Desney S. Tan’s research while affiliated with Microsoft and other places

An interactive concept learning image search technique that allows end-users to quickly create their own rules for re-ranking images based on the image characteristics of the images. The image characteristics can include visual characteristics as well as semantic features or characteristics, or may include a combination of both. End-users can then rank or re-rank any current or future image search results according to their rule or rules. End-users provide examples of images each rule should match and examples of images the rule should reject. The technique learns the common image characteristics of the examples, and any current or future image search results can then be ranked or re-ranked according to the learned rules.

Embodiments of the present invention relate to systems, methods, and computer storage media for identifying, authenticating, and authorizing a user to a device. A dynamic image, such as a video captured by a depth camera, is received. The dynamic image provides data from which geometric information of a portion of a user may be identified as well as motion information of a portion of the user may be identified. Consequently, a geometric attribute is identified from the geometric information. A motion attribute may also be identified from the motion information. The geometric attribute is compared to one or more geometric attributes associated with authorized users. Additionally, the motion attribute may be compared to one or more motion attributes associated with the authorized users. A determination may be made that the user is an authorized user. As such the user is authorized to utilize functions of the device.

An interactive tool is described for modifying the behavior of a system, such as, but not limited to, the behavior of a classification system. The tool uses an interface mechanism to present a current global state of the system. The tool accepts one or more refinements to this global state, e.g., by accepting individual changes to parameter settings that are presented by the interface mechanism. Based on this input, the tool computes and displays the global implications of the updated parameter settings. The process of iterating over one or more cycles of user updates, followed by computation and display of the implications of the attempted refinements, has the effect of advancing the system towards a global state that exhibits desirable behavior.

The description relates to user control gestures. One example allows a speaker and a microphone to perform a first functionality. The example simultaneously utilizes the speaker and the microphone to perform a second functionality. The second functionality comprises capturing sound signals that originated from the speaker with the microphone and detecting Doppler shift in the sound signals. It correlates the Doppler shift with a user control gesture performed proximate to the computer and maps the user control gesture to a control function.

The subject disclosure relates to a method and system for visual object categorization. The method and system include receiving human inputs including data corresponding to passive human-brain responses to visualization of images. Computer inputs are also received which include data corresponding to outputs from a computerized vision-based processing of the images. The human and computer inputs are processing so as to yield a categorization for the images as a function of the human and computer inputs.

In Experiments, researchers set up comparable situations in which they carefully manipulate variables and collect people’s behavior in each condition. Experiments are very effective in determining causation in controlled situations and complement techniques that investigate ongoing behavior in more natural settings. For example, experiments are excellent for determining whether increased audio quality reduces blood pressure of participants in a video conference, and can add important insights to the larger question of when people choose video conferences over audio-only ones.

A human input system is described herein that provides an interaction modality that utilizes the human body as an antenna to receive electromagnetic noise that exists in various environments. By observing the properties of the noise picked up by the body, the system can infer human input on and around existing surfaces and objects. Home power lines have been shown to be a relatively good transmitting antenna that creates a particularly noisy environment. The human input system leverages the body as a receiving antenna and electromagnetic noise modulation for gestural interaction. It is possible to robustly recognize touched locations on an uninstrumented home wall using no specialized sensors. The receiving device for which the human body is the antenna can be built into common, widely available electronics, such as mobile phones or other devices the user is likely to commonly carry.

A multiple model data exploration system and method for running multiple machine-learning models simultaneously to understand and explore data. Embodiments of the system and method allow a user to gain a greater understanding of the data and to gain new insights into their data. Embodiments of the system and method also allow a user to interactively explore the problem and to navigate different views of data. Many different classifier training and evaluation experiments are run simultaneously and results are obtained. The results are aggregated and visualized across each of the experiments to determine and understand how each example is classified for each different classifier. These results then are summarized in a variety of ways to allow users to obtain a greater understanding of the data both in terms of the individual examples themselves and features associated with the data.

Described herein is a system that includes a receiver component that receives gesture data from a sensor unit that is coupled to a body of a gloveless user, wherein the gesture data is indicative of a bodily gesture of the user, wherein the bodily gesture comprises movement pertaining to at least one limb of the gloveless user. The system further includes a location determiner component that determines location of the bodily gesture with respect to a touch-sensitive display apparatus. The system also includes a display component that causes the touch-sensitive display apparatus to display an image based at least in part upon the received gesture data and the determined location of the bodily gesture with respect to the touch-sensitive display apparatus.

Recent work has shown that the body provides an interesting interaction platform. We propose a novel sensing technique based on transdermal low-frequency ultrasound propagation. This technique enables pressure-aware continuous touch sensing as well as arm-grasping hand gestures on the human body. We describe the phenomena we leverage as well as the system that produces ultrasound signals on one part of the body and measures this signal on another. The measured signal varies according to the measurement location, forming distinctive propagation profiles which are useful to infer on-body touch locations and on-body gestures. We also report on a series of experimental studies with 20 participants that characterize the signal, and show robust touch and gesture classification along the forearm.