Tianyi Wang’s research while affiliated with Purdue University West Lafayette and other places

... The proposed work builds on prior research on thumb-to-finger interactions [50,54], microgestures [25,42,43,45], on the hand interfaces [10] and explorations using physical objects as part of the interaction design [19,24]. Yet, even with the increasing interest in making XR interfaces more practical for everyday real-world interactions, addressing the physical elements of these interactions is still a challenge for developers [1]. ...

Reference:

GraV: Grasp Volume Data for the Design of One-Handed XR Interfaces

... Similarly, MimiCook [43] combines a depth camera and projector to deliver on-the-spot guidance during cooking. Computer-vision-based approaches are popular to guide users through various tasks [20,22,32,47,55]. For instance, AR Cooking [20] used 3D animation of cookware on AR glasses. ...

Reference:

PrISM-Observer: Intervention Agent to Help Users Perform Everyday Procedures Sensed using a Smartwatch

... Implementation of Universal Design Specification for AR as interface for AI: utilized explainable AI recommendations from Xu et al. (2023) and UD heuristics for AR from Szentirmai & Murano (2023), based on the following aspects: Fig. 1). The prototype is compatible with system-level assistive technologies like voice commands and screen readers, allowing users to start the application seamlessly. ...

Reference:

Universally Designed Augmented Reality as Interface for Artificial Intelligence Assisted Decision-Making in Everyday Life Scenarios

... Hand-eye coordination is also crucial for the Apple Vision Pro, improving its spatial computing capabilities and accessibility [1]. This dual-modal interaction has been proven effective in numerous scenarios, e.g., menu selection [36,30], text input [19,29,61], and page browsing [1,35], significantly reducing the learning curve and aligning the user experience more closely with natural human behaviors. Additionally, Bao et al. have explored how hand-eye coordination can aid in selecting and translating objects within environments with heavy 3D occlusions, further illustrating the practical applications and advantages of this interaction technique [4]. ...

Reference:

GazeRing: Enhancing Hand-Eye Coordination with Pressure Ring in Augmented Reality

... Prior research has developed tools to support hand-object interaction data collection. One such tool is ARnnotate [40], which assists users in performing various hand poses while the system records 3D hand positions, 3D object bounding boxes, images, and additional metadata. Tools like ARnnotate could be utilized to gather data for expanding the GraV dataset we provide or for generating personalized GraV data in real-time. ...

Reference:

GraV: Grasp Volume Data for the Design of One-Handed XR Interfaces

... For example, in CAPturAR [54] and ProGesAR [55], users can prototype context-aware IoT applications by recording the human's movement or by using proxies through AR-based interfaces. In MechARSpace [56], users can author AR enhanced toys with a two-way biding between AR content and physical sensors and actuators from a plug-and-play IoT toolkit. While these tools do not focus on visualisation aspects, there are some that do: in PapARVis Designer [12] users can extend static visualisations in physical books by augmenting them with virtual content. ...

Reference:

RoboVisAR: Immersive Authoring of Condition-based AR Robot Visualisations

... Additionally, a new interactive scene synthesis tool was proposed to help designers group object arrangements or quickly identify potential synthesis results [33]. [34] proposed an environment in which designers can create scenes for augmented reality (AR) users in VR by extracting indoor scene information using Microsoft HoloLens2-based AR and reconstructing the extracted scene data into a virtual environment. Additionally, an authoring environment in which designers can directly reconstruct and synthesize scenes based on an immersive virtual environment was proposed [35]. ...

Reference:

Improved object arrangement and layout design considering interaction for indoor scene synthesis in immersive virtual reality

... Freehand gesture. Gesture interaction is widely used in current AR systems [40,50,37], due to its precision and resemblance to daily behaviour. Users can interact through various gestures, such as pinch, tap, drag, and dual-hand interaction [8,38]. ...

Reference:

GazeRing: Enhancing Hand-Eye Coordination with Pressure Ring in Augmented Reality

... Wang and others have combined light painting with augmented reality (AR) technology. Through moving a light source along these AR trajectories, users can create accurate traces on the photograph, thereby bridging the gap between actual output and planned trajectories, and enhancing the quality of the creative content (Wang et al., 2021). Nestor Z. Salamon and others have proposed using light painting videos combined with computation to optimise the photo environment and light painting patterns. ...

Reference:

Light-Tracing: A Novel Approach for Mixed-Reality (MR) Content Creation

... For instance, Uriu et al. [50] developed a sensor-equipped frying pan that offers context-sensitive information like the pan's current temperature. Also, AdapTutAR [22] is a machine task tutoring system designed to monitor users in following tutorials and offer feedback through AR glasses. However, we often need help with mundane tasks or in situations where instrumenting ourselves or the environment is not easy. ...

Reference:

PrISM-Observer: Intervention Agent to Help Users Perform Everyday Procedures Sensed using a Smartwatch