Fig 2 - uploaded by Mi Zhang
Content may be subject to copyright.
Source publication
With the advent of immersive virtual reality (VR) head-mounted displays (HMD), we envision that immersive VR will revolutionize the personal fitness experience in our daily lives. Toward this vision, we present JARVIS, a virtual exercise assistant that is able to provide an immersive and interactive gym exercise experience to a user. JARVIS is enab...
Context in source publication
Context 1
... 1 summarizes these previous works and compares them with JARVIS proposed in this paper. Figure 2 illustrates the system architecture of JARVIS. As shown, JARVIS operates with two types of devices: 1) a miniature IoT sensing device that is attachable to gym machines to track machine exercises; and 2) a VR HMD that processes the sensor data as well as visualizes the exercise information and the computer-generated virtual environment in real time. ...
Similar publications
The main purpose of this review was to systematically analyze the literature concerning studies which have investigated muscle activation when performing the Deadlift exercise and its variants. This study was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Statement (PRISMA). Original studies from incep...
Citations
... This technology system is sometimes known as the Internet of Objects, that is "the networked interconnection of everyday objects, which are often equipped with ubiquitous intelligence" [139]. Fusing VR technology with IoT has been used recently for the scientific and modern construction of physical education teaching systems [35], as well as to demonstrate to users the proper way of doing physical exercises in a virtual exercise environment in a gym [108]. Wang [136] provide insights into the characteristics of virtual reality technology while studying the application of virtual reality technology in environmental art design. ...
While Virtual Reality has shown the potential to improve learning experiences, its acceptance and adaptation within curriculum remains significantly limited. One possible reason for this is the lack of information regarding how faculty from different disciplines and with different levels of expertise in using VR perceive the challenges and benefits of its implementation. As such, this study uses a Phenomenography approach to share the views of faculty from Business, Instructional Design and English in this context. Interview and observation data findings suggest that while faculty are keenly interested in VR technology, the challenges posed by cost, hardware and software can sometimes be prohibitive to VR adaptation. The study offers multiple suggestions to mitigate such challenges so that students and faculty can benefit from a technology that is highly immersive and engaging. The recommendations support and add to the literature and highlight the deeper details of issues and recommendations for VR integration in curriculum.
... Numerous indoor AR systems have been proposed in the literature [17][18][19][20][21]. Of these systems, Endure [19], Climbing Gym [20], and Jarvis [21] are closely related to GAR. ...
... Numerous indoor AR systems have been proposed in the literature [17][18][19][20][21]. Of these systems, Endure [19], Climbing Gym [20], and Jarvis [21] are closely related to GAR. These three systems will be reviewed to understand today's state-of-the-art AR systems for the use in gyms. ...
... GAR is built based on the ILARF framework. To show that ILARF can indeed facilitate the development of desirable AR systems with multiple advantages, three related gym AR/VR (virtual reality) systems are reviewed, namely, Endure [19], Climbing Gym [20], and Jarvis [21], as described in Section 4.1. After that, GAR is compared with these related systems in Section 4.2. ...
This paper proposes an indoor location-based augmented reality framework (ILARF) for the development of indoor augmented-reality (AR) systems. ILARF integrates an indoor localization unit (ILU), a secure context-aware message exchange unit (SCAMEU), and an AR visualization and interaction unit (ARVIU). The ILU runs on a mobile device such as a smartphone and utilizes visible markers (e.g., images and text), invisible markers (e.g., Wi-Fi, Bluetooth Low Energy, and NFC signals), and device sensors (e.g., accelerometers, gyroscopes, and magnetometers) to determine the device location and direction. The SCAMEU utilizes a message queuing telemetry transport (MQTT) server to exchange ambient sensor data (e.g., temperature, light, and humidity readings) and user data (e.g., user location and user speed) for context-awareness. The unit also employs a web server to manage user profiles and settings. The ARVIU uses AR creation tools to handle user interaction and display context-aware information in appropriate areas of the device’s screen. One prototype AR app for use in gyms, Gym Augmented Reality (GAR), was developed based on ILARF. Users can register their profiles and configure settings when using GAR to visit a gym. Then, GAR can help users locate appropriate gym equipment based on their workout programs or favorite exercise specified in their profiles. GAR provides instructions on how to properly use the gym equipment and also makes it possible for gym users to socialize with each other, which may motivate them to go to the gym regularly. GAR is compared with other related AR systems. The comparison shows that GAR is superior to others by virtue of its use of ILARF; specifically, it provides more information, such as user location and direction, and has more desirable properties, such as secure communication and a 3D graphical user interface.
... However, with an ongoing process of miniaturization of electronic components, increasing density of pixels in head-mounted displays, and advancements in deep learning applications, XR is getting yet another round of attention. Notable advancements include Google Glass Enterprise 2 [1], North Focals [2]; a constantly growing entertainment-centered VR/MR market [3]; and relevant research projects [4], [5], [6], [7], [8], [9]. ...
... In the case when the user input C may contain a mistyped character, the probabilistic model fails and predicts no candidate words (see line 4). Then, while we search through the characters from reverse order (from c n to c 1 ), we predict the candidate words again after replacing the character c i with its neighboring characters (see lines [5][6][7][8][9][10][11][12]. The algorithm returns the found candidates up to three words if they exist. ...
... Moreover, we acknowledge other touch-based interactions designated for one-handed text entry on various touch-sensitive surfaces such as the spectacle frame of Google Glass (One-Dimensional Handwriting (4.67 WPM), SwipeZone [6] (8.73 WPM), a finger-worn addendum (8 WPM) [87], and fingertip addendum (11.9 WPM ) [76]. The text entry rate of MyoKey is comparable to the above stateof-the-art solutions but only slightly inferior to the fingertip addendum, with the below reasons. ...
Usability challenges and social acceptance of textual input in a context of extended realities (XR) motivate the research of novel input modalities. We investigate the fusion of inertial measurement unit (IMU) control and surface electromyography (sEMG) gesture recognition applied to text entry using a QWERTY-layout virtual keyboard. We design, implement, and evaluate the proposed multi-modal solution named MyoKey. The user can select characters with a combination of arm movements and hand gestures. MyoKey employs a lightweight convolutional neural network classifier that can be deployed on a mobile device with insignificant inference time. We demonstrate the practicality of interruption-free text entry with MyoKey, by recruiting 12 participants and by testing three sets of grasp micro-gestures in three scenarios: empty hand text input, tripod grasp (e.g., pen), and a cylindrical grasp (e.g., umbrella). With MyoKey, users achieve an average text entry rate of 9.33 words per minute (WPM), 8.76 WPM, and 8.35 WPM for the freehand, tripod grasp, and cylindrical grasp conditions, respectively.
... 3D Space Lofting Module. e interaction between vision and light is a typical problem of flexible interactive animation architecture [11]. e changes of the space scene are divided into different interfaces according to the different animation architectures, and the extremely high distribution and modular characteristics of the deformation field show the animation elements of the space scene and change according to the changes of the space scene. ...
With the development of computational simulation technology, the need for stable and immersive display effects of space scene animation in the field of life experience and visual art has gradually increased. In this case, the requirements for immersive characteristics of space scene animation have also been strengthened. The existing 3D space scene animation has a limited degree of stereoscopic display model and data visualization. However, the current space scene interactive animation adopts a plane layout as a whole, and the size of the view interface is generally fixed, including the size of virtual elements. However, the immersion in this paper interactive animation of spatial scene can effectively solve the problems of incomplete display of 3D effects and unstable view simulation in 3D effects. This paper takes immersive space scene animation as the research object and studies the 3D and characteristics of space scene animation based on computer parallel computing in different immersive space scenes and different virtual space technologies, as well as the animation effects of different scene transformations and art forms. The result of research shows that, with the continuous increase of the degree of virtuality within a certain range, the immersive effect of spatial scene interactive animation gradually becomes better. When the color of the space scene animation is below 15, the virtual immersive effect changes less. When the space scene is in the range of 15–20, the space scene will make people feel the atmosphere of a beautiful and mysterious illusion.
... In addition, the objects or entities in the environment can interact in a real-time manner with each other [26,43,49]. To increase the user engagement effectively, real-time IoT applications (e.g., public safety in smart cities) integrate VRLEs for disaster management and damage containment [17,22,25]. The social VRLE systems render immersive content from the network-connected IoT devices to the users [11] in order to enhance human cognition in several subjects i.e., psychology and physiology [37]. ...
Virtual Learning Environments (VLEs) are spaces designed to educate student groups remotely via online platforms. Although traditional VLEs have shown promise in educating students, they offer limited immersion that overall diminishes learning effectiveness. In this paper, we describe vSocial, a cloud-based virtual reality learning environment (VRLE) system that can be deployed over high-speed networks using the High Fidelity “social VR” platform. vSocial provides flexible control of group learning content and compliance with established VLE standards with improved immersive user experience for both instructor(s) and students. For our vSocial development, we build upon the use case of an existing special education VLE viz., iSocial that trains youth with Autism Spectrum Disorder by implementing the Social Competence Intervention (SCI) curriculum. The vSocial can be used to: (a) implement multiple learning modules using wearable VR technologies, (b) integrate cognitive state sensing devices, and (c) organize learning session data securely using web applications hosted on cloud resources. Our experiment results show that the VR mode of content delivery in vSocial better stimulates the generalization of lessons to the real world than non-VR lessons, and provides improved immersion when compared to an equivalent desktop version. Further, usability study results show that users can successfully use the web application features in vSocial for group learning activities with ease-of-use and consistency.
... To date, AmI applications have been largely explored in several contexts: education (Bureš, Tučník, Mikulecky, Mls, & Blecha, 2016;Korozi et al., 2019), health environments (Machado, Maran, Augustin, Wives, & de Oliveira, 2017;Islam, Kwak, Kabir, Hossain, & Kwak, 2015), smart cities and home design (Yaddaden, Adda, Bouzouane, Gaboury, & Bouchard, 2018;Hui, Sherratt, & Sánchez, 2017), neurocomputing (Roda, Rodríguez, López-Jaquero, Navarro, & González, 2017), factories business (Robinson, Sanders, & Mazharsolook, 2015), and for creating immersive exercises in a retail environments such as gyms (Rabbi, Park, Fang, Zhang, & Lee, 2018). ...
... In the AmI perspective, several works make use of sensors to control the users' performance during physical exercises. For instance, JARVIS (Rabbi et al., 2018), a virtual exercise assistant, able to provide an immersive and interactive gym experience to users through IoT and virtual reality. It employs small IoT sensor that can be easily attached to, and removed from, exercise machines to track a multitude of information, including exercise types, repetition counts, and progress within each repetition in real-time. ...
Ambient Intelligence (AmI) is an interdisciplinary research area of ICT which has evolved since the 90s, taking great advantage from the advent of the Internet of Things (IoT). AmI creates, by using Artificial Intelligence (AI), an intelligent ecosystem in which computers, sensors, lighting, music, personal devices, and distributed services, work together to improve the user experience through the support of natural and intuitive user interfaces. Nowadays, AmI is used in various contexts, e.g., for building smart homes and smart cities, providing healthcare, and creating an adequate atmosphere in retail and public environments.
In this paper, we propose a novel AmI system for gym environments, named Gym Intelligence, able to provide adequate music atmosphere, according to the users’ physical effort during the training. The music is taken from Spotify and is classified according to some music features, as provided by Spotify itself. The system is based on a multi-agent computational intelligence model built on two main components: (i) machine learning methods that forecast appropriate values for the Spotify music features, and (ii) a multi-objective dynamic genetic algorithm that selects a specific Spotify music track, according to such values. Gym Intelligence is built by sensing the ambient with a minimal, low-cost, and non-intrusive set of sensors, and it has been designed considering the outcome of a preliminary analysis in real gyms, involving real users. We have considered well-known regression methods and we have validated them using a collected data (i) about the users’ physical effort, through the sensors, and (ii) about the users’ music preferences, through an Android app that the users have used during the training. Among the regression methods considered, the one that provided the best results is the Random Forest, which predicted Spotify music features with a mean absolute error of 0.02 and a root mean squared error of 0.05. We have implemented Gym Intelligence and deployed it in five real gyms. We have evaluated it conducting several experiments. The experiments show how, with the help of Gym Intelligence, the users’ satisfaction about the provided background music, rose from 3.05 to 4.91 (on a scale from 1 to 5, where 5 is the maximum score).
... Given that VR is still nascent, studies have focused more on detection rather than intervention. Metrics like Electrocardiographic signal, galvanic skin response, blood volume pressure, electrodermal activity [69] [36], Flow Experience Analysis for taskbased activities [29], Miss Rate, Merge Rate, Fragmentation Rate in case of rehabilitation and trauma-based VR applications [61], Effect of Reachability [15], Vibrotactile Feedback [51], Interface quality, and Realism Index along with an immersive tendency survey [71], Heart Rate, and Skin Conductivity [16] are used to assess the quality of applications. In [81], PPG signals and EEG signals are captured to study the participant's attention in terms of learning VR applications. ...
... Novel Metrics: We observed a few novel metrics which are not found to be used in traditional software product development. Effect of Reachability [15], Pupillary Light Response [32], Miss Rate -Merge Rate -Fragmentation Rate [61], Temperature Distribution [41] and Vibrotactile Feedback [51] are few of the metrics. We observe that these metrics are novel and can be adopted by upcoming VR products that are planning to focus on Quality-of-Experience and VR scene quality. ...
... Rabbi et al. [49] and Rawassizadeh et al. [50] found that users are not motivated to manually log their exercise, especially when they are not accompanied by personal trainers. Therefore, external motivation, like gamification, is required to encourage manual data entry. ...
... Therefore, external motivation, like gamification, is required to encourage manual data entry. With that in mind, Rabbi et al. [49] proposed a virtual exercise assistant with a virtual reality head mounted display and an attached, miniature IoT sensing device that can track users' activities. Another prominent example of physical activity gamification is Pokémon GO [51], which resulted in US$100 million sales a month after its release [90], although this app does not have an indoor context. ...
There is growing consumer demand for digital technologies that help users track, motivate, and receive coaching for both aerobic and anaerobic activities. In this article, we provide a review of existing technological advances in tracking, coaching, and motivating users during indoor training in contexts such as gymnasiums. This study lists the advantages and limitations of various apparatuses and applications used for this purpose. Our review and discussion are intended to help entrepreneurs and engineers improve their products to better meet users? needs and aid researchers in identifying potential new areas.
... For instance, VR motion training systems were found effective to transfer knowledge from a ghost trainer to the trainee (Kim, 1999). Other examples include a VR track cycling platform developed by Yap and colleagues to simulate real-track conditions for training purpose (Yap et al., 2018) and a virtual exercise assistant designed by Rabbi and colleagues to provide immersive and interactive gym experience to users (Rabbi, Park, Fang, Zhang, & Lee, 2018). ...
Recent research suggests that VR games can engage players in physical activity with high levels of enjoyment. Understanding users’ motivation to engage and enjoy immersive VR exercise platforms is thus important to designers. We designed a VR exercise platform and conducted an experiment with two conditions, one with a static User Interface (UI) and the other with an Open World environment. Across participants there was significantly (p= .03*) greater enjoyment reported in an Open World compared to static UI. Enjoyment in both static UI and Open World conditions was positively correlated with user’s psychological needs and experience; autonomy and immersion. Participants’ future play intention was also predicted by autonomy and immersion, but only within the open world condition. Our findings also suggest players can be classified into entertainment-focused and exercise-focused with different expectations and therefore different engagement behaviours with each VR exercise environment. The study highlights the value of informing VR design with measures of psychological need satisfaction.
... These efforts include the tactile feedback of a shape display [19], tangible objects, both at-hand [9,29] and presented by a robot arm [2], and the expanded interactions of modified controllers [12,68]. Further attempts within the field of tangible and embodied interaction include varying degrees of sensory and/or real-world environments [30,51,60], as well as the possibility of the embodied, kinesthetic action of wakeboarding [33], swinging [64], or using gym equipment [50]. In the majority of these cases, VR appears to be the dominant medium. ...
We present two proof of concept experiences for a virtual reality (VR) game that draws on several medium-specific qualities of mobile, location-based, and tangible storytelling. In contemporary smartphone-VR, experiences are limited by short playtimes, limited interactions, and limited movement within a physical space. To address these limitations, we suggest a reconceptualization of smartphone-VR. Rather than design that deems the smartphone the least capable VR platform, we propose design that adds VR to an already rich mobile storytelling platform. We argue that by drawing on otherwise separate storytelling media, designers can circumvent limitations related to smartphone-VR while also extending the range of smartphone-based storytelling. We conclude by reflecting on possible implications of this extended design space.
