Holger Regenbrecht

• University of Otago

Presence, or the experience of being present in a computer-generated environment, is a defining element of virtual reality. While there are different methodologies to measure presence, questionnaires remain the most popular, particularly the Igroup Presence Questionnaire (IPQ). In this paper, we analyse the results of over 20 years of IPQ usage to...

Presence is a defining element of virtual reality (VR), but it is also increasingly used when assessing mixed reality (MR) experiences. The increased interest in measuring presence in MR and recent works underpinning the specific nature of presence in MR raise the question of the current state and practice of assessing presence in MR. To address th...

Cinematic Virtual Reality (CVR) is a style of narrative-based Virtual Reality (VR) experience built on filmed or computer-generated 360-degree videos. Since CVR is becoming more popular and widely accessible, researchers and practitioners have been trying to address challenges such as the conflict between the viewer’s freedom of choice and the crea...

Augmented reality (AR) glasses are likely to become omnipresent, providing a continuous and ubiquitous experience of computer-mediated reality. This new Pervasive AR will lead to perceptual, acceptance, and ethical issues which are increasingly discussed in the literature. However, given such Pervasive AR prototypes are currently not commercially a...

Acoustic noise control or cancellation (ANC) is a commonplace component of modern audio headphones. ANC aims to actively mitigate disturbing environmental noise for a quieter and improved listening experience. ANC is digitally controlling frequency and amplitude characteristics of sound. Much less explored is visual noise and active visual noise co...

The maturity of Augmented Reality (AR) technology allows for expansion into real-world applications, including visualizations for on-site sports spectating. However, it's crucial to understand the factors influencing user experience in AR applications. To optimize user experience, we conducted a user study where participants adjusted parameters to...

Co-design and development of technology with indigenous communities requires respect and close partnership. Here, we reflect on our experiences working with a Māori (indigenous New Zealand) community as Pākehā textasciimacron (non-Māori). <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> In parti...

Background Virtual reality (VR) has been used as a technological medium to deliver mirror therapy interventions with people after stroke in numerous applications with promising results. The recent emergence of affordable, off-the-shelf head-mounted displays (like the Oculus Rift or HTC Vive) has opened the possibility for novel and cost-effective a...

In the foreseeable future, mobile and wearable computing technology with an augmented reality (AR) interface can provide an omnipresent, environmentally adaptive, and everyday reality augmentation. This new pervasive AR technology will lead to a continuous moderation of experienced reality with the potential to support better and faster decision-ma...

bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Developing, evaluating, and disseminating IT research prototypes for and with indigenous partners is both challenging and rewarding. In conjunction with our domain expert collaborators, Te Rau Aroha Marae [Bluff, Aotearoa/New Zealand (NZ)] and our academ...

Content creators have been trying to produce engaging and enjoyable Cinematic Virtual Reality (CVR) experiences using immersive media such as 360-degree videos. However, a complete and flexible framework, like the filmmaking grammar toolbox for film directors, is missing for creators working on CVR, especially those working on CVR storytelling with...

This chapter is a collection of selected VR/AR case studies from academia and industry.

With recent technological advancements in sports broadcasting, remote spectators are presented with an enriched experience. These enriched experiences include additional content such as statistics and graphics that support game understanding. However, these technological advancements are often not accessible to on-site sports spectators. In this pa...

Developing, evaluating, and disseminating IT research prototypes for and with indigenous partners is both challenging and rewarding. In conjunction with our domain expert collaborators, Te Rau Aroha Marae (Bluff, Aotearoa/New Zealand) and our academic colleagues at the universities of Waikato and Canterbury, we are implementing a mixed reality tele...

Developing, evaluating, and disseminating IT research prototypes for and with indigenous partners is both challenging and rewarding. In conjunction with our domain expert collaborators, Te Rau Aroha Marae (Bluff, Aotearoa/New Zealand) and our academic colleagues at the universities of Waikato and Canterbury, we are implementing a mixed reality tele...

We have developed a novel and affordable way to texture virtual hands from individually taken photographs and integrated the virtual hands into a mixed reality neurorehabilitation system. This mixed reality system allows for serious game play with mirrored and non-mirrored hands, designed for patients with unilateral motor impairments. Before we ca...

Cinematic Virtual Reality (CVR) is a form of immersive storytelling widely used to create engaging and enjoyable experiences. However, issues related to the Narrative Paradox and Fear of Missing Out (FOMO) can negatively affect the user experience. In this paper, we review the literature about designing CVR content with the consideration of the vie...

Volumetric videos allow for a true three-dimensional experience where users can freely choose their viewing angles and be actually immersed in a video clip. High quality video productions are gaining attention and first volumetric video recordings are commercially provided at select places. Unfortunately, the production process is very time, labour...

Immersive Virtual Reality (VR) has shown to be effective in general training and learning applications, but whether it has potential in understanding and learning complex topics is not well researched. In addition, the role of users’ expertise in VR-based learning is not really understood. Here we present findings from our purpose-designed immersiv...

Augmented Reality (AR) enriches a user's real environment by adding spatially aligned virtual objects (3D models, 2D textures, textual annotations, etc) by means of special display technologies. These are either worn on the body or placed in the working environment. From a technical point of view, AR faces three major challenges: (1) to generate a...

The desire to stay connected to one another over large distances has guided decades of telepresence research. Most focuses on stationary solutions that can deliver high-fidelity telepresence experiences, but these are usually impractical for the wider population who cannot afford the necessary proprietary equipment or are unwilling to regress to no...

In recent years, the development of Augmented Reality (AR) frameworks made AR application development widely accessible to developers without AR expert background. With this development, new application fields for AR are on the rise. This comes with an increased need for visualization techniques that are suitable for a wide range of application are...

The Ātea project is investigating how technology can be used to help Māori share and preserve knowledge, culture, and language. One aspect of this is the use of telepresence systems to connect members of an iwi (tribe) or rūnanga (council) with their marae (meeting place, communal centre) and community. We describe the range of interactions we can...

Traditional sports events related data have no direct spatial relationship to what spectators see when attending a live sports event. The idea of our work is to address this gap and ultimately to provide spectators insights of a sports game by embedding sports statistics into their field of view of the game using mobile Augmented Reality. Research...

Background Providing timely follow-up care for patients with inflammatory bowel disease in remission is important but often difficult because of resource limitations. Using smartphones to communicate symptoms and biomarkers is a potential alternative. We aimed to compare outpatient management using 2 smartphone apps (IBDsmart for symptoms and IBDoc...

Augmented Reality (AR) sandtables facilitate the shaping of sand to form a surface that is transformed into a digital terrain map which is projected back onto the sand. Although a mature technology, there are still few instances of sandtables being used in surface analysis. Fundamentally there has not been any reported formal assessment of how well...

There are many techniques for locomotion and navigation that can support the exploration of large virtual environments in a limited physical area. Previous studies focused on measuring curvature gains and bending gains applied to the walking direction in the real world. However, the effects of different moving techniques and their relationship with...

Objective We describe a networked, two-user virtual reality (VR) power wheelchair (PWC) simulator system in which an actor (client) and an observer (clinician) meet. We then present a study with 15 observers (expert clinicians) evaluating the effect of three principal forms of viewpoint control (egocentric-egomotion, egocentric-tethered, and client...

For mixed reality applications, where reality and virtual reality are spatially merged and aligned in interactive real-time, we propose a pure voxel representation as a rendering and interaction method of choice. We show that voxels—gap-less volumetric pixels in a regular grid in space—allow for an actual user experience of a mixed reality environm...

The mobility and ubiquity of mobile head-mounted displays make them a promising platform for telepresence research asthey allow for spontaneous and remote use cases not possible with stationary hardware. In this work we present a system that providesimmersive telepresence and remote collaboration on mobile and wearable devices by building a live sp...

Recent advancements in computing research and technology will allow future immersive virtual reality systems to be voxel-based, i.e. entirely based on gap-less, spatial representations of volumetric pixels. The current popularity of pixel-based videoconferencing systems could turn into true telepresence experiences that are voxel-based. Richer, non...

Many categories of the illusion of self-motion have been widely studied with the potential support of virtual reality. However, the effects of directly and indirectly powering the movement on the possibility of perceiving changes in moving speed and their relationship with sensory feedback on users' speed change perception have not been investigate...

In this special issue of IEEE Transactions on Visualization and Computer Graphics (TVCG) , we are pleased to present the TVCG papers from the 17th IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2018), held October 16–20 in Munich, Germany. ISMAR continues the 20-year long tradition of IWAR, ISMR, and ISAR, and is undoubtedly t...

Prescription glasses are used by many people as a simple, and even fashionable way, to correct refractive problems of the eye. However, there are other visual impairments that cannot be treated with an optical lens in conventional glasses. In this work we present ChromaGlasses, Computational Glasses using optical head-mounted displays for compensat...

Given the advancements in ubiquitous computing we can nowadays link information to places and objects anywhere on the globe. For years, map-based interfaces have been the primary interfaces to browse and retrieve this situated media. While more recently also other interface concepts for situated media have found their way out of the research labs,...

Purpose: New rehabilitation strategies for post-stroke upper limb rehabilitation employing visual stimulation show promising results, however, cost-efficient and clinically feasible ways to provide these interventions are still lacking. An integral step is to translate recent technological advances, such as in virtual and augmented reality, into th...

The experience of Virtual Reality (VR) can lead to unwanted or wanted psychological stress reactions. Highly immersive VR games for instance utilise extreme, life-threatening, or dangerous situations to achieve those responses from their players. There is also sufficient evidence that in clinical settings and specific situations, such as fear of he...

Virtual and augmented reality, and other forms of mixed reality (MR), have become a focus of attention for companies and researchers. Before they can become successful in the market and in society, those MR systems must be able to deliver a convincing, novel experience for the users. By definition, the experience of mixed reality relies on the perc...

Virtual Reality based driving simulators are increasingly used to train and assess users’ abilities to operate vehicles in a controlled and safe way. For the development of those simulators it is important to identify and evaluate design factors affecting perception, behaviour, and driving performance. In an exemplary power wheelchair simulator set...

Virtual reality-based power wheelchair simulators can help potential users to be assessed and trained in a safe and controlled environment. Although now widely used and researched for several decades, many properties of virtual environments are still not yet fully understood. In this chapter, we evaluated the effects of the visual representation of...

Virtual reality-based power wheelchair simulators can help potential users to be assessed and trained in a safe and controlled environment. Although now widely used and researched for several decades, many properties of virtual environments are still not yet fully understood. In this study, we evaluated the effects of the visual representation of t...

Optical see-through head-mounted displays are currently seeing a transition out of research labs towards the consumeroriented market. However, whilst availability has improved and prices have decreased, the technology has not matured much. Most commercially available optical see-through head mounted displays follow a similar principle and use an op...

Augmented Reality is a technique that enables users to interact with their physical environment through the overlay of digital information. While being researched for decades, more recently, Augmented Reality moved out of the research labs and into the field. While most of the applications are used sporadically and for one particular task only, cur...

Collaborative Virtual Environments (CVE) with co-located or remote video communication functionality require a continuous experience of social presence. If, at any stage during the experience the communication interrupts presence then the CVE experience as a whole is affected - spatial presence is then decoupled from social presence. We present a s...

Background: Colorectal cancer (CRC) has a high incidence, especially in New Zealand. The reasons for this are unknown. While most cancers develop sporadically, a positive family history, determined by the number and age at diagnosis of affected first and second degree relatives with CRC is one of the major factors, which may increase an individual...

Research output metrics, in particular for peer-reviewed publications are of increasingly high importance for academics' careers. In Human-Computer Interaction (HCI) and related fields of research, like Computer Graphics and Multimedia, simple to use and objective measures like Thomson Reuter's impact factors aren't applicable. Ranking lists like t...

Providing portable and affordable virtual reality systems for upper limb stroke rehabilitation is still a challenge. Here we present a simple user interface that allows the integration of various upper limb stroke exercises that can be autonomously performed by patients without the presence of a therapist, yet is portable and assembled using afford...

New rehabilitation strategies for post-stroke upper limb rehabilitation employing visual stimulation show promising results. Cost-efficient ways to provide these techniques are still a challenge. Virtual and Augmented Reality Technologies could be suitable for this endeavor. Recent technological advances often are not translated into therapeutic pr...

The use of virtual reality technology can lead to better controlled and flexible forms of physical assessment and therapy, which can be more motivating for patients. The Nine Hole Peg Test (NHPT) is a standard instrument to practice and assess a patient's hand motor control abilities. A physical, wooden or plastic board with nine holes and cylindri...

Most interactive input devices for virtual reality-based simulators are proprietary and expensive. Can they be substituted with standard, inexpensive devices if the virtual representation of the input device looks and acts like the original? Visual dominance theory would appear to support such a possibility. According to this view the visual aspect...

Videoconferencing allows geographically dispersed parties to communicate by simultaneous audio and video transmissions. It is used in a variety of application scenarios with a wide range of coordination needs and efforts, such as private chat, discussion meetings, and negotiation tasks. In particular, in scenarios requiring certain levels of trust...

Augmented Reflection Technology for Autonomous Upper Limb Stroke Rehabilitation

There is increasing demand to improve the quality and efficiency of healthcare delivery. Mobile health self- reporting can play a vital role here, provided that user acceptance can be achieved. We developed a mobile health reporting app for patients suffering from inflammatory bowel diseases and tested its acceptance, perceived usefulness and usabi...

Videoconferencing systems available for end users do not allow for eye-to-eye contact between participants. The different locations of video camera and video display make it impossible to directly look into each others eyes. This issue is known as the lack of mutual gaze. Combined with a lack of a life-sized video image of the communication partner...

This paper evaluates the patient engagement and clinical feasibility of an Augmented Reflection Technology (ART) system for use in physical rehabilitation of the upper limb following stroke. In particular, TheraMem, an extension of the ART system, was assessed on its potential to engage patients in meaningful therapeutic exercise. Six patients part...

Augmented Reality (AR) has the potential to change the way therapy and rehabilitation is understood and administered. It can be used to manipulate the experience of reality resulting in novel rehabilitative applications including but not limited to augmented mirror box manipulations. We present a conceptual framework for the effective use of AR in...

We present a novel interface approach which combines 2D video-based AR with a partial voxel model, allowing for more convincing interactions with 3D objects and worlds. It enables users in a hand-controlled interface (a) to interact with a virtual environment (VE) and at the same time (b) allows for correct mutual occlusions between interacting fin...

This paper describes spatially aligned user-generated audio annotations and the integration with visual augmentations into a single mobile AR system. Details of our prototype system are presented, along with an explorative usability study and technical evaluation of the design. Mobile Augmented Reality applications allow for visual augmentations as...

This paper describes spatially aligned user-generated audio annotations and the integration with visual augmentations into a single mobile AR system. Details of our prototype system are presented, along with an explorative usability study and technical evaluation of the design. Mobile Augmented Reality applications allow for visual augmentations as...

Stereoscopic display technologies are becoming increasingly commonplace in cinemas, homes and offices. It appears that this form of presentation, also referred to as 3D, is more and more supplementing and substituting monoscopic (2D) forms. However, there is only little evidence for the actual usefulness of stereo presentations. We are investigatin...

We investigate the effect that visual occlusion plays on users' perception in a prototypical augmented reality post-stroke therapy system. We implemented a reach-based therapeutic exercise using the Microsoft Kinect to enable depth sensing and correct visual occlusion of the upper-limb. Thirty participants evaluated the exercise with three differen...

Mirror Box therapy is used for rehabilitating stroke patients who have lost control of a hand. The therapy uses mirrors to give patients the illusion that movements of their healthy hand are movements of their impaired hand. The goal of this stimulation is to enable the healthy part of the brain to learn to control the impaired hand. In this paper...

Training in a Virtual Environment simulating a power wheelchair, can enhance safety, cost-effectiveness, training quality, and can also be used for the assessment of potential future users. Power wheelchair simulators often lack correct physics simulation, do no t support peripheral vision, or do not provide features required for assessment. Here w...

Humans readily perceive ownership of a limb even when it is artificially induced as in the case of using a mirror reflection. However, mirror reflections are very constrained perceptions which do not allow transformations and varied contexts as often occurs in real life. The extent to which perceived limb ownership occurs with video-mediated manipu...

The way we are perceived and we are presenting ourselves in videoconferencing situations is influenced by many factors. Earlier work showed that there are strong effects of participant gender, partner gender, and body language availability on feelings of self-transmission efficacy in videoconferencing. Because participants rated their efficacy at d...

Mixed reality rehabilitation systems and games are demonstrating potential as innovative adjunctive therapies for health professionals in their treatment of various hand and upper limb motor impairments. Unilateral motor deficits of the arm, for example, are commonly experienced post stroke. Our Augmented Reflection Technology (ART) system provides...

This paper presents a clinical feasibility study of a novel Augmented Reflection Technology system, called TheraMem. The feasibility of the system for physical rehabilitation of the upper limb and the potential to improve motor impairments following stroke were evaluated. Five patients participated in a total of 20 sessions of upper limb training w...

We present a first version of a 3D power wheel chair (PWC) simulator to be used to assess whether a person is able to drive a real PWC. The project is in its very early stage but might lead to a comprehensive virtual assessment suite in the future.

Mixed reality rehabilitation systems and games are demonstrating potential as innovative adjunctive therapies for health professionals in their treatment of various hand and upper limb motor impairments. Unilateral motor deficits of the arm, for example, are commonly experienced post stroke. Our TheraMem system provides an augmented reality game en...

Video-based collaborative virtual environments (CVE) attempt to emulate face-to-face meetings by immersing remote collaborators in a shared 3D virtual setting. To investigate potential advantages of this novel type of collaborative user interfaces for creating a better sense of social presence and affording a more efficient collaborative process we...

Background: Augmented Reality (AR) is a relatively new technology which blends virtual objects with real scenes in real time and this may be an effective intervention to use in rehabilitation. Objectives: To systematically review the evidence for the effectiveness of AR applications on rehabilitation outcomes within a physical context. Method: A s...