Content uploaded by Grigore Burdea
Author content
All content in this area was uploaded by Grigore Burdea on Nov 06, 2014
Content may be subject to copyright.
Multimodal Virtual Reality: Input-Output Devices, System Integration, and Human Factors
International Journal of Human-Computer Interaction
Abstract
Virtual reality (VR) involves multimodal interactions with computer-simulated worlds through visual, auditory, and haptic feedback. This article reviews the state of the art in special-purpose input-output devices, such as trackers, sensing gloves, 3-D audio cards, stereo displays, and haptic feedback masters. The integration of these devices in local and network-distributed VR simulation systems is subsequently discussed. Finally, we present human-factor studies that quantify the benefits of several feedback modalities on simulation realism and sensorial immersion. Specifically, we consider tracking and dextrous manipulation task performance in terms of error rates and learning times when graphics, audio, and haptic feedback are provided.
... (Hammill, 2005) 1922-19451946-19641965-19801981-2000(Crampton & W.Hodge, 2009) 1925-19451946-19641965-19791980(Salahuddin, 2010) 1922-19431943-19601960-19801980-2000(Bova & Phelan, 2011) 1925-19421943-19611962-19811982-2000(Harber, 2011) 1922-19451946-19641965-19801980-20002000+ (Hansen & Leuty, 2012) 1924-19451946-19641965-19801980-(Agnieszka & Barbara, 2014) 1946-19641965-19761980-1991+ (Education, 2015) 1930-19451946-19641964-19811982-2000(Aniszewska, 2015) 1925-19451946-19641965-19801980-19941995-(Consultancy.uk, 2015) 1928-19441945-19651965-19791980-19941995+ (Cook, 2015) 1927-19451946-19641965-19801981-19951995+ (Demirağ & Gündüz, 2021Yüksekbilgili, 2015) 1925-19451946-19641965-19791980-1988-2012(Demirkaya, Akdemir, Karaman, & Atan, 2015) 1925-19451946-19641965-19791980-20002000+ (Sönmez, 2016) 1946-19651965-19791980-19941995-2000 (Cilliers, The Challenge of Teaching Generation Z People, 2017; Dolot, 2018) 1928-19441945-19651965-19791980-19951995+ (Gemlik, İlter, & Bektaş, 2018) 1925-19451946-19641965-197919802000+ (Bağçı & İçöz, 2019) 1946-19641965-197919802000-2012-2030(Dimock, 2019) 1928-19451946-19641965-19801981-1996-2012(Köktener & Algül, 2020) 1927-19461946-19641965-19801981 2000 + (Alptekin, Türkmen, & Arslantaş, 2021;Öz, 2015Öz, ) 1925Öz, -1945Öz, 1946Öz, -1964Öz, 1965Öz, -1979Öz, 1980Öz, -1999Öz, 2000Öz, + (Görmez, 2021Öz, ) 1925Öz, -1945Öz, 1946Öz, -1964Öz, 1965Öz, -1979Öz, 1980Öz, -1995Öz, 1996Öz, -2010 2010 + (Uysal, 2022) 1924-19451946-19641965-19801981-20002001-2012(Arslan & Sağlam, 2022) 1900-19251925-19451960-19801980-20002000+ (İçme, Yıldırım, & Büyük, 2022) 1924-19451946-19641965-19791980-2000 2021 + (Kocakaya & Dönmez, 2022) 1925-19451946-19641965-19791980-20002000 2011 + Bu çalışmada, yerel yapının kuşakların oluşmasındaki etkisi göz önünde bulundurulmuş, Tablo 1'den yola çıkılmış olup görece olarak yerli araştırmacılar içerisinde yaygın kullanılan yıllar esas alınmıştır. Sorunları çözme ve düşünce tarzları, yaşanılan coğrafyaya, çağa, o dönemin siyasi erkine göre değişiklik göstermektedir (Mannheim, 1927/28 (Aniszewska, 2015). ...
... (Alvim, 2022;Bowskill & Downie, 1995;Park, Bokijonov, & Cho, 2021), mekandaki objelerle olan etkileşimini ifade etmektedir (Kayabaşı, 2005). VR kullanıcıyı simüle edilmiş bir evrene çeken arayüzü temsil etmektedir (Burdea, Richard, & Coiffet, 1996). ...
Bilgi teknolojilerinde yaşanan gelişmeler neticesinde insan yaşamına giren Sanal
Gerçeklik ve Metaverse gibi pek çok kavram, dildeki değişiklerin yanı sıra yaşam
tarzında da köklü değişikliklere yol açmıştır. Dijital yerli olarak adlandırılan Z kuşağı
günümüzde bu yeni teknolojiye adaptasyonu tam olarak tamamlanmış ilk kuşak olarak
incelenmektedir. Dijital devriminin yaşanmasının ardından dünyaya gelen Z kuşağı
internete tam olarak hâkim olan ilk kuşaktır. Bu çalışmada Z kuşağı bireylerin, Metaverse
farkındalığı, onu kullanma ve fark etme biçimleri belirlenmeye çalışılmıştır. Çalışmanın
teorik bölümünde 1900- günümüz arasındaki kuşaklara, Metaverse’ün gelişim sürecine
ve onu oluşturan dinamiklere, Metaverse ile birlikte gündelik konuşma diline de hâkim
olan Metaverse’ü oluşturan teknolojileri hakkında alan yazın incelenmiştir. Uygulama
kısmında ise, Z kuşağı bireylerinden oluşan 500 kişi ile yapılan anket çalışmasına yer
verilmiştir. Anket çalışmasında elde edilen veriler SPSS 26 paket programı aracılığıyla
analiz edilmiş ve yorumlanmıştır. Araştırma sonucunda Z kuşağı bireylerin; cinsiyeti,
gelir durumu, medya ve teknoloji uyaranları ile Metaverse’e yönelik farkındalık arasında
ilişki olmadığı, eğitim durumları ve günlük teknolojiye harcanan zaman arttıkça
metaverse yönelik farkındalığın arttığı gözlemlenmiştir.
... This integration allows a more complete sensorial experience. In 1996, Burdea et al. [27] provided foundational insights into multimodal interactions in VR, emphasizing the integration of special-purpose input-output devices. Subsequently, the visual-haptic multimodality has continuously remained a focal point of research in VR. ...
Virtual reality (VR) technology has been increasingly focusing on incorporating multimodal outputs to enhance the sense of immersion and realism. In this work, we developed AirWhisper, a modular wearable device that provides dynamic airflow feedback to enhance VR experiences. AirWhisper simulates wind from multiple directions around the user's head via four micro fans and 3D-printed attachments. We applied a Just Noticeable Difference study to support the design of the control system and explore the user's perception of the characteristics of the airflow in different directions. Through multimodal comparison experiments, we find that vision-airflow multimodality output can improve the user's VR experience from several perspectives. Finally, we designed scenarios with different airflow change patterns and different levels of interaction to test AirWhisper's performance in various contexts and explore the differences in users' perception of airflow under different virtual environment conditions. Our work shows the importance of developing human-centered multimodal feedback adaptive learning models that can make real-time dynamic changes based on the user's perceptual characteristics and environmental features.
The so-called second wave of VR has brought to research and market a lot of new displays, input devices, and content solutions during the last few years. Not only has new hardware entered the consumer market with low-cost price patterns, but whole new technologies are also being designed and developed.
Bu çalışmada marka ve marka
oluşturma sürecinde işletmelerin bazı departmanlarındaki uygulamalarında
yapay zekanın kullanımı değerlendirilmiştir. Bu genel amaçla öncelikle, yapay
zekâ ve buna yönelik teknolojiler açıklanmıştır. Daha sonra yapay zekânın işletmelerde markalama sürecinde nasıl kullanıldığı üzerinde durulmaya
çalışılmıştır. Bu kapsamda öncelikle marka kavramına yönelik açıklamalar
yapılmış, daha sonra yapay zekâya yönelik kavramsal bir çerçeve çizilmiştir.
Gelecek dönemlerde yapay zeka kullanımının daha fazla yaygınlaşması beklenmektedir.
Bu nedenledir ki, işletmeler markalaşma sürecinde yapay zeka
teknolojisine yönlendirilmeli ve bu süreci iyi yönetebilir hale gelmelidir. Günümüzde
marka ve yapay zeka alanının beraber ele alınmasının, akademik
olarak çalışmalara konu olmasının bu sürece uyumluluğu kolaylaştırıcı bir
etki oluşturacağı düşünülmektedir.
Neurofeedback refers to the process of feeding a sensory representation of brain activity back to users in real time to improve a particular brain function, e.g., their focus and/or attention on a particular task. This study addressed the notable lack of research on methods used to visualize EEG data and their effects on the immersive quality of VR. We developed an algorithm to quantify focus, yielding a focus score. A pre-study with twenty participants confirmed its effectiveness in distinguishing between focused and relaxed mental states. Subsequently, we used this focus score to prototype a VR experience system visualizing the focus score in preconfigured manners, which was utilized in an exploratory study to assess the impact of different neurofeedback visualization methods on user engagement and focus in VR. Among all the visualization methods evaluated, the environmental scheme stood out due to its superior usability during task execution, its ability to evoke positive emotions through the visualization of objects or scenes, and its minimal deviation from user expectations. Additionally, we explored design guidelines based on collected results for future research to further refine the visualization scheme, ensuring effective integration of the focus score within the VR environment. These enhancements are crucial for designing neurofeedback visualization schemes that aim to boost participant focus in VR settings, offering significant insights into the optimization of such technologies.
Welding is an important operation in many industries, including construction and manufacturing, which requires extensive training and practices. Although welding simulators have been used to accommodate welding training, it is still challenging to enable novice trainees to effectively understand the kinesthetic experience of the expert in an egocentric manner, such as the proper way of force exertion in complex welding operations. This study implements a robot-assisted perceptual learning system to transfer the expert welders' experience to trainees, including both the positional and force control actions. A human-subject experiment (N = 30) was performed to understand the motor skill acquisition process. Three conditions (control, robotic positional guidance with force visualization, and force perceptual learning with position visualization) were tested to evaluate the role of robotic guidance in welding motion control and force exertion. The results indicated various benefits related to task completion time and force control accuracy under the robotic guidance. The findings can inspire the design of future welding training systems enabled by external robotic systems.
In studying grasping and manipulation we find two very different approaches to the subject: knowledge-based approaches based primarily on empirical studies of human grasping and manipulation, and analytical approaches based primarily on physical models of the manipulation process. This chapter begins with a review of studies of human grasping, in particular our development of a grasp taxonomy and an expert system for predicting human grasp choice. These studies show how object geometry and task requirements (as well as hand capabilities and tactile sensing) combine to dictate grasp choice. We then consider analytic models of grasping and manipulation with robotic hands. To keep the mathematics tractable, these models require numerous simplifications which restrict their generality. Despite their differences, the two approaches can be correlated. This provides insight into why people grasp and manipulate objects as they do, and suggests different approaches for robotic grasp and manipulation planning. The results also bear upon such issues such as object representation and hand design.
The control of exploratory and manipulative procedures in
teleoperation and virtual environments requires the availability of
adequate advanced interfaces capable not only of recording the movements
of the human hands and arms, but also of replicating sensations of
contact and collisions. In this paper the problem of replicating
external forces acting against the remote/virtual arm is addressed. The
design of an arm exoskeleton system developed in the authors' laboratory
is presented. The exoskeleton consists of a 7 DOF actuated and
sensorized mechanical structure wrapping up completely the human arm and
directly supported by the shoulders and the trunk of the human operator.
Emphasis is given to the implemented control procedures and to the
description of the transputer-based control architecture
Research on virtual environments (VE) produced significant advances in computer hardware (graphics boards and i/o tools) and software (real-time distributed simulations). However, fundamental questions remain about how user performance is affected by such factors as graphics refresh rate, resolution, control latencies, and multimodal feedback. This article reports on two experiments performed to examine dextrous manipulation of virtual objects. The first experiment studies the effect of graphics frame rate and viewing mode (monoscopic vs. stereoscopic) on the time required to grasp a moving target. The second experiment studies the effect of direct force feedback, pseudoforce feedback, and redundant force feedback on grasping force regulation. The trials were performed using a partially-immersive environment (graphics workstation and LCD glasses), a DataGlove, and the Rutgers Master with force feedback. Results of the first experiment indicate that stereoscopic viewing is beneficial for low refresh rates (it reduced task completion time by about 50% vs. monoscopic graphics). Results of the second experiment indicate that haptic feedback increases performance and reduces error rates, as compared to the open loop case (with no force feedback). The best performance was obtained when both direct haptic and redundant auditory feedback were provided to the user. The large number of subjects participating in these experiments (over 160 male and female) indicates good statistical significance for the above results.
This paper describes the PHANTOM haptic interface - a device which measures a user's finger tip position and exerts a precisely controlled force vector on the finger tip. The device has enabled users to interact with and feel a wide variety of virtual objects and will be used for control of remote manipulators. This paper discusses the design rationale, novel kinematics and mechanics of the PHANTOM. A brief description of the programming of basic shape elements and contact interactions is also given.
The Paper outlines the concepts of a robot technology experiment ROTEX we have proposed to fly with the next German spacelab mission D2 (originally planned for 88, now delayed for at least two years). It provides a small, six axis robot inside a space-lab rack, equipped with a multisensory gripper (force/torque, an array of range finders, stereo optical fibers). The robot is supposed to handle a biological experiment, to perform several assembly and "servicing tasks" and to grasp floating objects. The paper focusses on the man-machine and supervisory control concepts for teleoperation from the spacecraft and from ground and expecially explains the predictive estimation schemes for an extensive use of delay-compensating 3D-computer graphics.
The Virtual Academy is an educational model based on multiage teams of students and adults working through the Internet to build and use virtual worlds for educational purposes. These collaborations are mediated by a range of tools ranging from electronic mail to hypermedia and video links, and result in the creation of simulation‐based role‐playing adventure games within the ExploreNet software environment ExploreNet is an Internet‐based multimedia, multiuser domain constructed specifically for educational experimentation.This article describes the Virtual Academy Model, the ExploreNet software system, and an experiment conducted in the spring of 1995. The article describes the evolution of features of ExploreNet's user interface and their relevance to collaborative work by children.