Figure 4 - uploaded by Päivi Majaranta
Content may be subject to copyright.
Source publication
Proper feedback is essential in gaze based interfaces, where the same modality is used for both perception and control. We measured how vibrotactile feedback, a form of haptic feedback, compares with the commonly used visual and auditory feedback in eye typing. Haptic feedback was found to produce results that are close to those of auditory feedbac...
Context in source publication
Context 1
... for accomplishing the task and therefore had no effect on the task. We suspect that reason 2 was at least partially true. This was because of the closing circle feedback shown during the dwell time. It allowed the participants to anticipate when the selection would hap- pen. The most efficient way to utilize the system was probably to anticipate based on the circle and ignore the feedback that followed the selection. In fact, during interview in the end, some participants commented that it was hard to evaluate any differences between the feedback modes. Some of the participants commented that they mainly concentrated on the animated feedback given on dwell time progression. This supports our reasoning that the feedback after selection was not important for the completion of the task. Still, we learned that in the tested condition of dwell time based eye typing haptic feedback did not perform any worse than the auditory feedback. While statistically non-significant, there was a slight tendency for worse performance with visual feedback, which was also reflected in the participants’ preferences. In summary, the first exploratory study showed that haptic feedback seems to work just as well as the other modes of feedback. However, we were left with the sus- picion that perhaps the dwell-time feedback interfered with the measurement of the effects of the selection feedback modes. Thus, to measure the effect of the selection feedback modes, we needed an experiment that empha- sized that part of the feedback. Based on the experiences from the first exploratory study, we designed a follow-up study on the effects of haptic feedback on selection in eye typing. Twelve volunteers (4 male, 8 female, aged between 24 to 54, mean 39 years) participated in the experiment. The volunteers received movie tickets in return for their participation. Three wore eye glasses and one had contact lenses during the experiment. The one with contact lenses experienced some problems with eye tracker calibration and required a few re-calibrations during the experiment. Participants included people with varying backgrounds and expertise levels in gaze-based interaction and use of information technology in general (some participants were total novices from outside of the academia). Five had previous experience with eye tracking but not with eye typing. Most had some experience in haptics, usually from the vibration in cell phones (eight reported using haptic feedback during input in their smart phones). The eye tracker and the experimental software were the same as in the first experiment. Taking into account the lessons learned from the first experiment, we adjusted the feedback and modified the setup. We removed the dominating “closing circle” visual feedback on dwell. By removing the animated feedback on dwell (which also inherently indicated the selection in the end of dwell), we forced the participants to concentrate on the feedback on selection. However, some feedback on focus was necessary as the accuracy of the measured point of gaze is not always perfect and the users thus needed some indication that the gaze was pointing at the correct key. We used a simple, static visual feedback indicating focus: when the user fixated on a key, its background color changed to a slightly darker color. The color stayed darker as long as the key was fixated and returned to its default color when the gaze moved away from the key. A short delay of 100 ms was added before the feedback on focus, because the change in color was hard to detect if it happened immediately after the gaze landed on the key. The delay was included in the dwell time (500 ms), thus the feedback on dwelling was shown for 400 ms before selection. Visual feedback. Visual feedback for selection was the same as in the first experiment. Auditory feedback. Auditory feedback was the same as in the first experiment. Haptic feedback. In the first experiment, some participants did not like the haptic feedback on their wrist. Therefore, as suggested by the participants, we decided to give the feedback on the finger. The participant placed his or her right index finger on the actuator (Figure 3). One participant had an injury on the index finger and used the middle finger instead. To reduce sound from the vibration, the actuator was placed on a soft cushion. We had the option to use hearing protectors if the participant could hear the sound despite the cushioning but it was not needed. Again, we tested the setup before the actual experiment with each participant. Some participants complained that the 100 ms constant amplitude haptic feedback was uncomfortable because it “tickled” their skin. Considering that we moved the haptic stimulation to the finger tip, which is known to be more sensitive, we needed to reduce the intensity of the tactile signal. In informal testing we found that a signal of the same 250Hz frequency with decaying amplitude that reached 0 around 70 milliseconds (see Figure 4, bottom) was equally easy to detect, but significantly less “ticklish”. We extended the experiment from one session into three so that each participant used each feedback condition three times. We hoped that a longer experiment would better reveal the potential differences between feedback types and the effect of learning. The experiment followed a repeated measures design with counter- balancing of the order of the feedback types between sessions and participants. We ran a couple of pilot participants to test the modified setup. It was soon found that the original 860 ms dwell time felt too long already after the first session. After a few adjustments, it seemed even as short as 500 ms would work as people seemed to learn quite fast and they were eager to reduce the dwell time quite soon. This is in line with previous studies where participants have adjusted the dwell time already in the first session (Majaranta et al. 2009; Räihä & Ovaska, 2012). However, we were aware that 500 ms may be too fast for the participants’ first encountered with eye typing. Therefore, in the practice session that preceded the three sessions analyzed below it was set to 860 ms. Although the animated circle was not a part of the second experiment, it was used as the only feedback in the practice session. Task and Procedure. The task and the procedure were the same as in the first experiment except that the number of sessions was now three. In order to keep the time in the lab reasonable, the sessions were organized in two different days. The practice phase and the first experimental session were completed on the first day. The second and the third sessions were completed on another day within a week from the first day. Between the second and the third session, participants took a break of few minutes. During the break the participants were instructed to stretch or walk. Measurements. The same measurements were analyzed as in the first experiment (speed, accuracy, subjective experience). In addition, we collected two measures on gaze behavior. During eye typing, the user has to look at the virtual keys to select them. In order to review the text written so far, the user has to switch her gaze from the keyboard to the text entry field. Read text events (RTE) per character is a measure of the frequency of gaze switching between the virtual keyboard and the text entry field. Using dwell-selection, the user has to fixate on the key for long enough to select it. Re-focus events (RFE) is the measure of the number of premature exits per character, indicating the user looked away from the character too early and thus had to re-focus on it to select it. All focus-and-leave events were recorded, not only finally selected characters. A single gaze sample outside the key did not yet increase the RFE count, because the experimental software used an intelligent algorithm (described in (Räihä, 2015)) to filter out outliers: single gaze sam- ples outside of the currently focused key did not reset the dwell time counter for that key but only decreased its accumulated time by the time spent outside, allowing the dwelling to continue when the gaze returned to the key. Thus, a real fixation on the other key was required for a focus/unfocus event. The software includes an option for the dwell-time feedback delay, which was set to 100 ms in this experiment. The feedback on focus (change of the background color of the key) was only shown after the minimum dwell time was full. The results were analyzed using the same statistical methods as in the first experiment. 19 phrases (out of 1048 in total) were left out of the analysis because they were either unfinished (due to poor calibration that forced re-calibration in the middle of the phrase) or semantically incorrect (e.g. the participant misremembered the phrase and thus replaced whole words). Typing speed. The grand mean for typing speed was 11.20 wpm (SD 1.42), with the average of 11.80 wpm (SD 1.60) for auditory, 11.34 wpm (SD 1.31) for haptic and 10.46 wpm (SD 1.39) for visual feedback. The repeated measures ANOVA showed that the feedback mode had a statistically significant effect on the typing speed (F(1.73,18.99) = 10.02, p=.002, Greenhouse- Geisser corrected). Pairwise t-tests showed that visual feedback differed significantly from auditory (p=.003) and haptic feedback (p=.006) but there was no statistical significance between auditory and haptic feedback (p=.135). Most participants maintained a typing speed well above ten words per minute but one participant was clearly slower than the others (reaching 5.56 wpm on average). However, since that person finished all conditions and the effect (i.e. the average speed and deviation) was similar in all conditions and sessions, the data from this participant were included in the analysis. A small learning effect was seen in the increase of the typing speed, from an average of 10.91 wpm (SD 1.46) in the first session, 11.10 wpm (SD 1.42) ...
Similar publications
Today’s Augmented Reality (AR) devices enable users to interact almost naturally with their surroundings, e.g., by pinning digital content onto real-world objects. However, current AR display are mostly limited to optical and video see-through technologies. Nevertheless, extending Augmented Reality (AR) beyond screens by accommodating additional mo...
Citations
... The possibility to communicate in a gaze-based way is of special relevance for motor-impaired patients who might not be able to rely on traditional communication techniques such as mouse or keyboard use (Slobodenyuk, 2016). Previous research has demonstrated the applicability of eye movements as input devices in fields like drawing applications (Hornof & Cavender, 2005;van der Kamp & Sundstedt, 2011), gaming (Corcoran, Nanu, Petrescu, & Bigioi, 2012), typing (Akkil et al., 2016;Mott, Williams, Wobbrock, & Morris, 2017), or web browsing (Abe, Owada, Ohi, & Ohyama, 2008). Consequently, the gaze itself can take over several functions (see Majaranta, Räihä, Hyrskykari, & Špakov, 2019, for a recent review), such as pointing (Asai et al., 2000), zooming (Adams, Witkowski, & Spence, 2008;Halwani, Salcudean, Lessoway, & Fels, 2017), or object selection (Tanriverdi & Jacob, 2000;Urbina & Huckauf, 2008). ...
![Eva Katharina Riechelmann [verh. Steinbacher]](https://i1.rgstatic.net/ii/profile.image/778385039061000-1562592863919_Q64/Eva-Riechelmann.jpg)
Humans use their eyes not only as visual input devices to perceive the environment, but also as an action tool in order to generate intended effects in their environment. For instance, glances are used to direct someone else's attention to a place of interest, indicating that gaze control is an important part of social communication. Previous research on gaze control in a social context mainly focused on the gaze recipient by asking how humans respond to perceived gaze (gaze cueing). So far, this perspective has hardly considered the actor’s point of view by neglecting to investigate what mental processes are involved when actors decide to perform an eye movement to trigger a gaze response in another person. Furthermore, eye movements are also used to affect the non-social environment, for instance when unlocking the smartphone with the help of the eyes. This and other observations demonstrate the necessity to consider gaze control in contexts other than social communication whilst at the same time focusing on commonalities and differences inherent to the nature of a social (vs. non-social) action context. Thus, the present work explores the cognitive mechanisms that control such goal-oriented eye movements in both social and non-social contexts. The experiments presented throughout this work are built on pre-established paradigms from both the oculomotor research domain and from basic cognitive psychology. These paradigms are based on the principle of ideomotor action control, which provides an explanatory framework for understanding how goal-oriented, intentional actions come into being. The ideomotor idea suggests that humans acquire associations between their actions and the resulting effects, which can be accessed in a bi-directional manner: Actions can trigger anticipations of their effects, but the anticipated resulting effects can also trigger the associated actions. According to ideomotor theory, action generation involves the mental anticipation of the intended effect (i.e., the action goal) to activate the associated motor pattern. The present experiments involve situations where participants control the gaze of a virtual face via their eye movements. The triggered gaze responses of the virtual face are consistent to the participant’s eye movements, representing visual action effects. Experimental situations are varied with respect to determinants of action-effect learning (e.g., contingency, contiguity, action mode during acquisition) in order to unravel the underlying dynamics of oculomotor control in these situations. In addition to faces, conditions involving changes in non-social objects were included to address the question of whether mechanisms underlying gaze control differ for social versus non-social context situations. The results of the present work can be summarized into three major findings. 1. My data suggest that humans indeed acquire bi-directional associations between their eye movements and the subsequently perceived gaze response of another person, which in turn affect oculomotor action control via the anticipation of the intended effects. The observed results show for the first time that eye movements in a gaze-interaction scenario are represented in terms of their gaze response in others. This observation is in line with the ideomotor theory of action control. 2. The present series of experiments confirms and extends pioneering results of Huestegge and Kreutzfeldt (2012) with respect to the significant influence of action effects in gaze control. I have shown that the results of Huestegge and Kreutzfeldt (2012) can be replicated across different contexts with different stimulus material given that the perceived action effects were sufficiently salient. 3. Furthermore, I could show that mechanisms of gaze control in a social gaze-interaction context do not appear to be qualitatively different from those in a non-social context. All in all, the results support recent theoretical claims emphasizing the role of anticipation-based action control in social interaction. Moreover, my results suggest that anticipation-based gaze control in a social context is based on the same general psychological mechanisms as ideomotor gaze control, and thus should be considered as an integral part rather than as a special form of ideomotor gaze control.
... Previous studies where gaze and haptic interactions are used have explored different body locations for providing vibrotactile feedback. Vibrotactile feedback was provided on palm of the hand [Pakkanen et al., 2008], fingers Majaranta et al., 2016], wrist , back and head [Kangas et al., 2014b;Rantala et al., 2015]. ...
... In an eye-typing study, Majaranta et al. (2016) . ...
Eyes are the window to the world, and most of the input from the surrounding environment is captured through the eyes. In Human-Computer Interaction too, gaze based interactions are gaining prominence, where the user’s gaze acts as an input to the system. Of late portable and inexpensive eye-tracking devices have made inroads in the market, opening up wider possibilities for interacting with a gaze. However, research on feedback to the gaze-based events is limited. This thesis proposes to study vibrotactile feedback to gaze-based interactions.
This thesis presents a study conducted to evaluate different types of vibrotactile feedback and their role in response to a gaze-based event. For this study, an experimental setup was designed wherein when the user fixated the gaze on a functional object, vibrotactile feedback was provided either on the wrist or on the glasses. The study seeks to answer questions such as the helpfulness of vibrotactile feedback in identifying functional objects, user preference for the type of vibrotactile feedback, and user preference of the location of the feedback. The results of this study indicate that vibrotactile feedback was an important factor in identifying the functional object. The preference for the type of vibrotactile feedback was somewhat inconclusive as there were wide variations among the users over the type of vibrotactile feedback. The personal preference largely influenced the choice of location for receiving the feedback.
... Compared to the slow and deliberate way of operating a mouse or other input device, eye movements usually scan the screen involuntarily, for example the user is not aware of the jittery motions during a fixation. Moreover, eyes are used primarily for perception [18] and they typically precede actions [1,14,17,27]. Thus, the additional use for control requires careful design of human-computer interaction interfaces [3,13,20] in order to provide adequate feedback and to avoid false activation. ...
... Despite the problems of using the same modality for both perception and control, gaze estimation and eye tracking have important application areas ranging from medical diagnosis and psychological research to the design of interfaces and usability studies of gaze-controlled applications from the field of human-computer interaction [17]. Probably the most common example is eye-typing [15,16,18,19,25,28]. Other applications include object selection on interfaces [12] and in real world [26], target search and selection [28], computer game control (ranging from simple puzzles [1] and classical games [6] to role-playing and first-person shooter video games [11,27]), facilitating attention switching between multiple visual displays [13], robotic device control [7], web-browsing [5,14], interacting with geographic information systems [3], developing interactive graphical user interface elements [20], projected display control in automotive and military aviation environments [23]. ...
... Using gaze as an input method may not be comparable to the universal mouse and keyboard [14] because of the nature of the human eye movements and the physiology of the eye [17], yet it can still have several advantages. For people with physical disabilities gaze-based interaction provides a means to communicate and interact with technology and other people [5,6,17,18,20]. The performance of children with disabilities can be enhanced considerably through gaze controlled computers [10]. In the case of older adults, it may be able to compensate for the declined motor functions when using mouse input [21]. ...
In this work we investigate the effects of switching from mouse cursor control to gaze-based control in a computerized divided attention game. We conducted experiments with nine participants performing a task that requires continuous focused concentration and frequent shifts of attention. Despite carefully controlling experimental and design aspects, the performance of subjects was considerably impaired when using gaze-based control. The participants were experienced users of the mouse control version of the task, we adjusted the difficulty to the more demanding conditions and selected the parameters of gaze input based on previous research findings. In contrast to our assumptions, experienced users could not get used to gaze-based control in the amount of experiments we performed. Additionally we consider the strategies of users, i.e. their method of problem solving, and found that it is possible to make progress in our task even during a short amount of practice. The results of this study provide evidence that the adoption of interfaces controlled by human eye-gaze in cognitively demanding environments require careful design, proper testing and sufficient user training.
... We made no special effort to block out sounds generated by the vibrotactile actuators because they would be present in real products as well. However, in some cases (Köpsel, Majaranta, Isokoski, & Huckauf, 2016;Majaranta et al., 2016) the actuators were placed on a pillow to prevent disturbing sound from the vibration. Participants were recruited mainly from the local University community, and all signed informed consent forms before proceeding to the experiments. ...
... Therefore, feedback that is not co-located seems a more suitable design option in this context. To explore the possible locations, we gave vibrotactile feedback to the palm of the hand (Kangas et al., 2014a;Kangas et al., 2014b;Köpsel et al., 2016), fingers (Kangas et al., 2014d;Käki et al., 2014;Majaranta et al., 2016;Zhi, 2014), wrist Majaranta et al., 2016), back , and head (Kangas et al., 2014c;Rantala, Kangas, Akkil, Isokoski, & Raisamo, 2014;Rantala et al., 2015;Špakov et al., 2015). We investigated how people experience and react to vibrotactile feedback presented to different body locations. ...
... Therefore, feedback that is not co-located seems a more suitable design option in this context. To explore the possible locations, we gave vibrotactile feedback to the palm of the hand (Kangas et al., 2014a;Kangas et al., 2014b;Köpsel et al., 2016), fingers (Kangas et al., 2014d;Käki et al., 2014;Majaranta et al., 2016;Zhi, 2014), wrist Majaranta et al., 2016), back , and head (Kangas et al., 2014c;Rantala, Kangas, Akkil, Isokoski, & Raisamo, 2014;Rantala et al., 2015;Špakov et al., 2015). We investigated how people experience and react to vibrotactile feedback presented to different body locations. ...
Vibrotactile feedback is widely used in mobile devices because it provides a discreet and private feedback channel. Gaze based interaction, on the other hand, is useful in various applications due to its unique capability to convey the focus of interest. Gaze input is naturally available as people typically look at things they operate, but feedback from eye movements is primarily visual. Gaze interaction and the use of vibrotactile feedback have been two parallel fields of human-computer interaction research with a limited connection. Our aim was to build this connection by studying the temporal and spatial mechanisms of supporting gaze input with vibrotactile feedback. The results of a series of experiments showed that the temporal distance between a gaze event and vibrotactile feedback should be less than 250 milliseconds to ensure that the input and output are perceived as connected. The effectiveness of vibrotactile feedback was largely independent of the spatial body location of vibrotactile actuators. In comparison to other modalities, vibrotactile feedback performed equally to auditory and visual feedback. Vibrotactile feedback can be especially beneficial when other modalities are unavailable or difficult to perceive. Based on the findings, we present design guidelines for supporting gaze interaction with vibrotactile feedback.
We present a new application (“Sakura”) that enables people with physical impairments to produce creative visual design work using a multimodal gaze approach. The system integrates multiple features tailored for gaze interaction including the selection of design artefacts via a novel grid approach, control methods for manipulating canvas objects, creative typography, a new color selection approach, and a customizable guide technique facilitating the alignment of design elements. A user evaluation (N=24) found that non-disabled users were able to utilize the application to complete common design activities and that they rated the system positively in terms of usability. A follow-up study with physically impaired participants (N=6) demonstrated they were able to control the system when working towards a website design, rating the application as having a good level of usability. Our research highlights new directions in making creative activities more accessible for people with physical impairments.
Gaze provides an attractive input channel for human-computer interaction because of its capability to convey the focus of interest. Gaze input allows people with severe disabilities to communicate with eyes alone. The advances in eye tracking technology and its reduced cost make it an increasingly interesting option to be added to the conventional modalities in every day applications. For example, gaze-aware games can enhance the gaming experience by providing timely effects at the right location, knowing exactly where the player is focusing at each moment. However, using gaze both as a viewing organ as well as a control method poses some challenges. In this chapter, we will give an introduction to using gaze as an input method. We will show how to use gaze as an explicit control method and how to exploit it subtly in the background as an additional information channel. We will summarize research on the application of different types of eye movements in interaction and present research-based design guidelines for coping with typical challenges. We will also discuss the role of gaze in multimodal, pervasive and mobile interfaces and contemplate with ideas for future developments.
User-centered design questions in gaze interfaces have been explored in multitude empiri-cal investigations. Interestingly, the question of what eye should be the input device has never been studied. We compared tracking accuracy between the "cyclopean" (i.e., mid-point between eyes) dominant and non-dominant eye. In two experiments, participants performed tracking tasks. In Experiment 1, participants did not use a crosshair. Results showed that mean distance from target was smaller with cyclopean than with dominant or non-dominant eyes. In Experiment 2, participants controlled a crosshair with their cyclo-pean, dominant and non-dominant eye intermittently and had to align the crosshair with the target. Overall tracking accuracy was highest with cyclopean eye, yet similar between cyclopean and dominant eye in the second half of the experiment. From a theoretical viewpoint, our findings correspond with the cyclopean eye theory of egocentric direction and provide indication for eye dominance, in accordance with the hemispheric laterality approach. From a practical viewpoint, we show that what eye to use as input should be a design consideration in gaze interfaces.
