AltTyping virtual keyboard for eye typing. 

Contexts in source publication

Context 1

... bar) indicates the progression of time, making the dwell time process visible to the user (Lankford, 2000; Hansen, Hansen, & Johansen, 2001). Proper feedback helps the user to keep her gaze fixated on the focused item for dwell-selection (Majaranta et al. 2006). However, short dwell times do not leave much time for animated feedback. Thus, short dwell times may benefit from a sharp, distinct feedback on selection (Majaranta el al. 2006). A meta-analysis of several different studies (not relat- ed to eye typing) indicates that complementing visual feedback with auditory or tactile feedback improves performance (Burke, et al., 2006). This is likely because human response times to multimodal feedback are faster than to unimodal feedback. Hecht, Reiner and Havely (2006) found an average response time of 362 ms to unimodal visual, auditory and haptic feedback. For bi- modal and trimodal feedback the average response times were 293 and 270 ms, respectively. Response times vary also between modalities. Results of studies on measuring perceptual latency generally agree that both auditory and haptic stimuli alone are processed faster than visual stimuli, and that auditory stimuli are processed faster than haptic stimuli (Hirsh & Sherrick, 1961; Harrar & Harris, 2008). The latency for perceiving the haptic stimuli is dependent on the location; a signal from fingers takes a different time than a signal from the head (Harrar & Harris, 2005). Haptic feedback, i.e. feedback sensed through the sense of touch, has been used extensively in communication aids targeted for people with limited sight or the blind (Jansson, 2008). We were interested in measuring if and how haptic feedback may facilitate eye typing. To our knowledge, haptic feedback has not been studied in the context of eye typing. Haptic feedback has been found beneficial in virtual touchscreen keyboards operated by manual touch. A good-quality tactile feedback improves typing speed and reduces error rate (Hoggan, Brewster, & Johnston, 2008; Koskinen, Kaaresoja, & Laitinen, 2008). Haptic feedback can be used to inform the user about gaze events, provided the feedback is given within the duration of a fixation (Kangas, Rantala, Majaranta, Isokoski, & Raisamo, 2014c). If gaze is used to control a hand-held device such as a mobile phone, haptic feedback can be given via the phone. Kangas et al. (2014b) found that haptic feedback on gaze gestures improved user performance and satisfaction on handheld devices. In addition to hand-held devices, head-worn eye trackers provide natural physical contact point for haptic feedback (Rantala, Kangas, Akkil, Isokoski, & Raisamo, 2014). Finally, in addition to quantitative measures, feedback also has an effect on the qualitative measures. Proper feedback can ease learning and improve user satisfaction and user experience in general. This applies to visual and auditory (Majaranta et al. 2006) as well as haptic feedback (Pakkanen, Raisamo, Raisamo, Salminen, & Surakka, 2010). Despite the earlier findings cited above, it was not clear that haptic feedback would work well in eye typing, where the task is heavily focused on rapid serial pointing with the eyes. In addition, there is a good opportunity for giving visual and auditory feedback, because the gaze stays stationary during the dwell time and audio is also easy to perceive. On the other hand, it was tempting to speculate that adding the tactile feedback that is not natu- rally present in eye typing could improve the user experience and possibly also the eye typing performance. Thus, we found experiments necessary to better understand how haptic feedback works in eye typing. The purpose of the first experiment was to get first impressions on using haptic feedback in eye typing and to find out how well a haptic confirmation of selection performs in comparison with visual or auditory confirmations. Twelve university students (8 male, 4 female, aged between 20 to 35, mean 24 years) were recruited for the experiment. Students were rewarded with extra points that counted towards passing a course for their participation in the experiment. All were novices in eye typing; six had some experience in eye tracking (e.g. having participated in a demonstration). One participant wore eye glasses. None had problems in seeing, hearing or tactile perception and all were able-bodied. Eye tracking. Tobii T60 (Tobii Technology, Sweden) was used to track the eye movements, together with a Windows XP laptop. The tracker’s built-in 17-inch TFT monitor (with resolution of 1280x1024) was used as the primary monitor in the experiment. Experimental software . AltTyping, built on ETU- Driver (developed by Oleg Š pakov, University of Tampere) was used to run the experiment and to log event data. AltTyping includes a setup mode where one can ad- just the layout of the virtual keyboard. The layout of the keyboard was adjusted so that it included letters and punctuation required in the experiment (see Figure 1). In addition, there were a few function keys, differentiated from the characters by green background (letter keys had blue background). The backspace key (marked with an arrow symbol ‘ ! ’) was used to delete the last character. A shift key ( ⌂ ) changed the whole keyboard into capitals; after selection of a key, the keyboard returned to lower case configuration. A special ‘Ready’ key ( ☺ ) marked the end of the typing of the current sentence and loaded the next sentence. It was located separately from the other keys to avoid accidentally ending a sentence prematurely. Dwell time was used to select a key. Based on pilot tests, the dwell time for key selection was set to 860 ms. Progression of the dwell time was indicated by a dark blue animated, closing circle (see ‘n’ in Figure 1). This animation was same for all conditions. In the end of the animation, selection was confirmed either by an audible click, a short haptic vibration, or visual flash of the key, as described below. Visual feedback. Visual feedback for selection was shown on the screen as a “flash” of the selected key (the background color of the key changed to dark red for 100 ms, with constant intensity and tone). The confirmation feedback on selection was shown as soon as the dwell time was exceeded (immediately after the animated circle had closed). Auditory feedback. Auditory feedback played a ‘click’ sound to confirm selection via ordinary desktop loud- speakers. The sound was similar to the default click sound used by Microsoft Windows (see Figure 2, top). Haptic Feedback. EAI C2 Tactor vibrotactile actuator, controlled with a Gigaport HD USB sound card, was used for the haptic feedback. We chose vibrotactile actuators over other actuation technologies such as shear (Winfield, Glassmire, Colgate, & Peshkin, 2007) and indentation (Rantala, et al., 2011) because vibrotactile actuators are compact and easy to attach to different body locations. The C2 actuator converted an audio file (a 100 ms file with a sinusoidal wave, with a constant amplitude of 250 Hz) to vibration (see Figure 2, bottom). Our aim in creating feedbacks was to make them suit- able for the modality and as short as possible. The auditory feedback was easily perceived even at its 15ms length. The visual feedback was set to 100 ms long to make it clearly perceivable; with a shorter duration it could be missed e.g. due to a blink. Also the haptic feedback set to 100 ms duration would make it easy to perceive. Since it was possible that the participant heard some sound from the vibrotactile actuators, and we wanted to isolate the effect of auditory feedback from haptic feedback, the participants wore hearing protectors. The haptic actuator was placed on the participant’s dorsal wrist. It was held in place by an adjustable Velcro strap. We considered a wrist band as a convenient way to attach the haptic actuator. Since some intelligent wrist watches already include inbuilt haptic feedback (e.g. Apple Watch with haptic alerts), this setup is also relevant to design in the wristwatch form factor. The timing of initiation of the feedback was the same for all feedback types (they started immediately as the dwell time had run out). The experienced “strength” (intensity) of each type was set in pilot tests to be as similar as possible, although it may be considered rather subjective. In practice, for example, the same signal out- putted as audio and haptic would cause quite different perceived “strength”. Therefore, the haptic wave had to be amplified to produce an experience that is similar in intensity with the auditory click. In other words, a sound constructed from a signal of very low amplitude is still easy to perceive but vibrotactile feedback constructed from the same low amplitude signal would be barely noticeable. The task that the participants completed in the experiment was to transcribe a phrase of text repeatedly. The phrases were randomly drawn from the set of 500 phrases originally published by MacKenzie and Soukoreff (2003). Because the participants were Finnish, the Finnish translation of the phrase set was used. The phrase set was translated in Finnish by Isokoski and Linden (2004). A few typos were corrected and the punctuation was unified for this experiment. The stimulus phrase was shown on top of the experimental software and the text written by the participant appeared in an input field below the target text (Figure 1). The participants were first briefed about the motiva- tion of the study and the experimental procedure. Each participant was asked to read and sign an informed con- sent form and to fill in a demographics form. The participant was seated in front of the eye tracker so that the distance between the participant’s eyes and the camera was about 60-70 cm. The chair position was fixed but participant’s movements were not restricted. The eye tracker was calibrated before the task started and recali- brated again before each test block. ...

Context 2

... the eye movements, together with a Windows XP laptop. The tracker’s built-in 17-inch TFT monitor (with resolution of 1280x1024) was used as the primary monitor in the experiment. Experimental software . AltTyping, built on ETU- Driver (developed by Oleg Š pakov, University of Tampere) was used to run the experiment and to log event data. AltTyping includes a setup mode where one can ad- just the layout of the virtual keyboard. The layout of the keyboard was adjusted so that it included letters and punctuation required in the experiment (see Figure 1). In addition, there were a few function keys, differentiated from the characters by green background (letter keys had blue background). The backspace key (marked with an arrow symbol ‘ ! ’) was used to delete the last character. A shift key ( ⌂ ) changed the whole keyboard into capitals; after selection of a key, the keyboard returned to lower case configuration. A special ‘Ready’ key ( ☺ ) marked the end of the typing of the current sentence and loaded the next sentence. It was located separately from the other keys to avoid accidentally ending a sentence prematurely. Dwell time was used to select a key. Based on pilot tests, the dwell time for key selection was set to 860 ms. Progression of the dwell time was indicated by a dark blue animated, closing circle (see ‘n’ in Figure 1). This animation was same for all conditions. In the end of the animation, selection was confirmed either by an audible click, a short haptic vibration, or visual flash of the key, as described below. Visual feedback. Visual feedback for selection was shown on the screen as a “flash” of the selected key (the background color of the key changed to dark red for 100 ms, with constant intensity and tone). The confirmation feedback on selection was shown as soon as the dwell time was exceeded (immediately after the animated circle had closed). Auditory feedback. Auditory feedback played a ‘click’ sound to confirm selection via ordinary desktop loud- speakers. The sound was similar to the default click sound used by Microsoft Windows (see Figure 2, top). Haptic Feedback. EAI C2 Tactor vibrotactile actuator, controlled with a Gigaport HD USB sound card, was used for the haptic feedback. We chose vibrotactile actuators over other actuation technologies such as shear (Winfield, Glassmire, Colgate, & Peshkin, 2007) and indentation (Rantala, et al., 2011) because vibrotactile actuators are compact and easy to attach to different body locations. The C2 actuator converted an audio file (a 100 ms file with a sinusoidal wave, with a constant amplitude of 250 Hz) to vibration (see Figure 2, bottom). Our aim in creating feedbacks was to make them suit- able for the modality and as short as possible. The auditory feedback was easily perceived even at its 15ms length. The visual feedback was set to 100 ms long to make it clearly perceivable; with a shorter duration it could be missed e.g. due to a blink. Also the haptic feedback set to 100 ms duration would make it easy to perceive. Since it was possible that the participant heard some sound from the vibrotactile actuators, and we wanted to isolate the effect of auditory feedback from haptic feedback, the participants wore hearing protectors. The haptic actuator was placed on the participant’s dorsal wrist. It was held in place by an adjustable Velcro strap. We considered a wrist band as a convenient way to attach the haptic actuator. Since some intelligent wrist watches already include inbuilt haptic feedback (e.g. Apple Watch with haptic alerts), this setup is also relevant to design in the wristwatch form factor. The timing of initiation of the feedback was the same for all feedback types (they started immediately as the dwell time had run out). The experienced “strength” (intensity) of each type was set in pilot tests to be as similar as possible, although it may be considered rather subjective. In practice, for example, the same signal out- putted as audio and haptic would cause quite different perceived “strength”. Therefore, the haptic wave had to be amplified to produce an experience that is similar in intensity with the auditory click. In other words, a sound constructed from a signal of very low amplitude is still easy to perceive but vibrotactile feedback constructed from the same low amplitude signal would be barely noticeable. The task that the participants completed in the experiment was to transcribe a phrase of text repeatedly. The phrases were randomly drawn from the set of 500 phrases originally published by MacKenzie and Soukoreff (2003). Because the participants were Finnish, the Finnish translation of the phrase set was used. The phrase set was translated in Finnish by Isokoski and Linden (2004). A few typos were corrected and the punctuation was unified for this experiment. The stimulus phrase was shown on top of the experimental software and the text written by the participant appeared in an input field below the target text (Figure 1). The participants were first briefed about the motiva- tion of the study and the experimental procedure. Each participant was asked to read and sign an informed con- sent form and to fill in a demographics form. The participant was seated in front of the eye tracker so that the distance between the participant’s eyes and the camera was about 60-70 cm. The chair position was fixed but participant’s movements were not restricted. The eye tracker was calibrated before the task started and recali- brated again before each test block. In the haptic condi- tion, the setup was checked to make sure the haptic feedback was easily perceivable. Prior to the experiment, the participant had a chance to practice eye typing. The setup and the task during training were similar to the actual experiment but the feedback was somewhat different: During practice, we used the default visual feedback given by AltTyping. When the participant focused on a key, it visually rose (as if the eye was pulling it up). After the 1000 ms dwell time used during the training had elapsed, the key went down (like it was pressed by an invisible finger). After 150 ms the key returned to its default state. During the experiment, the phrases were presented one at a time. The participant was instructed to first read and memorize the phrase and then type it as fast and as accurately as possible. The presented phrase remained visible during the task, thus the participant could read it again if necessary. The participants were instructed to correct errors if they noticed them immediately but leave errors that they noticed later in the middle of the text. After finishing the phrase, the participant activated the Ready key that loaded the next phrase. The typing time for each condition was set to five minutes. For analysis purposes, this preset typing time included only active typing time (from the first entered key to the selection of the last key, before the selection of the Ready key). However, if the preset time had elapsed during typing, the participant could finish the last phrase without interrup- tion. After each 5-minute block of phrases, the participant filled in a questionnaire about his/her subjective experience of the feedback used in that block. The experiment then continued with different feedback. After all feedback conditions had been completed, the participant filled in a questionnaire comparing the feedback modes. The session ended with a short interview. The exploratory study included one session that con- sisted of the training and three 5-minute blocks (one for each feedback type). The whole experiment, including instructions and post-test questionnaires took about one hour. The experiment had a within-subject design where each participant tested all feedback conditions. The order of the feedback types was counter-balanced between participants. The feedback type was the independent variable. Dependent variables included the performance and user experience measurements described below. Speed. Text entry rate was measured in words per minute (WPM) , where a word is defined as a sequence of five characters, including letters, punctuation and space (MacKenzie, 2003). The phrase typing duration was measured as the interval between the first and last key selection. The selection of Ready key was not included, as it only served for stimuli changing, not for text entry. Accuracy . The error rate percentage was calculated by comparing the transcribed text with the presented text, using the minimum string distance method. Error rate only evaluated the result but did not take into account the corrected errors. Keystrokes per character (KSPC) was used to calculate the overhead incurred in correcting errors (Soukoreff & MacKenzie, 2001). Subjective experience. Participants’ perceived usability and subjective satisfaction was evaluated with questionnaires. The questionnaire was given after each feedback mode, concerning perceived speed and accuracy, learning, pleasure, arousal, concentration requirements, tiredness, consistency and understandability of each feedback, with a scale from 1 (e.g. very low) to 7 (e.g. very high). In the end (after all conditions), we also asked the participants’ feedback mode preferences. They were asked to indicate which feedback mode they liked the best and which the least, which was most clear, trustworthy and pleasant. We also asked if any of the feedback modes was more “dominant (stronger)” than the others. Repeated measures ANOVA was used to test for statistically significant differences. Pairwise T-tests were used to analyze differences between specific conditions when the ANOVA suggested a difference among more than two conditions. Unfinished phrases and semantically incorrect phrases were left out of the analysis (8 phrases out of 287 phrases in total). Semantic errors in the typing occurred when the participant skipped, added, or mis-recalled whole words (e.g. “Never too rich ” versus “Never ...

Context 3

... response time of 362 ms to unimodal visual, auditory and haptic feedback. For bi- modal and trimodal feedback the average response times were 293 and 270 ms, respectively. Response times vary also between modalities. Results of studies on measuring perceptual latency generally agree that both auditory and haptic stimuli alone are processed faster than visual stimuli, and that auditory stimuli are processed faster than haptic stimuli (Hirsh & Sherrick, 1961; Harrar & Harris, 2008). The latency for perceiving the haptic stimuli is dependent on the location; a signal from fingers takes a different time than a signal from the head (Harrar & Harris, 2005). Haptic feedback, i.e. feedback sensed through the sense of touch, has been used extensively in communication aids targeted for people with limited sight or the blind (Jansson, 2008). We were interested in measuring if and how haptic feedback may facilitate eye typing. To our knowledge, haptic feedback has not been studied in the context of eye typing. Haptic feedback has been found beneficial in virtual touchscreen keyboards operated by manual touch. A good-quality tactile feedback improves typing speed and reduces error rate (Hoggan, Brewster, & Johnston, 2008; Koskinen, Kaaresoja, & Laitinen, 2008). Haptic feedback can be used to inform the user about gaze events, provided the feedback is given within the duration of a fixation (Kangas, Rantala, Majaranta, Isokoski, & Raisamo, 2014c). If gaze is used to control a hand-held device such as a mobile phone, haptic feedback can be given via the phone. Kangas et al. (2014b) found that haptic feedback on gaze gestures improved user performance and satisfaction on handheld devices. In addition to hand-held devices, head-worn eye trackers provide natural physical contact point for haptic feedback (Rantala, Kangas, Akkil, Isokoski, & Raisamo, 2014). Finally, in addition to quantitative measures, feedback also has an effect on the qualitative measures. Proper feedback can ease learning and improve user satisfaction and user experience in general. This applies to visual and auditory (Majaranta et al. 2006) as well as haptic feedback (Pakkanen, Raisamo, Raisamo, Salminen, & Surakka, 2010). Despite the earlier findings cited above, it was not clear that haptic feedback would work well in eye typing, where the task is heavily focused on rapid serial pointing with the eyes. In addition, there is a good opportunity for giving visual and auditory feedback, because the gaze stays stationary during the dwell time and audio is also easy to perceive. On the other hand, it was tempting to speculate that adding the tactile feedback that is not natu- rally present in eye typing could improve the user experience and possibly also the eye typing performance. Thus, we found experiments necessary to better understand how haptic feedback works in eye typing. The purpose of the first experiment was to get first impressions on using haptic feedback in eye typing and to find out how well a haptic confirmation of selection performs in comparison with visual or auditory confirmations. Twelve university students (8 male, 4 female, aged between 20 to 35, mean 24 years) were recruited for the experiment. Students were rewarded with extra points that counted towards passing a course for their participation in the experiment. All were novices in eye typing; six had some experience in eye tracking (e.g. having participated in a demonstration). One participant wore eye glasses. None had problems in seeing, hearing or tactile perception and all were able-bodied. Eye tracking. Tobii T60 (Tobii Technology, Sweden) was used to track the eye movements, together with a Windows XP laptop. The tracker’s built-in 17-inch TFT monitor (with resolution of 1280x1024) was used as the primary monitor in the experiment. Experimental software . AltTyping, built on ETU- Driver (developed by Oleg Š pakov, University of Tampere) was used to run the experiment and to log event data. AltTyping includes a setup mode where one can ad- just the layout of the virtual keyboard. The layout of the keyboard was adjusted so that it included letters and punctuation required in the experiment (see Figure 1). In addition, there were a few function keys, differentiated from the characters by green background (letter keys had blue background). The backspace key (marked with an arrow symbol ‘ ! ’) was used to delete the last character. A shift key ( ⌂ ) changed the whole keyboard into capitals; after selection of a key, the keyboard returned to lower case configuration. A special ‘Ready’ key ( ☺ ) marked the end of the typing of the current sentence and loaded the next sentence. It was located separately from the other keys to avoid accidentally ending a sentence prematurely. Dwell time was used to select a key. Based on pilot tests, the dwell time for key selection was set to 860 ms. Progression of the dwell time was indicated by a dark blue animated, closing circle (see ‘n’ in Figure 1). This animation was same for all conditions. In the end of the animation, selection was confirmed either by an audible click, a short haptic vibration, or visual flash of the key, as described below. Visual feedback. Visual feedback for selection was shown on the screen as a “flash” of the selected key (the background color of the key changed to dark red for 100 ms, with constant intensity and tone). The confirmation feedback on selection was shown as soon as the dwell time was exceeded (immediately after the animated circle had closed). Auditory feedback. Auditory feedback played a ‘click’ sound to confirm selection via ordinary desktop loud- speakers. The sound was similar to the default click sound used by Microsoft Windows (see Figure 2, top). Haptic Feedback. EAI C2 Tactor vibrotactile actuator, controlled with a Gigaport HD USB sound card, was used for the haptic feedback. We chose vibrotactile actuators over other actuation technologies such as shear (Winfield, Glassmire, Colgate, & Peshkin, 2007) and indentation (Rantala, et al., 2011) because vibrotactile actuators are compact and easy to attach to different body locations. The C2 actuator converted an audio file (a 100 ms file with a sinusoidal wave, with a constant amplitude of 250 Hz) to vibration (see Figure 2, bottom). Our aim in creating feedbacks was to make them suit- able for the modality and as short as possible. The auditory feedback was easily perceived even at its 15ms length. The visual feedback was set to 100 ms long to make it clearly perceivable; with a shorter duration it could be missed e.g. due to a blink. Also the haptic feedback set to 100 ms duration would make it easy to perceive. Since it was possible that the participant heard some sound from the vibrotactile actuators, and we wanted to isolate the effect of auditory feedback from haptic feedback, the participants wore hearing protectors. The haptic actuator was placed on the participant’s dorsal wrist. It was held in place by an adjustable Velcro strap. We considered a wrist band as a convenient way to attach the haptic actuator. Since some intelligent wrist watches already include inbuilt haptic feedback (e.g. Apple Watch with haptic alerts), this setup is also relevant to design in the wristwatch form factor. The timing of initiation of the feedback was the same for all feedback types (they started immediately as the dwell time had run out). The experienced “strength” (intensity) of each type was set in pilot tests to be as similar as possible, although it may be considered rather subjective. In practice, for example, the same signal out- putted as audio and haptic would cause quite different perceived “strength”. Therefore, the haptic wave had to be amplified to produce an experience that is similar in intensity with the auditory click. In other words, a sound constructed from a signal of very low amplitude is still easy to perceive but vibrotactile feedback constructed from the same low amplitude signal would be barely noticeable. The task that the participants completed in the experiment was to transcribe a phrase of text repeatedly. The phrases were randomly drawn from the set of 500 phrases originally published by MacKenzie and Soukoreff (2003). Because the participants were Finnish, the Finnish translation of the phrase set was used. The phrase set was translated in Finnish by Isokoski and Linden (2004). A few typos were corrected and the punctuation was unified for this experiment. The stimulus phrase was shown on top of the experimental software and the text written by the participant appeared in an input field below the target text (Figure 1). The participants were first briefed about the motiva- tion of the study and the experimental procedure. Each participant was asked to read and sign an informed con- sent form and to fill in a demographics form. The participant was seated in front of the eye tracker so that the distance between the participant’s eyes and the camera was about 60-70 cm. The chair position was fixed but participant’s movements were not restricted. The eye tracker was calibrated before the task started and recali- brated again before each test block. In the haptic condi- tion, the setup was checked to make sure the haptic feedback was easily perceivable. Prior to the experiment, the participant had a chance to practice eye typing. The setup and the task during training were similar to the actual experiment but the feedback was somewhat different: During practice, we used the default visual feedback given by AltTyping. When the participant focused on a key, it visually rose (as if the eye was pulling it up). After the 1000 ms dwell time used during the training had elapsed, the key went down (like it was pressed by an invisible finger). After 150 ms the key returned to its default state. During the experiment, the phrases were presented one at a time. The participant was instructed to first read and memorize the phrase and then ...