Conference PaperPDF Available

Secondary Task Boundaries Influence Drivers' Glance Durations

Abstract and Figures

Drivers show a wide range of behavior while performing a secondary task behind the wheel. In the current study, we categorized drivers into groups based on their glance behavior at task boundary (i.e., pressing a touch screen button after reading driving-related messages) and compared driving capabilities of drivers in each group. The comparison between the groups identifies different eye glance strategies, or task switching decisions, and associated vehicle control behaviors. Senders' uncertainty model was adapted to explain the results and to suggest future directions in developing driver models.
Content may be subject to copyright.
Secondary Task Boundaries Influence
Drivers’ Glance Durations
Ja Young Lee
University of Wisconsin-
Madison
Madison, WI, USA
jayoung.lee@wisc.edu
Madeleine Gibson
University of Wisconsin-
Madison
Madison, WI, USA
mcgibson2@wisc.edu
John D. Lee
University of Wisconsin-
Madison
Madison, WI, USA
john.d.lee@wisc.edu
ABSTRACT
Drivers show a wide range of behavior while performing a
secondary task behind the wheel. In the current study, we
categorized drivers into groups based on their glance
behavior at task boundary (i.e., pressing a touch screen
button after reading driving-related messages) and compared
driving capabilities of drivers in each group. The comparison
between the groups identifies different eye glance strategies,
or task switching decisions, and associated vehicle control
behaviors. Senders’ uncertainty model was adapted to
explain the results and to suggest future directions in
developing driver models.
Author Keywords
Driving; distraction; eye glance; task switching; in-vehicle
system interface
ACM Classification Keywords
H.1.2. Models and Principles: User/Machine Systems –
Human factors; H.5.2. Information interfaces and
presentation (e.g., HCI): User Interfaces – Theory and
methods
INTRODUCTION
Vehicles today are equipped with systems to improve the
driving experience. Unfortunately, drivers interact with these
systems by interleaving attention between the forward
roadway and the system, potentially undermining safety. An
important requirement for building safer vehicles is
predicting how drivers interleave attention to driving and
secondary tasks. This prediction can be used to design in-
vehicle interface that does not require long glances that can
lead to crashes. However, such predictions are challenging
because of substantial variability between and within drivers:
different drivers exhibit different behaviors, and even a
single driver might perform the same task many different
ways.
For this reason, a growing body of research aims to explain
how drivers’ shift attention between the driving task and the
secondary task. Janssen, Brumby, and Garnett [5] proposed
that people are more likely to switch attention at subtask
boundaries that are signaled with cognitive cues (e.g., block
boundaries of phone number such as 1-800-867-5309) or
motor cues (e.g., when hand needs to move a long distance).
Suspending secondary tasks at such boundaries reduces
mental effort [1] by reducing working memory load and task
resumption easier [2, 4].
Switching attention back to the road depends on subtask
boundary, but also on the road state. Senders et al. [7]
suggested that drivers maintain off-road glances until the
estimated level of uncertainty reaches each individual’s
internal threshold. The uncertainty accumulates over time,
proportionally to the instability of the vehicle and variability
of the environment (speed of vehicle, information density of
the roadway, etc.) as well as the individual differences in
weighting and forgetting information. These factors are
expressed formally as:
∗∗
∗∗1
∗
The first term of this equation represents vehicle instability
(K: constant related to vehicle stability relative to road, V:
vehicle velocity, t: time occluded) and the second term
represents lost information over time (H: road information
density, D: weight on information, F: rate of forgetting). If
one’s uncertainty of the road is below threshold (U), the
driver will continue to look away from the road. Once the
uncertainty accrues over one’s threshold, the driver will look
back to the road, to reduce the discomfort of high
uncertainty. This model predicts long eye glances away from
the road when the uncertainty grows slowly and when the
threshold for the uncertainty is high.
Thus, the decision to switch attention back to the road
depends on how individuals balance cognitive effort and
uncertainty. If a driver continues performing a task beyond a
subtask boundary, a larger portion of the task would be
completed, but the uncertainty of roadway demands would
increase as the glance duration increases. If the driver
switches attention to the road at the subtask boundary, the
driver can better control the vehicle and more easily recall
the task state. For example, if the driver switches attention to
the road after reading a complete sentence, the task state
Permission to make digital or hard copies of all or part of this work fo
r
p
ersonal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
b
ear this notice and the full citation on the first page. Copyrights fo
r
components of this work owned by others than ACM must be honored.
Abstracting with credit is permitted. To copy otherwise, or republish, to
p
ost on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from Permissions@acm.org.
A
utomotiveUI '15, September 01 - 03, 2015, Nottingham, United Kingdom
© 2015 ACM. ISBN 978-1-4503-3736-6/15/09…$15.00
DOI: http://dx.doi.org/10.1145/2799250.2799269
(e.g., which word was I reading?) is easier to remember than
that of suspending the task in the middle of reading a
sentence. The total time to complete the task, however, is
likely to increase due to resumption cost.
In the current study, we analyzed drivers’ eye glance patterns
at subtask boundaries to examine strategies in task switching
decisions. Our task required participants to read messages
related to driving and press a touch screen button to display
the next page or to finish the task. The button press between
pages or at the end of the reading task represents a subtask
boundary, for two reasons. First, because a button press
demands manual resources, but not necessarily visual
resources, visual resources could be released for driving task,
and so the button press introduces a possible point for task
switching. Second, pressing button also refreshed the screen
and displayed new texts to read, which acted as a placeholder
that reduces the need to remember the current subtask state
(e.g., where I was reading). Thus, whenever a button press
was required, the driver could choose either to turn attention
to the road or to continue with the secondary task.
Based on the glance patterns at the subtask boundary, we
defined two groups of drivers and compared their driving
behavior to infer what influenced switching behavior. In
particular, we examined whether the task switching decision
was influenced more by the accumulation of uncertainty or
by the difference in threshold, from the perspective of the
Senders’ uncertainty model.
METHODS
Participants
24 participants from 4 different age groups (18-24, 25-39,
40-54, and 55+ years old, 6 from each age group) with an
equal number of males and females (3 from each gender)
completed the study. This sampling follows the NHTSA
visual-manual guidelines [6].
Driving Task
Following the NHTSA visual-manual guidelines,
participants were instructed to drive in the right lane at a
speed of 50 mph until the lead vehicle appeared. Participants
were instructed to maintain a fixed following distance of 220
feet for the entire experiment. The simulated environment
was an undivided, four lane highway with a shoulder on each
side.
Secondary Task
Participants completed all three conditions of the reading
task while driving: one screen, two screens, and two screens
with a system delay. Trials were randomized within a
condition block, and the execution order of condition block
was also randomized. Messages for the tasks were modified
from the study conducted by Boyle et al. (2013). To complete
each respective task, drivers had to perform the following
actions:
One screen: read texts press ‘enter’
Two screens: read texts press ‘next’ read texts
press ‘enter’
Two screens with delay: read texts press ‘next’
wait until the next texts appears on screen read texts
press ‘enter’
Once participants pressed the “enter” button, an auditory
statement was played that was related to the phrase they just
read. Participants decided whether that statement was true or
false and pressed the corresponding button on the
touchscreen. Participants were given 40 seconds to complete
each trial. Conditions contained 12 trials: 6 with short text
(20-40 characters; 4-6 words) and 6 with long text (120-140
characters; 20-26 words). Participants performed each of the
three conditions for a total of 36 trials. Conditions were
counter-balanced across participants.
For the two-screen tasks, the button press was introduced to
define subtask boundaries. At these subtask boundaries,
drivers could decide to either begin reading the second
screen or to shift their visual attention away from the screen
and back to the roadway.
Figure 1 One screen reading task with pressing ‘enter’ (left) and two-screen reading task with pressing ‘next’ in between the screens
and ‘enter’ at the end (right). Whenever there was a system delay, the ‘next’ button was greyed out and the second screen was not
displayed until the end of the delay.
Materials
The simulator experiment was configured as shown in Figure
2. Equipment included:
Simulator: The simulator used for the study was the
NADS MiniSim using the driving scenario defined by
the NHTSA visual-manual guidelines. The NADS
MiniSim consists of one 65” screen placed at a distance
of about 54” from the driver and includes a standard
steering and pedal setup.
Touch screen: A 7” touchscreen was attached to the
NADS MiniSim at a typical center-stack position. This
interface displayed the text messages. Participants
moved between screens with button-presses on the
touchscreen.
Camera setup: A GoPro camera was mounted to record
drivers’ head and eye movements.
Figure 2 Driving simulator with touchscreen to the right of the
steering wheel
Procedure
Upon arrival, participants read and signed the informed
consent form and completed a demographics and driving
history survey. The experimenter orally reviewed the
instructions for the tasks and participants were shown
examples of each task. Each participant completed a three-
minute practice drive and received feedback on their
performance maintaining a 220-foot following distance from
the lead vehicle. Participants repeated the practice drive until
they felt comfortable with the task. When the practice ended,
participants completed a wellness questionnaire to assess any
simulator sickness symptoms experienced.
Following the practice drive, participants completed a
baseline drive-only trial lasting five minutes. Then
participants performed the data collection trials blocked by
condition. Each condition began with four practice trials,
followed by twelve data collection trials. Each trial lasted
one minute, thus each condition took approximately 16
minutes to complete. When the data collection ended,
participants were compensated for their time.
Data Preparation
Driving measures such as steering angle and lane position
were directly collected by the NADS MiniSim. Eye glance
data was manually coded following the NHTSA visual-
manual guidelines by four researchers using a video logging
tool Morae®. The two data sets were time synced based on
the start time of each trial.
RESULTS
Eye Glance Strategies at Subtask Boundary
To analyze the eye glances related to subtask boundaries, we
selected off-road glances associated with button press for
each trial. These glances started before the button press and
ended before or after the button press. For the one-screen
condition, we used glances associated with ‘enter’ button
presses, which was the last eye glance away from the road
that related to the reading task. For the two-screen conditions
(both no delay and delay conditions), we used glances
associated with ‘next’ button press. When a driver pressed
the ‘next’ button, the second screen was displayed. Thus
these glances were the last eye glances for the first screen,
and were followed by eye glances for reading text on the
second screen.
Glances associated with button press were longer and more
dispersed compared to other glances not associated with a
button press (Figure 3). The 85th percentile of “button press”
glance was 2.54s, and that of “other (non-button press)”
glance was 2.00s. Particularly long glances occurred
relatively frequently when the drivers were pressing the
button, compared to all other glances. The proportion of long
glances over 2 seconds increased with button press in all
three conditions: 19.3% to 32.6% (1 Screen condition),
11.9% to 28.1% (2 Screen No Delay condition), and 14.5%
to 25.9% (2 Screen Delay condition). This implies that the
subtask boundary influenced drivers’ glance pattern in a way
that undermines safety in all three conditions.
Figure 3 Distribution of glance durations at button press (top)
and at other times (bottom)
Figure 4 shows glances near the time of the button press. The
horizontal time axis starts from 2 seconds before the button
press and ends at 2 seconds after the button press. The button
press happened at time zero, which is marked with dashed
vertical line. Each row consists of black, white, and grey
lines indicating glance types. Black lines indicate off-road
0.0
0.2
0.4
0.6
0.0
0.2
0.4
0.6
Buttonpress O ther
012345
Glance duration (s)
Density
Condit ion
1 Screen
2 Screen, No Delay
2 Screen, Delay
glances, white lines indicate glances to the forward roadway,
and grey lines indicate other types of glances, such as looking
at a dashboard, looking down, closing eyes, and so forth.
Each row indicates one trial for a participant, and rows are
sorted by the end time of the off-road glance near the button
press.
Figure 4 Timelines of off-road glances, shown as horizontal
black lines, relative to the button press, which occurred at time
zero and marked with vertical dashed line
Almost all glances in the one-screen condition ended within
500ms of the button press that signified task completion.
This means eye glances did not stay on the screen for more
than 500ms after the task was completed. However, in the
two-screen condition, where button presses indicated next-
screen request, a large proportion of the glances extended
beyond 500ms after the button press, and some extended to
almost 2000ms after the button press. This effect was less
pronounced when the delay occurred after the button press.
Figure 4 also shows that glances do not always overlap the
button press. The ones that ended before the button press are
noteworthy because they indicate that some participants did
not need to look at the screen to press the button.
By grouping the glances by participants, we found that
drivers tend to adopt a consistent glance pattern. Some of the
drivers looked back to the road before the button press, while
others looked at the screen. Figure 5 shows when eye glances
to the secondary screen ended relative to the button press,
which happened at zero. A negative y-value means that the
driver looked back on to the road before the button press,
while a positive y-value means that the driver looked back
on to the road after the button press. Each point in Figure 5
represents the mean value for a single driver and the order of
drivers is the same for each panel. The first point in the lower
left of the leftmost panel is the same driver as the first point
in the lower left of the rightmost panel. This pattern shows
that drivers exhibit consistent individual differences in how
they distribute glances around the button press. Based on the
mean ‘end of glance time’, we categorized drivers into three
groups. We first selected participants whose end of glance
time was lower than -1 standard deviation (five before type)
and then selected the matching number of participants from
the other end (five long after type). The rest of the
participants were then categorized into shortly after type.
Figure 5 The time to return eyes to the road, relative to the button press time at 0. Drivers are ordered identically across conditions
On average, before button press switchers looked back to the
road 199ms before the press the button, whereas long after
button press switchers looked back to the road 505ms after
the button press. The demographics of each group suggests a
relationship between age and glance strategy. Among five
before drivers, four were from 55+ years old age group, and
one was from the 40-54 group. Long after type switchers
were younger: of five, two were from 18-24 group, one was
from 25-39 group, and the other two were from 40-54 group.
Figure 6 Distribution of mean glance duration for the driver
types
1 Screen 2 Screen 2 Screen, D elay
-2 -1 0 1 2
Relative time (s)
1 Screen 2 Screen 2 Screen, Delay
0.0
0.5
1.0
Participants
End of glance time
relative to button press (s)
Switch type
Before
Shortly After
Long After
0.0
0.5
1.0
01234
Mean Glance Duration (s)
Density
Switch type
Before
Shortly After
Long After
The pattern observed in Figure 5 implies that before
switchers would have short glances in general. However,
compared to the overall mean glance duration for each trial,
the before switchers did not necessarily have consistently
shorter glances. Yet long after switchers had longer mean
glances away from the road than drivers in other groups.
Figure 6 shows that only long after type drivers show notably
long mean glance durations for each trial.
These driver types also showed different glance patterns
during the button press. As seen in Figure 7, before type
switchers showed reduced glance duration during a button
press. On the other hand, other types of drivers showed
extended glances during a button press. The magnitude was
greater for long after type switchers. The button press action,
glance type, and task condition significantly influenced
glance duration (F(1, 4983.8) = 143.14, p<.001; F(2, 22.6) =
11.03, p<.001; F(2, 4982.3)=12.02, p<.001). The interaction
between the button press and glance type was also significant
(F(2, 4983.9) = 75.35, p<.001).
Because the long after type switchers looked away from the
road longer than before type switchers, it might be expected
that vehicle control would suffer. In the next section, we
compare the vehicle control capabilities of before and long
after group by analyzing steering behavior and lane keeping
behavior.
Figure 7 Glance duration of button press glances and other glances in each condition and glance type. The vertical lines indicate 95%
confidence interval as a function of button press action, glance type, and task condition.
Vehicle Control of Drivers
Figure 8 presents the absolute value of steering angle over
six seconds, starting from one second before the button press
to five seconds after. Each line is a trace of steering angle in
one trial for one participant. Before switchers showed
quicker and harder steering (on the top panels) than long
after switchers (on the bottom panels). It implies that long
glances away from the road do not always require an
aggressive steering to recover.
Figure 8 Trace of absolute steering angle relative to the time
after the button press (button press at 0, represented with
vertical dashed line)
Figure 9 presents the lane position over the same six seconds
as Figure 8. The width of the lane was 12 feet with center at
y = 0, with y = 6 being the right most edge of the roadway.
Again, each line is a trace of lane position in one trial for one
participant. Any trace over 2.67 or under -2.67 was colored
dark, as the vehicle encroached on the neighboring lane when
the center of vehicle trace is not within this range.
Figure 9 Lane position on relative time axis (button press at 0,
represented with vertical dashed line)
Before type drivers’ vehicles trace were mostly centered in
the lane around y = 0. Their active steering shown in Figure
1 Screen 2 Screen 2 S creen Delay
1.0
1.5
2.0
2.5
Other Butt onpress Ot her Butt onpress Other Buttonpress
Glance duration (s)
Glance type
Before
Shortly After
Long After
8 was effective in keeping the vehicle in the center of the
road in general. On the other hand, the traces of long after
type drivers’ vehicle implied that they allowed the vehicle to
deviate from the center: they deviated further from the center
more but their steering was less active. Interestingly, these
drivers deviated to the right side of the road where there was
a shoulder (positive lane position in Figure 9), more than to
the left side of the road where there was an additional lane.
This might be an intentional movement to avoid traffic, as
the drivers were driving on the right most lane of four-lane
highway with shoulders, or an unintentional movement
caused by reaching to the tasks located to the right of the
driver.
Figure 10 shows the average distance of the vehicle from the
center of the lane, 500ms periods before and after the glance
returns to the roadway. This measure shows the average gap
between the lane center and a vehicle over time. The result
indicates that the long after type people stayed away from
the road center more than others, and it is consistent with the
more dark traces shown in Figure 9.
Figure 10 Absolute lane position 500ms before and after glance
returned to road
Yet, the standard deviation of lane position was higher in
before type drivers for the two screen conditions, as seen in
Figure 11. Taken together, this pattern indicates that the
before group people steered more aggressively to maintain a
consistent position within the lane.
Figure 11 Standard deviation of lane position 500ms before and
after glance returned to the road
This the better vehicle control of the long after group seemed
to compensate for the long glances. When we calculated the
time to cross the lane based on the lateral vehicle velocity
and lane position at the end of each glance, the predicted time
to cross the lane boundary did not differ in the before and
long after group. Long after group drivers could maintain the
heading of vehicle and drive parallel to the roadway even
though they were staying further away from the lane center.
With long glance away from the road, their ability to drive
straight was as well as the before group drivers who had
shorter glances away from the road.
DISCUSSION
The subtask boundary defined by the button press was
associated with longer off-road glances. More importantly,
drivers adopted distinctive patterns of glances at the subtask
boundary that were not reflected in the mean glance duration.
Some drivers switched attention to the road before the button
press. For these drivers the button press defined a subtask
boundary. Other drivers maintained attention to the
secondary screen long after the button press, showing greatly
extended glances, in addition to a generally long glance
duration overall. The steering profile of these drivers was
also different: before drivers were more active in steering but
less capable of smooth driving. Interestingly, those who had
long glances at the subtask boundary had less variable lane
position, but drove less closely to the center of the lane.
Nevertheless, these drivers tended to drift to the right road
shoulder when they moved away from the lane center. As a
result, the lateral vehicle control of the after group was as
effective as those who actively steered to stay in the center
of the lane. However, the similar control achieved even with
long glances does not reflect the risk associated with
longitudinal hazards, such as sudden breaking of the lead
vehicle. Long glances of these people (and even longer
glances at the button press) can increase the risk of missing
abrupt changes of situation.
Implications for Driver Model
Different eye glance patterns at subtask boundaries suggest
systematic individual differences in how drivers control the
vehicle and divide their attention between the road and a
secondary task. Although those with long eye glances look
away from road even longer at task boundaries, they were
still able to steer smoothly and maintain the vehicle within
the lane. This has implications for Senders’ uncertainty
model of the driving situation that guides glance duration.
There are two possible sources of individual difference in
task switching and associated glance patterns. Drivers’
tolerance for uncertainty might differ or drivers’ rate of
uncertainty accumulation might differ. The current study
supports the latter. If different thresholds determine task
switching, drivers with long glances might have let
uncertainty accumulate more for the longer duration and
result in less stable vehicle control. However, vehicle control
was smoother for drivers who took longer glances. This
suggests that control ability and the rate of uncertainty
growth differentiate between driver types.
1 Screen 2 Screen 2 Screen Delay
1.0
1.5
2.0
500ms
Befo re
500ms
After
500ms
Befo re
500ms
After
500ms
Befo re
500ms
After
Absolute Lane Position (ft)
Glance typ e
Bef ore
Shortly Af ter
Long Af ter
1 Screen 2 Screen 2 Screen Delay
0.02
0.03
0.04
0.05
0.06
500ms
Bef ore
500ms
After
500ms
Bef ore
500ms
After
500ms
Bef ore
500ms
After
SD Lane Posi tion (ft )
Glance typ e
Bef ore
Shortly Af ter
Long Af ter
Uncertainty accumulates slowly with a more capable
controller, and such a driver would perform secondary tasks
with longer eye glances. This driver might feel safe to press
a button, continue the task and extend glance duration across
a subtask boundary. However, this propensity can impede the
driver’s capability to detect sudden events in the forward
roadway. On the other hand, a driver whose uncertainty
accumulates more rapidly would have relatively short
glances. At subtask boundary, this driver would switch
attention back to the road and control the vehicle instead of
reading the next screen. This type of driver might be averse
to long glances. In the experiment, these people compensated
with aggressive vehicle control and relatively short glances.
The current experiment used a straight roadway and included
only one lead vehicle driving at a constant speed. This simple
driving scenario limits the amount of uncertainty associated
with the roadway and so uncertainty accumulates according
to the capacity of the driver. In more complex and dynamic
driving situations, uncertainty accumulation would depend
less on the drivers’ capacity. Whether the individual
differences observed in this study would generalize to more
complex roadway environments is unknown. Additionally,
this simple scenario does not reveal how different types of
drivers deal with longitudinal road hazards. Testing different
scenarios would be helpful to evaluate drivers’ vehicle
control in more detail.
The current study observed eye glance patterns at an explicit
subtask boundary (i.e., button press) that was defined by a
motor cue (i.e., move hand to press) and cognitive cue (i.e.,
end of a set of sentences). These cues made it clear to that
this is the natural break point. However, because the button
press action requires manual movement contrary to reading,
it can also be a task in and of itself. Thus, studying glance
patterns defined by implicit subtask boundaries, such as
compound word entry, would better address self-paced task
chunking, which is common.
The age difference in two groups is potentially an important
result, although it is not a focus of the current study. Younger
and experienced driver might exhibit different strategies in
controlling the vehicle and chunking a task into subtasks,
with younger drivers willing to extend glances over subtask
boundaries and look away from the road for relatively long
periods.
CONCLUSION
Individuals showed different glance patterns at subtask
boundaries. A majority of drivers extended glances when
encountering a button press and such a tendency was greater
for those who had a long mean glance duration. Other drivers
showed generally shorter glances, particularly at the task
boundary. These drivers were more intense but less capable
in controlling the vehicle, suggesting individual differences
in glance patterns may depend on the rate of uncertainty
growth rather than differences in tolerance to uncertainty.
These results highlight different glance strategies that can
trigger drivers’ latent unsafe behaviors. Future models to
estimate the distraction potential of in-vehicle devices should
consider individual difference associated with the rate of
uncertainty accumulation.
ACKNOWLEDGMENTS
This material is based upon work supported by the U.S. Dept.
of Transportation – National Highway Traffic Safety
Administration under Contract No. DTNH22-11-D-00237.
Any opinions, findings, and conclusions or
recommendations expressed in this publication are those of
the authors and do not necessarily reflect the views of the
NHTSA and/or the U.S. DOT.
REFERENCES
1. Adamczyk, P. D., & Bailey, B. P. 2004. If not now,
when?: the effects of interruption at different moments
within task execution. In Proceedings of the SIGCHI
conference on Human factors in computing systems
(CHI'04), 271-278.
2. Altmann, E. M., & Trafton, J. G. 2002. Memory for
goals: An activation-based model. Cognitive science,
26, 1: 39-83.
3. Anderson, J. R., Bothell, D., Lebiere, C., & Matessa,
M. 1998. An integrated theory of list memory. Journal
of Memory and Language, 38, 4: 341-380.
4. Borst, J. P., Taatgen, N. A., & van Rijn, H. 2010. The
problem state: a cognitive bottleneck in multitasking.
Journal of Experimental Psychology: Learning,
memory, and cognition, 36, 2: 363.
5. Janssen, C. P., Brumby, D. P., & Garnett, R. 2012.
Natural Break Points The Influence of Priorities and
Cognitive and Motor Cues on Dual-Task Interleaving.
Journal of Cognitive Engineering and Decision
Making, 6, 1: 5-29.
6. National Highway Traffic Safety Administration. 2014.
Visual-manual NHTSA driver distraction guidelines
for in-vehicle electronic devices. Washington, DC:
National Highway Traffic Safety Administration
(NHTSA), Department of Transportation (DOT).
7. Senders, J. W., Kristofferson, A. B., Levison, W. H.,
Dietrich, C. W., & Ward, J. L. 1967. The attentional
demand of automobile driving. Highway Research
Record, 195: 15-33.
... The model predicts long glances away from the road during secondary activities when the uncertainty grows slowly and/or when the individual threshold for the uncertainty is high. Lee et al. (2015) have utilized the model successfully in analysing and recognizing different off-road glancing patterns as well as task factors (number of screens, delay) affecting these while driving and concurrently reading messages at an in-car display. They concluded that the drivers who showed generally shorter glances off road were more intense but less capable of controlling the vehicle, suggesting the observed individual differences in glance patterns off road may depend more on the rate of the growth of uncertainty rather than differences in tolerance of uncertainty. ...
... The metric of OD has been utilized as a baseline for acceptable off-road glance durations for in-car user interface testing in dynamic, self-paced driving scenarios (Kujala, Grahn, Mäkelä, and Lasch, 2016). Based on the findings of Lee et al. (2015), the constant Umax k is likely to be determined by two individual factors instead of one, that is, of the uncertainty tolerance threshold as well as the rate of the growth of uncertainty. This complicates the interpretation of the OD measurements for a road environment, but for a sufficient driver sample, average OD can be taken as an estimate of the objective visual demands of the road environment. ...
... From the viewpoint of minimizing uncertainty, attentional demand for allocating attention to a source of information with an expected event and the related value of the sampled information can be actually rather low (Clark, 2013). From this perspective, SEEV and similar models are useful for predicting attention allocation for an experienced operator for a well-defined task and task environment (a prescriptive model), but are limited in their ability to describe or predict individual behaviours in uncertain tasks contexts (e.g., for operators with different uncertainty growth rates; see Lee et al., 2015 for a novice operator with under-developed expectancies, or for environments of high variability in task-relevant event rates). For instance, in the uncertainty model of , discussed earlier, the event density and visual demand is still very much coupled with the (objective) information bandwidth of the road, but overall, the uncertainty-based models of attention allocation seem to provide feasible solutions for predicting individual differences in the control of attention (of focal vision, in particular) in multitasking and are well in line with more general theories of brains as prediction engines (e.g., Clark, 2013Clark, , 2015. ...
Chapter
Our world today is increasingly technological and interconnected: from smart phones to smart homes, from social media to streaming video, technology users are bombarded with digital information and distractions. These many advances have led to a steep increase in user multitasking, as users struggle to keep pace with the dizzying array of functions at their fingertips. The field of human- computer interaction (HCI) has also struggled to keep pace, ever trying to understand how multitasking affects user behaviour and how new technologies might be better designed to alleviate the potential problems that arise from multi-tasking and distraction. This chapter surveys a number of recent approaches to understanding user multitasking using computational models. While strengths and weaknesses of the different approaches vary greatly, together they have shown great promise in helping to understand how users perform multiple tasks and the implications for task performance. Computational models are critical not only for scientific exploration of user behaviours, but also as computational engines embedded into running systems that mimic, predict, and potentially mitigate interruption and distraction.
... Yet another design factor that could diminish drivers' visual distraction while conducting secondary in-car tasks, are well designed task structures (i.e., "how a task breaks down into smaller subtasks" [Salvucci and Kujala, 2016]) that are based on scientific knowledge of human multitasking behavior. It has previously been observed that people have a tendency to switch tasks at subtask boundaries (e.g., Janssen et al., 2012;Lee et al., 2015;Lee and Lee, 2019;Salvucci and Kujala, 2016), for instance dialing a phone number in chunks or typing one word at a time (Janssen et al., 2012). Empirical evidence also suggests that when the duration of a secondary visual search task increases, the glance durations tend to increase as well (Kujala and Salvucci, 2015;Lee et al., 2012). ...
... Based on these findings, increase in the preferred number of visual or visual-manual interaction steps during an in-car glance (e.g., pressing one button vs. typing one word), increases both the duration of the in-car glance as well as its visual distraction potential. These observations of subtask boundaries support the previous findings of, for instance, Janssen et al. (2012), Lee et al. (2015) and Lee and Lee (2019). ...
Article
Visual distraction by secondary in-car tasks is a major contributing factor in traffic incidents. In-car user interface design may mitigate these negative effects but to accomplish this, design factors’ visual distraction potential should be better understood. The effects of touch screen size, user interface design, and subtask boundaries on in-car task's visual demand and visual distraction potential were studied in two driving simulator experiments with 48 participants. Multilevel modeling was utilized to control the visual demands of driving and individual differences on in-car glance durations. The 2.5” larger touch screen slightly decreased the in-car glance durations and had a diminishing impact on both visual demand and visual distraction potential of the secondary task. Larger relative impact was discovered concerning user interface design: an automotive-targeted application decreased the visual demand and visual distraction potential of the in-car tasks compared to the use of regular smartphone applications. Also, impact of subtask boundaries was discovered: increase in the preferred number of visual or visual-manual interaction steps during a single in-car glance (e.g., pressing one button vs. typing one word) increased the duration of the in-car glance and its visual distraction potential. The findings also emphasize that even if increasing visual demand of a task – as measured by in-car glance duration or number of glances – may increase its visual distraction potential, these two are not necessarily equal.
... For example, when studying the effect of subtask boundaries on longer off-road glances, amalgamated measures such as mean glance duration is not sensitive enough to assess such subtle addition of long glance or to verify the effect of the construct on glance (J. Y. Lee, Gibson, & Lee, 2015). In the study, J. Y. Lee et al. (2015) analyzed how microstructure of secondary task of text reading and entry is associated with long off-road glances which is not reflected in the mean glance duration. ...
... Y. Lee, Gibson, & Lee, 2015). In the study, J. Y. Lee et al. (2015) analyzed how microstructure of secondary task of text reading and entry is associated with long off-road glances which is not reflected in the mean glance duration. Glance extensions at subtask boundaries contribute to longer tail of the of the glance duration distribution and change the 85 th percentile relative to the median. ...
Article
This study assessed whether quantile regression can identify design specifications that lead to particularly long glances, which might go unnoticed with traditional analyses focusing on conditional means of off-road glances. Although substantial research indicates that long glances contribute disproportionately to crash risk, few studies have directly assessed the tails of the distribution. Failing to examine the distribution tails might underestimate the disproportionate risk on long glances imposed by secondary tasks. We applied quantile regression to assess the effects of secondary task type (reading or entry), system delay (delay or no delay), and text length (long or short) on off-road glance duration at 15th, 50th, and 85th quantiles. The results show that entry task, long text, and some combinations of variables led to longer glances than that would be expected given the central tendency of glance distributions. Quantile regression identifies secondary task features that produce long glances, which might be neglected by traditional analyses with conditional means.
... s) during the distraction testing. These drivers could have been insecure regarding their driving skills and therefore tried to keep their in-car glances as short as possible. In addition, the structure of the tested task may have allowed the drivers to use subtask boundaries as natural break points during the task completion (Janssen et al., 2012;J. Y. Lee et al., 2015;J. Y. Lee & Lee, 2019;Salvucci & Kujala, 2016) in order to avoid inappropriately long in-car glances. However, the glancing behavior of drivers with low occlusion distance resulted in passing the distraction testing -even when it failed with other randomly mixed samples which had similar occlusion distance distribution as in the original ...
Article
Driver distraction is a recognized cause of traffic accidents. Although the well-known guidelines for measuring distraction of secondary in-car tasks were published by the United States National Highway Traffic Safety Administration (NHTSA) in 2013, studies have raised concerns on the accuracy of the method defined in the guidelines, namely criticizing them for basing the diversity of the driver sample on driver age, and for inconsistent between-group results. In fact, it was recently discovered that the NHTSA driving simulator test is susceptible to rather fortuitous results when the participant sample is randomized. This suggests that the results of said test are highly dependent on the selected participants, rather than on the phenomenon being studied, for example, the effects of touch screen size on driver distraction. As an attempt to refine the current guidelines, we set out to study whether a previously proposed new testing method is as susceptible to the effects of participant randomization as the NHTSA method. This new testing method differs from the NHTSA method by two major accounts. First, the new method considers occlusion distance (i.e., how far a driver can drive with their vision covered) rather than age, and second, the new method considers driving in a more complex, and arguably, a more realistic environment than proposed in the NHTSA guidelines. Our results imply that the new method is less susceptible to sample randomization, and that occlusion distance appears a more robust criterion for driver sampling than merely driver age. Our results are applicable in further developing driver distraction guidelines and provide empirical evidence on the effect of individual differences in drivers’ glancing behavior.
... The task being split into multiple screens in the smartwatch condition may have provided natural breakpoints to enable the drivers to glance back at the road, mitigating the longer average glance durations observed with the smartwatch in Experiment I (as evidenced by a lack of difference between the two device conditions in Experiment II and by the larger number of glances per notification observed in Experiment II but not in Experiment I). These kinds of secondary task boundaries have been shown to influence driver glance and task switching behavior in other simulator studies (Lee, Gibson, & Lee, 2015). ...
Chapter
This work seeks to understand whether the unique features of a smartwatch, compared to a smartphone, mitigate or exacerbate driver distraction due to notifications, and to provide insights about drivers' perceptions of the risks associated with using smartwatches while driving. As smartwatches are gaining popularity among consumers, there is a need to understand how smartwatch use may influence driving performance. Previous driving research has examined voice calling on smartwatches, but not interactions with notifications, a key marketed feature. Engaging with notifications (e.g., reading and texting) on a handheld device is a known distraction associated with increased crash risks. Two driving simulator studies compared smartwatch to smartphone notifications. Experiment I asked participants to read aloud brief text notifications and Experiment II had participants manually select a response to arithmetic questions presented as notifications. Both experiments investigated the resulting glances to and physical interactions with the devices, as well as self-reported risk perception. Experiment II also investigated driving performance and self-reported knowledge/expectation about legislation surrounding the use of smart devices while driving. Experiment I found that participants were faster to visually engage with the notification on the smartwatch than the smartphone, took longer to finish reading aloud the notifications, and exhibited more glances longer than 1.6 s. Experiment II found that participants took longer to reply to notifications and had longer overall glance durations on the smartwatch than the smartphone, along with longer brake reaction times to lead vehicle braking events. Compared to the no device baseline, both devices increased lane position variability and resulted in higher self-reported perceived risk. Experiment II participants also considered that smartwatch use while driving deserves penalties equal to or less than smartphone use. The findings suggest that smartwatches may have road safety consequences. Given the common view among participants to associate smartwatch use with equal or less traffic penalties than smartphone use, there may be a disconnect between drivers' actual performance and their perceptions about smartwatch use while driving.
... It would be reasonable, if it is more challenging to handle two lagging systems (the car and the randomly delayed IVIS). Lee et al. (2015) also used a system with a delay in one experimental condition. The delay was further specified by online communication 3 : 800 ms-2500 ms. ...
Thesis
Full-text available
The assessment and empirical testing of the potential for interfaces to distract drivers is a time-consuming and costly issue in the automobile industry. This topic is addressed and supported by different guidelines and standards. For human factors engineering, it would be beneficial to obtain an approximate idea concerning the performance of a task in driver distraction testing before undertaking the experiments. This could improve suitable interaction design at an early stage e.g., during (paper) prototyping. In this thesis, a prediction model is implemented (open source) and evaluated. The approach is based on measuring subtasks and storing their results in a database. From the subtask database, complete tasks can be assembled. The subtasks were measured from 24 subjects. A separate prediction is calculated for each subject based on synthesized subtasks (virtual experiment). From these 24 values (distribution), characteristic values such as the 85 th percentile can be derived. After discussing the properties of delays, System Response Times are incorporated into the prediction model and are used in an evaluation experiment to test the model. It is demonstrated that System Response Times can have an impact on distraction metrics. These delays can (mathematically) lower Single Glance Durations. Typical driver distraction metrics are reviewed and enhanced (e.g., for lateral driving performance and Single Glance Durations). The prediction model incorporates 13 metrics: • Total Time on Task (TTT static; non-driving) • Total Time on Task while driving • Glance-Total Glance Time (task related) • Glance-Single Glance Duration (task related) • Glance-Number of Glances (task related) • Glance-Total Eyes-Off-Road Time • Glance-Single Glance Duration (eyes-off-road) • Glance-Number of Glances (eyes-off-road) • Occlusion-Total Shutter Open Time (TSOT) • Occlusion-R-Metric (TSOT/TTT) • Tactile Detection Response Task (TDRT)-Deterioration in Reaction Time (%) • Driving-Deterioration in Lateral Drift (%) • Driving-Deterioration in Longitudinal Drift of Headway (%) An evaluation experiment with 24 subjects revealed that most of these predictions could be a helpful support. When excluding the unreliably predictable Deterioration in Longitudinal Drift of Headway, the average percentage error of predictions to measurements was 16%, with a mean coefficient of determination R² = .614.
... In the future, we hope to extend the model and test whether a greater number of subtask boundaries benefits driving safety. We can also include more parameters that account for other aspects of behavior, such as individual differences in the tolerance of uncertainty or response to task boundary (Lee, Gibson, & Lee, 2015), to enhance external validity of the model to real world driving. The model developed in this study also has implications for future models that extend beyond driver. ...
Studies of multitasking while driving have shown that drivers tend to switch attention at subtask boundaries. It is also known that the uncertainty of roadway information plays a significant role in attention switching. Yet, these two approaches have not been modeled together. In this study, we create an attention switching model that accounts for both subtask boundaries and uncertainty, and use Approximate Bayesian Computation-Markov Chain Monte Carlo (ABC- MCMC) to determine the weight between the two factors, based on the empirical data. The weight was calculated for each of two different types of tasks, text reading and entry, that have subtask boundaries with different characteristics. We found that the subtask boundary in the text reading task nudged drivers to discontinue the distracting task and switch attention back to the road more than the subtask boundary in the text entry task. This study suggests that task structure may play a role in generating long glances.
... The task being split into multiple screens in the smartwatch condition may have provided natural breakpoints to enable the drivers to glance back at the road, mitigating the longer average glance durations observed with the smartwatch in Experiment I (as evidenced by a lack of difference between the two device conditions in Experiment II and by the larger number of glances per notification observed in Experiment II but not in Experiment I). These kinds of secondary task boundaries have been shown to influence driver glance and task switching behavior in other simulator studies (Lee, Gibson, & Lee, 2015). ...
Article
Full-text available
This work seeks to understand whether the unique features of a smartwatch, compared to a smartphone, mitigate or exacerbate driver distraction due to notifications, and to provide insights about drivers' perceptions of the risks associated with using smartwatches while driving. As smartwatches are gaining popularity among consumers, there is a need to understand how smartwatch use may influence driving performance. Previous driving research has examined voice calling on smartwatches, but not interactions with notifications, a key marketed feature. Engaging with notifications (e.g., reading and texting) on a handheld device is a known distraction associated with increased crash risks. Two driving simulator studies compared smartwatch to smartphone notifications. Experiment I asked participants to read aloud brief text notifications and Experiment II had participants manually select a response to arithmetic questions presented as notifications. Both experiments investigated the resulting glances to and physical interactions with the devices, as well as self-reported risk perception. Experiment II also investigated driving performance and self-reported knowledge/expectation about legislation surrounding the use of smart devices while driving. Experiment I found that participants were faster to visually engage with the notification on the smartwatch than the smartphone, took longer to finish reading aloud the notifications, and exhibited more glances longer than 1.6 s. Experiment II found that participants took longer to reply to notifications and had longer overall glance durations on the smartwatch than the smartphone, along with longer brake reaction times to lead vehicle braking events. Compared to the no device baseline, both devices increased lane position variability and resulted in higher self-reported perceived risk. Experiment II participants also considered that smartwatch use while driving deserves penalties equal to or less than smartphone use. The findings suggest that smartwatches may have road safety consequences. Given the common view among participants to associate smartwatch use with equal or less traffic penalties than smartphone use, there may be a disconnect between drivers' actual performance and their perceptions about smartwatch use while driving.
... It seems the NHTSA verification results are highly dependent on the distribution of 'short-glancers' and 'long-glancers' in the sample [1]. This finding is understandable given the large variance in the preferred occlusion times and distances the drivers are willing to tolerate (see Figures 9 and 10, see also [7] and [8]). ...
Conference Paper
This is the first controlled quantitative analysis on the visual distraction effects of audio-visual route guidance in simulated, but ecologically realistic driving scenarios with dynamic maneuvers and self-controlled speed (N = 24). The audio-visual route guidance system under testing passed the set verification criteria, which was based on drivers’ preferred occlusion distances on the test routes. There were no significant effects of an upcoming maneuver instruction location (up, down) on the in-car display on any metric or on the experienced workload. The drivers’ median occlusion distances correlated significantly with median in- car glance distances. There was no correlation between drivers’ median occlusion distance and intolerance of uncertainty but significant inverse correlations between occlusion distances and age as well as driving experience were found. The findings suggest that the visual distraction effects of audio-visual route guidance are low and provide general support for the proposed testing method.
Article
This article investigates the effects of the quantity and size of touch screen buttons and the task-interleaving strategies on drives’ eye glance behavior. An experiment was conducted on a fixed-base driving simulator with 20 participants. The participants were asked to perform a button search-and-press task on an in-vehicle touch screen while driving. A full-factorial within-subject design was used with three button quantities (4, 8, and 15) and three button sizes (14 mm, 24 mm, and 33 mm). Although a normal distribution was often assumed for the eye glance data in previous studies, our results show that the total eyes-off-road time (TEORT) and glance durations are generally not normally distributed (positively skewed) even after a log transformation. The results show that the number of buttons has an increasing effect on task completion time, TEORT, and long (2+ s) glances. However, in general, no such differences were found for button sizes. Further analysis shows that long glances were strongly associated with drivers completing the task with a single glance. It seems to suggest that a major cause of long glances is that drivers are reluctant to switch the task back to driving at subtask boundaries that are probably associated with the high cost of interruption. These findings confirm the importance of task resumability for in-vehicle user interfaces and have implications that careful task analysis needs to be conducted in the context of multitasking. Certain subtask combinations, such as a visual search followed by pressing the search target, may discourage task interleaving and ultimately compromise driving safety.
Article
Full-text available
What factors determine when people interleave tasks when multitasking? Here the authors look at the role of priorities and cognitive and motor cues. A study was conducted in which participants steered a simulated vehicle while also dialing two phone numbers that contained sets of repeating digits. Participants tended to interleave tasks after typing in a complete set of repeating digits and sometimes also at the cognitive chunk boundary. The exact pattern of how participants interleaved these tasks depended on their priority objective. A modeling analysis that explored performance for a series of alternative strategies for task interleaving, given the cognitive and task constraints, suggested why participants avoided interleaving at other points: Such strategies tend to move performance away from a trade-off curve that strikes an optimal balance between dialing and driving performance. The study highlights the role that cognitive and motor cues can play in dual-task performance and the importance of being aware, and acting on, priorities. Further implications and limitations are discussed.
Article
Full-text available
The main challenge for theories of multitasking is to predict when and how tasks interfere. Here, we focus on interference related to the problem state, a directly accessible intermediate representation of the current state of a task. On the basis of Salvucci and Taatgen's (2008) threaded cognition theory, we predict interference if 2 or more tasks require a problem state but not when only one task requires one. This prediction was tested in a series of 3 experiments. In Experiment 1, a subtraction task and a text entry task had to be carried out concurrently. Both tasks were presented in 2 versions: one that required maintaining a problem state and one that did not. A significant overadditive interaction effect was observed, showing that the interference between tasks was maximal when both tasks required a problem state. The other 2 experiments tested whether the interference was indeed due to a problem state bottleneck, instead of cognitive load (Experiment 2: an alternative subtraction and text entry experiment) or a phonological loop bottleneck (Experiment 3: a triple-task experiment that added phonological processing). Both experiments supported the problem state hypothesis. To account for the observed behavior, computational cognitive models were developed using threaded cognition within the context of the cognitive architecture ACT-R (Anderson, 2007). The models confirm that a problem state bottleneck can explain the observed interference.
Article
The ACT-R theory (Anderson, 1993; Anderson & Lebiere, 1998) is applied to the list memory paradigms of serial recall, recognition memory, free recall, and implicit memory. List memory performance in ACT-R is determined by the level of activation of declarative chunks which encode that items occur in the list. This level of activation is in turn determined by amount of rehearsal, delay, and associative fan from a list node. This theory accounts for accuracy and latency profiles in backward and forward serial recall, set size effects in the Sternberg paradigm, length–strength effects in recognition memory, the Tulving–Wiseman function, serial position, length and practice effects in free recall, and lexical priming in implicit memory paradigms. This wide variety of effects is predicted with minimal parameter variation. It is argued that the strength of the ACT-R theory is that it offers a completely specified processing architecture that serves to integrate many existing models in the literature.
Article
Goal-directed cognition is often discussed in terms of specialized memory structures like the “goal stack.” The goal-activation model presented here analyzes goal-directed cognition in terms of the general memory constructs of activation and associative priming. The model embodies three predictive constraints: (1) the interference level, which arises from residual memory for old goals; (1) the strengthening constraint, which makes predictions about time to encode a new goal; and (3) the priming constraint, which makes predictions about the role of cues in retrieving pending goals. These constraints are formulated algebraically and tested through simulation of latency and error data from the Tower of Hanoi, a means-ends puzzle that depends heavily on suspension and resumption of goals. Implications of the model for understanding intention superiority, postcompletion error, and effects of task interruption are discussed.
Conference Paper
User attention is a scarce resource, and users are susceptible to interruption overload. Systems do not reason about the effects of interrupting a user during a task sequence. In this study, we measure effects of interrupting a user at different moments within task execution in terms of task performance, emotional state, and social attribution. Task models were developed using event perception techniques, and the resulting models were used to identify interruption timings based on a user's predicted cognitive load. Our results show that different interruption moments have different impacts on user emotional state and positive social attribution, and suggest that a system could enable a user to maintain a high level of awareness while mitigating the disruptive effects of interruption. We discuss implications of these results for the design of an attention manager.
Natural Break Points The Influence of Priorities and Cognitive and Motor Cues on Dual-Task Interleaving
  • C P Janssen
  • D P Brumby
  • R Garnett
Janssen, C. P., Brumby, D. P., & Garnett, R. 2012. Natural Break Points The Influence of Priorities and Cognitive and Motor Cues on Dual-Task Interleaving. Journal of Cognitive Engineering and Decision Making, 6, 1: 5-29.