Conference PaperPDF Available

Driver Distraction Detection and Evaluation with Artificial Neural Network and Fuzzy Logic In-vehicle information system as a driver's secondary activity: Case study

Authors:

Abstract and Figures

A robust methodology for detecting and evaluating driver distraction induced by in-vehicle information system using artificial neural network and fuzzy logic is introduced in this paper. An artificial neural network is used to predict driver's performance on a specific road segment. The predicted performance-based measures are compared to the driving with secondary task accomplishment. Fuzzy logic is applied to fuse the variables into a single output, which constitutes a level of driver distraction in percentage. The technique was tested on a vehicle simulator by ten drivers that exploited in-vehicle information system as a secondary activity. The driver-in-the-loop experiment outcomes are discussed.
No caption available
… 
Content may be subject to copyright.
Driver Distraction Detection and Evaluation with
Artificial Neural Network and Fuzzy Logic
In-vehicle information system as a driver’s secondary activity: Case study
Andrei Aksjonov, Pavel Nedoma
Concept Development
ŠKODA AUTO a.s.
Mladá Boleslav, Czech Republic
{ andrei.aksjonov; pavel.nedoma }@skoda-auto.cz
Valery Vodovozov, Eduard Petlenkov
Department of Electrical Power Engineering and
Mechatronics
Tallinn University of Technology
Tallinn, Estonia
{ valery.vodovozov; eduard.petlenkov }@ttu.ee
Abstract A robust methodology for detecting and evaluating
driver distraction induced by in-vehicle information system using
artificial neural network and fuzzy logic is introduced in this
paper. An artificial neural network is used to predict driver’s
performance on a specific road segment. The predicted
performance-based measures are compared to the driving with
secondary task accomplishment. Fuzzy logic is applied to fuse the
variables into a single output, which constitutes a level of driver
distraction in percentage. The technique was tested on a vehicle
simulator by ten drivers that exploited in-vehicle information
system as a secondary activity. The driver-in-the-loop experiment
outcomes are discussed.
Keywordsartificial neural networks; fuzzy logic; vehicle
safety; machine learning; prediction methods
I. INTRODUCTION
Driver distraction (DD) causes serious environmental
problem every year. Not to mention injuries, DD contributes to
more than 5000 traffic fatalities yearly in the USA alone.
Unfortunately, this trend does not tend to decline [1].
DD is defined as anything that delays the recognition of
information necessary to safety maintain the lateral and
longitudinal control of the vehicle (driver’s primary task) due
to some event, activity, object or person, within or outside the
vehicle that compels or tends to induce the driver’s shifting
attention away from the fundamental driving task by
compromising the driver’s auditory, biomechanical, cognitive
or visual faculties or combinations thereof”. The activities not
related to primary tasks driver perform while driving are
defined as secondary activities [2, 3, 4, 5].
There are two types of secondary tasks: interaction with in-
vehicle information system (IVIS) (e.g. controlling comfort
and entertainment), and interaction with the items brought to a
vehicle (e.g. portable electronic devices, passengers, pets) [2].
DD minimization caused by IVIS is under vehicle
manufacturers’ responsibility. In particular, the vehicle cockpit
and human-machine interface (HMI) must be safe for
operation, intuitive, well organized, and, what is most
important, not distract a driver from her/his primary task.
Development of a robust DD detection and assessment
methodology while interacting with IVIS allows testing
different HMI technologies and cockpit designs before they are
accepted in series vehicle. DD evaluation is also applied in
advanced driver assistance systems impact on driver’s situation
awareness. Today, there are still no accurate evaluation
technologies for DD induced by IVIS [2, 6, 7].
Previously, many different attributes were proposed for DD
detection. Among them, psychological [8], behavioral [9],
subjective [10], performance-based or their combinations [11]
are known. For DD detection, machine learning (e.g. support
vector machine, graph-regularized extreme learning machine,
k-nearest neighbor) deserved special attention among other
algorithms [12].
Artificial neural network (ANN) is one of the most popular
machine learning approaches [13] due to its robustness, ability
to learn by example, and efficiency in intelligent systems. The
ANNs are used in various disciplines: physics, statistics,
psychology, cognitive science, neuroscience, and linguistics,
not to mention computer science, electrical engineering, and
adaptive control [14]. Many researchers on DD detection and
assessment applied ANN to solve their problems.
In [15], a three-dimensional convolutional neural network
(CNN) and gradient boosting algorithms combination were
proposed for drowsiness classification. Gaze zone
categorization was designed using CNN in [16]. A probabilistic
restricted Coulomb energy ANN was implemented for drowsy
driving prediction in [17]. In these works, the researchers
preferred behavioral attributes. Unfortunately, behavioral and
psychological attributes always require supplementary devices
(e.g. cameras and neuro-scan systems) that increase system
cost and complexity [18].
Another famous approach of DD detection - performance-
based - does not require additional devices. These methods
depend on vehicle dynamic performance, which is tracked by
the sensors available in modern vehicles (e.g. vehicle velocity
and steering wheel angle). In [19], the scholars proposed DD
detection using in-vehicle signals without planned distraction.
The machine learning schemes, ANN, and Gaussian mixture
models were combined to solve the problem. In [20], a real-
time DD detection classifier using vehicle dynamic data was
introduced. Different machine learning algorithms, including
static and dynamic ANNs, adaptive neuro-fuzzy inference
system, and support vector machine, were compared. The last
one outperformed all the other classifiers.
For accurate DD detection, different attributes can be
combined. For example, behavioral (i.e. head movement) and
performance-based data were integrated for the online DD
detection with long- and short-term memory recurrent ANN in
[21]. In [22], an adaptive neuro-fuzzy inference system for DD
prediction was developed. This approach showed better
performance comparing to the ANN and radial basic function
prediction algorithms.
Another computational intelligence method, fuzzy logic
(FL), is also widely used in automotive engineering, from
vehicle dynamics control [23] to DD evaluation [24]. A
thorough review on fuzzy methods in automotive engineering
applications can be found in [25].
Although the results of the related works were very positive
in DD detection, they all use the 2-class classification:
distracted or not distracted. This classification is not suitable
for measuring the DD level and consequent IVIS HMI
technology comparative evaluation. Hence, in this paper a
solution of the regression problem is proposed for DD
detection with performance-based attributes bearing in mind
that ANNs are actively used for nonlinear regression [14].
An ANN is designed as a driver performance predictor in a
name of lane keeping ability and speed limit maintenance.
First, the ANN is trained with data collected during the DD-
free driving. Next, the predicted variables are compared to the
same variables collected during driver’s run with completing
secondary task in parallel to driving. The comparison is used
for DD detection. As FL is efficient in data fusion [26, 27], it is
applied to merge vehicle dynamics into a single variable. This
variable depicts a level of DD in percentage caused by a
secondary activity.
Next Section is dedicated to the methodology based on
ANN and FL combination description. The driver-in-the-loop
experiment was conducted on an advanced driving simulator
provided by ŠKODA AUTO (Mladá Boleslav, Czech
Republic). The experiment procedure, subjects, and apparatus
are described in Section III. The DD detection and evaluation
methodology outcomes are stressed in Section IV. Finally, the
research conclusion is provided in Section V.
II. DRIVER DISTRACTION DETECTION METHOD
The block scheme of the DD detection and evaluation
method is presented in Fig. 1. The symbols’ description and
annotation are introduced in Table I. The method involves
three steps. First, the ANN predicts a vehicle dynamic
performance on a specific road segment taking as the inputs an
information about the road segment: speed limit Vl and
curvature (radius) r. The outputs of the ANN describe
predicted driver performance on a specific road segment:
ability to hold a speed limit ∆vp and to stay in the middle of the
road lane xp. The training data for the ANN are collected
during the first phase of the experiment, when the driver
demonstrates her/his normal vehicle operation, without
secondary activity.
Second, the predicted variables are compared to a real
driving performance ∆x and ∆v with IVIS interaction. The
comparison outcomes are the resultative variables ∆xr and vr
calculated using the following rules:
p
ppp
ppp
ppp
ppp
r
xxif
xxxxifxx
xxxxifxx
xxxxifxx
xxxxifxx
x
,0
;0;0,
;0;0,
;0;0,
;0;0,
(1)
p
ppp
ppp
ppp
ppp
r
vvif
vvvvifvv
vvvvifvv
vvvvifvv
vvvvifvv
v
,0
;0;0,
;0;0,
;0;0,
;0;0,
(2)
Negative ∆x means that the vehicle drives closer to the road
dividing line. Contrariwise, positive x means that the vehicle
drives towards off road from the middle of the lane. Negative
v shows that the vehicle velocity is lower than the speed limit
on a road segment whereas positive ∆v signifies speeding.
Finally, FL completes data fusion and outputs a level of DD in
percentage DD.
The evaluation method is programmed in MATLAB®
(Natick, MA, USA) environment. The Neural Network
Toolbox™ was exploited for the design of the ANN driver
performance predictor. One of the most popular methods due to
its optimal computation complexity - backpropagation [13] -
Fig. 1. DD detection and evaluation method block scheme.
TABLE I. PARAMETERS DESCRIPTION
Symbol
Description
Unit
r
Road radius
m
Vl
Speed limit
km/h
x
Real lane keeping offset
m
v
Real vehicle speed deviation
km/h
xp
Predicted lane keeping offset
m
vp
Predicted vehicle speed deviation
km/h
xr
Resultative lane keeping offset
m
vr
Resultative vehicle speed deviation
km/h
DD
Driver distraction in percentage
%
was used for computing gradients in the ANN. The algorithm
detailed description for training a multilayer perceptron for
regression is reported in [14].
The Levenberg-Marquardt method is the most suitable for
training an ANN with nonlinear regression (function fitting)
purposes. In practice, ANNs incorporate three and sometimes
four layers, including one or two hidden layers with 10 to 1000
neurons in each. More hidden layers do not guarantee better
ANN performance likewise the greater number of neurons
does. On the contrary, each additional layer increases the
computational burden exponentially [14, 15].
Two hidden layers were used for ANN in this study. The
network performance was tested on different number of
neurons in each layer. However, we did not notice the ANN
significant enhancement with more than 100 neurons. Due to
its simplicity and superior performance [13], the tangent-
sigmoid activation function was applied in the hidden layers.
As a rule, for the outputs the linear transfer functions are used
in regression [13]. In short, a feedforward ANN with two
hidden layers of 100 neurons each, with a tangent-sigmoid
transfer function in the hidden layer, and a linear transfer
function in the output layer was designed using the Levenberg-
Marquardt training method.
In this research, an FL Sugeno’s type inference mechanism
based on simple matrix operations suitable for C, C++,
MATLAB® script, and other programming languages was used
[24]. The FL DD evaluator has a “2 inputs - 1 output”
structure. The inputs are xr and ∆vr. The DD output represents
a level of DD in percentage (Fig. 1).
Five symmetrically dispersed and overlapped over the
whole universe of discourse (UOD) triangular membership
functions (MFs) capable for fast response and easy for
programming were designed for both inputs. Inputs’ equal
sensitivity is guaranteed by the symmetric MFs dispersion. The
xr UOD was bounded between [-1.5, 1.5] whereas the vr
UOD was narrowed in [-12, 12].
The output UOD is bounded within [0, 100]. The output
MFs represent eight singletons with equal step 14.3 between
each other: {0, 14.3, 28.6, 42.9, 57.2, 71.5, 85.8, 100}. The
step is calculated by dividing the highest value of UOD
boundary by an amount of MFs starting from the second
singleton, because the first one is equal to zero. Consequently,
the fixed step between the output MFs guarantees their equal
sensitivity inside of the UOD.
Modus-ponens-form rules “If-Then” are applied to connect
the inputs with the output. The input-output linguistic relation
is presented in Table II. The control surface for the designed
FL evaluator is shown in Fig. 2. An example of the linguistic
input-output mapping is performed as follows:
IF the vehicle center is “far from the road dividing line
AND driver’s speed is ”low” comparing to the road speed
limit, THEN driver distraction is ”14.3%”.
III. METHODOLOGY
A. Subjects
Ten diver-in-the-loop experiment participants were
members of the Interdisciplinary Training Network in Multi-
Actuated Ground Vehicles (ITEAM) project. In this study, only
male drivers participated. Their age ranged between 27 and 31
years old. Every participant owned a valid driver license. All
drivers took part in the experiment voluntarily.
The experiment participation was not payed. However, for
their contribution the participants were awarded with a guided
tour to ŠKODA AUTO museum and vehicle production plant
(Mladá Boleslav, Czech Republic) free of charge.
B. Apparatus
The experiment was conducted on available in ŠKODA
AUTO HMI laboratory (Mladá Boleslav, Czech Republic)
facilities that consist of a fixed-base vehicle mockup and a wall
screen in front of the driver, where the virtual world is
projected (Fig. 3). A driver settled down inside the vehicle
simulator, which has an automatic gearbox, steering wheel,
adjustable driver’s seat, acceleration and brake pedals. The
TABLE II. RULE-BASE OF THE FL EVALUATOR
DD [%]
xr [m]
very_close
close
zero
far
very_far
∆vr
[km/h]
very_low
100
71.5
42.9
71.5
100
low
85.8
14.3
0
14.3
85.8
zero
42.9
0
0
0
42.9
high
71.5
14.3
0
14.3
71.5
very_high
85.8
57.2
28.6
57.2
85.5
Fig. 3. Driver-in-the-loop experiment facility.
Fig. 2. The DD FL evaluator nonlinear control surface.
vehicle cockpit is identical to the one used in commercial
vehicles. The simulator’s head-up instrument panel displays
current vehicle velocity. All the secondary tasks are conducted
via the HMI display placed, like in most of the European
countries, on the driver’s right.
Vehicle dynamics and virtual world are modelled using an
open source library for simulating rigid body dynamics Open
Dynamics Engine™ v 0.5 in C++ programming language [28].
The vehicle model consists of the vehicle body, suspension
system, four wheels, and the tire model completed with
Pacejka’s Magic Formula [29]. The vehicle is parametrized
according to Škoda Yeti SK316 with 1.4 liters twincharged
stratified injection 77 kW engine specifications. The drivers
could understand when they drive off the road on a grass by the
screen vibration. However, they could not feel crossing the
road dividing line.
Three coordinates (i.e. x, y, z) are saved in the virtual world
for each wheel with 0.1 seconds sample period. These data are
exploited for lane keeping offset and for vehicle speed
deviation calculations. The drivers control the vehicle via the
steering wheel and the brake and acceleration pedals.
C. Procedure
The driver-in-the-loop experiment participants were asked
to drive the two-way lap of a total length of 10626 meters with
a 3.5 meters width lane. The road has two main segments: the
curvy road with a speed limit of 50 km/h and the almost
straight road with slight curvature of 90 km/h. When all the
speed limits are respected, one lap takes approximately 10
minutes for driving. There were no other traffic participants
modelled, neither pedestrians nor other vehicles.
To familiarize with the simulator, the drivers were allowed
to drive it unlimited time, before the start of the driver-in-the-
loop experiment. Thus, they were aware of the test rig and the
road shape preliminary. What is more, the participants were
instructed to the studied HMI display. They could try all the
secondary tasks in advance.
The experiment for each participant was divided into two
parts. First, the drivers were asked to pass two laps showing
their best performance in lane keeping ability and following all
the traffic regulations (i.e. speed limits). They were not
fulfilling a secondary activity during the first part of the
experiment. The data collected during the free from the
secondary task driving were used for the ANN training. During
the second part, the drivers were asked to continue driving
obeying the traffic rules and staying in the middle of the lane as
good as feasible. However, this time they drove the same road
performing a secondary activity in parallel. The participants
(a)
(b)
Fig. 4. Driver performance prediction versus real driver performance example for one of the experiment participants: gray background the secondary task
accomplishment period; red line real driver performance; black line predicted performance; green line road information; (a) speed difference ∆v and
road speed limit Vl; (b) center lane keeping offset ∆x and road radius r.
TABLE III. IN-VEHICLE SECONDARY TASKS
#
In-vehicle secondary task
1.
Volume
Volume regulation
2.
Context selection
Radio
3.
Media
4.
Telephone
5.
Navigation
6.
Radio
Radio station selection from a primary list
7.
Radio station selection from an overall list
8.
Media
Media source selection (e.g. CD, SD-card)
9.
Media item selection
10.
Song shuffle
11.
Telephone
Call a number from a favorite contact list
12.
Call a number from an overall contact list
13.
Navigation
Input location
14.
Input of a next target
15.
Zoom operation
were allowed to take a break between the experiment parts.
The in-vehicle secondary tasks are introduced in Table III.
The experiment organizer sent the vocal command to the driver
for a secondary activity request. When the secondary task is
accomplished, the driver sent a feedback to the experimenter
via a switch around the steering wheel. If the task is correct, the
driver heard a vocal signal informing that the submitted task is
correct. If the completed task is wrong and the driver did not
hear the sound signal, she/he had to perform the secondary
activity again. The participant had a reasonable time between
each secondary task request. No time restriction for a
secondary task realization was applied.
IV. RESULTS
The driver-in-the-loop experiment outcomes are presented
in this section. A random driver performance prediction, DD
detection and evaluation with FL is studied here in details. For
the rest of the participants the results are very similar: low DD
during normal driving and high level of DD while performing a
secondary activity.
In Fig. 4, the driver performance prediction for ∆vp and ∆xp
versus real performance v and x with secondary activity is
introduced. During the experiment, the driver passed more than
two full laps. However, in this paper the results only from the
last lap are presented.
Within normal driving, the driver tended to keep the speed
slightly lower its road segment limit (Fig 4a; black curve). In
average, the speed deviation did not exceed 5 km/h. When the
secondary task is performed (Fig. 4a; red curve), the speed
limit maintenance ability has higher oscillation. For example,
in the period from 820 s to 900 s the driver slowed the speed by
more than 25 km/h, while during the DD-free run the
participant was able to keep ∆v almost at zero.
The lane keeping ability along with road radiuses are
introduced in Fig. 4b. In average, the participant was holding
the middle line with 0.2 m error. On the contrary, when the
driver was performing a secondary activity, his driving
performance was significantly burdened. The lane keeping
ability lower than -1 m or higher than 1 m means that the
vehicle was driving out of the road bounds. When the
participant performed secondary tasks, in most of the cases the
vehicle went outside the road limits.
As it is noticed in Fig. 4, the interaction with the IVIS
influences the vehicle dynamic performance. The impact
depends on a task complexity. The proposed DD detection
method easily recognizes the difference between normal DD-
free driving and driving while performing the secondary task.
Thereafter, the FL evaluates the level of DD.
In Fig. 5, DD evaluation is presented. It is seen that DD is
noticeably higher when the driver interacts with HMI by
executing the secondary activity. For some tasks (Table III),
the DD is low, for example, in case of the task number 1 -
volume regulation. On the contrary, some tasks caused very
high DD. For instance, with the task number 13 - searching for
a new location in a navigation system, the driver performed
badly for a quite long period (Fig. 5, inset). It can also be
observed from Fig. 4 for the period between 820 s and 900 s.
During this secondary task accomplishment, the driver dropped
the speed and went off the road several times (Fig. 4). The
method easily detects these mistakes in vehicle dynamic
performance and provides an appropriate evaluation (Fig. 5).
V. CONCLUSION
A DD detection and evaluation method based on ANN and
FL combination is presented in this paper. An ANN is applied
to solve a regression problem in driver performance prediction
on a specific road segment. The prediction is based on normal
driving without performing a secondary activity. Next, the
predicted performance-based variables are compared to the
same vehicle dynamics data collected during the driver run
with secondary activity accomplishment. The performance
Fig. 5. Driver distraction evaluation results: gray background the secondary task accomplishment period; blue curve DD. The secondary task number
refers to Table III.
degradation is accepted as a DD detection. Finally, the data
pass through FL evaluator, which outputs a level of DD.
The method is verified in driver-in-the-loop experiment on
an advanced driver decoy simulator with ten participants. The
results show that the methodology is always capable to detect
DD when an experiment participant interacts with IVIS. The
evaluation by FL allows an accurate comparison of different
in-vehicle secondary tasks. In particular, it shows what
secondary tasks lead to higher level of DD comparing to
another ones. The proposed methodology provides a practical
tool for HMI technologies comparative analysis.
In the future, more driver-in-the-loop participants will be
studied on IVIS-induced DD detection and evaluation with the
proposed methodology. Moreover, other machine learning
algorithms (e.g. k-nearest neighbor, adaptive neuro-fuzzy
inference system, and long short-term memory) efficient in
nonlinear regression [14] will be applied and compared to an
ANN. Different attributes combination and more performance-
based variables will be used for the method improvement.
ACKNOWLEDGMENT
This project has received funding from the European
Union’s Horizon 2020 research and innovation program under
grant agreement No. 675999. The authors extend their gratitude
to the volunteers from ITEAM (https://iteam-project.net)
project for participating in the driver-in-the-loop experiment.
REFERENCES
[1] AAA Foundation for Traffic Safety. 2016 Traffic Safety Culture Index.
Washington, DC: AAA Foundation for Traffic Safety, 2017. [Online]
Available:
https://www.aaafoundation.org/sites/default/files/2016TrafficSafetyCult
ureIndexReportandCover_0.pdf
[2] M. Westin, R. Dougherty, C. Depcik, A. Hausmann, and C. Sprouse III,
“Development of an adaptive-human-machine-interface to minimize
driver distraction and workload”, in Proc. in the ASME 2013 Int.
Mechan. Engin. Cong. and Expos., San Diego, California, USA, 2013,
pp. 1-13.
[3] J. L. Hansen, C. Busso, Y. Zheng, and A. Sathyanarayana, “Driver
modeling for detection and assessment of driver distraction: Examples
from the UTDrive test bed,” IEEE Sig. Proce. Mag., vol. 34, no. 4, pp.
130-142, Jan. 2017.
[4] S. Arun, K. Sundaraj, and M. Murugappan, “Driver inattention detection
method: A review,” in 2012 IEEE Cong. On Sust. Utiliz. And Develop.
in Engin. and Technol. (STUDENT), Kuala Lumpur, Malaysia, 2013,
pp. 1-6.
[5] Q. Wu, “An overview of driver distance measure methods,” in IEEE
10th Intern. Conf. on Comp.-Aided Indust. Design & Concept. Design,
2009 (CAID & CD 2009), Wenzhou, China, 2009, pp. 2391-2394.
[6] C. A. Pickering, K. J. Bumnhamt, and M. J. Richardson, “A review of
automotive human machine interface technologies and techniques to
reduce driver distraction,” in 2nd IET Int. Conf. on Syst. Safety 2007,
London, UK, 2007, pp. 223-228.
[7] A. Amditis, L. Andreone, K. Pagle, G. Markkula, E. Deregibus, M. R.
Rue, F. Bellotti, A. Engelsberg, R. Brouwer, B. Peters, and A. De
Gloria, “Towards the automotive HMI of the future overview of the
AIDE-integrated project results,” IEEE Trans. on Intel. Transp. Syst.,
vol. 11, no. 3, pp. 567-578, Sept. 2010.
[8] S. Wang, Y. Zhang, C. Wu, F. Darvas, and W. A. Chaovalitwongse,
“Online prediction of driver distraction based on brain activity patterns,”
IEEE Trans. on Intel. Transp. Syst., vol. 16, no. 1, pp. 136-150, Feb.
2015.
[9] C. Ahlstrom and K. Kirscher, “Review of real-time visual driver
distraction detection algorithms,” in Proc. of the 7th intern. Conf. on
Methods and Techniq. in Behavi. Research, Eindhoven, The
Netherlands, 2010, pp. 310-313.
[10] T. Horberry, J. Anderson, M. A. Regan, T. J. Triggs, and J. Brown,
Driver distraction: The effect of concurrent in-vehicle tasks, road
environment complexity and age on driving performance,” Acc. Anal.
and Prev., vol. 38, pp. 185-191, January 2006.
[11] M. Miyaji, H. Kawanaka, and K. Oguri, “Study on effect of adding pupil
diameter as recognition features for driver's cognitive distraction
detection,” in 2010 7th Intern. Symp. On Communi. Syst., Netw. And
Digit. Signal Proc. (CSNDSP 2010), Newcastle upon Tyne, UK, 2010,
pp. 1-6.
[12] Y. Jiao, Y. Peng, B.-L. Lu, X. Chen, S. Chen, and C. Wang,
“Recognizing slow eye movement for driver fatigue detection with
machine learning approach,” in 2014 Intern. Joint Conf. on Neural
Netw. (IJCNN), Beijing, China, 2014, pp. 4035-4041.
[13] M. Negnevitsky, Artificial Intelligence: A Guide to Intelligent System.
2nd ed. Harlow, England: Adison-Wesley, 2005, pp. 165-217.
[14] E. Alpaydin, Introduction to Machine Learning. Cambridge,
Massachusetts: USA: The MIT Press, 2004, pp. 229-373.
[15] X.-P. Huynh, S.-M. Park, and Y.-G. Kim, “Detection of driver
drowsiness using 3D deep neural network and semi-supervised gradient
boosting machines,” in Asian Conf. on Comp. Vision (ACCV) Lecture
Notes in Computer Science, vol. 10118. Springer, 2016, pp. 134-145.
[16] I.-H. Choi, S. K. Hong, and Y.-G. Kim, “Real-time categorization of
driver’s gaze zone using the deep learning techniques,” in 2016 Intern.
Conf. on Big Data and Smart Comp. (BigComp), Hong Kong, China,
2016, pp. 143-148.
[17] H. Matsuo and A. Khiat, “Prediction of drowsy driving by monitoring
driver’s behavior,” in 21st Inter. Conf. on Pattern Recogn. (ICPR 2012),
Tsukuba, Japan, 2012, pp. 3390-3393.
[18] C. Ahlstrom, K. Kircher, and A. Kircher, “A gaze-based driver
distraction warning system and its effect on visual behavior,” IEEE
Trans. on Intel. Transp. Syst., vol. 14, no. 2, pp. 965-973, June 2013.
[19] S. Im, C. Lee, S. Yang, J. Kim, and B. You, “Driver distraction detection
by in-vehicle signal processing,” in 2014 IEEE Symp. On Comput. Intel
in Vehic. and Transp. Syst. (CIVTS), Orlando, FL, USA, 2015, pp. 1-5.
[20] F. Tango and M. Botte, “Real-time detection system of driver distraction
using machine learning, “ IEEE Trans. on Intel. Transp. Syst., vol. 14,
no. 2, pp. 894-905, June 2013.
[21] M. Wollmer, C. Blaschke, T. Schmidl, B. Schuller, B. Farber, S. Mayer,
and B. Trefflich, „Online driver distraction detection using long short-
term memory.“ IEEE Trans. on Intel. Transp. Syst., vol. 12, no. 2, pp.
574-582, June 2011.
[22] F. Tango, C. Calefato, L. Minin, and L. Canovi, “Moving attention from
the road: A new methodology for the driver distraction evaluation using
machine learning approach, in 2009 2nd Conf. on Human. Syst.
Interact., Catania, Italy, 2009, pp. 596-599.
[23] A. Aksjonov, K. Augsburg, and V. Vodovozov, “Design and simulation
of the robust ABS and ESP fuzzy logic controller on the complex
braking maneuvers,” Appl. Sci., vol. 6, pp. 382-390, Nov. 2016.
[24] A. Aksjonov, P. Nedoma, V. Vodovozov, E. Petlenkov, and M.
Herrmann, “A method of driver distraction evaluation using fuzzy
logic,” in 2017 XXVI Intern. Conf. on Inform., Commun. and Autom.
Techn. (ICAT), Sarajevo, Bosnia & Herzegovina, pp 23-28.
[25] V. Ivanov, “A review of fuzzy methods in automotive engineering
applications”, Europ. Transp. Res. Rev., vol. 7 no. 3, pp. 1 - 10, Aug.
2015.
[26] C. Craye, A. Rashwan, M. S. Kamel, and F. Karray, “A multi-modal
driver fatigue and distraction assessment system,” Intern. Jour. of Intel.
Transp. Syst. Research, vol. 14, no. 3, pp. 173-794, Sep. 2016.
[27] K. M. Passino and S. Yurkovich, Fuzzy Control. California: USA:
Addison Wesley, 1998.
[28] R. Smith, Open Dynamics Engine v0.5: User Guide, 2006. [Online]
Available: http://ode.org/ode-latest-userguide.html
[29] H. B. Pacejka, Tyre and Vehicle Dynamics: Second Edition, Oxford:
Great Britain: Butterworth-Heinemann, 2006.
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
In addition to vehicle control, drivers often perform secondary tasks that impede driving. Reduction of driver distraction is an important challenge for the safety of intelligent transportation systems. In this paper, a methodology for the detection and evaluation of driver distraction while performing secondary tasks is described and an appropriate hardware and a software environment is offered and studied. The system includes a model of normal driving, a subsystem for measuring the errors from the secondary tasks, and a module for total distraction evaluation. A new machine learning algorithm defines driver performance in lane keeping and speed maintenance on a specific road segment. To recognize the errors, a method is proposed, which compares normal driving parameters with ones obtained while conducting a secondary task. To evaluate distraction, an effective fuzzy logic algorithm is used. To verify the proposed approach, a case study with driver-in-the-loop experiments was carried out, in which participants performed the secondary task, namely chatting on a cell phone. The results presented in this research confirm its capability to detect and to precisely measure a level of abnormal driver performance.
Article
Full-text available
Vehicle technologies have advanced significantly over the past 20 years, especially with respect to novel in-vehicle systems for route navigation, information access, infotainment, and connected vehicle advancements for vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) connectivity and communications. While there is great interest in migrating to fully automated, self-driving vehicles, factors such as technology performance, cost barriers, public safety, insurance issues, legal implications, and government regulations suggest it is more likely that the first step in the progression will be multifunctional vehicles. Today, embedded controllers as well as a variety of sensors and high-performance computing in present-day cars allow for a smooth transition from complete human control toward semisupervised or assisted control, then to fully automated vehicles. Next-generation vehicles will need to be more active in assessing driver awareness, vehicle capabilities, and traffic and environmental settings, plus how these factors come together to determine a collaborative safe and effective driver-vehicle engagement for vehicle operation. This article reviews a range of issues pertaining to driver modeling for the detection and assessment of distraction. Examples from the UTDrive project are used whenever possible, along with a comparison to existing research programs. The areas addressed include 1) understanding driver behavior and distraction, 2) maneuver recognition and distraction analysis, 3) glance behavior and visual tracking, and 4) mobile platform advancements for in-vehicle data collection and human-machine interface. This article highlights challenges in achieving effective modeling, detection, and assessment of driver distraction using both UTDrive instrumented vehicle data and naturalistic driving data
Article
Full-text available
Automotive driving safety systems such as an anti-lock braking system (ABS) and an electronic stability program (ESP) assist drivers in controlling the vehicle to avoid road accidents. In this paper, ABS and the ESP, based on the fuzzy logic theory, are integrated for vehicle stability control in complex braking maneuvers. The proposed control algorithm is implemented for a sport utility vehicle (SUV) and investigated for braking on different surfaces. The results obtained for the vehicle software simulator confirm the robustness of the developed control strategy for a variety of road profiles and surfaces.
Article
Full-text available
Purpose The paper addresses evolution of fuzzy systems for core applications of automotive engineering. Methods The presented study is based on the analysis of bibliography dedicated to fuzzy sets and fuzzy control for ground vehicles. A special attention is given to fuzzy approaches used in the following domains of automotive engineering: vehicle dynamics control systems, driver and driving environment identification, ride comfort control, and energy management of electric vehicles. Results The bibliographical analysis, supplemented with statistics of relevant research publications, has allowed to allocate the most important fuzzy application cases for each domain. In particular, it concerns driver identification, human-machine interface, recognition of road conditions, and controllers of vehicle chassis and powertrain systems. It is found out that fuzzy methods have the primary use most of all for tasks requiring identification and forecasting procedures, especially in conditions of limited informational space. Additional observation that can be also derived from the presented survey points to reasonable integration of fuzzy technique with other control engineering methods to improve the performance of automotive control systems. Conclusions In the aggregate the performed review indicates that fuzzy computing can be considered as a versatile tool for automotive engineering applications of different nature.
Article
Full-text available
In this paper, we present a multi-modal approach for driver fatigue and distraction detection. Based on a driving simulator platform equipped with several sensors, we have designed a framework to acquire sensor data, process and extract features related to fatigue and distraction. Ultimately the features from the different sources are fused to infer the driver’s state of inattention. In our work, we extract audio, color video, depth map, heart rate, and steering wheel and pedals positions. We then process the signals according to three modules, namely the vision module, audio module, and other signals module. The modules are independent from each other and can be enabled or disabled at any time. Each module extracts relevant features and, based on hidden Markov models, produces its own estimation of driver fatigue and distraction. Lastly, fusion is done using the output of each module, contextual information, and a Bayesian network. A dedicated Bayesian network was designed for both fatigue and distraction. The complementary information extracted from all the mod- ules allows a reliable estimation of driver inattention. Our experimental results show that we are able to detect fatigue with 98.4 % accuracy and distraction with 90.5 %.
Conference Paper
Full-text available
Slow eye movement (SEM) regarded as a sign of onset of sleep is very significant for detecting driver fatigue, but its characteristics and detection algorithm have been rarely involved in the study of driver fatigue detection. In this study, some new features were extracted based on wavelet singularity analysis and statistics to detect SEMs. Six subjects participated in this simulated driving experiment, and for each subject, a more than 2 hours electro-oculogram (EOG) session was recorded. Each session was divided into SEM epochs and non-SEM epochs according to the common judgments made by the two of three experts by the visual recognition criteria of SEMs. Regarding the problem of detecting SEMs as an imbalance classification problem, and through the under-sampling and over-sampling methods a 2s horizontal electro-oculogram (HEO) signal could finally be recognized as the category of SEMs or non-SEMs with the classifiers SVM, GELM, and KNN respectively. Results prove that the proposed features was a little better than the wavelet energy features, and through the combination of the wavelet energy features and the new features based on wavelet singularity analysis and statistics, the classification results were improved obviously.
Article
Full-text available
This paper presents a new computational framework for early detection of driver distractions (map viewing) using brain activity measured by electroencephalographic (EEG) signals. Compared with most studies in the literature, which are mainly focused on the classification of distracted and nondistracted periods, this study proposes a new framework to prospectively predict the start and end of a distraction period, defined by map viewing. The proposed prediction algorithm was tested on a data set of continuous EEG signals recorded from 24 subjects. During the EEG recordings, the subjects were asked to drive from an initial position to a destination using a city map in a simulated driving environment. The overall accuracy values for the prediction of the start and the end of map viewing were 81% and 70%, respectively. The experimental results demonstrated that the proposed algorithm can predict the start and end of map viewing with relatively high accuracy and can be generalized to individual subjects. The outcome of this study has a high potential to improve the design of future intelligent navigation systems. Prediction of the start of map viewing can be used to provide route information based on a driver's needs and consequently avoid map-viewing activities. Prediction of the end of map viewing can be used to provide warnings for potential long map-viewing durations. Further development of the proposed framework and its applications in driver-distraction predictions are also discussed.
Conference Paper
Detecting drowsiness of the driver with a reliable and confident manner is a challenging task since it requires accurate monitoring of facial behavior such as eye-closure, nodding and yawning. It is even harder to deal with it when she wears sunglasses or scarf, appearing in the data set given for this challenge. One of the popular ways to analyze facial behavior has been using standard face models such as active shape model or active appearance model. These models work well for the frontal faces and yet often stumble for the extreme head pose cases. To handle these issues, we propose an approach based on recent machine learning techniques: first, 3D convolutional neural network to extract features in spatial-temporal domain; secondly, gradient boosting for drowsiness classification; thirdly, semi-supervised learning to enhance overall performance. The highest score from our submissions was 87.46% accuracy, suggesting that this approach has a potential for real application.
Conference Paper
In this paper, we present a system that monitors the increasing frequency of driver's subsidiary behavior as a revealing indicator of driver's decrease in arousal level that would cause drowsy driving. The considered involuntary subsidiary behavior events are yawning, self-touching hand motion and head twisting movement, which increased as a signal of struggle against sleepiness in a driving simulator experiment. We show that by combining this index with conventional indexes of eye closure rate and head sway, we obtained better results than conventional drowsiness detection systems. The results obtained in proving ground driving conditions were satisfactory and the timing of the generated feedback was acceptable to most of the volunteers who tested our prototype.