Content uploaded by Gerrit Bagschik
Author content
All content in this area was uploaded by Gerrit Bagschik on Jan 31, 2018
Content may be subject to copyright.
Platzhalter für Bild, Bild auf Titelfolie hinter das Logo einsetzen
René S. Hosse;Gerrit Bagschik; Markus Maurer; Klaus Bengler; Uwe Becker
April 23, 2017
Evolution Issues of Automated Driving Functions by
Application of Systemic Accident Analysis
On the Example of the Tesla Model S Fatality
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Disclaimer
This contribution refers to public available information about
accident #HWY16FH018 involving a Tesla Model S.
The investigation and models are developed according to
Autopilot Version 7.X.
The final report of the National Transportation Safety Board is
not taken into account.
The views and opinions expressed in this presentation are
those of the authors and do not necessarily reflect the
official policy or position of TU Braunschweig or TU Munich.
Examples of analysis performed within this presentation are
only examples. They should not be utilized in real-world
analytic products as they are based only on very limited and
dated public source information. Assumptions made within
the analysis are not reflective of the position of TU
Braunschweig or TU Munich.
23.04.2017 R.S. Hosse, G. Bagschik, M. Maurer, K. Bengler,
U. Becker | STAMP Workshop 2017 | MIT
Page 2
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Agenda
Automated Driving
and Safety –
Evolution Issues
Implications
Social Technical
Aspects of Automated
Driving
Application Example:
Tesla Model S
Fatality
Talk starts with
motivates
contributed
to deduces
make
safer
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Automated Driving and Safety –Evolution Issues
Increasing Automation throughout Domains
R.S. Hosse, G. Bagschik, M. Maurer, K. Bengler,
U. Becker | STAMP Workshop 2017 | MIT
Page 4
1960 20101980 1990 20001970 2020 Year
1950
Electric Electronic Control
Systems Computational and
Information Technology
Mechanic Control Systems
1930 1940
Cooperative
Networks
Road (L-Jetronic)
CAN C2C
ABS ESP
(Motronic)
e-Call
Totally Integrated Automation (TIA)
Programmable memory program
Control software
Manu-
facturing CNC machinery
Agricultural 1st Harvester
Automated filling
1st motorized
harvester
Overload detection syytems
Monitoring devices
Passenger information
systems
Railway
ETCS (L.2) ETCS (L.3)
Mobile
comm.
Electric interlocks
Standardised mechanical
interlocking. Electronically interlocking
Indusi LZB
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Automated Driving and Safety –Evolution Issues
„Operators act always as prescribed“
Common advices by automotive user guides:
„Drivers are required to remain engaged and aware when
piloting functions are engaged“
„Drivers must keep their hands on the wheel“
23.04.2017 R.S. Hosse, G. Bagschik, M. Maurer, K. Bengler,
U. Becker | STAMP Conference 2017 | MIT
Page 5
Source: youtube
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Agenda
Automated Driving
and Safety –
Evolution Issues
Implications
Social Technical
Aspects of Automated
Driving
Application Example:
Tesla Model S
Fatality
Talk starts with
motivates
contributed
to deduces
make
safer
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Socio-technical Aspects of Automated Driving
Role of human in automated vehicles
Name
Lateral &
long. control
Surveillance of
environment
Fallback
layer
Domain of
operation
Assisted
Driver &
System
Driver
Driver
Limited
Partial
automation
System
Driver
Driver
Limited
Conditional
automation
System
System
Fallback
ready user
Limited
High
automation
System
System
System
Limited
Full
automation
System
System
System
Unlimited
SAE, “J3016: Taxonomy and Definitions for Terms Related to Driving
Automation Systems for On-Road Motor Vehicles”, 2016
23.04.2017 R.S. Hosse, G. Bagschik, M. Maurer, K. Bengler,
U. Becker | STAMP Workshop 2017 | MIT
Page 9
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Socio-technical Aspects of Automated Driving
Role of human in automated vehicles
Name
Lateral &
long. control
Surveillance of
environment
Fallback
layer
Domain of
operation
Assisted
Driver &
System
Driver
Driver
Limited
Partial
automation
System
Driver
Driver
Limited
Conditional
automation
System
System
Fallback
ready user
Limited
High
automation
System
System
System
Limited
Full
automation
System
System
System
Unlimited
23.04.2017 R.S. Hosse, G. Bagschik, M. Maurer, K. Bengler,
U. Becker | STAMP Workshop 2017 | MIT
Page 10
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Socio-technical Aspects of Automated Driving
Role of human in the vehicle
Todays market systems provide level 2 automation
•Humans are designed as a permanent supervisor for the system
•Overruling is necessary
But: Studies from the early 80s show
• “that it is impossible for even a highly motivated human being to maintain effective visual
attention towards a source of information on which very little happens, for more than about
half an hour.”
Name
Lateral &
long. control
Surveillance of
environment
Fallback
layer
Domain of
operation
Partial
automation
System
Driver
Driver
Limited
L. Bainbridge, “Ironies of automation,” Automatica, vol. 19, no. 6,
pp. 775–779, 1983
23.04.2017 R.S. Hosse, G. Bagschik, M. Maurer, K. Bengler,
U. Becker | STAMP Workshop 2017 | MIT
Page 11
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Socio-technical Aspects of Automated Driving
Role of human in the vehicle
Name
Lateral &
long. control
Surveillance of
environment
Fallback
layer
Domain of
operation
Partial
automation
System
Driver
Driver
Limited
Warning: Traffic-Aware Cruise Control is designed for your driving comfort and
convenience and is not a collision warning or avoidance system.
It is your responsibility to stay alert, drive safely, and be in control of the vehicle
at all times.
Never depend on Traffic-Aware Cruise Control to adequately slow down Model
S. Always watch the road in front of you and be prepared to take corrective
action at all times.
Failure to do so can result in serious injury or death.
!
Tesla Model S Manual, p.68, 2016
23.04.2017 R.S. Hosse, G. Bagschik, M. Maurer, K. Bengler,
U. Becker | STAMP Workshop 2017 | MIT
Page 12
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Socio-technical Aspects of Automated Driving
Role of humans in the development process
•Automation of driving task is not a completely new topic
•First driver assistance systems came in 1995 (first ACC on Mitsubishi)
•Introduction of new systems must be planned and analyzed
•Project RESPONSE 3 gives a code of practice (2006)
•Guidelines on safe function definitions
•For example do not use „safe“ in the name of an assisting system
•Functional system boundaries like standing objects in early radar sensors
•Explicit communication of inadequacies
•Clear definition of responsibilities
•Supervision of responsibilities
•Create correct expectations of system performance
23.04.2017 R.S. Hosse, G. Bagschik, M. Maurer, K. Bengler,
U. Becker | STAMP Workshop 2017 | MIT
Page 13
A. Knapp, M. Neumann, M. Brockmann, R. Walz, and T. Winkle, Code of
Practice for the Design and Evaluation of ADAS. RESPONE 3
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Agenda
Automated Driving
and Safety –
Evolution Issues
Implications
Social Technical
Aspects of Automated
Driving
Application Example:
Tesla Model S
Fatality
Talk starts with
motivates
contributed
to deduces
make
safer
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Application Example: Tesla Model S Fatality
Accident Introduction
A. Singhvi and K. Russell, “Inside the Self-Driving Tesla Fatal
Accident,” The New York Times, 01-Jul-2016
23.04.2017 R.S. Hosse, G. Bagschik, M. Maurer, K. Bengler,
U. Becker | STAMP Workshop 2017 | MIT
Page 15
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Application Example: Tesla Model S Fatality
Step 0: Accidents and Hazards
No.
Accident
1
Vehicle crashes when Autopilot is active
No.
Hazards
1
Driver does not provide required attention to driving
tasks and environment
2
Autopilot does not react to other road crossing
vehicles/obstacles
23.04.2017 R.S. Hosse, G. Bagschik, M. Maurer, K. Bengler,
U. Becker | STAMP Workshop 2017 | MIT
Page 16
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Application Example: Tesla Model S Fatality
Step 0: Control Structures
Basic Model Concept:
23.04.2017 R.S. Hosse, G. Bagschik, M. Maurer, K. Bengler,
U. Becker | STAMP Workshop 2017 | MIT
Page 17
Driver
Vehicle
Vehicle in its Environment
Driver inputs VehicleInf.
VehicleActions Environment
Feedb.
Environment
Feedb.
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Application Example: Tesla Model S Fatality
Step 0: Control Structures
Basic Model Concept: Autopilot Control Structure
23.04.2017 R.S. Hosse, G. Bagschik, M. Maurer, K. Bengler,
U. Becker | STAMP Workshop 2017 | MIT
Page 18
Driver
Vehicle
Vehicle in its Environment
Driver inputs VehicleInf.
VehicleActions Environment
Feedb.
Environment
Feedb.
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Application Example: Tesla Model S Fatality
Step 0: Autopilot Control Structure
R.S. Hosse, G. Bagschik, M. Maurer, K. Bengler,
U. Becker | STAMP Workshop 2017 | MIT
Page 19
Driver
Autopilot HMI
Autopilot
Data Fusion and Assessment
INS Camera Radar
Steering
Wheel Brake Pedal Accelerator
Pedal
Steering
System Brake System Drivetrain
Trajectory Follow-Up
Controller
Vehicle Dynamics
Vehicle in its Environment
Driver
Control
Inputs
Vehicle
Dynamics
Controllers
steering angle engage target speed
mode,
objects,
target speed,
Signs
enables,
disables,
set speed
overtake request
enables,
disables,
set speed
provides trajectory assessed environment model
overrules/deactivates
provides delta/
target action
provides delta/
target action
provides delta/
target action
provides delta/
target action
controls controls controls
Autopilot Control Structure (System Operation)
Other tasks
performs
23.04.2017
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Application Example: Tesla Model S Fatality
Step 1: (Selected) Unsafe Control Actions by Autopilot Controls
Control action
Required but not
provided
Unsafe action
provided
Incorrect timing
Stopped too
soon/applied too
long
Overrule/
Deactivate
Driver inputs do
not overrule
Autopilot
Driver inputs
deactivate Autopilot
too late
Enable
Autopilot is enabled
unintended
Send mode
status
Autopilot does not
send mode status
Autopilot sends mode
status when not
enabled
Provide
assessed
environment
model
Environment
model not
provided
(not updated)
Environment model
provided when not
required
Environment model
provided too late
(Same) model
provided too long
R.S. Hosse, G. Bagschik, M. Maurer, K. Bengler,
U. Becker | STAMP Workshop 2017 | MIT
Page 20
23.04.2017
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Application Example: Tesla Model S Fatality
Step 0: Control Structures
Basic Model Concept: Driver Control Structure
23.04.2017 R.S. Hosse, G. Bagschik, M. Maurer, K. Bengler,
U. Becker | STAMP Workshop 2017 | MIT
Page 21
Driver
Vehicle
Vehicle in its Environment
Driver inputs VehicleInf.
VehicleActions Environment
Feedb.
Environment
Feedb.
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Application Example: Tesla Model S Fatality
Step 0: Driver Control Structure
R.S. Hosse, G. Bagschik, M. Maurer, K. Bengler,
U. Becker | STAMP Workshop 2017 | MIT
Page 22
23.04.2017
Tesla Motors
Driver Control Structure (System Development)
Tesla Model S
Driver
Digital
Manual Instruction
on Delivery Marketing
Instrument Cluster
Social Media
forms
attitude
Operator Process Model
contributes
customer
expectations
sources
disturbances by other
humans/activities, etc.
Autopilot mode
vehicles & road signs
action instructions
functions
instructions
limitations
drive (steer, accelerate, brake)
Autopilot on/off
status logs
driving data
update
enable functions
sent changelogs
driving information
changelog
Change log teaches instructs
authoring defines
coverage
sets goals
defines customer
defines image
Feedback
Control Action
surveys reviews
Driving Environment
Relevant Focus
Other Vehicles
Road Signs
Environmental Conditions
Other Tasks
Performs
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Application Example: Tesla Model S Fatality
Step 1: (Selected) Unsafe Control Actions by Driver Controls
R.S. Hosse, G. Bagschik, M. Maurer, K. Bengler,
U. Becker | STAMP Workshop 2017 | MIT
Page 23
Control action
Required but not
provided
Unsafe action
provided
Incorrect timing
Stopped too
soon/applied too
long
Steer
Driver
does not
steer Model S
when required
Driver steers Model
S too
late
Enable
Autopilot
Driver enables
Autopilot when not
allowed
Send
changelogs
Tesla
does not
send
changelogs
when
required
Authoring
Tesla does not
author the manual
when required
23.04.2017
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Application Example: Tesla Model S Fatality
Step 1: Violated Safey Constraints
R.S. Hosse, G. Bagschik, M. Maurer, K. Bengler,
U. Becker | STAMP Workshop 2017 | MIT
Page 24
23.04.2017
No
.
UCA
Safety
constraint
UCA 3
Autopilot does not send objects to Autopilot
HMI when required
Autopilot must send objects to Autopilot
HMI when required
UCA 4
Autopilot does not send road signs to Autopilot
HMI when required
Autopilot must send road signs to
Autopilot HMI when required
UCA 20
Data Fusion and Assessment does not provide
assessed environment model to Autopilot
when required
Data Fusion and Assessment must
provide assessed environment model to
Autopilot when required
UCA 34
Driver does not brake Model S when required
Driver must brake Model S when required
UCA 68
Driver performs other tasks when not allowed
Driver must not perform other tasks when
not allowed
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Application Example: Tesla Model S Fatality
Step 2: Autopilot Process Models and Contextual Factors
R.S. Hosse, G. Bagschik, M. Maurer, K. Bengler,
U. Becker | STAMP Workshop 2017 | MIT
Page 25
Autopilot HMI
Data Fusion and Assessment
Trajectory Follow-
Up Controller
Driver Control Inputs
mode
objects
target speed
signs
enables
disables
set speed
provides trajectory assessed environment model
overrules/deactivates
Autopilot
Autopilot Process Model
Control Actions:
•Provide Trajectory
•Status Mode
•Objects
•Signs
•Target Speed
Control Inputs:
Driver Control Inputs
•Overrule/Deactivate
Autopilot HMI
•Enable/Disable Autopilot
•Target Speed
Feedbacks:
Data Fusion and Assessment
•Assessed Environment Model
Driver
•Driver Operative Process
Awareness
INS Camera Radar
vehicle dynamics camera
Data radar
Data
Process Model
Control Actions:
•Assessed
Environment Model
Feedbacks:
•Vehicle Dynamics
•Visual Environment
•Radar Data
Accident Causation
Driver Operative
Process Awareness
Missing
Feedback
Missing Feedback
23.04.2017
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Application Example: Tesla Model S Fatality
Step 2: Driver Process Models and Contextual Factors
R.S. Hosse, G. Bagschik, M. Maurer, K. Bengler,
U. Becker | STAMP Workshop 2017 | MIT
Page 26
Tesla Model S
Driver
Digital
Manual
Instruction
on Delivery
Instrument Cluster
Social Media
forms
attitude
Operator Process Model
Available Control Actions:
•Steer
•Accelerate
•Brake
•Autopilot on/off
•Perform other Tasks
Available Feedbacks:
Instrument Cluster
•Autopilot Mode
•Vehicles displayed on instrument cluster
•Road signs displayed on instrument
cluster
•Action instructions
Changelog
•New functions
•New driving and handling instructions
•Driving Assistance limitations
Digital Manual
•Lessons learned by Digital Manual
Instruction on Delivery
•Lessons learned by Instruction on
Delivery
Social Media
•Attitude by Social Media
Disturbances
•Performing other activities
Driving Environment
•Other Vehicles
•Road Signs
•Environmental Conditions (weather, street
type, etc.)
disturbances by other
humans/activities, etc.
Autopilot mode
vehicles & road signs
action instructions
functions
instructions
limitations
drive (steer, accelerate, brake)
Autopilot on/off Change log
teaches
instructs
Driving Environment
Other Vehicles
Road Signs
Environmental Conditions
Other Tasks
performs
23.04.2017
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Agenda
Automated Driving
and Safety –
Evolution Issues
Implications
Social Technical
Aspects of Automated
Driving
Application Example:
Tesla Model S
Fatality
Talk starts with
motivates
contributed
to deduces
make
safer
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Application Example: Tesla Model S Fatality
Step 2: Autopilot Process Models and Contextual Factors
R.S. Hosse, G. Bagschik, M. Maurer, K. Bengler,
U. Becker | STAMP Workshop 2017 | MIT
Page 28
Autopilot HMI
Data Fusion and Assessment
Trajectory Follow-
Up Controller
Driver Control Inputs
mode
objects
target speed
signs
enables
disables
set speed
provides trajectory assessed environment model
overrules/deactivates
Autopilot
Autopilot Process Model
Control Actions:
•Provide Trajectory
•Status Mode
•Objects
•Signs
•Target Speed
Control Inputs:
Driver Control Inputs
•Overrule/Deactivate
Autopilot HMI
•Enable/Disable Autopilot
•Target Speed
Feedbacks:
Data Fusion and Assessment
•Assessed Environment Model
Driver
•Driver Operative Process
Awareness
INS Camera Radar
vehicle dynamics camera
Data radar
Data
Process Model
Control Actions:
•Assessed
Environment Model
Feedbacks:
•Vehicle Dynamics
•Visual Environment
•Radar Data
Accident Causation
Driver Operative
Process Awareness
Missing
Feedback
Missing Feedback
As far as the information available for this analysis
goes, the Autopilot itself did not work wrong.
Depending on the assessed environmental model and
sensory data, the calculated trajectory was correct.
The main process model flaw occurred in the Data
Fusion and Assessment control component. Here
the wrong hypothesis has been created,
misunderstanding the semi-trailer as a road sign. The
authors do not have any specific information how the
data assessment algorithms work and what lead to the
faulty hypothesis. Another aspect to mention is that
the sensors have not been capable of detecting
objects in the vehicles structure gauge.
23.04.2017
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Application Example: Tesla Model S Fatality
Step 2: Autopilot Process Models and Contextual Factors
R.S. Hosse, G. Bagschik, M. Maurer, K. Bengler,
U. Becker | STAMP Workshop 2017 | MIT
Page 29
Autopilot HMI
Data Fusion and Assessment
Trajectory Follow-
Up Controller
Driver Control Inputs
mode
objects
target speed
signs
enables
disables
set speed
provides trajectory assessed environment model
overrules/deactivates
Autopilot
Autopilot Process Model
Control Actions:
•Provide Trajectory
•Status Mode
•Objects
•Signs
•Target Speed
Control Inputs:
Driver Control Inputs
•Overrule/Deactivate
Autopilot HMI
•Enable/Disable Autopilot
•Target Speed
Feedbacks:
Data Fusion and Assessment
•Assessed Environment Model
Driver
•Driver Operative Process
Awareness
INS Camera Radar
vehicle dynamics camera
Data radar
Data
Process Model
Control Actions:
•Assessed
Environment Model
Feedbacks:
•Vehicle Dynamics
•Visual Environment
•Radar Data
Accident Causation
Driver Operative
Process Awareness
Missing
Feedback
Missing Feedback
As far as the information available for this analysis
goes, the Autopilot itself did not work wrong.
Depending on the assessed environmental model and
sensory data, the calculated trajectory was correct.
The main process model flaw occurred in the Data
Fusion and Assessment control component. Here
the wrong hypothesis has been created,
misunderstanding the semi-trailer as a road sign. The
authors do not have any specific information how the
data assessment algorithms work and what lead to the
faulty hypothesis. Another aspect to mention is that
the sensors have not been capable of detecting
objects in the vehicles structure gauge.
23.04.2017
Example: Vehicle Structure Gauge Principle
v
s
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Application Example: Tesla Model S Fatality
Step 2: Driver Process Models and Contextual Factors
R.S. Hosse, G. Bagschik, M. Maurer, K. Bengler,
U. Becker | STAMP Workshop 2017 | MIT
Page 30
Tesla Model S
Driver
Digital
Manual
Instruction
on Delivery
Instrument Cluster
Social Media
forms
attitude
Operator Process Model
Available Control Actions:
•Steer
•Accelerate
•Brake
•Autopilot on/off
•Perform other Tasks
Available Feedbacks:
Instrument Cluster
•Autopilot Mode
•Vehicles displayed on instrument cluster
•Road signs displayed on instrument
cluster
•Action instructions
Changelog
•New functions
•New driving and handling instructions
•Driving Assistance limitations
Digital Manual
•Lessons learned by Digital Manual
Instruction on Delivery
•Lessons learned by Instruction on
Delivery
Social Media
•Attitude by Social Media
Disturbances
•Performing other activities
Driving Environment
•Other Vehicles
•Road Signs
•Environmental Conditions (weather, street
type, etc.)
disturbances by other
humans/activities, etc.
Autopilot mode
vehicles & road signs
action instructions
functions
instructions
limitations
drive (steer, accelerate, brake)
Autopilot on/off Change log
teaches
instructs
Driving Environment
Other Vehicles
Road Signs
Environmental Conditions
Other Tasks
performs
The driver did not provide the required attention
to the driving process. If the driver would have
watched the road and processed the information
properly, the accident would not have happened.
Although the driver is informed by the manual that
he has to overview the Autopilot, different factors,
like Tesla’s image, social media, and other relatives
influenced the process model to create more trust in
the technology. Otherwise, the vehicle itself did not
do any supervision of the driver if he provides the
required attention. So there is a missing feedback in
the control structure.
23.04.2017
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Application Example: Tesla Model S Fatality
Step 2: Driver Process Models and Contextual Factors
R.S. Hosse, G. Bagschik, M. Maurer, K. Bengler,
U. Becker | STAMP Workshop 2017 | MIT
Page 31
Tesla Model S
Driver
Digital
Manual
Instruction
on Delivery
Instrument Cluster
Social Media
forms
attitude
Operator Process Model
Available Control Actions:
•Steer
•Accelerate
•Brake
•Autopilot on/off
•Perform other Tasks
Available Feedbacks:
Instrument Cluster
•Autopilot Mode
•Vehicles displayed on instrument cluster
•Road signs displayed on instrument
cluster
•Action instructions
Changelog
•New functions
•New driving and handling instructions
•Driving Assistance limitations
Digital Manual
•Lessons learned by Digital Manual
Instruction on Delivery
•Lessons learned by Instruction on
Delivery
Social Media
•Attitude by Social Media
Disturbances
•Performing other activities
Driving Environment
•Other Vehicles
•Road Signs
•Environmental Conditions (weather, street
type, etc.)
disturbances by other
humans/activities, etc.
Autopilot mode
vehicles & road signs
action instructions
functions
instructions
limitations
drive (steer, accelerate, brake)
Autopilot on/off Change log
teaches
instructs
Driving Environment
Other Vehicles
Road Signs
Environmental Conditions
Other Tasks
performs
The driver did not provide the required attention
to the driving process. If the driver would have
watched the road and processed the information
properly, the accident would not have happened.
Although the driver is informed by the manual that
he has to overview the Autopilot, different factors,
like Tesla’s image, social media, and other relatives
influenced the process model to create more trust in
the technology. Otherwise, the vehicle itself did not
do any supervision of the driver if he provides the
required attention. So there is a missing feedback in
the control structure.
23.04.2017
Example: Eye Detection and/or Hands on Wheel Detection
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Conclusions
The fatal crash of the Tesla Model S shows that development of safe automated vehicles
must take socio-technical aspects into account. STAMP and CAST, respective STPA for
forward analysis, can integrate the human in the roles of operator, traffic participant and
manufacturer of a system.
The proposed categorization of control actions to determine unsafe behavior do not explain
the main causes of the aforementioned fatal accident. For example, the assessed
environmental model is sent to the driving controller in right order and right time but
contained wrong information about the environment. To explain why the accident still
happened further explanation is needed.
R.S. Hosse, G. Bagschik, M. Maurer, K. Bengler,
U. Becker | STAMP Workshop 2017 | MIT
Page 32
23.04.2017
Institut für Verkehrssicherheit
und Automatisierungstechnik
Prof. Dr.-Ing. Dr. h.c. mult. E. Schnieder
Contact information
R.S. Hosse, G. Bagschik, M. Maurer, K. Bengler,
U. Becker | STAMP Workshop 2017 | MIT
Page 33
23.04.2017
René S. Hosse, M.Sc.
Email: r.hosse@tu-braunschweig.de
Gerrit Bagschik, M.Sc.
Email: bagschik@ifr.ing.tu-bs.de