Content uploaded by Daniel Brod
Author content
All content in this area was uploaded by Daniel Brod on Nov 08, 2022
Content may be subject to copyright.
Photo – JJ11 / Shutterstock.com (non-commercial use)
U.S. Department of
Transportation
Federal Railroad
Administration
A New Model for Highway-Rail Grade Crossing
Accident Prediction and Severity
Office of Research,
Development,
and Technology
Washington, DC 20590
DOT/FRA/ORD-20/40
Final Report
October 2020
NOTICE
This document is disseminated under the sponsorship of the
Department of Transportation in the interest of information exchange.
The United States Government assumes no liability for its contents or
use thereof. Any opinions, findings and conclusions, or
recommendations expressed in this material do not necessarily reflect
the views or policies of the United States Government, nor does
mention of trade names, commercial products, or organizations imply
endorsement by the United States Government. The United States
Government assumes no liability for the content or use of the material
contained in this document.
NOTICE
The United States Government does not endorse products or
manufacturers. Trade or manufacturers’ names appear herein solely
because they are considered essential to the objective of this report.
i
REPORT DOCUMENTATION PAGE
Form Approved
OMB No. 0704-0188
Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources,
gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this
collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson
Davis Highway, Suite 1204, Arlington, VA 22202-4302, and to the Office of Management and Budget, Paperwork Reduction Project (0704-0188), Washington, DC 20503.
1. AGENCY USE ONLY (Leave blank)
2. REPORT DATE
October 2020
3. REPORT TYPE AND DATES COVERED
Final Report 10/1/2019 – 7/31/2020
4. TITLE AND SUBTITLE
New Model for Highway-Rail Grade Crossing Accident Prediction and Severity
5. FUNDING NUMBERS
6. AUTHOR(S)
Daniel Brod 0000-0002-1213-0908 -, David Gillen 0000-0001-7047-2669
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)
Decisiontek, LLC
11821 Parklawn Drive, Ste. 260
Rockville, MD 20852
8. PERFORMING ORGANIZATION
REPORT NUMBER
9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)
U.S. Department of Transportation
Federal Railroad Administration
Office of Railroad Policy and Development
Office of Research, Development, and Technology
Washington, DC 20590
10. SPONSORING/MONITORING
AGENCY REPORT NUMBER
DOT/FRA/ORD-20/40
11. SUPPLEMENTARY NOTES
Project Manager: Karen McClure
12a. DISTRIBUTION/AVAILABILITY STATEMENT
This document is available to the public through the FRA website.
12b. DISTRIBUTION CODE
13. ABSTRACT (Maximum 200 words)
The U.S. Department of Transportation Accident Prediction and Severity (APS) model has been used by Federal, State, and local
authorities to assess accident risk at highway-rail grade crossings since the late 1980s. The Federal Railroad Administration
funded research for the development of a new model that employs current consensus analysis methods and recent data trends. The
new model also seeks to address several limitations of the current APS model and to provide a more robust tool for analysts.
This report presents the stages of the new model development, the statistical estimation of the new model, and validations
comparing the performance of the new model with the APS. The research shows that the new model described here out-performs
the APS by multiple measures.
The new model will support grade crossing management by enabling: more accurate risk ranking of grade crossings, more rational
allocation of resources for public safety improvements at grade crossings, and the ability to assess the statistical significance of
variances in the measured risk at grade crossings.
14. SUBJECT TERMS
Highway-rail grade crossing; risk assessment; accident rates; accident severity; accident types;
high-speed rail; highway safety; highway travel; highways; improvements; highway vehicles;
railroad crashes; railroad safety; railroad trains
15. NUMBER OF PAGES
62
16. PRICE CODE
17. SECURITY CLASSIFICATION
OF REPORT
Unclassified
18. SECURITY CLASSIFICATION
OF THIS PAGE
Unclassified
19. SECURITY CLASSIFICATION
OF ABSTRACT
Unclassified
20. LIMITATION OF ABSTRACT
NSN 7540-01-280-5500 Standard Form 298 (Rev. 2-89)
Prescribed by ANSI Std. 239-18
298-102
iii
METRIC/ENGLISH CONVERSION FACTORS
ENGLISH TO METRIC
METRIC TO ENGLISH
LENGTH (APPROXIMATE)
LENGTH (APPROXIMATE)
1 inch (in)
=
2.5 centimeters (cm)
1 millimeter (mm)
=
0.04 inch (in)
1 foot (ft)
=
30 centimeters (cm)
1 centimeter (cm)
=
0.4 inch (in)
1 yard (yd)
=
0.9 meter (m)
1 meter (m)
=
3.3 feet (ft)
1 mile (mi)
=
1.6 kilometers (km)
1 meter (m)
=
1.1 yards (yd)
1 kilometer (km)
=
0.6 mile (mi)
AREA (APPROXIMATE)
AREA (APPROXIMATE)
1 square inch (sq in, in2)
=
6.5 square centimeters (cm2)
1 square centimeter (cm2)
=
0.16 square inch (sq in, in2)
1 square foot (sq ft, ft2)
=
0.09 square meter (m2)
1 square meter (m2)
=
1.2 square yards (sq yd, yd2)
1 square yard (sq yd, yd2)
=
0.8 square meter (m2)
1 square kilometer (km2)
=
0.4 square mile (sq mi, mi2)
1 square mile (sq mi, mi2)
=
2.6 square kilometers (km2)
10,000 square meters (m2)
=
1 hectare (ha) = 2.5 acres
1 acre = 0.4 hectare (he)
=
4,000 square meters (m2)
MASS - WEIGHT (APPROXIMATE)
MASS - WEIGHT (APPROXIMATE)
1 ounce (oz)
=
28 grams (gm)
1 gram (gm)
=
0.036 ounce (oz)
1 pound (lb)
=
0.45 kilogram (kg)
1 kilogram (kg)
=
2.2 pounds (lb)
1 short ton = 2,000 pounds
(lb)
=
0.9 tonne (t)
1 tonne (t)
=
=
1,000 kilograms (kg)
1.1 short tons
VOLUME (APPROXIMATE)
VOLUME (APPROXIMATE)
1 teaspoon (tsp)
=
5 milliliters (ml)
1 milliliter (ml)
=
0.03 fluid ounce (fl oz)
1 tablespoon (tbsp)
=
15 milliliters (ml)
1 liter (l)
=
2.1 pints (pt)
1 fluid ounce (fl oz)
=
30 milliliters (ml)
1 liter (l)
=
1.06 quarts (qt)
1 cup (c)
=
0.24 liter (l)
1 liter (l)
=
0.26 gallon (gal)
1 pint (pt)
=
0.47 liter (l)
1 quart (qt)
=
0.96 liter (l)
1 gallon (gal)
=
3.8 liters (l)
1 cubic foot (cu ft, ft3)
=
0.03 cubic meter (m3)
1 cubic meter (m3)
=
36 cubic feet (cu ft, ft3)
1 cubic yard (cu yd, yd3)
=
0.76 cubic meter (m3)
1 cubic meter (m3)
=
1.3 cubic yards (cu yd, yd3)
TEMPERATURE (EXACT)
TEMPERATURE (EXACT)
[(x-32)(5/9)] F
=
y C
[(9/5) y + 32] C
=
x F
QUICK INCH - CENTIMETER LENGTH CONVERSION
1
02 3 4 5
Inches
Centimeters 0 1 3 452 6 1110987 1312
QUICK FAHRENHEIT - CELSIUS TEMPERATURE CONVERSION
-40° -22° -4° 14° 32° 50° 68° 86° 104° 122° 140° 158° 176° 194° 212°
°F
°C -40° -30° -20° -10° 0° 10° 20° 30° 40° 50° 60° 70° 80° 90° 100°
For more exact and or other conversion factors, see NIST Miscellaneous Publication 286, Units of Weights and
Measures. Price $2.50 SD Catalog No. C13 10286 Updated 6/17/98
iv
Acknowledgements
The authors appreciate the insightful review comments by Federal Railroad Administration
(FRA) staff Karen McClure and Francesco Bedini. We also acknowledge the ongoing support of
Sam Al-Ibrahim of FRA’s Office of Train Control and Communication.
v
Contents
Executive Summary ............................................................................................................................... 1
1. Introduction........................................................................................................................ 3
1.1 Background ........................................................................................................................ 3
1.2 Objectives .......................................................................................................................... 7
1.3 Overall Approach .............................................................................................................. 7
1.4 Scope ................................................................................................................................. 9
1.5 Organization of the Report ................................................................................................ 9
2. Preliminary Data Review ................................................................................................. 11
2.1 Risk by Warning Device Types ....................................................................................... 11
3. Data Selection and Analysis ............................................................................................ 18
3.1 Data Sources .................................................................................................................... 18
3.2 Data Selection .................................................................................................................. 18
3.3 Candidate Variables ......................................................................................................... 20
4. Accident Prediction Model .............................................................................................. 26
4.1 The Accident Prediction Model ....................................................................................... 26
4.2 ZINB Regression ............................................................................................................. 27
4.3 Predicting Accidents from the Regression Outputs ......................................................... 30
4.4 Empirical Bayes Prediction Adjustment .......................................................................... 30
4.5 Cumulative Residual (CURE) Analysis .......................................................................... 32
5. Accident Severity Model ................................................................................................. 35
5.1 Description of the Data .................................................................................................... 35
5.2 The Accident Severity Model .......................................................................................... 35
5.3 Model Specification and Regression Results .................................................................. 37
5.4 Accident Severity Forecast Equations ............................................................................. 39
6. Validation ........................................................................................................................ 41
6.1 Accident Prediction – Cumulative Risk .......................................................................... 41
6.2 Accident Prediction – Risk at Crossings by Accident Count Groups ............................. 44
6.3 Accident Prediction – Accident Risk for Groups of High-Risk Crossings...................... 46
6.4 Accident Severity – Model Comparisons ........................................................................ 46
7. Conclusion ....................................................................................................................... 51
8. References........................................................................................................................ 53
Appendix A. Interpreting Regression Outputs ..................................................................................... 54
Appendix B. Application of the New Model ....................................................................................... 56
vi
Figures
Figure 2-1. Accidents by Warning Device Type ................................................................................. 13
Figure 2-2. Risk per Exposure (Accident Rate) by Warning Device Type ......................................... 15
Figure 2-3. Relative Risk Levels by Warning Device ......................................................................... 17
Figure 3-1. Boxplot of Normalized Crossing Accidents by Grade Crossing Surface Type ................ 22
Figure 3-2. Boxplot of Normalized Crossing Accidents by Grade Crossing Angle ............................ 23
Figure 3-3. Boxplot of Normalized Crossing Accidents by Maximum Timetable Speed Deciles ...... 24
Figure 3-4. Boxplot of Normalized Crossing Accidents by Percent Truck Deciles ............................ 24
Figure 4-1. ZINB Predicted Accidents by Warning Device Type ....................................................... 30
Figure 4-2. ZINB+EB Predicted Accidents by Warning Device Type ................................................ 32
Figure 4-3 Cumulative Residual Analysis for Exposure ..................................................................... 33
Figure 5-1. Severity Predictions for 50 Crossings with the New Model ............................................. 39
Figure 6-1. Model Comparison (2014–2018, all crossings in sample) ................................................ 42
Figure 6-2. Model Comparison (2015–2019, all crossings in sample) ................................................ 42
Figure 6-3. Model Comparison (2014–2018, crossings in sample with non-zero accidents) .............. 43
Figure 6-4. Model Comparison (2015–2019, crossings in sample with non-zero accidents) .............. 43
Figure 6-5 Model Comparison, Accident Counts, and Predictions (2014–2018) ................................ 45
Figure 6-6. Model Comparison, Accident Counts, and Predictions (2015–2019) ............................... 45
Figure 6-7. Comparison of Predictions for Riskiest Crossings ............................................................ 46
Figure 6-8 Distribution of Predicted Accident Severities with the New Model .................................. 48
Figure 6-9. Distribution of Predicted Accident Severities with APS .................................................. 48
vii
Tables
Table 1-1. Estimates Required for Different Types of Analysis Focus ................................................. 8
Table 2-1. Warning Device Type Codes and Descriptions .................................................................. 12
Table 3-1. Summary of Data Selection ................................................................................................ 20
Table 3-2 Candidate Variables for Inclusion in the New Model ......................................................... 20
Table 3-3. Final Data Selection ........................................................................................................... 25
Table 4-1. ZINB Regression Output .................................................................................................... 28
Table 4-2. Revised ZINB Regression Output ...................................................................................... 34
Table 5-1. Accident Severity Ordered Logistic Regression Output .................................................... 38
Table 6-1. Predicted Severity (Percent of Total) by the New Model and APS ................................... 47
Table 6-2. Summary of Severity Category Skewness by Model ......................................................... 49
1
Executive Summary
This report presents research on a new model as an alternative to the U.S. Department of
Transportation grade crossing Accident Prediction and Severity (APS) model, which dates back
to 1986. This report follows the steps in developing the new model, presents the modeling
results, and validates the new model in comparison to the APS.
In the nomenclature of AASHTO’s Highway Safety Manual, the new model is a safety
performance function (SPF). SPFs generate metrics (e.g., predicted accidents by severity type)
indicating safety (or risk, insofar as more safety means less risk) and have been applied to a
range of highway facilities. The SPF approach is applicable to grade crossings, individually and
to aggregated collections (i.e., populations)
1
, as well.
The new model derives from a policy perspective on grade crossing safety, a review of the data,
statistical analysis, and validation. The authors conclude that the new model outperformed the
APS, and its adoption would result in more accurate risk ranking of grade crossings, more
rational allocation of resources for public safety improvements at grade crossings, and the ability
to assess the statistical significance of variances in the measured risk at grade crossings.
Key Conclusions
The preliminary data review indicates a new model could replace the APS based on the key
drivers of exposure and grade crossing warning device type (i.e., the data show that risk
increases with exposure, and decreases with a more protective warning device type).
There is justification for a single model with warning device type category as a variable rather
than separate models for each of the three warning device type categories.
In the U.S. there are 105,377 grade crossings that are public, not closed, not grade separated, and
that have non-missing, non-erroneous values for exposure and warning device type. From 2014–
2018, there were 8,467 accidents at these grade crossings.
An aggregate analysis of these grade crossings shows that relative to a passive crossing, an
average lights crossing had 73 percent less risk per exposure than a passive crossing. An average
gated crossing had 63 percent less risk per exposure than a lights crossing.
The findings of the above analysis indicate a functional form with exposure, warning device
type, and other grade crossing characteristics.
Model estimation using the zero-inflated negative binomial (ZINB) regression yielded
parameters of the expected sign and magnitude, and had strong statistical significance.
The empirical Bayes (EB) method accounted for accident history while correcting for
“regression to the mean” bias. Adjusted results with EB produced predictions that more closely
track the actual counts than did the APS with its (non-EB) adjustment process for accident
history.
1
The “population” of grade crossings refers to all public grade crossings in the U.S. that are not closed or grade
separated. The analysis sample is a large subset (over 100,000) of all grade crossings.
2
The new model severity component determines the probabilities that an accident will be of one
of three severity types: fatal, injury, or property damage only. The severity component of the
new model was derived using multinomial logistic regression (MNL) on the accidents in a 6-year
period, 2014–2019. In this period there were 11,131 accidents at public crossings. Of these, there
were 9,870 at grade crossings with non-missing, non-erroneous data.
These 9,870 accidents were included in the severity model estimation. The MNL regression
shows that the best results were obtained with explanatory variables: rural or urban, maximum
time table speed, number of daily trains, and whether a crossing had a lights warning device.
Validations indicate the new model outperformed the APS. One of the validations looks at
cumulative risk at crossings, with crossings ordered from greatest to least risk (i.e., accident
count). The riskiest crossings in the data sample include 7,822 accidents at 6,409 crossings in
2014-2018. Applying each model (new and APS) to the data, the new model predicted 4,853.3
accidents (62.0 percent of the actual count) whereas the APS predicted 2360.2 accident (30.2
percent of the actual count).
3
1. Introduction
1.1 Background
1.1.1 About the APS Model
The U.S. Department of Transportation (DOT) Accident Prediction and Severity (APS)
2
model
has been used to assess accident risk at highway-rail grade crossings by all levels of government
since the late 1980s. The assessments of accident risk at grade crossings are foundational
information that guide the management of grade crossings, the identification of high-risk
crossings (“hotspots”), and the allocation of resources for improving grade crossing safety.
The APS model was developed in 1986 based on grade crossing and accident data from the
preceding 20 years.
Additional modeling efforts intended to support and supplement the APS were conducted more
recently by the Volpe National Transportation System Center (Volpe) and FRA. Volpe
developed a High-Speed Rail (HSR) Accident Severity Model in 2000
3
to predict accidents and
their severity by types of traffic on the highway and railroad. In 2005 the FRA published the
final Train Horn Rule (49 CFR 222), which specified “supplementary safety measures” and their
impacts on risk reduction. Such measures include: four-quad gates, median barriers, mountable
curbs, and new technologies like photo enforcement.
Among its enhancements for assessing grade crossing risk, FRA’s GradeDec.Net online tool
gives users access to the HSR Accident Severity Model, and complements the APS model with
the supplementary safety measure impacts from the Train Horn Rule.
While these improvements are notable, a new replacement model for the APS is still required to
ensure that U.S. DOT, State Departments of Transportation, and local governments efficiently utilize
resources for reducing risk at grade crossings.
1.1.2 Grade Crossing Accident Trends and the APS
Grade crossing accidents declined sharply in the 25 years following APS development (from
about 3,000 per year to about 2,000 per year). This reduction was due to a number of factors,
indicating the relationship between grade crossing characteristics and accidents has likely
shifted.
FRA periodically updates the APS normalizing constants
4
so that the national aggregate number
of predicted accidents equals the actual number of accidents in the most recently ended calendar
year. While the normalizing constants are applied uniformly within each warning device type
2
Farr (1987) describes the APS.
3
See Mironer, et al. (2000) at https://rosap.ntl.bts.gov/view/dot/8433.
4
See Farr (1987), 3-7.
4
group, they do not account for the many factors influencing accident risk that have changed in
recent years, namely: rail and highway environments, technology, traffic trends, etc.
On the rail side, freight trains are longer, which causes longer block times at crossings. The
expansion of intermodal traffic and the growth of intermodal facilities have led to choke points
on highways in the vicinity of some major intermodal facilities. Longer waits at crossings
contribute to “incentivizing” risky behavior (e.g., driving around lowered gates) by some
highway users. In recent years there has been an uptick in grade crossing accidents.
One would also expect changes in highway user behavior to impact safety at crossings. Trends
toward larger vehicles (e.g., SUVs and light trucks replacing smaller cars) result in slower queue
dispersal at crossings. Changes in traffic mix, increases in number of delivery vehicles, and the
rise of ride-sharing – would all contribute to changes in crossing safety and its prediction based
on characteristics of grade crossings and traffic volumes by mode.
Moreover, since 1986, new technologies and traffic management measures have been deployed at
many crossings, including: constant warning time (CWT) devices, signal pre-emption, and queue
cutters.
1.1.3 APS Limitations
State and local government agencies have alerted the FRA Office of Research and Development
that the APS produces very similar results for a majority of crossings within their jurisdictions,
making it difficult to identify the highest-risk highway-rail grade crossings. Limited variance
among APS-generated assessments is attributed to the predominance of crossings with no
accidents in the preceding 5 years, and similar-site specific characteristics (like traffic counts and
warning devices). New consensus methods of analysis (see the Accident Prediction Model
section) directly address these issues.
The APS includes three separate models for accident prediction – one for each of the three major
grade crossing warning device type categories: passive (signage), flashing lights, and gates.
There is no clear rationale for splitting accident prediction into three separate models, as opposed
to treating the warning device type as a grade crossing characteristic in a single model for all
crossings.
Moreover, the separate models can generate inconsistent outcomes. For example, for some
combinations of grade crossing characteristics, the APS calculates higher risk for crossings with
the same characteristics except for a more protective warning device. It is easy to see how an
analysis of grade crossing risk in a corridor or region could yield results with measures of
relative risk between similar crossings with different warning device types that are highly
suspect.
Similarly, if seeking to estimate the effect of a warning device upgrade (say, from lights to
gates), one could not use the models, segregated by device type category, to estimate risk
reduction. The APS resource allocation procedure is to work around this issue by applying a
crash modification factor (CMF).
5
A CMF reduces the risk of the unimproved grade crossing by
a fixed percentage. The workaround uses the CMF-reduced risk result in place of the APS result
for the assessed risk of the improved crossing. The CMF method, while accepted practice, has
5
Farr (1987), p. 11, calls these “effectiveness factors.” The term crash modification factor was adopted later.
5
been critiqued in the safety research literature.
6
Regardless, the model should enable
recalculation of the risk at the crossing corresponding to a warning device upgrade without
relying on external methods.
Another limitation of the APS model is that it provides no method to determine if risk measures
at different crossings differ with statistical significance
7
(e.g., two crossings with predicted
annual accidents of, say, 0.21 and 0.23, respectively). If the difference in measured risk at two
crossings is not statistically significant, there is no evidentiary basis for treating these crossings
differently (e.g., applying an improvement to one of the crossings and not the other). The APS is
essentially a scoring model where a statistical model is needed (see the example in Appendix B.
Application of the New Model).
1.1.4 Purpose of a New Model
The overarching purpose of a new grade crossing safety model, an alternative to the APS, is to
effect evidence-based safety management of grade crossings. The new grade crossing safety
model should enable users to:
1. Estimate safety and risk at grade crossings.
2. Estimate safety gains due to prospective improvements to crossings and support the
estimation of benefits from these gains.
3. Screen for high risk crossings and develop strategies and programs for safety improvements.
4. Account for statistical significance of differences in measured risk at crossings.
5. Estimate changes in safety at crossings due to changes in some variable value (e.g., growth
of AADT over time).
1.1.5 Policy Perspective of Grade Crossing Safety
Grade crossings are “safety hotspots.” Fatalities in grade crossing accidents numbered 260
8
in
2018. While this may seem small in comparison to total U.S. highway fatalities (36,560
9
in
2018), fatalities and accidents at grade crossings are highly significant when considering the
amount of highway travel that actually traverses grade crossings.
Transportation agencies at all levels recognize that grade crossings are a significant source of
risk and have been singled-out for special programs and safety countermeasures over the years.
Accident risk at grade crossings is eliminated by closure or grade separation (closure, however,
could possibly re-direct the risk from the closed crossing to other grade crossings). Additional
measures like warning device upgrades, supplementary safety measures, and other engineering
solutions have been shown to significantly reduce risk at grade crossings.
6
See Hauer (2015), 186-188.
7
This is similar to asking whether the risk measures of the two crossings are within the “margin of error.”
8
https://safetydata.fra.dot.gov/OfficeofSafety/publicsite/Query/AccidentByRegionStateCounty.aspx
9
https://cdan.nhtsa.gov/tsftables/National%20Statistics.pdf
6
There is a definitional relationship between risk and exposure. Exposure is a measure of
opportunities for accidents to occur. The exposure
10
metric for grade crossing usage is based on
coincident arrivals of trains and highway vehicles at a crossing. It is not surprising to find that
more heavily trafficked grade crossings, in general, have more protection from warning devices.
The analysis in this report examines the relationship between accidents, exposure, and the
principal warning device type categories.
11
The current U.S. DOT APS model has three accident prediction models, one for each warning
device type category. For some ranges of input variables, APS calculates higher risk than with a
more protective warning device type. (For example, with exposure of 1,000 and maximum
timetable speed of 79 mph, the APS predicts more accidents at a gated crossing than at a lights-
only crossing.) This should give pause when considering APS predictions in a region or corridor.
If two crossings have similar data with the exception of the warning device type, do we have
confidence in the relative measure of their predicted accidents? Moreover, would proposed
improvements for the corridor or region be allocated to their most effective use? The new model,
based on modern techniques, replaces the three APS models with a single prediction model that
incorporates warning device type category as a variable. Its predictions consistently preserve
relative magnitudes of risk with different warning devices.
Moreover, the APS resource allocation procedure relies on “effectiveness values”
12
to account
for risk reduction with a warning device upgrade (in recent years, these have been renamed
“crash modification factors”). The road safety literature indicates that such mixed methods can
result in methodological inconsistencies.
13
The assessment of grade crossing risk and the planning and budgeting for improvements are the
sole responsibility of State and local authorities.
14
The public authority assessing grade crossing
risk relies on a model like the APS
15
and bases management decisions for improvements,
accordingly. The quality of those decisions will rely to a great extent on the quality of the risk
assessment.
The new model developed here as an alternative to APS seeks to address the issue of risk
assessment quality by:
• Relying upon current data, appropriate data analysis, and statistical methods
• Examining the relationship between exposure, warning device type, and other key grade
crossing characteristics
10
Exposure, or exposure to risk, is defined for grade crossings as average annual daily trains times average annual
daily highway vehicles at a crossing. This definition is imperfect because accident risk should consider the
correlation of vehicle arrivals by mode, accounting for both seasonality and diurnal distributions of traffic.
11
The APS is defined in Farr (1987). Warning device type categories are: passive, lights, and gates.
12
Farr (1987) p. 11, Table 3 “Effectiveness Values for Crossing Warning Devices.”
13
See, for example Hauer (2015), Appendix L.
14
Upon request, the owning railroad grants the public authority easement to build and maintain the road that
traverses its track. The railroad bears full responsibility for maintaining warning devices and any equipment within
the grade crossing right-of-way.
15
FRA maintains the APS and provides a web-based version at https://safetydata.fra.dot.gov/webaps/.
7
• Properly accounting for accident history
• Presenting a fully transparent model that allows for: single crossing estimates, estimates
of risk for groups of crossings, and determining whether differences in grade crossing
risk warrant similar or different treatment based on statistical measures.
1.2 Objectives
The objectives of the research are as follows:
• Develop a new model to serve as an alternative to the current U.S. DOT APS.
• Document the full development process of the model.
• Demonstrate that the model satisfies statistical criteria and is practical for practitioner
use.
• Validate the new model by comparing its performance against the APS and actual
accident data.
1.3 Overall Approach
1.3.1 About Safety Performance Functions
Since the late 1990s, there has been substantial progress in consensus methods for developing
safety prediction models. These new approaches are presented in AASHTO’s Highway Safety
Manual.
16
In the current mode of thinking, the APS is a type of “safety performance function”
(SPF), which yields a metric indicating the safety of a grade crossing. That metric can be either
the annual expected number of accidents at a crossing or expected accidents by severity type
(e.g., fatal, injury, property damage only – the APS accident severity types).
The SPF is derived in a multi-stage process. The key sources of data for this process are: 1) a set
of traits that characterize the facilities under consideration and 2) the 5-year accident history at
the grade crossings. The database of traits is the U.S. DOT Grade Crossing Inventory System
(GCIS). The database of U.S. DOT Form 57 (a form must be submitted for each highway-rail
crossing accident) captures the grade crossing accident history.
The SPF development involves: First, screen the data in the inventory to eliminate irrelevant or
erroneous data. Second, discover via analysis the functional forms that best describe the data,
and offer hints regarding possible relationships between accidents and traits. Third, derive the
safety model from a suitable statistical estimation procedure. Fourth, adjust the number of
predicted accidents at each crossing to account for the accident history using empirical Bayes
(EB) estimators, which derive from another statistical procedure.
This research covers the development of a new model, namely: the derivation of the SPF, its
validation, and the process for estimating safety risk at grade crossings as an alternative to the
APS.
16
AASHTO (2010).
8
1.3.2 Information the SPF Provides
The model, or SPF, provides estimates of four elements for a given set, or population, of grade
crossings:
1. E[μi], the expected or predicted number of accidents at crossing i
2. σ[μi], the standard deviation of the predicted number of accidents at crossing i
3. E{μ}, the mean of all the μs in a population (all crossings or a subset of crossings)
4. σ{μ}, the standard deviation of all the μs in a population
The following table shows situations for which the above estimates are needed:
Table 1-1. Estimates Required for Different Types of Analysis Focus
Analysis Focus
Average safety E{μ} for subsets of grade
crossings
Safety (μ, σ) of specific grade crossings
What is normal for grade crossings with given
traits?
Is the crossing “unsafe” or has unusually
high risk?
How do the E{μ} vary across subsets of crossings
(e.g., by states or region, by device type)?
Can we rank a collection of crossings and
divide into high- and low-risk groupings?
What would be the aggregate effect of making an
improvement over a population of crossings (e.g.,
eliminate humped crossings)?
What might be the safety effect and
benefit of applying some improvement to
a crossing?
Need E{μ} and σ{μ} to answer the questions
Need E[μ] and σ[μ] to answer the
questions
Source: based on Hauer (2015).
The estimate of the standard deviation of the safety metric is needed in the case of specific
crossings in order to determine whether:
• Predicted accidents are different from zero with statistical significance.
17
• Safety measures of two crossings are statistically different from one another (i.e., if
crossings A and B, say, have predicted accidents of 0.21 and 0.23, respectively, should
they be treated differently or with different priority on the basis of the evidentiary data).
To achieve an SPF, data about grade crossing characteristics, or traits, need to be cast as
statistical models that explain the accident counts at crossings. In developing a safety model for
crossings, there are two clues that the model needs to exploit:
• The first clue is the characteristics (or traits) of the grade crossing. These traits contain
information regarding the common features of grade crossings that contribute to
accidents.
• The second clue available for developing a safety model is the accident history. Accident
history captures the unique qualities of each crossing contributing to safety and risk.
17
“Statistical significance” means that a relationship between two or more variables is caused by something besides
chance. If the ratio of a crossing’s mean predicted accidents to its standard deviation exceeds a threshold value (e.g.,
1.65) then the predicted accidents is said to be “statistically significant at the (e.g.) 90% level.” This is equivalent to
saying that there is a 10 percent probability of a Type I error (falsely rejecting the null hypothesis).
9
As a general approach, the safety model will account for both clues by first predicting accidents
based on characteristics, and then adjust the outcome to account for accident history.
The principles outlined in this section guided the development of the new model for grade
crossing accident prediction and severity.
1.4 Scope
The analysis of the accident and GCIS data and the development of the new model focused on
methods described in AASHTO’s Highway Safety Manual.
18
The approach the project researchers
followed sought to:
• Make best use of their understanding of historical trends, the policy environment, and
practice in using the APS.
• Maximize the number of grade crossings included in the regression analysis.
Researchers did not conduct an exhaustive search of alternative approaches, such as: artificial
intelligence (AI) methods, like “k nearest neighbors” (KNN); methods for “slicing and dicing”
the data into smaller subsets; non-multiplicative (i.e., non-linear in logs) functional forms, etc.
The research team believes that alternative approaches may have merits, but also drawbacks in
comparison with the chosen approach.
The focus of the research was on developing the model. The team recognizes that additional
work is needed to further operationalize the model and provide guidance for use of the new
model by practitioners.
19
1.5 Organization of the Report
Section 2 is a preliminary data review. The section discusses well-established relationships (e.g.,
exposure drives risk, upgrading the warning device type at a crossing reduces risk). It concludes
with a generic functional form based on the principal drivers of risk (exposure and warning
device type) and accommodates additional variables as warranted by data analysis and the
estimation process.
Section 3 describes the data selection and data analysis.
Section 4, the Accident Prediction Model, presents the functional form of the new model
accident prediction, its estimation using the zero-inflated negative binomial (ZINB) regression
method, and the application of the EB method. The section concludes with the new model
equations for accident prediction.
Section 5, the Accident Severity Model, presents the accident severity component of the new
model. It describes the multinomial logistic (MNL) regression method used to develop the
model.
18
18
AASHTO (2010).
19
For example, guidance should provide rules for treating missing data or replacing data from the GCIS with more
current or more relevant estimates.
11
2. Preliminary Data Review
In this section, the research team identifies known relationships or well-supported theories
relating accident risk at grade crossings to grade crossing traits.
The team explored whether a single model could internalize warning device types and thus avoid
having separate models for each class of device. A unified model would ensure that a device
upgrade will be accompanied by accurate risk reduction measurements of accidents at grade
crossings. This would eliminate the need for employing a “crash modification factor” (CMF)
20
approach to estimate the effect of a device upgrade.
It is intuitively clear, and supported by research
21
, that upgrading a warning device type to one
that provides a higher level of protection reduces the accident risk at a crossing (given that all
other factors remain the same). That said, it does not follow that a device upgrade is cost-
beneficial or even a cost-effective way to improve safety at a crossing.
There are three warning device type categories: passive, lights, and gates. Within each category,
there are several warning device types with somewhat differing risk characteristics than the main
category. These will be discussed below.
It is also well understood that risk increases with exposure (although not at a uniform rate for
every level of exposure). As one would expect, for a given crossing the greater the exposure and
risk, the more likely it is that a local authority will (in coordination with the owning railroad)
upgrade the warning device. Consequently, nearly all very low-exposure crossings have passive
devices and nearly all very high-exposure crossings have gates. The researchers expected to
observe a high correlation between device type and exposure at crossings.
This section examines the relationship between accidents, exposure, and device types and
concludes with a general functional form for the accident prediction model.
2.1 Risk by Warning Device Types
Table 2-1 shows the warning device codes by super-category (passive, lights, gates) and their
meaning in GCIS.
20
The CMF approach, often based on before-and-after crash studies, provides a factor associated with risk reduction
for a particular safety countermeasure. For example, a CMF of 0.12 means that predicted accidents after applying
the safety countermeasure will equal predicted accidents before such application times one minus the CMF, i.e.,
Aafter=Abefore*(1-CMF).
21
Elvik, R. and Vaa, T. (2004).
12
Table 2-1. Warning Device Type Codes and Descriptions
Code
Description of Warning Device Type
PASSIVE
1
No sign or signal
2
Other signs or signals
3
Stop signs
4
Crossbucks
LIGHTS
5
Non-train-activated special protection
6
Highway traffic signals, wigwags or bells
7
Flashing lights
GATES
8
Gates
9
4-quadrant gates
Figure 2-1 shows the filtered crossings in the inventory grouped by device type category. The
bars indicate the number of crossings with the specified device type having the number of
accidents in the period shown on the x-axis. Note that the y-axis uses a log scale.
13
Figure 2-1. Accidents by Warning Device Type
2.1.1 Aggregate Risk Adjusted for Exposure by Warning Device Type
To support an accident prediction model with exposure and warning device type as core
variables, the research team examined aggregate risk at crossings by warning device type and
accident rates (i.e., accident count divided by exposure).
Accident per exposure is the most common way to express accident rates on a facility.
22
Note
that the accidents are for 5 years. The exposure data in GCIS
23
are for a typical day. Exposure for
the 5-year period is given by:
22
For example, “Highway Statistics 2018, Federal Highway Administration” gives fatality rates in terms of
“fatalities per 100 million VMT (vehicle-miles traveled).” VMT is the measure of exposure for general highway use.
23
As a caveat, note that the GCIS data are reported by State and local agencies with varying data quality. Moreover,
some data fields are not maintained as vigorously as others. For example, data for warning device type are, for the
most part, current and accurate. Data for the railroad and highway environments at crossings (e.g., AADT, train
traffic) tend to be less current and may be out-of-date.
14
Equation 1. Exposure in the Analysis Period (2014–2018)
where:
xp
Exposure in 5-year period
aadt
Average annual daily traffic
dt
Daily trains at the crossing
300
Number of annual traffic days
5
Number of years
Figure 2-2 shows the crossing risk divided by exposure for each device type category. The data
points (colored purple and orange) show the risk per exposure at each crossing grouped by
warning device type. The risk values are shown at the data points in bold and by the left y-axis.
The bars are the number of crossings in each group and their values are represented by the right
y-axis. Note also the number below the risk value, which is the count of accidents in the period
for each grouping of crossings.
Focusing for now on the orange data points, these represent the largest groupings in each of the
three super-categories: passive, lights, and gates. A lights crossing has 73 percent less risk per
exposure in comparison to the passive crossing. Compared to a lights crossing, the gated
crossings have 63 percent less risk per exposure.
The orange points were singled-out because they represent: 1) the main grouping in the super-
category, and 2) in each, there is a substantial number of crossings and accidents. The “Stop
Signs” category is also sizable and its risk per exposure is not that different than the risk per
exposure of the crossbucks grouping (1.122 vs 1.479; in other words, crossbucks are about 75
percent as risky per exposure as stop signs). Moreover, there are over 10,000 crossings in the
“Stop Signs” category and initial inspection indicates that it will likely be advantageous to merge
the two categories into the “passive” category.
The other warning device type categories within each super-category are somewhat small
samples of crossings and accidents with widely different risk characteristics than the main
grouping. The crossings with codes for these groupings (1, 2, 5, 6, 9) will be omitted from the
analysis. (For accident prediction of these device types, we would use the super-category and
then apply a CFM to scale the risk given the best available information).
15
Figure 2-2. Risk per Exposure (Accident Rate) by Warning Device Type
2.1.2 A Generic Functional Form for Accident Prediction
The following generic functional form follows from the above discussion.
Equation 2. Generic Functional Form for Accident Prediction
NOTES TO FIGURE 2-2
The bars in the chart are the number of grade crossings (shown on the right y-axis) for each
warning device type (shown on the x-axis). In the x-axis labels, the letters in parentheses indicate
the principal warning device type category (P – passive, L – lights, G – gates).
The square markers represent the average number of accidents per exposure at crossings with the
warning device type (shown on the left y-axis). Markers are colored orange for the warning
device type with the largest number of grade crossings in the warning device type category.
Upper number: Accident rate
Lower number: Accidents in sample
16
where:
xp
Exposure (= daily trains * aadt)
x
Other variables (vector)
D2
1 if crossing warning device is lights, 0 otherwise
D3
1 if crossing warning device is gates, 0 otherwise
Note: If D2 = D3 = 0 then the warning device at the crossing is passive
From an understanding of the impacts of exposure and warning device types on accident risk, the
parameter estimates of coefficients from a statistical estimation process would yield the
following:
that is, a crossing with lights warning device has less risk than a crossing with passive device,
and a gates crossing has less risk than a lights crossing. (The “hat” diacritical indicates an
estimated coefficient of the model.)
The following chart shows the relative risk of an example grade crossing for different warning
device types and at different levels of exposure. Note that for very low levels of exposure all
crossings have passive warning devices, and at very high levels of exposure grade crossings are
gated. Grade crossings with lights fall in the middle range of exposure.
0>2>3
17
Figure 2-3. Relative Risk Levels by Warning Device
24
The following sections show how this general form, together with additional model variables,
will combine in the new accident prediction model.
24
The elasticity of risk with respect to exposure (set to a value of 0.35) is drawn from the current APS and
preliminary data analysis. Elasticity is the percent change in one variable (e.g., accident risk) when another variable
(e.g., exposure) varies by 1 percent.
18
3. Data Selection and Analysis
The section describes the process of data selection for the development of the new model that
will serve as an alternative to APS. The goal was to produce a model that defines an SPF for
grade crossings. The first focus was on a model predicting accident occurrence, and later in this
document address accident severity prediction given an accident.
Following data analysis and selection of traits for inclusion in the new model, additional filters
may be applied to the data to account for missing/erroneous values for the new model traits. An
additional consideration that accompanied the data analysis was to retain as many grade
crossings in the dataset for model estimation as practical.
The research team sought variables that were likely to support a model. Since the researchers
proceeded from the assumption that key drivers are represented by exposure and warning device
type, they further assumed that f(x) from Equation 2 in the previous section was linear in its
variables (which were the explanatory variables the team sought to identify for inclusion in the
model).
3.1 Data Sources
The two sources of data for the development of the new model are:
• Grade Crossing Inventory System (GCIS) data. The reference document for the data is
“FRA Instructions for Electronic Submission of U.S. DOT Crossing Inventory Data,
Grade Crossing Inventory System (GCIS), v2.9.0.0, Released: 7/2/2019.” Grade crossing
data updates are electronic submissions of Form FRA F 6180.71 by railroads, transit
agencies, and States. GCIS uses Open Data (OData), a RESTful (REpresentational state
transfer), for data downloads. OData downloads provide a single table that includes all
five parts of the inventory – including header information. The data contain one row for
each grade crossing in the inventory representing the most current data per the submitting
agency’s most recent submission.
• The FRA safety data website provides downloading accident data by year. The accident
data source is Form 6180.57, which railroads submit to FRA following each grade
crossing accident. The Form 6180.57 data download as a single table (in Excel or Access
formats) with each accident represented as a single row in the table. For the analysis,
researchers looked at accidents in the 5-year period 2014–2018.
We downloaded and inserted the data into SQL server database tables. The tables were merged
into a single table with an additional column for total accidents in the period (2014–2018).
3.2 Data Selection
This section describes the process for filtering the data so as to include those crossings that are
the focus of the analysis, while eliminating from analysis those crossings that are not of interest
(e.g., closed or grade separated). Researchers also filtered out data that had missing or erroneous
values for several key analysis variables. Table 3-1 summarizes the data filters along with the
number of crossings, accidents, and number of crossings with accidents remaining after applying
each filter. The team sought to keep the number of grade crossings in the selection as large as
19
possible so that its practical application in prediction would not require an extensive set of rules
to account for missing or erroneous data. For example, if a variable seemed promising for
inclusion, yet only, say, 30 percent of grade crossings had data for the variable – researchers
opted to exclude it.
3.2.1 Public Crossings Only
GCIS identifies public crossings as those having a value of 3 in the TypeXing field. For private
crossings, the roadway is maintained by a private individual or entity. There is no legal
obligation for the road maintainers at private crossings to submit data to GCIS. Each year, on
average, 14 to 15 percent of accidents occur at private crossings. However, the data of crossing
characteristics at private crossings are extremely sparse. Consequently, these have been excluded
from the analysis.
3.2.2 At-Grade Crossings Only
Crossings that are grade separated pose no risk of collision between trains and highway vehicles,
hence these crossings are excluded. The field PosXing with value set to 1 identifies a crossing at-
grade.
3.2.3 Closed Crossings
GCIS identifies closed crossings when the ReasonID (reason for submitting a data update) field
is set to value 16. Crossings with ReasonID = 16 have been eliminated from the analysis. Note
that it may be the case that a closed crossing was subsequently updated for a different reason, in
which case there would be no indicator in GCIS that the crossing was closed.
3.2.4 Missing or Erroneous Values for AADT
Without a value for average annual daily traffic (AADT), risk exposure at the crossing could not
be evaluated (defined as AADT times the number of daily trains). Note that AADT, like other
variables in GCIS, may be out-of-date.
3.2.5 Missing or Erroneous Values for Number of Daily Trains
As with AADT, crossings that have missing or erroneous data for total number of daily trains
have been excluded.
3.2.6 Missing or Erroneous Values for Highway Lanes and Tracks
These two variables are the key descriptors of infrastructure at crossings and may be important
predictors of accidents.
20
Table 3-1. Summary of Data Selection
Filter Criterion
(with previous
filters)
Number of
Crossings
Remaining after
Filter
Total Number of
Accidents 2014-2018
at Remaining
Crossings
Of Remaining
Crossings, Number
with Accidents
None
429,463
10,675
8,814
Public only
266,304
9,147
7,538
At-grade only
220,289
9,110
7,503
Exclude closed
130,107
8,986
7,390
Exclude 0, missing,
erroneous AADT
128,378
8,922
7,334
Exclude 0, missing,
erroneous highway
lanes
127,755
8,895
7,308
Exclude 0, missing,
erroneous daily trains
105,383
8,467
6,944
Exclude 0, missing,
erroneous total tracks
105,362
8,465
6,942
3.3 Candidate Variables
Variables in the GCIS that were considered candidates for explaining accidents are shown in the
table below. Researchers eliminated from the list variables that are already accounted for in the
exposure variable (i.e., trains and AADT) and those that are likely highly correlated with these
variables. Warning devices were also excluded, as the team included these by default in the new
model. The variables are divided into two groups: discrete and continuous.
The analysis assesses whether a variable is a likely candidate for inclusion in the model.
Table 3-2 Candidate Variables for Inclusion in the New Model
Discrete
Continuous
Approach angle
Percent truck
Development type
Passenger train count
Main track?
Hwy speed
Traffic lane type
Max timetable speed
Paved/unpaved
Crossing surface type
Urban/rural
21
Discrete
Continuous
Highway functional class
Advanced warning
3.3.1 Discrete Explanatory Variables
The discrete variables are essentially category variables that indicate a crossing belongs to a
particular category among two or more possibilities. The variables are represented in the data as
integer values. However, there is no ordered relationship among the categories represented by
the integers.
The method for evaluating the discrete variables for inclusion in the model was to consider
crossings with 5-year accident history greater than 0. Researchers then examined a boxplot chart
of accidents normalized for exposure and warning device types
25
, grouped by the variable by its
different levels. If the boxplot indicated significant variance across groupings (i.e., the groupings
displayed different medians and other measures indicating variance), then the variable would be
considered for inclusion in estimation. If the boxplot displayed no such variance, the team
concluded that the variable did not have a strong impact on accident prediction and would be
excluded.
As an example, the following chart shows the boxplot for the variable of grade crossing surface
type. Researchers aggregated the two categories of “Concrete” and “Concrete and Rubber.” This
variable displays variance across its categories, so it was flagged for inclusion in the new model.
25
“Accidents normalized for exposure and warning device types” means accidents in 5-year history divided by the
product of exposure and a risk factor for the warning device type. The risk factors used were: passive = 1.0, lights =
0.3 and gates = 0.1. These values are based on the analysis of the previous Section.
22
Figure 3-1. Boxplot of Normalized Crossing Accidents by Grade Crossing Surface Type
23
The following chart shows the boxplot for the variable of grade crossing angle. There is very
little variance across the groupings. Consequently, this variable was excluded from the model.
Figure 3-2. Boxplot of Normalized Crossing Accidents by Grade Crossing Angle
Following the review of the discrete variables, it was found that the following variables
warranted inclusion in the model: 1) Crossing surface type, and 2) RuralUrban.
3.3.2 Continuous Explanatory Variables
The grade crossings characteristics that are continuous variables were ordered (i.e., all variable
values are comparable, and if values are different, then one is greater than the other). Each can
assume a range of values, not necessarily integers. However, data specifications typically restrict
the values to integers (e.g., maximum timetable speeds can assume values from 1 to 99).
The method for evaluating the continuous variables for inclusion in the model was to consider
crossings with 5-year accident history greater than 0. Researchers then examined a boxplot chart of
accidents normalized for exposure and warning device types, grouped by the variable for each of its
10 deciles. If the boxplot indicated a good distribution of the variable, and an observed functional
relationship across deciles, then the variable would be considered for inclusion in estimation,
otherwise it was not.
The following chart shows the boxplot for the variable of maximum timetable speed.
There was a clear increasing trend for increasing decile. Consequently, this variable was included in
the model.
24
Figure 3-3. Boxplot of Normalized Crossing Accidents by Maximum Timetable Speed
Deciles
The following chart shows the boxplot for the variable of percent truck of highway traffic.
Figure 3-4. Boxplot of Normalized Crossing Accidents by Percent Truck Deciles
25
There was no clear relationship that changes over deciles of the variable. Consequently, this
variable was excluded from the model.
Following the identification of variables for inclusion in the model estimation, researchers
further filtered the remaining crossings to exclude from the regression analysis crossings that
have a) non-standard warning device codes or b) missing or erroneous values for included
explanatory variables.
Table 3-3. Final Data Selection
Filter Criterion (with
previous filters)
Number of
Crossings
Remaining after
Filter
Total Number of
Accidents 2014-2018 at
Remaining Crossings
Of Remaining
Crossings,
Number with
Accidents
Exclude non-standard
warning device codes
(1, 2, 5, 6, 9). See
Section 2.1
102,054
8,204
6,743
RuralUrban missing or
erroneous values
101,838
8,187
6,730
XSurfaceIds2 missing
or erroneous values
94,033
7,822
6,409
MaxTtSpd missing or
erroneous values
94,029
7,822
6,409
26
4. Accident Prediction Model
This section presents the selected accident prediction model, its regression with the ZINB
estimation procedure, and the EB adjustment of the ZINB-predicted values.
ZINB is one type of zero-inflated models. It is used for count variables (e.g., accidents) that
exhibit excess zeroes. “Excess zeroes” means that of the many crossings with no accidents in the
preceding 5 years, some of those were crossings effectively had no risk of an accident.
The ZINB model assumes that:
• Each crossing has some non-zero probability of being a no-risk crossing.
• Each crossing has an expected number of annual accidents.
• Accident counts for the population of crossings conform to a negative binomial
distribution (the standard deviation of accidents for the population is greater than the
mean, indicating overdispersion).
ZINB has been adopted in numerous accident studies and is well-suited for the analysis of grade
crossing accidents.
The EB method adjusts the estimate of the expected number of accidents so as to account for
history, and correct for “regression to the mean”
26
bias. The equation relies on the ZINB
regression outputs to estimate a weighting factor. The EB-adjusted estimate is a linear
combination of the predicted accidents (from ZINB) and the actual count of accidents. If the
accident history indicates no accidents, then the EB adjustment will adjust the expected value of
accidents downwards toward zero. For crossings with non-zero accident history, EB will adjust
the expected value (usually upward) so that it is closer to the actual count.
R software was used in the model estimation.
4.1 The Accident Prediction Model
Based on the analysis described in the previous sections, the selected accident prediction mode is
shown below. The model has two components: 1) a count model and 2) a zero-inflated model.
Equation 3. The ZINB Count Model
26
“Regression to the mean” basically means that if a variable is extreme the first time you measure it, it will be
closer to the average the next time you measure it. For example, if we randomly selected a crossing that had several
accidents in its 5-year history (that is, a very high risk grade crossing), the next random selection would be a
crossing whose risk was much closer to the mean for all grade crossings.
27
Equation 4. The ZINB Zero-Inflated Model
Equation 5. The ZINB Combined Model
where:
NCountPredicted
Predicted accidents of count model (data for left-hand side of regression are
counts of accidents at crossings in 5-year period 2014–2018)
PInflatedZero
The probability that the grade crossing is an “excess zero”
NPredicted
Predicted accidents after accounting for excess zeroes
lExpo1
Exposure, equal to average annual daily traffic times daily trains
D2
If warning device type is lights =1, 0 otherwise
D3
If warning device type is gates =1, 0 otherwise
(note: if both D2 and D3 are zero, then warning device type is passive)
RurUrb
If Rural = 0, if Urban = 1
XSurfID2s
Timber = 1, Asphalt = 2, Asphalt and Timber OR Concrete OR Rubber = 3,
Concrete and Rubber = 4
lMaxTtSpd1
Maximum timetable speed (integer value between 0 and 99)
lAadt1
Average annual daily traffic
lTotalTrains1
Total number of daily trains
1These variables have been transformed as follows: lx = log(1+αx), where x is the original
variable and α is a factor. The factor α was selected so that for the median value of x, ln(1+αx) =
ln(x)
4.2 ZINB Regression
The ZINB regression model has two components: the count model and the zero-inflated model.
The count model is for predicted accidents before considering the probability of excess zeroes.
The zero-inflation model is for estimating the probability of an inflated zero. (An “inflated zero”
is a zero accident count that does not derive from a grade crossing’s traits; rather, it is zero
because the crossing accident risk is effectively 0.) Note that the explanatory variable for the
zero-inflated model is the total number of trains; that is, the fewer trains at a grade crossing the
higher the probability of an excess zero.
The predicted (fitted) values of the model are given by f(x)*(1-g(s)), where f is the count model
(operating on the vector of inputs x for each observation) and g is the zero-inflation model
(operating on the vector of inputs s for each observation).
28
The following table shows the output for the zero-inflated negative binomial regression for the
model in the previous section.
The final set of crossing data used in the regression included 94,029 grade crossings with 7,822
accidents at 6,409 crossings in 2014-2018 (see Table 3-3).
Table 4-1. ZINB Regression Output
Count model (negative binomial with log link)
Variable
Estimate
Std. Error
z-Value
Pr(>|z|)
(p-value)
Confidence
Level
(Intercept)
−8.3592
0.3208
−26.059
< 2e16
> 99.99
lExpo
0.1902
0.0287
6.638
3.18e11
> 99.99
D2
–0.2848
0.0481
−5.926
3.10e09
> 99.99
D3
−0.8577
0.0409
−20.976
< 2e16
> 99.99
RurUrb
0.3935
0.0316
12.444
< 2e16
> 99.99
XSurfaceID2s
0.1318
0.0172
7.686
1.52e14
> 99.99
lMaxTtSpd
0.6876
0.6876
22.702
< 2e16
> 99.99
lAadt
0.1063
0.1063
3.511
0.000446
> 99.99
Log(θ)
−0.2593
0.0887
−2.925
0.003447
> 99.00
Zero-inflated model (negative binomial with log link)
Variable
Estimate
Std. Error
z-Value
Pr(>|z|)
(p-value)
Confidence
Level
(Intercept)
1.1708
0.1900
6.1620
7.19e10
> 99.99
lTotalTr
−1.0109
0.0845
−11.9610
< 2e16
> 99.99
Summary Statistics
Log-Likelihood
AIC
–2.462e+04
49260.26
Pearson Residuals
Minimum
1st Quartile
Median
3rd Quartile
Maximum
−0.6559
−0.2742
−0.2072
−0.1504
28.5137
Notes to the regression output:
• The values in the “Estimate” column are estimates of the model coefficients and
correspond to the βs from the count model equation (Equation 3) and γs from the zero-
inflation model equation (Equation 4).
• The column “Std. Error” shows the standard error of the coefficient to the left.
• The “z-value” column is the coefficient divided by the standard error (larger absolute
values of z indicate that the coefficient has greater statistical significance).
• “Pr(>|z|)” is the probability of exceeding the absolute value of the z-value (smaller values
indicate greater statistical significance).
29
• The rightmost column shows the confidence level of the coefficient.
• θ
27
is the inverse of the overdispersion parameter (α) of the count model. The estimate of
θ is 0.7716 (and the imputed value of α=1.296). α was expected to be greater than 1.
• AIC is the Akaike Information Criteria for model quality given the dataset.
Key points to note from the regression output:
• The coefficients for lExpo and lAadt have positive signs with expected magnitudes.
• The coefficients for D2 and D3 are negative (i.e., compared to passive devices, lights,
and gates reduce risk). The coefficient of D3 is about three times that of D2, which
conforms to expectations.
• The signs and magnitudes of other coefficients in the count model seem to correspond to
expectations.
• The coefficient of lTotalTr (i.e., total trains) in the zero-inflation model is negative, i.e.,
the probability of an excess zero decreases with the number of trains, as expected.
• All the coefficients have strong statistical significance.
28
• The Akaike Information Criterion (AIC)
29
is the least value for all tested models.
• The estimated mean and standard deviations for the population are:
o Mean: 0.08316
o Standard deviation: 0.21377
Figure 4-1is a chart of the ZINB predicted values grouped by device type. The vertical lines on
the chart indicate the average log of exposure for each grouping. The horizontal lines on the
chart indicate the average predicted 5-year accidents for each grouping. The vertical line
indicates the average log of exposure for each grouping.
27
θ is the Greek letter “theta.”
28
“Strong statistical significance” for an estimated coefficient means there is a very small probability of falsely
rejecting the null hypothesis (i.e., the hypothesis that the coefficient is actually 0).
29
From Wikipedia: The Akaike information criterion (AIC) is an estimator of out-of-sample prediction error and
thereby relative quality of statistical models for a given set of data. For a statistical model, let k be the number of
estimated parameters in the model. Let L be the maximum value of the likelihood function for the model. Then the
AIC value of the model is the following: AIC = 2k - 2*ln(L)
30
Figure 4-1. ZINB Predicted Accidents by Warning Device Type
4.3 Predicting Accidents from the Regression Outputs
One can apply Equation 3, Equation 4, and Equation 5 above to calculate the predicted accident
at a grade crossing (prior to applying the EB adjust described in the following section). The
predicted accidents are the fitted values (i.e., Ŷ) of the model.
The βs in the equations are the ZINB count model coefficient estimates and the γs are the ZINB
zero-inflated model coefficients estimates.
4.4 Empirical Bayes Prediction Adjustment
The EB adjustment intends to correct the prediction for “regression to the mean” bias while
adjusting the expected value to account for accident history. The process is described in Hauer.
30
For each grade crossing, the expected number of accidents is given by:
Equation 6. Empirical Bayes Adjustment
30
E. Hauer, The Art of Regression Modeling in Road Safety, Springer 2015.
31
where:
NExpected
The adjusted number of predicted accidents
NPredicted
The number of predicted accidents from the ZINB regression procedure
NObserved
The number of observed accidents (i.e., count of accidents at the grade crossing)
and the weighting factor w is given by:
Equation 7. EB Weighting Factor
The variance of NPredicted is given by:
Equation 8. Variance of Crossing's Predicted Number of Accidents
where theta, as noted above, is the inverse of the overdispersion parameter α from the ZINB
regression (θ is estimated to be 0.7716).
Note that the underlying assumptions of the model indicate that the accident count data for a
population of crossings is best described by the NB distribution. The overdispersion parameter
describes the overdispersion of data relative to a Poisson distribution (where mean and variance
are assumed equal). R software defines the variance of the count variable as μ+μ2/θ.
31
Given this
definition of variance, θ should be less than 1 and greater than 0.
Figure 4-2 shows the predicted values grouped by device type, with this chart showing the
predicted values including the EB adjustment.
Compared to Figure 4-1 this chart shows the predicted values clustered around the values that
represent the accident counts in each grade crossing’s 5-year accident history.
31
Most other software packages (e.g., SAS, Stata, Limdep, SPSS, etc.) define the variance of the count variable as
μ+ α · μ2. R’s θ is equivalent to 1/ α in the other packages. α is the overdispersion parameter of the negative
binomial distribution, as defined in these other packages and most of the academic literature.
VN = N 1 + N P + 1
32
Figure 4-2. ZINB+EB Predicted Accidents by Warning Device Type
4.5 Cumulative Residual (CURE) Analysis
The parameter estimates from the ZINB regression in Table 4-1 exhibit strong statistical
significance. However, one needs to know that the model generates unbiased estimates over the
model variables’ ranges. One method for identifying the presence of bias is the cumulative
residual (CURE) analysis.
32
The residuals are the difference between the accident count and the
predicted (i.e., model-fitted) values. The residuals are ordered by increasing exposure, and the
CURE plot shows the cumulative residuals.
Figure 4-3 below shows the CURE plot. The black plot shows the cumulative residuals for the
above ZINB+EB model, and the exposure variable (for now, ignore the red plot).
The vertical lines on the chart divide it into five regions. Each region is labeled with a Roman
numeral and, below it:
• The number of grade crossings having exposure values within the region
• The accident count at grade crossings having exposure values within the region
Note the black CURE plot remains fairly flat in regions I and V; it climbs in regions II and IV
and declines in region III.
32
E. Hauer (2) devotes a chapter of his book to the CURE method.
33
Figure 4-3 Cumulative Residual Analysis for Exposure
The CURE plot should not have long runs of steady increases or decreases. Ideally, it should
resemble a symmetric “random walk,” about 0. When the plot is climbing it represents a region
of the exposure variable where the model is consistently underestimating predicted accidents.
Likewise, when the graph descends it is a region of the exposure variable where the model
consistently overestimates predicted accidents. These regions of consistent over- or
underestimation are called “bias-in-fit.”
The model requires adjustment to mitigate the bias-in-fit revealed by the CURE plot. A proposed
adjustment is to add two dummy variables to the ZINB regression, defined as follows:
(The regions in the above variable descriptions refer to those in Figure 4-3.)
The following equation shows the revised ZINB count model after adding the new dummy
variables (replacing Equation 3):
Equation 9. The Revised ZINB Count Model
3=0,
1,
3=0,
1,
=
0+1+23+34+42+53+5+62+6+7
34
The table below shows the outputs for the revised ZINB model of accident prediction. In
comparison with the previous ZINB model, note that:
• The parameters of the revised model are of the same signs and similar magnitudes.
• The original parameters remain highly significant and the parameters for the new dummy
variables are also significant.
• The AIC statistic is lower (indicating better overall fit) for the revised ZINB model.
The cumulative residuals for exposure with the new ZINB model and EB adjustment is shown by
the red graph in the CURE plot of Figure 4-3. While the graph is not “perfect,” the introduction
of the dummy variables seems to have had the desired effect: The graph crosses 0 multiple times
and its upward and downward oscillations are more constrained.
Table 4-2. Revised ZINB Regression Output
Count model (negative binomial with log link)
Variable
Estimate
Std. Error
z-Value
Pr(>|z|)
(p-value)
Confidence
Level
(Intercept)
–8.01314
0.32364
–24.759
< 2.00E–16
> 99.99
lExpo
0.16952
0.02867
5.913
3.37E–09
> 99.99
Dx3
–0.09801
0.0353
–2.777
0.005491
>99.00
Dx4
0.13392
0.0525
2.551
0.010741
>95.00
D2
–0.2283
0.04955
–4.607
4.08E–06
>99.99
D3
–0.81117
0.04248
–19.097
< 2.00E–16
> 99.99
RuralUrban
0.38484
0.03176
12.117
< 2.00E–16
> 99.99
XSurfaceID2s
0.1352
0.01716
7.877
3.35E–15
> 99.99
lMaxTtSpd
0.67161
0.03045
22.057
< 2.00E–16
> 99.99
laadt
0.11483
1.11111
3.777
0.000159
> 99.99
Log(theta)
–0.25711
0.08661
–2.969
0.002992
> 99.00
Zero-inflated model (negative binomial with log link)
Variable
Estimate
Std. Error
z-Value
Pr(>|z|)
(p-value)
Confidence
Level
(Intercept)
1.24505
0.18757
6.638
3.18E–11
> 99.99
lTotalTr
–1.05711
0.08682
–12.176
< 2.00E–16
> 99.99
Summary Statistics
Log-Likelihood
AIC
-2.46e+04 on 13 Df
49228.78
Pearson Residuals
Minimum
1st Quartile
Median
3rd Quartile
Maximum
−0.6820
−0.2705
−0.2054
−0.1515
28.7961
35
5. Accident Severity Model
Grade crossing management in the U.S. considers three severity categories: fatal, injury and
property damage only (PDO). A fatal accident is one with at least one fatality; an injury accident
has at least one injury; and a PDO accident has no injuries or fatalities.
The accident severity model seeks to determine the probabilities of prospective accidents at
grade crossings belonging to each severity category. The process for predicting accident severity
is one of allocating predicted accidents to each severity category. In the APS, there is no process
to calibrate accident severity. Over time, accident severity has been fairly stable: fatal accidents
are about 10 to 12 percent of the total, injury accidents about 27 percent, and PDO accidents
about 61 percent.
The remainder of the section describes the data, the logistic regression process used in the model
estimation, and the model results. Some comparisons of the new model with the APS are
discussed in the next section.
R software was used in the model estimation.
5.1 Description of the Data
Federal law requires filing a Form 57 accident report for each grade crossing accident. The
analysis used the Form 57 report database and GCIS. Researchers examined accidents in the
period 2014–2019 (6 years) during which there were 12,983 accidents. They excluded from the
model estimation process accidents from the following crossings:
• Private crossings
• Crossings where traits were missing data for key explanatory variables.
There were 11,131 accidents at public crossings. Of these, 9,870 contained all the data for key
explanatory variable, and these were included in the model estimation. Of the 9,870 accidents,
1,355 (13.7 percent) were fatal, 2,768 (28.0 percent) were injury accidents, and 5,747 (58.2
percent) were PDO.
These accidents were matched with the grade crossing data from GCIS for each crossing where
an accident occurred.
5.2 The Accident Severity Model
For the accident severity model, the researchers sought to estimate the probabilities that given an
accident, the accident will be one of three types: fatal, injury or PDO. The explanatory variables
for these estimates are grade crossing characteristics. The research sought, therefore, to model
three variables:
Equation 10. Probabilities to Estimate – Fatal
36
Equation 11. Probabilities to Estimate – Injury
Equation 12. Probabilities to Estimate – PDO
keeping in mind the following constraint:
Equation 13. Constraint that Severity Probabilities Sum to 1
Additionally, the categories of accident severity are ordered, that is:
Equation 14. Ordering of Severity Categories
Where S() indicates accident type severity. Note that the ordering is ordinal, that is, there is no
measure of relative severity. (While it can be said that a fatal accident is more severe than an
injury accident, it cannot be said that one accident type is two, three or five times more severe
than the other
33
.)
There are several methods for estimating a model with the dependent (also called the left-hand
side or LHS) variable representing several ordered categories. The chosen estimation process is
the ordered logit model (also called the proportional odds model or the parallel lines model).
5.2.1 The Ordered Logit Model
The dependent variable of the model is an observed ordinal variable Y (i.e., the accident severity
type). The model assumes that there is a continuous, unmeasured latent variable, Y*, whose
values determine the value of the observed ordinal variable Y. The variable Y* has two threshold
points represented by κ (the lowercase Greek letter kappa).
The value of the observed variable Y depends on whether Y* has crossed a threshold, as follows:
Equation 15. Relationship Between Y and Y*
The latent variable Y* is a function of grade crossing characteristics. Thus, the ordered logit
model to estimate for a given specification (i.e., for a selected set of explanatory variables) is
given by the following:
33
Introducing costs could support an analysis of relative severity, however, it would not assist in analyzing the
probability of an accident belonging to a specific severity category.
=, 1
, 12
, 2
37
Equation 16. Ordered Logit Model for Three Severity Categories
where:
j
Index of grade crossing
P(Y)
The probability that an accident is of type PDO, injury or fatal
k
Index of the selected set of K explanatory variables
Xkj
The kth explanatory variable (a characteristic of the jth grade crossing)
βk
Coefficient (to be estimated) of kth explanatory variable
κ1
Coefficient (to be estimated) of the threshold separating PDO from injury accident
κ2
Coefficient (to be estimated) of the threshold separating injury from fatal accident
5.3 Model Specification and Regression Results
A number of alternative model specifications were attempted. The selected specification is the
one that generated the smallest AIC (Akeike Information Criterion) value. The explanatory
variables in the selected specification include the following:
• lMaxTtSpdSq – this variable is based on the square of maximum time table speed (mtts)
at a grade crossing (transformed as shown in the next equation). The rationale for linking
severity to the square of mtts is that accident severity is largely a function of the kinetic
energy generated by an accident. The kinetic energy is proportional to the square of the
speed. The mtts variable is capped at 70 mph, that is, for mtts exceeding 70 the variable is
fixed at 70.
• lThru – this variable is the number of daily through trains at the crossing, transformed as
shown in the next equation.
• lSwitch – this variable is the number of daily switch trains at the crossing, transformed as
shown in the next equation.
• lAadt – this variable is the average annual daily highway traffic at the crossing,
transformed as shown in the next equation.
The above four variables were transformed as follows:
38
Equation 17. Transformation of Variables
Where is the mean value of the variable X. The transformation achieves two objectives. The
transformed variable is calculable at 0, and the value of the transformed variable is equal to the log of
the untransformed variable at its mean value.
• The next variable included in the variable was RuralUrban (assuming values 1 if grade
crossing is in a rural area, 0 otherwise).
• The last variable included in the variable was D1 (assuming values 1 if grade crossing has no
lights or gates, 0 otherwise).
The ordered logistic regression output is shown in the following table:
Table 5-1. Accident Severity Ordered Logistic Regression Output
Variable
Coeff.
Estimate
Std. Error
z-Value
Pr(>|z|)
(p-value)
Confidence
Level
(PDO | Injury)
κ1
–3.05946
0.19728
–15.5082
< 1e16
> 99.9
(Injury | Fatal)
κ2
–4.60832
0.20025
–23.0127
< 1e16
> 99.9
lMaxTtSpdSq
β1
–0.29043
0.02368
–12.2637
< 1e16
> 99.9
lThru
β2
–0.10696
0.02408
–4.44116
< 9e06
> 99.9
lSwitch
β3
0.13847
0.04140
3.34481
< 9e04
> 99.9
lAadt
β4
–0.03317
0.01354
–2.45074
< 2e02
> 99.0
Rural Urban
β5
–0.14500
0.05106
–2.83989
< 5e03
> 99.5
D1
β6
–0.20471
0.06004
–3.40951
< 7e04
> 99.9
Summary Statistics
Residual Deviance
AIC
18224.88
18224.88
The coefficient estimates exhibit a high level of confidence (high level of confidence coincides
with a low probability of a Type I error
34
). The value for the AIC is the least among all of the
variable combinations tested.
34
A Type I error occurs when rejecting a true null hypothesis.
=log1+(1)
39
5.4 Accident Severity Forecast Equations
Equation 18 shows forecast equations for the accident severity model.
Equation 18. Accident Severity Forecast Formulas
Notes to equations:
• The subscript j indicates a grade crossing.
• P() is the variable indicating probability of accident type (fatal, injury or PDO).
The following chart shows forecast severity for 50 accidents with the new model:
Figure 5-1. Severity Predictions for 50 Crossings with the New Model
40
41
6. Validation
The section presents validations for the new model (estimated with the ZINB and EB methods).
Note here that the term “prediction” means the expected value of accidents at the crossing. In
general, accidents are rare and the (annualized) expected value of accidents at a crossing will be
a real value between 0 and 1. A non-zero accident count will be larger in most cases than the
expected value of accidents at a crossing, which reflects the fact that the observed count in a
previous year is not expected to repeat frequently in subsequent years.
The first validation compares cumulative predicted accidents by the new model and the APS
with the actual risk as measured by accident counts.
The second validation shows the predicted accidents for the new model and the APS for
crossings grouped by accident count.
The third comparison examines the model results (the new model and APS) for different
groupings of high-risk crossings and shows the results in a chart. In this case, researchers
counted accidents at the 50 highest-risk crossings (and then at the subsequent groupings of
highest-risk crossings). The better of the two models will predict accidents at the groupings of
crossings that is closer to the actual accident counts.
For the severity model, this report shows comparisons of the model performance with that of the
APS.
6.1 Accident Prediction – Cumulative Risk
For this validation we order the grade crossings from high risk to low risk (according to total
accidents in 5-year history). The y-axis on the charts below shows the actual cumulative risk and
the predicted risk with each model. The better model is the one that tracks closer to the actual
cumulative risk.
The four charts below represent two cases and two periods. The first case displays cumulative
accident count and predictions for all crossings in the estimation sample (which includes 94,029
crossings). The second case focuses on the crossings with non-zero accidents. The first period is
the estimation period 2014–2018. The second period is the following year, which covers 5-year
accidents from 2015–2019.
The vertical line indicates the boundary between those crossings with non-zero accidents in the
period (to the left of the line) and those with zero accidents in the period (to the right of the line).
Figure 6-1 and Figure 6-2 show the counts and predictions, ordered from high to low risk, for the
complete set of crossings in the estimation sample. Figure 6-1 is for the period 2014–2018.
Figure 6-2 is for the period 2015-2019.
Figure 6-3 and Figure 6-4 show the same chart data as Figure 6-1 and Figure 6-2, but limit the
data displayed to those crossings with non-zero accident history.
The charts demonstrate that the new model was the better predictor of accident risk than the
APS.
42
Figure 6-1. Model Comparison (2014–2018, all crossings in sample)
Figure 6-2. Model Comparison (2015–2019, all crossings in sample)
43
Figure 6-3. Model Comparison (2014–2018, crossings in sample with non-zero accidents)
Figure 6-4. Model Comparison (2015–2019, crossings in sample with non-zero accidents)
On the riskiest crossings, the new model (ZINB+EB) predicted cumulative accident risk much
better than APS.
44
6.2 Accident Prediction – Risk at Crossings by Accident Count Groups
In the second validation, researchers grouped the crossings by the number of accidents in the 5-
year history. The chart shows the number of accidents in the grouping on the x-axis
The orange square markers show mean predicted accidents with the APS given traits at the
crossings with the specified accident history (shown on the x-axis). The square blue markers
show mean predicted accidents with the new model. The lines below and above the markers
indicate the 10th and 90th percentiles, respectively. The lines also indicate the bounds of the 80
percent confidence interval of the prediction for crossings in the period.
Figure 6-5 Model Comparison, Accident Counts, and Predictions (2014–2018)
Figure 6-5 below (displaying the period 2014–2018) at crossings having three accidents the new
model predicted between 1.6 and 2.0 accidents. The APS predicted 0.5 to 1.4. The new model
better predicted the crashes at crossings for each level of accident risk than the APS.
Figure 6-6 shows the results for the period 2015–2019.
45
Figure 6-5 Model Comparison, Accident Counts, and Predictions (2014–2018)
Figure 6-6. Model Comparison, Accident Counts, and Predictions (2015–2019)
46
6.3 Accident Prediction – Accident Risk for Groups of High-Risk Crossings
The third validation examines the model results (APS and new model) for groupings of high-risk
crossings and shows the results in a chart. The better of the two models will predict accidents at
each grouping of crossings that is closer to the actual accident counts.
Crossings in the estimation sample were ordered by decreasing risk, and then divided into groups
of 50. In the figure below, the x-axis shows groupings 1 to 20 (20 groups of 50 equals total of
1,000). The y-axis shows the actual and predicted crossings by model (new model and APS) for
each grouping.
For each grouping, the new model performed better than the APS. For the top 1,000 high-risk
crossings in 2014–2018 the accident count was 2,578 accidents. The APS predicted 791.3
accidents while the new model predicted 1,518.0 accidents at these 1,000 high-risk crossings.
Figure 6-7. Comparison of Predictions for Riskiest Crossings
6.4 Accident Severity – Model Comparisons
The table below shows the predicted accident severity for all accidents and by each accident type
in the severity estimation sample.
47
Table 6-1. Predicted Severity (Percent of Total) by the New Model and APS
With the new model, the aggregate percentage of accidents of each accident type exactly equaled
the percentages in the sample (as expected). The APS predictions in the aggregate diverged
somewhat from the sample data; for example, APS predicted the percent of fatal accidents to be
half of the actual percentage.
An indicator of the predictive performance of the severity model is to estimate the predicted
percentage of a severity category while only considering those accidents in that category. That
value should well exceed the percentage of a severity category when considering all accidents.
Table 6-1 shows that predicted fatal accidents with the new model increased from a mean of 13.7
percent for all accidents to 18.7 (a 36 percent increase). When considering only accidents that
were actually fatal. The comparable change with APS was 6.9 to 7.2 (a 4 percent increase).
Overall, the new model performed better, with more significant movements in the correct
direction when restricting to accidents of a particular type.
Figure 6-8 and Figure 6-9 below show boxplot charts of predicted accident severities for the new
model and APS.
For Accidents of Severity Type
All Accidents
Fatal
Injury
PDO
New Model
Predictions
Fatal
13.7
18.7
13.1
12.9
Injury
28.4
27.3
28.4
28.0
PDO
58.2
54.0
58.4
59.1
APS
Predictions
Fatal
6.9
7.2
6.9
6.9
Injury
27.3
26.9
27.4
27.4
PDO
65.7
65.9
65.6
65.7
48
Figure 6-8 Distribution of Predicted Accident Severities with the New Model
Figure 6-9. Distribution of Predicted Accident Severities with APS
49
The two charts indicate:
• The mean value from APS for fatal was about half that of the new model, while the
means for injury and PDO accidents were similar.
• The new model had a higher variance for the fatal and PDO categories, with smaller
variance for the injury category. (Standard deviations were 0.08182, 0.02986, and
0.07511 for fatal, injury, and PDO, respectively.)
• APS had a small variance for fatal, somewhat larger for injury, and a bit larger still for
PDO. (Standard deviations were 0.0208, 0.03704, and 0.0455 for fatal, injury, and PDO,
respectively.)
• APS had more and more unbalanced outliers. The injury category skewed downward, and
the PDO category skewed up. The table below shows a summary of the skewness values:
Table 6-2. Summary of Severity Category Skewness by Model
New Model
APS
Fatal
0.3132
0.5734
Injury
0.2782
-0.9523
PDO
0.4544
0.6588
51
7. Conclusion
The preliminary data review indicates that a new model could replace the APS based on the key
drivers of exposure and grade crossing warning device type. In other words, the data show that
risk increases with exposure and more protective warning device type reduces risk.
Other findings include:
• There is justification for a single model with category of warning device type as a
variable rather than separate models for each of the three warning device type categories.
• Grade crossings that are public, not closed, not grade separated, and that have non-
missing, non-erroneous values for exposure and warning device type, number 105,377
nationally. In the period 2014–2018 there were 8,467 accidents at these grade crossings.
• An aggregate analysis of these grade crossings showed that relative to a passive crossing,
a lights crossing had 73 percent less risk per exposure. A gated crossing had 63 percent
less risk per exposure than a lights crossing.
• The findings of the above analysis indicate a functional form with exposure, warning
device type, and other grade crossing characteristics.
• The analysis indicates additional variables that are likely to explain accident occurrence:
grade crossing is in rural or urban area, maximum timetable speed, and grade crossing
surface types.
• Model estimation using ZINB regression yielded parameters of the expected sign and
magnitude, and had strong statistical significance.
• Including the number of daily trains and the AADT at the crossing, which are
components of the exposure metric, improved the regression results as indicated by the
AIC.
• The EB method accounts for accident history while correcting for “regression to the
mean” bias. Adjusted results with EB produced predictions that more closely track the
actual counts than did the APS adjustment process for accident history.
• The new model severity component determined the probabilities that an accident would
be of one of three severity types: fatal, injury or PDO.
• The severity component of the new model was derived using multinomial logistic
regression on the accidents in the 6-year period 2014–2019.
• In the period there were 11,131 accidents at public crossings. Of these, the crossings
where these accidents occurred had non-missing, non-erroneous data for 9,870 grade
crossings. The accidents at these crossings were included in the severity model
estimation.
• The multinomial logistic regression showed that the best results were obtained with
explanatory variables: rural or urban, maximum time table speed, number of daily trains,
and whether a crossing has a lights warning device.
52
• Validations showed that the new model performed better than the APS by multiple
measures.
53
8. References
1. Farr, E.H. (1987). Summary of the DOT Rail-Highway Crossing Resource Allocation
Procedure – Revisited [DOT/FAR/OS-87/05].
2. Hauer, E. (2015). The Art of Regression Modeling in Road Safety. Springer
3. Hauer, E. (2001). Overdispersion in modeling accidents on road sections and in
Empirical Bayes estimation. Accident Analysis and Prevention 33, 799–808.
4. Hauer, E. (2004). Statistical Safety Modeling. Transportation Research Record 1897.
Washington, DC: National Academies Press, 81–87.
5. American Association of State Highway and Transportation Officials. (2010). Highway
Safety Manual, 1st edition.
6. Brod, D., Weisbrod, G., Moses, S.B., Gillen, D., & Martland, C.D. (2013)
Comprehensive Costs of Highway-Rail Grade Crossing Crashes. NCHRP Report #755.
Transportation Research Board.
7. Federal Railroad Administration. (2019). GradeDec.Net Reference Manual.
8. Mironer, M., Coltman, M. & McCown, R. (2000). Assessment of Risks for High-Speed
Rail Grade Crossings on the Empire Corridor [DOT-Volpe-FRA-00-03]. Washington,
DC: U.S. Department of Transportation.
9. Elvik, R. & Vaa, T. (2014). Handbook of Road Safety Measures. Oxford, UK: Elsevier.
10. R Core Team. (2020). R: A language and environment for statistical computing. R
Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/
54
Appendix A. Interpreting Regression Outputs
A regression analysis is a set of statistical processes for estimating the relationships between a
dependent variable and one or more independent variables. A dataset contains a number of
observations for each variable.
The independent variable is often called the left-hand side (LHS) variable because it is written to
the left of the equals sign. The dependent variables (also called explanatories) are the right-hand
side (RHS) variables.
In regression analysis, the analyst develops a model linking the LHS with RHS variables and
“runs” a regression. A statistical program examines the dataset and finds the values of model
coefficients that meet optimization criteria.
35
The regression output table contains general statistics along with coefficient estimates and
statistics.
The following describes the columns in the regression output table that relate to the coefficient
estimates:
Column Name
Column Description
Variable
Each row contains the name of a model variable. If the model has a constant,
the row will usually say “constant” or “intercept,” depending upon the
software used.
Estimate
The estimate of the variable model coefficient (in this report, coefficients are
subscripted and shown in model equations as lowercase Greek letters β
(beta) and γ (gamma)
Std. Error
The standard deviation of the coefficient estimate
z-value
This is the estimate divided by the standard error.
Pr(>|z|)
(p-value)
In statistical significance testing, the p-value is the largest probability of
obtaining test results at least as extreme as the results actually observed,
under the assumption that the null hypothesis is correct (i.e., assuming the
coefficient is actually 0). This is equivalent to the probability of falsely
rejecting the null hypothesis (also called a Type I error).
35
The two broad classes of regression techniques are least squares (LS) and maximum likelihood estimation (MLE).
With LS, the regression minimizes the sum of squared residuals (“residuals” are the differences between the LHS
values and the “fitted” calculated values of the model). With MLE, the regression seeks the point of maximum of a
likelihood function that is constructed from all the data observations. The datasets under consideration will usually
determine which technique is most appropriate.
55
Column Name
Column Description
Confidence
Level
This is the confidence level of the parameter estimate. It is one minus the p-
value (i.e., if the p-value is .01, then the confidence level is 0.99 – or, 99.0
percent).
The general statistics include descriptive statistics of the regression and its residuals. This study
examines the AIC, which enables model quality comparison and whose value is least for the
better model specification with the given set of data.
56
Appendix B. Application of the New Model
The APS enables risk ranking of grade crossings (in a corridor or region). However, it cannot
inform when two grade crossings with similar risk scores (e.g., predicted annual accidents)
should be treated the same or differently. The new model provides descriptive statistics of the
population of grade crossings, and these can be used to determine if scores are close enough to
warrant same or different treatment.
36
For example, suppose we have two grade crossings A and B, and the new model estimates they
have predicted annual accidents of 0.21 and 0.26, respectively. From the analysis of data in
developing the model, we know that:
1. Mean value of 5-year accidents for the population of grade crossings is E{μ} = 0.08319
2. The variance of 5-year accidents for the population of grade crossings is V{k} =
0.1220627.
3. The standard deviation of 5-year accidents for the population of grade crossings is:
Since the standard deviation is for 5-year accidents, divide by 5 for the standard deviation of
predicted annual accidents:
Crossing A has predicted annual accidents of 0.21, then adding the standard deviation to the
value 0.21 + 0.03945124 = 0.24945124. Crossing B has predicted annual accidents of 0.26,
which is greater than the previous value and outside a band of one standard deviation from the
mean value of predicted annual accidents of A. We would conclude that the predicted annual
accidents of the two crossings differ significantly and, therefore, the two warrant different
treatment based on the new model.
36
Following Hauer (2015) Chapter 2, “A Safety Performance Function for Real Populations.”
57
Abbreviations and Acronyms
AADT
Average Annual Daily Traffic
AASHTO
American Association of State Highway and Transportation Officials
AIC
Akaike Information Criterion (a measure of the relative quality of a model for a given
set of data)
APS
Accident Prediction and Severity
CMF
Crash Modification Factor (a safety countermeasure’s ability to reduce crashes and
crash severity)
CFR
Code of Federal Regulations
CWT
Constant Warning Time (device at grade crossings with active warning devices that
ensures the time between initial warning and a train’s arrival at the roadway is
constant, regardless of the speed of the train)
DOT
Department of Transportation
EB
Empirical Bayes (procedure for statistical inference in which prior distributions are
derived from data)
FRA
Federal Railroad Administration
GCIS
Grade Crossing Inventory System
GX
Grade crossing (used in this document’s figures)
HSR
High-Speed Rail
MLE
Maximum Likelihood Estimation (a class of model estimation procedures)
MNL
Multinomial Logistic (a regression analysis method)
NB
Negative Binomial (a probability distribution)
PDO
Property Damage Only (a severity type of train-highway vehicle accident at a grade
crossing)
SPF
Safety Performance Function (a function for evaluating the safety of a transportation
facility, or population of facilities, from a set of facility traits and accident history)
TRB
Transportation Research Board
Volpe
Volpe National Transportation Systems Center
ZINB
Zero-Inflated Negative Binomial (a regression analysis method)
