Content uploaded by Robin S. Engel
Author content
All content in this area was uploaded by Robin S. Engel on Jun 26, 2020
Content may be subject to copyright.
146 ANNALS, AAPSS, 687, January 2020
DOI: 10.1177/0002716219889328
Moving beyond
“Best Practice”:
Experiences in
Police Reform
and a Call for
Evidence to
Reduce Officer-
Involved
Shootings
By
ROBIN S. ENGEL,
HANNAH D. MMANUS,
and
GABRIELLE T. ISAZA
889328ANN The Annals Of The American AcademyMoving beyond “best practice”
research-article2019
In post-Ferguson America, police departments are
being challenged to implement evidence-based changes
in policies and training to reduce fatal police-citizen
encounters. Of the litany of recommendations believed
to reduce police shootings, five have garnered wide-
spread support: body-worn cameras, de-escalation
training, implicit bias training, early intervention sys-
tems, and civilian oversight. These highly endorsed
interventions, however, are not supported by a strong
body of empirical evidence that demonstrates their
effectiveness. Guided by the available research on
evidence-based policing and informed by the firsthand
experience of one of the authors in implementing
departmental reforms that followed the fatal shooting
of a civilian by an officer, this article highlights promis-
ing reform strategies and opportunities to build the
evidence base for effective use-of-force reforms. We
call upon police executives to engage in evidence-based
policing by scientifically testing interventions, and we
call on academics to engage in rapid research responses
for critical issues in policing.
Keywords: evidence-based policing; police shootings;
use of force; police reform
On July 19, 2015, Samuel DuBose was
stopped by University of Cincinnati Police
Division (UCPD) Officer Raymond Tensing
approximately 0.5 mile south of the University
of Cincinnati (UC) campus for a minor equip-
ment violation (missing front license plate, a
traffic violation in the State of Ohio). After a
brief exchange, DuBose, an unarmed 43-year-
old black male, was shot and killed by Officer
Robin S. Engel is director of the IACP/UC Center for
Police Research and Policy and a professor of criminal
justice at the University of Cincinnati. She recently
served as vice president for safety and reform at the
University of Cincinnati. Her work includes establish-
ing academic-practitioner partnerships and promoting
evidence-based practices in policing, with empirical
assessments of police behavior and evaluations of crime
reduction strategies.
Correspondence: robin.engel@uc.edu
MOVING BEYOND “BEST PRACTICE” 147
Tensing, a 25-year-old white male. The incident was captured on Tensing’s depart-
ment-issued body-worn camera (BWC), but still the circumstances surrounding
the shooting were widely debated within the Cincinnati community, sparking
protests, independent investigations, civil litigation, and criminal trials. An inde-
pendent consulting firm commissioned by the university to perform an external
review of the incident found that the shooting was “entirely preventable” and
concluded it was the officer’s “critical errors in judgment that created an elevated
risk of a serious or fatal bodily injury” (Kroll 2015, 46). Ten days after the incident,
Tensing was indicted by the Hamilton County Prosecutor’s Office for murder. His
first criminal trial concluded in November 2015, resulting in a hung jury, while a
second trial in June 2017 garnered the same outcome. Concerns regarding racial
bias overshadowed the criminal proceedings and flowed into the public discourse.
In July 2017, the county prosecutor announced a third trial would not be con-
ducted, and the charges against the defendant were dropped.1
This tragic incident and its aftermath sent shockwaves through the Cincinnati
community, a city with a troubled history of racial tension and damaged police-
community relations (Eck and Rothman 2006); however, it is far from a unique
story in American policing. The fatal DuBose-Tensing encounter in summer
2015 occurred amid a spate of other police-involved deaths of unarmed black
males.2 Described by Sherman (2018) as the “Second Great Awakening,” the
reaction to these high-profile incidents involving the killing of unarmed citizens
by American police has included public protests, civil unrest, widespread media
attention, and heightened public scrutiny of police.
Notably, the growth in public angst regarding police bias and use of force has
been accompanied by rising concerns regarding police officers’ safety. Violent
incidents, including the murder of five police officers and wounding of nine oth-
ers in Dallas, Texas, in July 2016, followed by an ambush attack killing three
officers and wounding three more in Baton Rouge, Louisiana, led to national
reflection and discussions regarding officer safety. Although overall assaults, inju-
ries, and death of officers in the line of duty have continually decreased over the
Hannah D. McManus is a research associate for the IACP/UC Center for Police Research and
Policy and a doctoral candidate in the School of Criminal Justice at the University of
Cincinnati. Her recent work includes the evaluation of police training, the study of public
perceptions of police, and the promotion of evidence-based practice to enhance police officer
safety and wellness.
Gabrielle T. Isaza is a research associate for the IACP/UC Center for Police Research and
Policy and a doctoral candidate in the School of Criminal Justice at the University of
Cincinnati. Her areas of research include police effectiveness; survey design; and evaluations
of police training programs, including de-escalation and implicit bias.
NOTE: Ideas and concepts presented within this article were generated in coordination with
other research conducted for the IACP / UC Center for Police Research and Policy, funded by
Arnold Ventures. The information and commentary within this article, however, are from the
authors and do not necessarily represent the official positions or policies of Arnold Ventures or
the International Association of Chiefs of Police.
148 THE ANNALS OF THE AMERICAN ACADEMY
last two decades, the number of officers injured or killed through ambush-style
attacks has increased in the last 10 years, reaching a peak in 2016 (Federal
Bureau of Investigation [FBI] 2019).
Acknowledging the dangers to both citizens and officers embedded in this
police-community relations crisis, efforts have intensified to identify “solutions” to
reduce the frequency and severity of violent encounters between police and the
public. Expert panels, such as the President’s Task Force on 21st Century Policing,
have convened to explore best practices in policing that “can promote effective
crime reduction while building public trust” (2015, 1). Simultaneously, American
citizens have organized to demand police reform, including changes regarding
officer use of force. These demands have emerged during a time when the applica-
tion of evidence-based policy and practice by law enforcement is increasingly
expected yet infrequently followed. Indeed, most “best practices” for police reform
are not based on a strong body of empirical evidence (see Lum etal. 2016).
In an effort to better inform the discussion regarding “what works” to reduce
the frequency and severity of potentially violent police-citizen encounters, this
article considers the evidence regarding the effectiveness of routinely recom-
mended police reforms, including (1) deployment of BWCs; (2) changes to use-
of-force policies and training that emphasize de-escalation; (3) implicit bias
training for officers; (4) early intervention (warning) systems; and (5) citizen
oversight (review) boards. Although each of these interventions has been recom-
mended by the President’s Task Force or other policing experts, careful consid-
eration of the evidence available demonstrates a patchwork of studies that
collectively provide little confidence that these reforms will directly impact
police shootings. Guided by the available research on police use of force and
evidence-based practices and informed by the experience of one of the authors
(Engel) in implementing reforms within the UCPD, this article highlights prom-
ising strategies, along with the remaining gaps in knowledge and opportunities to
build the evidence base for effective use-of-force reform.
We recognize that in the absence of evidence, police executives must still
move forward with recommended best practices. However, we further contend
that it is an ethical duty of police officials to combine the implementation of
innovative approaches with continuous review and testing to identify ineffective
practices and unintended consequences. We articulate the urgent need for
researchers to work collaboratively with police executives to generate and dis-
seminate knowledge regarding these and other police reform efforts, generating
a rapid research response (also see Engel, McManus, and Herold, forthcoming).
Lacking this research, it cannot be stated with confidence that the proposed use-
of-force reforms can prevent future fatal police-citizen encounters. As such, it is
of the utmost importance that our academic and practitioner colleagues, along
with local, state, and federal agencies and organizations and the philanthropic
community, prioritize research designed to advance rigorous scientific testing of
police reform efforts at the local level, including the reduction of fatal police
shootings. The safety of our nation’s police officers, and members of the public
they serve, hinges on this collective work.
MOVING BEYOND “BEST PRACTICE” 149
Police Use of Force
More than four decades ago, policing scholar Egon Bittner (1974) argued that
police are defined by their ability—and duty—to use force. It is the public’s
expectation that police use force when necessary that separates policing from all
other occupations. It is this ability to use force that shapes all other aspects of
police practice. Given this distinction, it is surprising that we still know relatively
little about how, when, and under what circumstances police officers use force
(Engel and Serpas 2017; Garner etal. 2018). As of this writing, the best available
national databases capturing fatal police-citizen encounters are not compiled by
government officials, but by reporters from The Guardian and the Washington
Post (Sherman 2018; Zimring 2017). Most notable of the findings from these data
sources are as follows:
1) Over half of all fatal police shootings occur in small jurisdictions with
fewer than 50,000 residents, and only one-third occur in cities with popu-
lations greater than 250,000 residents (Sherman 2018).
2) There was no firearm present in 44 percent of fatal police shootings
(Zimring 2017).
3) When compared to the national population, African Americans are 2.3
times more likely to be killed by police than whites (Zimring 2017).
Combined, these quantitative findings support the narratives generated
through the sharing and resharing of individual images and videos of police- citizen
encounters that have resulted in tragedy. What these figures do not provide, how-
ever, is an understanding of the details regarding the individual, situational, envi-
ronmental, and organizational factors likely to impact officer’s decision-making
during encounters with citizens. Despite this lack of information, Zimring (2017)
boldly predicts that fatal interactions with police could be cut in half without com-
promising officer safety. He bases this prediction, in part, on the large estimated
percentage of fatal encounters with police where the suspect does not have a
firearm. Referring to these fatal encounters as “unnecessary killings,” Zimring and
others argue that systematic changes in policing, implemented at the national
level, could reduce citizen injury and death (Sherman 2018).
Within a nation that claims approximately eighteen thousand distinct police
organizations with little consistency in training, policies, and practices (Reaves
2011), the prospect of implementing significant police reform systematically is
daunting. For police shootings specifically, it is even more challenging to impact
individual officer decision-making during potentially violent encounters. Sherman
(2018) argues for employing a system-accident framework to drive appropriate
policies and research necessary to reduce fatal encounters with police. Similarly,
David Klinger (2005, this volume) has renewed his proposal to use Perrow’s
(1984) normal accidents framework to guide understanding of violent police-
citizen encounters. Further, Zimring (this volume) argues that creating a statisti-
cal and research branch within the federal government is likely the only way to
implement needed police reforms designed to reduce police killings.
150 THE ANNALS OF THE AMERICAN ACADEMY
In contrast to the apparent logic of these recommendations, our premise is
that the decentralized nature of policing in the United States will make many
efforts to reduce officer shootings at the federal and state levels ineffective. Even
when mandated or otherwise influenced by oversight at the state or federal lev-
els, meaningful changes in police practice must be implemented and embraced
at the local level. Citizens intuitively recognize this. Although there is often pres-
sure placed on state and federal officials to “do something” about police use of
force, the majority of citizen demands involve changes directly targeted at indi-
vidual police agencies. Therefore, much of the movement in police reform is
occurring at the local level, agency by agency. In this context, this article consid-
ers the role of evidence-based policing (EBP) in reducing officer shootings at the
local level by presenting the most common calls for change, reviewing the evi-
dence-base supporting these recommendations, and considering the experience
of the University of Cincinnati Police Division in the implementation of evi-
dence-based police use-of-force reform.
The Role of Evidence-Based Policing in Reform
At its core, EBP is a movement encouraging the use of research to guide practice
and evaluate practitioners (Sherman 1998). Seeking to progress beyond the use of
anecdotal or experiential evidence alone, EBP suggests that police decision-making
on “what works” to address specific problems should be guided by objective facts
produced from scientific research. Further, EBP encourages police to embed sci-
ence within their respective agencies through the evaluation of current policies and
practices (Sherman 1998, 2013). Police agencies that use and generate research to
guide strategic and tactical decision-making should experience greater success in
identifying cost-effective approaches (Weisburd and Neyroud 2011).
The movement toward EBP has been a gradual one, gaining significant trac-
tion in the last 20 years due to the increased availability and application of
research and innovative technologies in policing (Lum, Koper, and Telep 2011;
Sherman 2013). Many scholars, practitioners, and policy-makers now adopt the
view that EBP is critical for successful reform across a number of outcomes,
including reducing police use of lethal force (Telep 2016). However, our review
of existing evidence on five commonly recommended police reforms provides a
limited picture on “what works” in addressing police use of force, particularly
lethal force, at the local level (also see Lum etal. 2016).
Regardless of the availability of scientific evidence to support critical decisions,
police administrators are responsible for crisis management. Furthermore, police
administrators are often pressed through public and political demands to make
significant changes to policies and training immediately following critical incidents,
leading to the perception that police accountability is best measured by the speed
of response. Police executives are acutely aware that their jobs often hinge on how
they respond to community pressure within the first few days following a critical
incident. Within this time constraint, the careful planning of an evidence-based
MOVING BEYOND “BEST PRACTICE” 151
response is often unrealistic. Therefore, although the clamor for police reform has
been pervasive, it also appears that these reform efforts are largely divorced from
the expectation that these interventions be evidence-based.
Reform within the University of Cincinnati Police Division
The UCPD is a fully empowered law enforcement agency with a complement of
seventy-two sworn officers that provides all public safety services for the UC com-
munity (seventy thousand students and employees), including primary jurisdiction
on all university-owned and operated buildings. Based on a Memorandum of
Understanding with the City of Cincinnati, UCPD also patrols and conducts police
activities in city neighborhoods in the one-mile corridor surrounding the UC
uptown campus. The UCPD has approximately five hundred or fewer Part I crime
incidents reported on campus each year, which are largely driven by reports of
theft/larceny (Isaza et al. 2017).3 In general, the UCPD does not record many
instances of police use of force in any given year. Between January 2017 and
December 2018, for example, the agency reported only four use-of-force incidents,
one of which included a display of force only (Exiger 2019). Overall, the UCPD is
representative of many midsize police agencies that have few critical incidents.
However, the UCPD’s experience with the fatal DuBose-Tensing encounter dem-
onstrates the magnitude of a single police-citizen interaction gone terribly wrong.
In the aftermath of the shooting, immediate changes in police executive lead-
ership were made at the university. The lead author of this article (Engel) was
appointed to the newly created position of vice president for safety and reform
to develop and implement an immediate police reform effort. Two weeks later,
she added executive leadership positions to her team, including a director of
public safety (with operational oversight of the UCPD) and director of police
community relations. She led her team in the design and implementation of over
three years of systematic reforms, including top-to-bottom organizational
changes in policies, procedures, training, and accountability systems, along with
the completion of voluntary external monitorship.4 Although the reform efforts
implemented by this team were intended to be evidence-based, the immediacy
of implementation, as well as the general lack of evidence available to inform
organizational change (as articulated below), characterizes these efforts more
appropriately as “best practices.” The selection and implementation of reform
strategies and the evidence-base surrounding the effectiveness of these efforts is
discussed in greater detail below.
The Evidence Base for Agency-Level Solutions
to Reduce Police Shootings
Following the shooting of Samuel DuBose, the political and public demands for
change in Cincinnati were highly predictable, echoing the approaches being
152 THE ANNALS OF THE AMERICAN ACADEMY
requested and implemented across the country to reduce fatal police encounters
and rebuild community trust. First, although UCPD officers were already
equipped with BWCs, calls for more transparency within the police department
reverberated. Second, UCPD’s use-of-force policy and training were heavily
scrutinized and, ultimately, dramatically changed to incorporate de-escalation
tactics. Third, direct calls for implicit bias training for officers were made and
answered with the rapid delivery of the leading curriculum in the field.
Additionally, an early intervention system—designed to identify potentially prob-
lematic officers prior to a critical incident—was hastily implemented. Finally, the
long and challenging task of rebuilding public trust through increased commu-
nity engagement and citizen oversight began. Each of these approaches—(1)
BWCs, (2) de-escalation training, (3) implicit bias training, (4) early intervention
systems, and (5) civilian oversight—are the most typical and frequently called for
reform efforts after a controversial shooting. As such, many might be surprised
to learn that the body of evidence supporting the effectiveness of most of these
initiatives is generally thin and, in some cases, nearly nonexistent. The remaining
sections of this article outline the research evidence for each of the five initiatives
outlined above, highlighting the UCPD’s experience in the implementation of
use-of-force reforms. Implications for evidence-based use-of-force reform are
discussed, and critical next steps for effective change are highlighted.
Body-worn cameras
Calls for police to adopt BWCs have proliferated in recent years across the
United States. A recent study using a nationally representative sample of police
agencies estimated that one-third of American law enforcement agencies cur-
rently deploy BWCs to some or all of their officers, and an additional 50 percent
of agencies have immediate plans to deploy them (Police Executive Research
Forum 2018). These devices are theorized to impact officer behavior through
their deterrent effect on unconstitutional interactions with the public, particu-
larly for excessive use of force (Ariel, Farrar, and Sutherland 2015). There is an
abundance of research suggesting that when people know they are being
observed, they alter their behavior to better adhere to social norms (Diener and
Wallbom 1976). Theory follows, then, that when officers know they are being
recorded, they will be more likely to behave in a constitutional manner. Despite
this assumption, a majority (over 90 percent) of surveyed police agencies indicate
that the primary reason for their adoption of BWCs is to improve community
trust by promoting accountability, transparency, and legitimacy (Police Executive
Research Forum 2018), not necessarily to change police behavior.
Research on BWCs has increased exponentially in recent years, spurred on, in
part, by significant interest and funding from the philanthropic community. This
research has examined multiple outcomes with varying units of analysis, includ-
ing the attitudes and behavior of officers and citizens, police investigations, and
organizational effects (Lum et al. 2019). A recent comprehensive review of
research on BWCs, conducted by Lum and colleagues (2019), identified seventy
studies, sixteen of which specifically measured officers’ reported use of force as
MOVING BEYOND “BEST PRACTICE” 153
a dependent variable. These studies report inconsistent findings concerning the
impact of BWCs on officers’ use of force. Specifically, multiple experimental
studies indicate that officers wearing cameras used significantly less force com-
pared to those who did not wear cameras. Yet several other studies using experi-
mental and quasi-experimental designs found no statistical differences among
officers wearing cameras and those who do not.
One particular study provides important insight regarding these mixed find-
ings. Ariel and colleagues (2016) conducted a subgroup analysis of ten rand-
omized control trials examining BWCs to better understand why BWCs did not
statistically impact officers’ use of force in a single direction. They conclude that
officers are more likely to use higher levels of force when turning BWCs on dur-
ing a citizen encounter is left to their discretion. While this finding offers an
interesting nuance related to the impact of BWCs on use of force, few research
studies have tracked the activation of cameras. Therefore, it is unclear whether
these findings would be replicated elsewhere (Lum etal. 2019).
Collectively, this research suggests that simply outfitting officers with BWCs is
insufficient to significantly reduce police use of force. Rather, it is likely that
deployment of the cameras needs to be coupled with strong policies and super-
visory oversight to achieve maximum benefits in reducing use of force. It also
remains unclear what impact, if any, BWCs could expect to have on reducing fatal
police encounters, which are significantly less likely than other uses of force.
Indeed, in the case of the UCPD, the introduction of BWCs did not decrease
officers’ use of force or the likelihood of a fatal police-citizen encounter. It did,
however, capture video of the fatal police-citizen encounter (Noble 2015). Rather
than bring clarity to that incident, the video footage demonstrated that citizens
(and competing experts) could watch the same images yet have significantly dif-
ferent interpretations regarding the legitimacy and legality of the use of force.
Indeed, Tensing’s two separate murder trials resulted in hung jury decisions
where the primary piece of evidence was the body camera footage (Grasha 2017).
Use-of-force/de-escalation policies and training
A primary recommendation for police reform has been the incorporation of
use-of-force/de-escalation policies and training within law enforcement. Many
law enforcement leaders supporting the adoption of de-escalation training
espouse that these techniques can help to resolve police-citizen encounters with
less frequent and severe uses of force and, thereby, also increase officer safety.
Notably, de-escalation tactics and training received an endorsement from the
President’s Task Force; and more recently, the National Consensus Policy on Use
of Force (International Association of Chiefs of Police 2017, 3) encouraged offic-
ers to use de-escalation techniques “whenever possible or appropriate before
resorting to force.”
De-escalation policies and training are not without critics, however, with a
growing number voicing serious concerns about perceived risks to officer safety
(Blake 2017; Landers 2017; Williams 2015). Several tactics common in
de-escalation training run counter to more traditional operational responses
154 THE ANNALS OF THE AMERICAN ACADEMY
in policing. Critics argue that changes in the traditional approach could increase
the risk of officer injury. In the absence of evidence to the contrary, Landers
(2017) argues that “by sending officers to de-escalation training courses, chiefs
and sheriffs have risked these men and women becoming hesitant about using
force”; and furthermore that “while [de-escalation] concepts are practical and
effective in some situations, they are useless and even dangerous in others.”
Concerns regarding officer safety run deep within police organizational cultures,
and some trainers have resorted to avoiding the use of the word de-escalation
altogether in training sessions, rather describing these techniques as opportuni-
ties to defuse situations.
In many ways, these concerns are amplified by the ambiguity of de-escalation
policies and training. Specifically, there is no uniformly accepted definition of
de-escalation within the policing field. Most recommendations, including from
the President’s Task Force, neglect to provide a specific definition, or even a
general description, of de-escalation. As such, de-escalation has become a catch-
all of sorts, symbolizing a different but perhaps more progressive policing
approach for handling potential use-of-force encounters. A systematic review of
literature across disciplines finds de-escalation to typically refer to a process or
tactics used to prevent, reduce, or manage behaviors associated with conflict—
including verbal or physical agitation, aggression, violence, or similar behaviors—
during an interaction between two or more individuals (for review, see Engel,
McManus, and Herold, forthcoming).
Similar to most other police training curricula, de-escalation training has not
been subjected to rigorous scientific testing (National Research Council 2004;
Lum at el. 2016; Skogan, Van Craen, and Hennessy 2015). As a result, little is
known about the development, delivery, and impact of police de-escalation train-
ing. To assess this gap in knowledge, Engel, McManus, and Herold (forthcoming)
conducted a multidisciplinary systematic review, identifying only a limited num-
ber of de-escalation training evaluations appearing across professions (n = 64),
and no evaluations within the realm of policing or criminal justice were identified.
The findings from studies outside of policing provide some insight on the attitudi-
nal (i.e., self-reported) and behavioral effects that de-escalation training could
have on officer use-of-force outcomes. Specifically, studies examining self-
reported outcomes suggest that de-escalation training in other professions has led
to favorable effects on the attitudes, perceptions, and self-reported experiences
and behaviors of trained individuals. The impact of de-escalation training on
behavioral outcomes, however, has not been established. Although the majority of
the studies report favorable effects of de-escalation training (e.g., reduction in
number and severity of violent incidents, increased application in use of de-esca-
lation techniques), there were also findings that suggest de-escalation training has
no effect and, in some cases, unfavorable effects on use-of-force outcomes.
Confidence in the validity and generalizability of these findings is threatened
by several factors, including the (1) substantial variation across de-escalation
training programs under examination; (2) reliance on pretest/posttest research
designs with no comparison groups; (3) heavy focus on self-reported, rather
than behavioral, outcomes; and (4) emphasis on short-term outcomes
MOVING BEYOND “BEST PRACTICE” 155
(Engel etal., forthcoming). As a result, recommendations for de-escalation, as
well as larger conversations on the safety and well-being of police officers and the
individuals they encounter, rely heavily on anecdotal evidence and untested
propositions about best practice. Nevertheless, a majority of police agencies in
large cities across the country are currently training officers in de-escalation tech-
niques, as this is one of the most common demands from community activists and
political officials (CBS News 2019). Although more rigorous research studies of
de-escalation training curriculums for police are under way (e.g., Engel et al.
2019; McLean etal., this volume; White and Pooley 2018), the current state of
evidence leaves many questions related to the effectiveness of de-escalation poli-
cies and training unanswered.
Based on the limited evidence from other disciplines, anecdotal accounts,
reviews of practice in other agencies, experts’ advice, and our best judgments, the
UCPD executive team engaged in a comprehensive and innovative overhaul of
all use-of-force policies, procedures, and training. The underlying principle of
the sanctity of life—respecting the value of every human life—along with the
implementation of a robust training and accountability structure necessary to
support the use of de-escalation tactics were the primary focus of the UCPD
team (Police Executive Research Forum 2016). While the scientific evidence
regarding the likelihood of reducing use of force based on these changes was
quite limited, a robust accountability structure was embedded, and several evalu-
ation opportunities to build our own body of evidence were undertaken. For
example, when officers were trained in use of force and de-escalation, a series of
surveys (pre, post, and follow-up) were created and administered to officers to
identify changes in officers’ attitudes, knowledge, and self-reported behaviors.
These survey instruments are now being used by other agencies looking to exam-
ine the impact of similar de-escalation trainings (see Engel etal. 2019). Coupled
with strong research designs (e.g., randomized controlled trials) for training
delivery, this evidence base will quickly grow, and soon critical information
regarding the impact of de-escalation training on officer use of force will be
available.
Implicit bias training
Racial and ethnic disparities in policing practices—including vehicle and
pedestrian stops, citations, searches, arrests, and use of force—have been noted
for decades. The reasons for these disparities, however, are multifaceted, and
researchers have traditionally struggled to disentangle critical factors to deter-
mine if disparities are the result of discrimination (Engel and Swartz 2014).
Nevertheless, based on the supposition that differences in policing outcomes are
the result of individual police bias or discrimination, the operational response to
reduce racial disparities has consistently involved the provision of additional
training for officers. For example, racial sensitivity training was adopted in the
1980s and 1990s, followed by antiracial profiling training in the 2000s, and bias-
free policing in the early 2010s. The most recent training proposed to address the
156 THE ANNALS OF THE AMERICAN ACADEMY
problems associated with racial/ethnic disparities in policing—including dispari-
ties in use of force—is implicit bias training (Fridell 2017; Nix etal. 2017).
Implicit bias refers to an unconscious prejudice that people may develop due to
differential life experiences (Devine 1989). In contrast to explicit bias, which is akin
to traditional “racism,” all humans are subject to some form of unconscious bias
that may impact perceptions and behaviors, resulting in discriminatory decision-
making (Dovidio, Kawakami, and Gaertner 2002). Due to the high rates of minor-
ity citizen contact with police, the impact of implicit bias in policing could be
especially profound. Indeed, the existence of implicit bias has been demonstrated
in research using police as subjects (Payne 2001).
The President’s Task Force (2015) specifically acknowledged implicit bias and
its role in producing disparities in outcomes, calling for police training to reduce
the impact of such biases. Based upon a large body of psychological research
demonstrating that biases can be managed (see Monteith, Arthur, and Flynn
2010; Pettigrew and Tropp 2005), interventions typically begin by educating indi-
viduals regarding the existence of implicit bias and, subsequently, teaching vari-
ous skills to reduce and manage these biases (Fridell 2017). A meta-analysis of
nearly five hundred studies identified that implicit bias is malleable (albeit with
relatively weak effects). However, changes in implicit bias measures may not
necessarily translate into changes to explicit bias or in behavior (Forscher etal.
2018; also see Atewologun, Cornish, and Tresh 2018). Specifically, Forscher and
colleagues (2018) conclude that although interventions to change implicit bias
may not consistently change behavior, some procedures for change are more
effective than others. Others conclude that an important key is to not only raise
awareness of implicit biases but to complement trainings with strategies to
reduce and manage biases (Atewologun, Cornish, and Tresh 2018).
Of concern, some research has suggested that there may be unintended con-
sequences of implicit bias training, including increases in the expression of bias.
For example, research has identified a “rebound effect,” which occurs when
individuals attempt to suppress stereotypical thoughts. The act of suppression
may cause the stereotypical thoughts to later reappear with even greater insist-
ence had they not been suppressed (Macrae et al. 1994). Other research has
suggested that trainings to increase awareness of stereotypes may also normalize
stereotyping, thereby undermining the desired effect of the training (Duguid and
Thomas-Hunt 2015). It is unknown how these potential negative consequences
may impact implicit bias training for law enforcement.
Collectively, this research presents a mixed picture regarding the impact of
implicit bias training on changes in behavior. And despite the accumulating evi-
dence from other disciplines, there remains a critical need to examine implicit
bias training for police, as reviews have found no experimental evaluations of
implicit bias training for police officers that have been completed (Mitchell and
James 2018). In short, agencies across the country are training officers in implicit
bias without evidence that this training is effective. For most police executives,
however, the cost-benefit analysis for providing implicit bias training may hinge
on whether they perceive that their agency will receive a boost in community
trust and improve police-community relations, rather than any expected changes
MOVING BEYOND “BEST PRACTICE” 157
in police officers’ attitudes and behaviors due to the training. For example,
despite being unaware of its likely impact or effectiveness, within three months
of the UCPD officer-involved shooting, all UCPD officers were trained in
implicit bias, and a special session was conducted with community members in
an attempt to both educate and rebuild trust (Jones 2015). The impact of this
training was never formally assessed within the UCPD. However, recognizing the
lack of evidence regarding the impact of implicit bias training, the lead author of
this article secured funding and is overseeing a larger research team conducting
a randomized control trial evaluation of implicit bias training with the New York
City Police Department. Findings are scheduled for release in spring 2020.
Given the popularity of implicit bias training within police agencies across the
United States, replication studies could and should be generated.
Early intervention systems
Another frequently mentioned opportunity for changing officer behavior is
the implementation of early intervention systems (EIS), also referred to as early
warning systems. EIS are data-driven administrative systems used to detect
officer misconduct. The goal of EIS is to identify officers whose performance
includes a pattern of undesirable behavior and to implement corrective interven-
tions quickly. By monitoring indicators of problematic behavior (e.g., citizen
complaints, use of force, high-speed pursuits), police administrators may be able
to identify officers at risk of future misconduct and intervene before that miscon-
duct occurs (Worden etal. 2013). EIS generally consist of four components:
performance indicators, procedures for officer identification, intervention, and
postintervention monitoring. Typically framed as a nonpunitive approach, these
systems often operate external to a law enforcement agency’s disciplinary pro-
cesses, providing the opportunity for officer retraining, counseling, or other
interventions (Worden etal. 2018).
The use of EIS spread rapidly across the landscape of American law enforce-
ment in the 1990s and 2000s. The proliferation of these systems was supported,
in particular, by the inclusion of recommendations for EIS in consent decrees
developed by the U.S. Department of Justice as part of their investigations of the
“pattern or practice” of citizen rights violations within police agencies (Walker
2003). By 2007, 40 percent of agencies serving populations of fifty thousand or
more reported having implemented an EIS within their agency (Worden etal.
2018). Among many, EIS are now considered a “best practice” for police account-
ability, particularly regarding officer use of force (Harmon 2009; Walker 2003).
The use of EIS is not without criticism, however. The ability of systems to accu-
rately identify problematic officers is a primary concern. Though law enforce-
ment agencies have begun to incorporate secondary screening measures (e.g.,
supervisory review) to bolster the review and selection of “problem” officers, the
process for identification remains an imperfect science (Worden etal. 2018).
Furthermore, despite the “best practice” label, EIS have not been a common
subject of evaluation. A recent review of EIS evaluations (Worden etal. 2018)
identified only six evaluations appearing in four research studies examining the
158 THE ANNALS OF THE AMERICAN ACADEMY
impact of agencies’ EIS interventions on citizen complaints against officers, officer
use of force, and officer arrest activity. Due to agency specificity in the structure of
EIS components, including the types of indicators used and the nature of interven-
tions, it has been suggested that each evaluation is best defined as a case study
(Worden etal. 2013). As a whole, these six case studies report positive findings,
suggesting that EIS interventions may reduce the likelihood of future complaints
against officers and subsequent uses of force. However, our confidence in the accu-
racy and generalization of these findings is limited by the considerable threats to
internal validity within each study (Worden etal. 2018). Further doubt is cast on
the effectiveness of EIS interventions because the most rigorous evaluation pro-
vides the least favorable results (see Worden etal. 2013). Overall, the evidence
supporting the use of EIS as a mechanism to decrease officer use of force, and in
particular officer-involved shootings, is not strong. The limited number of evalua-
tions, combined with weak research designs, and the limited scope of the outcomes
examined leaves many questions surrounding the positive effects, as well as the
potential unintended consequences, of EIS interventions.
The UCPD was already in the process of implementing an EIS when the
officer-involved shooting occurred. The purchased software was quickly inte-
grated into the records management system, and the use of the EIS was publicly
announced. However, inherent issues within the tracking system (e.g., setting
appropriate thresholds to identify outliers within a small agency) presented con-
tinuous challenges for effectiveness. Likewise, integrating the EIS into a more
comprehensive accountability and oversight strategy within the UCPD became
an overarching goal. The impact of this specific reform, however, has not been
adequately studied, as the small number of UCPD officers assigned to similar
shifts and tasks creates a challenge for identifying outliers and determining
impact based on statistical significance. What is measured and tracked, however,
is the use of the system by supervisors to identify subordinate behavior that is
potentially problematic.
Civilian oversight
A final popular call to reduce officer killings includes multiple forms of com-
munity engagement, including citizen oversight of the police. For example, the
President’s Task Force (2015) highlights the use of civilian oversight mechanisms
and specifically recommends that incidents involving suspect death or officer-
involved shootings be subject to external oversight, including external review
boards and serious incident review boards. The President’s Task Force suggests
that the purpose of these boards is to “identify any administrative, supervisory,
training, tactical, or policy issues that need to be addressed” (2015, 88). From a
broader perspective, these oversight mechanisms are viewed to provide an
important alternative to “police investigating complaints against police” and are
typically recommended to increase police transparency and accountability, as
well as help to build trust and perceptions of police legitimacy within the com-
munity (Briggs 2017, 146).
MOVING BEYOND “BEST PRACTICE” 159
Research on the prevalence of civilian oversight, particularly in the form of
civilian review boards, suggests oversight is fairly common among large police
agencies, with approximately 80 percent of the fifty largest U.S. police agencies
incorporating such oversight mechanisms (Lum etal. 2016), but much less com-
mon in mid- to small-size agencies, with estimates of fewer than 150 departments
nationwide that incorporate these oversight mechanisms (De Angelis, Rosenthal,
and Buchner 2016). Furthermore, significant variation in the composition and
role in oversight processes are observed among the civilian review boards that
exist. In many cases, this variation is a function of factors both internal and exter-
nal to specific law enforcement agencies (De Angelis, Rosenthal, and Buchner
2016; Ferdik, Rojek, and Alpert 2013).
Despite calls for research (National Research Council 2004; Walker 2001), the
evidence for civilian oversight of police agencies remains limited (Lum et al.
2016). The extreme variability across civilian review boards has proven a difficult
hurdle in the examination of the prevalence, effects, and best models for over-
sight. Additionally, the question remains of how best to measure the effectiveness
of civilian oversight within law enforcement agencies (De Angelis, Rosenthal,
and Buchner 2016), although available research studies typically opt to examine
rates of sustained complaints. Collectively, this research provides mixed findings
regarding the sustained rates of citizen complaints when civilian oversight is pre-
sent, with some noting fewer sustained complaints in agencies with some form of
civilian oversight (Hickman 2006), while others observe higher rates of sustained
complaints when oversight is present (Terrill and Ingram 2015).
In the immediate weeks following the officer-involved shooting in Cincinnati,
the UC executive team established the Community Advisory Council (CAC) to
provide community input and guidance regarding reform efforts of the UCPD.
Modeled after the Cincinnati City Manager’s Advisory Group (MAG), which has
been identified as a model for a successful citizen oversight organization (Eck
and Rothman 2006; Wasserman 2014), the CAC became a primary mechanism
for community input and collaboration. Representatives include UC students,
faculty, staff, and alumni; neighborhood community groups; city civic, faith, and
business leaders; and law enforcement officials. The varied membership of the
CAC was designed to reflect the diverse makeup of the larger Cincinnati com-
munity and to promote diversity of thought, ideas, and information exchange.
Although the effectiveness of this council was not quantitatively evaluated, quali-
tative assessments suggest a profound enhancement of police community rela-
tions. The CAC (renamed the Community Compliance Council) is now beginning
a new phase of work, including review of the processes followed for all com-
plaints against UCPD officers and use-of-force incidents.
Discussion
In response to the recent national crisis in police-community relations sparked by
controversial police-involved shootings, police executives in the United States are
160 THE ANNALS OF THE AMERICAN ACADEMY
routinely encouraged (and in some cases, mandated) to implement reforms,
including deploying BWCs, de-escalation training, implicit bias training, early
intervention systems, and civilian oversight. Similar to other recent reviews (e.g.,
Lum etal. 2016), our examination of the available policing research exposes vast
gaps in the collective knowledge regarding the effectiveness of reforms most
commonly recommended to reduce officer use of force. As police executives seek
to answer calls for reform while meeting expectations of evidence-based practice,
there is limited scientific research to support them. Yet the pressure is intensify-
ing for police executives to rapidly implement changes designed to reduce use-
of-force incidents, as the public is eager to move “beyond what is legal and start
focusing on what is preventable” (Kindy 2015). These recommended reforms,
however, likely do more in the way of “damage control” after a critical incident
than actually preventing future fatal police-citizen encounters.
In the immediate aftermath of the fatal shooting of Samuel DuBose, the UC
executive team focused on crisis management, while still seeking to implement
evidence-based strategies for long-term reform and sustainability. As protesters
gathered and politicians demanded action, we searched for evidence-based solu-
tions. What we found, instead, was a collection of best practices—that is, inter-
ventions that were supported by anecdotal accounts and experts’ endorsements,
but not scientifically validated. While the effectiveness and unintended conse-
quences of many of our reform efforts were largely unknown, the immediate
nature of police work did not allow time for years of study and debate. Ultimately,
we selected and implemented interventions based on the limited science availa-
ble, along with our best judgments and intuition. Our experience reflects a reality
in American policing: changes in policing practice are happening now, and police
executives cannot wait for the research field to keep pace. This experience also
highlights a reality in policing research: we often fail to adequately test innova-
tions in policing practices, and we rarely examine the impact of police training.
Even more concerning is the lack of scholarly interest (or perhaps opportunity or
ability) to test the effects of interventions designed to directly impact what many
have identified as the crucial issue underlying the current policing crisis—officer
use of force.
What can be done? To facilitate the implementation of evidence-based
approaches in reducing officer-involved shootings and other police reforms, law
enforcement agencies must (1) consider findings from available research; and (2)
where evidence is lacking, engage in opportunities to fill that void. Our review
demonstrates that much of the evidence base surrounding the most popular
reform initiatives to reduce officer shootings is limited in scope or nonexistent.
Therefore, to advance EBP principles for reform, agencies must monitor the
application and impact of training, policies, and procedures designed to reduce
police use of force. That is, law enforcement must take responsibility in collecting
data to generate evidence on the delivery and effects of their practices (Sherman
2013), moving beyond being consumers of research and instead engaging in the
production of knowledge.
This is the approach we used to facilitate the reform efforts within the UCPD.
Our first step was to redesign data collection systems to include the data necessary
MOVING BEYOND “BEST PRACTICE” 161
to evaluate the impact of our work. Our executive team modified existing data col-
lection processes and also mandated the collection of new data. Changes in data
collection instruments and practices resulted in new data generated during traffic
and pedestrian stops, during the citizen complaint process, through the review and
cataloging of BWC footage, during potential use-of-force encounters (e.g., when
officers draw their Tasers or firearms but do not deploy them), along with multiple
citizen and officer surveys. Each of these data collection changes required an
accompanying change in policy, training, and supervisory oversight to ensure that
the data were being properly collected and used. The UCPD is now in a better
position to test specific propositions about the effectiveness of our own reform
efforts. Based on these experiences, we believe that our best chance to reduce
officer-involved shootings is working at the local (agency) level by building and
sharing capacity for EBP and testing to determine what works.
Given the realities of policing, as new policies are introduced and officer training
advances, police executives must take it upon themselves to consider how they
might evaluate these practices to understand the effects on their agency, including
partnering with researchers when their expertise is needed. Unfortunately, with the
exception of a handful of progressive police agencies, this type of testing is rare.
Nevertheless, as the EBP movement continues to gain acceptance, a tipping point
in the profession is likely to occur. Similar to the impact in medicine and other
disciplines, the acceptance of evidence-based practices should ultimately propel
the proliferation and dissemination of research in policing.
For this change to come to light, however, policing researchers, as a profes-
sion, must also do better. We must develop the infrastructure to provide a rapid
research response to the most pressing issues in policing. Currently, our
approaches to research are too slow, and the topics selected for examination are
too limited. The scientific evaluation of training and technical assistance within
law enforcement must be recognized as a worthwhile area of research, deserving
dedication of time and resources. A faster and more relevant research response
will require the establishment of fully collaborative police-academic partnerships
(Engel and Henderson 2013; Engel and Whalen 2010). Academics must step
away from the ivory tower to take a seat at the table with police agencies and,
simultaneously, police agencies must be gracious hosts. In sum, we must work
together to build the evidence base necessary to better guide police agencies in
their reform efforts. In the specific case of police use of force, facilitating the
transition from “best practice” to evidence-based practice through research has
the potential to reduce officer and citizen injuries and fatalities. In the collective,
such research will fill a tremendous gap in knowledge, informing the field on
successful models and overall effectiveness to better direct the national conversa-
tions surrounding policing and police reform.
Notes
1. Prior to the criminal trials, the university settled a civil rights and wrongful death claim with
DuBose’s family valued at $5.3 million. Tensing ultimately settled a union grievance with the university for
162 THE ANNALS OF THE AMERICAN ACADEMY
wrongful termination and was awarded approximately $345,000 for back pay and legal fees but was not
reinstated within the UCPD.
2. During this time period, other high-profile deaths of unarmed black males by police included
Michael Brown (Ferguson, MO, 2014), Tamir Rice (Cleveland, OH, 2014), Eric Garner (New York, NY,
2014), Walter Scott (North Charleston, SC, 2015), Freddie Gray (Baltimore, MD, 2015), and Terence
Crutcher (Tulsa, OK, 2016), among others.
3. The Uniform Crime Report (UCR) produced by the FBI identifies Part I crime incidents to include
eight types of offenses: criminal homicide, rape, robbery, aggravated assault, burglary, larceny-theft, motor
vehicle theft, and arson (see UCR Offense Definitions at https://www.ucrdatatool.gov/offenses.cfm).
4. A comprehensive independent review of all UCPD policies, procedures, practices, and training
resulted in a hefty report containing 129 findings and 276 specific recommendations to meet best practices
(Exiger 2016). In the final stages of the planned reforms, all but one of these recommendations were
deemed as meeting substantial compliance by the external monitor. The lone recommendation not meet-
ing substantial compliance was found in partial compliance awaiting an additional assessment period
(Exiger 2019).
References
Ariel, Barak, William A. Farrar, and Alex Sutherland. 2015. The effect of police body-worn cameras on use
of force and citizens’ complaints against the police: A randomized controlled trial. Journal of
Quantitative Criminology 31 (3): 509–35.
Ariel, Barak, Alex Sutherland, Darren Henstock, Josh Young, Paul Drover, Jayne Sykes, Simon Megicks,
and Ryan Henderson. 2016. Report: Increases in police use of force in the presence of body worn
cameras are driven by officer discretion: A protocol based subgroup analysis of ten randomized
experiments. Journal of Experimental Criminology 12 (3): 453–63.
Atewologun, Doyin, Tinu Cornish, and Fatima Tresh. 2018. Unconscious bias training: An assessment of
the evidence for effectiveness. Manchester: Equality and Human Rights Commission. Available from
www.equalityhumanrights.com.
Bittner, Egon. 1974. Florence Nightingale in pursuit of Willie Sutton: A theory of the police. The Potential
for Reform of Criminal Justice 3:17–44.
Blake, David. 3 November 2017. Does de-escalation endanger police officers or save lives? PoliceOne.
Briggs, William. 2017. Police oversight: Civilian oversight boards and lessons learned from our neighbors
to the north. Suffolk Transnational Law Review 40 (1): 139–64.
CBS News. 7 August 2019. We asked 155 police departments about their racial bias training. Here’s what
they told us. CBS News.
De Angelis, Joseph, Richard Rosenthal, and Brian Buchner. 2016. Civilian oversight of law enforcement:
Assessing the evidence. Tucson, AZ: National Association for Civilian Oversight of Law Enforcement.
Available from www.ncjrs.gov.
Devine, Patricia G. 1989. Stereotypes and prejudice: Their automatic and controlled components. Journal
of Personality and Social Psychology 56 (1): 5–18.
Diener, Edward, and Mark Wallbom. 1976. Effects of self-awareness on antinormative behavior. Journal
of Research in Personality 10 (1): 107–11.
Dovidio, John F., Kerry Kawakami, and Samuel L. Gaertner. 2002. Implicit and explicit prejudice and
interracial interaction. Journal of Personality and Social Psychology 82 (1): 62–68.
Duguid, Michelle M., and Melissa C. Thomas-Hunt. 2015. Condoning stereotypes? How awareness of
stereotyping prevalence impacts expression of stereotypes. Journal of Applied Psychology 100 (2):
343–59.
Eck, John E., and Jay Rothman. 2006. Police-community conflict and crime prevention in Cincinnati,
Ohio. In Public security and police reform in the Americas, eds. John Bailey and Lucia Dammert,
225–44. Pittsburgh, PA: University of Pittsburgh Press.
Engel, Robin S., Nicholas Corsaro, Gabrielle T. Isaza, and Hannah D. McManus. 2019. Proposal for the
evaluation of ICAT training for the Louisville Metro Police Department. Submitted to the Louisville
Metro Police Department, Louisville, KY.
MOVING BEYOND “BEST PRACTICE” 163
Engel, Robin S., and Samantha Henderson. 2013. Beyond rhetoric: Establishing police-academic partner-
ships that work. In The future of policing, ed. Jennifer M. Brown, 247–66. London: Routledge.
Engel, Robin S., Hannah D. McManus, and Tamara D. Herold. Forthcoming. Does de-escalation training
work? A systematic review and call for evidence in police use of force reform. Criminology and Public
Policy.
Engel, Robin S., and Ronal Serpas. 2017. Evidence-based use-of-force policy: How research could
improve development and training. Police Chief, April: 28–36.
Engel, Robin S., and Kristin Swartz. 2014. Race, crime, and policing. In The Oxford handbook of ethnicity,
crime, and immigration, eds. Sandra M. Bucerius and Michael Tonry, 135–65. New York, NY: Oxford
University Press.
Engel, Robin S., and James L. Whalen. 2010. Police–academic partnerships: Ending the dialogue of the
deaf, the Cincinnati experience. Police Practice and Research: An International Journal 11 (2): 105–16.
Engel, Robin S., Robert E. Worden, Nicholas Corsaro, Hannah D. McManus, Danielle Reynolds, Hannah
Cochran, Gabrielle T. Isaza, and Jennifer Calnon Cherkauskas. 2019. The power to arrest: Lessons
from research. New York, NY: Springer.
Exiger. 2016. Final report for the comprehensive review of the University of Cincinnati Police Department.
Submitted to University of Cincinnati Office of Safety and Reform, Cincinnati, OH. Available from
www.uc.edu.
Exiger. 2019. Final report of the independent monitor for the University of Cincinnati Police Division.
Submitted to University of Cincinnati Board of Trustees Audit Committee, Cincinnati, OH. Available
from www.uc.edu.
Federal Bureau of Investigation (FBI). 2019. Law enforcement officers killed and assaulted. Available from
www.fbi.gov/leoka.
Ferdik, Frank V., Jeff Rojek, and Geoffrey P. Alpert. 2013. Citizen oversight in the United States and
Canada: An overview. Police Practice and Research 14 (2): 104–16.
Forscher, Patrick S., Calvin K. Lai, Jordan R. Axt, Charles R. Ebersole, Michelle Herman, Patricia G.
Devine, and Brian A. Nosek. 2018. A meta-analysis of procedures to change implicit bias measures.
Pre-Print.
Fridell, Lorie A. 2017. Producing bias-free policing: A science-based approach. New York, NY: Springer.
Garner, Joel H., Matthew J. Hickman, Ronald W. Malega, and Christopher D. Maxwell. 2018. Progress
toward national estimates of police use of force. PLos ONE 13 (2): 1–23.
Grasha, Kevin. 18 July 2017. No third trial for ex-UC cop Ray Tensing in shooting death of unarmed
motorist Sam DuBose. Cincinnati Enquirer.
Harmon, Rachel A. 2009. Promoting civil rights through proactive policing reform. Stanford Law Review
62 (1): 1–68.
Hickman, Matthew J. 2006. Citizen complaints about police use of force. Washington, DC: Bureau of
Justice Statistics. Available from www.bjs.gov.
International Association of Chiefs of Police. 2017. National consensus policy on use of force. Alexandria,
VA: International Association of Chiefs of Police. Available from www.theiacp.org.
Isaza, Gabrielle, Murat Yildirim, Jillian Shafer, and Murat Ozer. 2017. 2016 campus crime report.
Submitted to the University of Cincinnati Campus Crime Reduction Committee, Cincinnati, OH.
Available from www.uc.edu.
Jones, Hannah G. 12 October 2015. The science of bias policing. Local 12 News.
Kindy, Kimberly. 30 May 2015. Fatal police shootings in 2015 approaching 400 nationwide. Washington
Post.
Klinger, David. 2005. Social theory and the street cop: The case of deadly force. Washington, DC: Police
Foundation. Available from www.policefoundation.org.
Klinger, David. 2020. Organizational accidents and deadly police-involved violence: Theory, research,
public policy, and police practice. The ANNALS of the American Academy of Political and Social
Science (this volume).
Kroll. 2015. Review and investigation of Officer Raymond M. Tensing’s use of deadly force on July 19, 2015:
University of Cincinnati Police Department. Submitted to the University of Cincinnati Office of
General Counsel, Cincinnati, OH. Available from www.uc.edu.
Landers, Brian. 14 October 2017. Are de-escalation policies dangerous? Police Magazine. Available from
https://www.policemag.com/342333/are-de-escalation-policies-dangerous.
164 THE ANNALS OF THE AMERICAN ACADEMY
Lum, Cynthia, Christopher S. Koper, Charlotte Gill, Julie Hibdon, Cody Telep, and Laurie Robinson.
2016. An evidence-assessment of the recommendations of the President’s Task Force on 21st Century
Policing – Implementation and research priorities. Fairfax, VA: Center for Evidence-Based Crime
Policy, George Mason University. Available from www.cebcp.org.
Lum, Cynthia, Christopher S. Koper, and Cody W. Telep. 2011. The evidence-based policing matrix.
Journal of Experimental Criminology 7 (1): 3–26.
Lum, Cynthia, Megan Stoltz, Christopher S. Koper, and J. Amber Scherer. 2019. Research on body-worn
cameras: What we know, what we need to know. Criminology and Public Policy 18 (1): 1–26.
Macrae, C. Neil, Galen V. Bodenhasuen, Alan B. Milne, and Jolanda Jetten. 1994. Out of mind but back
in sight: Stereotypes on the rebound. Journal of Personality and Social Psychology 67 (5): 808–17.
McLean, Kyle, Scott E. Wolfe, Jeff Rojek, Geoffrey P. Alpert, and Michael R. Smith. 2020. Evaluating a
long-term social interaction police training program: A randomized controlled trial in the real world.
The ANNALS of the American Academy of Political and Social Science (this volume).
Mitchell, Renee J. and Lois James. 2018. Addressing the elephant in the room: The need to evaluate
implicit bias training effectiveness for improving fairness in police officer decision-making. Police Chief
Magazine. Available from https://www.policechiefmagazine.org/addressing-the-elephant-in-the-room.
Monteith, Margo J., Steven A. Arthur, and Sara M. Flynn. 2010. Self-regulation and bias. In The SAGE
handbook of prejudice, stereotyping and discrimination, eds. John F. Dovidio, M. Hewstone, P. Glick,
and V. M. Esses, 493–507. Thousand Oaks, CA: Sage Publications.
National Research Council. 2004. Fairness and effectiveness in policing: The evidence. Washington, DC:
National Academies Press.
Nix, Justin, Bradley A. Campbell, Edward H. Byers, and Geoffrey P. Alpert. 2017. A bird’s eye view of
civilians killed by police in 2015: Further evidence of implicit bias. Criminology & Public Policy 16 (1):
309–40.
Noble, Greg. 29 July 2015. Watch: Body cam video released in Sam DuBose shooting. WCPO Cincinnati.
Payne, B. Keith. 2001. Prejudice and perception: The role of automatic and controlled processes in mis-
perceiving a weapon. Journal of Personality and Social Psychology 81 (2): 181–92.
Perrow, Charles. 1984. Normal accidents: Living with high risk technologies. New York, NY: Basic Books.
Pettigrew, Thomas F., and Linda R. Tropp. 2005. Allport’s intergroup contact hypothesis: Its history and
influence. In On the nature of prejudice: Fifty years after Allport, eds. John F. Dovidio, Peter Glick,
and Laurie A. Rudman, 262–77. Malden, MA: Blackwell Publishing.
Police Executive Research Forum. 2016. Critical issues in policing series: Guiding principles on use of
force. Washington, DC: Police Executive Research Forum.
Police Executive Research Forum. 2018. Cost and benefits of body-worn camera deployments: Final
report. Washington, DC. Available from www.policeforum.org.
President’s Task Force on 21st Century Policing. 2015. Final report of the President’s Task Force on 21st
Century Policing. Washington, DC: Office of Community Oriented Policing Services.
Reaves, Brian A. 2011. Census of state and local law enforcement agencies, 2008. Washington, DC: U.S.
Department of Justice, Bureau of Justice Statistics. Available from www.bjs.gov.
Sherman, Lawrence W. 1998. Evidence-based policing. Ideas in American Policing Series. Washington,
DC: Police Foundation. Available from www.policefoundation.org.
Sherman, Lawrence W. 2013. The rise of evidence-based policing: Targeting, testing, and tracking. Crime
and Justice 42 (1): 377–451.
Sherman, Lawrence W. 2018. Reducing fatal police shootings as system crashes: Research, theory, and
practice. Annual Review of Criminology 1:421–49.
Skogan, Wesley G., Martin Van Craen, and C. Hennessy. 2015. Training police for procedural justice.
Journal of Experimental Criminology 11 (3): 319–34.
Telep, Cody W. 2016. Expanding the scope of evidence-based policing. Criminology & Public Policy 15
(1): 243–52.
Terrill, William, and Jason R. Ingram. 2015. Citizen complaints against the police: An eight city examina-
tion. Police Quarterly 19 (2): 150–79.
Walker, Samuel. 2001. Police accountability: The role of citizen oversight. Belmont, CA: Wadsworth.
Walker, Samuel. 2003. The new paradigm of police accountability: The U.S. Justice Department “pattern
or practice” suits in context. St. Louis University Public Law Review 22 (3): 3–52.
MOVING BEYOND “BEST PRACTICE” 165
Wasserman, R. 2014. Building relationships of trust: Recommended steps for chief executives. Tallahassee,
FL: Institute for Intergovernmental Research.
Weisburd, David, and Peter Neyroud. 2011. Police science: Toward a new paradigm. Washington, DC:
Office of Justice Programs, National Institute of Justice. Available from www.ncjrs.gov.
White, Michael D., and Michael Pooley. 2018. Testing the impact of de-escalation training on officer
behavior: The Tempe, Arizona SPI. Presented to Tempe Police Department, Tempe, AZ.
Williams, Timothy. 27 June 2015. Long taught to use force, police warily learn to de-escalate. New York
Times.
Worden, Robert E., Moonsun Kim, Christopher J. Harris, Mary Anne Pratte, Shelagh E. Dorn, and
Shelley S. Hyland. 2013. Intervention with problem officers: An outcome evaluation of an EIS inter-
vention. Criminal Justice Behavior 40 (4): 409–37.
Worden, Robert E., Sarah J. McLean, Kelly J. Becker, Eugene A. Paoline III, Hannah Cochran, and Chris
Harris. 2018. Police early intervention systems: The state of the art. Albany, NY: The John F. Finn
Institute for Public Safety, Inc.
Zimring, Franklin E. 2017. When police kill. Cambridge, MA: Harvard University Press.
Zimring, Franklin E. 2020. Police killings as a problem of governance. The ANNALS of the American
Academy of Political and Social Science (this volume).
