No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Justifying the use of non-experimental methods and disqualifying the use of randomized controlled trials: Challenging folklore in evaluation research in crime and justice
Abstract
The key limitation of non-experimental evaluation methods is that they require an assumption that all confounding factors
related to treatment are identified in the statistical models developed. The key advantage of randomized experiments is that
this assumption can be relaxed. In this paper, I describe and explain why this assumption is so critical for non-experiments
and why it can be ignored in randomized controlled trials (RCTs). I also challenge what I describe as “folklores” that are
used to justify the use of non-randomized studies despite this statistical limitation, and to justify the failure of evaluation
researchers in crime and justice to use randomized experiments despite their unique ability to overcome this limitation. I
conclude by reinforcing what Joan McCord had argued after a life time of review of evaluations: “(W)henever possible” evaluation
studies “should employ random assignment.”
KeywordsRandomized experiments-Randomized controlled trials (RCTs)-Non-experimental methods-Evaluation studies-Confounding variables
... There is a consensus among researchers that experimental designs improve internal validity and reduce selection bias (Burtless, 1995;Heckman and Smith, 1995). Four main characteristics of this research strategy make it best suited for establishing causality (Burtless, 1995;Heckman and Smith, 1995;Weisburd, 2010;Weisburd et al., 2013): ...
... This suggests that RCTs are better able to rule out selection effects and more accurately depict the program's impact (Weisburd et al., 2001). Weisburd (2010) also points out that the amount of "variance explained" using quasi-and nonexperimental designs is low. He warns that many potentially confounding variables are missing from such designs and that researchers are dismissing the systematic nature (and importance) of these missing variables. ...
... QEDs use statistical methods, time-series analyses, matching strategies, natural experiments and other approaches to produce equivalency in relevant characteristics prior to treatment implementation. Although QEDs improve causal inference in many ways, they are still unable to rule out confounding influences from unmeasured variables (Weisburd, 2010). ...
... Despite several important limitations, experimental evidence continues to serve as the gold-or at the very least the bronze (Berk 2005)-standard on the evidentiary hierarchy in the social and behavioral sciences (Banerjee and Duflo 2009;Imbens 2010;Weisburd 2010). 1 The fact that random assignment, in expectation, creates comparable treatment and control groups and allows us to credibly ascribe causality to group-based differences is a feature that has been appreciated since at least the 18th century (Dunn 1997) but which was popularized in the early 20th century by the work of Jerzy Neyman and Fisher (Fisher 1936;Splawa-Neyman et al. 1923). Recently, the foundational importance of experiments has been bolstered by concerns about the limitations of observational research, with an array of evidence from econometrics and statistics suggesting that experimental treatment effects cannot be reliably recovered using some of the most ubiquitous types of observational research designs (LaLonde 1986;Smith andTodd 2001, 2005;Rubin 2008;Gordon et al. 2019) even when analysts have a rich set of covariates upon which to condition (DiNardo and Pischke 1997;Arceneaux et al. 2010). ...
... Recently, the foundational importance of experiments has been bolstered by concerns about the limitations of observational research, with an array of evidence from econometrics and statistics suggesting that experimental treatment effects cannot be reliably recovered using some of the most ubiquitous types of observational research designs (LaLonde 1986;Smith andTodd 2001, 2005;Rubin 2008;Gordon et al. 2019) even when analysts have a rich set of covariates upon which to condition (DiNardo and Pischke 1997;Arceneaux et al. 2010). Reflecting these concerns, Weisburd (2010) has, channeling a sentiment found in McCord (2003) suggested that "whenever possible, evaluation studies should employ random assignment." ...
Objective
The increasing availability of large administrative datasets has led to an exciting innovation in criminal justice research—using administrative data to measure experimental outcomes in lieu of costly primary data collection. We demonstrate that this type of randomized experiment can have an unfortunate consequence: the destruction of statistical power. Combining experimental data with administrative records to track outcomes of interest typically requires linking datasets without a common identifier. In order to minimize mistaken linkages, researchers often use stringent linking rules like “exact matching” to ensure that speculative matches do not lead to errors in an analytic dataset. We show that this, seemingly conservative, approach leads to underpowered experiments, leaves real treatment effects undetected, and can therefore have profound implications for entire experimental literatures.
Methods
We derive an analytic result for the consequences of linking errors on statistical power and show how the problem varies across combinations of relevant inputs, including linking error rate, outcome density and sample size.
Results
Given that few experiments are overly well-powered, even small amounts of linking error can have considerable impact on Type II error rates. In contrast to exact matching, machine learning-based probabilistic matching algorithms allow researchers to recover a considerable share of the statistical power lost under stringent data-linking rules.
Conclusion
Our results demonstrate that probabilistic linking substantially outperforms stringent linking criteria. Failure to implement linking procedures designed to reduce linking errors can have dire consequences for subsequent analyses and, more broadly, for the viability of this type of experimental research.
... Researchers have examined the use of experiments in specific criminal justice areas and consistently find that experiments are in the minority of methods deployed (Farrington and Welsh 2006;Weisburd 2003;Weisburd 2010;Weisburd and Hinkle 2012). For example, only 11.4% of published evaluations on disciplinary boot camps used experimental methods, and less than 10% of drug court and drug treatment evaluations used experimental designs (Mackenzie et al. 2001;Mitchell et al. 2005;Wilson et al. 2005). ...
... For example, place-based experiments have been used to evaluate policing interventions by hot spots (Braga et al. 2013;Farrington and Welsh 2006;Weisburd 2005) and to evaluate prevention and treatment program contexts (Farrington 2008;Farrington and Welsh 2006). Encouraging the use of place-based designs in other settings (e.g., probation supervision, prison-based programs, school-based programs) could increase the use of experimental evaluations across criminology (Weisburd 2010). ...
Objectives
Despite experiments being termed the “gold standard,” criminology has been slow to adopt experimentation as a methodology. The goal of this research note is to better understand the use of experiments in criminology and the potential barriers in implementing this methodology.Methods
We conducted a systematic assessment of experiments in Criminology and Justice Quarterly by reviewing every article published since the inception of the journals and coding for numerous elements (i.e., field versus lab studies, research areas, outcomes, and funding sources).ResultsDespite a general increase, experiments make up only 2.8% (n = 44) of studies in Criminology and 3.1% (n = 31) of studies in Justice Quarterly.Conclusions
While it is not always clear why experiments are not used more often, we explore the possible reasons for a lack of experimentation (i.e., a lack of mentoring, challenges to practitioner buy in, etc.) and areas for future research.
... Without further elaborating, some authors have questioned the possibility of rigorously studying programs, such as the Sherbrooke's co-response team, implemented in real-world settings (Compton et al. 2008;Helfgott et al. 2016). Conversely, Weisburd (2010) have shown that RCTs can be implemented in real-world settings; feasibility concerns are mainly based in folklore rather than on scientific evidence. In the present study, the evaluation was simultaneously planned with the implementation of the program, precluding any randomization. ...
... Based on the available evidence, conducting RCTs to assess the impact of co-response programs seems more than feasible. Such a trial would not affect the autonomy or authority of police officers, which could facilitate the acceptance of an experimental design (Weisburd 2000(Weisburd , 2010. The dispatcher could randomly assign calls to the MCIT or the "treatment as usual" condition. ...
Objectives
Assess the capability of a mobile crisis intervention team (MCIT) to connect emotionally disturbed people (EDP) with community resources and decrease police use-of-force.
Method
In order to have equivalent groups, interventions managed by the MCIT were matched to incidents handled by traditional police officers with similar propensity scores.
Results
Average treatment effects (ATEs) were computed to assess the impact of the MCIT. The MCIT was associated with decreases in police use-of-force (ATE=-0.08; p≤0.01), EDP transported to the hospital against their will (ATE=-0.06; p≤0.10), and EDP transported to the hospital in general (ATE=-0.42; p≤0.01). EDP were more likely to be referred to community resources (ATE=0.19; p≤0.01) or managed by their social network (ATE=0.22; p≤0.01) when the MCIT was involved in the intervention.
Conclusion
The MCIT was effective in connecting EDP with community resources, avoiding unnecessary transports to the hospital, and reducing police use-of-force.
... Political, ethical, and/or practical concerns sometimes lead evaluators to conclude that random assignment is not feasible when evaluation plans are made (see, e.g., Clarke and Cornish 1972;Erez 1986). The continued growth of criminological experiments in a broad range of real-world settings that have been carried out in an ethical manner demonstrates that many of these concerns are, in most cases, based in folklore rather than facts (Weisburd 2010). Nevertheless, quasiexperiments are often used as alternative approaches in program evaluations. ...
... In addition, it is also important to obtain sound knowledge about what findings can be safely generalized. Although studies with nonexperimental and internally less valid designs may particularly emphasize external validity, they have the same problems of generalization (e.g., Weisburd 2010). For these and other reasons, policy makers and practitioners should seek to carry out experiments whenever feasible and preferably before implementing expensive programs. ...
Objectives
This article summarizes key points made in a panel at the American Society of Criminology (ASC) meeting in Atlanta in November 2018, entitled “20th Anniversary of the Academy of Experimental Criminology (AEC): Looking Back and Forward,” organized by Friedrich Lösel as the AEC president.
Method
Seven (current and former) presidents of AEC contribute short papers about the past and future of experimental criminology, focusing on different and emerging areas of criminological experimentation, as well as identifying topics that require more attention in future, including field experiments and experimental neurocriminology.
Results
This article informs readers about the history of AEC, its links with the Journal of Experimental Criminology, current issues, and potential future developments in experimental criminology. It also briefly deals with arguments that question whether experiments are the “gold standard,” which were addressed by Daniel Nagin and Robert Sampson in another ASC session at Atlanta. Experimental panel members did not view randomized controlled trials (RCTs) as a ritualized and general “gold standard” for criminological research, because many important topics cannot be investigated in this type of design.
Conclusions
This article is not intended to be a missionary statement for RCTs, but it does argue that experiments should be used whenever feasible, because they are most robust in ensuring internal validity as the basis for external validity and for generalizations that are necessary for effective practice and policy making.
... Persons with behavioral and mental illness are overrepresented in criminal-legal systems (Al-Rousan et al., 2017; National Institute on Drug Abuse, 2020; Prince & Wald, 2018) and the push toward evidence-based policing (Sherman, 2013) has resulted in many new strategies, like co-response teams, for diverting these persons into needed treatments and services in lieu of arrest. While these strategies often lack a rigorous evidence base (Lum & Yang, 2005;Weisburd, 2010), the current study represents the exception as it outlines an experimental study design-the first RCT of a police-mental health co-response team-implemented in a real-world setting with the goal of furthering efforts to rigorously evaluate alternative response programs and other police-based innovations. As demonstrated in our qualitative findings, this kind of rigorous research requires close collaboration with program stakeholders to develop and modify study procedures in a pragmatic manner. ...
Objectives
Describe the development, results, and stakeholder perceptions of randomization procedures for determining outcomes of a co-response police-mental health team.
Methods
We present randomization results using the CONSORT diagram and report on three semi-structured focus groups with eight co-response team members, including police officers, mental health clinicians, and program leaders.
Results
Study procedures resulted in randomization of 686 co-response team-eligible calls for service to either receive a co-response team (treatment group, n = 376) or police-as-usual response (control group, n = 310). Focus groups revealed lessons for randomization of a co-response team, including the importance of the researcher-practitioner partnership, considerations for study site selection and staffing, and suggestions to proactively address ethical concerns of randomizing calls for service.
Conclusions
Rigorous evaluation of alternative policing programs is possible through randomization at the call-for-service level, provided researchers and program stakeholders work together to determine feasible procedures.
... While most criminological research is non-experimental (Dezember et al. 2021), there has been a growing recognition that random allocation is not only necessary for establishing causal effects in evaluation research (Weisburd 2010), but that a broad scope of criminological topics can benefit from randomized controlled trials (Ridgeway 2019). This echoes earlier calls to make social science more experimental (Sherman 2003), ...
... Also, there was no control group in the study, so it is impossible to determine whether or not there may have been a difference in results between those surveyed and those interviewed. Another key limitation is it is not a randomized experiment (Weisburd, 2010), which means that one cannot say whether or not there would have been any difference between the results if authors had conducted another type of experiment with different participants or if they had used a different method to get our research subjects involved (e.g., through focus groups). ...
Purpose: The marketing practice for the leisure and sports industry is a complex process that requires a thorough understanding of the audience, their needs, and motivations. Thus, this niche market is focused on specific products, services, or experiences. The present research explores and suggests meaningful strategies based on the literature textual dataset to provide how to attract consumers in this sector. Research design, data, and methodology: We have conducted the 'Secondary Literature Analysis', reviewing and summarizing numerous findings in the relevant prior studies. As a result, we could obtain a total of 15 significant textual resources which are from only peer-reviewed journal articles. All resources had a high-quality instrument to prove their results. Results: The findings of this research pointed out that marketers in the leisure sports sector need to communicate via the following methods: (1) Understanding the Customers' Needs and Wants, (2) Social Media, (3) Advertising, (4) Promoting Brand Affinity, (5) Offering Discounts, and (6) Providing Value-Added Services. Conclusions: The present research concludes that the marketing practice in the leisure and sports industry should be performed using various channels. In addition, marketing practitioners are supposed to check if tailored marketing messages are compatible with products, services, and events that relate to their target audience's interests.
... Just as the medical model espoused the virtues of randomized controlled trials, so too did Sherman, who wrote about the need for experimental research in policing and has, along with others, continued to do so (Sherman, 1998(Sherman, , 2013Weisburd, 2010). Over the last two decades, this initial conception of EBP has evolved, with its followers now encouraging a broader definition, including a willingness to support methods other than experimental designs, and to conduct research on a wider range of outcomes than crime reduction and prevention. ...
Evidence-based policing (EBP) has become a key perspective for practitioners and researchers concerned with the future of policing. This volume provides both a review of where evidence-based policing stands today and a consideration of emerging trends and ideas likely to be important in the future. It includes comparative and international contributions, as well as researcher and practitioner perspectives. While emphasizing traditional evidence-based methods and approaches, the book also identifies barriers to the advancement of evidence-based policing and expands the vision of evidence-based policing by critically examining ethical and moral concerns and questions. The book's main focus is not on what has to happen in police agencies to advance EBP, but rather on an issue that has received far less attention - the science that is necessary to produce for EBP to be successfully integrated into policing.
... Solange die Wirksamkeit einer Maßnahme nicht überzeugend belegt ist, kann daher nicht beurteilt werden, ob eine Nichtbehandlung oder eine Behandlung das größere Übel darstellt. Es gibt keinen inhaltlichen Grund, dass in der Straftäterbehandlung geringere Qualitätsmaßstäbe angelegt werden als in der Medizin oder Pharmakologie, in denen RCTs ein unverhandelbarer Standard der Qualitätskontrolle sind (Weisburd 2010 Wenn wir wirklich wissen wollen, ob die in Deutschland durchgeführten Behandlungsmaßnahmen für Sexual-und Gewaltstraftäter wirksam sind und wie wir diese Maßnahmen verbessern können, sehen wir zu einer solchen zweistufigen Evaluationsstrategie keine Alternative, und auch andere Autoren haben parallel zu uns einen sehr ähnlichen Ansatz vorgeschlagen 4 . Wir sind der Meinung, dass sich aufgrund des zu erwartenden gesellschaftlichen Nutzens ein so groß angelegtes Projekt lohnt. ...
Die empirische Bewährung von Behandlungsprogrammen von Straftätern (insbesondere Sexualstraftätern) wird seit langem kritisch diskutiert. In diesem Kapitel geben wir einen kurzen Überblick über diese Kontroverse. Wir legen dar, dass nur experimentelle Evaluationsstudien mit zufälliger Zuordnung zu einer Behandlungsgruppe und einer Kontrollgruppe (Randomized Controlled
Trials, RCTs) Evidenz für die absolute Wirksamkeit eines Behandlungsprogrammes liefern können. Auch methodisch starke Evaluationen der absoluten Wirksamkeit von Interventionen liefern allerdings keine Information darüber, welche Effekte die einzelnen Komponenten komplexer Behandlungsprogramme haben und wie man Behandlungsprogramme verbessern kann. Dazu
werden Methode wie Komponenten-, Dismantling- und korrelative Untersuchungsdesigns
vorgestellt, die eine empirische Prüfung der relativen Wirksamkeit einzelner Programmkomponenten
erlauben. Um die aktuelle Ungewissheit über die Wirksamkeit von Straftäterbehandlungsprogrammen zu überwinden, wird eine zweistufige Evaluationsstrategie
vorgeschlagen: 1) Um die relative Wirksamkeit einzelner Programmelemente zu untersuchen, sollte
eine multizentrische und länderübergreifende Evaluationsstudie durchgeführt werden. Die Vielfalt aktuell durchgeführter Behandlungsprogramme kann dabei genutzt werden, um mit multivariat-korrelativen Methoden die Wirkung einzelner Behandlungskomponenten unter spezifischen
institutionellen Randbedingungen für spezifische Gruppen von Straftätern zu bestimmen. 2) Um die absolute Wirksamkeit der vielfältigen in Deutschland angebotenen Behandlungsprogramme im
Vergleich zum Strafvollzug ohne besondere Behandlungsmaßnahmen zu prüfen, gibt es keine überzeugende Alternative zu RCTs, deren Durchführung allerdings schon durch gesetzliche Vorgaben
in einigen Vollzugsformen extrem erschwert wird. Das Ziel einer solchen kombinierten Evaluationsstrategie ist die Entwicklung eines selbstkorrigierenden Systems der Behandlung von Straftätern, in dem eine Rückkopplung von Erfolgen und Misserfolgen genutzt wird, um die Qualität
von Behandlungsmaßnahmen zu verbessern und die Kriminalrückfälligkeit nachhaltig zu senken.
... While most criminological research is non-experimental (Dezember et al., 2021), there has been a growing recognition that random allocation is not only necessary for establishing causal effects in evaluation research (Weisburd, 2010), but that a broad scope of criminological topics can benefit from randomized controlled trials (Ridgeway, 2019). This echoes earlier calls to make social science more experimental (Sherman, 2003), including the prediction that "[c]riminology may soon resemble medicine more than economics" (Sherman, 2005, p. 132). ...
Objectives
Pair-matching with random allocation in prospective controlled trials represents a novel and highly rigorous design. First use of the design can be traced to medicine (in 1926) and criminology and the social sciences more generally (in 1935). Beginning with these trials, we examine the subsequent history of matched-pair RCTs (randomized controlled trials), and related attention to stratification prior to randomization, in both criminology and medicine over almost a century to illustrate shared interest in the design’s advantages and disadvantages.
Methods
We draw upon a wide range of historical and contemporary sources, including historical archives and writings on the first trials in criminology and medicine, prior reviews of RCTs and matched-pair RCTs, and searches of selected databases.
Results
The first trials draw attention to key factors that remain central to contemporary use, including concerns about covariate imbalance when randomization is used on its own, potential to improve study power when matching is effective, and the ability to deal with differential attrition in follow-ups. The evolution of the design also shows that the single most important application of matched-pair RCTs is when the units are clusters or places.
Conclusions
Over the twentieth and twenty-first centuries, criminology and medicine have continued to wrestle with methodologies to most efficiently and robustly compare like with like. Both, in this setting, have turned to matched-pair randomization, though less often than its advocates would like. It is this and other shared interests between criminology/social sciences and medicine/public health, including a movement toward evidence-based policy and practice, that help us reimagine possibilities for advancing knowledge and improving public policy.
... Criminologists commonly rely on non-experimental research to conduct quantitative tests of associations (e.g., Kelling, Graif, Korkmaz, & Haran, 2020;Lee & Kim, 2020;Nivette, Echelmeyer, Weerman, Eisner, & Ribeaud, 2021). Although non-experimental research in criminology has produced a variety of important findings (Kelling et al., 2020;Lee & Kim, 2020), concerns remain over the inability to adjust for unknown and unobserved confounders (Dezember et al., 2020;Weisburd, 2010). As such, it is commonplace to question the validity of an estimated association, as the influence of a confounder can bias key statistical estimates (Gelman & Tuerlinckx, 2000;Urbano, Lima, & Hanjalic, 2019). ...
Objectives
The inclusion of variables that: (1) are descendants of unobserved confounders and (2) do not theoretically or empirically cause variation in the constructs of interest in a multivariable regression model can potentially adjust for the bias generated from unobserved confounders. Nevertheless, the validity and utility of descendants for criminological research has yet to be evaluated.
Methods
Two studies were developed to address the gap in the literature. First, a randomly specified directed equation simulation analysis was performed. Second, using data from the Pathways to Desistance study, the technique was implemented to observe if the association between gang involvement and criminal involvement was attenuated after adjusting for exposure to violence (a potential descendant of an unobserved confounder).
Results
The simulation analysis demonstrated that adjusting for the descendants of unobserved confounders can reduce bias in key estimates. The magnitude of the association between gang involvement and criminal involvement was approximately half of the bivariate association after introducing exposure to violence into the model.
Conclusions
The findings suggest that adjusting statistical models for variables that are a descendant of an unobserved confounder and do not cause variation in the association of interest can reduce the bias generated by an unobserved confounder.
... The second problem associated with trying to balance the multivariate world and the parsimonious theoretical explanations is related to the concept of omitted variable bias. The problem of omitted variable bias, an issue frequently neglected in much of the criminological research (Weisburd et al., 2016), speaks to the fact that omitting an important explanatory variable from a regression model may yield problematic parameter estimates (Weisburd, 2010). Specifically, if a key variable that is correlated with the dependent variable (i.e., adhering to the code of silence) is omitted from the model, it could make the results of the entire model highly questionable, especially when correlated with another independent variable in the model (Angrist and Pischke, 2008). ...
This chapter summarizes the most recent calls for defunding and reforming police organizations around the country. A number of high-profile cases that focused primarily on the alleged notions of excessive use of force contributed to the explosion of criticism of the legitimacy of police actions. No time has been more critical in terms of the study of police integrity than now. This importance is further underscored by the need to connect the public outcry, mostly based on anecdotal accounts of police performance, to empirical research that has been in existence for over two decades and has produced valuable and implementable solutions. The pressing need to study the nature of the police code of silence and its resistance to change will be illustrated by our analyses of one case study, expanding the traditional police integrity framework developed by Klockars and colleagues in the past two decades.
... The second problem associated with trying to balance the multivariate world and the parsimonious theoretical explanations is related to the concept of omitted variable bias. The problem of omitted variable bias, an issue frequently neglected in much of the criminological research (Weisburd et al., 2016), speaks to the fact that omitting an important explanatory variable from a regression model may yield problematic parameter estimates (Weisburd, 2010). Specifically, if a key variable that is correlated with the dependent variable (i.e., adhering to the code of silence) is omitted from the model, it could make the results of the entire model highly questionable, especially when correlated with another independent variable in the model (Angrist and Pischke, 2008). ...
This chapter discusses the findings of a case study of a mid-size U.S. police agency in the context of extant research and elaborates on the lessons learned from our study. Based on the empirical study of the contours of the code of silence across behaviors that violate tenets of police integrity, including police corruption, use of excessive force, interpersonal deviance, and organizational deviance, the chapter illustrates the interconnectedness between the code of silence and the police agency’s organizational perspectives. The study emphasizes the role that the police officers’ organizational attitudes play in their willingness to adhere to the code of silence, from their perceptions of how willing other police officers are to report misconduct and the severity of the disciplinary threat that their police agency is making, to their perceptions of self-legitimacy and organizational justice. The chapter concludes the book by offering a discussion of theoretical and policy implications of the findings.
... The second problem associated with trying to balance the multivariate world and the parsimonious theoretical explanations is related to the concept of omitted variable bias. The problem of omitted variable bias, an issue frequently neglected in much of the criminological research (Weisburd et al., 2016), speaks to the fact that omitting an important explanatory variable from a regression model may yield problematic parameter estimates (Weisburd, 2010). Specifically, if a key variable that is correlated with the dependent variable (i.e., adhering to the code of silence) is omitted from the model, it could make the results of the entire model highly questionable, especially when correlated with another independent variable in the model (Angrist and Pischke, 2008). ...
This chapter expands the police integrity approach by focusing on the link between the evaluations of disciplinary fairness and the code of silence. Based on the writings by Klockars and Kutnjak Ivković (The code of silence and the Croatian police. In Pagon M (ed) Policing in central and Eastern Europe: organizational, managerial, and human resource aspects. College of Police and Security Studies, Ljubljana, Slovenia, 329–347, 1998), this chapter presents three potential theoretical approaches hypothesizing the relationship between police officers’ willingness to report misconduct and disciplinary fairness. We rely on the data from one mid-sized police department in the United States to test the effects across 12 scenarios depicting police corruption, use of excessive force, interpersonal deviance, and organizational deviance. Our multivariate models show that perceptions of disciplinary fairness are independently related to the police officers’ willingness to adhere to the code of silence. Discipline that is viewed as too harsh does not entice police officers to report; rather, in such cases, police officers are more likely to say that they would not report than police officers who evaluated discipline as fair. The effects are not as clear for the cases in which police officers evaluated discipline as too lenient.
... The second problem associated with trying to balance the multivariate world and the parsimonious theoretical explanations is related to the concept of omitted variable bias. The problem of omitted variable bias, an issue frequently neglected in much of the criminological research (Weisburd et al., 2016), speaks to the fact that omitting an important explanatory variable from a regression model may yield problematic parameter estimates (Weisburd, 2010). Specifically, if a key variable that is correlated with the dependent variable (i.e., adhering to the code of silence) is omitted from the model, it could make the results of the entire model highly questionable, especially when correlated with another independent variable in the model (Angrist and Pischke, 2008). ...
An emerging body of research suggests that the concept of self-legitimacy plays an important role in various outcomes associated with police officer attitudes and behaviors. In this chapter, we extend this literature by looking at the potential role that self-legitimacy may play in explaining the code of silence. The results suggest that self-legitimacy exerts an inconsistent effect on police officers’ decisions to adhere to the code of silence across the scenarios. Moreover, adding self-legitimacy to the model does not substantively change the effect of the traditional police integrity variables. The chapter concludes with a discussion of theoretical and policy implications of these findings.
... The second problem associated with trying to balance the multivariate world and the parsimonious theoretical explanations is related to the concept of omitted variable bias. The problem of omitted variable bias, an issue frequently neglected in much of the criminological research (Weisburd et al., 2016), speaks to the fact that omitting an important explanatory variable from a regression model may yield problematic parameter estimates (Weisburd, 2010). Specifically, if a key variable that is correlated with the dependent variable (i.e., adhering to the code of silence) is omitted from the model, it could make the results of the entire model highly questionable, especially when correlated with another independent variable in the model (Angrist and Pischke, 2008). ...
This chapter focuses on how the police officers’ willingness to report is shaped by their perceptions of organizational justice. With a few exceptions, bivariate analyses revealed a negative relationship between the adherence to the code of silence and the perceptions of organizational justice. Multivariate analyses indicated that the effect of perceived organizational justice varies across the types of scenarios. While the effect of organizational justice on the adherence to the code of silence is negative for the corruption scenarios, organizational justice exerts an inconsistent relationship on the adherence to the code of silence for organizational deviance scenarios and interpersonal deviance scenarios. Yet, the perceptions of organizational justice seem to have no effect on the use of excessive force scenarios. Additionally, the estimated other police officers’ adherence to the code of silence is the strongest predictor of an officer’s own adherence to the code of silence.
... The second problem associated with trying to balance the multivariate world and the parsimonious theoretical explanations is related to the concept of omitted variable bias. The problem of omitted variable bias, an issue frequently neglected in much of the criminological research (Weisburd et al., 2016), speaks to the fact that omitting an important explanatory variable from a regression model may yield problematic parameter estimates (Weisburd, 2010). Specifically, if a key variable that is correlated with the dependent variable (i.e., adhering to the code of silence) is omitted from the model, it could make the results of the entire model highly questionable, especially when correlated with another independent variable in the model (Angrist and Pischke, 2008). ...
This chapter nests the code of silence within the discussion of police integrity. It starts by presenting an overview of the tenets of police integrity theory and the methodology developed by Klockars CB, Kutnjak Ivković S, Haberfeld MR (2004a) The contours of police integrity. In Klockars CB, Kutnjak Ivković S, Haberfeld M R (eds). The contours of police integrity. Sage, Thousand Oaks, p 1–18.; Klockars CB, Kutnjak Ivković S, Haberfeld MR (2004b) Police integrity in the United States of America. In Klockars CB, Kutnjak Ivković S, Haberfeld M R (eds). The contours of police integrity. Sage, Thousand Oaks, p 265–282. Based on the data from one mid-sized police department in the United States, the chapter examines the extent of the code of silence across 12 different scenarios depicting lapses in police integrity, including police corruption, use of excessive force, organizational deviance, and interpersonal deviance. Our findings show that the strength of the code of silence varies across scenarios and that it is negatively related to how serious misconduct is evaluated to be. The multivariate models indicate that the perceptions of organizational peer culture are the strongest factor affecting the respondents’ own willingness to report. In addition, other factors based on the police integrity theory—seriousness of police misconduct, assessment that the behavior violates official rules, and severity of expected discipline—are all related to the respondents’ expressed unwillingness to adhere to the code of silence.
... The second problem associated with trying to balance the multivariate world and the parsimonious theoretical explanations is related to the concept of omitted variable bias. The problem of omitted variable bias, an issue frequently neglected in much of the criminological research (Weisburd et al., 2016), speaks to the fact that omitting an important explanatory variable from a regression model may yield problematic parameter estimates (Weisburd, 2010). Specifically, if a key variable that is correlated with the dependent variable (i.e., adhering to the code of silence) is omitted from the model, it could make the results of the entire model highly questionable, especially when correlated with another independent variable in the model (Angrist and Pischke, 2008). ...
... Criminologists commonly rely on non-experimental research to conduct quantitative tests of associations (e.g., Kelling et al., 2020;Lee & Kim, 2020;Nivette et al., 2021). Although nonexperimental research in criminology has produced a variety of important findings (Kelling et al., 2020;Lee & Kim, 2020), concerns remain over the inability to adjust for unknown and unobserved confounders (Dezember et al., 2020;Weisburd, 2010). As such, it is commonplace to question the validity of an estimated association, as the influence of a confounder can bias key statistical estimates -slope coefficients, standard errors, standardized coefficients, and p-values -and increase the likelihood of committing a null-hypothesis testing error (Type 1, Type 2, or Type S; Gelman & Tuerlinckx, 2000;Urbano et al., 2019). ...
Objectives: The inclusion of mechanisms that: (1) are a descendant of an unobserved confounder and (2) do not theoretically or empirically cause variation in one or more of the constructs in the association of interest in a multivariate regression model can potentially adjust for the bias generated from unobserved confounders. Nevertheless, the validity and utility of descendants for criminological research has yet to be evaluated.
Methods: Two studies were developed to address the gap in the literature. First, a randomly specified directed equation simulation analysis was performed. Second, using the Pathways to Desistance, the technique was implemented to observe if the association between gang involvement and criminal involvement was attenuated after adjusting for exposure to violence.
Results: The simulation analysis demonstrated that adjusting statistical models for the descendants of unobserved confounders can reduce bias in key regression estimates. The magnitude of the association between gang involvement and criminal involvement was approximately half of the bivariate association after introducing exposure to violence into the model.
Conclusions: The findings suggest that adjusting statistical models for mechanisms that are a descendant of an unobserved confounder and do not cause variation in the association of interest can reduce the bias generated from unobserved confounders.
... Solange die Wirksamkeit einer Maßnahme nicht überzeugend belegt ist, kann daher nicht beurteilt werden, ob eine Nichtbehandlung oder eine Behandlung das größere Übel darstellt. Es gibt keinen inhaltlichen Grund, dass in der Straftäterbehandlung geringere Qualitätsmaßstäbe angelegt werden als in der Medizin oder Pharmakologie, in denen RCTs ein unverhandelbarer Standard der Qualitätskontrolle sind (Weisburd 2010). Allerdings gibt es in Deutschland gesetzliche Bestimmungen, die unter der Prämisse, dass die Behandlungen in jedem Fall wirksam sind, z.B. die Durchführung einer Sozialtherapie oder einer Maßregel der Besserung und Sicherung verbindlich vorschreiben und so eine zufällige Zuweisung von Personen in eine Kontrollgruppe faktisch unmöglich machen. ...
Die empirische Bewährung von Behandlungsprogrammen von Straftätern (insbesondere Sexualstraftätern) wird seit langem kritisch diskutiert. In diesem Kapitel geben wir einen kurzen Überblick über diese Kontroverse. Wir legen dar, dass nur experimentelle Evaluationsstudien mit zufälliger Zuordnung zu einer Behandlungsgruppe und einer Kontrollgruppe (Randomized Controlled Trials, RCTs) Evidenz für die absolute Wirksamkeit eines Behandlungsprogrammes liefern können. Auch methodisch starke Evaluationen der absoluten Wirksamkeit von Interventionen liefern allerdings keine Information darüber, welche Effekte die einzelnen Komponenten komplexer Behandlungsprogramme haben und wie man Behandlungsprogramme verbessern kann. Dazu werden Methoden wie Komponenten-, Dismantling- und korrelative Untersuchungsdesigns vorgestellt, die eine empirische Prüfung der relativen Wirksamkeit einzelner Programmkomponenten erlauben. Um die aktuelle Ungewissheit über die Wirksamkeit von Straftäterbehandlungsprogrammen zu überwinden, wird eine zweistufige Evaluationsstrategie vorgeschlagen: 1) Um die relative Wirksamkeit einzelner Programmelemente zu untersuchen, sollte eine multizentrische und länderübergreifende Evaluationsstudie durchgeführt werden. Die Vielfalt aktuell durchgeführter Behandlungsprogramme kann dabei genutzt werden, um mit multivariat-korrelativen Methoden die Wirkung einzelner Behandlungskomponenten unter spezifischen institutionellen Randbedingen für spezifische Gruppen von Straftätern zu bestimmen. 2) Um die absolute Wirksamkeit der vielfältigen in Deutschland angebotenen Behandlungsprogramme im Vergleich zum Strafvollzug ohne besondere Behandlungsmaßnahmen zu prüfen, gibt es keine überzeugende Alternative zu RCTs, deren Durchführung allerdings schon durch gesetzliche Vorgaben in einigen Vollzugsformen extrem erschwert wird. Das Ziel einer solchen kombinierten Evaluationsstrategie ist die Entwicklung eines selbstkorrigierenden Systems der Behandlung von Straftätern, in dem eine Rückkopplung von Erfolgen und Misserfolgen genutzt wird, um die Qualität von Behandlungsmaßnahmen zu verbessern und die Kriminalrückfälligkeit nachhaltig zu senken.
... aleatoriamente, a cada uma das condições, tal processo permite assegurar que o indivíduo médio que é alvo da intervenção é equivalente ao indivíduo médio que não é alvo de intervenção, em todas as variáveis externas à intervenção(Farrington, 2006;Sherman, 2003;Weisburd, 2010). Este procedimento pretende assegurar que a verificação de qualquer diferença subsequente à intervenção entre os grupos possa ser, com maior certeza, atribuída ao efeito do programa de intervenção e não à influência de fatores externos de ordem biológica, individual, social ou contextual(Fagan, 2013;Farrington, 2006; procurando, assim, estabelecer uma ordem temporal entre causas e efeitos(Currie, 2001;Fagan & Lindsey, 2014;Farrington, 2006;Shadish et al., 2002). ...
... There are more valid procedures than ad hoc quasi-experimental matching techniques, as we detailed under blocking, minimisation and stratification processes (Lum & Yang, 2005;Nagin & Weisburd, 2013;Weisburd, 2010). ...
The fourth book in The SAGE Quantitative Research Kit, this resource covers the basics of designing and conducting basic experiments, outlining the various types of experimental designs available to researchers, while providing step-by-step guidance on how to conduct your own experiment.
As well as an in-depth discussion of Random Controlled Trials (RCTs), this text highlights effective alternatives to this method and includes practical steps on how to successfully adopt them. Topics include:
· The advantages of randomisation
· How to avoid common design pitfalls that reduce the validity of experiments
· How to maintain controlled settings and pilot tests
· How to conduct quasi-experiments when RCTs are not an option
Practical and succinctly written, this book will give you the know-how and confidence needed to succeed on your quantitative research journey.
... However, in circumstances such as incarceration, it is typically not ethical to consider RCTs. 35 For example, there would be an obvious ethical problem in assigning people to arbitrary sentence lengths at random, particularly for serious crimes deserving severe punishment. How many people would give informed consent to being sentenced to five or ten years in prison, at random? ...
In response to prison overcrowding concerns in recent years, many U.S. officials have undertaken efforts to reduce sentence lengths for certain crimes. However, it is unclear how these changes affect recidivism rates. Among the research on incarceration and recidivism, the majority of studies compare custodial with noncustodial sentences, while fewer examine the impact of varying incarceration lengths. This article reviews the research on the latter. Overall, the effect of incarceration length on recidivism appears too heterogeneous to draw universal conclusions, and findings are inconsistent across studies due to methodological limitations. For example, many study samples are skewed toward people with shorter sentences while others include confounds that render results invalid. Of the studies reviewed, some suggested that longer sentences provide additional deterrent benefits in the aggregate, though some studies also had null effects. None suggested a strong aggregate-level criminogenic effect. We argue that a conclusion that longer sentences have a substantial criminogenic effect, large enough to offset incapacitative effects, cannot be justified by the existing literature.
... We therefore believe that a systematic review evaluating existing evidence from randomised controlled trials (RCTs) might be helpful in identifying treatments for reducing drug use and criminal activity in this vulnerable population. In principle, the RCT design eliminates the threat to internal validity providing there is a sufficiently large number of units assigned as the experimental and control conditions and methods to reduce bias are implemented (Weisburd, 2010). Given the importance of relating economic cost to a reduction in drug use and related offending behaviour, good-quality economic evidence will help inform strategies which represent the best use of limited resource (Higgins and Green, 2011). ...
This updated systematic review assesses the effects of pharmacological interventions for drug-using offenders.
Methods
Systematic review protocols and conventions of the Cochrane Collaboration were followed to identify eligible studies. Studies were pooled in a meta-analysis to assess the impact of pharmacological interventions on drug use and criminal activity. An economic appraisal was conducted.
Results
The search strategies identified 22 studies containing 4372 participants. Meta-analyses revealed a small statistically significant mean difference favouring pharmacological interventions relative to psychological interventions in reducing drug use and criminal activity. When comparing the drugs to one another there were no significant differences between those included (methadone versus buprenorphine, naltrexone and cyclazocine).
Conclusion
Overall, the findings of this review suggest that methadone and naltrexone may have some impact on reducing drug use and reincarceration. Individual pharmacological drugs had differing (generally non-significant) effects. One study identified serious adverse events. Three studies reported cost and consequences information sufficient to conduct a full economic analysis but this was not comprehensive enough to be able to make judgements across all treatment options. Full economic analyses should be encouraged. The study findings were limited mainly to male adult offenders.
... Medicine offers helpful cues for many ethical concerns that confront experimental criminology. Criminologists note that some of those cues have calmed the worry that randomised experiments can be conducted ethically (Lum & Yang 2005;Weisburd 2010). But confidence in the ethical viability of criminological experiments generally focuses on issues arising within the experimental design, such as those pertaining to withholding treatment, group assignment based on treatment need, when experiments ought to be discontinued, and the vulnerability of research subjects (e.g., Baunach 1980;Boruch et al. 2000;Erez 1987;Farrington & Welsh 2006;etc.). ...
Experimental criminology promises a public good: when experiments generate findings about criminal justice interventions, everyone benefits from that knowledge. However, experimental criminology also produces a free-rider problem: when experiments test interventions on the units where problems concentrate, only the sample assumes the risk of backfire. This mismatch between who pays for criminological knowledge and who rides on it persists even after traditional critiques of experimental social science are addressed. We draw from medicine and economics to define experimental criminology’s free-rider problem and expose a dilemma. Either we distribute the costs of producing policy-actionable knowledge to the entire beneficiary population or we justify isolating the risk of experimental harm on that class of the population where ethical concerns are most acute.
... This approach is common in studies which seek to measure the outcome of an intervention where the population cannot be randomised or interventions such as health education given to large population groups (Stuart et al., 2009). The emphasis in this type of study is on the effect of the intervention on a group of people without necessarily considering the population variability (Weisburd, 2010). The school children from both schools were selected using stratified random sampling by grade. ...
... The methodological approach of this research is quantitative. This study is a descriptive non-experimental cross-sectional methodological design, since in the social research practice it is difficult to achieve control of all internal and external factors that can invalidate information [30]. Thus, existing situations, not intentionally provoked, are studied here [31]. ...
The incorporation of social networks in university teaching enables new communication channels and opens new channels for training meetings, which also provides a different dynamic to day-to-day teaching, promoting communication initiatives under social responsibility. This article presents an experience of digital innovation through the use of social networks, Twitter in particular. The overall objective of this innovation is to allow communication, to provide information and reflection as a learning tool, where the main reason for the existence of this community is the exchange and intercommunication of lessons related to the different subjects and to enable professional development. It is developed through eminently active pedagogical methods, considering the theory called connectivism. The results obtained corroborate the starting hypothesis that this teaching team had from the beginning, that a methodology enriched with the use of Twitter, a tool that allows complementing the content of the subjects and interacting with them without limitations of space and time, would increase and improve the process of teaching–learning.
... The estimated cost of this programme is $480 per person and delivery to the nearly 16,000 members of the QPS could cost $7.6 million. Conducting an RCT is viewed as the most reliable method for determining whether the treatment is effective (Weisburd 2010). ...
Research Question Did a values education program taught to Queensland police recruits change their attitudes towards police workplace diversity and equality, relative to recruits in the same cohorts who did not received the program? Data A survey designed to measure attitudes towards workplace diversity and related issues was administered three times to 260 police recruits, who were randomly assigned to receive a values education program or not over the 25-week initial police recruit course. The surveys were conducted in week two of the course, at the conclusion of the values education program, and six weeks after the program concluded. Methods Three separate cohorts were split by batch random assignment into experimental and controls, for 132 experimental recruits and 128 controls. Using a variety of validated scales and items, the attitudes of the two groups were compared at all three survey waves, and in comparative longitudinal trends. Findings While the values education program did not improve experimental group recruit attitudes towards diversity in the workplace over time, it protected that group from a clear decline in support for diversity associated with the standard recruit training experience. Because the design was an RCT, the study clearly revealed that the benefit of the program was as a successful buffer against what happened to reduce diversity support among the other recruits. Conclusions The findings show that in at least one police recruit experience, there is a clear shift away from support for diversity by race and gender in the police workplace in the course of initial training. Fortunately, the results also provide at least one possible preventative measure for that problem, in the form of a values education program similar to one used widely in many countries.
... Eine Kausalität zwischen intensiver Bewährungsunterstützung und Legalbewährung kann nur dann festgestellt werden, wenn sich die Legalbewährung von nicht in RUBIKON aufgenommenen, aber geeigneten Probanden bei zufälliger Zuweisung zu Experimental-und Kontrollgruppe negativer darstellt(vgl. Diekmann 2007, S. 337 ff.;Weisburd & Mazerolle 2007;Weisburd 2010). Eine Voraussetzung für die Bildung einer solchen Kontrollgruppe ist die Vergleichbarkeit der rückfallrelevanten Faktoren mit denen der Experimentalgruppe(Köberlein 2006, S. 250 f.; Englmann 2009, S. 187).Ein experimentelles Design konnte hier jedoch nicht durchgeführt werden, da das Modellprojekt bereits vor Beginn der Evaluation ohne Randomisierung, also zufälliger Zuordnung der Projektteilnehmer zu Experimental-und Kontrollgruppe, begonnen hatte. ...
... Since randomized experiments control for confounding factors by design, analyses of experimental data do not require extensive statistical modeling to ensure rival causal influences are identified and controlled (Weisburd 2010). As such, we used independent samples t tests and standardized mean differences (Cohen's d) to test the direct impact of the BWCs on treatment officer outcomes relative to control officer outcomes during the 12-month intervention period. ...
Objectives
Existing research on the effects of body-worn cameras (BWCs) have found largely consistent results regarding direct significant reductions in citizen complaints and often also report reductions in use of force reports. However, few studies have examined possible spillover effects onto untreated officers. This study explicitly tests for direct and spillover effects of BWCs on the civility of police-citizen encounters and police work activities.
Methods
This study assesses the direct effects of BWCs on citizen complaints, police use of force, and police proactivity and discretion during a 1-year randomized controlled trial in the Boston Police Department. Through a simultaneous quasi-experimental design, this study also investigates whether BWC deployment results in spillover effects onto control officers in treated districts as compared to comparison officers in untreated districts.
Results
Findings indicate that the use of BWCs reduces citizen complaints and police use of force but has no appreciable impact on officer activity or discretion. Furthermore, results indicate significant spillover reductions in citizen complaints for control officers in treated districts.
Conclusions
The results of this study suggest that a limited BWC adoption may generate spillover deterrent impacts as officers and citizens perceive an increased threat that inappropriate and illegal behaviors will be captured on video even when BWCs are not actually present during an encounter. Partial BWC implementation seems like a cost-effective alternative to full implementation. However, police executives and policy makers need to think carefully about possible negative externalities generated by uneven BWC coverage.
... That is, PSM is limited to including only observable variables as selection factors. As such, as a quasi-experimental method, PSM is unable to claim a strong assumption of "ignorability" of confounding that can only be made in experimental studies (Boruch 2007;Weisburd 2010). Nevertheless, when there are a large number of identified and relevant covariates, assumptions of equivalence between treatment and comparison groups are reasonable (Shadish 2013;Shadish et al. 2002). ...
Objectives
Despite the prevalence of religious-oriented rehabilitation programs, few studies have assessed the effects of these programs on recidivism. Prior studies have generally focused on Christian-based programs in the USA and also suffer from a range of methodological problems, making it difficult to draw sound conclusions. The current study evaluates the effects of two Jewish-based programs in Israel.
Methodology
Using data provided by the Israeli Prison Services, propensity score matching was used to examine the effects of two religious-oriented rehabilitation programs on recidivism. A sub-sample of prisoners who progressed from the less intensive to the more intensive program was used as a proxy for examining the role of motivation and commitment.
Findings
Compared to non-participants, only those who participated in both programs had a significantly lower risk of recidivism. This sub-sample also had significantly lower recidivism than those who participated in only the less intensive of the two programs. Comparing participants of the two programs, those from the more intensive program had significantly more positive recidivism outcomes than those from the less intensive program.
Conclusions
The theoretical foundations of religious-oriented programs are that causing changes in levels of religiosity can lead to less recidivism. While prior studies have had difficulties in assessing the motivation to change of program participants, our findings provide evidence to support these theoretical underpinnings. That is, when religious-oriented programs succeed in engendering legitimate changes in levels of religiosity, they can have a positive effect on reducing recidivism.
... Evidence-based approaches, including those based in part on randomized controlled trials (RCTs), have emerged as valuable tools to help both theorists and practitioners better understand crime prevention tactics, and their measurable effects in specific environments (Clarke 2008;Hayes 2003;Johns et al. 2017;Sherman 2010;Weisburd 2010;Farrington 2000, 2007). In recent years, practitioners have adopted evidence-based insights from SCP to reduce crime through application of more precisely focused loss prevention tactics and technologies (Hayes 2003;Hayes and Blackwood 2006;Hayes et al. 2012;Hollinger 2019;Welsh and Farrington 2007). ...
Objective
Anti-theft wire-wraps were tested on three high-theft product categories (cordless electric drills, weight loss supplements, and skincare products) in retail stores to estimate protective effects across product categories.
Methods
This study sampled 56 retail locations, stratified into high, medium, and low-inventory loss (also referred to as shrink or shrinkage) stores. Treatment was assigned randomly within strata to retail locations. Each retail location contained anywhere from 9 to 11 different product lines, categorized into three general product categories. The unit of analysis used in this study was the individual product type nested within individual stores (product type per store) and retailer category. A nested mixed effects model design for repeated measures was used to identify the effects of the treatment on retail loss. Four models were fitted: one for overall retail loss, and one for retail loss for each of the three product categories (i.e., cordless electric drills, weight loss supplements, and skincare products).
Results
The interaction effects between treatment and time were only significant at p < .05 for cordless electric drills, but nonsignificant for the other product categories used in this study, and for overall retail loss. Our results demonstrate the effects of anti-theft wraps on in-store retail loss vary by product category.
Conclusion
The tested treatment was conditionally efficacious. While experimental research designs have successfully tested the effectiveness of anti-theft technology in retail environments, these results may not always be generalizable across product types. Further research exploring variable anti-theft treatment effectiveness and mechanisms of action across asset types is necessary to more efficiently reduce theft events.
... More specifically, the possibilities for drawing causal conclusions will be reduced if the experiment and the control group are not sufficiently balanced, if the subjects sense how they are expected to respond and if the variables are unreliably measured (Dunoff & Pollack, 2017). However, strong voices defend the use of RCTs: This design gives the strongest empirical evidence of causal inference since it does not require that all confounding variables are accounted for (Dunoff & Pollack, 2017;Weisburd, 2010). The widespread influence of RCTs in current police research outside of the Scandinavian countries (see e.g. ...
In criminology and criminal justice research, correlated relationships are complex; therefore, it is necessary to understand spuriousness and its potential impacts on research. Spuriousness is often considered the third variable problem: what may appear as causation between two variables is, in reality, the result of a third (spurious) variable not accounted for in the analysis. Criminology and criminal justice research have many examples of this phenomenon, including the infamous ice cream and violent crime example. This chapter examines spurious correlations in detail to elucidate their impact on criminology and criminal justice research. To begin, a brief discussion of causation and correlation sets the groundwork for discussing the two main types of correlation: accidental and genuine. The discussion continues by describing those two subcategories. Finally, the chapter concludes with a discussion on the possible solutions to spuriousness found in research design and statistical analysis.
Experimental criminology is a part of a larger and increasingly expanding scientific research and evidence-based movement in social policy. The essays in this volume report on new and innovative contributions that experimental criminology is making to basic scientific knowledge and public policy. Contributors explore cutting-edge experimental and quasi-experimental methods and their application to important and topical issues in criminology and criminal justice, including neurological predictors of violence, peer influence on delinquency, routine activities and capable guardianship, early childhood prevention programs, hot spots policing, and correctional treatment for juvenile and adult offenders. It is the first book to examine the full scope of experimental criminology, from experimental tests - in the field and in the laboratory - of criminological theories and concepts to experimental and quasi-experimental evaluations of crime prevention and criminal justice interventions.
Compliance, or the behavioral response to legal rules, has become an important topic for academics and practitioners. A large body of work exists that describes different influences on business compliance, but a fundamental challenge remains: how to measure compliance or noncompliance behavior itself? Without proper measurement, it's impossible to evaluate existing management and regulatory enforcement practices. Measuring Compliance provides the first comprehensive overview of different approaches that are or could be used to measure compliance by business organizations. The book addresses the strengths and weaknesses of various methods and offers both academics and practitioners guidance on which measures are best for different purposes. In addition to understanding the importance of measuring compliance and its potential negative effects in a variety of contexts, readers will learn how to collect data to answer different questions in the compliance domain, and how to offer suggestions for improving compliance measurement.
This paper presents an assessment of the existing literature on validity and methodology relevant to crime prevention studies. Reports eligible for inclusion in the review focused on assessing the methodological quality of crime prevention evaluations. A narrative synthesis approach was used to review the included reports to examine how validity considerations are assessed and addressed in criminological impact evaluations. The reports reviewed included substantive discussions of the five types of validity, as well as discussions of interrelated issues of evaluation design, methodological quality scales, and evidence-based registries. We recommend that all crime prevention evaluations address the methodological issues discussed in this article. In addition, policymakers should consume research with a critical eye toward potential validity issues. Where valid evaluations show interventions to be promising, practitioners should make efforts to ensure fidelity in program implementation. Registries should support policymakers and practitioners in identifying and implementing evidence-based policy and programming through providing guidance on choosing interventions aligned to their priorities and settings, understanding the strengths and weaknesses of various strategies and programs and the conditions under which they are successful, and implementing programs and replicating evaluations with fidelity.
Objectives
This paper offers novel experimental evidence that violent crimes can be successfully reduced by changing the situational environment that potential victims and offenders face. We focus on a ubiquitous but understudied feature of the urban landscape—street lighting—and report the first experimental evidence on the effect of street lighting on crime.
Methods
Through a unique public partnership in New York City, temporary street lights were randomly allocated to 40 of the city’s public housing developments.
Results
We find evidence that communities that were assigned more lighting experienced sizable reductions in nighttime outdoor index crimes. We also observe a large decline in arrests indicating that deterrence is the most likely mechanism through which the intervention reduced crime.
Conclusion
Results suggests that street lighting, when deployed tactically, may be a means through which policymakers can control crime without widening the net of the criminal justice system.
This Campbell systematic review compares effects of custodial and non‐custodial sentences on re‐offending. The authors found 14 high‐quality studies, including three randomised controlled trials and two natural experiments.
Imprisonment is no more effective than community‐based sanctions in reducing re‐offending. Despite this evidence, almost all societies across the world continue to use custodial sentences as the main crime control strategy.
In terms of rehabilitation, short confinement is not better or worse than “alternative” solutions.
1 Synopsis
As part of a broad initiative of systematic reviews of experimental or quasi‐experimental evaluations of interventions in the field of crime prevention and the treatment of offenders, our work consisted in searching through all available databases for evidence concerning the effects of custodial and non‐custodial sanctions on re‐offending. For this purpose, we examined, in 2006, more than 3,000 abstracts, and identified more than 300 possibly eligible studies. For the update, nearly 100 additional potentially eligible studies published or completed between 2003 and 2013 have been identified. For the update, 10 matched‐pair design studies and one RCT have been abstracted. One study (Bergman 1976) that, in 2006, had been classified as an RCT turned out, after closer examination, to have been quasi‐experimental with respect to the comparison of the custodial and the non‐custodial groups. As a result, it has been “downgraded” and included among the quasi‐experimental studies in this update.
The findings of the update confirm one of the major results of the first report, namely that the rate of re‐offending after a non‐custodial sanction is lower than after a custodial sanction in most comparisons. However, this is true mostly for quasi‐experimental studies using weaker designs, whereas experimental evaluations and natural experiments yield results that are less favourable to non‐custodial sanctions. It can be concluded that results in favour of non‐custodial sanctions in the majority of quasi‐experimental studies may reflect insufficient control of pre‐intervention differences between prisoners and those serving “alternative” sanctions.
2 Abstract
BACKGROUND
Throughout the Western World, community‐based sanctions have become a popular and widely used alternative to custodial sentences. There have been many comparisons of rates of reconviction among former prisoners and those who have served any kind of community sanction. So far, the comparative effects on re‐offending of custodial and non‐custodial sanctions are largely unknown, due to many uncontrolled variables.
OBJECTIVE
The objective is to assess the relative effects of custodial sanctions (imprisonment) and non‐custodial (“alternative” or “community”) sanctions on re‐offending. By “custodial” we understand any sanction where offenders are deprived of freedom of movement, i.e. placed in a closed residential setting not their home, no matter whether they are allowed to leave these premises during the day or over weekends. Thus, jails and boot camps would be considered “custodial” settings according to the definition adopted here. By “non‐custodial”, we mean any form of sanction that does not involve any deprivation of liberty, such as community work, electronic monitoring, financial or suspended custodial sanctions. Thus, the category of non‐custodial sanctions includes a great variety of punishments that have in common leaving the offender in the community rather than putting him into confinement.
SEARCH STRATEGY
Relevant published and unpublished studies which meet the eligibility criteria have been identified, during the first as well as for the updated review, through multiple sources, including Abstracts, bibliographies, and contacts with experts in several countries. In particular, the following sources have been searched for abstracts: Criminal Justice Abstracts, Criminology and Penology Abstracts, National Criminal Justice Reference Service (NCJRS), Library Catalogues (University of Minnesota), http://www.google.ch. The following keywords have been used to identify relevant studies: Prison, jail, imprisonment, alternative sanctions, house arrest, electronic monitoring, community service, probation, day reporting, fines, shock incarceration, boot camps; further keywords: re‐conviction, re‐offending, self‐reported offenses, recidivism, re‐arrest and re‐incarceration.
ELIGIBILITY CRITERIA
Randomized or natural experiments, as well as quasi‐experimental comparisons between former prison inmates and those who served community sanctions have been included without exception, provided that propensity score matching methods were used. Other quasi‐experimental studies have been included, for the updated as well as the first review, if subject were matched or if three or more potentially relevant independent variables had been controlled for. Studies written in any language and prepared between 1961 and 2013 have been considered for inclusion. For the update, ten studies have been identified and considered that used propensity scores in order to control for pre‐existing differences between custodial and non‐custodial groups.
DATA COLLECTION AND ANALYSIS
A coding protocol has been prepared, following the guidelines of the Campbell Collaboration.
MAIN RESULTS
Although a majority of the selected studies (see Table 2) show non‐custodial sanctions to be more beneficial in terms of re‐offending than custodial sanctions, no significant difference is found in the meta‐analysis based on four controlled and one natural experiments. It should be noted that offences prevented through incapacitation of incarcerated offenders have not been considered in this assessment.
REVIEWERS' CONCLUSIONS
The review has allowed identifying several shortcomings of studies on this subject:
• 1) Controlled experiments are still rare exceptions, although obstacles to randomisation are often less formidable than claimed.
• 2) Follow‐up periods rarely extend beyond two years. Even in cases of controlled trials where later follow‐up studies might be feasible, periods considered rarely extended to significant parts of subjects' biographies.
• 3) Despite alternative (and presumably more valid) measures of re‐offending (such as self‐reports) have become widely available, most studies do not include measures of re‐offending beyond re‐arrest or re‐conviction.
• 4) In most studies, only the occurrence (prevalence) of re‐arrest or reconviction is considered, but not the frequency (incidence) of new offences. Some studies have shown, however, that most offenders reduce offending rates after any type of intervention. Thus, the relevant question may be to what extent improvement differs by type of sanction. Therefore, future studies should look at rates of improvement (or reductions in offending) rather than merely at “recidivism” as such.
• 5) Rehabilitation in other relevant areas, such as health, employment, family and social networks, is rarely considered, despite century‐old claims that short custodial sentences are damaging with respect to social integration in these other areas.
• 6) No study has addressed the possibility of placebo (or Hawthorn) effects. Even in controlled trials, it is not clear to what extent outcomes that favoured “alternative” sanctions were due to the fact that subjects assigned to non‐custodial sanctions may have felt treated more fairly, rather than to specific effects of “alternative” sanctions as such. Given experimental research on neurobiological effects of feelings of fairness (Fehr and Rockenbach, 2003), such a possibility should be envisaged with more attention in future research.
SOURCES OF SUPPORT
The update has been supported by a grant of the Swedish National Council for Crime Prevention. The original review, published in 2006, was supported by a grant of the Swiss National Science foundation.
The chapter illustrates both the objective and the methodology of the research presented in the book. Possible evaluation research designs are reviewed, highlighting their strengths and weaknesses and analysing which one is the more suitable for the purposes of the present research – i.e. a natural experiment with one group(s) post-test only design, using a non-equivalent qualitative pre-test. Then, strategies for data gathering and analysis are presented. In more detail, the methodology used consisted of systematic keyword searches on the three mayor social media platforms – i.e. Twitter, Facebook, and Instagram – after the implementation of the SCP measures in order to identify and analyse contents linked to the illegal online trade of medicines. The keyword list comprises 76 terms of two different types: 36 names of medicines, both generic and branded names, and 8 general terms in 5 languages – i.e. 40 keywords overall – referring to disease or condition and the medicines that are intended to treat them, or their therapeutic category. In order to assess the impact of the SCP interventions, data was analysed ‘vertically’ (i.e. by social media and by therapeutic category) and ‘horizontally’ (i.e. across social media and therapeutic category). Results (presented in Chap. 5) were compared to the situation existing before the implementation of the measures by using six qualitative indicators.
This study examines whether there is variability between the effects of three different drug rehabilitation programs operating in the Israeli Prison Service (IPS) on completers’ recidivism. By performing comparative analysis, this study attempts to address the problem of assessing the role of participants’ motivation. The study uses a rich administrative data obtained from the IPS system to develop a propensity score matching (PSM) approach where the treatment groups consist of only those who completed the programs, and the comparison groups consist of drug-addicted prisoners who have not taken part in any drug rehabilitation program. After matching, prisoners in the treatment and comparison groups are found to be similar on all known characteristics. Findings show that the only rehabilitation program that promised significant and positive outcomes for its completers was the more comprehensive one operating at Hermon Prison. Prisoners who completed the treatment were incarcerated and arrested less than their comparison group. The “golden strategy” for rehabilitating drug-using prisoners, then, will be twofold. The program should be based on the promising components of rehabilitation, that is, cognitive behavioral therapy, therapeutic community, long duration, intensity, and positive social climate. The program should also succeed in retaining its participants through completion.
This article presents a framework that extends the discussion of translational criminology and the dynamic process of translating research to practice. The goal is to provide an explicit dissection of the translation process into four phases to help outline a structured way of thinking about how to incorporate research into police practice. The four-phases include: Phase I: ‘Does it Work?’ Research and Evaluation; Phase II: ‘What Works?’ Synthesis and Dissemination; Phase III: ‘How to Make it Work?’ Implementation and Evaluation; and Phase IV: ‘Make it Work!’ Institutionalization and Sustainability. The process is founded in implementation science and the ‘Knowledge to Action’ model (KTA) used in the medical and public health fields, as well as current translation activities for policing, and the authors’ experience as practitioners, researchers, and ‘translators’ over the last 25 years. It is the hope that parsing out four distinct phases for the translation of research to practice will assist researchers and police leaders to identify and fill gaps in current and future translation activities.
This chapter considers how the outcomes of alternative care and treatment in child protection can be assessed and the potential promise of public health approaches to child maltreatment. Despite decades of research in the field of child welfare, it is not possible to make causal claims about the outcomes of alternative care and treatment in child protection. Randomised control trial research is not ethical or feasible because children may not be randomly allocated to different types of alternative care. Instead, other research designs are relied upon, but there are significant challenges arising from definitional issues, methodological limitations and potential bias. Prospective longitudinal research with population-based samples is best positioned not only to provide information on outcomes associated with alternative care, but also the contributions of risk and protective factors to outcomes. A public health approach to child maltreatment can provide a more systematic approach to achieving and measuring children’s outcomes. Public health approaches are also favoured by political and technological trends. Governments are increasingly adopting a social investment approach, demanding return on investment. The advent of big data enables researchers to track children’s outcomes through linking with data generated by other public systems such as education, health and criminal justice.
Halfway houses are a form of community supervision and correctional programming that have become a staple intervention in recent years. Despite the ingrained belief in their benefits with respect to successful reintegration, this assumption may not be justified based on the existing literature. The current study provides a systematic review and meta-analysis of nine studies examining the effects of halfway houses on recidivism. Overall, the findings suggest that halfway houses are an effective correctional strategy for successful reentry (log odds ratio [LOR] = 0.236, z = 9.27, p < .001). Further work is needed to determine best practices for programming and meeting the needs of different participants.
Day reporting centers (DRCs) are a community-based correctional option that provide punishment and control while simultaneously assisting with rehabilitation and reintegration. The purpose of this study was to examine the effects of DRCs on the criminal recidivism of adult offenders. The current study meta-analyzed nine studies contributing 21 effect sizes (12 of which were independent). The findings show that, overall, DRCs do not differ from traditional supervision options in terms of recidivism outcomes. Additional analyses suggest that the magnitude of the treatment impact varies depending on the type of recidivism outcome measured, with studies limited to conviction showing a positive and significant reduction for DRCs. Further, front end DRCs were related to stronger treatment impact than back end DRCs. Inconsistent reporting of important variables concerning program characteristics and participant characteristics limited our ability to make recommendations for best practices.
Evidence-based policy has much to recommend it, but it also faces significant challenges. These challenges reside not only in the dilemmas faced by policy makers but also in the quality of the evaluation evidence. Some of these problems are most effectively addressed by rigorous syntheses of the literature known as systematic reviews. Other problems remain, including the range of quality in systematic reviews and their general failure to be updated in light of new evidence or disseminated beyond the research community. Based on the precedent established in health care by the international Cochrane Collaboration, the newly formed Campbell Collaboration will prepare, maintain, and make accessible systematic reviews of research on the effects of social and educational interventions. Through mechanisms such as rigorous quality control, electronic publication, and worldwide coverage of the literature, the Campbell Collaboration seeks to meet challenges posed by evidence-based policy.
In the light of continuing debate over the applications of significance testing in psychology journals and following the publication of J. Cohen's (1994) article, the Board of Scientific Affairs (BSA) of the American Psychological Association (APA) convened a committee called the Task Force on Statistical Interference (TFSI) whose charge was "to elucidate some of the controversial issues surrounding applications of statistics including significance testing and its alternatives; alternative underlying models and data transformation; and newer methods made possible by powerful computers" (BSA, personal communication, February 28, 1996). After extensive discussion, the BSA recommended that publishing an article in American Psychologist, as a way to initiate discussion in the field about changes in current practices of data analysis and reporting may be appropriate. This report follows that request. Following each guideline are comments, explanations, or elaborations assembled by L. Wilkinson for the task force and under its review. The report is concerned with the use of statistical methods only and is not meant as an assessment of research methods in general. The title and format of the report are adapted from an article by J. C. Bailar and F. Mosteller (1988). (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Evidence-based policy has much to recommend it, but it also faces significant challenges. These challenges reside not only in the dilemmas faced by policy makers but also in the quality of the evaluation evidence. Some of these problems are most effectively addressed by rigorous syntheses of the literature known as systematic reviews. Other problems remain, including the range of quality in systematic reviews and their general failure to be updated in light of new evidence or disseminated beyond the research community. Based on the precedent established in health care by the international Cochrane Collaboration, the newly formed Campbell Collaboration will prepare, maintain, and make accessible systematic reviews of research on the effects of social and educational interventions. Through mechanisms such as rigorous quality control, electronic publication, and worldwide coverage of the literature, the Campbell Collaboration seeks to meet challenges posed by evidence-based policy.
This study investigated the effectiveness of a popular batterers's intervention program in reducing repeated violence among men who were convicted of misdemeanor domestic violence. A classical experimental design randomly assigned all 404 male defendants in Broward County Courthouse into an experimental (one-year probation and court-mandated counseling) or control (one-year probation only) conditions. The study followed these men for 12 months, collecting information from offenders' self-reports, victims' reports, and official measures of rearrests. No significant differences were found between the experimental and control groups in their attitudes, beliefs, and behaviors regarding domestic violence; both groups were equally likely to engage in both minor and severe partner abuse. In addition, no significant differences were found between the two groups in their rates of rearrest. Further analyses indicated that stake-in-conformity variables (employment and age) predicted both attendance at treatment and reoffending.
The social benefits of randomized controlled trials (RCTs) would be enhanced by general recognition of three problems of their interpretation and a redefinition of their mission in relation to program development and evaluation. One problem is that of “forest versus trees,” or the sampling relationship between each test of a hypothesis and the conclusions drawn from all such tests taken together. A second problem is interpreting RCTs as testing theory or policy when they cannot achieve a high correlation between the treatments assigned and treatments actually applied in each case. The third problem is what works for whom, or whether identical treatments cause different effects, on average, for different kinds of people, groups, situations, or other units of analysis that were different at the point of random assignment. Confronting these three problems suggests that RCTs should not only seek verdicts about what works but also should seek better inventions of crime prevention programs for further testing.
The propensity score is the conditional probability of assignment to a particular treatment given a vector of observed covariates. Both large and small sample theory show that adjustment for the scalar propensity score is sufficient to remove bias due to all observed covariates. Applications include: (i) matched sampling on the univariate propensity score, which is a generalization of discriminant matching, (ii) multivariate adjustment by subclassification on the propensity score where the same subclasses are used to estimate treatment effects for all outcome variables and in all subpopulations, and (iii) visual representation of multivariate covariance adjustment by a two- dimensional plot.
The study assessed 36-month recidivism outcomes for a prison therapeutic community (TC) program with aftercare using an intent-to-treat design with random assignment. Outcomes for 478 felons at 36 months replicated findings of an earlier report on 12- and 24-month outcomes, showing the best outcomes for those who completed both in-prison and aftercare TC programs. At 36 months, 27% of the prison TC plus aftercare completers recidivated, versus 75% for other groups. In addition, a significant positive relationship was found between the amount of time spent in treatment and the time until return for the parolees who recidivated. However, the reduced recidivism rates for in-prison treatment found only at 12 and 24 months was not maintained at 36 months.
This paper reports the results of a randomized experimental evaluation of an innovative drug enforcement strategy developed as part of the Drug Market Analysis Program. Using computer mapping techniques, we identified 56 “hot spots” of drug activity that were randomized in statistical blocks to experimental and control conditions. The experimental strategy followed a stepwise approach that sought to engage business owners and citizens in crime control efforts, to apply pressure to reduce drug and drug-related activity through police crackdowns, and to initiate a maintenance program with the assistance of the patrol division of the department. In line with tactics employed by street-level narcotics units in many other American cities, the control strategy involved unsystematic arrest-oriented narcotics enforcement based on ad hoc target selection. Comparing seven-month pre- and post-intervention periods, we find consistent and strong effects of the experimental strategy on disorder-related emergency calls for service. We also find little evidence of displacement of the crime control benefits of the study to areas near the experimental hot spots. Indeed, through two separate methods, our data suggest a “diffusion of benefits” around the experimental as compared with control locations.
This article considers the ethical problems engendered by random assignment and privacy concerns in randomized field experiments. The particular focus is on procedural, legislative, and technical approaches to reducing or avoiding the problems. Examples are given from a variety of disciplines, although the main emphasis is on research in crime and delinquency.
Increasing attention is being paid to the systematic review and synthesis of evaluations of large-scale, generic, crime prevention programs. The utility of these syntheses rests on the assumption that the programs are designed to work across a wide variety of contexts. But many police problem-solving efforts and situational prevention in- terventions are small-scale efforts specifically tailored to individual contexts. Do evaluation designs and methods applicable to generic programs apply to problem specific programs? Answering this ques- tions requires examining the differences between propensity-based and opportunity-blocking interventions; between internal and external validity; and between the needs of practitioner evaluators and aca- demic researchers. This paper demonstrates that in some common cir- cumstances, weak evaluation designs may have greater utility and produce more generalizable results than very strong evaluation de- signs. This conclusion has important implications for evaluations of
Many criminologists doubt that the dosage of uniformed police patrol causes any measurable difference in crime. This article reports a one-year randomized trial in Minneapolis of increases in patrol dosage at 55 of 110 crime “hot spots,” monitored by 7,542 hours of systematic observations. The experimental group received, on average, twice as much observed patrol presence, although the ratio displayed wide seasonal fluctuation. Reductions in total crime calls ranged from 6 percent to 13 percent. Observed disorder was only half as prevalent in experimental as in control hot spots. We conclude that substantial increases in police patrol presence can indeed cause modest reductions in crime and more impressive reductions in disorder within high crime locations.
Three different 12-month interventions for servicemen who had been substantiated as having physically assaulted their wives were used and the outcomes examined. The 861 couples of the study were randomly assigned to 4 groups: a men's group, a conjoint group, a rigorously monitored group, and a control group. Cognitive-behavioral interventions were implemented for the men's and conjoint groups, and outcome data were gathered from male perpetrators and female victims at roughly 6-month intervals over the approximately 18-month experimental period. Data analyses revealed nonsignificant differences between the experimental groups over a variety of outcome measures.
A feast of randomized controlled trials (RCTs) in medical science and comparative famine in criminology can be explained in terms of cultural and structural factors. Of central importance is the context in which the evaluation of interventions is done and the difference in status of situational research in the two disciplines. Evaluation of medical interventions has traditionally been led by practitioner (clinical) academics. This is not the case in criminal justice, where theory has had higher status than intervention research. Medical science has advanced in, or closely associated with, university teaching hospitals, but links between criminology, and criminal justice services are far more tenuous. The late development of situational crime prevention seems extraordinary from a medical perspective, as does the absence of university police schools in the United Kingdom and elsewhere. These structural and cultural factors explain concentration of expectation, resource, and RCT productivity in medical science. The Campbell Collaboration and the Academy of Experimental Criminology are forces which are reducing this polarization of feast and famine in RCTs. But unless scientific criminology is embedded in university schools which are responsible for the education and training of law, probation, and police practitioners, convergence in terms of RCTs and implementation of findings in practice seems unlikely.
This article discusses advantages of randomized experiments and key issues raised in the following articles. The main concern is the growth and decrease in the use of randomized experiments by the California Youth Authority, the U.S. National Institute of Justice, and the British Home Office, although other experiments are also discussed. It is concluded that feast and famine Periods are influenced by key individuals. It is recommended that policy makers, practitioners, funders, the mass media, and the general public need better education in research quality so that they can tell the difference between good and poor evaluation studies. They might then demand better evaluations using randomized experiments.
The propensity score is the conditional probability of assignment to a particular treatment given a vector of observed covariates. Previous theoretical arguments have shown that subclassification on the propensity score will balance all observed covariates. Subclassification on an estimated propensity score is illustrated, using observational data on treatments for coronary artery disease. Five subclasses defined by the estimated propensity score are constructed that balance 74 covariates, and thereby provide estimates of treatment effects using direct adjustment. These subclasses are applied within sub-populations, and model-based adjustments are then used to provide estimates of treatment effects within these sub-populations. Two appendixes address theoretical issues related to the application: the effectiveness of subclassification on the propensity score in removing bias, and balancing properties of propensity scores with incomplete data.
Randomized controlled trials (RCTs) have been considered a gold standard for health and social service research for generations of professionals. However, even as methods have developed to accommodate a large number of professional perspectives and fields of intervention, few have adequately addressed the complex nature of RCTs conducted in community settings. In this book, Drs. Solomon, Cavanaugh, and Draine draw on their extensive experience conducting randomized controlled trials to compile a practical and accessible guide to RCTs in community-based practice settings. While providing a detailed, common-sense manual, the authors address numerous design and implementation challenges that are unique to practice settings, which are less-controlled environments than the typical clinic or consultation room. Such issues include: Community and agency buy-in to support collaboration, addressing confounds to internal and external validity, and fidelity with complex interventions. These challenges are addressed through a mix of qualitative and quantitative methods that have supported RCT research in community-based settings. This pragmatic guide provides a thorough review of the basic ingredients for working through each step of the RCT process. It offers encouragement and support to enter this richly rewarding and challenging research area.
What Works in Corrections, first published in 2006, examines the impact of correctional interventions, management policies, treatment and rehabilitation programs on the recidivism of offenders and delinquents. The book reviews different strategies for reducing recidivism and describes how the evidence for effectiveness is assessed. Thousands of studies were examined in order to identify those of sufficient scientific rigor to enable conclusions to be drawn about the impact of various interventions, policies and programs on recidivism. Systematic reviews and meta-analyses were performed to further examine these results. This book assesses the relative effectiveness of rehabilitation programs (e.g., education, life skills, employment, cognitive behavioral), treatment for different types of offenders (e.g. sex offenders, batterers, juveniles), management and treatment of drug-involved offenders (e.g., drug courts, therapeutic communities, outpatient drug treatment) and punishment, control and surveillance interventions (boot camps, intensive supervision, electronic monitoring). Through her extensive research, MacKenzie illustrates which of these programs are most effective and why. © Doris Layton MacKenzie 2006 and Cambridge University Press, 2009.
A multistage therapeutic community treatment system has been instituted in the Delaware correctional system, and its effectiveness has captured the attention of the National Institutes of Health, the Department of Justice, members of Congress, and the White House. Treatment occurs in a three-stage system, with each phase corresponding to the client's changing correctional status-incarceration, work release, and parole. In this paper, 18 month follow-up data are analyzed for those who received treatment in: (1) a prison-based therapeutic community only, (2) a work release therapeutic community followed by aftercare, and (3) the prison-based therapeutic community followed by the work release therapeutic community and aftercare. These groups are compared with a no-treatment group. Those receiving treatment in the two-stage (work release and aftercare) and three-stage (prison, work release, and aftercare) models had significantly lower rates of drug relapse and criminal recidivism, even when adjusted for other risk factors. The results support the effectiveness of a multistage therapeutic community model for drug-involved offenders, and the importance era work release transitional therapeutic community as a component of this model.
Understanding of the phenomenon of crime lies at the heart of criminology. A century and a half of theory and research has accumulated, but there does not yet exist an evaluation of how much explanatory power (summarized as the amount of variance explained) there is in criminological research. Examination of empirical tests of criminological theory in Criminology between 1968 and 2005 yields three key findings. The overall level of variance explained is often very low with 80 or 90 percent unexplained. There has been no improvement over time. Individual-based models provide relatively weak explanatory power, but models that took a more crime-specific focus indicated some strength. Criminologists will need to pay much more attention to "what is not explained" in criminological modeling if they are to make significant advances in understanding crime.
Objective:
To evaluate the long-term efficacy of a school-based approach to drug abuse prevention.
Design:
Randomized trial involving 56 public schools that received the prevention program with annual provider training workshops and ongoing consultation, the prevention program with videotaped training and no consultation, or "treatment as usual" (ie, controls). Follow-up data were collected 6 years after baseline using school, telephone, and mailed surveys.
Participants:
A total of 3597 predominantly white, 12th-grade students who represented 60.41% of the initial seventh-grade sample.
Intervention:
Consisted of 15 classes in seventh grade, 10 booster sessions in eighth grade, and five booster sessions in ninth grade, and taught general "life skills" and skills for resisting social influences to use drugs.
Measures:
Six tobacco, alcohol, and marijuana use self-report scales were recorded to create nine dichotomous drug use outcome variables and eight polydrug use variables.
Results:
Significant reductions in both drug and polydrug use were found for the two groups that received the prevention program relative to controls. The strongest effects were produced for individuals who received a reasonably complete version of the intervention--there were up to 44% fewer drug users and 66% fewer polydrug (tobacco, alcohol, and marijuana) users.
Conclusions:
Drug abuse prevention programs conducted during junior high school can produce meaningful and durable reductions in tobacco, alcohol, and marijuana use if they (1) teach a combination of social resistance skills and general life skills, (2) are properly implemented, and (3) include at least 2 years of booster sessions.
This Article analyzes data from the Colorado Springs Spouse Abuse Experiment. In that experiment, suspects apprehended for misdemeanor spouse abuse were assigned at random to one of four treatments: (1) an emergency order of protection for the victim coupled with arrest of the suspect; (2) an emergency order of protection for the victim coupled with immediate crisis counseling for the suspect; (3) an emergency order of protection only; or (4) restoring order at the scene with no emergency order of protection. Outcome measures are taken from official police data and from follow-up interviews with victims. Using Bayesian procedures to take previous experiments into account, the balance of evidence supports a deterrent effect for arrest among "good risk" offenders, who presumably have a lot to lose by being arrested. The balance of evidence is far more equivocal for a "labeling effect" in which an arrest increases the likelihood of new violence.
Data from 300 homicides involving an aggravating felony were examined to determine what factors influence the prosecutor's decision to seek the death penalty. It was found that the race of the victim was significantly related to the decision to seek the death penalty even when several legally relevant factors were taken into account. The data also revealed that black killers of whites were more likely and black killers of blacks less likely to have the death penalty requested. A breakdown of homicides into those involving a single aggravating felony and those involving multiple felonies revealed that racial effects were stronger in the former category. There was some evidence that this difference in the effects of race reflected a different threshold of tolerance for white and black murders. Black victim homicides resulted in a death request only when they crossed a threshold of aggravation that was higher than that found for white deaths.
Eleven jurisdictions across the country are participating in the Intensive Supervision Demon stration Projectfunded by the Bureau of Justice Assistance (BJA). The demonstration is designed to assess the effects — and costs — of sentencing convicted felons to community-based programs. One of the unique aspects of the project is that it involves random assignment of offenders to intensive probation/parole supervision or control program conditions. The demonstration will run until 1990, but it has already provided instructive insights into the issues and problems involved in managing large randomized field experiments in criminal justice. The purpose of this article is to describe the programs and sites participating in the BJA Demonstration Project. The details of the RAND evaluation are then outlined, along with the data collection methods and random assignment procedures. The remaining sections describe and discuss the author's experiences, both positive and negative, in designing and evaluating the demonstration project. The author's hope is that the lessons learned from this experiment will prove instructive and help pave the way toward more refined randomized experiments in the future.
In theory, experimental designs provide the most reliable method to establish a relationship between interventions and outcomes. However, in practice, randomized experiments have remained a much less common choice for criminal justice evaluators than have nonexperimental methods. This article focuses on factors that have traditionally inhibited the use of randomized experiments as a tool for developing criminal justice policy. In this context, the main ethical, political, and practical barriers that face experimenters are described. General principles for identifying circumstances less or more amenable for developing randomized experiments are also defined. In concluding, it is argued that experiments are possible in many circumstances and can provide a powerful tool for developing criminal justice policy.
This article lays out five standards for judging the importance of randomized field trials in estimating the relative effects of new programs and new variations on existing programs. These standards include contemporary evaluation policy, the historical development of trials in diverse sciences, ethics, normative practice, and the credibility of alternative approaches to estimating the effects of programs or variations. Empirical evidence and a line of reasoning bearing on each standard are made plain.
In 1981, the Federal Judicial Center proposed guidelines for balancing risks of experimentation against their supposed benefits. The ethical rules were supposed to be within the scope of American legal and constitutional justification. These guidelines would not fulfill the same criterion in Germany, especially if risks to the individual are weighed against benefits toward the society. Randomization in criminal law cannot be justified by the constitutional principle of freedom of research or by the need to gain empirical knowledge about the effectiveness of sanctions. Unequal treatment in randomized designs can only be justified if randomization is permitted for reasons other than experimentation and if the individuals give their free and informed consent. The principle of informed consent must be extended from a procedural to a material content, which guarantees that participants are not forced to take part in an experiment.
In this article, more general lessons are drawn from two randomized experiments in hot spots policing that the author helped design and implement in the 1990s: the Minneapolis Hot Spots Experiment and the Jersey City Drug Market Analysis Experiment. Using a case study approach, factors that facilitate and inhibit development and implementation of randomized trials are identified with particular focus on the special problems and/or advantages of place-based experiments. While the author's main comments focus on the success of place-based randomized trials in evaluating hot spots policing approaches, he draws insight as well into the reasons why the successful example of experiments in hot spots policing has not inspired similar place-based experimentation in other areas of policing or criminal justice. Eight specific lessons regarding the implementation and development of place-based randomized trials and experimental methods more generally are identified.
This review highlights the importance of recognizing the possibility for doing harm when intentions are good. It describes several examples showing that well-planned and adequately executed programs provide no guarantee for safety or efficacy. The author concludes with recommendations for scientifically credible evaluations to promote progress in the field of crime prevention.
An experiment investigates the effect of changes in one factor (the independent variable) on another (the dependent variable). The independent variable is under the control of the researcher. A randomized experiment is one in which people (or other units) are assigned to conditions according to a table of random numbers, with every person having the same probability of being assigned to each condition. These experiments are especially useful for testing causal hypotheses. Their unique advantage over other methods is their high internal validity, or high ability to demonstrate the effect of one factor on another. The randomization ensures that people assigned to one condition are equivalent in every possible way to those assigned to another condition, within the limits of statistical fluctuation. Despite their methodological advantages, very few randomized experiments have been carried out on crime and justice topics. Most have investigated the effects of providing special help for offenders, and in most cases this help proved no more effective in reducing reoffending than did existing alternative treatment methods. It is difficult to arrange randomized experiments because program administrators are unwilling to relinquish control of assignment to experimenters, and because of ethical problems of denial of treatment. Randomized experiments are most feasible when the effects of a treatment are unknown and when it is impossible to treat everyone. Because of their high internal validity, hypotheses should be tested, and technologies should be evaluated, using randomized experiments whenever possible.
Experiments are known to be powerful tools in assessing the effectiveness of interventions. Notwithstanding this knowledge, randomized field trials have been little used in criminal justice settings. The researchers discuss their experiences implementing two experiments: one in a courthouse and the other in a police agency. Despite the different settings, common problems surfaced in both randomized field trials. The authors discuss obstacles faced in implementing and running experiments and suggest some strategies for avoiding these problems. It is the researchers' hope that such a candid discussion will help others to design and implement stronger experiments.
Does the type of research design used in a crime and justice study influence its conclusions? Scholars agree in theory that randomized experimental studies have higher internal validity than do nonrandomized studies. But there is not consensus regarding the costs of using nonrandomized studies in coming to conclusions regarding criminal justice interventions. To examine these issues, the authors look at the relationship between research design and study outcomes in a broad review of research evidence on crime and justice commissioned by the National Institute of Justice. Their findings suggest that design does have a systematic effect on outcomes in criminal justice studies. The weaker a design, indicated by internal validity, the more likely a study is to report a result in favor of treatment and the less likely it is to report a harmful effect of treatment. Even when comparing randomized studies with strong quasi-experimental research designs, systematic and statistically significant differences are observed.
Volunteering is viewed as a nonrandom event determined by general situational variables and specific personality attributes. Situational variables increasing probability of volunteering, other than having only a relatively less attractive alternative, include: (1) intensity of request to volunteer, (2) perception that others in similar situation would volunteer, (3) acquaintance with, perceived prestige of, and liking for E, (4) interest in subject matter, and (5) subjective probability of subsequently being favorably evaluated by E. Associated personal attributes include greater intellectual ability, interest and motivation, unconventionality, sociability, need for social approval, lower age, and less authoritarianism. Implications are discussed. (51 ref.) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
[This guide] untangles the complexities of randomized field experiments so that researchers can better evaluate the impact of new [educational or social] programs. Beginning with an exploration of how to judge whether or not a program worked (and for how much), [the author] explores the context of experiments; the standards used to judge the ethical propriety of randomized experiments; methods for studying populations; and sampling methods, including pilot tests and arrangements for random allocation. [The author] also covers baseline measures, including when to measure costs; missing data registry; and analyses of quality assurance, including analyses that have to be engineered to suit the departures from the design.
[This book] is for all professionals and practitioners in evaluation/research methods, statistics, psychology, sociology, political science, social work, and public health. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
One of the major changes in juvenile justice during the past decade has been the increased reliance on restitution as a sanction for juvenile offenders. Although a great deal has been learned during the past 10 years about the operation of restitution programs, much remains unknown regarding its impact on recidivism rates. This report contains the results from four random-assignment experiments conducted simultaneously in four communities: Boise, Idaho, Washington, D. C., Clayton County, Georgia, and Oklahoma County, Oklahoma.
In all four studies, youths were randomly assigned into restitution and into traditional dispositions. On the whole, the results show that restitution may have a small but important effect on recidivism. However, not all programs will be able to achieve this effect, either because of program management and strategy, community circumstances, or other factors.
Youths in the restitution groups never had higher recidivism rates than those in probation or detention conditions. In two of the four studies, the juveniles in restitution clearly had fewer subsequent recontacts with the court during the two-to-three-year follow-up.