Article

Justifying the use of non-experimental methods and disqualifying the use of randomized controlled trials: Challenging folklore in evaluation research in crime and justice

Authors:
To read the full-text of this research, you can request a copy directly from the author.

Abstract

The key limitation of non-experimental evaluation methods is that they require an assumption that all confounding factors related to treatment are identified in the statistical models developed. The key advantage of randomized experiments is that this assumption can be relaxed. In this paper, I describe and explain why this assumption is so critical for non-experiments and why it can be ignored in randomized controlled trials (RCTs). I also challenge what I describe as “folklores” that are used to justify the use of non-randomized studies despite this statistical limitation, and to justify the failure of evaluation researchers in crime and justice to use randomized experiments despite their unique ability to overcome this limitation. I conclude by reinforcing what Joan McCord had argued after a life time of review of evaluations: “(W)henever possible” evaluation studies “should employ random assignment.” KeywordsRandomized experiments-Randomized controlled trials (RCTs)-Non-experimental methods-Evaluation studies-Confounding variables

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the author.

... Although the actual impact on policy was variable (see, for example, Reuter, 2013), this embrace of rigorous methodologies and support for EBP based on the 'highest standards of science' (Welsh and Farrington, 2012: 129) laid the foundation for further increases in the rate, scope and methodological rigour of criminological research over the next dec- ades (Farrington, 2001;Sherman et al., 1998;Visher and Weisburd, 1997). Randomized experiments quickly became the 'gold standard' of evaluation research (see Farrington, 2003b;Sherman, 2003;Weisburd, 2003), as well-designed RCTs provide rigorous evi- dence of between-group equivalence, internal validity and the limitation of biases ( Boruch et al., 2000a;Farrington, 2003b;Weisburd, 2000Weisburd, , 2010. Failure to use RCTs, Sherman (2003) argues, may therefore undermine conclusions about the efficacy of pro- grammes and mislead the processes through which policy-makers select programmes for implementation. ...
... A 'gold standard' experiment requires more than simple random assignment; limiting attrition, maintaining between-group independence, delivering consistent treatment and collecting data from the field are essential, yet sometimes hard to achieve in practice. For instance, service providers may object to randomized designs when they believe that a beneficial programme is being withheld (for example, Cook and Payne, 2002) and RCTs can be expensive and time-consuming (Weisburd, 2000). 2 It is also important to acknowledge that causal estimates may not always be neither necessary or sufficient to test predictions from theories or influence public policy (Sampson, 2010;Weisburd, 2010) and, in in some cases, quasi-experimental designs may provide virtually identical results as field experiments ( Berk et al., 2010). The proclaimed superiority of RCTs as compared with other research designs has been criticized as overreaching and overly generalized (Berk, 2005;Sampson, 2010). ...
... First, it points to an interesting discrepancy between the overt and public goals of EBP and an absence of 'typical' evidence-based research designs. Internationally, experiments have become a key component of the movement towards EBPs (Sherman, 2009;Weisburd, 2010), and key actors in Scandinavia have demonstrated some formal support for the goals of evidence-based criminal justice policies and practices (for example, The Danish Crime Prevention Council, 2014; The Norwegian Ministry of Justice and the Police, 2008; The Swedish Prison and Probation Service, 2016). This has, however, not translated into the implementation of RCTs to date. ...
Article
Randomized controlled trials are reported on with increasing frequency within the criminological literature. This development, which is commonly seen as being a part of a global shift towards evidence-based policies, relies heavily on reviews of American research. However, other regions face distinct challenges and employ distinct policy solutions, potentially undermining the uniformity of this trend. In particular, the Scandinavian nations (Denmark, Norway and Sweden), with distinct penal philosophies, may offer a counter-narrative. Here, we conduct a multi-lingual systematic review of crime-related experiments in Scandinavia. Findings show that only eight experiments with an offending or delinquency outcome were published before 2015, six of which focused primarily on medical or psychological treatments. We suggest this distribution is driven by unique, regional epistemological traditions and conclude by outlining distinctive opportunities for experimental criminology in Scandinavia.
... There is a consensus among researchers that experimental designs improve internal validity and reduce selection bias (Burtless, 1995;Heckman and Smith, 1995). Four main characteristics of this research strategy make it best suited for establishing causality (Burtless, 1995;Heckman and Smith, 1995;Weisburd, 2010;Weisburd et al., 2013): ...
... This suggests that RCTs are better able to rule out selection effects and more accurately depict the program's impact (Weisburd et al., 2001). Weisburd (2010) also points out that the amount of "variance explained" using quasi-and nonexperimental designs is low. He warns that many potentially confounding variables are missing from such designs and that researchers are dismissing the systematic nature (and importance) of these missing variables. ...
... QEDs use statistical methods, time-series analyses, matching strategies, natural experiments and other approaches to produce equivalency in relevant characteristics prior to treatment implementation. Although QEDs improve causal inference in many ways, they are still unable to rule out confounding influences from unmeasured variables (Weisburd, 2010). ...
... The randomized controlled trial (RCT) is generally accepted as the best design for a valid assessment of intervention effects (Cook, 2003). The design is usually described as the "gold standard" and the only allocation method capable of controlling unknown confounders (Shadish & Cook, 2009;Weisburd, 2010). In RCTs, individuals or clusters, such as schools, are typically assigned randomly to a treatment or a control group (Shadish, Cook, & Campbell, 2002). ...
... Selection refers to bias in the way that participants or clusters are selected or assigned so that groups systematically possess different characteristics prior to the intervention or that groups are not representative of a population. Thus, the internal validity of the study, or the causal relationship between the intervention and the outcomes, may be questioned when a selection bias occurs (Deeks et al., 2003;Shadish et al., 2002;Ukoumunne et al., 1999;Weisburd, 2010). History refers to specific events occurring between the pre-and posttest independently of the intervention that can alter or affect the participants' performance or ratings and thereby influence the study results. ...
... In order to effectively prevent misconduct and school failure, teachers and politicians need evidence-based knowledge, which should be based on highly credible research (Ogden, 2012). Whenever feasible, the RCT design should be given priority over other designs (Deeks et al., 2003;Shadish & Cook, 2009;Weisburd, 2010), but when RCT designs are not possible, initiatives to reduce or eliminate threats to the validity introduced by nonrandomized approaches should be incorporated in the design of the study Shadish et al., 2002;Shadish, 2011). Knowledge stemming from RCTs and advanced nonrandomized experiments with high validity should thus be used to inform practice and policy. ...
Article
This paper reviews literature on the rationale, challenges, and recommendations for choosing a nonequivalent comparison (NEC) group design when evaluating intervention effects. After reviewing frequently addressed threats to validity, the paper describes recommendations for strengthening the research design and how the recommendations were implemented in a current study. Next, tests were performed in order to investigate whether the added design components had contributed to increased internal validity by creating comparable cluster groups at baseline. In order to estimate the external validity or the potential generalizability of results, the statistical power of the study was calculated up front and representativeness of the participating schools in relation to other schools in Norway was examined. Baseline comparisons indicated that the addition of several design elements, such as the random invitation of multiple, stratified, and active comparison groups, may have reduced significant validity threats stemming from selection bias. The need for more accurate and reliable effect estimates in school-based evaluation research is discussed. © 2014
... This design is often referred to as the "gold standard" as it has robust internal validity and arguably, best tells whether an intervention effect is due to the intervention (Macdonald, 2000). The randomization process allows for the removal of suspicion of systematic biases in the trial (Weisburd, 2010). Although some research suggests that quasi-experimental designs can produce similar results as RCTs, these comparisons have not yet been made in criminological interventions (Weisburd, 2010). ...
... The randomization process allows for the removal of suspicion of systematic biases in the trial (Weisburd, 2010). Although some research suggests that quasi-experimental designs can produce similar results as RCTs, these comparisons have not yet been made in criminological interventions (Weisburd, 2010). One such study compared results from high-quality nonrandomized studies with randomized experiments and found that the results significantly differed, with "weaker" designs being more likely to favor treatment (Weisburd, Lum, & Petrosino, 2001). ...
Article
Full-text available
The aim of this review is to assess the effectiveness of reentry programs designed to reduce recidivism and ensure successful reintegration among adult, male offenders. Studies were included if they (a) evaluated a reentry program incorporating elements dealing with the transition from prison to community for adult, male offenders; (b) utilized a randomized controlled design; and (c) measured recidivism as a primary outcome. In addition, secondary outcomes measures of reintegration were also included. The systematic search of 8,179 titles revealed nine randomized controlled evaluations that fulfilled eligibility criteria. The random-effects meta-analysis for rearrest revealed a statistically nonsignificant effect favoring the intervention (odds ratio [OR] = 0.89, 95% confidence interval [CI] [0.74, 1.07]). Similar results were found for reconviction (OR = 0.94, 95% CI [0.77, 1.12]) and reincarceration (OR = 0.90, 95% CI [0.78, 1.05]). Studies reported mixed results of secondary outcomes of reintegration. The results of this review reflect the variability of findings on reducing recidivism. The challenges faced in conducting this review highlight a need for further research and theory development around reentry programs.
... Without further elaborating, some authors have questioned the possibility of rigorously studying programs, such as the Sherbrooke's co-response team, implemented in real-world settings (Compton et al. 2008;Helfgott et al. 2016). Conversely, Weisburd (2010) have shown that RCTs can be implemented in real-world settings; feasibility concerns are mainly based in folklore rather than on scientific evidence. In the present study, the evaluation was simultaneously planned with the implementation of the program, precluding any randomization. ...
... Based on the available evidence, conducting RCTs to assess the impact of co-response programs seems more than feasible. Such a trial would not affect the autonomy or authority of police officers, which could facilitate the acceptance of an experimental design (Weisburd 2000(Weisburd , 2010. The dispatcher could randomly assign calls to the MCIT or the "treatment as usual" condition. ...
Article
Full-text available
Objectives Assess the capability of a mobile crisis intervention team (MCIT) to connect emotionally disturbed people (EDP) with community resources and decrease police use-of-force. Method In order to have equivalent groups, interventions managed by the MCIT were matched to incidents handled by traditional police officers with similar propensity scores. Results Average treatment effects (ATEs) were computed to assess the impact of the MCIT. The MCIT was associated with decreases in police use-of-force (ATE=-0.08; p≤0.01), EDP transported to the hospital against their will (ATE=-0.06; p≤0.10), and EDP transported to the hospital in general (ATE=-0.42; p≤0.01). EDP were more likely to be referred to community resources (ATE=0.19; p≤0.01) or managed by their social network (ATE=0.22; p≤0.01) when the MCIT was involved in the intervention. Conclusion The MCIT was effective in connecting EDP with community resources, avoiding unnecessary transports to the hospital, and reducing police use-of-force.
... Researchers have examined the use of experiments in specific criminal justice areas and consistently find that experiments are in the minority of methods deployed (Farrington and Welsh 2006;Weisburd 2003;Weisburd 2010;Weisburd and Hinkle 2012). For example, only 11.4% of published evaluations on disciplinary boot camps used experimental methods, and less than 10% of drug court and drug treatment evaluations used experimental designs (Mackenzie et al. 2001;Mitchell et al. 2005;Wilson et al. 2005). ...
... For example, place-based experiments have been used to evaluate policing interventions by hot spots (Braga et al. 2013;Farrington and Welsh 2006;Weisburd 2005) and to evaluate prevention and treatment program contexts (Farrington 2008;Farrington and Welsh 2006). Encouraging the use of place-based designs in other settings (e.g., probation supervision, prison-based programs, school-based programs) could increase the use of experimental evaluations across criminology (Weisburd 2010). ...
Article
Full-text available
Objectives Despite experiments being termed the “gold standard,” criminology has been slow to adopt experimentation as a methodology. The goal of this research note is to better understand the use of experiments in criminology and the potential barriers in implementing this methodology.Methods We conducted a systematic assessment of experiments in Criminology and Justice Quarterly by reviewing every article published since the inception of the journals and coding for numerous elements (i.e., field versus lab studies, research areas, outcomes, and funding sources).ResultsDespite a general increase, experiments make up only 2.8% (n = 44) of studies in Criminology and 3.1% (n = 31) of studies in Justice Quarterly.Conclusions While it is not always clear why experiments are not used more often, we explore the possible reasons for a lack of experimentation (i.e., a lack of mentoring, challenges to practitioner buy in, etc.) and areas for future research.
... Despite several important limitations, experimental evidence continues to serve as the gold-or at the very least the bronze (Berk 2005)-standard on the evidentiary hierarchy in the social and behavioral sciences (Banerjee and Duflo 2009;Imbens 2010;Weisburd 2010). 1 The fact that random assignment, in expectation, creates comparable treatment and control groups and allows us to credibly ascribe causality to group-based differences is a feature that has been appreciated since at least the 18th century (Dunn 1997) but which was popularized in the early 20th century by the work of Jerzy Neyman and Fisher (Fisher 1936;Splawa-Neyman et al. 1923). Recently, the foundational importance of experiments has been bolstered by concerns about the limitations of observational research, with an array of evidence from econometrics and statistics suggesting that experimental treatment effects cannot be reliably recovered using some of the most ubiquitous types of observational research designs (LaLonde 1986;Smith andTodd 2001, 2005;Rubin 2008;Gordon et al. 2019) even when analysts have a rich set of covariates upon which to condition (DiNardo and Pischke 1997;Arceneaux et al. 2010). ...
... Recently, the foundational importance of experiments has been bolstered by concerns about the limitations of observational research, with an array of evidence from econometrics and statistics suggesting that experimental treatment effects cannot be reliably recovered using some of the most ubiquitous types of observational research designs (LaLonde 1986;Smith andTodd 2001, 2005;Rubin 2008;Gordon et al. 2019) even when analysts have a rich set of covariates upon which to condition (DiNardo and Pischke 1997;Arceneaux et al. 2010). Reflecting these concerns, Weisburd (2010) has, channeling a sentiment found in McCord (2003) suggested that "whenever possible, evaluation studies should employ random assignment." ...
Article
Full-text available
Objective The increasing availability of large administrative datasets has led to an exciting innovation in criminal justice research—using administrative data to measure experimental outcomes in lieu of costly primary data collection. We demonstrate that this type of randomized experiment can have an unfortunate consequence: the destruction of statistical power. Combining experimental data with administrative records to track outcomes of interest typically requires linking datasets without a common identifier. In order to minimize mistaken linkages, researchers often use stringent linking rules like “exact matching” to ensure that speculative matches do not lead to errors in an analytic dataset. We show that this, seemingly conservative, approach leads to underpowered experiments, leaves real treatment effects undetected, and can therefore have profound implications for entire experimental literatures. Methods We derive an analytic result for the consequences of linking errors on statistical power and show how the problem varies across combinations of relevant inputs, including linking error rate, outcome density and sample size. Results Given that few experiments are overly well-powered, even small amounts of linking error can have considerable impact on Type II error rates. In contrast to exact matching, machine learning-based probabilistic matching algorithms allow researchers to recover a considerable share of the statistical power lost under stringent data-linking rules. Conclusion Our results demonstrate that probabilistic linking substantially outperforms stringent linking criteria. Failure to implement linking procedures designed to reduce linking errors can have dire consequences for subsequent analyses and, more broadly, for the viability of this type of experimental research.
... Political, ethical, and/or practical concerns sometimes lead evaluators to conclude that random assignment is not feasible when evaluation plans are made (see, e.g., Clarke and Cornish 1972;Erez 1986). The continued growth of criminological experiments in a broad range of real-world settings that have been carried out in an ethical manner demonstrates that many of these concerns are, in most cases, based in folklore rather than facts (Weisburd 2010). Nevertheless, quasiexperiments are often used as alternative approaches in program evaluations. ...
... In addition, it is also important to obtain sound knowledge about what findings can be safely generalized. Although studies with nonexperimental and internally less valid designs may particularly emphasize external validity, they have the same problems of generalization (e.g., Weisburd 2010). For these and other reasons, policy makers and practitioners should seek to carry out experiments whenever feasible and preferably before implementing expensive programs. ...
Article
Full-text available
Objectives This article summarizes key points made in a panel at the American Society of Criminology (ASC) meeting in Atlanta in November 2018, entitled “20th Anniversary of the Academy of Experimental Criminology (AEC): Looking Back and Forward,” organized by Friedrich Lösel as the AEC president. Method Seven (current and former) presidents of AEC contribute short papers about the past and future of experimental criminology, focusing on different and emerging areas of criminological experimentation, as well as identifying topics that require more attention in future, including field experiments and experimental neurocriminology. Results This article informs readers about the history of AEC, its links with the Journal of Experimental Criminology, current issues, and potential future developments in experimental criminology. It also briefly deals with arguments that question whether experiments are the “gold standard,” which were addressed by Daniel Nagin and Robert Sampson in another ASC session at Atlanta. Experimental panel members did not view randomized controlled trials (RCTs) as a ritualized and general “gold standard” for criminological research, because many important topics cannot be investigated in this type of design. Conclusions This article is not intended to be a missionary statement for RCTs, but it does argue that experiments should be used whenever feasible, because they are most robust in ensuring internal validity as the basis for external validity and for generalizations that are necessary for effective practice and policy making.
... Criminologists commonly rely on non-experimental research to conduct quantitative tests of associations (e.g., Kelling, Graif, Korkmaz, & Haran, 2020;Lee & Kim, 2020;Nivette, Echelmeyer, Weerman, Eisner, & Ribeaud, 2021). Although non-experimental research in criminology has produced a variety of important findings (Kelling et al., 2020;Lee & Kim, 2020), concerns remain over the inability to adjust for unknown and unobserved confounders (Dezember et al., 2020;Weisburd, 2010). As such, it is commonplace to question the validity of an estimated association, as the influence of a confounder can bias key statistical estimates (Gelman & Tuerlinckx, 2000;Urbano, Lima, & Hanjalic, 2019). ...
Article
Objectives The inclusion of variables that: (1) are descendants of unobserved confounders and (2) do not theoretically or empirically cause variation in the constructs of interest in a multivariable regression model can potentially adjust for the bias generated from unobserved confounders. Nevertheless, the validity and utility of descendants for criminological research has yet to be evaluated. Methods Two studies were developed to address the gap in the literature. First, a randomly specified directed equation simulation analysis was performed. Second, using data from the Pathways to Desistance study, the technique was implemented to observe if the association between gang involvement and criminal involvement was attenuated after adjusting for exposure to violence (a potential descendant of an unobserved confounder). Results The simulation analysis demonstrated that adjusting for the descendants of unobserved confounders can reduce bias in key estimates. The magnitude of the association between gang involvement and criminal involvement was approximately half of the bivariate association after introducing exposure to violence into the model. Conclusions The findings suggest that adjusting statistical models for variables that are a descendant of an unobserved confounder and do not cause variation in the association of interest can reduce the bias generated by an unobserved confounder.
... Evidence-based approaches, including those based in part on randomized controlled trials (RCTs), have emerged as valuable tools to help both theorists and practitioners better understand crime prevention tactics, and their measurable effects in specific environments (Clarke 2008;Hayes 2003;Johns et al. 2017;Sherman 2010;Weisburd 2010;Farrington 2000, 2007). In recent years, practitioners have adopted evidence-based insights from SCP to reduce crime through application of more precisely focused loss prevention tactics and technologies (Hayes 2003;Hayes and Blackwood 2006;Hayes et al. 2012;Hollinger 2019;Welsh and Farrington 2007). ...
Article
Full-text available
Objective Anti-theft wire-wraps were tested on three high-theft product categories (cordless electric drills, weight loss supplements, and skincare products) in retail stores to estimate protective effects across product categories. Methods This study sampled 56 retail locations, stratified into high, medium, and low-inventory loss (also referred to as shrink or shrinkage) stores. Treatment was assigned randomly within strata to retail locations. Each retail location contained anywhere from 9 to 11 different product lines, categorized into three general product categories. The unit of analysis used in this study was the individual product type nested within individual stores (product type per store) and retailer category. A nested mixed effects model design for repeated measures was used to identify the effects of the treatment on retail loss. Four models were fitted: one for overall retail loss, and one for retail loss for each of the three product categories (i.e., cordless electric drills, weight loss supplements, and skincare products). Results The interaction effects between treatment and time were only significant at p < .05 for cordless electric drills, but nonsignificant for the other product categories used in this study, and for overall retail loss. Our results demonstrate the effects of anti-theft wraps on in-store retail loss vary by product category. Conclusion The tested treatment was conditionally efficacious. While experimental research designs have successfully tested the effectiveness of anti-theft technology in retail environments, these results may not always be generalizable across product types. Further research exploring variable anti-theft treatment effectiveness and mechanisms of action across asset types is necessary to more efficiently reduce theft events.
... More specifically, the possibilities for drawing causal conclusions will be reduced if the experiment and the control group are not sufficiently balanced, if the subjects sense how they are expected to respond and if the variables are unreliably measured (Dunoff & Pollack, 2017). However, strong voices defend the use of RCTs: This design gives the strongest empirical evidence of causal inference since it does not require that all confounding variables are accounted for (Dunoff & Pollack, 2017;Weisburd, 2010). The widespread influence of RCTs in current police research outside of the Scandinavian countries (see e.g. ...
... While most criminological research is non-experimental (Dezember et al., 2021), there has been a growing recognition that random allocation is not only necessary for establishing causal effects in evaluation research (Weisburd, 2010), but that a broad scope of criminological topics can benefit from randomized controlled trials (Ridgeway, 2019). This echoes earlier calls to make social science more experimental (Sherman, 2003), including the prediction that "[c]riminology may soon resemble medicine more than economics" (Sherman, 2005, p. 132). ...
Article
Full-text available
Objectives Pair-matching with random allocation in prospective controlled trials represents a novel and highly rigorous design. First use of the design can be traced to medicine (in 1926) and criminology and the social sciences more generally (in 1935). Beginning with these trials, we examine the subsequent history of matched-pair RCTs (randomized controlled trials), and related attention to stratification prior to randomization, in both criminology and medicine over almost a century to illustrate shared interest in the design’s advantages and disadvantages. Methods We draw upon a wide range of historical and contemporary sources, including historical archives and writings on the first trials in criminology and medicine, prior reviews of RCTs and matched-pair RCTs, and searches of selected databases. Results The first trials draw attention to key factors that remain central to contemporary use, including concerns about covariate imbalance when randomization is used on its own, potential to improve study power when matching is effective, and the ability to deal with differential attrition in follow-ups. The evolution of the design also shows that the single most important application of matched-pair RCTs is when the units are clusters or places. Conclusions Over the twentieth and twenty-first centuries, criminology and medicine have continued to wrestle with methodologies to most efficiently and robustly compare like with like. Both, in this setting, have turned to matched-pair randomization, though less often than its advocates would like. It is this and other shared interests between criminology/social sciences and medicine/public health, including a movement toward evidence-based policy and practice, that help us reimagine possibilities for advancing knowledge and improving public policy.
... Police scholars are increasingly focused on using randomized, controlled experiments because comparisons of experimental and control conditions permit inferences about causality (Weisburd 2010). Here, we conduct a survey-based experiment, a particularly useful method for understanding community perceptions following the adoption of a new policy by an agency. ...
Article
Objectives We examine the extent to which individuals' knowledge of an advanced police technology (license plate recognition or "LPR") may impact perceptions of police. Technologies with the capacity to track individuals' movements are becoming increasingly common in police practice. Although these technologies may yield positive benefits, their use may also heighten community concerns about increased surveillance, data storage, and data security, thereby potentially negatively impacting community-police relationships. Methods We utilize a survey-based experiment with randomized assignment of participants (n=405) to investigate the impact of individuals' knowledge of LPR use on a variety of police perceptions, including trust in police, community approval, respect for citizens, and respect for individual rights. Results Most respondents were unaware of LPR use prior to the survey. When compared with a control group, respondents who encountered brief mentions of LPR functions on the survey expressed significantly lower levels of trust in police. Additionally, "strong agreement" with other positive statements about police also appears to have declined in this sample in response to LPR information. Notably, the sample contained high pre-existing levels of trust and support for police, factors which may have moderated the impacts of LPR information. Conclusions These results support the hypothesis that awareness of LPR use may negatively impact perceptions of police, including trust in police. More generally, although technologies like LPR represent technological innovations, they may also yield unintended consequences, including the potential to undermine police-community relations if adoption decisions are not accompanied by sufficient transparency or community support.
... The first challenge centers around the effect size that BWCs have on the outcome of interest. Studies that employ RCT designs purportedly have the highest level of internal validity (Weisburd, 2010), and the RCT research reviewed here generally demonstrates that BWCs had a significant effect on the outcome(s), although this effect can vary by officer discretion of when to turn the camera on/off (Ariel et al., 2016b). The need for additional effect sizes derived from RCTs (or from rigorous quasi-experimental designs) is especially important in the realm of BWCs because police agencies need to know their likely return on investment. ...
Article
Full-text available
Purpose: The purpose of this paper is to review the extant of the published literature on body-worn cameras (BWCs) in policing, specifically in the context of how BWCs affect both citizens and officers. Design/methodology/approach: The current study is a narrative review of the impact of BWCs on police and citizens generated through a search of four repositories (Google Scholar, Criminal Justice Abstracts, EBSCO Host, PsychInfo). Findings: The current narrative review identified 21 articles that matched the selection criteria. In general, this body of research demonstrates that: the police are supportive of BWC adoption; the evidence from BWC evaluations suggests that the use of BWCs can have benefits for police-public encounters. Practical implications: The practical implications derived from this narrative review suggest police administrators that the adoption and effective implementation of BWCs are one mechanism that can strengthen police-community relationships and decrease police misconduct through enhanced legitimacy and accountability. Originality/value: This study is useful for researchers who wish to further examine BWC issues in policing, for police managers/administrators who are currently utilizing BWC technology, and for those who are considering adopting BWC technology.
... The estimated cost of this programme is $480 per person and delivery to the nearly 16,000 members of the QPS could cost $7.6 million. Conducting an RCT is viewed as the most reliable method for determining whether the treatment is effective (Weisburd 2010). ...
Article
Full-text available
Research QuestionDid a values education programme taught to Queensland police recruits change their attitudes towards police workplace diversity and equality, relative to recruits in the same cohorts who did not receive the programme? DataA survey designed to measure attitudes towards workplace diversity and related issues was administered three times to 260 police recruits, who were randomly assigned to receive a values education programme or not over the 25-week initial police recruit course. The surveys were conducted in week two of the course, at the conclusion of the values education programme and six weeks after the programme concluded. Methods Three separate cohorts were split by batch random assignment into experimental and controls, for 132 experimental recruits and 128 controls. Using a variety of validated scales and items, the attitudes of the two groups were compared at all three survey waves and in comparative longitudinal trends. FindingsWhile the values education programme did not improve experimental group recruit attitudes towards diversity in the workplace over time, it protected that group from a clear decline in support for diversity associated with the standard recruit training experience. Because the design was a randomized controlled trial (RCT), the study clearly revealed that the benefit of the programme was as a successful buffer against what happened to reduce diversity support among the other recruits. Conclusions The findings show that in at least one police recruit experience, there is a clear shift away from support for diversity by race and gender in the police workplace in the course of initial training. Fortunately, the results also provide at least one possible preventative measure for that problem, in the form of a values education programme similar to one used widely in many countries.
... These and other examples suggest that the issue of differentiation in replication is not only relevant for developmental prevention and offender treatment. Rossi (1978Rossi ( , 1987 rightly emphasized the importance of methodologically sound evaluations and criminologists repeatedly have underlined the need for more RCTs (e.g., Farrington 2003;Weisburd 2010). The Academy of Experimental Criminology, the ASC Division of Experimental Criminology, and the Campbell Crime and Justice Collaboration promote this aim. ...
Article
Full-text available
Recent publications in Nature, Science, and other journals raised concerns about the reproducibility of empirical findings in psychology and other scientific disciplines. This article summarizes some of these arguments and results that led to discussions about a “replication crisis” in research. In criminology, there is not yet a similar discussion, although the need for more replications has been emphasized in the past. The present article addresses this topic with special consideration of program evaluations in early developmental crime prevention and offender treatment. In both fields, there has been substantial progress in research and practice. Most systematic reviews showed mean positive effects; however, nearly all of them demonstrated very heterogeneous findings that could not be attributed to the content of programs. This does not allow simple recommendations of “what works” for policy-making and practice. In addition, there is a serious lack of long-term follow-ups and independent evaluations. The article shows remarkable similarity of the findings and problems in both fields of intervention. Problems of reproducibility prove to be highly relevant for criminology, although there is no need for using the term “crisis”. The article proposes various strategies that can enhance the reproducibility of findings, i.e., more systematic investigation of those differentiated conditions under which interventions are most effective. An integrative model of relevant characteristics is briefly presented. It refers to factors of the programs, contexts, participants, and evaluation methods. Confirmatory meta-analyses can play an important role on the path toward more differentiated and replicated knowledge
... The first challenge centers around the effect size that BWCs have on the outcome of interest. Studies that employ RCT designs purportedly have the highest level of internal validity (Weisburd, 2010), and the RCT research reviewed here generally demonstrates that BWCs had a significant effect on the outcome(s), although this effect can vary by officer discretion of when to turn the camera on/off (Ariel et al., 2016b). The need for additional effect sizes derived from RCTs (or from rigorous quasi-experimental designs) is especially important in the realm of BWCs because police agencies need to know their likely return on investment. ...
Article
Full-text available
\textbf{Purpose}$ – The purpose of this study is to review the extant of the published literature on bodyworn cameras (BWCs) in policing, specifically in the context of how BWCs effect both citizens and officers. $\textbf{Design/methodology/approach}$ – The current study is a narrative review of the impact of BWCs on police and citizens generated through a search of four repositories (Google Scholar, Criminal Justice Abstracts, EBSCO Host, PsychInfo). $\textbf{Findings}$ – The current narrative review identified 21 articles that matched the selection criteria. In general, this body of research demonstrates that: (a) the police are supportive of BWC adoption; (b) the evidence from BWC evaluations suggests that the use of BWCs can have benefits for police-public encounters. $\textbf{Practical implications}$ – The practical implications derived from this narrative review suggest to police administrators that the adoption and effective implementation of BWCs are one mechanism that can strengthen police-community relationships and decrease police misconduct through enhanced legitimacy and accountability. $\textbf{Originality/value}$ – This study is useful for researchers who wish to further examine BWC issues in policing, for police managers/administrators who are currently utilizing BWC technology, and for those who are considering adopting BWC technology.
... Theft prompts retailers to develop and test more effective crime prevention measures (Bamfield and Hollinger 1996;Hayes 2003;Lab 2010). Therefore, this study was designed to provide situational and contextual decision-making data via a rigorous randomized controlled experimental design in retail locations (Clarke 2008;Hayes 2003;Sherman 2010;Weisburd 2010;Welsh et al. 2000;Welsh and Farrington 2007). ...
Article
Full-text available
Objectives This study tested the effectiveness of two situational loss prevention treatments (a protective display fixture and a public view monitor) in reducing loss levels of premium shaving blade replacement packs. Methods The study used a randomized controlled trial design with each treatment tested in 14 stores. The study also used a 14-store control group. Paired-sample t tests were used to determine whether differences between pre- and post-test product loss were statistically significant. The odds ratio (OR) is used to measure effect size. ResultsThe results indicate that the protective display fixture had a statistically significant reduction in loss, showing a difference of 21% between the pretest and posttest. The reduction in loss for the public view monitor was not statistically significant and the level of loss for the control group remained unchanged from the pretest to posttest period. Conclusions The results indicate that experimental research designs can be used in retail settings to determine the effectiveness of various theft control devices and technologies. The results of these analyses can help retailers and loss prevention practitioners make better informed decisions regarding which loss control approaches are most effective in reducing loss.
... RCTs are considered to be one of the most reliable forms of scientific evidence (Greenhalgh, 1997;Grossman and Mackenzie, 2005). RCTs can be used to shape policies, business practices, provide evidence that a treatment does or does not have a desired outcome (Weisburd, 2010) and many more utilizations. ...
Chapter
Full-text available
The criticality of life safety, significant financial losses and paralysing fear of crime mean private and public crime prevention efforts should be evidence based. And like medicine, law enforcement and security researchers are increasingly using theory-informed randomized controlled trials (RCTs) to provide practitioners with rigorous process, impact and cost–benefit evidence.
... As we noted earlier, PSM, like other observational data analysis methods, is constrained by the degree to which variables that differentiate selection factors can be identified. Only in experiments can a strong assumption of Bignorability^of confounding be made (Boruch 2007;Weisburd 2010). At the same time, given a large number of identified and relevant co-variates, an assumption of equivalence between the groups has been shown to be reasonable (Shadish 2013;Shadish et al. 2002). ...
Article
Full-text available
Objectives We present findings from an evaluation of a large work release program in the Israeli Prison Service. The Israeli program uses an integrative therapeutic approach which combines work release with a series of other program elements. Our main question is whether this integration of multiple program elements leads to strong benefits in terms of reincarceration and re-arrests. Methods The study uses a rich administrative database to develop a propensity score matching approach for creating matched treatment (n = 547) and control (n = 547) subjects. The prisoners in the groups are found to be similar on known characteristics after matching. MHbounds methods are used to assess the sensitivity of the results to bias. ResultsAverage standardized effect sizes in our study vary between .17 and .31. In general, rehabilitation outcomes observed are much larger than those reported for US evaluations of work release programs. Conclusions Work release in Israel is not simply work release, but involves a positive social environment, a high dosage of counseling and therapy, and more general privileges for inmates including furloughs and cultural activities. We argue that our findings suggest the importance of a broader more integrative approach to work release programs to achieve enhanced success in reducing recidivism.
... Since randomized experiments control for confounding factors by design, analyses of experimental data do not require extensive statistical modeling to ensure rival causal influences are identified and controlled (Weisburd 2010). As such, we used independent samples t tests and standardized mean differences (Cohen's d) to test the direct impact of the BWCs on treatment officer outcomes relative to control officer outcomes during the 12-month intervention period. ...
Article
Full-text available
Objectives Existing research on the effects of body-worn cameras (BWCs) have found largely consistent results regarding direct significant reductions in citizen complaints and often also report reductions in use of force reports. However, few studies have examined possible spillover effects onto untreated officers. This study explicitly tests for direct and spillover effects of BWCs on the civility of police-citizen encounters and police work activities. Methods This study assesses the direct effects of BWCs on citizen complaints, police use of force, and police proactivity and discretion during a 1-year randomized controlled trial in the Boston Police Department. Through a simultaneous quasi-experimental design, this study also investigates whether BWC deployment results in spillover effects onto control officers in treated districts as compared to comparison officers in untreated districts. Results Findings indicate that the use of BWCs reduces citizen complaints and police use of force but has no appreciable impact on officer activity or discretion. Furthermore, results indicate significant spillover reductions in citizen complaints for control officers in treated districts. Conclusions The results of this study suggest that a limited BWC adoption may generate spillover deterrent impacts as officers and citizens perceive an increased threat that inappropriate and illegal behaviors will be captured on video even when BWCs are not actually present during an encounter. Partial BWC implementation seems like a cost-effective alternative to full implementation. However, police executives and policy makers need to think carefully about possible negative externalities generated by uneven BWC coverage.
... That is, PSM is limited to including only observable variables as selection factors. As such, as a quasi-experimental method, PSM is unable to claim a strong assumption of "ignorability" of confounding that can only be made in experimental studies (Boruch 2007;Weisburd 2010). Nevertheless, when there are a large number of identified and relevant covariates, assumptions of equivalence between treatment and comparison groups are reasonable (Shadish 2013;Shadish et al. 2002). ...
Article
Full-text available
Objectives Despite the prevalence of religious-oriented rehabilitation programs, few studies have assessed the effects of these programs on recidivism. Prior studies have generally focused on Christian-based programs in the USA and also suffer from a range of methodological problems, making it difficult to draw sound conclusions. The current study evaluates the effects of two Jewish-based programs in Israel. Methodology Using data provided by the Israeli Prison Services, propensity score matching was used to examine the effects of two religious-oriented rehabilitation programs on recidivism. A sub-sample of prisoners who progressed from the less intensive to the more intensive program was used as a proxy for examining the role of motivation and commitment. Findings Compared to non-participants, only those who participated in both programs had a significantly lower risk of recidivism. This sub-sample also had significantly lower recidivism than those who participated in only the less intensive of the two programs. Comparing participants of the two programs, those from the more intensive program had significantly more positive recidivism outcomes than those from the less intensive program. Conclusions The theoretical foundations of religious-oriented programs are that causing changes in levels of religiosity can lead to less recidivism. While prior studies have had difficulties in assessing the motivation to change of program participants, our findings provide evidence to support these theoretical underpinnings. That is, when religious-oriented programs succeed in engendering legitimate changes in levels of religiosity, they can have a positive effect on reducing recidivism.
... Eine Kausalität zwischen intensiver Bewährungsunterstützung und Legalbewährung kann nur dann festgestellt werden, wenn sich die Legalbewährung von nicht in RUBIKON aufgenommenen, aber geeigneten Probanden bei zufälliger Zuweisung zu Experimental-und Kontrollgruppe negativer darstellt(vgl. Diekmann 2007, S. 337 ff.;Weisburd & Mazerolle 2007;Weisburd 2010). Eine Voraussetzung für die Bildung einer solchen Kontrollgruppe ist die Vergleichbarkeit der rückfallrelevanten Faktoren mit denen der Experimentalgruppe(Köberlein 2006, S. 250 f.; Englmann 2009, S. 187).Ein experimentelles Design konnte hier jedoch nicht durchgeführt werden, da das Modellprojekt bereits vor Beginn der Evaluation ohne Randomisierung, also zufälliger Zuordnung der Projektteilnehmer zu Experimental-und Kontrollgruppe, begonnen hatte. ...
... However, in circumstances such as incarceration, it is typically not ethical to consider RCTs. 35 For example, there would be an obvious ethical problem in assigning people to arbitrary sentence lengths at random, particularly for serious crimes deserving severe punishment. How many people would give informed consent to being sentenced to five or ten years in prison, at random? ...
Preprint
Full-text available
In response to prison overcrowding concerns in recent years, many U.S. officials have undertaken efforts to reduce sentence lengths for certain crimes. However, it is unclear how these changes affect recidivism rates. Among the research on incarceration and recidivism, the majority of studies compare custodial with noncustodial sentences, while fewer examine the impact of varying incarceration lengths. This article reviews the research on the latter. Overall, the effect of incarceration length on recidivism appears too heterogeneous to draw universal conclusions, and findings are inconsistent across studies due to methodological limitations. For example, many study samples are skewed toward people with shorter sentences while others include confounds that render results invalid. Of the studies reviewed, some suggested that longer sentences provide additional deterrent benefits in the aggregate, though some studies also had null effects. None suggested a strong aggregate-level criminogenic effect. We argue that a conclusion that longer sentences have a substantial criminogenic effect, large enough to offset incapacitative effects, cannot be justified by the existing literature.
... Criminologists commonly rely on non-experimental research to conduct quantitative tests of associations (e.g., Kelling et al., 2020;Lee & Kim, 2020;Nivette et al., 2021). Although nonexperimental research in criminology has produced a variety of important findings (Kelling et al., 2020;Lee & Kim, 2020), concerns remain over the inability to adjust for unknown and unobserved confounders (Dezember et al., 2020;Weisburd, 2010). As such, it is commonplace to question the validity of an estimated association, as the influence of a confounder can bias key statistical estimates -slope coefficients, standard errors, standardized coefficients, and p-values -and increase the likelihood of committing a null-hypothesis testing error (Type 1, Type 2, or Type S; Gelman & Tuerlinckx, 2000;Urbano et al., 2019). ...
Preprint
Objectives: The inclusion of mechanisms that: (1) are a descendant of an unobserved confounder and (2) do not theoretically or empirically cause variation in one or more of the constructs in the association of interest in a multivariate regression model can potentially adjust for the bias generated from unobserved confounders. Nevertheless, the validity and utility of descendants for criminological research has yet to be evaluated. Methods: Two studies were developed to address the gap in the literature. First, a randomly specified directed equation simulation analysis was performed. Second, using the Pathways to Desistance, the technique was implemented to observe if the association between gang involvement and criminal involvement was attenuated after adjusting for exposure to violence. Results: The simulation analysis demonstrated that adjusting statistical models for the descendants of unobserved confounders can reduce bias in key regression estimates. The magnitude of the association between gang involvement and criminal involvement was approximately half of the bivariate association after introducing exposure to violence into the model. Conclusions: The findings suggest that adjusting statistical models for mechanisms that are a descendant of an unobserved confounder and do not cause variation in the association of interest can reduce the bias generated from unobserved confounders.
... Solange die Wirksamkeit einer Maßnahme nicht überzeugend belegt ist, kann daher nicht beurteilt werden, ob eine Nichtbehandlung oder eine Behandlung das größere Übel darstellt. Es gibt keinen inhaltlichen Grund, dass in der Straftäterbehandlung geringere Qualitätsmaßstäbe angelegt werden als in der Medizin oder Pharmakologie, in denen RCTs ein unverhandelbarer Standard der Qualitätskontrolle sind (Weisburd 2010 Wenn wir wirklich wissen wollen, ob die in Deutschland durchgeführten Behandlungsmaßnahmen für Sexual-und Gewaltstraftäter wirksam sind und wie wir diese Maßnahmen verbessern können, sehen wir zu einer solchen zweistufigen Evaluationsstrategie keine Alternative, und auch andere Autoren haben parallel zu uns einen sehr ähnlichen Ansatz vorgeschlagen 4 . Wir sind der Meinung, dass sich aufgrund des zu erwartenden gesellschaftlichen Nutzens ein so groß angelegtes Projekt lohnt. ...
Chapter
Die empirische Bewährung von Behandlungsprogrammen von Straftätern (insbesondere Sexualstraftätern) wird seit langem kritisch diskutiert. In diesem Kapitel geben wir einen kurzen Überblick über diese Kontroverse. Wir legen dar, dass nur experimentelle Evaluationsstudien mit zufälliger Zuordnung zu einer Behandlungsgruppe und einer Kontrollgruppe (Randomized Controlled Trials, RCTs) Evidenz für die absolute Wirksamkeit eines Behandlungsprogrammes liefern können. Auch methodisch starke Evaluationen der absoluten Wirksamkeit von Interventionen liefern allerdings keine Information darüber, welche Effekte die einzelnen Komponenten komplexer Behandlungsprogramme haben und wie man Behandlungsprogramme verbessern kann. Dazu werden Methoden wie Komponenten-, Dismantling- und korrelative Untersuchungsdesigns vorgestellt, die eine empirische Prüfung der relativen Wirksamkeit einzelner Programmkomponenten erlauben. Um die aktuelle Ungewissheit über die Wirksamkeit von Straftäterbehandlungsprogrammen zu überwinden, wird eine zweistufige Evaluationsstrategie vorgeschlagen: 1) Um die relative Wirksamkeit einzelner Programmelemente zu untersuchen, sollte eine multizentrische und länderübergreifende Evaluationsstudie durchgeführt werden. Die Vielfalt aktuell durchgeführter Behandlungsprogramme kann dabei genutzt werden, um mit multivariat-korrelativen Methoden die Wirkung einzelner Behandlungskomponenten unter spezifischen institutionellen Randbedingen für spezifische Gruppen von Straftätern zu bestimmen. 2) Um die absolute Wirksamkeit der vielfältigen in Deutschland angebotenen Behandlungsprogramme im Vergleich zum Strafvollzug ohne besondere Behandlungsmaßnahmen zu prüfen, gibt es keine überzeugende Alternative zu RCTs, deren Durchführung allerdings schon durch gesetzliche Vorgaben in einigen Vollzugsformen extrem erschwert wird. Das Ziel einer solchen kombinierten Evaluationsstrategie ist die Entwicklung eines selbstkorrigierenden Systems der Behandlung von Straftätern, in dem eine Rückkopplung von Erfolgen und Misserfolgen genutzt wird, um die Qualität von Behandlungsmaßnahmen zu verbessern und die Kriminalrückfälligkeit nachhaltig zu senken.
... aleatoriamente, a cada uma das condições, tal processo permite assegurar que o indivíduo médio que é alvo da intervenção é equivalente ao indivíduo médio que não é alvo de intervenção, em todas as variáveis externas à intervenção(Farrington, 2006;Sherman, 2003;Weisburd, 2010). Este procedimento pretende assegurar que a verificação de qualquer diferença subsequente à intervenção entre os grupos possa ser, com maior certeza, atribuída ao efeito do programa de intervenção e não à influência de fatores externos de ordem biológica, individual, social ou contextual(Fagan, 2013;Farrington, 2006; procurando, assim, estabelecer uma ordem temporal entre causas e efeitos(Currie, 2001;Fagan & Lindsey, 2014;Farrington, 2006;Shadish et al., 2002). ...
... There are more valid procedures than ad hoc quasi-experimental matching techniques, as we detailed under blocking, minimisation and stratification processes (Lum & Yang, 2005;Nagin & Weisburd, 2013;Weisburd, 2010). ...
Preprint
Full-text available
The fourth book in The SAGE Quantitative Research Kit, this resource covers the basics of designing and conducting basic experiments, outlining the various types of experimental designs available to researchers, while providing step-by-step guidance on how to conduct your own experiment. As well as an in-depth discussion of Random Controlled Trials (RCTs), this text highlights effective alternatives to this method and includes practical steps on how to successfully adopt them. Topics include: · The advantages of randomisation · How to avoid common design pitfalls that reduce the validity of experiments · How to maintain controlled settings and pilot tests · How to conduct quasi-experiments when RCTs are not an option Practical and succinctly written, this book will give you the know-how and confidence needed to succeed on your quantitative research journey.
... We therefore believe that a systematic review evaluating existing evidence from randomised controlled trials (RCTs) might be helpful in identifying treatments for reducing drug use and criminal activity in this vulnerable population. In principle, the RCT design eliminates the threat to internal validity providing there is a sufficiently large number of units assigned as the experimental and control conditions and methods to reduce bias are implemented (Weisburd, 2010). Given the importance of relating economic cost to a reduction in drug use and related offending behaviour, good-quality economic evidence will help inform strategies which represent the best use of limited resource (Higgins and Green, 2011). ...
Article
Full-text available
This updated systematic review assesses the effects of pharmacological interventions for drug-using offenders. Methods Systematic review protocols and conventions of the Cochrane Collaboration were followed to identify eligible studies. Studies were pooled in a meta-analysis to assess the impact of pharmacological interventions on drug use and criminal activity. An economic appraisal was conducted. Results The search strategies identified 22 studies containing 4372 participants. Meta-analyses revealed a small statistically significant mean difference favouring pharmacological interventions relative to psychological interventions in reducing drug use and criminal activity. When comparing the drugs to one another there were no significant differences between those included (methadone versus buprenorphine, naltrexone and cyclazocine). Conclusion Overall, the findings of this review suggest that methadone and naltrexone may have some impact on reducing drug use and reincarceration. Individual pharmacological drugs had differing (generally non-significant) effects. One study identified serious adverse events. Three studies reported cost and consequences information sufficient to conduct a full economic analysis but this was not comprehensive enough to be able to make judgements across all treatment options. Full economic analyses should be encouraged. The study findings were limited mainly to male adult offenders.
... The estimated cost of this programme is $480 per person and delivery to the nearly 16,000 members of the QPS could cost $7.6 million. Conducting an RCT is viewed as the most reliable method for determining whether the treatment is effective (Weisburd 2010). ...
Article
Research Question Did a values education program taught to Queensland police recruits change their attitudes towards police workplace diversity and equality, relative to recruits in the same cohorts who did not received the program? Data A survey designed to measure attitudes towards workplace diversity and related issues was administered three times to 260 police recruits, who were randomly assigned to receive a values education program or not over the 25-week initial police recruit course. The surveys were conducted in week two of the course, at the conclusion of the values education program, and six weeks after the program concluded. Methods Three separate cohorts were split by batch random assignment into experimental and controls, for 132 experimental recruits and 128 controls. Using a variety of validated scales and items, the attitudes of the two groups were compared at all three survey waves, and in comparative longitudinal trends. Findings While the values education program did not improve experimental group recruit attitudes towards diversity in the workplace over time, it protected that group from a clear decline in support for diversity associated with the standard recruit training experience. Because the design was an RCT, the study clearly revealed that the benefit of the program was as a successful buffer against what happened to reduce diversity support among the other recruits. Conclusions The findings show that in at least one police recruit experience, there is a clear shift away from support for diversity by race and gender in the police workplace in the course of initial training. Fortunately, the results also provide at least one possible preventative measure for that problem, in the form of a values education program similar to one used widely in many countries.
... If the sample is of sufficient size, the only difference between the treated and comparison groups should be the relevant intervention. This ensures high internal validity as differences in outcomes for the groups are likely attributable to the intervention, rather than competing explanations (Sherman et al., 1997;Weisburd, 2010). ...
Article
Operating in Australia since 1999, drug courts are now present in the majority of Australian jurisdictions. This paper takes stock of the impact evaluations of Australia’s drug courts to date, and considers to what extent these evaluations support drug courts as being more effective than ‘conventional’ sanctions in reducing recidivism. While Australian evaluations indicate drug courts reduce recidivism more than conventional sanctions, certainty in these findings is tempered by mixed results and methodological limitations.
... The methodological approach of this research is quantitative. This study is a descriptive non-experimental cross-sectional methodological design, since in the social research practice it is difficult to achieve control of all internal and external factors that can invalidate information [30]. Thus, existing situations, not intentionally provoked, are studied here [31]. ...
Article
Full-text available
The incorporation of social networks in university teaching enables new communication channels and opens new channels for training meetings, which also provides a different dynamic to day-to-day teaching, promoting communication initiatives under social responsibility. This article presents an experience of digital innovation through the use of social networks, Twitter in particular. The overall objective of this innovation is to allow communication, to provide information and reflection as a learning tool, where the main reason for the existence of this community is the exchange and intercommunication of lessons related to the different subjects and to enable professional development. It is developed through eminently active pedagogical methods, considering the theory called connectivism. The results obtained corroborate the starting hypothesis that this teaching team had from the beginning, that a methodology enriched with the use of Twitter, a tool that allows complementing the content of the subjects and interacting with them without limitations of space and time, would increase and improve the process of teaching–learning.
... This approach is common in studies which seek to measure the outcome of an intervention where the population cannot be randomised or interventions such as health education given to large population groups (Stuart et al., 2009). The emphasis in this type of study is on the effect of the intervention on a group of people without necessarily considering the population variability (Weisburd, 2010). The school children from both schools were selected using stratified random sampling by grade. ...
Article
Objectives Begun in 1939, the Cambridge-Somerville Youth Study (CSYS) is recognized as the first delinquency prevention experiment and the earliest example of a longitudinal–experimental study with criminological outcomes. This paper aims to develop a historical understanding of the origins of the study’s research design. Methods The present study is guided by the socio-historical approach and informed by past historical research in criminology. It draws upon a wide array of archival records and published works from the late nineteenth century to the present day. Results Richard Clarke Cabot designed and directed the CSYS. Major influences on the study’s research design can be traced to Cabot’s medical practice and research, his advocacy for social work practice and research, and his professional relationship with the Gluecks. The beginnings of experimentation in the social sciences during the early twentieth century may have also played a role. Joan McCord’s early involvement in the study proved instrumental to its longitudinal component. Conclusions The rigorous and innovative research design of the CSYS marks an important chapter in the history of experimental criminology, and its influence continues to this day. New experimental studies on the prevention of crime and delinquency must continue to strive to advance scientific knowledge and improve public policy.
Chapter
Quasi-experimental research methods encompass a wide variety of research designs that, compared to true experiments, do not involve randomized selection of research participants. This wide variety of designs can include single group assessments, treatment and control group comparisons, assessments of behavior change over time, and observations of direct effects of exposure to social stimuli. There are many advantages and disadvantages to quasi-experimental methods that must be taken into account by researchers. These designs can be effectively used in the study of complex behaviors, policies, programs, and other factors related to crime and deviance.
Chapter
This chapter discusses the findings of a case study of a mid-size U.S. police agency in the context of extant research and elaborates on the lessons learned from our study. Based on the empirical study of the contours of the code of silence across behaviors that violate tenets of police integrity, including police corruption, use of excessive force, interpersonal deviance, and organizational deviance, the chapter illustrates the interconnectedness between the code of silence and the police agency’s organizational perspectives. The study emphasizes the role that the police officers’ organizational attitudes play in their willingness to adhere to the code of silence, from their perceptions of how willing other police officers are to report misconduct and the severity of the disciplinary threat that their police agency is making, to their perceptions of self-legitimacy and organizational justice. The chapter concludes the book by offering a discussion of theoretical and policy implications of the findings.
Article
This article presents a framework that extends the discussion of translational criminology and the dynamic process of translating research to practice. The goal is to provide an explicit dissection of the translation process into four phases to help outline a structured way of thinking about how to incorporate research into police practice. The four-phases include: Phase I: ‘Does it Work?’ Research and Evaluation; Phase II: ‘What Works?’ Synthesis and Dissemination; Phase III: ‘How to Make it Work?’ Implementation and Evaluation; and Phase IV: ‘Make it Work!’ Institutionalization and Sustainability. The process is founded in implementation science and the ‘Knowledge to Action’ model (KTA) used in the medical and public health fields, as well as current translation activities for policing, and the authors’ experience as practitioners, researchers, and ‘translators’ over the last 25 years. It is the hope that parsing out four distinct phases for the translation of research to practice will assist researchers and police leaders to identify and fill gaps in current and future translation activities.
Chapter
This chapter considers how the outcomes of alternative care and treatment in child protection can be assessed and the potential promise of public health approaches to child maltreatment. Despite decades of research in the field of child welfare, it is not possible to make causal claims about the outcomes of alternative care and treatment in child protection. Randomised control trial research is not ethical or feasible because children may not be randomly allocated to different types of alternative care. Instead, other research designs are relied upon, but there are significant challenges arising from definitional issues, methodological limitations and potential bias. Prospective longitudinal research with population-based samples is best positioned not only to provide information on outcomes associated with alternative care, but also the contributions of risk and protective factors to outcomes. A public health approach to child maltreatment can provide a more systematic approach to achieving and measuring children’s outcomes. Public health approaches are also favoured by political and technological trends. Governments are increasingly adopting a social investment approach, demanding return on investment. The advent of big data enables researchers to track children’s outcomes through linking with data generated by other public systems such as education, health and criminal justice.
Chapter
Der Beitrag setzt sich mit systematisch an experimenteller Forschung ausgerichteter Kriminalpolitik bezogen auf den Bereich strafrechtlicher Sanktionen auseinander. Die Vision diese zu testen wie Arzneimittel wird insbesondere unter Bezugnahme auf Sherman vorgestellt. Die Chancen solcher Ansätze liegen in einer Verankerung im verfassungsrechtlichen Verhältnismäßigkeitsprinzip, die sich allerdings aus dessen gegenwärtigem rechtswissenschaftlichen Verständnis (noch) nicht ableiten lässt. So begegnen jedoch randomisierte Zuweisungen von Sanktionen rechtlichen und ethischen Bedenken. Es bestehen grundlegende Unterschiede zwischen Kriminalprävention und Medizin oder Arzneimittelforschung. Zudem lassen sich einer Evidenzorientierung zuwiderlaufende kriminalpolitische Realitäten ausmachen, so dass sich die Frage nach einer Evidenzorientierung der Evidenzorientierung stellt, also nach empirischen Erkenntnissen darüber, inwieweit experimentelle Studien tatsächlich Einfluss auf die Entwicklung strafrechtlicher Sanktionen nehmen. Neben den Chancen birgt eine an systematisch experimenteller Erkenntnisgewinnung orientierte Politik zudem auch beachtliche Risiken. Nicht zuletzt zeigen sich Nebenwirkungen evidenzorientierter Kriminalprävention in ihrer aktuellen Praxis mit Blick auf den Vollzug von Strafen und Maßregeln.
Article
Full-text available
Objectives This paper offers novel experimental evidence that violent crimes can be successfully reduced by changing the situational environment that potential victims and offenders face. We focus on a ubiquitous but understudied feature of the urban landscape—street lighting—and report the first experimental evidence on the effect of street lighting on crime. Methods Through a unique public partnership in New York City, temporary street lights were randomly allocated to 40 of the city’s public housing developments. Results We find evidence that communities that were assigned more lighting experienced sizable reductions in nighttime outdoor index crimes. We also observe a large decline in arrests indicating that deterrence is the most likely mechanism through which the intervention reduced crime. Conclusion Results suggests that street lighting, when deployed tactically, may be a means through which policymakers can control crime without widening the net of the criminal justice system.
Article
This study examines whether there is variability between the effects of three different drug rehabilitation programs operating in the Israeli Prison Service (IPS) on completers’ recidivism. By performing comparative analysis, this study attempts to address the problem of assessing the role of participants’ motivation. The study uses a rich administrative data obtained from the IPS system to develop a propensity score matching (PSM) approach where the treatment groups consist of only those who completed the programs, and the comparison groups consist of drug-addicted prisoners who have not taken part in any drug rehabilitation program. After matching, prisoners in the treatment and comparison groups are found to be similar on all known characteristics. Findings show that the only rehabilitation program that promised significant and positive outcomes for its completers was the more comprehensive one operating at Hermon Prison. Prisoners who completed the treatment were incarcerated and arrested less than their comparison group. The “golden strategy” for rehabilitating drug-using prisoners, then, will be twofold. The program should be based on the promising components of rehabilitation, that is, cognitive behavioral therapy, therapeutic community, long duration, intensity, and positive social climate. The program should also succeed in retaining its participants through completion.
Article
Full-text available
This Campbell systematic review compares effects of custodial and non‐custodial sentences on re‐offending. The authors found 14 high‐quality studies, including three randomised controlled trials and two natural experiments. Imprisonment is no more effective than community‐based sanctions in reducing re‐offending. Despite this evidence, almost all societies across the world continue to use custodial sentences as the main crime control strategy. In terms of rehabilitation, short confinement is not better or worse than “alternative” solutions. 1 Synopsis As part of a broad initiative of systematic reviews of experimental or quasi‐experimental evaluations of interventions in the field of crime prevention and the treatment of offenders, our work consisted in searching through all available databases for evidence concerning the effects of custodial and non‐custodial sanctions on re‐offending. For this purpose, we examined, in 2006, more than 3,000 abstracts, and identified more than 300 possibly eligible studies. For the update, nearly 100 additional potentially eligible studies published or completed between 2003 and 2013 have been identified. For the update, 10 matched‐pair design studies and one RCT have been abstracted. One study (Bergman 1976) that, in 2006, had been classified as an RCT turned out, after closer examination, to have been quasi‐experimental with respect to the comparison of the custodial and the non‐custodial groups. As a result, it has been “downgraded” and included among the quasi‐experimental studies in this update. The findings of the update confirm one of the major results of the first report, namely that the rate of re‐offending after a non‐custodial sanction is lower than after a custodial sanction in most comparisons. However, this is true mostly for quasi‐experimental studies using weaker designs, whereas experimental evaluations and natural experiments yield results that are less favourable to non‐custodial sanctions. It can be concluded that results in favour of non‐custodial sanctions in the majority of quasi‐experimental studies may reflect insufficient control of pre‐intervention differences between prisoners and those serving “alternative” sanctions. 2 Abstract BACKGROUND Throughout the Western World, community‐based sanctions have become a popular and widely used alternative to custodial sentences. There have been many comparisons of rates of reconviction among former prisoners and those who have served any kind of community sanction. So far, the comparative effects on re‐offending of custodial and non‐custodial sanctions are largely unknown, due to many uncontrolled variables. OBJECTIVE The objective is to assess the relative effects of custodial sanctions (imprisonment) and non‐custodial (“alternative” or “community”) sanctions on re‐offending. By “custodial” we understand any sanction where offenders are deprived of freedom of movement, i.e. placed in a closed residential setting not their home, no matter whether they are allowed to leave these premises during the day or over weekends. Thus, jails and boot camps would be considered “custodial” settings according to the definition adopted here. By “non‐custodial”, we mean any form of sanction that does not involve any deprivation of liberty, such as community work, electronic monitoring, financial or suspended custodial sanctions. Thus, the category of non‐custodial sanctions includes a great variety of punishments that have in common leaving the offender in the community rather than putting him into confinement. SEARCH STRATEGY Relevant published and unpublished studies which meet the eligibility criteria have been identified, during the first as well as for the updated review, through multiple sources, including Abstracts, bibliographies, and contacts with experts in several countries. In particular, the following sources have been searched for abstracts: Criminal Justice Abstracts, Criminology and Penology Abstracts, National Criminal Justice Reference Service (NCJRS), Library Catalogues (University of Minnesota), http://www.google.ch. The following keywords have been used to identify relevant studies: Prison, jail, imprisonment, alternative sanctions, house arrest, electronic monitoring, community service, probation, day reporting, fines, shock incarceration, boot camps; further keywords: re‐conviction, re‐offending, self‐reported offenses, recidivism, re‐arrest and re‐incarceration. ELIGIBILITY CRITERIA Randomized or natural experiments, as well as quasi‐experimental comparisons between former prison inmates and those who served community sanctions have been included without exception, provided that propensity score matching methods were used. Other quasi‐experimental studies have been included, for the updated as well as the first review, if subject were matched or if three or more potentially relevant independent variables had been controlled for. Studies written in any language and prepared between 1961 and 2013 have been considered for inclusion. For the update, ten studies have been identified and considered that used propensity scores in order to control for pre‐existing differences between custodial and non‐custodial groups. DATA COLLECTION AND ANALYSIS A coding protocol has been prepared, following the guidelines of the Campbell Collaboration. MAIN RESULTS Although a majority of the selected studies (see Table 2) show non‐custodial sanctions to be more beneficial in terms of re‐offending than custodial sanctions, no significant difference is found in the meta‐analysis based on four controlled and one natural experiments. It should be noted that offences prevented through incapacitation of incarcerated offenders have not been considered in this assessment. REVIEWERS' CONCLUSIONS The review has allowed identifying several shortcomings of studies on this subject: • 1) Controlled experiments are still rare exceptions, although obstacles to randomisation are often less formidable than claimed. • 2) Follow‐up periods rarely extend beyond two years. Even in cases of controlled trials where later follow‐up studies might be feasible, periods considered rarely extended to significant parts of subjects' biographies. • 3) Despite alternative (and presumably more valid) measures of re‐offending (such as self‐reports) have become widely available, most studies do not include measures of re‐offending beyond re‐arrest or re‐conviction. • 4) In most studies, only the occurrence (prevalence) of re‐arrest or reconviction is considered, but not the frequency (incidence) of new offences. Some studies have shown, however, that most offenders reduce offending rates after any type of intervention. Thus, the relevant question may be to what extent improvement differs by type of sanction. Therefore, future studies should look at rates of improvement (or reductions in offending) rather than merely at “recidivism” as such. • 5) Rehabilitation in other relevant areas, such as health, employment, family and social networks, is rarely considered, despite century‐old claims that short custodial sentences are damaging with respect to social integration in these other areas. • 6) No study has addressed the possibility of placebo (or Hawthorn) effects. Even in controlled trials, it is not clear to what extent outcomes that favoured “alternative” sanctions were due to the fact that subjects assigned to non‐custodial sanctions may have felt treated more fairly, rather than to specific effects of “alternative” sanctions as such. Given experimental research on neurobiological effects of feelings of fairness (Fehr and Rockenbach, 2003), such a possibility should be envisaged with more attention in future research. SOURCES OF SUPPORT The update has been supported by a grant of the Swedish National Council for Crime Prevention. The original review, published in 2006, was supported by a grant of the Swiss National Science foundation.
Chapter
The chapter illustrates both the objective and the methodology of the research presented in the book. Possible evaluation research designs are reviewed, highlighting their strengths and weaknesses and analysing which one is the more suitable for the purposes of the present research – i.e. a natural experiment with one group(s) post-test only design, using a non-equivalent qualitative pre-test. Then, strategies for data gathering and analysis are presented. In more detail, the methodology used consisted of systematic keyword searches on the three mayor social media platforms – i.e. Twitter, Facebook, and Instagram – after the implementation of the SCP measures in order to identify and analyse contents linked to the illegal online trade of medicines. The keyword list comprises 76 terms of two different types: 36 names of medicines, both generic and branded names, and 8 general terms in 5 languages – i.e. 40 keywords overall – referring to disease or condition and the medicines that are intended to treat them, or their therapeutic category. In order to assess the impact of the SCP interventions, data was analysed ‘vertically’ (i.e. by social media and by therapeutic category) and ‘horizontally’ (i.e. across social media and therapeutic category). Results (presented in Chap. 5) were compared to the situation existing before the implementation of the measures by using six qualitative indicators.
Article
Day reporting centers (DRCs) are a community-based correctional option that provide punishment and control while simultaneously assisting with rehabilitation and reintegration. The purpose of this study was to examine the effects of DRCs on the criminal recidivism of adult offenders. The current study meta-analyzed nine studies contributing 21 effect sizes (12 of which were independent). The findings show that, overall, DRCs do not differ from traditional supervision options in terms of recidivism outcomes. Additional analyses suggest that the magnitude of the treatment impact varies depending on the type of recidivism outcome measured, with studies limited to conviction showing a positive and significant reduction for DRCs. Further, front end DRCs were related to stronger treatment impact than back end DRCs. Inconsistent reporting of important variables concerning program characteristics and participant characteristics limited our ability to make recommendations for best practices.
Article
Halfway houses are a form of community supervision and correctional programming that have become a staple intervention in recent years. Despite the ingrained belief in their benefits with respect to successful reintegration, this assumption may not be justified based on the existing literature. The current study provides a systematic review and meta-analysis of nine studies examining the effects of halfway houses on recidivism. Overall, the findings suggest that halfway houses are an effective correctional strategy for successful reentry (log odds ratio [LOR] = 0.236, z = 9.27, p < .001). Further work is needed to determine best practices for programming and meeting the needs of different participants.
Article
Experimental criminology promises a public good: when experiments generate findings about criminal justice interventions, everyone benefits from that knowledge. However, experimental criminology also produces a free-rider problem: when experiments test interventions on the units where problems concentrate, only the sample assumes the risk of backfire. This mismatch between who pays for criminological knowledge and who rides on it persists even after traditional critiques of experimental social science are addressed. We draw from medicine and economics to define experimental criminology’s free-rider problem and expose a dilemma. Either we distribute the costs of producing policy-actionable knowledge to the entire beneficiary population or we justify isolating the risk of experimental harm on that class of the population where ethical concerns are most acute.
Chapter
Jüngste Publikationen in renommierten Fachzeitschriften wie Nature und Science haben gezeigt, dass es Probleme bei der Replikation von Forschungsergebnissen gibt. Dieser Beitrag informiert über einige Befunde zum Replikationsproblem und befasst sich in diesem Kontext mit der Reproduzierbarkeit von Ergebnissen in der Kriminologie und exemplarisch mit der entwicklungsbezogenen Kriminalprävention. Es zeigt sich, dass zwar im Durchschnitt positive Evaluationsergebnisse vorliegen, aber auch erhebliche Unterschiede in den Effektstärken, die keineswegs nur auf die verschiedenen Inhalte von Programmen zurückzuführen sind. Dies erlaubt keine einfachen Empfehlungen über die Wirksamkeit. Darüber hinaus umfassen die meisten Evaluationen nur kurzfristig Follow-up-Zeiträume. Auch sehr gute Studien liefern nur teilweise konsistente Ergebnisse. Programmentwicklung und -implementierung sind oftmals nicht unabhängig. Die genannten und andere Probleme betreffen nicht nur die entwicklungsorientierte Kriminalprävention, sondern sind allgemeiner. Trotz deutlicher Fortschritte in der einschlägigen Evaluationsforschung zeigt der Beitrag, dass differenziertere Ansätze notwendig sind. Zahlreiche Merkmale der Programme, Kontexte, Zielgruppen und Forschungsmethodik spielen für die Programmwirkungen eine Rolle. Wesentliche Schritte auf dem Weg zu einer differenzierten und replizierten Evidenzbasis werden dargestellt.
Article
Full-text available
Evidence-based policy has much to recommend it, but it also faces significant challenges. These challenges reside not only in the dilemmas faced by policy makers but also in the quality of the evaluation evidence. Some of these problems are most effectively addressed by rigorous syntheses of the literature known as systematic reviews. Other problems remain, including the range of quality in systematic reviews and their general failure to be updated in light of new evidence or disseminated beyond the research community. Based on the precedent established in health care by the international Cochrane Collaboration, the newly formed Campbell Collaboration will prepare, maintain, and make accessible systematic reviews of research on the effects of social and educational interventions. Through mechanisms such as rigorous quality control, electronic publication, and worldwide coverage of the literature, the Campbell Collaboration seeks to meet challenges posed by evidence-based policy.
Article
Full-text available
Evidence-based policy has much to recommend it, but it also faces significant challenges. These challenges reside not only in the dilemmas faced by policy makers but also in the quality of the evaluation evidence. Some of these problems are most effectively addressed by rigorous syntheses of the literature known as systematic reviews. Other problems remain, including the range of quality in systematic reviews and their general failure to be updated in light of new evidence or disseminated beyond the research community. Based on the precedent established in health care by the international Cochrane Collaboration, the newly formed Campbell Collaboration will prepare, maintain, and make accessible systematic reviews of research on the effects of social and educational interventions. Through mechanisms such as rigorous quality control, electronic publication, and worldwide coverage of the literature, the Campbell Collaboration seeks to meet challenges posed by evidence-based policy.
Article
Full-text available
This study investigated the effectiveness of a popular batterers's intervention program in reducing repeated violence among men who were convicted of misdemeanor domestic violence. A classical experimental design randomly assigned all 404 male defendants in Broward County Courthouse into an experimental (one-year probation and court-mandated counseling) or control (one-year probation only) conditions. The study followed these men for 12 months, collecting information from offenders' self-reports, victims' reports, and official measures of rearrests. No significant differences were found between the experimental and control groups in their attitudes, beliefs, and behaviors regarding domestic violence; both groups were equally likely to engage in both minor and severe partner abuse. In addition, no significant differences were found between the two groups in their rates of rearrest. Further analyses indicated that stake-in-conformity variables (employment and age) predicted both attendance at treatment and reoffending.
Article
Full-text available
The social benefits of randomized controlled trials (RCTs) would be enhanced by general recognition of three problems of their interpretation and a redefinition of their mission in relation to program development and evaluation. One problem is that of “forest versus trees,” or the sampling relationship between each test of a hypothesis and the conclusions drawn from all such tests taken together. A second problem is interpreting RCTs as testing theory or policy when they cannot achieve a high correlation between the treatments assigned and treatments actually applied in each case. The third problem is what works for whom, or whether identical treatments cause different effects, on average, for different kinds of people, groups, situations, or other units of analysis that were different at the point of random assignment. Confronting these three problems suggests that RCTs should not only seek verdicts about what works but also should seek better inventions of crime prevention programs for further testing.
Article
Full-text available
The propensity score is the conditional probability of assignment to a particular treatment given a vector of observed covariates. Both large and small sample theory show that adjustment for the scalar propensity score is sufficient to remove bias due to all observed covariates. Applications include: (i) matched sampling on the univariate propensity score, which is a generalization of discriminant matching, (ii) multivariate adjustment by subclassification on the propensity score where the same subclasses are used to estimate treatment effects for all outcome variables and in all subpopulations, and (iii) visual representation of multivariate covariance adjustment by a two- dimensional plot.
Article
Full-text available
The study assessed 36-month recidivism outcomes for a prison therapeutic community (TC) program with aftercare using an intent-to-treat design with random assignment. Outcomes for 478 felons at 36 months replicated findings of an earlier report on 12- and 24-month outcomes, showing the best outcomes for those who completed both in-prison and aftercare TC programs. At 36 months, 27% of the prison TC plus aftercare completers recidivated, versus 75% for other groups. In addition, a significant positive relationship was found between the amount of time spent in treatment and the time until return for the parolees who recidivated. However, the reduced recidivism rates for in-prison treatment found only at 12 and 24 months was not maintained at 36 months.
Article
Full-text available
This paper reports the results of a randomized experimental evaluation of an innovative drug enforcement strategy developed as part of the Drug Market Analysis Program. Using computer mapping techniques, we identified 56 “hot spots” of drug activity that were randomized in statistical blocks to experimental and control conditions. The experimental strategy followed a stepwise approach that sought to engage business owners and citizens in crime control efforts, to apply pressure to reduce drug and drug-related activity through police crackdowns, and to initiate a maintenance program with the assistance of the patrol division of the department. In line with tactics employed by street-level narcotics units in many other American cities, the control strategy involved unsystematic arrest-oriented narcotics enforcement based on ad hoc target selection. Comparing seven-month pre- and post-intervention periods, we find consistent and strong effects of the experimental strategy on disorder-related emergency calls for service. We also find little evidence of displacement of the crime control benefits of the study to areas near the experimental hot spots. Indeed, through two separate methods, our data suggest a “diffusion of benefits” around the experimental as compared with control locations.
Article
Full-text available
This article considers the ethical problems engendered by random assignment and privacy concerns in randomized field experiments. The particular focus is on procedural, legislative, and technical approaches to reducing or avoiding the problems. Examples are given from a variety of disciplines, although the main emphasis is on research in crime and delinquency.
Article
Full-text available
Increasing attention is being paid to the systematic review and synthesis of evaluations of large-scale, generic, crime prevention programs. The utility of these syntheses rests on the assumption that the programs are designed to work across a wide variety of contexts. But many police problem-solving efforts and situational prevention in- terventions are small-scale efforts specifically tailored to individual contexts. Do evaluation designs and methods applicable to generic programs apply to problem specific programs? Answering this ques- tions requires examining the differences between propensity-based and opportunity-blocking interventions; between internal and external validity; and between the needs of practitioner evaluators and aca- demic researchers. This paper demonstrates that in some common cir- cumstances, weak evaluation designs may have greater utility and produce more generalizable results than very strong evaluation de- signs. This conclusion has important implications for evaluations of
Article
Full-text available
Many criminologists doubt that the dosage of uniformed police patrol causes any measurable difference in crime. This article reports a one-year randomized trial in Minneapolis of increases in patrol dosage at 55 of 110 crime “hot spots,” monitored by 7,542 hours of systematic observations. The experimental group received, on average, twice as much observed patrol presence, although the ratio displayed wide seasonal fluctuation. Reductions in total crime calls ranged from 6 percent to 13 percent. Observed disorder was only half as prevalent in experimental as in control hot spots. We conclude that substantial increases in police patrol presence can indeed cause modest reductions in crime and more impressive reductions in disorder within high crime locations.
Article
Three different 12-month interventions for servicemen who had been substantiated as having physically assaulted their wives were used and the outcomes examined. The 861 couples of the study were randomly assigned to 4 groups: a men's group, a conjoint group, a rigorously monitored group, and a control group. Cognitive-behavioral interventions were implemented for the men's and conjoint groups, and outcome data were gathered from male perpetrators and female victims at roughly 6-month intervals over the approximately 18-month experimental period. Data analyses revealed nonsignificant differences between the experimental groups over a variety of outcome measures.
Article
A feast of randomized controlled trials (RCTs) in medical science and comparative famine in criminology can be explained in terms of cultural and structural factors. Of central importance is the context in which the evaluation of interventions is done and the difference in status of situational research in the two disciplines. Evaluation of medical interventions has traditionally been led by practitioner (clinical) academics. This is not the case in criminal justice, where theory has had higher status than intervention research. Medical science has advanced in, or closely associated with, university teaching hospitals, but links between criminology, and criminal justice services are far more tenuous. The late development of situational crime prevention seems extraordinary from a medical perspective, as does the absence of university police schools in the United Kingdom and elsewhere. These structural and cultural factors explain concentration of expectation, resource, and RCT productivity in medical science. The Campbell Collaboration and the Academy of Experimental Criminology are forces which are reducing this polarization of feast and famine in RCTs. But unless scientific criminology is embedded in university schools which are responsible for the education and training of law, probation, and police practitioners, convergence in terms of RCTs and implementation of findings in practice seems unlikely.
Article
This article discusses advantages of randomized experiments and key issues raised in the following articles. The main concern is the growth and decrease in the use of randomized experiments by the California Youth Authority, the U.S. National Institute of Justice, and the British Home Office, although other experiments are also discussed. It is concluded that feast and famine Periods are influenced by key individuals. It is recommended that policy makers, practitioners, funders, the mass media, and the general public need better education in research quality so that they can tell the difference between good and poor evaluation studies. They might then demand better evaluations using randomized experiments.
Article
The propensity score is the conditional probability of assignment to a particular treatment given a vector of observed covariates. Previous theoretical arguments have shown that subclassification on the propensity score will balance all observed covariates. Subclassification on an estimated propensity score is illustrated, using observational data on treatments for coronary artery disease. Five subclasses defined by the estimated propensity score are constructed that balance 74 covariates, and thereby provide estimates of treatment effects using direct adjustment. These subclasses are applied within sub-populations, and model-based adjustments are then used to provide estimates of treatment effects within these sub-populations. Two appendixes address theoretical issues related to the application: the effectiveness of subclassification on the propensity score in removing bias, and balancing properties of propensity scores with incomplete data.
Article
Randomized controlled trials (RCTs) have been considered a gold standard for health and social service research for generations of professionals. However, even as methods have developed to accommodate a large number of professional perspectives and fields of intervention, few have adequately addressed the complex nature of RCTs conducted in community settings. In this book, Drs. Solomon, Cavanaugh, and Draine draw on their extensive experience conducting randomized controlled trials to compile a practical and accessible guide to RCTs in community-based practice settings. While providing a detailed, common-sense manual, the authors address numerous design and implementation challenges that are unique to practice settings, which are less-controlled environments than the typical clinic or consultation room. Such issues include: Community and agency buy-in to support collaboration, addressing confounds to internal and external validity, and fidelity with complex interventions. These challenges are addressed through a mix of qualitative and quantitative methods that have supported RCT research in community-based settings. This pragmatic guide provides a thorough review of the basic ingredients for working through each step of the RCT process. It offers encouragement and support to enter this richly rewarding and challenging research area.
Book
What Works in Corrections, first published in 2006, examines the impact of correctional interventions, management policies, treatment and rehabilitation programs on the recidivism of offenders and delinquents. The book reviews different strategies for reducing recidivism and describes how the evidence for effectiveness is assessed. Thousands of studies were examined in order to identify those of sufficient scientific rigor to enable conclusions to be drawn about the impact of various interventions, policies and programs on recidivism. Systematic reviews and meta-analyses were performed to further examine these results. This book assesses the relative effectiveness of rehabilitation programs (e.g., education, life skills, employment, cognitive behavioral), treatment for different types of offenders (e.g. sex offenders, batterers, juveniles), management and treatment of drug-involved offenders (e.g., drug courts, therapeutic communities, outpatient drug treatment) and punishment, control and surveillance interventions (boot camps, intensive supervision, electronic monitoring). Through her extensive research, MacKenzie illustrates which of these programs are most effective and why. © Doris Layton MacKenzie 2006 and Cambridge University Press, 2009.
Article
A multistage therapeutic community treatment system has been instituted in the Delaware correctional system, and its effectiveness has captured the attention of the National Institutes of Health, the Department of Justice, members of Congress, and the White House. Treatment occurs in a three-stage system, with each phase corresponding to the client's changing correctional status-incarceration, work release, and parole. In this paper, 18 month follow-up data are analyzed for those who received treatment in: (1) a prison-based therapeutic community only, (2) a work release therapeutic community followed by aftercare, and (3) the prison-based therapeutic community followed by the work release therapeutic community and aftercare. These groups are compared with a no-treatment group. Those receiving treatment in the two-stage (work release and aftercare) and three-stage (prison, work release, and aftercare) models had significantly lower rates of drug relapse and criminal recidivism, even when adjusted for other risk factors. The results support the effectiveness of a multistage therapeutic community model for drug-involved offenders, and the importance era work release transitional therapeutic community as a component of this model.
Article
Understanding of the phenomenon of crime lies at the heart of criminology. A century and a half of theory and research has accumulated, but there does not yet exist an evaluation of how much explanatory power (summarized as the amount of variance explained) there is in criminological research. Examination of empirical tests of criminological theory in Criminology between 1968 and 2005 yields three key findings. The overall level of variance explained is often very low with 80 or 90 percent unexplained. There has been no improvement over time. Individual-based models provide relatively weak explanatory power, but models that took a more crime-specific focus indicated some strength. Criminologists will need to pay much more attention to "what is not explained" in criminological modeling if they are to make significant advances in understanding crime.
Article
Objective: To evaluate the long-term efficacy of a school-based approach to drug abuse prevention. Design: Randomized trial involving 56 public schools that received the prevention program with annual provider training workshops and ongoing consultation, the prevention program with videotaped training and no consultation, or "treatment as usual" (ie, controls). Follow-up data were collected 6 years after baseline using school, telephone, and mailed surveys. Participants: A total of 3597 predominantly white, 12th-grade students who represented 60.41% of the initial seventh-grade sample. Intervention: Consisted of 15 classes in seventh grade, 10 booster sessions in eighth grade, and five booster sessions in ninth grade, and taught general "life skills" and skills for resisting social influences to use drugs. Measures: Six tobacco, alcohol, and marijuana use self-report scales were recorded to create nine dichotomous drug use outcome variables and eight polydrug use variables. Results: Significant reductions in both drug and polydrug use were found for the two groups that received the prevention program relative to controls. The strongest effects were produced for individuals who received a reasonably complete version of the intervention--there were up to 44% fewer drug users and 66% fewer polydrug (tobacco, alcohol, and marijuana) users. Conclusions: Drug abuse prevention programs conducted during junior high school can produce meaningful and durable reductions in tobacco, alcohol, and marijuana use if they (1) teach a combination of social resistance skills and general life skills, (2) are properly implemented, and (3) include at least 2 years of booster sessions.
Article
This Article analyzes data from the Colorado Springs Spouse Abuse Experiment. In that experiment, suspects apprehended for misdemeanor spouse abuse were assigned at random to one of four treatments: (1) an emergency order of protection for the victim coupled with arrest of the suspect; (2) an emergency order of protection for the victim coupled with immediate crisis counseling for the suspect; (3) an emergency order of protection only; or (4) restoring order at the scene with no emergency order of protection. Outcome measures are taken from official police data and from follow-up interviews with victims. Using Bayesian procedures to take previous experiments into account, the balance of evidence supports a deterrent effect for arrest among "good risk" offenders, who presumably have a lot to lose by being arrested. The balance of evidence is far more equivocal for a "labeling effect" in which an arrest increases the likelihood of new violence.
Article
Data from 300 homicides involving an aggravating felony were examined to determine what factors influence the prosecutor's decision to seek the death penalty. It was found that the race of the victim was significantly related to the decision to seek the death penalty even when several legally relevant factors were taken into account. The data also revealed that black killers of whites were more likely and black killers of blacks less likely to have the death penalty requested. A breakdown of homicides into those involving a single aggravating felony and those involving multiple felonies revealed that racial effects were stronger in the former category. There was some evidence that this difference in the effects of race reflected a different threshold of tolerance for white and black murders. Black victim homicides resulted in a death request only when they crossed a threshold of aggravation that was higher than that found for white deaths.
Article
In the light of continuing debate over the applications of significance testing in psychology journals and following the publication of J. Cohen's (1994) article, the Board of Scientific Affairs (BSA) of the American Psychological Association (APA) convened a committee called the Task Force on Statistical Interference (TFSI) whose charge was "to elucidate some of the controversial issues surrounding applications of statistics including significance testing and its alternatives; alternative underlying models and data transformation; and newer methods made possible by powerful computers" (BSA, personal communication, February 28, 1996). After extensive discussion, the BSA recommended that publishing an article in American Psychologist, as a way to initiate discussion in the field about changes in current practices of data analysis and reporting may be appropriate. This report follows that request. Following each guideline are comments, explanations, or elaborations assembled by L. Wilkinson for the task force and under its review. The report is concerned with the use of statistical methods only and is not meant as an assessment of research methods in general. The title and format of the report are adapted from an article by J. C. Bailar and F. Mosteller (1988). (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Eleven jurisdictions across the country are participating in the Intensive Supervision Demon stration Projectfunded by the Bureau of Justice Assistance (BJA). The demonstration is designed to assess the effects — and costs — of sentencing convicted felons to community-based programs. One of the unique aspects of the project is that it involves random assignment of offenders to intensive probation/parole supervision or control program conditions. The demonstration will run until 1990, but it has already provided instructive insights into the issues and problems involved in managing large randomized field experiments in criminal justice. The purpose of this article is to describe the programs and sites participating in the BJA Demonstration Project. The details of the RAND evaluation are then outlined, along with the data collection methods and random assignment procedures. The remaining sections describe and discuss the author's experiences, both positive and negative, in designing and evaluating the demonstration project. The author's hope is that the lessons learned from this experiment will prove instructive and help pave the way toward more refined randomized experiments in the future.
Article
In theory, experimental designs provide the most reliable method to establish a relationship between interventions and outcomes. However, in practice, randomized experiments have remained a much less common choice for criminal justice evaluators than have nonexperimental methods. This article focuses on factors that have traditionally inhibited the use of randomized experiments as a tool for developing criminal justice policy. In this context, the main ethical, political, and practical barriers that face experimenters are described. General principles for identifying circumstances less or more amenable for developing randomized experiments are also defined. In concluding, it is argued that experiments are possible in many circumstances and can provide a powerful tool for developing criminal justice policy.
Article
This article lays out five standards for judging the importance of randomized field trials in estimating the relative effects of new programs and new variations on existing programs. These standards include contemporary evaluation policy, the historical development of trials in diverse sciences, ethics, normative practice, and the credibility of alternative approaches to estimating the effects of programs or variations. Empirical evidence and a line of reasoning bearing on each standard are made plain.
Article
In 1981, the Federal Judicial Center proposed guidelines for balancing risks of experimentation against their supposed benefits. The ethical rules were supposed to be within the scope of American legal and constitutional justification. These guidelines would not fulfill the same criterion in Germany, especially if risks to the individual are weighed against benefits toward the society. Randomization in criminal law cannot be justified by the constitutional principle of freedom of research or by the need to gain empirical knowledge about the effectiveness of sanctions. Unequal treatment in randomized designs can only be justified if randomization is permitted for reasons other than experimentation and if the individuals give their free and informed consent. The principle of informed consent must be extended from a procedural to a material content, which guarantees that participants are not forced to take part in an experiment.
Article
In this article, more general lessons are drawn from two randomized experiments in hot spots policing that the author helped design and implement in the 1990s: the Minneapolis Hot Spots Experiment and the Jersey City Drug Market Analysis Experiment. Using a case study approach, factors that facilitate and inhibit development and implementation of randomized trials are identified with particular focus on the special problems and/or advantages of place-based experiments. While the author's main comments focus on the success of place-based randomized trials in evaluating hot spots policing approaches, he draws insight as well into the reasons why the successful example of experiments in hot spots policing has not inspired similar place-based experimentation in other areas of policing or criminal justice. Eight specific lessons regarding the implementation and development of place-based randomized trials and experimental methods more generally are identified.
Article
This review highlights the importance of recognizing the possibility for doing harm when intentions are good. It describes several examples showing that well-planned and adequately executed programs provide no guarantee for safety or efficacy. The author concludes with recommendations for scientifically credible evaluations to promote progress in the field of crime prevention.
Article
An experiment investigates the effect of changes in one factor (the independent variable) on another (the dependent variable). The independent variable is under the control of the researcher. A randomized experiment is one in which people (or other units) are assigned to conditions according to a table of random numbers, with every person having the same probability of being assigned to each condition. These experiments are especially useful for testing causal hypotheses. Their unique advantage over other methods is their high internal validity, or high ability to demonstrate the effect of one factor on another. The randomization ensures that people assigned to one condition are equivalent in every possible way to those assigned to another condition, within the limits of statistical fluctuation. Despite their methodological advantages, very few randomized experiments have been carried out on crime and justice topics. Most have investigated the effects of providing special help for offenders, and in most cases this help proved no more effective in reducing reoffending than did existing alternative treatment methods. It is difficult to arrange randomized experiments because program administrators are unwilling to relinquish control of assignment to experimenters, and because of ethical problems of denial of treatment. Randomized experiments are most feasible when the effects of a treatment are unknown and when it is impossible to treat everyone. Because of their high internal validity, hypotheses should be tested, and technologies should be evaluated, using randomized experiments whenever possible.
Article
Experiments are known to be powerful tools in assessing the effectiveness of interventions. Notwithstanding this knowledge, randomized field trials have been little used in criminal justice settings. The researchers discuss their experiences implementing two experiments: one in a courthouse and the other in a police agency. Despite the different settings, common problems surfaced in both randomized field trials. The authors discuss obstacles faced in implementing and running experiments and suggest some strategies for avoiding these problems. It is the researchers' hope that such a candid discussion will help others to design and implement stronger experiments.
Article
Does the type of research design used in a crime and justice study influence its conclusions? Scholars agree in theory that randomized experimental studies have higher internal validity than do nonrandomized studies. But there is not consensus regarding the costs of using nonrandomized studies in coming to conclusions regarding criminal justice interventions. To examine these issues, the authors look at the relationship between research design and study outcomes in a broad review of research evidence on crime and justice commissioned by the National Institute of Justice. Their findings suggest that design does have a systematic effect on outcomes in criminal justice studies. The weaker a design, indicated by internal validity, the more likely a study is to report a result in favor of treatment and the less likely it is to report a harmful effect of treatment. Even when comparing randomized studies with strong quasi-experimental research designs, systematic and statistically significant differences are observed.
Article
Volunteering is viewed as a nonrandom event determined by general situational variables and specific personality attributes. Situational variables increasing probability of volunteering, other than having only a relatively less attractive alternative, include: (1) intensity of request to volunteer, (2) perception that others in similar situation would volunteer, (3) acquaintance with, perceived prestige of, and liking for E, (4) interest in subject matter, and (5) subjective probability of subsequently being favorably evaluated by E. Associated personal attributes include greater intellectual ability, interest and motivation, unconventionality, sociability, need for social approval, lower age, and less authoritarianism. Implications are discussed. (51 ref.) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
[This guide] untangles the complexities of randomized field experiments so that researchers can better evaluate the impact of new [educational or social] programs. Beginning with an exploration of how to judge whether or not a program worked (and for how much), [the author] explores the context of experiments; the standards used to judge the ethical propriety of randomized experiments; methods for studying populations; and sampling methods, including pilot tests and arrangements for random allocation. [The author] also covers baseline measures, including when to measure costs; missing data registry; and analyses of quality assurance, including analyses that have to be engineered to suit the departures from the design. [This book] is for all professionals and practitioners in evaluation/research methods, statistics, psychology, sociology, political science, social work, and public health. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
One of the major changes in juvenile justice during the past decade has been the increased reliance on restitution as a sanction for juvenile offenders. Although a great deal has been learned during the past 10 years about the operation of restitution programs, much remains unknown regarding its impact on recidivism rates. This report contains the results from four random-assignment experiments conducted simultaneously in four communities: Boise, Idaho, Washington, D. C., Clayton County, Georgia, and Oklahoma County, Oklahoma. In all four studies, youths were randomly assigned into restitution and into traditional dispositions. On the whole, the results show that restitution may have a small but important effect on recidivism. However, not all programs will be able to achieve this effect, either because of program management and strategy, community circumstances, or other factors. Youths in the restitution groups never had higher recidivism rates than those in probation or detention conditions. In two of the four studies, the juveniles in restitution clearly had fewer subsequent recontacts with the court during the two-to-three-year follow-up.