Journal of Quantitative Criminology (2019) 35:631–662
Quantifying theLikelihood ofFalse Positives: Using
Sensitivity Analysis toBound Statistical Inference
KyleJ.Thomas1· JeanMarieMcGloin2· ChristopherJ.Sullivan3
Published online: 22 June 2018
© Springer Science+Business Media, LLC, part of Springer Nature 2018
Objective Criminologists have long questioned how fragile our statistical inferences are to
unobserved bias when testing criminological theories. This study demonstrates that sensi-
tivity analyses oﬀer a statistical approach to help assess such concerns with two empirical
examples—delinquent peer inﬂuence and school commitment.
Methods Data from the Gang Resistance Education and Training are used with models
that: (1) account for theoretically-relevant controls; (2) incorporate lagged dependent vari-
ables and; (3) account for ﬁxed-eﬀects. We use generalized sensitivity analysis (Harada in
ISA: Stata module to perform Imbens’ (2003) sensitivity analysis, 2012; Imbens in Am
Econ Rev 93(2):126–132, 2003) to estimate the size of unobserved heterogeneity neces-
sary to render delinquent peer inﬂuence and school commitment statistically non-signiﬁ-
cant and substantively weak and compare these estimates to covariates in order to gauge
the likely existence of such bias.
Results Unobserved bias would need to be unreasonably large to render the peer eﬀect
statistically non-signiﬁcant for violence and substance use, though less so to reduce it to a
weak eﬀect. The observed eﬀect of school commitment on delinquency is much more frag-
ile to unobserved heterogeneity.
Conclusion Questions over the sensitivity of inferences plague criminology. This paper
demonstrates the utility of sensitivity analysis for criminological theory testing in deter-
mining the robustness of estimated eﬀects.
Keywords Sensitivity analysis· False positives· Unobserved bias· Theory testing
“A fragile inference is not worth taking seriously.” Leamer (1985)
* Kyle J. Thomas
1 University ofMissouri-St. Louis, 1 University Blvd, 331 Lucas Hall, St.Louis, MO63121, USA
2 University ofMaryland, CollegePark, MD, USA
3 University ofCincinnati, Cincinnati, OH, USA