Content uploaded by Paul R Marchant
Author content
All content in this area was uploaded by Paul R Marchant on Feb 19, 2014
Content may be subject to copyright.
BRIT. J. CRIMINOL. (2004) 44, 441–447
Advance Access publication 8 April 2004
441
British Journal of Criminology 44(3) © the Centre for Crime and Justice Studies (ISTD) 2004; all rights reserved
RESEARCH NOTE
A Demonstration That the Claim That Brighter Lighting
Reduces Crime Is Unfounded
P. R. MARCHANT*
The major systematic review on street lighting and crime, Home Office Research Study 251, suggests
that claims for the effectiveness of lighting against crime are justified. The review at first sight
appears to be an appropriate statistical synthesis of all studies on street lighting and crime across
the world. However on close examination, the statistical claims and methods are unfounded. In
three cases examined there is a clear conflict between the evidence and the reviewers’ interpretation
of this. One of the principal problems is easily seen. The time-series of the original data from the
Bristol study shows no good evidence for the crime reduction benefit of lighting. However the review
gives the result for the same data as being extremely statistically significant. It is suggested that
such a difference between the newly lit and the control areas occurring purely by chance is less than
one in a billion, but this is manifestly wrong. Two other component studies, Birmingham and
Dudley, are examined.
Introduction
A major flaw with the review Home Office Research Study 251 (HORS 251) Farrington
and Welsh (2002) is to use methods that ignore the large variation (known as ‘overdis-
persion’) in the data and implicitly assume that crimes are independent events, which
is implausible in the extreme.
The review compared the ratio of number of crimes before and after in an area that
had brighter lighting introduced with the ratio of a similar ratio in a ‘control’ area
which had no change in lighting. The ratio of ratios is called by the authors an ‘odds
ratio’ (OR). If the OR is convincingly greater than one then it might be concluded that
crime has been reduced in the newly lit area compared with the control. The results of
the review are shown in essence in Figure 1 (Figure 3.1 in HORS 251). The rectangular
point for each study (labelled at the side) shows the OR, generated from its data, and is
surrounded by a 95 per cent confidence interval bar, within which the underlying OR
might be expected to reside. (The confidence interval has been calculated using a
standard formula for statistically independent occurrences but this is incorrect as is
shown later.) We see that there is a tendency for the 13 studies to be displaced to the
right of the vertical dashed line. This line represents an OR =1, suggesting that the
study ORs are generally greater than 1, indicating that there has been a fall in crime in
the newly lit area compared with the control. This conclusion seems to be formally
strengthened by the point labelled Mean as this represents the weighted average of all
13 studies and shows an OR greater than 1 with a narrow confidence interval around it.
* Centre for Research and Graduate Studies, Leeds Metropolitan University, Calverley Street, Leeds, UK.
MARCHANT
442
However it will be shown that the true confidence interval should be very much wider
and so it is not possible to say whether lighting reduces or indeed increases crime.
The Bristol Study
As but one example of the problem of overdispersion, one can examine the contribution
from the Bristol study that used data on crime from the beginning of 1986 to mid 1990.
The reviewers’ interpretation of the result of this study is given by the point, with its
narrow confidence interval halfway down the ‘forest plot’ shown (Figure 1, HORS 251
Figure 3.1). It is, according to Farrington and Welsh, key evidence showing the benefit
of lighting as the confidence interval associated with this point is clearly on the benefit
side of the no-effect dashed vertical line. But in fact their claim is incorrect. The treatment
area in the Bristol study had brighter lighting introduced between July 1987 and
Birmingham
Stoke
Dudley
Dover
Atlanta
Fort Wor th
Bristol
Milwaukee
Kansas City
Harrisburg
New Orleans
Portland
Indianapolis
Mean
Note: Odds Ratios and Confidence Intervals on logarithmic scale
1.0
FIG. 1 Street lighting evaluations
RESEARCH NOTE
443
March 1989 and the control area had its lighting left unchanged. The reviewers in
HORS 251compared the ratio of crimes committed in the first year and the final year
in both areas. It is claimed by the reviewers that the benefit of lighting shown by this con-
tributing study is clear. (The z-statistic of 6.6, calculated using the formula relying on
statistical independence is consistent with its confidence interval in the forest plot,
being very well displaced from the null line. The value of 6.6 corresponds to a chance
occurrence of such an extreme value of about one in a billion. It would be extremely
strong evidence that the new street lighting had reduced crime).
However, the original paper from which the data was taken (Shaftoe 1994) makes no
such claim for the crime reduction benefit of lighting. Indeed it is easy to check the
situation with the data from Shaftoe, plotted below, in Figure 2.
Just inspection of the plot of the two time series of number of crimes reported in the
comparison areas and noting when the new lighting went in, shows nothing convincing
to support a claim of the benefit of lighting. (Remember that the reviewers’ claim is
that this data shows a very highly statistically significant result for the benefit of lighting
with probability of occurring by chance of less than 1 in a billion. Their claim is literally
incredible).
The difficulty for HORS 251 is that a requirement of the statistical method used by
its authors is that a typical range for a fluctuation is equivalent to approximately the
square root of the mean. In this case this is say around 20 something for the control
area and 30 something for the treatment area. As the fluctuations are seen in Figure 2
1986.0 1987.0 1988.0 1989.0 1990.0
600
700
800
900
1000
1100
1200
1300
1400
1500
Year
Number of crimes
1414 1517 1429
1374 1424
1237
1160
1464
1311
630
685
838 833
648
727
846 840 840
Brighter lighting
Control: no
lighting change
the time axis.
New lighting introduced from July 87 to March 89 as marked on
FIG. 2 Bristol: Number of crimes reported in half-year periods
MARCHANT
444
to be instead on average around 100 or so, this shows that the wrong method has been
used. Thus the results are invalid, because the reviewers’ method is incorrect as it
underestimates the true variability. This is consistent with the conclusions of Shaftoe,
the original investigator in the Bristol study who could find no evidence that the new
lighting reduced crime, in stark contradiction of the reviewers’ claim.
Other Studies
It is not just the Bristol study, as the fundamental problem is with the method used in
the review. The Birmingham Market Study, Poyner and Webb (1997), is another of
those included in the review. The effect claimed in HORS 251 is shown at the top of
the forest plot. However the data from the original Poyner and Webb (1997) paper
clearly shows that there is excess variation, via the differences in the values recorded at
the two time points, in each of the four settings, (treatment and control and before and
after the intervention of brighter lighting). See Figure 3.
There is a much larger drop than the square root of the mean in the treatment area
and also an excessively large rise in the control area, both occurring before the new lighting
went in. It is therefore impossible to claim that the change in crime was anything to do
with the new lighting installed over a short period in late 1983.
The same problem, of more variability than allowed for, occurs too in the Dudley
study, Painter and Farrington (1997), also included in the review. This study was based
around a household crime survey. It is clear in that paper that the spread of the
number of crimes experienced by the households was much greater than the methods
1985.5
1984.51983.51982.5
90
80
70
60
50
40
30
20
10
0
Yea
r
Number of crimes
12
17
54
82
21
45
56
25
time axis.
New lighting introduced late 1983 as marked on the
Brighter lighting intr oduc ed la te 19 83
lighting change
Control area: no
FIG. 3 Birmingham markets: Number of crimes reported
RESEARCH NOTE
445
used in the review (and the original paper) warrant. The original paper also contains
shortcomings such as inappropriate use of one-tailed testing, so that if analysed appro-
priately the study did not in fact detect the conventionally statistically significant effect
that it was designed to find.
Overdispersion: Its Impact and Cause
Overdispersion is also indicated by the fact that the effect of lighting is more variable
between studies than the variability (confidence intervals) that the authors give for the
individual studies would suggest. This is consistent with the very large heterogeneity
statistic, Q =56.9, for the 13 studies.
One might ask why the variability in the individual studies is so much larger than that
required by the method used by the reviewers. An answer is that crime events are not
‘statistically independent’ as the method used by the reviewers assumes, but are instead
correlated. Crime is perpetrated by people. One criminal may be responsible for many
crimes and so this one person changing behaviour can cause a large change in the
number of crimes committed and recorded.
Different statistical methods are needed to deal with such variability. Where it has
been possible to re-analyse the data the appropriate methods have not provided satis-
factory evidence for brighter lighting reducing crime. The other studies included in
the review will suffer from the same problem of extra-variability that the reviewers fail
to account for. However we cannot estimate the effect for each study as the data, such
as repeated measurements, do not exist to allow us to do so. However we can be sure
that the confidence intervals of individual studies and the combined result will be
much wider than that given in HORS 251.
Other Problems with HORS 251
There are also other problems with the review. One is of not comparing like with like,
for the individual studies, in general. This is because brighter street lighting is applied
to more crime-ridden areas and the comparison areas are less crime-ridden and this
will lead to an effect known as ‘regression to the mean’. (See Bland and Altman (1994)
for a discussion of the regression to the mean effect. Many examples of errors, resulting
from the lack of recognition of the effect, are given by Andersen (1990).) The regression
to the mean effect is exemplified by the situation of performing a ‘controlled’ trial of
treatment for the common cold on a group of people suffering badly whereas the people
in the control group are not very poorly at all. After following up the patients some
time later and finding them all virtually cold-free, a great success is claimed for the new
treatment, as if this had been responsible for bringing down the cold symptoms in the
badly ill group more than the standard treatment did in the not-so-ill group. In fact the
groups have both returned to a more average state, naturally. Clearly this example
shows the difficulty of not having the ‘controls’ in the same average state as the ‘inter-
vention’ group at the start.
Indeed the Bristol data, over nine periods, show the regression to the mean effect.
For a crime count in the series which is below average (765 in the control area, 1,370 in
re-lit area) the next value tends to be greater i.e. above the line of equality corresponding
to the same value after as before (see Figures 4 and 5).The reverse tends to be the case
MARCHANT
446
850750650
850
750
650
Previous crime count
Crime count
Regression statis tics: S = 84.2804 R-Sq = 9.2 % R-Sq(adj) = 0.0 %
Regress ion Equation: Crime = 581.078 + 0.265979 Lagged Crim e
Control area: Crime count Y v. Previous value X
Regression Line
Line of Equality
FIG. 4 Control area
1200 1300 1400 1500
1200
1300
1400
1500
Previous crime count
Crime count
Regression Equation: C rime C ount = 1207.99 + 0.113627 Previous Crime Count
Regression statistic s: S = 129.482 R-Sq = 1.3 % R-Sq(adj) = 0.0 %
Re-lit area: Crime count Y v. Previous value X
Line of Equality
Regression Line
FIG. 5 Re-lit area
RESEARCH NOTE
447
where a crime count is above average. This is just as expected in the regression to the
mean effect. It is particularly strong when, as here, the correlation of crime counts to
their immediate previous values is small. This fact is easily seen in a scatter-plot of crime
count (Y) against its previous value (X) as the ‘best-fit’ (i.e. regression) line, for the
mean crime count given the count of crime previously, is flat. This indicates that the
correlation is indeed weak and regression to the mean is a serious problem.
There are further problems in HORS 251. For example, studies of small size are
excluded for no good reason. There are a number of other shortcomings with the
review, such as not giving the source of funding of the component studies as this infor-
mation would be useful to see if bias towards the interests of funders might be a problem.
However the Bristol study by itself is sufficient to indicate that the reviewers’ results are
untenable and that the claim that brighter lighting reduces crime is unfounded. Crime
reduction is frequently presented as a potent argument for increased lighting—here it
has been shown that there is no scientific basis for this claim.
REFERENCES
ANDERSEN, B. (1990), Methodological Errors in Medical Research. Oxford: Blackwell Scientific.
BLAND, J. M. and ALTMAN, D. G. (1994), ‘Statistics Notes: Regression towards the Mean’,
British Medical Journal, 308: 1499 http://bmj.com/cgi/content/full/308/6942/1499.
FARRINGTON, D. P. and WELSH, B. C. (2002), The Effects of Improved Street Lighting on Crime:
A Systematic Review, Home Office Research Study 251. http://www.homeoffice.gov.uk/
rds/pdfs2/hors251.pdf.
PAINTER, K. and FARRINGTON, D. P. (1997) ‘The Crime Reducing Effect of Improved Street
Lighting: The Dudley Project’, in R. V. Clarke, ed., Situational Crime Prevention: Successful
Case Studies, 209–26. Guilderland, NY: Harrow and Heston.
POYNER, B. and WEBB, B. (1997) ‘Reducing Theft from Shopping Bags in City Center
Markets’, in R. V. Clarke, ed. Situational Crime Prevention: Successful Case Studies, 2nd edn,
83–9. Guilderland, NY: Harrow and Heston.
SHAFTOE, H. (1994), ‘Easton/Ashley, Bristol: Lighting Improvements’, in S. Osborn, ed.,
Housing Safe Communities: An Evaluation of Recent Initiatives, 72–7. London: Safe Neigh-
bourhoods Unit.
