Page 1

Index

absolute risk reduction see risk difference

adjusted estimate of effect

analysis of variance (ANOVA)

150, 157, 169, 176, 191, 198, 205, 223,

397, 402

apples and oranges357, 378, 379–80

artifact(s)341–5

see also psychometric meta-analysis

artifact correction

compared with meta-regression

348–9

see also psychometric meta-analysis

attenuation341–8

see also psychometric meta-analysis

341–2

xxiii, 106,

bare-bones meta-analysis see psychometric

meta-analysis

Bayesian meta-analysis

316–19

BCG data set 188–97

Begg and Mazumdar test

between-study variation see heterogeneity

bias385–7

correcting for see adjusted estimate of

effect; trim and fill

outcome reporting

publication see publication bias

related to literature search

related to study design

small sample see small study effects

bias-corrected standardized mean difference

see Hedges’ g

binary data33–9, 240–1, 315, 393

combining with continuous

data46–9

effect size metrics for

worked example for

xxvii, 84, 295, 311,

397, 402

278, 369

46, 279

359–60, 380–1

33–9

92–7, 139–43

see also Mantel-Haenszel method; odds

ratio; one-step method; risk difference;

risk ratio

businessxxiv

Campbell Collaboration

change from baseline see matched (paired)

groups

change scores see matched (paired) groups

chi-squared test for heterogeneity see

heterogeneity, testing for

choosing a model83–5, 161–4, 205–7

clinical importance

clinical significance see clinical importance

clinical trials311, 381

see also randomized controlled trial

clustered groups32, 54–5, 361

cluster-randomized trials see clustered groups

CMA see Comprehensive Meta-Analysis

(CMA)

Cochrane Collaboration

Cochrane Database of Systematic

Reviews119, 262, 279

Cochrane Handbook

The Cochrane Library

Cochrane review(s)

see also Review Manager; (RevMan)

Cochran’s test see heterogeneity, testing for

coding see data extraction

Cohen’s d 27, 46, 401

see also standardized mean difference

combined effect across subgroups within

studies219–21

across outcomes or time-points

see also summary effect

combining

effect sizes see meta-analysis

407

12, 265

xxiii, 383, 398, 407

365, 405

279, 398

279, 280, 377

226–33

Introduction to Meta-Analysis. Michael Borenstein, L. V. Hedges, J. P. T. Higgins and H. R. Rothstein

© 2009 John Wiley & Sons, Ltd. ISBN: 978-0-470-05724-7

Page 2

combining (Continued)

outcomes or time-points

p-values

subgroups within a study

comparisons, multiple see multiple

comparisons; multiple comparison

groups

complex data structure(s)

243–5, 394, 397

Comprehensive Meta-Analysis

(CMA) 391–8, 403, 406

see also software

computational model(s)

211, 366

computer software see software

confidence interval

and forest plots

and heterogeneity

for I-squared124–5, 138–9, 142–3,

146–7

for meta-analysis

263, 363

for meta-regression

for tau-squared

145–6

see also prediction interval

conflicting findings

see also heterogeneity

confounding222, 307, 359

consistency of effects see heterogeneity

continuous data21–32

combining with binary data see effect

size(s), converting among

effect size metrics for

worked example for

controlled trials see clinical trials

conversion among effect sizes

correlated outcomes see outcomes, correlated

correlation

between outcomes or timepoints see

multiple outcomes or time-points

see also correlation coefficient

correlational studies

effect size metrics for

meta-analysis of396

worked example for

see also psychometric meta-analysis

correlation coefficient

as effect size metric

intraclass (ICC) 54

226–33

326–30

217–22

xxvii, 215–41,

xxvi, 83, 205–77,

51–5

5, 11, 366–8

107–9

80–5, 107–24, 132–3,

193, 199

122–4, 137–8, 141–2,

382

21–32

87–92, 135–9

45–9

41–3

97–102, 143–7

41–9, 216

41–3, 46

matched pairs see matched (paired) groups

pre-post see matched (paired) groups

covariates (moderator variables)

107, 118, 187–203, 205–11, 262, 316–17,

348–50, 394

credibility interval

in Bayesian meta-analysis

in psychometric meta-analysis

criminology xxiv

criticisms of meta-analysis

cross-over trials see matched (paired) groups

cumulative meta-analysis

371–6

xxii, 7, 70,

318

350–1

377–8

xxvii, 287–9, 355,

D (effect size metric) see raw mean

difference (D)

d (effect size metric) see Cohen’s d;

standardized mean difference

data extraction,349

data types18, 312–13

see also study design

DerSimonian-Laird method

101, 115

see also random-effects model

design see study design

diagnostic testsxxiv, xxviii

dichotomous data see binary data

dispersion106

see also heterogeneity

dummy coding187, 317

72, 91, 96,

ecology

education

effect size(s)

assumptions under fixed effect model

195, 205, 207

assumptions under random effects

model69–70, 195, 205, 207

for binary data33–9

choice of18

consistency of see heterogeneity

converting among

for correlational studies

for data based on means

difference between true and

observed 18–19, 62, 105–6

heterogeneity of see heterogeneity

precision of 51–5

summary effect

Egger’s test 397, 402

xxiv–xxv

xxiv

3–7, 17–19, 195

63,

45–9

41–3

21–32

77–9

416Index

Page 3

eligibility criteria see inclusion criteria

error

random error see sampling error

systematic error see bias

examples see worked examples

Excel functions66, 74, 129, 131, 137, 141,

145, 266, 269, 271, 273, 274, 326, 330,

335, 337

explanatory variables see covariates

(moderator variables)

fail-safe N

file drawer problem

Fisher’s method for combining

p-values

Fisher’s z (transformation of r)

49, 351, 396

fixed-effect model

151–6, 188–9, 205–6

assumptions of

compared with random-effects

model61–2, 78–86, 193–8

in meta-regression

statistical power for

weighting in65–7

worked examples for

forest plot10, 11, 78, 129, 131–3, 281–2,

287–9, 366–9, 394–402

funnel plot282–4, 286–7, 290, 394, 397,

400, 402

284–6, 290, 397

285, 377–9

328–30

41–3, 46,

6, 61, 63–7, 77–86,

6, 77, 85

188–96, 205–7, 211

260, 268–70, 275

90–1, 95–6, 99–100

g(smallsamplebiascorrectedd)seeHedges’g

gain see matched (paired) groups

garbage in, garbage out

genetic studies313–14

graphics see forest plot; funnel plot; software

377, 380

hazard ratio

Hedges’ g

worked example for

see also standardized mean difference

heterogeneity xxvi, 3, 6–7, 72, 83–6,

103–212, 358–63, 365–6, 393

measures of109–25

and model choice

7, 211

in psychometric meta-analysis

349–51

and statistical power

testing for 84–5, 111–13, 120–2, 206, 211,

262, 272–4

57, 313, 396

19, 25, 27–30, 32, 45–6, 401

87–92, 135–9

83–6, 193–7, 206–

341–4,

270–2, 274–5

worked examples

see also I-squared; meta-regression;

Q statistic; random-effects model; tau;

tau-squared

homogeneity, testing for see heterogeneity,

testing for

Hunter-Schmidt method see psychometric

meta-analysis

135–47

inclusion criteria

inconsistency see heterogeneity; I-squared

individual participant data meta

analysisxxvii, 295, 311, 316–18

interval estimation see confidence interval

intraclass correlation coefficient

inverse-variance weighting

73, 385

generality of method

IPD see individual participant data meta

analysis

I-squared105, 117–22, 125, 182, 202–3

worked examples for

61, 262, 278, 358–9, 380–1

54

xxvii, 5–7, 65–7,

311– 319

137, 141, 145

J (adjustment for small sample

bias)27–30, 90

log odds ratio

395–6, 401

see also odds ratio

log risk ratio

395–6, 401

see also risk ratio

log transformation

longitudinal studies see multiple outcomes or

time-points

lumping and splitting see criticisms of

meta-analysis

36–7, 45–9, 92–4, 142, 331–9,

35, 190–200, 281–7, 329–30,

31–6, 121–2

macros see SPSS; STATA

Mantel-Haenszel method

see also binary data

matched design see matched (paired) groups

matched (paired) groups

53–9, 361, 393–5

maximum likelihood methods

mean difference see raw mean difference (D);

standardized mean difference

means21–32, 312

measurement error see reliability

medicinexxiii–xxiv

315, 331–5, 339

19, 23–30, 32,

115, 207

Index417

Page 4

meta-analysis

compared with narrative review

criticisms of

cumulative see cumulative meta-analysis

limitations of 209–10, 363–4, 368–9,

378–9, 380–1

psychometric see psychometric

meta-analysis; publication bias,

rationale for

software for see software

steps in1–7

meta-regressionxxvii, 119, 122, 187–203,

205–11

fixed-effect model

random-effects model

statistical power for

and unexplained heterogeneity

Metawin392, 403

see also software

method of moments see DerSimonian-Laird

method

missing data278, 369

see also bias

mixed-effects model

models for meta-analysis choice of

see also fixed-effect model; random-effects

model

moderator analysis see meta-regression

moderator variable see covariates (moderator

variables)

multiple comparison groups

multiple comparisons

see also multiple comparison groups

multiple outcomes or time-points

225–38

combining across226–33

comparing 233–8

multiple regression see meta-regression

multiple treatments meta-analysis see multiple

comparison groups

385–6

377–8

188–96, 205–7, 211

193–203, 207–8

210–11

193–6

183

83–5

216, 239–41

208–9

216,

N see sample size calculation

narrative review

301–2, 377–8, 385–7

nested designs see clustered groups

network meta-analysis see multiple

comparison groups

normaldistribution

168, 176, 191, 197, 200, 264, 268, 270,

316, 318, 328–30

9–14, 251–4, 277, 297,

47–8,65,70–1,123,156,

null hypothesis

121, 168–9, 195–6, 206–7, 208, 210, 232,

249, 251, 254, 257–63, 297–8, 318–19,

325–30

number of studies81, 84, 112–13, 118,

119–20, 163, 363–4

11, 66, 74, 83–4, 112–13,

observational studies

380

meta-regression and subgroup

analysis as

odds ratio17–19, 33–9, 45–8, 49, 303–8,

313–16, 331–9, 363, 368, 393–401

worked example for

one-step method 331, 335–9

OR see odds ratio

ordinal data 313

Orwin’s fail safe N

outcomes, correlated

overall effect see summary effect

overdispersion see heterogeneity

303–7, 358, 359–61,

209–11

92–7, 139–43

285, 397

233–6

p-value(s)

combining

and confidence intervals

for heterogeneity test

for meta-analysis

for meta-regression

209, 210–11

and narrative reviews

and statistical power

paired data see matched (paired) groups

parameter18–19, 106

partitioning heterogeneity

see also heterogeneity

Pearson correlation see correlation coefficient

Peto method see one-step method

physical constants312

point estimate311

see also effect size

pooled estimate see summary estimate

pooled standard deviation

171–9

pooling vs. synthesizing

see also Simpson’s paradox

power257–76

for individual studies

for meta-analysis257–75

and statistical model

for test of homogeneity

251–5, 284–5, 297–302

251–5, 325–30

5, 11, 52

112–13, 119–22

6, 66, 74

106, 191–2, 198, 208,

10–14, 251–5

258–9, 264, 268, 270

105, 107–12

22, 26–8, 162–4,

303–8

257–9

260–2, 267–72

273–5

418 Index

Page 5

pre-post design see matched (paired) groups

precision5–6, 51–5, 368

and complex data structures

7, 243–4

factors influencing

and statistical power

267, 298–9

weighting see inverse-variance weighting

see also variance

prediction interval127–33

compared with confidence interval

worked example for

predictors see covariates (moderator variables)

presentation of results see reporting results

proportions312

see also risks

psychologyxxiv

psychometric meta-analysis

publication bias 277–92, 374–5, 379, 385,

394–402

rationale for 9–14

see also Begg and Mazumdar test; Egger’s

test, fail safe N, file drawer problem,

funnel plot, Orwin’s fail safe N; trim

and fill

226–7, 232–

51–5

258, 260, 263–5,

132–3

139, 143, 147

341–5

Q statistic

191–2, 198, 393–401

and meta-regression

and statistical power

and subgroup analysis

worked example for

139, 143

see also heterogeneity

quantifying heterogeneity see heterogeneity,

measures of

quasi-experimental designs

72–3, 105–6, 107–25, 150–86,

191–2, 198

272–4

150–86, 206

91, 96, 101, 135–7,

358–60

r (correlation coefficient) see correlation

coefficient

R (software)391–2

R-squared see variance, explained by

subgroups or covariates

random assignment

random-effects model

77–86, 161–86, 205–8, 260–3, 289,

315–18

assumptions of6, 61, 69–70, 85

compared with fixed-effect model

76–86, 193–8

360, 380

6, 61–2, 69–75,

61–2,

and meta-regression

statistical power for

weighting in

worked examples for

random-effects variance see tau-squared

random error see sampling error

randomized controlled trial

307, 358–61, 381–4, 406

cluster-randomized see clustered groups

cross-over see matched (paired) groups

range restriction342, 346, 349, 361

rank correlation test see Begg and

Mazumdar test

rate data57, 313, 396

ratio of means see response ratio

raw mean difference (D)

399, 401

RD see risk difference

regression coefficients

see also meta-regression

regression see regression coefficients;

meta-regression

relative risk see risk ratio

reliability118, 342–9

reporting350, 365–9

research design see study design

research synthesis9, 355, 365, 378, 386

response ratio18, 21, 30–2

restricted maximum likelihood (REML)

restriction of range see range restriction

Review Manager 391–4, 398–400

RevMan see Review Manager

risk difference 18–19, 37–9, 284, 291, 315,

362, 395–401

risk ratio 4–6, 11, 18, 33–9, 188–200, 281–

90, 358, 395–401

risks33–9, 312

see also risk difference; risk ratio

Rosenthal’s file drawer method

RR see risk ratio

193–203, 205–9

260–1, 270–2, 274–5

73–4, 78–9

91–2, 96–7, 101–2

209–11, 222,

21–2, 25, 32, 396,

314

115

284–5

sample size calculation

sampling distribution

207–8

sampling error

203, 283, 344–5, 348–50

of tau-squared

SAS391–2

see also software

searching for studies

262, 264–5

18–19, 65, 71, 85, 162,

61–7, 70–1, 78, 85, 106, 182,

114, 123–5

xxviii, 277–80

Index419

Page 6

selection bias see publication bias; criticisms

of meta-analysis

sensitivity analysis368–9, 393–402

shared comparison group see multiple

comparison groups

significance level(s)

see also p-value(s); statistical significance

significance, statistical see statistical

significance

sign test325–7, 330

Simpson’s paradox222, 303–8, 317

single-armed studies see single-group designs

single-group designs

small study effects79, 281, 291

SMD see standardized mean difference

software 209, 319, 391–403

see also Comprehensive Meta-Analysis

(CMA); Excel functions; Metawin;

R (software); Review Manager; SAS;

SPSS; STATA

SPSS 392

standard deviation18, 21–32, 81–2, 107,

121, 128–9, 221–2, 240–1, 265, 315,

317, 350

see also tau

standard error 51–5

see also variance

standardized mean change see matched

(paired) groups

standardized mean difference

87–92, 396–401

see also Cohen’s d; Hedges’ g

STATA393–4, 400–3

statistical power see power

statistical program see software

statistical significance

257–8, 263–4, 275, 375

and publication bias

Stouffer’s Z 328

streptokinase example

329–30, 371–3

study design 18, 19, 25, 30, 45, 53–4, 195,

342, 359–63, 384, 395

see also clustered groups; correlational

studies; matched (paired) groups;

observational studies; randomized

controlled trial

subgroup analyses 149–86, 205–12, 394

fixed-effect model

mixed-effects model

265

311–13

25–32, 45–9,

10–11, 85, 207, 253,

281

11–12, 254, 259, 326,

151–60

183

model choice for

obtaining a summary effect in

presence of

random-effects model

statistical power for

see also meta-regression; subgroups within

studies

subgroups within studies

combining across218–22

comparing222–3

summary effect 5–6, 61, 63, 67, 69, 77–82,

105, 206–7, 243–4, 378

computation of65–6, 72–4

meaning in fixed-effect model

67, 83

meaning in random-effects model

74, 83

in presence of subgroups

summary estimate,79, 100, 102, 315

sum of Z’s method see Stouffer’s Z

survival data see time-to-event data

systematic review xxi–xxiii, xxv,

xxvii–xxviii, 278–9, 365, 378, 380, 381,

385, 398, 405–6

161–2

184–6

164–82

210–11

217–23

6, 63,

6, 69,

183–6, 206–7

tau

tau-squared

33, 162–6, 171–7, 181, 185, 200–2, 363,

366, 393–401

worked example for

141, 145

time-points see multiple outcomes or

time-points

time-to-event data57, 313, 317

transformations35, 42–3

see also conversion among effect sizes

treatment effect 17–18

see also effect size

trim and fill 286–7, 288–90, 398, 402

see also publication bias

two by two table see binary data

Type I error208, 265, 267,

327, 375

see also power

Type II error265, 327

see also power

71, 105, 110–25, 129–33, 316, 318

71–5, 105, 106, 110–25, 129–

91, 96, 101, 137,

u, U see range restriction

unattenuated effect see psychometric

meta-analysis

420Index

Page 7

uncertainty interval

see also confidence interval; prediction

interval

unit of analysis23, 185, 218–23, 227

see also clustered groups, paired

(matched) groups, Simpson’s

paradox

unreliability see reliability

unstandardized mean difference

see raw mean difference

124–5

validity generalization see psychometric

meta-analysis

variability see variation

variance

of a composite or difference

223–38, 240–1, 243–5

computation of21–32, 34–8, 41–2, 46–9,

66, 72–5, 82, 87–102, 135–47

explained by subgroups or

covariates179–83, 200–3

factors influencing

partitioning 13, 105–2

between studies 72–3, 80–1, 84–5,

105–24, 163, 181–2, 192, 198, 200–2,

207, 261, 270–3, 344–5, 347–8, 363–4

within-study 65, 72–3, 81, 109, 113, 115,

164, 181, 192, 270–3

see also analysis of variance, precision,

tau-squared

variance inflation factor

218–22,

51–5, 80–2

231–2, 236–7

variation

observed

in true effects

see also heterogeneity

VIF see variance inflation factor

vote-counting

325, 364

7, 14, 105–14, 119

7, 70–1, 105–14, 122, 161

11, 84, 249, 251–5,

weights5–8, 53, 61–2, 220, 223, 233, 236,

240, 268, 289, 303, 311–15, 332–6, 339,

344–5, 351, 366–8, 385

fixed-effect65–7, 198

fixed-effect vs. random-effects

161–8, 196, 206

random-effects72–5, 114–15, 125, 199

within-study variance see variance,

within-study

within-subjects design see matched

(paired) groups

worked examples

for binary data 92–7, 139–43

for continuous data

for correlational data

for heterogeneity 135–47

78–85,

87–92, 135–9

97–102, 143–7

Z statistic41, 52, 66, 74, 91, 92, 95, 97,

100, 102, 150–6, 167–8, 175–7, 186,

190–2, 196–8, 207–10, 223, 230–7,

258, 264, 270, 298–9, 329–30, 332–3,

335–9

z transform see Fisher’s z (transformation of r)

Index421