- Access to this full-text is provided by Springer Nature.
- Learn more

Download available

Content available from BMC Medical Research Methodology

This content is subject to copyright. Terms and conditions apply.

Crippa and Orsini BMC Medical Research Methodology (2016) 16:91

DOI 10.1186/s12874-016-0189-0

RESEARCH ARTICLE Open Access

Dose-response meta-analysis of

differences in means

Alessio Crippa1* andNicolaOrsini

1

Abstract

Background: Meta-analytical methods are frequently used to combine dose-response findings expressed in terms

of relative risks. However, no methodology has been established when results are summarized in terms of differences

in means of quantitative outcomes.

Methods: We proposed a two-stage approach. A flexible dose-response model is estimated within each study (first

stage) taking into account the covariance of the data points (mean differences, standardized mean differences).

Parameters describing the study-specific curves are then combined using a multivariate random-effects model

(second stage) to address heterogeneity across studies.

Results: The method is fairly general and can accommodate a variety of parametric functions. Compared to

traditional non-linear models (e.g. Emax, logistic), spline models do not assume any pre-specified dose-response curve.

Spline models allow inclusion of studies with a small number of dose levels, and almost any shape, even non

monotonic ones, can be estimated using only two parameters. We illustrated the method using dose-response data

arising from five clinical trials on an antipsychotic drug, aripiprazole, and improvement in symptoms in shizoaffective

patients. Using the Positive and Negative Syndrome Scale (PANSS), pooled results indicated a non-linear association

with the maximum change in mean PANSS score equal to 10.40 (95 % confidence interval 7.48, 13.30) observed for

19.32 mg/day of aripiprazole. No substantial change in PANSS score was observed above this value. An estimated

dose of 10.43 mg/day was found to produce 80 % of the maximum predicted response.

Conclusion: The described approach should be adopted to combine correlated differences in means of quantitative

outcomes arising from multiple studies. Sensitivity analysis can be a useful tool to assess the robustness of the overall

dose-response curve to different modelling strategies. A user-friendly R package has been developed to facilitate

applications by practitioners.

Keywords: Meta-analysis, Dose-response, Mean differences, Random-effects

Background

The identification and characterization of dose-response

relationships is an essential part of the analysis in many

scientific disciplines such as toxicology, pharmacology,

and epidemiology. This is particularly important in the

development and testing of new compounds (e.g. a new

drug, pharmaceutical treatment) where trials at different

stages aim to evaluate the efficacy of increasing levels of

dosage (Phase II-III trials) or to derive a dose-response

curve for selection of optimal doses (Phase IV trials) [1, 2].

*Correspondence: alessio.crippa@ki.se

1Department of Public Health Sciences, Karolinska Institutet, Stockholm,

Sweden

Randomized clinical trials often investigate a continu-

ous outcome variable, such as the efficacy or safety of

a drug, reporting the change from baseline of a medical

score, or the final value of a clinical measurement. The

dose-response results are typically summarized by dose-

specific means and standard deviations [3]. Measures of

effect are expressed in terms of mean or standardized

mean differences using a dose level, usually the placebo

group, as referent [1]. Over the last few years method-

ological research focused on developing and improving

methods for performing dose-response analysis in a sin-

gle study [4, 5]. A conclusive result is hardly obtained by a

single investigation and there is often the need to synthe-

size information collected from multiple studies. In such a

© 2016 The Author(s). Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0

International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and

reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the

Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver

(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Crippa and Orsini BMC Medical Research Methodology (2016) 16:91 Page 2 of 10

case meta-analytic methods can be used to define an over-

all relation or to investigate heterogeneity across study

findings.

A method for pooling aggregated dose-response data

where the outcome is a log relative risk was originally pre-

sented by Greenland and Longnecker in 1992 [6]. Since

then, several papers have refined and covered specific

aspects of the methodology such as model specification

[7, 8], modeling strategies [9, 10], and software implemen-

tation [11, 12]. Other methodological articles extended

the approach for continuous outcome but in the case

where individual patient data are available, mainly in the

context of time-series environmental studies [13–15].

Only a few alternatives have been proposed to pool

aggregated dose-response data where the findings are

summarized by differences in means. Davis and Chen

[16] in 2004 described a methodology for summariz-

ing dose-response curves of first and second generation

antipsychotics in schizoaffective patients. The authors

reconstructed drug-specific dose-response curves and

conducted a meta-analysis to compare the effectiveness of

medium vs high dosages. A common alternative to analyze

the drug effect consists of fitting a random-intercept Emax

model, where the random component accounts for het-

erogeneity in placebo effect across trials [17]. Heterogene-

ity, however, may be related to other study characteristics

rather than differences in placebo response such as imple-

mentation, participants, intervention, and outcome defi-

nition. Thomas et al. [18] adopted hierarchical Bayesian

models to summarize and describe, independently, the

distribution of study-specific model parameters derived

from an Emax model.

The mentioned strategies assumed pre-specified mod-

els that do not allow for non-monotonic curves which

may occur in practice [19], as in case of dose-response

data of antipsychotics. In addition, fitting study-specific

sigmoidal curves such as the Emax model requires

that the single studies have assessed at least three

dose levels in order to estimate model parameters.

Discarding studies not providing enough data points

represents a loss of information and may introduce

bias.

The aim of this paper is to formalize and propose a

general and flexible methodology to pool dose-response

relations from aggregated data where the changes in the

distribution of the quantitative outcome are expressed

in terms of differences in means. We first present the

data necessary for a dose-response meta-analysis and

derive formulas for obtaining effect sizes and their vari-

ance/covariance structure. We describe flexible dose-

response models with particular emphasis on regression

splines. The method is then applied to dose-response data

from clinical trials on use of aripiprazole and symptoms

improvement in schizoaffective patients.

Methods

Dose-response data

The notation and data required for a dose-response meta-

analysis for a generic study are displayed in Table 1. We

consider Istudies indexed by i=1, ...,Ireporting the

results of a common treatment at different dose levels

xij,j=1, ...,Ji,wherex0i=0 indicates the control or

placebo group in the i-th study. The study-specific results

typically consist of dose-specific means of an outcome

variable, Yij, that measures the efficacy of the j-th dose

in the i-th study [3]. Additional information about the

number of patients allocated in each treatment, nij ,and

the sample standard deviations of Yij,sdij , is generally

reported or obtained from the study-specific results.

Effect sizes and their variance/covariance

A common way to reduce heterogeneity in placebo

response is to compute the effect size (or treatment effect)

as difference between dose-specific means and placebo

mean. In case all studies measure the outcome on a com-

mon and interpretable scale, the difference can be based

on the absolute scale

dij =¯

Yij −¯

Yi0,j=1, ...,Ji,i=1, ...,I(1)

Assuming common study-specific population standard

deviations, the variance of dij is defined as

Var dij =nij +ni0

nijni0

s2

pi,j=1, ...,Ji,i=1, ...,I

(2)

where s2

pi=Ji

j=0nij −1sd2

ij/Ji

j=0nij −1is the

square of the pooled standard deviation for the i-th study.

Since the study-specific mean differences dij use the same

referent values, ¯

Yi0, they cannot be regarded as indepen-

dent. The covariance term is defined as

Cov dij,dij=Va r ¯

Yj0=s2

i0

ni0

,j= j,i=1, ...,I

(3)

Table 1 Notation for aggregated data in the i-th study used in

dose-response meta-analysis of differences in meas

dose mean(Y)asd(Y) nbdcVar (d)d∗dVar d∗

0¯

Yi0sdi0ni00– 0 –

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

xij ¯

Yij sdij nij dij Var dijd∗

ij Var d∗

ij

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

xiJi¯

YiJisdiJiniJidiJiVar diJid∗

iJiVar d∗

iJi

aY is the continuous outcome

bNumber of patients

cMean difference

dStandardized mean difference

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Crippa and Orsini BMC Medical Research Methodology (2016) 16:91 Page 3 of 10

In case the outcome is measured on different scales the

effect sizes can be based on standardized mean differences

d∗

ij =¯

Yij −¯

Yi0

spi

,j=1, ...,Ji,i=1, ...,I(4)

with

Var d∗

ij=1

nij

+1

ni0

+d∗

ij

2

2Ji

j=0nij

,j=1, ...,Ji,

i=1, ...,I

Cov d∗

ij,d∗

ij=1

ni0

+

d∗

ijd∗

ij

2Ji

j=0nij

,j= j,i=1, ...,I

(5)

Dose-response analysis

The chosen effect sizes and the corresponding

(co)variances are used to estimate the study-specific dose-

response curves. The dose-response curves characterize

the relative efficacy of the dose under investigation using

the placebo effect as referent (i.e. the relative efficacy

for the placebo is zero by definition). The dose-response

models are expressed through the parametric model f,

which specifies how the effect size varies according to the

dose values. The functional relationship fis parametrized

in terms of θi,thep×1 vector of dose-response coef-

ficients. We consider the case of mean differences, dij,

but the same principles apply for standardized mean

differences, d∗

ij. The study-specific curves can be written

as

di=f(xi,θi)+εi,εi∼N0,

ˆ

i,i=1, ...,I

(6)

ˆ

iis the covariance matrix of the residual error term,

with Var dijalong the diagonal and Cov dij,dijoff-

diagonal.

Several alternatives are available to model the dose-

response curve (i.e. for the choice of f). Table 2 shows

the definition of 4 types of models ordered according to

the number of parameters, ranging from 1 for a linear

modelupto3foralogisticmodel.SeeBretzetal.fora

comprehensive description [2].

The most common choice in dose findings [2] is the use

of the Emax model which is expressed in terms of three

Table 2 Frequently used dose-response models

Model Equation No. of parameters

Linear E di|xi=θ1ixi1

Quadratic E di|xi=θ1ixi+θ2ix2

i2

Emax Edi|xi=θ1ixθ3i

i/θ2i+xθ3i

i3

Logistic E di|xi=θ1i/{1+exp [(θ2i−xi)]/θ3i}3

parameters: the maximum effect (θ1i),thedosetopro-

duce half of the maximum effect (θ2i) and the steepness

of the curve (θ3i) [20]. As other non-linear models, the

Emax model assumes a specific shape that does not allow

for non-monotonic curve and its estimation requires at

least three non reference dose levels. Quadratic models

are defined by only p=2 coefficients butmay poorly fit at

extreme dose values [9]. Other non-linear models such as

logistic and sigmoidal models, are commonly defined by

p≥3 coefficients so that study-specific aggregated data

may not be sufficient to estimate the parameters.

We propose the use of regression splines to flexibly

model the dose of interest. Splines represent a family of

smooth functions that can describe a wide range of curves

(i.e. U-shaped, J-shaped, S-shaped, threshold) [21]. The

curves consist of piecewise polynomials over consecu-

tive intervals defined by kknots. Their use may facilitate

curve fitting since many non-linear curves can be exam-

ined by estimating only a small number of coefficients.

For instance, a restricted cubic spline model with three

knots k=(k1,k2,k3)is defined only in terms of p=2

coefficients [22]

E[di|xi]=θ1ix1i+θ2ix2i(7)

with two transformations [23] defined as

x1=x

x2=(x−k1)3

+−k3−k1

k3−k2(x−k2)3

++k2−k1

k3−k2(x−k3)3

+

(k3−k1)2

(8)

wherethe‘+’notation,withu+=uif u≥0andu+=0

otherwise, has been used.

An alternative flexible approach to model the dose-

response association is represented by fractional poly-

nomials. In particular, a dose-response model based on

fractional polynomial of order two can be written as in

Eq. 6 with the two transformations defined as

x1=xp1and x2=xp2if p1= p2

x1=xp1and x2=xp1log(x)if p1=p2(9)

for each combination of p1and p2in the predefined set of

values {−2, −1, −0.5, 0, 0.5, 1, 2, 3};forp=0, xpbecomes

log(x). The best fitting fractional polynomial is typically

chosen based on the Akaike’s Information Criterion [24].

Once the functional relation fhas been selected, gen-

eralized least square estimation can be performed to effi-

ciently estimates the dose-response coefficients

ˆ

θjand the

corresponding (co)variance matrix

ˆ

Vj, by minimizing

di−f(xi,θi)Tˆ

−1

idi−f(xi,θi)(10)

that generally requires numerical optimization algo-

rithms. If fis a linear combination of the parameters

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Crippa and Orsini BMC Medical Research Methodology (2016) 16:91 Page 4 of 10

θi, as in the case of regression splines and fractional

polynomials, the close solution can be written as

ˆ

θi=XT

i

ˆ

−1

iXi−1

XT

i

ˆ

−1

idi

ˆ

Vi=Var ˆ

θi=XT

i

ˆ

−1

iXi−1(11)

where Xiindicates the Ji×pdesign matrix in the i-th study.

Meta-analysis

The estimated study-specific dose-response coefficients

ˆ

θiand the accompanying (co)variance matrices

ˆ

Viare

combined by means of multivariate meta-analysis

ˆ

θi∼Nθ,

ˆ

Vi+(12)

A fixed-effects model assumes no statistical hetero-

geneity among study results, i.e. differences in the dose-

response coefficients are only related to sampling error.

The assumption of homogeneity may not hold in prac-

tice, unless it is known that the studies are performed

in a similar way and are sampled from the same popula-

tion [25]. The Cochran’s Q test [26] is typically used to

test statistical heterogeneity across studies (H0:=0)

[27]. Selected studies, however, will typically differ with

respect to study design and implementation, selection of

participants, and type of analyses. A certain degree of het-

erogeneity is expected and should be taken into account

in the analysis. A random-effects model allows the dose-

response coefficients, θi, to vary across studies. Statistical

heterogeneity is captured by the between-studies variance

while θrepresents the mean of the distribution of dose-

response coefficients and an estimate,

ˆ

θ, can be obtained

using (restricted) maximum likelihood estimation [15].

As a final result, the pooled dose-response curve can be

presented in either a graphical or tabular form by predict-

ing the mean differences of the outcome for a set of xdose

values

E[

ˆ

d|x]=fX,

ˆ

θ(13)

with an approximate (1−α/2)% confidence interval (CI),

that in case fis a linear combination of θcan be expressed

as

E[

ˆ

d|x]∓zα

2diag fX,

ˆ

θTCov ˆ

θfX,

ˆ

θ(14)

where zα/2is the α/2-th quantile of a standard normal

distribution.

Dose findings

Once the pooled dose-response curve has been estimated,

it may be of interest to determine a set of target doses,

i.e. doses associated with prespecified outcome effects.

In development of new compounds it is often important

to select an optimal dose which is almost as effective as

the maximum effective dose but has less undesired side

effects, which often occur at high dosages. Suppose one

wants to determine which is the lowest dose (EDγ)topro-

duce an almost complete effect, e.g. γ%oftheobserved

maximum predicted response.

The EDγcan be determined as

EDγ=argmax

x∈(0,xmax]E[ ˆ

d|x]

E[ ˆ

d|xmax]≥γ(15)

where xmax is the dose corresponding to the maximum

predicted outcome.

An important step when presenting results from dose

findings analysis is to accompany the previous estimates

with a measure of precision, typically confidence intervals.

Pinheiro et al. [3] proposed the use a parametric bootstrap

approach based on the asymptotic normal distribution of

ˆ

θ, the pooled estimate of the dose-response coefficients.

The approach consists in re-sampling the dose-response

coefficients θfrom its approximate normal distribution

and derive the distribution of

EDγbased on the sam-

ples. Approximated confidence intervals for

EDγcan be

constructed using percentiles of the sampling distribution.

Results

To illustrate the methodology we examined the

dose-response relation between aripiprazole, a second-

generation antipsychotic, and symptoms improvement in

schizoaffective patients. We updated the search strategy

presented in a previous review by Davis and Chen [16]

by searching the Medline, International Pharmaceutical

Abstracts,CINAHL,andtheCochraneDatabaseofSys-

tematic Reviews. To reduce the exclusion of unpublished

papers, additional sources including Food and Drug

administration website, data from Cochrane reviews,

poster presentations and conference abstracts were also

searched. All random-assignment, double-blind, con-

trolled clinical trials of schizoaffective patients providing

dose-response results for at least two non-zero dosages of

aripiprazole were eligible.

Five studies [28–32] met the inclusion criteria and were

included in the analysis. All the studies reported mean

changes from baseline as main outcome variable, using

the Positive and Negative Syndrome Scale (PANSS). The

PANSS scale is an ordinal score derived from 30 items

ranging from 1 to 7. Computations of ratios such as per-

centage changes are not directly applicable and may lead

to erroneous results [33, 34]. To address this issue, the the-

oretical minimum (i.e. 30) needs to be subtracted from

the original score [35]. Information about the means, the

number of patients assigned to each treatment, and the

standard deviation was available from the published data.

Because all the studies measured the outcome variable on

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Crippa and Orsini BMC Medical Research Methodology (2016) 16:91 Page 5 of 10

the same scale, we computed PANSS mean differences as

effect sizes. Data are reported in Table 3.

We used the trial by Cutler et al. [28] to illustrate the

steps required for estimating the dose-response curve for

a single study. For example, the difference in mean PANSS

comparing the dose of 2 mg/day relative to 0 mg/day

is d11 =8.23 −5.3 =2.93 mg/day. Its variance is

Var (d11 )=(85 +92)/(85 ×92)×s2

p1= 7.59, where

s2

p1=119, 419/356 =335.4. The covariance of this

difference in means PANSS is 18.312/85 =3.94. The vari-

ance/covariance structure associated with the vector of

differences in means for this trial d1can be presented in a

matrix form

ˆ

1=⎡

⎣

7.59

3.94 7.72

3.94 3.94 7.52 ⎤

⎦

Toestimatethedose-responsecurveweneedfirstto

specify the model f. We characterized the dose-response

relation using a restricted cubic spline model with three

knots located at the 10th, 50th, and 90th percentiles

(0, 10, and 30 mg/day) of the overall dose distribu-

tion (p= 2). The restricted cubic spline dose-response

model is defined as in Eq. 7. Efficient estimates of the

dose-response coefficients and (co)variance matrix were

obtained by generalized least square estimation

ˆ

θ1=(1.215, −5.738)T

ˆ

V1=Var ˆ

θ1=0.49

−3.65 31.64 (16)

We applied the same procedure to the other stud-

ies included in the meta-analysis in order to obtain the

study-specific

ˆ

θiand

ˆ

Vi,i=1, ..., 5 (Table 4). The study-

specific predicted curves are presented in Fig. 1. Under

a random-effects model, restricted maximum likelihood

estimates were

ˆ

θ=(0.937, −1.156)T

Cov ˆ

θ=0.03

−0.05 0.10 (17)

Ap-value <0.001 for the multivariate Wald-type test

H0:θ=0provided strong evidence against the null

hypothesis of no relation between different doses of arip-

iprazole and mean change PANSS score. The Q test (Q=

3.5, p-value = 0.899) did not detect substantial statistical

heterogeneity across studies.

To communicate results of the pooled dose-response

analysis, we can estimate the pooled mean differences in

PANSS scores using 0 mg/day as referent as 0.937x1−

1.156x2, together with the corresponding 95 % confidence

interval for a generic dose xof interest as following

(0.937x1−1.156x2)∓1.960.03x2

1+0.1x2

2−0.1x1x2

Table 3 Aggregated dose-response data of five clinical trials investigating effectiveness of different dosages of aripiprazole in

schizoaffective patients. The continuous outcome is measured on the Positive and Negative Syndrome Scale and summarized by

mean values (mean(Y)) and standard deviations (sd(Y))

ID Author, Year dose mean(Y) sd(Y) n d Var(d)

1 Cutler, 2006 [28] 0 5.300 18.310 85 0.000 0.000

2 8.230 18.320 92 2.930 7.593

5 10.600 18.310 89 5.300 7.715

10 11.300 18.320 94 6.000 7.515

2 McEvoy, 2007 [29] 0 2.330 26.100 107 0.000 0.000

10 15.040 27.600 103 12.710 13.344

15 11.730 26.200 103 9.400 13.344

20 14.440 25.900 97 12.110 13.764

3 Kane, 2002 [30] 0 2.900 24.280 102 0.000 0.000

15 15.500 26.490 99 12.600 12.038

30 11.400 22.900 100 8.500 11.977

4 Potkin, 2003 [31] 0 5.000 21.140 103 0.000 0.000

20 14.500 20.160 98 9.500 8.563

30 13.900 20.880 96 8.900 8.654

5 Study 94202 [32] 0 1.400 25.730 57 0.000 0.000

2 11.000 25.000 51 9.600 25.447

10 11.500 25.200 51 10.100 25.447

30 15.800 28.510 54 14.400 24.701

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Crippa and Orsini BMC Medical Research Methodology (2016) 16:91 Page 6 of 10

Table 4 Study-specific dose-response coefficients and corresponding covariances for different dose-response models considered in

the analysis

Model id ˆ

θ1ˆ

θ2Var ˆ

θ1Cov ˆ

θ1,ˆ

θ2Var ˆ

θ2

Restricted cubic splines 1 0.55 0.55 0.065 0.065 0.065

2 0.59 0.59 0.031 0.031 0.031

30.84−1.69 0.054 −0.12 0.38

40.47−0.54 0.021 −0.043 0.15

50.78−1.25 0.23 −0.62 1.8

Fractional Polynomials 1 17.47 −7.84 2.9e+02 −2.2e+02 1.8e+02

2 29.59 −12.42 2.5e+02 −1.6e+02 1e+02

3 32.12 −13.47 1.4e+02 −70 37

4 18.48 −6.42 1.8e+02 −93 49

5 21.97 −7.00 2.3e+02 −1.1e+02 55

Emax 1 8.13 3.13 27 24 36

2 11.39 0.00 38 35 42

3 10.54 0.00 37 53 1e+02

4 9.20 0.00 60 1.4e+02 3.6e+02

5 13.28 0.94 23 2.7 2.7

Quadratic 1 1.54 −0.09 0.96 −0.086 0.0083

21.54−0.05 0.35 −0.017 0.00089

31.40−0.04 0.18 −0.0055 0.00018

40.83−0.02 0.14 −0.0047 0.00017

51.08−0.02 0.51 −0.016 0.00051

Piecewise linear 1 0.55 0.065

2 0.59 0.031

30.84−1.69 0.054 −0.12 0.38

40.47−0.54 0.021 −0.043 0.15

50.78−1.25 0.23 −0.62 1.8

where x1and x2are defined as in Eq. 8. For instance,

the model-based predicted mean changes in PANSS score

compared to placebo were 4.52 (95 % CI: 2.96, 6.08) for

5 mg/day, 8.08 (95 % CI: 5.43, 10.73) for 10 mg/day, 9.95

(95 % CI: 6.97, 12.94) for 15 mg/day, 10.38 (95 % CI: 7.49,

13.27) for 20 mg/day, 9.84 (95 % CI: 6.86, 12.83) for 25

mg/day, and 8.83 (95 % CI: 5.11, 12.54) for 30 mg/day. The

pooled predicted dose-response curve together with the

confidence intervals and the model mean differences is

provided in Fig. 2.

The results indicated a statistically significant positive

association between increasing doses of aripiprazole and

the mean change in PANSS score with the maximum value

of 10.39 (95 % CI: 7.48, 13.30) observed at xmax = 19.32

mg/day. The model suggested a slight decrease in the pre-

dicted mean PANSS score for dosages greater than 20

mg/day. The estimated dose to produce 50 % and 80 % of

the predicted maximum effect were

ED50 =5.82 mg/day

(95 % CI: 5.10, 8.58) and

ED80 =10.43 mg/day (95 % CI:

9.02, 16.73).

Sensitivity analysis

A sensitivity analysis is often required to evaluate the

robustness of the pooled dose-response curve. In the

spline model, for example, the location of the knots may

affect the shape of the dose-response curve. Therefore

we considered alternative knots locations including differ-

ent combinations of the 10th, 25th, 50th, 75th and 90th

percentiles of the overall dose distribution (0, 0.5, 10,

18.75, and 30 mg/day). A graphical comparison is pre-

sented in the left panel of Fig. 3. The alternative curves

roughly described the same dose-response shape with

no substantial variation, all indicating an increase in the

mean change PANSS score up to 20 mg/day of arip-

iprazole. We can assess whether there is an increasing

trend above 20 mg/day by simply re-defining x2equal

to (x−20)+in Eq. 8; this approach is known as piece-

wise linear model. The rate of change in the PANSS

mean differences was negative and not statistically sig-

nificant (θ1+θ2=-0.284, p= 0.18) after 20 mg/day of

aripiprazole.

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Crippa and Orsini BMC Medical Research Methodology (2016) 16:91 Page 7 of 10

0 5 10 15 20 25 30

0

5

10

15

20

25

Aripiprazole (mg/day)

Mean Difference

Cutler 2006

0 5 10 15 20 25 30

0

5

10

15

20

25

Aripiprazole (mg/day)

Mean Difference

McEvoy 2007

0 5 10 15 20 25 30

0

5

10

15

20

25

Aripiprazole (mg/day)

Mean Difference

Kane 2002

0 5 10 15 20 25 30

0

5

10

15

20

25

Aripiprazole (mg/day)

Mean Difference

Potkin 2003

0 5 10 15 20 25 30

0

5

10

15

20

25

Aripiprazole (mg/day)

Mean Difference

Study 94202

Fig. 1 Study-specific mean differences in Positive and Negative Syndrome Scale score for increasing dosages of aripiprazole. The first author and

year of publication of the subjects included in the original analyses are reported. Black squares indicate the mean differences and whiskers their

95 % confidence interval. Ariprazole dosage was modeled with restricted cubic splines. Solid lines represent the estimated dose-response curves,

dashed lines the corresponding 95 % confidence intervals. The placebo group (dose = 0) served as the referent group

Fig. 2 Pooled dose-response association between aripiprazole and mean change in Positive and Negative Syndrome Scale score (solid line).

Aripiprazole dosage was modeled with restricted cubic splines in a random-effects model. Dash lines represent the 95 % confidence intervals for the

spline model. The placebo group (dose = 0) served as the referent group. Circles indicate observed mean differences in individual studies; size of

bubbles is proportional to precision (inverse of variance) of the mean differences. Right axis represents percentage of the maximum predicted effect

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Crippa and Orsini BMC Medical Research Methodology (2016) 16:91 Page 8 of 10

Fig. 3 Graphical sensitivity analysis for the pooled dose-response curves between aripiprazole and mean change in Positive and Negative Syndrome

Scale score. The placebo group (dose = 0) served as the referent group. Right axis represents percentage of the maximum predicted effect. Left

panel: different location of the three knots in a restricted cubic spline model. Right panel: different models, restricted cubic splines (solid line),

fractional polynomials (dashed line), quadratic polynomial (dotted line), and Emax model (dot-dashed line). Circles indicate observed mean differences

in individual studies; size of bubbles is proportional to precision (inverse of variance) of the mean differences

To evaluate the sensitivity of the dose-response curve

to the choice of the parametric model fadopted, instead,

we considered three alternatives: fractional polynomials;

quadratic; and Emax. Since two studies only had two non-

referent doses, the study-specific (sigmoidal) Emax models

as described in Table 2 cannot be estimated. A common

solution is to fix the steepness of the curve θ3to be 1, also

referred to as hyperbolic Emax [20].

The “best” fractional polynomials (p1=0.5, p2=1)

provided overall a similar dose-response curve when com-

pared to the spline model, with slightly higher value for the

maximum predicted response (right panel of Fig. 3). The

hyperbolic Emax had substantially higher predicted mean

differences for low values of the dose. The non-linear

model assumes a specific hyperbolic dose-response curve

that did not seem to fit the data and may be dependent

from the choice of fixing θ3to be 1. The dose-response

curve described by the quadratic model fall in between the

spline and the hyperbolic Emax curves.

Discussion

In this paper we proposed a statistical method to com-

bine differences in means of quantitative outcomes.

The method consists of dose-response models estimated

within each study (first stage) and an overall curve

obtained by pooling study-specific dose-response coef-

ficients (second stage). The covariance among study-

specific mean differences is taken into account in the first

stage analysis using generalized least square estimators,

while statistical heterogeneity across studies is allowed by

multivariate random-effects model in the second stage.

One major strength of the proposed method is that

it is fairly general and can accommodate different mod-

eling strategies, including non-linear ones described by

Pinheiro et al. [3]. Non-linear models, however, are

defined by at least three or four parameters, and hence

require an equal number of dose levels for each single

study included in the analysis. Given that some stud-

ies may have investigated a lower number of dose lev-

els, exclusion of these studies may result in substantial

loss of information. In addition, many non-linear models

assume a specific behaviour (e.g. monotonicity) requir-

ing a strong a priori information about the dose-response

curve. The choice of the parametric model is critical,

since it highly influences the final results [3]. Indeed, the

selection of the dose-response model should be informed

by subject-matter knowledge as well as understanding of

the research questions at hand. We presented the use of

regression splines as a flexible tool for modeling any quan-

titative exposure. The major advantage is that a variety

of curves, even non monotonic ones, can be estimated

using only two parameters. It is considered to be closed

to non-parametric regression, since no major assump-

tions about the shape of the curve are needed [9]. A

possible alternative is the use of fractional polynomials.

In comparing the two strategies, we did not find impor-

tant differences between the two strategies and concluded

that both are useful tools to characterize a (non-linear)

dose-response curve. Nonetheless a sensitivity analysis

is generally required to evaluate the robustness of the

combined results.

A possible limitation of the proposed methodology is

that it requires information about dose-specific means

and standard deviations. Studies providing other sum-

mary measures, such as dose-specific medians, would

not be included the analysis. The dose-response analysis

presented in Eq. 6 is based on the asymptotic normal dis-

tribution of the conditional mean effect size. Extension of

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Crippa and Orsini BMC Medical Research Methodology (2016) 16:91 Page 9 of 10

the introduced methodology to percentiles is not straight-

forward and may represent an interesting topic of future

research.

An additional limitation of aggregated dose-response

data is that supplementary information for approximat-

ing the covariance terms may not be available. Articles

may report directly mean or standardized mean differ-

ences and standard errors for non-referent dose groups.

Whenever the standard deviation for the outcome vari-

able in the control group (s2

i0) cannot be obtained, it

may be approximated using the pooled standard deviation

based on the non-referent dose levels (s2

pij ). Alternatively

a specific value may be imputed and a sensitivity anal-

ysis can be performed to evaluate how the results of

the meta-analysis vary for different values of s2

0j.Fur-

ther limitations relate to the general application of meta-

analysis based on aggregated data. These include restric-

tions in subgroup analyses, the impossibility of assessing

the appropriateness of individual analyses, and to har-

monize variable definitions and analyses for reducing the

extent of heterogeneity, as well as specific biases such as

aggregation (or ecological) bias in meta-regression mod-

els. Meta-analysis of individual patient data, however, are

often difficult to undertake especially for the availabil-

ity of individual data, so that usage of aggregated data

may represent the only alternative [36]. Specific to aggre-

gated dose-response data, different dose references and

exposure range may complicate the analysis. The pre-

sented methodology assume that all the selected studies

share a common dose-response model. Important depar-

ture from this assumption may limit and/or impact the

pooling of individual dose-response coefficients. An alter-

native methods has been proposed based on a series of

univariate meta-analyses of effect sizes for a pre-specified

grid of dose-levels [37]. Further work is needed to analyze

this possibility and potential advantages. Depending on

the extent of heterogeneity of the dose-response curves,

however, it may not be opportune to pool study-specific

results, and meta-regression or stratified analyses should

be performed [38].

In our application, we considered the effectiveness

of increasing dosages of aripiprazole in shizoaffective

patients. We described the steps needed to obtain the

overall dose-response curve and to present it in a graph-

ical form. We observed a non-linear association with the

maximum efficacy corresponding to aripiprazole 19.32

mg/day. An estimated dose of 10.46 mg/day, however,

may be sufficient to obtain 80 % of the maximum effect,

which may be relevant for avoiding possible undesired

side effects. Sensitivity analysis showed similar results as

compared to fractional polynomials. The Emax model pre-

sented higher drug efficacy for low dosages. Compared to

the previous models, the Emax model did not seem to fit

properly the data at low dosages.

Conclusions

We described an approach to combine differences in

means of a quantitative outcome contrasting different

dose levels relative to a placebo in randomized trials.

The general framework of the proposed methodology can

include a variety of flexible models. Sensitivity analysis

can be a useful tool to assess the stability of the over-

all dose-response curve to different modelling strategies.

Although the method was presented for the analysis of

randomized trials, it may be extended to observational

studies where mean differences are further adjusted for

potential confounders. Future work is needed to evalu-

ate the properties of the statistical model and validity of

the underlying assumptions. A user friendly procedure is

implemented in the dosresmeta R package [39] with

worked examples available on GitHub.

Abbreviations

PANSS, positive and negative syndrome scale

Acknowledgements

We are grateful to Dr. Stefan Leucht and John M. Davis for providing the data

and raising the methodological question under study.

Funding

This work was supported by Karolinska Institutet’s funding for doctoral

students (KID-funding) (AC) and by a Young Scholar Award from the

Karolinska Institutet’s Strategic Program in Epidemiology (SfoEpi) (NO).

Availability of data and materials

The data on the effectiveness of aripiprazole are publicly available and also

contained in dosresmeta R package on github [39] (https://github.com/

alecri/dosresmeta/blob/master/data/ari.rda).

Authors’ contributions

AC developed the methods and prepared a draft. NO provided critical reviews,

corrections and revisions. Both authors read and approved the final version of

the manuscript.

Authors’ information

AC is a PhD student in Epidemiology and Biostatistics. Dr. NO is Associate

Professor of Medical Statistics.

Competing interests

The authors declare that they have no competing interest.

Consent for publication

Not applicable.

Ethics approval and consent to participate

Not applicable.

Received: 27 October 2015 Accepted: 13 July 2016

References

1. Bretz F, Pinheiro JC, Branson M. Combining multiple comparisons and

modeling techniques in dose-response studies. Biometrics. 2005;61(3):

738–48. doi:10.1111/j.1541-0420.2005.00344.x. Accessed 24 Feb 2015.

2. Bretz F, Hsu J, Pinheiro J, Liu Y. Dose finding - a challenge in statistics.

Biometrical Journal. Biometrische Zeitschrift. 2008;50(4):480–504.

doi:10.1002/bimj.200810438.

3. Pinheiro JC, Bretz F, Branson M. Analysis of dose–response studies.

modeling approaches. In: Dose Finding in Drug Development. New York

City: Springer; 2006. p. 146–71.

4. Ting N. Dose finding in drug development. New York City: Springer; 2006.

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Crippa and Orsini BMC Medical Research Methodology (2016) 16:91 Page 10 of 10

5. Chevret S. Statistical methods for dose-finding experiments. New York

City: Wiley; 2006.

6. Greenland S, Longnecker MP. Methods for trend estimation from

summarized dose-response data, with applications to meta-analysis. Am J

Epidemiol. 1992;135(11):1301–1309.

7. Berlin JA, Longnecker MP, Greenland S. Meta-analysis of epidemiologic

dose-response data. Epidemiology. 1993;4(3):218–28.

8. Dumouchel W. Meta-analysis for dose–response models. Stat Med.

1995;14(5-7):679–85.

9. Bagnardi V, Zambon A, Quatto P, Corrao G. Flexible meta-regression

functions for modeling aggregate dose-response data, with an

application to alcohol and mortality. Am J Epidemiol. 2004;159(11):

1077–1086. doi:10.1093/aje/kwh142. Accessed 25 Nov 2013.

10. Liu Q, Cook NR, Bergström A, Hsieh CC. A two-stage hierarchical

regression model for meta-analysis of epidemiologic nonlinear

dose–response data. Comput Stat Data Anal. 2009;53(12):4157–167.

11. Orsini N, Bellocco R, Greenland S, et al. Generalized least squares for

trend estimation of summarized dose-response data. Stata J. 2006;6(1):40.

12. Orsini N, Li R, Wolk A, Khudyakov P, Spiegelman D. Meta-analysis for

linear and nonlinear dose-response relations: examples, an evaluation of

approximations, and software. Am J Epidemiol. 2012;175(1):66–73.

13. Dominici F, Samet JM, Zeger SL. Combining evidence on air pollution

and daily mortality from the 20 largest us cities: a hierarchical modelling

strategy. J Royal Stat Soc Ser A (Stat Soc). 2000;163(3):263–302.

14. Simmonds MC, Higginsa JP, Stewartb LA, Tierneyb JF, Clarke MJ,

Thompson SG. Meta-analysis of individual patient data from randomized

trials: a review of methods used in practice. Clin Trials. 2005;2(3):209–17.

15. Gasparrini A, Armstrong B, Kenward MG. Multivariate meta-analysis for

non-linear and other multi-parameter associations. Stat Med. 2012;31(29):

3821–839. doi:10.1002/sim.5471. Accessed 11 Dec 2013.

16. Davis JM, Chen N. Dose response and dose equivalence of

antipsychotics. J Clin Psychopharmacol. 2004;24(2):192–208.

17. Holford NH, Sheiner LB. Understanding the dose-effect relationship:

clinical application of pharmacokinetic-pharmacodynamic models. Clin

Pharmacokinet. 1981;6(6):429–53.

18. Thomas N, Sweeney K, Somayaji V. Meta-analysis of clinical

dose-response in a large drug development portfolio. Stat Biopharm Res.

2014;6(4):302–17. doi:10.1080/19466315.2014.924876. Accessed 11 Feb

2015.

19. Geddes J, Freemantle N, Harrison P, Bebbington P. Atypical

antipsychotics in the treatment of schizophrenia: systematic overview

and meta-regression analysis. BMJ : British Medical Journal.

2000;321(7273):1371–1376. Accessed 24 Mar 2015.

20. Macdougall J. Analysis of dose–response studies. emax model. In: Dose

Finding in Drug Development. New York City: Springer; 2006. p. 127–45.

21. Durrleman S, Simon R. Flexible regression models with cubic splines. Stat

Med. 1989;8(5):551–61. doi:10.1002/sim.4780080504. Accessed30 Mar 2015.

22. Desquilbet L, Mariotti F. Dose-response analyses using restricted cubic

spline functions in public health research. Stat Med. 2010;29(9):

1037–1057. doi:10.1002/sim.3841. Accessed 25 Nov 2013.

23. Harrell F.E. Jr, Lee KL, Pollock BG. Regression models in clinical studies:

determining relationships between predictors and response. J Natl

Cancer Inst. 1988;80(15):1198–1202.

24. Royston P. A strategy for modelling the effect of a continuous covariate in

medicine and epidemiology. Stat Med. 2000;19(14):1831–1847.

25. Jackson D, White IR, Riley RD. Quantifying the impact of between-study

heterogeneity in multivariate meta-analyses. Stat Med. 2012;31(29):

3805–820.

26. Cochran WG. The combination of estimates from different experiments.

Biometrics. 1954;10(1):101–29. doi:10.2307/3001666. Accessed 09 Jan

2015.

27. Ritz J, Demidenko E, Spiegelman D. Multivariate meta-analysis for data

consortia, individual patient meta-analysis, and pooling projects. J Stat

Plann Infer. 2008;138(7):1919–1933.

28. Cutler AJ, Marcus RN, Hardy SA, O’Donnell A, Carson WH, McQuade RD.

The efficacy and safety of lower soses of aripiprazole for the treatment of

patients with acute exacerbation of schizophrenia. CNS Spectrums.

2006;11(09):691–702. doi:10.1017/S1092852900014784. Accessed 06 Feb

2015.

29. McEvoy JP, Daniel DG, Carson WH, McQuade RD, Marcus RN. A

randomized, double-blind, placebo-controlled, study of the efficacy and

safety of aripiprazole 10, 15 or 20 mg/day for the treatment of patients

with acute exacerbations of schizophrenia. J Psychiatr Res. 2007;41(11):

895–905. doi:10.1016/j.jpsychires.2007.05.002.

30. Kane JM, Carson WH, Saha AR, McQuade RD, Ingenito GG, Zimbroff DL,

Ali MW. Efficacy and safety of aripiprazole and haloperidol versus placebo

in patients with schizophrenia and schizoaffective disorder. J Clin

Psychiatry. 2002;63(9):763–71.

31. Potkin SG, Saha AR, Kujawa MJ, Carson WH, Ali M, Stock E, Stringfellow

J, Ingenito G, Marder SR. Aripiprazole, an antipsychotic with a novel

mechanism of action, and risperidone vs placebo in patients with

schizophrenia and schizoaffective disorder. Arch Gen Psychiatr.

2003;60(7):681–90. doi:10.1001/archpsyc.60.7.681.

32. Turner EH, Knoepflmacher D, Shapley L. Publication bias in antipsychotic

trials: an analysis of efficacy comparing the published literature to the US

Food and Drug Administration Database. PLoS Med. 2012;9(3).

doi:10.1371/journal.pmed.1001189. Accessed 30 Mar 2015.

33. Obermeier M, Schennach-Wolff R, Meyer S, Möller HJ, Riedel M, Krause

D, Seemüller F. Is the panss used correctly? a systematic review. BMC

psychiatry. 2011;11(1):1.

34. Leucht S, Kissling W, Davis JM. The PANSS should be rescaled.

Schizophrenia bulletin. 2010;36:461–462.

35. Leucht S, Davis JM, Engel RR, Kane JM, Wagenpfeil S. Defining response

in antipsychotic drug trials: recommendations for the use of scale-derived

cutoffs. Neuropsychopharmacology. 2007;32(9):1903–1910.

36. Lyman GH, Kuderer NM. The strengths and limitations of meta-analyses

based on aggregate data. BMC Med Res Methodol. 2005;5(1):14.

37. Sauerbrei W, Royston P. A new strategy for meta-analysis of continuous

covariates in observational studies. Stat Med. 2011;30(28):3341–360.

38. Greenland S. Invited commentary: a critical look at some popular

meta-analytic methods. Am J Epidemiol. 1994;140(3):290–6.

39. Crippa A, Orsini N. Multivariate dose-response meta-analysis: the

dosresmeta r package. J Stat Softw. 2016. In Press.

• We accept pre-submission inquiries

• Our selector tool helps you to ﬁnd the most relevant journal

• We provide round the clock customer support

• Convenient online submission

• Thorough peer review

• Inclusion in PubMed and all major indexing services

• Maximum visibility for your research

Submit your manuscript at

www.biomedcentral.com/submit

Submit your next manuscript to BioMed Central

and we will help you at every step:

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

1.

2.

3.

4.

5.

6.

Terms and Conditions

Springer Nature journal content, brought to you courtesy of Springer Nature Customer Service Center GmbH (“Springer Nature”).

Springer Nature supports a reasonable amount of sharing of research papers by authors, subscribers and authorised users (“Users”), for small-

scale personal, non-commercial use provided that all copyright, trade and service marks and other proprietary notices are maintained. By

accessing, sharing, receiving or otherwise using the Springer Nature journal content you agree to these terms of use (“Terms”). For these

purposes, Springer Nature considers academic use (by researchers and students) to be non-commercial.

These Terms are supplementary and will apply in addition to any applicable website terms and conditions, a relevant site licence or a personal

subscription. These Terms will prevail over any conflict or ambiguity with regards to the relevant terms, a site licence or a personal subscription

(to the extent of the conflict or ambiguity only). For Creative Commons-licensed articles, the terms of the Creative Commons license used will

apply.

We collect and use personal data to provide access to the Springer Nature journal content. We may also use these personal data internally within

ResearchGate and Springer Nature and as agreed share it, in an anonymised way, for purposes of tracking, analysis and reporting. We will not

otherwise disclose your personal data outside the ResearchGate or the Springer Nature group of companies unless we have your permission as

detailed in the Privacy Policy.

While Users may use the Springer Nature journal content for small scale, personal non-commercial use, it is important to note that Users may

not:

use such content for the purpose of providing other users with access on a regular or large scale basis or as a means to circumvent access

control;

use such content where to do so would be considered a criminal or statutory offence in any jurisdiction, or gives rise to civil liability, or is

otherwise unlawful;

falsely or misleadingly imply or suggest endorsement, approval , sponsorship, or association unless explicitly agreed to by Springer Nature in

writing;

use bots or other automated methods to access the content or redirect messages

override any security feature or exclusionary protocol; or

share the content in order to create substitute for Springer Nature products or services or a systematic database of Springer Nature journal

content.

In line with the restriction against commercial use, Springer Nature does not permit the creation of a product or service that creates revenue,

royalties, rent or income from our content or its inclusion as part of a paid for service or for other commercial gain. Springer Nature journal

content cannot be used for inter-library loans and librarians may not upload Springer Nature journal content on a large scale into their, or any

other, institutional repository.

These terms of use are reviewed regularly and may be amended at any time. Springer Nature is not obligated to publish any information or

content on this website and may remove it or features or functionality at our sole discretion, at any time with or without notice. Springer Nature

may revoke this licence to you at any time and remove access to any copies of the Springer Nature journal content which have been saved.

To the fullest extent permitted by law, Springer Nature makes no warranties, representations or guarantees to Users, either express or implied

with respect to the Springer nature journal content and all parties disclaim and waive any implied warranties or warranties imposed by law,

including merchantability or fitness for any particular purpose.

Please note that these rights do not automatically extend to content, data or other material published by Springer Nature that may be licensed

from third parties.

If you would like to use or distribute our Springer Nature journal content to a wider audience or on a regular basis or in any other manner not

expressly permitted by these Terms, please contact Springer Nature at

onlineservice@springernature.com

Content uploaded by Alessio Crippa

Author content

All content in this area was uploaded by Alessio Crippa on Aug 04, 2016

Content may be subject to copyright.