Sample size determinations in original research protocols for randomised clinical trials submitted to UK research ethics committees: review

Institut für Medizinische Informationsverarbeitung, Biometrie und Epidemiologie, Faculty of Medicine, Ludwig-Maximilians University, Munich, Germany.
BMJ (online) (Impact Factor: 16.38). 03/2013; 346:f1135. DOI: 10.1136/bmj.f1135
Source: PubMed

ABSTRACT To assess the completeness of reporting of sample size determinations in unpublished research protocols and to develop guidance for research ethics committees and for statisticians advising these committees.
Review of original research protocols.
Unpublished research protocols for phase IIb, III, and IV randomised clinical trials of investigational medicinal products submitted to research ethics committees in the United Kingdom during 1 January to 31 December 2009.
Completeness of reporting of the sample size determination, including the justification of design assumptions, and disagreement between reported and recalculated sample size.
446 study protocols were reviewed. Of these, 190 (43%) justified the treatment effect and 213 (48%) justified the population variability or survival experience. Only 55 (12%) discussed the clinical importance of the treatment effect sought. Few protocols provided a reasoned explanation as to why the design assumptions were plausible for the planned study. Sensitivity analyses investigating how the sample size changed under different design assumptions were lacking; six (1%) protocols included a re-estimation of the sample size in the study design. Overall, 188 (42%) protocols reported all of the information to accurately recalculate the sample size; the assumed withdrawal or dropout rate was not given in 177 (40%) studies. Only 134 of the 446 (30%) sample size calculations could be accurately reproduced. Study size tended to be over-estimated rather than under-estimated. Studies with non-commercial sponsors justified the design assumptions used in the calculation more often than studies with commercial sponsors but less often reported all the components needed to reproduce the sample size calculation. Sample sizes for studies with non-commercial sponsors were less often reproduced.
Most research protocols did not contain sufficient information to allow the sample size to be reproduced or the plausibility of the design assumptions to be assessed. Greater transparency in the reporting of the determination of the sample size and more focus on study design during the ethical review process would allow deficiencies to be resolved early, before the trial begins. Guidance for research ethics committees and statisticians advising these committees is needed.

  • [Show abstract] [Hide abstract]
    ABSTRACT: Sample size calculations for treatment trials that aim to assess health-related quality-of-life (HRQOL) outcomes are often difficult to perform. Researchers must select a target minimal clinically important difference (MCID) in HRQOL for the trial, estimate the effect size of the intervention, and then consider the responsiveness of different HRQOL measures for detecting improvements. Generic preference-based HRQOL measures are usually less sensitive to gains in HRQOL than are disease-specific measures, but are nonetheless recommended to quantify an impact on HRQOL that can be translated into quality-adjusted life-years during cost-effectiveness analyses. Mapping disease-specific measures onto generic measures is a proposed method for yielding more efficient sample size requirements while retaining the ability to generate utility weights for cost-effectiveness analyses. This study sought to test this mapping strategy to calculate and compare the effect on sample size of three different methods. Three different methods were used for determining an MCID in HRQOL in patients with incontinence: 1) a global rating of improvement, 2) an incontinence-specific HRQOL instrument, and 3) a generic preference-based HRQOL instrument using mapping coefficients. The sample size required to detect a 20% difference in the MCID for the global rating of improvement was 52 per trial arm, 172 per arm for the incontinence-specific HRQOL outcome, and 500 per arm for the generic preference-based HRQOL outcome. We caution that treatment trials of conditions for which improvements are not easy to measure on generic HRQOL instruments will still require significantly greater sample size even when mapping functions are used to try to gain efficiency. Copyright © 2015 International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Published by Elsevier Inc. All rights reserved.
    Value in Health 01/2015; 18(2). DOI:10.1016/j.jval.2014.11.004 · 2.89 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: The number of samples needed to identify significant effects is a key question in biomedical studies, with consequences on experimental designs, costs and potential discoveries. In metabolic phenotyping studies, sample size determination remains a complex step. This is due particularly to the multiple hypothesis-testing framework and the top-down hypothesis-free approach, with no a priori known metabolic target. Until now, there was no standard procedure available to address this purpose. In this review, we discuss sample size estimation procedures for metabolic phenotyping studies. We release an automated implementation of the Data-driven Sample size Determination (DSD) algorithm for MATLAB and GNU Octave. Original research concerning DSD was published elsewhere. DSD allows the determination of an optimized sample size in metabolic phenotyping studies. The procedure uses analytical data only from a small pilot cohort to generate an expanded data set. The statistical recoupling of variables procedure is used to identify metabolic variables, and their intensity distributions are estimated by Kernel smoothing or log-normal density fitting. Statistically significant metabolic variations are evaluated using the Benjamini-Yekutieli correction and processed for data sets of various sizes. Optimal sample size determination is achieved in a context of biomarker discovery (at least one statistically significant variation) or metabolic exploration (a maximum of statistically significant variations). DSD toolbox is encoded in MATLAB R2008A (Mathworks, Natick, MA) for Kernel and log-normal estimates, and in GNU Octave for log-normal estimates (Kernel density estimates are not robust enough in GNU octave). It is available at, with a tutorial at © The Author 2015. Published by Oxford University Press. For Permissions, please email:
    Briefings in Bioinformatics 01/2015; DOI:10.1093/bib/bbu052 · 5.92 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Central to the design of a randomised controlled trial is the calculation of the number of participants needed. This is typically achieved by specifying a target difference and calculating the corresponding sample size, which provides reassurance that the trial will have the required statistical power (at the planned statistical significance level) to identify whether a difference of a particular magnitude exists. Beyond pure statistical or scientific concerns, it is ethically imperative that an appropriate number of participants should be recruited. Despite the critical role of the target difference for the primary outcome in the design of randomised controlled trials, its determination has received surprisingly little attention. This article provides guidance on the specification of the target difference for the primary outcome in a sample size calculation for a two parallel group randomised controlled trial with a superiority question. This work was part of the DELTA (Difference ELicitation in TriAls) project. Draft guidance was developed by the project steering and advisory groups utilising the results of the systematic review and surveys. Findings were circulated and presented to members of the combined group at a face-to-face meeting, along with a proposed outline of the guidance document structure, containing recommendations and reporting items for a trial protocol and report. The guidance and was subsequently drafted and circulated for further comment before finalisation. Guidance on specification of a target difference in the primary outcome for a two group parallel randomised controlled trial was produced. Additionally, a list of reporting items for protocols and trial reports was generated. Specification of the target difference for the primary outcome is a key component of a randomized controlled trial sample size calculation. There is a need for better justification of the target difference and reporting of its specification.
    Trials 12/2015; 16(1):526. DOI:10.1186/s13063-014-0526-8 · 2.12 Impact Factor