Discrepancies in sample size calculations and data analyses reported in randomised trials: comparison of publications with protocols

Mayo Clinic, Rochester, USA.
BMJ (online) (Impact Factor: 16.38). 02/2008; 337:a2299. DOI: 10.1136/bmj.a2299
Source: PubMed

ABSTRACT To evaluate how often sample size calculations and methods of statistical analysis are pre-specified or changed in randomised trials.
Retrospective cohort study. Data source Protocols and journal publications of published randomised parallel group trials initially approved in 1994-5 by the scientific-ethics committees for Copenhagen and Frederiksberg, Denmark (n=70).
Proportion of protocols and publications that did not provide key information about sample size calculations and statistical methods; proportion of trials with discrepancies between information presented in the protocol and the publication.
Only 11/62 trials described existing sample size calculations fully and consistently in both the protocol and the publication. The method of handling protocol deviations was described in 37 protocols and 43 publications. The method of handling missing data was described in 16 protocols and 49 publications. 39/49 protocols and 42/43 publications reported the statistical test used to analyse primary outcome measures. Unacknowledged discrepancies between protocols and publications were found for sample size calculations (18/34 trials), methods of handling protocol deviations (19/43) and missing data (39/49), primary outcome analyses (25/42), subgroup analyses (25/25), and adjusted analyses (23/28). Interim analyses were described in 13 protocols but mentioned in only five corresponding publications.
When reported in publications, sample size calculations and statistical methods were often explicitly discrepant with the protocol or not pre-specified. Such amendments were rarely acknowledged in the trial publication. The reliability of trial reports cannot be assessed without having access to the full protocols.

Download full-text


Available from: Karsten Juhl Jørgensen, Jul 05, 2015
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Background All randomized clinical trials (RCTs) require a protocol; however, numerous studies have highlighted protocol deficiencies. Reporting guidelines may improve the content of research reports and, if developed using robust methods, may increase the utility of reports to stakeholders. The objective of this study was to systematically identify and review RCT protocol guidelines, to assess their characteristics and methods of development, and to compare recommendations. Methods We conducted a systematic review of indexed literature (MEDLINE, EMBASE and the Cochrane Methodology Register from inception to September 2010; reference lists; related article features; forward citation searching) and a targeted search of supplementary sources, including a survey of major trial funding agencies in six countries. Records were eligible if they described a content guideline in English or French relevant to RCT protocols. Guidelines were excluded if they specified content for protocols for trials of specific procedures or conditions or were intended to assess trial quality. We extracted guideline characteristics and methods. Content was mapped for a subset of guidelines that described development methods or had institutional endorsement. Results Forty guidelines published in journals, books and institutional reports were included in the review; seven were specific to RCT protocols. Only eight (20%) described development methods which included informal consensus methods, pilot testing and formal validation; no guideline described all of these methods. No guideline described formal consensus methods or a systematic retrieval of empirical evidence to inform its development. The guidelines included a median of 23 concepts per guideline (interquartile range (IQR) = 14 to 34; range = 7 to 109). Among the subset of guidelines (n = 23) for which content was mapped, approximately 380 concepts were explicitly addressed (median concepts per guideline IQR = 31 (24,80); range = 16 to 150); most concepts were addressed in a minority of guidelines. Conclusions Existing guidelines for RCT protocol content varied substantially in their recommendations. Few reports described the methods of guideline development, limiting comparisons of guideline validity. Given the importance of protocols to diverse stakeholders, we believe a systematically developed, evidence-informed guideline for clinical trial protocols is needed.
    09/2012; 1(1). DOI:10.1186/2046-4053-1-43
  • [Show abstract] [Hide abstract]
    ABSTRACT: Grid appears as an effective technology coupling geographically distributed resources for solving large-scale problems in the wide area network. Fault tolerance in grid system is a significant and complex issue to secure a stable and reliable performance. Until now, various techniques exist for detecting and correcting faults in distributed computing systems. Unfortunately, few energy focus on fault-tolerance in grid environment, especially with the emergence of OGSA. A new fault-tolerant mechanism is needed to detect and recover service faults and nodes crash. Based on our previous work on Java threads state capturing and existing mobile agent techniques, we put forward a fault-tolerant mechanism providing effective fault-handling and recovering methods.
    Industrial Informatics, 2003. INDIN 2003. Proceedings. IEEE International Conference on; 09/2003