Article

Empirical vs natural weighting in random effects meta-analysis.

Department of Epidemiology and Health Policy Research, College of Medicine, University of Florida, PO Box 100177, Gainesville, FL 32610-0177, USA.
Statistics in Medicine (impact factor: 1.88). 06/2009; 29(12):1259-65. DOI:10.1002/sim.3607
Source: PubMed

ABSTRACT This article brings into serious question the validity of empirically based weighting in random effects meta-analysis. These methods treat sample sizes as non-random, whereas they need to be part of the random effects analysis. It will be demonstrated that empirical weighting risks substantial bias. Two alternate methods are proposed. The first estimates the arithmetic mean of the population of study effect sizes per the classical model for random effects meta-analysis. We show that anything other than an unweighted mean of study effect sizes will risk serious bias for this targeted parameter. The second method estimates a patient level effect size, something quite different from the first. To prevent inconsistent estimation for this population parameter, the study effect sizes must be weighted in proportion to their total sample sizes for the trial. The two approaches will be presented for a meta-analysis of a nasal decongestant, while at the same time will produce counter-intuitive results for the DerSimonian-Laird approach, the most popular empirically based weighted method. It is concluded that all past publications based on empirically weighted random effects meta-analysis should be revisited to see if the qualitative conclusions hold up under the methods proposed herein. It is also recommended that empirically based weighted random effects meta-analysis not be used in the future, unless strong cautions about the assumptions underlying these analyses are stated, and at a minimum, some form of secondary analysis based on the principles set forth in this article be provided to supplement the primary analysis.

0 0
 · 
0 Bookmarks
 · 
57 Views
  • Article: A Simple and Robust Way of Concluding Meta-Analysis Results Using Reported P values, Standardized Effect Sizes, or Other Statistics.
    Po-Huang Chyou
    [show abstract] [hide abstract]
    ABSTRACT: Meta-analysis is a powerful tool to estimate measures of associations/effects based on published/unpublished reports. However, problems exist in many meta-analyses, particularly related to study heterogeneity. This article proposes a way of concluding meta-analysis results using P-values, taking heterogeneity into account. There is little research focused on evaluating conclusiveness of summary results of reported meta-analyses. Generally, a P-value is directly linked to the test statistic z=b/s(b) following a standard normal distribution with mean zero and unit variance, where b is an estimator of β and s(b) is the estimated standard error of b for any study included in a meta-analysis. This forms the basis of the proposed method for deriving overall test statistics and corresponding P-values used for comparing results of meta-analyses. Two published meta-analyses were chosen and specific software was applied. Results are consistent with the two published meta-analysis reports in terms of P-values for significance and direction of summary measure of treatment effect. This proposed method can be utilized to safeguard against improper conclusions of published meta-analyses due to heterogeneity. Exploring more sophisticated statistical methods for situations when the key assumption applied to this proposed method is violated could be pursued and could expand the scope of applications beyond this method.
    Clinical Medicine &amp Research 05/2012;
  • Article: Routine probiotics for premature infants: let's be careful!
    Josef Neu
    The Journal of pediatrics 01/2011; 158(4):672-4. · 4.02 Impact Factor
  • Article: Study factors influencing ventricular enlargement in schizophrenia: a 20 year follow-up meta-analysis.
    Angelo Sayo, Robin G Jennings, John Darrell Van Horn
    [show abstract] [hide abstract]
    ABSTRACT: A meta-analysis was performed on studies employing the ventricular-brain ratio to compare schizophrenic subjects to that of normal controls. This was a follow-up to a similar meta-analysis published in 1992 in which study-, in addition to clinical-, factors were found to contribute significantly to the reported difference between patients with schizophrenia and controls. Seventy-two (N=72) total studies were identified from the peer reviewed literature, 39 from the original meta-analysis, and 33 additional studies published since which met strict criteria for inclusion and analysis - thus representing ~30 years of schizophrenia ventricular enlargement research. Sample characteristics from schizophrenics and controls were coded for use as predictor variables against within sample VBR values as well as for between sample VBR differences. Additionally, a number of factors concerning how the studies were conducted and reported were also coded. Obtained data was subjected to unweighted univariate as well as multiple regression analyses. In particular, results indicated significant differences between schizophrenics and controls in ventricular size but also the influence of the diagnostic criteria used to define schizophrenia on the magnitude of the reported VBR. This suggests that differing factors of the diagnostic criteria may be sensitive to ventricular enlargement and might be worthy of further examination. Interestingly, we observed an inverse relationship between VBR difference and the number of co-authors on the study. This latter finding suggests that larger research groups report smaller VBR differences and may be more conservative or exacting in their research methodology. Analyses weighted by sample size provided identical conclusions. The effects of study factors such as these are helpful for understanding the variation in the size of the reported differences in VBR between patients and controls as well as for understanding the evolution of research on complex clinical syndromes employing neuroimaging morphometrics.
    NeuroImage 07/2011; 59(1):154-67. · 5.89 Impact Factor

Show more

Data provided are for informational purposes only. Although carefully collected, accuracy cannot be guaranteed. The impact factor represents a rough estimation of the journal's impact factor and does not reflect the actual current impact factor. Publisher conditions are provided by RoMEO. Differing provisions from the publisher's actual policy or licence agreement may be applicable.

Keywords

classical model
 
DerSimonian-Laird approach
 
empirical weighting risks substantial bias
 
nasal decongestant
 
patient level effect size
 
popular empirically
 
population parameter
 
primary analysis
 
random effects analysis
 
random effects meta-analysis
 
sample sizes
 
second method estimates
 
secondary analysis
 
serious question
 
study effect sizes
 
targeted parameter
 
total sample sizes
 
two approaches
 
weighted method
 
weighted random effects meta-analysis
 

Jonathan J Shuster