ArticlePDF Available

Performance of statistical methods for meta-analysis when true study effects are non-normally distributed: A comparison between DerSimonian-Laird and restricted maximum likelihood

Authors:
Evangelos Kontopantelis at The University of Manchester
Evangelos Kontopantelis
David Reeves at The University of Manchester
David Reeves
Download full-text PDF
Read full-text
Download citation

Abstract

Comment on: Performance of statistical methods for meta-analysis when true study effects are non-normally distributed: A simulation study.
Content uploaded by Evangelos Kontopantelis
Author content
All content in this area was uploaded by Evangelos Kontopantelis
Content may be subject to copyright.
1
Letter. Performance of statistical methods for meta-analysis when
true study effects are non-normally distributed: a comparison
between DerSimonian-Laird and Restricted Maximum
Likelihood.
Evangelos Kontopantelis
National Primary Care Research and Development Centre
University of Manchester, Williamson Building 5
th
floor.
Oxford Road, M13 9PL
UK
e.kontopantelis@manchester.ac.uk
David Reeves
Health Sciences Primary Care Research Group
University of Manchester, Williamson Building 5
th
floor.
Oxford Road, M13 9PL
UK
In a recent paper we evaluated the performance of seven different methods for random-
effects meta-analyses under various non-normal distributions for the effect sizes.
1
However,
due to computational limitations we did not include Restricted Maximum Likelihood
(REML) estimator for the between-study variance. Lately, we have observed that the iterative
REML approach has been increasingly replacing the non-iterative DerSimonian-Laird (DL)
as the method of choice in published meta-analyses. Jackson et al examined the performance
of the two methods in terms of coverage, for normally distributed effects only, and found that
results for the two methods were similar.
2
However, REML requires an assumption that study
effects are normally distributed, which DL does not, and so the two methods may differ more
substantially when this assumption is violated.
Using the same simulation method and scenarios as in our previous paper, we assessed
the performance of REML in terms of coverage, power and overall effect estimation when
effect sizes do not follow a normal distribution. REML is a computationally expensive
2
iterative method (it took several months and a few computers to complete the simulations in
STATA
3
) which estimates the between study variance
and effect by maximising the
restricted log-likelihood function:
log
(
,
)
=
1
2
log
{
2
(
+
)}
+
(
)
(
+
)


1
2
log
1
(
+
)

, &
0
(1)
where is the number of studies being combined,
and
are the effect and variance
estimates for study and is the overall effect estimate with =
[


]

[ 

]

. Non-
negativity for
must be enforced at each iteration and iteration continues until convergence
or the maximum number of iterations is reached. REML is considered an improvement over
Maximum Likelihood (ML) since it adjusts for the loss of degrees of freedom due to the
estimation of the overall effect .
4
Non-convergence is a possibility, although it was rare in
our simulations (around 0.1%). The method has been implemented in the STATA command
metaan.
5
Coverage, power and confidence interval estimation (estimated confidence interval as a
percentage of the interval based on the true between-study variance) for the REML method
are presented in table1, with results for DL provided for comparison. The two methods
performed very similarly across all scenarios. In terms of coverage DL outperformed REML
slightly, by a maximum of 2%, particularly when heterogeneity was low. As expected the
picture was reversed with regards to power, with REML performing slightly better
(maximum 2% for
= 0). As heterogeneity and/or the number of studies increased the two
methods converged to almost identical results. However, power for DL caught up somewhat
quicker than coverage for REML. Results for confidence interval estimation do not identify a
clear ‘winner’: DL performed better (maximum 2%) in cases of small or moderate
heterogeneity, and more so with larger number of studies, while REML returned a slightly
more accurate interval (maximum 1%) in certain large study number scenarios with high
heterogeneity. Although the form of the effect size distribution had some overall impact on
performance, it did not alter the comparison of results between methods.
In conclusion, it seems that REML’s performance does not justify the extra level of
complexity associated with the method. In general, DL performed just as well in most
scenarios and scored marginally better in some. We stand by our earlier recommendation to
3
meta-analysts to use either DerSimonian-Laird or Profile Likelihood, depending on the
scenario and the requirements, as described in our paper.
1. Kontopantelis E, Reeves D. Performance of statistical methods for meta-analysis
when true study effects are non-normally distributed: A simulation study. Stat
Methods Med Res.
2. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials 1986;
7(3): 177-88.
3. STATA Statistical Software for Windows: Release 10.0 [program]. 10 version.
College Station, TX: Stata Corporation, 2007.
4. Jackson D, Bowden J, Baker R. How does the DerSimonian and Laird procedure for
random effects meta-analysis compare with its more efficient but harder to compute
counterparts? Journal of statistical planning and inference 2010; 140(4): 961-970.
5. Kontopantelis E, Reeves D. metaan: random effects meta-analysis. The STATA
Journal 2010; 10(3): 395-407.
4
Table 1: Coverage, power and confidence interval estimation by degree of heterogeneity, between-study effect distribution, and number of studies,
assuming
2
1
-based within-study variances
Power (25
th
centile)
Confidence interval estimation
# of studies:
2-5
6-15
16-25
26-35
2-5
6-15
16-25
26-35
2-5
6-15
16-25
26-35
2
H
i
Distribution
(skew, kurtosis)
REML
DL
REML
DL
REML
DL
REML
DL
REML
DL
REML
DL
REML
DL
REML
DL
REML
DL
REML
DL
REML
DL
REML
DL
1
None
0.95
0.96
0.94
0.96
0.94
0.96
0.95
0.96
0.30
0.29
0.69
0.67
0.93
0.92
0.99
0.99
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.18
Normal (0,3)
0.93
0.94
0.92
0.94
0.93
0.94
0.93
0.94
0.31
0.29
0.66
0.65
0.91
0.90
0.98
0.98
0.96
0.96
0.94
0.95
0.96
0.97
0.97
0.98
1.18
Skew-normal (1,4)
0.93
0.94
0.93
0.94
0.93
0.94
0.93
0.94
0.29
0.28
0.66
0.65
0.91
0.90
0.98
0.98
0.96
0.96
0.94
0.95
0.95
0.97
0.97
0.98
1.18
Skew-normal (2,9)
0.93
0.95
0.93
0.94
0.93
0.94
0.94
0.94
0.29
0.28
0.66
0.65
0.92
0.91
0.98
0.98
0.96
0.96
0.94
0.95
0.94
0.96
0.96
0.97
1.18
Uniform
0.93
0.94
0.92
0.94
0.93
0.94
0.93
0.94
0.29
0.28
0.66
0.65
0.91
0.90
0.98
0.98
0.96
0.96
0.94
0.95
0.96
0.97
0.97
0.98
1.18
Bimodal
0.93
0.94
0.92
0.94
0.93
0.94
0.93
0.94
0.30
0.29
0.66
0.65
0.91
0.90
0.98
0.98
0.96
0.96
0.95
0.95
0.96
0.97
0.97
0.98
1.18
D-spike
0.93
0.94
0.92
0.94
0.93
0.94
0.93
0.94
0.30
0.29
0.65
0.64
0.91
0.90
0.98
0.98
1.00
1.00
1.06
1.07
1.13
1.13
1.14
1.15
1.54
Normal (0,3)
0.90
0.91
0.91
0.92
0.92
0.92
0.93
0.93
0.33
0.32
0.65
0.65
0.90
0.90
0.97
0.97
0.90
0.91
0.93
0.94
0.97
0.97
0.98
0.98
1.54
Skew-normal (1,4)
0.90
0.91
0.91
0.92
0.92
0.92
0.93
0.93
0.31
0.30
0.66
0.65
0.91
0.91
0.98
0.98
0.90
0.91
0.93
0.94
0.96
0.97
0.97
0.98
1.54
Skew-normal (2,9)
0.91
0.92
0.91
0.92
0.92
0.93
0.92
0.93
0.29
0.28
0.64
0.63
0.90
0.89
0.98
0.98
0.90
0.90
0.90
0.92
0.94
0.95
0.95
0.96
1.54
Uniform
0.89
0.90
0.90
0.91
0.92
0.92
0.93
0.93
0.32
0.31
0.65
0.64
0.89
0.89
0.98
0.98
0.91
0.91
0.94
0.95
0.97
0.98
0.98
0.99
1.54
Bimodal
0.89
0.90
0.90
0.91
0.91
0.92
0.92
0.92
0.33
0.33
0.67
0.67
0.91
0.91
0.98
0.98
0.91
0.91
0.94
0.95
0.98
0.98
0.99
0.99
1.54
D-spike
0.89
0.90
0.89
0.90
0.91
0.91
0.92
0.92
0.32
0.32
0.67
0.67
0.91
0.91
0.98
0.98
1.04
1.06
1.29
1.30
1.37
1.37
1.38
1.39
2.78
Normal (0,3)
0.86
0.87
0.90
0.90
0.92
0.92
0.93
0.93
0.35
0.35
0.64
0.64
0.87
0.87
0.96
0.96
0.85
0.85
0.94
0.94
0.98
0.97
0.98
0.98
2.78
Skew-normal (1,4)
0.86
0.86
0.89
0.90
0.91
0.92
0.93
0.93
0.32
0.32
0.64
0.63
0.90
0.90
0.98
0.98
0.84
0.84
0.93
0.93
0.97
0.96
0.98
0.97
2.78
Skew-normal (2,9)
0.87
0.88
0.88
0.89
0.90
0.91
0.92
0.92
0.30
0.30
0.64
0.63
0.90
0.90
0.98
0.98
0.80
0.81
0.89
0.88
0.93
0.92
0.95
0.94
2.78
Uniform
0.84
0.85
0.89
0.89
0.92
0.92
0.93
0.93
0.36
0.36
0.66
0.66
0.91
0.91
0.98
0.98
0.86
0.86
0.96
0.96
0.99
0.98
0.99
0.99
2.78
Bimodal
0.82
0.83
0.88
0.88
0.92
0.92
0.93
0.93
0.36
0.36
0.67
0.67
0.92
0.92
0.99
0.99
0.87
0.87
0.97
0.97
0.99
0.99
0.99
0.99
2.78
D-spike
0.80
0.81
0.87
0.87
0.92
0.92
0.93
0.93
0.34
0.34
0.66
0.67
0.92
0.92
0.99
0.99
1.35
1.36
1.80
1.80
1.87
1.87
1.89
1.89
... Means, standard deviations, and sample sizes were the input data for all meta-analyses. The DerSimonian-Laird estimator was used because it does not need normally distributed data [49][50][51]. The heterogeneity of parameters was assessed using Cochran's Q test, with a p < 0.10 indicating evidence of heterogeneity [51]. ...
... The DerSimonian-Laird estimator was used because it does not need normally distributed data [49][50][51]. The heterogeneity of parameters was assessed using Cochran's Q test, with a p < 0.10 indicating evidence of heterogeneity [51]. Heterogeneity was also quantified by the I 2 statistics [50], expressed as a percentage: values greater than 50% indicate substantial heterogeneity [52]. ...
Academic Skills in Students with Autism Spectrum Disorder and Intellectual Disability: A Systematic Review and Meta-Analysis
Article
Full-text available
  • Oct 2023
Background: A substantial proportion of children with autism spectrum disorder (ASD) also have an intellectual disability (ID). However, the academic achievement levels of students with ASD and ID (ASD-ID) are poorly documented and known. Method: We systematically reviewed studies on school skills (reading, spelling, and math) in children and adolescents with ASD-ID. The search was conducted in seven bibliographic databases: Embase, Pubmed/MEDLINE, PsycINFO, Cochrane Library, Ebscohost, Proquest, and Scopus until 28 May 2022. Results: We identified 33,750 reports, four of which met the inclusion criteria for the review. The studies, characterized by Level III evidence (non-randomized controlled trials), included 535 students, 266 in the ASD-ID group and 269 in the ASD-no ID group. A random-effects model meta-analysis revealed that students with ASD-ID had significantly lower reading, spelling, and math scores than students with ASD-no ID. The effect sizes associated with reading score differences were large, although with significant heterogeneity; similarly, the effect sizes associated with spelling and math score differences were also large, although to a lesser extent than for reading. Conclusions: The co-presence of ASD and ID is associated with significant deficits in reading, spelling, and math. However, the present meta-analytic results rest on a limited number of studies. This contrasts with the substantial proportion of children with ASD who have ID and highlights the need for further research to fill a significant gap regarding the profile of academic abilities of students with ASD-ID.
View
... The models in most cases generated residuals, which were non-normally distributed. Although this is a violation of the assumptions behind the models, but the works of Kontopantelis & Reeves (Kontopantelis and Reeves, 2012;Kontopantelis and Reeves, 2010) indicates that this does not have the potential to fundamentally alter the conclusions. To avoid any selection biases that may occur in our case by rejecting datapoints considered as influential outliers (Habeck and Schultz, 2015), all datapoints were included in the meta-analysis as long as the criteria for experiments in that particular analysis were met. ...
Effectiveness of bio-effectors on maize, wheat and tomato performance and phosphorus acquisition from greenhouse to field scales in Europe and Israel: a meta-analysis
Article
Full-text available
  • Apr 2024
Biostimulants (Bio-effectors, BEs) comprise plant growth-promoting microorganisms and active natural substances that promote plant nutrient-acquisition, stress resilience, growth, crop quality and yield. Unfortunately, the effectiveness of BEs, particularly under field conditions, appears highly variable and poorly quantified. Using random model meta-analyses tools, we summarize the effects of 107 BE treatments on the performance of major crops, mainly conducted within the EU-funded project BIOFECTOR with a focus on phosphorus (P) nutrition, over five years. Our analyses comprised 94 controlled pot and 47 field experiments under different geoclimatic conditions, with variable stress levels across European countries and Israel. The results show an average growth/yield increase by 9.3% (n=945), with substantial differences between crops (tomato > maize > wheat) and growth conditions (controlled nursery + field (Seed germination and nursery under controlled conditions and young plants transplanted to the field) > controlled > field). Average crop growth responses were independent of BE type, P fertilizer type, soil pH and plant-available soil P (water-P, Olsen-P or Calcium acetate lactate-P). BE effectiveness profited from manure and other organic fertilizers, increasing soil pH and presence of abiotic stresses (cold, drought/heat or salinity). Systematic meta-studies based on published literature commonly face the inherent problem of publication bias where the most suspected form is the selective publication of statistically significant results. In this meta-analysis, however, the results obtained from all experiments within the project are included. Therefore, it is free of publication bias. In contrast to reviews of published literature, our unique study design is based on a common standardized protocol which applies to all experiments conducted within the project to reduce sources of variability. Based on data of crop growth, yield and P acquisition, we conclude that application of BEs can save fertilizer resources in the future, but the efficiency of BE application depends on cropping systems and environments.
View
... Accordingly, considerable heterogeneity [I 2 = 99.5%, p < 0.001] was detected, and the pooled prevalence of MNM and each severe maternal complication was estimated using a random-effects model with the DerSimonian-Laird method [26]. Furthermore, the adjusted odds ratio (AOR) and 95% CIs were extracted, and the pooled estimates were computed using a random-or fixed-effect model based on their level of heterogeneity. ...
Road to maternal death: the pooled estimate of maternal near-miss, its primary causes and determinants in Africa: a systematic review and meta-analysis
Article
Full-text available
  • Feb 2024
Background Maternal near-miss (MNM) is defined by the World Health Organization (WHO) working group as a woman who nearly died but survived a life-threatening condition during pregnancy, childbirth, or within 42 days of termination of pregnancy due to getting quality of care or by chance. Despite the importance of the near-miss concept in enhancing quality of care and maternal health, evidence regarding the prevalence of MNM, its primary causes and its determinants in Africa is sparse; hence, this study aimed to address these gaps. Methods A systematic review and meta-analysis of studies published up to October 31, 2023, was conducted. Electronic databases (PubMed/Medline, Scopus, Web of Science, and Directory of Open Access Journals), Google, and Google Scholar were used to search for relevant studies. Studies from any African country that reported the magnitude and/or determinants of MNM using WHO criteria were included. The data were extracted using a Microsoft Excel 2013 spreadsheet and analysed by STATA version 16. Pooled estimates were performed using a random-effects model with the DerSimonian Laired method. The I2 test was used to analyze the heterogeneity of the included studies. Results Sixty-five studies with 968,555 participants were included. The weighted pooled prevalence of MNM in Africa was 73.64/1000 live births (95% CI: 69.17, 78.11). A high prevalence was found in the Eastern and Western African regions: 114.81/1000 live births (95% CI: 104.94, 123.59) and 78.34/1000 live births (95% CI: 67.23, 89.46), respectively. Severe postpartum hemorrhage and severe hypertension were the leading causes of MNM, accounting for 36.15% (95% CI: 31.32, 40.99) and 27.2% (95% CI: 23.95, 31.09), respectively. Being a rural resident, having a low monthly income, long distance to a health facility, not attending formal education, not receiving ANC, experiencing delays in health service, having a previous history of caesarean section, and having pre-existing medical conditions were found to increase the risk of MNM. Conclusion The pooled prevalence of MNM was high in Africa, especially in the eastern and western regions. There were significant variations in the prevalence of MNM across regions and study periods. Strengthening universal access to education and maternal health services, working together to tackle all three delays through community education and awareness campaigns, improving access to transportation and road infrastructure, and improving the quality of care provided at service delivery points are key to reducing MNM, ultimately improving and ensuring maternal health equity.
View
... Accordingly, considerable heterogeneity [I 2 = 99.5%, p < 0.001] was detected, and the pooled prevalence of MNM and each severe maternal complication was estimated using a random-effects model with the DerSimonian-Laird method [26]. Furthermore, the adjusted odds ratio (AOR) and 95% CIs were extracted, and the pooled estimates were computed using a random-or fixed-effect model based on their level of heterogeneity. ...
Road to maternal death: the pooled estimate of maternal near-miss, its primary causes and determinants in Africa: a systematic review and meta-analysis
Article
Full-text available
  • Feb 2024
Background Maternal near-miss (MNM) is defined by the World Health Organization (WHO) working group as a woman who nearly died but survived a life-threatening condition during pregnancy, childbirth, or within 42 days of termination of pregnancy due to getting quality of care or by chance. Despite the importance of the near-miss concept in enhancing quality of care and maternal health, evidence regarding the prevalence of MNM, its primary causes and its determinants in Africa is sparse; hence, this study aimed to address these gaps. Methods A systematic review and meta-analysis of studies published up to October 31, 2023, was conducted. Electronic databases (PubMed/Medline, Scopus, Web of Science, and Directory of Open Access Journals), Google, and Google Scholar were used to search for relevant studies. Studies from any African country that reported the magnitude and/or determinants of MNM using WHO criteria were included. The data were extracted using a Microsoft Excel 2013 spreadsheet and analysed by STATA version 16. Pooled estimates were performed using a random-effects model with the DerSimonian Laired method. The I² test was used to analyze the heterogeneity of the included studies. Results Sixty-five studies with 968,555 participants were included. The weighted pooled prevalence of MNM in Africa was 73.64/1000 live births (95% CI: 69.17, 78.11). A high prevalence was found in the Eastern and Western African regions: 114.81/1000 live births (95% CI: 104.94, 123.59) and 78.34/1000 live births (95% CI: 67.23, 89.46), respectively. Severe postpartum hemorrhage and severe hypertension were the leading causes of MNM, accounting for 36.15% (95% CI: 31.32, 40.99) and 27.2% (95% CI: 23.95, 31.09), respectively. Being a rural resident, having a low monthly income, long distance to a health facility, not attending formal education, not receiving ANC, experiencing delays in health service, having a previous history of caesarean section, and having pre-existing medical conditions were found to increase the risk of MNM. Conclusion The pooled prevalence of MNM was high in Africa, especially in the eastern and western regions. There were significant variations in the prevalence of MNM across regions and study periods. Strengthening universal access to education and maternal health services, working together to tackle all three delays through community education and awareness campaigns, improving access to transportation and road infrastructure, and improving the quality of care provided at service delivery points are key to reducing MNM, ultimately improving and ensuring maternal health equity.
View
... We will use random effects models using the statistical software SPSS (Statistical Package for Social Sciences), version 22 [93]. Random effects models are considered better than fixed-effects models in case of heterogeneity and a small number of studies [94,95]. We will also assess if study protocols are available for the included studies (particularly randomized controlled trials) and if they were published before recruitment of patients. ...
Factors influencing health-related quality of life among long-term care residents experiencing pain: a systematic review protocol
Article
Full-text available
  • Feb 2024
Background Pain is highly burdensome, affecting over 30% of long-term care (LTC) residents. Pain significantly reduces residents’ health-related quality of life (HRQoL), limits their ability to perform activities of daily living (ADLs), restricts their social activities, and can lead to hopelessness, depression, and unnecessary healthcare costs. Although pain can generally be prevented or treated, eliminating pain may not always be possible, especially when residents have multiple chronic conditions. Therefore, improving the HRQoL of LTC residents with pain is a priority goal. Understanding factors influencing HRQoL of LTC residents with pain is imperative to designing and evaluating targeted interventions that complement pain management to improve residents’ HRQoL. However, these factors are poorly understood, and we lack syntheses of available research on this topic. This systematic review protocol outlines the methods to identify, synthesize, and evaluate the available evidence on these factors. Methods This mixed methods review will follow the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines. We will systematically search Medline, EMBASE, PsycINFO, CINAHL, Scopus, Cochrane Database of Systematic Reviews and ProQuest Dissertation and Thesis Global from database inception. We will include primary studies and systematically conducted reviews without restrictions to language, publication date, and study design. We will also include gray literature (dissertation and reports) and search relevant reviews and reference lists of all included studies. Two reviewers will independently screen articles, conduct quality appraisal, and extract data. We will synthesize results thematically and conduct meta-analyses if statistical pooling is possible. Residents and family/friend caregivers will assist with interpreting the findings. Discussion This proposed systematic review will address an important knowledge gap related to the available evidence on factors influencing HRQoL of LTC residents with pain. Findings will be crucial for researchers, LTC administrators, and policy makers in uncovering research needs and in planning, developing, and evaluating strategies in addition to and complementary with pain management to help improve HRQoL among LTC residents with pain. Systematic review registration PROSPERO CRD42023405425
View
... A random-effects meta-analysis using restricted maximum likelihood estimation was conducted to pool Cronbach's alpha/ICC values across studies [23]. Cronbach's alpha and the ICC are correlation coefficients that range from 0 to 1, measuring the internal consistency and reliability of scales or ratings [24]. ...
Evaluating the reliability of the athlete sleep behavior questionnaire (ASBQ): a meta-analysis of Cronbach’s alpha and intraclass correlation coefficient
Article
Full-text available
  • Jan 2024
Background The Athlete Sleep Behavior Questionnaire (ASBQ) was designed to identify maladaptive sleep practices among athletes. The aim of this meta-analysis was to evaluate the internal consistency and the test-retest reliability coefficients of the ASBQ. Methods A systematic search across 10 databases from inception of the ASBQ to August 2023 was performed. Publications that reported estimates of internal consistency and/or test-retest reliability of the ASBQ were included. A random-effects model was employed to estimate the overall reliability measures of the ASBQ. Results Meta-analytic results demonstrated a good level of internal consistency within the ASBQ, evidenced by a Cronbach’s alpha of 0.73 (95% CI: 0.63 to 0.80). This suggests a modest correlation among the questionnaire items, supporting its reliability as an effective measure of sleep behavior. In terms of test-retest reliability, our meta-analysis revealed a very good degree of consistency (ICC = 0.88; 95% CI: 0.87 to 0.89), suggesting that the ASBQ can serve as an instrument for monitoring and evaluating changes in athletes’ sleep behavior over time. No evidence of publication bias was identified. Conclusion While the ASBQ demonstrates a moderate level of internal consistency, its test-retest reliability suggests that it can serve as an instrument for longitudinal assessments of athletes’ sleep behavior. Future studies focusing on refining the ASBQ to optimize its internal consistency and validate its applicability across diverse athletic populations are warranted.
View
Improving the Estimation of Site-Specific Effects and Their Distribution in Multisite Trials
Article
  • Aug 2024
In multisite trials, researchers are often interested in several inferential goals: estimating treatment effects for each site, ranking these effects, and studying their distribution. This study seeks to identify optimal methods for estimating these targets. Through a comprehensive simulation study, we assess two strategies and their combined effects: semiparametric modeling of the prior distribution and alternative posterior summary methods tailored to minimize specific loss functions. Our findings highlight that the success of different estimation strategies depends largely on the amount of within-site and between-site information available from the data. We discuss how our results can guide balancing the trade-offs associated with shrinkage in limited data environments.
View
View
Efficacy of virtual reality for pain relief in medical procedures: a systematic review and meta-analysis
Article
Full-text available
  • Feb 2024
  • BMC MED
Background Effective pain control is crucial to optimise the success of medical procedures. Immersive virtual reality (VR) technology could offer an effective non-invasive, non-pharmacological option to distract patients and reduce their experience of pain. We aimed to evaluate the efficacy of Immersive virtual reality (VR) technology in reducing patient’s pain perception during various medical procedures by conducting a systematic review and meta-analysis. Methods We searched MEDLINE, EMBASE, CENTRAL, CINAHL, and SIGLE until December 2022 for all randomised clinical trials (RCT) evaluating any type of VR in patients undergoing any medical procedure. We conducted a random effect meta-analysis summarising standardised mean differences (SMD) with 95% confidence intervals (CI). We evaluated heterogeneity using I² and explored it using subgroup and meta-regression analyses. Results In total, we included 92 RCTs (n = 7133 participants). There was a significant reduction in pain scores with VR across all medical procedures (n = 83, SMD − 0.78, 95% CI − 1.00 to − 0.57, I² = 93%, p = < 0.01). Subgroup analysis showed varied reduction in pain scores across trial designs [crossover (n = 13, SMD − 0.86, 95% CI − 1.23 to − 0.49, I² = 72%, p = < 0.01) vs parallel RCTs (n = 70, SMD − 0.77, 95% CI − 1.01 to − 0.52, I² = 90%, p = < 0.01)]; participant age groups [paediatric (n = 43, SMD − 0.91, 95% CI − 1.26 to − 0.56, I² = 87%, p = < 0.01) vs adults (n = 40, SMD − 0.66, 95% CI − 0.94 to − 0.39, I² = 89%, p = < 0.01)] or procedures [venepuncture (n = 32, SMD − 0.99, 95% CI − 1.52 to − 0.46, I² = 90%, p = < 0.01) vs childbirth (n = 7, SMD − 0.99, 95% CI − 1.59 to − 0.38, I² = 88%, p = < 0.01) vs minimally invasive medical procedures (n = 25, SMD − 0.51, 95% CI − 0.79 to − 0.23, I² = 85%, p = < 0.01) vs dressing changes in burn patients (n = 19, SMD − 0.8, 95% CI − 1.16 to − 0.45, I² = 87%, p = < 0.01)]. We explored heterogeneity using meta-regression which showed no significant impact of different covariates including crossover trials (p = 0.53), minimally invasive procedures (p = 0.37), and among paediatric participants (p = 0.27). Cumulative meta-analysis showed no change in overall effect estimates with the additional RCTs since 2018. Conclusions Immersive VR technology offers effective pain control across various medical procedures, albeit statistical heterogeneity. Further research is needed to inform the safe adoption of this technology across different medical disciplines.
View
Efficacy of Prescribed Opioids for Acute Pain after Being Discharged from the Emergency Department: A Systematic Review and Meta-Analysis
Article
  • Aug 2023
  • ACAD EMERG MED
Background Opioids are often prescribed for acute pain to emergency department (ED) discharged patients, but there is a paucity of data on their short-term use. The purpose of this study was to synthesize the evidence regarding the efficacy of prescribed opioids compared to non-opioid analgesics for acute pain relief in ED-discharged patients. Methods MEDLINE, EMBASE, CINAHL, PsycINFO, CENTRAL, and gray literature databases were searched from inception to January 2023. Two independent reviewers selected randomized controlled trials investigating the efficacy of prescribed opioids for ED-discharged patients, extracted data and assessed risk of bias. Authors were contacted for missing data and to identify additional studies. The primary outcome was the difference in pain intensity scores or pain relief. All meta-analyses used random-effect model and a sensitivity analysis compared patients treated with codeine versus those treated with other opioids. Results From 5,419 initially screened citations, 46 full texts were evaluated and six studies enrolling 1,161 patients were included. Risk of bias was low for five studies. There was no statistically significant difference in pain intensity scores or pain relief between opioids versus non-opioid analgesics (standardized mean difference [SMD]:0.12; 95%CI: −0.10 to 0.34). Contrary to children, adult patients treated with opioid had better pain relief (SMD: 0.28; 95%CI: 0.13-0.42) compared to non-opioids. In another sensitivity analysis excluding studies using codeine, opioids were more effective than non-opioids (SMD: 0.30; 95%CI: 0.15-0.45). However, there were more adverse events associated with opioids (odds ratio: 2.64; 95%CI: 2.04-3.42). Conclusions For ED-discharged patients with acute musculoskeletal pain, opioids do not seem to be more effective than non-opioid analgesics. However, this absence of efficacy seems to be driven by codeine, as opioids other than codeine are more effective than non-opioids (mostly NSAIDs). Further prospective studies on the efficacy of short-term opioid use after ED discharge (excluding codeine), measuring patient-centered outcomes, adverse events, and potential misuse, are needed.
View
Metaan: Random-effects Meta-analysis
Article
Full-text available
  • Aug 2010
This article describes the new meta-analysis command metaan, which can be used to perform fixed- or random-effects meta-analysis. Besides the stan- dard DerSimonian and Laird approach, metaan offers a wide choice of available models: maximum likelihood, profile likelihood, restricted maximum likelihood, and a permutation model. The command reports a variety of heterogeneity mea- sures, including Cochran’s Q, I2, HM2 , and the between-studies variance estimate τb2. A forest plot and a graph of the maximum likelihood function can also be generated.
View
Performance of statistical methods for meta-analysis when true study effects are non-normally distributed: A simulation study
Article
Full-text available
  • Dec 2010
  • STAT METHODS MED RES
Meta-analysis (MA) is a statistical methodology that combines the results of several independent studies considered by the analyst to be 'combinable'. The simplest approach, the fixed-effects (FE) model, assumes the true effect to be the same in all studies, while the random-effects (RE) family of models allows the true effect to vary across studies. However, all methods are only correct asymptotically, while some RE models assume that the true effects are normally distributed. In practice, MA methods are frequently applied when study numbers are small and the normality of the effect distribution unknown or unlikely. In this article, we discuss the performance of the FE approach and seven frequentist RE MA methods: DerSimonian-Laird, Q-based, maximum likelihood, profile likelihood, Biggerstaff-Tweedie, Sidik-Jonkman and Follmann-Proschan. We covered numerous scenarios by varying the MA sizes (small to moderate), the degree of heterogeneity (zero to very large) and the distribution of the effect sizes (normal, skew-normal and 'extremely' non-normal). Performance was evaluated in terms of coverage (Type I error), power (Type II error) and overall effect estimation (accuracy of point estimates and error intervals).
View
How does the DerSimonian and Laird procedure for random effects meta-analysis compare with its more efficient but harder to compute counterparts?
Article
  • Apr 2010
  • J STAT PLAN INFER
The procedure suggested by DerSimonian and Laird is the simplest and most commonly used method for fitting the random effects model for meta-analysis. Here it is shown that, unless all studies are of similar size, this is inefficient when estimating the between-study variance, but is remarkably efficient when estimating the treatment effect. If formal inference is restricted to statements about the treatment effect, and the sample size is large, there is little point in implementing more sophisticated methodology. However, it is further demonstrated, for a simple special case, that use of the profile likelihood results in actual coverage probabilities for 95% confidence intervals that are closer to nominal levels for smaller sample sizes. Alternative methods for making inferences for the treatment effect may therefore be preferable if the sample size is small, but the DerSimonian and Laird procedure retains its usefulness for larger samples.
View
Meta-Analysis in Clinical Trials
Article
  • Oct 1986
  • Contr Clin Trials
This paper examines eight published reviews each reporting results from several related trials. Each review pools the results from the relevant trials in order to evaluate the efficacy of a certain treatment for a specified medical condition. These reviews lack consistent assessment of homogeneity of treatment effect before pooling. We discuss a random effects approach to combining evidence from a series of experiments comparing two treatments. This approach incorporates the heterogeneity of effects in the analysis of the overall treatment efficacy. The model can be extended to include relevant covariates which would reduce the heterogeneity and allow for more specific therapeutic recommendations. We suggest a simple noniterative procedure for characterizing the distribution of treatment effects in a series of studies.
View