No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Grey matters; on the importance of publication bias in systematic reviews
Abstract
‘How can I treat my patient best?’ is a question clinicians ask themselves on a daily basis. For clinical inquiries, systematic reviews (SRs) deliver the highest level of evidence in medical sciences.1 They can cover a variety of domains, including treatment and diagnostics. Presenting a full overview of the literature and delivering an unbiased estimate of effect are key aims of SRs. In this editorial, we explore publication bias, one of the potential sources of bias in SRs.
Authors of SRs usually search multiple databases to find relevant publications. However, is all the available evidence identified with this approach? Publication bias is one threat to delivering an unbiased effect estimate.2 ,3 Publication bias is a phenomenon where journals are more likely to publish positive results, as these articles yield more citations. Non-significant publications are more prone to be rejected and remain unpublished.4 Unpublished resources are known as ‘grey literature’.
Publication bias has been shown in …
... Current automated translation technologies and the accessibility of native human translators mean that restricting literature searches by the language of publication is less necessary and justifiable. (iv) Consider many shades of gray: gray literature 3 (i.e., conference proceedings, PhD theses, pharmaceutical study reports) should be consulted in addition to published peer-reviewed literature [77]. Although some may argue that more reliable conclusions would be derived from peer-reviewed randomized studies, moderator analyses can be considered to compare between data derived from gray and non-gray literature. ...
Publication bias refers to a systematic deviation from the truth in the results of a meta-analysis due to the higher likelihood for published studies to be included in meta-analyses than unpublished studies. Publication bias can lead to misleading recommendations for decision- and policy-making. In this education review, we introduce, explain, and provide solutions to the pervasive misuses and misinterpretations of publication bias that afflict evidence syntheses in sport and exercise medicine, with a focus on the commonly used funnel-plot based methods. Publication bias is more routinely assessed by visually inspecting funnel plot asymmetry, although it has been consistently deemed unreliable, leading to the development of statistical tests to assess publication bias. However, most statistical tests of publication bias (i) cannot rule out alternative explanations for funnel plot asymmetry (e.g., between-study heterogeneity, choice of metric, chance), and (ii) are grossly underpowered, even when using an arbitrary minimum threshold of ≥10 studies. We performed a cross-sectional, meta-research investigation of how publication bias was assessed in systematic reviews with meta-analysis published in the top two sport and exercise medicine journals throughout 2021. This analysis highlights that publication bias is frequently misused and misinterpreted, even in top tier journals. Due to conceptual and methodological problems when assessing and interpreting publication bias, preventive strategies (e.g., pre-registration, registered reports, disclosing protocol deviations, and reporting all study findings regardless of direction or magnitude) offer the best and most efficient solution to mitigate the misuse and misinterpretation of publication bias. Because true publication bias is very difficult to determine, we recommend that future publications use the term “risk of publication bias”.
... Later, in the published manuscript, disclose all deviations from the pre-registered protocol and the reasons for these alterations. (iv) Consider many shades of grey: Grey literature 1 (i.e., conference proceedings, PhD theses, pharmaceutical study reports) should be consulted in addition to published, peer-reviewed literature [74]. Although some may argue that more reliable conclusions would be derived from peer-reviewed randomized studies, moderator analyses can be considered to compare between data derived from grey and nongrey literature. ...
... Publication bias is a substantial problem for the credibility of meta-analytic results, as it yields overestimated effects and may suggest the presence of non-existent effects [98]. Although attempts were made to limit publication bias through grey literature searches and visual inspections of funnel plots [99], some unpublished trials could have been missed which may have inflated effect size estimates. Furthermore, the trim-and-fill method has been criticised for having a high false positive rate which needs to be considered when interpreting the findings [100]. ...
Plain English summary Feeding and eating disorders can significantly impair health and psychosocial functioning. However, demand for eating disorder services is greater than services’ ability to deliver effective treatment. Low intensity psychological interventions, which are brief in nature and require less therapist input than standard treatments, have the potential to bridge this demand-capacity gap. The current review examined the effectiveness of low intensity psychological interventions for the treatment of feeding and eating disorders. Overall, findings suggest that low intensity psychological interventions can successfully treat eating disorder symptoms, particularly binge eating-related symptoms. Given their relatively low costs and ease of accessibility, such interventions can help people to access treatment at a time when this is so desperately needed. More research is needed to determine the value of low intensity psychological interventions for children and adolescents, and people with feeding and eating disorders that are not characterised by recurrent binge eating, such as anorexia nervosa, ARFID, pica and rumination disorder.
... For gray literature retrieval, only 4.55% (n = 1) of the SRs/MAs searched the gray literature database [14], and 27.27% (n = 6) searched the clinical trial registration platform [14,[30][31][32]40,46]. Winters and Weir [100] proposed three methods to control the publication bias, namely, (1) registering before the start of the trial, (2) using a funnel chart to evaluate the publication bias intuitively, and (3) using a comprehensive retrieval strategy to retrieve multiple gray literature data. Unpublished studies may significantly change the clinical effect of SRs/MAs, and more comprehensive literature retrieval strategies should be developed in the future. ...
Clinical guidelines need high-quality studies to support clinical decision-making, in which the evidence often was collected from systematic reviews (SRs) and/or meta-analyses (MAs). At present, the methodological quality and risk of bias (RoB) of SRs/MAs on stem cell therapy for the treatment of knee osteoarthritis (KOA) has been poorly investigated. This study aims to strictly evaluate the methodological quality and RoB in SRs/MAs of stem cell therapy for KOA. Four electronic databases (PubMed, Embase, Cochrane Library, and Web of Science databases) were searched, from inception to October 5th, 2021. SRs/MAs involving randomized control trials (RCTs) or cohort studies on stem cell therapy for the treatment of KOA were included. The methodological quality and RoB were assessed using AMSTAR 2 and ROBIS tool respectively. In total, 22 SRs/MAs were included. According to the results obtained by AMSTAR 2 tool, all SRs/MAs were rated as "Critically low". Main methodological weaknesses were as follows: up to 81.82% did not meet protocol registration requirements, only 13.64% provided a list of excluded studies and justification, and 13.64% investigated and discussed the publication bias.. ROBIS-based RoB assessment showed that all the SRs/MAs were rated as "High". Besides, the lack of following the implementation of the PRISMA reporting guideline seems to reduce the methodological quality of the studies. The overall methodological quality of the SRs/MAs concerning the application of stem cell therapy in treating KOA is "Critically low", while the RoB is high. It is difficult to provide effective evidence for the formulation of guidelines for KOA treatment. We suggest that the relevant methodological quality assessment should be carried out in the future before the SRs/MAs are used as clinical evidence. In addition, it may be necessary for many journals to include the checklist with a submitted manuscript.
... For gray literature retrieval, only 4.55% (n=1) of the SRs/MAs searched the gray literature database [11], and 27.27% (n=6) searched the clinical trial registration platform [11, 27-29, 37, 43]. Winters et al. [97] proposed three methods to control the publication bias, namely, (1) registering before the start of the trial, (2) using a funnel chart to evaluate the publication bias intuitively, and (3) using a comprehensive retrieval strategy to retrieve multiple gray literature data. Unpublished studies may signi cantly change the clinical effect of SRs/MAs, and more comprehensive literature retrieval strategies should be developed in the future. ...
Background: Clinical guidelines need high-quality studies to support clinical decision-making, in which the evidence often was collected from systematic reviews (SRs) and/or meta-analyses (MAs). At present, the methodological quality and risk of bias (RoB) of SRs/MAs on stem cell therapy for the treatment of knee osteoarthritis (KOA) has been poorly investigated. This study aims to strictly evaluate the methodological quality and RoB in SRs/MAs of stem cell therapy for KOA.
Methods: Four electronic databases (PubMed, Embase, Cochrane Library, and Web of Science databases) were searched, from inception to October 5th, 2021. SRs/MAs involving RCTs or cohort studies on stem cell therapy for the treatment of KOA were included. The methodological quality and RoB were assessed using AMSTAR 2 and ROBIS tool respectively.
Results: In total, 22 SRs/MAs were included. According to the results obtained by AMSTAR 2 tool, all SRs/MAs were rated as “Critically low”. Main methodological weaknesses were as follows: eight items accounted for more than 50% of “No”, including Items 2 Protocol registration (81.82%), Item 7 Study exclusion and justification (86.36%) and Item 15 Investigation and discussion of publication bias (63.64%) were critical items. ROBIS-based RoB assessment showed that all the SRs/MAs were rated as “High”.
Conclusions: The overall methodological quality of the SRs/MAs concerning the application of stem cell therapy in treating KOA is “Critically low”, while the RoB is high. It is difficult to provide effective evidence for the formulation of guidelines for KOA treatment. We suggest that the relevant methodological quality assessment should be carried out in the future before the SRs/MAs are used as clinical evidence.
PROSPERO registration number: CRD42021246924
... p = .03. Taken together, publication bias is quite possible, although the inclusion of grey literature, such as the undergraduate theses, in these findings would prevent publication bias from being a substantial concern (Winters & Weir, 2017). ...
Background: Multitasking while reading is a commonplace activity. Many studies have been conducted examining the effect of multitasking on reading comprehension and times. The purpose of this meta-analysis is to consolidate the empirical findings on reading comprehension and times in order to understand the overall effect of multitasking on reading. Characteristics of the reading situation, comprehension assessment, and the secondary task were examined to determine if they varied the effect of multitasking.
Methods: A systematic search of studies on multitasking and reading was conducted. Only studies that used random assignment and had participants reading independently were included. This screening yielded a total of 22 independent studies (20 reports) that met inclusion criteria, with 20 studies on reading comprehension and 9 studies on reading times. Most of the studies involved adults reading expository texts.
Results: Based on Robust Variance Estimation (RVE) analyses, multitasking had a negative effect on reading comprehension (g = -0.28, p = .002). The effect was similar after outliers were removed, (g = -0.26, p = .001). Based on moderator analyses, this negative effect may only occur when time was limited because the reading pace was controlled by the experimenter (g = -0.54, p < .001) as there was not a reliable effect when reading was self-paced (g = -0.14, p = .10). Multitasking during reading lead to longer reading times (g = 0.52, p < .001).
Conclusions: Multitasking during reading is detrimental to reading comprehension when time is limited. When readers control their own pace of reading, multitasking lengthens the time for the reading task. Therefore, multitasking while reading is less efficient than focusing attention on the primary task of reading.
... This combination of databases was chosen to reduce the risk of publication bias. 21 ...
Background:
The benefits and risks of performing popular wrist-loading sports at a young age have not been investigated systematically. We aimed to evaluate positive and negative long-term wrist-related health effects of sports performance requiring repetitive wrist loading during youth.
Methods:
Six databases were searched for cohort and cross-sectional studies. Three investigators selected studies evaluating quantitatively measured health effects of upper extremities in athletes practicing wrist-loading sports (gymnastics, tennis, volleyball, field hockey, rowing, and judo) for minimally 4 months before the age of 18.
Results:
A total of 23 studies with 5 outcome measures, nearly all of moderate to good quality, were eligible for inclusion. Bone mineral density and bone mineral content were higher in athletes compared with controls and in tennis players' dominant arm. Mixed results were found for ulnar variance in gymnasts. Handgrip strength was greater in tennis players' dominant arm and in experienced gymnasts.
Conclusions:
Wrist-loading sports performance during youth can promote bone strength in wrists and dominant handgrip strength, but evidence on the lasting of these effects and on prevalence of wrist joint degeneration in former young athletes is limited. For better counseling of young athletes and their parents, future studies with increased comparability are essential, for which recommendations are provided.
Meta-analyses constitute fundamental tools of the Evidence-Based Medicine (EBM) aiming at synthesizing outcome data from individual trials in order to produce pooled effect estimates for various outcomes of interest. Combining summary data from several studies increases the sample size, improves the statistical power of the findings as well as the precision of the obtained effect estimates. For all these reasons, meta-analyses are thought of providing the best evidence to support clinical practice guidelines. However, the strength of the provided evidence is closely dependent on the quality of included studies as well as the rigour of the meta-analytic process. In addition, over the course of the evolution of the current meta-analytic methodology, some concerns have been expressed on the ultimate usefulness of such a complex and time consuming procedure on establishing timely, valid evidence on various specified topics in the field of Orthopaedics and Trauma Surgery. This article provides an overview of the appropriate methodology, benefits and potential drawbacks of the meta-analytic procedure.
Standardized reading assessments are often used as an admissions criterion for college admittance, however, the relationship and predictive validity of reading assessments to academic achievement remains in question. Through a quantitative review of the literature, we conducted a meta-analysis to examine how well performance on college reading assessments (e.g., ACT, COMPASS, Nelson-Denny, SAT) correlated with academic achievement (GPA and college grades). Additionally, to help explain the variation in previous findings, we examined whether the type of reading assessment used, performance indicator, publication bias, or year of publication served as moderators. Results based on 26 studies and a total of 25,090 students revealed a small association between performance on reading comprehension assessments and college grades (r = .29, SE = .02, 95% CI [ .25, .33], p < .001), with no variation based on study moderators. These findings highlight the importance of college student’s reading comprehension skills for college academic achievement.
Reporting guidelines are international standards that all researchers are required to follow during the writing phase. The implementation of them help the researchers to ensure transparency and reproducibility of the research, the clinicians to understand what was done and why, and the policy makers to implement results in practice.
The PRISMA 2020 (Preferred Reporting Items for Systematic Reviews and Meta Analysis) is the reporting guideline for systematic reviews. To ease the application of it in musculoskeletal rehabilitation and sports medicine fields, it has been published the 'Prisma in Exercise, Rehabilitation, Sport medicine and SporTs science' (PERSiST), which is the reference standard for the systematic review writing in these fields.
For this reason, we have worked on an italian translation of this paper, which we think is useful for everyone who works on research and production of systematic reviews related to these topics. Lastly, we also believe that it can be a useful tool for the writing of a Degree, Master or Doctoral thesis.
Poor reporting of medical and healthcare systematic reviews is a problem from which the sports and exercise medicine, musculoskeletal rehabilitation, and sports science fields are not immune. Transparent, accurate and comprehensive systematic review reporting helps researchers replicate methods, readers understand what was done and why, and clinicians and policy-makers implement results in practice. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement and its accompanying Explanation and Elaboration document provide general reporting examples for systematic reviews of healthcare interventions. However, implementation guidance for sport and exercise medicine, musculoskeletal rehabilitation, and sports science does not exist. The Prisma in Exercise, Rehabilitation, Sport medicine and SporTs science (PERSiST) guidance attempts to address this problem. Nineteen content experts collaborated with three methods experts to identify examples of exemplary reporting in systematic reviews in sport and exercise medicine (including physical activity), musculoskeletal rehabilitation (including physiotherapy), and sports science, for each of the PRISMA 2020 Statement items. PERSiST aims to help: (1) systematic reviewers improve the transparency and reporting of systematic reviews and (2) journal editors and peer reviewers make informed decisions about systematic review reporting quality.
Keywords: evaluation; implementation; meta-analysis; methodology.
Introduction Patellofemoral pain (PFP) affects 1 in every 14 adults. Many treatments for PFP have been evaluated, but the comparative effectiveness of all available treatments has never been examined. Network meta-analysis is the only design to study the comparative effectiveness of all available treatments in one synthesis. This protocol describes the methods for a systematic review including network meta-analysis to assess which treatment is most likely to be effective for patients with PFP.
Methods and analysis The primary outcome measures of this network meta-analysis are the global rating of change scale at 6–12 weeks, 13–52 weeks and >52 weeks. The secondary outcome measures are patient-rated pain scales at 6–12 weeks, 13–52 weeks and >52 weeks. Completed published and unpublished randomised controlled trials with full-text reports are eligible for inclusion. We will search Embase, PubMed (including MEDLINE), CENTRAL, Scopus, Web of Science, and CINAHL, SPORTDiscus, OpenGrey, WorldCat, conference Proceedings and multiple trial registers for relevant reports. Two researchers will appraise the study eligibility and perform data extraction. Risk of bias will be assessed with the Cochrane Risk of Bias Tool V.2.0.
Bayesian network meta-analyses will be constructed for global rating of change scale and patient-rated pain. Consistency between direct and indirect comparisons will be assessed. Between study variability will be explored, and a threshold analysis for the credibility of the network meta-analyses’ conclusions will be performed.
Ethics and dissemination Ethical approval is not required, as this study will be based on published data. The study commenced at 1 February 2018, and its expected completion date is 15 January 2019. Full publication of the work will be sought in an international peer-reviewed journal, as well as translational articles to disseminate the work to clinical practitioners.
PROSPERO registration number CRD42018079502.
Medial tibial stress syndrome (MTSS), also known as shin splints, is one of the most common sports injuries. Although 20% of the jumping and running athletes have MTSS at some point while engaging in sporting activities, we know little about it. There is a lack of knowledge regarding making the diagnosis, how to treat it effectively and how to best measure outcomes that are relevant to the patient. The diagnosis MTSS is commonly made using history and physical examination. This seems the logical diagnostic approach because MTSS is a pain syndrome without an established tissue pathology. Whether this approach is reliable between different clinicians is unknown. A number of interventions have been studied in randomised controlled trials over the past 40 years. These include shockwave therapy, lower leg braces, dry needling, lower leg stockings, strengthening exercises, a graded running rehabilitation program and ice massage. Which intervention is most effective, however, has remained unclear. Measuring outcomes that matter to the patient are highly important. These so-called patient-reported outcome measures (PROMs) are considered the cornerstone for outcome assessment and measuring treatment success in medicine. There was no PROM for athletes with MTSS prior to this thesis’ commencement, which prevented the standardised measurement of outcomes in athletes with MTSS. A number of studies, described in this thesis, sought to fill these important gaps in the field. This thesis reports that making the diagnosis MTSS based on history and physical examination has almost perfect reliability among clinicians: k = 0.89 (95%
CI 0.74 to 1.00), p<0.000001. This supports making the diagnosis of MTSS clinically, without using additional and expensive imaging. A systematic review showed that the research previously performed on the treatment of MTSS is at such a high risk of bias, it’s impossible to recommend any particular choice for use in clinical practice. At this point in time, the most logical treatment is load management, increasing loading capacity through gradual load exposure and strengthening the calf muscles. Lastly, this thesis describes the development and validation of a new PROM: the MTSS score. Items for this outcome measure were developed together with researchers, clinicians and patients. The items and scale were tested for their validity, reliability and responsiveness in a large population of 133 athletes with MTSS recruited from 15 sports medicine, military medicine and physiotherapy practices in The Netherlands. This study showed that the MTSS score is a valid, reliable and responsive 4-item scale that can be used to assess outcomes relevant to the athlete with MTSS.
Introduction:
TXA has been evaluated in THA in several randomized controlled trials for the past 16 years. We attempted to evaluate the trends in the evidence using recursive cumulative meta-analysis and publication bias using Rosenthal and 'Trim and Fill' methods.
Methods:
Electronic databases were searched for randomized controlled clinical trials comparing TXA with either placebo (or no TXA administration) or TXA, administered through different routes in patients with osteoarthritis or osteonecrosis of the hip who underwent THA. We considered the total number of patients requiring blood transfusion as the clinical outcome for both the analyses and used quality effects model for assessing the changes in the pooled estimates with addition of new clinical trial data. We also assessed the publication bias by plotting individual study estimates with the standard errors using Rosenthal and 'Trim and Fill'methods.
Results:
A total of 20 studies were included. The pooled cumulative meta-analysis indicates that from 2014 the addition of estimate from new studies in the following years is only narrowing the confidence interval without any significant change in the point estimate. Rosenthal fail-safe-N for the comparisons of intravenous bolus TXA, intra-operative and post-operative intravenous TXA and topical with control groups were 54, 6 and 16 respectively. Fail-safe-N for the combined intravenous and topical TXA with intravenous TXA alone was 13.
Conclusion:
Adequate evidence exists supporting the use of intravenous TXA in reducing the need for blood transfusion in THA. There is possible existence of small studies with null effects evaluating the use of TXA in THA.
Background:
Systematic reviews (SRs) can help decision makers interpret the deluge of published biomedical literature. However, a SR may be of limited use if the methods used to conduct the SR are flawed, and reporting of the SR is incomplete. To our knowledge, since 2004 there has been no cross-sectional study of the prevalence, focus, and completeness of reporting of SRs across different specialties. Therefore, the aim of our study was to investigate the epidemiological and reporting characteristics of a more recent cross-section of SRs.
Methods and findings:
We searched MEDLINE to identify potentially eligible SRs indexed during the month of February 2014. Citations were screened using prespecified eligibility criteria. Epidemiological and reporting characteristics of a random sample of 300 SRs were extracted by one reviewer, with a 10% sample extracted in duplicate. We compared characteristics of Cochrane versus non-Cochrane reviews, and the 2014 sample of SRs versus a 2004 sample of SRs. We identified 682 SRs, suggesting that more than 8,000 SRs are being indexed in MEDLINE annually, corresponding to a 3-fold increase over the last decade. The majority of SRs addressed a therapeutic question and were conducted by authors based in China, the UK, or the US; they included a median of 15 studies involving 2,072 participants. Meta-analysis was performed in 63% of SRs, mostly using standard pairwise methods. Study risk of bias/quality assessment was performed in 70% of SRs but was rarely incorporated into the analysis (16%). Few SRs (7%) searched sources of unpublished data, and the risk of publication bias was considered in less than half of SRs. Reporting quality was highly variable; at least a third of SRs did not report use of a SR protocol, eligibility criteria relating to publication status, years of coverage of the search, a full Boolean search logic for at least one database, methods for data extraction, methods for study risk of bias assessment, a primary outcome, an abstract conclusion that incorporated study limitations, or the funding source of the SR. Cochrane SRs, which accounted for 15% of the sample, had more complete reporting than all other types of SRs. Reporting has generally improved since 2004, but remains suboptimal for many characteristics.
Conclusions:
An increasing number of SRs are being published, and many are poorly conducted and reported. Strategies are needed to help reduce this avoidable waste in research.
Background
The efficacy of antidepressant medication has been shown empirically to be overestimated due to publication bias, but this has only been inferred statistically with regard to psychological treatment for depression. We assessed directly the extent of study publication bias in trials examining the efficacy of psychological treatment for depression.
Methods and Findings
We identified US National Institutes of Health grants awarded to fund randomized clinical trials comparing psychological treatment to control conditions or other treatments in patients diagnosed with major depressive disorder for the period 1972–2008, and we determined whether those grants led to publications. For studies that were not published, data were requested from investigators and included in the meta-analyses. Thirteen (23.6%) of the 55 funded grants that began trials did not result in publications, and two others never started. Among comparisons to control conditions, adding unpublished studies (Hedges’ g = 0.20; CI95% -0.11~0.51; k = 6) to published studies (g = 0.52; 0.37~0.68; k = 20) reduced the psychotherapy effect size point estimate (g = 0.39; 0.08~0.70) by 25%. Moreover, these findings may overestimate the "true" effect of psychological treatment for depression as outcome reporting bias could not be examined quantitatively.
Conclusion
The efficacy of psychological interventions for depression has been overestimated in the published literature, just as it has been for pharmacotherapy. Both are efficacious but not to the extent that the published literature would suggest. Funding agencies and journals should archive both original protocols and raw data from treatment trials to allow the detection and correction of outcome reporting bias. Clinicians, guidelines developers, and decision makers should be aware that the published literature overestimates the effects of the predominant treatments for depression.
Background
The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias and outcome reporting bias have been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making.
Methodology/Principal Findings
In this update, we review and summarise the evidence from cohort studies that have assessed study publication bias or outcome reporting bias in randomised controlled trials. Twenty studies were eligible of which four were newly identified in this update. Only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Fifteen of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40–62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies.
Conclusions
This update does not change the conclusions of the review in which 16 studies were included. Direct empirical evidence for the existence of study publication bias and outcome reporting bias is shown. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.
Medial tibial stress syndrome (MTSS) is a common exercise-induced leg injury among athletes and military personnel. Several treatment options have been described in the literature, but it remains unclear which treatment is most effective.
The objective of this systematic review was to assess the effectiveness of any intervention in the treatment of MTSS.
Published or non-published studies, reporting randomized or non-randomized controlled trials of any treatment in subjects with MTSS were eligible for inclusion. Treatments were assessed for effects on pain, time to recovery or global perceived effect.
Computerized bibliographic databases (MEDLINE, CENTRAL, EMBASE, CINAHL, PEDro and SPORTDiscus) and trial registries were searched for relevant reports, from their inception to 1 June 2012. Grey literature was searched for additional relevant reports.
The Cochrane Risk of Bias Tool was used to appraise study quality of randomized clinical trials (RCTs) whereas the Newcastle Ottawa Scale was used to appraise non-randomized trials. The 'levels of evidence', according to the Oxford Centre for Evidence-Based Medicine, addressed the impact of the assessed trials. Two reviewers independently performed the search for articles, study selection, data extraction and appraised methodological quality.
Eleven trials were included in this systematic review. All RCTs revealed a high risk of bias (Level 3 of evidence). Both non-randomized clinical trials were found to be of poor quality (Level 4 of evidence). RCTs, studying the effect of a lower leg brace versus no lower leg brace, and iontophoresis versus phonophoresis, were pooled using a fixed-effects model. No significant differences were found for lower leg braces (standardized mean difference [SMD] -0.06; 95 % CI -0.44 to 0.32, p = 0.76), or iontophoresis (SMD 0.09; 95 % CI -0.50 to 0.68, p = 0.76). Iontophoresis, phonophoresis, ice massage, ultrasound therapy, periosteal pecking and extracorporeal shockwave therapy (ESWT) could be effective in treating MTSS when compared with control (Level 3 to 4 of evidence). Low-energy laser treatment, stretching and strengthening exercises, sports compression stockings, lower leg braces and pulsed electromagnetic fields have not been proven to be effective in treating MTSS (level 3 of evidence).
None of the studies are sufficiently free from methodological bias to recommend any of the treatments investigated. Of those examined, ESWT appears to have the most promise.
Our objective was to develop an instrument to assess the methodological quality of systematic reviews, building upon previous tools, empirical evidence and expert consensus.
A 37-item assessment tool was formed by combining 1) the enhanced Overview Quality Assessment Questionnaire (OQAQ), 2) a checklist created by Sacks, and 3) three additional items recently judged to be of methodological importance. This tool was applied to 99 paper-based and 52 electronic systematic reviews. Exploratory factor analysis was used to identify underlying components. The results were considered by methodological experts using a nominal group technique aimed at item reduction and design of an assessment tool with face and content validity.
The factor analysis identified 11 components. From each component, one item was selected by the nominal group. The resulting instrument was judged to have face and content validity.
A measurement tool for the 'assessment of multiple systematic reviews' (AMSTAR) was developed. The tool consists of 11 items and has good face and content validity for measuring the methodological quality of systematic reviews. Additional studies are needed with a focus on the reproducibility and construct validity of AMSTAR, before strong recommendations can be made on its use.