Article

Protocol - realist and meta-narrative evidence synthesis: Evolving Standards (RAMESES)

Healthcare Innovation and Policy Unit, Centre for Primary Care and Public Health, Blizard Institute, Barts and The London School of Medicine and Dentistry, London E1 2AB, UK.
BMC Medical Research Methodology (Impact Factor: 2.17). 08/2011; 11:115. DOI: 10.1186/1471-2288-11-115
Source: PubMed

ABSTRACT There is growing interest in theory-driven, qualitative and mixed-method approaches to systematic review as an alternative to (or to extend and supplement) conventional Cochrane-style reviews. These approaches offer the potential to expand the knowledge base in policy-relevant areas - for example by explaining the success, failure or mixed fortunes of complex interventions. However, the quality of such reviews can be difficult to assess. This study aims to produce methodological guidance, publication standards and training resources for those seeking to use the realist and/or meta-narrative approach to systematic review.
We will: [a] collate and summarise existing literature on the principles of good practice in realist and meta-narrative systematic review; [b] consider the extent to which these principles have been followed by published and in-progress reviews, thereby identifying how rigour may be lost and how existing methods could be improved; [c] using an online Delphi method with an interdisciplinary panel of experts from academia and policy, produce a draft set of methodological steps and publication standards; [d] produce training materials with learning outcomes linked to these steps; [e] pilot these standards and training materials prospectively on real reviews-in-progress, capturing methodological and other challenges as they arise; [f] synthesise expert input, evidence review and real-time problem analysis into more definitive guidance and standards; [g] disseminate outputs to audiences in academia and policy. The outputs of the study will be threefold:1. Quality standards and methodological guidance for realist and meta-narrative reviews for use by researchers, research sponsors, students and supervisors2. A 'RAMESES' (Realist and Meta-review Evidence Synthesis: Evolving Standards) statement (comparable to CONSORT or PRISMA) of publication standards for such reviews, published in an open-access academic journal.3. A training module for researchers, including learning outcomes, outline course materials and assessment criteria.
Realist and meta-narrative review are relatively new approaches to systematic review whose overall place in the secondary research toolkit is not yet fully established. As with all secondary research methods, guidance on quality assurance and uniform reporting is an important step towards improving quality and consistency of studies.

Download full-text

Full-text

Available from: Geoff Wong, Jun 27, 2015
1 Follower
 · 
226 Views
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: In this article we develop a typology of review types and provide a descriptive insight into the most common reviews found in top IS journals. Our assessment reveals that the number of IS reviews has increased over the years. The majority of the 139 reviews are theoretical in nature, followed by narrative reviews, meta-analyses, descriptive reviews, hybrid reviews, critical reviews, and scoping reviews. Considering the calls for IS research to develop a cumulative tradition, we hope more review articles will be published in the future and encourage researchers who start a review to use our typology to position their contribution.
    Information & Management 02/2015; 52(2):183-199. DOI:10.1016/j.im.2014.08.008 · 1.79 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Context: An evidence base that addresses issues of complexity and context is urgently needed for large-system transformation (LST) and health care reform. Fundamental conceptual and methodological challenges also must be addressed. The Saskatchewan Ministry of Health in Canada requested a six-month synthesis project to guide four major policy development and strategy initiatives focused on patient- and family-centered care, primary health care renewal, quality improvement, and surgical wait lists. The aims of the review were to analyze examples of successful and less successful transformation initiatives, to synthesize knowledge of the underlying mechanisms, to clarify the role of government, and to outline options for evaluation. Methods: We used realist review, whose working assumption is that a particular intervention triggers particular mechanisms of change. Mechanisms may be more or less effective in producing their intended outcomes, depending on their interaction with various contextual factors. We explain the variations in outcome as the interplay between context and mechanisms. We nested this analytic approach in a macro framing of complex adaptive systems (CAS). Findings: Our rapid realist review identified five "simple rules" of LST that were likely to enhance the success of the target initiatives: (1) blend designated leadership with distributed leadership; (2) establish feedback loops; (3) attend to history; (4) engage physicians; and (5) include patients and families. These principles play out differently in different contexts affecting human behavior (and thereby contributing to change) through a wide range of different mechanisms. Conclusions: Realist review methodology can be applied in combination with a complex system lens on published literature to produce a knowledge synthesis that informs a prospective change effort in large-system transformation. A collaborative process engaging both research producers and research users contributes to local applications of universal principles and mid-range theories, as well as to a more robust knowledge base for applied research. We conclude with suggestions for the future development of synthesis and evaluation methods.
    Milbank Quarterly 09/2012; 90(3):421-56. DOI:10.1111/j.1468-0009.2012.00670.x · 5.06 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: To evaluate current risk models and scores for type 2 diabetes and inform selection and implementation of these in practice. Systematic review using standard (quantitative) and realist (mainly qualitative) methodology. Inclusion criteria Papers in any language describing the development or external validation, or both, of models and scores to predict the risk of an adult developing type 2 diabetes. Medline, PreMedline, Embase, and Cochrane databases were searched. Included studies were citation tracked in Google Scholar to identify follow-on studies of usability or impact. Data were extracted on statistical properties of models, details of internal or external validation, and use of risk scores beyond the studies that developed them. Quantitative data were tabulated to compare model components and statistical properties. Qualitative data were analysed thematically to identify mechanisms by which use of the risk model or score might improve patient outcomes. 8864 titles were scanned, 115 full text papers considered, and 43 papers included in the final sample. These described the prospective development or validation, or both, of 145 risk prediction models and scores, 94 of which were studied in detail here. They had been tested on 6.88 million participants followed for up to 28 years. Heterogeneity of primary studies precluded meta-analysis. Some but not all risk models or scores had robust statistical properties (for example, good discrimination and calibration) and had been externally validated on a different population. Genetic markers added nothing to models over clinical and sociodemographic factors. Most authors described their score as "simple" or "easily implemented," although few were specific about the intended users and under what circumstances. Ten mechanisms were identified by which measuring diabetes risk might improve outcomes. Follow-on studies that applied a risk score as part of an intervention aimed at reducing actual risk in people were sparse. Much work has been done to develop diabetes risk models and scores, but most are rarely used because they require tests not routinely available or they were developed without a specific user or clear use in mind. Encouragingly, recent research has begun to tackle usability and the impact of diabetes risk scores. Two promising areas for further research are interventions that prompt lay people to check their own diabetes risk and use of risk scores on population datasets to identify high risk "hotspots" for targeted public health interventions.
    BMJ (online) 11/2011; 343:d7163. DOI:10.1136/bmj.d7163 · 16.38 Impact Factor