[show abstract][hide abstract] ABSTRACT: It is important to understand the decision-making process, and the role of research evidence within it, across sectors other than health, as interventions delivered within these sectors may have substantial impacts on public health and health inequalities.
Systematic review of qualitative evidence. Twenty-eight databases covering a range of sectors were searched. Studies were eligible if they included local decision-makers in a policy field relevant to the social determinants of health (including housing, transport, urban planning and regeneration, crime, licensing or trading standards), were conducted in a high-income country, and reported primary qualitative data on perceptions of research evidence. Study quality was assessed and a thematic synthesis undertaken.
Sixteen studies were included, most using interview designs, and most focusing on planning or transport policy. Several factors are seen to influence decision-makers' views of evidence, including practical factors such as resources or organizational support; the credibility of the evidence; its relevance or applicability to practice; considerations of political support or feasibility; and legislative constraints. There are limited data on how evidence is used: it is sometimes used to not only support decision-making, but also to lend legitimacy to decisions that have already been made.
Although cultures of evidence in non-health sectors are similar to those in health in some ways, there are some key differences, particularly as regards the political context of decision-making. Intersectoral public health research could benefit from taking into account non-health decision makers' needs and preferences, particularly around relevance and political feasibility.
The European Journal of Public Health 03/2014; · 2.52 Impact Factor
[show abstract][hide abstract] ABSTRACT: We report the main results, among adults, of a cluster-randomised-trial of Well London, a community-engagement programme promoting healthy eating, physical activity and mental well-being in deprived neighbourhoods. The hypothesis was that benefits would be neighbourhood-wide, and not restricted to intervention participants. The trial was part of a multicomponent process/outcome evaluation which included non-experimental components (self-reported behaviour change amongst participants, case studies and evaluations of individual projects) which suggested health, well-being and social benefits to participants.
Twenty matched pairs of neighbourhoods in London were randomised to intervention/control condition. Primary outcomes (five portions fruit/vegetables/day; 5×30 m of moderate intensity physical activity/week, abnormal General Health Questionnaire (GHQ)-12 score and Warwick-Edinburgh Mental Well-being Scale (WEMWBS) score) were measured by postintervention questionnaire survey, among 3986 adults in a random sample of households across neighbourhoods.
There was no evidence of impact on primary outcomes: healthy eating (relative risk [RR] 1.04, 95% CI 0.93 to 1.17); physical activity (RR:1.01, 95% CI 0.88 to 1.16); abnormal GHQ12 (RR:1.15, 95% CI 0.84 to 1.61); WEMWBS (mean difference [MD]: -1.52, 95% CI -3.93 to 0.88). There was evidence of impact on some secondary outcomes: reducing unhealthy eating-score (MD: -0.14, 95% CI -0.02 to 0.27) and increased perception that people in the neighbourhood pulled together (RR: 1.92, 95% CI 1.12 to 3.29).
The trial findings do not provide evidence supporting the conclusion of non-experimental components of the evaluation that intervention improved health behaviours, well-being and social outcomes. Low participation rates and population churn likely compromised any impact of the intervention. Imprecise estimation of outcomes and sampling bias may also have influenced findings. There is a need for greater investment in refining such programmes before implementation; new methods to understand, longitudinally different pathways residents take through such interventions and their outcomes, and new theories of change that apply to each pathway.
Journal of epidemiology and community health 01/2014; · 3.04 Impact Factor
[show abstract][hide abstract] ABSTRACT: Well London is a multicomponent community engagement and coproduction programme designed to improve the health of Londoners living in socioeconomically deprived neighbourhoods. To evaluate outcomes of the Well London interventions, a cluster randomised trial (CRT) was conducted that included a longitudinal qualitative component, which is reported here. The aim is to explore in depth the nature of the benefits to residents and the processes by which these were achieved.
The 1-year longitudinal qualitative study was nested within the CRT. Purposive sampling was used to select three intervention neighbourhoods in London and 61 individuals within these neighbourhoods. The interventions comprised activities focused on: healthy eating, physical exercise and mental health and well-being. Interviews were conducted at the inception and following completion of the Well London interventions to establish both if and how they had participated. Transcripts of the interviews were coded and analysed using Nvivo.
Positive benefits relating to the formal outcomes of the CRT were reported, but only among those who participated in project activities. The extent of benefits experienced was influenced by factors relating to the physical and social characteristics of each neighbourhood. The highest levels of change occurred in the presence of: (1) social cohesion, not only pre-existing but also as facilitated by Well London activities; (2) personal and collective agency; (3) involvement and support of external organisations. Where the physical and social environment remained unchanged, there was less participation and fewer benefits.
These findings show interaction between participation, well-being and agency, social interactions and cohesion and that this modulated any benefits described. Pathways to change were thus complex and variable, but personal well-being and local social cohesion emerged as important mediators of change.
BMJ Open 01/2014; 4(4):e003596. · 1.58 Impact Factor
[show abstract][hide abstract] ABSTRACT: To assess the utility of an acronym, place of residence, race/ethnicity/culture/language, occupation, gender/sex, religion, education, socioeconomic status, and social capital ("PROGRESS"), in identifying factors that stratify health opportunities and outcomes. We explored the value of PROGRESS as an equity lens to assess effects of interventions on health equity.
We assessed the utility of PROGRESS by using it in 11 systematic reviews and methodological studies published between 2008 and 2013. To develop the justification for each of the PROGRESS elements, we consulted experts to identify examples of unfair differences in disease burden and an intervention that can effectively address these health inequities.
Each PROGRESS factor can be justified on the basis of unfair differences in disease burden and the potential for interventions to reduce these differential effects. We have not provided a rationale for why the difference exists but have attempted to explain why these differences may contribute to disadvantage and argue for their consideration in new evaluations, systematic reviews, and intervention implementation.
The acronym PROGRESS is a framework and aide-memoire that is useful in ensuring that an equity lens is applied in the conduct, reporting, and use of research.
Journal of clinical epidemiology 11/2013; · 2.96 Impact Factor
[show abstract][hide abstract] ABSTRACT: Knowledge translation strategies are an approach to increase the use of evidence within policy and practice decision-making contexts. In clinical and health service contexts, knowledge translation strategies have focused on individual behavior change, however the multi-system context of public health requires a multi-level, multi-strategy approach. This paper describes the design of and implementation plan for a knowledge translation intervention for public health decision making in local government.
Four preliminary research studies contributed findings to the design of the intervention: a systematic review of knowledge translation intervention effectiveness research, a scoping study of knowledge translation perspectives and relevant theory literature, a survey of the local government public health workforce, and a study of the use of evidence-informed decision-making for public health in local government. A logic model was then developed to represent the putative pathways between intervention inputs, processes, and outcomes operating between individual-, organizational-, and system-level strategies. This formed the basis of the intervention plan.
The systematic and scoping reviews identified that effective and promising strategies to increase access to research evidence require an integrated intervention of skill development, access to a knowledge broker, resources and tools for evidence-informed decision making, and networking for information sharing. Interviews and survey analysis suggested that the intervention needs to operate at individual and organizational levels, comprising workforce development, access to evidence, and regular contact with a knowledge broker to increase access to intervention evidence; develop skills in appraisal and integration of evidence; strengthen networks; and explore organizational factors to build organizational cultures receptive to embedding evidence in practice. The logic model incorporated these inputs and strategies with a set of outcomes to measure the intervention's effectiveness based on the theoretical frameworks, evaluation studies, and decision-maker experiences.
Documenting the design of and implementation plan for this knowledge translation intervention provides a transparent, theoretical, and practical approach to a complex intervention. It provides significant insights into how practitioners might engage with evidence in public health decision making. While this intervention model was designed for the local government context, it is likely to be applicable and generalizable across sectors and settings.Trial registration: Australia New Zealand Clinical Trials Register ACTRN12609000953235.
[show abstract][hide abstract] ABSTRACT: Complex interventions present unique challenges for systematic reviews. Current debates tend to center around describing complexity, rather than providing guidance on what to do about it. At a series of meetings during 2009-2012, we met to review the challenges and practical steps reviewer could take to incorporate a complexity perspective into systematic reviews. Based on this, we outline a pragmatic approach to dealing with complexity, beginning, as for any review, with clearly defining the research question(s). We argue that reviews of complex interventions can themselves be simple or complex, depending on the question to be answered. In systematic reviews and evaluations of complex interventions, it will be helpful to start by identifying the sources of complexity, then mapping aspects of complexity in the intervention onto the appropriate sources of evidence (such as specific types of quantitative or qualitative study). Although we focus on systematic reviews, the general approach is also applicable to primary research that is aimed at evaluating complex interventions. Although the examples are drawn from health care, the approach may also be applied to other sectors (e.g., social policy or international development). We end by concluding that systematic reviews should follow the principle of Occam's razor: explanations should be as complex as they need to be and no more.
Journal of clinical epidemiology 08/2013; · 2.96 Impact Factor
[show abstract][hide abstract] ABSTRACT: Although there is increasing interest in the evaluation of complex interventions, there is little guidance on how evidence from complex interventions may be reviewed and synthesized, and the relevance of the plethora of evidence synthesis methods to complexity is unclear. This article aims to explore how different meta-analytical approaches can be used to examine aspects of complexity; describe the contribution of various narrative, tabular, and graphical approaches to synthesis; and give an overview of the potential choice of selected qualitative and mixed-method evidence synthesis approaches.
The methodological discussions presented here build on a 2-day workshop held in Montebello, Canada, in January 2012, involving methodological experts from the Campbell and Cochrane Collaborations and from other international review centers (Anderson L, Petticrew M, Chandler J, et al. Introduction: systematic reviews of complex interventions. In press). These systematic review methodologists discussed the broad range of existing methods and considered the relevance of these methods to reviews of complex interventions.
The evidence from primary studies of complex interventions may be qualitative or quantitative. There is a wide range of methodological options for reviewing and presenting this evidence. Specific contributions of statistical approaches include the use of meta-analysis, meta-regression, and Bayesian methods, whereas narrative summary approaches provide valuable precursors or alternatives to these. Qualitative and mixed-method approaches include thematic synthesis, framework synthesis, and realist synthesis. A suitable combination of these approaches allows synthesis of evidence for understanding complex interventions.
Reviewers need to consider which aspects of complex interventions should be a focus of their review and what types of quantitative and/or qualitative studies they will be including, and this will inform their choice of review methods. These may range from standard meta-analysis through to more complex mixed-method synthesis and synthesis approaches that incorporate theory and/or user's perspectives.
Journal of clinical epidemiology 08/2013; · 2.96 Impact Factor
[show abstract][hide abstract] ABSTRACT: This article outlines a research and development agenda for systematic reviews that ask complex questions about interventions varying in degree and type of complexity.
Consensus development by key authors of articles on methodological challenges in systematic reviews of complex interventions, based on a 2-day workshop in Montebello, Canada, January 2012.
There is an urgent need for a more precise and consistently applied lexicon and language to disaggregate several conceptually distinct dimensions of "complexity." Selected current evidence synthesis methods have potential application in reviews where complexity is important. There is a lack of evaluation of methods to better understand the nature of complex interventions and the optimal processes of synthesizing and interpreting evidence from these systematic reviews. Gaps in methods, knowledge, and know-how exist, and there is a need for additional guidance.
Understanding how complexity can impact on findings of systematic reviews is critical. Experience in applying methods that have been developed to facilitate this understanding is limited, and the degree to which these approaches improve the systematic review process or transparency is only partially understood. Future research should concentrate on the impact of complexity on the systematic review process and findings and on further methodological development.
Journal of clinical epidemiology 08/2013; · 2.96 Impact Factor
[show abstract][hide abstract] ABSTRACT: The Public Health Responsibility Deal (RD) in England was launched in 2011 as a public-private partnership which aims to 'tap into the potential for businesses and other influential organisations to make a significant contribution to improving public health by helping us to create this environment'. It has come under criticism from public health advocates and others, who have suggested that it will be ineffective or perhaps even harmful. Like many public health policies, there have also been demands to know whether it 'works'.
We conducted a scoping review and used this, supplemented with interviews with stakeholders, to develop a detailed logic model of the RD (presented here) to help understand its likely outcomes and the pathways by which these may be achieved as a basis for planning an evaluation.
Evaluations of complex interventions require not just assessment of effects (including outcomes), but also a clear conceptualization of the intervention and its processes. The way the RD and the pledges made by participant organizations has been presented makes it difficult at this stage to evaluate whether the RD 'works' in terms of improving health. Instead, any evaluation needs to put together a jigsaw of evidence about processes, mechanisms and potential future health and non-health impacts, in part using the current scientific evidence. This task is ongoing.
Journal of Public Health 07/2013; · 2.06 Impact Factor
[show abstract][hide abstract] ABSTRACT: To analyse available review-level evidence on the effectiveness of population-level interventions in non-clinical settings to reduce alcohol consumption or related health or social harm.
Health, social policy and specialist review databases were searched between 2002 and 2012 for systematic reviews of the effectiveness of population-level alcohol interventions on consumption or alcohol-related health or social outcomes. Data were extracted on review research aim, inclusion criteria, outcome indicators, results, conclusions and limitations. Reviews were quality-assessed using AMSTAR criteria. A narrative synthesis was conducted overall and by policy area.
Fifty-two reviews were included from ten policy areas. There is good evidence for policies and interventions to limit alcohol sales availability, to reduce drink-driving, to increase alcohol price or taxation. There is mixed evidence for family- and community-level interventions, school-based interventions, and interventions in the alcohol server setting and the mass media. There is weak evidence for workplace interventions and for interventions targeting illicit alcohol sales. There is evidence of the ineffectiveness of interventions in higher education settings.
There is a pattern of support from the evidence base for regulatory or statutory enforcement interventions over local non-regulatory approaches targeting specific population groups.
[show abstract][hide abstract] ABSTRACT: At the Rio Summit in 2011 on Social Determinants of Health, the global community recognized a pressing need to take action on reducing health inequities. This requires an improved evidence base on the effects of national and international policies on health inequities. Although systematic reviews are recognized as an important source for evidence-informed policy, they have been criticized for failing to assess effects on health equity.
This article summarizes guidance on both conducting systematic reviews with a focus on health equity and on methods to translate their findings to different audiences. This guidance was developed based on a series of methodology meetings, previous guidance, a recently developed reporting guideline for equity-focused systematic reviews (PRISMA-Equity 2012) and a systematic review of methods to assess health equity in systematic reviews.
We make ten recommendations for conducting equity-focused systematic reviews; and five considerations for knowledge translation. Illustrative examples of equity-focused reviews are provided where these methods have been used.
Implementation of the recommendations in this article is one step toward monitoring the impact of national and international policies and programs on health equity, as recommended by the 2011 World Conference on Social Determinants of Health.
[show abstract][hide abstract] ABSTRACT: INTRODUCTION: Process evaluations of environmental public health interventions tend not to consider issues of spatial equity in programme delivery. However, an intervention is unlikely to be effective if it is not accessible to those in need. Methods are required to enable these considerations to be integrated into evaluations. Using the Healthy Towns programme in England, we demonstrate the potential of spatial equity analysis in the evaluation of environmental interventions for diet and physical activity, examining whether the programme was delivered to those in greatest need. METHODS: Locations of new physical infrastructure, such as cycle lanes, gyms and allotments, were mapped using a geographic information system. A targeting ratio was computed to indicate how well-located the infrastructure was in relation to those at whom it was specifically aimed, as detailed in the relevant project documentation, as well as to generally disadvantaged populations defined in terms of UK Census data on deprivation, age and ethnicity. Differences in targeting were examined using Kruskal-Wallis and t-tests. RESULTS: The 183 separate intervention components identified were generally well located, with estimated targeting ratios above unity for all population groups of need, except for black and ethnic minorities and children aged 5--19 years. There was no evidence that clustering of population groups influenced targeting, or that trade-offs existed when components were specifically targeted at more than one group. CONCLUSIONS: The analysis of spatial equity is a valuable initial stage in assessing the provision of environmental interventions. The Healthy Towns programme can be described as well targeted in that interventions were for the most part located near populations of need.
International Journal for Equity in Health 06/2013; 12(1):43. · 1.71 Impact Factor
[show abstract][hide abstract] ABSTRACT: BACKGROUND: We used the introduction of free bus travel for young people in London in 2005 as a natural experiment with which to assess its effects on active travel, car use, road traffic injuries, assaults, and on one measure of social inclusion, total number of trips made. METHODS: A controlled before-after analysis was conducted. We estimated trips by mode and distances travelled in the preintroduction and postintroduction periods using data from London Travel Demand Surveys. We estimated rates of road traffic injury and assault in each period using STATS19 data and Hospital Episode Statistics, respectively. We estimated the ratio of change in the target age group (12-17 years) to the change in adults (ages 25-59 years), with 95% CIs. RESULTS: The proportion of short trips travelled by bus by young people increased postintroduction. There was no evidence for an increase in the total number of bus trips or distance travelled by bus by young people attributable to the intervention. The proportion of short trips by walking decreased, but there was no evidence for any change to total distance walked. Car trips declined in both age groups, although distance travelled by car decreased more in young people. Road casualty rates declined, but the pre-post ratio of change was greater in young people than adults (ratio of ratios 0.84; 95% CI 0.82 to 0.87). Assaults increased and the ratio of change was greater in young people (1.20; 1.13 to 1.27). The frequency of all trips by young people was unchanged, both in absolute terms and relative to adults. CONCLUSIONS: The introduction of free bus travel for young people had little impact on active travel overall and shifted some travel from car to buses that could help broader environmental objectives.
Journal of epidemiology and community health 06/2013; · 3.04 Impact Factor
[show abstract][hide abstract] ABSTRACT: BACKGROUND: There is continuing interest among practitioners, policymakers and researchers in the evaluation of complex interventions stemming from the need to further develop the evidence base on the effectiveness of healthcare and public health interventions, and an awareness that evaluation becomes more challenging if interventions are complex.We undertook an analysis of published journal articles in order to identify aspects of complexity described by writers, the fields in which complex interventions are being evaluated and the challenges experienced in design, implementation and evaluation. This paper outlines the findings of this documentary analysis. METHODS: The PubMed electronic database was searched for the ten year period, January 2002 to December 2011, using the terms "complex intervention*" in the title and/or abstract of a paper. We extracted text from papers to a table and carried out a thematic analysis to identify authors' descriptions of challenges faced in developing, implementing and evaluating complex interventions. RESULTS: The search resulted in a sample of 221 papers of which full text of 216 was obtained and 207 were included in the analysis. The 207 papers broadly cover clinical, public health and methodological topics. Challenges described included the content and standardisation of interventions, the impact of the people involved (staff and patients), the organisational context of implementation, the development of outcome measures, and evaluation. CONCLUSIONS: Our analysis of these papers suggests that more detailed reporting of information on outcomes, context and intervention is required for complex interventions. Future revisions to reporting guidelines for both primary and secondary research may need to take aspects of complexity into account to enhance their value to both researchers and users of research.
BMC Public Health 06/2013; 13(1):568. · 2.08 Impact Factor