Study Designs for Effectiveness and Translation Research: Identifying Trade-Offs

Guide to Community Preventive Services, National Center for Health Marketing, Centers for Disease Control and Prevention, Atlanta, Georgia 30333, USA.
American Journal of Preventive Medicine (Impact Factor: 4.53). 09/2007; 33(2):139-154. DOI: 10.1016/j.amepre.2007.04.005
Source: PubMed


Practitioners and policymakers need credible evidence of effectiveness to justify allocating resources to complex, expensive health programs. Investigators, however, face challenges in designing sound effectiveness and translation research with relevance for "real-world" settings.
Research experts and federal and foundation funders (n= approximately 120) prepared for and participated in a symposium, held May 4-5, 2004, to weigh the strengths, limitations, and trade-offs between alternate designs for studying the effectiveness and translation of complex, multilevel health interventions.
Symposium attendees acknowledged that research phases (hypothesis generating, efficacy, effectiveness, translation) are iterative and cyclical, not linear, since research in advanced phases may reveal unanswered questions in earlier phases. Research questions thus always need to drive the choice of study design. When randomization and experimental control are feasible, participants noted that the randomized controlled trial with individual random assignment remains the gold standard for safeguarding internal validity. Attendees highlighted trade-offs of randomized controlled trial variants, quasi-experimental designs, and natural experiments for use when randomization or experimental control or both are impossible or inadequately address external validity. Participants discussed enhancements to all designs to increase confidence in causal inference while accommodating greater external validity. Since no single study can establish causality, participants encouraged replication of studies and triangulation using different study designs. Participants also recommended participatory research approaches for building population relevance, acceptability, and usefulness.
Consideration of the study design choices, trade-offs, and enhancements discussed here can guide the design, funding, completion, and publication of appropriate policy- and practice-oriented effectiveness and translational research for complex, multilevel health interventions.


Available from: Lawrence W Green, Jul 25, 2014
  • Source
    • "The call for more practice-based evidence is a challenge to policymakers, practitioners, researchers, and funding agencies [8]. However, the gap between evidence and practice has still not been bridged, although some important progress has been made [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Background: The development of knowledge for health promotion requires an effective mechanism for collaboration between academics, practitioners, and policymakers. The challenge is better to understand the dynamic and ever-changing context of the researcher–practitioner–policymaker–community relationship. Aims: The aims were to explore the factors that foster Academic Practice Policy (APP) partnerships, and to systematically and transparently to review three cases. Methods: Three partnerships were included: Power and Commitment–Alcohol and Drug Prevention by Non-Governmental Organizations in Sweden; Healthy City–Social Inclusion, Urban Governance, and Sustainable Welfare Development; and Empowering Families with Teenagers–Ideals and Reality in Karlskoga and Degerfors. The analysis includes searching for evidence for three hypotheses concerning contextual factors in multi-stakeholder collaboration, and the cumulative effects of partnership synergy. Results: APP partnerships emerge during different phases of research and development. Contextual factors are important; researchers need to be trusted by practitioners and politicians. During planning, it is important to involve the relevant partners. During the implementation phase, time is important. During data collection and capacity building, it is important to have shared objectives for and dialogues about research. Finally, dissemination needs to be integrated into any partnership. The links between process and outcomes in participatory research (PR) can be described by the theory of partnership synergy, which includes consideration of how PR can ensure culturally and logistically appropriate research, enhance recruitment capacity, and generate professional capacity and competence in stakeholder groups. Moreover, there are PR synergies over time. Conclusions: The fundamentals of a genuine partnership are communication, collaboration, shared visions, and willingness of all stakeholders to learn from one another.
    Scandinavian Journal of Public Health 11/2014; 42(15 Suppl):88-95. DOI:10.1177/1403494814556926 · 1.83 Impact Factor
  • Source
    • "We used a one-arm interrupted time series design, a quasiexperimental research design [10] [11] [12] in which a series of pre-intervention assessments were conducted (18, 12, and 6 weeks prior to intervention and baseline), followed by a group, diabetes educator/dietitian-facilitated intervention and a series of post-intervention assessments (6, 12, and 18 weeks after intervention) (Box 1). Pre-and postintervention assessment visits were conducted at Meharry Medical College based on participants' schedules and intervention study sessions were conducted in the evening hours at a collaborating YMCA facility. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Objective: To assess the efficacy and acceptability of a group medical nutritional therapy (MNT) intervention, using motivational interviewing (MI). RESEARCH DESIGN & METHOD: African American (AA) women with type 2 diabetes (T2D) participated in five, certified diabetes educator/dietitian-facilitated intervention sessions targeting carbohydrate, fat, and fruit/vegetable intake and management. Motivation-based activities centered on exploration of dietary ambivalence and the relationships between diet and personal strengths. Repeated pre- and post-intervention, psychosocial, dietary self-care, and clinical outcomes were collected and analyzed using generalized least squares regression. An acceptability assessment was administered after intervention. Results: Participants (n = 24) were mostly of middle age (mean age 50.8 ± 6.3) with an average BMI of 39 ± 6.5. Compared to a gradual pre-intervention loss of HbA1c control and confidence in choosing restaurant foods, a significant post-intervention improvement in HbA1c (P = 0.03) and a near significant (P = 0.06) increase in confidence in choosing restaurant foods were observed with both returning to pre-intervention levels. 100% reported that they would recommend the study to other AA women with type 2 diabetes. Conclusion: The results support the potential efficacy of a group MNT/MI intervention in improving glycemic control and dietary self-care-related confidence in overweight/obese AA women with type 2 diabetes.
    Journal of obesity 08/2014; 2014:345941. DOI:10.1155/2014/345941
  • Source
    • "However, clinicians, funding bodies, academics and publishers may be beginning to detect chinks in the armour of RCTs, and in response to the need to economically justify research agendas and support evidence based practice, are starting to recognise the need to also consider studies focused towards relevancy and applicability rather than overwhelmingly prioritising analyses of cause and effect under highly controlled, artificial conditions [25]. This climate change is promoting, in line with Lewin’s [10] theory, the unfreezing of accepted norms and acknowledgement that a variety of study designs focused towards the patient, the question(s) under investigation, and applicability and relevancy of the research to routine clinical practice; rather than continuing to promote highly reductionist or post-positivist focused research as the standard by which all others are judged [26]. One such illustration is an extension to the Consort Guidelines for reporting pragmatic clinical trials which provide a clear mandate for clinicians and researchers to justify pragmatically focused trials as both meaningful and relevant to patients, clinicians, and the broader healthcare community [27]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Background Randomised controlled trials may be of limited use to evaluate the multidisciplinary and multimodal interventions required to effectively treat complex patients in routine clinical practice; pragmatic action research approaches may provide a suitable alternative. Methods A multiphase, pragmatic, action research based approach was developed to identify and overcome barriers to nutritional care in patients admitted to a metropolitan hospital hip-fracture unit. Results Four sequential action research cycles built upon baseline data including 614 acute hip-fracture inpatients and 30 purposefully sampled clinicians. Reports from Phase I identified barriers to nutrition screening and assessment. Phase II reported post-fracture protein-energy intakes and intake barriers. Phase III built on earlier results; an explanatory mixed-methods study expanded and explored additional barriers and facilitators to nutritional care. Subsequent changes to routine clinical practice were developed and implemented by the treating team between Phase III and IV. These were implemented as a new multidisciplinary, multimodal nutritional model of care. A quasi-experimental controlled, ‘before-and-after’ study was then used to compare the new model of care with an individualised nutritional care model. Engagement of the multidisciplinary team in a multiphase, pragmatic action research intervention doubled energy and protein intakes, tripled return home discharge rates, and effected a 75% reduction in nutritional deterioration during admission in a reflective cohort of hip-fracture inpatients. Conclusions This approach allowed research to be conducted as part of routine clinical practice, captured a more representative patient cohort than previously reported studies, and facilitated exploration of barriers and engagement of the multidisciplinary healthcare workers to identify and implement practical solutions. This study demonstrates substantially different findings to those previously reported, and is the first to demonstrate that multidisciplinary, multimodal nutrition care reduces intake barriers, delivers a higher proportional increase in protein and energy intake compared with baseline than other published intervention studies, and improves patient outcomes when compared with individualised nutrition care. The findings are considered highly relevant to clinical practice and have high translation validity. The authors strongly encourage the development of similar study designs to investigate complex health problems in elderly, multi-morbid patient populations as a way to evaluate and change clinical practice.
    BMC Medical Research Methodology 08/2014; 14(1):98. DOI:10.1186/1471-2288-14-98 · 2.27 Impact Factor
Show more