Applying what works: a systematic search of the transfer and implementation of promising Indigenous Australian health services and programs.
ABSTRACT BACKGROUND: The transfer and implementation of acceptable and effective health services, programs and innovations across settings provides an important and potentially cost-effective strategy for reducing Indigenous Australians' high burden of disease. This study reports a systematic review of Indigenous health services, programs and innovations to examine the extent to which studies considered processes of transfer and implementation within and across Indigenous communities and healthcare settings. METHODS: Medline, Informit, Infotrac, Blackwells Publishing, Proquest, Taylor and Francis, JStor, and the Indigenous HealthInfoNet were searched using terms: Aborigin* OR Indigen* OR Torres AND health AND service OR program* OR intervention AND Australia to locate publications from 1992-2011. The reference lists of 19 reviews were also checked. Data from peer reviewed journals, reports, and websites were included. The 95% confidence intervals (95% CI) for proportions that referred to and focussed on transfer were calculated as exact binomial confidence intervals. Test comparisons between proportions were calculated using Fisher's exact test with an alpha level of 5%. RESULTS: Of 1311 publications identified, 119 (9.1%; 95% CI: 7.6 % - 10.8%) referred to the transfer and implementation of Indigenous Australian health services or programs, but only 21 studies (1.6%; 95% CI: 1.0% - 2.4%) actually focused on transfer and implementation. Of the 119 transfer studies, 37 (31.1%; 95% CI: 22.9 - 40.2%) evaluated the impact of a service or program, 28 (23.5%; 95% CI: 16.2% - 32.2%) reported only process measures and 54 were descriptive. Of the 37 impact evaluation studies, 28 (75.7%; 95% CI: 58.8% - 88.2%) appeared in peer reviewed journals but none included experimental designs. CONCLUSION: While services and programs are being transferred and implemented, few studies focus on the process by which this occurred or the effectiveness of the service or program in the new setting. Findings highlight a need for partnerships between researchers and health services to evaluate the transfer and implementation of Indigenous health services and programs using rigorous designs, and publish such efforts in peer-reviewed journals as a quality assurance mechanism.
- SourceAvailable from: Christopher M Doran[Show abstract] [Hide abstract]
ABSTRACT: BACKGROUND: Indigenous peoples of Australia, Canada, United States and New Zealand experience disproportionately high rates of suicide. As such, the methodological quality of evaluations of suicide prevention interventions targeting these Indigenous populations should be rigorously examined, in order to determine the extent to which they are effective for reducing rates of Indigenous suicide and suicidal behaviours. This systematic review aims to: 1) identify published evaluations of suicide prevention interventions targeting Indigenous peoples in Australia, Canada, United States and New Zealand; 2) critique their methodological quality; and 3) describe their main characteristics. METHODS: A systematic search of 17 electronic databases and 13 websites for the period 1981--2012 (inclusive) was undertaken. The reference lists of reviews of suicide prevention interventions were hand-searched for additional relevant studies not identified by the electronic and web search. The methodological quality of evaluations of suicide prevention interventions was assessed using a standardised assessment tool. RESULTS: Nine evaluations of suicide prevention interventions were identified: five targeting Native Americans; three targeting Aboriginal Australians; and one First Nation Canadians. The main intervention strategies employed included: Community Prevention, Gatekeeper Training, and Education. Only three of the nine evaluations measured changes in rates of suicide or suicidal behaviour, all of which reported significant improvements. The methodological quality of evaluations was variable. Particular problems included weak study designs, reliance on self-report measures, highly variable consent and follow-up rates, and the absence of economic or cost analyses. CONCLUSIONS: There is an urgent need for an increase in the number of evaluations of preventive interventions targeting reductions in Indigenous suicide using methodologically rigorous study designs across geographically and culturally diverse Indigenous populations. Combining and tailoring best evidence and culturally-specific individual strategies into one coherent suicide prevention program for delivery to whole Indigenous communities and/or population groups at high risk of suicide offers considerable promise.BMC Public Health 05/2013; 13(1):463. · 2.08 Impact Factor
- [Show abstract] [Hide abstract]
ABSTRACT: The concepts and standard practices of implementation, largely originating in developed countries, cannot necessarily be simply transferred into diverse cultural contexts. There has been relative inattention in the implementation science literature paid to the implementation of interventions targeting minority Indigenous populations within developed countries. This suggests that the implementation literature may be bypassing population groups within developed countries who suffer some of the greatest disadvantage. Within the context of Aboriginal Australian health improvement, this study considers the impact of political and cultural issues by examining the transfer and implementation of the Family Wellbeing program across 56 places over a 20-year period. A theoretical model of program transfer was developed using constructivist-grounded theory methods. Data were generated by conducting in-depth interviews with 18 Aboriginal and non-Aboriginal research respondents who had been active in transferring the program. Data were categorised into higher order abstract concepts and the core impetus for and process of program transfer were identified. Organizations transferred the program by using it as a vehicle for supporting inside-out empowerment. The impetus to support inside-out empowerment referred to support for Aboriginal people's participation, responsibility for and control of their own affairs, and the associated ripple effects to family members, organizations, communities, and ultimately reconciliation with Australian society at large. Program transfer occurred through a multi-levelled process of embracing relatedness which included relatedness with self, others, and structural conditions; all three were necessary at both individual and organizational levels. Similar to international implementation models, the model of supporting inside-out empowerment by embracing relatedness involved individuals, organizations, and interpersonal and inter-organizational networks. However, the model suggests that for minority Indigenous populations within developed countries, implementation approaches may require greater attention to the empowering nature of the intervention and its implementation, and multiple levels of relatedness by individuals and organizations with self, others, and the structural conditions. Key elements of the theoretical model provide a useful blueprint to inform the transfer of other empowerment programs to minority Indigenous and other disadvantaged populations on a case-by-case basis.Implementation Science 10/2013; 8(1):129. · 2.37 Impact Factor
- [Show abstract] [Hide abstract]
ABSTRACT: While Aboriginal Australian health providers prioritise identification of local community health needs and strategies, they do not always have the opportunity to access or interpret evidence-based literature to inform health improvement innovations. Research partnerships are therefore important when designing or modifying Aboriginal Australian health improvement initiatives and their evaluation. However, there are few models that outline the pragmatic steps by which research partners negotiate to develop, implement and evaluate community-based initiatives. The objective of this paper is to provide a theoretical model of the tailoring of health improvement initiatives by Aboriginal community-based service providers and partner university researchers. It draws from the case of the Beat da Binge community-initiated youth binge drinking harm reduction project in Yarrabah. A theoretical model was developed using the constructivist grounded theory methods of concurrent sampling, data collection and analysis. Data was obtained from the recordings of reflective Community-Based Participatory Research (CBPR) processes with Aboriginal community partners and young people, and university researchers. CBPR data was supplemented with interviews with theoretically sampled project participants. The transcripts of CBPR recordings and interviews were imported into NVIVO and coded to identify categories and theoretical constructs. The identified categories were then developed into higher order concepts and the relationships between concepts identified until the central purpose of those involved in the project and the core process that facilitated that purpose were identified. The tailored alcohol harm reduction project resulted in clarification of the underlying local determinants of binge drinking, and a shift in the project design from a social marketing awareness campaign (based on short-term events) to a more robust advocacy for youth mentoring into education, employment and training. The community-based process undertaken by the research partnership to tailor the design, implementation and evaluation of the project was theorised as a model incorporating four overlapping stages of negotiating knowledges and meanings to tailor a community response. The theoretical model can be applied in spaces where local Aboriginal and scientific knowledges meet to support the tailored design, implementation and evaluation of other health improvement projects, particularly those that originate from Aboriginal communities themselves.BMC Public Health 08/2013; 13(1):726. · 2.08 Impact Factor
Applying what works: a systematic search of the
transfer and implementation of promising
Indigenous Australian health services and programs
* Corresponding author
1 The Cairns Institute and School of Education, James Cook University,
McGregor Rd, Smithfield 4878, Australia
2 The Cairns Institute and School of Education, James Cook University,
McGregor Rd, Smithfield 4878, Australia
3 Institute for Urban Indigenous Health, Edgar St, Bowen Hills 4006, Australia
4 The Cairns Institute and School of Arts and Social Sciences, James Cook
University, McGregor Rd, Smithfield 4878, Australia
5 University of NSW, King St, Sydney 2031, Australia
6 The Cairns Institute and School of Education, James Cook University,
McGregor Rd, Smithfield 4878, Australia
The transfer and implementation of acceptable and effective health services, programs and
innovations across settings provides an important and potentially cost-effective strategy for
reducing Indigenous Australians' high burden of disease. This study reports a systematic
review of Indigenous health services, programs and innovations to examine the extent to
which studies considered processes of transfer and implementation within and across
Indigenous communities and healthcare settings.
Medline, Informit, Infotrac, Blackwells Publishing, Proquest, Taylor and Francis, JStor, and
the Indigenous HealthInfoNet were searched using terms: Aborigin* OR Indigen* OR Torres
AND health AND service OR program* OR intervention AND Australia to locate
publications from 1992–2011. The reference lists of 19 reviews were also checked. Data from
peer reviewed journals, reports, and websites were included. The 95% confidence intervals
(95% CI) for proportions that referred to and focussed on transfer were calculated as exact
binomial confidence intervals. Test comparisons between proportions were calculated using
Fisher's exact test with an alpha level of 5%.
Of 1311 publications identified, 119 (9.1%; 95% CI: 7.6% - 10.8%) referred to the transfer
and implementation of Indigenous Australian health services or programs, but only 21 studies
(1.6%; 95% CI: 1.0% - 2.4%) actually focused on transfer and implementation. Of the 119
transfer studies, 37 (31.1%; 95% CI: 22.9 - 40.2%) evaluated the impact of a service or
program, 28 (23.5%; 95% CI: 16.2% - 32.2%) reported only process measures and 54 were
descriptive. Of the 37 impact evaluation studies, 28 (75.7%; 95% CI: 58.8% - 88.2%)
appeared in peer reviewed journals but none included experimental designs.
While services and programs are being transferred and implemented, few studies focus on the
process by which this occurred or the effectiveness of the service or program in the new
setting. Findings highlight a need for partnerships between researchers and health services to
evaluate the transfer and implementation of Indigenous health services and programs using
rigorous designs, and publish such efforts in peer-reviewed journals as a quality assurance
Indigenous, Transfer, Spread, Dissemination, Implementation, Adoption, Uptake.
The transfer and implementation of acceptable and effective health services, programs and
innovations across settings provides an important and potentially cost-effective strategy for
reducing Indigenous Australians’ high burden of disease [1,2]. The criteria of Australia’s
leading health research organisation, the National Health and Medical Research Council,
explicitly require researchers working in Indigenous settings to demonstrate the
transferability and sustainability of programs and research benefits . Transfer and
implementation can occur through 1) a hierarchical, centrally-driven and controlled process;
2) a more decentralised and participatory adaptive approach supported by experts, or 3) an
informal and largely uncontrolled grass roots process whereby organisations define their
problems and search for ‘packaged solutions’ which they can adapt to address local needs .
The Indigenous Australian health literature varies considerably in how the processes of
transfer and implementation are conceptualised. Included in this review, for example, are
studies that referred to dissemination, extension, transfer, translation, adaptation,
implementation, uptake and spread. We use the term transfer to refer to the shift of a service
or program across sites, or to a different target group within a site, and the term
implementation to refer to the uptake and delivery of a service or program in a new site.
There are good reasons for a research focus on the extent to which existing proven or
promising Indigenous-specific service delivery models or prevention programs are
transferred and implemented within and across communities and healthcare settings.
Research which examines the transfer and implementation of health services and programs
could: reduce the likelihood of successful programs being unsuccessfully transferred and
implemented in other locations; enhance the efficiency of processes of transfer and
implementation; and increase intervention research in the Indigenous health field generally.
Despite these potential benefits, research on the transfer and implementation of health
services and programs is lacking. For example, a recent review of evaluations of
dissemination strategies for improving the uptake of evidence-based smoking, nutrition,
alcohol and physical activity interventions published in the peer-reviewed literature found
only 11 publications . The authors concluded that more Indigenous-specific research is
needed to determine which dissemination strategies are most likely to be effective for
increasing uptake of evidence-based health care across Indigenous health settings .
This study used a systematic search to analyse the peer-reviewed and grey literature to
determine the extent to which research and reports focusing on Indigenous health strategies
have contained information on the transfer and implementation of these strategies within and
between Indigenous communities. It enhances the previous review  in three principal
ways. First it includes all publications relating to Indigenous Australian health services and
programs to examine the extent to which they refer to transfer. Second, it searches the grey
literature in addition to peer-reviewed literature. Third, it examines three hypotheses in order
to more precisely articulate directions for future research in the transfer and implementation
of health services and programs in Indigenous Australian settings. The three hypotheses
1. That few published studies/reports will evaluate or describe the transfer and
implementation of a service, program or innovation from one setting to another;
2. That a greater proportion of transfer evaluation studies will measure process, rather than
3. That a greater proportion of transfer outcome evaluations will use non-experimental than
The process used to identify and classify studies was consistent with Cochrane methods for
systematic searches .
To capture evidence of the transfer and implementation of Indigenous Australian health
services and programs, studies were included in this review if they evaluated, described or
reviewed Indigenous Australian health services or programs and were published between
1992 and 2011 (inclusive) in the peer review or grey literature. A substantial proportion of
Indigenous health research is published in the grey literature, making it an important source
. In cases where a relevant study was published in both the peer review and grey literature,
we included the grey literature only if it referred to a discrete aspect of a service or program
not included in its peer reviewed counterpart. Services, programs and innovations were
defined as systematic actions and approaches taken to address an identified Indigenous health
need . Health was defined broadly according to the Indigenous Australian definition which
includes physical, mental, emotional and spiritual wellbeing .
A two-step search strategy, summarised in Figure 1, was utilised. First, electronic databases
Informit, Infotrac, Blackwells Publishing, Proquest, Taylor and Francis, JStor, Medline and
the Australian Indigenous HealthInfoNet were separately searched (last date: 25 November
2011) for citations that included the following terms in the title, abstract or MeSH heading:
Aborigin* OR Indigen* OR Torres AND health AND service OR program* OR intervention
AND Australia. We identified 1554 references (after removal of duplicates). Second, the
reference lists of 19 Indigenous health-related literature reviews, identified through database
search, were examined. This process identified an additional 75 references.
Classification of studies
The 1629 references identified in step 1 were classified in a four step process.
Step 1: Identification of studies for exclusion: We excluded studies that were 1) not
Indigenous-Australian-specific; 2) not related to the provision of a service or program; or 3)
duplicates. Given that some services and programs changed their names during the 20 year
timeframe and were cited in different ways, the elimination process may have underestimated
the number of duplications. Step one excluded 318 publications.
Step 2: Identification of transfer studies and type of transfer: The remaining 1311 references,
which documented 1098 programs and services and 19 reviews, were entered into an Excel
spreadsheet. They comprised 309 peer reviewed papers and books/book chapters and 1002
reports and websites. Abstracts were searched by one author (JM) to classify studies
according to whether transfer and implementation was: 1) the focus of the study, 2)
considered as one of several key themes, or 3) not addressed. If an abstract suggested (but did
not make explicit) transfer, the conclusions were also searched. Step 2 identified a total of
119 “transfer studies” (9.1% of 1311). Transfer studies (n = 119) were further classified by
three authors (KT, RB and JM) to identify the extent to which they focused on the transfer
and implementation of a health service or program in Indigenous healthcare settings, with an
initial inter-rater agreement of 82.4%. The studies for which there was a discrepancy were re-
evaluated until consensus was reached by the three authors. The process of transfer described
in the studies which focused on transfer was classified according to the theory described
previously as a: 1) hierarchical, centrally-driven; 2) decentralised and participatory, or 3)
informal grass roots process .
Step 3: Classification of studies: The 119 transfer studies (which documented 97 services or
programs) were then classified as evaluative or descriptive studies. Impact/outcome
evaluation studies were defined as those that informed understanding about the effectiveness
or acceptability of Indigenous health services or programs. Process evaluation studies were
those which measured reach, satisfaction, quality and implementation (how to produce
change). Descriptive studies were “descriptions of methods or processes .... in which no data-
based evaluation was reported” [9,10]. Studies which reported both process and
impact/outcome measures were classified as impact/outcome evaluations. Step 3 found 37
studies (31.1% of 119) which reported impact measures.
Step 4: Quality of studies: The likely extent of scientific rigour of the 37 impact/outcome
evaluation studies was assessed in terms of whether they: 1) had been peer-reviewed, and 2)
used an experimental design. Peer-review was included as a quality indicator since papers
published in the scientific literature have been subject to peer review while those published in
the grey literature most likely have not. As per Sanson-Fisher et al.. , peer-reviewed
studies were then classified as either controlled experimental designs (randomised and non-
randomised controlled trials) or non-experimental (cohort/longitudinal analytic studies, case–
control studies, single group pre-post or post-evaluation measurement, and other).
The 95% confidence intervals (95% CI) for proportions were calculated as exact binomial
confidence intervals. Test comparisons between counts and proportions were conducted using
SPSS (IBM) version 20 and Fisher’s exact test with an alpha level of 5%.
Of the 1311 publications identified as dealing with Indigenous Australian health services,
programs or innovations, 119 (9.1%; 95% CI: 7.6% – 10.8%) referred to their transfer.
Transfer or implementation was the primary focus of 21 of these 1311 studies (1.6%; 95%
CI: 1.0% - 2.4%) and was only considered by the remaining 98 studies (7.5% of 1311) (Table
1). Of these 21 studies, seven merely described protocols for transfer while the other 14
evaluated or described transfer processes that had actually occurred. The most common
process for transfer (12/21 or 57.1% studies) was through the central development but
decentralised implementation of an initiative. This decentralised transfer involved
community-based participation and adaptation of the intervention, often with support from
researchers [11-22]. We also found five studies of informal, grass-roots transfer [23-27],
three cases of hierarchical transfer [28-30] and one review . Services and programs
targeted health professionals, health service clients, school students, community groups and
community members. Hence we accepted the first hypothesis of this study, that there are
relatively few published studies describing or evaluating the transfer of service delivery
models or prevention programs.
Table 1 Studies that focussed on transfer and implementation
Focus of study and evaluation or
Brady et. al.
(2002) and acceptability of implementing
brief intervention for alcohol
Clifford et al.
(2009) strategies for smoking, nutrition,
alcohol misuse and physical
inactivity interventions to health
Gardner et al
(2010) implementation of Audit and Best
Practice for Chronic Disease project
Type of transfer Target
Process evaluation of the feasibility Decentralised
Systematic review of dissemination Review Varied
Examines uptake and Decentralised
Gardner  et al,
Reviews the challenges of
implementing a primary health care
quality improvement project in
remote Australia and the South
Evaluation of implementation of
national recommendations for the
clinical management of alcohol-
related problems in Indigenous
primary health care settings
Overview of the spread of the
Mental Health First Aid program
across Australia and internationally
Hunter et al
transfer linked with
McCalman et al
The role of participatory action
research in transferring knowledge
roots transfer with
McKay et al.
Process evaluation of the
appropriateness and effectiveness of
an across-community knowledge
sharing suicide prevention project -
Impact and process evaluation of a
well women's health program
Mitchell (2006) Decentralised
Across NSW NSW Department
of Health (2010)
Evaluates the NSW SmokeCheck
Aboriginal Tobacco Prevention
Describes the use of traditional
Indigenous games in two schools
Parker et al.
Rowley et al.
Outcome evaluation of the Looma
Healthy Lifestyle project
roots transfer with
advice and support
Sheehan et al.
Process evaluation of the
acceptability of the mainstream
Mind Matters Program
in a remote
in a remote
Tsey et al. (2004) Process evaluation of adapting the
Family Wellbeing Program
roots transfer with
Bailie et al. (2008) Extension phase of Audit for Best
Practice in Chronic Disease project
to examine factors that influence
uptake and sustainability
Bailie et al. (2010) Examine factors associated with
variations in implementation of
Audit for Best Practice in Chronic
Disease project and effective
strategies to enhance clinical
performance and implementation
Kendall, (2011) by which the Stanford Chronic
Disease Self-Management program
should be delivered to enhance
Field et al. (2001) Process evaluation of Laramba
Family Wellness model
Process evaluation of the principles Hierarchical
Description of the transfer of the
Army Aboriginal community
care and other
Midford, Daly &
Provides a model for community
involvement in the care of public
Process evaluation of barriers to the
implementation of the SAFE
strategy – trachoma
Wright et al.
Of the 119 transfer studies, 37 (31.1%; 95% CI: 22.9 - 40.2%) evaluated the impact of the
service or program. This proportion was significantly higher than the proportion of impact
evaluation studies among the remaining 1192 publications in this review (16.7%; p < 0.05)
and the proportion of evaluation studies reported by other reviews of Indigenous Australian
health publications (5.8%; p < 0.05) . In comparison, 28 transfer studies (23.5%; 95% CI:
16.2% - 32.2%) measured only process indicators. Hence we rejected our second hypothesis
that most transfer studies evaluating a service or program are process evaluations. Twenty-
eight of the 37 transfer studies both appeared in the peer-reviewed literature and included an
impact evaluation of a service or program (75.7%; 95% CI: 58.8% - 88.2%). However, none
of the impact evaluations were based on experimental study designs. Hence we accepted our
third hypothesis, that transfer studies evaluating the impact/outcomes of a service or program
predominantly use non-experimental research designs. A meta-analysis of findings across
studies was deemed inappropriate due to the variability of transfer processes, target groups
and outcomes incorporated within the studies, and their methodological deficiencies.
The overall findings of this review have contributed to the pool of knowledge in the area of
Indigenous Australian program transfer. They indicate that there is a lack of published
evaluations of the transfer of services and programs within and across Indigenous Australian
health settings, and these evaluations are not employing rigorous study designs. The findings
have implications for the required steps to improve our understanding of how to successfully
transfer and implement health strategies within and between Indigenous communities.
The review provided an opportunity to assess the contribution of the grey literature. The grey
literature contributed 49/119 (40.8%) of the transfer studies, suggesting that: 1) health
practitioners and others document their transfer efforts in the grey literature and that such
reports and websites may potentially influence the transfer decisions of others; and 2) that
investing in the review of the grey literature was productive. However, our finding that the
grey literature contributed 1002/1311 (76.4%) of the initial publications reviewed raised
concerns that considerable resources are being invested into the documentation of Indigenous
health services and programs which are neither subjected to the quality assurance mechanism
of peer review nor readily available .
Consistent with hypothesis one, this review suggests that few published studies report the
transfer and implementation of a service or program across sites or groups. This makes it
difficult for health practitioners, researchers and others to identify transfer processes that
would reliably result in health improvement, assist in accessing hard-to-reach community
members, or provide best value for money . Contrary to hypothesis two, it is promising
that 31% of the 119 transfer studies in this review did attempt to evaluate the impact of a
program transferred to a new site. Yet consistent with hypothesis three, these evaluations
were all based on non-experimental research designs which provide weaker evidence of cause
and effect than experimental designs . This finding is not surprising since Indigenous
health research has been predominantly descriptive  and few intervention studies have met
rigorous methodological criteria [9,32].
The methods used to establish these findings have limitations. The publications in this review
were identified with a non-exhaustive search strategy designed to produce the bulk of peer-
and non-peer-reviewed Indigenous Australian health studies that described or evaluated
services or programs. It is therefore possible that some relevant publications were missed,
particularly those published in the grey literature which is more difficult to systematically
search than the peer-reviewed literature. However, given the two-step strategy of searching
electronic databases and reference lists of reviews, it is highly likely that the studies
represented in this review are representative of published transfer and implementation
research in the Indigenous health field.
While the study undertook a quality assessment, this did not extend to an assessment of bias
that may have characterised the identified studies. The measure of quality used was based
only on whether the study design was peer-reviewed and/or experimental or non-
experimental. This is a relatively superficial measure because experimental study designs can
be low quality if they are characterised by selection, measurement or other biases – even if
they have been peer-reviewed. However, we felt that these quality measures were sufficient
given the study’s exploratory nature and the qualitative nature of program transfer.
A pragmatic approach for improving Indigenous Australian health is to adapt effective
services and programs that have been successfully and routinely delivered in some health
settings to others. This systematic search, however, found evidence that few descriptions of
the successful transfer of programs or services are readily available (1.6% of publications),
and while one-third of transfer publications referred to services or programs whose impacts
have been evaluated, none reported an experimental evaluation design.
There has also been a lack of theoretical conceptualisation of the processes of transfer and
implementation. This gap is possibly due to a lack of capacity or authority to document the
processes and outcomes of transfer across sites, and little evidence of the effectiveness of the
service or program in the new setting. This implies a need for theorisation of the processes by
which transfer and implementation have occurred, and evaluation of the effects of transfer
and implementation through multi-partner collaborations between researchers and health
services. Rigorous evaluation designs that can be implemented simultaneously with the
transfer of programs or services are available, such as multiple baseline designs and are
endorsed by the Cochrane Effective Practice and Organization of Care Group . More
routine utilisation of these evaluation designs would provide greater confidence that any
improvements are reasonably attributable to the transferred program. The results of such
evaluation efforts need to be published in peer-reviewed journals to increase awareness of
effective processes for transfer and implementation, and as a quality assurance mechanism.
The authors declare that they have no competing interests.
JM, KT, WE and RB conceived of the study and participated in its design and coordination.
JM took the lead role in reviewing and assessing the literature with KT co-assessing a sample
of studies. JM drafted the paper. AS and AC revised the study hypotheses and methods and
edited the draft paper. All authors read and approved the final manuscript.
Thanks to Annita Virzi and Cath Brown for assistance with the literature search and Reinhold
Muller for statistical advice. We acknowledge financial support for this review from the
Queensland Centre for Social Science Innovation.
1. Wejnert B: A conceptual threshold model of adoption of innovations as a function of
innovation's value and actor's characteristics. Journal of Asia-Pacific Business 2010,
2. Ovretveit J: Widespread focused improvement: lessons from international health for
spreading specific improvements to health services in high-income countries.
International Journal for Quality in Health Care 2011, 23(3):239–246.
3. National Health and Medical Research Council: Criteria for health and medical research
of Indigenous Australians n.d. Canberra: Commonwealth of Australia; 2003:1.
4. Clifford A, et al: Disseminating best-evidence health care to Indigenous health-care
settings and programs in Australia: identifying the gaps. Heal Promot Int 2009,
5. Jackson N: Handbook for systematic reviews for health promotion and public health
interventions. Melbourne: Deakin University and Australian Department of Health and
6. Derrick G, et al: A bibliometric analysis of research on Indigenous health in Australia,
1972–2008. Australian and New Zealand Journal of Public Health 2011, online.
7. Clapham K, O'Dea K, Chenhall R: Interventions and sustainable programs, in. In Social
determinants of Indigenous health. Edited by Bronwyn Carson TD, Richard C, Ross B. Allen
and Unwin: Crows Nest; 2007.
8. National Aboriginal Health Strategy Working Party (NAHS: A National Aboriginal Health
Strategy. Canberra: AGPS; 1989.
9. Sanson-Fisher RW, et al: Indigenous health research: a critical review of outputs over
time. Australasian Medical Publishing Company 2006, 184(10):502–505.
10. Hawe P, Degeling D, Hall J: Evaluating health promotion: A health worker's guide.
Sydney: MacLennan Petty; 1990.
11. Brady M, et al: The feasibility and acceptability of introducing brief intervention for
alcohol misuse in an urban Aboriginal medical service. Drug and Alcohol Review 2002,
12. Gardner K, et al: Understanding uptake of continuous quality improvement in
Indigenous primary health care: lessons from a multi-site case study of the Audit and
Best practice for Chronic Disease project. Implement Sci 2010, 5(21):14.
13. Gardner K, et al: Reorienting primary health care for addressing chronic conditions
in remote Australia and the South pacific: review of evidence and lessons from an
innovatibe quality improvement process. Aust J Rural Heal 2011, 19:111–117.
14. Kitchener B, Jorm A: Early intervention in the real world. Mental Health First Aid:
An international programme for early intervention. Early Intervention in Psychiatry
15. Mitchell J: A matter of urgency! Remote Aboriginal women's health: Examining the
transfer, adaptation and implementation of an established holistic Aboriginal Well Women's
health program from one remote community to another with similar needs and
characteristics, in School of Nursing and Midwifery, Faculty of Health Sciences. Adelaide:
Flinders University; 2006.
16. NSW Department of Health, Cancer Institute, and University of Sydney, Final report. The
NSW Aboriginal SmokeCheck tobacco prevention project 2007–08: NSW Department of
Health, Cancer Institute. Sydney: University of Sydney; 2010:117.
17. Parker E, et al: Our games our health: a cultural asset for promoting health in
indigenous communities. Health Promotion Journal of Australia 2006, 17(2):103.
18. Sheehan M, Ridge D, Marshall B: 'This was a great project': reflections on a
'successful' mental health promotion project in a remote Indigenous school. Health
Promotion Journal of Australia 2002, 13(3):201–204.
19. Bailie R, et al: Study protocol: Audit and best practice for chronic disease extension
(ABCDE) project. BMC Heal Serv Res 2008, 8(184):7.
20. Bailie R, et al: Study protocol: national research partnership to improve health care
performance and outcomes for Indigenous people. NBMC Health Service Research 2010,
21. Field P, McLay A, Grundy J: Laramba family wellness model: Integration, sustainability
and transferability. 6th edition. Canberra: National Rural Health Conference; 2001:7.
22. Wright H, Keeffe J, Taylor H: Barriers to the implementation of the SAFE strategy to
combat hyperendemic trachoma in Australia. Opthalmic Epidemiology 2010, 17(6):349–
23. McCalman J, et al: Bringing back respect: the role of participatory action research in
transferring knowledge from an Aboriginal men's group to youth programs.
Australasian Psychiatry 2009, 17:S59–S63.
24. McKay K, et al: Building bridges: Learning from the experts. Brisbane: Australian
Institute for Suicide Research and Prevention, Griffith University; 2009:27.
25. Rowley K, et al: Effectiveness of a community-directed 'healthy lifestyle' program in
a remote Australian Aboriginal community. Australian & New Zealand Journal of Public
Health 2000, 24(2):136–144.
26. Tsey K, et al: Adapting the 'Family Wellbeing' empowerment program to the needs
of remote Indigenous school children. Australian and New Zealand Journal of Public
Health 2004, 29(2):112–116.
27. Midford R, Daly A, Holmes M: The care of public drunks in Halls Creek. Health
Promotion Journal of Australia 1994, 4(1):5–8.
28. Hunter E, Brown J, McCulloch B: Encouraging practitioners to use resources:
Evaluation of the national implementation of a resource to improve the clinical
management of alcohol-related problems in Indigenous primary care settings. Drug and
Alcohol Review 2004, 23:89–100.
29. Department of Families Community Services and Indigenous Affairs: Army Aboriginal
Community Assistance Program. Canberra: Australian National Audit Office,
Commonwealth of Australia; 2010.
30. Barnett L, Kendall E: Culturally appropriate methods for enhancing the participation
of Aboriginal Australians in health-promoting programs. Health Promotion Journal of
Australia 2011, 22(1):27–32.
31. Redman S: Towards a research strategy to support public health programs for
behaviour change. Australia and New Zealand Journal of Public Health 1996, 20(4):352–8.
32. Paul C, et al: Being sorry is not enough. The sorry state of the evidence base for
improving the health of Indigenous populations. American Journal of Preventive Medicine
33. Cochrane Effective Practice and Organisation of Care Group: The data collection
checklist. Ottawa: Cochrane Effective Practice and Organisation of Care Group; 2003.