ArticlePublisher preview available

Developing and Using a Common Framework to Evaluate FASD Programs: Results of a Three-Year Canadian Project

DOI:10.1007/s11469-015-9597-3
Authors:
Deb Rutman at University of Victoria
Deb Rutman
Carol Hubberstey
Carol Hubberstey
Carol Hubberstey
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Nancy Poole at Centre of Excellence for Women's Health
Nancy Poole
  • Centre of Excellence for Women's Health
Sharon Hume
Sharon Hume
Sharon Hume
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Show all 5 authorsHide
Read publisher preview
Download citation
To read the full-text of this research, you can request a copy directly from the authors.

Abstract and Figures

This article discusses a three-year Canadian project that created common Evaluation Frameworks for Fetal Alcohol Spectrum Disorder (FASD) support programs and for FASD prevention programs (i.e., programs serving people living with FASD and programs serving pregnant women and mothers). The project’s mixed-methods approach included a comprehensive literature search and consultations across Canada with multi-disciplinary service providers, program funders, researchers, and evaluators. These activities led to development of three visual “maps” depicting evaluation of: a) FASD support programs; b) FASD prevention programs; and c) FASD programming in Aboriginal communities. In addition, the team provided mentoring and evaluation-related support to program staff, funders and/or partners of five community-based FASD-related agencies. Informed by a social determinants of health lens, the maps are comprised of concentric rings showing programs’: theoretical foundations; activities; program outcomes; and wholistic participant, community and systemic outcomes. The article also shares findings regarding the applicability and utility of the Frameworks and of evaluation-related mentoring.
Figures - available from: International Journal of Mental Health and Addiction
This content is subject to copyright. Terms and conditions apply.
  • A preview of this full-text is provided by Springer Nature.
  • Learn more
Preview content only
Content available from International Journal of Mental Health and Addiction
This content is subject to copyright. Terms and conditions apply.
Developing and Using a Common Framework to Evaluate
FASD Programs: Results of a Three-Year
Canadian Project
Deborah Rutman
1,2
&Carol Hubberstey
1
&
Nancy Poole
2,3
&Sharon Hume
1
&Marilyn Van Bibber
1,2
Published online: 21 October 2015
#Springer Science+Business Media New York 2015
Abstract This article discusses a three-year Canadian project that created common Evaluation
Frameworks for Fetal Alcohol Spectrum Disorder (FASD) support programs and for FASD
prevention programs (i.e., programs serving people living with FASD and programs serving
pregnant women and mothers). The project’s mixed-methods approach included a comprehensive
literature search and consultations across Canada with multi-disciplinary service providers, program
funders, researchers, and evaluators. These activities led to development of three visual “maps”
depicting evaluation of: a) FASD support programs; b) FASD prevention programs; and c) FASD
programming in Aboriginal communities. In addition, the team provided mentoring and evaluation-
related support to program staff, funders and/or partners of five community-based FASD-related
agencies. Informed by a social determinants of health lens, the maps are comprised of concentric
rings showing programs’: theoretical foundations; activities; program outcomes; and wholistic
participant, community and systemic outcomes. The article also shares findings regarding the
applicability and utility of the Frameworks and of evaluation-related mentoring.
Keywords Fetal alcohol spectrum disorder.Program evaluation .Evaluation mentoring .
Evaluation framework .FASD support programs .Social determinants of health
In recent years, the research literature and practice evidence are increasingly emphasizing the
value of a social determinants of health approach in working to prevent Fetal Alcohol
Int J Ment Health Addiction (2016) 14:472–482
DOI 10.1007/s11469-015-9597-3
*Deborah Rutman
drutman@uvic.ca
1
Nota Bene Consulting Group, 1434 Vining St., Victoria, BC V8R 1P8, Canada
2
Canada FASD Research Network, Network Action Team on Prevention, Vancouver, BC, Canada
3
British Columbia Centre of Excellence for Women’s Health, Vancouver, Canada
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
... Establishing the effectiveness of such programs has been challenging, in part because they are community-based and/or lack resources, time, knowledge, or expertise to carry out an evaluation of their programs, or they are small programs that have not been the focus of larger studies [16]. As such, many FASD prevention services have not been formally evaluated, have not had the resources or opportunities to conduct a multi-year evaluation, or their evaluations have not been published [17]. ...
... In program evaluation, a Theory of Change is created to show the interconnections between the program's philosophy/approaches; activities; and anticipated outcomes at the client/family, community, and systems levels. As part of the developmental phase of this project, the team and the Program Coordinators/Managers of the eight participating sites collaboratively developed a Theory of Change (depicted in Figures 2-4); this was based on a framework for evaluation of FASD prevention and support programs [17]. In keeping with a collaborative and community approach to developing a Theory of Change, the CCE Theory of Change was created through multiple discussions during the initial face-to-face meeting with all programs in June 2017; following this, the research team revised and finalized the Theory of Change to incorporate all the Program Coordinators' feedback. ...
National Evaluation of Canadian Multi-Service FASD Prevention Programs: Interim Findings from the Co-Creating Evidence Study
Article
Full-text available
Since the 1990s, a number of multi-service prevention programs working with women who have substance use, mental health, or trauma and/or related social determinants of health issues have emerged in Canada. These programs use harm reduction approaches and provide outreach and “one-stop” health and social services on-site or through a network of services. While some of these programs have been evaluated, others have not, or their evaluations have not been published. This article presents interim qualitative findings of the Co-Creating Evidence project, a multi-year (2017–2020) national evaluation of holistic programs serving women at high risk of having an infant with prenatal alcohol exposure. The evaluation utilizes a mixed-methods design involving semi-structured interviews, questionnaires, focus groups, and client intake/outcome “snapshot” data. Findings demonstrated that the programs are reaching vulnerable pregnant/parenting women who face a host of complex circumstances including substance use, violence, child welfare involvement, and inadequate housing; moreover, it is typically the intersection of these issues that prompts women to engage with programs. Aligning with these results, key themes in what clients liked best about their program were: staff and their non-judgmental approach; peer support and sense of community; and having multiple services in one location, including help with mandated child protection.
... that provides the information and tools developed and/or compiled through the project. 21,22 what is an FAsd-informed approach? ...
Becoming FASD Informed: Strengthening Practice and Programs Working with Women with FASD
Article
Full-text available
  • May 2016
There is growing appreciation among health and social care providers, especially those working in community-based programs with women or young people with substance use problems and/or who have experienced violence, maltreatment, or trauma, that a high number of their program participants may have been prenatally exposed to alcohol or have fetal alcohol spectrum disorder (FASD). This article provides a conceptualization of the key components of an FASD-informed approach. Drawing on the emerging literature and the author's research identifying the support needs and promising approaches in working with women, young adults, and adults with FASD, as well as evaluations of FASD-related programs, the article discusses what an FASD-informed approach is, why it is centrally important in working with women, adults, and young people who may have FASD, underlying principles of an FASD-informed approach, and examples of FASD-informed adaptations to practice, programming, and the physical environment. In this discussion, the benefits of using an FASD-informed approach for service providers and women living with FASD and their families, as well as conceptualization of FASD-informed policy and systems are highlighted.
Challenges and resiliency in Aboriginal adults with Fetal Alcohol Spectrum Disorder
Article
  • Jan 2018
Neuropsychological Aspects of Prevention and Intervention for FASD in Canada
Article
  • Oct 2016
Fetal alcohol spectrum disorder (FASD) is the leading cause of developmental disability among Canadians. As a result, FASD has emerged as a public health priority in Canada. The current paper provides an overview of FASD in Canada with respect to (1) epidemiology; (2) conceptualizations of FASD by schools, the government, mental health services and the medical community; (3) current research; (4) prevention initiatives; and (5) intervention efforts. Overall, Canada has become a world leader in advancing knowledge about FASD, in part through the establishment of research networks aimed at preventing FASD and improving the lives of individuals affected by the disorder across the lifespan. A nationwide research framework has strengthened the links between research and practice in Canada, and major bodies in Canada appear to be moving towards a more integrated model for conceptualizing FASD, with increased investment in prevention and intervention. Although research and evaluation evidence is emerging, however, the strength of current evidence does not yet allow for prevalence rates or prevention and intervention best practices to be defined with certainty. With quality cross-sectoral research and evaluation that incorporates the lived experiences of parents and individuals with FASD, it will be possible to enhance understanding of facilitators and barriers to service delivery, access and utilization with respect to FASD prevention and intervention in Canada.
Breaking the cycle: Measures of progress 1995-2005
Article
Full-text available
  • Jan 2006
Building a framework for evaluation of Fetal Alcohol Spectrum Disorder prevention and support programs: A collaborative Canadian project
Article
Full-text available
  • Mar 2014
Rutman, D., Poole, N., Hume, S., Hubberstey, C., & Van Bibber, M. (2014). Building a framework for evaluation of Fetal Alcohol Spectrum Disorder prevention and support programs: A collaborative Canadian project. The International Journal Of Alcohol And Drug Research, 3(1), 81-89. doi:10.7895/ijadr.v3i1.122 Aims: This article discusses a Canadian project that is designed to identify promising evaluation methods and create common evaluation frameworks for FASD prevention programs serving pregnant women and mothers, and FASD supportive intervention programs serving youth and adults living with FASD. A social determinants of health perspective guided the project. Design: The project has employed a mixed-methods approach including a literature search, documentary review, and an iterative set of consultations with program providers, program managers, government managers and funders, researchers, and evaluators in the context of their work across Canada and internationally. Results: The project’s processes led to the development of three visual “maps” comprised of concentric rings that depict theoretical foundations; activities and approaches; formative outcomes; and participant, community and systemic outcomes. The three visual frameworks depict evaluation of 1) FASD prevention programs; 2) FASD support programs; and 3) FASD programs in Aboriginal communities. Conclusions: The development of visual maps to depict common evaluation frameworks promotes individual and collective action towards applying the frameworks on the part of community-based services and governments across Canada, on the service and systemic levels. Program providers, researchers, and system planners have indicated that the maps have wide-ranging applications.
Evaluation of a three-year Youth Outreach Program for Aboriginal youth with suspected Fetal Alcohol Spectrum Disorder
Article
Full-text available
  • Mar 2014
Aims: This article discusses the process, findings and lessons learned from the external evaluation of the Youth Outreach Program (YOP), a three-year intensive outreach and support program intended for at-risk Aboriginal youth, 13 to 18, with characteristics and/or behaviors associated with Fetal Alcohol Spectrum Disorder (FASD). The overall purpose of the formative and summative evaluation was to document the development and early implementation of the program, in order to make adjustments as the program unfolded; and to assess, describe, and document early outcomes for youth participants and community partners. Design: The project employed a mixed-method design using qualitative and quantitative data. Setting: The program and evaluation took place in a small, rural community in British Columbia, Canada. Participants: Program managers, program staff, program participants, community partners, family members of participants. Measures and methods: Triangulation, program output data; program-specific data collection tools, such as qualitative interview guides; participant outcome rating scale.
A Holistic Model for the Selection of Environmental Assessment Indicators to Assess the Impact of Industrialization on Indigenous Health
Article
Full-text available
  • Mar 2011
Mainstream environmental assessment (EA) methodologies often inadequately address health, social and cultural impacts of concern for Canadian indigenous communities affected by industrialization. Our objective is to present a holistic, culturally-appropriate framework for the selection of indigenous health indicators for baseline health assessment, impact prediction, or monitoring of impacts over time. We used a critical population health approach to explore the determinants of health and health inequities in indigenous communities and conceptualize the pathways by which industrialization affects these determinants. We integrated and extended key elements from three indigenous health frameworks into a new holistic model for the selection of indigenous EA indicators. The holistic model conceptualizes individual and community determinants of health within external social, economic and political contexts and thus provides a comprehensive framework for selecting indicators of indigenous health. Indigenous health is the product of interactions among multiple determinants of health and contexts. Potential applications are discussed using case study examples involving indigenous communities affected by industrialization. Industrialization can worsen indigenous health inequities by perpetuating the health, social and cultural impacts of historic environmental dispossession. To mitigate impacts, EA should explicitly recognize linkages between environmental dispossession and the determinants of health and health inequities and meaningfully involve indigenous communities in the process.
Preventing Alcohol and Drug Exposed Births in Washington State: Intervention Findings From Three Parent-Child Assistance Program Sites
Article
Full-text available
  • Feb 2005
Home visitation interventions show promise for helping at-risk mothers, yet few programs have been developed and evaluated specifically for alcohol and drug-abusing pregnant women. This study examines outcomes among 216 women enrolled in the Washington State Parent-Child Assistance Program, a three-year intervention program for women who abuse alcohol and drugs during an index pregnancy. Pretest-posttest comparison was made across three sites: the original demonstration (1991-1995), and the Seattle and Tacoma replications (1996-2003). In the original demonstration, the client group performed significantly better than controls. Compared to the original demonstration, outcomes at replication sites were maintained (for regular use of contraception and use of reliable method; and number of subsequent deliveries), or improved (for alcohol/drug treatment completed; alcohol/ drug abstinence; subsequent delivery unexposed to alcohol/drugs). Improved outcomes at replication sites are not attributable to enrolling lower-risk women. Public policies and programs initiated over the study period may have had a positive effect on outcomes. Study findings suggest that this community-based intervention model is effective over time and across venues.
Development Evaluation: Applying Complexity Concepts to Enhance Innovation and Use
Book
  • Jan 2011
“New Choices” for substance‐using mothers and their children: Preliminary evaluation
Article
Setting, participants, intervention and measurement: New Choices is a centralized “one‐stop shop” in Hamilton, Ontario, Canada, that provides multiple services in a supportive environment for substance‐using women and their young children. Measures of substance use, social support, mental health, nutrition, parenting and child development were administered to 13 women (21–36 years old) and their 13 children (0–5 years old) when they entered the programme, and at 3 and 6 months thereafter. Findings: Results revealed a significant improvement in maternal empathy at 3 months and children's social competence at 6 months, and a decrease in maternal support of children's power and independence that was significant at 3 months but not at 6 months. Improvements in other outcomes were not statistically significant, but effect sizes were promising. Results showed more improvements after 6 than 3 months, suggesting that length of programme involvement may be an important factor in outcome. Conclusions: The findings from this small pilot study suggest that a “one‐stop” intervention for substance‐using mothers and their children might have a positive impact on many outcomes. This evaluation involved a small cohort sample, no comparison group and parent‐report measures only, so it should be viewed as exploratory and interpretation of findings made cautiously.
The Effectiveness of a Community-Based Intervention for Parents with FASD
Article
The purpose of this study was to evaluate the effectiveness of the Step by Step program in which mentors work with parents affected by Fetal Alcohol Spectrum Disorder (FASD) on a one-to-one basis. Mentors help clients identify and work towards meeting their needs and achieving their goals. Data from 24 closed client files was collected and analyzed and as predicted, the program was effective in helping clients reduce their needs and achieve their goals. The clients' reason for leaving the program as well as whether or not they had a formal FASD diagnosis had an impact on their success in the program. Data collected on additional mental health issues, experience of abuse and addictions helped to characterize the sample of clients and correlations were found between clients' experience of abuse and their past and/or present addictions issues. Limitations of this study as well as future implications were also discussed.
The Effectiveness of a Community-Based Intervention Program for Women At-Risk for Giving Birth to a Child with Fetal Alcohol Spectrum Disorder (FASD)
Article
The goal of this study was to determine whether the First Steps program (modeled after the Parent-Child Assistance Program) resulted in improved outcomes among women at-risk for giving birth to a child with FASD. We conducted a retrospective analysis of data on 70 participants in the First Steps program. Clients were high risk and faced many life challenges, including: being on welfare, substance abuse, physical and sexual abuse as children, mental health issues, criminal activity, and unplanned pregnancies. We found a significant increase in birth control use and decrease in welfare rates from pre- to post-program. At program exit, many participants were abstinent from alcohol and/or drugs and the majority did not experience a subsequent pregnancy. Clients also showed significant increases in goals and decreases in needs from pre-to post-program. The First Steps program demonstrated promising outcomes for women at-risk for giving birth to a child with FASD.
Basics of community-based evaluation
  • K Hutchinson
  • K Van Der Woerd
Hutchinson, K. & van der Woerd, K. (2010). Basics of community-based evaluation. Retrieved from: http:// ebookbrowse.com/caiwebinar-basics-of-community-based-evaluation-nov-16-2010-pdf-d36769698