Evaluation and program planning

Publisher: Elsevier

Current impact factor: 0.89

Impact Factor Rankings

Additional details

5-year impact 0.99
Cited half-life >10.0
Immediacy index 0.07
Eigenfactor 0.00
Article influence 0.34
ISSN 1873-7870

Publisher details

Elsevier

  • Pre-print
    • Author can archive a pre-print version
  • Post-print
    • Author can archive a post-print version
  • Conditions
    • Authors pre-print on any website, including arXiv and RePEC
    • Author's post-print on author's personal website immediately
    • Author's post-print on open access repository after an embargo period of between 12 months and 48 months
    • Permitted deposit due to Funding Body, Institutional and Governmental policy or mandate, may be required to comply with embargo periods of 12 months to 48 months
    • Author's post-print may be used to update arXiv and RepEC
    • Publisher's version/PDF cannot be used
    • Must link to publisher version with DOI
    • Author's post-print must be released with a Creative Commons Attribution Non-Commercial No Derivatives License
    • Publisher last reviewed on 03/06/2015
  • Classification
    green

Publications in this journal

  • [Show abstract] [Hide abstract]
    ABSTRACT: This article presents a framework to identify key mechanisms for developing a logic model blueprint that can be used for an impending comprehensive evaluation of an undergraduate degree program in a Canadian university. The evaluation is a requirement of a comprehensive quality assurance process mandated by the university. A modified RUFDATA (Saunders, 2000) evaluation model is applied as an initiating framework to assist in decision making to provide a guide for conceptualizing a logic model for the quality assurance process. This article will show how an educational evaluation is strengthened by employing a RUFDATA reflective process in exploring key elements of the evaluation process, and then translating this information into a logic model format that could serve to offer a more focussed pathway for the quality assurance activities. Using preliminary program evaluation data from two key stakeholders of the undergraduate program as well as an audit of the curriculum's course syllabi, a case is made for, (1) the importance of inclusivity of key stakeholders participation in the design of the evaluation process to enrich the authenticity and accuracy of program participants' feedback, and (2) the diversification of data collection methods to ensure that stakeholders' narrative feedback is given ample exposure. It is suggested that the modified RUFDATA/logic model framework be applied to all academic programs at the university undergoing the quality assurance process at the same time so that economies of scale may be realized.
    No preview · Article · Apr 2016 · Evaluation and program planning
  • [Show abstract] [Hide abstract]
    ABSTRACT: While there is considerable support among the policy, practice and academic communities for the idea that practice in youth work should be based on evidence, Veerman and Van Yperen (2007) highlight the fact that little practice in the area of youth care is actually evidence based due to the paucity of experimental studies in the field and the difficulties associated with evaluation of non-standardised interventions such as youth work. They propose a model consisting of levels of evidence, the aim of which is to ‘bridge the gap between evidence based and evidence informed practice’ (p. 218). In order to address the dearth of examples in the international literature of how organisations have dealt with these complexities in practice, this article provides a case study of how a large Irish youth organisation, Foróige, has endeavoured to adopt an inclusive approach to evidence, working towards both evidence based and evidence informed practice. Drawing on Veerman and Van Yperen’s (2007) framework, the case study shows that Foróige committed itself to gathering different types of evidence, including logic models and theoretical understandings at the foundation level and experimental design studies of key programmes at the advanced level. The case study focuses in detail on the intermediate level, whereby practitioners were trained to evaluate aspects of their own work, with a view to embedding evaluation and reflection in day to day practice. The process is described and key lessons emerging are discussed.
    No preview · Article · Jan 2016 · Evaluation and program planning
  • [Show abstract] [Hide abstract]
    ABSTRACT: Systematic triangulation may address common challenges in evaluation, such as the scarcity or unreliability of data, or the complexities of comparing and cross-checking evidence from diverse disciplines. Used to identify key evaluation findings, its application has proven to be effective in addressing the limitations encountered in country-level evaluation analysis conducted by the Independent Evaluation Office of the Global Environment Facility (GEF). These include the scarcity or unreliability of national statistics on environmental indicators and data series, especially in Least Developed Countries; challenges in evaluating the impacts of GEF projects; and inherent difficulties in defining the GEF portfolio of projects prior to the undertaking of the evaluation. In addition to responding to the need for further developing triangulation protocols, procedures and/or methodologies advocated by some authors, the approach offers a contribution to evaluation practice. This applies particularly to those evaluation units tasked with country-level evaluations in international organizations, facing similar constraints.
    No preview · Article · Jan 2016 · Evaluation and program planning
  • [Show abstract] [Hide abstract]
    ABSTRACT: Preconception health programs have resulted in improved health behaviors among participants and have shown promise in reducing adverse birth outcomes. However, the role of health beliefs in preconception health program outcomes has been overlooked but warrants attention due to reported positive associations between women's views of control over fetal health and health behavior in pregnancy. Towards an ultimate aim of improving preconception health program reach and effectiveness, we examined properties of a fetal health locus of control (FHLC; Labs & Wurtele, 1986) measure in nulliparous, childbearing aged university women and men. Students (n=1467) completed an online survey that included the FHLC subscales maternal, powerful others', and chance control over fetal health. Factor analyses and correlations with related scales supported the soundness of FHLC constructs in both women and men. All participants rated maternal control in fetal health nearly twice as highly as powerful others' and chance. We therefore recommend that FHLC be integrated into preconception health program evaluation as personal agency in fetal health likely has an important role in women's and men's preconception health behavior and health behavior change.
    No preview · Article · Dec 2015 · Evaluation and program planning
  • [Show abstract] [Hide abstract]
    ABSTRACT: Exposure to secondhand smoke occurs primarily in the home due to passage of smoke-free legislation. Creation of a total household smoking ban can reduce associated health conditions such as asthma, lung cancer, heart disease and stroke. This paper describes the results of a randomized control trial of a minimal intervention to create smoke-free homes. 2-1-1 callers were invited to participate in the trial and were randomized to an intervention (mailings and a coaching call) or a control group (no intervention). We assessed reach, dose, fidelity, and receptivity to the intervention through program records and a 3-month follow-up survey with intervention participants. For the intervention materials, materials were mailed to 244 participants (99.2%) and 227 participants (92.3%) received the coaching call intervention. 92.3% received all intervention components. Participants who had full household bans at 3 months were more likely to conduct behaviors leading to a smoke-free home (i.e., making a list of reasons, having a family talk, posting a pledge) than were those with no/partial ban. The intervention materials also were rated higher in relevance and usefulness by non-smokers than smokers. Results demonstrate that this minimal intervention had high fidelity to the delivery of components and relatively high receptivity.
    No preview · Article · Dec 2015 · Evaluation and program planning
  • [Show abstract] [Hide abstract]
    ABSTRACT: This paper describes the approach and process undertaken to develop evaluation capacity among the leaders of a federally funded undergraduate research program. An evaluation toolkit was developed for Computer and Information Sciences and Engineering. 11Computer and Information Sciences and Engineering, CISE. Research Experiences for Undergraduates. 22Research Experiences for Undergraduates, REU. (CISE REU) programs to address the ongoing need for evaluation capacity among principal investigators who manage program evaluation. The toolkit was the result of collaboration within the CISE REU community with the purpose being to provide targeted instructional resources and tools for quality program evaluation. Challenges were to balance the desire for standardized assessment with the responsibility to account for individual program contexts. Toolkit contents included instructional materials about evaluation practice, a standardized applicant management tool, and a modulated outcomes measure. Resulting benefits from toolkit deployment were having cost effective, sustainable evaluation tools, a community evaluation forum, and aggregate measurement of key program outcomes for the national program. Lessons learned included the imperative of understanding the evaluation context, engaging stakeholders, and building stakeholder trust. Results from project measures are presented along with a discussion of guidelines for facilitating evaluation capacity building that will serve a variety of contexts.
    No preview · Article · Dec 2015 · Evaluation and program planning
  • [Show abstract] [Hide abstract]
    ABSTRACT: Minority racial and ethnic groups are at higher risk for developing type 2 diabetes. These groups also experience more severe complications from diabetes and have higher mortality rates as a result of the disease, such as cardiovascular disease, amputation and kidney failure. Underserved rural ethnically disparate populations benefit from health education outreach efforts that are conveyed and translated by specially-trained community health ambassadors. Project H.I.G.H. (Helping Individuals Get Healthy) was developed to target the priority areas of type 2 diabetes and cardiovascular disease. Utilizing trained community health ambassadors, CDC's The Road to Health Toolkit as well as New Beginnings: A Discussion Guide for Living Well with Diabetes was used as a model for a community-based educational program. The overall goal of Project H.I.G.H was to implement and evaluate: (1) a coordinated, behavior-focused, family-centered, community-based educational program and (2) a client service coordination effort resulting in improved health outcomes (BMI, Glucose Levels, BP) for individuals with type 2 diabetes and cardiovascular disease in Gadsden County, Florida. Overall, Project H.I.G.H. was very successful in its first year at motivating participants to delay or prevent diabetes and/or cardiovascular disease or at the very least to start taking better care of their health.
    No preview · Article · Dec 2015 · Evaluation and program planning
  • [Show abstract] [Hide abstract]
    ABSTRACT: Information on costs of programs addressing community integration for persons with serious mental illness in the United States, essential for program planning and evaluation, is largely lacking. To address this knowledge gap, community integration programs identified through directories and snowball sampling were sent an online survey addressing program costs and organizational attributes. 64 Responses were received for which annual per person costs (APPC) could be computed. Programs were categorized by type of services provided. Program types differed in median APPCs, though median APPCs identified were consistent with the ranges identified in the limited literature available. Multiple regression was used to identify organizational variables underlying APPCs such as psychosocial rehabilitation program type, provision of EBPs, number of volunteers, and percentage of budget spent on direct care staff, though effects sizes were moderate at best. This study adds tentative prices to the menu of community integration programs, and the implications of these findings for choosing, designing and evaluating programs addressing community integration are discussed.
    No preview · Article · Nov 2015 · Evaluation and program planning
  • [Show abstract] [Hide abstract]
    ABSTRACT: Maintaining contact with and collecting outcome data from adolescent study participants can present a significant challenge for researchers conducting longitudinal studies. Establishing an organized and effective protocol for participant follow-up is crucial to reduce attrition and maintain high retention rates. This paper describes our methods in using and adapting the evidence-based Engagement, Verification, Maintenance, and Confirmation (EVMC) model to follow up with adolescents 6 and 12 months after implementation of a health program. It extends previous research by focusing on two key modifications to the model: (1) the central role of cell phones and texting to maintain contact with study participants throughout the EVMC process and, (2) use of responsive two-way communication between staff and participants and flexible administration modes and methods in the confirmation phase to ensure that busy teens not only respond to contacts, but also complete data collection. These strategies have resulted in high overall retention rates (87-91%) with adolescent study participants at each follow-up data collection point without the utilization of other, more involved tracking measures. The methods and findings presented may be valuable for other researchers with limited resources planning for or engaged in collecting follow-up outcome data from adolescents enrolled in longitudinal studies.
    No preview · Article · Nov 2015 · Evaluation and program planning
  • [Show abstract] [Hide abstract]
    ABSTRACT: This study was designed to evaluate whether railway safety lessons are effective in increasing schoolchildren's safety knowledge and behaviour intention. The railway safety education in schools included a 45-min lesson on safe behaviour in a railway environment directed at 8-11 year old schoolchildren. The lessons were held in four schools located near railway lines in Finland. The effectiveness of this measure was evaluated based on a short survey directed at pupils before the lesson (base level) and around 2-3 months later (post-lesson) based on three variables which are considered as strong determinants of actual behaviour: behaviour intention, estimated dangerousness of the behaviour, and level of knowledge on the legality of the behaviour. The results show that the change in the share of correct answers was positive regarding all questions except for one question in which the difference was not significant. Based on this we can reasonably assume that railway safety education in schools can have a positive effect for all the measured variables, although the effect is not necessarily large. The results indicate that these positive changes can have a positive effect on the frequency of trespassing (i.e. fewer unsafe crossings in the future). We can further assume that reduction in the frequency of trespassing would reduce the frequency of trespassing accidents.
    No preview · Article · Nov 2015 · Evaluation and program planning
  • [Show abstract] [Hide abstract]
    ABSTRACT: Background: Cancer registration data is vital for creating evidence-based policies and interventions. Quantifying the resources needed for cancer registration activities and identifying potential efficiencies are critically important to ensure sustainability of cancer registry operations. Methods: Using a previously validated web-based cost assessment tool, we collected activity-based cost data and report findings using 3 years of data from 40 National Program of Cancer Registry grantees. We stratified registries by volume: low-volume included fewer than 10,000 cases, medium-volume included 10,000-50,000 cases, and high-volume included >50,000 cases. Results: Low-volume cancer registries incurred an average of $93.11 to report a case (without in-kind contributions) compared with $27.70 incurred by high-volume registries. Across all registries, the highest cost per case was incurred for data collection and abstraction ($8.33), management ($6.86), and administration ($4.99). Low- and medium-volume registries have higher costs than high-volume registries for all key activities. Conclusions: Some cost differences by volume can be explained by the large fixed costs required for administering and performing registration activities, but other reasons may include the quality of the data initially submitted to the registries from reporting sources such as hospitals and pathology laboratories. Automation or efficiency improvements in data collection can potentially reduce overall costs.
    No preview · Article · Nov 2015 · Evaluation and program planning
  • [Show abstract] [Hide abstract]
    ABSTRACT: Despite the long history of immigration in the United States, communities around the country struggle to integrate newcomers into the economic, cultural, and political spheres of society. Utilizing results from the program evaluation of one public library's Cultural Navigator Program, the authors illustrate how communities and public institutions can promote integration and relationship-building between newly arrived immigrants and long-time residents. Existing social networks within receiving communities, conceptualized in this article as social capital, were leveraged to build capacity among newly arrived immigrants and foster inclusivity and integration at the community level. As a place of intervention, public libraries are suggested as a safe and shared space where community integration can be fostered. Insights derived from the evaluation inform a discussion on engaging approaches to immigrant integration. Lessons learned and recommendations for program evaluators and administrators are provided.
    No preview · Article · Nov 2015 · Evaluation and program planning
  • [Show abstract] [Hide abstract]
    ABSTRACT: Families facing mental health challenges have very limited access to ongoing support. A formative evaluation of Families Healing Together (FHT), a new online family mental health recovery program was conducted using five waves (N= 108) of data. Exploratory factor analysis of the measures identified as important to the program theory found strong reliability evidence (α= .77-.86) for 6 constructs. A poor response rate (25%) did not allow for valid pre and postoutcome evaluation, however we did have enough information to assess the psychometric properties of the new measures. The new evaluation tool accounted for 34% of the variance in Capacity to Support Family Member, and nearly 50% of the variance in Hopefulness toward Recovery. New programs without existing measures require formative evaluation strategies that accurately describe program activities in order to develop outcome measures sensitive to novel aspects of program components. Most outcome measures are developed for individuals with mental health challenges not family members. These new measures may be beneficial to effectively evaluate programs that promote family recovery and wellness.
    No preview · Article · Nov 2015 · Evaluation and program planning
  • [Show abstract] [Hide abstract]
    ABSTRACT: This article is part of a long term project "Promoting the Occupational Well-Being of School Staff-Action Research Project in Finland and Estonia, 2009-2014." The purpose of this article is to describe the significance of action plans in the promotion of the occupational well-being of primary and upper secondary school staff in Finland and Estonia from 2010 to the turn of the year 2011-2012. An electronic open questionnaire was sent to occupational well-being groups in Finland (N=18) and in Estonia (N=39). In Finland, the questionnaire was responded to by 16 (n=16) occupational well-being groups, and in Estonia, by 38 (n=38) groups. The qualitative data were analyzed using the inductive-deductive method and content analysis. The obtained results indicate that the schools had named goals for action plans in all aspects of the promotion of occupational well-being in schools (worker and work, working conditions, professional competence, working community) and that these goals were mainly realized in the schools in a systematic way. Schools felt that the action plan for occupational well-being helped them to set goals for occupational well-being and that the planned actions were realized in a more systematic way than before.
    No preview · Article · Oct 2015 · Evaluation and program planning
  • [Show abstract] [Hide abstract]
    ABSTRACT: In this study, a utilization-focused process evaluation framework was used to explore client and service providers' experiences of Crisis Management Services (CMS), their perceptions of the services provided, and the process of client engagement CMS offers. CMS is a strength-based program that targets individuals who experience crises every day. The Community-University Institute for Social Research facilitated the evaluation by engaging academic expertise in a coordinated collaborative approach to community-university partnerships. Semi-structured interviews were conducted with the clients and service providers. The general inductive approach was used for transcript analysis with seven themes emerging. A conceptual model of service delivery is presented, which integrates the interviews conducted with clients and service providers. Results affirm that the establishment of a close personal strength-based relationship is key to client engagement. Collaborative goal setting with informal and formal community resources viewed as potential assets, characterizes the process that enables clients to live at their optimal level of independence. This study is unique as it provides valuable insight on the perspectives of vulnerable individuals in crisis situations. Through the establishment of community-university partnerships the gap between scholarly research and its applicability to community organizations is narrowed with opportunities for improving the quality of life enhanced.
    No preview · Article · Oct 2015 · Evaluation and program planning