No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Evaluation Standards
... In contrast to ethnographic methodology, evaluation methodology commonly comprises the use of both qualitative and quantitative techniques. Commonly defined as the systematic determination of something's merit, worth, or significance (Davidson, 2005; Sanders, 1994; Scriven, 1991). Scriven (1991) claims that evaluation is not only a methodology, but a distinct multi-and transdisciplinary field of study not to be regarded as merely a as a branch of applied social science. ...
... The main limitations of Guba and Lincoln's model is that it minimally acknowledges the fundamental role of evaluation in determining the merit, worth, significance, value, quality, or importance of the program, which are core elements within the definition of evaluation (see for examples Scriven, 1991; Davidson, 2005; and Sanders, 1994). Guba and Lincoln claim to offer a formative evaluation model placing little emphasis on making an evaluative conclusion and more on program improvement through consensus-building; however, to suggest improvement, they must determine deficits in the evaluand, thus they do actually evaluate. ...
Albeit some might argue that this review is a little late in coming, it is worth revisiting Pawson and Tilley’s 1997 book, Realistic Evaluation (reprinted in 1998, 2000, 2001, and 2002) as the debate about causation and evidence-based research and evaluation continues to be a topic of debate and concern in the evaluation and research communities (see A Call to Action: The First International Congress of Qualitative Inquiry and The Claremont Debate, in this issue of JMDE).
... Webster's New World Dictionary defines evaluation broadly as judging or determining the worth, value, or quality of an entity. The Joint Committee on Standards for Educational Evaluation in the United States of America (Sanders, 1994) defines evaluation as the systematic investigation of the worth or merit of an object (such as a project, program or policy). Carol Weiss (1998: 4) defines evaluation as the systematic assessment of the operation and/or the outcomes of a program or policy, compared to a set of explicit or implicit standards, as a means of contributing to the improvement of the program or policy. ...
... Gaebler, 1992; US General Accounting Office, 1993a Office, , 1993b Office, , 1994 Office, , 1995a Office, , 1995b Office, , 1997a Office, , 1997b Office, , 1997c Office, , 1997d Rossi and Freeman, 1994; Sanders, 1994; Friedlander and Burless,1994; National Governors' Association, 1994; Hatry, 1994 ; Leeuw and Rozendal , 1994; National Performance Review, 1994/1995; US Department of Labor, 1994–1996; Duran et al., 1995; Muller-Clemm, 1995; European Commission, 1995a, 1995b; Organization for Economic Cooperation and Development, 1995 Development, , 1997; Pawson and Tilley, 1997; Blalock, 1997; Scott, et al., 1997; Kettl, 1997; Pollit, 1998; Toulemonde et al., 1998. In a paper given at the International Evaluation Association conference in Vancouver , BC in 1994, Sue Funnel commented that Australia used a hierarchy of six stages of organization as a framework for studying continuous improvement, which also formed the basis for implementation studies. ...
The burgeoning performance management movement, with its emphasis on social program ‘results’ measured typically by a limited set of quantitative indicators, has developed a life of its own largely apart from the evaluation research movement. Reflecting the differences in the professional history, interests and training underlying the two movements, the relationship between these disparate approaches to establishing public accountability has lacked coordination and defied integration. This article discusses the basic concepts guiding the evolution of these movements in the context of the goals of information production, and explores the major conceptual, measurement and methodological problems resulting from the lack of accommodation between them. It also provides suggestions about how these two important approaches can be better integrated, both professionally and organizationally, for the purpose of enhancing the reliability and validity of social program assessments, and therefore for improving policy development and program management.
... Over the past 15 years, the development of codes of behaviour and practice has become a matter of concern in the evaluation profession Evaluators (American Evaluation Association [AEA], 1994). The Program Evaluation Standards (Sanders, 1994) are grouped into four areas as perFigure 2 below:Figure 2. Whilst the three standards summarised above are distinct, they are also complementary. Learning and developing the capacity to apply the different sets of standards selectively, with respect to particular programs and audiences, will enhance professionalism and versatility in conducting sound evaluations (Stufflebeam & Shinkfield, 2007). ...
... The presence of multiple objectives complicates the evaluation and priority-setting process but does not require that we abandon economic principles (page 503).'' In contrast, the broader professional evaluation community has moved away from discipline-based evaluations employing the ''single best method'' and now embraces mixed-method evaluation approaches in which decisions on methods follow efforts to understand user information requirements (Green and Caracelli, 1997; Sanders, 1994). Another important gap in evaluation practice in most agricultural research organizations is the lack of concern for institutional arrangements. ...
National agricultural research organizations as well as centres affiliated with the Consultative Group on International Agricultural Research (CGIAR) are under strong pressure to enhance their contributions to poverty alleviation, food security and protection of the environment and to demonstrate the results of their work. The dynamic world environment demands continuous changes in the ways in which research organizations operate and relate to other actors in agricultural innovation systems. To date, agricultural research evaluation has been dominated by economic impact assessment frameworks and methods. Designed as research studies, impact assessments have often served accountability and public awareness purposes. However, they have been of less use to further the understanding of how policies, programs and technologies may or may not contribute to agricultural innovation. There is an increasing awareness that appropriately designed and executed evaluations—going beyond traditional economic impact assessment—can contribute substantially to institutional learning and performance improvement. This article outlines the links between evaluation, learning and institutional change, and in doing so briefly summarizes the papers in this special issue of Agricultural Systems.
... low income) where families are perceived to be vulnerable (Portes et al. 1986a; Guralnick 1997; Aber et al. 2000). It is essential that the tools used to evaluate the outcomes of these interventions provide reliable, valid, and culturally appropriate assessments of parent–child interactions and provide normative data with which to make comparisons (Sanders 1994; Seideman et al . 1994; MacDonald-Clark & Harney-Boffman 1995 ). ...
Background:
Limited research is available on the performance of North American Aboriginal(1) people on the Nursing Child Assessment Teaching Scales (NCATS) and available research examines parent-child interactions within Aboriginal samples drawn from the USA. Most published normative data on low-income people's performance on the NCATS are also limited to samples drawn from the USA. The purpose of this research study was to use the NCATS measure to: (1) begin to develop a knowledge base that describes the parent-child interactions observed in Canadian low-income samples; and (2) explore the relationship between parent-infant interactions and ethnicity (Aboriginal or Non-Aboriginal) within a low-income Canadian sample.
Methods:
Secondary analysis was completed on data collected as part of a larger study designed to examine the impact of low-income situations on pre-school children's health and development in Edmonton, Alberta, Canada. The NCATS measure of parent-child interaction was administered to all children (1-36 months old) and their parents in the low-income non-probability sample. The sample derived for secondary analysis consists of 12 Aboriginal parent-child pairs (11 mothers and 1 father) and 48 Non-Aboriginal parent-child pairs (47 mothers and 1 father).
Results:
The data analysis suggests that although low-income Aboriginal parents may be less verbal with their children in interactions, the overall interaction quality is not different from that of other low-income parents. However, both groups' parent-child interaction scores were less than the published 10th percentile cut-off score, indicating 'worrisome scores' and less than optimal interactions.
Conclusion:
While the findings that compare the Aboriginal and Non-Aboriginal samples are limited by the small sample size, the fact that these findings agree with those from heterogeneous Aboriginal samples drawn from the USA are encouraging. Finally, the findings provide needed information about parent-child interactions in Canadian low-income urban samples including data from Aboriginal parents and children.
... Overall, evidence was considered valid only after contradictory information, spurious relations, and rival explanations were considered and subsequently accounted for (Sanders 1994). Unless otherwise stated, exemplars presented represent the conclusions drawn from this examination of the evidence. ...
This methodological study examines an original data collection model designed to incorporate human factors and enhance data richness in qualitative and evaluation research. Evidence supporting this model is drawn from in-depth youth and adult interviews in one of the largest policy/program evaluations undertaken in the United States, the Drug, Alcohol, and Tobacco Education evaluation (77 districts, 118 schools). When applying the explicit observation technique (EOT)--the strategic and nonjudgmental disclosure of nonverbal human factor cues by the interviewer to the respondent during interview--data revealed the observation disclosure pattern. Here, respondents linked perceptions with policy or program implementation or effectiveness evidence. Although more research is needed, it is concluded that the EOT yields richer data when compared with traditional semistructured interviews and, thus, holds promise to enhance qualitative and evaluation research methods. Validity and reliability as well as qualitative and evaluation research considerations are discussed.
... The final report analyzed the project against widely accepted program evaluation standards[8]. These standards treat multiple factors related to utility, feasibility, propriety, and accuracy. ...
Engineering Criteria 2000 requires faculty to define objectives for engineering programs, define and implement assessment processes to measure student knowledge and skill, and identify processes where feedback is used to achieve continuous quality improvement of instruction and learning. As part of the preparation for an accreditation visit under the new criteria, faculty in the Mechanical Engineering Department at the University of Idaho partnered with doctoral students in the Assessment and Evaluation Center at Washington State University to review the technical elective component of the undergraduate curriculum. This paper describes the process undertaken by the clients and evaluators in designing, implementing, and conducting the evaluation.
Die Novellierung des Hochschulrahmengesetzes hat eine Stärkung des Wettbewerbs zwischen den Bildungseinrichtungen zur Folge. Eine Konsequenz des zunehmenden Wettbewerbs ist die steigende Bedeutung von Evaluation und Akkreditierung in der deutschen Hochschullandschaft. Damit gewinnt das Qualitätsmanagement an universitären Einrichtungen an Wichtigkeit.
Bisher existiert in Deutschland kein etabliertes Modell zur Qualitätsmessung und -verbesserung in Hochschulen. In vorliegendem Beitrag wird ein Instrument — das AACSB International (The Association to Advance Collegiate Schools of Business)-Modell — zur Messung der Qualität von Fakultäten vorgestellt, das über eine bloße Evaluation von Studium und Lehre hinausgeht.
Die Fakultät für Betriebswirtschaftslehre der Universität Mannheim durchlief den Prozess der Leistungsbeurteilung und Akkreditierung durch AACSB. Die Autoren in ihrer Funktion als Prüfungsausschussvorsitzender, Vorsitzender der Strategiekommission, Prorektor und Projektbetreuerin waren Mitglieder der Akkreditierungskommission, welche die Fakultät in eineinhalbjähriger Arbeit auf die Akkreditierung vorbereitete und erfolgreich abschloss.
Neben der Vorstellung und Erläuterung des Modells zur Messung der Qualität von wirtschaftswissenschaftlichen Fakultäten besteht ein sekundäres Ziel dieses Beitrags in der informatorischen Begleitung des Wandels der deutschen Hochschullandschaft.
The main goal of the paper that is called "Comparison of Selected Evaluation Approaches" is to show the variety of how evaluators could solve the evaluation tasks. Results of the performed analysis present how uneven the outcomes of evaluation could be when different approach is taken. The article turns attention toward the basics of evaluative thinking by opening the range of alternative methodological and conceptual frameworks that could be used in the course of intervention program evaluation.
Evaluation is defined with use of several definitions that put emphasis on methodological aspects of evaluative inquiry. In detail, there are four different approaches compared: goal-based, goal-free, constructivists and theory-driven. Each of the approaches is briefly introduced, context of its dissemination is explained and it is also described how the approach relates to other approaches. Useful component of the analysis is an example that highlights the main distinctions among the four approaches and results that each of the approaches can offer.
Im Leistungssport wird Techniktraining täglich geplant und bewertet. Gerade die Änderungen in Regeln oder Technikentwicklungen machen Tischtennis zu einem interessanten Anwendungsfeld, wie ein Techniktraining verbessert werden kann. Der Fokus der vorliegenden Arbeit lag nicht nur auf den einzelnen Techniken, sondern ebenfalls darauf wie Übergänge zwischen den Techniken selbst optimiert werden können. Dazu wurde eine Diagnostik für die Techniken und Übergänge (Tischtennis Technikwechsel Leistungsdiagnostik) entwickelt, geprüft und verbessert. Aus den Ergebnissen der Diagnostik wurde eine Intervention der Techniken konzipiert (Phase I), die anschließend selbst- und fremdevaluiert wurde (Phase II). Die Erfahrungen der Intervention der Techniken und deren Evaluation wurden benutzt, um in einer Phase III eine zielgerichtete Trainingsmaßnahme zur Optimierung der Technikübergänge zu gestalten und zu evaluieren. Die Verbesserungen der Interventionsstichprobe gegenüber einer ebenfalls trainierenden Evaluationsstichprobe in kinematischen Analysen der Bewegungen und Trefferleistungen bestätigten die Hypothesen zu einer verbesserten Erreichung der Sollwertvorstellung. Trainingswettkämpfe sowie kurz- und langfristige videobasierte Wettkampfanalysen kamen zu dem Ergebnis, dass die Trainingseffekte auch die Wettkampfleistungen langfristig beeinflussen. Über die konkreten Fragestellungen des Forschungsprojektes hinaus konnten das theoretische Verständnis für sensomotorische Kontrollprozesse bei schnellen Bewegungen sowie methodologische Konsequenzen von langfristigen Evaluationsstudien erarbeitet werden. Ein Praxisteil mit Empfehlungen für die Planung von konkreten Trainingsinhalten schließt die Arbeit ab.
To address the need for women's health education by designing, implementing, and evaluating a self-study, web-based women's health curriculum.
Cohort of students enrolled in the ambulatory portion of the medicine clerkship with comparison group of students who had not yet completed this rotation.
Third- and fourth-year medical students on the required medicine clerkship (115 students completed the curriculum; 158 completed patient-related logs).
Following an extensive needs assessment and formulation of competencies and objectives, we developed a web-based women's health curriculum completed during the ambulatory portion of the medicine clerkship. The modules were case based and included web links, references, and immediate feedback on posttesting. We discuss technical issues with implementation and maintenance.
We evaluated this curriculum using anonymous questionnaires, open-ended narrative comments, online multiple-choice tests, and personal digital assistant (PDA) logs of patient-related discussions of women's health. Students completing the curriculum valued learning women's health, preferred this self-directed learning over lecture, scored highly on knowledge tests, and were involved in more and higher-level discussions of women's health with faculty (P<.001).
We present a model for the systematic design of a web-based women's health curriculum as part of a medicine clerkship. The web-based instruction resolved barriers associated with limited curriculum time and faculty availability, provided an accessible and standard curriculum, and met the needs of adult learners (with their motivation to learn topics they value and apply this knowledge in their daily work). We hypothesize that our web-based curriculum spurred students to later discuss these topics with faculty. Web-based learning may be particularly suited for women's health because of its multidisciplinary nature and need for vertical integration throughout medical school curricula.
Professional associations of evaluators have recently devoted much time and energy to the development of codes of behaviour to guide the work of evaluators. However, there is a body of anecdotal evidence that many evaluations are carried out by practitioners without reference to any code of practice. This paper sets out a training strategy we have used to encourage evaluators to make sense of codes of behaviour, and to incorporate them into their work. By grouping standards into broad categories evaluators, and those commissioning evaluations, can concentrate on the relevant aspects for the particular stage of the evaluation.
ResearchGate has not been able to resolve any references for this publication.