PresentationPDF Available

The Role of Evaluation in Curriculum Design

Authors:

Abstract and Figures

The Role of Evaluation in Curriculum Design
No caption available
… 
No caption available
… 
No caption available
… 
No caption available
… 
Content may be subject to copyright.
THE ROLE OF EVALUATION IN CURRICULUM DESIGN Seyed Ehsan Afsahi
October 2016
Objectives:
1. Define evaluation and describe its value to a curriculum.
2. Describe the relationship between needs assessment, objectives and evaluation.
3. Assure that the evaluation process measures the appropriate domains of learning (cognitive,
affective and psychomotor).
4. Distinguish between feedback and evaluation.
5. Contrast norm-referenced and criterion-referenced evaluation.
6. Describe the three approaches to evaluation (Learner, Instruction and Program).
7. Discuss the advantages and disadvantages of various evaluation strategies for the learner and
program.
8. Prepare an evaluation plan, including appropriate instruments, for an educational curriculum.
Goal
We should be able to understand the basic elements of evaluation in the curriculum development
and design process and utilize that information to prepare a new curriculum.
Review
Let's review
where we are:
We've decided what we intend
to cover (goals and objectives),
and how we plan to do it
(teaching/learning strategies).
Now we'll decide how to
measure whether our plan
works.
"If we wish to discover the truth about an educational system, we
must look into its assessment procedures...The spirit and style of
student assessment defines the defacto curriculum."
D. Rowntree. Assessing Students: How Shall We Know Them
Evaluation is defined as "determining the value of ....“
In the context of an educational curriculum, evaluation is the determination of the value of
learning, instruction and the overall program.
In most educational settings, evaluation traditionally means assessing the learner to see if they
met a particular standard. In reality, there are a variety of benefactors from evaluation.
The Definition of Evaluation
1.The learner. Did the learner improve during the educational process? Did the learner meet a
certain standard? Depending on the curriculum objectives, did the learner meet those standards set
for the curriculum?
2.The instruction. Did the instruction, or faculty meet expectations of the learner and the overall
program?
3.The overall program. Did the program, or curriculum content/format, accomplish what it set out to
do?
Why Is Evaluation Important?
In a curriculum, evaluation could be used to provide the following value:
1. Feedback to all stakeholders T F
2. Improvement of the educational process T F
3. Certification of learner competency T F
4. Data to meet accreditation requirements T F
5. Assessment of the cost of delivery T F
All of the items are true. Although each evaluation tool may not ask those specific questions, overall data
from the evaluation process could contribute value to each item in the curriculum.
"Participants in the curriculum's experience have an interest in the evaluation of their own performance
and in the performance of the curriculum. Evaluation can provide feedback and motivation for the
continued improvement for learners, faculty, and curriculum developers." Kern, Thomas, Howard and
Bass, Curriculum Development for Medical Education: A Six-step Approach.
Kern, et al, points out further that evaluation has the following purposes:
1. Determining a learners' accomplishments and confirming competence which leads to a grade, or
certification.
2. Measuring improvement in the learners, instruction (faculty) and program.
3. Meeting accreditation standards by measuring student and program performance.
4. Determining the worth of the curriculum to the institution.
In essence, evaluation is an important component of the curriculum design process for the learner, the
faculty and the overall program. How do you use evaluation in the curriculum process?
Formative Evaluation
This type of evaluation occurs
during
the educational process with the intent or
improving performance, often referred to as "feedback."
Formative Evaluation
means to assess the educational process while it is still
being used or developed. It can provide reassurance and reinforcement to the
individual, faculty member, or program. Its intent is
to improve performance.
Evaluation can be used in different ways - during, at the conclusion of, or some time after the
educational process. The terms that reflect these types of educational evaluation are the
following:
This module will focus primarily on summative evaluation.
These are the types of evaluations that are used in an educational curriculum, but what do you
evaluate?
Types of Evaluation
Summative Evaluation
This type of evaluation occurs at the
conclusion
of an educational activity with
the intent of documenting achievement or competence. For the purposes of
this module, the term "evaluation" will refer to this type
.
Summative Evaluation
means to make a judgment, or decision, at the
conclusion of the educational curriculum. It is a capstone assessment, the final
outcome or a certification of competency at completion. Summative
evaluation can be used for an individual learner, faculty member, or program.
Follow
-up Evaluation
This type of evaluation occurs sometime
after
an educational activity, with the
intent of determining whether the learner has applied the knowledge/skill in
practice
.
Follow
-up Evaluation means to measure the result/application of the
educational intervention at a future time
. It is a type of evaluation that
determines whether the learner has applied the knowledge/skill in practice.
Most often this is self
-report and may reflect long-term satisfaction.
Three Major Levels of Curriculum Evaluation
As noted earlier, there are three levels of evaluation in a curriculum:
The learner
The instruction (faculty)
The program
While these are the main levels of evaluation, keep in mind that you can use the formative, summative,
and follow-up evaluation processes with each. It is possible to use all three evaluation processes with the
same educational curriculum. The table below illustrates the relationships between the levels and the
uses.
Process
The instructional method/s used by the educational program or curriculum could
be assessed for its effectiveness. During, or at the conclusion of educational
program, the student and/or faculty could review how the method of instruction or
administration was implemented.
Outcome
The results of the educational program can be assessed for its effectiveness. How
effective is the program in achieving its educational objectives?
What do you Evaluate?
Evaluation can focus on different aspects of the educational program.
The main focus of this module is on the measurement of outcomes. For example, if the curriculum
objectives expect the learner to be able to conduct CPR at the end of the course, you would assess
the student's skill in performing CPR on a mannequin.
There are a number of outcomes from an educational intervention that could be measured.
Kirkpatrick describes a continuum of four different outcome levels from an educational process:
Satisfaction,
Learning (also referred to as competence),
Behavior, and
Societal outcomes
Usually the level of outcome measured is determined by the curriculum objectives. So in planning the
evaluation, the curriculum developer should review the curriculum goals and objectives to be sure
that the evaluation addresses the outcome level.
Pearl: An effective curriculum uses formative and summative evaluation to provide feedback to the
learner for improvement during the learning process and to make judgments or decisions about the
learner at the end of the process which will contribute to the ongoing improvement of the program.
Effective Evaluation can measure the process, or methods, and outcomes, or results of a curriculum. What criteria do
you use to measure the process, or outcomes?
How Do You Measure Process and Outcomes?
Evaluation data can be collected in two ways:
1.Norm-referenced Evaluation - A procedure designed to determine a student's performance in relation to the
performance of other students on the same test - "grading on a curve". These types of evaluations are usually
used that the conclusion of an educational intervention to provide a grade.
Example: At the completion of a course in the curriculum, the student takes written examination and is judged on
the basis of how he/she compares with other students who have taken the same examination. This reflects a
"relative standard" of performance of how that student has performed compared to other students on this
examination. The student that answers the most questions would receive the top grade, whether or not they have
accomplished the objectives for the course.
2.Criterion-referenced Evaluation - A procedure designed to determine whether a student has achieved an
established or selected standard as specified in the instructional objective(s). These types of evaluations can be
used as pre-tests, to assess a student's progress during the curriculum and/or, at the end of the educational
intervention, to assess whether the students have accomplished the specific objectives of the curriculum.
Example: A written evaluation of the knowledge and an observable demonstration of the behavior of the student
are evaluated at the completion of a cardio-pulmonary resuscitation (CPR) course based upon the specific
objectives of that curriculum. If the student does not meet the objective standards regardless of whether he/she is
the best of the group, the student does not pass, or is not certified to perform CPR.
Pearl: Norm-referenced Evaluation compares a learner's performance with other learners. Criterion-
referenced Evaluation compares learner performance with a standard (criterion). Criterion-
referenced Evaluation is best in medical education because mastery is the key.
Parts of an Evaluation Plan
An evaluation plan for a curriculum should have three parts:
Evaluation of learner performance
Evaluation of faculty/instruction
Program evaluation
Evaluation of Learner Performance
Much of what we think of as evaluation is assessment of learner performance. The intent here is to
measure whether/how well learners have achieved the education objectives identified for the program.
For example, the objectives of the curriculum might expect the learner to know the physiology of the
heart. The written examination could ask questions about the aorta and its function.
There are many methods that can be used to measure learner performance
Pearl: When preparing the evaluation, make sure that it measures the objectives for the course, or
curriculum. If you are teaching knowledge, evaluate knowledge. If you are teaching a skill, evaluate
the skill.
Evaluation of Faculty
Good Faculty are life long learners. In order to stay current and continually improve, they need
assessment of their own performance as teachers. In addition these data may provide value in terms of
promotion, since teaching is being considered more frequently as a scholarly activity.
These evaluations can be done by
Learners - students are often asked to complete a questionnaire at the end of a session, or course.
Learner satisfaction data can be helpful for faculty and program improvement but should not be relied
upon exclusively, as what/who students like do not always reflect who was the most effective in
facilitating their learning.
Peers - Observation and feedback by other faculty are very useful to incorporate into faculty
assessment, when resources allow.
Below are some examples of questions that might be asked in a faculty evaluation process:
Did the faculty member address the goals and objectives of the course?
Did the faculty member answer questions raised by the learner?
Was the format of the instruction relevant to the content and course objectives?
Were the topics presented in the course relevant to you as a learner?
Did the faculty member involve the learners in the educational process?
A formative, or summative, evaluation format may be used for instructional evaluation. Summative is the
most commonly used for learners, since they are more likely to give frank and helpful answers more
comfortably after the course is completed.
Evaluation of the Program
Program evaluation is designed to assess the effectiveness of the curriculum as a whole.
Program evaluation can determine
whether the curriculum was effective in meeting its goals and objectives,
whether the faculty members were effective as learning facilitators,
whether the learners viewed the curriculum content and process as a value to their learning,
cost effectiveness of the curriculum,
fit with other curriculum components of the school and/or
compliance with external requirements.
The data may include
aggregated learner evaluations about the instruction and the overall program in meeting their needs,
faculty evaluations about the needs of the program, goals and objectives of the curriculum, and
content and process of instruction,
learner performance data , such as application of the instructional material in the clinical setting,
outcome data on learner achievements, such as results of the application of the learning in
publications and/or clinical practice and
costs to offer the curriculum including the instructional material, support and faculty time.
Your Evaluation Plan
Think about the curriculum area you are working on. Look at your objectives and think about how you
would evaluate them.
How would you assess learner achievement of these objectives?
How would you measure your faculty’s effectiveness in teaching the material?
How would you determine if your program as a whole was successful in achieving its objectives?
Summary of Evaluation
Evaluation is one part of the circular educational process that measures whether the educational intervention meets its goals and
objectives.
Evaluation is the determination of the value of learning, instruction and
the overall program of a curriculum.
Evaluation can be done during (formative), or at the conclusion of the
educational process(summative), or at a future time (follow-up) that
determines the interventions success.
There are three levels of curriculum evaluation: the learner, the
instruction, the overall program.
Learner evaluation can use norm-referenced (comparison with other
students), or criteria referenced (comparison with a standard), evaluation
measures to assess different outcomes for various domains of learning.
Ideally, evaluation systems should include formative and summative
methods.
Multiple stakeholders should be involved in the program evaluation
process.
Four Models of Curricular Change
Model 1: Hotchpoch
Model 2: Follow the Leader
Model 3: Window dressing
Model 4: Bureaucratic top-down
Four Twentieth Century Theoretical Models of Curricular Change
Tyler's four-stage progression model (1949)
1. Objectives: What educational purpose should the school seek to attain?
2. Content: What educational experiences can be provided that are likely to attain the objectives?
3. Organization: How can these educational experiences be effectively organized?
4. Evaluation: How can it be determined whether the objectives are being attained?
Tyler's Basic principles of Curriculum & Instruction was regarded by many educators as the early bible of
curriculum design. In it, he provided a workable model for the systematic development of curricula.
However, it was criticized as being an archetypal 'ballistic' model; that is, curriculum development activity occurs
in a linear series of sequential stages. Also, it was criticized as representing an "ends-means" view of education.
Taba's model (1962)
Step 1: diagnosis of needs
Step 2: formulation of objectives
Step 3: selection of content
Step 4: organization of content
Step 5: selection of learning experiences
Step 6: organization of learning experiences
Step 7: determination of what to evaluate & means to evaluate.
In this early standard model of the curriculum process, there is a systematic movement through a series of
set steps.
However, its rigidity is its chief weakness.
This "means-ends" or objective model has proved a popular framework for developing language curricula
programs, given that teaching itself is usually seen in these terms.
This model is often referred to as the Rational Planning Model because it is "rational" to the specify ends
before engaging in any given activity.
A hybrid Taba-Tyler model
Nunan's model (1985)
References
Alderson, J. C., & Beretta A. (Eds.). (1992). Evaluating second language education. Cambridge: Cambridge University Press.
Beretta, A. (1992). Evaluation of language education: An overview. In J. Alderson and A. Beretta (Eds.). (1992). Evaluating second language education (pp. 5-24).
Cambridge: Cambridge University Press.
Beretta, A. (1990). The program evaluator: The ESL researcher without portfolio. Applied Linguistics, 11, 1, 1-14.
Beretta, A. (1986). Program-fair language teaching evaluation. TESOL Quarterly, 20 (3), 431-444.
Brown, J. D. (1989). Language program evaluation: A synthesis of existing possibilities. In R. Johnson (Ed.). (1989). The second language curriculum. (Which
Pages?????) Cambridge: Cambridge University Press.
Colman, J. (1988). Curriculum structures in adult language learning: Implications for the AMEP. Prospect, 4 (1), 25-37.
Johnson, R. K. (Ed.). (1989). The second language curriculum. Cambridge: Cambridge University Press.
Lynch, B. K. (1990). A context-adaptive model for program evaluation. TESOL Quarterly, 24, 1, 23-39.
Nunan, D. (1988). The learner-centered curriculum. Cambridge: Cambridge University Press.
Nunan, D. (1992). Research methods in language learning. Cambridge: Cambridge University Press.
Rhea-Dickins, P. & Germaine, K. (1992). Evaluation. Oxford: Oxford University Press.
Richards, J. C. (1990). The language teaching matrix. Cambridge: Cambridge University Press.
Tyler, R. W. (1949). Basic principles of curriculum and instruction. Chicago: The University of Chicago Press.
... Evaluation is described as a systematic process that enables the assaying and judgment of whether the instructional program is effective, and through which the nature and extent of behavioural change among students who may have been exposed to the instructional process can be determined (UNESCO, IBE, 2013). Afsahi (2016) outlines three main types of evaluation methods, which are formative, summative, and follow-up. Afsahi contends that formative evaluation occurs during the educational process with the intent of improving performance, often referred to as "feedback." ...
Article
Entrepreneurship Education (EE) has been identified as the requisite driver for improvingKenya’s tourism economy. Consequently, the sector has emphasizedentrepreneurship education to bolster entrepreneurial skills alongside core technical skills among hospitality graduates in Technical Training Institutes (TTIs), to boost their employability or capacity to start their ventures. However, concerns remain regarding the inadequacy of soft skills among youth despite having the necessary technical training. This study examined the relationship between evaluation methods of EE and acquisition of entrepreneurial skills among students in 5 selected public TTIs in Kenya. The study adopted an explanatory research design. The target population comprised 199 students pursuing a diploma course in Food and Beverage Production, Service and Sales Management from the selected TTIs. A sample of 132 students was selected using stratified and simple random sampling methods. Questionnaires were administered to students. The simple regression method was employed to analyze the data. The findings provide evidence of a positive relationship between evaluation methods of EE and acquisition of entrepreneurial skills (B=3.366, p=.000).It was evident that the use of end-term sit -in exams; participation in class by answering questions and, sit-in tests were the predominant methods adopted. Such methods were found to be theoretical-based and examination-orientedand thus inadequate to evaluate a high level of entrepreneurial skills. Therefore, recommendations are made to TTIsto adopt practical-oriented methods and establish self-regulatory practices to monitor quality compliance to the EE syllabus.
... Evaluation is described as a systematic process that enables the assaying and judgment of whether the instructional program is effective, and through which the nature and extent of behavioural change among students who may have been exposed to the instructional process can be determined (UNESCO, IBE, 2013). Afsahi (2016) outlines three main types of evaluation methods, which are formative, summative, and follow-up. Afsahi contends that formative evaluation occurs during the educational process with the intent of improving performance, often referred to as "feedback." ...
Article
Full-text available
Entrepreneurship Education (EE) has been identified as the requisite driver for improving Kenya’s tourism economy. Consequently, the sector has emphasized entrepreneurship education to bolster entrepreneurial skills alongside core technical skills among hospitality graduates in Technical Training Institutes (TTIs), to boost their employability or capacity to start their ventures. However, concerns remain regarding the inadequacy of soft skills among youth despite having the necessary technical training. This study examined the relationship between evaluation methods of EE and acquisition of entrepreneurial skills among students in 5 selected public TTIsin Kenya. The study adopted an explanatory research design. The target population comprised 199 students pursuing a diploma course in Food and Beverage Production, Service and Sales Management from the selected TTIs. A sample of 132 students was selected using stratified and simple random sampling methods. Questionnaires were administered to students. The simple regression method was employed to analyze the data. The findings provide evidence of a positive relationship between evaluation methods of EE and acquisition of entrepreneurial skills (B=3.366, p=.000). It was evident that the use of end-term sit-in exams; participation in class by answering questions and, sit-in tests were the predominant methods adopted. Such methods were found to be theoretical-based and examination-oriented and thus inadequate to evaluate a high level of entrepreneurial skills. Therefore, recommendations are made to TTIs to adopt practical-oriented methods and establish self-regulatory practices to monitor quality compliance to the EE syllabus.
... By planning of the teaching evaluation, the school's administration can address challenges of teaching for every teacher in the school by determining his strength and weakness positions. Based on the planning of teaching evaluation, the school' principal supported by his subordinates can take the essential procedures to improve and develop the educational process (Afsahi, 2016). Particularly, in the secondary school' stage, as a result of the planning of teaching evaluation, the school's administration may have to undergo some teachers to specific training and development programs, and in other cases it may replace one by another according to some needed educational competences. ...
Article
Full-text available
The study aims to investigate the levels of teaching planning problems in the secondary schools, in Salt City in Jordan, in its three stages; planning problems of lesson’s preparation, planning problems of teaching implementation, and planning problems of teaching evaluation. To achieve this aim, descriptive analytical approach was used with a questionnaire which was used as the study instrument. The study sample was calculated to be (152) teachers who were selected randomly. The study results found that teaching planning levels in salt secondary schools were found in the medium level. Moreover, it was found that there is not significant difference between means of the sample’s responses about the problems of teaching planning due to the gender variable of the secondary schools ‘teachers. Finally, based on the study results, a set of recommendations were provided.
Chapter
An authoritative collection of papers which argues for issues of language curriculum development to be made explicit. Issues of language curriculum development underlie all stages in the planning and implementation of language teaching programmes. This authoritative collection of papers argues for these isses to be made explicit. The stages which the book deals with are: curriculum planning, specification of ends and means, programme implementation, and classroom implementation. Evaluation is also of crucial relevance at each stage. The 'curriculum' is taken to mean all the factors which contribute to the teaching and learning situation, and the emphasis of the book is on the interdependence of these factors. The contributors are leading practitioners and researchers with experience in various parts of the world. They identify the problems faced and the directions to be followed in relating current theory with practice in curriculum development. This collection will be of key interest to teachers, teacher-trainers, course directors and designers, and all others concerned with designing and implementing language programmes.
Book
Richards explains how effective language teaching involves a network of interactions between curriculum, methodology, teachers, learners, instructional materials. Each chapter discusses and examines the theoretical and practical dimensions of a central issue in language teaching. Topics covered include the nature of effective teaching, self-monitoring in teacher development, language and content, and teaching listening, speaking, reading and writing. Richards presents key issues in an accessible and highly readable style, and shows how teachers and teachers-in-training can be involved in the investigation of classroom teaching and learning. The emphasis is not on prescriptions but rather on developing effective teaching through understanding the various factors that interact in second language learning and in the second language classroom.
Article
The literature on the evaluation of language teaching programs has focused almost entirely on specific issues of methodology and measurement. This article presents a generalized model for ESL program evaluation. The context-adaptive model consists of a series of seven steps designed to guide the program evaluator through consideration of the issues, information, and design elements necessary for a thorough evaluation. These steps are illustrated with examples from the evaluation of the Reading English for Science and Technology (REST) Project at the University of Guadalajara, Mexico. The model is intended to be flexible, lending itself to effective adaptation and refinement as it is implemented in a variety of ESL/EFL contexts.
Article
A basic, but inadequately fulfilled, requirement of evaluation studies comparing the effects of teaching programs is that the tests used should be program-fair. In this article, attention is first of all drawn to the potential for misinterpretation of experiments lacking program-fair criteria. Second, the strategies adopted by various researchers to come to terms with the issue are considered from the perspectives of (a) standardized tests, (b) specific tests for each program, (c) program-specific plus program-neutral measures, (d) common/unique objectives, and (e) appeal to consensus. Finally, it is concluded that while the issue remains largely unresolved, there are some general principles which can be gleaned from past experience.