The Role of Evaluation in Curriculum Design

Abstract

The Role of Evaluation in Curriculum Design

Full text

THE ROLE OF EVALUATION IN CURRICULUM DESIGN Seyed Ehsan Afsahi
October 2016

Objectives:
1. Define evaluation and describe its value to a curriculum.
2. Describe the relationship between needs assessment, objectives and evaluation.
3. Assure that the evaluation process measures the appropriate domains of learning (cognitive,
affective and psychomotor).
4. Distinguish between feedback and evaluation.
5. Contrast norm-referenced and criterion-referenced evaluation.
6. Describe the three approaches to evaluation (Learner, Instruction and Program).
7. Discuss the advantages and disadvantages of various evaluation strategies for the learner and
program.
8. Prepare an evaluation plan, including appropriate instruments, for an educational curriculum.
Goal
We should be able to understand the basic elements of evaluation in the curriculum development
and design process and utilize that information to prepare a new curriculum.

Review
Let's review
where we are:
We've decided what we intend
to cover (goals and objectives),
and how we plan to do it
(teaching/learning strategies).
Now we'll decide how to
measure whether our plan
works.

"If we wish to discover the truth about an educational system, we
must look into its assessment procedures...The spirit and style of
student assessment defines the defacto curriculum."
D. Rowntree. Assessing Students: How Shall We Know Them
Evaluation is defined as "determining the value of ....“
In the context of an educational curriculum, evaluation is the determination of the value of
learning, instruction and the overall program.
In most educational settings, evaluation traditionally means assessing the learner to see if they
met a particular standard. In reality, there are a variety of benefactors from evaluation.
The Definition of Evaluation

1.The learner. Did the learner improve during the educational process? Did the learner meet a
certain standard? Depending on the curriculum objectives, did the learner meet those standards set
for the curriculum?
2.The instruction. Did the instruction, or faculty meet expectations of the learner and the overall
program?
3.The overall program. Did the program, or curriculum content/format, accomplish what it set out to
do?
Why Is Evaluation Important?
In a curriculum, evaluation could be used to provide the following value:
1. Feedback to all stakeholders T F
2. Improvement of the educational process T F
3. Certification of learner competency T F
4. Data to meet accreditation requirements T F
5. Assessment of the cost of delivery T F

All of the items are true. Although each evaluation tool may not ask those specific questions, overall data
from the evaluation process could contribute value to each item in the curriculum.
"Participants in the curriculum's experience have an interest in the evaluation of their own performance
and in the performance of the curriculum. Evaluation can provide feedback and motivation for the
continued improvement for learners, faculty, and curriculum developers." Kern, Thomas, Howard and
Bass, Curriculum Development for Medical Education: A Six-step Approach.
Kern, et al, points out further that evaluation has the following purposes:
1. Determining a learners' accomplishments and confirming competence which leads to a grade, or
certification.
2. Measuring improvement in the learners, instruction (faculty) and program.
3. Meeting accreditation standards by measuring student and program performance.
4. Determining the worth of the curriculum to the institution.
In essence, evaluation is an important component of the curriculum design process for the learner, the
faculty and the overall program. How do you use evaluation in the curriculum process?

•Formative Evaluation
This type of evaluation occurs
during
the educational process with the intent or
improving performance, often referred to as "feedback."
Formative Evaluation
means to assess the educational process while it is still
being used or developed. It can provide reassurance and reinforcement to the
individual, faculty member, or program. Its intent is
to improve performance.
Evaluation can be used in different ways - during, at the conclusion of, or some time after the
educational process. The terms that reflect these types of educational evaluation are the
following:
•This module will focus primarily on summative evaluation.
These are the types of evaluations that are used in an educational curriculum, but what do you
evaluate?
Types of Evaluation

•Summative Evaluation
This type of evaluation occurs at the
conclusion
of an educational activity with
the intent of documenting achievement or competence. For the purposes of
this module, the term "evaluation" will refer to this type
.
Summative Evaluation
means to make a judgment, or decision, at the
conclusion of the educational curriculum. It is a capstone assessment, the final
outcome or a certification of competency at completion. Summative
evaluation can be used for an individual learner, faculty member, or program.
•Follow
-up Evaluation
This type of evaluation occurs sometime
after
an educational activity, with the
intent of determining whether the learner has applied the knowledge/skill in
practice
.
Follow
-up Evaluation means to measure the result/application of the
educational intervention at a future time
. It is a type of evaluation that
determines whether the learner has applied the knowledge/skill in practice.
Most often this is self
-report and may reflect long-term satisfaction.

Three Major Levels of Curriculum Evaluation
As noted earlier, there are three levels of evaluation in a curriculum:
•The learner
•The instruction (faculty)
•The program
While these are the main levels of evaluation, keep in mind that you can use the formative, summative,
and follow-up evaluation processes with each. It is possible to use all three evaluation processes with the
same educational curriculum. The table below illustrates the relationships between the levels and the
uses.

Process
The instructional method/s used by the educational program or curriculum could
be assessed for its effectiveness. During, or at the conclusion of educational
program, the student and/or faculty could review how the method of instruction or
administration was implemented.
Outcome
The results of the educational program can be assessed for its effectiveness. How
effective is the program in achieving its educational objectives?
What do you Evaluate?
Evaluation can focus on different aspects of the educational program.

The main focus of this module is on the measurement of outcomes. For example, if the curriculum
objectives expect the learner to be able to conduct CPR at the end of the course, you would assess
the student's skill in performing CPR on a mannequin.
There are a number of outcomes from an educational intervention that could be measured.
Kirkpatrick describes a continuum of four different outcome levels from an educational process:
•Satisfaction,
•Learning (also referred to as competence),
•Behavior, and
•Societal outcomes

Usually the level of outcome measured is determined by the curriculum objectives. So in planning the
evaluation, the curriculum developer should review the curriculum goals and objectives to be sure
that the evaluation addresses the outcome level.
Pearl: An effective curriculum uses formative and summative evaluation to provide feedback to the
learner for improvement during the learning process and to make judgments or decisions about the
learner at the end of the process which will contribute to the ongoing improvement of the program.

Effective Evaluation can measure the process, or methods, and outcomes, or results of a curriculum. What criteria do
you use to measure the process, or outcomes?
How Do You Measure Process and Outcomes?
Evaluation data can be collected in two ways:
1.Norm-referenced Evaluation - A procedure designed to determine a student's performance in relation to the
performance of other students on the same test - "grading on a curve". These types of evaluations are usually
used that the conclusion of an educational intervention to provide a grade.
Example: At the completion of a course in the curriculum, the student takes written examination and is judged on
the basis of how he/she compares with other students who have taken the same examination. This reflects a
"relative standard" of performance of how that student has performed compared to other students on this
examination. The student that answers the most questions would receive the top grade, whether or not they have
accomplished the objectives for the course.
2.Criterion-referenced Evaluation - A procedure designed to determine whether a student has achieved an
established or selected standard as specified in the instructional objective(s). These types of evaluations can be
used as pre-tests, to assess a student's progress during the curriculum and/or, at the end of the educational
intervention, to assess whether the students have accomplished the specific objectives of the curriculum.
Example: A written evaluation of the knowledge and an observable demonstration of the behavior of the student
are evaluated at the completion of a cardio-pulmonary resuscitation (CPR) course based upon the specific
objectives of that curriculum. If the student does not meet the objective standards regardless of whether he/she is
the best of the group, the student does not pass, or is not certified to perform CPR.

Pearl: Norm-referenced Evaluation compares a learner's performance with other learners. Criterion-
referenced Evaluation compares learner performance with a standard (criterion). Criterion-
referenced Evaluation is best in medical education because mastery is the key.

Parts of an Evaluation Plan
An evaluation plan for a curriculum should have three parts:
•Evaluation of learner performance
•Evaluation of faculty/instruction
•Program evaluation
Evaluation of Learner Performance
Much of what we think of as evaluation is assessment of learner performance. The intent here is to
measure whether/how well learners have achieved the education objectives identified for the program.
For example, the objectives of the curriculum might expect the learner to know the physiology of the
heart. The written examination could ask questions about the aorta and its function.
There are many methods that can be used to measure learner performance
Pearl: When preparing the evaluation, make sure that it measures the objectives for the course, or
curriculum. If you are teaching knowledge, evaluate knowledge. If you are teaching a skill, evaluate
the skill.

Evaluation of Faculty
Good Faculty are life long learners. In order to stay current and continually improve, they need
assessment of their own performance as teachers. In addition these data may provide value in terms of
promotion, since teaching is being considered more frequently as a scholarly activity.
These evaluations can be done by
•Learners - students are often asked to complete a questionnaire at the end of a session, or course.
Learner satisfaction data can be helpful for faculty and program improvement but should not be relied
upon exclusively, as what/who students like do not always reflect who was the most effective in
facilitating their learning.
•Peers - Observation and feedback by other faculty are very useful to incorporate into faculty
assessment, when resources allow.
Below are some examples of questions that might be asked in a faculty evaluation process:
•Did the faculty member address the goals and objectives of the course?
•Did the faculty member answer questions raised by the learner?
•Was the format of the instruction relevant to the content and course objectives?
•Were the topics presented in the course relevant to you as a learner?
•Did the faculty member involve the learners in the educational process?
A formative, or summative, evaluation format may be used for instructional evaluation. Summative is the
most commonly used for learners, since they are more likely to give frank and helpful answers more
comfortably after the course is completed.

Evaluation of the Program
Program evaluation is designed to assess the effectiveness of the curriculum as a whole.
Program evaluation can determine
•whether the curriculum was effective in meeting its goals and objectives,
•whether the faculty members were effective as learning facilitators,
•whether the learners viewed the curriculum content and process as a value to their learning,
•cost effectiveness of the curriculum,
fit with other curriculum components of the school and/or
•compliance with external requirements.
The data may include
•aggregated learner evaluations about the instruction and the overall program in meeting their needs,
•faculty evaluations about the needs of the program, goals and objectives of the curriculum, and
content and process of instruction,
•learner performance data , such as application of the instructional material in the clinical setting,
•outcome data on learner achievements, such as results of the application of the learning in
publications and/or clinical practice and
•costs to offer the curriculum including the instructional material, support and faculty time.

Your Evaluation Plan
Think about the curriculum area you are working on. Look at your objectives and think about how you
would evaluate them.
•How would you assess learner achievement of these objectives?
•How would you measure your faculty’s effectiveness in teaching the material?
•How would you determine if your program as a whole was successful in achieving its objectives?

Summary of Evaluation
•Evaluation is one part of the circular educational process that measures whether the educational intervention meets its goals and
objectives.
•Evaluation is the determination of the value of learning, instruction and
the overall program of a curriculum.
•Evaluation can be done during (formative), or at the conclusion of the
educational process(summative), or at a future time (follow-up) that
determines the interventions success.
•There are three levels of curriculum evaluation: the learner, the
instruction, the overall program.
•Learner evaluation can use norm-referenced (comparison with other
students), or criteria referenced (comparison with a standard), evaluation
measures to assess different outcomes for various domains of learning.
•Ideally, evaluation systems should include formative and summative
methods.
•Multiple stakeholders should be involved in the program evaluation
process.

Four Models of Curricular Change
Model 1: Hotchpoch

Model 2: Follow the Leader

Model 3: Window dressing

Model 4: Bureaucratic top-down

Four Twentieth Century Theoretical Models of Curricular Change
Tyler's four-stage progression model (1949)
1. Objectives: What educational purpose should the school seek to attain?
2. Content: What educational experiences can be provided that are likely to attain the objectives?
3. Organization: How can these educational experiences be effectively organized?
4. Evaluation: How can it be determined whether the objectives are being attained?
Tyler's Basic principles of Curriculum & Instruction was regarded by many educators as the early bible of
curriculum design. In it, he provided a workable model for the systematic development of curricula.
However, it was criticized as being an archetypal 'ballistic' model; that is, curriculum development activity occurs
in a linear series of sequential stages. Also, it was criticized as representing an "ends-means" view of education.

Taba's model (1962)
Step 1: diagnosis of needs
Step 2: formulation of objectives
Step 3: selection of content
Step 4: organization of content
Step 5: selection of learning experiences
Step 6: organization of learning experiences
Step 7: determination of what to evaluate & means to evaluate.
In this early standard model of the curriculum process, there is a systematic movement through a series of
set steps.
However, its rigidity is its chief weakness.
This "means-ends" or objective model has proved a popular framework for developing language curricula
programs, given that teaching itself is usually seen in these terms.
This model is often referred to as the Rational Planning Model because it is "rational" to the specify ends
before engaging in any given activity.

A hybrid Taba-Tyler model

Nunan's model (1985)

References
Alderson, J. C., & Beretta A. (Eds.). (1992). Evaluating second language education. Cambridge: Cambridge University Press.
Beretta, A. (1992). Evaluation of language education: An overview. In J. Alderson and A. Beretta (Eds.). (1992). Evaluating second language education (pp. 5-24).
Cambridge: Cambridge University Press.
Beretta, A. (1990). The program evaluator: The ESL researcher without portfolio. Applied Linguistics, 11, 1, 1-14.
Beretta, A. (1986). Program-fair language teaching evaluation. TESOL Quarterly, 20 (3), 431-444.
Brown, J. D. (1989). Language program evaluation: A synthesis of existing possibilities. In R. Johnson (Ed.). (1989). The second language curriculum. (Which
Pages?????) Cambridge: Cambridge University Press.
Colman, J. (1988). Curriculum structures in adult language learning: Implications for the AMEP. Prospect, 4 (1), 25-37.
Johnson, R. K. (Ed.). (1989). The second language curriculum. Cambridge: Cambridge University Press.
Lynch, B. K. (1990). A context-adaptive model for program evaluation. TESOL Quarterly, 24, 1, 23-39.
Nunan, D. (1988). The learner-centered curriculum. Cambridge: Cambridge University Press.
Nunan, D. (1992). Research methods in language learning. Cambridge: Cambridge University Press.
Rhea-Dickins, P. & Germaine, K. (1992). Evaluation. Oxford: Oxford University Press.
Richards, J. C. (1990). The language teaching matrix. Cambridge: Cambridge University Press.
Tyler, R. W. (1949). Basic principles of curriculum and instruction. Chicago: The University of Chicago Press.