ArticlePDF Available

Abstract

The authors are members of the Steering Committee of the PREAL Working Group on Standards and Assessments (GTEE). The opinions expressed in this study are those of the authors and do not necessarily reflect the positions of PREAL or its sponsoring institutions.
Programa de Pr omocióndelaReforma Educativa en AméricaLatinayel Caribe
Partnership forEducational Revitalization in theAmericas
Working Paper Series
No. 40
The Educational
Assessments That Latin
America Needs
by Pedro Ravela, Patricia Arregui,
Gilbert Valverde, Richard Wolfe,
Guillermo Ferrer, Felipe Martínez Rizo,
Mariana Aylwin, and Laurence Wolff
March 2008
The authors are members of the Steering Committee of the PREAL Working Group on Standards and Assess-
ments (GTEE). The opinions expressed in this study are those of the authors and do not necessarily reflect
the positions of PREAL or its sponsoring institutions.
©2009 Partnership for Educational Revitalization in the Americas (PREAL).
All rights reserved.
For additional copies, contact PREAL at the Inter-American Dialogue. This report can be downloaded from
www.preal.org/publicacion.asp.
Inter-American Dialogue
1211 Connecticut Avenue, NW, Suite 510
Washington, DC 20036
202-822-9002
iad@thedialogue.org
www.thedialogue.org and www.preal.org
Citation: Ravela, Pedro, Patricia Arregui, Gilbert Valverde, Richard Wolfe, Guillermo Ferrer, Felipe Martínez
Rizo, Mariana Aylwin, and Laurence Wolff, The Educational Assessments That Latin America Needs (Wash-
ington, DC: PREAL, 2009).
ISBN: 978-0-9800777-5-9
First edition
Published in the USA
Book design: Nita Congress
Cover design: Studio Grafik
The Educational Assessments That Latin America Needs|i
Contents
1. Introduction 1
2. The Importance of National Assessments of Educational
Achievement 1
3. Standardized Assessment in Latin America: Current
Situation 5
4. Purposes and Uses of Assessments 7
4.1 Assessments to Certify What Students Have Learned 8
4.2 Diagnostic and Formative Assessments  9
4.3 Assessments and Incentives  10
4.4 Disseminating and Using Results 11
4.5 Some Cautions about Comparing Results across Schools 12
4.6 Checklist for Decision Making 13
5. Technical Quality Challenges of Assessments 14
6. Establishing Assessment Units 15
7. Ten Recommendations on the Assessments That the Region
Needs 17
The Educational Assessments That Latin America Needs|1
1. Introduction
This working paper addresses the need for and uses
of large-scale standardized assessments, typically in
primary and secondary school, of learning and/or
educational achievement in Latin America and the
Caribbean. It is aimed at policymakers in the field
of education, teachers, academics, business people,
members of trade unions and social organizations,
and those working in financial agencies and the me-
dia. It seeks to contribute to the debate on standard-
ized testing in education systems and to decisions
made in that regard.
As used here, large-scale standardized assessment
refers to assessment that produces comparable infor-
mation on the performance of students from different
cultural and regional contexts, including from differ-
ent countries. It yields information that provides an
overview of the situation in a country, state, or prov-
ince, even where the sample is not particularly large
(for example, 5,000 students).1
This document focuses on assessments of learning,
defined as the change in each student’s knowledge
and capacities in the course of the school year, and/
or of educational achievement, meaning the accu-
mulation of knowledge and capacities throughout
a student’s whole life. The paper does not address
other important aspects of assessing educational en-
deavors, such as evaluations of teacher performance,
education policies, and schools or the assessments
that teachers carry out in the classroom. It also does
not examine tertiary-level assessments or entrance
exams for higher education.
This working paper can help those who make educa-
tion policy decisions understand and analyze various
1The term “large-scale” is not synonymous with census-
based assessments.
options regarding the purposes and uses of assess-
ment systems and the implications of each, and how
to devise an assessment strategy.
All too often, assessment is undertaken in a simplis-
tic and naïve manner, leading to poorly conceived
and inadequately implemented assessment exer-
cises with unintended consequences for the educa-
tion system as a whole—including wasting resources
and discrediting assessment in the eyes of teachers.
A wide range of issues must be taken into account
in implementing an assessment system or reforming
an existing scheme. If the assessment initiative is to
have any value, the education system in which it is
implemented must have a clear purpose, an under-
lying vision of shared responsibility in education, a
high-quality technical design consistent with its aims,
a strong motivation to support teachers in their work,
and the political will to take the steps necessary to
resolve the problems and weaknesses the assessment
uncovers. Significant resources must be invested in
creating a competent technical unit, and a compre-
hensive long-term plan must be drawn up; this pro-
cess takes time and cannot be improvised.
2. The Importance of National
Assessments of Educational
Achievement
Standardized assessments are used with growing fre-
quency to gain greater insight into processes and re-
sults in education systems throughout the region and
the world, in countries with very different cultures
and with governments of varying political affiliations.
This is evidenced by countries’ increasing involve-
ment in international assessments such as the Pro-
gramme for International Student Assessment (PISA),
the Trends in International Mathematics and Science
Study (TIMSS), and the Progress in International
Reading Literacy Study (PIRLS) as well as regional
2|The Educational Assessments That Latin America Needs
tests such as the Second Regional Comparative and
Explanatory Study (SERCE) in Latin America and the
Southern Africa Consortium for Monitoring Educa-
tional Quality (SACMEQ) in Africa. It is also apparent
in the development of various kinds of national and
subnational assessment systems.
In some cases, assessment is spurred by a desire to en-
gage in civic education and consolidate a democratic
society. In other cases, the objective is to increase the
productivity of the labor force and the competitiveness
of the national economy, to take advantage of the op-
portunities for the overall development of individuals
and their participation in the knowledge society, or
to use education as a path to securing greater equity
and overcoming poverty. Many assessment systems
are based on a combination of these concerns.
In almost all cases, it is assumed that assessment can
serve
as a basis for better-grounded education policies,
as a means of improving the management of edu-
cation systems,
as an instrument to foster collaboration and con-
tinuous learning within those systems.
The following describes the main contributions of
such assessments.
Standardized assessments provide an overview of
results for the entire student body. Education is an
opaque activity, in the sense that its outcomes are not
directly or immediately apparent. Good teachers can
determine whether their students are learning and
how, but not all teachers have the same assessment
criteria. Those criteria are closely related to teach-
ers’ professional experience: their general and spe-
cific training, their knowledge of the subject they are
teaching, their capacity to discern students’ processes
and difficulties, their familiarity with different kinds
of students, and so on. Given the diversity of a coun-
try’s teachers, an overview cannot be acquired simply
by aggregating individual viewpoints, however. Stan-
dardized assessment seeks to provide that overview.
Assessments offer information on the real extent
of knowledge and the capacities that students
achieve, measuring more than just years of school-
ing. In past decades, the relationship between staying
in the education system and developing knowledge
and capacities was taken for granted; thus, the indica-
tors used to appraise education systems were related
to access (enrollment, coverage, dropout rates, etc.).
In that period, the poorest majorities with the least
cultural capital either failed to enter the education
system or had just a few years of basic schooling. As
access to the system has gradually become universal,
students from the poorest social sectors who do not
speak or write their country’s official language flu-
ently arrive at school at a distinct disadvantage and
seldom perform at the grade level in which they are
enrolled. At the same time, increased access to the
teaching profession has not been matched by guaran-
tees of quality in teacher training, a circumstance that
casts doubt as to whether such an equivalence exists.
In sum, an increase in the number of years that pu-
pils spend in the system does not necessarily mean
that all children and youths are developing the in-
creasingly sophisticated and complex knowledge, at-
titudes, and capacities needed for their personal and
social lives. Assessments seek to shed light on what
is happening in this regard.
Standardized assessments help reveal a set of key
educational issues. Among other things, standard-
ized assessments provide information on
the extent to which students are learning what
is expected of them when they complete certain
grades or levels of schooling;
The Educational Assessments That Latin America Needs|3
the degree of equity or inequity in the acquisition
of such learning;
how achievement levels and equity in the access
of different social groups to knowledge evolve over
time;
how and to what extent socioeconomic and cul-
tural inequalities affect students’ learning oppor-
tunities;
the range of educational practices currently used in
schools and among teachers, and how those prac-
tices are related to students’ learning in different
social contexts;
how student progress is influenced by teaching
conditions (teachers’ wages and general working
conditions, access to teaching resources, availabil-
ity of time to prepare lessons and reflect on teach-
ing practices, etc.);
the effect that investments in educational programs,
changes in the education system’s structure, cur-
ricular reforms, training schemes, the acquisition
of teaching materials, and other factors have on
educational attainment.
A system for assessing learning and/or educational
achievement can provide important information to
various social actors. To the extent that the system
produces and appropriately delivers information on
the above matters, it can be a crucial instrument for
improvement, enriching understanding of educa-
tional circumstances and decision making in various
spheres.
Ministry of education officials and other policy-
makers can gain a better understanding of prob-
lems in teaching and learning; they can be made
aware of deficiencies in the context within which
teachers discharge their duties; and they can de-
velop relevant policies to support the work of the
schools. Assessments also give them a solid ba-
sis of empirical evidence on which to evaluate the
impact of policies and programs they have imple-
mented, and to gauge the probable effects of those
they plan to implement.
Principals and teachers
can, on the basis of an
external review of educational achievements
throughout the system, gain a better understand-
ing of the level at which their students are per-
forming, how they are learning, and the difficulties
they are facing. They can learn from the experi-
ences of other teachers and schools working with
students similar to and different from their own.
And they can make more informed decisions about
which aspects of the curriculum to emphasize and
can enhance their own methods of assessing stu-
dent learning.
Supervisors and those responsible for training
teachers can use information on systemic educa-
tional achievements and difficulties as the basis for
making a detailed study of weaknesses in overall
educational approaches or specific teaching prac-
tices that underlie the learning deficiencies the as-
sessments reveal. They can thereby improve their
work in guiding and training teachers. Supervisors
can particularly benefit from using assessment data
to map schools by student sociocultural composi-
tion and educational achievement.
Parents
, given appropriate information, can bet-
ter understand what their children are expected to
learn, what they are achieving, and what they can
do to collaborate with the school in their children’s
learning.
Citizens
in general may pay more attention to edu-
cational issues and problems when they are bet-
ter informed about what is happening within the
education system. They will be better positioned
to demand of public officials and teachers that the
4|The Educational Assessments That Latin America Needs
quality of education given to children and youths
be constantly improved, and that the resources de-
voted to education be used responsibly.
Development of a national standardized assessment
system necessitates informed debate as to which as-
pects of the formal curriculum should be manda-
tory and to define clearly what all students should
have learned at the end of each educational cycle. In
To Avoid Confusion…
When national assessment systems are being developed, their role and function must be stated explicitly,
along with their limitations. This explicit denition will obviate the risk that they, as well as the information
they gather and disseminate, might be misinterpreted.
A standardized assessment provides fundamental and vital information on educational quality, but is
not a complete indicator of said quality. Not all the valuable goals of education are included in na-
tional standardized assessments. Much important knowledge and learning, as well as many signicant
attitudes and values, cannot be part of an assessment—because they are difcult to measure—or
should not be part of one—because they are relevant to their local context and thus not applicable to
all students in a country.
Standardized assessment of learning and educational achievement is an essential component of a
comprehensive educational assessment system, but it is not the only form of relevant evaluation. Also
important are the assessments that teachers carry out in the classroom and assessment of teacher
performance, of educational institutions, of education policies, of resource use, of the relevance of the
curriculum, and so on.
Assessment is a necessary but insufcient condition for improving education.
There is some evidence
that the mere existence and dissemination of information has some effect on certain actors. But assess-
ment is only one of several key elements of education policy; others include preservice and in-service
teacher training, teacher working conditions, school management and supervision, curricular design,
textbooks and educational materials, investment of resources proportional to the needs of different
populations, and concerted action by those responsible for education to resolve any problems uncov-
ered. Signicant efforts should be made to ensure that these elements are in proper alignment.
Standardized assessment will only have positive effects on education if it is conceived, perceived, and
used as a mechanism to ensure the public accountability of all actors involved in education. There is
always a risk that education policy might concentrate on conducting assessments, but that no con-
crete action is subsequently taken to tackle and resolve the problems the assessments bring to light.
Often, authorities conne themselves simply to providing information on the assessment results and
transfer all responsibility for solving the problems to the schools and families. At other times, all responsi-
bility is assigned to the teachers, who themselves tend to shift responsibility to the parents or the social
context. This “blame game” is unproductive; instead, a vision of shared responsibility for education
should be forged.
Latin America and the Caribbean, most curricula con-
sist of long lists of goals and themes, all of which are
desirable but not all of which are feasible. Designing
national assessments makes it necessary to determine
what should be considered fundamental—i.e., what all
students should know and be able to do—regardless
of whether this knowledge set is defined as standards,
basic skills, achievement indicators, performance lev-
els, learning targets, proficiency criteria, etc.
The Educational Assessments That Latin America Needs|5
3. Standardized Assessment in
Latin America: Current Situation
National standardized assessment systems experienced
robust development in Latin America during the 1990s.
In some countries, the systems have been operating
continuously for decades, despite shifts in approach
or changes in institutional structure. Other countries
have experienced a significant lack of continuity and
have repeatedly had to start over almost from square
one, or will have to do so in the near future.
Sixteen countries are taking part in the UNESCO
Regional Bureau of Education for Latin America
and the Caribbean (OREALC/UNESCO) Second
Regional Study for third- and sixth-grade primary
school students: Argentina, Brazil, Chile, Colom-
bia, Costa Rica, Cuba, the Dominican Republic, Ec-
uador, El Salvador, Guatemala, Mexico, Nicaragua,
Panama, Paraguay, Peru, and Uruguay. The related
network of national assessment systems, in which
most Latin American countries participate, and
which constitutes OREALC/UNESCO´s Laboratorio
Latinoamericano de Evaluación de la Calidad de la
Educación (LLECE), has been active in the region
since 1997. It meets twice a year and is a useful
forum for training and the exchange of experiences
in the assessment field.
Six countries—Argentina, Brazil, Chile, Colombia,
Mexico, and Uruguay—took part in PISA 2006, and
another three—the Dominican Republic, Panama,
and Peru—will be involved in PISA 2009. A PISA
Ibero-American Group was recently established,
comprising Argentina, Brazil, Chile, Colombia,
Mexico, Portugal, Spain, and Uruguay.
Some countries have taken part (or are now partic-
ipating) in the studies for mathematics and science
(TIMSS), reading (PIRLS), and civic education led
by the International Association for the Evaluation
of Educational Achievement.
Large-scale assessments in the region are becom-
ing better and more frequent. Changes and improve-
ments in recent years include the following.
Greater transparency in the dissemination of re-
sults. Authorities no longer interfere in the disclo-
sure of “unfavorable” assessment results, as those
in several countries had done in the past.
A shift from normative tests, whose main aim
is to establish a comparative hierarchy among
students, to criterion-referenced tests, which fo-
cus on what students know and are able to do.
Increasingly, too, criterion-referenced tests include
a definition of the results that all students should
achieve if their performance is to be judged satis-
factory.
Improved technical/methodological capacities to
devise tests and process data. Efforts have been
made to develop tests that assess a broader range
of knowledge and capacities, and that include
open-ended questions. The use of more sophis-
ticated methods of data processing, such as item
response theory and multilevel analysis, has also
been growing.
Increased attention to the dissemination and use
of results. There is a rising awareness that it is not
enough to undertake an assessment and publish a
report. A dissemination strategy must also be de-
veloped, and a series of reports must be prepared
in line with the needs of each target audience. As-
sessment units are more aware of the appropriate
and inappropriate uses of each kind of assessment,
although those who make the policy decisions are
often not clear about this.
Greater concern for research into the factors
that affect learning. Though much remains to be
done in methodological and interpretive terms to
produce good research studies, there is, in most
6|The Educational Assessments That Latin America Needs
countries, an appreciation of the need to conduct
more research and to devise hypotheses about how
children’s opportunities to learn are influenced by
the dynamics of education system management,
school processes, teaching practices, and educa-
tion policy decisions.
Countries’ increasing involvement in interna-
tional assessments. This involvement has had
positive effects on assessment units and the qual-
ity of their work: it has fostered the development
and accumulation of technical capacity for assess-
ment, facilitated exchange and the development of
a common language among countries, and helped
improve the quality of various technical processes
(the design of tests, inclusion of open-answer
questions, sampling, quality control of the test ad-
ministration process, data analysis, dissemination
methods, etc.).
The primary weaknesses in the region’s standard-
ized assessment systems are in the areas of dis-
semination strategy, use of results, and technical
quality of assessments. Despite the aforementioned
improvements, a significant number of weaknesses
remain to be addressed.
Often, political authorities demand that large as-
sessments be conducted without clearly defined
purposes, with time frames and resources that are
insufficient if the assessments are to be technically
sound.
Authorities frequently fail to understand that not
just any assessment will serve any purpose, and
that an assessment system must be designed care-
fully for the long term in light of clearly established
purposes and uses. Without this definition, good
decisions cannot be made regarding the use of cen-
suses or samples, grades and subjects to be tested,
periodicity of assessments, types of tests and re-
porting scales to be used, among other things.
There should be more public discussion of what
should be assessed and which aspects of the cur-
riculum should have been taught to all students by
the end of certain levels of education. Absent such
a discussion, learning targets and/or standards are
unclear, and there can be no clarity for teaching
or assessment. This undertaking calls for a broad
social debate and for the coordination of technical
work between curriculum and assessment units.
In order to raise the quality of research on factors
associated with student learning, it is important to
improve the instruments that are applied with the
test to collect complementary information, as well
as to develop more ambitious research designs,
such as longitudinal studies, “value-added” mod-
els, qualitative studies, etc.
In general, the region’s education ministries have
had only limited capacity to conceive, formulate,
and implement policies that respond to the prob-
lems uncovered by the assessments. There should
thus be stronger links between the assessment
units and other important actors in the education
field, both within and outside the education minis-
tries. Closer ties among assessment, curricular de-
velopment, preservice teacher training, and teach-
ers’ professional development are also needed.
The activities undertaken for the dissemination and
use of the assessment results continue to be inade-
quate, especially in terms of ensuring that teachers
understand and use the results and that the results
are incorporated into the school culture.
Most countries persist in disseminating results
by type of school and/or by individual schools,
states, or provinces without due consideration for
the sociocultural contexts in which the institutions
and educational subsystems operate. This practice
gives rise to flawed interpretations and conclusions
regarding the educational effectiveness of those in-
The Educational Assessments That Latin America Needs|7
stitutions and subsystems. A proper appraisal calls
for value-added assessments, involving two evalu-
ations of the same population at different times.
The region has only minimal experience in this
area.
Many countries continue to exhibit significant tech-
nical weaknesses in their assessment design: most
of the questions in their assessments are overly
simplistic, with few if any questions devised that
allow complex cognitive capacities to be measured.
The assessments focus on the educational achieve-
ments of a specific grade, which makes it difficult
to determine what has been learned by those who
still fail to attain what is expected of that grade.
There are deficiencies in devising samples and
in estimating and reporting measurement errors.
Weaknesses also persist in conducting assessments
that are comparable over time.
There is a shortage of professionals who are quali-
fied to design and carry out these kinds of assess-
ments, a circumstance exacerbated by turnover
among the countries’ technical specialists, often for
political reasons. This lack of expertise and conti-
nuity hampers the accumulation of knowledge and
experience in the region. Many countries have had
to start from scratch in establishing their assess-
ment systems, years after having had a system in
operation and having dismantled it.
4. Purposes and Uses of
Assessments
Whether an assessment system is being implemented
or reformed, countries must evaluate certain basic
options in light of what the system is expected to de-
liver.
The first step is to define the purpose of the system
and the use to be made of its results. Several options
should be considered; they are not necessarily mutu-
ally exclusive. A system can in fact combine several
of them, but it is important to be aware that each
requires a particular design and has different techni-
cal demands and costs. The most basic decision is to
determine whether the national assessment system is
to be diagnostic in nature or whether it is to certify
achievement.
A certification assessment or graduation exami-
nation is a means of determining students’ edu-
cational achievements. Here, the main goal is to
determine which students have acquired the req-
uisite knowledge and reached the requisite perfor-
mance level to complete a course or grade and thus
to pass or fail.
A diagnostic or formative assessment
can focus
on students, schools, or the education system as
a whole, and entails no direct consequences for
The Risk of Poor Assessments
Only an assessment that is technically sound
and whose results are disclosed and used ap-
propriately can have an impact on improving
learning. An assessment whose results are little
known and little used is obviously a futile exer-
cise and a waste of resources.
More serious are those circumstances in which
the results of technically decient assessments
are broadly disseminated, or when assess-
ments are used for purposes other than those
for which they were conceived and designed.
These scenarios can be harmful to the educa-
tion system itself. Moreover, to simply carry out
an assessment and disseminate the results as
an accountability mechanism, irrespective of
the quality of the assessment, can cause as-
sessment systems to suffer setbacks and make
it impossible to hold a serious discussion about
the advantages and disadvantages of ac-
countability for results.
8|The Educational Assessments That Latin America Needs
students. Its main goal is to provide quality infor-
mation to enrich the perceptions, decisions, and
actions of various actors (authorities and techni-
cal staff, supervisors, principals, teachers, students
and their families), with a view to improving teach-
ing and learning.
A proper balance must be struck between the imple-
mentation of national tests and participation in in-
ternational tests. National tests can offer a better view
of what students have learned as compared with what
they have been taught, while international tests can
reveal what students know and can do compared to
their counterparts in other societies, thereby enriching
the debate on the national curriculum and approach
to teaching. Countries should carefully evaluate which
international tests they might choose to participate
in, mindful of what each test seeks to assess and its
relevance to national objectives. It seems advisable to
participate periodically in at least one regional or in-
ternational test, in light of each country’s priorities.
4.1 Assessments to Certify What
Students Have Learned
An assessment system to certify what students
have learned by means of a high-quality, national
examination has advantages in terms of transpar-
ency and accountability for results. It is common
for two students who have completed the same level
of education in two different regions of a country to
have very different levels of knowledge. Assessment
systems designed for certification purposes clarify the
value of educational qualifications in the eyes of soci-
ety. Moreover, such systems make both teachers and
students themselves responsible for acquiring the
knowledge that the test assesses, which has positive
effects on learning. Such graduation exams are much
more appropriate for the higher levels of the educa-
tion system than the lower, especially at the end of
secondary school.
An assessment system for certification purposes
has some costly requirements. First, the tests must
be census-based and each student must have more
than one chance to take them, thus requiring the con-
duct of several assessments every year. Additionally,
because the tests must have wide curricular coverage,
they must be extensive, covering several subjects or
disciplines. They should also, to the extent possible,
include constructed response questions, which entail
significant codification costs.
An assessment system geared to certification can
trigger considerable tensions which must be antic-
ipated. If the graduation tests are demanding, there
might be a very high failure rate. This would mainly
affect the more vulnerable social sectors and could in-
tensify such problems as dropout rates and youth un-
employment, making such tests unsustainable from
a social and political standpoint. Consequently, this
type of assessment—like all assessments—must be
accompanied by complementary and remedial teach-
ing activities. If high standards are proposed, systemic
responsibility must be taken to provide students with
every opportunity to learn, including the provision of
textbooks, materials, facilities, and teacher prepara-
tion. The following measures will also be useful in
implementing these assessments:
Establish that standardized assessment is only one
part of the process of certifying student learning
(for example, 40 percent of the final grade). The rest
of the score could be derived from teacher assess-
ments of students. Even though teachers’ assess-
ment criteria are diverse, this approach combines
external and internal assessment, and gives teach-
ers an “outsider’s” perspective regarding what is
happening in their schools, thereby allowing them
to reflect on their own evaluation criteria.
Establish a period of transition to universally ap-
plicable standards. During that period, the main
The Educational Assessments That Latin America Needs|9
focus would be on the extent to which the students
in each school had improved or made progress rel-
ative to their previous performance and not solely
on achievement of the standard for certification.
The worst way to approach the issue would be to set
a fixed failure rate—for example, to mandate that no
more than 10 percent of students should fail. Such
an approach entails the use of simpler tests and thus
sends the wrong signal to students, families, and edu-
cators about academic expectations.
4.2 Diagnostic and Formative
Assessments
A formative assessment system with no direct con-
sequences for students has advantages in terms of
cost, possibilities for designing and setting high
standards, and contributing to an assessment cul-
ture. The costs of applying such tests can be lower,
since they can be taken by a sample of students in key
grades and at certain multiyear intervals. The tests
can be matrix-type, in which not all students answer
the same questions but rather blocks of questions.
This method allows a very large number of questions
to be used and facilitates a more detailed analysis of
the various aspects of the curriculum. Such systems
make it possible to set demanding standards or ex-
pectation levels without causing widespread failure.
Moreover, they help promote an assessment culture
and allow technical capacity to be constructed, so
that when consideration is given to the possibility of
establishing a system with consequences, the condi-
tions are in place to do it properly. One of the alterna-
tives to be considered in using formative tests is to
release an entire test so that it can be applied autono-
mously by the teachers in order to help them identify
the difficulties of individual pupils and enrich their
repertoire of assessment instruments.
The main problem with diagnostic assessments is
that they may have no impact if complementary ac-
tions are not taken. Diagnostic and formative assess-
ments can be of little use if they are not matched by a
precise strategy and significant investment to ensure
that results are disseminated and used in later edu-
cational activities, since their effectiveness depends
on the various stakeholders receiving, understanding,
and using the results. In this regard, the following
considerations should be kept in mind:
If the results of such assessments are to have an
impact on education policies, time must be spent
in analyzing and discussing the results in various of-
fices of the education ministry and among other rel-
evant stakeholders, in understanding the problems
and weaknesses that the results bring to light, and
in devising appropriate activities and investments to
tackle those problems. The authorities must be will-
ing to submit their policies and decisions to public
scrutiny; hence the need to invest in appropriate and
ongoing communication of results to the public.
If the results are to have an impact on teach-
ing practices, time must be spent in analyzing and
interpreting their didactic implications: if students
are unable to resolve certain situations, what is
being done deficiently or incorrectly in the class-
room, and what should be done differently? This
kind of analysis should be undertaken by individu-
als who specialize in teaching the subjects under
assessment and by teachers, thereby creating per-
manent forums for in-service training and joint
work within the schools. Teachers must be able
to analyze the greatest possible number of items
in order to determine which reveal a significant
obstacle to the development of new concepts or
capacities. However, it will always be necessary to
keep some assessment items confidential so as to
enable comparable assessments over time.
If the results are to have an impact on students’
motivation and families’ attitudes toward learn-
10|The Educational Assessments That Latin America Needs
ing in school, students and families must be given
appropriate and comprehensible information as to
what is regarded as crucial for students to have
learned in each grade or educational level and on
the actions that might be conducive to attaining
such learning.
Diagnostic tests can be sample- or census-based,
depending on the strategy for educational change.
Whether the tests are sample-based or census-based
has various implications; it is also possible to combine
a test for controlled samples with a census-based dis-
tribution of tests that can be applied autonomously
by the schools. The purposes of the latter are forma-
tive, facilitating analysis of results and identification
of students who need additional support.
Sample-based tests
provide an overall diagnosis
of the system. Care must be taken in devising the
sample, so as to secure representative information
for the levels of disaggregation at which actions
and decisions are to be taken (regional, provin-
cial, and municipal; urban and rural; indigenous
schools; etc.). The impact of these test results de-
pends mainly on the education policy measures
taken at the central level and on an appropriate
outreach strategy targeting all schools.
Census-based tests
provide information on each
school and even on each pupil. The impact of
their results depends on conveying the informa-
tion to each educational community with a focus
and format that helps encourage greater participa-
tion and commitment at the local level. The in-
formation might also be highly useful in focusing
policies on districts or schools with more serious
problems, since census-based tests offer a “map”
of the results of every school, area, province, type
of school, and so forth.
4.3 Assessments and Incentives
Some standardized assessment policies are geared
to establishing economic incentives in light of as-
sessment results or to fostering a competitive mar-
ket among schools. These policies are of three types:
Using the results to draw up rankings of schools
and making them public as a means of encourag-
ing schools to take responsibility for their results,
helping families make informed decisions about
which school they want their children to attend,
and promoting interschool competition to achieve
the best results.
Using the results to give economic incentives to the
schools that achieve the best results or improve on
their results in previous assessments.
Using the results as an indicator of the quality of
each teacher’s work and as a criterion for offering
economic incentives.
Diagnostic Assessments Should Take a
Broad Perspective
Tests should not be conned to assessing the
knowledge and skills corresponding to a single
grade (that for which the test is designed), but
rather should take a broader perspective of
performance levels—from the most basic to
the most complex—across several grades. This
approach helps identify what students have
learned in previous grades and what they
need now. Teachers can thereby note and
rectify the weaknesses of learning in previous
courses that hinder students from advancing.
The results will be useful not only for teachers
working in the grade under assessment, but
also for those teaching earlier grades. The em-
phasis should not be on informing the teachers
that students have “passed” or “failed,” but
rather on conveying that they are at different
points on a continuum of learning, along which
everyone can and must advance.
The Educational Assessments That Latin America Needs|11
These approaches sometimes result—deliberately
or not—in the state relinquishing its responsibility
for the results of the education system. Sometimes,
by confining itself to carrying out assessments, deliv-
ering the results, and creating incentives on the basis
of the results, the state transfers responsibility for the
results to schools and families, as if it were a mat-
ter between private actors. The state does this rather
than endeavoring to create the conditions wherein
teaching is made effective by providing the necessary
resources, putting in place a properly trained teaching
staff, and establishing mechanisms for evaluating and
guiding the work of the schools. This approach does
not take into account the complexity of the educa-
tional endeavor, especially in socially disadvantaged
environments, and it disregards the need to invest in
capacity building as a key tool in improving teaching
and learning.
4.4 Disseminating and Using Results
Responsibility for results should be shared among
the various stakeholders, including national, re-
gional, and local authorities; the teaching staff; and
students and their families. Care must be taken to
avoid using the results for the deliberate or implicit
purpose of assigning sole blame or responsibility to
certain actors.
Responsibility for results requires a proper bal-
ance between the demands made on schools and
teachers and the support they are given. Schools
and teachers should take responsibility for ensuring
that all students learn what is expected of them. At
the same time, the authorities have the duty of estab-
lishing the support policies needed to allow schools
and teachers to do their work properly. Making heavy
demands of schools and teachers without providing
the corresponding support can only cause ill-will and
discouragement. On the other hand, providing sup-
port without making the corresponding demands can
cause complacency.
It is not appropriate to use the results of standard-
ized assessments as the main indicator of the qual-
ity of the work of the teacher or school. This prem-
ise is particularly important where the assessments
do not control for other factors within and outside the
education system, and where it is not kept in mind
that learning also depends on student motivation and
personal effort. Because the results of standardized
assessments are not the sole indicator of the quality
of education, they must be considered in conjunction
with other pertinent matters such as educational at-
tainment, the relevance of what is being taught, the
development of values and habits, and civic educa-
tion. Efforts must be made to avoid identifying the
term “quality” with the results of standardized tests.
To facilitate understanding and the use of results, it
is not enough to offer only numerical data. The var-
ious stakeholders must understand the kinds of tasks
students should be able to undertake in the tests.
Concurrently, however, the question set must be kept
confidential so it can be used in future assessments
to ensure comparable evaluations over time. While
it is not intended that the teachers use standardized
tests to assess their students, knowledge and under-
standing of these assessments can help improve their
own evaluation methods and develop an assessment
culture.
When the differences in results are reported in
terms of the sociocultural composition of the stu-
dent body, care must be taken to avoid creating a
system of differentiated achievement expectations
for diverse social groups. Education policies should
make distinctions so as to create the right conditions
in which to teach the least advantaged groups. No
attempt should be made to use the results in such a
12|The Educational Assessments That Latin America Needs
way as to encourage schools—directly or indirectly—
to select students with a view to improving their re-
sults.
4.5 Some Cautions about Comparing
Results across Schools
Although the results of standardized tests do not
provide an exhaustive picture of a school’s educa-
tional quality, they do provide important informa-
tion on the performance levels achieved. The infor-
mation on two standardized test results—normally,
language and mathematics—is not of itself an evalua-
tion of the schools’ “educational quality” and should
not be presented to the public as such. The “quality”
of a school includes other relevant matters that are of
importance to teachers, students, and families—such
as emotional development, interpersonal relations,
civic education, and the inculcation of values. None-
theless, comparative data on the performance levels
achieved by students in a range of schools can be use-
ful to the teaching staff, inasmuch as the information
enriches their perception of their own work, allowing
them to locate their students’ achievement in the con-
text of that of students in other schools.
If a comparison of student performance levels is to
be valid, the students’ social background must be
taken into account. Schools’ academic results must
be compared with those of schools with a similar so-
cial composition. This is because the challenges and
difficulties of teaching students from disadvantaged
backgrounds (or those whose native tongue is an in-
digenous language) are very different from those in-
volved in teaching students from family backgrounds
marked by complete secondary and/or tertiary edu-
cation. School dropout rates should also be kept in
mind, as should student selection policies, since a
school can improve its results by excluding students
who have difficulties.
Between-school comparisons should take account
of the difference between measuring “educational
achievement” and “learning.” Strictly speaking, if
the aim is to provide schools, families, and/or the
authorities with information on the teaching capacity
of teachers and schools, assessments must measure
both progress made by students throughout an aca-
demic period (learning) and the final result (achieve-
ment). The differences between the two terms are as
follows:
Learning
can be defined as the change in each
student’s knowledge and capacities throughout the
school year. Measuring it requires two tests, one
at the start and another at the end. This approach
makes it possible to determine the progress that
each student has made.
Educational achievement
, by contrast, is mea-
sured using a single test and reflects the accumu-
lation of knowledge and capacities throughout a
student’s whole life, including the family’s cul-
tural capital and the student’s experiences in other
schools or with other teachers.
Students’ learning and educational achievement de-
pend not only on what teachers and schools do, but
also on the effort made by the students themselves,
the support that the families give to their children’s
schooling, the community and cultural context, and
education policies. Problems in education cannot
be resolved by appealing solely or mainly to market
mechanisms.
Results expressed as school rankings should be
viewed with caution. Most school rankings give a
false impression of standing. One school might be
in 1st place and another in 40th, but the difference
between their averages might not be statistically sig-
nificant. Thus, it cannot be said that one average is
really higher than the other, because the differences
The Educational Assessments That Latin America Needs|13
fall within the assessments’ margin of error. But even
when the difference in the averages is statistically sig-
nificant, it might be irrelevant in terms of the percent-
age of students who attain the expected performance
levels.
4.6 Checklist for Decision Making
Given the various options for devising and imple-
menting an assessment policy, it is essential to deter-
mine the assessment system’s characteristics before
putting it into effect (or modifying an existing one).
The following checklist can be helpful in assessment
system decision making.
What is the purpose of the assessment? Who will
use the results and to what ends? What as yet un-
known information will the assessment provide?
What are the units of analysis for the results re-
port: individual students, class groups/teachers,
schools, types of schools, subnational governing
bodies, the education system?
Based on the defined purposes of the assessment,
which is more appropriate to assess at the end of
given grades or cycles, learning or educational
achievement?
What consequences will the results have, and for
whom?
According to the defined purposes, is a census-
based approach necessary, or are sample-based as-
sessments sufficient?
What grades and disciplines should be assessed?
How often is it necessary and appropriate to un-
dertake the assessments?
The answers to these questions should be integrated
in a clear and explicit assessment plan for the short,
medium, and long term. In drawing up the plan, the
“Commandments” for Making Policy
Decisions about the Assessment System
“Do no harm.” One of the rst precepts of
the Hippocratic Oath is that physicians will
refrain from doing anything that could harm
their patients. Similarly, when planning an as-
sessment system, it is important to consider
the risks of unintended and damaging ef-
fects that the selected assessment strategy
might have on the education system whose
improvement is sought.
“Everything in moderation; nothing in ex-
cess.” Assessment cannot take primacy over
education. Too many assessments can harm
the health of the education system, espe-
cially if countries only conduct assessments
and fail to devise policies that respond to
the problems thereby uncovered.
“Do not assess in vain.”
Collecting informa-
tion every year and never analyzing or using
it should be avoided. If an assessment sys-
tem is to have an impact, the assessments
should be carried out at intervals that guar-
antee that the data can be analyzed, dis-
cussed, understood, and used. It takes time
to absorb new information and translate it
into decisions and actions. Changes in the
education system need even more time.
“Dress me slowly, for I am in a hurry.”
Deci-
sion makers in education ministries must re-
ject the false belief in easy, quick solutions.
A serious assessment program cannot be
established in three months. There is no cir-
cumstance or window of opportunity that
justies it: sooner or later, the consequences
of improvisation will become apparent. As-
sessment demands careful reection on its
purposes and uses, public discussion of what
needs to be assessed, the involvement of
various actors and dialogue among them,
the creation of technical teams that are
competent in various matters, and prior infor-
mation that motivates the actors to become
involved in the assessment.
14|The Educational Assessments That Latin America Needs
monetary costs of each option and the human re-
sources needed to implement it properly should be
taken into account. Most particularly, a balance must
be struck between the investment required to collect
the information and that needed to disseminate and
use the results. Many education ministries devote sig-
nificant sums of money each year to gathering large
amounts of data that thereafter are scarcely analyzed,
disseminated, or used. It is pointless to undertake an
initial assessment exercise if no long-term work plan
has been prepared.
5. Technical Quality Challenges of
Assessments
Once the assessment policy has been defined, it must
be implemented in line with various appropriate stan-
dards of technical quality. This entails addressing the
following challenges.
Devise a referent or conceptual framework that
stipulates precisely the knowledge and perfor-
mance deemed appropriate at the end of the grade
or educational cycle to be assessed. While such an
endeavor calls in the first place for a political debate
and political decision making, the definitions thereby
decided upon must then be adequately translated
into technical specifications and standards. The edu-
cation policy debate must be fueled by information
on recent conceptual developments regarding teach-
ing and student performance in the disciplines under
consideration.
Include activities with various levels of complexity
in the tests. Thought should be given to including ac-
tivities that call for sufficiently complex cognitive ca-
pacities that are appropriate to the challenges of the
knowledge society, as well as simple activities that
reveal the level to which less advanced students have
progressed. The activities must be guaranteed to pos-
sess a series of psychometric properties, and control-
ling them demands pilot tests and careful analysis. To
the extent possible, it is also important to expand the
use of constructed response questions, with the two-
fold aim of assessing more complex capacities and
improving the links between external assessment on
the one hand and teaching and school culture on the
other.
Design the tests by properly integrating activities
in blocks and test booklets. This is a significant and
complicated technical challenge, calling for special-
ized knowledge and experience. Particular attention
should be paid to the decision of whether to use
classical theory or item response theory in devising
and analyzing the tests; this in turn calls for modern
processing programs, well-trained analysts, rigorous
analysis, and high-level advice.
Define the cut-off scores that set the limits between
performance levels in a test. Authorities must define
a methodology to establish which of the performance
levels should be deemed acceptable for a student
completing the grade or educational cycle under as-
sessment. An acceptable performance cannot be de-
fined automatically as equal to 51 percent or more of
the maximum possible score on a test.
Devise samples that are suitable for the purposes
of the assessment. The aim here is to avoid assess-
ment exercises that are larger and more costly than is
strictly necessary and that, at the same time, have an
appropriate degree of precision. In this latter regard,
the margins of error in the assessment should be esti-
mated and reported.
Determine how to equate the assessments. Equat-
ing the assessments refers to the methodology used
to establish comparability among the results of tests
applied in different years. This is one of the most im-
portant technical challenges assessment systems face
The Educational Assessments That Latin America Needs|15
if they seek to provide information on progress and
setbacks in educational achievement over time. The
process is crucial if the assessment is to be able to in-
dicate that any possible variations respond to changes
in real educational circumstances and not simply to
changes in the measuring instruments. Answering
this challenge means taking into account statistical
considerations, assessing the same knowledge and
skills in each subsequent assessment, and keeping the
structure and length of the test comparable over time.
Produce longitudinal data that reveal how the
learning of the same group of students has evolved
over time. Production of such data demands more
than measuring achievement over time. It assesses
“learning” as change and is more apt for establishing
which school processes influence learning. This ap-
proach provides more relevant information for educa-
tion policymaking and for research.
Establish mechanisms to monitor compliance with
the standardized conditions that should prevail
during test application. Compliance with the condi-
tions in which the tests should be applied involves
complex technical matters that are often neglected.
These issues are crucial to ensuring that the informa-
tion obtained is reliable and comparable and relates
to such issues as the following:
The quality of the test administrators’ training
Establishment of quality controls during test ad-
ministration
Student motivations to take the tests
Distribution logistics
The safe return of test materials
Combining assessments with qualitative studies.
It is important to combine standardized national as-
sessments, which offer an overview of the whole sys-
tem, with qualitative studies that reveal more about
processes in the schools and classrooms. This combi-
nation of approaches is the best means of providing
rich and complex information to guide thinking about
education policies and teaching practices.
6. Establishing Assessment Units
Countries need assessment units with the requisite
capacities and resources in order to move forward
with an appropriate policy for assessing educational
achievement or learning.
It takes at least two to three years to establish a se-
rious assessment system. That much time is needed
to move ahead with the following basic processes:
To discuss, define, and make publicly known the
purposes of the assessment system, the kind of
consequences it will have, its expected uses, and
what is to be assessed
Ensuring Needed Transparency
Technical processes need, above all, transpar-
ent and accessible information. Particular em-
phasis should be placed on documenting the
technical procedures followed in
designing instruments,
estimating the precision of measurements
(and, consequently, their margin of error),
designing samples and reporting actual re-
sponse rates,
applying the tests and controlling the quality
of their administration,
dening performance levels and cut-off
scores,
equating the results and making them com-
parable with previous assessments.
16|The Educational Assessments That Latin America Needs
To design a long-term assessment plan
To assemble technical teams with the necessary
range of skills (design of tests and questionnaires,
knowledge of the disciplines to be assessed and of
their didactic requirements, curriculum and stan-
dards, sampling, logistics and quality control of
the administration process, gathering and cleaning
data, processing and analysis, scale construction,
sociocultural contextualization of the results and
analysis of related factors, data interpretation in
terms of education policies and teaching practices)
To devise instruments, test them in a pilot phase, and
ensure external supervision of these processes
The technical staff of the assessment units must
be stable over time. It takes about 10 to 15 years of
planning to develop an assessment system. A high
rate of turnover among technical staff causes a loss of
knowledge and accumulated experience in a complex
field and can discredit assessment processes in the
eyes of educators and the public.
Assessment units must be independent in report-
ing the results of their work. As with units that pro-
vide social and economic statistics, those responsible
for educational assessment and for disseminating test
results cannot be dependent on the time frames and
interests of political parties. There has been much
discussion of whether assessment units should be lo-
cated within or outside of education ministries. The
main argument for the latter approach is precisely this
need for independence and transparency. Nonethe-
less, there have been examples in the region of coun-
tries with stable and independent units within educa-
tion ministries, and other cases where the units have
been unstable and unable to consolidate their work
even though they were part of an external institution.
In fact, the institutional location is not as important
as the culture of continuity and transparency created
around assessment. Such a culture is achieved when
assessment has a clear mandate and a solid structure,
which necessitates that the assessment system be un-
derpinned by some kind of legal statute. One of the
approaches to be considered is to establish this by
law, since it calls for broad agreement (that reaches
across party lines, if possible) that allows a long-term
educational assessment plan to be put in place. As
in the economic arena, there must be a certain level
of stability: if assessment policies change constantly,
distrust grows and credibility is lost.
A solid institutional structure requires indepen-
dence and pluralism among government bodies and
technical assistance agencies, an appropriate bud-
get, and human resources that guarantee the unit
can function to the necessary degree of technical
quality. The assessment unit’s independence should
not cause it to become dissociated from education
policy. On the contrary, assessment must respond to a
political-educational project with widespread support
and should remain closely linked to other key areas of
education policy such as teacher training, curricular
development, planning and project design, program
evaluation, and research.
If standardized tests are to have an impact on edu-
cation policies and practices, new interfaces and
working methods are needed that ensure that the
various actors and decision-making spheres are
aligned. The assessment units must assume that
their work consists of more than producing data. If
their mission is to provide information to other ac-
tors, they need trained individuals with the time to
establish a dialogue with other agencies and actors
responsible for the following:
Analysis of education policy
, with a goal of im-
proving the design of the assessments, as well as
processing plans and reporting results, and taking
into account some important matters for education
policy (for instance, the sample can be designed
The Educational Assessments That Latin America Needs|17
in such a way as to make it possible to assess the
impact of specific policies or programs on certain
groups of schools)
Didactic analysis
, with a goal of interpreting the
results and student learning problems from the
perspective of teaching and the didactics of the dis-
cipline under assessment, preparing reports with
didactic significance for teachers, designing in-ser-
vice training programs on the basis of the results,
and reflecting on the links between standardized
assessments and classroom evaluations
Communication
, with a goal of designing a range
of results reports in various formats and styles tai-
lored to and understandable by various audiences
Good assessment requires investment. It is better to
do no assessment at all than to do one that is poor or
inadequate. Also, it is better to have a modest assess-
ment system whose costs are affordable and sustain-
able over time rather than an extensive and sophis-
ticated assessment that can only be carried out once
and never repeated.
Investing in assessment should be seen in terms
of the use to be made of the results rather than on
the basis of other indicators, such as cost per pupil.
The costs of assessment are low relative to national
budgets and other investment possibilities. But any
investment in assessment, be it high or low, is worth-
less if the results are not put to use.
7. Ten Recommendations on the
Assessments That the Region
Needs
Assessment must be regarded as an element 1.
linked to others in a wider set of education poli-
cies and actions. Assessment in and of itself does
not produce improvement. There must be stable
links between the domains of assessment and
those of curriculum development, teacher train-
ing, research, policy design, communications and
outreach, among others.
Assessment should reflect coherent and com-
2.
prehensive consideration on the state of educa-
tion and the means of improving it. This process
should begin with public consultation and debate
as to what students should learn and the purposes
and consequences of assessment. It is crucial to
have a constructive public discussion of the re-
sults, with a view to tackling deficiencies and in-
equities in students’ access to knowledge. This
calls for investing in communication and out-
reach as much as (if not more) in the assessment
itself—before, during, and test administration.
Assessment should help develop a sense of
3.
shared responsibility for education as a public
good. It should foster all stakeholders’ commit-
ment to education in line with their position in
the system and their area of activity. Efforts must
be made to avoid using assessment as a means
of assigning blame to specific actors for problems
uncovered.
The region’s assessment systems should gradu-
4.
ally expand the range of educational objec-
tives that are subject to appraisal. Civic educa-
tion and other subjects apart from language and
mathematics should be included in order to cover
a broader spectrum of competencies and capaci-
ties than is currently captured.
The region’s assessment systems should gradu-
5.
ally design evaluations of student progress over
time, since these can provide additional informa-
tion on the impact of education policies, actions
by the schools, and teaching practices on student
learning.
18|The Educational Assessments That Latin America Needs
An assessment system is a long-term undertak-6.
ing, and thus requires commitment on the part
of the state and careful planning of its design.
Decisions must be made about its purposes and
consequences, the curricular areas to be assessed,
topics and grades to be covered, and the periodic-
ity of assessment, among other things. Because
careful planning takes time, it is not advisable to
try to implement assessment systems over a short
period.
A good assessment system needs investment
7. ,
primarily in terms of establishing qualified teams
as well as providing sufficient economic resources
to ensure proper implementation of all the pro-
cesses involved.
The assessment system should be fully trans-
8.
parent with regard to the results and accountabil-
ity to society.
Education ministries must assume a serious and
9.
consistent commitment to assessment results,
which entails fostering dialogue on the problems
uncovered and the means for approaching them,
devising appropriate strategies to resolve those
problems, and investing the necessary resources.
The assessment system should be evaluated
10.
periodically, with the aim of analyzing the tech-
nical quality of the information it provides and
its relevance for various educational and social
actors.
Inter-American Dialogue
1211 ConnecticutAve., NW,Suite 510
Washington, D.C. 20036 USA
Tel: (202) 822-9002
Fax: (202) 822-9553
E-mail: iad@thedialogue.org
Internet: www.thedialogue.org & www.preal.org
Corporación de Investigaciones parael Desarrollo
Santa Magdalena 75, Piso 10, Oficina 1002
Santiago, Chile
Tel: (56-2) 334-4302
Fax: (56-2) 334-4303
E-mail: infopreal@preal.org
Internet: www.preal.org
PREAL was established by the Inter-American Dialogue in
Washington, D.C., and the Corporation for Development
Research (CINDE) in Santiago, Chile, in 1995 as a multiyear
initiative to build a broad and active constituency for
education reform in many countries. It has become the
leading non-governmental voice on education in Latin
America and a strong advocate for involving leaders from
civil society in education reform. Most of PREAL’s activities
are implemented by a region-wide network of expert
public policy and research centers working to promote
education reform.
PREAL seeks to improve the quality and equity of education
by helping public and private sector organizations
throughout the hemisphere promote informed debate on
education policy, identify and disseminate best practices,
and monitor progress toward improvement. PREAL’s
activities are made possible by the generous support of
the American people through the United States Agency
for International Development (USAID), by the Inter-
American Development Bank (IDB), the GE Foundation, the
International Association for the Evaluation of Educational
Achievement (IEA), and the World Bank, among others.
The contents of this publication are the responsibility of the
authors and do not necessarily reect the views of PREAL or
any of its donors.
... Over the last few decades, the conduct of student evaluations of learning has received considerable attention across the globe (OECD, 2013;Benavot and Tanner, 2007). Setting up effective systems for testing student learning with a defined policy framework, organizational structure, and suitable operational procedures are important for education reform in both established and evolving economies (OECD, 2013;Greaney and Kellaghan, 2012;Clarke, 2012;Ravela et al., 2008). Moreover, the fourth Sustainable Development Goal (SDG) for equitable quality education along with lifelong learning opportunities for all by 2030 calls for a continued and sustained focus on monitoring learning (United Nations, 2015). ...
Technical Report
Full-text available
Abstract The education sector uses learning assessment data as an accountability measure for quality education and to guide sector reform. This paper begins with an overview of assessment systems across countries in terms of their experience, coverage of school populations and subjects, and institutional responsibility. The use of assessment data in education reform in three groups of countries is then reviewed. The first group includes countries with established systems of assessment. Countries with established systems provide rich examples of how the use of assessment data can positively impact student learning reflected in higher PISA and TIMSS scores. In these countries, based on an initial and critical alignment of curricula objectives with assessment content, test data are systematically used in three components relevant to the functioning of schools– infrastructure and instructional aids management, personnel management and support, and school oversight and support. Harmonization across these three components and coherence across the entire system positively influence teaching and learning in the classroom. The second group comprises those countries with evolving assessment systems. In evolving systems, information on the status of two of the components is weak (personnel management and school oversight) and therefore, the basis and direction for using assessment data appropriately to improve teaching and learning are unclear. The third group of countries operates learning assessments in federal systems with dispersed authority structures. In federal systems, when the national level is authorized to support harmonization and coherence at sub-national levels, there is potential for the use of assessment data for accountability and improvement in sector performance. The paper ends by highlighting three facets in the reform experience of countries successful in creating harmonization and coherence for quality teaching and learning. First, the availability of appropriate and in-depth analyses on the status of infrastructure, personnel and schools facilitates and steers harmonization and coherence (balanced knowledge base). Second, depending on the history of education, technical capacity and political, socio-cultural and economic contexts, the process of harmonization is expedited by prioritizing what requires more attention than others (negotiating embedded constraints). Finally, countries exhibiting a high degree of adaptability, adopting different strategies and amounts of time to ensure coherence across the system (deliberate but adaptive sequencing) are more poised toward improved student learning. ‘The sparrow may be small, but all its vital organs are present. [A Chinese Proverb]. The nation took pains to ensure not just that all its parts were working well, but also that they are working in tandem. Having the best teachers, schools, or policies is not sufficient; it is about having a broader understanding so that the parts work together.’ Pak Tee Ng, Associate Dean of Leadership and Learning at the National Institute of Education, Singapore.
... Indeed, there is a growing concern over the increasing dominance of ILSA data for country comparisons and evidence-informed policy-making (Grek 2009). Another issue important for the LAC region is the relative importance of evidence garnered from NLSA and ISLAs (Ravela et al. 2008;Wagner et al. 2012). ...
Article
Full-text available
This paper presents a single-country case study of the use of large scale assessment (LSA) data to generate actionable knowledge at school and system levels. Actionable knowledge is data-informed insight into school and system processes that can be used to direct corrective action. The analysis is framed from the perspective of the country’s evolving national policy on data use for educational improvement between 1990 and 2013. Trinidad and Tobago first participated in international large scale assessments (ILSAs) in 1991 but also developed a centralized system of national large scale assessments (NLSAs) in 2004. Analyses of both datasets consistently pointed to low quality and high inequality as the main actionable issues in the education system. NLSA data also hinted at notable variation in performance across schools and education districts. Analyses for and of policy point to the need for multiple school performance measures to better inform site-based, formative action. Over the period, actionable knowledge appears to have had greater impact at school level, with evidence being used by some low-performing schools to improve. However, at the system level, the frequent non-use and misuse of actionable knowledge suggest the need to promote and strengthen structures and processes related to evidence-informed policy-making.
... In almost all cases, P. Ravela et al. (2009) assumed that assessment can serve: (1) as a basis for better-grounded education policies; (2) as a means of improving the management of education systems; and (3) as an instrument to foster collaboration and continuous learning within those systems. ...
Article
Full-text available
Assessment has been studied by many researchers and educators from the prism of educational theories which means a purely instructional or pedagogical analysis of assessment. But there is an open space in which learning assessment could be further scrutinized using sociological perspective, so that we could explore and expose the social roles of assessment. This discussion is engaged in the analysis of the social consequences of assessment and its resistances from the various participants of teaching and learning process. Assessment has been widely used for accountability, control, and sorting mechanism of society to distribute the limited social positions that are available. Thus, only those students who are properly equipped with knowledge, values, and competencies are the ones who excel in the assessment devices. Students in the lower echelon of society are left with limited opportunities for social mobility and employment, due to their limited capacity to pass through this filtering machine called assessment. So, there some manifest and latent resistances that students as well as teachers about the negative consequences of assessment. Thus, assessment is not only a system to improve learning, account teachers, and schools but rather is an instrument of academic segregation and tracking. Assessment should be viewed beyond the walls of classroom by looking at the close but intricate linkage of assessment and society. Since assessment is a social fact, therefore, we need to consider how it is developed and practiced in everyday life of teachers and students. Education is never neutral, according to Paulo Freire, and also assessment is can never be neutral. It is resisted by some sector of society, but it is still a very strong mechanism to classify and rank people in a society. It serves a purpose in perpetuating a society's present order as well as a site to perturb the existing order by its results.
... See Wagner (2011) for an in-depth review. 9 See also: Siniscalco, 2006; Ravela et al., 2008; Wolff, 1998. 10 ...
Article
Full-text available
The following values have no corresponding Zotero field: PB - Teachers College, Columbia University. International and Transcultural Studies, PO Box 211, 525 West 120th Street, New York, NY 10027
Chapter
Based on an in-depth, ten-year study, this novel book examines the reform of Kazakhstan's education system, from the initial plans and models of change, through to the implementation at all stages and places in the education system. Through an exploration of a wide range of data, it maps the problems, models, challenges, interventions, and successes of educational change. It covers the viewpoints of all stakeholders involved – policy makers, teachers, regional officials, head teachers, parents, and pupils – to provide a comprehensive assessment of the perspectives of people at all levels. It will be invaluable to those interested in the implementation of radical development and change, and it is essential reading for researchers and students in education reform and education policy, as well as teachers and educational professionals. This title is part of the Flip it Open Programme and may also be available Open Access. Check our website Cambridge Core for details.
Article
Full-text available
The present study attempts to evaluate the availability of assessment facilities at government special education schools and to make a comparison of these available facilities among assessment professionals. Professionals (Speech Therapist, Psychologist, and Audiometerist) working at government special education schools of Faisalabad and Multan are the population of the study. The sample of the study is N=81 professionals (n= 39 Speech Therapists, n=36 Psychologists, and n=06 Audiometrists) selected through Simple random sampling technique. Assessment system evaluation questionnaire (ASEQ) is developed as a tool of study and reliability of the questionnaire is 0.886. It is found out after statistical analysis that the psychologist and speech therapist working at government special education institutes are in a greater amount as compare to audiometrists who are very few and it is found out that there is the significant difference among professionals related to the availability of assessment facilities for special students in government special education institutions. It is also analyzed that problems faced by the professionals regarding the assessment of special students at government special education schools have no significant difference. It is concluded that there should be arrangements for the availability of assessment material related to all assessment professionals in the schools so that they can do the best assessment that can be helpful in rehabilitation of special students.
Article
The present study attempts to evaluate the availability of assessment facilities at government special education schools and to make a comparison of these available facilities among assessment professionals. Professionals (Speech Therapist, Psychologist, and Audiometerist) working at government special education schools of Faisalabad and Multan are the population of the study. The sample of the study is N=81 professionals (n= 39 Speech Therapists, n=36 Psychologists, and n=06 Audiometrists) selected through Simple random sampling technique. Assessment system evaluation questionnaire (ASEQ) is developed as a tool of study and reliability of the questionnaire is 0.886. It is found out after statistical analysis that the psychologist and speech therapist working at government special education institutes are in a greater amount as compare to audiometrists who are very few and it is found out that there is the significant difference among professionals related to the availability of assessment facilities for special students in government special education institutions. It is also analyzed that problems faced by the professionals regarding the assessment of special students at government special education schools have no significant difference. It is concluded that there should be arrangements for the availability of assessment material related to all assessment professionals in the schools so that they can do the best assessment that can be
Article
Purpose – Although high-quality comparative data from international assessments are now more widely available, to what extent is that data being used to trigger, inform, and direct educational change in non-Organization for Economic Co-Operation and Development (OECD) countries? The purpose of this paper is to develop a theoretical framework to guide a case analysis of Trinidad and Tobago's system response to international assessment data. Design/methodology/approach – This is a single-nation explanatory case study using data from policy documents and elite interviews. Findings are generated through inductive thematic analysis. Findings – The four emerging themes were: first, weaknesses in the national evaluation system; second, policy-making practices not attuned to data; third, lack of collaboration and stakeholder involvement; and fourth, challenges in accessing and using data. Findings suggested that data rarely acted alone to trigger system change. Critical to initiating and sustaining effective data use for system reform were policy-making contexts and mental maps of system leaders, which in this context acted as barriers. Respondents believed that greater strategic leadership from politicians and technocrats could ensure data-informed systemic change. Research limitations/implications – The study focuses upon data use and data-driven decision making for whole system reform within a single country context. However, it advances theory that might be applied to other non-OECD cases. Originality/value – The findings contribute to the refinement of a conceptual model explaining data-driven system reform applicable to non-OECD contexts. The role of system leaders when using international assessment data is clarified.
ResearchGate has not been able to resolve any references for this publication.