After delineating the major rationale for computer education, data are presented from Stage 1 of the IEA Computers in Education Study showing international comparisons that may reflect differential priorities. Rapid technological change and the lack of consensus on goals of computer education impedes the establishment of stable curricula for “general computer education” or computer literacy. In this context the construction of instruments for student assessment remains a challenge. Seeking to anticipate and measure what educators will view as the essential computer-related abilities for students in the mid-1990s, the second stage of the IEA Computers in Education Study developed a student assessment instrument grounded in the perspective of “functionality,” student prerequisites to functioning effectively with practical information-related tasks. The threat of test obsolescence as well as philosophical differences among the experts in their goals for general computer education challenged traditional test construction procedures. The resulting content objectives and test procedures can serve as guideposts for research and planning in computer education.
The article examines the processes and challenges involved in conducting participatory research and evaluation in schools, by looking at a case study of a collaborative evaluation research that was conducted in a secondary school in Israel by two researchers and three teachers. It describes the experiences of the research team while conducting the study, the processes that developed during this collaboration, and the kinds of knowledge that emerged over the two years of teamwork. It also describes the features of the relationships between the various individuals who participated in the study. Implications for evaluation in schools are presented.
This study illustrates the development and validation of an admission test, labeled as Performance Samples on Academic Tasks in Educational Sciences (PSAT-Ed), designed to assess samples of performance on academic tasks characteristic of those that would eventually be encountered by examinees in an Educational Sciences program. The test was based on one of Doyle's (1983) categories of academic tasks, namely comprehension tasks. There were 108 examinees who completed the test consisting of nine comprehension tasks. Factor analysis indicated that the test is basically unidimensional. Furthermore, generalizability analysis indicated adequate reliability of the pass/fail decisions. Regression analysis then showed that the test significantly predicted later academic performance. The implications of using performance assessments such as PSAT-Ed in admission procedures are discussed.
This article reports findings obtained from a large-scale national study (299 schools; 2761 students) that examined academic achievements of immigrants in Israeli schools. It focused on two distinct groups of immigrant students – those from the former USSR and from Ethiopia, in two subject areas – mathematics and academic language (Hebrew), and in three grade levels – 5, 9 and 11. The scores of the immigrant students and those of a parallel group of native-borns were compared and analyzed. The findings demonstrate differences in achievements between the groups. The scores also demonstrate that immigrants require a substantial number of years to reach achievement levels similar to those of students who were born in Israel in academic subjects, specifically, 5–7, 9 or 11 years in mathematics, and 5–7, 8 or 11 years in academic Hebrew, depending on the grade levels and the groups. The study discusses the implications of using large-scale evaluation of educational achievement for educational policy and evaluation designs.
During the 1990s, Spain experienced a phenomenon that is slightly unusual in educational systems: the coexistence of two secondary education models and one single tertiary education model. The overall purpose of this article has been to study the possible differences in the academic performance of students in university education in terms of the model followed in secondary education. We carried out a comparative monitoring of various cohorts of students in the Universities of Barcelona, Oviedo, the Basque Country, Salamanca and Zaragoza (Spain). These institutions are located in regions of differing size and nature, and are representative of the Spanish university system as a whole. In all, we analysed the academic performance of nearly 150,000 university students. The main conclusion we reached is that there are no differences in student academic performance at university in terms of the secondary education system followed.
This study investigates how different types of prior knowledge influence student achievement and how different assessment measures influence the observed effect of prior knowledge. We introduce a model of prior knowledge that distinguishes between different types of prior knowledge and uses different assessment measures to assess different types of knowledge. The sample consists of 202 mathematics students who completed the prior knowledge test during the first lesson. The student achievement was measured by the final grade on the course. The results indicate that the type of prior knowledge makes a difference: The measures assessing procedural knowledge predicted the final grades best whereas measures assessing declarative knowledge did not predict final grades. Additionally, previous study success was the best predictor of student achievement. These results are discussed in relation to assessment measures and their implications for practice.
The studies reviewed here support the proposition that more accurate and informative measures of the home educational environment are possible than the status characteristics typically used. The stability of SES measures and the ease with which this type of data may be collected have promoted the continued use of static variables as measures of family background. But as we have indicated in this review, studies have shown that what parents do rather than who they are is the more important determinant of the home's influence on the child's achievement. Thus, the emphasis is on the process variables in the home, although the static variables are still relevant to gaining a complete picture of the home environment and its influence. The home-based intervention studies further support these results as well as the alterability of process characteristics. The alterability of home processes is an important factor in moving away from the restrictions of prediction and classification imposed by the traditional static measures of the home environment.
In general, studies on gender and mathematics show that the advantage held by boys over girls in mathematics achievement has diminished markedly over the last 40 years. Some researchers even argue that gender differences in mathematics achievement are no longer a relevant issue. However, the results of the Trends in Mathematics and Science Study of 2003 (TIMSS-2003), as well as the participation rates of girls in (advanced) mathematics courses, show that in some countries, such as the Netherlands, gender equity in mathematics is still far from a reality. Research on gender and mathematics is often limited to the relationship between gender differences in attitudes toward mathematics and gender differences in mathematics achievement. In school effectiveness research, theories and empirical evidence emphasize the importance of certain school and class characteristics (e.g., strong educational leadership, safe and orderly learning climate) for achievement and attitudes. However, there is little information available at to whether these factors have the same or a different influence on the achievement of girls and boys. This study used the Dutch data from TIMSS-2003 to explore the relationship between school- and class characteristics and the mathematics achievement and attitudes for both girls and boys in Grade 4 of the primary school. The explorations documented in this paper were guided by a conceptual model of concentric circles and involved multilevel analyses. Interaction effects with gender were assessed for each influencing factor that turned out to have a significant effect. The results of these analyses provide additional insight into the influence that non-school-related and school-related factors have on the mathematics achievement and attitudes of girls and boys.
The purpose of the present study was to improve a multivariate multilevel model in the research literature which estimates the consistency in the rates of growth between mathematics and science achievement among students and schools. We introduced a new multivariate multilevel model via a latent variable approach. Data from the Longitudinal Study of American Youth (LSAY) provided scores on basic skills, algebra, geometry, and quantitative literacy as indicators of the latent variable mathematics achievement, and scores on biology, physics, and environmental science as indicators of the latent variable science achievement. Using this multivariate multilevel model with latent variables, we examined the relationship between growth in mathematics and science achievement during middle and high school among students and schools, and we demonstrated that such a model was more sensitive to this relationship.
Opportunity to learn is considered an important contributing factor in learning outcomes. In some of the latest international comparative studies of mathematics achievement, such as SIMS and TIMSS, painstaking efforts have been made to find out what the participating students' opportunities to learn mathematics had been. However, there have been problems with relating the findings to student outcomes. This article approaches opportunity to learn in three different ways. Among these approaches, an item-based analysis of textbook contents resulted in fairly high correlations with student performance at the item level in TIMSS 1999. This implies that even a quite simple analysis of textbooks can produce valuable information when looking for explanations for student achievement in mathematics.
The topic of teacher credentials and student performance is revisited in an international setting using the TIMSS-99 data. The lack of consistent positive link between credentials and performance can be explained via three routes: measurement problem of “teacher quality” input, measurement problem of “student outcomes”, and the production function form that is assumed to link the input and the output. Although there is a small literature focusing on student outcome measurement problems, suggesting the use of cognitive achievements rather than test scores, in most cases those cognitive measures are nothing but math and science scores. This study contributes to the literature by borrowing from the measurement and psychometrics theories to decompose single scores into three categories of cognitive abilities. The hypothesis is that teachers may play a crucial role in the development of some student cognitive skills while not in the others. Using a “rule-space” model, this article identifies three cognitive skills: the process skill, the reading skill and the mathematical think skill. The study finds that: (1) in general teacher credentials have no effect on any type of cognitive skill development as well as on the test score, and (2) the within-teacher variance of student performance is much larger than between-teacher variance in Japan and Korea, whereas the reverse is true in the US and the Netherlands. The phenomenon of “private tutoring” is quoted as an explanation of this pattern.
Investigations of children's learning contexts have typically shown that, in relation to family environments, school measures have weak associations with children's achievements. Much of the research is restricted, however, by an inadequate conceptualization of both contexts or, if one of the environments is assessed by proximal social-psychological variables, then the other is generally defined by gross social indicators. In the present study a selective evaluation of prior environmental investigations is presented. Then two analyses are considered which attempt to overcome some of the restrictions of previous research. In the two studies regression surfaces are plotted to examine relations between school environments and children's achievements at different family environment levels, with both environments defined by proximal social-psychological variables.
One of the strongest traditions during the past decade of classroom environment research has involved investigation of the predictability of students' cognitive and affective learning outcomes from their perceptions of psychosocial characteristics of their classrooms. Moreover numerous research programs involving many thousands of students from various nations have provided convincing and consistent support for the incremental predictive validity of student perceptions in accounting for appreciable amounts of variance in learning outcomes beyond that attributable to initial student characteristics such as pretest performance and general ability (see Walberg, Singh and Rasher, 1977; Haertel, Walberg and Haertel, 1979; Walberg, 1979; Fraser, 1980; Walberg and Haertel's paper in this issue).
This paper investigates the usefulness of a verbal protocol approach in examining the underlying construct of a cloze test, i e. the reasons that a test writer had for deleting some lexical items from a passage to construct a cloze test. The informants were asked to “think aloud” while they were doing the cloze test. Observation of the informants verbalising their thoughts revealed inadequacies in using this approach. To compensate for this inadequacy, retrospective interviews in which the informants were asked about their choices after their verbal protocols, were conducted. The analyses of the informants' think-aloud and their retrospection showed that in their think-aloud they could not verbalize all the mental processes they used in taking the test. The results, however, suggest verbal protocols as useful instruments for collecting a particular type of data which is inaccessible when using other approaches.
Large scale surveys in education have to face non-response issues that might bias the results. Non-response can occur at three levels: (i) a school refuses to participate, (ii) a sample student fails to participate and (iii) a participating student refuses to answer a particular question. Until now schools and student non-response have been counterbalanced by a non-response weight adjustment. This assumes that both the school and student non-respondents have similar characteristics to the school and student respondents, respectively, within classes. In this article results of analyses conducted on the Student Tracking Form data of the OECD/PISA 2000 survey are presented. The non-randomness of student absenteeism or refusal is demonstrated. Then, a simulation that compares the relative efficiency of the student weight adjustment with a multiple imputation method is presented where the superiority of the multiple imputation method, in particular for educational systems with a small school variance is shown. Finally, the multiple imputation method applied to some PISA 2000 countries identifies biases that become substantial in a longitudinal perspective.
Data from the Health Behavior in School-Aged Children Study (Currie, Samdal, Boyce, & Smith, 2001) were used to analyze the differences in perceptions of educational experiences among over 10,000 sixth to tenth graders of different grades, genders, races and ethnicities. The relationships between students’ evaluations of their school experiences and their perceptions of their achievement were also examined. The results indicated that older students tended to feel more negative about their educational experience than younger students. Male students tended to have more negative attitudes than female students. African American students reported more negative evaluations of their school environment, but tended to report liking school more. Perceptions of achievement were associated significantly with liking of school and with perceptions of teacher caring.
This article employs the data collected during 1970 in Australia as part of the International Association for the Evaluation of Educational Achievement (IEA) Six Subject Survey, in a model which posits that affective factors mediate the influence of student background on performance at high school. The data used is that for Population II (students between the ages of 14.0 and 14.11) and is augmented by follow-up data on the same students two years later. This second data set was collected by the Australian Council for Educational Research and includes information on students still at school and those who had left school. Only those students who were at school at the time of both surveys are included in the analysis. An analysis of the data on school leavers has recently been published by Rosier (1978). As Australia collected data on only one of the six subjects in the survey program, this analysis is limited to performance in that subject-science.
Given South Africa's divided past, it is imperative to improve educational outcomes to overcome labour market inequalities. Historically white and Indian schools still outperform black and coloured schools in examinations, and intraclass correlation coefficients (rho) reflect far greater between-school variance than for other countries.SACMEQ's rich data sets provide new possibilities for investigating relationships between educational outcomes, socio-economic status (SES), pupil and teacher characteristics, and school resources and processes. As a different data generating process applied in affluent historically white schools (test scores showed bimodal distributions), part of the analysis excluded such schools, sharply reducing rho. Test scores were regressed on various SES measures and school inputs for the full and reduced sample, using survey regression and hierarchical (multilevel or HLM) models. This shows that poor schools were least able to systematically overcome inherited socio-economic disadvantage. Schools diverged in their ability to convert inputs into outcomes, with large random effects in the HLM models. Outside of the richest schools, SES had only a mild impact on test scores, which were quite low in SACMEQ context.
One of the most important components of teacher education is the practical part, the Practicum, and assessment of the candidates’ performance plays a major part in forming the future generation of teachers. Little is known about the extent of agreement between the two main actors in the Practicum, the candidates and the school-based teacher educators. The aim of this paper is to add information about a rather blurred area of assessment in teacher education. The findings indicate there is a considerably extent of disagreement about assessment in the Practicum between the mentors and the candidates. It is suggested that instead of seeing the disagreements merely as obstacles to valid assessment, they can be exploited to initiate professional learning for the candidates.
This article presents a method for computer-aided tutor evaluation: Bayesian Networks are used for organizing the collected data about tutors and for enabling accurate estimations and predictions about future tutor behavior. The model provides indications about each tutor's strengths and weaknesses, which enables the evaluator to exploit strengths to the benefit of the university and offer advice for tutors’ improvement. It also allows the evaluator to make hypotheses about potential tutor approaches and test the effect of such approaches on the educational procedure in advance. The article briefly discusses Bayesian Networks and introduces a model that has been used at the Hellenic Open University for aiding tutor evaluation.
Text anxiety research in West German schools is explored and measures currently used are described. Test anxiety is examined in relation to academic achievement research and theory. Distinctions between state and trait anxiety and between worry and emotionality are discussed. Test anxiety is investigated both in the traditional tripartite school system as well as in the non-traditional unitary comprehensive school. Results of several longitudinal studies point to the findings that there are socialization effects as explained by reference group theory and that these effects also are specific to the particular learning environment which the students experience in each school system. Future directions of test anxiety research in West German schools are explored.
From this evidence, it is clear that tutors do make assessments of personality during interview, and that this assessment determines selection. In general, their responses on the forms show that this is done with some care. Tutors try to put candidates at ease and give every opportunity for them to display their best; for example a number of forms make reference to initial nervousness of candidates, but this made no difference to the final outcome and ‘nervous’ is only included in the categories if it refers to a specific and persistent quality (e.g., ‘too nervous a disposition’). Although offset by unfavourable judgments and often qualified, all rejected candidates had something positive said about them. In general, the remarks about rejected candidates are fuller and more individual than those for accepted candidates.The expressions tutors record on the interview form may differ in detailed wording, but the kinds of qualities laid down by the DES are measured and the students selected are perceived to have those qualities to a greater degree than those rejected. The insights which can be gleaned from comparing the judgments made at interview with the outcomes of the course indicate that not only do interviewers take this responsibility seriously they also carry it out well in respect of subsequent performance in practical teaching.
Approaches utilized to appraise student progress depend upon the philosophy of education involved. Each specific philosophy uniquely determines that which learners should acquire.The testing and measurement movement stresses the utilization of predetermined objectives written in measurable terms. The objectives are written prior to instruction of learners. With appropriate learning opportunities, a student either does or does not achieve one or more precise objectives. Measuring student progress against the stated objectives emphasizes the concept of criterion referenced testing (CRT).The testing and measurement movement also advocates using norm referenced tests (NRT). Students are spread out on a continuum from highest to lowest based on test scores. Predetermined objectives tend not to exist when utilizing norm referenced tests to measure student achievement. Norm references tests spread students' results in terms of test scores much more so than criterion referenced testing. Students attempt to attain predetermined objectives with CRTs. The measurably stated objectives represent absolute standards. A high number of students might well achieve the measurably stated objectives, as the teacher usually intends.Self-evaluation by the student is an opposite approach to appraisal of learner progress as advocated by the testing and measurement movements. With self-evaluation, responsibility rests with the learner him/herself to acknowledge strengths, weaknesses, and modifications to attain at a higher level. Learners, when evaluating themselves, need to perceive the processes and products completed from the frame of reference of personal improvement. The truth resulting from evaluation may well reside within the student. Subjectivity in results is to be expected, since open-ended criteria are utilized to appraise progress. The tendency here will be not to utilize objective tests to ascertain progress. With self-evaluation, the student might well perceive increased purpose in assessing the self. The teacher is a stimulator and initiator guiding the self-evaluation process.Idealism advocates students' achievement in mental development. Mental maturity here is prized more highly than affective and psychomotor objectives. The affective dimension is salient to the point that learners attain well academically and intellectually. Students' attaining vital concepts and generalizations is of utmost importance to an idealist. To appraise learner progress effectively, the teacher must evaluate student growth in achieving worthwhile subject matter content, consisting of vital broad ideas. Discussions and essay tests, in particular, assist the idealist teacher to determine student acquisition of subject matter.Experimentalists depend upon teacher observation, basically, to evaluate student progress. The experimentalist teacher evaluates students in life-like situations in which they select information and solve problems. Hypotheses, tentative in nature, attempt to provide answers to identified problems. Since each hypothesis is to be tested within a social context, modifications or revisions may need to be made.Perennialism is a philosophy of conservation, rejecting a continually changing environment as identified and defined by experimentalists. The great ideas of thinkers of the past provide subject matter content. The abstract and academic are preferred to the concrete and the practical. Transitory ideas from the past have no place in a perennialist's curriculum. Rather, content must remain salient, vital, and significant as the decades and centuries pass. Endurance in time and in diverse geographical regions characterize that which is classic. Perennialists emphasize the liberal arts and general education for all. Preparing for jobs, careers, and the professions, has no place in such a curriculum. Vocational needs are not to be emphasized on the elementary, high school, or baccalaureate degree level, but only on graduate levels of study which should prepare the student for a niche in the world of work. Prior to that time, however, common learning should be acquired by students in the form of a liberal arts education.Liberal education, which is non-vocational, needs to emphasize as objectives of instruction the development of the mind towards maturity so that the great ideas of the past may be understood and accepted. The classics provide the intellectual system of subject matter which is offered to students. The content needs to be challenging intellectually and enable retention of major concepts and generalizations.
Pupils’ attitudes influence both learning and teaching processes and affect the way pupils will engage with art as adults. This article introduces an attitude scale, the Attitude Scale for Art Experienced in School (ASAES), which comprises four subscales: enjoyment, confidence, usefulness, and support. A three-step procedure was followed for the construction and validation of the scale which was administered to 420 primary school pupils in Cyprus. The scale's psychometric properties are evaluated through Confirmatory Factor analysis. The findings indicate that teachers’ art specialisation and attitudes towards art teaching, pupils’ perceived competence and pupils’ gender are three important variables that influence the formation of pupils’ attitudes. Important interactions between these variables are also reported.
Pupil monitoring systems support the teacher in tailoring teaching to the individual level of a student and in comparing the progress and results of teaching with national standards. The systems are based on the availability of an item bank calibrated using item response theory. The assessment of the students’ progress and results can be further supported by using computerized adaptive testing where the items selected from the item bank are targeted at the specific ability level of the student. The present article discusses psychometric issues of pupil monitoring systems, such as ability estimation, the optimal construction of tests from the item bank and monitoring of progress.
