Article
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

This paper examines the rise of value-added as a measure of quality in education. As a point of departure, the paper begins with an analysis of the rise of the concept of quality in education and discusses how, at times, various contradictory determinants of quality have managed to influence the evaluation and assessment frameworks of most countries. Leading on from this, the second part of the paper provides a discussion on the use of value-added as a determinant of quality in education. Finally, the study concludes with a discussion on the challenges relating to the introduction of value-added into the Irish education system, a development which will, arguably, become a contentious educational reform initiative within the future landscape of Irish education.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Used well, various researchers have suggested that data can lead to school improvement in terms of increased student achievement Schildkamp and Kuiper 2010;. A number of authors have also written on the significance of data in relation to decision-making in the context of school selfevaluation and improvement (Brown et al. 2016;Schildkamp and Visscher 2014;Bernhardt 2013;Matthews and Lewis 2009;Earl and Katz 2006;MacBeath 1999). Analysing school data, such as student attainment scores, can be a 'tin opener' with which to explore the inner life of the school (MacBeath 2013). ...
... Longitudinal data allows schools to monitor trends in student progress over time, providing key information on the effectiveness of teaching strategies and curricular programmes. Longitudinal data can also help to improve planning by identifying where students are in their programme in relation to comparable students in previous years, help form performance benchmarks or targets and enhance schemes of work (Brown et al. 2016;Sun et al. 2016). Mandinach and Jackson (2012) define DIDM as 'The process by which an individual collects, examines, and interprets empirical evidence to make a decision' (p.27), whereas Schildkamp et al. (2013b) refer to DIDM in an educational context: 'information that is collected and organised to represent some aspect of schools' (p.10). ...
... Although a significant amount of school supports exist, there remains an issue around the availability and integration of data in Irish education. There is relatively little statistical data about the Irish education system as a whole and, currently, there is very little data regarding standardised tests of ability scores of students at post-primary level, with which comparisons can be made (Brown et al. 2016). Gilleece (2014) also found considerable limitations to the datasets available on the Irish education system (for example, standardised student achievements, population profiles, student backgrounds) as most use samples of the student population only, longitudinal data is sparse, and most of this data is irreconcilable with, for example, the state examinations data. ...
Article
Full-text available
The concept of data-informed decision making (DIDM), a term used interchangeably with data-driven decision making (DDDM) and data-based decision making (DBDM), is relatively new to Irish education and the school planning process. This research sought to clarify what data principals use and how they use that information for school improvement considering new school self-evaluation requirements. The paper begins by charting the rise internationally of data use in school planning, decision making and accountability. It proceeds to describe the policy context in this area in Ireland and then reports recent research with school leaders around how data is collected and used in their work. Although the paper focusses on Ireland, it is tentatively suggested that school leaders, teachers and policymakers in other countries, and there are many, which have come late to the expectation that school improvement and accountability should be heavily data-informed may find the efforts of Irish principals in this regard of interest.
... As stated by OECD (2016a), 'Integration policies, and extra support targeted towards immigrant families and children, can make a significant difference in how immigrant students fare in their host communities' (p.16). Thus far; however, with numerous descriptions of cultural neglect such as the inattention given to linguistic interdependence (Cummins, 1979), the lack of culturally responsive leadership in schools (Brown, McNamara, O'Hara, Hood, Burns & Kurum, 2017), and the supremacy of standardized testing (Brown, McNamara & O'Hara, 2016;Padilla, 2001;Young, McNamara, Brown & O'Hara, 2018,); educational outcomes for migrant children have not always been as uniformly positive compared to that of their native counterparts (Brown, 2007;Griner & Stewart, 2013). The OECD Programme for International Student Assessment (PISA) starkly illustrates this point. ...
... Proposals for reform of the Leaving Certificate have also been suggested where a greater emphasis is also being placed on key skills and assessment for learning (Burns, Devitt, McNamara, O'Hara & Brown, 2018;NCCA, 2005). However, given the initial resistance to junior cycle reform in Ireland (Brown, McNamara & O'Hara, 2016), that is teachers devising and grading student assessments, reform efforts at this level have been delayed for the purpose of system level acceptance of new junior cycle assessment arrangements. Finally, as a unique situation to Ireland, almost all students in Ireland sit what is referred to as a Mock Junior and Leaving Certificate examination in rehearsal and preparation for the actual certificate examinations that occur at the end of each cycle. ...
... Acknowledging that there are varying conceptions of quality and indicators for educational achievement such as access to education, participation, progression, and youth unemployment (Brown et al. 2016;Caspersen, Smeby & Olaf Aamodt, 2017;Scheerens, Luyten & van Ravens, 2011); analysis of PISA test scores (OECD, 2000(OECD, -2017 demonstrates that, in most OECD countries, there have been constant achievement gaps between migrant and non-migrant children in the areas of Reading, Mathematics and Science. Bilgili, Volante & Klinger (2018), in reference to PISA 2015 (OECD, 2016a), also state that the majority of first and second-generation migrant students performed worse than those students without a migrant background. ...
Article
Full-text available
Purpose: The purpose of this research as part of an Erasmus+ funded project tilted Aiding Culturally Responsive Assessment in Schools (www.acras.eu) is to provide an exploratory analysis of survey responses related to culturally responsive assessment policies, professional development and practices that were administered to school principals in four European countries (Austria, Ireland, Norway and Turkey). Research Methods: The research method used in this study was a quantitative comparative analysis. A purposeful sampling strategy was adopted based on geographical spread of the participants in Austria, Ireland and Turkey. The survey was also administered to all principals in four out of eighteen Norwegian counties. Descriptive analysis and nonparametric analysis were used. Findings: There is evidence to suggest that the foundations for culturally responsive assessment practices are beginning to take shape albeit varying degrees of difference in each country. However, the survey results also indicate the need for training and professional development, and this study also implies that not enough emphasis is being placed on culturally responsive assessment despite the rhetoric that espouses interculturalism. Implications forResearch and Practice: The research points the need for upskilling in culturally responsive leadership as well as the development of an overarching culturally responsive assessment framework and toolkit that can be used by policy makers and schools to allay the various interpretations of what it means to satisfy the assessment needs of teachers and students with migrant backgrounds. Keywords diversity, culturally-responsive pedagogy, fairness, assessment, school assessment policies, migrant background students
... The unwavering influence of supranational organisations on educational policy and practice in Ireland is significant, such as the OECD, as outlined by both Mac Ruairc (2010) and Brown et al. (2016), among others. We maintain that this influence also extends to educational reform connected to EIP. ...
... On the one hand, the drive for the more frequent use of data for EIP has resulted in the various enabling actors and instruments for data-Informed decision making such as: The Department of Education and Skills data-informed Framework for School Self-Evaluation (DES, 2016a); the professional learning support provided to schools by various organisations such as the Professional Development Service for Teachers and the Centre for School Leadership (CSL); and the more widely available use of data for standardised testing (Brown, McNamara and O'Hara,2016). On the other hand, whilst the varying factors associated with the use of data in schools has been extensively researched by, for example, Schildkamp et al. (2014) and Young et al. (2018) among others, in the case of Ireland, a significant barrier concerning data use in EIP relates to the value placed on the types of data that exist and the purposes for which they are used (Murphy, 2020a). ...
Chapter
Evidence-informed practice (EIP), broadly conceived as a data and research-based approach to enhance practice, has recently come to the fore of the Irish education system. With changes to the structure and duration of professional education over the last decade, most notably Initial Teacher Education (ITE) programs, coupled with the implementation of a school leadership framework upon which a mandatory data-informed school improvement process of school self-evaluation (SSE) is based, multiple reforms connected to EIP have been introduced. Furthermore, in terms of compulsory education, assessment practices at the lower secondary level have also been significantly reformed. EIP has now become a core element of almost all educational reform initiatives in Ireland. This is a remarkable achievement given that prior to the Education Act (Government of Ireland, 1998) the conception that data and research-informed decision-making should form a core component part of school life was rarely conceived if at all in the policy discourse of educational reform. We draw on the Malin et al. (2020) interpretation of Hoods (1998) social cohesion/regulation matrix to describe and classify the Irish system. The chapter concludes with a discussion of key lessons for policy and practice based on Ireland's experience over the course of the last decade.
... It is interesting to note that as the school evaluation system has evolved and has become an excepted facet of school life, it has undoubtedly acquired elements which left-wing scholars might well perceive to be part of the neoliberal agenda. These include a much greater emphasis on evidence and data, including examination results and standardised test scores, both in setting targets for individual students and in the making of evaluative judgements about the performance of schools, and also in the increasing recognition of the rights of parents and students to have an input into school evaluation (Brown et al., , 2016a(Brown et al., , 2016b. The next section will chart, through a close analysis of policy documents, the gradual evolution of the school evaluation system since 2003. ...
... How can we improve?' This is similar enough to the existing guidance on conducting SSE in Ireland and other jurisdictions, except that, crucially, schools were now told that between 2012 and 2016 their SSEs must focus each year on literacy and numeracy and one other curriculum area, and in each case must result in a concise school improvement plan, with clear and measurable targets (Brown et al., 2016b). ...
Article
Full-text available
In this paper, we provide an overview of the development of school inspection in Ireland over the past twenty years using the analytic and critical lens developed by Richard Boyle in partnership with the current authors. The paper is fundamentally a reflection on the nature, purpose and operation of evaluation in the Irish public sector through the lens of education. The paper provides a historical overview of developments in the linked areas of school evaluation and inspection, and goes on to explore how the implementation of this mode of quality assurance has influenced, and been influenced by, a wide range of policy actors. The argument made is that education has embedded a culture of evaluation in a unique yet systemically resonant manner and that a reflection on this reality will help illuminate our understanding of the role of evaluation across the public sector as a whole.
... Time barriers were considered to be particularly challenging for both staff and parents, and to a lesser extent, students. Previous research reports that SSE is time consuming (Brown, McNamara, and O'Hara 2016) and often constrained by time (O'Brien et al. 2019). While there is a desire in schools for forms of training to assist and facilitate the inclusion of parents and students in SSE, the lack of current provision is a concern. ...
Article
As with school self-evaluation in most European countries, the Irish education system now promotes the involvement and inclusion of stakeholders such as parents and students in the evaluation process. Yet, in the Irish context, there is limited research exploring the role of these stakeholders in this internal mode of school evaluation. To address this lacuna, this research draws on a national survey of post-primary school principals and interviews conducted with 109 stakeholders now placed at the centre of the evaluation process: school staff, parents and students. While there are some optimistic indications in the data, this research highlights that only slight progress has been made in terms of including parents and students in school self-evaluation in Ireland. The data presented in this paper corroborate that many age-old obstacles in the Irish context still exist and continue to dominate. Ultimately, this research concludes that changes in policy do not necessarily produce changes in practice.
... School evaluation has become a key driver in the attempt to improve the quality of performance across the continuum of education (see, for example, Brown et al., 2016). In this context, Barber et al. (2010) in Mckinsey sparked a debate about a lack of improvement in many education systems. ...
Article
The core theory of polycentric inspection is that when schools reach a certain quality threshold, they can further improve best through a coordinated, collaborative effort between clusters of schools and external agencies such as, for example, social services and training providers. The suggested role of an external agency with the respect and resources of the inspectorate is to provide stimulus and support to make the network effective. Using a bounded case study method, this research seeks to assess the potential of polycentric inspection as a tool for improving school effectiveness and outcomes. Evidence from this study suggests that this mode of evaluation has had a significant impact on improving schools, supporting teachers’ practice and, arguably, increasing student examination outcomes in the network examined. In consequence, it is suggested that these findings have wider implications for the changing conception of school evaluation and how improvement can be achieved in education.
... In addition to the introduction of standardised testing (Brown, McNamara, and O'Hara 2016a) in Ireland, new models of risk-based inspections (Brown et al. 2016b) such as incidental/unannounced inspections have seen state inspectors assume new roles as the as sole arbiters of 'good teaching' (Mooney Simmie, Moles, and O'Grady 2019). What we are witnessing is a change in the relations of power between teachers and the state (Ball 2016). ...
Article
Full-text available
This paper discusses the various ways privatisation processes affect Irish education. Due to the long history of considerable church involvement, the notable absence of middle tiers of governance, and more recently, the embrace of neoliberal principles, in large part due to and for economic reasons, the Irish education system represents a fascinating example of a complex interplay between the public and private sectors. The conceptual and analytical tools provided by Cultural Political Economy are used in this paper to highlight why and how privatisation has unfolded and might yet unravel further in Irish education. These tools offer a useful lens through which to examine country-specific developments, whilst locating them in the global picture. Utilising the conceptualisations offered by Cultural Political Economy, this paper demonstrates how different forms of privatisation in Ireland have contributed to what can be described as a complex system of governance with strong private involvement.
... Beyond predicting college success from college readiness using these measures, the recent applied studies used the academic college readiness measures as covariates while they used college cumulative GPAs as measures of college outcomes in HE quality assessment studies (Coates, 2009;Liu, 2011;Jackson and Kurlaender, 2014). Also, the value-added HE quality assessment models emphasize direct measurement of student learning outcomes (Shavelson et al, 2016;Brown, McNamara and O'Hara, 2016;Sønderlund et al, 2019). Although the previous studies used the college readiness and success measures for modeling and estimating the HE quality of the value-added, they did not compare conditional effects of college readiness on college success and probabilities of college success for comparative analysis of university performance. ...
Article
Full-text available
Studies revealed that college readiness promotes college success and higher education student learning outcomes. This study opted to 1) analyze the total effect and the conditional effect of college readiness on college success by university generations and departments; 2) analyze the differences in the probability of college success across departments and university generations; 3) describe the quality of university generations in terms of the conditional effects and the probabilities of college success. The study is an ex post facto research. The Ethiopian 1st, 2nd, and 3rd generation universities; and the National Educational Assessments and Evaluations Agency officers were the population of the study. The total sample size was 551. The Ethiopian General Education School Leaving Certificate Examination Grade Point Average, the Ethiopian Higher Education Entrance Examination score, and the College Cumulative Grade Point Average of the students were sources of the data. Using the Process Procedure for Software Package for Social Sciences, the binomial logistic regression was conducted. Maintaining the highest total conditional effect of college readiness on college success while heightening the probability of college success at a value of college readiness has been interpreted as a trait of the high performing university generation. Highlights • College readiness affects college success. • The conditional effect of college readiness on college success varies by university generations and departments. • The probability of college success at the value of college readiness varies by university generations and departments. • Maintaining the effect of college readiness on college success while heightening the probability of college success is a trait of high-performing university generations.
... The decentralisation of governance and the consequent drive to more actively include citizens in decision-making processes have become part of the discourse of public sector 'reform' in most European countries particularly in services such as healthcare and education (Beckmann et al., 2009;Verger and Curran 2014). This policy direction can, it is suggested by its proponents, serve a variety of purposes such as reducing state bureaucracy, improvement of services by both regulation and competition, and enhanced 'stakeholder' voice and choice (Brown et al., 2016a). In the education sector, for example, while accountability through the process of school inspection has become emphasised in most Organisation for Economic Co-operation and Development (OECD) countries, inspection models have matured and been adapted as educational evaluation systems mature. ...
Article
The purpose of this paper, which is part of a three-year EU Erasmus+-funded study titled 'Distributed Evaluation and Planning in Schools' (DEAPS), is to provide an analysis of policies, structures, processes, supports and barriers that exist to enable or inhibit the involvement of students and parents in school evaluation in four European countries (Belgium, Ireland, Portugal and Turkey). Document analysis was used for this study and some 348 peer-reviewed articles, and 28 national and transnational policy documents were included in the analysis. Based on this review it would be reasonable to suggest that the student/parent voice agenda around evaluation in schools remains, by and large, aspirational. It is extolled in policy but in practice is mainly tokenistic with very little evidence of impact on the work of schools. In light of this, it is argued that government and school-level policies and strategies need to be reconsidered to enhance students' and parents' engagement in school evaluation. As a first step, significant further empirical research on the limitations on and conditions necessary for stakeholder voice in education is required.
Article
Full-text available
School inspection systems have undergone a transformation in response to the changing social and economic scenarios across Europe. However, two major approaches can easily be identified that also define the two ends of the continuum of the approaches to school inspection. On one end is a high stake sanctions oriented inspection while on the other end is the low stakes advisory inspection. The elements that contribute to the rigour of school inspection include governance arrangements, statutory powers of the inspectorate including powers of sanction, the forms and frequency of inspection visits, the level of emphasis on school self-evaluation and action planning for improvement; and availability of support services for the schools. The current paper presents four cases from Europe: Bulgaria, Ireland, Greece and Spain (Extremadura) describing how school inspection is organised in these countries as defined in the governing legislation and to which end of the continuum their inspection system is tilted in the light of that case's performance on the above indicators. Towards the end, this paper makes a comparison of the cases and suggests why the relationship of trust and respect between inspectors and the inspected is stronger in some cases and how developing inspection systems can benefit from analysing the established systems.
Chapter
Teacher autonomy has been a topic of heated debate in public discourses over recent decades, as well as an increasingly popular topic of investigation. A high degree of teacher autonomy has been associated with positive effects, since autonomous teachers are proven to be more effective in and more satisfied with their work, and to feel empowered and positive about their working climate (Parker in Curriculum Journal 26(3):452‒467, 2015; Wilches in Íkala, revista de lenguaje y cultura, 2007).
Article
Full-text available
School self-evaluation (SSE) has emerged as a widely used approach to school evaluation in recent decades. This has occurred in the context of what is referred to as “New Public Management,” an element of which seeks to empower public institutions to make decisions locally about improving their processes and standards. Inspection regimes in many countries have developed legislative, methodological, and support mechanisms for schools to carry out SSE. This paper, by using the evolution of the SSE process in Irish education, analyses the efficacy of SSE by exploring teachers and school principals’ perceptions of both the challenges and supports concerning the integration of SSE in their schools. Results derived from this study suggest that respondents were, overall, fully aware of the support services available to them. However, support capacity challenges also emerged, in particular as it relates to data use and target setting. Importantly, it is argued that since there are striking resemblances between SSE as it has developed in Ireland and other systems, the challenges and solutions identified in this paper will have wide application in other contexts.
Technical Report
Full-text available
Immigration has dramatically increased in recent years and meeting and satisfying the demands of a diverse multicultural classroom is taxing educators at all levels of the educational spectrum as well as across Europe and internationally. The effects can be particularly significant when assessment tests knowledge, competence and ability at a point of transition which determines future life path or a rite of passage. Towards equity in education, this report (derived from a three year Erasmus+ Project titled: Aiding Culturally Responsive Assessment in Schools - ACRAS) consists of a series of research papers, workshops and resources for the enhancement of assessment strategies and supports for migration background students.
Technical Report
Full-text available
This handbook is an outcome of a project exploring culturally responsive assessment in schools. This handbook is based on literature surveys, surveys of second level school personnel, case studies of second level schools and the subsequent conceptual framework that drew the central themes and concepts for the project. Details can be found on www.acras.eu. The project addresses the issue of culturally responsive assessment for students with a migration background because the immigration into each of the four countries has resulted in students of diverse cultural backgrounds in a classroom. Diversity is a desirable learning environment for all students and fairness in assessment contributes to social equity. As well as contributing to greater academic achievement for minority groups, more equal societies, research shows, “do better in terms of human development, economic performance and political stability” (Giannakaki, McMillan & Karamichas, 2018, p. 206). Findings of the project have included the challenge involved for teachers to design and conduct culturally responsive assessment in diverse classrooms. The challenges are both conceptual and methodological as educators are encountering at first hand the realities of the movement of peoples and the geo-political changes. Teacher professional understanding as “reflective practitioner” (Schon, 1983) leads teachers to awareness of “the limits of our knowledge, of how our own behaviour plays into organisational practices and why such practices might marginalise groups or exclude individuals” (Bolton, 2010, 14). The depth of the challenge for educators requires wide-ranging supports, from national and local policy formulations, professional development and resources for classroom assessment. This handbook aims to address and support in some measure the need for educator support in diverse educational settings.
Book
Full-text available
The compendium featuring Action Based School Evaluation Research Projects that were facilitated by Dr.Patrick Shevlin and Dr Martin Brown comprises School Evaluation Case Studies of all school types (cross-phase primary and post-primary, controlled and maintained across the religious divide, and including integrated and shared education schools throughout Northern Ireland). The key focus of the research was targeted school improvement in a variety of forms together with the use of innovative techniques for self-evaluation and continuing professional learning. As evidenced in the report, the significant impact of this research has been achieved through a series of school-based action research sessions, with accompanying and ongoing in-service at these sessions. Towards civic engagement, this has resulted in (challenging times) schools throughout Northern Ireland, taking ownership of Evaluation for improvement by developing and initiating a series of innovative school evaluation measures with data-informed measures of impact in core areas of education such as Leadership, Assessment and Health and Wellbeing in their school communities.
Article
Full-text available
This paper posits that almost all inspectorates are now following, if to varying degrees, a similar overarching ideology and methodology for school accountability and improvement. The first part of the paper provides an analysis of recent changes to school inspection policies across Frontiers. Using Ireland as a case example, the next part of the paper provides an analysis of Irish school inspection policies and practices that appear to mirror other school inspection systems. To test these assumptions, the paper then provides an analysis of a key informant interview with the Chief Inspector of schools in Ireland. The evidence suggests that there is an increasingly convergent approach to school evaluation discernible across all inspection frontiers. Among the many aspects of this changing landscape is the rapid pace at which schools have accepted school inspection frameworks and the emergence of a genuinely co-professional as opposed to co-existent mode of evaluation between the inspectorate and schools.
Article
Using multisite case studies in four European countries, the purpose of this paper was to explore school leaders and teachers views on School Self Evaluation (SSE), its role in school improvement and the capacity of schools to engage with the process. Evidence derived from the study suggests that although there is a consensus among school leaders concerning the potential utility of SSE; across some countries, there were also concerns relating to implementing the process and the potential misuse of SSE outcomes. When this was not the case, it is apparent that governments have driven the process with clearly defined legislation and defining the SSE agenda and outcomes to dispel school leaders’ apprehensions regarding the balance between SSE for accountability or school improvement.
Central to improving the quality of education is developing a teacher evaluation system that promotes teachers’ professional growth, the improvement of student learning, and educational equality for all students regardless of social factors such as socio-economic status, educational environment, race, and gender. To address these issues in South Korea and to respond to pressure from parents, educators, and policymakers, a new national teacher evaluation system was implemented in 2011. This paper briefly describes the evolution of the teacher evaluation system in South Korea and analyzes the newly developed mechanism by which teachers are evaluated. The new evaluation system resulted in a backlash from teachers and debate among educational stakeholders, including parents, educators, and policymakers. Both the support of and opposition to the new policy are discussed. Although the new evaluation system has not been viewed as effective in all schools, several cases of schools that have had success under the new system are examined to determine the factors that led to their success. This article argues that the teacher evaluation system consisting of fair and reliable components that measure teachers’ performance and support professional development can be an effective means of ensuring high-quality teaching, which, in turn, can positively impact student achievement. However, based on an examination of the case studies presented and grounded in the theoretical perspective on accountability proposed by Thorn and Harris, this study asserts that for the evaluation system to be successful in each school, accountability, necessary modification, and mutual adaptation are required. Implications for policymakers, researchers, and politicians are provided.
Chapter
Full-text available
This chapter distinguishes multiple aspects of teaching that together predict student engagement and learning. Using data from the Measures of Effective Teaching (MET) project, it applies two popular frameworks and associated assessment tools for measuring what teachers actually do in their classrooms. These are the Framework for Teaching (FfT) and 7Cs framework from the Tripod Project survey assessments. The chapter first demonstrates that specific domains and components of the FfT and Tripod 7Cs frameworks are compatible, not only conceptually, but also empirically. It also considers ways that combining and comparing data generated can contribute to quality control by helping to uncover dishonesty or other problems with implementation. The chapter proposes that paying attention to the components of the FfT and Tripod 7Cs frameworks can enrich the quality of reflection, discourse, and support that teachers experience in collaboration with supervisors and peers concerning their teaching.
Article
Full-text available
This study examined the SAS Education Value-Added Assessment System (EVAAS®) in practice, as perceived and experienced by teachers in the Southwest School District (SSD). To evaluate teacher effectiveness, SSD is using SAS EVAAS® for high-stakes consequences more than any other district or state in the country. A mixed-method design including a large-scale electronic survey was used to investigate the model’s reliability and validity; to determine whether teachers used the SAS EVAAS® data in formative ways as intended; to gather teachers’ opinions on SAS EVAAS®’s claimed benefits and statements; and to understand the unintended consequences that occurred as a result of SAS EVAAS® use in SSD. Results revealed that the reliability of the SAS EVAAS® model produced split and inconsistent results among teacher participants, and teachers indicated that students biased the SAS EVAAS® results. The majority of teachers disagreed with the company’s marketing claims and did not report similar SAS EVAAS® and principal observation scores, reducing the criterion-related validity of both measures of teacher quality. Many unintended consequences associated with the high-stakes use of SAS EVAAS® emerged through teachers’ responses, which revealed among others that teachers felt heightened pressure and competition, which reduced morale and collaboration, and encouraged cheating or teaching to the test in attempt to raise SAS EVAAS® scores. The results of this study, one of the first to investigate how the SAS EVAAS® model works in practice, should be considered by policymakers, researchers, and districts when considering implementing the SAS EVAAS®, or any value-added model for teacher evaluation.
Technical Report
Full-text available
Examining VfM in schools requires a detailed consideration of both the inputs and the outputs relating to the education system – in other words, the costs and benefits associated with schools. In this context, inputs are relatively straightforward to express: staffing typically represents around 70% of school costs, with the remainder comprising of operating and maintenance expenditure. However, defining the outputs from schools (pupil outcomes) is an altogether more complex task. The most tangible measure of pupil outcomes is the results they achieve in examinations. Even so, both in the UK and overseas, it is widely recognised that pupils’ levels of achievement are influenced by a range of factors other than the quality of schooling they receive (for example, their family background or their track record of achievement at earlier stages of their education). Put simply, pupils’ current levels of attainment are likely to be enhanced if they have a high level of prior educational attainment, and therefore schools may make a less significant contribution to student outcomes. In assessing the value for money which schools deliver through their day-to-day activities, it is therefore necessary to take account of the progress pupils make in their time at school (or ‘value added’ by the school), rather than just their levels of attainment, which are influenced by a host of factors other than schooling. There is no single accepted definition for VfM in schools. However, in our view, VfM could potentially be defined in terms of educational value added per pound of educational expenditure. As discussed later in the report, a wealth of data currently exists to allow the practical measurement of VfM in this way. Section 3 of the report examines a range of factors relevant to VfM in schools and sets out more fully the rationale for our suggested definition.
Article
Full-text available
What lessons can we learn from the relationship between policy-makers and schools over the life of the 'New' Labour and its predecessor Conservative government? What happened to 'Education, Education, Education' as it travelled from political vision to classroom practice? What are the lasting legacies of 13 years of a reforming Labour government? And what are the key messages for a coalition government? These are the questions addressed to the architects of educational reform, their critics and the prophets of better things to come. The 37 interviewees include ministers past and present, journalists, union officials, members of lobby groups and think tanks. Reinventing Schools, Reforming Teaching considers the impact of educational policies on those who have to translate political priorities into the day to day work of schools and classrooms. The authors argue that an evidence-informed view of policy-making has yet to be realised, graphically illustrating how many recent political decisions in education can be explained by the personal experiences, predilections and short-term needs of key decision-makers. The interviews, which explore the dynamics behind the creation of education policies, cover a wide range of themes and issues, including: policy-makers' attitudes to schools, the staff who work in them and the communities they serve the drivers of politicians' reform agendas and the constraints on radical reform the shaping and reshaping of curriculum and assessment the search for a more effective marriage between inspection and school self evaluation the relationship of academic research to policy making how a vision for teaching and teachers might be constructed for the 21st century Contributions from leading figures including; David Puttnam, Kenneth Baker, Estelle Morris, Gillian Shepherd, Jim Knight, Pauline Perry, Michael Barber, Peter Mortimore, Judy Sebba, Paul Black, Mary James, Kevan Collins, David Hargreaves, Mike Tomlinson, David Berliner, Andreas Schleicher, Tim Brighouse, Conor Ryan, Keith Bartley, Michael Gove and Philippa Cordingley are woven in with the insights of teachers and headteachers such as Alasdair MacDonald and William Atkinson. The book's findings and proposals will be of interest not only to professional educators and those with an interest in the current and future state of education but to those interested in the process of policy-making itself. © 2011 John Bangs, John MacBeath and Maurice Galton. All rights reserved.
Article
Full-text available
This paper puts forward an argument for the centrality of the concept of dialogue in the school effectiveness and improvement discourse. The work of the 'Leadership for Learning' group at Cambridge is conceptualised as knowledge building through disciplined dialogue framed by shared moral purpose. The paper includes exemplifications by focusing on four dimensions of the LfL group's work: pupil voice, teacher leadership, school self-evaluation and critical friendship. It concludes with a call for dialogic pedagogies as well as dialogic approaches to policy development and inquiry within the field of school effectiveness and improvement.
Article
Full-text available
There is a growing consensus that evidence of teachers' contributions to student learning should be a component of teacher evaluation systems, along with evidence about the quality of teachers' practice. Value-added models (VAMs), designed to evaluate student test score gains from one year to the next are often promoted as tools to accomplish this goal. However, current research suggests that VAM ratings are not sufficiently reliable or valid to support high-stakes, individual-level decisions about teachers. Other tools for teacher evaluation have shown greater success in measuring and improving teaching, especially those that examine teachers' practices in relation to professional standards.
Article
Full-text available
This article is a case study of the emergence of an evaluation culture in the public sector and particularly in education in Ireland. It suggests that the emergence of this culture was strongly influenced by external bodies, particularly the EU and, to a lesser but significant degree, the OECD. It is further argued that the continuation of systematic evaluation is still dependent on external forces, since a commitment to evaluation as a tool of governance has not taken hold among key policy-makers in Ireland. However it is postulated that, notwithstanding its arguably insecure foundations, evaluation practice has moved beyond the confines of externally funded EU programmes, which saw its first introduction into Ireland. In recent years a broad quality assurance agenda within the public service and to an extent beyond has emerged.The article concludes by making the point that an evaluation culture in a particular country is hugely contextualized and influenced by the constraints of existing ideologies and relationships between different interest groups.Thus, in Ireland, in line with the corporatist and partnership-driven approaches to economic policy and industrial relations which have been dominant in recent decades, the form of evaluation which has emerged is consensual, collaborative and negotiated.
Article
Full-text available
The purpose of this article is to re-examine the current use of what is to count as evaluation evidence to inform professional practice. The provision of evaluation evidence to inform decision making has long been an aspiration, albeit with varying degrees of success. However the need to re-examine the utility of this endeavour has arisen in the contemporary culture of evidence-based policy and practice. The main argument of the article is that the politically favoured approach to this task fails to recognize the holistic nature of professional practice and disregards the complexity of professional decision making and action. The article offers a critique of the dominant model of evidence-based in relation to the nature of professional practice and argues that it is time to reassert the value of qualitative methodologies to maximize the utilization of evaluation evidence by professional practitioners.
Article
Full-text available
Research Findings: Hong Kong launched the new mechanism of the Quality Assurance Inspection (QAI) in 2000 to enforce accountability and school improvement in the sector of early childhood education. The QAI comprises 3 stages: school self-evaluation, external inspection, and release of the QAI report to the public. To understand the roles of self-evaluation and external inspection and their interactions, we analyzed all 80 QAI reports released by the government from 2004 to 2007 and interviewed the principals and teachers of 3 kindergartens that had been ranked excellent, good, and satisfactory on Learning and Teaching performance in the QAI. The results indicated that (a) school performance on self-evaluation was significantly and positively correlated with Learning and Teaching performance but negatively correlated with the number of problems found in the QAI, (b) all kindergartens reported 4 types of challenges brought about by the external inspection (i.e., uncertainty, personnel, workload, and psychological burdens), and (c) the QAI was a facilitative and productive process for school self-evaluation and quality improvement. Practice or Policy: These findings suggest that an effective quality assurance mechanism should maintain a balance between external and internal evaluations and should work toward school empowerment and improvement.
Article
Full-text available
The issue of value‐added measures of school effectiveness is reviewed in relation to the Government's requirement for schools to publish raw examination results. The study reports a series of multilevel analyses of the 1993 GCSE examination results in Lancashire employing a wide variety of pupil intake and school context variables as well as a range of 1991 Census measures attached to each school's catchment area and each pupil's home area. The methodology involves evaluating and comparing the results of several different value‐added models controlling for different pupil and school background factors. The findings indicate that a substantial percentage of school level variation in pupil outcomes can be explained by pupil intake factors. In addition, comparisons between the different value‐added models indicate that when rich and wide‐ranging pupil level data (for example, prior attainment measures in different areas) are available and taken into account in the analysis, school‐context factors (such as percentage of pupils entitled to free school meals) are not significant in predicting pupil outcomes. However, when pupil prior attainment data are lacking, contextual factors are more useful and may be viewed as adequate but approximate measures of pupil intake. Within individual schools there is also evidence of inconsistent departmental effectiveness and differential effectiveness for pupils of different levels of prior attainment. The implications and limitations of the findings are discussed, as is the question of how the methodology may be employed to stimulate approaches to school improvement and to widen the scope of value‐added measures.
Article
Full-text available
Purpose – The objectives of this study were to investigate the degree to which quality initiatives, including quality standards, have been adopted in Irish organisations and to provide comparisons with survey results of surveys in 1995/1996. Design/methodology/approach – A national survey was conducted in 2002 of Irish employers in manufacturing/services within the private and public sectors. The questionnaire was sent to 2,487 organisations – 932 questionnaires were returned representing a 34 per cent response rate from the private sector organisations and 47 per cent in the public sector. Findings – The survey illustrates the continuing importance of, and reliance upon, standards. Among organisations that had been awarded ISO 9000 certification, 93 per cent believed that it had improved their reputation and 90 per cent that it resulted in better products or services. Nearly two‐thirds of respondent organisations had adopted a total quality approach compared with less than one‐third in 1995/1996. The results demonstrate that total quality organisations did not differ significantly from non‐TQ organisations in relation to having been awarded ISO 9000. However, there were statistically significant differences between TQ and non‐TQM organisations in relation to: a commitment to quality, quality structures, measures to evaluate effectiveness in relation to customers, employees and suppliers, team working, quality assurance systems, use of quality tools and techniques and investment in training. Originality/value – This study shows that interest and adoption of quality improvement in Irish organisations is growing rapidly, particularly since the mid‐1990s that has marked a distinct period of economic growth in the Irish economy.
Article
Full-text available
Background: This paper outlines the findings from a research project carried out in the UK that explored the relationship between parental engagement and student achievement. Purpose: The 12-month research project was commissioned to explore the relationship between innovative work with parents and the subsequent impact upon student achievement. A main aim of the research project was to capture the views and voices of parents, students and teachers and to explore the barriers to parental engagement and the respective benefits to learning. Programme description: The study was qualitative in design and collected in-depth case-study data from 20 schools and 314 respondents. In addition a range of documentary evidence plus performance data were collected at each case-study site. Sample: A sample of schools in England was selected on two main criteria: firstly, the type of development, and secondly, the particular focus of parental engagement. Schools in the sample were selected to ensure that there was a broad geographical spread and a mix of urban and rural schools. Other factors were also taken into account to ensure a diverse range of schools (e.g. number on roll, socio-economic status (SES) and black minority ethnic (BME) percentages). Design and methods: Case-study methodology was used as the prime method of data collection in the study. In addition, school data sets relating to student performance, behaviour and attendance were analysed. These data sets allowed patterns and trends to be identified. This analysis formed the basis of the more detailed interrogation of the case-study evidence at each of the 20 sites. Results: The research findings highlight a number of barriers facing certain parents in supporting their children's learning. It is clear that powerful social and economic factors still prevent many parents from fully participating in schooling. The research showed that schools rather than parents are often 'hard to reach'. The research also found that while parents, teachers and pupils tend to agree that parental engagement is a 'good thing', they also hold very different views about the purpose of engaging parents. It is also clear that there is a major difference between involving parents in schooling and engaging parents in learning. While involving parents in school activities has an important social and community function, it is only the engagement of parents in learning in the home that is most likely to result in a positive difference to learning outcomes. Conclusions: Parental engagement in children's learning in the home makes the greatest difference to student achievement. Most schools are involving parents in school-based activities in a variety of ways but the evidence shows but this has little, if any, impact on subsequent learning and achievement of young people.
Article
Full-text available
Value-added measures can be used to allocate funding to schools, to identify those institutions in need of special attention and to underpin government guidance on targets. In England, there has been a tendency to include in these measures an ever-greater number of contextualising variables and to develop ever-more complex models that encourage (or ‘impose’) in schools a single uniform method of analysing data, but whose intricacies are not fully understood by practitioners. The competing claims of robustness, usability and accessibility remain unresolved because it is unclear whether the purpose of the measurement is teacher accountability, pupil predictability or school improvement. This paper discusses the provenance and shortcomings of value-added measurement in England (and the Pupil Level Annual Schools Census that informs it) including the fact that although the metrics are essential for School Effectiveness Research, they fail to capture in its entirety the differential effectiveness of schools across the prior attainment range and across sub-groups of students and subjects. KeywordsValue-added measures-Pupil attainment data-Accountability in education-Complex models in school effectiveness
Article
Full-text available
Trends in skill bias and greater turbulence in modern labor markets put wages and employment prospects of unskilled workers under pressure. Weak incentives to utilize and maintain skills over the life-cycle become manifest with the ageing of the population. Policies to promote human capital formation reduce welfare state dependency among the unskilled and offset inefficiencies in human capital formation. Skill formation features strong dynamic complementarities over the life-cycle. Investments in the human capital of children have higher returns than investments in the human capital of older workers. There is no trade-off between equity and efficiency at early ages of human development but there is a substantial trade-off at later ages. Later remediation of skill deficits acquired in early years often does not meet the cost-benefit criterion. Positive returns to active labor market and training policies are doubtful. Skill formation is impaired when the returns to skill formation are low due to low skill use and insufficient skill maintenance later on in life. High marginal tax rates and generous benefit systems reduce labor force participation rates and hours worked and thereby lower the utilization rate of human capital. Tax-benefit systems redistribute resources from outsiders to insiders in labor markets, which can be both distortionary and inequitable. Actuarially fairer early retirement and pension schemes reduce the incentives to retire early and strengthen incentives for human capital investment by increasing the time-horizon over which returns to human capital are harvested.
Article
Assessment dominates our lives but its good intentions often produce negative consequences. An example that is central to this book is how current forms of assessment encourage shallow 'for-the-test' learning. It is true to say that as the volume of assessment increases, confidence in what it represents is diminishing. This book seeks to reclaim assessment as a constructive activity which can encourage deeper learning. To do this the purpose, and fitness-for-purpose, of assessments have to be clear. Gordon Stobart critically examines five issues that currently have high-profile status: intelligence testing; learning skills; accountability; the 'diploma disease'; formative assessment. Stobart explains that these form the basis for the argument that we must generate assessments which, in turn, encourage deep and lifelong learning. This book raises controversial questions about current uses of assessment and provides a framework for understanding them. It will be of great interest to teaching professionals involved in further study, and to academics and researchers in the field.
Article
This book looks at the foundations of school self-evaluation from a scientific as from a practical perspective. Planning concepts, restructuring of education systems, organizational theory on schools, evaluation methodology and models of school effectiveness and school improvement are discussed as contributing to the overall conceptualization of school self-evaluation. A broad range of approaches is presented and methodological requirements are discussed. School self-evaluation contains controversial issues that reflect tension between the need for objectivity in a context that is permeated by values and potential conflicts of interests. Similar tensions may be seen to exist with respect to the static and "reductionist" aspects of available data collection procedures in a complex and dynamic situation and the appeal for external accountability on the one hand and improvement oriented self-refection on the other. The mission of the book is to clarify these tensions and offer ways to deal with them in practical applications. The school effectiveness knowledge base is offered as a substantive educational frame of references that serves an important function in selecting relevant factors for data collection and the use of the evaluation results.
Article
Since the 1990s, programmes aimed at universalising elementary eduction in India have increasingly dichotomised ‘quality’ and ‘equality’ and heightened, a view of education which is essentially instrumental. The paper argues that this opposition is mistaken and that equality itself requires a nuanced approach which reflects the quality of education as an involvement in the long-term growth of the person. An example of how a school may be knowledgeably structured for educational quality, given the social reality of entrenched gender inequality, is discussed.
Article
This article reports an original investigation into school performance measures and the multilevel nature of pupil achievement data in the Chilean school system using a sample of 177,461 students, nested within 7146 classrooms, 2283 secondary schools and 313 municipalities. The data-set comprised Year 10 students’ 2006 SIMCE test’s results in two subject outcomes (language and maths) matched to their prior attainment in grade 8 and family characteristics. The analyses showed the lack of precision of two-level models to draw conclusions regarding the effectiveness of Chilean secondary schools as well as the extent to which different pupil intake, background and context features of Chilean secondary schools influence students’ performance. The results show substantial and statistically significant municipal, school and classroom effects in Chile, and how the estimation of school effects changes according to the explanatory variables controlled for in the analysis and the outcome analysed. These results are compared with similar studies carried out in Latin America, as well as in other countries (England and China), in order to situate the findings in the broader knowledge base of Educational Assessment and Effectiveness Research.
Article
In this article, Jennifer O'Day builds on her earlier work defining and examining the standards-based reform movement in the United States. Here, O'Day explores accountability mechanisms currently associated with standards-based reform efforts that "take the school as the unit of accountability and seek to improve student learning by improving the functioning of the school organization." She examines such accountability mechanisms using the theoretical framework of complexity theory and focuses on how information travels through complex systems, with the understanding that information, its existence and usage, is key to improving schools. Drawing on work conducted with researchers at the Consortium for Policy Research in Education (CPRE), she contrasts the Chicago Public Schools' outcomes-based bureaucratic accountability approach with the combination of administrative and professional accountability found in the Baltimore City Schools. She argues that the combination of administrative and professional accountability presents a much more promising approach for implementing lasting and meaningful school reform.
Article
The assessment of the achievement of students and the quality of schools has drawn increasing attention from educational researchers, policy makers, and practitioners. Various test-based accountability and feedback systems involving the use of value-added techniques have been developed for evaluating the effectiveness of individual teaching professionals and schools. A variety of models have been employed for calculating value-added measures, including the use of linear regression models which link students' present and prior achievements. One of the limitations associated with the use of the conventional linear regression methods in value-added calculation is that the value-added measures are likely to be overestimated for students with higher prior achievements while underestimated for students with low prior achievements. This study explores an alternative approach, the principal axis approach, to calculating value-added measures which can eliminate some of the limitations associated with the conventional linear regression approaches.
Article
The government of Ireland, like many European countries, is currently under severe pressure from external forces to grow the economy. One possible way to maintain and grow its economy is through the production of a highly educated and globally competitive workforce. In an effort to develop such a workforce, the government, through the Department of Education, is considering ways to increase accountability in its schools. This paper examines value-added accountability systems used in the USA and raises critical questions about their perceived value for Irish schools.
Article
Instability in the school population between school entrance and school leaving is not “just a problem of missing data” but often the visible result of the educational problems in some schools and is, therefore, not merely to be treated as missing data but as indicator for the quality of educational processes. Even the most superior value-added model with corrections for premeasurement, gender, intelligence, age, socioeconomic and ethnic backgrounds of students is only valid for the detection of schools with the highest raw scores and the highest learning gains, but it is a very moderate predictor for the detection of schools with the lowest amount of students that lag behind and a bad predictor for the equity and efficiency of schools.
Article
This paper examines the emergence of new accountabilities in teaching and teacher education in Ireland in the 15 years period 1997–2012. Framing accountability in terms of the three main approaches to it globally in education systems, that is, compliance with regulations, adherence to professional norms and attainment of results/outcomes, we identify significant changes, particularly, in compliance- and results-driven accountability. A ‘rising tide’ of accountability, due to the interrelated influences of the European higher education space, education legislation and professional self-regulation policies (i.e. Teaching Council), is evident since the late 1990s. This was punctuated by a ‘perfect storm’ in 2010 comprising ‘bad news’ from PISA 2009, the economic bailout and strategic leadership at a system level. The cumulative impact of the ‘rising tide’ and ‘perfect storm’ is evident in how they reframed both ‘to whom’ and ‘for what’ accountability in teacher education relates. Significantly, the new accountabilities in teaching and teacher education reflect a move towards the dominant global education reform movement (Sahlberg 2007) with its emphasis on standardisation, narrow focus on literacy and numeracy and higher stakes accountability.
Article
It is argued that the field of comparative and international education must be fundamentally reconceptualised and redeveloped in ways that better demonstrate its potential to contribute to: (I) the improvement of educational policy and practice world‐wide; and (2) advances in theoretical work relating specifically to education and to the social sciences more generally. Traditional strengths of the field upon which we can build are identified, but the discussion focuses upon the nature, significance and potential of emergent trends and new possibilities. This meta‐analysis is presented to stimulate and widen discussion and to help generate further avenues for future research and development. Particular attention is given to the research orientation and potential of our multidisciplinary field; to the implications of globalisation; to the challenges of post‐modern and post‐colonial perspectives; to new frames of reference and units of analysis; to the significance of culture and context; to new forms of discourse; and to the importance of forging stronger linkages between theory and practice, insiders and outsiders and the comparative and international dimensions of our field.
Article
The use of value-added modeling (VAM) in school accountability is expanding. However, trying to decide how to embrace VAM can be rather nettlesome. Some experts claim it is “too unreliable,” causes “more harm than good,” and has “a big margin for error,” while other experts assert VAM is “imperfect, but useful” and provides “valuable feedback.” This article attempts to parse these statements by exploring the underlying statistical assumptions of VAM, the reliability of VAM’s estimates, and the validity of the inferences commonly made based on the estimates of VAM. It then goes on to discuss the perverse incentives, unintended consequences, and gaming that might accompany the misuse of VAM. The article concludes that while, in many cases, VAM may be preferable to other commonly used measurement modes, it should never be used as the sole indicator of teacher effectiveness. Rather, it should just be a piece of a larger accountability system.
Article
There may be adult educators who wish to alleviate social inequalities but whose practices are wedded to human capital theory In this article, it is argued that educational practices based wholly or partly on human capital theory are unlikely to alleviate social inequities because the theory spawns pedagogical practices that are apolitical, adaptive, and individualistic. By alerting adult educators of the ominous pedagogical implications of human capital theory, it is hoped that they will distance themselves from it and embrace more socially responsible alternatives.
Article
This paper surveys the development of various approaches to quality that are essentially learning-centred: •In the Schools sector: a brief overview of the Victorian Quality in Schools project; •In Higher Education: work in progress at two Australian universities (Victoria University of Technology and Swinburne Universities of Technology in Melbourne); and •In Vocational Education and Training: work in progress in re-orienting the policy approach to Quality towards a more flexible and learning-centred model. This paper will argue that when looked at from the perspective of the individual learner, there is a strong case for student learning to be placed at the very heart of Quality Systems in all sectors of education, and also therefore in related sectoral Quality Assurance programs and processes. <br /
Article
In this article, the author explores the challenges and promises of value-added assessment. As NCLB approaches the end of statistically possible achievement gains in schools, value-added assessment is being employed to longitudinally measure student learning to determine a school's effect. Yet, value-added assessment is limited in its explanatory powers because it focuses only on certain types of knowledge and needs to be used in conjunction with other estimates. As such, the author provides a variety of perspectives to help educational stakeholders explore the assessment not just as a new test but rather as a promising and potentially damaging lever of change in school cultures.
Article
Use of tests and assessments as key elements in five waves of educational reform during the past 50 years are reviewed. These waves include the role of tests in tracking and selection emphasized in the 1950s, the use of tests for program accountability in the 1960s, minimum competency testing programs of the 1970s, school and district accountability of the 1980s, and the standards-based accountability systems of the 1990s. Questions regarding the impact, validity, and generalizability of reported gains, and the credibility of results in high-stakes accountability uses are discussed. Emphasis is given to three issues regarding currently popular accountability systems. These are (a) the role of content standards, (b) the dual goals of high performance standards and common standards for all students, and (c) the validity of accountability models. Some suggestions for dealing with the most severe limitations of accountability are provided.
Article
Production research in education has been dominated by attempts to estimate the structural parameters of what has been called the education production function. These estimation attempts are viewed in this article as only one way the production function concept can be drawn upon to inform debates over education policy. After exploring what it means to posit the existence of the education production function, the article critically reviews past estimation efforts and gives examples of how the production function can be used as a source of insight to guide policy-relevant inquiries into education productivity.
Article
This article gives an account of how and why the term 'value added' has come to be used in an educational context, focusing on the early years of that usage in the UK. It discusses the fact that the term value added has developed, has been understood and defined in diverse and sometimes conflicting ways; and identifies what are called 'functional ambiguities' in the term; that is to say, ambiguities which cannot be altogether eliminated because they are necessary to how the term is made to function. Nonetheless, some criteria emerge in the course of the article for making evaluative judgements of what should and should not count as meaningful definitions of value added.
Article
Accountability systems in education generally include indicators of student performance. However, these indicators often differ considerably among the various systems. More and more countries try to include value-added measures, mainly because they do not want to hold schools accountable for differences in their initial intake of students. This study presents a conceptual framework of these value-added measures, resulting in an overview of 5 different types. Using data from Dutch secondary schools, we empirically provide estimates of these different measures. Our analyses show that the correlation between the different types of school effects estimated is rather high, but that the different models implicate different results for individual schools. Based on theoretical considerations, arguments are given to use the following indicators in the value-added accountability models: prior achievement, student-level background characteristics, and compositional characteristics of the student population.
Article
Purpose – The purpose of this paper is to discuss some key aspects of quality in education in the light of over 30 years practical experience of doing quality assurance (QA). Design/methodology/approach – Reflection on three concepts, which are still the subject of debate, namely: “quality”; “total quality management (TQM)”; and “autonomy”. Findings – As this is not a research paper, it presents no findings. There are some research implications, if only to deter researchers from digging up old ground. More research into the diversity of and interactions between cultures in academia might prove useful. Practical implications – There are lessons to be learnt from the past. Doing quality improves quality. Talking about it or trying to impose it does not. Managers and leaders need to reflect more carefully than is their wont on the purposes and procedures of QA in education. Originality/value – This paper makes a contribution to the debate about quality in education in universities and schools and suggests that a clearer understanding across the education system of the scope and purpose of QA, the nature of TQM and the limitations of autonomy might lead to better embedded and more effective continuous improvement.
Article
Published indicators of school `performance', such as those shown annually in league tables in England, have been controversial since their inception. Raw-score figures for school outcomes are heavily dependent on the prior attainment and family background of the students. Policy-makers in Wales have reacted to this fundamental flaw by withdrawing the publication of school results. In England, on the other hand, they have reacted by asking for more information to be added to tables, in the form of student context such as the percentage with a special educational need, and `value-added' figures. In 2004, the Department for Education and Skills (DfES) value-added figures for England were based on student progress from Key Stage 2 at the end of primary education to GCSE at the end of compulsory secondary education. For 2005, at time of writing, the DfES plan to use context information in their model as well. This paper re-analyses the 2004 value-added figures and shows that they contain the same flaw as the original raw-score tables. The purported value-added scores turn out to be a proxy for the overall level of attainment in the school, and almost entirely independent of any differential progress made by the students. The paper concludes by considering the implications of these findings, if accepted, for policies based on identifying schools that are clearly more or less effective, and for the field of school effectiveness and improvement research.
Article
This article presents a policy sociology reflection on Bernard Barker's book, The Pendulum Swings: Transforming School Reform. The book represents Barker's attempt to intervene in education policy during the lead-up to the 2010 UK general election and is framed by what he imagined might be possible under a new Conservative government. Barker draws inspiration from the Red Tory communitarian position articulated by Phillip Blond. In hindsight, we are less sanguine about these possibilities in the context of the Coalition government and its ongoing response to the ongoing financial crises. Indeed, what has emerged is a rearticulated neo-liberalism in the guise of ‘Big Society’ rhetoric. We agree with Barker's critical deconstruction of the five illusions underpinning New Labour schooling policy, but argue for a broader agenda of redistribution, both in social policy and with respect to schools. Policy needs to recognise and support teachers and good pedagogies, and we also see a pressing need to rethink richer forms of educational accountability. All of this must be located within a politics that pursues a new social imaginary. Nonetheless, we commend Barker's contribution towards post neo-liberal thinking in respect of school policy, specifically in England, but with relevance to other locations and systems.
Article
For three consecutive years, opinion polls in the US have placed education as the top national issue (USA Today, 1999). `Education, Education, Education' has been the personal mantra of UK Prime Minister Tony Blair. These responses are not unique, governments across the globe are engaged in the education reform business, motivated by a range of concerns. Drawing on the experience of a number of countries, this article explores the context for education reform, offering insights into the reform process and questioning some of the conventional orthodoxies about the nature of education change. The author argues that the structural reforms which have been the pre-occupation of governments over recent years have been limited in their impact because of their lack of connection to learning – teacher learning and student learning – and because of the ways in which policy-makers have embarked on the change voyage. She argues that governments need to move away from a compliance models of educational reform, towards an approach which reflects the aspirations of schools and communities, and which is supported by forms of leadership connected to learning.
Article
Let me share with you how honored I am to receive an award named after the late Dr. Jay Millman. In 1983, after completing the first of our research studies that began our continuing work in value-added, our report was sent by officials in the Tennessee Department of Education for review by Dr. Millman. It is no secret that many in the Department at the time were assuming that his anticipated critical review would put an end to such a preposterous idea—that student achievement data could be used as part of teacher evaluation. Days turned into weeks; each time that I would inquire of the Department as to when we would hear from the review, I was always told that they had not received it. One day I called Dr. Millman and explained my frustration of not hearing from the review and inquired as to when it might be available. He immediately interrupted and explained that he had sent the review several weeks previous to that day and that he would be glad to send me a copy of his remarks, obviously very angry that they had not been passed on to me. Upon receiving and reading his review, it became obvious why I had not received a copy from the Department. Even though he raised many important questions, his review was most objective and generally very positive. Later, he asked us to submit chapters to the book on student outcomes assessment models that he edited. In all of my interactions with Jay, I developed the utmost respect for this distinguished scholar, and I am glad that fate let our paths cross.
Article
Purpose - To provide a history of the emergence of quality systems from the mid-1980s. To show how quality became a primary policy concern in higher education policy. To map the development of quality processes and raise questions about dominant approaches and express concerns for the future. Design/methodology/approach - Historical document analysis. Findings - The problems in institutionalising quality are analysed and it is concluded that the British system of quality monitoring failed to engage with transformative learning and teaching. Practical implications - As the UK developments guided many other countries into developing a system of quality, the UK history of the emergence and development of quality processes 1985-2005 is of interest to international readers. It identifies both good practice and what to avoid. Originality/value - The historical analysis reveals how quality evaluations were guided as much by political pragmatism as rational evaluation.
Article
Recent decades have witnessed a remarkable rise in the regulation of public services and servants, education being a case in point. External evaluation and inspection has been an important element of this trend. Increasingly, however as the limitations of external surveillance systems have become clear the concept of internal or self-evaluation has grown in importance. This paper explores the concept of self-evaluation in education and gives an account of some of the possibilities and problems associated with it. In particular it is argued that enabling individual schools and teachers to self-evaluate effectively is a complex task that will require help and support from the community of professional evaluators.