Content uploaded by Greg Thompson
Author content
All content in this area was uploaded by Greg Thompson on Jul 22, 2019
Content may be subject to copyright.
Toward an International Measure of Global Competence? A Critical Look at the PISA 2018
Framework
Laura C. Engela*
David Rutkowskiᵇ**
Greg Thompsonᶜ***
aInternational Education Program, The George Washington University, Washington, DC, United
States
School of Education, Indiana University, Bloomington, IN, United Statesᵇ
ᶜFaculty of Education, Queensland University of Technology, Australia
*Laura C. Engel, Associate Professor of International Education and International Affairs,
International Education Program, 2134 G Street, NW Washington, DC 20052 USA, Email:
Lce@gwu.edu
**David Rutkowski, Associate Professor, School of Education, Indiana University, Bloomington,
IN, United States
***Greg Thompson, Associate Professor, Faculty of Education, Queensland University of
Technology, Australia
Abstract: This paper focuses on the OECD’s framing of global competence measured in the
PISA 2018 assessment cycle and the issues that arise from having an international economic
policy organisation define global competence within multiple countries. Given the growing
national and international attention on educating for global competence, and in the absence of
other measures, it is highly likely that stakeholders in local, national, and international spaces
will turn to this international measure as an objective and neutral tool. We think, given this
possibility, there is an urgent need to scrutinise the framework that underlies global competence.
Our critical analysis is conceptually framed by academic literature related to (a) the OECD’s
influential role in facilitating neoliberal education policy trends, (b) points of disjuncture and
debate surrounding global competence, and (c) the ways in which influence is garnered through
measurement technologies. In our last section, we focus on the implications of the OECD leading
the development of such a measure, encouraging the OECD to be transparent in the reporting of
results and educational stakeholders to exercise caution in their interpretation of the results.
Keywords: Global competence, OECD, PISA, global citizenship
As a dynamic and multifaceted organization, the Organisation for Economic Co-
operation and Development (OECD) has expanded its reach in education by including more
countries, age groups and topics into its assessment, development, and warehousing work. One
recent example is the introduction of a global competence measure in the OECD’s Programme
for International Student Assessment (PISA). The OECD and the Asia Society argue that the
need for an assessment of global competence expresses “a remarkable moment of global
consensus” (Asia Society/OECD, 2018, p. 4). The OECD’s measurement of global competence
is a function of growing enthusiasm for global “21st century skills” within national school
systems and aligns with the United Nations’ Sustainable Development Goals’ (SDGs) focus on
global citizenship. However, given that many countries, including Japan, the U.S., England,
Scotland, Germany, France, and Finland have decided not to participate in this aspect of PISA
2018, it seems the claim of a consensus may be premature (Coughlan, 2018; Sälzer & Roczen,
2018). In fact, as will be shown in this paper, the conceptual space surrounding global
competence remains at best amorphous and at worst divisive, making valid cross-national
measurement difficult if not impossible.
By developing frameworks and subsequent measures, however problematic, the OECD
uses its global position as an international policy organization to influence education policy
around the world. To that end, this paper examines the OECD’s international framework for
global competence and asks what it means to allow an international economic policy
organisation to define and measure global competence across a diverse set of countries. We
operate from the perspective that when something is measured, values are both assumed and
claimed, such that over time what is measured often becomes what is valued. Furthermore, given
the political, institutional and expert investment required for nations and organisations to commit
to an assessment, future changes to that assessment can be difficult because of the political cost
of admitting that a measure is sub-optimal and/or fears that change compromises the
comparability of the data across years. In short, measures matter. This is especially relevant for
‘global competence’ given the political enthusiasm for the Sustainable Development Goals
(SDGs), ongoing concerns around the health of democratic institutions, and the status of
citizenship within many countries amid a refugee crisis. In the absence of cross-national
measures of global citizenship and global competence, it is highly likely that stakeholders in
local, national, and international spaces will adopt the OECD’s measure as an objective and
neutral tool, regardless of whether they have participated or not. Given this likelihood, there is an
urgent need to scrutinise the OECD’s global competence framework.
We develop our argument in four parts: We first focus on the OECD as a leading
international policy actor, able to wield influence vis-a-vis its work in international large-scale
assessment (ILSA). We then focus on the measurement of amorphous constructs, like global
competence, and the ways in which influence is garnered through measurement technologies.
Third, we explore the academic literature on global competence to elaborate the points of
disjuncture and debate, suggesting these as leading challenges in the OECD’s attempt to develop
a single cross-national measure of global competence. Against these bodies of literature, we
describe and critically analyse the OECD’s framework document of global competence. We
conclude with a discussion of the implications of the OECD’s international measure of global
competence, encouraging the OECD’s full transparency in reporting of results and caution of
educational stakeholders in their interpretation of the forthcoming results, and ranking, regarding
global competence.
The OECD and “soft governance”
One of the OECD’s significant foci is education, and specifically assessment and
evaluation, with prominent examples of its work being PISA, the Programme for the
International Assessment of Adult Competencies (PIAAC) and the Teaching and Learning
International Survey (TALIS). While there is not room in this paper to give an adequate
treatment of the copious literature that criticises the OECD’s education work, in broad terms
many critics see the OECD as a driving force of neoliberal education policy that produces
significant, and often perverse, effects within education systems (see, e.g., Bieber & Martens,
2011; Grek, 2009; Meyer & Benavot, 2013). Scholars argue the OECD facilitates neoliberal
policy reforms across systems largely through the production and reporting of league tables, as
well as its assistance to systems in determining targets, goal-setting, and directly providing
advice to education policy-makers on what actions to take to enhance PISA performance. In
doing so, Grek (2012) argued PISA has both created an interdependence among education
systems and a dependence on PISA produced evidence. More than a simple set of objective
measures, an assessment like PISA operates as a “social phenomena” rousing emotion, shifting
public ideas about the well-being of education systems, redirecting policy priorities, and
initiating action (Meyer & Benavot, 2013). Concern is also raised about the OECD’s role as a
new centre of global governance outside that of the traditional nation-state, steering education
policy formation toward neoliberal principles associated with enhanced competitiveness in a
global marketplace (Rizvi & Lingard, 2009).
Many of these characterisations seem to be rejected by the OECD. In fact, it has been
pointed out that the original design of ILSAs like PISA were to merely act as “thermometers,”
intended to take the temperature of education systems, rather than their current uses as reform
“whips” (Lockheed & Wagemaker, 2013). What is of interest in this article is less about how we
should categorise the OECD’s education work, or the uses/misuses of PISA, rather our focus is
on a new dimension of the OECD’s education work that is concentrating on identifying new
essential ‘skills’ for the worker-citizen to possess and, therefore, for systems to inculcate to
enable societies to flourish. The new global competence assessment might be seen as part of the
OECD’s commitment to human capital theory that they define as “the knowledge, skills,
competencies and attributes embodied in individuals that facilitate the creation of personal,
social and economic well-being” (Brian, 2007, p. 29). For the OECD “economic success relies
on human capital” (Brian, 2007, p. 3) and global competence is a critical element. Global
competence models “a ‘wider’ human capital” around non-cognitive skills and capacities (Sellar,
2015, p. 425).
Measuring global competence is a means to understand how to assist education systems
in the creation of globally competent citizens fit for future social and economic challenges. In
this, the OECD individualizes global competence as a core concern of education systems and
ensures its own prominent position as a leader in measuring progress in addressing such a core
concern. As argued by Auld and Morris (2019), who examined the historical development of the
PISA 2018 assessment of global competence, the OECD is actively positioning itself as the
major assessor of the UN SDGs, a move that (1) may undercut the UN’s broader notion of global
citizenship and (2) may result in simply propagating global elites and global elitism. Our focus is
on the ambiguity of such an amorphous construct from the perspective on the design and framing
of measurement, and how such ambiguity can be advantageous as it affords an organization, like
the OECD, the power to shape and lead future conversations about developing global
competence in individuals and societies.
The Measurement of Global Competence
In its most basic form, measurement is understood as the assignment of numbers to
objects or events according to rules (Stevens, 1946). The measurement process in education, as
well as any field in social science, requires that the resulting numbers are reflective of the
attribute that we intend to measure. In other words, because the attribute, construct, or latent trait
will be represented by numbers, there is an explicit assumption that the thing being measured can
be understood numerically. These numerical ratings are often useful and easily understood but
only when ratings represent simple and discrete behaviours or attributes that manifest themselves
similarly across subjects, context, and time.
According to Thorndike and Thorndike-Christ (2010) the process of measurement
involves three common steps: “1) identifying and defining the quality or the attribute that is to be
measured 2) determining the set of operations by which the attribute may be isolated and
displays for observation, and 3) establishing a set of procedures or definitions for translating our
observations into quantitative statements of degree or amount” (p. 10). These general steps,
which are reliant on one another, begin to highlight the difficult and time-consuming process that
needs to take place before we measure constructs internationally. For example, if the construct
and its attributes are not defined nor widely agreed upon, those doing the measuring have
nothing to isolate and display. On the other hand, when there is agreement concerning the
construct, the measurement process is more straightforward. For instance, the physical attribute
of weight is rarely disputed and although we can use different tools to do the measuring and units
to express weight, the underlying concept remains consistent. However, when the concept of
interest is amorphous or poorly defined, we violate initial and important assumptions about the
fundamental task of measurement, thereby introducing error and limiting or invalidating the
usefulness of the results. Further, when the construct is poorly defined there is a danger that
statistical findings, rather than the underlying theory, will determine what indicators ultimately
enter the measurement model. While choosing indicators based on statistical fit may ensure
statistical validity it does not necessarily lead to an acceptable construct unless it is supported by
a compelling theory that has been clearly articulated and taken into account in the design of the
study.
International assessment rankings based on poorly defined constructs are especially
problematic. For example, inaccurate rankings provide misinformation to participating systems
who often use the data to understand, compare and reform their educational systems. Further,
over time a poorly defined construct becomes normalized in discourse through the ways it is
measured and reported, which is a problem if we are not measuring what we claim to be
measuring, or we are measuring it poorly. Lingard and Sellar (2013) argued in the case of PISA
that the rankings from the literacy, science literacy and numeracy ILSAs often becomes “catalyst
data” that “pressure politicians, policy-makers and systems to respond to comparative measures
of performance and which have real and multiple effects beyond such measurement” (p. 652).
Once results and rankings emerge in a public realm, how the actual construct is defined and
measured is rarely discussed. In other words, at the national and international level, what is
measured becomes the focus of political actors regardless of the quality and/or theoretical
soundness of the measure.
The philosopher Ian Hacking (1986) argues that the categories that emerge from
measurement and classification technologies, particularly when they are taken up by
governments, effectively “make up people” so that they are knowable and subsequently
governable. Hacking (1986) refers to this as dynamic nominalism, the process whereby the
naming of things, particularly abstract concepts associated with human dispositions or
characteristics that are assumed to be measurable, comes to determine how we understand the
construct and the actions that we take as a result. Extending Hacking’s problematisation, we are
interested in the ways that the frameworks, constructs, items and administration of ILSAs
effectively “make up systems” as homogenous entities that can be understood and acted on.
There is power afforded by measurement technologies, often through ranking and categorisation,
in shaping how people come to understand the characteristics of that are measured. This lends
itself to an obvious question regarding drawing inferences regarding global competence from the
assessment – what will it mean for a system to be ranked as having a high global competence or
a low global competence? ILSA categorisations of educational systems (e.g. “failing”, “low
quality” or “low equity”) continue to influence national and sub-national policy and practice.
Given the force with which ILSAs can punctuate national policy discussions, it is important to
scrutinize the assumptions, claims and presuppositions that go into the OECD’s measurement of
global competence.
Defining Global Competence
Global competence is often associated with larger national and global discourses
expressing concern that education systems are not adequately preparing students for a modern,
fast-paced and interconnected world. For instance, the growing focus on “internationalized
schooling” (Engel, Maxwell & Yemini, 2019; Engel & Siczek, 2018) explicitly targets the need
to cultivate the globally competent citizen and worker (Pashby, 2011). Albeit not a new concept,
this more recent emphasis on global competence within schools and school systems brings a new
focus and energy to the perceived need to reform education systems to better take into account
the needs of a global economy – a common reference point for organizations like the OECD
which are committed to viewing fields like education through a predominant lens of human
capital theory (Rizvi & Engel, 2009; Sellar & Lingard, 2014). These needs include fostering
skills for a highly flexible, competitive, and technologically advanced society, citizens who are
poised to take action on global problems, and systems able to respond to ever more diverse
student populations, which Mitchell (2003) saw as a shift away from the liberal “multicultural
self” toward the neoliberal inspired “strategic cosmopolitan”.
While the OECD may have settled on an orientation to its work, definitions of global
competence, though sharing some similarities, are not consistent. Table 1 provides an overview
of the primary components of three common frameworks that focus on education for global
citizenship or education for global competence, developed by UNESCO (2015), Asia Society
(Boix Mansilla & Jackson, 2011), and the OECD (2018).
[Insert Table 1 about here]
Although the three organizations differ in their aims and goals, Table 1 suggests some
commonality across their approaches, including the need to develop specific knowledge,
dispositions, values, and behaviours needed in a globalized world. Additionally, there is a
specific focus on taking action in each of the frameworks illustrating that it is not enough for
individuals to think globally, they must also identify means to appropriately act on global
problems. However, the OECD deviates substantially in terms of the components identified as
global competence (see also Auld & Morris (2019) in their discussion of how the OECD’s more
limited definition of global competence may undermine the wider UN conception of global
citizenship, as elaborated by UNESCO). In fact, the OECD provides no justification as to why
these components were selected, and the extent to which these are universal, rather than
contextual, features of competence. Moreover, while the first two frameworks offer succinct
definitions, illuminated through specific target dimensions, the OECD’s definition and the four
target dimensions are one and the same. We elaborate this further in our analysis of the OECD’s
framework of global competence.
Challenges to defining global competence
Despite the frequently “slogan-like” (Popkewitz, 1980) use of the term global
competence by these policy focused organizations, academic literature points to important
debates and a lack of consensus around the concept of global competence. Specifically, academic
literature focuses on four key points of critique regarding the contested nature of global
competence, which have clear implications for constructing a cross-national measure: the overlap
and conflation with other terms; the narrow individualistic and skills-based orientation; the
multitude of definitions that are often geographically oriented and dependent on who, or indeed
which organisation, is doing the defining; and the implicit assumptions about who in fact can
develop such global competences. The following discussion will take up these critiques in further
detail.
First, a key challenge in defining global competence, a necessity for measurement, is its
considerable overlap with other frequently used concepts, including global citizenship,
cosmopolitanism, multiculturalism, intercultural competence, education for democratic
citizenship, and internationalisation, all of which have their own definitions albeit some more
ambiguous than others. Indeed, as Pashby (2009) has pointed out, there is often a conflation of
global, intercultural, and multicultural education discourses, leading to confusion about exactly
what each term means in different contexts. Rather than a single construct, these frameworks act
instead as a kind of “hub” for various orientations and understandings (Mannion, Biesta,
Priestley, and Ross; 2011; see also Frey & Whitehead, 2009). For example, discourses of global
citizenship entail a wide range of agendas, including education for sustainability, economic
competitiveness, equality and human rights, social justice, and intercultural understanding
(Marshall, 2011), thus challenging any attempt to provide concrete definition for measurement
purposes.
Second, debate continues over the individualistic skills-orientation rooted within global
competence. Some scholars have defined global competence as a set of gleaned skills from a
more expansive understanding of global citizenship (Mannion et al., 2011). The literature on
global competence refers to the specific aptitudes and actions of individual citizens (or workers),
thereby embodying more of a skills focus -- i.e., what a person can and knows to do. Frequently
the emphasis is on the specific proficiencies an individual possesses that provide “a competitive
edge” for upward social and economic mobility (Weenink, 2008, p. 1093). This might include,
for example, the ability to speak a world language with proficiency, adaptability and flexibility in
new circumstances, or to understand and behave appropriately in intercultural environments.
There is considerable debate about whether it is appropriate to distil broad notions of
global citizenship into a set of recognizable, agreed upon, and assessable competencies. For
example, critics point out that it is impossible to pre-define global competence as a set of
knowledge, skills and attitudes, and map those onto “a set of particular and predetermined
activities” (Bamber, Lewin & White, 2018, p. 224). These critiques place global competence into
a larger culture of performativity that “propagates the pre-specification of easily identifiable and
measurable outcomes from curriculum interventions” (p. 225). Additionally, the tendency to
prioritize the actions and achievements of the individual citizen “individualises citizenship by
seeing it in terms of what individuals have, rather than in terms of what individuals do together,”
a contrast with more expansive global citizenship orientations (Mannion et al., 2011, p. 454).
These features are often seen to be at odds as it is not possible to reconcile global consciousness
rooted in collaboration with the advancement of skills for advancing individual competiveness
(Dill, 2013).
These debates lead us to our third line of critique, which is the lack of a universal
international definition of global competence. Instead the definition “depends on the geopolitical
standpoint” of the individual (Baumgratz, 1995, p. 445). In fact, despite its frequent use in
Western contexts, there are differences between North American and European conceptions of
global competence (Baumgratz, 1995). Global competence in the U.S. tends to refer to the
professional relations of individuals and companies in the U.S. with other global market
competitors, and the preparation of American workers in the global marketplace (Watkins &
Cseh, 2009). In fact, a solid proportion of the North American literature on global competence is
rooted in human resource management and organisational learning, specifically “refer[ring] to
economic performance and the optimal way of qualifying American human resources to be up to
the challenges of international economic competition” (p. 445). This includes what Lambert
(1994) referred to as “task performance” in relation to organisational cultures, where
communication, cooperation, and negotiation abroad or with diverse partners are regular
occurrences in professional settings, and part of employee performance. In contrast, European
conceptions have tended to highlight more intercultural conceptions within the region, as
European employees are tasked with working across European member-states with multiple
cultural and linguistic differences (Baumgratz, 1995).
Connected to the above geopolitical differences in global competence, earlier research in
the 1990s suggested considerable variance in the ways that global education is perceived and
practiced in different systems (Hicks, 2003). One leading example is Tye’s (1999) study of the
perceptions of global education in over 50 countries, which revealed multiple orientations,
including environmental considerations, development, intercultural relations, peace, economics,
and human rights, which each embraces its own set of questions and concerns. Another example
is Pike’s (2000) study of the differences in the UK, the U.S., and Canada, which although shared
similar “big ideas,” the national orientations varied from common interest in human beings and
the planet over national development (Canada, the UK) to building general knowledge of
specific countries or places (the US). Within a single national context, there are diverse
definitions of global education, often shaped by the views that national citizens hold about “their
relationship with the rest of the world” (Frey & Whitehead, 2009, p. 273). These differences
within and across systems would most likely mean major variance in what is thought to
constitute global competence.
We might also consider other ILSAs, like the International Civic and Citizenship Study
(ICCS), which introduced regional modules to assess variance across Latin American, Asian, and
European systems. ICCS findings continue to reveal considerable regional differences in the
values that inform civic and citizenship education. For instance, the ICCS 2009 regional modules
revealed a Latin American orientation toward civic knowledge and rule of law, an Asian focus on
self-cultivation and moral development, and European concentration on equal rights and regional
harmonisation. Other research similarly suggests that both regional and national variance in
citizenship values and dispositions challenges the development of universal definitions
(Veugelers, 2011).
Lastly, literature raises concern about implicit assumptions built into frameworks of
global competence and how inclusive these frameworks may or may not be (Dower, 2008).
Related to global competence, Weenink (2008) described cosmopolitan capital as “a propensity
to engage in globalizing social arenas… in which the struggle is for privileged positions” (p.
1092). Cosmopolitan capital is more than simply the proficiencies required to collaborate and
broker deals in a multinational context, but also includes the display of knowledge, attitudes,
dispositions, and tastes aligned with dominant globalized societal spaces. It is a “competitive
edge, a head start vis- à-vis competitors” (p. 1092). Seen as an integral outcome (Yemini, 2014)
of global competence, cosmopolitan capital has been frequently criticized as propagating largely
liberal democratic views and ultimately privileging the already dominant transnational capitalist
classes (Balarin, 2011; Brooks & Waters, 2015; Dower, 2008; Marshall, 2011; Maxwell &
Yemini, 2019; Pashby, 2011; Weenink, 2008; Yemini, 2014). There are implications then for the
OECD’s international measure of global competence, and whether ultimately this assessment
will benefit liberal democratic systems and/or global elite classes (Auld & Morris, 2019).
In its framing as “a new social qualification… in the context of the global economy and
global interdependence” (Baumgratz, 1995, p. 444), global competence seems like an entirely
contemporary idea. Yet, some scholars, like Lambert (1994), Carter (1994), and Baumgratz
(1995) argue global competence has deeper historical roots in white European colonial traditions
related to internationalisation. Here, global competence refers to the kinds of concrete
knowledge, aptitudes, and behaviours needed to successfully negotiate relationships with local or
native populations on behalf of colonial interests. This bears some similarities to the literature on
global citizenship, which as Pashby (2011) argues, has an inherent tendency to simply widen the
scope of the liberal democratic orientations of the national citizen “upward” to the global scale,
furthering the hegemonic orientation of the Northern/Western national citizen. The core debate is
whether there even exists a truly global conception of global competence and if not, whose
global competence is prioritized and who benefits from such an international measurement. The
integration of theories, perspectives, and scholarship from the Global South with those of
Western traditions in the Global North are essential to an integrated “global” conception of
global competence, otherwise priority is given to late-stage capitalist, Westernized, and global
North discourses of global competence (Grotlüschen, 2018).
Overall, this diverse and growing body of literature suggests that considerable debates
remain on the specific focus and clear components aligned with global competence. Although
some broad agreements exist, definitions appear largely driven by different agendas and
orientations -- human rights and equality for all; sustainability and environmental awareness;
economic competition and upward social mobility -- as well as geopolitical contexts, all which
create a challenge for the development of a single internationally agreed upon measure. Yet, as
we discuss in the next section, the OECD has developed a tool that claims to assess global
competence in a diverse set of countries.
PISA 2018 Global Competence Frame
The OECD’s (2018) position paper, Preparing our Youth for an Inclusive and
Sustainable World: The OECD PISA Global Competence Framework, defines global competence
as “the capacity to examine local, global and intercultural issues, to understand and appreciate
the perspectives and world views of others, to engage in open, appropriate and effective
interactions with people from different cultures, and to act for collective well-being and
sustainable development” (p. 7). The framework argues that the aim of global competence is “to
live harmoniously in multicultural communities; to thrive in a changing labour market; to use
media platforms effectively and responsibly; to support sustainable development goals” (OECD,
2018, p. 4). Thriving in an interconnected world and managing the increasing prevalence of daily
intercultural encounters is heavily stressed. For example, the word intercultural is mentioned 74
times in the text portions of the 48 page handbook.
Importantly, the OECD defines competence as much more than a specific skill. It is “a
combination of knowledge, skills, attitudes and values successfully applied to face-to-face,
virtual or mediated encounters with people who are perceived to be from a different cultural
background, and to individuals’ experiences of global issues (i.e. situations that require an
individual to reflect upon and engage with global problems that have deep implications for
current and future generations)” (p. 7). The OECD stresses that “acquiring global competence is
a lifelong process -- there is no single point at which an individual becomes completely globally
competent. PISA will assess at what stage 15-year-old students are situated in this process, and
whether their schools effectively address the development of global competence” (p. 7).
PISA’s assessment of global competence includes both a cognitive portion taken by
students, and background questionnaires completed by students, teachers and school leaders. The
main cognitive assessment is “designed to elicit students’ capacities to critically examine global
issues; recognise outside influences on perspectives and world views; understand how to
communicate with others in intercultural contexts; and identify and compare different courses of
action to address global and intercultural issues” (p. 6). The student background questionnaire
asks students to answer questions on “how familiar they are with global issues; how developed
their linguistic and communication skills are; to what extent they hold certain attitudes, such as
respect for people from different cultural backgrounds; and what opportunities they have at
school to develop global competence” (p. 6). Teacher and school background questionnaires
focus on the integration of “global, international and intercultural perspectives throughout the
curriculum and in classroom activities” (p. 6).
The OECD’s (2018, p. 7-8) position paper outlines four “target dimensions” of global
competence:
1. the capacity to examine issues and situations of local, global and cultural significance
(e.g. poverty, economic interdependence, migration, inequality, environmental risks,
conflicts, cultural differences and stereotypes);
2. the capacity to understand and appreciate different perspectives and world views;
3. the ability to establish positive interactions with people of different national, ethnic,
religious, social or cultural backgrounds or gender; and
4. the capacity and disposition to take constructive action toward sustainable development
and collective well-being.
Each of these four target dimensions “are supported by four inseparable factors: knowledge,
skills, attitudes and values” (p. 11). Intriguingly, the framework identifies a series of skills
outside of the inseparable factors that are necessary to enable these target dimensions including
“reasoning with information, communication skills in intercultural contexts, perspective taking,
conflict resolution skills and adaptability” (p. 14). Further complicating the framework is the role
of attitudes or key dispositions that exemplify and drive forward global competence. Attitudes
are “the mind-set that an individual adopts towards a person, a group, an institution, an issue, a
behaviour, or a symbol” that “integrates beliefs, evaluations, feelings and tendencies to behave in
a particular way” (p. 16). The framework identifies an “attitude of openness, respect for people
from different cultural backgrounds” and a belief that one is a citizen of the world with
commitments and obligations (i.e. global mindedness) as key dispositions for global competence
(p. 16). After attitudes comes values, which “are more general beliefs about the desirable goals
that individuals strive for in life, reflecting modes of conduct or states of being that an individual
finds preferable to all other alternatives” (p. 17). Two values are identified as central to global
competence that can be influenced by education: Valuing human dignity and valuing cultural
diversity (p.18).
The OECD’s framing of global competence works like this (see Figure 1): Knowledge
and cognitive skills will be assessed in the cognitive test, while knowledge, cognitive skills, and
social skills and attitudes will be assessed in the student background questionnaire. Values are
considered to be beyond the scope of the PISA 2018 assessment, although no justification is
given for this. Thus, the OECD PISA 2018 is not assessing global competence, but rather claims
instead to be assessing what it refers to as global understanding. The OECD explains global
understanding as: Knowledge + Cognitive Skills + Social Skills and Attitudes - Values.
[Insert Figure 1 about here]
Curiously, after establishing their case for assessing global understanding as a proxy for
global competence, the OECD states that the cognitive test itself (see Figure 2) that measures
knowledge and cognitive skills/processes associated with global understanding is actually
assessing the four key “target dimensions” of global competence. The assessment includes four
content domains: (1) Culture and intercultural relations; (2) Socio-economic development and
interdependence; (3) Environmental sustainability, and (4) Institutions, conflicts, and human
rights. Many of the components of these content domains (e.g., identity in multicultural societies;
cultural expressions) are abstract concepts that relate to human dispositions and characteristics.
This seems to imply that by assessing global understanding, PISA 2018 will be assessing global
competence after all. It is entirely unclear how and why values are important in the framework of
global competence in the first place.
[Insert Figure 2 about here]
This issue, then, speaks to one of the key problems with the PISA 2018’s assessment of
global competence. The lack of an agreement within the research community about what global
competence is confuses the measurement frame. The OECD framework states that it is building
on these different global education models, which, while they have “differences in their focus
and scope (cultural differences or democratic culture, rather than human rights or environmental
sustainability), these models share a common goal to promote students’ understanding of the
world and empower them to express their views and participate in society” (p. 7). The OECD
explicitly states that “PISA contributes to the existing models by proposing a new perspective on
the definition and assessment of global competence” (p. 7). The evident problem, as elaborated
above, is that some of the differences found within these models and frameworks are not easily
reconciled. The difference, for example, between an orientation toward economic
competitiveness versus one of social justice and intercultural understanding, can be considerable
(Mitchell, 2003; Tarc, 2009; Torres, 2015).
Throughout the framework, there is a conflation of terminology of global and
intercultural competence, with frequent mention of “global and intercultural issues,” “global or
intercultural problems,” and developing students’ “global and intercultural outlooks.” For
example, on page 19 of the framework document is mention of the different cultural orientations
and understandings of global competence, and yet the literature and examples cited are drawn
explicitly from research on intercultural competence. In doing so, the assumption made is that
these two approaches – intercultural and global – work in tandem. What is overlooked is the
ways in which national contexts shape how students understand and respond to scenarios focused
on global issues. For example, Pashby (2009, 2011), Kymlicka (2003) and Dower (2008) have
argued that there can be serious contradictions between ideas of national and global belonging,
and that it is impossible for students to simultaneously articulate a universal idea of global
citizenship while being intolerant toward a particular group within a local or national culture.
Without clear measures to assess this apparent contradiction we can only deduce that PISA may
assess aspects of intercultural competence and not global outlooks of students.
The OECD provides a clear perspective on the policy rationales behind its efforts and
what it sees as the core outputs of its assessment of global competence. For instance, the OECD
states that PISA 2018 global competence measures will serve as the “first comprehensive
overview of education systems’ success in equipping young people to address global
developments and collaborate productively across cultural differences in their everyday lives” (p.
38). As such, the organization claims it will “provide insights on which policy approaches to
global education are most commonly used in school systems,” as well as stimulate policy
prescriptions on how to build teacher global competence, adapt curricula, and inspire new
school-level approaches (p. 38). Despite the lack of consensus around the definition of global
competence, there is a clear aim to shape the actions of governments. While allowing an
international organization to selectively choose and misinterpret academic literature in order to
develop its own definition is clearly problematic, it is even more problematic when we consider
the potential of global competence rankings to influence political decisions. A general notion of
global competence may be desirable but as with any other complex concept, the idea requires a
well-reasoned and exacting definition before it can be accurately measured. In this regard, the
OECD has fallen well short. By moving forward without a generally accepted definition, the
OECD implies that it is an authoritative source with the ability to define and measure constructs
it deems important. Further, by producing a framework on global competence the organization
appears to be engaging in a research process while at the same time ignoring the research
community’s apparent disagreements on the concept.
We should also extend this critique to the categorizations and biases inherent in the
framework, and the undiscussed impact of this on people and the social systems with which they
engage. Over time, the outcomes of many educational measurement systems such as ILSAs
come to represent the system itself. For example, Australia no longer has an education system as
much as it has a system ‘in decline’ (Sellar, Thompson, Rutkowski, 2015). Measuring constructs
is essentially a process of defining, displaying, then quantifying, so that the construct can be
understood numerically. Over time, the numerical output of the measurement becomes the thing-
in-itself, at least as far as the concept is in use in everyday communication. One thinks here of
the ways that intelligence tests took a cultural construct of European analytic problem-solving
and turned it into the (supposed) concept of universal intelligence. Now, when we talk about
intelligence we invariably think of someone's numerical IQ score and historically this has had
direct consequences for many people especially in the ways that they have been governed, and
encouraged to govern themselves. In other words, “the things that we ourselves do are intimately
connected to our descriptions” (Hacking, 1986). Describing a system as having low, mid or high
global competence may have repercussions to and for people within systems.
Conclusion
In this paper, we have focused on the PISA 2018 measure of global competence in order
to consider what it has proposed and the implications of allowing an international economic
policy organisation to lead the development of such a measure. A well-established body of
literature regularly suggests that the OECD is an influential policy actor in education, furthering
a primarily neoliberal agenda (cf. Meyer, 2014). It has been argued that it does this through
steering evidence-based policy dialogues through the creation and regulation of the evidence
(Grek, 2009). The OECD’s power as an objective and neutral global coordinator of educational
measurement means that it can define terms and assign values to them. We are not simply
arguing that cross-national measurement in education is on its own problematic. Rather we are
pointing toward the potential dangers of an international economic policy organization leading
this charge.
For many stakeholders, the OECD signifies collective wisdom, power, resources, and
expertise centrally located in an international space outside of national borders. It is therefore
often considered a neutral global space to develop cross-national educational measures. As an
influential technology, however, the OECD’s PISA does much more than provide neutral data
points. Once the OECD develops a measure and systems accept it, it can create momentum
around an issue that was not previously significant. This is particularly meaningful in a
conceptual space like global competence education, which similar to other amorphous concepts
like grit and creativity, seems to remain open, porous, and malleable, where there is considerable
conceptual room for a multitude of definitions, orientations, and practices.
The OECD has suggested that there is “global consensus” on global competence
suggesting the OECD’s position is to simply meet this demand and supply a product to countries
so that they can measure a universally agreed upon concept. Yet, we question the extent to which
there is global consensus about these concepts as there is no single, agreed upon definition.
Instead, scholars have pointed to the conflation of multiple and at times conflicting rationales
and orientations (Marshall, 2011; Torres, 2015). Moreover, there is not a global consensus about
who is imagined to be and gets the opportunity to become a globally competent citizen, with a
concern that a Western liberal tradition built into global competence may exclude orientations
that do not fit the profile and values of the Northern/Western national citizen (Balarin, 2011;
Pashby, 2011; Yemini, 2014).
To summarize, global competence as defined by the OECD’s framework might be
thought of as a set of desired dispositions that have been pulled together in a form to create
meaning, that will construct value and over time and ultimately become the authoritative
definition of global competence. And, as Hacking (1986) reminds us, intentional human action is
always rooted in description; when “new modes of description come into being, new possibilities
for action come into being as a consequence” (p. 231). Without a consensus definition,
particularly when the reasons for this are cultural, geopolitical and/or historical factors, the
power of testing regimes like PISA to impose new normative definitions remains a critical
problem. In other words, systems have given the OECD considerable power to define global
competence, and in this definition to shape the horizons of action for nations and jurisdictions in
regard to their educational systems (see, e.g., Bieber & Martens, 2011; Grek, 2009; Lewis, Sellar
& Lingard, 2015; Sellar & Lingard, 2014). It would therefore be important to ask what does it
mean to be globally competent at a system or at an individual level? For the actual system
participants, what is it that the categorisation, indeed a ranking as being either #1 or #20, enable?
What can a country do as a result of those rankings – is global competence receptive to
educational intervention and if so, in what ways?
Although many systems have reportedly declined to participate, the very development of
such a framework and subsequent measures can be influential in education policy-making. For
example, SDG Goal 4; Target 4.7, articulates a need for education systems worldwide to ensure
that students have skills and knowledge to promote sustainable development through global
citizenship, indicating that there is a growing mandate for policy-makers to measure progress on
meeting such a goal (Auld & Morris, 2019). Given the circulation of global discourses about the
need to educate for 21st century interconnected world and the international pressure to monitor
progress, a range of local, national, and global stakeholders, likely eager to develop avenues of
advocacy, may become enthusiastic users of such a measure, including in ways that the measure
was not necessarily intended for. Some indication of this is already evident in the U.S. Although
the U.S. is not participating in the global competence assessment, several organizations have
used the framework questionnaires as a non-validated assessment circulated to school districts
across the country (see e.g., World Affairs Council Dallas Fort Worth, n.d.).
Educational stakeholders and policy-makers may, of course, generate insightful ideas
from the PISA measure of global competence. For example, the background questionnaires
aimed at understanding whether and to the extent young people are learning about global issues,
like climate change, immigration, or digital literacy in schools, are interesting cross-national and
comparative data points. The findings may well offer educational stakeholders at local and
national levels new insights in determining how these issues or topics are being addressed in
their respective education systems. Notwithstanding these potentially positive outcomes, by
ignoring the inclusion of “values,” the OECD even under the most ideal circumstances is not
measuring global competence. As such, the OECD should only report specific constructs it is
measuring (e.g. student’s self-report of tolerance) and be forthcoming with both the research and
policy community by not claiming to have a universal assessment of global competency.
Perhaps, given the problems that we raise, there is no possibility of a meaningful global
assessment of global competence, and the OECD would be better served isolating key questions
their members would like answered and including these in the background questionnaires
administered as part of their already existing assessments. Finally, in highlighting the problems
around measuring global competence, we find ourselves echoing the sentiments of Mark
Schneider, the director of the U.S. Department of Education’s Institute of Education Sciences,
who argued in 2019 that one of the problems with PISA is that the OECD does not invest in
research to ensure the quality of the assessments (Schneider, 2019). Prior to administering the
assessment, the OECD should have been more methodical and transparent regarding which
aspects of global competence they consider to be universal, and which may well be better
defined and measured at a local level.
References
Asia Society/OECD (2018) Teaching for global competence in a rapidly changing world.
Available online at: https://asiasociety.org/sites/default/files/inline-files/teaching-for-global-
competence-in-a-rapidly-changing-world-edu.pdf
Auld, E., & Morris, P. (2019). Science by streetlight and the OECD’s measure of global
competence: A new yardstick for internationalisation? Policy Futures in Education.
Auld, E. & Morris, P. (in press). The OECD’s assessment of global competence: Measuring and
making global elites. Engel. L. C., Maxwell, C. and Yemini, M. 2019. (Eds.), Beyond the
established boundaries: the machinery of school internationalization in action. London:
Routledge.
Balarin, M. (2011) Global citizenship and marginalisation: contributions towards a political
economy of global citizenship, Globalisation, Societies and Education, 9(3-4), 355-366.
Bamber, P., Lewin, D. & White, M. (2018) (Dis-) locating the transformative dimension of
global citizenship education, Journal of Curriculum Studies, 50, 204-230.
Baumgratz, G. (1995) Language, culture and global competence: an essay on ambiguity,
European Journal of Education, 30(4), 437-447. doi:10.2307/1503516
Bieber, T. & Martens, K. (2011) The OECD PISA study as a soft power in education?
Lessons from Switzerland and the US, European Journal of Education, 46(1), 101–116.
doi:10.1111/j.1465-3435.2010.01462.x
Boix Mansilla, V. & Jackson, A. (2011) Preparing our youth to engage the world: educating for
global competence. Available online from: papers2://publication/uuid/CB6743CB-9C89-4AAF-
831E-5124907F4569.
Brooks, R. & Waters, J. L. (2015) The hidden internationalism of elite English schools,
Sociology, 49, 212-228.
Brian, K. (2007). OECD Insights Human Capital How what you know shapes your life: How
what you know shapes your life. Paris: OECD Publishing.
Carter, H. M. (1994) Multiculturalism, diversity and global competence, in: R.D. Lambert (Ed)
Educational exchange and global competence (New York, Council on International Educational
Exchange).
Carroll, P., & Kellow, A. (2011) The OECD: A study of organisational adaptation. (Cheltenham,
UK: Edward Elgar Pub).
Coughlan, S. (2018, January 24), England and US will not take PISA tests in tolerance, BBC
News. Available online from https://www.bbc.com/news/business-42781376
Dill, J. S. (2013) The Longings and limits of global citizenship education: The moral pedagogy
of schooling in a cosmopolitan age. New York: Routledge.Dower, N. (2008) Are we all global
citizens or are only some of us global citizens?, in: A. Abdi & L. Shultz (Eds.), Educating for
human rights and global citizenship (pp. 39–54) (Albany, NY: State University of New York
Press).
Engel, L. C., Maxwell, C., & Yemini, M. (2019) The machinery of school internationalisation in
action: Beyond the established boundaries. New York: Routledge.
Engel, L. C. & Siczek, M. (2018) Framing global education in the United States: policy
perspectives, in: L. D. Hill & F. J. Levine (Eds) Global perspectives in education research (London,
Routledge).
Frey, C. J. & Whitehead, D. M. (2009) International education policies and the boundaries of global
citizenship in the US, Journal of Curriculum Studies, 41(2), 269-290.
Grek, S. (2009) Governing by numbers: the PISA “effect” in Europe, Journal of Education
Policy, 24(1), 23–37. https://doi.org/10.1080/02680930802412669
Grek, S. (2012) What PISA knows and can do: Studying the role of national actors in the making
of PISA. European Educational Research Journal, 11(2), 243–254.
https://doi.org/10.2304/eerj.2012.11.2.243
Grotlüschen, A. (2018). Global competence – Does the new OECD competence domain ignore
the global South? Studies in the Education of Adults, 50(2), 185-202
Hacking, I. (1986) Making up people, in: T. Heller, M. Sosna & D. Wellbery (Eds)
Reconstructing individualism: autonomy, individuality and the self in Western thought (Stanford,
CA, Stanford University Press).
Hicks, D. (2003) Thirty years of global education: A reminder of key principles and precedents,
Education Review, 55(3), 265-275
Kymlicka, W. (2003) Multicultural states and intercultural citizens. Theory and Research in
Education, 1(2), 147–169.
Lambert, R. D. (1994) Educational exchange and global competence (New York, Council on
International Educational Exchange).
Lewis, S., Sellar, S. & Lingard, B. (2016) PISA for Schools: topological rationality and new
spaces of the OECD’s global educational governance, Comparative Education Review, 60(1),
27–57.
Lingard, B., & Sellar, S. (2013) ‘Catalyst data’: Perverse systemic effects of audit and
accountability in Australian schooling. Journal of Education Policy, 28(5), 634-656.
https://doi.org/10.1086/684458
Lockheed, M. E., & Wagemaker, H. (2013) International large-scale assessments: Thermometers,
whips, or useful policy tools? Research in Comparative and International Education, 8(3), 296-
306.
Mannion, G., Biesta, G., Priestley, M. & Ross, H. (2011) The global dimension in education and
education for global citizenship: genealogy and critique, Globalisation, Societies and Education,
9, 443–456.
Marshall, H. (2011) Instrumentalism, ideals and imaginaries: theorising the contested space of
global citizenship education in schools, Globalisation, Societies and Education, 9(3-4), 411–426.
Maxwell, C., & Yemini, M. (2019). Modalities of cosmopolitanism and mobility: parental
education strategies of global, immigrant and local middle-class Israelis. Discourse: Studies in
the Cultural Politics of Education, 1-17.
Meyer, H. D., & Benavot, A. (2013) PISA, power, and policy: the emergence of global
educational governance (Oxford: Symposium).
Meyer, H. D. (2014). The OECD as pivot of the emerging global educational accountability
regime: How accountable are the accountants. Teachers College Record, 116(9), 1-20.
Mitchell, K. (2003) Educating the national citizen in neoliberal times: from the multicultural self
to the strategic cosmopolitan. Transactions of the Institute of British Geographers, 28(4), 387–
403.
OECD. (n.d.) Beyond PISA 2015: a longer-term strategy of PISA. Available online from
http://www.oecd.org/pisa/pisaproducts/Longer-term-strategy-of-PISA.pdf
OECD (2018) Preparing our youth for an inclusive and sustainable world: the OECD PISA
global competence framework. Available online from http://www.oecd.org/pisa/Handbook-PISA-
2018-Global-Competence.pdf
Pashby, K. (2009) Related and conflated: A theoretical and discursive framing of
multiculturalism and global citizenship education in the Canadian context, PhD dissertation,
University of Toronto. Available online from
https://tspace.library.utoronto.ca/handle/1807/35921
Pashby, K. (2011) Cultivating global citizens: planting new seeds or pruning the perennials?
Looking for the citizen-subject in global citizenship education theory, Globalisation, Societies
and Education, 9(3-4), 427-442.
Pike, G. (2000) Global education and national identity: in pursuit of meaning, Theory into
Practice, 39(2), 64–74.
Popkewitz, T. (1980) Global education as a slogan system, Curriculum Inquiry, 10(3), 303-316.
doi:10.2307/1179617
Rizvi, F. & Engel, L. C. (2009) Neo-liberal globalization, educational policy, and the struggle for
social justice, in: W. Ayers, T. Quinn & D. Stovall (Eds) The handbook of social justice in
education (Rowman & Littlefield Publishers).
Rizvi, F., & Lingard, B. (2009) Globalizing education policy. (London: Routledge).
Sälzer, C., & Roczen, N. (2018) Assessing global competence in PISA 2018: Challenges and
approaches to capturing a complex construct. International Journal of Development Education
and Global Learning, 10(1): 5–20. DOI https://doi.org/10.18546/IJDEGL.10.1.02
Schneider, M. (2019, 31st January) Is PISA a Victim of Its Own Success? IES Head Calls for
Change. Education Week. Available online from
https://www.edweek.org/ew/articles/2019/02/01/is-pisa-a-victim-of-its-own.html
Sellar, S. (2015) A strange craving to be motivated: Schizoanalysis, human capital and education.
Deleuze Studies 9.3 (2015): 424–436.
Sellar, S., & Lingard, B. (2013) The OECD and global governance in education. Journal of
Education Policy, 28(5), 710-725.
Sellar, S. & Lingard, B. (2014) The OECD and the expansion of PISA: new global modes of
governance in education, British Educational Research Journal, 40(6), 917–936.
https://doi.org/10.1002/berj.3120
Sellar, S., Thompson, G., & Rutkowski, D. (2017) The global education race: Taking the
measure of PISA and international testing (Canada: Brush Education Inc).
Stevens, S. S. (1946) On the theory of scales of measurement, Science, 103(2684), 677-680.
Tarc, P. (2009). Global dreams, enduring tensions: International Baccalaureate in a Changing
World. New York: Peter Lang.
Thorndike, R. M. & Thorndike-Christ, T. M. (2010) Measurement and evaluation in psychology
and education (8th edition). Boston: Pearson.
Torres, C. A. (2015) Solidarity and competitiveness in a global context: comparable concepts in
global citizenship education?, The International Education Journal: Comparative Perspectives
14(2), 22–29.
Tye, K. A. (1999) Global Education: a worldwide movement (Orange, CA, Interdependent
Press).
UNESCO (2015) Global citizenship education: Topics and learning objectives (Paris,
UNESCO).
Veuglers, W. (2011) The moral and the political in global citizenship: Appreciating differences in
education, Globalisation, Societies and Education, 9(3-4), 473-485.
Watkins, K. E. & Cseh, M. (2009) Competence development in the United States of America:
limiting expectations or unleashing global capacities, in: K. Illeris (Ed) Competence development
(London, Routledge).
Weenink, D. (2008) Cosmopolitanism as a form of capital: Parents preparing their children for a
globalising world, Sociology, 42(6), 1089-1106.
Woodward, R. (2009) The organisation of economic cooperation and development (London:
Routledge).
World Affairs Council Dallas Fort Worth (n.d.). How globally aware are you? Put your global
knowledge to the test with a free 15 minute global competence survey: Brochure/Flyer.
Yemini, M (2014) Internationalization of secondary education: lessons from Israeli Palestinian-
Arab schools in Tel Aviv-Jaffa, Urban Education, 49, 471-498.
