1
• Forthcoming chapter in: Lütge, Christiane (ed.), Foreign Language Learning in the
Digital Age: Theory and Pedagogy for Developing Literacies, Routledge.
The Changing Dynamics of Online Education: Five Theses on the Future of Learning
1
Bill Cope and Mary Kalantzis
These are times of unprecedented change in education. Digitally-mediated online education
looms large as one of the most significant harbingers of change. Potentially, for better or for
worse, no classroom and no formal or informal learning process will be left unaffected.
Immediately, this statement demands qualification. On the one hand, online education is a
classical technological disruption of traditional practices of teaching and learning. Yet on the
other hand, some of the technological changes represent in pedagogical terms, little or no change
at all. In fact, worse than that, we will argue some forms of online learning can serve to ossify
anachronistic practices, to a point at times where they almost become back-to-the future parodies
of their past selves.
On the disruptive side of change, business theorists Joseph Bower and Clayton Christensen
speak to technology in general when they analyze “disruptive innovation” (Bower and
Christensen 1995). This is a variation on an older theme of technological and social change
where Joseph Schumpeter famously called capitalism a system of “creative destruction”
(Schumpeter 1950 [1976]: 81). Applying their analysis to education, Christensen and colleagues
predict enormous change in which some old education institutions and teaching practices die
while others thrive (Christensen, Horn and Johnson 2008).
In pedagogical terms, implementing technology need not bring about reform. We can video
our lectures, but the didactic form of the lecture does not change. We can move from print to e-
textbooks, but the genre of the textbook as a medium of content transmission remains the same.
We can deliver courses in learning management systems, but the lock-step logic of the traditional
syllabus stays the same. We can deploy online tests, but the process of assessment to
discriminate the few who succeed from the many who are destined to be mediocre or to fail,
remains unchanged. The paradox here is that the transition to new technology—the technological
infrastructures provided to teachers and learners by the decision makers in our schools and
colleges—may at times force us to replicate didactic patterns of teaching and learning. In this
case, technology stifles the possibility of pedagogical innovation, even when innovation is
needed and perhaps within reach.
Technology does not in itself determine the shape of change. We can put it to different kinds
of use; it only has “affordances,” or a range of possible applications. Psychologist James Gibson
coined this word, capturing the idea that meaning is shaped by the materiality of the media we
have at our disposal. His work is at an elemental, creaturely level: “The affordances of the
environment are what it offers the animal, what it provides or furnishes, either for good or ill”
(Gibson 1979 [2015]: 119).
1
First presented as a keynote address at the conference, Media Literacy in Foreign Language Education: Digital
and Multimodal Perspectives, Ludwig-Maximilians University, Munich, Germany, 12 March 2017. Video version at
https://newlearningonline.com/fivetheses
2
We use the word “affordance” to delineate the range of possibilities for use of an artifact or
tool (Kalantzis and Cope 2020a: 92). Technology offers opportunities for action as much as it
does constraints, for change as well as to ossify old practices. Elsewhere, we have sketched out
seven affordances of digital technologies for teaching and learning, seven things that with
thoughtful design and implementation, online learning technologies could possibly offer that
would enhance learning: ubiquitous learning, active knowledge making, multimodal meaning,
recursive feedback, collaborative intelligence, metacognitive awareness, and differentiated
learning (Cope and Kalantzis 2017, Kalantzis and Cope 2015). These are only possibilities,
because at every opportunity for change in the design and implementation of learning mediated
by digital technologies, we can also create and deploy online technologies that fossilize old
pedagogical practices, so making educational reform harder to achieve than ever.
Building on the idea of digital affordances, this chapter takes one further step. Reaching
beyond the immediate pedagogical affordances of online learning, we explore possible futures
for education in a larger institutional sense. Two possibilities arise: we can “hack” online
learning technologies, subverting the anachronistic constraints built into their designs. Or we can
create new designs for online learning. This second path is the one we have taken in our
Common Ground Scholar (CGScholar) research and development program.
2
Our guiding
research question has been: if a first generation of online learning technologies replicated and
fossilized old pedagogical processes, how might we create a new generation of these
technologies and deploy them as a medium for reform?
We articulate our argument in this chapter in terms of five theses. Martin Luther nailed his
ninety-five theses to the church door in Wittenberg, a lengthy complaint against Catholicism that
initiated the Protestant revolution. Karl Marx wrote eleven theses addressing the intellectuals of
his time, the final one of which was, “The philosophers have only interpreted the world in
various ways; the point is to change it.” Our theses are a more modest complaint about the
education of our in-person and recent online past, and our aspirations for change in the future of
education.
In our hoped-for future:
• THESIS 1. There will be no pedagogical differences between learning in person and
learning remotely.
• THESIS 2. There will be no distinction between instruction and assessment.
• THESIS 3. There will be no class scale.
• THESIS 4. Adaptive and personalized learning will not be at the expense of learning
community.
• THESIS 5. Educators will stop insisting on inequality of outcomes.
But before we start to argue our theses, a definition: online learning consists of learning
interactions mediated by connected digital devices. “Online” in this definition is not a synonym
for remote or distance learning. This is only one possible modality, because online can also be
used to support in-person learning. The semantics is important for us because we want to argue
2
US Department of Education, Institute of Education Sciences: “The Assess-as-You-Go Writing Assistant”
(R305A090394); “Assessing Complex Performance” (R305B110008); “u-Learn.net: An Anywhere/Anytime
Formative Assessment and Learning Feedback Environment” (ED-IES-10-C-0018); “The Learning Element” (ED-
IES-lO-C-0021); and “InfoWriter: A Student Feedback and Formative Assessment Environment” (ED-IES-
13-C-0039). Bill and Melinda Gates Foundation: “Scholar Literacy Courseware.” National Science Foundation:
“Assessing ’Complex Epistemic Performance’ in Online Learning Environments” (Award 1629161).
3
that the infrastructure and pedagogical potentials for online learning are essentially the same,
whether delivery is remote or in-person.
More than this, we want to reverse the case for online learning, as well as the usual direction
of technology and pedagogy design. The gold standard for modern education was traditional in-
person or face-to-face learning, where the technologists reverse-engineered face-to-face in order
to replicate its features online. However, when we push the affordances of online learning, this
can and indeed should become the new gold standard, transforming in-person as well as remote
learning.
More than this, we believe that a crisis in education was, in any event, headed our way.
Indeed, it has been heading our way for a long time, and progressive educators have been
attempting to address it for just as long. Each of our five theses addresses an aspect of this larger
crisis.
Machine solutions have been around for a long time too, variously to be interpreted as
progressive or regressive, as early as B.F. Skinner’s teaching machine of 1954, and the world’s
first computer-mediated learning system, PLATO of 1959, developed at the University of Illinois
(Cope and Kalantzis 2020: 297, Dear 2017).
The COVID-19 crisis of 2020 added a new intensity to the disruption represented by online
education and urgency to respond. Almost all educators were forced to teach remotely, the
medium for which necessarily was online learning. For some educators, the experience required
them to learn new things and gain insights into different processes for teaching and learning. For
others, forced to use horrible tools that hard-coded regressive pedagogies, this proved a negative
experience (Kalantzis and Cope 2020b).
In each of the theses that follow, we will contrast the pedagogical architecture of what we
characterize as the old school, with the not-so-new school of online learning that reverse
engineers and replicates the old school, and then a new generation of online learning tools that
may be a harbinger of genuine change in education.
THESIS 1. There will be no pedagogical differences between learning in person and
learning remotely.
In the old school, it was necessary for students to be in the same room at the same time. The
classical classroom is a communications architecture, in fact a knowledge technology of sorts,
with thirty or so children in the classroom or a hundred or so college students in the lecture
hall—just enough for the teacher to speak and be heard. The logistics of communication,
transmitting knowledge from teacher to learner, required confinement on two dimensions: space
and time. The walls of the classroom confined learners in space, while the cells of the timetable
confined them in time. Here we will parse three foundational artifacts of communication that
were brought into these four walls in these timeslots: the lecture, the textbook and teacher-led
student discussion.
As a form of communication peculiar to education, the teacher-lecture is perhaps the oldest.
“For it belongeth to the master to speak and to teach,” said St Benedict of Nursia, and “it
becometh the disciple to be silent and to listen. If, therefore, anything must be asked of the
Superior, let it be asked with all humility and respectful submission” (St Benedict c.530 [1949]:
chapter 6). St Benedict is a profoundly important person, the inventor of western monasticism, a
system of social and bodily separation from the world for the purposes of learning and
4
ideological devotion. The medieval monasteries become the first western colleges and
universities. In the lecture, the master speaks while the disciples sit silently, listening. Perhaps at
the end there might be space for a question or two from the bravest souls (Kalantzis and Cope
2020a: 54-6).
The second great innovation of modern education is the textbook. Petrus Ramus, appointed a
professor at the University of Paris in 1551, was perhaps the most prolific author in the century
after the invention of the printing press. He was author of over 750 editions, though today he is
almost entirely forgotten. Making no original intellectual contribution of his own, he was the
inventor of that eminently modern textual form, the textbook. His books were reconstructions of
classical authors—Euclid’s geometry and Aristotle’s logic among other subjects. Ramus divided
complex disciplines into small and digestible chunks, illustrating them with diagrams, ordering
and numbering them from simpler to more complex ideas, and logically arranging the subjects
according to the organizational principles of a table of contents or concept taxonomy. In this
way, knowledge which was in its classical sources dialogical was reordered into a static visual-
spatial array. As a way of organizing the delivery of learning, it was now possible to direct all the
students to be on the same page (Kalantzis and Cope 2020a: 166-8, Ong 1958 [1983]). In these
senses, Ramus was hugely innovative, creating the modern arrangement of school subjects with
which students were to become familiar over the centuries.
The lecture and the textbook perform the same communicative and pedagogical function,
something that Courtney Cazden calls “exophoric reference” (Cazden 2001: 74). Endophoric
reference points to something within sight or reach. Exophoric reference speaks to something not
present (Kalantzis and Cope 2020a: 177-8). The remarkable thing about education—its lectures
and its textbooks—is that there is nothing outside of the classroom to which the lecture and the
textbook cannot refer: volcanoes, illnesses, poets, atoms, insects, armies, vectors, or anything
and everything else in the world.
Lectures and textbooks are distinctive modes of referring which share features that become
fundamental to early modern education. The knowledge-making capacities of the learner must be
suppressed as students become passive receptors of authoritative meaning, conveyed by the
spoken word of the lecturer or the written word of the textbook author. This is transmission
pedagogy, or what Paulo Freire called “the ‘banking’ concept of education... in which the
students are the depositaries and the teacher is the depositor. Instead of communicating, the
teacher issues communiqués and ‘makes deposits’ which the students patiently receive,
memorize and repeat” (Freire 1972: 45-6). Very little scope is left for knowledge-making on the
part of students, let alone critical, creative or design thinking.
Lectures and textbooks are synoptic—the single speaker or writer summarizes the world; a
world of potentially many voices is reduced to a single authoritative voice. Problematic and
complex realities are reduced to definitive facts. Little scope is left for interpretation or
alternative perspectives beyond the received, officially sanctioned and conventional wisdoms.
These media and learning architectures become the conduits for the characteristic social
architectures of the modern world—or at least version of the modern world where some people’s
capacities to act are greater than others whose capacities are constrained: the rich versus the
poor; the boss versus the workers; the creator versus the consumer; the expert versus the
layperson; the powerful versus the powerless. The lecturer or the textbook author both position
themselves in a role of epistemic authority, so students become disciples who get used to
following.
5
And now, a third major communicative mode in the modern classroom, something Courtney
Cazden calls “classroom discourse.”
T[eacher]: “Three-quarters of the crayons in Mrs. R.’s box are broken. How many unbroken Crayons are
there?”
S[tudent]: Sean offers to show his solution. “It would be four.” (He draws and talks on the chalkboard.)
If the student’s answer is right, the teacher will say so or ask another student to respond to the
same question.
This is the beginning of a typical pattern in teacher/student talk: teacher initiates, then student
responds, and finally teacher evaluates, or “I.R.E.” The teacher asks “display questions,” as if
they didn’t know the answer (Cazden 2001: 30, 46). The student plays this game by guessing
what is in the teacher’s head. “Hands up!” after the question, then one student answers at a time,
a proxy for the others who must listen in absolute silence if the conversation is not to become
chaotic.
There are some fundamental and all-too-familiar problems with this pattern in classroom
discourse. The teacher is habitually the initiator. Only one student can respond at a time,
followed at most by a second if the first falters, enough to get at the answer the teacher expects.
These are almost invariably the students who need least affirmation as knowledge leaders. The
answer, moreover, is oral, ephemeral, eminently forgettable, and open only to subjective teacher
evaluation. The teacher develops a distorted sense of those who seem to be the keenest students,
and in further questioning, these become the favored ones. Meanwhile, the other students in the
class are positioned as listeners, passively. For them the experience must unavoidably be boring.
And the conversation is by dint of necessity confined to the architectonics of classroom space
and time.
Now comes online learning, and nothing really changes. The flipped classroom replicates the
lecture (Bishop and Verleger 2013). The e-textbook replicates the print textbook. The
videoconference screen replicates classroom discourse, its tiles as rigidly arrayed as the most
fixed of classroom desk formations or seating in the lecture theater. More than one person may
put up their hand, but only one person can speak at a time.
Of course, there are some changes, but they are marginal. There are a few minor concessions
to learner agency—you can speed up or slow down the video lecture, and skip or repeat parts.
There may be some moving images or responsive quizzes in the e-textbook. More students can
answer with clickers or online question, but the answers are canned, one answer correct, and no
space for student voice. The videoconference conversation may be recorded and transcribed for
people who miss it or who want to go back over a point, but the fundamental IRE patterns of oral
classroom discourse remain. Or worse, when there is no way of knowing whether the lecture has
been “attended” or whether the videoconference attendee is looking at the camera while scrolling
through their social media feed.
This is the not-so-new school. Certainly, we no longer have to be in the same space to learn.
But pedagogically, the underlying epistemic confinements remain. The cells of the timetable
become the cells of the learning management system, blocks of time in the syllabus, day after
day, week after relentless week. In the flipped classroom and videoconference classes, the
teacher still mostly talks while the students mostly listen. Or when there is class discussion just
the keenest students speak, and only one at a time. The communications architecture remains
one-to-many. Didactic, knowledge transmission pedagogy has migrated online.
6
How could things be different? What can the new school be like? Here, we are suggesting
that innovations in online learning which were created in the first instance for remote delivery
could offer pedagogical improvements for in-person classrooms. We are going to speak in a
narrative way here about our own research contribution to new designs for learning in set of
technology innovations we have called CGScholar and our teaching experiences in our Learning
Design and Leadership program using this online platform. It is not enough to know the issues
and critique the field if one does not attempt to test solutions. Elsewhere, we have reported in a
more formal fashion on our research results (Cope, Kalantzis and Searsmith 2020, Haniya et al.
2018, McMichael et al. 2020, Montebello et al. 2018).
We record the lectures and post them online in CGScholar, because we do buy the “flipped
classroom” idea that lectures are a waste of valuable class time, whether that time is in-person
and synchronous or remote and asynchronous. We've learned from our MOOC experience that
between six and nine minutes is an optimum length for a video lecture. The discursive logic,
moreover, is no longer a simple matter of telling.
Each of our videos ends with two kinds of prompt: the first is to comment, to discuss an idea,
and comment on others’ comments. The comment should be extended, say fifty words or more.
You have to have watched the video for your comments to make sense because they will be there
for everyone to see.
The other prompt is for the student to make an update themself, to research and contribute
content to the course—some ideas from their own experience, a video or infographic discovered
on the web with some meaningful explanation, or a linked article or data presentation. Then
students comment on other students’ updates. This means that the learners become co-designers
of the course, and there is much to be learned from them—valuing the stuff they already know or
just have discovered, arguing their perspectives and showcasing their identities. Where
traditional pedagogical architectures stamp sameness onto students—watching the same lecture,
reading on the same page, and demonstrating what they have remembered—the pedagogy
embedded in CGScholar can be characterized as productive diversity. This is possible in the
online environment in ways that were difficult to achieve in the in-person environment without
online support.
Not that this replaces the agency of teachers with the agency of learners. Rather, it strikes a
new balance of agency in learning. The role of the teacher changes, not that the teacher retreats
into a position of agentless facilitation, nor do they become a mere “guide on the side.” Rather,
they have a new, centrally important role to scaffold learner agency in generative way, to nurture
learners’ capacities as knowledge co-producers, and to give voice to their perspectives, interests
and identities. Video mini-lectures serve as prompts for the co-design of learning.
Then, instead of the textbook or a syllabus, we have a pedagogical artifact that we call a
“learning module.” Learning designs are as important as they ever were in the era of the
textbook. But there is no need to summarize the world in the manner of a traditional textbook,
because exophoric reference is never more than a hyperlink away. Instead of synopsis, a learning
module curates. Instead of single-minded authorship, the links can take the learner to multiple
voices, requiring critical discernment of perspectives and interests. And instead of the
monologue of the traditional textbook, each instructor or teacher update in a learning module is
dialogical, prompting argument, appraisal and further exploration of new realms of knowledge.
What then of classroom discourse? It’s just the same and utterly different. Initiate: the
teacher posts a prompt for discussion. Respond: all students respond, and their distinctive voices
can be heard. Evaluate: the teacher can respond to the student, but perhaps doesn’t need to
7
because other students have already responded exhaustively to each other’s responses. The
teacher can see all responses and learning analytics can track participation.
Here now are the radical differences, things that are impossible to achieve in oral discussions
whether in-person or via videoconference. Instead of one or two people responding to a
question—particular kinds of people who feel confident to respond (often those who least need
to respond)—everyone can respond. Indeed, everyone can be required to respond. Instead of an
oral response, the response is in writing, and so more carefully framed, a step along the path
from vernacular orality to academic literacy.
There are lowered barriers to response, because even if the discussion is in real time, the
students have a few seconds to reflect before they press the comment button to submit. The
confidence of the more hesitant students to comment can grow as they see other comments
coming through.
Instead of second-guessing the right answer—for this is all that is practicable in oral
discussion—the presence of more comments reveals diversity of perspectives, even if these are
just differently nuanced ways of framing and articulating a similar response.
Where listening to the teacher’s spoken question and another person and answering is sub-
optimal cognitive load (boring!), scanning a feed is more than sufficient cognitive load, aligning
also with habits of reading in the era of social media. This is also a good way to replace or
supplement ephemeral and eminently forgettable oral discourse with documented conversation, a
mnemonic to which it is easy to return to recall or clarify an idea.
And as for the confinements of time and space, there no difference and no particular benefits
whether this happens in simultaneous time and co-located space or asynchronous time and
remote space.
An interesting thing happens when we make this transition. Online infrastructure becomes an
indispensable supplement to face-to-face, because it can do things that face-to-face could never
do without it. Face-to-face learning is transformed. Now, there becomes no pedagogical
difference between remote and learning face to face, just a circumstantial difference of time and
space.
THESIS 2. There will be no distinction between instruction and assessment.
Next, old school assessment, and by “old school” we mean not terribly old at all. The artifact of
the school test is an invention of the late nineteenth century whose engineering was refined in the
twentieth century.
In the old school, there was a sharp distinction between instruction and the peculiar practices
and artifacts of assessment. Learning and assessment were built around different kinds of
knowledge artifacts and came at different times. The relation of learning to assessment was
linear: first learning, then assessment (before moving on to a new topic). By virtue of its
temporal arrangement, assessment was typically retrospective and judgmental. In order to be fair,
there was to be nothing new in the exam (the assessment theorists have a word for this,
“validity”), nothing to be learned except the scope of one’s previous learning or ignorance. So,
assessment was mostly summative (of learning, for administrative purposes) rather than
formative (for learning, or pedagogical purposes).
As for its contents, the test was a sample survey of knowledge that had to have been
remembered. To learn was to memorize; assessment was to find out what had been remembered.
8
Until today, long-term memory remains the measure of learning for educators who oppose
progressive approaches such as inquiry learning (Hmelo-Silver, Duncan and Chinn 2007,
Kirschner, Sweller and Clark 2006). And the definition of long-term, in the reality if not the
rhetoric of traditional summative assessment: until the day after the test.
Two strange knowledge evaluation artifacts were created to test memory: the select response
assessment on the supply response assessment. These are strange, we say, because learning and
knowledge activity is never so radically individualized, without social supports in the form of
things that can be looked up and collaborators who can be asked.
In the case of select response assessments this was a particularly strange game: to distinguish
the one correct answer hidden beside deceptively and only slightly wrong “distractor” answers
(Haladyna 2004). Perhaps a student might guess right, but without understanding. Perhaps the
distraction may have been effective, because the reasoning anticipated in the distraction was
sound. Getting the answer wrong for the right reason is built into distractor item—the answer is
nearly right, prompting some degree of correct reasoning for it to be “nearly.”
We test-takers have lived with the frustrations of this deeply flawed knowledge evaluation
system for long stretches of our lives. As the test is a game of trickery, test-taking is as much
about learning how to play this strange game and spot the tricks as it is about the answers
themselves. If the question is “discriminating” enough (to take a term from item response
theory), it’s at the edge if your knowledge and capacity, in which case it’s gambler’s luck
whether get an answer right or wrong.
The knowledge being tested in select response assessments is also about answers stripped of
reasoning—you are rewarded if you happen to land on a fact that is correct or if you determine
the right answer by mechanically applying a procedure. The methodological flaw of this
approach is that this requires an entirely tendentious inference about understanding on the part of
the student. If the student gets the answer right, they must have understood. So the assessment
argument proceeds by means of a triangle: test item <=> interpretation of the answer <=>
inference about cognition inference (Pellegrino, Chudowsky and Glaser 2001: 44). But of course,
without opportunity to expose their reasoning processes, the inference about the learner’s
understanding can only ever be a guess, and often that guess is wrong.
Then there is the issue of the kinds of knowledge that can be assessed in these ways. Take the
all-too-ubiquitous select response reading comprehension test, for example. It is not possible to
test for the most important things such as interpretation, for instance: which characters in the
novel or evidence presented in a scientific argument have the strongest cogency for you as
reader; the scientific or moral agenda of the author; or the narrative or reasoning moves that are
made. It is only possible to test for definitive “facts”, and these are the least germane to
understanding narrative or the processes of a scientific argumentation.
This leaves us to test for memory of the things that are least relevant and most forgettable.
The standard of knowledge is reduced to empirical facts and faithful application of procedures,
but not interpretation, not judgement, not usefully variable perspective, not critical reflection, not
innovative design thinking or practice.
And yet another problem: the assumption is that knowledge is a matter of templated
sameness. If I am correct and you are too in this system of knowledge measurement, my answer
must end up being the same as yours. This is an architecture that imposes epistemic sameness.
But in life, real knowledge is made by bouncing different perspectives off each other. Diversity
is a powerfully productive dynamic in collaborative learning.
9
As knowledge artifacts, supply response assessments—classical essay exams or short, open-
ended answers—are a little less tendentious than select response assessments. At least reasoning
can be exposed and perspectives articulated. However, the strange part is the “closed book”
exam in which informational responses have to be provided without access to information other
than that which can be memorized. These too become tests of the memory of artificially isolated
brains, rather than tests of durable and intrinsically sociable knowledge. For both of these
canonical forms of assessment, the focus on memory is in its nature an individual focus, erasing
the social provenance of knowledge, and eliminating collaboration from the measures of learning
that ultimately count.
Now comes digital technology and the not-so-new school. Computer adaptive testing is a
development of select response assessment in which students are served questions from an item
bank. If you get an answer right, you are given a harder question next; if you get it wrong, an
easier one. In terms of item response theory, the “discrimination” works better because the
questioning quickly moves away from questions that are too easy for you or too hard for you, to
questions which are just hard enough for the distraction tricks to be genuinely disorienting. Also,
no two students take the same test, which means the individualization of memory can be even
more rigorously enforced. For in this knowledge paradigm, to collaborate is to cheat.
One can be awed at the sophistication of computer adaptive testing (Chang 2004, Chang
2015), but the underlying logic of select response assessment stays the same, along with its
fundamental deficiencies. The only difference is that this kind of assessment now happens with
greater precision and on an industrial scale.
Automated essay assessment mechanizes supply response assessments. Humans grade a
sample of scripts across a small number of levels, then the computer assigns grades to new
scripts by a process of statistical comparison of dumb keystrokes (Shermis 2014). The statistical
reductions are never revealed, because that would make it too easy to game the test. But here are
some tricks: longer texts are ranked higher than shorter because students who write longer
answers are usually better than ones who write shorter. This is just a statistical fact. Use a wider
range of words (even if they are nonsensical) and write longer sentences (even if they are
convoluted) and you will get a better score (Cope et al. 2011, Vojak et al. 2011). Semantics is
never better than “latent” (Landauer et al. 2007), a tendentious guess, because computers can
only do sums about recorded typographic characters and the spaces between them.
As for the temporal separation, instruct then assess, the creators of online learning
environments love their select response tests—or “quizzes” if we need a euphemism to lower the
stakes or soften the blow. Having more of them, and imposing them more frequently at the end
of every chapter of the e-textbook or every few days in the learning management system, does
little or nothing to change their underlying logic as select response assessments. These are no
more than summative assessments even when disguised as formative.
Our gesture to a the future of education is to say that in the new school, there will be no
separation between instruction and assessment, no difference between the artifacts of learning
and the processes for the evaluation of learning outcomes, no assessment of learning (summative
assessment) that is not in the first instance for learning (formative assessment). Testing will
disappear because assessment is embedded everywhere. The test is dead. Long live assessment!
But how? For our model of online support infrastructure in the new school, we will return to
our experimental CGScholar platform. In CGScholar, everything that happens in-person or
remote learning context is documented: learner responses to instructor content updates; learner
contributions of knowledge in the form of updates and comments on each other’s comments;
10
projects that involve both quantitative and qualitative review judgements made by peers, teacher
or self against disciplinary criteria spelt out in rubrics; unstructured and structured (coded)
annotations; feedback-on-feedback; select response and open-ended knowledge or opinion
surveys; natural language processing analyses; and beyond this, pushing the potentials of
machine learning and artificial intelligence to deepen assessment and learning (Cope, Kalantzis
and Searsmith 2020). Everything is being assessed, and these include all the legacy processes of
measuring learning including rubric-based review in supply response assessments and item-
based analyses in select response assessments—plus of course much more.
Across twenty two assessment data types, all the work undertaken by students is potentially
assessable across three macro-measures: demonstrable knowledge; focus or effort and
perseverance; and help, or a collaboration measure where we set out to measure and reward
collective as well as individual intelligence. In Figure 1, we in the last week of an eight week
course with approximately forty students, where there have been over 14,000 pieces of
actionable feedback. Our learning analytics aster plot visualization draws on over 3.3m
datapoints, well beyond the capacity of any arrangement of teachers and learners without online
infrastructure support (Cope and Kalantzis 2016). Every student sees their own data in their
personal aster plot. Every piece of data a teacher has is transparently accessible to the student.
Fig. 1: CGScholar’s learning analytics
11
Here are some the potential changes for the new school. Instead of a limited, at-the-end
sample, assessment is embedded throughout all learning. Instead of one-shot assessment as the
end point in a linear process, feedback is incremental and cumulative and recursive. A learner
can keep asking for clarification and can offer feedback on feedback.
Instead of feedback that is retrospective and judgmental, against the same learning outcome
measures, assessments can be framed in ways that are prospective and constructive. In these
ways we have shifted the primary locus of and mode assessment from summative to formative.
Every moment of assessment is legible to students, available to them just-in-time and just-
enough. Summative assessment is no more than a retrospective view of learning progress, where
every assessment datapoint has in the first instance been legible as feedback and thus formative.
The measure of learning is the incremental processes of knowledge making including recorded
interactions with peers and resources. Digital devices serve as cognitive prostheses or the
extensions of memory and supplements to mental procedure, ubiquitously essential to the
complexities of our digital lives, and now integral to learning and its assessment as well.
Instead of highly tendentious inferences about cognition, learning is measured by the
knowledge representations that students make. Our focus is less focused on cognition, and more
on the knowledge artifacts that students make and the processes of their making.
Instead of individualized memory, social knowledge can be measured as effective leveraging
of peer collaborations and carefully navigating knowledge in the social memories we keep in our
pockets or and have at our fingertips. Critically careful and effective use of cognitive prostheses
helpfully replaces individualized mnemonics.
Instead of relying on strange testing artifacts, assessment is of authentic objects of knowledge
work. Instead of being a mysterious, distanced and somewhat arbitrary process, assessment
criteria are spelt out explicitly in rubrics and assessment processes democratized.
Instead of long term memory, the measure of learning is disciplinary practice—being a
writer, or being a scientist, or being a mathematician who shows their workings. Instead of low-
level thinking in the form of recall of facts and correct application of memorized procedures, we
can focus on higher order thinking including measures of interpretive, innovative, and applied
thinking.
Instead of assessment falling solely onto the shoulders of the teacher, it is crowdsourced.
Internet theory argues that the distributed “wisdom of the crowd” can be as perspicacious as
concentrated expertise (Surowiecki 2004). Instead of individual intelligence we are measuring
individual capabilities in contexts of distributed cognition and the quintessentially social mind
(Gee 1992 [2013]).
Instead of standardizing the minds of learners so they regurgitate correct answers, we nurture
productive diversity where differences of perspective, judgment and interpretation are made
visible and valuable for learning in a context of epistemic collaboration.
Instead of a gaming (and thus eminently gameable) system that relies on trickery, we create a
transparent system of knowledge collaborations. What previously we might have wanted to call
out as cheating, we now track and value as reciprocal learning and knowledge making. What
previously might have been regarded as plagiarism, we teach as the ethical requirement to
acknowledge sources and to document the social provenance of knowledge. In a world of
increasingly complex and challenging problems, such collaboration is vital.
The result: instead of instruction and assessment, now we have a newly integrated learning
reality that we call “reflexive pedagogy”. This adds up to something quite new in education,
12
though not-so new in terms the longstanding aspirations of pedagogical reformers and their
social epistemologies, from Rousseau, to Montessori, to Dewey. The main difference today is
that online learning infrastructures make reflexive pedagogy universally practicable.
THESIS 3. There will be no class scale.
In the old school, there was an institutional pragmatics of one teacher to thirty-something
students in the classroom or one teacher to more or less than one hundred in the lecture hall. This
is classical one-to-many communications, born in the era of “mass media” and “broadcast”
communications, an era of silent audiences and read-only culture. In institutions of education the
scope to scale up or down from these sizes was limited. Fewer students per teacher was too
expensive; too many more was impracticable.
Then e-learning comes along, but in the not-so-new school we keep classes about the same
size and the architecture of knowledge communication one-to-many. Class scale does not change
much in learning management and other e-learning systems with their old fashioned hub-and-
spoke model of knowledge transmission.
Scalability is important for the future of education from the point of view of a general crisis
of resourcing and access. Not enough resources go to education for it to be able to meet its goals,
either for individuals or society. Access remains more restricted than it should be for a purported
“knowledge society” (Peters, Marginson and Murphy 2008). Like every sector of the economy,
resources can be more efficiently and effectively used and wider access can be achieved at a
lower cost. Online learning, we argue, can increase the productivity of teachers, not to exploit
them further, but to make their work more effective and more rewarding. This will require a shift
in the allocation of resources away from bricks and mortar to teachers, instructional designers
and technical support. For in-person learning underpinned by online learning architectures, the
bricks and mortar structures to be redesigned to be more sociable.
MOOCs have become the poster-children of scaling and accessibility in the era of online
learning. However cheap it has become to enroll in courses created by professors who work in
expensive and previously hard-to-access institutions, there has been a pedagogical price to pay.
MOOCs mostly rely on select response assessment, with all its faults. Mostly, the learner is
isolated with their screen, watching videos and taking tests without interaction with the instructor
and no interaction with peers. At best, there might be peripheral conversations in discussion
boards. Peer review is occasionally used, but for summative evaluation rather than formative
assessments where learners might receive feedback that can improve their assessable work. If the
focus of traditional classroom or lecture theater is scale limited to seats, MOOCs are captive to
scale as well—this time economies of large scale, so they can be offered cheaply while still
pulling in a lot of revenue. Though new to education, MOOCs are quintessentially old,
transmission media. They are a twenty-first century educational equivalent of twentieth century
Fordist mass production.
However, in the genuinely new school, there is no necessary scale. An example: we have
been working with one of our graduate students and a research colleague in a middle school in
rural Wisconsin. Gradually, students and teachers have been making the choice to move from the
“legacy” classrooms of one-to-thirty or so, to a new part of the school where the walls have been
torn down. This new school-within-a-school is called “Create.” The teachers have a glass office
in the middle of the space. Small, glass-walled rooms allow half a dozen students to work
13
together, noisily if they need to be that. Pairs of students quietly work together in the open space,
or alone in comfortable furniture if they need silence for reading and reflection. In an
experimental intervention which connected them through CGScholar, sometimes the class was
one hundred, sometimes a handful of students working on a project together, sometimes a
teacher working on-on-one with a student. There is no scale in the sense that the scale is flexibly
variable. The learning that is happening scaled up or scaled down by the teacher from the whole
class level to individual students using the aster plot learning analytics visualization (van Haren
and Harroun 2019).
MOOCs could work this way too, but they don’t because their platforms have not been
designed for anything other than mass production. They could offer ways for learners to become
knowledge creators rather than consumers of memorized content. They could be highly
collaborative. They could nurture learning community. They could support variable scale, where
working closely giving and receiving the feedback of three peers is no different in a class of
three than it is in a class where three peers are randomly assigned from three thousand. But they
don’t. To demonstrate what is possible, this is what we have attempted in CGScholar with
groups at times very small, at other times upwards of a thousand learners (Muck 2015).
THESIS 4. Adaptive and personalized learning will not be at the expense of learning
community.
The old school was paradoxically an asocial place. Indeed, at times, the restrictions put in place
to limit sociability were explicitly anti-social. Here in the traditional classroom we have students
sitting closely together listening to the teacher, or in closely arrayed rows listening to the
professor in the lecture hall. “Quiet please!” For practical reasons, in this communications and
knowledge architecture, they can’t speak to each other. That would be disruptive (class “noise”),
and counterproductive to the extent that there is no way of knowing whether the peer interactions
are on-task.
Lines of sight are individualized too, centralization of vision in a hub-and-spoke model of
attention. The students look to the teacher and the blackboard, demonstrating their focus on the
learning by averting their eyes from those beside them. The teacher surveils the class, watching
whose eyes are watching theirs. This is the classical gaze of modernity. The social meaning of
these lines of sight is captured in Foucault’s description of Jeremy Bentham’s nineteenth century
design for prisons, the “Panopticon,” where the guards can see every prisoner but the prisoners
can’t see each other (Foucault 1975 [1977]: 200).
The great irony of Covid-19 for the old school was that students were close enough to
become pathogenic dangers to each other, even though the communications architecture strictly
distanced students from each other in pedagogical and epistemological terms.
In this old school, moreover, every student does their own work. Recall the anti-social
gesture of the child covering their work with their hand so another nearby student couldn’t copy
their answer. Then, when it came to the test, the student was absolutely alone. The measure of
learning is individual memory of facts and procedures, enacting the fiction that because brains
are individual, mind should be too.
Nevertheless, schools are very social places, where all kinds of deeply human relationships
can develop, from friendships to bullying. But because the formal spaces and times of learning
are so strictly based on practices of social isolation, the social part does not happen until release
14
from official learning into the playground or the cafeteria where the reins of surveillance is
loosened.
Meanwhile, learning management systems and videoconference classes operationalize the
old school processes of isolation, marginalizing dialogue to the discussion board or the chatbox.
Truly personalized discussion can only happen in private messaging, the twenty first century
version of the paper note flicked to another student while the teacher’s eyes are directed
elsewhere. Learning analytics in mainstream learning management systems are heavy-handed,
focusing on the student’s attention to the instructor’s delivered content (did they finish the video
and take the quiz?), and not the potential sociability of e-learning ecologies. We’re still in the
Panopticon.
“Personalized,” self-paced and “adaptive” learning appear in the era of digital learning. But
these are not so new, because textbooks allowed this too. You could always back over past
content when you needed to recall or revise, or work ahead of the class because you already
understood the current chapter.
Personalization in digital learning environments can produce greater isolation than ever,
where self-pacing reduces learning to one-to-one relationship of learner to the screen. With few
fundamental differences, the screen of digital learning becomes a replicant of the page of the
print textbook.
However, in plain sight social media point us towards the possibility of highly social
processes of learning interaction, where the dynamics of interaction are generated by the
“stickiness” of interpersonal responsiveness and senses of community belonging. In CGScholar
we achieve this by moving the discussion board from the unassessed and unaccountable margin
of the platform to the assessable center. We encourage students not only to respond to the
teacher’s texts and videos, but initiate and introduced content as well. And we have moved peer
reviewed feedback from the summative/judgmental end of a project to the formative/constructive
middle, so that peers can offer genuinely useful, actionable feedback, and officially become an
integral part of the learning process.
The changes we are suggesting align with a profound change in the zeitgeist that has
accompanied the rise of social media in the first decades of the twenty-first century. However
obvious the lessons of social media, these new ways of engaging don’t come automatically given
the legacy habits of learners and teachers in formal learning spaces. Change needs scaffolding
and nurturing.
Moreover, anxious commentators have analyzed new social problems in these media—the
narcissism of self-promotion, the manipulations of the tech giants that shape personal interests to
suit their commercial interests, and the enormous profits these companies make from content we
have given to them with no remuneration, our “gift” of social labor (Kalantzis-Cope 2016).
Some of this we can and must eliminate in digital learning environments. The logic of social
“stickiness” should be quite different. “Friends” and “followers” are terrible ideas for learning.
In CGScholar we have replaced them with “peers” when one-on-one and “community members”
according to flexible social scale, variably a group within a class, a whole class, or multiple
classes. We offer transparent feed preferences. We can eliminate the commercial pleading and
manipulation which constantly redirects eyes back towards an advertiser in social media feeds.
In CGScholar we attempt to leverage for learning the productive logics of social media while
bracketing away the educationally counterproductive ones. Given contemporary shifts in
sensibility, this move may have become an imperative. Our media dispositions today have
produced a crisis for the old school that we call “attentional.” Listening to lectures is an absurdly
15
sub-optimal cognitive load for today’s students who on their personal devices have become
habituated to designing their own information feeds then skipping through their messages.
Compared to these intrinsically participatory media, passive, knowledge reception media are
intolerably boring. We have become used to constant interaction, taking an active role in
dialogue rather than just being recipients of monologue. We have become used to being creators
of media as much as we are consumers.
Returning to the question of scale, we have become used to changing scale of our social
interactions according to need and interest, where large scale (the scope of social media, and the
large followings of some feeds) and small scale (our immediate friends and specialized affinity
followings) overlay and interleave. Mass and micro sociability are aspects of the same media
phenomenon.
In these ways, the logic of social media can be translated in the new school into an ecology
of social knowledge. There is a reciprocal obligation to contribute knowledge and offer
feedback, to give to the knowledge community as well as to receive, to be discerning about what
is found from social sources and what might be discovered. Intelligence is collaborative. The
glue that moves learning forward has become social: the “stickiness” of reciprocal learning. And
adaptivity and personalization are not at the expense of learning community; they are just
momentary changes in a system of flexible scale.
THESIS 5. Educators will stop insisting on inequality of outcomes.
“No child left behind?” “Every child succeeds?” These were the slogans that accompanied
attempts to reform schooling in the first decades of the twenty-first century in the USA and
elsewhere. But not much changed because the measure of success remained that century-old
invention, the standardized test.
The old school relentlessly insists on inequality. Here is the process, a kind of bait-and-
switch. We admit students into a grade, course or class because we have decided they can all do
the work. Then we work hard to differentiate them via two processes: standardization and
normalization. Standardization means we expect them to do the same work in the same way, a
logic first invented for intelligence tests (Binet 1905 [1916]) but later applied to anything and
everything that might conceivably be learnable. Of course, this is an impossibility.
Normalization means that before the learning even begins, we are have decided on the
distribution of learning outcomes: most are destined to be mediocre, a few will fail, and only a
select few will excel, so to be classified as the best and smartest students. This mathematical
insistence on inequality generates results that conveniently align with a radically unequal society.
Henry Goddard, a psychology professor at Ohio State University, invented this statistical
idea for the measure of comparative intelligence. On the right side of the normal distribution or
“bell” curve, approximately 1% can be geniuses, 2% gifted, 13% above average. On the left side,
about 1% are morons, 2% imbeciles, and 13% idiots. This leaves most people mediocre, a neat
symmetry where 34% are slightly below average and 34% slightly above (Goddard 1920).
On this measure of intelligence, if society is unequal, it is because inequality of mental
capacity is natural. This idea persists today for apologists of inequality (Herrnstein and Murray
1995, Phelps 2009). It’s not even a matter of offering equality of opportunity, because natural
inequality means the moderate agenda of giving all learners an equal chance is guaranteed to fail
by grading for minority excellence (Fischer et al. 1996, Gould 1981).
16
This is not only what intelligence tests must always find, but all tests that set out to
discriminate between students in order to determine inequality of learning outcomes. The
measure that is the success of the few, comes at the cost of the necessary mediocrity and failure
of the many. So, education becomes a sorting system, designed to rationalize inequality. No
matter how rigorously we profile and track learners into cohorts of similarity, we continue to
insist on inequality.
Then, in the not-so-new school, fancy new online assessment technologies arrive. Computer
adaptive testing takes to new technological heights the process of “discrimination” of different
student “understanding” that underlies item response theory. Natural language processing
technologies are applied to supply response assessments, where the key is the cut scores that
discriminate one level of outcome from another. The effect is more accurately and effectively to
discriminate those to be to be congratulated for being smart from those the system will thereby
label mediocre or stupid, and now on an industrial scale. Finely calibrated differences in test
results are turned into scientifically justified inequalities, and the less successful have only their
natural intelligence or lack of effort to blame. Education suggests that the victims of inequality
should blame nobody but themselves for their fate.
For a long time, we have known that education doesn’t have to be this way. Benjamin
Bloom, a professor of psychology at the University of Chicago, was no revolutionary. But in that
year of great social unrest, 1968, he created the idea of “mastery learning” or the idea that with
differently designed learning environments, perhaps more than ninety percent of students can
master what is being taught to them. Some may require more time or opportunity to persevere.
Others may require differentiated instruction and for this, formative assessment will point the
teacher to the particular needs of students. Non-competitive group work will also help learners
along the path to mastery. Bloom’s aim was to push the normal distribution curve to the right
(Bloom 1968).
Mastery learning has been shown to work (Kulik, Kulik and Bangert-Drowns 1990), though
it has not frequently been applied because it is harder to implement, requires teachers trained for
non-standardized teaching, and is more demanding of resources than traditional one-size-fits-all,
transmission or content-broadcast models of learning.
Bloom also addressed a phenomenon he called the “two sigma problem.” He and his
collaborators compared three conditions, conventional standardized one-to-thirty teaching,
mastery learning with differentiated instruction for thirty students, and one-on-one tutoring.
While mastery learning pushed the curve to the right, perhaps not surprisingly one on-one-
tutoring generated the best results. Student outcomes in this condition were two standard
deviations ahead of conventional teaching. Given that one-to-one tutoring does not feasibly
scale, the challenge Bloom set for educators was to design ways to achieve equivalent results to
one-on-one tutoring in conventional classes (Bloom 1984). Online learning, we want to argue,
makes this ambition achievable.
This should be the underlying institutional ethic of the new school: if we have allowed
learners into a class, it is because we have judged them teachable, and if so, it is our obligation to
teach in a way where all can achieve the learning performance objectives that we have set. As
educators, if our mission cannot realistically address the wider frame of social inequality, at the
very least it can offer opportunity for all in the design of our learning environments.
One means to achieve this will be online formative assessment and feedback systems. These
range from machine feedback to machine-mediated peer and teacher feedback. We can contend
that by these means, online learning—both for remote learning and as foundational infrastructure
17
for in-person learning, can bring the benefits of one-on-one tutoring to learning at any scale and
probably exceed them.
In designing the CGScholar platform, we have aimed to explore these potentials through the
learning analytics visualization tool (Figure 1). Most importantly for us, there is a social side to
this tool, because it captures a social immediacy and intricacy that is not possible in the
communications architecture of the traditional classroom. Peers are randomly assigned in
anonymously reviewed projects—three reviews times thirty students produces ninety pieces of
structured feedback to be judged quantitatively against a Likert scale with qualitative comments
and annotations, and ninety pieces of feedback on feedback. This would have been hard work for
a teacher to organize in pen-and-paper, but the machine manages this now. There are hundreds or
thousands of comments in classroom discussion, neatly arranged against teacher and peer inputs.
This amounts to perhaps fifty times the number of interactions that would be possible in an oral
classroom discussion, and more equitably distributed between students compared to the intrinsic
inequality of players in the discursive game of “hands up.” In online learning environments, we
can require, monitor and assess all of this.
Then there are peculiarities of machine-mediated learning which allow it to exceed the
capabilities of even a one-on-one personal tutor. Learning a musical instrument has been one
area where one-on-one tutoring has historically been the norm for teaching and learning. But
Smart Music software does something no music teacher could ever do, which is to highlight in
the score on the screen and in real time notes that are out of tune or out of tempo without
interrupting the performance, then to give an accurately calibrated, precisely evidence-based
rating at the end.
There also the potentials of big data, offering peer, teacher, self-regulated, and machine
feedback to learners in many tiny increments and on-the-fly (Cope and Kalantzis 2015). In
CGScholar, no assessment data is collected that is not in the first instance semantically legible
and actionable in a learnable moment.
Next comes artificial intelligence, where the machine “learns” from student interactions in
order to offer progressively more helpful feedback the more the system is used. We are working
to push these frontiers in our CGScholar research and development work (Cope and Kalantzis
2019, Cope, Kalantzis and Searsmith 2020). The challenge is to make artificial intelligence
explainable, where, unlike the secret and at times nefarious operations of artificial intelligence in
other domains of social life, its processes and outcomes in support of learning are fully exposed
to teachers and learners and in intelligible terms (Core et al. 2006).
If we do some or all of this, the role of assessment in the new school will have fundamentally
changed. Instead of being a fear-inducing tool, ready to judge you inadequate in relation to your
peers and pointing ominously towards unequal social destinies, embedded formative assessment
becomes the learner’s friend, always offering helpful feedback. Instead of a one-shot score,
assessment takes the form of a progress visualization based on a large number of datapoints
indicating tracking incremental progress towards mastery. Keep on going, and you can make it!
You can achieve the objectives set by the teacher and their learning designs! When mastery
replaces norming, every learner can succeed.
And instead of the forced inequalities of standardization, our measures are grounded in the
comparability of differences. Learners may start a unit of work with different knowledge,
interests and capabilities—of course. Standardized instruction turns these differences into
inequalities. When standardization succeeds, it is by imposing an architecture of epistemic and
cultural sameness where some comfortably fit the epistemic mold and others not. But in the new
18
school, student projects may be different in their contents and interpretations, while rubrics that
ensure comparability against disciplinary standards. Students may have their own voices in
community discussions, but their contributions can be equivalent. Teachers need to develop
repertoires of practice so that they can design purposeful instructional scaffolds and learning
sequences for these new learning environments.
Today, online learning infrastructures offer these possibilities, and a new frame for learning
that we call “reflexive pedagogy.” Learners are supported by “recursive feedback” that can lead
students at their own pace towards mastery learning while affirming and developing their own
distinctive voices and orientations to knowledge. Teachers can address the needs of all students
and track their progress. Today at last, online learning infrastructures make Bloom’s half-century
old aspiration, a practical possibility—and more.
So, in the new school let’s have every learner succeed. Without irony we plead, “let’s leave
no child behind!”
In Brief Conclusion...
Our five theses propose five ways in which the new school could be dramatically different. Each
leverages the affordances of online educational technologies, and their positive potentials to
support educational reform. We have demonstrated through the design and delivery of learning
via the experimental CGScholar platform, how it is possible to make full use of the affordances
of the digital in the service of learning outcomes that are better aligned with the complex and at
times vexing world in which we live.
Online learning could bring about the biggest change in education since the invention in early
modern times of the walled classroom, the textbook, and the teacher lecture and classroom
discussion. This could transform both in-person and remote learning. It could make education
dramatically better and its outcomes more equitable. If larger social inequalities persist, at least
education will have ceased to be their handmaiden.
Because now we can, we must.
References
Binet, Alfred. 1905 [1916]. "New Methods for the Diagnosis of the Intellectual Level of
Subnormals." in The Development of Intelligence in Children, edited by A. Binet and T.
Simon. New York.
Bishop, Jacob and Matthew Verleger. 2013. "The Flipped Classroom: A Survey of the
Research." Paper presented at the American Society for Engineering Education, 23-26
June, Atlanta GA.
Bloom, Benjamin S. 1968. "Learning for Mastery." Evaluation Comment 1(2):1-2.
Bloom, Benjamin S. 1984. "The 2 Sigma Problem: The Search for Methods of Group Instruction
as Effective as One-to-One Tutoring." Educational Researcher 13(6):4-16.
Bower, Joseph L and Clayton M. Christensen. 1995. "Disruptive Technologies: Catching the
Wave." Harvard Business Review (January-February):43-53.
Cazden, Courtney B. 2001. Classroom Discourse: The Language of Teaching and Learning.
Portsmouth, NH: Heinemann.
19
Chang, Hua-Hua. 2004. "Understanding Computerized Adaptive Testing: From Robbins-Monro
to Lord and Beyond." Pp. 117-33 in The Sage Handbook of Quantitative Methodology for
the Social Sciences, edited by D. Kaplan: Sage Publications.
Chang, Hua-Hua. 2015. "Psychometrics Behind Computerized Adaptive Testing."
Psychometrika 80(1):1-20. doi: 10.1007/s11336-014-9401-5.
Christensen, Clayton M., Michael B. Horn and Curtis W. Johnson. 2008. Disrupting Class: How
Disruptive Innovation Will Change the Way the World Learns. New York: McGraw Hill.
Cope, Bill, Mary Kalantzis, Sarah J. McCarthey, Colleen Vojak and Sonia Kline. 2011.
"Technology-Mediated Writing Assessments: Paradigms and Principles." Computers and
Composition 28(2):79-96.
Cope, Bill and Mary Kalantzis. 2015. "Sources of Evidence-of-Learning: Learning and
Assessment in the Era of Big Data." Open Review of Educational Research 2(1):194–
217. doi: http://dx.doi.org/10.1080/23265507.2015.1074869.
Cope, Bill and Mary Kalantzis. 2016. "Big Data Comes to School: Implications for Learning,
Assessment and Research." AERA Open 2(2):1-19. doi:
https://doi.org/10.1177/2332858416641907.
Cope, Bill and Mary Kalantzis. 2017. "Conceptualizing E-Learning." Pp. 1-45 in E-Learning
Ecologies: Principles for New Learning and Assessment, edited by B. Cope and M.
Kalantzis. New York NY: Routledge.
Cope, Bill and Mary Kalantzis. 2019. "Education 2.0: Artificial Intelligence and the End of the
Test." Beijing International Review of Education 1:528-43. doi:
https://doi.org/10.1163/25902539-00102009.
Cope, Bill and Mary Kalantzis. 2020. Making Sense: Reference, Agency and Structure in a
Grammar of Multimodal Meaning. Cambridge UK: Cambridge University Press.
Cope, Bill, Mary Kalantzis and Duane Searsmith. 2020. "Artificial Intelligence for Education:
Knowledge and Its Assessment in Ai-Enabled Learning Ecologies." Educational
Philosophy and Theory 52(16):1-17. doi: http://doi.org/10.1080/00131857.2020.1728732.
Core, Mark G., H. Chad Lane, Michael van Lent, Dave Gomboc, Steve Solomon and Milton
Rosenberg. 2006. "Building Explainable Artificial Intelligence Systems."
Dear, Brian. 2017. The Friendly Orange Glow: The Untold Story of the Plato System and the
Dawn of Cyberculture. New York NY: Pantheon Books.
Fischer, Claude S., Michael Hout, Martin Sanchez Jankowski, Samuel R. Lucas, Ann Swindler
and Kim Voss. 1996. Inequality by Design: Cracking the Bell Curve Myth. Princeton NJ:
Princeton University Press.
Foucault, Michel. 1975 [1977]. Discipline and Punish: The Birth of the Prison. Translated by A.
Sheridan. New York NY: Vintage Books.
Freire, Paulo. 1972. Pedagogy of the Oppressed. Harmonsdworth UK: Penguin.
Gee, James Paul. 1992 [2013]. The Social Mind: Language, Ideology, and Social Practice.
Champaign IL: Common Ground.
Gibson, James J. 1979 [2015]. The Ecological Approach to Visual Perception. New York NY:
Psychology Press.
Goddard, Henry H. 1920. Human Efficiency and Levels of Intelligence. Princeton NJ: Princeton
University Press.
Gould, Stephen Jay. 1981. The Mismeasure of Man. New York: W.W. Norton.
Haladyna, Thomas M. 2004. Developing and Validating Multiple-Choice Test Items. Mahwah
NJ: Lawrence Erlbaum Associates.
20
Haniya, Samaa, Matthew Montebello, Bill Cope and Richard Tapping. 2018. "Promoting Critical
Clinical Thinking through E-Learning." in Proceedings of the 10th International
Conference on Education and New Learning Technologies (EduLearn18). Palma de
Mallorca ES.
Herrnstein, Richard and Charles Murray. 1995. The Bell Curve: Intelligence and Class Structure
in American Life. New York: Free Press.
Hmelo-Silver, Cindy E., Ravit Golan Duncan and Clark A. Chinn. 2007. "Scaffolding and
Achievement in Problem-Based and Inquiry Learning: A Response to Kirschner, Sweller,
and Clark." Eductional Psychologist 42(2):99-107.
Kalantzis, Mary and Bill Cope. 2015. "Learning and New Media." Pp. 373-87 in The Sage
Handbook of Learning, edited by D. Scott and E. Hargreaves. Thousand Oaks CA: Sage.
Kalantzis, Mary and Bill Cope. 2020a. Adding Sense: Context and Interest in a Grammar of
Multimodal Meaning. Cambridge UK: Cambridge University Press.
Kalantzis, Mary and Bill Cope. 2020b. "After the Covid-19 Crisis: Why Higher Education May
(and Perhaps Should) Never Be the Same." Educational Philosophy and Theory:24-27.
doi: https://doi.org/10.1080/00131857.2020.1777655.
Kalantzis-Cope, Phillip. 2016. "Whose Data? Problematizing the ‘Gift' of Social Labour."
Global Media and Communication 12(3):295-309.
Kirschner, Paul A., John Sweller and Richard E. Clark. 2006. "Why Minimal Guidance During
Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery,
Problem-Based, Experiential, and Inquiry-Based Teaching." Educational Psychologist
41(2):75-86.
Kulik, Chen-Lin C., James A. Kulik and Robert L. Bangert-Drowns. 1990. "Effectiveness of
Mastery Learning Programs: A Meta-Analysis." Review of Educational Research
60(2):265-99.
Landauer, Thomas K., Danielle S. McNamara, Simon Dennis and Walter Kintsch, eds. 2007.
Handbook of Latent Semantic Analysis. New York NY: Routledge.
McMichael, Maureen A., Matthew C. Allender, Duncan Ferguson, Bill Cope, Mary Kalantzis,
Duane Searsmith and Samaa Haniya. 2020. "Use of a Novel Learning Management
System for Teaching Critical Thinking to First Year Veterinary Students." Journal of
Veterinary Medical Education.
Montebello, Matthew, Bill Cope, Mary Kalantzis, Duane Searsmith, Tabassum Amina, Anastasia
Olga Tzirides, Naichen Zhao, Min Chen and Samaa Haniya. 2018. "Critical Thinking
through a Reflexive Platform." in Proceedings of the 17th IEEE International Conference
on Information Technology Based Higher Education and Training (ITHET 2018). Olhao
PT.
Muck, Kátia. 2015. "The Role of Recursive Feedback: A Case Study of E-Learning in
Emergency Operations." The International Journal of Adult, Community, and
Professional Learning 23(1):1-25.
Ong, Walter J. 1958 [1983]. Ramus, Method and the Decay of Dialogue. Cambridge MA:
Harvard University Press.
Pellegrino, James W., Naomi Chudowsky and Robert Glaser, eds. 2001. Knowing What Students
Know: The Science and Design of Educational Assessment. Washington DC: National
Academies Press.
Peters, Michael A., Simon Marginson and Peter Murphy. 2008. Creativity and the Global
Knowledge Economy. New York: Peter Lang.
21
Phelps, Richard P. 2009. "Educational Achievement Testing: Critiques and Rebuttals." Pp. 89-
146 in Correcting Fallacies About Educational and Pychological Testing, edited by R. P.
Phelps. Washington DC: American Pyschological Association.
Schumpeter, Joseph A. 1950 [1976]. Capitalism, Socialism and Democracy. New York NY:
Harper and Row.
Shermis, Mark D. 2014. "State-of-the-Art Automated Essay Scoring: Competition, Results, and
Future Directions from a United States Demonstration." Assessing Writing 20:53-76. doi:
10.1016/j.asw.2013.04.001.
St Benedict. c.530 [1949]. The Holy Rule of St. Benedict. Translated by O. Rev. Boniface
Verheyen.
Surowiecki, James. 2004. The Wisdom of Crowds: Why the Many Are Smarter Than the Few and
How Collective Wisdom Shapes Business, Economies, Societies and Nations. New York:
Doubleday.
van Haren, Rita and Jessica Harroun. 2019. "Cgscholar’s Analytics: Progress to Mastery
Learning." Ubiquitous Learning 12(2):1-24.
Vojak, Colleen, Sonia Kline, Bill Cope, Sarah J. McCarthey and Mary Kalantzis. 2011. "New
Spaces and Old Places: An Analysis of Writing Assessment Software." Computers and
Composition 28(2):97-111.
