ArticlePDF Available

Abstract

This paper examines the ongoing challenge of defining what learning means from the perspective of the cognitive and learning sciences, especially as it unfolds in online environments. To better define learning as well as offer guiding principles, this paper uses Khan Academy as an example of what some high profile individuals, such as Bill Gates, are claiming to be the future of education. I offer five guiding observations that provide a structure for understanding the learning process and apply them to Khan Academy as a means of revealing what I call the illusion of understanding, and I replace that view with a more authentic understanding of the learning process and the means to achieve that understanding.
Khan Academy: The Illusion of Understanding
Journal of Asynchronous Learning Networks, Volume 17: Issue 4 67
KHAN ACADEMY: THE ILLUSION OF
UNDERSTANDING
Marc Schwartz
University of Texas at Arlington
ABSTRACT
This paper examines the ongoing challenge of defining what learning means from the perspective of the
cognitive and learning sciences, especially as it unfolds in online environments. To better define learning
as well as offer guiding principles, this paper uses Khan Academy as an example of what some high-
profile individuals, such as Bill Gates, are claiming to be the future of education. I offer five guiding
observations that provide a structure for understanding the learning process and apply them to Khan
Academy as a means of revealing what I call the illusion of understanding, and I replace that view with a
more authentic understanding of the learning process and the means to achieve that understanding.
KEYWORDS
Khan Academy, cognitive development, hierarchical development, understanding, practice, feedback,
context sensitive, online education, MOOC, e-learning, pedagogy, instructional design
I. INTRODUCTION: THE ILLUSION OF UNDERSTANDING
A serious challenge for educators and students is avoiding what I call the illusion of understanding. Most
often, the illusion arises when educators and students fall into the following relationship: “I’ll pretend to
teach as long as you pretend to understand.” The interaction is neither malicious nor necessarily
conscious. In fact, the relationship emerges out of a persistent and pervasive misunderstanding of the
learning process, one repeated in numerous contexts throughout the history of education, apparent in the
introductions of textbooks over the last 150 years (which justify the newest and latest textbook), and now
repeated at Khan Academy. Recognizing the relationship in educational contexts is challenging because
educators are embedded in the process and, if they are reading this article, are most likely a product of the
process. But the effort is necessary if we are to understand and address our misconceptions about teaching
and learning and, more importantly, to avoid succumbing to the illusion that real teaching and learning is
occurring.
A. What does the illusion of understanding look like?
An illustration from my own work of the illusion of understanding occurred years ago when my
colleagues and I were training science educators in a graduate course at the Harvard Graduate School of
Education [1, 2]. A significant part of the course involved students addressing a number of problems
involving basic principles of science. The problem students found the most intriguing is the following:
Imagine you come upon a canoe in a swimming pool and you remove the large anvil you find in
the canoe and submerge the anvil in the pool. If you note the level of water before commencing
the operation and again after the anvil is completely submerged, does the water level of the pool
change?
This question, describing an unlikely situation, might have created some dissonance, but our students
were well embedded in the process of answering similar questions and accepted the challenge. You might
want to consider this problem as well before reading further. This scenario was created to challenge
students’ understanding of Archimedes’ principle, a concept found in many school curricula around the
Khan Academy: The Illusion of Understanding
68 Journal of Asynchronous Learning Networks, Volume 17: Issue 4
world. At Khan Academy, this principle is directly addressed in two of twelve sessions on “Fluids” (parts
five and six): http://www.khanacademy.org/science/physics/fluids/v/fluids--part-5.
What you should note about the canoe scenario is that it can be solved entirely without mathematics. The
problem is purely conceptual and has just three possible solutions: the water level will go up, remain
unchanged, or go down. For five years my colleagues and I posed this and similar challenges to students,
and, surprisingly, for five years the distribution of their answers appeared to be no better than as if by
chance [2].
B. What does the illusion of understanding feel like?
Students were also surprised, although unpleasantly, at their inability to arrive at a definitive answer to the
canoe problem. They thought they understood Archimedes’ principle until they faced this or similar
conceptual problems to which they had to apply the principle. The level of distress these problems created
for students surprised us. Some expressed a feeling of panic at the thought that if they didn’t understand
this principle, perhaps they didn’t understand anything they had learned in school. This was perhaps the
most surprising insight for us: we are all subject to the illusion of understanding until we are somehow
forced to face it. But the most insidious aspect of the illusion of understanding is that it masks what I will
call authentic understanding.
C. What is authentic understanding?
Over time, my colleagues and I learned to appreciate a small number of important observations about the
nature of authentic understanding that distinguishes it from the illusion of understanding. The
observations are straightforward, but the nuances of each make them easy to dismiss because they are
hard to integrate into what many educators, including Khan, believe understanding to mean. To be fair,
those educators’ views are not often explicitly stated but are revealed through their choice of interventions
and assessments. Meeting this challenge is not trivial.
Understanding is a complex phenomenon. While difficult to define or measure precisely [3], it is
nonetheless possible to identify key characteristics and processes that support its development. The
insights and recommendations that follow have emerged not only from my own observations but also
from the work of numerous cognitive scientists, neuroscientists, and educators over the past century [4, 5,
6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]. Here, I describe five critical insights concerning the nature and
development of authentic understanding. The insights help confront the illusion of understanding, which
continues to survive and thrive in a variety of educational contexts. Khan Academy is just the latest
reminder of our collective struggle to profit from the learning sciences; however, more challenging in the
context of the virtual classroom is that the teacher-student relationship cannot profit from even the most
basic form of communicationthe student’s confused look. As everyone stares at the magic Smartboard,
the illusion that Khan is teaching and that the observer is learning is that much easier to perform.
To be fair, the problem inherent at Khan Academy is one that appears in many educational contexts, and
the academy’s potential strengths are noteworthy, but not because they necessarily support authentic
understanding. As Khan highlights in interviews, the central issues he addresses are student attention
span, the availability of instruction, and student control of the pace of instruction. While these features
allow students to more easily consume videos intended to be instructional, they are nonetheless peripheral
to the goal of developing students’ authentic understanding.
II. FIVE KEY OBSERVATIONS ABOUT AUTHENTIC UNDERSTANDING
The following five observations offer a working framework of authentic understanding:
Authentic understanding depends on hierarchically organized knowledge.
Authentic understanding is grounded in direct experience.
Authentic understanding is stabilized by practice (generally at every level within the hierarchy).
Authentic understanding requires formative feedback.
Khan Academy: The Illusion of Understanding
Journal of Asynchronous Learning Networks, Volume 17: Issue 4 69
Authentic understanding is context-sensitive.
The generality of these criteria is supported by the fact that our students were extremely bright, motivated,
and often just as surprised as we were by their inability to leverage the formulas they had learned by heart
and used for years in science classrooms to solve problems. Dismissing our students’ struggle with the
conceptual problems as being peculiar to them was not easy, as they were considered to be among the
best in the nation. Thus, the first important realization was that intelligence had little to do with the
challenges the students were facing. In any situation in which these five criteria are ignored,
understanding becomes fragile and unstable, and our ability to recall or apply what we remember is
compromised.
A. Observation one: authentic understanding depends on hierarchically
organized knowledge.
Understanding, the nature of which has been unpacked by over a century of research, is hierarchical in
structure [14, 15, 16, 17, 18, 19]. Each new achievement within the hierarchy becomes the foundation for
the next more complex, more integrated coordination of earlier achievements. The work of Piaget and
scores of neo-Piagetian scholars have documented this process, in which sensorimotor experiences (i.e.,
what we learn through our senses) become the foundation for representations [20] of the content of
experiences (e.g., words, pictures, graphs, tables, etc.), and later those representations are coordinated into
abstractions that transcend the concrete nature of our experiences and the representations and
relationships that emerged earlier from our sensory experiences. Thus, abstractions, such as democracy,
justice, or Archimedes principle, emerge as a new way of understanding the rich coordination of
representations accumulated through various contexts and practiced on multiple occasions. However, this
achievement, which is the outcome of a powerful synthesis of concepts, is often belied by simple-looking
words and phrases like “democracy” or “Archimedes’ principle,” which at face value do not appear to be
remarkably different from other words students use, such as “gavel” and “anvil.”
All of the three tiers (sensorimotor, representation, and abstraction) are qualitatively very different ways
of understanding, and each depends on the earlier tier to achieve the more complex understanding of later
tiers. Progress for students is, however, more nuanced than what is captured by these three major
developmental steps. Within each tier researchers have noted finer degrees of achievement, described as
levels [15, 16, 19, 21]. For example, the first level of the representational tier signifies the ability to name
objects, such as the object in the canoe (i.e., the anvil). The next level (in the representational tier)
involves coordinating exemplars from the first level into a new understanding, such as knowing that
putting the canoe (or any object) into the pool will raise the water level. There are still two more levels of
complexity in each tier that require additional exemplars of earlier levels as well as further coordination
before moving to the next tier. The transition to the abstract tier requires the consolidation of
representational skills from all four levels into a qualitatively new way of thinking about the world, which
is summarized in words such asjustice,” “democracy,” or Archimedes’ principle.Details regarding the
nature of levels and tiers, their structure and relationship to each other, the criteria for measuring the
complexity of each level, and movement from one level to the next are well documented and illustrated
by numerous researchers [16, 19].
For the purposes of this article, there are two important notions about any skill demonstrated at any level
in any tier. First, the skill is observable, which is of particular importance to educators and students.
Second, movement up the hierarchy involves the coordination of less complex skills into skills of greater
complexity, like a juggler adding more objects into a juggling routine. As skills become more complex,
the coordination becomes increasingly more complex. Time and practice will allow for some
consolidation of earlier skills into more stable skills of greater complexity. However, as every juggler
knows, practice is necessary to maintain the coordination of plates, balls and knives; otherwise ideas, like
objects, crash to the ground.
Khan Academy: The Illusion of Understanding
70 Journal of Asynchronous Learning Networks, Volume 17: Issue 4
1. Do Khan Academy lessons support the development of hierarchical knowledge?
Looking at Khan’s series of videos related to Archimedes’ principle as a representative example, the
layout of the lessons is not compatible with the observation that humans build knowledge hierarchically.
For the most part, he introduces ideas at the most complex levels of the representational tier or the next
higher tier (i.e., abstraction). While it is true that the video is sensitive to sequencing concepts, the work is
done from the perspective of an expert. This is an important distinction.
The expert’s perspective offers a global view of relevant concepts, the heuristics to understand
relationships between concepts, and the algorithms that reveal more precise understandings of those
concepts. From this vantage point, the expert sees the educational challenge as unpacking complex skills
into less complex skills. Thus, it would make sense from an expert’s point of view to introduce density
before discussing Archimedesprinciple, which depends on density. This approach generates a sequence
of abstractions relevant to the expert, but not a sequence that the learner can necessarily construct or see
as relevant.
Learners typically confront a different challengeone of building more complex understandings, as
encountered in higher levels and tiers. Every graph, formula, drawing, and arrow on Khan’s Smartboard
is a relevant representation (otherwise it wouldn’t be in the video); however, he unconsciously ignores the
sequences of experiences that allow students to coordinate or juggle these (and other necessary)
representations into concepts like density and, eventually, Archimedesprinciple.
One important exception to this general observation is the “Knowledge Map” in the area of math. While
the map is very important in understanding the development of a discipline, the map is not a picture of the
development of understanding in individual learners. It is important to note that while the map is a
hierarchical framework, the hierarchy emerges from the perspectives of experts, not learners. Similar
work can be seen in the Atlas of Science Literacy [22]. Here the authors claim something more explicit:
[The] Atlas of Science Literacy is a two-volume collection of conceptual strand maps … that show how
students’ understanding of the ideas and skills that lead to literacy in science, mathematics, and
technology might develop from kindergarten through 12th grade” [22, p.1].
This claim is seductive. The actual path to understanding might be broadly marked out by these maps (as
indicated by the authors), but this work was accomplished by experts retrospectively reflecting on how
they learned science, so we must emphasize that they do not necessarily characterize how students
actually build that knowledge. In similar fashion, the math Knowledge Map connects a number of
relevant topics, beginning with addition and subtraction and ending with calculus, and within each topic
there are exercises that test student understanding. I will underscore here and beyond that the knowledge
map as well as the Atlas of Science Literacy offer an important perspective on the development of a
discipline, but whether the maps lead students to authentic knowledge is not obvious; thus, we must be
circumspect with how we use these maps to create interventions and assessments in the belief that
students are re-creating the same understanding as experts.
B. Observation two: experience is the foundation of authentic understanding.
Educators recognize the importance of creating experiences, but it is important to note that not all
experiences are equal. Central to the first observation is that understanding develops first through the use
and integration of our senses. Formulas for concepts such as density or pressure (d = m/v and p = mgh)
belong to the representational tier and are far removed from the senses that students need to immediately
coordinate and employ when confronting a new and complex juggling routine. While many elementary or
middle school curricula do incorporate activities such as submerging small blocks of iron or wood to see
how much water is displaced, such experiences are too remote and inaccessible to high school students
trying to understand how the laws and algorithms their teachers are using encapsulate their earlier
experiences. The power in laws and principles is in the fact that they summarize numerous experiences,
but those insights cannot be transferred directly from one person to anotherthe abstraction, divorced
from the numerous experiences that gave rise to it in the first place, is lost upon the student who doesn’t
Khan Academy: The Illusion of Understanding
Journal of Asynchronous Learning Networks, Volume 17: Issue 4 71
have the foundation of experiences that give rise to personally constructed representations, which in turn
will support the abstractions valued by educators.
The task of teaching begins in carefully choosing experiences that challenge the student’s intuition about
the world, and allow for numerous interactions between the student, their experiences, and the challenges
they are facing. This dynamic process, which requires time and multiple opportunities to engage with the
content, is necessary to develop more complex representations [21, 23]. Eventually, this kind of work can
set the stage for watching and appreciating the complexity in Khan’s juggling of representations such as
pressure, volume, gravity, etc., embedded in parts five and six of Fluids.”
1. Do Khan Academy lessons ground complex ideas in sensorimotor experience?
Like many accomplished science educators, Khan demonstrates his ability to coordinate numerous
representations, which is analogous to observing a master juggler. Watching Khan carry out a complex
performance of understanding in 15 minutes or less is also much like watching a professional musician
play a piece of similar duration. In this case, audience members do not generally believe themselves
capable of reproducing the same performance afterward. However, what is curious in educational
contexts is that after listening to a lecture, teachers and students frequently believe that the student should
be capable of performing at the same level as the teacher and with the same level of authentic
understanding. But that isn’t the case, even after students spend time on questions, algorithms, charts, and
graphs found on worksheets or at the end of the chapter. Just practicing within the representational tier
doesn’t appear to support authentic understanding.
One promising aspect of Khan Academy is the effort to develop authentic understanding through intuitive
practice, again in the area of math. The technology embedded in the problems allows students to
experiment with the impact of different variables that give rise to important concepts, such as standard
deviation or average; however, changing the distribution of data on a two dimensional graph, for example,
still requires focusing on juggling representations and not yet how the representations connect to tangible
experiences. Authentic understanding requires a wide platform of experiences, which in turn provides a
foundation for the representations that are the basis of student work in any school environment, virtual or
real. Without the benefit of this foundation, representations practiced in school are reduced to borrowed
ideas that are limited in scope and decay rapidly [1].
C. Observation three: authentic understanding is stabilized by scaffolded
practice.
Students who can recall the formulas they practiced using in school often struggle to employ those
formulas outside the rigid contexts in which they were learned [24, 25, 26, 27, 28]. Such was the case
with my students. Their knowledge of the formulas left them with an illusion of understanding, which
unraveled when they confronted conceptual problems based upon the related algorithms. Formulas are
important tools that mathematically represent relationships observed in nature. They often show up in
demonstrations as a way of expressing complicated relationships, but they do not necessarily reveal the
conceptual basis for the relationships between the variables or the numerous experiences underlying the
formula, no matter where they appearclassroom whiteboards or Khan’s Smartboard.
Khan’s presentations look no different from what many practiced physics teachers create for their
students even though the duration may be much shorter in Khan’s case (see reference section for other,
longer, examples). Demonstrations that illustrate the teacher’s understanding also reinforce the illusion of
understanding [29]. The viewer watches the instructor demonstrate the outcome of years of their own
practice, creating for students the afterglow of an experience in which meaning was created, but not by
the student.
Students who report having trouble following demonstrations also report that they quickly lose interest;
thus, it is not surprising that the short attention span that Khan highlights as a universal educational
problem becomes a defining feature of his presentations. Educators must address the disconnect students
experience between their current understanding and the demonstration they are observing, which includes
Khan Academy: The Illusion of Understanding
72 Journal of Asynchronous Learning Networks, Volume 17: Issue 4
the less obvious knowledge embedded in the instructor’s choice of objects and tools, as well as the order
in which both are used and manipulated during the demonstration.
One approach we used to address the disconnect mentioned above was through the kinds of problems we
offered students. We found that without instruction or encouragement students would take control of
problems and practice with them at home, using whatever tools and supplies were available to them. They
surprised us by returning to class explaining what they had learned. While we were impressed by
individual performances of understanding, we unfortunately remained vulnerable to the illusion of
understanding. Initially, we succumbed to the easy belief that the students listening to these stories of
success had also achieved the same level of juggling demonstrated by their peers, but they quickly
punctured that illusion. A few promptly announced that they didn’t understand what their peers had said,
and did not understand until they attempted the demonstration. Then, in their own words they tried to
explain what they understood. In effect, authentic understanding emerged as a process of students
juggling the ideas and tools on their own. The ongoing challenge educators must face is identifying
demonstrations that challenge their students intuition and inviting them to assume control and practice
juggling the relevant elements of the demonstration.
Our students learned that they had to practice re-presenting their explanations so that their complex ideas
could develop stability over time. In one dramatic moment, a student correctly described what would
happen to the water level in the pool, but then paused, looked at the class, and publically admitted he
didn’t understand what he had just said. For several minutes he had created and sustained an
understanding that allowed him to see the causal connection between his experiences and representations
and what would happen to the level of water in the pool. But he also realized that the understanding was
temporal. Understanding does not sit in our minds like books on shelves [30]. We have to re-create
understanding so that it can achieve some level of permanence. But as jugglers know, if they don’t
practice there is little guarantee they can pick up the blocks (or concepts) and begin juggling again.
It is useful to practice with the same problem a number of times, like a musician practicing the same piece
until she is comfortable with every aspect of the score. An important dimension of practicing is that the
learner can easily observe changes in performance over time. The notes become easier to play; there are
fewer mistakes and greater fluidity, etc. As with the canoe problem, practice allows the student to
recognize the importance of volume or how the impact of volume on the water level is different when
evaluating the role of the canoe and the anvil (before it is removed from the canoe). Similarly, the mass of
the anvil plays a different role (on the outcome of the water level) depending on whether it is in the canoe
or in the water. Coordinating these ideas into a more complex representation is a necessary foundation for
achieving new abstractions (such as Archimedes’ principle).
1. Does Khan Academy scaffold practice?
There are currently a limited number of subjects taught at the Academy that offer opportunities to
practice. The sole area in which practice is offered, which is particularly well developed, is mathematics;
however, as pointed out earlier, practice is limited to the representational tier. While scaffolding is
provided within this tier to help students unpack, for example, the meaning of median or mode, this work
is executed through the use of other representations. There are no explicit connections to relevant
experiences that students can use to construct for themselves authentic representations at any level. The
risk, as noted earlier, is that students are left with an illusion of understanding that is fragile and highly
context specific (a problem explored in observation five) and disconnected from the real world.
Outside the area of math, as in the case of Fluidsand Archimedes principle (parts 5 and 6), practice is
not yet an option. As the Academy creates practice conditions for students, there are two important
considerations. First, practice must focus on the careful choice of similar problems (like musical
compositions that feature the same technical challenges, such as rhythm) that reinforce the skills used and
afford greater comfort with concepts, such as volume and mass (or notes, rhythm, and intonation, as the
case may be). Problems similar to each other in scope and complexity are instrumental in allowing
students and teachers to consider the important role context plays in teaching and learning. Second,
Khan Academy: The Illusion of Understanding
Journal of Asynchronous Learning Networks, Volume 17: Issue 4 73
practice must lead to meaningful feedback. Feedback not only helps adjust performance as a skill is
stabilized, but it should also provide the platform for reaching the next more complex level of
understanding.
D. Observation four: meaningful feedback is relevant and timely.
All life depends on relevant feedback. This is true across a range of complexity beginning with single
celled organisms looking for nutrients, to multicellular organs achieving homeostasis, to multicellular
organisms attempting to survive in their niche. Relevant feedback for students and educators is generated
through actions they believe will allow them to achieve a particular goal. Students need to recognize the
target they are attempting to reach and be able to identify likely responses they believe will lead them to
success. More importantly, they need the freedom to try out those actions. Only in the context of
attempting to achieve a goal do our actions make sense, and only then can we meaningfully interpret the
outcome of our actions [31, 32, 33]. While this might seem obvious, executing this in educational
contexts is challenging.
This task of identifying goals that students understand and that also match their current ability to juggle is
not easy but necessary. A number of researchers have pointed out that lessons need to be strongly guided
by goals that focus a student’s attention [7, 10, 21, 31, 33]. An effective goal provides students with the
opportunity to identify promising strategies to reach the goal, which in turn creates a meaningful
foundation for creating and coordinating more complex representations and abstractions.
The problem with school lessons whose “goals,” for example, require students to submerge blocks of iron
or copper or wood in a column of water is that for many students there is no obvious or explicit reason for
the activity. Students should be asking, what is the problem for which this lesson (i.e., submerging
blocks) is the solution? But students don’t ask this question because they are more accustomed to asking
how they should participate. Thus, the likelihood of recognizing the need to ask why Khan is making the
choices he makes during his video presentation is even more remote. It may not be clear to a student what
goals are guiding Khan’s actions. At times Khan’s indecision is obvious in that he will start a drawing and
then change his mind and start something different. Why? What problem was he facing that required a
change in teaching strategy? This style of teaching is not unique to Khan, as the teaching environment is
dynamic. Teachers often change their minds and approaches as they become involved in the teaching
moment. The energy and creativity that often characterize these moments easily support the illusion of
understanding, as the learner experiences the theater of the teacher’s goals successfully directing the
teacher’s actions. Unfortunately, the learner is often too passive during the whole experience. More
dramatic is the online experience, which creates a new challenge in that the “teaching moment” is not
really shared with the audience or influenced by it. As the dynamic nature of teaching becomes codified,
the video risks becoming a permanent reminder of the distance between the instructor and student, further
reinforcing the need to passively observe. As passive observers, students eventually let go of asking why
any demonstration (of the teacher’s expertise) is unfolding as it is, because students know they will not be
answered.
To confront this challenge, instructors need to consider how to set up problems that have clear actionable
goals, and that action on the part of students generates feedback that is meaningful to them, thus requiring
less feedback from the instructor. In regard to the canoe problem, one should note that nature, rather than
the instructor, provides the feedback. Students don’t need to ask the instructor what the answer is,
although they often will. In these cases the instructor can direct students to find the answer on their own
in the lab or at home.
Finding problems like the canoe problem also provides students with the opportunity to consider the
importance of their own work in creating new knowledge. These kinds of problems can provide high
intrinsic motivation similar to the state of “flow” in which the sense of time compresses, focus and
concentration increases, and distractions disappear [34]. The experience is also similar to what individuals
experience when playing certain video games [35].
In contrast, extrinsic motivation is what educators use when students don’t respond in ways intended by
Khan Academy: The Illusion of Understanding
74 Journal of Asynchronous Learning Networks, Volume 17: Issue 4
the lesson. This situation can result from a number of problems: students don’t understand the goal, don’t
recognize the appropriate action, don’t have the ability to execute the required action, or can’t interpret
feedback generated by their action (or given by the instructor). Extrinsic feedback often depends on
rewards (such as money, grades, extra time at recess, less homework, etc.) that are unrelated to the goal of
making students want or need to create more complex representations or abstractions. Extrinsic feedback
is less powerful at generating more complex understandings, as observed by drivers who get information
about how to drive from the person in the back seat. The risk of a system based upon extrinsic rewards is
that it easily contributes to the illusion of understanding.
1. What does feedback look like at Khan Academy?
Like many school environments, Khan Academy depends on extrinsic feedback, in which students
receive “badges” and “energy points” after completing lessons. If they wish, students can share with
selected audiences their progress through the lessons. Second, students can also ask questions in a blog
format, to which other members of the Academy can respond. It’s not clear that answers from the
community are any more helpful than Khan’s videos or my students’ explanations of their understanding
to their peers.
Both forms of feedback fall short of what counts as meaningful and timely feedback. Badges do not
contribute to developing an understanding of fluid dynamics. I, for example, earned three badges while
watching “Fluids; however, those badges are not predictors of how successful I would be with the canoe
problem or variations of the canoe problem or any similar problem in a new context (e.g., applying
Archimedesprinciple with gasses instead of fluids).
The third form of feedback, developed as part of the math Knowledge Map at Khan Academy, is
embedded in the available practice problems. The software evaluates the student’s answer, and if the
answer is incorrect the software offers one or more layers of suggestions that guide students to the right
answer. Alternatively, the student can choose to unpack the layers without committing to an answer, and
teachers can follow student progress.
This form of feedback resembles coaching, in which students are provided with specific support in
response to a problem in executing a skill. While coaching can help students achieve greater proficiency
with skills or practice more effectively, the more basic challenge of developing authentic knowledge
remains. If the short-term goal is for students to get better at solving quadratic equations, then they may
never discover the problem for which the quadratic equation was the solution in the first place.
Developing a richer palette of sensorimotor experiences that are tightly linked to representations still
needs to occur before teachers become overly focused on training students to solve quadratic equations.
Otherwise the students’ work becomes no more important or useful than a parlor trick restricted to very
specific contexts. This challenge is already one that teachers find difficult to meet in classrooms and,
expectedly, much more challenging in an online environment.
One possible solution is letting the video set up problems that students can execute on their own and in
turn allow them to judge the impact of their actions. In our case, the problems we posed students provided
this opportunity. In turn, course instructors began to play a more marginal role in providing answers
because nature could offer immediate feedback. Given this kind of experience, our students were better
able to adapt to situations where nature could not conveniently respond to their questions. In such cases,
students were better prepared to consider the answers of experts, and to compare them to their own. In
situations where nature cannot provide an answer, current technology can collect student responses and
allow them to compare and contrast answers through graphs or tables that are continually updated. A
distribution of student answers has the potential to encourage students to take a deeper look at the original
challenge and the answers.
E. Observation five: authentic understanding is context-sensitive.
While practice provides opportunities to challenge the stability of a complex idea, practicing in a variety
of contexts challenges the robustness of one’s grasp of the principle underlying any particular problem.
Khan Academy: The Illusion of Understanding
Journal of Asynchronous Learning Networks, Volume 17: Issue 4 75
Thus, initially the juggler may be comfortable juggling balls in front of friends, but the same act on stage
in front of strangers can create a different experience and result. In this case, changing contexts where
competence is demonstrated will challenge how easily ideasor ballsare coordinated. While there is
value in establishing this kind of stability with a problem, the greater challenge is establishing stability
with the principle behind related but novel problems [24, 28, 36, 37].
The canoe problem offered students the chance to practice and demonstrate their understanding through
different approaches. Inviting students to demonstrate their understanding through writing, oral
presentations and, in turn, using their understanding with their students created stability with this one
problem, but not necessarily with Archimedes’ principle. Developing comfort with Archimedesprinciple
requires changing the problem in slight ways to provide students the opportunity to challenge their
understanding. For example, changing the context slightly by asking what would happen to the level of
water if the anvil were replaced with a large piece of foam or a piece of balsa wood or a smaller version
of the anvil allows students to focus on particular elements of a problem growing in familiarity. Working
with the original problem provides a framework that remains constant over time so students can control
elements within the problem to judge the importance and role the variables (such as volume, mass,
floating, sinking, etc.) play.
After a trial period during which students explore the impact of the variables in the problem, the
curriculum can invite students to consider new and unfamiliar contexts where the principle is still
operating. Consider for example a planet like Mars where the atmosphere is predominantly carbon
dioxide. If an astronaut fills her birthday balloon with oxygen and releases it, will the balloon fall, float in
place, or climb? Although this problem requires an understanding of chemistry, the example illustrates the
kind of opportunity that educators need to create in which students can consider how Archimedes
principle is operating. Problems like this already exist in numerous textbooks [38, 39]; however, to make
effective use of such problems, careful and purposeful attention to earlier experiences and observations is
still necessary.
The experience students had with the canoe problem allowed them to begin considering more modest
changes in context. Asking students what variables they would change, and to consider the impact of
those changes, invited students to take ownership of the problem. The shift in responsibility for the
problem as well as the answer offers a unique opportunity to explore the role of feedback, in that students
are now looking for answers to their own questions.
1. Is the curriculum at Khan Academy context sensitive?
All academies of learning, including Khan Academy, face the challenge of how to vary the goals in
meaningful ways to allow students to evaluate how robust their understanding is. The virtual environment
provides the unique opportunity to layer numerous dynamic contexts to create a foundation of
understanding that leads to multiple applications of the principles being studied. Much of the material
necessary is already available on the web as products of teacher inventiveness to help students understand
the plethora of concepts now required for state testing. Building on this work while maintaining attention
to all five observations is a challenge not just for Kahn but for all educators.
This issue is especially relevant as universities try to reach larger audiences through Massive Open Online
Courses (MOOCs). The courses organized through initiatives such as edX or Coursera seek to distribute
“understanding” to hundreds of thousands of students. But new educational environments such as these
are not immune to the observations noted as necessary for authentic learning. For example, the common
use of multiple-choice exams and peer evaluations falls far short of creating effective feedback. While the
outcome of the MOOC experiment is still hard to gauge, without a meaningful framework of learning,
they are unlikely to succeed, a situation already noted by the press [40, 41, 42]. Currently, completion
rates hover around 10 percent for most MOOCS [43], which may be a signal that students are aware of
the illusion of understanding and are looking elsewhere for a meaningful learning experience.
The tools of technology do offer educators and students a future that is both promising and complex. The
revolution in education that Bill Gates describes when referring to Khan Academy is based upon the
Khan Academy: The Illusion of Understanding
76 Journal of Asynchronous Learning Networks, Volume 17: Issue 4
power of magnification that technology offers; but a technological platform that claims toprovide a free
world-class education for anyone anywhere,” as Khan Academy promises, still requires understanding
how students learn as well as how to effectively evaluate the impact of instruction at the individual level
[44]. Gates’ experience in technology is noteworthy, but the thousands of hours of practice and feedback
that unfolded as he pursued his goal of understanding computers is unique [45]; thus, educators must be
careful with how individual experiences and perspectives such as his are leveraged when making
educational decisions, especially decisions that will impact the millions Gates wants to help. While he
does recognize that caution is necessary and that placing a video in front of a captive audience is not a
universal solution for educators and students [46], Gates background does not reasonably support the
level of analysis required, nor the ability to identify the most promising solutions, without his also having
dedicated thousands of hours to studying teaching and learning.
Thus, the temptation is to quickly overreach with the use of technology without seriously considering the
implications for students and teachers. For example, this year Carlos Slim, the Mexican telecom
entrepreneur, agreed to provide the resources to translate Khan academy into Spanish, but whether Khan’s
representations and personal experiences will cross the cultural divide will be a real test of how context
sensitive the videos are. I’m reminded of a lecture I gave years ago in Quebec, Canada, where I
introduced the canoe problem. Several individuals asked me what an anvil was, and I referred to the
Looney Tunes episode in which Wile E. Coyote tries to drop an anvil on the Road Runner [47]. The
audience feedback was immediate. Blank stares informed me I was either outdated or culturally out of
touch, and I needed to try again.
III. CONCLUSIONTHE ILLUSION OF UNDERSTANDING
Unfortunately, there is no way that this paper, which has taken much more than 15 minutes to write and
read, will convince anyone that they now understand what we had to observe repeatedly, practice
constantly, and evaluate through feedback from our students over years. Educators must apply these
observations every day, just as musicians and artists practice every day to become masters of their crafts.
The observations noted in this paper will still be easily discounted by strong intuitions about teaching and
learning developed over years as a result of surviving an educational system that is and has been mostly
didactic. Factors such as the lack of time or resources will easily undermine the need to carefully scaffold
experiences that allow students to build more complex understandings. Those factors undermine the
importance of developing student experiences as a foundation for new representations or a new
coordination of familiar representations. They weaken our resolve to encourage students to practice or
seek new contexts in which student understandings can be challenged. And perhaps the most pernicious
outcome of a pedagogy based on the lack of time, resources, or student feedback is that students become
dependent on their instructor’s feedback to judge their success, and instructors miss the opportunity to
recognize and develop lesson goals in which student action becomes a source of feedback that students
use to judge their success in learning.
Our students had mastered numerous tests of knowledge over many years and demonstrated their skill in
using formulas learned in numerous carefully constructed contexts, such as books, problem sets, and tests.
This work did not provide them with the conceptual understanding underlying a basic principle in the
sciences. Perpetuating the illusion of understanding with students is an easy trap for instructors to fall into
whenever they assume that they are responsible for the entire feedback loop. Instructors need to focus on
carefully chosen goals that challenge a student’s intuition, and, in turn, allows that student to complete the
feedback loop largely on their own.
This operation will of course challenge the instructor’s intuition about what teaching means. Khan, like
many educators, falls victim to this illusion in his teaching, partly because he doesn’t get the kind of
feedback that forces educators to challenge their assumptions about learning. Bill Gates and Sal Khan are
not alone in using their experiences as the lens through which they understand and respond to educational
problems. Using experiences for decision-making is natural and to be expected when creating
understandings (a point made repeatedly throughout this article). However, despite the “naturalness” of
Khan Academy: The Illusion of Understanding
Journal of Asynchronous Learning Networks, Volume 17: Issue 4 77
such actions, the risk we face collectively is of making educational decisions that ignore the last hundred
years of cognitive science and emerge predominately from our intuition, a strategy that is usually
ineffective [48]. Shortening demonstrations to match an ever-shrinking attention span or creating a
plethora of on-demand videos completely misses the shift in perspective necessary to expose the illusion
and allow teachers and students to make the adjustments necessary to support and develop authentic
understanding.
IV. ACKNOWLEDGEMENTS
I wish to recognize my colleagues who read and commented on this piece as well as encouraged me to
write it: Mike Connell, Eugenia Garduno, Irwin Shapiro, Bruce Gregory, Jeanne Gerlach, and Ken
Williford. I also want to recognize my students who read drafts of this paper and interacted with me over
time on the observations noted in the article. The input of all these individuals led to a stronger, more
focused framework for the challenging work that all educators face. I am also thankful to Gary Miller,
who saw a purpose and need to make explicit a framework for challenging the way we think about
education and the way it unfolds in an online environment.
V. ABOUT THE AUTHORS
Marc Schwartz is Professor of Mind, Brain and Education at the University of Texas at Arlington. He is
also director of the Southwest Center for Mind, Brain and Education at UTA, which seeks to identify and
support promising research agendas at the intersection of neuroscience and cognitive science to inform
(and be informed by) educational practice and leadership.
Dr. Schwartz is a charter member of the International Mind, Brain and Education Society (IMBES), past
vice president, and current president. The mission of IMBES is to facilitate cross-cultural collaboration in
biology, education and the cognitive and developmental sciences. Dr. Schwartz is also an Associate
Researcher in the Science Education Department at the Harvard-Smithsonian Center for Astrophysics
(CfA).
VI. REFERENCES
1. Schwartz, M.S. and Fischer, K.W. Building vs. Borrowing: The Challenge of Actively
Constructing Ideas. Liberal Education (Summer): 2229 (2003).
2. Schwartz, M.S, Shapiro, I., & Gregory, B. Confronting the need for conceptual change in pre-
service science education. Higher Education Studies. 3(5), 11-28 (2013).
3. Elgin, C. From Knowledge to Understanding. In: Hetherington, S. (Ed.), Epistemology Futures,
Oxford: Clarendon, 2006.
4. Bransford, J.D., Brown, A.L., and Cocking, R.R., eds. How People Learn: Brain, Mind,
Experience and School. Washington, DC: National Academy Press, 2000.
5. Elman, J., Bates, E.A. Johnson, M.H., Karmiloff-Smith, A., Parisi, D., and Plunkett, K.
Rethinking Innateness: A Connectionist Perspective on Development. Cambridge, MA: MIT Press,
1996.
6. Klahr, D. Beyond Piaget: A Perspective From Studies of Children’s Problem Solving Abilities. In:
Slater, A. and Quinn, P. (Eds.), Refreshing Developmental Psychology: Beyond the Classic Studies,
London: Sage Publishers, in press.
7. Perkins, D. Smart Schools: From Training Memories to Educating Minds. New York: The Free
Press, 1992.
8. Piaget, J. The Equilibration of Cognitive Structures: The Central Problem of Intellectual
Development. Trans. by T.T. Brown and K.J. Thampy. Chicago, IL: The University of Chicago
Press, 1985.
9. Posner, M. and Rothbart, M. Educating the Human Brain, Human Brain Development Series.
Washington, DC: American Psychological Association, 2006.
Khan Academy: The Illusion of Understanding
78 Journal of Asynchronous Learning Networks, Volume 17: Issue 4
10. Powers, W. Making Sense of Behavior: The Meaning of Control. New Canaan, CT: Benchmark
Publications Inc., 1998.
11. Siegler, R.S. Emerging Minds: The Process of Change in Children’s Thinking. New York: Oxford
University Press, 1996.
12. Schank, R. Teaching Minds: How Cognitive Science Can Save Our Schools. New York: Teachers
College Press, 2011.
13. Schwartz, M.S. Cognitive Development and Learning: Analyzing the Building of Skills in
Classrooms. Mind, Brain and Education 3(4): 198208 (2009).
14. Case, R. The Structure and Process of Intellectual Development. International Journal of
Psychology 22: 571607 (1987).
15. Commons, M.L., Trudeau, E.J., Stein, S.A., Richards, S.A., and Krause, S.R. Hierarchical
Complexity of Tasks Shows the Existence of Developmental Stages. Developmental Review 18:
237278 (1998).
16. Fischer, K.W. and Bidell, T.R. Dynamic Development of Psychological Structures in Action and
Thought. In: Damon, W. and Lerner, R.M. (Eds.), Handbook of Child Psychology: Theoretical
Models of Human Development. New York: John Wiley & Sons, 1998.
17. Piaget, J. Piaget's Theory. In: Mussen, P.M. (Ed.), Handbook of Child Psychology. New York:
Wiley, 1983.
18. van Geert, P. A Dynamic Systems Model of Basic Developmental Mechanisms: Piaget, Vygotsky,
and Beyond. Child Development 105: 634677 (1998).
19. Dawson-Tunik, T.L., Commons, M., Wilson, M., and Fischer, K.W. The Shape of Development.
European Journal of Developmental Psychology 2(2): 163195 (2005).
20. The word “representation,” as used here, focuses on the act of “re-presenting” as in re-enacting or
re-creating understanding, and also intentionally underscores the ability to reproduce the same
understanding at a later time.
21. Schwartz, M.S. and Sadler, P. Empowerment in Science Curriculum Development: A
Microdevelopmental Approach. International Journal of Science Education 29(18): 987–1,017
(2007).
22. AAAS. Atlas of Science Literacy: Project 2061. Washington, DC: American Association for the
Advancement of Science, 2001.
23. Doucerain, M. and Schwartz, M.S. Analyzing Learning About Conservation of Matter in Students
While Adapting to the Needs of a School. Mind, Brain and Education 4(3): 112124 (2010).
24. Fischer, K.W., Bullock, D., Rotenberg, E.J., and Raya, P. The Dynamics of Competence: How
Context Contributes Directly to Skill. In: Wozniak, R. and Fischer, K.W. (Eds.), Development in
Context: Acting and Thinking in Specific Environments. Hillsdale, NJ: Erlbaum, 1993.
25. Lave, J. Word Problems: A Microcosm of Theories of Learning. In: Light, P. and Butterworth, G.
(Eds.), Context and Cognition: Ways of Learning and Knowing. Hillsdale, NJ: Lawrence Erlbaum
Associates, 1993.
26. Nardi, B. Studying Context: A Comparison of Activity Theory, Situated Action Models, and
Distributed Cognition. In: Nardi, B. (Ed.), Context and Consciousness: Activity Theory and Human-
Computer Interaction. Cambridge, MA: MIT Press, 1996.
27. Wertsch, J.V., Minick, N., and Arms, F.J. The Creation of Context in Joint Problem Solving. In:
Rogoff, B. and Lave, J. (Eds.), Everyday Cognition: Its Development in Social Context. Cambridge,
MA: Harvard University Press, 1984.
28. De Corte, E. Constructive, Self-Regulated, Situated, and Collaborative Learning: An Approach for
the Acquisition of Adaptive Competence. Journal of Education 192(2): 3347 (2012).
29. Tai, R.H. and Sadler, P.S. High School Chemistry Instructional Practices and Their Association
With College Chemistry Grades. Journal of Chemical Education 84(6): 1,040–1,046 (2007).
30. Fuster, J.M. Cortex and Memory: Emergence of a New Paradigm. Journal of Cognitive
Neuroscience 21(11): 2,0472,072 (2009).
31. Langer, E.J. The Power of Mindful Learning. Reading, MA: Addison-Wesley, 1997.
Khan Academy: The Illusion of Understanding
Journal of Asynchronous Learning Networks, Volume 17: Issue 4 79
32. Powers, W. Perceptual Control Theory. Hawthorne, NY: Aldine DeGruyter, 1973.
33. Schank, R. and Cleary, C. Engines for Education. Hillsdale, NJ: Lawrence Erlbaum Associates,
1995.
34. Csikszentmihalyi, M. Flow: The Psychology of Optimal Experience. New York: Harper & Collins,
1990.
35. Schwartz, M.S. and Sadler, P.S. Goals and Technology Education: The Example of Design
Challenges. Paper read at Proceedings of the Second AAAS Research in Technology Education
Conference, in Washington, DC (2001).
36. Granott, N., Fischer, K.W., and Parziale, J. Bridging to the Unknown: A Transition Mechanism
in Learning and Problem-Solving. In: Granott, N. and Parziale, J. (Eds.), Microdevelopment:
Transition Processes in Development and Learning. Cambridge, UK: Cambridge University Press,
2002.
37. Griffin, M.M. You Can't Get There From Here: Situated Learning, Transfer, and Map Skills.
Contemporary Educational Psychology 20(1): 6587 (1995).
38. Epstein, L. and Hewitt, P.G. Thinking Physics: Practical Lessons in Critical Thinking. San
Francisco: Insight Press, 1981.
39. Hewitt, P.G. Conceptual Physics. Boston, MA: Addison-Wesley Publishing, 2010.
40. Rees, J. The MOOC Racket. Slate (2013).
41. Wilson, E. The Huge Growth of MOOCs Threatens America's Great University System. Alternet
(2013). http://www.alternet.org/education/who-profits-
moocs?akid=10609.314356.jndowS&rd=1&src=newsletter858873&t=7.
42. Koh, A. Weekend Reading: The MOOC Catchup Edition. The Chronicle of Higher Education
(2013). http://chronicle.com/blogs/profhacker/weekend-reading-the-mooc-catchup-edition/50111.
43. Jordan, K. Synthesising MOOC Completion Rates. MoocMoocher (blog) (2013).
http://moocmoocher.wordpress.com/2013/02/13/synthesising-mooc-completion-rates/
44. Khan Academy. About. (2013). https://www.khanacademy.org/about
45. Gladwell, M. Outliers: The Story of Success. New York: Little, Brown and Company, 2008.
46. Gates, B. and Microsoft. Innovation & Opportunity: The Contribution of Computing to Improving
Our World. In Virtual Faculty Summit. Redmond: Microsoft, 2013.
47. Wikipedia. Wile E. Coyote and the Road Runner. Wikipedia Foundation, Inc. (2013).
http://en.wikipedia.org/wiki/Wile_E._Coyote_and_The_Road_Runner.
48. Kahneman, D. Thinking, Fast and Slow. New York: Farrar, Straus and Giroux, 2011.
Khan Academy: The Illusion of Understanding
80 Journal of Asynchronous Learning Networks, Volume 17: Issue 4
... Few students are capable of complex learning without focused facilitation from knowledgeable experts. Arbaugh & Benbunan-Fich, 2005;Bain, 2004;Benbunan-Fich et al., 2005;Benton & Pallett, 2013;Bozkurt et al., 2015;Chu, Zhang, Chen, Chan, Lee, Zou, & Lau, 2017;Curriculum Committee, 2012;El Tantawi, Abdelsalem, Mourady, & Elrifae, 2015;Holtslander, Racine, Furniss, Burles, & Turner, 2012;Holzweiss et al., 2014;Jones, 2015;Kim, 2013;Lai, 2015;Legg, Adelman, & Levitt, 2009;Mandel & Sussmuth, 2011;Maringe & Sing, 2014;Mbati & Minnaar, 2015;Picciano, 2017;Ravenna, 2012;Russell, 2015;Schellens & Valcke, 2006;Schwartz, 2014;Taft et al., 2011;Williams, Jaramillo, & Pesko, 2015 Bloom's taxonomy: A classic in education theory, categories of learning level align in a pyramid, from simple/concrete (lower levels) to complex/abstract (higher levels). Each category contains subcategories. ...
... (b) Developing students: faculty providing incremental student feedback, mentoring; may involve creativity, problemsolving, research, writing, communication, and other skills; students access faculty expertise; faculty use the CoI practice of teaching presence Artemiou et al., 2013;Benton & Pallett, 2013;Curriculum Committee, 2012;Holzweiss et al., 2014;Horning, 2007;Jones, 2015;Lai, 2015;Maringe & Sing, 2014;Mbati & Minnaar, 2015;Monks & Schmidt, 2011;Palmer & Smith, 2013;Ravenna, 2012;Russell, 2015;Russell & Curtis, 2013;Salley & Shaw, 2015;Schwartz, 2014;Sorensen, 2014Sorensen, , 2015Taft et al., 2011;Walls, 2016 22 One Size Does Not Fit All: Toward an Evidence-Based Framework for Determining Online Course Enrollment Sizes in Higher Education ...
... (c) Mastering complex phenomena: critical thinking; deep learning with constructivist pedagogy Bristol & Kyarsgaard, 2012;Colwell & Jenks, 2004;Curriculum Committee, 2012;Kim, 2013;Lai, 2015;Maringe & Sing, 2014;Mbati & Minnaar, 2015;Monks & Schmidt, 2011;Ravenna, 2012;Russell, 2015;Schwartz, 2014;Sorensen, 2014Sorensen, , 2015Taft et al., 2011;Tynan et al., 2015;Walls, 2016 17 (d) Attaining higher order learning: advanced content at the application level or above on Bloom's taxonomy Benton & Pallett, 2013;Colwell & Jenks, 2004;Curriculum Committee, 2012;Holzweiss et al., 2014;Kim, 2013;Kingma & Keefe, 2006;Lai, 2015;Maringe & Sing, 2014;Palmer & Smith, 2013;Ravenna, 2012;Russell, 2015;Schwartz, 2014;Taft et al., 2011 13 (e) Effects of instructor inspiration: more challenge for and effort by students; greater motivation and enthusiasm; better study/work habits; greater student average progress on course objectives Benton & Pallett, 2013;Haynie, 2014;Monks & Schmidt, 2011;Palmer & Smith, 2013;Sorensen, 2014Sorensen, , 2015 Re: instructor inspiration: Sorensen (2014Sorensen ( , 2015 reported that faculty with smaller enrollments created stronger intracourse relationships and more fully shared their knowledge and expertise about the subject matter, thereby enabling the expression of teacher and student social presences and engagement, than did faculty in larger classes. Meaningful relationships generate emotions that facilitate learning (Berg & Seeber, 2016). ...
Article
Full-text available
Class enrollment sizes for online learning in higher education, a topic of persistent interest in the academic literature, impact student learning, pedagogical strategies, school finances, and faculty workload. Yet in the research literature class size is addressed with insufficient specificity to provide enrollment direction. Seeking guidelines for determining online class sizes, the authors conducted a qualitative research synthesis from 43 recent higher education journals, yielding 58 evidence-based articles. It is clear that no one size fits all. Findings reflect that large class sizes (≥ 40 students) are effective for foundational and factual knowledge acquisition requiring less individualized faculty-student interaction. Small class sizes (≤ 15 students) are indicated for courses intending to develop higher-order thinking, mastery of complex knowledge, and student skill development. Pedagogical intent should dictate class size. Using well-established learning theories, the authors describe current understandings of online enrollments and propose an analytical framework for pedagogically-driven numerically-specific class sizes.
... Having explored several educational portals, we chose the open educational resource Khan Academy which meets all the mentioned criteria. Schwartz [30] summarized five key observations about authentic understanding: thanks to the pedagogical experience of the author, Khan Academy is a suitable basis for authentic understanding stabilized with practical examples and problems to solve, offers relevant feedback, it is context-sensitive, and the particular pieces of knowledge are ordered hierarchically. ...
Article
Full-text available
The recent months have shifted contact teaching to the online environment and distance learning and students are dealing more and more with digital materials in various e-learning systems. The question is whether the online electronic materials are as effective as their printed versions for the students using them for self-study purposes. This paper presents research focusing on university students' work with an electronic and printed version of a mathematics workbook. The main research focuses on differences regarding error rate, the number of used hints, and the time they need to spend to solve 111 mathematical problems covering four topics of their introductory course of Mathematics such as limits, graphs, differentiation, and applications of derivatives. One hundred fifty-seven university students participated in the research working with sets of mathematical problems with multi-choice answers taken from the Khan Academy, including step-by-step hints. At the same time, the students were recording their errors, time, and the number of used hints using a questionnaire. The electronic sets were transformed into an electronic workbook and afterward into a printed version of this workbook. Obtained data were analysed using the Random Mixed Model as it enables to mix the used mathematical problems with different variance. The most exciting finding of this research was that the students working with the electronic version of the workbook work significantly faster but at the expense of errors. Students working with the interactive version of the workbook used significantly fewer hints.
... Furthermore, YouTube's broad impact as an education media platform sparked studies summarizing helpful tips for usage grounded in learning theories [34], as well as literature reviews on using YouTube in teaching activities across disciplines [35,36]. One study cautioned about a pitfall known as the illusion of understanding, commonly observed when watching others perform an activity without engaging in that activity, and analyzed the extent to which Khan Academy supports the illusion of understanding versus authentic understanding [37]. Based on this wealth of research, it is clear that such popular online media outlets have had a significant impact on education research over the past decade. ...
Article
Full-text available
In this mixed-methods study of large enrollment introductory physics service courses, I investigated students’ perception and use of online education resources as supplements to course-provided materials and activities. Specifically, I focused on the increasing use of popular free online media resources such as YouTube and Khan Academy, and fee-based textbook solution repository services such as Chegg. In the quantitative portion of this study, I surveyed students from three courses on their textbook and online resource usage and found that most students relied primarily on online resources as they navigated the courses, and comparatively few used the textbook regularly. In the qualitative portion, I investigated the patterns and culture of textbook and online resource usage via semistructured interviews and found that students reported using online resources as supplements to, or in place of, the course-provided materials when engaging with online homework or studying for an exam. Students reported using online resources productively to guide learning efforts, but also acknowledged unproductive uses such as copying solutions to mitigate loss of assignment points. I provide suggestions for changes in course materials, practices, and expectations to better engage students in the course and in their learning.
... A common experience associated with the illusion of knowing is the feeling of understanding a concept, but then realizing that this is not the case when you try to explain it to someone else. In other words, our lack of knowing may only become apparent once we explain our reasoning (Schwartz, 2013). So, while courts do not have to respond to absurd explanations, they should be wary of their own biases. ...
Article
Full-text available
In criminal trials, defendants often offer alternative explanations of the facts when they plead for their innocence. In its ruling on the Venray murder case, the Dutch Supreme Court dealt with the question when and how courts can reject such alternative explanations. According to the Supreme court, while courts should typically refer to evidence that refutes the explanation, they can also argue that the explanation ‘did not become plausible’ or that it is ‘not credible’. Finally, courts can state that an explanation is so ‘highly improbable’ that it requires no response. However, the Supreme Court did not explain these terms, leading to confusion about how they ought to be interpreted. This case comment offers a Bayesian interpretation according to which these three terms relate to (i) the posterior probability of the explanation, (ii) the credibility of the defendant, and (iii) how obvious it is that the explanation is improbable. This interpretation clarifies the Supreme Courts ruling and ties it to the criminal law system’s goals of error minimization and of producing understandable decisions efficiently.
Article
Transcription factor binding sites (TFBS) and RNA-binding proteins (RBP) plays a key role in gene regulation, transcription, RNA editing. Identifying and locating these potential sites is essential for detecting pathogenic variation in many biological processes. Some portions of binding sites are recognized by biological experiments that are time-intensive and expensive. Many computational approaches are considered as possible alternative solutions and few deep learning methods are recently developed for fast and accurate prediction of binding sites. Although existing approaches achieve competent performance, many methods requires specialized feature set and moreover interpretability remains challenging. To overcome these problems, we propose an interpretable deep learning technique called protein binding variable pattern predictor (PBVPP), which uses a wide variety of experimental data and performance metrics to predict binding sites. The novelty of our proposed method is based on three key factors: (i) PBVPP along with its variant has the capability to extract vital features from large-scale genomic sequences obtained by high throughput technology to predict the occurrence of TFBS and RBP sites. (ii) The proposed interpretable model reveals how to mine vital features, and also extract variable length patterns for accurate prediction of binding sites. (iii) The obtained motifs are validated against the TFBSshape DNA (JASPAR) database’s known target motifs. The proposed model has shown an improvement of 5.88%, 5.01% over state-of-the-art methods in terms of receiver operating curve for TFBS, RBP and also shown tremendous improvement of 60% in precision recall curve for TFBS prediction.
Research
Full-text available
هدفتْ الدراسة للتعرّف على فاعلية تقنية المنصّات التعليميّة في تنمية المفاهيم العلميّة في مساق استراتيجيات تعليم العلوم، على طالبات كليّة مجتمع الأقصى للدراسات المتوسطة، والتعرّف على مستوى إتقان مجموعات الدراسة، واتّبع الباحث المنهج الوصفي والمنهج شبه التجريبي، القائم على تصميم مجموعتين تجريبيتين ومجموعة ضابطة، وتكوّن مجتمع الدراسة من (122) طالبة، وتكوّنتْ عيّنة الدراسة من (90) طالبةً من طالبات قسم تربية الطفل في كليّة مجتمع الأقصى، والمُسجلات لمساق "استراتيجيات تعليم العلوم" في الفصل الدراسي الأول للعام الجامعي (2018-2019)م، وتمّ اختيار العيّنة بطريقة قصدية ضمن شروط محدّدة، وتعيين مجموعات الدراسة بصورة عشوائية، وتكوّنتْ مجموعات الدراسة الثلاث من (30) طالبةً في كل مجموعة، المجموعة التجريبية الأولى تمّ تدريسها باستخدام منصّة إدمودو التعليميّة، والمجموعة التجريبية الثانية تمّ تدريسها باستخدام منصّة مودل التعليميّة، وأمّا المجموعة الضابطة، فتمّ تدريسها بالطريقة التقليدية، وأعدّ الباحث اختباراً للمفاهيم العلميّة، وبطاقة التقييم الذاتي للمفاهيم العلميّة، وتوصّلتْ الدراسة إلى عدة نتائج كان من أهمها: وجود فروق دالة إحصائياً بين متوسط درجات مجموعات الدراسة الثلاث في الاختبار البعدي للمفاهيم العلميّة، تُعزى للتقنية المُستخدمة؛ ولصالحِ استخدام المنصّات التعليميّة، ووجود فروق ذات دلالة إحصائية بين متوسط درجات مجموعات الدراسة الثلاث في بطاقة التقييم الذاتي للمفاهيم العلميّة، تُعزى للتقنية المُستخدمة؛ ولصالحِ استخدام المنصّات التعليميّة، كما توصّلتْ الدراسة إلى وصول مستوى متوسطات مجموعتي الدراسة التجريبيتين، واللتين درستا باستخدام منصتي التعليم (إدمودو ومودل) إلى مستوى الإتقان (80%) في الاختبار البعدي للمفاهيم العلميّة، في حين لم تصلْ المجموعة الضابطة لمستوى الإتقان المُقترح وهو (80%)، ويُحقِّق توظيف المنصّات التعليميّة درجة من الفاعلية وفق مُعامل الكسب المُعدّل لبلاك في تنمية المفاهيم العلميّة في المساق المُقرر، وفي ضوء تلك النتائج؛ أوصى الباحثُ بضرورة الاهتمام بتوظيف التعلُّم الإلكتروني والمنصات التعليميّة في العملية التعليميّة، باعتبارهما من مُتطلبات التعليم في القرنِ الحادي والعشرين، ولمواجهة الثورة المعرفية والتكنولوجية، وتشجيع المعلمين والمتعلّمين وحثِّهم على استخدام المنصّات التعليميّة، وتطوير المناهج التعليميّة بتضمينها لموضوعات توظّف المنصّات في العملية التعليميّة.
Article
Full-text available
This article uses a specific context and problem to explore how the brain's biology influences the learning process and how this perspective can help shape adult education policy. Set in Mexico in an area of high seismic activity, the population is faced with building challenges that will impact the lives of all. The role of the brain in addressing the problems the community faces is divided into four dimensions to highlight how each support the learning process and how they can ultimately be used to re-envision education policy.
Article
Full-text available
Through the most recent years Information and Communication Technologies (ICT) have been positioned as powerful and revitalizing mechanisms in the educational processes. Therefore, this study measures the incidence of a digital platform by the learning of real functions with a view of identifying didactic strategies that contributes to the strengthening and development of mathematical knowledge in reference to the variational thought. The original random sample was primary conformed by students of second semester at a technological academic formation of an institution of higher education based on Santander-Colombia (N = 176). The respective quantitative study is being considering by the correlation type, with the proper use of the Spearman's correlation coefficient and descriptive statistics. The used program to realize this kind of analysis was the statistical software SPSS. So far at this point the results shows that the average academic performance on the partial of functions evidenced was 2.61 with standard deviation (SD)0.92, meanwhile in another way by the independent work that is being taken in the Khan-Academy's platform scored 3.72 with a (SD) 0.2. Besides, it can be said that the bivariate correlation coefficient shown a substantial relation of 0.846 with a significance level of 1%. in all the process the conclusions that can be considered, it can be affirmed the strong relation between the study variables, denoting in fact that the correlation coefficient showed a clear association between the variables.
Article
Full-text available
In today's learning society, education must focus on fostering adaptive competence (AC) defined as the ability to apply knowledge and skills flexibly in different contexts. In this article, four major types of learning are discussed—constructive, self-regulated, situated, and collaborative—in relation to what students must learn in order to acquire AC in a particular domain. Two questions are addressed: What are the characteristics of productive learning processes that are required in order to acquire AC?, and How can such learning be stimulated and sustained through instruction? An illustrative study is presented that focuses on the design of a learning environment for improving problem-solving competence in primary school students. Concluding comments address the challenges to the implementation of innovative learning environments.
Article
Full-text available
During a five-year period the authors taught over 100 students in a graduate course (The Nature of Science) counting toward teacher certification at the Harvard Graduate School of Education. Despite the fact that students had undergraduate degrees in the sciences, most of them found the application of models in science challenging and the epistemological consequences unsettling. Moreover, students found it especially difficult to use a model to correctly generate predictions, which was starkly illustrated with the application of Archimedes’ principle during our unit on floating and sinking. We examine the deceptive belief that student success with algorithms and word problems leads to conceptual understanding as well as the conceptual change necessary to understand the relationship between evidence and inference as explored in the nature of science. Considering the apparently strong science backgrounds of our students, we doubt that typical pre-college students can achieve the goals described in the National Science Education Standards in the short time typically allotted for their science studies. We explore the issues students face in “understanding” science as well as the impact of science education on students and teachers, and implications for policy makers and pre-service programs.
Article
Full-text available
This project examines the shape of conceptual development from early childhood through adulthood. To do so we model the attainment of developmental complexity levels in the moral reasoning of a large sample (n= 747) of 5- to 86-year-olds. Employing a novel application of the Rasch model to investigate patterns of performance in these data, we show that the acquisition of successive complexity levels proceeds in a pattern suggestive of a series of spurts and plateaus. We also show that there are six complexity levels represented in performance between the ages of 5 and 86; that patterns of performance are consistent with the specified sequence; that these findings apply to both childhood and adulthood levels; that sex is not an important predictor of complexity level once educational attainment has been taken into account; and that both age and educational attainment predict complexity level well during childhood, but educational attainment is a better predictor in late adolescence and adulthood.
Article
Full-text available
Converging evidence from humans and nonhuman primates is obliging us to abandon conventional models in favor of a radically different, distributed-network paradigm of cortical memory. Central to the new paradigm is the concept of memory network or cognit--that is, a memory or an item of knowledge defined by a pattern of connections between neuron populations associated by experience. Cognits are hierarchically organized in terms of semantic abstraction and complexity. Complex cognits link neurons in noncontiguous cortical areas of prefrontal and posterior association cortex. Cognits overlap and interconnect profusely, even across hierarchical levels (heterarchically), whereby a neuron can be part of many memory networks and thus many memories or items of knowledge.
Book
How do children acquire the vast array of concepts, strategies, and skills that distinguish the thinking of infants and toddlers from that of preschoolers, older children, and adolescents? In this new book, Robert Siegler addresses these and other fundamental questions about children's thinking. Previous theories have tended to depict cognitive development much like a staircase. At an early age, children think in one way; as they get older, they step up to increasingly higher ways of thinking. Siegler proposes that viewing the development within an evolutionary framework is more useful than a staircase model. The evolution of species depends on mechanisms for generating variability, for choosing adaptively among the variants, and for preserving the lessons of past experience so that successful variants become increasingly prevalent. The development of children's thinking appears to depend on mechanisms to fulfill these same functions. Siegler's theory is consistent with a great deal of evidence. It unifies phenomena from such areas as problem solving, reasoning, and memory, and reveals commonalities in the thinking of people of all ages. Most important, it leads to valuable insights regarding a basic question about children's thinking asked by cognitive, developmental, and educational psychologists: How does change occur?
Article
A sub-sample of data from a nationally representative survey of introductory college science students entitled "Factors Influencing College Science Success" is studied. Final college chemistry grade (introductory college chemistry grade - ICCGRADE) is used as the outcome measure. Two control predictors are considered as follows: demographic identifiers and general educational background measures. Multiple linear regression analysis is applied because it has the capacity to analyze the significance of instructional practice variables as predictors of the outcome measure while controlling for background differences. Trends in the predictors show that peer teaching is associated positively with higher grades while individual work is negatively associated.