Recursive Hierarchical Recognition: A Brain-based Theory of Language Learning

Abstract

The advent of multimedia computers allows for multimodal input and practice, where learning activities can take advantage of the hierarchical structure of the human brain and the interplay between listening, speaking, memory, and the pattern-recognition logic that is at the heart of human intelligence. Listening and speaking-based activities can now be coordinated with visual, conceptual and phonological inputs not possible with textbooks, or even in classroom activities. This creates opportunities for fundamental changes in language learning, including a rethinking of the relationship between the 4 skills, with the skills of listening and speaking elevated to playing their key roles. It also brings into focus the realization that too much precision and language 'knowledge' may work against the learning process. In fact, a tolerance for ambiguity becomes a predictor of language learning success and guessing becomes one of the learning skills to be encouraged in the language learning process. Recursive Hierarchical Recognition (RHR) is a learning theory that addresses these issues. It has been developed to guide the development of learning materials and activities, and is supported by the study records of thousands of students studying in diverse circumstances in over 50 countries. As more data is collected, it continues to evolve.

Full text

First published as a plenary paper in FEELTA/NATE Conference Proceedings © 2008
Lance Knowles
DynEd International
lknowles@dyned.com
Recursive Hierarchical Recognition: A Brain-based
Theory of Language Learning
The advent of multimedia computers allows for multimodal input and practice,
where learning activities can take advantage of the hierarchical structure of the human brain
and the interplay between listening, speaking, memory, and the pattern-recognition logic
that is at the heart of human intelligence.
Listening and speaking-based activities can now be coordinated with visual,
conceptual and phonological inputs not possible with textbooks, or even in classroom
activities. This creates opportunities for fundamental changes in language learning,
including a rethinking of the relationship between the 4 skills, with the skills of listening and
speaking elevated to playing their key roles. It also brings into focus the realization that too
much precision and language ‘knowledge’ may work against the learning process. In fact, a
tolerance for ambiguity becomes a predictor of language learning success and guessing
becomes one of the learning skills to be encouraged in the language learning process.
Recursive Hierarchical Recognition (RHR) is a learning theory that addresses these
issues. It has been developed to guide the development of learning materials and activities,
and is supported by the study records of thousands of students studying in diverse
circumstances in over 50 countries. As more data is collected, it continues to evolve.

First published in FEELTA/NATE Conference Proceedings © 2008
This presentation offers an overview and key concepts of RHR, such as Multimodal
Input, Hebbian Learning, Temporal Tension, Conceptual Chunking, and Language
Bootstrapping.
Procedural Memory and Automaticity
From the neurosciences, we know that there are several kinds of memory systems.
Episodic memory is responsible for explicit memory (event and fact learning), that is,
learning with awareness. Procedural memory is responsible for implicit memory (skill
learning), that is, learning without awareness [Restak: p 79]. Procedural memory is used
for carrying out a skill. A skill involves the activation of an automatic sequence of actions
that have been acquired through repetition and/or practice over a suitable period of time.
Procedural memory depends on a network of neural structures that execute relatively
automatic subroutines. RHR assumes that unconscious neural routines – not knowledge
about a language – do the heavy work of breaking down, chunking, and reassembling
language for comprehension or oral expression. These neural routines involve pattern
recognition, and follow the learning sequence: (1) familiarization (2) recognition (3)
comprehension, (4) mastery, and (5) automaticity.
To accelerate language learning, we must facilitate the above sequence. This is
accomplished through the multi-modal input of language models that follow a learning path
that makes efficient use of Long Term (LT) memory. Language input and language practice
work in a recursive, circular manner to wire in the pattern-recognizing subroutines.
Multimodal Input
Also from the neural sciences we are learning about the nature of brain plasticity, the
kinds of changes in the brain that occur when learning takes place. We know that
2

First published in FEELTA/NATE Conference Proceedings © 2008
multimodal activities in particular enhance the creation of new or strengthened synaptic
connections, which is the stuff of new memories, especially procedural memories. As the
famous neuroscientist, Donald Hebb said: Neurons that fire together, wire together. This is
the basis for Hebbian learning: that repeated excitations of a sequence of neurons modifies
the synaptic connections between those neurons.[Hebb: pg 62] As a result, RHR stresses
the importance of multimodal practice: listening, seeing, speaking, acting, and processing
information.
By multimodal, I mean the coordinated, synchronized activation of visual, auditory,
conceptual, and other systems within the brain – something that well-designed multimedia
exercises can provide – unlike textbooks, which are page-based, non-temporal, and require
initial orthographic processing.
Language processing requires many neural systems to interact, with information
flowing upward and downward within the brain. Figure 1 is an oversimplified diagram that
shows how various processors in the brain communicate with each other and the working
memory.
Figure 1
Lon
g
-Term Memories
Working Memory
Visual
Processors
Auditory
Processors
Conceptual
Processors
3

First published in FEELTA/NATE Conference Proceedings © 2008
Well-designed multimedia exercises activate and synchronize the appropriate
processors in ways not previously possible. These processors work in parallel and interact
with the working memory and long-term (LT) memory to piece together and interpret
language and sensory input. A well-designed multimedia program optimizes this process,
both in the presentation of language models and in the interactive exercises that support
them. In particular, long term (LT) memory, visual information, and conceptual processors
work together to help decode and fill-in comprehension gaps.
Figure 2
The process begins by presenting visual inputs arranged so that the general meaning
can be inferred without any language or auditory input. This takes advantage of the brain’s
natural ability to make sense of things and fill-in details or
patterns to fit one’s expectations. In Figure 2, for example,
the brain instantly and naturally fills in the expected
pattern. In other words, it takes incomplete information
and extrapolates, fills in, or infers the rest. RHR takes
advantage of this natural ‘learning force.’
With well-designed multimedia exercises, we can develop the oral skills, step by
step, taking advantage of how brain systems work together, how memories are formed, and
helping the learner facilitate the learning process by using what is known to fill in gaps and
discover rules and patterns that lead to more efficient processing, which is the key to oral
fluency, and ultimately to all 4 skills.
During practice sessions, students are coached to listen multiple times to a language
model in context and supported by synchronized, visual input of an iconic nature, such as
geometric figures, charts, or arrangements of pictures designed to express causal
4

First published in FEELTA/NATE Conference Proceedings © 2008
relationships. These kinds of visuals help learners to infer the meaning of an utterance, or a
series of utterances, especially if they are animated or brought into focus so that the visual
and auditory inputs are appropriately synchronized. With each passing sentence or question,
the underlying language patterns and gaps are perceived, with or without conscious
awareness of the patterns themselves. As this process is repeated over several days, the
familiar patterns begin to carry meaning even into novel situations.
To accomplish this, the language models must be carefully arranged to help learners
to discover the underlying language framework and resolve ambiguities before they lead to
frustration. Learners can generally guess the meaning by using their knowledge of the world
and the conceptual logic that is wired into our brains. This guessing process, followed by the
elimination of wrong choices, appears to be a much faster way to learn than trying to learn
every detail and then piece things together.
Some neuroscientists believe this conceptual structuring is done through millions of
tiny cortical columns in the brain’s neocortex, each one of which processes a specific type of
sensory input. When groups of these columns are switched on repeatedly, they wire
together to form a networked assembly that can be instantly activated as a whole, thereby
increasing the speed with which language input can be processed and chunked. RHR
predicts that appropriate multi-modal practice activities accelerate this wiring process.
Chunking and Temporal Tension
When developing the oral skills, RHR follows the “4-skills path” [Knowles 2004].
Listening comes first, supported by visual, conceptual, and LT memory inputs. Oral
repetition follows, with the aim of developing the skill to organize language into phrases, or
chunks. This is done by having the student focus on parts of each sentence until the parts
5

First published in FEELTA/NATE Conference Proceedings © 2008
can be grouped together and repeated as a whole. For example, “The person on the left is a
woman” may be broken into three units at first: (1) The person (2) on the left (3) is a
woman. Then, with practice, the student can break it
into 2 units: (1) The person on the left (2) is a woman.
Then, with more practice, the students can repeat is as a
whole: “The person on the left is a woman.” Once the
student can do this, over a period of several days, the student will be able to process the
entire sentence even if spoken quickly.
During the above activity, RHR suppresses any text support, especially for older
learners and false beginners. The use of text can interfere with the listening process and
reduces the temporal tension that activates the pattern recognition logic of the brain.
Temporal tension, provided that it’s the right amount, helps to develop the chunking skill.
Another disadvantage of text is that it often causes graphical interference – where the
learner’s previous phonetic model of the text distorts what is actually heard.
Once students are able to listen to and repeat the entire sentence – with confidence
and relative fluency – they can begin to look at the text for confirmation. This provides
another form of repetition, and additional orthographic input, which reinforces the memory.
Beyond this, writing exercises can provide yet another opportunity for practice, input and
extension.
In our experience many students who consider themselves to be at an intermediate or
advanced level are surprised by their inability to process language without text support.
Their oral fluency level is much lower. Such students have never developed the
automaticity necessary to chunk language. This explains their lack of confidence and
6

First published in FEELTA/NATE Conference Proceedings © 2008
limited oral fluency. In RHR, chunking ability is proportional to fluency, and chunking is a
skill that can develop through frequent and sequenced practiced.
In RHR, listening and speaking are the primary language skills and should always
come first in the skill acquisition process. Though learners and teachers may find the use of
text a useful and comfortable support, this comfort comes at a high cost because it
eliminates the temporal tension. An appropriate amount of temporal tension leads to
attention, efficient practice, and language automaticity. Learners should be encouraged to
leave their comfort zone.
RHR predicts that reversing the order of skills – which is the common practice –
delays the language acquisition process. As argued above, relying on text support short-
circuits the process of developing the gap-filling, pattern recognition circuits necessary for
oral skills to develop quickly. Therefore, students who are uncomfortable or unable to
practice without text support should be given lower level material to work with, and coached
so that they can develop a more efficient way to practice.
The temporal nature of oral communication is fundamental. Oral communication is
temporal, not spatial. Unlike text, which is static and visible, speech input flows quickly
through the brain. Language processing must be done quickly and the input must be held in
memory buffers that are limited in size.
As the cognitive scientist Steven Pinker points out: “Phonological short-term
memory lasts between one and five seconds and can hold from four to seven “chunks.
(Short-term memory is measured in chunks rather than sounds because each item can be a
label that points to a much bigger information structure in long-term memory, such as the
content of a phrase or sentence.”[Pinker 1997: p 89]
7

First published in FEELTA/NATE Conference Proceedings © 2008
The pressure to hold auditory information in limited memory buffers creates
temporal tension, which can engage and motivate the learner – if done in short, frequent
sessions. However, too much tension can lead to frustration, so it is essential to place
learners into a learning sequence where the length and complexity of the target language is
appropriate. Hence a good placement test, monitoring, and frequent testing are important.
These only have utility, however, if they can assess the chunking skill of the learner. An
assessment of vocabulary, for example, would not be appropriate.
In addition to placement and ongoing assessment, language input should be designed
so that the key patterns are in abundance and appropriately sequenced. Without this
preparation, RHR cannot work, or will be severely limited. The patterns must be there to be
recognized and acquired. Without that, the language input becomes noise to the brain, not
music, and tension becomes frustration and defeating.
Conceptual Sequencing
In RHR, language chunks are built around concepts – which express elements of
information – or language functions – which signal the type of speech act (e.g. request,
suggestion) being expressed. Examples of concepts include: point of time (after arriving,
when it started), frequency (several times a week, sometimes), and events (the car went off
the road, they practiced).
Teaching discrete words is avoided. Instead, lexical items are presented in phrases,
such as ‘a book’, ‘a red book’, ‘a green book’ ‘open the red book’, etc. Presenting
vocabulary in this way – without text support at first – facilitates conceptual chunking while
also teaching the vocabulary.
8

First published in FEELTA/NATE Conference Proceedings © 2008
Processing a single word or number is a relatively shallow process. It’s fast and can
easily be remembered for a short time. However, research suggests that as the level of
processing deepens, more neural linkages and associations facilitate long-term learning
[Craik 1975]. Abstracting and generalizing are natural processes that are conceptually based
and provide a means for storing information and consolidating memories. Routines,
templates, and conceptual ‘patterns’ seem to be the building blocks of thought and language.
Many of the most common words work as indices, or switches, to concepts or sets of
concepts. These marker words switch on various concept areas. The preposition ‘at’ for
example signals location in time or space. Such marker words head a phrase that can be
chunked around a concept. The brain anticipates that some location in time or space is
forthcoming: ‛at her house’ or ‛at the end of the performance.’ Similarly, the word ‛for’
activates a set of conceptual areas, including duration (for a few minutes) and purpose (for
her school). Depending on what words actually follow (e.g. few minutes), the alternative
concepts (purpose, etc.) are eliminated.
These examples also indicate how the meaning of a word depends on the words and
context around it, which is another reason why RHR rejects word lists. When acquiring a
new language, the goal is to facilitate the recognition of patterns, not discrete lexical items.
The hierarchical structure of memories and concepts is a key feature in RHR. RHR
suggests that the optimum learning sequence moves from basic concepts such as object and
event to complex concepts where many concepts are embedded within other concepts, such
as “while he was driving home”, which expresses duration but which has other concepts
embedded within it (process, direction, etc.). Optimum learning sequences should resonate
9

First published in FEELTA/NATE Conference Proceedings © 2008
with how memories are associated in the brain and how concepts are organized in our
environment.
Iconic Presentation
In RHR, multimedia presentations make extensive use of icons to support language
input. Icons are visual objects that alone or in combination with other icons communicate
information independent of language input. RHR uses icons to provide visual cues that
work to activate LT memories and associations. This process stimulates the brain to guess
meaning which can then be used to fill-in language gaps and identify language patterns.
Examples of icons include: numbers, geometric shapes, symbols, pictures of objects
or actions, and charts. For an icon to work, it must connect to the long-term memory of the
learner so that it activates a set of concepts in memory. Shown a triangle, for example, the
brain immediately activates a set of attributes associated with a triangle. If we now say “A
triangle has 3 x,” then one anticipates that x means either side or angle. This is because the
attributes of a triangle are inherited in the target language. If the next visual input shows one
or more sides highlighted, then the meaning ‘angle’ is eliminated in favor of side. There is
no need of translation, provided that the icon is age-appropriate. Obviously if a learner
doesn’t know what a triangle is, then it isn’t appropriate as an icon.
Multimedia computers facilitate the use of icons. Animation and the sequential
presentation of iconic visuals cannot be done in a textbook, but is easily done in brain-based
programs like those developed at DynEd [www.dyned.com] where we specialize in this type
of design. The essence of an iconic presentation is simplicity and clarity – to work as an
effective mental trigger or mnemonic device. In contrast, the presentation of too much
information becomes no information at all.
10

First published in FEELTA/NATE Conference Proceedings © 2008
LT Memory and Language Bootstrapping
RHR makes extensive use of Long-Term Memory. Experience and real-world
knowledge is systematically used to aid the acquisition process. Steven Pinker used the term
‘bootstrapping’ when he hypothesized how children use meaning to acquire language syntax
[Pinker 1994]. Unlike an L1 learner, an L2 learner has an extensive LT memory of
academic and professional subject matter that can be drawn on to facilitate inductive
learning. As a result, learning can be more efficient and motivating, because the brain is
solving problems rather than memorizing or communicating about generic content of no
consequence to the learner.
An interesting example of how this has been applied is a course for airline pilots:
Aviation English [Knowles, 2007]. In situations where an airplane is about to land and the
wind suddenly shifts, we can predict and use the knowledge and experience of pilots to
anticipate what course of action to consider. This knowledge and experience is language
independent. Therefore, a Chinese pilot learning to speak English will use this knowledge
and experience to fill in the language gaps and ‘bootstrap’ the learning process. However,
this can only happen if the language input is designed with this in mind, and with the
requisite aviation knowledge that the pilot has.
In other words, a student can use knowledge of math and science to learn English;
because this knowledge is language independent. If I show you two parallel lines and say
“These two lines never X”, you know that X means intersect or cross. An example of this
approach is seen in the DynEd course, English for Success [Knowles 2004]. This course
uses the knowledge of school subjects such as math, science and geography to help
‘bootstrap’ English language acquisition, in particular academic English.
11

First published in FEELTA/NATE Conference Proceedings © 2008
RHR Blended Model
In relation to the classroom, RHR supports the notion that the most efficient
language learning approach is a blend, with well-designed multimedia programs and
coordinated classroom activities working together. There is no evidence to suggest that
computers can or should replace the classroom.
In the RHR blended model, both computers and the classroom have roles to play.
The strengths and the limitations of each are recognized. Language models are introduced
and practiced through multimedia-based listening and speaking activities. This is followed
up, personalized and extended through classroom activities, and then extended again through
paper-based reading and writing exercises in an expanding spiral. Learners are active, not
passive, and work at an optimal language level which is adjusted and monitored by the
software.
Compared to a classroom-only approach, the advantages of this kind of practice are
manifold, particularly in the total amount of productive time on task. If coached properly,
the number of learning encounters per session is significantly higher than in a classroom-
only scenario and can be monitored.
In addition to computer-based lessons, the classroom provides the human element,
accommodating the needs and lives of learners in a social context. Through communication
activities such as oral presentations, pair work, role plays, and discussions, learners extend
and personalize the language previously presented and practiced in their multimedia
sessions. And in the best case, the teacher guides and facilitates these activities, with very
little lecturing.
12

First published in FEELTA/NATE Conference Proceedings © 2008
In this skills-based approach, multimedia practice activities form the core of the
learning process and provide the conceptual framework for communication activities. The
teacher is in overall control, not only in the classroom, but in setting and monitoring the
learning paths for the students, who now rely on practice and acquired skills rather than
memorization.
Conclusion
RHR makes predictions that can be tested – under the right conditions and with an
awareness of the large number of variables that affect language acquisition, including the
teacher and testing instruments, both of which have built-in biases. Some of these
predictions are:
1. Delaying text and following the “4-Skills Path” accelerates fluency development.
2. Frequent speaking practice which focuses on chunks of increasing length and
conceptual complexity without text support results in accelerated fluency.
3. Vocabulary is best taught in phrases rather than in isolation. Word lists should be
avoided.
4. Oral fluency facilities reading and writing skills.
RHR offers a new and practical approach to language acquisition and materials
design. Brain-based CALL (BB-CALL) materials used in a blend with classroom activities
take advantage of this approach, and are now being used by several million students around
the world. The traditional, text-based approach needs to be challenged.
Whatever approach one takes, testing, monitoring and accountability should be
expected and systematically utilized. Now that computers are available and connected,
opportunities for rethinking language teaching principles abound, with plenty of data
available to test assumptions. And the insights from neuroscience should be a part of every
language teacher’s training.
13

First published in FEELTA/NATE Conference Proceedings © 2008
“Human children appear preadapted to guess the rules of syntax correctly, precisely because
languages evolve so as to embody in their syntax the most frequently guessed patterns. The brain has co-
evolved with respect to language, but languages have done most of the adapting.” [Deacon 1997: p122]
References
Brown, R. W. Learning, Hierarchical Storage, Assembly and Recall / R. W. Brown //
Proceedings of the 2003 ASEE/WFEO International Colloquium
Craik, F. Depth of processing and the retention of words in episodic memory / Craik, F. I.
M. Craik, E.Tulving // Journal of Experimental Psychology - 1975 - General, 104, 268-294.
Deacon, T. The Symbolic Species: The Co-evolution of Language & the Brain. (T.W.
Deacon) -NY: WW Norton, 1997
Feldman, J. The Simplicity Principle in Human Concept Learning / J. Feldman // Current
Directions in Psychological Science - 2003 - pp 227-232
Hawkins, J. On Intelligence (J. Hawkins) New York, Times Books, Henry Holt & Company
-2004
Hebb, D. The Organization of Behavior (D. Hebb) Wiley - 1949
Knowles, L. On the Cusp: New Developments in Language Teaching / L. Knowles // ESL
Magazine, Issue 40, July/August 2004 - available http://www.dyned.com/pdf/Teacher-
Guides/TGTHEORY.PDF [accessed: 08:05:2008]
Knowles, L. The Evolution of CALL / L. Knowles // Language Magazine, August 2004
Knowles, L. English for Success (L. Knowles) DynEd International - 2003 -
www.dyned.com/products/efs/
Knowles, L. Aviation English (L. Knowles et al) DynEd International - 2007 -
http://www.dyned.com/products/ae/
Knowles, L. Mind Blocks / L. Knowles // Language Magazine - August 2008
Krashen, S. The Input Hypothesis (S. Krashen) Beverly Hills: Laredo - 1985
Kuhl, P. Early Language Acquisition: Cracking The Speech Code / P. K. Kuhl // Nature
Reviews/Neuroscience |Vol 5| pp831-41 www.nature.com/reviews/ Nov. 2004
doi:10.1038/nm1533
Lazarus, R. Autonomic discrimination without awareness: A study of subception / R.
Lazarus, R. McCleary // Psychological Review - 1951 - 58, pp 113-22
LeDoux, J. The Emotional Brain (J. LeDoux) New York, Simon and Schuster - 1996
14

First published in FEELTA/NATE Conference Proceedings © 2008
15
Lidz, J. Understanding How Input Matters: Verb Learning and the Footprint of Universal
Grammar / J. Lidz, H. Gleitman et al // Cognition 87.3 - 2003 - pp 151-178
Palmer, H. The Oral Method of Teaching Language (H. E. Palmer) University College,
London - 1921
Pinker, S. How could a child use verb syntax to learn verb semantics? /S. Pinker // Lingua,
92 North Holland - 1994 - pp 377- 410
Pinker, S. How the Mind Works (S. Pinker) New York: W.W. Norton & Company - 1997
Restak, R. The Modular Brain. (R.M. Restak, M.D.) Macmillan, New York - 1994
Shukla, M. Revealing the Workings of Universal Grammar /M. Shukla // Journal of
Bioscience 28.5 (September 2003) pp 535-537
Snedeker, Jesse. Cross-Situational Observation and the Semantic Bootstrapping Hypothesis.
E Clark (ed) Proceedings of the Thirtieth Annual Child Language Research Forum. (2000)
Stanford , CA Center for the Study of Language and Information
Ulman, M. Contributions of memory circuits to language: the declarative/procedural model /
M. T. Ulman // Cognition 92 -2004 - pp 231-270 available at
http://www.brainlang.georgetown.edu/PUBS/Ullman_Cognition_04.pdf [accessed:
08:05:2008]
Wilkins, D. Notional Syllabuses. (D. Wilkins) London: Oxford University Press - 1976.
Zajonc, R. Feeling and Thinking: Preferences Need No Inferences / R. Zajonc // American
Psychologist 35 - 1980 - pp151-75
Author’s bio:
Lance Knowles, President and Head of Courseware Development
DynEd International, Inc. (www.dyned.com)
Mr Knowles has pioneered the development and use of CALL for more than 20 years. His
innovative learning theory, RHR, is based on neuroscience, and his award-winning programs
are used by students in over 50 countries.