ArticlePDF Available

Reflective Judgment: Theory and Research on the Development of Epistemic Assumptions Through Adulthood



The reflective judgment model (RJM) describes the development of complex reasoning in late adolescents and adults, and how the epistemological assumptions people hold are related to the way they make judgments about controversial (ill-structured) issues. This article describes the theoretical assumptions that have guided the development of the RJM in the last 25 years, showing how these ideas influenced the development of assessment protocols and led to the selection of research strategies for theory validation purposes. Strategies discussed here include a series of longitudinal studies to validate the proposed developmental sequence, cross-sectional studies examining age/educational level differences, and studies of domain specificity. Suggestions for assessing and promoting reflective thinking based on these findings are also offered here.
Reflective Judgment: Theory and Research on the
Development of Epistemic Assumptions Through
Patricia M. King
Center for the Study of Higher and Postsecondary Education
University of Michigan
Karen Strohm Kitchener
School of Education
University of Denver
The reflective judgment model (RJM) describes the development of complex reasoning in late
adolescents and adults, and how the epistemological assumptions people hold are related to the
way they make judgments about controversial (ill-structured) issues. This article describes the
theoretical assumptions that have guided the development of the RJM in the last 25 years, show-
ing how these ideas influenced the development of assessment protocols and led to the selection
of research strategies for theory validation purposes. Strategies discussed here include a series of
longitudinal studiestovalidate theproposeddevelopmentalsequence,cross-sectionalstudies ex-
amining age/educational level differences, and studies of domain specificity. Suggestions for as-
sessing and promoting reflective thinking based on these findings are also offered here.
Do the benefits of inoculating health care workers against
smallpox outweigh the risks? Will a proposed urban growth
policy protect farmland without sacrificing jobs? Is affirma
tive action an effective tool for promoting genuine access to
higher education? Controversial problems such as these
about which “reasonable people reasonably disagree” are
called ill-structured problems (Churchman, 1971; Wood,
1983); they are characterized by two features: that they can
not be defined with a high degree of completeness, and that
they cannot be solved with a high degree of certainty. In the
last 25 years, we have investigated how late adolescents and
adults come to understand and make judgments about these
kinds of controversial problems. In examining the responses
that hundreds of individuals across a wide range of age and
educational levels have given to such questions, we have
made three major observations: (a) there are striking differ
ences in people’s underlying assumptions about knowledge,
or epistemic assumptions; (b) these differences in assump
tions are related to the way people make and justify their own
judgments about ill-structured problems; and (c) there is a
developmental sequence in the patterns of responses and
judgments about such problems. The reflective judgment
model (RJM; King & Kitchener, 1994; K. S. Kitchener &
King, 1981) provides a theoretical framework for under
standing and organizing these observations.
In this article, we begin by presenting the original theoreti
cal grounding and underlying assumptions that have guided
the development of the RJM, as well as the influence of subse
quent theoretical developments, and then show how these as
sumptions guided the development of assessment protocols
and led to the selection of research strategies for theory valida
tion purposes. The next major section of this article summa
rizes research on the RJM that tested the theoretical claims
about the development of reflective judgment. Last, we ex
plore some of the implications for practice and research based
onthetheoreticalideasandempiricalfindingspresented here.
The RJM is a model of the development of reflective thinking
from late adolescence through adulthood. This construct was
Copyright © 2004, Lawrence Erlbaum Associates, Inc.
Requests for reprints should be sent to Patricia M. King, Center for the
Study of Higher and Postsecondary Education, University of Michigan, 610
E. University, Room 2117, Ann Arbor, MI 48109–1259. E-mail:
first defined by Dewey (1933), who argued that reflective
judgments are initiated when an individual recognizes that
there is controversy or doubt about a problem that cannot be
answered by formal logic alone, and involve careful consid
eration of one’s beliefs in light of supporting evidence. This
kind of reasoning remains a central goal of education, espe
cially higher education; this is evident in several recent na
tional reports on undergraduate education, each of which re
iterated the need for college graduates to think reflectively
(American Association of Colleges and Universities
[AAC&U], 2002; American Association of Higher Educa
tion, American College Personnel Association [ACPA], &
National Association of Student Personnel Administrators,
1998; ACPA, 1994). The RJM describes a progression of
seven major steps in the development of reflective thinking
leading to the capacity to make reflective judgments; each
step represents a qualitatively different epistemological per
spective. In defining these perspectives, we use K. S.
Kitchener’s (1983) definition of epistemic cognition (as dis
tinguished from cognition and metacognition), focusing on
individuals’ underlying assumptions about knowledge and
how it is gained. For each step in this progression (which we
call stages, as defined below) the RJM includes a description
of individuals’ views of knowledge and concepts of justifica-
tion, showing the relationship between the epistemological
assumptions people hold and the way they make judgments
about controversial (ill-structured) issues. The model shows
how the assumptions are interrelated and how they reflect an
internal logic within each stage. (See Table 1.) In the follow-
ing description, we offer a general overview of these seven
stages and offer examples from Stage 4, which is characteris-
tic of the reasoning of a majority of college students, and
Stage 7, which is indicative of the kind of reasoning many
colleges aspire to teach (AAC&U) and which has been asso
ciated with the kind of thinking skills adults need to function
effectively in today’s complex societies (Baxter Magolda, in
press; Kegan, 1994). These examples illustrate how qualita
tively different sets of epistemic assumptions are associated
with distinctly different ways of justifying beliefs in adult
hood. Because a detailed descriptions of each stage is avail
able elsewhere (King & Kitchener, 1994), here we offer only
a brief summary of the seven stages.
As an introduction to this developmental progression,
consider the seven stages grouped into three levels:
prereflective thinking (Stages 1–3), quasi-reflective thinking
(Stages 4–5), and reflective thinking (Stages 6–7). We are
aware that labels such as these (e.g., “absolutist”) risk being
interpreted as overly simplistic reflections of complex
epistemological perspectives and that similar terms are used
to refer to quite different epistemologies (as documented by
Hofer & Pintrich, 1997). For this reason, we use numbers to
reflect the order in which the epistemological perspective
typically emerges and offer these broader categories as a
more general introduction to the model. Although clustering
qualitatively different stages into levels reduces complexity
by collapsing stages within levels and highlighting similari
ties, this strategy also risks obscuring within-level
differences. Thus, the summaries that follow should be read
with the understanding that important differences exist be
tween stages, both within and across levels.
A major hallmark of prereflective thinking is that knowl
edge is assumed to be certain, and accordingly, that single
correct answers exist for all questions and may be known
with absolute certainty, usually from authority figures. Well-
and ill-structured problems are not differentiated, as all prob
lems are assumed to be well structured. Further, those using
prereflective assumptions do not use evidence to reason to
ward a conclusion, relying instead on a restatement of beliefs
or on unsubstantiated personal opinions. With quasi-reflec
tive thinking comes the recognition that uncertainty is a part
of the knowing process, the ability to see knowledge as an ab
straction, and the recognition that knowledge is constructed.
This is a major advance, as it lays a foundation for the con
struction of beliefs that are internally derived, not simply ac
cepted from others. Further, evidence is now understood as a
key part of the knowing process, as it provides an alternative
to dogmatic assertions that are characteristic of prereflective
thinking. Those using quasi-reflective assumptions are aware
that different approaches or perspectives on controversial is-
sues rely on different types of evidence and different rules of
evidence, and that factors like these contribute to different
ways of framing issues. Here are two examples of Stage 4
Stage 4 Reasoning, Example 1
Interviewer (I): “Can you say that one point of view is
better and another worse?”
Respondent (R): “No, I really can’t on this issue. It de
pends on your beliefs since there is no way of proving
either one.
I: “Can you say that one is more accurate than the
R: “No, I can’t. I believe they’re both the same as far as
Stage 4 Reasoning, Example 2
I: “Can you say one view of creation is right and one is
R: “No, because no one can prove how the world was
created or how man evolved. Scientists can get close to
it—an actual answer. When it comes right down to it,
as to the actual change, they don’t know because they
can’t draw a straight relationship between apes and
man. There isn’t a straight relationship … ”
In quasi-reflective reasoning, the link between gathering evi
dence and making a conclusion is tenuous; this link becomes
explicit in reflective thinking, the third level of the RJM.
Summary of Reflective Judgment Stages
Prereflective Thinking Quasireflective Thinking Reflective Thinking
Stage 1 Stage 4 Stage 6
View of knowledge: Knowledge is assumed to exist
absolutely and concretely; it is not understood as an
abstraction. It can be obtained with certainty by direct
View of knowledge: Knowledge is uncertain, and knowledge
claims are idiosyncratic to the individual because situational
variables (such as incorrect reporting of data, data lost over
time, or disparities in access to information) dictate that
knowing always involves an element of ambiguity.
View of knowledge: Knowledge is constructed into individual
conclusions about ill-structured problems on the basis of
information from a variety of sources. Interpretations that are
based on evaluations of evidence across contexts and on the
evaluated opinions of reputable others can be known.
Concept of justification: Beliefs need no justification because
there is assumed to be an absolute correspondence between
what is believed to be true and what is true. Alternate beliefs
are not perceived.
Concept of justification: Beliefs are justified by giving
reasons and using evidence, but the arguments and choice of
evidence are idiosyncratic (e.g., choosing evidence that fits
an established belief).
Concept of justification: Beliefs are justified by comparing
evidence and opinion from different perspectives on an issue
or across different contexts and by constructing solutions
that are evaluated by criteria such as the weight of the
evidence, the utility of the solution, or the pragmatic need
for action.
I know what I have seen.”“I would be more inclined to believe evolution if they had
proof. It is just like the pyramids: I do not think we will ever
know. Who are you going to ask? No one was there.
It is very difficult in this life to be sure. There are degrees of
sureness. You come to a point at which you are sure enough
for a personal stance on the issue.
Stage 2 Stage 5 Stage 7
View of knowledge: Knowledge is assumed to be absolutely
certain or certain but not immediately available. Knowledge
can be obtained directly through the senses (as in direct
observation) or via authority figures.
View of knowledge: Knowledge is contextual and subjective
because it is filtered through a person’s perceptions and
criteria for judgment. Only interpretations of evidence,
events, or issues may be known.
View of knowledge: Knowledge is the outcome of a process
of reasonable inquiry in which solutions to ill-structured
problems are constructed. The adequacy of those solutions is
evaluated in terms of what is most reasonable or probable
according to the current evidence, and it is reevaluated when
relevant new evidence, perspectives, or tools of inquiry
become available.
Concept of justification: Beliefs are unexamined and
unjustified or justified by their correspondence with the
beliefs of an authority figure (such as a teacher or parent).
Most issues are assumed to have a right answer, so there is
little or no conflict in making decisions about disputed
Concept of justification: Beliefs are justified within a
particular context by means of the rules of inquiry for that
context and by context-specific interpretations of evidence.
Specific beliefs are assumed to be context specific or are
balanced against other interpretations, which complicates
(and sometimes delays) conclusions.
Concept of justification: Beliefs are justified probabilistically
on the basis of a variety of interpretive considerations, such
as the weight of the evidence, the explanatory value of the
interpretations, the risk of erroneous conclusions,
consequences of alternative judgments, and the interrelations
of these factors. Conclusions are defended as representing
the most complete, plausible, or compelling understanding
of an issue on the basis of the available evidence.
If it is on the news, it has to be true.”“People think differently and so they attack the problem
differently. Other theories could be as true as my own, but
based on different evidence.
One can judge an argument by how well thought-out the
positions are, what kinds of reasoning and evidence are used
to support it, and how consistent the way one argues on this
topic is as compared with how one argues on other topics.
TABLE 1 (Continued)
Prereflective Thinking Quasireflective Thinking Reflective Thinking
Stage 3
View of knowledge: Knowledge is assumed to be absolutely
certain or temporarily uncertain. In areas of temporary
uncertainty, only personal beliefs can be known until
absolute knowledge is obtained. In areas of absolute
certainty, knowledge is obtained from authorities.
Concept of justification: In areas in which certain answers
exist, beliefs are justified by reference to authorities’ views.
In areas in which answers do not exist, beliefs are defended
as personal opinion because the link between evidence and
beliefs is unclear.
When there is evidence that people can give to convince
everybody one way or another, then it will be knowledge;
until then, it is just a guess.
Note. From Developing Reflective Judgment (p. 14–15), by P. M. King and K. S. Kitchener, 1994, San Francisco: Jossey-Bass. Copyright 1994 by John Wiley & Sons, Inc. Reprinted with permission.
Reflective thinkers consistently and comfortably use evi
dence and reason in support of their judgments. They argue
that knowledge claims must be understood in relation to the
context in which they were generated, but that they can be
evaluated for their coherence and consistency with available
information. Because new data or new perspectives may
emerge as knowledge is constructed and reconstructed, indi
viduals using assumptions of reflective thinking remain open
to reevaluating their conclusions and knowledge claims.
Stage 7 Reasoning, Example 1
I: “Can you ever say you know for sure?”
R: “It’s [the view that the Egyptians built the pyramids]
very far along the continuum of what is probable.
I: “Can you say one is right and one is wrong?”
R: “Right and wrong are not comfortable categories to
assign to this kind of item—more or less like or reason
able—more or less in keeping with what the facts seem
to be.
Stage 7 Reasoning, Example 2
R: “It’s my belief that you have to be very skeptical
about what you read for popular consumption … even
for professional consumption.
I: “How do you ever know what to believe?”
R: “I read widely of many points of view. Partly
[it’s] reliance on people you think you can rely on, who
seem to be reputable journalists, who make measured
judgments. Then reading widely and estimating where
the reputable people line up or where the weight of the
evidence lies.
These examples illustrate the kinds of developmentally or
dered differences in the way people reason about ill-struc
tured problems that are described in the RJM.
Theoretical Assumptions Underlying the RJM
We turn next to one of the questions that is the focus of this
volume, the paradigmatic assumptions underlying each the
ory of personal epistemology. Because research on the RJM
spans more than 25 years, we will introduce the theoretical
assumptions from both historical and contemporaneous per
Developmental traditions.
The RJM evolved out of a
careful examination of the few models of late adolescent and
adult intellectual development that existed in the late 1970s.
Our initial conceptualization (K. S. Kitchener & King, 1981)
was grounded in the cognitive-developmental tradition of
Piaget (1965; Piaget & Inhelder, 1969) and Kohlberg (1969).
Other developmental theorists in this tradition whose work
informed our early conceptualization of the RJM were Perry
(1968, 1970), Broughton (1975, 1978), Loevinger (1976),
and Harvey, Hunt, and Schroder (1961). The cognitive-de
velopmental tradition has much in common with more recent
constructive-developmental perspectives (e.g., Fischer &
Pruyne, 2002; Kegan, 1982, 1994). What these two ap
proaches share are (a) the underlying assumption that mean
ing is constructed, (b) the emphasis on understanding how in
dividuals make meaning of their experiences, and (c) the
assumption that development (not just change) occurs as
people interact with their environments. Another central de
fining feature is that patterns of meaning-making are de
scribed in developmental terms, that is, the frameworks peo
ple use for interpreting their experiences (e.g., categories and
organizing principles) are described as becoming more com
plex, integrated, and complete over time. These changes do
not occur automatically but rather through interaction with
an environment that both challenges and supports growth.
However, our data led us to reject two well-known as
sumptions espoused by prominent theorists from this tradi
tion. First, unlike Piaget, we do not assume that cognitive de
velopment is best measured by deductive reasoning, nor do
we assume that it is complete with the emergence of formal
operations at age 16 (indeed, our data show that this is not the
case). And in contrast to Kohlberg, we do not claim
cross-cultural universality, and we endorse Rest’s (1979)
concept of a complex rather than a simple stage model of de-
Stage theory.
At the time the RJM was being devel-
oped in the1970s, Rest (a faculty member who supervised
our initial research) was also working within the cogni-
tive-developmental tradition. As a researcher of moral devel-
opment, he was beginning to raise questions about the ade-
quacy of what he called the “simple stage” model being
advanced by Kohlberg (1969), a critique Rest (1979) later
published in his first book. We took the opportunity to ask
similar questions based on our initial study (K. S. Kitchener
& King, 1981), such as whether there was stage variability or
consistency among an individual’s responses. Our scoring
procedures were intentionally designed to allow for this
question to be tested (i.e., allowing raters to record multiple
stages if several were apparent in a given interview protocol).
We found that Rest’s alternative, the “complex stage” model
of development, provided a good explanatory framework for
our data. That is, we observed that development in reasoning
about ill-structured, controversial problems has stage-like
properties, but not that it evolves in a lock-step,
one-stage-at-a-time fashion. Hence, we refer to the major
categories of thinking and interrelated clusters of assump
tions as stages, but our use of this term is qualified, based on
specific assumptions and definitions that fall outside more
traditional usage. Below, we offer data illustrating develop
ment across stages that support this approach.
We acknowledge that stage models within the cogni
tive-developmental tradition have been criticized as provid
ing inadequate conceptual frameworks for describing devel
opment (e.g., Flavell, 1971). Two underlying assumptions
about stages (traditionally defined) have drawn considerable
criticism. The first is that individuals utilize only one orga
nizing framework (stage) at a time and, therefore, that devel
opment from stage to stage is abrupt with any overlap be
tween stages occurring briefly only during transitions
between stages. At the time of stage consolidation, stage us
age is assumed to peak at 100%, consistent with the common
phrase used when referring to stage theories, being “in a
stage. The second criticism is that the stages constitute an in
variant sequence that exists across all cultures. Kohlberg’s
(1969, 1984) claim to the universality of his sequence of
stages of moral development was based on his refutation of
moral relativism as an inadequate philosophical framework
(Kohlberg, 1991) and on cross-cultural studies indicating
that the pattern of development he proposed was also appar
ent among individuals across several cultures. We do not
make these claims. We do, however, support the other claims
within this tradition (that meaning is constructed, that these
constructions are developmentally ordered, and that develop
ment is the result of person-environment interactions).
Rest’s (1979) complex stage model better captures the na
ture of development of reflective judgment because it ac-
counts for the observed patterns in data gathered using the
Reflective Judgment Interview (RJI). For example, it is com-
mon to find an individual who relies heavily on Stage 4 as-
sumptions while reasoning about a controversial problem,
but who also makes statements that are consistent with Stage
3 and Stage 5 assumptions. By contrast, someone who relies
heavily on Stage 2 assumptions rarely uses assumptions of
any stage higher than Stage 3. As Rest noted, this approach
suggests a “much messier and complicated picture of devel-
opment” (p. 65) than does a simple stage approach.
Does the complex stage model proposed by Rest accu
rately capture reflective judgment data? We examined vari
ability of scores of those in our 10-year longitudinal sample
(described later in the review of RJM research) to answer this
question. In only two cases were the RJI ratings limited to a
single stage; in the vast majority of cases, a subdominant
score was assigned, and this was almost always an adjacent
stage. In a small proportion of cases, more than two stage
scores were assigned. Wood (1997) examined the variability
of RJI scores using data from 15 studies for which raw data
were available (n = 1,995 problem scores; reported in Wood,
1993). He constructed a “percent stage utilization score”
based on all responses across the four problems; this score in
dicated the proportion of time each stage was assigned. He
then calculated a series of spline regressions (Darlington,
1990), which predicted stage utilization on the basis of over
all RJI score. (A graph of these may be found in King &
Kitchener, 1994, Figure 6.2.) Here, development is pictured
as a series of uneven, overlapping waves, where usage of
given stage assumptions rises and falls in different propor
tions over time. As this figure shows, for those whose modal
score was Stage 2, 70% of the ratings were for Stage 2, with
less than 20% at Stage 3. About two-thirds of the ratings were
at Stage 3 for those with a modal Stage 3 rating, with the re
mainder fairly equally distributed between Stages 2 and 4. A
similar pattern was obtained for those with a modal Stage 4
rating; here, the remaining ratings were split fairly equally
between Stages 3 and 5. However, the shape of the “wave”
was much flatter for Stage 5, with only about half of the rat
ings at Stage 5; the remainder were spread two stages higher
and lower than the mode. The shape of the curve for Stage 6
was more similar to those for Stages 3 and 4. In other words,
variability in reasoning across stages was the norm and not
the exception in these ratings. No individuals evidenced
non-adjacent utilization patterns (3/5, 4/6, etc.). This evi
dence is consistent with the assumptions of complex stage
theory (Rest, 1979) and adds further evidence that character
izing individuals as being “in” or “at” a single stage is mis
leading. Based on these patterns, King, Kitchener, and Wood
(1994) suggested that development in reflective thinking be
characterized as
waves across a mixture of stages, where the peak of a
wave is the most commonly used set of assumptions. While
there is still an observable pattern to the movement between
stages, this developmental movement is better described as
the changing shape of the wave rather than as a pattern of uni-
form steps interspersed with plateaus. (p. 140)
This shift from simple to complex stage theory represents a
radical change in how development is conceptualized; in-
deed, it may be considered a change of paradigmatic propor-
tions within stage theory.
Skill theory.
The second theoretical model that has af
fected our thinking about RJM research is Fischer’s skill the
ory. Fischer and his colleagues (Fischer, 1980; Fischer, Bull
ock, Rosenberg, & Raya, 1993; Fischer & Lamborn, 1989;
K. S. Kitchener & Fischer, 1990) identified seven develop
mental levels that emerge between ages 2 and 30. These lev
els are divided into two overlapping tiers, the representa
tional tier and the abstract tier. The focus of the
representational tier is on individuals’ ability to manipulate
concrete representations, objects, people, or events; the focus
of the abstract tier is on individuals’ ability to integrate, ma
nipulate, and reason using abstract concepts. This portion of
skill theory has much in common with Kegan’s (1982, 1994)
theory of the development of mature capacity toward self-au
thorship. The upper levels of Fischer’s model also have much
in common with the RJM; in fact, the seven stages of the
RJM can be readily mapped onto Representational Levels
1–4 and Abstract Levels 2–4 (Fischer & Pruyne, 2002; King,
1985; K. S. Kitchener, 2002; K. S. Kitchener & Fischer,
1990). Reflective thinking requires the ability to think ab
stractly, which explains the correspondence between the ab
stract levels of skill theory and Stages 4–7. K. S. Kitchener
(2002) also suggested that skill theory provides a framework
for comparing the multiple models of folk epistemology (R.
Kitchener, 2002) and personal epistemology, such as those in
Hofer and Pintrich’s (1997, 2002) comprehensive reviews.
Another important and influential aspect of Fischer’s
work is his assumption that no skills exist independent of the
environment and that the skill levels a person demonstrates
will vary depending on the conditions under which they are
assessed. (Notably, the acknowledgement that performance
varies with task demands is incompatible with the simple
stage assumption that individuals are “in” one stage at a
time.) Fischer and his colleagues (Fischer & Pipp, 1984;
Lamborn & Fischer, 1988) posited that variability in individ
uals’ responses across tasks reflects the degree of “contextual
support” (e.g., memory prompts, feedback, opportunity to
practice) available at the time of the assessment. He sug
gested that tasks that require performance without support
elicit a person’s functional level capacity, but that tasks that
provide contextual support can elicit performance at levels
that are closer to the upper limit of the person’s cognitive ca
pacity, called optimal level. Contextual support can be pro
vided by offering participants a high-level example of the
skill, the opportunity to ask questions about the example, the
chance to practice the skill in a variety of settings, and so on:
It is the emergence of this general capacity [for abstract
thinking] that establishes an upper limit on the level of inde-
pendent functioning an individual can potentially achieve in
reflective thinking or other domains involving advanced ab-
stract thinking. This upper limit of skill development is
termed the optimal level. (Fischer & Pruyne, 2002, p. 169;
italics in original)
The space between functional and optimal levels is called
one’s “developmental range” and reflects the range of skills
that an individual can access and produce depending on the
circumstances. That is, the nature of the person’s experi
ence—including the structure of the learning and assessment
tasks—affects where within this developmental range a per
son’s performance will fall. If courses and other opportuni
ties for student learning do not provide contextual support for
developing the skills associated with forming abstract con
cepts like reflective thinking (a criticism commonly levied at
both schools and colleges), students will be more likely to
perform at functional rather than optimal levels. Further,
those who have access to higher levels of development would
also have access to a larger repertoire of responses from
which to choose, explaining Fischer’s (1980; Kitchener &
Fischer, 1990) hypothesis that optimal and functional level
will diverge to a greater degree as the person approaches
higher levels of development.
Fischer (1980; Kitchener & Fischer, 1990) also hypothe
sized that functional level performance would improve in a
slow, steady fashion, resulting in a gradual, even slope if
graphed over time. By contrast, he hypothesized that optimal
level performance would be less even and instead be charac
terized by spurts at given age levels, followed by plateaus
between spurts. Thus, researchers would expect different de
velopmental trajectories depending on whether their mea
sures yield data on functional or optimal level performance.
The ability to operate at an optimal level is influenced not
only by support and practice, but also by changes in brain ac
tivity and the reorganization of neural networks (Fischer &
Pruyne, 2002; Fischer & Rose, 1994). The emergence of ab
stractions and reflective thinking appears to involve brain de
velopment that does not occur until late adolescence and
early adulthood.
As this brief summary shows, skill theory provides an inno
vative approach to the study of human development in general
and the development of reflective thinking (with its grounding
in epistemic cognition) in particular. For example, the concept
of developmental range provides an alternative way of address
ing the question of being “in” or “at” a single stage on the RJM,
and a way of targeting educational interventions to students’ de
velopmental levels. Further, its differentiation of functional and
optimal level suggests the need to analyze measures of cognitive
development or personal epistemology for degree of contextual
support, and to develop measures that assess both levels. And
although skill theory is certainly consistent with the person-en-
vironment interaction assumptions inherent in the cognitive-de-
velopmental paradigm, it specifies particular environmental
variables (e.g., contextual support) that appear to affect how stu-
dents learn to engage in the production of more advanced be-
haviors (here, reflective thinking).
Measuring Reflective Judgment
Over the last 25 years, we have experimented with several as-
sessment procedures to measure reflective judgment and its
underlying epistemic assumptions. In order to illustrate the
links between theoretical assumptions stemming from our
research paradigm and our assessment approaches, we de
scribe how the development of several assessment proce
dures was grounded in theoretical considerations.
The Reflective Judgment Interview.
The RJI was
initially designed to measure reflective thinking as described
by the RJM and to inform theory development. We used an it
erative process between theory development and assessment
(“boot-strapping”) for much of the first decade of research on
the RJM, moving back and forth between theory develop
ment and validation efforts. The RJI uses a semistructured in
terview format to elicit responses from participants regarding
how they reason about ill-structured problems. A trained and
certified interviewer asks a series of predetermined but
open-ended questions regarding their reasoning in order to
get at their fundamental assumptions concerning knowledge
and how it is gained. The original interview consisted of four
controversial problems (the accuracy of news reporting, the
creation of human beings, the safety of chemical additives to
foods, and the building of the Egyptian pyramids). A di
lemma on the safety of nuclear power was added for the
10-year longitudinal retest, and several discipline-specific
dilemmas have been used in subsequent studies (business,
chemistry, and psychology). The RJI also includes a stan
dardized series of probe questions; each question is designed
to elicit comments that reflect individuals’ epistemic as
sumptions (specifically, their assumptions about knowledge,
how it is gained, how they decide what to believe). Probe
questions ask about the basis for their point of view, the cer
tainty with which they hold that view, whether differing opin
ions on the topic are right or wrong or better or worse, and
how it is possible people (including experts) disagree about
the topic. The one-hour interview was designed to yield a
picture of how people approach the task of knowing and
making judgments about controversial intellectual issues by
looking at ways they understand and make meaning of con
cepts such as evidence, differences of opinion, uncertainty,
and interpretation. (For a detailed description of the RJI, see
King & Kitchener, 1994, Chapter 5 and Resource A.)
The RJI is scored by trained and certified raters using the
Reflective Judgment Scoring Rules (K. S. Kitchener & King,
1985). Consistent with the complex stage assumptions noted
above, raters can assign three scores to each dilemma to reflect
whatever characteristics of reasoning they observein the inter-
view; these typically range across two adjacent stages. The
stage most clearly or frequently observed is coded first as the
dominant stage, followed by the subdominant stage(s). Occa-
sionally, one dilemma includes statements that reflect three
different stages; this is rare, but recording all three is on option
available to raters if they determine that this best captures the
reasoning in the interview. The point here is that scoring is de-
signed to reflect whatever stage-characteristic responses are
evident in the transcript and not to assume a priori that consis
tency (or inconsistency) will be observed. Assigned scores are
then weighted across dominant and subdominant stages, and
an overall dilemma score is calculated. The training and certif
ication programs for interviewers and raters were put into
place to assure comparability across studies and researchers.
(We have recently discontinued these programs in order to fo
cus on the development of other measures.)
Fischer’s (Fischer & Pruyene, 2002; Kitchener & Fischer,
1990) differentiation of functional and optimal levels of per
formance raised several questions for research on the RJM.
In particular, it called for the consideration of the level of per
formance characterized by the RJI: because no contextual
support is offered, the RJI may be considered a measure of
functional level. As such, it may underestimate a person’s ca
pacity to engage in reflective thinking, yielding a score at the
lower rather than the higher end of the individual’s develop
mental range. Implications of this insight for research and
teaching are explored later in the article.
The Prototypic Reflective Judgement Interview
As noted earlier, there are several points of corre
spondence between Fischer’s skill theory and the RJM. In
order to evaluate whether the developmental patterns
Fischer (1980) had predicted occurred for the development
of reflective thinking as defined by the RJM (K. S.
Kitchener & Fischer, 1990), a measure of optimal level per
formance was needed. Fischer argued that optimal mea
sures required two qualities: first, there had to be an inde
pendent assessment for each step in the developmental
sequence. Second, the research design had to vary relevant
characteristics of the participants (especially age) as well as
characteristics of the environment (e.g., the amount of envi
ronmental support provided for a response). Using these
criteria, K. S. Kitchener, Lynch, Fischer, and Wood (1993)
designed a new measure, the prototypic reflective judgment
interview (PRJI) to assess reflective judgment under condi
tions of support and practice. The study measured both
functional level (using the RJI) and optimal level (using the
PRJI) to determine whether scores differed between the two
measures of reflective thinking and whether there was evi
dence for age-related spurts and plateaus using the optimal
level measure. Because of its theoretical significance, this
study is summarized here.
To construct the PRJI, two problems from the RJI were
selected, and stage-prototypic responses were written for
reflective judgment Stages 2 through 7. These responses
were based on answers given by people for the same prob-
lems used in the RJI; they were presented in order to give
contextual support for high-level responses to the reflective
judgment problems. Participants were first asked to com-
plete the RJI, then to read one of the prototypic statements,
and to respond to a series of questions that directed their at-
tention to key elements of the statement. They were then
asked to explain the prototypic statement in their own
words; each answer was scored as a “hit” if it accurately
paraphrased the statement and as a “miss” if it did not. This
procedure was repeated for each reflective judgment stage
and both of the problems. They were then given two proto
typic statements addressing a different ill-structured prob
lem; the statements were selected to correspond with the
highest and second highest stages they had paraphrased in
the interview. Participants were asked to think about these
statements prior to the next testing, which took place two
weeks later; this strategy provided contextual support in the
form of exposure to and the opportunity to think about
higher stage responses to ill-structured problems. The pro
cedure was repeated at the next testing, fulfilling the prac
tice component of contextual support.
Three findings from the study are relevant for the current
discussion of how stage theory and skill theory as comple
mentary theoretical models have informed research on the
RJM. First, participants scored higher on the PRJI than on
the RJI. This supports the idea that individuals are not “in”
a stage but rather that they have access to several stages,
and that this reflects the effects of contextual support and
practice. In other words, contextual support appears to in
crease the individuals’ access to higher stage functioning,
yielding more advanced levels of performance (here, higher
reflective thinking scores). Second, there was an age-re
lated ceiling in the PRJI even after practice, suggesting that
optimal levels are age-related. This is consistent with the
developmental trends observed in RJM research (described
later) but offers new information about the nature and limits
of age-related trends. Third, there was evidence of age-re
lated developmental spurts on the PRJI at reflective judg
ment Stages 4 to 6. This is consistent with Fischer’s hy
pothesis that the emergence of optimal levels is marked by
spurts in performance and then plateaus. It also helps ex
plain the growth in reflective thinking that has been ob
served with samples of traditional-age college students
(whether this growth is consistent with collegiate goals is
discussed later).
The Reasoning About Current Issues Test
Although the RJI and PRJI provided extremely
rich information for theory development, their expense both
in time and money was problematic in terms of conducting
the kind of validation and application studies that were of in
terest to researchers and educators alike. In the process of de-
veloping an objectively scored measure of reflective think-
ing, we developed and tested several different approaches;
these are described by Wood, Kitchener, and Jensen (2002).
Here, we discuss the most recent measure, the Reasoning
About Current Issues Test (RCI). This is an objectively
scored instrument that was built on research using prior mea-
sures, but using a format that is amenable to large-scale ad-
ministration. Because this is described elsewhere, the focus
of this discussion is on ways our theoretical assumptions
guided measurement development.
The RCI was modeled after Rest’s Defining Issues Test
(DIT) of moral judgment (Rest, 1979, 1986; Rest, Narvaez,
Bebeau, & Thoma, 1999). The DIT has been found to be a
highly reliable measure of moral judgment, able to detect
developmental change over time, and sensitive to
macrolevel changes in reasoning about social issues among
adults. (For reviews of research using the DIT among col
lege students, see King & Mayhew, 2002, in press.) In the
RCI, respondents are asked to read a dilemma similar to
those used in the RJI. In addition to the chemicals in foods
dilemma, several others have been used that reflect contem
porary issues (causes of alcoholism, workforce preparation,
immigration policy, determinants of sexual orientation).
The RCI first asks respondents to write a short statement
describing their response in their own words. These written
statements served to “prime the pump” by encouraging re
spondents to start thinking about their views on the given
topics. Respondents are then asked to rate and rank in order
a series of short statements to indicate the statements’ simi
larity to the respondents’ own views; each statement re
flects the epistemic assumptions of one of the reflective
judgment stages.
Each statement was based on responses made by respon
dents taking the RJI and modified from the statements devel
oped for the PRJI. By merit of being a recognition task, this
format provides contextual support (in contrast to the RJI).
These brief statements are not written to capture an individ
ual’s whole network of underlying assumptions on which a
judgment is based, nor to yield a nuanced articulation of how
the individual approaches making judgments about contro
versial issues. Rather, it appears that responding to short
items serves to activate the internal organizing schemas that
individuals use to make judgments about the given issue, but
without filling in the details about the specific rationale used
and strategies employed, and without articulating the specific
epistemic principles underlying the approach. (For a more
detailed description of this rationale as applied to the DIT,
see Rest et al., 1999.)
In addition, we wished to control for the possibility that
the respondents would endorse statements that sounded im
pressive (e.g., that used sophisticated vocabulary) but that
were not similar to the approach they used, or even to an
approach they aspired to use. To address this concern, we
created a series of statements that are grammatically correct
but nonsensical. When these are selected, the responses for
that problem are excluded from the analyses. The RCI
score is calculated across all dilemma topics based on the
statements most often ranked as similar to the participant’s
own view. Internal consistency reliabilities have been in the
low to mid-.70s (depending on the sample). It takes 30 to
45 minutes to complete (Wood et al., 2002).
There are many trade-offs to be made when moving from
a production task with open-ended questions (the RJI) to a
recognition task where respondents are asked to choose from
a limited set of predefined options (the RCI). Although both
approaches are designed to tap into related skills required for
the production of reflective thinking, the two are not simply
different formats that yield comparable scores; rather, each
serves a different purpose, makes different demands on re
spondents, and yields a different “snapshot” of the develop
ment of reflective thinking. Having participants evaluate
statements provides more contextual support than respond
ing to open-ended interview questions; therefore, we would
anticipate that individuals would score higher on the RCI
than the RJI, and this has been the case.
Research on the RJM
In the last 25 years, we have learned a great deal about re
flective thinking and how it develops. The centerpiece of
our book, Developing Reflective Judgment (King &
Kitchener, 1994), is a review of this research base. It re
ports both the results of our 10-year longitudinal study of
the development of reflective judgment using three age/ed
ucational level cohorts (n = 80 at Time 1), other longitudi
nal studies of 120 other respondents, as well as a review of
cross-sectional studies in which more than 1,700 people
(high school students, college students, graduate students,
and nonstudent adults) completed the RJI. Since the publi
cation of that volume, Wood (1997) has completed a com
prehensive secondary analysis of these data, and an updated
literature review was published (King & Kitchener, 2002).
Interested readers should consult these works for details.
Here, we will summarize the major findings from this body
of research, especially as they pertain to the focus of this
special issue, how our theoretical framework has guided re
search on reflective judgment.
Validating the developmental sequence.
RJM was
proposed as a model of reflective thinking in the cogni
tive-developmental tradition, where the major claim is that
the stages constitute a developmental sequence. Docu
menting the existence of this sequence and validating the
model requires longitudinal data. Toward this end, we con
ducted a 10-year longitudinal study using three age/educa
tional cohorts (n = 80 at Time 1, n = 53 at Time 4). At Time
1, the three gender-balanced cohorts included high school
juniors, college juniors, and third-year doctoral students;
the younger two groups were matched to the doctoral stu
dents on gender and academic aptitude (based on scores
from the Minnesota Scholastic Aptitude Test). This was de-
signed as a check of the competing hypothesis that obtained
cohort differences on the RJI (e.g., if graduate students
scored higher than college students) could be attributed to
differences in aptitude. Even with this control, age and edu-
cational level remained confounded in this study. By the
time of the last testing, all but one of the high school cohort
had completed a bachelor’s degree, and about half of the
college cohort had completed post-baccalaureate degrees.
This yielded a well-educated sample and served as a level
ing factor for the age/educational level confounding at
Time 1.
Mean RJI scores were significantly different between
groups at Time 1 (1977), with the doctoral students scoring
the highest (M = 5.67), followed by the college students (M =
3.76), and the high school students (M = 2.77). The RJI mean
score increased consistently for the high school and college
student groups at each subsequent testing (1979, 1983, and
1987). Over the 10-year period, the former high school stu
dents’RJI scores increased over 2.5 stages to 5.29, the former
college students’ scores rose an average of 1.29 stages to
5.05, and the mean scores of the former doctoral students in
creased an average of .54, to 6.21. The overall rate of increase
(less than two stages in two years) suggests that reflective
thinking evolves slowly and steadily, even among those en
gaged in postsecondary education.
To determine whether a developmental sequence existed,
we carefully examined the pattern of scores over time—and
even reprinted the list of individual RJI scores in their en
tirety for other researchers to examine (King & Kitchener,
1994, Table B6.2). We found that during the 10 years of
this study, the use of higher stage reasoning increased, and
the use of lower stage reasoning decreased. This is evident
in the individual examples reported by King and Kitchener
(1994). For example, listed below are RJI scores for three
individuals (a high school student, #105; a college student,
#417; and a doctoral student, #310) at Times 1 (1977), 2
(1979), 3 (1983), and 4 (1987). The mean score is given
first, followed by the mode (dominant/subdominant stage
105, T1: 3.17; 3(4); T2: 3.96; 4(3); T3: 5.67; 6(5);
T4: 6.59; 7(6)
417, T1: 3.92; 4(3); T2: 3.63; 4(3); T3: 4.54; 4(5);
T4: 4.96; 5(6/4)
310, T1: 5.29; 5(6); T2: 5.63; 6(5); T3: 5.71 6(5);
T4: 6.02; 6(7/5)
Both types of analyses offer strong support for the claim that
the posited reflective judgment stages form a developmental
Similar patterns of change were obtained in six other lon
gitudinal studies involving an additional 180 individuals who
took the RJI (Brabeck & Wood, 1990; Polkosnik & Winston,
1989; Sakalys, 1984; Schmidt, 1985; Van Tine, 1990; Welfel
& Davison, 1986) and ranging in duration from 3 months to 4
years. The most noteworthy finding among these studies is
that the pervasive pattern is one of growth or stability. As
King and Kitchener (1994) reported, “in every sample tested,
the scores either stayed the same or increased over time. Fur-
ther, with two exceptions, the mean score increased signifi-
cantly for all groups tested at 1- to 4-year intervals” (p. 156).
The amount of change was smallest in studies of short dura-
tion (3–4 months); significant increases were consistently
observed in studies of at least a year’s duration. In studies re
porting incidence of regressions (Brabeck & Wood; King &
Kitchener; Sakalys; Schmidt; Welfel & Davison), 0–16% of
the mean scores declined between testings, while 84–100%
of the mean scores either remained consistent or increased.
This suggests that change in reflective thinking over time is
better reflected as stability or development rather than de
cline, and that earlier stage assumptions are rarely used once
they are replaced with more advanced assumptions.
Similarly, longitudinal data based on RCI scores obtained
at the beginning and end of the first year of college yielded
significant increases of about one third of a standard devia
tion, with comparable gain scores by gender and ethnicity
(K. S. Kitchener, Wood, & Jensen, 2003). These freshmen
were tested again as sophomores, and a sample of juniors was
retested as seniors; RCI scores again increased significantly
over time. These findings provide additional evidence that
the RJM describes a developmental sequence. However, the
growth correlation coefficient was significantly and nega
tively correlated with scores at Time 1 for the entire sample,
by class and by gender: Those who had the lowest scores at
Time 1 gained the most, and those who entered with the high
est scores gained the least. The pattern was similar for a small
subset of the participants who also completed the RJI (K. S.
Kitchener et al., 2003).
Differences by age/educational level.
Another de
sirable characteristic of a model of reflective thinking is that
it can detect predictable changes in thinking across educa
tional levels (e.g., that graduate students score higher than
undergraduate students). Over two dozen cross-sectional
studies have been used to examine educational level differ
ences in reflective judgment; these studies include samples
of high school students, traditional- and nontraditional-age
college students, graduate students, and nonstudent adults.
Questions related to educational level differences have been
of particular interest to those interested in using reflective
thinking as a college outcomes variable. Because promoting
intellectual development (and especially skills associated
with complex reasoning) is a common goal of higher educa
tion, studies documenting complex reasoning among college
students have been of interest among many higher education
researchers. As these studies have been summarized else
where, we present only a brief review of these findings.
King et al. (1994) reviewed 25 studies in which more than
1,500 respondents from across the United States took the RJI.
Student RJI scores increased slowly but steadily across educa-
tional levels, from high school (M = 3.2) to the first year of col-
lege (M = 3.6) to the senior year of college (M = 4.0) to early
graduate study (M = 4.6) to advanced doctoral study (M = 5.3).
The average RJI scores for nonstudent adults with and without
collegedegreeswere 4.3 and 3.6, respectively. The high school
students consistently evidenced the assumptions associated
with prereflective thinking, such as making decisions on the
basis of beliefs that are not subject to evaluation, especially
when a conclusion wasconsistent with what theywantedto be
lieve. Among the college samples, the shift to Stage 4 reason
ing indicates that the students had accepted uncertainty as part
of the knowing process and were using evidence more consis
tently to make judgments. Kroll (1992) eloquently captured
the shift from prereflective to quasi-reflective thinking as the
movement from “ignorant certainty” (the dogmatic assertions
characteristic of prereflective thinking) to “intelligent confu
sion” (acknowledging what you don’t know, and why). Al
though this represents an important step toward reflective
thinking, it is not the kind of thinking that is consistent with in
tended college outcomes (Brabeck, 1983; King, 1992). Only
advanced doctoral students consistently used the assumptions
of reflective thinking.
A similar pattern of findings was reported based on stud
ies that used the RCI. Wood, Kitchener, and Jensen (2003)
conducted a meta-analysis based on all available studies us
ing RCI data; this yielded a sample of 8,537 students who
were enrolled in college, graduate, and professional pro
grams at seven different colleges or universities. They found
significant differences by educational level, even after con
trolling for academic aptitude and prior academic achieve
ment. Graduate students scored significantly higher than did
medical students, who scored significantly higher than did
undergraduate students (p < .001). Among the undergraduate
students, significant differences were found between early
level college students (freshmen and sophomores) and more
advanced students (seniors). Thus, the educational level dif
ferences in reflective thinking that were found using the RJI
were also found using the RCI; however, scores by educa
tional level were about one stage higher on the RCI than on
the RJI.
Data from cross-sectional studies showing upward trends
in reflective judgment scores across age/educational levels
offer corroborating evidence that the RJM describes a devel
opmental sequence. In addition, this collection of studies (es
pecially those that controlled for age) offers evidence that de
velopment in reflective thinking is associated with
participation in educational programs.
Domain specificity.
Do individuals reason using simi
lar sets of epistemic assumptions across domains? That is, do
respondents score similarly or differently when reasoning
about controversies of different content? We have analyzed
score variability using several indices. Internal consistency,
as measured by coefficient alpha, has been high, with the me-
dian scores in the low .80s (King & Kitchener, 1994).
Inter-dilemma correlations have been lower, varying with the
heterogeneity of scores in the sample, typically in the mid
.40s. King, Kitchener, Wood, and Davison (1989) examined
individual modal RJI scores and found that the modal score
was consistent across dilemmas 75% of the time. However,
Wood et al. (2003) reported a significant main effect for di-
lemma topic using the RCI, as well as an interaction of topic
by education level. Students in all four collegiate class levels
(freshman through senior) plus graduate students tended to
score higher on the two psychology dilemmas (origins of al
coholism and of homosexuality) than on the other three (arti
ficial sweeteners, curricular reform, and immigration pol
icy); however the class difference was accounted for by the
higher scores of the seniors and graduate students. That is,
the magnitude of the dilemma differences was more pro
nounced for the more advanced students. Whether this is an
artifact of sampling (e.g., representation of behavioral sci
ence majors among the seniors) is not known. Interestingly,
the scores within the two sets of dilemmas (psychology and
nonpsychology) were quite similar.
These findings suggest that there is a relatively high rate
of consistency in people’s use of epistemic assumptions
when reasoning about ill-structured problems. This could be
because the RJM describes development in molar rather than
fine-grained terms, and therefore is less sensitive to differ
ences in dilemma content. Alternatively, it may be that
epistemic assumptions themselves provide a guiding frame
work for making interpretive judgments that individuals use
across a variety of problems such as those measured by the
RJI and RCI.
Another way to consider questions related to domain
specificity is to look at whether people reason similarly in
terms of reflective thinking as compared with how they rea
son about issues in other areas. Data from several studies that
have examined this question (reviewed in King & Kitchener,
1994) strongly suggest that development in reflective judge
ment is related to but distinct from development in other as
pects of cognitive development (verbal aptitude, formal oper
ations, academic ability, critical thinking) and from moral
and identity development, and strongly predictive of toler
ance for diversity (Guthrie, King, & Palmer, 2000).
How can educators apply their understanding of the nature of
the development of reflective thinking as described by the
RJM to educational practice? The theory and research pre
sented here offer many possibilities for answering this ques
tion. First, the strong effects associated with education offer a
hopeful sign that the educational experiences for many stu-
dents are effective in promoting growth toward reflective
thinking. However, the nature of these practices remains
largely unexplored, and there is considerable concern (e.g.,
Baxter Magolda, in press; King, 1992) that the observed re-
flective thinking skills are not as developed as those called
for in the national reports mentioned at the beginning of this
article. Nor are they at the level consistent with college goals
for students, nor at the level associated with the complex is-
sues and decisions college students will face upon gradua-
tion, whether as employees, citizens, consumers, or parents.
Second, consider the consistent finding that development
in reflective thinking appears to unfold in a slow, steady man
ner following the sequence of stages outlined in the RJM.
Without data on specific educational experiences affecting
this growth curve, it is reasonable to assume that theoreti
cally grounded interventions would yield increases in perfor
mance, but probably not in dramatic proportions. Given that
stage assumptions are organizing categories for viewing
knowledge and knowing, and given that each stage is more
like a molar than a fine-grained unit of analysis for develop
ment, slow, steady progress is a more reasonable expectation;
after all, each stage is a dramatic shift in world view and
one’s role as a knower. We have offered a number of sugges
tions elsewhere for promoting reflective thinking (King,
1992, 2000; King & Kitchener, 1994, 2002; K. S. Kitchener
et al., 2003; Wood & Lynch, 1998). These range from inten
tionally incorporating ill-structured problems into the curric
ulum to improving discipline-specific contextual support, to
structuring opportunities for practice and feedback to stimu
late optimal level thinking. In each of these practices, stu
dents are encouraged to examine their assumptions, gather
and interrogate the available evidence from multiple perspec
tives, and be responsible for offering their own conclusions
of the evidence.
Third, it is noteworthy that virtually all the studies that
comprise the database for the RJM have measured functional
level, not optimal level. In only one study (Kitchener et al.,
1993) did the measure of reflective judgment offer contextual
support, and probably not at a level that would elicit perfor
mance at the upper reaches of a participant’s developmental
range. According to skill theory, functional level measures
offer a low estimate of individuals’ability to engage in reflec
tive thinking. If the average educational level scores are low
estimates, then the concerns indicating deficits in student
performance around reflective thinking may be overstated.
Kroll (1992) also discouraged educators from directing their
efforts toward a student’s average performance; instead, he
encouraged teachers to focus on the leading edge of develop
ment, which would be at a higher level within the student’s
developmental range. Similarly, the finding of differences in
performance with and without contextual support suggests
that educators should be encouraged to evaluate the amount
and kind of contextual support they offer when assessing re
flective thinking, for example, in student papers.
K. S. Kitchener et al. (2003) provided new information
on the role of student involvement in an assortment of cam-
pus activities in promoting reflective thinking. In addition
to the RCI, they also administered the College Student Ex-
periences Questionnaire (CSEQ; Pace, 1990), which asks
students to indicate on a 4-point scale how frequently they
participated in particular collegiate activities. Findings from
the freshman sample highlight the complexity of the rela-
tionship between participation in college activities and
epistemological thinking. Predictably, those who entered
college with higher reflective judgment scores also gradu
ated with higher scores. For freshmen, the relationship be
tween Time 1 and Time 2 scores are consistent with expec
tations about students with higher and lower reflective
scores scores: Those who entered with higher scores en
dorsed an appreciation for challenging courses, a willing
ness to work harder in classes, and a commitment to think
ing through ideas themselves. They expressed enthusiasm
for being in college and indicated an appreciation for the
scientific method and further growth in understanding of
science. By contrast, the amount of growth in RCI scores
for freshmen was almost always negatively correlated with
the educational college activities on the CSEQ, including
almost all items having to do with seeking out experiences
that were different from prior experiences, or seeking out
others who were different from themselves. That is, those
who relied on prereflective assumptions were less open to
experiences that involved talking about different points of
view or interacting with others who are different. These
students simply may not seek out these experiences as fre
quently as students who enter with more advanced
epistemological assumptions. Studies such as this that link
types of collegiate experiences to patterns of college stu
dent growth would be particularly helpful in advancing our
understanding of the mechanisms of development in reflec
tive thinking. Baxter Magolda’s (1999, 2001) pedagogical
framework for promoting development offers a promising
conceptual tool for designing interventions to promote not
only reflective thinking but also advanced capacities in
identity and interpersonal domains (for examples in higher
education contexts, see Baxter Magolda & King, in press).
Educators have often reported that they are puzzled by
how students defend their beliefs—for example, why some
reduce complex controversies to simple, black-and-white
terms, and why others are so appreciative of the value of mul
tiple perspectives that they are unable to make their own
judgments. Our hope here is that educators can better inter
pret their observations about student behaviors by under
standing how such behaviors are grounded in their epistemic
assumptions, and how these assumptions about knowledge
and how it is gained are related to the ways students justify
their own judgments about controversial issues.
American Association of Colleges and Universities (2002). Greater expec-
tations: A new vision for learning as a nation goes to college. National
Panel Report. Washington, DC: Author.
American Association of Higher Education, American College Personnel
Association, National Association of Student Personnel Administrators
(1998). Powerful partnerships. Washington, DC: Author.
American College Personnel Association (1994). The student learning im-
perative. Washington, DC: Author.
Baxter Magolda, M. B. (1999). Creating contexts for learning and self-au-
thorship: Constructive-developmental pedagogy. Nashville, TN:
Vanderbilt University Press.
Baxter Magolda, M. B. (2001). Making their own way: Narratives for trans
forming higher education to promote self-development. Sterling, VA: Sty
lus Publishing.
Baxter Magolda, M. B. (in press). Self-authorship as the common goal of
21st century education. In M. B. Baxter & P. M. King (Eds.), Learning
partnerships: Educating for self-authorship. Sterling, VA: Stylus Publi
Baxter Magolda, M. B., & King, P. M. (Eds.). (in press). Learning part
nerships: Educating for self-authorship. Sterling, VA: Stylus Publica
Brabeck, M. (1983) Critical thinking skills and reflective judgment develop
ment: Redefining the aims of higher education. Journal of Applied Devel
opmental Psychology, 4(1), 23–34.
Brabeck, M., & Wood, P. K. (1990). Cross-sectional and longitudinal evi
dence for differences between well-structured and ill-structured problem
solving abilities. In M. L. Commons, C. Armon, L. Kohlberg, F. A. Rich
ards, R. A. Grotzer, & J. D. Sinnott (Eds.), Adult development 2: Models
and methods in the study of adolescent and adult thought (pp. 133–146).
New York: Praeger.
Broughton, M. (1975). The development of natural epistemology in years 11
to 16. Unpublished doctoral dissertation, Graduate School of Education,
Harvard University, Cambridge, MA.
Broughton, M. (1978). Development of concepts of self, mind, reality, and
knowledge. In W. Damon (Ed.), Social cognition. New directions for child
development (Vol. 1, pp. 75–100). San Francisco: Jossey-Bass.
Churchman, C. W. (1971). The design of inquiring systems: Basic concepts
of systems and organization. New York: Basic Books.
Darlington, R. B. (1990). Regression and linear models. New York:
Dewey, J. (1933). How we think: A restatement of the relations of reflective
thinking to the educative process. Lexington, MA: Heath.
Fischer, K. W. (1980). A theory of cognitive development: The control and
construction of hierarchies of skills. Psychological Review, 87, 477–531.
Fischer, K. W., Bullock, D., Rosenberg, E. J., & Raya, P. (1993). The dynam
ics of competence: How context contributes directly to skill. In R.
Wozniak & K. W. Fischer (Eds.), Development in context: Acting and
thinking in specific environments (pp. 93–117). Hillsdale, NJ: Lawrence
Erlbaum Associates, Inc.
Fischer, K. W., & Lamborn, S. D. (1989). Mechanisms of variation develop
mental levels: Cognitive and emotional transitions during adolescence. In
A. de Ribaupierre (Ed.), Transitions mechanisms in child development:
The longitudinal perspective (pp. 33–670). Cambridge, England: Cam
bridge University Press.
Fischer, K. W., & Pipp, S. L. (1984). Processes of cognitive development:
Optimal level and skill acquisition. In R. J. Sternberg (Ed.),
of cognitive development (pp. 45–80). New York: Freeman.
Fischer, K. W., & Pruyne, E. (2002). Reflective thinking in adulthood:
Emergence, development, and variation. In J. Demick & C. Andreoletti
(Eds.), Handbook of adult development (pp. 169–198). New York: Ple
Fischer, K. W., & Rose, S. P. (1994). Dynamic development of coordination
of components of brain and behavior. In G. Dawson & K. W. Fischer
(Eds.), Human behavior and the developing brain (pp. 2–66). New York:
Flavell, J. (1971). Stage related properties of cognitive development. Cogni-
tive Psychology, 2, 421–453.
Guthrie, V. L., King, P. M., & Palmer, C. J. (2000). Cognitive capabilities un-
derlying tolerance for diversity among college students. Manuscript sub-
mitted for publication.
Harvey, L. J., Hunt, D., & Schroder, H. M. (1961). Conceptual systems and
personality organization. New York: Wiley.
Hofer, B. K. & Pintrich, P. R. (1997). The development of epistemological
theories: Beliefs about knowledge and knowing and their relation to learn-
ing. Review of Educational Research, 67, 88–140.
Hofer, B. K., & Pintrich, P. R. (2002). Personal epistemology: The psychol
ogy of beliefs about knowledge and knowing. Mahwah, NJ: Lawrence
Erlbaum Associates, Inc.
Kegan, R. (1982). The evolving self. Cambridge, MA: Harvard University
Kegan, R. (1994). In over our heads: The mental demands of modern life.
Cambridge, MA: Harvard University Press.
King, P. M. (1985). Choice-making in young adulthood: A develop
mental double-bind. Counseling and Human Development, 18(3),
King, P. M. (1992). “How do we know? Why do we believe?” Learning to
make reflective judgments. Liberal Education, 78(1), 2–9.
King, P. M. (2000). Learning to make reflective judgments. In M. B. Baxter
Magolda (Ed.), Linking student development, learning, and teaching:
New directions for teaching and learning (Vol. 82, pp. 15–26). San Fran
cisco: Jossey-Bass.
King, P. M., & Kitchener, K. S. (Eds.). (1994). Developing reflective
judgment: Understanding and promoting intellectual growth and
critical thinking in adolescents and adults. San Francisco:
King, P. M., & Kitchener, K. S. (2002). The reflective judgment model:
Twenty years of epistemic cognition. In B. K. Hofer & P. R. Pintrich
(Eds.), Personal epistemology: The psychology of beliefs about knowl
edge and knowing (pp. 37–61). Mahwah, NJ: Lawrence Erlbaum Associ
ates, Inc.
King, P. M., Kitchener, K. S., & Wood, P. K. (1994). Research on the reflec
tive judgment model. Developing reflective judgment: Understanding
and promoting intellectual growth and critical thinking in adolescents
and adults. San Francisco: Jossey-Bass.
King, P. M., Kitchener, K. S., Wood, P. K., & Davison, M. L. (1989). Re
lationships across developmental domains: A longitudinal study of in
tellectual, moral and ego development. In M. L. Commons, J. D.
Sinnott, F. A. Richards, & C. Armon (Eds.), Adult development: Vol. 1.
Comparisons and applications of developmental models (pp. 57–72).
New York: Praeger.
King, P. M., & Mayhew, M. J. (2002). Moral judgement development in
higher education: Insights from the Defining Issues Test. Journal of
Moral Education, 33, 247–270.
King, P. M., & Mayhew, M. J. (in press). Theory and research on the devel
opment of moral reasoning among college students. Handbook of Higher
Kitchener, K. S. (1983). Cognition, metacognition and epistemic cognition:
A three-level model of cognitive processing. Human Development, 4,
Kitchener, K. S. (2002). Skills, tasks, and definitions: Discrepancies in the
understanding and data on the development of folk epistemology. New
Ideas in Psychology, 20, 309–328.
Kitchener, K. S., & Fischer, K. (1990). A skill approach to the development
of reflective thinking. In D. Kuhn (Ed.), Contributions to human develop
ment: Developmental perspectives on teaching and learning thinking
skills (Vol. 21, pp. 48–62). Basel, Switzerland: Karger.
Kitchener, K. S., & King, P. M. (1981). Reflective judgment: Concepts of
justification and their relationship to age and education. Journal of Ap
plied Developmental Psychology, 2(2), 89–116.
Kitchener, K. S., & King, P. M. (1985). The reflective judgment scoring
manual. Unpublished manuscript.
Kitchener, K. S., Lynch, C. L., Fischer, K. W., & Wood, P. K. (1993). Devel-
opmental range of reflective judgment: The effect of contextual support
and practice on developmental stage. Developmental Psychology, 29(5),
Kitchener, K. S., Wood, P. K, & Jensen, L. (2003). Individual differences in
gains in reflective judgment and their relationship to college experiences.
University of Denver, CO.
Kitchener, R. (2002). Folk epistemology: An introduction. New Ideas in
Psychology, 20(2/3), 89–105.
Kohlberg, L. (1969). Stage and sequence: The cognitive-developmental ap-
proach to socialization. In D. Goslin (Ed.), Handbook of socialization the
ory and research.(pp. 347–450). Chicago: Rand McNally.
Kohlberg, L. (1984). The psychology of moral development: The nature and
validation of moral stages. San Francisco: Harper & Row.
Kohlberg, L. (1991). My personal search for universal morality. In L.
Kuhmerker (Ed.), The Kohlberg legacy for the helping professions. Bir
mingham, AL: R.E.P. Books.
Kroll, B. (1992). Teaching hearts and minds: College students reflect on the
Vietnam War through literature. Carbondale, IL: Southern Illinois Univer
sity Press.
Lamborn, S. D., & Fischer, K. W. (1988). Optimal and functional levels in
cognitive development: The individual’s developmental range. Newsletter
of the International Society for the Study of Behavioral Development,
2(Serial No. 14), 1–4.
Loevinger, J. (1976). Ego development: Conceptions and theories. San
Francisco: Jossey-Bass.
Pace, R. (1990). College Student Experience Questionnaire: Norms for the
third edition. Bloomington, IN: Center for Postsecondary Research and
Planning, Indiana University.
Perry, W. (1968). Patterns of development in thought and values of students
in a liberal arts college: A validation of a scheme (Final Report, Project
No. 5-0825, Contract No. SAE-8973). Washington, DC: Department of
Health, Education, and Welfare.
Perry, W. (1970). Forms of intellectual and ethical development in the col
lege years: A scheme. Troy, MO: Holt, Rinehart & Winston.
Piaget, J. (1965). The moral judgment of the child. New York: Free Press.
Piaget, J., & Inhelder, B. (1969). The psychology of the child. New York: Ba
sic Books.
Polkosnik, M. C., & Winston, R. B. (1989). Relationships between students’
intellectual and psychological development: An exploratory investiga
tion. Journal of College Student Development, 30(1), 10–19.
Rest, J. R. (1979). Development in judging moral issues. Minneapolis: Uni
versity of Minnesota Press.
Rest, J. R. (1986). Moral development: Advances in research and theory.
New York: Praeger.
Rest, J. R., Narvaez, D., Bebeau, M., & Thoma, S. (1999). Postconventional
moral thinking: A neo-Kohlbergian approach. Mahwah, NJ: Lawrence
Erlbaum Associates, Inc.
Sakalys, J. (1984). Effects of an undergraduate research course on cognitive
development. Nursing Research, 33, 290–295.
Schmidt, J. A. (1985). Older and wise? A longitudinal study of the impact of
college on intellectual development. Journal of College Student Person
nel, 26, 388–394.
Van Tine, N. B. (1990). The development of reflective judgment in adoles-
cents. Dissertation Abstracts International, 51, 2659.
Welfel, E. R., & Davison, M. L. (1986). How students make judgments: Do
educational level and academic major make a difference? Journal of Col-
lege Student Personnel, 23, 490–497.
Wood, P. K. (1983). Inquiring systems and problem structure: Implica-
tions for cognitive development. Human Development, 26,
Wood, P. K. (1993). Context and development of reflective thinking: A sec-
ondary analysis of the structure of individual differences. Unpublished
manuscript, University of Missouri at Columbia.
Wood, P. K. (1997). A secondary analysis of claims regarding the Reflective
Judgment Interview: Internal consistency, sequentiality and intra-individ
ual differences in ill-structured problem solving. In J. C. Smart (Ed.),
Higher education: Handbook of theory and research (Vol. 12, pp.
243–312). Edison, NY: Agathon Press.
Wood, P. K., Kitchener, K. S., & Jensen, L. (2002). Considerations in the de
sign and evaluation of a paper-and-pencil measure of epistemic cognition.
In B. K. Hofer & P. R. Pintrich (Eds.), Personal epistemology: The psy
chology of beliefs about knowledge and knowing (pp. 277–294). Mahwah,
NJ: Lawrence Erlbaum Associates, Inc.
Wood, P. K., Kitchener, K. S., & Jensen, L. (2003). College students’ con
cepts of belief justification regarding open-ended controversies. Manu
script submitted for publication.
Wood, P. K., & Lynch, C. L. (1998). Using guided essays to assess and en
courage reflective thinking. Assessment Update, 10(2), 14–15.
... Educational psychologists like Kitchener/King (proposing the seven stage reflective judgment model, RJM, 1994) and Kuhn (1991) mirror Perry's early conceptualizations of intellectual development at different points in the life span or at different educational levels. But even if this research typically was qualitative, mostly based on interviews (King, Kitchener 2004), the focus always was on static positions and not on how different belief dimensions develop over time: There is a lack regarding the portrayal of transitional stages and their driving forces (Hofer 2001;Alexander 2006;Greene et al. 2008). Basically all authors describe intellectual positions from "naïveté" to "sophistication". ...
Full-text available
Do domain specific teaching/learning beliefs and epistemological beliefs exist and do they explain a domain specific approach to teaching and learning the social sciences? This paper reports on the first stage of an exploratory qualitative study carried out at the University of Bielefeld in 2010/11 on pre-service social studies teachers (PSST, n=61). It has a threefold aim: First, providing a very short overview of relevant trends in recent research on epistemological beliefs and teaching/learning beliefs and presenting some plausible hypotheses on if and how these belief dimensions are related to each other in the field of social studies education; second, presenting and discussing metaphors as a suitable qualitative research method for diagnosing and analysing the teaching and learning beliefs in the field; third, presenting the general outline of the Bielefeld project and a comparison of two emblematic cases of the project to evaluate the potential advantages and shortcomings of the research design and methods, especially of metaphor analysis.
... Then, a total of 12 specific processing contents subordinated to the above categories were constructed from three sources. Some specific contents were based on previous studies (i.e., prior investment, past experience, augmentation of scenario, alternatives, avoiding future waste, improvement, personal characteristics, and commitment; Fujino et al., 2020;Strough et al., 2011;Strough et al., 2016), some were added to ensure the integrity of the top-down 3 × 2 theoretical framework (i.e., past emotion, present emotion, and future emotion), and another specific content was included through a bottom-up technique that captured emergent themes in the participants' original responses (i.e., current situation assessment; King & Kitchener, 2004). The category, definition, example, and kappa coefficient of each specific processing content, as well as the entire structure of the coding system of processing contents, are shown in Table 1. ...
Full-text available
Previous studies have found that older adults are less susceptible to sunk costs than younger adults, but the underlying mechanism remains unclear. Recruiting 63 older adults and 60 younger adults, the current study examined the age differences in the sunk cost effect and then explored the potential mediating effect of processing mode and processing contents with the think-aloud technique in half of the participants who assigned to the think-aloud condition. The results showed that the sunk cost effect was smaller in the older adults than in the younger adults. And the processing mode did not mediate such age differences, although intuitive processing was related to a less sunk cost effect and more preferred by older adults. However, older people less frequently processed deliberately while considering past information or present information in the low prior investment scenarios, explaining their decision superiority when faced with sunk costs. From the integrated viewpoint of processing mode and processing contents, the current study proposed potential mental mechanism of the age-related sunk cost effect, which brought inspiration for future practice in the field of sunk-cost decision-making.
In this article, we use grounded practical theory to develop a practical theory for using “difficult data” in Language and Social Interaction university classes. “Difficult data” are transcribed, audio/video-recorded data that contain language and ideologies that could be offensive, bigoted, or otherwise disturbing. We provide contextual literature for understanding the pedagogical tensions around using these data from communication pedagogy research as well as social theory for handling the language within the data. We base our analysis on various types of data from academia: an international/intercultural pedagogical speech event, student surveys, field notes, interview, and recordings of classroom instruction using difficult data. We provide a practical theory focused on techniques for managing the central problem that instructors want students to analyze the social consequences of interaction without replicating the problematic impacts of discourse that make difficult data difficult. Our theory reveals four techniques including framing the data as difficult, surveying students before the class, using already-published research data, and metacommunicating about these dilemmas. These techniques do not promise inviolability, but instructors can foster deliberative reflection/co-orientation about classroom interaction that considers whether the use of difficult data can help students develop analysis-informed praxis.
To help students become better learners, educators need to be able to provide feedback to students on their learning processes. However, this can be challenging for educators due to the customized nature of such feedback. To that end, we have created and evaluated the Acknowledge, Affirm, Challenge, and Encourage (A²CE) framework as a practical and scalable way to provide formative feedback for engineering students. The A²CE framework was: 1) developed through a research project focused on developing metacognitive skills in engineering students, and 2) evaluated for functionality in a workshop setting where we taught educators how to use the A²CE framework. Within the workshop, we collected participants’ feedback on the framework in the form of open-ended surveys. Analysis of responses revealed that workshop participants believed the A²CE framework was useful as a method for having caring conversations with students about their learning, while giving constructive feedback that students can use to improve their learning in and out of the classroom; prior to the workshop, participants indicated a need for more support in having meaningful conversations with their students about learning processes. We conclude that the A²CE framework is an accessible tool educators can leverage to help students improve as learners.
This paper deals with an empirical study about the way information is used by 12–15 years students when they play Geome, a mixed reality game dedicated to museum school visits. With Geome, the students solve ill-structured problems about environmental issues and they deal with fake news or rumors. The playful learning experience is analyzed from the players’ perspective based on the dimensions of personal epistemology, i.e. thinking and beliefs about the nature of knowledge and knowing. Experiments in the museum have been conducted with 3 classes, and mixed data (audio, video and game interactions) were collected from specific moments of the gameplay and analyzed according to the epistemological dimensions. The verbatim analysis shows that the combination of an ill-structured problem and a ludic scenario may encourage students to actively process information and to develop critical thinking. Thus, we discuss the characteristics of the game's influence on students’ personal epistemology.KeywordsStudents’ personal epistemologyGame-based learningMixed reality gameMuseum school visits
Im Themenheft 1/2022 "Erfassung und Förderung von Reflexionskompetenz in der Lehrerbildung" der Lehrerbildung auf dem Prüfstand ( Reflexionsprozesse spielen bei der Entwicklung professioneller pädagogischer Kompetenz als Brückenfunktion zwischen Theorie und Praxis eine wesentliche Rolle. In Praxisphasen können Reflexionsprozesse eingeübt werden, z. B. wenn zur Erklärung von Unterrichtssituationen Informationen in Form von entsprechenden Literaturquellen genutzt werden. Diese Nutzung lässt sich durch verschiedene Betrachtungsebenen beschreiben, für die in diesem Beitrag ein Kodierschema vorgestellt wird. Die drei Ebenen des vorgestellten Schemas umfassen (1) die Art der genutzten Informationen, (2) die Informationsintegration sowie (3) die Nutzung epistemischer Prozesse zum Umgang mit Informationen. Der vorliegende Beitrag kontrastiert auf dieser Basis 15 Fälle im Sinne eines theoretischen Samplings und stellt beispielhaft drei Fälle mit Auszügen aus Reflexionsessays zur Erklärung einer pädagogischen Situation dar. Es zeigt sich, dass die untersuchten Studierenden vorwiegend Überblicksliteratur und einfache epistemische Prozesse verwenden. Zudem wird deutlich, dass erst ein Zusammenspiel der drei Ebenen zu einem umfassenden Bild des Umgangs mit Literatur beitragen kann. Schlagwörter: epistemische Prozesse – evidenzorientierte Praxis – Informationsintegration – Lehrkräftebildung – Reflexion By bridging the gap between theory and practice, reflection processes are essential in the development of professional pedagogical competence. Reflection processes can be practiced in practical phases, e.g. when information (corresponding literature sources) are used to explain teaching situations. This information use can be described by different levels of consideration, for which a coding scheme is presented in this paper. The three levels of consideration include (1) the type of information used, (2) the integration of information, and (3) use of epistemic processes to deal with information. On this basis, this paper contrasts 15 cases in terms of theoretical sampling and exemplifies three cases with excerpts from the essays explaining a pedagogical situation. It becomes apparent that students predominantly use survey literature and simple epistemic processes. Moreover, it becomes clear that an interplay of the three levels results in a comprehensive picture of how students deal with literature. Keywords: epistemic processes – evidence-oriented practice – multiple source integration – reflection – teacher education
Beliefs play a central role in our lives. They lie at the heart of what makes us human, they shape the organization and functioning of our minds, they define the boundaries of our culture, and they guide our motivation and behavior. Given their central importance, researchers across a number of disciplines have studied beliefs, leading to results and literatures that do not always interact. The Cognitive Science of Belief aims to integrate these disconnected lines of research to start a broader dialogue on the nature, role, and consequences of beliefs. It tackles timeless questions, as well as applications of beliefs that speak to current social issues. This multidisciplinary approach to beliefs will benefit graduate students and researchers in cognitive science, psychology, philosophy, political science, economics, and religious studies.
Beliefs play a central role in our lives. They lie at the heart of what makes us human, they shape the organization and functioning of our minds, they define the boundaries of our culture, and they guide our motivation and behavior. Given their central importance, researchers across a number of disciplines have studied beliefs, leading to results and literatures that do not always interact. The Cognitive Science of Belief aims to integrate these disconnected lines of research to start a broader dialogue on the nature, role, and consequences of beliefs. It tackles timeless questions, as well as applications of beliefs that speak to current social issues. This multidisciplinary approach to beliefs will benefit graduate students and researchers in cognitive science, psychology, philosophy, political science, economics, and religious studies.
Full-text available
Niniejszy artykuł prezentuje badania przeprowadzone w celu uzyskania odpowiedzi na pytanie, w jaki sposób tutorzy (nauczyciele akademiccy) i tutees (studenci) formują osądy na temat kilku trudnych poznawczo problemów. W badaniu skoncentrowano się na opisie intelektualnych procedur stosowanych w refleksyjnym osądzaniu tutorów i tutees. Rozstrzygano, na ile jest to myślenie na poziomie prerefleksyjnym, quasirefleksyjnym i refleksyjnym. Podczas badania tutorzy i tutees byli postawieni wobec kilku niejednoznacznych tematów i mieli za pomocą krótkich odpowiedzi ustosunkować się do własnych źródeł poznania, przekonań i wiedzy. Badanym zadawano serię pytań skoncentrowanych wokół pięciu tematów, np. stosowania leków w terapii zaburzeń psychicznych; przyczyn alkoholizmu. Badania zostały przeprowadzone za pomocą pisemnego kwestionariusza zainspirowanego Wywiadem osądzania refleksyjnego (Reflective Judgment Interwiew), opracowanego przez King i Kitchener w oparciu o Model osądzania refleksyjnego. Model zakłada trzy stadia myślenia: prerefleksyjne, quasirefleksyjne i refleksyjne, podzielone na siedem etapów. Analiza wyników badań pozwoliła ustalić to z jakich źródeł korzystają badani uzasadniając swe sądy, na ile są pewni swojej wiedzy, czy dopuszczają inne punkty widzenia. Tutorzy najczęściej prezentowali stadium refleksyjne, a tutees stadium quasi-refleksyjne. Ustalono jednak, że poziom refleksyjności niektórych badanych spośród tutees zmieniał się w zależności od rozważanego zagadnienia, natomiast w grupie tutorów nie zaobserwowano istotnych różnic w myśleniu reflek- syjnym po zmianie tematu. Dodatkowe analizy z uwzględnieniem płci i kierunku studiów nie wykazały znaczących różnic międzygrupowych w myśleniu studentów. Badani tutorzy i tutees okazali się grupą niezwykłą, kompetentną i wyrafinowaną w zakresie rozwiązywania trudnych poznawczo problemów. Rekomendowane jest zindywidualizowane podejście do edukacji studentów poprzez stosowanie metody tutoringu.
The main objective of this paper is to examine how teacher reflection could result in possible change in practice. The essential value of teacher reflection is to think about improvements to be made for practice. This paper adapted Ward and McCotter’s reflection rubric into a Teacher Reflection survey scale to explore and examine the varying years of teaching experience, teaching level and designation on teachers’ teaching practices. A total of 436 primary and secondary school mathematics teachers took part in an online survey. Confirmatory factor analyses were conducted to validate the survey instruments. Findings suggest that there were no major differences in teacher reflection based on their years of teaching experience, designation, and teaching levels. Correlational, regression, and path analyses of the three dimensions of teacher reflection (i.e. Focus, Inquiry, and Change) and teacher Change Practise revealed interesting associations among these variables. Findings indicate that the Focus dimension is the basis of teacher reflection and from there it extends to the Inquiry and Change dimensions and ultimately to Change Practise. This study suggests that the focus on a transformative mindset based on intentional reflection is a powerful stimulus for change and growth in teaching practice.
A model is said to be linear if the partial derivatives with respect to any of the model parameters are independent of the other parameters. This chapter introduces linear models and regression, both simple linear and multiple regression, within the framework of ordinary least squares and maximum likelihood. Influence diagnostics, conditional models, error in variables, and smoothers and splines are discussed. How to appropriately handle missing data in both the dependent and independent variables is discussed.
In linear regression the mean surface in sample space is a plane. In non-linear regression the mean surface may be an arbitrary curved surface but in other respects the models are similar. In practice the mean surface in most non-linear regression models will be approximately planar in the region(s) of high likelihood allowing good approximations based on linear regression techniques to be used. Non-linear regression models can still present tricky computational and inferential problems. (Indeed, the examples here exceeded the capacity of S-PLUS for Windows 3.1.)