ChapterPDF Available

Educational Design Research


Abstract and Figures

Educational design research is a genre of research in which the iterative development of solutions to practical and complex educational problems provides the setting for scientific inquiry. The solutions can be educational products, processes, programs, or policies. Educational design research not only targets solving significant problems facing educational practitioners but at the same time seeks to discover new knowledge that can inform the work of others facing similar problems. Working systematically and simultaneously toward these dual goals is perhaps the most defining feature of educational design research. This chapter seeks to clarify the nature of educational design research by distinguishing it from other types of inquiry conducted in the field of educational communications and technology. Examples of design research conducted by different researchers working in the field of educational communications and technology are described. The chapter concludes with a discussion of several important issues facing educational design researchers as they pursue future work using this innovative research approach.
Content may be subject to copyright.
Chapter 9: Educational Design Research
Susan McKenney
Open University & University of Twente, the Netherlands
Thomas C. Reeves
The University of Georgia, USA
Educational design research is a genre of research in which the iterative development of
solutions to practical and complex educational problems provides the setting for scientific
inquiry. The solutions can be educational products, processes, programs or policies.
Educational design research not only targets solving significant problems facing
educational practitioners, but at the same time it seeks to discover new knowledge that
can inform the work of others facing similar problems. Working systematically and
simultaneously toward these dual goals is perhaps the most defining feature of
educational design research. This chapter seeks to clarify the nature of educational design
research by distinguishing it from other types of inquiry conducted in the field of
educational communications and technology. Examples of design research conducted by
different researchers working in the field of educational communications and technology
are described. The chapter concludes with a discussion of several important issues facing
educational design researchers as they pursue future work using this innovative research
Design research; design-based research; formative research; design experiments
McKenney & Reeves
Educational Design Research
Page 3 of 29
Chapter 9: Educational Design Research
Educational design research is a genre of research in which the iterative development of
solutions to complex educational problems provides the setting for scientific inquiry. The
solutions that result from educational design research can be educational products (e.g. a
multiuser virtual world learning game), processes (e.g. a strategy for scaffolding student
learning in online courses), programs (e.g. a series of workshops intended to help
teachers develop more effective questioning strategies), or policies (e.g. year-round
schooling). Educational design researchers attempt to solve significant real world
problems while at the same time they seek to discover new knowledge that can inform
the work of others facing similar problems. This chapter summarizes arguments and
evidence presented by Barab and Squire (2004), Burkhardt, (2009), Reeves, (2011),
Schoenfeld (2009); van den Akker, Gravemeijer, McKenney, and Nieveen (2006a), and
others that educational design research is an innovative and exceptionally promising
approach to improving the quality and impact of educational research in general, and
educational communications and technology research in particular.
Educational design research origins
Design research is not defined by its methods but by the goals of those who pursue it.
Design research is constituted within communities of practice that have certain
characteristics of innovativeness, responsiveness to evidence, connectivity to basic
science, and dedication to continual improvement.” - Bereiter (2002) p. 321.
McKenney & Reeves
What has prompted scholars around the globe sharing the above-mentioned
characteristics of “innovativeness, responsiveness to evidence, connectivity to basic
science, and dedication to continual improvement” to come together in the pursuit of
educational design research? At least two main motives can be identified. Interestingly,
both perspectives have strong historical ties to educational psychology, and both
perspectives are concerned with making a contribution to educational practice. The first
motive is driven more by what society needs while the second has more to do with
finding adequate methods to meet those needs.
First, stemming from the notion that scientific understanding should be used to solve or at
least gain a better understanding of practical problems, the call for scientific inquiry to
yield what Lagemann (2002) refers to as ‘usable knowledge’ has been present for over a
century. Although this focus on demonstrable impact may be ignored by some who
recommend that educational researchers should emulate the methods of the so-called
“hard sciences” (e.g. physics) that seek knowledge without expectation of practical
application, the expectation for social science research to connect fundamental
understanding with applied use dates back to Münsterberg (1899) and Dewey (1900), if
not earlier. Both of these former American Psychological Association presidents
expressed the need for a linking science, which would use empirical insights and
theoretical advancements to inform problem-solving and improvement initiatives in
practice. This call has been taken up gradually within the fields of education and
psychology, for example in the work of Robert Glaser (1976) who laid out the elements
of a psychology of instruction and called for a science of design in education. Donald
Stokes (1997), an American political scientist, provided a fresh look at the goals of
science and their relation to application to real world problems, in his highly-acclaimed
book titled, Pasteur’s Quadrant: Basic Science and Technological Innovation. Stokes
Educational Design Research
Page 5 of 29
promoted more “use-inspired basic research” akin to work of the French chemist and
microbiologist, Louis Pasteur. He contrasted Pasteur’s pragmatic research approach with
that of the basic science goals of Danish physicist, Niels Bohr, and the applied research
aims of the American inventor, Thomas A. Edison.
Second, educational researchers have been searching for adequate methods to yield the
kinds of empirical insights and theoretical advancements that could be used to address
real concerns in educational practice. Acknowledging the limitations of laboratory
settings, the value of relinquishing control of variables in return for increased ecological
validity of the findings has been gaining support over the last 30 years. In 1992, two
landmark papers were published which are often credited with launching educational
design research as a specific genre of scientific inquiry. Brown’s (1992) article in the
Journal of the Learning Sciences discussed tensions between laboratory studies of
educational innovations and challenges inherent in integrating these innovations into real
world classrooms as background to describing her own design experiments. That same
year, Collins (1992) published a book chapter arguing that education should be viewed as
a design science akin to aeronautics, as opposed on an analytical science similar to
physics, emphasizing the fact that laboratory conditions could rarely approximate
conditions in real classrooms.
By the turn of the millennium, support was increasing for innovative research approaches
that might yield the kind of knowledge that can be put to use for the improvement of
education. Advocates for these new approaches accepted that the kinds of knowledge
needed would have to be constructed in the complex ‘laboratories’ of everyday learning
environments such as classrooms or online courses. The establishment of educational
design research is growing steadily. This momentum became apparent through several
McKenney & Reeves
special issues of highly respected journals, including Educational Researcher (2003,
31(1)), Journal of the Learning Sciences (2004, 13(1)), and Educational Psychologist
(2004, 39(4)). Since then, several books have been written about educational design
research. Books have focused on conceptualization (van den Akker, Gravemeijer,
McKenney, & Nieveen, 2006a) methodological considerations (Kelly, Lesh, & Baek,
2008), and the details of conducting design studies (McKenney & Reeves, in press)
across educational fields. Related volumes have appeared specifically in the domains of
literacy (Reinking & Bradley, 2008) and instructional design (Richey & Klein, 2007). In
addition to special issues and books about educational design research, numerous reports
of educational design research initiatives have been published in research journals such as
Instructional Science (cf. Xie & Sharma, 2011), the Journal of the Learning Sciences
(e.g. Schwarz & Asterhan, 2011), the Journal of Research on Technology in Education
(e.g. Basham, Meyer, & Perry, 2010), and Educational Technology Research and
Development (e.g. Reynolds & Caperton, 2011). In addition, doctoral dissertations using
educational design research have been completed at multiple institutions such as the
University of California, Berkeley (e.g. Brar, 2010), Unversity of Florida (e.g. Drexler,
2010), the University of Georgia (e.g. Oh, 2011), the Pennsylvania State University (e.g.
Lee, 2009), and the University of Twente (e.g. Raval, 2010).
Today we see many sectors within education that seem to embrace educational design
research, including: learning sciences, instructional design, curriculum development and
teacher professional development. While educational design research is not inherently
tied to any specific subject area, much of the work published so far has been related to
science or mathematics, perhaps because more funding has been available for research
related to STEM (science, technology, engineering, and mathematics) disciplines than for
other areas (Kelly et al., 2008). However, educational design research is also being
Educational Design Research
Page 7 of 29
increasingly used in language and literacy research (Reinking & Bradley, 2008), as well
as other disciplines. A wide variety is present across educational design study literature, a
development that is partly accounted for by the methodological traditions within the
various educational sectors, individual researcher preferences and the resources available
for specific projects. In addition, variance across the twofold motives driving educational
design research plays a large role in explaining the diversity of these kinds of studies.
While pursuing both goals simultaneously remains a defining feature of educational
design research, one goal may feature more prominently than the other. For example,
relating more to the motive of improving practice, educational design research may be
conducted primarily to:
- Solve a problem (e.g. increase the participation of women and other minorities in
engineering and science careers),
- Put knowledge to innovative use (e.g. use the affordances of smart phones to
enable mobile learning), and/or
- Increase robustness and systematic nature of design practices (e.g. establish a set
of design principles for implementing inquiry-based learning in middle school
Or, relating more to the motive of enhancing the quality of research findings, educational
design research may be conducted primarily to:
- Generate new knowledge (e.g., develop a theory of game-based learning),
- Generate different types of knowledge (e.g. enhance and extend knowledge
related to professional development for scaffolding strategies for math teachers),
- Increase the ecological validity of research-based knowledge (e.g. increase the
likelihood that educational innovations will be used to transform educational
McKenney & Reeves
Clarifying the nature of educational design research
What is educational design research?
While studies do differ in terms of which motives are more powerful determinants in
shaping the inquiry, educational design research in general distinguishes itself from other
forms of inquiry by attending to both solving problems by putting knowledge to use, and
through that process, generating new knowledge. As stated elsewhere (McKenney &
Reeves, in press), educational design research is a genre of research in which the iterative
development of solutions (e.g. educational products, processes, programs or policies) to
practical and complex educational problems, provides the setting for scientific inquiry,
and yields new knowledge that can inform the work of others. Working systematically
and simultaneously toward these dual goals may be considered the most defining feature
of educational design research.
Educational design research is not a methodology. It uses quantitative, qualitative and –
probably most often – mixed methods to answer research questions. In so doing,
educational design research is held to the same standards as other scientific work when it
comes to providing transparency of the process and adequate warrants for the knowledge
claims it yields (cf. Shavelson, Phillips, Towne, & Feuer, 2003). In addition to the
knowledge generated, the value of educational design research is measured in terms of its
ability to improve educational practice (Design-Based Research Collective, 2003).
How does educational design research compare to other approaches?
While both are concerned with developing new knowledge and are connected to design
processes, educational design research has commonalities but also differences from the
Educational Design Research
Page 9 of 29
instructional design focused design and development research described by Richey and
Klein (2007, and in this volume). If considered as a Venn diagram, educational design
research and design and development research would overlap in projects that are
concerned with actively solving problems in educational practice (e.g. design and testing
of software to help plan lessons). The area that would be unique to design and
development research would be those projects that are concerned with developing tools
or models to support education in the long run, but that do not function as educational
interventions (e.g. retrospective analysis of how instructional designers carry out their
tasks). Design research projects that would not overlap with design and development
research would be those not specifically concerned with advancing the field of
instructional design (e.g. design and testing of a learning sequence for early literacy).
Educational design research is also different from evaluation research (Clarke, 1999),
although formative and summative evaluation methods are among the main vehicles used
to study and fine-tune interventions in both cases. First, problem definition and solution
design are rarely featured in evaluation research. Second, a key difference is that
evaluation research is primarily concerned with evaluating and possibly improving the
qualities of a particular intervention. The broader scientific orientation of generating
usable knowledge (e.g. in the form of models to underpin design, theories about how
teachers learn, descriptions of what engages learners, etc.) is not as overtly present in
evaluation research as in educational design research.
Educational design research also entails more than research-based educational design.
They are both forms of scientific inquiry, and often, each values a rational approach.
They both embrace systems thinking and are both shaped by iterative, data-driven
processes to reach successive approximations of a desired intervention. However,
McKenney & Reeves
research-based educational design focuses solely on intervention development, whereas
design research strives explicitly to make a scientific contribution of value to others
outside the research/design setting in addition to the intervention development. This has
important implications for the entire process. Additional information on these differences
is available in (McKenney & Reeves, in press; Oh & Reeves, 2010). Similarly, action
research (cf. Mills, 2002) also lacks the emphasis on finding robust public knowledge
that is a hallmark of educational design research.
Distinguishing educational design research from other forms of inquiry in education is
made more difficult because it has been referenced in the literature by a number of
different terms such as “design-based research” (cf. Barab & Squire, 2004), “design
experiments” (cf. Brown, 1992), “development research” (cf. van den Akker, 1999),
“formative experiments” (cf. Reinking & Bradley, 2008), “formative research” (cf.
Newman, 1990), and simply “design research” (cf. Kelly et al., 2008). There are subtle
differences in how these terms are used by various researchers as delineated in
McKenney and Reeves (in press). The term “educational design research” is used in this
chapter and elsewhere (cf. Plomp & Nieveen, 2009; van den Akker et al., 2006) because
including the word “educational” in the term helps to avoid confusion with design
research as used in other fields. For example, Laurel’s (2003) book simply titled Design
Research concerns the field of human computer interface design and industrial
engineering rather than education.
Educational Design Research
Page 11 of 29
Conducting educational design research
Characteristics of educational design research have been offered in the literature (Kelly,
2003; Reinking & Bradley, 2008; van den Akker, Gravemeijer, McKenney, & Nieveen,
2006a; Wang & Hannafin, 2005). Common descriptors include: pragmatic, grounded,
interventionist, iterative, collaborative, adaptive and theory-oriented. Educational design
research is pragmatic because it is concerned with generating usable knowledge, and
usable solutions to problems in practice. It is grounded because it uses theory, empirical
findings and craft wisdom to guide the work. It is interventionist because it is undertaken
to make a change in a particular educational context. Educational design research is
iterative because it generally evolves through multiple cycles of design, development,
testing and revision. It is collaborative because it requires the expertise of
multidisciplinary partnerships, including researchers and practitioners, but also often
others (e.g. subject matter specialists, software programmers or facilitators). Educational
design research is adaptive because the intervention design and sometimes also the
research design are often modified in accordance with emerging insights. Finally, it is
theory-oriented not only because it uses theory to ground design, but also because the
design and development work is undertaken to contribute to a broader scientific
There is no set process for conducting the ‘manifold enterprise’ (Bell, 2004, p. 245) of
educational design research. This approach to inquiry is rich with variation in terms of
McKenney & Reeves
models and frameworks that describe, and in a few cases, guide the process. Across that
variation, some similarities can be identified:
- Educational design research uses scientific knowledge (and to varying degrees,
also other kinds of knowledge such as craft wisdom) to ground design work;
- Educational design research produces scientific knowledge (and in some cases,
also craft wisdom among the participants);
- Though the terminology and contents differ, three phases can be distinguished in
educational design research: an analysis/orientation phase; a design/development
phase; and an evaluation/retrospective phase; these are often revisited in the
lifespan of a project; and
- Educational design research strives to develop both interventions in practice and
reusable knowledge.
Rich variation
Thought-provoking differences in design research are also present. Some of the
differences stem from the units of analysis, scope of implementation, nature of the
subject areas addressed, or from the research domains and methodological traditions in
which studies originate. As mentioned earlier, the relative emphasis on each motive
(solution development, new knowledge or equally on both) can also wield strong
influence on the design research process. But other differences stem from the concerns of
those interpreting the concept and conducting the studies.
McKenney and Reeves (in press) surveyed models for educational design research and, in
addition to highlighting similarities like those mentioned above, noted unique
contributions each one has to offer. The Osmotic Model, offered by Eljersbo et al. (2008),
depicts the parallels of the design cycle and the research cycle. The authors point out that
Educational Design Research
Page 13 of 29
both cycles originate from the problem and would ideally run simultaneously, but state
that this ideal is often not the case. Bannan-Ritland and Baek (2008) developed the
Integrated Learning Design Framework, which depicts four main stages and across those,
14 steps, in a combined approach to research and development. Along with the process
model, guiding questions for research and examples of applicable methods for each main
phase are given. Reeves (2000) presented a minimalist model that highlights four main
phases of design research: problem analysis; solution development; iterative refinement;
and reflection to produce design principles. He compared these phases to the four phases
of predictive research. In contrast to the aforementioned three models, McKenney, van
den Akker and Nieveen (2006) offered a model which is more conceptually-oriented than
process-oriented. This model depicts tenets guiding a research and development cycle,
situated in a particular context, yielding three main outcomes: professional development
of the participants; the designed intervention; and design principles.
In addition to these visual models, Gravemeijer and Cobb (2006) described important
steps in the three main phases of their work: preparing for a design experiment;
conducting a design experiment; and retrospective analysis. Based on a review of
literature, Wang and Hannafin (2005) delineated and argued for nine principles of design-
based research. Finally, Reinking and Bradley (2008) posed six questions as a guide for
conducting formative experiments, relating to: pedagogical goals; classroom intervention;
factors affecting the intervention; modifications to the intervention; unpredicted effects of
the intervention; and changes in the instructional environment due to the intervention.
Based on their survey and analysis of existing models and frameworks for design
research, McKenney and Reeves (in press) created a generic model for design research
(see Figure 1). Through this basic visualization, this model shows only the core elements
McKenney & Reeves
of a flexible process that features the three main stages described earlier, taking place in
interaction with practice and yielding the dual outputs of knowledge and intervention.
Figure 1. Generic model for conducting educational design research (McKenney & Reeves, 2012)
Scientific outputs
Different terms have been used to describe the kinds of theoretical knowledge that are
produced by educational design research (cf. Edelson, 2002; McKenney & Reeves, in
press; van Aken, 2004; van den Akker, 1999). Descriptive, substantive or declarative
knowledge is generated to describe certain phenomena (e.g. what learner behaviors are
triggered by certain prompts). Prescriptive or procedural knowledge is generated to help
inform interventions in practice (e.g. how to facilitate learning through the strategic use
of certain prompt types under certain circumstances). Some projects may develop a
research agenda more attuned to one type of knowledge over another, though eventually
attending to both types seems to be the case more often than not.
Implementation & Spread
Educational Design Research
Page 15 of 29
Different terms have been used in literature to describe the kind of integrated procedural
and declarative knowledge that comes out of design research, but design principles is
probably the most prevalent (cf. Kali, 2008; Kim & Hannafin, 2008; Mishra & Koehler,
2006; Quintana et al., 2004; van den Akker, 1999). Bell, Hoadley, and Linn (2004)
describe design-principles as:
“…an intermediate step between scientific findings, which must be generalized and
replicable, and local experiences or examples that come up in practice. Because of the
need to interpret design-principles, they are not as readily falsifiable as scientific laws.
The principles are generated inductively from prior examples of success and are subject
to refinement over time as others try to adapt them to their own experiences.” (p. 83).
On the other hand, Van den Akker (1999) suggests that the knowledge encompassed in
design principles can be conveyed through heuristic statements, such as, “If you want to
design intervention X [for purpose/function Y in context Z]; then you are best advised to
give that intervention the characteristics C1, C2, …, Cm [substantive emphasis]; and do
that via procedures P1, P2, …, Pn [procedural emphasis]; because of theoretical
arguments T1, T2, …, Tp; and empirical arguments E1, E2, … Eq.” (p. 9).
Complementing these perspectives on design principles, Linn and Elyon (2006) also
describe design patterns, which illustrate promising instructional sequences, and may be
guided or fine-tuned by design principles.
Practical outputs
In educational design research, research and development are integrated to create
educational interventions that address practical problems. In early stages, this involves
analysis of the problem to be addressed. Using the findings from a needs and context
analysis, together with a clarified problem statement, design work commences.
McKenney & Reeves
Depending on the scope of the project, (re-)design work can last from several weeks to
several years. Especially the revisions are fed by field investigations using a range of
strategies and methods to study either the intervention itself (e.g. as a type of intervention
for which guidelines or design frameworks are needed); or phenomena that are
engendered by the interventions (e.g. learner reactions).
Different research reports are used here (Thomas, Barab, & Tuzun, 2009; Klopfer &
Squire, 2008; Oh, 2011) to illustrate the variety of educational design research conducted
within the field of educational communications and technology. One study (Thomas et
al., 2009) was conducted by a research team led by Sasha Barab, one of the most highly
respected senior professors in the field, with substantial funding from the National
Science Foundation and other sources; one study was co-led by an at-the-time early
career assistant professor, Kurt Squire, with start-up funding from Microsoft and other
sources; and the last was carried out by a doctoral student, Eunjung Oh, working with one
other doctoral student and a practitioner with no funding beyond a graduate teaching
assistantship. For each one, the problem addressed, the primary focus of the research, the
intervention that was developed, the theoretical contributions, the methods used, the
scope of the intervention involved as well as its practical contribution are summarized in
Table 1.
Table 1. Three examples demonstrating educational design research variation
Thomas, Barab, &
Tuzun, 2009
Klopfer & Squire,
Oh, 2011
Middle school students
were relatively
unengaged in
meaningful scientific
High school and
college students were
frequent users of
handheld devices such
as smart phones, but
Graduate student
collaboration in online
learning course was
superficial and
Educational Design Research
Page 17 of 29
were not using them to
Main focus
Investigating the
implementation of a
educational innovation
in a public elementary
school in the USA
Developing innovative
applications for mobile
computing for
environmental science
To optimize
collaborative group
work and student
learning in an online
higher education
learning environment
Quest Atlantis: A 3D
Multiplayer Virtual
A series of games that
can be played on
handheld devices such
as PDA and smart
Evaluation” course
based on authentic
tasks for online
created Theory of
Transformational Play
Theoretical framework
called ‘‘augmented
reality educational
Thirty design
principles and
associated strategies to
enhance group work in
online courses
methods used
Document analyses
Three qualitative case
Focus groups
Discourse analysis
Case Studies
Design narratives
Three sequential case
Research scope
This design research
initiative has been
underway for more
than a decade with
substantial funding
from NSF and other
The design research
study has been
underway since 20011
with initial funding
from Microsoft and
other sources.
This study lasted two
years with no direct
As of 2010, Quest
Atlantis had been used
by 50,000 students in
more than a dozen
The work started with
this project is now part
of the Games,
Learning, and Society
group at the University
of Wisconsin where
numerous learning
games can be found.
An online course
design for a graduate
level course based
around authentic tasks
was developed with
substantial support for
group work.
McKenney & Reeves
The three examples described here illustrate how different types of research reports are
published as sub-components of larger educational design research projects. Published in
the Journal of Educational Computing Research, Thomas, Barab, and Tuzun (2009) is
one of a series of journal papers in which Barab and his colleagues have described their
efforts to refine a theory of transformational play while at the same time seeking to
develop advanced forms of interactive learning games. This paper summarizes the results
of three qualitative studies focused on the challenges and successes involved in
implementing Quest Atlantis, a 3D multiplayer virtual environment (MUVE), which
serves as the primary vehicle for instantiating Barab’s transformational play learning
theory and for allowing it to be refined through iterative design-based research.
Published in the Educational Technology Research and Development Journal, Klopfer
and Squire (2008) describe a multi-year project to enhance student learning related to
environmental science through the development and refinement of learning games that
are accessed with handheld devices such as PDAs and smart phones. In addition to
developing an array of learning games, the project has sought to develop and refine a
theoretical framework called ‘augmented reality educational gaming’ that can be applied
by other games designers. The paper provides considerable detail about the development
of the learning games using a unique ‘design narrative’ approach. This particular paper
focuses on iterative design cycles based on five case studies conducted in real high school
Oh (2011) reports the findings of a doctoral dissertation that pursued two primary goals:
1) optimizing collaborative group work in an online graduate level course focused on ‘E-
Learning Evaluation,’ and 2) developing a refined model of group work in online courses
and identifying design principles for supporting online collaborative group work among
Educational Design Research
Page 19 of 29
adult learners The dissertation provides a comprehensive portrayal of a two-year design
research project using what Boote and Beile (2005) called the “compilation of research
articles” (p. 10) format for dissertations. The dissertation includes one published article,
three submitted papers, one detailed methodology chapter, and one detailed results
chapter. Oh (2011) documents how mixed methods were applied across several semester-
length iterations of an online course to yield 30 distinct design principles for supporting
group work by adults.
Addressing inherent challenges
Inspired by van den Akker’s (1999) design research challenges, this section briefly
touches on several important issues that often crop up in educational design research,
how they may be attended to, and areas that require further consideration.
Information richness and efficiency: Seeking a productive balance
When conducting educational design research, it is necessary to address questions about
appropriate tactics for increasing the information richness and efficiency of data
collection procedures and instruments without being over-whelmed with data. Design
researchers should not be driven by the misconception that ‘more is better.’ This notion is
aptly conveyed by Dede, (2004, p. 7) who noted in reference to a design study that
“everything that moved within a 15-foot radius of the phenomenon was repeatedly
interviewed, videotaped, surveyed and so-forth – this elephantine effort resulted in the
birth of mouse-like insights in their contribution to educational knowledge.”
McKenney & Reeves
Optimizing processes: Stacking smaller studies together
Other questions arise around the linkages among design, prototyping, implementation,
data collection, processing, analysis and re-design. Managing the process of
communicating evaluation findings and subsequently utilizing them for improvement of
interventions is difficult. Realistic timelines must be established with allowances for
flexibility. Educational design research projects must inevitably be divided into smaller,
more manageable chunks. These chunks and the smaller studies involved in them can
function as ‘bricks’ in a larger structure that forms both the evolving intervention and the
refined knowledge. Emerging insights can be shared through shorter (e.g. article-sized)
reports of smaller chunks, whereas books or other media might be more appropriate for
sharing new knowledge derived from the whole of long-term efforts. Often, the interim
(i.e. smaller chunk) reporting stands on its own and does not (need to) mention the larger
study; also, interim reporting for an external audience can be a timely vehicle for
fostering reflection among design research team members.
Measuring impact: Powerful examples needed
Ultimately, educational design researchers must address questions regarding the most
relevant indicators of quality, success and impact of the interventions and knowledge
advances that result from their efforts. Burkhardt (2006) writes about what is needed to
bring about greater acceptance of educational design research. He describes several
Nobel Prize winners for design and development in other fields and concludes that
educational design research candidates should be assessed on the basis of their: impact on
practice; contribution to theory and/or knowledge; and improvement in either research
and/or design methodology. While it is surely too early to be expecting Nobel Prizes for
educational design researchers, this approach will only gain wide acceptance when it can
Educational Design Research
Page 21 of 29
be shown to make the much-needed gains in demonstrating the impact educational
research (cf. Kaestle, 1999).
Generalizability: Toward uptake and use of new knowledge
The main conceptual vehicle through which new knowledge is transferred outside of the
research context, generalizability means different things to different researchers. All
researchers must seek to identify promising approaches to enable uptake and use of
research findings. Because educational design research takes place in natural settings
where more variables are present than can be controlled for, the findings from these
studies cannot yield immutable rules, easily transferred without consideration. But they
can yield useful insights to inform the work of others (design work or otherwise). For
example, when designs are tested in multiple settings and under varying conditions, or
when design features are systematically varied under similar conditions, theory
development can occur through analytic generalization. According to Yin (1989, p. 44),
analytic generalization is a process through which "the investigator is striving to
generalize a particular set of results to a broader theory” which can be of use to others.
Alternatively, knowledge produced through design research can be shared and used
through case-to-case generalization. Firestone (1993) refers to case-to-case generalization
as the transfer of ideas that takes place when a person in one setting considers adopting
an intervention, or its underlying propositions and frameworks in another setting. To do
this, the knowledge producer is obligated to explicate how the specific instance studied
compares to other instantiations of the phenomenon. In so doing, description of salient
characteristics of both the intervention and the context in which it is enacted are essential.
Clearly, when it comes to putting the knowledge of design research to use, the knowledge
producer must portray the work well enough. This could mean, for example, adhering to
McKenney & Reeves
Lincoln and Guba’s (1985) criteria for naturalistic inquiry: credibility, transferability,
dependability and confirmability (parallel to internal validity, external validity, reliability
and objectivity, respectively). At the same time, knowledge consumers are obliged to
critically assess the applicability of certain ideas for their own specific contexts.
On the horizon
Educational design researchers and arguably all educational researchers must seek to
balance rigor and relevance (Reeves, 2011). To find this balance, educational design
researchers might do well to learn from sister fields. For example, engineering and
product design tend to embrace creativity more than most educational researchers (e.g.
Laurel, 2003). Another perspective can be found in appreciative inquiry in health care
(e.g. Carter, Ruhe, Weyer, Litaker, Fry, & Stange, 2007) that emphasizes design based on
opportunity, as opposed to patching gaps uncovered by reductionist problem diagnostics.
Since the landmark design research articles in 1992, a growing appreciation for
educational design research in a wide variety of contexts has been evident. Gradually, the
design research literature is beginning to show more consideration of factors that affect
implementation. Instead of tossing innovations over the metaphorical walls of classrooms
and online learning environments, educational design researchers are working hand in
hand with practitioners to conduct design and research in ways that make substantive
change possible. The importance of collaborative approaches and on-the-ground
understanding of implementation issues, which were privileged topics of research in the
1970s (cf. Fullan & Pomfret, 1977; Hall, Wallace & Dossett, 1973; Havelock, 1971)
seem relatively new – but also quite dear – to many of those currently practicing design
research. Some researchers emphasize this perspective by referring to their work as
design-based implementation research (e.g. Penuel, Fishman, Cheng & Sabelli, 2011).
Educational Design Research
Page 23 of 29
We embrace the surge of interest in these concerns, and express our hope for a
renaissance of scholarship that brings researcher and practitioner expertise together to
bear on substantial educational issues. Educational design research is one of several
genres of inquiry that can lead the way in contributing to scientific understanding in the
long term through its study of meaningful implementation in the here and now.
Bannan-Ritland, B., & Baek, J. (2008). Teacher design research: An emerging paradigm for
teachers’ professional development. In A. Kelly, R. Lesh & J. Baek (Eds.), Handbook of
design research methods in education: Innovations in science, technology, engineering, and
mathematics learning and teaching. London: Routledge.
Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. Journal of
the Learning Sciences, 13(1), 1-14.
Basham, J. D., Meyer, H., & Perry, E. (2010). The design and application of the digital backpack.
Journal of Research on Technology in Education, 42(4), 339-359.
Bell, P. (2004). On the Theoretical Breadth of Design-Based Research in Education. Educational
Psychologist, 39(4), 243 - 253.
Bell, P., Hoadley, C., & Linn, M. (2004). Design-based research in education. In M. Linn, E.
Davis & P. Bell (Eds.), Internet environments of science education (pp. 73-85). Mahwah, NJ:
Lawrence Earlbaum Associates.
Bereiter, C. (2002). Design research for sustained innovation. Cognitive Studies, Bulletin of the
Japanese Cognitive Science Society, 9(3), 321-327.
Boote, D. N., & Beile, P. (2005). Scholars before researchers: On the centrality of the dissertation
literature review in research preparation. Educational Researcher, 34(6), 3-15.
Brar, R. (2010). The design and study of a learning environment to support growth and change in
students' knowledge of fraction multiplication. Unpublished doctoral dissertation from the
University of California, Berkeley.
* Brown, A. L. (1992). Design Experiments: Theoretical and Methodological Challenges in
Creating Complex Interventions in Classroom Settings. Journal of the Learning Sciences, 2(2),
Burkhardt, H. (2006). From design research to large-scale impact: Engineering research in
education. In J. van den Akker, K. Gravemeijer, S. McKenney & N. Nieveen (Eds.),
Educational design research. London: Routledge.
Carter, C. A., Ruhe, M. C., Weyer, S. M., Litaker, D., Fry, R. E., & Stange, K. C. (2007). An
appreciative inquiry approach to practice improvement and transformative change in health
care settings. Quality Management in Health Care, 16(3), 194204.
Clarke, A. (1999) Evaluation research: An introduction to principles, methods and practice.
London: Sage.
McKenney & Reeves
* Collins, A. (1992). Toward a design science of education. In E. Lagemann & L. Shulman (Eds.),
Issues in education research: problems and possibilities (pp. 15-22). San Francisco: Jossey-
Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for
educational inquiry. Educational Researcher, 32(1), 5-8.
Dewey, J. (1900). Psychology and social practice. The Psychological Review, 7, 105-124.
Drexler, W. (2010). The networked student: A design-based research case study of student
constructed personal learning environments in a middle school science course. Unpublished
doctoral dissertation from the University of Florida.
Edelson, D. (2002). Design research: What we learn when we engage in design. The Journal of the
Learning Sciences, 11(1), 105-121.
Ejersbo, L., Engelhardt, R., Frølunde, L., Hanghøj, T., Magnussen, R., & Misfeldt, M. (2008).
Balancing Product Design and Theoretical Insight. In A. Kelly, R. Lesh & J. Baek (Eds.), The
Handbook of Design Research Methods in Education (pp. 149-163). Mahwah, NJ: Lawrence
Erlbaum Associates.
* Firestone, W. A. (1993). Alternative arguments for generalizing from data as applied to
qualitative research. Educational Researcher, 22(4), 16-23.
Fullan, M., & Pomfret, A. (1977). Research on curriculum and instruction implementation. Review
of Educational Research, 47(2), 335-397.
Glaser, R. (1976). Components of a psychology of instruction: Toward a science of design.
Review of Educational Research, 46(1), 29-39.
Gravemeijer, K., & Cobb, P. (2006). Outline of a method for design research in mathematics
education. In J. v. d. Akker, K. Gravemeijer, S. McKenney & N. Nieveen (Eds.), Educational
design research. London: Routledge.
Hall, G. E., Wallace, R. C., & Dossett, W. A. (1973). A developmental conceptualization of the
adoption process within educational institutions. Austin, Texas: The Research and
Development Center for Teacher Education.
Havelock, R. (1971). Planning for Innovation through Dissemination and Utilization of
Knowledge. Ann Arbor: Center for Research on Utilization of Scientific Knowledge.
Kaestle, C. F. (1993). The awful reputation of education research. Educational Researcher, 22(1),
23, 26-31.
Kali, Y. (2008). The Design Principles Database as means for promoting design-based research. In
A. Kelly, R. Lesh & J. Baek (Eds.), Handbook of design research methods in education (pp.
423-438). London: Routledge.
Kelly, A. (2003). Research as design. Educational Researcher, 32(1), 3-4.
* Kelly, A., Lesh, R., & Baek, J. (Eds.). (2008). Handbook of design research methods in
education. New York: Routledge.
Kim, H., & Hannafin, M. (2008). Grounded design of web-enhanced case-based activity.
Educational Technology Research & Development 56, 161-179.
Klopfer, E., & Squire, K. (2008). Environmental Detectivesthe development of an augmented
reality platform for environmental simulations. Educational Technology Research and
Development, 56(2), 203-228.
Labaree, D. F. (2006). The trouble with ed schools. New Haven, CT: Yale University Press.
Educational Design Research
Page 25 of 29
Lagemann, E. (2002). An elusive science: The troubling history of education research. Chicago:
University of Chicago Press.
Laurel, B. (2003). Design research: Methods and perspectives. Cambridge, MA: MIT Press.
Lee, J. J. (2009). Understanding how identity supportive games can impact ethnic minority
possible selves and learning: A design-based research study. Unpublished dissertation study
from The Pennsylvania State University.
* Lincoln, YS. & Guba, EG. (1985). Naturalistic Inquiry. Newbury Park, CA: Sage Publications.
Linn, M., & Eylon, B. (2006). Science education: Integrating views of learning and instruction. In
P. Alexander & P. Winne (Eds.), Handbook of educational psychology (2 ed., pp. 511-544).
Mahwah, NJ: Lawrence Erlbaum Associates.
* McKenney, S., & Reeves, T. (2012). Conducting Educational Design Research: What it is, How
we do it, and Why. London: Routledge.
McKenney, S., van den Akker, J., & Nieveen, N. (2006). Design research from the curriculum
perspective. In J. Van den Akker, K. Gravemeijer, S. McKenney & N. Nieveen (Eds.),
Educational design research (pp. 67-90). London: Routledge.
Mills, G. E. (2002). Action research: A guide for the teacher researcher (2nd ed.). Englewood
Cliffs, NJ: Prentice-Hall.
Mishra, P., & Koehler, M. (2006). Technological pedagogical content knowledge: A framework
for teacher knowledge. Teachers College Record, 108(6), 1017-1054.
Münsterberg, H. (1899). Psychology and life. Boston: Houghton Mifflin.
Newman, D. (1990). Opportunities for research on the organizational impact of school computers.
Educational Researcher, 19(3), 8-13.
Oh, E. (2011). Collaborative group work in an online learning environment: A design research
study. Unpublished doctoral dissertation from The University of Georgia.
Oh, E., & Reeves, T. (2010). The implications of the differences between design research and
instructional systems design for educational technology researchers and practitioners.
Educational Media International, 47(4), 263-275.
Quintana, C., Reiser, B., Davis, E., Krajcik, J., Fretz, E., & etc. (2004). A scaffolding design
framework for software to support science inquiry. Journal of the learning sciences, 13(3),
* Penuel, W., Fishman, B., Cheng, B., & Sabelli, N. (2011). Organizing research and development
ad the intersection of learning, implementation and design. Educational Researcher, 40(7),
Plomp, T., & Nieveen, N. (Eds.). (2009). An introduction to educational design research:
Proceedings of the seminar conducted at the East China Normal University, Shanghai.
Enschede, The Netherlands: SLO - Netherlands Institute for Curriculum Institute.
Raval, H. (2010). Supporting para-teachers in an Indian NGO: The plan-enact-reflect cycle.
Unpublished doctoral dissertation from Twente University.
Reeves, T. C. (2011). Can educational research be both rigorous and relevant? Educational
Designer: Journal of the International Society for Design and Development in Education, 1(4).
Available online at:
McKenney & Reeves
Reeves, T. C., Herrington, J., & Oliver, R. (2005). Design research: A socially responsible
approach to instructional technology research in higher education. Journal of Computing in
Higher Education, 16(2), 97-116.
* Reinking, D., & Bradley, B. (2008). Formative and design experiments: Approaches to
language and literacy research. New York: Teachers College Press.
Reynolds, R., & Caperton, I. (2011). Contrasts in student engagement, meaning-making, dislikes,
and challenges in a discovery-based program of game design learning. Educational
Technology Research & Development, 59(2), 267-289.
Richey, R., & Klein, J. (2007). Design and Development Research. London: Lawrence Erlbaum
Schwarz, B. B., & Asterhan, C. S. (2011). E-moderation of synchronous discussions in
educational settings: A nascent practice. Journal of the Learning Sciences, 20(3), 395-442.
Shavelson, R., Phillips, D., Towne, L., & Feuer, M. (2003). On the science of education design
studies. Educational Researcher, 32(1), 25-28.
Stokes, D. (1997). Pasteurs Quadrant: Basic Science and Technological Innovation. Washington,
D.C.: Brookings Institution Press.
Thomas, M. K., Barab, S. A., & Tuzun, H. (2009). Developing critical implementations of
technology-rich innovations: A cross-case study of the implementation of Quest Atlantis.
Journal of Educational Computing Research, 41(2), 125-153.
van Aken, X. (2004). Management research based on the paradigm of the design sciences: The
quest for field-tested and grounded technological rules. Journal of Management Studies, 41(2),
* van den Akker, J. (1999). Principles and methods of development research. In J. van den Akker,
R. Branch, K. Gustafson, N. Nieveen & T. Plomp (Eds.), Design approaches and tools in
education and training (pp. 1-14). Dordrecht: Kluwer Academic Publishers.
van den Akker, J., Gravemeijer, K., McKenney, S., & Nieveen, N. (Eds.). (2006a). Educational
Design Research. London: Routledge.
van den Akker, J., Gravemeijer, K., McKenney, S., & Nieveen, N. (2006b). Introducing
educational design research. In J. Van den Akker, K. Gravemeijer, S. McKenney & N.
Nieveen (Eds.), Educational design research (pp. 3-7). London: Routledge.
Wang, F., & Hannafin, M. (2005). Design-based research and technology-enhanced learning
environments. Educational Technology Research and Development, 53(4), 5-23.
Xie, Y., & Sharma, P. (2011). Exploring evidence of reflective thinking in student artifacts of
blogging-mapping tool: A design-based research approach. Instructional Science, 39(5), 695-
Author Information
Complete name: Susan McKenney
Institutional affiliation: Open University & University of Twente
Educational Design Research
Page 27 of 29
Institutional address: OWK/GW/UTwente; PO Box 217; 7500AE Enschede; the
Permanent address: Beldershoekweg 22; 7548PZ Enschede; the Netherlands
Telephone number: +31 53 489 2890
Fax number: +31 53 489 3759
Email address:
Short biographical sketch: Dr. McKenney is Associate Professor in the Centre for
Learning Sciences and Technologies at the Open University and also within the
Education Department of the Faculty of Behavioral Sciences at the University of Twente.
Her research focuses on understanding and supporting the interplay between curriculum
development and teacher professional development. In various settings, she has
extensively studied the supportive role of technology in curriculum development. Dr.
McKenney is committed to exploring how educational research can serve the
development of scientific understanding while also creating sustainable solutions to real
problems in educational practice. Since educational design research lends itself to these
dual aims, she also works on developing and explicating ways to conduct this exciting
form of inquiry. In addition to authoring numerous journal articles, she co-edited the
book, Educational Design Research in 2006 and, together with Tom Reeves, wrote the
book, Conducting Educational Design Research in 2012. Dr. McKenney and is current
editor of Educational Designer, the journal of the International Society for Design and
Development in Education. Dr. McKenney has received multiple merit-based
scholarships from both the University of Twente and the Dutch Science Foundation
(NWO), as well as from other organizations. Professional organizations in which she is
active include: AERA, EARLI, ICO, ISDDE, ISLS and VOR. Dr. McKenney has taught
McKenney & Reeves
preschool and middle school, as well as bachelor, master and doctorate courses. In
addition to regular conference participation around the globe, she has given invited
speeches in Belgium, China, India, Italy, the Netherlands, South Africa and the USA.
Author Information
Complete name: Reeves, Thomas, C.
Institutional affiliation: The University of Georgia
Institutional address: Dept. of Educational Psychology and Instructional Technology
(EPIT), College of Education, The University of Georgia
Complete mailing address: 604 Aderhold Hall, The University of Georgia Athens, GA
Telephone number: 706-542-3849
Fax number (optional):
Email address:
Website (optional):
Short biographical sketch: Professor Emeritus Thomas C. Reeves taught in the Learning,
Design, and Technology (formerly Instructional Technology) program for nearly 30
years. He continues to work as a consultant and he is a frequent invited speaker around
the globe. He designed, developed and evaluated numerous interactive learning programs
for education and training. He is a past president of the Association for the Development
of Computer-based Instructional Systems (ADCIS) and a former Fulbright Lecturer.
From 1997-2000, he was the editor of the Journal of Interactive Learning Research. He
served as a member of the Scientific Panel on Interactive Consumer Health Information
for the U.S. Department of Health and Human Services and the National Visiting
Committee for the National Science Digital Library. In 2003, he was the first person to
receive the AACE Fellowship Award from the Association for the Advancement of
Educational Design Research
Page 29 of 29
Computing in Education. His Interactive Learning Systems Evaluation book (with John
Hedberg) was published in 2003; his A Guide to Authentic E-Learning book (with Jan
Herrington and Ron Oliver) was published in 2010; and his Conducting Educational
Design Research book (with Susan McKenney) was published in 2012. In 2010, Reeves
was made a Fellow of the Australasian Society for Computers in Learning in Tertiary
Education (ASCILITE). His current research interests include: 1) evaluation of
educational technology, 2) authentic learning tasks, 3) socially responsible research in
education, 4) educational design research, and 5) applications of instructional technology
in developing countries.
... According to Anderson and Shattuck (2012, p. 16), "DBR is a methodology designed by and for educators that seeks to increase the impact, transfer, and translation of education research into improved practice." It also emphasizes the necessity of conceptual or theoretical construction and the development of design principles to guide, inform, and improve practice and research in an educational environment (Armstrong et al., 2022;Barab & Squire, 2004;Design-Based Research Collective, 2003;Lim & Nguyen, 2021;McKenney & Reeves, 2014). DBR (1) aims to solve problems (by designing interventions) in real life and produce new knowledge for others to deal with similar problems; ...
... We adopted a three-phase process DBR (Armstrong et al., 2022;McKenney & Reeves, 2014 in which we analyzed, understood, and defined the problem to be addressed (Armstrong et al., 2022). We conducted a comprehensive literature review in the first phase (analysis and exploration); the review considered relevant readings on language material development and language material evaluation. ...
... We went through three phases of DBR (McKenney & Reeves, 2014, but found they could be further broken down into eight steps (see Figure 5). Analysis was the most fundamental step as it led us to guidance in addressing stakeholders' needs. ...
Teacher educators drive preservice teachers' learning about teaching particularly through feedback. The learning materials produced by preservice teachers also play a crucial role in English language teacher training but may not be receiving the necessary attention in teacher education programs. Preservice teachers lack formal teaching experiences, which may result in their overemphasizing material delivery while underemphasizing material suitability. Providing teacher educators with criteria for evaluating learning materials developed by preservice teachers could aid in their providing quality formative feedback. Therefore, this study was designed to construct a research-based contextualized evaluation checklist for learning materials. Adopting a systematic yet flexible design-based method allowed for the integration of design and intervention to develop a solution for this targeted educational problem. The researchers adopted a three-phase process for this study: analysis and exploration of a teacher educator's needs and extant literature on learning materials; design and construction; and evaluation and reflection. Learning materials designed by a preservice teacher were used to pilot the checklist. The results showed a need for refinement to form the final evaluation checklist. The checklist can be used by teacher educators for preservice teacher evaluation purposes or by preservice teachers as a quality check.
... El DBR, por tanto, "requiere una colaboración intensiva y a largo plazo en la que participen investigadores y profesionales" (Herrington et al., 2007, p. 4089) dentro del contexto y otros expertos en la materia. Se trata de un proceso colaborativo donde el investigador trabaja de forma conjunta con los profesionales que forman parte de ese mismo contexto a lo largo de todas las fases del estudio (Plomp, 2010;Wang y Hannafin, 2005) y reciben apoyo externo de expertos en la materia (Mckenney y Reeves, 2013). ...
... En los estudios de DBR es habitual que los prototipos diseñados sean revisados y/o apoyados por expertos externos al proyecto (Gros, 2019;Mckenney y Reeves, 2013) con el fin de facilitar y mejorar la calidad de los materiales diseñados. Las propuestas didácticas diseñadas en este estudio han sido revisadas y enriquecidas por 5 expertos y 2 expertas pertenecientes a las diferentes áreas de conocimiento que se abordan en el diseño de los materiales: la gramática pedagógica, la GC, procedimientos docentes y el uso de la tecnología en la práctica educativa. ...
Full-text available [spa] Esta investigación expone el proceso de cocreación docente de presentaciones gramaticales (PGs) con tecnología desde una perspectiva cognitiva en la enseñanza de español como lengua extranjera (ELE) en el Instituto Cervantes de Praga (IC de Praga) y tiene dos objetivos: analizar el proceso de diseño de PGs con tecnología y examinar el funcionamiento del equipo de cocreación docente. Para desarrollar este estudio, un equipo de siete profesores del IC de Praga diseñó cuatro PGs con tecnología para el nivel A2 en la enseñanza a adultos en línea siguiendo la metodología de los estudios basados en diseño (Design Based Research, DBR). Así, analizamos los cambios que se producen en el diseño de las PGs durante el diseño inicial (que comprende la ideación, el diseño de prototipos en papel, la revisión de los expertos y el desarrollo tecnológico) y durante los ciclos iterativos de implementación, análisis y rediseño de las cuatro PGs. Para analizar del funcionamiento del equipo nos hemos basado en el marco teórico del Teacher Design Team y hemos examinado el rol del gestor, la interacción del equipo para comunicarse de forma abierta, la apropiación y las emociones. Además, hemos evaluado la incidencia de la cocreación docente en la calidad de los materiales y en el desarrollo profesional de los miembros del equipo. Para alcanzar nuestros objetivos hemos examinado los datos procedentes del análisis de necesidades del contexto, los ocho talleres del equipo, la revisión de los expertos de los prototipos en papel, las iteraciones de las PGs y los cuestionarios y entrevistas de autoevaluación de los miembros del equipo. Además, para examinar la percepción del estudiante, se han analizado los cuestionarios de evaluación de las implementaciones y las entrevistas grupales. Los resultados del proceso de diseño de las PGs demuestran cómo la tecnología puede apoyar la secuencia de presentación de contenidos gramaticales, especialmente durante la fase de descubrimiento guiado. Además, ilustran el poder de la multimodalidad para representar visualmente el significado de las formas gramaticales, así como la necesidad de mantener la coherencia en el diseño y guiar el acceso a los elementos multimodales para facilitar la comprensión de los estudiantes. Asimismo, reflejan la transcendencia de organizar de forma sistemática las actividades de interacción de los estudiantes, y velar por la sencillez y flexibilidad del diseño de los materiales. Por otro lado, en cambio, los resultados reflejan que un uso inadecuado de la tecnología entorpece la enseñanza de la gramática desviando la atención de estudiantes y profesores del objetivo de aprendizaje hacia la tecnología y provocando la descoordinación entre el discurso del profesor y la secuencia de diapositivas. Con respecto al funcionamiento del equipo de cocreación docente, los resultados constatan la necesidad de que el gestor sea transparente, flexible y que transmita al equipo apoyo, confianza y seguridad con el objetivo de involucrar a los profesores y que participen activamente en la toma de decisiones, lo cual contribuye a crear un entorno propicio para el diseño que permite que el equipo funcione de forma efectiva. Adicionalmente, hemos comprobado que para aprovechar al máximo el tiempo disponible durante los talleres y que la interacción sea equilibrada, es necesario definir la organización de las actividades que se realizan dentro y fuera de los talleres. Por último, hemos constatado que la combinación de la metodología de DBR y la cocreación docente ofrece una panorámica muy amplia de qué sucede alrededor del proceso de diseño de materiales en equipo. En concreto, hemos encontrado que la cocreación docente optimiza los procedimientos característicos de la metodología DBR provocando que estos se alternen de forma recursiva a través de la interacción de los miembros del equipo, lo cual ha incidido en la calidad de los materiales y en el desarrollo profesional de los profesores. [eng] This research examines the process of teacher co-creation of grammatical presentations (GPs) with the use of technology in teaching of Spanish as a foreign language (Español como lengua extranjera, ELE) at the Instituto Cervantes de Praga (IC Prague) from a cognitive perspective and has two main objectives: to analyse the process of designing GPs with the use of technology and to examine the functioning of the teacher co-creation team. In this study, a team of seven teachers from IC Prague designed four technology- enabled GPs for adult A2-level students educated online, following the Design Based Research (DBR) methodology. We analysed the changes made in the design of GPs during their initial creation (comprising ideation, paper prototyping, expert review, and technology development) and during the iterative cycles of their implementation, analysis, and redesign. To study how the team works together, we based our analysis on the theoretical framework of the Teacher Design Team and examined the role of the team leader, team's ability to communicate openly, ownership and emotions. In addition, we evaluated the impact of teacher co-creation on the quality of the materials and on the professional development of the individual team members. To achieve our objectives, we have looked at the data from the context needs analysis, eight team workshops, expert review of the paper prototypes, GP iterations, questionnaires and self-evaluation interviews of the team members. In addition, to examine learner perception, implementation evaluation questionnaires and group interviews have been analysed. The results of the GPs creation process demonstrate how technology can support the process of grammatical content presentation, especially in the phase of guided discovery. Furthermore, they illustrate how multimodality can visually represent the meaning of grammatical forms, as well as the need to maintain consistency in design and guide learner’s access to multimodal elements to facilitate their comprehension. They also reflect the importance of systematically organizing interaction activities and ensuring simplicity and flexibility in the design of materials. On the other hand, the results show that inappropriate use of technology hinders the teaching of grammar, diverting the attention of students and teachers from the learning objective to the technology itself and causing a lack of coordination between the teacher's discourse and the sequence of slides. With respect to the functioning of the teacher co-creation team, the results show the team leader must be transparent, flexible and offer support, trust and confidence to the team to achieve teachers’ involvement and their active participation in decision-making, which contributes to creating a design-friendly environment where the team can work efficiently. Additionally, we have found that in order to make the most of the time available during the workshops and for the interaction to be balanced, it is necessary to organize the activities that take place inside and outside the workshops well. Finally, we have found that the combination of the DBR methodology with teacher co-creation process offers a very broad overview of the process of designing materials in teams. Specifically, we have found that teacher co-creation optimizes the processes characteristic of the DBR methodology by their constant alternation through the interaction of team members, which has had an impact not only on the quality of the materials but also on the professional development of the teachers.
... This study used a qualitative design-based research approach (DBR) and three prototypes of the TSCL PD were developed and improved on, using participant interviews and informal discussions. DBR is an approach to educational research that focuses on the iterative design and testing of educational interventions in authentic settings (McKenney & Reeves, 2014). It is also referred to as design research, design experiments, or educational design research. ...
... One of the features of DBR is that research needs to produce design outcomes (McKenney & Reeves, 2014). Based on the literature and the above findings from this research, guidelines for implementing TSCL PD have been developed to enhance teachers' SDL. ...
The goal of this study is to offer guidelines for the application of technology-supported cooperative learning professional development to enhance teachers’ self-directed learning skills. A qualitative approach was used which was informed by the interpretive paradigm. Many technology-supported cooperative learning professional development prototypes were created as part of the iterative design-based research methodology used in this study. Seven primary school Mathematics teachers from South African government schools participated in the technology-supported cooperative learning professional development for six weeks as part of a purposive sample. All the participants were scheduled for data collection through semi-structured interviews and follow-up interviews. The qualitative analysis computer data analysis system was used to prepare, process, and analyse the data collected from the semi-structured interviews. According to the findings of this research, teachers needed professional development that is affordable, flexible, intensive, and ongoing. The findings of the study provided insights into the best practices for implementing technology-supported cooperative learning professional development and enhancing teachers' self-directed skills. The study has the potential to contribute to the field of professional development for teachers as it will provide evidence-based recommendations for the effective and sustainable implementation of technology-supported cooperative learning professional development.
... Wawancara dilakukan untuk mendapatkan pemahaman yang lebih mendalam tentang permasalahan yang muncul di lapangan. Pedoman dan alat wawancara dalam penelitian ini didasarkan pada kerja lapangan dan mencakup tiga fokus penelitian utama, yaitu: (1) permasalahan, (2) konteks, dan (3) kebutuhan (McKenney & Reeves, 2014). Kuesioner diberikan melalui Google Sheets untuk melihat permasalahan yang muncul di lapangan. ...
Full-text available
Penelitian ini dilatarbelakangi oleh kurangnya penggunaan E-LKPD dalam pembelajaran teks eksposisi pada tingkat SMA. Tujuan dari penelitian ini adalah untuk menganalisis permintaan Lembar Kerja Peserta Didik Elektronik (E-LKPD) yang berbasis kearifan lokal. Metode yang digunakan dalam penelitian ini adalah deskriptif-kualitatif. Subjek penelitian ini adalah kepala sekolah dan guru SMA di Kota Padang. Alat pengumpulan data menggunakan pedoman wawancara dan dokumentasi. Analisis data menggunakan analisis Miles & Huberman, langkah-langkah analisis data adalah reduksi data, penyajian data, dan validasi data. Temuan penelitian ini mencakup tiga hal, yang meliputi: (1) analisis permasalahan yang mencakup permasalahan kurikulum, dan penerapan E-LKPD di SMA, (2) analisis konteks, dan (3) analisis solusi permasalahan. Temuan penelitian ini menunjukkan bahwa E-LKPD di sekolah belum memenuhi syarat untuk membuat E-LKPD yang benar. Oleh karena itu, sekolah memang membutuhkan E-LKPD yang sesuai untuk pembelajarannya, yang disesuaikan dengan kurikulum yang ada, dalam hal ini adalah Kurikulum Merdeka. Oleh karena itu, penelitian ini diharapkan dapat menjadi acuan untuk membuat atau mengembangkan E-LKPD berbasis HOTS dan berkearifan lokal sebagai media pembelajaran teks eksposisi pada tingkat SMA.
... In the following, we provide implications for design and practice, for MM teacher dashboards to facilitate teachers use. Future research should consider following an EDR/DBR approach [72] by inviting teachers as experts to participate alongside researchers, through iterative cycles of dashboard ideation, design, implementation, and evaluation, with the aim of improving the teachers' user experience and MM teacher dashboard adoption rates. EDR's ongoing teacher-researcher exchange may uncover teachers' various preferences and concerns (e.g., selection of important and reliable measurements, adoption concerns), thus producing knowledge early enough to negate several potential challenges encountered when MM teacher dashboard are used "in the wild", such as those which respondents identified during the interview phases. ...
Full-text available
Teacher dashboards provide insights on students' progress through visualisations and scores derived from data generated during teaching and learning activities (e.g., response times, task correctness) to improve teaching. Despite the potential usefulness of enhancing teacher dashboards, and the respective teaching practices, with rich information regarding students' cognitive and affective states (e.g., cognitive load), few studies on teacher dashboards have considered such information. In this study, we drew on contemporary developments of MultiModal Learning Analytics and designed a MultiModal (MM) teacher dashboard with notification system. The proposed system 1) receives data from various sensors, 2) computes relevant cognitive and affective measurements, 3) visualises the resulting measure- ments in a clean customisable interface, and 4) notifies instructors during moments of interest so they may determine an appropriate method to support struggling students. To evaluate our MM teacher dashboard, we first collected MultiModal Data (MMD), performance data, and video recordings of students' interactions during an in-situ study where 26 students engaged with a motion-based learning task. Then, we used our MM teacher dashboard to present the collected MMD and video recordings to 20 experienced teachers and educational researchers, and collected qualitative data regarding respondents' insights on the advantages and challenges of visualising students' MMD. Results showed that teachers found a MM teacher dashboard enhanced with notification system, useful to complement their pedagogical practices. We offer empirically founded guidelines for design and integration of a MM teacher dashboard with notification systems, aimed to enhance teachers' understanding of students' learning states (e.g., real-time awareness of students' stress).
Digital health and wellbeing are highly contested terms and range from carefully costed and evaluated software systems designed for patients to access their doctor; evidence-based mobile applications for supporting those living with long term health conditions such as diabetes; to the Coronavirus travel applications (app) developed to enable societies to come together post-pandemic. By way of contrast, numerous mental health ‘apps’ with tracking algorithms enabling individual personal data to be commercialized and sold on to third parties lack a robust evidence base and are problematic. Against a fast-changing backdrop, the European Union (EU) launched the revision of their Digital Framework Digital Competence (DigComp 2.2) in February of 2022. This paper reports on the findings of the ‘Safety and Security’ working group and their recommendations for the digital knowledge, skills, and attitudes (KSA) required for EU citizens negotiating a complex and constantly changing health sector.KeywordsEU DigCompDigital healthWellbeingLong term health
Full-text available
El desarrollo de habilidades tecnológicas y digitales por parte del profesorado en todos los niveles educativos es una necesidad imperante. Con este documento se propone una estrategia de formación para profesores de química mediante el diseño didáctico y la incorporación de recursos TIC. El objetivo de este artículo es aportar en términos de resultados, los alcances de una propuesta didáctica de formación docente en TIC orientada mediante una comunidad de desarrollo profesional para profesores de química (CODEP) en formación inicial (FI) gestionada mediante ciclos iterativos desde la perspectiva de la investigación de diseño educativo (IDE). Durante los ciclos iterativos de las diferentes fases de la propuesta de formación se aplicaron instrumentos como cuestionarios estandarizados, entrevistas y documentos para evidenciar el desarrollo de la propuesta. Mediante triangulación metodológica y de instrumentos se identifican las condiciones iniciales de los profesores participantes y su evolución durante la intervención. Se encuentra finalmente que los ciclos iterativos y las actividades que componen la unidad didáctica brinda herramientas didácticas, metodológicas y tecnológicas para mejorar las estrategias de diseño didáctico y la incorporación de TIC en propuestas para la enseñanza de la química a nivel de educación secundaria.
Sustainability and responsible use of the resources at our disposal are among the most important goals of our time. Employees are looking for next-generation employees with ever more skills. To ideally foster these in engineering and prepare them for future challenges, the integration of education for sustainable development (ESD) with a linkage of technical and sustainability-oriented issues in the curriculum is essential. This paper takes up two points: (1) An analysis of the research landscape in Engineering Education Research (EER) on the topic of “sustainability” is undertaken. For this purpose, more than 3500 conference papers of EDUCON and FIE of the years 2014 to 2018 as well as 2021 (EDUCON only) are evaluated. The methodology of the analysis as well as the set of main and sub categories (among them “sustainability”) will be presented at ICL 2022. The results of the analysis of the research landscape (1) show that the topic of sustainability has so far played a negligible role in the conference contributions. In (2), the focus is on the implementation process and the linking of technical and sustainability-oriented issues. The study programme concept of the Leuphana University offers two options for sustainable technical education by combining major and minor study programmes. The interdisciplinary combinations are presented and explained using module examples. The paper proves in a research-based manner that the topic of sustainability has so far been underrepresented in the engineering sciences, which is why evidence-based interdisciplinary implementation procedures such as those presented in (2) are highly relevant.
Full-text available
Abstract This article explores system geographical webbing, i.e., to use a specific connection web to analyze sustainability issues, as an object of knowing in upper secondary Geography teaching. A model task was developed based on systems principles and geographical concepts. Empirical data consists of teaching interventions aimed at encouraging students to perform webbing actions and to construct geographical relationships of sustainability issues. The article describes aspects of the geographical knowing needed for using a connection web and its embedded tools to explore sustainability issues. Finally, the article suggests avenues for teachers to support system geographical knowing development.
This chapter focuses on design and development research, a type of inquiry unique to the instructional design and technology field dedicated to the creation of new knowledge and the validation of existing practice. We first define this kind of research and provide an overview of its two main categories-research on products and tools and research on design and development models. Then, we concentrate on recent design and development research (DDR) by describing 11 studies published in the literature. The five product and tool studies reviewed include research on comprehensive development projects, studies of particular design and development phases, and research on tool development and use. The six model studies reviewed include research leading to new or enhanced ID models, model validation and model use research. Finally, we summarize this new work in terms of the problems it addresses, the settings and participants examined, the research methodologies employed used, and the role evaluation plays in these studies. © Springer Science+Business Media New York 2014. All rights reserved.
During the past decade, design-based research has demonstrated its potential as a methodology suitable to both research and design of technology-enhanced learning environments (TELEs). In this paper, we define and identify characteristics of design-based research, describe the importance of design-based research for the development of TELEs, propose principles for implementing design-based research with TELEs, and discuss future challenges of using this methodology. (