ArticlePDF Available

Design-based research and technology-enhanced learning environments. Educational Technology Research and Development, 53(4), 5-23

Authors:

Abstract

During the past decade, design-based research has demonstrated its potential as a methodology suitable to both research and design of technology-enhanced learning environments (TELEs). In this paper, we define and identify characteristics of design-based research, describe the importance of design-based research for the development of TELEs, propose principles for implementing design-based research with TELEs, and discuss future challenges of using this methodology.
AERA 2004 Proposal Using Design-based… - 1 -
Using Design-based Research in Design and Research of Technology-
Enhanced Learning Environments
Feng Wang
fwang@coe.uga.edu
Michael J. Hannafin
hannafin@coe.uga.edu
The University of Georgia
Abstract
In the past decade, a new research paradigm—design-based research—has
showed great potential as an ideal alternative research methodology. Design-based
research is suitable to both research and design of technology-enhanced learning
environments (TELEs). At the beginning of this presentation, we will propose the
definition and five characteristics of design-based research research methodology. Then,
we will discuss why design-based research is important to both research and design of
TELEs. Next, nine principles are provided that are essential to successfully implementing
design-based research methodology with TELEs. Finally, we will discuss its implications,
challenges, and payoffs to TELE designers.
Introduction
The application of technology in education remains unsatisfying; TELEs have not
been widely used by students and teachers (Cuban, 1986, 2001; Kent & McNergney,
1999). To address these problems, a new educational research paradigm — design-based
research — has showed great potential to change the disappointing disconnect between
educational research and design practice. The design-based research paradigm, advanced
initially by Ann Brown (1992) and Allan Collins (1992) as design experiments, posits
intimate relationships between design and engineering and tends to be both scientific and
educational (Kelly, 2003). Design-based research deconstructs the presumed integrity
that the research should not be contaminated by the external influence of the research
(Barab & Kirshner, 2001). Researchers in design-based research processes collaborate
intimately with participants to achieve theoretical and pragmatic goals that will ultimately
change educational practices in a maximum extent.
For TELE design and research, design-based research methodology is even more
promising. Many TELE projects proven spectacularly successful in recent years feature
this methodology, such as WISE (Bell & Linn, 2000), Adventures of Jasper Woodbury
(Cognition and Technology Group at Vanderbilt, 1992a, 1992b), and CSILE (Cohen &
Scardamalia, 1998). The purposes of this presentation are to summarize the key ideas of
design-based research, to describe the utility of design-based research for both TELE
research and design, to provide principles deemed essential to successful TELE s based
on design-based research methodology, and to discuss implications, challenges, and
payoffs for TELE designers.
Design-based research introduction
We use the term “design-based research” specifically to represent a research
paradigm comprising many labels in the literature, including: 1) design experiments
AERA 2004 Proposal Using Design-based… - 2 -
(Brown, 1992; Collins, 1992), 2) design-based research (The Design-Based Research
Collective, 2003), 3)design research (Cobb, 2001; Edelson, 2002; Collins, Joseph, &
Bielaczyc, in press, 4) development research (van den Akker, 1999) or developmental
research (Richey & Nelson, 1996), 5) formative research (Walker, 1992), and 6) action
research (Stringer, 1999). Though some have slightly different foci, the intrinsic ideas are
consistent. We propose the following definition:
Design-based research is a research methodology aimed to improve educational
practices through systematic, flexible, and iterative review, analysis, design, development,
and implementation, based upon collaboration among researchers and practitioners in
real-world settings, and leading to design principles or theories.
We summarize the key ideas of design-based research through five basic
characteristics:
Pragmatic research goal
Design-based research posits synergy between practice and research, propelling
the development of practice as much as possible. From a design-based research
perspective, research should refine both theory and practice (Collins, Joseph, Bielaczyc,
in press); ultimately, the value of theory will be appraised by the extent to which their
principles inform and improve practice (Greeno & Collins, 1996; Cobb, Confrey, diSessa,
Lehrer, & Schauble, 2003).
Grounded research methodology
As design-based research is usually conducted in a limited number of settings, the
research and design process need to be grounded. Design-based research is conducted in
real-world contexts with social interaction rather than laboratory settings with social
isolation (Collins, 1999). Before conducting a design-based research, researchers review
literature and available design cases to identify gaps to ensure the value of the research
(Edelson, 2002). Its theory driven nature is important in that design-based methods are
considered more a research paradigm than an evaluation method.
Interactive, iterative, and flexible research process
Design-based research process is characterized by an iterative cycle of design,
enactment or implementation, analysis, and redesign (The Design-Based Research
Collective, 2003). Through this cycle, a theory will be gradually formed and updated
based on the accumulated data collected in each design iteration as well as
implementation experiences of the designer(s) (Edelson, 2002).The interactive, iterative
and flexible research process also corresponds to timely factors in the local settings (The
Design-Based Research Collective). Tacitly, in the design-based research process, a
researcher needs to balance the role as a designer and a researcher. This is not necessarily
negative as it helps to balance different solutions and perspectives (van den Akker, 1999).
Integrative research methods
Design-based researchers use mixed methods to maximum the credibility and
adaptability of their methods. Methods used in design-based research are not unique to
design-based research, such as survey, expert review, evaluation, case study, interview,
inquiry methods, etc. (Richey & Nelson, 1996; McCandliss, Kalchman, & Bryant, 2003).
Retrospective analysis and formative evaluation are also considered methods of design-
based research (van den Akker, 1999; Edelson, 2002; Cobb et al., 2003).
Contextual research results
AERA 2004 Proposal Using Design-based… - 3 -
The results generated from design-based research take the form of a profile akin
to a consumer report (Collins et al., in press) or principles in the form of heuristic
statements. The research process, the research findings, and any change from the initial
plan are documented; some warrants or guidance on how to use these principles are also
provided (Shavelson, Phillips, Towne, & Feuer, 2003). Thus, other researchers or
designers can trace the emergence of an innovation or combinations of innovations
according to their interests; they also examine contexts or conditions that led to different
effects (Baumgartner & Bell, 2002).
Importance of design-based research to TELEs
Design and research on TELEs are mutually beneficial. Design processes,
especially research-driven designs, can be thought of as research processes. Design is a
strategy for developing, refining, and testing theories with design-based research rather
than a way to implement theories for testing with traditional research methodologies
(Edelson, 2002). Regarding the history of design TELEs, however, design and research
are often mismatched. Numerous learning environments were developed using
contradictory theory bases; designs are not supported by underlying theories so the
outcomes are not traceable to an emerging theory (Collins, 1992). With traditional
research methodologies, research is usually conducted after the design processes and
research results generated from traditional research methodologies are too abstract to be
referable (Cobb, et al., 2003). Thus, it is difficult for TELE designers to identify and
apply useful theories to guide their designs.
With the design-based methodology, design and research processes are integrated.
Design-based research, with its pragmatic goal, grounded methodology, integrative
methods, contextual results, and interactive, iterative, flexible processes, not only
supports TELE design but also generates useful, practical theories. In the following
section, we explicate the importance of design-based approaches to TELEs using three
propositions:
1) design-based research can support the design of TELEs;
2) TELE design processes can be research;
3) design-based research is an ideal research methodology on TELEs.
There are other important considerations. First, many problems associated with
educational research methodologies became apparent especially in the past decade; more
and more educational researchers voice the need to employ pragmatic research
methodologies capable of narrowing the gap between theory and practice. Some TELE
designers also argued the need to integrate research and design on TELEs. Research on
TELEs should take the form of design because knowledge generated from the design
processes are unique and can support the design of TELEs in a maximum extend. As
design-based research can bring so many benefits to TELE designers, we would argue
that it’s an ideal way to both design and research of TELEs.
Principles for design-based research
In the following section, we discuss the methodology of design-based research
through principles essential to the successful implementation of design-based research on
TELEs. We do not intend to build a general framework of design-based research, but
rather to inform the design-based research within the context of TELEs.
AERA 2004 Proposal Using Design-based… - 4 -
Support the design with research from the outset
Set pragmatic theory goals and initial plans
Conduct research in real-world settings without exception
Collaborate with participants intimately
Use research methods systematically
Analyze collected data retrospectively and constantly
Refine the design continuously and iteratively
Report contextual and usable design principles
Underscore the generizability of the design continuously
Challenges for design-based researchers
As a relatively new methodology, design-based research has both advantages and
limitations; important unanswered questions remain: What epistemology and theory
perspectives underlie design-based research initiatives? How do we reconcile the mixed
methods needed in the research processes? How do we balance the goal of improving
practice and the goal of generating design principles? The pragmatic nature of design-
based research also limits the applicability of traditional theories and related constructs
which may be necessary in some situations. After all, the culture of present-day practice
is affected deeply by empirical research methodologies that appears “scientifically
credible” because of the presumed discipline of the inquiry and statistical evidence
typically accompanying reports; thus, design-based research may not be as well-suited for
administrators or politicians as empirical methodologies given policy debates and
emphasis on “Scientifically-based research” (Cobb, 2001). Finally, as design-based
research requires documenting the whole research processes in authentic learning
environments, the amount of data collected is typically large, requiring both extended
time and resources to analyze (Collins, Joseph, & Bielaczyc, in revision).
Payoffs and conclusions
Design-based research affords many opportunities for improvement to both
design and research on TELEs. We view three areas as most promising: a) propelling the
application of TELEs in classrooms, b) productively and effectively generating credible
and useful theories, and c) transforming disappointing traditional research and design
methodologies. When design-based research is employed by more designers and
researchers, its benefits will be more evident. For the field of instructional technology
specifically, design-based research is critical to the evolution of its theory base (Richey &
Nelson, 1996). TELE designers should collect their efforts to accelerate the coming
systematic change brought by the design-based research methodology.
References
Barab, S. A., & Kirshner, D. E. (2001). Guest Editors' Introduction: Rethinking
methodology in the learning sciences. Journal of the Learning Sciences, 10(1&2),
5-15.
Baumgartner, E., & Bell, P. (2002). What will we do with design principles? Design
principles and principled design practice. Paper presented at the Annual
AERA 2004 Proposal Using Design-based… - 5 -
Conference of the American Educational Research Association, New Orleans,
Louisiana.
Bell, P., & Linn, M. C. (2000). Scientific arguments as learning artifacts: Designing for
learning from the Web with KIE. International Journal of Science Education,
Special Issue(22), 797-817.
Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in
creating complex interventions in classroom settings. The Journal of the Learning
Sciences, 2(2), 141-178.
Cobb, P. (2001). Supporting the improvement of learning and teaching in social and
institutional context. In S. Carver & D. Klahr (Eds.), Cognition and instruction:
25 years of progress (pp. 455-478). Cambridge, MA: Lawrence Erlbaum
Associates, Inc.
Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments
in educational research. Educational Researcher, 32(1), 9-13.
Cognition and Technology Group at Vanderbilt (1992a). The Jasper experiment: An
exploration of issues in learning and instructional design. Educational Technology
Research & Development, 40(1), 65-80.
Cognition and Technology Group at Vanderbilt (1992b). The Jasper Series as an example
of anchored instruction: Theory, program description, and assessment data.
Educational Psychologist, 27(3), 291-315.
Cognition and Technology Group at Vanderbilt (1997). The Jasper project: Lessons in
curriculum, instruction, assessment, and professional development. Mahwah, NJ:
Erlbaum.
Cohen, A., & Scardamalia, M. (1998). Discourse about ideas: Monitoring and regulation
in face-to-face and computer-mediated environments. Interactive Learning
Environments, 6(1-2), 93-113.
Collins, A. (1992). Towards a design science of education. In E. Scanlon & T. O’Shea
(Eds.), New directions in educational technology (pp. 15-22). Berlin: Springer.
Collins, A., Joseph, D., & Bielaczyc, K. (in press). Design Research: Theoretical and
Methodological Issues. In J. Campione (Ed.), Volume in honor of Ann Brown.
Collins, A., Joseph, D., & Bielaczyc, K. (in revision). Design Research: Theoretical and
Methodological Issues. Journal of the Learning Sciences.
Cuban, L. (1986). Teachers and machines: The classroom use of technology since 1920.
New York: Teachers College Press.
Cuban, L. (2001). Oversold and underused: Computers in the classroom. Cambridge,
MA: Harvard University Press.
Edelson, D. C. (2002). Design research: What we learn when we engage in design.
Journal of the Learning Sciences, 11(1), 105-121.
Kelly, A. E. (2003). Research as Design. Educational Researcher, 32(1), 3-4.
Kent, T. W., & McNergney, R. F. (1999). Will technology really change education: From
blackboard to web. Thousand oaks, CA: Corwin Press.
McCandliss, B. D., Kalchman, M., & Bryant, P. (2003). Design experiments and
laboratory approaches to learning: Steps toward collaborative exchange.
Educational Researcher, 32(1), 14-16.
AERA 2004 Proposal Using Design-based… - 6 -
Richey, R. C., & Nelson, W. A. (1996). Development research. In D. Jonassen (Ed.),
Handbook of research for educational communications and technology (pp. 1213-
1245). London: Macmillan.
Savery, J. R., & Duffy, T. M. (1996). Problem based learning: An instructional model
and its Constructivist framework. In B. G. Wilson (Ed.), Constructivist learning
environments: Case studies in instructional design (pp. 135-148). Englewood
Cliffs, NJ: Educatioal Technology Publications.
Shavelson, R. J., Phillips, D. C., Towne, L., & Feuer, M. J. (2003). On the science of
education design studies. Educational Researcher, 32(1), 25-28.
Stringer, E. (1999). Action research: A hand book for practitioners (2nd ed.). Thousand
Oaks, CA: Sage Publications.
The Design-Based Research Collective (2003). Design-based research: An emerging
paradigm for educational inquiry. Educational Researcher, 32(1), 5-8.
van den Akker, J. (1999). Principles and methods of development research. In J. van den
Akker, N. Nieveen, R. M. Branch, K. L. Gustafson & T. Plomp (Eds.), Design
methodology and developmental research in education and training (pp. 1-14).
The Netherlands: Kluwer Academic Publishers.
Walker, D. F. (1992). Methodological issues in curriculum research. In P. Jackson (Ed.),
Handbook of research on curriculum (pp. 98-118). New York: Macmillan.
This paper can be cited as:
Wang, F., & Hannafin, M. J. (2004). Using design-based research in
design and research of technology-enhanced learning environments.
Paper presented at the Annual Meeting of the American Educational
Research Association, San Diego, CA.
... To address this research gap, this study reviewed 20 years of literature (2000-2020), on AR-based instruction in K-12 educational settings and conducted a systematic analysis from the perspectives of publication trends, application contexts, technology features, and instructional design, and identified the overall effectiveness of AR and its moderators through a meta-analysis. According to Wang and Hannafin (2005) [34], those perspectives and their intertwined relationships are essential for understanding technology-enhanced learning environments. Consequently, the following five research questions guided our systematic review and meta-analysis: 1. ...
... To address this research gap, this study reviewed 20 years of literature (2000-2020), on AR-based instruction in K-12 educational settings and conducted a systematic analysis from the perspectives of publication trends, application contexts, technology features, and instructional design, and identified the overall effectiveness of AR and its moderators through a meta-analysis. According to Wang and Hannafin (2005) [34], those perspectives and their intertwined relationships are essential for understanding technology-enhanced learning environments. Consequently, the following five research questions guided our systematic review and meta-analysis: 1. ...
... After finalizing the master database, we carried out an in-depth reading of the selected articles, and the content of the selected articles was analyzed using the coding scheme shown in Table 1. The coding scheme was based on the work of Wang and Hannafin (2005) [34] and focused on the key aspects of the technology-enhanced learning environment. The final coding scheme included five interrelated aspects of AR-supported instruction in K-12 settings: basic information, instructional context, technology features, instructional design, and research findings. ...
Article
Full-text available
With its capacity to support student-centered learning through digital transformation and shared experience, augmented reality (AR) has received increasing attention from both researchers and practitioners as an emerging technology to achieve innovative and sustainable education. Therefore, this study systematically reviewed the literature on the application of augmented reality in K–12 education settings between 2000 and 2020. After two stages of screening, 129 articles were selected, and the key research results were analyzed and integrated by adopting a coding scheme including basic information, instruction contexts, technical features, instructional design, and research results. The results revealed interesting findings regarding the augmented reality literature in terms of publication patterns, application fields, technological affordances, instructional designs, and methods. Furthermore, a meta-analysis was conducted to examine the effectiveness of augmented reality-based instruction, and the results showed a large overall effect size (g = 0.919) with three significant moderators. Finally, the practical significance of AR-based instruction and a future research agenda are discussed.
... Design-based research (DBR) is a systematic, but flexible methodology that improves educational practices through iterative analysis, design, development, and implementation, in J o u r n a l P r e -p r o o f naturalistic learning environment (Wang & Hannafin, 2005). DBR aims to develop contextually sensitive design principles and theories in real-world settings (Tabak, 2004). ...
... However, there are several challenges concerning DBR. For example, it can be difficult to determine whether to continue or abandon an iterative design (Wang & Hannafin, 2005). ...
Article
The overarching goal of this design-based research is to develop and evaluate a set of design principles for a fully online flipped classroom to support students' learning outcomes, behavioural, emotional, and cognitive engagement. In a fully online flipped classroom, students are encouraged to complete online pre-class activities asynchronously. But unlike in the conventional flipped approach, students do not subsequently meet face-to-face in classrooms, but rather online synchronously. The testbed involved a conventional flipped class (Cycle 0), a fully online flipped class (Cycle 1), and a refined fully online flipped class (Cycle 2). The results showed that although all three groups of students performed equally well in learning, the refined online flipped model was more effective in supporting students' behavioural engagement in the synchronous online class sessions than the online flipped model. This study contributes to the extant literature by explicating the design principles that support student engagement in fully online flipped learning.
... Communication, socializing, entertainment, shopping, and education have all undergone fundamental shifts as a result of the rise of information and communications technology (ICT). Technology-enhanced learning is defined as learning and instructional methods that are based on the use of technology and through which students acquire skills or knowledge (Wang & Hannafin, 2005). ...
Conference Paper
Full-text available
The effects of Covid-19 and subsequent closure of schools for nearly two year period have robbed many children of their basic Right to Education. Particularly in the marginalized sections of the society, where education is the only hope for a better future, this break from education could potentially wipe out hard earned gains over several years. For most of these students, closure of schools meant no access to education or educational material beyond textbooks. Many villages still don't have phone networks or the internet, so online education is not an option. To address this new challenge to the education system, we returned to an old paper media, a Newsletter or just a letter, to interact with the students. We named this newsletter Anandi , meaning 'blissful'. 1 In this work we will discuss the genesis and design of the newsletter, and the learnings from our experiment. We emphasize the need to scale up such efforts during the pandemic period to complement the efforts taken by the schools and other educational bodies. We also briefly 1 The name ANANDI also honours the inspirational life of Dr. Anandibai Joshi, one of the first women from India to study modern medicine. present opinions of students and teachers about such an intervention.
Article
Full-text available
This paper describes the changes made to the administration of assessments in the second design trial of the Maths Intensive course at a comprehensive university in South Africa because of Covid-19. The course was designed in direct response to poor attainment in the Primary Teacher Education project’s mathematics test. Building on positive findings of improved attainment evident in the first design cycle, the Maths Intensive course was further refined for a second cohort of students, and shifts in attainment were once again analysed. The Covid-19 lockdown meant that student could not write the post-test on campus. As a result, the post-test data were not used to measure impact but, instead, were used to establish what the student teachers knew and were able to do at the end of the course. The Maths Intensive test framework mapped the test items to the learning outcomes for the course. Facility scores revealed areas where students were capable and areas where they still required support. The assumption that students would use calculators in un-invigilated circumstances was challenged, and requires further research. These findings may be of interest to lecturers and teachers seeking to develop assessment reports that provide useable information to improve their instructional design and teaching practice. They may also be of interest to academics and training providers designing mathematics education programmes for initial teacher education or for ongoing professional development courses for teachers in primary schools.
Article
Full-text available
Studies using co-design methods require the meaningful involvement of stakeholders in creating new knowledge and harnessing, mobilising, and transferring existing knowledge to support comprehensive and long-term solutions. In the health sector, co-design methodology is seen as a way of supporting and engaging local communities in critical decision-making about their health. However, little is known about which specific co-design methods have been adopted, used, and implemented within health education contexts. To address this gap, this paper presents a literature review of co-design methods used to design and implement health education interventions. This rapid evidence assessment (REA) was carried out by identifying 53 papers categorised into four themes: methods, stages, stakeholders, and outcomes. We examined specific co-design methods used in health education stages to support the involvement of stakeholders, second, we reviewed the outcomes of the application of these methods. Based on the review findings, the paper reflects two areas: first, the review shows that there are a wide number of co-design methods being used to support stakeholder collaboration to design health care services as products and processes. Second, there is no clear way co-design methods are evaluated for their outcomes. This review of literature contributes an evidence base to support the future development and use of co-design in health contexts by organising relevant literature into coherent themes in ways that can inform future research.
Article
The Jasper Woodbury Problem Solving Series is an example of a video-based instructional macrocontext for complex problem generation and problem solving. The Jasper series and curricular materials are described and illustrated in this article. The theoretical framework underlying the series includes assumptions about educational goals and the nature of learning, including the importance of generative activities and cooperative learning situations. The authors argue that the Jasper series affords generative and cooperative learning activities in a way that traditional mathematics problem-solving materials do not. However, whether these features are utilized depends on the teaching model at work in the classroom. Three models of teaching—basics first, structured problem solving, and guided generation—that can be applied to the Jasper series are outlined. The strengths and weaknesses of each are discussed, and associated assessment issues are raised. The article concludes by pointing to the need for research on the impact of differing instructional design decisions.