Content uploaded by Daniel Rees Lewis
Author content
All content in this area was uploaded by Daniel Rees Lewis on Jun 07, 2018
Content may be subject to copyright.
A preview of the PDF is not available
Planning to Iterate: Supporting Iterative Practices for Real-world Ill-structured Problem-solving
Abstract and Figures
Solving real-world highly ill-structured problems involves iteration: gathering information, building, testing, and revising products, experiments, and theories. However, we do not know how to create learning environments to teach iteration for highly ill-structured problems. How might we help student teams effectively iterate for highly ill-structured design problems? In this design-based research study we built on learning sciences research to implement Planning to Iterate—a weekly planning session in which teams create problem and planning representations. The study took place in a 6-week extracurricular undergraduate design program with five undergraduate project teams working on highly ill-structured problems. To understand team iterative practices, we analyzed videos of teams’ weekly planning sessions, and teams’ artifacts. Students significantly increased iterative practices, but infrequently integrated the practices together, suggesting re-design with additional coaching.
Figures - uploaded by Daniel Rees Lewis
Author content
All figure content in this area was uploaded by Daniel Rees Lewis
Content may be subject to copyright.
... • Scoping deliberations also provide templates for articulating a representation of the problem and solution (known as the theory of change), recording unanswered questions, and planning to seek new information (Strategy 7), which theoretically helps to structure the process of iteration (Puntambekar & Kolodner, 2005;Rees Lewis et al., 2018). In a design-based research study, Rees Lewis et al. used a template for undergraduate design teams to create a shared representation of a highly ill-structured problem as an important step for identifying gaps in knowledge that the team address through iteration. ...
... Puntambekar & Kolodner also used templates to teach middle-school students to learn science from design activities and found that scaffolds were important for supporting learners to incorporate inquiry into the design process, such as by prompting students to think about what design features needed to improve based on their own research (Puntambekar & Kolodner, 2005). • Finally, scoping deliberations create a recurring sequence of activities in which learners identify gaps in knowledge, plan investigations to seek new information, and incorporate that information into their theory of change (Strategy 8), which theoretically should help to structure the iteration process (Rees Lewis et al., 2018;Singer et al., 2000;Zivic et al., 2018). To help undergraduate design teams structure the iteration process, Rees Lewis and coauthors facilitated a planning process that helped them define elements of the problem and solution, identify risks associated with these elements, identify risks associated with those elements, prioritize risks to address, and make plans to iterate to reduce those risks (Rees Lewis et al., 2018). ...
... • Finally, scoping deliberations create a recurring sequence of activities in which learners identify gaps in knowledge, plan investigations to seek new information, and incorporate that information into their theory of change (Strategy 8), which theoretically should help to structure the iteration process (Rees Lewis et al., 2018;Singer et al., 2000;Zivic et al., 2018). To help undergraduate design teams structure the iteration process, Rees Lewis and coauthors facilitated a planning process that helped them define elements of the problem and solution, identify risks associated with these elements, identify risks associated with those elements, prioritize risks to address, and make plans to iterate to reduce those risks (Rees Lewis et al., 2018). Singer and coauthors describe pilot work on six inquiry projects utilizing design principles for helping students go through a sequence of multi-week activities to identify questions, gather data, analyze and reflect on findings from their inquiry to promote scientific learning (Singer et al., 2000). ...
Most social challenges fall outside of the authority of any single individual and therefore require collective action—coordinated efforts by many stakeholders to implement solutions. Despite growing interest in teaching students to lead collective action, we lack models for how to teach these skills. Collective action ostensibly involves design: the act of planning to change existing situations into preferred ones. In other domains, instructors commonly scaffold design using an instructional model known as studio critique in which students strengthen their plans by exchanging arguments with peers and instructors. This study explores whether studio critique can serve as the basis for an effective instructional model in collective action. Using design-based research methods, we designed and implemented scoping deliberations, a new instructional model that augments studio critique with domain-specific templates for planning collective action and repeats weekly to enable iterations. We used process tracing to analyze data from field notes, video, and artifacts to evaluate causal explanations for events observed in this case study. By implementing scoping deliberations in a 10-week undergraduate course, we found that this model appeared effective at scaffolding engagement in planning collective action: students articulated and refined their plans by engaging in argumentation and iteration, as expected. However, students struggled to contact the community stakeholders with whom they planned to work. As a result, their plans rested on implausible, untested assertions. These findings advance instructional science by showing that collective action may require new instructional models that help students to test their assertions against feedback from community stakeholders. Practically, scoping deliberations appear most useful for scaffolding thoughtful planning in conditions when students are already collaborating with stakeholders.
... This iterative refinement ensures the design's adaptability and alignment with user needs and accessibility standards, reflecting the dynamic and responsive nature of the framework. The iterative evaluation and learning mechanisms embedded in this stage reinforce the framework's capability for ongoing refinement and adaptation, crucial for maintaining relevance and effectiveness in a changing technological landscape (Cukurova et al., 2016;Legner et al., 2016;Lewis et al., 2018;Wang et al., 2023). ...
In the dynamic domain of web and mobile application development, the imperative to continuously evolve and enhance user experience is paramount. The EDIT UX Framework offers a robust, systematic approach to redesign, aimed at significantly enhancing user engagement, accessibility, and business performance. This framework is delineated into four pivotal stages: (1) Evaluation, which establishes a solid analytical foundation by synthesizing metrics analysis, heuristic evaluations, accessibility assessments, and user insights; (2) Design, where ideation and prototyping are driven by user-centric insights, fostering innovative solutions; (3) Iteration, a phase dedicated to refining designs through iterative user feedback and rigorous testing, with an unwavering focus on inclusivity and accessibility; and (4) Transformation, which transitions the refined product into the market, emphasizing continuous evaluation and iterative enhancements post-launch. By integrating principles of user-centered design, data-driven decision-making, and comprehensive accessibility, the EDIT UX Framework empowers design teams to create digital experiences that not only meet but exceed user expectations, ensuring a product’s resilience and adaptability in an ever-evolving digital landscape.
... First, students need to understand that solving any real-world problem is highly iterative (D. Crismond et al., 2013;Lewis et al., 2018). Similarly, scientific thinking is iterative (Dunbar, 2000) and to solve science problems, scientists may reengage in defining the problem at various stages, even when the problem has been represented. ...
In this article, we present a hierarchical model for teaching scientific thinking to gifted students. This article follows up on an article published 40 years ago in this journal. The problem now, as 40 years ago, is that gifted students often are taught science courses at a more intensive level, but without their truly learning how to think scientifically. We argue that students of science need not only learn the content of science courses, but also learn, at a deep level, how to think scientifically. Our model addresses the issue of what this deep level consists of. Level I involves Teaching Scientific Knowledge. Level II involves Teaching Scientific Problem Solving. Level III involves the deepest level of scientific thinking: Teaching Scientific Problem Finding. We end the article with conclusions about these issues.
... Perceiving learning situations as authentic will require involving role players outside of the formal classroom in representing how real-world problems are tackled (e.g. Rees Lewis et al. 2018). • Multiple perspectives -Different perspectives on the topic of the innovation project. ...
Background
Authenticity in schools has been highlighted as important for improving students’ engagement and learning, and to prepare them for future job markets, especially in science and technology.
Purpose
This study investigates students’ perceived authenticity of a developed innovation project when implemented in an upper secondary technology education program.
Sample
Three cohorts of students (n = 199) attended a first-year technology course at a Swedish upper secondary school in 2016, 2017 and 2018, respectively. In addition, eleven students from the 2016 cohort were interviewed two years later to obtain their views on how the innovation project in the first-year course influenced their performance in a subsequent advanced technology course taken in 2017–2018.
Design and Methods
Groups of students participated in the first phase of an innovation project in the first-year course, a five-week module, cooperatively designing solutions to real-world problems. A Likert scale questionnaire measured the degree of perceived authenticity in line with Herrington, Reeves and Oliver’s (2010) key elements. Focus group interviews were conducted after the second phase – a 20-week follow-up module in the subsequent advanced course – about how authentic they perceived the first and second phases to be.
Results
A questionnaire measured the degree of perceived authenticity of the students for the first phase, for each of the three years. Coaching and scaffolding received the highest ratings across all three years, whereas Reflection was perceived as having the lowest authenticity. In a qualitative component of the study students found both phases positive, and five new themes of students’ perception of their experiences were revealed.
Conclusions
The similarities in perceived authenticity between the three cohorts suggest consistency in students’ perceptions of authenticity. However, they did not feel that the project gave them the opportunity to reflect on their learning. According to interviews conducted two years later, they perceived their experiences of the innovation project as having induced creativity, commitment, ownership, motivation, and real-world connection, although at times it was also a challenge to think for themselves and to collaborate with others.
... These works illustrate the designing and inquiry processes through stages of constructing design concepts and alternatives. (2) The critical instructional elements for implementing a successful DBL curriculum in practice; For instance, some studies have pointed out the iterative design process as a vital element in orchestrating DBL activity (Adams and Atman 2000;Bekker et al. 2015;Gómez Puente et al. 2013a, b;Kolodner 2002;Kolodner et al. 2003;Lewis et al. 2018). Other elements include the teacher's role of coach (Gómez Puente et al. 2013a, b;van Diggelen et al. 2019), reflection (Bekker et al. 2015;Jun et al. 2017), the open-ended and authentic project feature (Bekker et al. 2015;Gómez Puente et al. 2013a, b), and collaboration in teamwork (Bekker et al. 2015;Fortus et al. 2004). ...
Educational settings are emotional places where students experience diverse emotions in relation to academic activities and their outcomes. Emotions, in turn, greatly influence students’ learning process and engagement. Research on emotions in Design-Based Learning (DBL) has so far been coarse-grained examining how students evaluate their overall feelings towards the DBL project. As yet, little is known regarding how specific DBL activities influence students’ emotional experience. Therefore, we report a three-month field study of a DBL project involving 30 middle school students (aged 13–14) addressing dual research purposes: (1) to faithfully reconstruct the manner and sequence of activities during DBL from a fine-grained perspective; and (2) to examine the relationship between these activities and students’ emotional experience. This study used a mixed research method and collected multiple data sources, including experience sampling surveys, classroom observations, and interviews. The research outcomes in this study are multiple. First, this paper reveals detailed inspection regarding the types of task students performed, the strategies of shifting, and executing tasks during the process student experienced. Second, this paper identifies specific types of activities that have a significant positive or negative relationship with students’ emotional experiences. Derived from reported empirical evidence in the present study, this paper furtherly proposes an Activity-and-Affect model of DBL. This model provides a fine-grained description of DBL activity as continuous along three dimensions: task (design thinking process, project management, social interaction) task strategy (single-tasking vs. multitasking), and collaboration strategy (individual or group). Our analysis highlights the variability in how different DBL activities can be associated with different emotions.
Background
To create design solutions experienced engineering designers engage in expert iterative practice. Researchers find that students struggle to learn this critical engineering design practice, particularly when tackling real‐world engineering design problems.
Purpose/Hypothesis
To improve our ability to teach iteration, this study contributes (i) a new teaching approach to improve student teams' expert iterative practices, and (ii) provides support to existing frameworks—chiefly the Design Risk Framework—that predict the key metacognitive processes we should support to help students to engage in expert iterative practices in real‐world engineering design.
Design/Method
In a 3‐year design‐based research study, we developed a novel approach to teaching students to take on real‐world engineering design projects with real clients, users, and contexts to engage in expert iterative practices.
Results
Study 1 confirms that student teams struggle to engage in expert iterative practices, even when supported by problem‐based learning (PBL) coaching. Study 2 tests our novel approach, Planning‐to‐Iterate, which uses (i) templates, (ii) guiding questions to help students to define problem and solution elements, and (iii) risk checklists to help student teams to identify risks. We found that student teams using Planning‐to‐Iterate engaged in more expert iterative practices while receiving less PBL coaching.
Conclusions
This work empirically tests a design argument—a theory for a novel teaching approach—that augments PBL coaching and helps students to identify risks and engage in expert iterative practices in engineering design projects.
One-to-many coaching is a common, yet difficult, coaching technique used in environments with many novices learning to solve ill-defined problems. Intelligent systems might be designed to support 1-to-many coaching but designing such systems requires a 1-to-many coaching model that details novices' challenges, coaches' strategies, and coaches' goals. To build such a model, we conducted interaction analysis on 24 1-to-many coaching sessions with novices developing new products in a university incubator and conducted retrospective analyses with 3 coaches and 30 novices. We contribute a model that demonstrates that coaches in a 1-to-many setting not only need to help novices develop metacognitive skills (just as in 1-to-1 coaching), but also need to utilize the presence and expertise of a group of novices to learn from each other, to mitigate their fear of failures, and provide them accountability. Our model informs design implications for future intelligent coaching systems to (1) assist coaches in monitoring and comparing many novices' progress, learning, and expertise; (2) provide novices with checklists, templates, and scaffolds to help them self-evaluate, seek-help, and summarize learning; (3) showcase failures and growth; and (4) publicize planning and progress to provide accountability.
Human-Centered Design (HCD) is a growing field that has the potential to positively impact students' learning. A general consensus on the terms, practices, scaffolds, and assessments of HCD can foster its effective implementation in K-12 and post-secondary education. This session brings together researchers whose work is focused on implementing HCD across K-16 classrooms. It aims to develop a coherent definition of HCD, its methods, practices, and assessments, to help frame the field and reduce ambiguity at a critical time in its broader adoption.
Electronic textiles are a part of the increasingly popular maker movement that champions existing do-it-yourself activities. As making activities broaden from Maker Faires and fabrication spaces in children's museums, science centers, and community organizations to school classrooms, they provide new opportunities for learning while challenging many current conventions of schooling. In this article, authors Yasmin Kafai, Deborah Fields, and Kristin Searle consider one disruptive area of making: electronic textiles. The authors examine high school students' experiences making e-textile designs across three workshops that took place over the course of a school year and discuss individual students' experiences making e-textiles in the context of broader findings regarding themes of transparency, aesthetics, and gender They also examine the role of e-textiles as both an opportunity for and challenge in, breaking down traditional barriers to computing.
This study shows that top-down decomposition is problematic in the early stages of design. Instead, an opportunistic decomposition is better suited to handle the ill-structuredness of design problems. Designers are observed interleaving decisions at various levels of abstraction in the solution decomposition. The verbal protocols of three professionals designing a software system of realistic complexity are analyzed to determine the frequency and causes of opportunistic decompositions. The sudden discovery of new requirements and partial solutions triggered by data-driven rules and associations, the immediate development of solutions for newly discovered requirements, and drifting through partial solutions are shown to be important causes of opportunistic design. A top-down decomposition appears to be a special case for well-structured problems when the designer already knows the correct decomposition. Two cognitive models are briefly discussed in relation to opportunistic design. Finally, implications for training, methods, and computational environments to support the early stages of design are outlined.
Design can change the world. Growing environmental and social concerns about the role we play as world citizens and caretakers of the planet have given rise to a green environmental movement and concerns of sustainability. But sustainability only attacks these problems in an incremental way. A more novel approach, geared towards real impact and breakthrough innovation, is to shift the frame of these growing issues of our time in a different way. Rather than engineer better solutions and implement better technology, this paper describes design and design thinking education practice and student examples that seek to change the context dramatically and break the mold of current means of thinking about sustainability. By examining mechanical engineering students doing design work in the context of a graduate level mechanical engineering design course at Stanford University, this paper will illustrate the benefits of the approach of breakthrough innovation through design thinking and highlight some difficulties students have in conceptualizing, framing, and designing for the future.
Educating effective engineering designers is an important goal. Exploring the extent to which this goal is being met hinges on our ability to characterise what contributes to effectiveness and to map students’ performance against such standards. In previous work, we used verbal protocol analysis to analyse differences in the design processes of freshmen and seniors, the effects of interventions on student design processes, and process factors that contribute to product quality. In this paper, we utilise Schön’s reflective practitioner theory to discuss our empirical results in the context of educating reflective practitioners. Such an approach may provide implications for enhancing engineering education.
Background: Design experiences play a crucial role in undergraduate engineering education and are increasingly important in K-12 settings. There are few efforts to purposefully connect research findings on how people design with what teachers need to understand and do to help K-16 students improve their design capability and learn through design activities. Purpose: This paper connects and simplifies disparate findings from research on design cognition and presents a robust framework for a scholarship of design teaching and learning that includes misconceptions, learning trajectories, instructional goals, and teaching strategies that instructors need to know to teach engineering design effectively. Method: A scholarship of integration study was conducted that involved a meta-literature review and led to selecting and bounding students' design performances with appropriate starting points and end points, establishing key performance dimensions of design practices, and fashioning use-inspired tools that represent design pedagogical content knowledge for teachers. Results: The outcome of this scholarship of integration effort is the Informed Design Learning and Teaching Matrix that contains nine engineering design strategies and associated patterns that contrast beginning versus informed design behaviors, with links to learning goals and instructional approaches that aim to support students in developing their engineering design abilities. Conclusions: This paper's theoretical contribution is an emergent educational theory of informed design that identifies key performance dimensions relevant to K-16 engineering and STEM educational contexts. Practical contributions include the Informed Design Teaching and Learning Matrix, which is fashioned to help teachers do informed teaching with design tasks while developing their own design pedagogical content knowledge.
In this article, we present two studies that helped us understand the kinds of support that students need to learn science successfully from design activities. Both were enacted in the context of an approach to learning science from design called learning by design (LBD). In our first study, we designed and integrated a paper-and-pencil scaffolding tool, the design diary, into an LBD unit to support students' design-related activities. We learned two important lessons from the first study. First, we refined our understanding of the processes involved in designing and the ways we might present those processes to students. Second, and more important, we observed that in the dynamic, complex environment of the classroom, not all of the scaffolding could be provided with any one tool or agent. We found that students need multiple forms of support and multiple learning opportunities to learn science successfully from design activities. In our next study, we provided additional support through an organized system of tools and agents. Our analysis of data from the second study leads us to believe that supporting multiple students in a classroom requires us to rethink the notion of scaffolding as it applied to groups of learners in a classroom. We put forth the notion of distributed scaffolding as an approach to supporting hands-on inquiry learning in a classroom. © 2005 Wiley Periodicals, Inc. J Res Sci Teach 42: 185–217, 2005
Problem-based approaches to learning have a long history of advocating experience-based education. Psychological research and theory suggests that by having students learn through the experience of solving problems, they can learn both content and thinking strategies. Problem-based learning (PBL) is an instructional method in which students learn through facilitated problem solving. In PBL, student learning centers on a complex problem that does not have a single correct answer. Students work in collaborative groups to identify what they need to learn in order to solve a problem. They engage in self-directed learning (SDL) and then apply their new knowledge to the problem and reflect on what they learned and the effectiveness of the strategies employed. The teacher acts to facilitate the learning process rather than to provide knowledge. The goals of PBL include helping students develop 1) flexible knowledge, 2) effective problem-solving skills, 3) SDL skills, 4) effective collaboration skills, and 5) intrinsic motivation. This article discusses the nature of learning in PBL and examines the empirical evidence supporting it. There is considerable research on the first 3 goals of PBL but little on the last 2. Moreover, minimal research has been conducted outside medical and gifted education. Understanding how these goals are achieved with less skilled learners is an important part of a research agenda for PBL. The evidence suggests that PBL is an instructional approach that offers the potential to help students develop flexible understanding and lifelong learning skills.
This work seeks to better understand how design processes affect design outcomes. Design process data were collected from journals kept as a part of mechanical engineering capstone design projects at Montana State University. Student processes were characterized by time coding journal entries using a 3×4 matrix of process variables. The data were modeled using a principal components artificial neural network, and the model used in a virtual designed experiment to obtain estimates for design process factors that significantly affect client satisfaction. Results indicate that greater client satisfaction is achieved through: greater problem definition (PD) activity and idea generation at conceptual design levels, and PD and engineering analysis activities at the system design level. Whereas, design activity at the detailed level associates with lower client satisfaction. These results support some aspects of existing models of “good” design process, and suggest adaptations of the models for novice designers.
Mental iteration in engineering design is a repetition of cognitive activities occurring in designers' thinking process. While the importance of mental iteration has been recognized, the current understanding about it is still very limited. In this paper, we propose a framework to study mental iteration in different design situations. A cognitive activity model of conceptual design is developed to identify and capture various loops of cognitive activities. An experiment is conducted to study the effect of design problems and constraints on the behavior of mental iteration. The results indicate differences of iterative behaviors in response to different problem types and constraint conditions.