No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Students' experimentation strategies in design: Is process data enough?
Abstract
Conducting experiments is an important practice for both engineering design and scientific inquiry. Engineers iteratively conduct experiments to evaluate ideas, make informed decisions, and optimize their designs. However, both engineering design and scientific experimentation are open‐ended tasks that are not easy to assess. Recent studies have demonstrated how technology‐based assessments can help to capture and characterize these open‐ended tasks using unobtrusive data logging. This study builds upon a model to characterize students' experimentation strategies in design (ESD). Ten undergraduate students worked on a design challenge using a computer‐aided design (CAD) tool that captured all their interactions with the software. This “process data” was compared to “think‐aloud data,” which included students' explanations of their rationale for the experiments. The results suggest that the process data and the think‐aloud data have both affordances and limitations toward the goal of assessing students' ESD. While the process data was an effective approach to identify relevant sequences of actions, this type of data failed to ensure that students carried them out with a specific purpose. On the other hand, the think‐aloud data captured students' rationale for conducting experiments, but it depended on students' ability to verbalize their actions. In addition, the implementation of think‐aloud procedures and their analysis are time consuming tasks, and can only be done individually.
... Some LA tools have higher requirements for learners. The tool appeared in the research of Vieira et al. (2018) required freshmen students basic high school level science knowledge and good English-speaking skills. ...
... Some LA tools demonstrates the ability to help set learning environments in which learners' complex problem-solving skills improved, like engineering design courses (Dasgupta et al., 2019;Vieira et al., 2018;Mio et al., 2019), marketing-based decision courses (Bucic et al., 2018) or game design courses (Richard & Giri, 2019). ...
... Interdisciplinary LA tools appears more often in STEM design course and the learning context where study items are divided by work fields for adult learners. The objects of this learning environment are more related to ability development (Tran et al., 2018), to assessing students' experimentation strategies in an engineering design challenge (Vieira et al., 2018), to suppling opportunities of computational thinking (Richard & Giri, 2019) and improving content-discover ability (Tran et al., 2018). ...
Applying learning analytics (LAs) to actual teaching scenarios is a huge challenge. One of the problems that is required to be solved is how to combine LAs with pedagogy. Activity theory (AT) provides a conceptional tool for social human activities including objects and tools. Combining AT and pedagogical strategies as an analysis framework, this chapter analyzes LA application scenarios in seven components: subject, objective, community, tools, rules, division of labor, and outcomes. And learning theories present an in-depth analysis of rules. Conclusion shows in the LA application: teachers and students are main subjects; knowledge mastery is a common object; researchers and administrators play important roles while teachers have no specific teaching guidance to follow; presentation strategies of content are abundant; LAs integrate with multiple assessments; behaviorism, cognitivism, and constructivism embodied at different degrees; measurement of LAs application are diverse; not only learners, but characteristics of tasks need to be further studied.
... In this case study [68,69], we explore how we can use LA to understand specific strategies of the engineering design process. The primary benefit of the design process is its support for solving real-world problems, which are complex issues with no single solution or approach to arrive at the solution. ...
... Vieira and colleagues [69] proposed a continuum of experiments implemented by designers with different levels of experience. Novice designers do not generate hypotheses or implement experiments to test their hypotheses [72]. ...
This chapter presents five case studies that explore various affordances for engineering educators in reimagining education through technology. The first case study delves into the integration of teamwork and technology, addressing challenges in online learning and proposing innovative solutions for effective engagement in teamwork within large-sized classrooms. The second case study explores the integration of computational science and engineering (CSE) and computational thinking (CT) into existing engineering courses, recognizing CSE as a crucial pillar in the digital era. The third case study discusses personalized digital learning through adaptive technologies to enhance participation in engineering education. The fourth case study highlights the nuanced art of managing uncertainty in makerspaces, emphasizing the role of educators in guiding students through productive uncertainty management. The final case study explores the application of learning analytics in engineering design, offering insights into student experimentation strategies. Through these case studies, the chapter aims to provide academic leaders with valuable insights and strategies to redefine engineering education through technology.
... Moreover, while videotaping can capture these strategies, analyzing video data is time-intensive and often only practical in research contexts. Learning analytics can offer an alternative for capturing students design strategies and practices that is unobtrusive (Shute, 2011), and scalable for all students in a classroom setting (e.g., Vieira et al., 2018) and produces rich and reliable data (Rahman et al., 2019). Learning analytics leverage digital platforms to collect detailed student behavior to support analysis of this data toward understanding and supporting learners (Siemens, 2013). ...
... Although still emerging as an analytical approach, learning analytics has been growing in popularity with several dedicated literature reviews published (e.g., see Aldowah et al., 2019;Mangaroska & Giannakos, 2019) and conferences such as International Conference on Learning Analytics and Knowledge. Moving specifically to the study of design learning or design education, learning analytics are also gaining more attention (Vieira et al., 2018;Akhtar et al., Vieira et al. (2016) used a computer aided design (CAD) system for creating sustainable homes that logged students' actions to identify the design experiment strategies of 55 middle school students, including systematic and non-systematic experiments. The results revealed a correlation between the number of experiments run, both systematic and non-systematic, and the quality of students' final design models. ...
Worldwide, engineering design is seeing an increase in pre-college settings due to changing educational policies and standards. Additionally, these projects can help students develop critical skills for a broad range of problem settings, such as design thinking and reflection. In design and other contexts, reflection is a mental process where someone returns to previous experience and uses this revisiting to aid in new actions. While there is substantial research studying design practices at the collegiate or professional level, the design practices of younger students remain understudied. Moreover, past research on reflection has tended to focus on how to support reflection or what impact reflection has and not how students engage in reflection strategies. We had 105 middle school students in the Midwestern United States design a green-energy home using a computer-aided design (CAD) tool, Energy3D. Students were instructed to use Energy3D’s design journal to reflect on their design process throughout the project, enabling students to employ different reflection strategies. Energy3D unobtrusively captures students’ design actions, including journal interactions; these were used to identify students' reflection strategies. Three features of journal interaction were developed, i.e., frequency of interaction across sessions, intensity of interaction, and relative frequency of journal use over other actions. We used k-means cluster analysis on these features and discovered four groups representing different strategies. Regression was used to understand the relationship between reflection strategies and design outcomes. Finally, we draw out implications for supporting pre-college students' productive beginnings of engagement in reflection and future study directions.
... Some of the OELEs are associated with standardized assessments, such as the Programme for International Student Assessment (PISA) and the National Assessment of Educational Progress (NAEP) (eg, PISA (Teig et al., 2020), NAEP (Chu & Leighton, 2019)). Furthermore, some OELEs are game-like, wrapping the scenarios to explore in a story (eg, Crystal Island (Taub et al., 2018), VPA (Jiang et al., 2015), Energy3D (Vieira et al., 2018), TugLet (Käser et al., 2017), BioWorld (Doleck et al., 2016a)), while other OELEs are more closely associ- ...
... At the same time, aggregated features based on raw actions alone were found to have limited predictive power for target competencies (Fratamico et al., 2017;Pedro et al., 2012;Wang, Salehi, et al., 2021). One way to address this challenge is through combining sub-sequences of discrete actions into semantically meaningful units, or events, and counting the instances of these units, such as experiment trials set up by students to collect data (Vieira et al., 2018) or test hypotheses . These features are informed by researchers' knowledge of theoretical frameworks or observations of students working through the task, thus better aligned with the underlying cognitive processes. ...
Technology‐based, open‐ended learning environments (OELEs) can capture detailed information of students' interactions as they work through a task or solve a problem embedded in the environment. This information, in the form of log data, has the potential to provide important insights about the practices adopted by students for scientific inquiry and problem solving. How to parse and analyse the log data to reveal evidence of multifaceted constructs like inquiry and problem solving holds the key to making interactive learning environments useful for assessing students' higher‐order competencies. In this paper, we present a systematic review of studies that used log data generated in OELEs to describe, model and assess scientific inquiry and problem solving. We identify and analyse 70 conference proceedings and journal papers published between 2012 and 2021. Our results reveal large variations in OELE and task characteristics, approaches used to extract features from log data and interpretation models used to link features to target constructs. While the educational data mining and learning analytics communities have made progress in leveraging log data to model inquiry and problem solving, multiple barriers still exist to hamper the production of representative, reproducible and generalizable results. Based on the trends identified, we lay out a set of recommendations pertaining to key aspects of the workflow that we believe will help the field develop more systematic approaches to designing and using OELEs for studying how students engage in inquiry and problem‐solving practices.
Practitioner notes
What is already known about this topic Research has shown that technology‐based, open‐ended learning environments (OELEs) that collect users' interaction data are potentially useful tools for engaging students in practice‐based STEM learning.
More work is needed to identify generalizable principles of how to design OELE tasks to support student learning and how to analyse the log data to assess student performance.
What this paper adds We identified multiple barriers to the production of sufficiently generalizable and robust results to inform practice, with respect to: (1) the design characteristics of the OELE‐based tasks, (2) the target competencies measured, (3) the approaches and techniques used to extract features from log files and (4) the models used to link features to the competencies.
Based on this analysis, we can provide a series of specific recommendations to inform future research and facilitate the generalizability and interpretability of results: Making the data available in open‐access repositories, similar to the PISA tasks, for easy access and sharing.
Defining target practices more precisely to better align task design with target practices and to facilitate between‐study comparisons.
More systematic evaluation of OELE and task designs to improve the psychometric properties of OELE‐based measurement tasks and analysis processes.
Focusing more on internal and external validation of both feature generation processes and statistical models, for example with data from different samples or by systematically varying the analysis methods.
Implications for practice and/or policy Using the framework of evidence‐centered assessment design, we have identified relevant criteria for organizing and evaluating the diverse body of empirical studies on the topic and that policy makers and practitioners can use for their own further examinations.
This paper identifies promising research and development areas on the measurement and assessment of higher‐order constructs with process data from OELE‐based tasks that government agencies and foundations can support.
Researchers, technologists and assessment designers might find useful the insights and recommendations for how OELEs can enhance science assessment through thoughtful integration of learning theories, task design and data mining techniques.
... Previous work has characterized students' strategies for generating ideas and conducting experiments in engineering design thinking (Goldstein et al., 2015;Seah & Magana, 2019;Vieira et al., 2018). For instance, Seah and Magana (2019) identified and characterized patterns of students' experimentation strategies while students were designing an energyefficient house using a CAD tool. ...
... For instance, Seah and Magana (2019) identified and characterized patterns of students' experimentation strategies while students were designing an energyefficient house using a CAD tool. Following a think-aloud protocol, Vieira et al. (2018) explored the connections between students' explanations and the process data generated by the software as students performed experiments while completing a design task. Goldstein et al. (2015) focused on idea fluency by investigating the patterns of students' design behaviors in a design challenge (Goldstein et al., 2015). ...
In this exploratory study, we investigated students’ design thinking strategies during a challenge involving the design of an energy-efficient house. We used the Informed Design Teaching and Learning Matrix as a framework for characterizing the students’ design thinking, focusing on four specific strategies—generating ideas, conducting experiments, revising and iterating, and troubleshooting. To elicit the use of design thinking strategies, we employed two pedagogical approaches—tell-and-practice (T&P) and contrasting cases (CC)—as conditions in a within-subjects design, where participants were exposed to one approach first and then the other. Findings suggest that students exposed to T&P then CC had more balanced use of all four design strategies as compared to the students exposed to CC first then T&P. Regarding changes in strategies used, there was a significant increase in conducting experiments, but a significant decrease in troubleshooting, after students were exposed to both approaches. This finding suggests that students spent more time experimenting and understanding how the system works rather than focusing on problematic areas and finding solutions to the problems they faced during the design process. Implications of the study include recommendations for using T&P and CC to elicit design strategies during design thinking.
... In general, these situations contain problems that must be resolved. Design is the process of using scientific and engineering knowledge to solve technological challenges and optimize solutions within a set of requirements and restrictions (Arastoopour Irgens et al., 2017;Vieira, Seah & Magana, 2018). Because engineering is the application of scientific and mathematical principles to real-world problems (Svarovsky & Shaffer, 2007), engineering design activities can provide a rich environment for students to gain fundamental science skills and understandings while working on projects that are personally significant to them (Buber & Unal Coban, 2020;Fortus, Reddy, & Dershimer, 2003). ...
... In interdisciplinary education, these relationships have not been explicitly examined. Although studies such as Yu et al. (2020) investigate the relationships between scientific knowledge, CT, and design thinking through modeling, they do not explicitly address students' engagement in practice and the role of CT in this engagement.However, there are studies that highlight students' challenges in applying scientific content knowledge to engineering design through inquiry (Vieira et al., 2018) and in using mathematics and scientific knowledge in modelling the design product (Magana, 2017). Therefore, the next step of the project will allow us to address our initial hypothesis by implementing the activities in primary pre-service teacher education at USC. ...
... In general, these situations contain problems that must be resolved. Design is the process of using scientific and engineering knowledge to solve technological challenges and optimize solutions within a set of requirements and restrictions (Arastoopour Irgens et al., 2017;Vieira, Seah & Magana, 2018). Because engineering is the application of scientific and mathematical principles to real-world problems (Svarovsky & Shaffer, 2007), engineering design activities can provide a rich environment for students to gain fundamental science skills and understandings while working on projects that are personally significant to them (Buber & Unal Coban, 2020;Fortus, Reddy, & Dershimer, 2003). ...
... In interdisciplinary education, these relationships have not been explicitly examined. Although studies such as Yu et al. (2020) investigate the relationships between scientific knowledge, CT, and design thinking through modeling, they do not explicitly address students' engagement in practice and the role of CT in this engagement.However, there are studies that highlight students' challenges in applying scientific content knowledge to engineering design through inquiry (Vieira et al., 2018) and in using mathematics and scientific knowledge in modelling the design product (Magana, 2017). Therefore, the next step of the project will allow us to address our initial hypothesis by implementing the activities in primary pre-service teacher education at USC. ...
P2D project (Progression and Pedagogy of Design [P2D]: Contextualizing Design based Pedagogy in Teacher Education Programs) is a 30-month project granted by the European Commission’s Erasmus+ program (Award# 2020-1TR01-KA203-094180). This project is
built on a new framework called Design based Pedagogical Content Knowledge (DPCK).
DPCK was a four-year journey to search for ways of supporting design-based pedagogy
in teacher education (Delen et al., 2020). The idea of DPCK was to build a coherent framework that could be implemented in other universities and classrooms. First of all,
I would like to thank all my colleagues for participating in numerous discussions in the
Promoting Instructional Coherence through Science Teacher Education (PICoSTE) Erasmus+ project (Award # 2017-1-DE03- KA201-035669). All of these discussions aided in the formation of our ideas about the DPCK. Next, I would like to thank my colleagues in the P2D project for exploring how to implement this new framework in four different universities and countries. The booklet will offer you a detailed baseline for the need to switch our discussion to teacher education. All chapters in this booklet will examine the
need to emphasize design-based pedagogy in teacher education. To accomplish this, we will begin Chapter 1 with a general description of pedagogy in the design education literature and how the DPCK concept is related to the existing body of literature. The following chapters in the book are closely linked with the DPCK framework (see Figure 1). The main emphasis of DPCK is putting the design challenges and products upfront. Connected with this goal, Chapter 2 and Chapter 3 will discuss how European studies
focused on design challenges and design products. The second emphasis of DPCK is to integrate and connect the three dimensions of scientific knowledge (disciplinary core ideas, crosscutting concepts and scientific/engineering practices). Chapter 4 will bring a new perspective to this process by integrating critical thinking into scientific practices and the design process. Chapter 4 also introduces our activity framework and this chapter also includes activities designed for integrating critical thinking with different core ideas. Chapter 2 and Chapter 3 will present examples of design challenges and products that support critical thinking. Then, Chapter 5 will discuss how we can create/adapt integrated assessment strategies/tools when implementing design pedagogy in teacher education
programmes.
All chapters in this book will provide a short overview of the literature to make it clear that a focus on developing pre-service teachers’ DPCK is missing in teacher education. Chapter 2 and Chapter 3 provide a review from European perspective to discuss the scarcity of design challenges/products. Chapter 4 and Chapter 5 review the literature from a broader perspective since there were very few examples linked to critical thinking and assessment. The final Chapter 6 concludes with a summary of P2D activity framework and our initial ideas for assessment. In the 2021-2022 academic year, the
examples presented in this book will be implemented in four different countries and in teacher education programs throughout Europe. This book will serve as the baseline for project activities and next year, project partners will work on utilizing the pedagogy of design across different countries and teacher education programs. Activities presented from Chapter 2 to Chapter 4 will be tested (Section 4 in these chapters include the activities). As stated in Chapter 5, new activities and assessment tools will be created during these implementations. Based on the results of our implementation, Chapter 6 will
be revised. The second edition of the book will also include different case studies to discuss the design-based pedagogy implementation in four different countries.
... With its increasing importance and popularity in education, research efforts have been put into better understanding the different design strategies applicable to engineering design, challenges in learning them, as well as pedagogical strategies to teach them (e.g., Atman, Cardella, Turns, & Adams, 2005;Crismond & Adams, 2012). Even though there has been some efforts on assessing idea fluency and experimentation through the student design process by exploring design replays and process data (Goldstein, Purzer, Mejia, Zielinski, & Douglas, 2015;Vieira, Hathaway Goldstein, Purzer, & Magana, 2016;Vieira, Seah, & Magana, 2018), efforts on operationalizing design strategies as forms of assessment or formative feedback, still remain largely unexplored. ...
Various research efforts have been put into understanding design strategies performed by novice designers versus informed designers, misconceptions and challenges in design, as well as design teaching strategies, just to name a few. However, the work on operationalizing design strategies as a form of assessment still remains largely unexplored. In this study, we aimed to (a) investigate ways to operationalize design strategies using process data, and (b) study the interplay between students' design strategies use and their design performance. The design strategies we targeted were generating ideas and troubleshooting.
... Learning analytics, for example, does not necessarily require large data sets because it essentially means collecting data on the interaction of the user/ student with an educational tool. For example, Vieira et al. [39] collected students' interaction with an educational CAD tool to characterize students' engineering design process by analyzing their decisions and actions on building an energy-efficient house. These text files were relatively small (i.e., kilobytes) but provided rich information about students' processes. ...
This study proposes and demonstrates how computer‐aided methods can be used to extend qualitative data analysis by quantifying qualitative data, and then through exploration, categorization, grouping, and validation. Computer‐aided approaches to inquiry have gained important ground in educational research, mostly through data analytics and large data set processing. We argue that qualitative data analysis methods can also be supported and extended by computer‐aided methods. In particular, we posit that computing capacities rationally applied can expand the innate human ability to recognize patterns and group qualitative information based on similarities. We propose a principled approach to using machine learning in qualitative education research based on the three interrelated elements of the assessment triangle: cognition, observation, and interpretation. Through the lens of the assessment triangle, the study presents three examples of qualitative studies in engineering education that have used computer‐aided methods for visualization and grouping. The first study focuses on characterizing students' written explanations of programming code, using tile plots and hierarchical clustering with binary distances to identify the different approaches that students used to self‐explain. The second study looks into students' modeling and simulation process and elicits the types of knowledge that they used in each step through a think‐aloud protocol. For this purpose, we used a bubble plot and a k‐means clustering algorithm. The third and final study explores engineering faculty's conceptions of teaching, using data from semi‐structured interviews. We grouped these conceptions based on coding similarities, using Jaccard's similarity coefficient, and visualized them using a treemap. We conclude this manuscript by discussing some implications for engineering education qualitative research.
... The following is a list of data mining methods that can be applied to learning analytics [40][41][42][43]. A student's relationships with other students and teachers in a learning setting can be used to predict how well they will succeed. ...
Data mining is a method for extracting information from enormous amounts of data. Data mining, also known as knowledge discovery from data, is the swift and straightforward detection of patterns that hint to knowledge that has been implicitly stored or recorded in big databases, data warehouses, the web, other massive data repositories, or information streams. In this essay, several data mining theories, approaches, tactics, and applications are discussed. The purpose of modern management promotion today is to improve the problem of inaccurate information transmission and increase productivity. The primary goal of contemporary administration is to increase the university potential to develop talent and serve society. The primary focus of this work is on the information management systems that colleges and universities construct utilizing data mining. For many disciplines in higher education, data mining offers useful solutions. Due to the abundance of student data that may be utilized to identify illuminating trends on how students learn, the area of education research is continuously growing. To evaluate student performance and assist them in showcasing the students' accomplishments, educational institutions might use educational data mining. This paper reviews various techniques used as knowledge extractors to tackle specific education challenges from large data sets of higher education institutions to the benefit of all educational stakeholders.
... Consequently, DS has become a crucial tool for the research and educational community [3]. Educational research has explored diverse topics using DS tools, including student admission [72] and retention [71], as well as student approaches to problem-solving [65]. However, less is known about the processes involved in DS education itself, with the persistent question still being "How do we train [a] workforce of professionals who can use data to its best advantage?" ...
This study implements a computational cognitive apprenticeship framework for knowledge integration of Data Science (DS) concepts delivered via computational notebooks. This study also explores students' conceptual understanding of the unsupervised Machine Learning algorithm of K‐means after being exposed to this method. The learning of DS methods and techniques has become paramount for the new generations of undergraduate engineering students. However, little is known about effective strategies to support student learning of DS and machine learning (ML) algorithms. The research questions are: How do students conceptualize their understanding of an unsupervised ML method after engaging with interactive visualizations designed using the computational cognitive apprenticeship approach? How do the affordances of the interactive visualizations support or hinder student knowledge integration of an unsupervised machine learning method? Design‐based research allowed for the iterative design, implementation, and validation of the pedagogy in the context of a working classroom. For this, data collection methods often take the form of student artifacts. We performed a qualitative content analysis of students' written responses and reflections elicited during the learning process. Results suggest that the computational cognitive apprenticeship promoted knowledge integration. After interacting with the computational notebooks, most students had accurate conceptions of the goal and the nature of the method and identified factors affecting the output of the algorithm. Students found it useful to have a concrete representation of the method, which supported its conceptual understanding and showcased the acquisition of strategic knowledge for its appropriate execution. However, we also identified important misconceptions students held about the algorithm.
... Así, diseñar y llevar a cabo experimentos controlados es una habilidad que todos los estudiantes deberían desarrollar. La Figura 11 muestra un continuo de los experimentos que logran implementar diseñadores con diferentes niveles de experiencia (Vieira, Seah y Magana, 2018). A menudo, los diseñadores principiantes no generan hipótesis ni implementan experimentos para probar sus hipótesis (Crismond y Adams, 2012). ...
... Así, diseñar y llevar a cabo experimentos controlados es una habilidad que todos los estudiantes deberían desarrollar. La Figura 11 muestra un continuo de los experimentos que logran implementar diseñadores con diferentes niveles de experiencia (Vieira, Seah y Magana, 2018). A menudo, los diseñadores principiantes no generan hipótesis ni implementan experimentos para probar sus hipótesis (Crismond y Adams, 2012). ...
... In general, these situations contain problems that must be resolved. Design is the process of using scientific and engineering knowledge to solve technological challenges and optimize solutions within a set of requirements and restrictions (Arastoopour Irgens et al., 2017;Vieira, Seah & Magana, 2018). Because engineering is the application of scientific and mathematical principles to real-world problems (Svarovsky & Shaffer, 2007), engineering design activities can provide a rich environment for students to gain fundamental science skills and understandings while working on projects that are personally significant to them (Buber & Unal Coban, 2020;Fortus, Reddy, & Dershimer, 2003). ...
This chapter will discuss how European studies focused on design challenges and design products. It will present examples of design challenges and products that support critical thinking. It provide a review from European perspective to discuss the scarcity of design challenges/products.
... Aladdin (formerly known as Energy3D) is a freely available, opensource CAD program that has been developed specifically to help students learn about designing energy-e icient buildings and renewable energy solutions (Xie, Schimpf, Chao, Nourian, & Massicotte, 2018). It also allows education researchers to study actions that students perform within the platform (e.g., Seah & Magana, 2019;Vieira, Seah, & Magana, 2018). Since the software is easy to use and has built-in tutorials, students can quickly learn how to model a simple building (e.g., see Figure 1). ...
Engineering learning, a three-dimensional construct that includes Engineering Habits of Mind, Engineering Practices, and Engineering Knowledge, has been well established and defined at the post-secondary level (Reed, 2018). Meanwhile, engineering within pre-kindergarten through 12th grade (P-12) classrooms continues to grow steadily. Changes introduced by A Framework for K-12 Science Education (National Research Council, 2012) and Next Generation Science Standards (NGSS Lead States, 2013) have started to place engineering within secondary science education, just as the inclusion of engineering design in Standards for Technological Literacy did within technology education classrooms at the turn of the century (International Technology and Engineering Educators Association, 2000/2002/2007). More and more students are now exposed to engineering learning prior to graduating from high school in a variety of courses like technology, science, and career/technical education classrooms, as well as informal learning programs. Nevertheless, engineering in its own right often remains a missing or minimal component of the learning experience for many students (Change the Equation, 2016; Miaoulis, 2010). To include engineering in a more prominent manner, the Framework for P-12 Engineering Learning (2020) has recently been published as a practical guide for developing coherent, authentic, and equitable engineering learning programs across schools. This guidance includes a definition of the three dimensions of engineering learning, principles for pedagogical practice, and common learning goals. The framework can support the development of in-depth and authentic engineering learning initiatives and provide building blocks toward the 2020 Standards for Technological and Engineering Literacy. As a component of the framework, engineering practices are detailed by describing core concepts that can support performing these practices with increased sophistication over time. Examples include making data-informed design decisions based on material properties and employing computational tools to analyze data to assess and optimize designs. In this Engineering in Action article, we introduce a freely available, open-source computer-aided design (CAD) software called Aladdin and discuss how it can support authentic engineering practice within secondary classrooms. Earlier works have suggested that Aladdin is an effective tool for implementing Next Generation Science Standards (e.g., Chao et al., 2018; Goldstein, Loy, & Purzer, 2017). Similarly, we make a case for using Aladdin in secondary engineering education and discuss how recommendations of the Framework for P-12 Engineering Learning map to specific features of the software.
... It is often not enough to assess students' proficiency in design thinking only from the final product, as one needs to interpret all the steps taken to make the final product to solve a problem. There have been some studies that characterized students' experimentation and generating ideas strategies in engineering design [3,4,5]. However, students' use of four design thinking strategies at the same time has not been assessed and categorized based on their strategic performance. ...
... Given the complexity of the design process, visualization techniques are commonly deployed to both characterize designers' processes and enable comparisons of how designers performed on a given task [21][22][23][24][25][26] . These visualizations may seek to visualize a designer's entire design process or relevant aspects of their design process. ...
Abstract: In design, reflection is a central practice that helps designers evaluate past strategies, synthesis knowledge they've gained and plan future actions. For novice designers, developing reflection abilities may be particularly important as it may both help them develop this specific ability and more broadly develop their design thinking abilities. However, the design process is fluid with distinct design stages that may happen in varying order and repeat or cycle in a sequence unique to the design context and designers involved. Reflection similarly can happen in different patterns across the design process. Thus, it is not clear when students may reflect. While past work in design has studied several aspects of reflection such as students' reflective practices or how reflection relates to design quality outcomes, less work has looked at how reflection is distributed over the design process. We seek to address this gap by proposing to analyze designers' reflection process through a data visualization approach to generate reflection plots of students' reflection activities in context with other key design activities. Data from a class of 28 students in a Midwestern middle school is visualized to discover what key factors distinguish how students reflect over their design processes and to uncover unique modes of how novices reflect temporally. Results identify four factors that distinguish when designers reflect: timing, duration, intensity and interweaving (with other activities). An examination of four cases unveil distinct ways in which students' reflection process can be characterized holistically. These results indicate there is high variability in regards to when and how students reflect, which can be used by design educators and researchers to better support student's future design learning.
... However, it has been only in recent years that educational researchers have investigated the potential of engaging students in engineering design practices via computer simulations for integrated science and engineering learning (Dasgupta et al. 2019;Seah and Magana 2019;Vieira et al. 2018;Xie et al. 2018). Some of this research suggests that by engaging in experimentation practices through design challenges, students may develop some science learning. ...
In science and engineering education, the use of heuristics has been introduced as a way of understanding the world, and as a way to approach problem-solving and design. However, important consequences for the use of heuristics are that they do not always guarantee a correct solution. Learning by Design has been identified as a pedagogical strategy that can guide individuals to properly connect science learning via design challenges. Specifically, we focus on the effect of simulation-enabled Learning by Design learning experiences on student-generated heuristics that can lead to solutions to problems. A total of 318 middle school students were exposed to a lesson that integrated design practices in the context of energy consumption and energy conservation considerations when designing buildings using an educational CAD tool. The students were pre- and posttested before and after the 2-week long intervention. The data analysis procedures combined qualitative with quantitative methods along with machine learning approaches. Our analysis revealed two distinct groups of students based on their learning achievement: the naive developing heuristic group and semi-knowledgeable fixated heuristic group. Differences between the groups are discussed in terms of performance, as well as implications for the use of computer simulations to improve student learning.
... Students often follow a trial and error method for evaluating multiple ideas (Schauble et al., 1991). The goal, for the students, is to pursue a systematic experimentation approach using the available resources (e.g., Vieira, Seah, & Magana, 2018). Technology in the form of simulation software has been found to support this experimentation process by providing repeated opportunities for simulating, testing 'what-if' conditions, and evaluating the proposed solutions (Kolodner et al., 2003). ...
There has been an increased emphasis on designing integrated STEM learning environments for K-12 students that facilitate seamless learning of disciplinary concepts infused with science inquiry, engineering design, mathematical reasoning, and technological skills. However, there is limited prior research investigating how to facilitate such integrated STEM learning in formal classrooms that go beyond a simple combination of the different subject areas and instead enable teaching and learning of disciplinary concepts infused with scientific inquiry, engineering design, mathematical reasoning, and 21st century technological skills. In this paper, we investigate the affordances of using an educational Computer Aided Design tool, Energy 3D, and corresponding curricular materials to support such integrated STEM learning anchored in the engineering design process. We present an exploratory case study that was conducted in a middle school in the US, where a project-based learning approach was followed and students were asked to design a low-cost energy-efficient home within a given budget using the Energy3D CAD tool. Findings indicate that students learned to engage in the design process and demonstrated practices of idea fluency and systematic experimentation; practices usually representative of informed designers. During the design process while analyzing the problem space, generating ideas, and evaluating solutions, they developed better understanding of the relationships between variables and underlying science concepts, used various mathematical analysis tools and graphical representations embedded in the available technology to inform their engineering design decisions. The learning environment using Energy3D afforded formative feedback to help students understand relationship between variables, provided converging evidences using multiple analytical tools, and enabled visual problem decomposition using suboptimal model to engage students in integrated STEM learning. This study provides a platform for future research investigating the effectiveness of educational CAD tools and curricular scaffolds designed specifically for K-12 students for supporting integrated STEM learning anchored in the design process.
Programming‐related courses prefer to put importance on practice rather than theoretical knowledge. However, the students' programming behavior data is difficult to be recorded and fed back to teachers anytime, allowing teachers to adjust their teaching in real‐time. Aiming at this problem, this study develops an online integrated programming platform that provides a unified programming environment for teachers and students and logs students' procedural programming behavior data automatically to assist teachers in providing real‐time feedback teaching. To verify the effectiveness of the data‐driven immediate feedback teaching method, an exploratory case was conducted in the 2020/2021 academic year autumn semester. As the control group, cloud201 adopted the traditional hybrid online/offline mode, while cloud202, as the experimental group, adopted the hybrid online/offline mode and immediate feedback teaching mode. The effect is demonstrated through the questionnaire, the programming behavior data analysis, and the performance analysis. The results of the questionnaire show that there are significant differences in the students' satisfaction with interaction, supervision, feedback, and evaluation, of which feedback is the biggest part. In addition, the two analysis results indicate that students' individual learning development and the learning status of the class are objectively characterized, the overall grade distribution of cloud202 is more concentrated in medium grades and the difference on average scores between cloud201 and cloud202 is gradually increased. Since the students' immediate and comprehensive programming behavior data is perceived by teachers, the teaching adjustment is so flexible that the improvement of teaching efficiency and teaching effect is promoted accordingly.
Tasarım Temelli Pedagoji kitabının çevirisi Uşak Üniversitesi ve Dokuz Üniversitesi'ndeki araştırmacılar tarafından tamamlanmıştır. Kitaba proje sayfamızdan da erişebilirsiniz:
http://p2dproject.eu/index.php/ciktilar/
With an increased emphasis on science, technology, engineering, and mathematics (STEM) education, educators have often adopted engineering projects for integrating interdisciplinary knowledge through engineering design process. However, there is scant research that has empirically documented the relationships among students’ understanding of the knowledge and their characteristics on the design product. This study uses structural equation modeling to examine the influence of students’ scientific knowledge, design process, and critical thinking on their design product of the engineering project. A theoretical model was constructed to identify the factors that affected the design product. Study data were collected from 613 high school students, aged 16–17 years, regarding their scientific knowledge, design processes, and critical thinking. The results showed that design process played a mediating role in the effects of knowledge and critical thinking on the final design product. Students who could deduce, explain, and evaluate had an increased ability to apply scientific knowledge during the design process. These results indicate that scientific knowledge has an effect on the process and products of engineering projects completed by high school students. This study also suggested that the science and technology education community should consider offering high school students learning opportunities to promote their cognitive skills and facilitate connections between knowledge and practice.
Since Fall 2000, mechanical engineering students at Montana State University have been required to keep design journals of their senior design projects. We have now accumulated over 70 journals on 21 design projects. We developed a coding scheme to code the journal data by design activity (problem definition, idea generation, engineering analysis, and design refinement), design level (concept, system, detail), planning, and formal reporting. The scheme was then used to code approximately one-third of the journal data collected to date.
This paper will first describe the coding scheme and its development. It will then report some preliminary findings from the journal coding. Specifically, we will show that journal data can be used to produce time profiles of design activity over the course of the projects, estimates of the proportion of time spent in activity associated with the different codes, and comparisons of student processes to “good” design processes as documented in the literature. These findings should be of interest to design educators interested in assessing design processes. Finally, the paper will posit a number of hypotheses that arise from the data, for future consideration.
This paper proposes a novel computational approach based on time series analysis to assess engineering design processes using a CAD tool. To collect research data without disrupting a design learning process, design actions and artifacts are continuously logged as time series by the CAD tool behind the scenes, while students are working on a design challenge. These fine-grained data can be used to reconstruct and analyze the entire design process of a student with extremely high resolution. Results of a pilot study in a high school engineering class, in which students solved a solar urban design challenge, suggest that these data can be used to measure the level of student engagement, reveal the gender differences in design behaviors, and distinguish the iterative and non-iterative cycles in a design process. From the perspective of engineering education, this paper contributes to the emerging fields of educational data mining and learning analytics that aim to expand evidence approaches for learning in a digital world.
This paper provides a theoretical perspective of how modeling and simulation on a CAD platform can be used to teach science concepts and inform design decisions. The paper discusses the educational implications of three recent advancements in CAD technologies: system integration, machine learning, and computational design. The challenges to design energy‐efficient buildings that harness solar energy are used as the engineering examples to illustrate the learning and teaching opportunities created by the modeling, simulation, and data mining capabilities of the Energy3D software, which is a CAD tool developed from scratch along the directions of the three advancements to support engineering research and education. Preliminary results from students in a physics classroom and an online course shed light on the effects of these features on guiding student to design cost‐effective rooftop solar power systems for their own home buildings.
In this paper, we review computer‐based assessment for learning (CBAfL), in elementary and secondary education, as a viable way to merge instruction and assessment of students' developing proficiencies. We begin by contextualizing our topic relative to summative and formative assessment before presenting the current literature, which we categorized into the following: (a) supplementary use in classrooms, (b) web‐based, and (c) data‐driven, continuous CBAfL. Examples of research studies per category are provided. Findings show that using CBAfL in the classroom, via the Internet, or embedded in a game, generally enhances learning and other outcomes across a range of content areas (e.g. biology, math, and programming). One conclusion is that feedback, to be most beneficial to learning, should not be overly complex and must be used to be effective. Findings also showed that the quality of the assessment (i.e. validity, reliability, and efficiency) is unimpaired by the inclusion of feedback. The possibilities created by advances in the learning sciences, measurement, and technology have paved the way toward new assessment approaches that will support personalized learning and that can accurately measure and support complex competencies. The next steps involve evaluating the new assessments regarding their psychometric properties and support of learning.
Lay Description
What is currently known about computer‐based assessment for learning (CBAfL)? Early CBAfL systems were divided into linear and branching programs with no diagnostics and evolved into systems possessing more personalized/adaptive remediation with AI.
Current CBAfL can support a range of competencies in various digital environments.
Advanced learning analytic methods include learning analytics and stealth assessment.
What our paper adds to what is already known about CBA for learning? Trends in our review suggest CBAs will improve in personalizing learning in a variety of contexts.
Innovative CBAfL techniques will move beyond the laboratory and into the mainstream.
Boundaries between instruction, learning and assessment will eventually become blurred, thus removing the need for high‐stake tests of learning.
What are the implications of our topic for practitioners? With CBAfL advances, teachers will have more time to provide targeted support to learners.
Students would not need to worry about taking exams if CBAfL is continuous and formative.
Educators will be able to provide personalized learning experiences for diverse students.
Students will be equipped with the knowledge and skills needed to succeed in the 21st century.
Learning analytics and educational data mining are introducing a number of new techniques and frameworks for studying learning. The scalability and complexity of these novel techniques has afforded new ways for enacting education research and has helped scholars gain new insights into human cognition and learning. Nonetheless, there remain some domains for which pure computational analysis is currently infeasible. One such area, which is particularly important today, is open-ended, hands-on, engineering design tasks. These open-ended tasks are becoming increasingly prevalent in both K–12 and post-secondary learning institutions, as educators are adopting this approach in order to teach students real-world science and engineering skills (e.g., the “Maker Movement”). This paper highlights findings from a combined human–computer analysis of students as they complete a short engineering design task. The study uncovers novel insights and serves to advance the field’s understanding of engineering design patterns. More specifically, this paper uses machine learning on hand-coded video data to identify general patterns in engineering design and develop a fine-grained representation of how experience relates to engineering practices. Finally, the paper concludes with ideas on how the specific findings from this study can be used to improve engineering education and the nascent field of “making” and digital fabrication in education. We also discuss how human–computer collaborative analyses can grow the learning analytics community and make learning analytics more central to education research.
This paper aims to provide the reader with a comprehensive background for understanding current knowledge on Learning Analytics (LA) and Educational Data Mining (EDM) and its impact on adaptive learning. It constitutes an overview of empirical evidence behind key objectives of the potential adoption of LA/EDM in generic educational strategic planning. We examined the literature on experimental case studies conducted in the domain during the past six years (2008-2013). Search terms identified 209 mature pieces of research work, but inclusion criteria limited the key studies to 40. We analyzed the research questions, methodology and findings of these published papers and categorized them accordingly. We used non-statistical methods to evaluate and interpret findings of the collected studies. The results have highlighted four distinct major directions of the LA/EDM empirical research. We discuss on the emerged added value of LA/EDM research and highlight the significance of further implications. Finally, we set our thoughts on possible uncharted key questions to investigate both from pedagogical and technical considerations.
Engineering design thinking combines concepts from heterogeneous sources like personal experience, colleagues, digital and hardcopy media. Despite this challenge, modes of thinking across levels of abstraction through multi-dimensional (spatial) representations are widely neglected in digital support systems. This paper aims to summarize lessons learned through years of experience with software tools that augment this visio-spatial conceptual thinking. This work cuts across disciplines to provide a needed, coherent starting point for other researchers to examine complex outstanding issues on a class of promising support tools which have yet to gain widespread popularity. Three studies are used to provide specific examples across design phases, from conceptual design to embodiment. Each study also focuses on an exemplar of diagrammatic software: the University of Cambridge Design Rationale editor (DRed), the Institute for Human Machine Cognition’s (IHMC) CmapTools and the Open University’s Compendium hypermedia tool. This synthesis reiterates how hypermedia diagrams provide many unique, valuable functions while indicating important practical boundaries and limitations. Future research proposed includes: a need to build more diagrammatic literacy into engineering practice, the need for more detailed studies with experts in industry and specific directions for refining the hypermedia diagram software interfaces.
This paper is based on the premises that the purpose of
engineering education is to graduate engineers who can design,
and that design thinking is complex. The paper begins by briefly
reviewing the history and role of design in the engineering
curriculum. Several dimensions of design thinking are then
detailed, explaining why design is hard to learn and harder still to
teach, and outlining the research available on how well design
thinking skills are learned. The currently most-favored
pedagogical model for teaching design, project-based learning
(PBL), is explored next, along with available assessment data on
its success. Two contexts for PBL are emphasized: first-year
cornerstone courses and globally dispersed PBL courses. Finally,
the paper lists some of the open research questions that must be
answered to identify the best pedagogical practices of improving
design learning, after which it closes by making recommendations
for research aimed at enhancing design learning.
Constructionism can be a powerful framework for teaching
complex content to novices. At the core of constructionism is the suggestion that by enabling learners to build creative artifacts that require complex content to function, those learners will have opportunities to learn this content in contextualized, personally meaningful ways. In this paper, we investigate the relevance of a set of approaches broadly called “educational data mining” or “learning analytics” (henceforth, EDM) to help provide a basis for quantitative research on constructionist learning which does not abandon the richness seen as essential by many researchers in that paradigm. We suggest that EDM may have the potential to support research that is meaningful and useful both to researchers working actively in the constructionist tradition but also to wider communities. Finally, we explore potential collaborations between researchers in the EDM and constructionist traditions; such collaborations have the potential to enhance the ability of constructionist researchers to make rich inferences about learning and learners, while providing EDM researchers with many interesting new research questions and challenges.
This paper proposes that Social Learning Analytics (SLA) can be usefully thought of as a subset of learning analytics approaches. SLA focuses on how learners build knowledge together in their cultural and social settings. In the context of online social learning, it takes into account both formal and informal educational environments, including networks and communities. The paper introduces the broad rationale for SLA by reviewing some of the key drivers that make social learning so important today. Five forms of SLA are identified, including those which are inherently social, and others which have social dimensions. The paper goes on to describe early work towards implementing these analytics on SocialLearn, an online learning space in use at the UK's Open University, and the challenges that this is raising. This work takes an iterative approach to analytics, encouraging learners to respond to and help to shape not only the analytics but also their associated recommendations.
Learning journals are implemented in undergraduate engineering design courses to encourage students to reflect actively on what they can learn from their personal experiences with team projects, lectures, and assignments. These journals are intended to catalyze student reflection and thus enhance student understanding, retention, and capacity for future application of course content. An invaluable side-effect of the journals is continuous feedback that enables real-time adjustments to course schedule and pedagogy. Implementation details are reported in this paper, including sample reflective prompts, grading rubrics, and sample journal entries with the corresponding instructor feedback provided. Results of student opinion surveys are also discussed. As described in this paper, our experiences and observations throughout the semester- long course motivate us to continue implementing refined versions of this pedagogical tool in design courses.
The premise of this research is that the engineering design process is partially driven by achieving consensus and reconciling points of view among team members. Characterizing the quality of the design performance by measuring the coherence of the description of related design concepts and events in design documentation is examined. Latent Semantic Analysis (LSA) was used to analyze design documentation written by self-managing, cross-functional engineering design teams. Computational measurements of document variance and textual coherence were applied to the teams' design documents, presentation materials and e-mail communication. The levels of semantic coherence were correlated to assessments by faculty and product designers and engineers from industry of the design teams' process and outcome quality. The results indicated a statistically significant positive correlation between design document coherence and design performance, especially for poorly performing teams. The impact of this research is to provide team managers (people who create teams and manage teams) or self-organizing teams (teams that focus on self-reflection and peer evaluation) computational tools that could be integrated with design information management technologies to assist them in the management of engineering design teams.
With the increase in available educational data, it is expected that Learning Analytics will become a powerful means to inform and support learners, teachers and their institutions in better understanding and predicting personal learning needs and performance. However, the processes and requirements behind the beneficial application of Learning and Knowledge Analytics as well as the consequences for learning and teaching are still far from being understood. In this paper, we explore the key dimensions of Learning Analytics (LA), the critical problem zones, and some potential dangers to the beneficial exploitation of educational data. We propose and discuss a generic design framework that can act as a useful guide for setting up Learning Analytics services in support of educational practice and learner guidance, in quality assurance, curriculum development, and in improving teacher effectiveness and efficiency. Furthermore, the presented article intends to inform about soft barriers and limitations of Learning Analytics. We identify the required skills and competences that make meaningful use of Learning Analytics data possible to overcome gaps in interpretation literacy among educational stakeholders. We also discuss privacy and ethical issues and suggest ways in which these issues can be addressed through policy guidelines and best practice examples.
The widespread availability of computers in elementary schools makes them an appealing option for presenting instructional materials in laboratory science. However, there are neither theoretical nor empirical grounds for predicting whether virtual or physical presentation of instructional materials will be more effective. The definition of "active manipulation" is poorly specified and there are few studies that directly compare the two approaches unaccompanied by other potential confounds. In this study, 4th- and 5th-grade children were taught how to design simple unconfounded experiments using 1 of 2 instructional methods differing only in whether children manipulated physical or virtual materials. The 2 types of materials were equally ef- fective in achieving several instructional objectives, including the design of uncon- founded experiments, the derivation of correct predictions from them, and explicit reference to the need for experiments to be unconfounded.
Computer simulations provide environments enabling exploratory learning. Research has shown that these types of learning environments are promising applications of computer assisted learning but also that they introduce complex learning settings, involving a large number of learning processes. This article reports on an instrument for supporting one of these learning processes: stating hypotheses.
The resulting instrument, an hypothesis scratchpad, was designed on the basis of a conceptual representation of the simulation model and tested in an experimental study. In this study three versions of the scratchpad, varying in structure, were compared. It was found that support offered for identifying variables, in the form of a selection list, is relatively successful: students who used this list were better in differentiating different types of variables. For identifying relations, a selection list of relations offered to the students proved unhelpful in finding accurate relations: students using this list stated their hypotheses mainly at a very global level.
There is great interest in assessing student learning in unscripted, open-ended environments, but students' work can evolve in ways that are too subtle or too complex to be detected by the human eye. In this paper, I describe an automated technique to assess, analyze and visualize students learning computer programming. I logged hundreds of snapshots of students' code during a programming assignment, and I employ different quantitative techniques to extract students' behaviors and categorize them in terms of programming experience. First I review the literature on educational data mining, learning analytics, computer vision applied to assessment, and emotion detection, discuss the relevance of the work, and describe one case study with a group undergraduate engineering students
Action research refers to the formalized, self-re fl ective research of practitioners. According to Cochran-Smith and Lytle (Inside and outside: Teacher research and knowledge. NY: Teachers College Press, 1993) action research is "systematic and intentional inquiry" (p. 7). It is often conducted collaboratively in research groups that meet in person or at a distance via communication technologies. Action research transforms the traditional "outside-in" relationship between practitioners and the educational community. It can provide a powerful means for bridging the divide between theory and practice and encouraging practitioners to engage in innovative practices. Action research includes a cyclical process of posing questions, collecting data, re fl ecting on findings, and reporting results. This chapter provides a comprehensive overview of action research and its history in the USA, Great Britain, and Australia. It also describes the epistemological and ontological differences between practical and critical action research. To inspire future action research in our field, we detail the action research method, including data collection and analysis techniques and provide example studies from the field of educational communications and technology. More specifically, we demonstrate the manner in which action research has already been used to better understand the impact of the integration of technology in classrooms and social settings. At the same time, we describe how action researchers have used educational communications and technology to conduct action research and to teach this research method through online or hybrid classes. Technology can be both the focus and part of the method of the action research. © Springer Science+Business Media New York 2014. All rights reserved.
The ability of future engineering professionals to solve complex real-world problems depends on their design education and training. Because engineers engage with open-ended problems in which there are unknown parameters and multiple competing objectives, they engage in fuzzy decision-making, a method of making decisions that takes into account inherent imprecisions and uncertainties in the real world. In the design-based decision-making field, few studies have applied fuzzy decision-making models to actual decision-making process data. Thus, in this study, we use datasets on student decision-making processes to validate approximate fuzzy models of student decision-making, which we call data-enabled cognitive modeling. The results of this study (1) show that simulated design problems provide rich datasets that enable analysis of student design decision-making and (2) validate models of student design cognition that can inform future design curricula and help educators understand how students think about design problems.
This paper considers the data science and the summaries significance of Big Data and Learning Analytics in education. The widespread platform of making high-quality benefits that could be achieved by exhausting big data techniques in the field of education is considered. One principal architecture framework to support education research is proposed.
Engineering design is a complex process. The design process cannot be assessed based solely on a product or as a simple test because there is no single perfect design for a problem. An important design strategy is the conduction of experiments. Informed designers carry out experiments and use their outcomes to inform their next steps. On the other hand, beginning designers do little or no experiments, and the few experiments they do involve confounding variables. These behaviours that differentiate beginning and informed designers are not easy to assess in educational settings because they occur throughout the design process.
This paper proposes and evaluates a model to analyze student interactions with a CAD tool in order to identify and characterize the different strategies students use to conduct experiments. A two-fold study is carried out to validate the model. The first phase uses the clickstream data of 51 middle school students working on a design project to create a net-zero energy house. The analysis of clickstream data is compared to a qualitative analysis of an open-ended posttest. The second phase correlates the number of experiments students did to the student prototype quality. The results suggest that the proposed model can be used to identify, characterize, and assess student strategies to conduct experiments.
Engineering design is an iterative process that supports the solution of problems by applying scientific knowledge to make informed decisions. Assessing different levels of expertise in experimentation is a difficult task since these are not usually visible as part of a student's final design solution. The purpose of this research is to investigate and characterize students' experimentation strategies while working on a design challenge. We conducted a concurrent think-aloud to capture students' thinking while they were working on a design challenge using an educational computer-aided design (CAD) software. We showed how the design replays generated from the log files collected from the CAD software can be used to represent students' experimentation strategies and how these representations can be validated by the data collected from the think-aloud. Our preliminary results show that technology-based assessment by the educational CAD tool allows us to identify the differences between different experimentation strategies and that the result of this assessment is supported by the result obtained from the concurrent think-aloud. Implications of this work would be relevant to engineering educators and researchers who are interested in understanding and assessing students' experimentation strategies in engineering design.
The article deals with the interplay between learning analytics and massive open online courses (MOOCs) and provides a conceptual framework to situate ongoing research in the MOOC and learning analytics innovation cycle (MOLAC framework). The MOLAC framework is organized on three levels: On the micro-level, the data collection and analytics activities are focused on individual reflection and individual prediction. On the meso-level, data from several open courses are combined to support benchmarking and to create insights about behaviour of groups of learners rather than the individual. These insights can inform the institution to adapt their educational model. On the macro-level of the framework, cross-institutional learning analytics enables to develop learning and teaching interventions that can be tested in a cluster of educational organizations to analyse the impact of these interventions beyond contextual factors. The article proposes four areas of future activities that are needed to enable the MOLAC framework. These are the standardization of the description of the educational design of MOOCs, data sharing facilities across institutions, joint policymaking and ethical guidelines and last but not least standardized evaluation approaches.
Children’s engineering involves design of a solution under specified constraints in response to a particular need or goal. Desktop manufacturing systems enable students to engineer complex solutions with tangible products, expanding the range of possible approaches to engineering education. Desktop manufacturing technologies encompass digital fabrication systems such as 3D printers and computer-controlled die cutting systems and related technologies such as 3D scanners. These systems offer an entry point for advancing children’s engineering as well as connecting to other STEM subjects.
Because desktop manufacturing systems have only recently become affordable in schools and are continuing to evolve rapidly, the conditions under which they may be best used in classrooms are not yet well defined. However, there are several promising directions that may guide future research in this area. The design process involved in desktop manufacturing affords an opportunity for connections among multiple representations. The virtual design on the computer screen and the corresponding physical object that is produced are two representations of the same underlying construct. Negotiating these representations offers connections to mathematics taught in schools such as ratios, proportion, and scaling. Computer-assisted design programs developed as learning tools can capture information about student design choices and underlying thought processes. Construction of physical prototypes through desktop manufacturing involves extensive involvement of motor skills that may have linkages with student achievement. Digital objects and designs developed at one school can be disseminated via the Internet and reproduced at other sites, allowing designs to be shared and adapted for specific educational goals.
This research aims to explore the benefits of a flipped learning approach for students who are taking an introductory-level curriculum on bridge computer aided design in terms of learning attitude and achievement within the curriculum. In this study, collaborative problem based learning (CPBL) supported by flipped learning, a blended learning design, was integrated into a high school bridge computer aided design curriculum. Ninety-one 17-year-old students from two K11 classes were assigned randomly to an experimental group and a control group for the study, respectively. To assess the students' achievements and learning attitudes in the different groups, an 8-week (16 h in total) pre- and post-test quasi-experimental study was designed. The results confirmed the effectiveness of the flipped learning approach. Significant differences were found between the experimental and control group in terms of students' achievements. In the experimental group, students' learning attitudes, motivation and self-evaluation were enhanced. In conclusion, the results show that the flipped learning approach has a positive effect on the transfer of learning. Based on the findings obtained, recommendations for the improvement of future K12 engineering education instruction using the flipped learning approach are provided. © 2015 Wiley Periodicals, Inc. Comput. Appl. Eng. Educ. 9999:1-13, 2015; View this article online at wileyonlinelibrary.com/journal/cae; DOI 10.1002/cae.21622
Abstract The purpose of this study was to investigate value of combining Real Experimentation (RE) with Virtual Experimentation (VE) with respect to changes in students' conceptual understanding of electric circuits. To achieve this, a pre–post comparison study design was used that involved 88 undergraduate students. The participants were randomly assigned to an experimental (45 students) and a control group (43 students). Each group attended a one semester course in physics for preservice elementary school teachers. Both groups used the same inquiry-based curriculum materials. Participants in the control group used RE to conduct the study's experiments, whereas, participants in the experimental group used RE in the first part of the curriculum and VE in another part. Conceptual tests were administered to assess students' understanding of electric circuits before, during and after the teaching intervention. Results indicated that the combination of RE and VE enhanced students' conceptual understanding more than the use of RE alone. A further analysis showed that differences between groups on that part of the curriculum in which the experimental group used VE and the control group RE, in favour of VE.
Students' creativity is an important focus globally and is interrelated with students' spatial abilities. Additionally, three-dimensional computer-assisted drawing (3D-CAD) overcomes barriers to spatial expression during the creative design process. Does 3D-CAD affect students' creative abilities? The purpose of this study was to explore the auxiliary effects of 3D-CAD applications on students' creative design capabilities and on students with different spatial abilities. A total of 349 students from a public senior high school in Taoyuan County, Taiwan, participated in the study. The study used a non-equivalent pretest and post-test quasi-experimental design. The key results from the study include the following: (1) students' spatial abilities were moderately correlated with their creative performance, especially their functional creativity; (2) the 3D-CAD applications enhanced students' creative performance, particularly with regard to aesthetics; and (3) in 3D-CAD applications, students with better spatial abilities were superior to those with relatively poor spatial abilities with regard to creative performance. Additionally, the effect sizes of novelty and aesthetics were stronger than that of functionality.
This study offers new insights into the ongoing debate about whether physical and virtual materials are equally effective in inquiry-based science instruction. Physical materials were predicted to have a surplus value when haptic feedback helps discern object characteristics or when the perceived credibility of experimental data can impede conceptual change. Both assumptions were tested by comparing the belief revisions and confidence ratings of children (n = 60) engaged in an inquiry task about falling objects. Children were assigned to one of three instructional conditions that differed with regard to the type of materials and the possibility to manipulate those materials. Main findings confirmed the alleged benefits of physical manipulation in correcting misconceptions about object characteristics that are perceived by touch. Belief revision about visually discernible characteristics proved independent of the type of material and type of manipulation, as was children's confidence in their post-instructional beliefs. Together, these findings indicate that tactile cues derived from physical manipulation can have a unique contribution to children's science learning.
Background: Design experiences play a crucial role in undergraduate engineering education and are increasingly important in K-12 settings. There are few efforts to purposefully connect research findings on how people design with what teachers need to understand and do to help K-16 students improve their design capability and learn through design activities. Purpose: This paper connects and simplifies disparate findings from research on design cognition and presents a robust framework for a scholarship of design teaching and learning that includes misconceptions, learning trajectories, instructional goals, and teaching strategies that instructors need to know to teach engineering design effectively. Method: A scholarship of integration study was conducted that involved a meta-literature review and led to selecting and bounding students' design performances with appropriate starting points and end points, establishing key performance dimensions of design practices, and fashioning use-inspired tools that represent design pedagogical content knowledge for teachers. Results: The outcome of this scholarship of integration effort is the Informed Design Learning and Teaching Matrix that contains nine engineering design strategies and associated patterns that contrast beginning versus informed design behaviors, with links to learning goals and instructional approaches that aim to support students in developing their engineering design abilities. Conclusions: This paper's theoretical contribution is an emergent educational theory of informed design that identifies key performance dimensions relevant to K-16 engineering and STEM educational contexts. Practical contributions include the Informed Design Teaching and Learning Matrix, which is fashioned to help teachers do informed teaching with design tasks while developing their own design pedagogical content knowledge.
This study investigates the hypothesis that when children are engaged in science experiments, the goal of which is to understand relations among causes and effects, they often use the engineering model of experimentation, characterized by the more familiar goal of manipulating variables to produce a desired outcome. Sixteen fifth- and sixth-graders worked on two experimentation problems consistent with the engineering and science models, respectively. The context in which these problems were framed was also varied, to encourage adoption of either an engineering or science model. Over six 40-min sessions, the group achieved significant increases in the percentages of inferences about variables that were both correct and valid. Improvement was greatest for those who began with the engineering problem and then went on to the science problem. The science model was associated with broader exploration, more selectiveness about evidence interpreted, and greater attention to establishing that some variables are not causal. The findings suggest that research on scientific inquiry processes should attend not only to the science content students are reasoning about, but also to their beliefs about the goals of inquiry.
Empirical data on design processes were obtained from a set of protocol studies of nine experienced industrial designers, whose designs were evaluated on overall quality and on a variety of aspects including creativity. From the protocol data we identify aspects of creativity in design related to the formulation of the design problem and to the concept of originality. We also apply our observations to a model of creative design as the co-evolution of problem/solution spaces, and confirm the general validity of the model. We propose refinements to the co-evolution model, and suggest relevant new concepts of ‘default’ and ‘surprise’ problem/solution spaces.
This paper presents the results of a survey of CAD users that examined the ways in which their computational environment may influence their ability to design creatively. This extensive online survey builds upon the findings of an earlier observational case study of the use Of Computer tools by a small engineering team. The case Study was conducted during the conceptual and detailed stages of the design of a first-to-world product. Four mechanisms by which CAD tools may influence the creative problem solving process were investigated: enhanced visualisation and communication, circumscribed thinking, premature design fixation and bounded ideation. The prevalence of these mechanisms was examined via a Series of questions that probed the user's mode of working, attitudes, and responses to hypothetical situations. The Survey showed good Support for the first three mechanisms and moderate support for the fourth. The results have important implications for both the users and designers of CAD tools.
- Dewey
In: Engineering in pre-college settings: Synthesizing research, policy, and practices
- M Hsu
- M E Cardella
- Ş Purzer
M. Hsu, M. E. Cardella, and Ş. Purzer, Assessing design. In:
Engineering in pre-college settings: Synthesizing research, policy,
and practices, Ş. Purzerm, M. E. Cardella, and J. Strobel (Ed.),
Purdue University Press, West Lafayette, IN, 2014, pp. 303-313.
- M Berland
- R S Baker
- P Blikstein
M. Berland, R. S. Baker, and P. Blikstein, Educational data mining
and learning analytics: Applications to constructionist research,
Technol. Know. Learn. 19 (2014), 205-220.
Assessment Governing Board Technology and Engineering Literacy Framework for the 2014 National Assessment of Educational Progress
- Nagb-National
NAGB-National Assessment Governing Board, Technology and
Engineering Literacy Framework for the 2014 National Assessment
of Educational Progress, 2013.
Assessing Idea Fluency through the Student Design Process Proceedings of the 45th Annual Frontiers in Education (FIE) Conference El
- M H Goldstein