Conference Paper

Roll It Wall: Developing a Framework for Evaluating Practices of Learning

To read the full-text of this research, you can request a copy directly from the authors.


We1 introduce a framework to describe visitor engagement at an interactive museum exhibit where visitors build and test ball roller coasters. Our framework consists of two dimensions. The first is levels of engagement, which describe what visitors are doing and how they are interacting with the exhibit. The second dimension is practices of learning, which are derived from other research and policy documents such as the Next Generation Science Standards (NGSS). Our matrix links those practices of learning with observed types of engagement.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Video and inperson observations were the central data source used to develop engagement types for the focal exhibits. Teams categorized the observed engagement types, which were then analyzed for evidence of STEM practices [2,17]. The STEM practices that the four teams identified were: 1) observing, 2) asking questions, 3) defining problems, 4) planning investigations, 5) carrying out investigations, 6) analyzing and interpreting data, 7) designing solutions, 8) testing solutions, 9) developing models, 10) using models, 11) mathematical and computational thinking, 12) constructing explanations, 13) argue from evidence, 14) obtaining and evaluating information, 15) cause and effect, 16) scale, proportion, and quantity, 17) perseverance to achieve a goal, and 18) patterns. ...
... Behaviors were coded according to four engagement types and analyzed for evidence of STEM practices (See Table IV). Choosing specific colored pegs to make a picture (e.g., red pegs to make a heart) 1, [15][16][17][18] Peg and Wall Play with inserting pegs into wall 1,2,3,14-18 Interaction Peg Play ...
... In museums, guests may interact with the exhibits for as little or long as they want. These four studies whose finding are summarized in Table V and others [17] describe the complex ways that visitors engage with exhibits which lead to powerful learning experiences. The work presented here focuses attention on how to design exhibits to engage guests in NGSS-aligned learning experiences. ...
... or "What do you think the arrow block does?". Depending on the setting, facilitators such as teachers, parents, or museum staff can also be equipped with discussion points to prompt conversation [38]. However, it is important that facilitators offer probing questions or thought provoking comments (e.g. ...
... Maybe you could find a way to connect them into one composition?") as opposed to trying to explain how the table components work to participants, which may lead to reduced exploration, as children often assume adult's explanations are "correct" and do not question them [1], [38]. ...
Museum visitors often come into the museum space receptive to exploring new ideas, and this may encourage members of visitor groups to be supportive and cooperative when engaging together with exhibits. However, as participant groups explore the concepts of the exhibit, interruptions, conflicts, or disagreements may result. We collectively label this social tension as discord. This paper studies discord among family groups interacting with TuneTable, a museum exhibit designed to promote middle school students' interest in and learning of basic computing concepts (e.g. loops, conditionals) through music programming. We analyzed video recordings of each participant group and found that discord often appears alongside three markers of high engagement: a) complex physical manipulation of exhibit components; b) conversation demonstrating an in-depth understanding of how the exhibit works; and c) instances of collaboration between group members. Our findings suggest that certain types of discord could potentially be indicators of productive learning experiences at museum exhibits related to computing. In addition, when designing informal learning experiences for computing education, our findings suggest that discord is a potential trigger for deeper engagement that warrants further exploration.
... The goal of the project is to develop suites of coordinated STEM activities. Each suite focuses on an engineering task completed at MOXI, The Wolf Museum of Exploration + Innovation (e.g., Harlow, Skinner & O'Brien, 2017 ). Activities done in classrooms prior to the fi eld trip exhibit develop physical science and engineering ideas that the students will use in their fi eld trip program and require students to use mathematics skills being developed in their classrooms. ...
... Our research on practice-based learning began by investigating visitors' experiences. Initially, we sought to understand the ways visitors engaged with open-ended exhibits [23][24][25] and how these aligned with our STEM practice goals. Our second focus of practice-based facilitation guided our analysis of the visitor experience to inform strategies for making facilitation decisions that would support visitors in engaging in STEM practices [17]. ...
... During the full time of the study, the instructor was collaborating on the design of the exhibits at a new museum, the Wolf Museum of Exploration + Innovation (MOXI), and that work on informal science education influenced her perspectives and course activities. MOXI's exhibits were being planned as the NGSS were authored, providing an opportunity to design a museum fully informed by the NGSS and a place for pre-service teachers to observe children learning NGSS-aligned content in an informal science space (Harlow et al. 2017). In the pre-service teacher education course studied here, teacher candidates read and discussed the documents about facilitating learning in these informal spaces, highlighting strategies they found particularly helpful or promising for their own future instruction and then visited MOXI to observe children's interactions in informal spaces. ...
Full-text available
Despite the potential of the maker movement to influence how we teach students in school, thus far, most research on maker activities have taken place in informal spaces, such as museums and after-school programs, which are inaccessible to some populations. To ensure maker education reaches all students, it must find its place at school. However, classroom-based maker activities have different constraints and may require teachers to hold different types of knowledge. We drew from the body of research on maker education to create a course that prepared pre-service elementary school teachers to implement activities that were consistent with the maker ethos and met state and district standards. As a course assignment, the teacher candidates designed and hosted a School Maker Faire for elementary school children, providing an opportunity for local children to participate in maker activities and for pre-service elementary school teachers to design, facilitate, and reflect on maker education as a method of teaching science. In this paper, we delineate the constituent parts of maker pedagogical content knowledge and describe how pre-service teachers developed the appropriate knowledge for integrating maker education activities into their classroom curriculum. We propose that the knowledge teachers need to facilitate and assess student learning through maker education is more complex than either science pedagogical content knowledge or engineering pedagogical content knowledge.
Conference Paper
Artificial intelligence (AI) is becoming increasingly pervasive in our everyday lives. There are consequently many common misconceptions about what AI is, what it is capable of, and how it works. Compounding the issue, opportunities to learn about AI are often limited to audiences who already have access to and knowledge about technology. Increasing access to AI in public spaces has the potential to broaden public AI literacy, and experiences involving co-creative (i.e. collaboratively creative) AI are particularly well-suited for engaging a broad range of participants. This paper explores how to design co-creative AI for public interaction spaces, drawing both on existing literature and our own experiences designing co-creative AI for public venues. It presents a set of design principles that can aid others in the development of co-creative AI for public spaces as well as guide future research agendas.
Full-text available
The scientific research and education communities have long had a goal of advancing the public's understanding of science. Another emerging area of research investigates science-related hobbies. Research conducted by Marni Berendsen, education researcher and project director of the NASA Night Sky Network, showed that amateur astronomy club members lacking college-level astronomy training often knew more general astronomy than did undergraduate astronomy majors. Supporting evidence for the important role that out-of-school experiences have on children's learning is emerging from a variety of fronts. For example, a recent meta-analysis of experimental and quasi-experimental evaluation findings for after-school programs showed that such programs need not be academically focused in order to have academic impact. It seems reasonable to assume that out-of-school science-learning experiences are fundamental to supporting and facilitating lifelong science learning. The dominant assumption behind much current educational policy and practice is that school is the only place where and when children learn.
Full-text available
The maker movement consists of a growing culture of hands-on making, creating, designing, and innovating. A hallmark of the maker movement is its do-it-yourself (or do-it-with-others) mindset that brings individuals together around a range of activities, both high- and low-tech, all involving some form of creation or repair. The movement's shared commitment to open exploration, intrinsic interest, and creative ideas can help to transform STEM and arts education. The authors profile several forms of making, makerspaces, and maker networks, and conclude with some ways the movement spreads innovation, providing potential guidance for educational reform.
Full-text available
The article highlights and problematizes some challenges that are faced in carrying out design-based research. It states that the emerging field of learning sciences is one that is interdisciplinary, drawing on multiple theoretical perspectives and research paradigms so as to build understandings of the nature and conditions of learning, cognition and development. A fundamental assumption of many learning scientists is that cognition is not a thing located within the individual thinker but is a process that is distributed across the knower, the environment in which knowing occurs and the activity in which the learner participates. In other words, learning, cognition, knowing and context are irreducibly co-constituted and cannot be treated as isolated entities or processes.
As learning institutions, museums have long been buffeted by currents, coming mostly out of formal education, defining what counts as learning. The field has struggled to find its way in these waters. In this article I propose a strategy for gaining a firmer footing and indeed (shifting metaphors) for beginning to shape the very landscape, in educational and cultural policy, that currently constrains how our work is perceived and understood as contributing to learning within broader educational ecosystems. Research-practice partnerships represent a new, more equitable and perhaps more ethical, strategy for producing evidence-based knowledge and practice. Leveraging perspectives and expertise of both researchers and museum practitioners, RPPs can help the field begin to redefine what learning looks like and how museums both support and expand it. This paper discusses the need to participate in this new approach to research and provides some strategies for getting started.
Science, engineering, and technology permeate nearly every facet of modern life and hold the key to solving many of humanity's most pressing current and future challenges. The United States' position in the global economy is declining, in part because U.S. workers lack fundamental knowledge in these fields. To address the critical issues of U.S. competitiveness and to better prepare the workforce, A Framework for K-12 Science Education proposes a new approach to K-12 science education that will capture students' interest and provide them with the necessary foundational knowledge in the field. © 2012 by the National Academy of Sciences. All rights reserved.
As tinkering and making spaces proliferate in museums, many researchers, practitioners, funders, and policy-makers seek to understand what constitutes learning-through-tinkering. To support discussion of tinkering-based learning, the Exploratorium sought to articulate and refine a valid, evidence-based definition of learning in its permanent on-floor Tinkering Studio. We studied and made videos of fifty learners and their companions in one of three tinkering activities in the Tinkering Studio. A team of researchers and practitioners used the videos to refine frameworks for learning and facilitation (initially developed in a prior project), leading to the identification of four Dimensions of Learning and three broad Facilitation Moves. We created a Tinkering Library of Exemplars that categorizes over one hundred video clips according to these frameworks. The Library may help articulate important aspects of learning and facilitation, give voice to practitioners’ values in defining learning-through-tinkering, and lay a methodological foundation for gathering evidence for such learning. The Library is available for download.
Next Generation Science Standards identifies the science all K-12 students should know. These new standards are based on the National Research Council's A Framework for K-12 Science Education. The National Research Council, the National Science Teachers Association, the American Association for the Advancement of Science, and Achieve have partnered to create standards through a collaborative state-led process. The standards are rich in content and practice and arranged in a coherent manner across disciplines and grades to provide all students an internationally benchmarked science education. The print version of Next Generation Science Standards complements the website and: Provides an authoritative offline reference to the standards when creating lesson plans. Arranged by grade level and by core discipline, making information quick and easy to find. Printed in full color with a lay-flat spiral binding. Allows for bookmarking, highlighting, and annotating.
A framework for K-12 science education: Practices, crosscutting concepts, and core ideas
  • Research National
  • Council
The 95 Percent Solution
  • H John
  • Falk
  • D Lynn
  • Dierking
Research and practice: One way, two way, no way, or new way?. Curator
  • Bevan Bronwyn