No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Data Collection Methods for Evaluating Museum Programs and Exhibitions
Abstract
Museums often evaluate various aspects of their audiences' experiences, be it what they learn from a program or how they react to an exhibition. Each museum program or exhibition has its own set of goals, which can drive what an evaluator studies and how an evaluation evolves. When designing an evaluation, data collection methods are purposefully selected to provide the data needed to measure those goals and answer the evaluation questions at hand. This article provides an overview of the data collection methods commonly used in museum-related evaluations, informed by the work of the Building Informal Science Education (BISE) project.
... In practice, for both visitor studies and evaluation, the museum sector relies on a wide range of qualitative and quantitative methods to conduct front-end, formative, and summative research (Lindauer, 2005;Screven, in Bitgood & Loomis, 2012). Surveys, interviews, video-recordings, observations of visitor behavior and monitoring of exhibits are some of the most common methods (Nelson & Cohn, 2015) but are limited by the ingenuity of the individual researcher (Hein, 1998) and there is no single, fixed approach to evaluation (Diamond et al., 2016). ...
... The second problem the MEUX Evaluation Method seeks to address is in relation to standardized and generalized evaluation results. Several authors have explored how a key limitation of evaluation reports is that they are solely focused on one particular exhibit or activity, utilizing individual methods and questions, which mean findings cannot be standardized or generalized into wider understandings of how visitors engage with exhibits (Fu et al., 2016;Nelson & Cohn, 2015;Peterman et al., 2020;Teasdale, 2022). Calls have been made for more standardizes approach (MacPherson et al., 2019;Ong & Ladenhead, 2015;Voiklis et al., 2023) and so the work presented here seeks to address such limitations. ...
... Another limitation of existing evaluation practices is that results are contextualized only within the exhibit or program that they are evaluating and thus their generalisability or use is limited (Fu et al., 2016;Peterman et al., 2020;Teasdale, 2022). Where there is no one fixed approach to evaluation (Diamond et al., 2016) with evaluation methods evolving depending on the particular goals of an exhibit (Nelson & Cohn, 2015), the outcome is that the evaluation only ever provides specific data tied to an exhibit, rather than more general understandings of visitor experience that can be carried from exhibition to exhibition. However, it is this general understanding that allows for the implementation of recommendations and streamlining of practices, and so without them, it is evident that the long-term impact of evaluation is limited. ...
Underpinned by the Model for Museum Exhibit User Experience (MEUX; King et al., Visitor Studies , 2023, 26, 59), this paper develops and presents an evaluation methodology for museum exhibits that utilizes existing methodologies from the user experience sector adapted for the museum and cultural heritage sectors. Two studies are presented: an in‐depth evaluation of the Meat the Future exhibition at Oxford University Museum of Natural History and then a comparative study between this exhibition and two other permanent exhibits at the museum. Quantitative and qualitative data provide a nuanced picture of each exhibit from the visitor perspective and showcase the benefits of the MEUX methods of evaluation. Results show how three different exhibits are constructed in different ways, providing different visitor experiences and outcomes. They are directly compared with identify statistical differences, but do not impose a judgment as to whether any exhibit is better than another. With detailed, nuanced and rigorous data capturing visitor experiences of engaging with exhibits, the MEUX evaluation methodology allows for more sophisticated, standardized and efficient evaluation practices within the sector, with results that directly support further development of exhibits and exhibitions.
... More specifically, questionnaires were created for both the students and the teachers that took part in the educational program. Through these, quantitative and qualitative data were gathered [34,31]. Thus, the teachers' questionnaire consisted of two closed-ended, Likert-scale 4 questions in order to collect quantitative data and measure the extent to which respondents agree or disagree with a particular statement as well as two open-ended questions where respondents could share their thoughts and opinions. ...
Museums are called to respond to contemporary challenges and contribute to the formation of more progressive societies. One of the major challenges of our time is climate change and museums could construct cocreated narratives around climate and implement targeted actions for the audiences through their sensorial and affective experiences. More specifically, Geology and Palaeontology museums are repositories of knowledge on geoheritage and geoconservation, recognized for their scientific, pedagogical, cultural and recreational values. The educational potential of geological and paleontological exhibits could be used in informal education activities regarding the climate emergency. Taking the above into consideration, this study focuses on both the creation and evaluation of an educational program for the Natural History Museum of Arsakeio Campus at Psychicko.
... Within these pluralistic contexts, collecting and making sense of the personal data of visitors has become an everyday activity in cultural institutions: To personalise museum visits to the individual or group [33,41,74], to understand and engage visitors as co-producers of culture [92] and to understand and research their visitors in greater depth [49]. Indeed, personal data, defined as ". . . ...
... All projects funded through the National Science Foundation's (NSF) Advancing Informal STEM Learning program must post a final evaluation report to the site (NSF, 2017), and other reports are posted voluntarily. Although the repository is likely not representative of all ISE evaluation reports-reports of internal evaluations, formative evaluations, and non-federally funded evaluations are likely underrepresented-it is regarded as a key resource for studying ISE evaluation (see, e.g., Fu et al., 2016;Grack Nelson & Cohn, 2015;Morrissey et al., 2014;Serrell, 2015). As such, it can provide a window into the criteria used in ISE evaluation. ...
Evaluative criteria define a “high quality” or “successful” evaluand and provide the basis for judgment of merit and worth, yet they are often assumed and implicit in the evaluation process. This article presents an empirically supported model that describes and integrates two aspects of criteria: domain and source. Domain identifies the focus or substance of a criterion, while source describes the individual, group, or document from which it is drawn. Developed from a synthesis of evaluation literature and empirical analysis of evaluation reports, the model defines 11 criteria domains and 10 sources and reveals the relationships among them. In this integrated model, the two dimensions can be used together as a thinking tool to guide evaluators in specifying criteria, in empirical research on the valuing process, and as a conceptual framework and language for theorists prescribing criteria selection.
... Lima & Green (2017) (Benton, 2010;. For many years, researchers have provided a wealth of knowledge regarding best management practices for professionals in the museum industry (e.g., Ahmad et al., 2013;Nelson & Cohn, 2015). Recent management practices often involve evaluation of visitor AM and MM, with a focus on finding ways to enhance visitors' museum experiences and to fulfill the museum's mission (Storksdieck et al., 2006). ...
Awareness-making (AM) describes a process by which visitors bring with them past experiences, knowledge, and ideas, all of which help them make sense of museum exhibits. Meaning-making (MM) is when museum visitors’ memories and experiences transform their museum experience into new knowledge and meaning. This article explores how AM elicits MM in museum visitors. I offer findings from a research study of a natural history museum exhibition called Minnesota Journeys, based on a moose natural habitat display and accompanying interactive touchscreen. The exhibition was developed in Minnesota by the Bell Museum of Natural History for all ages. I report findings from a mixed-methods study incorporating surveys (n=243) and interviews (n=30) with adult museum visitors. I found that moose biology and ecology were not well-known subjects for this audience. However, after visiting both the habitat display and touchscreen, most visitors learned to identify specific moose biology and ecology characteristics, such as behavior and habitat. Also, the exhibit was more likely to elicit MM for visitors who answered AM questions correctly or agreed to AM belief statements. This study demonstrates how in a natural history museum setting visitor awareness-making can facilitate visitor meaning-making.
... Nelson and Cohn [49] in a review of the museum programs' and exhibitions' evaluations before 2013 reports that 79% of the experiments use Interview, 60% observation and 55% survey for data collection. Cuncliffe et al. [29] provide an analysis on the advantages and disadvantages of each method category and they concluded that a combination of methods would be more effective. ...
ICTs and the web have changed the way the users interact with museum exhibits providing alternative interactions and many advantages. Indeed, studies have shown a tremendous increase of online museum visitors. However, for a website to attract more visitors to the museum, it should be evaluated as usable and functional. This paper makes a state of the art on the evaluation experiments of museums’ websites. More specifically, it focuses on the methods used and tries to categorize the experiments found in the literature with respect to the method used. It describes the tools that may be used for implementing an evaluation experiment. Emphasis is given on the evaluation of special aspects of the use of ICTs in museums such as the use of handheld devices as well as VR tours. The review concludes on the most common criteria used for the evaluation of museum websites by real or expert users.
Temos o prazer de lançar o PRIMEIRO livro internacional voltado a área do desenvolvimento do ano de 2023, que tem como título Methodology focused on the area of interdisciplinarity, essa obra é editada pela Seven Publicações Ltda, tendo a composição de diversos capítulos voltados ao desenvolvimento e disseminação do conhecimento nas diversas áreas do desenvolvimento.
As instituições museológicas transformaram sua perspectiva comunicativa, organizacional e avaliativa ao longo do tempo, atribuindo diretrizes voltadas ao público. Cientes disto, a avaliação de exposições se tornou uma importante ferramenta para identificar como sucede a interação do público no contexto das exposições. Portanto, este estudo teve como o objetivo avaliar a interação dos turistas/visitantes no Museu do Forte do Presépio (MFP) e no Museu de Arte Sacra (MAS), localizados no Centro Histórico de Belém – PA. A metodologia adotada se constitui de pesquisa bibliográfica, pesquisa documental e pesquisa de campo através de observação direta, utilizando o método Timing and Tracking de Yallowitz e Bronnekant (2009) e uso de formulário. Utilizou-se dados quantitativos relacionados aos tempos médios, número de paradas e indicadores de interação, que foram relacionados a conceitos qualitativos como Holding Power and Attracting Power (GOINS; GRIFFEHAGEN, 1957), e fadiga museal (ROBINSON, 1928) para avaliação das exposições. Constatou-se que na exposição permanente do MAS a média de paradas foi abaixo da metade do número mínimo a ser considerado em cada área da exposição, denotando um Holding Power regular. No MFP essa média de paradas foi acima do número mínimo, demonstrando um Holding Power bom. Quanto às interações dos turistas/visitantes com as exposições, em ambos os museus houve razoabilidade de Holding Power and Attracting Power. Conclui-se que a pesquisa com o Timing and Tracking contribuiu com dados essenciais para interação dos turistas/visitantes com exposições, e que o MAS e o MFP podem utilizar esses dados para fazer uma avaliação somativa a fim de melhorar a permanência e a interação dos visitantes com as areas onde essa interação não foi efetiva.
As instituições museológicas transformaram sua perspectiva comunicativa, organizacional e avaliativa ao longo do tempo, atribuindo diretrizes voltadas ao público. Cientes disto, a avaliação de exposições se tornou uma importante ferramenta para identificar como sucede a interação do público no contexto das exposições. Portanto, este estudo teve como o objetivo avaliar a interação dos turistas/visitantes no Museu do Forte do Presépio (MFP) e no Museu de Arte Sacra (MAS), localizados no Centro Histórico de Belém – PA. A metodologia adotada se constitui de pesquisa bibliográfica, pesquisa documental e pesquisa de campo através de observação direta, utilizando o método Timing and Tracking de Yallowitz e Bronnekant (2009) e uso de formulário. Utilizou-se dados quantitativos relacionados aos tempos médios, número de paradas e indicadores de interação, que foram relacionados a conceitos qualitativos como Holding Power and Attracting Power (GOINS; GRIFFEHAGEN, 1957), e fadiga museal (ROBINSON, 1928) para avaliação das exposições. Constatou-se que na exposição permanente do MAS a média de paradas foi abaixo da metade do número mínimo a ser considerado em cada área da exposição, denotando um Holding Power regular. No MFP essa média de paradas foi acima do número mínimo, demonstrando um Holding Power bom. Quanto às interações dos turistas/visitantes com as exposições, em ambos os museus houve razoabilidade de Holding Power and Attracting Power. Conclui-se que a pesquisa com o Timing and Tracking contribuiu com dados essenciais para interação dos turistas/visitantes com exposições, e que o MAS e o MFP podem utilizar esses dados para fazer uma avaliação somativa a fim de melhorar a permanência e a interação dos visitantes com as areas onde essa interação não foi efetiva.
Algunas exposiciones museísticas se diseñan con objetivos educativos; estas exposiciones, se enmarcan dentro de la educación no formal aunque, si se analizan desde el punto de vista de la instrucción, tienen muchos aspectos en común con las actividades educativas de la educación formal.
Existe software desarrollado para diseñar actividades propias de la educación formal; en cambio, no se ha encontrado software para el diseño de exposiciones educativas en museos.
El objetivo de esta investigación es responder dos preguntas:
1. ¿Puede algún software existente ser utilizado para diseñar exposiciones educativas en museos?
2. ¿Puede algún software existente servir de base para desarrollar un software específico para diseñar exposiciones educativas en museos?
Para responder estas preguntas, en primer lugar, se han seleccionado y definido los conceptos que debería incluir un software para el diseño de exposiciones; muchos de estos conceptos son compartidos con los utilizados para el diseño de las actividades propias de la educación formal; otros, son específicos y han sido definidos en esta investigación.
En segundo lugar, de entre diecisiete software de diseño de instrucción, se han seleccionado cinco que han podido ser instalados y evaluados. A continuación se ha verificado, uno a uno, si contemplan los conceptos que un software de diseño de exposiciones debería incluir.
Los resultados de la evaluación se presentan en forma de tabla y algunas observaciones; su análisis ha permitido responder las preguntas formuladas.
De los cinco evaluados, uno, Compendium LD, puede ser utilizado para el diseño de exposiciones educativas en museos aunque, con limitaciones. Ninguno de los analizados contempla conceptos importantes como la ubicación de los objetos educativos en el espacio o la descripción de los recorridos del visitante. Compendium LD puede ser, además, un buen punto de partida para desarrollar un software específico.
There is a long history of observing visitors in museums, with the majority of the systematic observational work being done in the past 20 years or so. This article reviews the history of timing and tracking in museums, and provides a detailed description of methods used to record, analyze and report timing and tracking data. New technologies that can be used to improve the data collection and entry process are discussed, and ways in which timing and tracking data can be used to improve exhibit design are suggested.