Content uploaded by Lynda Kelly
Author content
All content in this area was uploaded by Lynda Kelly on Apr 23, 2015
Content may be subject to copyright.
Measuring the impact of museums on their communities:
The role of the 21
st
century museum
Lynda KELLY
Abstract
Museums, their missions, their civic, social responsibilities, and their modes of engagement
with communities are in a constant process of transformation in response to social and
economic imperatives at local, national and global levels. There is a need for museums to stay
relevant and be responsive to pressing social and environmental issues such as population and
sustainability, social justice and Indigenous rights. Funding bodies and stakeholders now
acknowledge that museums and programs need to demonstrate impact and value within their
local communities in order to attract further funding and ongoing support. Several models of
impact have been developed in Europe and the United States, and a number of benefits are
claimed for participation in museum programs and museum visitation. However these have not
been subject to rigorous testing, particularly in Australia. This paper discusses challenges for
museums in measuring impact and meeting audience needs drawing on results from two major
research projects: Investigating the impact of small museums in their local communities and
Exhibitions as Contested Sites - the roles of museums in contemporary society.
1
INTERCOM 2006 Conference Paper
Project 1. Investigating the impact of
small museums in their local
communities
Understanding impact in relation to culture
and the arts has been a concern of a number
of researchers in the past decade (Evans,
2001; Matarosso, 1997; Parker, Waterston,
Michaluk & Rickard, 2002; Persson, 2000;
Sandell, 1998; Sheppard, 2000; Williams,
1997). The economic value of the arts and
culture sector is widely recognized as being
only one part of its net worth to the
community (Holden, 2004). Identifying social
impact has been one way to shift the focus
from economics to capture a more holistic
understanding of how arts and culture
contribute to communities. The strength of
research studies into the impact of arts and
culture generally is that they have attempted
to develop measurable indicators. They have
mapped out areas where outcomes and
differences can be detected. The weakness
lies primarily in the inability to consistently
apply these indicators in ways that
demonstrates the impact on an individual and
community. There is no template that can be
consistently used with confidence across a
number of situations (Reeves, 2002).
Investigating the impact of small museums
in their local communities was funded by the
University of Technology, Sydney; the
Australian Museum; Arts NSW; and
Museums Galleries NSW. The aim was to
develop methodologies to evaluate the range
of impacts of local museums on their local
communities and through this to identify types
of programs or processes that lead to positive
impacts of museums. A case study approach
was adopted using a mix of qualitative and
quantitative data gathered at three sites
across NSW-a south coast museum; an
Aboriginal Keeping Place in northern NSW
and a metropolitan museum in Sydney. The
project studied the question What are the
impacts of museums on their local community
and how can these be measured?, and was
undertaken within the context of social
inclusion policy initiatives and previous
cultural and social impact studies into the
benefits of arts participation conducted over
the past decade.
From the literature reviewed it was
determined that the conceptual framework for
measuring social capital, although
problematic, might be more useful than
adopting or adapting methodologies that have
been used by science museums or projects
that measure outcomes associated with
participation in the arts (Stone, 2001). As
Stone suggests, it may be difficult to prove
that a causal relationship exists between
museums and the social impact they
generate. What is more achievable is to show
how museums make or contribute to an
impact but do not necessarily cause an
impact to happen.
In considering the research question and
the definition of impact as changes to social
and cultural stock and distribution, social
impact becomes the dependent variable-how
museums contribute to increase or decrease
this stock and how do we know? The impact
is dependent on the presence (programs,
policies, activities) of the museum. This does
not assume that no social capital existed
before museums were created within the
community. What is being measured is how
museums contribute to the stock of social
capital not whether they created it, and what
type of social capital that can legitimately be
claimed as created by the existence of the
museum. The framework adopted for this
study was informed by social capital concepts
of trust, reciprocity and networks using a
belief/behaviour dichotomy within the
museum/community context (Diagram 1).
Limits were set on the nature and scope of
the pilot study so as to develop appropriate
methodologies, specifically related to the
impact of a subset of regional museums
rather than major state or national museums.
A subset was chosen as there are sufficient
Measuring the impact of museums on their communities: The role of the 21 century museum
st
2
INTERCOM 2006 Conference Paper
differences within regional museums to
identifyńrepresentativeŅcases of type;
regional museums are more likely to be
closely connected to the communities they
serve whereas larger museums are more
likely to serve a number of disparate and
complex stakeholders and require different
forms of measurement. Additionally in
Australia, smaller regional museums are
often reliant on the ńgoodwillŅof community
funders therefore making measuring impact a
high priority. Therefore, a case study
approach was developed with the three
museums selected on the basis of:
œ
A representative of ańmetropolitanŅ
museum within the City of Sydney-the
impacts that these museums create may be
significantly different from those in more
remote locations. In addition these
museums compete with significant state
and national museums in adjacent
locations.
œ
A representative of a small regional
museum-in some sense theńtypicalŅ
regional museum within regional and rural
communities that have undergone structural
change through drought or industry
displacement of jobs ( for example, logging,
fishing, mining) and are now reinventing
themselves as tourist destinations or
developing other new industries.
œ
A representative of a Keeping Place-
community museums that are established
by Indigenous people in their local areas to
house repatriated artefacts, host
exhibitions, conduct education and
research programs while providing
employment and a meeting place.
Both qualitative and quantitative
approaches were used. The qualitative stage
investigated the assumed social and cultural
benefits that accrued to the users, staff,
volunteers and funders of the local museums.
Interviews and focus group discussions were
conducted with a number of stakeholders
including staff, volunteers, users of programs
and Council staff. The purpose of the
discussions was to identify perceived impacts
the museums were delivering to those directly
linked to the centres.
Diagram 1: Relationship of trust, reciprocity and networks - belief and behaviour
3
INTERCOM 2006 Conference Paper
A quantitative survey was randomly
administered to the local population in the
three areas tested both beliefs and behaviour
in relation to using the museum, sourcing
information, participating in programs,
showcasing the museum to visitors and
willingness to pay additional funding to the
museum. A total of 294 local residents were
sampled across the three sites.
Project 1: Results
The analysis of the qualitative studies
identified concepts such as individual social
benefits (networking, quality of life,
interaction), community social benefits
(networking, services to community) and
individual and community economic and
cultural benefits.
Generally the qualitative findings
suggested that the value of local museums
were the links back to community;
opportunities for people to visit, including
attending events; the work opportunities (both
paid & unpaid) that were available; the wealth
that the museum creates in the local
community leading to generate money to go
back to the community. Broader outcomes
were also identified, such as developing an
appreciation of place and culture, community
pride, museums preserving heritage, and
opportunities for learning across all age
levels.
An example of one of the sites researched
(the Keeping Place) is presented to illustrate
some of the findings. The Keeping Place is
located in northern inland NSW. It has a fairly
large, new building with good facilities. A
diversity of funding sources is available as
the Keeping Place forms a key part of a
number of projects overseen by a
corporation. The Keeping Place serves two
communities: Indigenous and general. They
see their role to foster community pride and
educating all levels of visitors about
Australia's Indigenous culture generally, and
those of the region specifically. The
qualitative findings demonstrated that social
interaction at the facility was important at an
individual level, with the community social
level outcomes to represent the local
community, and to establish links with the
local and broader community, as well as
being recognised as a community resource
through functions and events and other
programs. In terms of community economic
outcomes the Keeping Place generates
income that stays in the local community; it is
a place for tourism and is the major employer
of local Indigenous people through
Government and privately-funded programs.
In the interviews some tensions were
identified, such as for whom the centre
served-was it inclusive of non-Indigenous
people or exclusively for the Indigenous
community? For example, one respondent
stated that:
But it's hard for the Aboriginal community
to understand that they can't, they get a free
cup of coffee and stuff, but just use it like they
want to use it. Telling us we're running it like a
white organisation.
Individual and community cultural
outcomes included pride, ownership, sharing
culture, promoting reconciliation and
developing skills within the community, both
practical job skills and an understanding and
appreciation of how to promote Indigenous
culture. However, one difficulty expressed
was the problem of racism:
We're trying to lift the profile but you get
the children listening to their parents, from
both sides, so they'll go and call someone
black or someone white. ... They dwell in the
past because it's something,
ņ
We wouldn't
be like this if you whites didn't ...
Ň
There's
still that blame.
In terms of community education the
Keeping Place saw that its role is to provide
factual information and as an educational
resource for a wider community, including
schools, for example:
I think it's the education, especially at the
Measuring the impact of museums on their communities: The role of the 21 century museum
st
4
INTERCOM 2006 Conference Paper
schools, school excursions. It's always the
children who tell their parents
ņ
you should go
there and look.
Ň
There were many ideas for future
development in this area, such as exhibitions
and events; promoting the Centre as a
meeting place; promoting greater links to the
broader community; and the planned future
building works. Additionally, opportunities
were seen in educating international visitors
and non-Indigenous people about their
culture:
I really love discussing things with the
Europeans because they've been brought up
with lies about the Aboriginal people. ... they
come in here, they ask questions and I tell
them straight about how Aboriginals were
treated and they walk out with a completely
different attitude and it's just wonderful to see.
However, there were perceived barriers by
the broader community. For example, the
Centre could be seen as exclusively
ņIndigenousŇby the rest of the local
community, coupled with difficult local issues
such as racism. Practical considerations like
restricted opening hours and general lack of
knowledge about what is there, with some in
the local community knowing that the
Keeping Place was there but not visited-it
was unclear as to why.
In summary, the qualitative findings across
the three sites showed that social benefits
were evident when the museum was a
conduit to the local community through
integration into the community and through
linkages outside of the institution, the
expression of local culture, and in developing
skills and providing social interaction. It was
widely acknowledged that economic benefits
were important. It was also recognised that
local museums needed a clear focus of their
purpose and a strong, identifiable identity. If
not, community divisions and tensions may
contribute to unclear goals.
The quantitative phase consisted of 294
interviews with people across the three local
areas. The interviews were conducted on the
streets, with respondents answering both
open-ended questions and rating scales that
looked at the museums' role and the
perceived benefits. Overall, the general
benefits were described as tourism, an
appreciation of history and culture, and
general education, among others.
Interestingly, and encouragingly, no
respondent could list any negatives. When
responding to a series of statements about
local museums derived from the literature
review there was strong agreement that local
museums:
œ
develop pride in local traditions and
customs
œ
play an important role in tourism
œ
should have exhibitions relevant to the local
area
œ
help people feel a sense of belonging and
involvement
œ
involve people in local projects
œ
promote contact and cooperation across
different cultures
œ
develop community and social networks
œ
develop contact across different age
groups.
It was found that few respondents agreed
that local museums help people take up or
develop careers, were places where people
can debate issues or were well-known in the
local community mainly for education.
Looking at where museums fit when people
were looking for specific local information the
results found that respondents mainly
accessed the Internet, with the local library
and museum the first place used by around a
quarter of respondents. In terms of museums
in their local communities, 45% thought
museums were just as important as other
community organisations.
Project 2. Exhibitions as contested
sites- The roles of museums in
contemporary society
The second research project, Contested
5
INTERCOM 2006 Conference Paper
Sites, examined current and potential roles of
museums, and how they could position
themselves better around contentious topics
and sensitive issues and build vibrant
participatory cultures (Cameron, 2003, 2006;
Ellison, 2003; Ferguson, 2006; Kelly, 2006).
Contested Sites was a three project funded
by the Australian Research Council, with
input from partners University of Sydney; the
Australian Museum, Sydney and the
Australian War Memorial, Canberra. The
Canadian Museums Association also
generously funded the Canadian component
of the project.
A range of museum audiences were
sampled to investigate their perceptions
about the roles museums could play in
contemporary society. This study moved
beyond the specifics of exhibition
controversies and theoretical rhetoric to
examine the relevance, plausibility and
practical operation of a broad range of
museums as civic centres and for the
engagement of topics of contemporary
relevance and importance. To this end
answers were sought to a range of questions
including:
œ
How can museums contribute to
discussions on issues of contemporary
relevance and importance?
œ
How might museums effectively engage
contentious topics in new ways that
acknowledge and embrace conflicting
opinions, are non-alienating and acceptable
to the majority of audiences?
œ
In what ways can museums navigate the
sensitive terrain between facts/opinion,
authority/expertise, advocacy/neutrality and
censorship/exposure?
Four stages of research were identified,
each using different methodologies to best
deliver the desired results. First, a literature
analysis was undertaken into prevailing
museological and theoretical debates about
the roles of museums in contemporary
society and in the fields of media and cultural
studies, sociology and conflict and peace
studies. The aim was to link contemporary
debates across a range of disciplines to
contribute to and extend understandings of
the capacity of museums to anticipate and
engage with controversial subjects outside
traditional thinking. This was followed by
literature analysis of exhibition controversies
in the United States, Canada, United
Kingdom and Australia to situate controversy
in a historical context by investigating how
particular exhibitions in the past have been
defined as controversial and how the
definition has affected the roles and
functioning of museums. From the themes
identified in this review a multi-method
strategy was developed and implemented
that was both quantitative and qualitative.
These investigated museum roles,
community, audience, staff, management and
stakeholder expectations and concerns. The
multi-method ensured both reliability and
validity (Cohen & Manion, 1994). A series of
statements were developed and used across
all samples that addressed the key issues
identified from the literature review.
In the second stage, telephone (also called
omnibus) surveys of the broader Australian
community were conducted, both museum
and non-museum goers detailing
demographic profiles including socio-
economic data correlated to a series of
questions on topics and museum roles.
Survey respondents were asked whether or
not museums should present exhibitions on
contentious topics and were then invited to
respond to a series of current and potential
role statements using a five-point Likert scale
(strongly agree to strongly disagree) based
on the following themes:
œ
Are museums information sources and safe
places to explore these topics by presenting
a range of viewpoints? Or should they take
a more active role, as transformative
spaces to challenge and change views?
œ
Should museums act as provocateurs and
Measuring the impact of museums on their communities: The role of the 21 century museum
st
6
INTERCOM 2006 Conference Paper
take a leading role as social and political
activists to bring about change, and to
assist in the resolution of issues on a
personal or political level?
œ
Alternatively, is the primary role of
museums to offer non-challenging social
experiences?
œ
Can museums be all of these things at
once?
Exit surveys were conducted at the
Australian Museum and the Australian War
Memorial in the third stage, drawing on
statistically significant samples of 197 and
248 respondents respectively. The same
range of questions were used in order to
compare the responses of the broader
community with audiences and to gather
more detailed demographic data about age,
gender, cultural or ethnic affiliation, social,
economic and family circumstances. With a
generous grant from the Canadian Museums
Association, visitor surveys were undertaken
at three Canadian Museums, the Museum of
Anthropology Vancouver, the Canadian War
Museum and the Musee d'Art in Montreal
with a total of 286 visitors. This survey was
administered in French and English in Ontario
and Quebec and English-only in British
Columbia, using the same set of questions as
in the Australian museums. Quantitative exit
surveys and questionnaires were analysed
using SPSS (data analysis software) to
enable comparisons between all data sets,
cross-correlations, comparing results from
different cultural contexts, while extending the
research sample.
The fourth stage of the research was
qualitative in nature in order to explore,
unpack and discuss the findings from both
the telephone and exit surveys on topics
about the civic roles and social
responsibilities of museums as well as
experiences of museum visiting, functions
and activities, authority, expertise, trust and
censorship. Five focus groups were
conducted with museum visitors in Sydney
and Canberra-adults aged 18-30 without
children, adults aged 30-49 with children, and
adults 50-64.
In contrast, the perspectives of museum
staff and stakeholders were also gathered
using the an online survey, in-depth
interviews and focus group discussions with
over 100 staff and stakeholders in 26
institutions in Australia, New Zealand,
Canada, the USA and UK. In the focus
groups, participants were asked to identify
any topics or issues that were particularly
controversial orńhotŅin that country, or for
that museum at that time. This enabled the
research to capture emerging controversies
and contemporary responses. Other
questions related to museums, social
responsibilities and civic roles, as information
sources including authority, expertise and
censorship, the impact of controversies on
institutional functioning, successful
programming and funding arrangements. By
comparing the different geo-political, social,
cultural and institutional contexts within which
the international museum community
operates we sought to illuminate the
challenges, limitations and opportunities that
institutions face in presenting contentious
subjects.
Project 2: Results
The results presented here are a selected
sample from the broader project, serving as a
general comparison to the specific findings
from the impact study described earlier.
Generally, respondents felt that museums
have a role in presenting controversial issues,
with over half of the general population
sampled agreeing or strongly agreeing with
the question Do museums have a role in
developing exhibitions on taboo and
controversial topics? Strongest support was
from the industry respondents, followed by
Canadian museum visitors and Australian
Museum visitors. A selection of other areas
that participants agreed or strongly agreed
with is outlined in Table 1.
7
INTERCOM 2006 Conference Paper
Strong agreement was noted in the areas
of information-provision, with less so on
museums focusing on non-challenging
experiences. Coupled with strong support for
museums as places to explore important
issues, these views provide positive feedback
for museums.
Some of the key questions that visitors
raised in the study that are worth further
thought and reflection are issues about
authority, whose voices are being
represented and the issue of trust. A final
consideration was to engage audiences in
ways that they like to learn through finding
the right balance between being popular and
being populist; being controversial and
critical; between providing information and
generating knowledge; and recognising the
relationship between learning and
entertainment.
What do the Contested Sites results
demonstrate about the roles for museums?
Museums as trusted, reliable and credible
sources for information were critical,
especially given these times of incessant
change. Museums are socially integrative and
inclusive experiences, with audiences
wanting to be challenged more than they are
currently. Whose voices are being heard in
the museum and how museums are dealing
with social change are critical. Trust in the
institutions and the authority of museums was
well-recognised. Overall, bringing out
important challenging and controversial
points of view in a democratic, free-thinking
society for many was seen as a key role for
museums-one of the few places where these
debates can happen:
œ
Museums aren't just for showcasing surface
aspects of culture. (visitor, Museum of
Anthropology, Vancouver)
œ
Museums are a public forum for issues that
should challenge society. (female, 30-39
years, business owner, Sydney)
œ
If museums don't do it who will? (visitor,
Australian War Memorial)
œ
Controversy is just one factor in diversity
and people have a choice to attend an
exhibition or program. (visitor, Museum of
Anthropology, Vancouver)
œ
Hiding something doesn't help kids and the
future of Canada. You have to air it in order
to get to the truth, whatever the truth is.
(visitor, Canadian War Museum, Ottawa)
Conclusion
These two studies have revealed a range
of outcomes about how museums are valued,
the impact they have on their local
communities and the roles museums play in
people's lives. Generally, they show that local
communities understand and value the role of
museums and that museums benefit the local
community, in a reciprocal relationship of
mutual benefit. The broader global
community also understand and have strong
views about the roles of museums,
particularly as places that provide information
about issues that might be difficult or
controversial.
The challenge presented by these studies
is to use the information generated to
convince governments and funding agencies
about the role of museums, their worth in the
community and the value placed on them by
people. How can we do this in ways that are
Industry (n=148)AWM (n=248) Canadian(n=286) AM (n=197)
Places to explore important issues 97% 91% 90% 94%
86% 89% 89% 92%
99% 96% 98% 94%
43% 40% 46% 24%
Places that allow visitors to make comments
Places that should provide information
Places for non-challenging social experiences
Table 1. Positioning Statements
Measuring the impact of museums on their communities: The role of the 21 century museum
st
8
INTERCOM 2006 Conference Paper
meaningful to funding agencies? Using key
economic indicators that go beyond numbers
is a start. This could be done through
highlighting, for example, the social benefits
of museums in areas such as mental health
and social wellbeing, demonstrating that
museums are an integral part of social
capital. Another example is to make clear the
contribution museums make to the monitoring
of environmental indicators, which is
becoming critical given the increased
emphasis in the world today on climate
change. Finally, museums have opportunities
to influence, challenge and sometimes
change how visitors think, inspiring them to
take action on big issues and be more
informed citizens in an increasingly globalised
world. Visitors want this; are museums ready
and willing to provide?
References
Cameron, F. (2003). Transgressing fear- Engaging emotions and opinion- A case for museums in the 21st century.
Open Museum Journal, 6.
Cameron, F. (2006). Beyond surface representations: Museums, edgy topics, civic responsibilities and modes of
engagement. Open Museum Journal, 8.
Cohen, L., & Manion, L. (1994). Research methods in education. London: Routledge.
Ellison, J. (2003). Re-visioning the media, museums and controversy: A preliminary case study. Open Museum Journal,
6.
Evans, M. (2001). Renaissance in the regions: A new vision for England's museums. London: Resource: The Council
for Museums Archives and Libraries.
Ferguson, L. (2006). Pushing buttons: Controversial topics in museums. Open Museum Journal, 8.
Holden, J. (2004). Capturing cultural value: How culture has become a tool of government policy. London: Demos.
Kelly, L. (2006). Museums as sources of information and learning: The decision-making process. Open Museum
Journal, 8.
Matarosso, F. (1997). Use or ornament? The social impact of participation in the arts. Stroud: Comedia.
Parker, S., Waterston, K., Michaluk, G., & Rickard, L. (2002). Neighbourhood renewal and social inclusion: The role of
museums, archives and libraries. Glasgow: Marketing Management Services International.
Persson, P. (2000). Community impact of science centres: Is there any?. Curator (Vol. 43, pp. 9-17).
Reeves, M. (2002). Measuring the economic and social impact of the arts: A review (pp. 131). London: Arts Council of
England.
Sandell, R. (1998). Museums as agents of social inclusion. Museum Management and Curatorship (Vol. 17, pp. 63-74).
Sheppard, B. (2000). Do museums make a difference? Evaluating programs for social change. Curator (Vol. 43, pp. 63-
74).
Stone, W. (2001). Measuring social capital: Towards a theoretically informed measurement framework for researching
social capital in family and community life. Melbourne: Australian Institute of Family Studies.
Williams, D. (1997). How the arts measure up: Australian research into social impact. Stroud, Glos.: Comedia.
About the author
Lynda Kelly is the Head of the Australian Museum Audience Research Centre (AMARC) and has been working in the
evaluation and audience research fields since 1994 and in the museum industry since 1987. Ms. Kelly has extensive
experience in planning and conducting quantitative and qualitative research for a variety of purposes and a range of
audiences. Ms. Kelly is particularly interested in Indigenous evaluation; visitor experiences and learning and how these
can be measured; use of digital media in research and evaluation; as well as the strategic uses of audience research in
organisational change.
Ms. Kelly is internationally recognised as a leader in evaluation and audience research in the cultural sector through
her extensive projects, publications, conference papers and workshops. Ms. Kelly, through AMARC, pioneered a
website (http://www.amonline.net.au/amarc/) that provides extensive information about audience research to assist in
the development of industry skills and information provision, both within Australia and internationally.
Ms. Kelly has participated as a Partner Investigator in a range of research projects conducted in collaboration with
9
INTERCOM 2006 Conference Paper
universities across Australia, most notably the international research project Exhibitions as Contested Sites - the role of
museums in contemporary society funded by the Australian Research Council with partners the University of Sydney,
the Australian Museum and the Australian War Memorial. This project examined current and potential roles of
museums. Other significant projects include Assessing the Impact of Museums on their Local Communities, with the
University of Technology, Sydney and Arts NSW; and New Literacies, New Audiences, part of the Centre for Creative
Industries at the Queensland University of Technology. Ms. Kelly has published widely in the field of audience research
including four books about museum audiences- Energised, Engaged, Everywhere: Older Australians and Museums
(2002); Indigenous Youth and Museums: A Report on the Indigenous Youth Access Project (2002); Knowledge Quest:
Australian Families Visit Museums (2004) and Many Voices Making Choices: Museum Audiences with Disabilities
(2005).
Ms. Kelly is a Board member of ICOM-CECA and the ICOM Australia National Committee, a member of the editorial
board for the Journal of Visitor Studies and an assessor for the Australian Research Council. Ms Kelly has qualifications
in psychology and personnel management, and has just submitted her doctoral thesis which investigated adult museum
visitors' learning identities and how museum exhibitions impacted on a visitor's self-concept and how they viewed
themselves as a learner.
Measuring the impact of museums on their communities: The role of the 21 century museum
st
10
INTERCOM 2006 Conference Paper
