Research Design : Qualitative, Quantitative, and Mixed Methods Approaches / J.W. Creswell.
- SourceAvailable from: vuw.ac.nz
- [Show abstract] [Hide abstract]
ABSTRACT: This paper provides an overview, including methodology and preliminary findings, of a current and ongoing doctoral research study of the impact of national library associations. The study uses the impact assessment framework provided by ISO16439:2014 Methods and procedures for assessing the impact of libraries. If the professional associations supporting our profession are to become sustainable we need to understand the difference they make to the individual members of the profession, to the employers of those individuals and to the profession of librarianship. This study applies the framework provided by ISO 16439:2014 to the national library association environment to explore and gather evidence of impact. To align with the ISO model, impact is differentiated into (a) impact on individuals; (b) social impact – institution (library or employing organization); and (c) social impact – community (the profession). Preliminary findings show evidence of impact in all categories. Data was gathered through semi-structured interviews that were conducted with members of national library associations in Australia, New Zealand, the United Kingdom and the United States during the third quarter of 2013. This study contributes to research methodologies by testing the use of a qualitative assessment tool in a way that could be transferable to other associations both within and external to the library environment and to enable it to be adapted more broadly for other purposes within the library and information environment.Performance Measurement and Metrics 11/2014; 15(3).
and Mixed Methods
John W. Creswell
University of Nebraska, Lincoln
International Educational and Professional Publisher
Thousand Oaks London New Delhi
~ 6 7 6 3
Copyright O 2003 by Sage Publications, Inc.
All rights reserved. No part of this book may be reproduced or utilized in any
form or by any means, electronic or mechanical, including photocopying, record-
ing, or by any information storage and retrieval system, without permission in
writing from the publisher.
Cover image copyright O Sheldan CollinsICorbis; used by permission.
Sage Publications, Inc.
2455 Teller Road
Thousand Oaks, California 91320
Sage Publications Ltd.
6 Bonhill Street
London EC2A 4PU
Sage Publications India Pvt. Ltd.
Greater Kailash I
New Delhi 110 048 India
Printed in the United States of America
Library of Congress Cataloging-in-Publication Data
Creswell, John W.
Research design: Qualitative, quantitative, and mixed method
approaches I by John W. Creswel1.- 2nd ed.
Includes bibliographical references and indexes.
ISBN 0-7619-2441-8 (c) - ISBN 0-7619-2442-6 (pbk.)
1. Social sciences-Research-Methodology.
2. Social sciences-Statistical methods. I. Title.
H62 .C6963 2002
C. Deborah Laughton
Diana E. Axelsen
A. J. Sobczak
C&M Digitals (P) Ltd., Chennai, India
framework be adopted to provide guidance about all facets of the
study, from assessing the general philosophical ideas behind the
inquiry to the detailed data collection and analysis procedures. Using
an extant framework also allows researchers to lodge their plans in
ideas well grounded in the literature and recognized by audiences
(e.g., faculty committees) that read and support proposals for
What frameworks exist for designing a proposal? Although differ-
ent types and terms abound In the literature, I will focus on three:
quantitative, qualitative, and mixed methods approaches. 'The first
has been available to the social and human scientist for years, the
second has emerged primarily during the last three or four decades,
and the last is new and still developing in form and substance.
This chapter introduces the reader to the three approaches to
research. I suggest that to understand them, the proposal developer
needs to consider three framework elements: philosophical assump-
tions about what constltutes knowledge claims; general procedures
of research called strategies of inquhy and detailed procedures of
data collection, analysis, and writing. called methods. Qualitative.
quantitative, and mixed methods approaches frame each of these
elements differently, and these diefences are identified and dis-
cussed in this chapter. 'Then typical scenarios that combine the three
elements are advanced, followed by the reasons why one would
choose one approach over another in designing a study. 'This discus-
sion will not be a philosophical treatise on the nature of knowledge,
but it will provide a practical grounding in some of the philosophical
ideas behind research.
n the past two decades, research approaches have multiplied to
a point at which investigators or inquirers have many choices. For
those designing a proposal or plan, I recommend that a general
4 Research Design
T H R E E ELEMENTS OF INQUIRY
In the first edition of this book, I used two approaches-qualitative and
quantitative. I described each in terms of different philosophical
assumptions about the nature of reality, epistemology, values, the
rhetoric of research, and methodology (Creswell, 1994). Several devel-
opments in the last decade have caused a reexamination of this stance.
Mixed methods research has come of age. To include only quantita-
tive and qualitative methods falls short of the major approaches
being used today in the social and human sciences.
Other philosophical assumptions beyond those advanced in 19 94
have been widely discussed in the literature. Most notably, critical
perspectives, advocacy/participatory perspectives, and pragmatic
ideas (e.g., see Lincoln & Guba, 2000; Tashakkori & Teddlie, 1998)
are being extensively discussed. Although philosophical ideas
remain largely "hidden" in research (Slife & Williams, 199 S), they
still influence the practice of research and need to be identified.
The situation today is less quantitative versus qualitative and more
how research practices lie somewhere on a continuum between the
two (e.g., Newrnan & Benz, 1998). The best that can be said is that
studies tend to be more quantitative or qualitative in nature. Thus,
later in the chapter I introduce typical scenarios of quantitative,
qualitative, and mixed methods research.
Finally, the practice of research (such as writing a proposal) involves
much more than philosophical assumptions. Philosophical ideas
must be combined with broad approaches to research (strategies)
and implemented with specific procedures (methods). Thus, a
framework is needed that combines the elements of philosophical
ideas, strategies, and methods into the three approaches to research.
Crotty's (1998) ideas established the groundwork for this framework.
He suggested that in designing a research proposal, we consider four
1. What epistemology-theory
theoretical perspective--informs the research (e.g., objectivism,
of knowledge embedded in the
2. What theoretical perspective-philosophical
behind the methodology in questions (e.g., positivism and
postpositivm, interpretivism, critical theory, etc.)?
A Framework for Design 5
Elements of Inquiry
Alternative Knowledge Claims
Strategies of Inquiry -
Approaches to Research
by the researcher
Mixed Methods -
~ ~ ~ l ~ t e d
Figure 1.1 Knowledge Claims. Strategies of Inquiry, and Methods
Leading to Approaches and the Design Process
3. What methodology-strategy
methods to outcomes-governs our choice and use of methods
(e.g., experimental research, survey research, ethnography, etc.)?
or plan of action that links
4. What methods-techniques and procedures40 we propose to
use (e.g., questionnaire, interview. focus group, etc.)?
These four questions show the interrelated levels of decisions that go
into the process of designing research. Moreover, these are aspects that
inform a choice of approach, ranging h m the broad assumptions that
are brought to a project to the more practical decisions made about how
to collect and analyze data.
With these ideas in mind, I conceptualized Crotty's model to address
three questions central to the design of research:
1. What knowledge claims are being made by the researcher
(including a theoretical perspective)?
2. What strategies of inquiry will inform the procedures?
3. What methods of data collection and analysis will be used?
Next, I drew a picture, as shown in Figure 1.1. This displays how three
elements of inquiry (i.e., knowledge claims, strategies, and methods)
combine to form different approaches to research. These approaches, in
turn, are translated into processes in the design of research. Preliminary
steps in designing a research proposal, then, are to assess the knowledge
claims brought to the study, to consider the strategy of inquiry that will
be used, and to identify specific methods. Using these three elements, a
Alternative Knowledge Claim Positions
Multiple participant meanings
Social and historical construction
Consequences of actions
Real-world practice oriented
researcher can then identify either the quantitative, qualitative, or mixed
methods approach to inquiry.
Alternative Knowledge Claims
Stating a knowledge claim means that researchers start a project with
certain assumptions about how they will learn and what they will
learn during their inquiry. These claims might be called paradigms
(Lincoln & Guba, 2000; Mertens, 1998); philosophical assumptions,
epistemologies, and ontologies (Crotty, 19 9 8); or broadly conceived
research methodologies (Neuman, 2000). Philosophically, researchers
make claims about what is knowledge (ontology), how we know it
(epistemology), what values go into it (axiology), how we write about it
(rhetoric), and the processes for studying it (methodology) (Creswell,
1994). Four schools of thought about knowledge claims will be
discussed: postpositivism, constructivism, advocacy/participatory, and
pragmatism. The major elements of each position are presented in
Table 1.1. In discussions to follow, I will attempt to translate the broad
philosophical ideas of these positions into practice.
Postpositive Knowledge Claims
Traditionally, the postpositivist assumptions have governed claims
about what warrants knowledge. This position is sometimes called
the "scientific method" or doing "science" research. It is also called
quantitative research, positivist/postpositivist research, empirical science,
A Framework for Design 7
and postpostivism. The last term, "postpositivism," refers to the thinking
after positivism, challenging the traditional notion of the absolute truth
of knowledge (Phillips & Burbules, 2000) and recognizing that we
cannot be "positive" about our claims of knowledge when studying the
behavior and actions of humans. The postpositivist tradition comes
from 19th-century writers such as Comte, Mill, Durkheim, Newton, and
Locke (Smith, 1983), and it has been most recently articulated by
writers such as Phillips and Burbules (2000).
Postpositivism reflects a deterministic philosophy in which causes
probably determine effects or outcomes. Thus, the problems studied by
postpositivists reflect a need to examine causes that influence outcomes,
such as issues examined in experiments. It is also reductionistic in that
the intent is to reduce the ideas into a small, discrete set of ideas to test,
such as the variables that constitute hypotheses and research questions.
The knowledge that develops through a postpositivist lens is based on
careful observation and measurement of the objective reality that exists
"out there" in the world. Thus, developing numeric measures of obser-
vations and studying the behavior of individuals become paramount for
a postpositivist. Finally, there are laws or theories that govern the world,
and these need to be tested or verified and refined so that we can under-
stand the world. Thus, in the scientific method-the
to research by postpostivists-an individual begins with a theory, col-
lects data that either supports or refutes the theory, and then makes
necessary revisions before additional tests are conducted.
In reading Phillips and Burbules (2000). one can gain a sense of the
key assumptions of this position, such as the following:
1. That knowledge is conjectural (and anti-foundationa1)-
absolute truth can never be found. Thus, evidence established
in research is always i m p e r f e c t and fallible. It is for this reason
that researchers do not prove hypotheses and instead indicate a
failure to reject.
2. Research is the process of making claims and then refining or
abandoning some of them for other claims more strongly war-
ranted. Most quantitative research. for example, starts with the
test of a theory.
3. Data, evidence, and rational considerations shape knowledge.
In practice, the researcher collects information on instruments
based on measures completed by the participants or by
observations recorded by the researcher.
8 Research Design
4. Research seeks to develop relevant true statements, ones that
can serve to explain the situation that is of concern or that
describes the causal relationships of interest. In quantitative
studies, researchers advance the relationship among variables
and pose this in terms of questions or hypotheses.
5. Being objective is an essential aspect of competent inquiry, and
for this reason researchers must examine methods and conclu-
sions for bias. For example, standards of validity and reliability
are important in quantitative research.
Socially Constructed Knowledge Claims
Others claim knowledge through an alternative process and set
of assumptions. Social constructivism (often combined with inter-
pretivism; see Mertens, 1998) is such a perspective. The ideas came from
Mannheim and from works such as Berger and Luckmann's The Social
Construction of Reality (1 9 6 7) and Lincoln and Guba's Naturalistic
Inquiry (1985). More recent writers who have summarized this position
are Lincoln and Guba (2000), Schwandt (2000), Neuman (2000), and
Crotty (1 998), among others. Assumptions identified in these works hold
that individuals seek understanding of the world in which they live and
work. They develop subjective meanings of their experiences-meanings
directed toward certain objects or things. These meanings are varied and
multiple, leading the researcher to look for the complexity of views rather
than narrowing meanings into a few categories or ideas. The goal of
research, then, is to rely as much as possible on the participants' views of
the situation being studied. The questions become broad and general so
that the participants can construct the meaning of a situation, a mean-
ing typically forged in discussions or interactions with other persons. The
more open-ended the questioning, the better, as the researcher listens
carefully to what people say or do in their life setting. Often these subject-
ive meanings are negotiated socially and historically. I n other words, they
are not simply imprinted on individuals but are formed through interact-
ion with others (hence social constructivism) and through historical and
cultural norms that operate in individuals' k.
researchers often address the "processes" of interaction among individu-
als. They also focus on the specific contexts in which people live and work
in order to understand the historical and cultural settings of the partici-
pants. Researchers recognize that their own background shapes their
interpretation, and they "position themselves" in the research to
acknowledge how their interpretation flows from their own personal,
A Framework for Design 9
cultural, and historical experiences. The researcher's intent, then, is to
make sense of (or interpret) the meanings others have about the world.
Rather than starting with a theory (as in postpostivism), inquirers
generate or inductively develop a theory or pattern of meaning.
For example, in discussing constructivism, Crotty (1 9 9 8) identified
1. Meanings are constructed by human beings as they engage
with the world they are interpreting. Qualitative researchers
tend to use open-ended questions so that participants can
express their views.
Humans engage with their world and make sense of it based on
their historical and social perspectivewe are all born into a
world of meaning bestowed upon us by our culture. Thus, qual-
itative researchers seek to understand the context or setting of
the participants through visiting this context and gathering
information personally. They also make an interpretation of
what they find, an interpretation shaped by the researchers'
own experiences and backgrounds.
3. The basic generation of meaning is always social, arising in and
out of interaction with a human community. The process of
qualitative research is largely inductive, with the inquirer gen-
erating meaning from the data collected in the field.
Advocacy/Participatow Knowledge Claims
Another group of researchers claims knowledge through an advo-
cacylparticipatory approach. This position arose during the 1980s and
1990s from individuals who felt that the postpostivist assumptions
imposed structural laws and theories that did not fit marginalized indi-
viduals or groups or did not adequately address issues of social justice.
Historically, some of the advocacy/participatory (or emancipatory)
writers have drawn on the works of Marx, Adorno, Marcuse, Habermas,
and Freire (Neuman, 2000). More recently, works by Fay (1987), Heron
and Reason (199 7), and Kemmis and WWson (1998) can be read for
this perspective. In the main, these inquirers felt that the constructivist
stance did not go far enough in advocating for an action agenda to help
marginalized peoples. These researchers believe that inquiry needs to be
intertwined with politics and a political agenda. Thus, the research
should contain an action agenda for reform that may change the lives of
10 Research Design
the participants, the institutions in which individuals work or live, and
the researcher's life. Moreover, specific issues needed to be addressed
that speak to important social issues of the day, issues such as empower-
ment, inequality, oppression, domination, suppression, and alienation.
The advocacy researcher often begins with one of these issues as the
focal point of research. This research also assumes that the inquirer will
proceed collaboratively so as to not further marginalize the participants
as a result of the inquiry. In this sense, the participants may help design
questions, collect data, analyze information, or receive rewards for par-
ticipating in the research. The "voice" for the participants becomes a
united voice for reform and change. This advocacy may mean providing
a voice for these participants, raising their consciousness, or advancing
an agenda for change to improve the lives of the participants.
Within these knowledge claims are stances for groups and individu-
als in society that may be marginalized or disenfranchised. Therefore,
theoretical perspectives may be integrated with the philosophical
assumptions that construct a picture of the issues being examined, the
people to be studied, and the changes that are needed. Some of these
theoretical perspectives are listed below.
Feminist perspectives center and make problematic women's diverse
situations and the institutions that b
topics may include policy issues related to realizing social justice for
women in specific contexts or knowledge about oppressive situa-
tions for women (Olesen, 2000).
e those situations. Research
Racialized discourses raise important questions about the control and
production of knowledge, particularly knowledge about people and
communities of color (Ladson-Billings, 2000).
Critical theory perspectives are concerned with empowering human
beings to transcend the constraints placed on them by race, class,
and gender (Fay, 19 8 7).
Queer theory focuses on individuals calling themselves lesbians, gay,
bisexuals, or transgendered people. The research can be less objecti-
fying, can be more concerned with cultural and political means, and
can convey the voices and experiences of individuals who have been
suppressed (Gamson, 2000).
Disability inquiry addresses the meaning of inclusion in schools
and encompasses administrators, teachers. and parents who have
children with disabilities (Mertens, 1998).
A Framework for Design 11
These are diverse groups and topics, and my summaries here are
inadequate generalizations. It is helpful to view the summary by
Kemrnis and Wilkinson (1998) of key features of the advocacy or
participatory forms of inquiry:
1. Participatory action is recursive or dialectical and is focused on
bringing about change in practices. Thus, at the end of advo-
cacylparticipatory studies, researchers advance an action
agenda for change.
2. It is focused on helping individuals free themselves from con-
straints found in the media, in language, in work procedures,
and in the relationships of power in educational settings. Advo-
cacy/participatory studies often begin with an important issue
or stance about the problems in society, such as the need for
3. It is emancipatory in that it helps unshackle people from the
constraints of irrational and unjust structures that limit self-
development and self-determination. The aim of advocacylpar-
ticipatory studies is to create a political debate and discussion so
that change will occur.
4. It is practical and collaborative because it is inquiry completed
"with" others rather than "on" or "to" others. In this spirit,
advocacy/participatory authors engage the participants as
active collaborators in their inquiries.
Pragmatic Knowledge Claims
Another position about claims on knowledge comes from the prag-
matists. Pragmatism derives from the work of Peirce, James, Mead, and
Dewey (Cherryholmes, 1992). Recent writers include Rorty (1990),
Murphy (1990), Patton (1990), and Cherryholmes (1992). There are
many forms of pragmatism. For many of them, knowledge claims arise
out of actions, situations, and consequences rather than antecedent
conditions (as in postpositivism). There is a concern with applica-
tions-"what worksv-and solutions to problems (Patton, 1990).
Instead of methods being important, the problem is most important,
and researchers use all approaches to understand the problem
(see Rossman & Wilson, 1985). As a philosophical underpinning for
mixed methods studies, Tashakkori and Teddlie (1998) and Patton
(1990) convey the importance for focusing attention on the research
12 Research Design
problem in social science research and then using pluralistic approaches
to derive knowledge about the problem. According to Cherryholmes
(1992). Murphy (1990), and my own interpretations of these writers,
pragmatism provides a basis for the following knowledge claims:
1. Pragmatism is not committed to any one system of philosophy
and reality. This applies to mixed methods research in that
inquirers draw liberally from both quantitative and qualitative
assumptions when they engage in their research.
2. Individual researchers have a freedom of choice. They are
"free" to choose the methods, techniques, and procedures of
research that best meet their needs and purposes.
3. Pragmatists do not see the world as an absolute unity. In a sim-
ilar way, mixed methods researchers look to many approaches
to collecting and analyzing data rather than subscribing to only
one way (e.g., quantitative or qualitative).
4. Truth is what works at the time; it is not based in a strict dual-
ism between the mind and a reality completely independent of
the mind. Thus, in mixed methods research, investigators use
both quantitative and qualitative data because they work to
provide the best understanding of a research problem.
5. Pragmatist researchers look to the 'what" and "how" to research
based on its intended consequen-where
it. Mixed methods researchers need to establish a purpose for
their "mixing," a rationale for the reasons why quantitative and
qualitative data need to be mixed in the first place.
they want to go with
6. Pragmatists agree that research always occurs in social, histori-
cal, political, and other contexts. In this way, mixed methods
studies may include a postmodern turn. a theoretical lens that
is reflexive of social justice and political aims.
7. Pragmatists believe (Cherryholmes, 1992) that we need to stop
asking questions about reality and the laws of nature. "They
would simply like to change the subject" (Rorty, 1983, p. xiv).
Thus, for the mixed methods researcher, pragmatism opens the door to
multiple methods, different worldviews, and diflerent assumptions, as
well as to different forms of data collection and analysis in the mixed
A Framework for Design 13
Strategies of Inquiry
The researcher brings to the choice of a research design assumptions
about knowledge claims. In addition, operating at a more applied level
are strategies of inquiry (or traditions of inquiry, Creswell, 1998; or
methodologies, Mertens, 1998) that provide specific direction for proce-
dures in a research design. Like knowledge claims, strategies have mul-
tiplied over the years as computer technology has pushed forward data
analysis and the ability to analyze complex models, and as individuals
have articulated new procedures for conducting social science research.
These strategies of inquiry contribute to our overall research approach.
The major strategies employed in the social sciences are discussed in
Chapters 9, 10, and 11 of this book. Rather than cover all or a large
number of strategies, these chapters focus on those frequently used in
the social sciences. Here I will introduce those that will be discussed
later and that are cited in examples of research throughout the book.
An overview of these strategies is shown in Table 1.2.
such as surveys
Strategies Associated With the Quantitative Approach
During the late 19th century and throughout the 20th, strategies of
inquiry associated with quantitative research were those that invoked
the postpositivist perspectives. These include the true experiments and
the less rigorous experiments called quasi-experiments and correlational
studies (Campbell & Stanley, 1963), and specific single-subject experi-
ments (Cooper, Heron, & Heward, 1987; Neuman & McCorrnick, 1995).
More recently, quantitative strategies involved complex experiments
with many variables and treatments (e.g., factorial designs and repeated
measure designs). They also included elaborate structural equation
models that incorporated causal paths and the identification of the
14 Research Design
collective strength of multiple variables. In this book, we will focus on
two strategies of inquiry: experiments and surveys.
Experiments include true experiments, with the random assignment
of subjects to treatment conditions, as well as quasi-experiments
that use nonrandomized designs (Keppel, 19 9 1). Included within
quasi-experiments are single-subject designs.
Surveys include cross-sectional and longitudinal studies using ques-
tionnaires or structured interviews for data collection, with the
intent of generalizing from a sample to a population (Babbie, 1990).
Strategies Associated With the Qualitative Approach
In qualitative research, the numbers and types of approaches also
became more clearly visible during the 1990s. Books have summarized
the various types (such as the 19 strategies identified by Wolcott, 2001),
and complete procedures are now available on specific qualitative
inquiry approaches. For example, Clandinin and Connelly (2000) have
constructed a picture of what "narrative researchers do," Moustakas
(1994) discussed the philosophical tenets and the procedures of the
phenomenological method, and Strauss and Corbin (1 990,199 8) have
explicated the procedures of grounded theory. Wolcott (1999) has sum-
marized ethnographic procedures, and Stake (199 5) has identified the
processes of case study research. In this book, illustrations will be drawn
from the following strategies:
Ethnographies, in which the researcher studies an intact cultural
group in a natural setting over a prolonged period of time by col-
lecting, primarily, observational data (Creswell, 1998). The research
process is flexible and typically evolves contextually in response to
the lived realities encountered in the field setting (LeCompte &
Grounded theory, in which the researcher attempts to derive a
general, abstract theory of a process. action, or interaction
grounded in the views of participants in a study. This process
involves using multiple stages of data collection and the refinement
and interrelationship of categories of information (Strauss &
Corbin, 1990,199 8). Two primary characteristics of this design are
the constant comparison of data with emerging categories and
theoretical sampling of different groups to maximize the similarities
and the differences of information.
A Framework for Design 15
Case studies, in which the researcher explores in depth a program,
an event, an activity, a process, or one or more individuals. The
case(s) are bounded by time and activity, and researchers collect
detailed information using a variety of data collection procedures
over a sustained period of time (Stake, 1995).
Phenomenological research, in which the researcher identifies the
"essence" of human experiences concerning a phenomenon, as
described by participants in a study. Understanding the "lived experi-
ences" marks phenomenology as a philosophy as well as a method,
and the procedure involves studying a small number of subjects
through extensive and prolonged engagement to develop patterns
and relationships of meaning (Moustakas, 1994). In this process, the
researcher "brackets" his or her own experiences in order to under-
stand those of the participants in the study (Nieswiadomy, 1993).
Narrative research, a form of inquiry in which the researcher studies
the lives of individuals and asks one or more individuals to provide
stories about their lives. This information is then retold or restoried
by the researcher into a narrative chronology. In the end, the nar-
rative combines views from the participant's life with those of the
researcher's life in a collaborative narrative (Clandinin &
Strategies Associated With the Mixed Methods Approach
Less well known than either the quantitative or qualitative strategies
are those that involve collecting and analyzing both forms of data in a
single study. The concept of mixing different methods probably origi-
nated in 1959, when Campbell and Fiske used multiple methods to
study validity of psychological traits. They encouraged others to employ
their "multimethod matrii" to examine multiple approaches to data
collection in a study. This prompted others to mix methods, and soon
approaches associated with field methods such as observations and
interviews (qualitative data) were combined with traditional surveys
(quantitative data) (S. D. Sieber, 1973). Recognizing that all methods
have limitations, researchers felt that biases inherent in any single
method could neutralize or cancel the biases of other methods. Trian-
gulating data sources-a means for seeking convergence across quali-
tative and quantitative methods-were
original concept of triangulation emerged additional reasons for mixing
different types of data. For example, the results from one method can
born (Jick, 1979). From the