ArticlePDF Available

Advancing Qualitative Research Using Qualitative Data Analysis Software (QDAS)? Reviewing Potential Versus Practice in Published Studies using ATLAS.ti and NVivo, 1994–2013

SAGE Publications Inc
Social Science Computer Review
Authors:

Abstract and Figures

Qualitative data analysis software (QDAS) programs are well-established research tools, but little is known about how researchers use them. This article reports the results of a content analysis of 763 empirical articles, published in the Scopus database between 1994 and 2013, which explored how researchers use the ATLAS.ti™ and NVivo™ QDAS programs.* The analysis specifically investigated who is using these tools (in terms of subject discipline and author country of origin), and how they are being used to support research (in terms of type of data, type of study, and phase of the research process that QDAS were used to support). The study found that the number of articles reporting QDAS is increasing each year, and that the majority of studies using ATLAS.ti™ and NVivo™ were published in health sciences journals by authors from the United Kingdom, United States, Netherlands, Canada, and Australia. Researchers used QDAS to support a variety of research designs and most commonly used the programs to support analyses of data gathered through interviews, focus groups, documents, field notes, and open-ended survey questions. Although QDAS can support multiple phases of the research process, the study found the vast majority of researchers are using it for data management and analysis, with fewer using it for data collection/creation or to visually display their methods and findings. This article concludes with some discussion of the extent to which QDAS users appear to have leveraged the potential of these programs to support new approaches to research.
This content is subject to copyright.
Article
Advancing Qualitative Research
Using Qualitative Data Analysis
Software (QDAS)? Reviewing
Potential Versus Practice in
Published Studies using ATLAS.ti
and NVivo, 1994–2013
Megan Woods
1
, Trena Paulus
2
, David P. Atkins
3
, and Rob Macklin
1
Abstract
Qualitative data analysis software (QDAS) programs are well-established research tools, but little is
known about how researchers use them. This article reports the results of a content analysis of 763
empirical articles, published in the Scopus database between 1994 and 2013, which explored how
researchers use the ATLAS.tiand NVivoQDAS programs.* The analysis specifically investigated
who is using these tools (in terms of subject discipline and author country of origin), and how they
are being used to support research (in terms of type of data, type of study, and phase of the research
process that QDAS were used to support). The study found that the number of articles reporting
QDAS is increasing each year, and that the majority of studies using ATLAS.tiand NVivowere
published in health sciences journals by authors from the United Kingdom, United States, Neth-
erlands, Canada, and Australia. Researchers used QDAS to support a variety of research designs and
most commonly used the programs to support analyses of data gathered through interviews, focus
groups, documents, field notes, and open-ended survey questions. Although QDAS can support
multiple phases of the research process, the study found the vast majority of researchers are using it
for data management and analysis, with fewer using it for data collection/creation or to visually
display their methods and findings. This article concludes with some discussion of the extent to
which QDAS users appear to have leveraged the potential of these programs to support new
approaches to research.
Keywords
qualitative data analysis software, CAQDAS, QDAS, ATLAS.ti, NVivo, qualitative research,
literature review
1
University of Tasmania, Sandy Bay, Tasmania, Australia
2
University of Georgia, Atlanta, GA, USA
3
University of Tennessee, Knoxville, TN, USA
Corresponding Author:
Megan Woods, University of Tasmania, Room 317, Commerce Building, Sandy Bay, Tasmania 7001, Australia.
Email: megan.woods@utas.edu.au
Social Science Computer Review
1-21
ªThe Author(s) 2015
Reprints and permission:
sagepub.com/journalsPermissions.nav
DOI: 10.1177/0894439315596311
ssc.sagepub.com
by guest on August 28, 2015ssc.sagepub.comDownloaded from
Introduction
In the 25 years since qualitative data analysis software (QDAS) programs were developed, the use of
such tools has been lauded as both a positive and worrying development for the field of qualitative
research. When the first generation of QDAS programs was developed in the 1980s, researchers
speculated that such tools could potentially advance qualitative research in several key ways. The
first was by extending paper-based techniques for coding, retrieving, and analyzing data ‘‘beyond
the feasible manual limits’’ (Richards & Richards, 1987, p. 29) by enabling more complex, adapta-
ble, extensive, and exhaustive coding schemes (Podolefsky, 1987; Richards & Richards, 1987;
Tallerico, 1991), easier, more efficient and more accurate retrieval of coded data for analysis
(Hesse-Biber, Dupois, & Kinder, 1991; Richards & Richards, 1987; Tallerico, 1991; Shelly &
Sibert, 1986) and better testing of the qualitative evidence for emerging theoretical propositions and
hypotheses (Hesse-Biber et al., 1991). The second possibility was that QDAS could improve quali-
tative analysis and interpretation by supporting forms of analyses that ‘‘would be impossible to carry
out manually’’ (Richards & Richards, 1991, p. 319), such as by enabling the ‘‘linking of text, anal-
ysis and non-text materials (graphics, sounds and video) in a single analytical space outside the
mind’s eye [which] is not possible manually’’ (Dohan & Sanchez-Jankowski, 1998, p. 484). The
third possibility was that by enabling more transparent analytical processes, QDAS could enable
validity, rigor and trustworthiness to be more readily demonstrated (Dainty, Bagilhole, & Neale,
1997; Fritz, 1990; Morison & Moir, 1998) and advance understanding of its practical application,
usefulness, and limitations (Blismas & Dainty, 2003; Bringer, Johnston, & Brackenridge, 2006).
This in turn could help others avoid common mistakes (Sin, 2007) and help the qualitative research
community identify how to best guide new QDAS users in avoiding pitfalls and problems in quali-
tative research (Mangabeira, Lee, & Fielding, 2004).
But the development of QDAS programs has also prompted concerns that the tools can influence
qualitative research in undesirable ways. Arguably the most enduring worry is that their develop-
ment for specific research contexts, such as grounded theory, might impose a methodological
‘straight jacket’’ around research activities (Holbrook & Butcher, 1996, p. 60). More narrowly, that
the technological parameters of QDAS programs can compromise the researchers’ ability to design
and execute research tailored to the needs of their projects is another concern (DeNardo & Levers,
2002). This could occur if researchers design their studies around the capabilities of software (Hol-
land, 2002), defer to program requirements (Gilbert, 2002), adopt ‘‘programmatic’’ approaches to
analysis (Morison & Moir, 1998, p. 114), or use a technique simply because the software allows
it (Garcia-Horta & Guerra-Ramos, 2009). This also suggests that software can dominate the research-
ers’ understanding of their practices (Bryman & Burgess, 1994; MacMillan & Koenig, 2004; Se´ror,
2005), especially if QDAS are used without a critical and reflexive awareness of how the software
influences qualitative research practices (Brown, 2002; Woods, Macklin, & Lewis, 2015).
Despite these hopes and fears, very few empirical studies have investigated how researchers actu-
ally use and experience QDAS tools (see Fielding & Lee, 1998; Gilbert, 2002; Mangabeira et al.,
2004; Marshall, 2002 for exceptions). Some scholars have examined the forms of analyses that can
be executed and have compared analyses using different programs (e.g., Evers, Silver, Mruck, &
Peeters, 2011; Hutchison, Johnston, & Breckon, 2010; MacMillan, 2005; Woods & Dempster,
2011), but such accounts primarily report the experiences of individual researchers and teams. Little
is known about how widely QDAS are used and what they are used for.
Greater understanding of how researchers use QDAS, and the implications of that usage, is
needed to determine whether QDAS programs ‘‘really provide a panacea to the analysis of qualita-
tive data’’ (Blismas & Dainty, 2003, p. 462) and/or have led to the feared problems. Better under-
standing of QDAS usage would also advance discourses about their implications, and thus inform
decision making around and training in the use of such tools.
2Social Science Computer Review
by guest on August 28, 2015ssc.sagepub.comDownloaded from
This article reports findings from a large-scale review of empirical studies undertaken to deter-
mine how QDAS are being used and reported by qualitative researchers. This article reports our spe-
cific findings about who QDAS are being used by, and how QDAS are being used: Our findings
about how researchers are reporting this usage are detailed elsewhere (Paulus, Woods, Atkins &
Macklin, Under Review).
Method
We conducted a qualitative content analysis (Finfgeld-Connett, 2014) of 763 empirical studies using
QDAS published in peer-reviewed journals between 1994 and 2013. We selected 1994 as it was the
year the Computer-Assisted Qualitative Data Analysis Software (CAQDAS) Networking Project at
the University of Surrey was established as a focal point for discussions about QDAS programs. We
specifically focused on studies using ATLAS.tior NVivo/NUD*IST(NUD*IST evolved into
NVivoin 2007), because they are two of the longest used QDAS tools (Muhr, 1991; Richards &
Richards, 1991). They are also the programs that we ourselves our familiar with; without this famil-
iarity of our analysis would not have been possible. Consistent with techniques for systematic liter-
ature searching (Bandara, Miskon, & Fielt, 2011), we defined the search strategies for identifying
and extracting relevant articles, including key words, sources and databases. We generated a data
set using Scopus (Elsevier), a broad-based, multidisciplinary journal citation database to give us a
comprehensive collection of peer-reviewed journal articles. Scopus was chosen over Web of Science
(Thomson Reuters) and Academic Search Premier (EBSCO) because our evaluation of all three
determined that Scopus offered comparable accessibility, multidisciplinarity, and data set size, but
superior post-search analytics and citation download formats: each Scopus citation contains a URL
for the full article.
To build our search set, we used Scopus to simultaneously search for citations by title, abstract,
and key word. Using the words ‘‘atlas’’ and ‘‘nudist’’ retrieved too many articles that had nothing to
do with QDAS. We thus limited our searching to proper program names (ATLAS.ti, NVivo,
NUD*IST). We then narrowed these search sets to peer-reviewed journal articles published in
English. We chose to analyze peer-reviewed journal articles because they are arguably the highest
quality academic publications. We chose English language publications because this is the dominant
language of the research team. Finally, we concentrated exclusively on empirical rather than meth-
odological articles because our focus was specifically on how researchers are incorporating QDAS
into their research practices and reporting. As Scopus does not support filtering using these charac-
teristics, we had to do this manually. We determined if an article was ‘‘empirical’’ or ‘‘methodolo-
gical’’ when coding each article.
These strategies produced a final data set of 763 articles: 349 studies used ATLAS.tiand 414
studies used NVivo/NUD*IST. Our final data set heavily represented researchers from the
health sciences fields, which we attribute to a possible subject-discipline bias within the Scopus
database. Elsevier reports that Scopus includes 100%of the journal citations found in Medline, the
premier health sciences database. Elsevier also categorizes 33%of Scopus’s 19,400 indexed peer-
reviewed journals (approximately 6,400 titles) as health sciences journals. Arts, Humanities, and
Social Sciences (AHSS) journals account for 20%of Scopus journal title coverage. In 2012, Scopus
expanded its AHSS journal coverage to 4,000 peer-reviewed journals. However, at that time, 20%of
these AHSS titles were not indexed before 2002 (Elsevier, 2014).
We analyzed the articles to determine the characteristics of the authors and how the authors
reported using ATLAS.tior NVivoin their research. Consistent with Bandara et al.’s (2011)
recommendations, we developed and tested a priori coding schemes and protocols. This included
defining the codes to be used, the data to be captured by each code, whether text would be coded
by line, sentence, or paragraph, and how coder observations would be captured. Our coding scheme
Woods et al. 3
by guest on August 28, 2015ssc.sagepub.comDownloaded from
used three sets of codes: bibliographic codes, application codes, and coder’s reflections (see Table 1
for the complete set of codes used in the analysis). Our Scopus database searches generated basic
bibliographic data, which was automatically populated into a spreadsheet. We then used a prede-
fined code set for manually recording bibliographic codes which included the corresponding
author’s country of origin and subject discipline of the journal.
Application codes were used to categorize the types of research study conducted, types of data
collected/created, and phases of the research process when QDAS was used (literature review, data
collection/creation, data analysis/management, and data display/representation of findings). We
used the author’s description of their study as the basis for coding the types of study. For example,
where the authors claimed to use grounded theory, we coded the article as ‘‘grounded theory,’’ with-
out assessing or evaluating how well they implemented this research design. In cases where the
authors described their research designs more generically (e.g., as an ‘‘exploratory study’’ or a ‘‘qua-
litative study’’), we coded the article ‘‘generic qualitative.’’ We used predefined codes for data types
(e.g., interview, focus groups, video), modifying the categories as the analysis proceeded. Coding
of the phase of the research process relied on authors’ descriptions of their QDAS use. Any
additional detail about the articles or their coding was noted as free text in the Coder’s Reflections
field. This allowed us to record and compare coder reflections and clarify coding decisions when
necessary.
Our analysis proceeded through two stages. Stage 1 involved developing and piloting our coding
strategies by using an Excel spreadsheet to code all articles in the data set that reported using
ATLAS.ti. We used Microsoft Excelbecause all research team members were proficient in
using it, it facilitated combining and reviewing team member coding, and it enabled reflection on
whether subsequent analyses might be enhanced by using a QDAS program. To ensure accurate data
entry and to enable filtering by code, we used techniques developed by Wickham, Dunn, and
Sweeney (2012) for coding large data sets of literature using spreadsheets. We created predefined
drop-down menus to ensure that we used a consistent set of codes, typographical errors didn’t com-
promise accurate coding, and spreadsheets could be merged and searched. We ensured intercoder
consistency by pilot testing the coding strategies with a sample of 10 articles and engaging in regular
discussions on emergent issues. The 10 articles were coded independently by each coder, followed
by discussions to rectify any inconsistencies. Once coding had been completed, individual spread-
sheets were merged into one data set.
By the end of Stage 1, we had identified two limitations to using Excel for the remainder of our
coding: The spreadsheets were unwieldy when working with 300þarticles and also unwieldy when
trying to review the coding for the entire dataset. During Stage 2, we completed pilot analyses of 20
of the previously coded articles—one undertaken by M.W. using NVivo, one undertaken by T.P.
using ATLAS.ti—to determine which program better supported our analytical approach and over-
came Excel’s limitations. Comparing the outputs of the analyses, we determined that coding with
NVivooffered the unique advantage of creating an indexing system of data categories (called
‘nodes’’ in the NVivolexicon) which all four team members could code into, providing an evol-
ving visual guide to the analysis and integrating our coding so by opening a node we could view the
data in any given category in its entirety. Consequently, we adopted NVivoto support the project,
first recoding the articles that reported ATLAS.tiusage and then analyzing the data set of NVivo
articles using the same coding process detailed above. This produced a final node system that dis-
played the full system of categories into which the data were coded, a quantitative tally of the num-
ber of articles coded into each category and, by opening the node, the source details and coded
content for each article coded to the node. The coded content was then retrieved and reviewed to
produce our findings about QDAS use. The node system was used to generate Tables 2–4; Figures 1
and 2 were generated by exporting the coded data and generating charts in Excel, as NVivocould
not support the generation of such complex graphs.
4Social Science Computer Review
by guest on August 28, 2015ssc.sagepub.comDownloaded from
Table 1. Final Set of Codes Used to Classify Publications.
Bibliographic Codes Application Codes
Coder’s Thoughts
and Reflections
Journal discipline
a
Country of
corresponding author Type of studies Types of data
Phase of research
process using QDAS
Other points
of interest
Coding rule:
Code for discipline using key words in
journal title, supplemented with
information located at publishers’
websites
Coding rule:
Code for country given for
corresponding author
Coding rule:
Excerpted from article text based
on author description
Coding rule:
Code for each separate type
of data mentioned in
Method section
Coding rule:
Excerpted from article text
based on author description
Coding rule:
Notes taken
by coder
Codes: Codes: Codes: Codes: Codes:
Agricultural sciences
Arts, language, music, and humanities
Business
Communication and information sciences
Computer science
Education
Engineering and Applied sciences
Health sciences
Math and statistics
Multidisciplinary
Physical and natural sciences
Social sciences
Social work
Other
Selected from country
list taken from ISO
3166 Country Codes
(http://www.iso.org/iso/
country_codes)
Action or participatory action research
Case study or analysis
Content analysis
Critical incident technique
Discourse or conversation analysis
Ethnography or observation study
Evaluation study
Feminist policy analysis
Focus group
Formative research or rapid assessment
Generic qualitative
Grounded theory
Interview study
Mixed methods
Narrative inquiry or analysis
Naturalistic inquiry
Phenomenology
Survey Research
Thematic analysis/qualitative content analysis
Verbal autopsy method
Conversational data
Documents
Focus groups
Interview data
Observational field notes
Online social media data
Other Survey or questionnaire
Video or image data
Websites
Data collection/creation
Data analysis/management
Data display/representation
of findings
Literature review
a
Adapted from Paulus, Phipps, Harrison, and Varga (2012).
5
by guest on August 28, 2015ssc.sagepub.comDownloaded from
Findings
This section reports on the users and uses of QDAS as detailed in the 763 empirical articles: 349 arti-
cles published between 2000 and 2013 reported studies using ATLAS.tiand 414 articles published
between 1994 and 2013 reported using NVivo. As Figure 1 illustrates, the number of articles
reporting QDAS use has increased each year over the last decade. In 2012, there was a 50%increase
in the numbers of articles published over the previous year. Whether these increases are due to more
researchers using the programs or QDAS users being increasingly successful in publishing their
studies cannot be determined from our data.
We present our findings in two parts. First, we report on who is using QDAS, as illustrated by the
subject disciplines of the journals and the geographic distribution of lead authors. Second, we report
on how scholars are using QDAS, as indicated by types of studies, types of data analyzed, and phases
of the research process where QDAS was used.
Table 2. Top 10 Countries of Corresponding Authors.
ATLAS.tiNVivoTotal
United States 158 109 267
United Kingdom 50 108 158
Australia 2 74 76
Canada 15 31 46
Netherlands 20 9 29
South Africa 9 4 13
Italy 10 1 11
New Zealand 2 8 10
Ireland 1 7 8
Germany 8 0 8
Figure 1. Qualitative data analysis software articles published by year, 1994–2013.
6Social Science Computer Review
by guest on August 28, 2015ssc.sagepub.comDownloaded from
Part 1: QDAS Users
Subject disciplines of journals publishing QDAS-supported studies. We found that studies conducted using
QDAS had been published in journals from a diverse array of disciplines (see Figure 2). The major-
ity of studies (72%) were in journals related to health sciences. Medicine, nursing, general health
care, and public health/epidemiology were the four disciplinary categories most represented in our
data set. Within the health sciences, more studies used NVivothan ATLAS.ti.
Geographic distribution/representation of QDAS usage. We found that ATLAS.tiand NVivouse
were reported by researchers from a total of 58 countries. ATLAS.tiusers were represented in
Table 4. Data Types Used.
ATLAS.tiNVivoTotal
Data Type Articles % Articles % Articles %
Interview data 233 66.8 326 78.7 559 73.3
Focus groups 115 33.0 64 15.5 179 23.5
Documents 37 10.6 55 13.3 92 12.1
Observational field notes 47 13.5 40 9.7 87 11.4
Survey or questionnaire 27 7.7 51 12.3 78 10.2
Video or image data 16 4.6 11 2.7 27 3.5
Conversational data 11 3.2 15 3.6 26 3.4
Online social media data 5 1.4 8 1.9 13 1.7
Other 6 1.7 5 1.2 11 1.4
Websites 1 0.3 6 1.4 7 0.9
Table 3. Types of Research Studies Conducted With QDAS Support.
ATLAS.tiNVivoTotal
Research Type Articles % Articles % Articles %
Generic qualitative 29 8.3 150 36.2 179 23.5
Interview study 112 32.1 35 8.5 147 19.3
Focus group 74 21.2 40 9.7 114 14.9
Grounded theory 45 12.9 55 13.3 100 13.1
Thematic analysis or qualitative content analysis 0 0.0 63 15.2 63 8.3
Content analysis 16 4.6 32 7.7 48 6.3
Ethnography or observation study 28 8.0 16 3.9 44 5.8
Case study or analysis 19 5.4 22 5.3 41 5.4
Mixed methods 18 5.2 18 4.3 36 4.7
Phenomenology 13 3.7 18 4.3 31 4.1
Narrative inquiry or analysis 16 4.6 5 1.2 21 2.8
Survey research 13 3.7 7 1.7 20 2.6
Action research or participatory action 8 2.3 7 1.7 15 2.0
Discourse analysis or conversation analysis 5 1.4 4 1.0 9 1.2
Evaluation study 0 0.0 7 1.7 7 0.9
Formative research or rapid assessment 2 0.6 5 1.2 7 0.9
Clinical study 0 0.0 4 1.0 4 0.5
Critical incident technique 0 0.0 3 0.7 3 0.4
Feminist policy analysis 0 0.0 1 0.2 1 0.1
Naturalistic inquiry 0 0.0 1 0.2 1 0.1
Verbal autopsy method 0 0.0 1 0.2 1 0.1
Woods et al. 7
by guest on August 28, 2015ssc.sagepub.comDownloaded from
46 countries and NVivousers in 37 countries. Our review’s focus on English language articles
undoubtedly contributed to the heavy distribution toward English-speaking countries. As Table 2
illustrates, the top four countries for ATLAS.tiwere the United States, United Kingdom, Nether-
lands, and Canada; the top four countries for NVivowere the United States, United Kingdom,
Australia, and Canada. The relatively high number of articles using NVivopublished by Austra-
lian authors may be partly explained by NVivo’s origins in Australia. ATLAS.tioriginated in
Germany, but we did not find a similarly high number of studies authored by Germans. We attribute
this to our focus on English language publications.
Part 2: QDAS Use
Types of studies conducted with QDAS support. Authors reported using QDAS to support many different
types of studies. The vast majority (95.3%) reported using QDAS to support qualitative studies;
the remaining 4.72%were mixed method studies. A finer-grained analysis of the types of research
using QDAS support was complicated by authors’ descriptions. Some researchers defined types of
research in terms of research traditions, for instance, by describing their research as an ethnographic
study. Others described their research in terms of the types of methodology being used, most nota-
bly, grounded theory. Still others described their study in terms of the methods used, such as
an ‘‘interview study’’ or ‘‘focus group study.’’ Additionally, we found that in almost a quarter of
Figure 2. Subject disciplines of journals publishing ATLAS.tiand NVivostudies.
8Social Science Computer Review
by guest on August 28, 2015ssc.sagepub.comDownloaded from
the articles, authors described their research generically, such as a ‘‘qualitative design’’ or simply a
‘qualitative study’’. Consequently, our exploration of the types of research being conducted using
QDAS identified 21 different types (summarized in Table 3), including a variety of methodologies,
research traditions, and methods.
The high number of studies citing a generic qualitative approach or describing their research as
thematic or qualitative content analysis may be partially explained by the reporting practices of
researchers in the health science disciplines. They commonly described their studies using general
terms such as ‘‘a qualitative study’’ (Mutemwa et al., 2013, p. 1), ‘‘descriptive qualitative research’
(Bedos, Loignon, Landry, Allison, & Richard, 2013, p. 1), a ‘‘qualitative, explorative study using
open patient interviews’’ (Boeckxstaens et al., 2012, p. 183), or a ‘‘qualitative longitudinal study’
(Grime, Richardson, & Ong, 2010, p. 597). However, cross-tabulating our coding by type of research
and journal discipline indicated that while such practices were more common among health science
researchers, they were also frequently used by researchers from other disciplines, including Busi-
ness, Communication and Information systems, and Engineering and Applied Sciences.
The types of studies most frequently described by researchers using ATLAS.tiand NVivo
differed for each program. For ATLAS.ti, the most frequently cited type was interviews
(32.1%), followed by focus groups (21.2%), grounded theory (12.9%), generic qualitative (8.3%),
and ethnographic or observational studies (8%). For NVivo, it was generic ‘‘qualitative studies’
(36.2%), thematic analysis/qualitative content analysis (15.2%), grounded theory (13.3%), focus
group (9.7%) and interviews (8.5%).
Data types used in QDAS-supported studies. We also explored the types of data that researchers
reported analyzing with ATLAS.tior NVivoas illustrated in Table 4. Many studies collected
more than one type of data: 24%used two types and 9%used three or more types. Nevertheless, the
five most frequently cited types of data were interview data (73%), focus group data (23%), docu-
ments (12%), observational field notes (11%), and responses to open-ended survey questions (10%).
While interview data were the most common type used by both ATLAS.tiand NVivousers,
interview data were more heavily used with NVivo(78.7%) than ATLAS.ti(66.8%). In contrast,
ATLAS.tiusers made heavier use of focus group data than NVivousers (33%and 15%
respectively).
The phase of the research process supported by QDAS. We found that researchers reported using
ATLAS.tiand NVivoto support three phases of the research process: data collection/creation,
data analysis/management, and data display/representation of findings. Data analysis/management
was the most frequently mentioned use, with 99.6%of the studies using software for this purpose.
No articles reported using the software to support literature reviews and only six articles reported
using software for data collection/creation. Only 10.4%reported using the software for data dis-
play/representation of findings. In the following sections, we detail how researchers reported using
ATLAS.tiand NVivoin each of these three phases of the research process.
Data collection/creation. Both ATLAS.tiand NVivooffer functions for creating text files and
for transcribing audio and video files, making it possible for researchers to create field notes, inter-
view notes, reflective journal entries, and interview transcripts within the software (Friese, 2014;
QSR International, 2014a). In our sample, authors reported using this functionality to create study
journals and memos (e.g., Jakobsen & McLaughlin, 2004) and to generate transcripts of recorded
interviews (Szeinbach, Seoane-Vazquez, & Summers, 2012) and focus groups (e.g., Hawthorne
et al., 2011).
Since 2008, it has been possible to use both ATLAS.tiand NVivoto code audio or video
multimedia files directly, eliminating the need to transcribe recorded interviews or focus group
Woods et al. 9
by guest on August 28, 2015ssc.sagepub.comDownloaded from
discussions prior to analysis. However, we only identified two studies (De Gregorio, 2011; Larkin,
2009) in which researchers reported coding directly from audio or video files. In both cases,
ATLAS.tiwas used to analyze observational data recorded on video. We did not identify any stud-
ies in which the authors reported coding interview or focus group data directly from audio and video
recordings. In 16 cases, this could not be determined, as the authors provided no detail as to how
their interview data were captured. In six other cases, authors reported recording their interviews
or focus group discussions but did not specify whether or how they analyzed the recordings or tran-
scripts. In the main, it appears researchers are not yet leveraging the functionality in programs to
code directly from multimedia files.
Data analysis/management. As noted above, the vast majority of the studies we reviewed (99.6%)
reported using ATLAS.tior NVivofor data analysis and/or data management. Authors used
both terms when describing how they used their QDAS program, but their descriptions left some
doubt as to what distinguished these aspects of the research process. Some authors (e.g., Hurley,
2009) used the phrases ‘‘data analysis’’ and ‘‘data organization’’ synonymously and their subsequent
descriptions indicated that by data management they primarily meant coding. Others distinguished
between data management and analysis (e.g., Bennett et al., 2011; Laditka et al., 2009) or data man-
agement and coding (Waldrop, 2006) but rarely provided any additional detail or explanation of how
data analysis and data management were distinct or executed distinctly. The few exceptions (e.g.,
Barton, Sulaiman, Clarke, & Abramson, 2005; Gilliam, 2007) indicated that ‘data management’
included tasks such as ‘‘organising and preparing the data for analysis’’ (Chirwa, Malata, & Norr,
2011, p. 33), with Hartel (2010) providing the clearest distinction between data management as the
process of managing the growing set of data records collected through their ethnographic study and
data analysis as the coding of materials. These findings suggest that researchers differentiate
between using QDAS tools for data management and data analysis, but more research is needed
to explore the distinction between these two usages.
Turning specifically to analysis, authors commonly reported using both programs to support cod-
ing and the development of coding schema summarizing the topics or concepts represented by the
data. Illustrative examples included coding schemas (see Figure 3) developed to reflect aspects of
lived experience (e.g., Newman, Bogo, & Daley, 2008), barriers and facilitators of behavioral pro-
cesses (e.g., Ali, Baynouna, & Bernsen, 2010), and evaluations of interventions and educational
models (e.g., Moore, Morris, Crouch, & Martin, 2003).
Most authors used ATLAS.tior NVivoto assign codes to the data and then review all the data
to which a specific code has been applied. In ATLAS.ti, this involves creating ‘‘quotations’’ of
data and assigning a code, after which all quotations with the same code can be retrieved by running
a report or viewed in context by using the code manager. Assigning a code to a data segment (e.g.,
paragraph of text) in NVivoalso assigns the data to a node (data category) and thus produces two
outputs: a coded data set and a node system that provides an index of the major and subsidiary cate-
gories into which the data have been coded.
Researchers reported using the programs to retrieve and review the data in various ways, such as
to ‘‘retrieve quotes to dimensionalize each theme ... [and] substantiate and describe the findings’’
(Armour, Bradshaw, & Roseborough, 2009, p. 606), enable constant comparison of newly coded and
previously coded material to determine whether the same concepts are apparent (Curry, Taylor,
Chen, & Bradley, 2012), facilitate ‘‘retrieval of related quotations in order to examine patterns and
trends in the data’’ (de Villiers, Koko-Mhlahlo, & Senekal, 2005, p. 523), enable auditing of analy-
tical process and interpretations by peers (Thongpriwan & McElmurry, 2009), construct lineal
narratives (Gibson, Callery, Campbell, Hall, & Richards, 2005), and enable the grouping of codes
into categories representing broader and more abstract themes (e.g., Hannes, Janssens, & Wets,
10 Social Science Computer Review
by guest on August 28, 2015ssc.sagepub.comDownloaded from
2009). For a fuller discussion of how researchers described using QDAS to support their analyses,
see Paulus, Woods, Atkins, and Macklin (Under Review).
Another analytical practice identified was researchers using the QDAS program to investigate
potential relationships between concepts or between concepts and participant characteristics.
ATLAS.tiusers achieved this by using the co-occurrence feature to identify any data to which
multiple codes have been applied. For example, O’Halloran (2011) used co-occurrences to compare
and contrast the types of discussions that occurred in different types of reading groups (see Figure 4).
Researchers using NVivoreported running a matrix coding query that identifies co-occurrences
in the data fitting specified criteria and then reports it in a table format. Researchers determine the
variables of interest by specifying the content of rows and columns and the Boolean search term that
Figure 4. Distribution of discussion types across reading groups (O’Halloran, 2011, p. 182).
Figure 3. Sample illustration of coding scheme reporting themes identified in qualitative data from Newman,
Bogo, and Daley (2008, p. 221).
Woods et al. 11
by guest on August 28, 2015ssc.sagepub.comDownloaded from
connects the data, enabling the development and testing of patterns in the data. Our study identified
that researchers used matrices to investigate relationships in two ways. The first was to illustrate the
distribution of data across the sample of cases or participants. The second approach uses matrices to
determine that nodes represent discrete concepts. Bastos, Warson, and Barbour (2012), for example,
use a two-part approach to explore whether any data occurred in more than one node and might indi-
cate conceptual overlaps. They first generated matrices to compare nodes to identify any instances
where the same terms came up more than once, thereby suggesting the concepts may overlap. They
then generated a second set of matrices comparing nodes that did not overlap. Although ATLAS.ti
can be used to generate a codes-primary documents table, which is somewhat similar to an NVivo
coding matrix, no authors reported using this functionality.
Finally, our study found that researchers specifically reported using QDAS to support collabora-
tive team-based analyses in which several researchers were involved in analyzing a data set. Colla-
borative analytical approaches primarily involved multiple coders, but there were several variations
in how this was done. For some research teams, one researcher was primarily responsible for coding
with other team members coding a sample to check for accuracy and validity. More commonly, col-
laborative approaches involved multiple analysts coding the same data set and then integrating their
coding. Some teams also reported using collaborative analysis to inductively develop codes and cod-
ing rules that they then used to analyze the remainder of the data.
The current versions of ATLAS.ti(v. 7) and NVivo(v. 10) support collaborative analyses in
a variety of ways. Both record and track which researchers have added codes to the data, thus making
it possible to identify coders later when files are merged. NVivoalso offers a ‘‘coding compari-
son’’ function to check the consistency and interrater reliability of two researchers’ coding. It cal-
culates percentage agreement as well as a kcoefficient (QSR International, 2014b). However,
only 7.1%of the articles explicitly discussed using software features to support collaboration but
broad statements such as ‘‘coding was conducted by three independent analysts and checked for con-
sistency using ATLAS.tiqualitative software’’ (Kennedy, Grant, Walton, & Sandall, 2013, p. 139)
meant it was often unclear which software features they used. Some, such as Pilling et al. (2010)
reported using QDAS to keep an audit trail of the analysis. Most referred to using the software to
enable constant comparison of coded and uncoded material thereby facilitating intercoder consis-
tency over time. Some authors did provide specific detail as to how this was done, with Kirchhoff
et al. (2013, p. 379) providing one of the most detailed explanations that:
Two members of the research team coded all data independently using NVivo 8. At each analysis phase,
the two coders compared their results to confirm intercoder reliability (final k00.88), resolving discre-
pancies through discussion with the principal investigator and comparison of the raw data.
Data display/representation of findings. Both NVivoand ATLAS.tioffer functions for visually
displaying data and research findings. However, only 10.4%of the studies reported using the soft-
ware for this purpose. Of these, most used screen shots to illustrate coding processes, or program
outputs that visually depicted coding/conceptual schemas and relationships between data and codes.
One approach was to use NVivo’s coding matrices or ATLAS.ti’s co-occurrence tables to
generate tables and charts of code distributions (see Figure 5).
Another practice was to use ATLAS.ti’s network view (see Figure 6) or NVivo’s modeling
tool (see Figure 7) to illustrate relationships between codes, categories, and concepts, and to theorize
conceptual relationships.
At times, we had difficulty ascertaining whether or not the visual displays included in the pub-
lished articles were generated using the software. Researchers used QDAS to produce two types
of output for illustrating analytical process or conclusions: ‘‘native’’ outputs and ‘‘hybrid’’ outputs.
12 Social Science Computer Review
by guest on August 28, 2015ssc.sagepub.comDownloaded from
Native outputs were generated withthesoftwareandreportedin the format produced by the pro-
gram (such as the network view from ATLAS.tiin Figure 6). Examples included screen shots of
data categorization systems or exported versions of tables generated to illustrate coding distribu-
tions. Hybrid outputs were figures that authors reported generating with the software but which
had then been adapted or converted into other formats for inclusion in the article. Examples
included tables of coding distributions generated using the software but then reformatted using
Microsoft Word. Native outputs clearly demonstrate that QDAS programs are being used to illus-
trate analytical processes and conclusions, but hybrid outputs leavethisunclearsoitmaybethat
researchers are making more use of QDAS to illustrate their analytical processes than our study
could determine.
Discussion
This study was undertaken to determine how researchers are using ATLAS.tiand NVivoin
empirical studies as reported in peer-reviewed journals. Our specific objective was to determine both
who is currently using QDAS and how these researchers are leveraging the features, functionality,
and methodological opportunities offered by the programs. Our analysis found that the programs are
being used by researchers in a wide range of geographic and disciplinary areas and are primarily
being used to analyze textual data from interviews, focus groups, documents, field notes, and
open-ended survey responses. We found that researchers claimed to use QDAS to conduct many dif-
ferent types of studies, indicating that such tools are used to support a diverse array of research meth-
ods, methodologies, and analytical approaches. However, as many authors described their studies
generically as ‘‘qualitative studies’’ rather than as studies adopting a specific research methodology,
it is possible that actual usage may embrace an even wider range of research types than we could
identify through our study.
Figure 5. Tables and charts generated with NVivo’s coding matrix function to illustrate linkages between
expanded scope of analysis and project management elements (Nair, Malhotra, & Ahire, 2011, p. 546).
Woods et al. 13
by guest on August 28, 2015ssc.sagepub.comDownloaded from
Our investigation of the ways in which researchers used ATLAS.tiand NVivoprovides
empirical evidence that researchers are using QDAS to engage in analytical practices extending
beyond the limits of manual/paper-based techniques, most notably to support coding and retrieval
of data, differentiate coded data by participant characteristics, and investigate conceptual relation-
ships. We also found some evidence that researchers are using QDAS to make their analytical pro-
cesses more transparent, primarily by using program outputs to illustrate their coding processes and
research outputs. However, our finding that only about 10%of the studies used program outputs for
this purposes suggests there is more potential for researchers to do so. We hope that by reporting how
Figure 6. Nursing students’ positive characterizations of the clinical seminar experience produced with
ATLAS.ti(Granero-Molina et al., 2012, p. 444).
14 Social Science Computer Review
by guest on August 28, 2015ssc.sagepub.comDownloaded from
researchers are using screen shots and other program outputs, we are providing insights into how it
can be done.
We found little evidence of researchers leveraging QDAS to analyze new forms of data or adapt-
ing their research practices to leverage new program features. In recent years, new features have
been added to both programs enabling transcription of multimedia files and supporting direct anal-
ysis of multimedia data, social media data, geodata, and survey data sets. We thought these features
may have prompted researchers to analyze multimedia recordings directly, rather than first tran-
scribing them. This would eliminate the need for and costs of transcription (Evers, 2011). However,
we found little evidence of QDAS being used to analyze these new forms of data. We acknowledge
that it takes time for researchers to adopt these programs, incorporate them into their research prac-
tices, and then publish their accounts of using the tools in their research which may create a time lag
in doing so. It is also possible, as one of our reviewers suggested, that researchers have tried using
QDAS in these ways but reverted to previous practices because they found the process too cumber-
some. It may also be that researchers are using QDAS to develop new analytical techniques but are
not reporting this because of space constraints, or because researchers consider such techniques part
of their intellectual property. Alternatively, they may be reporting them in methodological articles,
which our study did not examine.
Researchers are also not yet reporting the use of QDAS for literature reviews. This was
unexpected, given reviewing literature is arguably the most universally utilized form of quali-
tative research, and the value of QDAS tools for this task has been acknowledged within the
methodological literature (Beekhuyzen, 2007; Di Gregorio, 2000; Paulus, Lester, & Dempster,
2014) and promoted by program developers. It may be that researchers are using QDAS for this
Figure 7. Schematic representation of how working capital is managed in small business developed with NVivo
(Orobia, Byabashaija, Munene, Sejjaaka, & Musinguzi, 2013, p. 139).
Woods et al. 15
by guest on August 28, 2015ssc.sagepub.comDownloaded from
purpose but not reporting it. As noted in our Method section, we tested the utility of both
ATLAS.tiand NVivoand found that both programs had utility for supporting systematic
meta-reviews of literature such as this one. Hopefully, researchers experimenting with of using
QDAS to review literature will encourage others to consider how they can leverage QDAS in
this way.
Our research did not identify any examples of researchers using QDAS to develop ‘‘qualitatively
new ways of doing things’’ (Bourdon, 2002, p. 7) or analytical approaches ‘‘which can truly be
described as new methodologies in their own right’’ (Cousins & Macintosh, 2005, p. 597) but that
may be due to the focus and scope of our study. Our review only examined English-language empiri-
cal studies that used ATLAS.tior NVivoand were indexed in the Scopus database. Future stud-
ies on QDAS use that examine non-English publications, publications in other databases, or other
QDAS tools may determine whether researchers are using the tools in other ways. It is possible that
methodological innovations are being reported elsewhere, such as in conference papers, methodo-
logical articles, or user blogs. Future studies that explore how QDAS use is reported in these other
forums could provide additional insights into the ways in which QDAS tools can advance qualitative
research. Another possibility is that QDAS innovations are occurring in nonacademic settings
(Mangabeira et al., 2004). Accounts from researchers in construction management (Dainty et al.,
1997) and market research (Catterall & Maclaren, 1998) have demonstrated the pragmatic value
of QDAS for conducting analyses efficiently when dealing with time constraints and funding pres-
sures. Examining nonacademic usages of QDAS could provide insights for academic users into the
ways in which these tools could overcome such constraints. Future studies could also explore
QDAS-specific analytical techniques such as the use of auto coding/word searching strategies and
the use of visualizations to see whether and how researchers use them to generate new analytical
approaches.
Future studies could also explore the ways in which disciplinary norms might be influencing the
QDAS usage and other research practices identified in our study. We found, for instance, that 86%of
the articles where authors reported using generic ‘‘qualitative’’ approaches and 85%of the articles
where authors claimed to be conducting thematic analysis or qualitative content analysis were pub-
lished in health sciences journals. Examining how researchers are trained to use and describe QDAS
programs through, for example, tertiary education curricula and the CAQDAS Network training pro-
grams may offer some explanation for the different usages and reporting practices identified in this
study. We thank our anonymous reviewer for this suggestion.
Author’s Note
*A short abstract broadly detailing our preliminary findings about Atlas usage was previously published in the
Proceedings of the 2013 Atlas Users conference,: Fostering dialog on Qualitative Methods, University of Tech-
nology Berlin, available at http://nbn-resolving.de/urn:nbn:de:kobv:83-opus4-44295. Partial findings were also
presented at the 25 years of CAQDAS conference, University of Surrey, England in May 2014, and at the
Eleventh International Congress of Qualitative Inquiry in Champaign-Urbana, Illinois in May 2015, and have
been significantly expanded and extended in this article.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or
publication of this article.
Funding
The authors would like to thank the University of Tasmania and the University of Tennessee for providing fund-
ing support for this study. Author 1 received study leave funding to visit authors 2 and 3 at the University of
Tennessee to undertake the first stage of data analysis. Both institutions also provided visiting scholar funding
16 Social Science Computer Review
by guest on August 28, 2015ssc.sagepub.comDownloaded from
for Authors 2 and 3 to travel to the University of Tasmania to complete the data analysis and develop our pre-
liminary findings.
References
Ali, H. I., Baynouna, L. M., & Bernsen, R. M. (2010). Barriers and facilitators of weight management: perspec-
tives of Arab women at risk for type 2 diabetes. Health and Social Care in the Community,18, 219–228.
Armour, M. P., Bradshaw, W., & Roseborough, D. (2009). African-Americans and recovery from severe mental
illness. Social Work in Mental Health,7, 602–622.
Bandara, W., Miskon, S., & Fielt, E. (2011). A systematic, tool-supported method for conducting literature
reviews in information systems. ECIS 2011 Proceedings. Paper 221. http://aisel.aisnet.org/ecis2011/221
Barton, C., Sulaiman, N., Clarke, D., & Abramson, M. (2005). Experiences of Australian parents caring for
children with asthma: It gets easier. Chronic Illness,1, 303–314.
Bastos, E., Warson, E., & Barbour, S. (2012). Exploring American Indian adolescents’ needs through a
community-driven study. The Arts in Psychotherapy,39, 134–142.
Bedos, C., Loignon, C., Landry, A., Allison, P. J., & Richard, L. (2013). How health professionals perceive and
experience treating people on social assistance: A qualitative study among dentists in Montreal, Canada.
BMC Health Services Research,13, 464–473.
Beekhuyzen, J. (2007). Putting the pieces of the puzzle together: Using Nvivo for a literature review. In
Proceedings for the 4th QUALIT Conference Qualitative Research in IT & IT in Qualitative Research. Well-
ington, New Zealand, Victoria University of Wellington, 19-20 November.
Bennett, W. L., Ennen, C. S., Carrese, J. A., Hill-Briggs, F., Levine, D. M., Nicholson, W. K., & Clark, J. M.
(2011). Barriers to and facilitators of postpartum follow-up care in women with recent gestational diabetes
mellitus: A qualitative study. Journal of Women’s Health,20, 239–245.
Blismas, N. G., & Dainty, A. R. J. (2003). Computer-aided qualitative data analysis: Panacea or paradox?
Building Research & Information,31, 455–463.
Boeckxstaens, P., Deregt, M., Vandesype, P., Willems, S., Brusselle, G., & De Sutter, A. (2012). Chronic
obstructive pulmonary disease and comorbidities through the eyes of the patient. Chronic Respiratory
Disease,9, 183–191.
Bourdon, S. (2002). The integration of qualitative data analysis software in research strategies: Resistances and
possibilities. FORUM: Qualitative Social Research,3, 2, Art. 11. Retrieved from http://www.qualitative-
research.net/index.php/fqs/article/view/850/1846/
Bringer, J. D., Johnston, L. H., & Brackenridge, C. H. (2006). Using computer-assisted qualitative data analysis
software to develop a grounded theory project. Field Methods,18, 245–266.
Brown, D. (2002). Going digital and staying qualitative: Some alternative strategies for digitizing the qua-
litative research process. FORUM: Qualitative Social Research,3, 2, Art. 12. Retrieved from http://
www.qualitative-research.net/fqs/fqs-eng.htm/
Bryman, A., & R. G. Burgess. (1994). Reflections on qualitative data analysis. In A. Bryman & R. G. Burgess
(Eds.), Analyzing qualitative data (pp. 216–226). London, England: Routledge.
Catterall, M., & Maclaren, P. (1998). Using computer software for the analysis of qualitative market research
data. Journal of the Market Research Society,40, 207–222.
Chirwa, E., Malata, A., & Norr, K. (2011). HIV prevention awareness and practices among married couples in
Malawi. Malawi Medical Journal,23, 32–37.
Cousins, K., & Mcintosh, W. (2005). More than typewriters, more than adding machines: Integrating informa-
tion technology into political research. Quality & Quantity,39, 581–614.
Curry, L., Taylor, L., Chen, P. G., & Bradley, E. (2012). Experiences of leadership in health care in sub-Saharan
Africa. Human Resources for Health,10, 33–41.
Dainty, A. R. J., Bagihole, B. M., & Neale, R. H. (1997). Analytical strategies for dealing with qualitative data
in construction management research. In P. Stephenson (Ed.), 13th annual association of researchers in con-
struction management conference (pp. 491–500). Cambridge, MA: King’s College.
Woods et al. 17
by guest on August 28, 2015ssc.sagepub.comDownloaded from
De Gregorio, E. (2011). Dynamics of a robbery: Criminological aspects, security issues and prevention—An
exploratory study. Police Practice and Research,12, 253–264.
Di Gregorio, S. (2000). Using Nvivo for your literature review. Paper presented at Strategies in Qualitative
Research: Issues and Results from Analysis Using QSR Nvivo and NUD*IST conference, Institute of
Education, London.
De Villiers, A., Koko-Mhlahlo, K., & Seneka, M. (2005). Nutritional well-being of young children in Duncan
Village, East London, South Africa: Accessibility of primary health care clinics. Public Health Nutrition,8,
520–532.
DeNardo, A., & Levers, L. L. (2002, June 14). Using Nvivo to analyze qualitative data. Monograph presented
at the Ethnographic and Qualitative Research in Education 2002 Annual Conference, Pittsburgh, PA.
Dohan, D., & Sanchez-Jankowski, M. (1998). Using computers to analyze ethnographic field data: Theoretical
and practical considerations. Annual Review of Sociology,24, 477–498.
Elsevier. (2014). Scopus: Content coverage guide. Retrieved from http://www.elsevier.com/__data/assets/pdf_
file/0019/148402/SC_Content-Coverage-Guide_July-2014.PDF
Evers, J. C. (2011). From the past into the future. How technological development change our ways of data col-
lection, transcription and analysis. Forum: Qualitative Social Research,12, 1, Article 38. Retrieved from
http://www.qualitative-research.net/index.php/fqs/article/view/1636/
Evers, J. C., Silver, C., Mruck, K., & Peeters, B. (2011). Introduction to the KWALON experiment: Discussions
on qualitative data analysis software by developers and users. FORUM: Qualitative Social Research,12,1,
Art.1. Retrieved from http://www.qualitative-research.net/fqs/fqs-eng.htm
Fielding, N. G., & Lee, R. M. (1998). Computer analysis and qualitative research. London, England: Sage.
Finfgeld-Connett, D. (2014). Use of content analysis to conduct knowledge-building and theory-generating
qualitative systematic reviews. Qualitative Research,14, 341–352.
Friese, S. (2014). Qualitative data analysis with ATLAS.ti
TM
. London, England: Sage.
Fritz, R. (1990). Computer analysis of qualitative data. In E. Lambert (Ed.), The collection and interpretation of
data from hidden populations (pp. 59–79). National Institute on Drug Abuse Research Monograph 98.
DHHS Pub. No. (ADM) 90-1678. Washington, DC: Supt. of Docs., U.S. Govt. Print Office.
Garcia-Horta, J. B., & Guerra-Ramos, M. T. (2009). The use of CAQDAS in educational research: Some advan-
tages, limitations and potential risks. International Journal of Research & Method in Education,32,
151–165.
Gibson, W., Callery, P., Campbell, M., Hall, A., & Richards, D. (2005). The digital revolution in qualitative
research: Working with digital audio and data through ATLAS.ti.Sociological Research Online, 10,
Retrieved from http://www.socresonline.org.uk/10/1/gibson.html doi:10.5153/sro.1044
Gilbert, L. S. (2002). Going the distance: ‘Closeness’ in qualitative data analysis software. International
Journal of Social Research Methodology,5, 215–228.
Gilliam, M. L. (2007). The role of parents and partners in the pregnancy behaviors of young Latinas. Hispanic
Journal of Behavioral Sciences,29, 50–67.
Granero-Molina, J., Ferna´ndez-Sola, C., Castro-Sa´nchez, A. M., Jime´nez-Lo´pez, F. R., Aguilera-Manrique, G.,
&Ma´rquez-Membrive, J. (2012). The clinical seminar as a learning methodology: An evaluation of nursing
students’ views. Acta Paulista de Enfermagem,25, 441–447.
Grime, J., Richardson, J. C., & Ong, B. N. (2010). Perceptions of joint pain and feeling well in older people who
reported being healthy: A qualitative study. British Journal of General Practice,60, 597–603.
Hannes, E. L., Janssens, D., & Wets, G. (2009). Does space matter? Travel mode scripts in daily activity travel.
Environment and Behavior,41, 75–100.
Hartel, J. (2010). Managing documents at home for serious leisure: A case study of the hobby of gourmet cook-
ing. Journal of Documentation,66, 847–874.
Hawthorne, K., Bennert, K., Lowes, L., Channon, S., Ronling, M., & Gregory, J. W. (2011). The experiences of
children and their parents in paediatric diabetes services should inform the development of communication
skills for healthcare staff (the DEPICTED Study). Diabetic Medicine,28, 1103–1108.
18 Social Science Computer Review
by guest on August 28, 2015ssc.sagepub.comDownloaded from
Hesse-Biber, S., Dupois, P., & Kinder, T. S. (1991). HyperRESEARCH: A computer program for the analysis
of qualitative data with an emphasis on hypothesis testing and multimedia analysis. Qualitative Sociology,
14, 289–306.
Holbrook, A., & Butcher, L. (1996). Software in educational research: The literature, the hard questions and
some specific research applications. Australian Educational Researcher,23, 55–80.
Holland, M. (2002). Computer-aided qualitative data analysis software. In C. Daymon & I. Holloway (Eds.),
Qualitative research methods in public relations and marketing communications (pp. 244–246). London,
England: Routledge.
Hurley, J. (2009). A qualitative study of mental health nurse identities: Many roles, one profession. Interna-
tional Journal of Mental Health Nursing,18, 383–390.
Hutchison, A. J., Johnston, L. H., & Breckon, J. D. (2010). Using QSR-Nvivo to facilitate the development of
a grounded theory project: An account of a worked example. International Journal of Social Research
Methodology,13, 283–302.
Jakobsen, C. H., & McLauglin, W. J. (2004). Communication in ecosystem management: A case study of cross-
disciplinary integration in the assessment phase of the interior Columbia basin ecosystem management
project. Environmental Management,33, 591–605.
Kennedy, H. P., Grant, J., Walton, C., & Sandall, J. (2013). Elective caesarean delivery: A mixed method qua-
litative investigation. Midwifery,29, 138–144.
Kirchhoff, A. C., Kuhlthau, K., Pajolek, H., Leisenring, W., Armstrong, G. T., Robison, L. L., & Park, E. R.
(2013). Employer-sponsored health insurance coverage limitations: Results from the childhood cancer
survivor study. Support Care Cancer,21, 377–383.
Laditka, S. B., Corwin, S. J., Laditka, J. N., Liu, R., Friedman, D. B., Mathews, A. E., & Wilcox, S. (2009).
Methods and management of the healthy brain study: A large multisite qualitative research project. The
Gerontologist,49, S18–S22.
Larkin, S. (2009). Socially mediated metacognition and learning to write. Thinking Skills and Creativity,4,
149–159.
MacMillan, K. (2005). More than just coding? Evaluating CAQDAS in a discourse analysis of news texts.
FORUM: Qualitative Social Research,6, 3, Art. 25. Retrieved from http://www.qualitative-research.net/
fqs/fqs-eng.htm
MacMillan, K., & Koenig, T. (2004). The wow factor: Preconceptions and expectations for data analysis soft-
ware in qualitative research. Social Science Computer Review,22, 179–186.
Mangabeira, W. C., Lee, R. M., & Fielding, N. G. (2004). Computers and qualitative research: Adoption, use,
and representation. Social Science Computer Review,22, 167–178.
Marshall, H. (2002). What do we do when we code data? Qualitative Research Journal,2, 56–70.
Morison, M., & Moir, J. (1998). The role of computer software in the analysis of qualitative data: Efficient
clerk, research assistant or Trojan horse? Journal of Advanced Nursing,28, 106–116.
Moore, A., Morris, J., Crouch, V., & Martin, M. (2003). Evaluation of physiotherapy clinical educational mod-
els: Comparing 1:1, 2:1 and 3:1 placements. Physiotherapy,89, 489–501.
Muhr, T. (1991). ATLAS/ti—A prototype for the support of text interpretation. Qualitative Sociology,14,
349–371.
Mutemwa, R., Mayhew, S., Colombini, M., Busza, J., Kivunaga, J., & Ndwiga, C. (2013). Experiences of health
care providers with integrated HIV and reproductive health services in Kenya: A qualitative study. BMC
Health Services Research,13, 18–28.
Nair, A., Malhotra, M. K., & Ahire, S. L. (2011). Toward a theory of managing context in six sigma
process-improvement projects: An action research investigation. Journal of Operations Management,
29, 529–548.
Newman, P. A., Bogo, M., & Daley, A. (2008). Self-disclosure of sexual orientation in social work field
education: Field instructor and lesbian and gay student perspectives. The Clinical Supervisor,27,
215–237.
Woods et al. 19
by guest on August 28, 2015ssc.sagepub.comDownloaded from
O’Halloran, K. (2011). Investigating argumentation in reading groups: Combining manual qualitative coding
and automated corpus analysis tools. Applied Linguistics,32, 172–196.
Orobia, L. A., Byabashaija, W., Munene, J. C., Sejjaaka, S. K., & Musinguzi, D. (2013). How do small business
owners manage working capital in an emerging economy? Qualitative Research in Accounting& Manage-
ment,10, 127–143.
Paulus, T., Lester, J., & Dempster, P. (2014). Digital tools for qualitative research.London,England:
Sage.
Paulus, T., Phipps, G., Harrison, J., & Varga, M. A. (2012). Re-envisioning instructional technology research in
higher education environments: A content analysis of a grant program. Journal of Computing in Higher
Education,24, 164–181.
Paulus, T., Woods, M., Atkins, D., & Macklin, R. The discourse of QDAS: Reporting practices of ATLAS.ti
and NVivo users with implications for best practices. International Journal of Social Research Methodology
(Under Review).
Pilling, S. A., Williams, M. B., Brackett, R. H., Gourley, R., Vander Weg, M. W., Christensen, A. J., Kaboli, P.
J., & Reisinger, H. S. (2010). Activating patients to engage their providers in the use of evidence-based med-
icine: A qualitative evaluation of the VA Project to Implement Diuretics (VAPID). Implementation Science,
5, 23–34.
Podolefsky, A. (1987). New tools for old jobs: Computers in the analysis of field notes. Anthropology Today,3,
14–16.
QSR International. (2014a). NVivo 10 feature list. Retrieved from http://download.qsrinternational.com/
Resource/NVivo10/nvivo10-feature-list.pdf/
QSR International. (2014b). Running a coding comparison query. Retrieved from http://webhelp-nv10.qsrinter-
national.com/nv10_help.htm#procedures/run_a_coding_comparison_query.htm/
Richards, L., & Richards, T. (1987). Qualitative data analysis: Can computers do it? Journal of Sociology,23,
23–35.
Richards, T., & Richards, L. (1991). The NUD*IST qualitative data analysis system. Qualitative Sociology,14,
307–324.
Se´ror, J. (2005). Computers and qualitative data analysis: Paper, pens, and highlighters vs. screen, mouse and
keyboard. TESOL Quarterly,39, 321–328.
Shelly, A., & Sibert, E. (1986). Using logic programming to facilitate qualitative data analysis. Qualitative
Sociology,9, 145–161.
Sin, C. H. (2007). Using software to open up the ‘Black Box’ of qualitative data analysis in evaluations: The
experience of a multi-site team using NUD*IST version 6. Evaluation,13, 110–120.
Szeinbach, S. L., Seoane-Vazquez, E., & Summers, K. H. (2012). Development of a men’s preference
for testosterone replacement therapy (P-TRT) instrument. Patient Preference and Adherence,6,
631–641.
Tallerico, M. (1991). Applications of qualitative analysis software: A view from the field. Qualitative Sociol-
ogy,14, 275–285.
Thongpriwan, V., & McElmurry, B. J. (2009). Thai female adolescents’ perceptions of dating violence. Health
Care for Women International,30, 871–891.
Waldrop, D. P. (2006). At the eleventh hour: Psychosocial dynamics in short hospice stays. The Gerontologist,
46, 106–114.
Wickham, M., Dunn, A., & Sweeney, S. (2012). Analysis of the leading tourism journals 1999–2008. Annals of
Tourism Research,39, 1714–1718.
Woods, D. K., & Dempster, P. G. (2011). Tales from the bleeding edge: The qualitative analysis of complex
video data using Transana. FORUM: Qualitative Social Research,12, Art. 17, Retrieved from http://
www.qualitative-research.net/fqs/fqs-eng.htm/
Woods, M., Macklin, R., & Lewis, G. (2015). Researcher reflexivity: Exploring the impacts of CAQDAS use.
International Journal of Social Research Methodology. doi:10.1080/13645579.2015.1023964
20 Social Science Computer Review
by guest on August 28, 2015ssc.sagepub.comDownloaded from
Author Biographies
Megan Woods is a lecturer in the Tasmanian School of Business and Economics, at the University of Tasma-
nia. She has over 10 years’ experience in teaching and researching the use of qualitative data analysis software
(QDAS) programs, which she has developed by researching learning processes of individuals, groups, organi-
zations, and communities; email: megan.woods@utas.edu.au
Trena Paulus is a professor in the Qualitative Research Program in the Department of Lifelong Education,
Administration and Policy at the University of Georgia. Her research areas include digital tools for qualitative
research and adapting discourse and conversation analysis methods for investigating computer-mediated com-
munication. She has previously written blog posts and a webinar about using Atlas.ti for the program’s devel-
opers as an expert user of their product but has no commercial relationship with them. email: tpaulus@uga.edu
David Atkins is an associate professor and department head for branch libraries at the University of Tennessee.
His research interests included international library cooperation and academic library service assessment;
email: david.atkins@utk.edu
Rob Macklin is a senior lecturer in the Tasmanian School of Business and Economics at the University of
Tasmania. His research and teaching passion is the relationship between ethics and human resource manage-
ment. He also has a passion for qualitative research and has published several papers on the demands of
professional practice in qualitative research; email: rob.macklin@utas.edu.au
Woods et al. 21
by guest on August 28, 2015ssc.sagepub.comDownloaded from
... The condition of tourism sector development governance based on agile governance is studied using qualitative research methods [26]. In addition, through notes during focus group discussions (FGD) and interviews with key informants, primary data were obtained on the condition of agile governance-based tourism sector development governance in Dayun District, Siak Regency. ...
Article
Full-text available
Historically, the government and local communities have long sought to develop the tourism sector so that sustainable development goals are actively pursued for the development of the tourism sector at the village and sub-district levels. This study examines the urgency of agile governance in developing the tourism sector in Dayun District, Siak Regency. This study was conducted using a qualitative descriptive method from relevant primary and secondary data sources, then analyzed using NVivo12 Plus software to answer the research questions. Based on the analysis results, this study concludes that agile governance is a crucial component that must be applied to sustainably develop the tourism sector in Dayun District, Siak Regency. Handling forest and land fire disasters can be combined with ecotourism activities in tourist village areas using the principles of agile governance, which include the development of regional information systems, the development of business-oriented community facilities, increasing community participation in tourism governance, and emphasizing the rapid resolution of problems that arise.
... This study employs a qualitative descriptive technique to analyze the phenomenon in a systematic manner [17]. The data in this study are divided into two types: primary data and secondary data. ...
... Furthermore, data analysis was carried out through content analysis using Nvivo 12 Plus software in order to explain the variables and concepts that influence the empirical phenomena of the study by visualizing data based on crucial combinations (Hai-Jew, 2020;Jackson, 2013;Silver and Woolf, 2018;Woods et al., 2015). Data analysis in this study was carried out in several crucial phases: First, the collected data (primary and secondary) were imported into the Nvivo 12 Plus application. ...
Article
Full-text available
This study aims to explain the design of policy strengthening in forest and land fire disaster mitigation governance, through the integration of ecotourism development in Siak Regency. Based on the research topic, this study employs a qualitative approach to describe governance conditions and the design of policy strengthening in ecotourism-based disaster mitigation governance. Data analysis is performed using Nvivo 12 Plus software. The results of this study indicate that forest and land fire disaster mitigation governance based on ecotourism development still has shortcomings that need to be addressed in the principles of conservation, economy, and community involvement. Then, the design of a policy to strengthen ecotourism-based disaster mitigation governance includes three crucial policy recommendations, namely: the need for special regulations related to forest and land fire disaster mitigation prevention based on the integration of ecotourism principle development, the need for a balance of roles between actors in determining and implementing ecotourism-based disaster mitigation policies, and the need for effective and efficient implementation of ecotourism-based disaster mitigation policies through increasing the involvement of strategic actors. Substantially, the handling of forest and land fire disasters in Siak Regency can be combined with ecotourism activities, especially in tourist village areas, by developing policies to strengthen the utilization of village-owned disaster mitigation facilities such as reservoirs, lakes, or ponds that are converted into water supplies during the dry season for forest and land fire disaster prevention activities and local economy-based tourist destinations. Our findings are a strategic effort to raise awareness among actors and highlight the need for policy-strengthening design in ecotourism-based disaster mitigation. These findings can also contribute to the literature that will be useful for all stakeholders in developing future long-term disaster mitigation governance policies. This study relies heavily on information from key informants, who represent only the perspectives and expertise of the stakeholders encountered. However, it still refers to important elements based on the informants’ knowledge capabilities in the disaster and tourism sectors. Therefore, we propose to conduct future studies on a comprehensive analysis of sustainable ecotourism-based disaster mitigation governance to promote and accelerate the idea of disaster and tourism in the future.
... ntumok vizsgálatára alkalmas. Kiemelkedő előnye az adatok struktúrákba szervezése. A program használata mellett azonban a fogalomépítésre és az elemzésre szánt idő nem igazán csökken (Almaiah-Al-Khasawneh-Althuniba, 2020). Alkalmazása az ezredfordulótól napjainkig egyre elterjedtebb. Az NVivo mellett gyakran használják még az ATLAS.ti szoftvert is (Woods et. al., 2015) Bernschütz és szerzőtársai kutatásukban összehasonlították a gépi és a szövegelemzést. Eredményeik azt mutatják, hogy a két út között nincs mélyebb eltérés. "Van, amit az emberi elme nem vesz észre, és van olyan, amelyet az emberi tudat (szakirodalom, előzmények) már egy eleve létező fogalomhoz kapcsolja" (Bernschütz-T. Nagy-Számadó, 20 ...
Article
Full-text available
A vasúti árufuvarozás folyamatosan versenyben áll a közúti árufuvarozással. Az összehasonlítás leggyakoribb kategóriái az ár, a szállítási gyakoriság és idő, a megbízhatóság és az ügyfélnek nyújtott szolgáltatás minősége. Kutatásunkban ezen jellemzők alapján azt kívánjuk feltárni, hogy a jövőben melyik fuvarozási mód fog inkább teret hódítani. Erre a kérdésre a szakemberekkel készített interjúk adatainak feldolgozásával próbáltunk választ adni, kiegészítve a meglévő kutatásokból származó információkkal. A közlekedési vállalatok hangsúlyozzák a költségek minimalizálásának és a folyamatok optimalizálásának fontosságát, és úgy vélik, hogy az intermodális szállítás hozzájárulhat a vasúti szállítás arányának növeléséhez. Véleményük szerint azonban állami támogatás nélkül nem lehet jelentős javulásra számítani. Kulcsszavak: vasúti árufuvarozás, közúti árufuvarozás, intermodális szállítás, állami támogatás Rail freight is in constant competition with road freight. The most common categories for comparison are price, frequency and time of delivery, reliability, and quality of service to the customer. Based on these characteristics, our research aims to find out which mode of transport will become more popular in the future. We have tried to answer this question by processing data from interviews with professionals, in addition to information from existing research. Transport companies stress the importance of minimising costs and optimising processes and believe that intermodal transport can contribute to increasing the share of rail transport. However, they believe that without public support, no significant improvement can be expected. Keywords: rail freight, road freight transport, intermodal transport, public support
... All but one (conducted by phone) were collected on site. Interviews were transcribed and thematically analysed using the program NVivo (Woods, Paulus, and Atkins 2016). Emerging themes from the interview data were initially guided by the focus questions (Silverman 2006). ...
Article
Full-text available
This paper examines the contributions that community gardens can make in achieving Target 7 of UN Sustainable Development Goal (SDG) 11, namely, to ‘provide universal access to safe, inclusive and accessible, green and public spaces’ (UN Citation2021). The paper focuses on a subset of green and public spaces, namely community gardens, to examine if their features enhance the capacity of public spaces to contribute to achieving target 11.7. In doing so, the paper applies a more expansive definition of ‘inclusive’ than that currently suggested by the target. The paper draws on findings from interviews with participants in several types of community gardens in Australia. The data indicates that community gardens contribute to solving the challenges of UNSDG 11.7, by improving the inclusivity and activation of public spaces through their design features, governance and community involvement. The development of additional target indicators that planners and policy makers can use to measure progress towards meeting the target is an area in need of further research.
Article
Demokrasi di tempat kerja merupakan prinsip yang dapat meningkatkan partisipasi, keadilan, dan keterlibatan pekerja dalam pengambilan keputusan organisasi. Penelitian ini bertujuan untuk mengeksplorasi dinamika demokrasi di tempat kerja melalui analisis diskursus berbasis data wawancara dari 23 responden. Pendekatan metodologi yang digunakan adalah Classroom Action Research (CAR) yang dikombinasikan dengan analisis diskursus menggunakan perangkat lunak NVivo 15 untuk menganalisis diskursus demokrasi di tempat kerja melalui 5 fokus analisis dengan indicator yaitu: (1) Partisipasi pegawai dalam berbagai kegiatan dan tugas di tempat kerja; (2) Kejelasan informasi di tempat kerja; (3) Saluran formal untuk menyampaikan kritik, saran, atau keluhan; (4) Kesamaan kesempatan bagi semua karyawan untuk berpartisipasi dalam pengambilan keputusan, tanpa diskriminasi; dan (5) Program pelatihan untuk membangun budaya kerja yang inklusif dan partisipatif. Hasil peneltian menunjukan: Demokrasi di tempat kerja sudah berjalan dengan baik pada indicator 1 dan 2, namun perlu adanya perbaikan dan peningkatan pada indikator 3,4, dan 5. Penelitian ini telah memberikan kontrubusi bagimana pengukuran demokratisasi melalui aspek inklusifitas dan partisipasi dalam organisasi di tempat bekerja.
Article
Full-text available
Purpose This research aims to contribute substantively and methodologically to our understanding of CSR communications in the pharmaceutical sector targeted to employees. Of specific concern is how companies can most effectively communicate their CSR activities to employees. Design/methodology/approach Multi-methods were used to identify CSR-related communication trends and relationships in the largest pharmaceutical companies by market cap. Replicatable data are ROA from Bloomberg, ESG scores, employee satisfaction from Indeed.com and content of CSR and similar reports. Findings Significant findings include the content of CSR reports related to employee behaviors and human resource processes. Both HR behaviors and HR processes are related to return on assets ROA. Multinational differences were found in Indeed satisfaction scores and in trends in communications from the CSR reports. Research limitations/implications One of the limitations was that not all pharmaceutical companies consistently reported their data. Practical implications Given that the pharmaceutical sector is comprised of a high proportion of uniquely qualified types of employees, this type of information can be used by prospective employees to consistently include more HR processes. Especially missing is recruitment and selection data that can reasonably be expected to facilitate identifying prospective employees who align with the CSR mission. Originality/value The methodology used in this study allows for replication in the pharmaceutical sector. Moreover, it encourages using similar disaggregated ESG data sources to study CSR in other sectors.
Article
Full-text available
We still know relatively little about how researchers use qualitative data analysis software (QDAS) such as ATLAS.ti and NVivo. We conducted a discourse analysis of 763 empirical articles published from 1994 to 2013 that explored the language used by researchers when reporting QDAS use. We found that most researchers provided few details of their QDAS use beyond naming the program, but the detailed accounts provided by some authors provided valuable insights into the ways that QDAS programs can be used to support data analysis and the reporting of research outcomes. We conclude with suggestions for best practices in reporting QDAS use. We encourage researchers to provide more detail about their program usage, e.g. by choosing active rather than passive voice to avoid attributing agency to the software, defining specialized QDAS terminology to prevent confusion, and avoiding unsubstantiated claims of a relationship between QDAS use and improved quality.
Article
Full-text available
Computer-assisted qualitative data analysis software (CAQDAS) programs are established tools for qualitative research. Making informed decisions when using them requires researchers to understand how they affect research practices and outcomes. In this article we consider the impact of CAQDAS on researcher reflexivity. Reviewing three decades of literature, we identify specific ‘reflexive moments’ experienced by CAQDAS users, the contexts in which they occur, the issues they raise, and the reflexive awareness they generate. The ways in which CAQDAS can enhance or undermine researcher reflexivity are also reported. By doing so, we aim to help researchers and especially research students (and their supervisors) understand the relationship between CAQDAS and reflexivity and the reflexive moments they may encounter when using such software.
Article
Full-text available
Computer assisted qualitative data ana­lysis software (CAQDAS) is frequently described as a tool that can be used for "qualitative research" in general, with qualitative analysis treated as a "catch-all" homogeneous category. Few studies have detailed its use within specific methods, and even fewer have appraised its value for discourse analysis (DA). While some briefly comment that CAQDAS has technical limitations for discourse analysis, in general, the topic as a whole is given scant attention. Our aim is to investigate whether this limited interest in CAQDAS as a qualitative tool amongst discourse analysts, and in DA as a research method amongst CAQDAS users, is prac­tically based; due to an uncertainty about research methods, including DA; or because of method­ol­ogical incompatibilities. In order to address these questions, this study is based not only on a review of the literature on CAQDAS and on DA, but also on our own experience as discourse analysts put­ting some of the main CAQDAS to the test in a media analysis of news texts. URN: urn:nbn:de:0114-fqs0503257
Article
Full-text available
This article makes a contribution to understanding informal argumentation by focusing on the discourse of reading groups. Reading groups, an important cultural phenomenon in Britain and other countries, regularly meet in members' houses, in pubs or restaurants, in bookshops, workplaces, schools or prisons to share their experiences of reading literature, usually fiction. Investigating argumentation in reading groups offers an opportunity for obtaining insight into how people debate with one another in self-organized, informal circumstances. The article reports on an empirical study which investigates the nature of argumentation used in evaluation and interpretation of novels read in a variety of groups. Ten groups were chosen on the basis of differences in age, gender composition, sexuality, setting (e.g. prison, school, university medical department) and geographical location in the UK. The article will investigate this focus through novel synergy between a manual qualitative coding software tool (Atlas-ti) and an automated quantitative tool of corpus linguistics (WMatrix), providing insight into patterns of co-occurrence between linguistic form and discoursal function which illuminate argumentation in reading group discourse.
Chapter
Qualitative data analysis, used by researchers to make sense of their data, comes in a variety of approaches which tend to be aligned with particular conceptual frameworks and methods. This chapter focusses on three approaches: thematic analysis and category coding, qualitative content analysis, and discourse analysis. With thematic analysis and category coding, which can be considered a foundational inductive approach, the researcher looks for connections within data, often collected in the field, and identifies thematic patterns. The approach is demonstrated with suggested steps that a researcher might follow. The emergence of qualitative content analysis from quantitative content analysis introduces the second approach, which focusses on a wide range of media, is inductive and deductive, highly systematic, and starts with an initial framework. Steps for undertaking this type of analysis are also included. The last approach, discourse analysis, is also used for a wide range of media. Its focus is on exploring the underlying meaning of phenomena, including their social implications. Three types are discussed: linguistic, psychosocial and critical discourse analysis. The uses of discourse analysis in information studies are explored. A brief comparison of the three approaches concludes the chapter. © 2018 Kirsty Williamson, Lisa M. Given, and Paul Scifleet. Published by Elsevier Ltd. All rights reserved.
Article
Issues associated with the analysis of qualitative data receive much less attention in the market research literature than data collection issues. In the absence of defined analysis procedures and standards, qualitative data analysis is perceived as an idiosyncratic process, relying heavily on the personal biography and the philosophical stance of the analyst. One of the more recent developments in qualitative research is the increasing use of computers and software programs designed specifically to assist with data management and analysis. These programs operate on the assumption that there are commonalties in the various approaches to data analysis. There are few published reports on programs in the market research literature and the aim of this article is to examine the assumptions about data analysis on which these programs are based, review how programs can help with analysis and discuss some of the methodological issues associated with their use. Additionally, we suggest how programs can be used by qualitative market researchers.
Article
This paper discusses the journey of an information systems PhD research student using Nvivo for a literature review. In this paper Nvivo is proposed as a tool to help any researcher accomplish a rigorous and transparent literature review. Here a practical example of such a process is presented in seven steps, using a well-known qualitative research software that has in recent years moved from the margins to the mainstream.
Article
Findings from knowledge-building and theory-generating qualitative systematic reviews have the potential to help guide policy formation and practice in many disciplines. Unfortunately, this potential is currently hindered by the fact that rigorous data analysis methods have not been consistently used and/or articulated for purposes of conducting these types of reviews. Content analysis is a flexible data analysis method that can be used to conduct qualitative systematic reviews; however, its application in this context has not been fully explicated. Qualitative systematic reviewers who aim to build knowledge and generate theory are urged to adapt content analysis methods to accommodate data that are, by nature, highly organized and contextualized. In addition, they are encouraged to use reflective memoing and diagramming to ensure valid integration, interpretation, and synthesis of findings across studies. Finally, reviewers are advised to clearly and fully explain their data analysis methods in research reports.