A Content Analysis of Qualitative Research Published in Top Criminology and Criminal Justice Journals from 2010 to 2019

Abstract

With the growth of qualitative research within the fields of criminology and criminal justice (CCJ) it is important to examine discipline standards and expectations of how to collect and analyze qualitative data and to present research findings. Our aim here is to assess qualitative research published in 17 top CCJ journals during the period of 2010 to 2019. We found that the number of qualitative articles published in these years increased over the previous two decades; however, the relative percentage of all articles remained relatively stable. During this period, 11.3% of all articles in the 17 CCJ journals used qualitative methods. In addition, we provide general patterns related to methodology and to presentation of findings. The results give insights into discipline standards and expectations and points to substantive areas that are under-studied (e.g., victims) and to issues relating to methodological transparency.

Full text

A Content Analysis of Qualitative Research
Published in Top Criminology and Criminal Justice
Journals from 2010 to 2019
Heith Copes
1
&Blake Beaton
1
&David Ayeni
2
&Dean Dabney
2
&
Richard Tewksbury
3
Received: 7 February 2020 /Accepted: 3 June 2020/
#Southern Criminal Justice Association 2020
Abstract
With the growth of qualitative research within the fields of criminology and criminal
justice (CCJ) it is important to examine discipline standards and expectations of how to
collect and analyze qualitative data and to present research findings. Our aim here is to
assess qualitative research published in 17 top CCJ journals during the period of 2010
to 2019. We found that the number of qualitative articles published in these years
increased over the previous two decades; however, the relative percentage of all articles
remained relatively stable. During this period, 11.3% of all articles in the 17 CCJ
journals used qualitative methods. In addition, we provide general patterns related to
methodology and to presentation of findings. The results give insights into discipline
standards and expectations and points to substantive areas that are under-studied (e.g.,
victims) and to issues relating to methodological transparency.
Keywords Qualitative research .Ethnographic research .Content analysis
Despite the legacy of ethnographic methods in criminology, qualitative research in the
discipline has accounted for only a small proportion of published research in the last
several decades (Buckler 2008; Copes et al. 2011; Kleck et al. 2006; Tewksbury et al.
2010; Tewksbury et al. 2005; Woodward et al. 2016). The percent of qualitative
research published in criminology and criminal justice (CCJ) journals since 2000 has
ranged from 3.7% (Copes et al. 2011)to9.8%(Buckler2008), depending on how
American Journal of Criminal Justice
https://doi.org/10.1007/s12103-020-09540-6
*Heith Copes
Jhcopes@uab.edu
1
University of Alabama at Birmingham, 1201 University Boulevard; Suite 210, Birmingham,
AL 35040, USA
2
Georgia State University, Atlanta, GA 30302, USA
3
Louisville, USA

qualitative methods is defined, time frames for publishing, and the range of journals
examined. Explanations for this finding vary, but typically point to structural reasons
that emphasize quantitative research for grant funding, assumptions about scientific
rigor, and issues of exposure and training for emerging scholars, rather than systematic
bias on the part of editors (Armstrong 2020;Buckler2008).
Nevertheless, it appears that CCJ may be in the midst of a growth in qualitative
methods. There is now a journal specializing in quantitative research (Journal of
Qualitative Criminal Justice and Criminology) and several handbooks devoted to the
topic have been published (e.g., Boeri and Shukla 2019; Copes and Miller 2015;Miller
and Palacios 2015; Rice and Maltz 2018). In part this is due to the evolving nature of
questions that criminologists ask and new pushes for categorization, interpretation, and
application of ideas and theoretical concepts. New lines of research interest that
prioritize qualitative methods are gaining in popularity. We are seeing more research
centered on queer criminology, cultural criminology, narrative criminology, visual
criminology, and convict criminology—all these areas place a high value on interpre-
tation and understanding, the cornerstones of qualitative methods.
Finally, there has been considerable growth in the number of doctoral programs in
criminology and criminal justice. In 2006, the Association of Doctoral Programs in
Criminology and Criminal Justice (ADPCCJ) identified 26 doctoral programs in the
field (Clear 2006). The 2019 report contained information on 39 doctoral programs
(Lane et al. 2019). More programs mean more researchers and more research on crime
and justice topics. It also means increased systematic training on how to conduct said
research. In 2006, only 4 of 25 CCJ PhD programs required a course on qualitative
research methods and another 10 offered elective seminars on the topic (Buckler 2008).
Our own 2020 cross-referencing of ADPCCJ reports and online resources identified 47
doctoral programs that self-identify as criminology and criminal justice. Two thirds
(N= 31) of these curricula include a required (N= 10) or elective (N= 21) course
specific to qualitative research methods. With more programs exposing fledgling
researchers to the maxims of qualitative research, one can reasonably expect to see
an increase in the amount of qualitative research being conducted and published.
If in fact we are on the cusp of a qualitative revival, driven by a growth in the
number of fledgling criminologists training in qualitative methods or increases in the
number of published articles in the field, then it is important to assess the current state
of qualitative CCJ. Doing so can aid those engaging in this style of research when they
develop research designs, collect data, analyze data, and write up their results. Accord-
ingly, our aim is to provide an assessment of the qualitative research
1
published in the
top CCJ journals from 2010 to 2019. By updating research that examined similar
themes for the first decade of the century (Copes et al. 2011) we seek to determine the
relative frequency of qualitative research in CCJ and to discuss the methodological
content of these articles. Doing so will provide insights into current conventions related
to methodology and style and point to strengths and limitations of the method and its
application. In this way we seek to provide an overview of the current state of the field,
focusing on the ways and means by which knowledge is produced and perpetuated.
1
For the current analysis we include only articles that used interviews or observations with participants. We
exclude content analysis of existing documents (including online forums) and qualitative responses to
questionnaires.
American Journal of Criminal Justice

Methods
The study sample consists of qualitative, interview-based articles published in the most
prestigious American and European CCJ journals from 2010 to 2019. The list of
journals was based on the study by Copes et al. (2011) with the addition of the two
highest ranked non-U.S. English language journals (i.e., British Journal of Criminology
and European Journal of Criminology). The initial list of journals was selected based
on CCJ journal prestige rankings provided by Sorensen et al. (2006) and Sorenson
(2009). Journals that were listed in the top 15 of either ranking were included in the
original sample. We also consulted DeJong and St. George’s(2018)surveyof543
members of the American Society of Criminology and Academy of Criminal Justice
Sciences that sought to establish subjective rankings of journal prestige. In total there
were 18 journals ranked in the top 15 of these lists with 11 appearing on all three lists.
We excluded three journals from the list that we used to form our study sample. We
excluded Journal of Quantitative Criminology because of its specified methodological
orientation, Advances in Criminological Theory because it was determined to be an
edited collection rather than a peer reviewed academic journal, and Law and Society
Review because its focus was primarily on law rather than criminology or criminal
justice. Thus, the 17 journals included in the current analysis are British Journal of
Criminology (BJC),Crime and Delinquency (C&D), Criminal Justice and Behavior
(CJB), Criminology (CRIM), Criminology and Public Policy (CPP),
2
Deviant Behavior
(DB),
3
European Journal of Criminology (EJC), International Journal of Offender
Therapy and Comparative Criminology (IJOTCC), Journal of Criminal Justice (JCJ),
Journal of Criminal Law and Criminology (JCLC),
4
Journal of Interpersonal Violence
(JIV), Journal of Research in Crime and Delinquency (JRCD), Justice Quarterly (JQ),
Law and Human Behavior (LHB), Police Quarterly (PQ), Prison Journal (PJ), and
Theoretical Criminology (TC).
We concentrated our inquiry on a ten-year period, collecting all qualitative
interview-based research published in volumes from 2010 through 2019.
5
To be
consistent with previous efforts (Buckler 2008; Copes et al. 2011; Kleck et al. 2006;
Tewksbury et al. 2010), our focus was on articles that used qualitative data collection.
To be eligible for inclusion an article’s author(s) must have analyzed text-based data
collected through fieldwork, semi-structured or unstructured interviews, focus groups,
or some combination of these techniques. We included those studies that had a
quantitative component (e.g., survey data) if the paper also included qualitative data.
We excluded studies that relied on open-ended questions attached to a survey and those
that relied solely on content analysis of existing documents (e.g., court records, police
files, and internet forums) with no interactions with participants. We also excluded
2
Only articles listed as “Research Articles”were included in the total number count. We excluded those listed
as Editorials and Policy Responses.
3
We included only those articles that explicitly addressed crime, victimization, and criminal justice processes
or actors indetermining the overall article count and total qualitative articles. We excluded articles that focused
solely on non-criminal or non-criminal justice topics (e.g., sexual deviance, student cheating, etc.).
4
We only included articles that appeared in the Criminology, and not the Criminal Law, Comments, or
Symposia sections.
5
We included articles published in general issues as well as special or thematic issues, some of which
involved a guest editor. None of the special issues revolved around themes that precluded or prescribed articles
based on qualitative data collection.
American Journal of Criminal Justice

research that relied solely on autoethnographic data. Using these selection criteria, only
those articles that all authors agreed were qualitative in orientation were coded as such.
Additionally, only research driven articles were included in the data under consider-
ation here; that is, editorials, book reviews, replies, commentaries, and Presidential
Addresses were excluded from the analysis.
These 17 journals published 8522 total articles within their 2010–2019 volumes, and
we defined 965 (11.3%) of these as qualitative. The total number of articles and total
number of qualitative articles has increased considerably since the first decade of the
2000s, nearly doubling the overall article counts and quadrupling the qualitative article
tallies reported by Copes et al. (2011), Tewksbury et al. (2010), and Tewksbury et al.
(2005). Because of the high number of qualitative articles, we chose to sample from
these 965 articles to allow for a more manageable analysis. We included all articles in
the top five journals (n= 83) because we believe these articles may have a stronger
impact on how others conduct and present their qualitative data than articles in lower
tier journals.
6
For the remaining 12 journals, we sampled 40% of the qualitative articles
(n= 355) using a stratified design.
7
That is, we sample 40% of qualitative articles from
each of these 12 journals. Thus, we include 438 articles in the final analysis sample. We
believe that the relatively high sample rate coupled with the inclusion of all qualitative
articles in the top five journals presents an accurate representation of the qualitative
articles published in CCJ journals during the period.
We used a code sheet similar to the one used by Copes et al. (2011) to systematically
gather data. We pilot tested the code sheet as a means of assessing its face validity and
to ensure inter-rater reliability. Three authors began by coding ten articles indepen-
dently and then comparing findings. For the few elements we coded differently we
discussed why they were different and aligned our coding for future articles. Addition-
ally, based on this pre-test we adjusted the coding scheme to reflect emerging issues
with coding. We then recoded these 10 articles and a new set of 10 articles to further
ensure consistency in coding.
Measures
We coded the qualitative articles for variables relating to methodological issues. The
first set of codes focused on methodological issues including sample size,
8
age of the
data (time from initial data collection to publication), use of visual methods, coding
strategies, inter-coder reliability, use of software for coding/analysis, and follow-ups
with participants. Other codes relating to methodology included whether the authors
reported that they recorded the interviews, whether and how much they paid partici-
pants, and whether the research was funded. Based on the article authors’descriptions,
we coded each article for whether the authors mentioned using an audio recorder.
9
For
6
We make no claim as to whether these articles are “better”or more deserving of this attention. We simply
state that because of their high profile they may influence authors more than articles in other journals.
7
This sampling threshold was chosen to allow for a robust yet manageable number of articles to be included
in the analysis.
8
When determining sample size for focus groups we included the total number of participants who were
included in all focus groups.
9
We only coded an article as not recorded if the authors specifically mentioned that they chose not to record
interviews
American Journal of Criminal Justice

payment of interviewees, we noted whether payments were used and if so, the amount
of the payment. We also coded for whether authors discussed issues of positionality,
which includes the authors’social position or identity being described in relation to
their participants and the potential impact of these differences or similarities on the
findings. Finally, we coded whether the articles discussed policy implications of the
findings.
We coded the primary method authors used to collect data (e.g., semi-structured
interviews, unstructured interviews, focus groups, field research, or quantitative
methods). For each article we coded for up to two types of data collection and classified
them based on the primacy of the method. For example, if the author(s) conducted field
research that was supplemented by 15 semi-structured interviews, we then coded field
work as the primary method and interviews as the secondary method. We also coded
whether the author used mixed methods.
Next, we sought to determine the departments in which authors of qualitative
research were housed. To do this we collected the departments listed at time of
publication for the first author of each article. Because of the diversity of departments
we classified them into seven groups: Criminology and Criminal Justice (CCJ),
Humanities (e.g., History, English, and Music), Law, Health Sciences (e.g., Health
and Life Sciences, Health Research, Nursing, and Surgery), Policy (Public Policy,
Social Policy, Drug Policy, Public Administration, and Public Affairs), Psychology,
Social and Behavioral Sciences (Anthropology, Social Sciences, Economics, Gender
Studies, Ethnic Studies, and Behavioral Sciences), Sociology, Social Work, and Other
(Architecture, Computer Sciences, Management Services, and Independent
Researcher).
There is a common belief that ethnographic research is difficult to get funded by
state and federal agencies. Thus, we coded for whether the authors acknowledged some
type of funding. Funding sources were grouped into three categories: (1) unfunded, (2)
university, state, or private funding (e.g., foundation grants and state department of
corrections grants), or (3) federal funding (e.g., National Institute of Justice, National
Institute of Health, National Science Foundation).
We were also interested in the types of study participants, as one of the key
components of any research is selecting appropriate participants. We coded each article
to determine the type of participant studied. This included basic demographics, includ-
ing gender and age, noting if authors did not mention any of this demographic
information. We also included whether the sample included LGBTQ+ populations as
a general demographic characteristic or as the focus of the study. Additionally, we
coded for the status of participants (offenders, professionals, citizens, victims, or
mixed). For “offender”participants, we also coded for their criminal justice status at
the time they were solicited (active, probation/parole, half-way house/treatment center,
or prison/jail) and the type of crime they committed (property, violent, sex crimes,
white collar, drugs, gang related crimes, or non-specific). When the participants were
victims, we determined the type of crimes for which they were victims. These crimes
included property (e.g., burglary, motor vehicle theft, shoplifting), violent (e.g., rob-
bery, assault, domestic violence), sex, or white-collar crimes. For the participants who
were classified as professionals, we coded to determine if they were police, judges,
attorneys, correctional officers/authorities, others (e.g., civil servants, victim rights
advocates, and forensic lab workers), or some combination of the previous.
American Journal of Criminal Justice

After completion of our direct coding from articles’content, we conducted two
additional analyses. Namely, we grouped articles based on (1) the prestige ranking of
the journal in which they appeared and (2) the article’s total number of citations divided
by number of years in print. To determine the prestige level of the journal, we classified
all journals into two broad categories: top tier and other. Top tier articles included the
top five U.S. based journals according to the prestige rankings from DeJong and St.
George (2018), which included Criminology,Justice Quarterly,Journal of Research in
Crime and Delinquency,Criminology and Public Policy,andCrime and Delinquen-
cy.
10
While Journal of Quantitative Criminology was ranked fourth, we excluded it
from the list because of its methodological orientation.
We also sought to determine if there are differences between those articles that are
highly cited articles and those that are not. To determine the number of citations the
articles received we relied on Google Scholar. We determined the number of citations
that each article received as of November 20, 2019. We then divided the number of
citations by the number of years the article had been in print to create the average
number of cites per year. The mean number of cites per year was 4.36 for the 438
articles comprising the analysis sample. Following Copes et al. (2011), we then
identified the articles falling in the top two deciles in terms of their number of citations
per year (mean of 9.2 cites per year) and deemed them as comprising a subsample of
“high impact”articles. For analysis purposes, the characteristics of these 87 high impact
articles were compared to the remaining 351 articles in our sample that comprised the
bottom eight deciles of the analysis sample.
Findings
The relative representation of qualitative research in the top CCJ journals is low,
especially in U.S. journals. As Table 1shows, of the 8522 research articles appearing
in the 2010–2019 volumes of the 17 journals under study, only 965 (11.3%) relied on
interview-based research. This is a meaningful increase over the levels observed during
the previous decade where this type of research constituted 3.7% of articles (Copes
et al. 2011). The journals with the highest percentage of articles using these methods
were BJC (37.2%), TC (24.1%), and PJ (21.4%). The journals with the lowest
percentage of qualitative articles (all under 2.5%) were C&D, CJB, JCJ, JCLC, and
LHB.JCLC published no qualitative based research articles in the period.
In aggregate, 5.3% (83 of 1657 articles) of the articles published in the top five
journals 2010–2019 relied on a qualitative methodology, which is considerably lower
than the 12.8% (882 of 6865 total articles) observed among the other 12 journals
comprising the study sample. CRIM had the highest percentage among the top five
journals (10.4%), while C&D had the lowest (1.6%). Interestingly, in aggregate, the
three European journals (i.e., BJC,EJC,andTC) published a much higher percentage
of qualitative articles (27.5%) than did the 14 U.S. journals (8.6%) in the sample. The
U.S. journals with the highest percent of qualitative articles were PJ (21.4%), DB
(17.8%), and IJOTCC (15.1%).
10
We recognize that including these five as top tier is an artificial ranking and that others (i.e., criminologists
in non-US countries) likely have different assessments of top tier journals.
American Journal of Criminal Justice

A supplemental longitudinal analysis revealed that the percentage of articles using
qualitative methods slowly increased among the 17 CCJ journals over the most recent
decade. Annual averages ranged from a low of 8.3% in 2011 to a high of 14.8% in
2019. This trend was most notable in CRIM, where nearly one-third of all qualitative
articles appearing in the journal appeared in 2019 compared to none in 2010. This trend
of increased percentage of qualitative articles was similar for the European journals as
well as for the top tier journals, where 2019 had the largest percent of qualitative
articles.
Characteristics of Qualitative Articles
Tables 2through 7as well as the text that follows report findings on all qualitative
articles published in the top five journals (N= 83) plus the 40% random sample (N=
355) that we drew from the full complement of qualitative articles published in the
remaining 12 journals considered in this analysis. Qualitative researchers can choose
from a variety of methodologies to collect their data. Table 2presents frequencies of the
major styles of data collection for qualitative articles. Interviews, both unstructured and
semi-structured, were the primary style of data collection: 80.1% of all of our sampled
articles relied on interviews. The remainder relied on fieldwork (11%), focus groups
(5.9%), and quantitative analysis of qualitatively collected data (3%).
Table 3includes descriptive statistics (raw numbers or means, with percentages
appearing in parentheses) on methodological issues relating to the analysis sample.
Table 1 2010–2019 Journals’Inclusion of Qualitative Articles (N=8522)
Journal Total Articles Qualitative Percent
British Journal of Criminology 610 227 37.2%
Crime and Delinquency 549 9 1.6%
Criminal Justice & Behavior 793 19 2.4%
Criminology 279 29 10.4%
Criminology & Public Policy 147 6 4.1%
Deviant Behavior 540 96 17.8%
European Journal of Criminology 361 48 13.3%
International Journal of Offender Therapy and
Comparative Criminology
1001 151 15.1%
Journal of Criminal Justice 649 9 1.4%
Journal of Criminal Law and Criminology 43 0 0.0%
Journal of Interpersonal Violence 1651 190 11.5%
Journal of Research in Crime and Delinquency 265 9 3.4%
Justice Quarterly 417 30 7.3%
Law & Human Behavior 478 4 0.8%
Police Quarterly 190 14 7.4%
Prison Journal 304 65 21.4%
Theoretical Criminology 245 59 24.1%
Total 8522 965 11.3%
American Journal of Criminal Justice

Factors considered in this regard include sample size, payment amount, recording of
interviews, use of mixed methods and visual methods, and the age of the data when
published. The mean sample size for the 438 articles (see far left panel of Table 3)was
Table 2 Style of Data Collection for Ethnographic Articles
Overall Analysis Sample
(N = 438)
Journal Tier Citation Score
Top
(N = 83)
Other
(N = 355)
Top
(N = 87)
Other
(N = 351)
Primary Style
Interviews 351 (80.1) 68 (81.9) 283 (79.7) 77 (88.5) 274 (78.1)
Focus groups 26 (5.9) 3 (3.6) 23 (6.5) 3 (3.4) 23 (6.6)
Fieldwork 48 (11.0) 9 (10.8) 39 (11.0) 6 (6.9) 42 (12.0)
Quantitative 13 (3.0) 3 (3.6) 10 (2.8) 1 (1.1) 12 (3.4)
Combined Style
Interviews 391 (71.7) 77 (70.6) 314 (72.0) 83 (74.1) 308 (71.1)
Focus groups 39 (7.2) 5 (4.6) 34 (7.8) 9 (8.0) 30 (6.9)
Fieldwork 77 (14.1) 13 (11.9) 64 (14.7) 14 (12.5) 63 (14.5)
Quantitative 38 (7.0) 14 (12.8) 24 (5.5) 6 (5.4) 32 (7.4)
Percentages may not sum to 100 due to rounding
Table 3 Methodological Characteristics of 2010–2019 Qualitative Articles
Overall Analysis Sample
(N = 438)
Journal Tier Citation Score
Top
(N = 83)
Other
(N = 355)
Top
(N = 87)
Other
(N = 351)
Mean Sample Size 52.6 68.4 37.8 61.7 43.5
Mean Payments $31.57 $35.22 $29.88 $37.68 $30.25
Mean Age of Data 5.7 7.3 5.2 5.2 5.8
Mention Age of Data
No 194 (44.3) 25 (30.1) 169 (47.6) 39 (44.8) 155 (44.2)
Yes 244 (55.7) 58 (69.9) 186 (52.4) 48 (55.2) 196 (55.8)
Recorded
No 15 (3.4) 8 (9.6) 7 (2.0) 6 (6.9) 9 (2.6)
Yes 209 (47.7) 41 (49.4) 168 (47.3) 43 (49.4) 166 (47.3)
Not Mentioned 214 (48.9) 34 (41.0) 180 (50.7) 38 (43.7) 176 (50.1)
Mixed Methods
No 401 (91.6) 68 (81.9) 333 (93.8) 81 (93.1) 320 (91.2)
Yes 37 (8.4) 15 (18.1) 22 (6.2) 6 (6.9) 31 (8.8)
Visual Methods
No 416 (95.0) 81 (97.6) 335 (94.4) 83 (95.4) 333 (94.9)
Yes 22 (5.0) 2 (2.4) 20 (5.6) 4 (4.6) 18 (5.1)
Percentages may not sum to 100 due to rounding
American Journal of Criminal Justice

52.6, with a range of 2 to 351. Referring to the middle panel of data, we note that
articles published in the top five journals had larger sample sizes (mean 68.4) than those
studies in the other journals (mean 37.8). Additionally, referring to the far-right panel of
the table, we note that articles with high citation scores were associated with larger
sample sizes (mean 61.7) than others (mean 43.5).
Data saturation is an important consideration underlying many qualitative research
designs. Typically, researchers do not embark upon data collection with a preset sample
size, but rather monitor the trends observed in the data as a means of determining when to
cease participant recruitment. Data saturation occurs when no meaningful themes (or
subthemes) in the data begin to emerge. Datasaturationtendstobeamorepressing
consideration in studies built around semi-structured interviews (i.e., repeated use of set
questions) than it is in a more loosely organized fieldwork venture. With this in mind, we
conducted a supplemental calculation of sample sizes for those studies involving semi-
structured interviews. The mean sample size for these studies was 48. This gives some
indication of standard sample sizes, and presumably, expectations of ensuring saturation.
Those who engage in qualitative research have the option to compensate participants
with financial remuneration. However, more than three quarters (76.3%) of the articles
in the overall sample did not directly mention paying participants. We found that it was
more common for articles in top tier journals to mention paying participants than
articles in the other 12 journals (38.6% vs. 19.7%). There was no meaningful difference
in the use of payments for articles with high citation counts compared to the other
articles. For those who mentioned paying participants, the data in Table 3show that the
monetary rage was from $2.25 to $150. The overall mean amount paid to participants
in the full analysis sample was $31.57. Articles in top tier journals had a slightly higher
mean payment ($35.22) than other articles ($29.88). A closer examination of these data
revealed that researchers tend to pay known victims and offenders more often than they
do ordinary citizens or professionals.
One important decision all qualitative researchers must make is whether they will
record interviews with participants. Whereas some prefer to record to have accurate
representations of the interview, others believe a recording device can stifle conversa-
tion. Referring to Table 3, our analysis revealed that nearly half of the articles did not
mention audio recording devices (48.9%), 47.7% recorded the interview, and only 3.4%
specifically stated that they did not record. This trend was stable regardless of the journal
rank or the number of citations the article received. Of note, but not displayed in the
table, all 23 of the articles that relied on focus group interviews recorded the interviews.
We identified 77 articles that relied on a field work design for data collection. Only four
(5.2%) of these studies did not record the interviews, whereas 28 (36.4%) said they did
and 45 (58.4%) provide no specifics on the recording protocol.
Referring again to Table 3, the data suggest that mixed and visual methods in CCJ
research are still relatively infrequent. Only 8.4% of studies used a mixed methods
approach. Articles in top tier journals are substantially more likely to use mixed
methods (18.1%) than articles appearing in other journals (6.2%). No meaningful
differences were observed between the highly cited articles and those recording lower
citation counts. Research that uses visual methods is rare—only 5.0% of the articles
used some form of visual method. Table 3also shows that articles published in the top
journals or yielding high citation counts were no more or less likely to use visual
methods than were other journal articles.
American Journal of Criminal Justice

Qualitative research, especially field work, can take a long time to complete. This is
important, as the age of data can have an impact on our understanding of topics. Yet,
our analysis revealed that only 55.7% of articles made a direct mention of the age of the
data (Table 3). Articles in top tier journals are relatively more likely to mention data age
(69.9%) relative to articles published in other journals (52.4%). When age of the data is
mentioned, the age from time of first collected to when published ranged from 1 to
23 years, with a mean of 5.7 years. Articles’data in highly ranked journals were older
on average (mean 7.3) than data in other journals (5.2) and few differences were seen
among highly cited and lesser cited articles. Focus group research had the youngest
data (slightly over 4 years old on average) and field research had the oldest (about six
and half years old on average).
Table 4provides further information about the characteristics of the articles includ-
ing the mention of funding, the number of authors, discussion of policy implications,
and the department type of each article’s lead author. Only a slight majority of articles
comprising the full analysis sample were supported by funding (54.6%). Journal
ranking and article citation counts appeared to have no relationship to the existence
of funding support. Of those that mentioned funding, the majority of articles in the
overall analysis sample were funded through local, state, or private organizations
Table 4 Article Characteristics of 2010–2019 Qualitative Articles
Overall Analysis Sample
(N = 438)
Journal Tier Article Citation Score
Top
(N = 83)
Other
(N = 355)
High
(N = 87)
Other
(N = 351)
Mention Funding
No 199 (45.4) 40 (48.2) 159 (44.8) 35 (40.2) 164 (46.7)
Local, state or private 140 (32.0) 26 (31.3) 114 (32.1) 30 (34.5) 110 (31.3)
Federal 99 (22.6) 17 (20.5) 82 (23.1) 22 (25.3) 77 (21.9)
Mean Number Authors 2.4 2.5 2.5 2.5 2.3
Discuss Policy
No 403 (92.0) 67 (80.7) 336 (94.6) 80 (92.0) 323 (92.0)
Yes 35 (8.0) 16 (19.3) 19 (5.4) 7 (8.0) 28 (8.0)
Department of lead author
CCJ 205 (46.8) 63 (75.9) 142 (40.0) 48 (55.2) 157 (44.7)
Soc 69 (15.8) 17 (20.5) 52 (14.6) 14 (16.1) 55 (15.7)
SBS 42 (9.6) 0 (0.0) 42 (11.8) 5 (5.7) 37 (10.5)
Medical 29 (6.6) 0 (0.0) 29 (8.2) 8 (9.2) 21 (6.0)
Psych 27 (6.1) 1 (1.2) 26 (7.3) 2 (2.3) 25 (7.1)
Law 21 (4.8) 0 (0.0) 21 (5.9) 2 (2.3) 19 (5.4)
Social Work 19 (4.3) 0 (0.0) 19 (5.4) 3 (3.4) 16 (4.6)
Policy 12 (2.7) 2 (2.4) 10 (2.8) 4 (4.6) 8 (2.3)
Humanities 7(1.6) 0(0.0) 7(2.0) 0(0.0) 7(2.0)
Other 7 (1.6) 0 (0.0) 7 (2.0) 1 (1.1) 6 (1.7)
Percentages may not sum to 100 due to rounding
American Journal of Criminal Justice

(58.6%), while articles that received their funding through federal grants made up the
remaining 41.4%. These frequencies were similar when comparing articles that men-
tioned funding based on top ranked journal articles and top cited articles.
Co-authorship has been identified as a growing trend in CCJ articles (Roche et al.
2019; Tewksbury and Mustaine 2011;Woodwardetal.2016). We found that the mean
number of authors for the qualitative articles comprising our full analysis sample was
2.4 and the median was 2. In addition, 70.5% of articles included in our sample
included co-authored studies. Supplemental analysis revealed that articles using field
research had the lowest mean number of authors and those that incorporated quantita-
tive methods had the highest. This is consistent with the often solitary nature of
ethnographic research. Yet, for those articles based on other forms of qualitative data,
a team approach to the analysis and reporting is the norm.
In some circles there exists a general sentiment that qualitative researchers have little
interest in extrapolating policy implications from their work. This assumption appears
to stem from the idea that qualitative research is more likely to be theoretical,
exploratory, or descriptive in nature, rather than evaluative. The data in Table 4appear
to substantiate this claim, as only 8.0% of the articles in the overall analysis sample
offered up direct policy implications. Articles published in top ranking journals were
found to be somewhat more likely to include discussions of policy than articles in the
other journals. Namely, for top tier journal articles, 19.3% included policy implications,
while only 5.4% of articles in non-top tier journals included such discussions. Inter-
estingly, discussion of policy between the top cited articles and the remaining articles
held the same frequencies, with only 8% of each category including such content.
Table 4also provides details on scholarly background of the persons who authored
the qualitative articles comprising our samples. The majority of qualitative research in
CCJ journals is published by lead authors housed in Criminology or Criminal Justice
departments. Beginning with the overall analysis sample of 438 qualitative articles,
nearly half (46.8%) of the lead authors listed CCJ as a home academic department.
Sociology (15.8%) and Social and Behavioral Sciences (SBS) (9.6%) were the next
most prominent categories in this regard. Articles appearing in top tier journals were
almost entirely written by faculty in CCJ (75.9%) or Sociology departments (20.5%).
Articles in non-top tier journals came from a more diverse background than those in top
tier journals, including Health Sciences (8.2%), Psychology (7.3%), Law (5.9%), and
Social Work (5.4%), which is likely due to these journals outlining subject foci that are
more aligned with public health or psychology approaches (e.g., JIV and IJOTCC).
Coding of Qualitative Data
We examined the methods section of each article in our sample to learn more about
their underlying research protocols. In particular, we gathered information pertaining to
their discussions of coding, inter-coder/interrater reliability, saturation, use of grounded
theory, coding styles, follow-up with participants, software use, and discussions of
positionality. Summary findings of this analysis are presented in Table 5. Of the total
articles sampled, 48.8% discussed how the authors coded the data that were collected
while 51.1% omitted such text. We partitioned the articles that discussed coding into
two categories of discussion. The first category being “little detail,”which included
only cursory attention to coding (often with just a sentence or two devoted to the topic).
American Journal of Criminal Justice

We classified articles as “detailed”coding when they contained detailed information about
coding techniques, including entire paragraphs or sections dedicated to the discussion of the
topic. Most of the articles sampled that discussed coding were classified as detailed in nature.
The discussion of coding efforts presented in the sample of top cited articles very much
paralleled the trends observed in the overall analysis sample. However, articles in top ranked
journals were far more likely to discuss coding strategies than those in other journals. Three
quarters (75.9%) of the articles in the top journals mentioned coding strategies, with 71.4%
of those articles containing detailed notes about the authors’coding strategies. A more
granular analysis revealed that certain data collection modalities are accompanied by more
Table 5 Coding Protocols for 2010–2019 Qualitative Articles
Overall Analysis Sample
(N = 438)
Journal Tier Article Citation Score
Top
(N = 83)
Other
(N = 355)
High
(N = 87)
Other
(N = 351)
Discuss Coding
No 224 (51.1) 20 (24.1) 204 (57.5) 44 (50.6) 180 (51.3)
Yes, Little Detail 89 (20.3) 18 (21.7) 71 (20.0) 16 (18.4) 73 (20.8)
Yes, Detailed 125 (28.5) 45 (54.2) 80 (22.5) 27 (31.0) 98 (27.9)
Inter-Coder Reliability
No 393 (89.7) 59 (71.1) 334 (94.1) 72 (82.8) 321(91.5)
Yes, Informal 23 (5.3) 14 (16.9) 9 (2.5) 9 (10.3) 14 (4.0)
Yes, Formal 22 (5.0) 10 (12.0) 12 (3.4) 6 (6.9) 16 (4.6)
Theoretical Saturation
No 402 (91.8) 74 (89.2) 328 (92.4) 78 (89.7) 324 (92.3)
Yes 36 (8.2) 9 (10.8) 27 (7.6) 9 (10.3) 27 (7.7)
Grounded Theory
No 355 (81.1) 55 (66.3) 300 (84.5) 69 (79.3) 286 (81.5)
Yes 83(18.9) 28(33.7) 55(15.5) 18(20.7) 65(18.5)
Coding Style
Deductive 7 (1.6) 2 (2.4) 5 (1.4) 2 (2.3) 5 (1.4)
Inductive 62 (14.2) 20 (24.1) 42 (11.8) 9 (10.3) 53 (15.1)
Both 12 (2.7) 6 (7.2) 6 (1.7) 3 (3.4) 9 (2.6)
Not Mentioned 357 (81.5) 55 (66.3) 302 (85.1) 73 (83.9) 284 (80.9)
Participant Follow-up
No 400 (91.3) 71 (85.5) 329 (92.7) 82 (94.3) 318 (90.6)
Yes 38 (8.7) 12 (14.5) 26 (7.3) 5 (5.7) 33 (9.4)
Software
No 334 (76.3) 68 (81.9) 266 (74.9) 74 (85.1) 260 (74.1)
Yes 104 (23.7) 15 (18.1) 89 (25.1) 13 (14.9) 91 (25.9)
Positionality
No 418 (95.4) 72 (86.7) 346 (97.5) 82 (94.3) 336 (95.7)
Yes 20 (4.6) 11 (13.3) 9 (2.5) 5 (5.7) 15 (4.3)
Percentages may not sum to 100 due to rounding
American Journal of Criminal Justice

discussion of coding strategies than others. Specifically, the majority of articles based on
interviews (51.6%) and focus groups (50%) discussed coding, whereas only 29.2% of
articles based on field work discussed coding.
We also examined if the authors mentioned inter-coder/interrater reliability in their
methods sections. Only 10.3% of all sampled articles discussed inter-coder/interrater
reliability among the authors. Of the articles that did alert readers to inter-coder/interrater
reliability efforts, 51.5% did so with passing statements and the remaining 48.5% did so
with detailed discussions. Articles in top tier journals were more likely to have discussed
inter-coder/interrater reliability (28.9%) than articles in other journals (5.9%).
Only a fraction of the articles that we reviewed provided discussion on the topic of
theoretical saturation. Only 8.2% of articles mentioned or discussed saturation, a pattern
that was consistent regardless of where articles were published. Grounded theory was
mentioned in 18.9% of all sampled articles. That percent rose slightly to 20.7% among
the most cited articles while significantly higher levels (33.7%) were observed in
articles published in top tier journals.
When coding data, researchers can use inductive, deductive, or a combination of
modalities. We found that it was uncommon for authors to directly mention which style
of coding they used; less than one in five (18.5%) did so. Inductive coding was the
most common style of coding (14.2%), whereas deductive coding was discussed in
1.6% of articles and a combination of the two was noted in 2.7% of articles. Articles in
top tier journals showed an increase in mention of coding styles (up from 18.5% to
33.7%), with inductive coding styles remaining the most frequently discussed style.
One way to ensure the reliability of coding is to have participants review some
version of the accumulated data. Articles that included a discussion of follow-ups with
participants accounted for only 8.7% of the articles comprising the overall analysis
sample. Top cited articles were found to be less likely to mention a follow-up with
participants (5.7%), while articles in top tier journals showed an increase in the
likelihood of mentioning follow-up with participants (14.5%).
Approximately 23.7% of articles mentioned the use of some sort of software to assist
with data analysis. In addition, articles that appeared in top tier journals or those
yielding top citation counts were found to be less likely to mention the use of analysis
software, 18.1% and 14.9% respectively. This is an interesting finding, seemingly
contradicting the pattern of top tier journal articles being more likely to report on an
issue and to use “advanced”means of data collection and analysis.
Positionality is a concept that has recently emerged among qualitative researchers
that is used to describe the potential for the identities of researchers and participants to
shape the research process (Bourke 2014). Discussions of positionality seldom ap-
peared in the sampled articles, as only 4.6% contained any mention of the positionality
between authors and participants. However, returning to the idea of top tier journal
articles including a more advanced discussion/explanation of methodological issues,
the percentage of articles mentioning positionality nearly tripled to 13.3% in the sample
of articles published in top tier journals. The sample of top cited articles showed only a
slight increase in the number discussing positionality (5.7%). Overall, when differences
occur among codes in the sampled articles, articles in top tier journals are generally
more likely to mention almost every category of code.
Table 6presents data on the type of participants studied using qualitative methods.
Offenders were the most commonly studied group: 41.6% of articles comprising the
American Journal of Criminal Justice

overall analysis sample focused on this category of participant. Professionals (e.g.,
police, judges and correctional officers/authorities) were the next most frequently
studied group (22.1% of articles). The remaining three types of study participants were
citizens, victims, and mixed samples; these three categories were represented by 14.4%,
9.4% and 12.6% of articles respectively. Articles published in top tier journals were
more likely to use samples made up of offenders (55.4%).
Our analysis reveals that qualitative articles overwhelmingly rely on samples of adult
participants. In reference to Table 6, note that juvenile participants were consistently
represented in less than 7% of the articles. The overwhelming majority of all articles,
consistently more than three quarters, described participants generically as “adults.”
Our analysis reveals that qualitative articles tend not to focus on one particular gender
when amassing study participants. A full 44.1% of the articles comprising our overall
analysissamplereportedondataprovidedbyboth males and females. In addition, 21.5%
were focused on men, 15.3% on women, and .5% on transgender samples. Interestingly,
18.7% of the articles did not include information on the gender of participants. Of all
Table 6 Type of Participants in in 2010–2019 Qualitative Articles
Overall Analysis Sample
(N = 438)
Journal Tier Article Citation Score
Top
(N = 83)
Other
(N = 355)
High
(N = 87)
Other
(N = 351)
Type of Participant
Offender 182 (41.6) 46 (55.4) 136 (38.3) 37 (42.5) 145 (41.3)
Professional 97 (22.1) 15 (18.1) 82 (23.1) 15 (17.2) 82 (23.4)
Citizens 63 (14.4) 10 (12.0) 53 (14.9) 14 (16.1) 49 (14.0)
Victims 41 (9.4) 5 (6.0) 36 (10.1) 10 (11.5) 31 (8.8)
Mixed 55 (12.6) 7 (8.4) 48 (13.5) 11 (12.6) 44 (12.5)
Age
Juveniles 25 (5.7) 3 (3.6) 22 (6.2) 6 (6.9) 19 (5.4)
Adults 339 (77.4) 74 (89.2) 265 (74.6) 67 (77.0) 272 (77.5)
Elderly 3 (0.7) 0 (0.0) 3 (0.8) 0 (0) 3 (0.9)
Mixed 33 (7.5) 3 (3.6) 30 (8.5) 6 (6.9) 27 (7.7)
Unknown 38 (8.7) 3 (3.6) 35 (9.9) 8 (9.2) 30 (8.5)
Gender
Male 94 (21.5) 15 (18.1) 79 (22.3) 19 (21.8) 75 (21.4)
Female 67 (15.3) 10 (12.0) 57 (16.1) 9 (10.3) 58 (16.5)
Mixed 193 (44.1) 40 (48.2) 153 (43.1) 40 (46) 153 (43.6)
Unknown 82 (18.7) 16 (19.3) 66 (18.6) 18 (20.7) 64 (18.2)
Transgender 2 (0.5) 2 (2.4) 0 (0.0) 1 (1.1) 1 (0.3)
LGBTQ
No 430 (98.2) 81 (97.6) 349 (98.3) 85 (97.7) 345 (98.3)
Yes, as demographic 3 (0.7) 0 (0.0) 3 (0.8) 0 (0.0) 3 (0.9)
Yes, as focus 5 (1.1) 2 (2.4) 3 (0.8) 2 (2.3) 3 (0.9)
Percentages may not sum to 100 due to rounding
American Journal of Criminal Justice

articles appearing in the top journals, 48.2% included both men and women, 18.1%
included only men, and 12% included only women. Articles appearing in the other journals
relied on data from men (22.3%), women (16.1%), and mixed gender (43.1%) samples.
However, when comparing articles based on citations, men made up the sample for 21.8%
of top cited articles and 21.4% of the others; women made up 10.3% of high impact articles
and 16.5% of others; and mixed samples in 46% of top articles and 43.8% of others. Very
few articles focused on queer populations. With respect to the overall analysis sample, a full
98.2% did not mention LGBTQ participants, while only 0.7% used LGBTQ as a demo-
graphic and 1.1% used LGBTQ as a focus in their article. These patterns held steady across
the articles with top citation counts and those published in top tier outlets.
Characteristics of Study Participants
Table 7provides a more nuanced examination of the study participants. Where researchers
relied on samples comprised exclusively of known criminals, they tended to target offense
generalists as opposed to specialists. Mixed offender types comprised the modal category
across the samples that we analyzed. Roughly one in five articles comprising the overall
analysis sample relied on samples comprised exclusively of drug offenders and one in ten
targeted only violent offenders. Articles in top tier journals were more likely to study those
who committed violent (23.1%) or drug offenses (28.8%) than other types of offenses. This
specialization trend is also seen in articles based on citations. The top cited articles showed
more attention for violent offenders (15.6% compared to non-top cited articles’8.8%) and
drug offenders (24.4% compared to non-top cited articles’18.8%).
A sizable body of literature exists about the best place to locate and interview offender
participants (Copes et al. 2013; Topalli et al. 2020). Overall, the location in which offenders
were recruited for the sampled studies included prison/jail (47.8%), halfway-houses and
treatment centers (5.5%), offenders on probation/parole (6%), and active offenders
(40.7%). When observing top versus non-top tier journal articles, we found the former to
be more likely to recruit active offenders (50% compared to 36.9%) and the latter to exhibit
more interest in offenders who currently reside in prison/jail (51.5% compared to 38.5%).
Next, Table 7provides breakout data on the qualitative articles that relied on victims
as study participants. Those suffering from violent or sex offenses were overwhelm-
ingly the focus in this regard, consistently combining to account for over 90% of the
published articles.
11
Approximately 54.2% of these articles with victims focused on
intimate partner violence. Studies that featured victims of violent or sex offenses
accounted for four of the five articles published in top tier outlets. The articles in
non-top tier journals included participants who were victims of violence (18.2%), sex
crimes (76.1%), and cybercrimes (4.3%). Similar trends appear when comparing
articles based on citation numbers, with the top cited articles focusing on victims of
violent (18.2%), sex (63.6%), cybercrimes (9.1%), or organizational (9.1%) offenses.
Also included in Table 7is data on articles that used professionals as participants. In
the articles comprising the overall analysis sample, these research participants include
police (23.8%), courtroom personnel (e.g., judges, attorneys, and lawyers) (18.5%),
correctional officers/authorities (21.5%), other (17.7%), and mixed (18.5%). Top tier
journal articles were found to be more likely to use courtroom personnel as their study
11
For victims who experienced both sexual and physical victimization we classified them as “sex crimes.”
American Journal of Criminal Justice

participants (36.8% compared to 15.3% of non-top tier journal articles). However, top
tier journal articles did not include a single study using non-law enforcement partici-
pants, while 20.7% of non-top tier journal article studies did. Professional types
compared through citation scores did not yield any significant differences. However,
the top 20% of cited articles were less likely to have used a mix of professional types
(12% compared to 20% of the bottom 80% of articles based on citations cores).
Conclusion
The present study highlights some important evolutions and growth regarding the
methodologies of criminal justice and criminology (CCJ). Most notably, the present
Table 7 Characteristics of Study Participants in 2010–2019 Qualitative Articles
Overall Analysis Sample Journal Tier Article Citation Score
Top Other High Other
Offender Type N = 205 N = 52 N = 153 N = 45 N = 160
Violent 21 (10.2) 12 (23.1) 9 (5.9) 7 (15.6) 14 (8.8)
Sex Crimes 18 (8.8) 1 (1.9) 17 (11.1) 1 (2.2) 17 (10.6)
White Collar 4 (2.0) 2 (3.8) 2 (1.3) 2 (4.4) 2 (1.3)
Drugs 41 (20.0) 15 (28.8) 26 (17) 11 (24.4) 30 (18.8)
Cyber 1 (0.5) 0 (0.0) 1 (0.7) 1 (2.2) 0 (0.0)
Gangs 7 (3.4) 3 (5.8) 4 (2.6) 1 (2.2) 6 (3.8)
Mixed 102 (49.8) 16 (30.8) 86 (56.2) 18 (40.0) 84 (52.5)
Offender Recruit Location
Active in Society 74 (40.7) 26 (50.0) 48 (36.9) 20 (44.4) 54 (39.4)
Probation/Parole 11 (6.0) 4 (7.7) 7 (5.4) 3 (6.7) 8 (5.8)
Halfway-House 10 (5.5) 2 (3.8) 8 (6.2) 2 (4.4) 8 (5.8)
Prison/Jail 87 (47.8) 20 (38.5) 67 (51.5) 20 (44.4) 67 (48.9)
Victim Type N = 51 N = 5 N = 46 N = 11 N = 40
Violent 11 (21.5) 2 (40.0) 9 (19.6) 2 (18.2) 9 (22.5)
Sex Crimes 37 (72.5) 2 (40.0) 35 (76.1) 7 (63.6) 30 (75)
White Collar 0 (0.0) 0 (0.0) 0 (0.0) 0 (0.0) 0 (0.0)
Cyber 2 (4.0) 0 (0.0) 2 (4.3) 1 (9.1) 1 (2.5)
Organizational/Inst 1 (2.0) 1 (20.0) 0 (0.0) 1 (9.1) 0 (0.0)
Professional Type N = 130 N = 19 N = 111 N = 25 N = 105
Police 31 (23.8) 5 (26.3) 26 (23.4) 7 (28.0) 24 (22.9)
Court room personnel 24 (18.5) 7 (36.8) 17 (15.3) 5 (20.0) 19 (18.1)
Correctional Personnel 28 (21.5) 3 (15.8) 25 (22.5) 5 (20.0) 23 (21.9)
Other 23 (17.7) 0 (0.0) 23 (20.7) 5 (20.0) 18 (17.1)
Mixed 24 (18.5) 4 (21.1) 20 (18.0) 3 (12.0) 21 (20.0)
Percentages may not sum to 100 due to rounding
American Journal of Criminal Justice

study provides evidence of the, albeit slow, growth of qualitative methods in the
discipline. This growth is most notable among the top tier of CCJ journals—places
that past research has described as publishing relatively few qualitative articles (Copes
et al. 2011). While a substantially greater proportion of the articles published in these
journals during the study period are shown to rely on qualitative methods, this
proportion remains less than one-half of that found in non-top tier journals. Interest-
ingly, for several journals, the prominence of qualitative articles is highest in the most
recent year(s). Whether this is the start of a new trend or a statistical anomaly is unclear,
and an issue that should be addressed by future research. Hence, while the evidence
supports the growth and increasing popularity of qualitative methods, it appears that the
journals that are the most selective (at least in the United States) are those least likely to
publish qualitative based analyses.
Beyond simply publishing more qualitative based articles than in previous years, so
too are there trends in the types of methodological approaches used and the issues
reported in articles, especially in those pieces published in top tier journals. We do not
make claims that what appears in these journals is the proper or best way to conduct
and write up qualitative research. We are merely providing trends and patterns in what
has been done (at least from 2010 to 2019). Here, presumably due to expectations of
using positivist framings, there is a greater likelihood of using mixed methods and
reporting about the numerous methodological decisions that are made in the research
process (e.g., recording of interviews, payments to subjects, and discussion of coding.).
Similarly, articles appearing in the top ranked journals are nearly four times more likely
to include discussions of policy implications of the research (also a likely product of
positivist ideals). In short, the evidence appears to support the claim that there are
different standards and expectations for papers in top tier journals than for those in
other journals and these expectations may relate to larger philosophical ideas about the
nature of qualitative data collection (i.e., positivism vs. interpretivism).
However, it is not simply the fact that qualitative methods are being used and
reported more among scholars that is important. So too is it important that the types
of knowledge being produced by CCJ research are expanding and growing. Indeed, the
types of questions asked and the types of answers uncovered are different for qualita-
tive and quantitative studies. Just as contemporary society is evolving to the point of
having much clearer and more distinct differentiations between actions, persons, beliefs
and responses to crime and criminals, so too is our science coming to emphasize a more
interpretive stance on knowledge. Whereas just a decade or two ago “interpretive”
studies were considered nice accompaniments to “real”(e.g., quantitative) research,
today those same studies are coming to be seen as integral to complete understandings
of crime and criminal justice. The simple growth in the number of qualitative studies
attests to this, as does the increase in citations for qualitative studies.
While it is encouraging to see the definitions of science and knowledge expand in
the discipline, there remain areas in which growth has been slow, and as a consequence
knowledge remains restricted and limited. Our findings suggest that there are two broad
areas where improvements can be made among qualitative criminologists. The first is
with issues regarding transparency in coding. While issues of transparency typically
relate to quantitative research and how data are collected and analyzed, such issues
have relevance for qualitative research as well. Our findings suggest that more often
than not, qualitative criminologists are not discussing issues relating to coding, inter-
American Journal of Criminal Justice

coder reliability, or positionality. Less than one-half of all articles included text
detailing in any way the coding approach used in their study. Only in about 10% of
the articles did authors discuss inter-coder reliability, which often stands as a center-
piece for positivist research. Having more details about how interviews are coded
(including whether inductive or deductive coding was used) will allow for readers to
better assess findings from such research. While replication is only beginning to take
hold in CCJ and these efforts will undoubtedly start with quantitative studies, it is
undeniable that replication of qualitative studies will be impossible without clear details
on how the data were coded and analyzed. Transparency in coding can also be found in
discussions of inter-coder reliability. The growth of multi-authored research suggests
that inter-coder reliability should be occurring among authors. Similarly, qualitative
researchers have long pointed to the benefits of sharing findings with participants
before publication to increase the validity of the findings. It was rare for authors to
follow up with participants about the accuracy or validity of the findings. In short, those
who publish qualitative research in CCJ journals may find it beneficial to be more
detailed and transparent in how they collected and analyzed their data.
Similarly, the role of social positions of researchers and participants (i.e.,
positionality) on the research process (including analysis) has increasingly become an
important methodological issue in many social science disciplines. In fact, some
journals ask for a specific statement on positionality from authors for qualitative
research. It appears that this trend of discussing positionality has not yet made its
way into CCJ, especially in the non-top tier journals. Rather, as our data indicate,
positionality is a concept that has only just begun to hold relevance in CCJ.
The other area for improvement concerns the topical areas and participants under
investigation. Our findings suggest that there is an emphasis on studying those who
commit crime. This is certainly a valuable population to study, but so too are other
groups. If one of the goals of qualitative research is to provide a voice to the
marginalized, then we need to ensure that we study marginalized groups, including
victims and LGBTQ+ populations. Only a small proportion of articles focused on the
victims of crime. Victims often come from marginalized populations so including their
voices may aid in improving social justice. As our analysis presented above suggests,
as a discipline, we may not be paying enough attention to the victims of crime. Less
than 10% of examined qualitative articles sought the perspective of the victims of
crime. Similarly, another growing area within the broad social sciences is queer studies
and queer theory. Again, CCJ appears to be behind in this area. It is rare that qualitative
criminologists focus on (or even take into account) LGBTQ+ populations. This is not to
say that CCJ ignores or neglects LGBTQ+ populations and topics, but the field has yet
to embrace such studies and move such foci and populations to the center of its
inquiries (Panfil and Miller 2014).
Another topical area that qualitative researchers can improve on is the policy
implications of their findings. The low percentage of articles that discuss policy suggest
that few are using these methods in program evaluations. Incorporating qualitative
methods in program and policy evaluations has the potential to aid insights (from those
who implement the policy and those who are affected by it) unobtainable from
quantitative assessments alone.
These changes in the growth and content of qualitative articles, we believe, are all
very positive changes for CCJ. As the discipline continues to mature and evolve, so too
American Journal of Criminal Justice

will the knowledge that is produced mature and evolve. And, with more and better data,
enhanced understandings of data, and refined interpretation and application of our
expanding knowledge base it should be increasingly possible that our research can
make a difference. After all, CCJ at its core is an applied discipline. Yes, we wish to
understand crime, those who commit it, and societal responses to such, but so too do we
seek to use our knowledge to reduce crime, to identify causes and correlates of criminal
offending, to help crime victims, and to identify and facilitate a fair, just, swift and
crime-reducing response to crime. This study provides us hope that we can, and will,
achieve such goals.
References
Armstrong, E. K. (2020). Political ideology and research: How neoliberalism can explain the paucity of
qualitative criminological research. Alternatives:Global,Local,Politicaldoi:https://doi.org/10.1177
/0304375419899832.
Boeri, M., & Shukla, R. K. (Eds.). (2019). Inside ethnography: Researchers reflect on the challenges of
reaching hidden populations. University of California Press.
Bourke, B. (2014). Positionality: Reflecting on the research process. The Qualitative Report, 19(33), 1–9.
Buckler, K. (2008). The quantitative/qualitative divide revisited: A study of published research, doctoral
program curricula, and journal editor perceptions. Journal of Criminal Justice Education, 19,383–403.
Clear, T.R. (2006). 2006 Survey Report.http://www.adpccj.com/documents/2006survey.pdf .
Copes, H., Brown, A., & Tewksbury, R. (2011). A content analysis of ethnographic research published in top
criminology and criminal justice journals from 2000-2009. Journal of Criminal Justice Education, 22,
341–359.
Copes, H., Hochstetler, A., & Brown, A. (2013). Inmates’retrospective experiences of prison interviews. Field
Methods, 25,182–196.
Copes, H., & Miller, J. M. (Eds.). (2015). eThe Routledge handbook of qualitative criminology. Routledge.
DeJong, C., & St. George, S. (2018). Measuring journal prestige in criminal justice and criminology. Journal
of Criminal Justice Education, 29(2), 290–309.
Kleck, G., Tark, J., & Bellows, J. J. (2006).What methods are most frequently used in research in criminology
and criminal justice? Journal of Criminal Justice, 34, 147–152.
Lane, J., Myers, D.L., Koetzele, D. & Armstrong, G. (2019). Association of Doctoral Programs in
Criminology & Criminal Justice (ADPCCJ) 2019 Survey Report.http://www.adpccj.
com/documents/2019survey.pdf.
Miller, J., & Palacios, W. R. (Eds.). (2015). Qualitative research in criminology. Transaction Publishers.
Panfil, V. R., & Miller, J. (2014). Beyond the straight and narrow: The import of queer criminology for
criminology and criminal justice. The Criminologist, 39(4), 1–9.
Rice, S., & Maltz, M. (2018). Doing ethnography in criminology: Discovery through fieldwork. Springer.
Roche, S. P., Fenimore, D. M., & Jennings, W. G. (2019). Trends in top journals in criminology and criminal
justice. Journal of Criminal Justice Education, 30(4), 551–566.
Sorensen, J. (2009). An assessment of the relative impact of criminal justice and criminology journals. Journal
of Criminal Justice, 37, 505–511.
Sorensen, J., Snell, C., & Rodriguez, J. J. (2006). An assessment of criminal justice and criminology journal
prestige. Journal of Criminal Justice Education, 17,297–322.
Tewksbury, R., Dabney, D., & Copes, H. (2010). The prominence of qualitative research in criminology and
criminal justice scholarship. Journal of Criminal Justice Education, 21,391–411.
Tewksbury, R., DeMichele, M. T., & Miller, J. M. (2005). Methodological orientations of articles appearing in
criminal justice’s top journals: Who publishes what and where. Journal of Criminal Justice Education,
16(2), 265–279.
Tewksbury, R., & Mustaine, E. E. (2011). How many authors does it take to write an article? An assessment of
criminology and criminal justice research article author composition. Journal of Criminal Justice
Education, 22(1), 12–23.
Topalli, V., Dickinson, T., & Jacques, S. (2020). Learning from criminals: Active offender research for
criminology. Annual Review of Criminology, 3,189–215.
American Journal of Criminal Justice

Woodward, V. H., Webb, M. E., Griffin, O. H., & Copes, H. (2016). The current state of criminological
research in the United States: An examination of research methodologies in criminology and criminal
justice. Journal of Criminal Justice Education, 27(3), 340–361.
Publisher’sNote Springer Nature remains neutral with regard to jurisdictional claims in published maps and
institutional affiliations.
Heith Copes is a professor in the Department of Criminal Justice at the University of Alabama at Birming-
ham. His research focuses on narrative sense making among those who engage in crime and deviance.
Blake Beaton is working on his Master of Science in Criminal Justice at the University of Alabama at
Birmingham. His research interests include illegal drug use and qualitative methodology.
David Ayeni is a doctoral student in the Department of Criminal Justice and Criminology at Georgia State
University. His research agenda focuses on the study of police culture, satisfaction and leadership. He is also
interested in terrorism, specifically that which involves Boko Haram.
Dean A. Dabney Ph.D. is a Professor and Chair in the Department of Criminal Justice and Criminology at
Georgia State University. His research agenda is principally focused on the study of police culture and their
efforts to combat violent crime. In recent years, he has studied the operation of homicide units, the use of
confidential informants, police response to gun violence, and officer use of discretion.
Richard Tewksbury is a retired Professor of criminal justice at the University Of Louisville.
American Journal of Criminal Justice