ArticlePDF Available

Issues to Consider Prior to Using the R-PAS in a Forensic Context

  • Consultant/author/artist
  • Maverick Psychology


It is essential to understand that CS validity research does not translated directly to the R-PAS. In this article we dicuss essential issues to consider prior to using the R-PAS in an applied context.
SIS J. Proj. Psy. & Ment. Health (2021) 28: 01-00
Carl B. Gacono, PhD, ABAP, Private Practice, Asheville, NC,; Jason M. Smith, PsyD, ABPP, Adjunct Faculty,
Pierpont Community & Technical College, Fairmont, WV.
Key Words: Rorschach, R-PAS, Comprehensive System, Court testimony, assessment
Issues to Consider Prior to Using the R-PAS in a Forensic Context
Carl B. Gacono and Jason M. Smith
After the sudden death of John Exner (developer of the Comprehensive System; CS; Exner, 1969, 2003), a new scoring
system, the Rorschach Performance Assessment System (R-PAS; Meyer, Viglione, Mihura, Erard, & Edberg, 2011) was
introduced. While the CS was developed with extensive research over approximately 40 years, this new system has only been
studied for approximately fifteen years. When considering its forensic usage, it is essential to understand that CS validity
research does not translate directly to R-PAS. One primary reason for the lack of translation is the differing administration
procedures. Other factors such as the absence of comparison descriptive group data (non-patient, clinical, & forensic) and
internal validity problems with studies utilized in the primary meta-analyses which supports the R-PAS variable selection
should be considered in determining the methods admissibility. In this article, we discuss essential issues to consider prior to
using the R-PAS in an applied context.
After the premature death of John Exner
(developer of the Comprehensive System; CS;
Exner, 1969, 2003), a new scoring system, the
Rorschach Performance Assessment System (R-
PAS; Meyer, Viglione, Mihura, Erard, & Edberg,
2011) was introduced. While the CS was
developed with extensive research over
approximately 40 years, this new system has only
been studied for around fifteen years. For several
reasons CS validity research does not translate
directly to the R-PAS. A primary reason for the
lack of translation is the differing administration
The R-PAS has been criticized related to its
administration procedures (Khadivi, & Evans,
2012; Mattlar, 2011; Ritzler & Sciara, 2010;
Smith, 2012), the lack of descriptive comparative
data (this article), its use of the international
Rorschach norms (Andronikof, 2019; Ritzler &
Sciara, 2009; Sciara & Ritzler, 2018), the
declaration that it has no cross cultural differences
(Andronikof, 2019; Fontan, 2014, 2019; Fontan &
Smith, 2018), problems with the indices used (i.e.,
Complexity, Suicide Constellation; Fontan, 2014;
Fontan & Smith, 2018), and the lack of forensic
admissibility (Gurley, Piechowski, Sheehan, &
Gray, 2014; Kivisto, Gacono, & Medoff, 2013). At
this time, these criticisms have not been adequately
Altered Administration Procedures:
R-PAS administration procedures have created a
“Rorschach” distinct from the CS Rorschach.
Reportedly, Meyer et al. (2011) conducted initial
research on this new procedure and stated the
following reasons to accept their new
administrations: 1) reduced short and long records
(smaller SD for R), 2) eased the administration
task (stating no need to re-administer), 3) very
little effect on variables, 4) better fit for protocols
to the norms, and 5) potential for less examiner
variation across sites. R-PAS administration
requires cards to be “pulled” after four responses.
This procedure allows for 40 (range 20-40) to be
the highest number of responses possible.
The need for limiting lengthy protocols, is quite
likely a straw person “concern.” In administering
hundreds of Rorschachs, we have never had more
than about 1% that required the CS instructions
related to lengthy protocols, and not many more
that were too “short” (4%). While rates for
experienced clinicians who routinely administer
the CS Rorschach deserves study, this does not
seem to be a legitimate reason for modifying the
CS’ well thought out administration procedures1.
As noted by Gacono and Meloy (1994) related to
assessing antisocial and psychopathic subjects:
We have found, contrary to our expectations,
that psychopaths generally produce a normative
amount of Rorschach responses, at least in a
research setting. We have purged our group
samples of protocols with less than 14 responses
without great heartache, because they were few
in number (p. 5)2.
Though Exner (2003) slightly altered the
administration through different versions of the CS
(and the five American Rorschach systems prior to
00: Gacono & Smith
the CS: Klopfer, Beck, Hertz, Piotrowski, and
Rapport-Schafer), his administration instructions
were based on an assimilation of the best
procedures from the five systems. Exner (1974)
stated the CS “closely parallels that of the
traditional systems [and] it is designed to
provide the Rorschach community with a common
language and a common methodology” (p. 17).
The R-PAS procedures, on the other hand, were,
based primarily on concerns with the problems
caused by the number of responses influencing the
“distributions of other Rorschach variables and
confounds their interpretation” and a comparison
to the CS instructions (Meyer et al., 2011, p. 409).
Specifically, “R-PAS was developed with one of
its aims being to better manage the number of
responses given to protocols, and thus improve its
psychometrics” (Meyer et al., 2020, p. 144).
Managing R may or may not improve the general
psychometrics (reliability) of the Rorschach. In
some cases, it may decrease its validity.
Rather than just the open-ended CS instruction of
“What might this be?” the R-PAS also includes
these implicit instructions “Try to give two
responses… or maybe three, to each card. That is,
for each card try to see two different things;
possibly three” (Meyer et al., 2011, p. 29). This
directive approach is a significant departure from
the open-ended CS instructions. The impetus of the
R-PAS instructions is to “control” or “manage”
rather than to facilitate a client directed process.
The existing Holtzman inkblot test provides an
alternative Inkblot method for examiners who
prefers a more directive, “controlled” approach.
Ironically, in their CS validity meta-analyses,
Mihura, Meyer, Dumitrascu, and Bombel (2013),
endorse the impact that different administration
procedures have on data production as they
“excluded results derived from studies employing
Rorschach test administration procedures that
markedly deviated from standard procedures” (p.
560). R-PAS authors have acknowledged the need
to have uniform Rorschach administration
procedures (Meyer et al., 2020). Yet, the
implications of this concern are seemingly ignored
when utilizing statistically manipulated CS
administered protocols for promoting the R-PAS
Statistics and meta-analyses have been utilized to
justify an absence of meaningful differences
among Rorschach administration procedures
(Hosseininasab et al., 2019; Meyer et al., 2020;
Pianowski, Meyer, de Villemor-Amaral, Zuanazzi,
& do Nascimento, 2019). These studies have found
some statistical differences which were largely
ignored when statistical comparisons of select
variables did not produce significantly different
quantities. It should be noted that six studies in the
meta-analysis did not use exact R-PAS
administration manual guidelines (Hosseininasab
et al., 2019; Meyer et al., 2020). Also statistics
deal with ranges and probabilities and fail to
capture, when non-patient protocols are utilized,
the multitude of differences that become apparent
when the Rorschach is used in clinical samples.
This is one reason why understanding the history
of a variable’s development is essential to selecting
variables for inclusion in any system as opposed to
using a statistical chopping block.
A study by Hartmann (2001) altered the Exner
administration by providing longer instructions
prior to administering the test. The instructions
I am going to show you some pictures. There are
10 in all, and none of them resemble anything in
particular. They are made by putting ink on a
piece of paper, folding it, and pressing the sides
together, so that they become figures. Now I
would like you to tell me what they look like,
remind you of, or make you think of. There is no
right or wrong answers. Different people see
different things, and I want to know what you
think they might resemble. Tell me everything
you see in each picture, and when you have
finished with a card, you may put it down
(Hartmann, 2001, p. 462).
At no time did Hartmann instruct the examinee to
give a certain amount of responses a la R-PAS
administration. In response to this lengthy
introduction (instructions) fewer brief protocols
were produced when compared to the standard
Exner instructions (“What might this be?”);
however, she found no significant group
Rorschach variable differences between the two
instructions. Pianowski et al. (2019) looked at two
groups of non-patient protocols and found no
differences in CS Rorschach variables when
Using the R-PAS: 00
administering protocols with the CS (N = 50) or R-
PAS (N = 50) administration methods. In this
study, 53 statistical analyses were done between
Rorschach variables from an R-PAS and CS
administration. They found that a CS
administration had higher means for FQ-% and the
R-PAS equivalent of the Affective ratio and lower
means on R and FQu% compared to an R-PAS
administration. Three other differences were found
on variables used only in R-PAS. Instead of
specifically hypothesizing what could be different
in the two administrations, the large number of
comparisons may have resulted in lost data.
Statistical approaches absent the historical
rationale (sound theory) utilized in developing the
CS instructions are limited by many factors.
Significant differences can manifest in clinical
samples that were not evident with non-patients.
The extent to which alterations in administration
procedures impact results in a range of clinical
samples, nor the impact of altered administration
on important interpretative approaches such as
sequence analysis, have not yet been fully assessed
(see Meloy, Acklin, Gacono, Murray & Peterson,
Directive versus opened ended administration
instructions initiate distinctly different cognitive
test taking sets which can interact with a given
personality style in creating significantly different
results. The validity of the Rorschach rests on its
ability to elucidate different variable pattern of
variables dependent on the patients’ clinical issues
and personality style. How might an individual
with a perfectionistic style be impacted by the R-
PAS instructions? Does the direct instructional set
create a template for judging their performance?
What is the impact on a paranoid individual who
now has a template for responding? Does a
directive approach reduce ambiguity and
subsequently impact their anxiety (Y, m)? Will it
negate the need for them to end Cards with a Dd; a
pattern where the stimulus field is narrowed to
make the previously ambiguous, testing experience
manageable? Will the Obsessive-Compulsive
person feel pressure to conform to the demands of
the R-PAS instructions? Or will the structure
reduce their anxiety? While the answers to these
questions are unknown, what is known is that the
CS open ended administration procedures have
weathered the test of applied usage, and “work”
effectively in eliciting valuable patterns consistent
with the theoretical understanding of a multitude of
clinical and non-clinical populations (Gacono,
2019; Mattlar, 2011; Ritzler, 2014; Smith, Gacono,
Fontan, Cunliffe, & Andronikof, 2020; Smith et
al., 2018). The impact of a more directive approach
has not yet been demonstrated.
The Rorschach’s sensitivity to variations in
administration, such as altered lighting, using dirty
cards, seating position, etc. has also been
demonstrated (Exner, 2003). There is a plethora of
social science literature concerning the impact of
instructional sets on subsequent performance
(McDaniel, Hartman, Whetzel, & Grubb, 2007;
McDaniel & Nguyen, 2001). One example from
Gacono and Meloy (1994) demonstrates how
changing even a single word can significantly
impact results. In her dissertation, Margolis (1992)
attempted to replicate Gacono and Meloy’s (1994)
list for scoring AgC. She, however, changed the
original research instructions from “Please rate
those objects popularly perceived as predatory,
dangerous, malevolent, injurious, or harmful”, to,
“Please rate those objects you perceive to be
predatory, dangerous, malevolent, injurious or
harmful” (p. 273). By doing so, she evoked a
distinctly different cognitive set in the survey
participants. Gacono and Meloy’s (1994) choice of
popular was intentional and meant to be
ambiguous. It allowed subjects to rate objects as to
what most or many people consider to be
aggressive. Respondents could externalize
responsibility for their choice, be less likely to
assume their answers has any relationship to their
own aggression (essential in non-patients where
aggression is likely ego-dystonic) and would
decrease any concerns about being judged for their
answers. Not surprisingly, Margolis’ altered
instructions produced different results. Statements
by the respondents supported the presence of
censoring, “other(s) might see these words as
being aggressive, I don’t see them in that way,”
and “I must be really aggressive because I checked
a lot of the words (Margolis, 1992, p. 41; Gacono
& Meloy, 1994, p. 273). In a similar vein
comments made by clients taking the R-PAS, “Oh
wait, you said I have to give two or three
responses” support the contention that a distinct
00: Gacono & Smith
and different (from CS) cognitive set is elicited by
R-PAS instructions.
Several early studies examined the impact of
altered Rorschach administration procedures on
Rorschach production. Blais, Norman, Quintar,
and Herzog (1995) found the CS administration
showed more color, blends, and shading responses
than a Rapaport Rorschach administration.
Bornstein, Bonner, Kildow, and McCall (1997),
studied an undergraduate population and found no
differences between individual and group
administered Rorschach protocols; however, they
only focused on their non-CS derived ROD score.
Might their results have been different in clinical
samples? How about psychopathic offenders who
might be motivated to conform (females) or rebel
(males)? In considering our recent study of
psychopathic women, any instructional set that
provides a template for comparing one’s
performance is likely to increase their need to
interact with others to aid in regulating emotions
and issues with self-worth that manifest during the
Rorschach administration3. Would these altered
instructions create another point from which they
can evaluate themselves negatively? Would it
increase the production of PERs, Cgs, projection,
and other indications of defensiveness? These
issues have not been adequately addressed by the
R-PAS research. The meaning from the many
years of study (with links to theory) that went into
developing the CS administration procedures
seems to have been ignored. This is just one of the
common fallacies (bias) in psychology research
that stems from a failure to understand fully, or
even ignoring, the history of what precedes one’s
current “discoveries” (see Chapter 2 in Smith,
Gacono, & Cunliffe, 2021).
The R-PAS lacks adequate descriptive group data:
Currently, the normative data for R-PAS consists
of statically manipulated CS administered
protocols (Meyer et al., 2011). A statistical
procedure that has been called R-Optimized. This
method intends to mimic how a Rorschach
protocol would look if it were administered using
R-PAS instructions. This method is in contrast to
developing descriptive group data based on using
R-PAS administration procedures.
The R-PAS manual indicates that the International
CS non-patient normative data was utilized
(Meyer, Erdberg, & Shaffer, 2007; N = 1396), with
a statistical modelling procedure, to mimic a non-
patient normative sample that was administered
with R-PAS guidelines (N = 640). Sciara and
Ritzler (2018) also discussed cautionary notes
about the international sample itself:
(a) the use of inexperienced examiners; (b) the
ways sampling might affect generalizability (i.e.,
questionable selection procedures, insufficient
exclusion criteria, inadequate sample size); and
(c) the use of less than ideal administration
procedures, including the lack of a warmup
phase as part of the administration (p. 343).
In order to model the normative data to look like
R-Optimized administration protocols 123
protocols administered by students primarily under
Don Viglione’s supervision using R-PAS
administration of patients and non-patients were
used in the statistical procedure. This sample
consisted of “30 non-patients, 30 chronic but stable
schizophrenia patients, 15 adult and 15 adolescent
sex offenders, and 33 outpatients” (Meyer et al.,
2011, p. 470). Of note, however, there are no
comprehensive descriptive data sets of R-PAS
administered normative, clinical, or forensic
This can be contrasted with descriptive data
contained in The Rorschach Workbook for the
Comprehensive System (4th & 5th edition; Exner,
1995, 2001) which includes descriptive CS data for
Nonpatient Adults (N=700; also listed by
Introversive, Extratensives, & Ambitents),
Nonpatient Children and Adolescents (N=1390;
also listed by age), Inpatient Schizophrenics
(N=320), Inpatient Depressives (N=315), and
Character Disorders (N=180). Exner (2007) also
provided a new 450 non-patient normative sample.
Directly related to using the Rorschach in a
forensic context, descriptive data is available for
Conduct Disordered Children (N=72; Gacono,
Gacono, & Evans, 2008), Conduct Disordered
Adolescents (N=179; Gacono et al., 2008),
Antisocial Personality Disordered Women
(N=232; Smith et al., 2021), Female Psychopaths
(N=115; Smith et al., 2021), Antisocial Personality
Disordered Males (N=108; Gacono et al., 2008),
Non-Sexually Offending Psychopathic Males
Using the R-PAS: 00
(N=32; Gacono, Meloy & Bridges, 2008), Non-
violent Male Pedophiles (N=39; Gacono, Meloy &
Bridges, 2008), Sexual Homicide Perpetrators
(N=38; Gacono, Meloy & Bridges, 2008), Sexually
Offending Females (N=39; Smith et al., 2021),
Antisocial Disordered Schizophrenic Men (N=100;
Gacono et al., 2008), Inmates in Forensic
Psychiatric Treatment (N=211; Young, Erdberg, &
Justice, 2008), three groups of Forensic
Outpatients (paranoid schizophrenics N=90,
undifferentiated schizophrenics, N=38, Bipolar
patients N=40; Gacono & Gacono, 2008) and
Child Custody Litigants (N=728, also presented by
gender, Singer, Hoppe, Lee, Olesen, & Walters,
This translates to 2061 forensic protocols
administered with CS procedures (see also Gacono
& Evans, 2008; Gacono & Meloy, 1994;
Piotrowski, 1996a, 1996b, 1996c, 2007, 2017).
The R-PAS has two forensic samples (N=30), 15
adult and 15 adolescent sex offenders (Meyer et
al., 2011). Forensic “case” studies, not a substitute
for comparative descriptive data, were provided in
edited texts by Erard and Evans (2017) and Mihura
and Meyer (2018).
Issues with Mihura et al. (2013) Meta-analyses:
The Mihura et al. (2013) meta-analyses played a
significant role in determining which variables
would be included in the R-PAS. The following
guidelines guide the inclusion of a variable in R-
PAS if 1) the notion that there needed to be a
conceptual association between the code and the
interpretation, 2) parsimony, and 3) clinical
experience based on experienced practitioners
(Meyer et al., 2011). These meta-analyses
contained 210 studies; however, not all were used
to demonstrate validity for the CS variables.
Specifically, Mihura stated, associations between
the Rorschach and self-report measures were not
used to determine the validity of Rorschach
variables” because “Rorschach studies using self-
report measures for validity criteria are typically
published in less rigorous journals” (i.e., journal
impact factor, Mihura, 2019, p. 171).
When examining all 210 studies within the Mihura
et al. (2013) meta-analyses using five criteria
(IQ/Education level, R, Lambda/F%, inter-rater
reliability, sample size), every article had
methodological or internal validity issues that
made a full interpretation of their findings
problematic (Smith et al., 2018). Four articles only
had one issue (1.9%), 17 had two (8.1%), 85 had
three (40.5%), 87 had four (41.7%) and 17 had five
(8.1%). For example, 51.4% did not provide any
statistics for R, 67.1% did not provide this data for
Lambda/F%, 9.5% used IQ scores < 80, 34.2% did
not include an inter-rater reliability statistic, and
26.7% had a sample size < 20.
Twenty-two (10.5%) of the studies included
protocols with less than 14 responses. These older
CS Rorschach studies with shortened protocols
(current CS guidelines consider that < 14 response
protocols may be statistically invalid/unreliable)
were included in the Mihura et al. (2013) meta-
analyses (Smith et al., 2018). Besides the problem
with including low R studies in this type of
analyses it should be noted that R-PAS instructions
would preclude these protocols from being
produced (push for 2 and pull after 4; Meyer et al.,
Smith et al. (2018) also found that of the 210
studies in the Mihura et al. (2013) meta-analyses,
only 104 of the 210 (49.5%) studies were research
on the Rorschach (validation studies). None met all
five Rorschach criteria assessed (appropriate inter-
rater reliability, R, Lambda, sample size, &
IQ/educational level). Internal validity issues with
90% of the studies examined had three or more
issues related to the above criteria. Without this
information, one cannot determine whether the
statistical procedures in the studies were
appropriate and/or whether the findings can be
generalized beyond the sample studied.
Contradictions were also noted in how the meta-
analyses results were used. For example, the HVI
variable was included in R-PAS with little support
found in the meta-analyses while the DEPI which
had moderate support in the meta-analyses was
excluded from R-PAS. Of note, the CS Human
Experience score (Hx) was removed from R-PAS
based on a statistical chopping block. We found
this to be an important indicator of primitive
hysteria in psychopathy (female offenders M =
0.84, 35%; female psychopaths M = 1.03, 38%;
male offenders M = 0.15; 11%; male psychopaths
M = 0.18, 12%, Chapter 3 in Smith et al., 2021).
00: Gacono & Smith
This suggests there is some clinical utility for this
Failure to adequately address critics of the R-PAS:
Criticisms of R-PAS have generally been met with
statistical analyses that fail to address the
conceptual points raised (Smith et al., 2020).
Applying statistics without putting them into
context is never justified (i.e., Simpson paradox). It
may lead to large effect sizes and statistical
significance, but the results can still be
meaningless (lacking in practical significance and
logical soundness; see Chapter 2 in Smith et al.,
2021; Smith et al., 2020). This over-evaluation of
statistical prowess at the expense of conceptual
understanding has resulted in the tail (statistics)
wagging the dog (sound conceptual driven
methodology; Gacono 2019). It can lead to
important variables being eliminated without
adequate consideration of their historical value
within certain clinical populations. There is some
truth to the notion that, “There are three kinds of
lies: lies, damned lies, and statistics” (unknown).
Or, as a colleague, Taka Endo, has stated, “The
study of statistics is a method to measure if we are
in the right direction not a method of finding the
truth.” Perhaps, Sigmund Freud said it best,
“Those critics who limit their studies to
methodological investigations remind me of people
who are always polishing their glasses instead of
putting them on and seeing with them” (in Reik,
1956, p. 54).
In the interest of clarifying some of the issues with
the R-PAS, we requested that the data from the
Mihura et al. (2013) study be shared with us so that
we might verify the hypothesized Rorschach
predictorcriterion associations database, validity
coefficients decisions, and interrater reliability for
the predictor criterion associations (Mihura et al.,
2013). As Mihura noted in her commentary (2019,
p. 179), in the spirit of ethical guidelines and
scientific cooperation, we provided her with our
data from Smith et al. (2018) for her review and
analysis. However, we were denied access to her
meta-analyses data. Mihura stated “I (the first
author) am using some of the secondary data in
new analyses andin some cases (not all)it
would be difficult or impossible to accurately trace
these databases back to their original format and
reconfigure a new database.” She also stated it was
more than five years after publication and that the
data we requested would be explained in her reply
(it was not; Joni Mihura, Personal Communication,
March 17, 2019). This first point is of significant
concern--that data used in a significant way to
support and promote R-PAS is no longer available.
Unlike the issue with sharing the original CS
protocols, where there are confidentiality issues,
the data from the Mihura et al. meta-analyses have
no such constraints.
Finally, the lead creator of R-PAS (Meyer) has
discussed similar issues related to other
researchers. Meyer (2000) has been critical of
“arguments … designed to convince readers of a
conclusion, regardless of its accuracy” (p. 77) and
[others] were not sufficiently careful in their
scholarship (e.g., the erroneous citations), were
not aware of some key literature on a topic (e.g.,
the composite variables), presented a limited and
slanted portrayal of relevant issues and evidence,
… and repeatedly dismissed corrective feedback
(p. 77).
Meyer also stated,
As the Rorschach evidence base continues to
grow and develop, sound and balanced criticism
of the literature will help advance scientific
knowledge and applied practice. Conversely,
publishing assertions that are known to be wrong
or misleading can only serve political purposes
that thwart the goals of science and retard
genuine evolution in the field (2000, p. 78).
In addition the above issues, Meyer et al. (2020)
have now advocated that during the COVID-19
pandemic, it is acceptable to administer the
Rorschach virtually or remotely using the above
mentioned Rorschach administration studies,
supported by three dissertations on virtual
Rorschach administrations (one dissertation was
not complete; Berry, 2020; Logid, 2018;
Whitehouse, 2019). The numerous modifications
to standardized procedures that they are needed to
do this make it inappropriate to administer the R-
PAS virtually4. Remote Rorschach administration
is also likely unethical (CSIRA/ARISI, 2020). The
results have certainly not been proven to be valid
(CSIRA/ARISI5, 2020). It should be noted that two
prominent R-PAS creators, Meyer and Viglione
(2008), have previously stated that the “standard
Using the R-PAS: 00
CS test administration does not lend itself to
automated, computer-adapted administration or to
computer automated scoring” (p. 306) in direct
contradiction to offering or suggesting virtual
administration. This represents another
inconsistency between ethical and competent
practice and the promotion of R-PAS.
The above information offers a blueprint for
challenging the R-PAS should it be used in any
forensic context.
1. There are a number of criticisms of R-PAS that have
not been as of yet adequately addressed. These
include its altered administration procedures
(Khadivi, & Evans, 2012; Mattlar, 2011; Ritzler &
Sciara, 2010; Smith, 2012), its use of the
international Rorschach norms (Andronikof, 2019;
Ritzler & Sciara, 2009; Sciara & Ritzler, 2018), the
declaration that it has no cross cultural differences
(Andronikof, 2019; Fontan, 2014, 2019; Fontan &
Smith, 2018), problems with the indices used (i.e.,
Complexity, Suicide Constellation; Fontan, 2014;
Fontan & Smith, 2018), and the lack of forensic
admissibility (Gurley et al., 2014; Kivisto et al.,
2. The meta-analysis on R-PAS administration
included studies with instructions different from
those found in the R-PAS manual (Hosseininasab et
al., 2019; Meyer et al., 2020).
3. The R-PAS altered “directive” administration
procedures to create a different test than the CS
“open ended” administration procedures.
Consequently, CS validity does not translate directly
to the R-PAS.
4. The R-PAS validity lacks the necessary R-PAS
administered comparative descriptive data to be used
clinically or forensically. Other than 15 adult and 15
adolescent sex offenders and a few forensic case
studies, no forensic samples exist from R-PAS
administered protocols.
5. The majority (90%) of the Mihura et al. (2013) CS
meta-analyses studies had internal validity issues
(Smith et al., 2018, 2020) such as studies did not
include IQ/Educational level, did not report R or
Lambda statistics, had Rorschach protocols < 14, did
not report inter-rater reliability, and had sample sizes
less than 20.
6. Statistical approaches used by R-PAS proponents
fail to address the conceptual issues raised by R-PAS
critics (Smith et al., 2020).
7. The data from the Mihura et al. (2013) meta-
analyses that has been used to promote R-PAS is,
according to Mihura, no longer available in “their
original format.”
8. Administering the R-PAS virtually is likely to be
unethical (CSIRA/ARISI, 2020).
Given the weight of the above issues, the applied
usage of R-PAS is premature and will likely add
legitimate grounds for critics of the Rorschach to
devalue the test (Piotrowski, 1996a, 1996b, 1996c,
2007, 2017). It will aid in creating one more “straw
person” or “apparent” controversy that detractors
will use to discredit the Rorschach Method and
further influence practitioners to avoid using the
Rorschach in forensic contexts. Rather than
discarding the use of the Rorschach and the
valuable information it provides, however, we
recommend that psychologists continue to use the
CS in clinical and forensic contexts and increase
their sophistication for presenting testimony
(Piotrowski, 2007; Wiener, 2008). This is really
not a difficult task (Wiener, 2008).
As outlined by Andronikof (2019, p. 9) “a) the
validity of the CS has a solid evidence-base which
has not been disproved, b) the attempt to discredit
its creator is produced by promoters of a
competing system, [and] c) the R-PAS, although
founded on Exner’s work, deviates from and
betrays Exner’s interpretation principles and is
therefore not a continuation or development of the
CS.” It should also be noted that while many of the
CS Rorschach critics of the past two decades were
individuals who were not experts in assessment,
the Rorschach, or even clinical practice, the
majority of those who now criticize R-PAS are
Rorschach experts; many of whom have conducted
original Rorschach research.
1An “experienced clinician” might be defined as a licensed
psychologist who has administered several hundred Rorschach
protocols in applied contexts over a reasonable length of time
(say 10 years). It would not include student administrators
working only in a research setting. Proper rapport frequently
reduces the number of protocols that might conclude with < 14
2While “at least in research settings” was included in this
statement, most of the protocols throughout the forensic
samples where administered in clinical contexts and then
included as archival data (Gacono & Evans, 2008). A more
accurate statement would be “at least within our forensic
3We have found these women (N=115) to manifest primarily
extratensive and ambitent styles (Smith et al., 2021).
00: Gacono & Smith
4 i.e., size of the image, ability to turn the card/image, needing
the examiner to see what the person is seeing, possibility using
a layperson to help administer, etc.
5CS International Rorschach Association/Association
Rorschach Internationale Système Intégré; https://www.csira-
Andronikof, A. (2019). What future for the
Comprehensive System? Paper presented at the
3rd Congress of the Comprehensive System
International Rorschach Association
Berry, B. A. (2020). The effects of manipulating visual
processes on Rorschach performance,
Unpublished doctoral dissertation. University of
Toledo, Toledo, OH.
Blais, M. A., Norman, D. K., Quintar, B., & Herzog, D.
B. (1995). The effect of administration method:
A comparison of the Rapaport and Exner
Rorschach systems. Journal of Clinical
Psychology, 51(1), 100107.
Bornstein, R. F., Bonner, S., Kildow, A. M., & McCall,
C. M. (1997). Effects of individual versus group
test administration on Rorschach Oral
Dependency scores. Journal of Personality
Assessment, 69, 215228.
CSIRA/ARISI. (2020). CSIRA position paper on
psychological tele assessment.
Erard, R. E., & Evans, F. B. (Eds.). (2017). The
Rorschach in multimethod forensic assessment:
Conceptual foundations and practical
applications. Routledge/Taylor & Francis Group.
Exner, J. E. (1969). The Rorschach systems. New York:
Grune & Stratton.
Exner, J. E. (1974). The Rorschach: A Comprehensive
System (1st ed.). New York: Wiley.
Exner, J. E., Jr. (1995). A Rorschach workbook for the
Comprehensive System (4th ed.). Asheville, NC:
Rorschach Workshops.
Exner, J. E., Jr. (2001). A Rorschach workbook for the
Comprehensive System (5th ed.). Asheville, NC:
Rorschach Workshops.
Exner, J. E. (2003). The Rorschach: A Comprehensive
System (4th ed.). New York: Wiley.
Exner, J. E. (2007). A new US adult nonpatient sample.
Journal of Personality Assessment, 89(Suppl. 1),
Fontan, P. (2014). Approche Psychométrique du test de
Rorschach, Unpublished doctoral dissertation,
Université Paris Ouest Nanterre, Paris, France.
Fontan, P. (2019). The Future of the Rorschach test.
Journal of Japan Rorschach Society for the
Comprehensive System, 23(1), 2-8.
Fontan, P., & Smith, J.M. (2018). A Critical Review of
R-PAS, Symposium conducted at the Society for
Personality Assessment at the 2018 Society of
Personality Assessment Annual Convention,
Washington DC, USA.
Gacono, C.B. (2019). The importance of Lambda to the
generalizability of Rorschach findings reported
in the literature. SIS Journal of Projective
Psychology & Mental Health, 26(2), 104-106.
Gacono, C. B., & Evans, F. B. (Eds.). (2008). The LEA
series in personality and clinical psychology.
The handbook of forensic Rorschach
assessment. (N. Kaser-Boyd & L. A. Gacono,
Collaborators). Routledge/Taylor & Francis
Gacono, C. B., Gacono, L. A., & Evans, F. B. (2008).
The Rorschach and antisocial personality
disorder. In C. B. Gacono, F. B. Evans (Eds.) &
N. Kaser-Boyd, L. A. Gacono (Collaborators),
The LEA series in personality and clinical
psychology. The handbook of forensic
Rorschach assessment (p. 323359).
Routledge/Taylor & Francis Group.
Gacono, C.B., & Meloy, J.R. (1994). The Rorschach
assessment of aggressive and psychopathic
personalities. Hillsdale, NJ: Erlbaum.
Gacono, C. B., Meloy, J. R., & Bridges, M. R. (2008). A
Rorschach understanding of psychopaths, sexual
homicide perpetrators, and nonviolent
pedophiles. In C. Gacono, B. Evans, N. Kaser-
Boyd, & L. Gacono (Eds.), The handbook of
forensic Rorschach assessment (pp. 379402).
Mahwah, NJ: Lawrence Erlbaum.
Gacono, L. A., & Gacono, C.B. (2008). Some
considerations for the Rorschach Assessment of
forensic Psychiatric outpatients. In C. B.
Gacono, F. Evans, N. Kaser-Boyd, & L. Gacono
(Eds.), The Handbook of Forensic Rorschach
Using the R-PAS: 00
Assessment (pp. 421-444). New York, NY US:
Routledge/Taylor & Francis Group.
Gurley, J. R., Sheehan, B. L., Piechowski, L. D., &
Gray, J. (2014). The admissibility of the R-PAS
in court. Psychological Injury and Law, 7(1), 9
Hartmann, E. (2001). Rorschach administration: A
comparison of the effect of two instructions.
Journal of Personality Assessment, 76(3), 461
Hosseininasab, A., Meyer, G.J., Viglione, D.J., Mihura,
J.L., Berant, B., Resende, A.R., Reese, J., &
Mohammadi, M.R. (2019). The effect of CS
administration or an R-Optimized alternative on
RPAS variables: A meta-analysis of findings
from six studies. Journal of Personality
Assessment, 101(2), 199-212,
Khadivi, A., & Evans, F. B. (2012). The brave new
world of forensic Rorschach assessment:
Comments on the Rorschach special section.
Psychological Injury and Law, 5(2), 145149.
Kivisto, A. J., Gacono, C. B., & Medoff, D. (2013).
Does the R-PAS meet standards for forensic
use? Considerations with introducing a new
Rorschach coding system. Journal of Forensic
Psychology Practice, 13(5), 389-410.
Logid, G. M. (2018). Presentation of Rorschach Inkblot
stimuli via the iPad®: An equivalence
investigation. Unpublished doctoral dissertation,
California School of Professional Psychology,
San Diego, CA.
Margolis, J. (1992). Aggressive and borderline-level
content on the Rorschach: An exploratory study
of some proposed scoring criteria. Unpublished
doctoral dissertation, California School of
Professional Psychology, Fresno, CA.
Mattlar, C. E. (2011). The issue of an evolutionary
development of the Rorschach Comprehensive
System (RCS) versus a revolutionary change (R-
PAS). Retrieved from
McDaniel, M.A., & Nguyen, N.T. (2001). Situational
judgment tests: A review of practice and
constructs assessed. International Journal of
Selection and Assessment, 9, 103 113.
McDaniel, M. A., Hartman, N. S., Whetzel, D. L., &
Grubb, W. L., III (2007). Situational judgment
tests, response instructions, and validity: A
meta-analysis. Personnel Psychology, 60, 63-91.
Meloy, R., Acklin, M., Gacono, C., Murray, J., &
Peterson, C. (1997). Contemporary Rorschach
Interpretation. Mahwah: NJ: Lawrence Erlbaum
Meyer, G. J. (2000). On the science of Rorschach
research. Journal of Personality Assessment,
75(1), 4681.
Meyer, G. J., Erdberg, P., & Shaffer, T. W. (2007).
Toward international normative reference data
for the Comprehensive System. Journal of
Personality Assessment, 89(Suppl1), S201
Meyer, G.J., Hosseininasab, A., Viglione, D.J., Mihura,
J.L., Berant, E., Resende, A.C., & Reese, J.
(2020). The effect of CS administration or an R-
Optimized alternative on potential projective
material in Rorschach responses from six studies
and a meta-analysis of their findings. Journal of
Personality Assessment, 102 (1), 135-146. DOI:
Meyer, G. J., & Viglione, D. J. (2008). An introduction
to Rorschach assessment. In R. P. Archer & S.
R. Smith (Eds.), Personality assessment (p. 281
336). Routledge/Taylor & Francis Group.
Meyer, G. J., Viglione, D. J., Mihura, J. L., Erard, R. E.,
& Erdberg, P. (2011). Rorschach Performance
Assessment System: Administration, coding,
interpretation, and technical manual. Toledo,
OH: Rorschach Performance Assessment
Meyer, G.M., Viglione, D.J., Mihura, J.L., Erdberg, P.,
Bram, A., Giromini, L., Grønnerød, C., Kleiger,
J., Lipkind, J., de Ruiter, C., Pianowski, G., &
Vanhoyland, M. (2020). Recommendations
concerning remote administration of the
Rorschach. https://r-
Mihura, J. L. (2019). Correcting Smith et al.’s (2018)
criticisms of all Rorschach studies in Mihura,
Meyer, Dumitrascu, and Bombel (2013).
Rorschachiana, 40(2), 169-186.
Mihura, J. L., & Meyer, G. J. (Eds.). (2018). Using the
Rorschach Performance Assessment System®
(R-PAS®). The Guilford Press.
Mihura, J. L., Meyer, G. J., Dumitrascu, N., & Bombel,
G. (2013). The validity of individual Rorschach
variables: Systematic reviews and meta-analyses
00: Gacono & Smith
of the comprehensive system. Psychological
Bulletin, 139, 548605. doi:10.1037/a0029406
Pianowski, G., Meyer, G.J., de Villemor-Amaral, A.E.,
Zuanazzi, A.C., & do Nascimento, R.S.G.F.
(2019). Does the Rorschach Performance
Assessment System (R-PAS) differ from the
Comprehensive System (CS) on variables
relevant to interpretation?. Journal of Personality
Assessment, DOI:
Piotrowski, C. (1996a). The Rorschach in contemporary
forensic psychology. Psychological Reports,
78(2), 458.
Piotrowski, C. (1996b). The status of Exner’s
Comprehensive System in contemporary
research. Perceptual and Motor Skills,
82(3_suppl), 13411342.
Piotrowski, C. (1996c). Use of the Rorschach in forensic
practice. Perceptual and Motor Skills, 82,254.
Piotrowski, C. (2007). Forensic psychological testing as
a function of affiliation and organizational
setting. Organization Development Journal,
25(1), 94-98
Piotrowski, C. (2017). Rorschach research through the
lens of bibliometric analysis: Mapping
investigatory domain. SIS Journal of Projective
Psychology & Mental Health, 24(1), 34-38.
Reik, T. (1956). The search within; the inner experiences
of a psychoanalyst. Farrar, Strauss, and Cudahy.
Ritzler, B. A. (2014). Society for Personality
Assessment (SPA). Rorschach Training
Programs, Inc. Newsletter, 6(2). Retrieved from
Ritzler, B., & Sciara, A. (2009). Rorschach
Comprehensive System international norms:
Cautionary notes.
Ritzler, B., & Sciara, A. (2010). Rorschach Training
Programs comments on RPAS.
Sciara, A.D. & Ritzler, B.A. (2018). Comment on
“Addressing Issues in the Development and Use
of the Composite International Reference Values
as Rorschach Norms for Adults” (Meyer,
Shaffer, Erdberg, & Horn, 2015), Journal of
Personality Assessment, 100 (3), 343-344.
Singer, J., Hoppe, C. F., Lee, S. M., Olesen, N. W., &
Walters, M. G. (2008). Child custody litigants:
Rorschach data from a large sample. In C. B.
Gacono, F. B. Evans (Eds.) & N. Kaser-Boyd, L.
A. Gacono (Collaborators), The LEA series in
personality and clinical psychology. The
handbook of forensic Rorschach assessment (p.
445464). Routledge/Taylor & Francis Group.
Smith, B. L. (2012). Clinical interpretations using the CS
and R-PAS: Discussion. This is not a horserace:
Comparison of clinical interpretation using the
CS and R-PAS. A symposium presented at the
Society for Personality Assessment Annual
Meeting, March 16, 2012, Chicago, IL.
Smith, J.M., Gacono, C.B., & Cunliffe, T.B. (2021).
Understanding Female Offenders: Psychopathy,
Criminal Behavior, Assessment, & Treatment.
San Diego, CA: Elsevier.
Smith, J.M., Gacono, C.B., & Fontan, P., Cunliffe, T.B.,
& Andronikof, A. (2020). Understanding
Rorschach research: Using the Mihura (2019)
commentary as a reference. SIS Journal of
Projective Psychology & Mental Health, 27(2),
Smith, J.M., Gacono, C.B., Fontan, P., Taylor, E.E.,
Cunliffe, T.B., & Andronikof, A. (2018). A
scientific critique of Rorschach research:
Revisiting Exner’s Issues and Methods in
Rorschach Research (1995). Rorschachiana,
39(2), 180-203.
Whitehouse, J. (2019). The Rorschach in the 21st
century: A tablet-based administration
equivalence study, Unpublished doctoral
dissertation, William James College, Newton,
Wiener, I. (2008). Presenting and defending Rorschach
testimony. In C. B. Gacono, F. B. Evans (Eds.)
& N. Kaser-Boyd, L. A. Gacono (Collaborators),
The LEA series in personality and clinical
psychology. The handbook of forensic
Rorschach assessment (p. 121137).
Routledge/Taylor & Francis Group.
Young, M. H., Erdberg, P. S., & Justice J. (2008).
Inmates in prison psychiatric treatment: A
multimethod description. In C. B. Gacono, F.
Evans (Eds.), The handbook of forensic
Rorschach assessment (pp. 403-421). New York,
NY US: Routledge/Taylor & Francis Group.
... The PCL-R has a two-factor, fourfacet structure: the callous, selfish, remorseless use of others and a chronically unstable and antisocial lifestyle factors, and the Interpersonal, Affective, Impulsive Lifestyle, and Antisocial Behavior facets (Hare, 2003). A total PCL-R score of 30 or higher is needed for categorical designations of psychopathy for both males and females to increase the likelihood that psychopaths are present in the sample (Cunliffe et al., 2016;Gacono, 2016;Gacono & Smith, 2021;Hare, 2003;Neumann et al., 2016;Smith et al., 2021b). ...
... Underneath their respective personality styles, psychopathic individuals display similar levels of personality organization (borderline or psychotic; Gacono & Meloy, 1994;Kernberg, 1967Kernberg, , 1975Kernberg, , 1976Smith et al., 2021b) resting at a paranoid position (Klein, 1946;Mahler, 1975;Gacono & Smith, 2021;Smith et al., 2021b). When the defensive purposes of pathological narcissism or malignant hysteria fail to maintain homeostasis, the psychopath's paranoid style becomes behaviorally evident (Gacono & Meloy, 1994). ...
... SD = 7.62, range = 14-52; see Table 1) while Lambda were slightly lower (MP M = 0.92, SD = 0.62, range = 0.13-2.50). This difference in Lambda represents a gender-based trend consistent with the psychopathic women producing significantly more CS determinants, indicative of greater affectivity (Gacono, 2019), or pseudo-affect (Gacono & Smith, 2021;Smith et al., 2021b). ...
Full-text available
The psychopathic personality is organized at a borderline level of personality. Additionally, while male psychopaths are considered to manifest pathological narcissism, female psychopaths are best understood in terms of malignant hysteria. Using Kernberg's three components of borderline personality functioning (identity diffusion, reliance on primitive defenses, and transient lapses in reality testing),
Full-text available
Managing the incarcerated population is the primary task within correctional settings. Using psychological assessment to predict institutional behavior, the psychologist has a unique set of skills essential to the management of prisoners. PCL-R, PAI, and Rorschach data were compared with institutional infractions (total, physical, verbal, non-aggressive) among 126 incarcerated women. Multiple binary logistic regression analyses were used which found significant correlations between PCL-R total score, PAI scales (BOR, ANT, VPI), and Rorschach variables (ROD, EGOI, TCI, AgPot, AgPast, SumV, SumC’, MOR) with total, verbal, physical, and nonviolent incident reports. Each of these measures adds incrementally to the assessment and understanding of institutional misbehavior for incarcerated women. Clinical implications of the findings were presented.
Full-text available
This article is part I of a two part series. It discusses the interpretative importance of anchoring Rorschach data within a larger body of comparative (both nonpatient and clinical) group data.
Long before psychology, bias has existed in science. From the beginning, concerns have been raised about the reliability, validity, and accuracy of social science research (Meehl, 1954). In this chapter, we define and discuss the origins of bias and how it can erode the scientific method. We focus specifically on bias in psychological research, theory, assessment, and treatment. We discuss the range of common misconceptions and misinformation that permeates the female offender literature. Finally, we conclude with ten myths about female offenders and offer guidelines for identifying bias and how to avoid it.
Historically, the cornerstone of the psychologist’s identity rested on providing competent in-depth psychological assessment (Rapaport, Gill, & Schafer, 1946). The ability to utilize a battery of assessment methods to elucidate complex issues makes the psychologist unique among other mental health professionals. Recent trends, however, have tarnished that cornerstone. Not surprisingly, the movement away from proficiency in psychological assessment has led to a decline in the need for psychologists. In this chapter, we discuss these harmful trends, define psychological assessment, offer a model for assessing female offenders, and provide examples of how record review, clinical interview, the Hare Psychopathy Checklist-Revised (PCL-R), Personality Assessment Inventory (PAI), and Rorschach Inkblot test can be useful with female offenders. We discuss the interpersonal aspects of the assessment process, evaluate gender specific patterns for several PCL-R criteria (also see Appendices A & B), and provide caveats for assessing female offenders. We conclude with a case study.
In this chapter, we provide a theoretical and empirically based understanding of antisocial and psychopathic women. We begin by clarifying the differences between psychopathy, sociopathy, and ASPD, and then provide a historical perspective of hysteria. While the underlying personality of the female psychopath is paranoid, malignant hysteria is their predominant personality style (Gacono & Meloy, 1994). Overviews of the Hare Psychopathy Checklist-Revised (PCL-R), Personality Assessment Inventory (PAI), and Rorschach are offered as a refresher for those experienced clinicians and as a resource for those that are not. Finally, we present group PAI and Rorschach data (also Trauma Symptom Inventory-2 [TSI-2]) for 337 female offenders including subsets of psychopathic (N = 124) and non-psychopathic (N = 57) females. We make note of the differences between female and male psychopaths.
Full-text available
The histories of incarcerated women exhibit a multitude of personality issues including psychopathy, trauma, and interpersonal dependency. Two studies were undertaken to better understand these issues with psychopathic (PCL-R ≥ 30; N = 115) and non-psychopathic (PCL-R ≤ 24; N = 53) women incarcerated for drug, theft, fraud, violence, and sex offenses. In the first study, trauma symptoms were compared on Rorschach variables, TSI-2, and PAI scales. The female psychopathic group experienced more problems related to intrusive experiences and dissociation (TSI-2, Rorschach). In the second study, interpersonal dependency was also examined with the PAI, TSI-2, and Rorschach. The psychopathic females had higher rates of interpersonal dependency (PAI, Rorschach). Based on our findings we discuss the relationship between trauma and interpersonal dependency and the meaning of these testing variables and concepts within the personality functioning of these antisocial women.
Full-text available
Between 1980 and 2018, the number of incarcerated women increased by more than 750%, rising from a total of 26,378 to 225,345. Managing and treating these women can be challenging. Understanding Female Offenders: Psychopathy, Criminal Behavior, Assessment, and Treatment provides a guide for meeting these challenges. In addition to providing a sound conceptualization for both psychopathic and non-psychopathic women, the text includes suggestions for interviewing, assessing, managing, and treating them. Throughout this text, clinical caveats and case examples enliven the application of research to practice. Consistent with the scientist-practitioner model, clinical experience and research guide the authors’ formulations which are translated for direct application. Additionally, guidelines are provided for identifying bias and interpreting the female offender literature. This is the first in-depth resource of its kind.
Full-text available
In several articles, my colleagues and I have addressed concerns regarding methodological issues with the recent RIM research (see Cunliffe et al., 2012; Gacono, Loving, &Bodholdt, 2001; Smith et. al., 2018). In many studies, the impact of these issues is frequently hidden, masked in a description of statistical procedures and shrouded by the umbrella of a meta-analysis. Findings from these flawed studies influence conclusions that may appear "controversial" but are in actuality an artifact of the individual studies' inadequate design. Alarmingly, such method-related practices negatively impact the scholarly perception of the Rorschach and are frequently cited by editors for rejecting submitted Rorschach studies for publication, based largely on the presence of "controversy" which does not exist. In this brief commentary, the importance of Lambda to the generalizability of Rorschach findings is discussed.
Full-text available
Mihura’s (2019) recent commentary on Smith et al.’s (2018) article, “A Scientific Critique of Rorschach Research: Revisiting Exner’s Issues and Methods in Rorschach Research (1995)” raised several issues surrounding our internal validity criteria and our approach to validating Rorschach research. Mihura also conducted additional statistical analyses that failed to address important, critical issues. In this article, we further clarify the importance of refining internal validity criteria for Rorschach research and Rorschach meta-analytic studies (our 2018 article). We offer this information and analysis to guide the Rorschach consumer toward a better understanding of how to assess the validity of Rorschach data and empirical findings. Criticisms of our criteria for evaluating Rorschach research (inter-rater reliability, IQ/Education level, Rorschach Responses, Lambda/F%, sample size) and our recommendation to include descriptive data for critical variables in all published Rorschach studies contradict accepted standards for Rorschach research. Mihura stated that “the associations between the Rorschach and self-report measures were not used to determine the validity of Rorschach variables” (p. 171) in her meta-analyses. Consequently, we applied our methodological criteria to only Mihura’s externally assessed criteria studies (removing self-report studies; 67%) and found that 91% had three or more problems related to internal validity (13 had counter-intuitive findings). In addition, other central issues related to meta-analyses, application/validation studies, and counter-intuitive findings regarding the Rorschach were discussed.
Full-text available
Exner’s (1995a) Issues and Methods in Rorschach Research provided a standard of care for conducting Rorschach research; however, the extent to which studies have followed these guidelines has not been examined. Similarly, meta-analytic approaches have been used to comment on the validity of Exner’s Comprehensive System (CS) variables without an evaluation as to the extent that individual studies have conformed to the proposed methodological criteria (Exner, 1995a; Gacono, Loving, & Bodholdt, 2001). In this article, 210 studies cited in recent meta-analyses by Mihura, Meyer, Dumitrascu, and Bombel (2013) were examined. The studies were analyzed in terms of being research on the Rorschach versus research with the Rorschach and whether they met the threshold of validity/generalizability related to specific Rorschach criteria. Only 104 of the 210 (49.5%) studies were research on the Rorschach and none met all five Rorschach criteria assessed. Trends and the need for more stringent methods when conducting Rorschach research were presented.
Smith et al. (2018) describe their article as “an evaluation as to the extent that individual studies have conformed to [ Exner’s (1995a) ] proposed methodological criteria” (Abstract). However, the authors did not conduct analyses to compare research before and after Exner (1995a) in order to assess its impact nor were the set of criteria they used Exner’s. Instead, they critiqued the individual studies in Mihura and colleagues’ (2013) meta-analyses, declaring all methodologically unsound (including Exner’s). They conjectured that Mihura et al. omitted studies with less “methodological bias” that would have provided more support for Rorschach validity. I explain why most of the criteria they use to criticize the studies’ methodology are not sound. But to directly test their hypotheses, I requested their ratings of study methodology. Findings from studies they rated as having more methodological “issues” (e.g., not reporting IQ or Lambda range) or as being “application studies” – which they said should be excluded – were not less supportive of Rorschach validity as they assumed would be the case. The small effect size associations ( r < |.10|) were also in the opposite direction of which Smith et al. argued to be true, indicating that the criteria by which they evaluated other researchers’ studies were not sound. Our findings do indicate that researchers are responding to the one criterion that is clearly stated in Exner (1995a) , which is Weiner’s (1991) recommendation to report interrater reliability; before 1991, 12% of studies reported interrater reliability, which afterward jumped to 78.4%. Other claims in the article by Smith et al. are also addressed.
We examined the impact of the changes in administration and coding introduced by the Rorschach Performance Assessment System (R-PAS) relative to the Comprehensive System (CS) on the Rorschach response process, as manifested in variables relevant to interpretation. We also examined the efficiency of each system to obtain protocols in an optimal range of responses (R) for interpretation. As hypothesized, when comparing 50 CS and 50 R-PAS nonpatient protocols, R-PAS produced many more protocols in the optimal R range (18-27) than the CS (78% vs. 24%) and it eliminated the need for re-administration, which was required for five CS protocols. As expected, R was less variable with R-PAS, as were two variables derived from it, R8910% and Complexity. In addition, as expected because of different Form Quality tables, R-PAS showed notably fewer and less variable perceptual distortions than the CS, and an increase in more conventional perceptions. The other 58 variables showed no reliable differences in means or standard deviations, though modest power precluded definitive inferences about equivalence. Overall, our results support previous findings about the benefit of R-PAS to obtain protocols in an optimal range for interpretation, while keeping the core manifestations of the response process unchanged.
A critical review of R-PAS. Symposium conducted at the Society for Personality Assessment 2018 Annual Convention (SPA 2018), Washington DC, USA
The Rorschach Performance Assessment System (R-PAS; Meyer, Viglione, Mihura, Erard, & Erdberg, 2011 Meyer, G. J., Viglione, D. J., Mihura, J. L., Erard, R. E., & Erdberg, P. (2011). Rorschach Performance Assessment System: Administration, coding, interpretation, and technical manual. Toledo, OH: Rorschach Performance Assessment System. [Google Scholar]) introduced R-optimized administration to reduce variability in the number of Responses (R). We provide new data from six studies of participants randomly assigned to receive a version of this method or Comprehensive System (CS; Exner, 2003 Exner, J. E. (2003). The Rorschach: A Comprehensive System. Vol. 1. Basic foundations (4th ed.). Hoboken, NJ: Wiley. [Google Scholar]) administration. We examine how administration methods affect 3 types of codes most likely to contain potential projective material and the frequency of these codes for the 1st, 2nd, 3rd, 4th, or last response to a card (R in Card). In a meta-analytic summary, we found 37% of responses have 1 type of code, 19% have 2 types, and 3% have all 3 types, with stable proportions across responses within cards. Importantly, administration method had no impact on potential projective variable means. Differential skew across samples made variability harder to interpret. Initial results suggesting differences in 3 of the 18 specific Type by R in Card pairs did not follow a coherent pattern and disappeared when using raw counts from all participants. Overall, data do not support concerns that R-optimized administration might alter potential projective processes, or make potentially “signature” last responses to the card any different in R-PAS than the CS.