No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Does the Rorschach Performance Assessment System (R-PAS) Differ from the Comprehensive System (CS) on Variables Relevant to Interpretation?
Journal of Personality Assessment
Abstract
We examined the impact of the changes in administration and coding introduced by the Rorschach Performance Assessment System (R-PAS) relative to the Comprehensive System (CS) on the Rorschach response process, as manifested in variables relevant to interpretation. We also examined the efficiency of each system to obtain protocols in an optimal range of responses (R) for interpretation. As hypothesized, when comparing 50 CS and 50 R-PAS nonpatient protocols, R-PAS produced many more protocols in the optimal R range (18-27) than the CS (78% vs. 24%) and it eliminated the need for re-administration, which was required for five CS protocols. As expected, R was less variable with R-PAS, as were two variables derived from it, R8910% and Complexity. In addition, as expected because of different Form Quality tables, R-PAS showed notably fewer and less variable perceptual distortions than the CS, and an increase in more conventional perceptions. The other 58 variables showed no reliable differences in means or standard deviations, though modest power precluded definitive inferences about equivalence. Overall, our results support previous findings about the benefit of R-PAS to obtain protocols in an optimal range for interpretation, while keeping the core manifestations of the response process unchanged.
... This is because R-PAS administration dramatically reduces variability in R relative to the CS. Because R is part of Complexity, Complexity also is less variable in R-PAS than the CS (Meyer et al., 2011;Pianowski et al., 2016Pianowski et al., , 2021. With R and Complexity more variable in the CS than R-PAS, the FUPC also is larger, such that any problems that may exist from Complexity or the FUPC actually affect the CS more than they affect R-PAS. ...
... These norms consist of 640 adult protocols modeled for R-Optimized administration. That mode of administration greatly improves the proportion of protocols in an optimal range for interpretation and produces protocols with scores that are more valid than their CS counterparts (Dean et al., 2007;Pianowski et al., 2021Pianowski et al., , 2023. (47), Spain (54), and the United States (137 from three subsamples). ...
... It is also what led the R-PAS authors to take even more notable steps to manage responding, which allowed them to attain the goal of reduced variability in R (Hosseininasab et al., 2019;Meyer et al., 2011). Reducing variability in R has had the expected benefit of enhancing the utility and validity of R-PAS scores over their CS counterparts (Pianowski et al., 2021(Pianowski et al., , 2023. In addition, it has had the unanticipated benefit of providing two helpful new variables for interpretation related to under-and overproductive behavior (i.e., Prompts and Pulls). ...
This article serves three goals. First, I review complexity in Rorschach responding. As a construct, complexity illuminates ways people differentially register experiences, which produces distinct patterns of expressed behavior when completing the task. Rorschach first described this dimension, creating novel terminology for it, and it was central to Rapaport, Gill, and Schafer’s system and Schachtel’s classic text. As a scored variable, Viglione and Meyer defined it when they were brought together by Exner to work on advancing the Comprehensive System (CS) through his Rorschach Research Council; later it was adopted in the Rorschach Performance Assessment System (R-PAS). Second, I review early factor analytic research identifying complexity and provide new data to document how Complexity as a scored variable is an excellent index of the first unrotated principal component when factoring individually assigned Rorschach scores. Third, I document a number of assertions published about Complexity by Fontan and Andronikof that are incorrect and misleading. I correct those assertions by means of explanation and also statistical results from two data sets. I close by offering 10 basic conclusions about complexity.
... Research comparing CS and R-PAS administration among nonpatients (Pianowski et al., 2021), experimental studies of participants receiving CS administration versus early versions of R-PAS administration (Hosseininasab et al., 2019;Meyer et al., 2020), and statistical modeling of R to convert its distribution in CS administration to fit its distribution in R-PAS administration Pianowski et al., 2016aPianowski et al., , 2016b all provide similar conclusions. R-PAS administration achieves its aim of slightly increasing the average number of responses in a protocol while dramatically decreasing its variability across respondents. ...
... Thus, FQ scores obtained by coding protocols using R-PAS guidelines are not comparable with FQ scores obtained by coding protocols using CS guidelines. Research with patients and nonpatients Pianowski et al., 2021) shows that relative to the CS, the R-PAS guidelines produce more responses classified as having conventional or ordinary form quality (FQo) and adequate but unusual form quality (FQu) and many fewer responses classified as having distorted form quality (FQ-; see also Su et al., 2015). FQ scores provide data on the adequacy and conventionality of one's perceptions and thus are reality-testing measures. ...
... Building on this foundation, the current study contributes to the literature on the differential validity of R-PAS and CS administration procedures, coding standards, and scoring algorithms. We do so by extending Pianowski et al.'s (2021) collaborative research effort, which brought together different teams of R-PAS and CS researchers to compare the relative merits of these systems with each other for the purpose of generating normative standards. Pianowski et al. (2021) focused specifically on the potential impact of R-Optimized Administration on nonpatient normative reference values, studying 100 nonpatients, 50 assessed by each system. ...
Building on the comparative nonpatient study of Pianowski et al., we examine data from the Rorschach Performance Assessment System (R-PAS) and Comprehensive System (CS) in 100 nonpatients and 100 patients, 50 of each per system. Replicating their results but now in a patient sample, R-PAS produced more patient protocols having an optimal number of responses (R) for interpretation and eliminated the need for readministration due to low R. The R-PAS protocols were also much less variable in R, despite having about 2.5 more responses. Extending their results, we document that the primary markers of psychopathology in each system validly differentiate patients from nonpatients. However, R-PAS produced stronger effects. Finally, Complexity added to the valid discrimination of patients from nonpatients just for R-PAS, with patients producing less complex and rich records. The more erratic variability in R for the CS produced larger Complexity standard deviations (SDs) that obscured these genuine differences in people. We discuss implications for research and applied practice, along with directions for future research.
... R-PAS sets 16 responses as a minimum and limits evaluees to no more than four responses per card. Data indicate R-PAS administration achieves its aim to produce an average range for R of 18 to 27 responses (Hosseininasab et al., 2019;Meyer et al., 2011;Pianowski et al., 2021). ...
... However, research shows that CS and R-PAS administration lead to similar norms for variables shared by both systems (Hosseininasab et al., 2019;Meyer et al., 2020). A non-randomized study of 100 Brazilian nonpatients by Pianowski et al. (2021) also found that for corresponding variables, the protocols collected and coded using CS guidelines had means and standard deviations that were similar to those obtained from protocols collected and coded using R-PAS guidelines. These studies address concerns about R-PAS changes to administration and suggest some support for the generalizability of CS research findings to corresponding R-PAS variables. ...
... It took the R-PAS evaluators 20 protocols to reach this standard, but it took the CS evaluators 30 protocols. Pianowski et al. (2021) relied on evaluators experienced with either R-PAS or the CS and did not ask them to calibrate to attain a minimal level of agreement. They found the R-PAS evaluators produced significantly better coding consistency (M ¼ .80, ...
The special issue editors selected us to form an "adversarial collaboration" because our publications and teaching encompass both supportive and critical attitudes toward the Rorschach and its recently developed system for use, the Rorschach Performance Assessment System (R-PAS). We reviewed the research literature and case law to determine if the Rorschach and specifically R-PAS meet legal standards for admissibility in court. We included evidence on norms, reliability, validity, utility, general acceptance, forensic evaluator use, and response style assessment, as well as United States and selected European case law addressing challenges to mental examination motions, admissibility, and weight. Compared to other psychological tests, the Rorschach is not challenged at unusually high rates. Although the recently introduced R-PAS is not widely refer-enced in case law, evidence suggests that information from it is likely to be ruled admissible when used by a competent evaluator and selected variables yield scores that are sufficiently reliable and valid to evaluate psychological processes that inform functional psycholegal capacities. We identify effective and ethical but also inappropriate uses (e.g., psychological profiling) of R-PAS in criminal, civil, juvenile, and family court. We recommend specific research to clarify important aspects of R-PAS and advance its utility in forensic mental health assessment.
... Finally, the most recent study (Pianowski et al., 2019) that conducted protocol-level IR analyses involved 22 adults (56% female; Age M = 29.08, SD = 12.45) from clinical and nonclinical settings. ...
... Because Viglione et al. (2012), Pignolo et al. (2017), and Pianowski et al. (2019) focused on protocol-level scores, they cannot be compared with the response-level analyses conducted by Kivisalu et al. (2016). However, as a general idea of agreement, mean kappa was considered good for Kivisalu et al. (2016), and the mean ICCs for Viglione et al. (2012), Pignolo et al. (2017), andPianowski et al. (2019) were considered excellent. ...
... Because Viglione et al. (2012), Pignolo et al. (2017), and Pianowski et al. (2019) focused on protocol-level scores, they cannot be compared with the response-level analyses conducted by Kivisalu et al. (2016). However, as a general idea of agreement, mean kappa was considered good for Kivisalu et al. (2016), and the mean ICCs for Viglione et al. (2012), Pignolo et al. (2017), andPianowski et al. (2019) were considered excellent. ...
We examine interrater reliability for scoring the Rorschach Performance Assessment System (R-PAS) in a sample of 89 adolescents ( M Age = 13.2, SD = 1.01) from Brazil using exact agreement intraclass correlations coefficient (ICCs) for the 60 protocol-level scores that are the focus of interpretation. The first author completed or reviewed all of the primary coding, and seven R-PAS proficient psychologists trained at different sites independently produced secondary coding. Overall, excellent agreement was found ( M ICC = 0.89; SD = 0.09). When averaged across this study and three other comparison studies, stronger reliability was present, in general, for commonly coded variables ( M = 0.87) as opposed to rare or infrequent variables ( M = 0.78). In addition, 78.3% of the variables showed excellent interrater reliability and an additional 20.0% had good reliability. The results also showed that the ICCs for most variables had low variability across studies, suggesting clear coding guidelines. However, variables with higher ICC variability across studies indicated domains where it would be desirable to expand guidelines with more detailed parameters. Overall, the findings indicate excellent interrater reliability for the great majority of codes and present solid grounds for future research on interrater reliability with R-PAS.
... Statistics and meta-analyses have been utilized to justify an absence of meaningful differences among Rorschach administration procedures (Hosseininasab et al., 2019;Pianowski, Meyer, de Villemor-Amaral, Zuanazzi, & do Nascimento, 2019). These studies have found some statistical differences which were largely ignored when statistical comparisons of select variables did not produce significantly different quantities. ...
... In response to this lengthy introduction (instructions) fewer brief protocols were produced when compared to the standard Exner instructions ("What might this be?"); however, she found no significant group Rorschach variable differences between the two instructions. Pianowski et al. (2019) looked at two groups of non-patient protocols and found no differences in CS Rorschach variables when administering protocols with the CS (N = 50) or R-PAS (N = 50) administration methods. In this study, 53 statistical analyses were done between Rorschach variables from an R-PAS and CS administration. ...
It is essential to understand that CS validity research does not translated directly to the R-PAS. In this article we dicuss essential issues to consider prior to using the R-PAS in an applied context.
... Em um estudo sobre a inteligência emocional, Miguel et al., (2017) verificaram a baixa incidência de cores em vários protocolos e buscaram realizar investigação com amostras dentro de uma faixa otimizada, ou seja, tiveram como critério a produção de duas ou mais respostas de cor por protocolo. Diante desses achados, estudos passaram a considerar o aprimoramento nos instrumentos de manchas de tinta e instruções otimizadas baseadas no Rorschach R-PAS (Meyer, 2017;Pianowski et al., 2019) e motivaram a busca de aperfeiçoamento do teste de ZSC utilizando uma nova forma de aplicação denominada de R-Otimizada com o objetivo de conseguir protocolos com, no mínimo, nove e, no máximo, 15 respostas (Gonçalves et al., 2019;Gonçalves & Villemor-Amaral, 2020;Villemor-Amaral et al., 2016;Villemor-Amaral & Gomes, 2020). Tais estudos, ainda que incipientes, evidenciaram as propriedades psicométricas do teste de ZSC e buscaram assegurar avanços com o uso do instrumento de forma válida e confiável. ...
The psychometric properties of a psychological test instrument seek to ensure that it is used in a scientifically secure way. This study aimed to gather empirical evidence on the psychometric properties, reliability estimates, and validity evidence, for the Zulliger test in the Comprehensive System (ZCS) used in adult investigations. This research followed the PRISMA guidelines and was based on articles published between the years 2009 to December 2022. The search was performed in the SciELO, PePSIC, IndexPsi Articles, Lilacs, PsycNET, and Redalyc DOAJ databases. The searches produced 24 publications. The analyzed studies show that the ZSC has psychometric properties that support the evaluation of psychic and personality dynamics in adults, however, they suggest the improvement in the ZSC using instructions with optimized responses. The review presents an overview of the historical and current situation of the ZCS test and demonstrates advances and limitations in research with the instrument.
... That is, if there was a difference in the Complexity variable exclusively due to the number of studied years, the groups would not differ due to the statistical model used. Thirdly, this study followed Exner's Comprehensive System methodology: although there are similarities between this method and the R-PAS in many of the relevant variables for interpretation, Complexity is a variable that proved to be different among systems precisely because it depends on the number of responses offered in each protocol (Pianowski et al., 2021). Despite that, a recent meta-analysis argued that it is possible that data obtained with SC can be generalized to R-PAS (Hosseininasab et al., 2019). ...
Objective This investigation presents a comparison among students from public and private schools in specific variables of the Rorschach test, Comprehensive System. Method One hundred eighteen adolescents from the city of São Paulo participated in the study, aged 13-17 years, from public (N = 49) and private schools (N = 69). Mixed linear effects models were used to test the effect of school, controlled by sex, years of education (fixed effects), and evaluator (random effect). Results Private schools students presented higher values of R (p = 0.015, d = 0.47) and Complexity scores (p = 0.007, d = 0.53). After inserting Complexity as a fixed factor in the model, the only variable that remained significant was Xu% (p = 0.008, d = -0.52), lower in private schools students. Conclusion The small number of differences found in this study indicates, at least for this age range, a trend to equivalent performance when comparing private and public schools. Even though the Rorschach is a method of personality assessment, it is clear that it can also signalize information related to each group's personal experiences.
... As a result, some authors raised concerns that evidence gathered using the CS may not generalize to the R-PAS (e.g., Khadivi & Evans, 2012;Kivisto et al., 2013), especially as other differences exist in the two systems. More recently, summaries across multiple samples suggest the two forms of administration have little effect on Rorschach variable scores except for variables directly reflecting the number of responses, while R-Optimized tends to result in more efficient administration and less variability in the number of responses (Hosseininasab et al., 2019;Pianowski et al., 2021). That said, in the following sections we focus on results using the more defensible R-PAS but indicate when results came from the CS instead. ...
... From the first development of the test in 1921 to the present days, various systems and methods of administration and coding have been developed. These include the Exner Comprehensive System (CS) (Exner, 2003;Exner et al., 2022;Meyer et al., 2002) and the Rorschach Performance Assessment System (R-PAS) (Meyer, 2011;Meyer and Eblin, 2012;Pianowski et al., 2021). From the earliest days, exponents of different Rorschach interpretation methodologies have fueled a debate about several aspects of the test. ...
The Rorschach inkblot test allows access to psychological processes that usually do not emerge in self-report measures and it has been widely used in clinical psychological and psychiatric settings. Recordings of brain activity during the administration of the Rorschach inkblots test could provide information on neural correlates of the underlying perceptual-cognitive processing and potentially identify neuroimaging markers of psychopathology risk. The present paper offers a systematization of the available literature on the Rorschach inkblot test and neuroimaging research. The 13 selected studies had been conducted with healthy participants and using fMRI, EEG, and fNIRS to investigate the neural underpinnings of Rorschach inkblot test responses. The neural processes underlying the visual, social, and emotional processes described by the included papers are systematically summarized. Research on the neural correlates of the Rorschach inkblot test is promising and would further benefit from studies on clinical populations, broader samples, and younger age groups.
... Recently, Pianowski et al. (2021) analyzed the impact of using R-PAS compared to the CS system in two nonpatient samples. They found higher interrater reliability for R-PAS protocols (88.5% good or excellent) than for the CS protocols (73.1% good or excellent). ...
Unlabelled:
Rorschach and self-report instruments represent methodologically different types of assessment, which together may yield incremental information about the test-taker. There is little evidence on whether and when results from these methods converge.
Objective:
To examine possible convergences between Rorschach trauma-related personality variables and self-reported variables.
Method:
Before and after psychotherapy 22 traumatized adult refugee patients were assessed with the Rorschach Performance Assessment System (R-PAS), symptom checklists of posttraumatic stress, anxiety and depression, and a quality of life questionnaire. Correlational analyses between eight R-PAS variables and 10 self-reported variables were performed.
Results:
The findings showed inconsistent and nonsignificant correlations pretherapy. Posttherapy, however, all R-PAS variables except Complexity correlated positively with symptoms of mental disorder, and negatively with the quality of life variables, as predicted. The R-PAS variables Mutuality of Autonomy-Pathology, Poor Human Representation, Critical Content, and Form Quality-minus%, converged significantly with most of the self-reported variables, with medium to large correlations.
Conclusion:
The finding of convergence only after psychotherapy, may tentatively suggest greater self-knowledge and internal consistency through the therapy experience, and increased trust and self-disclosure through the repeated meetings with the researchers. The findings represent a promising contribution to a cumulative validation process of convergence between Rorschach and self-report data.
... For the future research, best results may be obtained by controlling R during the test administration, as this would drastically reduce any distortion of the results due to transformation. Pianowski et al. (2021) demonstrated through a comparison of scores obtained using both RCS and the new Rorschach Performance Assessment System (R-PAS), that an administration procedure to reduce R variability had little impact on scores. ...
Objective: The present study examines the validity of 11 new Holtzman Inkblot Technique indices. These were chosen from Exner’s Comprehensive System (RCS) indices using two criteria: first, they had to be valid according to meta-analysis, and second, they must be computed using the HIT standard scoring system.
Methods: Both techniques were administrated with a retest interval from 1 to 7days to a sample of 139 subjects (63 males and 76 females) from the general population. The validity of the new indices was studied through Pearson correlation (r) with the corresponding RCS indices.
Results: Nine of the 11 new indices (R-HIT, F%-HIT, M-HIT, m-HIT, C'-HIT, Blends-HIT, PureH-HIT, DQ+HIT, and X-%-HIT) showed significant correlations with Rorschach scales, confirming our hypotheses. The correlation ranged from a minimum of 0.144 to a maximum of 0.414.
Conclusions: The results provide support for the validity of the new HIT indices and have important implications for both clinical and research fields.
... The following case example illustrates how the R-PAS coding system was used in a criminal responsibility evaluation. A number of studies and a meta-analysis have demonstrated that R-PAS keeps the response process of the Rorschach task, as reflected in coded variables, similar to that found in the CS, while optimizing interpretability by controlling R (Hosseininasab et al., 2019;Pianowski et al., 2021). ...
Over the years, a significant number of Rorschach studies have been conducted with forensic adult and adolescent samples, partly motivated by the use of the test in forensic psychological evaluations. Could the Rorschach, as a performance-based personality assessment tool, provide unique information that is not as vulnerable to distortion on the part of the examinee as self-report measures are? This article provides a review of Rorschach studies on relevant Rorschach variables, including those with different forensic samples. Empirical findings are mixed; there is not a one-on-one relationship between certain Rorschach variables and forensically relevant traits, such as psychopathy or hostility. This does not mean the Rorschach cannot provide useful information in answering psychological questions before the court. A case illustration of a male college student, who committed a (first) violent offense, illustrates the unique contribution of the Rorschach for understanding the psychological dynamics behind a violent act that was seemingly out of character.
... Regarding the two techniques used in this study, about the Rorschach test, it is possible to verify a large number of studies, which gives the technique the accumulation of favorable evidence as well as the constant development of new interpretations, which can be verified with the recent systematization performed by Meyer, Viglione, Mihura, Erard and Erdberg (2011), proposing the Rorschach Perfomance Assessment System (R-PAS), with the objective of improving the use of some variables used as well as optimizing the application (Mihura & Meyer, 2018;Pianowski, Meyer, Villemor-Amaral, Zuanazzi, & Nascimento, 2019). ...
Incremental validity indicates how much a measure can add prevision to a criterion, more than what can be previewed by other sources of data. In other words, it means how an instrument can complement and aid on information comprehension derived from another. The objective of the study was to verify evidence of incremental validity between the Wartegg and the Rorschach tests (R-PAS). A total of 40 subjects with ages varying between 21 to 70 years participated, divided into two groups, one composed by schizophrenia diagnosis and another, by subjects with a history of psychiatric diseases. Everybody responded to the Rorschach and Wartegg tests. The results indicated predictive capacity among the instruments of 75% for the variable Formal Quality, 98% for Movement and 100% for Content. New studies are suggested about validity evidences with larger samples as well as the analysis of other variables, not explored in this study.
... As a result, some authors raised concerns that evidence gathered using the CS may not generalize to the R-PAS (e.g., Khadivi & Evans, 2012;Kivisto et al., 2013), especially as other differences exist in the two systems. More recently, summaries across multiple samples suggest the two forms of administration have little effect on Rorschach variable scores except for variables directly reflecting the number of responses, while R-Optimized tends to result in more efficient administration and less variability in the number of responses (Hosseininasab et al., 2019;Pianowski et al., 2021). That said, in the following sections we focus on results using the more defensible R-PAS but indicate when results came from the CS instead. ...
• The term projective tests is often used to encompass a variety of procedures that allow the target individual to provide free-form responses to ambiguous stimuli. The participant’s responses are thought to be sensitive to implicit processes, and consequently they may be somewhat resistant to efforts at misrepresentation. This chapter summarizes the current status of projective instruments as scientific instruments. The first section offers a conceptual analysis of the nature of projective assessment. Drawing on recent discussions of projective assessment and comparisons with other psychological measurement methods, it is suggested that applying both the word projective and the word test to these instruments is problematic and probably should be discontinued. Current evidence on each of three projective instruments—the Rorschach, Thematic Apperception Test, and figure drawings—is reviewed. Although other projective instruments are used in clinical assessment, particularly various forms of Incomplete Sentences Blank, these three techniques are the most extensively researched and effectively reflect the current status of projective instruments. (PsycINFO Database Record (c) 2015 APA, all rights reserved)
• The term projective tests is often used to encompass a variety of procedures that allow the target individual to provide free-form responses to ambiguous stimuli. The participant’s responses are thought to be sensitive to implicit processes, and consequently they may be somewhat resistant to efforts at misrepresentation. This chapter summarizes the current status of projective instruments as scientific instruments. The first section offers a conceptual analysis of the nature of projective assessment. Drawing on recent discussions of projective assessment and comparisons with other psychological measurement methods, it is suggested that applying both the word projective and the word test to these instruments is problematic and probably should be discontinued. Current evidence on each of three projective instruments—the Rorschach, Thematic Apperception Test, and figure drawings—is reviewed. Although other projective instruments are used in clinical assessment, particularly various forms of Incomplete Sentences Blank, these three techniques are the most extensively researched and effectively reflect the current status of projective instruments. (PsycINFO Database Record (c) 2015 APA, all rights reserved)
University students are particularly susceptible to suicidal ideation and behavior due to issues inherent to this vital lives' moment. The Rorschach test can help to understand these students' suffering. The objective was to evaluate perception, thinking, stress, and distress in the Rorschach test domains in university students attended at a public university mental health service, comparing students with suicidal ideation with those without it. A total of 36 students aged 18 or over were assessed. The instruments were: Rorschach Performance Assessment System (R-PAS), Columbia Suicide Severity Rating Scale (C-SSRS) and Self Report Questionnaire (SRQ-20). Two analyses were performed using Wilcoxon Mann-Whitney test, according to the presence of suicidal ideation in the last 30 days (SRQ-20) and in the last six months (C-SSRS). Statistically significant differences were found in the two analyses, indicative of greater perceptual distortion in students without suicidal ideation and of stress and distress in students with ideation.
A lo largo de cien años de investigación científica, el Test de Rorschach ha suscitado debates sobre los contextos en los que se utiliza, objetos de estudio, metodología de interpretación, tipos de validez que le son aplicables y cómo estimar la calidad de la investigación que lo sustenta. Esta investigación lleva a cabo un análisis exploratorio a 86 estudios científicos con Rorschach de los últimos 10 años realizados en Hispanoamérica y España, describiendo categorías y dimensiones comunes. Además, se discute y concluye que: 1) existe una activa producción científica en la región; 2) en la región conviven diversas metodologías de análisis del protocolo; 3) los objetos de investigación son de carácter clínico, pero también psicosocial; 4) se investiga desde una perspectiva idiográfica y no solo nomotética, y 5) existe una serie de estudios que participan en la discusión sobre la validez del método.
In this chapter, we provide a theoretical and empirically based understanding of antisocial and psychopathic women. We begin by clarifying the differences between psychopathy, sociopathy, and ASPD, and then provide a historical perspective of hysteria. While the underlying personality of the female psychopath is paranoid, malignant hysteria is their predominant personality style (Gacono & Meloy, 1994). Overviews of the Hare Psychopathy Checklist-Revised (PCL-R), Personality Assessment Inventory (PAI), and Rorschach are offered as a refresher for those experienced clinicians and as a resource for those that are not. Finally, we present group PAI and Rorschach data (also Trauma Symptom Inventory-2 [TSI-2]) for 337 female offenders including subsets of psychopathic (N = 124) and non-psychopathic (N = 57) females. We make note of the differences between female and male psychopaths.
Scientists should be able to provide support for the absence of a meaningful effect. Currently, researchers often incorrectly conclude an effect is absent based a nonsignificant result. A widely recommended approach within a frequentist framework is to test for equivalence. In equivalence tests, such as the two one-sided tests (TOST) procedure discussed in this article, an upper and lower equivalence bound is specified based on the smallest effect size of interest. The TOST procedure can be used to statistically reject the presence of effects large enough to be considered worthwhile. This practical primer with accompanying spreadsheet and R package enables psychologists to easily perform equivalence tests (and power analyses) by setting equivalence bounds based on standardized effect sizes and provides recommendations to prespecify equivalence bounds. Extending your statistical tool kit with equivalence tests is an easy way to improve your statistical and theoretical inferences.
This study investigated the cultural and linguistic adaptability of the Rorschach Performance Assessment System (R-PAS), a new Rorschach administration, scoring, and interpretation system that minimizes psychometric weaknesses of the Comprehensive System (CS). This investigation addressed the validity of R-PAS measures of psychotic characteristics and psychopathology severity in Taiwan, including the incremental validity of the R-PAS relative to the CS variables measuring the same constructs. Ninety Taiwanese individuals (75 psychiatric patients and 15 nonpatients) were tested with standard R-PAS administration and scoring. Two non-Rorschach severity of disturbance measures and 2 psychosis measures served as independent criterion measures. The R-PAS measures were found to be valid in Taiwan in assessing psychotic symptoms and psychopathology severity, thus demonstrating cultural and linguistic adaptability. Moreover, hierarchical regression analyses demonstrated incremental validity for the R-PAS variables over their CS counterparts, providing support that the R-PAS revisions enhance the test psychometrically. These research findings also demonstrate the viability of the R-PAS as a Rorschach system that can be effectively employed outside the U.S. in a different language and culture. (PsycINFO Database Record
(c) 2015 APA, all rights reserved).
Gurley et al. (Psychological Injury and Law 7:9-17, 2014) express reservations about the admissibility of testimony based on the Rorschach Performance Assessment System (R-PAS) in court. They question whether there is sufficient evidentiary foundation in the underlying psychometrics and adequate general acceptance among psychologists for R-PAS-based testimony to meet either the Daubert or Frye criteria for admissibility and also raise doubts about how well it meets the criteria for the use of forensic tests proposed by Heilbrun (Law and Human Behavior 16:257-272, 1992). This invited comment addresses their concerns about the admissibility of R-PAS-based testimony and corrects some erroneous statements about the psychometrics of R-PAS and the pertinent empirical literature. Gurley et al. characterize R-PAS as being in competition with the established Comprehensive System (CS; Exner 2003), though we clarify that it is actually an evolutionary development from the CS and designed to be a replacement for it. We also point out how their conclusion that R-PAS-based forensic testimony may be hazardous or premature is based on an insufficient familiarity with the R-PAS scientific and professional literature, a misinterpretation of the Frye and Daubert evidentiary standards, and a mischaracterization of several of Heilbrun's (Law and Human Behavior 16:257-272, 1992) criteria for the use of tests in forensic testimony.
The present study consisted of a meta-analytic review of the criterion validity of the Rorschach Mutuality of Autonomy (Urist, 1977) scale. Search procedures yielded 27 independent samples (total N = 1,803, average n = 67, SD = 31) for inclusion in the meta-analysis. Results support the criterion validity of the Mutuality of Autonomy with an average overall weighted effect size of r = .24, p r = .18, p Q = 37.67, df = 26, p = .07), and all between-study moderator analyses were nonsignificant (ps > .19) with the exception of the specific type of criterion variable. Implications for future research and clinical practice are discussed. (PsycINFO Database Record (c) 2013 APA, all rights reserved)
The admissibility of the Rorschach has been a concern of forensic psychologists for many years. The focus of this debate has been the Comprehensive System, which is the most researched of the current Rorschach Systems available in the USA. However, recently, a new, competing system has been published: the Rorschach Performance Assessment System (R-PAS; Meyer et al., Rorschach Performance Assessment System: Administration, Coding, Interpretation, and Technical Manual, 2011). Using Heilbrun’s (Law and Human Behavior 16:257–272, 1992) guidelines as a framework, we examine the admissibility of this new system according to the standards outlined in Daubert (1993) and Frye (1923). We conclude that we have reservations about the admissibility of the R-PAS in court at the present time, notwithstanding ongoing work on this system.
Monte Carlo computer simulations were used to investigate the performance of three χ–2 test statistics in confirmatory factor analysis (CFA). Normal theory maximum likelihood χ–2 (ML), Browne's asymptotic distribution free χ–2 (ADF), and the Satorra-Bentler rescaled χ–2 (SB) were examined under varying conditions of sample size, model specification, and multivariate distribution. For properly specified models, ML and SB showed no evidence of bias under normal distributions across all sample sizes, whereas ADF was biased at all but the largest sample sizes. ML was increasingly overestimated with increasing nonnormality, but both SB (at all sample sizes) and ADF (only at large sample sizes) showed no evidence of bias. For misspecified models, ML was again inflated with increasing nonnormality, but both SB and ADF were underestimated with increasing nonnormality. It appears that the power of the SB and ADF test statistics to detect a model misspecification is attenuated given nonnormally distributed data. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
The Rorschach Performance Assessment System (R-PAS) was introduced
in 2011 as an alternative to Exner’s Comprehensive System
(CS). This article critically evaluates the contemporary use of
the R-PAS in forensic contexts. The standing of the R-PAS in
relation to relevant legal standards, professional guidelines, and
published professional models is considered. Normative reference
data, revised administration procedures, and general acceptance
are discussed, and questions surrounding the comparability of
the R-PAS to the CS are explored. Historical transitions to revised
versions of existing assessment procedures are reviewed with particular
attention to the recurrent question of comparability and how
such concerns were addressed. Although the R-PAS shows significant
promise, fully adopting it into current forensic practice may
be premature.
A review of research examining the construct validity of J. M. Masling, L. Rabie, and S. H. Blondheim's (1967) Rorschach Oral Dependency (ROD) scale as a measure of interpersonal dependency revealed that this scale has demonstrated good interrater reliability, and that evidence regarding the convergent and discriminant validity of the ROD scale is generally strong. Mixed results have been obtained in studies assessing the internal reliability of ROD scale scores. There have been very few studies examining the predictive validity and retest reliability of the ROD scale, and few studies assessing the relationship of ROD scores to scores on other objective and projective measures of dependency. Implications of these findings for laboratory and clinical research involving the ROD scale are discussed, and suggestions for future studies in this area are offered. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
In the context of the development of prototypic assessment instruments in the areas of cognition, personality, and adaptive functioning, the issues of standardization, norming procedures, and the important psychometrics of test reliability and validity are evaluated critically. Criteria, guidelines, and simple rules of thumb are provided to assist the clinician faced with the challenge of choosing an appropriate test instrument for a given psychological assessment. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
A meta-analysis of published studies in which scores on objective (i.e., self-report) or projective measures of interpersonal dependency were used to predict some aspect of dependency-related behavior revealed that validity coefficients for projective tests (number of comparisons=32) were generally larger than validity coefficients for objective tests (number of comparisons=54). The relationships of setting in which data were collected, source of behavioral ratings, and participant classification method on observed test score–behavior correlations were also assessed. Implications of these findings for use of objective and projective dependency measures in clinical, laboratory, and field settings are discussed. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
[See Table 4 for Rorschach variables Assessing Psychosis] We systematically evaluated the peer-reviewed Rorschach validity literature for the 65 main variables in the popular Comprehensive System (CS). Across 53 meta-analyses examining variables against externally assessed criteria (e.g., observer ratings, psychiatric diagnosis), the mean validity was r = .27 (k = 770) as compared to r = .08 (k = 386) across 42 meta-analyses examining variables against introspectively assessed criteria (e.g., self-report). Using Hemphill's (2003) data-driven guidelines for interpreting the magnitude of assessment effect sizes with only externally assessed criteria, we found 13 variables had excellent support (r ≥ .33, p < .001; ∴ FSN > 50), 17 had good support (r ≥ .21, p < .05, FSN ≥ 10), 10 had modest support (p < .05 and either r ≥ .21, FSN < 10, or r = .15-.20, FSN ≥ 10), 13 had little (p < .05 and either r = < .15 or FSN < 10) or no support (p > .05), and 12 had no construct-relevant validity studies. The variables with the strongest support were largely those that assess cognitive and perceptual processes (e.g., Perceptual-Thinking Index, Synthesized Response); those with the least support tended to be very rare (e.g., Color Projection) or some of the more recently developed scales (e.g., Egocentricity Index, Isolation Index). Our findings are less positive, more nuanced, and more inclusive than those reported in the CS test manual. We discuss study limitations and the implications for research and clinical practice, including the importance of using different methods in order to improve our understanding of people. (PsycINFO Database Record (c) 2012 APA, all rights reserved).
One possible reason for the continued neglect of statistical power analysis in research in the behavioral sciences is the inaccessibility of or difficulty with the standard material. A convenient, although not comprehensive, presentation of required sample sizes is provided. Effect-size indexes and conventional values for these are given for operationally defined small, medium, and large effects. The sample sizes necessary for .80 power to detect effects at these levels are tabled for 8 standard statistical tests: (1) the difference between independent means, (2) the significance of a product-moment correlation, (3) the difference between independent rs, (4) the sign test, (5) the difference between independent proportions, (6) chi-square tests for goodness of fit and contingency tables, (7) 1-way analysis of variance (ANOVA), and (8) the significance of a multiple or multiple partial correlation.
Research suggests that productivity could impact the stability of Rorschach scores. To explore for this effect, we conducted secondary analyses of test–retest data gathered using the Rorschach Comprehensive System (Exner, 200312.
Exner , J. E. Jr. 2003. The Rorschach: A Comprehensive System: Vol. 1. Basic foundations, , 4th ed., Hoboken, NJ: Wiley. View all references) and available for 75 French, nonpatient adults (Sultan, Andronikof, Réveillère, & Lemmel, 200637.
Sultan , S. ,
Andronikof , A. ,
Réveillère , C. and
Lemmel , G. 2006. A Rorschach stability study in a non-patient adult sample. Journal of Personality Assessment, 87: 330–348. [Taylor & Francis Online], [PubMed], [Web of Science ®]View all references). We examined how response frequency (R) impacted stability using hierarchical regression models. Results on 83 variables from the lower part of the structural summary showed that stability was impacted by the mean level of productivity in 12 variables with medium to large effects (including Zf, HVI, and W location). Stability was also impacted by variations of productivity in 9 variables with medium to large effects (including Passive Movement, D Location, or Human Contents). Higher mean R and variability of R impacted stability levels negatively. Transforming scores into proportions (i.e., dividing scores by R) was beneficial for some important variables (including FM+m, Zf, DQ+). Procedures should be developed to limit productivity and control for R variations across time if one wishes to derive more reliable descriptions of individuals from the Rorschach.
A large body of empirical evidence supports the reliability, validity, and utility of the Rorschach. This same evidence reveals that the recent criticisms of the Rorschach are largely without merit. This article systematically addresses several significant Rorschach components: interrater and temporal consistency reliability, normative data and diversity, methodological issues, specific applications in the evaluation of thought disorder and suicide, meta-analyses, incremental validity, clinician judgment, patterns of use, and clinical utility. Strengths and weaknesses of the test are addressed, and research recommendations are made. This information should give the reader both an appreciation for the substantial, but often overlooked, research basis for the Rorschach and an appreciation of the challenges that lie ahead.
In this article, we describe interrater reliability for the Comprehensive System (CS; Exner. 1993) in 8 relatively large samples, including (a) students, (b) experienced re- searchers, (c) clinicians, (d) clinicians and then researchers, (e) a composite clinical sample (i.e., a to d), and 3 samples in which randomly generated erroneous scores were substituted for (f) 10%, (g) 20%, or (h) 30% of the original responses. Across samples, 133 to 143 statistically stable CS scores had excellent reliability, with median intraclass correlations of.85, .96, .97, .95, .93, .95, .89, and .82, respectively. We also demonstrate reliability findings from this study closely match the results derived from a synthesis of prior research, CS summary scores are more reliable than scores assigned to individual responses, small samples are more likely to generate unstable and lower reliability estimates, and Meyer's (1997a) procedures for estimating response segment reliability were accurate. The CS can be scored reliably, but because scoring is the result of coder skills clinicians must conscientiously monitor their accuracy.
Brief and lengthy Rorschach records have been identified as common problems in protocol administration. Clinicians have debated how to prevent overly short and long records, but they have been reluctant to alter standardized administration for fear of introducing bias. The present study examines a nonintrusive method for constraining responses by prompting for an extra response when only one is offered per card and by removing the card after four responses are given. Among patients who typically produce brief records, consisting of a residential sample of civil and forensic patients with a range of disordered thinking, the alternative administration method demonstrated improved Comprehensive System validity in assessing thought disorder and eliminated the need to readminister the test due to fewer than 14 responses. The findings have clinical implications for protocol administration with thought-disordered populations that typically produce brief records.
We build on the work of all the authors contributing to this Special Supplement by summarizing findings across their samples of data, and we also draw on samples published elsewhere. Using 21 samples of adult data from 17 countries we create a composite set of internationally-based reference means and standard deviations from which we compute T-scores for each sample. Figures illustrate how the scores in each sample are distributed and how the samples compare across variables in eight Rorschach Comprehensive System (CS; Exner, 200311.
Exner , J. E. Jr. 2003. The Rorschach: A comprehensive system, , 4th ed., New York: Wiley. View all references) clusters. The adult samples from around the world are generally quite similar, and thus we encourage clinicians to integrate the composite international reference values into their clinical interpretation of protocols. However, the 31 child and adolescent samples from 5 countries produce unstable and often quite extreme values on many scores. Until the factors contributing to the variability among these samples are more fully understood, we discourage clinicians from using many CS scores to make nomothetic, score-based inferences about psychopathology in children and adolescents.
The Rorschach Performance Assessment System (R-PAS; Meyer, Viglione, Mihura, Erard, & Erdberg, 2011 Meyer, G. J., Viglione, D. J., Mihura, J. L., Erard, R. E., & Erdberg, P. (2011). Rorschach Performance Assessment System: Administration, coding, interpretation, and technical manual. Toledo, OH: Rorschach Performance Assessment System. [Google Scholar]) introduced R-optimized administration to reduce variability in the number of Responses (R). We provide new data from six studies of participants randomly assigned to receive a version of this method or Comprehensive System (CS; Exner, 2003 Exner, J. E. (2003). The Rorschach: A Comprehensive System. Vol. 1. Basic foundations (4th ed.). Hoboken, NJ: Wiley. [Google Scholar]) administration. We examine how administration methods affect 3 types of codes most likely to contain potential projective material and the frequency of these codes for the 1st, 2nd, 3rd, 4th, or last response to a card (R in Card). In a meta-analytic summary, we found 37% of responses have 1 type of code, 19% have 2 types, and 3% have all 3 types, with stable proportions across responses within cards. Importantly, administration method had no impact on potential projective variable means. Differential skew across samples made variability harder to interpret. Initial results suggesting differences in 3 of the 18 specific Type by R in Card pairs did not follow a coherent pattern and disappeared when using raw counts from all participants. Overall, data do not support concerns that R-optimized administration might alter potential projective processes, or make potentially “signature” last responses to the card any different in R-PAS than the CS.
Controlling the number of Rorschach responses (R) as a method to reduce variability in the length of records has stimulated controversy among researchers for many years. Recently, the Rorschach Performance Assessment System (R-PAS; Meyer, Viglione, Mihura, Erard, & Erdberg, 2011 ) introduced an R-Optimized method to reduce variability in R. Using 4 published and 2 previously unpublished studies (N = 713), we examine the extent to which 51 Comprehensive System-based scores on the R-PAS profile pages are affected as a result of receiving Comprehensive System (CS; Exner, 2003 ) administration versus a version of R-Optimized administration. As hypothesized, R-the intended target of R-Optimized administration-showed reliable weighted average differences across each method of administration. As expected, its mean modestly increased and its standard deviation notably decreased. Also as hypothesized, the next largest effects were decreases in the variability (SD) of 2 variables directly related to R, R8910% and Complexity. No other reliable differences were observed. Therefore, because R-Optimized administration does not notably modify the existing CS-based normative values for other profiled R-PAS variables, the data do not support concerns that R-Optimized administration notably modifies the Rorschach task or that existing CS research data would not generalize to R-PAS. However, because R-Optimized administration reduces variability in R, it allows a single set of norms to apply readily to more people.
Recently, psychologists have emphasized the response process-that is, the psychological operations and behaviors that lead to test scores-when designing psychological tests, interpreting their results, and refining their validity. To illustrate the centrality of the response process in construct validity and test interpretation, we provide a historical, conceptual, and empirical review of the main uses of the background white space of the Rorschach cards, called space reversal (SR) and space integration (SI) in the Rorschach Performance Assessment System. We show how SR and SI's unique response processes result in different interpretations, and that reviewing their literatures with these distinct interpretations in mind produces the expected patterns of convergent and discriminant validity. That is, SR was uniquely related to measures of oppositionality; SI was uniquely related to measures of cognitive complexity; and both SR and SI were related to measures of creativity. Our review further suggests that the Comprehensive System use of a single space code for all uses of white space likely led to its lack of meta-analytic support as a measure of oppositionality (Mihura, Meyer, Dumitrascu, & Bombel, 2013 ). We close by discussing the use of the response process to improve test interpretation, develop better measures, and advance the design of research.
Recently, the Rorschach Performance Assessment System (R–PAS; Meyer, Viglione, Mihura, Erard, & Erdberg, 2011) was introduced to overcome some possible limitations of the Comprehensive System (CS; Exner, 2003) while continuing its efforts to link Rorschach inferences to their evidence base. An important, technical modification to the scoring system is that R–PAS interpretations are based on both standard scores and complexity-adjusted scores. Two previous U.S. studies reported good to excellent interrater reliability (IRR) for the great majority of R–PAS variables; however, IRR of complexity-adjusted scores has never been investigated. Furthermore, no studies have yet investigated R–PAS IRR in Europe. To extend this literature, we examined R–PAS IRR of Page 1 and Page 2 raw and complexity-adjusted scores with 112 Italian Rorschach protocols. We collected a large sample of both clinical and nonclinical Rorschach protocols, each of which was coded separately by 2 independent raters. Results demonstrated a mean intraclass correlation of .78 (SD = .14) for raw scores and.74 (SD = .14) for complexity-adjusted scores. Overall, for both raw and complexity-adjusted values, most of the variables were characterized by good to excellent IRR.
This is the definitive, supplementary reference book for accurate coding of the Rorschach Comprehensive System.
http://www.rorschachcodingsolutions.com/login.asp
To generate normative reference data for the Rorschach Performance Assessment System (R–PAS), modeling procedures were developed to convert the distribution of responses (R) in protocols obtained using Comprehensive System (CS; Exner 20037.
Exner, J. E. (2003). The Rorschach: A Comprehensive System. Vol. 1. Basic foundations (4th ed.). Hoboken, NJ: Wiley.View all references) administration guidelines to match the distribution of R in protocols obtained using R-Optimized Administration (Meyer, Viglione, Mihura, Erard, & Erdberg, 201115.
Meyer, G. J., Viglione, D. J., Mihura, J. L., Erard, R. E., & Erdberg, P. (2011). Rorschach Performance Assessment System: administration, coding, interpretation, and technical manual. Toledo, OH: Rorschach Performance Assessment System.View all references). This study replicates the R–PAS study, examining the impact of modeling R-Optimized Administration on Brazilian normative reference values by comparing a sample of 746 CS administered protocols to its counterpart sample of 343 records modeled to match R-Optimized Administration. The results were strongly consistent with the R–PAS findings, showing the modeled records had a slightly higher mean R and, secondarily, slightly higher means for Complexity and V-Comp, as well as smaller standard deviations for R, Complexity, and R8910%. We also observed 5 other small differences not observed in the R–PAS study. However, when comparing effect sizes for the differences in means and standard deviations observed in this study to the differences found in the R–PAS study, the results were virtually identical. These findings suggest that using R-Optimized Administration in Brazil might produce normative results that are similar to traditional CS norms for Brazil and similar to the international norms used in R–PAS.
Although projective techniques continue to be widely used in clinical and forensic settings, their scientific status remains highly controversial. In this monograph, we review the current state of the literature concerning the psychometric properties (norms, reliability, validity, incremental validity, treatment utility) of three major projective instruments: Rorschach Inkblot Test, Thematic Apperception Test (TAT, and human figure drawings. We conclude that there is empirical support for the validity of a small number of indexes derived from the Rorschach and TAT. However, the substantial majority of Rorschach and TAT indexes are not empirically supported. The validity evidence for human figure drawings is even more limited. With a few exceptions, projective indexes have not consistently demonstrated incremental validity above and beyond other psychometric data. In addition, we summarize the results of a new meta-analysis intended to examine the capacity of these three instruments to detect child sexual abuse. Although some projective instruments were better than chance at detecting child sexual abuse, there were virtually no replicated findings across independent investigative terms. This meta-analysis also provides the first clear evidence of substantial file drawer effects in the projectives literature, as the effect sizes from published studies markedly exceeded those from unpublished studies. We conclude with recommendations regarding the la) construction of projective techniques with adequate validity: lbl forensic and clinical rise of projective techniques, and Ic) education and training of future psychologists regarding projective techniques.
Exner (19896.
Exner, J. E. (1989). Searching for projection in the Rorschach. Journal of Personality Assessment, 53, 520–536.View all references) and Weiner (200333.
Weiner, I. B. (2003). Principles of Rorschach interpretation (2nd ed.). Mahwah, NJ: Erlbaum.View all references) identified 3 types of Rorschach codes that are most likely to contain personally relevant projective material: Distortions, Movement, and Embellishments. We examine how often these types of codes occur in normative data and whether their frequency changes for the 1st, 2nd, 3rd, 4th, or last response to a card. We also examine the impact on these variables of the Rorschach Performance Assessment System's (R–PAS) statistical modeling procedures that convert the distribution of responses (R) from Comprehensive System (CS) administered protocols to match the distribution of R found in protocols obtained using R-optimized administration guidelines. In 2 normative reference databases, the results indicated that about 40% of responses (M = 39.25) have 1 type of code, 15% have 2 types, and 1.5% have all 3 types, with frequencies not changing by response number. In addition, there were no mean differences in the original CS and R-optimized modeled records (M Cohen's d = –0.04 in both databases). When considered alongside findings showing minimal differences between the protocols of people randomly assigned to CS or R-optimized administration, the data suggest R-optimized administration should not alter the extent to which potential projective material is present in a Rorschach protocol.
This second edition of Irving Weiner's classic comprehensive, clinician-friendly guide to utilizing the Rorschach for personality description has been revised to reflect both recent modifications in the Rorschach Comprehensive System and new evidence concerning the soundness and utility of Rorschach assessment. It integrates the basic ingredients of structural, thematic, behavioral, and sequence analysis strategies into systematic guidelines for describing personality functioning. It is divided into three parts. Part I concerns basic considerations in Rorschach testing and deals with conceptual and empirical foundations of the inkblot method and with critical issues in formulating and justifying Rorschach inferences. Part II is concerned with elements of interpretation that contribute to thorough utilization of data in a Rorschach protocol: the Comprehensive System search strategy; the complementary roles of projection and card pull in determining response characteristics; and the interpretive significance of structural variables, content themes, test behaviors, and the sequence in which various response characteristics occur. Each of the chapters presents and illustrates detailed guidelines for translating Rorschach findings into descriptions of structural and dynamic aspects of personality functioning. The discussion throughout emphasizes the implications of Rorschach data for personality assets and liabilities, with specific respect to adaptive and maladaptive features of the manner in which people attend to their experience, use ideation, modulate affect, manage stress, view themselves, and relate to others. Part III presents 10 case illustrations of how the interpretive principles delineated in Part II can be used to identify assets and liabilities in personality functioning and apply this information in clinical practice. These cases represent persons from diverse demographic backgrounds and demonstrate a broad range of personality styles and clinical issues. Discussion of these cases touches on numerous critical concerns in arriving at different diagnoses, formulating treatment plans, and elucidating structural and dynamic determinants of behavior. © 2003 by Lawrence Erlbaum Associates, Inc. All rights reserved.
Meta‐analysis has become a standard way of summarizing empirical studies in many fields, including ecology and evolution. In ecology and evolution, meta‐analyses comparing two groups (usually experimental and control groups) have almost exclusively focused on comparing the means, using standardized metrics such as Cohen's / Hedges’ d or the response ratio.
However, an experimental treatment may not only affect the mean but also the variance. Investigating differences in the variance between two groups may be informative, especially when a treatment influences the variance in addition to or instead of the mean.
In this paper, we propose the effect size statistic ln CVR (the natural logarithm of the ratio between the coefficients of variation, CV, from two groups), which enables us to meta‐analytically compare differences between the variability of two groups. We illustrate the use of lnCVR with examples from ecology and evolution.
Further, as an alternative approach to the use of lnCVR, we propose the combined use of ln s (the log standard deviation) and (the log mean) in a hierarchical (linear mixed) model. The use of ln s with overcomes potential limitations of lnCVR and it provides a more flexible, albeit more complex, way to examine variation beyond two‐group comparisons. Relevantly, we also refer to the potential use of ln s and lnCV (the log CV) in the context of comparative analysis.
Our approaches to compare variability could be applied to already published meta‐analytic data sets that compare two‐group means to uncover potentially overlooked effects on the variance. Additionally, our approaches should be applied to future meta‐analyses, especially when one suspects a treatment has an effect not only on the mean, but also on the variance. Notably, the application of the proposed methods extends beyond the fields of ecology and evolution.
This article describes 3 studies evaluating normative reference data for the Rorschach Comprehensive System (CS; Exner, 2003, 2007), with a particular focus on the viability of the Composite International Reference Values (CIRVs) that were compiled from 21 adult studies by Meyer, Erdberg, and Shaffer (2007). Study 1 documented how the CIRV norms are virtually identical when organized into 3 groups differentiated by the quality of their data collection effort, including an optimal group of 4 samples that relied on multiple experienced examiners and provided ongoing quality control over administration and coding. Analyses also showed that relative to the group of more optimal samples, the group of less optimal samples did not produce more variability in summary scores within or across samples or lower interrater reliability for coding. Study 2 used the existing CS reference norms to generate T scores for the CIRV means and documented how the CS norms make other samples of healthy nonpatients look psychologically impaired in multiple domains. Study 3 documented with examples from 4 different countries how 2 sets of within-country local norms produced notably different results on some variables, which compromises the ability of local norms to be used instead of the CIRVs. Taken together, the 3 studies provide support for the use of CIRVs in clinical practice as norms that are generalizable across samples, settings, languages, and cultures and that account for the natural variability that is present when clinicians and researchers contend with the ambiguity contained in the standard CS reference materials concerning the proper ways to administer and code. We conclude by urging CS users to rely on the CIRVs when making clinical inferences and to adopt alternative methods of ensuring they are following cohesively standardized administration and coding guidelines.
Key practitioner message:
Alternative administration method successfully eliminated overly short and excessively long records. ○Utility is potentially increased by greatly reducing both short records that often lack reliability and validity, as well as long records that consume an excessive amount of examiner administration and scoring time. Psychometric properties and the ability to apply parametric statistics are likely increased across variables given that the distribution of R is more normal. Re-administration due to inadequate R is almost never needed. Results are consistent with the conclusion that this alternative procedure reduces examiner variability by offering simple, but explicit instructions for encouraging sufficient productivity. Overall variability of R produced using the refined alternative procedure was significantly less than that produced using the traditional CS method, although more in line with Exner's (2003) normative expectations. ○Suggests that when using the alternative method, R becomes less of a confound for all other scores that are moderately to highly correlated with R. Also demonstrated that the reduced variability of R and the reduced number of less useful short and long records are generalized to clinical samples. ○Additional research (Reese, Viglione, & Giromini, 2014) provides support for these conclusions with child clinical samples.
We examined the association of gender, ethnicity, age, and education with 60 Rorschach scores using three clinical and nonclinical samples of adults and youths (ns = 640, 249, and 241). As anticipated for our data sets, there were no reliable associations for gender, ethnicity, or adult age. However, in adults years of education was associated with variables indicative of complexity, the articulation of subtlety and nuance, cognitive synthesis, and coping resources. In the clinical sample of youths, increasing age was primarily associated with more conventional perception and less illogical thought processes. Limitations are discussed in conjunction with further research that could address them, along with implications for applied practice.
The Mutuality of Autonomy scale (MA) is a Rorschach variable designed to capture the degree to which individuals mentally represent self and other as mutually autonomous versus pathologically destructive (Urist, 1977). Discussions of the MA's validity found in articles and chapters usually claim good support, which we evaluated by a systematic review and meta-analysis of its construct validity. Overall, in a random effects analysis across 24 samples (N = 1,801) and 91 effect sizes, the MA scale was found to maintain a relationship of r =.20, 95% CI [.16,.25], with relevant validity criteria. We hypothesized that MA summary scores that aggregate more MA response-level data would maintain the strongest relationship with relevant validity criteria. Results supported this hypothesis (aggregated scoring method: r =.24, k = 57, S = 24; nonaggregated scoring methods: r =.15, k = 34, S = 10; p =.039, 2-tailed). Across 7 exploratory moderator analyses, only 1 (criterion method) produced significant results. Criteria derived from the Thematic Apperception Test produced smaller effects than clinician ratings, diagnostic differentiation, and self-attributed characteristics; criteria derived from observer reports produced smaller effects than clinician ratings and self-attributed characteristics. Implications of the study's findings are discussed in terms of both research and clinical work.
Retest reliability coefficients were calculated for 72 pairs of Rorschach records. In the target group of 36 pairs, one of the tests contained fewer than 14 responses, and the second record in the pair contained more than 14 responses. The remaining 36 pairs of protocols were used as a control group, and both records in each pair contained at least 15 responses. The resulting correlations indicate that if a protocol contains at least 15 answers, there is a substantial probability that a retest will show essentially the same distribution of scores, but if a protocol contains less than 14 answers, only two variables, the χ + % and the Basic D Score, are likely to remain in the same interpretive range if a longer protocol is taken subsequently from the same subject. A comparison of pairs of protocols in which both records have Lambda values of .85 or less indicates that some brief records may be reliable for some variables, and some have interpretive value, but even that type of brief record should probably be approached with much caution.
Although projective techniques continue to be widely used in clinical and forensic settings, their scientific status remains highly controversial. In this monograph, we review the current state of the literature concerning the psychometric properties (norms, reliability, validity, incremental validity, treatment utility) of three major projective instruments: Rorschach Inkblot Test, Thematic Apperception Test (TAT), and human figure drawings. We conclude that there is empirical support for the validity of a small number of indexes derived from the Rorschach and TAT. However, the substantial majority of Rorschach and TAT indexes are not empirically supported. The validity evidence for human figure drawings is even more limited. With a few exceptions, projective indexes have not consistently demonstrated incremental validity above and beyond other psychometric data. In addition, we summarize the results of a new meta-analysis intended to examine the capacity of these three instruments to detect child sexual abuse. Although some projective instruments were better than chance at detecting child sexual abuse, there were virtually no replicated findings across independent investigative teams. This meta-analysis also provides the first clear evidence of substantial file drawer effects in the projectives literature, as the effect sizes from published studies markedly exceeded those from unpublished studies. We conclude with recommendations regarding the (a) construction of projective techniques with adequate validity, (b) forensic and clinical use of projective techniques, and (c) education and training of future psychologists regarding projective techniques.
© 2000 Association for Psychological Science.
We surveyed practicing clinicians who regularly used the Rorschach about the perceived clinical validity of specific Rorschach scores from many coding systems. The survey included quantitative feedback on the validity of specific variables as well as qualitative input in several areas, including the validity of specific variables, the potentially unique information that can be obtained from them, coding challenges associated with Comprehensive System (CS) codes, and recommendations for CS developments. Participants were recruited by applying a snowball sampling strategy. Based on responses from 246 experienced clinicians from 26 countries, composite judgments on rated variables were quite reliable (e.g., M α = .95 across 88 CS variables), despite limited agreement among any 2 judges. The aggregated judgments clearly differentiated among scores that were considered more and less clinically valid and the overall results aligned with recently obtained meta-analytic conclusions from the traditional validity literature (Mihura, Meyer, Dumitrascu, & Bombel, 2012). The judges also provided guidance concerning revisions and enhancements that would facilitate Rorschach-based assessment in the future. We discuss the implication of the quantitative and qualitative findings and provide suggestions for future directions based on the results.
Using a multiple regression approach with a large developmental sample (N = 460) of Rorschach protocols from psychiatric, forensic, and nonclinical control groups, the authors created continuous multivariable Composite scores corresponding to the Comprehensive System (CS) Perceptual-Thinking Index, Hypervigilance Index, and Suicide Constellation. Within a validation sample (N = 230), these three new scores, called the Thought and Perception Composite, Vigilance Composite, and Suicide Concern Composite were strongly associated with the three original CS Indices. Additional analyses suggest that the new Composite scores were more reliable than and at least as valid as the original Indices. Interpretive guidelines are offered.
Based on available research findings, the Rorschach Performance Assessment System (Meyer, Viglione, Mihura, Erard, & Erdberg, 201126.
Meyer , G. J. ,
Viglione , D. J. ,
Mihura , J. L. ,
Erard , R. E. and
Erdberg , P. 2011,2012. A manual for the Rorschach Performance Assessment System, Edited by:
Mihura , J. L. ,
Meyer , G. J. ,
Dumitrascu , N. and
Bombel , G. Toledo, OH: R–PAS Manuscript submitted for publication. A systematic review and meta-analysis of the Rorschach Comprehensive System validity literature View all references) was recently developed in an attempt to ground the administration, coding, and interpretation of the Rorschach in its evidence base, improve its normative foundation, integrate international findings, reduce examiner variability, and increase utility. This study sought to establish inter-rater reliability for the coding decisions in this new system. We randomly selected 50 Rorschach records from ongoing research projects using R-Optimized administration. The records were administered by 16 examiners and came from a diverse sample in terms of age, sex, ethnicity, educational background, and patient status. Results demonstrated a mean intraclass correlation of .88 and median of .92. Overall, the findings indicate good to excellent inter-rater reliability for the great majority of codes and are consistent with previous findings of strong inter-rater reliability for alternative Rorschach systems and scores.
The present study examined the relationship between the Rorschach Ego Impairment Index (EII) and psychiatric severity. Search procedures yielded 13 independent samples (total N = 1402, average n = 108, standard deviation = 90) for inclusion in the meta‐analysis. Inter‐rater reliability analyses demonstrated that coding of effect sizes and moderator variables was completed with good to excellent reliability. Results indicated that higher EII scores were associated with greater psychiatric severity, with an overall weighted effect size of r = 0.29, p = 0.000002 (95% confidence interval = 0.17–0.40), supporting the EII's validity as a measure of psychological impairment. Publication bias analyses did not indicate any significant cause for concern regarding the results. The data were demonstrably heterogeneous ( Q = 56.82, p = 0.0000001), and results of post‐hoc tests indicated that effect sizes with dependent variables obtained via researcher ratings were significantly larger than any of the following: effect sizes with dependent variables obtained via clinician ratings, informant ratings, information about level of treatment or placement status or self‐report ratings ( p 's = 0.0005, 0.003, <0.001, <0.001, respectively). In addition, there was a trend for effect sizes based on performance‐based measures to be larger than those based on information about level of treatment or placement status ( p = 0.098) as well as those based on self‐report measures ( p = 0.076). Other moderator analyses were non‐significant ( p 's > 0.10). Copyright © 2010 John Wiley & Sons, Ltd.
Key Practitioner Message:
• The Rorschach Ego Impairment Index (EII) demonstrated validity in measuring psychiatric severity across a range of normative, outpatient, residential, and inpatient samples.
• The degree of the EII's validity in assessing psychiatric severity compared favorably to the overall validity of the Rorschach and the MMPI.
• The EII appears to be most valid in capturing psychiatric severity as measured by researcher ratings of social competency or estimated ego impairment.
The issue of assumptions concerning projection and the Rorschach is discussed in relation to the Rorschach response process and the process of projection. Some data suggest that some minus answers may evolve because of projection, but the limited nature of the findings indicate that these types of projected answers will be difficult to identify in most cases. Findings concerning passive and aggressive movement responses, morbid content responses, and movement answers, coded using a new special score for positive cooperative interaction (COP) in human and animal movement responses, suggest that most projections will manifest in responses that depart from and/or embellish the stimulus field. Caution is recommended about attempting to interpret single answers tht appear to be projections, because most any might be the product of simple stimulus classification.
I first examined the effects of response frequency (R) on the Comprehensive System's constellation indices (Exner, 1991). R is significantly associated with 26 of the 29 constellation criteria that are based on raw numbers and significantly correlated with total scores on each of the constellations. I then examined how R impacts the external validity of the constellations. The ability of the Schizophrenia Index and the Suicide Constellation to discriminate diagnostic groups appears to be impaired when protocols deviate from average length, whereas the Hypervigilance Index (HVI) appears most diagnostic of a paranoid condition when it is relatively elevated in brief records. R also clearly mediates the relationship between the Rorschach and Minnesota Multiphasic Personality Inventory-2 (MMPI-2) criteria. In lengthy protocols, many of the cross-method convergent correlations between appropriate MMPI-2 scales and the Depression Index, Obsessive Style Index, and HVI are quite high and approach the maximum found in personality research. In brief protocols, there are strong negative correlations between these constellations and self-reports of depression and interpersonal distress. Implications from these findings for the integration of assessment methods are discussed.
The coefficient of variation is often used as a guide of the repeatability of measurements in clinical trials and other medical work. When possible, one makes repeated measurements on a set of individuals to calculate the relative variability of the test with the understanding that a reliable clinical test should give similar results when repeated on the same patient. There are times, however, when repeated measurements on the same patient are not possible. Under these circumstances, to combine results from different clinical trials or test sites, it is necessary to compare the coefficients of variation of several clinical trials. Using the work of Miller, we develop a general statistic for testing the hypothesis that the coefficients of variation are the same for k populations, with unequal sample sizes. This statistic is invariant under the choice of the order of the populations, and is asymptotically chi 2. We provide an example using data from Yang and HayGlass. We compare the size and the power of the test to that of Bennett, Doornbos and Dijkstra and a statistic based on Hedges and Olkin.
As interest in Rorschach research has increased, an old issue-whether differences in R require that other scores be normalized in relation to R when Rorschach data are analyzed-has been resurrected. Most who advocate normalizing or partialing for R cite Cronbach (1949) or Fiske and Baughman (1953) as the basis for their argument. Although Cronbach offered several useful suggestions regarding this problem, the data base to which he referred and that used by Fiske and Baughman allowed for a much greater variation in R than occurs when Rorschach's standard instructions are applied. Intercorrelations between R and most parametric Rorschach variables are not significant among groups of nonpatients, schizophrenics, and depressives. Thus, although the issue of R can be critical when some variables are included in a data matrix, most analyses can proceed without partialing or normalization. When nonparametric variables are included in an analysis, partialing or normalizing is inappropriate because the resulting distributions will be distorted.
This article focuses on clinical and research problems associated with response frequency (R) being a variable in the Rorschach. Despite the fact that variations in R directly contribute to 50% of the explainable variance among Rorschach raw scores, there is a dearth of empirical evidence to document what R actually measures. Furthermore, in the practical use of the Rorschach's structural data, R is considered to be a nuisance variable that is controlled and not deemed interpretively significant. Given this information, two research agendas are proposed. The first is to more thoroughly determine whether R measures anything of substantial clinical importance. The second is to evaluate systematically the relative merits of making R a constant rather than a variable through use of an R-controlled method of Rorschach administration. This strategy would resolve many of the psychometric problems related to R. Introducing greater structure and clearer expectations to the task may also sharpen the Rorschach's ability to assess and predict important aspects of personality. However, significant disadvantages would also result from this change in administration. Both sides of the issue are discussed in some detail.
The degree to which projection plays a role in Rorschach (Rorschach, 1921/194266.
Rorschach , H. 1942. Psychodiagnostik, Bern, , Switzerland: Bircher. (Original work published 1921) View all references) responding remains controversial, in part because extant data have yielded inconclusive results. In this investigation, I examined the impact of social projection on Rorschach Oral Dependency (ROD) scores using methods adapted from social cognition research. In Study 1, I prescreened 85 college students (40 women and 45 men) with the ROD scale and a widely used self-report measure of dependency, the Interpersonal Dependency Inventory (IDI; Hirschfeld et al., 197734.
Hirschfeld , R. M. A. ,
Klerman , G. L. ,
Gough , H. G. ,
Barrett , J. ,
Korchin , S. J. and
Chodoff , P. 1977. A measure of interpersonal dependency. Journal of Personality Assessment, 41: 610–618. [Taylor & Francis Online], [PubMed], [Web of Science ®]View all references). Results show that informing participants who scored low on the IDI that they were in fact highly dependent led to significant increases in ROD scores; I did not obtain parallel ROD increases for participants who scored high on the IDI or for participants who received low-dependent feedback. In Study 2, I examined a separate sample of 80 prescreened college students (40 women and 40 men) and showed that providing low self-report participants an opportunity to attribute dependency to a fictional target person prior to Rorschach responding attenuated the impact of high-dependent feedback on ROD scores. These results suggest that projection played a role in at least one domain of Rorschach responding. I discuss theoretical, clinical, and empirical implications of these results.
This is a normative study with 409 adult nonpatients living in the state of Sao Paulo, Brazil. The Rorschach was administered by a team of nine psychologists; eight had had further training in the Rorschach method by the Brazilian Rorschach Society and one intensively was prepared by the project coordinator. Of the study participants, 200 lived in the state capital (Sao Paulo) and the other 209 were in other large and small cities in the state, including a coastal city and one in the mountains. Previous psychological or psychiatric treatments were criteria for exclusion. Each protocol was coded independently by two examiners, and then agreement of the two codings was checked. Differences between the two codings were discussed in a meeting of the whole team, which was supervised by the project coordinator to guarantee codification quality control. Upon completion of the codings, an analysis of examiner differences was undertaken, the results of which are in the text. Interrater reliability statistics among examiners were calculated, including percentage of agreement and kappa. Reliability statistics among examiners at the response level are presented as are Comprehensive System (CS; 1999, 2003) findings.