No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Building Confidence in Psychological Assessment as a Therapeutic Intervention: An Empirically Based Reply to Lilienfeld, Garb, and Wood (2011)
Abstract
Lilienfeld, Garb, and Wood (2011) commented on our recently published meta-analysis (Poston & Hanson, 2010). In their thorough, fine-grained critique, they questioned the study's inclusion criteria, sampling, and results, suggesting the latter are "overstated" (p. 1048). Additionally, they discussed the Barnum effect, suggesting that it accounts, at least partially, for the potential efficacy and/or effectiveness of psychological assessment as a therapeutic intervention (PATI). In addressing Lilienfeld et al.'s (2011) concerns, we incorporate their suggestions and present new findings based on the original data set, including 15 independent and 56 dependent effect sizes (ESs). Upon reanalysis, a comparable, significant overall Cohen's d ES of 0.403 (95% CI [0.302, 0.503]) was found, thus building confidence in our original results, as well as the overall efficacy of PATI. Given PATI's potential treatment utility, we mention possible change mechanisms, above and beyond Barnum effects. And finally, in hopes of launching the field forward, we offer suggestions for future research and collaboration. (PsycINFO Database Record (c) 2011 APA, all rights reserved).
... Most notably, guidance counselors and psychologists working in the clinical, educational, and occupational fields use these instruments for various assessment needs, such as mental disorders diagnostics, career guidance, and personnel selection (Hogan, 2013;Urbina, 2014). Several studies support the efficiency of psychometric instruments in these fields (Hanson & Poston, 2011;Meyer et al., 2001;Morris et al., 2015;Sackett et al., 2008;Schmidt et al., 2016). For example, the validity of psychometric instruments seems to be comparable to that of other psychological and medical interventions (e.g., psychotherapy, electrocardiogram) (Meyer et al., 2001). ...
... In sum, in the context of the study of social representations regarding psychological assessment, it seems appropriate to retain characteristics from both psychological assessment models to examine professionals' preferred approaches to assessment. In that respect, it is interesting to note that some studies have examined the frequency of specific C/T assessment practices related to test feedback among psychologists (Curry & Hanson, 2010;Jacobson et al., 2015), while other studies have addressed the efficacy of C/T assessment compared to other approaches, including IG Hanson & Poston, 2011). However, no other study has attempted to examine the actual place of the C/T and IG assessment models in psychological assessment practices. ...
... With respect to differences in motives for using psychometric instruments, they were consistent with the literature (e.g., Finn, 2007;Finn & Tonsager, 1997;Hanson & Poston, 2011). As expected, IG assessors consider the objectivity and necessity of psychometric instruments for some assessment activities, as well as their ability to improve decision-making as more important factors compared to C/T assessors. ...
... Specifically, Lilienfeld, Garb, and Wood [12] criticized the review for having too wide inclusion criteria leading to overly wide sampling and results. Hanson and Poston (2011) [13] replied to this critique by publishing an updated version of their meta-analysis in which they excluded three studies and reanalyzed the remaining 14 studies (n = 1375). This updated version resulted in a statistically significant weighted Cohen's d = 0.40 [95% CI, 0.30 to 0.50] across outcomes but added no improved reporting of populations, diagnostic criteria, or assessment of study quality or certainty of evidence. ...
... Specifically, Lilienfeld, Garb, and Wood [12] criticized the review for having too wide inclusion criteria leading to overly wide sampling and results. Hanson and Poston (2011) [13] replied to this critique by publishing an updated version of their meta-analysis in which they excluded three studies and reanalyzed the remaining 14 studies (n = 1375). This updated version resulted in a statistically significant weighted Cohen's d = 0.40 [95% CI, 0.30 to 0.50] across outcomes but added no improved reporting of populations, diagnostic criteria, or assessment of study quality or certainty of evidence. ...
... Available evidence suggests that standardized psychological tests can be administered with a psychotherapyoriented approach, which might benefit patients suffering from psychiatric illness [13]. The review described in the present protocol will provide an evidence synthesis of the effectiveness of administering standardized psychological tests as an integrative part of the treatment of patient populations with psychiatric illnesses. ...
Background
Clinicians usually conduct diagnostic assessments in order to establish a diagnosis or to evaluate the effect of treatment. Two meta-analyses suggest that diagnostic assessment administered in collaboration with the patient and personalized feedback might have a therapeutic effect.
Methods
We aim to conduct a systematic review and meta-analysis of the effect on symptomatology when using assessment as a therapeutic intervention for patients with psychiatric illnesses.
We will search in five relevant electronic databases. Two reviewers will independently select papers following pre-defined eligibility criteria, extract data, and assess the quality of included studies. Randomized controlled trials comparing the effect of psychological assessment with other psychotherapeutic interventions in populations of patients will be included in the meta-analysis. We will extract data on symptom-related outcomes, quality of life, dropout, and re-diagnosis and use meta-analysis techniques to compute the effect size of interventions using assessment as a psychotherapeutic intervention.
The review will be conducted and reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement. Risk of bias will be assessed by using the Risk of Bias tool RoB 2.0 of the Cochrane Collaboration, and the certainty of the body of evidence will be assessed with the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach.
Discussion
The results will be able to inform clinicians and policymakers on the effect of assessment and, depending on the results, could lead to a recommendation for modified assessment procedures and approaches in mental health services. Ultimately, it might improve the treatment outcome in mental health services
Systematic review registration
PROSPERO CRD42021270567
... There is mounting evidence supporting the psychological benefits of psychological assessment feedback as a therapeutic intervention for patients within the clinical psychological and counselling assessment settings, where smallto-medium sized effects (d = 0.403-0.423) are typically reported following feedback (Aschieri et al., 2016;Hanson & Poston, 2011;Poston & Hanson, 2010). However, there is currently no comparable supporting evidence within the clinical neuropsychological assessment (NPA) setting. ...
... Furthermore, (3) within-subjects analysis of the combined sample showed clinically significant (small-medium sized) improvements in three wellbeing domains: perceived everyday cognitive functioning (a primary outcome), self-efficacy for managing MS, and mood (stress and depression) symptoms (secondary outcomes) at 1month follow-up. The effect size of the psychological benefits found at 1month follow-up (d = −0.369 to −0.488) are comparable in magnitude to the therapeutic benefits found in the clinical psychological and counselling assessment settings (Hanson & Poston, 2011;Poston & Hanson, 2010). ...
... There is mounting evidence supporting the psychological benefits of psychological assessment feedback as a therapeutic intervention for patients within the clinical psychological and counselling assessment settings, where smallto-medium sized effects (d = 0.403-0.423) are typically reported following feedback (Aschieri et al., 2016;Hanson & Poston, 2011;Poston & Hanson, 2010). However, there is currently no comparable supporting evidence within the clinical neuropsychological assessment (NPA) setting. ...
... Furthermore, (3) within-subjects analysis of the combined sample showed clinically significant (small-medium sized) improvements in three wellbeing domains: perceived everyday cognitive functioning (a primary outcome), self-efficacy for managing MS, and mood (stress and depression) symptoms (secondary outcomes) at 1month follow-up. The effect size of the psychological benefits found at 1month follow-up (d = −0.369 to −0.488) are comparable in magnitude to the therapeutic benefits found in the clinical psychological and counselling assessment settings (Hanson & Poston, 2011;Poston & Hanson, 2010). ...
Evidence supporting the direct therapeutic benefits of neuropsychological assessment (NPA) feedback relies mostly upon post-feedback consumer surveys. This randomized-controlled trial with cross-over investigated the benefits of NPA feedback in multiple sclerosis (MS). Seventy-one participants were randomly allocated to NPA with feedback or a “delayed-treatment” control group. The primary hypotheses were that NPA feedback would lead to improved knowledge of cognitive functioning and improved coping. Outcome instruments were administered by a research assistant blinded to group allocation. At 1-week post-NPA feedback there were no significant group-by-time interaction effects, indicating no improvement. But nor was there any significant deterioration in psychological wellbeing, despite most participants receiving “bad news” confirming cognitive impairment. At 1-month follow-up, within-subjects’ analyses not only found no evidence of any delayed deterioration, but showed clinically significant improvement (small-medium effects) in perceived everyday cognitive functioning, MS self-efficacy, stress and depression. Despite lack of improvement in the RCT component at 1-week post-NPA feedback, the absence of deterioration at this time, in addition to significant improvements in perceived cognitive functioning, self-efficacy and mood at follow-up, together with high satisfaction ratings, all support NPA feedback as a safe psycho-educational intervention that is followed by improved psychological wellbeing over time.
Trial registration: Uniform Trial Number identifier: U1111-1127-1585.
Trial registration: Australian New Zealand Clinical Trials Registry identifier: ACTRN12612000161820.
... L'évaluation psychométrique, pratique qui consiste essentiellement en l'utilisation d'instruments psychométriques, est ainsi employée dans divers secteurs, notamment dans le domaine clinique, en recherche, en éducation et en employabilité, à des fins d'orientation, de sélection, de classement, de dépistage et de diagnostic (Hogan, 2017 ;Neukrug & Fawcett, 2015 ;Urbina, 2014). De nombreuses études montrent l'efficacité de l'évaluation psychométrique dans le domaine clinique (Hanson & Poston, 2011 ;Meyer et al., 2001 ;Poston & Hanson, 2010), en contexte de sélection de personnel (Morris et al., 2015 ;Sackett et al., 2008 ;Schmidt et al., 2016) et en éducation (Sackett et al., 2008). 2 Bien que les instruments psychométriques présentent des avantages considérables, ils comportent aussi certaines limites fondamentales. Ceuxci varient en termes de qualité (Cohen & Swerdlik, 2010 ;Miller & Lovler, 2016), génèrent des données qui peuvent évoluer (Guédon, 1994 ;Guédon et al., 2011), offrent une représentation partielle des caractéristiques de la personne évaluée (Guédon et al., 2011 ;Meyer et al., 2001) et sont vulnérables à une variété de sources d'erreur (Guédon, 1994 ;Hogan, 2017 ;Kaplan & Saccuzzo, 2017 ;Miller & Lovler, 2016 ;Urbina, 2014). ...
... En proposant d'examiner le lien entre les approches d'évaluation thérapeutique et de collecte d'informations dans leur globalité et les pratiques d'évaluation psychométrique, la présente étude tente de développer la recherche portant sur ces modèles au-delà de l'analyse de leur efficacité relative (Aschieri et al., 2015 ;Hanson & Poston, 2011 ;Poston & Hanson, 2010) et de la fréquence de pratiques thérapeutiques strictement liées à la rétroaction de résultats (Curry & Hanson, 2010 ;Jacobson et al., 2015). ...
... Some meta-analyses, though, have found significant effect sizes between individual characteristics of clients (e.g., Beutler, Edwards, & Someah, 2018;Beutler, Kimpara, et al., 2018;Constantino et al., 2018;Krebs et al., 2018;Peluso & Freund, 2018;Swift et al., 2018)-all easily accessible via psychological assessment-and therapeutic outcomes in different types of therapy. Moreover, the evidence base supporting a semi-structured form of collaborative assessment, therapeutic assessment (TA; Finn, 2007;Finn & Tonsager, 1992, to improve outcomes is somewhat more robust (Hanson & Poston, 2011;), yet the mechanism through which psychological assessment works as a therapeutic intervention remains unclear (Lilienfeld et al., 2011) and its impact on therapeutic outcomes has not been as well demonstrated (De Saeger et al., 2014). ...
... Dans le cadre de l'évaluation formelle, basée sur des normes et des critères prédéterminés dans une approche objective (Renou, 2005), les professionnels peuvent donc recueillir un large éventail d'informations sur les caractéristiques psychologiques des individus, telles que leurs traits de personnalité et leurs habiletés, de façon efficace (Guédon et al., 2011;Urbina, 2014). La recherche empirique atteste d'ailleurs de la valeur et de l'utilité de l'évaluation psychométrique dans le domaine clinique (Hanson & Poston, 2011;Poston & Hanson, 2010), en sélection de personnel (Morris, Daisley, Wheeler & Boyer, 2015;Sackett, Borneman & Connelly, 2008;Schmidt, Oh & Shaffer, 2016) et en éducation (Sackett et al., 2008). ...
RÉSUMÉ :
Cet article présente les résultats d’une enquête en ligne réalisée à la suite de l’entrée en vigueur de la Loi modifiant le Code des professions et d’autres dispositions législatives dans le domaine de la santé mentale et des relations humaines. Cette enquête a été menée auprès d’un échantillon de 243 psychologues, 392 conseillers d’orientation (c.o.) et 90 psychoéducateurs (ps. éd.) québécois. Les résultats montrent d’abord que les modifications apportées au Code des professions semblent avoir eu un moindre impact sur les pratiques évaluatives des professionnels ciblés que ce qui était pressenti. Considérant la réserve de nouvelles activités d’évaluation pour les c.o. et les ps. éd., ces deux groupes professionnels perçoivent toutefois plus de changements dans leurs pratiques évaluatives que les psychologues. Par ailleurs, un relativement faible degré d’appropriation est observé pour plusieurs activités réservées en termes de fréquence d’exercice et de sentiment de compétence. Enfin, la fréquence moyenne d’exercice des activités réservées varie selon l’appartenance professionnelle et la clientèle desservie. Ainsi, les psychologues s’approprient davantage les activités qui leur sont réservées que les c.o. et les ps. éd. C’est aussi le cas pour les professionnels dont les clientèles vulnérables desservies (p. ex. : individus présentant des troubles du développement) sont ciblées par les modifications apportées au Code des professions. Des hypothèses d’explication sont proposées par rapport à l’impact modeste des modifications apportées au Code des professions et au relativement faible degré d’appropriation des activités d’évaluation réservées.
ABSTRACT :
This article presents the results of an online survey conducted following the coming into force of the Act to amend the Professional Code and other legislative provisions in the field of mental health and human relations. This survey was conducted on a sample of 243 psychologists, 392 guidance counselors (g.c.) and 90 psychoeducators (ps. ed.) from Quebec. Results first show that the amendments to the Professional Code appear to have had a lesser impact on the assessment practices of targeted professionals than was anticipated. Considering the attribution of new reserved assessment activities to g.c. and ps. ed., they, however, perceive more changes in their assessment practices than psychologists. Furthermore, a relatively low degree of appropriation is observed for several reserved activities in terms of frequency of exercise and perceived competence. Lastly, the average frequency of exercise varies according to the professional affiliation and the served clientele. Thus, psychologists appropriate their reserved activities to a greater extent than g.c. and ps. ed. This is also the case for professionals whose vulnerable clientele (e.g. individuals with developmental disabilities) are targeted by the amendments to the Professional Code. Plausible explanations for the modest impact of the amendments to the Professional Code and the relatively low degree of appropriation of the reserved assessment activities are proposed.
... A respeito da eficácia da AT, existem diversas pesquisas, empregando metodologias diferentes, que tem trazido resultados importantes. Dentre elas pode-se citar os ensaios quase-experimentais e clínicos (Ougrin, Ng, & Low, 2008;Smith, Eichler, Norman, & Smith, 2015), ensaios randomizados (De Saeger et al., 2014;Newman, & Greenway, 1997) e meta-análises (Hanson, & Poston, 2011;Poston, & Hanson, 2010). As evidências empíricas acumuladas são muito promissoras e apontam que AT tem beneficiado os clientes de várias maneiras, tais como ajudar na redução de sintomas, no aumento da autoestima, da esperança, da colaboração e adesão ao tratamento, bem como tem facilitado a aliança terapêutica e a psicoterapia simultânea ou subsequente (Blonigen, Timko, Jacob, & Moos, 2015;Del Giudice, Yanovsky, & Finn, 2014;Hilsenroth, Peters, & Ackerman, 2004;Holm-Denoma et al., 2008;Lerner, 2005;Little, & Smith, 2008;Smith et al., 2015;Smith, & George, 2012;Tarocchi, Aschieri, Fantini, & Smith, 2013;Tharinger, & Pilgrim, 2012). ...
Resumo: Este trabalho começa discutindo eventuais problemas nos processos de avaliação psicológica, tal como são conduzidos tradicionalmente, mostrando fragilidades que nem sempre são consideradas ao se planejar uma determinada avaliação. O resultado disto é que frequentemente o cliente não sentirá os reais benefícios daquele processo para si mesmo ou sua vida, podendo mesmo ocorrer o contrário e o processo se transformar em uma experiência negativa de sentimento de incompreensão, desesperança e baixa motivação para seguir as orientações finais. Há, entretanto, uma nova proposta que vem se expandindo na prática da Psicologia, na forma de um processo semiestruturado denominado de Avaliação Terapêutica. Por meio de uma estratégia cuidadosamente planejada, desenvolve-se uma avaliação inteiramente colaborativa e já se realiza algum tipo de intervenção durante o processo. Estas duas características costumam produzir uma experiência terapêutica imediata. O processo é pautado pelo uso imprescindível de testes de autorrelato e projetivos (expressivos ou de desempenho) por meio dos quais o avaliador busca aprofundar o conhecimento sobre o cliente e intervir da forma mais adequada para produzir uma efetiva mudança nele. Neste artigo são apresentadas as etapas claramente definidas deste processo, finalizando-se com os contextos possíveis de sua aplicação.
... Second, given the empirical support for CTA (Poston & Hanson, 2010), we were surprised that clinical science programs were much less likely to offer this training (53% compared to 95% scientist-practitioner and 73% practitioner-focused programs). Perhaps this could be due to Lilienfeld, Garb, and Wood's (2011) critique of Poston and Hanson's meta-analysis, but Hanson and Poston's (2011) reply soundly addressed the issues they raised. Regarding individual tests, there was an increase in training in different cognitive tests and self-report personality tests. ...
We surveyed American Psychological Association-accredited clinical psychology doctoral programs' (n = 83) training in psychological assessment-specifically, their coverage of various assessment topics and tests in courses and practica, and whether the training was optional or required. We report results overall and separately per training model (clinical science, scientist-practitioner, and practitioner-focused). Overall, our results suggest that psychological assessment training is as active, or even more active, than in previous years. Areas of increased emphasis include clinical interviewing and psychometrics; multimethod, outcomes, health, and collaborative or therapeutic assessment; and different types of cognitive and self-report personality tests. All or almost all practice-focused programs offered training with the Thematic Apperception Test and Rorschach compared to about half of the scientist-practitioner programs and a third of the clinical science programs. Although almost all programs reported teaching multimethod assessment, what constitutes different methods of assessing psychopathology should be clarified in future studies because many programs appear to rely on one method-self-report (especially clinical science programs). Although doctoral programs covered many assessment topics and tests in didactic courses, there appears to be a shortage of program-run opportunities for students to obtain applied assessment training. Finally, we encourage doctoral programs to be familiar with (a) internships' assessment expectations and opportunities, (b) the professional guidelines for assessment training, and
... Specifically, they suggested that some included studies confounded the interpretability of TA's effect (e.g., additional treatment components), that the researchers omitted nonsignificant results, and that the authors failed to consider potential Barnum effects (Lilienfeld et al., 2011). In a response to these critiques, Hanson and Poston (2011) reexamined their data specifically with regard to these suggestions and concluded that strong positive effects on process and outcome are still robust, despite the criticisms. Barnum effects may be more pronounced with positive feedback, as opposed to negative feedback (Snyder & Cowles, 1979). ...
Utilizing personality science within clinical assessment and intervention can aid in treatment planning. General personality constructs also are related to clinically relevant areas of dysfunction. However, personality continues to be underutilized in clinical settings. This article reviews current literature pertaining to the clinical applications of personality with a focus on dimensional models such as the Five-Factor Model. With the advent of a dimensional personality model in DSM-5, the clinical use of traits is an important topic of exploration. This review discusses the clinical significance of personality and personality pathology in various aspects of living (i.e., functioning, physical health, mental health), clinical applications and utility within clinical and treatment settings, and future research directions, as well as suggestions for further utilization of personality traits.
... effets positifs du retour donnée aux patients s'expliquait en fait par le fameux effet Barnum (les réactions positives déclenchées par un retour général et positif). Hanson et Poston (2011) ont réagi à ces critiques par une réanalyse des données, composée de 14 études et 1375 participants et résultant également en effets significatifs (Hanson & Poston, 2011), confirmant ainsi l'efficacité du retour individualisé pour les adultes. Les tailles d'effet relevées sont comparables aux tailles d'effet générales de la psychothérapie. ...
Cet article traite de la recherche empirique sur l’utilité et l’efficacité de l’évaluation thérapeutique ou de ses éléments (par exemple : le retour individuel proposé aux patients). Les études varient selon les méthodes et les problèmes traités mais toutes présentent l’évaluation thérapeutique comme une intervention brève qui augmente les variables du processus en rapport avec le traitement préconisé et a des effets directs de régression des symptômes de divers problèmes psychologiques. D’autres pistes d’études dans ce domaine sont fournies en fin d’article.
... A reanalysis of the data, which included 14 studies with 1,375 participants, also resulted in a significant effect (d = .403; Hanson & Poston, 2011), confirming the effectiveness of individualized assessment feedback for adults. ...
... A reanalysis of the data (14 studies, 1,375 participants) resulted in a significant overall effect of d = .40 (Hanson & Poston, 2011). Further, Poston and Hanson found different effects for symptom (d = .37; ...
This pragmatic study evaluates the effectiveness of a collaborative assessment intervention as an approach to midtherapy consultation, which has yet to be empirically tested. Ten adult participants in ongoing psychotherapy with a variety of presenting concerns, primarily consisting of general mood and adjustment issues, received a brief assessment-based intervention based on Finn’s (2007) Therapeutic Assessment model. Following the collection of assessment questions and the administration of a multimethod assessment battery, clients and therapists participated in a joint feedback session with the assessor. Clients were then followed as their psychotherapy continued. The results of idiographic and aggregate analytic approaches revealed significant reductions in client-reported symptomatic distress, as evidenced by a medium effect size (d = –.50) and a significant change in the trajectory of distress. Client reports of the process of psychotherapy revealed a significant increase in the clients’ ratings of the working alliance. The findings suggest that a midtherapy consultation using collaborative assessment is beneficial but that further rigorous investigation is needed.
... Die Arbeiten der Gruppe von Michael J. Lambert finden in ihrem Setting den Effekt zuverlässig (Lambert, 2005(Lambert, , 2007Lambert et al., 2003;Sapyta, Riemer, & Bickman, 2005;Shimokawa, Lambert, & Smart, 2010), mit dem System der Gruppe um Duncan und Miller werden ähnliche Befunde erzielt ("The Partners for Change Outcome Management System"; Lambert & Shimokawa, 2011;Miller, Duncan, Sorrell, & Brown, 2005). Und eine Meta-Analyse über Feedback als therapeutisches Mittel im Allgemeinen belegt diesen Effekt darüber hinaus (Hanson & Poston, 2011;Lilienfeld, Garb, & Wood, 2011;Poston & Hanson, 2010). Durch die verwendeten Designs (randomisiert kontrollierte Studien mit Messwiederholung und ohne Rückmeldungen in einer Bedingung) kann weitestgehend ausgeschlossen werden, dass der gefundene Effekt nur auf die wiederholte Erhebung zurückzuführen ist, und auch eine systematische Untersuchung von Wiederholungseffekten zeigte, dass dieser Effekt zwar vorhanden ist, aber nur sehr gering (Durham et al., 2002). ...
[Scroll down for for English version]
Psychotherapeutische Maßnahmen wirken im Mittel, doch ist unklar, ob eine Therapie bei einem konkreten Patienten auch ihre (maximale) Wirkung zeigt. Befunde der Patientenorientierten Versorgungsforschung zur Wirksamkeit von Feedback zeigen, dass eine Verbesserung des Therapieergebnisses durch Qualitätssicherungsmaßnahmen wie z.B. kontinuierliches Monitoring möglich ist. Diese Forschung und ihre praktische Anwendung machen es nötig, Daten am Einzelfall wiederholt zu erheben. Damit wird es unumgänglich, die Messungen effizienter zu gestalten. Diese Arbeit widmet sich der Frage, wie Mischverteilungsmodelle (Item Response/ Rasch-Modell sowie Latent Profile Analysis) dazu genutzt werden können, Fragebögen (weiter-) zu entwickeln, die mit kürzerem Umfang für diese Zwecke besser eingesetzt werden können.
Gegen die Verwendung von Mischverteilungsmodellen sprach lange, dass spezielle Software und Training erforderlich waren und dies im Praxiskontext nicht machbar war. Mit R steht eine freie Softwareumgebung zur Verfügung, die die Schätzung einer ganzen Fülle von Modellen möglich macht, auch von Mischverteilungsmodellen. Da Qualitätssicherung bei frei verfügbarer Software nötig ist, widmet sich Studie I der Frage, ob drei zentrale Pakete zur Schätzung von Rasch-Modellen in der R-Umgebung (eRm, ltm, mixRasch; Details siehe unten) zu akzeptablen Schätzergebnissen führen, d.h. zur Nutzung empfohlen werden können. Hierzu wurden in einer Simulationsstudie die Itemzahl, Stichprobengröße und Verteilung der Stichprobe systematisch variiert und der Effekt auf die Schätzgenauigkeit geprüft. Es zeigte sich, dass alle drei Schätzalgorithmen unter den realisierten Bedingungen zu zufriedenstellenden Genauigkeiten kommen und die Verteilungsform unter den gewählten Bedingungen keinen Einfluss auf die Genauigkeit hatte.
Studie II nutzte das Rasch-Modell um für ein Maß psychischer Belastung Kurzformen für spezifische Erhebungszwecke zu entwickeln: (1) verkürzte Erhebung beim Screening und (2) verkürzte Erfassung im hohen Belastungsbereich. Die Kurzformen wurden mittels Bootstrap und Kreuzvalidierung dahingehend geprüft, ob sie replizierbar eine bessere Messqualität erbrachten als andere Itemauswahlen aus dem Fragebogen, was sich bestätigte. Durch die Verwendung des Rasch-Modells sind die so erstellten Kurzformen miteinander und auch mit der Vollversion vergleichbar. Dies macht auch ohne die Verwendung spezieller Software (teil-)adaptives Testen möglich.
Studie III untersuchte wie drei Methoden genutzt werden können um festzustellen, welche Items eines Tests sich über den Verlauf einer Therapie als veränderungssensitiv erweisen. Hierzu wurden mittels einer Bevölkerungsstichprobe und den Prä- und Post-Erhebungen einer ambulanten Behandlungsstichprobe Items aus einer Beschwerden-Skala verwendet. Die drei Methoden waren (1) herkömmliche Mittelwertsvergleiche, (2) Auswahl über Bootstrap-Konfidenzintervalle und (3) Auswahl mittels einer Latent Profile Analysis, die latente Klassen von Varianzmustern um die Itemmittelwerte schätzte. Das Bootstrap-Verfahren erwies sich am Konservativsten (4 Items) während die Auswahl mittels herkömmlicher Mittelwertsvergleiche am liberalsten war (9 Items). Die Effektstärken und Reliabilitäten der Kurzfassungen waren alle im akzeptablen Bereich.
Die Diskussion beginnt mit einer knappen Zusammenfassung der Ergebnisse der drei Studien. Im Anschluss werden die Ergebnisse der Studien auf übergreifende Aspekte bezogen. Dies sind faktorielle Validität, die Angemessenheit von Item Response Modellen zur Repräsentation psychische Belastung und die Anforderungen, die Kurzversionen letztlich erfüllen können. Insgesamt lässt sich festhalten, dass die Methoden nützliche Werkzeuge zur spezifischen Untersuchung von Skalen und zur Erstellung von Kurzformen darstellen. Besonders der in Studie II vorgestellte Bootstrap-Test der Itemauswahl stellt eine relevante Ergänzung der etablierten Vorgehensweise dar, da er empirisch belegt, dass die Auswahl für den jeweiligen Zweck einer Kurzform besser geeignet ist, als andere Items. Klinisch lässt sich festhalten, dass mit statischen Kurzversionen etablierter Messinstrumente auch in Erhebungskontexten ohne computerisierte Erhebungsmethoden hochqualitative Erhebungen durchgeführt werden können.
[English Abstract]
Psychotherapeutic interventions show reliable mean effects, but it is not always clear whether an intervention is also (maximally) effective with a specific client. Results from patient-focused research and on the efficacy of feedback show that it is possible to increase the effect of interventions by the use of quality assurance measures, e.g. continuous monitoring. This line of research and its practical application make it necessary to assess patients repeatedly over the course of an intervention. Therefore, assessments should be as efficient as possible. In the current application it is explored how mixture distribution models (Item Response/ Rasch-Model, Latent Profile Analysis) can be used, to develop short versions of questionnaires for the use in quality assurance and feedback systems.
Mixture distribution models have not been applied for some time, since specialized training and software were needed, which was difficult to get in practice contexts. R provides a free software environment that can be employed for a whole range of models, including mixture distribution models. Since quality assurance is necessary when using free software, the first study compares three packages for the estimation of Rasch Models that are available in R (eRm, ltm, mixRasch): do they provide acceptable estimates and can they be recommended for use? Test length, sample size, and trait distribution were varied and their effects on measurement efficiency was checked. All three packages provided acceptable estimates in the realized conditions.
In the second study the Rasch-Model was used to create short versions of an established instrument for the assessment of psychological distress for specific purposes: (1) screening applications, and (2) assessments in highly distressed populations. It was checked for the short versions via bootstrap tests and cross-validation, whether they showed a reliably higher measurement quality compared to other possible short versions. Because of the use of the Rasch-Model, trait scores from these short versions are comparable with each other as well as the full version of the scale. This makes adaptive testing possible without the use of software or computers.
The third study compared three methods to investigate which items of an instrument are especially sensitive to change over the course of psychotherapy. With data from a population sample and pre-/ post assessments from an outpatient sample this study assessed at the functioning of the items of a German instrument of psychological distress. The three methods investigated were: (1) simple comparisons of means, (2) selection via bootstrapped comparisons of means, and (3) selection based on the results from a Latent profile Analysis, that estimated patterns of variances. The bootstrapped selection proved to be the most conservative (4 Items), while the simple comparison of means was the most liberal method (9 Items). The effect sizes and reliabilities of all short versions were acceptable.
After summarizing the results of the studies, overarching aspects of the three studies are discussed. These are in turn factorial validity, the appropriateness of Item Response Models for the measurement of psychological distress, and the standards of measurement which short versions can realistically fulfil. Overall it can be said these methods provide useful tools for the investigation of the functioning of instruments. Especially, the bootstrap test in the second study is a relevant extension of current methods, since it provides the opportunity to demonstrate that a specific short version is more efficacious for a specific assessment purpose than other short versions from the same instrument. With view on clinical application it can be said that fixed short-versions of established instruments allow assessments of high quality, even if no computerized assessments are possible.
... Critics might point to the lack of change in psychotherapy outcome measures. However, we believe that both the design and the targeted population of our study argue against overly negative appraisals and, in fact, point to the need for a contextualized evaluation of treatment utility (Hayes, Nelson, & Jarrett, 1987). First, the design of our study precluded benefit from the indirect effects of the intervention. ...
The field of clinical personality assessment is lacking in published empirical evidence on its treatment and clinical utility. This paper reports on a randomized controlled clinical trial (N = 74) allocating patients awaiting treatment in a specialized clinic for personality disorders to either four sessions of (a) Therapeutic Assessment (TA) or (b) a structured Goal Focused Pre-Treatment Intervention (GFPTI). In terms of short-term outcome, TA demonstrated superior ability to raise outcome expectancies, and patient perceptions of progress towards treatment (Cohen’s d = .65 and .56, respectively), and yielded higher satisfaction (d = .68). Moreover, patients reported marginally stronger alliance to the TA clinicians than to GFPT clinicians (d = .46), even though therapists perceived the alliance as equally positive in both groups. No differences in symptomatic ratings were observed. Results are discussed with reference to the concept of treatment utility in this particular patient group.
Mental health assessment and evaluation models have waxed and waned over the years. However, a contemporary humanistic approach, Collaborative/Therapeutic Assessment (C/TA), holds considerable promise and staying power. When combined with Measurement-Based Care (MBC), therapeutic processes and outcomes are enhanced. This chapter focuses on the integration of C/TA and MBC into clinical practice. It includes relevant theory and research, real-life applications and examples, and answers fundamental questions, like “Can mental health assessment and testing actually be collaborative and humanistic in nature?” and “As clinicians, why should we care about measuring clients’ treatment progress?” At the end of the chapter, illustrative graphs and verbatim scripts are provided for clinical use.
Keywords
Mental health assessmentHumanistic assessmentCollaborative/therapeutic assessmentMeasurement-based careRoutine outcome monitoringProgress monitoringPsychotherapy process
This pre–post follow-up randomized trial investigated the receptiveness and responsiveness of 82 incarcerated men undergoing reentry to feedback (discussion-based, form-based, or none–minimal) regarding their criminogenic risk–needs assessment results. Both short-term outcomes (self-perceived risk–needs, motivation for change, treatment readiness, and feedback satisfaction) and longer-term outcomes (intuitional conduct, rearrest, or halfway house return) were examined. As hypothesized, among study completers ( n = 67), motivation for change was significantly higher following discussion feedback, and both feedback formats were rated favorably by participants. Contrary to hypotheses, feedback recipients, including those who showed gains at post, did not appear reliably distinct from others on longer-term outcomes; nor were most outcomes significantly associated with baseline risk scores. Feedback about risk and needs may be useful in correctional treatment for motivation enhancement and treatment orienting, but special attention to measurement, contextual, and intensity factors is warranted.
Résumé
Depuis sa création, l’évaluation thérapeutique (l’ET) a été appliquée à différentes situations. Pour certains contextes, le modèle a été adapté. Aujourd’hui, l’évaluation thérapeutique prend quatre formes différentes : adulte, couple, enfant et adolescent. Cet article reprend les étapes de l’ET et présente les adaptations apportées pour répondre au mieux aux besoins de chaque situation. Les similitudes et les différences entre ces différents modèles sont abordées.
This is the first study to evaluate a quality assurance and feedback tool with regard to its practical feasibility in routine care and within the German health insurance system. This paper summarizes the evaluators' perspective on issues regarding the representativeness of the sample, consequences for therapy length and central assumptions and concepts of the project. The evaluation team comments on the paper "Pilot project of the Techniker-Krankenkasse - A comment from the perspective of the academic advisory council".
© Georg Thieme Verlag KG Stuttgart · New York.
This article reviews four decades of research on test interpretation. It concludes that the intended effects of this intervention have been demonstrated. There seems no clear evidence, however, for the effectiveness of one interpretation modality over another, though clients seem to prefer individual interpretations. Also examined in the article are client and counselor variables that alone and in interaction may affect test interpretations. Literature on the Barnum Effect is discussed briefly as a possible area of 'basic research " in test interpretation. Finally, several issues (e.g., criteria) that merit attention both by researchers and research consumers are discussed
Two styles of test interpretation, delivered and interactive, were compared in ongoing career counseling for 26 university honors students. The styles were used in interpreting the Personality Research Form and the Vocational Preference Inventory in the second session. Clients listed their thoughts in session after each test was interpreted. After the session, clients evaluated session impact and counselor influence. Clients in the two conditions did not differ in the number of thoughts listed or in the favorability of their thoughts. However, clients who received an interactive interpretation also considered their sessions to be deeper and their counselors to be more expert, trustworthy, and attractive than did clients who received a delivered interpretation. Implications for test-interpretation practice and research are discussed. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Two styles of test interpretation (TI), delivered and interactive, were manipulated in a 1‐session counseling interview. The favorability of the interpretations (i.e., positive only and mixed) was also manipulated. After completing a well‐known personality test, 46 participants completed the TI session. Participants' thoughts and perceptions of their sessions and counselors did not differ across the 4 experimental conditions. However, their TI acceptance and helpfulness ratings did differ. Implications for TI research and practice are discussed.
There is general agreement that meta-analysis is an important tool for synthesizing study results in quantitative educational research. Yet, a shared feature of many meta-analyses is a failure to report sufficient information for readers to fully judge the reported findings, such as the populations to which generalizations are to be made, inclusion and exclusion criteria for studies, the rationale underlying coded variables, including their scaling, relationship to study goals, and the sensitivity of conclusions to the chosen methodology. The authors provide (a) evidence of the prevalence of deficiencies of reporting and (b) suggestions about the kind of information that meta-analysts should typically provide. Providing readers with that information should significantly improve readers' ability to evaluate the credibility of a meta-analysis.
Current suggestions for assessing clients across cultures fail to adequately aid the average practitioner This failure arises from unresolved issues and problems, interfering with the ability of most counselors and therapists to render sound clinical judgments. In response to these issues and problems, a procedure is described that sensitizes counselors to cultural data in assessment and case conceptualization. Grounded in a guiding philosophy of assessment, the Multicultural Assessment Procedure (MAP) was developed in consideration of a number of relevant critical issues. The procedure entails identifying cultural data through multiple methods of data collection, interpreting cultural data to formulate a working hypothesis, incorporating cultural data with other relevant clinical information to test the working hypothesis, and arriving at a sound (i.e., comprehensive and accurate) assessment decision. A case illustration demonstrates how to use the proposed procedure.
Classes of community college students completed the Strong Interest Inventory (SII), the Myers-Briggs Type Indicator (MBTI), both instruments, or neither instrument. Contrary to previous findings that different vocational treatments produce similar effects, 12 weeks after test interpretation participants who had completed both instruments showed a pattern of more change in career goal, specificity of career goal, or level of certainty with reference to career goal in analyses of (a) the total sample (N = 427), (b) males only (n = 120), (c) females only (n = 307), (d) traditional age students only (n = 337), and (e) reentry students only (n = 90). The MBTI was rated to be as helpful as the SII in career decision making. Significant gender and age effects were found. Results support theoretical models relating understanding of one's personality with effective career development, and the joint use of the SII and MBTI in vocational counseling.
Undergraduates (64 women and 35 men) enrolled in a university orientation course completed measures of career decision-making self-efficacy and career beliefs. Participants were then randomly assigned to one of three experimental groups: the Strong Interest Inventory (SII) with feedback group, the SII completion-only group, or the control group. Students who completed the SII and participated in a social cognitive-based group feedback and interpretation session exhibited higher levels of posttest career decision-making self-efficacy and differential career beliefs relative to students in the other experimental groups. Theoretical and practical implications of the results are presented, and ideas for additional research in this domain are discussed.
This study examined the therapeutic effects of sharing Minnesota Multiphasic Personality Inventory—2 (MMPI-2) assessment results with clients. It is based on an earlier study by S. E. Finn and M. E. Tonsager (1992). Participants were 60 university students seeking psychological services from a university counseling service. All participants completed the MMPI-2 and several outcome measures. Within 2 weeks of completing the assessment, clients in the experimental group received test feedback, using a collaborative model developed by S. E. Finn (1996). Clients in the control group received test feedback only after having completed the final outcome measures and following a delay of 1 week. Compared with the control group, those who received test feedback within the time frame of the experimental conditions reported a significant increase in self-esteem immediately following the feedback session and a significant decrease in symptomatic distress at a 2-week follow-up. Overall, the findings provide further evidence for the efficacy of psychological assessment as an effective therapeutic intervention. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
This is an investigation to determine if any 1 of 4 methods would be most effective in interpreting test results. The degree of counselor and student interaction varied in 3; while in the 4th there was none, since results were mailed. Immediately after the interview and 1 week later 156 students completed the Counselor-Inventory Rating Scale to express attitudes toward counselor and value of receiving the information. Conclusions are: (a) no significant differences on attitudes toward the counselor among the 4 methods, (b) when attitudes of students toward counselor were analyzed among the 4 counselors there was a significant difference, (c) in attitudes toward the value of receiving test information there was a significant difference between students who received their results through the mail and those who did not. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Compared 3 different methods of test interpretation (individual, small group, and written) on accuracy of self-estimates of test scores, change in concept of self and others, and evaluation of test interpretation. Data obtained from the 3 experimental groups were compared with each other on 2 occasions and each group was compared to a control group. Results indicate that: (1) all groups receiving test interpretations were more accurate in self-estimates of test scores than the controls, (2) those receiving individual interpretations expressed greatest satisfaction with the test interpretation procedures, and (3) all groups made significant change in concept of self and others, but no significant difference among groups was found. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Many counseling psychologists provide marital assistance to couples who have relationship problems and those who seek to enrich their relationships. The authors investigated the effects of individualized relationship assessment and feedback in relation to merely completing written questionnaires about the relationships on couples' satisfaction and commitment. Student couples (
N = 48; 26 married, 15 cohabiting, 7 engaged) participated either in (a) 3 sessions of assessment feedback (
n = 28) or (b) written assessment only (
n = 20). Assessment-feedback couples improved more over time than did written-assessment-only couples. The authors concluded that assessment and feedback produce small positive changes in already well-functioning relationships. Those changes may account for a substantial proportion of the changes produced by relationship enrichment programs. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
The relative effectiveness of individual and multiple (group) counseling in test interpretation interviews was evaluated in terms of pre- and postcounseling measures on 4 criteria: (a) accuracy of self ratings, (b) acquisition of information about tests, (c) feasibility of vocational choice, and (d) counselee satisfaction. The 2 experimental groups were also compared with a noncounseled control group, each group comprising a random sample of 100 university freshmen. Although no differences existed among the study groups at the outset of the study, postcounseling criteria measures showed significant gains by both experimental groups over the control. Relatively few postcounseling differences of any significance, however, were observed between the individual and multiple-counseled. The results of the study supported the use of multiple counseling as an effective approach for disseminating and interpreting test information. (27 ref.) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
40 university students were identified as either people oriented (PO) or task oriented (TO), on the basis of their responses to the Vocational Preference Inventory. Half of the PO Ss were given group vocational counseling using a learning through interaction method (LTI), and the other half were counseled through a learning through individual problem solving method (LTIPS). Similarly, half of the TO Ss were counseled through LTI and half through LTIPS. Counseling approaches that were congruent with personality types (i.e., LTIPS approach with TO Ss and LTI with PO Ss) were more effective than counseling approaches that did not match personality type. Effectiveness was measured in terms of amount of occupational information sought by Ss, Ss' evaluation of the group experience, and scores on the Attitude scale of the Career Maturity Inventory. (17 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Part 1 examines salient methodological and technical issues in research on counseling and related processes: the state of the art of counseling research, research strategies, issues in criteria selection, and considerations in selecting samples and control groups. Discussion is guided by the "bubble hypothesis," a view of research recognizing the costs inherent in each solution offered for methodological problems. Part 2 examines issues surrounding the scientific productivity of counseling psychologists and their specialty. Assets and negative features of the scientist-practitioner model are examined, and 2 propositions are offered about the research/science training situation in counseling psychology: First, students enter training with deeply ambivalent attitudes as to the value and meaning of science in their careers and their ability to be effective as scholars. Second, training programs show little that is deliberate and systematic in attempting to foster positive motivation and attitudes. 10 ingredients of effective research training are discussed. (4 p ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
The provocative results of Blair, Marcus, and Boccaccini (2008) suggest that the allegiance effect, previously suggested in psychotherapy outcome studies, may apply to studies of actuarial risk assessment. Despite this finding, the mechanisms of the effect, particularly in assessment research, are unknown and warrant further investigation. We discuss the file drawer effect, selective reporting, and “data massaging” as three potential explanations for allegiance effects in the assessment domain. Furthermore, we offer four suggestions for minimizing allegiance effects and their impact: routinely coding for allegiance in meta-analytic studies, operationalizing allegiance in multiple ways, encouraging collaborations among authors with differing allegiances, and creating study registries to track all dependent variables measured in studies.
In a recent article in this journal, Poston and Hanson (2010) reported a meta-analysis of 17 studies on the use of psychological assessment as a therapeutic intervention (PATI) and concluded that "psychological assessment procedures--when combined with personalized, collaborative, and highly involving test feedback--have positive, clinically meaningful effects on treatment" (Poston & Hanson, 2010, p. 203). Although extant data suggest that PATI can sometimes exert positive effects, Poston and Hanson's (2010) meta-analysis may overstate the magnitude of these effects because the authors (a) included several studies that combined assessment with treatment components that are irrelevant to PATI, sometimes rendering it impossible to attribute any observed effects to PATI per se and (b) excluded numerous nonsignificant results. Moreover, the studies Poston and Hanson (2010) reviewed neglected to rule out Barnum effects as alternative explanations for client improvement, raising the possibility that PATI works for reasons other than those proposed by its advocates. We conclude that Poston and Hanson's (2010) review leaves a number of lingering questions concerning the treatment utility of PATI unanswered.
In this empirical, mixed methods study, we explored test feedback training, supervision, and practice among psychologists, focusing specifically on how feedback is provided to clients and whether feedback skills are taught in graduate programs. Based on a 48.5% return rate, this national survey of clinical, counseling, and school psychologists' suggests psychologists provide test feedback to clients but inconsistently. Most respondents, 91.7%, indicated they give verbal feedback at least some of the time, whereas 35% do so every time. However, 2.8% indicated they never give feedback. A negative correlation exists for clinical psychologists between years since graduation and providing verbal feedback. Of particular interest, approximately one third of respondents indicated predoctoral coursework, practica, and internship were of little-to-no help in preparing them to provide feedback. Also, feedback training in predoctoral coursework, practica, and internship was not correlated to actually providing feedback. There was, however, a significant correlation between postdoctoral training and providing feedback. Consistent with existing ethical exceptions, the most frequent reason for not providing feedback was using assessments in forensic settings. Individuals who indicated their training was not helpful cited "trial and error" and self-instruction as ways in which they learned feedback skills. We discuss implications and suggestions for feedback training, research, and practice.
Collaborative, individualized assessment is an approach to psychological assessment in which the assessor and the client work together to develop productive understandings. Collaboration is a means of individualizing the assessment--its process, resulting suggestions, and written accounts. In this approach, life events are regarded as primary data. Test scores, categories, and related research are used as bridges into a particular life and as tools for then exploring that life. This article, an invited Master Lecture presented at the 1999 Society for Personality Assessment (SPA) meeting, presents examples of contextualizing, intervening, describing in life-world terms, and writing individualized reports with suggestions. Historical struggles to promote individualized assessment and current openness to its practices, especially within SPA, are mentioned. A hermeneutic approach to impression formation is described.
Brief alcohol interventions typically have been directed to heavy-drinking patients seeking primary health care and college students. This study examined the efficacy of mailing brief personalized assessment-feedback to interested drinkers recruited from the general public. We hypothesized that problem drinkers would benefit more from the intervention than individuals who were not problem drinkers.
A two-arm, double-blinded, community-based randomized controlled trial with 6-month follow-up.
A screening interview was administered to a stratified random sample of 10 014 Canadians 18 years of age and older (5621 women and 4393 men; M age = 43.3 years, SD = 15.99; response rate = 65.4%).
Current drinkers interested in receiving alcohol self-help materials (n = 1727) were assigned randomly to receive brief personalized assessment-feedback on male and female population drinking norms by mail, or to a delayed-treatment control group, and were contacted 6 months later (76% retention rate).
Problem drinking status at baseline [using sex-specific Alcohol Use Disorders Identification Test (AUDIT) cut scores], and frequency and quantity of alcohol use at follow-up.
Analysis of covariance identified the hypothesized interaction of baseline problem drinking status and treatment condition (P < 0.01). Among problem drinkers identified at baseline the intervention caused a 10.1% reduction in per-occasion binge drinking compared to controls, whereas there was no difference in binge drinking across conditions for non-problem drinkers.
The continuum of care for alcohol problems can be broadened by providing brief interventions to interested drinkers in the general population.
Although a growing body of scholarly work explores the unique utility and therapeutic uses of psychological assessment, less work has focused specifically on feedback, with few studies that have explored empirically the underlying processes that may describe the role of feedback in a successful assessment. The purpose of this project was to add to the discourse on this topic by engaging in an empirical study exploring assessee and assessor experiences of significant events in psychological assessment feedback. The methodology is qualitative and modeled after significant events research in the study of psychotherapy process. I analyze the accounts of 6 assessment clients and 6 assessment clinicians regarding key events in their experience of feedback.