Matthew Holcomb’s research while affiliated with Louisiana State University Health Sciences Center New Orleans and other places

Objective This study was design to evaluate the potential of the recognition trials for the Logical Memory (LM), Visual Reproduction (VR), and Verbal Paired Associates (VPA) subtests of the Wechsler Memory Scales–Fourth Edition (WMS-IV) to serve as embedded performance validity tests (PVTs). Method The classification accuracy of the three WMS-IV subtests was computed against three different criterion PVTs in a sample of 103 adults with traumatic brain injury (TBI). Results The optimal cutoffs (LM ≤ 20, VR ≤ 3, VPA ≤ 36) produced good combinations of sensitivity (.33–.87) and specificity (.92–.98). An age-corrected scaled score of ≤5 on either of the free recall trials on the VPA was specific (.91–.92) and relatively sensitive (.48–.57) to psychometrically defined invalid performance. A VR I ≤ 5 or VR II ≤ 4 had comparable specificity, but lower sensitivity (.25–.42). There was no difference in failure rate as a function of TBI severity. Conclusions In addition to LM, VR, and VPA can also function as embedded PVTs. Failing validity cutoffs on these subtests signals an increased risk of non-credible presentation and is robust to genuine neurocognitive impairment. However, they should not be used in isolation to determine the validity of an overall neurocognitive profile.

This study was designed to evaluate the classification accuracy of the Memory module for the Inventory of Problems (IOP-M) in a sample of real-world patients. Archival data were collected from a mixed clinical sample of 90 adults clinically referred for neuropsychological testing. The classification accuracy of the IOP-M was computed against psychometrically defined invalid performance. IOP-M ≤30 produced a good combination of sensitivity (.46-.75) and specificity (.86-.95). Lowering the cutoff to ≤29 improved specificity (.94-1.00) at the expense of sensitivity (.29-.63). The IOP-M correctly classified between 73% and 91% of the sample. Given its low cost, ease of administration/scoring in combination with robust classification accuracy, the IOP-M has the potential to expand the existing toolkit for the evaluation of performance validity during neuropsychological assessments.

This study was designed to compare the validity of the Inventory of Problems (IOP-29) and its newly developed memory module (IOP-M) in 150 patients clinically referred for neuropsychological assessment. Criterion groups were psychometrically derived based on established performance and symptom validity tests (PVTs and SVTs). The criterion-related validity of the IOP-29 was compared to that of the Negative Impression Management scale of the Personality Assessment Inventory (NIMPAI) and the criterion-related validity of the IOP-M was compared to that of Trial-1 on the Test of Memory Malingering (TOMM-1). The IOP-29 correlated significantly more strongly (z = 2.50, p = .01) with criterion PVTs than the NIMPAI (rIOP-29 = .34; rNIM-PAI = .06), generating similar overall correct classification values (OCCIOP-29: 79–81%; OCCNIM-PAI: 71–79%). Similarly, the IOP-M correlated significantly more strongly (z = 2.26, p = .02) with criterion PVTs than the TOMM-1 (rIOP-M = .79; rTOMM-1 = .59), generating similar overall correct classification values (OCCIOP-M: 89–91%; OCCTOMM-1: 84–86%). Findings converge with the cumulative evidence that the IOP-29 and IOP-M are valuable additions to comprehensive neuropsychological batteries. Results also confirm that symptom and performance validity are distinct clinical constructs, and domain specificity should be considered while calibrating instruments.

Objective: This study was designed to replicate previous research on critical item analysis within the Word Choice Test (WCT). Method: Archival data were collected from a mixed clinical sample of 119 consecutively referred adults (Mage = 51.7, Meducation = 14.7). The classification accuracy of the WCT was calculated against psychometrically defined criterion groups. Results: Critical item analysis identified an additional 2%-5% of the sample that passed traditional cutoffs as noncredible. Passing critical items after failing traditional cutoffs was associated with weaker independent evidence of invalid performance, alerting the assessor to the elevated risk for false positives. Failing critical items in addition to failing select traditional cutoffs increased overall specificity. Non-White patients were 2.5 to 3.5 times more likely to Fail traditional WCT cutoffs, but select critical item cutoffs limited the risk to 1.5-2. Conclusions: Results confirmed the clinical utility of critical item analysis. Although the improvement in sensitivity was modest, critical items were effective at containing false positive errors in general, and especially in racially diverse patients. Critical item analysis appears to be a cost-effective and equitable method to improve an instrument's classification accuracy. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Background: Abbreviated neurocognitive tests offer a practical alternative to full-length versions but often lack clear interpretive guidelines, thereby limiting their clinical utility. Objective: To replicate validity cutoffs for the Boston Naming Test —Short Form (BNT–15) and to introduce a clinical classification system for the BNT–15 as a measure of object-naming skills. Method: We collected data from 43 university students and 46 clinical patients. Classification accuracy was computed against psychometrically defined criterion groups. Clinical classification ranges were developed using a z-score transformation. Results: Previously suggested validity cutoffs (≤11 and ≤12) produced comparable classification accuracy among the university students. However, a more conservative cutoff (≤10) was needed with the clinical patients to contain the false-positive rate (0.20–0.38 sensitivity at 0.92–0.96 specificity). As a measure of cognitive ability, a perfect BNT–15 score suggests above average performance; ≤11 suggests clinically significant deficits. Demographically adjusted prorated BNT–15 T-scores correlated strongly (0.86) with the newly developed z-scores. Conclusion: Given its brevity (< 5 minutes), ease of administration and scoring, the BNT–15 can function as a useful and cost-effective screening measure for both object-naming/English proficiency and performance validity. The proposed clinical classification ranges provide useful guidelines for practitioners. Key Words: Boston Naming Test, performance validity, normative data

To examine the potential of the Five-Variable Psychiatric Screener (V-5) to serve as an embedded symptom validity test (SVT). In Study 1, 43 undergraduate students were randomly assigned to a control or an experimental malingering condition. In Study 2, 150 undergraduate students were recruited to examine the cognitive and emotional sequelae of self-reported trauma history. The classification accuracy of the V-5 was computed against the Inventory of Problems (IOP-29), a free-standing SVT. In Study 1, the V-5 was a poor predictor of experimental malingering status, but produced a high overall classification against the IOP-29. In Study 2, the V-5 was a stronger predictor of IOP-29 than self-reported trauma history. Results provide preliminary support for the utility of the V-5 as an embedded SVT. Given the combination of growing awareness of the need to determine the credibility of subjective symptom report using objective empirical methods and systemic pressures to abbreviate assessment, research on SVTs within rapid assessment instruments can provide practical psychometric solutions to this dilemma.

This study was designed to determine the clinical utility of embedded performance validity indicators (EVIs) in adults with intellectual disability (ID) during neuropsychological assessment. Based on previous research, unacceptably high (>16%) base rates of failure (BRFail) were predicted on EVIs using on the method of threshold, but not on EVIs based on alternative detection methods. A comprehensive battery of neuropsychological tests was administered to 23 adults with ID (MAge = 37.7 years, MFSIQ = 64.9). BRFail were computed at two levels of cut-offs for 32 EVIs. Patients produced very high BRFail on 22 EVIs (18.2%-100%), indicating unacceptable levels of false positive errors. However, on the remaining ten EVIs BRFail was <16%. Moreover, six of the EVIs had a zero BRFail, indicating perfect specificity. Consistent with previous research, individuals with ID failed the majority of EVIs at high BRFail. However, they produced BRFail similar to cognitively higher functioning patients on select EVIs based on recognition memory and unusual patterns of performance, suggesting that the high BRFail reported in the literature may reflect instrumentation artefacts. The implications of these findings for clinical and forensic assessment are discussed.

Objective This project was designed to cross-validate existing performance validity cutoffs embedded within measures of verbal fluency (FAS and animals) and develop new ones for the Emotion Word Fluency Test (EWFT), a novel measure of category fluency. Method The classification accuracy of the verbal fluency tests was examined in two samples (70 cognitively healthy university students and 52 clinical patients) against psychometrically defined criterion measures. Results A demographically adjusted T-score of ≤31 on the FAS was specific (.88–.97) to noncredible responding in both samples. Animals T ≤ 29 achieved high specificity (.90–.93) among students at .27–.38 sensitivity. A more conservative cutoff (T ≤ 27) was needed in the patient sample for a similar combination of sensitivity (.24–.45) and specificity (.87–.93). An EWFT raw score ≤5 was highly specific (.94–.97) but insensitive (.10–.18) to invalid performance. Failing multiple cutoffs improved specificity (.90–1.00) at variable sensitivity (.19–.45). Conclusions Results help resolve the inconsistency in previous reports, and confirm the overall utility of existing verbal fluency tests as embedded validity indicators. Multivariate models of performance validity assessment are superior to single indicators. The clinical utility and limitations of the EWFT as a novel measure are discussed.

Objective Administering the noose item of the Boston Naming Test (BNT) has been questioned given the cultural, historical, and emotional salience of the noose in American culture. In response, some have modified the BNT by skipping/removing this item and giving the point as if the examinee responded correctly. It is unknown, however, whether modifying standardized administration and scoring in this manner affects clinical interpretation. In the present study, we examined the prevalence of noose item failure, whether demographic and clinical characteristics differed between those who responded correctly versus failed the item, and whether giving a point to those who failed affected clinical interpretation. Method Participants included a mixed clinical sample of 762 adults, ages 18–88 years, seen for neuropsychological evaluation at one of five sites within the USA. Results Those who failed the item (13.78%) were more likely to be female, non-White, and have primary diagnoses of major neurocognitive disorder, epilepsy, or neurodevelopmental disorder. Noose item failure was associated with lower BNT total score, fewer years of education and lower intellectual functioning, expressive vocabulary, and single word reading. Giving a point to those who failed the item resulted in descriptor category change for 17.1%, primarily for patients with poor overall BNT performance. Conclusions Only a small percentage of patients fail the noose item, but adding a point for these has an impact on score interpretation. Factors associated with poorer overall performance on the BNT, rather than specific difficulty with the noose item, likely account for the findings.