Figure - uploaded by Laszlo A Erdodi
Content may be subject to copyright.
The VI-9: Components, Base Rates of Failure, Cutoffs, and References

The VI-9: Components, Base Rates of Failure, Cutoffs, and References

Source publication
Article
Full-text available
A link between noncredible responding and low scores on the Grooved Pegboard Test (GPB) is well documented in the clinical literature. However, no specific validity cutoffs have emerged in previous research. This study was designed to examine the classification accuracy of various demographically adjusted cutoffs on the GPB against established meas...

Context in source publication

Context 1
... the same time, components that were not administered were counted as a Pass. Table 2 displays base rates of failure (BR Fail ) for each component of the VI-9, the cutoff for each validity indicator, and references from the clinical literature. ...

Citations

... A.Abeare et al., 2021;Deloria et al., 2021;Hurtubise et al., 2020); BNT: Boston Naming Test (L. A.Deloria et al., 2021;Nussbaum et al., 2022;Whiteside et al., 2015); CIM BDAE : Complex Ideational Material subtest of the Boston Diagnostic Aphasia ExaminationErdodi, 2019;Erdodi & Lichtenstein, 2017;Erdodi, Tyson, et al., 2016); DS WAIS-IV : Digit Span subtest of the Wechsler Adult Intelligence Scale -Fourth Edition(Babikian et al., 2006;Erdodi & Abeare, 2020;Greve et al., 2007;Heinly et al., 2005;Iverson & Tulsky, 2003;Shura et al., 2020;Young et al., 2012;Whitney et al., 2009); GPB-DH: Grooved Pegboard Test dominant handErdődi et al., 2017;Link et al., 2021); LM Recog : Logical Memory recognition trial(Bortnik et al., 2010;Dunn et al., 2021;Langeluddecke & Lucas, 2003;Pearson, 2009;J. S. Ord et al., 2008); TMT-B: Part B of the Trail Making Test (C.Abeare et al., 2019;Ashendorf et al., 2017;Erdodi, Hurtubise, et al., 2021;. ...
Article
Full-text available
This study was designed to evaluate the classification accuracy of the Memory module for the Inventory of Problems (IOP-M) in a sample of real-world patients. Archival data were collected from a mixed clinical sample of 90 adults clinically referred for neuropsychological testing. The classification accuracy of the IOP-M was computed against psychometrically defined invalid performance. IOP-M ≤30 produced a good combination of sensitivity (.46-.75) and specificity (.86-.95). Lowering the cutoff to ≤29 improved specificity (.94-1.00) at the expense of sensitivity (.29-.63). The IOP-M correctly classified between 73% and 91% of the sample. Given its low cost, ease of administration/scoring in combination with robust classification accuracy, the IOP-M has the potential to expand the existing toolkit for the evaluation of performance validity during neuropsychological assessments.
... In a mixed clinical sample of 234 adults referred for neuropsychological assessment, the Borderline range was significantly different from both Pass (i.e., stronger evidence of non-credible responding) and Fail (i.e., weaker evidence of non-credible responding). These findings are consistent with the results of previous (Erdodi & Rai, 2017;Erdodi, Sagar, et al., 2018;Erdodi, Seke, et al., 2017) and subsequent Cutler et al., 2021;Dunn et al., 2021;Erdodi, Hurtubise, et al., 2020) investigations. ...
Article
Full-text available
This study was designed to examine the classification accuracy of the Erdodi Index (EI-5), a novel method for aggregating validity indicators that takes into account both the number and extent of performance validity test (PVT) failures. Archival data were collected from a mixed clinical/forensic sample of 452 adults referred for neuropsychological assessment. The classification accuracy of the EI-5 was evaluated against established free-standing PVTs. The EI-5 achieved a good combination of sensitivity (.65) and specificity (.97), correctly classifying 92% of the sample. Its classification accuracy was comparable to that of another free-standing PVT. An indeterminate range between Pass and Fail emerged as a legitimate third outcome of performance validity assessment, indicating that the underlying construct is an inherently continuous variable. Results support the use of the EI-model as a practical and psychometrically sound method of aggregating multiple embedded PVTs into a single-number summary of performance validity. Combining free-standing PVTs with the EI-5 resulted in a better separation between credible and non-credible profiles, demonstrating incremental validity. Findings are consistent with recent endorsements of a three-way outcome for PVTs (Pass, Borderline and Fail).
... Given that all patients were referred by treating physicians and assessed in a clinical context, there was no information available on external incentive status (i.e., motivation to appear impaired). Parts of the sample were used in previous publications focused on different topics (Erdodi, 2019;Erdodi, Pelletier, et al., 2018;Erdodi, Seke, et al., 2017;. ...
Article
Full-text available
Objective : Replicate previous research on Logical Memory Recognition (LMRecog) and perform a critical item analysis. Method : Performance validity was psychometrically operationalized in a mixed clinical sample of 213 adults. Classification of the LMRecog and nine critical items (CR-9) was computed. Results : LMRecog ≤20 produced a good combination of sensitivity (.30-.35) and specificity (.89-.90). CR-9 ≥5 and ≥6 had comparable classification accuracy. CR-9 ≥5 increased sensitivity by 4% over LMRecog ≤20; CR-9 ≥6 increased specificity by 6–8% over LMRecog ≤20; CR-9 ≥7 increased specificity by 8–15%. Conclusions : Critical item analysis enhances the classification accuracy of the optimal LMRecog cutoff (≤20).
... The multidimensional aspect of the updated MND criteria highlights the need for PVTs that can assess non-credible performance in all areas of functioning, including motor performance.While limited, investigations on the use of motor tasks as embedded PVTs have been promising. The majority of these studies have focused on the Finger Tapping Test (Arnold et al., 2005;Arnold & Boone, 2007;Backhaus et al., 2004;Greiffenstein et al., 1996;Larrabee, 2003;) and despite common use of the Grooved Pegboard (GPB), the test has not been routinely considered as a potential PVT (Arnold & Boone, 2007;Ashendorf et al., 2009;Erdodi et al., 2017;Tolle et al., 2020). ...
... To address this gap, Erdodi et al. (2017) evaluated the utility of the GPB as an embedded PVT and sought to establish demographically adjusted T-score cutoffs. ...
... T-score cutoffs set at ≥ 29 for either hand or ≥ 31 for both handsyieldedsensitivity and specificity values that were consistent with minimum levels generally accepted within the field (Boone, 2013;Larrabee, 2003Larrabee, , 2008Lichtenstein et al., 2017;Sherman et al., 2020). Table 1 provides a summary of Erdodi et al. (2017) classification statistics for the recommended T-score cutoffs. Erdodi et al., concluded that their recommended GPB cutoffs were sufficient in detecting non-credible performance in a clinical sample. ...
Article
Objective Using embedded performance validity (PVT) comparisons, Erdodi et al. suggested that Grooved Pegboard (GPB) T-score cutoffs for either hand (≤ 29) or both hands (≤ 31) could be used as additional embedded PVTs. The current study evaluated the relationship between these proposed cutoff scores and established PVTs (Medical Symptom Validity Test [MSVT]; Non-Verbal Medical Symptom Validity Test [NV-MSVT], and Reliable Digit Span [RDS]). Method Participants (N = 178) were predominately Caucasian (84%) males (79%) with a mean age and education of 41 (SD = 11.7) and 15.8 years (SD = 2.3), respectively. Participants were stratified as “passing” or “failing” the GPBviaErdodi’s proposed criteria. “Failures” on the MSVT, NV-MSVT, and RDS were based on conventional recommendations. Results Moderate correlations between GPB classification and a condition of interest (COI; i.e. at least two failures on reference PVTs) were observed for dominant (χ² (1, n = 178) = 34.72, ϕ = .44, p < .001), non-dominant (χ² (1, n = 178) = 16.46, ϕ = .30, p = .001), and both hand conditions (χ² (1, n = 178) = 32.48, ϕ = .43, p < .001). Sensitivity, specificity, and predictive power were generally higher than Erdodi et al.’s initial findings. Conclusion These findingsprovide supportfor the clinical utility of the GPB as an additional embedded PVT. More specifically, dominant and both hand cutoffs were found to be more robust measures ofnon-genuine performance in those without motor deficits. While promising, sensitivity continues to be low; therefore, it is ill-advised to use the GPB as a sole measure of ­performance validity.
... see Supplementary Table 2). Finally, we scored three embedded performance validity measures within the dataset using the CVLT-II (Wolfe et al., 2010), grooved Pegboard (Erdődi et al., 2017), and Trail-Making Test (Iverson et al., 2002). Only 29 individuals (13.4%) scored below cut-offs on any single index and zero participants failed two or more of these embedded measures. ...
Article
Objective Black Americans are at high risk for HIV disease and associated morbidity. The impact and clinical correlates of HIV-associated neurocognitive impairment among Black Americans is not fully understood. The current study uses a full factorial design to examine the independent and combined effects of race and HIV disease on neurocognitive functioning, including its associations with everyday functioning and clinical disease markers in Black and White persons with HIV (PWH). Method Participants included 40 Black PWH, 83 White PWH, 28 Black HIV- and 64 White HIV- individuals. Neurocognition was measured by raw sample-based z-scores from a clinical battery. Everyday functioning was assessed using self- and clinician-rated measures of cognitive symptoms and activities of daily living. HIV-associated neurocognitive disorders were also classified using demographically adjusted normative standards and the Frascati criteria. Results We observed a significant three-way interaction between HIV, race, and domain on raw neurocognitive z-scores. This omnibus effect was driven by medium and large effect size decrements in processing speed and semantic memory, respectively, in Black PWH compared to other study groups. Black PWH also demonstrated higher frequencies of HIV-associated neurocognitive disorders as compared to White PWH. Unexpectedly, global neurocognitive performance was negatively related to everyday functioning impairments for White PWH, but not for Black PWH. Conclusions Systemic disadvantages for Black Americans may combine with HIV disease to compound some neurocognitive impairments in this under-served population. Prospective studies are needed to identify better ways to prevent, measure, diagnose, and manage HIV-associated neurocognitive disorders among Black Americans.
... Empirical support for the PIH comes from several different sources: correlations between PVT failures or performance on cognitive tests and psychiatric diagnoses/symptoms in clinical patients (Bell et al., 1999;Erdodi et al., 2018a, e;Erdodi et al., 2017c;Eysenck et al., 2007;Jak et al., 2019;Miskey et al., 2020;Qureshi et al., 2011;Rock et al., 2014;Rowland et al., 2017;Snyder, 2013;Tyson et al., 2018), experimental studies (Bakvis et al., 2010), systematic observations of individuals during the natural course of recovery (Losoi et al., 2016), and changes in PVT failure rates as a result of interventions targeting the underlying emotional problems . ...
... Dr. Erdodi provides forensic consultation and medicolegal assessments, for which he receives financial compensation. Curtis et al., 2006;Millis et al., 1995;Shura et al., 2016;Trueblood, 1994); RH: Yes/No Recognition hits (true positives) raw score (Persinger et al., 2018;Sweet et al., 2000;Trueblood, 1994;Wolfe et al., 2010); TMT Trail Making Test demographically adjusted T-score (Abeare et al., 2019;Erdodi & Lichtenstein, 2020) based on norms by Heaton et al. (2004); BNT Boston Naming Test demographically adjusted T-score Erdodi et al., 2018b) based on norms by Heaton et al. (2004); FTT DH Finger Tapping Test dominant hand demographically adjusted T-score based on norms by Heaton et al. (2004); GPB DH Grooved Pegboard Test dominant hand demographically adjusted T-score (Erdodi et al., 2018c;Erdodi et al., 2017c) ...
Article
Full-text available
This study was designed to examine the relative contribution of symptom (SVT) and performance validity tests (PVTs) to the evaluation of the credibility of neuropsychological profiles in mild traumatic brain injury (mTBI). An archival sample of 326 patients with mTBI was divided into four psychometrically defined criterion groups: pass both SVT and PVT; pass one, but fail the other; and fail both. Scores on performance-based tests of neurocognitive ability and self-reported symptom inventories were compared across the groups. As expected, PVT failure was associated with lower scores on ability tests (ηp2 .042–.184; d 0.56–1.00; medium-large effects), and SVT failure was associated with higher levels of symptom report (ηp2 .039–.312; d 0.32–1.58; small-very large effects). However, SVT failure also had a marginal deleterious effect on performance based measures (ηp2 .017–.023; d 0.23–0.46; small-medium effects) and elevations on self-report inventories were observed in the context of PVT failure (ηp2 .026; d 0.23–0.57; small-medium effects). SVT failure was associated with not only inflated symptom reports but also distorted configural patterns of psychopathology. Patients with clinically elevated somatic and depressive symptoms were twice as likely to fail PVTs. Consistent with previous research, SVTs and PVTs provide overlapping, but non-redundant information about the credibility of neuropsychological profiles associated with mTBI. Therefore, they should be used in combination to afford a comprehensive evaluation of cognitive and emotional functioning. The heuristic value of validity tests has both clinical and forensic relevance.
... The multidimensional aspect of the updated MND criteria highlights the need for PVTs that can assess non-credible performance in all areas of functioning, including motor performance.While limited, investigations on the use of motor tasks as embedded PVTs have been promising. The majority of these studies have focused on the Finger Tapping Test (Arnold et al., 2005;Arnold & Boone, 2007;Backhaus et al., 2004;Greiffenstein et al., 1996;Larrabee, 2003;) and despite common use of the Grooved Pegboard (GPB), the test has not been routinely considered as a potential PVT (Arnold & Boone, 2007;Ashendorf et al., 2009;Erdodi et al., 2017;Tolle et al., 2020). ...
... To address this gap, Erdodi et al. (2017) evaluated the utility of the GPB as an embedded PVT and sought to establish demographically adjusted T-score cutoffs. ...
... T-score cutoffs set at ≥ 29 for either hand or ≥ 31 for both handsyieldedsensitivity and specificity values that were consistent with minimum levels generally accepted within the field (Boone, 2013;Larrabee, 2003Larrabee, , 2008Lichtenstein et al., 2017;Sherman et al., 2020). Table 1 provides a summary of Erdodi et al. (2017) classification statistics for the recommended T-score cutoffs. Erdodi et al., concluded that their recommended GPB cutoffs were sufficient in detecting non-credible performance in a clinical sample. ...
Article
Objective Previously, the Grooved Pegboard Test (GPB) has shown potential as an embedded measure of performance validity (PVT) using a T-score cutoff for either hand (≤ 29) or both hands (≤ 31). This study sought to validate these cutoffs with established PVTs (Medical Symptom Validity Test [MSVT], Non-Verbal Medical Symptom Validity Test [NV-MSVT], and Reliable Digit Span [RDS]). Method Subjects (N = 190) were primarily Caucasian (85%) and male (81%). Average age and education were 41 (SD = 11.62) and 16 years (SD = 2.35), respectively. Participants were stratified as GPB “pass or fail” based on previously proposed cutoff criteria. MSVT, NV-MSVT, and RDS were also dichotomized as pass or fail based on manual or conventional recommendations. Results Chi-Square analyses revealed significant associations between GPB “fails” for both hands and MSVT, NV-MSVT, and RDS (χ2 (1, n = 190) ranging from 5.80 to 15.98, Phi ranging from .18 to .29, p ranging from < .05 to < .0001). Similar findings were observed for dominant hand “fails”; however, non-dominant hand was only related to the MSVT. Sensitivity and specificity values from the GPB measures ranged from .47 to .58 and from .89 to .92, respectively. Positive and negative predictive power ranged from .38 to .45 and .93 to .94, respectively. Conclusion These data demonstrate the relative utility of the GPB as an embedded PVT. In particular, dominant and both hand cutoffs are likely to be more clinically useful in determining sub-optimal performance. However, as sensitivity is relatively low, this measure should not be employed as the sole PVT administered.
... EI-7: Erdodi Index Seven; BNT-15: Boston Naming Test -Short Form; CD WAIS-III : Coding subtest of the Wechsler Adult Intelligence Scale -Third Edition (age-corrected scaled score;Erdodi & Abeare, 2020;Etherton, Bianchini, Heinly, & Greve, 2006;Inman & Berry, 2002;N. Kim et al., 2010;Trueblood, 1994); CIM: Complex Ideational Material subtest of the Boston Diagnostic Aphasia BatteryErdodi et al., 2016;; FMS WCST-64 : Failure to maintain set (raw score) for the 64-card version of the Wisconsin Card Sorting Test(Greve, Bianchini, Mathias, Houston, & Crouch, 2002;Greve et al., 2009); GPB Dominant: Grooved Pegboard Test dominant hand (demographically adjusted Tscore using norms byHeaton et al., 2004;Erdodi, Seke et al., 2017); RDS: Reliable Digit Span(Greiffenstein, Gola, & Baker, 1994); TMT-A: Trail Making Test -Part A (T-score;Abeare, Sabelli, et al., 2019;Ashendorf, Clark, & Sugarman, 2017;Erdodi & Lichtenstein, 2020). ...
Article
Full-text available
Objective This study was designed to replicate previous research on embedded validity indicators (EVIs) in the Hopkins Verbal Learning Test – Revised (HVLT-R) and introduce a new forced choice recognition trial (FCR). Method Undergraduate research volunteers were randomly assigned (M Age = 21.7; M Education = 14.5 years, 85% female) to either the control or experimental malingering condition, and were administered a brief battery of neuropsychological tests. Results Recognition memory based EVIs (both existing and newly introduced) effectively discriminated credible and non-credible response sets. An FCR ≤11 produced .59 sensitivity and perfect specificity to invalid responding. A Recognition Discrimination (RD) score ≤8 also produced a good combination of sensitivity (.35) and specificity (.96). The FCR trial made unique contributions to performance validity assessment above and beyond previously published EVIs. Conclusions RD achieved ≥.90 specificity at higher cutoffs than previously reported. The newly introduced FCR trial has the potential to enhance the existing arsenal of EVIs within the HVLT-R. However, it must demonstrate its ability to differentiate genuine impairment from non-credible responding before it can be recommended for clinical use.
... 32 BR Fail for the EVIs in this study may not be representative, due to the restrictive nature of the sample (all male collegiate football athletes). Future research should examine the interaction of demographic 33,34 and clinical characteristics, such as psychiatric and psychological factors [35][36][37] BR Fail and classification accuracies of the EVIs ImPACT . In addition, reasons behind the poor signal detection performance of certain EVIs ImPACT may warrant further investigation. ...
Article
Full-text available
Objective: To create novel Immediate Post-Concussion and Cognitive Testing (ImPACT)-based embedded validity indicators (EVIs) and to compare the classification accuracy to 4 existing EVIImPACT. Method: The ImPACT was administered to 82 male varsity football players during preseason baseline cognitive testing. The classification accuracy of existing EVIImPACT was compared with a newly developed index (ImPACT-5A and B). The ImPACT-5A represents the number of cutoffs failed on the 5 ImPACT composite scores at a liberal cutoff (0.85 specificity); ImPACT-5B is the sum of failures on conservative cutoffs (≥0.90 specificity). Results: ImPACT-5A ≥1 was sensitive (0.81), but not specific (0.49) to invalid performance, consistent with EVIImPACT developed by independent researchers (0.68 sensitivity at 0.73-0.75 specificity). Conversely, ImPACT-5B ≥3 was highly specific (0.98), but insensitive (0.22), similar to Default EVIImPACT (0.04 sensitivity at 1.00 specificity). ImPACT-5A ≥3 or ImPACT-5B ≥2 met forensic standards of specificity (0.91-0.93) at 0.33 to 0.37 sensitivity. Also, the ImPACT-5s had the strongest linear relationship with clinically meaningful levels of invalid performance of existing EVIImPACT. Conclusions: The ImPACT-5s were superior to the standard EVIImPACT and comparable to existing aftermarket EVIImPACT, with the flexibility to optimize the detection model for either sensitivity or specificity. The wide range of ImPACT-5 cutoffs allows for a more nuanced clinical interpretation.
... While individuals with mental health disorders may have suppressed cognitive scores in the context of valid PVT performance (e.g., Considine, Weisenbach, Walker, McFadden, Franti et al., 2011), individuals with mental health disorders are also more likely to obtain PVT scores that suggest insufficient engagement (Clark et al., 2014;Rowland et al., 2017;Williamson et al., 2014). This may reflect "psychogenic interference" (Erdődi et al., 2017), in which individuals with mental health disorders are unable to put forth maximal effort due to emotional distress, resulting in internally inconsistent patterns of performance. Problematically, PVT studies with college students either exclude participants with psychiatric or cognitive disorders (e.g., Santos, Kazakov, Reamer, Park, & Osmon, 2014;Ross et al., 2016) or do not report screening for clinical conditions (e.g., DeRight & Jorgensen, 2015;Silk-Eglit et al., 2014). ...
Article
Full-text available
Performance validity tests (PVTs) effectively detect suboptimal test performance, but cutoff scores for a given test may differ between populations. This research examines how optimal PVT cutoffs differ in a collegiate research population when mental health and clinical conditions are considered. Healthy controls (n = 328) and non-referred students with self-reported clinical conditions (n = 42) were assigned to perform their best while others simulated ADHD symptoms (n = 123). PVT indices were derived from a stand-alone measure (Victoria Symptom Validity Test) and embedded measures (California Verbal Learning Test – Second Edition; Wechsler Adult Intelligence Scale – Fourth Edition, Digit Span). PVT cutoffs with the highest sensitivity, while maintaining adequate specificity, were identified when the control groups were considered together, and when students with reported clinical conditions were considered separately. Mean differences in PVT performances were found between the simulation group and control groups, but not between clinical and nonclinical controls. The optimized cutoffs differed for five of eight PVT indices when all controls were considered together versus the clinical control group, only. When discordance was observed, cutoffs tended to be lower (less stringent) for the non-referred clinical control group. Together, these optimized cutoffs tended to be more stringent than previously established cutoffs. This study suggests that PVT cutoffs may be responsibly altered in a research context in the presence of a clinical condition. Future research should investigate if PVT classification accuracies can be improved in clinical and forensic samples while considering clinical conditions.