Fig 2 - uploaded by Laszlo A Erdodi
Content may be subject to copyright.
SCL-90-R profiles associated with the three levels of EI-5 REC performance; number of participants in the Pass range (0-1) is 51; number of participants in the Borderline range (2-3) is 18; number of participants in the Fail range (≥4) is 29. 

SCL-90-R profiles associated with the three levels of EI-5 REC performance; number of participants in the Pass range (0-1) is 51; number of participants in the Borderline range (2-3) is 18; number of participants in the Fail range (≥4) is 29. 

Source publication
Article
Full-text available
Objective The Forced Choice Recognition (FCR) trial of the California Verbal Learning Test—Second Edition (CVLT-II) was designed to serve as a performance validity test (PVT). The present study was designed to compare the classification accuracy of a more liberal alternative (≤15) to the de facto FCR cutoff (≤14). Method The classification accurac...

Contexts in source publication

Context 1
... was between the Pass and Fail groups, with effect sizes ranging from .43 (medium) to .87 (large). Unlike with FCR, there was a linear relationship between level of PVT failure and self-reported emotional distress, with the Pass group reporting the least, the Fail group reporting the most emotional distress, with the Borderline group in the middle (Fig. 2). Note. ...
Context 2
... between the Pass and Fail groups, with effect sizes ranging from .43 (medium) to .87 (large). Unlike with FCR, there was a linear relationship between level of PVT failure and self-reported emotional distress, with the Pass group re- porting the least, the Fail group reporting the most emotional distress, with the Borderline group in the middle (Fig. 2). Note. ...

Citations

... This is not standard practice, as typically studies will utilize another robust PVT as a means of calculating sensitivity and specificity values for another PVT such as the TOMM. While an embedded PVT is not as robust as a stand-alone measure, the CVLT-II is still a reasonable embedded test to use when calculating sensitivity and specificity, as its features are consistent with stand-alone PVTs (Erdodi et al., 2018). ...
... Therefore, only an embedded PVT was available for use. An embedded PVT is not as robust as a stand-alone measure; however, the CVLT-II is still a reasonable embedded test to use when calculating sensitivity and specificity, as its features are consistent with standalone PVTs, and was developed with the primary purpose of evaluating response validity (Bauer et al., 2005;Erdodi et al., 2018;Schwartz et al., 2016). ...
Article
Full-text available
The accuracy of neuropsychological assessments relies on participants exhibiting their true abilities during administration. The Test of Memory Malingering (TOMM) is a popular performance validity test used to determine whether an individual is providing honest answers. While the TOMM has proven to be highly sensitive to those who are deliberately exaggerating their symptoms, there is a limited explanation regarding the significance of using 45 as a cutoff score. The present study aims to further investigate this question by examining TOMM scores obtained in a large sample of active-duty military personnel (N = 859, M = 26 years, SD = 6.14, 97.31% males, 72.44% white). Results indicated that no notable discrepancies existed between the frequency of participants who scored a 45 and those who scored slightly below a 45 on the TOMM. The sensitivity and specificity of the TOMM were derived using the forced-choice recognition (FCR) scores obtained by participants on the California Verbal Learning Test, Second Edition (CVLT-II). The sensitivity for each trial of the TOMM was 0.84, 0.55, and 0.63, respectively; the specificity for each trial of the TOMM was 0.69, 0.93, and 0.92, respectively. Because sensitivity and specificity rates are both of importance in this study, balanced accuracy scores were also reported. Results suggested that various alternative cutoff scores produced a more accurate classification compared to the traditional cutoff of 45. Further analyses using Fisher's exact test also indicated that there were no significant performance differences on the FCR of the CVLT-II between individuals who received a 44 and individuals who received a 45 on the TOMM. The current study provides evidence on why the traditional cutoff may not be the most effective score. Future research should consider employing alternative methods which do not rely on a single score.
... FCR has been deemed highly specific and moderately sensitive to invalid responses in nonimpaired individuals (Schwartz et al., 2016), and has reasonable concurrent validity with other measures, such as the Test of Memory Malingering . It has also been employed in a brain injured sample, with results suggesting that even a single error on FCR is sufficient to identify invalid responses (Erdodi et al., 2018). The sole piece of the literature identified about the CVLT-II-SF FCR in dementia suggests a similar conclusion. ...
... In the literature, there have been several reports of appropriate cut scores for performance validity determination using FCR. For example, Erdodi et al. (2018) reported a potentially useful cutoff of one error on the CVLT-II FCR in a TBI sample, finding it unrelated to TBI-sensitive measures. A similar finding was reported by Fogel et al. (2013) in their dementia sample. ...
... They reported a potential cutoff of 8/9 for the CVLT-II-SF FCR as a measure of performance validity in a dementia sample, while acknowledging the significant relationship between performance validity test performance and cognition that has been replicated here and elsewhere (e.g., Burton et al., 2015;Dean et al., 2009). Given that the present mean CVLT-II-SF FCR value for those diagnosed with AD was 7.96 (SD ¼ 0.12), it may not be appropriate to ascertain poor performance validity based on existing suggestions for single-error cutoff scores (Erdodi et al., 2018;Fogel et al., 2013) in dementia. It is still possible that the proposed literature-based cutoff of one error is adequate for discerning performance validity (aligning with Schwartz et al. 2016) when combined with additional measures of performance validity and clinical judgment. ...
Article
Full-text available
Performance validity tests are susceptible to false positives from genuine cognitive impairment (e.g., dementia); this has not been explored with the short form of the California Verbal Learning Test II (CVLT-II-SF). In a memory clinic sample, we examined whether CVLT-II-SF Forced Choice Recognition (FCR) scores differed across diagnostic groups, and how the severity of impairment [Clinical Dementia Rating Sum of Boxes (CDR-SOB) or Mini-Mental State Examination (MMSE)] modulated test performance. Three diagnostic groups were identified: subjective cognitive impairment (SCI; n = 85), amnestic mild cognitive impairment (a-MCI; n = 17), and dementia due to Alzheimer's Disease (AD; n = 50). Significant group differences in FCR were observed using one-way ANOVA; post-hoc analysis indicated the AD group performed significantly worse than the other groups. Using multiple regression, FCR performance was modeled as a function of the diagnostic group, severity (MMSE or CDR-SOB), and their interaction. Results yielded significant main effects for MMSE and diagnostic group, with a significant interaction. CDR-SOB analyses were non-significant. Increases in impairment disproportionately impacted FCR performance for persons with AD, adding caution to research-based cutoffs for performance validity in dementia. Caution is warranted when assessing performance validity in dementia populations. Future research should examine whether CVLT-II-SF-FCR is appropriately specific for best-practice testing batteries for dementia.
... The cognitive impairment group obtained a mean score of 43.9 (5.3) for trial 1 and a mean score of 48.6 (3.1) for trial 2, suggesting that performance on the TOMM is very resistant to different types of severe cognitive impairment (Tombaugh, 1996). While it is traditional to use the cutoff score of <45 on trial 2 as suggested in the manual, this study also examined alternative cutoffs for trial 2 based on existing literature (e.g., Erdodi et al., 2018;Martin et al., 2020) in addition to examining trial 1 data as part of secondary analyses. ...
... Table 5 contains the number of individuals who performed at various cutoffs on the CVLT-FC at three time points. Cutoff scores were determined using existing literature (e.g., Erdodi et al., 2018;Schwartz et al., 2016). One cutoff was based on a systematic review conducted by Schwartz and colleagues (2016) in which they applied a cutoff of 14 on the forced choice trial (sensitivity 50% and specificity 93%). ...
... One cutoff was based on a systematic review conducted by Schwartz and colleagues (2016) in which they applied a cutoff of 14 on the forced choice trial (sensitivity 50% and specificity 93%). Another, more stringent cutoff, was based on a study conducted with a group of mixed TBI individuals (75% mTBI) (Erdodi et al., 2018). In that study, they used a cutoff of 15 on the forced choice trial (sensitivity 56% and specificity 92%). ...
Article
Full-text available
Objective Assessing performance validity is imperative in both clinical and research contexts as data interpretation presupposes adequate participation from examinees. Performance validity tests (PVTs) are utilized to identify instances in which results cannot be interpreted at face value. This study explored the hit rates for two frequently used PVTs in a research sample of individuals with and without histories of bipolar disorder (BD). Method As part of an ongoing longitudinal study of individuals with BD, we examined the performance of 736 individuals with BD and 255 individuals with no history of mental health disorder on the Test of Memory Malingering (TOMM) and the California Verbal Learning Test forced choice trial (CVLT-FC) at three time points. Results Undiagnosed individuals demonstrated 100% pass rate on PVTs and individuals with BD passed over 98% of the time. A mixed effects model adjusting for relevant demographic variables revealed no significant difference in TOMM scores between the groups, a = .07, SE = .07, p = .31. On the CVLT-FC, no clinically significant differences were observed ( ps < .001). Conclusions Perfect PVT scores were obtained by the majority of individuals, with no differences in failure rates between groups. The tests have approximately >98% specificity in BD and 100% specificity among non-diagnosed individuals. Further, nearly 90% of individuals with BD obtained perfect scores on both measures, a trend observed at each time point.
... In contrast, malingering is often associated with indiscriminate symptom exaggeration and fabrication (American Psychiatric Association, 2013). Distinguishing between these two presentations is a challenging (Boone, 2017;Sullivan & King, 2010) yet consequential differential diagnosis that has a significant impact on patient outcome and resource allocation (Chafetz & Underhill, 2013;Erdodi, Abeare et al., 2018;. Therefore, the combination of the two samples provides a valuable comparison of the V-5's classification accuracy in two different signal detection environments: experimentally induced gross symptom exaggeration in the absence of genuine psychopathology versus no incentive to over-report in the context of possible genuine psychopathology. ...
Article
Full-text available
To examine the potential of the Five-Variable Psychiatric Screener (V-5) to serve as an embedded symptom validity test (SVT). In Study 1, 43 undergraduate students were randomly assigned to a control or an experimental malingering condition. In Study 2, 150 undergraduate students were recruited to examine the cognitive and emotional sequelae of self-reported trauma history. The classification accuracy of the V-5 was computed against the Inventory of Problems (IOP-29), a free-standing SVT. In Study 1, the V-5 was a poor predictor of experimental malingering status, but produced a high overall classification against the IOP-29. In Study 2, the V-5 was a stronger predictor of IOP-29 than self-reported trauma history. Results provide preliminary support for the utility of the V-5 as an embedded SVT. Given the combination of growing awareness of the need to determine the credibility of subjective symptom report using objective empirical methods and systemic pressures to abbreviate assessment, research on SVTs within rapid assessment instruments can provide practical psychometric solutions to this dilemma.
... During the past few decades, many embedded and freestanding PVTs have been developed, validated, researched, and adapted for use in the forensic context (Boone, 2013;Rogers & Bender, 2018). Well-known examples of PVTs are the Test of Memory Malingering (TOMM; Tombaugh, 1996) and Word Memory Test (WMT; Green et al., 1996), among free-standing measures, and the Reliable Digit Span (RDS) of the Wechsler Adult Intelligence Scale (e.g., Axelrod et al., 2006;Babikian et al., 2006;Erdodi & Abeare, 2020;Greiffenstein et al., 1994;Reese et al., 2012) and Forced Choice Recognition Trial of the California Verbal Learning Test (Erdodi et al., 2018;Greve et al., 2009;Slick et al., 2000;Wolfe et al., 2010), among embedded measures. ...
... Moreover, from a practical perspective, assessing symptom and performance validity is notably different from assessing other constructs in medicine and neuropsychology (Chafetz, 2020). For example, NRB levels are likely to vary across different measures of NRB, because examinees often deliberately choose to restrict their NRB to a very limited number of domains of psychological functioning and do well in other domains (e.g., to appear cognitively impaired, someone might deliberately try to pretend to be unable to perform mathematical calculations but perform well on memory tasks) (Cottingham et al., 2014;Erdodi et al., 2018). From this standpoint, using SVTs that provide information about the same evaluee from different angles, e.g., by relying on different detection strategies or by focusing on multiple domains (such as somatic, cognitive, and psychiatric) might, therefore, be more beneficial than using SVTs that use the same detection strategy or focus on the same one symptom domain. ...
... That said, they should not be dismissed when there are significantly elevated results. These findings would need to be considered carefully for all possible interpretations, with the best one based on the overall pattern of information, data inconsistencies, and data gathered throughout the evaluation (Erdodi et al., 2018;Merten & Merckelbach, 2013;Young, 2019Young, , 2021. ...
Article
Full-text available
In psychological injury and related forensic evaluations, two types of tests are commonly used to assess Negative Response Bias (NRB): Symptom Validity Tests (SVTs) and Performance Validity Tests (PVTs). SVTs assess the credibility of self-reported symptoms, whereas PVTs assess the credibility of observed performance on cognitive tasks. Compared to the large and ever-growing number of published PVTs, there are still relatively few validated self-report SVTs available to professionals for assessing symptom validity. In addition, while several studies have examined how to combine and integrate the results of multiple independent PVTs, there are few studies to date that have addressed the combination and integration of information obtained from multiple self-report SVTs. The Special Issue of Psychological Injury and Law introduced in this article aims to help fill these gaps in the literature by providing readers with detailed information about the convergent and incremental validity, strengths and weaknesses, and applicability of a number of selected measures of NRB under different conditions and in different assessment contexts. Each of the articles in this Special Issue focuses on a particular self-report SVT or set of SVTs and summarizes their conditions of use, strengths, weaknesses, and possible cut scores and relative hit rates. Here, we review the psychometric properties of the 19 selected SVTs and discuss their advantages and disadvantages. In addition, we make tentative proposals for the field to consider regarding the number of SVTs to be used in an assessment, the number of SVT failures required to invalidate test results, and the issue of redundancy when selecting multiple SVTs for an assessment.
... A subsequent study by Erdodi et al. (2018) based on a mixed clinical sample of 214 found that a more liberal cutoff (T ≤ 37) achieved high specificity (.87-.95), but low and variable sensitivity (.15-.35). A raw score cutoff of ≤12 on the short form (BNT-15) produced uniformly high specificity (.92-.98), but even lower sensitivity (.03-.27). ...
Article
Full-text available
This study was designed to examine alternative validity cutoffs on the Boston Naming Test (BNT). Archival data were collected from 206 adults assessed in a medicolegal setting following a motor vehicle collision. Classification accuracy was evaluated against three criterion PVTs. The first cutoff to achieve minimum specificity (.87-.88) was T ≤ 35, at .33-.45 sensitivity. T ≤ 33 improved specificity (.92-.93) at .24-.34 sensitivity. BNT validity cutoffs correctly classified 67–85% of the sample. Failing the BNT was unrelated to self-reported emotional distress. Although constrained by its low sensitivity, the BNT remains a useful embedded PVT.
... Indeed, numerous studies have demonstrated response invalidity across multiple medical groups that are not in litigation. For example, response invalidity has been noted among patients with fibromyalgia (Gervais et al., 2000), memory disorder clinic outpatients (Howe et al., 2007), adults with traumatic brain injury (Erdodi et al., 2018;Carone, 2008), university students undergoing assessments of attention-deficit hyperactivity and learning disorders (Sullivan et al., 2007), and individuals experiencing psychogenic non-epileptic events (Drane et al., 2006). ...
Article
Full-text available
As overreporting of symptoms threatens the integrity and utility of the neuropsychological evaluation, symptom validity test (SVT) usage has become a standard of practice. The Miller Forensic Assessment of Symptoms Test (M-FAST) SVT has been validated in various forensic contexts, though its utility in specific medical settings remains under-explored. The current study examined the validity and diagnostic utility of the M-FAST among 123 inpatient veterans on a long-term video electroencephalogram (V-EEG) epilepsy monitoring unit (EMU). Select scores from the Structured Inventory of Malingered Symptomatology and the Minnesota Multiphasic Personality Inventory-2-Restructured Form were combined into an SVT composite used as a criterion for validating the M-FAST with receiver operating characteristic (ROC) curves. A Student’s t-test revealed significantly higher total M-FAST scores among the symptom invalidity group compared to individuals with valid symptom reports (Cohen’s d = 1.24). An optimal M-FAST cut score of ≥ 5 was identified to detect symptom overreporting, with .65 sensitivity and .85 specificity (AUC = .82). However, ROC curve analysis indicated that the M-FAST had poor diagnostic classification accuracy for V-EEG-confirmed epilepsy (ES; n = 21) versus V-EEG-confirmed psychogenic non-epileptic events (PNEE; n = 46; AUC = .56). Although diagnostic utility of the M-FAST for PNEE versus ES was not supported within our sample, results demonstrate the validity of the M-FAST as an effective SVT in an EMU. Findings highlight that various cut scores may be considered to optimize detection of symptom overreporting in settings where comorbid physiological and psychological conditions may influence response validity.
... Classification accuracy ranged from 0.68 (C-FCR ≤ 15) to 0.77 (C-RH ≤ 10 and B-RH ≤ 4). Various cutoff scores and their sensitivity, specificity, and total accuracy values are displayed in Table 2. [49]. b [37]. ...
Article
Full-text available
Within the neuropsychological assessment, clinicians are responsible for ensuring the validity of obtained cognitive data. As such, increased attention is being paid to performance validity in patients with multiple sclerosis (pwMS). Experts have proposed batteries of neuropsychological tests for use in this population, though none contain recommendations for standalone performance validity tests (PVTs). The California Verbal Learning Test, Second Edition (CVLT-II) and Brief Visuospatial Memory Test, Revised (BVMT-R)—both of which are included in the aforementioned recommended neuropsychological batteries—include previously validated embedded PVTs (which offer some advantages, including expedience and reduced costs), with no prior work exploring their utility in pwMS. The purpose of the present study was to determine the potential clinical utility of embedded PVTs to detect the signal of non-credibility as operationally defined by below criterion standalone PVT performance. One hundred thirty-three (133) patients (M age = 48.28; 76.7% women; 85.0% White) with MS were referred for neuropsychological assessment at a large, Midwestern academic medical center. Patients were placed into “credible” (n = 100) or “noncredible” (n = 33) groups based on a standalone PVT criterion. Classification statistics for four CVLT-II and BVMT-R PVTs of interest in isolation were poor (AUCs = 0.58–0.62). Several arithmetic and logistic regression-derived multivariate formulas were calculated, all of which similarly demonstrated poor discriminability (AUCs = 0.61–0.64). Although embedded PVTs may arguably maximize efficiency and minimize test burden in pwMS, common ones in the CVLT-II and BVMT-R may not be psychometrically appropriate, sufficiently sensitive, nor substitutable for standalone PVTs in this population. Clinical neuropsychologists who evaluate such patients are encouraged to include standalone PVTs in their assessment batteries to ensure that clinical care conclusions drawn from neuropsychological data are valid.
... Nonetheless, including information on the presence of financial incentives in future studies would be helpful, as they have been associated with prolonged endorsement of post-concussive symptoms and poorer cognitive test performance after mTBI (Belanger et al., 2005;Binder & Rohling, 1996). Furthermore, while one prior study of mTBI did not observe a relationship between BRIEF-A index scores and performance on the Test of Memory Malingering (Donders et al., 2015), a recent study involving university students and a mixed clinical sample observed that BRIEF-A validity scales (using cutoff scores suggested in the test manual) were not sensitive to invalid test performance as reflected by PVTs (Abeare et al., 2021) It should be noted that while some studies have not found an association between symptom validity measures and PVTs (Aase et al., 2020;Menatti et al., 2020), many others have observed a relationship (e.g., Boone & Lu, 1999;Erdodi et al., 2018;Gaasedelen et al., 2019;Lange et al., 2012;Sabelli et al., 2021). Thus, further research examining the relationship of subjective executive functioning to both independent symptom validity measures and PVTs in mTBI would be informative. ...
Article
Full-text available
Prior research has found that individuals with a pre-injury psychiatric history report greater difficulty with executive functioning after mild traumatic brain injury (mTBI). The present study examined self-rated executive functioning after the acute phase of recovery in individuals with mTBI having no prior psychiatric history. Participants included 59 individuals with mTBI and 27 healthy comparison subjects (HC). They completed the Behavior Rating Inventory of Executive Function-Adult Version (BRIEF-A) and Beck Depression Inventory-II. Participants had no prior history of psychiatric diagnosis based on semi-structured interview. In those with a valid BRIEF-A, the mTBI group (n = 54; days since injury: M = 465.04, SD = 146.92, range = 315–1144) endorsed poorer executive functioning than the HC group (n = 24) on several BRIEF-A scales. Only the Working Memory and Inhibit scales remained significant after adjusting for group differences in education and depression score, but they did not survive adjustment for multiple comparisons. The mean standardized scores for the mTBI group were well within normal limits across scales. Furthermore, there were no group differences on any BRIEF-A scales for percentage of participants with clinically elevated scores. Individuals with mTBI and no prior psychiatric history did not endorse significant executive dysfunction. Our findings support previous literature indicating that self-rated executive dysfunction in those with mTBI after the acute phase of recovery is likely due to factors other than brain injury, such as subclinical symptoms of depression.
... 32 BR Fail for the EVIs in this study may not be representative, due to the restrictive nature of the sample (all male collegiate football athletes). Future research should examine the interaction of demographic 33,34 and clinical characteristics, such as psychiatric and psychological factors [35][36][37] BR Fail and classification accuracies of the EVIs ImPACT . In addition, reasons behind the poor signal detection performance of certain EVIs ImPACT may warrant further investigation. ...
Article
Full-text available
Objective: To create novel Immediate Post-Concussion and Cognitive Testing (ImPACT)-based embedded validity indicators (EVIs) and to compare the classification accuracy to 4 existing EVIImPACT. Method: The ImPACT was administered to 82 male varsity football players during preseason baseline cognitive testing. The classification accuracy of existing EVIImPACT was compared with a newly developed index (ImPACT-5A and B). The ImPACT-5A represents the number of cutoffs failed on the 5 ImPACT composite scores at a liberal cutoff (0.85 specificity); ImPACT-5B is the sum of failures on conservative cutoffs (≥0.90 specificity). Results: ImPACT-5A ≥1 was sensitive (0.81), but not specific (0.49) to invalid performance, consistent with EVIImPACT developed by independent researchers (0.68 sensitivity at 0.73-0.75 specificity). Conversely, ImPACT-5B ≥3 was highly specific (0.98), but insensitive (0.22), similar to Default EVIImPACT (0.04 sensitivity at 1.00 specificity). ImPACT-5A ≥3 or ImPACT-5B ≥2 met forensic standards of specificity (0.91-0.93) at 0.33 to 0.37 sensitivity. Also, the ImPACT-5s had the strongest linear relationship with clinically meaningful levels of invalid performance of existing EVIImPACT. Conclusions: The ImPACT-5s were superior to the standard EVIImPACT and comparable to existing aftermarket EVIImPACT, with the flexibility to optimize the detection model for either sensitivity or specificity. The wide range of ImPACT-5 cutoffs allows for a more nuanced clinical interpretation.