Figure - uploaded by Laszlo A Erdodi
Content may be subject to copyright.
Source publication
This study was designed to develop performance validity indicators embedded within the Delis-Kaplan Executive Function Systems (D-KEFS) version of the Stroop task. Archival data from a mixed clinical sample of 132 patients (50% male; MAge= 43.4; MEducation= 14.1) clinically referred for neuropsychological assessment were analyzed. Criterion measure...
Context in source publication
Similar publications
Objective:
Determine whether preinjury concussion symptom factors predict postinjury concussion symptom factors.
Setting:
Community concussion surveillance program between 2008 and 2017.
Participants:
A total of 290 high school and collegiate-aged athletes (n = 168, aged 14-17 years; n = 122, aged 18-21 years, respectively).
Design:
Retrospe...
Citations
... In the hearing threshold test, an audible stimulus was played in each trial, and the subjects were asked to press the "1" button on the keyboard when they could hear these stimuli or, otherwise, to press the "0" button. In the duration discrimination test [49] and the frequency discrimination test [50], the subjects would always hear one tone after the other tone, and they had to judge which tone was longer for the former and which tone was higher for the latter. A sound test was performed before the formal tests to make sure that the subjects could hear the audible stimulus clearly, and then the sound volume remained unchanged during the experiments. ...
Individuals exposed to elevated noise levels experience heightened emotional intensity, leading to increased cognitive disruption and a higher likelihood of accidents. This study seeks to investigate the impact of noise exposure on human cognitive performance, and the moderating role of emotion. Twelve healthy male college-age students underwent exposure to three noise conditions, each characterized by different sound pressure levels and sharpness. Each condition included an initial acoustic/thermal adaption period lasting approximately 40 min, followed by intermittent questionnaire tests and a battery of computerized cognitive tests. Statistical analysis revealed that reducing noise levels proved advantageous, enhancing perceived sound quality, positive emotions, and auditory perception abilities, while concurrently reducing false alerts and accelerating execution speed. Many of these effects were found to be counteracted by elevated sharpness. Correlation analyses and partial least squares structural equation modeling (PLS-SEM) results suggested that human emotions mediate the relationship between noise exposure and cognitive performance. The potential underlying mechanism suggests that negative feelings towards noise contribute to poor emotional states, subsequently influencing cognitive processes and impairing executive function. The outcomes of this study provide valuable insights into the mechanism of noise exposure and its effects on human cognition and subjective perceptions.
... Abeare et al., 2019;C. A. Abeare, An, et al., 2022;Kim et al., 2010;Lichtenstein et al., 2017;Rai et al., 2019), indicators or cutoffs (Arentsen et al., 2013Ashendorf, 2019;Ashendorf et al., 2017Ashendorf et al., , 2021Blaskewitz et al., 2009;Bortnik et al., 2010;Boskovic et al., 2018;Boucher et al., 2023;Chang et al., 2023;Curtis et al., 2008Curtis et al., , 2009Deloria et al., 2021;Dunn et al., 2021;Erdodi et al., 2016Erdodi et al., , 2018Glassmire et al., 2019;Greve et al., 2009;Guise et al., 2014;M. J. Holcomb et al., 2022;Hurtubise et al., 2020;Johnson et al., 2012;C. ...
This editorial article introduces the second special issue of Psychology & Neuroscience devoted to performance and symptom validity testing. The reason for including the second special issue is that we received an unusually large number of high-quality submissions that could not fit into a single volume. The articles included in this second part offer practical, immediately actionable knowledge to assessors while simultaneously advancing the methodology for calibrating instruments designed to evaluate the credibility of a given clinical presentation. In this introduction, we briefly summarize each article and reflect on an emerging epistemological question about the interpretation of noncredible results in the context of a clinical research study: If a relatively large proportion of clinical patients fail a validity test without any apparent external incentives to appear impaired, should this be interpreted as a possible vulnerability of that validity test to false-positive classifications or as evidence that noncredible responding is relatively common outside of medicolegal/forensic assessments? The methodological implications of symptom and performance validity research are discussed.
... Similarly, ePVTs nested within Digit Span, verbal fluency and TMT-A underperformed relative to the rest of them (Fig. 2). Shading, capitalization and bold face provide a visual representation of the change in confidence in correctly classifying a given score as invalid (darker background, capital letters and bold mean increased likelihood of non-credible performance) (Denning, 2012;Erdodi, 2022Erdodi, , 2023Greve et al., 2006;Jones, 2013;Kulas et al., 2014;Rai & Erdodi, 2021;Schroeder et al., 2013); WCT: Word Choice Test (accuracy score; Barhon et al., 2015;Cutler et al., 2022;Davis, 2014;Erdodi, 2021;Erdodi et al., 2014;Holcomb et al., 2022;Pearson, 2009;Tyson & Shahein, 2023;Zuccato et al., 2018); WRT: Rey Word Recognition Test (true positives; Bell-Sprinkel et al., 2013;Goworowski et al., 2020;Nitch et al., 2006;Smith et al., 2014); Animals RO : Animal fluency T-score (Romanian administration; Abeare et al., 2021a;Deloria et al., 2021;Hurtubise et al., 2020;Sugarman & Axelrod, 2015); CD WAIS-III : Coding subtest of the Wechsler Adult Intelligence Scale -Third edition (age-corrected scaled score; Ashendorf et al., 2017;Erdodi & Abeare, 2020;Erdodi et al., 2017a;Etherton et al., 2006); FCR HVLT-R : Forced Choice Recognition trial of the Hopkins Verbal Learning Test -Revised; (accuracy score; Abeare et al., 2021b;Ali et al., 2022a, b;Cutler et al., 2021); RDS: Reliable Digit Span (Babikian et al., 2006;Heinly et al., 2005;Mathias et al., 2002;Pearson, 2009;Schoeder et al., 2012;Shura et al., 2020;Whitney et al., 2009;Young et al., 2012); WORD D-KEFS : Word Reading subtest of the Delis-Kaplan Executive System (age-corrected scaled score; Cutler et al., 2022;Eglit et al., 2020;Erdodi et al., 2018a) ...
This study was designed to evaluate the susceptibility of various performance validity tests (PVTs) to limited English proficiency (LEP). A battery of free-standing and embedded PVTs was administered to 95 undergraduate students at a Romanian university, randomly assigned to the control (n = 65) or experimental malingering group (n = 30). Overall correct classification (OCC) at the first cutoff to clear .90 specificity (with group membership as criterion) was used as the main metric to compare PVTs. Mean OCC for free-standing PVTs (.784) was comparable to mean OCC for embedded PVTs (.780). Cutoffs on embedded PVTs often had to be adjusted (more conservative) to meet the specificity standard. Contrary to our predictions, embedded PVTs with high verbal mediation outperformed those with low verbal mediation (mean OCC .807 versus .719). Although multivariate models of PVTs performed very well (mean OCC = .892), several individual freestanding and embedded PVTs produced comparable mean OCC (.863-.895). Other embedded PVTs had trivial sensitivity (.03-.13) at ≥ .90 specificity. PVTs administered in both languages (English and Romanian) provided conclusive evidence of both the deleterious effects of LEP and the cross-cultural validity of existing methods of performance validity testing. Results defied most of our a priori predictions: level of verbal mediation was an influential, but not a decisive factor in the classification accuracy of PVTs; free-standing PVTs were not necessarily superior to embedded PVTs; multivariate models of performance validity assessment outperformed most, but not all their individual components. Our findings suggest that some PVTs may be inherently unfit to be used with examinees with LEP. The multiple unexpected findings signal a fundamental uncertainty about the psychometric properties of instruments developed and validated in North America when applied to examinees outside the US or Canada. Although several existing PVTs have the potential to be useful in examinees with LEP, their relevant psychometric properties should be independently verified in new target populations to ensure the validity of their clinical interpretation. The classification accuracy observed in native speakers of English cannot be assumed to transfer to members of linguistically and culturally different communities – doing so risks potentially consequential errors in performance validity assessment. Of course, the abundance of counterintuitive findings also serves as a note of caution: our findings may not generalize to different samples.
... Cognitive tests can be classified according to the specific brain functions needed to address task demands. For example, the Stroop test can be used to measure human executive ability [31,32]; the psychomotor vigilance test (PVT) and the visual search (VS) test can be used to evaluate attention [33][34][35]; and the 2-back test and the digit span memory test can be used to measure short-term memory [36,37]. Pang et al. [38] reported a significant decline in subjects' attention (according to the PVT) at CO 2 concentrations above 3500 ppm. ...
... Fiene and colleagues [54] further noted that the utility of reaction time variability found in CPTs could also be observed in a simple reaction time task, a finding later independently reported in other studies [55,56]. Variations of the Stroop Test [57] have also shown promising results across studies [47,[58][59][60][61][62][63], with the word reading trial oftentimes emerging as the most sensitive outcome measure within this test. ...
The evaluation of performance validity is an essential part of any neuropsychological evaluation. Validity indicators embedded in routine neuropsychological tests offer a time-efficient option for sampling performance validity throughout the assessment while reducing vulnerability to coaching. By administering a comprehensive neuropsychological test battery to 57 adults with ADHD, 60 neurotypical controls, and 151 instructed simulators, we examined each test’s utility in detecting noncredible performance. Cut-off scores were derived for all available outcome variables. Although all ensured at least 90% specificity in the ADHD Group, sensitivity differed significantly between tests, ranging from 0% to 64.9%. Tests of selective attention, vigilance, and inhibition were most useful in detecting the instructed simulation of adult ADHD, whereas figural fluency and task switching lacked sensitivity. Five or more test variables demonstrating results in the second to fourth percentile were rare among cases of genuine adult ADHD but identified approximately 58% of instructed simulators.
... The CWIT is the D-KEFS version of the Stroop task (Stroop, 1935) which includes an additional trial where examinees switch between color naming and word reading of color words printed in incongruent colors. The first investigation on the utility of the CWIT as an EVI was conducted by Erdodi et al. (2018) in a sample of mixed clinical patients (predominantly psychiatric and mild traumatic brain injury [TBI]). The authors found that an Age-Corrected Scaled Score (ACSS) ≤6 on any of the four trials of the CWIT yielded high specificity (.82-.94) and variable sensitivity (.29-.71). ...
... The authors found that an Age-Corrected Scaled Score (ACSS) ≤6 on any of the four trials of the CWIT yielded high specificity (.82-.94) and variable sensitivity (.29-.71). However, scoring ≤6 on 3 or more conditions yielded better combinations of sensitivity (.36-.57) and specificity than the individual conditions (Erdodi et al., 2018). ...
... Minimal acceptable classification accuracy was set at an area under the curve (AUC) of ≥0.70 for all ROC curve analyses (Hosmer et al., 2013). Sensitivity and specificity were calculated for cutoffs derived from previous investigations (Cutler et al., 2022;Eglit et al., 2020;Erdodi et al., 2018). See Table 3, for a list of cutoffs. ...
Objective: Previous investigations have yielded divergent results regarding the clinical utility of the Delis–Kaplan Executive Function System (D-KEFS) color–word interference test (CWIT) as an embedded validity indicator. The present study sought to cross-validate previously identified indicators and cutoffs in a sample of adults referred for psychoeducational testing. Method: Archival data from 257 students and community members self- referred for a psychoeducational evaluation at a university clinic in the South were analyzed. Individuals who failed 0 criterion measures were included in the credible group and individuals failing 2 or more criterion measures were included in the noncredible group. Individuals who failed only one of the criterion measures were excluded (n = 53), resulting in a final sample of 204 (credible group n = 164; noncredible group n = 40). Results: Individual conditions yielded high specificity (.88–.91) and moderate sensitivity (.30–.45) at a cutoff of ≤6. Composite indicators yielded high specificity (.94–.99) and low to moderate sensitivity (.20–.40) at previously established cutoffs. Utilizing more liberal cutoffs increased sensitivity (.28–.53) at the predictable cost to specificity (.83–.91). Conclusions: While a cutoff of ≤6 resulted in high specificity for most conditions, composite indicators appear to be more robust. Additional research is needed before recommending any specific cutoff be used in clinical practice with individuals seeking psychoeducational assessment.
... Research over the last decades considered a large number of validity measures, so-called performance validity tests (PVTs), that can be classified into stand-alone (freestanding) validity tests and validity indicators embedded within routine measures of neuropsychological functions (for an overview, see Boone, 2021). While stand-alone PVTs are considered the gold standard for producing high diagnostic accuracy in distinguishing credible and noncredible cognitive performance, embedded PVTs have certain advantages over stand-alone PVTs Erdodi et al., 2014Erdodi et al., , 2018. Empirical evidence and current practice standards recommend the use of multiple PVTs to assess validity and stress the need to sample validity continuously throughout the assessment and across cognitive domains (e.g., Rhoads et al., 2021;Soble, 2021;Sweet et al., 2021). ...
... As a major limitation of the present study, it must be noted that only one PVT (i.e., the WMT) was used as the criterion to determine noncredible cognitive performance. The use of a single criterion PVT has been shown to distort accuracy estimates of the predictor PVT, especially when high congruence between predictor and criterion is given (see domain-specificity hypothesis, Erdodi et al., 2018). In favor of the present study, the congruence between the cognitive domains of the predictor PVT (COG as an attention test) and criterion PVT (WMT as a verbal memory test) was low, and the WMT is presumably one of the best studied PVTs and de facto gold standard in clinic and research. ...
The assessment of performance validity is essential in any neuropsychological evaluation. However, relatively few measures exist that are based on attention performance embedded within routine cognitive tasks. The present study explores the potential value of a computerized attention test, the Cognitrone, as an embedded validity indicator in the neuropsychological assessment of early retirement claimants. Two hundred and sixty-five early retirement claimants were assessed with the Word Memory Test (WMT) and the Cognitrone. WMT scores were used as the independent criterion to determine performance validity. Speed and accuracy measures of the Cognitrone were analyzed in receiver operating characteristics (ROC) to classify group membership. The Cognitrone was sensitive in revealing attention deficits in early retirement claimants. Further, 54% (n = 143) of the individuals showed noncredible cognitive performance, whereas 46% (n = 122) showed credible cognitive performance. Individuals failing the performance validity assessment showed slower (AUC = 79.1%) and more inaccurate (AUC = 79.5%) attention performance than those passing the performance validity assessment. A compound score integrating speed and accuracy revealed incremental value as indicated by AUC = 87.9%. Various cut scores are suggested, resulting in equal rates of 80% sensitivity and specificity (cut score = 1.297) or 69% sensitivity with 90% specificity (cut score = 0.734). The present study supports the sensitivity of the Cognitrone for the assessment of attention deficits in early retirement claimants and its potential value as an embedded validity indicator. Further research on different samples and with multidimensional criteria for determining invalid performance are required before clinical application can be suggested.
... Stroop test (ST). An experimental trial-by trial version of the ST (Erdodi et al., 2018;Jensen, 1965) was employed. Stimuli consisted of 48 congruent and incongruent colored (green, yellow, and blue) words. ...
Almost 30% of ADHD adults do not respond to standard pharmaceuticals. Transcranial direct current stimulation (tDCS) is a method for modulation of cortical excitability. On the other hand, dialectical behavioral therapy (DBT) is a cognitive-behavioral approach that might be utilized for adults with ADHD. The effects of integration of these interventions are only beginning to be explored. In the present work, we used both subjective and objective measures to investigate the effects of tDCS, DBT, and the integration of the two in treating adult ADHD symptoms. A total of 80 adults with ADHD (63 females, 17 males) participated in the study and were grouped into control, DBT, tDCS, and combined groups. Based on the observed results, the combination of DBT and tDCS was significantly effective in improving the mentioned variables compared to administration of each method in isolation. The results are discussed in terms of neurophysiological and psychological aspects of treatment methods.
... The EFT is widely used to measure inhibition ability and selective attention [47]. The ST is commonly used to measure the inhibition of cognitive interference, processing speed, cognitive flexibility, and executive function [48]. The RVIPT was used to assess target sensitivity, decision-making, and sustained attention and alertness [49]. ...
This study aimed to investigate the effects of caffeine on performances of simulated match, Wingate Anaerobic Test (WAnT), and cognitive function test of elite taekwondo athletes. Ten elite taekwondo athletes in Hong Kong volunteered to participate in two main trials in a randomized double-blinded crossover design. In each main trial, 1 h after consuming a drink with caffeine (CAF) or a placebo drink without caffeine (PLA), the participants completed two simulated taekwondo match sessions followed by the WAnT. The participants were instructed to complete three cognitive function tests, namely the Eriksen Flanker Test (EFT), Stroop Test, and Rapid Visual Information Processing Test, at baseline, before exercise, and immediately after the simulated matches. They were also required to wear functional near-infrared spectroscopy equipment during these tests. Before exercise, the reaction time in the EFT was shorter in the CAF trial than in the PLA trial (PLA: 494.9 ± 49.2 ms vs. CAF: 467.9 ± 38.0 ms, p = 0.035). In the WAnT, caffeine intake increased the peak power and mean power per unit of body weight (by approximately 13% and 6%, respectively, p = 0.018 & 0.042). The performance in the simulated matches was not affected by caffeine intake (p = 0.168). In conclusion, caffeine intake enhances anaerobic power and may improve certain cognitive functions of elite taekwondo athletes in Hong Kong. However, this may not be enough to improve the simulated match performance.
... In a mixed clinical sample of 234 adults referred for neuropsychological assessment, the Borderline range was significantly different from both Pass (i.e., stronger evidence of non-credible responding) and Fail (i.e., weaker evidence of non-credible responding). These findings are consistent with the results of previous (Erdodi & Rai, 2017;Erdodi, Sagar, et al., 2018;Erdodi, Seke, et al., 2017) and subsequent Cutler et al., 2021;Dunn et al., 2021;Erdodi, Hurtubise, et al., 2020) investigations. ...
This study was designed to examine the classification accuracy of the Erdodi Index (EI-5), a novel method for aggregating validity indicators that takes into account both the number and extent of performance validity test (PVT) failures. Archival data were collected from a mixed clinical/forensic sample of 452 adults referred for neuropsychological assessment. The classification accuracy of the EI-5 was evaluated against established free-standing PVTs. The EI-5 achieved a good combination of sensitivity (.65) and specificity (.97), correctly classifying 92% of the sample. Its classification accuracy was comparable to that of another free-standing PVT. An indeterminate range between Pass and Fail emerged as a legitimate third outcome of performance validity assessment, indicating that the underlying construct is an inherently continuous variable. Results support the use of the EI-model as a practical and psychometrically sound method of aggregating multiple embedded PVTs into a single-number summary of performance validity. Combining free-standing PVTs with the EI-5 resulted in a better separation between credible and non-credible profiles, demonstrating incremental validity. Findings are consistent with recent endorsements of a three-way outcome for PVTs (Pass, Borderline and Fail).
