ArticlePDF Available

The predictive validity of cognitive ability tests: A UK meta-analysis

Authors:
Cristina Bertua
Cristina Bertua
Cristina Bertua
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Neil Robert Anderson at University of Bradford
Neil Robert Anderson
Jesus F. Salgado at University of Santiago de Compostela
Jesus F. Salgado
Download full-text PDF
Read full-text
Download citation

Abstract

A meta-analysis on the validity of tests of general mental ability (GMA) and specific cognitive abilities for predicting job performance and training success in the UK was conducted. An extensive literature search resulted in a database of 283 independent samples with job performance as the criterion (N=13,262), and 223 with training success as the criterion (N=75,311). Primary studies were also coded by occupational group, resulting in seven main groups (clerical, engineer, professional, driver, operator, manager, and sales), and by type of specific ability test (verbal, numerical, perceptual, and spatial). Results indicate that GMA and specific ability tests are valid predictors of both job performance and training success, with operational validities in the magnitude of .5–.6. Minor differences between these UK findings and previous US meta-analyses are reported. As expected, operational validities were moderated by occupational group, with occupational families possessing greater job complexity demonstrating higher operational validities between cognitive tests and job performance and training success. Implications for the practical use of tests of GMA and specific cognitive abilities in the context of UK selection practices are discussed in conclusion.
Content uploaded by Jesus F. Salgado
Author content
All content in this area was uploaded by Jesus F. Salgado on Dec 05, 2018
Content may be subject to copyright.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The predictive validity of cognitive ability tests: A UK meta-analysis
Bertua, Cristina;Anderson, Neil;Salgado, Jesà Âos F
Journal of Occupational and Organizational Psychology; Sep 2005; 78, ABI/INFORM Global
pg. 387
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
... Intelligence is a central construct in psychology and related fields such as education, sociology, neuroscience, and economics (Lubinski 2004;Nisbett et al. 2012). It is linked to key life outcomes such as health and longevity (Gottfredson and Deary 2004), academic performance (Roth et al. 2015;Zaboski et al. 2018), career success (Bertua et al. 2005;Ng et al. 2005), socioeconomic achievements (Strenze 2007), and overall subjective wellbeing (Sternberg et al. 2001). Intelligence is often conceptualized as "the most stable psychological trait" (Plomin and Von Stumm 2018, p. 149). ...
Crystallized Intelligence, Fluid Intelligence, and Need for Cognition: Their Longitudinal Relations in Adolescence
Article
Full-text available
  • Oct 2024
Investment theory and related theoretical approaches suggest a dynamic interplay between crystallized intelligence, fluid intelligence, and investment traits like need for cognition. Although cross-sectional studies have found positive correlations between these constructs, longitudinal research testing all of their relations over time is scarce. In our pre-registered longitudinal study, we examined whether initial levels of crystallized intelligence, fluid intelligence, and need for cognition predicted changes in each other. We analyzed data from 341 German students in grades 7–9 who were assessed twice, one year apart. Using multi-process latent change score models, we found that changes in fluid intelligence were positively predicted by prior need for cognition, and changes in need for cognition were positively predicted by prior fluid intelligence. Changes in crystallized intelligence were not significantly predicted by prior Gf, prior NFC, or their interaction, contrary to theoretical assumptions. This pattern of results was largely replicated in a model including all constructs simultaneously. Our findings support the notion that intelligence and investment traits, particularly need for cognition, positively interact during cognitive development, but this interplay was unexpectedly limited to Gf.
View
... Moreover, PSW methods in general are supported by the references in the following arguments: a. Academic achievement (Barnes et al., 2022;Berkowitz et al., 2022;Buckley et al., 2019;Caemmerer et al., 2018;Gerst et al., 2021;Hajovsky et al., 2014;Höffler, 2010;Kendeou et al., 2014;McGrew & Wendling, 2010;Niileksela et al., 2016;Vanderwood et al., 2002;Zheng et al., 2011) b. Occupational choice andachievement (ALMamari &Traynor, 2021;Bertua et al., 2005;Coyle, 2022;Coyle et al., 2014;Goertz et al., 2014;Grobelny, 2018;Ludwikowski et al., 2019;Robertson et al., 2010;Schneider & Newman, 2015;Wai et al., 2009) c. Creative accomplishment (Benedek et al., 2017;Forthmann et al., 2019;Karwowski et al., 2021;Kaufman, 2015; Runco & Acar, 2012) d. ...
View
Cognitive decline and political leadership
Article
Full-text available
  • Sep 2024
  • Polit Life Sci
The cognitive deterioration of politicians is a critical emerging issue. As professions including law and medicine develop and implement cognitive assessments, their insights may inform the proper strategy within politics. The aging, lifetime-appointed judiciary raises legal and administrative questions of such assessments, while testing of older physicians experiencing cognitive decline provides real-life examples of implementation. In politics, cognitive assessment must contend with the field’s unique challenges, also taking context-dependent interpretations of cognitive-neuropsychological status into account. These perspectives, from legal and medical experts, political scientists, and officeholders, can contribute toward an equitable, functioning, and non-discriminatory system of assessing cognition that educates the public and enables politicians to maintain their public responsibilities. With proper implementation and sufficient public knowledge, we believe cognitive assessments for politicians, particularly political candidates, can be valuable for maintaining properly functioning governance. We offer recommendations on the development, implementation, and execution of such assessments, grappling with their democratic and legal implications.
View
Making a Permanent Deacon: Theorising Recruitment, Selection, Formation, and Deployment in the Catholic Archdiocese of Liverpool
Article
  • Aug 2024
The restoration of the permanent diaconate was one of the major changes introduced by the Second Vatican Council. Despite considerable scholarly attention to this Roman Catholic sacred order, no empirically-based study has yet aimed to theorise the process of becoming a permanent deacon. This paper addresses that gap by introducing a substantive theory, which elucidates the dominant behavioural patterns underlying the recruitment, selection, formation, and deployment of suitable candidates from the perspective of decision-makers. Employing classic grounded theory methodology in its entirety, this study draws upon interview, focus group, and textual data collected from the Liverpool Archdiocese in the United Kingdom. The article presents the resulting theory and its components, including the concepts of awakening sense of calling, assessing potentiality, enabling actuality, and facilitating discernment. While the study’s focus is specific to the context of the Liverpool Archdiocese, its insights hold broad relevance for decision-makers involved in Catholic dioceses worldwide.
View
The need for “Considered Estimation” versus “Conservative Estimation” when ranking or comparing predictors of job performance
Article
  • Jun 2024
A recent attempt to generate an updated ranking for the operational validity of 25 selection procedures, using a process labeled “conservative estimation” (Sackett et al., 2022), is flawed and misleading. When conservative estimation's treatment of range restriction (RR) is used, it is unclear if reported validity differences among predictors reflect (i) true differences, (ii) differential degrees of RR (different u values), (iii) differential correction for RR (no RR correction vs. RR correction), or (iv) some combination of these factors. We demonstrate that this creates bias and introduces confounds when ranking (or comparing) selection procedures. Second, the list of selection procedures being directly compared includes both predictor methods and predictor constructs, in spite of the substantial effect construct saturation has on validity estimates (e.g., Arthur & Villado, 2008). This causes additional confounds that cloud comparative interpretations. Based on these, and other, concerns we outline an alternative, “considered estimation” strategy when comparing predictors of job performance. Basic tenets include using RR corrections in the same manner for all predictors, parsing validities of selection methods by constructs, applying the logic beyond validities (e.g., d s), thoughtful reconsideration of prior meta‐analyses, considering sensitivity analyses, and accounting for nonindependence across studies.
View
View
A Contemporary Look at the Relationship Between General Cognitive Ability and Job Performance
Article
Full-text available
  • Dec 2023
The relationship between general cognitive ability (GCA) and overall job performance has been a long-accepted fact in industrial and organizational psychology. However, the most prominent data on this relationship date back more than 50 years. This meta-analysis examines the relationship between GCA and overall job performance using studies from the current century. Results across 153 samples and a total sample size of 40,740 show a mean observed validity of .16, with a residual SD of .09. Correcting for unreliability in the criterion and correcting predictive studies for range restriction produces a mean corrected validity of .22 and a residual SD of .11. While this is a much smaller estimate than the .51 value offered by Schmidt and Hunter (1998), that value has been critiqued by Sackett et al. (2022), who offered a mean corrected validity of .31 based on integrating findings from prior meta-analyses of 20th century data. We obtain a lower value (.22) for 21st century data. We conclude that GCA is related to job performance, but our estimate of the magnitude of the relationship is lower than prior estimates.
View
A critical review of the use of cognitive ability testing for selection into graduate and higher professional occupations
Article
  • Oct 2023
This article presents a critical review of the use of cognitive ability testing for access to graduate and higher professional occupations to promote further debate and reflection in both the academic and practitioner community. The main contentions are that the practice of applying cognitive ability testing in these contexts has strong potential to both maintain and exacerbate social inequality in access to higher occupations and professions, and that validity evidence does not justify this to the extent that has previously been presumed. Five critical observations are examined, namely (1) evidence of adverse impact in test outcomes; (2) the tendency to position cognitive ability testing early in selection processes in high‐volume recruitment; (3) recent evidence challenging the meta‐analytic validity of cognitive ability tests; (4) weaknesses in historical primary validity studies; (5) conceptually flawed examination of differential validity evidence in the literature. Implications for practice are discussed, contrasting strategies that involve modifying selection systems that include cognitive testing, versus removing and replacing cognitive tests.
View
View
Structural brain correlates of non-verbal cognitive ability in 5-year-old children: Findings from the FinnBrain birth cohort study
Article
Non-verbal cognitive ability predicts multiple important life outcomes, for example, school and job performance. It has been associated with parieto-frontal cortical anatomy in prior studies in adult and adolescent populations, while young children have received relatively little attention. We explored the associations between cortical anatomy and non-verbal cognitive ability in 165 5-year-old participants (mean scan age 5.40 years, SD 0.13; 90 males) from the FinnBrain Birth Cohort study. T1-weighted brain magnetic resonance images were processed using FreeSurfer. Non-verbal cognitive ability was measured using the Performance Intelligence Quotient (PIQ) estimated from the Block Design and Matrix Reasoning subtests from the Wechsler Preschool and Primary Scale of Intelligence (WPPSI-III). In vertex-wise general linear models, PIQ scores associated positively with volumes in the left caudal middle frontal and right pericalcarine regions, as well as surface area in left the caudal middle frontal, left inferior temporal, and right lingual regions. There were no associations between PIQ and cortical thickness. To the best of our knowledge, this is the first study to examine structural correlates of non-verbal cognitive ability in a large sample of typically developing 5-year-olds. The findings are generally in line with prior findings from older age groups, with the important addition of the positive association between volume / surface area in the right medial occipital region and non-verbal cognitive ability. This finding adds to the literature by discovering a new brain region that should be considered in future studies exploring the role of cortical structure for cognitive development in young children.
View
View
Fixed Effects vs. Random Effects Meta-Analysis Models: Implications for Cumulative Research Knowledge
Article
Full-text available
  • Dec 2000
Research conclusions in the social sciences are increasingly based on meta-analysis, making questions of the accuracy of meta-analysis critical to the integrity of the base of cumulative knowledge. Both fixed effects (FE) and random effects (RE) meta-analysis models have been used widely in published meta-analyses. This article shows that FE models typically manifest a substantial Type I bias in significance tests for mean effect sizes and for moderator variables (interactions), while RE models do not. Likewise, FE models, but not RE models, yield confidence intervals for mean effect sizes that are narrower than their nominal width, thereby overstating the degree of precision in meta-analysis findings. This article demonstrates analytically that these biases in FE procedures are large enough to create serious distortions in conclusions about cumulative knowledge in the research literature. We therefore recommend that RE methods routinely be employed in meta-analysis in preference to FE methods.
View
View
Meta-Analysis of Interrater Reliability of Job Performance Ratings in Validity Studies of Personnel Selection
Article
Full-text available
  • Dec 1996
Rating scales are most frequently used to assess the criterion in studies of validity in personnel selection. However, only a few articles report the interrater reliability for these scales. This paper presents four meta-analyses in which the interrater reliability was estimated for Civil Composite Criterion, Military Composite Criterion, Total Composite Criterion, and Single Civil Criterion. Mean reliabilities were .64, .53, .62, and .40, respectively. The implications of these findings for single and meta-analytic studies are discussed.
View
View
View
View
Why g matters: The complexity of everyday life
Article
  • Jan 1997
  • INTELLIGENCE
Personnel selection research provides much evidence that intelligence (g) is an important predictor of performance in training and on the job, especially in higher level work. This article provides evidence that g has pervasive utility in work settings because it is essentially the ability to deal with cognitive complexity, in particular, with complex information processing. The more complex a work task, the greater the advantages that higher g confers in performing it well. Everyday tasks, like job duties, also differ in their level of complexity. The importance of intelligence therefore differs systematically across different arenas of social life as well as economic endeavor. Data from the National Adult Literacy Survey are used to show how higher levels of cognitive ability systematically improve individual's odds of dealing successfully with the ordinary demands of modern life (such as banking, using maps and transportation schedules, reading and understanding forms, interpreting news articles). These and other data are summarized to illustrate how the advantages of higher g, even when they are small, cumulate to affect the overall life chances of individuals at different ranges of the IQ bell curve. The article concludes by suggesting ways to reduce the risks for low-IQ individuals of being left behind by an increasingly complex postindustrial economy.
View
Interrater Reliability of Job Performance Ratings: Growth to Asymptote Level With Increasing Opportunity to Observe
Article
  • Jun 1990
The interrater reliabilities of ratings of 9,975 ratees from 79 organizations were examined as a function of length of exposure to the ratee. It was found that there was a strong, nonlinear relationship between months of exposure and interrater reliability. The correlation between a logarithmic transformation of months of experience and reliability was .73 for one type of ratings and .65 for another type. The relationship was strongest during the first 12 months on the job. Changes in reliability were accounted for mostly by changes in criterion variance. Asymptotic levels of reliability were only about .60, even with 10-20 years of experience. Implications for estimating reliabilities in individual and meta-analytic studies and for performance appraisal were presented, and possible explanations of the reliability-variance relationship were advanced.
View
Personality and Job Competences: A comment on the Robertson and Kinder (1993) study
Article
  • Dec 1996
Robertson & Kinder (1993) carried out an exploration of the criterion-related validity of some personality variables as measured by the Occupational Personality Questionnaire. Using meta-analytic integrations of validity coefficients, they suggested that in some circumstances personality measures can fail as predictors of job performance. Although their estimation of the average validity for each criterion area is correct, their estimation of sampling error is based on a computational error. The error influenced the estimation of the residual variance and the 90 per cent credibility values. A reanalysis using the data reported by Robertson & Kinder shows that, against the Robertson & Kinder conclusions, the OPQ generalizes its validity for 11 criterion areas and the lower boundary of the credibility values are substantially far from zero in all cases, except for the energy area. If the Robertson & Kinder findings are taken together with those of this reanalysis, the results offer support for the criterion-related validity of personality variables.
View