ArticlePDF Available

The predictive validity of cognitive ability tests: A UK meta-analysis

DOI:10.1348/096317905X26994
Authors:
Cristina Bertua
Cristina Bertua
Cristina Bertua
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Neil Robert Anderson at University of Bradford
Neil Robert Anderson
Jesus F. Salgado at University of Santiago de Compostela
Jesus F. Salgado
Download full-text PDF
Read full-text
Download citation

Abstract

A meta-analysis on the validity of tests of general mental ability (GMA) and specific cognitive abilities for predicting job performance and training success in the UK was conducted. An extensive literature search resulted in a database of 283 independent samples with job performance as the criterion (N=13,262), and 223 with training success as the criterion (N=75,311). Primary studies were also coded by occupational group, resulting in seven main groups (clerical, engineer, professional, driver, operator, manager, and sales), and by type of specific ability test (verbal, numerical, perceptual, and spatial). Results indicate that GMA and specific ability tests are valid predictors of both job performance and training success, with operational validities in the magnitude of .5–.6. Minor differences between these UK findings and previous US meta-analyses are reported. As expected, operational validities were moderated by occupational group, with occupational families possessing greater job complexity demonstrating higher operational validities between cognitive tests and job performance and training success. Implications for the practical use of tests of GMA and specific cognitive abilities in the context of UK selection practices are discussed in conclusion.
Content uploaded by Jesus F. Salgado
Author content
All content in this area was uploaded by Jesus F. Salgado on Dec 05, 2018
Content may be subject to copyright.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The predictive validity of cognitive ability tests: A UK meta-analysis
Bertua, Cristina;Anderson, Neil;Salgado, Jesà Âos F
Journal of Occupational and Organizational Psychology; Sep 2005; 78, ABI/INFORM Global
pg. 387
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
... Individual differences exist in both sets of abilities, yet they remain malleable and need to be developed to achieve individual goals. The predictive validity of general and domain-specific abilities is well established in academic domains, with domain-specific abilities having a strong and consistent positive effect on creative productivity (e.g., Lubinski, 2016) or in the work place for the prediction of job performance and training success (Bertua, Anderson, & Salgado, 2005 & Worrell, 2011, p. 34). (a) Domains have developmental trajectories with different start, peak, and end times. ...
... In their meta-analyses, Zaboski, Kranzler, and Gage (2018) found positive relations of general and more specific cognitive abilities with academic achievement, with general cognitive abilities having the largest effect across all achievement domains and ages (mean effect size of r 2 = .54). Moreover, earlier domain-specific abilities have a strong and consistent positive effect on later creative productivity, job performance, or training success (Bertua et al., 2005;Lubinski, 2016). ...
View
... Par conséquent, ils accordent une très grande importance à la capacité de prédiction des outils retenus. Plusieurs études ont été menées sur la capacité de prédiction de nombreux outils/méthodes, et ce, depuis longtemps (p.ex., Bertua, Anderson et Salgado, 2005;Hunter et Hunter, 1984;Rojon, McDowall et Saunders, 2015). Parmi les outils qui s'avèrent être de bons prédicteurs de la performance ultérieure en emploi, on retrouve les tests d'aptitude cognitive générale (ACG) (Hunter et Hunter, 1984;Rojon et al., 2015), les tests de connaissances (Lievens et Patterson, 2011), les inventaires de personnalité (Hurtz et Donovan, 2010;Rojon et al., 2015), l'entrevue structurée (Schmidt et Hunter, 1998), les tests de jugement situationnel (Christian, Edwards et Bradley, 2010;McDaniel, Morgeson, Finnigan, Campion et Braverman, 2001;McDaniel, Hartman, Whetzel et Lee Grubb, 2007) et les échantillons de travail (Schmidt et Hunter, 1998;Pettersen, 2000). ...
View
... I. Brown et al., 2021), occupational status (M. I. Brown et al., 2021;Lang & Kell, 2020;Schmidt & Hunter, 2004;Strenze, 2007), job performance (Bertua et al., 2005;Neisser et al., 1996;Schmidt & Hunter, 2004N. Schmitt, 2014), income (M. ...
Structural brain correlates of non-verbal cognitive ability in 5-year-old children: findings from the FinnBrain Birth Cohort study
Preprint
Full-text available
  • Feb 2023
Non-verbal cognitive ability predicts multiple important life outcomes, e.g., school and job performance. It has been associated with parieto–frontal cortical anatomy in prior studies in adult and adolescent populations, while young children have received relatively little attention. We explored the associations between cortical anatomy and non-verbal cognitive ability in 165 5-year-old participants (mean scan age 5.40 years, SD 0.13; 90 males) from the FinnBrain Birth Cohort study. T1-weighted brain magnetic resonance images were processed using FreeSurfer. Non-verbal cognitive ability was measured using the Performance Intelligence Quotient (PIQ) estimated from the Block Design and Matrix Reasoning subtests from the Wechsler Preschool And Primary Scale Of Intelligence (WPPSI-III). In vertex-wise general linear models, PIQ scores associated positively with volumes in left caudal middle frontal and right pericalcarine regions, as well as surface area in left caudal middle frontal, left inferior temporal, and right lingual regions. There were no associations between PIQ and cortical thickness. To the best of our knowledge, this is the first study to examine structural correlates of non-verbal cognitive ability in a large sample of typically developing 5-year-olds. The findings are generally in line with prior findings from older age groups.
View
... Career counseling has a long history of assessment of cognitive aptitudes and abilities. In fact, some studies have indicated that tests of general intelligence and specific aptitudes can predict career and training success relatively well (e.g., Bertua et al., 2005;Salgado et al., 2003). ...
Mapping Career Counseling Interventions: A Guide for Career Practitioners
Book
  • May 2022
Mapping Career Counseling Interventions provides an innovative insight into identifying and resolving problems within career management. In recent years, myriad of counselling tools has been developed so that practitioners can offer the most appropriate support possible. To guide the selection of such tools, this book offers a "map" that introduces and links 63 frequent career management difficulties with more than 160 qualitative interventions and solutions. Through its highly organized layout and extensive detail, major career management difficulties—namely lack of motivation, indecisiveness, dysfunctional beliefs, lack of information, conflicts, and lack of employability—are scrutinized and divided into specific, more manageable sub-difficulties. Targeted interventions to address each of those difficulties, originating from vocational, motivational, and positive psychology, are then suggested and described. From these conclusions, the book delivers practical advice to help conduct these interventions in a sensitive and respectful manner. Overall, this book outlines the most effective ways to foster a good working alliance. One of the book’s standout features is its inclusion of an inventory and card sort to facilitate the identification of obstacles to professional choice and career management. This book is a pivotal resource for the large and diverse community of career development practitioners, such as career counsellors, career advisors, employment counsellors, and career coaches, regardless of their specific audience and work settings. https://www.routledge.com/Mapping-Career-Counseling-Interventions-A-Guide-for-Career-Practitioners/Rochat/p/book/9781032212289
View
... Cognitive abilities [which include the abilities involved in mentally handling information (Carroll, 1993)], and soft skills [or the personal qualities that positively regulate emotions, thoughts, and goal-directed behaviors (Park et al., 2004;Robles, 2012)] are fundamentally important to an individual's wellbeing and success in adulthood (McClelland, 1973;Sternberg, 1997;Bertua et al., 2005;Strenze, 2007;Heckman and Kautz, 2012;Bruna et al., 2019). They are also essential in adolescence, as school students rely on these skills for their academic achievement (Lounsbury et al., 2009;Roth et al., 2015;MacCann et al., 2020;Feraco et al., 2021b). ...
Sport Practice, Fluid Reasoning, and Soft Skills in 10- to 18-Year-Olds
Article
Full-text available
Engaging in physical activity and sports has been associated with various cognitive abilities and other personal characteristics. The contemporary link between doing sports and personal attributes such as soft skills and an individual’s cognitive abilities have yet to be investigated, however. This study aims to analyze the association between years of practicing a sport, cognitive abilities (in terms of fluid reasoning), and personal attributes (in terms of soft skills). A large sample of 1,115 individuals (10–18 years old) completed the Cattell test (measuring fluid reasoning) and answered a questionnaire measuring six soft skills (adaptability, curiosity, initiative, leadership, perseverance, and social awareness). A multivariate regression analysis show that, after controlling for age and gender, participants’ years of practicing a sport were positively associated with three soft skills (i.e., initiative, leadership, and perseverance) and with fluid reasoning. No differences emerged between team and individual sport practitioners. Our findings suggest an association between practicing sports, which entails more than just physical activity, and both cognitive abilities (fluid reasoning) and other important personal characteristics, such as soft skills.
View
Structural brain correlates of non-verbal cognitive ability in 5-year-old children: Findings from the FinnBrain birth cohort study
Article
Non-verbal cognitive ability predicts multiple important life outcomes, for example, school and job performance. It has been associated with parieto-frontal cortical anatomy in prior studies in adult and adolescent populations, while young children have received relatively little attention. We explored the associations between cortical anatomy and non-verbal cognitive ability in 165 5-year-old participants (mean scan age 5.40 years, SD 0.13; 90 males) from the FinnBrain Birth Cohort study. T1-weighted brain magnetic resonance images were processed using FreeSurfer. Non-verbal cognitive ability was measured using the Performance Intelligence Quotient (PIQ) estimated from the Block Design and Matrix Reasoning subtests from the Wechsler Preschool and Primary Scale of Intelligence (WPPSI-III). In vertex-wise general linear models, PIQ scores associated positively with volumes in the left caudal middle frontal and right pericalcarine regions, as well as surface area in left the caudal middle frontal, left inferior temporal, and right lingual regions. There were no associations between PIQ and cortical thickness. To the best of our knowledge, this is the first study to examine structural correlates of non-verbal cognitive ability in a large sample of typically developing 5-year-olds. The findings are generally in line with prior findings from older age groups, with the important addition of the positive association between volume / surface area in the right medial occipital region and non-verbal cognitive ability. This finding adds to the literature by discovering a new brain region that should be considered in future studies exploring the role of cortical structure for cognitive development in young children.
View
Inwiefern vermitteln metaanalytische Übersichten einen zutreffenden Eindruck der Kriteriumsvaliditäten von Personalauswahlverfahren? Anmerkungen zu möglichen Risiken und Nebenwirkungen
Article
Full-text available
  • Jun 2023
Der Artikel von Schmidt und Hunter (1998) bietet einen Überblick und Vergleich der Kriteriumsvalidität verschiedenster Personalauswahlverfahren. Für ihren Artikel haben Schmidt und Hunter jeweils metaanalytisch ermittelte Validitäten ausgewählt, bei denen Korrekturen für Varianzeinschränkung im Auswahlverfahren und Unreliabilität des Kriteriums (d. h. der Messung der Arbeitsleistung) vorgenommen wurden. Hiermit wollten Schmidt und Hunter einen umfassenden Vergleich der wahren Validität der verschiedenen Auswahlverfahren ermöglichen. Trotzdem gibt es eine Reihe von Aspekten, die die Interpretation dieses Vergleichs erschweren bzw. die zu falschen Schlussfolgerungen führen: Erstens die Qualität und Datengrundlage der Metaanalysen, die für diesen Vergleich herangezogen wurden, zweitens die fehlenden Informationen zu unkorrigierten metaanalytischen Validitäten der verschiedenen Auswahlverfahren, drittens unterschiedliche Vorgehensweisen und Annahmen im Rahmen der durchgeführten Korrekturen in den verwendeten Metaanalysen, viertens das gewählte Kriteriums, das durch die verschiedenen Auswahlverfahren vorhergesagt werden soll und fünftens das Problem von nicht berücksichtigte Auswahlverfahren in dieser Übersicht. Im Rahmen dieses Beitrags werden die verschiedenen Aspekte und ihre Bedeutung jeweils genauer erläutert, und es wird dargestellt, inwiefern sich Schlussfolgerungen bzgl. der Validität von Auswahlverfahren ändern können, wenn andere Annahmen und Vorgehensweisen als bei Schmidt und Hunter (1998) gewählt werden.
View
View
Personnel selection: A review of ways to maximize validity, diversity, and the applicant experience
Article
Full-text available
Personnel Psychology has a long tradition of publishing important research on personnel selection. In this article, we review some of the key questions and findings from studies published in the journal and in the selection literature more broadly. In doing so, we focus on the various decisions organizations face regarding selection procedure development (e.g., use multiple selection procedures, contextualize procedure content), administration (e.g., provide pre‐test explanations, reveal target KSAOs), and scoring (e.g., weight predictors and criteria, use artificial intelligence). Further, we focus on how these decisions affect the validity of inferences drawn from the procedures, how use of the procedures may affect organizational diversity, and how applicants experience the procedures. We also consider factors such as cost and time. Based on our review, we highlight practical implications and key directions for future research. This article is protected by copyright. All rights reserved
View
Uncertainty about Rater Variance and Small Dimension Effects Impact Reliability in Supervisor Ratings
Article
Full-text available
We modelled the effects commonly described as defining the measurement structure of supervisor performance ratings. In doing so, we contribute to different theoretical perspectives, including components of the multifactor and mediated models of performance ratings. Across 2 samples from the Jackson et al. (2020) data set (Sample 1, Nratees = 392, Nraters = 244; Sample 2, Nratees = 342, Nraters = 397), we found a structure primarily reflective of general ( > 27% of variance explained) and rater-related (> 49%) effects, with relatively small performance dimension effects (between 1% and 11%). We drew on findings from the assessment center literature to approximate the proportion of rater variance that might theoretically contribute to reliability in performance ratings. We found that even moderate contributions of rater-related variance to reliability resulted in a sizable impact on reliability estimates, drawing them closer to accepted criteria.
View
View
Fixed Effects vs. Random Effects Meta-Analysis Models: Implications for Cumulative Research Knowledge
Article
Full-text available
  • Dec 2000
Research conclusions in the social sciences are increasingly based on meta-analysis, making questions of the accuracy of meta-analysis critical to the integrity of the base of cumulative knowledge. Both fixed effects (FE) and random effects (RE) meta-analysis models have been used widely in published meta-analyses. This article shows that FE models typically manifest a substantial Type I bias in significance tests for mean effect sizes and for moderator variables (interactions), while RE models do not. Likewise, FE models, but not RE models, yield confidence intervals for mean effect sizes that are narrower than their nominal width, thereby overstating the degree of precision in meta-analysis findings. This article demonstrates analytically that these biases in FE procedures are large enough to create serious distortions in conclusions about cumulative knowledge in the research literature. We therefore recommend that RE methods routinely be employed in meta-analysis in preference to FE methods.
View
View
Meta-Analysis of Interrater Reliability of Job Performance Ratings in Validity Studies of Personnel Selection
Article
Full-text available
  • Dec 1996
Rating scales are most frequently used to assess the criterion in studies of validity in personnel selection. However, only a few articles report the interrater reliability for these scales. This paper presents four meta-analyses in which the interrater reliability was estimated for Civil Composite Criterion, Military Composite Criterion, Total Composite Criterion, and Single Civil Criterion. Mean reliabilities were .64, .53, .62, and .40, respectively. The implications of these findings for single and meta-analytic studies are discussed.
View
View
View
View
Why g matters: The complexity of everyday life
Article
Personnel selection research provides much evidence that intelligence (g) is an important predictor of performance in training and on the job, especially in higher level work. This article provides evidence that g has pervasive utility in work settings because it is essentially the ability to deal with cognitive complexity, in particular, with complex information processing. The more complex a work task, the greater the advantages that higher g confers in performing it well. Everyday tasks, like job duties, also differ in their level of complexity. The importance of intelligence therefore differs systematically across different arenas of social life as well as economic endeavor. Data from the National Adult Literacy Survey are used to show how higher levels of cognitive ability systematically improve individual's odds of dealing successfully with the ordinary demands of modern life (such as banking, using maps and transportation schedules, reading and understanding forms, interpreting news articles). These and other data are summarized to illustrate how the advantages of higher g, even when they are small, cumulate to affect the overall life chances of individuals at different ranges of the IQ bell curve. The article concludes by suggesting ways to reduce the risks for low-IQ individuals of being left behind by an increasingly complex postindustrial economy.
View
Interrater Reliability of Job Performance Ratings: Growth to Asymptote Level With Increasing Opportunity to Observe
Article
  • Jun 1990
The interrater reliabilities of ratings of 9,975 ratees from 79 organizations were examined as a function of length of exposure to the ratee. It was found that there was a strong, nonlinear relationship between months of exposure and interrater reliability. The correlation between a logarithmic transformation of months of experience and reliability was .73 for one type of ratings and .65 for another type. The relationship was strongest during the first 12 months on the job. Changes in reliability were accounted for mostly by changes in criterion variance. Asymptotic levels of reliability were only about .60, even with 10-20 years of experience. Implications for estimating reliabilities in individual and meta-analytic studies and for performance appraisal were presented, and possible explanations of the reliability-variance relationship were advanced.
View
Personality and Job Competences: A comment on the Robertson and Kinder (1993) study
Article
  • Dec 1996
Robertson & Kinder (1993) carried out an exploration of the criterion-related validity of some personality variables as measured by the Occupational Personality Questionnaire. Using meta-analytic integrations of validity coefficients, they suggested that in some circumstances personality measures can fail as predictors of job performance. Although their estimation of the average validity for each criterion area is correct, their estimation of sampling error is based on a computational error. The error influenced the estimation of the residual variance and the 90 per cent credibility values. A reanalysis using the data reported by Robertson & Kinder shows that, against the Robertson & Kinder conclusions, the OPQ generalizes its validity for 11 criterion areas and the lower boundary of the credibility values are substantially far from zero in all cases, except for the energy area. If the Robertson & Kinder findings are taken together with those of this reanalysis, the results offer support for the criterion-related validity of personality variables.
View