About
250
Publications
804,159
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
58,760
Citations
Introduction
Current institution
Publications
Publications (250)
As a vital tool for advancing cumulative science and guiding evidence-based practice, meta-analysis has advanced many research fields of the behavioral and social sciences, including worker age and aging. Nonetheless, there is still room for improvement, particularly in terms of meta-analytic methods and procedures. In this article, we overview som...
This meta-analytic path analysis presents evidence that first-year academic performance (FYAP), measured by first-year grade point average (FYGPA) plays the major role in determining second-year student retention and that socioeconomic status (SES), measured by parental income, plays a negligible role. Based on large sample data used in a previous...
Using a database of 171 studies across 62,965 organizational units with job attitude data and business performance outcomes ranging from 1994 to 2015, we tested the hypothesis that positive employee job attitudes relate more strongly to business unit success during bad economic times than during favorable economic times. Results showed that althoug...
This article explores an important credibility problem in the research literature beyond the issue of questionable data analysis methods: the problem of omission of relevant previous research in published research articles. This article focuses on this problem in 2 areas: (a) studies purporting to demonstrate the effects of people’s experiences on...
Purpose
Meta-regression is widely used and misused today in meta-analyses in psychology, organizational behavior, marketing, management, and other social sciences, as an approach to the identification and calibration of moderators, with most users being unaware of serious problems in its use. The purpose of this paper is to describe nine serious me...
The lengthy and complex focal article by Tett, Hundley, and Christiansen (2017) is based on a fundamental misunderstanding of the nature of validity generalization (VG): It is based on the assumption that what is generalized in VG is the estimated value of mean rho ( $\bar{\rho}$ ). This erroneous assumption is stated repeatedly throughout the arti...
In this commentary, we build on Bracken, Rose, and Church's (2016) definition stating that 360° feedback should involve “the analysis of meaningful comparisons of rater perceptions across multiple ratees, between specific groups of raters” (p. 764). Bracken et al. expand on this component of the definition later by stressing that “the ability to co...
This might be the most opportune time for Human Resource Management (HRM) to benefit from psychometric meta-analysis. Explosion of empirical research, often with conflicting results, hide important takeaways that can guide evidence-based applications of HRM. The science of HRM can turn to meta-analyses and meta-analytic thinking as the antidote to...
There have been frequent expressions of concern over the supposed failure of researchers to conduct replication studies. But the large number of meta-analyses in our literatures shows that replication studies are in fact being conducted in most areas of research. Many who argue for replication as the “gold standard” consider a nonsignificant replic...
In the current study, we present a more accurate method for correcting for range restriction (Case V) that expands upon Bryant and Gokhale’s (1972) method. We further present detailed steps to incorporate the Case V method into Schmidt and Hunter’s (2015; Hunter & Schmidt, 2004) psychometric meta-analysis methods (both individual correction and art...
Intelligence is the major determinant of job performance, and therefore hiring people based on intelligence leads to marked improvements in job performance improvements that have high economic value to the firm. This principle is very broad: it applies to all types of jobs at all job levels. Intelligence is often referred to as general mental abili...
In this article, I provide answers to the questions posed by Will Shadish about the history and development of the Schmidt-Hunter methods of meta-analysis. In the 1970s, I headed a research program on personnel selection at the US Office of Personnel Management (OPM). After our research showed that validity studies have low statistical power, OPM f...
Our objective was to apply the rigorous test developed by Browne (1992) to determine whether the circumplex model fits Big Five personality data. This test has yet to be applied to personality data. Another objective was to determine whether blended items explained correlations among the Big Five traits. We used two working adult samples, the Eugen...
This meta-analysis examines the strength of the relationships of ACT® Composite scores, high school grades, and socioeconomic status (SES) with academic performance and persistence into the 2nd and 3rd years at 4-year colleges and universities. Based upon a sample of 189,612 students at 50 institutions, ACT Composite scores and high school grade po...
Validity generalization refers to the demonstration that the validities of various selection devices generalize across different settings. The degree of validity generalization is typically established through the use of meta-analysis. Results have usually supported the case for validity generalization indicating that it is not necessary to conduct...
All measurements must contend with unreliability. No measure is free of measurement error. More attention must be paid to measurement error in all psychological research. The problem of reliability is more severe when rating scales are involved. Many of the constructs in industrial–organizational (I–O) psychology and organizational behav-ior/human...
Measurement Error Obfuscates Scientific Knowledge: Path to Cumulative Knowledge Requires Corrections for Unreliability and Psychometric Meta-Analyses - Volume 7 Issue 4 - Chockalingam Viswesvaran, Deniz S. Ones, Frank L. Schmidt, Huy Le, In-Sue Oh
There have been frequent expressions of concern over the supposed failure of researchers to conduct replication studies. But the large number of meta-analyses in our literatures shows that replication studies are in fact being conducted in most areas of research. Many who argue for replication as the “gold standard” consider a non-significant repli...
This commentary integrates the contents of four recent articles on individual differences (Nye, Su, Rounds, & Drasgow, 2012; Schmidt, 2011; Valla & Ceci, 2011; von Stumm, Hell, & Chamorro-Premuzic, 2011) in a causal theoretical model. In this model, introversion and fluid intelligence cause interest in general learning (intellectual curiosity), whi...
a b s t r a c t This paper presents methods for second order meta-analysis along with several illustrative applications. A second order meta-analysis is a meta-analysis of a number of statistically independent and methodo-logically comparable first order meta-analyses examining ostensibly the same relationship in different contexts. First order met...
Relations between constructs are estimated based on correlations between measures of constructs corrected for measurement error. This process assumes that the true scores on the measure are linearly related to construct scores, an assumption that may not hold. We examined the extent to which differences in distribution shape reduce the correlation...
The American Psychological Foundation (APF) Gold Medal Awards recognize distinguished and enduring records of accomplishment in four areas of psychology: the application of psychology, the practice of psychology, psychology in the public interest, and the science of psychology. The 2013 recipient of the Gold Medal Award for Life Achievement in the...
This paper used meta-analysis to synthesize the relation between visual skills and Chinese reading acquisition based on the empirical results from 34 studies published from 1991 to 2011. We obtained 234 correlation coefficients from 64 independent samples, with a total of 5,395 participants. The meta-analysis revealed that visual skills as a global...
Employment interviews are one of the most widely used selection tools across organizations, industries, and countries (Dipboye, 1992, 1997; Dip-boye & Jackson, 1999; Ryan, McFarland, Baron, & Page, 1999; Salgado, Viswesvaran, & Ones, 2001; Wilk & Cappelli, 2003, Table 1). Interviews also play an important role in government employment decisions, pa...
Employment interviews are one of the most widely used selection tools across organizations, industries, and countries (Dipboye, 1992, 1997; Dip-boye & Jackson, 1999; Ryan, McFarland, Baron, & Page, 1999; Salgado, Viswesvaran, & Ones, 2001; Wilk & Cappelli, 2003, Table 1). Interviews also play an important role in government employment decisions, pa...
The small sample studies typical of psychological research produce seemingly contradictory results, and reliance on statistical significance tests causes study results to appear even more conflicting. Meta-analysis integrates the findings across such studies to reveal the simpler patterns of relations that underlie research literatures, thus provid...
Examination of the Van Iddekinge, Roth, Raymark, and Odle-Dusseau (2012) meta-analysis reveals a number of problems. They meta-analyzed a partial database of integrity test validities. An examination of their coded database revealed that measures coded as integrity tests and meta-analyzed as such often included scales that are not in fact integrity...
Many industrial/organizational (I/O) psychologists, both academics and practitioners, believe that the content validity model is not appropriate for cognitive ability measures used in personnel selection. They believe that cognitive tests can have criterion validity and construct validity but not content validity. Based on a review of the broader d...
The most important fact emerging from the combination of my article and the three commentaries is the consensus judgment that content validity is appropriate scientifically and professionally for use with tests of specific cognitive skills used in job performance. This is important because the 1978 Uniform Guidelines on Employee Selection Procedure...
In this article, I present a theory that explains the origin of sex differences in technical aptitudes. The theory takes as proven that there are no sex differences in general mental ability (GMA), and it postulates that sex differences in technical aptitude (TA) stem from differences in experience in technical areas, which is in turn based on sex...
Meta-analysis has been increasingly used as a knowledge cumulation tool by IS researchers. In recent years many meta-analysts have conducted moderator analyses in an attempt to develop and test theories. These studies suffer from several methodological problems and, as a result, may have contributed to rather than resolved inconsistent research fin...
Aripiprazole is approved for the acute and maintenance treatment of manic and mixed episodes associated with bipolar I disorder. The aim of the present work was to review and meta-analyze the findings of all the available randomized double-blind controlled trials (RCTs) on the efficacy of aripiprazole in the treatment of bipolar disorder.
Aripipraz...
Our focus is on the difficulties that synthetic validity encounters in attempting to achieve discriminant validity and the implications of these difficulties. Johnson et al. (2010) acknowledge the potential problems involved in attaining discriminant validity in synthetic validity. For example, they report that Peterson et al. (2001), Johnson (2007...
Our focus is on the difficulties that synthetic validity encounters in attempting to achieve discriminant validity and the implications of these difficulties. Johnson et al. (2010) acknowledge the potential problems involved in attaining discriminant validity in synthetic validity. For example, they report that Peterson et al. (2001), Johnson (2007...
We performed a quantitative review of associations between the higher order personality traits in the Big Three and Big Five models (i.e., neuroticism, extraversion, disinhibition, conscientiousness, agreeableness, and openness) and specific depressive, anxiety, and substance use disorders (SUD) in adults. This approach resulted in 66 meta-analyses...
Perceptions of work conditions have proven to be important to the well-being of workers. However, customer loyalty, employee retention, revenue, sales, and profit are essential to the success of any business. It is known that these outcomes are correlated with employee attitudes and perceptions of work conditions, but the research into direction of...
This paper presents an account of the impact that research synthesis methods, in the form of psychometric meta-analysis, has had on industrial/organizational (I/O) psychology. This paper outlines the central contributions of psychometric meta-analysis in providing a method for developing cumulative knowledge. First, this paper describes the concern...
Construct empirical redundancy may be a major problem in organizational research today. In this paper, we explain and empirically illustrate a method for investigating this potential problem. We applied the method to examine the empirical redundancy of job satisfaction (JS) and organizational commitment (OC), two well-established organizational con...
Because of the way in which data are typically analyzed and interpreted, they frequently lie to researchers, leading to conclusions that are not only false but more complex than the underlying reality. The several examples of this presented in this article illustrate the possibility that although data may appear to indicate complex phenomena at the...
Meta-analysis is a set of methods for integrating findings across studies to reveal the patterns of relations that underlie conflicting research literatures, thus providing a basis for theory development. Meta-analysis can correct for the distorting effects of sampling error, measurement error, and other artifacts that produce the illusion of confl...
We present a systematic review and meta-analysis of the available clinical trials concerning the usefulness of aripiprazole in the treatment of the psychotic symptoms in bipolar disorder.
A systematic MEDLINE and repository search concerning clinical trials for aripiprazole in bipolar disorder was conducted.
The meta-analysis of four randomised con...
The methods presented in this chapter for correcting meta-analysis results for sampling error and biases in individual study results might appear complicated, but the reader should be aware that more elaborated and extended descriptions of these procedures are presented in Hunter and Schmidt (2004) and that Windows-based software (Schmidt and Le 20...
Recent research in the field of meta-analysis has provided a procedure of improved accuracy to correct for range restriction, which allows for more accurate calibrations of the validities of various admission and selection tools. For illustrative purposes, we reanalyzed the database meta-analyzed by Kuncel, Credé , and Thomas (2007) on the validity...
A recently developed procedure produces substantial improvements in the accuracy of corrections for range restriction and reveals that predictive validities of employment selection methods are higher than previously thought. This study applied this procedure to meta-analytic validities of 2 personality measures (Conscientiousness and Emotional Stab...
Comments on the original article, "The meaning of employee engagement" by W. H. Macey and B. Schneider (see record
2010-13366-002), which provides an informative treatise on the psychological construct of employee engagement. In recent years, the term "employee engagement" has taken on a life of its own, probably based, in part, on its resonance a...
Today most conclusions about cumulative knowledge in psychology are based on meta-analysis. We first present an examination of the important statistical differences between fixed-effects (FE) and random-effects (RE) models in meta-analysis and between two different RE procedures, due to Hedges and Vevea, and to Hunter and Schmidt. The implications...
Measurement artifacts, including measurement errors and scale-specific factors, distort observed correlations between measures of psychological and organizational constructs. The authors discuss two alternative procedures, one using the generalized coefficient of equivalence and stability (GCES) and one based on structural equation modeling, to cor...
We discuss how meta-analysis, a method for synthesizing research findings, can meaningfully impact personnel selection practices. Specifically, we review important changes in professional and legal practices from the past 30 years resulting from meta-analytic findings. The implications of using meta-analysis methods for evaluating utilities of sele...
During the past 30 years, meta-analysis has been an indispensable tool for revealing the hidden meaning of our research literatures. The four articles in this special section on meta-analysis illustrate some of the complexities entailed in meta-analysis methods. Although meta-analysis is a powerful tool for advancing cumulative knowledge, researche...
We respond to the three comments on our recent article in the August issue of Academy of Management Perspectives (Le, Oh, Shaffer, & Schmidt, 2007), which highlights the importance of methodological advances in human resource research. By concentrating on tangential aspects of our article, these comments miss its central points. Further, the commen...
The authors examine 3 methods of combining new studies into existing meta-analyses: (a) adding the new study or studies to the database and recalculating the meta-analysis (the medical model); (b) using the Bayesian procedure advocated by F. L. Schmidt and J. E. Hunter (1977) and F. L. Schmidt, J. E. Hunter, K. Pearlman, and G. S. Shane (1979) to u...
Executive Overview We respond to the three comments on our recent article in the August issue of Academy of Management Perspectives (Le, Oh, Shaffer, & Schmidt, 2007), which highlights the importance of methodological advances in human resource research. By concentrating on tangential aspects of our article, these comments miss its central points....
Executive Overview We respond to the three comments on our recent article in the August issue of Academy of Management Perspectives (Le, Oh, Shaffer, & Schmidt, 2007), which highlights the importance of methodological advances in human resource research. By concentrating on tangential aspects of our article, these comments miss its central points....
Using computer simulation, the authors assessed the accuracy of J. E. Hunter, F. L. Schmidt, and H. Le's (2006) procedure for correcting for indirect range restriction, the most common type of range restriction, in comparison with the conventional practice of applying the Thorndike Case II correction for direct range restriction. Hunter et al.'s pr...
This study uses meta-analysis of an extensive predictive validity data-base to explore the boundary conditions for the validity of. the structured interview as presented by McDaniel, Whetzel, Schmidt, and Maurer (1994). The interview examined here differs from traditional structured interviews in being empirically constructed, administered by telep...
Presents 40 questions, commentaries, and rebuttals on the generalizability of validities of employment tests of cognitive abilities and meta analysis. (BL)
The situational specificity hypothesis in personnel selection holds that variation in observed validity coefficients across studies for the same test and job is due to subtle variations from setting to setting in what constitutes job performance. This hypothesis therefore predicts that, if the setting does not vary, validity will not vary. Using da...
This study examined the construct validity of personality scales, a personality-based integrity test, and homogenous biodata scales as reflected in their ability to discriminate white collar criminals from other white collar employees. The sample included 365 prison inmates incarcerated in 23 federal correctional institutions for white collar offen...
This article describes and illustrates the adaptation of the linear-regression-based decision-theoretic equations used to estimate the dollar impact of valid selection procedures on workforce productivity to the evaluation of intervention programs designed to improve job performance. The appropriate equations are derived and explained, methods for...
Using an improved version of the Bayesian validity generalization model presented in Schmidt and Hunter (1977), this study showed that most of the between-study variation in observed validity coefficients of various aptitude and ability tests for two types of clerical work and for the job of first-line supervisor is artifactual in nature. These res...
As a result of rejection by personnel psychologists of the erroneous law of small numbers and of the adoption of correct inferential procedures, the future of criterion-related validity promises to be bright. Probable future developments include: (a) widespread appreciation of the low statistical power characteristic of small samples; (b) rejection...
Interrater correlations do provide an index of reliability of job performance ratings. We show that the arguments presented by Murphy and DeShon (2000) lead to the radical conclusion that traditional measurement models–both classical theory and generalizability theory models–can be used neither with job performance ratings nor with other measures u...
This study examined the questions of whether employment and educational tests demonstrate either differential validity or test unfairness to Hispanic Americans relative to the majority group. Relevant data from 19 published and unpublished studies formed the basis of the review and analysis. Results for employment tests, based on a very large amoun...
The Schmidt-Hunter interactive validity generalization procedure was applied to validity data for cognitive abilities tests for law enforcement occupations. Both assumed artifact distributions, and distributions of artifacts constructed from information contained in the current sample of studies were used to test the hypothesis of situational speci...
This comment shows that the conclusion of Schmitt, Gooding, Noe, and Kirsch (1984) that their meta-analytic findings are inconsistent with earlier validity generalization work is in error. The findings in their study that less variance than previously reported was due to sampling error are a result of their larger average sample sizes. Their claim...
The situational specificity hypothesis of selection procedure validity makes two predictions. The first is that variation in observed validities across settings is caused by real differences in what constitutes job performance. Validity generalization studies to date have provided disconfirming evidence for this prediction by showing that the obser...
This note examines two potential pitfalls in applying the Cleary or regression model of test fairness. The first lies in a misinterpretation of significance tests on intercept differences which can result when the researcher is unaware of the properties of analysis of covariance tests for intercept differences and relies on computer printouts of re...
In a previous study, Schmidt, Hunter, Croll and McKenzie (1983) demonstrated that estimates of the validity of cognitive tests made by highly trained and experienced judges are more accurate than empirical estimates obtained from small-sample validity studies. The present study examined whether less experienced judges could also produce accurate es...
Range restriction in most data sets is indirect, but the meta-analysis methods used to date have applied the correction for direct range restriction to data in which range restriction is indirect. The authors show that this results in substantial undercorrections for the effects of range restriction, and they present meta-analysis methods for makin...
There has been controversy over the years about whether specific mental abilities increment validity for predicting performance above and beyond the validity for general mental ability (GMA). Despite its appeal, specific aptitude theory has received only sporadic empirical support. Using more exact statistical and measurement methods and a larger d...
The common practice in meta-analyses and in individual studies of correcting for direct range restriction even though range restriction is actually indirect has long been known to lead to undercorrection, but this error has been assumed to be small. Using validity generalization data sets for 4 jobs, this study calibrated this error by comparing me...
A database integrating 90 years of empirical studies reporting intercorrelations among rated job performance dimensions was used to test the hypothesis of a general factor in job performance. After controlling for halo error and 3 other sources of measurement error, there remained a general factor in job performance ratings at the construct level a...
This study found mixed support for the hypothesis that the difference in criterion-related validity between unstructured and structured employment interviews is due solely to the greater reliability of structured interviews. Using data from prior meta-analyses, this hypothesis was tested in 4 data sets by using standard psychometric procedures to r...
Meta-analysis is arguably the most important methodological innovation in the social and behavioral sciences in the last 25 years. Developed to offer researchers an informative account of which methods are most useful in integrating research findings across studies, this book will enable the reader to apply, as well as understand, meta-analytic met...
The psychological construct of general mental ability (GMA), introduced by C. Spearman (1904) nearly 100 years ago, has enjoyed a resurgence of interest and attention in recent decades. This article presents the research evidence that GMA predicts both occupational level attained and performance within one's chosen occupation and does so better tha...
On the basis of an empirical study of measures of constructs from the cognitive domain, the personality domain, and the domain of affective traits, the authors of this study examine the implications of transient measurement error for the measurement of frequently studied individual differences variables. The authors clarify relevant reliability con...
Scientists have known for centuries that little can be proven from single studies. The elimination of alternate hypotheses requires the work of many researchers in many studies. Because of the unavoidable presence of sampling error, this problem is magnified when the basic data in the study are statistical. Yet the dominant pattern of review papers...
Until recently, research focus has been on a variety of demographic, attitudinal, and organizational variables in predicting and explaining absenteeism. If personality traits predict absenteeism, then it may be possible to use measures of these traits to identify and select job applicants and thereby reduce absenteeism rates. In this research, our...
The construct meaning of assessment center evaluations is an important unresolved issue in I/O psychology. This study hypothesized that Cognitive Ability and personality traits are primary correlates of evaluators’ overall assessment ratings (OARs). Meta–analysis results based on 65 correlations indicate the following mean construct–level correlati...
Most theorizing on the relationship between corporate social/environmental performance (CSP) and corporate financial performance (CFP) assumes that the current evidence is too fractured or too variable to draw any generalizable conclusions. With this integrative, quantitative study, we intend to show that the mainstream claim that we have little ge...
Reports the death of John E. Hunter. His various contributions to the areas of validity generalization, differential validity/selection fairness, and selection utility are noted. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
