Article

A Meta-Analysis of Incremental Validity and Nomological Networks for Subordinate and Peer Rating

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

Use of subordinates and peers for job performance ratings has increased markedly in recent years, on the assumption that these sources cover different parts of the criterion space and provide incremental validity over and above supervisors. This study directly tested this assumption using a meta-analytically derived correlation matrix including supervisor-, subordinate-, and peer-ratings as well as objective performance measures. Hierarchical regression results showed that both subordinates and peers accounted for significant variance in objective measures, over and above other sources. Further meta-analyses investigated nomological networks of subordinate and peer ratings, specifically relations with personality and ability measures. Ability correlations for peers and especially subordinates were lower than those previously found for supervisors. Correlations for personality dimensions affiliation and agreeableness, although low, were somewhat higher than previous supervisor values. Results provide justification for the added expense of including subordinates and peers in a rating system.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Alcuni contributi scientifici hanno esaminato la convergenza tra le valutazioni dei diversi raters, evidenziando che le diverse fonti di valutazione presentano bassi livelli di accordo tra loro (Conway & Huffcutt, 1997;Heidemeier & Moser, 2009), probabilmente perché esse osservano i comportamenti organizzativi da differenti prospettive (Murphy & Cleveland, 1995). In tal senso si possono leggere i contributi che avvalorano la validità "incrementale" delle fonti multiple in un sistema di valutazione; ciò significa che le valutazioni di colleghi e collaboratori spiegano una porzione di varianza aggiuntiva nelle misure oggettive di prestazione rispetto alla percentuale di varianza spiegata dai capi (Conway, Lombardo & Sanders, 2001). Inoltre, alcune ricerche hanno messo in luce la relazione dei criteri oggettivi di prestazione con le valutazioni dei capi (Bommer, Johnson, Rich, Podsakoff & MacKenzie, 1995), e successivamente con le valutazioni di colleghi e collaboratori (Conway et al., 2001). ...
... In tal senso si possono leggere i contributi che avvalorano la validità "incrementale" delle fonti multiple in un sistema di valutazione; ciò significa che le valutazioni di colleghi e collaboratori spiegano una porzione di varianza aggiuntiva nelle misure oggettive di prestazione rispetto alla percentuale di varianza spiegata dai capi (Conway, Lombardo & Sanders, 2001). Inoltre, alcune ricerche hanno messo in luce la relazione dei criteri oggettivi di prestazione con le valutazioni dei capi (Bommer, Johnson, Rich, Podsakoff & MacKenzie, 1995), e successivamente con le valutazioni di colleghi e collaboratori (Conway et al., 2001). ...
... Una successiva meta-analisi (Conway et al., 2001) conferma che ogni fonte di valutazione spiega una porzione di varianza unica, e che il disaccordo che ne deriva fornisce dunque informazioni valide. ...
Article
Full-text available
The present study aims to contribute to the research on 360° feedback by investigating inter-rater agreement and the predictive validity of the different sources. The sample consisted of 35 managers from an Italian manufacturing firm in the auto-motive sector; each was assessed by his- or herself, the supervisors, the colleagues and the co-workers (the average number of received evaluations was 14), through a questionnaire tapping five broad domains of competence. Results showed that self-evaluations did not correlate with any other sources, whereas the colleagues’ evaluations correlated with those expressed by supervisors and co-workers. Furthermore, colleagues’ evaluations were the only significant predictors of subsequent ratees’ goal achievement. The practical implications of these findings will be discussed, along with future research perspectives.
... Other studies have focused on organizational outcomes, with measures such as organizations' net profit margin (Conway et al., 2001;Koene, Vogelaar, & Soeters, 2002;Waldman, Ramirez, House, & Puranam, 2001), businessunit sales (e.g., Barling, Weber, & Kelloway, 1996;De Hoogh et al., 2004), and the percentage of business unit performance goals that were met (J. M. Howell & Avolio, 1993). ...
... Subjective and objective effectiveness measures are shown to yield different information and might reflect a different perspective of managerial effectiveness (e.g., Conway et al., 2001). Therefore, we include both types of outcome measures in the current study and relate them to managerial competencies as perceived by different rater groups. ...
... Supervisors expect their reporting managers to produce good results, to contribute to the unit, and to approve behaviors that motivate others (Conway et al., 2001). This may reflect an ILT in which both relationshipsand results-oriented competencies play a role. ...
Article
This study examined the predictive value of multisource ratings of managerial competencies for managerial and organizational effectiveness. Data from 155 subordinates, 59 peers, and 28 supervisors were gathered in order to provide insight into their perceptions on managerial competencies for their managers. With regard to the outcome variable (i.e., effectiveness), both individual-level (subordinates’, peers’, and supervisors’ ratings of managers) and organizational-level (Balanced Scorecard) measures were used. As expected, subordinates, peers, and supervisors have distinct perspectives on the managerial competencies that are relevant for effectiveness. Moreover, the specific managerial competencies differ in terms of their predictive validity respectively for managerial and organizational effectiveness. The outcomes of our study suggest that a multisource and multimethod approach is valuable in assessing both managerial competencies and managerial and organizational effectiveness. Several implications for human resource management practices are discussed.
... Specifically, the Reputation by subordinates reflects what is unique in the perspective provided by subordinates, and not shared with the views provided by superiors, peers, or the leader themselves. Considering that much of a leader's behavior involves and is directed towards subordinates (Hansbrough et al., 2015), and subordinates have many opportunities to witness their leader in a leadership role (Conway et al., 2001), subordinate-ratings are probably the most common source of observer-rated leadership. Moreover, compared to the leader's peers and superiors, subordinates should have a unique perspective on various leader behaviors including their motivational style and individualized consideration (Hiller et al., 2011). ...
... In a second research question (RQ2), we will focus on the relative size of the different LARI factors. Grounded in the realistic accuracy model (Funder, 2012), the relative size of the unique observer perceptions of subordinates is generally expected to be larger, compared to the unique perceptions of peers and superiors, as subordinates would have more opportunities to observe and interact with focal leaders (e.g., Conway et al., 2001;Hansbrough et al., 2015;Hiller et al., 2001;Lee & Carpenter, 2018). Importantly, however, in the current study, a number of leadership dimensions will be taken into account, each focusing on substantially different aspects of leadership. ...
Article
Full-text available
Multisource leadership ratings rely on the assumption that –in addition to the leader’s self-evaluation– different rater groups (i.e., subordinates, peers and superiors) bring in unique perspectives and thus provide a more well-rounded analysis of the leader’s behavior. However, the way in which multisource data are typically treated in research offers little information about the precise levels of overlap and uniqueness that are encapsulated in these different perspectives. Drawing on the Trait-Reputation-Identity (TRI) model, we propose a model that conceptualizes these shared and unique perspectives in terms of latent factors reflecting respectively (i) the consensus about the leader (i.e., the Leadership Arena), (ii) the impressions conveyed to others that are distinct from self-perceptions (i.e., the leader’s Reputation), and (iii) the unique self-perceptions of the leader (i.e., the leader’s Identity). This Leadership Arena-Reputation-Identity (LARI) model is formalized by means of bifactor modeling, which allows to statistically decompose the variance captured by multisource ratings. The LARI model was tested against five alternative models in two large multisource samples (N1 leaders = 537, N1 observers = 7,337; N2 leaders = 1,255, N2 observers = 15,777), each using different leadership instruments. In both samples, the LARI bifactor model outperformed the alternative models. A subsequent variance decomposition showed that each rater source indeed provides unique information about the target’s behavior, although in varying degree. Across all leadership dimensions in both samples, superiors consistently provided the largest share of unique information among the three observer groups. Implications and future directions are discussed.
... This is surprising considering the possible consequences of second-order sampling variation. Although it is difficult for even two meta-analyses to generate estimates in exactly the same way, it is important to be aware of the problem and to try to minimize differences (refer to Eby et al., 1999;Conway et al., 2001 for good examples). ...
... For example, Conway et al. (2001) explained clearly that because they chose not to correct for artifacts in their own meta-analysis, they uncorrected the values from other meta-analysis to make sure all values in the input matrix were uncorrected. Butts et al. (2012) mentioned that when producing the input correlation matrix, they combined their corrected correlations with "previously published meta-analytic correlations that were derived with the same techniques and artifact corrections (i.e. ...
Article
Researchers have become increasingly interested in conducting analyses on meta-analytic correlation matrices. Methodologists have provided guidance and recommended practices for the application of this technique. The purpose of this article is to review current practices regarding analyzing meta-analytic correlation matrices, to identify the gaps between current and best practices, and to offer a comprehensive set of recommendations regarding the planning, collection, analysis, and interpretation of studies that utilize meta-analytic correlation matrices. Copyright © 2016 John Wiley & Sons, Ltd.
... While the current results based on objective adaptive performance scores appear to be less in line with the meta-analytic findings devoted to other performance facets (Rudolph et 33 al., 2018), it is important to note that many of the studies included in the mentioned metaanalysis assessed performance via self-reports rather than objectively (Weikamp & Göritz, 2016;Zacher et al., 2010). Previous research has shown that subjective and objective performance measures (Christian et al., 2017;Conway et al., 2001;Stasielowicz, 2019Stasielowicz, , 2020 differ in their relationships with other constructs. The varying strength of the relationship confirms that subjective and objective measures cannot be used interchangeably. ...
Article
Full-text available
People who perceive more opportunities in their occupational future often report better well-being and motivation. A positive correlation with performance has also been reported, but until now, only the relationship with two broad performance dimensions, task and contextual performance, has been examined extensively. Furthermore, performance ratings often rely on self-reports, which can be biased. The present study (N = 258) helps close these research gaps. First, it examines the relationship between focus on opportunities and a specific performance facet (i.e., adaptive performance). Second, performance is assessed both subjectively and objectively. The distinction proved critical. After adjusting for education, focus on opportunities was related to self-reported adaption to change, but there was no substantial evidence that focus on opportunities promotes objective adaptive performance . Thus, it cannot yet be recommended to use focus on opportunities in organizational and educational contexts for selecting and training highly adaptable employees or students.
... While the current results based on objective adaptive performance scores appear to be less in line with the meta-analytic findings devoted to other performance facets (Rudolph et 33 al., 2018), it is important to note that many of the studies included in the mentioned metaanalysis assessed performance via self-reports rather than objectively (Weikamp & Göritz, 2016;Zacher et al., 2010). Previous research has shown that subjective and objective performance measures (Christian et al., 2017;Conway et al., 2001;Stasielowicz, 2019Stasielowicz, , 2020 differ in their relationships with other constructs. The varying strength of the relationship confirms that subjective and objective measures cannot be used interchangeably. ...
Preprint
Full-text available
People who perceive more opportunities in their occupational future often report better well-being and motivation. A positive correlation with performance has also been reported, but until now, only the relationship with two broad performance dimensions, task and contextual performance, has been examined extensively. Furthermore, performance ratings often rely on self-reports, which can be biased. The present study (N = 258) helps close these research gaps. First, it examines the relationship between focus on opportunities and a specific performance facet (i.e., adaptive performance). Second, performance is assessed both subjectively and objectively. The distinction proved critical. After adjusting for education, focus on opportunities was related to self-reported adaption to change, but there was no substantial evidence that focus on opportunities promotes adaptive performance on a specific task. Thus, it cannot be recommended to use focus on opportunities in organizational and educational contexts for selecting and training highly adaptable employees or students.
... One potential contributing factor reflects the differences between assessment methods of performance adaptation that were described in the preceding sections (subjective ratings vs objective scores). Interestingly, in other research areas the relationship between cognitive abilities and performance facets is usually stronger when objective performance scores rather than subjective ratings are used (i.e., Conway, Lombardo, & Sanders, 2001). Also within the performance adaptation research field the two assessment methods (subjective and objective) are not interchangeable as indicated by the reported intercorrelations ranging from small (r = 0.10) to medium (r = 0.43) at best (Baumgartner, 2015;Stokes et al., 2010). ...
Preprint
Performance adaptation is required in many areas (e.g., dealing with emergencies or newsoftware in the work and educational context). Therefore, predicting successful reactions tochanges is important. Motivated by heterogeneous findings, the present meta-analysisexamines to what extent individual differences in cognitive abilities are relevant toperformance adaptation. A three-level meta-analysis based on 133 correlations (N = 37,963)was conducted in order to estimate the mean strength of the relationship. Furthermore, severalmoderator analyses were carried out (e.g., task complexity, subjective vs objective assessmentof performance adaptation) to explain the variability of the effect sizes. As expected,cognitive ability was positively related to performance adaptation (r = .21). However, therelationship was stronger when objective performance adaptation scores (r = .25) rather thansubjective ratings were used (r = .11). Furthermore, cognitive ability seems to be particularlyrelevant for highly dynamic tasks (r = .31). Overall, cognitive ability can be a useful predictorof actual performance adaptation, which has implications for the selection context (e.g.,selection of students or employees with high adaptability). Furthermore, cognitive abilityseems to be more relevant than other personality-based predictors of performance adaptationthat have been examined meta-analytically (Big Five and goal orientation).
... All meta-analytic data were systematically extracted, including the name, source, and description of the focal variable, its total number of independent samples (i.e., k), total sample size (i.e., N), mean sample-size weighted observed effect size (e.g., r, d , or z), and an index of between-studies variability (e.g., standard error, standard deviation, confidence and/or credibility intervals). Select meta-analyses (i.e., Conway, Lombardo, & Sanders, 2001;Hogan & Holland, 2003;Hough, 1992;Huang, Ryan, Zabel, & Palmer, 2014) reported results for lower order traits of extraversion, which correspond to trait taxonomies that are similar to, but not identical with, the Big Five. In such cases, we used composites to estimate extraversion-level effects (Ghiselli, Campbell, & Zedeck, 1981). ...
Article
Full-text available
How and to what extent does extraversion relate to work relevant variables across the lifespan? In the most extensive quantitative review to date, we summarize results from 97 published meta-analyses reporting relations of extraversion to 165 distinct work relevant variables, as well as relations of extraversion's lower order traits to 58 variables. We first update all effects using a common set of statistical corrections and, when possible, combine independent estimates using second-order meta-analysis (Schmidt & Oh, 2013). We then organize effects within a framework of four career domains-education, job application, on the job, and career/lifespan-and five conceptual categories: motivations, values, and interests; attitudes and well-being; interpersonal; performance; and counterproductivity. Overall, extraversion shows effects in a desirable direction for 90% of variables (grand mean ρ̄ = .14), indicative of a small, persistent advantage at work. Findings also show areas with more substantial effects (ρ̄ ≥ .20), which we synthesize into four extraversion advantages. These motivational, emotional, interpersonal, and performance advantages offer a concise account of extraversion's relations and a new lens for understanding its effects at work. Our review of the lower order trait evidence reveals diverse relations (e.g., the positive emotions facet has consistently advantageous effects, the sociability facet confers few benefits, the sensation-seeking facet is largely disadvantageous), and extends knowledge about the functioning of extraversion and its advantages. We conclude by discussing potential boundary conditions of findings, contributions and limitations of our review, and new research directions for extraversion at work. (PsycINFO Database Record (c) 2019 APA, all rights reserved).
... In short, supervisory ratings are reliable job performance measures, and even more so for the quality dimension of job performance, a core dimension for knowledge workers. Many organizations complement supervisory ratings of job performance with peer ratings and 360-degree appraisals, which provide some incremental validity over and above supervisory ratings (Conway, Lombardo, & Sanders, 2001). However, the consensus among researchers is that the standalone reliability and validity of these measures do not compare to supervisory ratings, an also imperfect -albeit superior -measure of job performance for knowledge workers. ...
Article
Full-text available
An evidence-based research to find out which skills and abilities will allow knowledge workers to thrive in the age of automation.
... Conscientiousness was related to overall job performance. Also consistent with research on domestic managers and leaders (Bono/Judge 2004;Conway et al. 2001;Hough et al. 1998;Mount et al. 1998), Extraversion and Agreeableness also showed substantial relations with performance, particularly interpersonal performance, among expatriates (most of whom have managerial responsibility). Contrary to common beliefs among laypeople and practitioners, Openness showed negligible to weak relations with expatriate performance. ...
... Because the identity theory posits that individuals develop their sense of self through the information they receive from their environment, looking at LMX as well as trust in leader allows us to further validate this claim by considering multiple leader-subordinate viewpoints. Moreover, previous research has established that the source of the performance rating (i.e., supervisor and peer) is likely to yield different evaluations (Conway & Huffcutt, 1997) and is likely to account for unique variance in performance ratings (Conway, Lombardo, & Sanders, 2001). By looking at two performance assessment sources in a two-study constructive replication design, we expand the purview of the leader relationship-performance associations. ...
Article
Full-text available
Despite being a subject of scholarly inquiry for nearly a century, some components of person–environment fit remain enigmatic. This research seeks to explore the relational factors that are associated with employee identity and how this lens can provide explanatory factors that link leader–employee relationships to employee performance. Across a two‐study, multi‐rater constructive replication design, our results support the tenets of the identity theory that suggests the quality of the leader–employee relationships informs employees of their person–organization (PO) and person–job (PJ) fit, leading to higher performance. Across both the studies, high‐quality leader–employee relations (conceptualized as LMX and trust in leader) were related to higher levels of PO and PJ fit. Our results also indicate that PO and PJ fit uniquely drive higher levels of performance as rated by multiple constituents, helping delineate the impact of PO and PJ fit on work outcomes. Specifically, in Study One (N = 111), PJ fit mediated the relationship between LMX and leader‐rated performance, whereas in Study Two (N = 94), PO fit mediated the relationship between trust in leader and peer‐rated performance. This work provides preliminary support that leaders, through their relationships, can help shape employee fit perceptions and ultimately impact performance. Such knowledge can inform organizations and leaders, and emphasizes the formative role that leaders play in the organizational lives of their employees.
... There are several studies that has focused on non-self-report based organisational outcomes, as criterion measures to assess the effects of for example leadership style on organisational outcomes (Koene, Vogelaar, & Soeters, 2002;Waldman, Ramirez, House, & Puranam, 2001;Conway, Lombardo, & Sanders, 2001;De Hoogh et al., 2004;Barling, Weber, & Kelloway, 1996;Howel & Avolio, 1993). While reducing common-source and common-method bias, organisational measures raise criticism by being overly narrow (Bommer, Johnson, Rich, Podsakoff, & MacKenzie, 1995), thus suffering from criterion deficiency. ...
... All meta-analytic data were systematically extracted, including the name, source, and description of the focal variable, its total number of independent samples (i.e., k), total sample size (i.e., N), mean sample-size weighted observed effect size (e.g., r, d , or z), and an index of between-studies variability (e.g., standard error, standard deviation, confidence and/or credibility intervals). Select meta-analyses (i.e., Conway, Lombardo, & Sanders, 2001;Hogan & Holland, 2003;Hough, 1992;Huang, Ryan, Zabel, & Palmer, 2014) reported results for lower order traits of extraversion, which correspond to trait taxonomies that are similar to, but not identical with, the Big Five. In such cases, we used composites to estimate extraversion-level effects (Ghiselli, Campbell, & Zedeck, 1981). ...
... Although multisource feedback relies on observers' perceptions, there is evidence that multisource ratings correspond to objective measures of leadership effectiveness such as production, profit, revenue and sales (Conway, Lombardo & Sanders, 2001;Smither, London & Reilly, 2005). In this study we also took steps to reduce the potential for rater bias, by controlling for politicians' experience and gender (Paustian-Underdahl, Walker & Woehr, 2014), and by using multiple ratings for the same leader provided by observers with a range of perspectives (i.e. ...
Article
Full-text available
Are the traits preferred by voters also associated with success in political office? Drawing on the ascription-actuality trait theory of leadership the present study examines whether traits ascribed to politicians predict leadership outcomes differently to the actual traits they possess. We collected self-ratings of politicians' personality (N = 138) using the NEO-PI-R (actual traits) and observer ratings of politicians' facial appearance (ascribed traits) to examine their relationship with (a) leadership emergence, measured using share of vote in election, and (b) in-role leadership effectiveness, rated anonymously by political and local authority colleagues. Facial appearance predicted leadership emergence but not effectiveness. Personality had a more nuanced relationship with leadership outcomes. Conscientiousness predicted effectiveness but not emergence, and Agreeableness revealed a trait paradox, positively predicting emergence and negatively predicting effectiveness. These findings suggest a need to understand the contested nature of political leadership and qualities required for different aspects of political roles.
... Such ratings are very suitable criterion measures as there is clear evidence demonstrating a link between such ratings and workplace effectiveness. For example, in a seven year follow up study by McEvoy & Beatty (1989), team member ratings predict managerial performance better than assessment center outputs, while a meta-analysis by Conway, Lombardo, and Sanders (2001) found peer and subordinate ratings accounted for significant and unique variance in a variety of objective performance measures. Numerous studies (e.g., Smither & Walker, 2004) have shown that 360 ratings correlate with performance appraisal ratings. ...
Article
Individual Psychological Assessment (IPA) is a very widely offered service in Organizational Psychology. It generally consists of a psychologist or HR practitioner using a combination of interview and psychometrics to arrive at a detailed assessment of an individual's capabilities in relation to a job they are being considered for. Although much used, this practice has limited supporting evidence of its validity—not least because of the methodological difficulties in conducting research on this subject—and has been criticized accordingly. The current study examines the use of IPAs with 115 middle and senior management level candidates in a civil service context. All candidates completed a set of psychometric measures and had an in-depth interview with a psychologist as part of a standardized process. The ratings made by the assessors were correlated with a criterion measure of potential for promotion derived from multisource feedback ratings obtained on these candidates some months later. Analysis of the results indicated that three of the four attributes rated by assessors correlated significantly with the criterion measure. Further, assessors’ ratings were found to show incremental validity over that provided by psychometric test scores alone. These findings are discussed in terms of the use of IPAs in senior level assessment.
... This leads to avoiding a very common problem that performance assessments are based on social and political considerations, not on objective criteria of performance (Longenecker et al., 1987). Equally important, feedback should be provided in such a timely and constructive manner that is unlikely to result in either acrimony or unnecessary defensiveness (Conway et al., 2001). ...
Article
Full-text available
One of the most important criteria in showing the development level of a country is that financial markets of the country have gained depth. The most important function of financial system is reconstruction of economy and as a result of this, contribution to economical growth by playing a crucial role in the distribution process of capital. From this respect, Stock exchange is the leading indicator in showing countries’ financial depth .The crisis that develop either internally and have a growing contagion effect with globalization, make themselves felt in stock exchange with an increasing severe. After the crisis in 2000-2001, Turkey has lost the growing trend lasting for twenty seven quarter in the following of 2008 crisis. With this study, we have tried to explain where the crisis has started via national share indices of Istanbul stock exchange, how it has developed and how it has gained and in which way it has continued oscillation with the help of tables and graphics. Key Words: Financial Crisis, Stock Exchange, Istanbul Stock Exchange.
... Multi-rater feedback, or 360-degree assessment, provides leaders insights into how their leadership behaviors are perceived by others with whom they commonly interact. Multi-rater feedback offers the benefit of summarizing input from several distinct sources, all reflective of the assessed individual's workplace performance and all relevant to incorporate in one's perspective of one's own behavior and performance at work (Conway, Lombardo, & Sanders, 2001;London & Smither, 1995). ...
... Multi-rater performance evaluations (like 360degree feedback) are increasing in popularity as organizations are realizing the benefits of triangulating performance data by gathering different perspectives (Conway, Lombardo, & Sanders, 2001;Van Hooft, Van Der Flier, & Minne, 2006). Interestingly, research results obtained from such instruments has substantiated Thornton's (1980) observation that "individuals have a significantly different view of their own job performance than that held by other people" (Harris & Schaubroeck, 1988, p. 43). ...
Article
Full-text available
Multi-rater feedback systems are increasingly being used in organizations for assessment and development. Research suggests self-ratings are often uncorrelated with other-ratings, which has implications for ratee receptivity to feedback. Prior research suggests ratee personality may offer insight into self-other rating discrepancy. We explored the extent to which five work-related personality dimensions offer explanatory variance in self-other rating discrepancy using archival data of 527 employees from a large financial organization. Analyses of 487 self-peer dyads and 501 self-supervisor dyads revealed low agreement among self-, supervisor-, and peer-ratings of performance. Self-raters high on extraversion, dominance, cultural conformity, cynicism, and detail orientation were somewhat likely to over-rate their performance as compared to peer ratings. Cynical and detail-oriented raters were likely to over-rate their performance as compared with supervisor ratings. Detail orientation and cultural conformity interacted to predict self-other rating discrepancies for both supervisors and peers. Implications for research and practice are discussed.
... Examples of peer-supported systems include student-led workshops (Hess, 2007), student-to-student learning partnerships (Chio & Fandt, 2007), and peer mentoring (Parker, Hall, Kram, in press). Interestingly, the use of student peers as evaluators also builds upon contemporary multisource assessment practices found in industry (e.g., Conway, Lombardo, & Sanders, 2001;Drexler, Beehr, & Stetz, 2001). ...
Article
We describe the impact of a centralized electronic peer evaluation system on the group effectiveness of undergraduate business students over a pair of semesters. Using a quasi-experimental design, 389 undergraduate students evaluated, and were evaluated by, their peers using a web-based system that captures peer evaluations in quantitative and qualitative formats and allows for the reception of anonymous feedback. Results show that the effectiveness of students, as perceived by their peers, increased over semesters. This effect could be directly linked to the use of the system. The results of this study underscore the benefit of centralizing peer evaluations for the assessment of important skills and their development in higher education. The implication of these results and possible avenues of research are detailed. A common challenge for business school profes-sors is the assessment and management of indi-vidual contribution to group work (Brown, Rust, & Gibbs, 1994; Mello, 1993). While the outcomes of group work are typically under the purview of ed-ucators, much value resides in monitoring the in-dividual activities that precede the final group re-port, project, or presentation. Peer evaluations are meant to capture information related to internal group processes—information that is inaccessible to anyone other than group members. Peer evalu-ations not only offer a useful window into "what happened" (evaluation tool) but also they repre-sent a valuable learning tool in providing students with the experience of evaluating others and re-ceiving feedback from their peers on their behav-iors. These processes also expose students to an important and difficult organizational duty, that of evaluating, and being evaluated by, others.
... The scores are usually averaged into one leader effectiveness score. Subordinate ratings of leader performance have been 22 found to correlate with objective measures of performance (Conway, Lombardo, & Sanders, 2001). ...
Article
Information technology (IT)-business alignment continues to be a challenge for businesses. A potential reason may be the absence of relevant IT leadership capable of engaging IT and business units in a way that leads to alignment. Based on the knowledge integration view of organizations and the strategic leadership literature, we propose a model linking chief information officer (CIO) leadership style to IT-business alignment. We examine how ambidextrous CIO leadership behavior influences IT-business alignment through knowledge integration mechanisms, namely mechanisms that develop shared domain knowledge between IT and business specialists and mechanisms for collaborative integration of specialized IT and business knowledge. Analysis of cross-industry data collected from CIOs, top management team members, and IT specialists indicates that CIOs who can display both transformational and transactional leadership promote these mediating mechanisms. Moreover, the results show that these mechanisms have a substitutive effect, in that the effect of one mechanism diminishes when the other exists at a high level.
Article
Full-text available
Although affect is a factor likely to impact the success of innovation, little research has been done on the relationship between affect and innovation implementation performance (i.e., an employee’s ability to successfully implement innovative ideas and practices). We address this oversight by adopting a social network approach to examine relational energy (i.e., how energized one person is when interacting with another) as a form of high activation positive affect likely to influence innovation implementation. We test our hypotheses using a sample of employees in a pharmaceutical research firm (Study 1). Our results indicate that the number of people an employee goes to for political support who report being energized by that employee is positively related to innovation implementation performance. In contrast, the number of people an employee seeks out for political support who are not energized by that employee has negative implications for innovation implementation performance. The average network centrality of an individual’s energized network contacts also relates to implementation performance, with this effect being stronger for employees not in a managerial position. A scenario-based experiment (Study 2) provides support for the causal linkage between feeling energized by a co-worker and one’s willingness to provide instrumental help to the co-worker.
Article
Organizations must make decisions about which resources to prioritize and to whom to allocate them. In order to do so, evaluation practices are used to clarify the quality of employees. The purpose of this study is to examine the relationship between performance appraisal and personal reputation, which are two types of personnel appraisal practices used in organizations. Using questionnaire data and HR data analysis from a Japanese manufacturing company, we empirically took on this problem. The results of questionnaire data and HR data analysis from a Japanese manufacturing company showed that the capital that affects the two evaluation scores was found to be different from each other. In addition, psychological capital was found to be an antecedent of both.
Article
Agreeableness impacts people and real-world outcomes. In the most comprehensive quantitative review to date, we summarize results from 142 meta-analyses reporting effects for 275 variables, which represent N > 1.9 million participants from k > 3,900 studies. Arranging variables by their content and type, we use an organizational framework of 16 conceptual categories that presents a detailed account of Agreeableness’ external relations. Overall, the trait has effects in a desirable direction for 93% of variables (grand mean 𝜌M = .16). We also review lower order trait evidence for 42 variables from 20 meta-analyses. Using these empirical findings, in tandem with existing theory, we synthesize eight general themes that describe Agreeableness’ characteristic functioning across variables: self- transcendence, contentment, relational investment, teamworking, work investment, lower results emphasis, social norm orientation, and social integration. We conclude by discussing potential boundary conditions of findings, contributions and limitations of our review, and future research directions.
Book
Full-text available
A obra é destinada a profissionais da educação em formação, investigadores(as) do campo da educação e da saúde e educadores(as) e estudantes que atuam com formação docente e que buscam, por meio das atividades aquáticas, a promoção da saúde e a melhoria da qualidade de vida. A obra tem como pretensão tocar o(a) leitor(a), de forma a reconhecer o(a) discente como sujeito de conhecimento, aquele capaz de tomar decisões e de fazer escolhas em meio aquático. O conteúdo é diferenciado, uma vez que entende as atividades aquáticas para além dos quatro estilos de nado e faz abordagem das diversas possibilidades pedagógicas de explorar o ambiente aquático, na perspectiva da indissociabilidade entre o ensino, a pesquisa e a extensão
Article
Full-text available
Organizations frequently rely on peer performance ratings to capture employees' unique and difficult to observe contributions at work. Though useful, peers exhibit meaningful variance in the accuracy and informational utility they offer about ratees. In this research, we develop and test theory which suggests that raters' social network positions explains this variance in systematic ways. Drawing from information processing theory, we posit that members who occupy core (peripheral) positions in the network have greater (less) access to firsthand and secondhand performance information about ratees, which is in turn associated with more (less) accurate performance ratings. To overcome difficulties in obtaining a "true" performance score in interdependent field settings, we employ an external criterion comparison method to benchmark our arguments, such that larger validity coefficients between established predictors of performance (i.e., a ratee's general mental ability [GMA] and conscientiousness) and peer performance ratings should reflect more (less) accurate ratings for core (peripheral) members. In Study 1, we use an organization-wide network in a technology startup company to examine the validity coefficient of a ratee's GMA on performance as rated by central versus peripheral members. In Study 2, we attempt to replicate and extend Study 1's conclusions in team networks using ratee conscientiousness as a benchmark indicator. Findings from both studies generally support the hypotheses that core network members provide distinct, and presumably more accurate, peer performance ratings than peripheral network members. (PsycInfo Database Record (c) 2020 APA, all rights reserved).
Data
Full-text available
Supplementary Information for "A century of research on conscientiousness at work"
Article
Full-text available
Significance Conscientiousness (C) is the most potent noncognitive predictor of occupational performance. However, questions remain about how C relates to a plethora of occupational variables, what its defining characteristics and functions are in occupational settings, and whether its performance relation differs across occupations. To answer these questions, we quantitatively review 92 meta-analyses reporting relations to 175 occupational variables. Across variables, results reveal a substantial mean effect of ρ ¯ M = 20 . We then use results to synthesize 10 themes that characterize C in occupational settings. Finally, we discover that performance effects of C are weaker in high-complexity versus low- to moderate-complexity occupations. Thus, for optimal occupational performance, we encourage decision makers to match C’s goal-directed motivation and behavioral restraint to more predictable environments.
Article
Several meta-analyses have investigated the job-related validities of the traits associated with the Five-Factor Model (FFM). The presence of second-order sampling error, however, might complicate the interpretation of these meta-analyses (i.e., random error across different meta-analyses). The current paper therefore evaluates variability across different meta-analyses and tests for key moderators (i.e., performance criteria, sources of ratings, and context). Results suggested that the variation in the predictive validity of several FFM traits and facets across multiple performance criteria was attributable to second-order sampling error. However, true variation existed for some moderators such as sources of personality information (self-reports versus informant reports). The current analyses therefore enhance knowledge about the usefulness and generalizability of personality traits in work contexts.
Article
Die vorliegende Arbeit beschäftigt sich mit der Messung berufsbezogener Kompetenzen, sofern diese in der Persönlichkeit von Personen begründet liegen. Modelliert werden dabei leistungs- und kompetenzrelevante Daten aus dem Feld der Managementberufe. Der erste Schwerpunkt liegt im Bereich der Messung, indem (intervallskalierte) Rangordnungen von Kompetenzitems durch graphische Drag&Drop-Umgebungen erstellt und gängigen Fragebögen (auf Mehrpunktskalen) gegenübergestellt werden. Solche Forced-Choice-Verfahren, welche man zur Klasse der ipsativen Messinstrumente zählen kann, erfordern ein anderes Auswertemodell als normative Messinstrumente, auf die die klassische Testtheorie anwendbar ist. Deswegen wird – und das ist der zweite methodische Schwerpunkt der Arbeit – ein MDS-basiertes relationales Vorgehen für die Modellierung der Daten vorgeschlagen und anhand von drei Datensätzen aus einem Grossunternehen der Energiewirtschaft überprüft. Es zeigt sich, dass den Vorteilen der Rangordnungen (deutlich höhere Effizienz, Transparenz in der Messung und Unterdrückung von Beurteilertendenzen) kein Nachteil in der Qualität der Modellierung der Daten gegenüber steht, sondern dass das effizientere Forced-Choice-Verfahren ähnliche Gütemasse erreicht wie die in der Praxis wesentlich weiter verbreiteten normativen Verfahren. Bei der MDS werden zudem die relationalen Beziehungen zwischen einzelnen Kompetenzprofilen berücksichtigt und die gesamte Struktur der Daten anhand zweidimensionaler euklidischer Karten abgebildet. Dadurch können Personalbeurteilungsdaten in einer neuen und sowohl für die Wissenschaft als auch für die Praxis zweckmässigen Weise ganzheitlich dargestellt und interpretiert werden. The current work explores the measurement of managerial competencies. The first focus is on the type of measurement. A forced-choice procedure, where the rater puts competency items through drag&drop into an (interval scaled) rankorder is compared to common (multipoint) questionnaires. The data collected through a forced-choice procedure, which belong to the class of ipsative measurement, requires different methods of analysis than normative instruments, where classical test theory can be applied. Therefore – and this is the second methodological focus of this work – a multidimensional scaling technique is suggested in order to modellize the relational structure of the data collected in three different samples of a large utility company. It is shown, that the advantages of a rankorder-methodoligy (highly efficient, transparency of what is measured and avoidance of rater response tendencies) do not compete with the disadvantage of a reduced quality in modelling the data. The introduced forced-choice application does show similar reliability measures as the much more frequently used normative instruments. In addition, when applying MDS, the relational structure of all objects (i.e. competency profiles) is considered and visualized through two-dimensional Euclidian maps. This allows research and practice a new holistic form of visualising and interpreting human performance data.
Article
There is increasing use of peer evaluations in academic settings to assess student performance in team activities. Building on Kenny’s social relations model, the authors analyzed peer ratings from 85 teams of graduate students enrolled in a teamwork and leadership course. Results indicate the presence of greater assimilation at mid-semester and increased consensus for all performance criteria and uniqueness for most performance criteria at the end of the semester. Results further indicate the source of variance varied for different performance dimensions. Implications for the use of peer evaluations are discussed as well as future application and research direction.
Article
The agreement between a leader's self-rating of leadership and ratings from the leader's subordinates, peers, and superiors (i.e., self-other agreement) is critical to understanding leadership, but questions remain regarding the extent to which leaders are aware of their behaviors. This meta-analysis investigates whether leader-observer agreement is influenced by type of observer and type of leadership. First, we examined the convergence (i.e., correlation) between leader- and observer-ratings along several dimensions of leadership (e.g., initiating structure, consideration, contingent reward, and transformational leadership). Our results indicated that leader-observer correlations were generally moderate and of similar magnitudes for task- and relation-oriented behaviors (with the exception of a strong correlation for contingent reward). Next, we compared leaders' and observers' mean-level ratings (i.e., Cohen's d), and found that leaders generally reported lower or similar levels of task-oriented behaviors but higher levels of relation-oriented behaviors. Last, several variables (e.g., sampling method and study purpose) moderated leader-observer convergence. Implications of these findings for research, theory, and practice are discussed.
Article
It is common to add an additional predictor to a selection system with the goal of increasing criterion-related validity. Research on the incremental validity of a second predictor is generally based on forming a regression-weighted composite of the predictors. However, in practice predictors are commonly used in ways other than regression-weighted composites, and we examine the robustness of incremental validity findings to other ways of using predictors, namely, unit weighting and multiple hurdles. We show that there are settings in which the incremental value of a second predictor disappears, and can even produce lower validity than the first predictor alone, when these alternatives to regression weighting are used. First, we examine conditions under which unit weighting will negate gain in predictive power attainable via regression weights. Second, we revisit Schmidt and Hunter's (1998) summary of incremental validity of predictors over cognitive ability, evaluating whether the reported incremental value of a second predictor is different when predictors are unit weighted rather than regression weighted. Third, we analyze data reported in the published literature to discern the frequency with which unit weighting might affect conclusions about whether there is value in adding a second predictor to a first. Finally, we shift from unit weighting to multiple hurdle selection, examining conditions under which conclusions about incremental validity differ when regression weighting is replaced by multiple-hurdle selection. (PsycINFO Database Record
Chapter
This chapter reviews the performance appraisal theory and research to examine the expectations that (a) performance ratings are accurate and (b) performance improves as a result of receiving feedback. It provides a critical analysis of the performance appraisal literature and identify successes and challenges in living up to these expectations. The chapter organizes the review in two major sections. The first major section discusses the predominant strategies and challenges to improve the quality of ratings. The second major section also reviews a research on performance feedback, focusing on the characteristics of feedback that are likely to lead to employee development. To address the influence of rater motivations, rater training could incorporate interventions designed to increase motivation to rate accurately and reduce competing motivations. Future research can work to develop and test the efficacy of rater training programs that address motivations in order to improve the quality of ratings.
Article
In reviewing peer-leader feedback within Multi-Source Feedback programs, the group aggregate agreement (GAA) method is contrasted with self-other agreement (SOA). Past research (Markham, Smith, et al., 2014) has demonstrated convergence problems with GAA for groups of peer raters. To evaluate dyadic convergence, we used the Benchmarks data to investigate two derailment factors (Building & Mending Relationships and Interpersonal Problems) for 4607 peers describing 1505 focal respondents. For high-agreement dyads, the rtotal = − 0.66** with 88% of the combined variance and covariance based on dyadic averages converging as whole units. Only 50% of all dyads demonstrated this type of high convergence. For low agreement dyads, the matching correlation (rtotal = − 0.56**) was almost exclusively a function of within-dyad divergence with only 4% stemming from between-dyad sources. Research implications for evaluating SOA under these agreement conditions are highlighted. Practitioner applications for using an entity-based visualization of dyads also are prototyped and discussed.
Article
To examine complex relationships among variables, researchers in human resource management, industrial-organizational psychology, organizational behavior, and related fields have increasingly used meta-analytic procedures to aggregate effect sizes across primary studies to form meta-analytic correlation matrices, which are then subjected to further analyses using linear models (e.g., multiple linear regression). Because missing effect sizes (i.e., correlation coefficients) and different sample sizes across primary studies can occur when constructing meta-analytic correlation matrices, the present study examined the effects of missingness under realistic conditions and various methods for estimating sample size (e.g., minimum sample size, arithmetic mean, harmonic mean, and geometric mean) on the estimated squared multiple correlation coefficient (R²) and the power of the significance test on the overall R² in linear regression. Simulation results suggest that missing data had a more detrimental effect as the number of primary studies decreased and the number of predictor variables increased. It appears that using second-order sample sizes of at least 10 (i.e., independent effect sizes) can improve both statistical power and estimation of the overall R² considerably. Results also suggest that although the minimum sample size should not be used to estimate sample size, the other sample size estimates appear to perform similarly.
Article
The other articles in this special issue of Human Resource Management Review present meta-analyses of specific topic areas, or articles on methodological issues associated with meta-analyses, within the human resources management field. Ours is a bit different in that we do not present actual meta-analytic results, but instead conduct a thorough review of the field in order to identify areas where meta-analyses have not been conducted. Then, we discuss why such analyses have not been provided, suggestions for how we might like to see research proceed in such areas, and also implications for theory development in these areas of the field. We conclude our paper with some additional thoughts on issues to keep in mind as we seek to utilize meta-analysis to its fullest potential, and thus yield the best results possible.
Article
We extend multisource performance rating (MSPR) construct validity research by examining the pattern of relationships between factor analytically derived MSPR rating source and performance dimension factors and externally measured constructs (e.g., assessment center dimensions, personality constructs, and intelligence). The pattern of relationships among MSPR dimensions and external constructs provides modest construct validity evidence for the MSPR dimensions. In addition, MSPR source factors were differentially correlated with externally measured constructs, suggesting that MSPR source effects represent substantively meaningful source specific variance, as opposed to bias. These findings are discussed in the context of managerial skill diagnosis and the efficacy of collecting performance data from multiple sources.
Article
An Overview of Multisource Assessment The nature of work within corporations is constantly changing, as are the tools used to evaluate performance. Among the most important changes that have occurred are the increasing complexity of the tasks to be performed and the frequency and rapidity with which change occurs in the work environment. Performance evaluation practices, which aim to assess the behaviour of individuals in a work situation, have had to be adjusted in order to account for this new reality. This context has given rise to a significant rise in the use of multisource assessment as a management tool over the past decade. Based on a review of the literature available to date, this article proposes a critical analysis of multisource assessment. The authors present the historical context, followed by a definition of multisource assessment. They then explore the psychometric qualities of multisource assessment as well as the impact of these qualities on the individual being evaluated and on the organization in general. Finally, avenues of future research and outstanding questions are discussed.
Article
Motivating students to productively engage course material and perform to their potential is among our most urgent challenges as STEM educators. Professional organizations commonly seek to motivate employees by providing performance feedback from multiple raters who may include supervisors, peers, subordinates, clients, and others. With support from National Science Foundation grant EEC #1158728, the present study used a newly developed online delivery system to provide personalized multisource feedback to a sample of 206 undergraduate STEM students in a science and technology problem-solving course. PersonalityPad.org is an automated multisource feedback platform that allows users to generate their own personalized multisource feedback. This process incorporates prevalent 360-degree feedback strategies and "best practices" for effective feedback administration. A longitudinal experiment within an interventional framework evaluated the hypothesis that multisource conscientiousness feedback would provoke goal-directedness and motivate adaptive action. Students who received conscientiousness feedback from multiple sources - including friends, parents, and teachers - participated more in class and submitted higher quality homework assignments afterward, leading to significantly higher final course grades (M = 83.90) compared to a control group in the same class (M = 78.79). A structural analysis of relationships among key variables indicated that post-intervention goal-directedness plays a key intermediary role between receiving personalized feedback and achieving subsequent self-development goals. Implications are discussed from academic and social perspectives.
Article
Clear and accessible, Personnel Selection will continue to be the guide that students and professionals alike turn to for practical advice in this complex field. Fully updated edition of highly successful text. Clear, accessible and practical in its approach. Now includes emotional intelligence and counterproductive work behaviours â" not covered in any other text at this level.
Book
It is generally understood that some effective leadership behaviors of Chinese managers differ from those of Western managers. It has also been debated controversially whether Chinese learners can benefit from Western learning approaches. Taking these two aspects into consideration, Jingjing Wang examines whether a global leadership development program from Western countries has as much impact on Chinese managers as on Western managers. She conducts the empirical study within one global corporation originating from Germany and the data were collected from Germany and China. Based on the core results of the study, implications for the globalization of leadership development are discussed.
Article
Full-text available
This study quantified the effects of 5 factors postulated to influence performance ratings: the ratee's general level of performance, the ratee's performance on a specific dimension, the rater's idiosyncratic rating tendencies, the rater's organizational perspective, and random measurement error. Two large data sets, consisting of managers (n = 2,350 and n = 2,142) who received developmental ratings on 3 performance dimensions from 7 raters (2 bosses, 2 peers, 2 subordinates, and self) were used. Results indicated that idiosyncratic rater effects (62% and 53%) accounted for over half of the rating variance in both data sets. The combined effects of general and dimensional ratee performance (21% and 25%) were less than half the size of the idiosyncratic rater effects. Small perspective-related effects were found in boss and subordinate ratings but not in peer ratings. Average random error effects in the 2 data sets were 11% and 18%.
Article
Full-text available
This article summarizes the practical and theoretical implications of 85 years of research in personnel selection. On the basis of meta-analytic findings, this article presents the validity of 19 selection procedures for predicting job performance and training performance and the validity of paired combinations of general mental ability (GMA) and the 18 other selection procedures. Overall, the 3 combinations with the highest multivariate validity and utility for job performance were GMA plus a work sample test (mean validity of .63), GMA plus an integrity test (mean validity of .65), and GMA plus a structured interview (mean validity of .63). A further advantage of the latter 2 combinations is that they can be used for both entry level selection and selection of experienced employees. The practical utility implications of these summary findings are substantial. The implications of these research findings for the development of theories of job performance are discussed.
Article
Full-text available
In this study, we investigated a neglected form of extrarole behavior called taking charge and sought to understand factors that motivate employees to engage in this activity. Taking charge is discretionary behavior intended to effect organizationally functional change. We obtained both self-report and coworker data for 275 white-collar employees from different organizations. Taking charge, as reported by coworkers, related to felt responsibility, self-efficacy, and perceptions of top management openness. These results expand current understanding of extrarole behavior and suggest ways in which organizations can motivate employees to go beyond the boundaries of their jobs to bring about positive change.
Article
Full-text available
The purpose of this study was to investigate the relationship of hardiness to stress and to performance, and to explore its moderating effect on the rela- tionship between stress and performance. Three hundred twenty-six officer cadets in the Israel Defense Forces completed self-report stress questionnaires during four critical course events and hardiness questionnaires at the beginning and end of the course. As hypothesized, hardiness was consistently negatively related to experienced stress. It was also consistently positively related to objectively assessed performance throughout the course and in the subsequent course, and to the first on-the-job performance appraisal a year later. In addition to its main effects, hardiness buffered the cadets from the detri- mental effects of stress on performance. The processes by which hardiness affects performance are discussed.
Article
Full-text available
The purpose of the present study was to investigate the influence of individual and contextual variables on attitudes toward 360-degree feedback systems. It was hypothesized that individual differences as well as contextual factors would influence employees' receptivity to the implementation of a multirater appraisal system. Respondents were 52 employees from a manufacturing company and 23 employees from a telecommunications organization. All respondents had participated in the piloting of a multirater feedback system. The identified variables accounted for nearly 50% of the variance in 360-degree feedback system attitudes. These findings support the notion that organizations must begin to recognize the variety of needs of the workforce and understand the importance of providing a multitude of tools to improve performance. Further analyses indicated that contextual factors appear more important than personality in determining 360-degree attitudes. Implications for organizational practices are suggested along with areas of future research.
Article
Full-text available
360-degree feedback and/or multirater or multisource assessment methods have become one of the most popular and yet least understood management and organizational interventions in recent years. The editors of this special issue devoted to the subject provide (a) an introduction to the current usage and efficacy of these feedback systems in organizational settings, (b) a brief overview of the major themes and issues raised in the 6 articles included in the volume, and (c) a set of general comments and reflections about the state of research and practice on multirater assessment methodologies, with directions for future research.
Article
Full-text available
This article presents a perspective on the relation between personality, job performance, and performance appraisal. The article is largely descriptive and discursive, but the key assumptions are primarily data-based. We begin by defining personality in terms of motivation, identity, and reputation - as opposed to traits. We then describe the role of social skills in allowing people successfully to pursue their major occupational goals. Finally, we suggest that performance appraisals reflect supervisors' judgments of rewardingness - the degree to which an incumbent meets or fulfills his or her boss's expectations regarding his or her performance - and these expectations may not necessarily concern the degree to which important organizational goals are met.
Article
Full-text available
The purpose of this investigation was to examine the psychometric properties (interrater reliabilities within source and correlations between sources) of subordinate, supervisor, peer, and self-ratings of job performance. Different job types and dimension types were compared. Using meta-analytic methodology, we found that subordinates showed the lowest mean reliability (.30) and supervisors showed the highest (.50), with peers in between (.37). Mean correlations between sources were low for subordinate ratings (.22 with supervisor, .22 with peer, and .14 with self-ratings) and for self-ratings (.22 with supervisor and .19 with peer ratings). The mean supervisor-peer correlation was higher at .34. Both reliabilities and correlations between sources tended to be higher for nonmanagerial and lower complexity jobs. Comparisons of between-source correlations with within-source reliabilities indicated that, with some qualifications, the different sources had somewhat different perspectives on performance. Dimension reliabilities differed somewhat for interpersonal and cognitive dimensions.
Article
Full-text available
Accumulated empirical evidence, some telling criticisms, and even the most cursory glance at the business press compel us to rethink the defining character of organizational citizenship behavior (OCB). It no longer seems fruitful to regard OCB as extra-role, beyond the job, or unrewarded by the formal system. A more tenable position is one that defines OCB much along the lines of what Borman and Motowidlo (1993) called contextual performance. Some preliminary suggestions are offered for the repositioning and articulation of the OCB construct as redefined; due attention is given to the problems that nonetheless remain, and will occupy us for some time to come as we reckon with root changes in the very character of organizations.
Book
Full-text available
Meta-analysis is arguably the most important methodological innovation in the social and behavioral sciences in the last 25 years. Developed to offer researchers an informative account of which methods are most useful in integrating research findings across studies, this book will enable the reader to apply, as well as understand, meta-analytic methods. Rather than taking an encyclopedic approach, the authors have focused on carefully developing those techniques that are most applicable to social science research, and have given a general conceptual description of more complex and rarely-used techniques. Fully revised and updated, Methods of Meta-Analysis, Second Edition is the most comprehensive text on meta-analysis available today. New to the Second Edition: * An evaluation of fixed versus random effects models for meta-analysis* New methods for correcting for indirect range restriction in meta-analysis* New developments in corrections for measurement error* A discussion of a new Windows-based program package for applying the meta-analysis methods presented in the book* A presentation of the theories of data underlying different approaches to meta-analysis
Article
Full-text available
This study attempts to refine the construct of contextual performance by dividing it into 2 narrower constructs, interpersonal facilitation and job dedication. Supervisors rated 975 U.S. Air Force mechanics on at least 1 of 4 aspects of job performance (different supervisors rated each aspect of performance), and 515 of these mechanics also completed self-report individual difference measures. Correlations between performance ratings and individual difference variables support distinguishing task performance from interpersonal facilitation but not from job dedication. Thus this study suggests the need to redefine task performance to include motivational elements of job dedication. Then task performance would include task proficiency and motivation to perform one's own tasks effectively, and contextual performance would include interpersonal skills, the motivation to maintain good working relationships and help others perform their tasks.
Article
Full-text available
Examined the validity of observer (supervisor, coworker, and customer) ratings and self-ratings of personality measures. Results based on a sample of 105 sales representatives supported the 2 hypotheses tested. First, supervisor, coworker, and customer ratings of the 2 job-relevant personality dimensions—conscientiousness and extraversion—were valid predictors of performance ratings, and the magnitude of the validities was at least as large as for self-ratings. Second, supervisor, coworker, and customer ratings accounted for significant variance in the criterion measure beyond self-ratings alone for the relevant dimensions. Overall, the results suggest that validities of personality measures based on self-assessments alone may underestimate the true validity of personality constructs. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
The Golem effect is the negative impact on subordinates' performance that results from low leader expectations toward them. The effects of low expectations were studied experimentally by preventing the natural formation of low expectations toward individuals who had obtained low test scores in some units and by refraining from doing so in other units. Randomly selected squad leaders ( n = 17) were led to believe that low scores on a physical fitness test were not indicative of subordinates' ineptitude. Control squad leaders ( n = 17) were not told how to interpret the test scores. Two posttests showed that low-scoring personnel in the experimental squads improved more than those in the control squads, maintained the difference, rated their squad leaders more favorably, and were more satisfied, but they were not significantly more anxious. The potential for low scores on valid tests to produce unintended Golem effects is discussed. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
The study examines the effects of a wide array of rater–ratee relationship and ratee-characteristic variables on supervisor and peer job-performance ratings. Interpersonal ratings, job performance ratings, and ratee scores on ability, job knowledge, and technical proficiency were available for 493 to 631 first-tour US Army soldiers. Results of supervisor and peer ratings-path models showed ratee ability, knowledge, and proficiency accounted for 13% of the variance in supervisor performance ratings and 7% for the peer ratings. Among the interpersonal variables, ratee dependability had the strongest effect for both models. Ratee friendliness and likability had little effect on the performance ratings. Inclusion of the interpersonal factors increased the variance accounted for in the ratings to 28% and 19%, respectively. Discussion focuses on the relative contribution of ratee technical and contextual performance to raters' judgments. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
This article summarizes the practical and theoretical implications of 85 years of research in personnel selection. On the basis of meta-analytic findings, this article presents the validity of 19 selection procedures for predicting job performance and training performance and the validity of paired combinations of general mental ability (GMA) and the 18 other selection procedures. Overall, the 3 combinations with the highest multivariate validity and utility for job performance were GMA plus a work sample test (mean validity of .63), GMA plus an integrity test (mean validity of .65), and GMA plus a structured interview (mean validity of .63). A further advantage of the latter 2 combinations is that they can be used for both entry level selection and selection of experienced employees. The practical utility implications of these summary findings are substantial. The implications of these research findings for the development of theories of job performance are discussed. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
The study examines the effects of a wide array of rater-ratee relationship and ratee-characteristic variables on supervisor and peer job-performance ratings. Interpersonal ratings, job performance ratings, and ratee scores on ability, job knowledge, and technical proficiency were available for 493-631 first-tour U.S. Army soldiers. Results of supervisor and peer ratings-path models showed ratee ability, knowledge, and proficiency accounted for 13% of the variance in supervisor performance ratings and 7% for the peer ratings. Among the interpersonal variables, ratee dependability had the strongest effect for both models. Ratee friendliness and likability had little effect on the performance ratings. Inclusion of the interpersonal factors increased the variance accounted for in the ratings to 28% and 19%, respectively. Discussion focuses on the relative contribution of ratee technical and contextual performance to raters' judgments.
Article
"Construct validation was introduced in order to specify types of research required in developing tests for which the conventional views on validation are inappropriate. Personality tests, and some tests of ability, are interpreted in terms of attributes for which there is no adequate criterion. This paper indicates what sorts of evidence can substantiate such an interpretation, and how such evidence is to be interpreted." 60 references. (PsycINFO Database Record (c) 2006 APA, all rights reserved).
Article
Skarlicki and Folger (1997) found that distributive, procedural, and interactional justice interacted to predict workplace retaliation. In this follow-up and extension of that study, we investigated whether a person-by-situation interaction explained variance in workplace retaliation beyond what could be attributed to fairness perceptions alone. Negative affectivity and agreeableness were found to moderate the relationship between fairness perceptions and retaliation.
Article
5 tests were given to 48 supervisors, in 2 metal fabricating plants, who were divided into 3 groups. The tests were the Personal Audit, How Supervise?, the Bennett, Form AA, the Kuder, and the Otis Mental Ability Test. Ratings by fellow supervisors were used as criteria of supervisory ability. The results indicate that, at least in some instances, supervisory ability can be measured fairly accurately by psychological tests by selecting the best predictors from a relatively large number of likely indices (29 in this study). It is also necessary to have reliable criteria such as pooled ratings by a large number of raters who know the supervisors being rated exceedingly well. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
This study investigates the construct validity of three methods used to evaluate clinical competence in medicine: standardized test, supervisor performance ratings, and peer performance ratings. Three attributes of clinical competence are investigated: cognitive abilities, interpersonal skills, and professional qualities. Measures representing each attribute-method combination include: National Board of Medical Examiners (NBME) examination (standardized test of cognitive abilities); two scales derived from the California Psychological Inventory (CPI) (standardized test of interpersonal skills and professional qualities); and the three scales derived from the Resident Evaluation Form (REF) (peer and supervisor ratings of all three attributes). Scores for each attribute-method combination were obtained from a convenience sample of 166 resident physicians in three primary care specialties. These scores were cast into a multitrait-multimethod matrix design and analyzed using confirmatory factor analysis. Results suggest a lack of construct validity for the CPI and REF scales, moderate convergent validity for the NBME, and substantial method variance in the REF-derived ratings. Findings are discussed in terms of the implications for a theory of medical clinical competence, further research and development in clinical competence measurement, and current measurement practice in medical education.
Article
Personality researchers are converging on the “Big Five?’ personality constructs as a taxonomy for describing the basic dimensions of personality. If, however, prediction rather than description is important, the Big Five may not be an adequate taxonomy. This article presents data that suggest the Big Five is not an adequate taxonomy of personality variables for predicting important criteria. The Big Five constructs are too heterogeneous and incomplete. A nine-factor taxonomy is presented along with correlational evidence demonstrating differential prediction of criteria for each of the nine personality constructs.
Article
This study investigated the relation of the "Big Five" personality di- mensions (Extraversion, Emotional Stability, Agreeableness, Consci- entiousness, and Openness to Experience) to three job performance criteria (job proficiency, training proficiency, and personnel data) for five occupational groups (professionals, police, managers, sales, and skilled/semi-skilled). Results indicated that one dimension of person- ality. Conscientiousness, showed consistent relations with all job per- formance criteria for all occupational groups. For the remaining per- sonality dimensions, the estimated true score correlations varied by occupational group and criterion type. Extraversion was a valid pre- dictor for two occupations involving social interaction, managers and sales (across criterion types). Also, both Openness to Experience and Extraversion were valid predictors of the training proficiency criterion (across occupations). Other personality dimensions were also found to be valid predictors for some occupations and some criterion types, but the magnitude of the estimated true score correlations was small (p < .10). Overall, the results illustrate the benefits of using the 5- factor model of personality to accumulate and communicate empirical findings. The findings have numerous implications for research and practice in personnel psychology, especially in the subfields of person- nel selection, training and development, and performance appraisal.
Article
The relationship between personality attributes and transformational leadership and the effects of transformational leadership on subordinate satisfaction and performance were examined. High scores on transformational leadership were associated with a distinct personality pattern characterized by higher levels of pragmatism, nurturance, and feminine attributes and lower levels of criticalness and aggression. This enabling pattern formed the core of transformational leadership. Although leader scores on transformational leadership correlated significantly with subordinate satisfaction, no performance effects of transformational leadership were found for six different objective measures of performance. Implications for models of transformational leadership are discussed.
Article
This study examined the validity of the Sixteen Personality Factors Questionnaire (16PFQ) as a predictor of the success of female military officers. Subjects were 84 volunteers for officers' training, 18-19 years old. Of the 84 who participated in a three-day assessment-center-type selection program, 34 were admitted to officers' school. Three independent criteria were measured: (a) officers' ratings (n ≅ 80), (b) peer nominations (n = 84), and final course grades (n = 34). The results showed that Factors E, H. L, M, N, Q,, Q2, and Q 3 of the 16PFQ were correlated significantly with at least one of the criteria. Although the findings generally supported the interpretation of many of the 16PF scales, as rendered by Cattell, Eber, and Tatsuoka (1970), the outcomes suggested an alternative inter pretation of Factor M.
Article
Relationships between military academy leaders' personal attributes (e.g., traits, thinking styles) and others' ratings of transformational and transactional leadership were assessed in this study. Regression analyses indicated that personal attributes significantly predicted leadership ratings. Consistent with implicit leadership theories, however, the relevant predictors differed as a function of the positions of the individuals who rated leadership (i.e., superiors vs. subordinates of focal leaders). For example, leader intelligence and athletic experience were related to subordinate ratings of leadership while the leader's degree of conformity was associated with superiors' ratings of leadership. Implications for organizations and for understanding the leadership rating process are discussed.
Article
A meta-analysis was conducted with 23 studies to assess the relationship between supervisory ratings and results-oriented measures of performance. As hypothesized, the corrected mean correlation was higher when a relative (versus absolute) rating format was used and when a composite (versus overall) rating method was used. These differences did not, however, account for all of the remaining variance around the relationship between ratings and results. Suggestions are offered for the direction of future research and practice.
Article
The goals of this study were (a) to identify managerial performance development constructs through factor analysis, (b) to understand their motivational determinants using personality correlates, and (c) to examine differences between rating sources. Factor analyses identified 5 developmental constructs: Interpersonal Effectiveness, Willingness to Handle Difficult Situations, Teamwork and Personal Adjustment, Adaptability, and Leadership and Development. Comparisons with Borman and Brush's (1993) managerial performance megadimensions showed that the developmental constructs overlapped with but also added to the day-to-day performance domain. Each of the five factors showed a distinct pattern of personality correlates. Personality correlates supported hypotheses based on socioanalytic theory regarding the motive to get along with others (e.g., Interpersonal Effectiveness correlated with empathy and agreeableness) versus the motive to get ahead (e.g., Willingness to Handle Difficult Situations correlated with potency measures). Rating sources (supervisor, peer, subordinate, and self) showed some differences in their results.
Article
Method and trait effects in multitrait-multirater (MTMR) data were examined in a sample of 2,350 managers who participated in a developmental feedback program. Managers rated their own performance and were also rated by two subordinates, two peers, and two bosses. The primary purpose of the study was to determine whether method effects are associated with the level of the rater (boss, peer, subordinate, self) or with each individual rater, or both. Previous research which has tacitly assumed that method effects are associated with the level of the rater has included only one rater from each level; consequently, method effects due to the rater's level may have been confounded with those due to the individual rater. Based on confirmatory factor analysis, the present results revealed that of the five models tested, the best fit was the 10-factor model which hypothesized 7 method factors (one for each individual rater) and 3 trait factors. These results suggest that method variance in MTMR data is more strongly associated with individual raters than with the rater's level. Implications for research and practice pertaining to multirater feedback programs are discussed.
Article
Hierarchical regressions using the responses of 138 subordinates about 55 managers show that, as predicted, Bass' (1985b) transformational leader model adds to initiation and consideration in explaining the variance of subordinates' satisfaction and ratings of leader effectiveness. Further analysis, using leader ratings by one subordinate and outcome ratings by a second subordinate, largely confirms this augmentation effect. There was disconfirmation for the effect on reported extra effort, suggesting that the relation of transformational leadership to subordinate's extra effort is a dyadic rather than a group effect.
Article
D. T. Hall's (Careers in organizations. Pacific Palisades, Calif.: Goodyear Pub., 1976) psychological success-based model of career development was investigated using path analyses across three faculty member roles and several performance indices. The model received moderate support for each role. While alternative paths from performance to job involvement and satisfaction with promotional opportunities were observed, they did not contradict the portions of the psychological success model studied.
Article
In order to determine the psychological influences contributing to scientific eminence, data from interviews, self-reports, peer ratings, observer ratings, and objective productivity measures were obtained on 99 full professors of physics, chemistry, and biology at major research universities in California. A structural model in which psychological factors were predicted to have a direct influence on both productivity and eminence and an indirect influence on eminence yielded a very good fit to the data. These findings support previous research on scientific creativity and eminence, and extend it by providing the first integrated structural model of scientific eminence.
Article
The research described was performed under Project A, the U.S. Army's large-scale, multiyear manpower and personnel effort for improving the selection, classification, and utilization of Army enlisted personnel. This report is one of three derived from an extensive literature review aimed at identifying many types of constructs that might be used to enhance the accuracy of the present Army screening system (the other two reports deal, respectively, with cognitive abilities and psychomotor abilities). The present report is divided into three sections, each dealing with the utility of one type of information for predicting job performance. The section on temperament discusses traits as the basis for temperament assessment, several methods of scale construction, comparison of psychometric properties, a proposed taxonomy of temperament scales, criterion-related validity, and various moderator variables. The section on biographical data discusses measurement methods and concerns, structure and conceptual issues, and validity research. The section on interest assessment discusses various methods of measuring interests, models and theories, and validity research.
Article
This volume describes 24 publicly available multiple-perspective management-assessment instruments that relate self-view to the views of others on multiple management and leadership domains. Each instrument also includes an assessment-for-development focus that scales managers along a continuum of psychometric properties, and "best practices" for management development. The instruments reviewed are: (1) "Benchmarks"; (2) "Campbell Leadership Index" (CLI); (3) "COMPASS: The Managerial Practices Survey"; (4) "Executive Success Profile" (ESP); (5) "Survey of Executive Leadership" (EXEC); (6) "Leader Behavior Analysis II" (LBAII); (7) "The Visionary Leader: Leader Behavior Questionnaire" (LBQ); (8) "Leadership Effectiveness Analysis" (LEA); (9) "Acumen Leadership Skills" LEADERSHIP SKILLS; (10) "Leadership/Impact" (L/I); (11) "Leadership Practices Inventory" (LPI); (12) "Life Styles Inventory" (LSI); (13) "MANAGER VIEW/360"; (14) "Matrix: The Influence Behavior Questionnaire" (MATRIX); (15) "Management Effectiveness Profile System" (MEPS); (16) "Multifactor Leadership Questionnaire" (MLQ); (17) "The PROFILER"; (18) "PROSPECTOR"; (19) "Survey of Leadership Practices" (SLP); (20) "The Survey of Management Practices" (SMP); (21) "System for the Multiple Level Observation of Groups" (SYMLOG); (22) "Types of Work Index" (TWI); (23) "VOICES"; and (24) "Acumen Leadership Work Styles" (WORKSTYLES). Three aspects are described for each instrument: (1) descriptive: author; vendor, copyright date, purpose, target audience, cost, scoring and certification procedures, duration, format, and raters; (2) research: origins, scales, scale definitions, samples, cautionary statement, and instrument reports; and (3) training: sample instrument, sample feedback report, and training materials. (RIB)