Article

Validity and Dimensions of Descriptive Adjectives Used in Reference Letters for Engineering Applicants

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Because the tone and content of most reference letters are at least superficially positive (Muchinsky, 1979;Schneider & Schmitt, 1986), the reader may use one or several structural characteristics of the message to infer the writer's "true" evaluation of an applicant. For example, Peres and Garcia (1962) examined 625 letters of recommendation sent to a nuclear research laboratory. They found that candidates were frequently described by trait names or adjectives. ...
... Adjectives in the mental agility category were best at discriminating between job performance levels. Whitcomb and Bryan (1988;see also Aamodt, Bryan, & Whitcomb, 1993;Bryan, 1989;Whitcomb, 1989) examined the criterion-re- lated validity of the Peres and Garcia (1962) categories for predicting graduate school grades. Whitcomb and Bryan found that the number of adjectives placed in the mental agility category was associated with graduate school GPA (r = .32, ...
... Although theoretically appealing, Semin and Fiedler's (1988) lin- guistics-based measures (i.e., action verbs, state verbs, and adjectives) were not significantly associated with readers' reactions. While, Aamodt et al. (1993) found some support for the criterion-related validity of a modified Peres and Garcia (1962) adjective scoring scheme, their study focused on prediction of graduate grades and teaching evaluations and not on reader reactions. ...
Article
References continue to be a required part of selection systems in employment settings and higher education. Previous research on references has often involved atheoretical validation studies. We propose that the reference message itself is but one step in a communication process and that letter tone, content, and structure can be analyzed to determine their contribution to process outcomes. A sample of 85 reference statements was used to demonstrate potential antecedents, measures, and outcomes suggested by this conceptualization. Antecedents including letter characteristics were significantly related to readers' evaluations and accept or reject recommendations. Results from regression analyses indicated initial support for measures of letter tone, content, and structure in predicting readers' evaluations. A communication-based framework is presented and implications for research and practice are discussed.
... Other-Rated Noncognitive Assessment ToolsFive-Trait Category Tool16 Aamodt et al16 evaluated the Five-Trait Category Tool(FTCT) in an attempt to standardize evaluation of noncognitive constructs in the form of NLORs. The FTCT was originally created in 1962 by Peres and Garcia22 but had not been psychometrically tested. The FTCT is intended to evaluate character traits (dependability/reliability, consideration/ cooperation, mental agility, urbanity, vigor) in NLORs that would be indicative of academic performance.22 ...
... The FTCT was originally created in 1962 by Peres and Garcia22 but had not been psychometrically tested. The FTCT is intended to evaluate character traits (dependability/reliability, consideration/ cooperation, mental agility, urbanity, vigor) in NLORs that would be indicative of academic performance.22 Trait categories were the only significant predictors of teaching performance. ...
Article
In the graduate admission process, both cognitive and noncognitive instruments evaluate a candidate's potential success in a program of study. Traditional cognitive measures include the Graduate Record Examination or graduate grade point average, while noncognitive constructs such as personality, attitude, and motivation are generally measured through letters of recommendation, interviews, or personality inventories. Little consensus exists as to what criteria constitute valid and effective measurements of graduate student potential. This integrative review of available tools to measure noncognitive constructs will assist graduate faculty in identifying valid and reliable instruments that will enhance a more holistic assessment of nursing graduate candidates. Finally, as evidence-based practice begins to penetrate academic processes and as graduate faculty realize the predictive significance of noncognitive attributes, faculty can use the information in this integrative review to guide future research.
... For example, research on online health information provides consistent evidence that specific health information is viewed more credible (Sargeant, Mann, & Ferrier, 2005;Webb & Eves, 2007). Furthermore, organizational scholars argued that letters of recommendation should describe job candidates' traits and their work performance in detail because such information can reduce readers' uncertainty and demonstrate great value (French, 1982;Peters & Garcia, 1962;Range et al., 1991). Specifically, Knouse (1982Knouse ( , 1983) manipulated the level of specificity of examples used in letters of recommendation and replicated his study in two different groups of letter readers-college students and personnel specialists. ...
Article
Other-generated information about specific targets available online reduces third parties’ uncertainty of strangers. Yet only information perceived as credible can shapes our their impressions. This study examines how impressions form after exposure to online recommendations, a type of other-generated information via LinkedIn, through credibility assessment of these recommendations. The source-target relationship and recommendation specificity were manipulated in a 2 X 2 between-groups subjects experiment (N = 213). Main effects for both independent variables were found. Perceived manipulation likelihood and perceived credibility of recommendation mediated the impact of the source-target relationship on impressions. Additionally, perceived familiarity with the target and perceived credibility of recommendation mediated the impact of recommendation specificity on impressions. This study extends warranting theory and highlights the impact of new communication technology on credibility assessment of online information and impression formation.
... Our findings indicate that the technique developed by Peres and Garcia (1962) shows promise as a predictor of performance. With both samples, the significant validity coefficients were more than twice the magnitude of the .13 ...
Article
Full-text available
Although letters of recommendation or some form of reference checking are used by over 80% of organizations in the United States (Muchinsky, 1979), research investigating the validity of such techniques has not yielded promising results. In a study of references used in industry, Mosel and Goheen (1959) found that the validity of references was only .13. These results were supported by Browning (1968) who found the validity of references also to be .14 in predicting teaching success. Research has identified several potential reasons for this low validity. As with the employment interview, factors other than the relevant content of the letters are used to form impressions of the applicant. For example, Cowan and Kasen (1984) found that letters referring to applicants by their first name were perceived as being more positive than letters referring to applicants by a title such as, "Mr. Jones" and Knouse (1983) found that letters of recommendation containing specific examples were evaluated more positively than letters without examples. In an attempt to focus the attention of letter readers on the important content of the letter, Peres and Garcia (1962) developed a technique in which the traits contained in a letter of recommendation are highlighted and placed into one of five categories which were developed based on a content analysis of 625 letters of recommendation written for engineering applicants. These five categories and representative traits for each category are: Mental Agility: Adaptable, analytical, bright, intelligent, logical, resourceful Cooperation-Consideration: Altruistic, congenial, friendly, helpful, sincere Dependability-Reliability: Alert, critical, dependable, methodical, prompt Urbanity: Assured, chatty, cultured, forward, gregarious, sociable, talkative Vigor: Active, eager, energetic, enthusiastic, independent, industrious Unfortunately, Peres and Garcia (1962) did not attempt to validate this technique. Thus, it is the purpose of this study to investigate the reliability and validity of the technique using two separate samples.
... Letters that have ratings or are converted into numerical ratings may fare better. In the studies examined here, letters either contained numerical ratings to begin with or were coded for favorability using either structured methods that code specific adjectives or other content (e.g., the Peres and Garcia (1962) method) or unstructured impressionistic ratings of content. If possible, we contrast the predictive power of letter rating made by the writer versus those made by the reader. ...
Article
Full-text available
Letters of recommendation are used extensively in academic admissions and personnel selection. Despite their prominence, comparatively little is known about their predictive power for multiple outcomes. This meta-analysis combine the existing literature for college grade point average (GPA), academic outcomes of GPA, performance ratings, degree attainment, and research productivity for nonmedical school graduate programs, and GPA and internship performance ratings for medical school students. Intercorrelations with other commonly used predictors are also estimated and used to estimate incremental predictive power. Overall, letters of recommendation, in their current form, are generally positively but weakly correlated with multiple aspects of performance in post-secondary education. However, letters do appear to provide incremental information about degree attainment, a difficult and heavily motivationally determined outcome.
... For instance, quality of LORs only modestly correlates with important graduate school outcomes (e.g., GPA; faculty ratings; Kuncel et al., 2014). Moreover, LORs suffer from construct and method confusion (Arthur & Villado, 2008); unstructured LORs purportedly "assess" a variety of constructs such as students' motivation, persistence (Kuncel et al., 2014), personality (e.g., Aamodt et al., 1993;Peres & Garcia, 1962), self-efficacy, and/or creativity (Kyllonen et al., 2005). Table 1 presents a selected list of constructs posited to relate to academic success, measurable via LORs, and highlights the variety of things LORs purport to measure. ...
... To circumvent these problems, more organizations use largescale data analytic techniques to comb through open-ended text fields in electronic applications. Most HR professionals are familiar with automated keyword searches of applications, a method that far predates the use of electronic systems (e.g., Peres & Garcia, 1962). The development of these lists, however, is seldom linked to a conceptual or theoretical understanding of qualifications. ...
Article
Work history information reflected in resumes and job application forms is commonly used to screen job applicants; however, there is little consensus as to how to systematically translate information about one’s work-related past into predictors of future work outcomes. In this article, we apply machine learning techniques to job application form data (including previous job descriptions and stated reasons for changing jobs) to develop interpretable measures of work experience relevance, tenure history, and history of involuntary turnover, history of avoiding bad jobs, and history of approaching better jobs. We empirically examine our model on a longitudinal sample of 16,071 applicants for public school teaching positions, and predict subsequent work outcomes including student evaluations, expert observations of performance, value-added to student test scores, voluntary turnover, and involuntary turnover. We found that work experience relevance and a history of approaching better jobs were linked to positive work outcomes, whereas a history of avoiding bad jobs was associated with negative outcomes. We also quantify the extent to which our model can improve the quality of selection process above the conventional methods of assessing work history, while lowering the risk of adverse impact. (PsycINFO Database Record (c) 2019 APA, all rights reserved)
... Recientes meta-análisis acerca de la validez de las dimensiones de personalidad han mostrado que el factor de Conciencia es un buen predictor del desempeño (Barrick & Mount, 1991;Poropat, 2009;Salgado, 1997). Para la descripción de la personalidad de los individuos se han desarrollado inventarios y cuestionarios de autoinforme basados en los diferentes modelos de personalidad (Cattell, 1943;Eysenck, 1963;Norman, 1963); además, otro método de descripción corresponde al modelo utilizado por Peres and Garcia (1962) con el fin de analizar los adjetivos y descriptores del comportamiento presenten en diversas declaraciones que unas personas habían hecho sobre otras en diversos procedimientos de selección de personal, promoción, acceso e organizaciones, etc. El procedimiento utilizado por Peres y García (1962) ha sido replicado posteriormente en otros estudios por Aamodt, Bryan y Whitcomb (1989, 1993 y Aamodt (2006), para identificar los adjetivos descriptores de comportamientos y evaluar, así, el contenido de las cartas de recomendación en diversos procesos de selección de personal. ...
Conference Paper
Full-text available
El objetivo del estudio es describir con imágenes mentales la personalidad de los integrantes de equipos de trabajo y conocer la influencia de variables demográficas. Recientes meta-análisis mostraron que la Conciencia, perteneciente al modelo de los cinco factores de personalidad, es un buen predictor del desempeño académico. Existe escasa literatura acerca de la imagen mental como proceso mental para describir el perfil de personalidad. 226 estudiantes de Grado participaron en el estudio y los resultados indicaron que los adjetivos relacionados con Conciencia y Amigabilidad eran los más frecuentes. No existen diferencias significativas en las variables demográficas, excepto la variable del género.
Conference Paper
Full-text available
El objetivo del estudio era determinar el perfil del puesto de Policía Local a través de la declaración de descriptivos de competencias laborales y de las imágenes mentales de los policías locales. 24 policías locales aportaron descriptores referidos a la imagen del mejor Policía Local, y evaluaron la relevancia de las pruebas de selección. Los resultados indicaron que los descriptivos más frecuentes se referían a las competencias de Colaboración, Flexibilidad, Excelencia, Conocimientos técnicos y Trabajo en Equipo, y que las pruebas de evaluación más relevantes eran los conocimientos, los psicotécnicos y las aptitudes físicas.
Conference Paper
Full-text available
We will discuss the use of references and recommendation letters. Presentations will include data on the reliability, validity, and adverse impact of recommendations; advice on what should and should not be included in references; and discussions on the ethical and legal issues that can arise when serving as a reference.
Article
Despite the widespread use of letters of recommendation (LORs) in selection systems, research has largely failed to consider the potential emergence of bias in interpretations of LORs. The present study fills this void by examining both race and gender bias in evaluations of LORs and assessing the efficacy of elaboration as a strategy for reducing such bias. Undergraduate students (n = 423) rated four LORs that varied with regard to applicant race and gender. Results suggest that bias does exist in evaluations of LORs, but that requiring raters to expand on their evaluations (i.e., elaborate) reduces this bias. Implications include elaboration as a strategy organizations can implement to reduce bias from emerging when relying on LORs as a selection tool.
Article
Full-text available
Reference check is a term that reports information about personal characteristics related to job performance of one person and that another person (usually a supervisor) provide about the first. References are based on the assumption that "the best way to find out about someone is to ask someone who knows" and they are widely used and potentially effective for personnel selection. The main objective of this paper is to answer the following question: what do the references actually measure? There have been collected the primary studies in the literature on this subject and two meta-analyses were conducted. The results indicate that reference check measure General Mental Ability and Experience. Suggestions for research to find other constructs that could be integrated in the references are provided.
Article
Letters of reference are commonly used in acquiring a job in education. Despite serious issues of validity and reliability in writing and evaluating letters, there is a dearth of research that systematically examines the evaluation process and defines the constructs that define high quality letters. The current study used NVivo to examine 160 letters of reference to determine component constructs and relate these components to overall letter quality. Results suggest that testimonials are most predictive of overall letter quality. Implications and limitations of this study are also discussed.
Article
Billions of Dollars, Lires, Yens, etc. have been spent worldwide in researching the selection process in an effort to find a solution to identifying successful sales people. Though the process has been refined and improved, selecting those who would be successful is one of the greatest costs and challenges to sales management today. This article presents a review of several methods currently being utilized.
Conference Paper
Full-text available
El objeto del estudio era conocer la influencia de variables demográficas sobre el carácter emprendedor y conocer su relación con el estrés académico. Existe abundante literatura acerca del perfil del emprendedor y su disposición a involucrarse en proyectos complejos; sin embargo, escasean estudios que relacionen el estrés académico con el carácter emprendedor. 336 estudiantes universitarios de diversos Grados participaron en este estudio. Los resultados indicaron que existen diferencias en el carácter emprendedor en función de la edad y del curso; además, los resultados indicaron que carácter emprendedor y estrés académico son dos variables inversas aunque con correlaciones bajas o moderadas.
Article
Full-text available
This article develops a theoretical framework for understanding the appeal and tone of letters of recommendation using an evolutionary psychological perspective. Several hypotheses derived from this framework are developed and tested. The authors’ theoretical argument makes two major points. First, over the course of human evolution, people developed a preference for narrative information about people, and the format of letters of recommendation is compatible with that preference. Second, because recommenders are acquaintances of applicants, the tone of letters should reflect the degree to which the relationship with the applicant favors the recommender’s interests. We hypothesized that, over and above an applicant’s objective qualifications, letters of recommendation will reflect cooperative, status and mating interests of recommenders. We used 532 letters of recommendation written for 169 applicants for faculty positions to test our hypotheses. The results indicated that the strength of the cooperative relationship between recommenders and applicants influenced the favorability and length of letters. In addition, male recommenders wrote more favorable letters for female than male applicants, suggesting that male mating interests may influence letter favorability. We conclude with implications for practice and future research.
Article
Full-text available
Personnel selection involves exchanges of information between job market actors (applicants and organizations). These actors do not have an incentive to exchange accurate information about their ability and commitment to the employment relationship unless it is to their advantage. This state of affairs explains numerous phenomena in personnel selection (e.g., faking). Signaling theory describes a mechanism by which parties with partly conflicting interests (and thus an incentive for deception) can nevertheless exchange accurate information. We apply signaling theory to personnel selection, distinguishing between adaptive relationships between applicants and organizations, among applicants, and among organizations. In each case, repeated adaptations and counteradaptations between actors can lead to situations of equilibrium or escalation (arms races). We show that viewing personnel selection as a network of adaptive relationships among job market actors enables an understanding of both classic and underexplored micro- and macro-level selection phenomena and their dynamic interactions.
Article
Clear and accessible, Personnel Selection will continue to be the guide that students and professionals alike turn to for practical advice in this complex field. Fully updated edition of highly successful text. Clear, accessible and practical in its approach. Now includes emotional intelligence and counterproductive work behaviours â" not covered in any other text at this level.
Article
The origin of the use of efficiency reports for Officers of the U. S. Army is lost in history, as is the story of the evolution of the formal procedures of reporting. Sometime after the first World War, however, a standard form was adopted and a procedure regularized for accomplishing this report. Thereafter, twice each year—on June 30th and on December 31st—every officer in the Army has been rated by his immediate superior, and this rating submitted to the War Department. Though early recognized as not completely satisfactory, the original rating form remained in force (with sporadic minor amendments) until it was superseded in July of 1947.The new form is the product of many months of concentrated research. It is radically different in many respects from the old form, and from other rating devices currently in use in industry. Its most novel feature is the use of what has been called the “forced-choice” rating method. Rather than indicating how much or how little of each characteristic an officer possesses, the rater is required to choose, from several sets of four adjectives or phrases, which best characterizes the officer, and which is least descriptive. In other words, it calls for objective reporting and minimizes subjective judgment. And because of the way in which the tetrads—sets of four rating elements—are constructed, it reduces the rater's ability to produce any desired outcome by the choice of obviously good or obviously bad traits. It thus diminishes the effects of favoritism and personal bias.The technique, and the form embodying it, has been tried out on fifty thousand officers—in both experimental and official trials—and the results obtained with it have been compared with independent criteria of efficiency arrived at through group ratings. The new method is superior to all other methods examined. It produces a better distribution of ratings relatively free from the usual pile-up at the top of the scale. It is less subject to influence by the rank of the officer being rated. It is quickly and objectively scored by machine. And above all, it produces ratings which are more valid indices of real worth.The particular form developed for rating Army officers would probably be of little value for other groups—largely because of the specificity of the rating elements it contains. The technique, however, has already proved of value in other situations and there is every reason to believe that it is even more generally applicable.
Article
A method is presented for securing a hierarchical factor solution which achieves simple structure at each hierarchical level without rotation or even preliminary arbitrary orthogonal or oblique solutions. The method is based upon the assumption that if overlap is removed from clusters the remaining specifics will achieve simple structure automatically. The problem presented earlier by Schmid and Leiman, using oblique simple structural rotation as a basis, is reworked by this new approach.
Article
The computation of intercorrelation matrices involving large numbers of variables and the subsequent factoring of these matrices present a formidable task. A method for estimating factor loadings without computing the intercorrelation matrix is developed. The estimation procedure is derived from a theoretical model which is shown to be a special case of the multiple-group centroid method of factoring. Empirical checks have indicated that the model, even though it makes some stringent assumptions, can be applied to a variety of variables found in psychological factoring problems. It has been found to be particularly useful in factoring test items.
Relationship Between Behavior Trait Ratings by Peers and Later Officer Performance of USAF Officer Candidate School Graduates Research Report AFPTRC-TNdY-126. Lackland Air Force Base, Texas Stability of Personality Trait Rating Factors Obtained Under Diverse Conditions
  • E C Tupes
  • E C Tupes
  • R E And Christal
TUPES, E. C. " Relationship Between Behavior Trait Ratings by Peers and Later Officer Performance of USAF Officer Candidate School Graduates. " Research Report AFPTRC-TNdY-126. Lackland Air Force Base, Texas, 1957. TUPES, E. C. AND CHRISTAL, R. E. " Stability of Personality Trait Rating Factors Obtained Under Diverse Conditions. " Research Report WADC-TN-68-61, Lackland Air Force Base, Texas, 1958.
Behavior Trait Ratings by Peers and References
  • R J Wherry
  • N E Stander
  • J J And Hopilins
WHERRY, R. J., STANDER, N. E., AND HOPILINS, J. J. " Behavior Trait Ratings by Peers and References. " Research Report WADC-TR-69360, Lackland Air Force Base, Texas, 1959.