No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
"Hey, This is not like me!" Convergent validity of computerized personality reports
Abstract
Introduction/objective. - There is a dearth of research on the validity of computerized personality reports. The present study examined the validity of such reports using various methodologies. Method. - Convergence between personality self-ratings on items and self- and peer-ratings on personality descriptive text blocks used for compiling computerized reports were examined in a sample of 175 psychology undergraduates who were administered the Personality for Professionals Inventory (PfPI; Rolland 82 De Fruyt, 2009). Some weeks after these test administrations, a subsample of individuals was given feedback on their actual sex-normative scores, with half of the sample receiving random scores. Results. - Overall, PfPI-self-ratings on items showed strong rank-order convergence with self- and peer-ratings on 25 sets of three personality descriptive text blocks describing respectively low, medium or high positions on the 25 PfPI traits, underscoring the validity of the report text blocks. However, absolute ratings on text blocks were usually somewhat higher in the socially desirable direction. Assessees were able to discriminate genuine from fake reports, and perception of the text blocks as accurate was negatively associated with the discrepancy between genuine and fake reports. Conclusion. - PfPI-computerized reports are accurate reflections of self-descriptions on PfPI items. Implications for career counseling and development practice are discussed.
... Putting the two perspectives together, the observed differences could be due to methodological choices. Generally, participants discern more easily between real and randomly generated personality feedback compared to real personality feedback and Barnum-effect feedback (Andersen & Nordvik, 2002;Christian & Bringmann, 1982;De Fruyt & Wille, 2013). For example, participants were able to correctly identify descriptions based on randomly generated personality scores that deviated from their original scores (Andersen & Nordvik, 2002). ...
... For example, participants were able to correctly identify descriptions based on randomly generated personality scores that deviated from their original scores (Andersen & Nordvik, 2002). Also, these participants tended to disagree more with the false feedback reports when the difference between the real and random scores was larger (De Fruyt & Wille, 2013). However, participants still endorsed false reports if they were based on neutral scores (De Fruyt & Wille, 2013). ...
... Also, these participants tended to disagree more with the false feedback reports when the difference between the real and random scores was larger (De Fruyt & Wille, 2013). However, participants still endorsed false reports if they were based on neutral scores (De Fruyt & Wille, 2013). ...
Despite personality measurement and feedback being pervasive practices, there are self-judgment biases that may impair their usage. We set out to analyze the differences between two kinds of false feedback and real feedback on personality regarding perceived accuracy and preference. We propose that there would be no differences between false and real feedback regarding perceived accuracy, but we expect differences regarding feedback preference. A sample of 146 students completed the IPIP-50 instrument that measured the Big 5 Factors and received three kinds of feedback - a general one (Barnum effect as false feedback), a positive one (Better-than-average effect as false feedback), and a real one. They rated each regarding accuracy and preference. Results indicate differences regarding both dependent variables. Participants perceive false feedback as more accurate than the real one. Moreover, they prefer positive feedback over the other two, and general feedback compared to the real one. We discuss both theoretical and practical implications, alongside a series of limitations and future research directions.
... Psychometric properties, including reliability, construct, content, and predictive validity, are typically quoted as important in choosing a personality assessment (Cicchetti, 1994;Cook, 2004). Other considerations gaining attention include the candidate experience (Ekuma, 2012), acceptance of feedback (Atwater & Brett, 2006;Krings, Jacobshagen, Elfering, & Semmer, 2015), and quality of interpretive reports (De Fruyt & Wille, 2013). Nevertheless, much training in and critique of the use of tests and questionnaires concentrates on construct, content, and predictive validity in a selection context (e.g., British Psychological Society, 2017). ...
The acceptance and popularity of personality assessments in organizational contexts has grown enormously over the last 40 years. Although these are used across many applications, such as executive coaching, team building, and hiring and promotion decisions, the focus of most published research on the use of personality assessments at work is biased toward assessment for employee selection. Reviews have therefore tended to use criteria that are appropriate for selection, neglecting the additional and different criteria that are important in relation to employee development. An illustration of the often-discussed scientist–practitioner divide is that the Myers–Briggs Type Indicator is the most widely known and used personality assessment in organizations, despite harsh criticism by the academic community. This article reviews this debate, and draws implications for the appropriate choice of personality assessments for use in individual and team development, and a new direction for scientific research.
Many studies have been performed on the interpretation of a person’s personality along the Five-factor model that includes the following traits: openness to experience, emotional stability, conscientiousness, extraversion and agreeableness. However, very little research has been done specifically on the personality of internal auditors. This study tries to establish insight into the personality of internal auditors by comparing them with other professionals.
Based on a literature review and discussion, it is hypothesized whether or not the personality traits of internal auditors differ from those of other professionals. The hypotheses on each of the five factors have been tested for internal auditors and other professionals in The Netherlands.
Results show that, for four personality traits, the internal auditor’s personality is significantly different from other professionals; only the trait agreeableness shows no significant difference. Limitations of the study lie in its exploratory nature.
Relationships of physical resemblance to personality similarity and social affiliation have generated considerable discussion among behavioral science researchers. A “twin-like” experimental design (involving genetically unrelated look-alikes, U-LAs) explores associations among resemblance in appearance, the Big Five personality traits, self-esteem, and social attraction within an evolutionary framework. The Personality for Professionals Inventory (PfPI), NEO/NEO-FFI-3, Rosenberg Self-Esteem Scale, and a Social Relationship Survey were variously completed by 45 U-LA pairs, identified from the “I’m Not a Look-Alike” project, Mentorn Media, and personal referrals. The mean U-LA intraclass correlations were negligible for all Big Five personality traits on the PfPI and NEO/NEO-FFI-3 (ri = −.02 and − .04, respectively). In contrast, mean ri values of .53 and .15 for monozygotic (MZA) and dizygotic (DZA) reared-apart twins, respectively, have been reported for these personality measures. The U-LA self-esteem correlation (ri = −.18) was also below the correlations reported for MZ and DZ reared-together twins (ri = .31 and .13, respectively). Finally, far fewer U-LAs expressed close social relationships (20%) than MZA (80%) and DZA (65%) twins. The present study extends earlier findings indicating that appearance is not meaningfully related to personality similarity and social relatedness. The criticism that MZ twins are alike in personality because their matched looks invite similar treatment by others is refuted. A more judicious interpretation is reactive genotype-environment correlation, namely that MZ twins’ similar personalities evoke similar reactions from others. MZ twins’ close social relations most likely derive from their perceptions of genetically based within-pair similarities that are lacking in U-LAs.
Au Québec, la situation en termes d’apprentissage des savoirs grammaticaux, et surtout de mobilisation de ces derniers en situation d’écriture, est inquiétante. Pour rendre son intervention plus efficiente, l'enseignant peut choisir une approche pédagogique déductive (explication d'un concept suivi d'exercices) ou inductive (observation d'exemples avant la vérification d'hypothèse et les exercices). Notre objectif principal est d'évaluer les incidences d’une intervention éducative en grammaire selon des approches pédagogiques inductive ou déductive sur l’apprentissage par des élèves de 1re secondaire du complément du nom et son impact dans le développement de la compétence savoir écrire. Notre objet d’enseignement, le complément du nom (CN) est un concept de la grammaire actuelle, dont les caractéristiques facilitent son utilisation réfléchie en rédaction, surtout lors des phases de révision et de réécriture. Une méthodologie mixte de recherche a permis de recueillir des données auprès d'un échantillon de 269 élèves de première secondaire, de même qu’auprès de trois enseignants. Pour cette recherche exploratoire d’épistémologie pragmatique, nous avons réalisé des tests de connaissances, des analyses de rédactions et des entrevues auprès d’élèves et d'enseignants. Les résultats obtenus aux tests de connaissances nous montrent qu’il n’y a pas d’écart significatif entre l’amélioration des élèves ayant vécu l’approche inductive et ceux ayant vécu l’approche déductive, si ce n’est d’un léger avantage en ce qui concerne les accords pour les élèves ayant vécu l’approche déductive. Ce qui a par contre émergé de cette expérimentation est d’une part l’effet évident de l’apprentissage des CN sur leurs mobilisations en écriture, peu importe l’approche. D’autre part, les élèves ayant eu une forte amélioration ont confirmé que pour une approche ou une autre, leur implication cognitive demeure l’élément fondamental de l’efficacité de l’intervention éducative. Nous pouvons donc conclure qu’une approche n’est pas significativement supérieure à l’autre dans le cadre de l’enseignement des CN, mais que l’enseignant doit adapter son approche à la situation en tenant compte du contexte, des apprenants et des aspects de l'objet concernés.
Personality traits are a useful component of clinical assessment, and have been associated with positive and negative life outcomes. Assessment of both general and maladaptive personality traits may be beneficial practice, as they may complement each other to comprehensively and accurately describe one's personality. Notably, personal preferences regarding assessment feedback have not been studied. The current study examined the acceptability of personality assessment feedback from the perspective of the examinee. Treatment-seeking participants from a university (n = 72) and Amazon.com MTurk (n = 101) completed measures of the 5-factor model and the DSM-5 alternative model of personality disorder, and were then provided feedback on their general and maladaptive personality traits. Individuals then provided feedback on which aspects they found most useful. Results demonstrated strong participant agreement that the personality trait feedback was accurate and relevant. (PsycINFO Database Record
We attempt to measure the efficiency of inductive or deductive approaches to short and long-term learning of knowledge about the "complément du nom" by French as a first language secondary 1 students. Our results show that, in the context of our experiment, the deductive approach allows greater appropriation of morphological aspects. Moreover, it seems that the pedagogical approach has less impact on the learning than the teacher effect, regardless of the approach used. We will discuss our results in putting forward the importance of some conditions of the educative intervention.
The "rule-of-thumb method" for generating personality descriptions from tests involves the clinician's personal experience, skill, and creative artistry; the "cookbook method" refers to "the transition from psychometric pattern to personality description is an automatic, mechanical, 'clerical' kind of task, proceeding by the use of explicit rules set forth in the cookbook." Data are cited indicating the superiority of the "cookbook method." Characteristics and expectations regarding this method are discussed. 15 references. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
College students (23 men and 31 women) who believed that they possessed 1 of 4 personality traits considered whether a trait description of them based on an objective personality test was accurate. Traits had positive (i.e., trusting and independent) or negative (i.e., suspicious and dependent) implications. Ss were likely to agree that they were accurately described by a personality trait if the trait was consistent with an existing self-perception (i.e., schema). This confirmatory tendency was observed whether the trait presented was positive or negative and was observed whether persons believed they possessed these same positive or negative traits. Findings extend and clarify earlier research on hypothesized mood states by D. C. Strohmer et al (1988). It is argued that confirmatory bias, such as that observed in this study, affects the process of counseling. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Computers have been important to applied psychology since their introduction, and the application of computerized methods has expanded in recent decades. The application of computerized methods has broadened in both scope and depth. This article explores the most recent uses of computer-based assessment methods and examines their validity. The comparability between computer-administered tests and their pencil-and-paper counterparts is discussed. Basic decision making in psychiatric screening, personality assessment, neuropsychology, and personnel psychology is also investigated. Studies on the accuracy of computerized narrative reports in personality assessment and psychiatric screening are then summarized. Research thus far appears to indicate that computer-generated reports should be viewed as valuable adjuncts to, rather than substitutes for, clinical judgment. Additional studies are needed to support broadened computer-based test usage.
The suitability of the Revised NEO Personality Inventory (NEO PI-R) to assess adolescents' personality traits was investigated in an unselected heterogeneous sample of 469 adolescents aged 12 to 17 years. They were further administered the Hierarchical Personality Inventory for Children (HiPIC) to allow an examination of convergent and discriminant validity. The adult NEO PI-R factor structure proved to be highly replicable in the sample of adolescents, with all facet scales primarily loading on the expected factors, independent of the age group. Domain and facet internal consistency coefficients were comparable to those obtained in adult samples, with less than 12% of the items showing corrected item-facet correlations below absolute value .20. Although, in general, adolescents reported few difficulties with the comprehensibility of the items, they tend to report more problems with the Openness to Ideas (05) and Openness to Values (06) items. Correlations between NEO PI-R and HiPIC scales underscored the convergent and discriminant validity of the NEO facets and HiPIC scales. It was concluded that the NEO PI-R in its present form is useful for assessing adolescents' traits at the primary level, but additional research is necessary to infer the most appropriate facet level structure.
This article contributes to the understanding of why the use of a frame-of-reference leads to increased criterion-related validity of personality inventories. Two competing explanations are described and tested. A between-subjects (N = 337) and a within-subject (N = 105) study are conducted to test the hypothesized effects of use of a frame of reference on reliability and validity. Regarding the effects on reliability, use of a frame of reference reduces within-person inconsistency (instead of between-person variability) in responding to generic items. Use of a frame of reference further leads to higher validity as a result of the reduction of between-person variability and within-person inconsistency. Yet, reducing these inconsistencies is not enough. It is also important to use a frame of reference that is conceptually relevant to the criterion. Besides implications for contextualized personality inventories, these results provide an explanation for the moderate validities of generic personality inventories.
Numerous twin studies have demonstrated genetic influence on personality traits, yet twin methods continue to be challenged. A common misconception is that monozygotic co-twins’ personality resemblance results from similar treatment by others, due to their matched physical appearance. The present study brings unique evidence to this question by assessing the similarities in personality and self-esteem of 23 pairs of unrelated look-alike individuals. Intraclass correlations for the Big Five personality traits (ris = −.27 to .29) and the Rosenberg Self-Esteem Scale (ri = −. 03) demonstrated little within-pair resemblance. It is concluded that (1) MZ co-twins’ personality similarity mostly reflects their shared genes, and (2) reactive gene-environment correlation best explains MZ co-twins’ similar treatment by others.
This study investigated a variety of factors which were hypothesized, on the basis of past research, to influence subjects' acceptance of feedback of test results—report content, the source of interpretation, and individual differences. The Cattell 16PF was administered to 83 subjects, each of whom subsequently attempted to identify their own report from a group of four. Half the subjects were led to believe that the personality data had been interpreted and the report generated by computer, while half were told a human expert was responsible for this. Subjects later rated their reports in terms of perceived accuracy and how satisfied they felt with them. The results showed that subjects were able to identify their test-derived personality at a level greater than chance, but that they did not place more confidence in computer analysis. However, the data suggest that an individual's level of formal education may influence confidence in human analysts. Gender and other individual differences were found to yield significant results in terms of correct identification of own report, females and those higher in self-perceptive traits being more accurate. The findings are discussed in terms of self-perception of traits and social desirability, and their implications for feedback of test results are discussed.
Three studies are reported concerned with people's perception of their own personality, their acceptance of bogus ‘personality’ feedback, and the relationship between their ‘actual’ personality scores and their willingness to accept bogus feedback. In the first study subjects attempted to predict their own and a well-known other person's personality scores. They were fairly good at predicting some of their own scores (extraversion, neuroticism) but less so others, suggesting that people can recognize their own ‘correct’ personality feedback. In the second study subjects were given either positive (Barnum Statements) or negative (reverse Barnum Statements) ‘bogus’ feedback after a personality test. They tended to accept the positive feedback as more accurate than the negative feedback though this was not related to their actual scores. In the third study subjects were given four types of feedback statements after a personality test: general positive, general negative, specific positive and specific negative. As predicted, people tend to accept general rather than specific, and positive rather than negative feedback as true. Furthermore, acceptance was closely related to neuroticism and extraversion in a predicted direction. These results are discussed in terms of the uses and abuses of validation of personality feedback.
This study examined the relationship between subjects' actual test derived scores and their estimates of what those scores would be. Fifty-six subjects completed three questionnaires (Morningness-Eveningness Questionnaire; FIRO-B; Myers—Briggs Type Indicator MBTI), and then estimated the scores on each dimension (15 in all) for themselves and another person that they knew well. The results showed significant positive correlations on 10 of the 15 dimensions for themselves. The dimensions that they were best at estimating were Morningness-Eveningness; Extraversion, and Thinking on the MBTI; and Wanted and Expressed Inclusion on the FIRO-B. Eight correlations reached significance concerning their ability to predict another known person's scores but were lower than for their own estimate-actual score correlations. Whereas subjects believed that they were like the other person they nominated (12 of the 15 correlations were significantly positive), in actual fact their test derived scores showed only five significant findings, two positive and the others negative. The results are discussed in terms of lay theories of personality and their relationship to personality assessment.
The paper explores a range of issues relating to the design and use of computer-based test interpretation (CBTI) systems. It describes a number of dimensions along which CBTI systems vary and considers both the positive and negative implications of these for test users, for their clients and for test candidates. A distinction is drawn between systems that generate finished reports and those that generate questions or guidance for the user. The question of how the validity of narrative reports can be assessed is considered. A study on the discriminative validity of reports generated by the PREVUE ICES CBTI system is reported which shows clear differences between judgements made about true and bogus reports. It is concluded that there are two main issues to consider when evaluating a CBTI system: first, the validity of the information generated by it and, second, the ways in which that information is likely to be used.
We investigated feedback acceptance by participants (N = 141) in an operational developmental assessment center (AC). Consistent with predictions based on self-enhancement theory and the Affective Infusion Model, results indicated that higher assessor ratings were associated with higher feedback acceptance, and this relationship was partially mediated by the participant's affective reaction to the feedback session. Participants' self-ratings of their AC performance did not affect this relationship suggesting no support for our prediction based on self-verification theory. We also investigated the role of participants' self-ratings of agreeableness, extraversion, and emotional stability in feedback acceptance by testing specific moderated mediation hypotheses. Extraversion was related to feedback acceptance through the affective reaction to the feedback session. Agreeableness moderated the affective reaction to the feedback session and feedback acceptance relationship such that there was a strong relationship between affective reaction and feedback acceptance for highly agreeable participants, but no relationship for low agreeable individuals. For low agreeable individuals, there was a positive direct relationship between the assessor ratings and feedback acceptance which was not observed for highly agreeable individuals. The hypothesized role of emotional stability in predicting feedback acceptance was not supported. Implications for the delivery of feedback in developmental ACs are discussed in light of these findings. Copyright © 2008 John Wiley & Sons, Ltd.
The purpose of this study was to increase our understanding of applicant perceptions of feedback by drawing upon feedback process models. In Study 1, participants (N=125) completed a personality questionnaire as a first stage of a selection simulation. Results showed that the effect of feedback on attitudes toward the organization was mediated by feedback acceptance. In Study 2, participants (N=252) completed two parallel versions of an in-basket exercise and received informative feedback between the two versions. Results showed that the effect of feedback on subsequent test performance was partially mediated by feedback acceptance. Together, these results highlight the important role of feedback acceptance in selection and suggest new strategies to enhance applicant perceptions in selection.
This study investigated a possible Barnum effect in personality traits, i.e., that persons accept a false trait profile as correct. Having answered the Norwegian translation of the NEO Personality Inventory-Revised, a group of 75 students, 56 (five women military aviation cadets and 19 (10 women) graduate students, received a random T-score profile and were asked to rate their agreement on a 7-point scale. For all personality traits, i.e., facets and domains measured by the NEO Personality Inventory-Revised, positive correlations ranging from .08 to .64 were found between the agreement ratings and the similarity between the random false scores and the actual scores. The respondents identified and rejected random T scores that deviated far from their actual scores, which works against the Barnum effect. Correct identification correlated negatively with Neuroticism r = -.41) and positively (r = .53) with Conscientiousness.
Examined the ability of college students at two levels of defensiveness to discriminate among three types of personality feedback: actual personality feedback based on the student's test scores, trivial feedback which is generally true of all students, and inaccurate feedback which was the exact opposite of the student's actual test scores. The students could reliably discriminate the three types of feedback on the dimensions of uniqueness, usefulness, accuracy, and as a source of new information about themselves. The results are discussed in the context of the Barnum literature to the effect that students can offer valid perceptions of personality descriptions of themselves.
Personality for professionals inventory. Handleiding
- De Fruyt
- F Rolland
De Fruyt, F., & Rolland, J. P. (2013). Personality for professionals inventory. Handleiding.
Amsterdam: Pearson.
The fallacy of personal validation: A classroom demonstration of gullibility Can people accurately estimate their own personality test-scores
- B R A Forer
Forer, B. R. (1949). The fallacy of personal validation: A classroom demonstration of gullibility. Journal of Abnormal and Social Psychology, 44, 118–123.
http://dx.doi.org/10.1037/h0059240
Furnham, A. (1990). Can people accurately estimate their own personality test-scores. European Journal of Personality, 4(4), 319–327.
http://dx.doi.org/10.1002/per.2410040405
The role of computer-based test interpretation (cbti) in occupational assessment. Paper presented at the Continuing Professional Development Workshop on Advances in Selection and Assessment
- D Bartram
Bartram, D. (1994). The role of computer-based test interpretation (cbti) in occupational assessment. Paper presented at the Continuing Professional Development
Workshop on Advances in Selection and Assessment, Nottingham, England.
Acceptance of personality questionnaire feedback: The role of individual difference variables and source of interpretation
- C Fletcher
- P Taylor
- K Glanfield
Fletcher, C., Taylor, P., & Glanfield, K. (1996). Acceptance of personality questionnaire feedback: The role of individual difference variables and source
of interpretation. Personality and Individual Differences, 20(2), 151-156.
http://dx.doi.org/10.1016/0191-8869(95)00169-7