Article

The influence of cognitive ability on interviewee performance in traditional versus relaxed behavior description interview formats

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

Using an experimental condition that relaxed cognitive constraints in a behavior description interview (BDI), our results uncovered a pattern of low cognitive saturation in the traditional BDI format but significantly higher in the relaxed condition. Well over half of the time interviewees reported different experiences in the relaxed condition, and those experiences were rated higher by the interviewers and correlated more strongly with job performance. A potential implication is that inhibitive cognitive demands in traditionally administered BDIs result in a number of interviewees reporting relevant experiences that come to mind easily rather than ones that maximally portray their capabilities, thereby shifting BDIs more towards assessment of typical behavior. With reduced cognitive constraints, interviewees had greater opportunity to locate more maximally oriented experiences, with higher ability individuals benefitting the most.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Cognitive difficulties may prevent applicants from producing effective stories during the interview (Bangerter et al., 2014;Ralston et al., 2003). Storytelling involves processes of autobiographical recall and response planning and production (Bradburn, Rips, & Shevell, 1987;Brosy, Bangerter, & Mayor, 2016;Huffcutt, Culbertson, Goebl, & Toidze, 2016). Behavioural questions may create strong memory demands because applicants have to recall a suitable episode. ...
... Because excessive delays in initiating a response may lead to negative impressions, applicants experiencing difficulties in choosing an example may choose to start talking faster but perhaps less appropriately (Brosy et al., 2016). They may be more likely to talk about recent or highly available events (Huffcutt et al., 2016), or produce pseudostories (generic descriptions of event types rather than a unique event; Bangerter et al., 2014) which may not serve them best. ...
... This manipulation allows investigating the role of cognitive factors like memory in recalling appropriate examples. More precisely, we expected the manipulation of information would allow applicants to search in memory for a relevant past episode in a context less stressful than during the interview and with less time pressure inherent to the question-answering process (Huffcutt et al., 2016) and to facilitate response planning and production. This leads to Hypothesis 1 (H1): Information increases production of stories. ...
Article
In behavioural interviews, past-behaviour questions invite applicants to tell a story about a past job-related situation. Nevertheless, applicants often do not produce stories on demand, resorting to less appropriate responses. In a sample of real selection interviews (Study 1), only 50% of applicants’ responses to past-behaviour questions were indeed stories. We explored two factors that may increase applicants’ storytelling tendencies: probing and information about past-behaviour questions. In two experiments simulating selection interviews, we manipulated recruiter probing during the interview (Study 2) and the level of participants’ information about the expected answer format of past-behaviour questions (Studies 2 and 3). Probing induced participants to tell more stories and to include more narrative diversity in their stories, but there was no effect of giving participants information or not. More information did help participants to tell less pseudostories (generic descriptions of situations). Analyses of participants’ thoughts and emotions experienced during question-answering suggest that finding an appropriate example to narrate is a major problem. Storytelling rate also varied by competency. Findings are relevant for theories of behaviour elicitation in selection situations.
... Cognitive difficulties may prevent applicants from producing effective stories during the interview (Bangerter et al., 2014;Ralston et al., 2003). Storytelling involves processes of autobiographical recall and response planning and production (Bradburn, Rips, & Shevell, 1987;Brosy, Bangerter, & Mayor, 2016;Huffcutt, Culbertson, Goebl, & Toidze, 2016). Behavioural questions may create strong memory demands because applicants have to recall a suitable episode. ...
... Because excessive delays in initiating a response may lead to negative impressions, applicants experiencing difficulties in choosing an example may choose to start talking faster but perhaps less appropriately (Brosy et al., 2016). They may be more likely to talk about recent or highly available events (Huffcutt et al., 2016), or produce pseudostories (generic descriptions of event types rather than a unique event; Bangerter et al., 2014) which may not serve them best. ...
... This manipulation allows investigating the role of cognitive factors like memory in recalling appropriate examples. More precisely, we expected the manipulation of information would allow applicants to search in memory for a relevant past episode in a context less stressful than during the interview and with less time pressure inherent to the question-answering process (Huffcutt et al., 2016) and to facilitate response planning and production. This leads to Hypothesis 1 (H1): Information increases production of stories. ...
... The candidate's job performance after being selected is the common measure of interview validity (Huffcutt and Arthur, 1994;Kleinmann et al., 2011). However, most of the results were drawn using mock interviews and experimental data (e.g., Ingold et al., 2015;Huffcutt et al., 2017). Our study further confirmed the current literature empirically by using data from a real work environment. ...
Article
Full-text available
Despite the popularity of the employment interview in the employee selection literature and organizational talent selection process, few have examined the comments interviewers give after each interview. This study investigated the predictability of the match between interviewer post-interview notes and radar charts from job analysis on the candidate’s later career performance using text mining techniques and data from one of the largest internet-based technology companies in China. A large sample of 7,650 interview candidates who passed the interviews and joined the company was obtained to show that the number of job-related capabilities interviewers mentioned in their notes was positively related to candidate’s job performance, the number of promotions, and negatively related to turnover. Moreover, the dimensions of the radar chart from job analysis covered in the interview moderated the predictability of interview post-interview notes. Our results indicated that a smaller number of radar chart dimensions by which interviewers assessed the candidates in the interview positively moderated candidates’ promotion for product development jobs, and negatively moderated turnover for technical jobs. The implications of these results for structured interview research in both theory and practice are discussed.
... Huffcutt, Culbertson, Goebl, & Toidze (2017) conducted research to look into the evaluation of the ability of cognitive skills during the interview. In the study, Huffcutt et al. (2017) concluded that the conversational job interview is a poor predictor of job performance. Similarly, Hartvigsson, & Ahlgren, (2018) found that candidates can practise or conduct mock sessions among themselves to familiarise themselves with the interview process. ...
Article
Full-text available
Recruiting the right talent into an organisation has become more challenging because humans have become more complex in terms of their cognitive, affective and behaviour. When it comes to choosing the right candidate, many interview techniques have been used previously but it can only be described as much as flipping a coin. All these limitations in interview techniques contribute to the needs to identify a more holistic way to choose the right candidate. This research is conducted through literature review to build a model that can help to recruit the right candidates at the workplace. From the literature review, a new conceptual framework is developed that focused on the compassionate element during the interview that would take place. This method helps to overcome the impromptu situation when a candidate attends an interview session. The fundamentals of the compassionate assessment and interview include truthfulness (Siddiq), trustworthiness (Amanah), speak the advocacy (Tabligh) and wisdom (Fathanah).
... Therefore, control on the exact experimental procedures, lighting on the computer screen or the noise level of the experimental environment was limited. This field context may have been stressful or cognitively demanding which can compromise participant performance (Huffcutt et al., 2017). A research assistant was assigned to each pair of participants to ensure proper implementation of the experimental procedures. ...
Article
Purpose Deception detection is instrumental in business management but professionals differ widely in terms of deception detection performance. The purpose of this paper is to examine the genetic basis of deception detection performance using the classic twin study design and address the research question: how much variance in individual differences in deception detection performance can be accounted for by the variance in genetics vs environmental influences? Design/methodology/approach In total, 192 twins, with 65 pairs of monozygotic (identical) twins and 31 pairs of dizygotic (fraternal) twins participated in an experiment. A series of behavioral genetic analyses were performed. Findings The variability in deception detection performance was largely determined by differences in shared and non-shared environments. Research limitations/implications The subjects were solicited during the Twins Days Festival so the sample selection and data collection were limited to the natural settings in the field. In addition, the risks and rewards associated with deception detection performance in the study are pale in comparison with those in practice. Practical implications Deception detection performance may be improved through training programs. Corporations should continue funding training programs for deception detection. Originality/value This is the first empirical study that examines the complementary influences of genetics and environment on people’s ability to detect deception.
Article
The purpose of this investigation was to provide a direct assessment of typical versus maximal responding in a Behavior Description Interview. A total of 109 participants were recruited from three universities and tested as applicants for a general retail position. When asked to describe a time when they had to deal with a difficult person, a core aspect of retail positions, responses reflected typical tendencies roughly half of the time and more maximal tendencies the other half. Convenience factors (e.g., recent, frequent) appeared to exert a strong influence on responding. Such mixing is problematic from a psychometric perspective given the relatively low correlation between these two aspects of performance. Typical versus maximal recall was not correlated with mental ability. Directions for future research are discussed.
Article
Full-text available
The primary purpose of this investigation was to test two key characteristics hypothesized to influence the validity of situational (SI) and behavior description (BDI) structured interviews. A meta-analysis of 54 studies with a total sample size of 5536 suggested that job complexity influences the validity of SIs, with decreased validity for high-complexity jobs, but does not influence the validity of BDIs. And, results indicated a main effect for study design across both SIs and BDIs, with predictive studies having 0.10 lower validity on average than concurrent studies. Directions for future research are discussed.
Article
Full-text available
Based on total of 290 undergraduates, the split-half reliability of the Wonderlic Personnel Test was .87 and the Pearson correlation between test score and mean grade was .21. Implications are presented for the use of this test in an academic setting.
Article
Full-text available
The current study tested whether candidates' ability to identify the targeted interview dimensions fosters their interview success as well as the interviews' convergent and discriminant validity. Ninety-two interviewees participated in a simulated structured interview developed to measure three different dimensions. In line with the hypotheses, interviewees who were more proficient at identifying the targeted dimensions received better evaluations. Furthermore, interviewees' ability to identify these evaluation criteria accounted for substantial variance in predicting their performance even after controlling for cognitive ability. Finally, the interviewer ratings showed poor discriminant and convergent validity. However, we found some support for the hypothesis that the quality of the interviewer ratings improves when one only considers ratings from questions for which interviewees had correctly identified the intended dimensions.
Article
Full-text available
This study contributes to the literature on why selection procedures that are based on the behavioral consistency logic (e.g., structured interviews and assessment centers) are valid predictors of job performance. We rely on interactionist theories to propose that individual differences in assessing situational demands explain true variance in performance in selection procedures and on the job. Results from 124 individuals in a simulated selection process showed that the assessment of situational demands was related to both selection and job performance. Individual differences in assessing situational demands also contributed to the criterion-related validity of assessment center and structured interview ratings, offering a complementary explanation as to why selection procedures based on the notion of behavioral consistency predict job performance. (PsycINFO Database Record (c) 2012 APA, all rights reserved).
Article
Full-text available
A predictive validation study was conducted to determine the extent to which the behavior description (BDI) and situational (SI) interviews predict typical versus maximum performance. Incoming MBA-students (n = 79) were interviewed regarding teamplaying behavior. Four months later, peers within study groups anonymously evaluated each person's typical teamplaying behavior, whereas other peers within project groups anonymously evaluated each person's maximum teamplaying behavior. Both the BDI and the SI predicted typical performance. The SI also predicted maximum performance. Implications and directions for future research are discussed.
Article
Full-text available
The purpose of this investigation was to explore the extent to which employment interview evaluations reflect cognitive ability. A meta-analysis of 49 studies found a corrected mean correlation of .40 between interview ratings and ability test scores, suggesting that on average about 16% of the variance in interview constructs represents cognitive ability. Analysis of several design characteristics that could moderate the relationship between interview scores and ability suggested that (a) the correlation with ability tends to decrease as the level of structure increases; (b) the type of questions asked can have considerable influence on the magnitude of the correlation with ability; (c) the reflection of ability in the ratings tends to increase when ability test scores are made available to interviewers; and (d) the correlation with ability generally is higher for low-complexity jobs. Moreover, results suggest that interview ratings that correlate higher with cognitive ability tend to be better predictors of job performance. Implications for incremental validity are discussed, and recommendations for selection strategies are outlined. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
The situational interview is based on the critical-incident technique. The incidents are turned into interview questions in which job applicants are asked to indicate how they would behave in a given situation. Each answer is rated independently by 2 or more interviewers on a 5-point Likert-type scale. To facilitate objective scoring, job experts develop behavioral statements that are used as benchmarks of illustrations of 1, 3, and 5 answers. In Studies 1 and 2, the interobserver reliability coefficients for situational interviews of 49 hourly workers and 63 foremen were .76 and .79, respectively. Similarly, the internal consistencies of the interview questions for the hourly workers and foremen were .71 and .67, respectively. The respective concurrent validity coefficients were .46 and .30. In Study 3, predictive validity coefficients of .39 and .33 were obtained with 56 women and Blacks, respectively. All of these values were significant at the .05 level. (18 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
Based on a study of federal investigative agents, Pulakos and Schmitt (1995) hypothesized that situational interviews are less effective for higher-level positions than behavior description interviews. To evaluate their hypothesis we analyzed data from 2 new structured interview studies. Both of these studies involved higher-level positions, a military officer and a district manager respectively, and had matching SI and BDI questions written to assess the same job characteristics. Results confirmed that situational interviews are much less predictive of performance in these types of positions. Moreover, results indicated very little correspondence between situational and behavior description questions written to assess the same job characteristic, and a link between BDI ratings and the personality trait Extroversion. Possible reasons for the lower situational interview effectiveness are discussed.
Article
Full-text available
Criterion-related validities and inter-rater reliabilities for structured employment interview studies using situational questions (e.g. “Assume that you were faced with the following situation … what would you do?”) were compared meta-analytically with studies using past behaviour questions (e.g. “Can you think of a time when … what did you do?”). Validities and reliabilities were further analysed in terms of whether descriptively-anchored rating scales were used to judge interviewees' answers, and validities for each question type were also assessed across three levels of job complexity.While both question formats yielded high validity estimates, studies using past behaviour questions, when used with discriptively anchored answer rating scales, yielded a substantially higher mean validity estimate than studies using the situational question format with descriptively-anchored answer rating scales (.63 versus .47). Question type (situational versus past behaviour) was found to moderate interview validity, after controlling for whether studies used answer rating scales. No support was found for the hypothesis that situational questions are less valid for predicting job performance in high-complexity jobs.Sample-weighted mean inter-rater reliabilities were similar for both situational and past behaviour questions, provided that descriptively-anchored rating scales were used (.79 and .77, respectively), although they were slightly lower (.73) for past behaviour question studies lacking such rating scales.
Article
This study investigated impression management tactic use during structured interviews containing both experience-based and situational questions. Specifically, the authors examined whether applicants' use of impression management tactics depended on question type. Results from 119 structured interviews indicated that almost all of the applicants used some form of impression management. Significantly more assertive than defensive impression management tactics were used, and among assertive tactics, applicants tended to use self-promotion rather than ingratiation. However, different question types prompted the use of different impression management tactics. Ingratiation tactics were used significantly more when applicants answered situational questions, whereas self-promotion tactics were used significantly more when applicants answered experience-based questions. Furthermore, the use of self-promotion and ingratiation tactics was positively related to interviewer evaluations.
Article
In a variety of situations in psychological research, it is desirable to be able to make statistical comparisons between correlation coefficients measured on the same individuals. For example, an experimenter may wish to assess whether two predictors correlate equally with a criterion variable. In another situation, the experimenter may wish to test the hypothesis that an entire matrix of correlations has remained stable over time. The present article reviews the literature on such tests, points out some statistics that should be avoided, and presents a variety of techniques that can be used safely with medium to large samples. Several illustrative numerical examples are provided.
Article
The interrater reliabilities of ratings of 9,975 ratees from 79 organizations were examined as a function of length of exposure to the ratee. It was found that there was a strong, nonlinear relationship between months of exposure and interrater reliability. The correlation between a logarithmic transformation of months of experience and reliability was .73 for one type of ratings and .65 for another type. The relationship was strongest during the first 12 months on the job. Changes in reliability were accounted for mostly by changes in criterion variance. Asymptotic levels of reliability were only about .60, even with 10-20 years of experience. Implications for estimating reliabilities in individual and meta-analytic studies and for performance appraisal were presented, and possible explanations of the reliability-variance relationship were advanced.
Article
In this study we proposed that understanding of interrelations among criterion measures can be improved by differentiating between measures of typical and maximum job performance. We offered defining characteristics of typical and maximum performance measures and requirements for a fair examination of relations between the two. Typical and maximum performance measures of the speed and accuracy with which supermarket cashiers process items were obtained from two large samples ( Ns = 635 and 735) of cashiers from 12 supermarket chains. Relatively low correlations between typical and maximum performance measures were found. We discuss the implications of these findings for the use of maximum performance measures as criteria in the validation of selection systems and for the understanding of the impact of ability and motivation on job performance. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
The interrater reliabilities of ratings of 9,975 ratees from 79 organizations were examined as a function of length of exposure to the ratee. It was found that there was a strong, nonlinear relationship between months of exposure and interrater reliability. The correlation between a logarithmic transformation of months of experience and reliability was .73 for one type of ratings and .65 for another type. The relationship was strongest during the first 12 months on the job. Changes in reliability were accounted for mostly by changes in criterion variance. Asymptotic levels of reliability were only about .60, even with 10–20 yrs of experience. Implications for estimating reliabilities in individual and meta-analytic studies and for performance appraisal were presented, and possible explanations of the reliability–variance relationship were advanced. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
reviews the literature on one particular type of structured interview, the patterned behavior description interview (PBDI) discusses two unpublished studies which compare two specific types of interview content—opinions versus behavior descriptions expands on the theoretical arguments for PBDI distinctions are made between maximum and typical performance . . . to explain the substantial validity of PBDIs (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
In psychological research, it is desirable to be able to make statistical comparisons between correlation coefficients measured on the same individuals. For example, an experimenter (E) may wish to assess whether 2 predictors correlate equally with a criterion variable. In another situation, the E may wish to test the hypothesis that an entire matrix of correlations has remained stable over time. The present article reviews the literature on such tests, points out some statistics that should be avoided, and presents a variety of techniques that can be used safely with medium to large samples. Several numerical examples are provided. (18 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
15 teaching assistants (TAs) were interviewed twice by interviewers trained in patterned behavior-based interviewing, and twice by interviewers trained in standard techniques. Interviewers predicted the ratings students would give TAs, and actual student ratings were gathered after all interviews had been completed. The reliability of the standard interviewers was higher (.71 vs .46) but the validity of the behavior interviews was .54 vs .07 for standard interviews. 45 of the 60 interviews were taped and the content of the interview questions and answers were categorized. Standard interviews contained significantly more credentials information and self-perception content, whereas the patterned, behavioral interviews contained significantly more behavior description content. (14 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Early studies of intuitive judgment and decision making conducted with the late Amos Tversky are reviewed in the context of two related concepts: an analysis of accessibility, the ease with which thoughts come to mind; a distinction between effortless intuition and deliberate reasoning. Intuitive thoughts, like percepts, are highly accessible. Determinants and consequences of accessibility help explain the central results of prospect theory, framing effects, the heuristic process of attribute substitution, and the characteristic biases that result from the substitution of nonextensional for extensional attributes. Variations in the accessibility of rules explain the occasional corrections of intuitive judgments. The study of biases is compatible with a view of intuitive thinking and decision making as generally skilled and successful.
Article
In this study the effects of stress-induced cortisol increases on long-term memory retrieval during and after acute psychosocial stress were examined. Seventy male students were exposed to either a psychosocial stress task or to a non-stressful control task. During and after this task, retrieval was tested for idiosyncratic emotionally negative and neutral word pair associations that were learned 1 day or 5 weeks earlier. Within the stress condition, retrieval of negative words, 5 weeks after learning, was impaired both during and after the stress task compared to the control group. Further, during the stress task, when sympathetic activity was enhanced, impaired retrieval of both neutral and emotional words was significantly related to enhanced cortisol response. In contrast, after the stress task, when cortisol levels were still increased but sympathetic activity was low again, no association was found between cortisol increase and retrieval of either neutral or emotional material. These results are in line with the previous animal research showing that when arousal is high, cortisol increase can impair memory retrieval.
Theoretical and systematic approaches in cross-cultural psychology
  • G Jahoda
Jahoda, G. (1980). Theoretical and systematic approaches in cross-cultural psychology. In H. C. Triandis, & W. W. Lambert (Eds.), 1. Handbook of crosscultural psychology (pp. 69e141). Boston: Allyn and Bacon.