Content uploaded by Zvonimir Galic
Author content
All content in this area was uploaded by Zvonimir Galic on Mar 12, 2018
Content may be subject to copyright.
Content uploaded by Zvonimir Galic
Author content
All content in this area was uploaded by Zvonimir Galic on Mar 12, 2018
Content may be subject to copyright.
Validity Evidence for a Croatian
Version of the Conditional Reasoning
Test for Aggression
Zvonimir Galic´*, Kelly T. Scherer** and James M. LeBreton***
*Department of Psychology, University of Zagreb, Luciceva 3, 10 000 Zagreb, Croatia. zgalic@ffzg.hr
**Purdue University, West Lafayette, IN, USA
***Penn State University, State College, PA, USA
The Conditional Reasoning Test for Aggression (CRT-A) is based on the idea that aggress-
ive individuals use motive-based cognitive biases to see their behavior as reasonable and
that those biases can be measured with inductive reasoning tasks. Although the initial valid-
ation efforts for the CRT-A in the United States have been reasonably successful, there has
been no attempt to determine if the evidence of validity and reliability generalizes to other
cultural contexts. In this paper, we describe four studies designed to systematically accumu-
late validity evidence for the CRT-A using Croatian participants. Our analyses revealed that
the Croatian adaptation of the CRT-A yielded psychometric characteristics that were simi-
lar to those obtained on the US samples (Study 1). CRT-A scores that predicted counter-
productive work behaviors occurrence beyond self-reported personality (Study 2) were
independent from general mental ability as measured with an abstract reasoning test
(Study 3), and not susceptible to faking (Study 4).
1. Introduction
James and his associates (James et al., 2005; James &
LeBreton, 2010, 2012) recently introduced the Con-
ditional Reasoning Test for Aggression (CRT-A) that
assesses aspects of implicit personality and has import-
ant implications for understanding and predicting
counterproductive work behaviors (CWBs; Wu &
LeBreton, 2011). The test is based on the assumption
that individuals with a strong motive to aggress reason
differently than the majority of people who have so-
cially adaptive values and prosocial ideologies. The
reasoning of aggressive individuals is characterized by
frequent use of justifications that help them to resolve
the conflict between the motive to aggress (i.e., the
motive to harm) and the motive to hold a favorable
view of self. James and colleagues (James et al., 2005;
James & LeBreton, 2010, 2012) identified six justifica-
tion mechanisms that they believed aggressive individu-
als use to enhance the logical appeal of aggressive
behaviors. These mechanisms and their definitions are
listed in Table 1. James and his associates suggested
that these justification mechanisms are essentially cog-
nitive biases that impact how information is perceived,
processed, and used to make decisions. Consequently,
they argued that it is possible to measure these
motive-based cognitive biases using a special form of
inductive reasoning task that they called conditional
reasoning (see James & LeBreton, 2010, 2012 for ex-
ample items).
In the review of the studies conducted on large
samples of US respondents, James and LeBreton (2012)
reported satisfactory levels of internal consistency and
test–retest stability for CRT-A scores. Moreover, their
studies garnered support for the construct and conver-
gent validity of the CRT-A. The factor analysis of tasks
revealed a structure coherent with the theory, and
CRT-A scores did not show significant relationships
with cognitive abilities or self-reported aggression
(James & LeBreton, 2010, 2012). Importantly, LeBreton,
Barksdale, Robin, and James (2007) showed that the
CRT-A is resistant to faking when the indirect nature of
measurement is preserved. Finally, the CRT-A scores
predicted aggressive behaviors/CWBs in both laborat-
ory and field settings. Meta-analytical estimates of the
uncorrected validities of CRT-A scores in predicting ag-
bs_bs_banner
International Journal of Selection and Assessment Volume 22 Number 4 December 2014
©2014 John Wiley & Sons Ltd,
9600 Garsington Road, Oxford, OX4 2DQ, UK and 350 Main St., Malden, MA, 02148, USA
gressive behaviors differed between sources. Whereas
first estimates indicated high validities (.44; James et al.,
2005), recent meta-analyses revealed more modest
numbers (validity of CRT-A scores in predicting CWBs
was .28 in James & LeBreton, 2012, and .16 in Berry,
Sackett, & Tobares, 2010; .08 in Banks, Kepes, &
McDaniel, 2012). The variability in these meta-analytic
estimates is driven largely by differences in the subject-
ive decision each team of researchers made in conduct-
ing their meta-analysis (Wanous, Sullivan, & Malinak,
1989). Nevertheless, across these meta-analyses, there
is evidence that scores on the CRT-A may be predictive
of important organizational criteria (e.g., CWBs).
1.1The present study
The work on the conditional reasoning approach to
measurement of aggression implies that motive-based
cognitive biases that aggressive individuals use to see
their behavior as reasonable and logical are universal
and should operate in any cultural context. But, interna-
tional research on the CRT-A is scarce. For example, all
three meta-analyses that tested the usefulness of this ap-
proach to personality assessment relied exclusively on
the US samples (Banks et al., 2012; Berry et al., 2010;
James & LeBreton, 2012).
The purpose of the current study is to explore the
extent to which the justification mechanisms identified
by James and associates may generalize to a substantially
different societal context. Indeed, it is quite possible
that certain justification mechanisms are not operative
in certain cultures. For example, one might argue that
the victimization by powerful others bias or the social
discounting bias (i.e., the proclivity to frame social
norms as repressive and restrictive of free will) might
not work as justification mechanisms for aggressive be-
haviors in societies that are less libertarian and more
collectivistic than the United States. In less individualistic
societies with larger power distances (Hofstede, 2001),
unequal distribution of power is expected and therefore
these biases may not be used to justify aggressive behav-
iors. Moreover, the selection of justification mechanisms
for aggression might depend on cultural experiences
with aggressive behaviors. For example, in countries
with a recent broad experience of aggression (e.g., due
to a recent war), the hostile attribution bias might to a
certain extent reflect a veridical perception of social in-
teractions and not a justification mechanism used by ag-
gressive individuals. Dependence of the CRT-A on
cultural experience could seriously limit its usefulness in
international contexts.
In our study, we tested the CRT-A’s validity in Croa-
tia. We believe that Croatia represents an interesting
context for examining the CRT-A’s assumptions at least
for two reasons. First, Croatia represents a different
cultural context characterized by lower individualism
and higher power distance than the United States
(Hofstede, 2001). Second, the experience of war (1991–
1995) and the difficult transition from a socialistic to a
free-market country (Tanner, 2010) gave Croatian
citizen an extensive (and compared with US samples, a
unique) experience with acts of aggression (and the ac-
companying justifications for those acts).
In this paper, we report the results of the four studies
where data were collected using a Croatian version of
the CRT-A. In the first study, we explored the psycho-
metric evidence (i.e., reliability estimates, item total
correlations, and factor structure) of the Croatian ad-
aptation of CRT-A in a large heterogeneous sample of
Croatian respondents. In the second study, we explored
the relationship between CRT-A scores and scores on a
measure of CWBs, and we also tested whether scores
on the CRT-A provided incremental prediction of
CWBs above and beyond commonly used self-report
personality inventories. In the third study, we examined
the relationship between scores on the CRT-A, and
scores on a test of general mental ability (GMA). Finally,
in the fourth study, we examined whether scores on the
CRT-A changed with demands for positive presenta-
tions by comparing scores across applicant, incumbent,
and student samples faking good.
2. Study 1
Within the first study, we tested psychometric proper-
ties of the CRT-A problems with data collected from a
heterogeneous sample of participants consisting of stu-
dents, incumbents, and personnel selection applicants.
Per the recommendations of James and McIntyre (2000),
Table 1. Justification mechanism for motive to aggress (James
& LeBreton, 2012)
Justification
mechanism Definition
Hostile attribution
bias
A tendency to see hostile and malevolent
intention in the behavior of others
even when that behavior is friendly
Derogation of
target bias
A tendency to derogate the targets of
aggression as evil, immoral, and
untrustworthy
Potency bias A tendency to see interactions with
others as a contest for dominance that
pit the strong/brave against the
weak/cowardly
Retribution bias A tendency to favor revenge and
retaliation over acts of reconciliations
Victimization by
powerful
A tendency of aggressive individuals to
see themselves as victims of powerful
others who exploit and oppress them
Social discounting
bias
A tendency to perceive social norms as
repressive and restrictive of free will
344 Zvonimir Galic´, Kelly T. Scherer and James M. LeBreton
International Journal of Selection and Assessment
Volume 22 Number 4 December 2014
©2014 John Wiley & Sons Ltd
all respondents were instructed that they were solving a
reasoning test and to complete the testing battery to
the best of their ability.
2.1. Method
2.1.1. Participants
In total, data obtained on 988 respondents were in-
cluded in the study (642 students, 196 incumbents,
and 150 applicants in an actual selection situation). In
accordance to the test manual instructions (James &
McIntyre, 2000), participants who endorsed five or
more illogical answers were omitted from the database.
In total 21of 988 participants were omitted (2.22%).
Thus, the factor analyses were conducted on the data
from 967 participants. The participants were 47.7%
male, with an average age of 26.28 (standard deviation
[SD]=9.05). The final sample size, gender structure, and
average age according to sample type were: students
(n=628; 46.4% male, Mage =21.91,SDage =3.61), incum-
bents (n=192; 46.7% male, Mage =36.61,SDage =11.88),
and applicants (n=147) 54.4% male, Mage =30.93,
SDage =7.17).
2.1.2. Instrument
The CRT-A consists of 25 inductive reasoning problems
– 22 of these problems are conditional reasoning prob-
lems designed to reveal the justification mechanisms as-
sociated with aggression in a respondent’s reasoning
(James & McIntyre, 2000; James & LeBreton, 2010,
2012). Each of these 22 problems has four response op-
tions: an inductively logical aggressive response, an in-
ductively logical response based on socially adaptive
ideology and reasoning, and two illogical responses.
Three of the test problems (Problems 1, 2, and 6) are
‘classic’ inductive reasoning problems containing a single
inductively valid solution. These three items are not
scored as part of the CRT-A, but instead are included
early in the test to further enhance its face validity as a
reasoning test and thus maximize the indirect nature of
assessment.
2.1.3. Translation and adaptation process
The translation and adaptation of the CRT-A to Croa-
tian was consistent with the recommendation for a
cross-cultural test adaptation (Geisinger, 1994; Hui &
Triandis, 1985). Three Croatian researchers proficient
in English independently translated the CRT-A into
Croatian. Considering that all translators judged that
some information included in the CRT-A problems
were culture specific, certain adaptations had to be
made in the Croatian version. Most of the changes were
minor and related to changes in the names of individuals
and places from American names/places to Croatian
names/places. Only the problem exploring reasons of
recent improvements of the American car industry
underwent a major change such that ‘American car
industry’ was replaced with ‘Croatian industry of refri-
gerators’ in order to appeal to Croatian respondents.
The response options for this problem in the Croatian
version were formulated so that the same aggression
justification mechanisms would be operative as were in
the original problem. The adapted version of the CRT-A
was then reviewed by two psychologists experienced in
personality assessment and familiar with the conditional
reasoning approach. The Croatian version of the CRT-A
was finalized after incorporating a few minor suggestions
raised by these two psychologists.
2.1.4. Procedure
In all situations the CRT-A was administered according
to the instructions provided in the test manual (i.e., as
an inductive reasoning test; James & McIntyre, 2000).
Data from students were collected during a regular class
meeting. The students were motivated to complete the
study package with the information that they would
have the opportunity to gain practice by completing the
employment selection tests commonly used by organiza-
tions to select graduate students for entry-level posi-
tions. The package contained various psychological
instruments, including the CRT-A.
Incumbents were recruited by students from an I/O
psychology course who, as part of an extra credit
assignment, were asked to identify individuals who
worked more than 6 months, for 20 or more hours per
week. The students were instructed to give the survey
package to participants, who fit the above description, at
their homes, and to make sure participants completed
the CRT-A within the 25-min time limit. Both particip-
ants and students received instructions that the CRT-A
represents an ordinary reasoning test, and were not
aware of its measurement objective during the data col-
lection period.
The applicants were recruited in two mid-sized Croa-
tian companies. In both firms, the CRT-A was adminis-
tered as a part of a selection test battery for
professional positions related to sales or marketing
(e.g., the largest number of applicants was seeking
employment as brand/product managers, key account
managers, or marketing specialists).
2.2. Results and discussion
2.2.1. Descriptive statistics
Average item ‘difficulties’ and item total correlations are
reported in Table 2. The proportion of respondents
who selected the aggressive answer varied between .03
(‘bosses and employees’) and .60 (‘good product at low
price’). The average proportion of aggressive responses
Conditional Reasoning Test for Aggression 345
©2014 John Wiley & Sons Ltd International Journal of Selection and Assessment
Volume 22 Number 4 December 2014
Table 2. Comparison of psychometric characteristics for the items of the Conditional Reasoning Test for Aggression (CRT-A) obtained on the Croatian and US samples: item diffi-
culties, item total correlations, and factor saturations
Problem
no. Items
Our study (n=967) James and LeBreton (2012; n=4772)
Item characteristics Factor saturationaItem characteristics Factor saturationsa
pr
bF1F2 pr
bF1F2 F3
3 Late for meetings .21.26 −.05 .23 .31.35 .04 .05 .23
4 Aggressively going after customers .19 .32 .05 .14.21.38 .12.21.09
5 Generals .11 .40 .33 .12 .05 .41.38 .49 .08
7 An eye for an eye .05 .45 .30 .30 .05 .41.42 .63 −.05
8 Bosses and employees .03 .43 .41.30 .07 .38 .30 .37 .16
9 New technology and workplace .22 .36 .14.14.35 .41.04 .19.28
10 Girl scouts and boy scouts .26 .35 −.12.42 .06 .51.45 .43 .48
11 A homeless man .10 .47 .64 −.09 .19 .42 .32 .13 .23
12 Good product at a low price .60 .36 .06 .00 .37 .49 .19.10.40
13 Duels with swords .13.41.02 .70 .21.45 .11 .43 .20
14 A new girl at the high school .53 .41.09 .17.26 .49 .28 .19.29
15 Permits to carry guns .12.31.19.07 .15 .39 .29 .22 .06
16 American cars (adapted to Croatian fridges) .29 .46 .41−.13.13 .39 .48 .12.10
17 Store employees versus shoplifters .15.41.38 −.02 .19.41.29 .22 −.01
18 Bonuses for employees .20 .45 .43 .01.04 .46 .56 .34 .25
19 Search on employees .22 .46 .31.08 .29 .45 .25 .23 .17
20 Gangs .34 .42 .38 .02 .22 .45 .36 .14 .25
21Wild animals .14 .32 −.12.49 .22 .42 .11 .35 .18
22 Hold up victims .15 .24 .28 −.21.08 .24 .44 .18 .05
23 Divorces .29 .25 .25 −.16 .28 .37 .24 .17−.09
24 Employee’s revenge .33 .52 .45 .02 .14 .46 .53 .27 .13
25 Agreement between countries .26 .50 .49 .02 .04 .46 .59 .40 .11
Note:aThe largest absolute structure correlations are given in bold.
p=proportion of aggressive responses; rb=biserial correlation coefficient between item response and total score on the CRT-A.
346 Zvonimir Galic´, Kelly T. Scherer and James M. LeBreton
International Journal of Selection and Assessment
Volume 22 Number 4 December 2014
©2014 John Wiley & Sons Ltd
was .22, slightly higher than the one reported on the US
sample (.18; James & LeBreton, 2012). For only two
problems (the ‘good product at low price’ [p=.60] and
the ‘a new girl at the high school’ [p=.53]), the propor-
tion of respondents who selected the aggressive altern-
ative exceeded .50.
Item total correlations calculated as biserial correla-
tion coefficients (Lord & Novick, 1968) ranged between
.24 (‘hold up victims’) and .52 (‘employee revenge’). The
average biserial coefficient in our study was .39, which
was very similar to the .42 reported by James and
LeBreton (2012) in their review based on US samples.
The internal reliability coefficient for the Croatian
CRT-A calculated based on item total biserial coeffi-
cients (see Equation 21, p. 389; Gulliksen, 1950) was .73,
which was quite similar to the value of .76 reported by
James and LeBreton (2012).
2.2.2. Factor analyses
In accordance with the instructions given in James and
LeBreton (2012), for the purposes of factor analysis, we
scored responses to the problems on a 3-point scale
such that −1(non-aggressive alternative),0(illogical or no
response), and 1(aggressive alternative). Using recoded
responses, we conducted exploratory factor analysis on
polychoric correlations analyses using fa.poly procedure
in R (R Development Core Team, 2010). The procedure
was used in order to take discrete categorical scoring
into account (Joreskog & Sorbom, 2005) and to avoid
underestimation due to skewed data (Olsson, Drasgow,
& Dorans, 1982).
The maximum likelihood was used as the model fit-
ting procedure. Comparisons of one-, two-, and three-
factor solutions revealed that the last two showed
acceptable model fit (two factor: root mean square
error of approximation [RMSEA] =.078; confidence in-
terval [CI] =.073–.081; three-factor: RMSEA =.075;
CI =.070–.078) whereas the fit of the one-factor model
was above the .08 threshold (RMSEA =.086; CI =.082–
.090). The two-factor solution converged with our
subsequent interpretation of various solutions, which
ultimately yielded a two-factor solution. The two factors
were rotated obliquely using promax rotation. The cor-
relation between the two factors was .26 (p<.001). The
rotated factor structures obtained on the Croatian
sample and the factor solution from a large US sample
(n=4772; James & LeBreton, 2012) are also shown in
Table 2.
James and LeBreton (2012) in their analysis labeled
the first factor as the external justification bias because
most of the problems that loaded on the factor indic-
ated hostile attribution and victimization by powerful other
biases. Essentially, people who have a high score on this
factor justify their aggressive acts by others’ behavior.
In the original analysis (James & LeBreton, 2012), the
second factor was labeled as internal justification because
the problems on which it loads reflect the retribution
and potency biases. Common to these two biases is an
internal justification for aggressive behavior such that
aggressive individuals see retribution as more appealing
than forgiveness or reconciliation, or value interactions
with others as a context for bravery. Finally, James and
LeBreton (2012) interpreted the third factor obtained
with US data as powerlessness because it was related to
aggressive scores on conditional reasoning problems for
which ‘a key source for building a false sense of rational-
ity for aggression is anger and frustration with the lack
of control over one’s life’ (James & LeBreton, 2012,
p. 121).
As might be seen from Table 2, Factor 1from the
Croatian data was reasonably similar to the external jus-
tification obtained on the US samples. All 11 problems
that had primary loadings on the first factor in United
States-based studies showed the largest correlations
with the first factor for the Croatian sample. The most
important difference was that the internal justification
and powerlessness factors from the James and LeBreton
(2012) factor analysis merged into a single factor in the
Croatian sample. Most of the problems that loaded on
the second and the third factor in the US sample had
primary factor loadings on the second factor in the
Croatian sample. This finding seems reasonably aligned
with theory behind the instrument because justification
for aggression on the conditional reasoning problems
related to both the internal justification and powerless-
ness factors from James and LeBreton (2012) is found
within aggressive individuals. Further differentiation of
the internal justification factor into two factors ob-
served in the James and LeBreton (2012) analysis could
be related to the much smaller number of participants in
our study.
There are some exceptions in primary loadings be-
tween our analysis and earlier factor analyses but those
differences could be traced either to the sampling or
specific context of the data collection. In our analysis,
the ‘generals’ and ‘bosses and employees’ problems
loaded more strongly on the external justification, than
in the original analysis in which it was more strongly
saturated with the internal justification bias. However, it
should be observed that both in our analysis and the
original analyses (James & LeBreton, 2012) that problem
had relatively high loadings on the first two factors.
Therefore, the differences in primary projections may
be related to simple sampling error. In our analysis the
problem ‘good product at low price’ had low satur-
ations with both factors revealing that it did not share
variance with other problems. An interaction between
the content of the conditional reasoning problem and
the context of the data collection might have contrib-
uted to this finding. For the problem, respondents were
asked to find the most logical conclusion that follows
Conditional Reasoning Test for Aggression 347
©2014 John Wiley & Sons Ltd International Journal of Selection and Assessment
Volume 22 Number 4 December 2014
the story about labor cost cuts (i.e., downsizing, wage
reductions) to which organizations are forced in order
to keep prices of their products low. The aggressive al-
ternative ascribed ‘evil’ intention to the employing orga-
nizations that reflects hostile attribution bias. During the
period of data collection, Croatian citizens experienced
economic downturn followed by widely enforced lay-
offs and wage cuts that were not always justified by an
economic
necessity (Galic´ & Plec´aš, 2012). This means that the
selection of the aggressive alternative could reflect re-
spondents’ actual experience and not a cognitive bias
used by dispositionally aggressive individuals to justify
hurting others.
Regardless of minor differences, the psychometric
properties of the Croatian adaptation of the CRT-A
seemed reasonably similar to those obtained for the
US sample and coherent with the theory behind the
instrument.
3. Study 2
Within the second study, we examined the validity of
the Croatian adaptation of the CRT-A in predicting self-
reported CWBs. We also explored incremental validity
of CRT-A scores over self-reports on the Big Five ques-
tionnaire and a self-report measure of aggression. These
two instruments were selected because they represent
the dominant conceptualization of personality for scien-
tific and practical purposes (Big Five; Rothstein & Goffin,
2006) or represent a personality trait most often
related to CWBs (anger; Douglas & Martinko, 2001;
Hershcovics et al., 2007).
3.1. Method
3.1.1. Participants
Participants in this study were full-time employees
(n=192) from a variety of occupations whose charac-
teristics have already been described within Study 1(the
incumbents sample).
3.1.2. Instruments
3.1.2.1. Personality: conditional reasoning. The CRT-A
measured the justification mechanisms for aggression.
Internal consistency of the test was slightly lower than
reported in previous studies with a value of 0.67.
3.1.2.2. Personality: self-report. The 50-item question-
naire from the International Personality Item Pool (IPIP-
50) measured the Big Five factors (Goldberg et al.,
2006). In addition, we also used the IPIP 10-item angry
hostility scale as our indicator of self-reported aggres-
sion. Reliability of all scales was satisfactory and ranged
between 0.77 (conscientiousness) and 0.88 (emotional
stability). A Croatian version of this instrument was
developed following the translation-back translation
procedures.
3.1.2.3. Counterproductive work behaviors. Participants
filled in a 34-item measure taken from the CWB check-
list (Spector et al., 2006). The scale encompasses vari-
ous CWBs directed toward other individuals and the
organization. Respondents reported their frequency of
CWBs on a 5-point scale for which 1(never),2(once or
twice), 3 (once or twice per month), 4 (once or twice per
week), and 5 (every day). We aggregated all of the items
in order to obtain a total score of CWBs. Reliability of
the total score was 0.86. A Croatian version of this in-
strument was developed following the translation–back
translation procedure.
3.1.3. Procedure
The procedure of the recruitment and the incumbents’
CRT-A data collection is already given in the description
of the procedure within Study 1. In addition to the
CRT-A, the participants received a package containing
the self-reported personality measures and the CWBs
scale. After they completed all instruments, participants
were instructed to put all of the questionnaires into an
envelope, seal it, and give it to the student researchers
who then brought the envelopes to the head re-
searcher. All participants responded to the instruments
anonymously, without giving any information about their
identity.
3.2. Results and discussion
The reported percentages revealed that CWBs were
relatively common in our sample. For example, over
two thirds of participants admitted that they took a lon-
ger break than they were allowed to take, and more
than a half of the participants tried to look busy while
doing nothing or ignored someone at work. A relatively
high frequency was reported even for less desirable be-
havior (e.g., 42.7% of participants reported they pur-
posely wasted their employer’s materials and supplies,
and 35.4% had been nasty or rude to a client or cus-
tomer). The percentages of reported CWBs suggest
that participants were probably honest in reporting un-
desirable behaviors.
In Table 3, we present the correlations among Study
2 variables. The CRT-A score had a significant positive
correlation with CWBs total score (r=.22; p<.01), but
also showed a significant relationship with gender
(r=−.16), age (−.16), and tenure (−.14, all p<.05). No
significant relationships were found between CRT-A
score and self-reported personality, which was consist-
ent with United States-based studies of the CRT-A
348 Zvonimir Galic´, Kelly T. Scherer and James M. LeBreton
International Journal of Selection and Assessment
Volume 22 Number 4 December 2014
©2014 John Wiley & Sons Ltd
(James & LeBreton, 2012). In addition to correlating
with CRT-A score, the total score on the CWBs scale
correlated with gender, agreeableness (negatively), con-
scientiousness (negatively), emotional stability (negat-
ively), and angry hostility (positively). The finding that
the CWBs scale correlated negatively with agreeable-
ness, conscientiousness, and emotional stability and
positively with the angry hostility scale was in accord-
ance with the literature (Berry, Ones, & Sackett, 2007;
Hershcovics et al., 2007).
In order to test the potential incremental validity of
the CRT-A scale in predicting CWBs, we conducted a
hierarchical regression analysis wherein the CWBs scale
total score was regressed first on self-report personality
traits (entered in Step 2) before adding the CRT-A
scores (entered in Step 3). Considering that age and
gender showed a relationship with CRT-A and/or
CWBs scale score, we treated them as covariates (en-
tered in Step 1). By including these variables as we es-
sentially allowed them to take credit for any variance
they might have shared with CWBs and scores on the
CRT-A. Thus, inclusion of these covariates provides a
conservative test of the incremental validity of the
CRT-A. Because of the high intercorrelation with age
(.95, p<.001), tenure was not included in the regression
analysis as a covariate.
The angry hostility scale was highly correlated with
the overall emotional stability scale (.83; p<.001).
Thus, to minimize problems with multicollinearity (e.g.,
inflated standard errors; Cohen, Cohen, West, &
Aiken, 2003) we opted to conduct two separate re-
gression analyses. In the first, we tested the incre-
mental validity of the CRT-A over the Big Five traits,
and in the second analysis, over the angry hostility
scale. The results of the two analyses are shown in
Tables 4a and b, respectively.1
The results of Study 2 offer two important insights
about the usefulness of Croatian adaptation of the
CRT-A. First, CRT-A score showed a significant rela-
tionship with CWBs. The size of the correlation was
relatively low (.22), but in accordance with the size of
the correlations for the other personality constructs.
Second, our data support the proposition that the
CRT-A measures a construct that is unrelated to self-
reported personality; the CRT-A score adds to the
Table 3. Descriptive statistics and intercorrelations of variables in Study 2
Variable M(SD) 2.3.4. 5.6. 7. 8. 9. 10. 11.12.
1. Gendera1.54 (0.49) .06 .06 .06 .02 .27** −.02 −.16* −.13.13−.16* −.14*
2. Age 36.71(11.64) – −.03 .94** −.09 .04 .21** −.08 −.18* .07 −.19** −.12
3. Educationb1.48 (0.50) – −.15* .04 .07 .06 −.03 .17* −.05 −.05 .04
4. Tenure 14.42 (11.57) – −.08 .07 .24** −.05 −.20 .04 −.16* −.12
5. Extraversion 33.75 (6.00) – .31** −.10.21** .37** −.05 .00 .13
6. Agreeableness 38.99 (5.07) – .17* .12 .20** −.15* −.08 −.21**
7. Conscientiousness 37.47 (5.28) – .13 .04 −.15* −.11 −.25**
8. Emotional stability 33.10 (7.04) – .15* −.83** −.02 −.18*
9. Intellect 35.40 (5.51)–−.14−.04 .00
10. Angry hostility 27.00 (7.12) – .02 .19**
11. CRT-A 4.57 (2.18) .22**
11. CWB 49.81(11.21) –
Note:aMale =1, female =2. b1=higher education, 2 =college education.
CRT-A =Conditional Reasoning Test for Aggression; CWBs =counterproductive work behaviors; M=mean; SD =standard deviation.
Table 4. (a) Summary of hierarchical regression analyses in
predicting counterproductive work behaviors (CWBs): incre-
mental validity of the Conditional Reasoning Test for Aggres-
sion (CRT-A) over the Big Five self-reports. (b) Summary of
hierarchical regression analyses in predicting counterproduc-
tive work behaviors (CWBs): incremental validity of the Con-
ditional Reasoning Test for Aggression (CRT-A) over angry
hostility self-reports
(a) ΔR2β
Predictor
Step 1.03*
Control variablesa
Step 2 .14**
Extraversion .22**
Agreeableness −.18*
Conscientiousness −.15*
Emotional stability −.21*
Intellect .04
Step 3 .02*
CRT-A .16*
Total R2.19**
n192
(b) ΔR2β
Predictor
Step 1.03*
Control variablesa
Step 2 .05**
Angry hostility .22**
Step 3 .03*
CRT-A .18*
Total R2.11**
n
Note:aControl variables included gender and age.
*p<.05. **p<.01.
Conditional Reasoning Test for Aggression 349
©2014 John Wiley & Sons Ltd International Journal of Selection and Assessment
Volume 22 Number 4 December 2014
prediction of CWBs above sociodemographic variables
and self-reported personality.
We should add that as an anonymous reviewer
noted, the use of a self-report criterion as less than
ideal, in part because previous research has documented
that the optimal criteria against which to validate meas-
ures of implicit personality tend to be more objective
behaviors (Bornstein, 1999; James & LeBreton, 2012).
Objective criteria are often preferred because they cir-
cumvent potential concerns associated with response
biases, faking, and the overall introspective accuracy
(or lack thereof) that associated with some self-report
measures (Haidt, 2001; Morgeson et al., 2007; Nisbett &
Wilson, 1977).
Indeed, a growing number of researchers (Bing,
LeBreton, Davison, Migetz, & James, 2007; Bornstein,
1999, 2002; Frost, Ko, & James, 2007; James &
LeBreton, 2010, 2012; McClelland, Koestner, &
Weinberger, 1989; Spangler, 1992; Winter, John,
Stewart, Klohnen, & Duncan, 1998) have found that
measures of implicit and explicit personality often pre-
dict different types of criteria: (a) implicit measures are
more closely linked to objective behavioral criteria
whereas (b) explicit measures tend to be more closely
related to subjectively measured criteria (e.g., self-
reports or other reports). Thus, although the use of a
self-report criterion measure in the current study was
suboptimal, the most likely consequence of using such a
criterion would be to decrease the likelihood of finding
significant relationships with the CRT-A.
4. Study 3
One possible explanation for the predictive validity of
the CRT-A in Study 2 was that the observed correlation
with CWB may have resulted from a spurious relation-
ship both variables might have with GMA. In the third
study, we wanted to see if the finding that scores on the
CRT-A are uncorrelated with measures of GMA (James
& LeBreton, 2012) generalized to the Croatian adapta-
tion of the test. In addition to an abstract reasoning test
that served as our measure of GMA, we also included
two additional variables known to be correlated with
GMA: need for cognition (Cacioppo, Petty, & Kao,
1984) and cognitive reflection (Frederick, 2005). Al-
though the need for cognition represents a well-known
individual difference variable of inclination toward
effortful cognitive activities (Cacioppo et al., 1984),
the cognitive reflection test measures a tendency of
respondents to inhibit a prepotent response alternative
that is incorrect and to engage in more effortful reflec-
tion that leads to the correct response (Frederick,
2005). The relationship of the two variables with GMA
served as a benchmark for our evaluation of the rela-
tionship between the CRT-A and GMA.
4.1. Method
4.1.1. Participants
The study was conducted on a sample of undergradu-
ate and graduate students from a Croatian business
school (n=100). Analogous to earlier described
studies, participants who had five or more illogical
answers were omitted from the database (n=3, 3%
of the sample). Thus, the analyses were conducted
on the data from 97 participants. The participants
were 48.5% male, with an average age of 23.80
(SD =6.62).
4.1.2. Instruments
Croatian versions of all instruments were developed
using the translation-back translation procedures.
4.1.2.1. Personality: conditional reasoning. The CRT-A
measured the justification mechanisms for aggression.
The internal consistency reliability of the test was .75.
4.1.2.2. General mental ability. The abstract reasoning
test from the Differential Aptitude Test battery (Bennet,
Seashore, & Wessman, 1990) was used as a GMA
measure. The internal consistency of the test using the
KR 20 formula was .87.
4.1.2.3. Need for cognition. The need for cognition was
measured using an 18-item (Cacioppo et al., 1984)
scale. Sample items from the scale are ‘I would prefer
complex to simple problems’ and ‘I only think as hard
as I have to’ (negatively coded). The participants gave
their responses on a 5-point scale ranging from 1(com-
pletely disagree)to5(completely agree). The internal
consistency of the scale in our study was .73.
4.1.2.4. Cognitive reflection. The Cognitive Reflection
Test (Frederick, 2005) consists of three items that gen-
erate an intuitive incorrect answer and measures
cognitive impulsivity, that is, the ability to resist report-
ing responses that come to mind. A sample item is ‘If
it takes 5 machines to make 5 widgets, how long
would it take 100 machines to make 100 widgets?’ The
intuitive and incorrect answer is 100, and correct
answer 5. The internal consistency of the test in this
study was .55.
4.1.3. Procedure
During the regular class, students were asked to com-
plete a booklet consisting of three tests. They were
instructed that all three tests measured aspects of
cognitive functioning.
350 Zvonimir Galic´, Kelly T. Scherer and James M. LeBreton
International Journal of Selection and Assessment
Volume 22 Number 4 December 2014
©2014 John Wiley & Sons Ltd
4.2. Results and discussion
The correlations given in Table 5 support our assump-
tion that the CRT-A is not significantly related to
GMA as measured with the DAT abstract reasoning
test. The CRT-A also showed a nonsignificant correla-
tion with the need for cognition but significantly cor-
related with the Cognitive Reflection Test. Considering
that this observed relationship could result from sam-
pling issues (e.g., restriction of range), we compared
our correlations with those from another study con-
taining the same three ‘cognitive’ variables (GMA,
cognitive reflection, and need for cognition). The
correlations reported by Frederick (2005) gave us con-
fidence in our findings. In his study, the correlations
between GMA (measured with the Wonderlic Person-
nel Test) and cognitive reflection, GMA and need for
cognition, and cognitive reflection and need for cogni-
tion were almost identical to those obtained in our
study (.43, .19, and .22, respectively).
The significant positive correlation between the
CRT-A score and cognitive reflection obtained in our
study is somewhat surprising (especially given the relat-
ively meager reliability of our measure of cognitive re-
flection). These results indicate that individuals who
tended to rely on justification mechanisms linked to the
motive to aggress were also less cognitively impulsive.
5. Study 4
Within the fourth study, we sought to test the assump-
tion that the CRT-A is relatively immune to faking
(LeBreton et al., 2007), at least when the indirect nature
of measurement is preserved. We compared CRT-A
data obtained from the sample of incumbents, with stu-
dent respondents who were asked to fake good, and
with actual job applicants who completed the CRT-A
within a personnel selection test battery. Considering
that incumbents anonymously responded to the CRT-A
outside of their organizations, we believed that we en-
hanced the probability of honest responding. On the
other hand, respondents faking good (i.e., those in-
structed to fill out the survey as if they were an ideal
job candidate) and responding in a personnel selection
situation should have an incentive to improve their self-
presentation.
In accordance with earlier studies (LeBreton et al.,
2007), we expected that the Croatian adaptation of the
CRT-A should show resistance to faking. However, in
order to have a standard against which we can compare
the ‘fakeability’ of the CRT-A, in the honest and ‘fake
good’ conditions, we also collected data for the self-
report angry hostility scale. Unfortunately, angry hostil-
ity data were not available in the applicant situation.
5.1. Method
5.1.1. Participants
In addition to incumbents and applicants described in
Study 1, an independent sample of participants was
asked to complete the CRT-A in a ‘fake good’ situation
(n=100). Because the number of illogical responses was
5 or higher, data from seven participants from the ‘fake
good’ condition were omitted from further analyses.
Participants in the ‘fake good’ condition were students
from a large Croatian university (gender: 83.7% female,
age: M=22.41;SD =1.42).
Therefore, the final number of respondents in Study 3
was 192 in the honest condition, 93 in ‘fake good’ con-
dition, and 147 in the personnel selection condition.
5.1.2. Instruments
In all of the conditions, participants completed the
Croatian adaptation of the CRT-A. In the honest condi-
tion and the ‘fake good’ condition, participants also
completed the IPIP angry hostility scale described in
Study 2. Internal consistencies of the CRT-A were .67
under honest condition, .65 under ‘fake good’, and .74
within selection. The reliability of the angry hostility
scale were .89 under honest responding and .90 in ‘fake
good’ situation.
5.1.3. Procedure
The procedure of data collection from honest respond-
ents and applicants was described earlier in Study 2. In
other words, participants were instructed to take the
CRT-A in accordance with the guidelines provided in
Table 5. Descriptive statistics and intercorrelations between Study 3 variables
M(SD)
Abstract
reasoning
Cognitive
reflection
Need for
cognition
CRT-A 4.86 (2.46) .13 .27** .09
Abstract reasoning 30.10(10.07) – .45** .25*
Cognitive reflection 0.50 (0.81) – – .24*
Need for cognition 57.83 (6.41)– – –
Note:*p<.05. **p<.01.
CRT-A =Conditional Reasoning Test for Aggression; M=mean; SD =standard deviation.
Conditional Reasoning Test for Aggression 351
©2014 John Wiley & Sons Ltd International Journal of Selection and Assessment
Volume 22 Number 4 December 2014
the test manual (James & McIntyre, 2000). Participants in
the ‘fake good’ condition were asked to complete the
questionnaire anonymously during a regular class ses-
sion. They received instructions to complete the sur-
veys in a survey package consisting of the CRT-A
and self-report personality questionnaires, as an ideal
candidate for a nursing job. The nursing job was se-
lected because aggressiveness (i.e., the motive to harm)
represents an undesirable trait for a vocation focused
on helping and healing others.
5.2. Results and discussion
The descriptive statistics for the CRT-A in the three
conditions and for the angry hostility scale from the
honest responding and ‘fake good’ conditions are given
in Table 6.
The analysis of variance (ANOVA) results showed
that differences in the CRT-A scores between situations
were nonsignificant, F(2, 429) =0.71,p=.49.2At the
same time, the difference between the angry hostility
scale scores in the honest responding and ‘fake good’
conditions was highly significant, t(282) =12.26;
p<.001. The size of the difference on angry hostility
scale (Cohen’s d=1.61) revealed that our instruction
was successful in creating a faking response set. The
correlation between CRT-A score and angry hostility
score was nonsignificant both under the honest re-
sponding (r=.02; p=.820) and ‘fake good’ instructions
(r=.10; p=.350).
Therefore, when the indirect nature of measurement
is preserved, the CRT-A score did not show sensitivity
to situational cues that are sometimes observed for self-
report personality measures. These results were con-
sistent with the assumption that the CRT-A represents
an indirect measure of cognitive biases.
6. General discussion
To the best of our knowledge, construct and criterion-
related validity evidence for the inferences drawn from
scores on the CRT-A has not been accumulated in a cul-
tural context other than the United States where the in-
strument was developed. Our studies were conducted
in Croatia, a post-socialistic country that, during the
1990s, went through the war for independence and a
turbulent postwar period (Tanner, 2010). Moreover, the
context in which our participants and their families de-
veloped was marked by the transition from a socialistic
planned economy to a democratic free-market society
characterized by corruption, injustice, and growing
social inequalities (Galic´ & Plec´aš, 2012; Nestic´, 2002;
Vojnic´, 1994). All of these characteristics created an in-
teresting context for using the CRT-A because in many
social situations, our participants likely experienced hos-
tile intention behind others’ behavior (e.g., during the
war), or were victimized by powerful others (e.g., cor-
rupted politicians or ‘tycoons’). Thus, selecting ‘aggress-
ive’ responses to conditional reasoning problems may
be driven in part by extended exposure to the hostile
intentions of others and being victimized by powerful
others. If this were the case, we would expect to find
limited validity evidence for the CRT-A in this setting.
However, across four studies we accumulated validity
and reliability evidence that was quite similar to that re-
ported by James and his colleagues using samples from
the United States. In Study 1, we observed a relatively
coherent factor structure that could be meaningfully
linked to the structure reported by James and LeBreton
(2012). The results of Studies 2 and 3 indicated that
scores on the CRT-A predicted CWBs and explained
variance that was unrelated to that explained using an
array of self-report personality surveys, or GMA tests.
Finally, comparison of CRT-A scores between contexts
differing in a need for positive self-presentation provided
an initial test of the indirect nature of the CRT-A in
Croatian settings and its general resistance to faking.
Our study has several limitations and related sugges-
tions for future research. We will mention three that
seem the most salient. First, the main objection some-
one could give to this study is that the data were
collected in a small European country, and, thus,
the generalizability of obtained conclusions to other
countries/populations is limited. Although this might be
true, the goal of our study was to examine if the psy-
chometric characteristics and the validity evidence for
the CRT-A would generalize to samples obtained from a
very diverse cultural context. We hope our findings en-
courage other international researchers to begin testing
whether the CRT-A could be used in their countries.
Second, the relationship between CRT-A scores and
CWBs obtained in our study are, at best, modest, and
the usefulness of the test might be seen as limited. The
zero-order correlation between the CRT-A and CWBs
was .22, which can be categorized as a small to medium
effect size (Cohen, 1988). If the psychometric properties
(i.e., reliability) of the test were improved with a longer
set of inductive problems, and separate scales developed
for specific forms of bias, and then related to specific
forms of CWBs, we could expect higher correlations
Table 6. Means and standard deviations on the Conditional
Reasoning Test for Aggression (CRT-A) and the angry hostility
scale in Study 4 situations
Trait
Honest Fake good Selection
M(SD)M(SD)M(SD)
CRT-A 4.57 (2.18) 4.59 (2.29) 4.31(2.34)
Angry hostility scale 2.71(0.71)1.62 (0.64) –
M=mean; SD =standard deviation.
352 Zvonimir Galic´, Kelly T. Scherer and James M. LeBreton
International Journal of Selection and Assessment
Volume 22 Number 4 December 2014
©2014 John Wiley & Sons Ltd
(e.g., if victimization by powerful others is related with
organizational deviance, and hostile attribution bias with
interpersonal deviance).
Finally, as first mentioned in the results section of
Study 2, the fact that we used self-report surveys to as-
sess CWBs may be seen as problematic by some re-
searchers. For example, to the extent that respondents
either engaged in socially desirable responding or simply
lacked introspective insight into their behavior, the ob-
served relationship between CWBs and CRT-A would
be attenuated. Such attenuation would be consistent
with previous research using the CRT-A. For example,
James and LeBreton (2012) reported smaller criterion-
related validities when correlating the CRT-A with sub-
jective data, especially self-report data.
On the other hand, some researchers might view
the use of self-reports of CWBs as advantageous. For
example, the most recent meta-analysis by Berry,
Carpenter, and Barratt (2012) showed that self-reports
of CWBs might generally be a preferable alternative to
other reports. Their analyses showed that self-raters re-
vealed more CWBs than their coworkers/supervisors,
and that other reports added little incremental variance
over self-reported CWBs for the most often-studied
covariates (e.g., personality traits, organizational justice).
Although our results provide tentative support for the
use of the CRT-A in a Croatian context, considering
that this conditional reasoning represents a new tech-
nology of personality conceptualization and measure-
ment, future research using additional criteria for
aggressive behavior is certainly warranted.
Notes
1. The substantive conclusions concerning the incremental
prediction of the CRT-A did not change when a single re-
gression analysis was conducted containing all six of the
self-report surveys in Step 2.
2. Considering that groups differed in gender and age com-
position, and that age on the total sample was correlated
with the CRT-A score, we also conducted analyses of co-
variance where age and gender were used as covariates.
Substantive conclusions were the same; thus, we reported
the more parsimonious ANOVA results.
References
Banks, G. C., Kepes, S., & McDaniel, M. A. (2012). Publication
bias: A call for improved meta-analytic practice in the
organizational sciences. International Journal of Selection and
Assessment, 20, 182–196.
Bennet, G. K., Seashore, H. G., & Wessman, A. G. (1990). Dif-
ferential aptitude test for selection – general abilities battery
(GAB): Manual. New York: The Psychological Corporation.
Berry, C. M., Carpenter, N. C., & Barratt, C. L. (2012). Do
other-reports of counterproductive work behavior provide
an incremental contribution over self-reports? A meta-
analytic comparison. Journal of Applied Psychology, 97, 613–
636.
Berry, C. M., Ones, D. S., & Sackett, P. R. (2007). Interper-
sonal deviance, organizational deviance, and their common
correlates: A review and meta-analysis. Journal of Applied Psy-
chology, 92, 410–424.
Berry, C. M., Sackett, P. R., & Tobares, V. (2010). A meta-
analysis of conditional reasoning test of aggression. Personnel
Psychology, 63, 361–384.
Bing, M. N., LeBreton, J. M., Davison, H. K., Migetz, D. Z., &
James, L. R. (2007). Integrating implicit and explicit social
cognitions for enhanced personality assessment: A general
framework for choosing measurement and statistical meth-
ods. Organizational Research Methods,10, 346–389.
Bornstein, R. F. (1999). Criterion validity of objective and pro-
jective dependency tests: A meta-analytic assessment of be-
havioral prediction. Psychological Assessment,11, 48–57.
Bornstein, R. F. (2002). A process dissociation approach to
objective-projective test score interrelationships. Journal of
Personality Assessment, 78, 47–68.
Cacioppo, J. T., Petty, R. E., & Kao, C. F. (1984). The efficient
assessment of need for cognition. Journal of Personality As-
sessment, 48, 306–307.
Cohen, J. (1988). Statistical power analysis for the behavioral sci-
ences. Mahwah, NJ: Lawrence Erlbaum.
Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied
multiple regression/correlation analysis for the behavioral sciences
(3rd ed.) Mahwah, NJ: Lawrence Erlbaum Associates.
Douglas, S. C., & Martinko, M. J. (2001). Exploring the role of
individual differences in the prediction of workplace aggres-
sion. Journal of Applied Psychology, 86, 547–559.
Frederick, S. (2005). Cognitive reflection and decision making.
Journal of Economic Perspectives,19, 25–42.
Frost, B. C., Ko, C. E., & James, L. R. (2007). Implicit and ex-
plicit personality: A test of a channeling hypothesis for ag-
gressive behavior. Journal of Applied Psychology, 92, 1299–
1319.
Galic
´, Z., & Plec´aš, M. (2012). Quality of working life during
the recession: The case of Croatia. Croatian Economic Survey,
14, 5–41.
Geisinger, K. F. (1994). Cross-cultural normative assessment:
Translation and adaptation issues influencing the normative
interpretation of assessment instruments. Psychological As-
sessment, 6, 304–312.
Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton,
M. C., Cloninger, C. R., & Gough, H. G. (2006). The interna-
tional personality item pool and the future of public-domain
personality measures. Journal of Research in Personality, 40,
84–96.
Gulliksen, H. (1950). Theory of mental tests. New York: Wiley.
Haidt, J. (2001). The emotional dog and its rational tail: A so-
cial intuitionist approach to moral judgment. Psychological Re-
view,108, 814–834.
Hershcovics, M. S., Turner, N., Barling, J., Arnold, K. A.,
Dupré, K. E., Inness, M., et al. (2007). Predicting workplace
aggression: A meta-analysis. Journal of Applied Psychology, 92,
228–238.
Hofstede, G. H. (2001). Culture’s consequences: Comparing val-
ues, behaviors, institutions and organizations across nations.
Thousand Oaks, CA: Sage.
Conditional Reasoning Test for Aggression 353
©2014 John Wiley & Sons Ltd International Journal of Selection and Assessment
Volume 22 Number 4 December 2014
Hui, C. H., & Triandis, H. C. (1985). Measurement in cross-
cultural psychology: A review and comparison of strategies.
Journal of Cross-Cultural Psychology,16, 131–152.
James, L. R., & LeBreton, J. M. (2010). Assessing aggression
using conditional reasoning. Current Directions in Psychological
Science,19, 30–35.
James, L. R., & LeBreton, J. M. (2012). Assessing the implicit per-
sonality through conditional reasoning (1st ed.) Washington,
DC: American Psychological Association.
James, L. R., & McIntyre, M. D. (2000). Conditional reasoning test
of aggression: Test manual. Knoxville, TN: Innovative Assess-
ment Technology.
James, L. R., McIntyre, M. D., Glisson, C. A., Green, P. D.,
Patton, T. W., LeBreton, J. M., & Mitchell, T. R. (2005). A
conditional reasoning measure for aggression. Organizational
Research Methods, 8, 69–99.
Joreskog, K. G., & Sorbom, D. (2005). Lisrel 8.72. Chicago: Sci-
entific Software International.
LeBreton, J. M., Barksdale, C. D., Robin, J., & James, L. R.
(2007). Measurement issues associated with conditional
reasoning tests: Indirect measurement and test faking. Jour-
nal of Applied Psychology, 92, 1–16.
Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental
test scores. Reading, MA: Addison-Wesley.
McClelland, D. C., Koestner, R., & Weinberger, J. (1989). How
do self-attributed and implicit motives differ? Psychological
Review, 96, 690–702.
Morgeson, F. P., Campion, M. A., Dipboye, R. L., Hollenbeck,
J. R., Murphy, K., & Schmitt, N. (2007). Reconsidering the
use of personality tests in personnel selection contexts. Per-
sonnel Psychology, 60, 683–729.
Nestic´, D. (2002). Ekonomske nejednakosti u Hrvatskoj 1973–
1998. (Economic inequalities in Croatia 1978–1998).
Financijska Teorija I Praksa, 26, 595–613.
Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we
can know: Verbal reports on mental processes. Psychological
Review, 84, 231–259.
Olsson, U., Drasgow, F., & Dorans, N. J. (1982). The polyserial
correlation coefficient. Psychometrika, 47, 337–347.
R Development Core Team. (2010). R: A language and environ-
ment for statistical computing. Available at http://www.R-
project.org (accessed 22 September 2012).
Rothstein, M. G., & Goffin, R. D. (2006). The use of personal-
ity measures in personnel selection: What does current re-
search support? Human Resource Management Review,16,
155–180.
Spangler, W. D. (1992). Validity of questionnaire and TAT
measures of need for achievement: Two meta-analyses. Psy-
chological Bulletin,112, 140–154.
Spector, P. E., Fox, S., Penney, L. M., Bruursema, K., Goh, A., &
Kessler, S. (2006). The dimensionality of counter-
productivity: Are all counterproductive behaviors created
equal? Journal of Vocational Behavior, 68, 446–460.
Tanner, M. (2010). Croatia: A nation forged in war (3rd ed.) Yale:
University Press.
Vojnic´, D. (1994). European integration processes and the
countries in transition – with special reference to Croatia
and former Yugoslavia. Ekonomski Pregled,9–10, 203–239.
Wanous, J. P., Sullivan, S. E., & Malinak, J. (1989). The role of
judgment calls in meta-analysis. Journal of Applied Psychology,
74, 259–264.
Winter, D. G., John, O. P., Stewart, A. J., Klohnen, E. C., &
Duncan, L. E. (1998). Traits and motives: Toward an integ-
ration of two traditions in personality research. Psycholo-
gical Review,105, 230–250.
Wu, J., & LeBreton, J. M. (2011). Reconsidering the disposi-
tional basis of counterproductive work behavior: The role
of aberrant personality. Personnel Psychology, 64, 593–626.
354 Zvonimir Galic´, Kelly T. Scherer and James M. LeBreton
International Journal of Selection and Assessment
Volume 22 Number 4 December 2014
©2014 John Wiley & Sons Ltd
