Improving Feedback Reports: The Role of Procedural Information and Information Specificity

Abstract

We investigated the effects of varying two types of information in feedback reports on feedback reactions in the context of managerial skill development. We found that favorable reactions increased when a high amount of procedural information was given. Furthermore, unfavorable reactions diminished when participants received low specific information. Fifteen months after the assessment of feedback reactions, we also measured students' self-reported involvement in developmental activities and found a significant and positive relationship between favorable feedback reactions and developmental activities. These results provide useful suggestions for management educators to enhance feedback reactions in managerial skill development.

Full text

1
IMPROVING FEEDBACK REPORTS: THE ROLE OF PROCEDURAL
INFORMATION AND INFORMATION SPECIFICITY
ABSTRACT
We investigated the effects of varying two types of information in feedback reports on
feedback reactions in the context of managerial skill development. We found that favorable
reactions increased when a high amount of procedural information was given. Furthermore,
unfavorable reactions diminished when participants received low specific information. Fifteen
months after the assessment of feedback reactions, we also measured students’ self-reported
involvement in developmental activities 15 months after receiving feedback and found a
significant and positive relationship between favorable feedback reactions and developmental
activities. These results provide useful suggestions for management educators to enhance
feedback reactions in managerial skill development.
Keywords: Procedural information, information specificity, feedback reports, feedback
reactions, managerial skill development

2
IMPROVING FEEDBACK REPORTS: THE ROLE OF PROCEDURAL
INFORMATION AND INFORMATION SPECIFICITY
A key challenge for educators is increasing learners’ awareness of developmental
needs to create a strong commitment to future developmental activities. A recent meta-
analysis of self-assessments of knowledge in education and workplace training showed that
learners’ self-assessments correlated only moderately with actual cognitive learning,
suggesting that self-awareness of developmental needs and progress remains a potential
biasing factor in management education (Sitzmann, Ely, Brown, & Bauer, 2010).
Interestingly, results further showed that self-awareness was considerably higher in education
programs that provided external feedback to participants. This highlights the need for
management education programs to include powerful feedback interventions that are designed
to maximally increase self-awareness and developmental commitment in participants (e.g.,
Brutus & Donia, in press; Sitzmann et al., 2010; Van Fleet, Peterson, & Van Fleet, 2005).
A large body of evidence suggests that one of the key factors to focus on when
designing feedback interventions is how feedback recipients initially react to the feedback
provided (e.g., Ilgen, Fisher, & Taylor, 1979; Ryan, Brutus, Greguras, & Hakel, 2000). When
people feel good about the feedback they receive, they will be more open to act upon the
feedback and engage more in future developmental activities than when they feel unhappy
about the feedback. Thus, it seems crucial for research to develop and gain insight into
practical strategies that management educators can use during skill development to influence
how feedback recipients react to feedback (Kluger & DeNisi, 1996; London & Smither,
1995).
One way to improve outcomes following feedback is by varying the type and amount
of information provided in feedback reports (Brutus, 2009; Goodman & Wood, 2004a;

3
Smither & Walker, 2004). Given the widespread practice of providing learners with
customized feedback reports both in electronic and paper format (e.g., Brutus, 2009), a better
understanding of the effects of different types and amounts of information in these feedback
reports might lead to cost-effective and practical strategies to enhance feedback interventions
in management education.
The overall purpose of this study was, therefore, to investigate the causal effects of
different types of feedback reports on recipients’ reactions to feedback. We argue that two
types of information are crucial in a feedback report for determining reactions to feedback. A
first type is information about the procedures used to generate feedback. Previous survey
research suggests that perceptions of and knowledge about feedback procedures may be
associated with initial reactions to feedback (e.g., Jawahar, 2007; Leung, Su, & Morris, 2001).
A second type is information specificity. Some studies (e.g., Goodman & Wood, 2004a,
2004b) have shown that the specificity of feedback information provided in feedback reports
impacts on subsequent task performance. We argue that the specificity of feedback
information may also be helpful in understanding initial reactions to feedback messages.
Therefore, in this study, a first objective is to examine the moderating effects of different
levels of procedural information and information specificity on the relationship between
feedback scores and favorable and unfavorable feedback reactions. By means of a field
experiment, we aim to offer a better understanding of the causal effects of two information
types in strengthening or weakening feedback reactions after feedback. As a second objective,
we aim to examine the relationship between these initial feedback reactions and self-reported
involvement in development activities 15 months later. If management educators are to be
encouraged to focus on and enhance learners’ initial reactions to feedback, it is important to
demonstrate that these reactions are indeed predictive of future developmental behaviors. As a

4
guiding framework, an overview of all hypothesized relationships in this study are depicted in
Figure 1.
Insert Figure 1 about here
DETERMINANTS OF FEEDBACK REACTIONS
Feedback Sign
The determinant that has received most empirical support in feedback reactions
research is the valence of feedback or feedback sign. Several studies have found that feedback
recipients are more likely to react favorably to positive feedback than to negative feedback
(Anseel & Lievens, 2006; Atwater & Brett, 2005; Brett & Atwater, 2001; Illies, De Pater, &
Judge, 2006; Love, Love, & Northcraft, 2010). Paradoxically, this means that people who do
not perform up to the required standards, will most likely receive negative feedback and react
unfavorably to it. Thus, managers who need feedback the most in order to develop, are the
ones who are most likely to react unfavorably. Explanations for these effects posit that
feedback that is consistent with individuals’ existing image of themselves, which is usually
the case with positive information, will be processed uncritically and lead to positive
emotions (e.g., Mitchell & Beach, 1990). Feedback that is inconsistent with the existing
image on the other hand, which is usually the case with negative information, will not be
easily accepted and lead to negative emotions. Given the dominance of feedback sign on
learners’ reactions, studies of feedback reactions typically start from this basic relationship
and then explore potentially influencing factors (e.g., Anseel, Lievens, & Schollaert, 2009;
Atwater & Brett, 2005). In line with this approach, we will examine whether these main
relationships, as represented in Hypotheses 1a and 1b (Figure 1), are moderated by procedural
information and information specificity, and whether the two outcome variables, favorable

5
and unfavorable feedback reactions, have an impact on involvement in skill development.
Note that we conceptualized feedback reactions as two separate outcomes. Research in the
emotion domain has shown that emotional reactions are best not seen as one-dimensional, but
rather as two- or even multi-dimensional (Fontaine, Scherer, Roesch, & Ellsworth, 2007). In
line with Atwater and Brett (2006), we thus distinguished favorable (e.g., happy, motivated)
from unfavorable (e.g., worried, angry) feedback reactions. Conceptually, feedback reactions
are assumed to be driven by learners’ immediate affective response to the feedback message
(i.c. satisfaction with feedback) more than their immediate affective evaluation of their own
performance (i.c. satisfaction with performance) (Swann & Schroeder, 1995).
H1a. There will be a positive relationship between feedback score and favorable
feedback reactions with higher scores leading to more favorable reactions.
H1b. There will be a negative relationship between feedback score and unfavorable
feedback reactions with lower scores leading to more unfavorable reactions.
Type and Amount of Information
Ilgen and Davis (2000) suggested that the way in which (negative) feedback is framed
and delivered may influence how recipients respond to the feedback. In response to this call,
researchers have recently started to investigate the effects of type, amount and specificity of
the information provided in feedback messages on performance and other feedback-related
outcomes. For instance, the use of numeric, normative, or text feedback (Atwater & Brett,
2006), the specificity of the feedback presented (Goodman & Wood, 2004b), the amount of
comments and whether they contain behavior- or task-focused information (Smither &
Walker, 2004) and whether feedback is precise (Brutus, 2009; Brutus & Facteau, 2003) are all
characteristics that have been found to be important in determining outcomes of feedback and
performance improvement. In a recent review, Brutus (2009) cogently concluded that “the

6
format in which feedback is presented probably matters a great deal because it is so
intimately linked fundamental elements of the evaluation and communication of performance”
(Brutus, 2009: 11).
To our knowledge, only one study so far has investigated the reactions recipients
experience following feedback delivered in different feedback formats. The authors of this
study found that recipients’ reactions were more favorable after they had read numeric and
normative feedback in contrast to text feedback (Atwater & Brett, 2006). This study provides
preliminary evidence for our argument that variations in information characteristics presented
in feedback messages can indeed shape recipients’ reactions following feedback.
Procedural Information
A first important factor is information about the procedures used to determine a
feedback score. Ilgen and Davis (2000) suggested that one possible cause for unfavorable
reactions may be the attributions people make when receiving negative information. People
will generally attribute positive feedback to internal controllable factors, whereas negative
feedback will be mostly attributed to causes the individual has no control over. Individuals
will thus only use negative feedback for development if they believe they can exert control
over these behaviors, and if they are aware of the ways in which this feedback was gathered.
Two cross-sectional studies have shown that people react more favorably to feedback
messages if they report that they have insight into the procedures used (Jawahar, 2007; Leung
et al., 2001). In other words, knowledge about how the information is gathered has an
important influence on and can possibly determine how people react to feedback.
However, to date, most studies have measured people’s perceptions rather than
actively varied the amount of information (Cohen-Charash & Spector, 2001; Colquitt,
Conlon, Wesson, Porter, & Ng, 2001) making it difficult to draw strong conclusions. The

7
present study extends this line of research by experimentally varying the amount of
information participants receive in a personal feedback report. On the one hand, we expect that
participants with a high feedback score will react more favorably and that receiving information
about the procedures will enhance this positive effect. On the other hand, participants who
receive a low feedback score will react more unfavorably, but receiving procedural information
might diminish these unfavorable reactions.
Hypothesis 2a. The positive relationship between feedback score and favorable
feedback reactions will be moderated by procedural information. The positive
relationship will be more pronounced for feedback recipients in the high procedural
information group than for recipients in the low procedural information group.
Hypothesis 2b. The negative relationship between feedback score and unfavorable
feedback reactions will be moderated by procedural information. The negative
relationship will be more pronounced for feedback recipients in the low procedural
information group than for recipients in the high procedural information group.
Information Specificity
Previous research further suggests that the specificity of the information provided in
feedback messages can also shape reactions to feedback. Feedback specificity refers to the
level of detail presented in feedback information messages. For instance, Goodman and Wood
(2004a, 2004b) investigated the effects of specific feedback on learning, learning
opportunities and exploration in two studies. In a lab environment, they found that increasing
the specificity of feedback positively affected practice performance, although these benefits
did not endure over time and depended on what was to be learned.
To date, research concerning the specificity of feedback has mainly focused on the
effects of specific feedback on performance in a lab environment. A question that gained little

8
attention is whether the augmentation of performance feedback has an impact on learners’
reactions in the field. For recipients to believe they can actively change their performance and
learn from feedback, causes for poor performance should be attributed to factors over which
the actor has some control. We argue that if people feel they have control over their own
performance, they will react more favorably to feedback, even if this feedback is negative.
One way to facilitate the formation of such attributions is to convey to the recipient why the
feedback provided is negative and to clarify that the key for development is in their own
hands. When people receive negative feedback underpinned by more in-depth comments
detailing exactly why their feedback was negative, these comments supply them with valuable
and usable information as to how to improve their own performance. Hence, as feedback
information specificity increases, so does its capability to perform its informational role
(Goodman & Wood, 2004b). Thus, as feedback recipients receive more specific feedback, we
expect that they will see more value in the feedback and that they will attribute the causes for
their performance more to controllable factors, leading to more favorable reactions.
Hypothesis 3a. The positive relationship between feedback score and favorable
feedback reactions will be moderated by information specificity. The positive
relationship will be more pronounced for feedback recipients in the high specific
information group than for recipients in the low or moderate specific information
group.
Hypothesis 3b. The negative relationship between feedback score and unfavorable
feedback reactions will be moderated by information specificity. This negative
relationship will be more pronounced for feedback recipients in the low or moderate
specific information group than for recipients in the high specific information group.
Involvement in Developmental Activities

9
It is important to demonstrate that favorable feedback reactions are related to actual
involvement in developmental activities. Although the assumption that favorable reactions
automatically lead learners to engage in training and development activities (Kudisch, Ladd,
& Dobbins, 1997; Smither, London, & Richmond, 2005) seems intuitively appealing, more
empirical support is needed. As Bono and Colbert (2005) recently reported, satisfaction with
feedback does not necessarily lead to commitment to one's development goals. Furthermore, a
meta-analysis on the correlations among training criteria also revealed that affective reactions
to training interventions do not correlate with actual learning or behavior change (Alliger,
Tannenbaum, Bennett, Traver, & Shotland, 1997).
Drawing from self-efficacy theory, we argue that favorable feedback reactions induce
heightened self-efficacy (feeling good about oneself), which will in turn lead to more
involvement in activities where one can potentially receive more positive feedback about
oneself. In general, a situation that creates an environment supportive of learning and
development should help to enhance both self-confidence for development and beliefs that
favorable outcomes will result from that supported behavior (cf. Baldwin & Magjuka, 1997;
Mathieu & Martineau, 1997; Maurer, 2001). This self-efficacy for development should
subsequently be positively related to attitudes toward development activities. Research has
further shown that self-efficacy is a key predictor of choosing to perform a behavior or
pursuing a task as well as of persistence, thoughts, and feelings during the task (Bandura,
1997; Gist & Mitchell, 1992; Sadri & Robertson, 1993). Maurer, Weiss and Barbeite (2003)
provided empirical support for the theoretical link between affective/motivational constructs
such as favorable attitudes and behavioral outcomes such as participating and engaging in
development activities.

10
In the current study, we focused on self-reported involvement in developmental
activities 15 months after receiving feedback. Previous studies have shown that self-reported
involvement in development activities is highly correlated with objective measures of
involvement in developmental activities (e.g., estimates by organizational representatives;
Zoogah, 2010). Finally, a review of the literature shows that development behaviors are
crucial for organizations as they facilitate achievement of individual (performance,
compensation, and careers; Hall, 1996; Kozlowski & Farr, 1988; London & Smither, 1999;
Noe, 1996) and organizational (productivity and return on investment; Maurer et al., 2003;
Tharenou, Saks, & Moore, 2007) outcomes. Therefore, we propose the following hypotheses:
Hypothesis 4a. There will be a positive relationship between favorable feedback
reactions and involvement in developmental activities.
Hypothesis 4b. There will be a negative relationship between unfavorable feedback
reactions and involvement in developmental activities.
Control Variables
When introducing new interventions, it is important to show that they add value above
and beyond what is already known. Therefore, we controlled for positive affect, learning goal
orientation and core self-evaluations. First, positive affectivity refers to a relatively stable
dispositional tendency for people to feel generally enthusiastic, active, and alert (Judge &
Larsen, 2001; Watson, Clark, & Tellegen, 1988). Several studies found that people’s affective
disposition may have an impact on how they respond to performance feedback (Forgas &
George, 2001). Hammer and Stone-Romero (1996) reported that feedback was perceived as
more accurate when recipients’ affective dispositions were consistent with the
(un)favorability of the feedback. Other authors (Trope, Ferguson, & Ragunanthan, 2001) also

11
showed that positive affectivity helps people to deal more effectively with the negative
feedback they receive from others, by functioning as a psychological resource.
Second, research has shown that goal orientations influence how individuals interpret
feedback and react to it (e.g., Payne, Youngcourt, & Beaubien, 2007). Individuals with high
levels of learning goal orientation are inclined to seek feedback (e.g., Payne et al., 2007;
Vandewalle & Cummings, 1997), to interpret feedback as useful and positive (e.g., Farr,
Hofmann, & Ringenbach, 1993), and to react negatively when receiving unfavorable feedback
(Vandewalle, Cron, & Slocum, 2001).
Third, core self-evaluations are described as a broad, latent, higher-order trait
indicated by three well-established traits in the personality literature, namely global self-
esteem, trait- based self-efficacy and emotional stability/adjustment (Judge, Erez, Bono, &
Thoresen, 2003). There is consistent research evidence that traits, such as self-esteem, affect
how individuals respond to negative feedback (e.g., Brockner, Derr, & Laing, 1987; Ilgen et
al., 1979). Furthermore, a recent study showed that more positive core self-evaluations are
associated with higher satisfaction and stronger goal commitment after receiving feedback
(Kamer & Annen, 2010).
METHOD
Participants and Procedure
The sample consisted of final year master students (N = 274) from different
backgrounds (e.g., engineering, pharmaceutical sciences, economics, agricultural and plant
sciences, educational sciences) from a large public university in Belgium enrolled in a class
on managerial skill development. The sample consisted mostly of women (66.4%) and the
participant’s ages ranged from 21 to 48 years (M = 22.22).

12
Students attended a series of workshops on managerial skills (e.g., communication,
feedback giving, negotiating, meeting, decision making and teamwork) over the course of six
months. Given that this was a new course and we had no prior strong arguments regarding the
information that needed to be provided in the feedback reports, this course offered an ideal
opportunity to vary and test different types and amounts of information in an educational field
setting focused on management skill development. All skill workshops were led by trained
Psychology graduate students. For trainees, enrolment in the courses on managerial skills was
voluntary and they received course credit for participation. Before enrolment in this course,
all students were informed about the goals of this optional course and the importance of their
motivation for managerial skill development. Students interested in following the course were
hence aware that the workshops would enable them to better prepare themselves for their
future careers as junior managers. As participation was voluntary, people who chose to follow
this course were genuinely interested in feedback about their managerial skill development
and were likely to pursue a management career after graduation. We assessed this by using
one item that asked participants what organizational position they aspired to after their
graduation approximately six months after the course. The responses showed that 87.5% of
the participants aspired to a managerial or executive function. At the start of the course, three
months prior to the workshops (Time 1), all participants completed a number of online
questionnaires assessing their teamwork and leadership styles for use in the actual training
program. All six workshops consisted of practice exercises and role-plays, and during all of
them, participants were closely observed by trained observers who rated their relevant
behaviors. The observer training consisted of an intensive workshop in which a group of 15 to
20 students were instructed on how to use the checklists and how these were developed. They
were also given numerous behavioral examples for all six skills in order to create a sense of

13
concordance among the observers during the workshops they attended and observed. We
developed behavioral checklists for each workshop so the observers could easily indicate the
frequency of behavior displayed by the participants. Each checklist consisted of four items
that were developed based on the behaviors that were typically elicited during the role-plays.
The observers were asked to give each participant a score from 1 (= totally disagree) to 5 (=
totally agree) on each item, and to indicate which overall score (1 = extremely weak to 5 =
excellent) they would assign this participant on the particular managerial skill that was dealt
with in the observed workshop. After they had attended all six workshops, participants
received an e-mail with a personal feedback report (Time 2) with varying types and amount of
information (see below) and several questions to assess their reactions about the feedback.
Participants were asked to send back this feedback questionnaire one week after they had
received their feedback report (Time 3). Finally, approximately 15 months after Time 3, all
participants were contacted again and were asked to complete a questionnaire measuring their
involvement in developmental activities over the year following the course (Time 4).
Design
The between-persons study design consisted of two levels of procedural information
(high versus low procedural information) and three levels of information specificity (high
versus moderate versus low information specificity). Subjects were randomly assigned to the
different groups, and group frequencies ranged from 20 to 25.
Procedural information. Participants in the high procedural information group
received detailed information about the different raters and the rating process that was used
for their personal feedback scores. This information was given in the e-mail they received as
well as on the first page of their feedback report. For instance, the report read: “… several
trained observers (all Psychology students) observed you during all six workshops. These

14
observers used newly developed behavioral checklists for each workshop on which they were
asked to indicate the frequency of behavior displayed by you during the exercises. You were
thus evaluated on 24 different items during the course of the seminar.” Participants in the low
procedural information group were told that all scores were out of a maximum of five, and did
not receive any other information.
Information specificity. Following Goodman and Wood (2004a), participants were
randomly assigned to one of three information specificity groups (low, moderate, or high
specific information). Similar to Goodman and Wood (2004a), in the low information
specificity group, participants received outcome feedback only in the form of a quantitative
performance feedback score with a brief explanation for each score that gave them the
opportunity to quickly assess how they performed during the workshops (e.g., “You received
a high score on ‘teamwork’, which indicates that you continuously cooperated with your
teammates in an efficient manner during the teambuilding exercise. This means that you
succeeded in working constructively on a common goal, and that you actively contributed to
the team achievement.”). We chose this group as the baseline information specificity group to
see what effects the adding of information generated. In the moderate information specificity
group, participants received the same outcome feedback as participants in the low information
specificity group, but also standardized diagnostic feedback. This means that, consistent with
Goodman and Wood (2004a), we provided them with brief and standardized information on
how they could perform better in a future situation (e.g., “You received a low score on
‘negotiating’. This means that you did not follow the rules for negotiating that are appropriate
when negotiating with another party. People like you who achieved a low score on
‘negotiating’, are not yet capable to put into practice all the different aspects that are typical of
an efficient negotiation. In the future, when negotiating, you should for instance try to strive

15
for win-win solutions so that both parties are satisfied with the achieved results, you should
respect the other party and be assertive when trying to explain the priorities of you and your
party.”). For each workshop on managerial skills, participants were provided with
standardized information about the things they typically did right and wrong during this
particular workshop. All participants in these groups thus received the same additional
information. Finally, in the high information specificity group, participants received the same
outcome feedback that participants in the other groups received, supplemented by more
specific feedback about how they behaved in the workshops with specific behavioral
observations about the things they actually did right or wrong. In this group, the feedback
message thus included actual observations of behavior displayed during the workshops and
observed by the raters (e.g., “Apart from the high score you received on teamwork,
observations also showed that you master this skill to great extent. It was noticed for instance
that you actively helped the first team member that had to complete the exercise, and that you
encouraged her when she was scared to go through the construction the team built. You also
offered to hold the frame that held the construction so that this wouldn’t collapse during the
exercise. Finally, you continually encouraged your team members during the course of the
exercises, and helped them when necessary”). As the observed behaviors were idiosyncratic
for each management trainee in the high information specificity group, their feedback reports
did not contain exactly the same information. However, we believe this practice of providing
actual examples of behavior corresponds most closely with organizational feedback practices
in management education.
Measures
Control variables (Time 1). Positive affect was assessed by 10 items that are part of
the Positive and Negative Affect Scale (PANAS; Watson et al., 1988). This scale consists of a

16
number of words that describe different feelings and emotions. Respondents were asked to
respond to these items using a 5-point Likert-type scale ranging from 1 (very slightly or not at
all) to 5 (extremely). Sample words are “Interested”, “Strong”, “Active” and “Proud”. Internal
consistency of this positive affect scale was .76. Learning goal orientation was assessed by
four items developed by Vandewalle et al. (2001). Sample items are “I often look for
opportunities to develop new skills and knowledge” and “I enjoy challenging and difficult
tasks at work where I’ll learn new skills”. Participants responded to these items on a 7-point
Likert-type scale ranging from 1 (= totally disagree) to 7 (= totally agree). Internal
consistency of this learning goal orientation scale was .83. Core self-evaluations were
assessed by 12 items developed by Judge et al. (2003). Sample items are “I complete tasks
successfully", “I determine what will happen in my life” and “When I try, I generally
succeed”. Participants were asked to respond to these items on a 5-point Likert-type scale
ranging from 1 (= strongly disagree) to 5 (= strongly agree). Internal consistency of this core
self-evaluations scale was .84.
Feedback score (Time 2). Participants received feedback scores for each managerial
skill they had trained during the workshops, namely communication, dealing with feedback,
meeting, negotiating, decision making and teamwork. The highest possible overall score
participants could receive for each managerial skill was 5 (= excellent), an average score was
3 (= sufficient), and the lowest possible feedback score was 1 (= extremely weak). Because
participants were asked to describe their overall reactions towards the feedback they received,
and not towards each feedback score separately, these different feedback scores were
aggregated into one overall score for all workshops.
Feedback reactions (Time 3). Favorable and unfavorable reactions to feedback were
measured using a scale developed by Atwater and Brett (2005). After reading their personal

17
feedback report, participants were asked to indicate their reactions to feedback. For each of
the 24 reactions, recipients indicated on a 5-point scale the extent to which they feel this way
now, with 1 = not at all and 5 = extremely. We factor analyzed the 24 reactions and two clear
factors emerged. They represented favorable reactions on the one hand and unfavorable
reactions on the other. The reactions that were part of Atwater and Brett’s (2005) motivation
factor all loaded on the factor favorable reactions in our sample. Favorable reactions included
“pleased”, “proud”, “happy”, and “informed”. The alpha for this scale was .86. The
unfavorable reaction scale included “angry”, “frustrated”, “unhappy”, and “disappointed”.
The alpha for this scale was .73. Items were averaged to create scores for each of the two
scales for each participant.
Manipulation checks and coding qualitative material (Time 3). After reading their
feedback report, participants were asked, “After reading your feedback report, what is the first
thing you think about? Please write down as much as you can and want”. To check the
procedural information and information specificity manipulations, we relied on these
qualitative comments and examined whether there was a difference in the content and amount
of comments made by participants regarding procedural information and feedback specificity.
Two independent raters coded all comments made by participants on these two aspects on the
two independent variables. Detailed coding rules are available from the authors.
From an exploratory perspective, we also coded qualitative comments on perceptions
of (dis)satisfaction with feedback and (dis)satisfaction with performance. Again, two
independent coders rated all comments made by participants on the same question as
described earlier. All comments were coded using a bipolar coding scale ranging from 0 to 2.
Detailed coding rules for these two variables can be found in Appendix A. We calculated

18
Cohen’s kappa for the concordance of the coded data (1960). Inter-rater agreement was .64
for satisfaction with performance and .95 for satisfaction with feedback.
Involvement in skill development activities (Time 4). We used a developmental
activity scale developed by Smither et al. (2005) to measure participants’ involvement in skill
development activities 15 months after receiving feedback. This scale consists of 14 items
measuring to what extent participants used their feedback for further development. We
adapted the items to the specific context of the current study. Respondents were asked to
indicate to what extent they had engaged in certain behaviors during the year following the
course. Participants responded to these items using a 5-point Likert-type scale ranging from 1
(never) to 5 (regularly). Sample items are “To what extent did you look for additional
information to further improve your skills?”, “To what extent did you look for situations in
which you could practice the skills?” and “To what extent did you try to apply your new-
found knowledge in your job or study?”. Internal consistency of this involvement in skill
development scale was .88. To test for attrition effects, we compared feedback reaction scores
and involvement in skill development scores of those who participated in this follow-up to the
scores of those who dropped out. With regard to Time 2 feedback scores and Time 3 feedback
reactions, no mean differences and thus no selectivity effects were found between continuers
and dropouts.
RESULTS
Descriptive statistics, correlations, and internal consistency reliabilities for all
measured variables are presented in Table 1. In all hierarchical regressions testing our
hypotheses, we controlled for gender, learning goal orientation, positive affect and core self-
evaluations in the first step. For Hypotheses 2a, 2b, 3a and 3b, we controlled for feedback
scores and the main effect of the information manipulations (dummy coded) in the second

19
step of our analyses. To enhance interpretation, we mean-centered feedback score variables
prior to computing cross- product terms (Aiken & West, 1991).
Insert Table 1 about here
Manipulation Checks and Preliminary Analyses
Examination of the manipulation checks suggested that participants were sensitive to
both the procedural information and information sensitivity manipulations. An analysis of
variance (ANOVA) was conducted for each manipulation check variable. Feedback score was
entered as a control variable in all analyses. First, the effect of procedural information on the
amount of comments made by participants about their knowledge of the observation process
and observers, was significant, F(1,271) = 15.99, p < .001, ŋ² = .06. The mean ratings differed
significantly from one another in the expected order (low M = .20, SD = .55; high M = .56, SD
= .88). These mean ratings show that participants in the low procedural information condition
reported to know less about the procedures and observers compared to participants in the high
procedural information condition. Second, the effect of information specificity on the amount
of comments made by participants about the specificity of feedback was significant, F (2,270)
= 5.72, p < .01, ŋ² = .04. Here as well, mean ratings differed significantly from one another in
the expected order (low M = .20, SD = .45; moderate M = .33, SD = .52; high M = .50, SD =
.82). These means show that participants in the low information specificity condition made
significantly less comments concerning the specificity of information or the uniqueness of
their feedback compared to participants in the moderate and high information specificity
conditions. The results show that both manipulations in our study had the desired effect, and
that participants perceived the procedural information and information specificity in the
intended manner.

20
To provide some preliminary evidence for the construct validity of the feedback
reaction measure, we first tested the assumption that feedback reactions were driven by
respondents’ satisfaction with feedback instead of their satisfaction with their performance.
As can be seen in Table 1, correlational analysis showed that satisfaction with feedback was
positively related to favorable feedback reactions (r = .37, p = < .01) and negatively related
to unfavorable feedback reactions (r = -.24, p = < .01). We did not find significant
correlations between satisfaction with performance and both feedback reactions. Next, we
analyzed these data with regression analysis to see whether the relationship remained
significant when controlling for gender, positive affect, learning goal orientation and feedback
score. These analyses showed that satisfaction with feedback explained 6% of the variance in
favorable feedback reactions (∆R² = .06, F(1,245) = 17.32, p < .001) (β = .23, p < .001) and
2% of the variance in unfavorable feedback reactions (∆R² = .02, F(1,240) = 5.87, p < .05) (β
= -.07, p < .05). No effects were observed for satisfaction with performance. These results
indicate that the reactions to feedback were caused by the respondents’ (dis)satisfaction with
the feedback, rather than their (dis)satisfaction with their own performance.
Hypotheses 1a and 1b
As can be seen in Table 2 (1st part), the effect of feedback score on favorable reactions
was significant (β = .64, p < .001) and explained 14% of the variance in favorable feedback
reactions (∆R² = .14, F(1,244) = 41.74, p < .001). Thus, Hypothesis 1a was supported. As can
be seen in Table 2 (2nd part), the effect of feedback score on unfavorable feedback reactions
was significant (β = -.33, p < .001), and explained 14% of the variance in unfavorable
feedback reactions (∆R² = .14, F(1,239) = 42.25, p < .001). Hypothesis 1b was also supported.
Insert Table 2 about here

21
Hypotheses 2a and 2b
Quantitative analyses. We entered the interactive term between feedback scores and
procedural information in the third step to test Hypotheses 2a and 2b. As can be seen in Table
3 (1st part), we found a positive interaction between feedback score and procedural
information on favorable feedback reactions (β = .43, p < .05). This indicates that the slope
for high procedural information was more positive than the slope for low procedural
information. Furthermore, the interaction term explained 2% of the variance in favorable
feedback reactions above the previous predictors (∆R² = .02, F(1,242) = 4.83, p < .05). To
determine if the pattern of the interaction was consistent with our hypothesis, we plotted the
interaction in Figure 2 (Aiken & West, 1991). As predicted by Hypothesis 2a, Figure 2
revealed that the relationship between feedback score and favorable feedback reactions was
slightly more pronounced for individuals in the high procedural information group.
As can be seen in Table 3 (2nd part), we did not find a significant interaction effect
between procedural information and feedback score for unfavorable feedback reactions (β = -
.05, p > .05, ∆R² = .00, F(1,237) = .28, p > .05). Thus, Hypothesis 2b was not supported.
Qualitative analyses. To provide a rich and in-depth understanding of learners’
reactions, we analyzed the qualitative comments that respondents provided. We summarized
and sampled the typical responses for the hypotheses that were supported in the quantitative
analysis. We believe these comments are exemplary of the reasons why participants reacted
more favorably to their feedback scores when procedural information was high. An overview
of qualitative comments made by respondents in all different conditions of this study can be
found in Appendix B.
First, respondents reacted favorably to positive feedback, and this positive relationship
was more pronounced when they received a high amount of procedural information in their

22
feedback reports. Participants’ comments indicated that a main reason for this finding was
that the information about the rating process gave them the confidence that raters did a good
job at observing them during the different workshops. As respondents noted:
“I was surprised that my scores were exactly how I thought they would be! This must
have been a very difficult task for the raters! I didn’t think they would have been able
to paint a correct picture of my performance, but they did. Congratulations to all of
them!” (Respondent 36)
“All in all I think the feedback is correct. I am surprised that the observers were able
to make such good observations, as the remarks they made are absolutely true! I think
it’s great that they paid so much attention to observing us, it makes you feel as if
though they really cared for improving our performance!” (Respondent 55)
As reflected in the comments, a second reason for the more favorable reactions when
receiving procedural information may be that the respondents knew who observed them and
deemed the raters to be credible. Consequently, they attached greater value to the comments
made, and hence believed they could use the feedback for further improvement. This
assumption is supported by some of the respondents who noted:
“I am satisfied with my score on most skills, and I agree with the somewhat lower
scores I received. I definitely agree with the observers that I am not that good at
negotiating, and that I should try to use the leads that were given during the
workshops.” (Respondent 11)
“I feel as if though my report shows how I performed during the different workshops…
I am really happy that, for the first time, I received a clear picture about how I
perform on several skills, and that I have some guidelines about what I can do to
improve my performance on these skills.” (Respondent 255)

23
Insert Table 3 about here
Insert Figure 2 about here
Hypotheses 3a and 3b
Quantitative analyses. To test Hypotheses 3a and 3b, we included the main effect of
feedback score and two dummy coded variables reflecting the three information specificity
levels (low, moderate and high) in the equation. When coding the variables, we used the
‘low’- information specificity group as the ‘focal’ or ‘base’ group. In the third step the
interactive terms computed using the centered variable of feedback score and the two dummy
coded variables were entered. As can be seen from Table 4 (1st part), the interaction between
information specificity and feedback score was not significant for favorable feedback
reactions (∆R² = .01, F(2,240) = .91, p > .05), thus Hypothesis 3a was not supported.
For Hypothesis 3b, and as can be seen in Table 4 (2nd part), the interaction between
information specificity and feedback score explained additional variance in unfavorable
reactions beyond the main effects and explained 3% of the variance in unfavorable feedback
reactions above the previous predictors (∆R² = .03, F(2,235) = 4.21, p < .05). We found a
significant Z1 by feedback score interaction term (β = .35, p < .01). This indicates that the
slope for moderate information specificity is more positive than the low information
specificity slope. Contrary to our predictions in Hypothesis 3b, Figure 3 reveals that the
relationship between feedback score and unfavorable feedback reactions was slightly more

24
pronounced for individuals in the low information specificity group than for people in the
moderate information specificity group. Thus, Hypothesis 3b was not supported.
Qualitative analyses. Here, the results from quantitative analyses show that
respondents reacted unfavorably to negative feedback, but this positive relationship was less
pronounced when they received a low amount of information specificity in their feedback
reports. This unexpected pattern was also reflected in participants’ comments. For instance, an
exemplary commentary was:
“I don’t understand why I got the scores that I got… There are so many reasons that
can influence these feedback scores, so I don’t think these scores paint a correct
picture of my performance during the workshops….” (Respondent 85)
“I have a low score on teamwork, although I think I did quite well in this workshop.
Even so I am not disappointed, as the feedback wasn’t explained to me, and I thus
attach little value to my feedback report and the scores in it.” (Respondent 237)
Thus, a possible explanation for this result may be that respondents did not feel
inclined to accept their negative feedback scores because of the lack of information that was
given to them, and hence did not feel the need to respond unfavorably. It seems that a low
amount of information enables respondents to attribute their low feedback score to factors
other than their performance such as low-quality ratings or extraneous conditions. As
respondents noted:
“I wasn’t surprised that I got some low scores in the report. I was very tired during
the different workshops so I didn’t perform as well as I usually do… I know that I do
much better under ‘normal’ circumstances.” (Respondent 193)
“I think these scores are rather subjective, as I didn’t get an explanation for them.
However, I know I had a bad day the day of the workshop, so that may be an

25
explanation for my low scores… I am quite positive that I would score higher on a
good day.” (Respondent 225)
An overview of qualitative comments made by participants in the other conditions of
this study can be found in Appendix B.
Insert Table 4 about here
Insert Figure 3 about here
Hypotheses 4a and 4b
To investigate the relationship between favorable and unfavorable feedback reactions
and involvement in skill development, we looked at the correlations between reactions and the
dependent variable. As can be seen in Table 1, we found a significant correlation between
involvement in skill development activities and favorable reactions (r = .28, p < .01), but a
non-significant correlation between this variable and unfavorable reactions (r = -.07, p > .05).
Next, we conducted a more stringent test of these relationships involving all control
variables that can influence the dependent variable. We conducted two hierarchical multiple
regression analyses with satisfaction with performance as an additional control variable next
to the four control variables that were also included in all previous analyses. As can be seen in
Table 5 (1st part), we found a positive relationship between favorable feedback reactions and
involvement in skill development (β = .20, p < .05), and favorable feedback reactions
explained 5% of the variance in involvement in skill development above the previous
predictors (∆R² = .05, F(1,135) = 6.92, p < .05). Thus, Hypothesis 4a was supported. Finally,
we conducted a relative weights analysis (Tonidandel & Lebreton, 2011) to provide an

26
estimate of the relative importance of each of the different independent variables in predicting
involvement in skill development. As can be seen in Table 5 (1st part), favorable reactions had
the highest relative importance of all predictors of involvement in skill development activities
(50.4%), whereas satisfaction with performance was the least important predictor (0.4%).
As can be seen in Table 5 (2nd part), the effect of unfavorable feedback reactions on
involvement in skill development was not significant (∆R² = .00, F(1,133) = .06, p > .05).
Hypothesis 4b was thus not supported. We again conducted a relative weights analysis. Table
5 (2nd part) shows that in this case positive affectivity had the highest relative importance
(40.2%) whereas satisfaction with performance was the least important predictor (0.7%).
Finally, in an exploratory sense we also tested whether the manipulations and their
interactions with feedback scores had an effect on involvement in skill development activities,
but found no significant effects.
Insert Table 5 about here
DISCUSSION
The present study examined informational factors that were proposed to enhance
feedback reactions in managerial skill development. First, we found that learners reacted more
favorably to positive feedback and that this positive effect was strengthened if the amount of
procedural information they received was high. This result indicates that feedback recipients
react more favorably to a higher score when they are aware of the process and procedures
used to reach the feedback decision. Qualitative comments by respondents suggest that
participants attached greater value to their feedback when they knew it came from trained
observers. When feedback providers were seen as credible sources, respondents saw the
feedback as a helpful means to improve their performance.

27
Second, learners reacted unfavorably to negative feedback but this effect was less
pronounced when the specificity of feedback information they received was low. This finding
is surprising as we expected unfavorable feedback reactions to diminish when the participants
received high specific information. A viable explanation is that, under conditions of low
specificity, learners are able to protect their self-image by attributing poor performance to
uncontrollable or external causes (Ilgen & Davis, 2000; Taylor, Fisher, & Ilgen, 1984). This
was also supported by qualitative comments made by participants. We found that, in case of
low information specificity, participants referred to external factors as the cause of their low
performance. The underlying mechanism is that when people receive negative feedback
substantiated by specific, personal comments explaining exactly why the feedback message
was negative, it becomes difficult, if not impossible, to attribute this to external uncontrollable
causes. When learners receive the same feedback without these personalized remarks, making
external attributions for this feedback is more likely. A study by Schinkel, van Dierendonck
and Anderson (2004) supports this explanation. Providing participants with detailed
performance feedback in the context of a negative selection decision sometimes led to more
negative participants’ reactions, suggesting that the provision of detailed performance
feedback is not always as advantageous as often assumed. In an exploratory sense, we further
probed this explanation by coding and analyzing qualitative comments of participants on their
perceptions of controllability. However, exploratory analysis with qualitative data did not
yield any significant results in the proposed direction.
Third, we found a positive relationship between favorable feedback reactions and
involvement in skill development activities 15 months after receiving feedback. This is an
important finding as it corroborates our central assumption that initial reactions to feedback
are predictive of future development activities, even over longer periods of time. It invites

28
management educators to pay more attention to learner’s immediate reactions and to invest
effort in feedback interventions that are supportive of favorable feedback reactions as we
proposed in the current study. In an exploratory sense we also tested whether feedback
manipulations affected development 15 months later, but found no significant effects. Thus,
some caution is needed. Although information specificity and procedural information are
important educational strategies for shaping immediate feedback reactions, they may be less
important for developmental activity in the long term.
Theoretical Implications
Theoretically, our study contributes to a better understanding of how different types of
information, and how information is presented, can affect learners’ reactions to feedback.
Recently, calls have been made to develop new interventions for enhancing feedback
processes that have the potential to impact on immediate reactions to feedback (e.g., Anseel et
al., 2009). We think that our study fills this gap in the literature and extends the current
theoretical focus on how the processing of information may facilitate feedback interventions.
Furthermore, this study also addresses an important concern in the literature by developing
and applying a feedback intervention in the field, rather than merely measuring participants’
post-hoc perceptions. Greenberg (2009) recently criticized researchers for focusing too much
on generating knowledge, rather than investigating how these theoretical principles should be
applied. In this study, we tried to address this critique by actively developing and
experimentally testing a feedback intervention that can readily be implemented in the context
of management education. Thus, we are the first to show that altering the information
presented in feedback reports causes changes in feedback reactions.
Recommendations for Educators

29
From a practical perspective, developing solid feedback interventions has been a
challenge for management educators for quite some time now. Given the practical design of
the current study, the main findings of this study should be appealing and easily
implementable for practitioners. Based on the insight in the present study, we offer four
strategies that educators may want to consider when providing feedback to students. A first
strategy deals with the level of information in the feedback report. Our study showed that a
lack of procedural information may undermine favorable reactions. Therefore, we advise
educators to ensure that feedback recipients are aware of the procedures used to reach the
feedback decision and be honest about the process that led to the feedback (score). We also
found that high levels of information specificity increased unfavorable feedback reactions.
We encourage educators to be cautious with the immediate provision of detailed negative
performance feedback. However, at the same time educators should be aware of the potential
pitfall of hiding the specifics of negative feedback to avoid negative reactions. It is therefore
crucial to strike a balance between being clear on the one hand and being specific on the
other. Research investigating performance-enhancing feedback strategies has shown that
reflecting on feedback can enhance performance improvement, but only in combination with
external feedback and guidance (Anseel et a l., 2009). Therefore, one way of conveying
negative feedback without being overly specific is by providing feedback recipients with
overall outcome feedback and helping them to find out the explanations for potential negative
outcomes on their own by means of reflection and after event-reviews. It is important to
realize that these guidelines are especially helpful to enhance reactions to feedback, but they
may have less direct impact on developmental activities in the long term. Still, focusing on
immediate feedback reactions seem warranted as feedback reactions are predictive of
development activities 15 months later.

30
A second strategy concerns individual differences among learners. Relative weights
analysis showed that positive affectivity and learning goal orientation are the most important
traits in determining students’ involvement in skill development activities. Given that
individual differences may make learners more or less open to feedback, we recommend that
educators try to make students aware of their natural dispositions towards feedback and
encourage them to engage in introspection when dealing with feedback messages. Educators
should also train themselves in paying attention to these individual differences and tailor
feedback messages to students individually.
Third, learners’ comments suggest that they attribute low feedback scores to external
causes (e.g., “I was tired” or “I had a bad day”) whereas others take responsibility for their
actions (e.g., “I didn’t put as much effort in as I should have done”). We argue that it is
crucial to help learners deal with critical feedback. Educators may guide them during
feedback interventions in how to act less defensively when receiving criticism. The qualitative
comments suggest that internal and external attributions are a crucial mechanism for learners
to take responsibility for the feedback received. Thus, it is important for educators to manage
students’ attributional style as a means to reduce unfavorable reactions to negative feedback
and stimulate learning.
Fourth, comments provided by learners suggest rater issues play an important role.
Management educators may experience difficulties in communicating negative feedback. We
propose that feedback that is provided and discussed by a feedback facilitator who helps
recipients to interpret the feedback message in an appropriate manner, will lead to more
favorable reactions and stronger development. In the context of an educational setting, this
may well be another (credible) teacher that was not involved in the development of the
feedback message and who can act as an independent facilitator alongside the person

31
responsible for the feedback. However, taking into account the workload teachers often have
to deal with, this may not always be the most realistic option. A more practical possibility is
therefore to use web-based feedback systems (e.g., ‘Expert Systems’, Van Fleet et al., 2005)
that provide standardized feedback based on the information inserted by the teacher. We
believe using these feedback systems can be a valuable tool for educators to provide students
with objective feedback on a regular basis. We argue that the strategies formulated here can
provide educators with a more integrated sense of actions they can take when giving feedback
to students and when dealing with their subsequent reactions.
Suggestions for Future Research
Apart from its theoretical implications, our results may also guide future studies on
feedback reactions. First, given the frequent use of multisource feedback systems in
management development programs (e.g., Brutus, Petosa, & Aucoin, 2005; Hooijberg &
Lane, 2009; Shipper, Hoffman, & Rotondo, 2007), an interesting avenue for future studies is
how these different types of information and feedback formats can be implemented in the
context of multisource feedback systems. Second, research should address whether people
react differently to feedback reports that are provided face-to-face in contrast to electronic
channels (such as e-mail). Research has for instance revealed that employees seek more
feedback when feedback can be requested and/or provided via a computer (Ang & Cummings
1994; Ang, Cummings, Straub, & Early, 1993; Kluger & Adler 1993). Although studies have
shown that both face-to-face (e.g., Hwang, Ang, & Francesco, 2002) as well as electronic
(e.g., Arbaugh & Benbunan-Fich, 2006) feedback channels are important in the learning
environment, no research has investigated the impact of both types of channels on reactions to
feedback (Hwang & Francesco, 2010). Third, future research should examine the effects of
other changes in information in feedback reports. For instance, self-determination theory

32
would suggest that the tone used in feedback reports (e.g., “good, you did as you should” vs.
“good, this is exemplary”) would also affect feedback reactions. Fourth, the generation of
children born between 1976 and 1994, the so-called “Generation Y” or “Millenials”, are
currently entering our labour market and classrooms (Gardner, 2006). These ‘Millenials’ are
characterised as optimistic, tenacious, hard-working, and civic-minded. However, some
describe them as self-absorbed, unable to entertain themselves, and not tough enough to
handle the workplace (Zemke, Raines, & Filipczak, 2000). This generation is said to be
unable to handle negative feedback and critique. It would be very interesting to explore
whether and how these cohort effects have an impact on recipients’ reactions to feedback.
Finally, future studies might look at the effects of feedback source credibility. Students in the
high procedural information did know their raters were trained psychology students. Although
not quantitatively measured, respondents’ qualitative comments suggested that rater
credibility played a role. When participants knew who rated them, their reactions seemed
more favorable than when they did not receive information about the observers and the
observation process.
Limitations
Of course, the current study is not without its limitations. A first limitation is that we
conducted an experimental field study rather than a controlled lab study. Therefore, the
information provided in the high specificity information group (e.g., actual behavioral
observation) was not exactly the same across participants. People in this group received
personal comments observed in a workshop that could not be completely standardized.
However, all measures were taken to maximize standardization. Participants in the high
specificity group received a maximum of three sentences in their report all formulated in a
similar way. We believe that this approach corresponds most closely to feedback practices

33
where people receive feedback that is tailored to their actual behavior in the specific work or
developmental environment. A second limitation is that we investigated only two types of
information in feedback reports. It would be interesting to investigate whether other types of
information (e.g., text versus numeric feedback, normative versus self-referenced feedback)
are also related to feedback reactions. Third, we relied on self-reported involvement in
developmental activities but had no objective data on actual behavioral learning or job
performance. Fourth, although we assumed that the attributions made by participants about
the feedback would be likely mechanisms for the reactions they displayed, we could not find
support for this in exploratory analyses of the qualitative data. Clearly, an in-depth
examination of the attributions made by recipients about the positive or negative feedback
they receive is an important issue for future research.
Conclusion
In conclusion, we found that the effects of feedback scores on feedback reactions are
altered by the presence of procedural information and information specificity in feedback
reports. Furthermore, we showed that favorable feedback reactions were predictive of
involvement in skill development over a period of 15 months. These findings should
encourage management educators to take a closer look at the type and amount of information
given in feedback reports and to pay more attention to initial feedback reactions during skill
development.

34
REFERENCES
Aiken, L. S., & West, S. G. 1991. Multiple regression: Testing and interpreting interactions.
Thousand Oaks, CA: Sage.
Alliger, G. M., Tannenbaum, S. I., Bennett, W., Traver, H., & Shotland, A. 1997. A meta-
analysis of the relations among training criteria. Personnel Psychology, 50: 341-358.
Ang, S., & Cummings, L. L. 1994. Panel analysis of feedback-seeking patterns in face-to-
face, computer-mediated, and computer-generated communication environments.
Perceptual and Motor Skills, 79: 67-73.
Ang, S., Cummings, L. L., Straub, D. W., & Early, C. P. 1993. The effects of information
technology and the perceived mood of the feedback giver on feedback seeking.
Information Systems Research, 4: 240–261.
Anseel, F., & Lievens, F. 2006. Certainty as a moderator of feedback reactions? A test of the
strength of the self-verification motive. Journal of Occupational and Organizational
Psychology, 79: 533-551.
Anseel, F., Lievens, F., & Schollaert, E. 2009. Reflection as a strategy to enhance task
performance after feedback. Organizational Behavior and Human Decision Processes,
110: 23-35.
Arbaugh, J. B., & Benbunan-Fich, R. 2006. An investigation of epistemological and social
dimensions of teaching in online learning environments. Academy of Management
Learning & Education, 5: 435-447.
Atwater, L. E., & Brett, J. F. 2005. Antecedents and consequences of reactions to
developmental 360° feedback. Journal of Vocational Behavior, 66: 532-548.
Atwater, L. E., & Brett, J. F. 2006. Feedback format: Does it influence manager's reactions
to feedback? Journal of Occupational and Organizational Psychology, 79: 517-532.

35
Baldwin, T. T., & Magjuka, R. J. 1997. Training as an organizational episode: Pre-training
influences on trainee motivation. In J. K. Ford & Associates (Eds.). Improving training
effectiveness in work organizations: 99-128. Mahwah, NJ: Erlbaum.
Bandura, A. 1997. Self-efficacy: The exercise of control. New York: Freeman.
Bono, J. E., & Colbert, A. E. 2005. Understanding responses to multi-source feedback: The
role of core self-evaluations. Personnel Psychology, 58: 171-203.
Brett, J. F., & Atwater, L. E. 2001. 360° Feedback: Accuracy, reactions, and perceptions of
usefulness. Journal of Applied Psychology, 86: 930-942.
Brockner, J., Derr, W. R., & Laing, W. N. 1987. Self-esteem and reactions to negative
feedback – Toward greater generalizability. Journal of Research in Personality, 21:
318-333.
Brutus, S. 2009. Words versus numbers: A theoretical exploration of giving and receiving
narrative comments in performance appraisal. Human Resource Management Review,
20: 144-157.
Brutus, S., & Donia, M. In press. Improving the effectiveness of students in groups with a
centralized peer evaluation system. Academy of Management Learning & Education,
In press.
Brutus, S., & Facteau, J. 2003. Short, simple, and specific: The influence of item design
characteristics in multi-source assessment contexts. International Journal of Selection
and Assessment, 11: 313–325.
Brutus, S., Petosa, S., & Aucoin, E. 2005. Who will evaluate me ? Rater selection in multi-
source assessment contexts. International Journal of Selection and Assessment, 13:
129–138.

36
Cohen, J. 1960. A coefficient of agreement for nominal scales. Educational and
Psychological Measurement, 20: 37-46.
Cohen-Charash, Y., & Spector, P. E. 2001. The role of justice in organizations: A meta-
analysis. Organizational Behavior and Human Decision Processes, 86: 278-321.
Colquitt, J. A., Conlon, D. E., Wesson, M. J., Porter, C. O. L. H., & Yee Ng, K. 2001.
Justice at the millennium: A meta-analytic review of 25 years of organizational justice
research. Journal of Applied Psychology, 86: 425-445.
Farr, J. L., Hofmann, D. A., & Ringenbach, K. L. 1993. Goal orientation and action control
theory: Implications for industrial and organizational psychology. In C. L. Cooper & I.
T. Robertson (Eds.), International review of industrial and organizational psychology:
193–232. New York: Wiley.
Fontaine, J. R. J., Scherer, K. R., Roesch, E. B., & Ellsworth, P. C. 2007. The world of
emotions is not two-dimensional. Psychological Science, 18: 1050-1057.
Forgas, J. P., & George, J. M. 2001. Affective influences on judgments and behavior in
organizations: An information processing perspective. Organizational Behavior and
Human Decision Processes, 86: 3-34.
Gardner, S. F. 2006. Preparing for the Nexters. American Journal of Pharmaceutical
Education, 70: Article 87.
Gist, M. E., & Mitchell, T. R. 1992. Self-efficacy – A theoretical-analysis of its determinants
and malleability. Academy of Management Review, 17: 183-211.
Goodman, J. S., & Wood, R. E. 2004a. Feedback specificity, learning opportunities, and
learning. Journal of Applied Psychology, 89: 809-821.
Goodman, J. S., & Wood, R. E. 2004b. Feedback specificity, exploration, and learning.
Journal of Applied Psychology, 89: 248-262.

37
Greenberg, J. 2009. Everybody talks about organizational justice, but nobody does anything
about it. Industrial and Organizational Psychology. Perspectives on Science and
Practice, 2: 181-195.
Hall, D. T., & Associates, 1996. The career is dead - long live the career: A relational
approach to careers. San Francisco, CA: Jossey Bass.
Hammer, L. B., & Stone-Romero, E. F. 1996. Effects of mood state and favorability of
feedback on reactions to performance feedback. Perceptual and Motor Skills, 83: 923–
934.
Hooijberg, R., & Lane, N. 2009. Using multisource feedback coaching effectively in
executive education. Academy of Management Learning & Education, 8: 483-493.
Hwang, A., Ang, S., & Francesco, A. M. 2002. The silent Chinese: The influence of face and
Kiasuism on student feedback-seeking behaviors. Journal of Management Education,
26: 70-98.
Hwang, A., & Francesco, A. M. 2010. The influence of individualism-collectivism and power
distance on use of feedback channels and consequences for learning. Academy of
Management Learning & Education, 9: 243-257.
Ilgen, D. R., & Davis, C. A. 2000. Bearing bad news: Reactions to negative performance
feedback. Applied Psychology: An International Review, 49: 550–565.
Ilgen, D. R., Fisher, C. D., & Taylor, M. S. 1979. Consequences of individual feedback on
behavior in organizations. Journal of Applied Psychology, 64: 349–371.
Illies, R., De Pater, I. E., & Judge, T. A. 2006. Emotional reactions to performance feedback:
The effect on goal-regulation. Journal of Managerial Psychology, 22: 590-609.
Jawahar, I. M. 2007. The influence of perceptions of fairness on performance appraisal
reactions. Journal of Labor Research, 28: 735-754.

38
Judge, T. A., Erez, A., Bono, J. E., & Thoresen, C. J. 2003. The core self-evaluations scale:
Development of a measure. Personnel Psychology, 56: 303-331.
Judge, T. A., & Larsen, R. J. 2001. Dispositional affect and job satisfaction: A review and
theoretical extension. Organizational Behavior and Human Decision Processes, 86:
67-98.
Kamer, B., & Annen, H. 2010. The role of core self-evaluations in predicting performance
appraisal reactions. Swiss Journal of Psychology, 69: 95-104.
Kluger, A. N., & Adler, S. 1993. Person-versus computer-mediated feedback. Computers in
Human Behavior, 9: 1-16.
Kluger, A. N., & DeNisi, A. 1996. The effect of feedback interventions on performance: A
historical review, a meta-analysis, and a preliminary feedback intervention theory.
Psychological Bulletin, 119: 254-284.
Kozlowski, S. J. W., & Farr, J. L. 1988. An integrative model of updating and performance.
Human Performance, 1: 5-29.
Kudisch, J. D., Ladd, R. T., & Dobbins, G. H. 1997. New evidence on the construct validity
of diagnostic assessment centers: The findings may not be so troubling after all. Journal
of Social Behavior and Personality, 12: 129-144.
Leung, K., Su, S., & Morris, M. W. 2001. When is criticism not constructive ? The role of
fairness perceptions and dispositional attributions in employee acceptance of critical
supervisory feedback. Human Relations, 54: 1155-1187.
London, M., & Smither, J. W. 1995. Can multi-source feedback change perceptions of goal
accomplishment, self-evaluations, and performance-related outcomes? Theory based
applications, and directions for research. Personnel Psychology, 48: 803-839.

39
London, M., & Smither, J. W. 1999. Empowered self-development and continuous learning.
Human Resource Management, 38: 3-15.
Love, E. G., Love, D. W., & Northcraft, G. B. 2010. Is the end in sight? Student regulation of
in-class and extra-credit effort in response to performance feedback. Academy of
Management Learning & Education, 9: 81-97.
Mathieu, J. E., & Martineau, J. W. 1997. Individual and situational influences on training
motivation. In J. K. Ford, S.W.J. Kozlowski, K. Kraiger, E. Salas, & M. S. Teachout
(Eds.), Improving training effectiveness in work organizations: 193–221. Hillsdale,
NJ: Erlbaum.
Maurer, T. J. 2001. Career-relevant learning and development, worker age, and beliefs about
self-efficacy for development. Journal of Management, 27: 123-140.
Maurer, T. J., Weiss, E. M., & Barbeite, F. G. 2003. A model of involvement in work-related
learning and development activity: The effects of individual, situational, motivational,
and age variables. Journal of Applied Psychology, 88: 707-724.
Mitchell, T. R., & Beach, L. R. 1990. Do I love thee – Let me count – Toward an
understanding of intuitive and automatic decision-making. Organizational Behavior
and Human Decision Processes, 47: 1-20.
Noe, R. A. 1996. Is career management related to employee development and performance?
Journal of Organizational Behavior, 17: 119-133.
Payne, S. C., Youngcourt, S. S., & Beaubien, J. M. 2007. A meta-analytic examination of the
goal orientation nomological net. Journal of Applied Psychology, 92: 128-150.
Ryan, A. M., Brutus, S., Greguras, G., & Hakel, M. D. 2000. Receptivity to assessment-based
feedback for management development: Extending our understanding of reactions to
feedback. Journal of Management Development, 19: 252-276.

40
Sadri, G., & Robertson, I. T. 1993. Self-efficacy and work-related behavior – A review and
meta-analysis. Applied Psychology: An International Review, 42: 139-152.
Schinkel, S., Van Dierendonck, D., & Anderson, N. 2004. The impact of selection encounters
on applicants: An experimental study into feedback effects after a negative selection
decision. International Journal of Selection and Assessment, 12: 197–205.
Shipper, F., Hoffman, R., & Rotondo, D. 2007. Does the 360 feedback process create
actionable knowledge equally across cultures? Academy of Management Learning &
Education, 6: 33-50.
Sitzmann, T., Ely, K., Brown, K. G., & Bauer, K. N. 2010. Self-assessment of knowledge: A
cognitive learning or affective measure? Academy of Management Learning &
Education, 9: 169-191.
Smither, J. W., London, M., & Richmond, K. R. 2005. The relationship between leaders’
personality and their reactions to and use of multisource feedback – A longitudinal
study. Group & Organization Management, 30: 181-210.
Smither, J. W., & Walker, A. G. 2004. Are the characteristics of narrative comments related
to improvement in multirater feedback ratings over time? Journal of Applied
Psychology, 89: 575-581.
Swann, W. B., & Schroeder, D. G. 1995. The search for beauty and truth: A framework for
understanding reactions to evaluations. Personality and Social Psychology Bulletin,
21: 1307–1318.
Taylor, M. S., Fisher, C. D., & Ilgen, D. R. 1984. Individuals’ reactions to performance
feedback in organizations: A control theory perspective. In K. Rowland & J. Ferris
(Eds.), Research in Personnel and Human Resource Management: 81-124.
Greenwich, CT: JAI Press.

41
Tharenou, P., Saks, A. M., & Moore, C. 2007. A review and critique of research on training
and organizational-level outcomes. Human Resource Management Review, 17: 251-
273.
Tonidandel, S., & LeBreton, J. M. 2011. Relative importance analysis: A useful supplement
to regression analysis. Journal of Business and Psychology, 26: 1-9.
Trope, Y., Ferguson, S., & Ragunanthan, R. 2001. Mood as a resource in processing self-
relevant information. In J. P. Forgas (Ed.), The handbook of affect and social
cognition. Mahwah, NJ: Erlbaum.
Vandewalle, D., Cron, W. L., & Slocum, J. W. 2001. The role of goal orientation following
performance feedback. Journal of Applied Psychology, 86: 629-640.
Vandewalle, D., & Cummings, L. L. 1997. A test of the influence of goal orientation on the
feedback-seeking process. Journal of Applied Psychology, 82: 390–400.
Van Fleet, D. D., Peterson, T. O., & Van Fleet, E. W. 2005. Closing the performance
feedback gap with expert systems. Academy of Management Executive, 19: 38-53.
Watson, D., Clark, L. A., & Tellegen, A. 1988. Development and validation of brief measures
of positive and negative affect: The PANAS scales. Journal of Personality and Social
Psychology, 54: 1063-1070.
Zemke, R., Raines, C., & Filipczak, B. 2000. Generations at work: Managing the clash of
veterans, boomers, xers, and nexters in your workplace. New York: American
Management Association.
Zoogah, D. B. 2010. Why should I be left behind? Employees’ perceived relative deprivation
and participation in development activities. Journal of Applied Psychology, 95: 159-
173.