Content uploaded by Gary Yukl
Author content
All content in this area was uploaded by Gary Yukl
Content may be subject to copyright.
Effects of Multisource Feedback and a Feedback Facilitator on the
Influence Behavior of Managers Toward Subordinates
Charles F. Seifert
Siena College
Gary Yukl
University at Albany
Robert A. McDonald
Rensselaer Polytechnic Institute
The authors compared a feedback workshop with both a no-feedback control group and a comparison
group of managers who received a feedback report but no feedback workshop. The multisource feedback
was based on ratings of a manager’s influence behavior by subordinates, peers, and bosses. Managers in
the feedback workshop increased their use of some core influence tactics with subordinates, whereas
there was no change in behavior for the control group or for the comparison group. The feedback was
perceived to be more useful by managers who received it in a workshop with a facilitator than by
managers who received only a printed feedback report.
A widely used developmental method for managers and other
professional employees is multisource feedback, which is often
referred to as 360-degree feedback. In a multisource-feedback
program, managers generally receive information about how they
are perceived by various groups of people with whom they interact
regularly (e.g., subordinates, peers, bosses, and clients). The feed-
back is usually provided in a report that contains descriptive
information and graphical presentations of the data from the var-
ious sources, including self-ratings by the managers. In most
programs the feedback consists of ratings on specific types of
behavior or skill. Widespread interest in multisource feedback is
affirmed by the number of recent books on the topic (Edwards &
Ewen, 1996; Lepsinger & Lucia, 1997; Tornow, London, & CCL
Associates, 1998; Waldman & Atwater, 1998). The popularity of
multisource feedback reflects the assumption that it is an effective
developmental method (Edwards & Ewen, 1996).
Potential Benefits of Multisource Feedback
The proponents of multisource feedback contend that it offers
several potential benefits. First, managers can gain a more com-
prehensive perspective of their work performance when feedback
is provided by people with a different perspective on it (Yam-
marino & Atwater, 1993). Consistent with this proposition, Ber-
nardin, Dahmus, and Redmon (1993) found that feedback was
viewed as more useful if it came from both bosses and subordi-
nates rather than from subordinates alone. Second, developmental
feedback is less likely to be ignored if the sources include peers
and superiors, because these raters have more power and status
than do subordinates. The knowledge that important people per-
ceive a deficiency may increase efforts to demonstrate competence
(London & Smither, 1995).
Consistent with control theory, one may be more motivated to
change one’s behavior when ratings from others are lower than
self-ratings (Carver & Scheier, 1981). However, negative feedback
may be discounted if the rater is perceived as being biased or
uninformed (Waldman & Atwater, 1998). When feedback from
multiple sources is consistent, it is more likely to be perceived as
accurate and useful for guiding behavior change (London &
Smither, 1995; Meyer, 1980).
Potential Benefits of a Feedback Facilitator
The potential advantages of multisource feedback may depend
in part on how it is presented to managers. There is usually a
feedback report, but it can be delivered in different ways (Edwards
& Ewen, 1996). Sometimes managers receive the report to inter-
pret for themselves, and sometimes there is a feedback facilitator
who meets with managers, either individually or collectively, to
review the feedback report. The type of process may affect the
perceived relevance and utility of the feedback. The complexity of
feedback is usually greater when there are multiple sources, and
the potential benefit of more information may be offset by the
difficulties of interpreting it. A facilitator can help recipients
interpret the feedback and understand why it is relevant (DeNisi &
Kluger, 2000; Yukl & Lepsinger, 1995). Although most feedback
reports include interpretive guides, a facilitator is able to provide
more attention and assistance.
A feedback intervention is more likely to have a beneficial
effect if recipients perceive that there is a need for improvement
and are optimistic about learning how to make improvements
(Hellervik, Hazucha, & Schneider, 1992; Kluger & DeNisi, 1996;
Charles F. Seifert, School of Business, Siena College; Gary Yukl,
School of Business, University at Albany; Robert A. McDonald, Lally
School of Management Technology, Rensselaer Polytechnic Institute.
Correspondence concerning this article should be addressed to Charles
F. Seifert, School of Business, Siena College, 515 Loudon Road, Loudon-
ville, New York 12211. E-mail: seifert@siena.edu
Journal of Applied Psychology Copyright 2003 by the American Psychological Association, Inc.
2003, Vol. 88, No. 3, 561–569 0021-9010/03/$12.00 DOI: 10.1037/0021-9010.88.3.561
561
London & Smither, 1995). With the help of a supportive facilitator,
recipients are more likely to set improvement goals and develop
plans for improving their performance (Bracken, 1994; DeNisi &
Kluger, 2000). A feedback workshop provides managers with
uninterrupted time to analyze their feedback and develop improve-
ment plans (Bracken, 1994). In the absence of a workshop, busy
managers may spend little time thinking about the feedback or how
to apply it.
Feedback about performance deficiencies is not likely to be
beneficial if recipients are defensive about it. A facilitator can help
to keep the recipient focused on opportunities for self-development
rather than on coping with threats to self-esteem (DeNisi &
Kluger, 2000). When feedback is distributed in a workshop with a
facilitator, there are also opportunities for mutual support and
encouragement among the participants (Edwards & Ewen, 1996).
It is probably less threatening to have a feedback facilitator who is
a consultant or human resource professional than to have a supe-
rior responsible for evaluating the manager’s performance (Anto-
nioni, 1996). A neutral facilitator is less likely to elicit a defensive
reaction and may be more capable of providing advice and coach-
ing to help managers improve their performance. Moreover, when
a feedback intervention is managed by a neutral facilitator, recip-
ients may be more confident that the feedback will remain
confidential.
Research on Effects of Feedback to Managers
The general effects of feedback have been studied for decades,
and there have been many laboratory experiments. After a meta-
analysis of 131 feedback studies, Kluger and DeNisi (1996) con-
cluded that there was only a small overall improvement in perfor-
mance, and performance actually declined one third of the time.
They proposed several reasons why feedback may not be effective,
including characteristics of the feedback, the task, and the recipi-
ents. Their meta-analysis provides some important insights, but
most of the studies were laboratory experiments on feedback about
task performance or field studies on performance feedback from an
authority figure or an expert (e.g., a teacher). The review did not
examine the effects of upward or multisource feedback about
managerial behavior.
Despite the widespread use of behavioral feedback as a tech-
nique for management development, there has been relatively little
research to directly evaluate its effectiveness. We found only 14
studies that assessed the effects of upward or multisource feedback
on managers and other formal leaders. The criterion in these
studies was a change in ratings of managerial behavior or skill, not
changes in objective measures of effectiveness. Table 1 describes
the 14 studies and summarizes results based on ratings of mana-
gerial skill or managerial behavior made by others before and after
the feedback intervention. The table does not show results based
on anecdotal accounts of improvement, retrospective ratings of
perceived change, or self-ratings by managers, all of which are
likely to be biased indicators for evaluating feedback interventions.
Results from this research were generally weak and inconsistent.
The research method used in most of the studies involved com-
paring premeasures and postmeasures of behavior or skill for a
single group of managers. This type of study has no control group
and cannot verify that positive outcomes are a direct result of the
feedback intervention. There are several sources of contamination
for single group pre–post designs, and they are more likely to
occur in studies conducted over a long period of time.
One potential source of contamination (high manager mortality)
occurs when many of the managers who are in the sample at the
premeasure elect to drop out before the postmeasure. If the drop-
outs consist mostly of managers who are unmotivated to improve
or very competent managers with little potential to improve, then
the apparent effects of the feedback intervention on the remaining
managers may be inflated.
Rater instability is another source of contamination. The com-
mon practice of aggregating scores for the raters of a manager
often obscures the fact that some of the raters were not the same
for the premeasure and postmeasure. In studies in which raters are
not held constant, the results may reflect differences in the raters
rather than changes in manager behavior. The effect of the feed-
back can be inflated if many dropout raters give low premeasure
ratings because they disliked the manager, and new raters who
provide only a postmeasure rating are lenient because they do not
know the manager very well.
Even when the same managers are rated each time by the same
people, the results can be biased by changes in perceptions (Lon-
don & Smither, 1995). After the initial experience with the rating
process, the raters may gain a better understanding of the behaviors
and begin to observe them more carefully. If raters know the
managers are participating in developmental activities, then
the postmeasure ratings may be inflated by expectations of
improvement.
Many types of organizational changes can affect the behavior or
performance of a manager (e.g., structural changes, job redesign,
culture change, new incentives, and follow-up training or coach-
ing). Without random assignment of managers to an experimental
or control group, one cannot determine how much these other
factors affected managerial behavior and performance. To illus-
trate the potential for contamination, in the study by Smither et al.
(1995), there was a significant increase in behavior ratings, but
managers who received a feedback report did not improve more
than managers without such feedback.
Only four of the prior studies were field experiments with
random assignment to experimental and control conditions. These
experiments all involved upward feedback, and only one found
clear evidence that the feedback was effective. We did not find any
field experiments on the effects of multisource feedback on be-
havior or performance. The three nonexperimental studies on the
effects of multisource feedback did not provide consistent results.
In summary, there is little evidence that a multisource feedback
intervention is effective for changing managerial behavior.
There has been little research on the benefits of a feedback
facilitator. Antonioni (1995) compared the effects of giving man-
agers only a feedback report with the effects of having managers
meet with the boss to review the feedback report, but neither type
of intervention had any significant effect on managerial behavior.
The three other field experiments on upward feedback (Atwater,
Waldman, Atwater, & Cartier, 2000; Hegarty, 1974; Nemeroff &
Cosentino, 1979) had a neutral facilitator, but the results were
inconsistent across studies. Managerial behavior improved in the
study by Hegarty (1974), it did not improve in the study by
Atwater et al. (2000), and it improved in the Nemeroff and Co-
sentino (1979) study only if the facilitator encouraged managers to
set specific improvement goals. Most of the nonexperimental
562
RESEARCH REPORTS
studies on upward or multisource feedback did not clearly explain
whether a feedback facilitator was used. In summary, the benefits
of having a neutral facilitator for multisource feedback have yet to
be determined.
Research Objectives
The lack of strong, consistent results in the studies on upward
and multisource feedback echoes the conclusions reached in the
meta-analysis by Kluger and DeNisi (1996) and commentary by
other scholars (e.g., Waldman, Atwater, & Antonioni, 1998). The
widespread belief that behavioral feedback is a highly effective
method for management development has little empirical support.
Theory and research on this developmental method have lagged
behind practice (Greguras & Robie, 1998; London & Smither,
1995; Waldman et al., 1998). One objective of our study was to
find more definitive evidence about the effectiveness of a multi-
source feedback workshop for changing managerial behavior. A
second objective was to explore the proposition that a skilled,
neutral facilitator can enhance the effectiveness of multisource
feedback.
Method
The field experiment included an experimental group and a control
group in the same organization. There was also a quasi-experimental
comparison group in another organization. Managers in the experimental
condition received a feedback report in a workshop with a facilitator.
Managers in the comparison group received the same type of feedback
report, but there was no workshop. Managers in the control group received
no feedback, but after the study was completed they also had an opportu-
nity to participate in a feedback workshop. A premeasure survey was
conducted in all three conditions to assess each focal manager’s behavior
toward subordinates, peers, and bosses. The same respondents were sur-
veyed again in a postmeasure survey conducted 3 months later. The
Table 1
Summary of Studies on Behavioral Feedback to Managers
Feedback
study
Research
design
Sample for
analysis
Type of
feedback
Time
interval
(months)
Other
factors
Significant
improvements
Hegarty (1974) 2-group
experiment
56 supervisors in
a university
Upward 3 A Yes
Nemeroff &
Cosentino
(1979)
3-group
experiment
30 insurance
managers
Upward 3 F Inconclusive
(only if set
specific
goals)
Wilson et al.
(1990)
1 group
pre–post
41 hospital
managers
Upward 12 A, C Yes
Wilson et al.
(1990)
1 group
pre–post
29 bank
managers
Upward 6 A, C Yes
Hazucha et al.
(1993)
1 group
pre–post
48 utility
managers
360 24 A, B, C Inconclusive
(p ⬍ .10)
Atwater et al.
(1995)
2 groups
pre–post
978 navy cadet
leaders
Upward 4 A, D, E Yes
Antonioni (1995) 4-group
experiment
96 managers Upward 3 A No
Hezlett &
Ronnkvist
(1996)
1 group
pre–post
110 managers in
global firm
360 24 A, B No
Smither et al.
(1995)
1 group
pre–post
238 managers Upward 6 A, B, D Inconclusive
(only if had
low initial
rating)
Reilly et al.
(1996)
1 group
pre–post
92 managers Upward 30 A, B, D Inconclusive
(only if had
low initial
rating)
Rosti & Shipper
(1998)
2 groups
pre–post
53 job corps
managers
360 3 A, C, E Inconclusive
(p ⬍ .10
for a few
skills)
Johnson & Ferstl
(1999)
1 group
pre–post
2,171 managers Upward 12 A, B No
Walker & Smither
(1999)
1 group
pre–post
252 bank
managers
Upward 48 A, C, D Yes (more if
low initial
rating)
Atwater et al.
(2000)
2-group
experiment
96 police
supervisors
Upward 10 A No
Note. Time interval is between feedback and final postmeasure. A ⫽ Unmatched raters across time periods;
B ⫽ High manager mortality; C ⫽ Follow-up training was available; D ⫽ Feedback process was repeated; E ⫽
Nonrandom assignment of managers to conditions; F ⫽ Feedback involved behavior in a special context.
563
RESEARCH REPORTS
feedback was based on the premeasure data from all sources. The effect of
the feedback interventions was evaluated in terms of a manager’s change
in behavior toward subordinates.
The feedback involved a manager’s use of proactive influence tactics
with subordinates, peers, and bosses. The effectiveness of managers de-
pends on their influence over other people in the same organization, and for
many types of influence attempts one must use proactive influence tactics
(Yukl, 2002). Several studies have been conducted to identify specific
types of influence tactics (e.g., Kipnis, Schmidt, & Wilkinson, 1980;
Schriesheim & Hinkin, 1990; Yukl & Tracey, 1992). Building on the
earlier work, Yukl and Seifert (2002) developed an extended taxonomy
of 11 influence tactics that are relevant for managers. The four core tactics
most likely to result in commitment by the target person include rational
persuasion (presenting facts, evidence, and logical arguments), inspira-
tional appeals (appealing to the person’s values, ideals and emotions),
consultation (involving the person in planning how to accomplish a task or
objective), and collaboration (offering to make it easier to carry out a
request). Other tactics (e.g., exchange, pressure, or coalition) are more
likely to result in compliance or resistance (Yukl, 2002).
Sample
The sample for the experimental part of the study included most of the
midlevel managers from a regional savings bank in the northeastern United
States. The 14 middle managers included 6 men and 8 women. The
managers were randomly assigned to the experimental and control groups,
with an equal number in each group. The sample for the comparison group
included 7 middle managers (5 women and 2 men) in another regional
savings bank who volunteered to participate in the study.
In each condition we randomly selected five peers and five subordinates
of each manager to serve as raters. If a manager had fewer than five
subordinates or peers, all of them were included. In most cases the
managers had only one boss to provide feedback. In the bank used for the
experimental and control conditions, the response rate on the premeasure
survey was 80% for subordinates, 85% for peers, and 100% for bosses. The
response rate for the comparison group in the other bank was 96% for
subordinates, 91% for peers, and 100% for bosses.
Although managers in the experimental and comparison groups received
feedback from multiple sources, we were able to evaluate only the effect of
the feedback on manager behavior toward subordinates. In a small orga-
nization, insulating the peer and boss raters from possible contamination is
difficult. There were very few bosses, each boss had to rate several focal
managers, and the bosses knew which managers were in each condition.
Most of the peer raters were also focal managers. In the first bank, one half
of the peer raters participated in the feedback workshop, where they
learned how to identify the behaviors better and received feedback about
their own behavior. In the second bank, most of the peer raters were also
in the comparison group that received feedback reports. Like the bosses, all
peer raters filled out the premeasure questionnaire multiple times, and this
process may have changed their perception and awareness of the influence
tactics. In contrast, none of the subordinate raters in either bank received
feedback, and each subordinate had to rate only a single focal manager.
Therefore, we used only subordinates to evaluate the feedback interven-
tions. The number of subordinates who provided usable data from both the
premeasure and postmeasure surveys was 21 for the feedback work-
shop, 18 for the control group, and 20 for the comparison group.
Procedure
The first author presented a brief overview of the project at a regularly
scheduled meeting of the executive group for the first bank, and the bank
president indicated strong support for it. A letter from the bank president
explaining the project was sent to all participating managers and raters,
along with consent forms to be returned directly to the researchers. The
managers were assured that they would all have an opportunity to attend
either the initial feedback workshop or a second one to be held after the
postmeasure survey. Once the signed consent forms were received, the
premeasure questionnaire was mailed directly to the participants. A similar
procedure was followed in the second bank to obtain volunteers for the
feedback-only condition.
The multisource raters used the target version of the questionnaire to
describe the influence behavior of the focal managers. The name of the
manager to be rated was written on the questionnaire to make it possible to
combine responses from different raters of the same manager. Raters were
also asked to write a four-digit number on both the premeasure and
postmeasure questionnaires so that the two could be matched for the
analysis of behavior change. The focal managers in all conditions received
a self-report questionnaire designed to measure their use of different
influence tactics with subordinates, peers, and bosses.
The respondents mailed their completed questionnaires directly to the
researchers in postage-paid envelopes provided to preserve confidentiality.
Subordinate and peer responses remained anonymous, and managers who
received feedback saw only composite scale scores for each type of rater.
The feedback from bosses was not anonymous if the manager had only one
boss, which was usually the case.
Measure of Influence Behavior
The measure for assessing influence behavior was an updated version of
the Influence Behavior Questionnaire (Yukl & Tracey, 1992). Yukl and
Seifert (2002) have provided validation evidence for the 11 influence
tactics in the updated target version of the questionnaire, including a
confirmatory factor analysis. Each item had five anchored response choices
indicating how frequently the focal manager used the tactic in influence
attempts with the target person. The responses in the target version ranged
from 1 (I can’t remember him/her ever using this tactic with me)to5
(He/she uses this tactic very often with me).
The agent self-report version of the questionnaire had parallel items and
a similar rating format, with only minor differences in wording. For each
item in the agent version, the focal managers rated their use of the tactic in
influencing a specified target group (e.g., subordinates, peers, or bosses).
To make the agent version easier to complete, we created three columns
next to each question (one column for each target group).
Feedback was based on the 11 tactic scales. Evaluation of the feedback
interventions was based primarily on a composite score for the core tactics
(the mean of the 12 items on these tactics), but supplementary analyses of
feedback effects were made for the tactic scales.
Feedback Reports
Before the study began, we pilot tested the feedback report to check on
its readability and relevance. The final report included a detailed descrip-
tion of the influence tactics and their relative effectiveness. Next, there was
an explanation of how to interpret the feedback. The actual feedback was
presented as a mean rating for each tactic by target source (i.e., subordi-
nates, peers, or boss). The mean scores for each tactic were shown both in
numerical form and as a bar graph. The manager’s self-ratings for each
target source were included in the feedback report to facilitate comparison
of self-ratings to ratings by the targets. The feedback report also showed
norms for each tactic based on all premeasure data for each type of target
rater. Managers were encouraged to pay special attention to any large
discrepancies between self- and target ratings or between target ratings and
the norms for that source. Finally, there was a section to assist the focal
managers in planning how to improve their use of any core tactics that
seemed to be underutilized.
The feedback report was similar to ones used in many feedback pro-
grams, but it incorporated one uncommon element: Self-ratings were made
separately for agent influence behavior toward each type of target (i.e.,
564
RESEARCH REPORTS
subordinates, peers, or bosses). The usual procedure is to have agents make
self-ratings for their general use of a behavior for all targets. The new
procedure made it possible to compare self-ratings and target ratings
separately by type of target.
Feedback Workshop
The feedback workshop was conducted over a 7-hr period at the bank’s
training facility, and the researchers were the facilitators for the workshop.
The first part of the workshop explained the various influence tactics and
how they are related to the success of an influence attempt. The facilitators
showed a video to participants with examples of the core tactics, and we
stopped it several times to identify the tactics and discuss their relevance to
the situation. In the second part of the workshop, participants were given
their feedback reports. The facilitators explained each section of the report,
answered questions, and provided advice on how to interpret the feedback.
In the third part of the workshop, a scenario exercise provided the partic-
ipants an opportunity to consider how the influence tactics can be used in
specific situations that bank managers are likely to encounter. Three
scenarios were used, one involving downward influence, one involving
lateral influence, and one involving upward influence. The managers
worked in small groups to develop an influence strategy for each scenario,
and then the groups presented their recommendations and gave examples
of what they would say or do in each situation. In the final part of the
workshop, the facilitators helped individuals develop action plans for using
the feedback to improve their influence behavior. The facilitator empha-
sized opportunities to make better use of any core tactics that were relevant
to their situation.
Evaluation of the Feedback and the Workshop
We administered a short questionnaire at the end of the workshop to
evaluate the content of the workshop and the facilitator effectiveness. Each
scale had three items of the type commonly used to evaluate training
workshops (Noe, 1999). A 5 point anchored rating scale was used with
response categories ranging from poor to outstanding. Ratings were ob-
tained not only from managers who attended the workshop for the exper-
imental group but also from managers who attended the workshop held for
the control group a few weeks after the postmeasure survey was completed.
The two workshops are referred to as Workshop 1 and Workshop 2.
We used another short questionnaire to assess perceptions regarding
accuracy of the feedback, utility of the feedback, and capacity to improve
performance. Each scale had three items. A 5 point anchored Likert scale
was used with response categories ranging from 1 (strongly disagree)to5
(strongly agree). This questionnaire was filled out by managers in the two
workshops and managers in the second bank who received only written
feedback reports. For the comparison group, the questionnaire was mailed
to the managers along with the feedback report, and they were asked to
return it directly to the researchers.
Results
Reliability of Measures
Internal consistency reliability was determined separately for
the premeasures and postmeasures of influence behavior, and it
was adequate for all scales (
␣
⬎ .70). For the composite measure
of core tactics, the alpha value for subordinate raters was .91 for
the premeasure and .93 for the postmeasure. For subordinate raters
on the premeasure and postmeasure respectively, the alpha values
were as follows: .78 and .82 for rational persuasion, .80 and .70 for
inspirational appeals, .79 and .84 for consultation, and .71 and .88
for collaboration.
Reliability was also determined for the scales used to measure
manager reactions to the feedback and the workshop. The alpha
values were .94 for feedback accuracy, .86 for feedback utility, .94
for capacity to improve, .89 for quality of workshop content, and
.87 for facilitator effectiveness.
Evaluation of Feedback Workshop
To serve as an adequate basis for assessing the effects of
multisource feedback, the workshop must be relevant, and the
feedback report must be accurate and useful. The ratings of the
workshop were computed separately for the managers in the ex-
perimental group and the managers in the control group (who
attended the workshop after the postmeasure survey). The mean
scale score for the quality of workshop content was 4.3 for the first
workshop and 4.6 for the second workshop. The mean scale score
for facilitator effectiveness was 4.4 for the first workshop and 4.7
for the second workshop. These mean ratings (on a scale ranging
from 1.0 to 5.0) indicate a favorable assessment of the feedback
workshop. The similarity of the ratings for the two workshops (the
means were not significantly different) provides additional evi-
dence that they are reliable.
The mean ratings of feedback accuracy, relevance, and utility by
managers in the two feedback workshops and by managers in the
comparison group are shown in Table 2. There was no significant
difference for perceived accuracy of the feedback, which is con-
sistent with the fact that the feedback report was similar for each
group of managers. However, the managers in the workshops gave
significantly higher ratings of feedback utility (
2
⫽ .37) and
capacity to improve performance (
2
⫽ .38) than did the managers
in the feedback-only comparison group. The results suggest that
feedback was more useful and easier to apply when there was a
facilitator.
Table 2
Comparison of Feedback Evaluations by Three Groups of Managers
Measure
Workshop 1 Workshop 2 Feedback only
F(2,18)MSDMSDMSD
Feedback accuracy 4.05 0.30 3.71 1.08 3.38 0.49 1.56
Feedback utility 4.38
a
0.62 4.43
a
0.42 3.62
b
0.52 5.18*
Capacity to improve 4.67
a
0.58 4.71
a
0.41 3.67
b
0.92 5.45*
Number of managers 7 7 7
Note. Means that do not share subscripts have a significant pairwise difference, using Duncan’s (1995)
Multiple Range Test (p ⬍ .05).
* p ⬍ .05.
565
RESEARCH REPORTS
Effects of the Feedback Workshop on Behavior
As in most of the prior studies, the effects of the feedback
intervention were assessed in terms of changes in behavior from
the premeasure to the postmeasure. The primary analysis was
carried out at the individual level, because the questionnaire mea-
sures dyadic influence behavior toward the individual respondent,
and managers varied their use of the tactics when influencing
different subordinates. The means and standard deviations for the
composite core tactic scores are shown in Table 3. An initial
analysis of premeasure scores did not reveal any significant dif-
ferences between the experimental group and the control group for
any of the core tactics or for the composite score, which verified
that random assignment created equivalent groups for the study.
A two-way repeated measures analysis of variance (ANOVA)
was used to assess the effect of the feedback workshop on mana-
gerial behavior. Condition (feedback workshop vs. control group)
and time (premeasure vs. postmeasure) were the independent vari-
ables, and use of core tactics with subordinates was the dependent
variable. The premeasure and postmeasure ratings by subordinates
were used as the within-subjects variables, and condition was used
as the between-subjects variable. When used with matched pre-
measure and postmeasure data, the repeated measures ANOVA
allows for a control of premeasure scores when evaluating the
effects of an intervention on postmeasure scores. The results of the
ANOVA (see Table 4), indicate a significant two-way interaction
with a moderate effect size (
2
⫽ .12). To help interpret the
meaning of the interaction, we compared the premeasure and
postmeasure means within each condition (see Table 4). Managers
in the experimental condition significantly increased their use of
the core tactics with subordinates, whereas there was no significant
change for managers in the control group. The results are depicted
graphically in Figure 1.
A supplementary analysis was conducted at the group level to
ensure that the results would generalize across levels of analysis.
Responses were aggregated for each manager’s subordinates, and
the aggregate scores were used in the ANOVA. The means and
standard deviations for the group-level analysis are in Table 3, and
results for the ANOVA are in Table 4. The findings were parallel
to those for the individual level of analysis, although the effect size
was larger (
2
⫽ .51). Once again, there was a significant increase
in use of the core tactics in the experimental group but no signif-
icant increase in the control group (see Table 3).
To investigate the nature of the behavior change in more detail,
we computed the repeated measures ANOVA separately for each
core tactic at the individual level of analysis. There was a signif-
icant interaction for consultation, F(1, 37) ⫽ 9.13, p ⬍ .01,
2
⫽ .20, and it is accounted for by a significant increase in the
experimental group, t(20) ⫽ 3.44, p ⬍ .01, but not in the control
group, t(17) ⫽⫺0.68. There was also a significant interaction for
collaboration, F(1, 37) ⫽ 5.08, p ⬍ .05,
2
⫽ .12, and it is
accounted for by a significant increase in the experimental group,
t(20) ⫽ 3.30, p ⬍ .01, but not in the control group, t(17) ⫽⫺0.45.
Consistent results were found at the group level of analysis for the
effect of the feedback workshop on consultation (
2
⫽ .43) and
collaboration (
2
⫽ .46). There was no significant effect of the
feedback workshop on the other two core tactics or on any of the
noncore tactics.
Enhancing Effects of Feedback Facilitator
A preliminary analysis of premeasure data for the feedback
workshop and the feedback-only comparison group showed that
the two conditions did not differ significantly in any direction on
any of the core tactics. Thus, even though managers were not
randomly assigned to the comparison group, it was equivalent to
the experimental group on the premeasures. Manager self-ratings
of the core tactics on the premeasures were also compared, and
once again there were no significant differences between the two
conditions. Because a key aspect of the feedback is comparison of
self-ratings to ratings by others, the lack of significant differences
in either component indicates that the content of the feedback
reports was equivalent for the experimental group and the com-
parison group. That is, one group of managers did not receive
feedback that was more negative than the feedback for the other
group.
The enhancing effects of a feedback facilitator for changing
managerial behavior were evaluated with a two-way repeated
measures ANOVA. The results are shown in Tables 3 and 4. The
Condition ⫻ Time interaction was significant, both for the indi-
vidual level of analysis (
2
⫽ .21) and for the group level of
analysis (
2
⫽ .53). The follow-up t tests indicated that there was
a significant increase in use of the core tactics for the feedback
workshop but no significant change in the comparison group.
Table 3
Means and Standard Deviations for Scores on Composite
Measures of Core Tactics in Each Condition and Time Period
Condition tdf
Premeasure Postmeasure
MSDM SD
Individual level of analysis
Feedback workshop 3.53** 20 3.05 0.76 3.40 0.82
Control group ⫺0.21 17 3.06 0.68 3.00 0.83
Feedback only ⫺1.69 19 2.92 1.14 2.68 1.01
Group level of analysis
Feedback workshop 4.16** 6 2.80 0.83 3.20 0.77
Control group ⫺0.36 6 3.15 0.52 3.12 0.57
Feedback only ⫺1.58 6 3.02 1.02 2.80 0.97
** p ⬍ .01.
Table 4
Results from Repeated Measures Analysis of Variance on Core
Tactics
Conditions compared df Condition Time Interaction
Individual level of analysis
Workshop vs. control group 1, 37 0.72 2.42 5.08*
Workshop vs. feedback only 1, 39 2.33 0.34 10.58**
Group level of analysis
Workshop vs. control group 1, 12 0.13 9.31* 12.25**
Workshop vs. feedback only 1, 12 0.04 1.07 13.30**
* p ⬍ .05. ** p ⬍ .01.
566
RESEARCH REPORTS
Discussion
The results from our field experiment on multisource feedback
provide evidence for the potential effectiveness of this popular
developmental technique. The managers in the feedback workshop
significantly increased their use of two core tactics to influence
subordinates, whereas managers in the control group did not
change their use of any core tactics. Our study was also the first to
evaluate the effects of a feedback facilitator who was not the
manager’s boss. The results show that having a competent, sup-
portive facilitator increases the perceived utility of the feedback
and results in more behavior change for the managers. Effect sizes
ranged from moderate to large for all analyses (Cohen, 1988).
Limitations
Like all studies, ours had some limitations. We were able to
assess the effect of the feedback workshop on influence behavior
toward subordinates, but we could not adequately assess the effects
on influence behavior toward peers and bosses. This research
question deserves more attention, but the research would require a
much larger organization in which it would be possible to insulate
peer and boss raters from confounding factors.
Like the 14 prior studies on the effects of feedback to managers,
our study used behavior change as the criterion. Without a measure
of managerial effectiveness, it is not possible to determine if
changes in influence behavior also increase the effectiveness of the
managers. However, because the core tactics have been found to
be related to independent criteria of managerial effectiveness in
other studies (e.g., Yukl & Tracey, 1992), it is reasonable for one
to infer that managerial effectiveness would be increased by
greater use of the core tactics. The implications of multisource
feedback for managerial effectiveness need to be directly exam-
ined in future research.
Unlike the main part of the study, the comparison of the feed-
back workshop to the feedback-only group was a quasi-
experimental design. Possible differences between the two banks
may account for the findings. The results should be verified in a
follow-up study conducted in an organization large enough to
permit random assignment to all conditions.
Effects of Feedback on Different Tactics
Because this study was the first to assess the utility of a feed-
back workshop for improving influence behavior, there was little
basis for determining in advance if some tactics would be easier to
change than others. We had hoped that the feedback workshop
would increase the use of all four core tactics, and it is not clear
why only consultation and collaboration were affected. One ex-
planation why rational persuasion did not increase is that there was
already considerable use of this tactic at the time of the pretest
(M ⫽ 3.8). There is some evidence that managers tend to focus
more on changing behaviors that seem to be deficient (Hezlett &
Ronnkvist, 1996). In addition, our workshop did not attempt to
teach participants the necessary skills to use complex forms of
rational persuasion (e.g., how to present facts and evidence in a
convincing way).
The lack of an increase in the use of inspirational appeals may
reflect the nature of the work in a savings bank. Comments by
some of the managers during the workshop indicated that they did
not feel comfortable using ideological appeals in their influence
attempts with other members of the bank. Another explanation is
that our workshop did not teach the skills necessary to use inspi-
Figure 1. Change in mean rating of core tactics in the experimental and control groups.
567
RESEARCH REPORTS
rational appeals effectively (e.g., how to identify the values and
ideals of the target person, how to compose relevant ideological
and emotional appeals). It is likely that more skill is required for
use of inspirational appeals than for consultation or collaboration.
Facilitating Conditions
The effectiveness of multisource feedback may depend in part
on facilitating conditions during the feedback process and after-
ward. Examples include relevant skill training, incentives for be-
havior change, and a supportive climate (Antonioni, 1996; Kluger
& DeNisi, 1996; London & Smither, 1995). As yet, it is still not
clear how much these conditions can enhance the effects of feed-
back or the extent to which one condition may substitute for
another. A better understanding of facilitating conditions would be
valuable in designing effective feedback interventions. The rele-
vance of each facilitating condition for our study will be consid-
ered briefly.
Providing training or follow-up coaching can enhance the ef-
fectiveness of behavioral feedback. The potential benefits include
a better understanding of the behaviors, more skill in the effective
use of the behaviors, and increased self-confidence about using the
behaviors. Skill training can be included in a feedback workshop
or as a follow-up activity. In the latter case, the feedback can be
used to help managers decide what type of training or coaching is
most relevant. Evidence that feedback is more effective when
followed by relevant training was found in studies by Hazucha,
Hezlett, & Schneider (1993) and Wilson, O’Hare, & Shipper
(1990) but not in the study by Rosti and Shipper (1998). Our
feedback workshop included some activities to increase under-
standing of the influence tactics, but there was little opportunity for
developing skill in actually using the tactics. The 3 hrs of training
we provided was much less than the 24 hrs of training (over a 3
month period) in the study by Rosti and Shipper, or the 3 weeks of
residential training in the bank study by Wilson et al. Further
increasing the influencing skills of managers would require much
more training than we provided. At the minimum, each participant
should have an opportunity to practice using each core tactic (e.g.,
in a role play) and receive specific feedback (e.g., by viewing a
videotape of the role play). This type of training would involve
extending the workshop by at least another day or adding
follow-up training.
Another facilitating condition is a supportive climate for behav-
ior change (Facteau, Facteau, Russell, & Poteet, 1998; Hazucha et
al., 1993). The climate appeared supportive at the beginning of our
study, but it deteriorated rapidly. After the premeasure survey was
completed, there was a hostile takeover attempt of our primary
bank by two other regional banks. The focal managers in the study
were heavily involved in dealing with the external threats. Many of
the managers reactively negatively, because a takeover would
result in elimination of some jobs at their level and would require
difficult adjustments for anyone who remained. In the absence of
a takeover crisis to distract attention from applying what was
learned, the behavior change may have been greater. The second
bank was also involved in the takeover, which may be one reason
why the feedback-only intervention had no beneficial effect on
behavior.
The appraisal process and reward system can also affect moti-
vation to use behavioral feedback. The absence of stronger effects
in most feedback studies may reflect a lack of participant concern
for correcting any weaknesses revealed by the feedback. In our
study, as in most of the prior studies, the feedback intervention was
entirely developmental. Some scholars (e.g., London, Smither, &
Adsit, 1997) have proposed that multisource feedback would be
more effective if recipients were accountable for using it to im-
prove their performance. Even when the primary objective is
development rather than appraisal, the improvement goals and
action plans can provide a basis for subsequent evaluation of the
manager. Another way to increase motivation to use developmen-
tal feedback would be to ensure that it includes behaviors and
skills used to appraise manager performance and make promotion
decisions.
In summary, we found that a multisource feedback workshop
can improve managerial behavior even when there is little skill
training, a nonsupportive climate, and no extrinsic inducements.
Increasing the effectiveness of the feedback beyond the level
achieved in our study would probably require more of the facili-
tating conditions. Learning how to create these conditions should
be a primary objective of future research on behavioral feedback
programs.
References
Antonioni, D. (1995). Problems associated with implementation of an
effective upward appraisal process: An experimental field study. Human
Resource Development Quarterly, 6, 157–171.
Antonioni, D. (1996). Designing an effective 360-degree appraisal feed-
back process. Organizational Dynamics, 25, 24–38.
Atwater, L., Roush, P., & Fischthal, A. (1995). The influence of upward
feedback on self and follower ratings of leadership. Personnel Psychol-
ogy, 48, 35–59.
Atwater, L., Waldman, D. A., Atwater, D., & Cartier, P. (2000). An upward
feedback field experiment: Supervisors’ cynicism, reactions, and com-
mitment to subordinates. Personnel Psychology, 53, 275–297.
Bernardin, H. J., Dahmus, S. A., & Redmon, G. (1993). Attitudes of
first-line supervisors toward subordinate appraisals. Human Resource
Management, 32, 315–324.
Bracken, D. W. (1994). Straight talk about multirater feedback. Training
and Development, 48, 44–50.
Carver, C. S., & Scheier, M. F. (1981). Attention and self-regulation: A
control theory approach to human behavior. New York: Springer-
Verlag.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences.
Hillsdale, NJ: Erlbaum.
DeNisi, A. S., & Kluger, A. N. (2000). Feedback effectiveness: Can
360-degree appraisals be improved? Academy of Management Execu-
tive, 14, 129–139.
Duncan, D. B. (1995). Multiple range and multiple F tests. Biometrics, 11,
1–42.
Edwards, M. R., & Ewen, A. J. (1996). 360-degree feedback: The powerful
new model for employee assessment and performance improvement.
New York: American Management Association.
Facteau, C. L., Facteau, J. D., Russell, J. E. A., & Poteet, M. L. (1998).
Reactions of leaders to 360-degree feedback from subordinates and
peers. Leadership Quarterly, 9, 427–448.
Greguras, G. J., & Robie, C. (1998). A new look at within-source interrater
reliability of 360-degree feedback ratings. Journal of Applied Psychol-
ogy, 83, 960–968.
Hazucha, J. F., Hezlett, S. A., & Schneider, R. J. (1993). The impact of
360-degree feedback on management skills development. Human Re-
source Management, 32, 325–351.
568
RESEARCH REPORTS
Hegarty, W. H. (1974). Using subordinate ratings to elicit behavior
changes in supervisors. Journal of Applied Psychology, 59, 764–766.
Hellervik, L. W., Hazucha, J. F., & Schneider, J. (1992). Behavior change:
Models, methods, and a review of the evidence. In M. D. Dunnette &
L. M. Hough (Eds.), Handbook of industrial and organizational psy-
chology (2nd ed., Vol. 3, pp. 821–895). Palo Alto, CA: Consulting
Psychologists Press.
Hezlett, S. A., & Ronnkvist, A. M. (1996). The effects of multi-rater
feedback on managers’ skill development: Factors influencing behavior
change. Paper presented at the Society for Industrial and Organizational
Psychology, San Diego, CA.
Johnson, J. W., & Ferstl, K. L. (1999). The effects of interrater and
self–other agreement on performance improvement following upward
feedback. Personnel Psychology, 52, 271–303.
Kipnis, D., Schmidt, S. M., & Wilkinson, I. (1980). Intra-organizational
influence tactics: Explorations in getting one’s way. Journal of Applied
Psychology, 65, 440–452.
Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions
on performance: A historical review, a meta-analysis, and a preliminary
feedback intervention theory. Psychological Bulletin, 119, 254–284.
Lepsinger, R., & Lucia, A. D. (1997). The art and science of 360-degree
feedback. San Francisco: Pfeiffer.
London, M., & Smither, J. W. (1995). Can multi-source feedback change
perceptions of goal accomplishment, self-evaluations, and performance-
related outcomes? Theory-based applications and directions for research.
Personnel Psychology, 48, 803–839.
London, M., Smither, J. W., & Adsit, D. J. (1997). Accountability: The
Achilles’ heel of multisource feedback. Group and Organization Man-
agement, 22, 162–184.
Meyer, H. (1980). Self-appraisal of job performance. Personnel Psychol-
ogy, 33, 291–295.
Nemeroff, W. F., & Cosentino, J. (1979). Utilizing feedback and goal
setting to increase performance appraisal interview skills of managers.
Academy of Management Journal, 22, 566–576.
Noe, R. A. (1999). Employee training & development. New York:
McGraw-Hill/Irwin.
Reilly, R. R., Smither, J. W., & Vasilopoulos, N. L. (1996). A longitudinal
study of upward feedback. Personnel Psychology, 49, 599–612.
Rosti, R. T., & Shipper, F. (1998). A study of the impact of training in a
management development program based on 360-degree feedback. Jour-
nal of Managerial Psychology, 13, 77–89.
Schriesheim, C. A., & Hinkin, T. R. (1990). Influence tactics used by
subordinates: A theoretical and empirical analysis and refinement of the
Kipnis, Schmidt, and Wilkinson subscales. Journal of Applied Psychol-
ogy, 75, 246–257.
Smither, J. W., London, M., Vasilopoulos, N. L., Reilly, R. R., Millsap,
R. E., & Salvemini, N. (1995). An examination of the effects of an
upward feedback program over time. Personnel Psychology, 46, 1–34.
Tornow, W. W., London, M., & CCL Associates. (1998). Maximizing the
value of 360-degree feedback: A process for successful individual and
organizational development. San Francisco: Jossey-Bass.
Waldman, D. A., & Atwater, L. E. (1998). The power of 360-degree
feedback: How to leverage performance evaluations for top productivity.
Houston, Texas: Gulf Publishing Company.
Waldman, D. A., Atwater, L. E., & Antonioni, D. (1998). Has 360-degree
feedback gone amok? Academy of Management Executive, 12, 86–94.
Walker, A. G., & Smither, J. W. (1999). A five-year study of upward
feedback: What managers do with their results matters. Personnel Psy-
chology, 52, 393–423.
Wilson, C. L., O’Hare, D., & Shipper, F. (1990). Task cycle theory: The
processes of influence. In K. E. Clark & M. B. Clark (Eds.), Measures
of Leadership (pp. 185–204). West Orange, NJ: Leadership Library of
America.
Yammarino, F., & Atwater, L. (1993). Understanding self perception
accuracy: Implications for human resource management. Human Re-
sources Management, 32, 231–247.
Yukl, G. (2002). Leadership in organizations (5th ed.). Upper Saddle
River, NJ: Prentice Hall.
Yukl, G., & Lepsinger, R. (1995). How to get the most out of 360-degree
feedback. Training, 32, 45–50.
Yukl, G., & Seifert, C. (2002). Validation of the extended version of the
Influence Behavior Questionnaire. Poster session presented at the annual
meeting of the Society for Industrial and Organizational Psychology,
Toronto, Ontario, Canada.
Yukl, G., & Tracey, J. B. (1992). Consequences of influence tactics used
with subordinates, peers, and the boss. Journal of Applied Psychol-
ogy, 77, 525–535.
Received January 22, 2002
Revision received September 4, 2002
Accepted September 11, 2002 䡲
569
RESEARCH REPORTS
