Content uploaded by Kenneth P. De Meuse
Author content
All content in this area was uploaded by Kenneth P. De Meuse on Jul 27, 2015
Content may be subject to copyright.
PLEASE SCROLL DOWN FOR ARTICLE
This article was downloaded by:
[De Meuse, Kenneth P.]
On:
28 October 2009
Access details:
Access Details: [subscription number 916243541]
Publisher
Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,
37-41 Mortimer Street, London W1T 3JH, UK
Coaching: An International Journal of Theory, Research and Practice
Publication details, including instructions for authors and subscription information:
http://www.informaworld.com/smpp/title~content=t762290973
Evaluating the effectiveness of executive coaching: beyond ROI?
Kenneth P. De Meuse a; Guangrong Dai a; Robert J. Lee b
a Korn/Ferry Leadership and Talent Consulting, Minneapolis, USA b iCoach, New York, USA
Online Publication Date: 01 September 2009
To cite this Article De Meuse, Kenneth P., Dai, Guangrong and Lee, Robert J.(2009)'Evaluating the effectiveness of executive
coaching: beyond ROI?',Coaching: An International Journal of Theory, Research and Practice,2:2,117 — 134
To link to this Article: DOI: 10.1080/17521880902882413
URL: http://dx.doi.org/10.1080/17521880902882413
Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf
This article may be used for research, teaching and private study purposes. Any substantial or
systematic reproduction, re-distribution, re-selling, loan or sub-licensing, systematic supply or
distribution in any form to anyone is expressly forbidden.
The publisher does not give any warranty express or implied or make any representation that the contents
will be complete or accurate or up to date. The accuracy of any instructions, formulae and drug doses
should be independently verified with primary sources. The publisher shall not be liable for any loss,
actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directly
or indirectly in connection with or arising out of the use of this material.
Evaluating the effectiveness of executive coaching: beyond ROI?
Kenneth P. De Meuse
a
*, Guangrong Dai
b
, and Robert J. Lee
c
a,b
Korn/Ferry Leadership and Talent Consulting, 5051 Highway 7, Minneapolis, 55416 USA;
c
iCoach, 1 Penn Plaza, New York, 10119 USA
(Received 21 November 2008; final version received 16 February 2009)
The popularity of executive coaching has increased dramatically in both the
practitioner world and academia during the past decade. However, evaluating the
effectiveness of coaching has lagged behind. Executive coaching is a multi-
disciplinary practice, and professionals from many different scholarly back-
grounds provide coaching services. The paucity of empirical research may be
attributed to the lack of a consensus among these divergent professionals
regarding whether and how to evaluate the effectiveness of coaching. In this
article, we conducted a meta-analysis of the empirical research as well as reviewed
the retrospective studies evaluating coaching effectiveness. Subsequently, we
discussed six areas that impact the way researchers evaluate coaching effectiveness
and the conclusions they may draw from their studies. Although the Return On
Investment (ROI) index provides a straightforward, overall measure of effective-
ness, its veracity and usefulness is questioned. It is hoped that the clarification of
these areas will help guide the future of coaching evaluation research and practice.
Keywords: coaching; research; measurement; ROI; evaluation; effectiveness;
methodology
Introduction
The use of executive coaching has increased markedly during recent years and now is
regarded as a major leadership development practice in corporate organizations
around the world (Corporate Leadership Council, 2003). It has been estimated that
more than 70% of organizations with formal leadership development initiatives
employ coaching as an important part of that mix (Zenger & Stinnett, 2006). The
growing popularity of executive coaching also can be witnessed in the academic
literature. Grant and Cavanagh (2004) conducted a bibliographical review of
coaching studies and found a sharp increase in peer-reviewed journal publications
during recent years. Similarly, English (2006) investigated the executive coaching
literature, finding an increase of 300% in published articles in scholarly journals from
19941999 to 20002004.
While executive coaching is gaining popularity, the professional application of
coaching, our understanding of when to use coaching, and the evaluation of its
effectiveness has lagged far behind. A recent Harvard Business Review Research
Report asserted: ‘The coaching field is filled with contradictions. Coaches themselves
disagree over why they’re hired, what they do, and how to measure success’ (Coutu &
*Corresponding author. E-mail: ken.demeuse@kornferry.com
ISSN 1752-1882 print/ISSN 1752-1890 online
#2009 Taylor & Francis
DOI: 10.1080/17521880902882413
http://www.informaworld.com
Coaching: An International Journal of Theory, Research and Practice
Vol. 2, No. 2, September 2009, 117134
Downloaded By: [De Meuse, Kenneth P.] At: 02:10 28 October 2009
Kauffman, 2009, p. 26). In practice, only a small proportion of organizations appear
to be formally evaluating the impact of coaching (Bolch, 2001; Hay Group, 2002). In
academia, review studies consistently have concluded that there is a paucity of
empirical data to support the anecdotal evidence that coaching produces positive
outcomes (Feldman & Lankau, 2005; Kampa-Kokesch & Anderson, 2001; MacKie,
2007).
Although the published literature has begun to reflect the emergence of
professional coaches (Grant & Cavanagh, 2004), rigorous empirical investigations
of the outcomes associated with coaching are greatly outnumbered by practitioner
articles purporting the benefits of it. The majority of the studies evaluating coaching
are retrospective in nature (Evers, Brouwers, & Tomic, 2006) and use short-term,
affective reactions or perceptions of effectiveness as outcomes (Feldman & Lankau,
2005). To improve our understanding of whether executive coaching actually makes
a difference on leadership behavior as well as on other individual and organizational
outcomes (e.g., employee job satisfaction, employee productivity, organizational
performance), much more rigorous research is needed.
However, executive coaching is a multidisciplinary practice. Professionals from
many different academic backgrounds such as psychology, human resources, social
work, marketing, training and development, and business provide coaching services
(Liljenstrand & Nebeker, 2008). There is no professional consensus as to what
constitutes effective coaching and little in the way of guidelines and protocols to
inform coaching evaluation. MacKie (2007) reviewed the training and psychother-
apy literatures and explored how we can learn lessons from these two bodies of
research in evaluating the effectiveness of executive coaching. The purpose of this
paper is to continue this effort.
The current paper contains two parts. First, we briefly review the literature to
understand the current state of the evidence supporting the efficacy of executive
coaching. When possible, we use empirically based studies to conduct a meta-
analysis of executive coaching outcomes. The remaining studies are carefully
reviewed to glean additional information regarding coaching effectiveness. We then
address six key issues related to the evaluation of coaching effectiveness. These areas
reflect factors which likely impact the efficacy of coaching and should be considered
by researchers and practitioners alike when attempting to measure its effectiveness.
Review of executive coaching studies
Our goal was to conduct a comprehensive meta-analysis of the research evaluating
the effectiveness of executive coaching. We used PsychInfo and Business Source
Premier to search for published executive coaching effectiveness studies. We also
searched for unpublished studies through cross-references. A total of 22 studies were
identified. Unfortunately, many of the studies used a retrospective methodology,
failing to collect pre-post assessments necessary for performing a meta-analysis. The
focus of those studies was to ask participating individuals (and in some instances, the
coaches themselves or the coachees’ managers) to simply report their perceptions of
the effectiveness of coaching after it was completed. Other studies did not report
sufficient information to calculate an effect size (cf. Burke & Day, 1986; Smither,
London, & Reilly, 2005). Consequently, we divided the 22 research studies into two
categories: (a) empirical studies from which the gain or improvement from coaching
118 K.P. De Meuse, G. Dai, and R.J. Lee
Downloaded By: [De Meuse, Kenneth P.] At: 02:10 28 October 2009
can be directly observed or indirectly inferred, and (b) retrospective studies where the
perceived coaching effectiveness was reported (typically through a survey or an
interview). Initially, we performed a meta-analysis on those empirical studies that
met the necessary methodological and statistical requirements. Subsequently, we
conducted a content analysis on the remaining retrospective studies.
Meta-analysis of the empirical studies
The following four criteria had to be satisfied for inclusion in the meta-analysis: (a)
the focus of the intervention had to be exclusively executive coaching; (b) the services
of an external coach were used; (c) the methodological design was one that provided
both pre- and post-coaching ratings; and (d) statistics, such as means, standard
deviations or tvalues, and sample sizes had to be reported in the article so that an
effect size could be computed.
Overall, 12 empirical studies were identified in the literature. Two of the studies
used internal coaches and therefore were excluded from further analysis (Ellinger,
1999; Olivero, Bane, & Kopelman, 1997). One study used a student sample and was
excluded (Sue-Chan & Latham, 2004). In addition, one study did not report pre- and
post-coaching ratings and was eliminated (Thach, 2002). Two other studies were not
suitable for the meta-analysis, because they were based on single cases (Libri &
Kemp, 2006; Orenstein, 2006). Consequently, only six of the original 12 empirical
studies met these criteria and were included in our meta-analysis.
All six of the studies examined executive skill and/or job performance
improvement. Two studies also investigated coaching outcomes related to individuals
reporting to the coached executives (e.g., their job attitudes; Luthans & Peterson,
2003) and their organizations (e.g., clarification of organizational vision and mission;
Wofred, 2003). Overall, two different types of ratings were collected. Four of the
studies measured the responses from the coachee himself or herself (referred to as
‘self-ratings’; Evers et al., 2006; Luthans & Peterson, 2003; Peterson, 1993; Wofred,
2003). In contrast, four of the studies assessed the responses of the coach or the
coachee’s manager (referred to as ‘others’ ratings’; Luthans & Peterson, 2003;
Peterson, 1993; Smither, London, Flautt, Vergas, & Kucine, 2003; Togel &
Nicholson, 2005). We conducted separate meta-analyses on skill/performance
improvement for the two categories of rater sources. We readily acknowledge that
the number of studies included in our analysis is small. Thus, caution should be used
when generalizing the findings.
We followed the meta-analytic procedure described in Burke and Day (1986).
Initially, a specific effect size was calculated on the criterion variable in each study. If
a study had more than one criterion measure, the average effect size was used for that
study to remove sampling redundancy. Subsequently, this effect size was corrected
for unreliability. The ‘estimated population true effect size’ is the sample size of the
study weighted by the observed effect size and corrected for sampling error and
unreliability. Table 1 presents the results of the meta-analyses for self and others’
ratings, respectively.
An effect size (d) simply is a way of quantifying the difference between pre- and
post-coaching ratings. It is a statistical way of measuring the effectiveness of a
coaching engagement. According to Cohen (1988), an effect size less than 0.30 would
be considered small, an effect size between 0.31 and 0.50 would be moderate, and an
Coaching: An International Journal of Theory, Research and Practice 119
Downloaded By: [De Meuse, Kenneth P.] At: 02:10 28 October 2009
effect size greater than 0.50 would be large. As can be seen, executive coaching
generally leads to a moderate-to-large amount of improvement in the coachees’ skill
and/or performance ratings. This improvement is perceived to be relatively larger by
the coachees themselves an effect size of 1.27 as opposed to others (M
d
0.50).
Thus, self-ratings of improvement were higher than others’ ratings of improvement.
Sampling error variance explained only a small portion of variation (i.e., less than
10%) across studies, suggesting that the effectiveness of coaching is really situation
specific (Hunter & Schmidt, 1995). This finding reinforces Corporate Leadership
Council’s (2003) observation that coaching results in inconsistent returns.
The 90% credibility value was above zero for self-ratings, indicating that coachees
most of the time perceived a positive impact from coaching. In contrast, the 90%
credibility value for others’ ratings was below zero, suggesting that coaching at times
lead to some negative outcomes. There could be two reasons for this result. First, the
assessment method might be flawed. We will discuss this possibility in detail in a later
section of this paper. Secondly, executive coaching indeed may have an adverse
impact on some coachees. For example, MacKie (2007) questioned whether executive
coaching always had a positive impact. Kilburg (2000) identified several factors in
coachees (e.g., lack of motivation, unrealistic expectations, lack of follow-up) and
coaches (e.g., insufficient empathy, lack of expertise in the area of concern, poor
techniques) that mitigate against a successful coaching outcome.
In sum, our meta-analysis found an overall positive effect for executive coaching.
However, readers should be cautioned that this impact can be inconsistent from one
coaching engagement to another. Unfortunately, the inadequate number of studies in
the meta-analysis and the failure of authors to report sufficient information about
the coaching intervention itself preclude us from identifying when coaching will or
will not lead to positive results.
Content analysis of retrospective studies
We located 10 retrospective studies in the literature. Because many retrospective
studies included evaluations of multiple criteria, we applied Kirkpatrick’s (1977)
model of evaluating training interventions to frame the review. However, we used
three levels of evaluation rather than four as prescribed in the Kirkpatrick model.
Unlike training, coaching usually does not intend to enhance the coachees’
knowledge base in an area. Rather, the purpose of most coaching engagements is
to change or improve skills or performance. Consequently, the three levels of
evaluation used in our analysis include: (a) reactions to coaching; (b) coaching
effectiveness (as assessed through change or improvement in skills or performance at
Table 1. Results of executive coaching on skill/performance ratings.
Rater Category KNV
obs
V
e
Variance Accounted For M
d
V
d
90% C.V.
Self Ratings 4 173 0.72 0.04 5% 1.27 0.67 0.23
Others’ Ratings 4 591 0.40 0.01 2% 0.50 0.45 0.35
Note.Kreflects the number of studies measuring this rating. Ndenotes the cumulative sample size of these
studies. V
obs
is the observed effect size variance. V
e
is sampling error variance. M
d
is the estimated
population true effect size corrected for sampling error and unreliability. V
d
is the estimated true effect size
variance. C.V. denotes credibility value.
120 K.P. De Meuse, G. Dai, and R.J. Lee
Downloaded By: [De Meuse, Kenneth P.] At: 02:10 28 October 2009
the individual level); and (c) coaching impact at the organizational level. The
evaluation methodology employed by most of the retrospective studies was a post-
test measure, with ratings typically obtained via a survey immediately after the
coaching was completed. Reported success was in terms of percentage of favorable
responses. Table 2 presents the results from these 10 retrospective studies. In
addition, we reviewed the three empirical based studies that were excluded from the
meta-analysis due to statistical or sampling concerns (cf. Libri & Kemp, 2006;
Orenstein, 2006; Thach, 2002).
Overall, the findings from this second group of studies were very positive.
Coachees generally were favorable to and satisfied with the coaching provided to
them. The percentage of participants reporting favorable ratings in different studies
ranged between 75% and 95%. For example, Wasylyshyn (2003) surveyed executives
who had been coached between 1985 and 2001. Of the 87 executives, nearly 47% were
‘positive toward the coaching’ and an additional 29% expressed ‘enthusiastic
anticipation of coaching.’ In general, respondents rated coaching effective at
improving their skills and performance, particularly in areas related to the coaching
objectives. The coaching also was perceived to have a significant impact on some
business results (e.g., productivity, business deliverables). Several authors also
provided estimates of return on investment (ROI) of the coaching intervention.
For example, McGovern, Lindemann, Vergara, Murphy, Barker, and Warrenfeltz
(2001) reported that executives realized improvements in productivity, quality,
organizational strength, customer service, and shareholder value, realizing an
average ROI of almost six times the cost of coaching. Likewise, Parker-Wilkins
(2006) and Anderson (2001) estimated an ROI approximating 700% due to coaching.
While the retrospective studies support the effectiveness of executive coaching,
they suffer from several methodological problems. For example, coachees may
possess hindsight bias, in that their memories may have influenced the data collected
(Evers et al., 2006). Furthermore, the managers who experienced coaching might
have perceived that it was in their personal interest to report that the coaching
worked. Cognitive dissonance also might have been a factor (Festinger & Carlsmith,
1959). After all, their organization spent a lot of money on hiring a coach, plus they
personally devoted much effort and time to various coaching activities. It is
reasonable to expect that they should report their performance improved. In
addition, the evaluation of coaching success in some retrospective studies was
conducted by the consulting firm providing the coaching. Although we are not
questioning the veracity of the effectiveness ratings, it is in the self-interest of the
consulting organization to find positive results. MacKie (2007) declared that data
from such studies are ‘at the level of collective anecdotes’ (p. 311) and have limited
usefulness in demonstrating that coaching is effective. Hence, caution is needed when
interpreting reported outcomes of these studies, particularly when the research has
been conducted by individuals or companies with a vested interest in demonstrating
success.
Key factors related to the evaluation of executive coaching
As evident in our review, the effectiveness of coaching has been investigated in a
variety of different ways (e.g., use of retrospective versus pre-post research designs,
soliciting the coachee’s views versus the coach’s observations, obtaining global
Coaching: An International Journal of Theory, Research and Practice 121
Downloaded By: [De Meuse, Kenneth P.] At: 02:10 28 October 2009
Table 2. Results of retrospective studies.
Reaction to Coaching
80% of the participants were favorable to the coaching (Talboom, 1999).
‘Very Satisfactory’ ... This was the way clients most frequently rated the overall
effectiveness of their coaching experience on a 5-point scale, where 5 was very
satisfactory (Hall, Otazo & Hollenback, 1999).
Over 75% of executive reactions to the idea of working with a coach were significantly
positive (Wasylyshyn, 2003).
Respondents were very satisfied with coaching 86% rated coaching as very effective;
95% stated they were doing things differently as a result of coaching; and 95%
indicated they would recommend coaching to other staff members (Parker-Wilkins,
2006).
Coaching Effectiveness on the Individual Level
To determine whether coaching contributed to sustained behavioral change, Genger
combined the percentages of ‘highly effective’ and ‘somewhat effective’ statements for
awareness and responsibility as these statements were considered coaching outcomes.
Percentages ranged from 70.7% to 93.8%, suggesting that coaching contributed to
sustained behavioral change (Genger, 1997).
At the level of learning, 7090% of the participants were favorable to the coaching. At
the behavioral level, it was over 50% (Talboom, 1999).
Participants considered 73% of goals to have been achieved ‘very effectively’ or
‘extremely effectively.’ Other stakeholders were more conservative, evaluating 54% of
goals as having been achieved with this level of effectiveness, and 85% as having
delivered results ‘effectively’ or higher (McGovern et al., 2001).
55% of the participants increased leadership effectiveness as rated by others; 52%
increased as rated by self (Thach, 2002).
The top three indications of successful coaching were (a) sustained behavioral change
(63%), (b) increased self-awareness and understanding (48%), and (c) more effective
leadership (45%). On a 110 scale, over half of these coached executives reported a
sustainability level between 6 and 8; over a third was at the 910 level (Wasylyshyn,
2003).
75% of survey respondents rated the effectiveness of coaching as a 3 or higher on a
scale of 1 to 5, with 1 being ‘not effective’ and 5 being ‘very effective.’ Only 15% of
respondents rated coaching as a 1 or a 2 (‘Is Coaching Worth the Money,’ 2005).
Executives improved significantly and mostly on behavioral dimensions related to the
coaching objectives (15 of the 19 items, 79%), some on behavioral dimensions
indirectly related to the coaching objectives (4 of the 11 items, 36%), but none on
behavioral dimensions that were unrelated to the coaching objectives (Orenstein,
2006).
Respondents stated that coaching assisted them in the development of three main
competencies: (a) leadership behaviors (82%), (b) building teams (41%), and (c)
developing staff (36%; Parker-Wilkins, 2006).
Coaching enhanced the executive’s sales performance, core self-evaluation, and global
self-ratings of performance (Libri & Kemp, 2006).
96% of organizations reported that individual performance improved since coaching
was introduced. Nearly as many (92%) also have seen improvements to leadership and
management effectiveness. Of these organizations, 45% and 39%, respectively, have
seen significant or major improvements (‘Coaching Counts,’ 2007).
122 K.P. De Meuse, G. Dai, and R.J. Lee
Downloaded By: [De Meuse, Kenneth P.] At: 02:10 28 October 2009
perceptions versus dimensional ratings, measuring behavioral changes versus skill
improvement, assessing individual-level improvement versus organizational-level
outcomes). Some studies have attempted to measure ROI. Obviously, researchers
have different opinions regarding how coaching should be evaluated. In this section,
we address several key issues related to the evaluation of executive coaching. Initially,
we review three factors directly related to the measurement of coaching effectiveness,
including: (a) the purpose of evaluation; (b) the evaluation criteria used; and (c) the
rigor of the evaluation employed. Subsequently, we consider factors related to (d)
the type of coaching provided; (e) the content of coaching engagement itself; and (f)
the coaching methodology a coach applies. These latter factors likely serve as
moderating conditions influencing the overall effectiveness of the executive coaching
assignment. It is hoped that a careful examination of these six factors will help
develop a consensus among researchers and practitioners regarding how executive
coaching should or should not be evaluated.
Measurement and methodology issues
Purpose of the coaching evaluation. From a program evaluation perspective, there
basically are two different types of evaluation summative and formative. A
summative evaluation measures the outcome of a completed intervention program. In
contrast, a formative evaluation assesses current, ongoing program activities,
provides an internal process that compares the planned program with the actual
program, and measures the progress toward meeting the program goals.
Executive coaching practitioners and researchers alike need to conduct summa-
tive evaluations to demonstrate the merits of coaching. Did the coaching accomplish
the client’s objectives or not? However, executive coaching very often is characterized
as a sequential process. Coaches have to carefully plan and execute a number of
stages during the engagement for the achievement of the ultimate coaching
objectives. Process elements, such as the relationship between coach and coachee,
are critical for coaching success (Kampa-Kokesch & Anderson, 2001). Leedham
(2005) identified five success criteria of coaching. Three of them are related to the
coaching process itself. Similarly, Bush (2005) in her dissertation study investigated
how clients consider an executive coaching event effective. She found that process-
related factors (e.g., focus on development, rapport with the coach) were one type of
success criteria. Given the impact that the quality of coaching has on its
effectiveness, it is necessary for coaches to integrate formative, ongoing evaluation
Coaching Impact at the Organizational Level
At the organizational level, the participants asserted they witnessed lower rates of
absence among subordinates (Talboom, 1999).
77% of the respondents indicated that coaching had significant or very significant
impact on at least one of nine business measures. Productivity (60% favorable) and
employee satisfaction (53%) were cited as the most significantly impacted by the
coaching (Anderson, 2001).
35% improved on leadership. 28% improved as management team. 33% improved on
business deliverables. And 67% improved on personal balance (‘When Coaching
Measures Up,’ 2005).
Coaching: An International Journal of Theory, Research and Practice 123
Downloaded By: [De Meuse, Kenneth P.] At: 02:10 28 October 2009
into their coaching. Coaches should continuously monitor the coaching process,
identify problems in a timely manner, and adjust and readjust their coaching
activities as the process unfolds. Evaluating executive coaching requires an integrated
framework that includes both summative and formative components (Ely, Nelson,
Boyce, & Zaccaro, 2008).
Criteria used to evaluate coaching. Related to this issue are two questions. First, is the
ultimate criterion of coaching ROI? Second, can there be a single outcome criterion
for all coaching interventions? Coaching is expensive. A six-month engagement for
one executive can range between $15,000 and $75,000. Moreover, coaching requires
regular intervals of highly compensated executive time (Johnson, 2007). Organiza-
tions are asking hard questions about the ROI. Human Resources (HR) and talent
management leaders want to control expenses spent on external coaches, ascertain
the organizational and financial impact of such coaching, and justify coaching
budgets to senior leadership. The request for the evidence of ROI is legitimate, but
not always practical. Executive coaching has been used to enhance skills and improve
performance in a wide range of organizational arenas. It can have tangible and
intangible effects on organizational effectiveness to varying degrees. McDermott,
Levenson, and Newton (2007) found that coaching has a larger positive impact on
micro-level outcomes (e.g., improving leadership behaviors and individual employ-
ees’ performance) than on macro-level outcomes (e.g., strategy execution and change
management). It is likely that a coaching intervention is too many causal linkages
away from financial results to demonstrate direct and significant relationships
(Feldman & Lankau, 2005).
Some researchers have tried to quantify the ROI of executive coaching. For
example, one case study in a Fortune 500 company reported that coaching produced
an ROI of 529%. This figure was boosted to an overall ROI of 788% when including
the financial benefits of enhanced employee retention (Anderson, 2001). Another
study claimed that when calculated conservatively, ROI averaged nearly $100,000 or
5.7 times the organization’s initial investment in coaching (McGovern et al., 2001).
A close examination of these two studies indicates that the ROI numbers were
derived subjectively through retrospective questionnaires and interviews. We do not
criticize such methodology out of hand. It simply becomes very tenuous to draw
firm cause-and-effect financial conclusions through such a perceptual-based,
qualitative process, and generalize the numbers to other coaching events. Overall,
the difficulty with ROI investigations may be inherent in executive coaching itself.
It should be noted that the computation of ROI appears to be largely context
specific. The ROI metric depends on two key factors: (a) the overall cost an
organization incurs implementing the coaching engagement, and (b) the financial
benefits it obtains. These items are extremely difficult to measure and are highly
unique to the specific coaching assignment. Obviously, the fee structure of different
coaches varies greatly. Likewise, the organizational level and size of company of the
coachee varies greatly. A Chief Executive Officer (CEO) can have a powerful impact
on an organization; a mid-level manager less so. Some executives work on million-
dollar deals, while others work on billion-dollar ones. In order to make meaningful
conclusions regarding ROI and to compare ROI across studies all facets of the
coaching engagement must be considered and measured, including the coaching
costs and executives’ enhanced organizational value. Such factors as the expertise of
124 K.P. De Meuse, G. Dai, and R.J. Lee
Downloaded By: [De Meuse, Kenneth P.] At: 02:10 28 October 2009
the coach, length of coaching engagement, and organizational support for the
coaching also may play a pivotal role in success. In addition, the level of motivation
and commitment of the executive who is being coached must be considered. We are
not stating that it is impossible to compute ROI, but it certainly is very, very difficult
to do so.
After reviewing the psychotherapy literature, MacKie (2007) suggested that a
professional consensus on appropriate outcome domains needs to emerge for valid
comparisons to be made across interventions and studies. We echo MacKie’s
contention. However, based on the above discussion around ROI, we believe that the
omnibus outcome criterion cannot be ROI. Perhaps, the best single outcome
criterion is the coaching objective. Coaching interventions can be evaluated against
the goal(s) for which the coaching is designed to achieve. It has been widely
recommended that executive coaching should have clear coaching objectives (cf.
Valerio & Lee, 2005). Research has found that when organizations fail to clearly
articulate objectives, coaching is far less likely to succeed (McDermott et al., 2007).
For any engagement, the coaching goals and the success criteria should be
understood and agreed to between the coaches and the clients at the outset of the
coaching relationship (Valerio & Lee, 2005). In practice, however, the coaching
objectives frequently evolve during the coaching engagement (Peterson, 2002;
Valerio & Lee, 2005). Recently, Coutu and Kauffman (2009) surveyed 140 coaches
to understand current realities surrounding executive coaching. They found that
coaches overwhelmingly agreed the focus of coaching sessions typically shifts during
the course of an engagement. Consequently, it seems important to continue to
communicate coaching objectives among the coach, coachee, and other stakeholders
throughout the engagement. It would appear that the ROI of coaching largely
depends on how directly the coaching objectives correspond to business or
organizational requirements (Corporate Leadership Council, 2003). Some coaching
assignments have the potential for creating great change in organizational results;
whereas, others may change only one individual’s satisfaction with the job or another
individual’s satisfaction with the executive being coached.
Rigor of the coaching evaluation. Finally, there are three topics related directly to the
rigor of evaluation of coaching itself. The first topic focuses on the research design
implemented to assess executive coaching. It is generally agreed upon that the
research design with the highest methodological rigor is a pre-post, control group
design (Campbell & Stanley, 1963). If researchers used such a design to evaluate
executive coaching, they would be able to control many threats to internal and
external validity. However, typically executive coaching is a one-to-one relationship
between a coach and an executive. The fact that different executives have different
coaching objectives may make a control study not possible. Indeed, it may be
unrealistic to conduct control studies in many field settings (Goldstein, 1986; Sackett
& Mullen, 1993). A pre-post design is probably the most appropriate study with an
adequate level of methodological rigor. Whatever design is implemented, a concerted
effort to employ rigorous evaluation procedures is needed (Feldman & Lankau,
2005).
A second topic related to evaluation rigor focuses on the assessment instrument
employed to measure coaching effectiveness. Many authors contend that a 360-
degree assessment is an important part of the coaching process (Feldman, 2001;
Coaching: An International Journal of Theory, Research and Practice 125
Downloaded By: [De Meuse, Kenneth P.] At: 02:10 28 October 2009
Judge & Cowell, 1997; Wasylyshyn, 2003). Logically, we can compare the pre- and
post-coaching 360-degree assessment data to examine whether the coaching lead to
higher ratings. However, many researchers have opposed direct pre-post comparison
as a measure of improvement for several reasons (see Buda, Reilly, & Smither, 1991;
Smither, Reilly, & Buda, 1988). For instance, the raters themselves may change
between 360-degree evaluations due to employee attrition or the hiring of new
employees. Further, the observed improvement is confounded by beta change and/or
gamma change (Golembiewski, Billingsley, & Yeager, 1976). When beta change
occurs, the rater’s yardstick for the variable shifts or stretches, such that benchmarks
on the scale do not remain constant between the two assessments. Gamma change,on
the other hand, involves a redefinition or reconceptualization of key variables.
Consequently, researchers need to improve the psychometric quality of the 360-
degree assessment or design alternative assessment methods. Smither and Walker
(2001) suggested approaches such as retrospective ratings of the degree of change;
whereby, coworkers rate the extent to which the coachee’s performance has improved
with respect to each of the improvement goals he or she set.
The last topic related to the evaluation rigor of coaching pertains to the
integration of executive coaching with other leadership developmental interventions.
Executive coaching frequently is implemented in conjunction with other develop-
mental activities, such as 360-degree feedback (Luthans & Peterson, 2003; Smither
et al., 2003), goal setting (Grant & Cavanagh, 2007), or managerial training (Olivero
et al., 1997). Preferably, researchers can use control groups to separate the effects of
different interventions. However, as discussed previously, it often is unrealistic to
design control studies in field settings. More importantly, it may be unnecessary to
isolate the effects of the executive coaching engagement from the other develop-
mental activities. If coaching truly is a holistic process which integrates a variety of
interventions, we simply have to demonstrate its overall value to organizational
decision makers rather than delineate specific effects. From a research perspective,
obviously, it is useful to carefully isolate the unique contribution of coaching. A
potential approach to do this is to measure the overall effectiveness of the
intervention. Subsequently, researchers can estimate the effectiveness of coaching
by subtracting the effects due to the other interventions.
Factors moderating the effectiveness of coaching
Type of coaching provided. In general, the literature has shown that recipients of
executive coaching services typically fall into two broad categories: (a) executives
who had performed highly in the past but whose behaviors now are interfering with,
or not sufficient for, current job requirements; and (b) managers who have been
targeted for advancement to the executive level but are missing some specific skills
for those future jobs (London, 2002). Historically, the focus of coaching has been to
help executives who were derailing (McCall, Lombardo, & Morrison, 1988). Today,
it is increasingly common to provide coaches for managers to work on personal
learning, transition, and leadership developmental issues (Coutu & Kauffman, 2009;
Kiel, Rimmer, Williams, & Doyle, 1996).
The effectiveness of executive coaching is probably related to its overall coaching
purpose. The objective of coaching when addressing a ‘derailing’ executive is to
126 K.P. De Meuse, G. Dai, and R.J. Lee
Downloaded By: [De Meuse, Kenneth P.] At: 02:10 28 October 2009
eliminate or modify a behavioral pattern. In contrast, the purpose of developmental
coaching is to strengthen or expand a behavioral pattern. The two types of coaching
require different levels of effort from executives. Executives may be more motivated
toward one type of coaching than the other. Coaching is a helping relationship,
which is formed between an executive who has managerial authority and
responsibility in an organization and a coach. The coach has no formal authority
over the executive (Kilburg, 2000). Consequently, it is largely up to the executive’s
self-control and motivation to follow the coaching agenda and act on the
improvement plan. A coach can be instrumental in encouraging and motivating,
but ultimately, the changes must be embraced by the executive (Witherspoon &
White, 1996). Many executives resist coaching (Goldberg, 2005; McDermott, 1996).
This resistance may be stronger when the type of coaching is remedial rather than
developmental. Coaching would seem to be less successful when executives are
defensive and unwilling to look inward, than in situations of growth, development,
and advancement (Coutu & Kauffman, 2009).
Content of the coaching engagement. The specific coaching content and objectives can
vary widely across engagements, addressing issues beyond what is defined typically as
leadership behavior, such as career counseling and business acumen. Coaching is
probably effective to varying degrees at improving job skills and performance,
depending upon the specific content area being tackled. Some areas are explicit (e.g.,
task and cognitive skills). They can be understood readily and internalized by
executives. Other areas are tacit, embedded typically in individual experiences and
involve intangible factors, such as personal beliefs, long-held assumptions, and
internal perspectives. Because of their unstated nature, executives may be unaware of
them. The development of these areas requires self-discovery, deep learning, and
sometimes fundamental changes in one’s value system and behavior pattern
(Brantley, 2006). Future researchers should consider coaching effectiveness in light
of the specific content area being coached.
Coaching methodologies. The effectiveness of executive coaching may also be related
to the coaching methods employed. Coaches from different academic disciplines are
likely to use different methodologies to help their clients. These methods include
psychodynamic techniques (Kilburg, 2004), behavioral therapy (Sherin & Caiger,
2004), cognitive-behavioral approaches (Ducharme, 2004), and adult learning
(Axelrod, 2005). Different coaching methodologies may be effective at dealing
with different content areas. In other words, coaching effectiveness may be an
interaction between the coaching method employed and the coaching topic
addressed. Liljenstrand and Nebeker (2008) surveyed over 2,000 coaches and found
a significant interaction between academic backgrounds and client goals, suggesting
that coaches from different disciplines are hired to accomplish different objectives for
their clients. For example, they discovered that coaches with business backgrounds
were hired more often than other groups to coach individuals on task skills (e.g.,
sales) than for interpersonal relations (e.g., building trust, improving listening skills,
and adapting to change). On the other hand, coaches from an industrial/
organizational psychology or clinical background were hired more often than other
groups for engagements to enhance interpersonal skills and make work relationships
more effective (e.g., improving communication and listening, building trust). Overall,
Coaching: An International Journal of Theory, Research and Practice 127
Downloaded By: [De Meuse, Kenneth P.] At: 02:10 28 October 2009
it seems reasonable that coaching would be most likely to be successful when the
coaching methodologies ‘fit’ the coaching objectives.
Discussion and conclusion
The purpose of this paper was to continue the efforts initiated by other scholars to
address the paucity of coaching effectiveness research in the literature (Feldman &
Lankau, 2005; MacKie, 2007). Since coaching practitioners come from many
different professional backgrounds, an exploration of issues fundamental to
coaching evaluation can help build a common awareness that can guide future
research and practice. Toward achieving this objective, we first reviewed the literature
to understand the current state of executive coaching effectiveness. Both the meta-
analysis of the empirical research and the review of the retrospective studies revealed
an overall positive effect of executive coaching. We highlight the following
observations:
(1) Overall, executives and companies are favorable toward coaching.
(2) Coaching works. Specifically, the meta-analysis demonstrated that executives
made a moderate-to-large gain in skill and/or performance.
(3) Individuals who experience the coaching (self-ratings) report stronger effects
than do others (others’ ratings).
(4) Coaching impacts a wide array of individual and organizational outcomes,
such as individual skills and behaviors, team performance, productivity,
employee job satisfaction, and some measures of business deliverables.
(5) Some research suggests that coaching has the most positive impact when tied
directly to coaching objectives.
(6) The return from coaching was inconsistent. Coaching impact varies from
situation to situation, and even may lead to negative outcomes in some
circumstances.
We then explored and discussed six areas that influence the way researchers and
practitioners evaluate coaching effectiveness and affect the corresponding conclu-
sions that they may draw from their findings. The six areas and the major discussion
points include:
(1) The purpose of coaching evaluation summative versus formative evaluation.
(2) Criteria used to measure coaching effectiveness ROI versus coaching
objectives.
(3) Rigor of the coaching evaluation the research design employed to assess
coaching.
(4) Type of coaching implemented developmental versus remedial.
(5) Content of the coaching engagement skill acquisition and behavioral
change versus unconscious discovery and deep learning.
(6) Coaching methodologies the interaction between the coaching methods
and coaching content.
128 K.P. De Meuse, G. Dai, and R.J. Lee
Downloaded By: [De Meuse, Kenneth P.] At: 02:10 28 October 2009
In some cases, we offered specific suggestions for future assessment. For example,
we recommended that executive coaching should be evaluated against a single or
several individual-level criteria rather than using an overall organizational-level index
such as ROI. It appears to us that the objectives of the coaching engagement, which are
developed between the coach and the client are the most appropriate criteria on which
to gauge coaching efficacy. Likewise, we suggested that researchers and practitioners
should conduct ongoing formative evaluations, as well as a summative evaluation at
the end of the engagement, to enhance the overall quality of the coaching process. We
discussed how coaching effectiveness may be contingent on several factors. As our
meta-analysis indicated, there is an inconsistent relationship between executive
coaching and coaching outcomes. Researchers should systematically investigate how
coaching effectiveness is moderated by the purpose, content, and type of coaching.
Researchers should consider adopting a stage approach to coaching evaluation.
For example, Hicks and Peterson (1999) propose that coaches facilitate the
development of executives through five phases insight, motivation, capabilities,
real-world practice, and accountability. Similarly, Lombardo and Eichinger (2001)
articulate an ‘ABC Coaching Model’ that guides the coaching process through the
following six steps awareness, acceptance, acting, building, blending, and having
consequences. Regardless of the coaching model one follows, each stage has its
unique coaching objective. Very often the achievement of the objective at one stage is
necessary for the achievement of the objective at a subsequent stage. A complete
coaching evaluation should be able to assess all the stages, so we can understand
which stages are most critical to coaching success.
Based on this logic, Leedham (2005) proposed a holistic approach to evaluating
executive coaching. It is a pyramidal model of evaluation. At the bottom of the
pyramid are four key factors: (a) the skills of the coach, (b) the personal attributes of
the coach, (c) the coaching process, and (d) the coaching environment. At the next
stage are the levels of inner personal benefits (e.g., clarity and focus, confidence, and
motivation). The realization of the inner benefits is conducive to the outer personal
benefits (e.g., enhanced skills, knowledge, and understanding; improved behaviors).
Finally, with these enhanced skills and improved behaviors, the executive will be
equipped and empowered to achieve the pinnacle of the pyramid (i.e., business
results). Leedham (2005) is a good example of the stage model of coaching
evaluation. We do not suggest that this model can be applied to all coaching
interventions, because the coaching methods and the coaching topics vary from one
intervention to the next. An ideal evaluation model should be simple and capture the
essentials of the different types of coaching.
We also raised the question of the rigor of evaluation implemented. Researchers
should clarify what constitutes rigorous empirical research in the literature of
executive coaching, and how to quantify effectiveness. Likely, researchers will have to
decide if it is necessary to separate the contribution of coaching from other
leadership developmental interventions. And, if it is necessary, how do we ascertain
coaching’s unique impact in a field setting?
Retrospective questionnaires are likely to be the workhorse methodology for
coaching studies for some time to come, given the difficulties of implementing
rigorous research designs in the coaching field. It would be helpful if we develop
better instruments for this purpose. Appropriate scaling, relevant dimensions, rater
bias issues, and so on can be refined toward the goal of having a few reasonably
Coaching: An International Journal of Theory, Research and Practice 129
Downloaded By: [De Meuse, Kenneth P.] At: 02:10 28 October 2009
standard questionnaire formats for use by researchers. Such questionnaires should
be appropriate to the type of coaching, the organizational level of the coachee, other
developmental efforts affecting the coachee, and of course the specific objectives of
each coaching engagement.
During the 1970s, Schmidt, Hunter, and Urry (1976) presented the concept of
validity generalization to demonstrate the validity of various personnel assessments.
The authors persuasively argued that one should be able to accumulate validity
coefficients across several different empirical studies, weight the findings by the
respective sample sizes, correct for the unreliability of the psychological measure, and
then derive an overall index of validity. Their approach became the foundation for
meta-analysis. A popular measure advocated by some authors today is the concept of
ROI. By using such an all inclusive, straightforward index, the ‘return on investment’
of an intervention such as coaching can be calibrated easily. Unfortunately, the
realities of executive coaching engagements are not that simple. The expertise of the
coach, the length and intensity of the coaching assignment, the level of the
organization where the coachee resides, the receptivity of the manager receiving the
coaching, whether the coaching is for developmental or remedial reasons, the culture
of the organization, and a myriad of other individual and organizational factors all
can directly and indirectly affect that ROI index. We believe that the desire to sum
findings across studies and deduce a single measure of effectiveness must be weighed in
light of the danger of arriving at an erroneous or misleading ROI answer.
Many senior executives and HR directors probably understand the limited value
of an overall ROI number; therefore, they look to a panoply of global indicators of
coaching success (e.g., job satisfaction; organizational commitment; attitudinal,
behavioral, and relationship changes). A coach is responsible for helping a number
of stakeholders set reasonable expectations based on what seems achievable once the
coaching engagement is under way. In other words, formative evaluations and
emerging objectives should impact the way the summative evaluation is designed.
We also should consider the possibility that a more useful measure of validity
may be one that is associated with the coach rather than with the coaching process in
general. As with other professionals, some practitioners are consistently more
effective than others. There is almost no research assessing the effectiveness of
individual coaches; nor is there an easy way to design such research. In practice,
however, the reputations and references of experienced coaches carry the greatest
weight when it comes to decisions about whom to hire. Certainly, it is here that we
must look if we are to identify what makes for excellence in coaching.
Also, at the individual level, there continues to be a great need for well-reported
case studies of successful and not-so-successful coaching assignments. Some of us
may not see this as ‘research’ in a pure sense, yet instructive cases are perhaps the
best way to open the ‘closed door’ that Hall, et al. (1999) identified as a barrier to
understanding coaching. The most important learnings about coaching can be lost in
research designs that examine only correlation coefficients, statistical averages, and
curvilinear trends.
In conclusion, the empirical research assessing the effects of coaching has lagged
significantly behind the practice of coaching (Grant & Cavanagh, 2004). We are
hopeful that the six key areas related to coaching evaluation we identified in this
paper will serve as a springboard to improve assessment practices in the future. The
possibilities of coaching are endless. The necessity for coaching tomorrow’s leaders is
130 K.P. De Meuse, G. Dai, and R.J. Lee
Downloaded By: [De Meuse, Kenneth P.] At: 02:10 28 October 2009
critical. And the opportunities for practitioners to substantially impact the business
world are great. However, the effectiveness of executive coaching needs to be clearly
and scientifically demonstrated for us to achieve these desired outcomes.
References
Anderson, M.C. (2001). Executive briefing: Case study on the return on investment of
executive coaching. Retrieved November 27, 2007, from http://www.metrixglobal.net
Axelrod, S.D. (2005). Executive growth along the adult development curve. Consulting
Psychology Journal: Practice & Research,57, 118125.
Bolch, M. (2001). Proactive coaching. Training,38(5), 58.
Brantley, M.E. (2006). Executive coaching and deep learning (Doctoral dissertation, Fielding
Graduate University). Dissertation Abstract International Section A: Humanities and Social
Sciences,68(3-A), 848.
Buda, R., Reilly, R.R., & Smither, J.W. (1991). The influence of indirect knowledge of prior
performance on evaluations of present performance: The generalizability of assimilation
effects. Journal of Psychology & the Behavioral Science,6,8999.
Burke, M.J., & Day, R.R. (1986). A cumulative study of the effectiveness of managerial
training. Journal of Applied Psychology,71, 232245.
Bush, M. (2005). Client perceptions of effectiveness in executive coaching. Dissertation
Abstract International Section A: Humanities & Social Sciences,66(4-A), 1417.
Campbell, D.T., & Stanley, J.C. (1963). Experimental and quasi-experimental designs for
research on teaching. In N.L. Gage (Ed.), Handbook of research. Boston: Houghton Mifflin
Company.
Coaching counts. (2007). London: Chiumento research report. Retrieved December 19, 2007,
from http://www.chiumento.co.uk
Cohen, J. (1988). Statistical power analysis for the behavioral science (2nd ed.). Hillsdale, NJ:
Lawrence Erlbaum.
Corporate Leadership Council. (2003). Maximizing returns on professional executive coaching.
Washington, DC: Corporate Leadership Council.
Coutu, D., & Kauffman, C. (2009). What can coaches do for you? Harvard Business Review,
January, 2632.
Ducharme, M. (2004). The cognitive-behavioral approach to executive coaching. Consulting
Psychology Journal: Practice & Research,56, 214224.
Ellinger, A.D. (1999). Antecedents and consequences of coaching behavior. Performance
Improvement Quarterly,12,4570.
Ely, K., Nelson, J.K., Boyce, L., & Zaccaro, S.J. (2008). Evaluation methodologies of leadership
coaching. Paper presented at the 2008 Annual Meeting of the Society for Industrial and
Organizational Psychology, San Francisco.
English, M. (2006). Business print media coverage of executive coaching: A content analysis
(Doctoral Dissertation, Capella University).
*Evers, W.J.G., Brouwers, A., & Tomic, W. (2006). A quasi-experimental study on manage-
ment coaching effectiveness. Consulting Psychology Journal: Practice & Research,58,
174182.
Feldman, D.C. (2001). Career coaching: What HR professionals and managers need to know.
Human Resource Planning,24,2635.
Feldman, D.C., & Lankau, M.J. (2005). Executive coaching: A review and agenda for future
research. Journal of Management,31(6), 829848.
Festinger, L., & Carlsmith, J.M. (1959). Cognitive consequences of forced compliance. Journal
of Abnormal & Social Psychology,58, 203211.
Genger, C. (1997). Coaching: Theory and practice. Unpublished master’s thesis, University of
San Francisco.
Goldberg, R.A. (2005). Resistance to coaching. Organizational Development Journal,23,916.
Goldstein, I. L. (1986). Training in organizations: Needs assessment, development, and
evaluation (2nd ed.). Monterey, CA: Brooks/Cole.
Coaching: An International Journal of Theory, Research and Practice 131
Downloaded By: [De Meuse, Kenneth P.] At: 02:10 28 October 2009
Golembiewski, R., Billingsley, K., & Yeager, S. (1976). Measuring change and persistence in
human affairs: Types of change generated by OD designs. Journal of Applied Behavioral
Science,12, 133157.
Grant, A.M., & Cavanagh, M. (2004). Toward a profession of coaching: Sixty-five years of
progress and challenges for the future. International Journal of Evidence Based Coaching &
Mentoring,2(1), 116.
Grant, A.M., & Cavanagh, M.J. (2007). The goal-focused coaching skills questionnaire:
Preliminary findings. Social Behavior & Personality,35, 751760.
Hall, D.T., Otazo, K.L., & Hollenback, G.P. (1999). Behind closed doors: What really happens
in executive coaching. Organizational Dynamics,27,3953.
Hay Group (2002). Executive coaching: What makes a great executive? London: Hay Group.
Hicks, M.D., & Peterson, D.B. (1999). Leaders coaching across boarders. In W.H. Mobley
(Ed.), Advances in global leadership (Vol. 1, pp. 297315). U.S.: Elsevier Science/JAI Press.
Hunter, J.E., & Schmidt, F.L. (1995). Methods of meta-analysis: Correcting error & bias in
research findings. London: Sage.
Is coaching worth the money? (2005). Assessing the ROI of executive coaching. In H. Morgan,
P. Harkins, & M. Goldsmith (Eds.), The art and practice of leadership coaching. pp. 245254.
Hoboken, NJ: John Wiley & Sons.
Johnson, L.K. (2007). Getting more from executive coaching. Harvard Management Update,
12(1), 36.
Judge, W.Q., & Cowell, J. (1997). The brave new world of executive coaching. Business
Horizons,40,7177.
Kampa-Kokesch, S., & Anderson, M.Z. (2001). Executive coaching: A comprehensive review
of the literature. Consulting Psychology Journal: Practice & Research,53, 205228.
Kiel, F., Rimmer, E., Williams, K., & Doyle, M. (1996). Coaching at the top. Consulting
Psychology Journal: Practice and Research,48,6777.
Kilburg, R.R. (2000). Executive coaching: Developing managerial wisdom in a world of chaos.
Washington, DC: American Psychological Association.
Kilburg, R.R. (2004). When shadows fall: Using psychodynamic approaches in executive
coaching. Consulting Psychology Journal: Practice & Research,56, 246268.
Kirkpatrick, D.L. (1977). Evaluating training programs: Evidence vs. proof. Training &
Development Journal,31(11), 912.
Leedham, M. (2005). The coaching scorecard: A holistic approach to evaluating the benefits of
business coaching. International Journal of Evidence Based Coaching & Mentoring,3(2),
3044.
Libri, V., & Kemp, T. (2006). Assessing the efficacy of a cognitive behavioral executive
coaching programme. International Coaching Psychology Review,1(2), 920.
Liljenstrand, A.M., & Nebeker, D.M. (2008). Coaching services: A look at coaches, clients,
and practices. Consulting Psychology Journal: Practice & Research,60,5777.
Lombardo, M.M., & Eichinger, R.W. (2001). The leadership machine. Minneapolis: Lominger
Limited.
London, M. (2002). Leadership development: Paths to self-insight and professional growth.
Mahwah, NJ: Lawrence Erlbaum.
*Luthans, F., & Peterson, S.J. (2003). 360-degree feedback with systematic coaching:
Empirical analysis suggests a winning combination. Human Resource Management,43,
243256.
MacKie, D. (2007). Evaluating the effectiveness of executive coaching: Where are we now and
where do we need to be? Australian Psychologist,42, 310318.
McCall, M.W., Lombardo, M.M., & Morrison, A.M. (1988). The lessons of experience: How
successful executives develop on the job. New York: The Free Press.
McDermott, L. (1996). Wanted: Chief executive coaching. Training & Development,50(5),
6769.
McDermott, M., Levenson, A., & Newton, S. (2007). What coaching can and cannot do for
your organization. Human Resource Planning,30(2), 3037.
McGovern, J., Lindemann, M., Vergara, M., Murphy, S., Barker, L., & Warrenfeltz, R. (2001).
Maximizing the impact of executive coaching: Behavioral change, organizational outcomes,
and return on investment. Manchester Review,6(1), 19.
132 K.P. De Meuse, G. Dai, and R.J. Lee
Downloaded By: [De Meuse, Kenneth P.] At: 02:10 28 October 2009
Olivero, G., Bane, K.D., & Kopelman, R.E. (1997). Executive coaching as a transfer of
training tool: Effects on productivity in a public agency. Public Personnel Management,26,
461469.
Orenstein, R. L. (2006). Measuring executive coaching efficacy? The answer was right here all
the time. Consulting Psychology Journal: Practice & Research,58, 106116.
Parker-Wilkins, V. (2006). Business impact of executive coaching: Demonstrating monetary
value. Industrial & Commercial Training,38, 122.
*Peterson, D.B. (1993). Measuring change: A psychometric approach to evaluating individual
coaching outcomes. Research paper presented at the Society for Industrial and Organiza-
tional Psychology Conference, San Francisco.
Peterson, D.B. (2002). Management development: Coaching and mentoring programs. In
K. Kraiger (Ed.), Creating, implementing, and managing effective training and development
(pp. 160191). San Francisco: Jossey-Bass.
Sackett, P.R., & Mullen, E.J. (1993). Beyond formal experimental design: Towards an
expanded view of the training evaluation process. Personnel Psychology,46, 613627.
Schmidt, F.L., Hunter, J.E., & Urry, V.W. (1976). Statistical power in criterion-related
validation studies. Journal of Applied Psychology,61, 473485.
Sherin, J., & Caiger, L. (2004). Rational-emotive behavior therapy: A behavioral change model
for executive coaching? Consulting Psychology Journal: Practice & Research,56, 225233.
*Smither, J.W., London, M., Flautt, R., Vargas, Y., & Kucine, I. (2003). Can working with an
executive coach improve multisource feedback ratings over time? A quasi-experimental field
study. Personal Psychology,56,2344.
Smither, J.W., London, M., & Reilly, R.R. (2005). Does performance improve following
multisource feedback? A Theoretical model, meta-analysis, and review of empirical findings.
Personnel Psychology,58,3366.
Smither, J.W., Reilly, R.R., & Buda, R. (1988). The effect of prior performance information on
rating of present performance: Contrast versus assimilation revisited. Journal of Applied
Psychology,73, 487496.
Smither, J.W., & Walker, A.G. (2001). Measuring the impact of multisource feedback. In
D. Bracken, C. Timmreck & A. Church (Eds.), Handbook of multisource feedback (pp. 256
274). San Francisco: Jossey-Bass.
Sue-Chan, C., & Latham, G.P. (2004). The relative effectiveness of external, peer, and self-
coaches. Applied Psychology: An International Review,53, 260278.
Talboom, A.M. (1999). The welfare sector hits hard: Exploratory research after the role of
coaching and counseling within organizations. Nijmegen: Business School.
Thach, E.C. (2002). The impact of executive coaching and 360 feedback on leadership
effectiveness. Leadership & Organizational Development Journal,23, 205214.
*Togel, G., & Nicholson, N. (2005). Multisource feedback, coaching, and leadership develop-
ment: Gender homophily in coaching dyads. Academy of Management Best Conference
Paper.
Valerio, A.M., & Lee, R.J. (2005). Executive coaching: A guide for the HR professional.
San Francisco: Pfeiffer.
Wasylyshyn, K.M. (2003). Executive coaching: An outcome study. Consulting Psychology
Journal: Practice & Research,55,94106.
When coaching measures up. (2005, October). Training Magazine,p.20.
Witherspoon, R., & White, R.P. (1996). Executive coaching: A continuum of roles. Consulting
Psychology Journal: Practice & Research,48, 124133.
*Wofred, T. (2003). Executive coaching project: Evaluation of findings. San Francisco:
CompassPoint.
Zenger, J.H., & Stinnett, K. (2006). Leadership coaching: Developing effective executives.
Chief Learning Officer,5(7), 4447.
Note. Asterisked (*) references were included in meta-analysis.
Coaching: An International Journal of Theory, Research and Practice 133
Downloaded By: [De Meuse, Kenneth P.] At: 02:10 28 October 2009
Notes on contributors
Dr. Kenneth P. De Meuse is Associate Vice President of Research at
Korn/Ferry Leadership & Talent Consulting. Prior to coming to
work at Korn/Ferry, he was on the faculties at the University of
Wisconsin Eau Claire, Iowa State University, and the University of
Nebraska at Omaha. Ken has published five books and numerous
journal articles on employee attitudes and organizational behavior in
several leading professional journals.
Guangrong Dai, Ph.D., is the research scientist at Korn/Ferry
Leadership & Talent Consulting. His current research focuses on
leadership development, including topics such as multi-source feed-
back, high potential identification, leadership transition, executive
onboarding, and global leadership effectiveness.
Dr. Robert J. Lee is a management consultant in private practice in
New York City and serves as a coach to senior executives regarding
leadership style, managerial effectiveness, and team development.
Bob is on the adjunct faculty at the Milano Graduate School of New
School University, and is a Senior Fellow with the Zicklin School of
Business, Baruch College, CUNY.
134 K.P. De Meuse, G. Dai, and R.J. Lee
Downloaded By: [De Meuse, Kenneth P.] At: 02:10 28 October 2009
