ArticlePDF Available

Benchmarking In Hospitals: More Than a Scorecard

Authors:

Figures

Content may be subject to copyright.
Benchmarking
In Hospitals: More
Than a Scorecard
by Victor E. Sower
f you look at many hospitals’ websites and
other resources they make publicly available,
you will often see charts, graphs or tables
showing hospital performance on some metric
compared with a national standard.
An example is Table 1, which was taken from a
hospital website and compares patient satisfac-
tion in numerous areas with the national average.
The hospital highlighted the areas in which it
exceeded the national average.
The term “benchmarking” is often mentioned in
hospital quality literature, but the process of bench-
marking is often misunderstood. True benchmark-
ing is not simply comparing outcome measures
with industry averages.
ASQ defines benchmarking as “a technique in
which a company measures its performance against
that of best in class companies, determines how
those companies achieved their performance levels
and uses the information to improve its own perfor-
I
HEALTHCARE
In 50 Words
Or Less
Benchmarking should not just involve comparing
your hospital with national averages; it should
involve looking at best in class hospitals and
finding out what they do.
Hospitals shouldn’t limit their benchmarking to
just the healthcare industry; there’s much to
learn from the service industry, too.
Benchmarking Outcome Measures
TABLE 1
Item Our hospital National average
Overall patient satisfaction 84.9 82.5
Check-in 82.0 79.0
Nurses 79.3 84.4
Doctors 84.1 85.4
Tests 85.5 85.8
Family or friends 88.9 85.5
Waiting time 83.6 87.5
Black: Our hospital lower than national average
Blue: Our hospital higher than national average
58
I
AUGUST 2007
I
www.asq.org
mance. Subjects that can be benchmarked include
strategies, operations and processes.”
1
Doing a simple comparison with a national aver-
age is more like a scoreboard showing who is win-
ning. It only answers the question “Am I above or
below average?” This doesn’t tell the hospital how
to improve operations. The approach might be of
interest to the general public, government and
accreditation agencies, but it is of limited value as
input to a hospital’s process of continuous quality
improvement (CQI).
Ask yourself which would contribute more to
your CQI program: knowing your hospital is
slightly above average nationally in controlling
Methicillin resistant Staphylococcus aureus
(MRSA) or understanding the processes that the
University of Virginia Hospital used to achieve
best in class MRSA control?
2
There is value in comparisons with national
averages. Residents of the hospital’s service area
can judge the quality of the hospital compared
with national averages. The hospital’s quality
director and quality improvement teams can use
this information to determine which areas most
need improvement. The improvement efforts’
progress can be monitored over time to determine
whether the actions taken are effective in closing
the gaps between a hospital’s performance and
national averages.
However, for all its usefulness, comparison with
national averages is insufficient. Meeting the nation-
al average does not equate excellence. It might not
even equate sufficiency.
A Canadian study found that 7.5% of patients
experienced at least one adverse event because of
medical errors in 2000.
3
If your hospital has a med-
ical error rate of 7%, it is better than the national
average. Is that sufficient? Wouldn’t it be better to
know what the error rate is at the best hospitals?
Wouldn’t it be even better to understand how
those best in class hospitals achieved the bench-
mark standard medical error rates?
The Leapfrog Hospital Quality and Safety
Survey found that 50% of hospitals do not have
procedures to prevent bedsores.
4
If your hospital
has any such procedures, you are above the nation-
al average. Is that sufficient? Wouldn’t it be better
to know what the procedures are at the hospitals
with the lowest incidence of bedsores?
Without information about the processes that the
best hospitals use, we must approach improvement
by reinventing the wheel. We are doomed to make
the same mistakes other hospitals have made and
learned from.
A problem with national averages is that we
don’t even know which hospitals are the best per-
formers and we don’t know what best in class per-
formance is. National averages provide no measure
of variation in performance and no information
about the level for best in class performers. Varia-
tion in performance can be a bigger problem than
average performance.
The Nebraska Medical Center’s interventional
radiology department undertook a project to
improve major problems in treatment delays,
which were creating patient dissatisfaction and
patients’ seeking treatment elsewhere.
5
The
department found it took an average of 1.4 calls
to schedule an appointment. Further analysis
revealed the standard deviation was 0.989 calls,
with a maximum of seven calls.
After several improvement projects had been
completed, the average was still 1.4 calls.
However, the standard deviation had been
reduced to 0.52 calls with a maximum of three
calls. If they had used a comparison to national
averages, the significant improvement in this
process would not be visible.
Benchmarking is an improvement process in
which an organization measures its performance
against that of best in class organizations within or
outside its industry, determines how those organi-
zations achieved their performance levels, and uses
that information to improve its own performance.
Benchmarking can be a valuable tool in moving
beyond national average performance to best in
class performance.
Benchmarking can be a
valuable tool in moving
beyond national average
performance to best in
class performance.
QUALITY PROGRESS
I
AUGUST 2007
I
59
Best In Class
While it is useful to discuss improvement efforts
with other hospitals that are nearby, easily accessible
or otherwise convenient, to aspire for excellence you
must compare yourself with excellent hospitals. One
such best in class hospital is Robert Wood Johnson
(RWJ) University Hospital in Hamilton, NJ.
RWJ received a 2004 Malcolm Baldrige National
Quality Award. It had a quality program in place
in 1999 that was based on five pillars of excel-
lence—service, finance, quality, people and growth.
But, looking for ways to better serve its customers,
the hospital’s management decided to use the
Baldrige criteria as a “framework for leadership
and acceleration of [its] quality journey.”
6
One of RWJ’s achievements is best in class ser-
vice in its emergency department (ED). Its 15/30
program guarantees every patient will see a nurse
within 15 minutes and a doctor within 30 minutes
of entering the ED. RWJ backs this program with
an extraordinary guarantee—if it fails to meet the
guarantee, the ED portion of the bill will be waived
upon patient request. The hospital’s payout is less
than 1%, indicating it has a process in place to
achieve the desired results. Patient satisfaction
with ED increased from 85% in 2001 to 90% in
2004. Because 70% of the hospital’s inpatients
enter through the ED, this program has con-
tributed to overall hospital success.
Another hospital has an average time from enter-
ing the ER to seeing a physician of 47 minutes. The
graph on its website shows this is better than the
national norm of about 55 minutes. Clearly, this is
an above average hospital. But it is not best in class.
It should benchmark against RWJ’s best in class
performance—not the national norm.
Inside or Outside the Industry
While numerous hospitals have been recog-
nized for excellence—four have received the
Baldrige award since 2002—other hospitals need
not restrict their searches for benchmarking part-
ners to other hospitals. Joseph Juran wrote, “As
the health industry undertakes … change, it is
well advised to take into account the experience
of other industries in order to understand what
has worked and what has not. The health indus-
try is different … however, the decisive factors in
what works and what does not are the managerial
processes, which are alike for all industries.”
7
For example, hospitals share several processes
with hotels. The Ritz-Carlton Hotel Co., which
received a Baldrige award in 1992, has approaches
to employee training, room service, custodial ser-
vices, customer orientation and quality metrics
that hospitals could learn from. Disney is well
known for employee training and customer orien-
tation—both important to hospitals. Both of these
organizations were used as benchmark standards
by Bronson Methodist Hospital in Kalamazoo,
MI—also a Baldrige recipient.
8
Benchmarking is not just copying what other
successful organizations are doing. It involves
not just understanding what best in class organi-
zations’ goals are and how they have achieved
those goals through process and operations
improvement; it is also taking that information
back to your own organization to determine how
to achieve comparable results given your unique
internal and external conditions. This process
will make yours a better hospital.
REFERENCES
1. “The Quality Glossary,” Quality Progress, June 2007.
2. Thomas G. Dolan, “Staph Infections—Stealthy Killers,”
Radiology Today, May 2006.
3. Anne McIlroy and Rod Mickleburgh, “Hospital Errors
Kill Thousands in Canada, Study Estimates,” The Toronto
Globe and Mail, May 24, 2004.
4. Leapfrog Hospital Quality and Safety Survey, 2005,
www.leapfroggroup.org.
5. Jennifer Volland, “Quality Intervenes at a Hospital,”
Quality Progress, February 2005.
6. Dave Nelsen, “Baldrige—Just What the Doctor
Ordered,” Quality Progress, October 2005.
7. Joseph Juran, foreword to Curing Health Care, by
Donald Berwick, A. Blanton Godfrey and Jane Roessner,
Jossey-Bass, 1990.
8. Michele Serbenski, executive director, corporate effec-
tiveness and customer satisfaction, Bronson Healthcare
Group, personal communication, Oct. 3, 2006.
VICTOR E. SOWER is a professor of management at Sam
Houston State University in Huntsville, TX. He earned a
doctorate in operations management from the University of
North Texas in Denton. Sower is a senior member of ASQ,
a member of the Health Care Management Division, a
Quality Press reviewer and a certified quality engineer.
HEALTHCARE
60
I
AUGUST 2007
I
www.asq.org
... Person-centeredness involves participation of PWMI and their families in decision-making regarding service design and treatment choices (Sower, 2007;Johnson, Abraham, & Shelton, 2009). Table 6 summarizes the results. ...
... In this study, more than 50 percent of the respondents had not conducted a benchmark exercise in the past 12 months. This is incongruent with the notion that benchmarking is a strategy that helps health care professionals to compare their clinical performances to those of their peers in order to improve their service (Sower, 2007). ...
Article
Full-text available
Addressing the inadequate and poor provision of mental health services in rural areas is a world-wide challenge. Most people with mental illness in these areas do not have access to mental health services. Using eight attributes of good mental health service as criteria, the purpose of this study was to assess mental health services at Mashashane, a rural area in the Limpopo Province, South Africa. A survey was conducted with a purposively selected sample of health care professionals from four health establishments serving Mashashane. Data was collected using a questionnaire with closed and open-ended questions. Quantitative and qualitative data analyses were used. The results show that out of eight attributes assessed only comprehensiveness was positively perceived. This is an indication of the inadequacy of mental health services, hence their inability to improve the well-being of people with mental illness and their families. Lack of resources was identified as the major hindrance to the delivery of appropriate mental health services. The findings highlighted aspects that contributed to the development of a framework for a community-based program to improve the well-being of people with mental illness and their families in a rural setting.
... Almost as frequently, a best-in-class organization was selected as the benchmarking target. This is the preferred approach according to most benchmarking literature (Sower, et al., 2007;Camp, 1989Camp, , 1993. ...
... The fact that fewer than 30% of respondents benchmarked against targets outside their industries indicates that opportunities exist for increasing the effectiveness of benchmarking activities by expanding the scope of possible benchmarking targets. That operations, internal processes, and strategies are more frequently benchmarked than outcomes is consistent with best practices (Sower, 2007). Outcome benchmarking is required in some industries (e.g. ...
Article
Benchmarking is a proven way to make radical improvements in processes as diverse as surgical error prevention and physical distribution relatively quickly by targeting best-in-class organizations. This project surveys 196 operations and supply chain professionals with supply chain improvement responsibilities in the US to determine the current extent of use and effectiveness of benchmarking as part of their efforts to improve supply chain operations. The most frequently used criterion for selecting benchmarking targets were convenience and recognized best-in-class performance. The benefits and difficulties associated with benchmarking were found to vary by the size of the organization. The findings suggest that benchmarking can be a significant resource for improving supply chain performance and therefore, by extension, competitive advantage.
... According to Sower (2007), benchmarking is not a tool to copy what other successful organizations are doing; this implies not only a better understanding of what they are doing to be successful, but also what is there to do to achieve their goals and objectives. The top management with this tool can understand that the financial improvement efforts and progress can be monitored over time, to determine whether the measures taken are effective and close all the gaps between the performance of a hospital and the industry benchmark or not. ...
Article
Full-text available
El benchmarking se considera un componente clave del sistema de medición del desempeño de una organización. Este estudio examina una muestra de 53 hospitales con y sin fines de lucro adscritos al Directorio Americano de Hospitales, a través de cuatro dimensiones financieras: liquidez, eficiencia, rentabilidad y estructura de capital. El propósito del estudio es validar si el benchmark de la industria difiere o no de un grupo de 17 índices financieros seleccionados, para determinar si su desempeño es eficiente o ineficiente en el sistema hospitalario de Puerto Rico. Los resultados muestran que el 53% o más de los índices financieros seleccionados, comparados globalmente, sugieren ser eficientes en ambos tipos de hospitales. Esto significa que estos índices financieros fueron mayores o iguales que el benchmark de la industria.
Article
Purpose The purpose of this paper is to identify reported barriers to benchmarking and strategies to surmount these barriers. Design/methodology/approach The study is a qualitative meta‐analysis of 32 peer‐reviewed sources from January 2005‐July 2010. The authors sought recently published authoritative information on the topic of benchmarking reluctance to formulate an up‐to‐date framework explaining this phenomenon. Content analysis was used to identify reasons for benchmarking reluctance and ways to overcome reluctance. Findings The study concludes that organizational leadership best practices have been found to counter each of the four major benchmarking reluctance concerns: soundness of benchmarking theory/practices; lack of resources for benchmarking; inertia impeding pursuit of new practices; and specific impacts of implementing new practices. Practical implications The findings of this literature review may assist leaders to anticipate potential benchmarking barriers and to follow best practices for addressing these impediments. Originality/value Given the limited amount of recent literature focused on the topic of benchmarking reluctance, this paper provides valuable resources to help organizations succeed in their benchmarking efforts.
Article
The role of quality methods in improvement of radiology department at Nebraska Medical Center is discussed. The Nebraska Medical Center used quality methods in its interventional radiology department to increase the number of patients and the volume of procedures the department was conducting. Further improvement was made by the interventional radiology physicians reaching consensus on time allocation for procedures and lab work needed prior to procedures. By using Six Sigma tools, the department was able to collaborate with referring clinics and its own staff on improving its scheduling process.
  • G Thomas
  • Dolan
Thomas G. Dolan, " Staph Infections—Stealthy Killers, " Radiology Today, May 2006.
Baldrige-Just What the Doctor Ordered
  • Dave Nelsen
Dave Nelsen, "Baldrige-Just What the Doctor Ordered," Quality Progress, October 2005.
Hospital Errors Kill Thousands in Canada, Study Estimates
  • Anne Mcilroy
  • Rod Mickleburgh
Anne McIlroy and Rod Mickleburgh, "Hospital Errors Kill Thousands in Canada, Study Estimates," The Toronto Globe and Mail, May 24, 2004.
foreword to Curing Health Care
  • Joseph Juran
Joseph Juran, foreword to Curing Health Care, by
executive director, corporate effectiveness and customer satisfaction
  • Michele Serbenski
Michele Serbenski, executive director, corporate effectiveness and customer satisfaction, Bronson Healthcare Group, personal communication, Oct. 3, 2006.
Staph Infections-Stealthy Killers
  • Thomas G Dolan
Thomas G. Dolan, "Staph Infections-Stealthy Killers," Radiology Today, May 2006.