Hospital quality reports in Germany: patient and physician opinion of the reported quality indicators.

Abstract

Starting in 2005, Germany's health law required hospital quality reports to be published every two years by all acute care hospitals. The reports were intended to help patients and physicians make informed choices of hospitals. However, while establishing the quality indicators that form the content of the reports, the information needs of the target groups were not explicitly taken into account. Therefore, the aim of our study was to determine patient and physician opinion of the relevance of the reported quality indicators for choosing or referring to a hospital.
Convenience samples of 50 patients and 50 physicians were asked to rate the understandability (patients), suitability (physicians) and relevance (both groups) of a set of 29 quality indicators. The set was drawn from the reports (24 indicators) and supplemented by five indicators commonly used in hospital quality reports. We analysed the differences in patient and physician ratings of relevance of all indicators by applying descriptive statistics, t-tests and Wilcoxon tests.
Only three indicators were considered not understandable by the interviewed patients and unsuitable by the interviewed physicians. The patients rated 19 indicators as highly or very relevant, whereas the physicians chose 15 indicators. The most relevant indicator for the patients was "qualification of doctors", and for the physicians "volume of specified surgical procedures". Patient and physician rankings of individual indicators differed for 25 indicators. However, three groups of indicators could be differentiated, in which the relevance ratings of patients and physicians differed only within the groups. Four of the five indicators that were added to the existing set of reported indicators ranked in the first or second group ("kindness of staff", "patient satisfaction", "recommendation", and "distance to place of living").
Most of the content of Germany's hospital quality reports seems to be useful for patients and physicians and influence their choice of hospitals. However, the target groups revealed that approximately one third of the indicators (mostly hospital structural characteristics), were not useful and hence could have been omitted from the reports. To enhance the usefulness of the reports, indicators on patient experiences should be added.

Full text

Page 1

BioMed Central
Page 1 of 6
(page number not for citation purposes)
BMC Health Services Research
Open Access
Research article
Hospital quality reports in Germany: patient and physician opinion
of the reported quality indicators
Max Geraedts*, David Schwartze and Tanja Molzahn
Address: Public Health Programme, University Hospital of the Heinrich-Heine-University, Moorenstraße 5, 40225 Düsseldorf, Germany
Email: Max Geraedts* - geraedts@uni-duesseldorf.de; David Schwartze - d-schwartze@versanet.de; Tanja Molzahn - tanjamolzahn@t-online.de
* Corresponding author
Abstract
Background: Starting in 2005, Germany's health law required hospital quality reports to be
published every two years by all acute care hospitals. The reports were intended to help patients
and physicians make informed choices of hospitals. However, while establishing the quality
indicators that form the content of the reports, the information needs of the target groups were
not explicitly taken into account. Therefore, the aim of our study was to determine patient and
physician opinion of the relevance of the reported quality indicators for choosing or referring to a
hospital.
Methods: Convenience samples of 50 patients and 50 physicians were asked to rate the
understandability (patients), suitability (physicians) and relevance (both groups) of a set of 29 quality
indicators. The set was drawn from the reports (24 indicators) and supplemented by five indicators
commonly used in hospital quality reports. We analysed the differences in patient and physician
ratings of relevance of all indicators by applying descriptive statistics, t-tests and Wilcoxon tests.
Results: Only three indicators were considered not understandable by the interviewed patients
and unsuitable by the interviewed physicians. The patients rated 19 indicators as highly or very
relevant, whereas the physicians chose 15 indicators. The most relevant indicator for the patients
was "qualification of doctors", and for the physicians "volume of specified surgical procedures".
Patient and physician rankings of individual indicators differed for 25 indicators. However, three
groups of indicators could be differentiated, in which the relevance ratings of patients and
physicians differed only within the groups. Four of the five indicators that were added to the
existing set of reported indicators ranked in the first or second group ("kindness of staff", "patient
satisfaction", "recommendation", and "distance to place of living").
Conclusion: Most of the content of Germany's hospital quality reports seems to be useful for
patients and physicians and influence their choice of hospitals. However, the target groups revealed
that approximately one third of the indicators (mostly hospital structural characteristics), were not
useful and hence could have been omitted from the reports. To enhance the usefulness of the
reports, indicators on patient experiences should be added.
Published: 28 September 2007
BMC Health Services Research 2007, 7:157doi:10.1186/1472-6963-7-157
Received: 24 January 2007
Accepted: 28 September 2007
This article is available from: http://www.biomedcentral.com/1472-6963/7/157
© 2007 Geraedts et al; licensee BioMed Central Ltd.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0),
which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Page 2

BMC Health Services Research 2007, 7:157http://www.biomedcentral.com/1472-6963/7/157
Page 2 of 6
(page number not for citation purposes)
Background
By law, all German hospitals are required to publish a
structured quality report every two years. The first series of
reports of 1,983 acute care hospitals was published in
PDF-format on the internet in September of 2005 [1]. The
aim of these quality reports was to allow insurees/patients
and referring physicians to make an informed choice of
hospitals [2].
Publishing these reports on the quality of hospital care is
comparable to efforts in many countries of the world [e.g.
[3-6]]. However, the process of choosing the right quality
indicators for the report did not follow state of the art
methodology [7-12], as reflected in the set of indicators
that was ultimately chosen. A group of representatives of
the self-administration of the German health care system
– representing the statutory social health insurance funds
and the German hospital association – established the
content of the reports without explicitly taking into
account international experiences and the information
needs of patients and referring physicians [Daniela Riese,
association of substitute health insurance funds for
employees, personal communication].
Therefore, the aim of our study was to examine whether
the quality reports and their respective quality indicators
could nevertheless be useful for informed decision mak-
ing by determining the relevance attributed to the
reported hospital quality indicators by patients and refer-
ring physicians.
Methods
Quality indicators
The German quality reports are divided into two parts:
- A structured and comparable part with information on
the scope and volume of services provided by each hospi-
tal
- An unstructured and virtually comparable part with
information on hospital quality management systems.
Since these quality reports do not contain quality indica-
tors as exactly defined parameters [13], we extracted all
items from the first part of the reports that may be relevant
for informed decision making while choosing a hospital
(see list of indicators in the table). To study the usefulness
of the structured quality reports as they were published in
2005, i.e. as PDF-files without glossary, we used the exact
wording of the reports and explained the meaning of the
different indicators to the interviewed patients and physi-
cians in more detail only on request and only after receiv-
ing their answers to our questions.
The reports missed many indicators that are commonly
used in quality reports and we did not want to excessively
strain our interviewees. We therefore added only five
internationally acknowledged indicators to the former list
of 24 indicators: "patient satisfaction", "kindness of staff",
"distance to place of living", "recommendation by rela-
tives or physicians", and "hospital regularly visited or
referred to". We thus ended up with 29 indicators to be
examined in our study.
Patients, physicians and settings
Since surveys that do not collect personal data do not
require approval by the ethics committee of the University
of Duesseldorf, we only asked the interviewees for their
consent to participate in the study. On four consecutive
days, a medical student interviewer (DS) visited the wait-
ing rooms of four general practitioners' surgeries in the cit-
ies of Cologne, Duesseldorf, Wuppertal, and Duisburg,
which are located in the Rhine-Ruhr area of Germany.
There, he recruited 50 patients who agreed to take part in
an interview on choosing hospital care. The sample was
therefore drawn as a convenience sample of consecutive
patients of the respective private practices.
Table 1 presents the patient group's age and gender struc-
ture, as well as the proportion of the proportion of statu-
torily insured study patients compared to this proportion
in average GP patients in Germany [Zentralinstitut für die
kassenaerztliche Versorgung, Berlin: Patient-Physician-
Panel (ADT-Panel), I/2007, personal communication]. In
the study patient group, women and patients holding pri-
vate insurance were overrepresented. Concerning experi-
ence with in-hospital care in Germany, data for the
average group of a GP's patients do not exist. In our sam-
ple, two patients did not yet experience a hospital stay, the
remaining 48 experienced an average of 4.7 periods of
hospital care (range of 1–30 episodes of in-hospital care).
DS performed the 50 structured face-to-face interviews
that took an average of 25 minutes in a separate room of
the doctor's office. The patients were asked to imagine suf-
fering from an inguinal hernia. In order to choose a hos-
pital for performing the required surgery, they were
handed a fictitious "regional hospital quality information
package". This package was comprised of different com-
parative graphs on the quality of care at ten hospitals
Table 1: Socio-demographic data of study patients compared to
average GP patients in Germany
Study patientsAverage GP patients
Mean age (range)
% women
% statutorily insured
53 years (18–81)
68%
68%
50 years (0–>100)
57%
90%

Page 3

BMC Health Services Research 2007, 7:157http://www.biomedcentral.com/1472-6963/7/157
Page 3 of 6
(page number not for citation purposes)
along with the list of indicators that were used to compare
the hospitals. The patients were then asked to rate a) the
understandability of the indicators (understandable/not
understandable), and b) the relevance of the 29 indicators
for making an informed choice of which hospital to go to.
The rating scale ranged from 1–6 and was based on the
grading system in German schools:
- One (1) = highly relevant
- Two (2) = very relevant
- Three (3) = somewhat relevant
- Four (4) = somewhat irrelevant
- Five (5) = very irrelevant
- Six (6) = completely irrelevant
Another medical student interviewer (TM) visited 50 gen-
eral practitioners (GPs) who agreed to take part in the
study. All practitioners worked in the cities of Hagen and
Bochum and their surrounding regions in North Rhine-
Westphalia. The convenience sample was drawn by
informing all practitioners (N = 164) of those two regions
about the study in writing and choosing the first 50 that
agreed to participate. Their mean age was 50 years and 10
were women. Only 13 worked in group practices; the
remaining 37 in single private practices.
The GPs were asked to imagine a patient needing an
inguinal hernia operation. In order to make an informed
referral decision, the physicians received a fictitious infor-
mation package from the region's social health insurance
funds. This package was comprised of information about
the quality of care in regional hospitals. The physicians
were asked to rate a) the suitability (yes/no) and, b) the
relevance of the 29 indicators for making an informed
referral decision (rating scale from 1 to 6 as explained
above).
Analysis
We compared the number of patients who rated the differ-
ent indicators as understandable and the number of phy-
sicians who rated the different indicators as suitable. The
mean ratings and standard deviations of every indicator
were examined with respect to their relevance rating. We
analysed differences between the patient and physician
ratings of relevance of the individual indicators by apply-
ing t-tests. Within the groups of patients and physicians,
we analysed the ratings by performing Wilcoxon tests,
comparing the ratings of those neighbouring indicators
that differed in their median ratings between grades two
and three.
Results
Table 2 (see additional file 1) shows each of the 29 indi-
cators with the ratings of patients and physicians in rela-
tion to their relevance, understandability and suitability.
The indicators are ranked according to the mean relevance
ratings of the patient group, showing the highest ranking
indicator ("qualification of doctors") at the top of the list.
In addition, the table shows the relevance ranking by the
physician group.
Considering understandability, most indicators (22 out of
29) were understandable for more than 40 interviewed
patients (80%). However, only five indicators were under-
stood by the entire group of patients. In the physician
group, only one indicator was suitable for all of the inter-
viewed doctors and only 11 out of 29 indicators were suit-
able for more than 80% of them. Four indicators were
judged as not understandable by more than half of the
patients compared to seven indicators deemed not suita-
ble in the group of physicians.
Considering the patient relevance ratings for each of the
indicators, 19 indicators were rated highly or very relevant
(median grades one or two). The remaining 10 received
grades of three to six, meaning somewhat relevant to com-
pletely irrelevant. Only one indicator received a median
grade of four ("hospital owner"). However, 16 out of 29
indicators were graded completely irrelevant by at least
one of the interviewed patients.
The majority of physicians rated fifteen indicators as
highly or very relevant and fourteen as somewhat relevant
to completely irrelevant. Five indicators received a median
grade of four or five. 24 out of 29 indicators were graded
as completely irrelevant by at least one physician.
Within the groups of patients and physicians, the indica-
tor ratings differed significantly for those indicators that
were rated grade three (somewhat relevant) ("profes-
sional training for doctors" in both groups) compared to
the indicators with a median rating of grade two (very rel-
evant) or better (in the patients, the indicator ranked on
position 18 "distance to place of living", in the physicians,
the indicator ranked on position 12 and better "specialist
outpatient department").
The comparison of patient and physician relevance rank-
ings shows no difference for four indicators, whereas the
rankings of all other indicators differed between one and
seven positions. However, the analysis indicates three
groups of indicators that may be differentiated:
- Indicators ranked 1st to 10th position ("qualification of
doctors" to "range of therapeutic facilities"),

Page 4

BMC Health Services Research 2007, 7:157http://www.biomedcentral.com/1472-6963/7/157
Page 4 of 6
(page number not for citation purposes)
- Indicators ranked 11th to 19th position ("recommenda-
tion" to "professional training for doctors")
- Indicators ranked 20th to 29th position ("hospital oper-
ates an outpatient department" to "hospital owner").
The relevance rankings of patients and physicians differed
only within these groups and did not overlap with the
other groups. Within the first group of ten indicators, only
one showed a significant difference between the patient
and physician ratings of relevance. In both the second and
third group of indicators, four differed significantly.
The first ten indicators were judged most relevant by both
groups. However, while the patients ranked the indicators
"qualification of doctors", "kindness of staff" and "patient
satisfaction" in the 1st to 3rd positions, these indicators
received the 6th, 7th and 5th rank respectively, by the group
of physicians. On the other hand, the physicians ranked
the indicators "volume of specified surgical procedures",
"24-hours-availability of technical equipment", and
"range of therapeutic facilities" 1st to 3rd, whereas the
patients ranked these indicators on the 7th, 6th, and 10th
position, respectively.
Five indicators were added to the list published in the offi-
cial German hospital quality reports. Two of them ranked
in the first group ("kindness of staff" and "patient satisfac-
tion") received 2nd and 3rd positions from the patient
group and 7th and 5th position from the physicians. Two in
the second group ("recommendation" and "distance to
place of living") received 11th and 18th positions from the
patient group and 14th and 13th position from the physi-
cians. The last in the third group ("hospital regularly vis-
ited or referred to") received the 25th position from the
patient group and the 27th position from the physicians.
Discussion
Most but not all indicators published recently as part of
the first series of obligatory federal hospital quality
reports in Germany were judged by patients and referring
physicians as useful in choosing a hospital. At least 10 out
of 24 indicators could be omitted from the quality reports
because both target groups (patients and physicians) rated
these indicators as less relevant for choosing a hospital. In
addition, three of these indicators were not understanda-
ble or suitable to more than half of both target groups, fur-
ther justifying their omission. However, at least three
indicators ("kindness of staff", "patient satisfaction", and
"recommendation") should be added to the reports in
order to enhance their usefulness. They received high rel-
evance and at the same time high understandability or
suitability rankings.
Understandability and suitability of the indicators
The interviewed patients apparently understood almost
all indicators included in the hospital quality reports.
Only four indicators were judged "understandable" by
less than half of the patients. The physicians' judgements
of suitability of the listed indicators as an information
tool for their referral decision were more diverse. There
was only one indicator deemed suitable to all physicians
("volume of specified surgical procedures") and only
eleven indicators were suitable to more than 80% of the
physicians.
Relevance of the indicators
The patients judged the indicators more positively than
the physicians by rating a greater number of indicators as
highly or very relevant for choosing a hospital. Only one
indicator received a median grade of four by the patients.
The number of patients treated as well as some structural
characteristics of hospitals (e.g. "certified to treat insured
accidents") appeared to be of less relevance to patients. By
way of contrast, patients considered the number of proce-
dures performed and the range and availability of spe-
cialty services as very relevant. Especially the qualification
of doctors and nurses and indicators assessing patient sat-
isfaction seemed to be relevant for choosing a hospital.
The kindness of the staff and the reputation of a hospital
were also highly relevant to patients.
The indicators concerning patient experiences played an
important role for physicians as well. However, the top
four positions in the physician ranking of relevance were
held by indicators focussing on the number of procedures
and range and availability of services. Those indicators
judged less relevant by the patients were judged even less
relevant by the physicians.
However, based on the comparison of the rankings
depicted in the table, the patient and physician rankings
did not differ entirely. There were three groups of indica-
tors within which the positions of the individual indica-
tors differed marginally between the patients and
physicians but did not overlap with the other groups
(positions 1 to 10, 11 to 19, and 20 to 29). Especially the
indicators ranked position 1 to 10 in both groups seemed
to be highly useful for hospital quality reports. Within this
group, only one indicator differed significantly between
patients and physicians. Based on these results, the quality
reports could be condensed by omitting at least the third
group of indicators (positions 20–29) judged more or less
irrelevant by both groups. At the same time, the quality
reports should be expanded by incorporating those indi-
cators of patient experiences that were rated highly or very
relevant by both patients and physicians.

Page 5

BMC Health Services Research 2007, 7:157http://www.biomedcentral.com/1472-6963/7/157
Page 5 of 6
(page number not for citation purposes)
Generalisation of the results
Some methodological issues question the generalisation
of the results. First of all, the small group of interviewees
(convenience samples of 50 patients and 50 physicians)
might have caused a selection bias. In the group of
patients, women and privately insured individuals were
overrepresented. Private insurance is usually correlated to
social class in Germany, thereby creating an overrepresen-
tation of higher social class patients in that group. This
may be an explanation for the small differences between
the patient and physician ratings of the indicator rele-
vance and for the high ratings of understandability in the
group of patients. There might have been much higher dif-
ferences in the indicator ratings if a more representative
sample of the German population had been questioned.
However, the results of our small scale study have recently
been supported by a survey of a representative sample of
the German adult population. Out of 33 indicators that
were rated according to their relevance for choosing a hos-
pital, those indicators that were rated highly or very rele-
vant in our study were rated almost the same way by the
representative population sample [14].
In addition, the overrepresentation of privately insured
individuals reflects the actual users of the hospital quality
reports. These reports were exclusively published via the
internet and higher social class individuals generally tend
to show higher internet access and use in Germany.
In regard to the physicians, the group of interviewees were
well represented by general practitioners (GPs) in Ger-
many. German GPs are required by law to counsel
patients and support them in choosing a hospital for treat-
ment. This provides justification for the selection of GPs
and the exclusion of specialized physicians for the study.
However, interviewing the first 50 GPs who agreed to par-
ticipate in the study might have caused a bias, since these
physicians might have judged the suitability and rele-
vance of the indicators more positively than a non-con-
venience sample of GPs.
Moreover, the lack of detailed explanations of the indica-
tors provided to the interviewees before collecting their
ratings could have led to misunderstandings. This could
have caused false results. As an example, there were signif-
icant differences between the ratings of relevance for the
indicators "realization of minimum volume standards"
(patients: 23rd position, physicians: 21st position) and
"volume of specified surgical procedures" (patients: 7th
position, physicians: 1st position). However, the meaning
of both indicators overlap to a great extent: the first indi-
cator shows whether a hospital fulfils the volume require-
ments set by law for five specific surgical procedures, the
second indicator stands for the volume of surgical proce-
dures in general. Since the indicator "realization of mini-
mum volume standards" has been judged not
understandable by two thirds of the patients, this large
difference might well be explained. Because most other
indicators were considered understandable, the problem
of misunderstandings might be negligible. And since the
interview setting – not commenting the indicators before
the interviewees rated understandability (patients) or suit-
ability (physicians) and relevance (both groups) – corre-
sponds to the situation in which the patient or physician
performs an internet information search of the quality
reports, the results of our study may be especially valid.
Even when using the internet, patients and physicians
might not be able to find additional explanatory informa-
tion.
Finally, the study results might be of limited value because
the indicators of the German hospital quality reports
reflect unique features of the German health care system
and only marginally compare to indicators used in other
countries. Indeed, most of the indicators refer to structural
characteristics of hospitals that are specific to Germany.
Maintaining an outpatient department is usually only per-
mitted in bigger hospitals, for example. Only those indi-
cators that have been added reflect some qualitative
features of hospitals that are commonly measured in per-
formance assessment. In addition, the methodology of
setting up the German hospital quality report did not fol-
low state of the art approaches [7-12]. There might be no
protocol that uses identical indicators.
Nevertheless, some lessons remain to be learned if one
looks at the meaning of the indicators rather than at the
individual indicator itself. Patients and physicians as tar-
get groups of hospital quality reports preferred indicators
of range, volume and availability of services in hospitals,
and of staff qualification and patient experiences over
indicators that only state numbers of cases, structures or
equipment. Since stakeholders around the world are
struggling to issue comparable information on the quality
of hospitals, it might be worthwhile not only to follow
internationally acknowledged methodology but also to
consider the views of patients and physicians as principal
target groups of the reports.
Conclusion
Considering the understandability and relevance ratings
of the indicators that form the basis of the German hospi-
tal quality reports, most of the report content seems to be
useful to patients and physicians in their choice of hospi-
tals. Participation of the target groups in the design of the
quality reports could have revealed that about one third of
the indicators were not useful and hence could have been
omitted. In addition, this study shows that at least some
qualitative indicators on patient experiences should have