ArticlePDF Available

Patient-reporting improves estimates of postoperative complication rates: a prospective cohort study in gynaecological oncology


Abstract and Figures

Background: Most studies use hospital data to calculate postoperative complication rates (PCRs). We report on improving PCR estimates through use of patient-reporting. Methods: A prospective cohort study of major surgery performed at 10 UK gynaecological cancer centres was undertaken. Hospitals entered the data contemporaneously into an online database. Patients were sent follow-up letters to capture postoperative complications. Grade II–V (Clavien–Dindo classification) patient-reported postoperative complications were verified from hospital records. Postoperative complication rate was defined as the proportion of surgeries with a Grade II–V postoperative complication. Results: Patient replies were received for 1462 (68%) of 2152 surgeries undertaken between April 2010 and February 2012. Overall, 452 Grade II–V (402 II, 50 III–V) complications were reported in 379 of the 1462 surgeries. This included 172 surgeries with 200 hospital-reported complications and 231 with 280 patient-reported complications. All (100% concordance) 36 Grade III–V and 158 of 280 (56.4% concordance) Grade II patient-reported complications were verified on hospital case-note review. The PCR using hospital-reported data was 11.8% (172 out of 1462; 95% CI 11–14), patient-reported was 15.8% (231 out of 1462; 95% CI 14–17.8), hospital and verified patient-reported was 19.4% (283 out of 1462; 95% CI 17.4–21.4) and all data were 25.9% (379 out of 1462; 95% CI 24–28). After excluding Grade II complications, the hospital and patient verified Grade III–V PCR was 3.3% (48 out of 1462; 95% CI 2.5–4.3). Conclusion: This is the first prospective study of postoperative complications we are aware of in gynaecological oncology to include the patient-reported data. Patient-reporting is invaluable for obtaining complete information on postoperative complications. Primary care case-note review is likely to improve verification rates of patient-reported Grade II complications.
Content may be subject to copyright.
Patient-reporting improves estimates of
postoperative complication rates: a
prospective cohort study in gynaecological
R Iyer
, A Gentry-Maharaj
, A Nordin
, R Liston
, M Burnell
, N Das
, R Desai
, R Gornall
, A Beardmore-Gray
K Hillaby
, S Leeson
, A Linder
, A Lopes
, D Meechan
, T Mould
, J Nevin
, A Olaitan
, B Rufford
, A Ryan
S Shanbhag
, A Thackeray
, N Wood
, K Reynolds
and U Menon
Gynaecological Cancer Research Centre, Women’s Cancer, Institute for Women’s Health, University College London, 1st Floor
Maple House, 149 Tottenham Court Road, London W1T 7DN, UK;
National Cancer Intelligence Network Gynaecology Clinical
Reference Group, 18th Floor, Portland House, Bressenden Place, London SW1E 5RS, UK;
Department of Gynaecological Cancer,
Royal Cornwall Hospitals NHS Trust, Truro, Cornwall TR1 3LJ, UK;
Department of Gynaecological Oncology, Cheltenham General
Hospital, Sandford Road, Cheltenham, Gloucestershire GL53 7AN, UK;
Department of Obstetrics and Gynaecology, Betsi
Cadwaladr University Health Board, Penrhosgarnedd, Bangor, Gwynedd, North Wales LL57 2PW, UK;
Department of
Gynaecological Oncology, The Ipswich Hospital NHS Trust, Heath Road, Ipswich, Suffolk IP4 5PD, UK;
Trent Cancer Registry,
5 Old Fulwood Road, Sheffield S10 3TG, UK;
Department of Gynaecological Oncology, University College London Hospital NHS
Foundation Trust, 2nd Floor North, 250 Euston Road, London NW1 2PG, UK;
Pan Birmingham Gynaecological Cancer Centre,
Birmingham City Hospital, Dudley Road, Birmingham, West Midlands B18 7QH, UK;
Department of Gynaecological Oncology,
Glasgow Royal Infirmary, 84 Glasgow Street, Glasgow, UK;
Department of Gynaecological Oncology, Lancashire Teaching
Hospitals NHS Foundation trust, Royal Preston Hospital, Sharoe Green Lane, North Fulwood, Preston, Lancashire PR2 9HT, UK and
Department of Gynaecological Cancer, Barts Cancer Centre, Barts and the London NHS Trust, St Bartholomew’s Hospital (Barts),
West Smithfield, London EC1A 7BE, UK
Background: Most studies use hospital data to calculate postoperative complication rates (PCRs). We report on improving PCR
estimates through use of patient-reporting.
Methods: A prospective cohort study of major surgery performed at 10 UK gynaecological cancer centres was undertaken.
Hospitals entered the data contemporaneously into an online database. Patients were sent follow-up letters to capture
postoperative complications. Grade II–V (Clavien–Dindo classification) patient-reported postoperative complications were verified
from hospital records. Postoperative complication rate was defined as the proportion of surgeries with a Grade II–V postoperative
Results: Patient replies were received for 1462 (68%) of 2152 surgeries undertaken between April 2010 and February 2012. Overall, 452
Grade II–V (402 II, 50 III–V) complications were reported in 379 of the 1462 surgeries. This included 172 surgeries with 200 hospital-
reported complications and 231 with 280 patient-reported complications. All (100% concordance) 36 Grade III–V and 158 of 280 (56.4%
concordance) Grade II patient-reported complications were verified on hospital case-note review. The PCR using hospital-reported
data was 11.8% (172 out of 1462; 95% CI 11–14), patient-reported was 15.8% (231 out of 1462; 95% CI 14–17.8), hospital and verified
*Correspondence: Dr U Menon; E mail:
Received 30 January 2013; revised 16 June 2013; accepted 22 June 2013; published online 11 July 2013
& 2013 Cancer Research UK. All rights reserved 0007 0920/13
Keywords: patient-reported; postoperative complications; follow-up; gynaecological oncology surgery; UKGOSOC
British Journal of Cancer (2013) 109, 623632 | doi: 10.1038/bjc.2013.366 | DOI:10.1038/bjc.2013.366 623
patient-reported was 19.4% (283 out of 1462; 95% CI 17.4–21.4) and all data were 25.9% (379 out of 1462; 95% CI 24–28). After
excluding Grade II complications, the hospital and patient verified Grade III–V PCR was 3.3% (48 out of 1462; 95% CI 2.5–4.3).
Conclusion: This is the first pro specti ve s tudy of postoper ative complications we are awar e o f i n gyna ecol ogical oncology t o
include the patient-reported data. Patient-reporting is invaluable for obtaining complete information on postoperative
complications. Primary care case-note review is likely to impr ove verification rates of p atient-reported G rade II complications.
There is growing interest in using patient-reported outcomes as
benchmarking standards for surgery, with complications one of the
many measures being explored. Data on complications can be
collected in a variety of ways. The most common method is
retrospective case-note review, which provides first-hand information
on the clinical course and complications experienced by the patient
while in hospital. However, it is time consuming and subject to
recording bias. Prospective data collection is an alternative used in
clinical audits such as the UK National Cardiac Surgery Register
(Bridgewater, 2010). Prospectively collected data on a limited number
of surrogate complication measures such as return to theatre and
readmission are also available as part of Hospital Episode Statistics
(HES;, an administrative database
used in the National Health Service, UK. With all approaches,
there is under reporting as a result of poor compliance with
completing audit data, incorrect coding of procedures (Cockbain
et al, 2012) and treatment of patients for complications in primary
care or in institutions other than the hospital where the initial
surgery was performed.
One strategy for overcoming some of these biases is to use
patient-reported complication data. This is distinct from the
patient-reported outcomes measures (PROMs) that are used to
evaluate the effects of treatments including surgery (Guldberg et al,
2012). Patient-reported outcomes measures are quality of life
measures of patient experience, focused on those outcomes that
matter to patients (person-focused) (McHorney, 1997). They cover
the domains of physical, emotional and social health (Howell et al,
2013) and complement disease focused outcomes such as survival,
mortality and surgical complications.
Currently, there is limited literature on the additional value of
patient-reported complications following surgery. We are only
aware of three studies (Dushey et al, 2011; Alazzawi et al, 2012;
Greenbaum et al, 2012) examining concordance of clinical and
patient-reported complications, in elective hip and knee replace-
ment surgery. These suggest variable rates of correct reporting for
different complications with good concordance for clearly defined
complications such as deep vein thrombosis (DVT) and pulmon-
ary embolism (PE) and poor concordance for those less clearly
defined such as ‘major bleeding’.
We report on whether patient-reporting of complications
in the multicentre prospective audit, the United Kingdom
Gynaecological Oncology Surgical Outcomes and Complications
(UKGOSOC) was able to improve the estimation of postoperative
morbidity resulting from gynaecological cancer surgery. Our
research questions for this study were (1) What is the concordance
between patient-reported and hospital-reported postoperative
complications? (2) What is the difference in the estimates of
overall postoperative morbidity according to data source? (3) What
is the sensitivity of hospital- and patient-reporting for detection of
postoperative complications? (4) What is the most feasible
questionnaire format for collecting patient-reported data on
postoperative complications?
Ten gynaecological cancer centres in the United Kingdom
participated in UKGOSOC. This i ncluded eight centres from
England, one f rom Wales and one from Scotland. Eligible
surgeries were defined as all major procedures undertaken on a
gynaecological oncology theatre list. In addition to cancer
surgery, this included surgery for benign conditions in women
with a complicated surgical history or high suspicion of cancer
and risk-reducing surgery in women at risk of familial
gynaecological cancer. While all operative laparoscopy was
included, diagnostic procedures and surgery for complications
were excluded.
A surgical complication was defined as ‘an undesirable and
unintended result of an operation affecting the patient that occurs
as a direct result of the operation’ (Sokol and Wilson, 2008).
Postoperative complications were defined as those occurring
following the surgery and individual complications were defined
as per Supplementary Table 1.
Data on surgery and the postoperative course in hospital were
entered by the clinicians onto a central online database. Access to
the system was restricted to users nominated by the audit lead in
each centre who were given individual unique usernames and
passwords. The surgeon entered the details of the surgery,
co-morbidity and any intraoperative complications usually in
theatre. The hospital-reported postoperative complications were
entered by the clinical team contemporaneously as they occurred
on the ward and while entering discharge details. The system was
set up in such a way that any data once entered by
the hospital team could not be amended unless they contacted
the coordinating centre (CC).
Ethics approval was sought from the Joint UCL/UCLH
Committees on the Ethics of Human Research in June 2008,
which advised that the project was considered to be an audit, not
requiring ethical review. W omen were provided written i nforma-
tion about the study. Consent was obtained to include their
personal identifiers on the central database so that follow-up
letters (FULs) could be sent postoperatively from the CC at
University College London B8 weeks following the surgery.
Women had the option of only allowing their anonymised data to
be entered, in which case they were not included in the central
follow-up from the CC. The list of women eligible for follow-up
was checked by the respective clinical teams before mailing to
ensure no letters were sent to deceased patients or those still in
Initially an open, free-text question was used to collect patient-
reported data. Women were asked, Have you had a complication
following your gynaecological surgery? If so, please give details’
(Supplementary document 1). Women were also requested to
provide their telephone numbers to allow for clarifications. Interim
analysis of the FUL was undertaken in July 2011 to elucidate the
common postoperative complications experienced by women so
that a closed format questionnaire could be developed to capture
data in a uniform manner that could be easily interpreted
and analysed. A list of 11 common postoperative complications
was derived, which included wound breakdown, infections,
pelvic/abdominal abscess/haematoma, heavy vaginal bleeding,
lymphoedema, lymphocyst, constipation, other bowel problems,
bladder problems (including incontinence and urinary retention),
DVT and PE. Every complication was briefly described and the
questions included a subset on management (whether readmission
or reoperation had been necessary). Space was provided after each
BRITISH JOURNAL OF CANCER Patient-reporting improves estimates of PCRs
624 | DOI:10.1038/bjc.2013.366
question for the patient to add any additional details if they so
wished. Women were also asked questions about the main
language spoken in their home, whether the questionnaire was in
a language that they could easily understand, and their educational
status. The responses were kept to simple ‘yes/no’ answers, with a
view to minimise free text. Initially, two formats of the
questionnaire were designed. These were then circulated among
eight non-medical female colleagues and two lay volunteers. They
were asked to comment on the questions and the format of the
questionnaire. In the first format, women were asked if they had
suffered a particular complication from the surgery. Following the
main question, space was provided to enter details regarding the
management. In the second format, the main question was
followed by a subset of specific questions regarding management
with yes/no answers. Seven out of the ten women who had been
asked to evaluate the questionnaire preferred the second format as
the questionnaire though longer than the first was easier to
complete with minimum writing required. Hence, this latter format
was adopted (Supplementary document 2).
All replies were entered on the central audit database. The data
were cleaned and analysed by a single clinician (RI), a fully trained
obstetrician and gynaecologist. The same researcher contacted all
women for clarification of equivocal replies. The postoperative
complications were classified according to the Clavien and Dindo
system in which complications are graded from I to V (with two
subsets each in Grades III and IV), based on their severity and the
intervention required (Dindo et al, 2004; Table 1). Hernias that had
been managed conservatively were included under Grade II rather
than Grade I. Grade I complications being the least severe (not
requiring any specific pharmacological/surgical/radiological inter-
vention) were excluded from future analysis as it was felt these
could be subject to individual variation. Clinical teams were
contacted for individual confirmation of all Grade II–V post-
operative complications not previously reported by the hospital.
Patient-reported readmissions, reoperations and admissions to
intensive care were forwarded as soon as the replies were received
and all other patient-reported complications were forwarded at
quarterly intervals.
The postoperative complication rate (PCR) was calculated as the
proportion of eligible surgeries with a Grade II–V postoperative
complication. Concordance was calculated as proportion of Grade
II–V patient-reported complications that were verified by the
hospital clinician.
The audit consisted of a pilot phase from 1 April 2010 to 31
January 2011 that included 3 centres and a main phase from 1
February 2011 to 29 February 2012 that included all 10 centres. In
all, 2575 surgical procedures were undertaken in women who had
provided personal identifiers. Follow-up letters were not sent
following 423 surgeries, which included 24 where the women had
died and 399 with missing or incomplete addresses. The remaining
2152 were sent FUL and replies were received for 1462 (68%)
surgeries (Figure 1).
Median age at surgery was 63 years (IQR 53–72). The final
diagnosis in the 1462 surgeries included ovarian cancer in 481,
uterine cancer in 427, cervical cancer in 80, vulval cancer in 79 and
benign pathology in 395.
In 256 of 265 (97%) questionnaire format replies, women
reported that English was the main language spoken at home. Of
the remaining nine (3%) only two women reported having
difficulty understanding English and requiring help to complete
the questionnaire. In all, 30% of the women had left school before
15 years of age and 15% had completed a bachelor’s degree
(Table 2).
Hospital-reported complications. In 172 of these 1462 surgeries,
hospitals reported 200 Grade II–V postoperative complications.
The commonest complications reported were infections (51, 26%),
wound breakdown (48, 24%), ileus (13, 7%) and bladder-related
complications (13, 7%).
Patient-reported complications
Free-text format FUL. In 1787 of the 2152 surgeries (1 November
2010 to 31 December 2011), FUL was sent using the free-text
format (Figure 2a). Replies were received for 1197 (67%). There
were 289 patient-reported complications in 265 surgeries. In all,
91 were excluded as they were Grade I postoperative complications
(67), intra-operative (four) or related to chemo/radiotherapy or
care in hospital (20).
Patient-reported Grade II–V complications: There were 198
complications related to 188 surgeries, which included 26
readmissions, 22 reoperations, four complications requiring
management in intensive care and two perioperative deaths. The
Table 1. Clavien and Dindo’s classification of complications
Grade 1 Any deviation from the normal postoperative course without
the need for pharmacological treatment or surgical,
endoscopic and radiological interventions
Allowed therapeutic regimens are drugs such as antiemetics,
antipyretics, analgesics, diuretics, electrolytes and
physiotherapy. This grade also includes wound infections
opened at the bedside
Grade II Requiring pharmacological treatment with drugs other than
such allowed for grade I complications. Blood transfusions and
TPN are also included
Grade III Requiring surgical, endoscopic or radiological intervention
IIIa Intervention not under gener al anaesthesia
IIIb Intervention under general anaesthesia
Grade IV Life-threatening complication (including CNS complications
excludes TIA) requiring IC/ICU management
IVa Single organ dysfunction (including dialy sis)
IVb Multiorgan dysfunction
Grade V Death of a patient
Abbreviations: CNS ¼ central nervous system; IC ¼ intensive care; ICU ¼ intensive care
management; TIA ¼ transient ischaemic attack; TPN ¼ total parenteral nutrition. Source:
Dindo et al (2004).
*Surgeries where women had given consent to include their identifiers
to receive follow-up letters.
with patient
2152 Surgeries
Follow-up letters
Replies for
423 Surgeries
Follow-up letters not
(reasons detailed in
373 surgeries
patient details
Figure 1. Response to follow-up letters.
Patient-reporting improves estimates of PCRs BRITISH JOURNAL OF CANCER | DOI:10.1038/bjc.2013.366 625
commonest patient-reported complications were wound break-
down, infections (mostly urinary tract and chest infections) and
lymphocysts/lymphoedema (Table 3). In all, 57 (53 surgeries) of
the 198 patient-reported complications had already been reported
by the hospitals.
Patient-only-reported complications: The remaining 141 com-
plications (135 surgeries) were reported solely on FUL. They
included 125 Grade II, 2 Grade IIIa, 9 Grade IIIb and 3 Grade IVa
complications (Table 3). In reply to the FUL, the family members
of two patients informed the CC of their relatives’ perioperative
deaths (Grade V), one due to cardiac failure and the other due to
bowel perforation. In this subgroup of patient-only-reported
complications, the commonest complications were wound break-
down, infections and lymphocysts/lymphoedema (Table 3).
Hospital-only-reported complications: For this cohort, there
were an additional 113 Grade II–V complications in 104 surgeries
reported by hospitals but not reported by patients on FUL, with the
commonest being infections followed by wound breakdown
and lymphocysts/lymphoedema. This included 10 readmissions,
3 reoperations and 3 admissions to intensive care (Table 3).
Patient comments: Women were able to add comments on the
FULs. In all, 3 of 1197 women commented that they were unsure of
what was meant by a ‘complication’. One woman also felt that the
question had been ‘too poorly defined to answer’.
Questionnaire format FUL. Following 365 surgeries between
January and February 2012, FUL was sent using the closed
questionnaire format (Figure 2b). In all, 265 (72%) replies were
received. In all, 217 complications were reported in 165 surgeries.
Ninety-nine complications were excluded as they were Grade I
(94), intra-operative complications (4), incorrectly reported
Grade II complication (1) and not related to surgery (1). The
latter was one where the family had reported death of the patient
due to progression of cancer, as a postoperative complication.
Patient-reported Grade II–V complications: The remaining 117
Grade II–V postoperative complications (101 surgeries) included 9
readmissions, 2 reoperations and 2 requiring intensive care
management (Table 4). The commonest patient-reported compli-
cations were infection, wound breakdown and lymphocyst/
lymphoedema. Six of the complications had already been reported
by the hospitals.
Patient-only-reported complications: The 111 complications
(96 surgeries) reported only on FUL included 108 Grade II,
1 Grade IIIa, 1 Grade IIIb and 1 Grade IVa. In this subgroup, once
again the commonest complications were infection, wound break-
down and lymphocyst/lymphoedema (Table 4).
Hospital-only-reported complications: For this cohort, there were
an additional 24 complications in 21 surgeries that were reported
by the hospitals but not by patients with the commonest being
infections, bladder problems and wound breakdown. This included
three readmissions, three reoperations and one admission to
intensive care (Table 4).
Patient comments: Women were asked to for their views on how
the questionnaire could be improved. Two women felt inclusion of
‘Gynaecological Cancer Research Centre’ in the return address
(printed at the back of the envelope), breached confidentiality
about their diagnosis. The other comments included the question
on educational status being inappropriate, request for larger print
size and for the questionn aire to be sent soon after surgery to avoid
surgical complications being confused with those related to chemo/
Concordance of complications
Grade III-V complications. There were 36 patient-reported
complications with significant sequelae such as reoperations,
admissions to intensive care and perioperative deaths of which
17 had been previously reported by the hospitals. The Grade of the
remaining patient-only-reported 19 complications was confirmed
by the clinician resulting in 100% concordance for complication
Grade. The details of 1 of these 19 patient-only-reported
complications were found to be incorrect. This was a case of
patient-reported vault dehiscence requiring resuturing in theatre
when in fact the vault was intact and only an examination under
anaesthesia had been performed. This resulted in 97.2% (35 out of 36)
concordance for complication type for Grade III–V patient-reported
Grade II complications. There were 280 patient-reported Grade II
complications of which 46 had been previously reported by the
hospitals. The remaining 234 patient-only-reported complications
were forwarded to the respective centres for the clinicians to verify
from hospital records. Case notes for 221 (94.4%) of these
complications were checked and the complication grade and type
was confirmed for 113. These included 34 infections (25 urinary
tract infections, 5 pyrexia of unknown origin, 1 each of chest
infection, cellulitis, gastroenteritis and Clostridium difficile
Table 2. Demographics of women who underwent surgery
Number Median (IQR)
Age 1462 63 years (53–72)
Diagnosis %
481 33
427 29
Cervical 80 6
79 5
Benign 395 27
Total 1462
Main language spoken at home
(n ¼ 265)
English 256 97
Questionnaire in a language that could be understood
(n ¼ 265)
Yes 263 99
No 2 1
Help required to complete questionnaire
(n ¼ 265)
Yes 2 1
No 263 99
Educational status
(n ¼ 265)
Finished school at or before 15 years of age 79 30
Completed GCSEs, O levels or equivalent 67 25
Completed A levels or equivalent 21 8
Completed further education but not a
45 17
Completed a bachelor’s degree/master’s
41 15
Other (please specify) 5 2
Missing 7 3
Total 265
Abbreviations: IQR ¼ interquartile range; GCSE ¼ General Certificate of Secondary Education.
Includes primary ovarian, fallopian tube, primary peritoneal, non-gynae primary, unknown
primary and synchronous cancers.
Includes primary endometrial cancer, carcinosarcomas and uterine sarcomas.
Includes primary vulval and vaginal cancers.
Limited to women who completed the questionnaire format of follow-up letters.
Five Welsh, two Polish, one Greek and one French.
BRITISH JOURNAL OF CANCER Patient-reporting improves estimates of PCRs
626 | DOI:10.1038/bjc.2013.366
diarrhoea), 33 wound breakdowns, 9 lymphoedema, 6 lympho-
cysts, 5 haematomas, 4 DVTs, 4 PEs, 3 secondary haemorrhages,
3 readmissions to hospital with vomiting and abdominal pains
(no obvious cause found), 2 cases of ileus, 2 cases of severe
constipation, 2 hernias, 1 case each of dural tap, colovaginal
fistula, urinary retention, pressure sore, haematemesis (secondary
to stres s ulcer) and allergic reaction to antibiotics. One case of PE
had been wrongly reported by the patient as a postoperative
complication when in fact it had occurred before surgery and
therefore was excluded. The concordance for complication Grade
for patient- report ed Grade II complications was 56 .4%
({46 þ 112 ¼ 158}/280). Excluding the incorrectly reported PE,
279 patient-reported Grade II complications were included in
further analysis.
In the case of allergic reaction, the patient had reported allergy
to antibiotics when in fact the allergy was transfusion-related to
pooled platelets. There was also a case of readmission for diarrhoea
which was confirmed by the clinician. Although the complication
type was correctly reported, the causative agent was not
Clostridium difficile as reported by the patient. This resulted in
55.7% (156 out of 280) concordance for complication type for
patient-reported Grade II complications.
The centres were unable to confirm the remaining 108 Grade II
complications, which included 43 wound breakdowns, 39 infec-
tions (25 urinary tract infections, 8 chest and 6 pyrexia of unknown
origin), 9 lymphoedema (5 treated with compression stocking and
4 treated with physiotherapy), 5 lymphocysts (drained in the
outpatients department), 4 bowel-related complications (2 cases
of severe constipation requiring readmission and enemas, 1 case
of ileus requiring nasogastric tube insertion, 1 case of bowel
obstruction requiring readmission and steroids), 3 bladder-related
complications (2 cases of urinary retention requiring recatheter-
isation and 1 case of extreme urge incontinence requiring
treatment by urologists), 2 hernias, 1 case each of neuropathic
pain, depression and pressure sores.
Postoperative complication rate. A postoperative Grade II–V
complication was reported in 379 of the 1462 surgeries. This
included a total of 452 (402 Grade II that includes 4 hernias, 50
Grade III–V) complications. Of the 379 surgeries with a reported
postoperative Grade II–V complication, 172 had at least one
hospital-reported complication, 231 had at least 1 patient-reported
complication of which 124 were verified and 107 were not
(Table 5).
On hospital-reporting, the proportion of surgeries with a
postoperative complication was 11.8% (172 out of 1462; 95% CI
11–14) and on patient-only-reporting it was 15.8% (231 out of
1462; 95% CI 14–17.8). Using hospital and hospital verified FUL
data, this rate increased to 19.4% (283 out of 1462; 95% CI
17.4–21.4). Using hospital and all FUL data, the rate was 25.9%
(379 out of 1462; 95% CI 24–28).
1787 Surgeries
sent FUL
Replies for 1197
265 Surgeries reported
265 Post-op
complications in 241
198 Grade II–V
complications following
188 surgeries
141 Grade II–V complications
following 135 surgeries
reported only in FUL
57 Grade II–V complications
following 53 surgeries
previously reported by centres
67 Grade I
following 63 surgeries
20 Surgeries with complications
related to
chemo/radiotherapy/care in
4 intra-op complications
1016 Surgeries
No complications
reported on FUL
97 Grade II–V post-op
complications in 89 surgeries
reported by the centres
365 Surgeries
sent FUL
Replies for
265 surgeries
165 Reported
1 Complication not
related to surgery
4 intra-op
212 Post-op
complications in 159
117 Grade II–V
complications following
101 surgeries
111 Grade II–V complications
following 96 surgeries reported
only in follow-up letters
6 Grade II–V complications
following 5 surgeries
previously reported by hospital
94 Grade I complications
following 77 surgeries
and one incorrectly
reported grade II
complication excluded
100 Surgeries
No complications
reported on FUL
14 Grade II–V complications in
9 surgeries reported by
Figure 2. Patient-reported postoperative complications. (A) Follow-up letters (FULs) that used free-text format. (B) FULs that used questionnaire
Patient-reporting improves estimates of PCRs BRITISH JOURNAL OF CANCER | DOI:10.1038/bjc.2013.366 627
Excluding Grade II complications, the hospital-reported Grade
III-V PCR was 2.0% (29 out of 1462; 95% CI 1.4–2.8). Using
hospital and hospital-verified FUL data, this rate increased to 3.3%
(48 out of 1462; 95% CI 2.5–4.3). Since all the Grade III–V patient-
only-reported complications had been confirmed and found to be
correct, this rate was the same when all FUL data were included.
Post-op complication rate for cancer surgery. The hospital-
reported Grade II–V PCR for gynaecological cancer surgery
(1067), after excluding surgery for benign disease (395), was 14%
(146 out of 1067; 95% CI 12–17). Using hospital and hospital
verified FUL data, this rate increased to 21.5% (229 out of 1067;
95% CI 19–24). Using hospital and all FUL data, the rate was 27%
(289 out of 1067; 95% CI 25–30).
Excluding Grade II complications, the hospital-reported Grade
III–V PCR for gynaecological cancer surgery (1067) was 2.3%
(24 out of 1067; 95% CI 1.5–3.3). Using hospital and hospital
verified FUL data, this rate increased to 3.5% (37 out of 1067; 95%
CI 2.5–4.7). Again, since all the Grade III–V patient-only-reported
complications had been confirmed and found to be correct, this
rate was the same when all FUL data were included.
Sensitivity for detection of postoperative complications. The
sensitivity of hospital-reporting for detection of all 379 surgeries
with Grade II–V postoperative complications was 44% (200 out of
452; 95% CI 40–49) and that of patient-reporting was 70% (315 out
of 452; 95% CI 65–74) (Table 6). When the free-text format was
used for FUL, sensitivity for hospital-reporting was 55% (95% CI
49–60) and 64% (95% CI 58–69) for patient-reporting. With the
questionnaire format, sensitivity of hospital-reporting of complica-
tions was 21% (95% CI 15–29) with patient-reporting being 83%
(95% CI 76–88).
Excluding the 121 (108 Grade II and 13 notes not checked)
complications not confirmed by the hospital, the sensitivity for
patient-reporting was 59% (194 out of 331; 95% CI 53–64) using
both questionnaire formats and for hospital-reporting was 60%
(200 out of 331; 95% CI 55–66) (Table 6).
Grade II complications accounted for 402 (89%) (279 patient-
reported, 123 hospital-only-reported) out of the total 452
complications. Excluding these, the overall sensitivity of hospital-
reporting for detection of Grade III–V postoperative complications
was 62% (31 out of 50; 95% CI 48–74) and patient-reporting was
72% (36 out of 50; 95% CI 58–83) (Table 6).
Types of postoperative complications reported by hospital and
patients. Hospital-reporting appeared better for cardiac complica-
tions, ileus, bladder complications, bowel obstruction and
respiratory complications. Patients were better at reporting hernia,
wound breakdown, DVT, lymphocysts/lymphoedema, neurological
complications and pelvic/abdominal abscess/haematoma. Both
hospital and patients had similar reporting rates for anastomotic
leak, fistula, primary haemorrhage, bowel perforation and
Table 3. Grade II–V postoperative complications from follow-up letters which used free-text format
Patient-reported complications all (only reported by patient)
Complication category Total Grade II Grade IIIa Grade IIIb Grade IVa Grade V
Only reported
by hospital Overall total
Wound breakdown 73 (54) 63 (47) 9 (6) 1 (1) 22 95
Infection 42 (32) 42 (32) 35 77
Lymphocyst/Lymphoedema 19 (18) 19 (18) 827
Abscess/Haematoma 8 (4) 5 (3) 2 (1) 1 4 12
Bladder problems 5 (4) 5 (4) 611
Ileus 5 (2) 5 (2) 611
Bowel obstruction 2 (1) 1 1 (1) 3 5
Bowel perforation 1 (1) 1 (1) 0 1
Bowel other 4 (2) 4 (2) 26
Fistula 4 (1) 4 (1) 2 6
Primary haemorrhage 4 (1) 3 1 (1) 1 5
Secondary haemorrhage 2 2 46
Deep vein thrombosis 2 (2) 2 (2) 13
Pulmonary embolism 2 (2) 2 (2) 24
Cardiac 3 (1) 2 1 (1) 4 7
Respiratory 2 (2) 1 (1) 1 (1) 4 6
Neurological 3 (2) 3 (2) 14
Hernia 3 (3) 3 (3) 03
Anastomotic leak 2 2 0 2
Psychiatric 1 (1) 1 (1) 12
Other complications 11 (8) 8 (7) 2 (1) 1 7 18
Total 198 (141) 167 (125) 3 (2) 22 (9) 4 (3) 2 (2) 113 311
Complications reported by both hospital and patients ¼ all patients reported those only reported by patient. Readmissions ¼ 26 (21 patient-only-reported, 5 patient- and hospital-
reported) þ 10 hospital-only-reported. Reoperations ¼ 22 (9 patient-only-reported, 13 patient- and hospital-reported) þ 3 hospital-only–reported. Admissions to intensive care ¼ 4 (3 patient-
only-reported, 1 patient- and hospital-reported) þ 3 hospital-only-reported. Perioperative deaths ¼ 2 (both patient-only-reported).
BRITISH JOURNAL OF CANCER Patient-reporting improves estimates of PCRs
628 | DOI:10.1038/bjc.2013.366
psychiatric complications (Supplementary Table 2). The numbers
were too small for any formal statistical comparisons.
To our knowledge, this is the first study to use both hospital- and
patient-reported information to estimate the overall postoperative
morbidity in gynaecological oncology surgery. Concordance of
patient-reported complications with hospital case-note review was
100% for Grade III–V and 56.4% for Grade II postoperative
complications. The hospital-reported postoperative Grade II–V
complication rate for major surgery undertaken in gynaecological
oncology centres of 11.8% increased to 19.4% if hospital verified
patient-reported complications were also included and 25.9% on
inclusion of all patient-reported complications. The hospital and
patient verified Grade III–V PCR was 3.3%. Overall, sensitivity for
patient-reporting was 70% and hospital-reporting was 44%. During
the study, a closed format questionnaire was developed that
enabled more accurate capture of complication rates. The
questionnaire and the process set-up in UKGOSOC could
therefore better inform future data capture of complications in
gynaecological oncology surgery.
Patients reported a higher proportion of the overall 452 Grade
II–V complications when compared with hospitals (55.8% vs
44.2%). A survey of patients following radical prostatectomy also
noted that patients reported more complications in comparison
with previous hospital/clinician reported rates (Fowler et al, 1993).
However, the sensitivity of patient- and hospital-reporting was
similar (59% vs 60%) when the 121 Grade II complications not
Table 4. Grade II–V postoperative complications from follow-up letters which used questionnaire format
Patient-reported complications all (only reported by patient)
Complication category Total Grade II Grade IIIa Grade IIIb Grade IVa
Only reported
by hospital
Overall Total
Infection 44 (43) 43 (43) 1 5 49
Wound breakdown 41 (38) 40 (37) 1 (1) 4 45
Lymphocyst/Lymphoedema 12 (12) 12 (12) 0 12
Bladder problems 1 1 56
Ileus 1 (1) 1 (1) 4 5
Bowel obstruction 1 (1) 1 (1) 1 2
Bowel perforation 11
Bowel other 3 (3) 3 (3) 2 5
Fistula 1 (1) 1 (1) 0 1
Secondary haemorrhage 4 (3) 3 (3) 1 0 4
Abscess/Haematoma 2 (2) 2 (2) 0 2
Deep vein thrombosis 2 (2) 2 (2) 0 2
Pulmonary embolism 1 (1) 1 (1) 0 1
Hernia 1 (1) 1(1) 0 1
Ureteric obstructi on 1 (1) 1 (1) 0 1
Other complications 2 (2) 1 (1) 1 (1) 2 4
Total 117 (111) 112 (108) 1 (1) 2 (1) 2 (1) 24 141
Complications reported by both hospital and patients ¼ all patients reported those only reported by patient. Readmissions ¼ 9 (9 patient-only-reported)þ 3 hospital-only-reported.
Reoperations ¼ 2 (1 patient-only-reported, 1 patient- and hospital-reported) þ 3 hospital-only–reported. Admissions to intensive care ¼ 2 (1 patient-only-reported, 1 patient- and hospital-
reported) þ 1 hospital-only–reported.
Table 5. Proportion of surgeries with a post-operative complication
Post-operative complications
Highest grade of
complication Hospital-reported
Verified on hospital
notes review
Not verified on
hospital notes review Total
Hospital and
patient verified
All hospital- and
II 143 105 107 212 235 331
III–V 29 19 0 19 48 48
Total surgery 172 124 107 231 283 379
Patient-reporting improves estimates of PCRs BRITISH JOURNAL OF CANCER | DOI:10.1038/bjc.2013.366 629
confirmed on hospital case-note review were excluded. Patients
were better at reporting complications that had occurred following
discharge such as wound breakdown, pelvic abscess/haematoma,
DVT, lymphocysts/lymphoedema and hernias while hospitals
seemed better at reporting complications that had occurred
during the hospital stay such as ileus, bowel obstruction, bladder
(e.g. urinary retention), cardiac (e.g. atrial fibrillation) and
respiratory complications (e.g. pulmonary oedema).
Hospital notes of 94.8% (240 out of 253) of those with patient-
reported complications previously undocumented by the clinical
staff were reviewed. The clinical team confirmed all Grade III–V
patient-reported complications. This probably reflects the fact that
these were complications with significant sequelae requiring
secondary care management. Grade II complications such as
infections treated with antibiotics and lymphoedema treated
with compression stockings and physiotherapy were less likely
(concordance 56.4%) to be confirmed. While it is unlikely that
patients incorrectly reported use of antibiotics or compression
stockings, the possibility cannot be entirely ruled out. However, the
more likely explanation is that the surgical teams did not manage
these complications. A significant proportion was probably
managed in primary care. The wording of patient consent meant
that the coordinating centre team was unable to request review
of primary care records. In addition, some of the readmissions
are likely to have involved local hospitals, different from where
the initial surgery had been performed. Both these issues
were noted in the elective hip and knee replacement studies in
which about half of the surgical complications were managed
outside the institution where the initial surgery was undertaken
(Dushey et al, 2011; Greenbaum et al, 2012) and would have
been missed if only clinician-reported data were used. Logistic
issues may also have contributed to clinicians not entering s ome
of the post discharge Grade II–V complications that they were
aware of. As it is medical treatment that defines a complication
as Grade II, the issue of variation in threshold for prescription
of antibiotics for p ostoperative infections also needs to be
The open free-text format for collecting patient data proved
time consuming to analyse, requiring a clinician’s input to decipher
and enter the complications into the database. A minority of
women did not understand what was meant by a complication and
some women mentioned complications related to non-surgical
Table 6. Sensitivity of patient- and hospital-reporting for Grade II–V postoperative complications
Sensitivity (95% CI)
Data source
No. of Grade II-–V
Patient-reporting Hospital-reporting
All Grade II–V complications
Patient-reporting using free-text format
Patient-reported alone 141 64% (58–69) 55% (49–60)
Patient- and Hospital-reported 57
Hospital-reported alone 113
Total 311
Patient-reporting using questionnaire format
Patient-reported alone 111 83% (76–88) 21% (15–29)
Patient- and Hospital-reported 6
Hospital-reported alone 24
Total 141
Patient-reporting using both formats
Patient-reported alone 252 70% (65–74) 44% (40–49)
Patient- and Hospital-reported 63
Hospital-reported alone 137
Total 452
Patient-reporting using both formats excluding complications not confirmed by the hospital (n ¼ 121
Patient-reported alone 131 59% (53–64) 60% (55–66)
Patient- and Hospital-reported 63
Hospital-reported alone 137
Total 331
Grade III–V complications only
Patient-reporting using both formats
Patient-reported alone 19 72% (58–83) 62% (48–74)
Patient- and Hospital-reported 17
Hospital-reported alone 14
Total 50
108 Grade II, 13 notes not checked.
BRITISH JOURNAL OF CANCER Patient-reporting improves estimates of PCRs
630 | DOI:10.1038/bjc.2013.366
treatments or detailed problems related to their care in hospital.
The structured questionnaire (closed) format for patient-reporting
developed in the course of the study allowed easier interpretation
and grading of the complications. It comprises specific questions
pertaining to the management of 11 common postoperative
complications that were highlighted on analysis of the free-text
format of FULs. Every question included a brief description of the
complication with management options clearly specified. Simple
Yes/No answers also probably made completion easier for women.
The closed format also decreased the number of replies with
complications not related to surgery. The proportion of replies
reporting a complication was higher with this format (63% vs 22%)
when compared with the free-text format. However, a large
proportion (44% vs 25%) were Grade I complications, with the
commonest being constipation requiring diet changes/laxatives
and urinary incontinence not requiring any medication. This was
probably related to the inclusion of specific questions regarding
bowel and bladder problems. At present, there is no nationally
agreed list of complications that could be used to audit surgical
outcomes in gynaecological oncology. It might be feasible to
shorten the list of complications in the closed format from eleven
to five or six core complications for use in future local and/or
national audits. The reliability of this approach would however
have to be tested in a further prospective study.
In our study, the overall response rate was 68% with a similar
rate (72% vs 68%) associated with the use of a closed vs free-text
format for postal follow-up. Studies investigating patient-reported
postoperative complications following elective surgery have
reported response rates ranging from 80% (hip and knee
replacement surgery), 73% for hernia repair and 65% for varicose
vein surgery (Alazzawi et al, 2012; Greenbaum et al, 2012; Grosse
Frie et al, 2012). These studies also used a questionnaire format
containing questions regarding specific postoperative complica-
tions and simple yes/no answers. It is likely that response rates
could have been improved by sending reminders to non-
Strengths of our study included the size, multicentre design and
prospective online data collection by clinical teams, 68% patient
response rate, the same clinician (a general obstetrician and
gynaecologist) undertaking all patient interviews where data were
equivocal, hospital case-note review of patient-only-reported
complications and central-independent data analysis. The main
limitation was that the coordinating centre could not contact the
primary care teams to verify complications that were not managed
by the surgical team. Only those women who had provided
telephone numbers could be contacted directly for clarification. In
the absence of a validated questionnaire on postoperative
complications in gynaecological oncology, we designed one to
capture more accurate and precise information regarding compli-
cations. Although it was piloted and women provided feedback on
its content, it requires further validation in future studies. In
common with all questionnaire studies, one could speculate that
women were more likely to respond to the questionnaire if they
had experienced a complication.
Finally though the intention was to send the FULs 8 weeks
postoperatively this was not always possible due to delays in
receiving updates from the hospitals regarding any patients who
might have died or were terminally ill. The latter step was essential
to avoid causing unnecessary distress to family members. Despite
this, four (0.2%) FULs were sent to deceased patients and one of
the families complained prompting a written apology. Delays in
sending the FUL probably contributed to recall bias causing some
women to confuse surgical complications with side effects from
chemo/radiotherapy (commenced usually within 6 weeks of
There is growing interest in using PROMs to assess outcomes of
cancer treatment (Howell et al, 2013). The PROMs are designed to
assess the quality of life and long-term disability from treatment
and not surgical complications in particular. A recent study
(Andikyan et al, 2012) in gynaecological cancer looked at the
feasibility of capturing patient-reported symptoms electronically in
the immediate 6-week postoperative period following major
surgery. The authors concluded that this method was highly
acceptable to the women and provided useful information
regarding problems experienced by patients which could be helpful
to the clinicians in providing timely and appropriate interventions
where required. The Royal College of Obstetricians and Gynaecol-
ogists (RCOG) recently published a scientific impact paper
evaluating the use of PROMs in gynaecology and gynaecological
oncology (RCOG, 2012). It is envisaged that in future PROMs will
routinely be collected in the United Kingdom for all gynaecological
cancer patients. Linking or combining our follow-up questionnaire
to PROMs would be a cost-effective method of collecting data on
postoperative complications.
Conclusions. Follow-up letters provide valuable additional infor-
mation on complications experienced by women following
gynaecological cancer surgery and both hospital and patient
reporting are essential to obtain a complete picture. Future studies
on surgical complications should endeavour to use both dat a
sources and include primary care record verification of uncon-
firmed patient-reported Grade II complications.
We are very grateful to all the women who participated in this
study and to all the medical, nursing and administrative staff who
worked on UKGOSOC. In addition, we would like to thank all the
members of the gynaecological oncology multidisciplinary teams at
the participating hospitals and in particular: Philip Toon, Richard
Peevor, Sara Roberts (Betsi Cadwaladr University Health Board),
Janos Balega, Charlie Chan, Ahmed Elattar, David Luesley, Esther
Moss, Kavita Singh, Sudha Sundar, Mary Wright (City Hospital
Birmingham), Alta Viljoen (Cheltenham General Hospital),
Mohamed Ismail, Vivek Nama, Branislav Potancok, Cheryl Walke
(East Kent University Hospital NHS Foundation Trust), Deborah
Woods, Alison Garnham (Ipswich General Hospital), Pierre
Martin-Hirsch, Patrick Keating, Ketan Gajjar, Deborah Parkinson
(Royal Preston Hospital), Arnold Kruse, Emma Arthur (Royal
Cornwall Hospital), Kostas Doufekas, Ranjit Manchanda, Nicola
MacDonald and Martin Widschwendter (University College
London Hospital). The authors would like to thank the Eve
Appeal for their support of the study. The study was supported by
researchers at the National Institute for Health Research University
College London Hospitals Biomedical Research Centre.
The authors declare no conflict of interest.
Alazzawi S, Bardakos NV, Hadfield SG, Butt U, Beer ZH, Field RE (2012)
Patient-reported complications after elective joint replacement surgery: are
they correct? J Bone Joint Surg Br 94-B(8): 1120.
Andikyan V, Rezk Y, Einstein MH, Gualtiere G, Leitao Jr MM, Sonoda Y,
Abu-Rustum NR, Barakat RR, Basch EM, Chi DS (2012) A prospective
study of the feasibility and acceptability of a Web-based, electronic
patient-reported outcome system in assessing patient recovery after major
gynecologic cancer surgery. Gynecol Oncol 127(2): 273–277.
Bridgewater B (2010) Cardiac registers: the adult cardiac surgery register.
Heart 96(18): 1441–1443.
Patient-reporting improves estimates of PCRs BRITISH JOURNAL OF CANCER | DOI:10.1038/bjc.2013.366 631
Cockbain AJ, Carolan M, Berridge D, Toogood GJ (2012) Performance and
Quality Indicators: The importance of accurate coding. Bull Royal College
Surg Engl 94(2): 46–50.
Dindo D, Demartines N, Clavien PA (2004) Classification of surgical
complications - A new proposal with evaluation in a cohort of 6336
patients and results of a survey. Ann Surg 240(2): 205–213.
Dushey CH, Bornstein LJ, Alexiades MM, Westrich GH (2011) Short-term
coagulation complications following total knee arthroplasty. A comparison
of patient-reported and surgeon-verified complication rates. J Arthroplasty
26(8): 1338–1342.
Fowler FJ, Barry MJ, Lu Yao G, Roman A, Wasson J, Wennberg JE (1993)
Patient-reported complications and follow-up treatment after radical
prostatectomy. The National Medicare Experience: 1988-1990 (updated
June 1993). Urology 42(6): 622–629.
Greenbaum J, Bornstein L, Lyman S, Alexiades M, Westrich G (2012)
The validity of self-report as a technique for measuring short-term
complications after total hip arthroplasty in a joint replacement registry.
J Arthroplasty 27(7): 1310.
Grosse Frie K, van der Meulen J, Black N (2012) Relationship between
patients’ reports of complications and symptoms, disability and quality of
life after surgery. Br J Surg 99(8): 1156.
Guldberg R, Kesmodel U, Hansen J, Gradel K, Brostrøm S, Kærlev L,
Nørgård B (2012) Patient-reported outcome measures in women
undergoing surgery for urinary incontinence and pelvic organ prolapse in
Denmark, 2006–2011. Int Urogynecol J 0937-3462: 1–8.
Howell D, Fitch M, Bakker D, Green E, Sussman J, Mayo S, Mohammed S,
Lee C, Doran D (2013) Core domains for a person-focused
outcome measurement system in cancer (PROMS-Cancer Core) for
routine care: a scoping review and Canadian Delphi Consensus. Value
Health 16(1): 76.
McHorney CA (1997) Generic health measurement: past accomplishments
and a measurement paradigm for the 21st century. Ann Intern Med
127(8 Pt 2): 743.
RCOG (2012) Patient-Reported Outcome Measures in Gynaecology
(Scientific Impact Paper 31) RCOG,
Sokol D, Wilson J (2008) What is a surgical complication? World J Surg 32(6):
This work is published under the standard license to publish agree-
ment. After 12 months the work will become freely available and
the license terms will switch to a Creative Commons Attribution-
NonCommercial-Share Alike 3.0 Unported License.
Supplementary Information accompanies this paper on British Journal of Cancer website (
BRITISH JOURNAL OF CANCER Patient-reporting improves estimates of PCRs
632 | DOI:10.1038/bjc.2013.366
... complications were comparable to those reported in medical files in orthopedic and gynecologic study populations [21,22]. To our knowledge, no prior studies have been performed to evaluate differences between complications reported by clinicians and by patients or parents in a hypospadias or pediatric urology population. ...
... Although previous studies into patient-reported complications showed good agreement with medical records [21,22,26], our results are in line with the study of Bream et al. who found much lower agreement [20]. The former studies are hard to compare with our study owing to methodological differences. ...
... Firstly, we used parent instead of patient-reported complications, because of the young age at which children had surgery. Secondly, we focused on comparison between hospital records and parents' reports, whereas others verified reported complications with records from general practitioners (GPs) and other medical institutions [21,22,26]. Thirdly, owing to the detailed information from our questionnaires, we were able to analyze agreement for subtypes of complications instead of for complications in general only. ...
Purpose: To analyze agreement on postoperative complications after hypospadias surgery according to medical records and parents' reports. Materials & methods: In this retrospective cohort study, data were collected from 409 children who received an initial one-stage hypospadias correction in the Radboudumc, The Netherlands. Postoperative complications according to medical records were compared with parent-reported complications in an online questionnaire. Main complications studied were wound-related complications, urinary tract infections, fistulas, stenosis, and prepuce-related complications. Agreement was determined by Cohen's kappa coefficient. Results: Slightly less complications were mentioned in medical records (37%) compared to parents' reports (42%). Overall agreement was moderate (κ = 0.50, 95% confidence interval (CI):0.41-0.59), but poor for some specific complications. Agreement was higher for complications that needed reoperation compared to when no reoperation was performed (κ = 0.53, 95% CI: 0.43-0.62 and κ = 0.18, 95% CI: 0.06-0.31) and for patients with recent surgery (<5 years before questionnaire completion) compared to less recent surgeries (κ = 0.69, 95% CI: 0.55-0.84 and κ = 0.43, 95% CI: 0.33-0.54). Conclusions: Agreement on complications according to medical records and parents' reports was poor to moderate, but better after reoperation and more recent surgery. Some complications mentioned in medical records were missing from parents' reports and the other way around. Better agreement will give physicians and parents a more reliable view on postoperative outcome after hypospadias surgery. Type of study: Diagnostic test. Level of evidence: Level III.
... 18 The accuracy of patient reports of medical complications has not been investigated as thoroughly among surgical patients. Existing studies have been limited to either a single type of surgery [19][20][21] or to a single postoperative complication. [22][23][24] Additional characterization of PROs in a diverse surgical population is necessary if they are to be used to identify postoperative complications for clinical care, research, or quality improvement. ...
... Low positive agreement (0 to 71%) and high negative agreement (71 to greater than 99%) for patient-reported postoperative complications have been observed after hernia repair 19,20 and after gynecologic oncology surgery. 21 Some studies of orthopedic populations provide limited information because the medical record was only consulted if patient report of a postoperative complication was positive. [27][28][29] The only complication that has been investigated in diverse surgical populations is wound infection, for which similar rates of positive agreement (47 to 83%) and negative agreement (95 to 98%) have been found. ...
Full-text available
Background: Anesthesiologists need tools to accurately track postoperative outcomes. The accuracy of patient report in identifying a wide variety of postoperative complications after diverse surgical procedures has not previously been investigated. Methods: In this cohort study, 1,578 adult surgical patients completed a survey at least 30 days after their procedure asking if they had experienced any of 18 complications while in the hospital after surgery. Patient responses were compared to the results of an automated electronic chart review and (for a random subset of 750 patients) to a manual chart review. Results from automated chart review were also compared to those from manual chart review. Forty-two randomly selected patients were contacted by telephone to explore reasons for discrepancies between patient report and manual chart review. Results: Comparisons between patient report, automated chart review, and manual chart review demonstrated poor-to-moderate positive agreement (range, 0 to 58%) and excellent negative agreement (range, 82 to 100%). Discordance between patient report and manual chart review was frequently explicable by patients reporting events that happened outside the time period of interest. Conclusions: Patient report can provide information about subjective experiences or events that happen after hospital discharge, but often yields different results from chart review for specific in-hospital complications. Effective in-hospital communication with patients and thoughtful survey design may increase the quality of patient-reported complication data.
... They also showed that imperfect timing of QALY could bias economic conclusions. If nothing else, PROMS provides an opportunity to improve the quality of reporting of conventional complications [59] or to enhance clinical screening for emerging complications [60]. ...
Full-text available
Purpose of Review Population-based increases in ageing and medical co-morbidities are expected to substantially increase the incidence of expensive postoperative complications. This threatens the sustainability of essential surgical care, with negative impacts on patients’ health and wellbeing. Recent Findings Identification of key high-risk areas, and implementation of proven cost-effective strategies to manage both outcome and cost across the end-to-end journey of the surgical episode of care, is clearly feasible. However, good programme design and formal cost-effectiveness analysis is critical to identify, and implement, true high value change. Summary Both outcome and cost need to be a high priority for both fundholders and clinicians in perioperative care, with the focus for both groups on delivering high-quality care, which in itself, is the key to good cost management.
... The continuously growing number of oncological procedures delivered by means of robotic surgical systems and the controversial debate regarding their cost-effectiveness with respect to traditional surgery led to the need of performance measurement systems to evaluate the effects of treatments, including surgery, that are able to go beyond the traditional clinical view of mortality and complications rates [12][13][14][15] . Beside the objective outcome measurements evaluated by clinicians, there are also subjective measures such as patient satisfaction, quality of life, and patient experience. ...
Full-text available
Patient Reported Outcome and Experience Measures (PROMs and PREMs) play an increasingly important role in monitoring the quality of the oncological pathway. The aim of this study is to describe the case of five hospitals a year after the adoption of PROMs and PREMs for robotic oncological colorectal surgery in Tuscany and to investigate how the clinicians can impact the process of implementation and the efficacy of such measures. We used 14 months of data from the five robotic centers in Tuscany. Above all, the physician's personal motivation to improve the treatment of patients, the teamwork, and the possibility to use data for research purposes proved to be the essential factors for their engagement and the successful implementation of patient reported measures. Physicians play a key role in the adoption of systematic PROMs and PREMs. The higher their level of engagement, the higher the collection success, both in terms of number of patients enrolled and response rates. Moreover, the collection of patient reported measures may become part of physicians' daily practice and may lead to a change in their relationship and communication with patients, as clinicians accept to have their job reviewed and are not afraid to be evaluated by their patients.
... Previous studies have shown agreement of patient-reported post-surgical complications with surgeon-evaluated clinical diagnoses or chart review (31,32). Given the lack of data available from hospital records (33), potential for selection bias with chart review, and the general lack of consistent documentation of clinical visits in hospital records, complications data was collected via 30-day calls through the following algorithm (more information in Appendix S1). ...
... 31 In this context, it should be argued that patient reporting would improve our appreciation of 'real' postoperative complication rates. 21 The retrospective nature of our analysis resulted in our study having a number of limitations. Although the questionnaire we used had been previously validated, we cannot guarantee the accuracy of all the information received as we were unable (in real time) to clarify responses to some of the questions or to independently confirm complications. ...
Full-text available
Objectives To identify the frequency of postoperative complications, including problems identified by patients and complications occurring after discharge from hospital. To identify how these impact on quality of life (QoL) and the patient’s perception of the success of their treatment. Design Data from three prospective sources: surgical audit, a telephone interview (2 weeks after discharge) and a patient-focused questionnaire (2 months after surgery) were retrospectively analysed. Setting Dunedin Hospital, Dunedin, New Zealand. Participants Of the 500 patients, 100 undergoing each of the following types of surgeries: anorectal, biliary, colorectal, hernia and skin. Primary and secondary outcome measures The primary outcomes were complications and the 36-item Short Form Health Survey (SF-36). Secondary outcomes included the patient’s ratings of their treatment and a questionnaire-derived patient satisfaction score. Results 226 patients reported a complication; there were 344 separate complications and 411 reports of complications (16% of complications were reported on more than one occasion). The audit, telephone interview and questionnaire captured 12.6%, 36.3% and 51% of the 411 reports, respectively. Patients with complications had a lower SF-36 Physical Composite Summary (PCS) score (48.5 vs 43.9, p=0.021) and a lower Patient Satisfaction Score (85.6 vs 74.6, p<0.001). Rating of information received, care received, symptoms experienced, QoL and satisfaction with surgery were all significantly worse for patients with complications. On linear regression analysis, surgical complications, American Society of Anaesthesiologists score and age all made a similar contribution to the SF-36 PCS score, with standardised beta coefficients between 0.19 and 0.21. Conclusions Following surgery, over 40% of patients experienced complications. The QoL and satisfaction score were significantly less than for those without complications. The majority of complications were diagnosed after discharge from hospital. Taking more notice of the patient perspective helps us to identify problems, to understand what is important to them and may suggest ways to improve perioperative care.
... Previous studies have compared patient-reported complications (PRC) to clinical examination and medical records in general, inguinal hernia repair, bone marrow transplant, varicose vein, spinal, prostate, gynaecological oncology and orthopaedic surgery [7][8][9][10][11][12][13][14][15][16][17][18][19]. Overall, these studies have indicated high negative predictive values (NPV, 95.0 to 98.2%) but low positive predictive values (PPV, 26.0 to 83.3%) and varying levels of concordance (56.4 to 97.2%) and agreement (11.0 to 100.0%). ...
Full-text available
Background Patient reported outcomes are increasingly used to assess the success of surgical procedures. Patient reported complications are often included as an outcome. However, these data must be validated to be accurate and useful in clinical practice. Methods This was a retrospective descriptive study of 364 patients who had completed their six-month follow-up review questionnaire in the Arthroplasty Clinical Outcomes Registry, National (ACORN), an Australian orthopaedic registry. Patient-reported complications following total hip arthroplasty (THA) and total knee arthroplasty (TKA) were compared to surgeon-reported complications recorded in their electronic medical records at their various follow-up appointments. Sensitivity, specificity, positive predictive value and negative predictive value were calculated. Agreement was assessed using percentage agreement and Cohen’s kappa. Results Patient-reported data from the ACORN registry returned overall low sensitivity (0.14), negative predictive value (0.13) and kappa values (0.11), but very high specificity (0.98), positive predictive value (0.98) and agreement values (96.3%) for reporting of complications when compared to surgeon-reported data. Values varied depending on the type and category of complication. Conclusion Patients are accurate in reporting the absence of complications, but not the presence. Sensitivity of patient-reported complications needs to be improved. Greater attention to the clarity of the questions asked may help in this respect. Electronic supplementary material The online version of this article (10.1186/s12874-018-0645-0) contains supplementary material, which is available to authorized users.
Background The Global Gynaecological Oncology Surgical Outcomes Collaborative (GO SOAR) aims to develop a network of gynecological oncology surgeons, surgical departments, and other interested parties that will have the long-term ability to collaborate on outcome studies. The protocol for the first collaborative study is presented here. Primary Objective To evaluate international variation in 30-day post-operative morbidity and mortality following gynecological oncology surgery between very high/high and medium/low human development index country settings. Hypothesis There is no variation in post-operative morbidity and mortality following gynecological oncology surgery between very high/high and medium/low human development index country settings. Study Design International, multicenter, prospective cohort study. Patient data will be collected over a consecutive 30-day period through gynecological oncology multidisciplinary teams/tumor boards and clinics across different human development index country groups. All data are collected on a customized, secure, password protected, central REDCap database. Major Inclusion/Exclusion Criteria Inclusion criteria include women aged ≥18 years undergoing elective/emergency, curative/palliative surgery for primary/recurrent tubo-ovarian/peritoneal, endometrial, cervical, vulval, vaginal, gestational trophoblastic malignancies. Surgical modality may be open, minimal access (laparoscopic/robotic), or vaginal. Primary Endpoint 30-day post-operative morbidity and mortality defined as per Clavien-Dindo classification system. Sample Size 1100 (550/arm). Estimated Dates for Completing Accrual and Presenting Results It is estimated recruitment will be completed by 2022 and results published by 2023. Trial Registration registry: NCT04579861 ( ).
Background: The accuracy of patient self-report of health care utilization and complications has yet to be determined. If patients are accurate and engaged self-reporters, collecting this information in a manner that is temporally proximate to the health care utilization events themselves may prove valuable to health care organizations undertaking quality improvement initiatives for which such data are often unavailable. Objective: The objective of this study was to measure the accuracy of patient self-report of health care utilization and complications in the 90 days following orthopedic procedures using an automated digital patient engagement platform. Methods: We conducted a multicenter real-world observational cohort study across 10 orthopedic practices in California and Nevada. A total of 371 Anthem members with claims data meeting inclusion criteria who had undergone orthopedic procedures between March 1, 2015, and July 1, 2016, at participating practices already routinely using an automated digital patient engagement platform for asynchronous remote guidance and telemonitoring were sent surveys through the platform (in addition to the other materials being provided to them through the platform) regarding 90-day postencounter health care utilization and complications. Their self-reports to structured survey questions of health care utilization and complications were compared to claims data as a reference. Results: The mean age of the 371 survey recipients was 56.5 (SD 15.7) years, 48.8% (181/371) of whom were female; 285 individuals who responded to 1 or more survey questions had a mean age of 56.9 (SD 15.4) years and a 49.5% (141/285) female distribution. There were no significant differences in demographics or event prevalence rates between responders and nonresponders. With an overall survey completion rate of 76.8% (285/371), patients were found to have accuracy of self-report characterized by a kappa of 0.80 and agreement of 0.99 and a kappa of 1.00 and agreement of 1.00 for 90-day hospital admissions and pulmonary embolism, respectively. Accuracy of self-report of 90-day emergency room/urgent care visits and of surgical site infection were characterized by a kappa of 0.45 and agreement of 0.96 and a kappa of 0.53 and agreement of 0.97, respectively. Accuracy for other complications such as deep vein thrombosis, hemorrhage, severe constipation, and fracture/dislocation was lower, influenced by low event prevalence rates within our sample. Conclusions: In this multicenter observational cohort study using an automated internet-based digital patient engagement platform, we found that patients were most accurate self-reporters of 90-day hospital admissions and pulmonary embolism, followed by 90-day surgical site infection and emergency room/urgent care visits. They were less accurate for deep vein thrombosis and least accurate for hemorrhage, severe constipation, and fracture/dislocation. A total of 76.8% (285/371) of patients completed surveys without the need for clinical staff to collect responses, suggesting the acceptability to patients of internet-based survey dissemination from and collection by clinical teams. While our methods enabled detection of events outside of index institutions, assessment of accuracy of self-report for presence and absence of events and nonresponse bias analysis, low event prevalence rates, particularly for several of the complications, limit the conclusions that may be drawn for some of the findings. Nevertheless, this investigation suggests the potential that engaging patients in self-report through such survey modalities may offer for the timely and accurate measurement of matters germane to health care organizations engaged in quality improvement efforts post discharge.
Full-text available
The objectives of this scoping review study were 1) to identify core domains and dimensions for inclusion in a person-focused and self-reported outcome measurement system for cancer and 2) to reach consensus among key stakeholders including cancer survivors on the relevance, acceptability, and feasibility of a core outcome set for collection in routine clinical care. Following a scoping review of the literature, a Rand Delphi consensus method was used to engage key interdisciplinary decision makers, clinicians, and cancer survivors in reaching consensus on a core patient-reported outcome domain taxonomy and outcome measures. Of the 21,900 citations identified in the scoping review, 1,503 citations were included in the full article review (380 conceptual articles, 461 psychometric evaluation articles, and 662 intervention studies) and subjected to data abstraction and mapping. Final consensus was reached on 20 domains, related subdimensions, and 45 self-report measures considered relevant and feasible for routine collection in cancer by the Delphi panel (PROMS-Cancer Core). Standardization of patient-reported outcome data collection is key to assessing the impact of cancer and treatment on the person for population comparison and monitoring the quality of clinical care. The PROMS-Cancer Core taxonomy of domains and outcome measures can be used to guide the development of a patient-reported outcome information system for cancer.
Full-text available
Introduction and hypothesis: The aim of this study was to evaluate the impact of urogynecological surgery on quality of life based on patient reported outcome measures (PROMs). Methods: Data were retrieved from the Danish Urogynaecological Database. Inclusion criteria were Danish women undergoing surgery for urinary incontinence (UI) or pelvic organ prolapse (POP) from 2006 to 2011. Using frequency of symptoms and a visual analogue scale (VAS) both pre- and postoperatively, their severity of symptoms and quality of life were measured by questionnaires. Results: During the study period, 20,629 urogynecological procedures were performed. The questionnaires on severity of symptoms and the VAS had been completed both pre- and postoperatively for approximately one third of women undergoing surgery. For UI surgery, 83% had improved symptoms, 13% were unchanged, and 4% had worse symptoms postoperatively. For POP surgery, 80, 17, and 3% were improved, unchanged, and worsened, respectively. The postoperative bother of symptoms and interference in everyday life evaluated by VAS were significantly reduced for both UI [preoperative median VAS score 9, postoperative median score 1 (p < 0.001)] and POP [8 preoperatively and 0 postoperatively (p < 0.001)]. Conclusions: Based on PROMs, surgery for UI and POP is effective in alleviating symptoms associated with UI or POP, and it can improve quality of life in symptomatic women. Pre- and postoperative questionnaires are useful tools in assessing symptomatic outcome measures after surgery.
Objective: Although quality assessment is gaining increasing attention, there is still no consensus on how to define and grade postoperative complications. This shortcoming hampers comparison of outcome data among different centers and therapies and over time. Patients and methods: A classification of complications published by one of the authors in 1992 was critically re-evaluated and modified to increase its accuracy and its acceptability in the surgical community. Modifications mainly focused on the manner of reporting life-threatening and permanently disabling complications. The new grading system still mostly relies on the therapy used to treat the complication. The classification was tested in a cohort of 6336 patients who underwent elective general surgery at our institution. The reproducibility and personal judgment of the classification were evaluated through an international survey with 2 questionnaires sent to 10 surgical centers worldwide. Results: The new ranking system significantly correlated with complexity of surgery (P < 0.0001) as well as with the length of the hospital stay (P < 0.0001). A total of 144 surgeons from 10 different centers around the world and at different levels of training returned the survey. Ninety percent of the case presentations were correctly graded. The classification was considered to be simple (92% of the respondents), reproducible (91%), logical (92%), useful (90%), and comprehensive (89%). The answers of both questionnaires were not dependent on the origin of the reply and the level of training of the surgeons. Conclusions: The new complication classification appears reliable and may represent a compelling tool for quality assessment in surgery in all parts of the world.
Since the seminal work of Jarman et al in 1999, standardised mortality ratios have been published for all English hospitals in the NHS. These have been widely digested by the media, clinicians, managers and the public alike, with differences in mortality rates taken to represent variation in the overall quality of care provided by institutions. The appetite for comparative data has continued and a wealth of performance data is now made publicly available, be it from the department of Health, the Care Quality Commission, professional bodies maintaining their own specialty registries or from third-party agencies such as dr Foster.
The purposes of this study are to evaluate the feasibility of capturing patient-reported outcomes (PROs) electronically and to identify the most common distressing symptoms in women recovering from major gynecologic cancer surgery. This was a prospective, single-arm pilot study. Eligible participants included those scheduled for a laparotomy for presumed or known gynecologic malignancy. Patients completed a Web-based "STAR" (Symptom Tracking and Reporting for Patients) questionnaire once preoperatively and weekly during the 6-week postoperative period. The questionnaire consisted of the patient adaptation of the NCI CTCAE 3.0 and EORTC QLQ-C30 3.0. When a patient submitted a response that was concerning, an automated email alert was sent to the clinician. The patient's assessment of STAR's usefulness was measured via an exit survey. Forty-nine patients completed the study. The procedures included the following: hysterectomy±staging (67%), resection of tumor (22%), salpingo-oophorectomy (6%), and other (4%). Most patients (82%) completed at least 4 sessions in STAR. The CTC generated 43 alerts. These alerts resulted in 25 telephone contacts with patients, 2 ER referrals, one new appointment, and one pharmaceutical prescription. The 3 most common patient-reported symptoms generating an alert were as follows: poor performance status (19%), nausea (18%), and fatigue (17%). Most patients found STAR useful (80%) and would recommend it to others (85%). Application of a Web-based, electronic STAR system is feasible in the postoperative period, highly accepted by patients, and warrants further study. Poor performance status, nausea, and fatigue were the most common distressing patient-reported symptoms.
Using general practitioner records and hospital notes and through direct telephone conversation with patients, we investigated the accuracy of nine patient-reported complications gathered from a self-completed questionnaire after elective joint replacement surgery of the hip and knee. A total of 402 post-discharge complications were reported after 8546 elective operations that were undertaken within a three-year period. These were reported by 136 men and 240 women with a mean age of 71.8 years (34 to 93). A total of 319 reported complications (79.4%; 95% confidence interval 75.4 to 83.3) were confirmed to be correct. High rates of correct reporting were demonstrated for infection (94.5%) and the need for further surgery (100%), whereas the rates of reporting deep-vein thrombosis (DVT), pulmonary embolism, myocardial infarction and stroke were lower (75% to 84.2%). Dislocation, peri-prosthetic fractures and nerve palsy had modest rates of correct reporting (36% to 57.1%). More patients who had knee surgery delivered incorrect reports of dislocation (p = 0.001) and DVT (p = 0.013). Despite these variations, it appears that post-operative complications may form part of a larger patient-reported outcome programme after elective joint replacement surgery.
Patient-reported complications are increasingly being used to compare the performance of surgical departments. The objectives of this study were to explore the validity of patient-reported complications and to determine their influence on patients' reports of the benefits of surgery (health gain). This was an observational study of patients in England who underwent hip replacement (68,391), knee replacement (77,705), inguinal hernia repair (48,199) or varicose vein surgery (16,535) during 2009-2011. Health gain was assessed with condition-specific measures of symptoms and disability (Oxford Hip Score, Oxford Knee Score, Aberdeen Varicose Vein Questionnaire), health-related quality of life (EQ-5D™ index) and a single item on the success of surgery. Adverse outcomes included four complications, readmission and further surgery. There was evidence that patient-reported complications were valid. Patients with three or more co-morbid conditions reported more complications, whereas age, sex and socioeconomic status (adjusted for co-morbidity) had little, or no association. Complications were strongly associated with readmission and further surgery. Among patients reporting a complication, the Oxford Hip Score or Oxford Knee Score was about 3 points (or 15 per cent) lower than the value in patients not reporting a complication. The EQ-5D™ score was about 0·07 lower for joint replacement, 0·06 lower for hernia repair and 0·04 lower for varicose vein surgery. Patients' reports of complications can be used for statistical comparisons of surgical departments. If the relationship between complications and health gain is causal, there is scope for improving health gain indicators after surgery by minimizing the risk of a complication.
This study evaluated concordance between self-reports and surgeon assessments of short-term complications. A total of 3976 primary total hip arthroplasty patients consented for an institutional registry (5/2007-12/2008); 3186 (80.1%) completed a 6-month survey; 137 (4.4%) reported deep venous thrombosis, pulmonary embolism, major bleeding, fracture, or dislocation. Patients reporting complications were called. Positive predictive values and 95% confidence intervals (95% CI) for patient self-report were measured, using surgeon assessment for comparison: pulmonary embolism, 88.9% (95% CI, 78.4%-99.4%); dislocation, 81.1% (95% CI, 75.9%-86.5%); fracture, 73.7% (95% CI, 63.8%-83.5%); deep venous thrombosis, 69.7% (95% CI, 61.9%-77.5%); major bleeding, 32.0% (95% CI, 19.4%-44.5%); any bleeding, 88.0% (95% CI, 75.3%-99.9%). Of 97 confirmed complications, 64.95% presented to outside institutions. Registry data on self-reported complications may overcome limitations of traditional methods, but data should be interpreted cautiously. Concordance was high for PE and dislocation but low for major bleeding.
Data from 3278 patients from a prospective patient-centered total joint registry was used to investigate the rates of complications in the first 6 months following primary total knee arthroplasty. All patients reporting deep venous thrombosis (DVT), pulmonary embolism (PE), or major bleeding were identified through patient self-reporting. Complications were verified by a review of imaging records and direct communication with patients by a surgeon. Surgeon-verified rates of DVT, PE, and major bleeding were 1.52%, 0.49%, and 0.34%, respectively. When compared with the complication rates as reported by patients, concordance was 86.2% for DVT, 84.2% for PE, and 36.7% for major bleeding. More than half (54.5%) of the complications were diagnosed and treated at the original operative hospital, and 45.5% were diagnosed and treated at other institutions.
Aims of the scts adult cardiac surgery database To measure the quality of care of adult cardiac surgery in GB and Ireland and provide information for quality improvement and research. Quality of care interventions Feedback of structured data to hospitals, publication of named hospital and surgeon mortality data, publication of benchmarked activity and risk adjusted clinical outcomes through intermittent comprehensive database reports, annual screening of all hospital and individual surgeon risk adjusted mortality rates by the professional society. Setting All NHS hospitals in England, Scotland and Wales with input from some private providers and hospitals in Ireland. Years 1994-ongoing. Population Consecutive patients, unconsented. Current number of records: 400000. Startpoints Adult cardiac surgery operations excluding cardiac transplantation and ventricular assist devices. Baseline data 129 fields covering demographic factors, pre-operative risk factors, operative details and post-operative in-hospital outcomes. Data capture Entry onto local software systems by direct key board entry or subsequent transcription from paper records, with subsequent electronic upload to the central cardiac audit database. Non-financial incentives at hospital level. Data quality Local validation processes exist in the hospitals. There is currently no external data validation process. Endpoints and linkages to other data All cause mortality is obtained through linkage with Office for National Statistics. No other linkages exist at present. Access to data Available for research and audit by application to the SCTS database committee at