Content uploaded by Lisa M. Wintner
Author content
All content in this area was uploaded by Lisa M. Wintner on Dec 20, 2022
Content may be subject to copyright.
Review – Renal Disease
A Systematic Review of Heterogeneity in Outcome Definition
and Reporting in Localised Renal Cancer
Katharina Beyer
a,
*
,1
, Christiaan Widdershoven
b,1
, Lisa M. Wintner
c
, Saeed Dabestani
d
,
Lorenzo Marconi
e
, Charlotte Moss
a
, Netty Kinsella
a,f
, Yuhong Yuan
g
, Rachel H. Giles
h
,
Ravi Barod
i
, Mieke Van Hemelrijck
a
, Axel Bex
c,i
, Patricia Zondervan
c,2
, Steven MacLennan
j,2
a
Translational Oncology and Urology Research, King’s College London, London, UK;
b
Department of Urology, Amsterdam University Medical Centers,
Amsterdam, The Netherlands;
c
University Hospital of Psychiatry II, Medical University of Innsbruck, Innsbruck, Austria;
d
Department of Translational
Medicine, Division of Urological Cancers, Lund University, Kristianstad Central Hospital, Lund, Sweden;
e
Department of Urology, Coimbra University Hospital,
Coimbra, Portugal;
f
Department of Urology, Royal Marsden Hospital, London, UK;
g
Department of Medicine, McMaster University, Hamilton, Canada;
h
International Kidney Cancer Coalition, Duivendrecht, The Netherlands;
i
Specialist Centre for Kidney Cancer, Royal Free Hospital, London, UK;
j
Academic
Urology Unit, Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, UK
Article info
Article history:
Accepted November 21, 2022
Associate Editor:
M. Carmen Mir
Keywords:
Renal cell carcinoma
Treatment choice
Patient decision-making
Oncology
Barriers
Facilitators
Treatment selection
Abstract
Context: Outcomes in renal cell carcinoma (RCC) are reported inconsistently, with
variability in definitions and measurement. Hence, it is difficult to compare inter-
vention effectiveness and synthesise outcomes for systematic reviews and to create
clinical practice guidelines. This uncertainty in the evidence makes it difficult to
guide patient-clinician decision-making. One solution is a core outcome set
(COS): an agreed minimum set of outcomes.
Objective:
To describe outcome reporting, definitions, and measurement hetero-
geneity as the first stage in co-creating a COS for localised renal cancer.
Evidence acquisition:
We systematically reviewed outcome reporting heterogeneity
in effectiveness trials and observational studies in localised RCC. In total, 2822
studies (randomised controlled trials, cohort studies, case-control studies, system-
atic reviews) up to June 2020 meeting our inclusion criteria were identified.
Abstracts and full texts were screened independently by two reviewers; in cases
of disagreement, a third reviewer arbitrated. Data extractions were double-
checked.
Evidence synthesis:
We included 149 studies and found that there was inconsistency
in which outcomes were reported across studies and variability in the definitions
used for outcomes that were conceptually the same. We structured our analysis
using the outcome classification taxonomy proposed by Dodd et al. Outcomes
linked to adverse events (eg, bleeding, outcomes linked to surgery) and renal injury
outcomes (reduced renal function) were reported most commonly. Outcomes
https://doi.org/10.1016/j.euros.2022.11.014
2666-1683/Ó2022 The Author(s). Published by Elsevier B.V. on behalf of European Association of Urology. This is an open access article
under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
1
These authors contributed equally and are joint first authors.
2
These authors contributed equally and are joint senior authors.
*Corresponding author. Translational Oncology and Urology Research, School of Cancer and
Pharmaceutical Studies, Guy’s Hospital, Great Maze Pond, London SE1 9RT, UK. Tel. +44 207 188
5594.
E-mail address: katharina.beyer@kcl.ac.uk (K. Beyer).
EUROPEAN UROLOGY OPEN SCIENCE 48 (2023) 1–11
available at www.sciencedirect.com
journal homepage: www.eu-openscience.europeanurology.com
related to deaths from any cause and from cancer were reported in 44% and 25% of
studies, respectively, although the time point for measurement and the analysis
methods were inconsistent. Outcomes linked to life impact (eg, global quality of
life) were reported least often. Clinician-reported outcomes are more frequently
reported than patient-reported outcomes in the renal cancer literature.
Conclusions:
This systematic review underscores the heterogeneity of outcome
reporting, definitions, and measurement in research on localised renal cancer. It
catalogues the variety of outcomes and serves as a first step towards the develop-
ment of a COS for localised renal cancer.
Patient summary:
We reviewed studies on localised kidney cancer and found that
multiple terms and definitions have been used to describe outcomes. These are
not defined consistently, and often not defined at all. Our review is the first phase
in developing a core outcome set to allow better comparisons of studies to improve
medical care.
Ó2022 The Author(s). Published by Elsevier B.V. on behalf of European Association of
Urology. This is an open access article under the CC BY-NC-ND license (http://creative-
commons.org/licenses/by-nc-nd/4.0/).
1. Introduction
Renal cell carcinoma (RCC) represents 2.2% of all new can-
cers worldwide [1,2]. With the increase in reporting of inci-
dental findings, a greater proportion of patients newly
diagnosed with renal cancer currently present with stage I
disease [3,4]. Historically, surgery has been the standard
of care for localised renal cancer, but international guideli-
nes have more recently proposed ablative treatments and
active surveillance as alternative options [5,6]. Currently,
oncological outcomes across treatments are similar and
treatment decision-making is multifactorial [7].
Across many clinical areas including urology, patient-
reported outcomes and clinical outcomes are reported
inconsistently, with variability in definitions and measure-
ment, for instance in the settings of localised prostate can-
cer and bladder cancer [8–10]. This makes it very difficult
to compare and synthesise outcomes to improve guidelines
to better direct and support patients and clinicians during
treatment decision-making and ultimately improve results
in clinical practice [11,12]. A core outcome set (COS) is a
standardised set of prioritised outcomes and is proposed
by current research as a solution to decrease heterogeneity
in collection, reporting, and analysis of outcomes. COS in
urology are needed because inconsistencies and variability
cause not only frustration but also potentially problematic
conclusions [9]. This issue is also clearly apparent for loca-
lised renal cancer, and ultimately results in barriers for
the multifactorial process of decision-making [7].
The aim of this systematic review was to identify which
outcomes are reported in intervention effectiveness
research in localised kidney cancer and to assess hetero-
geneity in outcome definitions and measurements. It con-
stitutes the initial stage in the development of a COS for
localised renal cancer with the intention of identifying a
minimum set of outcomes that are potentially important
to health care professionals and patients. The outcomes
identified in this systematic review are organised under
the taxonomy developed by Dodd et al. [13], which helps
to structure general health research vocabularies to reduce
inconsistencies. It is embedded in a larger project registered
in the Core Outcome Measures in Effectiveness Trials
(COMET) database [14], and uses the same robust method-
ology that was already followed for the prostate cancer COS
[15] developed in collaboration with the European Associa-
tion of Urology.
2. Evidence acquisition
This systematic review followed the guidelines of the
COMET initiative, an international expert body that estab-
lished guidelines on how to develop methodologically
robust COS. We report our study in accordance with the Pre-
ferred Reporting Items for Systematic Reviews and Meta-
Analyses (PRISMA) statement and the COS-STAR reporting
guidelines, which are relevant to this stage of COS develop-
ment [16]. A project steering committee (S.M., M.V.H., P.Z.,
A.B., L.M., S.D., R.B., N.K.) supported the development from
a methodological and clinical perspective. The study proto-
col was registered on PROSPERO (ID: CRD42020198605).
2.1. Aims and objectives
The aim of this project was to systematically review which
outcomes have been reported in effectiveness trials and
observational studies in localised renal cancer, and how
they were defined and measured.
2.2. Identification of relevant studies
We searched Medline, EMBASE, Cochrane CENTRAL, and
Cochrane Database of Systematic Reviews (via Ovid) from
inception to June 2020. We worked with an information sci-
entist to design the search strategy (Supplementary Fig. 1).
To balance the feasibility and precision of the search, we
used a two-step approach. First, we identified all published
systematic reviews and intervention trials related to RCC
without limiting the search to localised renal cancer, and
we screened the reference lists in all the articles as a prag-
matic way to identify primary studies potentially meeting
our inclusion criteria. Second, we searched for and screened
EUROPEAN UROLOGY OPEN SCIENCE 48 (2023) 1–11
2
all interventional studies on localised RCC from 2015
onwards without limiting the study design. We included
randomised controlled trials (RCTs), cohort studies, and
case-control studies that reported on eligible interventions
for localised renal cancer. We excluded case studies owing
to their low level of evidence according to the Oxford Centre
for Evidence-Based Medicine (level of evidence 4 or lower
[14]) and the unlikelihood of changing clinical practice.
We also excluded conference abstracts.
2.2.1. Study participants
Adults (male and female) with suspected localised renal
cancer (N0M0 according to the TNM classification; all ver-
sions of the TNM staging system) on magnetic resonance
imaging, computed tomography, or ultrasound imaging
were included.
Those undergoing treatment for renal metastasis or
other tumours were excluded.
2.2.2. Intervention and comparator
Studies reporting on any intervention for localised renal
cancer were retained, including but not limited to active
surveillance, radical nephrectomy (all modes and
approaches), partial nephrectomy (all modes and
approaches), cryoablation, radiofrequency ablation, micro-
wave ablation, irreversible electroporation, watchful wait-
ing, high-intensity focused ultrasound, or radiotherapy.
2.2.3. Eligibility of studies
All abstracts and full texts were screened independently by
at least two reviewers (C.W., K.B.). Any disagreements were
arbitrated by a third review author (S.M).
2.3. Data extraction
Data were independently extracted from the studies
included by two researchers (C.W., K.B.) and checked for
accuracy by another reviewer (S.M.). We extracted data on
study design; author details; year and journal of
publication; intervention(s) under investigation; each effec-
tiveness outcome reported; whether the outcome was
defined or not; the definition used; the indicators and/or
tool(s) used to operationalise or measure the outcome;
the time point or period for outcome measurement; and
how the outcome was reported.
2.4. Data analysis and synthesis
The outcome names extracted were coded and categorised
according to the outcome reporting taxonomy developed
by Dodd et al. [13], which has been suggested by COMET
for classification of outcomes and group domains.
2.5. Assessment of risk of bias
A risk-of-bias assessment was not conducted, as no estima-
tion of the effect size of treatments was conducted and only
qualitative information containing terminology was
extracted.
3. Evidence synthesis
3.1. Characteristics of the studies included
Our initial search returned 2785 abstracts. Of these, we
assessed 319 full-text articles, of which 149 were included
(Fig. 1). Of the 149 studies included, 97% were observational
studies and five (3%) were RCTs.
3.2. Heterogeneity in outcome reporting, detection, and
definitions
A suitable outcome taxonomy for health research must dif-
ferentiate between high-level outcome domain classifica-
tions, and comprehensively classify all outcomes, while
also proposing a standardised terminology. Therefore, we
reported and organised the outcomes in the studies under
the taxonomy developed by Dodd et al. [13] and recom-
mended by the COMET initiative. Taxonomies help to
Reports screened
(n= 2822)
Reports excluded
(n= 2466)
Reports assessed for eligibility
(n = 319) Reports excluded:
Not accessible
Study type
Outside of time frame
Type of population
Other reasons
Studies included
(n= 149)
Identification
Screening
Included
Fig. 1 – Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) diagram.
EUROPEAN UROLOGY OPEN SCIENCE 48 (2023) 1–11
3
structure vocabularies for general health research to reduce
inconsistencies and ambiguities in how current studies
describe and define outcomes. The Dodd taxonomy has
been proposed to increase the reuse value of outcome data.
The taxonomy comprises 38 outcome domains within five
core areas: death, adverse events, life impact, physiologi-
cal/clinical, and resource use [13]. The core outcomes are
further subclassified as shown in Figure 2.
Table 1 lists the outcomes reported by the studies by
domain and highlights the heterogeneity of outcomes iden-
tified. We have merged the synonyms and redundant terms.
The next section explains the heterogeneity of the terminol-
ogy in more detail. Table 2 shows which outcomes were
reported in the studies included in the review.
3.3. Death (mortality/survival)
Death was reported 103 times. We categorised these out-
comes according to the Dodd mortality/survival classifica-
tion into ‘‘overall survival’’ and ‘‘cancer-specific survival’’.
Overall survival (OS) was measured in 65 studies (44%;
Table 2) as OS, death, or mortality; more details are pro-
vided in Table 1. Cancer-specific survival (CSS) was reported
in 43 (29%) of the studies (Table 2) as CSS, death from renal
cancer, or cancer-specific mortality.
Definitions of OS and CSS differed across studies
(Table 1). The heterogeneity for the definitions was linked
to time points. For instance, some studies started measure-
ment at diagnosis, whereas others used the treatment date
as the starting point for survival. The time endpoint also dif-
fered, with studies reporting either a rate at a defined time
(eg, at 10 yr) or a hazard ratio based on survival analyses.
3.4. Adverse events (adverse events/effects)
Adverse events (AEs) were the most common outcome
reported (n= 101, 68%). However, many different types of
AE were reported, sometimes as the number or percentage
of patients experiencing the outcome and sometimes sub-
sumed in a classification system linked to severity or conse-
quences (eg, the Clavien-Dindo scheme). Examples of
events that play a role in AE assessment include bleeding,
operation time, warm ischaemia time, intra-abdominal
pressure, surgical time, drainage time, serum creatinine,
blood loss, trifecta/pentafecta outcomes, and dialysis, which
are linked to the complexity of the surgery. Many articles
reported several AEs within one study, but the AEs reported
varied across studies (eg, surgical complications were mea-
sured as intraoperative complications, conversion to
nephrectomy, or short-term complications).
‘‘Adverse events/effects or resource use: hospital’’ out-
comes were reported in six studies (4%). Outcome reporting
and measurement were inconsistent; examples include
dialysis-free probability, number requiring dialysis, tempo-
rary dialysis, and permanent dialysis (Table 1).
3.5. Life impact/functioning (perceived health status; global
quality of life)
Only eight studies (5%) reported outcomes reflecting life
impact. Five studies (3%) reported outcomes classified as
perceived health status. Three studies (2%) reported on glo-
bal quality of life (QoL), one study using the Short Form
(SF)-36 and another using the Functional Assessment of
Cancer Therapy-Kidney Symptom Index (FKSI)-15 patient-
reported outcome measures (PROMs; Table 2).
3.6. Physiological/clinical (physiological or clinical)
Physiological or clinical outcomes were subclassified as ‘‘re-
nal and injury outcomes’’ (eg, new chronic kidney disease
[CKD], CKD stage, time to CKD), which were defined very
heterogeneously and reported in 87 (58%) of the studies;
and ‘‘outcomes relating to neoplasms’’ (linked to cancer
follow-up and progression, reported in 55 studies [37%])
according to the Dodd taxonomy [13] (Tables 1 and 2).
3.7. Resource use (economic, need for further intervention,
hospital)
Resource use consisted of the subcategories ‘‘economic
resource’’ (eg, health care expenditure; reported in four
Fig. 2 – The Dodd [13] taxonomy applied for classification.
EUROPEAN UROLOGY OPEN SCIENCE 48 (2023) 1–11
4
Table 1 – Outcomes classified according to the taxonomy of Dodd et al. [13]
Death Adverse events/effects Physiological or clinical Resource use Life impact
Mortality/survival 38. Adverse events/effects 2–24. Physiological/clinical Resource use Functioning
Mortality/survival
Overall survival
OS rate
Cumulative survival
Stage-related OS
Mean OS
Survival probability 1 yr
Deaths
Deaths
Death from any cause
Mortality
Other-cause mortality
Mortality events
Overall mortality
Total mortality
X-day mortality
Cancer-specific survival
RCC-specific survival
Recurrence-free survival
Death from kidney cancer
Number of patients deceased at
last follow-up
Death from kidney cancer
RCC death
Death from RCC
Death due to cancer
Cancer-specific mortality
Cancer-specific mortality
Death from nonRCC, other-cause
mortality
38. Adverse events
Complications
Surgical complications
Intraoperative complications
Conversion to nephrectomy
Short-term complications
Conversions
Grade I and grade II complications
Highest complication grade
Overall complications
30-d postoperative complications
Bleeding
Bleeding severity
Units of blood transfused during
hospitalisation
Estimated bleeding
Bleeding-related complications
Haemoglobin postoperatively
Perioperative
Surgical margins
Surgical margins
Negative margins
Positive surgical margins
Outcomes linked to surgery
Operation time
WIT
Surgical time
Drainage time
Procedure time
Pneumoperitoneum time
Suture time
WIT ≤25 min
Conversions
Open conversion
Average clamping time
Haemoglobin after surgery
Postoperative drainage time
Intra-abdominal pressure
Adverse health outcomes
Clampless rate
Blood loss
Mean estimated blood loss
Estimated blood loss
Changes in estimated blood loss
Units of blood transfused during
surgery
Transfusion requirement
Transfusion rate
Intraoperative transfusion
Intraoperative ES transfusion
Transfusions received
Perioperative allogenic blood
transfusion
19. Renal and injury outcomes
New CKD
CKD probability
CKD stage
CKD stage
Upgrading to CKD grade III–V
CKD upstaging
No CKD upstaging
Postoperative CKD stage
Postoperative new onset of stage III or IV CKD
Final CKD stage
Patients with acquired stage III–V CKD at follow-up, compared to preop-
erative
Time to CKD
Decline in CKD stage
Progression to CKD
CKD upstaged–free survival
De novo CKD stage III
Survival without CKD upstaging
Time to diagnosis of CKD
Outcomes linked to procedure
Mean ablation time
Laser excision time
Median procedure time
Renal outcomes
Urinary function
Oncological outcomes
Collecting system entry
Haemostatic agent
eGFR
Mean eGFR change
Median eGFR preservation
Median percentage eGFR change
Change in GFR
eGFR preservation (%)
Latest eGFR preservation
D
GFR change
Last eGFR
eGFR 1-yr post operation
Percentage change in eGFR
eGFR decrease
Postoperative eGFR change (%) from baseline to 1-yr follow-up
Serum creatinine
Preoperative creatinine
Creatinine level
Serum creatinine
Difference in serum creatinine between preoperative and postoperative
levels
Postoperative creatinine level
Postoperative creatinine
Latest creatinine level
Percentage change in creatinine
Variation of creatinine
34. Economic
Cost
Health care expenditure
Medical cost
Total cost
Imaging (linked to costs)
Medications?
36. Need for further inter-
vention
Readmission
35. Hospital
Length of stay
Postoperative HSP time
HSP time
Hospital stay
Average hospital stay
Duration of HSP
Duration of postoperative
hospital stay
Median hospital stay
Surgical supplies and
devices
Operating room
31. Perceived health
status
Perceived health
Pain
Adverse health out-
comes
30. Global quality
of life
Health-related
quality of life
(continued on next page)
EUROPEAN UROLOGY OPEN SCIENCE 48 (2023) 1–11
5
Table 1 (continued)
Death Adverse events/effects Physiological or clinical Resource use Life impact
Percentage blood transfusion
BUN after 1 d and 1 mo
Trifecta/pentafecta
Trifecta
Trifecta rate
Pentafecta reached
Highest complication grade
Low-grade complication
D
creatinine
Recurrence-free survival
RFS + time
38. Adverse events/effects or 35.
Resource use: hospital
Dialysis free probability
No. requiring dialysis
Temporary dialysis
Permanent dialysis
16. Outcomes relating to neoplasms: benign, malignant and unspecified
(including cysts and polyps)
Metastasis
Distant metastasis–free survival
Distant metastasis
Extrarenal metastasis
Follow-up
Follow-up
Long-term outcomes
Median postoperative follow-up time
Average length of follow-up
Median follow-up time
Progression-free survival
PFS
Systemic PFS
Clinical PFS
Progression
Local tumour progression
Disease progression
Recurrence
Local recurrence
Disease-free survival
Recurrence rate
Recurrence-free survival
Recurrence result
Recurrence linked to time
Time to local recurrence
Events of local recurrence
Delayed recurrence
Time to recurrence
Local recurrence rate
Local recurrence–free survival
Recurrence (local or metastatic)
Local ipsilateral recurrence
Disease-free survival
BUN = blood urea nitrogen; CKD = chronic kidney disease; eGFR = estimated glomerular filtration rate; ES = erythrocyte suspension; HSP = hospitalisation; OS = overall survival; PFS = progression-free survival; RCC = renal cell
carcinoma; WIT = warm ischaemia time.
EUROPEAN UROLOGY OPEN SCIENCE 48 (2023) 1–11
6
Table 2 – Outcomes reported in each study after classification according to the taxonomy suggested by Dodd et al. [13]
First author Design Location Death Adverse
events
Physiological
or clinical
Resource use Life impact
OS CSS AEs AEs/ E H NFI PHI GQL
ERUH RIO ORN
Patel OBS US X X X
Li OBS China X X X
Wang OBS China X X X X
Morkos OBS USA X X X X X
Alshyarba OBS Saudi Arabia X X
Wu OBS China X X X
Packiam OBS USA X X X
Yang OBS Not stated X X X
Rembeyo OBS France X X X X X
Uhlig OBS USA X
Yu OBS China X X X X X
Kartal OBS Turkey X X X X
Jalbani OBS Pakistan X X X
Seon OBS South Korea X X X X X
Choi OBS South Korea X X X
Tan OBS USA, Puerto Rico X
Grant OBS USA X
Chen RCT China X X X
Liu OBS China X X X
Sandbergen OBS Netherlands X X
Shapiro OBS USA X X X X
De Cobelli OBS Italy X X
Nayan OBS Canada X X X X
Jin OBS China X X X
Mourao OBS USA, Spain X X X X X
Anglickis OBS Lithuania X X X X X
Marchioni OBS USA X X
Li OBS China X
Liao OBS USA X X
Simone OBS Italy X X X X X
Shao OBS Taiwan X X X
Antonelli RCT Italy X X X
Kitley OBS USA X
Zhou OBS China X X
Andrews OBS USA X X
Zhou OBS USA X X X
Fraisse OBS France X X X X
Hu OBS China X
Abu-Ghanem OBS Israel X X X X
Kavaric OBS Montenegro X X X
Ziegelmueller OBS Germany X X X
Talenfeld OBS USA X X X
Bhindi OBS USA X X X X X
Larcher OBS Netherlands, Italy X X X X X
Xing OBS USA X X X
Ristau OBS USA, Puerto Rico X
Zhao OBS China X X X X
Gershman OBS USA X X X X
Benoit OBS France X X X X
Paulucci OBS USA X X X X
Abdel Raheem OBS South Korea X X X X X
Lourenco OBS Canada X X X X X
Hasegawa OBS Japan X X X
Streja OBS USA X X
Borghesi OBS Globally X X X
Uhlig OBS USA X X X
Ye OBS China X X X
Park OBS Not stated
Venkatramani OBS USA X X
Uhlig OBS USA X
Zhang OBS USA X X
Lee OBS South Korea X X X X
Chong OBS USA X X X
Chang OBS Taiwan X X X
Yang OBS China X X X
Veys OBS Belgium X X X X X
Banapour OBS USA X X X
Cai OBS China X X X
Lanchon OBS France X X X
Venkatramani OBS USA X X
Karalli RCT Sweden X
(continued on next page)
EUROPEAN UROLOGY OPEN SCIENCE 48 (2023) 1–11
7
Table 2 (continued)
First author Design Location Death Adverse
events
Physiological
or clinical
Resource use Life impact
OS CSS AEs AEs/ E H NFI PHI GQL
ERUH RIO ORN
Dong OBS Not stated X
Wang OBS USA X
Tang OBS USA X X
Yin OBS China X X
Shah OBS USA X
Annino OBS Italy X X
Wang OBS China X X
Shum OBS USA X X X X
Luo OBS USA X X
Lee OBS South Korea X X X X
Caputo OBS USA X X X X
Lu OBS China X X X
Maric OBS Serbia X
Matei OBS Italy X X X X X
Paulucci OBS USA X X
Rassweiler OBS Germany X
Larcher OBS USA X X X X X X
Lenis OBS USA X X
Wang OBS USA X X
Peng OBS China X X X X X
Malkoc OBS USA X X X X
Long OBS France X X X X X X X
Yoo OBS South Korea X
Redondo OBS Spain X X
Carrion OBS Spain X X
Shah OBS USA X X
Moskowitz OBS USA X
Huang RCT China X X X
Larcher OBS USA X
Jang OBS South Korea X X X X X
Forbes OBS Canada X X X X
Kara OBS Not stated X X X X
Takagi OBS Not stated X X X X
Oh OBS Not stated X
Andrade OBS Not stated X X X X
Dong OBS Not stated X X X X X
Trudeau OBS USA X X
Lai OBS China X X X X
Liu OBS China X X X X
Pantelidou OBS UK X X X X
Liu OBS China X X X X
Larcher OBS USA X X
Hossein OBS Iran X X
Komatsuda OBS Japan X X
Janicic OBS Serbia X X
Lyon OBS USA X X
Satkunasivam OBS USA X X X X X
Thompson OBS USA X X X
Tabayoyong OBS USA X
Alanee OBS USA X
Zargar OBS USA X X X
Mano OBS USA X X X X X
Chang OBS China X X X X X X
Serni OBS Italy X X X X X
Chung OBS Korea X X X
Yu OBS Not stated X X
Weinberg OBS USA X X X
Park RCT South Korea X X X X
Balasar OBS Turkey X X
O’Malley OBS USA X X X
Kim OBS South Korea X X
Chang OBS China X X
Cooper OBS USA X X
Alam OBS USA X X X
Çömez OBS Turkey X X X
Kopp OBS USA X X X X
Danzig OBS USA X X
Hussein OBS Egypt X
Simsek OBS Turkey X X X
Fossati OBS Italy X X X
Ji OBS Italy, China X X X X X X X
EUROPEAN UROLOGY OPEN SCIENCE 48 (2023) 1–11
8
studies (3%) as mean or median costs), ‘‘need for further
intervention’’ (eg, readmission; reported in eight studies
[5%] as a binary yes/no result or median value), and ‘‘hospi-
tal’’ (eg, length of hospital stay, reported in 58 studies [39%]
as mean or median length of hospital stay in days).
3.8. Discussion
To the best of our knowledge, this is the first systematic
review of outcome reporting heterogeneity in the literature
on localised renal cancer. Our results build a framework for
developing a COS for localised renal cancer with the aim of
reducing heterogeneity for outcome definitions, measure-
ment, and reporting.
Our systematic review highlights the persisting problem
of outcome reporting heterogeneity in studies on localised
renal cancer. Multiple terms are used to refer to conceptu-
ally similar outcomes, and there is variation in the outcome
definitions used. This has not improved over time and is
problematic when summarising the evidence base for treat-
ment effectiveness to inform decision-making, because it is
not advisable to synthesise data with different outcome def-
initions within a meta-analysis. Such a practice can produce
meaningless summary statistics that may be given more
credibility than they are due. Therefore, a cumbersome
and often less-informative narrative synthesis must be
undertaken instead. Furthermore, our work highlights vari-
ety in data reporting and measurement. For instance, if
dichotomous outcomes such as OS and CSS are reported
using different methods (eg, some studies report adjusted
and some unadjusted hazard ratios, others report a rate at
median follow-up or at specified time points such as 1 yr
or 5 yr), then these data cannot be easily or reliably synthe-
sised in a meta-analysis. When these problems all occur
together, then it is difficult to interpret the body of evidence
and clinical practice guideline panels encounter challenges
in drawing up recommendations and applying certainty-
of-evidence attributes such as those proposed by the GRADE
working group [17].
Worryingly, we identified very few patient-reported out-
comes (PROs), which might be related to the limited num-
ber of specific tools available for capturing QoL for renal
cancer. In their systematic review, Rossi et al. [18] identified
three generic PROMs (RAND medical outcome survey SF-36
and SF-12, EuroQol [EQ-5D], Convalescence and Recovery
Evaluation [CARE]) and eight cancer-specific PROMs (Cancer
Rehabilitation Evaluation System-Short Form [CARES-SF],
European Organisation for Research and Treatment of Can-
cer [EORTC] Quality of Life Questionnaire [QLQ]-C30, Func-
tional Assessment of Cancer Therapy-General [FACT-G],
FKSI, Renal Cell Carcinoma-Symptom Index [RCC-SI], Instru-
ments to assess psychological wellbeing Impact of Events
Scale [IES], Hospital Anxiety and Depression Scale [HADS],
Mishel Uncertainty in Illness Scale (MUIS)) which are cur-
rently being used in renal cancer. However, of the PROM
instruments used, only two are specific to renal cancer
and are not stage-specific (FKSI, RCC-SI) [18].
In their study of the symptom index most commonly
used for renal cancer, Rosenblad et al. [19] assessed the psy-
chometric properties of the FKSI-19 (which captures physi-
cal and emotional disease-related symptoms, function/
wellbeing, and treatment side effects) among patients with
RCC and reported that it is barely fit for this purpose. Decat
Bergerot et al. [20] conducted a patient survey that identi-
fied many of the FKSI-19 questions as irrelevant from a
patient perspective and stressed the need to incorporate
patients in the development of PRO tools to determine areas
of importance to them. The EORTC Quality of Life group is
currently developing an RCC-specific module to be used in
combination with their QLQ-C30 instrument.
Our project steering group includes clinical RCC experts,
patient advocacy groups, methodologists, and guideline
developers from the European Association of Urology (most
are co-authors of this study). We aim to use these networks
to improve recruitment to our research stages, and to sub-
sequently endorse and disseminate the final COS as part
of our implementation strategy.
We curated the different terms used and collated them
using a standardised outcome classification taxonomy [13]
as a first step in creating a COS for localised renal cancer.
In the next step we will use consensus processes in a mul-
tistakeholder group to prioritise which outcomes are core
and to recommend definitions for each outcome. Once we
know which outcomes are considered core by our stake-
holders, we will systematically review the psychometric
properties of PROMs available, with coverage of core out-
comes using the COSMIN criteria [21,22] and will recom-
mend one to be used in future research on treatment
effectiveness. This is a medium- to long-term vision to stan-
dardise the definition, measurement, and reporting of
Table 2 (continued)
First author Design Location Death Adverse
events
Physiological
or clinical
Resource use Life impact
OS CSS AEs AEs/ E H NFI PHI GQL
ERUH RIO ORN
Mason OBS USA X
Chehab OBS Not stated X
An OBS USA X X
Rosen OBS USA X X
Ramirez OBS USA X X
Malkoc OBS Turkey X X X
AEs = adverse events; CSS = cancer-specific survival; E = economic resource use; ERUH = effects or resource use: hospital; GQL = global quality of life;
H = hospital resource use; NFI = need for further intervention; OBS = observational study; ORN = outcomes relating to neoplasms; OS = overall survival;
PHI = perceived health impact; RCT = randomised controlled trial; RIO = renal and injury outcomes.
EUROPEAN UROLOGY OPEN SCIENCE 48 (2023) 1–11
9
outcomes in research on localised renal cancer, with the
ultimate aim of improving the decision-making process at
all levels.
3.9. Limitations
We may have missed studies reporting PROs and/or QoL
because we did not search specifically for primary qualita-
tive studies of patient experiences of renal cancer treat-
ment. However, we will supplement the list of outcomes
presented here with outcomes identified in our own pri-
mary interview study with patients who have been treated
for renal cancer, and further review work. Furthermore, as
part of our prioritisation process, participants will be able
to propose outcomes they think are missing from our list.
4. Conclusions
Our review indicates that clinical research on localised renal
cancer is impeded by heterogeneity in outcome selection,
definitions, and reporting. This work represents the first
step in the development of a COS that will ultimately
improve the evidence basis for treatment of patients with
localised renal cancer and the process for creating clinical
practice guidelines, and will facilitate treatment decision-
making by health care professionals and patients.
Author contributions: Katharina Beyer had full access to all the data in
the study and takes responsibility for the integrity of the data and the
accuracy of the data analysis.
Study concept and design: Beyer, Widdershoven, Van Hemelrijck, Bex,
Zondervan, MacLennan.
Acquisition of data: Beyer, Widdershoven, MacLennan.
Analysis and interpretation of data: Beyer, Zondervan, MacLennan.
Drafting of the manuscript: Beyer, Zondervan, MacLennan.
Critical revision of the manuscript for important intellectual content: Beyer,
Widdershoven, Wintner, Dabestani, Marconi, Moss, Kinsella, Yuan, Giles,
Barod, Van Hemelrijck, Bex, Zondervan, MacLennan.
Statistical analysis: None.
Obtaining funding: None.
Administrative, technical, or material support: Beyer, Widdershoven, Yuan.
Supervision: Zondervan, MacLennan.
Other: None.
Financial disclosures: Katharina Beyer certifies that all conflicts of inter-
est, including specific financial interests and relationships and affiliations
relevant to the subject matter or materials discussed in the manuscript
(eg, employment/affiliation, grants or funding, consultancies, honoraria,
stock ownership or options, expert testimony, royalties, or patents filed,
received, or pending), are the following: None.
Funding/Support and role of the sponsor:None.
Appendix A. Supplementary data
Supplementary data to this article can be found online at
https://doi.org/10.1016/j.euros.2022.11.014.
References
[1] Ferlay J, Steliarova-Foucher E, Lortet-Tieulent J, et al. Cancer
incidence and mortality patterns in Europe: estimates for 40
countries in 2012. Eur J Cancer 2013;49:1374–403. https://doi.
org/10.1016/j.ejca.2012.12.027.
[2] Sung H, Ferlay J, Siegel RL, et al. Global cancer statistics 2020:
GLOBOCAN estimates of incidence and mortality worldwide for 36
cancers in 185 countries. CA Cancer J Clin 2021;71:209–49. https://
doi.org/10.3322/caac.21660.
[3] Sun M, Abdollah F, Bianchi M, et al. A stage-for-stage and grade-for-
grade analysis of cancer-specific mortality rates in renal cell
carcinoma according to age: a competing-risks regression
analysis. Eur Urol 2011;60:1152–9. https://doi.org/10.1016/j.
eururo.2011.07.064.
[4] Rosiello G, Larcher A, Montorsi F, et al. Renal cancer: overdiagnosis
and overtreatment. World J Urol 2021;39:2821–3. https://doi.org/
10.1007/s00345-021-03798-z.
[5] Martini A, Larcher A, Bravi CA, et al. How to select the optimal
candidates for renal mass biopsy. Eur Urol Oncol 2021;4:506–9.
https://doi.org/10.1016/j.euo.2020.10.001.
[6] Ljungberg B, Albiges L, Abu-Ghanem Y, et al. European Association
of Urology guidelines on renal cell carcinoma: the 2022 update. Eur
Urol 2022;82:399–410. https://doi.org/10.1016/j.eururo.2022.
03.006.
[7] Beyer K, Barod R, Fox L, et al. The current evidence for factors that
influence treatment decision making in localized kidney cancer: a
mixed methods systematic review. J Urol 2021;206:827–39.
https://doi.org/10.1097/ju.0000000000001901.
[8] Williamson PR, Altman DG, Bagley H, et al. The COMET handbook:
version 1.0. Trials 2017;18(Suppl 3):280. https://doi.org/10.1186/
s13063-017-1978-4.
[9] MacLennan S, Williamson PR. The need for core outcome sets in
urological cancer research. Transl Androl Urol 2020;10:2832–3285.
[10] Meregaglia M, Ciani O, Banks H, et al. A scoping review of core
outcome sets and their ‘mapping’ onto real-world data using
prostate cancer as a case study. BMC Med Res Methodol
2020;20:41. https://doi.org/10.1186/s12874-020-00928-w.
[11] Violette PD, Jewett MAS, Richard PO. Evidence-based urology:
trustworthy guidelines. Eur Urol Focus 2021;7:1243–2126. https://
doi.org/10.1016/j.euf.2021.10.012.
[12] Kirkham JJ, Dwan KM, Altman DG, et al. The impact of outcome
reporting bias in randomised controlled trials on a cohort of
systematic reviews. BMJ 2010;340:c365.
[13] Dodd S, Clarke M, Becker L, et al. A taxonomy has been developed
for outcomes in medical research to help improve knowledge
discovery. J Clin Epidemiol 2018;96:84–92. https://doi.org/
10.1016/j.jclinepi.2017.12.020.
[14] MacLennan S, Marconi L, Van Hemelrijck M, Beyer K, Bex A,
Zondervan P. Renal cancer Core Outcome Sets (R-COS) programme
encompassing 1) localised renal cell carcinoma (RCC) 2) locally
advanced RCC and 3) metastatic RCC. COMET Initiative 2019.
https://www.comet-initiative.org/studies/details/1406.
[15] Beyer K, Moris L, Lardas M, et al. Updating and integrating core
outcome sets for localised, locally advanced, metastatic, and
nonmetastatic castration-resistant prostate cancer: an update
from the PIONEER Consortium. Eur Urol 2022;81:503–14. https://
doi.org/10.1016/j.eururo.2022.01.042.
[16] Kirkham JJ, Gorst S, Altman DG, et al. Core Outcome Set-Standards
for Reporting: the COS-STAR statement. PLoS Med 2016;13:
e1002148.
[17] Schünemann H, Bro _
zek J, Guyatt G, Oxman A. GRADE handbook.
Updated October 2013. https://gdt.gradepro.org/app/handbook/
handbook.html.
[18] Rossi SH, Klatte T, Stewart GD. Quality of life outcomes in patients
with localised renal cancer: a literature review. World J Urol
2018;36:1961–72. https://doi.org/10.1007/s00345-018-2415-3.
[19] Rosenblad AK, Sundqvist P, Westman B, et al. A psychometric
evaluation of the Functional Assessment of Cancer Therapy-Kidney
Symptom Index (FKSI-19) among renal cell carcinoma patients
suggesting an alternative two-factor structure. Qual Life Res
2021;30:2663–70. https://doi.org/10.1007/s11136-021-02839-9.
[20] Decat Bergerot C, Psutka S, Plimack E, et al. Relevance of items in
the FACT Kidney Cancer Symptom Inventory-19 (FKSI-19): results
of a patient survey. Presented at the 2022 European International
Kidney Cancer Symposium (EIKCS), Antwerp, April 22–24, 2022.
EUROPEAN UROLOGY OPEN SCIENCE 48 (2023) 1–11
10
[21] Mokkink LB, de Vet HCW, Prinsen CAC, et al. COSMIN risk of bias
checklist for systematic reviews of patient-reported outcome
measures. Qual Life Res 2018;27:1171–9. https://doi.org/10.1007/
s11136-017-1765-4.
[22] Prinsen CAC, Mokkink LB, Bouter LM, et al. COSMIN guideline for
systematic reviews of patient-reported outcome measures. Qual
Life Res 2018;27:1147–57. https://doi.org/10.1007/s11136-018-
1798-3.
EUROPEAN UROLOGY OPEN SCIENCE 48 (2023) 1–11
11
