Prospective multi-institutional study evaluating the performance of prostate cancer risk calculators.

Abstract

Prostate cancer risk calculators incorporate many factors to evaluate an individual's risk for prostate cancer. We validated two common North American-based, prostate cancer risk calculators.
We conducted a prospective, multi-institutional study of 2,130 patients who underwent a prostate biopsy for prostate cancer detection from five centers. We evaluated the performance of the Sunnybrook nomogram-based prostate cancer risk calculator (SRC) and the Prostate Cancer Prevention Trial (PCPT) -based risk calculator (PRC) to predict the presence of any cancer and high-grade cancer. We examined discrimination, calibration, and decision curve analysis techniques to evaluate the prediction models.
Of the 2,130 patients, 867 men (40.7%) were found to have cancer, and 1,263 (59.3%) did not have cancer. Of the patients with cancer, 403 (46.5%) had a Gleason score of 7 or more. The area under the [concentration-time] curve (AUC) for the SRC was 0.67 (95% CI, 0.65 to 0.69); the AUC for the PRC was 0.61 (95% CI, 0.59 to 0.64). The AUC was higher for predicting aggressive disease from the SRC (0.72; 95% CI, 0.70 to 0.75) compared with that from the PRC (0.67; 95% CI, 0.64 to 0.70). Decision curve analyses showed that the SRC performed better than the PRC for risk thresholds of more than 30% for any cancer and more than 15% for aggressive cancer.
The SRC performed better than the PRC, but neither one added clinical benefit for risk thresholds of less than 30%. Further research is needed to improve the AUCs of the risk calculators, particularly for higher-grade cancer.

Full text

Page 1

Prospective Multi-Institutional Study Evaluating the
Performance of Prostate Cancer Risk Calculators
Robert K. Nam, Michael W. Kattan, Joseph L. Chin, John Trachtenberg, Rajiv Singal, Ricardo Rendon,
Laurence H. Klotz, Linda Sugar, Christopher Sherman, Jonathan Izawa, David Bell, Aleksandra Stanimirovic,
Vasundara Venkateswaran, Eleftherios P. Diamandis, Changhong Yu, D. Andrew Loblaw, and Steven A. Narod
See accompanying editorial doi: 10.1200/JCO.2011.36.1329; listen to the podcast by Dr
Cooperberg at www.jco.org/podcasts
Robert K. Nam, Laurence H. Klotz,
Linda Sugar, Christopher Sherman,
Aleksandra Stanimirovic, Vasundara
Venkateswaran, and D. Andrew
Loblaw, Sunnybrook Health Sciences
Centre; John Trachtenberg, Princess
Margaret Hospital; Rajiv Singal, Toronto
East General Hospital; Steven A. Narod,
University of Toronto; Eleftherios
Diamandis, Mount Sinai Hospital,
Toronto; Joseph L. Chin and Jonathan
Izawa, University of Western Ontario,
London, Ontario; Ricardo Rendon and
David Bell, Queen Elizabeth II Health
Sciences Centre, Halifax, Nova Scotia,
Canada; and Michael W. Kattan and
Changhong Yu, The Cleveland Clinic,
Cleveland, OH.
Submitted September 9, 2010;
accepted March 29, 2011; published
online ahead of print at www.jco.org on
June 20, 2011.
Supported by operating Grants No.
010284 and 015164 from the Canadian
Cancer Society Research Institute.
Authors’ disclosures of potential con-
flicts of interest and author contribu-
tions are found at the end of this
article.
Corresponding author: Robert K. Nam,
MD, Sunnybrook Health Sciences
Centre, 2075 Bayview Ave, Room
MG-406, Toronto, Ontario, Canada,
M4N 3M5; e-mail: robert.nam@
utoronto.ca.
© 2011 by American Society of Clinical
Oncology
0732-183X/11/2999-1/$20.00
DOI: 10.1200/JCO.2010.32.6371
ABSTRACT
Purpose
Prostate cancer risk calculators incorporate many factors to evaluate an individual’s risk for prostate
cancer. We validated two common North American–based, prostate cancer risk calculators.
Patients and Methods
We conducted a prospective, multi-institutional study of 2,130 patients who underwent a prostate
biopsy for prostate cancer detection from five centers. We evaluated the performance of the
Sunnybrook nomogram–based prostate cancer risk calculator (SRC) and the Prostate Cancer
Prevention Trial (PCPT) –based risk calculator (PRC) to predict the presence of any cancer and
high-grade cancer. We examined discrimination, calibration, and decision curve analysis tech-
niques to evaluate the prediction models.
Results
Of the 2,130 patients, 867 men (40.7%) were found to have cancer, and 1,263 (59.3%) did not
have cancer. Of the patients with cancer, 403 (46.5%) had a Gleason score of 7 or more. The
area under the [concentration-time] curve (AUC) for the SRC was 0.67 (95% CI, 0.65 to 0.69);
the AUC for the PRC was 0.61 (95% CI, 0.59 to 0.64). The AUC was higher for predicting
aggressive disease from the SRC (0.72; 95% CI, 0.70 to 0.75) compared with that from the
PRC (0.67; 95% CI, 0.64 to 0.70). Decision curve analyses showed that the SRC performed
better than the PRC for risk thresholds of more than 30% for any cancer and more than 15%
for aggressive cancer.
Conclusion
The SRC performed better than the PRC, but neither one added clinical benefit for risk thresholds
of less than 30%. Further research is needed to improve the AUCs of the risk calculators,
particularly for higher-grade cancer.
J Clin Oncol 29. © 2011 by American Society of Clinical Oncology
INTRODUCTION
The results of randomized studies that evaluate
the efficacy of prostate cancer screening that
usestheprostate-specificantigen(PSA)testhigh-
light the need to reduce the overdiagnosis of pa-
tients with indolent and nonaggressive forms of
prostate cancer.1,2New approaches are needed to
better select patients who are at increased risk for
having prostate cancer, particularly the aggres-
sive, high-grade forms.
The use of prostate cancer risk calculators for
decidingwhomtobiopsyhasbeendemonstratedto
be superior to conventional decision making based
onPSAanddigitalrectalexamination(DRE).3,4The
Prostate Cancer Prevention Trial (PCPT) showed
thatage,familyhistoryofprostatecancer,ethnicity,
and DRE results increased the positive predictive
valueofPSAanddevelopedanonlineprostatecan-
cer risk calculator (PRC) based on these factors.4
Our group developed a Sunnybrook nomogram–
based online prostate cancer risk calculator (SRC)
that estimates an individual’s risk for any prostate
cancerandhigh-gradeprostatecancer.3Weshowed
that the combination of age, family history of
prostate cancer, ethnicity, urinary voiding symp-
tom score, DRE, PSA, and free:total PSA ratio
performed significantly better than conventional
use of PSA and DRE in predicting the presence of
prostate cancer.
JOURNAL OF CLINICAL ONCOLOGY
O R I G I N A L R E P O R T
© 2011 by American Society of Clinical Oncology
1

http://jco.ascopubs.org/cgi/doi/10.1200/JCO.2010.32.6371 The latest version is at
Published Ahead of Print on June 20, 2011 as 10.1200/JCO.2010.32.6371

Copyright 2011 by American Society of Clinical Oncology
202.120.88.12
Information downloaded from jco.ascopubs.org and provided by at East-China Normal University on October 23, 2012 from
Copyright © 2011 American Society of Clinical Oncology. All rights reserved.

Page 2

Tovalidatetheseprostatecancerriskcalculators,weconducteda
prospective,multi-institutionalstudythatevaluatedtheperformance
of these instruments. We compared the ability of these prediction
models with respect to their discrimination and calibration tech-
niques5in predicting the presence of prostate cancer and aggressive,
high-gradeprostatecancer.Wealsoperformeddecisioncurveanalysis
tocomparethemodels.
PATIENTS AND METHODS
Study Patients
From 2007 to 2009, patients were prospectively assembled from five
centersacrossCanadawithlargeprostatebiopsycohorts:SunnybrookHealth
Sciences Centre (n ? 604), Toronto East General Hospital (n ? 363), and
Princess Margaret Hospital (n ? 382) all in the Toronto area, and London
Health Sciences Centre in London, Ontario (n ? 420); and Victoria General
Hospital,Halifax,NovaScotia,Canada(n?361).
Patients were eligible for inclusion if they had an abnormal PSA level
(?2.6ng/mL)oranabnormalDREtest.Allpatientswereofferedatransrectal
ultrasound-guidedprostatebiopsy.However,theprobabilityforprostatecan-
cer based on the prostate cancer risk calculators was not factored into the
decision, and both patients and investigators were blinded to these estimates.
All patients underwent transrectal ultrasound-guided needle core biopsy (10
to12needlecoresamples).PatientswereexcludediftheirPSAlevelwasmore
than 50 ng/mL, if the decision to biopsy would be considered unequivocal
(n ? 67), if they had incomplete risk factor information (n ? 20), or if they
were unable to provide consent (n ? 10). Of the 2,227 total patients, 2,130
agreedtoparticipateinthestudy.
Primary End Point and Baseline Information
The primary end point was the histologic presence of adenocarcinoma
of the prostate biopsy specimen and aggressive cancer. All grading was based
on the Gleason scoring system.6No central pathology review was obtained.
However,eachtertiary-basedcenterwasconsideredareferentprostatebiopsy
center. Biopsy samples from Toronto East General Hospital (a community-
basedhospital)werealsoreviewedbyareferentprostatebiopsycenter.
Patient age at time of biopsy, urologic voiding history (American Uro-
logical Association [AUA] symptom score7), ethnic background, family his-
tory of prostate cancer, PSA level, free:total PSA ratio, and DRE results were
obtained by questionnaires. These questionnaires were carefully and system-
aticallyadministeredbyeachcenterbeforeprostatebiopsyorknowledgeofthe
primaryendpoint.Alldataandfollow-upinformationwereassembledateach
center by the designated principal investigator and then sent to a centralized
0
Ideal
Sunnybrook RC (C = 0.67)
PCPT RC (C = 0.61)
Sensitivity
A
1-Specificity
1.0
0.8
0.6
0.4
0.2
0.20.4 0.60.8 1.0
0
Ideal
Sunnybrook RC (C = 0.72)
PCPT RC (C = 0.67)
Sensitivity
B
1-Specificity
1.0
0.8
0.6
0.4
0.2
0.2 0.4 0.60.8 1.0
Fig 1. Receiver operating characteristic (ROC) curves in predicting the presence of
all prostate cancer (A) and aggressive prostate cancer (B) by using the Sunnybrook
risk calculator (RC) and the Prostate Cancer Prevention Trial (PCPT) RC. “C” refers to
the concordance index, area under the ROC curve.
Table 1. Multivariable Logistic Regression Analysis of Factors Associated
With Prostate Cancer
Factor
Frequency
Distribution
OR95% CIP
Cases Controls
No.% No.%
Age, years
? 60
60-70
? 70
Family history of prostate
cancer
Absent
Present
Ethnicity
Asian
White
Black
Urinary symptoms by
AUA symptom
score?
? 6
? 6
DRE
Normal
Abnormal
PSA (ng/mL)
? 2.6
2.6-4.0
? 4.0-10.0
? 10.0
Free:total PSA†
? 0.22
0.16-0.22
0.11-0.22
? 0.11
245
412
210
28
48
24
491
572
200
39
45
16
1.0
1.4
2.1
1.1 to 1.7
1.6 to 2.8
.002
? .001
661
206
76 1,003
24
79
21
1.0
1.32601.0 to 1.6.02
42513711
85
4
1.0
2.5
3.3
776
49
89 1,072
6
1.7 to 3.7
1.9 to 5.7
? .001
? .00154
436
431
50
50
543
720
43
57
1.0
0.7 0.6 to 0.8
? .001
679
188
78 1,120
22
89
11
1.0
2.6143 2.0 to 3.4
? .001
32
79
591
165
4
9
167
165
767
164
13
13
61
13
1.0
2.9
4.6
4.9
1.7 to 4.7
3.0 to 7.0
3.0 to 7.8
? .001
? .001
? .001
68
19
152
137
229
349
18
16
26
40
301
303
343
316
24
24
27
25
1.0
0.9
1.3
2.0
0.6 to 1.1
1.0 to 1.7
1.5 to 2.6
.24
.07
? .001
Abbreviations: AUA, American Urological Association; DRE, digital rectal
examination; OR, odds ratio; PSA, prostate-specific antigen.
?Based on AUA symptom score index from 0 to 35.
†Quartile of free:total PSA ratio based on the distribution of controls.
Nam et al
2
© 2011 by American Society of Clinical Oncology
Information downloaded from jco.ascopubs.org and provided by at East-China Normal University on October 23, 2012 from
Copyright © 2011 American Society of Clinical Oncology. All rights reserved.
JOURNAL OF CLINICAL ONCOLOGY
202.120.88.12

Page 3

database.Randomsampleswerechosenfromeachcenterandauditedfordata
accuracy by the principal study team. Each center had approval from its
respectiveresearchethicsboard.
Data Analysis
Thedistributionsofthebaselinepredictorvariableswereexamined.Multi-
variable unconditional logistic regression was used to examine the association
betweenpredictorvariablesofage,familyhistoryofprostatecancer,ethnicity,the
presenceofurinarysymptoms,PSA,free:totalPSAratio,andDRE.
To evaluate the performance of each prostate cancer risk calculator,
we obtained the predicted probability for any prostate cancer and for
aggressive prostate cancer for each patient from the PRC4(http://deb.uthscsa
.edu/URORiskCalc/Pages/uroriskcalc.jsp)andfromtheSRC3(http://www
.prostaterisk.ca) to evaluate each prediction model performance as estab-
lishedbySteyerbergetal.5SimilarstudypatientswitheitheranormalPSA
level (? 4.0 ng/mL) or an abnormal PSA level (? 4.0 ng/mL) or DRE
results were used to help develop the risk calculators. A random sample of
25% of the probabilities was reviewed to ensure accuracy of data entry.
Each calculator provides probabilities for the presence of any cancer and
aggressivecancerdefinedbyahistologicgradeofGleasonscore7ormore.
Wequantifiedthediscriminationabilityoftheriskcalculatorsbycalculat-
ing the concordance index, which is identical to the nonparametric area
underthereceiveroperatingcharacteristiccurve(AUC).Itgivestheprob-
abilitythat,inarandomlyselectedpairofpatientsinwhichonepatienthas
prostatecancerandtheotherdoesnot,thepatientwithprostatecancerwill
beassignedtheworsepredictedrisk.Itrangesfrom0.5(nodiscrimination)
to1(perfectdiscrimination).TotestthesignificancebetweentheAUCsof
the risk calculators, we created 2,000 concordance indices for each model
by using bootstrapping analysis and then calculated the differences be-
tween the paired concordance indices. Significance was determined if the
2.5th quantile of the sorted difference was greater than zero.
Wethencomparedthecalibrationofthetworiskcalculatorsbyplotting
the predictions on the x-axis and the observed outcomes on the y-axis in the
same plot. In the calibration plot, the 45-degree line represents the perfect
predictions. Because of binary outcomes, a smoothing technique was used to
generatetheobservedprobabilitiesofprostatecanceronthey-axis.Finally,we
conducted a decision curve analysis that was proposed by Vickers et al8to
assess the clinical usefulness of the prediction tools by quantifying the net
benefits when different threshold probabilities were considered. All statistical
analyseswereperformedbyusingRversion2.9.0(http://www.R-project.org)
withtheDesignandHmisclibrariesadded.
RESULTS
Of 2,130 men, the median PSA level was 5.7 ng/mL (interquartile
range,4.2to8.1ng/mL),and331men(15.5%)hadanabnormalDRE.
The median age at biopsy was 63 years (interquartile range, 58.0 to
69.0 years). In addition, 466 (22%) had a positive family history of
prostatecancer.Themajorityofpatientswerewhite(87%;n?1,848),
with 8% (n ? 179) having an Asian background and 5% (n ? 103)
havingAfricanancestry.Ofthe443patientswithaPSAlevellessthan
4.0ng/mL,343(77%)hadanormalDRE.
A total of 867 men (41%) were found to have prostate cancer at
biopsy(cases),and1,263(59%)hadnoevidenceofcancer(controls).
Of the patients with cancer, 464 (54%) had a histologic grade of
Gleason score 6, 235 (27%) had Gleason score 7, and 168 (19%) had
Gleason score 8 to 10. All of the factors that our earlier studies con-
firmedweresignificantlyassociatedwithprostatecancerrisk,includ-
ing age, family history of prostate cancer, ethnicity, urinary voiding
symptoms, PSA, free:total PSA ratio, and DRE were also significant
predictorsforprostatecancer(Table1).
To quantify the discrimination ability of the risk calculators
amongthe2,130men,weexaminedtheconcordanceindex(AUC)for
eachmodelinpredictinganycancerandhigh-grade,aggressivecancer
(Gleasonscore7orhigher).TheAUCfortheSRC(0.67;95%CI,0.65
to0.69)inpredictingprostatecancerwassignificantlyhigherthanthat
forthePRC(0.61;95%CI,0.59to0.64;P?.001forcomparison).The
AUC was also higher for predicting aggressive disease for the SRC
(0.72; 95% CI, 0.70 to 0.75) compared with the PRC (0.67; 95% CI,
0.64to0.70;P?.001forcomparison;Fig1).Whenwecomparedthe
sensitivity, specificity, positive predictive value, and negative predic-
tive value for predicting any cancer, the risk calculators appeared to
havesimilartestcharacteristicsoverarangeofnomogramprobabili-
ties for predicting any cancer (Table 2). For predicting aggressive
prostate cancer, the SRC appeared to have better sensitivity and neg-
ativepredictivevalueforthelowthresholdrange(Table2).
Table 2. Sensitivity, Specificity, PPV, and NPV for the Risk Calculators for a Range of Probability Thresholds for Prostate Cancer
Nomogram Probability (%)
Sensitivity (%) Specificity (%) PPV (%)NPV (%)
SRCPRC SRC PRCSRC PRCSRC PRC
Test Characteristics for Predicting Any Cancer
10
15
25
40
50
75
Test Characteristics for Predicting Aggressive Cancer
100
98
83
54
37
100
99
98
77
48
1
8
1
2
6
41
42
46
55
60
82
43
43
44
47
52
69
90
86
74
69
66
61
83
82
82
67
63
58
34
70
83
99
35
67
9886
10
15
25
40
50
75
94
82
56
30
17
83
66
41
22
12
23
47
75
93
97
100
34
57
75
94
97
100
22
27
34
49
55
77
24
28
35
47
57
78
94
92
88
85
83
81
89
87
84
83
82
8024
Abbreviations: NPV, negative predictive value; PPV, positive predictive value; PRC, Prostate Cancer Prevention Trial (PCPT) –based risk calculator; SRC, Sunnybrook
nomogram–based prostate cancer risk calculator.
Prostate Cancer Risk Calculators
www.jco.org
© 2011 by American Society of Clinical Oncology
3
202.120.88.12
Information downloaded from jco.ascopubs.org and provided by at East-China Normal University on October 23, 2012 from
Copyright © 2011 American Society of Clinical Oncology. All rights reserved.

Page 4

To assess the agreement between the predicted and actual out-
comes,wegeneratedcalibrationcurvesforeachriskcalculatorforany
and high-grade cancer. When compared with the ideal curve (45-
degree line) in which there was perfect agreement between the pre-
dictedandactualoutcomes,theSRCwascloserthanthePRCforboth
anyandaggressivecancer(Fig2).
Becausethereisnoestablishedcutoffprobabilityforthepros-
tate cancer risk calculators, we examined the theoretical relation-
shipbetweenarangeofthresholdprobabilitiesforcancerinhowit
affects the relative value of false-positive and false-negative results
(termed net benefit) by using decision curve analysis described by
Vickers et al.8When compared with a theoretical scenario of per-
forming a prostate biopsy for every patient, the ability of both risk
calculators to predict cancer provided a better net benefit over a
range of threshold probabilities for any and aggressive prostate
cancer (Fig 3). The risk calculators did not provide further benefit
comparedwithascenarioofconductingabiopsyforallpatientsfor
threshold probabilities of ? 30% for any cancer and ? 15% for
aggressive cancer. The risk calculators also did not provide further
0
Ideal
Sunnybrook RC (C = 0.67)
PCPT RC (C = 0.61)
Actual All PCa (probability)
A
Predicted All PCa (probability)
0.8
0.6
0.4
0.2
0.20.40.60.8
0
Ideal
Sunnybrook RC (C = 0.72)
PCPT RC (C = 0.67)
Actual High-Grade PCa
(probability)
B
Predicted High-Grade PCa (probability)
0.8
0.6
0.4
0.2
0.20.4 0.60.8
Fig 2. Calibration curve comparisons between the Prostate Cancer Preven-
tion Trial (PCPT) –based and Sunnybrook nomogram–based prostate cancer
(PCa) risk calculators (RCs) for any cancer (A) and high-grade, aggressive
cancer (B). The y-axis represents observed prostate cancer rate. Diagonal
black 45-degree dotted line represents perfect prediction by ideal model. “C”
refers to the concordance index, area under the receiver operating character-
istic curve. The histograms represent the predicted probabilities of the
models that display the density distribution of predicted risks. Because the
histograms were constructed on the basis of the density distribution instead
of the absolute frequency of each predicted risk, the relative frequency can
be observed.
None
All
Sunnybrook RC
PCPT RC
Net Benefit
A
Threshold Probability (%)
0.4
0.3
0.2
0.1
0.0
20 40 60 80
Net Benefit
B
Threshold Probability (%)
0.20
0.15
0.10
0.05
0.00
-0.05
-0.10
0 20 40 6080
None
All
Sunnybrook RC
PCPT RC
Fig 3. Decision curve analysis for prostate cancer biopsy prediction for each
model for any cancer (A) and high-grade, aggressive cancer (B). The y-axis
measures net benefit, calculated by summing the benefits (true positives) and
subtracting the harms (false positives), in which the latter are weighted by a
factor related to the relative harm of a missed cancer compared with the harm
of an unnecessary biopsy. A model is of clinical value if it has the highest net
benefit compared with both other models and simple strategies such as
biopsying all patients (gray dashed line) or no patients (horizontal red line)
across the full range of threshold probabilities at which a patient would
choose to be biopsied. The unit is in terms of cancers found: a model with a
net benefit of 0.1 is the equivalent of a strategy that biopsied 10 men per 100
with no negative biopsies. For example, if we take a threshold probability of
40%, the use of the Prostate Cancer Prevention Trial (PCPT) risk calculator
(RC; dashed blue line) as a strategy would equate to 0.08 of net benefit which
translates to the identification and biopsy of eight men, all with cancer on
biopsy, of 100 men eligible (with no negative biopsies). Of those same 100
men, the Sunnybrook RC would have a net benefit of 0.12, which means that
it would identify 12 men, all with cancer with no negative biopsies. Thus, for
a threshold probability of 40% to perform a prostate biopsy, the Sunnybrook
RC would theoretically identify four more cases of prostate cancer per 100
men with no negative biopsies compared with the PCPT RC. The dashed blue
and solid gold lines evaluate the performance of the RCs for various strategies
over the wide spectrum of threshold probabilities to perform a prostate biopsy
from the decision analytic model.
Nam et al
4
© 2011 by American Society of Clinical Oncology
Information downloaded from jco.ascopubs.org and provided by at East-China Normal University on October 23, 2012 from
Copyright © 2011 American Society of Clinical Oncology. All rights reserved.
JOURNAL OF CLINICAL ONCOLOGY
202.120.88.12

Page 5

benefitcomparedwithascenarioofnotconductingabiopsyforall
patients for threshold probabilities of more than 55% for any
cancer and more than 40% for aggressive cancer. Within these
probabilities ranges, the SRC provided better value in net benefit
compared with the PRC (Fig 3).
DISCUSSION
To the best of our knowledge, this is the first study to prospectively
validate the current and most widely used North American–based
prostatecancerriskcalculatorsamongdifferentcenters.Amonganew
cohort of 2,130 men derived from multiple centers that underwent a
prostatebiopsy,theSRCperformedbetterthanthePRCinpredicting
thepresenceofanyandaggressiveprostatecancer.
Since the original publications of the risk calculators, many in-
vestigators have called for external validation of the risk calculators.9
Several studies have examined the PRC with reported AUCs widely
ranging from 0.57 to 0.67, likely due to their retrospective study
designs.10-13The advantages of the current prospective study com-
paredwithpastretrospectivestudiesarethattheyarefreefromascer-
tainmentandrecallbiasesoftheriskfactorsthatareimportanttothe
predictormodels,includingfamilyhistoryofprostatecancer,ethnic-
ity, and urinary voiding symptoms. Importantly, the decision to un-
dergoabiopsywasnotbasedontheriskcalculators,whichminimized
potentialselectionbiases.
TheAUC(0.72)oftheSRCforpredictingaggressive,high-grade
prostatecancerwashighercomparedwiththatforthePRC.Although
theabsolutelevelislow,thisvalueisaffectedbymanyfactors.14From
a decision-making perspective, the comparison of options is what
matters and selecting the most accurate calculator available. This su-
periorpredictivevalueoftheSRCislikelyduetotheincorporationof
additional factors including free:total PSA ratio and urinary voiding
symptoms. We acknowledge that this validation study is limited to
patients referred for a prostate cancer evaluation and may not reflect
the general population for prostate cancer screening, as observed in
our high cancer detection rate of 40%. Nevertheless, this cohort re-
flects a contemporary cohort of screened patients as reflected in a
medianPSAof5.7ng/mLandanabnormalDREinonly15%.
Another limitation of the study was that there was no central
pathology review. However, we chose only centers that were consid-
ered a referent genitourinary pathology site, which minimized inter-
observervariability.
The main clinical utility of prostate cancer risk calculators is to
facilitatethedecisiononwhetherapatientrequiresaprostatebiopsy.
This depends on which threshold probability is used. It has been
suggested that a range of threshold probabilities for cancer between
10%and40%woulddeterminetheneedforprostatebiopsy.15Thatis,
a risk of 10% for cancer would likely dissuade a patient from under-
going an invasive biopsy, although a risk of 40% would compel a
patient to have a biopsy (especially if the risk was for high-grade
cancer). By using this arbitrarily defined range, both risk calculators
wouldnotimproveclinicaloutcomerelativetobiopsyingallmenwith
anincreasedPSAorabnormalDRE.Fromourresults,ourriskcalcu-
lator (SRC) would have the most benefit in decision making for
patientswhorequirea30%orgreaterriskofcancerbeforetheywould
agree to a biopsy, or physicians who would do no more than three
biopsiestofindonecancer.
However, it is important to emphasize that there are no stud-
iestodatethathaveevaluatedwhatwouldbeanacceptablerangeof
threshold probabilities that would help with prostate biopsy man-
agement. Although 10% to 40% appears reasonable, it is possible
that patients and physicians may accept lower or higher risk
thresholds, depending on how risk averse an individual would be.
ResultsofthePCPTshowedaprevalenceofprostatecancerof15%
amongpatientsintheplaceboarmwithanormalPSAwhounder-
wentanend-of-studybiopsy,16whichcouldserveasabaselinerisk
for prostate cancer. In the absence of definitive risk thresholds, it
would be important to provide a range of threshold probabilities,
although extreme ranges would not be helpful. The decision ana-
lytic curves (Fig 3) show that the SRC in the range of these proba-
bilities was consistently better in net benefit than the PRC (Fig 3),
particularly for predicting aggressive, high-grade cancer. An indi-
vidualpatientcanreceiveadifferentmessagewhenusingthemore
accurate SRC relative to the PRC.
The method of decision analytic curves is novel and is used to
addresstheproblemsofcurrentmethodsofhowweevaluatediagnos-
tictests.Alimitationofthedecisionanalyticcurvesisthattheconcept
ofnetbenefitisdifficulttoapplyclinicallysinceitisamathematically
derived definition. Another limitation is that decision curve analysis
does not help decide which probability threshold should be consid-
eredacceptable.Nonetheless,theanalysisprovidesarelativecompar-
isonbetweenpredictivemodelstoassesshowtheyperformwithinthe
spectrumofbeingeitherallpositiveorallnegativepredictions.
Thus,theSRCcanbeconsideredbyphysiciansandpatientswho
are considering undergoing a prostate biopsy to determine the pres-
enceofprostatecancer.Giventheimportanceofselectivelyidentifying
men with aggressive, high-grade cancer, prostate cancer risk calcula-
tors can play an important role in prostate biopsy decision making.
Further research will be required to evaluate what risk thresholds
wouldbeacceptableforpatientsandphysicians.
AUTHORS’ DISCLOSURES OF POTENTIAL CONFLICTS
OF INTEREST
The author(s) indicated no potential conflicts of interest.
AUTHOR CONTRIBUTIONS
Conception and design: Robert K. Nam, Michael W. Kattan,
Steven A. Narod
Financial support: Robert K. Nam
Administrative support: Robert K. Nam, Ricardo Rendon,
D. Andrew Loblaw
Provision of study materials or patients: Robert K. Nam, Joseph L.
Chin, John Trachtenberg, Rajiv Singal, Ricardo Rendon, Laurence H.
Klotz, Jonathan Izawa, David Bell
Collection and assembly of data: Robert K. Nam, Joseph L. Chin, John
Trachtenberg, Rajiv Singal, Ricardo Rendon, Laurence H. Klotz,
Jonathan Izawa, David Bell, Aleksandra Stanimirovic
Data analysis and interpretation: Robert K. Nam, Michael W. Kattan,
John Trachtenberg, Rajiv Singal, Laurence H. Klotz, Linda Sugar,
Christopher Sherman, Jonathan Izawa, David Bell, Vasundara
Venkateswaran, Eleftherios P. Diamandis, Changhong Yu, D. Andrew
Loblaw, Steven A. Narod
Manuscript writing: All authors
Final approval of manuscript: All authors
Prostate Cancer Risk Calculators
www.jco.org
© 2011 by American Society of Clinical Oncology
5
202.120.88.12
Information downloaded from jco.ascopubs.org and provided by at East-China Normal University on October 23, 2012 from
Copyright © 2011 American Society of Clinical Oncology. All rights reserved.

Page 6

REFERENCES
1. Schro ¨der FH, Hugosson J, Roobol MJ, et al:
Screening and prostate-cancer mortality in a ran-
domized European study. N Engl J Med 360:1320-
1328, 2009
2. Andriole GL, Crawford ED, Grubb RL 3rd, et
al: Mortality results from a randomized prostate-
cancer screening trial. N Engl J Med 360:1310-1319,
2009
3. Nam RK, Toi A, Klotz LH, et al: Assessing
individual risk for prostate cancer. J Clin Oncol
25:3582-3588, 2007.
4. Thompson IM, Ankerst DP, Chi C, et al:
Assessing prostate cancer risk: Results from the
Prostate Cancer Prevention Trial. J Natl Cancer Inst
98:529-534, 2006
5. Steyerberg EW, Vickers AJ, Cook NR, et al:
Assessing the performance of prediction models: A
framework for traditional and novel measures. Epi-
demiology 21:128-138, 2010
6. Gleason DF, Mellinger GT: Prediction of prog-
nosis for prostatic adenocarcinoma by combined
histological grading and clinical staging. J Urol 111:
58-64, 1974
7. Barry MJ, Fowler FJ Jr, O’Leary MP, et al: The
American Urological Association symptom index for
benign prostatic hyperplasia: The Measurement
Committee of the American Urological Association.
J Urol 148:1549-1557, 1992; discussion 1564
8. Vickers AJ, Elkin EB: Decision curve analysis:
A novel method for evaluating prediction models.
Med Decis Making 26:565-574, 2006
9. Ross RW, Kantoff PW: Predicting outcomes
in prostate cancer: How many more nomograms do
we need? J Clin Oncol 25:3563-3564, 2007
10. Nguyen CT, Yu C, Moussa A, et al: Perfor-
mance of prostate cancer prevention trial risk calcu-
lator in a contemporary cohort screened for prostate
cancer and diagnosed by extended prostate biopsy.
J Urol 183:529-533, 2010
11. Eyre SJ, Ankerst DP, Wei JT, et al: Validation
in a multiple urology practice cohort of the Prostate
Cancer Prevention Trial calculator for predicting
prostate cancer detection. J Urol 182:2653-2658,
2009
12. Parekh DJ, Ankerst DP, Higgins BA, et al:
External validation of the Prostate Cancer Preven-
tion Trial risk calculator in a screened population.
Urology 68:1152-1155, 2006
13. Hernandez DJ, Han M, Humphreys EB, et al:
Predicting the outcome of prostate biopsy: Compar-
ison of a novel logistic regression-based model, the
prostate cancer risk calculator, and prostate-specific
antigen level alone. BJU Int 103:609-614, 2009
14. Kattan MW: Factors affecting the accuracy of
prediction models limit the comparison of rival pre-
diction models when applied to separate data sets.
Eur Urol 59:566-567, 2011
15. Steyerberg EW, Vickers AJ: Decision curve
analysis: A discussion. Med Decis Making 28:146-
149, 2008
16. Thompson IM, Pauler DK, Goodman PJ, et al:
Prevalence of prostate cancer among men with a
prostate-specific antigen level ? or ? 4.0 ng per
milliliter. N Engl J Med 350:2239-2246, 2004
■ ■ ■
Nam et al
6
© 2011 by American Society of Clinical Oncology
Information downloaded from jco.ascopubs.org and provided by at East-China Normal University on October 23, 2012 from
Copyright © 2011 American Society of Clinical Oncology. All rights reserved.
JOURNAL OF CLINICAL ONCOLOGY
202.120.88.12