Establishing guidelines to harmonize tumor mutational burden (TMB): In silico assessment of variation in TMB quantification across diagnostic platforms: Phase I of the Friends of Cancer Research TMB Harmonization Project

Abstract

Background
Tumor mutational burden (TMB), defined as the number of somatic mutations per megabase of interrogated genomic sequence, demonstrates predictive biomarker potential for the identification of patients with cancer most likely to respond to immune checkpoint inhibitors. TMB is optimally calculated by whole exome sequencing (WES), but next-generation sequencing targeted panels provide TMB estimates in a time-effective and cost-effective manner. However, differences in panel size and gene coverage, in addition to the underlying bioinformatics pipelines, are known drivers of variability in TMB estimates across laboratories. By directly comparing panel-based TMB estimates from participating laboratories, this study aims to characterize the theoretical variability of panel-based TMB estimates, and provides guidelines on TMB reporting, analytic validation requirements and reference standard alignment in order to maintain consistency of TMB estimation across platforms.

Methods
Eleven laboratories used WES data from The Cancer Genome Atlas Multi-Center Mutation calling in Multiple Cancers (MC3) samples and calculated TMB from the subset of the exome restricted to the genes covered by their targeted panel using their own bioinformatics pipeline (panel TMB). A reference TMB value was calculated from the entire exome using a uniform bioinformatics pipeline all members agreed on (WES TMB). Linear regression analyses were performed to investigate the relationship between WES and panel TMB for all 32 cancer types combined and separately. Variability in panel TMB values at various WES TMB values was also quantified using 95% prediction limits.

Results
Study results demonstrated that variability within and between panel TMB values increases as the WES TMB values increase. For each panel, prediction limits based on linear regression analyses that modeled panel TMB as a function of WES TMB were calculated and found to approximately capture the intended 95% of observed panel TMB values. Certain cancer types, such as uterine, bladder and colon cancers exhibited greater variability in panel TMB values, compared with lung and head and neck cancers.

Conclusions
Increasing uptake of TMB as a predictive biomarker in the clinic creates an urgent need to bring stakeholders together to agree on the harmonization of key aspects of panel-based TMB estimation, such as the standardization of TMB reporting, standardization of analytical validation studies and the alignment of panel-based TMB values with a reference standard. These harmonization efforts should improve consistency and reliability of panel TMB estimates and aid in clinical decision-making.

Full text

1
MerinoDM, etal. J Immunother Cancer 2020;8:e000147. doi:10.1136/jitc-2019-000147
Open access
Establishing guidelines to harmonize
tumor mutational burden (TMB): in
silico assessment of variation in TMB
quantication across diagnostic
platforms: phase I of the Friends of
Cancer Research TMB
Harmonization Project
Diana M Merino ,1 Lisa M McShane,2 David Fabrizio,3 Vincent Funari,4
Shu- Jen Chen,5 James R White,6 Paul Wenz,7 Jonathan Baden,8 J Carl Barrett,9
Ruchi Chaudhary,10 Li Chen,11 Wangjuh (Sting) Chen,12 Jen- Hao Cheng,5
Dinesh Cyanam,10 Jennifer S Dickey,13 Vikas Gupta,14 Matthew Hellmann,15
Elena Helman,16 Yali Li,3 Joerg Maas,17 Arnaud Papin,18 Rajesh Patidar,11
Katie J Quinn,16 Naiyer Rizvi,19 Hongseok Tae,12 Christine Ward,8 Mingchao Xie,20
Ahmet Zehir,15 Chen Zhao,7 Manfred Dietel,17 Albrecht Stenzinger,21 Mark Stewart,1
Jeff Allen,1 On behalf of the TMB Harmonization Consortium
To cite: MerinoDM,
McShaneLM, FabrizioD, etal.
Establishing guidelines to
harmonize tumor mutational
burden (TMB): in silico
assessment of variation in TMB
quantication across diagnostic
platforms: phase I of the Friends
of Cancer Research TMB
Harmonization Project. Journal
for ImmunoTherapy of Cancer
2020;8:e000147. doi:10.1136/
jitc-2019-000147
►Additional material is
published online only. To view,
please visit the journal online
(http:// dx. doi. org/ 10. 1136/ jitc-
2019- 000147).
Accepted 11 February 2020
For numbered afliations see
end of article.
Correspondence to
Dr Diana M Merino;
dmerino@ focr. org
Original research
© Author(s) (or their
employer(s)) 2020. Re- use
permitted under CC BY- NC. No
commercial re- use. See rights
and permissions. Published by
BMJ.
ABSTRACT
Background Tumor mutational burden (TMB), dened
as the number of somatic mutations per megabase of
interrogated genomic sequence, demonstrates predictive
biomarker potential for the identication of patients with
cancer most likely to respond to immune checkpoint
inhibitors. TMB is optimally calculated by whole exome
sequencing (WES), but next- generation sequencing
targeted panels provide TMB estimates in a time- effective
and cost- effective manner. However, differences in panel
size and gene coverage, in addition to the underlying
bioinformatics pipelines, are known drivers of variability
in TMB estimates across laboratories. By directly
comparing panel- based TMB estimates from participating
laboratories, this study aims to characterize the
theoretical variability of panel- based TMB estimates, and
provides guidelines on TMB reporting, analytic validation
requirements and reference standard alignment in order to
maintain consistency of TMB estimation across platforms.
Methods Eleven laboratories used WES data from The
Cancer Genome Atlas Multi- Center Mutation calling in
Multiple Cancers (MC3) samples and calculated TMB from
the subset of the exome restricted to the genes covered by
their targeted panel using their own bioinformatics pipeline
(panel TMB). A reference TMB value was calculated from
the entire exome using a uniform bioinformatics pipeline
all members agreed on (WES TMB). Linear regression
analyses were performed to investigate the relationship
between WES and panel TMB for all 32 cancer types
combined and separately. Variability in panel TMB values
at various WES TMB values was also quantied using 95%
prediction limits.
Results Study results demonstrated that variability within
and between panel TMB values increases as the WES
TMB values increase. For each panel, prediction limits
based on linear regression analyses that modeled panel
TMB as a function of WES TMB were calculated and found
to approximately capture the intended 95% of observed
panel TMB values. Certain cancer types, such as uterine,
bladder and colon cancers exhibited greater variability in
panel TMB values, compared with lung and head and neck
cancers.
Conclusions Increasing uptake of TMB as a predictive
biomarker in the clinic creates an urgent need to bring
stakeholders together to agree on the harmonization of
key aspects of panel- based TMB estimation, such as
the standardization of TMB reporting, standardization
of analytical validation studies and the alignment of
panel- based TMB values with a reference standard.
These harmonization efforts should improve consistency
and reliability of panel TMB estimates and aid in clinical
decision- making.
BACKGROUND
Immune checkpoint inhibitors (ICIs) have
recently emerged as a pillar of cancer care,
providing the potential for durable responses
and improved survival for patients across
multiple cancer types.1–3 An intensive clinical
development pipeline investigating ICIs is
ongoing as a result. However, not all patients
with cancer respond to ICIs, with modest
on March 26, 2020 by guest. Protected by copyright.http://jitc.bmj.com/J Immunother Cancer: first published as 10.1136/jitc-2019-000147 on 26 March 2020. Downloaded from

2MerinoDM, etal. J Immunother Cancer 2020;8:e000147. doi:10.1136/jitc-2019-000147
Open access
response rates for several approved indications (approx-
imately 20% or less in lung cancer, bladder cancer and
cancers of the head and neck, among others) and high
treatment costs. There is a crucial interest in the devel-
opment of biomarker assays to predict which patients
are most likely to respond and benefit from ICIs, and to
improve clinical decision- making and disease manage-
ment.4 5
Expression of the programmed cell death ligand
protein-1 (PD- L1) by immunohistochemistry (IHC) has
been studied extensively as a biomarker of response to
anti- PD- L1 and programmed cell death protein 1 (PD-1)
therapy. Several assays have been developed to quantify
tumor PD- L1 immuno- positivity; however, quantitation is
imperfect, and lack of standardization across platforms
and scoring systems precludes assay interchangeability.6
Tumor mutational burden (TMB), which measures the
number of somatic mutations per megabase (Mb) of the
interrogated genomic sequence of a tumor, has been
most recently identified as a biomarker of response
to ICIs in several cancer types. High TMB is associated
with improved outcomes in patients with melanoma
treated with cytotoxic T- lymphocyte- associated protein 4
(CTLA-4) blockade7–9 and PD-1/PD- L1 blockade across
several cancer types, including melanoma,10 11 non- small-
cell lung carcinoma,12–15 bladder cancer,16 microsatellite
instability cancers3 17 and pan- tumor cohorts.18–20 High
TMB has also been associated with improved outcomes in
patients treated with a combination of PD-1/PD- L1 and
CTLA-4 inhibitors.21–24
Initial assessments of TMB involved whole exome
sequencing (WES) of matched tumor tissue and normal
specimens using next- generation sequencing (NGS).3 8–10
However, WES is not currently routine in clinical practice
due to substantial cost and turnaround time, which has
led assay manufacturers and commercial and academic
labs to develop targeted NGS panels. These targeted
panels, which cover several hundred genes, are already
routinely used in clinical practice, and are currently being
adapted to estimate TMB. TMB estimated from targeted
NGS panels has generally correlated well with TMB deter-
mined by WES, however the reliability of this technology
is still being assessed.13–16 20 22 25–30
There are several targeted NGS panels at different
stages of development that estimate TMB. To date,
the Foundation Medicine FoundationOne CDx test31
is currently the only Food and Drug Administration
(FDA)- approved panel, which includes TMB as part
of its tumor profiling claim, while the Memorial Sloan
Kettering Cancer Center MSK- IMPACT (Integrated
Mutation Profiling of Actionable Cancer Targets)32
has received FDA authorization. Additionally, there
are many more commercial and laboratory- developed
test panels currently under development. Each panel
has unique features integrated into their design that
may impact TMB estimation. For example, each panel
may include different numbers and types of genes,
use different sequencing platforms, have different
methods of filtering germline mutations, incorporate
different mutation types in the quantification of TMB
and use proprietary bioinformatics protocols to calcu-
late TMB.33 34 Thus, TMB estimates will vary according
to the targeted panel used.35 This is a crucial time to
understand the differences in TMB estimation across
panels, standardize the way TMB is reported, begin to
harmonize methods for TMB quantification and iden-
tify optimal approaches to promote TMB alignment
across different targeted NGS panels.
Friends of Cancer Research (Friends) convened a
consortium of key stakeholders, including diagnostic
manufacturers, academics, pharmaceutical companies,
the National Cancer Institute and the FDA, to recom-
mend best practices and approaches for TMB measure-
ment, validation, alignment and reporting well ahead
of the adoption of this powerful biomarker in clinical
decision- making. Leveraging the expertise and insights
of this comprehensive group of stakeholders, the Friends
TMB harmonization project seeks to establish a uniform
approach to measure and report TMB across different
sequencing panels by harmonizing the definition of
TMB, proposing best practices for analytic validation
studies and ensuring consistency of TMB calculation
through alignment with a universal reference standard.
The project consists of a stepwise approach broken down
into three phases: phase I, reported here, comprises the
in silico analysis, which by using publicly available data
from The Cancer Genome Atlas (TCGA) representing 32
cancer types, aims to identify the theoretical variability of
panel- derived TMB estimates (panel TMB) relative to a
common, standardized WES- derived TMB (WES TMB)
across various panels. Building on the results of the in
silico analysis, phase II will analyze human tumor clinical
sample material to objectively measure variation across
panels using patient formalin- fixed paraffin- embedded
(FFPE) tissue samples. This empirical analysis will also
compare panel TMB results to an agreed on universal
reference standard, consisting of a collection of human
tumor- derived reference cell lines that span a clinically
meaningful TMB dynamic range. FFPE tissue samples will
also be used to validate the use of the cell line standard.
Finally, phase III will involve a clinical study that seeks to
retrospectively analyze samples from patients treated with
ICIs to evaluate optimal cut- off values that will help guide
the clinical application of TMB (see online supplemen-
tary figure 1).
The need for harmonization of TMB is a global effort,
which is portrayed by the representation of national and
international diagnostic companies in the consortium.
Moreover, in seeking to complement the consortium’s
work, the Friends TMB harmonization project has part-
nered with the technical comparability study conducted
by Quality in Pathology in Germany,36 leading to the
identification of common and panel- specific factors
that influence TMB estimation and the development of
global recommendations, which have been published
previously.33
on March 26, 2020 by guest. Protected by copyright.http://jitc.bmj.com/J Immunother Cancer: first published as 10.1136/jitc-2019-000147 on 26 March 2020. Downloaded from

3
MerinoDM, etal. J Immunother Cancer 2020;8:e000147. doi:10.1136/jitc-2019-000147
Open access
Due to the large scale and collaborative nature of
this effort, study results will greatly contribute to under-
standing and refining how to best quantify and interpret
TMB as a biomarker, help establish standards that will
facilitate harmonization across different testing platforms
and inform future harmonization efforts that seek to
ensure consistency across diagnostic platforms.
METHODS
In silico dataset
Mutation calls generated using Multi- Center Mutation
calling in Multiple Cancers (MC3) WES data from TCGA
project were used for this analysis.37 Variants that over-
lapped with the CCDS, using bedtools (- wa option)38
were extracted from the publicly available mc3. v0. 2. 8.
PUBLIC. maf file (https:// gdc. cancer. gov/ about- data/
publications/ pancanatlas). Finally, the data were filtered
for any overlap or redundancy using the ‘merge’ func-
tion. The consortium created a final bed file that covered
32.102 Mb of the genome after intersecting the data
found in the MAF files and filtering for any overlap or
redundancy (see online supplementary methods). The
final bed file size was used as the denominator for calcu-
lating WES TMB in this study. Three different consortium
laboratories independently calculated WES TMB using
the same dataset and analytical methodology with 100%
concordance.
Ten thousand two hundred ninety- five tumor samples
with matched normal initially composed part of the
cohort. Only samples with at least one variant which
PASSED variant review filter were used (see online
supplementary methods for variant quality filters). Low
quality samples based on variant filters and those with
low purity were also removed from further analysis. The
remaining cases (n=8291) were randomly assigned to
training (n=4157) and validation (n=4134) datasets with
similar median candidate mutations and cancer types
(online supplementary figure 2). Participants, though
not required, could use the ‘training’ set for their own
algorithm or parameter testing. However, all analyses
described herein were conducted using the validation
dataset.
The evaluations reported in the present study are those
comparing panel TMB to WES TMB on the validation
set, with no adjustments made to the panel TMB algo-
rithms once the validation set analyses began. All analyses
focused on tumors for which WES TMB was ≤40 because
>98% of the TCGA dataset tumors investigated had TMB
≤40 in the TCGA dataset and all members of the consor-
tium agreed that this range would have the greatest rele-
vance for clinical decision- making. Of the 4134 tumors
initially represented in the validation set, 4065 remained
after excluding those with WES TMB >40. All results were
blinded to the entire consortium, with the exception of
the project statistician and data manager (LMMS and
DMM) who were regarded as neutral parties not affiliated
with any of the participating laboratories.
Statistical analysis
Statistical analyses interrogated the relationship between
WES TMB and panel TMB values. The first analysis
focused on the combined data from all 32 tumor types.
Spearman’s R correlation values were calculated, and
scatterplots and difference plots were created to assess
linearity of the relationship between panel TMB and WES
TMB and to evaluate whether variance of panel TMB was
constant across the range of WES TMB values.
Next, the 32 tumors were divided into three strata
according to the number of samples within each tumor
type that had TMB values spanning the range 0–40 mut/
Mb (see online supplementary methods and figure 3).
Stratum 1 contained eight tumor types (see online supple-
mentary table 1A—stratum 1) displaying a good distribu-
tion of TMB values spanning the range of interest (0–40
mut/Mb). Seventy- seven per cent of samples (1257/1627)
had TMB ≤10 mut/Mb, 19% (306/1627) had TMB 10–40
mut/Mb and 4% (64/1627) had TMB ≥40 mut/Mb and
were thus eliminated from further analyses. Stratum 2 was
represented by 11 tumor types (see online supplemen-
tary table 1B—stratum 2) whose samples had generally
low TMB values (≤10 mut/Mb, 98%, 1723/1754), and
only 1.5% (26/1754) of samples had TMB 10–40 mut/
Mb. Only five samples (0.29%) had TMB ≥40 mut/Mb
and were thus eliminated from further analyses. Stratum
3 was represented by 13 tumor types (see online supple-
mentary table 1C—stratum 3) whose samples had very
low TMB values (≤5 mut/Mb, 99.5%, 749/753) and only
4 samples (0.5%) had samples with TMB between 5 and
10 mut/Mb. Regression modeling using weighted least
squares was implemented to account for the heterosce-
dasticity in errors, referring to the variability in panel
TMB values about the fitted regression line, which was
observed to increase with the mean and with WES TMB.
This modeling was conducted for all strata, although we
focused on stratum 1 considering strata 2 and 3 provided
less stable and unreliable estimates due to the large
number of samples that concentrated in the lower end of
the TMB range.
For each regression, the mean panel TMB was modeled
as a simple linear function of the WES TMB, and five
different models for the error variance were consid-
ered (see online supplementary methods). Restricted
maximum likelihood analysis using the gls function avail-
able in the R package nlme was performed to estimate
the model parameters and select a best fitting variance
structure based on minimum Akaike information and
Bayesian information criteria.
Whole exome analysis
The whole exome analysis of the TCGA MC3 validation
dataset used an agreed on methodology to calculate the
WES TMB values, termed the Uniform TMB Calculation
Method (see online supplementary table 2). The goal of
phase I of this harmonization study is to assess the theo-
retical variability across panels. Given that the partici-
pating panels were at different stages of development and
on March 26, 2020 by guest. Protected by copyright.http://jitc.bmj.com/J Immunother Cancer: first published as 10.1136/jitc-2019-000147 on 26 March 2020. Downloaded from

4MerinoDM, etal. J Immunother Cancer 2020;8:e000147. doi:10.1136/jitc-2019-000147
Open access
Table 1 Description of the 11 participating diagnostic NGS panels
Laboratory Panel name # genes
Total region
covered
(Mb)
TMB region
covered*
(Mb)
Type of exonic
mutations included in
TMB estimation
Published
performance
characteristics
(ref.)
ACT Genomics ACTOnco+ 440 1.8 1.12 Non- synonymous†,
synonymous
NA
AstraZeneca AZ600 607 1.72 1.72 Non- synonymous,
synonymous
NA
Caris SureSelect XT 592 1.60 1.40 Non- synonymous Vanderwalde et al40
Foundation
Medicine
FoundationOne CDx‡ 324 2.20 0.80 Non- synonymous,
synonymous
Frampton et al41
Chalmers et al25
Fabrizio et al42
US FDA SSED31
Guardant Health GuardantOMNI§ 500 2.15 1.00 Non- synonymous,
synonymous
Quinn et al43
Illumina TSO500 (TruSight Oncology
500)
523 1.97 1.33 Non- synonymous,
synonymous
NA
Memorial Sloan
Kettering Cancer
Center
MSK- IMPACT¶ 468 1.53 1.14 Non- synonymous Cheng et al,44 Zehir et
al,30 US FDA32
NeoGenomics NeoTYPE Discovery Prole
for Solid Tumors
372 1.10 1.03 Non- synonymous,
synonymous
NA
Personal Genome
Diagnostics
PGDx elio tissue complete 507 2.20 1.33 Non- synonymous,
synonymous
Wood et al45
QIAGEN QIAseq TMB panel 486 1.33 1.33 Non- synonymous,
synonymous
NA
Thermo Fisher
Scientic
Oncomine Tumor Mutation
Load Assay
409 1.70 1.20 Non- synonymous Chaudhary et al46
Endris et al35
*Coding region used to estimate TMB regardless of the size of the region assessed by the panel.
†Non- synonymous mutations include single nucleotide variants, splice- site variants and short insertions and deletions (indels).
‡FoundationOne CDx assay has been approved by the US FDA as an IVD.31
§GuardantOMNI is a plasma- based circulating tumor DNA assay.
¶MSK- IMPACT assay has been authorized by the US FDA32
NA, not available.
had different sensitivity levels, the consortium decided
to use the Uniform TMB Calculation Method, which
would enable the selection of high- quality variants that
all laboratories were able to assess as part of their panels.
The consortium created a custom bed file covering
32.102 Mb of the genome which was used to calculate the
reference WES TMB values. The calculated WES TMB
values comprised the reference dataset for this study.
The uniform method for analysis of WES TMB included
minimum thresholds for median target coverage (median
300X as this was identified as the point where sensitivity
for the lower allele frequency variants drops drasti-
cally) (see online supplementary figure 4), variant allele
frequency (≥0.05), read depth (≥25) and variant count
(≥3), and synonymous variants were excluded.
Panel analysis
Each participating laboratory calculated TMB from the
subset of the exome restricted to the genes covered by
their targeted panel and using their own unique bioinfor-
matics pipeline (panel TMB). If available, the laborato-
ry’s bioinformatics analysis has been reported in table 1.
The panel- derived TMB datasets were sent to a neutral
third party (DMM) who assigned coded identifiers to the
laboratories to mask which laboratory contributed each
dataset. All subsequent data analyses were conducted by
LMMS and DMM. Participating laboratories were not
involved in the analyses and were not provided the key to
the coded lab identifiers.
RESULTS
In silico assessment of theoretical TMB variation across
panels
Eleven academic and commercial laboratories with
targeted gene panels in different stages of development
participated in this study (table 1). The size of the coding
region used to estimate TMB from these gene panels
ranged between 0.80 and 1.72 Mb. And the number of
genes in each of the gene panels ranged between 324 and
607 genes. All participating laboratories included exonic
somatic non- synonymous, frameshift and splice site vari-
ants and short indels when estimating TMB. Eight panels
(8/11, 73%) also included synonymous variants in their
on March 26, 2020 by guest. Protected by copyright.http://jitc.bmj.com/J Immunother Cancer: first published as 10.1136/jitc-2019-000147 on 26 March 2020. Downloaded from

5
MerinoDM, etal. J Immunother Cancer 2020;8:e000147. doi:10.1136/jitc-2019-000147
Open access
Figure 1 Estimated regression lines for panel tumor mutational burden (TMB) as a function of whole exome sequencing
(WES) TMB for each of the 11 participating laboratories analyzing (A) all cancer types combined and (B) stratum 1 cancer types
combined. Solid lines represent the tted regression lines. Red dashed line represents 45o line.
estimation. Each laboratory used their own bioinfor-
matics algorithms and workflows, which were optimized
using the sequencing methods, mutation types and filters
that best suited their own panel specifications. Since the
participating panels were in different stages of develop-
ment, only a few had published panel performance char-
acteristics (table 1).
The WES TMB values were calculated using the TCGA
MC3 Mutation Annotation Format (MAF) validation
dataset and an agreed on methodology (see ‘Whole
exome analysis’ section and online supplementary table
2). The panel TMB values on the same validation dataset
were estimated by down- sampling to the regions covered
by each of the laboratories’ panels and applying their own
bioinformatics algorithms. To prevent the misinterpre-
tation of this study’s results as an interlab performance
study, all laboratories agreed for the results to be blinded
with respect to the lab generating each dataset.
First, all 32 cancer types in the TCGA MC3 dataset
were investigated together using weighted linear regres-
sion analysis (generalized least squares, see ‘Methods’
section). Some variation was observed across panels, with
Spearman’s rank correlation values (R) ranging from
0.79 to 0.88, and slope values ranging from 0.87 to 1.47
(figure 1A, online supplementary figure 5). Eight labo-
ratories (73%) had slope values >1, demonstrating an
overestimation of TMB. Panel factors that may influence
TMB overestimation were not assessed due to the blinded
study design but may have included the type of mutations
counted for the panel TMB value (eg, synonymous alter-
ations included in panel TMB that were excluded from
the WES estimation), among others.
In silico assessment of theoretical TMB variation across
panels by cancer type
A limitation of analyzing all cancer types together is the
variable distribution of TMB across different cancer types,
with some cancer types displaying large dynamic ranges of
TMB values up to several hundred mutations per Mb and
others with very limited distributions with very few samples
reaching 20 mutations per Mb (see online supplementary
figure 3). To account for this limitation, cancer types were
categorized into strata by their distribution of WES TMB
values. Stratum 1 (n=1563 samples with <40 mut/Mb)
had samples with a good distribution of WES TMB values
covering 0–40 mut/Mb, which enabled a more robust
regression analysis across a clinically relevant TMB range.
The eight cancer types in stratum 1 were: bladder urothe-
lial carcinoma (BLCA, n=195), colon adenocarcinoma
(COAD, n=128), head and neck squamous cell carcinoma
(HNSC, n=232), lung adenocarcinoma (LUAD, n=228),
lung squamous cell carcinoma (LUSC, n=228), skin cuta-
neous melanoma (SKCM, n=166), stomach adenocarci-
noma (STAD, n=189) and uterine corpus endometrial
carcinoma (UCEC, n=197).
Regression analyses restricted to stratum 1 tumors
revealed an association between WES TMB and panel
TMB similar to that for all cancer types analyzed together
(Spearman’s R: 0.81–0.90 and slope 0.80–1.32, figure 1B,
per laboratory online supplementary figure 6 and table
3). The slopes calculated when stratum 1 tumors were
analyzed were consistently lower than when all cancers
were analyzed. The greatest differences in slope values
when comparing slopes estimated for all cancers and
for stratum 1 tumors only, were observed for labs 8 (all
cancers 1.47 vs stratum 1 1.32) and 9 (all cancers 1.24 vs
stratum 1 1.1) (both ∆ 0.15), while labs 4 (all cancers 0.904
vs stratum 1 0.897) and 2 (all cancers 1.087 vs stratum 1
1.076) had the least differences (∆ 0.007 and 0.01, respec-
tively). When stratum 1 tumors were analyzed, only six
laboratories (55%) reported overestimation of TMB with
slope values >1.
Regression analyses with stratum 2 and 3 were not
robust, as the WES TMB values did not adequately cover
the entire clinically meaningful range (see online supple-
mentary figures 7 and 8, and table 3).
Lastly, the eight cancers in stratum 1 were analyzed sepa-
rately. UCEC, BLCA and COAD had the broadest range of
slope values (UCEC: range 0.755–1.602, ∆ 0.847; BLCA:
range 1.042–1.79, ∆ 0.748; COAD: range 0.75–1.486, ∆
0.736) (figure 2, online supplementary table 4), and most
laboratories consistently overestimated these cancer types,
with BLCA as the only cancer type for which all 11 labora-
tories (100%) consistently overestimated their panel TMB
values relative to WES TMB. Conversely, LUAD, LUSC
and HNSC had the tightest range of slope values with no
consistent bias to overestimating or underestimating TMB
on March 26, 2020 by guest. Protected by copyright.http://jitc.bmj.com/J Immunother Cancer: first published as 10.1136/jitc-2019-000147 on 26 March 2020. Downloaded from

6MerinoDM, etal. J Immunother Cancer 2020;8:e000147. doi:10.1136/jitc-2019-000147
Open access
Figure 2 Estimated regression lines for panel tumor mutational burden (TMB) as a function of whole exome sequencing (WES)
TMB for the eight cancer types within stratum 1. All cancer types had a good distribution of WES TMB values from 0 to 40 mut/
Mb. Solid lines represent the tted regression lines. Red dashed line represents 45o line. BLCA, bladder urothelial carcinoma;
COAD, colon adenocarcinoma; HNSC, head and neck squamous cell carcinoma; LUAD, lung adenocarcinoma ; LUSC,
lung squamous cell carcinoma; SKCM, skin cutaneous melanoma; STAD, stomach adenocarcinoma; UCEC, uterine corpus
endometrial carcinoma.
(LUAD: range 0.817–1.135, ∆ 0.318; LUSC: range 0.741–
1.099, ∆ 0.358; HNSC: range 0.854–1.244, ∆ 0.39).
Dening the theoretical variation in TMB across panels and by
cancer type
Prediction limits for the observed panel TMB at fixed
WES TMB (5, 10, 15 and 20 mut/Mb) were calculated
to quantify the variability around the regression line at
those selected WES TMB values. The limits were designed
to capture approximately 95% of the panel TMB values
expected to be observed at a given WES TMB. Some
laboratories had consistently tighter (narrower) predic-
tion intervals, while others demonstrated more variability
(wider intervals), but for all laboratories, the prediction
intervals became tighter with decreasing WES TMB value,
indicating greater variability in panel TMB at larger WES
TMB values (figure 3). Generally, the prediction intervals
observed for each participating laboratory were similar,
with laboratories demonstrating intervals that spanned as
small as ±4.7 mut/Mb or as large as ±12.3 mut/Mb when
the WES TMB was 10 mut/Mb, which is a TMB threshold
that has been previously used to define a TMB- high cohort
using NGS panels.33 When prediction limits were assessed
by strata, the variability of the intervals was very large for
cancer types in strata 2 and 3 compared with stratum 1
because most TMB values for the cancers in these strata
accumulate in the lower end of the TMB spectrum, thus
resulting in more uncertainty in the fitted regression lines
and wide scatter in panel TMB values around those lines
(see online supplementary figure 9). When the eight
stratum 1 cancers were analyzed separately, prediction
intervals at the discreet value of WES TMB=10 mut/Mb
were observed to be wider for BLCA and UCEC, while
LUAD, LUSC, HNSC and SKCM had the tightest inter-
vals (figure 4). This is similar to the observed variation
in fitted regression lines for BLCA and UCEC across
laboratories (figure 2). The theoretical variability around
the regression was also seen to increase (wider intervals)
with increasing TMB value in individual cancer types (see
online supplementary figure 10).
DISCUSSION
Eleven laboratories with distinct NGS targeted gene
panels and bioinformatic approaches participated in
phase I of the Friends of Cancer Research TMB Harmo-
nization Project and provided early insights into the theo-
retical variability across different targeted gene panels
on March 26, 2020 by guest. Protected by copyright.http://jitc.bmj.com/J Immunother Cancer: first published as 10.1136/jitc-2019-000147 on 26 March 2020. Downloaded from

7
MerinoDM, etal. J Immunother Cancer 2020;8:e000147. doi:10.1136/jitc-2019-000147
Open access
Figure 3 Ninety- ve per cent prediction intervals for panel tumor mutational burden (TMB) estimated at discreet whole exome
sequencing (WES) TMB values (5, 10, 15 and 20 mut/Mb), by laboratory across all laboratories. Blue arrows represent the
estimated mean panel TMB for each laboratory. Red dashed line represents the discreet WES TMB value at which prediction
interval is calculated.
that estimate TMB. The goal of the first phase of the
project was to describe the variability in TMB estimates
across several uniquely designed panel- based diagnostic
assays and to further elucidate the theoretical variation
in TMB quantification using an in silico approach with
a large publicly available dataset with high- quality reads
and a common reference TMB standard calculated from
the entire exome. Moreover, dependence of the associa-
tion between panel TMB and WES TMB on cancer type
was investigated.
Variability in panel TMB across different panels was
observed, with some panels consistently overestimating
or underestimating TMB, suggesting that panel size and
composition, as well as laboratories’ bioinformatics algo-
rithms, including types of mutations counted and variant
filters used in the TMB calculation, were likely contribu-
tors to the differences. Because of the blinded design of
this study, the influence of these factors on panel TMB
variability was not evaluated in this early phase of the
project but will be assessed in the following empirical
phase to be reported subsequently. Additionally, other
studies have recently reported on the impact of panel
size, DNA input and variant filtering on panel- based TMB
estimates.35 39
The study evaluated a robust dataset containing 32
cancer types, with very few cases having TMB >40 mut/Mb
(n=69/4134, 1.7%), so it was not possible to robustly esti-
mate the association between panel TMB and WES TMB
for cases with values >40 mut/Mb. TMB data were thus
capped at 40 mut/Mb and a linear relationship was used
to model the relationship in that range. Factoring the
limited dynamic range of TMB values observed in some
cancer types, a subset of eight cancers was identified and
named stratum one for the primary analysis. Stratum 1
cancer types included lung, bladder, head and neck, skin,
colon, uterine and gastric cancers, all of which have been
shown to respond to immune checkpoint inhibitors. Eval-
uating these cancers separately revealed distinct levels
of variability in the association between panel TMB and
WES TMB across panels, with some cancer types having
less variability (eg, lung and head and neck cancers), and
some having greater variability (eg, uterine, bladder and
colon cancers). As our initial findings suggest that panels
may perform differently on certain cancer types, further
on March 26, 2020 by guest. Protected by copyright.http://jitc.bmj.com/J Immunother Cancer: first published as 10.1136/jitc-2019-000147 on 26 March 2020. Downloaded from

8MerinoDM, etal. J Immunother Cancer 2020;8:e000147. doi:10.1136/jitc-2019-000147
Open access
Figure 4 Ninety- ve per cent prediction intervals for panel TMB (x- axis) of stratum 1 tumor types at whole exome sequencing
tumor mutational burden (WES TMB) 10 mut/Mb by laboratory (y- axis). Blue arrows represent the estimated mean panel TMB
for each laboratory. Red dashed line indicates WES TMB value=10 mut/Mb. BLCA, bladder urothelial carcinoma; COAD, colon
adenocarcinoma; HNSC, head and neck squamous cell carcinoma; LUAD, lung adenocarcinoma ; LUSC, lung squamous cell
carcinoma; SKCM, skin cutaneous melanoma; STAD, stomach adenocarcinoma; UCEC, uterine corpus endometrial carcinoma.
work is required to understand the factors contributing to
any disease- specific TMB variability, and the relationships
beyond the analyzed TMB range. However, the composi-
tion of the panels’ genes, types of mutations counted or
methods used to train their respective TMB algorithms
could be future areas of focus.
Despite these cancer- dependent findings, our study
found that panel TMB values were strongly correlated
with WES TMB across laboratories. Additionally, the
calculated 95% prediction intervals permitted estimation
of the linear relationship between panel TMB and WES
TMB as well as quantification of the range in which 95%
of the observed TMB panel values would be expected to
fall for tumors with various fixed WES TMB values. This
provides a framework for understanding the theoretical
variability likely to be incurred in the clinical applica-
tion of TMB estimation across panels, but also suggests
that harmonization of TMB estimates could be achieved
through alignment using external reference materials.
There is still, however, much that can be done to improve
the reliability of using NGS panels for TMB estimation.
The selection of high- quality variants from the TCGA
MC3 dataset was used to assess the theoretical variability
of TMB across panels in this study ensuring the interpret-
ability of the findings where the assessment of variability
was limited to factors such as panel size and composition
or bioinformatics pipeline, instead of perceived differ-
ences regarding sensitivity and specificity of individual
variant calling. However, we acknowledge that in a clin-
ical setting the estimation of TMB from FFPE tissue may
introduce variants of lesser quality and panels should
aim to validate the sensitivity and specificity of individual
variant calling separately from TMB validation.
As TMB measurements are most likely to be impactful
in treatment decisions for stratum one cancer types,
including these tumors as part of a laboratory’s analytical
validation studies to achieve optimal accuracy and consis-
tency is critical. On the other hand, it is also important
to recognize that there are cancer types with generally
low TMB values that may have a few cases with high TMB
values that may benefit from reliable panel TMB results.
Moreover, because of the cancer type- dependent distri-
bution of TMB values, studies aiming to evaluate the clin-
ical utility of TMB and determine optimal TMB cut- offs
for treatment decisions may need to account for specific
cancer types. This would be consistent with a recent
on March 26, 2020 by guest. Protected by copyright.http://jitc.bmj.com/J Immunother Cancer: first published as 10.1136/jitc-2019-000147 on 26 March 2020. Downloaded from

9
MerinoDM, etal. J Immunother Cancer 2020;8:e000147. doi:10.1136/jitc-2019-000147
Open access
Table 2 Consensus recommendations for the standardization of analytical validation studies of targeted NGS panels that estimate TMB
Parameters Recommendations
Accuracy* ►Accuracy or agreement should be measured by comparing the TMB values generated by the assay requiring validation against reference TMB
values generated either from:
– A comparable companion diagnostic approved by a regulatory agency, such as the FDA, if available, OR
– A WES assay with validated performance characteristics and using an accepted WES TMB calculation method, such as the common method
reported in this study (see online supplementary table 1).
►The minimum number of samples used for evaluation of accuracy should be at least 30. Samples should have TMB values that span the entire
analytical range being investigated (0–40 mut/Mb is recommended currently).
►TMB as a continuous score: This analysis will characterize the analytical performance of the assay over the analytical range of interest per the
intended use.
– Quantication of performance based on TMB as a continuous variable should include an appropriate regression analysis and a scatter plot
showing the association between the panel and reference TMB values.
– Additionally, quantication of performance should examine the pointwise prediction intervals for panel TMB values as obtained from the
regression analysis, at predened reference TMB values. A description of the absolute deviation from the mean should also be reported.
►For TMB as a categorical call: accuracy should be analytically validated using a single or multiple discreet TMB cut- off values within the analytical
range of interest.
– Quantication of performance of TMB as a categorical call should be based on 2×2 agreement tables to inform the positive per cent agreement,
negative per cent agreement and overall per cent agreement informed by 2×2 agreement tables.
– For assays pursuing a companion diagnostic claim, accuracy should be examined using a predetermined discrete cut- off value investigated in a
study using clinical samples covering the spectrum of conditions from a dened, or intent- to- treat (ITT) population. If the ITT population includes
multiple cancer types, stratied analyses should be conducted.
►Alteration level agreement: as a supplemental analysis, the agreement between alteration level calls (including single nucleotide variants (SNVs)
and short indels) between the platform and the reference should be provided for each alteration included in the TMB panel and restricted to the
overlapping genomic regions between the two assays.
– Report the concordance of variant calls between variants identied by WES and panel as a function of the panel variant allele frequency (VAF).
►Characterize the percentage of tests passing QC by reporting rst pass acceptability rate and overall acceptability rate (after samples have been
retested, if necessary).
Continued
on March 26, 2020 by guest. Protected by copyright.http://jitc.bmj.com/J Immunother Cancer: first published as 10.1136/jitc-2019-000147 on 26 March 2020. Downloaded from

10 MerinoDM, etal. J Immunother Cancer 2020;8:e000147. doi:10.1136/jitc-2019-000147
Open access
Parameters Recommendations
Precision* ►Precision should be evaluated using several samples. For each sample, separate analyses should be performed as described in the TMB as a
continuous score and TMB as a categorical call sections below.
►Analytical validation of precision of TMB as both a continuous score and a categorical call will improve reliability of TMB as a biomarker.
►Because TMB is a composite estimate composed of different variants, its precision should be evaluated as a composite score (mut/Mb).
►TMB as a continuous score:
– Precision studies of quantitative TMB estimates should evaluate the mean, SD and coefcient of variation of TMB values obtained from testing
aliquots of the same sample under stipulated precision conditions (eg, replicates, runs, instruments, lots, operators) for a range of samples (5–6
samples with 20 TMB results distributed across the precision conditions each) with TMB values within the analytical range (0–40 mut/Mb is
recommended currently), and include different levels of tumor content and VAF values.
– Note: identication of the TMB range to be evaluated should be guided by the most recently published clinically relevant studies.
– Precision of the TMB score should be estimated using a variance component analysis to estimate between- run, within- run, between instruments,
between lots and between operator SD for each sample.
– Quantication of performance should include calculation of repeatability and within- lab SD for each sample corresponding to several discreet
TMB values, given that a single average value of variation (eg, coefcient of variation pooled across several samples having different TMB levels)
may not best reect the changing variability across the TMB range.
►TMB as a categorical call: consistency of the categorical calls (eg, TMB high vs TMB low according to a discreet cut- off) should be evaluated based
on both repeatability and within- laboratory precision for the TMB results according to a single or multiple discreet cut- off values.
– For repeatability, calculate the per cent of TMB high calls (if majority call is high) or per cent of TMB low calls (if majority call is low) between
replicate samples tested under the same lab conditions.
– For within laboratory precision, calculate the per cent of TMB high calls (if majority call is high) or per cent of TMB low calls (if majority call is low)
and the mean TMB score from replicate samples tested under varying within- lab conditions.
– Note: the number of aliquots tested per sample should be sufcient to account for the various sources of assay variability, such as the ones
described above (TMB as a continuous score). Moreover, the number of samples tested should be similar.
– Emphasis should be placed on evaluating samples with TMB values:
– Signicantly below cut- off (approximates limit of blank, expect TMB low almost 100% of time).
– Near and below cut- off (expect TMB low 95% of time).
– Near and above cut- off (expect TMB high 95% of time).
– Signicantly above cut- off (expect very high TMB almost 100% of time).
►Alteration level precision: the evaluation of precision for each individual alteration call used to estimate TMB is not necessary but may be performed
as an exploratory analysis to provide insight into the mechanisms that contribute to the TMB score variability.
►Per cent tumor content should be collected when evaluating precision and reported, if applicable.
►Characterize the percentage of tests passing QC by reporting rst pass acceptability rate and overall acceptability rate (after samples have been
retested, if necessary).
Table 2 Continued
Continued
on March 26, 2020 by guest. Protected by copyright.http://jitc.bmj.com/J Immunother Cancer: first published as 10.1136/jitc-2019-000147 on 26 March 2020. Downloaded from

11
MerinoDM, etal. J Immunother Cancer 2020;8:e000147. doi:10.1136/jitc-2019-000147
Open access
Parameters Recommendations
Sensitivity* ►The impact of tumor content of a sample on the TMB categorical call (high, low) should be evaluated using multiple samples, taking into
consideration the precision of the TMB score as a function of decreasing tumor content.
– Undiluted samples should have a range of expected TMB scores, a range of VAF values for somatic mutations and a ratio of SNVs and indels
that are representative of clinical samples.
►The evaluation of panel sensitivity to tumor content should be done using:
– Samples: 6–10 undiluted samples where each sample is diluted to at least 5 levels of tumor content.
– Each sample should have a dilution series ranging from well above and below the expected sensitivity limits for tumor content.
– Note: It is likely that matched normal will be required to generate each respective dilution value. Consideration should be given to technical
and biological factors that may impact the choice of the normal sample and design of the dilution series.
– At each dilution level, at least 10 replicate samples should be tested.
►For each sample, the evaluation should include a calculation of the per cent of TMB high (above a predened TMB threshold) across replicates at
each dilution and a probit regression of per cent TMB high versus tumor content. From the regression, report the estimated tumor content where the
probability of detecting TMB high is 95%.
Limit of blank* ►Non- tumor samples should be used to establish the limit of blank for TMB, yielding results close to, but not always equal to 0 mut/Mb.
– Considerations should be given to technical and biological factors, such as age of patient and distance from tumor lesion, among others.
Percentage of
tests passing QC
►Report percentage of tests passing TMB QC metrics in routine testing.
►Example QC metrics for TMB might include: median exon coverage, coverage uniformity, contamination rate.
*The denitions of the terms used in this table are based on the Clinical and Laboratory Standards Institute Harmonized Terminology Database at http://htd.clsi.org/.
FDA, Food and Drug Administration; NGS, next- generation sequencing; QC, quality control; TMB, tumor mutational burden ; WES, whole exome sequencing .
Table 2 Continued
on March 26, 2020 by guest. Protected by copyright.http://jitc.bmj.com/J Immunother Cancer: first published as 10.1136/jitc-2019-000147 on 26 March 2020. Downloaded from

12 MerinoDM, etal. J Immunother Cancer 2020;8:e000147. doi:10.1136/jitc-2019-000147
Open access
report that found that in patients who received ICI, those
who had high TMB had longer survival than those who
had low TMB, but TMB- high cut- offs were cancer- type
dependent.20
The Friends of Cancer Research TMB Harmonization
Consortium includes the participation of several leading
commercial and academic laboratories as well as a diverse
group of stakeholders, who together identified oppor-
tunities for standardization to promote the harmoniza-
tion of TMB estimation. These have led the consortium
to recommend best practices for panel developers that
seek to promote consistency in alignment and facilitate
commutability across panels table 2. These recommenda-
tions revolve around the following three items.
1. Ensure reporting consistency: TMB should be reported in
mutations/megabase (mut/Mb)
The current practice of reporting WES- derived TMB
values as number of somatic mutations, while panel-
derived TMB values are reported as a density of somatic
mutations per Mb of genomic region covered by the
panel (mut/Mb), precludes the aggregate analysis of
TMB being derived from WES or targeted panels, espe-
cially since the size of the exome interrogated using
different platforms may not be consistent. Reporting
TMB as mutations/megabase (mut/Mb) in order to keep
these values consistent and comparable is recommended
by the consortium.
2. Analytical validation studies for TMB estimation should be
standardized to include assessment of analytical accuracy,
precision and sensitivity
Size of targeted gene panel, technical sensitivity of the
assay and pre- analytical and analytical variables are
known to contribute to variability in panel- based TMB
estimates.33 The same in silico data were used by every
participating laboratory, which created a theoretical
setting that focused the investigation on potential sources
of variability that are unique to the technical specifica-
tions of the panel (eg, size and composition), and the
bioinformatics approaches of each laboratory (eg, muta-
tion types counted and germline and hotspot mutation
filtering). Some of these factors cannot be easily modified
and standardized across laboratories as panel assays are,
for the most part, proprietary and have been designed
to optimize their respective technical specifications and
conditions. However, harmonization of TMB estimates
may be achieved across laboratories by ensuring that the
analytical validation studies for each panel follow a stan-
dard approach including alignment of panel TMB values
to an external reference standard. Recommendations for
analyzing accuracy, precision and sensitivity of TMB values
to tumor content when used both as a continuous score
and a categorical call have been proposed by the consor-
tium (table 2). These recommendations will ensure that
regardless of the type of panel or bioinformatics pipeline
a laboratory decides to use, TMB estimates are held to a
standard of acceptable reliability.
3. Consistency across panels could be ensured through
alignment of panel TMB values to WES-derived universal
reference standard
Comparison to WES TMB is currently the most recog-
nized way to determine accuracy of panel TMB. However,
it should be noted that differences in performance
between panel TMB and WES TMB are to be expected
based on differences in coverage depth between the two
methods, with typically greater depth and higher vari-
ability observed in panels.
Universal reference standards, with TMB values span-
ning a clinically relevant range (eg, 0–40 mut/Mb),
represent a promising tool to achieve alignment or cali-
bration in order to ensure consistency of the TMB esti-
mation across platforms, regardless of known sources of
variability. An ideal reference standard for TMB estima-
tion should be generated from a renewable source and its
TMB values should be calculated using WES. To mitigate
differences resulting from comparisons using multiple
different WES assays, a universally accepted, predefined
bioinformatics pipeline and statistical methods should
be implemented. A calibration curve generated using
the reference standard should be used to normalize and
compare across panels, which should promote alignment
and aid in the analytical validation of panel TMB values.
CONCLUSIONS
Harmonization of methodologies for the accurate
measurement of complex continuous biomarkers is an
ongoing effort. The Friends of Cancer Research TMB
Harmonization Project has convened key stakeholders
early in the development of NGS assays that estimate
TMB to more effectively identify avenues for the harmo-
nization of estimation approaches and to emphasize the
need for the uptake and implementation of these harmo-
nization recommendations. The results included in this
report are the initial results from this stepwise approach,
but future studies will focus on assessing the feasibility of
using tumor- derived cell lines as external reference stan-
dards to help facilitate alignment of panel TMB values.
Additional empirical analyses will also be conducted to
investigate the influence of biologic factors (eg, specimen
type, cancer type) and technical factors (eg, sequencing
technology) on panel TMB, continue refining best prac-
tices for panel assessment of TMB, and developing align-
ment approaches to improve interchangeability between
TMB estimates generated from different targeted gene
panels.
Lastly, the collaborative efforts of the TMB Harmoni-
zation Consortium will serve as a framework for future
harmonization initiatives that seek to standardize complex
quantitative biomarker assays and promote the reliability
of biomarker testing.
Author afliations
1Friends of Cancer Research, Washington, DC, USA
2National Cancer Institute, Bethesda, Maryland, USA
on March 26, 2020 by guest. Protected by copyright.http://jitc.bmj.com/J Immunother Cancer: first published as 10.1136/jitc-2019-000147 on 26 March 2020. Downloaded from

13
MerinoDM, etal. J Immunother Cancer 2020;8:e000147. doi:10.1136/jitc-2019-000147
Open access
3Foundation Medicine Inc, Cambridge, Massachusetts, USA
4NeoGenomics Laboratories, Aliso Viejo, California, USA
5ACT Genomics, Taipei, Taiwan
6Resphera Biosciences, Baltimore, Maryland, USA
7Clinical Genomics, Illumina Inc, San Diego, California, USA
8Bristol- Myers Squibb Co, Princeton, New Jersey, USA
9Translational Medicine, Oncology Research and Early Development, AstraZeneca
Pharmaceuticals LP, Boston, Massachusetts, USA
10Clinical Sequencing Division, Thermo Fisher Scientic, Ann Arbor, Michigan, USA
11Molecular Characterization Laboratory, Frederick National Laboratory for Cancer
Research, Frederick, Maryland, USA
12Caris Life Sciences Inc, Phoenix, Arizona, USA
13Personal Genome Diagnostics, Baltimore, Maryland, USA
14QIAGEN Inc, Aarhus, Denmark
15Memorial Sloan Kettering Cancer Center, New York, New York, USA
16Bioinformatics, Guardant Health Inc, Redwood City, California, USA
17Quality in Pathology (QuIP), Berlin, Germany
18QIAGEN Inc, Waltham, Massachusetts, USA
19Division of Hematology/Oncology, Department of Medicine, Columbia University,
New York, New York, USA
20AstraZeneca Pharmaceuticals LP, Waltham, Massachusetts, USA
21Institute of Pathology, University Hospital Heidelberg, Heidelberg, Baden-
Württemberg, Germany
Twitter Diana M Merino @d2merino
Acknowledgements The authors would like to thank all members of the Friends
of Cancer Research TMB Harmonization Consortium for their support and scientic
contributions to the study design and interpretation. The authors would also like to
thank Dr Alexander Lazar for helpful discussions.
Contributors DMM: conception, design, acquisition, analysis, interpretation, draft
and manuscript preparation. LMMS: design, acquisition, analysis, interpretation,
draft and manuscript preparation. DF: conception, design, acquisition, analysis,
interpretation, draft and manuscript preparation. VF: analysis, interpretation,
draft and manuscript preparation. S- JC: analysis, interpretation, draft and
manuscript preparation. JRW: design, acquisition, analysis, interpretation, draft
and manuscript preparation. PW: analysis, interpretation, draft and manuscript
preparation. JB: analysis, interpretation, draft and manuscript preparation. JCB:
conception, design, acquisition, analysis, interpretation, draft and manuscript
preparation. RC: analysis, interpretation, draft and manuscript preparation.
LC: analysis, interpretation, draft and manuscript preparation. WSC: analysis,
interpretation, draft and manuscript preparation. J- HC: analysis, interpretation,
draft and manuscript preparation. DC: analysis, interpretation, draft and manuscript
preparation. JSD: analysis, interpretation, draft and manuscript preparation. VG:
analysis, interpretation, draft and manuscript preparation. MH: conception, design,
interpretation, draft and manuscript preparation. EH: analysis, interpretation,
draft and manuscript preparation. YL: design, acquisition, analysis, interpretation,
draft and manuscript preparation. JM: conception, design, interpretation, draft
and manuscript preparation. AP: analysis, interpretation, draft and manuscript
preparation. RP: analysis, interpretation, draft and manuscript preparation. KJQ:
analysis, interpretation, draft and manuscript preparation. NR: conception, design,
interpretation, draft and manuscript preparation. HT: analysis, interpretation,
draft and manuscript preparation. CW: conception, design, acquisition, analysis,
interpretation, draft and manuscript preparation. MX: analysis, interpretation, draft
and manuscript preparation. AZ: analysis, interpretation, draft and manuscript
preparation. CZ: analysis, interpretation, draft and manuscript preparation.
MD: conception, design, interpretation, draft and manuscript preparation.
AS: conception, design, interpretation, draft and manuscript preparation. MS:
conception, design, acquisition, analysis, interpretation, draft and manuscript
preparation. JA: conception, design, acquisition, analysis, interpretation, draft and
manuscript preparation. All authors have approved the submitted version of the
manuscript.
Funding The Friends initiative uses a distributed research model and costs
incurred are borne by each participating organization. Additional sources of funding
were provided by AstraZeneca; Bristol- Myers Squibb; EMD Serono; Genentech and
Merck & Company Inc.
Competing interests DF: employment with Foundation Medicine and stockholder
in Roche. VF: employment with NeoGenomics Inc and stockholder in NeoGenomics
Inc. S- JC: employment with ACT Genomics and stockholder in ACT Genomics.
JRW: founder and owner of Resphira Biosciences and paid consultant to PGDx. PW:
employment with Illumina and stockholder in Illumina. JB: employment with Bristol-
Myers Squibb, shareholder in Bristol- Myers Squibb and shareholder in Johnson
& Johnson. JCB: employment with AstraZeneca Pharmaceuticals and stocks in
AstraZeneca Pharmaceuticals. RC: employment with Thermo Fisher Scientic.
WSC: employment with Caris Life Sciences. JHC: employment with ACT Genomics.
DC: employment with Thermo Fisher Scientic and stockholder in Thermo Fisher
Scientic. JSD: employment with Personal Genome Diagnostics. VG: employment
with QIAGEN. MH: received research funding from Bristol- Myers Squibb; is paid
a consultant to Merck, Bristol- Myers Squibb, AstraZeneca, Genentech/Roche,
Janssen, Nektar, Syndax, Mirati and Shattuck Labs; has received travel support/
honoraria from AztraZeneca and BMS and a patent has been led by MSK related
to the use of tumor mutation burden to predict response to immunotherapy (PCT/
US2015/062208), which has received licensing fees from PGDx. EH: employment
with Guardant Health Inc and stockholder in Guardant Health Inc. YL: employment
with Foundation Medicine while engaged in the research project (March 2019).
Currently, an employee of Thrive Sciences, Inc and stockholder of Thrive Sciences,
Inc. AP: employment with QIAGEN. KJQ: employment with Guardant Health Inc and
stockholder in Guardant Health Inc. NR: NR and Memorial Sloan Kettering Cancer
Center have a patent ling (PCT/US2015/062208) for the use of tumor mutation
burden and HLA for prediction of immunotherapy efcacy, which is licensed to
Personal Genome Diagnostics. NR is a founder, shareholder and serves on the
scientic advisory board of Gritstone Oncology. NR has also consulted for AbbVie,
AstraZeneca, BMS, EMD Sorono, Genentech, GSK, Janssen, Lilly, Merck, Novartis,
Pzer, Regeneron. HT: employment with Caris Life Sciences. CW: employment
with Bristol- Myers Squibb, shareholder in Bristol- Myers Squibb ad shareholder
in Johnson & Johnson. MX: employment with AstraZeneca Pharmaceuticals. CZ:
employment with Illumina and stockholder in Illumina. AS: advisory board and/or
speech honoraria from: Bayer, BMS, MSD, Novartis, AstraZeneca, Roche, Seattle
Genomics, Illumina, Thermo Fisher, Takeda. Research funding from: Chugai, BMS.
Patient consent for publication Not required.
Ethics approval As leaders in the forefront of the collection and analysis of human
tissue specimens, the TCGA formed the Ethics, Law and Policy Group to generate
and oversee policies related to ethical and logistical considerations of the samples
they have made publicly available. The TCGA Ethics and Policies was originally
published by the National Cancer Institute. The TCGA study received ethics approval
for conducting their research work. No additional ethics approval was requested to
use the publicly available TCGA data.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available in a public, open access
repository. The mutation le ( mc3. v0. 2. 8. PUBLIC. maf. gz) analyzed in this study was
downloaded from the National Cancer Institute Genomics Data Commons: https://
gdc. cancer. gov/ about- data/ publications/ pancanatlas. All datasets generated by the
TMB Harmonization Consortium as part of this Project have been made available at
https:// precision. fda. gov/. Interested users should make a precision. FDA account
and request access to the data.
Open access This is an open access article distributed in accordance with the
Creative Commons Attribution Non Commercial (CC BY- NC 4.0) license, which
permits others to distribute, remix, adapt, build upon this work non- commercially,
and license their derivative works on different terms, provided the original work is
properly cited, appropriate credit is given, any changes made indicated, and the use
is non- commercial. See http:// creativecommons. org/ licenses/ by- nc/ 4. 0/.
ORCID iD
Diana MMerino http:// orcid. org/ 0000- 0002- 6173- 6957
REFERENCES
1 Topalian SL, Drake CG, Pardoll DM. Immune checkpoint blockade:
a common denominator approach to cancer therapy. Cancer Cell
2015;27:450–61.
2 Sharma P, Allison JP. Immune checkpoint targeting in cancer
therapy: toward combination strategies with curative potential. Cell
2015;161:205–14.
3 Le DT, Uram JN, Wang H, etal. PD-1 blockade in tumors with
mismatch- repair deciency. N Engl J Med 2015;372:2509–20.
4 Delamarre L, Mellman I, Yadav M. Neo approaches to cancer
vaccines. Science 2015;348:760–1.
5 Brahmer JR. Immune checkpoint blockade: the hope for
immunotherapy as a treatment of lung cancer? Semin Oncol
2014;41:126–32.
on March 26, 2020 by guest. Protected by copyright.http://jitc.bmj.com/J Immunother Cancer: first published as 10.1136/jitc-2019-000147 on 26 March 2020. Downloaded from

14 MerinoDM, etal. J Immunother Cancer 2020;8:e000147. doi:10.1136/jitc-2019-000147
Open access
6 Rimm DL, Han G, Taube JM, etal. A prospective, multi- institutional,
Pathologist- Based assessment of 4 immunohistochemistry assays
for PD- L1 expression in non- small cell lung cancer. JAMA Oncol
2017;3:1051–8.
7 Snyder A, Makarov V, Merghoub T, etal. Genetic basis for clinical
response to CTLA-4 blockade in melanoma. N Engl J Med
2014;371:2189–99.
8 Van Allen EM, Miao D, Schilling B, etal. Genomic correlates of
response to CTLA-4 blockade in metastatic melanoma. Science
2015;350:207–11.
9 Hugo W, Zaretsky JM, Sun L, etal. Genomic and transcriptomic
features of response to anti- PD-1 therapy in metastatic melanoma.
Cell 2016;165:35–44.
10 Johnson DB, Pollack MH, Sosman JA. Emerging targeted therapies
for melanoma. Expert Opin Emerg Drugs 2016;21:195–207.
11 Riaz N, Havel JJ, Makarov V, etal. Tumor and microenvironment
evolution during immunotherapy with nivolumab. Cell
2017;171:934–49.
12 Rizvi NA, Hellmann MD, Snyder A, etal. Cancer immunology.
mutational landscape determines sensitivity to PD-1 blockade in
non- small cell lung cancer. Science 2015;348:124–8.
13 Kowanetz M, Zou W, Shames DS, etal. Tumor mutation load
assessed by FoundationOne (FM1) is associated with improved
efcacy of atezolizumab (atezo) in patients with advanced NSCLC.
Ann Oncol 2016;27:vi23.
14 Carbone DP, Reck M, Paz- Ares L, etal. First- Line nivolumab in
stage IV or recurrent non- small- cell lung cancer. N Engl J Med
2017;376:2415–26.
15 Rizvi H, Sanchez- Vega F, La K, etal. Molecular determinants
of response to anti- programmed cell death (PD)-1 and anti-
programmed death- ligand 1 (PD- L1) blockade in patients with
non- small- cell lung cancer proled with targeted next- generation
sequencing. J Clin Oncol 2018;36:633–41.
16 Rosenberg JE, Hoffman- Censits J, Powles T, etal. Atezolizumab in
patients with locally advanced and metastatic urothelial carcinoma
who have progressed following treatment with platinum- based
chemotherapy: a single- arm, multicentre, phase 2 trial. Lancet
2016;387:1909–20.
17 Le DT, Durham JN, Smith KN, etal. Mismatch repair deciency
predicts response of solid tumors to PD-1 blockade. Science
2017;357:409–13.
18 Goodman AM, Kato S, Bazhenova L, etal. Tumor mutational burden
as an independent predictor of response to immunotherapy in
diverse cancers. Mol Cancer Ther 2017;16:2598–608.
19 Cristescu R, Mogg R, Ayers M, etal. Pan- tumor genomic biomarkers
for PD-1 checkpoint blockade- based immunotherapy. Science
2018;362:eaar3593.
20 Samstein RM, Lee C- H, Shoushtari AN, etal. Tumor mutational load
predicts survival after immunotherapy across multiple cancer types.
Nat Genet 2019;51:202–6.
21 Hellmann MD, Nathanson T, Rizvi H, etal. Genomic features
of response to combination immunotherapy in patients with
advanced non- small- cell lung cancer. Cancer Cell 2018;33:843–52.
22 Hellmann MD, Ciuleanu T- E, Pluzanski A, etal. Nivolumab plus
ipilimumab in lung cancer with a high tumor mutational burden. N
Engl J Med 2018;378:2093–104.
23 Hellmann MD, Callahan MK, Awad MM, etal. Tumor mutational
burden and efcacy of nivolumab monotherapy and in
combination with ipilimumab in small- cell lung cancer. Cancer Cell
2018;33:853–61.
24 Peters S, Cho BC, Reinmuth N, etal. Tumor mutational burden (TMB)
as a biomarker of survival in metastatic non- small cell lung cancer
(mNSCLC): Blood and tissue TMB analysis from MYSTIC, a Phase III
study of rst- line durvalumab ± tremelimumab vs chemotherapy. In:
Proceedings of the American association for cancer research AACR
Anual meeting 2019. Atlanta, GA, 2019.
25 Chalmers ZR, Connelly CF, Fabrizio D, etal. Analysis of 100,000
human cancer genomes reveals the landscape of tumor mutational
burden. Genome Med 2017;9:1–14.
26 Johnson DB, Frampton GM, Rioth MJ, etal. Targeted next generation
sequencing identies markers of response to PD-1 blockade. Cancer
Immunol Res 2016;4:959–67.
27 Campbell BB, Light N, Fabrizio D, etal. Comprehensive analysis of
hypermutation in human cancer. Cell 2017;171:1042–56.
28 Fabrizio DA, George TJ, Dunne RF, etal. Beyond microsatellite
testing: assessment of tumor mutational burden identies subsets of
colorectal cancer who may respond to immune checkpoint inhibition.
J Gastrointest Oncol 2018;9:610–7.
29 Sorscher S, Desnoyers R. A patient with a microsatellite stable (MSS)
and high mutational burden metastatic colorectal cancer responding
to checkpoint inhibitor therapy. MOJ Clin Med Case Reports
2016;5:230–1.
30 Zehir A, Benayed R, Shah RH, etal. Mutational landscape of
metastatic cancer revealed from prospective clinical sequencing of
10,000 patients. Nat Med 2017;23:703–13.
31 US Food and Drug Administration. Summary of safety and
effectiveness data (SSED): FoundationOne CDxTM, 2017. Available:
https://www. accessdata. fda. gov/ cdrh_ docs/ pdf17/ P170019B. pdf
32 U.S. Food and Drug Administration. Evaluation Of Automatic Class
III Designation For MSK- IMPACT (Integrated Mutation Proling of
Actionable Cancer Targets) - Decision Summary, 2017. Available:
https://www. accessdata. fda. gov/ cdrh_ docs/ reviews/ den170058. pdf
33 Stenzinger A, Allen JD, Maas J, etal. Tumor mutational burden
standardization initiatives: recommendations for consistent
tumor mutational burden assessment in clinical samples to guide
immunotherapy treatment decisions. Genes Chromosom Cancer
2019;58:578–88.
34 Chan TA, Yarchoan M, Jaffee E, etal. Development of tumor
mutation burden as an immunotherapy biomarker: utility for the
oncology clinic. Ann Oncol 2019;30:44–56.
35 Endris V, Buchhalter I, Allgäuer M, etal. Measurement of tumor
mutational burden (TMB) in routine molecular diagnostics: in silico
and real- life analysis of three larger gene panels. Int J Cancer
2019;144:2303–12.
36 Stenzinger A, Endris V, Budczies J, etal. Harmonization and
standardization of panel- based tumor mutational burden (TMB)
measurement: real- world results and recommendations of the QuIP
study. J Thorac Oncol 2020. doi:10.1016/j.jtho.2020.01.023. [Epub
ahead of print: 28 Feb 2020].
37 Ellrott K, Bailey MH, Saksena G, etal. Scalable open science
approach for mutation calling of tumor Exomes using multiple
genomic pipelines. Cell Syst 2018;6:271–81.
38 Quinlan AR, Hall IM. BEDTools: a exible suite of utilities for
comparing genomic features. Bioinformatics 2010;26:841–2.
39 Buchhalter I, Rempel E, Endris V, etal. Size matters: dissecting key
parameters for panel- based tumor mutational burden analysis. Int J
Cancer 2019;144:1–11.
40 Vanderwalde A, Spetzler D, Xiao N, etal. Microsatellite instability
status determined by next- generation sequencing and compared
with PD- L1 and tumor mutational burden in 11,348 patients. Cancer
Med 2018;7:746–56.
41 Frampton GM, Fichtenholtz A, Otto GA, etal. Development
and validation of a clinical cancer genomic proling test
based on massively parallel DNA sequencing. Nat Biotechnol
2013;31:1023–31.
42 Fabrizio DA, Milbury C, Yip W- K, etal. Analytic validation of tumor
mutational burden as a companion diagnostic for combination
immunotherapy in non- small cell lung cancer. Annals of Oncology
2018;29:viii16–57.
43 Quinn K, Helman E, Nance T, etal. Development and analytical
validation of a plasma- based tumor mutational burden (TMB) score
from next- generation sequencing panels. Ann Oncol 2018;29:viii41.
44 Cheng DT, Mitchell TN, Zehir A, etal. Memorial sloan kettering-
integrated mutation proling of actionable cancer targets (MSK-
IMPACT): a hybridization capture- based next- generation sequencing
clinical assay for solid tumor molecular oncology. J Mol Diagn
2015;17:251–64.
45 Wood DE, White JR, Georgiadis A, etal. A machine learning
approach for somatic mutation discovery. Sci Transl Med
2018;10:eaar7939.
46 Chaudhary R, Quagliata L, Martin JP, etal. A scalable solution for
tumor mutational burden from formalin- xed, parafn- embedded
samples using the Oncomine tumor mutation load assay. Transl.
Lung Cancer Res 2018;7:616–30.
on March 26, 2020 by guest. Protected by copyright.http://jitc.bmj.com/J Immunother Cancer: first published as 10.1136/jitc-2019-000147 on 26 March 2020. Downloaded from