Why Your New Cancer Biomarker May Never Work: Recurrent Patterns and Remarkable Diversity in Biomarker Failures

Author's Affiliation: The Sidney Kimmel Comprehensive, Cancer Center at Johns Hopkins, Baltimore, Maryland.
Cancer Research (Impact Factor: 9.33). 11/2012; 72(23). DOI: 10.1158/0008-5472.CAN-12-3232
Source: PubMed


Less than 1% of published cancer biomarkers actually enter clinical practice. Although best practices for biomarker development are published, optimistic investigators may not appreciate the statistical near-certainty and diverse modes by which the other 99% (likely including your favorite new marker) do indeed fail. Here, patterns of failure were abstracted for classification from publications and an online database detailing marker failures. Failure patterns formed a hierarchical logical structure, or outline, of an emerging, deeply complex, and arguably fascinating science of biomarker failure. A new cancer biomarker under development is likely to have already encountered one or more of the following fatal features encountered by prior markers: lack of clinical significance, hidden structure in the source data, a technically inadequate assay, inappropriate statistical methods, unmanageable domination of the data by normal variation, implausibility, deficiencies in the studied population or in the investigator system, and its disproof or abandonment for cause by others. A greater recognition of the science of biomarker failure and its near-complete ubiquity is constructive and celebrates a seemingly perpetual richness of biologic, technical, and philosophical complexity, the full appreciation of which could improve the management of scarce research resources. Cancer Res; 72(23); 1-5. ©2012 AACR.

17 Reads
  • Source
    • "The resulting multi-gene prognostic biomarkers (sometimes called signatures) can identify patient subgroups that would be particularly likely to derive benefit from more intense therapy [5,6]. However, there have been numerous challenges in the development of clinically-useful biomarkers; most published biomarkers fail to enter routine clinical practice [7]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Background The reproducibility of transcriptomic biomarkers across datasets remains poor, limiting clinical application. We and others have suggested that this is in-part caused by differential error-structure between datasets, and their incomplete removal by pre-processing algorithms. Methods To test this hypothesis, we systematically assessed the effects of pre-processing on biomarker classification using 24 different pre-processing methods and 15 distinct signatures of tumour hypoxia in 10 datasets (2,143 patients). Results We confirm strong pre-processing effects for all datasets and signatures, and find that these differ between microarray versions. Importantly, exploiting different pre-processing techniques in an ensemble technique improved classification for a majority of signatures. Conclusions Assessing biomarkers using an ensemble of pre-processing techniques shows clear value across multiple diseases, datasets and biomarkers. Importantly, ensemble classification improves biomarkers with initially good results but does not result in spuriously improved performance for poor biomarkers. While further research is required, this approach has the potential to become a standard for transcriptomic biomarkers.
    BMC Bioinformatics 06/2014; 15(1):170. DOI:10.1186/1471-2105-15-170 · 2.58 Impact Factor
  • Source
    • "A miRNA signature trial has been recently announced (MRX34 trial). A key issue for miRNA signatures for cancer is their validation in diverse cohorts (Kern, 2012). Such task generates difficulties even though many public cohorts are available. "
    [Show abstract] [Hide abstract]
    ABSTRACT: MicroRNAs (miRNAs) play a key role in post-transcriptional regulation of RNAm levels. Their function in cancer has been studied by high throughput methods generating valuable sources of public information. Thus miRNA signatures predicting cancer clinical outcomes are emerging. An important step to propose miRNA-based biomarkers before clinical validation is their evaluation in independent cohorts. Although it can be carried out using public data, such task is time consuming, and requires a specialized analysis. Therefore to aid and simplify the evaluation of prognostic miRNA signatures in cancer, we developed SurvMicro, a freely and easy to use web tool that assesses miRNA signatures from publicly available miRNA profiles using multivariate survival analysis. SurvMicro is composed of a wide and updated database of more than 40 cohorts in different tissues and a web tool where survival analysis can be done in minutes. We presented evaluations to portrait the straightforward functionality of SurvMicro in Liver and Lung cancer. To our knowledge SurvMicro is the only bioinformatic tool that aids the evaluation of multivariate prognostic miRNA signatures in cancer. SurvMicro and its tutorial are freely available at
    Bioinformatics 02/2014; 30(11). DOI:10.1093/bioinformatics/btu087 · 4.98 Impact Factor
  • Source
    • "In one study, for example, samples from spatially separated sites within glioblastoma multiforme (GBM) tumors found that multiple molecular subtypes were present in all of the examined tumors [3]. It is clear that this molecular heterogeneity may significantly limit efforts to personalize cancer treatment based on the use of molecular profiling to identify druggable targets [7] [8] [9]. However, there has, thus far, been little effort to relate the spatial heterogeneity observed in clinical imaging with the genetic heterogeneity found in molecular studies. "
    [Show abstract] [Hide abstract]
    ABSTRACT: We examined pretreatment magnetic resonance imaging (MRI) examinations from 32 patients with glioblastoma multiforme (GBM) enrolled in The Cancer Genome Atlas (TCGA). Spatial variations in T1 post-gadolinium and either T2-weighted or fluid attenuated inversion recovery sequences from each tumor MRI study were used to characterize each small region of the tumor by its local contrast enhancement and edema/cellularity ("habitat"). The patient cohort was divided into group 1 (survival < 400 days, n = 16) and group 2 (survival > 400 days, n = 16). Histograms of relative values in each sequence demonstrated that the tumor regions were consistently divided into high and low blood contrast enhancement, each of which could be subdivided into regions of high, low, and intermediate cell density/interstitial edema. Group 1 tumors contained greater volumes of habitats with low contrast enhancement but intermediate and high cell density (not fully necrotic) than group 2. Both leave-one-out and 10-fold cross-validation schemes demonstrated that individual patients could be correctly assigned to the short or long survival group with 81.25% accuracy. We demonstrate that novel image analytic techniques can characterize regional habitat variations in GBMs using combinations of MRI sequences. A preliminary study of 32 patients from the TCGA database found that the distribution of MRI-defined habitats varied significantly among the different survival groups. Radiologically defined ecological tumor analysis may provide valuable prognostic and predictive biomarkers in GBM and other tumors.
    Translational oncology 02/2014; 7(1):5-13. DOI:10.1593/tlo.13730 · 2.88 Impact Factor
Show more