Receiver-operating characteristic curve analysis in diagnostic, prognostic and predictive biomarker research

Department of Surgery, Stavanger University Hospital, Stavanger, Norway.
Journal of clinical pathology (Impact Factor: 2.55). 10/2008; 62(1):1-5. DOI: 10.1136/jcp.2008.061010
Source: PubMed

ABSTRACT From a clinical perspective, biomarkers may have a variety of functions, which correspond to different stages in the disease development, e.g. in the progression of cancer. Biomarkers can assist in the care of patients for screening, diagnosis, prognosis, prediction and surveillance. Fundamental for the use of biomarkers in all situations is biomarker accuracy - the ability to correctly classify one condition and/or outcome from another. Receiver-operating characteristic (ROC) curve analysis is a useful tool in assessment of biomarker accuracy. Its advantages include testing accuracy across the entire range of scores and thereby not requiring a predetermined cut-off point, in addition to easily examined visual and statistical comparisons across tests or scores, and, finally, independence from outcome prevalence. Further, ROC curve analysis is a useful tool for evaluating the accuracy of a statistical model that classifies subjects into one of two categories. Diagnostic models are different from predictive and prognostic models in that the latter incorporate time-to-event analysis, for which censored data may pose a weakness of the model, or the reference standard. However, with the appropriate use of ROC curves, investigators of biomarkers can improve their research and presentation of results. ROC curves help identify the most appropriate classification rules. ROC curves avoid confounding resulting from varying thresholds with subjective ratings. The ROC curve results should always be put in perspective, because a good classifier does not guarantee the eventual clinical outcome, in particular for time-dependant events in screening, prediction, and/or prognosis studies where particular statistical precautions and methods are needed.

  • [Show abstract] [Hide abstract]
    ABSTRACT: Necrotizing enterocolitis is an acute inflammatory disease, which primarily affects preterm infants, and is a leading cause of morbidity and mortality in the neonatal intensive care unit. Unfortunately, necrotizing enterocolitis can be difficult to distinguish from other diseases and clinical conditions especially during the early course of the disease. This diagnostic uncertainty is particularly relevant to clinical evaluation and medical management and potentially leads to unnecessary and extended periods of cessation of enteral feedings and prolonged courses of parenteral nutrition and antibiotics. Biomarkers are molecular indicators of a disease process, diagnosis, prognosis and can be used to monitor the effects of disease management. Historically, there has been a paucity of reliable and robust biomarkers for necrotizing enterocolitis. However, several studies have recently identified promising biomarkers. Noninvasive samples for biomarker measurement are preferred and may have certain advantages in the preterm infant. In this review article, we focus on recent exciting and promising discoveries in noninvasive biomarkers for necrotizing enterocolitis.
    The Journal of perinatal & neonatal nursing 01/2015; 29(1):69-80. DOI:10.1097/JPN.0000000000000082 · 1.01 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Diseases and death caused by exposure to tobacco smoke have become the single most serious preventable public health concern. Thus, biomarkers that can monitor tobacco exposure and health effects can play a critical role in tobacco product regulation and public health policy. Biomarkers of exposure to tobacco toxicants are well established and have been used in population studies to establish public policy regarding exposure to second-hand smoke, an example being the nicotine metabolite cotinine, which can be measured in urine. Biomarkers of biological response to tobacco smoking range from those indicative of inflammation to mRNA and microRNA patterns related to tobacco use and/or disease state. Biomarkers identifying individuals with an increased risk for a pathological response to tobacco have also been described. The challenge for any novel technology or biomarker is its translation to clinical and/or regulatory application, a process that requires first technical validation of the assay and then careful consideration of the context the biomarker assay may be used in the regulatory setting. Nonetheless, the current efforts to investigate new biomarker of tobacco smoke exposure promise to offer powerful new tools in addressing the health hazards of tobacco product use. This review will examine such biomarkers, albeit with a focus on those related to cigarette smoking. © 2014 Elsevier Inc. All rights reserved.
    Advances in clinical chemistry 01/2014; 67:1-45. DOI:10.1016/bs.acc.2014.09.001 · 4.30 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Binary classifiers are routinely evaluated with performance measures such as sensitivity and specificity, and performance is frequently illustrated with Receiver Operating Characteristics (ROC) plots. Alternative measures such as positive predictive value (PPV) and the associated Precision/Recall (PRC) plots are used less frequently. Many bioinformatics studies develop and evaluate classifiers that are to be applied to strongly imbalanced datasets in which the number of negatives outweighs the number of positives significantly. While ROC plots are visually appealing and provide an overview of a classifier's performance across a wide range of specificities, one can ask whether ROC plots could be misleading when applied in imbalanced classification scenarios. We show here that the visual interpretability of ROC plots in the context of imbalanced datasets can be deceptive with respect to conclusions about the reliability of classification performance, owing to an intuitive but wrong interpretation of specificity. PRC plots, on the other hand, can provide the viewer with an accurate prediction of future classification performance due to the fact that they evaluate the fraction of true positives among positive predictions. Our findings have potential implications for the interpretation of a large number of studies that use ROC plots on imbalanced datasets.
    PLoS ONE 03/2015; 10(3):e0118432. DOI:10.1371/journal.pone.0118432 · 3.53 Impact Factor