Receiver-operating characteristic curve analysis in diagnostic, prognostic and predictive biomarker research

Department of Surgery, Stavanger University Hospital, Stavanger, Norway.
Journal of clinical pathology (Impact Factor: 2.55). 10/2008; 62(1):1-5. DOI: 10.1136/jcp.2008.061010
Source: PubMed

ABSTRACT From a clinical perspective, biomarkers may have a variety of functions, which correspond to different stages in the disease development, e.g. in the progression of cancer. Biomarkers can assist in the care of patients for screening, diagnosis, prognosis, prediction and surveillance. Fundamental for the use of biomarkers in all situations is biomarker accuracy - the ability to correctly classify one condition and/or outcome from another. Receiver-operating characteristic (ROC) curve analysis is a useful tool in assessment of biomarker accuracy. Its advantages include testing accuracy across the entire range of scores and thereby not requiring a predetermined cut-off point, in addition to easily examined visual and statistical comparisons across tests or scores, and, finally, independence from outcome prevalence. Further, ROC curve analysis is a useful tool for evaluating the accuracy of a statistical model that classifies subjects into one of two categories. Diagnostic models are different from predictive and prognostic models in that the latter incorporate time-to-event analysis, for which censored data may pose a weakness of the model, or the reference standard. However, with the appropriate use of ROC curves, investigators of biomarkers can improve their research and presentation of results. ROC curves help identify the most appropriate classification rules. ROC curves avoid confounding resulting from varying thresholds with subjective ratings. The ROC curve results should always be put in perspective, because a good classifier does not guarantee the eventual clinical outcome, in particular for time-dependant events in screening, prediction, and/or prognosis studies where particular statistical precautions and methods are needed.

  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Low molecular weight (LMW) respiratory sensitizers can cause occupational asthma but due to a lack of adequate test methods, prospective identification of respiratory sensitizers is currently not possible. This paper presents the evaluation of Structure-Activity Relationship models (SARs) as potential methods to prospectively conclude on the sensitization potential of LMW chemicals. The predictive performance of the SARs calculated from their training sets was compared to their performance on a dataset of newly identified respiratory sensitizers and non-sensitizers, derived from literature. The predictivity of the available SARs for new substances was markedly lower than their published predictive performance. For that reason, no single SAR model can be considered sufficiently reliable to conclude on potential LMW respiratory sensitization properties of a substance. The individual applicability domains of the models were analyzed for adequacies and deficiencies. Based on these findings, a tiered prediction approach is subsequently proposed. This approach combines the two SARs with the highest positive and negative predictivity taking into account model specific chemical applicability domain issues. The tiered approach provided reliable predictions for one third of the respiratory sensitizers and non-sensitizers of the external validation set compiled by us. For these chemicals, a positive predictive value of 96% and a negative predictive value of 89% was obtained. The tiered approach was not able to predict the other two thirds of the chemicals, meaning that additional information is required and that there is an urgent need for other test methods, e.g. in chemico or in vitro, to reach a reliable conclusion.
    Toxicological Sciences 09/2014; 142(2). DOI:10.1093/toxsci/kfu188 · 4.48 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Creatine kinase (CK) has been utilized as a diagnostic marker for Duchenne muscular dystrophy (DMD), but it correlates less well with the DMD pathological progression. In this study, we hypothesized that muscle-specific microRNAs (miR-1, -133 and -206) in serum may be useful for monitoring the DMD pathological progression, and explored the possibility of these miRNAs as potential non-invasive biomarkers for the disease. By using real-time quantitative reverse transcription-polymerase chain reaction (qRT-PCR) in a randomized and controlled trial, we detected that miR-1, -133 and -206 were significantly over-expressed in the serum of 39 children with DMD (up to 3.20 ± 1.20, 2(-ΔΔCt) ): almost 2- to 4- fold enriched in comparison to samples from the healthy controls (less than 1.15 ± 0.34, 2(-ΔΔCt) ). To determine whether these miRNAs were related to the clinical features of children with DMD, we analyzed the associations compared to CK. There were very good inverse correlations between the levels of these miRNAs, especially miR-206, and functional performances: high levels corresponded to low muscle strength, muscle function, and quality of life (QoL). Moreover, by receiver operating characteristic (ROC) curves analyses, we revealed that these miRNAs, especially miR-206, were able to discriminate DMD from controls. Thus, miR-206 and other muscle-specific miRNAs in serum are useful for monitoring the DMD pathological progression, and hence as potential non-invasive biomarkers for the disease. This article is protected by copyright. All rights reserved.
    Journal of Neurochemistry 01/2014; DOI:10.1111/jnc.12662 · 4.24 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: The association between DNA methylation status and embryogenic competency in oil palm tissue culture was examined through Representational Difference Analysis (RDA) approach, using methylation-sensitive restriction endonucleases. “Difference Products” (DPs) of RDA derived from palms of similar genetic backgrounds but exhibiting different embryogenesis rates during the regeneration process were isolated. The DPs were sequenced using a pyrosequencing platform. To our knowledge, this is the first study profiling partial HpaII methylation sites in oil palm young leaf tissues which are potentially associated with embryogenic amenability through a genomic subtractive approach. Quantitative real-time PCR analysis demonstrated that the methylation status of a novel fragment, EgNB3, was higher in highly embryogenic leaf explants compared to low embryogenesis rate materials. These differences are likely to be contributed by the 5′-mCCGG-3′ and/or 5′-mCmCGG-3′ methylation patterns. Our data suggest that the differentially methylated site in EgNB3 has potential as a molecular biomarker for the screening of oil palm leaf explants for their embryogenic potentials.
    Tree Genetics & Genomes 08/2013; 9(4):1099. DOI:10.1007/s11295-013-0625-9 · 2.44 Impact Factor