Analysis of a Multilevel Diagnosis Decision Support System and Its Implications: A Case Study

Centre for Plant Biotechnology and Genomics UPM-INIA, Polytechnic University of Madrid, Parque Científico y Tecnológico de la U.P.M. Campus de Montegancedo, Pozuelo de Alarcón, 28223 Madrid, Spain.
Computational and Mathematical Methods in Medicine (Impact Factor: 0.77). 12/2012; 2012(8):367345. DOI: 10.1155/2012/367345
Source: PubMed


Medical diagnosis can be performed in an automatic way with the use of computer-based systems or algorithms. Such systems are usually called diagnostic decision support systems (DDSSs) or medical diagnosis systems (MDSs). An evaluation of the performance of a DDSS called ML-DDSS has been performed in this paper. The methodology is based on clinical case resolution performed by physicians which is then used to evaluate the behavior of ML-DDSS. This methodology allows the calculation of values for several well-known metrics such as precision, recall, accuracy, specificity, and Matthews correlation coefficient (MCC). Analysis of the behavior of ML-DDSS reveals interesting results about the behavior of the system and of the physicians who took part in the evaluation process. Global results show how the ML-DDSS system would have significant utility if used in medical practice. The MCC metric reveals an improvement of about 30% in comparison with the experts, and with respect to sensitivity the system returns better results than the experts.

Download full-text


Available from: Giner Alor-Hernández, Aug 26, 2014

  • No preview · Article · Jan 2013 · Foundations and Trends® in Web Science
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: In this paper, we examine the model for a chemical exposure decision support algorithm. Our purpose is to suggest the model frame to describe possibility of exposure with low-dose VOC chemicals for long time under normal circumstances at working place. Forensic rhetoric terms, non-exclusion exposure suspicion (NES) and exclusion exposure suspicion (EES), were defined and various statistical methods were combined basis of Bayesian approach. Decisiontree (DT) methods of linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), and naïve Bayes model were evaluated to classify 3 VOCs (toluene, xylene, and ehtybenzene) by means of the results of urinary test, gene expression and methylation expression experiments. Overall procedure is conducted by leave-one-out cross-validation that error rate of NES resulted in 11%.
    Full-text · Article · Mar 2013 · Molecular and Cellular Toxicology
  • [Show abstract] [Hide abstract]
    ABSTRACT: This paper proposes a new methodology for assessing the efficiency of medical diagnostic systems and clinical decision support systems by using the feedback/opinions of medical experts. The methodology behind this work is based on a comparison between the expert feedback that has helped solve different clinical cases and the expert system that has evaluated these same cases. Once the results are returned, an arbitration process is carried out in order to ensure the correctness of the results provided by both methods. Once this process has been completed, the results are analyzed using Precision, Recall, Accuracy, Specificity and Matthews Correlation Coefficient (MCC) (PRAS-M) metrics. When the methodology is applied, the results obtained from a real diagnostic system allow researchers to establish the accuracy of the system based on objective facts. The methodology returns enough information to analyze the system's behavior for each disease in the knowledge base or across the entire knowledge base. It also returns data on the efficiency of the different assessors involved in the evaluation process, analyzing their behavior in the diagnostic process. The proposed work facilitates the evaluation of medical diagnostic systems, having a reliable process based on objective facts. The methodology presented in this research makes it possible to identify the main characteristics that define a medical diagnostic system and their values, allowing for system improvement. A good example of the results provided by the application of the methodology is shown in this paper. A diagnosis system was evaluated by means of this methodology, yielding positive results (statistically significant) when comparing the system with the assessors that participated in the evaluation process of the system through metrics such as recall (+27.54%) and MCC (+32.19%). These results demonstrate the real applicability of the methodology used.
    No preview · Article · Sep 2013 · Computers in Biology and Medicine
Show more