Analysis of a Multilevel Diagnosis Decision Support System and Its Implications: A Case Study

Centre for Plant Biotechnology and Genomics UPM-INIA, Polytechnic University of Madrid, Parque Científico y Tecnológico de la U.P.M. Campus de Montegancedo, Pozuelo de Alarcón, 28223 Madrid, Spain.
Computational and Mathematical Methods in Medicine (Impact Factor: 0.77). 12/2012; 2012(8):367345. DOI: 10.1155/2012/367345
Source: PubMed


Medical diagnosis can be performed in an automatic way with the use of computer-based systems or algorithms. Such systems are usually called diagnostic decision support systems (DDSSs) or medical diagnosis systems (MDSs). An evaluation of the performance of a DDSS called ML-DDSS has been performed in this paper. The methodology is based on clinical case resolution performed by physicians which is then used to evaluate the behavior of ML-DDSS. This methodology allows the calculation of values for several well-known metrics such as precision, recall, accuracy, specificity, and Matthews correlation coefficient (MCC). Analysis of the behavior of ML-DDSS reveals interesting results about the behavior of the system and of the physicians who took part in the evaluation process. Global results show how the ML-DDSS system would have significant utility if used in medical practice. The MCC metric reveals an improvement of about 30% in comparison with the experts, and with respect to sensitivity the system returns better results than the experts.

Download full-text


Available from: Giner Alor-Hernández, Aug 26, 2014
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: In this paper, we examine the model for a chemical exposure decision support algorithm. Our purpose is to suggest the model frame to describe possibility of exposure with low-dose VOC chemicals for long time under normal circumstances at working place. Forensic rhetoric terms, non-exclusion exposure suspicion (NES) and exclusion exposure suspicion (EES), were defined and various statistical methods were combined basis of Bayesian approach. Decisiontree (DT) methods of linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), and naïve Bayes model were evaluated to classify 3 VOCs (toluene, xylene, and ehtybenzene) by means of the results of urinary test, gene expression and methylation expression experiments. Overall procedure is conducted by leave-one-out cross-validation that error rate of NES resulted in 11%.
    Molecular and Cellular Toxicology 03/2013; 9(1). DOI:10.1007/s13273-013-0011-6 · 1.27 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: This paper proposes a new methodology for assessing the efficiency of medical diagnostic systems and clinical decision support systems by using the feedback/opinions of medical experts. The methodology behind this work is based on a comparison between the expert feedback that has helped solve different clinical cases and the expert system that has evaluated these same cases. Once the results are returned, an arbitration process is carried out in order to ensure the correctness of the results provided by both methods. Once this process has been completed, the results are analyzed using Precision, Recall, Accuracy, Specificity and Matthews Correlation Coefficient (MCC) (PRAS-M) metrics. When the methodology is applied, the results obtained from a real diagnostic system allow researchers to establish the accuracy of the system based on objective facts. The methodology returns enough information to analyze the system's behavior for each disease in the knowledge base or across the entire knowledge base. It also returns data on the efficiency of the different assessors involved in the evaluation process, analyzing their behavior in the diagnostic process. The proposed work facilitates the evaluation of medical diagnostic systems, having a reliable process based on objective facts. The methodology presented in this research makes it possible to identify the main characteristics that define a medical diagnostic system and their values, allowing for system improvement. A good example of the results provided by the application of the methodology is shown in this paper. A diagnosis system was evaluated by means of this methodology, yielding positive results (statistically significant) when comparing the system with the assessors that participated in the evaluation process of the system through metrics such as recall (+27.54%) and MCC (+32.19%). These results demonstrate the real applicability of the methodology used.
    Computers in Biology and Medicine 09/2013; 43(8):975-86. DOI:10.1016/j.compbiomed.2013.05.003 · 1.24 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Accurate and evidence-based diagnosis is a key step in clinical practice. High-quality diagnoses depend on several factors, including physician's training and experience. To assist physicians, medical diagnosis systems can be used, as part of clinical decision support systems (CDSS), to improve the accuracy of diagnoses, as well as inform the clinician regarding the bases of the diagnostic decisions in the context of prior knowledge. To support such CDSS systems, it is important to have accurate and well-formed knowledge bases with thoroughly annotated diagnostic criteria, as well as models for representing clinical observations that allow them to more easily be analyzed by expert-systems. We propose the use of Nan publications as a way to store provenance data related to the content of diagnostic knowledge bases, as well as the clinical diagnoses themselves. The primary goal is to be able to rigorously track the complete diagnostic process: from the knowledge base construction and its supporting evidence, to the clinical observations and the context within which they were made, through to the diagnosis itself, and the rationale behind it.
    2014 IEEE 27th International Symposium on Computer-Based Medical Systems (CBMS); 05/2014
Show more