Distributed Medical Image Analysis and Diagnosis through Crowd-Sourced Games: A Malaria Case Study

Electrical Engineering Department, University of California Los Angeles, Los Angeles, California, United States of America.
PLoS ONE (Impact Factor: 3.23). 05/2012; 7(5):e37245. DOI: 10.1371/journal.pone.0037245
Source: PubMed


In this work we investigate whether the innate visual recognition and learning capabilities of untrained humans can be used in conducting reliable microscopic analysis of biomedical samples toward diagnosis. For this purpose, we designed entertaining digital games that are interfaced with artificial learning and processing back-ends to demonstrate that in the case of binary medical diagnostics decisions (e.g., infected vs. uninfected), with the use of crowd-sourced games it is possible to approach the accuracy of medical experts in making such diagnoses. Specifically, using non-expert gamers we report diagnosis of malaria infected red blood cells with an accuracy that is within 1.25% of the diagnostics decisions made by a trained medical professional.

Download full-text


Available from: Swati Padmanabhan
  • Source
    • "The function uploading images of ROIs to the cloud image storage service allows the user to store, manage, and share images online. It can also gather and merge observation results from multiple or off-site microscopes, showing feasibility for tele-diagnosis (Lehman and Gibson, 2013) or further applications such as crowd-sourced biomedical image analysis (Mavandadi et al., 2012). From these features, a time-lapse imaging method using a smartphone makes a microscopic experiment more flexible and simpler. "
    [Show abstract] [Hide abstract]
    ABSTRACT: A prototype system that replaces the conventional time-lapse imaging in microscopic inspection for use with smartphones is presented. Existing time-lapse imaging requires a video data feed between a microscope and a computer that varies depending on the type of image grabber. Even with proper hardware setups, a series of tedious and repetitive tasks is still required to relocate to the region-of-interest (ROI) of the specimens. In order to simplify the system and improve the efficiency of time-lapse imaging tasks, a smartphone-based platform utilizing microscopic augmented reality (μ-AR) markers is proposed. To evaluate the feasibility and efficiency of the proposed system, a user test was designed and performed, measuring the elapse time for a trial of the task starting from the execution of the application software to the completion of restoring and imaging of an ROI saved in advance. The results of the user test showed that the average elapse time was 65.3 ± 15.2 s with 6.86 ± 3.61 μm of position error and 0.08 ± 0.40 degrees of angle error. This indicates that the time-lapse imaging task was accomplished rapidly with a high level of accuracy. Thus, simplification of both the system and the task was achieved via our proposed system. Microsc. Res. Tech., 2014. © 2014 Wiley Periodicals, Inc.
    Full-text · Article · Apr 2014 · Microscopy Research and Technique
  • Source
    • "In the past few years, scientific researchers have begun to take an even more active and creative approach to public participation, using the ubiquity of the Internet and its interactive affordances to solve a nexus of problems [19-24]. For example, biochemists studying protein folding have teamed up with computer scientists to build an online game, Fold It, that invites public users to compete in devising solutions to protein folding problems that are then used to 'train' computers to improve their protein structure algorithms [25-28]. The website Zooniverse is a platform where scientists studying problems as diverse as galaxy formation, bat vocalization, and cancer can recruit the public to aid in massive, labor-intensive data coding efforts [29-32]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: This article provides a framework for disentangling the concept of participation, with emphasis on participation in genomic medicine. We have derived seven 'dimensions' of participation that are most frequently invoked in the extensive, heterogeneous literature on participation. To exemplify these dimensions, we use material from a database of 102 contemporary cases of participation--focusing here on cases specific to science and medicine. We describe the stakes of public participation in biomedical research, with a focus on genomic medicine and lay out the seven dimensions. We single out five cases of participation that have particular relevance to the field of genomic medicine, we apply the seven dimensions to show how we can differentiate among forms of participation within this domain. We conclude with some provocations to researchers and some recommendations for taking variation in participation more seriously.
    Full-text · Article · Jan 2014 · Genome Medicine
  • Source
    • "[1] In the biological sciences it has shown great potential in the determination of protein folding structure which has limited feasibility with conventional computational approaches. [2] In healthcare, crowdsourcing has been used in drug discovery, analysis of imaging, clinical diagnosis and to improve service efficiency [3]–[6]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Crowdsourcing is the process of outsourcing numerous tasks to many untrained individuals. Our aim was to assess the performance and repeatability of crowdsourcing for the classification of retinal fundus photography. One hundred retinal fundus photograph images with pre-determined disease criteria were selected by experts from a large cohort study. After reading brief instructions and an example classification, we requested that knowledge workers (KWs) from a crowdsourcing platform classified each image as normal or abnormal with grades of severity. Each image was classified 20 times by different KWs. Four study designs were examined to assess the effect of varying incentive and KW experience in classification accuracy. All study designs were conducted twice to examine repeatability. Performance was assessed by comparing the sensitivity, specificity and area under the receiver operating characteristic curve (AUC). Without restriction on eligible participants, two thousand classifications of 100 images were received in under 24 hours at minimal cost. In trial 1 all study designs had an AUC (95%CI) of 0.701(0.680-0.721) or greater for classification of normal/abnormal. In trial 1, the highest AUC (95%CI) for normal/abnormal classification was 0.757 (0.738-0.776) for KWs with moderate experience. Comparable results were observed in trial 2. In trial 1, between 64-86% of any abnormal image was correctly classified by over half of all KWs. In trial 2, this ranged between 74-97%. Sensitivity was ≥96% for normal versus severely abnormal detections across all trials. Sensitivity for normal versus mildly abnormal varied between 61-79% across trials. With minimal training, crowdsourcing represents an accurate, rapid and cost-effective method of retinal image analysis which demonstrates good repeatability. Larger studies with more comprehensive participant training are needed to explore the utility of this compelling technique in large scale medical image analysis.
    Full-text · Article · Aug 2013 · PLoS ONE
Show more