Article

Effect of case selection on the performance of computer-aided detection schemes.

Kurt Rossmann Laboratories for Radiologic Image Research, Department of Radiology, University of Chicago, Illinois 60637.
Medical Physics (impact factor: 2.83). 03/1994; 21(2):265-9. pp.265-9
Source: PubMed

ABSTRACT The choice of clinical cases used to train and test a computer-aided diagnosis (CAD) scheme can affect the test results (i.e., error rate). In this study, we deliberately modified the components of our testing database to study the effects of this modification on measured performance. Using a computerized scheme for the automated detection of breast masses from mammograms, it was found that the sensitivity of the scheme ranged between 26% and 100% (at a false positive rate of 1.0 per image) depending on the cases used to test the scheme. Even a 20% change in the cases comprising the database can reduce the measured sensitivity by 15%-25%. Because of the strong dependence of measured performance on the testing database, it is difficult to estimate reliably the accuracy of a CAD scheme. Furthermore, it is questionable to compare different CAD schemes when different cases are used for testing. Sharing databases, creating a common database, or using a quantitative measure to characterize databases are possible solutions to this problem. However, none of these solutions exists or is practiced at present. Therefore, as a short-term solution, it is recommended that the method used for selecting cases, and histograms or mean and standard deviations of relevant image features be reported whenever performance data are presented.

0 0
 · 
0 Bookmarks
 · 
23 Views
  • Source
    Article: Colon polyp detection using smoothed shape operators: preliminary results.
    P Sundaram, A Zomorodian, C Beaulieu, S Napel
    [show abstract] [hide abstract]
    ABSTRACT: Computer-aided detection (CAD) algorithms identify locations in computed tomographic (CT) images of the colon that are most likely to contain polyps. Existing CAD methods treat the CT data as a voxelized, volume image. They estimate a curvature-based feature at the mucosal surface voxels. However, curvature is a smooth notion, while our data are discrete and noisy. As a second order differential quantity, curvature amplifies noise. In this paper, we present the smoothed shape operators method (SSO), which uses a geometry processing approach. We extract a triangle mesh representation of the colon surface, and estimate curvature on this surface using the shape operator. We then smooth the shape operators on the surface iteratively. Throughout, we use techniques explicitly designed for discrete geometry. All our computation occurs on the surface, rather than in the voxel grid. We evaluate our algorithm on patient data and provide free-response receiver-operating characteristic performance analysis over all size ranges of polyps. We also provide confidence intervals for our performance estimates. We compare our performance with the surface normal overlap (SNO) method for the same data. A preliminary evaluation of our method on 35 patients yielded the following results (polyp diameter range; sensitivity; false positives/case): (10mm; 100%; 17.5), (5-10 mm; 89.7%, 21.23), (<5 mm; 59.1%; 23.9) and (overall; 80.3%; 23.9). The evaluation of the SNO method yielded: (10 mm; 75%; 17.5), (5-10 mm; 43.1%; 21.23), (<5 mm; 15.9%; 23.9) and (overall; 38.5%; 23.9).
    Medical image analysis 05/2008; 12(2):99-119. · 3.09 Impact Factor
  • Source
    Article: Computer-aided detection of clustered microcalcifications on digitized mammograms: a robustness experiment.
    Y H Chang, B Zheng, D Gur
    [show abstract] [hide abstract]
    ABSTRACT: The authors assessed the performance of an existing computer-aided diagnosis (CAD) scheme for the detection of clustered microcalcifications in a large image database. A previously developed, rule-based system was used to assess detectability of microcalcification clusters in a set of 386 digitized mammograms with 239 verified clusters visible on 191 images. The test was performed without any reoptimization of the scheme. None of the 386 images had been used in any previous scheme development or testing procedures. The CAD scheme achieved 89.5% sensitivity at an average false-positive detection rate of 0.39 per image. In 75% of all images, no false-positive findings occurred. Twenty-three of 25 false-negative findings (misses) occurred during the last two stages in the detection process. This scheme produced reasonable results in a large data set of images with a large variety of cluster characteristics.
    Academic Radiology 07/1997; 4(6):415-8. · 1.69 Impact Factor
  • Source
    Article: Detecting sleep apnea by heart rate variability analysis: assessing the validity of databases and algorithms.
    María J Lado, Xosé A Vila, Leandro Rodríguez-Liñares, Arturo J Méndez, David N Olivieri, Paulo Félix
    [show abstract] [hide abstract]
    ABSTRACT: Obstructive sleep apnea (OSA) is a serious disorder caused by intermittent airway obstruction which may have dangerous impact on daily living activities. Heart rate variability (HRV) analysis could be used for diagnosing OSA, since this disease affects HRV during sleep. In order to validate different algorithms developed for detecting OSA employing HRV analysis, several public or proprietary data collections have been employed for different research groups. However, for validation purposes, it is obvious and evident the lack of a common standard database, worldwide recognized and accepted by the scientific community. In this paper, different algorithms employing HRV analysis were applied over diverse public and proprietary databases for detecting OSA, and the outcomes were validated in terms of a statistical analysis. Results indicate that the use of a specific database may strongly affect the performance of the algorithms, due to differences in methodologies of processing. Our results suggest that researchers must strongly take into consideration the database used when quoting their results, since selected cases are highly database dependent and would bias conclusions.
    Journal of Medical Systems 08/2011; 35(4):473-81. · 1.13 Impact Factor

Show more

Data provided are for informational purposes only. Although carefully collected, accuracy cannot be guaranteed. The impact factor represents a rough estimation of the journal's impact factor and does not reflect the actual current impact factor. Publisher conditions are provided by RoMEO. Differing provisions from the publisher's actual policy or licence agreement may be applicable.

Keywords

automated detection
 
breast masses
 
CAD scheme
 
cases
 
clinical cases
 
common database
 
computer-aided diagnosis
 
databases
 
different CAD schemes
 
different cases
 
error rate
 
measured sensitivity
 
performance data
 
quantitative measure
 
relevant image features
 
Sharing databases
 
short-term solution
 
solutions
 
strong dependence
 
testing database