Breast US computer-aided diagnosis workstation: performance with a large clinical diagnostic population.
ABSTRACT To evaluate the performance of a computer-aided diagnosis (CAD) workstation in classifying cancer in a realistic data set representative of a clinical diagnostic breast ultrasonography (US) practice.
The database consisted of consecutive diagnostic breast US scans collected with informed consent with a protocol approved by the institutional review board and compliant with the HIPAA. Images from 508 patients with a total of 1046 distinct abnormalities were used. One hundred one patients had breast cancer. Results both for patients in whom the lesion abnormality was proved with either biopsy or aspiration (n = 183) and for all patients irrespective of biopsy status (n = 508) are presented. The ability of the CAD workstation to help differentiate malignancies from benign lesions was evaluated with a leave-one-out-by-case analysis. The clinical specificity of the radiologists for this dataset was determined according to the biopsy rate and outcome.
In the task of differentiating cancer from all other lesions sent to biopsy, the CAD workstation obtained an area under the receiver operating characteristic curve (AUC) value of 0.88, with 100% sensitivity at 26% specificity (157 cancers and 362 lesions total). The radiologists' specificity at 100% sensitivity for this set was zero. When analyzing all lesions irrespective of biopsy status, which is more representative of actual clinical practice, the CAD scheme obtained an AUC of 0.90 and 100% sensitivity at 30% specificity (157 cancers and 1046 lesions total). The radiologists' specificity at 100% sensitivity for this set was 77%.
Current levels of computer performance warrant a clinical evaluation of the potential of US CAD to aid radiologists in lesion work-up recommendations.
Article: Receiver operating characteristic analysis: a tool for the quantitative evaluation of observer performance and imaging systems.[show abstract] [hide abstract]
ABSTRACT: Receiver operating characteristic (ROC) analysis provides the most comprehensive description of diagnostic accuracy available to date, because it estimates and reports all of the combinations of sensitivity and specificity that a diagnostic test is able to provide. After sketching the 6 levels at which diagnostic efficacy can be assessed, this paper explains the conceptual foundations of conventional ROC analysis, describes a variety of indices that can be used to summarize ROC curves, and describes several forms of generalized ROC analysis that address situations in which more than 2 decision alternatives are available. Key issues that arise in ROC curve fitting and statistical testing are addressed in an intuitive way to provide a basis for judging the validity of ROC studies reported in the literature. A list of sources for free ROC software is provided. Receiver operating characteristic methodology has reached a level of maturity at which it can be recommended broadly as the approach of choice for radiologic imaging system comparisons.Journal of the American College of Radiology: JACR 07/2006; 3(6):413-22.
[show abstract] [hide abstract]
ABSTRACT: In this paper we present a computationally efficient segmentation algorithm for breast masses on sonography that is based on maximizing a utility function over partition margins defined through gray-value thresholding of a preprocessed image. The performance of the segmentation algorithm is evaluated on a database of 400 cases in two ways. Of the 400 cases, 124 were complex cysts, 182 were benign solid lesions, and 94 were malignant lesions. In the first evaluation, the computer-delineated margins were compared to manually delineated margins. At an overlap threshold of 0.40, the segmentation algorithm correctly delineated 94% of the lesions. In the second evaluation, the performance of our computer-aided diagnosis method on the computer-delineated margins was compared to the performance of our method on the manually delineated margins. Round robin evaluation yielded Az values of 0.90 and 0.87 on the manually delineated margins and the computer-delineated margins, respectively, in the task of distinguishing between malignant and nonmalignant lesions.Medical Physics 09/2001; 28(8):1652-9. · 2.83 Impact Factor
[show abstract] [hide abstract]
ABSTRACT: To develop and evaluate a two-stage computerized method that first detects suspicious regions on ultrasound images, and subsequently distinguishes among different lesion types. The first stage of detecting potential lesions was based on expected lesion shape and margin characteristics. After the detection stage, all candidate lesions were classified by a Bayesian neural net based on computer-extracted lesion features. Two separate tasks were performed and evaluated at the classification stage: the first classification task was the distinction between all actual lesions and false-positive detections; the second classification task was the distinction between actual cancer and all other detected lesion candidates (including false-positive detections). The neural nets were trained on a database of 400 cases (757 images), consisting of complex cysts and benign and malignant lesions, and tested on an independent database of 458 cases (1,740 images including 578 normal images). In the distinction between all actual lesions and false-positive detections, Az values of 0.94 and 0.91 were obtained with the training and testing data sets, respectively. Sensitivity by patient of 90% at 0.45 false-positive detections per image was achieved for this detection-plus-classification scheme for the testing data set. Distinguishing cancer from all other detections (false-positives plus all benign lesions) proved to be more challenging, and Az values of 0.87 and 0.81 were obtained during training and testing, respectively. Sensitivity by patient of 100% at 0.43 false-positive malignancies per image was achieved in the detection and classification of cancerous lesions for the testing dataset. The results show promising performance of the computerized lesion detection and classification method, and indicate the potential of such a system for clinical breast ultrasound.Academic Radiology 06/2004; 11(5):526-35. · 1.69 Impact Factor