CT colonography with computer-aided detection as a second reader: observer performance study.
ABSTRACT To evaluate the effect of computer-aided detection (CAD) as second reader on radiologists' diagnostic performance in interpreting computed tomographic (CT) colonographic examinations by using a primary two-dimensional (2D) approach, with segmental, unblinded optical colonoscopy as the reference standard.
This HIPAA-compliant study was IRB-approved with written informed consent. Four board-certified radiologists analyzed 60 CT examinations with a commercially available review system. Two-dimensional transverse views were used for initial polyp detection, while three-dimensional (3D) endoluminal and 2D multiplanar views were available for problem solving. After initial review without CAD, the reader was shown CAD-identified polyp candidates. The readers were then allowed to add to or modify their original diagnoses. Polyp location, CT Colonography Reporting and Data System categorization, and reader confidence as to the likelihood of a candidate being a polyp were recorded before and after CAD reading. The area under the receiver operating characteristic (ROC) curve (AUC), sensitivity, and specificity were estimated for CT examinations with and without CAD readings by using multireader multicase analysis.
Use of CAD led to nonsignificant average reader AUC increases of 0.03, 0.03, and 0.04 for patients with adenomatous polyps 6 mm or larger, 6-9 mm, and 10 mm or larger, respectively (P > or = .25); likewise, CAD increased average reader sensitivity by 0.15, 0.16, and 0.14 for those respective groups, with a corresponding decrease in specificity of 0.14. These changes achieved significance for the 6 mm or larger group (P < .01), 6-9 mm group (P < .02), and for specificity (P < .01), but not for the 10 mm or larger group (P > .16). The average reading time was 5.1 minutes +/- 3.4 (standard deviation) without CAD. CAD added an average of 3.1 minutes +/- 4.3 (62%) to each reading (supine and prone positions combined); average total reading time, 8.2 minutes +/- 5.8.
Use of CAD led to a significant increase in sensitivity for detecting polyps in the 6 mm or larger and 6-9 mm groups at the expense of a similar significant reduction in specificity.
SourceAvailable from: Andrew Plumb[Show abstract] [Hide abstract]
ABSTRACT: We examined the design, analysis and reporting in multi-reader multi-case (MRMC) research studies using the area under the receiver-operating curve (ROC AUC) as a measure of diagnostic performance. We performed a systematic literature review from 2005 to 2013 inclusive to identify a minimum 50 studies. Articles of diagnostic test accuracy in humans were identified via their citation of key methodological articles dealing with MRMC ROC AUC. Two researchers in consensus then extracted information from primary articles relating to study characteristics and design, methods for reporting study outcomes, model fitting, model assumptions, presentation of results, and interpretation of findings. Results were summarized and presented with a descriptive analysis. Sixty-four full papers were retrieved from 475 identified citations and ultimately 49 articles describing 51 studies were reviewed and extracted. Radiological imaging was the index test in all. Most studies focused on lesion detection vs. characterization and used less than 10 readers. Only 6 (12%) studies trained readers in advance to use the confidence scale used to build the ROC curve. Overall, description of confidence scores, the ROC curve and its analysis was often incomplete. For example, 21 (41%) studies presented no ROC curve and only 3 (6%) described the distribution of confidence scores. Of 30 studies presenting curves, only 4 (13%) presented the data points underlying the curve, thereby allowing assessment of extrapolation. The mean change in AUC was 0.05 (-0.05 to 0.28). Non-significant change in AUC was attributed to underpowering rather than the diagnostic test failing to improve diagnostic accuracy. Data reporting in MRMC studies using ROC AUC as an outcome measure is frequently incomplete, hampering understanding of methods and the reliability of results and study conclusions. Authors using this analysis should be encouraged to provide a full description of their methods and results.PLoS ONE 12/2014; 9(12):e116018. DOI:10.1371/journal.pone.0116018 · 3.53 Impact Factor
[Show abstract] [Hide abstract]
ABSTRACT: Different methods of evaluating diagnostic performance when comparing diagnostic tests may lead to different results. We compared two such approaches, sensitivity and specificity with area under the Receiver Operating Characteristic Curve (ROC AUC) for the evaluation of CT colonography for the detection of polyps, either with or without computer assisted detection.PLoS ONE 10/2014; 9(10):e107633. DOI:10.1371/journal.pone.0107633 · 3.53 Impact Factor
[Show abstract] [Hide abstract]
ABSTRACT: In recent years, dual-energy computed tomography (DECT) has been widely used in the clinical routine due to improved diagnostics capability from additional spectral information. One promising application for DECT is CT colonography (CTC) in combination with computer-aided diagnosis (CAD) for detection of lesions and polyps. While CAD has demonstrated in the past that it is able to detect small polyps, its performance is highly dependent on the quality of the input data. The presence of artifacts such as beam-hardening and noise in ultra-low-dose CTC may severely degrade detection performances of small polyps. In this work, we investigate and compare virtual monochromatic images, generated by image-based decomposition and projection-based decomposition, with respect to CAD performance. In the image-based method, reconstructed images are firstly decomposed into water and iodine before the virtual monochromatic images are calculated. On the contrary, in the projection-based method, the projection data are first decomposed before calculation of virtual monochromatic projection and reconstruction. Both material decomposition methods are evaluated with regards to the accuracy of iodine detection. Further, the performance of the virtual monochromatic images is qualitatively and quantitatively assessed. Preliminary results show that the projection-based method does not only have a more accurate detection of iodine, but also delivers virtual monochromatic images with reduced beam hardening artifacts in comparison with the image-based method. With regards to the CAD performance, the projection-based method yields an improved detection performance of polyps in comparison with that of the image-based method.Proceedings - Society of Photo-Optical Instrumentation Engineers 02/2015; 9412. DOI:10.1117/12.2081982