Article

Some practical issues of experimental design and data analysis in radiological ROC studies.

Department of Radiology, University of Chicago, IL 60637.
Investigative Radiology (Impact Factor: 4.45). 04/1989; 24(3):234-45. DOI: 10.1097/00004424-198903000-00012
Source: PubMed

ABSTRACT Receiver operating characteristic (ROC) analysis has been used in a broad variety of medical imaging studies during the past 15 years, and its advantages over more traditional measures of diagnostic performance are now clearly established. But despite the essential simplicity of the approach, workers in the field often find--sometimes only after an ROC study is under way--that a number of subtle issues related to experimental design and data analysis must be confronted in practice. Many of these issues have not been discussed in the literature in detail, and most are not well known. The purposes of this paper are to make users of ROC methodology in medical imaging aware of potential problems that should be confronted before an ROC study is begun and to indicate, at least broadly, how those problems may be dealt with, given the present state of the art. Some of the issues raised here can be addressed adequately by easily prescribed techniques, whereas others remain difficult and will be resolved fully only by new methodologic developments.

7 Followers
 · 
176 Views
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Detective performance of radiologists for "obvious" targets should be evaluated by visual search task instead of ROC analysis, but visual task have not been applied to radiology studies. The aim of this study was to set up an environment that allows visual search task in radiology, to evaluate its feasibility, and to preliminarily investigate the effect of career on the performance. In a darkroom, ten radiologists were asked to answer the type of lesion by pressing buttons, when images without lesions, with bulla, ground-glass nodule, and solid nodule were randomly presented on a display. Differences in accuracy and reaction times depending on board certification were investigated. The visual search task was successfully and feasibly performed. Radiologists were found to have high sensitivity, specificity, positive predictive values and negative predictive values in non-board and board groups. Reaction time was under 1 second for all target types in both groups. Board radiologists were significantly faster in answering for bulla, but there were no significant differences for other targets and values. We developed an experimental system that allows visual search experiment in radiology. Reaction time for detection of bulla was shortened with experience.
    SpringerPlus 11/2013; 2:607. DOI:10.1186/2193-1801-2-607
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: For the multislice CT (MSCT) systems with a larger number of detector rows, it is essential to employ dose-reduction techniques. As reported in previous studies, edge-preserving adaptive image filters, which selectively eliminate only the noise elements that are increased when the radiation dose is reduced without affecting the sharpness of images, have been developed. In the present study, we employed receiver operating characteristic (ROC) analysis to assess the effects of the quantum denoising system (QDS), which is an edge-preserving adaptive filter that we have developed, on low-contrast resolution, and to evaluate to what degree the radiation dose can be reduced while maintaining acceptable low-contrast resolution. The low-contrast phantoms (Catphan 412) were scanned at various tube current settings, and ROC analysis was then performed for the groups of images obtained with/without the use of QDS at each tube current to determine whether or not a target could be identified. The tube current settings for which the area under the ROC curve (Az value) was approximately 0.7 were determined for both groups of images with/without the use of QDS. Then, the radiation dose reduction ratio when QDS was used was calculated by converting the determined tube current to the radiation dose. The use of the QDS edge-preserving adaptive image filter allowed the radiation dose to be reduced by up to 38%. The QDS was found to be useful for reducing the radiation dose without affecting the low-contrast resolution in MSCT studies.
    International Journal of Biomedical Imaging 02/2008; 2008:379486. DOI:10.1155/2008/379486
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: A local linear method for estimating the conditional ROC curve under the presence of continuous and categorical covariates is introduced. A data driven smoothing parameter selector based on the bootstrap is proposed. The methods are illustrated with real data from a discrimination problem emerging in the context of computer-aided diagnosis. The bootstrap approach is also used to construct pointwise confidence intervals for the area under the ROC curve. (c) 2007 Elsevier B.V. All rights reserved.
    Computational Statistics & Data Analysis 01/2008; 52(5-52):2623-2631. DOI:10.1016/j.csda.2007.09.013 · 1.15 Impact Factor