Accuracy assessment of global and local atrophy measurement techniques with realistic simulated longitudinal Alzheimer's disease images

Centre for Medical Image Computing, Department of Medical Physics and Bioengineering, University College London, WC1E 6BT, UK.
NeuroImage (Impact Factor: 6.13). 06/2008; 42(2):696-709. DOI: 10.1016/j.neuroimage.2008.04.259
Source: PubMed

ABSTRACT The evaluation of atrophy quantification methods based on magnetic resonance imaging have been usually hindered by the lack of realistic gold standard data against which to judge these methods or to help refine them. Recently [Camara, O., Schweiger, M., Scahill, R., Crum, W., Sneller, B., Schnabel, J., Ridgway, G., Cash, D., Hill, D., Fox, N., 2006. Phenomenological model of diffuse global and regional atrophy using finite-element methods. IEEE Trans. Med.l Imaging 25, 1417-1430], we presented a technique in which atrophy is realistically simulated in different tissue compartments or neuroanatomical structures with a phenomenological model. In this study, we have generated a cohort of realistic simulated Alzheimer's disease (AD) images with known amounts of atrophy, mimicking a set of 19 real controls and 27 probable AD subjects, with an improved version of our atrophy simulation methodology. This database was then used to assess the accuracy of several well-known computational anatomy methods which provide global (BSI and SIENA) or local (Jacobian integration) estimates of longitudinal atrophy in brain structures using MR images. SIENA and BSI results correlated very well with gold standard data (Pearson coefficient of 0.962 and 0.969 respectively), achieving small mean absolute differences with respect to the gold standard (percentage change from baseline volume): BSI of 0.23%+/-0.26%; SIENA of 0.22%+/-0.28%. Jacobian integration was guided by both fluid and FFD-based registration techniques and resulting deformation fields and associated Jacobians were compared, region by region, with gold standard ones. The FFD-based technique outperformed the fluid one in all evaluated structures (mean absolute differences from the gold standard in percentage change from baseline volume): whole brain, FFD=0.31%, fluid=0.58%; lateral ventricles, FFD=0.79%; fluid=1.45%; left hippocampus, FFD=0.82%; fluid=1.42%; right hippocampus, FFD=0.95%; fluid=1.62%. The largest errors for both local techniques occurred in the sulcal CSF (FFD=2.27%; fluid=3.55%) regions. For large structures such as the whole brain, these mean absolute differences, relative to the applied atrophy, represented similar percentages for the BSI, SIENA and FFD techniques (controls/patients): BSI, 51.99%/16.36%; SIENA, 62.34%/21.59%; FFD, 41.02%/24.95%. For small structures such as the hippocampi, these percentages were larger, especially for controls where errors were approximately equal to the small applied changes (controls/patients): FFD, 92.82%/43.61%. However, these apparently large relative errors have not prevented the global or hippocampal measures from finding significant group separation in our study. The evaluation framework presented here will help in quantifying whether the accuracy of future methodological developments is sufficient for analysing change in smaller or less atrophied local brain regions. Results obtained in our experiments with realistic simulated data confirm previously published estimates of accuracy for both evaluated global techniques. Regarding Jacobian Integration methods, the FFD-based one demonstrated promising results and potential for being used in clinical studies alongside (or in place of) the more common global methods. The generated gold standard data has also allowed us to identify some stages and sets of parameters in the evaluated techniques--the brain extraction step in the global techniques and the number of multi-resolution levels and the stopping criteria in the registration-based methods--that are critical for their accuracy.

  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Brain atrophy measured using structural MRI has been widely used as an imaging biomarker for disease diagnosis and tracking of pathological progression in neurodegenerative diseases. In this work, we present a generalised and extended formulation of the Boundary Shift Integral (gBSI) using probabilistic segmentations in order to estimate anatomical changes between 2 time points. This method adaptively estimates a non-binary XOR region-of-interest from probabilistic brain segmentations of the baseline and repeat scans, in order to better localise and capture the brain atrophy. We evaluate the proposed method by comparing the sample size requirements for a hypothetical clinical trial of Alzheimer’s disease to that needed for the current implementation of BSI as well as a fuzzy implementation of BSI. The gBSI method results in a modest, but reduced sample size, providing increased sensitivity to disease changes through the use of the probabilistic XOR region.
    Neurobiology of Aging 08/2014; 36. DOI:10.1016/j.neurobiolaging.2014.04.035 · 4.85 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Brain atrophy is considered an important marker of disease progression in many chronic neuro-degenerative diseases such as multiple sclerosis (MS). A great deal of attention is being paid toward developing tools that manipulate magnetic resonance (MR) images for obtaining an accurate estimate of atrophy. Nevertheless, artifacts in MR images, inaccuracies of intermediate steps and inadequacies of the mathematical model representing the physical brain volume change, make it rather difficult to obtain a precise and unbiased estimate. This work revolves around the nature and magnitude of bias in atrophy estimations as well as a potential way of correcting them. First, we demonstrate that for different atrophy estimation methods, bias estimates exhibit varying relations to the expected atrophy and these bias estimates are of the order of the expected atrophies for standard algorithms, stressing the need for bias correction procedures. Next, a framework for estimating uncertainty in longitudinal brain atrophy by means of constructing confidence intervals is developed. Errors arising from MRI artifacts and bias in estimations are learned from example atrophy simulations and anatomies. Results are discussed for three popular non-rigid registration approaches with the help of simulated localized brain atrophy in real MR images.
    Computerized medical imaging and graphics: the official journal of the Computerized Medical Imaging Society 08/2013; 37(7-8). DOI:10.1016/j.compmedimag.2013.07.002 · 1.50 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: SIENA and similar techniques have demonstrated the utility of performing "direct" measurements as opposed to post-hoc comparison of cross-sectional data for the measurement of whole brain (WB) atrophy over time. However, gray matter (GM) and white matter (WM) atrophy are now widely recognized as important components of neurological disease progression, and are being actively evaluated as secondary endpoints in clinical trials. Direct measures of GM/WM change with advantages similar to SIENA have been lacking. We created a robust and easily-implemented method for direct longitudinal analysis of GM/WM atrophy, SIENAX multi-time-point (SIENAX-MTP). We built on the basic halfway-registration and mask composition components of SIENA to improve the raw output of FMRIB's FAST tissue segmentation tool. In addition, we created LFAST, a modified version of FAST incorporating a 4th dimension in its hidden Markov random field model in order to directly represent time. The method was validated by scan-rescan, simulation, comparison with SIENA, and two clinical effect size comparisons. All validation approaches demonstrated improved longitudinal precision with the proposed SIENAX-MTP method compared to SIENAX. For GM, simulation showed better correlation with experimental volume changes (r=0.992 vs. 0.941), scan-rescan showed lower standard deviations (3.8% vs. 8.4%), correlation with SIENA was more robust (r=0.70 vs. 0.53), and effect sizes were improved by up to 68%. Statistical power estimates indicated a potential drop of 55% in the number of subjects required to detect the same treatment effect with SIENAX-MTP vs. SIENAX. The proposed direct GM/WM method significantly improves on the standard SIENAX technique by trading a small amount of bias for a large reduction in variance, and may provide more precise data and additional statistical power in longitudinal studies.
    NeuroImage 12/2013; DOI:10.1016/j.neuroimage.2013.12.004 · 6.13 Impact Factor