Conference PaperPDF Available

The IronTract challenge: Validation and optimal tractography methods for the HCP diffusion acquisition scheme

Authors:
  • Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital and Harvard Medical School
Conference Paper

The IronTract challenge: Validation and optimal tractography methods for the HCP diffusion acquisition scheme

Abstract and Figures

Synopsis We present results from IronTract, the first challenge to evaluate tractography on the two-shell diffusion scheme of the Human Connectome Project (HCP). Accuracy was evaluated by comparison to tracer injections in the same macaque brains as the diffusion data. Training and validation datasets involved different injection sites. We observed that optimizing data analysis with respect to one injection site does not guarantee optimality for another; encouragingly, two teams could achieve consistently high performance in both datasets. We also found that, when analysis methods are optimized, the HCP scheme may achieve similar accuracy as a more demanding diffusion spectrum imaging acquisition. Introduction The error-prone nature of diffusion MRI (dMRI) tractography has received considerable attention in recent years, in great part due to tractography challenges that have increased our awareness of the limitations of this technique. Prior challenges, however, used dMRI data that had been either synthesized or acquired with a single, low b-value. This precluded the use of state-of-the-art analysis methods that require multi-shell or Cartesian sampling schemes. Furthermore, it is not clear whether the conclusions of those studies are applicable to the multi-shell, high-angular-resolution dMRI data that are now widely available thanks to large-scale initiatives like the Human Connectome Project (HCP). The IronTract challenge seeks to address this gap by investigating i) which data processing strategies lead to optimal tractography accuracy for the two-shell dMRI acquisition scheme of the lifespan and disease HCP, and ii) whether those methods could achieve even higher accuracy with a different acquisition scheme. Here we present initial results of the challenge and discuss next steps. Methods The training and validation cases are part of a previously described dataset that consists of in-vivo tracing and ex-vivo dMRI acquired in the same macaque brains. Tracer data: Bidirectional tracers were injected as previously described. The training and validation cases consisted of two different brains each of which received a single injection, in the anterior frontal and ventrolateral prefrontal cortex respectively. dMRI data: After fixation, the brains were scanned in a small-bore 4.7T Bruker scanner using 3D EPI, (0.7x0.7x0.7mm, TR=750ms, TE=43ms, =15ms, Δ=19ms, maximum b=40,000s/mm), with 515 volumes corresponding to a Cartesian lattice in q-space. These data were resampled on q-space shells, using a fast implementation of the non-uniform fast Fourier transform (NUFFT). We generated data on the two q-shells of the HCP lifespan acquisition scheme (b=1500/ 3000s/mm , multiplied here by the 4x factor required to achieve comparable diffusion contrast ex-vivo as in-vivo). Challenge: The challenge was administered through the QMENTA platform (qmenta.com/irontract-challenge/). Participants were blind to the tracer data. For the training case, they uploaded their tractography results and received a score (see below) and ranking. They could repeat this any number of times while they fine tuned the free parameters of their methods to optimize their score. They then applied their optimized analysis pipelines to the validation case, which was used as the basis for the final ranking (Figure 1). Figure of merit: In contrast to prior challenges, participants were asked to upload tractography volumes obtained with multiple thresholds. The thresholding strategy (e.g., angle or probability-based) was left up to the participants. For each tractography volume, true and false positive rates were computed by voxel-wise comparison to the tracer data. The score was the area under the curve (AUC). It was computed for false positive rates in [0,0.3], hence the maximum score was 0.3. We separated the rankings into: i) Overall/DSI: participants were allowed to use any sampling scheme ii) HCP: participants were restricted to the HCP-like, two-shell scheme. Results We report results submitted before the MICCAI 2019 conference. Out of 30 registered teams, 12 completed the challenge. There were 227 total submissions (training: 187, validation: 39) and 17 final submissions that were ranked. The diffusion reconstruction and tractography algorithms used are reported in Table 1. Overall, better performance was achieved for the training (mean AUC=0.20) than the validation case (mean AUC=0.15) (Figure 2). Higher AUC scores were obtained using the DSI scheme, probabilistic tractography, spherical deconvolution, and additional constraining masks (Figure 3). We localized the true positives and false negatives for each submission in terms of pathways in the validation case (Figure 4). At a false positive rate=0.
Content may be subject to copyright.
06/11/2019, 15*59
Page 1 of 5https://submissions2.mirasmart.com/ISMRM2020/ViewSubmission.aspx?sbmID=269
The IronTract challenge: Validation and optimal tractography methods for the HCP
diusion acquisition scheme
Chiara Maei , Gabriel Girard , Kurt G. Schilling , Nagesh Adluru , Dogu Baran Aydogan , Andac Hamamci , Fang-Cheng Yeh , Matteo Mancini , Ye Wu , Alessia Sarica , Achille Teillac , Steven H.
Baete , Davood Karimi , Ying-Chia Lin , Fernando Boada , Nathalie Richard , Bassem Hiba , Aldo Quattrone , Yoonmi Hong , Dinggang Shen , Pew-Thian Yap , Tommy Boshkovski , Jennifer S.
W. Campbell , Nikola Stikov , G. Bruce Pike , Barbara B. Bendlin , Andrew L. Alexander , Vivek Prabhakaran , Adam Anderson , Bennett A. Landman , Erick J.Z. Canales-Rodrígue , Muhamed
Barakovic , Jonathan Rafael-Patino , Thomas Yu , Gaëtan Rensonnet , Simona Schiavi , Alessandro Daducci , Marco Pizzolato , Elda Fischi-Gomez , Jean-Philippe Thiran , George Dai , Giorgia
Grisot , Nikola Lazovski , Albert Puente , Matt Rowe , Irina Sanchez , Vesna Prchkovska , Robert Jones , Julia Lehman , Suzanne Haber , and Anastasia Yendiki
Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital and Harvard Medical School, Charlestown, MA, United States, Radiology Department, Centre Hospitalier Universitaire
Vaudois and University of Lausanne, Lausanne, Switzerland, Signal Processing Lab (LTS5), École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland, Institute of Imaging Science, Vanderbilt University,
Nashville, TN, United States, University of Wisconsin, Madison, WI, United States, Department of Neuroscience and Biomedical Engineering, Aalto University, Helsinki, Finland, Department of Biomedical
Engineering, Faculty of Engineering, Yeditepe University, Instanbul, Turkey, Department of Neurological Surgery, University of Pittsburgh, Pittsburgh, PA, United States, Department of Neuroscience, Brighton and
Sussex Medical School, University of Sussex, Brighton, United Kingdom, NeuroPoly Lab, Polytechnique Montreal, Montreal, QC, Canada, Department of Radiology and BRIC, University of North Carolina,
Chapel Hill, NC, United States, Neuroscience Research Center, University Magna Graecia of Catanzaro, Catanzaro, Italy, CNRS/ISC, Bron, France, Université de Bordeaux, Bordeaux, France, CNRS/INCIA,
Bordeaux, France, Center for Advanced Imaging Innovation and Research (CAI2 R), NYU School of Medicine, New York, NY, United States, Center for Biomedical Imaging, Dept. of Radiology, NYU School of
Medicine, New York, NY, United States, Boston Children's Hospital, Boston, MA, United States, Montreal Neurological Institute, McGill University, Montreal, QC, Canada, Hotchkiss Brain Institute and
Department of Radiology, University of Calgary, Calgary, AB, Canada, Department of Electrical Engineering, Vanderbilt University, Nashville, TN, United States, FIDMAG Germanes Hospitalàries, Sant Boi de
Llobregat, Barcelona, Spain, Mental Health Research Networking Center (CIBERSAM), Madrid, Spain, Translational Imaging in Neurology (ThINK), Department of Medicine and Biomedical Engineering,
University Hospital and University of Basel, Basel, Switzerland, ICTEAM Institute, Université Catholique de Louvain, Louvain-la-Neuve, Belgium, Computer Science Department, University of Verona, Verona,
Italy, Wellesley College, Wellesley, Wellesley, MA, United States, DeepHealth, Inc., Cambridge, MA, United States, QMENTA, Inc., Barcelona, Spain, Department of Pharmacology and Physiology, University
of Rochester School of Medicine, Rochester, NY, United States
Synopsis
We present results from IronTract, the first challenge to evaluate tractography on the two-shell diusion scheme of the Human Connectome
Project (HCP). Accuracy was evaluated by comparison to tracer injections in the same macaque brains as the diusion data. Training and
validation datasets involved dierent injection sites. We observed that optimizing data analysis with respect to one injection site does not
guarantee optimality for another; encouragingly, two teams could achieve consistently high performance in both datasets. We also found
that, when analysis methods are optimized, the HCP scheme may achieve similar accuracy as a more demanding diusion spectrum
imaging acquisition.
Introduction
The error-prone nature of diusion MRI (dMRI) tractography has received considerable attention in recent years, in great part due to tractography
challenges that have increased our awareness of the limitations of this technique . Prior challenges, however, used dMRI data that had been either
synthesized or acquired with a single, low b-value. This precluded the use of state-of-the-art analysis methods that require multi-shell or Cartesian
sampling schemes. Furthermore, it is not clear whether the conclusions of those studies are applicable to the multi-shell, high-angular-resolution
dMRI data that are now widely available thanks to large-scale initiatives like the Human Connectome Project (HCP). The IronTract challenge seeks to
address this gap by investigating i) which data processing strategies lead to optimal tractography accuracy for the two-shell dMRI acquisition
scheme of the lifespan and disease HCP, and ii) whether those methods could achieve even higher accuracy with a dierent acquisition scheme. Here
we present initial results of the challenge and discuss next steps.
Methods
The training and validation cases are part of a previously described dataset that consists of in-vivo tracing and ex-vivo dMRI acquired in the same
macaque brains . Tracer data: Bidirectional tracers were injected as previously described . The training and validation cases consisted of two
dierent brains each of which received a single injection, in the anterior frontal and ventrolateral prefrontal cortex respectively. dMRI data: After
fixation, the brains were scanned in a small-bore 4.7T Bruker scanner using 3D EPI, (0.7x0.7x0.7mm, TR=750ms, TE=43ms, 𝛿=15ms, Δ=19ms,
maximum b=40,000s/mm ), with 515 volumes corresponding to a Cartesian lattice in q-space. These data were resampled on q-space shells, using a
fast implementation of the non-uniform fast Fourier transform (NUFFT) . We generated data on the two q-shells of the HCP lifespan acquisition
scheme (b=1500/ 3000s/mm , multiplied here by the 4x factor required to achieve comparable diusion contrast ex-vivo as in-vivo ). Challenge:
The challenge was administered through the QMENTA platform (qmenta.com/irontract-challenge/). Participants were blind to the tracer data. For the
training case, they uploaded their tractography results and received a score (see below) and ranking. They could repeat this any number of times
while they fine tuned the free parameters of their methods to optimize their score. They then applied their optimized analysis pipelines to the
validation case, which was used as the basis for the final ranking (Figure 1). Figure of merit: In contrast to prior challenges, participants were asked
to upload tractography volumes obtained with multiple thresholds. The thresholding strategy (e.g., angle or probability-based) was left up to the
participants. For each tractography volume, true and false positive rates were computed by voxel-wise comparison to the tracer data. The score was
the area under the curve (AUC). It was computed for false positive rates in [0,0.3], hence the maximum score was 0.3. We separated the rankings
into: i) Overall/DSI: participants were allowed to use any sampling scheme ii) HCP: participants were restricted to the HCP-like, two-shell scheme.
Results
We report results submitted before the MICCAI 2019 conference. Out of 30 registered teams, 12 completed the challenge. There were 227 total
submissions (training: 187, validation: 39) and 17 final submissions that were ranked. The diusion reconstruction and tractography algorithms used
are reported in Table 1. Overall, better performance was achieved for the training (mean AUC=0.20) than the validation case (mean AUC=0.15) (Figure
2). Higher AUC scores were obtained using the DSI scheme, probabilistic tractography, spherical deconvolution, and additional constraining masks
(Figure 3). We localized the true positives and false negatives for each submission in terms of pathways in the validation case (Figure 4). At a false
positive rate=0.1, the sensitivity was variable across dierent pathways and overall low (HCP=0.57, DSI=0.56). Almost all submissions label regions
close to the injection site correctly, but most fail to reconstruct pathways far from it or that require splitting from the main trajectory (eg. brainstem
and thalamic fibers). Majority voting analysis confirms this trend.
1 2,3 4 5 6 7 8 9,10 11 12 13,14,15
16,17 18 16,17 16,17 13 13 12 11 11 11 10
19 10 20 5 5 5 21 4,21 3,22,23
3,24 3 3 3,25 3,26 26 3 3,24 2,3 27
28 29 29 29 29 29 1 30 30 1
1 2
3 4
5 6 7
8 9
10 11
12 13 14 15
16 17
18 19 20
21 22
23 24
25 26
27 28 29 30
1-7
8-10 11
2
12
2 13
06/11/2019, 15*59
Page 2 of 5https://submissions2.mirasmart.com/ISMRM2020/ViewSubmission.aspx?sbmID=269
Discussion and Conclusion
Our results show that, when processing methods are tuned appropriately, it is possible to achieve similar tractography accuracy with the HCP and
DSI schemes, even though the latter involves 2.8 times more directions and 3.3 times higher maximum b-value. Thus the HCP scheme represents an
advantageous trade-o between accuracy and acquisition time. For many of the pipelines employed here, optimizing the methods with respect to
accuracy for one seed/injection region did not guarantee optimal performance for another region. This highlights the importance of using anatomical
studies from a variety of regions as guidance for tractography. The two injection sites used here project through similar white-matter pathways but
reach those pathways from very dierent angles. The tracing data reveal complex systems of small bundles that travel within and jump between
dierent pathways . The present results confirm the limited accuracy of tractography when traveling longer distances and through bottle-neck
regions, where fibres align and diverge . Encouragingly, two teams could achieve consistently high performance in both training and validation
datasets. In next steps, we will investigate which of their pre/post-processing and tractography methods led to this robustness. We expect our
findings to have implications for analyzing the thousands of datasets acquired with the HCP scheme that will soon be publicly available.
Acknowledgements
Data acquisition was supported by the National Institute of Mental Health (R01-MH045573). Additional research support was provided by the National
Institute of Biomedical Imaging and Bioengineering (R01-EB021265). Imaging was carried out at the Athinoula A. Martinos Center for Biomedical
Imaging at the Massachusetts General Hospital, using resources provided by the Center for Functional Neuroimaging Technologies, P41-EB015896, a
P41 Biotechnology Resource Grant, and instrumentation supported by the NIH Shared Instrumentation Grant Program (S10RR016811,
S10RR023401, S10RR019307, and S10RR023043).
Additional grants that supported part of this work: NIH grants (NS093842, EB022880, and
EB006733).
References
1. Daducci A, Canales-Rodriguez EJ, Descoteaux M, Garyfallidis E, Gur Y, Lin YC, et al. Quantitative comparison of reconstruction methods for intra-
voxel fiber recovery from diusion MRI. IEEE transactions on medical imaging. 2014;33(2):384-99.
2. Ning L, Laun F, Gur Y, DiBella EV, Deslauriers-Gauthier S, Megherbi T, et al. Sparse Reconstruction Challenge for diusion MRI: Validation on a
physical phantom to determine which acquisition scheme and analysis method to use? Med Image Anal. 2015;26(1):316-31.
3. Cote MA, Girard G, Bore A, Garyfallidis E, Houde JC, Descoteaux M. Tractometer: towards validation of tractography pipelines. Med Image Anal.
2013;17(7):844-57.
4. Neher PF, Laun FB, Stieltjes B, Maier-Hein KH. Fiberfox: facilitating the creation of realistic white matter software phantoms. Magn Reson Med.
2014;72(5):1460-70.
5. Maier-Hein, K.H., Neher, P.F., Houde, J.C., et al. The challenge of mapping the human connectome based on diusion tractography. Nat. Commun.
2017;8(1349).
6. Nath V, Schilling KG, Parvathaneni P, et al. Tractography Reproducibility Challenge with Empirical Data (TraCED): The 2017 ISMRM Diusion Study
Group Challenge. J Magn Reson Imaging. 2019
7. Schilling KG, Nath V, Hansen C, Parvathaneni P, et al. Limits to anatomical accuracy of diusion tractography using modern approaches.
NeuroImage. 2019; 185:1–11
8. Z. Safadi, G. Grisot, S. Jbabdi, T. Behrens, S. R. Heilbronner, J. Mandeville, A. Versace, M. L. Phillips, A. Yendiki, S. N. Haber, Functional
segmentation of the internal capsule: Linking white matter abnormalities to specific connections, Journal of Neuroscience. 2018; 38(8):2106-17.
9.G. Grisot. S. N. Haber, A. Yendiki, Validation of diusion MRI models and tractography algorithms using chemical tracing, Proc. Intl. Soc. Mag. Res.
Med.. 2018:734
10.W. Tang, S. Jbabdi, Z. Zhu, M. Cottaar, G. Grisot, J. Lehman, A. Yendiki, S. N. Haber A connectional hub in the rostral anterior cingulate cortex
links areas of emotion and cognitive control, eLife, In Press, 2019.
11. Suzanne Haber. Tracing intrinsic fiber connections in postmortem human brain with WGA-HRP. Journal of Neuroscience Methods. 1988;23(1):15–
22.
12. Jerey AF and Bradley PS. Nonuniform fast Fourier transforms using min-max interpolation. IEEE Transactions on Signal Processing.
2003;51(2):560–574.
13. Dyrby TB, William FC, Alexander DC, Jelsing J, Garde E, Søgaard LV. An ex vivo imaging pipeline for producing high quality and high-resolution
diusion-weighted imaging datasets. Human Brain Mapping, 32(4):544–563, 2011.
14. Lehman JF, Greenberg BD, McIntyre CC, Rasmussen SA, Haber SN. Rules ventral prefrontal cortical axons use to reach their targets: implications
for diusion tensor imaging tractography and deep brain stimulation for psychiatric illness. Journal of neuroscience. 2011;31:10392–10402.
14
15
06/11/2019, 15*59
Page 3 of 5https://submissions2.mirasmart.com/ISMRM2020/ViewSubmission.aspx?sbmID=269
15. Aydogan DB, Jacobs R, Dulawa S, Thompson SL, Francois MC, Toga AW, Dong H, Knowles JA, Shi Y. When tractography meets tracer injections:
a systematic study of trends and variation sources of diusion-based connectivity. Brain Struct. Funct. 2018;223: 2841–2858.
16. Tran G and Yonggang Shi. "Fiber orientation and compartment parameter estimation from multi-shell diusion imaging." IEEE transactions on
medical imaging. 2015;34(11):2320-2332.
17. Wu, Y. et al. Asymmetry spectrum imaging for baby diusion tractography. In International Conference on Information Processing in Medical
Imaging. 319–331 (Springer, 2019).
18. Tournier JD, Calamante F, Connelly A. Robust determination of the fibre orientation distribution in diusion MRI: non-negativity constrained super-
resolved spherical deconvolution. Neuroimage. 2017;35(4):1459-1472.
19. Yeh FC, Wedeen VJ, Tseng WY. Generalized q-sampling imaging. IEEE TMI. 2010;29(9)5.
20. Jeurissen B, Tournier JD, Dhollander T, Connelly A, Sijbers J. Multi-tissue constrained spherical deconvolution for improved analysis of multi-shell
diusion mri data. NeuroImage. 2014;103:411-426, 2014.
21. Dhollander T, Raelt D, Connelly A. Unsupervised 3-tissue response function estimation from single-shell or multi-shell diusion MR data without
a co-registered T1 image. ISMRM Workshop on Breaking the Barriers of Diusion MRI. 2016.
22. Baete SH, Cloos MA, Lin YC, Placantonakis DG, Shepherd T, Boada FE. Fingerprinting Orientation Distribution Functions in diusion MRI detects
smaller crossing angles. NeuroImage. 2019;198:231-41.
23. Baete SH, Yutzy S, Boada F. Radial q-space sampling for DSI. Magnetic resonance in medicine : ocial journal of the Society of Magnetic
Resonance in Medicine / Society of Magnetic Resonance in Medicine. 2016;76:769-80.
24 Dell'Acqua F, Scifo P, Rizzo G, Catani M, Simmons A, Scotti G, and Fazio F. A modified damped richardson-lucy algorithm to reduce isotropic
background eects in spherical deconvolution. Neuroimage. 2010;49(2):1446-1458.
25. Canales-Rodríguez EJ, Daducci A, Sotiropoulos S, Caruyer E, Aja-Fernández S, Radua J, Yurramendi Mendizabal JM, Iturria-Medina Y, Melie-
García L, Alemán-Gómez Y, Thiran JP, Sarró S, Pomarol-Clotet E, Salvador R. Spherical Deconvolution of Multichannel Diusion MRI Data with Non-
Gaussian Noise Models and Spatial Regularization. PLoS One. 2015;10(10):e0138910.
Figures
Figure 1. Challenge pipeline. Data from two monkey brains served as training and validation cases. For both we had in-vivo tracing with dierent
injection sites in the frontal cortex, and ex-vivo dMRI acquired on a Cartesian grid (515 directions, max b-value=40,000s/mm ) and resampled
through non-uniform fast Fourier transform (NUFFT) on the HCP multi-shell scheme. Data were shared via the Qmenta platform; participants could
tune tractography parameters on the basis of the accuracy score obtained for the training data. Submissions for the validation case were then
evaluated.
2
06/11/2019, 15*59
Page 4 of 5https://submissions2.mirasmart.com/ISMRM2020/ViewSubmission.aspx?sbmID=269
Figure 2. Receiver Operator Characteristic (ROC) curves and the corresponding Area Under the Curve (AUC) are shown for each submission for both
training (top) and validation case (bottom), for HCP (solid lines) and Overall/DSI (dashed line) ranking. We set the maximum false positive rate (FPR) =
0.3, as previous studies showed this to be the maximum FPR that can be achieved by deterministic tractography methods . Bar graphs show the
AUC score for each team for the training (green) and validation (lightblue) case for HCP (top) and overall/DSI ranking (bottom).
Figure 3. Area under the curve (AUC) scores for dierent tractography methods, diusion models, masking strategies and acquisition schemes for
training (top) abd validation data (bottom) across all submissions. Overlay scatterplots show submissions for HCP () and overall/DSI (). SD=
spherical deconvolution; 3Comp = three compartment model; ASI = asymmetry spectrum imaging; GQI = generalized Q-ball imaging; ODF-FP =
orientation distribution function fingerprinting; RDSI = radial diusion spectrum imaging.
Figure 4. Schematic of the main pathways present in the tracing for the validation case (top-left). Boxplots and overlaid scatterplots show the ratio of
true positive voxels for each bundle for each submission and for the majority vote (HCP: gray; overall/DSI: lightblue). All submissions were evaluated
at FPR=0.1. ALIC=anterior limb of the internal capsule; CB=cingulum bundle; CC=corpus callosum; EC=external capsule; EmC=extreme capsule;
LPFC_WM=lateral pre-frontal cortex white-matter; UF=uncinate fasciculus.
9
06/11/2019, 15*59
Page 5 of 5https://submissions2.mirasmart.com/ISMRM2020/ViewSubmission.aspx?sbmID=269
Table 1. Details of the methods used by each team. Model=diusion model; Method=Tractography algorithm; Masks=use of additional masks to
constrain tractography. 3-Comp=Three compartment model ; ASI= Asymmetry Spectrum Imaging ; CSD=constrained spherical deconvolution ;
GQI=Generalized Q-ball Imaging ; MSMT-CSD=Multi Shell Multi Tissue Constrained Spherical Deconvolution ; ODF-FP=ODF Fingerprinting ;
RDSI= Radial DSI ; RL-SD=Richardson-Lucy Spherical Deconvolution ; RUMBA-SD=Robust and Unbiased Model-Based Spherical
Deconvolution .
16 17 18
19 20,21 22
23 24
25
... We have shown that this resampling approach can approximate q-shell data from data collected on a grid in q-space with high accuracy . Thus, from a single scan, we can generate data with multiple q-space sampling schemes and perform systematic, side-by-side comparisons, as we did most recently in the IronTract Challenge (Maffei et al., , 2020. ...
... Our findings indicate that, with an acquisition time similar or shorter than typical high angular resolution multi-shell scans, CS-DSI data can be used to approximate both fully sampled DSI and multi-shell data with high accuracy. This provides the flexibility to take advantage of the high angular accuracy of DSI (Daducci et al., 2013;Jones et al., 2020;Maffei et al., 2020Maffei et al., , 2021 while also allowing microstructural analyses either on q-shells or on the full EAP. For in vivo population studies, this gives access to a wider range of potential biomarkers. ...
... The findings of recent validation studies suggest that DSI may provide more accurate fiber reconstructions than single-or multi-shell acquisition schemes. DSI produced more accurate fiber orientation estimates in simulations (Daducci et al., 2013) and comparisons to optical imaging measurements , as well as more accurate tractography when compared to ground-truth anatomic tracing in non-human primates Maffei et al., 2020). The present study shows that a sparsely sampled CS-DSI protocol preserves the high angular accuracy of fully sampled DSI. ...
Article
Full-text available
While many useful microstructural indices, as well as orientation distribution functions, can be obtained from multi-shell dMRI data, there is growing interest in exploring the richer set of microstructural features that can be extracted from the full ensemble average propagator (EAP). The EAP can be readily computed from diffusion spectrum imaging (DSI) data, at the cost of a very lengthy acquisition. Compressed sensing (CS) has been used to make DSI more practical by reducing its acquisition time. CS applied to DSI (CS-DSI) attempts to reconstruct the EAP from significantly undersampled q-space data. We present a post mortem validation study where we evaluate the ability of CS-DSI to approximate not only fully sampled DSI but also multi-shell acquisitions with high fidelity. Human brain samples are imaged with high-resolution DSI at 9.4T and with polarization-sensitive optical coherence tomography (PSOCT). The latter provides direct measurements of axonal orientations at microscopic resolutions, allowing us to evaluate the mesoscopic orientation estimates obtained from diffusion MRI, in terms of their angular error and the presence of spurious peaks. We test two fast, dictionary-based, L2-regularized algorithms for CS-DSI reconstruction. We find that, for a CS acceleration factor of R=3, i.e., an acquisition with 171 gradient directions, one of these methods is able to achieve both low angular error and low number of spurious peaks. With a scan length similar to that of high angular resolution multi-shell acquisition schemes, this CS-DSI approach is able to approximate both fully sampled DSI and multi-shell data with high accuracy. Thus it is suitable for orientation reconstruction and microstructural modeling techniques that require either grid- or shell-based acquisitions. We find that the signal-to-noise ratio (SNR) of the training data used to construct the dictionary can have an impact on the accuracy of CS-DSI, but that there is substantial robustness to loss of SNR in the test data. Finally, we show that, as the CS acceleration factor increases beyond R=3, the accuracy of these reconstruction methods degrade, either in terms of the angular error, or in terms of the number of spurious peaks. Our results provide useful benchmarks for the future development of even more efficient q-space acceleration techniques.
... The first round was organized in the context of the 2019 international conference on Medical Image Computing and Computer-Assisted Intervention. Preliminary results from the first and second rounds were presented, respectively, at the 2020 and 2021 annual meetings of the International Society for Magnetic Resonance in Medicine ( Maffei et al., , 2020. In the first round, two teams outperformed all others, achieving both high accuracy and robustness to the location of the seed region. ...
Article
Full-text available
Limitations in the accuracy of brain pathways reconstructed by diffusion MRI (dMRI) tractography have received considerable attention. While the technical advances spearheaded by the Human Connectome Project (HCP) led to significant improvements in dMRI data quality, it remains unclear how these data should be analyzed to maximize tractography accuracy. Over a period of two years, we have engaged the dMRI community in the IronTract Challenge, which aims to answer this question by leveraging a unique dataset. Macaque brains that have received both tracer injections and ex vivo dMRI at high spatial and angular resolution allow a comprehensive, quantitative assessment of tractography accuracy on state-of-the-art dMRI acquisition schemes. We find that, when analysis methods are carefully optimized, the HCP scheme can achieve similar accuracy as a more time-consuming, Cartesian-grid scheme. Importantly, we show that simple pre- and post-processing strategies can improve the accuracy and robustness of many tractography methods. Finally, we find that fiber configurations that go beyond crossing (e.g., fanning, branching) are the most challenging for tractography. The IronTract Challenge remains open and we hope that it can serve as a valuable validation tool for both users and developers of dMRI analysis methods.
Article
Full-text available
Anatomic tracing is recognized as a critical source of knowledge on brain circuitry that can be used to assess the accuracy of diffusion MRI (dMRI) tractography. However, most prior studies that have performed such assessments have used dMRI and tracer data from different brains and/or have been limited in the scope of dMRI analysis methods allowed by the data. In this work, we perform a quantitative, voxel-wise comparison of dMRI tractography and anatomic tracing data in the same macaque brain. An ex vivo dMRI acquisition with high angular resolution and high maximum b-value allows us to compare a range of q-space sampling, orientation reconstruction, and tractography strategies. The availability of tracing in the same brain allows us to localize the sources of tractography errors and to identify axonal configurations that lead to such errors consistently, across dMRI acquisition and analysis strategies. We find that these common failure modes involve geometries such as branching or turning, which cannot be modeled well by crossing fibers. We also find that the default thresholds that are commonly used in tractography correspond to rather conservative, low-sensitivity operating points. While deterministic tractography tends to have higher sensitivity than probabilistic tractography in that very conservative threshold regime, the latter outperforms the former as the threshold is relaxed to avoid missing true anatomical connections. On the other hand, the q-space sampling scheme and maximum b-value have less of an impact on accuracy. Finally, using scans from a set of additional macaque brains, we show that there is enough inter-individual variability to warrant caution when dMRI and tracer data come from different animals, as is often the case in the tractography validation literature. Taken together, our results provide insights on the limitations of current tractography methods and on the critical role that anatomic tracing can play in identifying potential avenues for improvement.
Article
Full-text available
We investigated afferent inputs from all areas in the frontal cortex (FC) to different subregions in the rostral anterior cingulate cortex (rACC). Using retrograde tracing in macaque monkeys, we quantified projection strength by counting retrogradely labeled cells in each FC area. The projection from different FC regions varied across injection sites in strength, following different spatial patterns. Importantly, a site at the rostral end of the cingulate sulcus stood out as having strong inputs from many areas in diverse FC regions. Moreover, it was at the integrative conjunction of three projection trends across sites. This site marks a connectional hub inside the rACC that integrates FC inputs across functional modalities. Tractography with monkey diffusion magnetic resonance imaging (dMRI) located a similar hub region comparable to the tracing result. Applying the same tractography method to human dMRI data, we demonstrated that a similar hub can be located in the human rACC.
Article
Full-text available
Tractography is a powerful technique capable of non-invasively reconstructing the structural connections in the brain using diffusion MRI images, but the validation of tractograms is challenging due to lack of ground truth. Owing to recent developments in mapping the mouse brain connectome, high-resolution tracer injection-based axonal projection maps have been created and quickly adopted for the validation of tractography. Previous studies using tracer injections mainly focused on investigating the match in projections and optimal tractography protocols. Being a complicated technique, however, tractography relies on multiple stages of operations and parameters. These factors introduce large variabilities in tractograms, hindering the optimization of protocols and making the interpretation of results difficult. Based on this observation, in contrast to previous studies, in this work we focused on quantifying and ranking the amount of performance variation introduced by these factors. For this purpose, we performed over a million tractography experiments and studied the variability across different subjects, injections, anatomical constraints and tractography parameters. By using N-way ANOVA analysis, we show that all tractography parameters are significant and importantly performance variations with respect to the differences in subjects are comparable to the variations due to tractography parameters, which strongly underlines the importance of fully documenting the tractography protocols in scientific experiments. We also quantitatively show that inclusion of anatomical constraints is the most significant factor for improving tractography performance. Although this critical factor helps reduce false positives, our analysis indicates that anatomy-informed tractography still fails to capture a large portion of axonal projections.
Article
Full-text available
Tractography based on non-invasive diffusion imaging is central to the study of human brain connectivity. To date, the approach has not been systematically validated in ground truth studies. Based on a simulated human brain data set with ground truth tracts, we organized an open international tractography challenge, which resulted in 96 distinct submissions from 20 research groups. Here, we report the encouraging finding that most state-of-the-art algorithms produce tractograms containing 90% of the ground truth bundles (to at least some extent). However, the same tractograms contain many more invalid than valid bundles, and half of these invalid bundles occur systematically across research groups. Taken together, our results demonstrate and confirm fundamental ambiguities inherent in tract reconstruction based on orientation information alone, which need to be considered when interpreting tractography and connectivity results. Our approach provides a novel framework for estimating reliability of tractography and encourages innovation to address its current limitations.
Conference Paper
Full-text available
To overcome the fact that the fibre orientation distribution (FOD) from constrained spherical deconvolution (CSD) assumes a single-fibre white matter (WM) response function—and is thus inappropriate and distorted in voxels containing grey matter (GM) or cerebrospinal fluid (CSF)—multi-shell multi-tissue CSD (MSMT-CSD) was proposed. MSMT-CSD can resolve WM, GM and CSF signal contributions, but requires multi-shell data. Very recently, we proposed a novel method that can achieve the same results using just single-shell data. We refer to this method as "single-shell 3-tissue CSD" (SS3T-CSD). Both MSMT-CSD and SS3T-CSD require WM, GM and CSF response functions. These can be obtained from manually selected exemplary voxels of the tissue classes, or via the procedure described initially in the MSMT-CSD paper, which relies on a highly accurately co-registered T1 image. We propose an unsupervised procedure that does not depend on a T1 image, nor registration, and works for both single-shell and multi-shell data.
Article
Full-text available
Diffusion magnetic resonance imaging (dMRI) is the modality of choice for investigating in-vivo white matter connectivity and neural tissue architecture of the brain. The diffusion-weighted signal in dMRI reflects the diffusivity of water molecules in brain tissue and can be utilized to produce image-based biomarkers for clinical research. Due to the constraints on scanning time, a limited number of measurements can be acquired within a clinically feasible scan time. In order to reconstruct the dMRI signal from a discrete set of measurements , a large number of algorithms have been proposed in recent years in conjunction with varying sampling schemes, i.e., with varying b-values and gradient directions. Thus, it is imperative to compare the performance of these reconstruction methods on a single data set to provide appropriate guidelines to neuroscientists on making an informed decision while designing their acquisition protocols. For this purpose, the SParse Reconstruction Challenge (SPARC) was held along with the workshop on Computational Diffusion MRI (at MICCAI 2014) to validate the performance of multiple reconstruction methods using data acquired from a physical phantom. A total of 16 reconstruction algorithms (9 teams) participated in this community challenge. The goal was to reconstruct single b-value and/or multiple b-value data from a sparse set of measurements. In particular, the aim was to determine an appropriate acquisition protocol (in terms of the number of measurements, b-values) and the analysis method to use for a neuroimag-ing study. The challenge did not delve on the accuracy of these methods in estimating model specific measures such as fractional anisotropy (FA) or mean diffusivity, but on the accuracy of these methods to fit the data. This paper presents several quantitative results pertaining to each reconstruction algorithm. The conclusions in this paper provide a valuable guideline for choosing a suitable algorithm and the corresponding data-sampling scheme for clinical neuroscience applications.
Article
Diffusion tractography is routinely used to study white matter architecture and brain connectivity in vivo. A key step for successful tractography of neuronal tracts is the correct identification of tract directions in each voxel. Here we propose a fingerprinting-based methodology to identify these fiber directions in Orientation Distribution Functions, dubbed ODF-Fingerprinting (ODF-FP). In ODF-FP, fiber configurations are selected based on the similarity between measured ODFs and elements in a pre-computed library. In noisy ODFs, the library matching algorithm penalizes the more complex fiber configurations. ODF simulations and analysis of bootstrapped partial and whole-brain in vivo datasets show that the ODF-FP approach improves the detection of fiber pairs with small crossing angles while maintaining fiber direction precision, which leads to better tractography results. Rather than focusing on the ODF maxima, the ODF-FP approach uses the whole ODF shape to infer fiber directions to improve the detection of fiber bundles with small crossing angle. The resulting fiber directions aid tractography algorithms in accurately displaying neuronal tracts and calculating brain connectivity.
Article
Diffusion MRI fiber tractography is widely used to probe the structural connectivity of the brain, with a range of applications in both clinical and basic neuroscience. Despite widespread use, tractography has well-known pitfalls that limits the anatomical accuracy of this technique. Numerous modern methods have been developed to address these shortcomings through advances in acquisition, modeling, and computation. To test whether these advances improve tractography accuracy, we organized the 3-D Validation of Tractography with Experimental MRI (3D-VoTEM) challenge at the ISBI 2018 conference. We made available three unique independent tractography validation datasets – a physical phantom and two ex vivo brain specimens - resulting in 176 distinct submissions from 9 research groups. By comparing results over a wide range of fiber complexities and algorithmic strategies, this challenge provides a more comprehensive assessment of tractography's inherent limitations than has been reported previously. The central results were consistent across all sub-challenges in that, despite advances in tractography methods, the anatomical accuracy of tractography has not dramatically improved in recent years. Taken together, our results independently confirm findings from decades of tractography validation studies, demonstrate inherent limitations in reconstructing white matter pathways using diffusion MRI data alone, and highlight the need for alternative or combinatorial strategies to accurately map the fiber pathways of the brain.
Article
The anterior limb of the internal capsule (ALIC) carries thalamic and brainstem fibers from prefrontal cortical regions that are associated with different aspects of emotion, motivation, cognition processing, and decision-making. This large fiber bundle is abnormal in several psychiatric illnesses and a major target for deep brain stimulation. Yet, we have very little information about where specific prefrontal fibers travel within the bundle. Using a combination of tracing studies and diffusionMRIin malenonhumanprimates, as well as diffusion MRI in male and female human subjects, we segmented the human ALIC into five regions based on the positions of axons from different cortical regions within the capsule. Fractional anisotropy (FA) abnormalities in patients with bipolar disorder were detected whenFAwas averaged in the ALIC segment that carries ventrolateral prefrontal cortical connections. Together, the results set the stage for linking abnormalities within the ALIC to specific connections and demonstrate the utility of applying connectivity profiles of large white matter bundles based on animal anatomic studies to human connections and associating disease abnormalities in those pathways with specific connections. The ability to functionally segment large white matter bundles into their components begins a new era of refining how we think about white matter organization and use that information in understanding abnormalities.
Article
Diffusion MRI offers the unique opportunity of assessing the structural connections of human brains in vivo.With the advance of diffusion MRI technology, multi-shell imaging methods are becoming increasingly practical for large scale studies and clinical application. In this work, we propose a novel method for the analysis of multi-shell diffusion imaging data by incorporating compartment models into a spherical deconvolution framework for fiber orientation distribution (FOD) reconstruction. For numerical implementation, we develop an adaptively constrained energy minimization approach to efficiently compute the solution. On simulated and real data from Human Connectome Project (HCP), we show that our method not only reconstructs sharp and clean FODs for the modeling of fiber crossings, but also generates reliable estimation of compartment parameters with great potential for clinical research of neurological diseases. In comparisons with publicly available DSI-Studio and BEDPOSTX of FSL, we demonstrate that our method reconstructs sharper FODs with more precise estimation of fiber directions. By applying probabilistic tractography to the FODs computed by our method, we show that more complete reconstruction of the corpus callosum bundle can be achieved. On a clinical, two-shell diffusion imaging data, we also demonstrate the feasibility of our method in analyzing white matter lesions.