About
36
Publications
8,135
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
736
Citations
Introduction
Additional affiliations
April 2017 - present
January 2015 - March 2017
Education
September 2009 - June 2014
September 2006 - June 2009
September 2003 - June 2006
Publications
Publications (36)
Deep neural networks (DNNs) have recently been applied successfully to brain decoding and image reconstruction from functional magnetic resonance imaging (fMRI) activity. However, direct training of a DNN with fMRI data is often avoided because the size of available data is thought to be insufficient for training a complex network with numerous par...
The mental contents of perception and imagery are thought to be encoded in hierarchical representations in the brain, but previous attempts to visualize perceptual contents have failed to capitalize on multiple levels of the hierarchy, leaving it challenging to reconstruct internal imagery. Recent work showed that visual cortical activity measured...
Supplementary materials and methods.
(PDF)
Reconstruction quality of seen natural images for individual subjects.
Evaluations on individual subjects’ results are separately shown (VC activity; DNN1–8; N = 50; chance level, 50%; cf., Fig 3B), indicating that overall tendency was almost consistent across different subjects, except that the human judgment accuracy of reconstructions from Subje...
Reconstructions from different initial states.
The black and gray surrounding frames indicate presented and reconstructed images respectively (VC activity, DNN 1–8). We used different initial states for reconstructions with and without the DGN. For reconstructions with the DGN, we additionally performed the reconstruction analysis using a Gaussian-...
Other examples of reconstructions with a variable number of multiple DNN layers.
The black and gray surrounding frames indicate presented and reconstructed images respectively (VC activity, without the DGN).
(PDF)
All examples of artificial shape reconstructions.
The black and gray surrounding frames indicate presented and reconstructed images respectively (VC activity, DNN 1–8, without the DGN). The three rows of reconstructed images correspond to reconstructions from three subjects.
(PDF)
Reconstruction quality of alphabetical letters for individual subjects.
Evaluations on individual subjects’ results are separately shown (VC activity; DNN1–8; without the DGN; N = 10; chance level, 50%; cf., Fig 6C right). Evaluations of reconstructions using pixel-wise spatial correlation showed 98.9%, 87.8%, and 100.0% for Subject 1–3, respective...
Reconstruction quality of shape and color for different visual areas for individual subjects.
Evaluations on individual subjects’ results are separately shown (DNN1–8; without the DGN; N = 40; chance level, 50%; cf., Fig 7B). Evaluations by pixel-wise correlations and human judgment both showed almost consistent tendency across different subjects,...
Vividness scores for imagery images reported by subjects.
Vividness scores reported during the imagery experiment are shown in descending order of mean vividness scores across trials for individual images. For each subject, the vividness scores were averaged across trials for the same imagery images (N = 20). For the pooled results, to eliminate ba...
Imagery image reconstructions from V1.
The black and gray surrounding frames indicate presented and reconstructed images respectively (V1 activity, DNN 1–8, without the DGN). The three rows of reconstructed images correspond to reconstructions from three subjects. The rightmost images in the bottom row show reconstructions during maintenance of fix...
Deep image reconstruction: Natural images.
The iterative optimization process is shown (left, presented images; right, reconstructed images).
(MOV)
Deep image reconstruction: Imagery images.
The iterative optimization process is shown (left, imagined images; right, reconstructed images).
(MOV)
Correlations between feature decoding accuracy and reconstruction quality.
To investigate the relations between feature decoding accuracy and reconstruction quality, we first evaluated feature decoding accuracies for individual samples instead of those for individual DNN units (cf., S1 Fig; see Materials and Methods: “Evaluation of reconstruction q...
All examples of artificial shape reconstructions obtained from different visual areas (Subject 1).
The black and gray surrounding frames indicate presented and reconstructed images respectively (DNN 1–8, without the DGN).
(PDF)
DNN feature decoding accuracy.
DNN feature decoding accuracy obtained from VC activity was evaluated by the correlation coefficient between the true and decoded feature values of each feature unit following the procedure in Horikawa & Kamitani (2017) [10]. The evaluation was individually performed for each of the three types of seen images (natural...
Other examples of imagery image reconstructions.
The black and gray surrounding frames indicate presented and reconstructed images respectively (VC activity, DNN 1–8, without the DGN). The three rows of reconstructed images correspond to reconstructions from three subjects. The rightmost images in the bottom row show reconstructions during maintena...
Reconstruction quality of imagined artificial shapes for individual subjects.
Evaluations on individual subjects’ results are separately shown (VC activity; DNN 1–8; without the DGN; N = 15; chance level, 50%; cf., Fig 8D). Evaluations of reconstructions using pixel-wise spatial correlation showed 49.5%, 52.4%, and 53.8% for Subject 1–3, respective...
Reconstruction quality of imagined artificial shapes separately evaluated for color and shape by human judgment (reconstructed from V1).
Evaluations on individual subjects’ results and their pooled result are separately shown (V1 activity; DNN 1–8; without the DGN; N = 15 for individual subjects and N = 45 for the pooled result; chance level, 50%;...
Deep image reconstruction: Artificial shapes.
The iterative optimization process is shown (left, presented images; right, reconstructed images).
(MOV)
Examples of natural image reconstructions obtained with the DGN.
The black and gray surrounding frames indicate presented and reconstructed images respectively (VC activity, DNN 1–8, with the DGN). The three columns of reconstructed images correspond to reconstructions from three subjects. For copyright reasons, we present only a subset of the 50 t...
Reconstructions from the generic object decoding dataset.
The same reconstruction analysis was performed with a previously published dataset [10] (VC activity, DNN 1–8, with the DGN). See Horikawa & Kamitani (2017) [10] for details of the data. The black and gray surrounding frames indicate presented and reconstructed images respectively. The five...
Other examples of natural image reconstructions obtained without the DGN.
The black and gray surrounding frames indicate presented and reconstructed images respectively (VC activity, DNN 1–8, without the DGN). The three columns of reconstructed images correspond to reconstructions from three subjects.
(PDF)
Examples of reconstructions from individual DNN layers.
The black and gray surrounding frames indicate presented and reconstructed images respectively (without the DGN). We used DNN features from individual layers (DNN1, DNN2, …, or DNN8) as well as the combination of all DNN layers (DNN1–8) for the reconstruction analysis, in which either of true...
All examples of alphabetical letter reconstructions.
The black and gray surrounding frames indicate presented and reconstructed images respectively (VC activity, DNN 1–8, without the DGN). The three rows of reconstructed images correspond to reconstructions from three subjects.
(PDF)
Reconstruction quality of artificial shapes for individual subjects.
Evaluations on individual subjects’ results are separately shown (VC activity; DNN1–8; without the DGN; N = 40; chance level, 50%; cf., Fig 6C left). Evaluations of reconstructions using pixel-wise spatial correlation showed 69.6%, 72.1%, and 69.8% for Subject 1–3, respectively. E...
Reconstruction quality of imagined artificial shapes for individual subjects separately evaluated for color and shape by human judgment.
Evaluations on individual subjects’ results are separately shown (VC activity; DNN 1–8; without the DGN; N = 15; chance level, 50%; cf., Fig 8E). Evaluations of reconstructions with respect to color showed 71.1%,...
Reconstruction quality of imagined artificial shapes (reconstructed from V1).
Evaluations on individual subjects’ results and their pooled result are separately shown (V1 activity; DNN 1–8; without the DGN; N = 15 for individual subjects and N = 45 for the pooled result; chance level, 50%; cf., Fig 8D). Evaluations of reconstructions using pixel-wi...
Deep neural networks (DNNs) have recently been applied successfully to brain decoding and image reconstruction from functional magnetic resonance imaging (fMRI) activity. However, direct training of a DNN with fMRI data is often avoided because the size of available data is thought to be insufficient to train a complex network with numerous paramet...
Machine learning-based analysis of human functional magnetic resonance imaging (fMRI) patterns has enabled the visualization of perceptual content. However, it has been limited to the reconstruction with low-level image bases (Miyawaki et al., 2008; Wen et al., 2016) or to the matching to exemplars (Naselaris et al., 2009; Nishimoto et al., 2011)....
Previous research has demonstrated that there are specific white matter abnormalities in patients with attention deficit/hyperactivity disorder (ADHD). However, the results of these studies are not consistent, and one of the most important factors that affects the inconsistency of previous studies maybe the ADHD subtype. Different ADHD subtypes may...
Jing Zhang Du Lei Jun Ma- [...]
Xiaoxia Du
Nocturnal enuresis is a common developmental disorder in children; primary monosymptomatic nocturnal enuresis (PMNE) is the dominant subtype. Previous literature has suggested that the prefrontal cortex and the pons are both involved in micturition control. This study aimed to investigate the metabolic levels of the left prefrontal cortex and the p...
Multivariate pattern classification analysis (MVPA) has been applied to functional magnetic resonance imaging (fMRI) data to decode brain states from spatially distributed activation patterns. Decoding upper limb movements from non-invasively recorded human brain activation is crucial for implementing a brain–machine interface that directly harness...
Nocturnal enuresis is a common developmental disorder in children, and primary nocturnal enuresis (PNE) is the dominant subtype. The main purpose of this study was to investigate brain functional abnormalities specifically related to motor response inhibition in children with PNE using fMRI in combination with a Go/NoGo task. Twenty-two children wi...
Primary monosymptomatic nocturnal enuresis (PMNE) is a common disorder in school-aged children. Previous studies have suggested that a developmental delay might play a role in the pathology of children with PMNE. However, microstructural abnormalities in the brains of these children have not been thoroughly investigated.
In this work, we evaluated...
Primary monosymptomatic nocturnal enuresis (PMNE) is a common disorder in school-aged children. However, little is known about resting-state neural function in individuals with PMNE. In this work, resting-state functional magnetic resonance imaging (fMRI) was used to investigate changes in spontaneous brain activity in children with PMNE. We analyz...