ArticlePDF Available

Sense of Agency Beyond Sensorimotor Process: Decoding Self-Other Action Attribution in the Human Brain

Authors:

Abstract and Figures

The sense of agency is defined as the subjective experience that "I" am the one who is causing the action. Theoretical studies postulate that this subjective experience is developed through multistep processes extending from the sensorimotor to the cognitive level. However, it remains unclear how the brain processes such different levels of information and constitutes the neural substrates for the sense of agency. To answer this question, we combined two strategies: an experimental paradigm, in which self-agency gradually evolves according to sensorimotor experience, and a multivoxel pattern analysis. The combined strategies revealed that the sensorimotor, posterior parietal, anterior insula, and higher visual cortices contained information on self-other attribution during movement. In addition, we investigated whether the found regions showed a preference for self-other attribution or for sensorimotor information. As a result, the right supramarginal gyrus, a portion of the inferior parietal lobe (IPL), was found to be the most sensitive to self-other attribution among the found regions, while the bilateral precentral gyri and left IPL dominantly reflected sensorimotor information. Our results demonstrate that multiple brain regions are involved in the development of the sense of agency and that these show specific preferences for different levels of information.
(A) Trial timeline. After moving a cursor to the start position (shown as a square) during the 5-s ready period, participants traced a sinusoidal target-path with a cursor controlled by a joystick during the 10-s move period. Following a 6-s delay period, participants assigned a score (on a 9-point Likert scale) to their self-other attribution by pushing buttons. (B) Target path (top). Numbers were sequentially presented every second to help participants maintain the required pace of tracing. Cursor visibility (bottom). The cursor was invisible on the screen during the first 2.0 s (first cycle). Visibility linearly increased from zero to one over the next 2.0 s (second cycle). Here, zero corresponds to black (RGB: 0, 0, 0), which is the same brightness as the background, while one corresponds to white (RGB: 255, 255, 255). The cursor continued to be clearly visible during the next 4.0 s (third and fourth cycles) and then linearly became darker from 8.0 to 10 s (fifth cycle). (C) Morphing method. Cursor position on the screen (X, Y) was the weighted summation of the joystick position controlled by the current participant (self) (x, y) and a pre-recorded joystick position (other) (x', y'). Weights were modified by a morphing ratio (α). (D) Cursor trajectories. Circles labeled with numbers (1-5) illustrate how the cursor position was changed according to the five morphing ratios (0 ≤ α ≤ 1): self 0% (number 1) to self 100% (number 5) at every 25% step. In the self-other mixed conditions (number 2, 3, or 4), the cursor was displayed between the position of the participant's own joystick and the position of the other person's joystick.
… 
Relationship between tracing behavior and rating score of self-other attribution of movement. (A) Schematic of the four behavioral measures whose relationships with the self-other attribution score were investigated. Blue, green and red lines in the left panel indicate the target-cursor, target-joystick, and cursorjoystick distances, respectively. Light and dark gray arrows in the right panel denote the joystick and cursor velocity, respectively. Orange line represents the cursorjoystick velocity difference. (B) Time courses of Fisher-transformed Pearson's correlation coefficients between each behavioral measure and the self-other rating scores during the 10-s move period. Values of behavioral measures were averaged within every second. Colored shaded areas denote 95% confidence intervals. Bottom panel denotes visibility of the cursor during the move period. Hatched area denotes the period during which the cursor was invisible (i.e., cursor visibility was zero). Here, the data for self 50% condition are shown. Negative correlation indicates that the greater the behavioral measure became, the lower the score the participants gave (i.e., more other attribution). (C) Time courses of Fisher-transformed Pearson's correlation coefficients between accumulated value of each behavioral measure and rating scores. Values of behavioral measures were averaged from movement onset to every second. Colored shaded areas denote 95% confidence intervals. Hatched area denotes the period during which the cursor was invisible. Note that the data for self 50% condition are shown. Negative correlation indicates that the greater the behavioral measure was accumulated, the lower the score the participants gave.
… 
Content may be subject to copyright.
© The Author(s) 2020. Published by Oxford University Press.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/),
which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited
Cerebral Cortex, 2020;00: 1–16
doi: 10.1093/cercor/bhaa028
Advance Access Publication Date: 20 January 2001
Original Article
ORIGINAL ARTICLE
Sense of Agency Beyond Sensorimotor Process:
Decoding Self-Other Action Attribution in the
Human Brain
Ryu Ohata 1,2, Tomohisa Asai2, Hiroshi Kadota3,4, Hiroaki Shigemasu3,4,
Kenji Ogawa2,5 and Hiroshi Imamizu 1,2
1Department of Psychology, Graduate School of Humanities and Sociology, The University of Tokyo,
Bunkyo-ku, Tokyo 113-0033, Japan, 2Department of Cognitive Neuroscience, Cognitive Mechanisms
Laboratories, Advanced Telecommunications Research Institute International (ATR), Keihanna Science City,
Kyoto 619-0288, Japan, 3School of Information, Kochi University of Technology, Kami, Kochi 782-8502, Japan,
4Research Institute, Kochi University of Technology, Kami, Kochi 782-8502, Japan and 5Department of
Psychology, Graduate School of Humanities and Human Sciences, Hokkaido University, Sapporo, Hokkaido
060-0810, Japan
Address correspondence to Hiroshi Imamizu. Email: imamizu@gmail.com; Ryu Ohata. Email: ryu.oohata@gmail.com
Ryu Ohata and Tomohisa Asai have contributed equally to this work
Abstract
The sense of agency is defined as the subjective experience that “I” am the one who is causing the action. Theoretical
studies postulate that this subjective experience is developed through multistep processes extending from the
sensorimotor to the cognitive level. However, it remains unclear how the brain processes such different levels of
information and constitutes the neural substrates for the sense of agency. To answer this question, we combined two
strategies: an experimental paradigm, in which self-agency gradually evolves according to sensorimotor experience, and a
multivoxel pattern analysis. The combined strategies revealed that the sensorimotor, posterior parietal, anterior insula, and
higher visual cortices contained information on self-other attribution during movement. In addition, we investigated
whether the found regions showed a preference for self-other attribution or for sensorimotor information. As a result, the
right supramarginal gyrus, a portion of the inferior parietal lobe (IPL), was found to be the most sensitive to self-other
attribution among the found regions, while the bilateral precentral gyri and left IPL dominantly reflected sensorimotor
information. Our results demonstrate that multiple brain regions are involved in the development of the sense of agency
and that these show specific preferences for different levels of information.
Key words: functional magnetic resonance imaging, inferior parietal lobe, multivoxel pattern analysis, sense of agency,
supramarginal gyrus
Introduction
How the brain makes us aware of our selfhood, as an individual
separate from other individuals, is a long-standing question in
the field of neuroscience. The sense of agency is defined as a
subjective experience that “I” am the one who is causing or
generating an action (Gallagher 2000;Haggard 2017). This def-
inition illustrates the interaction between body and environ-
ment (i.e., a sensorimotor process), specifying the self as the
subject of action and perception (Legrand 2007;Legrand and
Downloaded from https://academic.oup.com/cercor/advance-article-abstract/doi/10.1093/cercor/bhaa028/5775460 by guest on 04 May 2020
2Cerebral Cortex, 2020, Vol. 00, No. 00
Ruby 2009;Christoff et al. 2011). Previous neuroimaging studies
have reported multiple brain regions associated with the sense
of agency, such as the supplementary motor area (Tsakiris et al.
2010;Yomogida et al. 2010;Miele et al. 2011), the cerebellum
(Blakemore et al. 2001;Yomogida et al. 2010), the posterior
parietal cortex (Farrer and Frith 2002;Farrer et al. 2003,2008;
Ogawa and Inui 2007;Schnell et al. 2007;Spengler et al. 2009;
Yomogida et al. 2010;Miele et al. 2011;Nahab et al. 2011;Cham-
bon et al. 2013;Fukushima et al. 2013;Beyer et al. 2018), the
lateral prefrontal cortex (Schnell et al. 2007;Nahab et al. 2011;
Chambon et al. 2013), the higher visual cortex (Astafiev et al.
2004;David et al. 2007;Yomogida et al. 2010), and the insula
(Farrer et al. 2003;Tsakiris et al. 2010;Fukushima et al. 2013)
(see also reviews and meta-analysis studies in David et al. 2008;
Miele et al. 2011;Sperduti et al. 2011). Most of these studies
manipulated the discrepancy between participants’ own actions
and the sensory consequences for controlling the participants’
attribution of the observed action to the self or to another.
Then, they made contrasts between the self-attribution condi-
tion (the observed action is attributed to oneself) and the other-
attribution condition (the action is attributed to another). These
studies indicated that the reported regions are recruited at a
certain stage of the sense of agency; however, it has not been
clarified how the brain regions are involved in the process to
develop sensorimotor information into agency attribution.
Theoretical models provide a clue for elucidating the neural
process behind the sense of agency grounded on sensorimotor
information. The comparator model, one of the most influential
models of the sense of agency, has suggested the importance
of sensorimotor processing for agency attribution (Fig. 1 left).
This model suggests that the brain compares predicted and
actual sensory consequences of an action (Miall and Wolpert
1996;Blakemore et al. 1998,2000). The result of this compari-
son, known as a prediction error, determines whether people
attribute the observed action to their own or to another agent
(Blakemore et al. 2000;Frith et al. 2000). As the comparator
model highlights the sensorimotor process (i.e., calculation of
sensory prediction error), there exists a gap between the sen-
sorimotor process and agency attribution. Synofzik et al. (2008)
postulated the necessity of an intermediate process, named a
nonconceptual feeling of agency, to take over the outcome of
the comparator model and develop the information needed to
achieve a conceptual judgment of agency (Fig. 1 right, see also
Synofzik et al. 2013). Summarizing the above, the two theoretical
models suggest that there need to be multistep processes for
the sense of agency extending from the lower sensorimotor to
higher cognitive level (Fig. 1).
The current study reflected the theoretical models’ implica-
tions in the hypothesis regarding the neural process. Namely, we
hypothesized that there exists gradation in neural information
from the lower sensorimotor to higher cognitive level (Fig. 1,yel-
low gradation): Some brain regions preferentially represent the
information closely related to agency attribution, while others
represent the immediate output of sensorimotor processing. We
tested the hypothesis by combining the following two strate-
gies. First, we used an experimental paradigm in which self-
or other agency gradually evolves according to the amount of
sensorimotor experience (Nahab et al. 2011;Asai 2016). Partici-
pants continuously traced a target path by controlling a joystick
under ambiguous conditions of agency. That is, we morphed
visual feedback of the movement (a cursor position) by incor-
porating another person’s pre-recorded movement into the par-
ticipant’s online movement. Accordingly, they could gradually
perceive whether sensory feedback was attributed to self- or
other control (self-other attribution). The second strategy was
multivoxel pattern analysis (MVPA) of functional magnetic res-
onance imaging (fMRI) data (Haynes and Rees 2005;Kamitani
and Tong 2005;Norman et al. 2006). MVPA makes it possible to
explore neural information represented in distinct patterns of
fMRI voxel signals (Haynes 2015;Hebart and Baker 2018). In the
current study, we separately decoded self-other attribution and
sensorimotor information (e.g., sensory prediction error), which
was correlated with self-other attribution, and examined the
preference for agency attribution (or for sensorimotor informa-
tion) in the found regions. As a result, we found that the inferior
parietal lobe (IPL), sensorimotor, anterior insula, and higher
visual cortices contained the information that determined self-
other attribution. Among the found regions, the right supra-
marginal gyrus (SMG) showed the highest preference for self-
other attribution, compared with sensorimotor information, at
the final stage of a movement. We acknowledge that the sense of
agency is associated with not only sensorimotor information but
also with external cues such as subliminal/supraliminal priming
(Moore et al. 2009;Wenke et al. 2010;Chambon et al. 2013)or
social context (Beyer et al. 2018),andthatitisfinallydeter-
mined as a result of integrating multiple sources of information
(Pacherie 2007;Moore and Fletcher 2012;Synofzik et al. 2013).
Although multiple sources other than sensorimotor information
also play a crucial role in the sense of agency, the current study
focuses on the sense of agency grounded on the sensorimotor
system illustrated in Figure 1.
Materials and Methods
Participants
Eighteen right-handed and healthy volunteers (six females) with
a mean age of 25.9 years (20–42 years) participated in our exper-
iment. Our previous study (Asai 2016), which adopted a similar
experimental paradigm as a behavioral study, found a main
effect of a morphing ratio condition on attribution judgment (for
details of the experimental paradigm, see below). We calculated
the sample size for our behavioral data because it is critical in
the current study to obtain the effect of the morphing condi-
tion. We calculated this based on a power analysis for repeated
measures analysis of variance (ANOVA) using Gpower 3.1 with
power selected at 0.8, effect size (f) at 0.4, and alpha at 0.05.
According to the requirements of this analysis, sample size for
behavioral data was nine participants. Eventually, we chose the
sample size of this study to be 18 based on the estimated sample
size for behavioral data and those used in previous fMRI studies
of the sense of agency (David et al. 2007;Farrer et al. 2008;Nahab
et al. 2011). Written informed consent was obtained from all of
the volunteers in accordance with the latest version of the Dec-
laration of Helsinki. The experimental protocol was approved by
the ethics committee of Kochi University of Technology.
Behavioral Task
Trial Timeline
Participants were required to trace a five-cycle sinusoidal wave
(target path) with a cursor (Fig. 2A)(Asai 2016). They manipu-
lated a joystick with their right index finger to control the cursor
on the screen.
Cue and ready periods. At the beginning of each trial, the trial
number was displayed on the screen in a cue period of 1 s.While
listening to four countdown sounds, participants set their cursor
Downloaded from https://academic.oup.com/cercor/advance-article-abstract/doi/10.1093/cercor/bhaa028/5775460 by guest on 04 May 2020
Sense of Agency Beyond Sensorimotor Process Ohata et al. 3
Figure 1. Multistep processes behind the sense of agency extending from lower sensorimotor to higher cognitive level processing. This schematic is an overview
combing the two influential theories of the sense of agency: 1) the comparator model (Miall and Wolpert 1996;Blakemore et al. 1998,2000) and 2) the two-step account
of agency (Synofzik et al. 2008,2013). Considering the two theoretical models together, agencyattribution is achieved through the multistep processes extending from
lower sensorimotor to higher cognitive level.The hypothesis in the current study is that some brain regions represent the immediate output of sensorimotor processing,
while others represent the information directly leading to agency attribution. The background yellow gradation depicts the level of the information represented in
the brain from sensorimotor to cognitive level, which is the main target of the current study. In Synofzik et al. (2008), the sense of agency encompasses two levels
of representation: a nonconceptual feeling and a conceptual judgment of agency. The term “agency attribution” (or “self-other attribution”) in the current study
corresponds to a conceptual judgment of agency. We assume that the process leading to the conceptual judgment of agency (not including the judgment process
itself) based on the lower sensorimotor information is a nonconceptual “feeling of agency.” We use the term “sense of agency”to include both a nonconceptual feeling
and a conceptual judgment of agency.
at the starting point located near the lower-left corner of the
screen within a ready period of 5 s.
Move period. As soon as the target path was presented on
the screen, participants started tracing it toward the goal point
located near the lower-right corner of the screen. They were
required to trace each half cycle of the target path in 1 s (i.e.,
0.5 Hz) and to complete the entire movement within a move
period of 10 s. We sequentially displayed numbers from 1 to
10 every second at the top and bottom of the sinusoidal target-
path to help participants maintain the required pace of tracing
(Fig. 2Btop and Supplementary Movie 1). The cursor position
on the screen was determined by the weighted summation of
the participant’s online joystick and the pre-recorded other’s
joystick position (for details see Morphing Visual Feedback from
Self to Other). Each participant was instructed that the cursor
movement on the screen corresponded to the participant’s own
or to someone else’s joystick movement and that he or she
should trace the target path with the cursor as accurately and as
smoothly as possible. Note that the cursor was invisible on the
screen during the first 2.0 s of the move period (first cycle) and
then gradually appeared (i.e., brightness of the cursor linearly
increased) during the next 2.0 s (second cycle). Furthermore,
the cursor gradually disappeared (i.e., brightness of the cursor
linearly decreased) from 8.0 to 10 s (fifth cycle) (Fig. 2Bbottom
and Supplementary Movie 1). The reason for this cursor-visibility
control is that the onset and the offset of the cursor move-
ment are sensitive to the mismatch between the participants’
own joystick and the cursor movements, which predominantly
affects self-other attribution judgment (i.e., temporal delay and
spatial deviation at the onset/offset timing might strongly affect
judgment).
Delay and rating periods. A blank screen was presented during
a 6-s delay period after the move period. Then participants
reported how much they felt the cursor movement could be
attributed to their own joystick movement on a 9-point Likert
scale from 1 (completely the other’s movement) to 9 (completely
their own movement). The number 5 was displayed on the
screen at the beginning of a rating period for 8 s. The number
was incremented or decremented by pressing the right or left
button, respectively. The buttons were attached to the joystick
box, and participants were instructed to press the buttons with
their right hand.
Morphing Visual Feedback from Self to Other
Visual feedback (i.e., cursor movement) during the tracing
movement was morphed by incorporating another person’s
movement into the participant’s movement. We calculated
the weighted summation of the participant’s online joystick
position (x,y) and the other’s position (x’, y’) at 60 Hz (refresh
rate of the monitor) and displayed the cursor in the calculated
position (X,Y) on the screen (Fig. 2C). This weight corresponded
to the morphing ratio (α). The other persons’ movements were
recorded prior to the fMRI experiment, and 240 trajectories
(15 trajectories recorded from each of 16 participants) were
stored in a dataset. A trajectory was randomly chosen for each
trial from this dataset. Five morphing ratios were set at 25%
intervals, from self 0% (other 100%, α=0) to self 100% (other 0%,
α=1.0). In the self 100% condition, the visible cursor position
fully corresponded to the participant’s joystick position (cursor
labeled number 5 in Fig. 2D). By contrast, in the self 0% condition,
the visible cursor position was independent of the participant’s
own joystick position (cursor labeled number 1 in Fig. 2D). In
the self-other mixed conditions, the cursor was displayed at a
position between the position of the participant’s own joystick
and that of the other person’s pre-recorded joystick (cursor
labelednumber2,3,or4inFig. 2D).
Experimental Procedure
Before the main fMRI runs, participants performed two types
of practice runs inside the fMRI scanner. In the first practice
run, the participants were trained to trace the target path with
a cursor moving in accordance with 1-Hz metronomic sounds
to become accustomed to the cyclic movement. In this run, the
cursor movement precisely reflected their joystick movement
(self 100% condition). In the second practice run, they conducted
the same task as the main fMRI runs but with a smaller number
of trials (10 trials) than that of the main runs (50 trials/run). After
Downloaded from https://academic.oup.com/cercor/advance-article-abstract/doi/10.1093/cercor/bhaa028/5775460 by guest on 04 May 2020
4Cerebral Cortex, 2020, Vol. 00, No. 00
Figure 2. (A) Trial timeline.After moving a cursor to the start position (shown as a square) during the 5-s ready period, participants traced a sinusoidal target-path with
a cursor controlled by a joystick during the 10-s move period. Following a 6-s delay period, participants assigned a score (on a 9-point Likert scale) to their self-other
attribution by pushing buttons. (B) Target path (top). Numbers were sequentially presented every second to help participants maintain the required pace of tracing.
Cursor visibility (bottom). The cursor was invisible on the screen during the first 2.0 s (first cycle). Visibility linearly increased from zero to one over the next 2.0 s
(second cycle). Here, zero corresponds to black (RGB: 0, 0, 0), which is the same brightness as the background, while one corresponds to white (RGB: 255, 255, 255). The
cursor continued to be clearly visible during the next 4.0 s (third and fourth cycles) and then linearly became darker from 8.0 to 10 s (fifth cycle).(C) Morphing method.
Cursor position on the screen (X,Y) was the weighted summation of the joystick position controlled by the current participant (self) (x,y) and a pre-recorded joystick
position (other) (x’, y’). Weights were modified by a morphing ratio (α). (D) Cursor trajectories. Circles labeled with numbers (1–5) illustrate how the cursor position was
changed according to the five morphing ratios (0 α1): self 0% (number 1) to self 100% (number 5) at every 25% step. In the self-other mixed conditions (number 2,
3, or 4), the cursor was displayed between the position of the participant’s own joystick and the position of the other person’s joystick.
Downloaded from https://academic.oup.com/cercor/advance-article-abstract/doi/10.1093/cercor/bhaa028/5775460 by guest on 04 May 2020
Sense of Agency Beyond Sensorimotor Process Ohata et al. 5
the practice runs, participants conducted three main runs/day
(150 trials) for a total of six runs (300 trials) over 2 days. The
participants performed the task using the five morphing ratios
10 times in random order during each of the main runs.
MRI Data Acquisition
A 3-T Magnetom Verio scanner (Siemens) with a 32-channel
head coil was used to acquire T2-weighted echo-planar images
(EPI). In total, 753 volumes were acquired in each run with
a gradient echo EPI sequence under the following scanning
parameters: repetition time (TR), 2000 ms; echo time (TE),30 ms;
flip angle (FA), 70; field of view (FOV), 192×192 mm; matrix,
64 ×64; 30 axial slices; and thickness, 4 mm with a 1-mm gap.T2-
weighted turbo spin echo images were scanned to acquire high-
resolution anatomical images of the same slices used for the
EPI (TR, 6000 ms; TE, 58 ms; FA, 160; FOV, 192 ×192 mm; matrix,
256 ×256; 30 axial slices; and thickness, 4 mm with a 1-mm gap).
T1-weighted structure images were obtained with 1 ×1×1-mm
resolution with a gradient echo sequence (repetition time,
2250 ms; echo time, 3.06 ms; flip angle, 9; matrix, 256 ×256;
192 axial slices; and thickness, 1 mm without gap).
Preprocessing of fMRI Data
The fMRI data were analyzed using SPM8 (Wellcome Trust Cen-
tre for Neuroimaging, London, UCL) on MATLAB. We discarded
the first three volumes of the functional images in each run
to allow for T1 equilibration. The remaining image volumes
were temporally realigned to correct for the sequence of slice
acquisition and then spatially realigned to the first image to
adjust for motion-related artifacts. Rigid-body transformations
were performed to align the functional images to the structural
image for each subject. The images were spatially normalized
with the Montreal Neurological Institute (MNI) (Montreal, Que-
bec, Canada) reference brain and resampled into 3×3×4-mm
cuboid voxels. Note that spatial smoothing was not applied to
the data, since this might blur the fine-grained information
contained in multivoxel activity (Mur et al. 2009). After linear-
trend removal within each run, we calculated the percentage of
signal change relative to the mean of activity for each run.
Decoding Self-Other Attribution with Multivoxel
Pattern Regression
We performed multivoxel pattern regression to decode a
self-other rating score (ranging from 1 to 9) from the fMRI
activity patterns during movement. A linear support vector
regression (SVR) model implemented in LIVSVM (http://www.
csie.ntu.edu.tw/~cjlin/libsvm/) was applied to the voxel patterns
with the trial-by-trial rating score as a dependent variable.
The SVR model was trained using the data from four out of
five morphing conditions and then tested using the data in
the remaining condition (i.e., leave-one-condition-out cross
validation) to prevent the differences among conditions from
becoming a confounding factor. In addition, we evaluated
the decoding performance using a leave-one-run-out cross-
validation procedure to prevent the differences among runs
from becoming a confounding factor. More specifically, we
trained the model with the fMRI data from four out of five
conditions in five out of six runs and tested it with the
independent data of the remaining condition in the remaining
run (e.g., the fifth condition in the sixth run). This procedure
was repeated 30 times (five conditions times six runs) so that
each condition in each run was used as test data once. We
performed a volume-based searchlight analysis using voxels
within a 9-mm-radius sphere (see Searchlight Decoding Over
the Brain) extracted from each volume of fMRI data scanned
every 2 s (TR= 2 s) during the move periods. Here, each volume
corresponded to a cycle of the sinusoidal movement.
Evaluation of Decoding Accuracy in Individual Analyses
We evaluated the above decoding accuracy in the test phase by
calculating the z-scores of the Fisher-transformed Pearson’s
correlation coefficient following the permutation procedure
(Langfelder et al. 2011;Shibata et al. 2016). We generated
1000 surrogate correlation coefficients by permutating the
relationship between the actual and predicted value 1000 times
to get an empirical distribution of the correlation coefficients.
The z-scores of the original (without permutation) value was
calculated based on the empirical distribution. The above steps
were then applied to the test dataset in each condition. We
regarded z-scores averaged across conditions as indicative of
the decoding performance by each participant.
Searchlight Decoding Over the Brain
We performed a volume-based searchlight decoding analysis
(Kriegeskorte et al. 2006;Haynes et al. 2007). We repeatedly
extracted voxel patterns within a 9-mm-radius sphere con-
taining at least 65 voxels to perform regression analysis. This
sphere was moved over the gray matter of the entire brain, and
the mean of the z-scores was assigned to the sphere’s central
voxel, resulting in a 3-D z-score map for each participant. A
random-effects group analysis was performed on the z-score
maps by using SPM8. To satisfy the assumptions of Gaussian
random field theory for statistical inference at the group level,
the z-score maps were smoothed with a 4-mm full-width at
half-maximum (FWHM) Gaussian kernel (Soon et al. 2008;Bode
and Haynes 2009;Wisniewski et al. 2015,2016). We applied a
statistical analysis of the entire brain with a threshold of P<0.01
(family-wise error (FWE) corrected at cluster level with a cluster-
forming threshold of P<0.0005). The anatomical localization
was determined according to the automated anatomical labeling
(AAL) atlas (Tzourio-Mazoyer et al. 2002).
Decoding Cursor-Joystick Distance and Velocity
Difference with Multivoxel Pattern Regression
We performed multivoxel pattern regression to investigate
whether the brain regions contained enough information
to predict sensorimotor information (distance or velocity
difference between the cursor and joystick; for details and
definition, see Results: Relationship Between Tracing Behavior
and Rating Score of Self-Other Attribution). Note that we used
the mean of the distance or velocity difference from 4 to 8 s
after the onset of the move period in order to exclude periods
when the cursor was not clearly visible on the screen (for details
see Behavioral Task). The procedure for decoding sensorimotor
information was almost the same as that for decoding self-other
attribution. The only exception was that the dataset in the self
100% condition was not included in the calculation of decoding
performance (i.e., a correlation coefficient between actual and
predicted value) because the cursor-joystick distance or velocity
difference was zero in this condition. We applied a volume-
based searchlight analysis to each volume of fMRI data scanned
every 2 s during the move period. We assigned z-scores of the
Downloaded from https://academic.oup.com/cercor/advance-article-abstract/doi/10.1093/cercor/bhaa028/5775460 by guest on 04 May 2020
6Cerebral Cortex, 2020, Vol. 00, No. 00
Figure 3. Self-other attribution rating scores averaged across participants for
each morphing ratio. The higher the score was, the more strongly participants
felt that the cursor movement was attributed to their own joystick movement.
Error bars indicate standard error of the mean. A linear regression model was
fitted to each participant’s rating scores. The regression lines of all participants
are shown behind the bars.
Fisher-transformed Pearson’s correlation coefficient calculated
by the permutation procedure (see Evaluation of Decoding
Accuracy in Individual Analysis) to the sphere’s central voxel.
Next, we performed group analysis on the smoothed 3-D z-score
maps for each participant.
Results
Self-Other Rating Score on Morphing Ratio Condition
Figure 3 shows the rating score averaged across all participants
as a function of the morphing (self-movement) ratio. We found
a significant main effect of the morphing ratio (F(4, 85) = 151.2,
P<0.001) according to a one-way repeated-measures ANOVA
with the five morphing ratios as a within-subject factor. The
rating scores linearly increased as the ratio of self-movement
increased, which was also found in a previous study (Asai
2016). The lines behind bars represent regression lines fitted to
each participant’s rating scores: (rating score)= w0+w1×(self-
movement ratio). The mean of the line slopes (w1) was 0.049
(SD: 0.015), which was significantly larger than zero (two-tailed
t-test: t(17) = 13.6, P<0.001).
Relationship Between Tracing Behavior and Rating
Score of Self-Other Attribution
We investigated the relationship between trial-by-trial rating
scores and behavioral measures to specify the sensorimotor
factors that affected the self-other attribution of the partici-
pants. We examined four behavioral measures as possible fac-
tors: 1) the target-cursor distance, which is the vertical distance
between the target path and the cursor position (blue line in
left panel of Fig. 4A), 2) the target-joystick distance, which is
the vertical distance between the target path and the joystick
position (green line in left panel of Fig. 4A), 3) the cursor-joystick
distance, which is the Euclidean distance between the cursor
and the joystick positions (red line in left panel of Fig. 4A), and
4) the cursor-joystick velocity difference. The velocity difference
was the norm of the difference between the cursor and the
joystick velocities: (vxc vxj)2+(vyc vyj)2,wherevxc and vyc
are x-andy-direction velocities of the cursor, respectively, and
vxj and vyj are those of the joystick, respectively (orange line
in right panel of Fig. 4A). We calculated the Fisher-transformed
Pearson’scorrelation coefficients between each behavioral mea-
sure (mean value within every second) and self-other rating
scores (one value for each trial).
Figure 4Bshows the time course of the correlation for the
self 50% condition (Supplementary Fig. 1 shows time courses for
all conditions). In the self 50% condition, the upper limit of the
95% confidence interval (CI) of the cursor-joystick distance was
lower than zero from 2 s after the onset of movement (red line
in Fig. 4B). Similarly, the 95% CI of the velocity difference was
less than zero from 3 s after the onset of movement (orange line
in Fig. 4B). This negative correlation indicates that the greater
the cursor-joystick distance or velocity difference was, the more
likely were the participants to judge the cursor movements to
be attributed to the other’s motion, and vice versa.
As shown in the bottom panel of Figure 4B, we controlled the
cursor visibility to avoid participants’ highly sensitive reaction
to the initial and final mismatch between their joystick and
the cursor movements. Since the cursor visibility was different
among the periods between 0 and 2 s (first cycle), between 2
and 4 s (second cycle), between 4 and 8 s (third and fourth
cycles), and between 8 and 10 s (fifth cycle), the correlation
coefficients for the four periods were not considered compa-
rable. The correlations were nearly zero between any of the
behavioral measures and the rating score in the first cycle.
This result is reasonable because the cursor was completely
invisible in this period. The cursor-joystick distance and veloc-
ity difference were highly correlated with each other: Pear-
son’s correlation coefficients averaged across participants were
0.85 (SD: 0.10), 0.83 (0.10), 0.72 (0.17), and 0.38 (0.22) in the self
75%, 50%, 25%, and 0% conditions, respectively. Note that we
used the mean of the distance or velocity difference from 4
to 8 s after the onset of the move period. This high correla-
tion seems reasonable because these measures are not inde-
pendent but determined by the relationship between the cur-
sor and joystick movements, which largely affected the rating
score.
Figure 4Cshows the time courses of the correlation between
the rating score and the accumulated value of each measure,
which was averaged from movement onset to each second. The
negative correlation gradually became larger according to the
distance or velocity-difference accumulation (red and orange
lines in Fig. 4Cshown for self 50% condition). The time courses
for all conditions are shown in Supplementary Fig. 2.Thisfind-
ing indicates that the accumulation of distance or velocity differ-
ence between cursor and joystick is essential for the judgment
of self-other attribution. By contrast, the correlation coefficients
for the target-cursor and target-joystick distances were stable
around zero (blue and green lines in Fig. 4B,C).
Decoding Self-Other Attribution During Movement
We decoded self-other attribution of cursor movement, which
was evaluated by the participants after movement, from fMRI
voxel patterns during the tracing. A searchlight analysis found
clusters in which self-other attribution could be significantly
decoded from their voxel patterns (red regions in Fig. 5,P<0.01
FWE-corrected at cluster level with a cluster-forming threshold
Downloaded from https://academic.oup.com/cercor/advance-article-abstract/doi/10.1093/cercor/bhaa028/5775460 by guest on 04 May 2020
Sense of Agency Beyond Sensorimotor Process Ohata et al. 7
Figure 4. Relationship between tracing behavior and rating score of self-other attribution of movement. (A) Schematic of the four behavioral measures whose
relationships with the self-other attribution score were investigated. Blue, green and red lines in the left panel indicate the target-cursor, target-joystick, and cursor-
joystick distances, respectively. Light and dark gray arrows in the right panel denote the joystick and cursor velocity, respectively. Orange line represents the cursor-
joystick velocity difference. (B) Time courses of Fisher-transformed Pearson’scorrelation coeff icients betweeneach behavioral measure and the self-other rating scores
during the 10-s move period. Values of behavioral measures were averaged within every second.Colored shaded areas denote 95% confidence intervals. Bottom panel
denotes visibility of the cursor during the move period. Hatched area denotes the period during which the cursor was invisible (i.e., cursor visibility was zero). Here,
the data for self 50% condition are shown. Negative correlation indicates that the greater the behavioral measure became, the lower the score the participants gave
(i.e., more other attribution). (C) Time courses of Fisher-transformed Pearson’s correlation coeff icients between accumulated value of each behavioral measure and
rating scores. Values of behavioral measures were averaged from movement onset to every second. Colored shaded areas denote 95% confidence intervals. Hatched
area denotes the period during which the cursor was invisible. Note that the data for self 50% condition are shown. Negative correlation indicates that the greater the
behavioral measure was accumulated, the lower the score the participants gave.
Downloaded from https://academic.oup.com/cercor/advance-article-abstract/doi/10.1093/cercor/bhaa028/5775460 by guest on 04 May 2020
8Cerebral Cortex, 2020, Vol. 00, No. 00
Figure 5. Decoding performance for self-other attribution during movement, with clusters of significant decoding accuracy (P<0.01 FWE corrected at cluster level with
a cluster-forming threshold of P<0.0005). A searchlight decoding analysis was applied to a volume scanned every 2 s during the 10-s move period to create accuracy
maps. The sinusoidal waves represent a typical cursor movement along the timeline shifted by 6 s from the actual time considering the hemodynamic response delay
(HRD). All clusters larger than 50 voxels are reported.IFG: inferior frontal gyrus, IPL: inferior parietal lobe, pre CG: precentral gyrus, MOG: middle occipital gyrus, MTG:
middle temporal gyrus, SMG: supramarginal gyrus, STG: superior temporal gyrus.
Tab l e 1 Summary of searchlight decoding of self-other attribution of movement during move period
Brain region Side Cluster size MNI coordinates (peak voxel)
xyz
Cycle 2
1. Inferior frontal gyrus Left 115 51 11 26
2. Inferior parietal lobe Left 52 39 58 54
Cycle 3
3. Inferior parietal lobe Left 147 42 52 38
4. Precentral gyrus Left 198 39 138
5. Precentral gyrus Right 193 36 10 50
6. Superior temporal gyrus Right 56 57 43 18
Cycle 4
7. Precentral gyrus Left 367 45 13 50
8. Inferior parietal lobe Left 81 39 58 50
9. Precentral gyrus Right 339 33 10 58
10. Insula Right 79 33 17 6
11. Supramarginal gyrus Right 77 57 46 26
Cycle 5
12. Precentral gyrus Left 82 36 10 54
13. Middle occipital gyrus Left 150 36 73 22
14. Middle temporal gyrus Right 84 54 58 14
15. Supramarginal gyrus Right 50 60 34 30
Note: A threshold at P<0.05 (FWE-corrected at cluster level with a cluster-forming threshold of P<0.0005) was set for statistical testing. Clusters larger than 50 voxels
are reported. Cycles correspond to those illustrated at the bottom of Figure 5, and they are shifted by 6 s from the actual time considering the HRD.
of P<0.0005; all clusters are reported in Table 1). At first, the
clusters in the left inferior frontal gyrus and IPL appeared in the
second cycle of the move period. Then, the bilateral precentral
gyrus, right superior temporal gyrus, right IPL (mainly the SMG),
and right anterior insula showed significant decoding accura-
cies from the third to fourth cycles. In the last cycle, the left
middle occipital gyrus and right middle temporal gyrus showed
significant accuracies. These results indicate that the infor-
mation that could predict the following self-other attribution
is contained in the regions reported as the neural correlates
of the sense of agency by the previous studies, such as the
posterior parietal (Farrer and Frith 2002;Farrer et al. 2003,2008;
Downloaded from https://academic.oup.com/cercor/advance-article-abstract/doi/10.1093/cercor/bhaa028/5775460 by guest on 04 May 2020
Sense of Agency Beyond Sensorimotor Process Ohata et al. 9
Figure 6. (A) Clusters showing significant decoding accuracy for cursor-joystick distance and (B) those for cursor-joystick velocity difference (blue regions; P<0.01 FWE-
corrected at cluster level with a cluster-forming threshold of P<0.0005). A searchlight decoding analysis (Kriegeskorte et al. 2006) was applied to a volume scanned
every 2 s during the 10-s move period to create accuracy maps. The sinusoidal waves represent a typical cursor movement along the timeline shifted by 6 s from the
actual time considering the HRD. All clusters larger than 50 voxels are reported.
Ogawa and Inui 2007;Schnell et al. 2007;Yomogida et al. 2010;
Nahab et al. 2011), sensorimotor (David et al. 2008;Sperduti
et al. 2011), anterior insula (Farrer et al. 2003;Tsakiris et al.
2010), and higher visual cortices (Astafiev et al. 2004;David
et al. 2007;Yomogida et al. 2010). We also showed that the
neural representation shifted from region to region during the
tracing.
When assessing an individual’s self-other discriminability
by fitting the linear regression model to each participant’s
rating scores (regression lines in Fig. 3), we found some of the
participants showing relatively low and unstable discriminabil-
ity. Therefore, as an additional analysis, we examined whether
the poor discriminability affected our decoding result. We
excluded seven (out of 18) participants with relatively low
and unstable discriminability according to certain criteria (see
Supplementary Results) and performed a random-effects group
analysis on the z-score maps of the remaining participants.
Consequently, we found a result similar to that of the 18
Downloaded from https://academic.oup.com/cercor/advance-article-abstract/doi/10.1093/cercor/bhaa028/5775460 by guest on 04 May 2020
10 Cerebral Cortex, 2020, Vol. 00, No. 00
participants (for details see Supplementary Results: Exclusion
of participants with low and unstable discriminability of action
attribution and Supplementary Fig. 3).
Preference for Self-Other Attribution or for
Sensorimotor Information
We next decoded the sensorimotor information that was
correlated with self-other attribution (i.e., cursor-joystick
distance and velocity difference, Fig. 4) from fMRI voxel patterns
using a searchlight analysis. As a result, we were able to
decode the cursor-joystick distance in many regions (Fig. 6A,
P<0.01 FWE-corrected at cluster level with a cluster-forming
threshold of P<0.0005), including some of the clusters shown in
Figure 5. Similar results were obtained for the velocity difference
(Fig. 6B). Note that we could not find any cluster showing
significant decoding performance in the first cycle. This is
reasonable because the cursor was not displayed for the first
2s(Fig. 2B). These results indicate the possibility that decoding
performance in some clusters in Figure 5 more dominantly
reflected sensorimotor information than self-other attribution.
We assessed whether the voxel patterns in each cluster in
Figure 5 were more sensitive to self-other attribution than to
sensorimotor information or vice versa as follows. We computed
difference (diff) in decoding performance (measured by z-score,
see “Evaluation of Decoding Accuracy in Individual Analysis”
in Materials and Methods) between self-other attribution and
sensorimotor information. Here, diff was calculated for each
participant (i=1, 2, ... 18) and each cluster (j=1,2, ... 15):
diff i,j=z-scoreself-other attribution i,j
z-scoresensorimotor information i,j.(1)
As mentioned above, the cursor-joystick distance highly cor-
related with velocity difference. Thus, we compared the mean
z-score across participants between the two measures for each
cluster and chose the higher one as z-scoresensorimotor information for
the cluster (j) to simplify further analysis. Note that the mean
z-score was higher for the cursor-joystick distance than for the
velocity difference in all clusters except cluster 15 (right SMG,
see Tab l e 1). We computed mean diff across participants for
each cluster (Mdiff(j)) and divided this by the standard deviation
(SDdiff(j)) to get the effect size (Cohen’s dz,Lakens 2013):
Cohensdz(j)=Mdiff(j)SDdiff(j),(2)
where SDdiff(j)=18
i=1(diff(i,j)Mdiff (j))2
181. Importantly, this effect
size has a sign. The positive sign means that the cluster (j)is
more sensitive to self-other attribution than to sensorimotor
information, while the negative sign means the opposite case
(see eq. 1).
Figure 7Arepresents the signed effect sizes of the 15 clusters
using a color code. Reddish colors indicate a bias toward the
self-other attribution (positive sign), while bluish colors indicate
a bias toward the sensorimotor information (negative sign).
Figure 7Bshows the effect sizes of the clusters sorted in ascend-
ing order (see Supplementary Fig. 4 for decoding performances
for self-other attribution and sensorimotor information in all
clusters). According to Figure 7B, the areas near the central sul-
cus, including the bilateral precentral gyrus (clusters 5, 7, and 9)
and left IPL (cluster 3), showed prominent biases toward senso-
rimotor information. By contrast, effect sizes in the right SMG
in the fourth and fifth cycles (clusters 11 and 15) showed the
highest and second-highest values among the clusters, respec-
tively. Consistent with this result, the decoding performance in
the right SMG was significantly higher for self-other attribu-
tion than for sensorimotor information (cursor-joystick velocity
difference) in the fifth cycle (paired t-test: t(17) = 2.38, P= 0.029,
Fig. 7C). In addition, we found the left IFG in the second cycle
(cluster 1), right MTG in the f ifth cycle (cluster 14), right anterior
insula in the fourth cycle (cluster 10), and right STG in the
third cycle (cluster 6) to show relatively high effect sizes. Note
that sensorimotor information was not considered comparable
among the periods between 2 and 4 s (second cycle), between
4 and 8 s (third and fourth cycles), and between 8 and 10 s
(fifth cycle) since we controlled the cursor visibility (for details
see Materials and Methods: Behavioral task). Taken together,
our results reveal that the preference for self-other attribution
(or sensorimotor information) was different among the 15 clus-
ters. Notably, the right SMG is the most sensitive to self-other
attribution among these clusters at the final stage of movement.
We further investigated the temporal changes in decoding
performances (z-scores) of the cluster in the right SMG. Figure 8
shows z-scores as a function of the time bin (2 s) for self-
other attribution (Fig. 8A), cursor-joystick distance (Fig. 8B), and
velocity difference (Fig. 8C) at the peak coordinate (x= 60, y=34,
z= 30 in MNI coordinates). The z-score for self-other attribution
reached the peak value in the fourth cycle (time bin 4) and
maintained a significant value in the last cycle (time bin 5:
gray bars). Meanwhile, the z-score for sensorimotor information
reached the peak before the last cycle (third cycle for cursor-
joystick distance and fourth cycle for velocity difference) but
abruptly declined in the last cycle. Consequently, while the
decoding performance for sensorimotor information reached a
significant level during the middle stage of the move period, the
performance for self-other attribution remained high at the end
of the move period.
Discussion
In the current study, we first found the sensorimotor, posterior
parietal, anterior insula, and higher visual cortices as the regions
where the self-other attribution could be decoded from their
voxel patterns (Fig. 5). As we found a tight relationship between
agency attribution and the sensorimotor information based on
the correspondence between the cursor and joystick movement
(cursor-joystick distance and velocity difference; Fig. 4), some of
the found regions overlapped those in which the sensorimotor
information could be decoded. Then, we investigated whether
information represented in the found regions showed a prefer-
ence for self-other attribution or for sensorimotor information
according to the effect size of the difference in decoding perfor-
mance. As a result,the right SMG (at the late stage of movement)
and left IFG (at the early stage of movement) were sensitive
to self-other attribution, while the bilateral precentral gyri and
left IPL dominantly reflected sensorimotor information (Fig. 7).
Our findings demonstrate that individual regions processed the
different levels of information during movement according to
their preference.
The comparator model proposed that the prediction error
is the main factor to determine agency attribution (Blakemore
et al. 2000;Frith et al. 2000). In our experiment, the joystick
Downloaded from https://academic.oup.com/cercor/advance-article-abstract/doi/10.1093/cercor/bhaa028/5775460 by guest on 04 May 2020
Sense of Agency Beyond Sensorimotor Process Ohata et al. 11
Figure 7. (A) Clusters color-coded according to the effect size of the difference between decoding performance for self-other attribution and for sensorimotor
information. Warmercolors denote bias toward self-other attribution (positive value of the effect size), while colder colors denote bias towardsensorimotor information
(negative value). The clusters are identical to those in Figure 5 (see Tab le 1 for anatomical details). The sinusoidal wave represented at the bottom is shifted by 6 s from
the actual time considering the HRD. (B) Effect sizes in the 15 clusters sorted in ascending order. The number and color of the bars correspond to those of clusters in (A).
(C) Decoding performance (z-score) for self-other attribution and cursor-joystick velocity difference in the right SMG in the fifth cycle (red and blue bars, respectively).
Error bars show standard error of the mean.
position was not displayed on the screen during the task. How-
ever, participants could predict the actual position and velocity
of their joystick on the screen according to their proprioception
and a forward model of the relationship between the cursor
and joystick. The participants could acquire the forward model
during the practice run under the self 100% condition (for details
see Materials and Methods: Behavioral Task). A cursor appeared
in the position shifted by the addition of the other’s joystick
position to the predicted position (i.e., actual position) in the
self 0–75% conditions (Fig. 2D). Therefore, we can regard the
Downloaded from https://academic.oup.com/cercor/advance-article-abstract/doi/10.1093/cercor/bhaa028/5775460 by guest on 04 May 2020
12 Cerebral Cortex, 2020, Vol. 00, No. 00
Figure 8. Time courses of z-scores for self-other attribution (A), cursor-joystick distance (B), and velocity difference (C)atthepeakvoxel(x= 60, y=34, z=30inMNI
coordinates) in the right SMG (cluster no. 15 in Fig. 7). Error bars show standard error of the mean. Asterisks indicate z-scores that were significantly larger than zero
according to two-tailed one-sample t-test (:P<0.01 uncorrected, ∗∗:P<0.05 Bonferroni corrected for multiple comparisons). Each time bin corresponds to a volume
scanned every 2 s during the 10-s move and 6-s delay periods.Gray bars highlight the fifth bin (last cycle of the sinusoidal movement). The events denoted under the
time bins are shifted by 6 s from the actual time considering the HRD.
cursor-joystick distance and velocity difference (Fig. 4A)as
proxies for a prediction error of sensory feedback. Our results
demonstrate that the accumulation of these behavioral
measures explains a large part of the variance in self-other
attribution (red and orange lines in Fig. 4B,C). These results are
consistent with the implications suggested by the comparator
model.
Previous studies have reported multiple brain regions as neu-
ral bases for the sense of agency. We found many such regions to
be those in which self-other attribution could be decoded from
their voxel patterns, such as the posterior parietal (Farrer and
Frith 2002;Farrer et al. 2003,2008;Ogawa and Inui 2007;Schnell
et al. 2007;Yomogida et al. 2010;Nahab et al. 2011), sensori-
motor (David et al. 2008;Sperduti et al. 2011), anterior insula
(Farrer et al. 2003;Tsakiris et al. 2010), and higher visual cortices
(Astafiev et al. 2004;David et al. 2007;Yomogida et al. 2010).
Since the concept of the sense of agency covers multiple aspects
from the sensorimotor to judgment level, previous studies have
not necessarily shown a difference in the level of the process
in which each of various regions was involved within a single
experiment (but see Farrer et al. 2008;Miele et al. 2011 for disso-
ciation of brain regions associated with agency attribution from
sensorimotor stages). The current study questioned whether
the found regions preferentially processed immediate output
of sensorimotor processing (mainly sensory prediction error) or
information closely related to a conceptual judgment of agency
during movement. We answered the question by showing the
gradual difference in preference for self-other attribution in
contrast to sensorimotor information (Fig. 7). Such gradation of
neural representation would give us a clue to understanding
how each region intermediates between the prediction error and
the conceptual judgment of agency attribution.
We found the clusters in the right SMG at the late stage
and left IFG at the early stage of movement as the top three
clusters that were more sensitive to agency attribution than
to sensorimotor information among the 15 clusters in which
self-other attribution could be decoded (Fig. 7). The SMG is a
part of the IPL, and previous studies also suggested that the
right IPL plays a critical role in the subjective experience of
self-agency. First, the right IPL is the region most frequently
reported as a neural correlate of the sense of agency (Farrer and
Frith 2002;Farrer et al. 2003,2008;Ogawa and Inui 2007;Schnell
et al. 2007;Nahab et al. 2011). In particular, Nahab et al. (2011)
morphed visual feedback (finger movement) by incorporating
pre-recorded movement into the participant’s actual one, and
found two distinct networks involving the sense of agency: the
leading and lagging networks. They found that blood-oxygen-
level-dependent responses in the lagging network regions were
slightly later than those in the leading network regions. By
adding the functional connectivity analysis to this finding, they
suggested that the lagging network played a role in mediating
sensorimotor information to a conscious awareness of self-
agency. The right IPL was found to be one of the main compo-
nents of the lagging network. Second, a single-pulse transcranial
magnetic stimulation (TMS) over the right IPL induced a change
in the participant’s action attribution (Preston and Newport
2008;Ritterband-Rosenbaum et al. 2014;Chambon et al. 2015).
Finally, the right IPL reflects the subjective experience of self-
agency even without receiving a prediction-error signal calcu-
lated in the sensorimotor system (Wenke et al. 2010;Haggard
and Chambon 2012;Chambon et al. 2013,2014,2015;Haggard
2017;Beyer et al. 2018). The previous studies reported that the
prospective signals, particularly the fluency of action selection
controlled by subliminal priming, affected the sense of agency
(Wenke et al. 2010). Their fMRI studies suggested that a region
in the right IPL received the prospective signals carried from
the dorsolateral prefrontal cortex and constructed the subjective
experience of self-agency (Chambon et al. 2013). Taken together,
the previous findings support our conclusion that the right IPL
is dominantly responsible for the higher-order function in the
neural process of the sense of agency (Eddy 2016), compared
with the low-level sensorimotor component.
Regarding the role of the right IPL, many neuroimaging stud-
ies have reported the right angular gyrus (AG), not the right
SMG, to be sensitive to the difference in the sense of agency
(Farrer et al. 2008;Chambon et al. 2013;Beyer et al. 2018;but
see Koreki et al. 2019). In the current study, we found the cluster
mostly in the right SMG in the last cycle (cluster 15 in Fig. 7A),
Downloaded from https://academic.oup.com/cercor/advance-article-abstract/doi/10.1093/cercor/bhaa028/5775460 by guest on 04 May 2020
Sense of Agency Beyond Sensorimotor Process Ohata et al. 13
which did not overlap the right AG defined in the AAL atlas
(Supplementary Fig. 5). There are two possible interpretations
why the cluster was found in the right SMG, but not the right AG,
in the current study. The first is that the SMG may preferentially
code for sensorimotor conflicts, which are critical for the sense
of agency, compared with an intersensory conflict between
vision and proprioceptive information, which affects both sense
of agency and body ownership (Tsakiris et al. 2010). Tsakiris et al.
(2010) found the most prominent activation in the right SMG
when visual feedback was asynchronized with the active finger
movements. In contrast, the right AG was activated when visual
feedback was asynchronized with passive movements as well
as active movements. Based on their findings, it is possible to
interpret that the neural representation in the right SMG found
in our study reflected the sensorimotor-based sense of agency,
not that affected by gain/loss of body ownership. The second
is due to the difference in the analysis method between the
previous studies and our study (i.e., univariate vs. multivariate
analysis). To our knowledge, all of the previous neuroimaging
studies reporting the AG as the neural basis of the sense of
agency performed univariate analysis (i.e., increase or decrease
in voxel-wise activation). By contrast, we performed MVPA to
find the brain regions where we could decode agency attribution.
Thus, it might be possible that the SMG mainly represents infor-
mation reflecting the sense of agency at a multivoxel pattern
level.
Our findings on the time courses of decoding performances
in the right SMG (Fig. 8) have significant implications for the
neural process of the sense of agency. The sensorimotor infor-
mation could be decoded in the middle of the move period
(third and fourth cycles in Fig. 8B,C). But more importantly, we
found that the decoding performance for self-other attribution
remained at a significant level (fifth cycle in Fig. 8A), despite
the sudden decline in performance for sensorimotor informa-
tion at the end of the move period (fifth cycle in Fig. 8B,C).
The results suggest that the right SMG possibly contained the
sensorimotor information for translating it into the conscious
experience of self-agency. The right IPL has connections with
many brain regions (Bzdok et al. 2013)andtheroleofmultisen-
sory integration (Ionta et al. 2011;Jakobs et al. 2012). Therefore,
we can hypothesize that the right SMG receives and integrates
the information processed in the regions that have a preference
for sensorimotor processing and then calculates the sense of
agency. For validation of this hypothesis, we need further stud-
ies on the information flow between the right SMG and other
sensorimotor regions (Koreki et al. 2019).
As well as the right SMG, the left IFG in the second cycle
(cluster 1 in Fig. 7A) also showed relatively high effect size
among 15 clusters. The cluster on the left IFG in the second
cycle largely overlaps those found in the previous studies on
body ownership and peripersonal space (Ehrsson et al. 2004,
2005;Petkova et al. 2011;Gentile et al. 2013;Guterstam et al.
2013,2015;Blanke et al. 2015;Grivaz et al. 2017). In the second
cycle, we manipulated the cursor to gradually become visible
as noted in the Materials and Methods section (for details see
Behavioral Task section, Fig. 2Band Supplementary Movie 1).
At the initial stage of a trial in this task, a possible strategy
for participants was to explore the correspondence between
their joystick control and the gradually visible cursor. Thus, we
assumed that the degree of this correspondence determined
how much the boundary of peripersonal space extended to
the cursor on the screen. Bassolino et al. (2010) has already
demonstrated that the peripersonal space around the hand can
be extended toward a cursor (computer mouse) on the screen.
This subjective feeling of extending their body to the cursor
could affect their evaluation of agency attribution. The above
interpretation is also supported by the fact that the other cluster
in the second cycle (cluster 2 in Fig. 7A), which is located on the
left IPL, also overlaps the area associated with body ownership
and peripersonal space.
We combined two strategies to test our hypothesis. The first
was to use a unique experimental paradigm that considers the
temporal evolution of self-agency (Asai 2015,2016,2017). Most
previous studies required participants to perform an intermit-
tent action such as a button press (e.g., Farrer and Frith 2002)or
a reaching movement (e.g., David et al. 2007). However, such sim-
ple tasks made it difficult to shed light on the processes of how
the sense of agency was built in the brain. In our experiment,
participants continuously received sensorimotor evidence while
tracing a target path under an ambiguous condition so that
they could gradually realize whether a cursor movement was
attributed to the self or other. Thus, our task paradigm was
appropriate to investigate the neural representation reflecting
self-other attribution, which shifts from time to time. The sec-
ond strategy was to apply an MVPA to fMRI data (Haynes and
Rees 2005;Kamitani and Tong 2005;Norman et al. 2006). The
MVPA enabled us to find the regions whose voxel patterns
were more sensitive to self-other attribution than to sensori-
motor information. Thus, the combination of these strategies
has revealed a temporal change in the neural representation
of self-agency grounded on the sensorimotor system. Note that
MVPA evaluates whether the nonuniform response of voxels in
aregionisinformativeaboutthevariableofinterest(Hebart
and Baker 2018). Therefore, it is basically impossible to discuss
how the uniform (i.e., positive or negative) responses in the
found clusters were related to self- or other agency. Related to
the above, we have confirmed that it is unlikely that our MVPA
result (Fig. 5) could be explained only by the activation level in
the found clusters (for more details see Supplementary Results:
Mass univariate analysis of voxel-wise activation modulated by
self-other attribution and Supplementary Fig. 6).
In the current study, participants were required to report
self-other attribution of cursor movement as a judgment of
agency. By contrast,some studies required participants to report
different types of agency judgments such as those regarding
controllability (i.e., how much control they felt). The current
experiment was not designed to answer the question of whether
the type of agency judgment influences the neural process for
determining the agency judgment. To answer this question, we
need future studies designed to reveal the difference in neural
substrates implicated in different types of agency judgments.
For instance, we will acquire fMRI data under two conditions as
follows. In one condition, participants will be required to report
their explicit judgment about “self-other attribution” (i.e., how
much they felt that the cursor movement was attributed to their
own action). In the other condition, participants will report their
judgment of “controllability” (i.e., how much they felt that they
could control the cursor). For MVPA, a decoder will be trained to
predict a rating score of self-other attribution in the same way
as the current study. In the test phase, we will evaluate whether
the trained decoder can predict the rating score of controllability
(i.e., cross-decoding). This study might shed light on whether a
common brain region is recruited in different types of agency
judgment.
The MVPA in our study has several possible confounding
factors. The first factor is the difference in attention level used
Downloaded from https://academic.oup.com/cercor/advance-article-abstract/doi/10.1093/cercor/bhaa028/5775460 by guest on 04 May 2020
14 Cerebral Cortex, 2020, Vol. 00, No. 00
to control the cursor depending on whether the participants felt
the cursor movement was attributed to the self or controlled by
the other. A possible scenario could be as follows: Participants
might find it more difficult to precisely control the cursor in
the presence of an external agent controlling the cursor. This
scenario would suggest that the more strongly the participants
felt that an external agent controlled the cursor,the more atten-
tion they would have given toward controlling the cursor. In that
case, we could have only decoded different levels of attention
that covary with action attribution. We checked whether the
attention level correlated with the rating score of the self-other
attribution. Although we cannot directly measure the level of
attention given to cursor control, it can be inferred from the
accuracy of the participant’s tracing performance, such as the
error between the target path and the cursor position (target-
cursor distance in Fig. 4A). Note that we instructed participants
to precisely trace the target path with the cursor even in the self-
other mixed conditions. We found that the target-cursor dis-
tance did not correlate with the rating score (blue line in Fig. 4B).
This result suggests that the attention placed on cursor control
was not a crucial factor in decoding the self-other attribution
judgment. The second factor is the difference in the rating
score, which participants prepared in their minds before the
rate period. We instructed them to judge action attribution on
a 9-point Likert scale. Although the rate period was temporally
distinct from the move periods, participants might have kept a
rating score in mind before the rate period. It has been suggested
that the right IPL is involved in a magnitude system of numerical
processing (Walsh 2003). Thus, we might have decoded the
difference in the rating score from activity patterns in the right
SMG, even before the rate period. However, the decoding perfor-
mance of the cluster in the right SMG declined once the delay
period began (time bin 6 in the “self-other attribution” panel in
Fig. 8A). This decline suggests that numerical processing was not
a crucial factor in our successful regression.
In 1890, William James proposed the concept of the “I”as one
aspect of the self: experiencing oneself as a subjective agent of
thought, perception, and action (James 1890). Our study tackled
the neural substrate underlying the awareness of ourselves as
agents of action through interaction with the external world. As
emphasized in the comparator model, our findings support the
idea that the sense of agency is grounded on the sensorimotor
system. More importantly, our study demonstrated the neural
process that bridges the gap between lower level sensorimotor
processing and higher level processing for agency attribution.
In this process, the right SMG plays a critical role in translating
sensorimotor information (obtained from interaction with the
external world) into an awareness of the subjective agent of an
action.
Supplementary Material
Supplementary material is available at Cerebral Cortex online.
Funding
JSPS KAKENHI (grant 26120002, 18H01098, 19H05725 to H.I.);
JSPS KAKENHI (grant 15J05135, Grant-in-Aid for JSPS Fellows
to R.O.); JSPS KAKENHI (grant 17K13971 to T.A.); “Research and
development of technology for enhancing functional recovery of
elderly and disabled people based on noninvasive brain imag-
ing and robotic assistive devices,” Commissioned Research of
National Institute of Information and Communications Technol-
ogy (NICT) to T.A. and H.I.; Japan Agency for Medical Research
and Development (AMED) (grant JP18dm0307008) to T.A. and H.I.
Notes
The authors are grateful to Dr Nobuhiro Hagura (Center for Infor-
mation and Neural Networks) and Dr Shu Imaizumi (Ochano-
mizu University) for their insightful comments. Conflict of Inter-
est: None declared.
References
Asai T. 2015. Feedback control of one’s own action: self-
other sensory attribution in motor control. Conscious Cogn.
38:118–129.
Asai T. 2016. Self is “other”, other is “self”: poor self-other dis-
criminability explains schizotypal twisted agency judgment.
Psychiatry Res. 246:593–600.
Asai T. 2017. Know thy agency in predictive coding: meta-
monitoring over forward modeling. Conscious Cogn. 51:82–99.
Astafiev SV, Stanley CM, Shulman GL, Corbetta M. 2004. Extras-
triate body area in human occipital cortex responds to the
performance of motor actions. Nat Neurosci. 7:542–548.
Bassolino M, Serino A, Ubaldi S, Ladavas E. 2010. Everyday use of
the computer mouse extends peripersonal space representa-
tion. Neuropsychologia. 48:803–811.
Beyer F, Sidarus N, Fleming S, Haggard P. 2018. Losing control in
social situations: how the presence of others affects neural
processes related to sense of agency. eNeuro. 5:ENEURO.0336-
0317.2018.
Blakemore SJ, Frith CD, Wolpert DM. 2001. The cerebellum is
involved in predicting the sensory consequences of action.
Neuroreport. 12:1879–1884.
Blakemore SJ, Rees G, Frith CD. 1998. How do we predict the
consequences of our actions? A functional imaging study.
Neuropsychologia. 36:521–529.
Blakemore SJ, Wolpert D, Frith C. 2000. Why can’t you tickle
yourself? Neuroreport. 11:R11–R16.
Blanke O, Slater M, Serino A. 2015. Behavioral, neural, and com-
putational principles of bodily self-consciousness. Neuron.
88:145–166.
Bode S, Haynes JD. 2009. Decoding sequential stages of task
preparation in the human brain. NeuroImage. 45:606–613.
Bzdok D, Langner R, Schilbach L, Jakobs O, Roski C, Caspers S,
Laird AR, Fox PT, Zilles K, Eickhoff SB. 2013. Characteriza-
tion of the temporo-parietal junction by combining data-
driven parcellation, complementary connectivity analyses,
and functional decoding. NeuroImage. 81:381–392.
Chambon V, Moore JW, Haggard P. 2015. TMS stimulation over
the inferior parietal cortex disrupts prospective sense of
agency. Brain Struct Funct. 220:3627–3639.
Chambon V, Sidarus N, Haggard P. 2014. From action intentions
to action effects: how does the sense of agency come about?
Front Hum Neurosci. 8:320.
Chambon V, Wenke D, Fleming SM, Prinz W, Haggard P. 2013.
An online neural substrate for sense of agency. Cereb Cortex.
23:1031–1037.
Christoff K, Cosmelli D, Legrand D, Thompson E. 2011. Spec-
ifying the self for cognitive neuroscience. Trends Cogn Sci.
15:104–112.
David N, Cohen MX, Newen A, Bewernick BH, Shah NJ, Fink
GR, Vogeley K. 2007. The extrastriate cortex distinguishes
Downloaded from https://academic.oup.com/cercor/advance-article-abstract/doi/10.1093/cercor/bhaa028/5775460 by guest on 04 May 2020
Sense of Agency Beyond Sensorimotor Process Ohata et al. 15
between the consequences of one’s own and others’ behav-
ior. NeuroImage. 36:1004–1014.
David N, Newen A, Vogeley K. 2008. The “sense of agency” and its
underlying cognitive and neural mechanisms.Conscious Cogn.
17:523–534.
Eddy CM. 2016. The junction between self and other? Temporo-
parietal dysfunction in neuropsychiatry. Neuropsychologia.
89:465–477.
Ehrsson HH, Holmes NP, Passingham RE. 2005. Touching a rubber
hand: feeling of body ownership is associated with activity in
multisensory brain areas. J Neurosci. 25:10564–10573.
Ehrsson HH, Spence C, Passingham RE. 2004. That’s my hand!
Activity in premotor cortex reflects feeling of ownership of a
limb. Science. 305:875–877.
Farrer C, Franck N, Georgieff N, Frith CD, Decety J, Jeannerod
A. 2003. Modulating the experience of agency: a positron
emission tomography study. NeuroImage. 18:324–333.
Farrer C, Frey SH, Van Horn JD, Tunik E, Turk D, Inati S, Grafton
ST. 2008. The angular gyrus computes action awareness
representations. Cereb Cortex. 18:254–261.
Farrer C, Frith CD. 2002. Experiencing oneself vs another person
as being the cause of an action: the neural correlates of the
experience of agency. NeuroImage. 15:596–603.
Frith CD, Blakemore S, Wolpert DM. 2000. Explaining the symp-
toms of schizophrenia: abnormalities in the awareness of
action. Brain Res Rev. 31:357–363.
Fukushima H, Goto Y, Maeda T, Kato M, Umeda S. 2013. Neural
substrates for judgment of self-agency in ambiguous situa-
tions. PLoS One. 8:e72267.
Gallagher S. 2000. Philosophical conceptions of the self: implica-
tions for cognitive science. Trends Cogn Sci. 4:14–21.
Gentile G, Guterstam A, Brozzoli C, Ehrsson HH. 2013. Dis-
integration of multisensory signals from the real hand
reduces default limb self-attribution: an fMRI study. JNeu-
rosci. 33:13350–13366.
Grivaz P, Blanke O, Serino A. 2017. Common and distinct brain
regions processing multisensory bodily signals for periper-
sonal space and body ownership. NeuroImage. 147:602–618.
Guterstam A, Bjornsdotter M, Gentile G, Ehrsson HH. 2015. Pos-
terior cingulate cortex integrates the senses of self-location
and body ownership. Curr Biol. 25:1416–1425.
Guterstam A, Gentile G, Ehrsson HH. 2013. The invisible hand
illusion: multisensory integration leads to the embodiment
of a discrete volume of empty space. J Cogn Neurosci.
25:1078–1099.
Haggard P. 2017. Sense of agency in the human brain. Nat Rev
Neurosci. 18:196.
Haggard P, Chambon V. 2012. Sense of agency. Curr Biol. 22:R390–
R392.
Haynes JD. 2015. A primer on pattern-based approaches to fMRI:
principles, pitfalls, and perspectives. Neuron. 87:257–270.
Haynes JD, Rees G. 2005. Predicting the orientation of invisible
stimuli from activity in human primary visual cortex. Nat
Neurosci. 8:686–691.
Haynes JD, Sakai K, Rees G, Gilbert S, Frith C, Passingham RE.
2007. Reading hidden intentions in the human brain. Curr
Biol. 17:323–328.
Hebart MN, Baker CI. 2018. Deconstructing multivariate decoding
for the study of brain function. NeuroImage. 180:4–18.
Ionta S, Heydrich L, Lenggenhager B, Mouthon M, Fornari E,
Chapuis D, Gassert R, Blanke O. 2011. Multisensory mecha-
nisms in temporo-parietal cortex support self-location and
first-person perspective. Neuron. 70:363–374.
Jakobs O, Langner R, Caspers S, Roski C, Cieslik EC, Zilles
K, Laird AR, Fox PT, Eickhoff SB. 2012. Across-study and
within-subject functional connectivity of a right temporo-
parietal junction subregion involved in stimulus-context
integration. NeuroImage. 60:2389–2398.
James W. 1890. The Principles of Psychology. Dover Publications.
Kamitani Y, Tong F. 2005. Decoding the visual and sub-
jective contents of the human brain. Nat Neurosci.8:
679–685.
Koreki A, Maeda T, Okimura T, Terasawa Y, Kikuchi T, Umeda
S, Nishikata S, Yagihashi T, Kasahara M, Nagai C et al. 2019.
Dysconnectivity of the agency network in schizophrenia: a
functional magnetic resonance imaging study. Front Psych.
10:171.
Kriegeskorte N, Goebel R, Bandettini P. 2006. Information-
based functional brain mapping. Proc Natl Acad Sci USA.
103:3863–3868.
Lakens D. 2013. Calculating and reporting effect sizes to facil-
itate cumulative science: a practical primer for t-tests and
ANOVAs. Front Psychol. 4:863.
Langfelder P, Luo R, Oldham MC, Horvath S. 2011. Is my net-
work module preserved and reproducible? PLoS Comput Biol.
7:e1001057.
Legrand D. 2007. Pre-reflective self-as-subject from experiential
and empirical perspectives. Conscious Cogn. 16:583–599.
Legrand D, Ruby P. 2009. What is self-specif ic? Theoretical inves-
tigation and critical review of neuroimaging results. Psychol
Rev. 116:252–282.
Miall RC, Wolpert DM. 1996. Forward models for physiological
motor control. Neural Netw. 9:1265–1279.
Miele DB, Wager TD, Mitchell JP, Metcalfe J. 2011. Dissociating
neural correlates of action monitoring and metacognition of
agency. J Cogn Neurosci. 23:3620–3636.
Moore JW, Fletcher PC. 2012. Sense of agency in health and
disease: a review of cue integration approaches. Conscious
Cogn. 21:59–68.
Moore JW, Wegner DM, Haggard P. 2009. Modulating the sense of
agency with external cues. Conscious Cogn. 18:1056–1064.
Mur M, Bandettini PA, Kriegeskorte N. 2009. Revealing represen-
tational content with pattern-information fMRI—an intro-
ductory guide. Soc Cogn Affect Neurosci. 4:101–109.
Nahab FB, Kundu P, Gallea C, Kakareka J, Pursley R, Pohida T,
Miletta N, Friedman J, Hallett M. 2011. The neural processes
underlying self-agency. Cereb Cortex. 21:48–55.
Norman KA, Polyn SM, Detre GJ, Haxby JV. 2006. Beyond mind-
reading: multi-voxel pattern analysis of fMRI data. Trends
Cogn Sci. 10:424–430.
Ogawa K, Inui T. 2007. Lateralization of the posterior parietal
cortex for internal monitoring of self-versus externally gen-
erated movements. J Cogn Neurosci. 19:1827–1835.
Pacherie E. 2007. The sense of control and the sense of agency.
Psyche. 13:1–30.
Petkova VI, Bjornsdotter M, Gentile G, Jonsson T, Li TQ , Ehrs-
son HH. 2011. From part- to whole-body ownership in the
multisensory brain. Curr Biol. 21:1118–1122.
Preston C, Newport R. 2008. Misattribution of movement agency
following right parietal TMS. Soc Cogn Affect Neurosci. 3:26–32.
Ritterband-Rosenbaum A, Karabanov AN, Christensen MS,
Nielsen JB. 2014. 10 Hz rTMS over right parietal cortex alters
sense of agency during self-controlled movements.Front Hum
Neurosci. 8:471.
Schnell K, Heekeren K, Schnitker R, Daumann J, Weber J, Hessel-
mann V, Moller-Hartmann W, Thron A, Gouzoulis-Mayfrank
Downloaded from https://academic.oup.com/cercor/advance-article-abstract/doi/10.1093/cercor/bhaa028/5775460 by guest on 04 May 2020
16 Cerebral Cortex, 2020, Vol. 00, No. 00
E. 2007. An fMRI approach to particularize the frontopari-
etal network for visuomotor action monitoring: detection of
incongruence between test subjects’ actions and resulting
perceptions. NeuroImage. 34:332–341.
Shibata K, Watanabe T, Kawato M, Sasaki Y. 2016. Differential
activation patterns in the same brain region led to opposite
emotional states. PLoS Biol. 14:e1002546.
Soon CS, Brass M, Heinze HJ, Haynes JD. 2008. Unconscious deter-
minants of free decisions in the human brain. Nat Neurosci.
11:543–545.
Spengler S, von Cramon DY, Brass M. 2009. Was it me or was
it you? How the sense of agency originates from ideomotor
learning revealed by fMRI. NeuroImage. 46:290–298.
Sperduti M, Delaveau P, Fossati P, Nadel J. 2011. Different
brain structures related to self- and external-agency attri-
bution: a brief review and meta-analysis. Brain Struct Funct.
216:151–157.
Synofzik M, Vosgerau G, Newen A. 2008. Beyond the comparator
model: a multifactorial two-step account of agency. Conscious
Cogn. 17:219–239.
Synofzik M, Vosgerau G, Voss M. 2013. The experience of agency:
an interplay between prediction and postdiction. Front
Psychol. 4:127.
Tsakiris M, Longo MR, Haggard P. 2010. Having a body versus
moving your body: neural signatures of agency and body-
ownership. Neuropsychologia. 48:2740–2749.
Tzourio-Mazoyer N, Landeau B, Papathanassiou D, Crivello F,
Etard O, Delcroix N, Mazoyer B, Joliot M. 2002. Automated
anatomical labeling of activations in SPM using a macro-
scopic anatomical parcellation of the MNI MRI single-subject
brain. NeuroImage. 15:273–289.
Walsh V. 2003. A theory of magnitude: common cortical metrics
of time, space and quantity. Trends Cogn Sci. 7:483–488.
Wenke D, Fleming SM, Haggard P. 2010. Subliminal priming of
actions influences sense of control over effects of action.
Cognition. 115:26–38.
Wisniewski D, Goschke T, Haynes J-D. 2016. Similar coding of
freely chosen and externally cued intentions in a fronto-
parietal network. NeuroImage. 134:450–458.
Wisniewski D, Reverberi C, Momennejad I, Kahnt T, Haynes JD.
2015. The role of the parietal cortex in the representation of
task-reward associations. J Neurosci. 35:12355–12365.
Yomogida Y, Sugiura M, Sassa Y, Wakusawa K, Sekiguchi A,
Fukushima A, Takeuchi H, Horie K, Sato S, Kawashima R.
2010. The neural basis of agency: an fMRI study. NeuroImage.
50:198–207.
Downloaded from https://academic.oup.com/cercor/advance-article-abstract/doi/10.1093/cercor/bhaa028/5775460 by guest on 04 May 2020
... However, the SoA can be reduced by non-self-initiated interventions because the outcomes of the actions of an individual influenced by others differ from those predicted by internal models. In other words, autonomous transition control or shared control in which robots assist human operations can reduce the SoA [23], [24], [25], [26]. Therefore, methods that improve performance and reflect the intentions of operators in the perceived outcomes of their actions, even under time-varying delays, are needed. ...
... They rated the question "Did you feel that your actions were reflected in the pouring task?" on a Likert scale from 1 to 7 (7: strongly agree; 4: neither agree nor disagree (neutral); 1: strongly disagree). This question was adapted from those used in previous studies [19], [24], [26], [25] to fit the context of this experiment. A SoA is defined in this study as being elicited when the score is significantly higher than the neutral score of 4. In addition, task performance was assessed by measuring the weight of beads poured into the target bottle during each trial and converting this weight to the number of beads. ...
Article
Full-text available
Intuitive teleoperation enables operators to embody remote robots, providing the sensation that the robot is part of their own body during control. The sense of agency (SoA), i.e., the feeling of controlling the robot, contributes to enhanced motivation and embodiment during teleoperation. However, the SoA can be diminished by time-varying communication delays associated with teleoperation. We propose a visual guidance system to assist operations while maintaining a high SoA when teleoperating robots with time-varying delays, thereby improving positioning accuracy. In the proposed system, a recurrent neural network (RNN) model, trained on the pouring tasks of skilled operators, predicts the input position 500ms ahead of the input from the novice operator and visually presents it in real-time as the end-effector target position. Experiments with time-varying delays confirmed that the proposed method provides a visual representation of the target position interpolated in time and space from the real-time input of the operator, guiding the operator to align with the trajectory of the skilled operator. The proposed method significantly improves task performance even under time-varying delays while maintaining a high SoA compared with other conditions. Applying the prediction system developed in this study to humanârobot collaborative control may enable interventions that maintain the SoA.
... The supramarginal gyrus (SMG) in the inferior parietal lobe has been linked to sense of agency. Apraxic patients with lesions in the left SMG confuse whether a hand movement is their own or someone else's [23] and SMG activity has been linked to realizing whether an action was one's own or not [24,25]. More generally, the SMG has been linked to directing attention to movement [26][27][28]. ...
... Other brain regions such as area 47/12o on the border between ventrolateral and orbitofrontal cortex have been linked to credit assignment in non-social contexts [2,4,13,55]. However, monitoring the consequences of one's own and others' actions may depend on SMG [23][24][25], and therefore, make SMG important in establishing agents' relative levels of control -their degrees of agency -during social interaction. ...
Preprint
Full-text available
There has been considerable interest in how we ascertain whether an environment is controllable and the neural mechanisms mediating this process. An especially acute version of this problem occurs when multiple people collaborate towards a joint outcome. Here we demonstrate that in such ambiguous social contexts, people engage in specific patterns of behaviour that we refer to as active disambiguation. This process helps individuals establish what they themselves, as opposed to others, control and what consequence they themselves cause or that another person causes. People identify when active disambiguation is needed and engage in it at that time. A pattern of activity in the supramarginal gyrus that emerges during and after active disambiguation is linked to establishing controllability and tracking the outcomes from which control is inferred. Activity in this brain region also signals a second learning mechanism, by which individuals attribute outcomes to themselves versus others, in proportion to their perceived control.
... In a typical active motion, self-motion is determined by comparing afferent signals from proprioceptive sensations with efferent copies of motor commands [23], [24]. The discrepancy between the actual motion and that predicted by efferent copying decreases the sense of agency [25], [26]. When this illusion is induced during active motion, the illusory directional sensation may be added to the sensation of the motion, resulting in a discrepancy from the predicted sensation. ...
... The phenomenon of a decreased sense of agency, despite the participants speeding up the wrist motion by themselves, was examined based on the comparator model (Fig. 8), as outlined in Ref. [35]. This model incorporates the generation of a body schema, wherein the agency arises from a comparison of predicted and actual motions [23], [24], [25]. This model is deemed reasonable because the agency contributes to updating the body schema [27], which is recognized as an internal model in motor control [36]. ...
Article
Full-text available
Illusory directional sensations are generated through asymmetric vibrations applied to the fingertips and have been utilized to induce upper-limb motions in the rehabilitation and training of patients with visual impairment. However, its effects on motor control remain unclear. This study aimed to verify the effects of illusory directional sensations on wrist motion. We conducted objective and subjective evaluations of wrist motion during a motor task, while inducing an illusory directional sensation that was congruent or incongruent with wrist motion. We found that, when motion and illusory directional sensations were congruent, the sense of agency for motion decreased. This indicates an induction sensation of the hand being moved by the illusion. Interestingly, although no physical force was applied to the hand, the angular velocity of the wrist was higher in the congruent condition than that in the no-stimulation condition. The angular velocity of the wrist and electromyography signals of the agonist muscles were weakly positively correlated, suggesting that the participants may have increased their wrist velocity. In other words, the congruence between the direction of motion and illusory directional sensation induced the sensation of the hand being moved, even though the participants’ wrist-motion velocity increased. This phenomenon can be explained by the discrepancy between the sensation of active motion predicted by the efferent copy, and that of actual motion caused by the addition of the illusion. The findings of this study can guide the design of novel rehabilitation methods.
... P-VMI significantly increased the activity level in the right (contralateral) superior parietal lobule compared with S-VMI and also significantly increased the perception of somatization sensation (SoO and SoA). Ohata et al. reported that the right inferior parietal lobule plays a critical role in SoA [23]. This study showed increased activity in the right superior parietal lobule, a region previously reported to have connectivity with the inferior parietal lobule [24]. ...
Article
Full-text available
Background Visual-motor illusion (VMI) is a cognitive approach used to evoke kinesthetic sensations. Research suggests that VMI can modulate brain activity depending on the specific joint movement observed. This study aimed to identify differences in brain activity when observing video images of joint movements at different intensities of movement in VMI. Methodology The study included 14 healthy adult participants. Two types of video images were used: pure ankle dorsiflexion movements (Standard-VMI) and ankle dorsiflexion movements with added resistance (Power-VMI). The brain activity measurement protocol employed a block design with one set of 15 seconds rest, 30 seconds VMI task, and 30 seconds follow-up. Each participant performed the VMI task twice, alternating between Standard-VMI and Power-VMI. Brain activity was measured using functional near-infrared spectroscopy, focusing on motor-related regions. Subjective impressions were assessed using visual analog scales (VAS) for kinesthetic illusions. Results The results revealed that Power-VMI stimulated significantly greater brain activity in the premotor and supplementary motor cortex, supramarginal gyrus, and superior parietal lobule compared with Standard-VMI. Power-VMI resulted in higher VAS values for kinesthetic illusion than Standard-VMI. Additionally, a positive correlation was observed between brain activity in the superior parietal lobule and the degree of kinesthetic illusion. Conclusions These findings indicate that Power-VMI enhances both motor-related brain areas and motor-sensory illusions, potentially having a greater impact on improving motor function. This study provides valuable insights for developing VMI interventions for rehabilitation, particularly for individuals with paralysis or movement impairments.
... In this section, we model SoA for delayed continuous operations. The classical SoA model is known as the comparator model (Blakemore et al., 1998Miall and Wolpert, 1996;Ohata et al., 2020;Wolpert et al., 1995). In the comparator model, the lack of SoA originates from prediction error, that is, the error between the predicted state and the estimated actual state (see Comparator 3 in Fig. 1). ...
Article
Full-text available
Action-feedback delay during operation reduces sense of agency (SoA). In this study, using information-theoretic free energy, we formalized a novel mathematical model for explaining the influence of delay on SoA in continuous operations. Based on the mathematical model, we propose that visualization of predicted future outcomes prevents SoA degradation resulting from response delays. Model-based simulations and operational experiments with participants confirmed that operational delay considerably reduces SoA. Furthermore, the proposed visualization mitigates these problems. Our findings support the model-based interface design for continuous operations with delay to prevent SoA degradation.
Article
The concept of forward models in the brain, classically applied to describing on-line motor control, can in principle be extended to action planning, i.e. assuming forward sensory predictions are issued during the mere preparation of movements. To test this idea, we combined a delayed movement task with a virtual reality based manipulation of visuomotor congruence during functional magnetic resonance imaging. Participants executed simple hand movements after a delay. During the delay, two aspects of the upcoming movement could be cued: the movement type and the visuomotor mapping (i.e. congruence of executed hand movements and visual movement feedback by a glove-controlled virtual hand). Frontoparietal areas showed increased delay period activity when preparing pre-specified movements (cued > uncued). The cerebellum showed increased activity during the preparation for incongruent > congruent visuomotor mappings. The left anterior intraparietal sulcus showed an interaction effect, responding most strongly when a pre-specified (cued) movement was prepared under expected visuomotor incongruence. These results suggest that motor planning entails a forward prediction of visual body movement feedback, which can be adjusted in anticipation of nonstandard visuomotor mappings, and which is likely computed by the cerebellum and integrated with state estimates for (planned) control in the anterior intraparietal sulcus.
Article
Sense of agency (SoA) is the sensation that self-actions lead to ensuing perceptual consequences. The prospective mechanism emphasizes that SoA arises from motor prediction and its comparison with actual action outcomes, while the reconstructive mechanism stresses that SoA emerges from retrospective causal processing about the action outcomes. Consistent with the prospective mechanism, motor planning regions were identified by neuroimaging studies using the temporal binding (TB) effect, a behavioral measure often linked to implicit SoA. Yet, TB also occurs during passive observation of another’s action, lending support to the reconstructive mechanism, but its neural correlates remain unexplored. Here, we employed virtual reality (VR) to modulate such observation-based SoA and examined it with functional magnetic resonance imaging (fMRI). After manipulating an avatar hand in VR, participants passively observed an avatar’s “action” and showed a significant increase in TB. The binding effect was associated with the right angular gyrus and inferior parietal lobule, which are critical nodes for inferential and agency processing. These results suggest that the experience of controlling an avatar may potentiate inferential processing within the right inferior parietal cortex and give rise to the illusionary SoA without voluntary action.
Article
Full-text available
Background: Self-disturbances in schizophrenia have recently been explained by an abnormality in the sense of agency (SoA). The cerebral structures of SoA in healthy people are considered to mainly include the insula and inferior parietal lobule. In contrast, the functional lesion of aberrant SoA in schizophrenia is not yet fully understood. Considering the recent explanation of establishing SoA from the standpoint of associative learning, the “agency network” may include not only the insula and inferior parietal lobule but also the striatum. We hypothesized that aberrant SoA in schizophrenia is based on a deficit in the “agency network.” Methods: Functional magnetic resonance imaging data were acquired while patients with schizophrenia (n = 15) and matched controls (n = 15) performed our adaptation method of agency attribution task on a trial-by-trial basis to assess participants' explicit experience of the temporal causal relationship between an action and an external event with temporal biases. Analysis of functional connectivity was done using the right supramarginal gyrus and the right middle frontal gyrus as seed regions. Results: In healthy controls, analyses revealed increased activation of the right inferior parietal lobule (mainly the supramarginal gyrus), right insula, and right middle frontal gyrus as an activation of the agency condition. We defined activated Brodmann areas shown in the agency condition of healthy controls as the seed region for connectivity analysis. The connectivity analysis revealed lower connectivity between the head of the left caudate nucleus and right supramarginal gyrus in the patients compared to healthy controls. Conclusions: This dysconnectivity of the agency network in schizophrenia may lead to self-disturbance through deficits in associative learning of SoA. These findings may explain why pathological function of the striatum in schizophrenia leads to self-disturbance.
Article
Full-text available
Social contexts substantially influence individual behavior, but little is known about how they affect cognitive processes related to voluntary action. Previously, it has been shown that social context reduces participants’ sense of agency over the outcomes of their actions and outcome monitoring. In this fMRI study on human volunteers, we investigated the neural mechanisms by which social context alters sense of agency. Participants made costly actions to stop inflating a balloon before it burst. On “social” trials, another player could act in their stead, but we analyzed only trials in which the other player remained passive. We hypothesized that mentalizing processes during social trials would affect decision-making fluency and lead to a decreased sense of agency. In line with this hypothesis, we found increased activity in the bilateral temporo-parietal junction (TPJ), precuneus, and middle frontal gyrus during social trials compared with nonsocial trials. Activity in the precuneus was, in turn, negatively related to sense of agency at a single-trial level. We further found a double dissociation between TPJ and angular gyrus (AG): activity in the left AG was not sensitive to social context but was negatively related to sense of agency. In contrast, activity in the TPJ was modulated by social context but was not sensitive to sense of agency.
Article
Full-text available
Multivariate decoding methods were developed originally as tools to enable accurate predictions in real-world applications. The realization that these methods can also be employed to study brain function has led to their widespread adoption in the neurosciences. However, prior to the rise of multivariate decoding, the study of brain function was firmly embedded in a statistical philosophy grounded on univariate methods of data analysis. In this way, multivariate decoding for brain interpretation grew out of two established frameworks: multivariate decoding for predictions in real-world applications, and classical univariate analysis based on the study and interpretation of brain activation. We argue that this led to two confusions, one reflecting a mixture of multivariate decoding for prediction or interpretation, and the other a mixture of the conceptual and statistical philosophies underlying multivariate decoding and classical univariate analysis. Here we attempt to systematically disambiguate multivariate decoding for the study of brain function from the frameworks it grew out of. After elaborating these confusions and their consequences, we describe six, often unappreciated, differences between classical univariate analysis and multivariate decoding. We then focus on how the common interpretation of what is signal and noise changes in multivariate decoding. Finally, we use four examples to illustrate where these confusions may impact the interpretation of neuroimaging data. We conclude with a discussion of potential strategies to help resolve these confusions in interpreting multivariate decoding results, including the potential departure from multivariate decoding methods for the study of brain function.
Article
Full-text available
Author Although it is well studied how averaged activation of a brain region relates to behavior, it is still unclear if specific patterns of activation within regions also relate to cognitive function. In recent years, several methods have been developed for manipulating brain activity in humans. Real-time functional magnetic resonance imaging decoded neurofeedback (fMRI DecNef) is a method that allows the induction of specific patterns of brain activity by measuring the current pattern, comparing this to the pattern to be induced, and giving the subjects feedback on how close the two patterns of neuronal activity are. Using fMRI DecNef, we manipulated the pattern of activation in the cingulate cortex?a part of the cerebral cortex that plays a role in preference to different categories including faces and daily items?and tested whether we could change these preferences. In the experiment, a specific activation pattern in the cingulate cortex corresponding to higher (or lower) preference was induced by fMRI DecNef while subjects were seeing a neutrally preferred face. As a result, these neutrally preferred faces became more (or less) preferred. Our finding suggests that different patterns of activation in the cingulate cortex represent, and are sufficient to determine, different emotional states. Our new approach using fMRI DecNef may reveal the importance of activation patterns within a brain region, rather than activation in a whole region, in many cognitive functions.
Article
Though the computation of agency is thought to be based on prediction error, it is important for us to grasp our own reliability of that detected error. Here, the current study shows that we have a meta-monitoring ability over our own forward model, where the accuracy of motor prediction and therefore of the felt agency are implicitly evaluated. Healthy participants (N=105) conducted a simple motor control task and SELF or OTHER visual feedback was given. The relationship between the accuracy and confidence in a mismatch detection task and in a self-other attribution task was examined. The results suggest an accuracy-confidence correlation in both tasks, indicating our meta-monitoring ability over such decisions. Furthermore, a statistically identified group with low accuracy and low confidence was characterized as higher schizotypal people. Finally, what we can know about our own forward model and how we can know it is discussed.
Article
In adult life, people normally know what they are doing. This experience of controlling one's own actions and, through them, the course of events in the outside world is called 'sense of agency'. It forms a central feature of human experience; however, the brain mechanisms that produce the sense of agency have only recently begun to be investigated systematically. This recent progress has been driven by the development of better measures of the experience of agency, improved design of cognitive and behavioural experiments, and a growing understanding of the brain circuits that generate this distinctive but elusive experience. The sense of agency is a mental and neural state of cardinal importance in human civilization, because it is frequently altered in psychopathology and because it underpins the concept of responsibility in human societies.
Article
We take the feeling that our body belongs to us for granted. However, recent research has shown that it is possible to alter the subjective sensation of body ownership (BO) by manipulating multisensory bodily inputs. Several frontal and parietal regions are known to specifically process multisensory cues presented close to the body, i.e., within the peripersonal space (PPS). It has been proposed that these PPS fronto-parietal regions also underlie BO. However, most previous studies investigated the brain mechanisms of either BO or of PPS processing separately and by using a variety of paradigms. Here, we conducted an extensive meta-analysis of functional neuroimaging studies to investigate PPS and BO processing in humans in order to: a) assess quantitatively where each one of these functions was individually processed in the brain; b) identify whether and where these processes shared common or engaged distinct brain mechanisms; c) characterize these areas in terms of whole-brain co-activation networks and functions, respectively. We identified (i) a bilateral PPS network including superior parietal, temporo-parietal and ventral premotor regions and (ii) a BO network including posterior parietal cortex (right intraparietal sulcus, IPS; and left IPS and superior parietal lobule, SPL), right ventral premotor cortex, and the left anterior insula. Co-activation maps related to both PPS and BO encompassed largely overlapping fronto-parietal networks, but whereas the PPS network was more frequently associated with sensorimotor tasks, the BO network was rather associated with attention and awareness tasks. Finally, the conjunction analysis showed that (iii) PPS and BO tasks anatomically overlapped only in two clusters located in the left parietal cortex (dorsally at the intersection between the SPL, the IPS and area 2 and ventrally between areas 2 and IPS). Distinct activations were located at the temporo-parietal junction (PPS) and the anterior insula (BO). These results highlight the brain regions consistently involved in PPS and BO and provide evidence-based insight about the overlap of the two processes in the IPS region and the extensive connectivity between the two associated co-activation networks. They also show significant dissociations, with PPS fronto-parietal areas located more proximal to the central sulcus than BO areas. Such anatomical distinction may also reflect the different functions of the two processes, whereby PPS areas underlie a multisensory-motor interface for body-objects interaction and BO areas being involved in bodily awareness and self-consciousness.
Article
Anomalous agency has been reported clinically and empirically for people with schizophrenia. This finding is expected to contribute to understanding positive symptomatology in schizophrenia in terms of a general neurocomputational model of motor control, because anomalous agency has also been reported in schizotypal traits in the general population. However, superficially opposite conclusions have been suggested: over-attributed or under-attributed agency in patients. In this work, healthy participants (N=104) were presented continuous morphed self-other visual feedback of their reaching movements and rated the agency they felt for it. The slope of the regression line in stimuli-response coordination as a function of self-other discriminability was estimated for each participant. The estimated slopes were significantly associated with positive schizotypal symptomatology. Higher schizotypal participants exhibited a lower slope, indicating poorer discriminability between their own and others’ movements. Furthermore, the estimated regression lines in the high and low groups are predicted to cross at the approximately center point in the coordinates, which should produce both over-attribution and under-attribution errors for the high group compared with low group. The pattern of schizotypal attribution error depends on the S/N (signal-to-noise) ratio of the given stimuli within our sensorimotor system where the self-originated stimulus is the signal to be detected. The current study, for the first time, suggests both over- and under-attribution within participants scoring high on schizotypy.
Article
The temporo-parietal junction (TPJ) is implicated in a variety of processes including multisensory integration, social cognition, sense of agency and stimulus-driven attention functions. Furthermore, manipulation of cortical excitation in this region can influence a diverse range of personal and interpersonal perceptions, from those involved in moral decision making to judgments about the location of the self in space. Synthesis of existing studies places the TPJ at the neural interface between mind and matter, where information about both mental and physical states is processed and integrated, contributing to self-other differentiation. After first summarising the functions of the TPJ according to existing literature, this narrative review aims to offer insight into the potential role of TPJ dysfunction in neuropsychiatric disorders, with a focus on the involvement of the right TPJ in controlling representations relating to the self and other. Problems with self-other distinctions may reflect or pose a vulnerability to the symptoms associated with Tourette syndrome, Schizophrenia, Autistic Spectrum Disorder and Obsessive Compulsive Disorder. Further study of this most fascinating neural region will therefore make a substantial contribution to our understanding of neuropsychiatric symptomatology and highlight significant opportunities for therapeutic impact.
Article
Intentional action is essential to human behavior, yet its neural basis remains poorly understood. In order to identify neural networks specifically involved in intentional action, freely chosen and externally cued intentions have previously been contrasted. This has led to the identification of a fronto-parietal network, which is involved in freely choosing one's intentions. However, it remains unclear whether this network encodes specific intentions, or whether it merely reflects general preparatory or control processes correlated with intentional action. Here, we used MVPA on fMRI data to identify brain regions encoding non-motor intentions that were either freely chosen or externally cued. We found that a fronto-parietal network, including the lateral prefrontal cortex, premotor, and parietal cortex, contained information about both freely chosen and externally cued intentions. Importantly, MVPA cross-classification indicated that this network represents the content of our intentions similarly, regardless of whether these intentions are freely chosen or externally cued. This finding suggests that the intention network has a general role in processing and representing intentions independent of their origin.