Modeling Sense of Agency using Free Energy

Abstract

Sense of agency is the sense that an event is caused by oneself. In the context of man-machine interfaces, it has been proposed that the prediction error of a sensory outcome decreases the sense of agency. We assume that the information content of the sensory response represents the prediction error and thus explains the extent of sense of agency. We use the negative free energy as the information content and formulate the extent of sense of agency as a function of prediction error and sensory precision. The model predicts an interaction effect between the prediction error and sensory precision on the sense of agency. We conducted an experiment through a mouse-clicking task with participants. Between single cue and multiple cues, sensory responses are compared with respect to the effect of response delay as a prediction error on participants’ reported sense of agency value. According to the cue integration model, multiple cues provide more temporal precision of sensory response than a single cue. The results of the experiment supported the interaction effect predicted by the proposed free energy model. Therefore, negative free energy works as a mathematical index of sense of agency.

Full text

1
ISASE 2021
Modeling Sense of Agency using Free Energy
Kensaku TANIYAMA*, Takuma MAKI ** and Hideyoshi YANAGISAWA ***
* The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656, Japan
taniyama2032@mail.design.t.u-tokyo.ac.jp
** The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656, Japan
maki427@mail.design.t.u-tokyo.ac.jp
*** The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656, Japan
hide@mech.t.u-tokyo.ac.jp
Abstract: Sense of agency is the sense that an event is caused by oneself. In the context of man-machine interfaces, it has been
proposed that the prediction error of a sensory outcome decreases the sense of agency. We assume that the information content of the
sensory response represents the prediction error and thus explains the extent of sense of agency. We use the negative free energy as the
information content and formulate the extent of sense of agency as a function of prediction error and sensory precision. The model
predicts an interaction effect between the prediction error and sensory precision on the sense of agency. We conducted an experiment
through a mouse-clicking task with participants. Between single cue and multiple cues, sensory responses are compared with respect
to the effect of response delay as a prediction error on participants’ reported sense of agency value. According to the cue integration
model, multiple cues provide more temporal precision of sensory response than a single cue. The results of the experiment supported
the interaction effect predicted by the proposed free energy model. Therefore, negative free energy works as a mathematical index of
sense of agency.
Keywords: sense of agency, free energy, Bayesian surprise, cue integration, interface design
1. INTRODUCTION
Sense of agency (SoA) is the feeling of control over
one’s actions [1]. It affects one’s sense of responsibility
over the consequences or the joy resulting from one’s
action [2,3]. SoA is important in the context of man-
machine interface design. We believe that SoA can be
designed properly through mathematical modeling.
SoA is affected by prediction error [4-6]. Humans
predict the results of the action before it is initiated, and
estimate its results based on the sensory outcome after it
is completed. The prediction error is the discrepancy
between the predicted results and the estimated actual
results. We can obtain a high SoA when the prediction
error is small. For example, response delay is usually
manipulated experimentally as a prediction error. People
predict an immediate sensory outcome when they operate
a machine. SoA decreases in the case where response
delay is added.
The brain is a machine that processes information. A
large prediction error indicates an information overload.
We considered modeling SoA using information content
on information theory.
2. MODELING
2.1 Formalizing SoA using free energy
SoA is affected by the difference between the predicted
state and estimated actual state. In case the difference is
large, we are surprised because we receive significant new
information. We considered that the information content
in information theory corresponds to the amount of
perceived prediction error.
According to information theory, the information
content we obtain through an event is expressed as
!"#$%
when the probability of the event is
%
. Following this, the
information content we obtain when we obtain the sensory
outcome
&
is expressed as (1).
!"#$%
'
&
( '
)
(
Humans estimate the state of the outside world based
on the sensory outcome. By applying the Bayesian
theorem to (1), we obtain equation (2).
!"#$%
'
&
(
*
+
!"#$%
'
,-&
(.
!
(
,
|
&
)
!/
0
%
'
,
1
&
(2
34
'
5
(

2
where
,
is the state of the outside world.
4
is called free
energy as per the analogy of thermodynamics. The first
and second terms correspond to the internal energy and
entropy, respectively. Free energy is decomposed into two
other terms, as shown in equation (3).
4*67
'
%
'
,
1
&
(8
%
'
,
((
9
+
!"#$%
'
&
1
,
(.
!
(
,
|
&
)'
:
(
where
;%
'
,
( is the prior distribution,
%
'
&
1
,
( is the
likelihood, and
%
'
,
1
&
( is the posterior distribution. The
first term on the right side is called Bayesian surprise or
information gain. The second term on the right side is
called uncertainty or accuracy. Bayesian surprise is the
Kullback-Leibler divergence between posterior and prior
distribution. It can be interpreted as the information
content derived by calculating the difference between the
predicted state and the perceived state. Uncertainty is the
negative log likelihood averaged by the posterior
distribution and can be interpreted as the information
content derived by evaluating the plausibility of the
estimation of the actual state. We believe that both
Bayesian surprise and uncertainty affect SoA.
We propose a negative free energy model as a
mathematical model that explains SoA as (4).
<=>?!4
'
@
(
2.2 Hypothesis
We approximately represent prior distribution and
likelihood as Gaussian distributions. Negative free energy
is expressed as shown in equation (5).
!4* !)
5
A
)
&!9&%B&9"#$5C
0
&!9&%
2D'
E
(
where the means of prior distribution and likelihood are
F!
and
F%
. The variances of prior distribution and
likelihood are
&!
and
&%
.
B
is the prediction error meaning
the difference between
F!
and
F%
. Prediction error refers
to the difference between the predicted and estimated
actual states.
&!
is the prior uncertainty of prediction,
which is simply called uncertainty.
&%
is likelihood
variance. Likelihood variance is interpreted as the inverse
of the sensory reliability.
Equation (5) is a quadratic equation of
B
. We conducted
partial derivatives of (5) as (6) and (7).
!4* GB&9H
G* ! )
5
0
&!9&%
2
-H* !)
5"#$5C
0
&!9&%
2
IG
I&!JK-IH
I&!LK
'
M
(
IG
I&%JK-IH
I&%LK
'
N
(
This result indicates that negative free energy decreases
and prior uncertainty or likelihood variance increases
when the prediction error is zero. Furthermore, the degree
of negative free energy decreases because the prediction
error increase becomes small when prior uncertainty or
likelihood variance increases. Figure 1 shows the negative
free energy as a function of the prediction error for
different prior uncertainties. Figure 2 shows the negative
free energy as a function of the prediction error for
different likelihood variances. This simulation predicts
two things. The first is that in the case where the prediction
error is small, SoA increases as prior uncertainty
decreases. On the other hand, in the case where the
prediction error is large, SoA increases as prior
uncertainty increases. We previously verified this
phenomenon [7]. The second prediction is that in the case
where the prediction error is small, SoA increases as the
likelihood variance decreases. However, in the case where
the prediction error is large, SoA increases as the
likelihood variance increases. This phenomenon has not
yet been verified.
We hypothesized that a large SoA is achieved by a
small likelihood variance under a small prediction error
condition. On the other hand, it is also achieved by a large
likelihood variance under a large prediction error
condition.
Figure 1: Negative free energy as a function of
prediction error for different prior uncertainties

3
Figure 2: Negative free energy as a function of
prediction error for different likelihood variances
2.3 Cue integration model
As shown in (7), we predicted the effect of likelihood
variance on SoA. Likelihood variance is interpreted as the
inverse of sensory reliability.
According to the cue integration model [8], the
uncertainty of the estimation based on sensory outcome
decreases as multiple cues are integrated. Cue is the
sensory outcome or memory needed to estimate the actual
state. The likelihood variance of the total sensory outcome
O&
is expressed as equation (8) when sensory outcomes 1
and 2 are integrated.
)
O&*)
O'
&9)
O&
&
'
P
(
This model shows that the likelihood variance of
multiple sensory outcomes is smaller than that of one
sensory outcome. Figure 3 shows an outline of the model.
Figure 3: Model of the effect of the sensory outcome
on SoA
3. EXPERIMENT
3.1 Method
We verified the hypothesis derived from the negative
free energy model and cue integration model. Our
hypothesis is that a high SoA is achieved by multiple
sensory responses under a small prediction error condition
and by one sensory response under a large prediction error
condition.
When subjects click the mouse of a computer, they
receive sensory responses after a certain delay. To verify
this hypothesis, we manipulated response delay as a
prediction error and the number of sensory responses as
the likelihood variance. The response delays utilized were
0, 50, 100, 150, 200, 250, 300, 400 and 500ms. The types
of sensory responses were only sound (S) and only
vibration (V) as large likelihood variance and sound-
vibration (SV) as small likelihood variance.
The experiment consisted of two blocks. In the
adaptation block, subjects learned the 0ms response delay
freely for 2 minutes. After the adaptation block, an
evaluation block was conducted. The evaluation block
was subdivided into S, V, and SV sessions. In each session,
all 18 subjects experienced 9 levels of response delay
randomly and evaluated SoA on each level using a
questionnaire.
3.2 Results
Figure 4 shows result of the value of SoA as functions
of response delays with respect to different sensory
response conditions. The value of SoA decreased as
response delay increased for all modality conditions. In
the case of a 0ms response delay, the value of SoA under
the SV condition is higher than that under the S condition
or V condition. In the case of a 500ms response delay, the
value of SoA under the S condition and V condition is
higher than that under the SV condition.
Figure 4: Value of SoA for different response delays
with respect to conditions of response modalities

4
Table 1 shows the results of the two-way ANOVA.
Both the response delay and the number of response
modalities affected the SoA value (p<0.01). The response
delay and the number of modalities displayed interaction
(p<0.01).
Table 1: Interaction effect of response delay and
number of response modalities
(Results of two-way ANOVA)
4. DISCUSSION
In the case of a 0ms response delay, the SoA value under
the SV condition was higher than that under the S
condition or the V condition. In the case of a 500ms
response delay, the SoA score under the S condition and
the V condition was higher than that under the SV
condition. The interaction effect between the response
delay and the number of response modalities was
observed. These results correspond to the model
predictions illustrated in Figure 2. Thus, the results of the
experiment supported our negative free energy model of
SoA.
No interaction effect was observed between the S
condition and the V condition. This suggests that the
difference between auditory and tactile organs with
respect to the estimation of time is negligible.
Auditory and tactile stimulus timings are perceived
precisely. Participants perceived a large SoA by receiving
auditory and tactile stimuli when the machine’s response
delay was short. On the other hand, the SoA is secured by
receiving a single modality stimulus or a stimulus that low
reliability in terms of timing estimation.
5. CONCLUSION
In this study, we proposed a negative free energy model
of SoA. The model predicts that a high SoA is achieved
by small likelihood variance under a small prediction error
condition and by a big likelihood variance under a large
prediction error condition. We verified this model
prediction by comparing the SoA under varying degrees
of response delay and various response modalities.
This model prediction provides a guideline for
designing SoA in a man-machine interface. If the
machine’s response delay is long, the modality of the
sensory response should be single. If the machine
possesses multiple response modalities, the response
delay should be as short as possible.
ACKNOWLEDGMENTS
This work was supported by Sony Global Manufacturing
& Operations Corporation.
REFFERENCES
[1] Haggard, P. (2017). Sense of agency in the human
brain. Nature Reviews Neuroscience, 18(4), 196–207.
[2] Moretto, G., Walsh, E., & Haggard, P. (2011).
Experience of agency and sense of responsibility.
Consciousness and cognition, 20(4), 1847–1854.
[3] Karsh, N., & Eitam, B. (2015). I control therefore I do:
judgments of agency influence action selection. Cognition,
138, 122–131.
[4] Sato, A., & Yasuda, A. (2005). Illusion of sense of
self-agency: discrepancy between the predicted and actual
sensory consequences of actions modulates the sense of
self-agency, but not the sense of self-ownership.
Cognition, 94(3), 241–255.
[5] Frith, C. D., Blakemore, S. J., & Wolpert, D. M. (2000).
Abnormalities in the awareness and control of action.
Philosophical transactions of the Royal Society of London.
Series B, Biological sciences, 355(1404), 1771–1788.
[6] Yang, Q., & Yanagisawa, H. (2019). Effect of
Latency and Space Discrepancy on Sense of Agency.
International Symposium on Affective Science and
Engineering, ISASE2019(0), 1–6.
[7] Bamba, M., & Yanagisawa, H. (2021). Modelling
sense of agency using information gain (An experimental
evidence using varied response delay). Transactions of
the JSME (in Japanese), 87(893), 20-00035-20–00035.
[8] Moore, J. W., & Fletcher, P. C. (2012). Sense of
agency in health and disease: a review of cue integration
approaches. Consciousness and cognition, 21(1), 59–68.