Content uploaded by Hideyoshi Yanagisawa
Author content
All content in this area was uploaded by Hideyoshi Yanagisawa on Jul 16, 2019
Content may be subject to copyright.
ORIGINAL ARTICLE
A computational model of perceptual expectation eﬀect based
on neural coding principles
Hideyoshi Yanagisawa
Department of Mechanical Engineering, The
University of Tokyo, Tokyo, Japan
Correspondence
H. Yanagisawa, Department of Mechanical
Engineering, The University of Tokyo, 73
1, Hongo, Bunkyo, Tokyo, Japan.
Email: hide@mail.design.t.utokyo.ac.jp
Funding Information
This work was supported by KAKEN (No.
23760127).
Abstract
Prior expectation aﬀects posterior perceptual experience. This contextual bias is called expectation
eﬀect. Previous studies have observed two diﬀerent patterns of expectation eﬀect: contrast and
assimilation. Contrast magniﬁes the perceived incongruity, and assimilation diminishes the incon
gruity. This study proposes a computational model that explains the conditions of contrast and
assimilation based on neural coding principles. This model proposed that prediction error, uncer
tainty, and external noise aﬀected the expectation eﬀect. Computer simulations with the model
show that the pattern of expectation eﬀect shifted from assimilation to contrast as the prediction
error increased, uncertainty decreased the extent of the expectation eﬀect, and external noise
increased the assimilation. We conducted an experiment on the size–weight illusion (SWI) as a
case of the crossmodal expectation eﬀect and discussed correspondence with the simulation. We
discovered conditions where the participants perceived bigger object to be heavier than smaller
one, which contradicts to conventional SWI.
Practical applications
Expectation eﬀect in sensory perception represents a perceptual bias caused by prior expectation,
such as illusions and crossmodality. The computational model proposed in this study guides
researchers and practitioners who investigate this bias in sensory studies to set a hypothesis with
appropriate experimental factors. For example, the model suggests that prediction error can be
used as a main factor to identify a condition at which assimilation switches over to contrast. The
model provided how expectation uncertainty and noise of stimulus aﬀect the switchover point of
prediction error and extent of expectation eﬀect. Uncertainty, which may diﬀer from person to
person, can be used as a factor to explain personal diﬀerences in the extent of expectation eﬀect.
1

INTRODUCTION
Expectation congruity works as an appraisal component that evokes
emotions, such as surprise (Ludden, Schiﬀerstein, & Hekkert, 2012),
satisfaction (Oliver, 1977, 1980), and disappointment (Demir, Desmet,
& Hekkert, 2009). For example, a user expects the usability from a
product appearance (“This looks easy to use.”), and determines the
actual usability (“This is really easy to use.”). A positive expectation dis
crepancy (“This is easier to use than expected!”) provides a satisfaction.
In contrast, a negative expectation discrepancy (“This is harder to use
than expected.”) disappoints the user. Furthermore, prior expectations
aﬀect and change posterior perception and experience. Researchers
from a broad range of ﬁelds have observed this psychological bias, the
socalled expectation eﬀect, with regard to diﬀerent cognitive proc
esses, such as desire for rewards (Schultz, Dayan, & Montague, 1997),
emotions (Geers & Lassiter, 1999; Wilson, Lisle, Kraft, & Wetzel, 1989),
and sensory perceptions (Deliza & MacFie, 1996; Schiﬀerstein, 2001).
The expectation eﬀect changes the disconﬁrmation between expecta
tion and experience. Thus, the expectation eﬀect is an essential factor
to ensure the satisfactory design of products and services.
In a time sequence of UX of a product and service, users shift
from one sensory state to another in cyclic interactions involving
action, sensation, and perception (Krippendorﬀ, 2005). Figure 1 illus
trates an example of such state transitions while using a camera. A user
perceives the appearance by looking (vision), the tactile feeling by
grasping (touch), and shutter sound by popping a shutter (audition). We
expect that users would predict subsequent states between these tran
sitions of state (e.g., we expect a meal to taste a certain way based on
how it looks, the weight of a product before lifting it, the usability of a
mouse by looking at it). This prior prediction aﬀects posterior
430

V
C2016 Wiley Periodicals, Inc. wileyonlinelibrary.com/journal/joss J. Sens. Stud. 2016; 31: 430439
Received: 28 April 2016

Revised: 25 June 2016

Accepted: 16 August 2016
DOI 10.1111/joss.12233
perception, that is, the expectation eﬀect. For example, visual expecta
tion changes tactile perceptions of surface texture (Yanagisawa &
Takatsuji, 2015a).
We can explain a kind of perceptual illusion using the expectation
eﬀect. For example, people perceive a smaller object as heavier than a
larger one although the weight of both objects is identical (Flanagan,
Bittner, & Johansson, 2008). This wellknown size–weight illusion
(SWI) can be explained as a visual expectation eﬀect. People expect a
larger object to be heavier than a smaller one. Prior visual expectation
of the objects’weights magniﬁes the perception of diﬀerence between
the expected and actual weights. Although many experimental ﬁndings
exist on the expectation eﬀect in diﬀerent disciplines, the general
mechanism on why and how the eﬀect occurs is not yet clearly eluci
dated. A mathematical model of the expectation eﬀect based on a fun
damental mechanism enables us to estimate user perception of
product and service. Yanagisawa and Takatsuji (2015b) proposed a
mathematical model of the expectation eﬀect using information theory.
They modeled prior expectation as a subjective probability distribution
and hypothesized that Shannon’s entropy of the distributions repre
senting uncertainty of prior expectation determines the occurrence of
the expectation eﬀect. An experimental result of the visual expectation
eﬀect of tactile texture showed that the lower entropy invoked expec
tation eﬀect in a higher rate than another. This ﬁnding suggests that
uncertainty is a factor of occurrence of expectation eﬀect.
On the other hand, two diﬀerent patterns of expectation eﬀect,
contrast and assimilation, were observed (Deliza & MacFie, 1996). Con
trast is a bias that magniﬁes the diﬀerence between prior expectation
and posterior experience. Assimilation is a bias that diminishes expecta
tion incongruence. It is important to understand whether the expecta
tion eﬀect is contrasting or assimilating, because they exaggerate or
diminish the perception of expectation disconﬁrmation as a factor of
satisfaction. However, the mechanisms and conditions governing the
contrasting and assimilating patterns are not yet clearly elucidated. In
this study, we propose a simulation model of the expectation eﬀect
that explains the conditions of contrast and assimilation by applying
neural coding principles, such as Bayesian decoding and the eﬃcient
encoding principles. Based on the proposed model, we conduct com
puter simulations of the expectation eﬀect and obtain an accurate
hypothesis of the conditions of assimilation and contrast. Finally, we
apply the obtained hypothesis to externalize a perceptual law behind
SWI as an expectation eﬀect from a result of an experiment with the
participants.
2

COMPUTATIONAL MODEL OF
EXPECTATION EFFECT
2.1

Bayesian coding in perception involving prior
expectation
We deﬁne perception as an estimation of external physical property,
such as the weight of an object. Sensory stimulus from the external
physical world, such as pressure applied to a hand, is transformed to
patterns of neural signals. We call the neural representation of an
external physical variable encoding. Based on the pattern of neural sig
nals, our brain estimates the physical variable. We call this estimation
process decoding. We assume that sensory stimuli are encoded as cer
tain ﬁring rates of neural populations. This type of neural coding is
called rate coding. Another type of encoding is a temporal coding
hypothesis where the precise timing of spikes adds information. How
ever, most studies of temporal coding have concentrated on single
brain areas and only a few have reported how timing codes are trans
formed across diﬀerent stages of a sensory pathway. Evidence for tem
poral coding is less established in higher cognitive brain area (Quian
Quiroga & Panzeri, 2009). Thus, we assume rate coding for further dis
cussion. Based on the ﬁring rate distributions from a sensory stimulus,
R, our brain forms the likelihood function,PRjuðÞ, of a physical variable,
u. On the other hand, a physical property has certain frequency distri
butions in the world. Human beings learn these frequency distributions
throughout their life. Based on these learnt distributions, human beings
predict a physical variable, u, before experiencing sensory stimulus. For
example, in the SWI, people predict the weight of an object by looking
at it before actually lifting it up. The predicted physical variable should
follow certain probability distributions. We deﬁne this distribution as
prior,PðuÞ. Recent studies in neuroscience showed that the estimation
of a physical variable, that is, decoding, follows the Bayesian estimator
(Brayanov & Smith, 2010; Ernst & Banks, 2002). Based on Bayes’theo
rem, our brain estimates the distributions of perceptions or posterior,
PujR
ðÞ
, using prior and likelihood.
PujRðÞ5PRju
ðÞ
Pu
ðÞ
XXPRjuðÞPuðÞ (1)
Since the denominator of the righthand side of Equation 1isa
constant for normalization, the posterior is proportional to the product
of prior and likelihood.
PujRðÞ/PRjuðÞPuðÞ (2)
If the distributions of prior and likelihood follow the Gaussian dis
tributions and are independent from each other, we can calculate the
optimal estimate of posterior ^
uby the following equation (K€
ording &
Wolpert, 2004).
^
u5V½PðuÞE½PðRjuÞ1V½PðRjuÞE½PðuÞ
V½PðuÞ1V½PðRjuÞ (3)
where EðpÞdenotes mean of distribution p,andVðpÞdenotes a var
iance of p. The numerator of the right side in Equation 3represents
sum of prior mean EðPðuÞÞ weighted with likelihood variation VðPðRjuÞÞ
and likelihood mean EðPðRjuÞÞ weighted with prior variation VðPðuÞÞ.
appearance
vision
look
tactile feel
touch
grasp
shutter sound
audition
press button
FIGURE 1 Sensory transitions of product use (a case of camera)
YANAGISAWA

431
The denominator of the right side in Equation 3 is a sum of variations
for normalization. The estimate is in the middle of prior mean and likeli
hood mean when the prior variation and likelihood variation are same.
The estimate comes close to the prior mean when prior variation is big
ger than likelihood variation. In an opposite manner, the estimate
comes close to likelihood when likelihood variation is bigger than prior
variation. Equation 3 indicates that the posterior estimate, ^
u,isalways
between prior mean and likelihood peak. Figure 2 illustrates how pos
terior estimate comes close to a peak of prior from a peak of the likeli
hood estimate of the sensory stimulus. We call the eﬀect attractive
inﬂuence of prior.
2.2

Expectation eﬀect and prediction error
Now, we can write the expectation eﬀect, E,andprediction error,d
E5
^
u2u
lik (4)
d5ulik2E½PðuÞ (5)
where ulik denotes a likelihood peak. Using Eand d, we can formalize
the condition of contrast as jd1Ej>jdjand the condition of assimilation
as jd1Ej<jdj.jd1Ejrepresents the extent of perceived discrepancy
with expectation eﬀect. Figure 3 illustrates contrast and assimilation.
Contrast is a type of expectation eﬀect that exaggerates perceived dis
crepancy, whereas assimilation is another type of expectation eﬀect
that diminishes perceived discrepancy. Assume that the horizontal axis
of Figure 3 is weight, contrast makes people feel heavier than actual
weight, whereas assimilation makes people feel lighter than the actual
weight.
2.3

Eﬃcient coding causes contrast
The attractive inﬂuence of prior alone involves assimilation as an
expectation eﬀect. The question then arises: How does contrast occur?
We applied a neural encoding framework based on the eﬃcient coding
principle (Wei & Stocker, 2012). According to the encoding framework,
the Bayesian estimate shifts away from the peaks of the prior distribu
tion. This phenomenon corresponds to the contrast pattern of the
expectation eﬀect. Eﬃcient coding hypothesis proposes that the tuning
characteristics of a neural population are adapted to the prior distribu
tion of a sensory variable such that the neural population optimally rep
resents the sensory variable (Barlow, 1961). This optimal neural coding
makes the total expected spike count of each neuron in the population
similar. To mathematically represent this code, the neural encoding
framework assumed an equivalent homogeneous space with units ~
uin
which the stimulus prior distribution is uniform and mapped the physi
cal space with units uto the homogeneous space using a mapping
function F. The framework used cumulative of the prior Pas the map
ping function as follows.
~
u5FuðÞ5ðu
21
PxðÞdx (6)
To map from ~
uto u, we can use the inverse function F21ð~
uÞ.Fig
ure 4 illustrates the tuning curves of eﬃcient neural populations and
the mapping to/from the homogeneous space for a prior of normal dis
tributions pðuÞ. In the homogeneous space, we assume similar turning
curves for each neural population. The turning curves are assigned at
equally spaced intervals so that the expected spike count of each pop
ulation is the same for a uniform prior. Then, we map the tuning curve
of each neural population to the physical space using F21.Figure4
shows that the tuning curves in the physical space are dense around
prior expectation (high prior density) and sparse toward away from the
expectation (on the side of low prior density). In summary, more neural
populations are allocated to sense a range of physical values that are
more likely to occur than others. This coding strategy maximizes
expected information amount for a prior. Due to the mapping using F,
the shape of tuning curve in the physical space gradually becomes
asymmetrical towards the side of lower prior density. This is attributable
to the shape of the mapping function F. The mapping function shapes S
curve that closes to a linear around the prior expectation, but becomes
nonlinear toward away from prior mean. We assume that the likelihood
shape follows the shape of the tuning curves in the physical space.
The likelihood shape is constrained in physical space by the prior
distribution. The likelihood distributions in physical space transformed
from symmetric distributions, such as Gaussian, in homogeneous space
FIGURE 2 Attractive inﬂuence of prior in Bayesian estimate
FIGURE 3 Illustration of contrast and assimilation
432

YANAGISAWA
shows heavier tails on the side of lower prior density. In sum, the eﬃ
cient encoding typically leads to an asymmetrical likelihood function
whose mean value is away from the peak of prior. As shown in Equa
tion 2, the Bayesian estimate is determined by a combination of prior
and shifted likelihood means,
ulik, and it shifts away from the prior peak
(Figure 5).
We apply this eﬃcient encoding to explain contrast in our model.
The Bayesian estimate (perceived value), ^
u, shifts from a peak of the
asymmetric likelihood function away from a peak of prior. We call the
perceptual shift repulsion inﬂuence.
2.4

Perceptual model with prior expectation
From the above discussion using Bayesian decoding and eﬃcient cod
ing principle, we can summarize our hypothetical model of perception
as shown in Figure 6. Based on the eﬃcient encoding principle, prior
changes the shape of the likelihood function asymmetry while encod
ing the sensory stimulus of the physical variable, u,asaﬁring rate of
the neuron population R. The Bayesian decoder integrates the prior
distribution, PðuÞ, and asymmetric likelihood function, PðRjuÞ, and forms
posterior distributions. As a result, we perceive a peak of the posterior,
^
u, as an estimate of the physical variable, that is, perception.
2.5

Factors of expectation eﬀect: prediction error,
uncertainty, and external noise
Based on the eﬃcient coding, the likelihood shape shows heavier tails
on the side of lower prior density, and the likelihood means shifts away
from the prior peak. The extent of likelihood asymmetry and the likeli
hood mean shift increases towards away from a prior peak. Thus, we
can predict that the repulsion bias increases towards away from a prior
peak. (Wei & Stocker, 2012) mathematically derived the prediction.
Based on their mathematic derivation, we can approximate the repul
sion inﬂuence at prediction error, bðuÞ, in the following diﬀerential
equation.
bðuÞCd
du
1
PðuÞ2
! (7)
where Cis a positive constant, and PðuÞis prior distributions. If the
prior is Gaussian with zero mean, the gradient of 1=PðjdjÞ2is positive
for any prediction error dand increases away from mean of prior. Thus,
Equation 7 indicates that the repulsion bias increases as the absolute
prediction error jdjincreases. This is because the extent of asymmetry
of likelihood increases away from the peak of prior. Repulsion inﬂuence
involves contrast. Thus, the prediction error is a key factor that decides
the condition of the expectation eﬀect.
We assume two more factors of the expectation eﬀect: external
noise and uncertainty. The shape of the likelihood function is aﬀected
by the noise of the external stimulus. An external noise modiﬁes the
FIGURE 4 How eﬃcient coding makes likelihood in physical space asymmetry
lik
θ
lik
θ
ˆ
θ
δ
ε
FIGURE 5 Repulsion inﬂuence due to asymmetry of likelihood
function
YANAGISAWA

433
shape of the likelihood function by convolving it with noise
distributions.
ðfgÞðuÞ5ðfðxÞgðu2xÞdx (8)
Where fis the asymmetric likelihood function, and gis a symmet
ric external noise distribution. We assume that the external noise distri
bution follows the Gaussian distribution.
gðuÞNðulik;r2noise Þ(9)
Symmetric external noise distributions do not change the mean of
likelihood, but they increase its overall width. Thus, the attractive inﬂu
ence of prior relatively increases, and the Bayesian estimate, ^
u,shifts
toward the peak of prior. If the attractive inﬂuence of prior exceeds
the repulsion inﬂuence of the asymmetric likelihood, the expectation
eﬀect may change into assimilation from contrast.
Variations of prior distributions are indicators of prediction uncer
tainty. As shown in formula (3), the variation in prior impacts the attrac
tive inﬂuence. In the Bayesian estimation, a small variation in prior
denotes small uncertainty and involves a strong attractive inﬂuence.
Conversely, a big variation in prior means uncertain prediction and
involves weak attractive inﬂuence.
From the above discussions, we can assume that the condition and
extent of expectation eﬀect alters with the three factors, namely, pre
diction error, d, uncertainty, V½PðuÞ, and external noise, r2noise.
3

EFFECT OF PREDICTION ERROR,
EXTERNAL NOISE, AND UNCERTAINTY ON
EXPECTATION EFFECT
3.1

Method
To investigate the eﬀects of the three factors, that is, prediction error,
uncertainty, and external noise on the expectation eﬀect, we con
ducted a computer simulation using abovementioned model. We used
Gaussian distributions for prior, homogeneous likelihood, and posterior.
The following conditions were selected as simulation parameters: pre
diction error, d, within [0, 100]; uncertainty, V½PðuÞ, within [50, 200];
and external noise, r2noise , within [5, 50]. The standard deviation of the
homogeneous likelihood is set as 0.04. We calculated the expectation
eﬀect, E, using the simulation model for all combinations of the above
mentioned conditions for the three factors. We use MATLAB® to con
duct the simulations. The code is provided as a supporting information.
3.2

Result and discussion
Figure 7 shows an example of the simulation result of the expectation
eﬀect as a function of the prediction error. Each line represents a con
dition of uncertainty (small: 150, big: 180) and external noise (small: 30,
big: 40). A positive value represents contrast, and a negative value,
assimilation. Figure 7 reveals three ﬁndings.
The expectation eﬀect functions as an assimilating eﬀect when
the prediction error is small. As the prediction error increases, the
expectation eﬀect increases and changes to the contrasting condition
((1) in Figure 7). Around the peak of prior, where the prediction error is
small, the shape of the likelihood function was close to symmetry, the
repulsion inﬂuence was small, and the attractive inﬂuence of prior is
dominant. Thus, assimilation occurs. As the prediction error increases,
the extent of the likelihood asymmetry increases, and the repulsion
inﬂuence increases. Thus, the expectation eﬀect shifts to the contrast
condition.
ˆ
θ
P( )
θ
P(  )R
θ
θ
P( )R
θ
FIGURE 6 A model of perception involving prior expectation
0 20 40 60 80 100 120 140 160 180 200
Prediction error
1.5
1
0.5
0
0.5
1
1.5
2
2.5
Expectation effect
uncertainty=150, noise=30
uncertainty=150, noise=40
uncertainty=180, noise=30
uncertainty=180, noise=40
(2)
(2)
(2)
(2)
(3)
(1)
FIGURE 7 Expectation eﬀect as a function of expectation error
for diﬀerent conditions of expectation uncertainty and external
noise
434

YANAGISAWA
The extent of the expectation eﬀect, jEj, is bigger when uncer
tainty is lower for both assimilation and contrast ((2) in Figure 7). With
respect to assimilation, the attractive inﬂuence of prior increases in the
Bayesian estimation as the variation of prior (uncertainty) decreases.
On the other hand, the repulsive inﬂuence increases from a certain
value of prediction error as the variation of prior decreases. In sum, cer
tain predictions involve a sharp expectation eﬀect regardless of the
condition (contrast or assimilation).
The prediction error at which assimilation changes to contrast
increases as the external noise increases ((3) in Figure 7). External noise
weakens the repulsive inﬂuence. In the Bayes estimation, the attractive
inﬂuence of prior becomes stronger than the repulsive inﬂuence of
likelihood. Thus, the area of assimilation in the prediction error
increases when the external noise exceeds prediction error and uncer
tainty. We observed the abovementioned trends for all possible combi
nations of conditions for uncertainty and external noise.
Figure 8 shows normalized prediction error when assimilation
switches over to contrast as a function of uncertainty for diﬀerent
external noise condition. The prediction error, the vertical axis, is nor
malized between zero and one. In sum, the vertical axis represents the
ratio of assimilation area in the range of prediction error. Zero in the
vertical axis represents a case where only contrast occurs, whereas one
represents a case where only assimilation occurs. With respect to small
external noise such as 5, only contrast occurs for all uncertainty condi
tions. This is because the repulsion inﬂuence due to eﬃcient coding is
dominant with small external noise. Except the condition of noise 5,
only assimilation occurs when uncertainty is small, such as 50 for all
noise conditions. This is because attractive inﬂuence with small uncer
tainty is dominant in Bayesian decoding (as indicated in formula [3]).
The assimilation area decreases as uncertainty increases from certain
uncertainty condition. This is because the attractive inﬂuence of prior
decreases against repulsion inﬂuence caused by eﬃcient coding.
Uncertainty at the area where assimilation starts to decrease increases
as noise increases because attractive inﬂuence becomes relatively
stronger than noisier likelihood in Bayesian decoding. The above results
suggest that the switchover prediction error, assimilation area, depends
on the balance between attractive inﬂuence of prior uncertainty in
Bayesian decoding and repulsion inﬂuence due to asymmetric likeli
hood in eﬃcient coding.
Figure 9 shows contours of prediction errors when assimilation
changes to contrast for all combinations of uncertainty and external
noise. The prediction error, the zaxis, is normalized between zero and
one. Zero in the contour represents a case where only contrast occurs,
whereas one represents a case where only assimilation occurs. Figure 9
shows that the area where uncertainty is high and external noise is
small denotes cases where only contrast occurs. In this area, the repul
sive inﬂuence of asymmetry likelihood is dominant compared to the
attractive inﬂuence of uncertain prediction. On the other hand, the
area with low uncertainty and big external noise shows only assimila
tion. The attractive inﬂuence of prior is dominant for certain predic
tions compared to the repulsive inﬂuence, which is weakened by the
external noise.
4

EXPERIMENT WITH SIZE–WEIGHT
ILLUSION
4.1

Method
As discussed in the Introduction Section, SWI is a signiﬁcant case of
the cross modal expectation eﬀect, where prior visual expectation
biases weight perception. To investigate the actual eﬀect of the
three factors (i.e., prediction error, uncertainty, and external noise)
on SWI, we conducted a sensory experiment with the participants.
We manipulated the three factors of the expectation eﬀect. For
each condition of the factors, we obtained responses from the par
ticipants with respect to perceptions of weight, and evaluated the
extent of weight illusions as expectation eﬀects. We presented the
participants with pairs of cubic metal objects. The objects in each
pair had identical weights but diﬀerent sizes. We asked the partici
pants to compare the weights and obtained their responses for the
diﬀerence in weights. We used the perceptual diﬀerence of weight
as the extent of the expectation eﬀect.
Uncertainty
Switchover from assimilation to contrast
5
15
25
35
45
FIGURE 8 Expectation errors when assimilation switch over to
contrast as a function of uncertainty for diﬀerent external noise
10 20 30 40 50
50
100
150
0.2
0.4
0.6
0.8
1
External noise
Uncertainty
Normalized expectation error
0
FIGURE 9 Expectation errors when assimilation shifts to contrast
for diﬀerent conditions of uncertainty and external noise. Data is
provided as a supporting information
YANAGISAWA

435
According to conventional SWI, human beings perceive that a
smaller object is heavier than a larger object, although both objects
may have the same weight. This illusion can be viewed as a contrast of
the expectation eﬀect, where the perception of diﬀerence between
the weight predicted by the object’s size and its actual weight, the pre
diction error, is exaggerated. However, our simulation result in Figure 4
showed that assimilation, an opposite eﬀect to contrast, occurs when
the prediction error is less than a certain value. To validate the simula
tion results in SWI, we controlled the prediction errors in the experi
ment. We prepared pairs of objects, called target and reference.
Participants evaluated the weight of the target by comparing it with
that of the reference. Figure 10 illustrates the weight and size of tar
gets (t1–t4) and references (r1–r4). All samples have identical surface
material. References are solid objects made by identical material with
equal density. All targets have identical size in appearance. The visual
expectation of weight for all targets are marked “x”in Figure 10 from
the identical appearance (size and surface material). However, we
adjusted each target weight similar to each compared reference. The
diﬀerence between the visually expected weight and the actual weight
works as the prediction error. In this setting, the target was always big
ger than the reference for each pair. Therefore, when target is lighter
than the reference denotes contrast, whereas when target is heavier
than the reference denotes assimilation.
To control uncertainty of visual predictions, we used a fogged glass
so that transparency between the participants and target object was
manipulated. We assumed that fuzzy visual images of the targets
would increase the uncertainties of size and weight predictions. To
control the external noise of somatosensory sensation while lifting an
object, we asked the participants to add a weight to the wrist he/she
used for lifting the samples. According to the Weber–Fechner Law, the
diﬀerence threshold increases as the intensity of stimulus increases.
We assumed that the additional weight of each participant’swrist
serves as the external noise of weight perception due to the increasing
diﬀerence threshold.
4.2

Material
We prepared ten solid cubic shapes made of duralumin (A2017) as a
set of reference samples. The weights of the reference samples ranged
from 350 g to 1250 g. Their sides were 50–76.5 mm long. For the tar
get samples, we prepared hollow cubes made of duralumin (A2017),
with sides measuring 80 mm. We inserted additional weights into the
hollow cubes so that the weights of both samples were identical for
each pair. We attached a wire to the top of each target and reference
sample and hung them from a steel framework to straighten the wires
without tension. We placed a ring in the middle of each wire to enable
the participants to lift the samples using their index ﬁngers.
4.3

Participants
Fifteen (twelve male and three female) volunteers aged 21 to 24 years
served as experiment evaluators. They were undergraduate or graduate
students studying mechanical engineering at the University of Tokyo.
All participants were physically healthy.
4.4

Procedure
The participants were invited individually into the isolated test room.
Each participant was seated on a chair in front of the framework, which
was set on a table. After obtaining informed consent, the participants
received written instructions for the procedure. Before starting the
comparison of the pairs, we asked the participants to lift up the ten ref
erence samples with their index ﬁngers using the wired ring to perceive
the density of the duralumin. After the learning session, the partici
pants compared the weights of the target and reference samples under
four diﬀerent combinations of external noise and uncertainty (Table 1).
To simulate the condition of big uncertainty (B and D in Table 1), we
placed a fogged glass between the target and the participant so that
the visual image of the target was fuzzy. For the big external noise con
dition (C and D in Table 1), a participant added a weight to the wrist
he/she used for lifting the samples. For each condition, we randomly
presented each pair of the 10 pairs of the target and reference samples
with identical weights. The presentation order diﬀered between partici
pants. We asked participants to alternately lift the target sample and
reference sample with the index ﬁnger of the dominant arm using the
wired ring. After they had lifted both samples, we asked them to rank
the target weight as “very much heavier,”“heavier,”“kind of heavier,”
“almost the same,”“kind of lighter,”“lighter,”and “very much lighter”in
comparison to the reference sample in that pair. We repeated the
paired comparisons of sample pair weights for all ten pairs for the four
conditions.
size (cm3)
mass (g)
reference
target
prediction error
expectation
comparison
x
r1
r2
r3
r4
t1
t2
t3
t4
FIGURE 10 Method for manipulating expectation error.
Participants compared the weights of the reference and target. The
diﬀerence between the predicted weight (x) and the actual weight
of the target were controlled as prediction errors for each
reference–target pair
TABLE 1 Experimental conditions regarding uncertainty and exter
nal noise
External noise
Small Big
Uncertainty Small A C
Big B D
436

YANAGISAWA
4.5

Data analysis
We used the participants’responses regarding the relative weights of
the target samples in comparison to those of the reference samples as
an index of the expectation eﬀect. As explained previously, the physical
weight of the target and reference samples in each pair were identical.
If the response was “almost the same,”we can infer that no expecta
tion eﬀect was observed. Due to the combination of learned density of
the material and the visually estimated volume, all the target samples
were actually lighter than the expected weight, whereas the reference
samples, which were solid and had congruent expected and actual
weights, were heavier. Thus, the participants’responses that the target
samples were heavier than the reference samples denote contrast.
Conversely, the participants’responses that the target samples were
lighter than the reference samples represent assimilation. We com
pared the participants’responses of the expectation eﬀect for diﬀerent
combinations of prediction error, uncertainty, and external noise.
4.6

Experimental results
Figure 11 shows the averaged responses of the participants regarding
the relative weight of each target sample for four combinations of
uncertainty and external noise. A positive value shows how much
heavier the target (smaller object) was than the reference (bigger
object), whereas the negative value shows the opposite. In sum, the
positive value represents contrast, and the negative value represents
assimilation. The horizontal axis denotes the diﬀerences between the
expected weight and the actual weight of each target (see Figure 10),
that is, the extent of prediction errors for each pair.
The result shows that under all combinations of uncertainty and
external noise, the expectation eﬀect began with assimilation and then
shifted to contrast as the prediction error increased. This trend corre
sponds to the simulation results shown in Figure 7. As we hypothe
sized, assimilation occurred in the presence of small prediction errors,
which contradicts the SWI.
We conducted threeway ANOVA with prediction error,uncer
tainty,andexternal noise as independent variables and the response of
the relative weight of the target sample (i.e., expectation eﬀect, E)as
the dependent variable. The main eﬀects of the prediction error
[F524.83, p<0.001, h250.276, 12b 1:0] were signiﬁcant. The
average score of the expectation eﬀect with bigger noise was signiﬁ
cantly smaller than one with smaller noise [F54.65, p50.03,
h250.00576, 12b 5 0.578]. In sum, bigger external noise tended to
have more assimilations than smaller external noise. The main eﬀects
of uncertainty [F50.29, p50.58, h250.00036, 12b 5 0.084] and
interactions were not signiﬁcant.
To investigate the inﬂuences of uncertainty and external noise in
detail, we conducted a twoway repeated measure ANOVA with uncer
tainty and external noise as independent variables and the response of
the relative weight of the target sample as the dependent variable for
each target sample, that is, each prediction error. The results indicate
statistical signiﬁcance at the prediction errors of 180 g for assimilation
and of 780 g for contrast.
With respect to the prediction error of 180 g (assimilation), the
main eﬀects of uncertainty [F52.25, p50.16, h250.016,
12b 5 0.341] and external noise [F50.38, p50.55, h250.0049,
12b 5 0.150] were not signiﬁcant. However, we observed marginally
signiﬁcant interaction between uncertainty and external noise [F52.92,
p50.1, h250.0445 12b 5 0.517]. Figure 7 shows a prominent nega
tive response, namely, assimilation, for small uncertainty and small
noise at a prediction error of 180 g. We also observed a similar trend
at 280 g (Figure 7). We compared the responses for the prediction
error of 180 g for diﬀerent conditions of uncertainty and external noise
using Bonferronicorrected paired comparisons. We found that the
negative response (assimilation) for small uncertainty and big noise was
signiﬁcantly bigger than the response for big uncertainty and small
noise. For the prediction error of 780 g (contrast), we found signiﬁcant
main eﬀects for both uncertainty [F55.18, p50.039, h250.076,
12b 5 0.681] and external noise [F57.88, p50.014, h250.116,
12b 5 0.626]. The interaction between uncertainty and external noise
is not signiﬁcant [F50.25, p50.62, h250.0036 12b 5 0.142]. We
observed that smaller uncertainty involves a signiﬁcantly bigger expec
tation eﬀect, namely, contrast, than bigger uncertainty. For external
2
1.5
1
0.5
0
0.5
1
1.5
2
180 280 380 480 580 680 780 880 980 1080
Average score of weight
expectation effec t
Prediction error (g)
small u ncertai nty & small no ise
big u ncertainty & smal l noise
small u ncertai nty & big noise
big u ncertainty & big no ise
FIGURE 11 Expectation eﬀects in SWI as functions of expectation error for each condition of uncertainty and external noise. Each bar
represents the average responses of the relative weight of each target. The error bars denote standard errors
YANAGISAWA

437
noise, smaller noise involves signiﬁcantly bigger contrast than bigger
noise.
5

DISCUSSION
Both results of the computer simulation (Figure 7) and the experiment
using the SWI (Figure 11) showed that prediction error dominantly
aﬀected the extent of the expectation eﬀect and worked as a factor of
either the assimilation or the contrast condition. The pattern of expec
tation eﬀect shifted from assimilation to contrast as the prediction
error increased. This correspondence between the simulation and
experiment supports our hypothesis, namely, that the prediction error
increases the likelihood repulsive inﬂuence against prior attractive
inﬂuence during Bayesian estimation (decoding). As we hypothesized,
assimilation occurred in the presence of small errors. In sum, human
beings perceive bigger object as heavier than a smaller object if the
prediction error is small. This result contradicts the idea in the conven
tional SWI. The law of expectation eﬀect using the model revealed
these undiscovered perceptual phenomena.
We discuss the signiﬁcance of the psychological phenomenon
from an ecological viewpoint. Contrast exaggerates expectation discon
ﬁrmation so that human beings pay attention to novel stimuli with sur
prise (Itti & Baldi, 2009) and try to gain information from unexpected
phenomena. This biological function may provide an opportunity to
learn novel information and renew prior knowledge, that is, prior distri
butions. However, due to limitations of cognitive resources, such as
shortterm memory content and energy, human beings cannot pay
attention to each unexpected phenomenon. Assimilation may work as
aﬁlter to select which unexpected phenomena should be paid atten
tion to. In sum, human beings ignore marginal prediction error. This bio
logical function is reasonable in that it saves the energy resources of
the human brain.
The second hypothesis was that the trend in the relationship
between the expectation eﬀect and prediction error depends on uncer
tainty and external noise. The simulation results in Figure 4 show that
uncertainty decreased the extent of the expectation eﬀect and external
noise increased the assimilation due to the decreasing repulsive inﬂu
ence during the Bayesian estimation. Although these eﬀects were
weak in the SWI experiment, the trend supported the simulation result.
The condition of small uncertainty with big external noise involved
prominent assimilation at 180 g prediction error. The result of 780 g
prediction error showed that the extent of contrast with smaller noise
signiﬁcantly exceeded that with bigger noise. Smaller uncertainty
involved a signiﬁcantly bigger contrast than bigger uncertainty.
We can explain these phenomena with our hypothetical model as
follows. Prior distributions of low variation, namely, certain predictions,
attracted a Bayesian estimate against the likelihood function of noisy
stimuli when the prediction error and likelihood asymmetry are small.
The repulsive inﬂuence decreased as uncertainty and external noise
increased. The contrast weakened with big uncertainty and big noise.
Human beings rely on their prior distributions when the external stimu
lus is noisy. Certain prior predictions may increase this dependency,
and thus, the extent of assimilation becomes prominent. On the other
hand, human beings should pay attention to big prediction errors of
certain predictions and clear external stimuli. Therefore, contrast
increased with small uncertainty (certain prediction) and small external
noise (clear stimulus).
6

CONCLUSION
We proposed a simulation model of expectation eﬀect based on neural
coding principles. The simulation results using the model revealed that
the condition of expectation eﬀect is altered depending on the three
factors, that is, prediction error, uncertainty, and external noise. We
observed that the trend of the SWI experiment supported the simula
tion result. The simulation result renewed the conventional SWI thesis
based on the proposed expectation eﬀect model.
Discussions from ecological and neural viewpoints suggested that
the simulation result was reasonable. Although we need to validate the
model with other perceptual phenomena in future study, the proposed
modelhasapotentialtoworkasafundamentaloftheexpectation
eﬀect.
The proposed model guides researchers and practitioners to select
parameter combinations that diﬀerentiate sensory illusions caused by
prior expectation. This provides a guideline to design eﬃcient experi
ment that maximize response variance with limited trials of partici
pants. Furthermore, the model helps to explain perceptual bias caused
by prior expectation in the participants’responses in sensory
experiment.
REFERENCES
Barlow, H. B. (1961). Possible principles underlying the transformation of
sensory messages. In Sensory Communication, (W. Rosenblith, ed.) pp.
217–234. MIT Press: Cambridge, MA.
Brayanov, J. B., & Smith, M. A. (2010). Bayesian and “antiBayesian”
biases in sensory integration for action and perception in the size–
weight illusion. Journal of Neurophysiology,103, 1518–1531.
Deliza, R., & Macﬁe, H. J. H. (1996). The generation of sensory expecta
tion by external cues and its eﬀect on sensory perception and
hedonic ratings: A review. Journal of Sensory Studies,11, 103–128.
Demir, E., Desmet, P., & Hekkert, P. (2009). Appraisal patterns of emo
tions in humanproduct interaction. International Journal of Design,3,
41–51.
Ernst, M. O., & Banks, M. S. (2002). Humans integrate visual and haptic
information in a statistically optimal fashion. Nature,415, 429–433.
Flanagan, J. R., Bittner, J. P., & Johansson, R. S. (2008). Experience can
change distinct sizeweight priors engaged in lifting objects and judg
ing their weights. Current Biology,18, 1742–1747.
Geers, A. L., & Lassiter, G. D. (1999). Aﬀective expectations and informa
tion gain: Evidence for assimilation and contrast eﬀects in aﬀective
experience. Journal of Experimental Social Psychology,35, 394–413.
Itti, L., & Baldi, P. (2009). Bayesian surprise attracts human attention.
Vision Research,49, 1295–1306.
K€
ording, K. P., & Wolpert, D. M. (2004). Bayesian integration in sensori
motor learning. Nature,427, 244–247.
Krippendorﬀ, K. 2005. The semantic turn: A new foundation for design.
Boca Raton, London, New York: Taylor & Francis, CRC Press.
438

YANAGISAWA
Ludden,G.D.S.,Schiﬀerstein, H. N. J., & Hekkert, P. (2012).
Beyond surprise: A longitudinal study on the experience of
visualtactual incongruities in products. International Journal of
Design,6,1–10.
Oliver, R. L. (1977). Eﬀect of expectation and disconﬁrmation on postex
posure product evaluations: An alternative interpretation. Journal of
Applied Psychology,62, 480–486.
Oliver, R. L. (1980). A cognitive model of the antecedents and conse
quences of satisfaction decisions. Journal of Marketing Research,7,
460–469.
Quian Quiroga, R., & Panzeri, S. (2009). Extracting information from neu
ronal populations: Information theory and decoding approaches.
Nature Review Neuroscience,10, 173–185.
Schiﬀerstein, H. N. J. (2001). Eﬀects of product beliefs on product per
ception and liking. In Food, people and societyed (Lynn J. Frewer, Einar
Risvik, Hendrik Schiﬀerstein, eds.), pp. 73–96. Springer: Berlin
Heidelberg.
Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of
prediction and reward. Science,275, 1593–1599.
Wei, X. X., & Stocker, A. (2012). Bayesian Inference with eﬃcient neural
population codes. In Artiﬁcial neural networks and machine learning –
ICANN 2012 (A. P. Villa, W. Duch, P.
Erdi, F. Masulli, & G. Palm eds.)
(pp. 523–530). Springer: Switzerland.
Wilson, T. D., Lisle, D. J., Kraft, D., & Wetzel, C. G. (1989). Preferences as
expectationdriven inferences: Eﬀects of aﬀective expectations on aﬀec
tive experience. Journal of Personality and Social Psychology,56,519–530.
Yanagisawa, H., & Takatsuji, K. (2015a). Eﬀects of visual expectation on
perceived tactile perception: An evaluation method of surface texture
with expectation eﬀect. International Journal of Design,9,39–51.
Yanagisawa, H., & Takatsuji, K. (2015b). Expectation eﬀect of perceptual
experience in sensory modality transitions: Modeling with informa
tion theory. Journal of Intelligent Manufacturing,1–10. doi: 10.1007/
s1084501510967.
SUPPORTING INFORMATION
Additional Supporting Information may be found in the online ver
sion of this article.
YANAGISAWA

439