A computational model of perceptual expectation effect based on neural coding principles

Abstract

Abstract: Prior expectation affects posterior perceptual experience. This contextual bias is called expectation effect. Previous studies have observed two different patterns of expectation effect: contrast and assimilation. Contrast magnifies the perceived incongruity, and assimilation diminishes the incongruity. This study proposes a computational model that explains the conditions of contrast and assimilation based on neural coding principles. This model proposed that prediction error, uncertainty, and external noise affected the expectation effect. Computer simulations with the model show that the pattern of expectation effect shifted from assimilation to contrast as the prediction error increased, uncertainty decreased the extent of the expectation effect, and external noise increased the assimilation. We conducted an experiment on the size–weight illusion (SWI) as a case of the cross-modal expectation effect and discussed correspondence with the simulation. We discovered conditions where the participants perceived bigger object to be heavier than smaller one, which contradicts to conventional SWI.
Practical applications: Expectation effect in sensory perception represents a perceptual bias caused by prior expectation, such as illusions and cross-modality. The computational model proposed in this study guides researchers and practitioners who investigate this bias in sensory studies to set a hypothesis with appropriate experimental factors. For example, the model suggests that prediction error can be used as a main factor to identify a condition at which assimilation switches over to contrast. The model provided how expectation uncertainty and noise of stimulus affect the switchover point of prediction error and extent of expectation effect. Uncertainty, which may differ from person to person, can be used as a factor to explain personal differences in the extent of expectation effect.

Full text

ORIGINAL ARTICLE
A computational model of perceptual expectation effect based
on neural coding principles
Hideyoshi Yanagisawa
Department of Mechanical Engineering, The
University of Tokyo, Tokyo, Japan
Correspondence
H. Yanagisawa, Department of Mechanical
Engineering, The University of Tokyo, 7-3-
1, Hongo, Bunkyo, Tokyo, Japan.
Email: hide@mail.design.t.u-tokyo.ac.jp
Funding Information
This work was supported by KAKEN (No.
23760127).
Abstract
Prior expectation affects posterior perceptual experience. This contextual bias is called expectation
effect. Previous studies have observed two different patterns of expectation effect: contrast and
assimilation. Contrast magnifies the perceived incongruity, and assimilation diminishes the incon-
gruity. This study proposes a computational model that explains the conditions of contrast and
assimilation based on neural coding principles. This model proposed that prediction error, uncer-
tainty, and external noise affected the expectation effect. Computer simulations with the model
show that the pattern of expectation effect shifted from assimilation to contrast as the prediction
error increased, uncertainty decreased the extent of the expectation effect, and external noise
increased the assimilation. We conducted an experiment on the size–weight illusion (SWI) as a
case of the cross-modal expectation effect and discussed correspondence with the simulation. We
discovered conditions where the participants perceived bigger object to be heavier than smaller
one, which contradicts to conventional SWI.
Practical applications
Expectation effect in sensory perception represents a perceptual bias caused by prior expectation,
such as illusions and cross-modality. The computational model proposed in this study guides
researchers and practitioners who investigate this bias in sensory studies to set a hypothesis with
appropriate experimental factors. For example, the model suggests that prediction error can be
used as a main factor to identify a condition at which assimilation switches over to contrast. The
model provided how expectation uncertainty and noise of stimulus affect the switchover point of
prediction error and extent of expectation effect. Uncertainty, which may differ from person to
person, can be used as a factor to explain personal differences in the extent of expectation effect.
1
|
INTRODUCTION
Expectation congruity works as an appraisal component that evokes
emotions, such as surprise (Ludden, Schifferstein, & Hekkert, 2012),
satisfaction (Oliver, 1977, 1980), and disappointment (Demir, Desmet,
& Hekkert, 2009). For example, a user expects the usability from a
product appearance (“This looks easy to use.”), and determines the
actual usability (“This is really easy to use.”). A positive expectation dis-
crepancy (“This is easier to use than expected!”) provides a satisfaction.
In contrast, a negative expectation discrepancy (“This is harder to use
than expected.”) disappoints the user. Furthermore, prior expectations
affect and change posterior perception and experience. Researchers
from a broad range of fields have observed this psychological bias, the
so-called expectation effect, with regard to different cognitive proc-
esses, such as desire for rewards (Schultz, Dayan, & Montague, 1997),
emotions (Geers & Lassiter, 1999; Wilson, Lisle, Kraft, & Wetzel, 1989),
and sensory perceptions (Deliza & MacFie, 1996; Schifferstein, 2001).
The expectation effect changes the disconfirmation between expecta-
tion and experience. Thus, the expectation effect is an essential factor
to ensure the satisfactory design of products and services.
In a time sequence of UX of a product and service, users shift
from one sensory state to another in cyclic interactions involving
action, sensation, and perception (Krippendorff, 2005). Figure 1 illus-
trates an example of such state transitions while using a camera. A user
perceives the appearance by looking (vision), the tactile feeling by
grasping (touch), and shutter sound by popping a shutter (audition). We
expect that users would predict subsequent states between these tran-
sitions of state (e.g., we expect a meal to taste a certain way based on
how it looks, the weight of a product before lifting it, the usability of a
mouse by looking at it). This prior prediction affects posterior
430
|
V
C2016 Wiley Periodicals, Inc. wileyonlinelibrary.com/journal/joss J. Sens. Stud. 2016; 31: 430-439
Received: 28 April 2016
|
Revised: 25 June 2016
|
Accepted: 16 August 2016
DOI 10.1111/joss.12233

perception, that is, the expectation effect. For example, visual expecta-
tion changes tactile perceptions of surface texture (Yanagisawa &
Takatsuji, 2015a).
We can explain a kind of perceptual illusion using the expectation
effect. For example, people perceive a smaller object as heavier than a
larger one although the weight of both objects is identical (Flanagan,
Bittner, & Johansson, 2008). This well-known size–weight illusion
(SWI) can be explained as a visual expectation effect. People expect a
larger object to be heavier than a smaller one. Prior visual expectation
of the objects’weights magnifies the perception of difference between
the expected and actual weights. Although many experimental findings
exist on the expectation effect in different disciplines, the general
mechanism on why and how the effect occurs is not yet clearly eluci-
dated. A mathematical model of the expectation effect based on a fun-
damental mechanism enables us to estimate user perception of
product and service. Yanagisawa and Takatsuji (2015b) proposed a
mathematical model of the expectation effect using information theory.
They modeled prior expectation as a subjective probability distribution
and hypothesized that Shannon’s entropy of the distributions repre-
senting uncertainty of prior expectation determines the occurrence of
the expectation effect. An experimental result of the visual expectation
effect of tactile texture showed that the lower entropy invoked expec-
tation effect in a higher rate than another. This finding suggests that
uncertainty is a factor of occurrence of expectation effect.
On the other hand, two different patterns of expectation effect,
contrast and assimilation, were observed (Deliza & MacFie, 1996). Con-
trast is a bias that magnifies the difference between prior expectation
and posterior experience. Assimilation is a bias that diminishes expecta-
tion incongruence. It is important to understand whether the expecta-
tion effect is contrasting or assimilating, because they exaggerate or
diminish the perception of expectation disconfirmation as a factor of
satisfaction. However, the mechanisms and conditions governing the
contrasting and assimilating patterns are not yet clearly elucidated. In
this study, we propose a simulation model of the expectation effect
that explains the conditions of contrast and assimilation by applying
neural coding principles, such as Bayesian decoding and the efficient
encoding principles. Based on the proposed model, we conduct com-
puter simulations of the expectation effect and obtain an accurate
hypothesis of the conditions of assimilation and contrast. Finally, we
apply the obtained hypothesis to externalize a perceptual law behind
SWI as an expectation effect from a result of an experiment with the
participants.
2
|
COMPUTATIONAL MODEL OF
EXPECTATION EFFECT
2.1
|
Bayesian coding in perception involving prior
expectation
We define perception as an estimation of external physical property,
such as the weight of an object. Sensory stimulus from the external
physical world, such as pressure applied to a hand, is transformed to
patterns of neural signals. We call the neural representation of an
external physical variable encoding. Based on the pattern of neural sig-
nals, our brain estimates the physical variable. We call this estimation
process decoding. We assume that sensory stimuli are encoded as cer-
tain firing rates of neural populations. This type of neural coding is
called rate coding. Another type of encoding is a temporal coding
hypothesis where the precise timing of spikes adds information. How-
ever, most studies of temporal coding have concentrated on single
brain areas and only a few have reported how timing codes are trans-
formed across different stages of a sensory pathway. Evidence for tem-
poral coding is less established in higher cognitive brain area (Quian
Quiroga & Panzeri, 2009). Thus, we assume rate coding for further dis-
cussion. Based on the firing rate distributions from a sensory stimulus,
R, our brain forms the likelihood function,PRjuðÞ, of a physical variable,
u. On the other hand, a physical property has certain frequency distri-
butions in the world. Human beings learn these frequency distributions
throughout their life. Based on these learnt distributions, human beings
predict a physical variable, u, before experiencing sensory stimulus. For
example, in the SWI, people predict the weight of an object by looking
at it before actually lifting it up. The predicted physical variable should
follow certain probability distributions. We define this distribution as
prior,PðuÞ. Recent studies in neuroscience showed that the estimation
of a physical variable, that is, decoding, follows the Bayesian estimator
(Brayanov & Smith, 2010; Ernst & Banks, 2002). Based on Bayes’theo-
rem, our brain estimates the distributions of perceptions or posterior,
PujR
ðÞ
, using prior and likelihood.
PujRðÞ5PRju
ðÞ
Pu
ðÞ
XXPRjuðÞPuðÞ (1)
Since the denominator of the right-hand side of Equation 1isa
constant for normalization, the posterior is proportional to the product
of prior and likelihood.
PujRðÞ/PRjuðÞPuðÞ (2)
If the distributions of prior and likelihood follow the Gaussian dis-
tributions and are independent from each other, we can calculate the
optimal estimate of posterior ^
uby the following equation (K€
ording &
Wolpert, 2004).
^
u5V½PðuÞE½PðRjuÞ1V½PðRjuÞE½PðuÞ
V½PðuÞ1V½PðRjuÞ (3)
where EðpÞdenotes mean of distribution p,andVðpÞdenotes a var-
iance of p. The numerator of the right side in Equation 3represents
sum of prior mean EðPðuÞÞ weighted with likelihood variation VðPðRjuÞÞ
and likelihood mean EðPðRjuÞÞ weighted with prior variation VðPðuÞÞ.
appearance
vision
look
tactile feel
touch
grasp
shutter sound
audition
press button
FIGURE 1 Sensory transitions of product use (a case of camera)
YANAGISAWA
|
431

The denominator of the right side in Equation 3 is a sum of variations
for normalization. The estimate is in the middle of prior mean and likeli-
hood mean when the prior variation and likelihood variation are same.
The estimate comes close to the prior mean when prior variation is big-
ger than likelihood variation. In an opposite manner, the estimate
comes close to likelihood when likelihood variation is bigger than prior
variation. Equation 3 indicates that the posterior estimate, ^
u,isalways
between prior mean and likelihood peak. Figure 2 illustrates how pos-
terior estimate comes close to a peak of prior from a peak of the likeli-
hood estimate of the sensory stimulus. We call the effect attractive
influence of prior.
2.2
|
Expectation effect and prediction error
Now, we can write the expectation effect, E,andprediction error,d
E5
^
u2u
lik (4)
d5ulik2E½PðuÞ (5)
where ulik denotes a likelihood peak. Using Eand d, we can formalize
the condition of contrast as jd1Ej>jdjand the condition of assimilation
as jd1Ej<jdj.jd1Ejrepresents the extent of perceived discrepancy
with expectation effect. Figure 3 illustrates contrast and assimilation.
Contrast is a type of expectation effect that exaggerates perceived dis-
crepancy, whereas assimilation is another type of expectation effect
that diminishes perceived discrepancy. Assume that the horizontal axis
of Figure 3 is weight, contrast makes people feel heavier than actual
weight, whereas assimilation makes people feel lighter than the actual
weight.
2.3
|
Efficient coding causes contrast
The attractive influence of prior alone involves assimilation as an
expectation effect. The question then arises: How does contrast occur?
We applied a neural encoding framework based on the efficient coding
principle (Wei & Stocker, 2012). According to the encoding framework,
the Bayesian estimate shifts away from the peaks of the prior distribu-
tion. This phenomenon corresponds to the contrast pattern of the
expectation effect. Efficient coding hypothesis proposes that the tuning
characteristics of a neural population are adapted to the prior distribu-
tion of a sensory variable such that the neural population optimally rep-
resents the sensory variable (Barlow, 1961). This optimal neural coding
makes the total expected spike count of each neuron in the population
similar. To mathematically represent this code, the neural encoding
framework assumed an equivalent homogeneous space with units ~
uin
which the stimulus prior distribution is uniform and mapped the physi-
cal space with units uto the homogeneous space using a mapping
function F. The framework used cumulative of the prior Pas the map-
ping function as follows.
~
u5FuðÞ5ðu
21
PxðÞdx (6)
To map from ~
uto u, we can use the inverse function F21ð~
uÞ.Fig-
ure 4 illustrates the tuning curves of efficient neural populations and
the mapping to/from the homogeneous space for a prior of normal dis-
tributions pðuÞ. In the homogeneous space, we assume similar turning
curves for each neural population. The turning curves are assigned at
equally spaced intervals so that the expected spike count of each pop-
ulation is the same for a uniform prior. Then, we map the tuning curve
of each neural population to the physical space using F21.Figure4
shows that the tuning curves in the physical space are dense around
prior expectation (high prior density) and sparse toward away from the
expectation (on the side of low prior density). In summary, more neural
populations are allocated to sense a range of physical values that are
more likely to occur than others. This coding strategy maximizes
expected information amount for a prior. Due to the mapping using F,
the shape of tuning curve in the physical space gradually becomes
asymmetrical towards the side of lower prior density. This is attributable
to the shape of the mapping function F. The mapping function shapes S-
curve that closes to a linear around the prior expectation, but becomes
nonlinear toward away from prior mean. We assume that the likelihood
shape follows the shape of the tuning curves in the physical space.
The likelihood shape is constrained in physical space by the prior
distribution. The likelihood distributions in physical space transformed
from symmetric distributions, such as Gaussian, in homogeneous space
FIGURE 2 Attractive influence of prior in Bayesian estimate
FIGURE 3 Illustration of contrast and assimilation
432
|
YANAGISAWA

shows heavier tails on the side of lower prior density. In sum, the effi-
cient encoding typically leads to an asymmetrical likelihood function
whose mean value is away from the peak of prior. As shown in Equa-
tion 2, the Bayesian estimate is determined by a combination of prior
and shifted likelihood means, 
ulik, and it shifts away from the prior peak
(Figure 5).
We apply this efficient encoding to explain contrast in our model.
The Bayesian estimate (perceived value), ^
u, shifts from a peak of the
asymmetric likelihood function away from a peak of prior. We call the
perceptual shift repulsion influence.
2.4
|
Perceptual model with prior expectation
From the above discussion using Bayesian decoding and efficient cod-
ing principle, we can summarize our hypothetical model of perception
as shown in Figure 6. Based on the efficient encoding principle, prior
changes the shape of the likelihood function asymmetry while encod-
ing the sensory stimulus of the physical variable, u,asafiring rate of
the neuron population R. The Bayesian decoder integrates the prior
distribution, PðuÞ, and asymmetric likelihood function, PðRjuÞ, and forms
posterior distributions. As a result, we perceive a peak of the posterior,
^
u, as an estimate of the physical variable, that is, perception.
2.5
|
Factors of expectation effect: prediction error,
uncertainty, and external noise
Based on the efficient coding, the likelihood shape shows heavier tails
on the side of lower prior density, and the likelihood means shifts away
from the prior peak. The extent of likelihood asymmetry and the likeli-
hood mean shift increases towards away from a prior peak. Thus, we
can predict that the repulsion bias increases towards away from a prior
peak. (Wei & Stocker, 2012) mathematically derived the prediction.
Based on their mathematic derivation, we can approximate the repul-
sion influence at prediction error, bðuÞ, in the following differential
equation.
bðuÞCd
du
1
PðuÞ2
! (7)
where Cis a positive constant, and PðuÞis prior distributions. If the
prior is Gaussian with zero mean, the gradient of 1=PðjdjÞ2is positive
for any prediction error dand increases away from mean of prior. Thus,
Equation 7 indicates that the repulsion bias increases as the absolute
prediction error jdjincreases. This is because the extent of asymmetry
of likelihood increases away from the peak of prior. Repulsion influence
involves contrast. Thus, the prediction error is a key factor that decides
the condition of the expectation effect.
We assume two more factors of the expectation effect: external
noise and uncertainty. The shape of the likelihood function is affected
by the noise of the external stimulus. An external noise modifies the
FIGURE 4 How efficient coding makes likelihood in physical space asymmetry
lik
θ
lik
θ
ˆ
θ
δ
ε
FIGURE 5 Repulsion influence due to asymmetry of likelihood
function
YANAGISAWA
|
433

shape of the likelihood function by convolving it with noise
distributions.
ðfgÞðuÞ5ðfðxÞgðu2xÞdx (8)
Where fis the asymmetric likelihood function, and gis a symmet-
ric external noise distribution. We assume that the external noise distri-
bution follows the Gaussian distribution.
gðuÞNðulik;r2noise Þ(9)
Symmetric external noise distributions do not change the mean of
likelihood, but they increase its overall width. Thus, the attractive influ-
ence of prior relatively increases, and the Bayesian estimate, ^
u,shifts
toward the peak of prior. If the attractive influence of prior exceeds
the repulsion influence of the asymmetric likelihood, the expectation
effect may change into assimilation from contrast.
Variations of prior distributions are indicators of prediction uncer-
tainty. As shown in formula (3), the variation in prior impacts the attrac-
tive influence. In the Bayesian estimation, a small variation in prior
denotes small uncertainty and involves a strong attractive influence.
Conversely, a big variation in prior means uncertain prediction and
involves weak attractive influence.
From the above discussions, we can assume that the condition and
extent of expectation effect alters with the three factors, namely, pre-
diction error, d, uncertainty, V½PðuÞ, and external noise, r2noise.
3
|
EFFECT OF PREDICTION ERROR,
EXTERNAL NOISE, AND UNCERTAINTY ON
EXPECTATION EFFECT
3.1
|
Method
To investigate the effects of the three factors, that is, prediction error,
uncertainty, and external noise on the expectation effect, we con-
ducted a computer simulation using abovementioned model. We used
Gaussian distributions for prior, homogeneous likelihood, and posterior.
The following conditions were selected as simulation parameters: pre-
diction error, d, within [0, 100]; uncertainty, V½PðuÞ, within [50, 200];
and external noise, r2noise , within [5, 50]. The standard deviation of the
homogeneous likelihood is set as 0.04. We calculated the expectation
effect, E, using the simulation model for all combinations of the above-
mentioned conditions for the three factors. We use MATLAB® to con-
duct the simulations. The code is provided as a supporting information.
3.2
|
Result and discussion
Figure 7 shows an example of the simulation result of the expectation
effect as a function of the prediction error. Each line represents a con-
dition of uncertainty (small: 150, big: 180) and external noise (small: 30,
big: 40). A positive value represents contrast, and a negative value,
assimilation. Figure 7 reveals three findings.
The expectation effect functions as an assimilating effect when
the prediction error is small. As the prediction error increases, the
expectation effect increases and changes to the contrasting condition
((1) in Figure 7). Around the peak of prior, where the prediction error is
small, the shape of the likelihood function was close to symmetry, the
repulsion influence was small, and the attractive influence of prior is
dominant. Thus, assimilation occurs. As the prediction error increases,
the extent of the likelihood asymmetry increases, and the repulsion
influence increases. Thus, the expectation effect shifts to the contrast
condition.
ˆ
θ
P( )
θ
P( | )R
θ
θ
P( )|R
θ
FIGURE 6 A model of perception involving prior expectation
0 20 40 60 80 100 120 140 160 180 200
Prediction error
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
Expectation effect
uncertainty=150, noise=30
uncertainty=150, noise=40
uncertainty=180, noise=30
uncertainty=180, noise=40
(2)
(2)
(2)
(2)
(3)
(1)
FIGURE 7 Expectation effect as a function of expectation error
for different conditions of expectation uncertainty and external
noise
434
|
YANAGISAWA

The extent of the expectation effect, jEj, is bigger when uncer-
tainty is lower for both assimilation and contrast ((2) in Figure 7). With
respect to assimilation, the attractive influence of prior increases in the
Bayesian estimation as the variation of prior (uncertainty) decreases.
On the other hand, the repulsive influence increases from a certain
value of prediction error as the variation of prior decreases. In sum, cer-
tain predictions involve a sharp expectation effect regardless of the
condition (contrast or assimilation).
The prediction error at which assimilation changes to contrast
increases as the external noise increases ((3) in Figure 7). External noise
weakens the repulsive influence. In the Bayes estimation, the attractive
influence of prior becomes stronger than the repulsive influence of
likelihood. Thus, the area of assimilation in the prediction error
increases when the external noise exceeds prediction error and uncer-
tainty. We observed the abovementioned trends for all possible combi-
nations of conditions for uncertainty and external noise.
Figure 8 shows normalized prediction error when assimilation
switches over to contrast as a function of uncertainty for different
external noise condition. The prediction error, the vertical axis, is nor-
malized between zero and one. In sum, the vertical axis represents the
ratio of assimilation area in the range of prediction error. Zero in the
vertical axis represents a case where only contrast occurs, whereas one
represents a case where only assimilation occurs. With respect to small
external noise such as 5, only contrast occurs for all uncertainty condi-
tions. This is because the repulsion influence due to efficient coding is
dominant with small external noise. Except the condition of noise 5,
only assimilation occurs when uncertainty is small, such as 50 for all
noise conditions. This is because attractive influence with small uncer-
tainty is dominant in Bayesian decoding (as indicated in formula [3]).
The assimilation area decreases as uncertainty increases from certain
uncertainty condition. This is because the attractive influence of prior
decreases against repulsion influence caused by efficient coding.
Uncertainty at the area where assimilation starts to decrease increases
as noise increases because attractive influence becomes relatively
stronger than noisier likelihood in Bayesian decoding. The above results
suggest that the switchover prediction error, assimilation area, depends
on the balance between attractive influence of prior uncertainty in
Bayesian decoding and repulsion influence due to asymmetric likeli-
hood in efficient coding.
Figure 9 shows contours of prediction errors when assimilation
changes to contrast for all combinations of uncertainty and external
noise. The prediction error, the z-axis, is normalized between zero and
one. Zero in the contour represents a case where only contrast occurs,
whereas one represents a case where only assimilation occurs. Figure 9
shows that the area where uncertainty is high and external noise is
small denotes cases where only contrast occurs. In this area, the repul-
sive influence of asymmetry likelihood is dominant compared to the
attractive influence of uncertain prediction. On the other hand, the
area with low uncertainty and big external noise shows only assimila-
tion. The attractive influence of prior is dominant for certain predic-
tions compared to the repulsive influence, which is weakened by the
external noise.
4
|
EXPERIMENT WITH SIZE–WEIGHT
ILLUSION
4.1
|
Method
As discussed in the Introduction Section, SWI is a significant case of
the cross modal expectation effect, where prior visual expectation
biases weight perception. To investigate the actual effect of the
three factors (i.e., prediction error, uncertainty, and external noise)
on SWI, we conducted a sensory experiment with the participants.
We manipulated the three factors of the expectation effect. For
each condition of the factors, we obtained responses from the par-
ticipants with respect to perceptions of weight, and evaluated the
extent of weight illusions as expectation effects. We presented the
participants with pairs of cubic metal objects. The objects in each
pair had identical weights but different sizes. We asked the partici-
pants to compare the weights and obtained their responses for the
difference in weights. We used the perceptual difference of weight
as the extent of the expectation effect.
Uncertainty
Switchover from assimilation to contrast
5
15
25
35
45
FIGURE 8 Expectation errors when assimilation switch over to
contrast as a function of uncertainty for different external noise
10 20 30 40 50
50
100
150
0.2
0.4
0.6
0.8
1
External noise
Uncertainty
Normalized expectation error
0
FIGURE 9 Expectation errors when assimilation shifts to contrast
for different conditions of uncertainty and external noise. Data is
provided as a supporting information
YANAGISAWA
|
435

According to conventional SWI, human beings perceive that a
smaller object is heavier than a larger object, although both objects
may have the same weight. This illusion can be viewed as a contrast of
the expectation effect, where the perception of difference between
the weight predicted by the object’s size and its actual weight, the pre-
diction error, is exaggerated. However, our simulation result in Figure 4
showed that assimilation, an opposite effect to contrast, occurs when
the prediction error is less than a certain value. To validate the simula-
tion results in SWI, we controlled the prediction errors in the experi-
ment. We prepared pairs of objects, called target and reference.
Participants evaluated the weight of the target by comparing it with
that of the reference. Figure 10 illustrates the weight and size of tar-
gets (t1–t4) and references (r1–r4). All samples have identical surface
material. References are solid objects made by identical material with
equal density. All targets have identical size in appearance. The visual
expectation of weight for all targets are marked “x”in Figure 10 from
the identical appearance (size and surface material). However, we
adjusted each target weight similar to each compared reference. The
difference between the visually expected weight and the actual weight
works as the prediction error. In this setting, the target was always big-
ger than the reference for each pair. Therefore, when target is lighter
than the reference denotes contrast, whereas when target is heavier
than the reference denotes assimilation.
To control uncertainty of visual predictions, we used a fogged glass
so that transparency between the participants and target object was
manipulated. We assumed that fuzzy visual images of the targets
would increase the uncertainties of size and weight predictions. To
control the external noise of somatosensory sensation while lifting an
object, we asked the participants to add a weight to the wrist he/she
used for lifting the samples. According to the Weber–Fechner Law, the
difference threshold increases as the intensity of stimulus increases.
We assumed that the additional weight of each participant’swrist
serves as the external noise of weight perception due to the increasing
difference threshold.
4.2
|
Material
We prepared ten solid cubic shapes made of duralumin (A2017) as a
set of reference samples. The weights of the reference samples ranged
from 350 g to 1250 g. Their sides were 50–76.5 mm long. For the tar-
get samples, we prepared hollow cubes made of duralumin (A2017),
with sides measuring 80 mm. We inserted additional weights into the
hollow cubes so that the weights of both samples were identical for
each pair. We attached a wire to the top of each target and reference
sample and hung them from a steel framework to straighten the wires
without tension. We placed a ring in the middle of each wire to enable
the participants to lift the samples using their index fingers.
4.3
|
Participants
Fifteen (twelve male and three female) volunteers aged 21 to 24 years
served as experiment evaluators. They were undergraduate or graduate
students studying mechanical engineering at the University of Tokyo.
All participants were physically healthy.
4.4
|
Procedure
The participants were invited individually into the isolated test room.
Each participant was seated on a chair in front of the framework, which
was set on a table. After obtaining informed consent, the participants
received written instructions for the procedure. Before starting the
comparison of the pairs, we asked the participants to lift up the ten ref-
erence samples with their index fingers using the wired ring to perceive
the density of the duralumin. After the learning session, the partici-
pants compared the weights of the target and reference samples under
four different combinations of external noise and uncertainty (Table 1).
To simulate the condition of big uncertainty (B and D in Table 1), we
placed a fogged glass between the target and the participant so that
the visual image of the target was fuzzy. For the big external noise con-
dition (C and D in Table 1), a participant added a weight to the wrist
he/she used for lifting the samples. For each condition, we randomly
presented each pair of the 10 pairs of the target and reference samples
with identical weights. The presentation order differed between partici-
pants. We asked participants to alternately lift the target sample and
reference sample with the index finger of the dominant arm using the
wired ring. After they had lifted both samples, we asked them to rank
the target weight as “very much heavier,”“heavier,”“kind of heavier,”
“almost the same,”“kind of lighter,”“lighter,”and “very much lighter”in
comparison to the reference sample in that pair. We repeated the
paired comparisons of sample pair weights for all ten pairs for the four
conditions.
size (cm3)
mass (g)
reference
target
prediction error
expectation
comparison
x
r1
r2
r3
r4
t1
t2
t3
t4
FIGURE 10 Method for manipulating expectation error.
Participants compared the weights of the reference and target. The
difference between the predicted weight (x) and the actual weight
of the target were controlled as prediction errors for each
reference–target pair
TABLE 1 Experimental conditions regarding uncertainty and exter-
nal noise
External noise
Small Big
Uncertainty Small A C
Big B D
436
|
YANAGISAWA

4.5
|
Data analysis
We used the participants’responses regarding the relative weights of
the target samples in comparison to those of the reference samples as
an index of the expectation effect. As explained previously, the physical
weight of the target and reference samples in each pair were identical.
If the response was “almost the same,”we can infer that no expecta-
tion effect was observed. Due to the combination of learned density of
the material and the visually estimated volume, all the target samples
were actually lighter than the expected weight, whereas the reference
samples, which were solid and had congruent expected and actual
weights, were heavier. Thus, the participants’responses that the target
samples were heavier than the reference samples denote contrast.
Conversely, the participants’responses that the target samples were
lighter than the reference samples represent assimilation. We com-
pared the participants’responses of the expectation effect for different
combinations of prediction error, uncertainty, and external noise.
4.6
|
Experimental results
Figure 11 shows the averaged responses of the participants regarding
the relative weight of each target sample for four combinations of
uncertainty and external noise. A positive value shows how much
heavier the target (smaller object) was than the reference (bigger
object), whereas the negative value shows the opposite. In sum, the
positive value represents contrast, and the negative value represents
assimilation. The horizontal axis denotes the differences between the
expected weight and the actual weight of each target (see Figure 10),
that is, the extent of prediction errors for each pair.
The result shows that under all combinations of uncertainty and
external noise, the expectation effect began with assimilation and then
shifted to contrast as the prediction error increased. This trend corre-
sponds to the simulation results shown in Figure 7. As we hypothe-
sized, assimilation occurred in the presence of small prediction errors,
which contradicts the SWI.
We conducted three-way ANOVA with prediction error,uncer-
tainty,andexternal noise as independent variables and the response of
the relative weight of the target sample (i.e., expectation effect, E)as
the dependent variable. The main effects of the prediction error
[F524.83, p<0.001, h250.276, 12b 1:0] were significant. The
average score of the expectation effect with bigger noise was signifi-
cantly smaller than one with smaller noise [F54.65, p50.03,
h250.00576, 12b 5 0.578]. In sum, bigger external noise tended to
have more assimilations than smaller external noise. The main effects
of uncertainty [F50.29, p50.58, h250.00036, 12b 5 0.084] and
interactions were not significant.
To investigate the influences of uncertainty and external noise in
detail, we conducted a two-way repeated measure ANOVA with uncer-
tainty and external noise as independent variables and the response of
the relative weight of the target sample as the dependent variable for
each target sample, that is, each prediction error. The results indicate
statistical significance at the prediction errors of 180 g for assimilation
and of 780 g for contrast.
With respect to the prediction error of 180 g (assimilation), the
main effects of uncertainty [F52.25, p50.16, h250.016,
12b 5 0.341] and external noise [F50.38, p50.55, h250.0049,
12b 5 0.150] were not significant. However, we observed marginally
significant interaction between uncertainty and external noise [F52.92,
p50.1, h250.0445 12b 5 0.517]. Figure 7 shows a prominent nega-
tive response, namely, assimilation, for small uncertainty and small
noise at a prediction error of 180 g. We also observed a similar trend
at 280 g (Figure 7). We compared the responses for the prediction
error of 180 g for different conditions of uncertainty and external noise
using Bonferroni-corrected paired comparisons. We found that the
negative response (assimilation) for small uncertainty and big noise was
significantly bigger than the response for big uncertainty and small
noise. For the prediction error of 780 g (contrast), we found significant
main effects for both uncertainty [F55.18, p50.039, h250.076,
12b 5 0.681] and external noise [F57.88, p50.014, h250.116,
12b 5 0.626]. The interaction between uncertainty and external noise
is not significant [F50.25, p50.62, h250.0036 12b 5 0.142]. We
observed that smaller uncertainty involves a significantly bigger expec-
tation effect, namely, contrast, than bigger uncertainty. For external
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
180 280 380 480 580 680 780 880 980 1080
Average score of weight
expectation effec t
Prediction error (g)
small u ncertai nty & small no ise
big u ncertainty & smal l noise
small u ncertai nty & big noise
big u ncertainty & big no ise
FIGURE 11 Expectation effects in SWI as functions of expectation error for each condition of uncertainty and external noise. Each bar
represents the average responses of the relative weight of each target. The error bars denote standard errors
YANAGISAWA
|
437

noise, smaller noise involves significantly bigger contrast than bigger
noise.
5
|
DISCUSSION
Both results of the computer simulation (Figure 7) and the experiment
using the SWI (Figure 11) showed that prediction error dominantly
affected the extent of the expectation effect and worked as a factor of
either the assimilation or the contrast condition. The pattern of expec-
tation effect shifted from assimilation to contrast as the prediction
error increased. This correspondence between the simulation and
experiment supports our hypothesis, namely, that the prediction error
increases the likelihood repulsive influence against prior attractive
influence during Bayesian estimation (decoding). As we hypothesized,
assimilation occurred in the presence of small errors. In sum, human
beings perceive bigger object as heavier than a smaller object if the
prediction error is small. This result contradicts the idea in the conven-
tional SWI. The law of expectation effect using the model revealed
these undiscovered perceptual phenomena.
We discuss the significance of the psychological phenomenon
from an ecological viewpoint. Contrast exaggerates expectation discon-
firmation so that human beings pay attention to novel stimuli with sur-
prise (Itti & Baldi, 2009) and try to gain information from unexpected
phenomena. This biological function may provide an opportunity to
learn novel information and renew prior knowledge, that is, prior distri-
butions. However, due to limitations of cognitive resources, such as
short-term memory content and energy, human beings cannot pay
attention to each unexpected phenomenon. Assimilation may work as
afilter to select which unexpected phenomena should be paid atten-
tion to. In sum, human beings ignore marginal prediction error. This bio-
logical function is reasonable in that it saves the energy resources of
the human brain.
The second hypothesis was that the trend in the relationship
between the expectation effect and prediction error depends on uncer-
tainty and external noise. The simulation results in Figure 4 show that
uncertainty decreased the extent of the expectation effect and external
noise increased the assimilation due to the decreasing repulsive influ-
ence during the Bayesian estimation. Although these effects were
weak in the SWI experiment, the trend supported the simulation result.
The condition of small uncertainty with big external noise involved
prominent assimilation at 180 g prediction error. The result of 780 g
prediction error showed that the extent of contrast with smaller noise
significantly exceeded that with bigger noise. Smaller uncertainty
involved a significantly bigger contrast than bigger uncertainty.
We can explain these phenomena with our hypothetical model as
follows. Prior distributions of low variation, namely, certain predictions,
attracted a Bayesian estimate against the likelihood function of noisy
stimuli when the prediction error and likelihood asymmetry are small.
The repulsive influence decreased as uncertainty and external noise
increased. The contrast weakened with big uncertainty and big noise.
Human beings rely on their prior distributions when the external stimu-
lus is noisy. Certain prior predictions may increase this dependency,
and thus, the extent of assimilation becomes prominent. On the other
hand, human beings should pay attention to big prediction errors of
certain predictions and clear external stimuli. Therefore, contrast
increased with small uncertainty (certain prediction) and small external
noise (clear stimulus).
6
|
CONCLUSION
We proposed a simulation model of expectation effect based on neural
coding principles. The simulation results using the model revealed that
the condition of expectation effect is altered depending on the three
factors, that is, prediction error, uncertainty, and external noise. We
observed that the trend of the SWI experiment supported the simula-
tion result. The simulation result renewed the conventional SWI thesis
based on the proposed expectation effect model.
Discussions from ecological and neural viewpoints suggested that
the simulation result was reasonable. Although we need to validate the
model with other perceptual phenomena in future study, the proposed
modelhasapotentialtoworkasafundamentaloftheexpectation
effect.
The proposed model guides researchers and practitioners to select
parameter combinations that differentiate sensory illusions caused by
prior expectation. This provides a guideline to design efficient experi-
ment that maximize response variance with limited trials of partici-
pants. Furthermore, the model helps to explain perceptual bias caused
by prior expectation in the participants’responses in sensory
experiment.
REFERENCES
Barlow, H. B. (1961). Possible principles underlying the transformation of
sensory messages. In Sensory Communication, (W. Rosenblith, ed.) pp.
217–234. MIT Press: Cambridge, MA.
Brayanov, J. B., & Smith, M. A. (2010). Bayesian and “anti-Bayesian”
biases in sensory integration for action and perception in the size–
weight illusion. Journal of Neurophysiology,103, 1518–1531.
Deliza, R., & Macfie, H. J. H. (1996). The generation of sensory expecta-
tion by external cues and its effect on sensory perception and
hedonic ratings: A review. Journal of Sensory Studies,11, 103–128.
Demir, E., Desmet, P., & Hekkert, P. (2009). Appraisal patterns of emo-
tions in human-product interaction. International Journal of Design,3,
41–51.
Ernst, M. O., & Banks, M. S. (2002). Humans integrate visual and haptic
information in a statistically optimal fashion. Nature,415, 429–433.
Flanagan, J. R., Bittner, J. P., & Johansson, R. S. (2008). Experience can
change distinct size-weight priors engaged in lifting objects and judg-
ing their weights. Current Biology,18, 1742–1747.
Geers, A. L., & Lassiter, G. D. (1999). Affective expectations and informa-
tion gain: Evidence for assimilation and contrast effects in affective
experience. Journal of Experimental Social Psychology,35, 394–413.
Itti, L., & Baldi, P. (2009). Bayesian surprise attracts human attention.
Vision Research,49, 1295–1306.
K€
ording, K. P., & Wolpert, D. M. (2004). Bayesian integration in sensori-
motor learning. Nature,427, 244–247.
Krippendorff, K. 2005. The semantic turn: A new foundation for design.
Boca Raton, London, New York: Taylor & Francis, CRC Press.
438
|
YANAGISAWA

Ludden,G.D.S.,Schifferstein, H. N. J., & Hekkert, P. (2012).
Beyond surprise: A longitudinal study on the experience of
visual-tactual incongruities in products. International Journal of
Design,6,1–10.
Oliver, R. L. (1977). Effect of expectation and disconfirmation on postex-
posure product evaluations: An alternative interpretation. Journal of
Applied Psychology,62, 480–486.
Oliver, R. L. (1980). A cognitive model of the antecedents and conse-
quences of satisfaction decisions. Journal of Marketing Research,7,
460–469.
Quian Quiroga, R., & Panzeri, S. (2009). Extracting information from neu-
ronal populations: Information theory and decoding approaches.
Nature Review Neuroscience,10, 173–185.
Schifferstein, H. N. J. (2001). Effects of product beliefs on product per-
ception and liking. In Food, people and societyed (Lynn J. Frewer, Einar
Risvik, Hendrik Schifferstein, eds.), pp. 73–96. Springer: Berlin
Heidelberg.
Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of
prediction and reward. Science,275, 1593–1599.
Wei, X. X., & Stocker, A. (2012). Bayesian Inference with efficient neural
population codes. In Artificial neural networks and machine learning –
ICANN 2012 (A. P. Villa, W. Duch, P.

Erdi, F. Masulli, & G. Palm eds.)
(pp. 523–530). Springer: Switzerland.
Wilson, T. D., Lisle, D. J., Kraft, D., & Wetzel, C. G. (1989). Preferences as
expectation-driven inferences: Effects of affective expectations on affec-
tive experience. Journal of Personality and Social Psychology,56,519–530.
Yanagisawa, H., & Takatsuji, K. (2015a). Effects of visual expectation on
perceived tactile perception: An evaluation method of surface texture
with expectation effect. International Journal of Design,9,39–51.
Yanagisawa, H., & Takatsuji, K. (2015b). Expectation effect of perceptual
experience in sensory modality transitions: Modeling with informa-
tion theory. Journal of Intelligent Manufacturing,1–10. doi: 10.1007/
s10845-015-1096-7.
SUPPORTING INFORMATION
Additional Supporting Information may be found in the online ver-
sion of this article.
YANAGISAWA
|
439