ArticlePDF Available

Understanding gloss perception through the lens of art: Combining perception, image analysis, and painting recipes of 17th century painted grapes


Abstract and Figures

To understand the key image features that we use to infer the glossiness of materials, we analyzed the pictorial shortcuts used by 17th century painters to imitate the optical phenomenon of specular reflections when depicting grapes. Gloss perception of painted grapes was determined via a rating experiment. We computed the contrast, blurriness, and coverage of the grapes' highlights in the paintings' images, inspired by Marlow and Anderson (2013). The highlights were manually segmented from the images, and next the features contrast, coverage, and blurriness were semiautomatically quantified using self-defined algorithms. Multiple linear regressions of contrast and blurriness resulted in a predictive model that could explain 69% of the variance in gloss perception. No effect was found for coverage. These findings are in agreement with the instructions to render glossiness of grapes contained in a 17th century painting manual (Beurs, 1692/in press), suggesting that painting practice embeds knowledge about key image features that trigger specific material percepts.
Content may be subject to copyright.
Understanding gloss perception through the lens of art:
Combining perception, image analysis, and painting recipes of
17th century painted grapes
Francesca Di Cicco
Perceptual Intelligence Lab,
Faculty of Industrial Design Engineering,
Delft University of Technology, Delft, the Netherlands $
Maarten W. A. Wijntjes
Perceptual Intelligence Lab,
Faculty of Industrial Design Engineering,
Delft University of Technology, Delft, the Netherlands $
Sylvia C. Pont
Perceptual Intelligence Lab,
Faculty of Industrial Design Engineering,
Delft University of Technology, Delft, the Netherlands $
To understand the key image features that we use to
infer the glossiness of materials, we analyzed the
pictorial shortcuts used by 17th century painters to
imitate the optical phenomenon of specular reflections
when depicting grapes. Gloss perception of painted
grapes was determined via a rating experiment. We
computed the contrast, blurriness, and coverage of the
grapes’ highlights in the paintings’ images, inspired by
Marlow and Anderson (2013). The highlights were
manually segmented from the images, and next the
features contrast, coverage, and blurriness were
semiautomatically quantified using self-defined
algorithms. Multiple linear regressions of contrast and
blurriness resulted in a predictive model that could
explain 69% of the variance in gloss perception. No
effect was found for coverage. These findings are in
agreement with the instructions to render glossiness of
grapes contained in a 17th century painting manual
(Beurs, 1692/in press), suggesting that painting practice
embeds knowledge about key image features that
trigger specific material percepts.
In the last two decades, artists and vision scientists
have boosted joint efforts to mutually profit from each
other’s knowledge (Adelson, 2001; Wade, Ono, &
Lillakas, 2001; Cavanagh, 2005; Pinna, 2007; Conway
& Livingstone, 2007; Cavanagh, Chao, & Wang, 2008;
Melcher & Cavanagh, 2011; Pepperell & Ruschkowski,
2013; DiPaola, Riebe, & Enns, 2013). Via careful
observation of the world, painters have developed
implicit knowledge of the key image features needed to
render different materials, and they have transferred
that to the canvas. Vision scientists can thus use
artworks produced throughout the centuries to extract
these features and understand visual perception.
Naturalistic paintings offer novel learning possibil-
ities to the ongoing research on gloss perception. When
rendering glossy materials, painters did not retrieve the
exact reflectance function of the object they were
depicting. Most likely, they rather portrayed the optical
phenomenon representing its most salient characteris-
tic, namely its specular peak, by applying a bright spot
following the curvature of the surface (Cavanagh et al.,
2008). Specular highlights are, indeed, the most
common monocular cues used by painters to induce a
glossy impression (Wendt, Faul, & Mausfeld, 2008; van
Assen, Wijntjes, & Pont, 2016). Here we assume ‘‘real-
world illumination’’, allowing reliable and accurate
estimations of the gloss (Fleming et al., 2003), from one
primary light source (e.g., a window, as was common in
17th century painting studios), and we ignore illumi-
nation variations.
According to Marlow, Kim, and Anderson (2012),
the key visible features of a highlight that affect gloss
perception are coverage, sharpness, and contrast. They
represent respectively the width, steepness, and height
of the specular peak of the reflectance function.
Citation: Di Cicco, F., Wijntjes, M. W. A., & Pont, S. C. (2019). Understanding gloss perception through the lens of art: Combining
perception, image analysis, and painting recipes of 17th century painted grapes. Journal of Vision,19(3):7, 1–15,
Journal of Vision (2019) 19(3):7, 1–15 1 116 7/ 19 .3.7 ISSN 1534-7362 Copyright 2019 The AuthorsReceived July 20, 2018; published March 21, 2019
This work is licensed under a Creative Commons Attribution 4.0 International License.
Downloaded from on 03/26/2019
Perceptual effects of contrast, coverage, and sharpness
of the specular reflections were already considered
separately in literature (Beck & Prazdny, 1981;
Ferwerda, Pellacini, & Greenberg, 2001; Berzhanskaya,
Swaminathan, Beck, & Mingolla, 2005), and found to
influence gloss perception. Marlow et al. (2012)
investigated their combined effect. Following the study
of Ho, Landy, and Maloney (2008), Marlow et al.
(2012) extended the research on the perceptual inter-
action between bumpiness and glossiness, as a function
of the illumination geometry. By varying the surface
reliefs and illumination directions, but keeping con-
stant the reflectance function, they observed variations
in perceived glossiness. Such variations could be
predicted by modeling the perceived values of contrast,
sharpness, and coverage of the specular reflections. In a
follow-up study, Marlow and Anderson (2013) tested
the efficacy of perceptual ratings of these highlights’
features as predictors, by systematically varying their
contribution in the stimuli. By tuning the extrinsic
factors of surface geometry and illumination, they
could manipulate the highlight features. They demon-
strated that the weighted combination of the perceived
highlight features can generate and predict the glossy
appearance of spherical and planar rendered objects.
Here, we tested if the same holds for glossy materials
depicted in paintings. We tested glossiness perception
of grapes in Dutch 17th century paintings via a rating
experiment, and whether those data can be predicted by
the image features of the highlights. To do this, we
developed a novel method to semiautomatically com-
pute the features directly from segmented images of the
paintings. The reason for preferring paintings from the
17th century over other periods is the accurate
rendering of reality that characterizes this age. We
chose to study grapes because they represent an
accessible starting point, given their more or less
spherical shape. Moreover, grapes offer the advantage
of having been painted often during the 17th century,
allowing us to collect a high number of stimuli.
In addition to linking perceptual judgments and
image analyses, we investigated whether we could find
hints to the three highlights’ features in the pictorial
recipe for grapes contained in a 17th century treatise on
oil painting, The big world painted small (Beurs, 1692/in
press). This treatise represents one of the most valuable
art historical records of the 17th century studio
practice. Throughout the six chapters that make up the
treatise, Beurs (1692/in press) provides detailed in-
structions and practical tips on how to render all kind
of materials and surface textures. As the recipes of
Beurs (1692/in press) were shown to match the painting
practice of some of his contemporaries (Wallert, 1999,
2012; De Keyser et al., 2017), we can use this written
source to grasp 17th century painters’ implicit knowl-
Previous work
It is known that gloss perception interacts with the
3D shape of the target object (Vangorp, Laurijssen, &
e,2007; Ho et al., 2008; Marlow & Anderson,
2015), its surface structure (Pont, van Doorn, Wijntjes,
& Koenderink, 2015), and the light field in which it is
embedded (Fleming et al., 2003; Pont & te Pas, 2006;
Zhang, de Ridder, & Pont, 2015; Wendt & Faul, 2017).
Certain combinations of illumination directions and
surface reliefs can even render a matte Lambertian
surface to look glossy (Wijntjes & Pont, 2010).
The presence of either highlights or lowlights is
recognized to be the minimum requisite to convey a
glossy impression (Beck & Prazdny, 1981; Berzhan-
skaya et al., 2005; Kim et al., 2012; van Assen et al.,
2016), as long as they are placed at the ‘‘right’’ position
on the surface, i.e., along the direction of minimal
curvature (Koenderink & van Doorn, 1980; Beck &
Prazdny, 1981; Fleming, Torralba, & Adelson, 2004;
Anderson & Kim, 2009; Kim, Marlow, & Anderson,
2011). Moreover, highlights with simple shapes, as
squares or circles, were found to be more effective than
complex ones in producing a glossy impression (van
Assen et al., 2016).
In 1937, Hunter pioneered the idea of the multidi-
mensionality of glossiness, identifying six classes of
gloss that differ in their appearance. In doing so, he laid
the groundwork for the perceptual dimensions often
used in the subsequent investigations on gloss. Fer-
werda et al. (2001) revealed the limitations of the
dimensions proposed by Hunter, as being defined a
priori. They suggested, instead, a psychophysically
based model to predict gloss perception. Via multidi-
mensional scaling, Ferwerda et al. (2001) built a visual
gloss space, whose perceptually meaningful axes were
contrast and sharpness of the reflected image, for the
specific set of conditions and stimuli they used. This
gloss space should probably be extended with more
dimensions, for extensions of the range of stimuli
beyond dielectrics.
One popular approach to understand material
perception from low-level image cues involves image
statistics. For gloss perception, it has been proposed
that the statistical moments of the luminance histogram
of the image, such as the skewness, could be used as
predictor (Motoyoshi, Nishida, Sharan, & Adelson,
2007; Sharan, Li, Motoyoshi, Nishida, & Adelson,
2008). However, further researches have demonstrated
that the skewness is not enough to explain perceived
glossiness (Anderson & Kim, 2009; Kim & Anderson,
2010; Kim, Tan, & Chowdhury, 2016), and it fails to
account for the influence of illumination geometry
(Olkkonen & Brainard, 2010, 2011).
Wiebel, Toscani, and Gegenfurtner (2015) found an
effect of skewness on glossiness, but only for computer
Journal of Vision (2019) 19(3):7, 1–15 Di Cicco, Wijntjes, & Pont 2
Downloaded from on 03/26/2019
rendered stimuli. When they tested photographs of
natural surfaces, the main discriminative statistic for
glossiness was the standard deviation of the luminance
histogram, a measure for the contrast. Given the wide
variety of glossy materials present in the world, Wiebel
et al. (2015) proposed the use of photographs of real
surfaces as alternative to the time-consuming procedure
of computer rendering.
Here, we aim to extend the study of gloss
perception to true-to-life paintings, starting with
paintings of grapes. We tested the three highlight
features proposed by Marlow and Anderson (2013),
but instead of relying on human judgment to estimate
them, we developed a new method to semiautomat-
ically compute them from the segmented images of
the paintings. Previous research concerning the
influence of image cues on material perception have
hitherto either used human judgments (Marlow et al.,
2012; Marlow & Anderson, 2013) or luminance
histogram-based moment statistics (Motoyoshi et al.,
2007; Sharan et al., 2008). The former approach
appears to be motivated by difficulties designing
robust algorithms that capture image properties like
coverage, contrast, and sharpness of the highlights
(but see also Qi, Chantler, Siebert, & Dong, 2014) for
segmentation of highlights based on pixel intensity
threshold and pixel wise calculation of the features for
the case of rendered surfaces with identical parame-
ters settings). Yet, the drawback of relying on human
judgments is that there could be interaction effects
(e.g., an object appears glossier, causing the contrast
to be perceived higher). Please note that the compu-
tation of these features also involves more than only
the luminance histogram; in order to determine the
acteristics of highlights need to be taken into account
too, complicating the computation. Therefore, we
propose an intermediate approach where human
annotations assist the computation of the image
properties. Understanding the effectiveness of this
approach will not only answer our specific research
questions but may also be useful for other studies
concerning the relation between image cues and
material perception.
Glossiness rating experiments were conducted on the
cropped (A) and original (B) versions of the stimuli.
The first experiment (A) was performed as main
experiment, while the second (B) was done to
investigate the influence of context of the whole
painting on the gloss judgment of grapes. The
highlights of the grapes were manually segmented from
the images of the paintings. The luminance profile of
each segmented highlight was semiautomatically ex-
tracted to quantify the features.
The stimuli used were high-resolution, digital images
of 17th century paintings (78 in total), downloaded
from the online repositories of several museums (see
Supplementary Figure S1 for a numbered list of all the
squared cut-outs used for rating experiment A. Each
image in the list has the embedded link to the relative
museum repository website, where the original images
used in experiment B can be found).
For experiment A, the stimuli consisted of squared
cut-outs containing the target bunch of grapes (Figure
1, left). The gloss rating experiment B was conducted
using images of the entire paintings (Figure 1, right).
The segmentation task was performed using the latter
(Figure 1, right).
Experimental set up
For both rating experiments and the highlights
segmentation task, the stimuli were presented in a
darkened room, on an EIZO LCD monitor (CG277),
with built-in self-calibration sensor. To ensure color
consistency across the experiments, the monitor was
calibrated before each session, using the software
‘‘Color Navigator 6’’ (EIZO, version The
brightness level was always set to 100 cd/m
and the
color temperature to 5500 K. The interfaces of the
experiments were programmed in MATLAB R2016b,
using the Psychtoolbox Version 3.0.14 (Brainard, 1997;
Pelli, 1997; Kleiner et al., 2007).
The two rating experiments were conducted with
different groups of participants. Nine observers took
part in rating experiment A, using the cut-out stimuli
containing the grapes. For experiment B, six observ-
ers were asked to rate glossiness for the images of the
entire paintings. All participants had normal or
corrected-to-normal vision. They were na¨
ıve to the
purpose of the experiments. They provided written
consent prior to the experiment and received a
compensation for their participation. The experi-
ments were conducted in agreement with the Decla-
ration of Helsinki and approved by the Human
Research Ethics Committee of the Delft University of
Journal of Vision (2019) 19(3):7, 1–15 Di Cicco, Wijntjes, & Pont 3
Downloaded from on 03/26/2019
Procedure rating experiments
Rating experiments A and B differed in that either a
part of the painting or the whole was shown (Figure 1),
and in the number of material properties rated. For
both experiments, the images were presented against a
black background. Before starting the experiments,
participants went through all the stimuli in order to get
an overview of the stimulus range. No time limit was
given to complete the task.
Experiment A
In experiment A the squared cut-outs containing the
target grapes were used as stimuli. The rating of gloss
was part of a larger experiment. Observers were asked
to rate five different attributes on a continuous 7-point
scale. Apart from the attribute glossiness, they also
rated translucency, bloom, three-dimensionality, and
convincingness. The ratings of the other four attributes
were not considered in the analysis since they are not
relevant for the purpose of the current discussion.
Before starting the experiment, a written definition of
each attribute was provided to the observers, and their
understanding of the meaning of glossiness, translu-
cency, and bloom was verified with a paired compar-
ison test. A pair of photographs of real grapes was
shown to the participants to test the three attributes,
with one photo having the attribute and one not.
Observers were asked to choose which one was glossier
(or more translucent/bloomy). They were given feed-
back on the answers and, if they were able to choose the
right option, they could start the experiment. The
question presented on the screen was ‘‘how [attribute] is
this bunch of grapes on average?’’ The attributes were
tested separately in five blocks, in a random order
(between and within each block). Altogether the 78
stimuli were rated five times, once for each attribute,
resulting in 390 trials per observer.
In the data analysis, the possible differences in rating
due to having rated glossiness as first attribute or as
last, i.e., after having seen a certain stimulus for the first
time or the fifth time, were tested via interrater
reliability analysis.
Experiment B
Rating experiment B, using the entire paintings, was
performed to check the assumption that the context of
the painting does not play a significant role on judging
the grapes’ glossiness. The term ‘‘glossiness’’ was
explained to the observers, and their understanding was
checked as in experiment A, with the same two-
alternative choice test. If multiple bunches were
depicted, the researcher indicated to the observer the
bunch of grapes in the painting to be rated. The images
were presented in a random order to each participant.
The rating was done on a continuous 7-point scale.
Procedure image segmentation
For the segmentation analysis of the stimuli, the full
images of the paintings were used. The highlights’
segmentation was performed by the first author. On
average, 17.64 grapes were segmented from the images
of the bunches used in the rating experiments, in order
to have a representative set of samples. The segmen-
Figure 1. Left, squared cut-out (experiment A); right, whole painting (experiment B). Abraham Mignon, Still Life with Fruit and Oysters,
1660–1679. Downloaded from the online repository of the Rijksmuseum, Amsterdam.
Journal of Vision (2019) 19(3):7, 1–15 Di Cicco, Wijntjes, & Pont 4
Downloaded from on 03/26/2019
tation procedure consisted of drawing a polygon
around the grape’s contour (blue line, Figure 2),
followed by another polygon around the outline of the
corresponding highlight (green line, Figure 2). The
image could be freely zoomed in (up to the pixel level)
and out in order to perform the segmentation.
Procedure computation of highlights’ features
from the images
Since no conventional method can be found in
literature on how to compute the image features that
are diagnostic for material properties, we propose a
novel approach. We developed a series of functions in
Mathematica (version 11.2) for the semiautomated
computation of the contrast, coverage, and sharpness
of the highlights, which were manually segmented from
the images. Although we tried to make the algorithm to
extract the highlights’ features fully automated, a
manual inspection of the luminance profiles was still
needed to correct the data. Because of the extremely
uncontrolled nature of the stimuli, several factors
interfered with the analysis of the images. The major
factor was that, since we used photographs of old
paintings, cracks on the surface of the paint (visible as
dark lines) added noise to the pixel wise analysis of the
The coverage was calculated as the ratio between the
area of the highlight and the total area of the grape.
Sharpness and contrast were derived from the lumi-
nance profiles of the segmented grapes. We extracted
the luminance profile from a cross-section of the
segmented grape, centered in the middle of the
highlight. The cross-sections covered the width of the
grapes and were 3 pixels high; the luminance profiles
were averaged over these 3 pixels, smoothing out
potential outliers.
Contrast values were calculated as the Michelson’s
contrast (Michelson, 1891), taking the maximum and
minimum luminance values of the peak profile as
shown by the horizontal lines pink and yellow in Figure
3. The choice of the orientation for the extraction of the
luminance profile (blue line crossing the highlight on
the grape in Figure 3) will be addressed later.
Instead of sharpness we considered the inverse,
which we named the blurriness and quantified as
Blurriness ¼Dy
Figure 2. Full painting with one grape and its highlight manually
segmented. Abraham Mignon, Still Life with Fruit and Oysters,
1660–1679. Downloaded from the online repository of the
Rijksmuseum, Amsterdam.
Figure 3. Illustration of the values extraction from a luminance profile for the computation of Michelson’s contrast and of the slope on
the two sides of the peak. The horizontal lines pink and yellow show how the minimum and maximum values of the highlight’s peak
were extracted, whereas the oblique lines green and red show the computation of the maximum derivative. Note that the xaxis
shows the pixel width of the grape from the original image, but for the data analysis all the values were normalized for the pixel width
of the grape shown on the screen.
Journal of Vision (2019) 19(3):7, 1–15 Di Cicco, Wijntjes, & Pont 5
Downloaded from on 03/26/2019
where Dy is the difference between the maximum and
minimum luminance values of the peak and Dy/Dx
corresponds to the maximum derivative, taken and
averaged over the two sides of the highlight profile
(oblique red and green lines in Figure 3). The Dx
values were normalized to the visual size of the
grapes shown on the screen during the rating
experiment A.
Dx represents the transition area from the back-
ground (diffuse scattering) to the highlight (specular
reflections). The relationship between Dx and blurriness
is illustrated in Figure 4.Dx increases with the
blurriness, and thus it is inversely related to sharpness.
Throughout the rest of the paper, we will refer to
blurriness instead of sharpness. Figure 4 also shows
that changing the contrast does not affect Dx.
Because of the irregular shapes of the highlights, the
luminance profiles were extracted at 36 different angles,
between 08and 1758, in steps of 58. The results were
averaged over the different angles. Two examples of
Figure 4. Illustration of the relationships of Dx with blurriness and contrast. For the sharp circles, the luminance profile shows a step,
with Dx¼0. As blurriness increases, the transition becomes more gradual and Dx increases. Dx is not dependent on the contrast.
Journal of Vision (2019) 19(3):7, 1–15 Di Cicco, Wijntjes, & Pont 6
Downloaded from on 03/26/2019
luminance profiles acquired at 08and 908are shown in
Figure 5.
The highlights do not only show irregular shapes,
but many also show an internal spatial structure. They
were often rendered as a window reflection (van Assen
et al., 2016). Thus, the inner structure of the window,
visible in the reflection, constitutes an additional term
of variation in the luminance profile, depending on the
angle of computation. This is evident in Figure 5.
Extracting the profile either perpendicular or parallel to
the internal line of the window drastically changes the
shape of the luminance profile. Hence, in some cases
the maximum derivative is detected in the middle of the
highlight instead of at the outer edges. We assume that
the visual system detects the sharpest edges and they do
not necessarily need to be the outer ones.
Finally, the features’ values for each bunch of grapes
were obtained from the average of the segmented
grapes, analyzed as just described.
Glossiness rating experiments
As mentioned before, the gloss rating experiment
was performed once using the squared cut-outs
containing the target grapes, and again with the entire
images of the paintings. Before comparing the data,
we analyzed the internal consistency of the ratings in
experiment A. Here, glossiness was rated as part of a
larger experiment in which observers were asked to
judge also four other attributes, in random order.
Their evaluation of glossiness may thus have been
biased by the order of the attributes and the number of
times the stimuli were seen before rating glossiness. A
reliability analysis resulted in a Cronbach’s alpha
coefficient of 0.83, demonstrating high consistency in
the ratings. A Spearman rank test was also performed,
showing that all the observers’ data were significantly
Figure 5. Example of a segmented grape and its highlight. The luminance profile above was taken at 08, the one below at 908.
Journal of Vision (2019) 19(3):7, 1–15 Di Cicco, Wijntjes, & Pont 7
Downloaded from on 03/26/2019
correlated (p,0.05) with each other. Nevertheless,
the reliability analysis between observers of experi-
ment B, who rated only glossiness of the grapes seeing
the entire paintings, gave a Cronbach’s alpha coeffi-
cient of 0.97. Ttest showed that the two Cronbach’s
alpha values are significantly different (p,0.05). This
may indicate that increasing the number of material
properties to be rated decreased the agreement
between observers, but gloss ratings from experiment
A are still reliable.
To minimize possible effects of unequal interval
judgments, the data of each observer for both rating
experiments were rescaled from the 7-point scale to the
0–1 range before averaging. The average gloss ratings
of experiments A and B were correlated, in order to test
possible effects of the painting context on the
judgment. The trend of the correlation is shown in
Figure 6. The ratings resulted in a strong and
significant correlation (r¼0.74, p,0.001). The
regression line that best fit the data gives an offset of
0.02 nonsignificantly different from 0, but a slope of
0.96 significantly (p,0.05) different from 1. This
means that the participants of experiment A perceived
a wider range of glossiness levels than the ratings used
by participants in experiment B. Such systematic effect
of the slope may be due to the grapes’ bunch size shown
in the two experiments. They were clear and close-up in
the cut-outs (A), but small when shown in the entirety
of big paintings (B). The values of the mean gloss
ratings for the two experiments and their standard
deviations are reported in Supplementary Table S2.
In Figure 7, a bar chart shows average ratings from
experiment A for the three stimuli judged most and the
three judged least glossy. We do not know the ground
truth of the glossiness levels of the painted grapes, but
since the average minimum and maximum levels were
more than 0.6 apart, whereas random data would have
shown both the minimum and maximum around 0.5,
we can conclude that the ratings were internally
consistent and the stimuli obviously cover a perceptual
well distinguishable range.
Glossiness prediction based on the segmented
highlights’ features
Using the image processing technique described in
the method section, we quantified the features of the
segmented highlights. To explore the relationships
between the features contrast, blurriness, and coverage
and the perceived glossiness, we employed principal
component analysis (PCA) and multiple linear regres-
sion. In the PCA biplot, shown in Figure 8, we can see
how the scores, i.e., the images (numbered points; see
Supplementary Figure S1 for the image corresponding
to each number) were distributed with respect to the
variables. The variables represent the three highlight
features and the mean gloss rating from experiment A
(a PCA biplot representing the relationships between
only the three highlight features is shown in Supple-
mentary Figure S2, with the corresponding factor
loadings in Supplementary Table S1). To account for
the different scales of the variables, we performed the
PCA based on the correlation matrix.
The first two principal components together account
for 83.4%of the variance. From the factor loadings
(Table 1), we see that the first component is strongly
Figure 6. Scatterplot of the correlation between the average
glossiness ratings of the squared cut-outs (experiment A) and of
the whole paintings (experiment B). Results show r¼0.74, p,
0.001; the area around the fit line represents the 95% CI.
Figure 7. Mean ratings for experiment A (cut-outs) of the three
bunches of grapes judged most glossy and the three least
glossy. The corresponding images are shown below each bar.
The error bars indicate the standard errors of the mean.
Journal of Vision (2019) 19(3):7, 1–15 Di Cicco, Wijntjes, & Pont 8
Downloaded from on 03/26/2019
loaded by contrast and perceived gloss in one direction
and by blurriness in the opposite direction. This means
that glossiness varied positively with contrast and
negatively with blurriness. The correlation between
perceived gloss and contrast is indeed positive and
significant with r¼0.80, p,0.001, and it is negative
and significant between perceived gloss and blurriness
with r¼0.61, p,0.001. On the second component,
the variable with the highest loading is coverage. This
suggests that coverage was not correlated with gloss-
iness, and indeed r¼0.03, p.0.05 between glossiness
and coverage. Correlation plots for each highlight
feature with perceived glossiness are shown in Supple-
mentary Figure S3, and the corresponding values of the
average gloss rating and highlight features are reported
in Supplementary Table S2.
To predict the perception of glossiness based on the
highlight features, we used multiple linear regression.
We found the best fit (Equation 2) for a model carrying
contrast and blurriness as significant (p,0.001)
Perceived gloss ¼0:32 þ1:1 Contrast
2:05 Blurriness ð2Þ
This model explains (r
)69%of the variance of
perceived glossiness.
One aim of the study presented in this paper was to
test whether the diagnostic power of highlight features
proposed by Marlow and Anderson (2013) could be
transferred from computer rendered to painted stimuli.
We therefore first measured the perception of glossiness
of grapes in bunches, extracted as squared cut-outs
from the images of the paintings. Alongside this rating
experiment, a second experiment was performed,
showing to the observers the entire images of the
paintings in order to test whether the context influences
the perceived glossiness of the grapes. The strong and
significant correlation, and the lack of systematic effect
of the fit offset that we found for the average ratings of
the two experiments, show that a potential influence
due to the context was not critical. However, the
systematic effect of the slope indicates that a wider
range of ratings was used for the cut-outs compared to
the entire paintings. The different sizes of the bunches
of grapes shown in the two experiments may have
caused this effect. In the cut-outs of experiment A, they
were all shown with similar, close-up sizes; thus,
smaller variations of glossiness image cues may have
been more visible.
To measure the three highlight features (contrast,
blurriness, and coverage), we segmented the grapes
from the images and computed the features via image
Figure 8. PCA biplot showing the scores (images) distribution with respect to the variables (highlights’ features and gloss ratings), and
the relationships between the variables themselves.
Gloss 0.61 0.02
Contrast 0.59 0.03
Blurriness 0.53 0.04
Coverage 0.054 0.99
Table 1. Factor loadings of the first two principal components.
Journal of Vision (2019) 19(3):7, 1–15 Di Cicco, Wijntjes, & Pont 9
Downloaded from on 03/26/2019
analysis. Contrast and blurriness were found to be the
predictors for the best fit model of gloss perception,
accounting for 69%of the explained variance. The
amount of explained variance (r
) given by Marlow and
Anderson (2013) for their set of experiments ranged
between 0.91 and 0.97. We cannot make a direct
comparison with their r
values because of the
fundamental difference with our stimuli and for the
method used to quantify the highlight features.
However, we assume that the main explanation for our
lower r
can be imputed to the uncontrolled nature of
the paintings. As a future step, the algorithms we used
to quantify the highlight features should be correlated
to their perceptual measures (i.e., via human estimation
of the cues), in order to validate the psychophysical
relevance of our method.
Contrast and blurriness were found to be the main
contributors regarding gloss rendering of grapes in
paintings, as shown by the high correlation between the
variables in the PCA biplot (Figure 8). Dutch 17th
century painters may have been aware of the impor-
tance of the highlights’ contrast and may have
intentionally emphasized such feature by placing a dark
line or area along the highlight contour as a pictorial
trick (Figure 9). The importance of contrast for
rendering glossiness in paintings is also confirmed by
the findings of Cavanagh et al. (2008). They found that
in paintings the only requirements for highlights on
curved surfaces are to be brighter than the surrounding
and to be appropriately curved.
Blurriness had the expected negative correlation with
gloss perception. For coverage we found no significant
effect. This is comparable with what Marlow and
Anderson (2013) reported for rendered spheres. They
did find an effect of perceived coverage on glossiness,
but this effect and the perceived variation of the
highlights’ coverage were the lowest compared to
perceived contrast and sharpness. If the light source has
one main direction, only a small part of a spherical
surface will be covered by highlights, because a sphere
has a uniform distribution of surface normals. Grapes
are spherical (or ellipsoidal) objects, and in still life
paintings it was common practice to suggest the
presence of a single source of light coming from a
window, usually placed top left (Mamassian, 2008).
Thus, the coverage is rather small and constant
throughout the various paintings and the different
levels of glossiness.
From previous works it is clear that the research on
gloss perception cannot be reduced to the highlights
only, since the appearance of the highlights is
influenced by other factors like the illumination field
(Fleming et al., 2003; Pont & te Pas, 2006; Zhang et al.,
2015; Wendt & Faul, 2017) and the 3D shape of the
object (Vangorp et al., 2007; Ho et al., 2008; Marlow &
Anderson, 2015). However, it is also known that
painters often abstract the rules of physics into an
Figure 9. Details of two bunches of grapes used as stimuli showing an example of the use of dark lines around the contour of some of
the highlights. Left: Nicolaes van Gelder, Still Life, 1664. Right: Pieter de Ring, Still Life with Golden Goblet, 1640–1660. Downloaded
from the online repository of the Rijksmuseum, Amsterdam.
Journal of Vision (2019) 19(3):7, 1–15 Di Cicco, Wijntjes, & Pont 10
Downloaded from on 03/26/2019
‘‘alternative physics’’ (Cavanagh, 2005), which allows
portraying just the key information for an efficient
recognition of the scene, leaving errors and incongru-
ences unnoticed at first glance.
This is, for example, the case for the congruency
between the orientation of the highlights and of the
grapes’ shapes. It is well known in literature that one of
the fundamental requisites for highlights is to be placed
at the ‘‘right’’ position on the surface (Koenderink &
van Doorn, 1980; Beck & Prazdny, 1981; Fleming et
al., 2004; Anderson & Kim, 2009; Kim, Marlow, &
Anderson, 2011). Still, when we measured the orienta-
tion of an ellipse fitted onto the highlight and that of an
ellipse fitted on the grape, we did not find a correlation
for the set of bunches perceived as highly glossy. The
orientations were found to be more congruent instead
(r¼0.57, p,0.001) for the medium to low glossy
grapes. This finding contradicts the literature as well as
the physics. Figure 10 shows on the left a photo of a
real bunch of grapes and on the right one of the painted
bunches considered among the glossiest. In the photo,
each grape has its own orientation, as indicated by the
black arrows, and their highlights are always coherently
aligned (red arrows). The painting, on the other hand,
shows visible incongruences. Nonetheless, such inac-
curate orienting of the highlights does not seem to
hinder the perception of glossiness, nor improve it
when they are more coherently aligned on the low and
medium glossy grapes. Another discrepancy between
the laws of physics and the ‘‘physics of paintings’
concerns the elongation of the highlights’ shape with
respect to the distance of the highlight from the center
of the grape, which is related to the slant angle of the
light direction. Assuming a spherical shape for the
grapes, we calculated the highlights’ position. We
retrieved the light direction as the tilt and the slant
angle. With an average of 1438for the tilt angle and of
518for the slant angle (Figure 11), we could confirm the
top-left convention for the illumination orientation,
Figure 10. Left: Photo of a real bunch of grapes taken by the authors in the lab. Right: A bunch of grapes considered among the
glossiest of our set of stimuli (Pieter de Ring, Still Life with Golden Goblet, 1640–1660. Downloaded from the online repository of the
Rijksmuseum, Amsterdam). The black arrows indicate the orientation of the grapes while the red arrows show the orientation of the
highlights. The arrows were drawn by hand. In the photo, the grapes are oriented differently, but each highlight follows the shape
orientation of the grapes; in the painting, the orientations of the highlights appear to be randomly scattered across the bunch, and
they are not consistently congruent with the grapes’ shape.
Figure 11. Polar plot of the average tilt and slant angles. The
plot shows the top-left convention of the light orientation used
in paintings.
Journal of Vision (2019) 19(3):7, 1–15 Di Cicco, Wijntjes, & Pont 11
Downloaded from on 03/26/2019
which is a well-known perceptual prior (Mamassian &
Goutcher, 2001; Morgenstern, Murray, & Harris,
2011), also found in paintings (Mamassian, 2008;
Carbon & Pastukhov, 2018; Wijntjes, 2018). We found
that the highlights’ elongations were not consistent with
the slant angles of the illumination. Nevertheless, this
did not influence gloss judgments throughout our
stimulus set, as no correlation was found.
We found that breaking the rules of the orientation
congruency and of the elongation of highlights with the
light slant do not affect glossiness perception. We
assume that this is the case, because the highlights’
contrast has the predominant effect in our set of
As will be discussed later, the artistic conventions,
including the recipes given by Beurs (1692/in press),
state to use white to render the highlights on grapes.
This can be an example of the above-mentioned key
information representing statistical regularities of real
scenes and transferred to the canvas by the painter. In
fact, grapes are dielectric materials, so they have
specular highlights of the same color as the light source
(Klinker, Shafer, & Kanade, 1990; Nishida et al., 2008).
Measuring the chroma of the segmented highlights, we
found a significant negative correlation with glossiness
(r¼0.35, p,0.01), which means that the more
colored the highlights are (which can also be due to
ageing and yellowing of the painting), the less glossy
the grapes will be perceived.
One of the next steps would be to include the
perceptual attribute of ‘‘haze gloss’’ (Hunter, 1937;
Vangorp, Barla, & Fleming, 2017) to the representation
of the glossy appearance of grapes. Grapes are
naturally covered by bloom, a waxy coating that looks
like a whitish matte layer. Usually, it is not evenly
spread over the fruit surface, since it can be easily
deleted by handling or transportation, and it can also
have various thickness, but in general the more bloom
is present, the less glossy the fruit appears (Mukhtar,
Damerow, & Blanke, 2014; Loypimai, Paewboonsom,
Damerow, & Blanke, 2017). However, the highlight can
be also placed next to a highly bloomy area, making the
role of bloom in tuning gloss perception far from
Our findings on the use of the highlights’ features to
render glossiness of grapes in 17th century painting
practice are supported by the painting manual of Beurs
(1692/in press). In his recipe for grapes, no instruction
can be found on how much of the fruit surface should
be covered with highlights. He may not have mentioned
it, either because experience and observation would
have been enough to get this notion, or because, as we
found, the coverage has no significant role in the case of
The recipe contains less ambiguous indications for
what concerns contrast and blurriness. It states that the
highlight should be placed where the surface is not
covered with bloom, and it should be painted white. In
the area where no bloom is present, the skin color of
the grape is visible. Applying a white spot on a colored
background mainly affects the contrast. For blurriness,
Beurs (1692/in press) specified that care should be
taken, when applying the white highlight, to ‘‘gently
blend it in.’’ He referred to the edges of the reflection,
blending the white of the specular reflection with the
color of the diffuse body scattering, resulting in more
gradual edges. Since grapes are not mirror-like
materials, this procedure would increase the natural
appearance of the fruit and thus its convincingness.
It would be interesting to apply the model, having
contrast, blurriness, and coverage of the highlights as
predictors, to other glossy materials depicted in
paintings, and see whether the contributions of the
predictors change.
Lastly, we showed that it is possible to extract image
cues by manually indicating the highlight and the
contour of the grape. Using this input, highlight
profiles can be generated that contain information
about contrast, blurriness, and coverage. To our
knowledge, this approach is relatively new and seems a
valuable addition to research on visual material cues.
Until now, research has either focused on the physical
parameters (that lead to the image cues, e.g., Ferwerda
et al., 2001), global image statistics (Motoyoshi et al.,
2007) or human estimates of cue strength (Marlow &
Anderson, 2013). Almost all of these studies were
performed on well-controlled computer rendered stim-
uli. Although our stimuli are clearly also artificial, they
are uncontrolled. Our approach can be readily gener-
alized to ‘‘natural images,’’ like the Flickr Material
Database (FMD) (Sharan, Rosenholtz, & Adelson,
We have measured the amount of glossiness per-
ceived in paintings of grapes from the Dutch Golden
Age, a period characterized by the detailed realistic
imitation of nature. We have predicted perceived
glossiness using the key features of the highlights,
which can be observed in the image (Marlow et al.,
2012; Marlow & Anderson, 2013). The novelty of our
work consisted in the use of uncontrolled stimuli and in
the method we have used to measure the features.
Contrast, coverage, and blurriness were mathematically
defined, and calculated directly from the segmented
Contrast and blurriness were found to be the main
predictors for gloss perception. Coverage, on the other
hand, was found to have no influence at all. We could
Journal of Vision (2019) 19(3):7, 1–15 Di Cicco, Wijntjes, & Pont 12
Downloaded from on 03/26/2019
find hints to the same conclusions in the painting
instructions for grapes given by Beurs (1692/in press).
We also found support for the idea that painters used
to sacrifice the true physics of light and instead use key
factors of the optical phenomena, and that does not
affect glossiness perception (or perhaps even enhance
We have shown that the research on gloss perception
can be extended to paintings, and eventually also to the
study of historical sources. Via image analysis, we have
demonstrated that two of the three cues proposed by
Marlow and Anderson (2013), were used by 17th
century painters to elicit gloss perception of grapes.
Keywords: material perception,gloss perception,
paintings,image analysis,image features
This work is part of the research program NICAS
‘‘Recipes and Realities’’ with project number
628.007.005, which is partly financed by the
Netherlands Organization for Scientific Research
(NWO) and partly by Delft University of Technology.
Maarten Wijntjes was financed by the VIDI project
‘‘Visual communication of material properties,’
number 276.54.001.
Commercial relationships: none.
Corresponding author: Francesca Di Cicco.
Address: Perceptual Intelligence Lab, Faculty of
Industrial Design Engineering, Delft University of
Technology, Delft, the Netherlands.
Adelson, E. H. (2001). On seeing stuff: The perception
of materials by humans and machines. Proceedings
SPIE 4299, Human Vision and Electronic Imaging
VI,4299, 1–12.
Anderson, B. L., & Kim, J. (2009). Image statistics do
not explain the perception of gloss and lightness.
Journal of Vision,9(11):10, 1–17,
1167/9.11.10. [PubMed] [Article]
Beck, J., & Prazdny, S. (1981). Highlights and the
perception of glossiness. Attention, Perception, &
Psychophysics,30(4), 407–410.
Berzhanskaya, J., Swaminathan, G., Beck, J., &
Mingolla, E. (2005). Remote effects of highlights on
gloss perception. Perception,34, 565–575.
Beurs, W. (1692). De groote waereld in ’t kleen
geschildert, of schilderagtig tafereel van ’s Weerelds
schilderyen. Kortelijk vervat in ses boeken. Ver-
klarende de hooftverwen, haare verscheide menge-
lingen in oly en der zelver gebruik (The big world
painted small, or colorful tableau of the world in
paintings. Concisely presented in six books ex-
plaining the main colors, their various mixtures in
oil and their use). Amsterdam, the Netherlands:
van Waesberge.
Beurs, W. (in press). The big world painted small (M.
Scholz, trans.). Los Angeles, CA: The Getty
Research Institute.
Brainard, D. H. (1997). The Psychophysics Toolbox.
Spatial Vision,10, 433–436.
Carbon, C. C., & Pastukhov, A. (2018). Reliable top-
left light convention starts with early Renaissance:
An extensive approach comprising 10k artworks.
Frontiers in Psychology,9:454, 1–7.
Cavanagh, P. (2005, March 17). The artist as neuro-
scientist. Nature,434, 301–307.
Cavanagh, P., Chao, J., & Wang, D. (2008). Reflec-
tions in art. Spatial Vision,21, 261–270.
Conway, B. R., & Livingstone, M. S. (2007). Perspec-
tives on science and art. Current Opinion in
Neurobiology,17, 476–482.
De Keyser, N., Van der Snickt, G., Van Loon, A.,
Legrand, S., Wallert, A., & Janssens, K. (2017). Jan
Davidsz. de Heem (1606-1684): A technical exam-
ination of fruit and flower still lifes combining MA-
XRF scanning, cross-section analysis and technical
historical sources. Heritage Science,5(38), 1–13.
DiPaola, S., Riebe, C., & Enns, J. T. (2013). Following
the masters: Portrait viewing and appreciation is
guided by selective detail. Perception,42, 608–630.
Ferwerda, J. A., Pellacini, F., & Greenberg, D. P.
(2001). A psychophysically based model of surface
gloss perception. Proceedings of SPIE, Human
Vision and Electronic Imaging VI,4299, 291–301.
Fleming, R. W., Dror, R. O., & Adelson, E. H. (2003).
Real-world illumination and the perception of
surface reflectance properties. Journal of Vision,
3(5):3, 347–368,
[PubMed] [Article]
Fleming, R. W., Torralba, A., & Adelson, E. H. (2004).
Specular reflections and the perception of shape.
Journal of Vision,4(9):10, 798–820,
10.1167/4.9.10. [PubMed] [Article]
Ho, Y.-X., Landy, M. S., & Maloney, L. T. (2008).
Conjoint measurement of gloss and surface texture.
Psychological Science,19(2), 196–204.
Hunter, R. S. (1937). Methods of determining gloss.
Journal of Vision (2019) 19(3):7, 1–15 Di Cicco, Wijntjes, & Pont 13
Downloaded from on 03/26/2019
Journal of Research of the National Bureau of
Standards,18(1), 19–41.
Kim, J., & Anderson, B. L. (2010). Image statistics and
the perception of surface gloss and lightness.
Journal of Vision,10(9):3, 1–17,
1167/10.9.3. [PubMed] [Article]
Kim, J., Marlow, P. J., & Anderson, B. L. (2011). The
perception of gloss depends on highlight congru-
ence with surface shading. Journal of Vision,11(9):
4, 1–19, [PubMed]
Kim, J., Marlow, P. J., & Anderson, B. L. (2012,
November 15). The dark side of gloss. Nature
Neuroscience,15, 1590–1595, doi:10.1038/nn.3221.
Kim, J., Tan, K., & Chowdhury, N. S. (2016). Image
statistics and the fine lines of material perception. i-
Perception,7(4), 1–11.
Kleiner, M., Brainard, D., Pelli, D., Ingling, A.,
Murray, R., & Broussard, C. (2007). What’s new in
psychtoolbox-3. Perception,36(14), 1–16.
Klinker, G., Shafer, S., & Kanade, T. (1990). A
physical approach to color image understanding.
International Journal of Computer Vision,4, 7–38.
Koenderink, J. J., & van Doorn, A. (1980). Photo-
metric invariants related to solid shape. Optica
Acta,27(7), 981–996.
Loypimai, P., Paewboonsom, S., Damerow, L., &
Blanke, M. M. (2017). The wax bloom on
blueberry: Application of luster sensor technology
to assess glossiness and the effect of polishing as a
fruit quality parameter. Journal of Applied Botany
and Food Quality,90, 154–158.
Mamassian, P. (2008). Ambiguities and conventions in
the perception of visual art. Vision Research,48,
Mamassian, P., & Goutcher, R. (2001). Prior knowl-
edge on the illumination position. Cognition,81,
Marlow, P. J., & Anderson, B. L. (2013). Generative
constraints on image cues for perceived gloss.
Journal of Vision,13(14):2, 1–23,
1167/13.14.2. [PubMed] [Article]
Marlow, P. J., & Anderson, B. L. (2015). Material
properties derived from three-dimensional shape
representations. Vision Research,115, 199–208.
Marlow, P. J., Kim, J., & Anderson, B. L. (2012). The
perception and misperception of specular surface
reflectance. Current Biology,22, 1909–1913.
Melcher, D., & Cavanagh, P. (2011). Pictorial cues in
art and visual perception. In F. Bacci & D. Melcher
(Eds.), Art and the senses (pp. 359–394). London,
UK: Oxford University Press.
Michelson, A. A. (1891). On the application of
interference methods to spectroscopic measure-
ments. I. The London, Edinburgh and Dublin
Philosophical Magazine and Journal of Science,
Fifth Series,31, 338–346 and Plate VII.
Morgenstern, Y., Murray, R. F., & Harris, L. R.
(2011). The human visual system’s assumption that
light comes from above is weak. Proceedings of the
National Academy of Sciences, USA,108, 12551–
Motoyoshi, I., Nishida, S., Sharan, L., & Adelson, E.
H. (2007, May 10). Image statistics and the
perception of surface qualities. Nature,447, 206–
Mukhtar, A., Damerow, L., & Blanke, M. (2014). Non-
invasive assessment of glossiness and polishing of
the wax bloom of European plum. Postharvest
Biology and Technology,87, 144–151.
Nishida, S. Y., Motoyoshi, I., Nakano, L., Li, Y.,
Sharan, L., & Adelson, E. (2008). Do colored
highlights look like highlights? Journal of Vision,
8(6): 339,
Olkkonen, K. M., & Brainard, D. H. (2010). Perceived
glossiness and lightness under real-world illumina-
tion. Journal of Vision,10(9):5, 1–19, https://doi.
org/10.1167/10.9.5. [PubMed] [Article]
Olkkonen, K. M., & Brainard, D. H. (2011). Joint
effects of illumination geometry and object shape in
the perception of surface reflectance. i-Perception,
2, 1014–1034.
Pelli, D. G. (1997). The VideoToolbox for visual
psychophysics: Transforming numbers into movies.
Spatial Vision,10, 437–442.
Pepperell, R., & Ruschkowski, A. (2013). Double
vision as a pictorial depth cue. Art & Perception,1,
Pinna, B. (2007). Art as a scientific object: Toward a
visual science of art. Spatial Vision,20(6), 493–508.
Pont, S. C., & te Pas, S. F. (2006). Material-
illumination ambiguities and the perception of solid
objects. Perception,35, 1331–1350.
Pont, S. C., van Doorn, A. J., Wijntjes, M. W. A., &
Koenderink, J. J. (2015). Texture, illumination, and
material perception. Proceedings SPIE 9394, Hu-
man Vision and Electronic Imaging XX,93940E,1
Qi, L., Chantler, M. J., Siebert, J. P., & Dong, J. (2014).
Why do rough surfaces appear glossy? Journal of
the Optical Society of America A,31(5), 935–943.
Sharan, L., Li, Y. Z., Motoyoshi, I., Nishida, S., &
Adelson, E. H. (2008). Image statistics for surface
Journal of Vision (2019) 19(3):7, 1–15 Di Cicco, Wijntjes, & Pont 14
Downloaded from on 03/26/2019
reflectance perception. Journal of the Optical
Society of America A,25(4), 846–865.
Sharan, L., Rosenholtz, R., & Adelson, E. H. (2014).
Accuracy and speed of material categorization in
real-world images. Journal of Vision,14(9):12, 1–24, [PubMed] [Article]
Van Assen, J. J. R., Wijntjes, M. W. A., & Pont, S. C.
(2016). Highlight shapes and perception of gloss for
real and photographed objects. Journal of Vision,
16(6):6, 1–14,
[PubMed] [Article]
Vangorp, P., Barla, P., & Fleming, R. W. (2017). The
perception of hazy gloss. Journal of Vision,17(5):
19, 1–17, [PubMed]
Vangorp, P., Laurijssen, J., & Dutr´
influence of shape on the perception of material
reflectance. ACM Transactions on Graphics,26,19.
Wade, N. J., Ono, H., & Lillakas, L. (2001). Leonardo
da Vinci’s struggles with representations of reality.
Leonardo,34(3), 231–235.
Wallert, A. (1999). Still lifes: Techniques and style. An
examination of paintings from the Rijksmusuem,
Amsterdam. Amsterdam, the Netherlands: Rijks-
museum; Zwolle, the Netherlands: Waanders.
Wallert, A. (2012). De Groote Waereld in ‘t Kleen
Geschildert [The Big World Painted Small]: A
Dutch 17
century treatise on oil painting tech-
nique. In S. Eyb-Green, J. H. Townsend, M.
Clarke, J. Nadolny, & S. Kroustallis (Eds.), The
artist’s process: Technology and interpretation (pp.
130–137), London, UK: Archetype Publications
Wendt, G., & Faul, F. (2017). Increasing the com-
plexity of the illumination may reduce gloss
constancy. i-Perception,8(6), 1–40.
Wendt, G., Faul, F., & Mausfeld, R. (2008). Highlight
disparity contributes to the authenticity and
strength of perceived glossiness. Journal of Vision,
8(1):14, 1–10,
[PubMed] [Article]
Wiebel, C. B., Toscani, M., & Gegenfurtner, K. R.
(2015). Statistical correlates of perceived gloss in
natural images. Vision Research,115, 175–187.
Wijntjes, M. (2018). Annotating shadows, highlights
and faces: The contribution of a ’human in the
loop’ for digital art history. arXiv:1809.03539 [cs.
Wijntjes, M., & Pont, S. C. (2010). Illusory gloss on
Lambertian Surfaces. Journal of Vision,10(9):13, 1–
12, [PubMed]
Zhang, F., de Ridder, H., & Pont, S. (2015). The
influence of lighting on visual perception of
material qualities. Proceedings SPIE 9394, Human
Vision and Electronic Imaging XX,93940Q, 1–10.
Journal of Vision (2019) 19(3):7, 1–15 Di Cicco, Wijntjes, & Pont 15
Downloaded from on 03/26/2019
... In this study, we studied the perception of painted fabrics in 17 th century Dutch paintings, a class of paintings unanimously acknowledged for the convincing representation of materials and their properties. The economical yet effective rendering of material properties exploited by 17 th century painters (Parraman, 2014) resonates with the mechanisms of the human visual system (Adelson, 2001;Koenderink & van Doorn, 2001;Cavanagh, 2005;Sayim & Cavanagh, 2011;Wijntjes, Doerschner, Kucukoglu, & Pont, 2012;Marlow, Kim, & Anderson, 2017;Di Cicco, Wijntjes & Pont, 2019;Van Zuijlen, Pont, & Wijntjes, 2020;Wijntjes, Spoiala & de Ridder, 2020). Painters carefully chose the image features to include and could choose to omit perceptually irrelevant or hindering features, as was shown to be the case for the orientation of the highlights on grapes which do not need to be congruent with the object shape in order to communicate a glossy appearance (Di Cicco et al., 2019). ...
... The economical yet effective rendering of material properties exploited by 17 th century painters (Parraman, 2014) resonates with the mechanisms of the human visual system (Adelson, 2001;Koenderink & van Doorn, 2001;Cavanagh, 2005;Sayim & Cavanagh, 2011;Wijntjes, Doerschner, Kucukoglu, & Pont, 2012;Marlow, Kim, & Anderson, 2017;Di Cicco, Wijntjes & Pont, 2019;Van Zuijlen, Pont, & Wijntjes, 2020;Wijntjes, Spoiala & de Ridder, 2020). Painters carefully chose the image features to include and could choose to omit perceptually irrelevant or hindering features, as was shown to be the case for the orientation of the highlights on grapes which do not need to be congruent with the object shape in order to communicate a glossy appearance (Di Cicco et al., 2019). Materials were often painted according to standard, well-established instructions which assured the painter of getting the best possible rendering. ...
... The different scattering behaviors of velvet and satin result in distinctive optical cues. Previous studies have shown that image features of the highlights, such as coverage, contrast and sharpness, can influence the perception of glossiness (Marlow, Kim, and Anderson 2012;Marlow & Anderson, 2013;Qi, Chantler, Siebert, & Dong, 2014;Di Cicco, Wijntjes & Pont, 2019;Schmid, Barla & Doerschner, 2020). To test whether the perception of shininess and softness depended on the choice of the cropped area, and therefore on the image features of the highlights present in the crop, we computed the mean luminance of the crops, the relative coverage of the highlights and the mean contrast of the highlights. ...
Full-text available
Dutch 17th century painters were masters in depicting materials and their properties in a convincing way. Here, we studied the perception of the material signatures and key image features of different depicted fabrics, like satin and velvet. We also tested whether the perception of fabrics depicted in paintings related to local or global cues, by cropping the stimuli. In Experiment 1, roughness, warmth, softness, heaviness, hairiness, and shininess were rated for the stimuli shown either full figure or cropped. In the full figure, all attributes except shininess were rated higher for velvet, whereas shininess was rated higher for satin. This distinction was less clear in the cropped condition, and some properties were perceived significantly different between the two conditions. In Experiment 2 we tested whether this difference was due to the choice of the cropped area. On the basis of the results of Experiment 1, shininess and softness were rated for multiple crops from each fabric. Most crops from the same fabric differed significantly in shininess, but not in softness perception. Perceived shininess correlated positively with the mean luminance of the crops and the highlights' coverage. Experiment 1 showed that painted velvet and satin triggered distinct perceptions, indicative of robust material signatures of the two fabrics. The results of Experiment 2 suggest that the presence of local image cues affects the perception of optical properties like shininess, but not mechanical properties such as softness.
... For example, to depict glossiness, painters usually place a lighter spot of paint aligned with the curvature of the objects (Cavanagh et al., 2008). The contrast and sharpness of this spot provide a strong cue of the perceived glossiness (Di Cicco et al., 2019). However, preceding studies have been made on already painted stimuli that (a) usually depict identifiable objects in context and (b) prevent from comparing the material perception between the painting and other renditions of the same scene. ...
... In accordance with previous studies that have shown that glossiness depends on both the contrast and the sharpness of highlights (Pellacini et al., 2000;Marlow & Anderson, 2013;Di Cicco et al., 2019), the glossiness attribute is highly correlated with both the image contrast (r = 0.85) and the highlights sharpness (r = 0.91). ...
... Our findings fully align with what was previously shown in perceptual studies that only considered computer-generated images. We show in this study that the exact same image features are linked to the perception of material properties in realistic paintings, similar to the observations of Di Cicco et al. (2019). This suggests that, despite other visual differences between paintings and renderings, the reproduction of these key visual features by the painter was sufficient to provide a good appearance perception of the materials. ...
Full-text available
Painters are masters in replicating the visual appearance of materials. While the perception of material appearance is not yet fully understood, painters seem to have acquired an implicit understanding of the key visual cues that we need to accurately perceive material properties. In this study, we directly compare the perception of material properties in paintings and in renderings by collecting professional realistic paintings of rendered materials. From both type of images, we collect human judgments of material properties and compute a variety of image features that are known to reflect material properties. Our study reveals that, despite important visual differences between the two types of depiction, material properties in paintings and renderings are perceived very similarly and are linked to the same image features. This suggests that we use similar visual cues independently of the medium and that the presence of such cues is sufficient to provide a good appearance perception of the materials.
... In previous studies, Di Cicco et al. (2019Cicco et al. ( , 2020aCicco et al. ( , b, 2021 have shown that the key image features in Beurs' pictorial recipes are exploited by the visual system to perceive materials and their properties. For example, when describing how to paint grapes, Beurs provides in terms of key features all the optical phenomena that define their appearance. ...
... The highlights were used as perceptual cues for glossiness and translucency perception, in agreement with the literature (Chadwick and Kentridge, 2015;Fleming and Bülthoff, 2005), and the edge reflections also increased the translucent appearance, again in agreement with literature about the cues for translucency (Fleming and Bülthoff, 2005;Gigilashvili et al., 2021;Gkioulekas et al., 2015;Nagai et al., 2013;Wijntjes et al., 2020;Xiao et al., 2020), whereas the image feature of the bloom was the direct trigger of bloom perception. Di Cicco et al. (2019) further showed that the glossiness of grapes is predicted by high contrast, blurry highlights, in agreement with Beurs' instructions. ...
Full-text available
Dutch Golden Age painters could convincingly depict all sorts of materials. How did they do it and how do we perceive them as such, are questions that only recently have started to be addressed by art historians and vision scientists, respectively. This paper aims to discuss how a booklet of pictorial recipes written by the Dutch painter Willem Beurs in 1692 constitutes an index of key image features for material depiction and perception. Beurs' recipes connect different materials according to their shared visual features, and offer the profiles, i.e., the optimal combinations, of these features to render a wide range of materials. By combining representation and perception, the knowledge of painters about the depiction of materials can help to understand the mechanisms of the visual system for material perception, and these in turn can explain the pictorial features that make the pictorial representation of materials so convincing.
... Increasingly, artistic practice has itself become the source of insights for the science of material perception. For example, a series of studies have investigated the depiction of material properties like glossiness in still life and the fidelity of paintings that follow Beurs' instruction (di Cicco et al. 2019(di Cicco et al. , 2020, and Phillips and Fleming (2020) investigated the ability of subjects to distinguish apparent layers of different materials in stimuli inspired by sculptures like the Veiled Christ. This reflects a more general belief that the scientific study of material perception has something to gain from artmaking, both past and present (Schmidt, 2019). ...
Full-text available
Common everyday materials such as textiles, foodstuffs, soil or skin can have complex, mutable and varied appearances. Under typical viewing conditions, most observers can visually recognize materials effortlessly, and determine many of their properties without touching them. Visual material perception raises many fascinating questions for vision researchers, neuroscientists and philosophers, yet has received little attention compared to the perception of color or shape. Here we discuss some of the challenges that material perception raises and argue that further philosophical thought should be directed to how we see materials.
... The present study can be seen as an extension of this previous investigation of perceived reflectance strength to an additional dimension of the gloss impression. The question of whether Fresnel effects also influence the perceived roughness of glossy surfaces is of particular interest because Fresnel effects directly influence only the intensity of the mirror image, but not its sharpness, which according to previous findings is closely related to perceived surface roughness (Cicco, Wijntjes, & Pont, 2019;Kim, Tan, & Chowdhury, 2016;Marlow & Anderson, 2013;Marlow, Kim, & Anderson, 2012). ...
Full-text available
The roughness of a shiny surface determines how sharp the reflected image of the surroundings is, and thus whether the surface appears highly glossy or more or less matte. In a matching experiment, subjects were asked to reproduce the perceived roughness of a given surface (standard) in a comparison stimulus (match), where the standard and the match could differ in both shape and illumination. To compare the effect of the reflection model on the accuracy of the settings, this was done for two different reflectance models (bidirectional reflectance distribution function [BRDF]). The matching errors were smaller, that is, the constancy under shape and illumination changes higher, when Fresnel effects were physically correctly reproduced in the reflectance model (Fresnel-BRDF) than when this was not the case (Ward-BRDF). The subjects' settings in the experiment can be predicted very well by two image statistics, one of which is based on the mean edge strength and the other on a local discrete cosine transform. In particular, these predictions also reflect the empirically observed advantage of the Fresnel-BRDF. These results show that the constancy of perceived roughness across context changes may depend on the BRDF used, with Fresnel effects playing a significant role. The good prediction of subjects' settings using the two image statistics suggests that local brightness variance, which affects both image statistics, can be used as a valid cue for surface roughness.
... Because the "tail" of the luminance histogram typically corresponds to specular highlights, such a claim suggests the importance of specular highlights or high luminance regions on the object surface for glossiness perception. This is consistent with previous studies supporting the importance of specular highlights in glossiness perception (Beck and Prazdny, 1981;Ferwerda et al., 2001;Marlow et al., 2012;Marlow and Anderson, 2013;van Assen et al., 2016;Di Cicco et al., 2019). For instance, Marlow et al. (2012) suggested that the coverage, contrast, sharpness, and depth of the specular highlights are informative predictors of human glossiness perception based on the results indicating that perceived gloss can be accurately predicted from such features, at least for their stimulus set. ...
Full-text available
It has been suggested that luminance edges in retinal images are potential cues for glossiness perception, particularly when the perception relies on low-luminance specular regions. However, a previous study has shown only statistical correlations between luminance edges and perceived glossiness, not their causal relations. Additionally, although specular components should be embedded at various spatial frequencies depending on the micro-roughness on the object surface, it is not well understood what spatial frequencies are essential for glossiness perception on objects with different micro-roughness. To address these issues, we examined the impact of a sub-band contrast enhancement on the perceived glossiness in the two conditions of stimuli: the Full condition where the stimulus had natural specular components and the Dark condition where it had specular components only in dark regions. Object images with various degrees of surface roughness were generated as stimuli, and their contrast was increased in various spatial-frequency sub-bands. The results indicate that the enhancement of the sub-band contrast can significantly increase perceived glossiness as expected. Furthermore, the effectiveness of each spatial frequency band depends on the surface roughness in the Full condition. However, effective spatial frequencies are constant at a middle spatial frequency regardless of the stimulus surface roughness in the Dark condition. These results suggest that, for glossiness perception, our visual system depends on specular-related information embedded in high spatial frequency components but may change the dependency on spatial frequency based on the surface luminance to be judged.
... In previous work, Beurs' manual has supported the notion that contrast and blurriness, but not coverage of highlights, were the image features used to render the glossiness of grapes in 17th-century paintings. The grapes recipe contained in the manual also confirmed the artistic convention of using white to render highlights, thus providing an example of using key perceptual information to produce an efficient yet effective rendering of material properties (Di Cicco, Wijntjes, & Pont, 2019). We also considered Beurs' recipes for additional insights into the image features and perceptual shortcuts exploited by painters to render translucency and juiciness. ...
Full-text available
Citrus fruits are characterized by a juicy and translucent interior, important properties that drive material recognition and food acceptance. Yet, a thorough understanding of their visual perception is still missing. Using citrus fruits depicted in 17th-century paintings as stimuli, we ran three rating experiments. In Experiment 1, participants rated the perceived similarity in translucency or juiciness of the fruits. In Experiment 2, different groups of participants rated one image feature from a list obtained in a preliminary experiment. In Experiment 3, translucency and juiciness were rated. We constructed two-dimensional perceptual spaces for both material properties and fitted the ratings of the image features into the spaces to interpret them. "Highlights," "peeled side," "bumpiness," and "color saturation" fit the juiciness space best and were high for the highly juicy stimuli. "Peeled side," "intensity of light gradient," "highlights," and "color saturation" were the most salient features of the translucency space, being high for the highly translucent stimuli. The same image features were also indicated in a 17th-century painting manual for material depiction (Beurs, 1692; Beurs, in press). Altogether, we disclosed the expertise of painters with regard to material perception by identifying the image features that trigger a visual impression of juiciness and translucency in citrus fruits.
Gloss is one of the main attributes to describe the appearance of surfaces and objects, as it contributes to the general quality perception. Gloss is a multidimensional quantity of which ‘specular gloss’ is the most commonly applied attribute. Specular gloss meters are standardized and widely used in industry. However, their readings correlate only partially to the general visual gloss impression, which also comprises distinctness-of-the-reflected-image (DOI), haze, contrast and surface-uniformity attributes. This study presents a more profound image-based gloss meter (iGM) which incorporates a CMOS camera detector. This concept is not new, but limited research has been conducted on the inclusion of various image processing evaluations for gloss attributes. The designed iGM is compatible to 60° specular gloss meter standards. The CMOS detector captures the reflected source image, which is processed to measure four perceptual attributes of surface gloss. The obtained results validate the 60° specular gloss evaluation and indicate a promising capability in characterizing DOI, haze, and contrast. Contrast is an important attribute that is not available yet in industrial gloss meters. It is measured using a diffuse aspecular light source. Generally, this iGM maintains the hardware principles of specular gloss meters, while evolving toward a representative gloss perception meter.
Single‐image appearance editing is a challenging task, traditionally requiring the estimation of additional scene properties such as geometry or illumination. Moreover, the exact interaction of light, shape and material reflectance that elicits a given perceptual impression is still not well understood. We present an image‐based editing method that allows to modify the material appearance of an object by increasing or decreasing high‐level perceptual attributes, using a single image as input. Our framework relies on a two‐step generative network, where the first step drives the change in appearance and the second produces an image with high‐frequency details. For training, we augment an existing material appearance dataset with perceptual judgements of high‐level attributes, collected through crowd‐sourced experiments, and build upon training strategies that circumvent the cumbersome need for original‐edited image pairs. We demonstrate the editing capabilities of our framework on a variety of inputs, both synthetic and real, using two common perceptual attributes (Glossy and Metallic), and validate the perception of appearance in our edited images through a user study. Single‐image appearance editing is a challenging task, traditionally requiring the estimation of additional scene properties such as geometry or illumination. We present an image‐based editing method that allows to modify the material appearance of an object by increasing or decreasing high‐level perceptual attributes (e.g. Glossy or Metallic), using a single image as input.
Deep learning has paved the way for strong recognition systems which are often both trained on and applied to natural images. In this paper, we examine the give-and-take relationship between such visual recognition systems and the rich information available in the fine arts. First, we find that visual recognition systems designed for natural images can work surprisingly well on paintings. In particular, we find that interactive segmentation tools can be used to cleanly annotate polygonal segments within paintings, a task which is time consuming to undertake by hand. We also find that FasterRCNN, a model which has been designed for object recognition in natural scenes, can be quickly repurposed for detection of materials in paintings. Second, we show that learning from paintings can be beneficial for neural networks that are intended to be used on natural images. We find that training on paintings instead of natural images can improve the quality of learned features and we further find that a large number of paintings can be a valuable source of test data for evaluating domain adaptation algorithms. Our experiments are based on a novel large-scale annotated database of material depictions in paintings which we detail in a separate manuscript.
Full-text available
Art history claims that Western art shows light from the top left, which has been repeatedly shown with narrow image sets and simplistic research methods. Here we employed a set of 10,000 pictures for which participants estimated the direction of light plus their confidence of estimation. From 1420 A.D., the onset of Early Renaissance, until 1900 A.D., we revealed a clear preference for painting light from the top left—within the same period, we observed the highest confidence in such estimations of the light source. One sentence summary This study demonstrates a robust preference for painting light from the top left for Western art history, starting from Early Renaissance until 1900.
Full-text available
We examined in which way gradual changes in the geometric structure of the illumination affect the perceived glossiness of a surface. The test stimuli were computer-generated three-dimensional scenes with a single test object that was illuminated by three point light sources, whose relative positions in space were systematically varied. In the first experiment, the subjects were asked to adjust the microscale smoothness of a match object illuminated by a single light source such that it has the same perceived glossiness as the test stimulus. We found that small changes in the structure of the light field can induce dramatic changes in perceived glossiness and that this effect is modulated by the microscale smoothness of the test object. The results of a second experiment indicate that the degree of overlap of nearby highlights plays a major role in this effect: Whenever the degree of overlap in a group of highlights is so large that they perceptually merge into a single highlight, the glossiness of the surface is systematically underestimated. In addition, we examined the predictability of the smoothness settings by a linear model that is based on a set of four different global image statistics.
Full-text available
This article discusses the technical examination of five flower and fruit still life paintings by the seventeenth century artist Jan Davidsz. de Heem (1606-1684). The painter is known for his meticulously composed and finely detailed still life paintings and is a master in imitating the surface textures of various fruits, flowers, and objects. Macro X-ray fluorescence (MA-XRF) scanning experiments were supplemented with a study of paint cross-sections and contemporary art technical sources with the aim of reconstructing the complex build-up of the overall lay-in of the composition and individual subjects. MA-XRF provided information on the distribution of key chemical elements present in painting materials and made it possible to recapture evidence of the different phases in the artist's working methods: from the application of the ground layers, to De Heem's characteristic oval-shaped underpaintings, and finally, the superposition of multiple paint layers in the working up of the paintings. SEM-EDX analysis of a limited number of paint cross-sections complemented the chemical images with local and layer-specific information on the microscale, providing more accuracy on the layer sequence and enabling the study of elements with a low atomic number for which the non-invasive technique is less sensitive. The results from this technical examination were in addition compared with recipes and paint instructions, to obtain a better understanding of the relation between the general practice and actual painting technique of Jan Davidsz. de Heem. Ultimately, this combined approach uncovered new information on De Heem's artistic practice and demonstrated the complementarity of the methods.
Full-text available
Most previous work on gloss perception has examined the strength and sharpness of specular reflections in simple bidirectional reflectance distribution functions (BRDFs) having a single specular component. However, BRDFs can be substantially more complex and it is interesting to ask how many additional perceptual dimensions there could be in the visual representation of surface reflectance qualities. To address this, we tested materials with two specular components that elicit an impression of hazy gloss. Stimuli were renderings of irregularly shaped objects under environment illumination, with either a single Ward specular BRDF component (Ward, 1992), or two such components, with the same total specular reflectance but different sharpness parameters, yielding both sharp and blurry highlights simultaneously. Differently shaped objects were presented side by side in matching, discrimination, and rating tasks. Our results show that observers mainly attend to the sharpest reflections in matching tasks, but they can indeed discriminate between single-component and twocomponent specular materials in discrimination and rating tasks. The results reveal an additional perceptual dimension of gloss-beyond strength and sharpness- akin to ''haze gloss'' (Hunter & Harold, 1987). However, neither the physical measurements of Hunter and Harold nor the kurtosis of the specular term predict perception in our tasks. We suggest the visual system may use a decomposition of specular reflections in the perception of hazy gloss, and we compare two possible candidates: a physical representation made of two gloss components, and an alternative representation made of a central gloss component and a surrounding halo component.
Full-text available
We experience vivid percepts of objects and materials despite complexities in the way images are structured by the interaction of light with surface properties (3D shape, albedo, and gloss or specularity). Although the perception of gloss (and lightness) has been argued to depend on image statistics (e.g., sub-band skew), studies have shown that perceived gloss depends critically on the structure of luminance variations in images. Here, we found that separately adapting observers to either positive or negative skew generated declines in perceived gloss, contrary to the predictions of theories involving image statistics. We also found similar declines in perceived gloss following adaptation to contours geometrically correlated with sharp specular edges. We further found this aftereffect was stronger when contour adaptors were aligned with specular edges compared with adaptation to the same contours rotated by 90°. These findings support the view that the perception of gloss depends critically on the visual system’s ability to encode specular edge structure and not image skew.
Full-text available
In this paper we will present an overview of our research into perception and biologically inspired modeling of illumination (flow) from 3D textures and the influence of roughness and illumination on material perception. Here 3D texture is defined as an image of an illuminated rough surface. In a series of theoretical and empirical papers we studied how we can estimate the illumination orientation (in the image plane) from 3D textures of globally flat samples. We found that the orientation can be estimated well by humans and computers using an approach based on second order statistics. This approach makes use of the dipole-like structures in 3D textures that are the results of illumination of bumps/throughs. For 3D objects, the local illumination direction varies over the object, resulting in surface illuminance flow. This again results in image illuminance flow in the image of a rough 3D object: the observable projection in the image of the field of local illumination orientations. Here we present results on image illuminance flow analysis for images from the Utrecht Oranges database, the Curet database and two vases. These results show that the image illuminance flow can be estimated robustly for various rough materials. In earlier studies we have shown that the image illuminance flow can be used to do shape and illumination inferences. Recently, in psychophysical experiments we found that adding 3D texture to a matte spherical object improves judgments of the direction and diffuseness of its illumination by human observers. This shows that human observers indeed use the illuminance flow as a cue for the illumination.
The wax bloom is responsible of the fruit is responsible for the visible quality of blueberries. This study aimed to investigate a new technology using the effect of polishing on micromorphology, wax content and weight loss of blueberries. Luster sensor (type CZ-H72, Keyence, Japan) technology was used to assess glossiness of polished blueberries compared with berries with a natural (unpolished) wax layer during 9 days after harvest. Blueberries were rubbed twice by hand within a soft microfibre tissue to obtain polished fruit. Unpolished blueberries contained ca. 120 μg wax cm-2, which was reduced by ca. 22% to ca. 95 μg cm-2 by polishing. This reduction was associated with an increase in luster levels from ca. 65 to 80 a.u.. Weight loss was larger from polished than from unpolished blueberries with a concomitant 40% increase in luster levels from 60 to 85 a.u. in polished fruit. Luster levels sharply decreased from 85 a.u. in the first 5 days after harvest and then leveled off to remain almost constant at ca. 20 a.u. with significantly larger values for polished blueberries of ca. 30 a.u. with a larger magnitude of glossiness. Overall, luster sensor technology may offer a new effective, affordable, possibly portable, non-destructive technique to assess glossiness or other surface features in real time for classifying not only blueberry, but also other waxy fruit such as aubergine/eggplant, plum, Juniperus, blue grape berry etc..
Gloss perception strongly depends on the three-dimensional shape and the illumination of the object under consideration. In this study we investigated the influence of the spatial structure of the illumination on gloss perception. A diffuse light box in combination with differently shaped masks was used to produce a set of six simple and complex highlight shapes. The geometry of the simple highlight shapes was inspired by conventional artistic practice (e.g., ring flash for photography, window shape for painting and disk or square for cartoons). In the box we placed spherical stimuli that were painted in six degrees of glossiness. This resulted in a stimulus set of six highlight shapes and six gloss levels, a total of 36 stimuli. We performed three experiments of which two took place using digital photographs on a computer monitor and one with the real spheres in the light box. The observers had to perform a comparison task in which they chose which of two stimuli was glossiest and a rating task in which they rated the glossiness. The results show that, perhaps surprisingly, more complex highlight shapes were perceived to produce a less glossy appearance than simple highlight shapes such as a disk or square. These findings were confirmed for both viewing conditions, on a computer display and in a real setting. The results show that variations in the spatial structure of "rather simple" illumination of the "extended source" type highlight influences perceived glossiness.