ArticlePDF Available

Modeling first impressions from highly variable facial images

Authors:

Abstract and Figures

First impressions of social traits, such as trustworthiness or dominance, are reliably perceived in faces, and despite their questionable validity they can have considerable real-world consequences. We sought to uncover the information driving such judgments, using an attribute-based approach. Attributes (physical facial features) were objectively measured from feature positions and colors in a database of highly variable "ambient" face photographs, and then used as input for a neural network to model factor dimensions (approachability, youthful-attractiveness, and dominance) thought to underlie social attributions. A linear model based on this approach was able to account for 58% of the variance in raters' impressions of previously unseen faces, and factor-attribute correlations could be used to rank attributes by their importance to each factor. Reversing this process, neural networks were then used to predict facial attributes and corresponding image properties from specific combinations of factor scores. In this way, the factors driving social trait impressions could be visualized as a series of computer-generated cartoon face-like images, depicting how attributes change along each dimension. This study shows that despite enormous variation in ambient images of faces, a substantial proportion of the variance in first impressions can be accounted for through linear changes in objectively defined features.
Summary of methods. (A) For each of 1,000 highly variable face photographs, judgments for first impressions of 16 social traits ("traits") were acquired from human raters. These 16 trait scores were reduced to scores on three underlying dimensions ("factors") by means of factor analysis [see Sutherland et al. (21) and Methods for further details]. (B) Faces were delineated by the placement of 179 fiducial points outlining the locations of key features. The fiducial points were used to calculate 65 numerical attributes ("attributes") summarizing local or global shape and color information (e.g., "lower lip curvature, nose area"). Neural networks were trained to predict factor scores based on these 65 attributes (Table S1 describes the fiducial points and attributes in detail). The performance of the trained networks (i.e., the proportion of variance in factor scores that could then be explained by applying the network to untrained images) was evaluated using a 10-fold cross-validation procedure. (C) Using the full set of images a separate network was trained to reproduce 393 geometric properties (e.g., fiducial point coordinates) from the 65 image attributes (pixel colors were recovered directly from the attribute code). This process permits novel faces to be reconstructed accurately on the basis of the attributes illustrating the information available to the model (see Fig. S2 for additional examples). (D) A cascade of networks was used to synthesize cartoon face-like images corresponding to specified factor scores. This process entailed training a new network to translate three factor scores into 65 attributes that could then be used as the input to the network shown in C, and generate the face-like image. The social trait impressions evoked by these synthesized face-like images were then assessed by naive raters using methods equivalent to Sutherland et al. (21).
… 
Content may be subject to copyright.
Modeling first impressions from highly variable
facial images
Richard J. W. Vernon, Clare A. M. Sutherland, Andrew W. Young, and Tom Hartley
1
Department of Psychology, University of York, Heslington, York YO10 5DD, United Kingdom
Edited by Susan T. Fiske, Princeton University, Princeton, NJ, and approved July 7, 2014 (received for review May 27, 2014)
First impressions of social traits, such as trustworthiness or
dominance, are reliably perceived in faces, and despite their
questionable validity they can have considerable real-world con-
sequences. We sought to uncover the information driving such
judgments, using an attribute-based approach. Attributes (physi-
cal facial features) were objectively measured from feature
positions and colors in a database of highly variable ambient
face photographs, and then used as input for a neural network
to model factor dimensions (approachability, youthful-attractive-
ness, and dominance) thought to underlie social attributions. A
linear model based on this approach was able to account for
58% of the variance in ratersimpressions of previously unseen
faces, and factor-attribute correlations could be used to rank
attributes by their importance to each factor. Reversing this pro-
cess, neural networks were then used to predict facial attributes
and corresponding image properties from specific combinations of
factor scores. In this way, the factors driving social trait impres-
sions could be visualized as a series of computer-generated car-
toon face-like images, depicting how attributes change along each
dimension. This study shows that despite enormous variation in
ambient images of faces, a substantial proportion of the variance
in first impressions can be accounted for through linear changes in
objectively defined features.
face perception
|
social cognition
|
person perception
|
impression formation
Avariety of relatively objective assessments can be made
upon perceiving an individuals face. Their age, sex, and
often their emotional state can be accurately judged (1). How-
ever, inferences are also made about social traits; for example,
certain people may appear more trustworthy or dominant than
others. These traits can be readfrom a glimpse as brief as
100 ms (2) or less (3), and brain activity appears to track social traits,
such as trustworthiness, even when no explicit evaluation is re-
quired (4). This finding suggests that trait judgments are first
impressions that are made automatically, likely outside of con-
scious control. Such phenomena link facial first impressions
to a wider body of research and theory concerned with inter-
personal perception (5).
For many reasons, including the increasingly pervasive use of
images of faces in social media, it is important to understand how
first impressions arise (6). This is particularly necessary because
although first impressions are formed rapidly to faces, they are by
no means fleeting in their consequences. Instead, many studies
show how facial appearance can affect our behavior, changing the
way we interpret social encounters and influencing their out-
comes. For example, the same behavior can be interpreted as
assertive or unconfident depending on the perceived dominance
of an accompanying face (7), and inferences of competence based
on facial cues have even been shown to predict election results
(8). Nonetheless, support for the validity of these first impres-
sions of faces is inconclusive (915), raising the question of why
we form them so readily.
One prominent theory, the overgeneralization hypothesis,
suggests that trait inferences are overgeneralized responses to
underlying cues. For example, a person may be considered to
have other immature characteristics based on a babyfaced
appearance (16). Exploring this general approach of seeking the
factors that might underlie facial first impressions, Oosterhof
and Todorov (17) found that a range of trait ratings actually seem
to reflect judgments along two near-orthogonal dimensions:
trustworthiness (valence) and dominance. The trustworthiness
dimension appeared to rely heavily on angriness-to-happiness
cues, whereas dominance appeared to reflect facial maturity or
masculinity. The use of such cues by human perceivers can be
very subtle; even supposedly neutral faces can have a structural
resemblance to emotional expressions that can guide trait
judgments (18).
Oosterhof and Todorovs (17) findings imply that trait judg-
ments are likely to be based upon both stable (e.g., masculinity)
and more transient (e.g., smiling) physical properties (attrib-
utes) of an individuals face. However, the trait ratings they
analyzed were derived from constrained sets of photographs or
computer-generated neutral facial images. Although this ap-
proach allows careful control over experimental stimuli, such
image sets do not fully reflect the wide range of variability between
real faces and images of faces encountered in everyday life.
Jenkins et al. (19) introduced the concept of ambient images
to encompass this considerable natural variability. The term
ambient imagesrefers to images typical of those we see every
day. The variability between ambient images includes face-
irrelevant differences in angle of view and lighting, as well as the
range of expressions, ages, hairstyles, and so forth. Such vari-
ability is important: Jenkins et al. (19) and Todorov and Porter
Significance
Understanding how first impressions are formed to faces is
a topic of major theoretical and practical interest that has been
given added importance through the widespread use of
images of faces in social media. We create a quantitative
model that can predict first impressions of previously unseen
ambient images of faces (photographs reflecting the variability
encountered in everyday life) from a linear combination of
facial attributes, explaining 58% of the variance in raters
impressions despite the considerable variability of the photo-
graphs. Reversing this process, we then demonstrate that face-
like images can be generated that yield predictable social trait
impressions in naive raters because they capture key aspects
of the systematic variation in the relevant physical features of
real faces.
Author contribut ions: A.W.Y. and T.H. designed research ; R.J.W.V. and C.A. M.S. per-
formed research; C.A.M.S. contributed new reagents/analytic tools; R.J.W.V. and C.A.M.S.
analyzed data; R.J.W.V., C.A.M.S., A.W.Y., and T.H. wrote the paper; R.J.W.V. contributed
to the development of the model and face coding scheme.; C.A.M.S. contributed to the
design of the validation experiment.; A.W.Y. conceived the study; and T.H. conceived the
study and contributed to modeling methods.
The authors declare no conflict of interest.
This article is a PNAS Direct Submission.
Freely available online through the PNAS open access option.
1
To whom correspondence should be addressed. Email: tom.hartley@york.ac.uk.
This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.
1073/pnas.1409860111/-/DCSupplemental.
www.pnas.org/cgi/doi/10.1073/pnas.1409860111 PNAS Early Edition
|
1of9
PSYCHOLOGICAL AND
COGNITIVE SCIENCES
PNAS PLUS
(20) found that first impressions can show as much variability be-
tween photographs of the same individual as between photographs
of different individuals. Hence, the normally discounted range of
everyday image variability can play a substantial role in facial
judgments.
An important demonstration of the potential value of the
ambient images approach is that Sutherland et al. (21) found
that a third factor (youthful-attractiveness) emerged when ana-
lyzing first impressions of highly variable, ambient images, in
addition to Oosterhof and Todorovs (17) trustworthiness and
dominance dimensions. This finding does not of course undermine
the value of Oosterhof and Todorovs model, but shows how it can
be extended to encompass the range of natural variability.
The critical test for any model of facial first impressions is
therefore to capture these impressions from images that are as
variable as those encountered in daily life. Because human
perceivers do this so easily and with good agreement, it seems
likely that the underlying cues will be closely coupled to image
properties, but previous studies have not been able to capture
the physical cues underlying a wide range of trait impressions
across a wide range of images.
Here, we offer a novel perspective, modeling the relationships
between the physical features of faces and trait factor dimensions
for a sample of 1,000 ambient facial images. The models take the
form of artificial neural networks, in which simple processing
elements represent both facial features and social trait dimen-
sions and compute the translation between these representations
by means of weighted connections.
With this general technique we were able to show that a linear
network can model dimensions of approachability, youthful-
attractiveness, and dominance thought to underlie social attribu-
tions (21), and critically that its performance generalizes well to
previously untrained faces, accounting on average for 58% of the
variance in ratersimpressions. To validate this method, we then
reversed the approach by using a neural network to generate
facial feature attributes and corresponding image properties
expected to produce specific trait impressions. In this way, the
factors driving first impressions of faces could be visualized as
a series of computer-generated cartoon images, depicting how
attributes change along each dimension, and we were able to test
the predictions by asking a new group of human raters to judge
the social traits in the synthesized images.
Our approach offers a powerful demonstration of how ap-
parently complex trait inferences can be based on a relatively
simple extracted set of underlying image properties. Further-
more, by grounding the approach in such physical properties, it is
possible to directly relate the explained variance back to the fea-
tures that are driving the model, allowing us to quantify the ways
in which physical features relate to underlying trait dimensions.
Procedure and Results
Our overall aim was to characterize the relationship between
physical features of faces and the perception of social traits. Fig.
1 provides an overview of the methods. To quantify first
impressions, we used a database of 1,000 highly variable ambient
face photographs of adults of overall Caucasian appearance,
which had been previously rated for a variety of social traits by
observers with a Western cultural background (2124). We fo-
cused on first impressions of Caucasian-looking faces by partic-
ipants with a Western upbringing to avoid possible confounding
influences of other-raceeffects (25, 26). As in previous work
(21), we used factor analysis to reduce these ratings of perceived
traits to three factors, which together account for the majority of
the variance in the underlying trait judgments: approachability
[corresponding closely to trustworthiness/valence in Oosterhof
and Todorovs model (17)], youthful-attractiveness, and domi-
nance (Fig. 1A). In this way, each of the 1,000 ambient face
photographs could be given a score reflecting its loading on each
of the three underlying factors.
To identify the corresponding objective features of the same
faces, we calculated a range of attributes reflecting physical
properties (Fig. 1B). Consistent with our approach of using un-
constrained ambient images, we sought to include a wide range
of attributes without predetermining which might prove useful
on any theoretical grounds. We first defined the locations of 179
fiducial points used in the Psychomorph program (27) on each
face image (Fig. S1), thereby outlining the shapes and positions
of internal and external facial features. Then, using the coor-
dinates of these fiducials and the brightness and color properties
of pixels within regions defined by them, we calculated a range of
image-based measures summarizing physical characteristics of
each face (see Methods for further details). These included
measures related to the size and shape of the face as a whole
(e.g., head width), individual facial features (e.g., bottom lip cur-
vature), their spatial arrangement (e.g., mouth-to-chin distance),
the presence/absence of glasses and facial hair, and information
about the texture and color of specified regions (e.g., average
hue of pixels within polygons defined by fiducial points around
the irises). Where applicable, size and area values were first
standardized by dividing by a proxy for apparent head size.
To characterize the relationship between these physical
measures and the social trait ratings, we used a subset of the face
images to train artificial neural networks that were then used to
predict the trait factor scores of novel(untrained) faces from
their physical attributes alone. A variety of different network
architectures were evaluated. These included a simple linear
architecture (with factor score predictions being modeled as
a weighted combination of attributes plus constant) and a range
of nonlinear architectures (i.e., neural networks including a non-
linear hidden layer containing varying numbers of hidden units).
The former (linear architecture) approach is equivalent to mul-
tiple linear regression and is capable of characterizing univariate
linear relationships between individual attributes and factor scores.
The latter (nonlinear, with hidden units) approach is in principle
capable of exploiting nonlinear andmultivariaterelationships.
The neural network approach is beneficial in that it allows us
to evaluate both linear and nonlinear models using a closely
matched 10-fold cross-validation procedure. For this procedure
the set of 1,000 faces was divided into 10 subsamples, each
containing the physical attributes derived from 100 images to-
gether with the corresponding factor scores. For each foldone
set of 100 image attributes was reserved for use as test cases, and
the remaining 900 were used to train (800) and then validate
(100) a freshly initialized network. The network was trained to fit
the physical measures to the factor scores from the training
cases. After training, the predicted factor scores for the reserved
100 untrained faces were computed. The predictions were then
set aside and the process repeated until all 1,000 images had
been used as test cases. The correlation between predicted and
observed factor scores for the full set of untrained images was
then calculated. The procedure was repeated 100 times (using
random initialization of the networks and sampling of training
and test images), with performance calculated as the average
correlation over all iterations.
The average correlations between the network predictions and
actual factor scores were r
approachability
=0.90, r
youthful-attractiveness
=
0.70, and r
dominance
=0.67 (all P<0.001) (Fig. 2). Across iter-
ations of training, the SDs of these correlations were all less than
0.01, showing consistent network performance regardless of the
particular selection of images chosen for training/validation and
testing. Overall, the linear model accounted for a substantial
proportion of the variance for all three trait factors (58% on
average). These findings therefore demonstrate that linear
properties of objective facial attributes can be used to predict
trait judgments with considerable accuracy. Indeed, predicted
2of9
|
www.pnas.org/cgi/doi/10.1073/pnas.1409860111 Vernon et al.
factor scores based on unseen test cases were only marginally less
accurate than the fit obtained for training cases (r
approachability
=
0.92; r
youthful-attractiveness
=0.75; r
dominance
=0.73), indicating that
the linear model generalizes very well to novel cases.
Because of their greater ability to capture nonlinear and
multivariate relationships, we might intuitively have expected
nonlinear architectures to outperform linear models. Perhaps
surprisingly, however, we found no additional benefits of a non-
linear approach. For example, using our standard training and
validation procedures, a network with five nonlinear hidden units
generated correlations of r
approachability
=0.88, r
youthful-attractiveness
=
0.65, r
dominance
=0.62 (all P<0.001). Furthermore, there were
significant negative correlations between the number of hidden
units and performance for all three factors (r
approachability
=0.96,
BA
16 Traits 3 Factors
Factor Analysis
Acquire human trait
impression ratings
+
Supervised Learning
& Cross-Validation
Delineate/extract
attributes
65 Attributes 3 Factors
D
Specify target
factor scores
Pre- Trained
(See C*)
Supervised
Learning
Factor Loadings
(See A+)
Acquire human trait
impression ratings
Synthesize
3 Factors 65 Attributes 393 Image Properties
9 Traits 3 Factors
C
65 Attributes 393 Image Properties
Supervised
Learning*
Delineate/extract
attributes
Reconstruct image
properties
Fig. 1. Summary of methods. (A) For each of 1,000 highly variable face photographs, judgments for first impressions of 16 social traits (traits) were ac-
quired from human raters. These 16 trait scores were reduced to scores on three underlying dimensions (factors) by means of factor analysis [see Sutherland
et al. (21) and Methods for further details]. (B) Faces were delineated by the placement of 179 fiducial points outlining the locations of key features. The
fiducial points were used to calculate 65 numerical attributes (attributes) summarizing local or global shape and color information (e.g., lower lip cur-
vature, nose area). Neural networks were trained to predict factor scores based on these 65 attributes (Table S1 describes the fiducial points and attributes in
detail). The performance of the trained networks (i.e., the proportion of variance in factor scores that could then be explained by applying the network to
untrained images) was evaluated using a 10-fold cross-validation procedure. (C) Using the full set of images a separate network was trained to reproduce 393
geometric properties (e.g., fiducial point coordinates) from the 65 image attributes (pixel colors were recovered directly from the attribute code). This process
permits novel faces to be reconstructed accurately on the basis of the attributes illustrating the information available to the model (see Fig. S2 for additional
examples). (D) A cascade of networks was used to synthesize cartoon face-like images corresponding to specified factor scores. This process entailed training
a new network to translate three factor scores into 65 attributes that could then be used as the input to the network shown in C, and generate the face-like
image. The social trait impressions evoked by these synthesized face-like images were then assessed by naive raters using methods equivalent to Sutherland
et al. (21).
Vernon et al. PNAS Early Edition
|
3of9
PSYCHOLOGICAL AND
COGNITIVE SCIENCES
PNAS PLUS
r
youthful-attractiveness
=0.98, r
dominance
=0.97, all P<0.001). It
seems that any nonlinear or multivariate relationships that the
more complex architectures are able to exploit in fitting the
training data, do not generalize. Instead, nonlinear network per-
formance for untrained test cases suffers from overfitting. Im-
portantly then, the critical relationships in the data are largely
captured by the simple linear model, which generalizes well to the
new cases.
The fact that the linear model works so well allows us to
quantify which physical features are correlated with each social
trait dimension. Table 1 summarizes statistically significant
associations (Pearson correlations surviving Bonferroni correc-
tion for multiple comparisons) between physical attributes and
factor scores (a full description and numerical key to all 65
attributes is provided in Table S1).
The most striking thing in Table 1 is that almost all attributes
we considered are significantly correlated with one or more of
the dimensions of evaluation. It is clear that social traits can be
signaled through multiple covarying cues, and this is consistent
with findings that no particular region of the face is absolutely
essential to making reliable social inferences (24).
That said, the substantial roles of certain types of attribute for
each dimension also emerge clearly from closer inspection of
Table 1. The five features that are most strongly positively cor-
related with the approachability dimension are all linked to the
mouth and mouth shape (feature #25 mouth area, #26 mouth
height, #29 mouth width, #30 mouth gap, #32 bottom lip
curve), and this is consistent with Oosterhof and Todorovs (17)
observation that a smiling expression is a key component of an
impression of approachability. Four of the five features that are
most strongly positively correlated with the youthful-attractive-
ness dimension relate to the eyes (feature #11 eye area, #12 iris
area, #13 eye height, #14 eye width), in line with Zebrowitz
et al.s (16) views linking relatively large eyes to a youthful ap-
pearance. In Oosterhof and Todorovs (17) model the dominance
dimension is linked to stereotypically masculine appearance, and
here we find it to be most closely correlated with structural fea-
tures linked to masculine face shape (feature #8 eyebrow height,
#35 cheek gradient, #36 eye gradient) and to color and texture
differences that may also relate to masculinity (28) or a healthy or
tanned overall appearance (feature #49 skin saturation, #62 skin
value variation).
Although this agreement between the features, which we
found to be most closely linked to each dimension and theo-
retical approaches to face evaluation, is reassuring, it is none-
theless based on correlations, and correlational data are of
course notoriously susceptible to alternative interpretations. We
therefore sought to validate our approach with a strong test. The
largely linear character of the mapping we have identified
implies that it might be possible to reverse-engineer the process,
using trait-factor scores as inputs (instead of outputs) to a neural
network that will generate 65 predicted features from the input
combination of factor scores. From these 65 attributes, the
requisite image properties can then be recovered and used to
reconstruct a face-like image (Fig. 1). The critical test is then
whether the reconstructed image exemplifies the intended social
traits. This process provides us with a way to visualize the pat-
terns of physical change that are expected to drive the perception
of specific social traits, and to test the validity of these pre-
dictions with naive human raters.
We carried out this process in three steps. We first created
a linear model, allowing us to generate physical attribute scores
(including pixel colors) characteristic of specific combinations of
social trait judgments. We then created a linear model relating
attribute scores to normalized image coordinates, allowing us to
reconstruct face-like images from specified attribute scores (see
Methods and Fig. 1Cfor details). We then combined these
models to reconstruct faces expected to elicit a range of social
trait judgments along each dimension (see Fig. 3 for examples),
and obtained a new set of ratings of these images, which we
compared with the models predictions (Fig. 1D).
In all cases the predicted scores on a given dimension corre-
lated significantly with the new obtained ratings on that di-
mension (Table 2), showing that the intended manipulation of
each dimension was effective. However, it is also evident from
Table 2 that the dimensional manipulations were not fully in-
dependent from each other, as would be expected from the fact
that many features are correlated with more than one dimension
evident in Table 1. Nonetheless, for both approachability and
dominance, the magnitude of the correlation of ratings with the
corresponding predicted trait dimension was significantly greater
than the correlation with either of the remaining dimensions,
showing clear discriminant validity (all P<0.011) (see Methods
for details). For the youthful-attractiveness dimension, the
magnitude of the correlation between predicted youthful-
attractiveness and ratings of youthful-attractiveness was greater
than that of its correlation with ratings of approachability
(P<0.001), and its correlation with dominance approached
statistical significance (P=0.081).
In sum, the generated faces were successful in capturing the
key features associated with each dimension. Furthermore, the
faces strikingly bear out the conclusions reached from the pat-
tern of feature-to-dimension correlations reported in Table 1.
The multiple cues for each dimension are clearly captured in the
cartoon faces, and the importance of the mouth to approach-
ability, the eyes to youthful-attractiveness, and the masculine
face shape and change in skin tone for dominance are all evident.
Increased youthful-attractiveness is also apparently linked to
a more feminizedface shape in Fig. 3. This result was in fact
also apparent in Table 1, where the jaw heightfeature (no. 21
in Table 1) was the among the top 5positive correlations
with youthful-attractiveness (together with the four eye-related
features we listed previously), and the other jawline features
(2224) were all in the top 11.
General Discussion
To our knowledge, we have shown for the first time that trait
dimensions underlying facial first impressions can be recovered
from hugely variable ambient images based only on objective
measures of physical attributes. We were able to quantify the
relationships between those physical attributes and each di-
mension, and we were able to generate new face-like repre-
sentations that could depict the physical changes associated with
each dimension.
The fact that our results are consistent with a number of
previous studies based on ratings rather than physical measures
suggests that this approach has successfully captured true changes
in each dimension. To validate this claim, we first trained a sepa-
rate model to reconstruct face-like images capturing the relevant
featural variation in our ambient image database. We then gen-
erated images whose features varied systematically along each
dimension and demonstrated that these yield predictable social
trait impressions in naive raters.
Critically, our approach has been based entirely on highly
variable ambient images. The results did not depend in any way
on imposing significant constraints on the images selected, or on
any preconceived experimental manipulation of the data. Faces
were delineated, and a wide range of attributes chosen/calcu-
lated, with no a priori knowledge of how individual faces were
judged, or which attributes might be important. This approach
minimizes any subjectivity introduced by the researcher, and is as
close to a double-blind design as can be achieved. The fact that
the findings are based on such a diverse set of images also lends
considerable support to both the replicated and novel findings
described above. Furthermore, the approach used allowed us
explore attribute-factor relationships.
4of9
|
www.pnas.org/cgi/doi/10.1073/pnas.1409860111 Vernon et al.
Oosterhof and Todorovs (17) demonstration that first
impressions of many traits can be encompassed within an over-
arching dimensional model offered an important and elegant
simplification of a previously disparate set of studies, and helped
demystify how we as humans seem capable of reliably making
such an extraordinary range of social inferences. Our findings
take this demystification a significant step further by showing
that these dimensions of evaluation can be based on simple
linear combinations of features. Previous studies with computer-
generated stimuli had also shown that specific linear changes in
a morphable computer model can capture such dimensions (17,
29). Achieving an equivalent demonstration here is noteworthy
because the features present in the ambient images were not
predetermined or manipulated on theoretical grounds and be-
cause their highly varied nature will clearly affect the reliability
of individual feature-trait associations; unnatural lighting can
compromise skin-tone calculations, angle of view affects the
perceived size and the retinal shape of many features, and so on.
It is therefore particularly impressive that our approach was able
to capture the majority of the variance in ratings despite these
potential limitations in individual attribute calculations. Part of
the reason for this result surely lies in the point emphasized by
Bruce (30) and by Burton (31) for the related field of face rec-
ognition: that naturally occurring image variations that are usu-
ally dismissed as noisecan actually be a useful pointer that
helps in extracting any underlying stability.
A question for future research concerns the extent to which
the models success in accounting for social trait impressions is
dependent on the particular selection of attributes we used. Our
65 input features were intended to provide a reasonably com-
plete description of the facial features depicted in each image.
The success of these features is demonstrated by our capacity to
reconstruct distinctive facsimiles of individual faces based only
on the attributes (e.g., Fig. 1C; see Fig. S2 for further examples).
In generating these attributes, our approach was to establish
numerical scores that would, where possible, relate to a list of
characteristics we could label verbally based on the Psychomorph
template (Fig. S1 and Table S1). In line with the overarching
ambient image approach, our strategy was to be guided by the
data as to the role of different attributes, and thus we included
the full set in our analyses. Given the linearity we found, though,
we broadly expect that any similarly complete description of the
relevant facial features should yield similar results. However, it is
also likely that our attribute description is overcomplete in the
sense that there is substantial redundancy between the attributes.
Because of intercorrelations between features, it might well be
possible to achieve comparable performance with fewer, or-
thogonal, components, but these would be much harder to in-
terpret in terms of individual verbally labeled cues.
Although the current model already includes a fairly detailed
representation of the geometry of shape cues, the textural in-
formation we incorporate is less fine-grained, and an obvious
development of our approach will be to increase the resolution
of this aspect of the model, which may yield some improvement
in its performance and will potentially allow for much more re-
alistic reconstructions. For the present purposes, though, we
consider it a strength that a relatively simple representation can
capture much of the underlying process of human trait attribution.
Another important issue to which our approach can be
addressed in future concerns the extent to which social trait
impressions may depend on specific image characteristics, such
as lighting and camera position, changeable properties of an
individuals face, such as pose and expression, or alternatively,
more stable characteristics determined by the faces underlying
structure. It is important to note that the former, variable fea-
tures are potentially specific to each image and therefore even to
different images of the same individual. This critical distinction
between individual identities and specific photographs has a long
history in work on face perception (32, 33), and indeed recent
work (19, 20) has demonstrated clearly that intraindividual and
image cues play a role in determining social trait judgments
alongside any interindividual cues. Our results are consistent
with this in demonstrating that changeable features of the face
(such as head tilt and bottom lip curvature) covary reliably with
social trait impressions, but our approach could also be extended
to allow estimation of the relative contributions of these differ-
ent contributory factors.
Observed Dominance Score
Predicted Dominance Score
Observed Approachability Score
Predicted Approachability Score
Observed Youthful-Attractiveness Score
Predicted Youthful-Attractiveness Score
-1.0 0
-
1.00
-1.0 0 1.00
r = .67
-
-1.0 0
1.00
1.00 1.00
r = .90
-
-
r = .70
1.00
1.00
-1.0 0 1.00
A
B
C
Fig. 2. Scatterplots indicating the correlations between experimentally
derived factor scores (from human raters) with the corresponding pre-
dictions, for untrained images (see Methods for details), derived from a lin-
ear neural network (as illustrated in Fig. 1B). Each point (n=1,000, for all
axes) represents the observed and predicted ratings for a distinct face image
in our database. Both experimental and predicted scores have been scaled
into the range (1:1) (A) approachability, (B) youthful-attractiveness, or (C)
dominance.
Vernon et al. PNAS Early Edition
|
5of9
PSYCHOLOGICAL AND
COGNITIVE SCIENCES
PNAS PLUS
Our methods provide a means to estimate first-impression
dimensions objectively from any face image and to generate face-
like images varying on each dimension. These are significant
steps that offer substantial benefits for future research. Our
results are also of practical significance (e.g., in the context of
social media) because we have shown how images of a given
Table 1. Significant associations between objective attributes and social trait impressions in
1,000 ambient face photographs
Attribute type Attribute App Yo-At Dom
Head size and posture 01. Head area 0.14
03. Head width 0.14 0.18 0.20
04. Orientation (front-profile) 0.12
05. Orientation (up-down) 0.17 0.28
06. Head tilt 0.19 0.20
Eyebrows 07. Eyebrow area 0.16 0.21 0.23
08. Eyebrow height 0.15 0.33 0.27
09. Eyebrow width 0.22 0.12
10. Eyebrow gradient 0.31 0.15
Eyes 11. Eye area 0.26 0.40 0.22
12. Iris area 0.20 0.41 0.31
13. Eye height 0.30 0.39 0.23
14. Eye width 0.13 0.34 0.19
15. % Iris 0.31 0.24
Nose 16. Nose area 0.26 0.14
17. Nose height 0.24
18. Nose width 0.45 0.16
19. Nose curve 0.37
20. Nose flare 0.37
Jawline 21. Jaw height 0.17 0.35
22. Jaw gradient 0.18 0.33
23. Jaw deviation 0.25 0.14
24. Chin curve 0.18 0.31
Mouth 25. Mouth area 0.69 0.14 0.15
26. Mouth height 0.51 0.15 0.22
27. Top Lip height 0.24 0.24 0.25
28. Bottom lip height 0.35 0.34 0.15
29. Mouth width 0.73
30. Mouth gap 0.71
31. Top lip curve 0.36 0.12
32. Bottom lip curve 0.75
Other structural features 33. Noseline separation 0.22
34. Cheekbone position 0.16
35. Cheek gradient 0.17 0.37
36. Eye gradient 0.23 0.21 0.32
Feature positions 38. Eyebrows position 0.27
39. Mouth position 0.38 0.28
40. Nose position 0.22
Feature spacing 41. Eye separation 0.23 0.21
42. Eyes-to-mouth distance 0.39 0.19
43. Eyes-to-eyebrows distance 0.44
46. Mouth-to-chin distance 0.38 0.13
47. Mouth-to-nose distance 0.60 0.12
Color and texture 49. Skin saturation 0.28
50. Skin value (brightness) 0.13 0.23
51. Eyebrow hue
52. Eyebrow saturation 0.13 0.15
53. Eyebrow value (brightness) 0.13 0.22
55. Lip saturation 0.12 0.19
59. Iris value (brightness) 0.24
60. Skin hue variation 0.21
61. Skin saturation variation 0.22 0.21
62. Skin value variation 0.24 0.25
Other features 63. Glasses 0.26
64. Beard or moustache 0.20 0.24
65. Stubble 0.15 0.24
App, Approachability; Dom, Dominance; Yo-At, Youthful-attractiveness. Significant attribute-factor correla-
tions, after Bonferroni correction (P<0.050/195). Highly significant results (P<0.001/195) are highlighted in
bold. See Table S1 for attribute descriptions.
6of9
|
www.pnas.org/cgi/doi/10.1073/pnas.1409860111 Vernon et al.
individual can be selected on the basis of quantifiable physical
features of the face to selectively convey desirable social trait
impressions.
Methods
Social Trait Judgments. Each of the ambient faces had been rated for 16 social
trait impressions, as reported in previous work (2124). Briefly (Fig. 1A), each
trait was rated for the full set of 1,000 faces by a minimum of six in-
dependent judges using a seven-point Likert scale (for example, attractive-
ness: 1 =very unattractive, 7=very attractive). All traits had acceptable
interrater reliability (Cronbachsα>0.70). The means of the different raters
scores for each face and each trait were subjected to principal axis factor
analysis with orthogonal rotation, yielding factor scores (AndersonRubin
method, to ensure orthogonality) for each of the previously identified
dimensions (approachability, youthful-attractiveness, and dominance) (21).
Delineation and Quantification of Facial Features. The shape and internal
features of each face were identified by manually placing 179 fiducial points
onto each image using PsychoMorph (27). The initial delineation was
checked visually by two experimenters, with further checking of the orga-
nization of fine-scale features using a custom Matlab script that identified
errors that would not be found by visual inspection (e.g., incorrect se-
quencing of the fiducials). To facilitate the modeling of image shapes, the
resulting 2D fiducials were then rotated to a vertical orientation such that
the centroids of left- and right-sided points were level.
Sixty-five attributes were derived using these coordinates. These attributes
corresponded to a range of structural, configurational, and featural mea-
surements (see Table S1 for full details). For example, head widthwas
calculated as the mean horizontal separation between the three leftmost
and three rightmost points; bottom lip curvaturewas calculated by fitting
a quadratic curve through the points representing the edge of the bottom
lip (curvature being indicated by the coefficient of the squared term);
mouth-to-chin distancewas the vertical separation between lowest point
on the lower lip and the lowest point on the chin. Area measurements were
calculated using polygons derived from subsets of the fiducial points. Overall
colors within specified areas were calculated for specified regions (i.e., lips,
iris, skin, eyebrows) by averaging RGB values of pixels within polygons de-
fined by sets of fiducial points, converting these to hue, saturation, value
(HSV) measures, and (for skin pixels) additionally calculating a measure of
dispersion, entropy, for each of the H, S, and V channels (15 texture
attributes in total). A HSV description for color was chosen on the basis that
hue and saturation would be relatively insensitive to the large overall lu-
minance variations in the ambient images. Three Boolean attributes described
the presence of glasses, facial hair (beards and moustaches), or stubble.
The raw attribute values were separately normalized to place them into
a common range necessary for the neural network training. These normal-
ization steps also helped to reduce the impact of image-level nonlinearities in
the raw attributes of the highly varied images.
First, a square-root transform was applied to attributes measuring area.
The overall scale of all geometric measures was normalized by dividing by the
average distance between all possible pairs of points outlining the head (a
robust measure of the size of the head in the 2D image). Finally the resulting
values were scaled linearly into the range (1:1).
The HSV color values were similarly scaled into the range (1:1). As hue is
organized circularly, it was necessary to apply a rotation to the raw hue
values such that the median, reddish, hues associated with typical Caucasian
skin tones were represented by middling values.
Neural Network Training, Validation, and Cross-Validation. Neural networks
were implemented using the MatLab Neural Network toolbox (MathWorks).
For initial modeling of the determinants of social trait judgments, input units
represented physical attributes as described above, with output units rep-
resenting social trait factor scores. For the two-layer (linear) networks, both
input and output units used a linear activation function, and these were
connected in a fully feed-forward manner with each input unit being con-
nected to each output unit (Fig. 1B). For the three-layer (potentially non-
linear) networks an additional intervening layer of hidden units (with
sigmoid activation function) was placed between input and output layers,
such that each input unit was connected to each hidden unit, which was in
turn connected to each output unit.
During training weights were adjusted using the MatLab toolboxsdefault
LevenbergMarquardt algorithm. In essence, the weighted connections be-
tween input and output units were gradually and iteratively varied so as to
minimize (on a least-squares measure) the discrepancy between the models
output and the social trait judgments obtained from human raters.
Training, validation, and 10-fold cross-validation was carried out as de-
scribed above. The 1,000 images were randomly partitioned into 10 discrete
Fig. 3. Synthesized face-like images illustrating the changes in facial features that typify each of the three social trait dimensions. The images were gen-
erated using the methods described in the text and in Fig. 1 Cand D, based on the same attributes as those used to derive social trait predictions in Fig. 2. The
lowend of each dimension is shown at the left end of each row and the highend is at the right. Faces are shown in upright orientation for easy comparison,
but the model also suggests some systematic variation in the tilt of the face (indicated below each image). A sample of such synthesized images was used to validate
the predicted trait impressions in a group of naive human raters (see Table 2 and the text for details). See Movies S1S3 for video animations of these changes.
Vernon et al. PNAS Early Edition
|
7of9
PSYCHOLOGICAL AND
COGNITIVE SCIENCES
PNAS PLUS
sets of 100, with 8 sets used to train the network, a further set used to
determine when training had converged, and the remaining set used to
evaluate the performance of the trained network. The predicted social trait
judgment outputs were noted, and the whole process repeated until each of
the 10 image sets had served as the test. This entire cross-validation procedure
was then repeated 100 times using a new random partitioning of the data to
ensure that the results did not depend on a specific partitioning of the data.
Statistical Analysis. The Pearson correlation between the outputs of the
model to unseen (i.e., untrained) test case images and the corresponding
factor scores provides a measure of how well the model predicts variation in
the human ratings. We calculated these correlations for each of the three
previously identified social trait dimensions (Fig. 2). We also report the corre-
lations (Bonferroni-corrected) between individual attribute scores and each
social trait dimension (Table 1). Note that our analysis suggests that social trait
judgments are determined by multiple small contributions from different
physical attributes, which means that it is impossible to unambiguously de-
termine the contribution of each attribute (multicollinearity), although the
correlations may serve to indicate relationships worthy of further investigation.
Generating Face-Like Images. To address interpretational difficulties arising
from multicollinearity, we reversed the modeling process to generate face-
like images with attributes expected to convey certain social trait impressions.
To solve this engineering problem, we used a cascade of linear networks
trained on the full set of 1,000 images. First (Fig. 1C), a network was trained
to convert physical attributes (as described above) into the coordinates of
fiducial points, which could be rendered using custom MatLab scripts (for
this purpose we subdivided the output units, representing the full set of
fiducial points (Fig. S1), feature centroids, and global rotation, into 19 sub-
networks that were each trained separately for memory efficiency). Then
(Fig. 1D), a new network was trained to generate the attributes corre-
sponding to a specified social trait factor scores [each having been scaled
into the range (1:1)], which then acted as input to the face-rendering
network. The resulting cascade generates synthetic face-like images that are
expected to yield specific social percepts. For example, a face-like image
generated using a trait factor score of (1, 0, 0) is expected to be perceived as
highly approachable (first dimension) but neutral with respect to youthful-
attractiveness (second dimension) and dominance (third dimension).
Validating Synthesized Faces. We tested the validity of the generated face-like
images by having new participants assess them in an online questionnaire,
closely following the procedure in Sutherland et al. (21). Trait ratings (as
a proxy for scores on the three dimensions) were collected for 19 generated
faces that covered the range of potential factor scores [each scaled into the
range (1:1)]. One face represented the neutral point (0, 0, 0), 6 faces rep-
resented the extremes (high and low) of each dimension [e.g., (0.8, 0, 0),
approachable], and 12 faces represented all possible pairs of those extremes
[e.g., (0.8, 0, 0.8) unapproachable, dominant]. We chose a value of 0.8 for
to represent the extreme of each dimension because this was typical of the
scaled social trait factor scores of the most extreme 30 of the 1,000 faces in
our ambient image database.
To evaluate these images, we solicited social trait judgments from 30 naive
participants (15 male, mean age 23.93 y) who took part after consenting
to procedures approved by the ethics committee of the Psychology De-
partment, University of York. All spoke fluent English and were from a cul-
turally Western background.
To generate proxy factor scores for each trait dimension identified in our
earlier factor analysis, we selected the most heavily loading traits and asked
raters to evaluate those traits for each image in the set of synthesized face
images. These ratings were then combined in an average, weighted by the
traits loadings on that dimension. For the approachability dimension raters
assessed each face for smiling,”“pleasantness,and approachability.For
youthful-attractiveness they rated images for attractiveness,”“health,and
age.For dominance they rated dominance,”“sexual dimorphism,and
confidence.
The participants were randomly allocated into three sex-balanced groups,
one per dimension. Each group only rated the three traits making up one
dimension, to avoid the risk of judgments on one factor biasing another. Each
trait was rated in a block of consecutive trials (with random block order), and
within each block the 19 generated faces were randomly presented. Before
rating the actual stimuli, the participants saw six practice images (created
using random factor scores, indistinguishable from actual stimuli). All faces
were rated on a 17 Likert scale with endpoints anchored as previous re-
search (2124).
To assess the correspondence between ratersjudgments and the pre-
dicted factor scores (i.e., those used to synthesize the faces), we determined
the correlation for each pairing of judged and predicted dimensions (Table 2).
Independent ttests were used to compare these correlations at the level
of individual raters. First, we calculated a Spearmans correlation for each
pairing of rated and synthesized trait dimensions, then we used independent
ttests to test the hypothesis that the absolute value of the Fisher transformed
correlations was significantlygreater for the predicted trait dimension than for
each of the other dimensions.
Animations. By generating a series of images with incremental changes along
each dimension, we were also able to create short movies to encapsulate each
dimension in the model (Movies S1S3). As well as the changes already
noted, these movies show that the neural network has also captured su-
perordinate variation resulting from synchronized changes across combi-
nations of features; for example, increased dominance involves raising the
head (as if to look downupon the viewer).
ACKNOWLEDGMENTS. The research was funded in part by an Economic and
Social Research Council Studentship ES/I900748/1 (to C.A.M.S.).
1. Bruce V, Young AW (2012) Face Perception (Psychology Press, London).
2. Willis J, Todorov A (2006) First impressions: Making up your mind after a 100-ms
exposure to a face. Psychol Sci 17(7):592598.
3. Todorov A, Pakrashi M, Oosterhof NN (2009) Evaluating faces on trustworthiness after
minimal time exposure. Soc Cogn 27(6):813833.
4. Engell AD, Haxby JV, Todorov A (2007) Implicit trustworthiness decisions: Auto-
matic coding of face properties in the human amygdala. JCognNeurosci19(9):
15081519.
5. Fiske ST, Cuddy AJC, Glick P (2007) Universal dimensions of social cognition: Warmth
and competence. Trends Cogn Sci 11(2):7783.
6. Perrett DI (2010) In Your Face: The New Science of Human Attraction (Palgrave
Macmillan, London).
7. Hassin R, Trope Y (2000) Facing faces: Studies on the cognitive aspects of physiognomy.
J Pers Soc Psychol 78(5):837852.
8. Todorov A, Mandisodza AN, Goren A, Hall CC (2005) Inferences of competence from
faces predict election outcomes. Science 308(5728):16231626.
9. Snyder M, Tanke ED, Berscheid E (1977) Social perception and interpersonal behaviour:
On the self-fulfilling nature of social stereotypes. J Pers Soc Psychol 35(9):656666.
10. Bond JCF, Berry DS, Omar A (1994) The kernel of truth in judgments of deceptiveness.
Basic Appl Soc Psych 15(4):523534.
11. Zebrowitz LA, Andreoletti C, Collins MA, Lee SY, Blumenthal J (1998) Bright, bad,
babyfaced boys: Appearance stereotypes do not always yield self-fulfilling prophecy
effects. J Pers Soc Psychol 75(5):13001320.
12. Efferson C, Vogt S (2013) Viewing mens faces does not lead to accurate predictions of
trustworthiness. Sci Rep 3:1047.
13. Carré JM, McCormick CM (2008) In your face: Facial metrics predict aggressive be-
haviour in the laboratory and in varsity and professional hockey players. Proc Biol Sci
275(1651):26512656.
Table 2. Spearmans correlations between expected and rated factor scores for synthesized
face-like images
Rated factor scores
Expected factor scores
Approachability Youthful-attractiveness Dominance
Approachability 0.93** 0.03 0.09
Youthful-attractiveness 0.78** 0.56* 0.10
Dominance 0.19 0.53* 0.74**
**P<0.001, *P<0.050.
8of9
|
www.pnas.org/cgi/doi/10.1073/pnas.1409860111 Vernon et al.
14. Stirrat M, Perrett DI (2010) Valid facial cues to cooperation and trust: Male facial
width and trustworthiness. Psychol Sci 21(3):349354.
15. Gómez-Valdés J, et al. (2013) Lack of support for the association between facial shape
and aggression: A reappraisal based on a worldwide population genetics perspective.
PLoS ONE 8(1):e52317.
16. Zebrowitz LA, Fellous JM, Mignault A, Andreoletti C (2003) Trait impressions as
overgeneralized responses to adaptively significant facial qualities: Evidence from
connectionist modeling. Pers Soc Pscycol Rev 7(3):194215.
17. Oosterhof NN, Todorov A (2008) The functional basis of face evaluation. Proc Natl
Acad Sci USA 105(32):1108711092.
18. Said CP, Sebe N, Todorov A (2009) Structural resemblance to emotional expressions
predicts evaluation of emotionally neutral faces. Emotion 9(2):260264.
19. Jenkins R, White D, Van Montfort X, Burton AM (2011) Variability in photos of the
same face. Cognition 121(3):313323.
20. Todorov A, Porter JM (2014) Misleading first impressions: Different for different facial
images of the same person. Psychol Sci 25(7):14041417.
21. Sutherland CAM, et al. (2013) Social inferences from faces: Ambient images generate
a three-dimensional model. Cognition 127(1):105118.
22. Santos IM, Young AW (2005) Exploring the perception of social characteristics in faces
using the isolation effect. Vis Cogn 12:213247.
23. Santos IM, Young AW (2008) Effects of inversion and negation on social inferences
from faces. Perception 37(7):10611078.
24. Santos IM, Young AW (2011) Inferring social attributes from different face regions:
Evidence for holistic processing. Q J Exp Psychol (Hove) 64(4):751766.
25. Malpass RS, Kravitz J (1969) Recognition for faces of own and other race. JPersSoc
Psychol 13(4):330334.
26. Meissner CA, Brigham JC (2001) Thirty years of investigating the own-race bias
in memory for facesA meta-analytic review. Psychol Public Policy Law 7(1):
335.
27. Tiddeman B, Burt M, Perrett D (2001) Prototyping and transforming facial textures for
perception research. Computer Graphics and Applications, IEEE 21(5):4250.
28. Russell R (2009) A sex difference in facial contrast and its exaggeration by cosmetics.
Perception 38(8):12111219.
29. Walker M, Vetter T (2009) Portraits made to measure: Manipulating social judgments
about individuals with a statistical face model. J Vis 9(11):113.
30. Bruce V (1994) Stability from variation: The case of face recognition. QJExpPsycholA
47(1):528.
31. Burton AM (2013) Why has research in face recognition progressed so slowly? The
importance of variability. Q J Exp Psychol (Hove) 66(8):14671485.
32. Hay DC, Young AW (1982) The human face. Normality and Pathology in Cognitive
Functions, ed Ellis AW (Academic, London), pp 173202.
33. Bruce V, Young A (1986) Understanding face recognition. Br J Psychol 77(Pt 3):
305327.
Vernon et al. PNAS Early Edition
|
9of9
PSYCHOLOGICAL AND
COGNITIVE SCIENCES
PNAS PLUS
Supporting Information
Vernon et al. 10.1073/pnas.1409860111
Fig. S1. Key to fiducial points.
Vernon et al. www.pnas.org/cgi/content/short/1409860111 1of7
Fig. S2. Examples of individual face reconstructions based on modeled attributes.
Vernon et al. www.pnas.org/cgi/content/short/1409860111 2of7
Table S1. Description of attributes derived from ambient face images
Attribute Calculation
01. Head area Area enclosed by points 135:145, 113:115, 126:134, 110:112
02. Head height Vertical distance between centroid of 139:141 and centroid of 129:131
03. Head width Horizontal distance between centroid of 110:112 and centroid of 113:115
04. Head orientation 1 Absolute xaxis coordinate of middle of nose (centroid of 54:56, 60, 61, 66, 71).
Should increase as individual looks to the left or right.
05. Head orientation 2 Absolute yaxis coordinate of middle of nose (centroid of 54:56, 60, 61, 66, 71).
Should increase when looking down (may be confounded by nose length).
06. Head tilt* Gradient of line used for xaxis before standardization. Should increase as individuals
face becomes tilted (i.e., rotation about nose).
07. Eyebrow area Area enclosed by points 72:77, 85, 84
08. Eyebrow height Vertical distance between centroid of 72, 84, 85, 77 and centroid of 73:76
09. Eyebrow width Horizontal distance between point 72 and point 77
10. Eyebrow gradient* Absolute gradient of linear polynomial fitted through points 84, 85, 77.
Should increase as eyebrows become arched, or slant downward.
11. Eye area Area enclosed by points 19:23, 31, 30, 29
12. Iris area Area enclosed by points 3:10
13. Eye height Vertical distance between centroid of 20:22 and centroid of 29:31
14. Eye width Horizontal distance between points 19 and 23
15. % Iris* (1/πr
2
)*Iris area, where ris 1/2 horizontal distance between points 5 and 9. Intended
to show what percentage of the iris is visible, should approach one as iris becomes
less hidden and more circular
16. Nose area Area enclosed by points 51:54,62:66,56,71:67,60:57
17. Nose height Vertical distance between points 56 and centroid of 51,57
18. Nose width Horizontal distance between centroid of 67:69 and centroid of 62:64
19. Nose curve* Coefficient of x
2
from quadratic polynomial fitted through points 64:66, 56, 71:69.
Should increase as the bottom of the nose becomes less flat
20. Nose flare Vertical distance between centroid of 65, 70 and centroid of 64, 66, 71, 69
Should increase as nostrils become dilated/larger.
21. Jaw height Vertical distance between centroid of 112,115 and centroid of 129:131
22. Jaw gradient* Absolute gradient of linear polynomal fitted through points 128:130
23. Jaw deviation SD of distances between all points on jaw (126:134) and point at the top of the
jaw (x=average of 112, 115, 126:134; y=average of 112, 115)
Should increase as jaw becomes longer/less rounded.
24. Chin curve* Coefficient of x
2
from quadratic polynomial fitted through points 128:132. Similar
to jaw gradient (should increase as chin gets less rounded)
25. Mouth area Area enclosed by points 88:94, 109:105
26. Mouth height Vertical distance between centroid of 88:94 and centroid of 88, 105:109.94
27. Top lip height Vertical distance between centroid of 88:94 and centroid of 88, 95:99,94
28. Bottom lip height Vertical distance between centroid of 88, 100:104, 94 and centroid of 88, 105:109.94
29. Mouth width Horizontal distance between points 94 and 88
30. Mouth gap Vertical distance between centroid of 88, 95:99, 94 and centroid of 88, 100:104, 94.
Should increase with an open mouth.
31. Top lip curve* Coefficient of x
2
from quadratic polynomial fitted through points 8895:9994
32. Bottom lip curve* Coefficient of x
2
from quadratic polynomial fitted through points 88100:10494.
The above two attributes should increase with a smile.
33. Nose line sep. Horizontal distance between centroid of 171:172 and centroid of 173:174
34. Cheekbone position* (1/head height) ×(vertical distance between centroid of 165:170 and centroid of 129:131)
35. Cheek gradient* Absolute gradient of linear polynomal fitted through points 165:167
36. Eye line gradient* Absolute gradient of linear polynomal fitted through points 45:47
37. Eyes position* (1/head height) ×(vertical distance between centroid of 13:34 and centroid of 129:131)
38. Eyebrows position* (1/head height) ×(vertical distance between centroid of 72:87 and centroid of 129:131)
39. Mouth position* (1/head height) ×(vertical distance between centroid of 88:94,105:109 and centroid of 129:131)
40. Nose position* (1/head height) ×(vertical distance between centroid of 51:71 and centroid of 129:131)
These attributes all return a value between 0 (bottom of face) and 1 (top of face). If nose position
was 0.5, it would suggest that the middle of the nose is vertically located in the middle of the face.
41. Eye separation Horizontal distance between centroid of 2, 11:18, 24:28, 32:34 and centroid of 1, 3:10, 19:23, 29:31
42. Eyes-mouth distance Vertical distance between centroid of 23:24 and centroid of 90:92
43. Eyes-eyebrows distance Vertical distance between centroid of 72, 84, 85, 77, 78, 86, 87, 83, and centroid of 20:22, 25:27
44. Left jead to left eye Horizontal distance between centroid of 110:112 and 19
45. Right head to right eye Horizontal distance between centroid of 113:115 and 28
46. Mouth-chin dististance Vertical distance between centroid of 106:108 and centroid of 129:131
47. Mouth-nose distance Vertical distance between centroid of 66, 56, 71 and centroid of 90:92
Vernon et al. www.pnas.org/cgi/content/short/1409860111 3of7
Table S1. Cont.
Attribute Calculation
48. Skin hue* Color information (HSV format) for area enclosed by points points 135:145, 113:115,
134:126, 112:110, excluding the eyebrows, eyes, and mouth.49. Skin saturation*
50. Skin value*
51. Eyebrow hue* Color information (HSV format) for area enclosed by points points 72:77, 85,84 and 78:83, 87,86.
52. Eyebrow saturation*
53. Eyebrow value*
54. Lip hue* Color information (HSV format) for area enclosed by points points 88:94, 99:95 and 88,
100:104, 94, 109:105.55. Lip saturation*
56. Lip value*
57. Iris hue* Color information (HSV format) for area enclosed by points points 3:10 and 11:18.
58. Iris saturation*
59. Iris value*
Hue represents color, it should increase as the hue becomes redder, and decrease as it
becomes yellower.
Saturation is how vibrant that color is; as this attribute decreases the color becomes
more faded.
Value (or brightness) is how light or dark the color is, as this increases the color
becomes lighter.
60. Hue entropy* These attributes are based on Matlabsentropyfilt,used on the hue, saturation and
value channels of the area classed as skin (see skin color above). As these attributes decrease,
the respective channel should become more uniform (smoother).
61. Saturation entropy*
62. Value entropy*
63. Glasses* Signifies whether the person has glasses (1) or not (0)
64. Facial hair* Signifies whether the person has facial hair (beard, moustache; 1) or not (0)
65. Stubble* Signifies whether the person has stubble (1) or not (0)
Refer to Fig. S1 for the location of the numbered fiducial points mentioned in the Description column [specified using Psychomorph (1)], m:nindicates
inclusive ranges of points mthrough n. An asterisk indicates attributes where (further) normalization with reference to head size was not applicable. HSV, hue,
saturation, value.
1. Tiddeman B, Burt M, Perrett D (2001) Prototyping and transforming facial textures for perception research. Computer Graphics and Applications, IEEE 21(5):4250.
Vernon et al. www.pnas.org/cgi/content/short/1409860111 4of7
Movie S1. Animation showing changes in facial features associated with the Approachability dimension. Face-like images generated using the approach
illustrated in Fig. 1 Cand Dwith target factor scores varying from (1, 0, 0) to (1, 0, 0) (and back again).
Movie S1
Vernon et al. www.pnas.org/cgi/content/short/1409860111 5of7
Movie S2. Animation showing changes in facial features associated with the Youthful-Attractiveness dimension. Face-like images generated using the ap-
proach illustrated in Fig. 1 Cand Dwith target factor scores varying from (0, 1, 0) to (0, 1, 0) (and back again).
Movie S2
Vernon et al. www.pnas.org/cgi/content/short/1409860111 6of7
Movie S3. Animation showing changes in facial features associated with the Dominance dimension. Face-like images generated using the approach illus-
trated in Fig. 1 Cand Dwith target factor scores varying from (0, 0, 1) to (0, 0, 1) (and back again).
Movie S3
Vernon et al. www.pnas.org/cgi/content/short/1409860111 7of7
... Santos and Young (2011) found that the internal features of a face, enclosing the eyes, nose, and mouth, were critical for facial impressions of trustworthiness. In addition, Vernon et al. (2014) found evidence supporting that the mouth region is likely a major cue to these impressions. Such findings support the prediction that an occlusion of the mouth region would reduce the trustworthiness signal conveyed by a face, thereby impairing the discriminability between high and low trustworthiness. ...
... Importantly, previous studies found that while both the eye and the mouth regions were important for trustworthiness judgments, dominance judgments were mostly supported by the eyebrows, skin saturation, and facial shape features including the delineation of a face, wider chins, and narrower heads (Dotsch & Todorov, 2012;Robinson et al., 2014;Toscano et al., 2014;Vernon et al., 2014;Windhager et al., 2011). These findings converge with evidence presented by Oosterhof and Todorov (2008, p. 11090, Study 10) showing that variations of facial shape were predictive of dominance, but not of trustworthiness judgments. ...
Article
Full-text available
Recognizing the role that facial appearance plays in guiding social interactions, here we investigated how occlusions of the bottom-face region affect facial impressions of trustworthiness and dominance. Previous studies suggesting that different facial features impact inferences on these traits sustain the hypothesis that wearing a face mask will differently affect each trait inference. And specifically, that trustworthiness impressions will be more disrupted by this type of face occlusion than dominance impressions. In two studies, we addressed this possibility by occluding the bottom face region of faces that were previously shown to convey different levels of dominance and trustworthiness, and tested differences in the ability to discriminate between these trait levels across occlusion conditions. In Study 1 faces were occluded by a mask, and in Study 2 by a square image. In both studies, results showed that although facial occlusions generally reduced participants’ confidence on their trait judgments, the ability to discriminate facial trustworthiness was more strongly affected than the ability to discriminate facial dominance. Practical and theoretical implications of these findings are discussed.
... All of the tweets featured several key elements intended to make a positive first impression with audiences and encourage them to click. Profile "approachability" plays a role in user engagement with tweets (Vernon et al. 2014), thus the @ SeanYoungPhD account features a close-up photograph, as opposed to a medium or long shot. We refrained from using scientific terminology, institutional verbiage, or acronyms that would require prior knowledge of public health issues to comprehend the messages. ...
Book
Full-text available
Addresses the motivation and enablers for digital health innovations Contextualizes the application, technical considerations, as well as socio-psycho-economical ones influencing many digital health technologies’ acceptance and widespread use Presents a comprehensive state-of the-art approach to digital health technologies and practices
... Certain facial features like light skin tone, facial symmetry, and specific mouth/nose outlines are often used to judge whether a person appears attractive, trustworthy, or approachable (Linke et al., 2016;Perrett et al., 1999;R. Russell, 2003;Scheib, Gangestad, & Thornhill, 1999;Todorov, Baron, & Oosterhof, 2008;Vernon, Sutherland, Young, & Hartley, 2014). Invariant characteristics like sex and age, together with variable emotional expressions, are also known to influence the inference of these trait-like characteristics. ...
Article
Full-text available
Faces convey a lot of information about a person. However, the usage of face masks occludes important parts of the face. There is already information that face masks alter the processing of variable characteristics such as emotional expressions and the identity of a person. To investigate whether masks influenced the processing of facial information, we compared ratings of full faces and those covered by face masks. 196 participants completed one of two parallel versions of the experiment. The data demonstrated varying effects of face masks on various characteristics. First, we showed that the perceived intensity of emotional expressions was reduced when the face was covered by face masks. This can be regarded as conceptual replication and extension of the impairing effects of face masks on the recognition of emotional expressions. Next, by analyzing valence and arousal ratings, the data illustrated that emotional expressions were regressed toward neutrality for masked faces relative to no-masked faces. This effect was grossly pronounced for happy facial expressions, less for neutral expressions, and absent for sad expressions. The sex of masked faces was also less accurately identified. Finally, masked faces looked older and less attractive. Post hoc correlational analyses revealed correlation coefficient differences between no-masked and masked faces. The differences occurred in some characteristic pairs (e.g., Age and Attractiveness, Age and Trustworthiness) but not in others. This suggested that the ratings for some characteristics could be influenced by the presence of face masks. Similarly, the ratings of some characteristics could also be influenced by other characteristics, irrespective of face masks. We speculate that the amount of information available on a face could drive our perception of others during social communication. Future directions for research were discussed.
... As a process rooted in human biology, it seems clear that human biology is social biology. For example, with just 50 ms of exposure, we already know whether the other inspires confidence or not, and this assessment correlates with activity in the amygdala (Freeman et al., 2014); after 100 ms, evaluation of the confidence aroused by the presence of the other is complete (Vernon et al., 2014). Surprisingly, these studies are conducted such that an explicit trust assessment is not necessary, showing how natural relationships and interpersonal encounters are. ...
Article
Full-text available
Rather than occurring abstractly (autonomously), ethical growth occurs in interpersonal relationships (IRs). It requires optimally functioning cognitive processes [attention, working memory (WM), episodic/autobiographical memory (AM), inhibition, flexibility, among others], emotional processes (physical contact, motivation, and empathy), processes surrounding ethical, intimacy, and identity issues, and other psychological processes (self-knowledge, integration, and the capacity for agency). Without intending to be reductionist, we believe that these aspects are essential for optimally engaging in IRs and for the personal constitution. While they are all integrated into our daily life, in research and academic work, it is hard to see how they are integrated. Thus, we need better theoretical frameworks for studying them. That study and integration thereof are undertaken differently depending on different views of what it means to live as a human being. We rely on neuroscientific data to support the chosen theory to offer knowledge to understand human beings and interpersonal relational growth. We should of course note that to describe what makes up the uniqueness of being, acting, and growing as a human person involves something much more profound which requires too, a methodology that opens the way for a theory of the person that responds to the concerns of philosophy and philosophical anthropology from many disciplines and methods ( Orón Semper, 2015 ; Polo, 2015 ), but this is outside the scope of this study. With these in mind, this article aims to introduce a new explanatory framework, called the Interprocessual-self (IPS), for the neuroscientific findings that allow for a holistic consideration of the previously mentioned processes. Contributing to the knowledge of personal growth and avoiding a reductionist view, we first offer a general description of the research that supports the interrelation between personal virtue in IRs and relevant cognitive, emotional, and ethic-moral processes. This reveals how relationships allow people to relate ethically and grow as persons. We include conceptualizations and descriptions of their neural bases. Secondly, with the IPS model, we explore neuroscientific findings regarding self-knowledge, integration, and agency, all psychological processes that stimulate inner exploration of the self concerning the other. We find that these fundamental conditions can be understood from IPS theory. Finally, we explore situations that involve the integration of two levels, namely the interpersonal one and the social contexts of relationships.
Article
Full-text available
Human personality is significantly represented by those words which he/she uses in his/her speech or writing. As a consequence of spreading the information infrastructures (specifically the Internet and social media), human communications have reformed notably from face to face communication. Generally, Automatic Personality Prediction (or Perception) (APP) is the automated forecasting of the personality on different types of human generated/exchanged contents (like text, speech, image, video, etc.). The major objective of this study is to enhance the accuracy of APP from the text. To this end, we suggest five new APP methods including term frequency vector-based, ontology-based, enriched ontology-based, latent semantic analysis (LSA)-based, and deep learning-based (BiLSTM) methods. These methods as the base ones, contribute to each other to enhance the APP accuracy through ensemble modeling (stacking) based on a hierarchical attention network (HAN) as the meta-model. The results show that ensemble modeling enhances the accuracy of APP.
Article
Children as young as 3 years can make trait attributions based on behavioral and emotional cues, but such skills continue to develop across childhood. Theory of mind understanding, the ability to attribute mental states to oneself and others, may provide a foundation for early development of trait attributions. The purpose of the current study was to explore the impact of behavioral and affective cues on children’s trait attributions, if their attributes changed incrementally across five repeated instances of an observed behavior, and to what extent such patterns of attributions are related to false belief, a key concept of theory of mind. A total of 115 3- to 5-year-olds completed theory of mind tasks and two trait attribution tasks with affect and the nature of behavior (helpful/unhelpful) varied. Use of a quantitative histogram enabled identification of subtle changes in attributions across episodes. Results indicated that preschool-aged children rated characters as less likable with repeated instances of unhelpful behavior, with meaningful changes occurring after a second case of behavior. The 5-year-olds were more sensitive to differences in helpfulness than the other two age groups. In addition, the 4-year-olds rated smiling helpful characters more positively across time, suggesting a potential impact of emotional cues. Moreover, false belief was related to, yet did not account for, children’s attributions. Factors affecting young children’s formation of trait attributions are discussed.
Article
Humans spontaneously attribute character traits to strangers based on their facial appearance. Although these ‘first impressions’ typically have no basis in reality, some authors have assumed that they have an innate origin. By contrast, the Trait Inference Mapping (TIM) account proposes that first impressions are products of culturally acquired associative mappings that allow activation to spread from representations of facial appearance to representations of trait profiles. According to TIM, cultural instruments, including propaganda, illustrated storybooks, art and iconography, ritual, film, and TV, expose many individuals within a community to common sources of correlated face–trait experience, yielding first impressions that are shared by many, but typically inaccurate. Here, we review emerging empirical findings, many of which accord with TIM, and argue that future work must distinguish first impressions based on invariant facial features (e.g., shape) from those based on facial behaviours (e.g., expressions).
Article
Using machine learning–based algorithms, we measure key impressions about sell‐side analysts using their LinkedIn photos. We find that impressions of analysts’ trustworthiness (TRUST) and dominance (DOM) are positively associated with forecast accuracy, especially after recent in‐person meetings between analysts and firm managers. High TRUST also enhances stock return sensitivity to forecast revisions, especially for stocks with high institutional ownership. In contrast, the impression of analysts’ attractiveness (ATTRACT) is only positively associated with accuracy for new analysts or when a firm has a new CEO or CFO. Furthermore, while high DOM helps male analysts’ chances of attaining All‐Star status, it reduces female analysts’ accuracy and the likelihood of winning the All‐Star award. In addition, the relation between TRUST and accuracy is modulated by the disclosure environment and is attenuated by Regulation Fair Disclosure. Our results suggest that face impressions influence analysts’ access to information and the perceived credibility of their reports. This article is protected by copyright. All rights reserved
Article
Full-text available
The current article reviews the own-race bias (ORB) phenomenon in memory for human faces, the finding that own-race faces are better remembered when compared with memory for faces of another, less familiar race. Data were analyzed from 39 research articles, involving 91 independent samples and nearly 5,000 participants. Measures of hit and false alarm rates, and aggregate measures of discrimination accuracy and response criterion were examined, including an analysis of 8 study moderators. Several theoretical relationships were also assessed (i.e., the influence of racial attitudes and interracial contact). Overall, results indicated a "mirror effect" pattern in which own-race faces yielded a higher proportion of hits and a lower proportion of false alarms compared with other-race faces. Consistent with this effect, a significant ORB was also found in aggregate measures of discrimination accuracy and response criterion. The influence of perceptual learning and differentiation processes in the ORB are discussed, in addition to the practical implications of this phenomenon.
Article
Full-text available
Bond, Charles F; Berry, Diane S; Omar (Atoum), Adnan (1994). The kernel of truth in judgments of deceptiveness. Basic & Applied Social Psychology. Vol. 15(4), 523-534. This article describes an investigation of the relationship between appearance-based impressions of honesty and individuals' willingness to engage in deceptive behaviors. Neutral-expression photographs were taken of 133 study participants, and these photographs were judged by other participants for whether the person looked honest or dishonest. The study participants then were provided with an opportunity to engage in deceptive behavior. Participants who were rated as looking dishonest by the third parties (via the photographs), were more likely to volunteer to participate in research that was described as requiring deception than were participants who were perceived to look honest. The results suggested that naive judgments of deception are more accurate than has been supposed.
Article
Full-text available
Antisocial and criminal behaviors are multifactorial traits whose interpretation relies on multiple disciplines. Since these interpretations may have social, moral and legal implications, a constant review of the evidence is necessary before any scientific claim is considered as truth. A recent study proposed that men with wider faces relative to facial height (fWHR) are more likely to develop unethical behaviour mediated by a psychological sense of power. This research was based on reports suggesting that sexual dimorphism and selection would be responsible for a correlation between fWHR and aggression. Here we show that 4,960 individuals from 94 modern human populations belonging to a vast array of genetic and cultural contexts do not display significant amounts of fWHR sexual dimorphism. Further analyses using populations with associated ethnographical records as well as samples of male prisoners of the Mexico City Federal Penitentiary condemned by crimes of variable level of inter-personal aggression (homicide, robbery, and minor faults) did not show significant evidence, suggesting that populations/individuals with higher levels of bellicosity, aggressive behaviour, or power-mediated behaviour display greater fWHR. Finally, a regression analysis of fWHR on individual's fitness showed no significant correlation between this facial trait and reproductive success. Overall, our results suggest that facial attributes are poor predictors of aggressive behaviour, or at least, that sexual selection was weak enough to leave a signal on patterns of between- and within-sex and population facial variation.
Article
Full-text available
The evolution of cooperation requires some mechanism that reduces the risk of exploitation for cooperative individuals. Recent studies have shown that men with wide faces are anti-social, and they are perceived that way by others. This suggests that people could use facial width to identify anti-social men and thus limit the risk of exploitation. To see if people can make accurate inferences like this, we conducted a two-part experiment. First, males played a sequential social dilemma, and we took photographs of their faces. Second, raters then viewed these photographs and guessed how second movers behaved. Raters achieved significant accuracy by guessing that second movers exhibited reciprocal behaviour. Raters were not able to use the photographs to further improve accuracy. Indeed, some raters used the photographs to their detriment; they could have potentially achieved greater accuracy and earned more money by ignoring the photographs and assuming all second movers reciprocate.
Article
Studies on first impressions from facial appearance have rapidly proliferated in the past decade. Almost all of these studies have relied on a single face image per target individual, and differences in impressions have been interpreted as originating in stable physiognomic differences between individuals. Here we show that images of the same individual can lead to different impressions, with within-individual image variance comparable to or exceeding between-individuals variance for a variety of social judgments (Experiment 1). We further show that preferences for images shift as a function of the context (e.g., selecting an image for online dating vs. a political campaign; Experiment 2), that preferences are predictably biased by the selection of the images (e.g., an image fitting a political campaign vs. a randomly selected image; Experiment 3), and that these biases are evident after extremely brief (40-ms) presentation of the images (Experiment 4). We discuss the implications of these findings for studies on the accuracy of first impressions.
Article
Most of the actions we carry out on a daily basis require timing on the scale of tens to hundreds of milliseconds. We must judge time to speak, to walk, to predict the interval between our actions and their effects, to determine causality and to decode information from our sensory receptors. However, the neural bases of time perception are largely unknown. Scattered confederacies of investigators have been interested in time for decades, but only in the past few years have new techniques been applied to old problems. Experimental psychology is discovering how animals perceive and encode temporal intervals, while physiology, fMRI and EEG unmask how neurons and brain regions underlie these computations in time. This symposium will capitalize on new breakthroughs, outlining the emerging picture and highlighting the remaining confusions about time in the brain. How do we encode and decode temporal information? How is information coming into different brain regions at different times synchronized? How plastic is time perception? How is it related to space perception? The experimental work of the speakers in this symposium will be shored together to understand how neural signals in different brain regions come together for a temporally unified picture of the world, and how this is related to the mechanisms of space perception. The speakers in this symposium are engaged in experiments at complementary levels of exploring sub-second timing and its relation to space.
Article
[Formula: see text] Despite many years of research, there has been surprisingly little progress in our understanding of how faces are identified. Here I argue that there are two contributory factors: (a) Our methods have obscured a critical aspect of the problem, within-person variability; and (b) research has tended to conflate familiar and unfamiliar face processing. Examples of procedures for studying variability are given, and a case is made for studying real faces, of the type people recognize every day. I argue that face recognition (specifically identification) may only be understood by adopting new techniques that acknowledge statistical patterns in the visual environment. As a consequence, some of our current methods will need to be abandoned.
Article
Three experiments are presented that investigate the two-dimensional valence/trustworthiness by dominance model of social inferences from faces (Oosterhof & Todorov, 2008). Experiment 1 used image averaging and morphing techniques to demonstrate that consistent facial cues subserve a range of social inferences, even in a highly variable sample of 1000 ambient images (images that are intended to be representative of those encountered in everyday life, see Jenkins, White, Van Montfort, & Burton, 2011). Experiment 2 then tested Oosterhof and Todorov's two-dimensional model on this extensive sample of face images. The original two dimensions were replicated and a novel 'youthful-attractiveness' factor also emerged. Experiment 3 successfully cross-validated the three-dimensional model using face averages directly constructed from the factor scores. These findings highlight the utility of the original trustworthiness and dominance dimensions, but also underscore the need to utilise varied face stimuli: with a more realistically diverse set of face images, social inferences from faces show a more elaborate underlying structure than hitherto suggested.
Article
The age, sex, and distinctiveness of faces can be judged from objective and partially independent facial features. In contrast, the physical basis of other social judgements, such as attractiveness, intelligence, and trustworthiness is not as yet entirely understood, despite the consistency of these judgements from faces. The present set of experiments investigated the perception of social characteristics in faces, using an adaptation of the von Restorff/isolation paradigm to determine which social characteristics are spontaneously encoded from the face. The isolation effect involves enhanced memory for perceptually salient items in a list (items that are isolated in the sense that they are in numeric minority) and our results suggested that its locus is at the encoding stage of the recognition memory experiment. Isolation in the present experiments was achieved by manipulating the number of faces included in the sets on the basis of certain characteristics. Because the manipulation of the characteristics of faces in a set to be learnt was not mentioned in most of the experiments, any resulting memory increment for the isolated items could be taken as an index of spontaneous processing of the manipulated social characteristic. Age and sex were found to be spontaneously encoded from faces. Results for other characteristics were mixed, ranging from distinctiveness and attractiveness, for which there was some indication of spontaneous processing, to intelligence and trustworthiness, which did not seem to be spontaneously encoded from faces. For intelligence, an isolation effect was found only when the experiment required a judgement that led to activation of the appropriate stereotype.