ArticlePDF Available

In a world of big data, small effects can still matter: a reply to Boyce, Daly, Hounkpatin, and Wood (2017)

Authors:

Abstract and Figures

We make three points in response to Boyce, Daly, Hounkpatin, and Wood (2017). First, we clarify a misunderstanding of the goal of our analyses, which was to investigate the links between life satisfaction and spending patterns, rather than spending volume. Second, we report a simulation study we ran to demonstrate that our results were not driven by the proposed statistical artifact. Finally, we discuss the broader issue of why, in a world of big data, small but reliable effect sizes can be valuable.
Content may be subject to copyright.
Title: In the World of Big Data, Small Effects Can Still Matter: A Reply to Boyce et al. 2016
Authors: Sandra Matz1, Joe Gladstone2, David Stillwell3
Affiliations:
1 Department of Psychology, University of Cambridge, Cambridge, CB2 3EB, UK.
2 UCL School of Management, London, RM NE3, UK
3 Cambridge Judge Business School, University of Cambridge, Cambridge, CB2 1AG, UK
Correspondence to: sm917@cam.ac.uk; +447858786958; Department of Psychology,
Downing Street, Cambridge, CB2 3EB, UK.
Keywords: effect size, happiness, consumption, psychological fit, Big Five personality
We make three points in response to Boyce, Daly, Hounkpatin and Wood (2016). First, we
clarify a misunderstanding of the goal of our analyses, which was to investigate the links
between life satisfaction and spending patterns, rather than spending volume. Second, we run
a simulation study to demonstrate that our results are not driven by the proposed statistical
artefact. Finally, we discuss the broader issue of why, in a world of big data, small but reliable
effect sizes can be valuable.
Our goal is to study spending patterns, not spending volume.
Boyce et al. argue that our findings do not provide sufficient evidence for the contention that
spending more money buys happiness, when that money is spent on products and services that
fit a person’s personality. We agree, because this is not the argument we make in the paper.
Our argument is that spending the money one already has on products and services which
match one’s personality (and thus psychological needs) results in greater happiness. That is,
we studied spending patterns (what we buy) rather than spending volume (how much of it we
buy). In fact, we emphasized the relative unimportance of “spending more money” in our
measures, analyses, and study designs. First, we considered the fact that many small
purchases can result in greater happiness than a few large ones (e.g. Nelson and Meyvis,
2008) in the calculation of the person-basket match variable itself. Rather than weighting all
spending categories by the amount spent, we assigned an equal weight to each of them.
Second, we highlighted that the coefficients for income and total spending on life-satisfaction
are non-significant (Table 3, page 6), suggesting that earning or spending more money does
not impact life satisfaction and happiness. Finally, we did not vary the amount spent in our
experiment, but instead focused exclusively on manipulating the spending pattern (i.e.
whether or not the voucher fitted the personality of the recipient).
Our results are not exclusively driven by the proposed statistical artefact.
Boyce et al. (2016) suggest that the effect of the match between customers shopping baskets
and their personality might be driven “by participant personality, which is known to relate to
well-being”. We also had this concern. Which is why the original analysis controlled for
participant personality as well as the overall extremity of profiles and found that the effects of
basket-participant matched remained stable when controlling for these variables (see Table 3;
Matz et al., 2016).
To provide further evidence for our effect’s robustness, we ran a simulation on our dataset
that is similar to the one reported by Boyce et al. (2016). In 1,000 iterations, we randomly
allocated the basket personality calculated for each participant in Study 1 to another
participant (i.e., we ‘swap’ participants’ shopping baskets at random). We then calculated the
basket-participant match and ran regression analyses including the control variables used by
Boyce et al. (Model 1 without total spend) and the original control variables of Model 2 (see
online supplementary materials for more details). Out of 1000 iterations, basket-personality
match was significant at an alpha level of 0.05 in only 3.5% of cases when including the
controls used by Boyce et al., and in only 2.3% of cases when including all control variables
of Model 2. The average coefficient of the randomly generated basket-participant match was
B = 0.013 in Model 1 (SD = 0.03; average ß = 0.018; left panel of Figure 1) and B = -0.004 in
Model 2 (SD = 0.03; average β = -0.005; right panel of Figure 1). This compares to
coefficients of B = 0.06 and ß = 0.10 reported for real basket-participant match in our original
analysis (using the same controls). We take this as strong evidence that the relationship
between basket-participant match and life satisfaction is not (or at least not exclusively)
driven by the confounding artefacts suggested by Boyce et al. (2016).
Figure 1. Distribution of regression coefficients (B) for randomly
distributed basket personality across 1,000 iterations, using the
controls of Boyce et al. (left; original Model 1 without total
spend) and the original controls of Model 2 (right).
The importance of small effects.
Boyce et al. (2016) argue that the relationship we report between a consumer’s purchases and
their wellbeing is of no practical relevance. This raises an important debate about the
magnitude of meaningful effect sizes.
Traditional social science research uses small samples (Bertamini & Munafò, 2012; Button et
al., 2013), which require large effect sizes to reach statistical significance and be published.
Large effect sizes are more likely to be replicable (Open Science Collaboration, 2015), but
focusing exclusively on them hinders a nuanced exploration of complex psychological
phenomena such as life satisfaction, which are unlikely to be explained by a few strong
predictors. Instead of dismissing small effect sizes altogether, researchers should explore new
methodologies that enable the discipline of psychology to build a body of small but robust
predictors of behavior. New computational social science approaches (Big Data; see e.g.
Kosinski, Matz, Gosling, Popov, & Stillwell, 2015), for example, make it possible to collect
behavioral data at an unprecedented scale, providing psychologists with opportunities to
identify weak but reliable signals in a complicated world. However, the danger of using these
new methods is that even trivial effects will become statistically significant with large enough
samples. Therefore, how can we distinguish between small effects which are meaningful,
from those which are not?
This requires a scientist’s judgement rather than simply following arbitrary cut-offs. A full
review of situations when small effect sizes are important would require more space than
available in a commentary, but other authors have already discussed some situations. Cortina
and Landis (2009), for example, note that small effects can suggest strong support for a given
phenomenon if they (1) occur in the context of intentionally inauspicious designs, (2)
challenge fundamental theoretical assumptions, and (3) have enormous cumulative
consequences. One illustration of the latter is Abelson’s Paradox (Abelson, 1985).
Investigating the batting performance of major-league baseball players, Abelson showed that
historical batting performance was barely predictive of a player’s performance each time they
stepped up to the batting plate (less than 0.5%). Yet, the cumulative effect over several
hundred bats is so meaningful in practice that it leads to some batters being paid 100,000%
more than others.
In our paper, one challenge in determining the meaning in effect sizes is that it is difficult to
directly compare them across studies with different methodologies, such as self-reported
surveys and objective behavioral data. Study 1 is a ‘Big Data’ field study, where over 76,000
transactions are mined from customers’ bank accounts and where the measurement of
wellbeing is far removed, in both context and time, from when the bank customers’ purchases
were being made. Such a procedure is likely to result in smaller yet probably more realistic -
effect sizes than panel surveys because they eliminate well-documented response biases (e.g.,
consistency motive, covariation bias, or common method variance). For example, Powdthavee
(2008; cited by Boyce et al., 2016), reports a large effect of social contact on subjective
wellbeing in the British Household Panel Survey. While social contact is surely important for
wellbeing, the reported effect size might be over-estimated because respondents are
subjectively reporting both their social contact and their wellbeing at the same time.
Respondents who felt negatively about their lives, for example, might be less likely to
remember and report short and superficial social contact, such as saying “hello” to a neighbor,
than respondents who felt more positive (mood congruence; Bower, 1981). Indeed, the results
of our Study 2 underline the importance of considering the methodological context when
discussing a given effect size; Study 2’s behavioral field experiment found a medium to large
effect size for the personality-fit interaction (ß = 0.38).
A final consideration when deciding whether a small effect may have practical relevance is
the extent to which the factor can be manipulated. In their commentary, Boyce et al. (2016)
point to other factors that are stronger predictors of wellbeing (in terms of standardized effect
sizes) such as relationships (Powdthavee, 2008), personality (Diener & Lucas, 1999) or stable
employment (McKee-Ryan, Song, Wanberg, & Kinicki, 2005). Such factors certainly play an
important role in predicting life satisfaction. However, personality cannot easily be changed
(McCrae & Costa, 1994) and it is often outside the control of an unemployed person to
arrange for stable employment. Consumption choices, on the other hand, are usually under the
control of the individual and can be changed relatively easily. Even the very poorest groups in
the world (i.e., those living on less than $2 per day), spend substantial amounts on
discretionary goods, such as entertainment, celebrations, clothing and tobacco (Banerjee &
Duflo, 2007). Given that consumption is such a universal phenomenon, even small changes
can have a big impact if they occur on a large scale. A 1% increase in life satisfaction may be
negligible for a single consumer, but if a retail giant like Amazon aimed at making its
customers happier by personalizing their product recommendations, a 1% increase in life
satisfaction across its 244 million customers (Kline, 2014) could turn a small effect size into a
huge social effect.
Psychologists have typically focused on how their findings apply to individuals. However, by
providing the opportunity to understand and influence the behaviors of billions of people
around the world, the era of big data encourages and possibly requires - us to think bigger.
In this new world, small effects can still matter.
(Cortina & Landis, 2009)
(Boyce, Daly, Hounkpatin, & Wood, n.d.)
References
Abelson, R. P. (1985). A variance explanation paradox: When a little is a lot. Psychological
Bulletin, 97(1), 129.
Banerjee, A. V, & Duflo, E. (2007). The economic lives of the poor. The Journal of Economic
Perspectives, 21(1), 141167.
Bertamini, M., & Munafò, M. R. (2012). Bite-size science and its undesired side effects.
Perspectives on Psychological Science, 7(1), 6771.
Bower, G. H. (1981). Mood and memory. American Psychologist, 36(2), 129. JOUR.
Boyce, C., Daly, M., Hounkpatin, H., & Wood, A. (n.d.). Money may buy happiness, but so
little that it often doesn’t matter. Psychological Science.
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., &
Munafò, M. R. (2013). Power failure: why small sample size undermines the reliability
of neuroscience. Nature Reviews Neuroscience, 14(5), 365376.
Collaboration, O. S. (2015). Estimating the reproducibility of psychological science. Science,
349(6251), aac4716.
Cortina, J. M., & Landis, R. S. (2009). When small effect sizes tell a big story, and when large
effect sizes don’t. In C. E. Lance & R. J. Vandenberg (Eds.), Statistical and
methodological myths and urban legends: Doctrine, verity and fable in the
organizational and social sciences (pp. 287308). Routledge New York.
Diener, E., & Lucas, R. E. (1999). 11 Personality and Subjective Well-Being. Well-Being:
Foundations of Hedonic Psychology, 213.
Kline, D. B. (2014). How many customers does Amazon have? Retrieved from
http://www.fool.com/investing/general/2014/05/24/how-many-customers-does-amazon-
have.aspx
Kosinski, M., Matz, S. C., Gosling, S. D., Popov, V., & Stillwell, D. (2015). Facebook as a
Research Tool for the Social Sciences. American Psychologist, 70(6), 543556.
McCrae, R. R., & Costa, P. T. (1994). The stability of personality: Observations and
evaluations. Current Directions in Psychological Science, 3(6), 173175.
McKee-Ryan, F., Song, Z., Wanberg, C. R., & Kinicki, A. J. (2005). Psychological and
physical well-being during unemployment: a meta-analytic study. Journal of Applied
Psychology, 90(1), 53.
Powdthavee, N. (2008). Putting a price tag on friends, relatives, and neighbours: Using
surveys of life satisfaction to value social relationships. The Journal of Socio-Economics,
37(4), 14591480.
Supplementary Online Material
Simulation Model 1
Simulation Model 2
Predictors
B
β
t
B
SE(B)
β
t
B-P-match
0.01
0.02
0.46
-0.004
0.03
-0.01
-0.12
Income (log)
0.08
0.07
1.70
0.05
0.06
0.04
0.77
Gender
0.02
0.03
0.32
-0.03
0.08
-0.04
-0.40
Age
-0.01
-0.09
-2.23
-0.01
0.003
-0.12
-2.69
Total spend (log)
-
-
-
0.03
0.07
0.03
0.42
Person-O
-
-
-
0.04
0.03
0.04
1.16
Person-C
-
-
-
<0.001
0.04
0.00
0.001
Person-E
-
-
-
0.08
0.04
0.09
2.26
Person-A
-
-
-
0.01
0.03
0.01
0.34
Person-N
-
-
-
-0.23
0.04
-0.27
-6.09
Extremity
-
-
-
-0.05
0.11
-0.02
-0.47
Product-O
-
-
-
-0.11
0.07
-0.13
-1.67
Product-C
-
-
-
0.07
0.06
0.09
1.18
Product-E
-
-
-
0.16
0.09
0.19
1.87
Product-A
-
-
-
0.12
0.08
0.13
1.53
Product-N
-
-
-
0.04
0.11
0.05
0.41
Note. In Model 1 the B-P match predictor reached significance in 35 out of the 1000
iterations. In Model 2 the B-P match predictor reached significant in 23 out of 1000 iterations
... The availability of rich records of Reddit conversations also made it possible to go back in time to retrieve clues about the pre-COVID-19 psychological states of people across cities. As is common in social media studies, our effect sizes are smaller than the effects typically reported in controlled laboratory studies that artificially control for most extraneous factors (51,52). Even in the current research, the social connection effects captured in survey data are substantially larger than parallel Reddit effects testing the same relationships. ...
... Even in the current research, the social connection effects captured in survey data are substantially larger than parallel Reddit effects testing the same relationships. Echoing other scholars, we believe that the big data approach provides reliable representations of real-world effects (53) and that small but reliable effects are most likely to be real and can scale up to produce substantial consequences at national or global levels (52). ...
Article
Full-text available
The current research chronicles the unfolding of the early psychological impacts of coronavirus disease 2019 (COVID-19) by analyzing Reddit language from 18 U.S. cities (200,000+ people) and large-scale survey data (11,000+ people). Large psychological shifts were found reflecting three distinct phases. When COVID-19 warnings first emerged (“warning phase”), people’s attentional focus switched to the impending threat. Anxiety levels surged, and positive emotion and anger dropped. In parallel, people’s thinking became more intuitive rather than analytic. When lockdowns began (“isolation phase”), analytic thinking dropped further. People became sadder, and their thinking reflected attempts to process the uncertainty. Familial ties strengthened, but ties to broader social groups weakened. Six weeks after COVID-19’s onset (“normalization phase”), people’s psychological states stabilized but remained elevated. Most psychological shifts were stronger when the threat of COVID-19 was greater. The magnitude of the observed shifts dwarfed responses to other events that occurred in the previous decade.
... However, it is to be expected that people are also buying items for other people -something which cannot be controlled for here. Although the literature on personality homophily suggests that friends and partners have similar personalities (Wu et al., 2017), and these people are likely to be for whom customers are buying for, purchasing for other people will still introduce noise into the data -contributing to smaller effect sizes that would be expected from a lab or field experiment with more controls (Matz, Gladstone, & Stillwell, 2017). ...
... Despite the smaller effect sizes commonly being found in big data analyses (Boyce, Daly, Hounkpatin, & Wood, 2017;Matz, Gladstone, & Stillwell, 2017), clear indication of personality homophily's relationship to well-being were found in this study (whilst controlling for demographics, total spend, income, individual's personality, and indeed additional personality controls), highlighting the potential for further investigation in a consumer domain. One would not expect consumption to affect well-being more than other established effects such as Extroversion and Neuroticism, but that such a signal for the influence of homophily on well-being is identifiable, provides a useful basis for understanding how we as consumers can get the best from our consumption decisions. ...
Preprint
Full-text available
Recommender systems and personalised marketing algorithms now proliferate our daily lives, harnessing similarity to other profiles to guide our purchasing decisions. Recent research indicates that this personalisation can be good for our well-being, as spending in a way that fits our personality, can engender happiness. Less well-understood however, is the impact personality homophily (where people prefer to connect with others of similar personalities) has on consumer well-being. In this study, we investigate whether individuals who exhibit homophily in their consumption patterns, by buying what others of a similar personality buy, report higher well-being. We analyse over 12,000 personality questionnaires measuring the Big Five and well-being, linked to 3 million loyalty card transaction logs from a multi-national retailer. Personality homophily, or `buying what people like you buy', is quantified by introducing a novel metric, Personality Alignment (PA). Findings show that PA on Extroversion, Agreeableness, and Neuroticism positively predicts well-being, and that effects are strongest for those higher in Neuroticism. Given the rise of algorithm-assisted decision making, along with growing attention to ethical-AI, these results show that personality homophily can be leveraged in the design of future personalised marketing mechanisms, not just for greater sales, but also customer’s long term well-being.
... Data from social media groups, like all naturalistic data, are noisy, leading to small effects (36,37). This is evidenced by the smaller effect sizes in the Reddit analysis in Study 3 relative to the effects detected via Study 1's surveys. ...
Article
Full-text available
To what degree can we determine people's connections with groups through the language they use? In recent years, large archives of behavioral data from social media communities have become available to social scientists, opening the possibility of tracking naturally occurring group identity processes. A feature of most digital groups is that they rely exclusively on the written word. Across 3 studies, we developed and validated a language-based metric of group identity strength and demonstrated its potential in tracking identity processes in online communities. In Studies 1a–1c, 873 people wrote about their connections to various groups (country, college, or religion). A total of 2 language markers of group identity strength were found: high affiliation (more words like we, togetherness) and low cognitive processing or questioning (fewer words like think, unsure). Using these markers, a language-based unquestioning affiliation index was developed and applied to in-class stream-of-consciousness essays of 2,161 college students (Study 2). Greater levels of unquestioning affiliation expressed in language predicted not only self-reported university identity but also students’ likelihood of remaining enrolled in college a year later. In Study 3, the index was applied to naturalistic Reddit conversations of 270,784 people in 2 online communities of supporters of the 2016 presidential candidates—Hillary Clinton and Donald Trump. The index predicted how long people would remain in the group (3a) and revealed temporal shifts mirroring members’ joining and leaving of groups (3b). Together, the studies highlight the promise of a language-based approach for tracking and studying group identity processes in online groups.
... Therefore, while the effect sizes from our analysis are small, this does not imply that they lack practical relevance [88][89][90] . Effect sizes that are considered small by arbitrary standards can have a large impact when evaluated over time 77,91 or at scale [92][93][94] . This is particularly true for human psychology, where effects can accumulate over time, thus underscoring the fact that while an effect might be small when measured at a single point in time, it can have large ultimate consequences 77 . ...
Preprint
Full-text available
A fundamental characteristic of modern societies is economic inequality, where deprived individuals experience chronic economic scarcity. While such experiences have been shown to produce detrimental outcomes in regards to human judgment and decision-making, the consequences of such scarcity for our morality remain debated. We conduct one of the most comprehensive tests of the relationship between experiences of relative chronic economic scarcity and various measures linked to morality. In a pre-registered study, we analyse data from a large, cross-national survey ( N = 46,450 across 67 countries) allowing us to address important limitations related to measurement validity and external validity in past research. Our findings demonstrate that experiences of relative chronic economic scarcity, as indexed by (1) low subjective socioeconomic status at the individual level, and (2) income inequality at the national macro level, predict higher levels of moral identity, higher morality-as-cooperation, a larger moral circle, and importantly; more prosocial behaviour. The results appear robust to several advanced control analyses. Finally, exploratory analyses indicate that observed income inequality at the national level does not significantly moderate the predicted effect of subjective socioeconomic status. Our findings have vital implications for understanding human morality under chronic resource scarcity.
... Is this practically relevant (Twenge et al., 2020) or not (Orben & Przybylski, 2019? Similarly, research teams have debated whether the impact of money and spending on happiness is large enough to be practically relevant, given that only 1% of the variance in life satisfaction is explained by whether people spend money in a way that's consistent with their personality (e.g., Matz et al., 2016Matz et al., , 2017Boyce et al, 2017). As a final example, there is ongoing disagreement about whether the impact of growthmindset interventions on academic performance is large enough to be practically relevant, with different groups of researchers debating whether aggregated effect sizes are large enough to merit resource investment (e.g., Denworth, 2019;Dweck, 2018;Gelman, 2018;Miller, 2019;Sisk et al., 2018;Yeager et al., 2019). ...
Preprint
Full-text available
Psychological researchers currently lack guidance for how to evaluate the practical relevance of observed effect sizes, i.e. whether a finding will have impact when translated to a different context of application. Although psychologists have recently highlighted theoretical justifications for why small effect sizes might be practically relevant, such justifications are simplistic and fail to provide the information necessary for evaluation and falsification. Claims about whether an observed effect size is practically relevant need to consider both the mechanisms amplifying and counteracting practical relevance, as well as the assumptions underlying each mechanism at play. To provide guidance for systematically evaluating whether an observed effect size is practically relevant, we present examples of widely applicable mechanisms and the key assumptions needed for justifying whether an observed effect size can be expected to generalize to different contexts. Routine use of these mechanisms to justify claims about practical relevance has the potential to make researchers’ claims about generalizability substantially more transparent. This transparency can help move psychological science towards a more rigorous assessment of when psychological findings can be applied in the world.
... In other domains (e.g., education), interventions that change lay theories such as those about the nature of intelligence have also produced small, but arguably useful, effects (Dweck 2018). Moreover, effects that are small in terms of explained variance can be large when considered in the original metrics (Matz, Gladstone, and Stillwell 2017). As examples from the present research, respondents low versus high in SIW beliefs had a difference in savings of over £1500 (roughly $1950 at the time of the study) in study 1, and intended to transfer 6% (study 2, manipulated beliefs) or 17% (study 5, measured beliefs) more to savings. ...
Article
Spending is influenced by many factors. One that has received little attention is the meaning that people give to the act of spending. Spending money might imply that someone is relatively wealthy—since they have money to spend—or relatively poor—since spending can deplete assets. We show that people differ in the extent to which they believe that spending implies wealth (SIW beliefs). We develop a scale to measure these beliefs and find that people who more strongly believe that spending implies wealth spend their own money relatively lavishly and are, on average, more financially vulnerable. We find correlational evidence for these relationships using objective financial-transaction data, including over 2 million transaction records from the bank accounts of over 2,000 users of a money management app, as well as self-reported financial well-being. We also find experimental evidence by manipulating SIW beliefs and observing causal effects on spending intentions. These results show how underlying beliefs about the link between spending and wealth play a role in consumption decisions, and point to beliefs about the meaning of spending as a fruitful direction for further research.
... However, subjective well-being is a multifaceted construct that is unlikely to be explained by a few strong predictors. Consequently, the ability to detect even small effects in large-scale behavioural data makes it possible to incrementally add to a better understanding of the subjective well-being puzzle (Matz, Gladstone & Stillwell, 2017). Taken together, our findings contribute to a better understanding of how everyday mobility behaviours that reflect the ways in which people interact with their physical environments are related to specific indicators of subjective well-being. ...
Article
People interact with their physical environments every day by visiting different places and moving between them. Such mobility behaviours likely influence and are influenced by people's subjective well-being. However, past research examining the links between mobility behaviours and well-being has been inconclusive. Here, we provide a comprehensive investigation of these relationships by examining individual differences in two types of mobility behaviours (movement patterns and places visited) and their relationship to six indicators of subjective well-being (depression, loneliness, anxiety, stress, affect, and energy) at two different temporal levels of analysis (two-week tendencies and daily level). Using data from a large smartphone-based longitudinal study (N = 1765), we show that (i) movement patterns assessed via GPS data (distance travelled, entropy, and irregularity) and (ii) places visited assessed via experience sampling reports (home, work, and social places) are associated with subjective well-being at the between and within person levels. Our findings suggest that distance travelled is related to anxiety, affect, and stress, irregularity is related to depression and loneliness, and spending time in social places is negatively associated with loneliness. We discuss the implications of our work and highlight directions for future research on the generalizability to other populations as well as the characteristics of places. © 2020 European Association of Personality Psychology.
... First, Study 1 captures authenticity using observations of actual behavior rather than self-reports. Given that such behavioral data captured in the wild do not suffer from the same response biases as self-reports which can inflate relationships between variables (e.g. common method bias 50 ), and are often noisier than self-reports, their effect sizes cannot be directly compared 51 . In fact, the effect sizes obtained in Study 2 which was conducted in a much more controlled, experimental setting shows that the effect of authenticity on subjective well-being is substantially larger when measured with more traditional methods (max(d) = 0.45). ...
Article
Full-text available
Social media users face a tension between presenting themselves in an idealized or authentic way. Here, we explore how prioritizing one over the other impacts users’ well-being. We estimate the degree of self-idealized vs. authentic self-expression as the proximity between a user’s self-reported personality and the automated personality judgements made on the basis Facebook Likes and status updates. Analyzing data of 10,560 Facebook users, we find that individuals who are more authentic in their self-expression also report greater Life Satisfaction. This effect appears consistent across different personality profiles, countering the proposition that individuals with socially desirable personalities benefit from authentic self-expression more than others. We extend this finding in a pre-registered, longitudinal experiment, demonstrating the causal relationship between authentic posting and positive affect and mood on a within-person level. Our findings suggest that the extent to which social media use is related to well-being depends on how individuals use it. It is often tempting for social media users to present themselves in an idealized way. Here, based on analyses of a large set of Facebook profiles together with a longitudinal experiment, the authors find evidence that more authentic self-expression may be psychologically beneficial, as it is related to greater well-being.
Article
Full-text available
This thesis investigates what big data can add to the psychological study of human behaviour; and how Psychological theory can inform developments in machine learning models predicting human behaviour. It works through the difficulties that arise when the fields of machine learning and psychology meet. While machine learning models deal well with big datasets, they are designed for prediction, neglecting psychologists' desire to, not just predict, but understand behaviour. Psychology does well at using theory to specify models and explain the variance within a sample, yet can fail to consider how transferable the findings are to new samples. This research harnesses over a million loyalty card transaction records from a high-street health and beauty retailer linked to 12,968 questionnaire responses measuring demographics, shopping motivations, and individual differences. Equipped with real world behavioural records, and information on potential psychological and demographic drivers of behaviour, this thesis explores the ways in which psychological research can be undergone using big data to better understand three main areas: well-being, environmental behaviours, and anxiety symptoms. This thesis has the goal of marrying the strengths of traditional psychological methodology (utilising theoretical knowledge, quantifying uncertainty, and building interpretable models) with the exciting possibilities afforded by big data, all whilst ensuring that the models are generalisable and do not overfit. The following chapters discuss and evaluate novel research in this space, as well as the difficulties encountered, and compromises made, in undertaking `Big Data Psychology’.
Article
Full-text available
Regional differences in personality are associated with a range of consequential outcomes. But which factors are responsible for these differences? Frontier settlement theory suggests that physical topography is a crucial factor shaping the psychological landscape of regions. Hence, we investigated whether topography is associated with regional variation in personality across the United States (n = 3,387,014). Consistent with frontier settlement theory, results from multilevel modelling revealed that mountainous areas were lower on agreeableness, extraversion, neuroticism and conscientiousness but higher on openness to experience. Conditional random forest algorithms confirmed mountainousness as a meaningful predictor of personality when tested against a conservative set of controls. East–west comparisons highlighted potential differences between ecological (driven by physical features) and sociocultural (driven by social norms) effects of mountainous terrain.
Article
Full-text available
Facebook is rapidly gaining recognition as a powerful research tool for the social sciences. It constitutes a large and diverse pool of participants, who can be selectively recruited for both online and offline studies. Additionally, it facilitates data collection by storing detailed records of its users' demographic profiles, social interactions, and behaviors. With participants' consent, these data can be recorded retrospectively in a convenient, accurate, and inexpensive way. Based on our experience in designing, implementing, and maintaining multiple Facebook-based psychological studies that attracted over 10 million participants, we demonstrate how to recruit participants using Facebook, incentivize them effectively, and maximize their engagement. We also outline the most important opportunities and challenges associated with using Facebook for research, provide several practical guidelines on how to successfully implement studies on Facebook, and finally, discuss ethical considerations. (PsycINFO Database Record
Article
Full-text available
Short and rapid publication of research findings has many advantages. However, there is another side of the coin that needs careful consideration. We argue that the most dangerous aspect of a shift toward "bite-size" publishing is the relationship between study size and publication bias. Findings based on a single study or a study based on a limited sample size are more likely to be false positive, because the false positive rate remains constant, whereas the true positive rate (the power) declines as sample size declines. Pressure on productivity and on novelty value further exacerbates the problem. © Association for Psychological Science 2012.
Article
Full-text available
There is substantial evidence in the psychology and sociology literature that social relationships promote happiness for the individual. Yet the size of their impacts remains largely unknown. This paper explores the use of shadow pricing method to estimate the monetary values of the satisfaction with life gained by an increase in the frequency of interaction with friends, relatives, and neighbours. Using the British Household Panel Survey, I find that an increase in the level of social involvements is worth up to an extra £85,000 a year in terms of life satisfaction. Actual changes in income, on the other hand, buy very little happiness.
Article
INTRODUCTION: Reproducibility is a defining feature of science, but the extent to which it characterizes current research is unknown. Scientific claims should not gain credence because of the status or authority of their originator but by the replicability of their supporting evidence. Even research of exemplary quality may have irreproducible empirical findings because of random or systematic error. RATIONALE: There is concern about the rate and predictors of reproducibility, but limited evidence. Potentially problematic practices include selective reporting, selective analysis, and insufficient specification of the conditions necessary or sufficient to obtain the results. Direct replication is the attempt to recreate the conditions believed sufficient for obtaining a previously observed finding and is the means of establishing reproducibility of a finding with new data. We conducted a large-scale, collaborative effort to obtain an initial estimate of the reproducibility of psychological science. RESULTS: We conducted replications of 100 experimental and correlational studies published in three psychology journals using high-powered designs and original materials when available. There is no single standard for evaluating replication success. Here, we evaluated reproducibility using significance and P values, effect sizes, subjective assessments of replication teams, and meta-analysis of effect sizes. The mean effect size (r) of the replication effects (Mr = 0.197, SD = 0.257) was half the magnitude of the mean effect size of the original effects (Mr = 0.403, SD = 0.188), representing a substantial decline. Ninety-seven percent of original studies had significant results (P < .05). Thirty-six percent of replications had significant results; 47% of original effect sizes were in the 95% confidence interval of the replication effect size; 39% of effects were subjectively rated to have replicated the original result; and if no bias in original results is assumed, combining original and replication results left 68% with statistically significant effects. Correlational tests suggest that replication success was better predicted by the strength of original evidence than by characteristics of the original and replication teams. CONCLUSION: No single indicator sufficiently describes replication success, and the five indicators examined here are not the only ways to evaluate reproducibility. Nonetheless, collectively these results offer a clear conclusion: A large portion of replications produced weaker evidence for the original findings despite using materials provided by the original authors, review in advance for methodological fidelity, and high statistical power to detect the original effect sizes. Moreover, correlational evidence is consistent with the conclusion that variation in the strength of initial evidence (such as original P value) was more predictive of replication success than variation in the characteristics of the teams conducting the research (such as experience and expertise). The latter factors certainly can influence replication success, but they did not appear to do so here. Reproducibility is not well understood because the incentives for individual scientists prioritize novelty over replication. Innovation is the engine of discovery and is vital for a productive, effective scientific enterprise. However, innovative ideas become old news fast. Journal reviewers and editors may dismiss a new test of a published idea as unoriginal. The claim that “we already know this” belies the uncertainty of scientific evidence. Innovation points out paths that are possible; replication points out paths that are likely; progress relies on both. Replication can increase certainty when findings are reproduced and promote innovation when they are not. This project provides accumulating evidence for many findings in psychological research and suggests that there is still more work to do to verify whether we know what we think we know.
Article
Reproducibility is a defining feature of science, but the extent to which it characterizes current research is unknown. We conducted replications of 100 experimental and correlational studies published in three psychology journals using high-powered designs and original materials when available. Replication effects were half the magnitude of original effects, representing a substantial decline. Ninety-seven percent of original studies had statistically significant results. Thirty-six percent of replications had statistically significant results; 47% of original effect sizes were in the 95% confidence interval of the replication effect size; 39% of effects were subjectively rated to have replicated the original result; and if no bias in original results is assumed, combining original and replication results left 68% with statistically significant effects. Correlational tests suggest that replication success was better predicted by the strength of original evidence than by characteristics of the original and replication teams.
Article
Describes experiments in which happy or sad moods were induced in Ss by hypnotic suggestion to investigate the influence of emotions on memory and thinking. Results show that (a) Ss exhibited mood-state-dependent memory in recall of word lists, personal experiences recorded in a daily diary, and childhood experiences; (b) Ss recalled a greater percentage of those experiences that were affectively congruent with the mood they were in during recall; (c) emotion powerfully influenced such cognitive processes as free associations, imaginative fantasies, social perceptions, and snap judgments about others' personalities; (d) when the feeling-tone of a narrative agreed with the reader's emotion, the salience and memorability of events in that narrative were increased. An associative network theory is proposed to account for these results. In this theory, an emotion serves as a memory unit that can enter into associations with coincident events. Activation of this emotion unit aids retrieval of events associated with it; it also primes emotional themata for use in free association, fantasies, and perceptual categorization. (54 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
A study with low statistical power has a reduced chance of detecting a true effect, but it is less well appreciated that low power also reduces the likelihood that a statistically significant result reflects a true effect. Here, we show that the average statistical power of studies in the neurosciences is very low. The consequences of this include overestimates of effect size and low reproducibility of results. There are also ethical dimensions to this problem, as unreliable research is inefficient and wasteful. Improving reproducibility in neuroscience is a key priority and requires attention to well-established but often ignored methodological principles.
Article
Discusses the paradoxical discrepancy involved in the statistical calculation of the percentage of variation in batting performance that is attributable to skill differentials among major-league baseball players, which is discrepant with intuitions about the influence of skill in batting performance. This discrepancy is discussed in terms of habits of thought about variance explanation. It is argued that percentage variance explanation is a misleading index of the influence of systematic factors in cases where there are processes by which influences cumulate to produce meaningful outcomes. (6 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)