An empirical study of sample sizes in Experimental Psychology

Practical guide on how to conduct online experiments

People across the world and throughout history have gone to great lengths to enhance their physical appearance. Evolutionary psychologists and ethologists have largely attempted to explain this phenomenon via mating preferences and strategies. Here, we test one of the most popular evolutionary hypotheses for beauty-enhancing behaviors, drawn from mating market and parasite stress perspectives, in a large cross-cultural sample. We also test hypotheses drawn from other influential and non-mutually exclusive theoretical frameworks, from biosocial role theory to a cultural media perspective. Survey data from 93,158 human participants across 93 countries provide evidence that behaviors such as applying makeup or using other cosmetics, hair grooming, clothing style, caring for body hygiene, and exercising or following a specific diet for the specific purpose of improving ones physical attractiveness, are universal. Indeed, 99% of participants reported spending >10 min a day performing beauty-enhancing behaviors. The results largely support evolutionary hypotheses: more time was spent enhancing beauty by women (almost 4 h a day, on average) than by men (3.6 h a day), by the youngest participants (and contrary to predictions, also the oldest), by those with a relatively more severe history of infectious diseases, and by participants currently dating compared to those in established relationships. The strongest predictor of attractiveness-enhancing behaviors was social media usage. Other predictors, in order of effect size, included adhering to traditional gender roles, residing in countries with less gender equality, considering oneself as highly attractive or, conversely, highly unattractive, TV watching time, higher socioeconomic status, right-wing political beliefs, a lower level of education, and personal individualistic attitudes. This study provides novel insight into universal beauty-enhancing behaviors by unifying evolutionary theory with several other complimentary perspectives.

Undertaking an experience-sampling study via smartphones is complex. Scheduling and sending mobile notifications often requires the use of proprietary software that imposes limits on participants’ operating systems (whether iOS or Android) or the types of questions that can be asked via the application. We have developed an open-source platform— Samply— which overcomes these limitations. Researchers can access the entire interface via a browser, manage studies, schedule and send notifications linking to online surveys or experiments created in any Internet-based service or software, and monitor participants' responses—all without the coding skills usually needed to program a native mobile application. Participants can download the Samply Research mobile application for free from Google Play or the App Store, join a specific study, receive notifications and web links to surveys or experiments, and track their involvement. The mobile application leverages the power of the React Native JavaScript library, which allows it to be rendered in the native code of Android and iOS mobile operating systems. We describe Samply , provide a step-by-step example of conducting an experience-sampling study, and present the results of two validation studies. Study 1 demonstrates how we improved the website’s usability for researchers. Study 2 validates the mobile application’s data recording ability by analyzing a survey’s participation rate. The application’s possible limitations and how mobile device settings might affect its reliability are discussed.

The present article reviews web-based research in psychology. It captures principles, learnings, and trends in several types of web-based research that show similar developments related to web technology and its major shifts (e.g., appearance of search engines, browser wars, deep web, commercialization, web services, HTML5…) as well as distinct challenges. The types of web-based research discussed are web surveys and questionnaire research, web-based tests, web experiments, Mobile Experience Sampling, and non-reactive web research, including big data. A number of web-based methods are presented and discussed that turned out to become important in research methodology. These are one-item-one-screen design, seriousness check, instruction manipulation and other attention checks, multiple site entry technique, subsampling technique, warm-up technique, and web-based measurement. Pitfalls and best practices are described then, especially regarding dropout and other non-response, recruitment of participants, and interaction between technology and psychological factors. The review concludes with a discussion of important concepts that have developed over 25 years and an outlook on future developments in web-based research.