John Protzko

Central Connecticut State University | CCSU

The replication crisis in psychology and related sciences contributed to the adoption of large-scale research initiatives known as Big Team Science (BTS). BTS has made significant advances in addressing issues of replication, statistical power, and diversity through the use of larger samples and more representative cross-cultural data. However, whi...

Measurement involves numerous theoretical and empirical steps—ensuring our measures are operating the same in different groups is one step. Measurement invariance occurs when the factor loadings and item intercepts or thresholds of a scale operate similarly for people at the same level of the latent variable in different groups. This is commonly as...

While meditation research has primarily focused on mental health outcomes and immediate cognitive and affective benefits, little is known about its influence on higher-order beliefs, such as views on free will and personal agency. Despite substantial references to free will and related concepts in Buddhist philosophy, to our knowledge, there has be...

Terror Management Theory (TMT) proposes that when people are made aware of their own death, they are more likely to endorse cultural values. TMT is a staple of social psychology, being featured prominently in textbooks and the subject of much research. The implications associated with TMT are significant, as its advocates claim it can partially exp...

Background: The Psychological Science Accelerator (PSA) recently completed a large-scale moral psychology study using translated versions of the Oxford Utilitarianism Scale (OUS). However, the translated versions have no validity evidence. Objective: The study investigated the structural validity evidence of the OUS across 15 translated versions an...

Given how commonly GPS is now used in everyday navigation, it is surprising how little research has been dedicated to investigating variations in its use and how such variations may relate to navigation ability. The present study investigated general GPS dependence, how people report using GPS in various navigational scenarios, and the relationship...

According to cognitive-dissonance theory, performing counterattitudinal behavior produces a state of dissonance that people are motivated to resolve, usually by changing their attitude to be in line with their behavior. One of the most popular experimental paradigms used to produce such attitude change is the induced-compliance paradigm. Despite it...

Failures to replicate evidence of new discoveries have forced scientists to ask whether this unreliability is due to suboptimal implementation of methods or whether presumptively optimal methods are not, in fact, optimal. This paper reports an investigation by four coordinated laboratories of the prospective replicability of 16 novel experimental f...

Introduction Throughout history, technological and societal changes consistently receive suspicion. Their influences appear damaging, corrupting, and potential precursors to societal downfall, with today’s youth often portrayed as the primary victims. This study aims to explore an underlying reason for these perceptions and to investigate why socie...

Do you persist as the same person over time because you keep the same mind or because you keep the same body? Philosophers have long investigated this question of personal identity with thought experiments. Cognitive scientists have joined this tradition by assessing lay intuitions about those cases. Much of this work has focused on judgments of id...

Open Science is a movement to get researchers to adopt reforms such a preregistration of analysis plans, materials and data being publicly posted, analysis plans being described and made open before data collection. While a noble movement, it requires adoption by the next generation of researchers to continue to grow. One place students learn to co...

Big Team Science involves dozens if not hundreds of independent research labs from potentially all over the world coming together to run the same study. The power with which these studies tackle issues of replicability and generalizability are unprecedented in Psychological science. Here, we discuss new places Big Team Science can go on top of inve...

People’s confidence in their face memory, both generally and in specific cases, is somewhat calibrated to their ability—but not strongly. Investigating N=23,893 American adults, we show those most confident in their judgments (state confidence) are less accurate than those who are slightly less confident. This occurs due to a unique nonlinearity in...

How well can social scientists predict societal change, and what processes underlie their predictions? To answer these questions, we ran two forecasting tournaments testing the accuracy of predictions of societal change in domains commonly studied in the social sciences: ideological preferences, political polarization, life satisfaction, sentiment...

Survey researchers take great care to measure respondents’ answers in an unbiased way; but, how successful are we as a field at remedying unintended and intended biases in our research? The validity of inferences drawn from studies has been found to be improved by the implementation of preregistration practices. Despite this, only 3 of the 83 publi...

In the majority of moral decision-making research, we are asked to consider the action of someone we know little about—an anonymous actor. This is inconsistent with our everyday judgments of the actions of others. Here we test the novel prediction of whether actions are considered as comparably virtuous or malignant when performed by a good person,...

Verbalizing visual memories can interfere with later accurate recall. Whereas changes in the magnitude of this verbal overshadowing effect (VOE) as a function of delay have been reported, no study has systematically investigated multiple shorter nonimmediate delays. Does VOE happen when verbalization occurs 5-min postencoding? 10 min? 15 min? We sh...

How well can social scientists predict societal change, and what processes underlie their predictions? To answer these questions, we ran two forecasting tournaments testing accuracy of predictions of societal change in domains commonly studied in the social sciences: ideological preferences, political polarization, life satisfaction, sentiment on s...

Research using online research participants is here to stay. Understanding who these subjects are, and specifically what kind of biases and selection mechanisms they bring to our research studies will be important for the future of human-subjects research as our studies rely more and more on the Online Volunteer Subject. Here we extend a method val...

Throughout history, new technological/societal changes have been viewed with suspicion, their influences interpreted as damaging, corrupting, a potential cause of societal downfall. The youth of the day are particularly seen as the victims of such developments. Why do we keep seeing technology/societal change as being harmful to young people? Two s...

Survey researchers take great care to measure and report respondents’ answers in an unbiased way; but, how well are we as a field at remedying our own unintended and intended biases in our research? The validity of inferences drawn from studies has been found to be improved by the implementation of preregistration practices. Despite this, only thre...

This initiative examined systematically the extent to which a large set of archival research findings generalizes across contexts. We repeated the key analyses for 29 original strategic management effects in the same context (direct reproduction) as well as in 52 novel time periods and geographies; 45% of the reproductions returned results matching...

The relation between religiosity and well-being is one of the most researched topics in the psychology of religion, yet the directionality and robustness of the effect remains debated. Here, we adopted a many-analysts approach to assess the robustness of this relation based on a new cross-cultural dataset (N = 10, 535 participants from 24 countries...

"It is a common belief that the attention of new generations of children is in decline (Protzko & Schooler, 2019). However, such devastating claims about new generations are challenged when inspecting the evidence (Protzko, 2020). The current meta-analysis aimed to explore comprehensively whether attention problems have increased, decreased, or rem...

Ever since some scientists and popular media put forward the idea that free will is an illusion, the question has risen what would happen if people stopped believing in free will. Psychological research has investigated this question by testing the consequences of experimentally weakening people’s free will beliefs. The results of these investigati...

The study of moral judgements often centres on moral dilemmas in which options consistent with deontological perspectives (that is, emphasizing rules, individual rights and duties) are in conflict with options consistent with utilitarian judgements (that is, following the greater good based on consequences). Greene et al. (2009) showed that psychol...

Adults perceive the youth of the present as being worse than from when they were young. This phenomenon has been shown to be a product of a memory bias, adults are unable to accurately recall what children were like in the past so they impose their current selves onto their memories. In two studies using American adults ( N = 2,764), we seek to con...

Measurement in Psychology is a complicated process involving numerous theoretical and empirical steps. Ensuring our measures are operating the same way in different groups of people is a further step referred to as Measurement Invariance testing. Measurement Invariance occurs when the factor loadings and item intercepts or thresholds of a scale ope...

Verbalizing visual memories can interfere with later accurate recall. Whereas changes in the magnitude of this Verbal Overshadowing effect as a function of delay have been reported, no study has systematically investigated multiple shorter non-immediate delays. Does VOE happen when verbalization occurs 5-minutes post-encoding? 10-minutes? 15-minute...

Our basic beliefs about reality can be impossible to prove and yet we can feel a strong intuitive conviction about them, as exemplified by insights that imbue an idea with immediate certainty. Here we presented participants with worldview beliefs such as “people’s core qualities are fixed” and simultaneously elicited an aha moment. In the first exp...

Do you persist as the same person over time because you keep the same mind or because you keep the same body? Philosophers have long investigated this question of personal identity with thought experiments. Cognitive scientists have joined this tradition by assessing lay intuitions about those cases. Much of this work has focused on judgments of id...

Dehumanization has played a prominent role in myriad human atrocities, which inspired us to investigate its social–cognitive basis. Since dehumanization consists of perceiving another to lack a defining human essence, scholars have suggested the process is grounded in psychological essentialism, the belief that members of a group all share an under...

Here we examine three classes of models regarding the structure of human cognition: common cause models, sampling/network models, and interconnected models. That disparate models can accommodate one of the most globally replicated psychological phenomena—namely, the positive manifold—is an extension of underdetermination of theory by data. Statisti...

Early life interventions impacting cognitive abilities are most often followed by post-treatment fadeout. Some have hypothesized that persistence is unlikely when gains are specific to trained skills and distinguishable from impacts on general cognitive ability (classically modeled as a hierarchical factor, so-called psychometric g). Using measurem...

Here we present several points for designing a probable playground concerning a new beginning of intelligence research within the XXI Century: the nature, definition, and measurement of the construct of interest, its development across the lifespan, its enhancement by varied means, and its place within the already identified human psychological tra...

In this crowdsourced initiative, independent analysts used the same dataset to test two hypotheses regarding the effects of scientists' gender and professional status on verbosity during group meetings. Not only the analytic approach but also the operationalizations of key variables were left unconstrained and up to individual analysts. For instanc...

Our basic beliefs about reality can be impossible to prove and yet we can feel a strong intuitive conviction for them, as exemplified by insights that imbue an idea with immediate certainty. Here we presented participants with worldviews such as “people’s core qualities are fixed”, and simultaneously elicited an aha moment. In the first experiment...

Our basic beliefs about reality can be impossible to prove and yet we can feel a strong intuitive conviction for them, as exemplified by insights that imbue an idea with immediate certainty. Here we presented participants with worldviews such as “people’s core qualities are fixed”, and simultaneously elicited an aha moment. In the first experiment...

Our basic beliefs about reality can be impossible to prove and yet we can feel a strong intuitive conviction for them, as exemplified by insights that imbue an idea with immediate certainty. Here we presented participants with worldviews such as “people’s core qualities are fixed”, and simultaneously elicited an aha moment. In the first experiment...

How can we maximize what is learned from a replication study? In the creative destruction approach to replication, the original hypothesis is compared not only to the null hypothesis, but also to predictions derived from multiple alternative theoretical accounts of the phenomenon. To this end, new populations and measures are included in the design...

How can we maximize what is learned from a replication study? In the creative destruction approach to replication, the original hypothesis is compared not only to the null hypothesis, but also to predictions derived from multiple alternative theoretical accounts of the phenomenon. To this end, new populations and measures are included in the design...

How can we maximize what is learned from a replication study? In the creative destruction approach to replication, the original hypothesis is compared not only to the null hypothesis, but also to predictions derived from multiple alternative theoretical accounts of the phenomenon. To this end, new populations and measures are included in the design...

The One-Group Pretest-Posttest Design, where the same group of people is measured before and after some event, can be fraught with statistical problems and issues with causal inference. Still, these designs are common from political science to developmental neuropsychology to economics. In cases with cognitive data, it has long been known that a se...

Whether free will exists is a longstanding philosophical debate. Cognitive neuroscience and popular media have been putting forward the idea that free will is an illusion, raising the question of what would happen if people stopped believing in free will altogether. Psychological research has investigated this question by testing the consequences o...

Over the past 10 years, Oosterhof and Todorov’s valence–dominance model has emerged as the most prominent account of how people evaluate faces on social dimensions. In this model, two dimensions (valence and dominance) underpin social judgements of faces. Because this model has primarily been developed and tested in Western regions, it is unclear w...

People who know more about science are more interested in science and related fields and show more positive attitudes towards science and scientists. This covariation may be important for advocating the beneficial effects of continued science education in the American public. This covariance, however, may be confounded by methodological artifacts o...

Failures to replicate evidence of new discoveries have forced scientists to ask whether this unreliability is due to suboptimal implementation of optimal methods or whether presumptively optimal methods are not, in fact, optimal. This paper reports an investigation by four coordinated laboratories of the prospective replicability of 16 novel experi...

People often fail to keep their mind from wandering. Here, we examine how the tendency to mind wander is affected by people’s beliefs, or lay theories. Building on research on lay theories and self-regulation, we test whether differences in people’s beliefs about the extent to which mind wandering is controllable affect thought control strategies a...

Daydreaming—engaging in an internal stream of thought unrelated to the here and now—is often linked to creativity; but not all daydreams are creative or lead to creative ideas. To better understand the relationship between daydreaming and creativity, we distinguish between types of daydreaming that differ in style or content (future planning, pleas...

When do people believe that biological explanations, such as genes, brain damage, or abnormal hormones, mitigate punishment for crimes? We propose the way in which biology is viewed as impacting the true self of the actor—who the actor really is, deep down—is the key element for predicting biologically-based mitigation. Across four preregistered st...

Early life interventions impacting cognitive abilities are most often followed by post-treatment fadeout. Some have hypothesized that persistence is unlikely when gains are specific to trained skills, or specifically, distinguishable from impacts on psychometric g. Using measurement invariance, we investigated impacts on IQ subtests from the Abeced...

What explanation is there when teams of researchers are unable to successfully replicate already established ‘canonical’ findings? One suggestion that has been put forward, but left largely untested, is that those researchers who fail to replicate prior studies are of low ‘expertise and diligence’ and lack the skill necessary to successfully replic...

Over the last ten years, Oosterhof and Todorov’s valence-dominance model has emerged as the most prominent account of how people evaluate faces on social dimensions. In this model, two dimensions (valence and dominance) underpin social judgments of faces. Because this model has primarily been developed and tested in Western regions, it is unclear w...

Prosociality increases when decisions are made under time pressure. Here, we investigated whether time pressure increases socially desirable responding outside social interactions (Study 1). Finding that it did, we then examined whether this is because people align their responses with the concept of their “true” self or because of an intuitive ten...

In five preregistered studies, we assess people’s tendency to believe “kids these days” are deficient relative to those of previous generations. Across three traits, American adults ( N =3,458; Mage = 33-51 years) believe today’s youth are in decline; however, these perceptions are associated with people’s standing on those traits. Authoritarian pe...

How, if at all, do scientists adjust their beliefs based on observed evidence? Few studies have examined how beliefs change over the course of a study, from developing the protocol to collecting and analyzing original data. N scientists who were contributing to one or more of six multi-lab replication projects were asked to estimate their degree of...

Much research on moral judgment is centered on moral dilemmas in which deontological perspectives (i.e., emphasizing rules, individual rights and duties) are in conflict with utilitarian judgements (i.e., following the greater good defined through consequences). A central finding of this field Greene et al. showed that psychological and situational...

Mental simulation theories of language comprehension propose that people automatically create mental representations of real objects. Evidence from sentence-picture verification tasks has shown that people mentally represent various visual properties such as shape, color, and size. However, the evidence for mental simulations of object orientation...

Concerns about the veracity of psychological research have been growing. Many findings in psychological science are based on studies with insufficient statistical power and nonrepresentative samples, or may otherwise be limited to specific, ungeneralizable settings or populations. Crowdsourced research, a type of large-scale collaboration in which...

The oldest method in psychology of trying to gain access to one part of a divided mind is to instruct participants to answer quickly. Here we propose an alternative account for this procedure, namely, that it makes people give the socially desirable response. We randomly assigned 1,500 Americans to answer a social desirability scale either quickly...

Pre-registration of analysis plans involves making data-analysis decisions before the data is run in order to prevent flexibly re-running it until a specific result appears (p-hacking). Just because a model and results is pre-registered does not make it true, however. The complement to p-hacking, null-hacking, is the use of the same questionable re...

Concerns have been growing about the veracity of psychological findings. Many findings in psychological science are based on studies with insufficient statistical power and non-representative samples, or may otherwise be limited to specific, ungeneralizable settings or populations. Large-scale collaboration, in which one or more research projects a...

Despite the miraculous progress of science—it’s practitioners continue to run into mistakes, either discrediting research unduly or making leaps of causal inference where none are warranted. In this we isolate two of the reasons for such behavior involving the misplaced understanding of the role of mechanisms and mechanistic knowledge in the establ...

In this paper, we examine nearly every available randomized controlled trial that attempts to raise IQ in children from once they begin kindergarten until pre-adolescence. We use meta-analytic procedures when there are more than three studies employing similar methods, reviewing individual interventions when too few replications are available for a...

How does belief in free will affect altruistic behavior? In an online experiment we undermine subjects’ belief in free will through a priming task. Subjects subsequently conduct a series of binary dictator games in which they can distribute money between themselves and a charity that supports low-income people in developing countries. In each decis...

Experimental instructions. (DOCX)

This chapter considers four general types of declining effect sizes, include false positive decline effects, inflated decline effects, under-specified decline effects, and genuinely decreasing decline effects, each of which relates to the hypothetical true effect size of the finding in question at the time it was originally reported. Publication pr...

What explanation is there when teams of researchers are unable to successfully replicate already established ‘canonical’ findings? One suggestion put forward, but left largely untested, has been that those researchers who fail to replicate other studies are of lower ‘caliber’, lacking the expertise and skill necessary to successfully replicate such...

Alpha waves correlate with intelligence. Closing your eyes has been found to be an artifact in EEG research, whereby when a participant closes their eyes there is a spike in EEG alpha activity. If the correlation of alpha waves to intelligence is causal, then increasing alpha waves should increase intelligence. Therefore, people may be more intelli...

Targeted cognitive training, such as n-back or speed of processing training, in the hopes of raising intelligence is of great theoretical and practical importance. The most important theoretical contribution, however, is not about the malleability of intelligence. Instead, I argue the most important and novel theoretical contribution is understandi...

Science magazine #NextGenSci Six word stories.

We attempted to replicate a self-affirmation intervention that produced a 40% reduction in the academic achievement gap among at-risk students. The intervention was designed as a protection against stereotype threat—which creates stress and suppresses the performance, engagement, and learning of students stereotyped as intellectually inferior. In p...

Alpha power correlates with psychometric intelligence. Closing your eyes creates an artifactual spike in alpha. Therefore, if the alpha to intelligence correlation is causal, people should be more intelligent when they close their eyes. We tested this on N=96 participants randomly assigned to a fluid intelligence subtest with their eyes either open...