Daniël Lakens’s research while affiliated with Eindhoven University of Technology and other places

Despite the increasing recognition for the scientific and societal potential of interdisciplinary research, selection committees struggle with the evaluation of interdisciplinary proposals. Interdisciplinary proposals include a wider range of theories and methods, involve a more diverse team, pose a higher level of uncertainty, and their evaluation requires expertise from multiple disciplines. In this study, we investigate the possibility to support the evaluation of interdisciplinary research proposals with measures of interdisciplinary research quality. Based on the literature, we curated a set of qualitative criteria and bibliometric indicators. Subsequently, we examined their feasibility using interviews with interdisciplinary researchers and a re-assessment session of a grant-allocation procedure. In the re-assessment session members of an original evaluation panel assessed four original research proposals again, but now supported with our measures. This study confirmed the potential of qualitative criteria to assess the interdisciplinarity or research proposals. These indicators helped to make explicit what different people mean with interdisciplinary research, which improved the quality of the discussions and decision-making. The utility of bibliometric indicators turned out to be limited, due to technical limitations and concerns about unintended side effects.

In this paper we test the content validity of 246 academic research value items in two interrelated methodological steps. First, we analyze the in-depth evaluation of two experts to assess formal aspects of our items and their relatedness to the construct. Next, we evaluate the relevance of each item based on feedback gathered from 20 experts. Based on these assessments, we review and refine the original item pool and propose a total of 97 value items spread through 10 dimensions. We relate our items to existing work on researchers' values and provide recommendations for future measurement development. A validated measure of academic research values can assess the effectiveness of responsible research conduct courses and highlight personal differences among researchers from diverse backgrounds, as well as aid our understanding of values unique to academic careers.

Affective responses during stressful, high-stakes situations can play an important role in shaping performance. For example, feeling shaky and nervous at a job interview can undermine performance, whereas feeling excited during that same interview can optimize performance. Thus, affect regulation—the way people influence their affective responses—might play a key role in determining high-stakes outcomes. To test this idea, we adapted a synergistic mindsets intervention (SMI) (Yeager et al. 2022 Nature 607, 512–520 (doi:10.1038/s41586-022-04907-7)) to a high-stakes esports context. Our approach was motivated by the idea that (i) mindsets both about situations and one’s stress responses to situations can be shaped to help optimize stress responses, and (ii) challenge versus threat stress responses will be associated with improved outcomes. After a baseline performance task, we randomly assigned gamers (n = 300) either to SMI or a control condition in which they learned brain facts. After two weeks of daily gaming, gamers competed in a cash-prize tournament. We measured affective experiences before the matches and cardiovascular responses before and throughout the matches. Contrary to predictions, gamers did not experience negative affect (including feeling stressed), thus limiting the capacity for the intervention to regulate physiological responses and optimize performance. Compared with the control participants, synergistic mindsets participants did not show greater challenge responses or improved performance outcomes. Though our adaptation of Yeager et al.’s SMI did not optimize esports performance, our findings point to important considerations regarding the suitability of an intervention such as this to different performance contexts of varying degrees of stressfulness.

When analyzing data, researchers make some choices that are either arbitrary, based on subjective beliefs about the data-generating process, or for which equally justifiable alternative choices could have been made. This wide range of data-analytic choices can be abused and has been one of the underlying causes of the replication crisis in several fields. Recently, the introduction of multiverse analysis provides researchers with a method to evaluate the stability of the results across reasonable choices that could be made when analyzing data. Multiverse analysis is confined to a descriptive role, lacking a proper and comprehensive inferential procedure. Recently, specification curve analysis adds an inferential procedure to multiverse analysis, but this approach is limited to simple cases related to the linear model, and only allows researchers to infer whether at least one specification rejects the null hypothesis, but not which specifications should be selected. In this paper, we present a Post-selection Inference approach to Multiverse Analysis (PIMA) which is a flexible and general inferential approach that considers for all possible models, i.e., the multiverse of reasonable analyses. The approach allows for a wide range of data specifications (i.e., preprocessing) and any generalized linear model; it allows testing the null hypothesis that a given predictor is not associated with the outcome, by combining information from all reasonable models of multiverse analysis, and provides strong control of the family-wise error rate allowing researchers to claim that the null hypothesis can be rejected for any specification that shows a significant effect. The inferential proposal is based on a conditional resampling procedure. We formally prove that the Type I error rate is controlled, and compute the statistical power of the test through a simulation study. Finally, we apply the PIMA procedure to the analysis of a real dataset on the self-reported hesitancy for the COronaVIrus Disease 2019 (COVID-19) vaccine before and after the 2020 lockdown in Italy. We conclude with practical recommendations to be considered when implementing the proposed procedure.

From a methodological falsificationist philosophy of science it is desirable that scientific claims are severely tested. We review evidence about the self-reported prevalence of research practices that reduce the severity of tests. Such biasing selection effects are relatively common, although the extent to which they lead to biased claims is impossible to empirically establish. These practices are amplified by publication bias, where desired results are more likely to end up in the published literature. We explain how, from an error-statistical epistemology, the goal of preregistration is to allow peers to transparently evaluate how severely a claim has been tested, and how the goal of Registered Reports is to reduce publication bias. A review of metascientific research on preregistration and Registered Reports confirms both practices achieve these goals. The literature also reveals researchers perceive additional benefits of preregistration, mainly concerning the improved quality of their studies. We review criticisms on preregistration, and find conflicting viewpoints, several misunderstandings, and a general lack of empirical support. An exception is the valid concern that the mere presence of a preregistration will mindlessly be used as a proxy for high quality. We conclude with general recommendations to increase the quality and practice of preregistration.

Habits often conflict with goal-directed behaviors and this phenomenon continues to attract interests from neuroscientists, experimental psychologists, and applied health psychologists. Recent computational models explain habit-goal conflicts as the competitions between two learning systems, arbitrated by a central unit. Based on recent research that combined reinforcement learning and sequential sampling, we show that habit-goal conflicts can be more parsimoniously explained by a dynamic integration of habit and goal values in a sequential sampling model, without any arbitration. A computational model was developed by extending the multialternative decision field theory with the assumptions that habits bias starting points of preference accumulation, and that goal importance and goal relevance determine sampling probabilities of goal-related attributes. Simulation studies demonstrated our approach’s ability to qualitatively reproduce important empirical findings from three paradigms – classic devaluation, devaluation with a concurrent schedule, and reversal learning, and to predict gradual changes in decision times. In addition, a parameter recovery exercise using approximate Bayesian computation showcased the possibility of fitting the model to empirical data in future research. Implications of our work for habit theories and applications are discussed.