Dora Matzke

• PhD
• Professor (Assistant) at University of Amsterdam

We introduce EMC2, an R package for Bayesian hierarchical analysis of cognitive models of choice. EMC2 bridges the gap between standard regression analyses and cognitive modeling through linear-model specifications for each type of cognitive model parameter. The flexible implementation of the linear modeling language allows users to map model param...

For most researchers, academic publishing serves two goals that are often misaligned—knowledge dissemination and establishing scientific credentials. While both goals can encourage research with significant depth and scope, the latter can also pressure scholars to maximize publication metrics. Commercial publishing companies have capitalized on the...

The estimated latency of the unobservable stop response, the so-called stop-signal reaction time (SSRT), has been the established measure of performance in the stop-signal task. While it is currently debated whether SSRT is a suitable marker of inhibition performance, other markers such as the reliability of triggering the stop process (“stop trigg...

A variety of different evidence-accumulation models (EAMs) account for common response time and accuracy patterns in two-alternative forced choice tasks by assuming that subjects collect and sum information from their environment until a response threshold is reached. Estimates of model parameters mapped to components of this decision process can b...

Studying individual differences in psychology often involves examiningcorrelations across various measures. However, research involving high-dimensional data—such as in task batteries or neuroscience—often targetslatent constructs rather than individual correlations. Furthermore, the num-ber of correlations grows quadratically with increasing dimen...

In this study, we explored whether the key benchmarks of working memory processing identified in adults by Oberauer and colleagues (2018) also apply to children, using data from a large adaptive learning environment. Over nine thousand children from Dutch primary schools (age between 6 and 12) played two serial recall tasks (verbal domain and visuo...

Sequences of choice response times exhibit ubiquitous and strong multi-scale dynamics (i.e., sequential dependencies across a broad range of temporal scales). Despite their pervasive nature, multi-scale dynamics are poorly understood. We show that dynamics in the seconds to minutes range can be explained by the superposition of several distinct lea...

While decision theories have evolved over the past five decades, their focus has largely been on choices among a limited number of discrete options, even though many real-world situations have a continuous-option space. Recently, theories have attempted to address decisions with continuous-option spaces, and several computational models have been p...

Many-analysts studies explore how well an empirical claim withstands plausible alternative analyses of the same dataset by multiple, independent analysis teams. Conclusions from these studies typically rely on a single outcome metric (e.g. effect size) provided by each analysis team. Although informative about the range of plausible effects in a da...

There is a growing realization that experimental tasks that produce reliable effects in group comparisons can simultaneously provide unreliable assess- ments of individual differences. Proposed solutions to this “reliability para- dox” range from collecting more test trials to modifying the tasks and/or the way in which effects are measured from th...

Evidence accumulation models (EAMs) are powerful tools for making sense of human and animal decision-making behaviour. EAMs have generated significant theoretical advances in psychology, behavioural economics, and cognitive neuroscience, and are increasingly used as a measurement tool in clinical research and other applied settings. Obtaining valid...

Researchers have become increasingly aware that data-analysis decisions affect results. Here, we examine this issue systematically for multinomial processing tree (MPT) models, a popular class of cognitive models for categorical data. Specifically, we examine the robustness of MPT model parameter estimates that arise from two important decisions: t...

We review model-based neuroscience work on cognitive control of choices and actions. We consider both strategically deployed executive processes and more automatic influences, first in binary choice tasks and then in more complex tasks. These include “conflict” tasks, where automatic and executive control processes sometimes act in opposition; dela...

Researchers have become increasingly aware that data-analysis decisions affect results. Here, we examine this issue systematically for multinomial processing tree (MPT) models, a popular class of cognitive models for categorical data. Specifically, we examine the robustness of MPT model parameter estimates that arise from two important decisions: t...

We introduce EMC2, an R package for the Bayesian hierarchical analysisof cognitive models of choice. EMC2 bridges the gap between standardregression analyses and cognitive modeling through linear-model specifica-tions for each type of cognitive-model parameter. The flexible implemen-tation of the linear modelling language allows users to map model...

We introduce EMC2, an R package for the Bayesian hierarchical analysisof cognitive models of choice. EMC2 bridges the gap between standardregression analyses and cognitive modeling through linear-model specifica-tions for each type of cognitive-model parameter. The flexible implemen-tation of the linear modelling language allows users to map model...

Many-analysts studies explore how well an empirical claim withstands plausible alternative analyses of the same data set by multiple, independent analysis teams. Conclusions from these studies typically rely on a single outcome metric (e.g., effect size) provided by each analysis team. Although informative about the range of plausible effects in a...

Dynamic cognitive psychometrics measures mental capacities based on the way behavior unfolds over time. It does so using models of psychological processes whose validity is grounded in research from experimental psychology and the neurosciences. However, these models can sometimes have undesirable measurement properties. We propose a “hybrid” model...

The ability to stop simple ongoing actions has been extensively studied using the stop signal task, but less is known about inhibition in more complex scenarios. Here we used a task requiring bimanual responses to go stimuli, but selective inhibition of only one of those responses following a stop signal. We assessed how proactive cues affect the n...

Stopping an already initiated action is crucial for human everyday behavior and empirical evidence points toward the prefrontal cortex playing a key role in response inhibition. Two regions that have been consistently implicated in response inhibition are the right inferior frontal gyrus (IFG) and the more superior region of the dorsolateral prefro...

The ability to stop simple ongoing actions has been extensively studied using the stop signal task, but less is known about inhibition in more complex scenarios. Here we used a task requiring bimanual responses to go stimuli, but selective inhibition of only one of those responses following a stop signal. We assessed how proactive cues affect the n...

Standard, well-established cognitive tasks that produce reliable effects in group comparisons also lead to unreliable measurement when assessing individual differences. This reliability paradox has been demonstrated in decision-conflict tasks such as the Simon, Flanker, and Stroop tasks, which measure various aspects of cognitive control. We aim to...

Response inhibition and interference resolution are often considered subcomponents of an overarching inhibition system that utilizes the so-called cortico-basal-ganglia loop. Up until now, most previous functional magnetic resonance imaging (fMRI) literature has compared the two using between-subject designs, pooling data in the form of a meta-anal...

Standard, well-established cognitive tasks that produce reliable effects in group comparisons also lead to unreliable measurement when assessing individual differences. This reliability paradox has been demonstrated in decision-conflict tasks such as the Simon, Flanker, and Stroop tasks, which measure various aspects of cognitive control. We aim to...

Response inhibition is a key attribute of human executive control. Standard stop-signal tasks require countermanding a single response; the speed at which that response can be inhibited indexes the efficacy of the inhibitory control networks. However, more complex stopping tasks, where one or more components of a multi-component action are cancelle...

van Doorn et al. (2021) outlined various questions that arise when conducting Bayesian model comparison for mixed effects models. Seven response articles offered their own perspective on the preferred setup for mixed model comparison, on the most appropriate specification of prior distributions, and on the desirability of default recommendations. T...

van Doorn et al. (2021) outlined various questions that arise when conducting Bayesian model comparison for mixed effects models. Seven response articles offered their own perspective on the preferred setup for mixed model comparison, on the most appropriate specification of prior distributions, and on the desirability of default recommendations. T...

Interest in the processes that mediate between stimuli and responses is at the heart of most modern psychology and neuroscience. These processes cannot be directly measured but instead must be inferred from observed responses. Race models, through their ability to account for both response choices and response times, have been a key enabler of such...

Response inhibition is frequently measured using the stop-signal paradigm, where responses must be withheld when a “stop” signal appears. This paradigm assumes that go and stop stimuli trigger competing runners. The first runner crossing a boundary wins, and determines whether a response is performed. In this domain, as in many others, a tension ex...

We propose a new approach to cognitive aging research, in which detailed cognitive modelling of a single complex task affords simultaneous measures of the major mechanisms proposed to explain age-related deficits: capacity limits, processing speed, inhibition, and executive function. The validity of these measures rests on their well-defined roles...

The effects of distraction on responses manifest in three ways: prolonged reaction times, and increased error and response omission rates. However, the latter effect is often ignored or assumed to be due to a separate cognitive process. We investigated omissions occurring in two paradigms that manipulated distraction. One required simple stimulus d...

Human operators often experience large fluctuations in cognitive workload over seconds timescales that can lead to sub-optimal performance, ranging from overload to neglect. Adaptive automation could potentially address this issue, but to do so it needs to be aware of real-time changes in operators’ spare cognitive capacity, so it can provide help...

Any large dataset can be analyzed in a number of ways, and it is possible that the use of different analysis strategies will lead to different results and conclusions. One way to assess whether the results obtained depend on the analysis strategy chosen is to employ multiple analysts and leave each of them free to follow their own approach. Here, w...

The stop-signal paradigm has become ubiquitous in investigations of inhibitory control. Tasks inspired by the paradigm, referred to as stop-signal tasks, require participants to make responses on go trials and to inhibit those responses when presented with a stop-signal on stop trials. Currently, the most popular version of the stop-signal task is...

The “marginality principle” for linear regression models states that when a higher order term is included, its constituent terms must also be included. The target article relies on this principle for the fixed-effects part of linear mixed models of ANOVA designs and considers the implication that if extended to combined fixed-and-random-effects mod...

Roberts (2020, Learning & Behavior, 48 [2], 191–192) discussed research claiming honeybees can do arithmetic. Some readers of this research might regard such claims as unlikely. The present authors used this example as a basis for a debate on the criterion that ought to be used for publication of results or conclusions that could be viewed as unlik...

The Adolescent Brain Cognitive Development (ABCD) Study is a longitudinal neuroimaging study of unprecedented scale that is in the process of following over 11,000 youth from middle childhood though age 20. However, a design feature of the study's stop-signal task violates "context independence", an assumption critical to current non-parametric met...

Cognitive models provide a substantively meaningful quantitative description of latent cognitive processes. The quantitative formulation of these models supports cumulative theory building and enables strong empirical tests. However, the non-linearity of these models and pervasive correlations among model parameters pose special challenges when app...

The stop-signal paradigm has become ubiquitous in investigations of inhibitory control. Tasks inspired by the paradigm, referred to as stop-signal tasks, require participants to make responses on go trials and to inhibit those responses when presented with a stop-signal on stop trials. Currently, the most popular version of the stop-signal task is...

We present consensus-based guidance for conducting and documenting multi-analyst studies. We discuss why broader adoption of the multi-analyst approach will strengthen the robustness of results and conclusions in empirical sciences.

Human operators often experience large fluctuations in cognitive workload over seconds time scales that can lead to sub-optimal performance, ranging from overload to neglect. Adaptive automation could potentially address this issue, but to do so it needs to be aware of real-time changes in operators’ spare cognitive capacity, so it can provide help...

The ability to inhibit ongoing responses that suddenly become inappropriate is essential for safe and effective interaction with an ever-changing and often unpredictable world. This ability is quantified by the stop-signal reaction time (SSRT), the completion time of an inhibitory process triggered by a signal to stop responding. Because SSRT canno...

Parametric cognitive models are increasingly popular tools for analyzing data obtained from psychological experiments. One of the main goals of such models is to formalize psychological theories using parameters that represent distinct psychological processes. We argue that systematic quantitative reviews of parameter estimates can make an importan...

Parametric cognitive models are increasingly popular tools for analysing data obtained from psychological experiments. One of the main goals of such models is to formalize psychological theories using parameters that represent distinct psychological processes. We argue that systematic quantitative reviews of parameter estimates can make an importan...

This study investigates the neural correlates underpinning response inhibition using a parametric ex‐Gaussian model of stop‐signal task performance, fit with hierarchical Bayesian methods, in a large healthy sample (N = 156). The parametric model accounted for both stop‐signal reaction time (SSRT) and trigger failure (i.e., failures to initiate the...

Stephens, Matzke, and Hayes (SMH; 2019) used state-trace analysis to re-analyze databases of studies of reasoning and category learning. They found that many behavioral dissociations that had been viewed as support for distinct cognitive processes (or systems) were consistent with the operation of only one latent psychological variable. Ashby (2019...

The Peer Reviewers’ Openness (PRO) Initiative promotes the sharing of data and code. PRO signatories pledge to provide a full review only for manuscripts that publicly share data and code, or include a justification why sharing is not possible. Since the punitive element of this approach attracted criticism, we conducted a survey to assess signator...

Despite an ongoing stream of lamentations, many empirical disciplines still treat the p value as the sole arbiter to separate the scientific wheat from the chaff. The continued reign of the p value is arguably due in part to a perceived lack of workable alternatives. In order to be workable, any alternative methodology must be (1) relevant: it has...

The stop-signal paradigm is a popular procedure to investigate response inhibition—the ability to stop ongoing responses. It consists of a choice response time (RT) task that is occasionally interrupted by a stop stimulus signaling participants to withhold their response. Performance in the stop-signal paradigm is often formalized as race between a...

L’analyse de variance (ANOVA) est la procédure standard utilisée pour l’inférence statistique dans les plans factoriels. En règle générale, les analyses de variance sont exécutées à l’aide de statistiques fréquentistes, où les valeurs p déterminent la significativité statistique en termes de « tout ou rien ». Ces dernières années, l’approche bayési...

Over the last decade, the Bayesian estimation of evidence-accumulation models has gained popularity, largely due to the advantages afforded by the Bayesian hierarchical framework. Despite recent advances in the Bayesian estimation of evidence-accumulation models, model comparison continues to rely on suboptimal procedures, such as posterior paramet...

Analysis of variance (ANOVA) is the standard procedure for statistical inference in factorial designs. Typically, ANOVAs are executed using frequentist statistics, where p-values determine statistical significance in an all-or-none fashion. In recent years, the Bayesian approach to statistics is increasingly viewed as a legitimate alternative to th...

The target article on robust modeling (Lee et al. in review) generated a lot of commentary. In this reply, we discuss some of the common themes in the commentaries; some are simple points of agreement while others are extensions of a practical or abstract nature. We also address a small number of disagreements or confusions.

The target article on robust modeling (Lee et al.) generated a lot of commentary. In this reply, we discuss some of the common themes in the commentaries; some are simple points of agreement while others are extensions of a practical or abstract nature. We also address a small number of disagreements or confusions.

The article A SIMPLE METHOD FOR COMPARING COMPLEX MODELS:

Human operators often experience large fluctuations in cognitive workload that can lead to sub-optimal performance, ranging from overload to neglect. Help from automated support systems could potentially address this issue, but to do so the system would ideally need to be aware of real-time changes in operators’ cognitive workload, so it can provid...

In recent years, multiple initiatives have sought to improve the transparency and reproducibility of psychological research. One example is the Peer Reviewers’ Openness Initiative (PRO), which aims to promote the sharing of data and code. PRO signatories pledge to provide a full review only for manuscripts that publicly share data and code, or incl...

The current crisis of confidence in psychological science has spurred on field-wide reforms to enhance transparency, reproducibility, and replicability. To solidify these reforms within the scientific community, student courses on open science practices are essential. Here we describe the content of our Research Master course “Good Research Practic...

Despite an ongoing stream of lamentations, many empirical disciplines still treat the p-value as the sole arbiter to separate the scientific wheat from the chaff. The continued reign of the p-value is arguably due in part to a perceived lack of workable alternatives. In order to be workable, any alternative methodology must be (1) relevant: it has...

Sir Ronald Fisher’s venerable experiment “The Lady Tasting Tea” is revisited from a Bayesian perspective. We demonstrate how a similar tasting experiment, conducted in a classroom setting, can familiarize students with several key concepts of Bayesian inference, such as the prior distribution, the posterior distribution, the Bayes factor, and seque...

The current crisis of confidence in psychological science has spurred on fieldwide reforms to enhance transparency, reproducibility, and replicability. To solidify these reforms within the scientific community, student courses on open science practices are essential. Here we describe the content of our Research Master course “Good Research Practice...

Although advancements in electrophysiological methods to explore response inhibition have been substantial, the methods to describe behavioural differences in response inhibition have remained relatively unchanged. Here we use a model-based neuroscience approach to understand the neural correlates underpinning response inhibition as estimated using...

Response inhibition is essential for navigating everyday life. Its derailment is considered integral to numerous neurological and psychiatric disorders, and more generally, to a wide range of behavioral and health problems. Response-inhibition efficiency furthermore correlates with treatment outcome in some of these conditions. The stop-signal task...

Sir Ronald Fisher's venerable experiment "The Lady Tasting Tea'' is revisited from a Bayesian perspective. We demonstrate how a similar tasting experiment, conducted in a classroom setting, can familiarize students with several key concepts of Bayesian inference, such as the prior distribution, the posterior distribution, the Bayes factor, and sequ...

Motorists often engage in secondary tasks unrelated to driving that increase cognitive workload, resulting in fatal crashes and injuries. An International Standards Organization method for measuring a driver’s cognitive workload, the detection response task (DRT), correlates well with driving outcomes, but investigation of its putative theoretical...

Mean stop-signal reaction time (SSRT) is frequently employed as a measure of response inhibition in cognitive neuroscience research on attention deficit/hyperactivity disorder (ADHD). However, this measurement model is limited by two factors that may bias SSRT estimation in this population: (a) excessive skew in “go” RT distributions and (b) trigge...

In an attempt to increase the reliability of empirical findings, psychological scientists have recently proposed a number of changes in the practice of experimental psychology. Most current reform efforts have focused on the analysis of data and the reporting of findings for empirical studies. However, a large contingent of psychologists build mode...

Response inhibition is essential for navigating everyday life. Its derailment is considered integral to numerous neurological and psychiatric disorders, and more generally, to a wide range of behavioral and health problems. Response-inhibition efficiency furthermore correlates with treatment outcome in these conditions. The stop-signal task is an e...

People around the world endanger the lives of themselves and others every day by dividing their attention across multiple tasks, such as driving and talking on a cell phone. These dangers result from splitting and overtaxing our limited voluntary attentional efforts. Current tools for measuring attentional effort, also known as cognitive workload,...

Despite the increasing popularity of Bayesian inference in empirical research, few practical guidelines provide detailed recommendations for how to apply Bayesian procedures and interpret the results. Here we offer specific guidelines for four different stages of Bayesian statistical reasoning in a research setting: planning the analysis, executing...

In an attempt to increase the reliability of empirical findings, psychological scientists have recently proposed a number of changes in the practice of experimental psychology. Most current reform efforts have focused on the analysis of data and the reporting of findings for empirical studies. However, a large contingent of psychologists build mode...

The ability to control behaviour is thought to rely at least partly on adequately suppressing impulsive responses to external stimuli. However, the evidence for a relationship between response inhibition ability and impulse control is weak and inconsistent. This study investigates the relationship between response inhibition and both self-report an...

The stop-signal paradigm is a widely used procedure to study response inhibition. It consists of a 2-choice response-time task (a “go” task) that is occasionally interrupted by a stop signal instructing participants to withhold their responses. The paradigm owes its popularity to the underlying race model that enables estimation of the otherwise un...

This paper introduces JASP, a free graphical software package for basic statistical procedures such as t tests, ANOVAs, linear regression models, and analyses of contingency tables. JASP is open-source and differentiates itself from existing open-source solutions in two ways. First, JASP provides several innovations in user interface design; specif...

We outline a network method to synthesize a literature overview from search results obtained by multiple team members. Several network statistics are used to create a single representativeness ranking. We illustrate the method with the dispersed literature on a common misinterpretation of analysis of covariance (ANCOVA). The network method yields a...

For many years the Diffusion Decision Model (DDM) has successfully accounted for behavioral data from a wide range of domains. Important contributors to the DDM’s success are the across-trial variability parameters, which allow the model to account for the various shapes of response time distributions encountered in practice. However, several resea...

Dissociations have served as a key source of evidence for theory development in experimental psychology. Claims about the existence of multiple distinct psychological processes or systems are often based on demonstrations that manipulations such as working memory load, mood or instructions have differential effects on task performance. For example,...

Multinomial processing trees (MPTs) are a popular class of cognitive models for categorical data. Typically, researchers compare several MPTs, each equipped with many parameters, especially when the models are implemented in a hierarchical framework. A Bayesian solution is to compute posterior model probabilities and Bayes factors. Both quantities,...

The ability to control behaviour is thought to rely on adequately suppressing impulsive responses to external stimuli. However, the evidence for this relationship between response inhibition ability and impulse control is weak and inconsistent. This study investigates the relationship between response inhibition and both self-report and behavioural...

Over the last decade, the Bayesian estimation of evidence-accumulation models has gained popularity, largely due to the advantages afforded by the Bayesian hierarchical framework. Despite recent advances in the Bayesian estimation of evidence-accumulation models, model comparison continues to rely on suboptimal procedures, such as posterior paramet...

Parameter estimation in evidence-accumulation models of choice response times is demanding of both the data and the user. We outline how to fit evidence-accumulation models using the flexible, open-source, R-based Dynamic Models of Choice (DMC) software. DMC provides a hands-on introduction to the Bayesian implementation of two popular evidence-acc...

Psychological experiments often yield data that are hierarchically structured. A number of popular shortcut strategies in cognitive modeling do not properly accommodate this structure and can result in biased conclusions. To gauge the severity of these biases, we conducted a simulation study for a two-group experiment. We first considered a modelin...

Psychological experiments often yield data that are hierarchically structured. A number of popular shortcut strategies in cognitive modeling do not properly accommodate this structure and can result in biased conclusions. To gauge the severity of these biases we conducted a simulation study for a two-group experiment. We first considered a modeling...

Our understanding of the neural correlates of response inhibition has greatly advanced over the last decade. Nevertheless the specific function of regions within this stopping network remains controversial. The traditional neuroimaging approach cannot capture many processes affecting stopping performance. Despite the shortcomings of the traditional...

For many years the Diffusion Decision Model (DDM) has successfully accounted for behavioral data from a wide range of domains. Important contributors to the DDM’s success are the between-trial variability parameters, which allow the model to account for the various shapes of response time distributions encountered in practice. However, several rese...

Response inhibition—the ability to stop responses that are no longer appropriate—is frequently studied with the stop‐signal paradigm. In the stop‐signal paradigm, participants perform a choice response time task that is occasionally interrupted by a stop signal. The stop signal prompts participants to withhold their response on that trial. Performa...

The marginal likelihood plays an important role in many areas of Bayesian statistics such as parameter estimation, model comparison, and model averaging. In most applications, however, the marginal likelihood is not analytically tractable and must be approximated using numerical methods. Here we provide a tutorial on bridge sampling (Bennett, 1976;...

We demonstrate the use of three popular Bayesian software packages that enable researchers to estimate parameters in a broad class of models that are commonly used in psychological research. We focus on WinBUGS, JAGS, and Stan, and show how they can be interfaced from R and MATLAB. We illustrate the use of the packages through two fully worked exam...

Motorists often engage in secondary tasks unrelated to driving that increase cognitive workload, resulting in fatal crashes and injuries. An International Standards Organization (ISO) method for measuring a driver’s cognitive workload, the Detection Response Task (DRT), correlates well with driving outcomes, but investigation of its putative theore...

Whenever parameter estimates are uncertain or observations are contaminated by measurement error, the Pearson correlation coefficient can severely underestimate the true strength of an association. Various approaches exist for inferring the correlation in the presence of estimation uncertainty and measurement error, but none are routinely applied i...

We demonstrate the use of three popular Bayesian software packages that enable researchers to estimate parameters in a broad class of models that are commonly used in psychological research. We focus on WinBUGS, JAGS, and Stan, and show how they can be interfaced from R and MATLAB. We illustrate the use of the packages through two fully worked exam...

Multinomial processing trees (MPTs) are a popular class of cognitive models for categorical data. In typical applications, researchers compare several MPTs, each equipped with many parameters, especially when the models are implemented in a hierarchical framework. The principled Bayesian solution is to compute posterior model probabilities and Baye...

Bayesian parameter estimation and Bayesian hypothesis testing present attractive alternatives to classical inference using confidence intervals and p values. In part I of this series we outline ten prominent advantages of the Bayesian approach. Many of these advantages translate to concrete opportunities for pragmatic researchers. For instance, Bay...

Bayesian hypothesis testing presents an attractive alternative to p value hypothesis testing. Part I of this series outlined several advantages of Bayesian hypothesis testing, including the ability to quantify evidence and the ability to monitor and update this evidence as data come in, without the need to know the intention with which the data wer...

This preregistered replication attempt focuses on the finding from Ackerman, Nocera, & Bargh (2010; ANB) that holding a heavy object triggers concepts related to importance. ANB reported that participants who were holding a heavy clipboard rated a job candidate as better overall and more seriously interested in the job than participants holding a l...

The marginal likelihood plays an important role in many areas of Bayesian statistics such as parameter estimation, model comparison, and model averaging. In most applications, however, the marginal likelihood is not analytically tractable and must be approximated using numerical methods. Here we provide a tutorial on bridge sampling (Bennett, 1976;...