Rianne de Heide's research while affiliated with Centrum Wiskunde & Informatica and other places

Publications (14)

Preprint
Full-text available
We study worst-case growth-rate optimal (GROW) E-variables for hypothesis testing between two group models. If the underlying group G acts freely on the observation space, there exists a maximally invariant statistic of the data. We show that among all E-statistics, invariant or not, the likelihood ratio of the maximally invariant is GROW and that...
Preprint
Full-text available
Top Two algorithms arose as an adaptation of Thompson sampling to best arm identification in multi-armed bandit models (Russo, 2016), for parametric families of arms. They select the next arm to sample from by randomizing among two candidate arms, a leader and a challenger. Despite their good empirical performance, theoretical guarantees for fixed-...
Preprint
Different users of machine learning methods require different explanations, depending on their goals. To make machine learning accountable to society, one important goal is to get actionable options for recourse, which allow an affected user to change the decision $f(x)$ of a machine learning system by making limited changes to its input $x$. We fo...
Preprint
Full-text available
We consider a stochastic bandit problem with a possibly infinite number of arms. We write $p^*$ for the proportion of optimal arms and $\Delta$ for the minimal mean-gap between optimal and sub-optimal arms. We characterize the optimal learning rates both in the cumulative regret setting, and in the best-arm identification setting in terms of the pr...
Article
Full-text available
Volker Kessler (‘God becomes beautiful … in mathematics’ – HTS 2018) argues two points to Rudolf Bohren’s list of four areas where (1) God becomes beautiful should be extended with a fifth one: mathematics and (2) mathematics can be argued as a place where God becomes beautiful. In this response, we would like to argue that (1) the extension of Boh...
Article
Full-text available
Wenmackers and Romeijn (2016) formalize ideas going back to Shimony (1970) and Putnam (1963) into an open-minded Bayesian inductive logic, that can dynamically incorporate statistical hypotheses proposed in the course of the learning process. In this paper, we show that Wenmackers and Romeijn's proposal does not preserve the classical Bayesian cons...
Preprint
We investigate and provide new insights on the sampling rule called Top-Two Thompson Sampling (TTTS). In particular, we justify its use for fixed-confidence best-arm identification. We further propose a variant of TTTS called Top-Two Transportation Cost (T3C), which disposes of the computational burden of TTTS. As our main contribution, we provide...
Preprint
Full-text available
We study generalized Bayesian inference under misspecification, i.e. when the model is `wrong but useful'. Generalized Bayes equips the likelihood with a learning rate $\eta$. We show that for generalized linear models (GLMs), $\eta$-generalized Bayes concentrates around the best approximation of the truth within the model for specific $\eta \neq 1...
Preprint
Full-text available
We present a new theory of hypothesis testing. The main concept is the S-value, a notion of evidence which, unlike p-values, allows for effortlessly combining evidence from several tests, even in the common scenario where the decision to perform a new test depends on the previous test outcome: safe tests based on S-values generally preserve Type-I...
Preprint
Full-text available
It is often claimed that Bayesian methods, in particular Bayes factor methods for hypothesis testing, can deal with optional stopping. We first give an overview, using only most elementary probability theory, of three different mathematical meanings that various authors give to this claim: stopping rule independence, posterior calibration and (semi...
Article
Full-text available
Invited Discussion : Bertrand Clarke - Meng Li - Peter Grunwald and Rianne de Heide Contributed Discussion : A. Philip Dawid - William Weimin Yoo - Robert L. Winkler, Victor Richmond R. Jose, Kenneth C. Lichtendahl Jr., and Yael Grushka-Cockayne - Kenichiro McAlinn, Knut Are Aastveit, and Mike West - Minsuk Shin - Tianjian Zhou - Lennart Hoogerheid...
Article
Full-text available
Recently, optional stopping has been a subject of debate in the Bayesian psychology community. Rouder (2014) argues that optional stopping is no problem for Bayesians, and even recommends the use of optional stopping in practice, as do Wagenmakers et al. (2012). This article addresses the question whether optional stopping is problematic for Bayesi...

Citations

... 3.In Kessler (2018), I argued that 'God becomes beautiful … in mathematics'. This article received a response by Smit and De Heide (2021), in which they criticise that I did 'not discuss in detail the current state of research in the field of philosophy of mathematics' (p. 3). ...
... Of course, the coverage bound rests on the assumption that the data model is correctly specified and a misspecified data model will result in incorrect coverage. Furthermore, the bound is based on simple null hypotheses, but it can also be shown to hold for composite null hypotheses when special types of priors are assigned to the nuisance parameters (Hendriksen et al., 2021). ...
... Beyond computing ELPD estimates as in (3), the LOO predictive probabilities {p(y i |y −i )} n i=1 are also of interest in themselves, as they allow to implement methodologies aiming at optimizing predictive performances such as Bayesian stacking (see e.g. Yao et al. [2018] and references therein) or at identify discording observations (see e.g. the notion of conditional predictive ordinate Pettit, 1990) which then leads to model improvements and refinements. See also Weiss and Cho [1998], Epifani et al. [2008] and references therein for related discussion. ...
... Once the scientist has found a combination of choices that produces the preferred result, she would publish the experiment without mentioning combinations that results in a deviation from the preferred result (see e.g., Simmons et al., 2011). Similarly, she may decide to continue with collecting evidence (for example by adding participants or extending the length of the study) when the results do not support her preferred conclusion and stop the experiment as soon as she gets the result (see Heide & Grünwald, 2017). Those and similar covert manipulations called Questionable Research Practices (QRP) are possible because of the flexibility of scientific methodology. ...