Michael David Lee

Michael David Lee
University of California, Irvine | UCI · Department of Cognitive Sciences

BSc, BA(Hons), Grad Dip Ed, PhD

About

240
Publications
48,828
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
8,777
Citations
Introduction
My research interests are in developing and evaluating mathematical and computational psychological models, and in Bayesian methods for implementing and evaluation models. I have worked in areas ranging from stimulus representation, to categorization and memory, to decision making and problem solving. I have a particular interest in incorporating individual differences into cognitive models, including studying collective and social cognition and the "wisdom of the crowd".
Additional affiliations
March 2006 - present
University of California, Irvine
Position
  • Professor (Full)
January 2001 - March 2006
University of Adelaide
Position
  • Professor (Associate)

Publications

Publications (240)
Article
The last 25 years have shown a steady increase in attention for the Bayes factor as a tool for hypothesis evaluation and model selection. The present review highlights the potential of the Bayes factor in psychological research. We discuss six types of applications: Bayesian evaluation of point null, interval, and informative hypotheses, Bayesian e...
Article
Many commonly‐used multi‐trial wordlist memory (WLM) tests present list words in a shuffled order across learning trials, while others maintain a fixed order across trials. The ADAS‐Cog employs shuffling across trials in only one sequence for all assessed subjects. Therefore, to examine the impact of presentation order shuffling on the cognitive pr...
Article
The Balloon Analogue Risk Task (BART) is widely-used to measure risk propensity in theoretical, clinical, and applied research. In the task, people choose either to pump a balloon to increase its value at the risk of the balloon bursting and losing all value, or to bank the current value of the balloon. Risk propensity is most commonly measured as...
Article
We study the wisdom of the crowd in three sequential decision-making tasks: the Balloon Analogue Risk Task (BART), optimal stopping problems, and bandit problems. We consider a behavior-based approach, using majority decisions to determine crowd behavior and show that this approach performs poorly in the BART and bandit tasks. The key problem is th...
Article
We use cognitive models to evaluate three theories of the change in semantic memory caused by Alzheimer’s disease. We use data from 14,096 clinical assessments of 3602 Alzheimer’s patients and their caregivers. Each patient completed a semantic memory task involving the odd-one-out comparison of animal names. Each patient was also independently eva...
Article
Full-text available
Student evaluations of teaching are widely used to assess instructors and courses. Using a model-based approach and Bayesian methods, we examine how the direction of the scale, labels on scales, and the number of options affect the ratings. We conduct a within-participants experiment in which respondents evaluate instructors and lectures using diff...
Article
This study sought to validate a pragmatic method to predict impending cognitive decline in Alzheimer’s disease (AD) patients, prior to the development of hallmark symptoms. The method, a Hierarchical Bayesian Cognitive Process (HBCP) model, uses item responses to a wordlist memory (WLM) test to generate digital cognitive biomarkers (Table 1). We re...
Article
Full-text available
Background: Recent Alzheimer’s disease (AD) trials have faced significant challenges to enroll pre-symptomatic or early stage AD subjects with biomarker positivity, minimal or no cognitive impairment, and likelihood to decline cognitively during a short trial period. Our previous study showed that digital cognitive biomarkers (DCB), generated by a...
Preprint
Full-text available
The last 25 years have shown a steady increase in attention for the Bayes factor as a tool for hypothesis evaluation and model selection. The present review highlights the potential of the Bayes factor in psychological research. We discuss six types of applications: Bayesian evaluation of point null, interval, and informative hypotheses, Bayesian e...
Article
We develop and demonstrate a method for inferring changes in strategy use, applicable to decision making in multi-attribute choice. The method is an extension of one developed by Lee, Gluck, and Walsh (Decision 6:335–368, 2019) and continues to rely on a Bayesian approach for inferring strategy switches based on spike-and-slab priors. The extension...
Chapter
The principle of predictive irrelevance states that when two competing models predict a data set equally well, that data set cannot be used to discriminate the models and – for that specific purpose – the data set is evidentially irrelevant. To highlight the ramifications of the principle, we first show how a single binomial observation can be irre...
Article
Optimal stopping problems require people to choose from a sequence of values presented sequentially, under the constraint that it is not possible to return to an earlier option. Usually, the distribution from which values are drawn is the same for each option in the sequence. We consider an extension in which the distributions change in a known way...
Article
We model word-list learning over sequences of immediate and delayed free recall tasks to study the impact of memory impairment on episodic memory. We use a previously developed Multinomial Processing Tree (MPT) model of encoding, retrieval, and learning (Alexander et al., 2016), and apply it to behavioral data from thousands of patients tested tens...
Article
Full-text available
Why is there no consensual way of conducting Bayesian analyses? We present a summary of agreements and disagreements of the authors on several discussion points regarding Bayesian inference. We also provide a thinking guideline to assist researchers in conducting Bayesian inference in the social and behavioural sciences.
Article
The target article on robust modeling (Lee et al. in review) generated a lot of commentary. In this reply, we discuss some of the common themes in the commentaries; some are simple points of agreement while others are extensions of a practical or abstract nature. We also address a small number of disagreements or confusions.
Article
In most decision-making domains, people recognize some stimuli but not others. The validity of recognition as a basis for making decisions about the stimuli is empirically unclear, but has strong implications for the development of theories and models of decision-making. Following the framework used to motivate the recognition heuristic (Gigerenzer...
Article
In an attempt to increase the reliability of empirical findings, psychological scientists have recently proposed a number of changes in the practice of experimental psychology. Most current reform efforts have focused on the analysis of data and the reporting of findings for empirical studies. However, a large contingent of psychologists build mode...
Article
Full-text available
Differential strategy use is a topic of intense investigation in developmental psychology. Questions under study are as follows: How do strategies change with age, how can individual differences in strategy use be explained, and which interventions promote shifts from suboptimal to optimal strategies? In order to detect such differential strategy u...
Article
We apply the “wisdom of the crowd” idea to human category learning, using a simple approach that combines people's categorization decisions by taking the majority decision. We first show that the aggregated crowd category learning behavior found by this method performs well, learning categories more quickly than most or all individuals for 28 previ...
Article
Full-text available
Whenever parameter estimates are uncertain or observations are contaminated by measurement error, the Pearson correlation coefficient can severely underestimate the true strength of an association. Various approaches exist for inferring the correlation in the presence of estimation uncertainty and measurement error, but none are routinely applied i...
Article
Full-text available
The Iowa Gambling Task (IGT) is one of the most popular experimental paradigms for comparing complex decision-making across groups. Most commonly, IGT behavior is analyzed using frequentist tests to compare performance across groups, and to compare inferred parameters of cognitive models developed for the IGT. Here, we present a Bayesian alternativ...
Article
In optimal stopping problems, people are asked to choose the best option out of a sequence of alternatives, under the constraint that they cannot return to an earlier option once it is rejected. We examine human performance on variations of the optimal stopping problem, with different environments and with different goals. Specifically, we consider...
Article
People often interact with environments that can provide only a finite number of items as resources. Eventually a book contains no more chapters, there are no more albums available from a band, and every Pok?mon has been caught. When interacting with these sorts of environments, people either actively choose to quit collecting new items, or they ar...
Article
The development of cognitive models involves the creative scientific formalization of assumptions, based on theory, observation, and other relevant information. In the Bayesian approach to implementing, testing, and using cognitive models, assumptions can influence both the likelihood function of the model, usually corresponding to assumptions abou...
Article
Full-text available
Take-the-best is a decision-making strategy that chooses between alternatives, by searching the cues representing the alternatives in order of cue validity, and choosing the alternative with the first discriminating cue. Theoretical support for take-the-best comes from the “fast and frugal” approach to modeling cognition, which assumes decision-mak...
Article
The practical advantages of Bayesian inference are demonstrated here through two concrete examples. In the first example, we wish to learn about a criminal’s IQ: a problem of parameter estimation. In the second example, we wish to quantify and track support in favor of the null hypothesis that Adam Sandler movies are profitable regardless of their...
Article
In a top-n task, people produce a list of items that they believe are ordered relative to a criterion, and can include any number of items in their list. We develop Thurstonian cognitive models of the individual differences and decision-making processes involved in producing top-n lists, and apply it to the problem of inferring an aggregated list f...
Article
Multidimensional scaling (MDS) models of mental representation assume stimuli are represented by points in a low-dimensional space, such that more similar stimuli are represented by points closer to each other. We consider possible individual differences in MDS representations, using the recently proposed K-INDSCAL model, which allows for both sub-...
Article
The article reviews the field from a historical perspective, starting from the works of Fechner and Thurstone, and outlining the basic theoretical concepts in four traditional areas: learning, psychophysics, measurement and choice, and response times. More recent topics that have emerged from these areas are also briefly mentioned together with the...
Article
Full-text available
We study the effect of memory impairment on triadic comparisons of animal names in a large clinical data set. We define eight groups of subjects in terms of their delayed free recall performance, and present standard analyses of the triadic comparison and free recall data that provide little insight into the effect of memory impairment on semantic...
Article
Full-text available
Interval estimates – estimates of parameters that include an allowance for sampling uncertainty – have long been touted as a key component of statistical analyses. There are several kinds of interval estimates, but the most popular are confidence intervals (CIs): intervals that contain the true parameter value in some known proportion of repeated s...
Article
Full-text available
Many decisions in the lives of animals and humans require a fine balance between the exploration of different options and the exploitation of their rewards. Do you buy the advertised car, or do you test drive different models? Do you continue feeding from the current patch of flowers, or do you fly off to another one? Do you marry your current part...
Conference Paper
Full-text available
People have all sorts of different knowledge and are able to express this information in many ways. One ubiquitous form of expression is through ranking, in which people generate a list of items with respect to the criterion of interest. In some structured circumstances — like experts predicting the final standings of all the teams in a sporting co...
Article
Full-text available
Interval estimates – estimates of parameters that include an allowance for sampling uncertainty – have long been touted as a key component of statistical analyses. There are several kinds of interval estimates, but the most popular are confidence intervals (CIs): intervals that contain the true parameter value in some known proportion of repeated s...
Article
Full-text available
Hilbig and Moshagen (Psychonomic Bulletin & Review, 21, 1431-1443, 2014) recently developed a method for making inferences about the decision processes people use in multi-attribute forced choice tasks. Their paper makes a number of worthwhile theoretical and methodological contributions. Theoretically, they provide an insightful psychological moti...
Article
Full-text available
The less-is-more effect predicts that people can be more accurate making paired-comparison decisions when they have less knowledge, in the sense that they do not recognize all of the items in the decision domain. The traditional theoretical explanation is that decisions based on recognizing one alternative but not the other can be more accurate tha...
Book
p>Interval estimates -- estimates of parameters that include an allowance for sampling uncertainty -- have long been touted as a key component of statistical analyses. There are several kinds of interval estimates, but the most popular are confidence intervals (CIs): intervals that contain the true parameter value in some known proportion of repeat...
Article
Full-text available
Diffusion models are widely-used and successful accounts of the time course of two-choice decision making. Most diffusion models assume constant boundaries, which are the threshold levels of evidence that must be sampled from a stimulus to reach a decision. We summarize theoretical results from statistics that relate distributions of decisions and...
Article
Full-text available
The power fallacy refers to the misconception that what holds on average -across an ensemble of hypothetical experiments- also holds for each case individually. According to the fallacy, high-power experiments always yield more informative data than do low-power experiments. Here we expose the fallacy with concrete examples, demonstrating that a pa...
Article
We study how people terminate their search for information when making decisions in a changing environment. In 3 experiments, differing in the cost of search, participants made a sequence of 2-alternative decisions, based on the information provided by binary cues they could search. Whether limited or extensive search was required to maintain accur...
Article
Full-text available
Interval estimates – estimates of parameters that include an allowance for sampling uncertainty – have long been touted as a key component of statistical analyses. There are several kinds of interval estimates, but the most popular are confidence intervals (CIs): intervals that contain the true parameter value in some known proportion of repeated s...
Article
Full-text available
We develop a cognitive modeling approach, motivated by classic theories of knowledge representation and judgment from psychology, for combining people's rankings of items. The model makes simple assumptions about how individual differences in knowledge lead to observed ranking data in behavioral tasks. We implement the cognitive model as a Bayesian...
Article
Full-text available
We demonstrate the usefulness of cognitive models for combining human estimates of probabilities in two experiments. The first experiment involves people's estimates of probabilities for general knowledge questions such as "What percentage of the world's population speaks English as a first language?" The second experiment involves people's estimat...
Article
In most decision-making situations, there is a plethora of information potentially available to people. Deciding what information to gather and what to ignore is no small feat. How do decision makers determine in what sequence to collect information and when to stop? In two experiments, we administered a version of the German cities task developed...
Article
Full-text available
Faced with probabilistic relationships between causes and effects, quantum theory assumes that deterministic causes do not exist, and that only incomplete probabilistic expressions of knowledge are possible. As in its application to physics, this fundamental epistemological stance severely limits the ability of quantum theory to provide insight and...
Article
The scale-invariant memory, perception, and learning (SIMPLE) model developed by Brown, Neath, and Chater (2007) formalizes the theoretical idea that scale invariance is an important organizing principle across numerous cognitive domains and has made an influential contribution to the literature dealing with modeling human memory. In the context of...
Article
Bayesian inference has become a standard method of analysis in many fields of science. Students and researchers in experimental psychology and cognitive science, however, have failed to take full advantage of the new and exciting possibilities that the Bayesian approach affords. Ideal for teaching and self study, this book demonstrates how to do Ba...
Article
Wills and Pothos (2012) reviewed approaches to evaluating formal models of categorization, raising a series of worthwhile issues, challenges, and goals. Unfortunately, in discussing these issues and proposing solutions, Wills and Pothos (2012) did not consider Bayesian methods in any detail. This means not only that their review excludes a major bo...
Article
Identifying disease-modifying treatment effects in earlier stages of Alzheimer's disease (AD)-when changes are subtle-will require improved trial design and more sensitive analytical methods. We applied hierarchical Bayesian analysis with cognitive processing (HBCP) models to the Alzheimer's Disease Assessment Scale-Cognitive subscale (ADAS-Cog) an...
Article
Full-text available
Formal models in psychology are used to make theoretical ideas precise and allow them to be evaluated quantitatively against data. We focus on one important-but under-used and incorrectly maligned-method for building theoretical assumptions into formal models, offered by the Bayesian statistical approach. This method involves capturing theoretical...
Data
Full-text available
Game theory has been useful for understanding risk-taking and cooperative behavior. In the present study, subjects played the Hawk-Dove game with simulated and embodied (robotic) neu-ral agents which used a neurobiologically plausible model of action selection and adaptive behaviors. Subjects had their serotonin levels temporarily altered through a...
Article
Full-text available
Heuristic decision-making models, like Take-the-best, rely on environmental regularities. They conduct a limited search, and ignore available information, by assuming there is structure in the decision-making environment. Take-the-best relies on at least two regularities: DIMINISHING RETURNS, which says that information found earlier in search is m...
Article
Full-text available
Two experiments were conducted examining the effectiveness of visualizations of unstructured texts. The first experiment presented transcriptions of unrehearsed dialog and the second used emails. Both experiments showed an advantage in overall performance for semantically structured two-dimensional (2D) spatialized layouts, such as multidimensional...
Article
Full-text available
Despite their theoretical appeal, Bayesian methods for the assessment of poor effort and malingering are still rarely used in neuropsychological research and clinical diagnosis. In this article, we outline a novel and easy-to-use Bayesian latent group analysis of malingering whose goal is to identify participants displaying poor effort when tested....
Article
Full-text available
MacGregor and Ormerod (1996) have presented results purporting to show that human performance on visually presented traveling salesman problems, as indexed by a measure of response uncertainty, is strongly determined by the number of points in the stimulus array falling inside the convex hull, as distinct from the total number of points. It is argu...
Article
Determining how cognition affects functional abilities is important in Alzheimer disease and related disorders. A total of 280 patients (normal or Alzheimer disease and related disorders) received a total of 1514 assessments using the functional assessment staging test (FAST) procedure and the MCI Screen. A hierarchical Bayesian cognitive processin...
Article
The "wisdom of the crowd" phenomenon refers to the finding that the aggregate of a set of proposed solutions from a group of individuals performs better than the majority of individual solutions. Most often, wisdom of the crowd effects have been investigated for problems that require single numerical estimates. We investigate whether the effect can...
Article
We apply a cognitive modeling approach to the problem of measuring expertise on rank ordering problems. In these problems, people must order a set of items in terms of a given criterion (e.g., ordering American holidays through the calendar year). Using a cognitive model of behavior on this problem that allows for individual differences in knowledg...
Chapter
Book synopsis: Over a century ago, William James proposed that people search through memory much as they rummage through a house looking for lost keys. We scour our environments for territory, food, mates, and information. We search for items in visual scenes, for historical facts, and for the best deals on Internet sites; we search for new friends...
Article
Inductive generalization, where people go beyond the data provided, is a basic cognitive capability, and it underpins theoretical accounts of learning, categorization, and decision making. To complete the inductive leap needed for generalization, people must make a key ''sampling'' assumption about how the available data were generated. Previous mo...
Article
Full-text available
Hierarchical Bayesian methods offer a principled and comprehensive way to relate psychological models to data. Here we use them to model the patterns of information search, stopping and deciding in a simulated binary comparison judgment task. The simulation involves 20 subjects making 100 forced choice comparisons about the relative magnitudes of t...