No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
The ignored alternative: An application of Luce’s low-threshold model to recognition memory
... Relaxing the assumption of pure detection produces a low-threshold model with three possible states (i.e., two thresholds). We will call this the two-low-threshold (2LT) model to distinguish it from low-threshold models with a single threshold (Kellen, Erdfelder, Malmberg, Dube, & Criss, 2016;Luce, 1963;Starns & Ma, in press). 2 Fig. 2B displays this model, which is identical to the 2HT model except that any item type can potentially produce any internal evidence state. The Detect Studied and Detect Not Studied states are relabeled as Probably Studied and Probably Not Studied as they no longer provide perfect information about stimulus class. ...
... Some recent studies have evaluated Luce's (1963) low-threshold (LT) model in recognition memory (Kellen et al., 2016;Starns & Ma, in press). This model differs from the 2LT model that we considered because it has a single threshold instead of two (and thus two memory states instead of three), so we will refer to the Luce model as a 1LT model. ...
... This is because only one of the response categories is a mixture of responses from different internal states. The 1LT model assumes that people adopt either a liberal or conservative response strategy (Kellen et al., 2016;Luce, 1963). Under the liberal strategy, participants respond "Studied" for all items producing the Probably Studied state and for some proportion of the items producing the Probably Not Studied state, so "Studied" responses come from a mixture of states but all "Not Studied" responses are from the Probably Not Studied state. ...
... For instance, when only two lures are detected (see the three branches with two D n and one 1 − D n in Fig. 2A), they are assigned Ranks 3 and 4, whereas the non-detected target and the non-detected lure receive Ranks 1 and 2 by guessing (i.e., with a probability of 1 2 for each of the two possible combinations). The proposed model for the KART paradigm builds on similar psychological assumptions as Province and Rouder's (2012) 2HT model for two-alternative forced-choice tasks, Luce's (1963) low-threshold (LT) model for two-alternative forcedchoice tasks, and Kellen, Erdfelder, Malmberg, Dubé, and Criss's (2016) extension of the LT model to ranking tasks (see also Iverson & Bamber, 1997). ...
... Further support for discrete-state models was reported by Kellen et al. (2016). The authors pointed out that Luce's (1963) LT model is also in line with Kellen and Klauer's (2014) critical test. ...
The question of whether recognition memory should be measured assuming continuous memory strength (signal detection theory) or discrete memory states (threshold theory) has become a prominent point of discussion. In light of limitations associated with receiver operating characteristics, comparisons of the rival models based on simple qualitative predictions derived from their core properties were proposed. In particular, K-alternative ranking tasks (KARTs) yield a conditional probability of targets being assigned Rank 2, given that they were not assigned Rank 1, which is higher for strong than for weak targets. This finding has been argued to be incompatible with the two-high-threshold (2HT) model (Kellen & Klauer, 2014). However, we show that the incompatibility only holds under the auxiliary assumption that the probability of detecting lures is invariant under target-strength manipulations. We tested this assumption in two different ways: by developing new model versions of 2HT theory tailored to KARTs and by employing novel forced-choice-then-ranking tasks. Our results show that 2HT models can explain increases in the conditional probability of targets being assigned Rank 2 with target strength. This effect is due to larger 2HT lure-detection probabilities in test displays in which lures are ranked jointly with strong (as compared to weak) targets. We conclude that lure-detection probabilities vary with target strength and recommend that 2HT models should allow for this variation. As such models are compatible with KART performance, our work highlights the importance of carefully adapting measurement models to new paradigms.
... Kellen and Klauer (2014) found evidence supporting continuous mediation ( " #$%& < " '()*+, ), and McAdoo and Gronlund (2016) found similar results using faces as the study stimuli. However, Kellen et al. (2016) and McAdoo and Gronlund (under revision) found that the low-threshold model (Luce, 1963), a discrete model, fit the c2 data and curvilinear ROCs just as well as SDT did. This finding suggests that more converging evidence is required before concluding in favor of continuous models. ...
... Note the LTM can account for performance in the ranking task (seeKellen et al., 2016; McAdoo & Gronlund, under revision) but the authors are unaware of any extensions of the LTM made to confidence-rating tasks. ...
How recognition memory is mediated has been of interest to researchers for decades. But the apparent consensus implicating continuous mediation has been challenged. McAdoo, Key, and Gronlund (Journal of Experimental Psychology: Learning, Memory, and Cognition,2018. Advanced online publication) demonstrated that recognition memory can be mediated by either discrete or continuous evidence, depending on target-filler similarity. The present paper expands on this research by showing that different recognition tasks also can be mediated by different evidence. Specifically, recognition memory was mediated by continuous evidence in a ranking task, but by discrete evidence in a confidence-rating task. We posit that participants utilize a control process that induces a reliance on discrete or continuous evidence as a function of efficiency (Malmberg, 2008) to suit the demands of the task.
... However, Province and Rouder showed that if you relax the certainty assumption and allow for the possibility that detected items can be recognized with a broader range of confidence, a discrete-state model can indeed predict curvilinear ROCs. Recently, Kellen, Erdfelder, Malmberg, Dube, and Criss (2016) showed that an alternative discrete model (the low-threshold model, Luce 1963), which assumes that New items can exceed a threshold for detection, also can approximate empirical ROC curves. Therefore, ROC analysis is unable to definitively test between discrete-state and continuous mediation. ...
... This study also would be important given the call for the use of confidence judgments to construct ROC curves to assess eyewitness performance Wixted & Mickes 2012). Kellen et al. (2016) also found that Luce's (1963) discrete low-threshold model fit the Kellen and Klauer (2014) ranking data as well as did a continuous SDT model. Clearly, more work needs to be done to understand the c 2 ranking paradigm, as well as determine what factors, or strategies, affect when recognition memory is found to be continuously or discretely mediated. ...
Many in the eyewitness identification community believe that sequential lineups are superior to simultaneous lineups because simultaneous lineups encourage inappropriate choosing due to promoting comparisons among choices (a relative judgment strategy), but sequential lineups reduce this propensity by inducing comparisons of lineup members directly to memory rather than to each other (an absolute judgment strategy). Different versions of the relative judgment theory have implicated both discrete-state and continuous mediation of eyewitness decisions. The theory has never been formally specified, but (Yonelinas, J Exp Psychol Learn Mem Cogn 20:1341–1354, 1994) dual-process models provide one possible specification, thereby allowing us to evaluate how eyewitness decisions are mediated. We utilized a ranking task (Kellen and Klauer, J Exp Psychol Learn Mem Cogn 40:1795–1804, 2014) and found evidence for continuous mediation when facial stimuli match from study to test (Experiment 1) and when they mismatch (Experiment 2). This evidence, which is contrary to a version of relative judgment theory that has gained a lot of traction in the legal community, compels reassessment of the role that guessing plays in eyewitness identification. Future research should continue to test formal explanations in order to advance theory, expedite the development of new procedures that can enhance the reliability of eyewitness evidence, and to facilitate the exploration of task factors and emergent strategies that might influence when recognition is continuously or discretely mediated.
... However, this does not necessarily imply that this model is uncontested. It has, for example, been argued that a lowthreshold model (Kellen et al., 2016;Luce, 1963;Starns, 2020) of SDAI could be a viable competitor to IOMs based on SDT (Meyer-Grant & Klauer, 2021). Moreover, unequalvariance Gaussian SDT models are themselves known to have certain properties that are at least controversial. ...
For modeling recognition decisions in a typical eyewitness identification lineup task with multiple simultaneously presented test stimuli (also known as simultaneous detection and identification), essentially two different models based on signal detection theory are currently under consideration. These two models mainly differ with respect to their assumptions regarding the interplay between the memory signals of different stimuli presented in the same lineup. The independent observations model (IOM), on the one hand, assumes that the memory signal of each simultaneously presented test stimulus is separately assessed by the decision-maker, whereas the ensemble model (EM), on the other hand, assumes that each of these memory signals is first compared with and then assessed relative to its respective context (i.e., the memory signals of the other stimuli within the same lineup). Here, we discuss some reasons why comparing confidence ratings between trials with and without a dud (i.e., a lure with no systematic resemblance to the target) in an otherwise fair lineup—results of which have been interpreted as evidence in favor of the EM—is in fact inconclusive for differentiating between the EM and the IOM. However, the lack of diagnostic value hinges on the fact that in these experiments two aspects of between-item similarity (viz. old–new and within-lineup similarity) are perfectly confounded. Indeed, if separately manipulating old–new similarity, we demonstrate that EM and IOM make distinct predictions. Following this, we show that previously published data are inconsistent with the predictions made by the EM.
... In part my choice of working within UVSDT is also due to its successes as a model of the decision stage in a variety of recognition memory designs (Kellen, Winiger, Dunn, & Singmann, 2021;Wixted, 2020). However, it is entirely possible that "Accessibility" can find mathematical support within some other framework, such as discrete-state theories of memory (Kellen, Erdfelder, Malmberg, Dubé, & Criss, 2016;Starns, Dubé, & Frelinger, 2018). ...
Laming (2021) has recently proposed a way to measure the accessibility (as opposed to availability) of memories via recognition testing. His measure “Accessibility” is calculated by subtracting the Hit Rate and False Alarm Rate that fall at the point where the ROC curve’s derivative is 1. I prove that, if one works within the framework of Unequal-Variance Signal Detection Theory (UVSDT), as Laming does, the measure “Accessibility” depends on the
location of the response criterion (though always with a neutral likelihood ratio). Furthermore, I prove that the measure varies with the underlying variances of UVSDT regardless of which definition of bias (criterion or likelihood ratio) is used and, crucially, this holds even when the accuracy of discrimination performance or “sensitivity” (da) in UVSDT is constant. As such, from the standpoint of (at least) UVSDT, it is questionable whether or to what extent the new measure of “Accessibility” actually measures the accessibility of any memory.
... Model originally proposed by Luce (1963; see also Kellen, Erdfelder, Malmberg, Dubé, & Criss, 2016;McAdoo & Gronlund, 2020;Starns et al., 2018). One distinctive property of this Low-Threshold model is that assumes that noise stimuli can be incorrectly signal-detected. ...
Signal detection theory (SDT) plays a central role in the characterization of human judgments in a wide range of domains, most prominently in recognition memory. But despite its success, many of its fundamental properties are often misunderstood, especially when it comes to its testability. The present work examines five main properties that are characteristic of existing SDT models of recognition memory: (a) random-scale representation, (b) latent-variable independence, (c) likelihood-ratio monotonicity, (d) ROC function asymmetry, and (e) nonthreshold representation. In each case, we establish testable consequences and test them against data collected in the appropriately designed recognition-memory experiment. We also discuss the connection between yes-no, forced-choice, and ranking judgments. This connection introduces additional behavioral constraints and yields an alternative method of reconstructing yes-no ROC functions. Overall, the reported results provide a strong empirical foundation for SDT modeling in recognition memory. (PsycInfo Database Record (c) 2021 APA, all rights reserved).
... This guessing strategy can be implemented with a guessing parameter g that goes from 0 to 1 and switches from conservative to liberal guessing at the value g = .5. Similar to Kellen et al. (2016), the predicted hits and false alarms are: p hit = q t + q t (2g − 1), if g < 0.5, = q t + (1 − q t ) (2g − 1), if g >= 0.5, and p FA = q d + q d (2g − 1), if g < 0.5, = q d + (1 − q d ) (2g − 1), if g >= 0.5. ...
Divided attention has little effect for simple tasks, such as luminance detection, but it has large effects for complex tasks, such as semantic categorization of masked words. Here, we asked whether the semantic categorization of visual objects shows divided attention effects as large as those observed for words, or as small as those observed for simple feature judgments. Using a dual-task paradigm with nameable object stimuli, performance was compared with the predictions of serial and parallel models. At the extreme, parallel processes with unlimited capacity predict no effect of divided attention; alternatively, an all-or-none serial process makes two predictions: a large divided attention effect (lower accuracy for dual-task trials, compared to single-task trials) and a negative response correlation in dual-task trials (a given response is more likely to be incorrect when the response about the other stimulus is correct). These predictions were tested in two experiments examining object judgments. In both experiments, there was a large divided attention effect and a small negative correlation in responses. The magnitude of these effects was larger than for simple features, but smaller than for words. These effects were consistent with serial models, and rule out some but not all parallel models. More broadly, the results help establish one of the first examples of likely serial processing in perception.
... Note that this assumption differs from the 2-HTT assumption that lures can correctly exceed the high detect-new threshold. LTT has enjoyed a recent resurgence (e.g., Kellen, Erdfelder, Malmberg, Dubé, & Criss, 2016;McAdoo & Gronlund, 2019;Starns & Ma, 2018), and for good reason. First, it can provide a good fit to even apparently curvilinear ROC data. ...
The relationship between confidence and accuracy in recognition memory is important in real-world settings (e.g., eyewitness identification) and is also important to understand at a theoretical level. Signal detection theory assumes that recognition decisions are based on continuous underlying memory signals and therefore inherently predicts that the relationship between confidence and accuracy will be continuous. Almost invariably, the empirical data accord with this prediction. Threshold models instead assume that recognition decisions are based on discrete-state memory signals. As a result, these models do not inherently predict a continuous confidence-accuracy relationship. However, they can accommodate that result by adding hypothetical mapping relationships between discrete states and the confidence rating scale. These mapping relationships are thought to arise from a variety of factors, including demand characteristics (e.g., instructing participants to distribute their responses across the confidence scale). However, until such possibilities are experimentally investigated in the context of a recognition memory experiment, there is no sense in which threshold models adequately explain confidence ratings at a theoretical level. Here, we tested whether demand characteristics might account for the mapping relationships required by threshold models and found that confidence was continuously related to accuracy (almost identically so) both in the presence of strong experimenter demands and in their absence. We conclude that confidence ratings likely reflect the strength of a continuous underlying memory signal, not an attempt to use the confidence scale in a manner that accords with the perceived expectations of the experimenter.
... Just as the model of Hautus et al. (2008) stands as an exception to the generalization that continuous strength models predict source memory for unrecognized items, it is also important acknowledge that some threshold models could be capable of predicting source memory in the absence of recognition. While high-thresholds models postulate that retrieval is an all-or-none process, with retrieval failures resulting in guess responses, low-threshold models (Kellen et al., 2016;Luce, 1963) postulate that retrieval is a noisy process that can produce either a high or low state of evidence (Starns & Ma, 2018). If a low-threshold source memory model were devised, then it would be possible for a target to produce sub-threshold item evidence (leading to a "NEW" response), but for the target to produce above-threshold source evidence (leading to an accurate source response). ...
In episodic memory research, there is a debate concerning whether decision-making in item recognition and source memory is better explained by models that assume all-or-none retrieval processes or continuous underlying strengths. One aspect in which these classes of models tend to differ is their predictions regarding the ability to retrieve contextual details (or source details) of an experienced event, given that the event itself is not recognized. All-or-none or high-threshold models predict that when items are unrecognized, source retrieval is not possible and only guess responses can be elicited. In contrast, models assuming continuous strengths predict that it is possible to retrieve the source of unrecognized items, albeit with low accuracy. Empirically, there have been numerous studies reporting either chance accuracy or above-chance accuracy for source memory in the absence of recognition. Crucially, studies presenting recognition and source judgements for the same item in immediate succession (simultaneous design) have revealed chance-level accuracy, while studies presenting a block of recognition judgements followed by a block of source judgements (blocked design) have revealed slightly above-chance accuracy. Across three sets of experiments involving multiple design manipulations, the present investigation demonstrated: (a) that source memory for unrecognized items is indeed higher in blocked designs; (b) that evidence for the effect in blocked designs is likely artifactual due to item memory changing between blocks; and (c) that the effect does exist in simultaneous designs, but is highly subtle and is more easily detected when uncertainty in the participant-level data is low or is accounted for in a hierarchical Bayesian model. It is suggested that findings of a null effect in the prior literature may be attributable to design elements that hinder source memory as a whole, and to high degrees of uncertainty in the participant-level source data when conditioned on unrecognized items.
... It is conceivable that a new model could be constructed to account for our data, perhaps incorporating response oscillation over discrete mental states, in which there is no memory strength signal to speak of. As the discrete-state framework is expanded to produce more and more sophisticated accounts of recognition data (Kellen, Erdfelder, Malmberg, Dubé, & Criss, 2016;Klauer & Kellen, 2018), such a possibility seems increasingly likely. ...
Recent work by Benjamin and colleagues (Benjamin, Diaz, & Wee, 2009; Benjamin, Tullis, & Lee, 2013) suggests that recognition memory decisions are corrupted by random variability in decision criteria. This conclusion, which explains several anomalies in the recognition literature, was based on fits of the Noisy Decision Theory of Signal Detection (NDT) to a novel task: ensemble recognition. In the ensemble task, participants make Old/New decisions to ensembles of items rather than single items. The NDT assumption that criteria are fixed across ensembles was criticized by Kellen, Klauer, and Singmann (2012), and defended by Benjamin (2013). Little attention, however, has been paid to the assumption of the best-fitting NDT model that participants solve the ensemble task by calculating the average memory strength of items in the probe display. We review evidence of summary statistical representation in visual perception and short-term memory that suggests the aggregation hypothesis is plausible, and hold it up to test in three experiments using the direct ratings procedure (Criss, 2009; Mickes, Wixted, & Wais, 2007). Although we conclude that participants can produce estimates of average probe memory strength at test, in line with the assumptions of NDT, the mechanisms and strategies used to produce such estimates remain unclear.
... Unlike the single-step unequal-variance SDT model, the two-step model can capture this feature, and does so in a psychologically plausible fashion. Of course, other models may also be able to capture the elbow, such as a low-threshold discrete-state model (e.g., Kellen, Erdfelder, Malmberg, Dubé, & Criss, 2016), and could be explored in the future. Regardless of the modeling framework that is applied, we have argued that if the two-step task is used, models should take its structure into account. ...
When asked to determine whether a syllogistic argument is deductively valid, people are influenced by their prior beliefs about the believability of the conclusion. Recently, two competing explanations for this belief bias effect have been proposed, each based on signal detection theory (SDT). Under a response bias explanation, people set more lenient decision criteria for believable than for unbelievable arguments. Under the alternative argument strength explanation, believability affects the reasoning stage of processing an argument, with believable and unbelievable arguments differing in subjective strength for both valid and invalid items. Two experiments tested these accounts by asking participants to make validity judgments for categorical syllogisms and to rate their confidence. Conclusion-believability was manipulated both within group (Experiment 1) and between groups (Experiment 2). A novel two-step version of the signal detection model was fit to receiver operating characteristic (ROC) curves for believable and unbelievable arguments. Model fits confirmed that in both experiments there was a shift in decision criterion but not argument discriminability as a function of argument believability. Crucially, when believability is manipulated between groups, this shift is expected under the response bias account but not under the argument strength account. Therefore, the results support the view that belief bias primarily reflects changes in response bias: people require less evidence to endorse a syllogism as valid when it has a believable conclusion. This has important implications for theories of deductive reasoning.
... Further, there is low threshold theory (Luce, 1963), which relaxes the assumption of high thresholds and allows for observers to erroneously enter detection states (for example, entering the 'detect target' state when presented with a non-target). This model predicts ROCs made of two line segments and has recently been shown to provide a reasonable fit to data from recognition memory tasks (Kellen, Erdfelder, Malmberg, Dubé, & Criss, 2016;Starns & Ma, 2018). Thus, researchers should be aware of this alternative model and may consider applying it in their work, although it should be noted that this model does not produce a single point measure of sensitivity. ...
Here we use simulation to assess previously unaddressed problems in the assessment of statistical interactions in detection and recognition tasks. The proportion of hits and false-alarms made by an observer on such tasks is affected by both their sensitivity and bias, and numerous measures have been developed to separate out these two factors. Each of these measures makes different assumptions regarding the underlying process and different predictions as to how false-alarm and hit rates should covary. Previous simulations have shown that choice of an inappropriate measure can lead to inflated type I error rates, or reduced power, for main effects, provided there are differences in response bias between the conditions being compared. Interaction effects pose a particular problem in this context. We show that spurious interaction effects in analysis of variance can be produced, or true interactions missed, even in the absence of variation in bias. Additional simulations show that variation in bias complicates patterns of type I error and power further. This under-appreciated fact has the potential to greatly distort the assessment of interactions in detection and recognition experiments. We discuss steps researchers can take to mitigate their chances of making an error.
... McAdoo and Gronlund (2016) replicated these results using faces. However, D. Kellen, Erdfelder, Malmberg, Dubé, and Criss (2016) successfully fit a discrete-state model, the low-threshold model (LTM), to Kellen and Klauer's c 2 ranking data, and found no difference in measures of goodness of fit between the LTM and SDT. We applied the LTM to the McAdoo and Gronlund (2016) data and also found little difference between the LTM and SDT (we return to this point later). ...
Two broad approaches characterize the type of evidence that mediates recognition memory: discrete state and continuous. Discrete-state models posit a thresholded memory process that provides accurate information about an item (it is detected) or, failing that, no mnemonic information about the item. Continuous models, in contrast, posit the existence of graded mnemonic information about an item. Evidence favoring 1 approach over the other has been mixed, suggesting the possibility that the mediation of recognition memory may be adaptable and influenced by other factors. We tested this possibility with 2 experiments that varied the semantic similarity of word targets and fillers. Experiment 1, which used semantically similar fillers, displayed evidence of continuous mediation (contrary to Kellen & Klauer, 2015), whereas Experiment 2, which used semantically dissimilar fillers, displayed evidence of discrete mediation. The results have implications for basic theories of recognition memory, as well as for theories of applied domains like eyewitness identification. (PsycINFO Database Record
... Since the ICCM conference was held for the first time together with the Society for Mathematical Psychology, an important question is whether the fields of mathematical psychology and cognitive modeling can learn from each other, and whether more interaction is warranted. Mathematical psychology typically concerns itself with relatively simple models consisting of one or two equations that focus on a single phenomenon (such as signal detection theory for recognition memory; see Kellen, Erdfelder, Malmberg, Dub e, & Criss, 2016, for a recent example), while cognitive modeling typically concerns itself with large-scale cognitive architectures that can be used to model different tasks. Below we outline a few key characteristics of both approaches to modeling cognition that can be of benefit. ...
Cognitive modeling is the effort to understand the mind by implementing theories of the mind in computer code, producing measures comparable to human behavior and mental activity. The community of cognitive modelers has traditionally met twice every 3 years at the International Conference on Cognitive Modeling (ICCM). In this special issue of topiCS, we present the best papers from the ICCM meeting. (The full proceedings are available on the ICCM website.) These best papers represent advances in the state of the art in cognitive modeling. Since ICCM was for the first time also held jointly with the Society for Mathematical Psychology, we use this preface to also reflect on the similarities and differences between mathematical psychology and cognitive modeling.
We examine different models of recognition memory in a simultaneous detection and identification task, which features multiple simultaneously presented test stimuli. A common finding from eyewitness identification research investigating such tasks is that the more confident decision makers are about detecting the presence of a target, the higher the probability that they also correctly identify it. We demonstrate that for members of the signal detection theory (SDT) model framework, predicting such a relationship is — contrary to previous assertions — not entailed by a monotonic diagnosticity ratio. Instead, it can be shown that this prediction follows if latent memory signals’ rank order probabilities exhibit monotonicity under changes in the response criterion. For a selection of common SDT models, we prove that this monotonicity property holds in situations in which two test stimuli are presented simultaneously. Threshold models such as the two-high-threshold model (2HTM), however, do not necessarily possess this feature. Leveraging this fact, we show that in the presence of lures which resemble a target, the 2HTM is unable to make the same predictions as many reasonable SDT models with monotonic rank order probabilities. This enables us to construct a critical, distribution-free test between these models. An empirical investigation implementing this test reveals a clear failure of the 2HTM to account for the qualitative response patterns, which are consistent with the predictions of SDT models with monotonic rank order probabilities.
Confidence-accuracy characteristic (CAC) plots were developed for use in eyewitness identification experiments, and previous findings show that high confidence indicates high accuracy in all studies of adults with an unbiased lineup. We apply CAC plots to standard old/new recognition memory data by calculating response-based and item-based accuracy, one using false alarms and the other using misses. We use both methods to examine the confidence-accuracy relationship for both correct old responses (hits) and new responses (correct rejections). We reanalysed three sets of published data using these methods and show that the method chosen, as well as the relation of lures to targets, determines the confidence-accuracy relation. Using response-based accuracy for hits, high confidence yields quite high accuracy, and this is generally true with the other methods, especially when lures are unrelated to targets. However, when analyzing correct rejections, the relationship between confidence and accuracy is less pronounced. When lures are semantically related to targets, the various CAC plots show different confidence-accuracy relations. The different methods of calculating CAC plots provide a useful tool in analyzing standard old/new recognition experiments. The results generally accord with unequal-variance signal detection models of recognition memory.
The modeling of multinomial data has seen tremendous progress since Riefer and Batchelder’s (1988) seminal paper. One recurring challenge, however, concerns the availability of relative performance measures that strike an ideal balance between goodness of fit and functional flexibility. One approach to the problem of model selection is Normalized Maximum Likelihood (NML), a solution derived from the Minimum Description Length principle. In the present work we provide an R implementation of a Gibbs sampler that can be used to compute NML for models of joint multinomial data. We discuss the application of NML in different examples, compare NML with Bayes Factors, and show how it constitutes an important addition to researchers’ toolboxes.
We define here two novel ROC properties, discrete-state representability and shift representability, which are signatures of processing mediated by discrete states and continuous latent-strength models. We show how these properties provide for superior principled inference about processing without recourse to tenuous assumptions, and that existing recognition memory data from our lab are compatible with a discrete-state model.
In "A Critical Comparison of Discrete-State and Continuous Models of Recognition Memory: Implications for Recognition and Beyond," Pazzaglia, Dube, and Rotello (2013) explored the threshold multinomial processing tree (MPT) framework as applied to several domains of experimental psychology. Pazzaglia et al. concluded that threshold MPT analyses require assumptions at the representation and measurement levels that are contradicted by existing data in several domains. Furthermore, they showed that this flaw in the threshold MPT framework produces systematic errors in data interpretation. Pazzaglia et al. suggested measures derived from the empirically validated unequal-variance signal detection theory framework as a viable alternative and provided a simple tutorial for implementing such measures in an Excel spreadsheet. In their reply, Batchelder and Alexander (2013) disputed the conclusions advanced by Pazzaglia et al. Their arguments consisted of a small number of strong assertions, some of which were accompanied by references and/or data. In this reply, we demonstrate that both types of assertions-those with and without supporting references and/or data-are, at best, contradicted by several existing studies (many of which were already discussed in Pazzaglia et al., 2013) and, at worst, patently false. We conclude that the conclusions of Pazzaglia et al. are valid. (PsycINFO Database Record (c) 2013 APA, all rights reserved).
Multinomial processing tree (MPT) models such as the single high-threshold, double high-threshold, and low-threshold models are discrete-state decision models that map internal cognitive events onto overt responses. The apparent benefit of these models is that they provide independent measures of accuracy and response bias, a claim that has motivated their frequent application in many areas of psychological science including perception, item and source memory, social cognition, reasoning, educational testing, eyewitness testimony, and psychopathology. Before appropriate conclusions about a given analysis can be drawn, however, one must first confirm that the model's assumptions about the underlying structure of the data are valid. The current review outlines the assumptions of several popular MPT models and assesses their validity using multiple sources of evidence, including receiver operating characteristics, direct model fits, and experimental tests of qualitative predictions. We argue that the majority of the evidence is inconsistent with these models and that, instead, the evidence supports continuous models such as those based on signal detection theory (SDT). Hybrid models that incorporate both SDT and MPT processes are also explored, and we conclude that these models retain the limitations associated with their threshold model predecessors. The potentially severe consequences associated with using an invalid model to interpret data are discussed, and a simple tutorial and model-fitting tool is provided to allow implementation of the empirically supported SDT model. (PsycINFO Database Record (c) 2013 APA, all rights reserved).
Model comparison in recognition memory has frequently relied on receiver operating characteristics (ROC) data. We present a meta-analysis of binary-response ROC data that builds on previous such meta-analyses and extends them in several ways. Specifically, we include more data and consider a much more comprehensive set of candidate models. Moreover, we bring to bear modern developments in model selection on the current selection problem. The new methods are based on the minimum description length framework, leading to the normalized maximum likelihood (NML) index for assessing model performance, taking into account differences between the models in flexibility due to functional form. Overall, NML results for individual ROC data indicate a preference for a discrete-state model that assumes a mixture of detection and guessing states.
We introduce MPTinR, a software package developed for the analysis of multinomial processing tree (MPT) models. MPT models represent a prominent class of cognitive measurement models for categorical data with applications in a wide variety of fields. MPTinR is the first software for the analysis of MPT models in the statistical programming language R, providing a modeling framework that is more flexible than standalone software packages. MPTinR also introduces important features such as (1) the ability to calculate the Fisher information approximation measure of model complexity for MPT models, (2) the ability to fit models for categorical data outside the MPT model class, such as signal detection models, (3) a function for model selection across a set of nested and nonnested candidate models (using several model selection indices), and (4) multicore fitting. MPTinR is available from the Comprehensive R Archive Network at http://cran.r-project.org/web/packages/MPTinR/ .
Reviews the general theory of memory as a dynamic and interactive system, develops and evaluates a mathematical model for recognition memory, and applies the model to relevant experiments: memory for items in short-term store, memory involving both long- and short-term target sets, and effects of target-list length and tests repetitions. (3 p ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Conducted 3 numerical-decision experiments to examine the cause of conservation cutoff placement in probabilistic categorization decisions. 118 Ss, predominantly university students, were shown 5-digit numbers representing the heights of men and women, and they responded to each observation depending on how certain they were that it represented the height of a man. Payoff matrices and prior probability levels were varied orthogonally and within Ss in each experiment. Prior probabilities had larger effects on cutoff location than payoff matrices, even when the bias prescribed by the payoffs was greater than that prescribed by the priors. Results suggest a probability-matching rule according to which Ss chose a cutoff point. (14 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Two experiments investigated the effects on auditory signal detection of introducing visual cues that were partially correlated
with the signal events. The results were analyzed in terms of a detection model that assumes that such cue-signal correlations
will not affect sensitivity, but will instead cause the subject to develop separate response biases for each cue. The model
specifies a functional relationship between the asymptotic values of these cuecontingent biases. The overall results of the
experiments supported the detection assumptions of the model and the general bias learning assumption, but indicated a more
complex learning process than that specified by the model.
In recognition memory, a classic finding is that receiver operating characteristics (ROCs) are curvilinear. This has been taken to support the fundamental assumptions of signal detection theory (SDT) over discrete-state models such as the double high-threshold model (2HTM), which predicts linear ROCs. Recently, however, Bröder and Schütz (2009) challenged this argument by noting that most of the data on which support for SDT is based have involved confidence ratings. The authors argued that certain types of rating scale usage may result in curved ROCs even if the generating process is thresholded in nature. From this point of view, only ROCs constructed via experimental bias manipulations are useful for discriminating between the models. Bröder and Schütz conducted a meta-analysis and new experiments that compared SDT and the 2HTM using binary (yes-no) ROCs and found that many of these functions were linear, supporting 2HTM over SDT. We examine all the data reported by Bröder and Schütz, noting important limitations in their methodology, analyses, and conclusions. We report a new meta-analysis and 2 new experiments to examine the issue more closely while avoiding the limitations of Bröder and Schütz's study. These new data indicate that binary ROCs are curved in recognition, consistent with previous findings in perception and reasoning. Our results support classic arguments in favor of SDT and indicate that curvature in ratings ROCs is not task specific. We recommend the ratings procedure and suggest that analyses based on threshold models be treated with caution.
Following G. T. Fechner (1966), thresholds have been conceptualized as the amount of intensity needed to transition between mental states, such as between states of unconsciousness and consciousness. With the advent of the theory of signal detection, however, discrete-state theory and the corresponding notion of threshold have been discounted. Consequently, phenomena such as subliminal priming and perception have a reduced theoretical basis. The authors propose a process-neutral definition of threshold that allows for graded perception and activation throughout the system. Thresholds correspond to maximum stimulus intensities such that the distribution of mental states does not differ from that when an appropriate baseline stimulus is presented. In practice, thresholds are maximum intensities such that the probability distribution on behavioral events does not differ from that from baseline. These thresholds, which the authors call task thresholds, may be estimated with modified item response psychometric measurement models.
An ongoing discussion in the recognition-memory literature concerns the question of whether recognition judgments reflect a direct mapping of graded memory representations (a notion that is instantiated by signal detection theory) or whether they are mediated by a discrete-state representation with the possibility of complete information loss (a notion that is instantiated by threshold models). These 2 accounts are usually evaluated by comparing their (penalized) fits to receiver operating characteristic data, a procedure that is predicated on substantial auxiliary assumptions, which if violated can invalidate results. We show that the 2 accounts can be compared on the basis of critical tests that invoke only minimal assumptions. Using previously published receiver operating characteristic data, we show that confidence-rating judgments are consistent with a discrete-state account. (PsycINFO Database Record
(c) 2015 APA, all rights reserved).
A classical question for memory researchers is whether memories vary in an all-or-nothing, discrete manner (e.g., stored vs. not stored, recalled vs. not recalled). or whether they vary along a continuous dimension (e.g., strength, similarity, or familiarity). For yes-no classification tasks, continuous- and discrete-state models predict nonlinear and linear receiver operating characteristics (ROCs), respectively (D. M, Green & J. A. Swets, 1966; N. A. Macmillan & C. D. Creelman, 1991). Recently, several authors have assumed that these predictions are generalizable to confidence ratings tasks (J. Qin, C. L. Raye, M. K. Johnson, & K. J. Mitchell, 2001; S. D. Slotnick, S. A. Klein, C. S. Dodson, & A. P. Shimamura, 2000, and A. P. Yonelinas, 1999). This assumption is shown to be unwarranted by showing that discrete-state ratings models predict both linear and nonlinear ROCs. The critical factor determining the form of the discrete-state ROC is the response strategy adopted by the classifier.
With the theory of signal detectability as a framework, two psychophysical experiments were conducted in which each observation interval was well defined for the listener. Each interval contained noise, and it either did or did not (p=0.5) contain a signal (1000 cps, 0.5 sec in duration). In separate sessions of the first experiment, either the listener gave a yes‐no decision or he responded with a rating (1–4) after each observation interval. Operating characteristics were obtained with E/N 0 equal to 15.8. It is clear from the data that the trained listener can perform as well when he adopts the multiple criteria necessary for the rating method as when he adopts the single criterion required by the binary‐decision procedure. In the second experiment, only the rating method was used to determine the relation between d′ and E/N 0. The resulting function, for d′ ⩽ 3.0, approximates a straight line which passes through the origin and which has nearly the same slope as that obtained in other laboratories. [This research was supported by the Operational Applications Laboratory, Air Force Cambridge Research Center, under Contract AF 19(604)‐1962.]
The 2-high-threshold (2HT) model of recognition memory assumes that test items result in distinct internal states: they are either detected or not, and the probability of responding at a particular confidence level that an item is "old" or "new" depends on the state-response mapping parameters. The mapping parameters are independent of the probability that an item yields a particular state (e.g., both strong and weak items that are detected as old have the same probability of producing a highest-confidence "old" response). We tested this conditional independence assumption by presenting nouns 1, 2, or 4 times. To maximize the strength of some items, "superstrong" items were repeated 4 times and encoded in conjunction with pleasantness, imageability, anagram, and survival processing tasks. The 2HT model failed to simultaneously capture the response rate data for all item classes, demonstrating that the data violated the conditional independence assumption. In contrast, a Gaussian signal detection model, which posits that the level of confidence that an item is "old" or "new" is a function of its continuous strength value, provided a good account of the data. (PsycINFO Database Record (c) 2014 APA, all rights reserved).
Recollection is currently modeled as a univariate retrieval process in which memory probes provoke conscious awareness of contextual details of earlier target presentations. However, that conception cannot explain why some manipulations that increase recollection in recognition experiments suppress false memory in false memory experiments, whereas others increase false memory. Such contrasting effects can be explained if recollection is bivariate—if memory probes can provoke conscious awareness of target items per se, separately from awareness of contextual details, with false memory being suppressed by the former but increased by the latter. Interestingly, these 2 conceptions of recollection have coexisted for some time in different segments of the memory literature. Independent support for the dual-recollection hypothesis is provided by some surprising effects that it predicts, such as release from recollection rejection, false persistence, negative relations between false alarm rates and target remember/know judgments, and recollection without remembering. We implemented the hypothesis in 3 bivariate recollection models, which differ in the degree to which recollection is treated as a discrete or a graded process: a pure multinomial model, a pure signal detection model, and a mixed multinomial/signal detection model. The models were applied to a large corpus of conjoint recognition data, with fits being satisfactory when both recollection processes were present and unsatisfactory when either was deleted. Factor analyses of the models’ parameter spaces showed that target and context recollection never loaded on a common factor, and the 3 models converged on the same process loci for the effects of important experimental manipulations. Thus, a variety of results were consistent with bivariate recollection.
A classic discussion in the recognition-memory literature concerns the question of whether recognition judgments are better described by continuous or discrete processes. These two hypotheses are instantiated by the signal detection theory model (SDT) and the 2-high-threshold model, respectively. Their comparison has almost invariably relied on receiver operating characteristic data. A new model-comparison approach based on ranking judgments is proposed here. This approach has several advantages: It does not rely on particular distributional assumptions for the models, and it does not require costly experimental manipulations. These features permit the comparison of the models by means of simple paired-comparison tests instead of goodness-of-fit results and complex model-selection methods that are predicated on many auxiliary assumptions. Empirical results from 2 experiments are consistent with a continuous memory process such as the one assumed by SDT. (PsycINFO Database Record (c) 2014 APA, all rights reserved).
The study of thresholds for discriminability has been of long-standing interest in psychophysics. While threshold theories embrace the concept of discrete-state thresholds, signal detection theory discounts such a concept. In this paper we concern ourselves with the concept of thresholds from the discrete-state modelling viewpoint. In doing so, we find it necessary to clarify some fundamental issues germane to the psychometric function (PF), which is customarily constructed using psychophysical methods with a binary-response format. We challenge this response format and argue that response confidence also plays an important role in the construction of PFs, and thus should have some impact on threshold estimation. We motivate the discussion by adopting a three-state threshold theory for response confidence proposed by Krantz (, Psychol. Rev., 76, 308–324), which is a modification of Luce's (, Psychol. Rev., 70, 61–79) low-threshold theory. In particular, we discuss the case in which the practice of averaging over order (or position) is enforced in data collection. Finally, we illustrate the fit of the Luce–Krantz model to data from a line-discrimination task with response confidence.
A general comparison is made between the multinomial processing tree (MPT) approach and a strength-based approach for modeling recognition memory measurement. Strength models include the signal-detection model and the dual-process model. Existing MPT models for recognition memory and a new generic MPT model, called the Multistate (MS) model, are contrasted with the strength models. Although the ROC curves for the MS model and strength model are similar, there is a critical difference between existing strength models and MPT models that goes beyond the assessment of the ROC. This difference concerns the question of stochastic mixtures for foil test trials. The hazard function and the reverse hazard function are powerful methods for detecting the presence of a probabilistic mixture. Several new theorems establish a novel method for obtaining information about the hazard function and reverse hazard function for the latent continuous distributions that are assumed in the strength approach to recognition memory. Evidence is provided that foil test trials involve a stochastic mixture. This finding occurred for both short-term memory procedures, such as the Brown–Peterson task, and long-term list-learning procedures, such as the paired-associate task. The effect of mixtures on foil trials is problematic for existing strength models but can be readily handled by MPT models such as the MS model. Other phenomena, such as the mirror effect and the effect of target-foil similarity, are also predicted accurately by the MPT modeling framework.
Pazzaglia, Dube, and Rotello (2013) have provided a lengthy critique of threshold and continuous models of recognition memory. Although the early pages of their article focus mostly on the problems they see with 3 vintage threshold models compared with models from signal detection theory (SDT), it becomes clear rather quickly that Pazzaglia et al. are concerned more generally with problems they see with multinomial processing tree (MPT) models. First, we focus on Pazzaglia et al.'s discussion of the evidence concerning receiver operating characteristics (ROCs) in simple recognition memory, then we consider problems they raise with a subclass of MPT models for more complex recognition memory paradigms, and finally we discuss the difference between scientific models and measurement models in the context of MPT and SDT models in general. We argue that Pazzaglia et al. have not adequately considered the evidence relevant to the viability of the simple threshold models and that they have not adequately represented the issues concerning validating a cognitive measurement model. We further argue that selective influence studies and model flexibility studies are as important as studies showing that a model can fit behavioral data. In particular, we note that despite over a half century of effort, no generally accepted scientific theory of recognition memory has emerged and that it is unlikely to ever emerge with studies using standard behavioral measures. Instead, we assert that useful measurement models of both the SDT and the MPT type have been and should continue to be developed. (PsycINFO Database Record (c) 2013 APA, all rights reserved).
The use of rating scale isosensitivity data to test a discrete, two‐state model for binary choice detection experiments is shown to be inappropriate unless new assumptions are made. It is argued that, whatever assumptions are added, rating‐scale isosensitivity curves will be different from binary‐choice isosensitivity curves. The suggestion is made that a multistate model may be more appropriate than a two‐state model for rating experiments, but the resulting isosensitivity curves appear to be too similar to those of signal‐detectability theory to make rating‐scale data useful for deciding between the theories.
Signal Detection models as well as the Two-High-Threshold model (2HTM) have been used successfully as measurement models in recognition tasks to disentangle memory performance and response biases. A popular method in recognition memory is to elicit confidence judgements about the presumed old/new status of an item, allowing for the easy construction of ROCs. Since the 2HTM assumes fewer latent memory states than response options are available in confidence ratings, the 2HTM has to be extended by a mapping function which models individual rating scale usage. Unpublished data from 2 experiments in Bröder and Schütz (2009) validate the core memory parameters of the model, and 3 new experiments show that the response mapping parameters are selectively affected by manipulations intended to affect rating scale use, and this is independent of overall old/new bias. Comparisons with SDT show that both models behave similarly, a case that highlights the notion that both modelling approaches can be valuable (and complementary) elements in a researcher's toolbox.
Source monitoring refers to the discrimination of the origin of information. Multinomial models of source monitoring (W. H. Batchelder & D. M. Riefer, 1990) are theories of the decision processes involved in source monitoring that provide separate parameters for source discrimination, item detection, and response biases. Three multinomial models of source monitoring based on different models of decision in a simple detection paradigm (one-high-threshold, low-threshold, and two-high-threshold models) were subjected to empirical tests. With a 3 (distractor similarity) × 3 (source similarity) factorial design, the effect of difficulty of item detection and source discrimination on corresponding model parameters was examined. Only the source-monitoring model that is based on a two-high-threshold model of item recognition provides an accurate analysis of the data. Consequences for the use of multinomial models in the study of source monitoring are discussed. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
D. A. Kinchla (see record
1994-16291-001) criticizes W. H. Batchelder and D. M. Reifer's multinomial model for source monitoring, primarily its high-threshold assumptions, and advocates an approach based on statistical decision theory (SDT). The authors lay out some of the considerations that led to their model and then raise some specific concerns with Kinchla's critique. The authors point out that most of his criticisms are drawn from contrasting the high threshold and the Gaussian, equal-variance SDT models on receiver operating characteristics (ROC) curves for yes-no recognition memory. The authors indicate how source monitioring is more complicated than yes-no recognition and question the validity of standard ROC analyses in source monitoring. The authors argue that their model is a good approximation for measuring differences between sources on old-new detection and that it has the ability to measure source discrimination as well as detection. The authors also explore a low-threshold multinomial model and discuss the application of SDT models to source monitoring. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
We provide evidence that recognition memory is mediated by a detect-or-guess mental-state model without recourse to concepts of latent-strength or multiple-memory systems. We assess performance in a two-alternative forced-choice recognition memory task with confidence ratings. The key manipulation is that sometimes participants are asked which of two new items is old, and the resulting confidence distribution is unambiguously interpreted as arising from a guessing state. The confidence ratings for other conditions are seemingly the resultant of mixing this stable guessing state with an additional stable detect state. Formal model comparison supports this observation, and an analysis of associated response times reveals a mixture signature as well.
Several signal detection experiments employing a forced-choice procedure are analysed in terms of a model that incorporates two distinct processes: a sensory process and a decision process. The sensory process specifies the relation between external signal events and hypothesized sensory states of the subject. The decision process specifies the relation between the sensory states and the observable responses of the subject. The sensory process is assumed to be fixed throughout an experiment, whereas the decision process is viewed as varying from trial to trial as a function of the particular sequence of preceding events. The changes in the decision process are assumed to be governed by a simple stochastic learning model. There are several ways of formulating the learning model and the experiments reported here were designed to select among these alternative approaches. The empirical results favour a linear-operator process with trial-to-trial changes in response probabilities that are a function not only of the signal and information events, but also of the particular sequence of sensory states activated.
An outline is given of the behavioral properties (axioms) that have been proposed, and to some extent empirically evaluated,
concerning uncertain (often risky) alternatives, the joint receipt of alternatives, and possible linking properties. Recent
theoretical work has established the existence of three inherently distinct risk types of people—risk seeking, risk neutral,
and risk averse—and so evaluations of theories must take respondent type into account. A program of experiments is sketched
making clear exactly which empirical studies need to be repeated with respondents partitioned by risk type.
Sensory differential thresholds are studied by models which partition the sensory process into two successive stages: a detection stage independent of motivational variables, and a decision stage which selects a response using the observer's motivations and expectations.The detection model considered is the neural quantum theory. Several decision models are applied to the detection model, and the existing psycho-physical literature relevant to thresholds is analyzed.Two aspects of the data, the psychometric function and isosensitivity curves (Receiver Operating Characteristic), are examined for three different experimental conditions: (1) the yes-no detection experiment; (2) the random presentation experiment; and (3) the two-alternative temporal forced-choice experiment.The observer's task is to decide whether a particular observation is caused by signal or noise. In one decision strategy, the rigid criterion, the observer reports a signal only if he detects an increase in the number of excited states of at leask k. Another decision strategy proposes a response bias: the observer reports a signal with probability tk if the number of excited states increases by exactly k. This model provides an explanation for the way in which the psychometric functions of quantum theory are transformed by changes in experimental procedure.
Three experiments were performed to extend the previous finding that number of cate-gories (NC) in organized, categorized lists determines the number of words recalled. The NC also influences recognition both in immediate tests and in a delay of two weeks. False alarm rates in recognition are generally unaffected by the use of synonyms as fillers, suggesting that perceptual features of words are used at least in addition to semantic features. To accommodate the novel finding that organization affects recognition, a model for the case of subject-organized lists was presented which introduces the notion of a postrecognition retrieval check. Previous findings on the relation between NC and recall were replicated.
When items on one list receive more encoding than items on another list, the improvement in performance usually manifests as an increase in the hit rate and a decrease in the false alarm rate (FAR). A common account of this strength based mirror effect is that participants adopt a more strict criterion following a strongly than weakly encoded list (e.g.,
[6] and [56]). Differentiation models offer an alternative: more encoding leads to a more accurate memory representation for the studied item. A more accurate representation is less confusable with an unrelated item, resulting in a decrease in the FAR (
[38] and [54]). Differentiation models make additional predictions about reversals in FARs for foils similar to a studied item as a function of the composition of the study list. These predictions were empirically tested and confirmed.
This article is a personal commentary on the following, to my mind, unresolved issues of mathematical psychology: (1) the failure of the field to have become a fully accepted part of most departments of psychology; (2) the great difficulty we have in studying dynamic mechanisms, e.g., learning, because large samples are difficult to obtain: time samples wipe out the phenomena and subject samples are unrepresentative because of profound and ill understood individual differences; (3) the failure to unify successfully statistical and measurement theories which, I believe, are two facets of a common problem; (4) the proliferation of free parameters in many types of theories with little success in developing theories of such parameters; (5) the difficulties we have had in successfully formulating the mathematics of uncertainty and vagueness; and (6) the issues of modeling what are presumably discrete attributes and phenomena by continuous mathematics: when and how is this justified?
Steingrimsson (Attention, Perception, & Psychophysics, 71, 1916–1930, 2009) outlined Luce’s (Psychological Review, 109, 520–532 2002, 111, 446–454 2004) proposed psychophysical theory and tested, for brightness, behavioral properties that, separately, gave rise to two psychophysical functions, Ψ
⊕ and \( {\Psi_{{ \circ_p}}} \). The function Ψ
⊕ maps pairs of physical intensities onto positive real numbers and represents subjective summation, and the function \( {\Psi_{{ \circ_p}}} \) represents a form of ratio production. This article, the second in a series expected to consist of three articles, tests the properties linking summation and production such that it forces \( {\Psi_{{ \circ_p}}} = {\Psi_\oplus } = \Psi \). The properties tested are a form of distributivity and, in three experiments, were subjected to an empirical evaluation. Considerable support is provided for the existence of a single function Ψ for both summation and ratio production. The scope of this series of articles is to establish the theory as a descriptive model of binocular brightness perception.
In source-monitoring experiments, participants study items from two sources (A and B). At test, they are presented Source A items, Source B items, and new items. They are asked to decide whether a test item is old or new (item memory) and whether it is a Source A or a Source B item (source memory). Hautus, Macmillan, and Rotello (2008) developed models, couched in a bivariate signal detection framework, that account for item and source memory across several data sets collected in a confidence-rating response format. The present article enlarges the set of candidate models with a discrete-state model. The model is a straightforward extension of Bayen, Murnane, and Erdfelder's (1996) multinomial model of source discrimination to confidence ratings. On the basis of the evaluation criteria adopted by Hautus et al., it provides a better account of the data than do Hautus et al.'s models.
In differentiation models, the processes of encoding and retrieval produce an increase in the distribution of memory strength for targets and a decrease in the distribution of memory strength for foils as the amount of encoding increases. This produces an increase in the hit rate and decrease in the false-alarm rate for a strongly encoded compared with a weakly encoded list, consistent with empirical data. Other models assume that the foil distribution is unaffected by encoding manipulations or the foil distribution increases as a function of target strength. They account for the empirical data by adopting a stricter criterion for strongly encoded lists relative to weakly encoded lists. The differentiation and criterion shift explanations have been difficult to discriminate with accuracy measures alone. In this article, reaction time distributions and accuracy measures are collected in a list-strength paradigm and in a response bias paradigm in which the proportion of test items that are targets is manipulated. Diffusion model analyses showed that encoding strength is primarily accounted for by changes in the rate of accumulation of evidence (i.e., drift rate) for both targets and foils and manipulating the proportion of targets is primarily accounted for by changes in response bias (i.e., starting point). The diffusion model analyses is interpreted in terms of predictions of the differentiation models in which subjective memory strength is mapped directly onto drift rate and criterion placement is mapped onto starting point. Criterion shift models require at least 2 types of shifts to account for these findings.
Models of recognition memory assume that memory decisions are based partially on the subjective strength of the test item. Models agree that the subjective strength of targets increases with additional time for encoding however the origin of the subjective strength of foils remains disputed. Under the fixed strength assumption the distribution of memory strength for foils is invariant across experimental manipulations of encoding. For example, the subjective strength of foils may depend solely on the pre-experimental history of the item, thus encoding manipulations have no impact. In contrast, under the differentiation assumption the subjective strength of foils depends on the nature of the traces stored in episodic memory. If those traces are well encoded, the subjective strength of foils will be lower than the case where noisy traces are stored (e.g., when targets received minimal encoding). The fixed strength and differentiation accounts are tested by measuring direct ratings of memory strength. In Experiments 1 and 2, item strength is varied via repetition and in Experiment 3 response bias is varied via the relative proportion of targets on the test list. For all experiments empirical distributions of memory strength were obtained and compared to the distributions predicted by the two accounts. The differentiation assumption provides the most parsimonious account of the data.
Recent reviews of recognition receiver operating characteristics (ROCs) claim that their curvilinear shape rules out threshold models of recognition. However, the shape of ROCs based on confidence ratings is not diagnostic to refute threshold models, whereas ROCs based on experimental bias manipulations are. Also, fitting predicted frequencies to actual data is a more sensitive method for model comparisons than ROC regressions. In a reanalysis of 59 published data sets, the 2-high-threshold model (2HTM) fit the data better than an unequal variance signal detection model in about half of the cases. Three recognition experiments with experimental bias manipulation were conducted that yielded linear ROCs and a better fit of the 2HTM in all cases. On the basis of actual data and a simulation, the authors argue that both models are at least equally valid as measurement tools and can perhaps be integrated theoretically.
Four issues are discussed concerning Thurstone's discriminal processes: the distributions governing the representation, the
nature of the response decision rules, the relation of the mean representation to physical characteristics of the stimulus,
and factors affecting the variance of the representation. A neural schema underlying the representation is proposed which
involves samples in time of pulse trains on individual neural fibers, estimators of parameters of the several pulse trains,
samples of neural fibers, and an aggregation of the estimates over the sample. The resulting aggregated estimate is the Thurstonian
representation. Two estimators of pulse rate, which is monotonic with signal intensity, are timing and counting ratios and
two methods of aggregation are averaging and maximizing. These lead to very different predictions in a speed-accuracy experiment;
data indicate that both estimators are available and the aggregation is by averaging. Magnitude estimation data are then used
both to illustrate an unusual response rule and to study the psychophysical law. In addition, the pattern of variability and
correlation of magnitude estimates on successive trials is interpreted in terms of the sample size over which the aggregation
takes place. Neural sample size is equated with selective attention, and is an important factor affecting the variability
of the representation. It accounts for the magical number seven phenomenon in absolute identification and predicts the impact
of nonuniform distributions of intensities on the absolute identification of two frequencies.
SUCH TESTS REQUIRE THE VALIDITY OF CERTAIN ADDITIONAL ASSUMPTIONS ABOUT THE OPERATION OF THE DECISION SYSTEM WHICH MAPS THE STATES OF THE PERCEPTUAL OR MEMORY SYSTEM INTO RESPONSES. IN THE CASE OF OPERATING CHARACTERISTICS (OCS) GENERATED BY THE PAYOFF METHOD USING BINARY RESPONSES, THE REQUIRED DECISION-MAKING ASSUMPTION IS SPECIFIED BY LUCE. HIS 2-STATE DECISION RULE HAS BEEN TESTED AND SHOWN TO BE VALID. THUS, THE BINARY-RESPONSE OC GENERATED BY THE PAYOFF METHOD CAN BE USED TO BOTH ACCEPT AND REJECT THE 2-STATE HYPOTHESIS IN ANY SITUATION. QUESTIONS ARE RAISED WITH REGARD TO OTHER METHODS OF GENERATING OCS AND THEIR SUITABILITY FOR TESTING THE 2-STATE HYPOTHESIS. CONFIDENCE JUDGMENTS ARE FOUND TO BE LESS USEFUL FOR TESTING 2-STATE THEORIES, WHETHER THE RESULTS ARE ANALYZED BY OCS OR A POSTERIORI PROBABILITY FUNCTIONS.