Content uploaded by Travis M Seale-Carlisle
Author content
All content in this area was uploaded by Travis M Seale-Carlisle on Sep 29, 2019
Content may be subject to copyright.
A preview of the PDF is not available
Confidence and Response Time as Indicators of Eyewitness Identification Accuracy in the Lab and in the Real World
Abstract
The criminal justice system should consider the confidence an eyewitness expresses when making an identification at the time the initial lineup procedure is conducted. High confidence expressed at this time typically indicates high accuracy in the identification. Because the suspect identification – not filler identifications or no identifications – matters most in the court of law, confidence-accuracy characteristic (CAC) analysis provides information most relevant to stakeholders. However, just as high confidence identifications indicate high accuracy, fast identifications may also indicate high accuracy. We tested whether a new technique that is similar to CAC analysis, called response time-accuracy characteristic (RAC) analysis, could inform stakeholders about the likely accuracy of an identification while usefully summarizing response time data. We argue this is the case in the lab and in the real world. Furthermore, CAC and RAC results are not completely redundant so both, considered together, are useful to the criminal justice system.
... Researchers have discovered boundary conditions or exceptions to the specifications of high confidenceaccuracy calibration made by Wixted and Wells (Colloff et al., 2016;Giacona et al., 2021;Grabman et al., 2019;Lockamyeir et al., 2020;Seale-Carlisle et al., 2019;Semmler et al., 2018). For example, when multiple estimator variable conditions are poor, high confidence identifications are less reliable no matter how unspoiled the identification conditions (Giacona et al., 2021). ...
... To test our hypotheses about confidence, we report Confidence-Accuracy Characteristic analyses (CAC; Mickes, 2015;Seale-Carlisle et al., 2019). To conduct the confidence-accuracy analyses we used the following R packages: here (Muller & Bryan, 2020), readxl (Wickham & Bryan, 2022), dplyr , r4lineups (Tredoux & Naylor, 2018), boot (Canty & Ripley, 2021;Davison & Hinkley, 1997), ggplot2 (Wickham, 2016), psych (Revelle, 2022), and tidyverse (Wickham et al., 2019). ...
... Most recently, Wixed and Wells's (2017) influential paper found that confidence and accuracy are well calibrated under what they called 'pristine' lineup procedure conditions. However, several studies have found that this relationship does not hold when encoding conditions are poor (e.g., Colloff et al., 2016;Giacona et al., 2021;Grabman et al., 2019;Lockamyeir et al., 2020;Seale-Carlisle et al., 2019;Semmler et al., 2018) though others show that it does (e.g., Semmler, et al., 2018). In the present study, participants were fairly well calibrated (analysis of c), but there were no significant differences between conditions (OU analyses), and confidence ratings did not discriminate between correct and incorrect responses well (ANDI). ...
Research on eyewitness identification often involves exposing participants to a simulated crime and later testing memory using a lineup. We conducted a systematic review showing that pre-event instructions, instructions given before event exposure, are rarely reported and those that are reported vary in the extent to which they warn participants about the nature of the event or tasks. At odds with the experience of actual witnesses, some studies use pre-event instructions explicitly warning participants of the upcoming crime and lineup task. Both the basic and applied literature provide reason to believe that pre-event instructions may affect eyewitness identification performance. In the current experiment, we tested the impact of pre-event instructions on lineup identification decisions and confidence. Participants received non-specific pre-event instructions (i.e., "watch this video") or eyewitness pre-event instructions (i.e., "watch this crime video, you'll complete a lineup later") and completed a culprit-absent or -present lineup. We found no support for the hypothesis that participants who receive eyewitness pre-event instructions have higher discriminability than participants who receive non-specific pre-event instructions. Additionally, confidence-accuracy calibration was not significantly different between conditions. However, participants in the eyewitness condition were more likely to see the event as a crime and to make an identification than participants in the non-specific condition. Implications for conducting and interpreting eyewitness identification research and the basic research on instructions and attention are discussed.
... for the judge and jurors in the courtroom, information that indicates how accurate a witness's response is likely to be is of primary importance (Mickes, 2015(Mickes, , 2016. The confidence of young adult witnesses when they make an initial identification has been shown to be informative about the accuracy of their choice (Brewer & Wells, 2006;Grabman et al., 2019;Juslin et al., 1996;Seale-Carlisle et al., 2019a, 2019bSemmler et al., 2018;Wilson et al., 2018). This has also been reported with older adults when identification is made after a short delay, even when their overall discriminability is lower than younger adults (Colloff et al., 2017). ...
... It has also been shown that, for young adult witnesses, the speed with which initial lineup identification decisions are made is informative about the accuracy of their responses with faster responses being more accurate (e.g. Dodson & Dobolyi, 2016;Dunning & Perretta, 2002;Sauerland & Sporer, 2009;Seale-Carlisle et al., 2019a, 2019bSmith et al., 2001;Sporer, 1992Sporer, , 1993Weber et al., 2004). Furthermore, response time and confidence together predicted suspect identification accuracy better than each alone (Seale-Carlisle et al., 2019a, 2019b. ...
... Dodson & Dobolyi, 2016;Dunning & Perretta, 2002;Sauerland & Sporer, 2009;Seale-Carlisle et al., 2019a, 2019bSmith et al., 2001;Sporer, 1992Sporer, , 1993Weber et al., 2004). Furthermore, response time and confidence together predicted suspect identification accuracy better than each alone (Seale-Carlisle et al., 2019a, 2019b. For a particular level of confidence suspect identification accuracy was higher when the response was made quickly than when it was made slowly (Seale-Carlisle et al., 2019a, 2019b. ...
The world population is getting older and, as a result, the number of older victims of crime is expected to increase. It is therefore essential to understand how ageing affects eyewitness identification, so procedures can be developed that enable victims of crime of all ages to provide evidence as accurately and reliably as possible. In criminal investigations, witnesses often provide a description of the perpetrator of the crime before later making an identification. While describing the perpetrator prior to making a lineup identification can have a detrimental effect on identification in younger adults, referred to as verbal overshadowing, it is unclear whether older adults are affected in the same way. Our study compared lineup identification of a group of young adults and a group of older adults using the procedure that has consistently revealed verbal overshadowing in young adults. Participants watched a video of a mock crime. Following a 20-min filled delay, they either described the perpetrator or completed a control task. Immediately afterwards, they identified the perpetrator from a lineup, or indicated that the perpetrator was not present, and rated their confidence. We found that describing the perpetrator decreased subsequent correct identification of the perpetrator in both young and older adults. This effect of verbal overshadowing was not explained by a change in discrimination but was consistent with participants adopting a more conservative criterion. Confidence and response time were both found to predict identification accuracy for young and older groups, particularly in the control condition.
... A previous study examined the response time-accuracy characteristic (RAC) analysis that reveals the correlation between witnesses' response time and the suspect identification accuracy. Results of the lab study showed that as response time increased, accuracy decreased [12]. In the field study on a police department, response time was associated with confidence identifications in both suspect and filler identification, but only significant when the level of confidence was high [12]. ...
... Results of the lab study showed that as response time increased, accuracy decreased [12]. In the field study on a police department, response time was associated with confidence identifications in both suspect and filler identification, but only significant when the level of confidence was high [12]. Hence, response time is another strong indicator of suspect identification accuracy besides confidence level, and the RAC analysis should be used along with the confidence-accuracy characteristic analysis. ...
False memory, modulated by factors, such as emotion and semantic information about an event, can impede accurate suspect identification. Negative mood and semantic processing are found to create false memory by establishing associations between events. The associations include confounding the events’ order, mixing the events’ contents, and using schemas that sometimes deviate from the actual circumstances. The inconsistency of emotion while encoding and retrieving information can also cause the failure of recalling correct crime scene details. To increase the value of eyewitness testimony, this review discussed several factors that can impact eyewitness memory accuracy. Lab studies are largely based on individual cognitive differences, the level of confidence made with suspect identifications, and individual cross-cultural differences. High executive functional (EF) availability and high confidence level were found to corroborate higher accuracy of identification across tasks. Individuals from individualistic cultures were found to focus more on central details than those from collectivistic cultures. Additionally, eyewitness memory accuracy can be assessed by the response time of witnesses, which is another powerful indicator of eyewitness memory accuracy besides confidence. The difference in eyewitness memory accuracy between lab studies and reality is a gap in this field. It is recommended to use the two one-sided tests (TOST) to show that there is no statistical association between suspect identification accuracy and memory strength to minimise the gap. This review analysed several factors impacting eyewitness memory accuracy and provided insights into how the lab studies can be applied to the real world.
... But can decision time be considered a reliable predictor for accuracy when an eyewitness has identified someone? Our results echo previous findings (Seale-Carlisle et al., 2019) suggesting that it can: identifications were more likely to be accurate as decision time decreased. However, this relationship held true only when the lineup members did not wear masks. ...
While research has shown that wearing a disguise hinders lineup identifications, less is known about how to conduct lineups in cases of disguised perpetrators. We examined the influence of surgical masks, worn during a crime event (encoding) and within lineups (retrieval), on eyewitness identification accuracy. In our experiment, 452 participants watched a mock-crime video and identified the perpetrator from either a target-present or a target-absent simultaneous lineup. Contrary to expectations based on the encoding specificity principle, we did not find that matching the presence of masks in the lineup to the encoding condition increased identification accuracy. Instead, compared to the condition with no masks at encoding and retrieval, the presence of masks at either stage negatively affected discriminability and undermined the predictive utility of confidence and decision time. Our findings indicate that when a witness has encountered a masked perpetrator, presenting them with a masked lineup may not be necessary.
... Aside from repeated identification procedures, confidence in initial identification procedures is also a considerable factor that can avoid misidentification. Although a police-department-based field study involving witnesses of real crimes [12] showed that high confidence and rapid identifications were highly accurate, some data also suggests that confidence is unreliable evidence. Previous studies [8,9] on repeated identification procedures described that repeated identifications might exaggerate eyewitness confidence. ...
Misidentification is detrimental in judicial proceedings. People perceive memories as the reality that has happened, but they are not exact images in their minds, and thus memory can be distorted, and false memories can be implanted into people's minds. Numerous studies highlight the frequent adverse effects of eyewitness identification methods. In order to reduce misidentification in judicial trials, this study explored the influencing factors of misidentification. Based on the previous studies, the present study describes misidentification and the influencing factors, including suggestive information, repeated identification procedures, and the witnesses' confidence. Recommendations are made regarding suggestive information that can influence eyewitnesses decisions, the evidence is contaminated from repeated identification procedures, and only confidence in the initial identification can be considered reliable evidence. The significance of this study is beneficial for researchers to increase their insights into the influencing factors in misidentification, and it also has implications for reducing misidentification in judicial trials in the future.
... To be able to further estimate not only the speed and the accuracy of a decision, it also appears worthwhile to assess the confidence of the judgment of a decision (Ratcliff et al., 2016;Seale-Carlisle et al., 2019). Only a few studies in the team sports domain exist that have put the focus on the athletes' confidence in decisions. ...
Expert athletes are determined to make faster and better decisions, as revealed in several simple heuristic studies using verbal reports or micro-movement responses. However, heuristic decision-making experiments that require motor responses, also being considered as the embodied-choice experiments, are still underrepresented. Furthermore, it is less understood how decision time and confidence depend on the type of embodied choices players make. To scrutinize the decision-making processes (i.e., decision time, decision confidence), this study investigated the embodied choices of male athletes with different expertise in a close-to-real-life environment; 22 elite ( M age = 17.59 yrs., SD = 3.67), and 22 amateur ( M age = 20.71 yrs., SD = 8.54) team handball players performed a sport-specific embodied-choice test. Attack sequences ( n = 32) were shown to the players, who had to choose between four provided options by giving a respective sport-specific motor response. We analyzed the frequencies of specific choices and the best choice , as well as the respective decision time and decision confidence. Elite and amateur players differed in the frequencies of specific choices (i.e., forward/tackling; passive blocking), and elite players made the best choice more often. Slower decision times of elite players were revealed in specific choices and in best choices , the confidence of decisions was rated equally high by both player groups. Indications are provided that elite players make better choices rather slower, instead of faster. We suppose this is due to specific sensorimotor interactions and speed-accuracy-tradeoffs in favor of accuracy in elite players. Our findings extend expert decision-making research by using an embodied-choice paradigm, highlighting considerations of decision time and confidence in future experiments.
pyWitness is a python toolkit for recognition memory experiments, with a focus on eyewitness identification (ID) data analysis and model fitting. The current practice is for researchers to use different statistical packages to analyze a single dataset. pyWitness streamlines the process. In addition to conducting key data analyses (e.g., receiver operating characteristic analysis, confidence accuracy characteristic analysis), statistical comparisons, signal-detection-based model fits, simulated data generation, and power analyses are also possible. We describe the package implementation and provide detailed instructions and tutorials with datasets so that users can follow. There is also an online manual that is regularly updated. We developed pyWitness to be user-friendly, reduce human interaction with pre-processing and processing of data and model fits, and produce publication-ready plots. All pyWitness features align with open science practices, such that the algorithms, fits, and methods are reproducible and documented. While pyWitness is a python toolkit, it can also be used from R for users more accustomed to this environment.
Is confidence most diagnostic of accuracy if expressed in numbers or is confidence best expressed in words? This question bears immense importance in many real-world contexts especially within the confines of eyewitness identification. In an eyewitness identification task, we compared the diagnostic value of numeric confidence across rating scales that varied in grain size (3-point vs. 6-point vs. 21-point vs. 101-point rating scales). We also compared the diagnostic value of numeric confidence to verbal confidence statements using several machine-learning algorithms. We find that fine-grain ratings are more diagnostic of identification accuracy than coarse-grain ratings, which suggests that the former provide a closer correspondence to memory strength than the latter. Moreover, we find that verbal confidence statements capture diagnostic information about the likely accuracy of an identification that numeric confidence ratings do not capture. This suggests that verbal confidence statements and numeric confidence ratings reflect partially independent, non-overlapping sources of information. These results shed light on the processes that provide diagnostic value to confidence. From an applied standpoint, these results suggest that verbal confidence statements and numeric confidence ratings ought to be collected from eyewitnesses after an identification decision. Collecting both captures more diagnostic information than either can capture in isolation.
Adult Eyewitness Testimony: Current Trends and Developments provides an overview of empirical research on eyewitness testimony and identification accuracy, covering both theory and application. The volume is organized to address three important issues. First, what are the cognitive, social and physical factors that influence the accuracy of eyewitness reports? Second, how should lineups be constructed and verbal testimony be taken to improve the chances of obtaining accurate information? And third, whose testimony should be believed? Are there differences between accurate and inaccurate witnesses, and can jurors make such a distinction? Adult Eyewitness Testimony: Current Trends and Developments is crucial reading for memory researchers, as well as police officers, judges, lawyers and other members of the judicial system.
Increasing research shows that high eyewitness confidence at the time of an initial identification is a strong predictor of accuracy (Wixted & Wells, 2017). However, as with all forms of criminal evidence, this relationship is imperfect. This study addresses whether there are variables that systematically influence the rate of high confidence misidentifications. Notably, this is the first study to document the influence of face recognition ability on the confidence-accuracy relationship. Participants viewed photos of individuals of their same race or a different race, and performed a lineup recognition test after either a 5-minute (n = 277) or 1-day (n = 292) delay. High confidence identification errors were more likely when a) individuals are worse face recognizers, b) decision-times are slow, and c) responses are justified with references to familiarity (e.g., “He looks familiar).
How can lineups be designed to elicit the best achievable memory performance? One step toward that goal is to compare lineup procedures. In a recent comparison of US and UK lineup procedures, discriminability and reliability was better when memory was tested using the US procedure. However, because there are so many differences between the procedures, it is unclear what explains this superior performance. The main goal of the current research is therefore to systematically isolate the differences between the US and UK lineups to determine their effects on discriminability and reliability. In five experiments, we compared (1) presentation format: simultaneous vs. sequential; (2) stimulus format: photos vs. videos; (3) number of views: 1-lap vs. 2-lap vs. choice in both video and photo lineups; and (4) lineup size: 6- versus 9-lineup members. Most of the comparisons did not show appreciable differences, but one comparison did: simultaneous presentation yielded better discriminability than sequential presentation. If the results replicate, then policymakers should recommend using a simultaneous lineup procedure. Moreover, consistent with previous research, identifications made with high confidence were higher in reliability than identifications made with low confidence. Thus, official lineup protocols should require collecting confidence because of the diagnostic value added.
Researchers use a wide range of confidence scales when measuring the relationship between confidence and accuracy in reports from memory, with the highest number usually representing the greatest confidence (e.g., 4-point, 20-point, and 100-point scales). The assumption seems to be that the range of the scale has little bearing on the confidence-accuracy relationship. In two old/new recognition experiments, we directly investigated this assumption using word lists (Experiment 1) and faces (Experiment 2) by employing 4-, 5-, 20-, and 100-point scales. Using confidence-accuracy characteristic (CAC) plots, we asked whether confidence ratings would yield similar CAC plots, indicating comparability in use of the scales. For the comparisons, we divided 100-point and 20-point scales into bins of either four or five and asked, for example, whether confidence ratings of 4, 16–20, and 76–100 would yield similar values. The results show that, for both types of material, the different scales yield similar CAC plots. Notably, when subjects express high confidence, regardless of which scale they use, they are likely to be very accurate (even though they studied 100 words and 50 faces in each list in 2 experiments). The scales seem convertible from one to the other, and choice of scale range probably does not affect research into the relationship between confidence and accuracy. High confidence indicates high accuracy in recognition in the present experiments.
Estimator variables are factors that can affect the accuracy of eyewitness identifications but that are outside of the control of the criminal justice system. Examples include (1) the duration of exposure to the perpetrator, (2) the passage of time between the crime and the identification (retention interval), (3) the distance between the witness and the perpetrator at the time of the crime. Suboptimal estimator variables (e.g., long distance) have long been thought to reduce the reliability of eyewitness identifications (IDs), but recent evidence suggests that this is not true of IDs made with high confidence and may or may not be true of IDs made with lower confidence. The evidence suggests that while suboptimal estimator variables decrease discriminability (i.e., the ability to distinguish innocent from guilty suspects), they do not decrease the reliability of IDs made with high confidence. Such findings are inconsistent with the longstanding “optimality hypothesis” and therefore require a new theoretical framework. Here, we propose that a signal-detection-based likelihood ratio account – which has long been a mainstay of basic theories of recognition memory – naturally accounts for these findings.
Verbally describing a face has been found to impair subsequent recognition of that face from a photo lineup, a phenomenon known as the verbal overshadowing effect (Schooler & Engstler-Schooler, 1990). Recently, a large direct replication study successfully reproduced that original finding (Alogna et al., 2014). However, in both the original study and the replication studies, memory was tested using only target-present lineups (i.e., lineups containing the previously seen target face), making it possible to compute the correct identification rate (correct ID rate; i.e., the hit rate) but not the false identification rate (false ID rate; i.e., the false alarm rate). Thus, the lower correct ID rate for the verbal condition could reflect either reduced discriminability or a conservative criterion shift relative to the control condition. In four verbal overshadowing experiments reported here, we measured both correct ID rates and false ID rates using photo lineups (Experiments 1 and 2) or single-photo showups (Experiments 3 and 4). The experimental manipulation (verbally describing the face or not) occurred either immediately after encoding (Experiments 1 and 3) or 20-min after encoding (Experiments 2 and 4). In the immediate condition, discriminability did not differ between groups, but in the delayed condition, discriminability was lower in the verbal description group (i.e., a verbal overshadowing effect was observed). A fifth experiment found that the effect of the immediate-versus-delayed manipulation may be attributable to a change in the content of verbal descriptions, with the ratio of diagnostic to generic facial features in the descriptions decreasing as delay increases. (PsycINFO Database Record
The U.S. legal system increasingly accepts the idea that the confidence expressed by an eyewitness who identified a suspect from a lineup provides little information as to the accuracy of that identification. There was a time when this pessimistic assessment was entirely reasonable because of the questionable eyewitness-identification procedures that police commonly employed. However, after more than 30 years of eyewitness-identification research, our understanding of how to properly conduct a lineup has evolved considerably, and the time seems ripe to ask how eyewitness confidence informs accuracy under more pristine testing conditions (e.g., initial, uncontaminated memory tests using fair lineups, with no lineup administrator influence, and with an immediate confidence statement). Under those conditions, mock-crime studies and police department field studies have consistently shown that, for adults, (a) confidence and accuracy are strongly related and (b) high-confidence suspect identifications are remarkably accurate. However, when certain non-pristine testing conditions prevail (e.g., when unfair lineups are used), the accuracy of even a high-confidence suspect ID is seriously compromised. Unfortunately, some jurisdictions have not yet made reforms that would create pristine testing conditions and, hence, our conclusions about the reliability of high-confidence identifications cannot yet be applied to those jurisdictions. However, understanding the information value of eyewitness confidence under pristine testing conditions can help the criminal justice system to simultaneously achieve both of its main objectives: to exonerate the innocent (by better appreciating that initial, low-confidence suspect identifications are error prone) and to convict the guilty (by better appreciating that initial, high-confidence suspect identifications are surprisingly accurate under proper testing conditions).
Crimes can occur in a matter of seconds, with little time available for an eyewitness to encode a
perpetrator’s face. The presence of a weapon can further exacerbate this situation. Few studies
have featured mock crimes of short duration, especially with a weapon manipulation. We
conducted an experiment to investigate the impact of weapon presence and short perpetrator
exposure times (3 versus 10 s) on eyewitness confidence and accuracy. We found that recall
concerning the perpetrator was worse when a weapon was present, replicating the weapon focus
effect. However, there was no effect on eyewitness identification accuracy. Calibration analyses
revealed that all conditions produced a strong confidence-accuracy relationship. Confidence-
Accuracy Characteristic curves illustrated almost perfect accuracy for suspect identifications at
the highest levels of confidence. We conclude that weapon presence during a brief crime does
not necessarily result in negative consequences for either eyewitness identification accuracy or
the confidence-accuracy relationship.
This article documents a contradiction between objective eyewitness accuracy and perceived eyewitness accuracy. Objectively, eyewitness identification accuracy (and the confidence-accuracy relationship) is comparably strong when a lineup identification is accompanied by a justification that refers to either an observable feature about the suspect (“I remember his eyes”), an unobservable feature (“He looks like a friend of mine”) or just a statement of recognition (“I recognize him”). There is, however, a weaker relationship between confidence and accuracy and an increase in high confidence errors for identifications that are accompanied by references to familiarity than by references to any other type of justification. With respect to perceived accuracy, we document a robust cognitive bias—the featural justification effect—that causes eyewitnesses to be regarded by others as less accurate and less confident when they justify their identification by referring to an observable feature as compared to when they give any other kind of justification, except for a reference to familiarity.
For decades, sequential lineups have been considered superior to simultaneous lineups in the context of eyewitness identification. However, most of the research leading to this conclusion was based on the analysis of diagnosticity ratios that do not control for the respondent’s response criterion. Recent research based on the analysis of ROC curves has found either equal discriminability for sequential and simultaneous lineups, or higher discriminability for simultaneous lineups. Some evidence for potential position effects and for criterion shifts in sequential lineups has also been reported. Using ROC curve analysis, we investigated the effects of the suspect’s position on discriminability and response criteria in both simultaneous and sequential lineups. We found that sequential lineups suffered from an unwanted position effect. Respondents employed a strict criterion for the earliest lineup positions, and shifted to a more liberal criterion for later positions. No position effects and no criterion shifts were observed in simultaneous lineups. This result suggests that sequential lineups are not superior to simultaneous lineups, and may give rise to unwanted position effects that have to be considered when conducting police lineups.