Project

The dubious reliability of implicit cognition research: Current problems and ways forward

Goal: The study of unconscious cognitive processes is becoming one of the most vibrant and promising areas of research in psychological science. Many of these studies adopt the view, supported by a large volume of research, that elementary learning and memory processes demand few cognitive resources and can take place in the absence of awareness. This includes popular effects like sequence learning, artificial grammar learning, or visual statistical learning. The most common strategy to show that these effects are unconscious relies on a simple dissociation: After repeated exposure to a set of materials, participants become faster or more accurate at responding to these stimuli, revealing successful learning. However, when asked whether they recognise having been exposed to these stimuli in the past, they say no or they seem to be unable to discriminate between the stimuli they have seen and completely new stimuli. Furthermore, their score in this awareness test seems to be uncorrelated with performance in the learning task. In all cases, the lack of awareness is inferred from a null statistical result. In previous studies, we have claimed that this strategy is methodologically unsound because null results have an ambiguous interpretation in traditional, null hypothesis significance testing: They can mean that the null hypothesis is true or simply that the data are not sensitive enough to discriminate between the null and the alternative hypothesis. In the present project, we will highlight an additional problem in the interpretation of these results: The limited reliability of the dependent measures used to quantify learning and awareness. Our preliminary results suggest that, at least in the case of contextual cueing –an increasingly popular implicit learning paradigm– these reliabilities might be in the order of .10 to .20. The main goal of the present proposal is to explore and ameliorate the impact of low reliability in implicit cognition research. We will conduct several large-scale studies to estimate the reliability of the measures collected in contextual cueing and other implicit learning paradigms (probabilistic cueing of visual attention and multiple-cue probability learning). This information will allow us to use cutting-edge meta-analytic methods designed to correct for the attenuating effect of low reliability. The results of these meta-analyses can change dramatically our understanding of implicit learning, revealing, for instance, that the contribution of awareness to these effects is much larger than previously thought. If these reliabilities turn out to be as low as we expect, then it also follows that much of the applied research showing large and significant correlations between implicit learning processes and personal characteristics, like specific language impairment or dyslexia, must be biased, possibly due to selective publication or selective reporting. Our research will explore this possibility by using new methods for the detection and correction of these biases. Finally, the present proposal will devise new analytic methods, based on Bayesian hypothesis testing and computational modeling, to overcome the limitations of previous research on implicit learning.

Date: 1 January 2018 - 1 May 2021

Updates
0 new
0
Recommendations
0 new
0
Followers
0 new
26
Reads
0 new
104

Project log

Miguel A. Vadillo
added a research item
It is usually easier to find objects in a visual scene as we gain familiarity with it. Two decades of research on contextual cuing of visual search show that repeated exposure to a search display can facilitate the detection of targets that appear at predictable locations in that display. Typical accounts for this effect attribute an essential role to learned associations between the target and other stimuli in the search display. These associations improve visual search either by driving attention towards the usual location of the target or by facilitating its recognition. Contrary to this view, we show that a robust contextual cuing effect can also be observed when repeated search displays do not allow the location of the target to be predicted. These results suggest that, in addition to the mechanisms already explored by previous research, participants learn to ignore the locations usually occupied by distractors, which in turn facilitates the detection of targets even when they appear in unpredictable locations.
Miguel A. Vadillo
added a research item
People usually become faster at finding a visual target after repeated exposure to the same search display. This effect, known as contextual cueing, is often thought to rely on a highly efficient learning mechanism, relatively unconstrained by the availability of attentional resources. Consistent with this view, experimental evidence suggests that contextual cueing can be found even when participants are instructed to ignore the repeated visual context, although this learning remains latent until the context receives full attention. The present study explores the contribution of selective attention to contextual cueing in four high-powered preregistered experiments. None of them supported the hypothesis that latent learning can occur without selective attention. In general, our results suggest that selective attention to visual context plays an essential role in both the acquisition and the expression of contextual cueing.
Miguel A. Vadillo
added 2 research items
Recent debate about the reliability of psychological research has raised concerns about the prevalence of false positives in our discipline. However, false negatives can be just as concerning in areas of research that depend on finding support for the absence of an effect. This risk is particularly high in unconscious learning experiments, where researchers commonly seek to demonstrate that people can learn to perform a task in the absence of any explicit knowledge of the information that drives performance. The fact that some unconscious learning effects are typically studied with small samples and unreliable awareness measures makes false negatives especially likely. In the present article we focus on a popular unconscious learning paradigm, probabilistic cuing of visual attention, as a case study. Firstly, we show that, at the meta-analytic level, previous experiments reveal positive signs of participant awareness, although individual studies are severely underpowered to detect this. Secondly, we report the results of two empirical studies in which participants’ awareness was tested with alternative and more sensitive dependent measures, both of which manifest positive evidence of awareness. We also show that, based on the predictions of a formal model of probabilistic cuing and given the reliabilities of the dependent measures collected in these experiments, any statistical test aimed at detecting a significant correlation between learning and awareness is doomed to return a non-significant result, even if at the latent level both constructs are actually related and participants’ knowledge is completely explicit.
Miguel A. Vadillo
added a research item
West et al. (2018) examined the relationship between implicit learning and reading and language attainment in 7‐ to 8‐year‐old children. The implicit learning tasks had poor reliability and did not correlate with language or reading skills. These findings raise problems for the claim that Developmental Language Disorder (DLD) and Dyslexia are caused (at least in part) by a deficit in procedural learning (the Procedural Deficit Hypothesis (PDH)). This article is protected by copyright. All rights reserved.
Miguel A. Vadillo
added a project goal
The study of unconscious cognitive processes is becoming one of the most vibrant and promising areas of research in psychological science. Many of these studies adopt the view, supported by a large volume of research, that elementary learning and memory processes demand few cognitive resources and can take place in the absence of awareness. This includes popular effects like sequence learning, artificial grammar learning, or visual statistical learning. The most common strategy to show that these effects are unconscious relies on a simple dissociation: After repeated exposure to a set of materials, participants become faster or more accurate at responding to these stimuli, revealing successful learning. However, when asked whether they recognise having been exposed to these stimuli in the past, they say no or they seem to be unable to discriminate between the stimuli they have seen and completely new stimuli. Furthermore, their score in this awareness test seems to be uncorrelated with performance in the learning task. In all cases, the lack of awareness is inferred from a null statistical result. In previous studies, we have claimed that this strategy is methodologically unsound because null results have an ambiguous interpretation in traditional, null hypothesis significance testing: They can mean that the null hypothesis is true or simply that the data are not sensitive enough to discriminate between the null and the alternative hypothesis. In the present project, we will highlight an additional problem in the interpretation of these results: The limited reliability of the dependent measures used to quantify learning and awareness. Our preliminary results suggest that, at least in the case of contextual cueing –an increasingly popular implicit learning paradigm– these reliabilities might be in the order of .10 to .20. The main goal of the present proposal is to explore and ameliorate the impact of low reliability in implicit cognition research. We will conduct several large-scale studies to estimate the reliability of the measures collected in contextual cueing and other implicit learning paradigms (probabilistic cueing of visual attention and multiple-cue probability learning). This information will allow us to use cutting-edge meta-analytic methods designed to correct for the attenuating effect of low reliability. The results of these meta-analyses can change dramatically our understanding of implicit learning, revealing, for instance, that the contribution of awareness to these effects is much larger than previously thought. If these reliabilities turn out to be as low as we expect, then it also follows that much of the applied research showing large and significant correlations between implicit learning processes and personal characteristics, like specific language impairment or dyslexia, must be biased, possibly due to selective publication or selective reporting. Our research will explore this possibility by using new methods for the detection and correction of these biases. Finally, the present proposal will devise new analytic methods, based on Bayesian hypothesis testing and computational modeling, to overcome the limitations of previous research on implicit learning.