ArticlePDF Available

Publication Bias and the Validity of Evidence: What's the Connection?

Authors:
A preview of the PDF is not available
... Some have rendered positive assessments in this domain e.g., [31,32], while others cast doubt e.g., [33][34][35]. Still others serve to underscore publication bias as a key factor underlying findings of a language-related advantage [36,37]; but see [32,38]. ...
... Critically, this debate is serving to identify oversights in earlier research. For example, researchers have identified variable operationalizations of bilingualism [39][40][41], methodological confounds of various kinds (and corresponding misinterpretations of findings) [19], publication bias that favors significant findings (and relegates null results to the file drawer) [34,37,42]; but see [32,38], and overly strong conclusions that are not, in fact, supported by the data [33]. Many recent findings reveal the importance of factors that vary across individuals-including age, socioeconomic status (assessed as various amalgams of education, earnings, and neighborhood), culture, and, in particular, second language (L2) usage profiles-in modulating the existence of the bilingual advantage e.g., [43,44]. ...
Article
Full-text available
A debate over the past decade has focused on the so-called bilingual advantage—the idea that bilingual and multilingual individuals have enhanced domain-general executive functions, relative to monolinguals, due to competition-induced monitoring of both processing and representation from the task-irrelevant language(s). In this commentary, we consider a recent study by Pot, Keijzer, and de Bot (2018), which focused on the relationship between individual differences in language usage and performance on an executive function task among multilingual older adults. We discuss their approach and findings in light of a more general movement towards embracing complexity in this domain of research, including individuals’ sociocultural context and position in the lifespan. The field increasingly considers interactions between bilingualism/multilingualism and cognition, employing measures of language use well beyond the early dichotomous perspectives on language background. Moreover, new measures of bilingualism and analytical approaches are helping researchers interrogate the complexities of specific processing issues. Indeed, our review of the bilingualism/multilingualism literature confirms the increased appreciation researchers have for the range of factors—beyond whether someone speaks one, two, or more languages—that impact specific cognitive processes. Here, we highlight some of the most salient of these, and incorporate suggestions for a way forward that likewise encompasses neural perspectives on the topic.
... Studies have identified numerous biases in psychology research indicative of questionable research practices including 'data dredging' or 'p-hacking' (Head, Holman, Lanfear, Kahn, & Jennions, 2015), hypothesizing after results are known ('HARK-ing'; Kerr, 1998;Rubin, 2017;Szucs, 2016), and selective reporting (Simmons et al., 2011). Studies have also identified dissemination practices that may introduce bias such as journal reviewers and editors favoring positive, statistically significant findings (Bialystok, Kroll, Green, MacWhinney, & Craik, 2015;Sterling, Rosenbaum, & Weinkam, 1995), and research incentive systems that equate positive findings with success and career progression (Heise & Pearce, 2020;Higginson & Munafò, 2016;Smaldino, 2016). Many of these questionable practices were identified many years previously, but received limited widespread attention (e.g. ...
Article
Full-text available
Background Identification of widespread biases present in reported research findings in many scientific disciplines, including psychology, such as failures to replicate and the likely extensive application of questionable research practices, has raised serious concerns over the reliability and trustworthiness of scientific research. This has led to the development of, and advocacy for, ‘open science’ practices, including data, materials, analysis, and output sharing, pre-registration of study predictions and analysis plans, and increased access to published research findings. Implementation of such practices has been enthusiastic in some quarters, but literacy in, and adoption of, these practices has lagged behind among many researchers in the scientific community. Advances In the current article I propose that researchers adopt an open science ‘mindset’, a comprehensive approach to open science predicated on researchers’ operating under the basic assumption that, wherever possible, open science practices will be a central component of all steps of their research projects. The primary, defining feature of the mindset is a commitment to open science principles in all research projects from inception to dissemination. Other features of the mindset include the assumption that all components of research projects (e.g. pre-registered hypotheses, protocols, materials, analysis plans, data, and output) will be accessible broadly; pro-active selection of open fora to disseminate research components and findings; open and transparent dissemination of reports of the research findings in advance of, and after, formal publication; and active promotion of open science practices through education, modeling, and advocacy. Conclusion The open science mindset is a ‘farm to fork’ approach to open science aimed at promoting comprehensive quality in application of open science, and widening participation in open science practices so that they become the norm in research in health psychology and behavioral medicine going forward.
... More recently, specific aspects of the field, namely the putative cognitive and neural consequences (often framed in the form of advantages) of bilingualism, have become a hotspot for controversy tied to the replication crisis in psychology. The critique of this research appears to be broad, addressing issues of power and sample size (e.g., Brysbaert, 2020;Nichols, Wild, Stojanoski, Battista, & Owen, 2020), failures to replicate (e.g., Paap & Greenberg, 2013), noise in samples and methods (e.g., García-Pentón, Fernández García, Costello, Duñabeitia, & Carreiras, 2016a, 2016bValian, 2015), and publication bias (e.g., de Bruin, Treccani, & Della Sala, 2015a; but see Bialystok, Kroll, Green, MacWhinney, & Craik, 2015), suggesting that the effects of bilingualism on cognitive and brain functioning are the result of questionable research practices. Consequently, several prescribed remedies, such as large samples (Brysbaert, 2020) and uniform 1 experimental procedures (García-Pentón et al., 2016a, 2016b, have been marketed as solutions (see also Szucs & Ioannidis, 2020 for an example involving neuroscience more generally). ...
Article
Full-text available
An important aim of research on bilingualism is to understand how the brain adapts to the demands of using more than one language. In this paper, we argue that pursuing such an aim entails valuing our research as a discovery process that acts on variety. Prescriptions about sample size and methodology, rightly aimed at establishing a sound basis for generalization, should be understood as being in the service of science as a discovery process. We propose and illustrate by drawing from previous and contemporary examples within brain and cognitive sciences, that this necessitates exploring the neural bases of bilingual phenotypes: the adaptive variety induced through the interplay of biology and culture. We identify the conceptual and methodological prerequisites for such exploration and briefly allude to the publication practices that afford it as a community practice and to the risk of allowing methodological prescriptions, rather than discovery, to dominate the research endeavor. "We have to remember that what we observe is not nature in itself, but nature exposed to our method of questioning." Werner Heisenberg (1958)
... The comparison of bilinguals and monolinguals is not exhausted within the executive function literature, see for exampleHartsuiker et al. (2004) for a comparison in terms of lexical and syntactic information andBialystok et al. (2015) for an investigation of how bilingualism affects particular aspects of the languages used. ...
Article
Full-text available
This paper introduces a novel approach to evaluate performance in the executive functioning skills of bilingual and monolingual children. This approach targets method- and analysis-specific issues in the field, which has reached an impasse (Antoniou et al., 2021). This study moves beyond the traditional approach towards bilingualism by using an array of executive functioning tasks and frontier methodologies, which allow us to jointly consider multiple tasks and metrics in a new measure; technical efficiency (TE) . We use a data envelopment analysis technique to estimate TE for a sample of 32 Greek–English bilingual and 38 Greek monolingual children. In a second stage, we compare the TE of the groups using an ANCOVA, a bootstrap regression, and a k-means nearest-neighbour technique, while controlling for a range of background variables. Results show that bilinguals have superior TE compared to their monolingual counterparts, being around 6.5% more efficient. Robustness tests reveal that TE yields similar results to the more complex conventional MANCOVA analyses, while utilising information in a more efficient way. By using the TE approach on a relevant existing dataset, we further highlight TE’s advantages compared to conventional analyses; not only does TE use a single measure, instead of two principal components, but it also allows more group observations as it accounts for differences between the groups by construction.
Article
Full-text available
A key question in studies of cognitive development is whether bilingual environments impact higher-cognitive functions. Inconclusive evidence in search of a “bilingual cognitive advantage” has sparked debates on the reliability of these findings. Few studies with infants have examined this question, but most of them include small samples. The current study presents evidence from a large sample of 6- and 10-month-old monolingual- and bilingual-exposed infants (N = 152), which includes a longitudinal subset (n = 31), who completed a cueing attentional orienting task. The results suggest bilingual infants showed significant developmental gains in latency performance during the condition that was most cognitively demanding (Incongruent). The results also revealed bilingual infants’ performance was associated with their parents’ dual-language switching behavior. Taken together, these results provide support that bilingual experiences (i.e., dual-language mixing) influence infants’ shifting and orienting of attention.
Article
Full-text available
While there is evidence for bilingual enhancements of inhibitory control and auditory processing, two processes that are fundamental to daily communication, it is not known how bilinguals utilize these cognitive and sensory enhancements during real-world listening. To test our hypothesis that bilinguals engage their enhanced cognitive and sensory processing in real-world listening situations, bilinguals and monolinguals performed a selective attention task involving competing talkers, a common demand of everyday listening, and then later passively listened to the same competing sentences. During the active and passive listening periods, evoked responses to the competing talkers were collected to understand how online auditory processing facilitates active listening and if this processing differs between bilinguals and monolinguals. Additionally, participants were tested on a separate measure of inhibitory control to see if inhibitory control abilities related with performance on the selective attention task. We found that although monolinguals and bilinguals performed similarly on the selective attention task, the groups differed in the neural and cognitive processes engaged to perform this task, compared to when they were passively listening to the talkers. Specifically, during active listening monolinguals had enhanced cortical phase consistency while bilinguals demonstrated enhanced subcortical phase consistency in the response to the pitch contours of the sentences, particularly during passive listening. Moreover, bilinguals’ performance on the inhibitory control test related with performance on the selective attention test, a relationship that was not seen for monolinguals. These results are consistent with the hypothesis that bilinguals utilize inhibitory control and enhanced subcortical auditory processing in everyday listening situations to engage with sound in ways that are different than monolinguals.
Thesis
Background: For many years, research has focused on the cognitive consequences of bilingualism. While earlier studies reported that bilinguals were more efficient in executive control, particularly with respect to inhibitory processes (bilingual advantage), more recent studies have often failed to replicate this effect. Moreover, studies have shown the unity and diversity of inhibitory control and distinguished between response inhibition and interference suppression. Aim: This literature review aims to elucidate whether the bilingual inhibitory control advantage, especially in its two components, exists across the life span and to investigate its modulating factors. Method: A literature search was conducted via EBSCO in many databases and reference lists for all original data on bilingualism and inhibitory control, with a cut-off date of April 30, 2020. Following the preferred reporting items for systematic reviews and meta-analysis (PRISMA) protocols, 21 original studies were eligible to be included in this review. Results: The review yielded little evidence of bilingual inhibitory control advantage with different patterns of results for response inhibition and interference suppression. The heterogeneous outcomes might be related to individual differences, such as age and methodological issues, such as the use of different tasks. Conclusion: This literature review found heterogeneous results regarding the bilingual advantage in response inhibition and interference suppression. It stresses the importance of accounting for possible modulating factors when investigating the relationship between bilingualism and inhibition. If significant progress is to be made, accounting for confounding factors and reevaluating the inhibitory control measurement is required. Keywords: bilingual advantage, inhibitory control, interference suppression, response inhibition
Article
Full-text available
Speech production in multilinguals involves constant inhibition of the languages currently not in use. In relation to phonological development, higher inhibitory skills may lead to the improved suppression of interference from the remaining languages in one’s repertoire and more accurate production of target features. The participants were 20 sequential multilingual learners (13-year-olds with L1 Polish, L2 English, L3 German), acquiring their L2 and L3 by formal instruction in a primary school. Inhibition was measured in a modified flanker task (Eriksen & Eriksen 1974; Poarch & Bialystok 2015). Multilingual production of voice onset time (VOT) and rhotic consonants was tested in a delayed repetition task (e.g. Kopečková et al. 2016; Krzysik 2019) in their L2 and L3. The results revealed that higher inhibitory control was related to increased global accuracy in the L2 and L3 production. Moreover, higher inhibitory control was also linked to higher accuracy in the overall L2 production, but there was no significant relationship with the L3 accuracy. These findings suggest that inhibition may play a role in phonological speech production, however, it may depend on one’s level of proficiency.
Article
An increasing body of research has investigated how bilingual language experience changes brain structure and function, including changes to task-free, or “resting-state” brain connectivity. Such findings provide important evidence about how the brain continues to be shaped by different language experiences throughout the lifespan. The neural effects of bilingual language experience can provide evidence about the additional processing demands placed on the linguistic and/or executive systems by dual-language use. While considerable research has used MRI to examine where these changes occur, such methods cannot reveal the temporal dynamics of functioning brain networks at rest. The current study used data from task-free electroencephalograms (EEGs) to disentangle how the linguistic and cognitive demands of bilingual language use impact brain functioning. Data analyzed from 106 bilinguals and 96 monolinguals revealed that bilinguals had greater alpha power, and significantly greater and broader coherence in the alpha and beta frequency ranges than monolinguals. Follow up analyses showed that higher alpha was related to language control: more second-language use, higher native-language proficiency, and earlier age of second-language acquisition. Bilateral beta power was related to native-language proficiency, whereas theta was related to native-language proficiency only in left-hemisphere electrodes. The results contribute to our understanding of how the linguistic and cognitive requirements of dual-language use shape intrinsic brain activity, and what the broader implications for information processing may be.
Article
Perhaps the main advantage of being bilingual is the capacity to communicate with interlocutors that have different language backgrounds. In the life of a bilingual, switching interlocutors hence sometimes involves switching languages. We know that the capacity to switch from one language to another is supported by control mechanisms, such as task-set reconfiguration. This study investigates whether similar neurophysiological mechanisms support bilingual switching between different listeners, within and across languages. A group of 48 unbalanced Dutch-English bilinguals named pictures for two monolingual Dutch and two monolingual English life-size virtual listeners in an immersive virtual reality environment. In terms of reaction times, switching languages came at a cost over and above the significant cost of switching from one listener to another. Analysis of event-related potentials showed similar electrophysiological correlates for switching listeners and switching languages. However, it was found that having to switch listeners and languages at the same time delays the onset of lexical processes more than a switch between listeners within the same language. Findings are interpreted in light of the interplay between proactive (sustained inhibition) and reactive (task-set reconfiguration) control in bilingual speech production. It is argued that a possible bilingual advantage in executive control may not be due to the process of switching per se. This study paves the way for the study of bilingual language switching in ecologically valid, naturalistic, experimental settings.
Article
Full-text available
It is a widely held belief that bilinguals have an advantage over monolinguals in executive-control tasks, but is this what all studies actually demonstrate? The idea of a bilingual advantage may result from a publication bias favoring studies with positive results over studies with null or negative effects. To test this hypothesis, we looked at conference abstracts from 1999 to 2012 on the topic of bilingualism and executive control. We then determined which of the studies they reported were subsequently published. Studies with results fully supporting the bilingual-advantage theory were most likely to be published, followed by studies with mixed results. Studies challenging the bilingual advantage were published the least. This discrepancy was not due to differences in sample size, tests used, or statistical power. A test for funnel-plot asymmetry provided further evidence for the existence of a publication bias. © The Author(s) 2014.
Article
Full-text available
Normal aging is an inevitable race between increasing knowledge and decreasing cognitive capacity. Crucial to understanding and promoting successful aging is determining which of these factors dominates for particular neurocognitive functions. Here, we focus on the human capacity for language, for which healthy older adults are simultaneously advantaged and disadvantaged. In recent years, a more hopeful view of cognitive aging has emerged from work suggesting that age-related declines in executive control functions are buffered by life-long bilingualism. In this paper, we selectively review what is currently known and unknown about bilingualism, executive control, and aging. Our ultimate goal is to advance the views that these issues should be reframed as a specific instance of neuroplasticity more generally and, in particular, that researchers should embrace the individual variability among bilinguals by adopting experimental and statistical approaches that respect the complexity of the questions addressed. In what follows, we set out the theoretical assumptions and empirical support of the bilingual advantages perspective, review what we know about language, cognitive control, and aging generally, and then highlight several of the relatively few studies that have investigated bilingual language processing in older adults, either on their own or in comparison with monolingual older adults. We conclude with several recommendations for how the field ought to proceed to achieve a more multifactorial view of bilingualism that emphasizes the notion of neuroplasticity over that of simple bilingual versus monolingual group comparisons.
Article
Full-text available
Speech comprehension and production are governed by control processes. We explore their nature and dynamics in bilingual speakers with a focus on speech production. Prior research indicates that individuals increase cognitive control in order to achieve a desired goal. In the adaptive control hypothesis we propose a stronger hypothesis: Language control processes themselves adapt to the recurrent demands placed on them by the interactional context. Adapting a control process means changing a parameter or parameters about the way it works (its neural capacity or efficiency) or the way it works in concert, or in cascade, with other control processes (e.g., its connectedness). We distinguish eight control processes (goal maintenance, conflict monitoring, interference suppression, salient cue detection, selective response inhibition, task disengagement, task engagement, opportunistic planning). We consider the demands on these processes imposed by three interactional contexts (single language, dual language, and dense code-switching). We predict adaptive changes in the neural regions and circuits associated with specific control processes. A dual-language context, for example, is predicted to lead to the adaptation of a circuit mediating a cascade of control processes that circumvents a control dilemma. Effective test of the adaptive control hypothesis requires behavioural and neuroimaging work that assesses language control in a range of tasks within the same individual.
Article
Full-text available
Three studies compared bilinguals to monolinguals on 15 indicators of executive processing (EP). Most of the indicators compare a neutral or congruent baseline to a condition that should require EP. For each of the measures there was no main effect of group and a highly significant main effect of condition. The critical marker for a bilingual advantage, the Group×Condition interaction, was significant for only one indicator, but in a pattern indicative of a bilingual disadvantage. Tasks include antisaccade (Study 1), Simon (Studies 1-3), flanker (Study 3), and color-shape switching (Studies 1-3). The two groups performed identically on the Raven's Advanced Matrices test (Study 3). Analyses on the combined data selecting subsets that are precisely matched on parent's educational level or that include only highly fluent bilinguals reveal exactly the same pattern of results. A problem reconfirmed by the present study is that effects assumed to be indicators of a specific executive process in one task (e.g., inhibitory control in the flanker task) frequently do not predict individual differences in that same indicator on a related task (e.g., inhibitory control in the Simon task). The absence of consistent cross-task correlations undermines the interpretation that these are valid indicators of domain-general abilities. In a final discussion the underlying rationale for hypothesizing bilingual advantages in executive processing based on the special linguistic demands placed on bilinguals is interrogated.
Article
Full-text available
The regular use of two languages by bilingual individuals has been shown to have a broad impact on language and cognitive functioning. In this monograph, we consider four aspects of this influence. In the first section, we examine differences between mono-linguals and bilinguals in children's acquisition of language and adults' linguistic processing, particularly in terms of lexical retrieval. Children learning two languages from birth follow the same milestones for language acquisition as mono-linguals do (first words, first use of grammar) but may use different strategies for language acquisition, and they generally have a smaller vocabulary in each language than do monolin-gual children learning only a single language. Adult bilinguals typically take longer to retrieve individual words than monolin-guals do, and they generate fewer words when asked to satisfy a constraint such as category membership or initial letter. In the second section, we consider the impact of bilingualism on nonverbal cognitive processing in both children and adults. The primary effect in this case is the enhancement of executive control functions in bilinguals. On tasks that require inhibition of distract-ing information, switching between tasks, or holding information in mind while performing a task, bilinguals of all ages outperform comparable monolinguals. A plausible reason is that bilinguals recruit control processes to manage their ongoing linguistic per-formance and that these control processes become enhanced for other unrelated aspects of cognitive processing. Preliminary evi-dence also suggests that the executive control advantage may even mitigate cognitive decline in older age and contribute to cognitive reserve, which in turn may postpone Alzheimer's disease. In the third section, we describe the brain networks that are responsible for language processing in bilinguals and demon-strate their involvement in nonverbal executive control for bilinguals. We begin by reviewing neuroimaging research that identifies the networks used for various nonverbal executive control tasks in the literature. These networks are used as a ref-erence point to interpret the way in which bilinguals perform both verbal and nonverbal control tasks. The results show that bilinguals manage attention to their two language systems using the same networks that are used by monolinguals performing nonverbal tasks. In the fourth section, we discuss the special circumstances that surround the referral of bilingual children (e.g., language delays) and adults (e.g., stroke) for clinical intervention. These referrals are typically based on standardized assessments that use normative data from monolingual populations, such as vocabulary size and lexical retrieval. As we have seen, however, these measures are often different for bilinguals, both for children and adults. We discuss the implications of these linguistic differences for standardized test performance and clinical approaches. We conclude by considering some questions that have important public policy implications. What are the pros and cons of French or Spanish immersion educational programs, for example? Also, if bilingualism confers advantages in certain respects, how about three languages—do the benefits increase? In the healthcare field, how can current knowledge help in the treatment of bilingual aphasia patients following stroke? Given the recent increase in bilingualism as a research topic, answers to these and other related questions should be available in the near future.
Article
Full-text available
In this review we will focus on delineating the neural substrates of the executive control of language in the bilingual brain, based on the existing neuroimaging, intracranial, transcranial magnetic stimulation, and neuropsychological evidence. We will also offer insights from ongoing brain-imaging studies into the development of expertise in multilingual language control. We will concentrate specifically on evidence regarding how the brain selects and controls languages for comprehension and production. This question has been addressed in a number of ways and using various tasks, including language switching during production or perception, translation, and interpretation. We will attempt to synthesize existing evidence in order to bring to light the neural substrates that are crucial to executive control of language.
Article
The present paper summarizes research showing that bilingualism affects linguistic and cognitive performance across the lifespan. The effect on linguistic performance is generally seen as a deficit in which bilingual children control a smaller vocabulary than their monolingual peers and bilingual adults perform more poorly on rapid lexical retrieval tasks. The effect on cognitive performance is to enhance executive functioning and to protect against the decline of executive control in aging. These effects interact to produce a complex pattern regarding the effect of bilingualism on memory performance. Memory tasks based primarily on verbal recall are performed more poorly by bilinguals but memory tasks based primarily on executive control are performed better by bilinguals. Speculations regarding the mechanism responsible for these effects are described.
Article
The "weaker links" hypothesis proposes that bilinguals are disadvantaged relative to monolinguals on speaking tasks because they divide frequency-of-use between two languages. To test this proposal we contrasted the effects of increased word use associated with monolingualism, language dominance, and increased age on picture naming times. In two experiments, younger and older bilinguals and monolinguals named pictures with high- or low-frequency names in English and (if bilingual) also in Spanish. In Experiment 1, slowing related to bilingualism and language dominance was greater for producing low- than high-frequency names. In Experiment 2, slowing related to aging was greater for producing low-frequency names in the dominant language, but when speaking the nondominant language, increased age attenuated frequency effects and age-related slowing was limited exclusively to high-frequency names. These results challenge competition based accounts of bilingual disadvantages in language production, and illustrate how between-group processing differences may emerge from cognitive mechanisms general to all speakers.
Language control in bilinguals: The adaptive control hypothesis
  • D W Green
  • J Abutalebi
Green, D. W., & Abutalebi, J. (2013). Language control in bilinguals: The adaptive control hypothesis. Journal of Cognitive Psychology, 25, 515-530. doi:10.1080/20445911.2013.796377