Content uploaded by Fergus I M Craik
Author content
All content in this area was uploaded by Fergus I M Craik on May 12, 2015
Content may be subject to copyright.
A preview of the PDF is not available
Publication Bias and the Validity of Evidence: What's the Connection?
... Studies have identified numerous biases in psychology research indicative of questionable research practices including 'data dredging' or 'p-hacking' (Head, Holman, Lanfear, Kahn, & Jennions, 2015), hypothesizing after results are known ('HARK-ing'; Kerr, 1998;Rubin, 2017;Szucs, 2016), and selective reporting (Simmons et al., 2011). Studies have also identified dissemination practices that may introduce bias such as journal reviewers and editors favoring positive, statistically significant findings (Bialystok, Kroll, Green, MacWhinney, & Craik, 2015;Sterling, Rosenbaum, & Weinkam, 1995), and research incentive systems that equate positive findings with success and career progression (Heise & Pearce, 2020;Higginson & Munafò, 2016;Smaldino, 2016). Many of these questionable practices were identified many years previously, but received limited widespread attention (e.g. ...
Background
Identification of widespread biases present in reported research findings in many scientific disciplines, including psychology, such as failures to replicate and the likely extensive application of questionable research practices, has raised serious concerns over the reliability and trustworthiness of scientific research. This has led to the development of, and advocacy for, ‘open science’ practices, including data, materials, analysis, and output sharing, pre-registration of study predictions and analysis plans, and increased access to published research findings. Implementation of such practices has been enthusiastic in some quarters, but literacy in, and adoption of, these practices has lagged behind among many researchers in the scientific community.
Advances
In the current article I propose that researchers adopt an open science ‘mindset’, a comprehensive approach to open science predicated on researchers’ operating under the basic assumption that, wherever possible, open science practices will be a central component of all steps of their research projects. The primary, defining feature of the mindset is a commitment to open science principles in all research projects from inception to dissemination. Other features of the mindset include the assumption that all components of research projects (e.g. pre-registered hypotheses, protocols, materials, analysis plans, data, and output) will be accessible broadly; pro-active selection of open fora to disseminate research components and findings; open and transparent dissemination of reports of the research findings in advance of, and after, formal publication; and active promotion of open science practices through education, modeling, and advocacy.
Conclusion
The open science mindset is a ‘farm to fork’ approach to open science aimed at promoting comprehensive quality in application of open science, and widening participation in open science practices so that they become the norm in research in health psychology and behavioral medicine going forward.
... More recently, specific aspects of the field, namely the putative cognitive and neural consequences (often framed in the form of advantages) of bilingualism, have become a hotspot for controversy tied to the replication crisis in psychology. The critique of this research appears to be broad, addressing issues of power and sample size (e.g., Brysbaert, 2020;Nichols, Wild, Stojanoski, Battista, & Owen, 2020), failures to replicate (e.g., Paap & Greenberg, 2013), noise in samples and methods (e.g., García-Pentón, Fernández García, Costello, Duñabeitia, & Carreiras, 2016a, 2016bValian, 2015), and publication bias (e.g., de Bruin, Treccani, & Della Sala, 2015a; but see Bialystok, Kroll, Green, MacWhinney, & Craik, 2015), suggesting that the effects of bilingualism on cognitive and brain functioning are the result of questionable research practices. Consequently, several prescribed remedies, such as large samples (Brysbaert, 2020) and uniform 1 experimental procedures (García-Pentón et al., 2016a, 2016b, have been marketed as solutions (see also Szucs & Ioannidis, 2020 for an example involving neuroscience more generally). ...
An important aim of research on bilingualism is to understand how the brain adapts to the demands of using more than one language. In this paper, we argue that pursuing such an aim entails valuing our research as a discovery process that acts on variety. Prescriptions about sample size and methodology, rightly aimed at establishing a sound basis for generalization, should be understood as being in the service of science as a discovery process. We propose and illustrate by drawing from previous and contemporary examples within brain and cognitive sciences, that this necessitates exploring the neural bases of bilingual phenotypes: the adaptive variety induced through the interplay of biology and culture. We identify the conceptual and methodological prerequisites for such exploration and briefly allude to the publication practices that afford it as a community practice and to the risk of allowing methodological prescriptions, rather than discovery, to dominate the research endeavor. "We have to remember that what we observe is not nature in itself, but nature exposed to our method of questioning." Werner Heisenberg (1958)
... The comparison of bilinguals and monolinguals is not exhausted within the executive function literature, see for exampleHartsuiker et al. (2004) for a comparison in terms of lexical and syntactic information andBialystok et al. (2015) for an investigation of how bilingualism affects particular aspects of the languages used. ...
This paper introduces a novel approach to evaluate performance in the executive functioning skills of bilingual and monolingual children. This approach targets method- and analysis-specific issues in the field, which has reached an impasse (Antoniou et al., 2021). This study moves beyond the traditional approach towards bilingualism by using an array of executive functioning tasks and frontier methodologies, which allow us to jointly consider multiple tasks and metrics in a new measure; technical efficiency (TE) . We use a data envelopment analysis technique to estimate TE for a sample of 32 Greek–English bilingual and 38 Greek monolingual children. In a second stage, we compare the TE of the groups using an ANCOVA, a bootstrap regression, and a k-means nearest-neighbour technique, while controlling for a range of background variables. Results show that bilinguals have superior TE compared to their monolingual counterparts, being around 6.5% more efficient. Robustness tests reveal that TE yields similar results to the more complex conventional MANCOVA analyses, while utilising information in a more efficient way. By using the TE approach on a relevant existing dataset, we further highlight TE’s advantages compared to conventional analyses; not only does TE use a single measure, instead of two principal components, but it also allows more group observations as it accounts for differences between the groups by construction.
The term “Bilingual Advantage” had come to represent a general executive functioning advantage observed for bilinguals. While findings originally looked promising, two key publications released in 2015 drew attention to serious methodological problems in the research body and presented evidence that implied that the literature was affected by a publication and confirmation bias, which favoured significant findings in support of a bilingual advantage.KeywordsBilingualismExecutive functioningAttentionReplication crisis
We compared Spanish (L1)-English (L2) bilinguals and Spanish monolinguals in a semantic judgment relationship task in L1 that produced within-language conflict due to the coactivation of the two meanings of a Spanish homophone (e.g., "hola" and "ola" meaning "hello" and "a wave" in English). In this task, participants indicated if pairs of words were related or not ("agua-hola," "water-hello"). Conflict arose because a word ("agua," "water") not related to the orthographic form of a homophone ("hola," "hello") was related to the alternative orthographic form ("ola," "wave"). Compared to a control condition with unrelated word pairs ("peluche-hola," "teddy-hello"), the behavioral results revealed greater behavioral interference in monolinguals compared to bilinguals. In addition, electrophysiological results revealed N400 differences between monolinguals and bilinguals. These results are discussed around the impact of bilingualism on conflict resolution.
Full text available here: https://theses.lib.polyu.edu.hk/handle/200/12383
Bilingualism has been attracting interest from the cognitive science field for years as it is suggested to be a protective factor against cognitive decline in ageing. It is often reported that bilinguals performed better than monolinguals in inhibitory control tasks. The mechanism behind the better inhibitory control was that bilinguals would have to suppress the interference from the unwanted language all the time, and such linguistic control is thought to be, at least partially, overlapped with the general inhibitory control network. However, inconsistent results have been reported. It is common for the literature to compare monolinguals with bilinguals as two homogenous groups without considering the individual variations between and among them. Moreover, as the Adaptive Control Hypothesis (Green & Abutalebi, 2013) suggested, the interaction context affects the cognitive demand in controlling the languages. Three experiments were designed to explore how different aspects of bilingualism contribute to cognition and the bilingual advantage effect.
The first experiment recruited older adults to complete a comprehensive set of cognitive tests together with questionnaires on their language and demographic profiles. Comparing the monolinguals and bilinguals, we found the classic bilingual advantage effect: bilinguals scored higher in the Montreal Cognitive Assessment (MoCA), indicating better cognitive status. Moreover, within the bilinguals, the scores in the cognitive battery were predicted with demographic and linguistic variables using linear regression analysis. We found that L2 proficiency predicts better inhibitory control and verbal ability performance in lifelong bilinguals. We propose that, because our participants are L1-dominant speakers, only the sufficiently proficient L2 would provide enough interference in the practice of linguistic inhibition control.
The second experiment investigated the cognitive changes in older foreign language learners. Older adults were recruited to study in an elementary English course for six weeks, with cognitive tests taken before and after the course. Although the statistical results between the intervention group and the active and passive control groups were not significant, the language learning-induced differences were observed in some tasks, including the accuracy of Picture Naming and the Conflicting Effect in the Attention Network Task. Correlation analysis suggested that successful language learners showed an improvement in inhibitory control and a decline in verbal fluency.
The third experiment investigated the organisation of the mental lexicon through an interesting language phenomenon in Hong Kong: dense code-switching. Whereas the literature often suggested that the comprehension of code-switching requires a switch in lexicon and is therefore challenging, we found that switching lexicon was needed only in the case of non-habitual word usage, regardless of whether it was unilingual and code-switching. From the result of this experiment, we proposed that the language input from the community had formed the bilingual prefabs, which integrated into the dominantly Cantonese lexicon.
Collectively, we suggest that the environment, language and cognition form a looping circle in that each component is interrelated. Moreover, they each affect the organisation of the bilingual mental lexicon and the retrieval of concepts from the lexicon. In view of that, we propose the Experience-based Bilingual Mental Lexicon Model, which is modified based on the Revised Hierarchical Model (Kroll & Stewart, 1994). Two critical assumptions are incorporated into the existing model: (1) the language lexicon is organised by experience but not by language origin, and (2) language dominance is dynamic. We believe the proposed model could better capture the dynamic change of language by experience. It could explain how individual differences contribute to the bilingual advantage effect.
References:
Green, D. W., & Abutalebi, J. (2013). Language control in bilinguals: The adaptive control hypothesis. Journal of Cognitive Psychology, 25(5), 515-530. https://doi.org/10.1080/20445911.2013.796377
Kroll, J. F., & Stewart, E. (1994). Category interference in translation and picture naming: Evidence for asymmetric connections between bilingual memory representations. Journal of Memory and Language, 33(2), 149-174. https://doi.org/10.1006/jmla.1994.1008
A key question in studies of cognitive development is whether bilingual environments impact higher-cognitive functions. Inconclusive evidence in search of a “bilingual cognitive advantage” has sparked debates on the reliability of these findings. Few studies with infants have examined this question, but most of them include small samples. The current study presents evidence from a large sample of 6- and 10-month-old monolingual- and bilingual-exposed infants (N = 152), which includes a longitudinal subset (n = 31), who completed a cueing attentional orienting task. The results suggest bilingual infants showed significant developmental gains in latency performance during the condition that was most cognitively demanding (Incongruent). The results also revealed bilingual infants’ performance was associated with their parents’ dual-language switching behavior. Taken together, these results provide support that bilingual experiences (i.e., dual-language mixing) influence infants’ shifting and orienting of attention.
While there is evidence for bilingual enhancements of inhibitory control and auditory processing, two processes that are fundamental to daily communication, it is not known how bilinguals utilize these cognitive and sensory enhancements during real-world listening. To test our hypothesis that bilinguals engage their enhanced cognitive and sensory processing in real-world listening situations, bilinguals and monolinguals performed a selective attention task involving competing talkers, a common demand of everyday listening, and then later passively listened to the same competing sentences. During the active and passive listening periods, evoked responses to the competing talkers were collected to understand how online auditory processing facilitates active listening and if this processing differs between bilinguals and monolinguals. Additionally, participants were tested on a separate measure of inhibitory control to see if inhibitory control abilities related with performance on the selective attention task. We found that although monolinguals and bilinguals performed similarly on the selective attention task, the groups differed in the neural and cognitive processes engaged to perform this task, compared to when they were passively listening to the talkers. Specifically, during active listening monolinguals had enhanced cortical phase consistency while bilinguals demonstrated enhanced subcortical phase consistency in the response to the pitch contours of the sentences, particularly during passive listening. Moreover, bilinguals’ performance on the inhibitory control test related with performance on the selective attention test, a relationship that was not seen for monolinguals. These results are consistent with the hypothesis that bilinguals utilize inhibitory control and enhanced subcortical auditory processing in everyday listening situations to engage with sound in ways that are different than monolinguals.
Background: For many years, research has focused on the cognitive consequences of bilingualism. While earlier studies reported that bilinguals were more efficient in executive control, particularly with respect to inhibitory processes (bilingual advantage), more recent studies have often failed to replicate this effect. Moreover, studies have shown the
unity and diversity of inhibitory control and distinguished between response inhibition
and interference suppression.
Aim: This literature review aims to elucidate whether the bilingual inhibitory control
advantage, especially in its two components, exists across the life span and to investigate
its modulating factors.
Method: A literature search was conducted via EBSCO in many databases and reference
lists for all original data on bilingualism and inhibitory control, with a cut-off date of
April 30, 2020. Following the preferred reporting items for systematic reviews and meta-analysis (PRISMA) protocols, 21 original studies were eligible to be included in this review.
Results: The review yielded little evidence of bilingual inhibitory control advantage with different patterns of results for response inhibition and interference suppression. The heterogeneous outcomes might be related to individual differences, such as age and methodological issues, such as the use of different tasks.
Conclusion: This literature review found heterogeneous results regarding the bilingual advantage in response inhibition and interference suppression. It stresses the importance of accounting for possible modulating factors when investigating the relationship between bilingualism and inhibition. If significant progress is to be made, accounting for
confounding factors and reevaluating the inhibitory control measurement is required.
Keywords: bilingual advantage, inhibitory control, interference suppression, response inhibition
Speech production in multilinguals involves constant inhibition of the languages currently not in use. In relation to phonological development, higher inhibitory skills may lead to the improved suppression of interference from the remaining languages in one’s repertoire and more accurate production of target features. The participants were 20 sequential multilingual learners (13-year-olds with L1 Polish, L2 English, L3 German), acquiring their L2 and L3 by formal instruction in a primary school. Inhibition was measured in a modified flanker task (Eriksen & Eriksen 1974; Poarch & Bialystok 2015). Multilingual production of voice onset time (VOT) and rhotic consonants was tested in a delayed repetition task (e.g. Kopečková et al. 2016; Krzysik 2019) in their L2 and L3. The results revealed that higher inhibitory control was related to increased global accuracy in the L2 and L3 production. Moreover, higher inhibitory control was also linked to higher accuracy in the overall L2 production, but there was no significant relationship with the L3 accuracy. These findings suggest that inhibition may play a role in phonological speech production, however, it may depend on one’s level of proficiency.
It is a widely held belief that bilinguals have an advantage over monolinguals in executive-control tasks, but is this what all studies actually demonstrate? The idea of a bilingual advantage may result from a publication bias favoring studies with positive results over studies with null or negative effects. To test this hypothesis, we looked at conference abstracts from 1999 to 2012 on the topic of bilingualism and executive control. We then determined which of the studies they reported were subsequently published. Studies with results fully supporting the bilingual-advantage theory were most likely to be published, followed by studies with mixed results. Studies challenging the bilingual advantage were published the least. This discrepancy was not due to differences in sample size, tests used, or statistical power. A test for funnel-plot asymmetry provided further evidence for the existence of a publication bias.
© The Author(s) 2014.
Normal aging is an inevitable race between increasing knowledge and decreasing cognitive capacity. Crucial to understanding and promoting successful aging is determining which of these factors dominates for particular neurocognitive functions. Here, we focus on the human capacity for language, for which healthy older adults are simultaneously advantaged and disadvantaged. In recent years, a more hopeful view of cognitive aging has emerged from work suggesting that age-related declines in executive control functions are buffered by life-long bilingualism. In this paper, we selectively review what is currently known and unknown about bilingualism, executive control, and aging. Our ultimate goal is to advance the views that these issues should be reframed as a specific instance of neuroplasticity more generally and, in particular, that researchers should embrace the individual variability among bilinguals by adopting experimental and statistical approaches that respect the complexity of the questions addressed. In what follows, we set out the theoretical assumptions and empirical support of the bilingual advantages perspective, review what we know about language, cognitive control, and aging generally, and then highlight several of the relatively few studies that have investigated bilingual language processing in older adults, either on their own or in comparison with monolingual older adults. We conclude with several recommendations for how the field ought to proceed to achieve a more multifactorial view of bilingualism that emphasizes the notion of neuroplasticity over that of simple bilingual versus monolingual group comparisons.
Speech comprehension and production are governed by control processes. We explore their nature and
dynamics in bilingual speakers with a focus on speech production. Prior research indicates that
individuals increase cognitive control in order to achieve a desired goal. In the adaptive control
hypothesis we propose a stronger hypothesis: Language control processes themselves adapt to the
recurrent demands placed on them by the interactional context. Adapting a control process means
changing a parameter or parameters about the way it works (its neural capacity or efficiency) or the way
it works in concert, or in cascade, with other control processes (e.g., its connectedness). We distinguish
eight control processes (goal maintenance, conflict monitoring, interference suppression, salient cue
detection, selective response inhibition, task disengagement, task engagement, opportunistic planning).
We consider the demands on these processes imposed by three interactional contexts (single language,
dual language, and dense code-switching). We predict adaptive changes in the neural regions and circuits
associated with specific control processes. A dual-language context, for example, is predicted to lead to
the adaptation of a circuit mediating a cascade of control processes that circumvents a control dilemma.
Effective test of the adaptive control hypothesis requires behavioural and neuroimaging work that
assesses language control in a range of tasks within the same individual.
Three studies compared bilinguals to monolinguals on 15 indicators of executive processing (EP). Most of the indicators compare a neutral or congruent baseline to a condition that should require EP. For each of the measures there was no main effect of group and a highly significant main effect of condition. The critical marker for a bilingual advantage, the Group×Condition interaction, was significant for only one indicator, but in a pattern indicative of a bilingual disadvantage. Tasks include antisaccade (Study 1), Simon (Studies 1-3), flanker (Study 3), and color-shape switching (Studies 1-3). The two groups performed identically on the Raven's Advanced Matrices test (Study 3). Analyses on the combined data selecting subsets that are precisely matched on parent's educational level or that include only highly fluent bilinguals reveal exactly the same pattern of results. A problem reconfirmed by the present study is that effects assumed to be indicators of a specific executive process in one task (e.g., inhibitory control in the flanker task) frequently do not predict individual differences in that same indicator on a related task (e.g., inhibitory control in the Simon task). The absence of consistent cross-task correlations undermines the interpretation that these are valid indicators of domain-general abilities. In a final discussion the underlying rationale for hypothesizing bilingual advantages in executive processing based on the special linguistic demands placed on bilinguals is interrogated.
The regular use of two languages by bilingual individuals has been shown to have a broad impact on language and cognitive functioning. In this monograph, we consider four aspects of this influence. In the first section, we examine differences between mono-linguals and bilinguals in children's acquisition of language and adults' linguistic processing, particularly in terms of lexical retrieval. Children learning two languages from birth follow the same milestones for language acquisition as mono-linguals do (first words, first use of grammar) but may use different strategies for language acquisition, and they generally have a smaller vocabulary in each language than do monolin-gual children learning only a single language. Adult bilinguals typically take longer to retrieve individual words than monolin-guals do, and they generate fewer words when asked to satisfy a constraint such as category membership or initial letter. In the second section, we consider the impact of bilingualism on nonverbal cognitive processing in both children and adults. The primary effect in this case is the enhancement of executive control functions in bilinguals. On tasks that require inhibition of distract-ing information, switching between tasks, or holding information in mind while performing a task, bilinguals of all ages outperform comparable monolinguals. A plausible reason is that bilinguals recruit control processes to manage their ongoing linguistic per-formance and that these control processes become enhanced for other unrelated aspects of cognitive processing. Preliminary evi-dence also suggests that the executive control advantage may even mitigate cognitive decline in older age and contribute to cognitive reserve, which in turn may postpone Alzheimer's disease. In the third section, we describe the brain networks that are responsible for language processing in bilinguals and demon-strate their involvement in nonverbal executive control for bilinguals. We begin by reviewing neuroimaging research that identifies the networks used for various nonverbal executive control tasks in the literature. These networks are used as a ref-erence point to interpret the way in which bilinguals perform both verbal and nonverbal control tasks. The results show that bilinguals manage attention to their two language systems using the same networks that are used by monolinguals performing nonverbal tasks. In the fourth section, we discuss the special circumstances that surround the referral of bilingual children (e.g., language delays) and adults (e.g., stroke) for clinical intervention. These referrals are typically based on standardized assessments that use normative data from monolingual populations, such as vocabulary size and lexical retrieval. As we have seen, however, these measures are often different for bilinguals, both for children and adults. We discuss the implications of these linguistic differences for standardized test performance and clinical approaches. We conclude by considering some questions that have important public policy implications. What are the pros and cons of French or Spanish immersion educational programs, for example? Also, if bilingualism confers advantages in certain respects, how about three languages—do the benefits increase? In the healthcare field, how can current knowledge help in the treatment of bilingual aphasia patients following stroke? Given the recent increase in bilingualism as a research topic, answers to these and other related questions should be available in the near future.
In this review we will focus on delineating the neural substrates of the executive control of language in the bilingual brain, based on the existing neuroimaging, intracranial, transcranial magnetic stimulation, and neuropsychological evidence. We will also offer insights from ongoing brain-imaging studies into the development of expertise in multilingual language control. We will concentrate specifically on evidence regarding how the brain selects and controls languages for comprehension and production. This question has been addressed in a number of ways and using various tasks, including language switching during production or perception, translation, and interpretation. We will attempt to synthesize existing evidence in order to bring to light the neural substrates that are crucial to executive control of language.
The present paper summarizes research showing that bilingualism affects linguistic and cognitive performance across the lifespan. The effect on linguistic performance is generally seen as a deficit in which bilingual children control a smaller vocabulary than their monolingual peers and bilingual adults perform more poorly on rapid lexical retrieval tasks. The effect on cognitive performance is to enhance executive functioning and to protect against the decline of executive control in aging. These effects interact to produce a complex pattern regarding the effect of bilingualism on memory performance. Memory tasks based primarily on verbal recall are performed more poorly by bilinguals but memory tasks based primarily on executive control are performed better by bilinguals. Speculations regarding the mechanism responsible for these effects are described.
The "weaker links" hypothesis proposes that bilinguals are disadvantaged relative to monolinguals on speaking tasks because they divide frequency-of-use between two languages. To test this proposal we contrasted the effects of increased word use associated with monolingualism, language dominance, and increased age on picture naming times. In two experiments, younger and older bilinguals and monolinguals named pictures with high- or low-frequency names in English and (if bilingual) also in Spanish. In Experiment 1, slowing related to bilingualism and language dominance was greater for producing low- than high-frequency names. In Experiment 2, slowing related to aging was greater for producing low-frequency names in the dominant language, but when speaking the nondominant language, increased age attenuated frequency effects and age-related slowing was limited exclusively to high-frequency names. These results challenge competition based accounts of bilingual disadvantages in language production, and illustrate how between-group processing differences may emerge from cognitive mechanisms general to all speakers.
Language control in bilinguals: The adaptive control hypothesis
- D W Green
- J Abutalebi
Green, D. W., & Abutalebi, J. (2013). Language control in bilinguals: The adaptive control hypothesis. Journal of Cognitive
Psychology, 25, 515-530. doi:10.1080/20445911.2013.796377