Olavo B Amaral

Federal University of Rio de Janeiro | UFRJ · Instituto de Bioquímica Médica - UFRJ

Diagnostic screening models for the interpretation of null hypothesis significance test (NHST) results have been influential in highlighting the effect of selective publication on the reproducibility of the published literature, leading to John Ioannidis' much-cited claim that most published research findings are false. These models, however, are t...

Introduction: Preprints have been increasingly used in biomedical sciences, providing the opportunity for research to be publicly assessed before journal publication. With the increase in attention over preprints during the COVID-19 pandemic, we decided to assess the content of comments left on preprint platforms. Methods: Preprints posted on bioRx...

All data should get checked, but not every article needs an expert. All data should get checked, but not every article needs an expert. Olavo Amaral Olavo Amaral

Systematic reviews and meta-analyses have been increasingly recognized for their potential value in pre-clinical research, but their multiple applications have not been extensively explored in behavioral neuroscience. In this work, we studied protein synthesis inhibition, a classic intervention used to disrupt fear learning, reconsolidation, and ex...

Objectives Previous studies about the replicability of clinical research based on the published literature have suggested that highly cited articles can be contradicted or found to have inflated effects and depending of the area under study these founds can vary. An estimate done by John Ioannidis between 1993 and 2004 was the first of its kind, us...

Objectives Previous studies about the replicability of clinical research based on the published literature have suggested that highly cited articles can be contradicted or found to have inflated effects and depending of the area under study these founds can vary. An estimate done by John Ioannidis between 1993 and 2004 was the first of its kind, us...

Introduction Previous studies about the replicability of clinical research based on the published literature have suggested that highly cited articles are often contradicted or found to have inflated effects. Nevertheless, there are no recent updates of such efforts, and this situation may have changed over time. Methods We searched the Web of Sci...

Forecasting the reproducibility of research findings is one of the key challenges of Metascience. To date, reliable above-chance predictions have mainly been achieved by pooling the subjective ratings of experts through surveys or via prediction markets. Obtaining such data is laborious, however, and unlikely to increase the theoretical understandi...

Thousands of systematic reviews are published every year. To conduct a systematic review,researchers spend hours of work extracting data from and about the individual papers that areincluded. This step often leads to the production of spreadsheets with structured data andmetadata about scientific publications, which are the basis for later steps in...

Remembering is not a static process: When retrieved, a memory can be destabilized and become prone to modifications. This phenomenon has been demonstrated in a number of brain regions, but the neuronal mechanisms that rule memory destabilization and its boundary conditions remain elusive. Using two distinct computational models that combine Hebbian...

Make science more reliable by placing the burden of replicability on the community, not on individual laboratories. Make science more reliable by placing the burden of replicability on the community, not on individual laboratories.

Diagnostic screening models for the interpretation of null hypothesis significance test (NHST) results have been influential in highlighting the effect of selective publication on the reproducibility of the published literature, leading to John Ioannidis’ much-cited claim that most published research findings are false. These models, however, are t...

Remembering is not a static process: when retrieved, a memory can be destabilized and become prone to modifications. This phenomenon has been demonstrated in a number of brain regions, but the neuronal mechanisms that rule memory destabilization and its boundary conditions remain elusive. Using two distinct computational models that combine Hebbian...

Uma significativa parcela dos achados contidos na literatura biomédica não são reprodutíveis (Baker, 2016). Nesse contexto, a Iniciativa Brasileira de Reprodutibilidade visa avaliar a replicação sistemática de 60 experimentos publicados pela ciência biomédica brasileira nos últimos 20 anos (Neves et al., 2020). Os protocolos para os experimentos re...

A credibilidade dos achados científicos vem sendo analisada nos diversos campos da ciência ao longo dos últimos anos (Forsell, et al. 2019). Entre vários projetos de replicação sistemática, a Iniciativa Brasileira de Reprodutibilidade busca quantificar quão reprodutível é a ciência biomédica nacional, a partir da replicação direta de 60 experimento...

Background Preprint usage is growing rapidly in the life sciences; however, questions remain on the relative quality of preprints when compared to published articles. An objective dimension of quality that is readily measurable is completeness of reporting, as transparency can improve the reader’s ability to independently interpret data and reprodu...

Scientists have increasingly recognised that low methodological and analytical rigour combined with publish-or-perish incentives can make the published scientific literature unreliable. As a response to this, large-scale systematic replications of the literature have emerged as a way to assess the problem empirically. The Brazilian Reproducibility...

Meta‐analytic methods are powerful resources to summarize the existing evidence concerning a given research question and are widely used in many academic fields. Meta‐analyses can also be used to study sources of heterogeneity and bias among results, which should be considered to avoid inaccuracies. Many of these sources can be related to study aut...

Scientists have increasingly recognized that low methodological and analytical rigor combined with publish-or-perish incentives can make the published scientific literature unreliable. As a response to this, large-scale systematic replications of the literature have emerged as a way to assess the problem empirically. The Brazilian Reproducibility I...

The pressure for every research article to tell a clear story often leads researchers in the life sciences to exclude experiments that 'did not work' when they write up their results. However, this practice can lead to reporting bias if the decisions about which experiments to exclude are taken after data have been collected and analyzed. Here we d...

The pressure for every research article to tell a clear story often leads researchers in the life sciences to exclude experiments that 'did not work' when they write up their results. However, this practice can lead to reporting bias if the decisions about which experiments to exclude are taken after data have been collected and analyzed. Here we d...

The pressure for every research article to tell a clear story often leads researchers in the life sciences to exclude experiments that 'did not work' when they write up their results. However, this practice can lead to reporting bias if the decisions about which experiments to exclude are taken after data have been collected and analyzed. Here we d...

Preprint usage is growing rapidly in the life sciences; however, questions remain on the relative quality of preprints when compared to published articles. An objective dimension of quality that is readily measurable is completeness of reporting, as transparency can improve the reader's ability to independently interpret data and reproduce findings...

Aversive memories are at the heart of psychiatric disorders such as phobias and post-traumatic stress disorder (PTSD). Here, we present a new behavioral approach in rats that robustly attenuates aversive memories. This method consists of 'deconditioning' animals previously trained to associate a tone with a strong footshock by replacing it with a m...

Articles describing experimental data in the life sciences are meant to tell a clear story to the reader. This means that not every experimental attempt ends up published, as failed experiments and uninformative data are typically filtered out by researchers. Freedom to exclude data from an article, however, can lead to reporting bias when exclusio...

Freezing behavior is commonly used as a measure of associative fear memory. It can be measured by a trained observer, but this task is time-consuming and subject to variation. Commercially available software packages can also be used to quantify freezing; however, they can be expensive and usually require various parameters to be adjusted by the re...

Meta-analytic methods are powerful resources to summarize the existing evidence concerning a given research question, and are widely used in many fields of biomedical science. However, meta-analyses can be vulnerable to various sources of bias, which should be considered to avoid inaccuracies. Many of these sources can be related to study authorshi...

Freezing behavior is commonly used as a measure of associative fear memory. It can be measured by a trained observer, but this task is time-consuming and subject to variation. Commercially available software packages can also be used to quantify freezing; however, they can be expensive and usually require various parameters to be adjusted by the re...

Traumatic memories are at the heart of psychiatric disorders such as PTSD. Here, we present a new behavioral approach in rats that robustly attenuates aversive memories in a long-lasting way. This method consists of "deconditioning" animals previously trained to associate a tone with a strong footshock by replacing it with a much weaker one during...

Background Preprint usage is growing rapidly in the life sciences; however, questions remain on the relative quality of preprints when compared to published articles. An objective dimension of quality that is readily measurable is completeness of reporting, as transparency can improve the reader’s ability to independently interpret data and reprodu...

Most efforts to estimate the reproducibility of published findings have focused on specific areas of research, even though science is usually assessed and funded on a regional or national basis. Here we describe a project to assess the reproducibility of findings in biomedical science published by researchers based in Brazil. The Brazilian Reproduc...

Reconsolidation is a process in which re-exposure to a reminder causes a previously acquired memory to undergo a process of destabilisation followed by subsequent restabilisation. Different molecular mechanisms have been postulated for destabilisation in the amygdala and hippocampus, including CB1 receptor activation, protein degradation and AMPA r...

Raw freezing data. Unit-level % time contextual freezing for the data presented in Figs 1 & 2 and Table 3. (XLSX)

Reconsolidation is a process in which re-exposure to a reminder causes a previously acquired memory to undergo a process of destabilisation followed by subsequent restabilisation. Different molecular mechanisms have been postulated for destabilisation in the amygdala and hippocampus, including CB1 receptor activation, protein degradation and AMPA r...

With concerns over research reproducibility on the rise, systematic replications of published science have become an important tool to estimate the replicability of findings in specific areas. Nevertheless, such initiatives are still uncommon in biomedical science, and have never been performed at a national level. The Brazilian Reproducibility Ini...

Continuous excitation can trigger cell-wide homeostatic responses in neurons, altering membrane channels, promoting morphological changes and ultimately reducing synaptic weights. However, how synaptic downscaling interacts with classical forms of Hebbian plasticity is still unclear. In this study, we investigated whether chronic optogenetic stimul...

Proposals to increase research reproducibility frequently call for focusing on effect sizes instead of p values, as well as for increasing the statistical power of experiments. However, it is unclear to what extent these two concepts are indeed taken into account in basic biomedical science. To study this in a real-case scenario, we performed a sys...

Distribution of statistical power at the article level. (A) Distribution of mean sample size within articles. Gray bars show the distribution of mean sample size for the whole sample of articles (10.8 ± 4.2 [10.1–11.4], n = 149), while lines show the distribution of the mean sample size of memory-impairing (red, 11 ± 4.2, n = 60), memory-enhancing...

Correlations between effect sizes in Cohen’s d, variation and statistical power. (A) Correlation between effect size expressed in Cohen’s d and mean sample size. Correlation of the whole sample of experiments yields r = -0.15, p = 0.007, suggesting the presence of effect size inflation and/or publication bias. Grey line indicates the effect size ne...

Effect sizes and coefficients of variation across different types of intervention. Colors indicate memory-enhancing (red), memory-impairing (blue) or non-effective (yellow) experiments, all of which are pooled together in the analysis. Line and whiskers express median and interquartile interval. (A) Distribution of effect sizes across surgical (n =...

Correlation of region of origin with effect size, power and quality scores. The region of origin of articles were defined according to the affiliation of the corresponding author (or authors) of each article. Line and whiskers express median and interquartile interval. (A) Distribution of mean effect size for significant results across regions of o...

Effect size distribution in Cohen’s d. Distribution of effect sizes, expressed as standardized mean differences calculated on the basis of pooled standard deviations (i.e. Cohen’s d). Interventions were divided into memory-impairing (1.5 ± 0.7, n = 120), memory-enhancing (1.4 ± 0.5, n = 39) or non-effective (0.4 ± 0.3, n = 177) for graphical purpos...

Freezing levels during training and test sessions for memory-impairing interventions in contextual and tone conditioning. (A) Mean freezing (%) in response to context during training and testing for control (left) and treated (right) groups (n = 25 experiments for which baseline freezing to context–i.e. freezing percentage before any footshocks wer...

Distribution of statistical power calculated for absolute effect sizes. (A) Distribution of statistical power for memory-impairing interventions: based on each experiment’s variance and sample size, power varies according to the absolute difference to be detected for α = 0.05. Dashed lines show the three effect sizes used for point estimates of pow...

Correlation of quality assessment items with % of significant results, mean effect size for significant results and power. (A) Percentage of significant results in articles with (n = 18) and without (n = 59) randomization. Line and whiskers express median and interquartile interval. Student’s t test, p = 0.13. (B) Mean normalized effect size of sig...

Correlation of impact factor with effect size, power and quality scores. Impact factors were obtained from the 2013 Journal Citation Report. (A) Correlation between mean normalized effect size and impact factor. r = -0.05, p = 0.63 (n = 98). (B) Correlation between % of significant results per article and impact factor. r = -0.08, p = 0.37 (n = 121...

Correlations of effect sizes and coefficients of variation with freezing levels. (A) Correlation between the highest mean freezing between both groups and relative effect size, normalized as the % of freezing levels in the highest group. r = -0.20, p<0.0001* (n = 410). (B) Correlation between mean freezing in the control group and normalized effect...

Distribution of mean effect sizes at the article level. (A) Distribution of mean effect sizes for articles, calculated as percentage of control group freezing. The mean effect size for each type of experiment for each article is shown in the figure–thus, each article can contribute to more than one category if it contains more than one type of expe...

Distribution of statistical power calculated for effect sizes in Cohen’s d. (A) Distribution of statistical power calculated for Cohen’s d = 0.8 (i.e. large effects). for memory-impairing (red, 0.46 ± 0.15, n = 120), memory-enhancing (blue, 0.42 ± 0.10, n = 39), non-effective (yellow, 0.44 ± 0.12, n = 177) and all (grey, 0.44 ± 0.13, n = 336) exper...

Correlations between absolute effect size, variation and statistical power. (A) Correlation between absolute effect size and mean sample size. No significant correlation is found (r = -0.05, p = 0.34), although this is largely due to the presence of two outliers. (B) Correlation between absolute effect size and coefficient of variation. Correlation...

Classification of terms describing significant results. Volunteers were asked to judge each term on the left column as representing a strong, neutral or weak effect. Each term was given a score (strong, 2; neutral, 1; weak, 0) by each respondent and the mean score for each term (right column) was calculated based on an average of all 14 researchers...

Complete dataset in Microsoft access format and database info. All extracted and calculated data are present in the .accdb file. Each spreadsheet is connected to the level above it by an automatic field code. The complete description of each variable is presented in the database info PDF file. (ZIP)

Distribution of effect sizes calculated as absolute differences in freezing. (A) Distribution of effect sizes for experiments, expressed as the absolute difference in freezing between groups. Interventions were divided into memory-impairing (-24.4 ± 10.6, n = 146), memory-enhancing (21.6 ± 8.6, n = 53) or non-effective (-1.09 ± 7.7, n = 211) for gr...

Classification of terms describing non-significant results. Volunteers were asked to judge each term on the left column as representing similar means between both groups, a trend of difference or no information on the presence of a trend (neutral). Each term was given a score (trend, 2; neutral, 1; similar, 0) by each respondent and the mean score...

The search for biomarkers has been a leading endeavor in biological psychiatry. To analyze its evolution over the years, we performed a systematic review to evaluate (a) the most studied peripheral molecular markers in major psychiatric disorders, (b) the main features of studies proposing them as biomarkers and (c) whether their patterns of variat...

Retrieval of an associative memory can lead to different phenomena. Brief reexposure sessions tend to trigger reconsolida-tion, whereas more extended ones trigger extinction. In appetitive and fear cued Pavlovian memories, an intermediate " null point " period has been observed where neither process seems to be engaged. Here we investigated whether...

Transdiagnostic aspects of peripheral biomarkers in major psychiatric disorders: a systematic review. In: 19th Annual Conference of the International Society for Bipolar Disorders, 2017, Washington DC. Bipolar disorders, 2017. v. 19. p. 70-157.

Proposals to increase research reproducibility frequently call for focusing on effect sizes instead of p values, as well as for increasing the statistical power of experiments. However, it is unclear to what extent these two concepts are indeed taken into account in basic biomedical science. To study this in a real-case scenario, we performed a sys...

The search for biomarkers has been one of the leading endeavors in biological psychiatry; nevertheless, in spite of hundreds of publications, it has yet to make an impact in clinical practice. To study how biomarker research has progressed over the years, we performed a systematic review of the literature to evaluate (a) the most studied peripheral...

The concepts of effect size and statistical power are often disregarded in basic neuroscience, and most articles in the field draw their conclusions solely based on the arbitrary significance thresholds of statistical inference tests. Moreover, studies are often underpowered, making conclusions from significance tests less reliable. With this in mi...

In order to keep their activity constant, neurons are capable of altering their synaptic properties according to their overall degree of excitation and/or firing rate, thus regulating their excitability. This form of homeostatic plasticity, known as synaptic scaling, is thought to be necessary to maintain neuronal network stability. Additionally, i...

Although different hypotheses have been formulated to explain schizophrenia pathogenesis, the links between them are weak. The observation that five psychotic patients on chronic warfarin therapy for deep-vein thrombosis showed long-term remission of psychotic symptoms made us suspect that abnormalities in the coagulation pathway, specifically low...

Memory extinction involves the formation of a new associative memory that inhibits a previously conditioned association. Nonetheless, it could also depend on weakening of the original memory trace if extinction is assumed to have multiple components. The phosphatase calcineurin (CaN) has been described as being involved in extinction but not in the...

The search for biological causes of mental disorders has up to now met with limited success, leading to growing dissatisfaction with diagnostic classifications. However, it is questionable whether most clinical syndromes should be expected to correspond to specific microscale brain alterations, as multiple low-level causes could lead to similar sym...

Reconsolidation and extinction are two processes occurring upon memory retrieval that have received great attention in memory research over the last decade, partly due to their purported potential in the treatment of anxiety disorders. Due to their opposite behavioral effects, the two phenomena have usually been considered as separate entities, wit...

Established fear-related memories can undergo phenomena such as extinction or reconsolidation when recalled. Extinction probably involves the creation of a new, competing memory trace that decreases fear expression, whereas reconsolidation can mediate memory maintenance, updating, or strengthening. The factors determining whether retrieval will ini...

When 5-lipoxygenase (5-LO) is inhibited, roughly half of the CNS effect of the prototypic endocannabinoid anandamide (AEA) is lost. Therefore, we decided to investigate whether inhibiting this enzyme would influence physiological functions classically described as being under control of the endocannabinoid system. Although 5-LO inhibition by MK-886...

Alzheimer's disease (AD) and type 2 diabetes appear to share similar pathogenic mechanisms. dsRNA-dependent protein kinase (PKR) underlies peripheral insulin resistance in metabolic disorders. PKR phosphorylates eukaryotic translation initiation factor 2α (eIF2α-P), and AD brains exhibit elevated phospho-PKR and eIF2α-P levels. Whether and how PKR...