ArticlePDF Available

A Summary Evaluation of the Top-Five Brazilian Psychology Journals by Native English-Language Scholars

Article

A Summary Evaluation of the Top-Five Brazilian Psychology Journals by Native English-Language Scholars

Abstract and Figures

In the current century, English is the language for the research and dissemination of scientific findings. But for many scholars, English is a foreign language. This is especially true among the emerging and developing nations (EDNs), such as the BRICS nations, encompassing Brazil, Russia, India, China, and South Africa. The present study conducted a survey examining the translational integrity and overall impression of translated summary materials (abstracts and titles) from the five highest ranking (SCImago Journal Rank) Brazilian journals in the field of psychology. Analysis proceeded with two models. In the first model, translated summary materials from 12 randomly-selected articles from four of the five journals were evaluated by a panel of three native English-language scholars. Findings indicated an inverse relationship between the overall impression of the materials and their: abstract errors, r(34) = -0.61, p < .001; and total errors, r(34) = -0.62, p < .001; suggesting a direct relationship between the translational integrity of these EDN materials and the overall impression they leave with native English-language scholars. A second model added 3 additional articles from the fifth journal (English-language only) to the materials described. The findings from this second model suggested that for EDN journals, an investment in language resources may substantially improve the impression they leave with native English-language scholars, and thus promote wider dissemination of their findings. Keywords: English, translation, Brazil, lost science, lingua franca
Content may be subject to copyright.
Psychology
/Psicologia Reexão e Crítica, 28(S), 99-111. – DOI: http://dx.doi.org/10.1590/1678-7153.20152840014
99
ISSN 1678-7153
A Summary Evaluation of the Top-Five Brazilian Psychology
Journals by Native English-Language Scholars1
Uma Avaliação do Sumário das Top-Cinco Revistas Brasileiras de Psicologia por
Professores de Língua Nativa Inglesa
Chris Fradkin*
Centro Universitário La Salle, Canoas, RS, Brazil
Abstract
In the current century, English is the language for the research and dissemination of scientic ndings.
But for many scholars, English is a foreign language. This is especially true among the emerging
and developing nations (EDNs), such as the BRICS nations, encompassing Brazil, Russia, India,
China, and South Africa. The present study conducted a survey examining the translational integrity
and overall impression of translated summary materials (abstracts and titles) from the ve highest
ranking (SCImago Journal Rank) Brazilian journals in the eld of psychology. Analysis proceeded
with two models. In the rst model, translated summary materials from 12 randomly-selected articles
from four of the ve journals were evaluated by a panel of three native English-language scholars.
Findings indicated an inverse relationship between the overall impression of the materials and their:
abstract errors, r(34) = -0.61, p < .001; and total errors, r(34) = -0.62, p < .001; suggesting a direct
relationship between the translational integrity of these EDN materials and the overall impression
they leave with native English-language scholars. A second model added 3 additional articles from
the fth journal (English-language only) to the materials described. The ndings from this second
model suggested that for EDN journals, an investment in language resources may substantially
improve the impression they leave with native English-language scholars, and thus promote wider
dissemination of their ndings.
Keywords: English, translation, Brazil, lost science, lingua franca
Resumo
No século atual, o inglês tem sido a língua usada preferencialmente para pesquisa e a divulgação
cientíca. Mas para muitos pesquisadores, o inglês é uma língua estrangeira. Essa constatação é
muito verdadeira, especialmente, para nações emergentes e em desenvolvimento, (EDNs – Emerging
and Developing Nations) tais como as nações do BRICS, abrangendo Brasil, Rússia, Índia, China,
e África do Sul. O presente estudo é um levantamento da integridade translacional e a compreensão
geral de sumários (resumos e títulos) das revistas brasileiras que ocupam os cinco primeiros lugares
da classicação do SCImago Journal Rank, no campo da psicologia. A análise foi organizada em dois
modelos. No primeiro, três professores de língua nativa inglesa avaliaram a tradução dos sumários
de 12 artigos escolhidos aleatoriamente de quatro das cinco revistas. Os achados indicaram uma
relação inversa entre a impressão geral e seus respectivos: erros no resumo r(34) = -0.61, p < .001;
e erros totais r(34) = -0.62, p < .001; sugerindo uma relação direta entre a integridade translacional
e a impressão geral que os artigos deixaram em professores de língua nativa inglesa. Um segundo
modelo acrescentou 3 artigos de uma quinta revista, toda ela escrita em língua inglesa, aos materiais
descritos. Os achados deste segundo modelo sugeriram que para as revistas EDNs, um investimento
em recursos de linguagem poderão aumentar, substancialmente, a impressão que elas estão deixando
em professores de língua nativa inglesa, e incrementar a divulgação dos seus achados.
Palavras-chave: inglês, tradução, Brasil, ciência perdida, língua franca
* Corresponding author: Chris Fradkin, Programa de Estudos em Educação,
Centro Universitário La Salle, UNILASALLE, Av. Victor Barreto, 2288,
Canoas, Rio Grande do Sul, 92010-000, Brasil. Email: chrisfradkin@gmail.com
Acknowledgements: Funded by a pós-doc through Coordenação de
Aperfeiçoamento de Pessoal de Nivel Superior (CAPES), Ministério da
Educação, Brasil.
1 Consulting editors for this article: Claudio Hutz & Gustavo Gauer (UFR-
GS), J. Landeira-Fernandez & Daniel Mograbi (PUC-Rio)
Psychology
/Psicologia Reexão e Crítica, 28(S), 99-111.
100
The enterprise of scholarship consists largely of writing
manuscripts. It is a process of construction, in which
oor-by-oor, level-by-level, an argument is presented
(the hypothesis), the argument is tested, and the ndings
are interpreted and posited for value in the future. This
level-by-level process involves a literature search, in which
foundational material is gathered. In the 21st century, this
search is conducted primarily on computers, with the
scholar typing key words into worldwide databases (e.g.,
PubMed, PsycINFO, JSTOR, Google Scholar), which
return published articles related to the search. From these
published articles, the scholar gleans background and
theory that will serve as the foundation—for the manuscript
that he or she will write.
With worldwide databases, the scholar now has access
to published science from all corners of the globe. While
the diversity of information is a boon for hungry scholars,
this abundance also presents challenges. Foremost of
these challenges is language. For the English-language
scholar, this challenge may entail deciphering non-English-
language texts from non-English-language foreign research
journals. However, with English as the lingua franca of
the world, as the universal language of science (de Swaan,
2001; Meneghini & Packer, 2007), there is increasingly
inclusion of summary materials in English, within the
covers of these journals.
For example, when native-English scholars browse
through non-English-language journals, they typically
encounter full texts in native language with abstracts and
titles—the summary materials—in both native language
and in English. While foreign-language journals on
occasion issue special English-language issues, for the
most part this format is the norm. What this offers to the
English-language scholar is the opportunity to grab the
“gist” of an article—its purpose, methodology, results and
implications. But does this hybrid format serve the reader
or the author? Is information or confusion conveyed?
There has been considerable writing about the so-called
“lost science” (e.g., Hanes, 2014; Meneghini & Packer,
2007; Montgomery, 2013; Packer & Meneghini, 2007).
Coined by Gibbs (1995), this term refers to the unaccessed
scientic output of the “emerging” or “developing” nations
(EDNs). A frequently grouped constellation of EDNs is
the BRICS nations, consisting of Brazil, Russia, India,
China, and South Africa (Wilson & Purushothaman, 2003)
and constituting 43% of the world’s population (Thussu
& Nordenstreng, 2015). Unfortunately, even in the digital
age, with open-access format, science from the BRICS
nations remains inaccessible or “lost” to many scholars in
the English-speaking world. Why? Because the publishing
is not in lingua franca (English) (Gibbs, 1995). In response
to this conundrum, the dual-language abstract and title—at
best a compromise—attempts to bridge the current gap. But
does it work? Do these hybrid summary materials engage
the English-language scholar? Or do they crumble in the
process of translation? In our search for answers to these
questions, we will sample several journals from Brazil.
For our survey, we will focus on psychology.
Psychology is a eld in which Brazil has grown quite
amply in recent years (Gamba, Packer, & Meneghini, 2015),
with some authorities describing Brazil’s recent boon as
a golden age, similar to what took place in the US in the
1940s and the 1950s (Hutz, McCarthy, & Gomes, 2004).
This increase is substantiated by the SCImago database
2014 edition, which reveals that between 2004 and 2013,
“the number of Brazilian psychology publications leaped
from 136 to 1,032 articles per year, which corresponds
respectively to a 0.41% and 1.59% share of worldwide
publications” (Gamba, Packer, & Meneghini, 2015, p.
67). Additionally, among the nations of Latin America
and the Caribbean, SCImago notes that Brazil contributes
more than 54% of published output in the eld, and on the
worldwide stage, ranks 15th (SCImago, 2015).
In spite of Brazil’s impressive EDN statistics, on the
international stage there are problems. In fact, the top
Brazilian psychology journals perform below the mean,
when ranked within the global perspective (SCImago,
2015). As to reasons for this contrast, most authorities
attribute it to language (Collazo-Reyes, Luna-Morales,
Russell, & rez-Angón, 2008; Meneghini & Packer,
2007; Packer, 2014). The editorial from this special
issue cites language as the major challenge for Brazilian
scientists (Gomes & Fradkin, 2015). Gamba, Packer, and
Meneghini (2015) attribute the struggling performance of
Brazilian journals on the international stage to the scarcity
of English language articles. A recent examination of
several top-ranked Brazilian journals found “prociency
problems” in the English-language texts in terms of:
awkward collocation, nominal group error, punctuation
and capitalization, and preposition use error (Hanes,
2014, p. 127). In this study, grammatical error rates
among the journals were quite variable, ranging from
2.41 errors/1000 words to 113.59 errors/1000 words
(Hanes, 2014). Based on the wide variability in error
rates, one might assume that higher error rates would
correlate with poorer rst impression of the journal,
especially for the native English-language scholar. But
is that necessarily so?
While the Hanes (2014) study highlights the grammatical
shortcomings of translation in Brazilian journals, there is
no work to this date that has evaluated the translational
integrity of summary materials (i.e., the abstract and title)
from Brazilian or other EDN journals. Yet, authorities
believe these summary materials are critical. For one,
the American Psychological Association (APA) stresses
the importance of conciseness and coherence in a title
(American Psychological Association [APA], 2010, p.
23) and also stresses that a “well-prepared abstract can be
the most important single paragraph in an article” (APA,
2010, p. 26). After all, the abstract is the “rst contact” the
reader has with the article when browsing in a literature
search (APA, 2010, p. 26).
101
Fradkin, C. (2015). A Summary Evaluation of the Top-Five Brazilian Psychology Journals by Native English-Language Scholars
We should note that while the translational integrity of
summary materials does depend on grammatical integrity,
it also depends on structured formats. For example,
researchers browsing through an empirical study would,
at minimum, expect the abstract to inform them of: (1)
the research problem; (2) the participants; (3) features
of the study methodology; (4) the findings (including
effect sizes, confidence intervals, indications of statistical
significance); and (5) the conclusions and the implications
of the study (APA, 2010, p. 26). Thus, these English-
language summary materials provide opportunity for both
non-native English-language scholars and the journals that
disseminate their work. At their best, they are a bridge that
serves to span the mighty distance between the lingua-
franca and non-lingua-franca worlds.
The aim of this study, therefore, was to empirically
examine the translational integrity and overall impression
that translated EDN summary materials leave with native
English-language scholars. We focused on materials from
Brazilian psychology journals. Based on the weight of
findings from past research, we hypothesized that: (a)
there would be mixed levels of translational integrity in
the summary materials; (b) the summary materials would
leave a mixed impression with native English-language
scholars; and (c) there would be a positive correlation
between translational integrity and the overall impression
the summary materials left with native English-language
scholars.
Method
A random sampling of abstracts and titles from the
five highest-ranked Brazilian psychology journals was
presented to a discriminating group of English-language
scholars. The scholars, in turn, evaluated the materials for
translational integrity and overall impression.
Participants
Participants included English-language scholars
recruited from the University of California. Inclusion
criteria were: (1) native English-language speaker; (2)
psychologist; (3) tenured faculty member; and (4) strong
publication history. Candidates were restricted to associate
or full professors; with a minimum of 2,000 citations
in their publication history. First-wave invitations were
disseminated to a set of three candidates who satisfied the
criteria. The candidates were informed that the study was
examining the accessibility of foreign-language scientific
articles that have been translated into English; and that
half an hour of their time was requested to evaluate
materials. All three first-wave candidates accepted; this
resulted in a sample distribution consisting of 67% male,
100% Caucasian, 67% full (vs. associate) professor, age
M = 57 years, and M citation count = 2,582. The sample
represented the sub-areas of developmental (33%), health
(33%), and quantitative (33%) psychology. Citation
counts were ascertained through Google Scholar. Native
English-language speaker status was ascertained through
educational history (CV) and in person. Compensation
was not offered to participants.
Materials
Of the 1,042 worldwide psychology journals listed in
the 2013 SCImago Journal Ranking (SJR), 16 (1.5%) are
from Brazil. Three original research articles from the most-
recent issues of the five highest-ranked Brazilian journals
in this field (SCImago, 2015) were randomly selected
for analysis. All journals were peer-reviewed. Articles
in review, editorial, and commentary were excluded.
Abstracts and titles were gathered from the final articles (N
= 15) and assembled on evaluation sheets, one article per
sheet, with instructions for the evaluating scholars. With
three evaluators for each article, this resulted in 45 sheets
total (3 evaluators x 15 articles). Journal titles and names
of authors were excluded from the sheets. Because one
journal of the five did not include Portuguese-and-English
summary materials, we considered data from that journal
in a secondary model, hereafter referred to as Model 2.
Descriptive details on the journals appear in Table 1. For
a sample evaluation sheet, see Appendix A.
Measures
Error counts. Error counts served as proxy for
translational integrity (i.e., low error count: high
translational integrity). Title errors were phrases, words
or sections in the title that “bumped” or inhibited the flow
of information intake (e.g., grammatical errors, sentence
structure errors, structural anomalies). Title errors were
noted by the evaluating scholars by their underlining or
circling the offending phrases, words or sections on each
evaluation sheet. Abstract errors were phrases, words,
sentences or sections in the abstract that “bumped” or
inhibited the flow of information intake (e.g., grammatical
errors, sentence structure errors, structural anomalies).
As with title errors, abstract errors were noted by the
evaluating scholars by their underlining or circling the
offending phrases, words, sentences or sections on each
evaluation sheet. Importantly, it should be noted that
for its usage in this study, the term abstract errors refers
specifically to errors in the abstract of the paper, vs. vague
or non-specific errors. Total errors were calculated by
summing title errors with abstract errors.
Overall impression. Overall impression was indexed
on a 5-point Likert scale, with evaluators rating their
impression of the publication, based on their review of
the materials: “Based on your review of the abstract and
title of this article, rate the likelihood of your revisiting
this publication in the future: 0, non-existent; 1, unlikely;
2, possible; 3, likely; 4, very likely.” Post evaluation,
these responses were recoded into a tri-category scale
(0-1, Negative; 2, Neutral; 3-4, Positive), as is presented
in Table 2.
Psychology
/Psicologia Reexão e Crítica, 28(S), 99-111.
102
Procedure
Evaluators and facilitator signed consent forms.
Evaluators were then handed the 15 sheets/articles for
evaluation, and instructed to approach the task as if they
were browsing for articles relevant to their research.
The instructions were reviewed for: (1) notating errors
or “bumps” in the material; and (2) registering overall
impression (see Appendix A). Evaluators were also
reminded to focus on errors or structural anomalies that
distracted them from the information-intake experience.
They were reminded not to nitpick. They were also
reminded that one phrase could qualify for more than
one error. Evaluations were completed between May 14
and June 12, 2015, with each of the evaluators returning
a full set (15) of completed sheets to facilitator. The
three sets of completed sheets (N = 45) comprised the
data set.
Models
Model 1 was the primary model for our study and consisted
of the four journals with Portuguese-and-English summary
materials: Teoria e Pesquisa (B ra sil ia ), Psicologia e Sociedade,
Paidéia (Ribeirão Preto), and Psicologia: Reexão e Crítica.
Model 2 was the secondary model for our study and consisted
of the four previously mentioned journals plus Psychology &
Neuroscience, the journal published exclusively in English,
in partnership with the APA. As Model 1 addresses the main
questions of our study; accordingly, our ndings will focus on
that model. Having said that, when Model 2 offers contrasts
or pertinent perspectives, the ndings from that model will
come forward. Whenever possible, however, data from and
ndings for each model appear jointly in tables and in gures.
And nally, for the sake of economy, the journals Teoria e
Pesquisa (B ra sil ia ) an d Paidéia (Ribeirão Preto) will hereafter
be referred to without their identifying cities in parentheses.
Rank Rank Quintile Journal Worldwide (N) SJR Impact SciELO Qualis Other
Brazil World World Rating Factor Inclusion Rating
(1,042)
1666 4th Psicologia: Teoria e
Pesquisa (Brasilia) 0.31 0.21 Yes A1 --
2 685 4th Psicologia e Sociedade 0.29 0.15 Yes A2 --
3689 4th Paidéia (Ribeirão Preto) 0.29 0.18 Yes A1 --
4720 4th Psychology &
Neuroscience 0.26 0.62 NoaA2 APA
5725 4th Psicologia: Reexão e
Crítica 0.26 0.27 Yes A1 --
Table 1
Descriptive Statistics of the Top Brazilian Psychology Journals
Note. Rankings per SCImago; SJR Rating, SCImago Journal Rank indicator; Impact Factor, 2013 citations of articles published
2010-12; SciELO, Scientic Electronic Library Online database; APA, journal published by American Psychological Association
as of 2015.
aIndexed in SciELO 2008-2014; withdrawn from SciELO upon partnership with APA in 2015.
Included only in Model 2 analyses.
Overall Impression ------ Negative ------ Neutral ------ Positive ------
Likert Scale 0 1 23 4
Table 2
Overall Rating Scale
Based on your review of the abstract and title of this article, rate the likelihood of your revisiting this journal in the future.
_____ 0, non-existent;
_____ 1, unlikely;
_____ 2, possible;
_____ 3, likely;
_____ 4, very likely.
103
Fradkin, C. (2015). A Summary Evaluation of the Top-Five Brazilian Psychology Journals by Native English-Language Scholars
Errors Overall
Journal / Article… Evaluator Title Abstract Total Rating
Psicologia: Teoria e Pesquisa (Brasilia): vol.31, no.1
Art. 1 1 0 0 0 2
20 3 3 2
3 0 1 1 3
Art. 2 1 1 6 7 2
216 7 1
3 1 7 8 2
Art. 3 1 0 2 2 2
206 6 1
3 0 2 2 2
Psicologia e Sociedade: vol.27, no.1
Art. 1 1 0 4 4 1
2189 0
3210 12 1
Art. 2 1 0 5 5 1
206 6 1
3 1 3 4 2
Art. 3 1 0 10 10 1
208 8 1
3 1 9 10 2
Paidéia (Ribeirão Preto): vol.25, no.60
Art. 1 1 0 1 1 2
20 1 1 3
3 0 1 1 3
Art. 2 1 0 5 5 2
20 3 3 3
3 0 3 3 3
Art. 3 1 0 7 7 2
20 5 5 2
3 0 4 4 2
Psychology & Neuroscience: vol.8, no.1
Art. 1 1 0 1 1 2
202 2 4
3 0 4 4 3
Art. 2 1 0 1 1 2
206 6 3
3 0 3 3 3
Art. 3 1 0 0 0 2
202 2 4
3 0 1 1 3
Psicologia: Reexão e Crítica: vol.28, no.1
Art. 1 1 0 5 5 2
20 9 9 2
3 0 4 4 3
Art. 2 1 0 1 1 2
20 1 1 4
3 0 2 2 3
Art. 3 1 0 1 1 2
2 2 3 5 2
3 1 3 4 2
Table 3
Summary Material Ratings by Native English-Language Scholars: Raw Scores
Note. Errors, # of errors cited; Overall Rating, “Based on your review of the abstract and title of this article, rate the likelihood
of your revisiting this journal in the future. (0, nonexistent; 1, unlikely; 2, possible; 3, likely; 4 very likely).” Art. 1/Art. 2/Art. 3,
randomly selected articles.
Included only in Model 2 analyses.
Psychology
/Psicologia Reexão e Crítica, 28(S), 99-111.
104
Analysis
Analyses were performed using SPSS version 16.0. First,
title errors, abstract errors, and overall rating were manually
transcribed from the 45 evaluation sheets. Although not
included in the analyses, evaluator comments were transcribed
as well. The variable total errors was then calculated,
summing title and abstract errors. These raw scores are
displayed in Table 3. Inter-rater reliability (IRR) was then
addressed separately for both models using two-way mixed,
absolute agreement, average-measures intraclass correlations
(ICCs) (McGraw & Wong, 1996) to assess the degree that the
three evaluators provided consistency in their ratings of the
articles. ICCs were calculated separately for each of the four
variables (title errors, abstract errors, total errors, and overall
rating) to estimate consistency of agreement across evaluators.
A robust ICC would suggest that a minimal amount of
measurement error was introduced by the independent
evaluators, and that their ratings would be suitable for use in
the hypothesis tests of the present study.
Variability in error count (i.e., translational integrity)
was addressed separately for both models through GLM
Univariate Analysis. To test Hypothesis 1, signicant main
effects for the three error variables were separately examined
with Wald F tests set at p < .05. In the event of a signicant
main effect, post hoc pairwise comparison tests, set at p <
.05 using Bonferonni correction for multiple comparisons,
were used to evaluate differences between journals. To test
Hypothesis 2, a signicant main effect for overall rating
was examined with a Wald F test set at p < .05. In the event
of a signicant main effect, post hoc pairwise comparison
tests, set at p < .05 using Bonferonni correction for multiple
comparisons, were used to evaluate differences between
journals. Although not a primary aim of the study, signicant
main effects for the three error variables and overall rating
were separately examined with Wald F tests set at p < .05
across evaluators. As with the previous analyses, in the event
of a signicant main effect, post hoc pairwise comparison
tests, set at p < .05 using Bonferonni correction, were used
to evaluate differences across evaluators.
To test Hypothesis 3, Pearson correlation coefcients
were calculated separately for both models between
overall rating and (1) title errors, (2) abstract errors, and
(3) total errors. Four sets of correlations were calculated:
one for each of three evaluators, plus a fourth for the
aggregated mean. Values for the 5-point Likert scale for
overall rating were then recoded into a tri-category scale
(negative, neutral, positive) for overall impression. Using
this scale, articles and journals were ranked accordingly
by the overall impression that they left with the evaluating
scholars.
Results
Descriptive Statistics
As shown in Table 1, all ve Brazilian journals in our
sample rank in the fourth quintile internationally, placing
them below the international average. On the domestic front,
all but one of the journals (Psychology & Neuroscience)
are currently indexed by the Scientic Electronic Library
Online (SciELO) database, attesting to their presence in
the Latin American and Caribbean markets. Additionally,
the CAPES (Coordenadoria de Aperfeiçoamento de
Pessoal de Nível Superior) Qualis indicator, a rating
administered by Brazil’s Ministry of Education, assigns
three of the ve journals (Psicologia: Teoria e Pesquisa;
Paidéia; Psicologia: Reexão e Crítica) their highest
rating of A1, and two (Psicologia e Sociedade; Psychology
& Neuroscience) their second-highest rating of A2.
Note. SJR, SCImago Journal Rank Indicator (2015); Ann Rev of Psych, Annual Review of Psychology;
Psych Bulletin, Psychological Bulletin; Pers & Soc Psych Rev, Personality and Social Psychology Re-
view; Ann Rev of Clin Psych, Annual Review of Clinical Psychology; Psi: Teoria e Pesquisa, Psicologia:
Teoria e Pesquisa (Brasilia); Psi e Sociedade, Psicologia e Sociedade; Paidéia, Paidéia (Ribeirão Preto);
Psych & Neurosci, Psychology and Neuroscience; Psi: Reexão e Crítica, Psicologia: Reexão e Crítica.
Figure 1. SCImago Journal Ranking (SJR) of highest-ranking international (left) and Brazilian
(right) psychology journals.
105
Fradkin, C. (2015). A Summary Evaluation of the Top-Five Brazilian Psychology Journals by Native English-Language Scholars
Of the five journals, the one included in the second
model only (Psychology & Neuroscience) is published in
partnership with the American Psychological Association
(APA), while the remaining four are self-contained inside
Brazil. As measured by the SCImago Journal Ranking (SJR)
indicator (SCImago, 2015), Figure 1 reveals an impact
differential of 29 to 1 between the top-ve international
psychology journals (M = 8.05) and the top-ve Brazilian
psychology journals in our sample (M = 0.28).
Interrater Reliability
IRR was addressed for both models. For Model 1, ICCs
were calculated separately for the four variables of interest
and yielded values, per Cicchetti (1994), in the good (title
errors) to excellent (abstract errors, total errors, overall
impression) range (title errors ICC = 0.630 [95% CI =
0.103, 0.880], abstract errors ICC = 0.863 [95% CI = 0.647,
0.957], total errors ICC = 0.849 [95% CI = 0.613, 0.952],
overall impression ICC = 0.768 [95% CI = 0.407, 0.926]).
These values indicated that coders had a reasonably strong
degree of agreement and suggested that ratings were
consistent across evaluators. Incorporating the fth journal
into the analyses, the ICCs for Model 2 were substantially
the same, with values in the good (title errors, overall
impression) to excellent (abstract errors, total errors) range
(title errors ICC = 0.655 [95% CI = 0.225, 0.871], abstract
errors ICC = 0.849 [95% CI = 0.647, 0.944], total errors
ICC = 0.844 [95% CI = 0.636, 0.943], overall impression
ICC = 0.737 [95% CI = 0.393, 0.903]). The moderately
high ICCs suggest that a minimal amount of measurement
error was introduced by the independent evaluators; and
that evaluator ratings were therefore deemed to be suitable
for use in the hypothesis tests of the study.
Error Counts (Hypothesis 1)
Model 1. As seen in Table 4, while mean title error
counts per article ranged from 0.00 (Paidéia) to 0.56
(Psicologia e Sociedade), the main effect, F(3, 32) = 1.48,
p = .239, was not signicant. Mean abstract error counts
ranged from 3.22 (Psicologia: Reexão e Crítica) to 7.00
(Psicologia e Sociedade), with a signicant main effect,
F(3, 32) = 4.76, p = .007. Post hoc analysis found the mean
abstract error count of Psicologia e Sociedade statistically
higher than the mean abstract error counts of the other three
journals. Congured from the sum of title and abstract error
counts, mean total error counts ranged from 3.33 (Paidéia)
to 7.56 (Psicologia e Sociedade), and likewise, univariate
analysis revealed a signicant main effect, F(3, 32) = 4.87,
p = . 007 . Po st ho c an alys is f oun d the mea n to tal e rro r co unt
of Psicologia e Sociedade statistically higher than the mean
total error counts of Paidéia and Psicologia: Reexão e
Crítica; however there was no statistical difference in the
mean total error count of Psicologia: Teoria e Pesquisa
and the mean total error counts of the other three journals.
As with the other post hoc pairwise comparison tests,
these analyses were conducted at p < .05 with Bonferonni
multiple pairwise correction. Error counts were also
examined across evaluators, but consistent with the IRR
Errors Overall
Journal NTitle Abstract Total Rating
Model 1
Psicologia: Teoria e Pesquisa (Brasilia) 9 0.33a (0.50) 3.67a (2.60) 4.00ab (3.00) 1.89ab (0.60)
Psicologia e Sociedade 9 0.56a (0.73) 7.00b (2.60) 7.56a (2.92) 1.11a (0.60)
Paidéia (Ribeirão Preto) 9 0.00a (0.00) 3.33a (2.12) 3.33b (2.12) 2.44b (0.53)
Psicologia: Reexão e Crítica 90.33a (0.71) 3.22a (2.59) 3.56b (2.65) 2.44b (0.73)
Aggregated 36 0.31 (0.58) 4.31 (2.86) 4.61 (3.11) 1.97 (0.81)
Model 2
Psicologia: Teoria e Pesquisa (Brasilia) 9 0.33a (0.50) 3.67a (2.60) 4.00ab (3.00) 1.89ab (0.60)
Psicologia e Sociedade 9 0.56a (0.73) 7.00b (2.60) 7.56a (2.92) 1.11b (0.60)
Paidéia (Ribeirão Preto) 9 0.00a (0.00) 3.33a (2.12) 3.33b (2.12) 2.44ac (0.53)
Psychology & Neuroscience 9 0.00a (0.00) 2.22a (1.86) 2.22b (1.86) 2.89cd (0.78)
Psicologia: Reexão e Crítica 90.33a (0.71) 3.22a (2.59) 3.56b (2.65) 2.44ad (0.73)
Aggregated 45 0.24 (0.53) 3.89 (2.80) 4.13 (3.04) 2.16 (0.88)
Table 4
Mean (and Standard Deviation) for Error and Overall Rating Variables: Across Journals
Note. a,b,c,dDifferent superscripts indicate values in the column that are signicantly different from one another by pairwise comparison
test at p < .05, with Bonferroni correction for multiple pairwise comparisons.
Model 1, four-journal model with summary materials in Portuguese and English.
Model 2, ve-journal model with Psychology & Neuroscience, which publishes exclusively in English.
Included only in Model 2 analyses.
Psychology
/Psicologia Reexão e Crítica, 28(S), 99-111.
106
analysis, no main effect was found: title errors, F(2, 33) =
1.65, p = .208; abstract errors, F(2, 33) = 0.41, p = .668;
total errors, F(2, 33) = 0.47, p = .629. With absence of a
main effect, post hoc analyses were not pursued. Means
and standard deviations are displayed in Table 5.
Model 2. As seen in Table 4, when incorporating
the fth journal (Psychology & Neuroscience) into the
analyses, the mean error counts were reduced for all three
variables while the mean overall rating was increased.
However, there were no substantial changes in effects: title
error counts, F(4, 40) = 2.04, p = .107; abstract error counts,
F(4, 40) = 5.30, p = .002; and total error counts, F(4, 40) =
5.66, p = .001. Post hoc analyses were also consistent with
Model 1. As in Model 1, error counts were also examined
across evaluators, but consistent with the IRR analysis, no
main effect was found: title errors, F(2, 42) = 1.55, p =
.225; abstract errors, F(2, 42) = 0.86, p = .432; total errors,
F(2, 42) = 0.96, p = .393. With absence of a main effect,
post hoc analyses were not pursued. Means and standard
deviations are displayed in Table 5.
Evaluator Comments
Evaluator comments were unsolicited and were offered
in response to materials from three journals. Psicologia:
Teoria e Pesquisa and Paidéia received one and two
comments, respectively, on inconsistent or inappropriate
verb tense. These comments were from Evaluator 2.
Psicologia e Sociedade received the harshest comments
(e.g., “very convoluted,” “generally poor,” “bad”). These
comments came from all three evaluators. However, the
presence or absence of evaluator comments did not factor
into the analyses.
Overall Rating (Hypothesis 2)
Model 1. As seen in Table 4, mean overall ratings
ranged from 1.11 (Psicologia e Sociedade) to 2.44 (Paidéia
and Psicologia: Reexão e Crítica), with a signicant
main effect, F(3, 32) = 9.38, p < .001. Post hoc analysis
found the mean overall rating of Psicologia e Sociedade
statistically lower than the mean overall ratings of Paidéia
and Psicologia: Reexão e Crítica; however there was
no statistical difference in the mean overall rating of
Psicologia: Teoria e Pesquisa and the mean overall rating
of the other three journals. Post hoc pairwise comparison
tests were conducted at p < .05 with Bonferonni multiple
pairwise correction. Overall rating was also examined across
evaluators, but consistent with the IRR analysis, no main
effect was found, F(2, 33) = 1.92, p = .163. With absence
of a main effect, post hoc analyses were not pursued. Means
and standard deviations are displayed in Table 5.
Model 2. As seen in Table 4, when incorporating
the fth journal (Psychology & Neuroscience) into the
analyses, the mean overall ratings now range from 1.11
(Psicologia e Sociedade) to a higher 2.89 (Psychology
& Neuroscience). However, there was no substantial
change in effect size, F(4, 40) = 9.82, p < .001. Post hoc
analysis found the mean overall rating of Psychology &
Neuroscience statistically higher than the mean overall
ratings of Psicologia e Sociedade and Psicologia: Teoria
e Pesquisa; and the mean overall rating of Psicologia e
Sociedade statistically lower than the mean overall ratings
of Paidéia, Psychology & Neuroscience, and Psicologia:
Reexão e Crítica. Post hoc pairwise comparison tests were
conducted at p < .05 with Bonferonni multiple pairwise
correction. Overall rating was also examined across
Errors Overall
Evaluator NTitle Abstract Total Rating
Model 1
Evaluator 1 12 0.08a (0.29) 3.92a (3.00) 4.00a (3.08) 1.75a (0.45)
Evaluator 2 12 0.33a (0.65) 4.92a (2.71) 5.25a (2.80) 1.83a (1.12)
Evaluator 3 12 0.50a (0.67) 4.08a (3.00) 4.58a (3.55) 2.33a (0.65)
Aggregated 36 0.31 (0.58) 4.31 (2.86) 4.61 (3.11) 1.97 (0.81)
Model 2
Evaluator 1 15 0.07a (0.26) 3.27a (2.99) 3.33a (3.06) 1.80a (0.41)
Evaluator 2 15 0.27a (0.59) 4.60a (2.64) 4.87a (2.75) 2.20a (1.27)
Evaluator 3 15 0.40a (0.63) 3.80a (2.78) 4.20a (3.30) 2.47a (0.64)
Aggregated 45 0.24 (0.53) 3.89 (2.80) 4.13 (3.04) 2.16 (0.88)
Table 5
Mean (and Standard Deviation) for Error and Overall Rating Variables: Across Evaluators
Note. a,bDifferent superscripts indicate values in the column that are signicantly different from one another by pairwise comparison
test at p < .05, with Bonferroni correction for multiple pairwise comparisons.
Model 1, four-journal model with summary materials in Portuguese and English: Psicologia: Teoria e Pesquisa (Brasilia), Psicologia e
Sociedade, Paidéia (Ribeirão Preto), Psicologia: Reexão e Crítica. Model 2, ve-journal model: the above journals plus Psychology
& Neuroscience, which publishes exclusively in English.
107
Fradkin, C. (2015). A Summary Evaluation of the Top-Five Brazilian Psychology Journals by Native English-Language Scholars
evaluators, but consistent with the IRR analysis, no main
effect was found, F(2, 42) = 2.32, p = .110. With absence of
a main effect, post hoc analyses were not pursued. Means
and standard deviations are displayed in Table 5.
Associations Between Overall Rating and Error Counts
(Hypothesis 3)
Model 1. Table 6 reports the association between mean
overall rating and the three mean error counts: individually
for each evaluator and aggregated, for the panel, as a
whole. Considered as a whole, ndings indicated a strongly
negatively correlated relationship between mean overall
rating and mean abstract errors, r(34) = -0.61, p < .001; and
mean overall rating and mean total errors, r(34) = -0.62, p
< .001. There was no statistically signicant relationship
between mean overall rating and mean title errors, r(34) =
-0.29, p = .090. For evaluator 1, there were no statistically
signicant relationships between mean overall rating and
any of the mean error counts: title errors, r(10) = 0.17, p
= .588; abstract errors, r(10) = -0.49, p = .109; and total
errors, r(10) = -0.46, p = .135. For evaluator 2, ndings
indicated a strongly negatively correlated relationship
between mean overall rating and mean abstract errors,
r(10) = -0.79, p = .002; as well as mean overall rating
and mean total errors, r(10) = -0.83, p = .001; while
the relationship between mean overall rating and mean
title errors, r(10) = -0.29, p = .357, was not statistically
signicant. For evaluator 3, ndings indicated a strongly
negatively correlated relationship between mean overall
rating and all three mean error counts: title errors r(10) =
-0.83, p = .001; abstract errors r(10) = -0.71, p = .009; and
total errors r(10) = -0.76, p = .004.
Model 2. As seen in Table 6, incorporating the
fth journal (Psychology & Neuroscience) into the
analyses of the sample as a whole, elicited a moderately
ne gativel y corr ela ted re lat ions hip betwe en me an
overall rating and mean title errors, r(43) = -0.33, p
= .028; a strongly negatively correlated relationship
between mean overall rating and mean abstract errors,
r(43) = -0.55, p < 0.001; and a strongly negatively
correlated relationship between mean overall rating
and mean total errors, r(43) = -0.56, p < .001. For
evaluator 1, ndings now indicated a strongly negatively
correlated relationship between mean overall rating
and mean abstract errors, r(13) = -0.53, p = .042; while
relationships between mean overall rating and mean
title errors, r(13) = 0.14, p = .635; and mean overall
rating and mean total errors, r(13) = -0.51, p = .054,
remained not statistically signicant. For evaluator 2,
the associations were not substantially changed; as they
were not, also, for Evaluator 3.
Overall Impression (Hypotheses 1 & 2)
Table 7 presents an overall impression ranking of the
Brazilian journals in the sample, based on a tri-category
(negative, neutral, positive) scale (see Table 2 for
conversion). For Model 1, which includes the journals with
dual-language summary materials, Paidéia an d Psicologia:
Reexão e Crítica ranked highest, both with positive
overall impression ratings based on two positive (++) and
one neutral (±) article rating. Psicologia: Teoria e Pesquisa
ranked second lowest with a negative overall impression
rating based on one positive (+) and two negative (--)
article ratings. And Psicologia e Sociedade ranked lowest
of the journals, with a negative overall impression rating
based on three negative (---) article ratings. For Model 2,
the fth journal (Psychology & Neuroscience) ranked the
highest, with its positive overall impression rating based
on three positive (+++) article ratings. This, consequently,
moved the other journals down a spot.
Overall Rating:
Evaluator NTitle Abstract Total
Model 1
Evaluator 1 12 0.17 -0.49 -0.46
Evaluator 2 12 -0.29 -0.79** -0.83**
Evaluator 3 12 -0.83** -0.71** -0.76**
Aggregated 36 -0.29 -0.61*** -0.62***
Model 2
Evaluator 1 15 0.13 -0.53* -0.51
Evaluator 2 15 -0.36 -0.77** -0.81***
Evaluator 3 15 -0.85*** -0.71** -0.76**
Aggregated 45 -0.33* -0.55*** -0.56***
Table 6
Pearson Correlation Coefcients Between Overall Rating and Error Variables
Note. Title, title error count; Abstract, abstract error count; Total, total error count.
Model 1, four-journal model with summary materials in Portuguese and English: Psicologia: Teoria e Pesquisa (Brasilia), Psicologia e
Sociedade, Paidéia (Ribeirão Preto), Psicologia: Reexão e Crítica. Model 2, ve-journal model: the above journals plus Psychology
& Neuroscience, which publishes exclusively in English.
Correlation coefcients that reached signicance: *p < .05; **p < .01; ***p < .001.
Psychology
/Psicologia Reexão e Crítica, 28(S), 99-111.
108
Discussion
This study is the rst that we are aware of that establishes
a relationship between the translational integrity of EDN
translated summary materials and the impression they
leave with native English-language scholars. Consistent
with Hypothesis 1, as well as recent research (e.g., Hanes,
2014), is the nding of mixed levels of translational
integrity among the journals in our sample. Consistent
with Hypothesis 2 is the nding that the summary materials
left a mixed overall impression with our native English-
language scholars. And consistent with Hypothesis 3 is
the nding of a positive correlation between translational
integrity and the overall impression the summary materials
left with our native English-language scholars. This nal
nding provides answers to questions that we posed, as to
the efcacy of dual-language summary materials. As to the
potential that this hybrid format has in narrowing the gap
between the lingua franca and non-lingua franca worlds?
Our prognosis is conditionally positive. It is conditionally
positive because—as our ndings indicate—the overall
impression of the translated summary materials is related
to its translational integrity. This nding forecasts both
a narrowing of the gap and a widening of the gap that
separates the lingua franca and non-lingua franca worlds.
We forecast a narrowing of the gap, for those scientists
and authors who rate high in their translational integrity—
through access to resources or afnity for language; or
a hunger to share their science with the world—; and a
widening of the gap, for scientists who rate substantially
lower, regardless of their circumstance or reason.
Therefore, connection can be built or distance be made
wider, depending on the resource of translation (see Curry
& Lillis, 2014).
Regarding the specics of our ndings, there were
signicant differences in translational integrity and overall
impression ratings between two of the journals that we
surveyed: Psychology & Neuroscience and Psicologia
e Sociedade. A partial explanation may be resources.
As previously mentioned, of the five journals in the
sample, Psychology & Neuroscience is the only journal
in the group that is partnered with an English-language
entity; it is published by the APA. It is important to
remember, however, that while other Brazilian journals
have attempted to publish exclusively in English,
Psychology & Neuroscience was the first Brazilian
psychology journal that did so and succeeded. This
continued success, as well as partnership with the APA,
is at least partially attributable to the journal having native
English-language resources that predated the move to
APA. And also, due to its partnership with the APA—the
world’s largest association of psychologists—the quality
of submissions to Psychology & Neuroscience, in terms
of content and coherence, may be at a slightly higher
level than submissions offered to the other journals. As
to the consistently low ratings of Psicologia e Sociedade,
several observations bear discussion. The rst concerns its
presentation style. Of the ve journals that we surveyed,
Psicologia e Sociedade was the o nly one of a ll t he  ve th at
styled its titles in an “all caps” format. While title styling
is at the discretion of the journal, most native English-
language scholars—psychologists especially—are used
to titles in an APA format: “uppercase and lowercase”
(APA, 2010, p. 23). Conceivably, therefore, an all caps
presentation may have negatively inuenced the ratings.
The second observation concerns evaluator comments.
Of the materials evaluated, the materials from Psicologia
e Sociedade were the only ones that garnered comments
such as “generally poor” or “really didn’t understand this
at all.” Although these comments were not included in
the analyses, they are consistent with our ratings of the
journal. An explanation for this phenomenon may be that
Psicologia e Sociedade represents a segment of psychology
very “Brazilian” in its thought; so Brazilian, in fact, that
it has difculty moving its native scientic concepts into
sister terminology in English.
There were particular challenges we encountered in
this study. The rst was our evaluation of translational
integrity. In contrast to the Hanes study (2014), in which
translated text was evaluated by a linguistic specialist for
specic grammar errors, the evaluation in our study was
more nuanced. In our study, the native English-language
scholars were not only evaluating grammar, they were
dealing with a larger whole. They were dealing with the
all-elusive abstract. The abstract that sums the essence of
the study: its point and purpose, its participants, its method,
what it showed us, where it leads us, what it tells us of the
future. In one paragraph: the essence of our work. By its
very nature, the scientic abstract is a complex beast to
quantify. For grading, there were no specic rubrics we
adhered to. The evaluators focused on disruptions in the
ow; and when encountering them: underlined or circled.
And because disruptions in the ow may vary person-to-
Ratings
Journal Articles Overall
Psychology & Neuroscience + + + +
Paidéia (Ribeirão Preto) + + ± +
Psicologia: Reexão e Crítica + + ± +
Psicologia: Teoria e Pesquisa
(Brasilia) + - - -
Psicologia e Sociedade - - - -
Table 7
Overall Impression of the Top-Five Brazilian Psychology
Journals by Native English-Language Scholars
Note. +, positive: mean overall rating > 2; -, negative: mean
overall rating < 2; ±, neutral: mean overall rating = 2.
See Table 2 for conguration of categories.
Included only in Model 2 analyses.
109
Fradkin, C. (2015). A Summary Evaluation of the Top-Five Brazilian Psychology Journals by Native English-Language Scholars
person, the grading of an abstract is elusive. Nonetheless,
within our study, as reported earlier, there was consistency
across evaluators.
A se con d cha llenge wa s in ou r ove ral l rating
assessment. The instructional wording for this rating was:
“Based on your review of the abstract and title of this
article, rate the likelihood of your revisiting this journal
in the future.” We were reminded, however, that browsing
on-line scholars pay less attention to the journal than the
article itself. When encountering this issue, the facilitator
backed up and claried our intent and answered questions
the evaluator had. And again, as mentioned earlier, the IRR
found consistency across evaluators.
This study aimed to empirically examine the
translational integrity and overall impression that
translated EDN summary materials leave with native
English-language scholars. Previous studies by Theresa
Lillis and Mary Jane Curry have examined writing for
publication practices among non-native English-speaking
scholars (Curry & Lillis, 2014; Lillis & Curry, 2010;
Lillis, Hewings, Vladimirou, & Curry, 2010); as the
previously-mentioned Hanes (2014) study has examined
EDN grammatical integrity from a linguistic perspective.
However, ours is the rst study we are aware of that
empirically examined the association between translational
integrity and overall impression, from the perspective of a
panel of well-published native English-language scholars.
An obvious limitation in this research is that it focuses
on summary materials from one country in one discipline:
Brazil and psychology, respectively. Accordingly, it would
be helpful if we had data on summary materials from other
EDNs, across a spectrum of varied disciplines. Should this
opportunity create itself, it would be especially helpful if
the same instruments were used, which would facilitate
the merging and comparison of data. Comparisons
across disciplines would be interesting to have; as would
comparisons across culture and several other variables.
Expanded research of this nature would obviate another
limitation: in this study, we looked at Portuguese-to-
English translation only. Would ndings be consistent
across Chinese-to-English or Russian-to-English?
Accordingly, future research would address these and other
questions, as we struggle with the issues of lost science.
The present study established a direct relationship
between the translational integrity of EDN summary
materials and their overall impression with native English-
language scholars. In the case of our highest-rated journal
(Psychology & Neuroscience), we see the likely inuence
that language resources has on translational integrity and,
consequently, on the overall impression that the journal
leaves with the native English-speaking scholar. In the case
of our lowest-rated journal (Psicologia e Sociedade), we
see the converse inuence that language resources has on
translational integrity and, consequently, on the overall
impression the journal leaves with the native English-
speaking scholar, through summary materials that, to this
author’s eyes, were not proofread by a native English-
language speaker.
The mention of proofreading brings up our nal issue:
Are the EDN journal editors aware of the importance of
proofreading? Especially translated summary materials?
Through my experience at several EDN journals, in the role
of English-language editor, reviewer, and guest editor, the
answer is emphatically no. Through my experience, albeit
anecdotal, the English-language translation of titles and
abstracts is mostly viewed a nuisance or a necessary task. It
is rarely viewed an opportunity to spread the gospel of one’s
science to the global universe. Hopefully, upon reading this,
a few converts will be made. There is also the distinction
between a “rote translation” and a translation transparent
to the native English-language eye. Using the example of
Portuguese-to-English translation, a rote translation would
translate all the words, with a minimal regard for sentence-
structure stylings or the myriad of subtleties between the
languages. See Marlow (2014) for a top ten list of worst
offenders. A transparent translation, of the same material,
would read as if a native English-speaker were the writer.
And, in accordance with our ndings (Model 1 and Model
2), a transparent translation of coherent subject matter
would engender a positive impression on the native English-
language reader. And a positive impression on the journal.
And, in doing so, recover a scrap of the lost science—in
whatever modicum or tiny bit.
References
American Psychological Association. (2010). Publication
manual of the American Psychological Association (6th ed.).
Washington, DC: American Psychological Association.
American Psychological Association. (2015). Retrieved from
http://www.apa.org/
Cicchetti, D. V. (1994). Guidelines, criteria, and rules of
thumb for evaluating normed and standardized assessment
instruments in psychology. Psychological Assessment, 6(4),
284-290.
Collazo-Reyes, F., Luna-Morales, M., Russell, J., & Pérez-
Angón, M. A. (2008). Publication and citation patterns of
Latin American & Caribbean journals in the SCI and SSCI
from 1995 to 2004. Scientometrics, 75(1), 145-161.
Curry, M. J., & Lillis, T. M. (2014). Strategies and tactics in
academic knowledge production by multilingual scholars.
Education Policy Analysis Archives, 22(32).
de Swaan, A. (2001). Words of the world: The global language
system. Cambridge: Polity Press.
Gamba, E. C., Packer, A. L., & Meneghini, R. (2015). Pathways
to internationalize Brazilian journals of psychology.
Psicologia: Reexão e Crítica, 28(suppl. 1), 66-71.
Gibbs, W. W. (1995). Lost science in the third world. Scientic
American, 273, 92-99.
Gomes, W. B., & Fradkin, C. (2015). Editorial. Psicologia:
Reexão e Crítica, 28(suppl. 1), 1.
Hanes, W. F. (2014). Nominal groups as an indicator of non-
native English communication problems in top-ranked
Brazilian science journals. Belas Inéis, 2, 127-139.
Hutz, C., McCarthy, S., & Gomes, W. (2004). Psychology in
Psychology
/Psicologia Reexão e Crítica, 28(S), 99-111.
110
Brazil: The road behind and the road ahead. In M. J. Stevens
& D.Wedding (Eds.), Handbook of International Psychology
(pp. 151-168). New York: Brunner-Routledge.
Lillis, T. M., & Curry, M. J. (2010). Academic writing in global
context. London: Routledge.
Lillis, T., Hewings, A., Vladimirou, D., & Curry, M. J. (2010).
The geolinguistics of English as an academic lingua franca:
Citation practices across English-medium national and
English-medium international journals. International Journal
of Applied Linguistics, 20(1), 111-135.
Marlow, M. A. (2014). Writing scientic articles like a native
English speaker: Top ten tips for Portuguese speakers. Clinics,
69, 153-157.
McGraw, K. O., & Wong, S. P. (1996). Forming inferences
about some intraclass correlation coefcients. Psychological
Methods, 1(1), 30-46.
Meneghini, R., & Packer, A. L. (2007). Is there science beyond
English? EMBO reports, 8, 112-116.
Montgomery, S. L. (2013). Does science need a global
language?: English and the future of research. Chicago:
University of Chicago Press.
Packer, A. L. (2014). The emergence of journals of Brazil and
scenarios for their future. Educação e Pesquisa, 40(2), 301-
323.
Packer, A. L., & Meneghini, R. (2007). Learning to communicate
science in developing countries. INTERCIENCIA, 32, 643.
SCImago. (2015). SJR SCImago Journal & Country Rank.
Retrieved from http://www.scimagojr.com
Thussu, D. K., & Nordenstreng, K. (2015). Contextualizing the
BRICS media. In K. Nordenstreng, & D. K. Thussu (Eds.),
Mapping BRICS Media (p. 2). New York: Routledge.
Wilson, D., & Purushothaman, R. (2003). Dreaming with BRICs:
The path to 2050 (Vol. 99). Goldman, Sachs & Company.
Received: July 08, 2015
Reviewed: July12, 2015
Accepted: July 15, 2015
111
Fradkin, C. (2015). A Summary Evaluation of the Top-Five Brazilian Psychology Journals by Native English-Language Scholars
Appendix A
Title:
Phasellus nec nisi interdum aliquam nulla vitae: luctus
tellus
Abstract:
Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Fusce pellentesque diam ac leo ultricies lobortis. Nunc
hendrerit in ipsum nec eleifend. Vestibulum sit amet
magna vitae ipsum eleifend pulvinar. Donec semper
tincidunt metus, non euismod tortor. Proin aliquet
condimentum augue, eu pharetra lacus porta nec. Donec
risus lorem, pellentesque quis tellus in, ornare ultrices
diam. Etiam nec magna nec augue feugiat scelerisque.
Curabitur faucibus laoreet neque sit amet imperdiet.
Donec molestie ligula non tincidunt porta. Vestibulum
sed tellus eu ex dictum sollicitudin. Maecenas in elit
dapibus, convallis massa vitae, cursus tortor. Aliquam
consectetur vel dui quis varius. Praesent dapibus
vestibulum ipsum, non sollicitudin urna convallis a.
Donec odio erat, porta ac laoreet quis, maximus id
tortor.
Keywords: dignissim semper, nisl, odio bibendum
ipsum, et quis
INSTRUCTIONS:
1) READ TITLE AND ABSTRACT AS IF YOU WERE
BROWSING FOR ARTICLES RELEVANT TO YOUR
RESEARCH.
2) WHILE READING, CIRCLE OR UNDERLINE
PHRASES OR WORDS OR SENTENCES
THAT ‘BUMP’ OR INHIBIT THE FLOW OF
INFORMATION INTAKE (e.g., grammar errors,
sentence structure errors, structural anomalies).
3) WHEN DONE WITH 1 & 2, RATE THE
LIKELIHOOD OF YOUR REVISITING THE
JOURNAL THESE SPECIFIC MATERIALS CAME
FROM BASED ON THE READ-THROUGH YOU
JUST DID (0 = non-existent through 4 = very likely).
PLEASE:
FOCUS ON ERRORS OR STRUCTURAL
ANOMALIES THAT DISTRACT FROM THE
READING/SCANNING EXPERIENCE: NO NEED
TO NITPICK
AND REMEMBER:
ONE PHRASE COULD QUALIFY FOR MORE
THAN ONE ERROR
OVERALL IMPRESSION RATING:
Based on your review of the abstract and title of this
article, rate the likelihood of your revisiting this journal
in the future.
_____ 0, non-existent;
_____ 1, unlikely;
_____ 2, possible;
_____ 3, likely;
_____ 4, very likely.
... Over the last decade, psychology journals in Brazil have undertaken a commitment to internationalization. Evidence is seen in summary materials (abstracts and titles) and supplementary issues, many of which now appear in English (Fradkin, 2015). Evidence is also seen in the increased presence of Brazilian journals in the international Scopus database (Gamba, Packer, & Meneghini, 2015). ...
... But just because a journal publishes articles in English, that of itself is not assurance of strong science. In fact, a recent study (Fradkin, 2015) on the translational integrity of summary materials in Brazilian psychology journals found substantial variability in the integrity of such materials. The study also found a significant relationship (r = 0.62, p < .001) between the translational integrity of the journal and the overall impression the journal made with native English-language scholars (Fradkin, 2015). ...
... In fact, a recent study (Fradkin, 2015) on the translational integrity of summary materials in Brazilian psychology journals found substantial variability in the integrity of such materials. The study also found a significant relationship (r = 0.62, p < .001) between the translational integrity of the journal and the overall impression the journal made with native English-language scholars (Fradkin, 2015). This suggests that the simple presence of articles in English is not a guarantee of scientific rigor. ...
Article
Full-text available
There is considerable variability among psychology journals in Brazil, in terms of presence on the international stage. However, research as to why is very scarce. This study empirically examined the relationship between several indices of internationalization and real-world internationalization, among these journals. 661 articles from the top-17 psychology journals in Brazil were coded for: English-language text, editorial board makeup, lead author institution, and article type. Analyses revealed that successful internationalization was associated with: (a) lead author institution from a native English-speaking country; (b) empirical articles; and (c) editorial board members from a native English-speaking country. Use of English-language text was not associated with successful internationalization. These findings suggest that the path to internationalization for psychology journals in Brazil may depend on increased publishing of findings from English-speaking countries; or at the very least: increased collaboration between Brazilian and native English-speaking scholars. Keywords: globalization, scientific journals, psychology, Brazil, bibliometric analysis
... With my publication of a series of articles on Brazilian science (Fradkin, 2015(Fradkin, , 2017a(Fradkin, , 2017b(Fradkin, , 2018(Fradkin, , 2019 ...
... 3. What differences were there between the operationalization of SES in the main study and the operationalization of SES in the author's earlier work (Fradkin et al., 2016;Fradkin et al., 2015)? ...
Book
Full-text available
The main objective of this case study is to acquaint the reader with the challenges of conducting cross-cultural research. The vehicle is a study I conducted that examined variability in obesity and being overweight among a sample of Brazilian adolescents. As this was my first time working with an emerging-nation sample, there were conceptual and methodological adjustments I made. These adjustments concerned the operationalization (i.e., construction) of key variables in the study, including race and socioeconomic status. Besides detailing these adjustments, I also discuss how a foundational concept I had used in my previous research in this area was not applicable with my emerging-nation sample. I discuss the construction of a literature review, which included non-English language studies as well as communication challenges I faced in my collaboration with non-native English-speaking colleagues. The need for flexibility and openness is real. My goal is that the reader gains a deeper understanding of the conceptual and methodological issues involved in conducting research on these population groups.
... These withstanding, there has been no comprehensive tally of international collaboration or non-Brazilian contribution, among the psychology journals of Brazil. This absence is surprising, because these variables are linked inextricably to publishing performance (Smith et al., 2014;Bordons et al., 2015;Fradkin, 2015;Gamba et al., 2015). ...
... With the higher impact factor journals publishing in English, the acceptance rate for nonnative English speaking scholars is much lower than for their lingua franca counterparts (Vasconcelos et al., 2007;Paiva et al., 2017). Likewise, the publishing performance for emerging nation journals is much lower than for their lingua franca counterparts (Tijssen et al., 2006;Packer, 2014;Fradkin, 2015Fradkin, , 2017b. This reality creates a less-than-even playing field for the dissemination of newly published science. ...
Article
Full-text available
There is considerable variability in publishing performance among psychology journals in Brazil. However, research as to why is very scarce. This study empirically examined the relationship between non-Brazilian contribution and publishing performance, among these journals. A total of 746 articles from the top-18 psychology journals in Brazil were coded for study type, international collaboration, and non-Brazilian contribution. Analyses revealed that publishing performance was associated with the following: (i) international collaboration and (ii) non-Brazilian contribution. Collaboration with, and contribution from, English-speaking authors was more prevalent among the higher performing journals; while contribution from non-Brazilian Ibero-American authors was more prevalent among the lower performing journals. These findings suggest that publishing performance for psychology journals in Brazil may be strongly tied to non-Brazilian contribution. Implications may be relevant to journal publishers and editors, as well as arbiters of scientific policy.
... The seeking of researchers for international journals has intensified in recent decades, leading to possible global portray of local results. According to Fradkin (2015), despite the difficulty encountered by some Brazilian authors-whose native language is not English-it can be observed that the relevance of the international publications leverages the academic performance. Thus, greater discussion of elements or concepts is expected at a global level; that is, disseminating themes at international level, as well as the appropriation of global tools for the Brazilian debate. ...
... This analysis, regarding academic publications, was not known, therefore, a rigorous Bibliometric survey was required to understand the research framework. Although for some authors international publications involve a higher level of difficulty (Fradkin 2015), the seeking for journals and publications of greatest impact has been widely disseminated globally-as depicting the results of Science Direct database. The pre-salt, for example, has a relatively low number of academic studies, primarily because it is a Brazilian resource. ...
Article
Full-text available
The theme related to the oil production in the Brazilian territory is not limited to the extraction means, distribution and use of this product, and involves political, cultural and environmental axes. The paradigm shift in relation to the market aspects of petroleum is proving to be an issue for new academic discussions and future possibilities. This research aims to analyze the framework of the oil production in Brazil, based on the main contributions of this theme to the academic environment in the national and international spheres. The profiling of oil production in Brazil was conducted by bibliometric analysis, which is a fundamental tool to identify ideas and concepts inserted in a given context. This tool through a query of data from two databases was applied, the national platform Scielo Brazil, and the international platform Science Direct. As a result of the research, it was possible to verify the centralization of the theme in some research axes, verifying new approaches to the oil context at different scales of research.
... Com efeito, as questões e políticas de internacionalização receberam atenção redobrada no XV Simpósio. A proposta era trazer diferentes perspectivas de internacionalização, isto é, como política internacionalização estava sendo entendida na Provavelmente, foi essa instrutiva experiência que levou Fradkin (2015Fradkin ( , 2017 anteriores. Se retornarmos a Gomes (1988) veremos que este periódico contraria a tendência dominante no Brasil, pois tem um foco temático bem definido, mas aberto as suas muitas variações, é criativo e inovador quanto ao escopo e forma dos artigos, e é publicado por uma sociedade científica, a ABRAPSO. ...
Article
Full-text available
Resumo O presente memorandum traz anotações documentadas sobre questões concernentes à pertinência da língua franca (inglês) na formação e publicação em pesquisa, definidas respectivamente como internacionalidade e visibilidade. Reviso marcos históricos como: (1) criação da Revista Psicologia: Reflexão e Crítica, (2) fundação de pós-graduação stricto sensu (UFRGS), (3) proposição do Brazilian Psychological Abstract, base para o Index Psi Periódicos; (4) Fóruns de Internacionalização da Associação Nacional de Pesquisa e Pós-graduação em Psicologia; e (5) projeto do International Journal of Psychological Reviews. Os documentos consultados indicam que a internacionalidade como formação em pesquisa é hoje fato assimilado e aceito pelos programas de pós-graduação, assim como o interesse pela visibilidade das publicações. Reconhece-se que a prática da língua franca acarreta dificuldades operacionais e financeiras. Trazer a língua franca para o cotidiano dos programas mostra-se iniciativa oportuna e promissora. Contudo, o incremento da internacionalidade e da visibilidade dependem de políticas científicas adequadas, consistentes e continuadas. Abstract This present memorandum brings documented notes on questions concerning the relevance of the lingua franca (English) in research training and publication, defined respectively as internationality and visibility. I reviewed historical milestones such as: (1) the creation of the Psicologia: Reflexão e Crítica, (2) the foundation of stricto sensu post-graduation at UFRGS, (3) the proposal of the Brazilian Psychological Abstract that was the basis for the Index Psi Periodicals; (4) the Forum of Internationalization at National Association of Research and Postgraduate in Psychology; (5) the project of the International Journal of Psychological Reviews. The documents consulted indicated that internationality as training in research is now a fact assimilated and accepted by postgraduate programs, as well as interest in the visibility of publications. The lingua franca practice is highly considered, but it entails operational and financial difficulties. Bringing the lingua franca to the daily program activities is a timely and promising initiative. However, increasing internationality and visibility depends on sound, consistent and continuous scientific policies. Introdução O intercâmbio de estudiosos de diferentes países quanto à formação e ao debate das grandes questões do conhecimento é prática vigente através dos tempos. 1 Estudiosos na Grécia percorriam diversas cidades em regiões diferentes e mesmo distantes em busca do encontro com grandes pensadores. A história dos sofistas mostra como a internacionalização do mercado e o intercâmbio entre comerciantes de diferentes regiões levaram à exposição de variações culturais, trazendo mudanças na maneira de pensar a educação. As primeiras universidades criadas nos séculos XI e XII reuniram estudantes de várias regiões da Europa, tendo o latim como língua franca. As grandes transformações do conhecimento ocorridas na época da chamada renascença foram frutos de mudanças sociopolíticas, econômicas, culturais, e da expansão geográfica com a descoberta da América. A internacionalização também foi influência marcante no célebre e reconhecido Instituto de Psicologia Experimental da Universidade de Leipzig na Alemanha, sob a orientação de Wilhelm Wundt (1832-1920), para onde afluíam estudantes do Reino Unido, França, Rússia, Estados Unidos da América, entre outros países, todos interessados no desenvolvimento da nova ciência. Em suma, internacionalidade e sua exigência por visibilidade não é uma questão nova para a história da ciência (Hearnshaw, 1987). No presente memorandum trago aspectos da minha carreira como professor universitário e pesquisador para afirmar que atividades de pesquisa estão vinculadas, por sua natureza, às exigências da internacionalidade e da visibilidade. No meu argumento, a internacionalidade é condição inerente ao diligente trabalho de pesquisa, e está no cerne da formação do pesquisador, na proficiência do método, e na divulgação da ciência. Assim, mostro como a internacionalidade e a visibilidade foram presenças ora implícita, ora explicita, em projetos e ações acadêmicas das quais participei ativamente, seja (1) na criação de um periódico científico e de um programa de pós-graduação (o caso da fundação da Psicologia: Reflexão e Crítica e dos Programas de Mestrado e Doutorado em Psicologia na Universidade Federal do Rio Grande do Sul-UFRGS); (2) na defesa da visibilidade da produção científica brasileira na área da psicologia (o caso do Brazilian Psychological Abstracts); (3) no programa e anais de evento científico nacional (o caso do uso da língua franca da ciência na organização do 15º Simpósio de Intercâmbio Científico da Associação
... I felt guilty for my background, for the color of my skin, for the fact that the first words I spoke were English. Then I pulled myself together and proceeded to explain the Dual-Stage Approach to translation (Fradkin, 2015). ...
Article
Full-text available
While English is the language of science, for many scientists it is a foreign language. Such is the case in South America, where Portuguese and Spanish are spoken. In this commentary, the author shares the experience of presenting a workshop on “English-Language Publication for Nonnative English Speakers,” at a scientific conference in Brazil. Issues of translation and “lost science” are addressed. For the author, this experience raised awareness of the challenges nonnative English scholars have in disseminating their science to an English language market. Recommendations are offered, with the hopes of building bridges to connect the lingua franca and the non–English-speaking worlds.
... This brief review of the past two New Horizons Seminars enables us to understand the progress made so far. The articles in this supplement update us to issues that have been regularly discussed, and bring new and vital issues to the forefront: Brazilian scientific policies (Bastos, Tomanari, Trindade, & Andery, 2015;Guzzo, Linhares, Teodoro, & Koller, 2015), ethical standards for psychological research (Leitão, Falcão, & Maluf, 2015), internationalization (Hutz, Yamamoto, & Lo Bianco, 2015;Menandro, Linhares, Bastos, & Dell'Aglio, 2015), Latin American international publication (Fradkin, 2015;Gamba, Packer, & Meneghini, 2015;Lopez-Lopez, Anegón, Acevedo-Triana, & Garcia, 2015;VandenBos & Winkler, 2015), and transference of journal management to international publishers (Gracia, 2015). ...
Article
Full-text available
This article presents a brief overview of the history of psychology in Brazil. It highlights how the Brazilian Association of Research and Postgraduate Studies in Psychology (ANPEPP – Associação Nacional de Pesquisa e Pós-graduação em Psicologia) has fulfilled its mission of fostering discussion on scientific policy and stimulating interchange among researchers. First, it provides a retrospect of ANPEPP meetings, considering both: 1) the thematic working groups, which have served to bring together researchers, and to inspire the emergence of thematic associations and journals; 2) the discussion forums, which have contributed in the critical review of scientific policy, and the mission of postgraduate studies. Second, it focuses on the history of psychology in Brazil, from colonial times to the recent national commitment to postgraduate studies. The paper argues that the plans and strategies led by national funding agencies have been successful and that their results are evidenced by the role played by Brazil in the international arena, both in scientific production and the training of its researchers. By sustaining current policies, it seems certain that, even with the oscillations in the national economy, postgraduate education will grow steadily in its advance of the psychological sciences; and will be working towards a better quality of life, social justice and ecological sustainability. Keywords: ANPEPP, CAPES, CNPq, history of psychology, postgraduate studies, Brazil.
Article
Full-text available
As with most emerging-nation journals, South American scientific journals struggle in their efforts to reach the global market. Nonetheless, there are exceptions to this trend. This paper chronicles the journey of one of those exceptions, the Brazilian journal Psychology & Neuroscience. I focus on events leading up to and consummating the publishing partnership between Psychology & Neuroscience and its current publisher, The American Psychological Association. An analysis is presented that examines the journal in terms of its publication language, business model, editorial stability, editorial board makeup, and the international qualities of its editors. I present this with the hope it may be useful to emerging-nation scientists and scholars, as well as publishers and institutions, that share similar objectives of distributing their findings to the global market.
Article
Full-text available
Article
Full-text available
A Psicologia no Brasil é uma área científica cujo desenvolvimento remonta aos últimos 50 anos. A maior parte do conhecimento produzido por seus cientistas é publicada em periódicos nacionais. Por conseguinte, tal condição tem imposto barreiras na disseminação deste conhecimento para comunidades internacionais. O potencial para internacionalização da ciência produzida por pesquisadores brasileiros da psicologia encontra dois desafios: um seria levar suas publicações a revistas internacionais; o outro seria internacionalizar as revistas brasileiras de psicologia em geral restritas às fronteiras nacionais. Sobre o segundo desafio, a análise da base de dados da cientometria mostra que as razões para o baixo impacto dos artigos publicados em periódicos nacionais são múltiplas, mas é explicada principalmente pelo escasso número de artigos em língua inglesa. Discute-se o papel que o SciELO, um Publisher científico brasileiro, na ampliação dos horizontes dos melhores periódicos brasileiros, promovendo maior inserção internacional, e na sua afirmação como um importante instrumento nacional para estudos de psicologia.
Article
Full-text available
This study is an attempt to document the problematic nature of an intermediary linguistic system, the lingua franca used by the scientific community, on the production and impact of science from the broad area beyond the inner circle of native English speakers. To this end, a random cross-sectional sample (n=5) of current English-language articles from top-ranked journals in the Brazil-based metapublisher Scientific Electronic Library Online (SciELO) was examined for grammatical issues, especially nominative group construction. In the studied sample, varying and in some cases elevated levels of L1 interference were found, indicating that on the best collective level, there are proficiency problems with the lingua franca, that these problems are not evenly distributed and that systematic language management yielded vastly different language quality outcomes.
Article
Full-text available
In the past decade, academic evaluation systems worldwide have markedly increased the use of mechanisms that privilege the use of English in journal publishing. In the context of these trends, this article highlights our findings from more than 12 years of research on the experiences and perspectives of 50 multilingual European scholars with writing for publication, particularly in English. We draw on de Certeau's (1984) notions of strategies and tactics to explore key ways in which scholars manage often-competing demands and interests in writing for publication. Scholars both adopt strategies that align with official publication policies and use tactics that support scholars' sometimes competing agendas. At different moments scholars embrace, accommodate, or resist the perceived dominance of English in knowledge production regimes and evaluation systems. We conclude by summarizing the value of drawing on the notions of strategy and tactics in an era of increasing debates over evaluation systems.
Article
Full-text available
Quality journals in Brazil have achieved remarkable progress in recent years, with an increasing presence in international bibliographic indexes and on the Web. This is primarily due to Brazilian authors from different fields of knowledge, communicating the results of their research in English and/or Portuguese. At least 80% of their original and review articles that are internationally indexed have a Brazilian affiliation, representing approximately 30% of the total indexed scientific production from Brazil. However, this national centrality has a low impact when measured by the number of citations received in the indexes in comparison to that of journals from developed countries. Despite their performance being comparable to that of journals from emerging countries, and although open access publishing results in an extraordinary number of article downloads, most journals of Brazil face the challenge of becoming qualified to compete on a national and international level for better quality manuscripts, as well as of improving their performance in the international indexes. This sought-after qualification demands that these journals overcome the inherent limitations of institutional, management, and financing conditions in which they operate, while advancing professionalization, internationalization, and innovation in the editing, publishing, and dissemination processes, in order to be aligned with international state-of-the-art standards. This article presents an overview of the main bibliometric and editorial management characteristics of the 400 journals of Brazil indexed in Scientific Electronic Library Online (SciELO), Scopus, and Web of Science (WoS). It also projects scenarios for changing the current framework by promoting journals considered as international benchmarks and the way journals are evaluated and funded.
Article
Full-text available
Can you identify a single colleague who has not had a manuscript returned with the comment “needs to be reviewed by a native English speaker”? Many researchers receive this response even after translation or revision by an official translator or a native English-speaking coauthor. Over the past four years, while conducting my doctoral, and now my postdoctoral, work here in Brazil, I have been asked to both translate and help revise numerous manuscripts for my fellow Brazilian researchers. However, despite being a native English speaker and a researcher, I have found these tasks to be quite stressful at times. The truth is, just like it is one thing to write in Portuguese and another to write well in Portuguese, the same applies to writing well in English. Furthermore, not every native English speaker who writes well in English can write well for the scientific literature. Scientific English writing has its own style and rhythm, such as the use of passive voice. Passive voice is considered poor English in most forms of writing (news, novels, blogs, etc.) outside of science. The most recent version of Microsoft Office Word will even highlight passive voice as poor grammar and ask you if you want to rephrase. However, the use of passive voice is acceptable and even encouraged in some scientific writing.
Article
Full-text available
Many researchers in the developing world feel trapped in a vicious circle of neglect and -- some say -- prejudice by publishing barriers they claim doom good science to oblivion.
Article
Good quality science has been produced in developing countries, as shown by the number of papers published in prestigious journals. However, the competence to produce good journals in these countries has lagged behind for several reasons, particularly the establishment of an international publishing system relying on the increasing value attributed to the ISI-JCR journal ranking, a view adopted by authors worldwide and by funding and evaluation systems. Developing countries became integrated to this international context and the efforts to produce good local journals can be pinpointed to individual initiatives that in most cases failed to progress. One important consequence of this gap is that dealing with the peer review procedure, a major instrument to produce good journals and to foster scientific progress, is a limited experience in developing countries. Under this scenery an enterprise that began in Brazil in 1997 and thereafter spread over twelve other Iberoamerican countries is discussed in the light of recent data. SciELO (Scientific Electronic Library Online) is a program fundamentally supported by public funding, aimed at launching online the best existing journals in several countries, in an open access mode, based on peer-reviewing and bibliometric/scientometric analysis for the purpose of journal indexation and maintenance in its database. SciELO covers the functions of a meta-publisher and aims to operate in accordance with the open access movement, rendering scientific knowledge more widely available. The data presented show encouraging evidences that a new auspicious panorama is being established in the context of producing scientific journals in Brazil.