Available via license: CC BY 4.0
Content may be subject to copyright.
UiT goes open: Et festlig skrift til Stein Høydalsvik
https://doi.org/10.7557/15.5499
© Aysa Ekanger and Solveig Enoksen. This is an open access article distributed
under the terms and conditions of the Creative Commons Attribution 4.0
International license.
- 53 -
Support for Good Peer Review in OJS-based
Journals: A Library Publisher’s Perspective1
Aysa Ekanger and Solveig Enoksen
Abstract
How can a library publishing service with limited resources help
editorial teams of peer-reviewed journals in their work? This paper
focuses on the technical aspects of the peer review workflow that, if
set up and adhered to properly, can contribute to improving the
standard of the peer review process – and to some degree also the
quality of peer review. The discussion is based on the work done at
Septentrio Academic Publishing, the institutional service provider for
open access publishing at UiT The Arctic University of Norway.
Introduction
Septentrio Academic Publishing is a service offered by the University
Library of Tromsø to open access journals and series that are
associated (through editors or scope) with UiT The Arctic University
of Norway.2 The service is part of the library’s and the university’s
commitment to the open access mission: Septentrio offers its support
for free and strives for its journals to have good quality and be visible
in the open access infrastructure. One aim is that all peer-reviewed
journals in Septentrio are indexed in the Directory of Open Access
Journals, DOAJ. As of May 2020, Septentrio publishes ten peer-
reviewed journals in a variety of disciplines (eight of them in the
DOAJ) and eight series that are not formally peer-reviewed.
A handful of library employees are involved in running the publishing
platform (Open Journal Systems from the Public Knowledge Project),
providing technical support to the users, and giving advice to the
1 This paper is a result of our presentation about support for peer review
given at the midterm meeting of the IFLA Library Publishing Special
Interest Group that took place at OsloMet in March 2020. IFLA stands for
The International Federation of Library Associations and Institutions,
https://www.ifla.org/.
2 As this paper is published in a Ravnetrykk issue in honour of Stein
Høydalsvik, it is appropriate to mention that Stein was one of the founders
of the publishing service – and establishment plans for the service were
mentioned in Høydalsvik (2002), https://doi.org/10.7557/15.3844.
Support for Good Peer Review in OJS-based Journals
Ravnetrykk 39, 2020
- 54 -
editorial teams about best practice in various aspects of the
publishing process, such as peer review. The amount of (wo)man-
hours that we can invest in Septentrio work is limited – as all of us
have other tasks. How can a library publishing service with limited
resources help the editorial teams in their work? Our goal is to make
the publishing process smoother, through appropriate workflows set
up according to a journal’s needs on the journal platform and
through relevant information provided to editors, reviewers and
authors at crucial points.
This paper focuses on the technical aspects of the peer review
workflow that, if set up and adhered to properly, can contribute to
improving the standard of the peer review process – and to some
degree also the quality of peer review.
Support for peer review
The publishing software Open Journal Systems (OJS) offers a good
workflow for the peer review stage of the publishing process, with a
number of options that allow to model the peer review process in
accordance with what a specific journal needs. However, not all our
journals use OJS for peer review: some editors prefer the more
familiar method of managing peer reviews via email, outside of OJS.
When we try to explain to editors why they should use OJS for peer
review, we usually point to the benefits and importance of keeping
manuscript history in one place.3 Improvement of the standards of
the peer review process is another reason for why peer reviews should
be handled inside OJS – instead of via email.
So, how can we as technical support staff help editorial teams with
the peer review process? Peer review-related issues that can be
affected by technical configurations, can be roughly divided into two
groups: (adherence to) ethical standards and technicalities pertaining
to blind review (single-blind or double-blind).
Authors, reviewers and editors should be aware of these standards
and technicalities, it must be technically possible for them to comply
with the requirements – and they probably also need to be reminded
of them at various stages in the publishing process. Editors also have
the added responsibility of making sure that authors and reviewers
are following all of these standards and technicalities.
The Public Knowledge Project provides openly available user guides
for the publishing workflow in OJS, where almost every step of the
3 During the publication process, the platform provides information on when
review reports are due and sends automatic reminders to users when an
action needs to be taken – so that it becomes easier for an editor to manage
a journal. In addition, overview over both active and published or archived
submissions is important when there are multiple editors or section
editors involved, when editorial teams change, or when guest editors come
in to publish a specific issue.
UiT goes open: Et festlig skrift til Stein Høydalsvik
Ravnetrykk 39, 2020
- 55 -
editorial process or the review process is described in detail. An
important task of the support staff at Septentrio is to make editorial
teams and other users aware of the OJS functionality, help to set up
the review process in a way that excludes slip-ups, and make
adherence to various guidelines and requirements easier by making
them visible at various steps in the publishing process. The
Septentrio team have also put together a number of guides on the
publishing process, openly available to editors, reviewers and authors
on the Septentrio website.
Ethical standards
There are guidelines for ethical standards in peer review from various
associations. A good place to start for both the library support staff
and journal editors are Ethical Guidelines for Peer Reviewers from
the Committee on Publication Ethics (COPE) 4, as well as the
Principles of Transparency and Best Practice in Scholarly
Publishing 5. Journal editors also need to be aware of discipline-
specific guidelines relevant to their specific journal.
COPE advises reviewers to follow the journal’s instructions for peer
review – this emphasizes how important it is for a journal to have
clear guidelines available for reviewers.
In November 2019, in connection with the 14th Munin Conference on
Scholarly Publishing organized by the university library, COPE was
invited to hold a workshop on the standards of peer review. Editors
of Septentrio journals were personally invited to participate in the
workshop, without admission fees. In 2020, Septentrio is planning to
help its journals to assess whether their peer review processes are
carried out in accordance with the standards of COPE.
The ethical standard issues that can be taken care of with the help
of technical adjustments, include:
• ensuring against reviewer bias and competing interests
• accommodating for appropriate reviewer feedback and
dealing with inappropriate reviews
• dealing with suspected ethics violations on the part of the
authors
Review feedback should be unbiased – not influenced by the
reviewer’s possible competing interests, such as a close connection
to the author or, on the contrary, being part of a competing research
group.
4COPE Council (2017). Ethical guidelines for peer reviewers.
https://doi.org/10.24318/cope.2019.1.9
5 COPE, DOAJ, OASPA, WAME (2018). Principles of Transparency and Best
Practice in Scholarly Publishing. Version 3.
https://doi.org/10.24318/cope.2019.1.12
Support for Good Peer Review in OJS-based Journals
Ravnetrykk 39, 2020
- 56 -
A journal should have a clear conflict of interest disclosure policy
displayed in the reviewer guidelines on the journal’s website. In
addition, in the set-up of the review workflow in OJS, a journal
should choose to make the conflict of interest disclosure policy visible
to reviewers when they log onto the publishing platform. The
workflow settings also allow a journal to choose to request that
reviewers submit a competing interests statement.
Reviewers are supposed to give appropriate feedback. This, according
to COPE, consists of a fair, honest, and unbiased assessment of the
strengths and weaknesses of the manuscript. The journal will of
course be reminding reviewers about this in its guidelines, but there
is also a tool in OJS that journals can use – namely review forms.
A review form consists of predefined questions or statements that
reviewers can react to in a variety of ways: by filling in a text box, by
checking off several checkboxes, by choosing one radio button or an
item from a dropdown menu. It is possible to set up different review
forms for different types of papers that a given journal accepts
(articles, essays, squibs). By setting up review forms, a journal
ensures that reviewers address the issues they need to address.
Another benefit of review forms is that they help to standardize the
review process and make it easier for editors to compare feedback on
the same manuscript from different reviewers.
However thoroughly the review workflow may be set up, there is
always a risk of inappropriate reviews. You may have heard about
cases where e.g. the author's intellect is put into question, or there is
gender bias. It may also happen that the reviewer clearly does not
know the subject, so the review feedback is practically worthless.
There are guidelines on how to deal with inappropriate reviews (from
COPE, or from disciplinary associations), but editors should also
know about technical options OJS provides them with in such
situations.
If an inappropriate review has been submitted in OJS, an editor can
unconsider it – the author will then not receive this review feedback.
For future reference, the editor can then rate the reviewer and include
notes with an explanation of the rating – the rating and notes will not
be visible to the reviewer.6
While reviewing manuscripts, reviewers not only have to adhere to
ethical guidelines themselves, but also to make sure that authors do
so as well. What if a reviewer suspects a violation of ethics in the
research process presented in the manuscript or detects a case of
plagiarism?7 In OJS, a reviewer can inform the editors of a suspected
6 The option of reviewer rating is always there – not only in the case of
inappropriate reviews. The rating system is designed to make it easier for
editors to choose reviewers from the database on the publishing platform.
7 As a member of Crossref, Septentrio has access to the plagiarism-checking
service Similarity Check, powered by iThenticate. All journals at Septentrio
UiT goes open: Et festlig skrift til Stein Høydalsvik
Ravnetrykk 39, 2020
- 57 -
violation without including the author(s) in the communication.
There are different options for doing this. A reviewer can express her
concerns in the “For editor only” textbox when submitting the review.
A reviewer can also use the “Add discussion” option in the system
(choosing the editor as the only other participant in the discussion)
during the review process, before submitting the final review
feedback. Using OJS for this type of communication also ensures that
suspicions of ethics violations are saved for future reference, rather
than being lost in an editor's email inbox.
Blind review technicalities
Of our ten peer-reviewed journals, two practice single-blind review,
and eight – double-blind review. The fact that the identities of the
reviewers and the authors cannot be disclosed to the other party
means more things for authors, reviewers and editors to remember.
An author may inadvertently reveal her identity in the manuscript,
replication data, the Conflict of Interest statement, and by making a
copy of the manuscript available on a preprint server before
submission. A reviewer may slip up and identify herself in the review
report. Both author and reviewer may leave identifying traces in file
properties and comments. OJS has built-in functionality for ensuring
anonymity of reviewers and authors: when the blind or double-blind
review option is chosen in the set-up of the review workflow, the
system controls that authors and reviewers do not get access to
identifying metadata. In addition, authors and reviewers are
presented with the link to “Ensuring blind review” guidelines (pre-set
by the OJS developers) at the relevant stages of the publishing
process. Journals should also provide information about how to
ensure anonymity in their Author Guidelines and Reviewer
Guidelines, and set up the submission and review process in a way
that precludes unwanted identification.
The author needs to make sure that there is no identifying
information in the manuscript. This includes names and affiliations,
acknowledgements, and the phrasing of self-citations. If a journal
provides submitting authors with article templates, it may be
preferable not to include the field for author name – so that there is
one less thing for an editor to remember to take care of before sending
a manuscript off to a reviewer. Authors – and also reviewers, if they
are uploading any documents as part of their review reports – must
remember to remove identifying information from file properties (the
“last edited by” and “author” fields) and from any comments that have
can use the service (and the ones indexed in the DOAJ have to use it), and
the library covers the Similarity Check fee for the journals. It is the editors’
responsibility to run the submitted manuscripts through plagiarism check
before reviewers are assigned.
Support for Good Peer Review in OJS-based Journals
Ravnetrykk 39, 2020
- 58 -
been made using the “insert comment” functionality in their text-
processing program.
If replication data are submitted to a data repository and made
available to the journal’s reviewers, the author needs to inform the
data repository curators that the dataset is meant to be made
available to reviewers in a journal that practices double-blind peer
review. The curators will then take the necessary precautions, e.g. by
providing reviewers with a copy of the dataset that does not contain
identifying information in its metadata or ReadMe-file.
When submitting to a journal using double-blind reviews, authors
need to be careful not to include the Conflict of Interest (CoI)
statement in the manuscript. It is advised that the full statement is
sent as a Comment to the Editor during the submission in OJS, i.e.
outside of the manuscript. The author should also provide a short
version of the CoI (e.g. only stating whether there is a conflict of
interest) in the Comment to the Editor (which can be forwarded to the
reviewer by the editor) or included in the submitted manuscript (to
be substituted by the full CoI before publication).8
Ensuring double-blind review may be challenging if the author's
manuscript is openly available as a preprint. A journal's editorial
team need to think in advance about their position on this issue: do
they want to accept manuscripts for which preprints are available
somewhere? The answer to this question should be “yes”: preprints
contribute to the transparency and efficiency of scholarly
communication, and there are fewer and fewer journals who have
restrictive policies with regard to preprints. There is then not much
for a journal to do except inform its authors and reviewers (in Author
Guidelines and Reviewer Guidelines) that preprint availability is not
a tragedy – even if the journal standardly operates with double-blind
review. The author has to accept that double-blind procedure will not
be guaranteed in this case – and reviewers should not reject an
invitation to review a manuscript merely due to the availability of a
preprint.
A reviewer may accidentally reveal her identity in the review report,
e.g. by signing her name or by referring to something that can identify
her. In OJS, the default, free form for a review report is separated into
a field that is addressed to both editor and author, and another one
that is addressed to the editor only. When forwarding a review report
to an author, an editor should ensure that no identifying information
is included in the forwarded part. A pre-defined review form – that we
mentioned above in connection with appropriate review feedback –
reduces the risk of identifying information being included in a review
report. A review form does not contain the fields addressed to the
8 A reviewer’s CoI statement will not be visible to the author, as long as the
review process is marked as blind or double-blind in OJS.
UiT goes open: Et festlig skrift til Stein Høydalsvik
Ravnetrykk 39, 2020
- 59 -
author and/or the editor – so there is less possibility that a reviewer
may confuse the fields. Also, a review form – as it consists of a set of
predefined questions and statements – may help to keep reviewers
focused.
To sum up
There are standards and technicalities in the peer review process that
authors, reviewers and editors need to be aware of, be able to comply
with and be reminded of. The editor has a central role in monitoring
adherence to ethical standards and anonymization technicalities.
A library publishing service can help the editors make the review
process as smooth as possible, with a focus on the importance of
adhering to ethical standards and ensuring the anonymity of the
review process. The library should offer training in the use of OJS to
the editors and may consider, as in our case, compiling a set of
editorial resources. Producing editorial resources specific to your
publishing service may seem unnecessary when there are good user
guides made available by OJS and COPE. However, workflows differ
– on publisher and journal level – so it may be useful for editors to
have access to step-by-step guides that are compiled specifically for
their needs.
When the technical part – including easily accessible guidelines – is
set up properly, there are fewer possibilities for authors and reviewers
to make mistakes and the review process is standardised for all
submissions. This, in turn, improves the overall quality of the review
process. A good quality of the peer review process from start to finish
is something all journals should strive for, and, as shown here, a
library publishing service can play an important role in achieving
this.