Can AI replace Human Peer Reviewer for scientific articles and manuscrits ?

Samir Touzani

Dear Professor,

Thank you for the interesting topic of discussion.

I am not an expert on AI. As far as I understand AI, it is a programmed method of doing something. Human peer reviewing scientific articles and manuscripts, is just not a science, but also is an art. If AI is used for this purpose, then that individuality (please read: art) will get lost. So, I would prefer to believe that AI cannot replace human peer review of scientific articles and manuscripts.

Best regards,

Anamitra.

AI was not consulted for this answer.

Today‘s AI is capable of mimicking scholarly feedback. AI grammar, syntax, flow and logic will be good enough. However AI does not actually “know” that it is reviewing academic research. It may not be capable of subtle (or even some obvious) observations that a human expert might make. The output would be flat and mediocre, but acceptable.

If in the near future strong AI is achieved, then the reviews will be at the highest level. But of course if AI is that strong then it will be both author and reviewer.

I think humans can be seduced by AI mimicry. There is quite possible even today that human reviewers will submit work that is actually processed by AI. If today’s weak AI becomes the norm for fast reviews, I think the quality of reviews will be no better than a review from a human subject matter expert.

Confidentiality concerns, if any, will be swept aside as AI utility improves.

Thank you All, George McIlvaine your first sentence motivated me to request ChatGPT to verify his answer about its capability for this , you can find the attached mp4 recorded discussion, so my question were : Are you able to review scientific manuscripts to be published in scientific journals authored by doctors, bachelors and engineers?

ChatGpt answer's were :

Yes, I can review and provide feedback on scientific manuscripts authored by professionals from various fields, including doctors, bachelors, engineers, and many others. Here's what I can do:

However, it's important to note the following:

If you'd like to proceed, you can share a section or the main points of your manuscript, and I'll do my best to help!

I agree with the responses above, particularly Dr Anamitra Roy that AI would lose the human aspect of peer reviewing.

I used to peer review nursing articles, and would try to help - if necessary - authors (particularly new authors) who had written well but needed a little bit of advice. When I saw the amended version, it was usually fit for publication. These authors were usually nurses and had knowledge and experience but had not yet become authors.

However, other authors had written fine articles but sometimes another pair of eyes looking at their submission, just added a little something. When I was peer reviewed, I often found this and was grateful for that extra bit of help with what I had felt was really good work on my part, but the peer reviewers just added that little bit extra.

Mary C R Wilson yes sure, peer reviews is necessary , thank you for sharing your experience about that, but AI will for sure make dramatic changes every where, i remember when internet became available to public, some one telling us internet is a phone showing pages, now we can see how internet transformed all process every where, AI will do that soon in the next two (2) ot three (3) decades, and i m optimistic about that despite the fear expressed many times, i think we should adapt the scientific methodology to have AI as part of it like reasoning and experimentation.

No, at current level of AI , it is not possible. But, In the future , AI Technology with consciousness may replace human peer reviewer.

Tools like ChatGPT can help, but transparency is vital, say Mohammad Hosseini and Serge Horbach

In May, Northwestern University, Chicago researcher Mohammad Hosseini and Aarhus University, Denmark researcher Serge Horbach took a closer look at how generative AI programmes like Chat GPT could change peer review. As the technology develops at breakneck speed, their exploration offers a guide to the positives and negatives of using AI...

The integration of AI tools into the peer review process can be beneficial in assisting with certain tasks such as language editing and conflict of interest detection. However, the use of AI tools must be continually evaluated and responsibly implemented to ensure that they are not perpetuating biases or impacting the quality and reliability of scholarly literature. The expertise and judgment of human reviewers will always be essential in ensuring the rigor and dependability of the peer review process, and the continued integration of AI tools should be viewed as a complementary tool rather than a replacement...

Generative AI’s potential role in peer review is complex, with the capacity for great time-saving efficiency as well as for severe ethical violations and misinformation. In theory, generative AI platforms could be used throughout the peer review process, from the initial drafting to the finalization of a decision letter or a reviewer’s critiques. An editor or reviewer could input a manuscript (either in whole or individual sections) into a generative AI platform and then prompt the tool for either an overall review of the paper or for a specific analysis, such as evaluating the reproducibility of the article’s methods or the language clarity. However, this comes with a myriad of potential benefits and drawbacks...

Some journals have proposed adopting generative AI tools to augment the current peer review process and to automate some processes that are currently completed by editors or reviewers, which could meaningfully shorten the time required to complete a thorough peer review. Currently, few publishers have posted public position statements regarding the use of generative AI during peer review; an exception is Elsevier, who has stated that book and commissioned content reviewers are not permitted to use generative AI due to confidentiality concerns. The future of generative AI integration into journals’ manuscript evaluation workflows remains unclear...

The advent of easily accessible large scale natural language processing tools like ChatGPT is opening a new realm of ethical and practical considerations. Non-article research outputs like data, methods, and code are gaining prominence, evolving from nice-to-have supporting documentation to citable published artifacts, formally preserved in the scientific record. Peer reviewers face increasing demands on their time and expertise, making it more challenging to secure reviewers. As it did in so many other areas, the pandemic has accelerated that trend—and that is just the beginning.

But in spite of these seismic shifts, peer review itself remains largely unchanged, both in its value to the scholarly community and its day-to-day practice at journals. Peer review is the primary way that journals evaluate the rigor, credibility, and potential interest of research submitted for publication consideration. What does the changing publishing landscape mean for the practice of peer review, and for peer reviewers themselves?

Ljubomir Jacić i think peerreviewer will have AI as powerful tool to accelerate demands treatment, this pushs to peer reviewers to learn the prompts to leverage AI advantages, as you said demands increased, rigor can be verified primarily by AI like math proofs for example and interest as well using prompts. The change i think is the reviewers will use the AI prompt , special extensions during the review process.

There are beneficial uses for AI, if done carefully, like automating various aspects of the process — matching manuscripts with the right reviewers, identifying potential ethical issues, assessing the language quality and writing. All these things can be done reliably with AI now, and they can increase efficiency and take those tasks away from editors and reviewers, to allow them to focus on the science.

There's also the possibility that AI becomes so good that it actually can do peer review. Of course, nobody believes that right now, but we also didn't believe that open AI would be at the stage it is today. ChatGPT is passing college exams.

The challenge, though, is that AI algorithms can inherit biases from the data they're trained on. It could lead to even more bias, like biased reviewer recommendations. We have to ensure we're making efforts to eliminate that and reduce unintended bias.

There are also ethical considerations around privacy and data security and transparency. Authors and reviewers need to be aware of how their data is being used and who has access to it.

And there are some things AI tools are still not capable of doing — evaluation that you need human judgment for. AI algorithms can't yet determine what's novel or groundbreaking. They’ve been trained on existing research, and it's new discoveries we're looking for...

Can AI replace Human Peer Reviewer for scientific articles and manuscrits? No, not at all. Al cannot replace the totality of human potential and experience in reviewing scientific manuscripts.

Overview: Artificial intelligence and machine learning software are developed to catch common errors or shortcomings, allowing peer reviewers to focus on more conceptually-based criticism, such as the paper’s novelty, rigor, and potential impact. This strategy is more widely seen in humanities and social sciences research.

Pros: Increases efficient use of peer reviewers’ time; improves standardization of review; can automate processes like copyediting or formatting

Cons: Requires extensive upfront cost and development time as well as ongoing maintenance; prone to unintentional bias; ethically dubious; requires human oversight...

"Fixed knowledge differs from the ever-evolving living knowledge that humans engage with critically when reviewing and critiquing articles and publications. Artificial intelligence cannot substitute for humans in such a task."

We hear so often about the things that are broken in peer review. Not enough reviewers, slow turnaround times, and imperfect measures of impact. Rarely do we raise our glasses to the things that are going well, or that have greatly improved in peer review. However, as we move into an environment assisted by AI, being aware of what is going well and what we should celebrate enhances our motivation to get things done...

Peer review is having a bit of a renaissance, especially with the growth of papermills, predatory publishing, compromised peer review, and a growing list of retractions. While it is not perfect, peer review is one of the best processes we have to reduce the amount of junk science that gets published!

On the humanity of peer review - the importance of people in the process of creating, reviewing, and publishing research.

"While systems, tools, and standards can provide helpful (and often essential) support, it’s the contributions of individuals that will be most critical to the future of both peer review and publishing..."

Peer reviewers are not paid, which is not fair. It is just about altruism of peer reviewers.

The following article offer the help of AI:

The only way to make the situation fairer is by ending the human-dependent review system. We should invest more in AI-related components of journal review system, and gradually move away from the current human-dependent one. The arguments against the AI-dependent review are that AI can’t do critical thinking like humans and is based on algorithms that are frequently biased. We need to properly train AI to reduce the prevailing algorithmic limitations. Reviews of the existing AI-run article review software and models show certain degrees of effectiveness and efficiency, but that they are not quite ready to replace the human reviewers...

Interesting Proposal Prof. Samir Touzani

No human peer review and I will add, An AI test on text similarity and on vocabulary meaning, to make it robust.

I agree with Prof. Ljubomir Jacić statement.

Kind Regards.

Using AI as an assistant is a growing trend among academic editors, as journals field more submissions while tapping a depleting well of peer reviewers. In this reality, an AI tool that can quickly identify whether a paper’s subject matter falls within a journal’s scope, or can expeditiously find potential peer reviewers with relevant expertise, can be valuable...

Trust, after all, is the metaphorical Everest for artificial intelligence...

I agree with Murtala Ismail Adakawa

It depends on the discipline. I have always been published in journals for nurses or carers and I think it is important that the peer reviewer has insight into what is necessary for articles related to caring for people.

Dear Samir Touzani, I fully agree with Ljubomir Jacić. I would like to add that perhaps AI, being objective in reviewing, could not consider the fact that many journals to peer reviewers strongly recommend helping authors make the manuscript publishable. Therefore, perhaps journals could lose a few percentage points of their earnings but scientific literature would gain a lot.

The Peer Review Renaissance: An Urgent Call for Transformation

How can authors and reviewers unite and work together as a team instead of being on opposite sides? How can reviewers have ongoing discussions and conversations with the author during the review process itself? And what measures can authors and reviewers take to harmonize their efforts with the common aim of refining the paper to its highest potential?

This peer review renaissance is not merely a destination but a journey of continual improvement. There will be a lot of failed experiments as the process evolves. But every experiment will take us one step closer to our goal of making the process more robust, efficient, diverse, and collaborative...

How can a combination of human expertise and AI make the peer review process more efficient? I envision a future where AI is not a threat or a cause for worry but a tool for enhancing efficiency...

The selection of an AI model is influenced by various aspects, including accuracy, availability, reliability, and ethics. What are your thoughts on improving the database? The AI system must provide evidence of its performance by achieving an R2 value of 1 and a MAPE value of 0.00%. Is this possible?

Generative artificial intelligence (AI) has the potential to transform many aspects of scholarly publishing. Authors, peer reviewers, and editors might use AI in a variety of ways, and those uses might augment their existing work or might instead be intended to replace it. We are editors of bioethics and humanities journals who have been contemplating the implications of this ongoing transformation. We believe that generative AI may pose a threat to the goals that animate our work but could also be valuable for achieving those goals. In the interests of fostering a wider conversation about how generative AI may be used, we have developed a preliminary set of recommendations for its use in scholarly publishing. We hope that the recommendations and rationales set out here will help the scholarly community navigate toward a deeper understanding of the strengths, limits, and challenges of AI for responsible scholarly work...

Milan Marija Mirkovic in this Example the ChatGpt plus is acting as data scientist better than humain

humain https://medium.com/@teja.btc07/data-science-made-super-easy-with-chatgpt-plus-83745cf4fb9b

Perhaps RG should use AI to prevent all the pointless answers from seeing the light of day.

And the deliberately faux-ignorant questions that are posted to boost the author's visibility.

James Garry Karl Pfeifer please post relevant answers , i may ignore simply your answers , but posting such answers is not useful for no purpose , good or bad, some funny answers would be acceptable.

Samir Touzani

Maybe you should learn to read between the lines. Or, to repurpose an old saying, peer review begins at home. Cleaning up or preempting the crap on RG would be an easy form of peer review. If even that small step isn't possible, then there is no hope for more nuanced peer review that requires recognition of creativity and insight, and yes, sometimes the ability to read between the lines.

Karl Pfeifer sorry the answer is not clear at all, there is no scientific statement that says "read between the lines", if we have something to say we say it that's all, I am not following up on this suspicious discussion.

Samir Touzani

Scientists and scholars don't always explicitly say what they have to say:

"This study gives some more examples of the benefits of AI. However, we are now at the stage where artificial intelligence can carry out tasks which require creativity and judgement, such as recommending acceptance or rejection, creating reviewer reports, and identifying cases of image manipulation, duplication, and plagiarism. This is where the ethical issues really come to the fore..."