AI-Assisted Peer Review in Rehabilitation Research: Opportunities, Blind Spots, and the Need for Human Verification

Abstract

Peer review is straining under expanding submission volumes and reviewer fatigue, and journals are increasingly testing whether large language models (LLMs) can help, But the evidence so far supports a narrow role of it. An AI model in a sports medicine journal performed just better than chance at matching final editorial decisions. However, when it flagged a manuscript for rejection, it’s accuracy was 90.5% of the time, proving as a screening tool rather than a final decision maker.¹ When tested on RCT bias assessments, AI models agreed with each other 91% of the time but differed sharply from human experts on complex, interpretive domains like blinding and allocation concealment.² One promising approach is a human AI hybrid model, in which AI performs rapid screening and consistency checks under clearly defined protections, while human experts retain ultimate responsibility for scientific judgment and decision making.³

Where AI genuinely helps. While AI struggles with complex decisions and interpretive analysis, it excels at narrow tasks like screening rejections and maintaining consistency, a role currently being tested in a formal systematic review trial.⁴ AI can sort submissions before they reach a human desk,¹ and it can reduce the burden of drafting and editing.⁵

Where it fails. Artificial intelligence struggles with rehabilitation research due to its inability to assess context and clinical judgment. Technology cannot reliably determine whether trial populations and settings translate to routine outpatient care or if statistical results are functionally meaningful for patients,^{1, 2} AI frequently fabricates academic sources despite citing a reference for every single claim. When two leading AI Models were evaluated on academic writing tasks, one model made up 36.7% of its citations, and the other fabricated 2.6%.⁶ Relying on AI for peer review introduces severe risks of fabrication and confidentiality breaches. Reviewers who use AI tools often violate journal policies by uploading confidential manuscripts and neglect their professional duties by outsourcing their judgment. Ironically, articles warning about AI flaws can end up containing invented references themselves.³

Toward responsible use. In a study, anesthesia and pain-medicine chief editors established strict boundaries for AI use: AI may assist with language editing, summarizing, and organization, but are strictly banned from generating references, data, conclusions, or editorial verdicts and any AI assistance must be fully disclosed.⁷ Reviewers welcome AI's efficiency but demand strict, explicit governance over informal norms. While they appreciate the reduced workload, they remain highly cautious regarding bias, transparency, and data privacy.⁸ Narrative reviews mandate treating AI as an augmentative tool rather than a replacement in scientific writing. These evaluations conclude that integrating AI requires structured training and ensuring a named human remains legally and academically accountable for every claim.⁵

Rehabilitation journals must prioritize equity because algorithmic bias and infrastructure gaps actively skew research data, distorting which populations and interventions are accurately modeled. This systemic distortion directly undermines the evidence base that under-resourced regions rely upon for rehabilitation research.

Keywords: Artificial intelligence, Clinical significance Human verification, Peer review Rehabilitation research, Research integrity.

Conclusion

AI serves well for sorting but fails as a final judge. It should flag and draft content, but never decide, self-verify, or invent evidence. Rehabilitation journals must embed these strict boundaries into reviewer guidelines and disclosure policies to strengthen the field rather than erode it.

DOI: 10.53389/JRCRS.2026140301