Peering into Peer-Review

Cincinnati Children's Hospital Medical Center, Cincinnati, OH 45229, USA.
The Journal of pediatrics (Impact Factor: 3.79). 03/2011; 159(1):150-1. DOI: 10.1016/j.jpeds.2011.02.012
Source: PubMed
2 Reads
  • [Show abstract] [Hide abstract]
    ABSTRACT: The Journal of the American Society of Nephrology (JASN) gives authors submitting original research the option of suggesting qualified reviewers or those they wish to exclude. This historical habit often leaves us wondering whether author preferences correlate with reviewer recommendations and whether differences related to reviewer selection affect decisions by editors. In a self-study presented here, we found that author-suggested reviewers, as a group, make more positive recommendations than editor-suggested reviewers (P = 0.01), although the difference disappears when recommendations are compared with those of editor-suggested reviewers of the same manuscript (P = 0.081). The distribution of recommendations by author-excluded reviewers, as a group, did not differ from those by editor-suggested reviewers; however, author-excluded reviewers impart significantly more negative recommendations than other reviewers of the same manuscript (P = 0.029). We further explored whether such differences result from individual reviewer tendencies to give generally more positive or more negative recommendations than editor-suggested reviewers and found no such tendency. Finally, editorial decisions on manuscripts reviewed by author-suggested or author-excluded reviewers do not differ from those decisions on manuscripts assigned but not reviewed by them. JASN's policy of editors making decisions independent from individual reviewer recommendations minimizes the effect of selection bias on publication decisions.
    Journal of the American Society of Nephrology 08/2011; 22(9):1598-602. DOI:10.1681/ASN.2011070643 · 9.34 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Abstract Objectives To assess whether reports from reviewers recommended by authors show a bias in quality and recommendation for editorial decision, compared with reviewers suggested by other parties, and whether reviewer reports for journals operating on open or single-blind peer review models differ with regard to report quality and reviewer recommendations. Design Retrospective analysis of the quality of reviewer reports using an established Review Quality Instrument, and analysis of reviewer recommendations and author satisfaction surveys. Setting BioMed Central biology and medical journals. BMC Infectious Diseases and BMC Microbiology are similar in size, rejection rates, impact factors and editorial processes, but the former uses open peer review while the latter uses single-blind peer review. The Journal of Inflammation has operated under both peer review models. Sample Two hundred reviewer reports submitted to BMC Infectious Diseases, 200 reviewer reports submitted to BMC Microbiology and 400 reviewer reports submitted to the Journal of Inflammation. Results For each journal, author-suggested reviewers provided reports of comparable quality to non-author-suggested reviewers, but were significantly more likely to recommend acceptance, irrespective of the peer review model (p<0.0001 for BMC Infectious Diseases, BMC Microbiology and the Journal of Inflammation). For BMC Infectious Diseases, the overall quality of reviewer reports measured by the Review Quality Instrument was 5% higher than for BMC Microbiology (p=0.042). For the Journal of Inflammation, the quality of reports was the same irrespective of the peer review model used. Conclusions Reviewers suggested by authors provide reports of comparable quality to non-author-suggested reviewers, but are significantly more likely to recommend acceptance. Open peer review reports for BMC Infectious Diseases were of higher quality than single-blind reports for BMC Microbiology. There was no difference in quality of peer review in the Journal of Inflammation under open peer review compared with single blind.
    BMJ Open 09/2015; 5(9):e008707. DOI:10.1136/bmjopen-2015-008707 · 2.27 Impact Factor