How effective is the journal peer-review process in minimizing bias in industry-sponsored (IS) trials?

From the Guidelines

The journal peer-review process is only moderately effective at minimizing bias in industry-sponsored trials, with significant limitations in detecting and addressing systematic biases. The process of peer review often fails to identify important methodological flaws, selective outcome reporting, and publication bias in industry-sponsored research, as highlighted by studies such as 1 and 1. Several studies have shown that industry-sponsored trials are more likely to report favorable outcomes for sponsor products compared to independently funded studies, even after peer review, with a systematic review finding that research funded by the pharmaceutical industry had four times the odds of having outcomes favoring the sponsor than research funded by other sources 1. Reviewers typically lack access to raw data, detailed protocols, or statistical analysis plans, making it difficult to detect manipulation of analyses or selective reporting. Additionally, peer reviewers themselves may have conflicts of interest or limited expertise in detecting sophisticated forms of bias. The process is further hampered by time constraints, inconsistent reviewer quality, and the inability to verify adherence to pre-specified protocols.

Some key points to consider in this context include:

• The importance of transparency in reporting conflicts of interest and funding sources, as emphasized by guidelines such as those outlined in 1.
• The need for mandatory trial registration and data sharing requirements to enhance the integrity of industry-sponsored research.
• The potential benefits of independent statistical analyses in reducing bias and ensuring the accuracy of research findings.
• The limitations of peer review in detecting and addressing bias, and the need for a broader system of checks and balances to ensure research integrity.

Given these considerations, it is essential to view peer review as just one component in a broader system needed to ensure research integrity in industry-sponsored clinical trials, and to implement additional measures to minimize bias and ensure the accuracy and reliability of research findings, as supported by evidence from studies such as 1 and 1.

From the Research

Effectiveness of Journal Peer-Review Process in Minimizing Bias

The journal peer-review process is a crucial component of scientific publishing, aiming to ensure the quality and validity of research findings. However, its effectiveness in minimizing bias, particularly in industry-sponsored trials, is a topic of ongoing debate.

• The peer-review process can introduce bias, and various types of bias and confounding can affect a study before, during, or after the intervention has been delivered 2.
• A double-blind peer review system is supported on equipoise and fair-play principles, and triple- and quadruple-blind systems have also been described, although they are not commonly used 2.
• The open peer review system introduces "Skin in the Game" heuristic principles for both authors and reviewers and has a small favorable effect on the quality of published reports 2.
• Peer review has been defined as a process of subjecting an author's scholarly work, research, or ideas to the scrutiny of others who are experts in the same field, functioning to encourage authors to meet the accepted high standards of their discipline 3.
• Despite its wide-spread use by most journals, the peer review process has also been widely criticized due to the slowness of the process to publish new findings and due to perceived bias by the editors and/or reviewers 3.
• Peer review bias is one of the major focuses of today's scientific assessment of the literature, and various types of peer review bias include content-based bias, confirmation bias, bias due to conservatism, bias against interdisciplinary research, publication bias, and the bias of conflicts of interest 4.
• The peer review process would benefit from various changes and improvements, with appropriate training of reviewers to provide quality reviews to maintain the quality and integrity of research without bias 4.
• An appropriate, transparent peer review is not only ideal, but necessary for the future to facilitate scientific progress 4, 5.

Limitations and Challenges

• Unblinded peer review might suffer from positive or negative bias towards certain authors, specialties, and institutions 4.
• Peer review can also suffer when editors and/or reviewers might be unable to understand the contents of the submitted manuscript, resulting in an inability to detect major flaws or revelations of major flaws after acceptance of publication by the editors 4.
• Other concerns include potentially long delays in publication and challenges uncovering plagiarism, duplication, corruption, and scientific misconduct 4.

Improving the Peer-Review Process

• Improving the peer review process is crucial to maintain the quality and integrity of research, and various strategies can be implemented to mitigate bias, such as double-blind peer review, open peer review, and training of reviewers 2, 3, 4.
• The peer review process provides a foundation for the credibility of scientific findings in medicine, and journal editors should arrive at a decision on submitted manuscripts based on the quality of the research and the validity of the findings 5.