1. Sign our petition calling on Cochrane to withdraw their review of Exercise Therapy for CFS here.
    Dismiss Notice
  2. Guest, the 'News in Brief' for the week beginning 18th March 2024 is here.
    Dismiss Notice
  3. Welcome! To read the Core Purpose and Values of our forum, click here.
    Dismiss Notice

Risk of bias assessments for selective reporting were inadequate in the majority of Cochrane reviews, 2019, Saric et al

Discussion in 'Research methodology news and research' started by Andy, Feb 11, 2022.

  1. Andy

    Andy Committee Member

    Messages:
    21,810
    Location:
    Hampshire, UK
    Abstract

    Objectives
    The aim of the study was to analyze adequacy of risk of bias (RoB) judgments for selective reporting in Cochrane systematic reviews.

    Study Design and Setting
    We extracted RoB assessments, including judgment (low, high, or unclear risk) and supporting comment from Cochrane reviews of randomized controlled trials using computer parser. We analyzed sources of information mentioned in supporting comments. We compared judgments of Cochrane authors with guidance from the Cochrane Handbook for Systematic Reviews of Interventions (Cochrane Handbook) and categorized them into adequate or inadequate.

    Results

    At least 60% of judgments for risk of selective reporting bias of trials in analyzed Cochrane reviews were not in line with the Cochrane Handbook. Few Cochrane authors mentioned the trial protocol as a source of data for assessing selective reporting. Most of the inadequate judgments were made among trials that were judged with low risk of selective reporting bias; more than 90%. In 9% of analyzed RoB tables, Cochrane authors did not use this RoB domain at all.

    Conclusion

    Cochrane authors frequently make RoB judgments about selective reporting that are not in line with Cochrane Handbook and not mentioning trial protocol. Interventions aimed at helping Cochrane authors to make adequate RoB assessments in Cochrane reviews would be beneficial.

    Paywall, https://www.jclinepi.com/article/S0895-4356(18)31135-1/fulltext
     
  2. Caroline Struthers

    Caroline Struthers Senior Member (Voting Rights)

    Messages:
    821
    Location:
    Oxford UK
    This is from three years ago...!
     
    FMMM1, Mithriel, MeSci and 6 others like this.
  3. rvallee

    rvallee Senior Member (Voting Rights)

    Messages:
    12,299
    Location:
    Canada
    Who will police Cochrane the police Cochrane?

    Since this was 3 years ago, it looks like it's their Neptunian division, which is only available on the 5th Tuesday of odd bissextile years during the month of Smarch. Also all mail is immediately forwarded to the core of the planet.
     
  4. FMMM1

    FMMM1 Senior Member (Voting Rights)

    Messages:
    2,595
    Only glanced at this but perhaps the the problem is that the Cochrane authors don't care enough to properly assess RoB - or maybe they have a conflict of interest. Either way it questions NICEs collaboration with Cochrane --- might check but I can't recall that the risk of bias was clearly identified in the business case, re NICE/Cochrane collaboration.

    Also, if the Cochrane reviews can't be relied on then, as per rvallee's comment, NICE ---
    So much the the "savings" identified in the NICE business case.

    Then there's the "living guideline" goal, if NICE have to review the Cochrane review then will (reliable) reviews be available any sooner?
     
    bobbler, rvallee and Peter Trewhitt like this.

Share This Page