Risk of bias assessments for selective reporting were inadequate in the majority of Cochrane reviews, 2019, Saric et al

Andy

Andy

Senior Member (Voting rights)
Abstract

Objectives
The aim of the study was to analyze adequacy of risk of bias (RoB) judgments for selective reporting in Cochrane systematic reviews.

Study Design and Setting
We extracted RoB assessments, including judgment (low, high, or unclear risk) and supporting comment from Cochrane reviews of randomized controlled trials using computer parser. We analyzed sources of information mentioned in supporting comments. We compared judgments of Cochrane authors with guidance from the Cochrane Handbook for Systematic Reviews of Interventions (Cochrane Handbook) and categorized them into adequate or inadequate.

Results
At least 60% of judgments for risk of selective reporting bias of trials in analyzed Cochrane reviews were not in line with the Cochrane Handbook. Few Cochrane authors mentioned the trial protocol as a source of data for assessing selective reporting. Most of the inadequate judgments were made among trials that were judged with low risk of selective reporting bias; more than 90%. In 9% of analyzed RoB tables, Cochrane authors did not use this RoB domain at all.

Conclusion
Cochrane authors frequently make RoB judgments about selective reporting that are not in line with Cochrane Handbook and not mentioning trial protocol. Interventions aimed at helping Cochrane authors to make adequate RoB assessments in Cochrane reviews would be beneficial.

Paywall, https://www.jclinepi.com/article/S0895-4356(18)31135-1/fulltext
 
Andy said:
Abstract

Objectives
The aim of the study was to analyze adequacy of risk of bias (RoB) judgments for selective reporting in Cochrane systematic reviews.

Study Design and Setting
We extracted RoB assessments, including judgment (low, high, or unclear risk) and supporting comment from Cochrane reviews of randomized controlled trials using computer parser. We analyzed sources of information mentioned in supporting comments. We compared judgments of Cochrane authors with guidance from the Cochrane Handbook for Systematic Reviews of Interventions (Cochrane Handbook) and categorized them into adequate or inadequate.

Results
At least 60% of judgments for risk of selective reporting bias of trials in analyzed Cochrane reviews were not in line with the Cochrane Handbook. Few Cochrane authors mentioned the trial protocol as a source of data for assessing selective reporting. Most of the inadequate judgments were made among trials that were judged with low risk of selective reporting bias; more than 90%. In 9% of analyzed RoB tables, Cochrane authors did not use this RoB domain at all.

Conclusion
Cochrane authors frequently make RoB judgments about selective reporting that are not in line with Cochrane Handbook and not mentioning trial protocol. Interventions aimed at helping Cochrane authors to make adequate RoB assessments in Cochrane reviews would be beneficial.

Paywall, https://www.jclinepi.com/article/S0895-4356(18)31135-1/fulltext
Click to expand...
This is from three years ago...!
 
Andy said:
Interventions aimed at helping Cochrane authors to make adequate RoB assessments in Cochrane reviews would be beneficial.
Click to expand...

Only glanced at this but perhaps the the problem is that the Cochrane authors don't care enough to properly assess RoB - or maybe they have a conflict of interest. Either way it questions NICEs collaboration with Cochrane --- might check but I can't recall that the risk of bias was clearly identified in the business case, re NICE/Cochrane collaboration.

Also, if the Cochrane reviews can't be relied on then, as per rvallee's comment, NICE ---
rvallee said:
Who will police Cochrane the police Cochrane?
Click to expand...
So much the the "savings" identified in the NICE business case.

Then there's the "living guideline" goal, if NICE have to review the Cochrane review then will (reliable) reviews be available any sooner?
 
You must log in or register to reply here.
Back
Top Bottom