The Guardian's Science Weekly podcast - 2 November 2018 - What role should the public play in science?

Does anyone have it stored or perhaps only the complaint by Bob?

 

I think there were probably good reasons why it was withdrawn. I was very uncomfortable with what JC wrote, and others were too.

 

AYFKM?!

The paper has not, as a matter of fact, been revised. In making a correction on basic factual errors they make yet another basic factual error. Come on! And on the subject of misleading science no less. They paint "activists" as being on the wrong side yet make multiple factual errors and only present the perspective of the very person accused of misleading with fraudulent research.

It was right there in the first article from Kelland, the direct quote from Tovey. What is this trash excuse of journalism? They need to do much more than this weak-ass excuse of a correction filed in the 4th basement on the moon of Pluto. They need to dedicate a full episode to hearing from Tuller or anyone willing to give the facts as a counterpoint to the misleading opinion of a controversial researcher accused of serious malpractice.

Yellow "journalism". For shame. And not even the basic decency of an apology, especially as yet another hit piece on the exact same misleading framing was published in the same newspaper.

 

Wasn't that about one of the Cochrane reviews?

 

Ah, but *which* moon? (I'm going for Nix)

 

Maybe, but I thought it cropped up regarding how they researched for the existing guideline. Confess I can't find it but I'll try digging a bit more. But my memory is not the best.

 

This is what I was thinking of:

https://www.nice.org.uk/guidance/cg53/documents/surveillance-review-proposal

It includes:

In fairness though I'm not sure if this was simply an initial screening process, from which to choose a shortlist to then fully read. If so then this may have been take out of context in documents such as:

http://www.investinme.org/Documents/NICE/Invest in ME Research Comments-to-NICE-Consultation-Guidelines-Review-2017.pdf
(see para 15).

If the short listed papers were not read in full, then that would be very bad, but not sure if that was the case or not.

 

I should have specified that too

I vaguely remember this but it's about 50:50 on either.

 

Then the study about spin in psych research abstracts suddenly becomes very relevant to the NICE committee. MAJOR relevance. This is seriously seriously bad considering the 2017 rubber-stamping of "nothing new happened between 2007 and 2018" and the dismissal of the judicial review. What a massive, well-documented failure.

I would assume that it was pointed out in the 2007 committee and for the judicial review that the research used to justify the chosen psychosocial model is weak and possibly misleading. Especially given the very relevant confirmation by one author of a significant number of those poor papers, one precisely accused of being a major source of misleading interpretation.

But considering how thoroughly the Cochrane reviews were demolished by Mark Vink's papers, it seems likely that the same approach was used, that no scrutiny was applied to the papers that were selected to support the predetermined conclusion. This is failure that is endemic to this entire area of research, one that is being given increased power and influence despite delivering precisely nothing of value, at great cost no less.

 

I've tried to write a short blog post about this: "The Guardian’s Science Weekly podcast and the Cochrane review." https://mecfsskeptic.wordpress.com/...ience-weekly-podcast-and-the-cochrane-review/

Feel free to point out errors, regarding both grammar and content.

 

The Guardian’s Science Weekly podcast and the Cochrane review

Yesterday, the Guardian’s Science Weekly podcast published an update of its episode titled: “What role should the public play in science?” [1] For those who might have missed it, the episode revolved around a Cochrane review on exercise therapy for chronic fatigue syndrome (CFS). [2] A Reuter’s news article had announced that a withdrawal of the review was imminent following a formal complaint to Cochrane’s editor in chief. [3]

The Science Weekly podcast gave the impression that the decision to withdraw the Cochrane review was due to pressure from activists. Unfortunately, none of the critics of the Cochrane review were interviewed on the program. Instead, two authors of the controversial PACE trial were allowed to present their view without counterargument. They said the news about Cochrane, if true, was “very alarming” and an example of evidence being revised “not by a change in science but by campaigns.” The update of the podcast seems to admit that this narrative was incorrect. In the added introduction, host Ian Sample, explains that “the details of the complaint have been made public and they show that it was based in science.”

In their defense, the Science Weekly team seems to argue that they could not have known this at the time. The update explains:

“…in the podcast we talk about a paper on CFS interventions that appeared in the prestigious Cochrane database of systematic reviews. It was later temporarily removed after a complaint from the public. This was our starting point when looking at the role of public pressure when it comes to science and when we made the podcast we didn't know the exact nature of the complaints or who had made them. We asked, but we weren't told.”​

This argument is problematic for several reasons (for one, the Cochrane review was never removed, not even temporarily).

Let’s go back to when the podcast was aired, 2 November 2018, and see what was known at the time. The issues raised in the formal complaint were already mentioned in the comment section of the Cochrane review by the same person: the late Robert Courtney. Similar arguments were raised by Tom Kindlon. Their comments were published in the same document [3] as the Cochrane review, so any journalist who read the paper in full would have been aware of them. On 21 October 2018, renowned skeptic and health psychologist James Coyne published a blog post about Robert Courtney and the importance of his critique on the exercise review for CFS. [4] It included the cover letter and documents Courtney had used in his formal complaint to Cochrane. [5] In addition, detailed analysis and criticism of the Cochrane review were published by Vink & Vink-Niese in Health Psychology Open on 8 October 2018. [6] In other words, there was sufficient information to understand that valid criticism of the Cochrane review was being made.

It might be true that the Science Weekly podcast could not have known the exact nature or origin of the complaint Cochrane was responding to. But the issues raised in the complaint were already in the public domain. So it should have been evident to any investigative journalist that complaints about the review were based on sound scientific arguments. One could argue that uncertainty about the nature and origin of the complaint to Cochrane made it difficult to reach strong conclusions about the revision process. But in that case, one shouldn’t make a podcast accusing Cochrane and the ME/CFS patient community of damaging the integrity of science, either.

There is very little evidence that does support the narrative highlighted in the podcast. The main source for Cochrane giving in to a campaign of activists instead of scientific arguments is a Reuter’s news article that appeared on 17 October 2018. [3] The article suggested that Cochrane would temporarily withdraw the review due to pressure and complaints from online activists. Yet, when David Tovey, Cochrane’s editor in chief at the time, was contacted he stressed that “this is not about patient pressure.” [3] So why did the Guardian decide to do a podcast on this news story as an example of patient pressure overruling science?

The formal statement Cochrane gave to the Science Weekly podcast was as follows:

“The Cochrane review of exercise therapy for chronic fatigue syndrome was last updated in 2014 and is subject to an ongoing process of review and revision following the submission of a formal complaint to the Editor in Chief. Cochrane carefully considers all feedback and complaints and revises or updates reviews when appropriate to do so. The review author team have advised us that a resubmission of this review is imminent. A decision on the status of this review will be made once this resubmission has been through editorial process, which we anticipate will be towards the end of November 2018.”​

Notice there is no mention of withdrawal or pressure from campaigners. Instead, the statement speaks of a formal complaint, an ongoing review process, and an imminent resubmission. Despite the update, it remains unclear what information prompted the Science Weekly podcast to promote a narrative that conflicts with what Cochrane told them.

It is equally unclear why it took 9 months to issue this update. Cochrane already announced on 30 November 2018 that the review was being revised because of the complaint submitted by Robert Courtney (they mention him by name). [7] Journalist and Senior Fellow at Berkeley, David Tuller, had written about the details of Courtney’s complaint and confirmation of its validity by Cochrane on 14 March 2019. [8] Why did it take the science weekly podcast many more months to issue an update?

When a mistake is made, the right thing to do is to correct it. I doubt that many listeners of the podcast will check to see if an episode that was aired 9 months ago, has any updates or corrections. A more reasonable option would be to invite those who have criticized the Cochrane review on a new episode of the podcast so that they can explain to regular listeners that their complaints were indeed based on science.


References

[1] What role should the public play in science? - Science Weekly podcast. The Guardian, 7 August 2019. https://www.theguardian.com/science...public-play-in-science-science-weekly-podcast

[2] Larun L, Brurberg KG, Odgaard-Jensen J, Price JR. Exercise therapy for chronic fatigue syndrome. Cochrane Database Syst Rev. 2017 Apr 25;4:CD003200. https://doi.org/10.1002/14651858.CD003200.pub7

[3] [7] Kelland K. Exclusive: Science journal to withdraw chronic fatigue review amid patient activist complaints. Reuters. October 17, 2018. https://www.reuters.com/article/us-...mid-patient-activist-complaints-idUSKCN1MR2PI

[4] Coyne C. How to get a flawed systematic review and meta-analysis withdrawn from publication: a detailed example. Mind the Brain. 21 October 2018. https://web.archive.org/web/20190503184207/https://webcache.googleusercontent.com/search?q=cache:Qwoqlcv7C_MJ:https://mindthebrain.blog/2018/10/21/how-to-get-a-flawed-systematic-review-and-meta-analysis-withdrawn-from-publication-a-detailed-example/+&cd=1&hl=en&ct=clnk&gl=uk&client=firefox-b-d

[5] ME/CFS notes. Comments Submitted to the Cochrane Review of Exercise Therapy for Chronic Fatigue Syndrome. https://sites.google.com/site/mecfsnotes/

[6] Vink M, Vink-Niese A. Graded exercise therapy for myalgic encephalomyelitis/chronic fatigue syndrome is not effective and unsafe. Re-analysis of a Cochrane review. Health Psychol Open. 2018 Oct 8;5(2):2055102918805187. https://doi.org/10.1177/2055102918805187

[7] Cochrane Database of Systematic Reviews. Exercise therapy for chronic fatigue syndrome, see what’s new. https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD003200.pub7/information#whatsNew

[8] Tuller D. Trial By Error: Cochrane’s Report on Courtney’s Complaint. Virology Blog. 12 March 2019. http://www.virology.ws/2019/03/12/trial-by-error-cochranes-report-on-courtneys-complaint/

 

Nice job; maybe the bottom line should be something along the lines that the 'amended' review is due to be published by the end of next week.

So it remains to be seen whether the authors have adequately addressed the issues or if Cochrane have bowed to pressure by those (such as Professor Sharpe) who are objecting to the existing review being withdrawn.

 

I really think we should consider making a formal complaint to IPSO about this. What do others think? Will it do any good?

eta: although I guess that's exactly what they want us to do, so that they can continue to complain about us in the press.

 

I suspect that the regulations offer very little protection... otherwise the UK press wouldn't be so consistently terrible!