Cochrane ME/CFS GET review temporarily withdrawn

Jonathan Edwards said:
My understanding is that she was a physiotherapist. I doubt she had previous experience in designing trials.
Click to expand...

Thanks. Do you know if it is normal practice for Cochrane to choose someone who isn't a medical doctor and who has no or little previous experience in clinical trials to lead such a review?

Edit: And surely anyone leading such a review should have a considerable knowledge of statistics?
 
Last edited:
Daisymay said:
Thanks. Do you know if it is normal practice for Cochrane to choose someone who isn't a medical doctor and who has no or little previous experience in clinical trials to lead such a review?
Click to expand...

My understanding is that Cochrane does not so much choose someone as someone offers and Cochrane approves. I wrote a review on treatment of heel pain. The review team was led by a podiatrist who had worked with me. There was only one reasonably well structured trial - which we had done. The podiatrist had not done other trails but was highly intelligent and critical.

So it is quite normal for people to do reviews of their own work and I do not think there is any requirement for leads to be doctors.

One of the things I think recent events of various sorts has brought to light is that although Cochrane by and large seems to be a well meaning project with some quality control there is absolutely no guarantee that any articular review is any good. Or that doing reviews like this is a particularly useful thing anyway. By and large you only need systematic reviews in areas where things are not very clear. And it might be best just to admit that things are not very clear and do some definitive studies.
 
Jonathan Edwards said:
My understanding is that Cochrane does not so much choose someone as someone offers and Cochrane approves. I wrote a review on treatment of heel pain. The review team was led by a podiatrist who had worked with me. There was only one reasonably well structured trial - which we had done. The podiatrist had not done other trails but was highly intelligent and critical.

So it is quite normal for people to do reviews of their own work and I do not think there is any requirement for leads to be doctors.

One of the things I think recent events of various sorts has brought to light is that although Cochrane by and large seems to be a well meaning project with some quality control there is absolutely no guarantee that any articular review is any good. Or that doing reviews like this is a particularly useful thing anyway. By and large you only need systematic reviews in areas where things are not very clear. And it might be best just to admit that things are not very clear and do some definitive studies.
Click to expand...

Thanks for the explanation.

You said that "....Cochrane does not so much choose someone as someone offers and Cochrane approves " does that not leave things very open to manipulation especially in an area of contention?

I find that quite shocking actually, surely any supposed gold standard review, whatever the area of medicine, should be seen to be scrupulously beyond such possibilities of manipulation?
 
It's apparent that I've misunderstood how Cochrane works.
Daisymay said:
Thanks for the explanation.

You said that "....Cochrane does not so much choose someone as someone offers and Cochrane approves " does that not leave things very open to manipulation especially in an area of contention?

I find that quite shocking actually, surely any supposed gold standard review, whatever the area of medicine, should be seen to be scrupulously beyond such possibilities of manipulation?
Click to expand...
This.
 
Daisymay said:
"....Cochrane does not so much choose someone as someone offers and Cochrane approves " does that not leave things very open to manipulation especially in an area of contention?
Click to expand...

Er, yes. Everyone should have remembered that all along. Maybe some people did remember it and made use of it.

Of course if Cochrane did the choosing things might be much worse - since Cochrane was probably set up by a group of GPs and other anti-Pharma enthusiasts who were probably as biased as sin.

It all boils down to peer review, which is a broken system too. Fortunately, there is some sign that it might be working a bit better recently.
 
Jonathan Edwards said:
My understanding is that Cochrane does not so much choose someone as someone offers and Cochrane approves.
Click to expand...

Yes, the process is explained here, it's the authors who choose an available topic:

https://community.cochrane.org/revi...roposing-and-registering-new-cochrane-reviews

Proposing and registering new reviews

What’s the process for writing a review?

All Cochrane systematic reviews are first registered as titles with a Cochrane Review Group (CRG). Registering your title is important to prevent duplication of effort with other authors, and to make sure your topic is appropriate for a Cochrane review. There are 52 CRGs that each focus on a particular area of health. CRGs are coordinated by an editorial team who edit and publish completed reviews in the Cochrane Library. Unlike other journals, your CRG will provide support and advice throughout the review process.

1. The author team chooses an available topic and registers a title with the appropriate CRG.
2. Authors prepare a protocol and submit it to the CRG for peer review and publication.
3. When the protocol has been approved, authors begin searching for studies and analysing their results.
4. Authors prepare the final review report and submit it for peer review and publication.
5. After the review is published, the authors continue to update it periodically.


So, did Larun et al decide for themselves, completely unsolicited, to do this review or was there any involvement from the the PACE authors/BPS brigade/MRC ?



 
Daisymay said:
Yes, the process is explained here, it's the authors who choose an available topic:

https://community.cochrane.org/revi...roposing-and-registering-new-cochrane-reviews

Proposing and registering new reviews

What’s the process for writing a review?

All Cochrane systematic reviews are first registered as titles with a Cochrane Review Group (CRG). Registering your title is important to prevent duplication of effort with other authors, and to make sure your topic is appropriate for a Cochrane review. There are 52 CRGs that each focus on a particular area of health. CRGs are coordinated by an editorial team who edit and publish completed reviews in the Cochrane Library. Unlike other journals, your CRG will provide support and advice throughout the review process.

1. The author team chooses an available topic and registers a title with the appropriate CRG.
2. Authors prepare a protocol and submit it to the CRG for peer review and publication.
3. When the protocol has been approved, authors begin searching for studies and analysing their results.
4. Authors prepare the final review report and submit it for peer review and publication.
5. After the review is published, the authors continue to update it periodically.


So, did Larun et al decide for themselves, completely unsolicited, to do this review or was there any involvement from the the PACE authors/BPS brigade/MRC ?
Click to expand...

I think that just about says it all. Why haven't we ever done the Cochrane review ourselves as a community?

Larun was obviously put up by the BPS crowd.
 
It seems that by 2011 Larun had already done a 'systematic review' of exercise in CFS, but using a technique called 'meta-ethnography', which seems to have more to do with feeling good than science. At least part of the study includes 'qualitative studies' which are essentially of no value in terms of efficacy evidence. The abstract seems to start with the assumption that exercise is useful and to conclude the same without any critical analysis of evidence.

Of note in her abstract she notes that the quality of documentation in the studies was moderate to low on the GRADE system.

At some point others joined her for a Cochrane review, including a member of Michael Sharpe's department. That would make it pretty impossible not to produce a favourable conclusion.
 
Jonathan Edwards said:
It seems that by 2011 Larun had already done a 'systematic review' of exercise in CFS, but using a technique called 'meta-ethnography', which seems to have more to do with feeling good than science. At least part of the study includes 'qualitative studies' which are essentially of no value in terms of efficacy evidence. The abstract seems to start with the assumption that exercise is useful and to conclude the same without any critical analysis of evidence.

Of note in her abstract she notes that the quality of documentation in the studies was moderate to low on the GRADE system.

At some point others joined her for a Cochrane review, including a member of Michael Sharpe's department. That would make it pretty impossible not to produce a favourable conclusion.
Click to expand...

Grief I hadn't realised one the authors is from MS's department, well there's a surprise......

I'm assuming this is acceptable procedure for Cochrane?

It seems quite appalling to me.
 
Daisymay said:
Grief I hadn't realised one the authors is from MS's department, well there's a surprise......

I'm assuming this is acceptable procedure for Cochrane?

It seems quite appalling to me.
Click to expand...

This thread is getting so long I'd forgotten I'd found that information too. I was horrified it could be allowed. Post #628 on this thread:

Trish said:
This has probably been discussed before, but I notice that the one person who has been involved with the different versions of the Cochrane reviews of Exercise Therapy for CFS all the way through from the first version in 2001 is Jonathan Price, who works in the Department of Psychiatry at Oxford University, so he is a colleague of Michael Sharpe.

I looked up Jonathan Price. He is listed as a clinical tutor in the Department of Psychiatry. He has published on various aspects of psychiatric treatment. This one is of interest:
https://www.medicinejournal.co.uk/article/S1357-3039(16)30195-5/fulltext
The note about him on that paper says:
''His interests include patient attitudes to treatment and cognitive behavioural treatments.''
Edit to add:
The paper I mention is:
Functional disorders and ‘medically unexplained physical symptoms’ published in 2016.
The abstract concludes:
If reassurance and simple techniques of reattribution of symptoms do not lead to symptom resolution, cognitive behavioural therapy and the use of antidepressants as neuromodulating agents (rather than as antidepressants per se) should be considered.
Click to expand...

So he is biased from the start.
This is a clear case demonstrating the importance of removing ME/CFS from the mental health section of Cochrane.
Click to expand...

Edited to add post number and re-insert quote within quote that didn't transfer.
 
Last edited:
Jonathan Edwards said:
At some point others joined her for a Cochrane review, including a member of Michael Sharpe's department. That would make it pretty impossible not to produce a favourable conclusion.
Click to expand...

I think it's actually the opposite. Price did the 2004 version: https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD003200.pub2/full

Larun then joined, and used the 2004 version of the basis/protocol for her review (although she then deviated from that protocol in the manner noted by Courtney).
 
Daisymay said:
Grief I hadn't realised one the authors is from MS's department, well there's a surprise......

I'm assuming this is acceptable procedure for Cochrane?
Click to expand...
Trish said:
This thread is getting so long I'd forgotten I'd found that information too. I was horrified it could be allowed. Post #628 on this thread:
Click to expand...
From what @Jonathan Edwards says, this is normal practice.
Jonathan Edwards said:
So it is quite normal for people to do reviews of their own work and I do not think there is any requirement for leads to be doctors.
Click to expand...
My response being ...
Barry said:
This surprises me. Marking one's own homework would seem susceptible to honest but very human bias at best, and decidedly dishonest and inhuman bias at worst.
Click to expand...
 
Jonathan Edwards said:
At least part of the study includes 'qualitative studies' which are essentially of no value in terms of efficacy evidence.
Click to expand...

Which is a shame, because that's why patient surveys are dismissed so readily (I know, I know, there are other issues too).

But all of these studies are still based on qualitative data. They've simply taken qualitative data (the questions in the SF-36, CFQ etc) and produced a number out of them, that they can then claim is 'quantitative'. Bingo! That it actually strips out all the interesting information is disregarded.

The reason that 'qualitative studies' are considered to be of 'no value' is because they are incredibly hard to analyse properly.
 
You must log in or register to reply here.
Back
Top Bottom