I am concerned about this too. However the fact it was withdrawn at all is still a positive thing in many ways, even if they use it to bullet-proof the study against criticism. Furthermore if they do this they risk severely damaging their credibility, at a time when Cochrane's credibility is already damaged.
-
New Forum Software Installed (Still a few issues but reopening the forum) More details.
Cochrane ME/CFS GET review temporarily withdrawn
- Thread starter Trish
- Start date
Yes, almost everything about PACE can be bad science except this. Its deliberate manipulation of data, given that PDW co-authored a paper in 2007 showing he knows the problems. In other research where manipulation has occurred the research has often been retracted immediately. Why not here?
(edited to add "about PACE")
JemPD
Senior Member (Voting Rights)
Thats very good to hear @Jonathan Edwards thank you for sharing
Jonathan Edwards
Senior Member (Voting Rights)
In retrospect, maybe the reason for withdrawing the Chinese Dim Sum review first lies in here somewhere.
And Larun's camp wouldn't have leaked this to the press in an effort to apply public pressure on Cochrane unless all other efforts to resolve this had failed. This is the ultimate hail mary. It's their highest card they can play at this juncture, and they know it's a low card.
Jonathan Edwards
Senior Member (Voting Rights)
It looks more and more as if someone has shot themselves somewhere above the foot.
Dolphin
Senior Member (Voting Rights)
One person challenged them on this in a published letter:Yes - because the first quote (from the 2006 protocol) is actually about improvement.
It's because they didn't understand what they were doing, that they ended up making the recovery criteria *less strict* (>60) than the criteria they had already set for improvement (>70), which is why they then had to change the improvement criteria too (increase of 8 pts - which I've just noticed they say is 0.5 SD - oh what a mess!).
[ETA: and that's 0.5SD of the SF36 at baseline - which is already a highly selected population]
https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(11)60689-2/fulltext
CORRESPONDENCE| VOLUME 377, ISSUE 9780, P1831, MAY 28, 2011
The PACE trial in chronic fatigue syndrome
Published:May 17, 2011DOI:https://doi.org/10.1016/S0140-6736(11)60689-2
In their randomised trial of treatments for patients with chonic fatigue syndrome, Peter White and colleagues (March 5, p 823)1
define a clinically useful difference between the means of the primary outcomes as “0·5 of the SD of these measures at baseline, equating to 2 points for Chalder fatigue questionnaire and 8 points for short form-36”. They cite achieving a mean clinically useful difference in the graded exercise therapy or cognitive behaviour therapy groups, compared with specialist medical care alone, as evidence that these interventions are “moderately effective treatments”.
The source for this definition of clinically useful difference states that such a method has a “fundamental limitation”: “estimates of variability will differ from study to study…if one chooses the between-patient standard deviation, one has to confront its dependence on the heterogeneity of the population under study”.2
In White and colleagues' study, we do not have heterogeneous samples on the Chalder fatigue questionnaire and short-form 36 physical function subscale, since both are used as entry criteria.
1 Patients had to have scores of 65 or less on short-form 36 to be eligible for the study.1 However, most, in practice, would probably need to have scores of 30 or more to be able to participate in this clinic-based study. Indeed, only four of 43 participants in a previous trial of graded exercise therapy scored less than 30.
3, 4 Guyatt and colleagues2 suggest that “an alternative is to choose the standard deviation for a sample of the general population”, which White and colleagues have given as 24. 1 An SD of 24 gives a clinically useful difference of 12; both graded exercise therapy and cognitive behaviour therapy fail to reach this threshold. Whether they “moderately improve outcomes”, as claimed, 1 is therefore questionable.
Figure
Copyright © 2011 Photolibrary
I am chair of a myalgic encephalomyelitis support and advice group—an unpaid voluntary position.
References
References
Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial.
- White PD
- Goldsmith KA
- Johnson AL
- et al.
on behalf of the PACE trial management group
Lancet. 2011; 377: 823-836
View in Article
Methods to explain the clinical significance of health status measures.
- Guyatt GH
- Osaba D
- Wu AW
- et al.
Mayo Clinic Proc. 2002; 77: 371-383
View in Article
Physiological and psychological responses of patients with chronic fatigue syndrome to regular physical activity. Loughborough University of Technology, Loughborough; 1997
- Fulcher KY
http://hdl.handle.net/2134/6777
(accessed March 4, 2011).
View in Article
Randomised controlled trial of graded exercise in patients with the chronic fatigue syndrome.
- Fulcher KY
- White PD
BMJ. 1997; 314: 1647-1652
View in Article
Last edited:
ME/CFS Skeptic
Senior Member (Voting Rights)
Isn't it strange that no other major news outlet has reported about this (except Medscape taking over the Reuters article). It makes me wonder what's going on behind the scenes... Are they waiting for Cochrane to issue a press release or something? Hard to believe that the media doesn't see an interesting story in this..
Jonathan Edwards
Senior Member (Voting Rights)
Yes, if Clare Gerada is to be believed this is a matter for immediate international concern. Where are the journalists when you want them?
They are probably so confused by what Donald is saying about Middle Eastern murders and Teresa is saying about how nearly almost she has had a breakthough, or sort of, that they have lost the plot completely.
They are probably so confused by what Donald is saying about Middle Eastern murders and Teresa is saying about how nearly almost she has had a breakthough, or sort of, that they have lost the plot completely.
ME/CFS Skeptic
Senior Member (Voting Rights)
Maybe journalists called Cochrane, but were told that the decision is still in process and new information will be provided in due time etc. It feels like journalists are waiting for something...
Lucibee
Senior Member (Voting Rights)
And their reply is the well-worn tactic of simply repeating what they said in the paper:
and...
No acknowledgment that someone might have actually spotted an error.
Isn't the Reuters thing a SMC release? Didn't the SMC lose a lot of credit over the past year(s)?
Hoopoe
Senior Member (Voting Rights)
I am not sure I understand. We probably made fun of whatever papers were in that review. Are we so powerful that Cochrane caved in and withdrew a review of some quality papers on chinese herbs for CFS (or was it CF)?
Last edited:
Jonathan Edwards
Senior Member (Voting Rights)
Yes.
Or maybe they thought it would be an idea to withdraw the herbs first to give the impression these withdrawals were part of a strategic policy and then oops there goes exercise. I think it more likely that withdrawing the herbs turned out to be easier. Maybe one day we will know.
It is odd about the Chinese herbs review being withdrawn.
From the information I saw here on S4ME, my impression was that the conclusion of that review (that none of the studies were strong enough to support the use of Chinese herbs) was reasonable. Given that the study was done in 2009, I guess it's fine to review things to see if any further studies have been done since, but I'm not sure why that requires the existing review to be withdrawn.
From the information I saw here on S4ME, my impression was that the conclusion of that review (that none of the studies were strong enough to support the use of Chinese herbs) was reasonable. Given that the study was done in 2009, I guess it's fine to review things to see if any further studies have been done since, but I'm not sure why that requires the existing review to be withdrawn.
Last edited:
Jonathan Edwards
Senior Member (Voting Rights)
I agree, that is what was odd. The Chinese herb review seemed fine - no harm to anyone so why withdraw it? But then why not withdraw it - unless you have a policy of withdrawing everything for a certain illness.
What I forget is what Cochrane has said about CBT. My impression is that the review is old and rather non-committal.
Esther12
Senior Member (Voting Rights)
Yeah - it could just be that they decided it was worthless, so might as well be withdrawn. I wonder if they were hoping that withdrawing that one too would make this all seem like a low interest house cleaning exercise?
Jonathan Edwards
Senior Member (Voting Rights)
SMC have certainly been regarded with more scepticism recently. But maybe other outlets just see that if you read the piece it looks very flimsy and amateur.
'We got these leaked emails saying Cochrane were caving in to patients and we spoke to Cochrane who pointed out it was nothing to do with patients. And so we believed the authors and it is scandalous.'
Barry
Senior Member (Voting Rights)
Like they needed a practice run first, just to get the hang of it
.
Barry
Senior Member (Voting Rights)
Two thoughts.
- Having 'temporarily' withdrawn the review, is there anything in Cochrane's rules that sets a maximum time limit before a decision has to be made? Or can it now just lie fallow forever without any closure? Because it feels to me that if not properly concluded in some reasonable time frame, then the real lessons from this will be lost, rather than exposed as truths that need to be told.
- Cochrane categorisations. I appreciate that categorising things is important to keep things manageable and to avoid chaos. But it feels to me there is a risk of biased reviews before anyone even lifts a finger. This CFS exercise review exemplifies an issue that could be the tip of a very large iceberg, especially with regards to MUS etc. The Cochrane system is such (unless I'm missing something), that the choice of reviewers is made based on a presumption of what medical category the review fits into. And yet one of the most crucial aspects that should be under review surely, is what medical category the review is best fitted to! But reviewers are surely going to (maybe unconsciously) favour their own 'home' category, and find it very hard to transfer responsibility across to a different category, both at a personal level, and probably up against procedural hurdles. Feels like the system is set up to strongly risk "pre-determined category" bias.
Taking the ME/CFS reviews as examples, once in the mental illnesses category, how likely were those reviewers to proactively suggest the reviews were being undertaken in the wrong category? Not very likely at all would be my guess. To me this is really important, because there are likely a whole raft of other reviews down the line, for MUS issues etc, that risk the same problem. Cannot help thinking Cochrane need to do some really serious thinking about this.