Cochrane Review: 'Exercise therapy for chronic fatigue syndrome', Larun et al. - New version October 2019

Given the glaring flaws in this measuring instrument, and the fact out of range readings may be misconstrued as valid in-range readings, I would love to see this measuring instrument properly and rigorously reassessed. If it were something you bought in a high street shop, it would be deemed not fit for purpose, and I'm sure fall foul of consumer protection laws.

If the limitations of this measurement scale were properly and formally exposed, and its use where no careful account were taken of those limitations, then that alone would crucify PACE, as well as those who invented it ... oh yes, they were also PACE authors.

Is there a way to get an independent assessment of medical measuring instruments, by bodies with the clout to say yay or nay?

 

It is not a measuring instrument. It's a diagnostic instrument. And it doesn't even do that job properly, because Oxford requires 6 months of fatigue, not 3.

[edit: Sorry, I'm not remembering things correctly. No mention of 3 months. The instructions say this:

"We would like to know more about any problems you have had with feeling tired, weak or lacking in energy in the last month. Please answer ALL the questions by ticking the answer which applies to you most closely. If you have been feeling tired for a long while, then compare yourself to how you felt when you were last well."

Options are "Less than usual", "No more than usual", "More than usual", & "Much more than usual".

It does not define what "usual" is. It does not define what "a long while" is. "When you were last well" may not be the same thing as "before you were ill".]

 

Thank you for that because with your link I managed to find an Excel version but I wasn't allowed by the forum to upload the Excel version that's why I have uploaded the PDF version

 

OK, so my medical terminology is incorrect. But whatever it is correctly called, it is nonetheless being used to measure fatigue in scenarios it is not valid for. An instrument of any kind is only valid when used within its limitations; one of the first things an engineer learns.

What is the difference between the two please?

ETA: I can take a guess. Something that diagnoses you have or do not have ME, will not necessarily have the sensitivity nor the range to measure differences in your ME severity between to points in time. Is that close?

 

This is well-known problem in many areas, not just medicine. That it is so well-known and still allowed to be used is frankly malfeasance.

In the early 2000's, the US education system made a big shift towards standardized testing in order to determine allocation of funds for schools. School that repeatedly had low scores on those tests were defunded. The end result was predictably that schools started building their curriculum strictly around those tests. They did not teach a normal curriculum anymore, instead most class time was spent on material straight from the standardized testing pool.

Many schools resorted to systematically cheating, literally helping their students to cheat on the tests, as otherwise they would be defunded and so out of a job. The end result was also predictable: students didn't learn much since all they did was prepare for the tests, not by learning but by memorizing as many answers as possible.

Using the very thing that is ultimately tested as an ongoing target means everything is done to optimize that endpoint. It creates the environment for maximum bias. In this area of research, it additionally compounds to bias from the researchers themselves, who are frankly borderline cultists in their beliefs about the BPS ideology, bias from the questionnaires, which are mediocre tools at best, and maximally biasing participants by making the "treatments" all about influencing responses on the questionnaires and telling them explicitly that it is their improved responses on those questionnaires that will free them of this horrible illness.

As usual, the problem isn't even with the PACE gang, it is with people who know better and let them whatever they want. It's a failure of oversight and perverse incentives. None of this is real science and even less real medicine and I am pretty certain that everyone involved understand what they are doing, they just think it's all fine since we're not actually sick so it makes no difference either way because Simon said so.

 

It's not you, it's them!

The problem is that any time someone puts a number on something, folks assume that it must therefore "measure" something. In this case, the number is supposed to give an indication of fatigue severity (more tired than usual vs much more tired than usual), but what does that actually mean? And once it has a number, all semblance of meaning is transferred and potentially lost. The number becomes more meaningful than what it originally signified. Subjecting it to analysis gives it yet more meaning. (But without it actually meaning anything at all.)

At least with the PF (physical function) scale, the number has some sort of a definition behind it (being able to walk more than a block or a mile is quantifiable thing), but what on earth does "feeling a bit more tired than usual" mean in that context? And all the questions in the CFQ are incredibly vague (Do you feel weak? How's your memory?). Relating/comparing that to another person is impossible. And even within a person, your impression of fatigue will change over time, particularly if you are undergoing a programme of fatigue re-education to actively redefine what is "normal".

 

Ah yes, I see what you mean. Arbitrary in the extreme, yet treated as if accurate and precise ... when no one even knows what the reference is in the first place.

Seems to be the first thing this breed of psychiatrists seem to do. Ascribe a number to something totally woolly to give it some parasitic credibility.

 

They seem to have built their whole careers around the concept .

 

In case of interest to you or anybody:


J Health Psychol. 2017 Aug;22(9):1192-1197. doi: 10.1177/1359105317707531. Epub 2017 May 12.
PACE-GATE: An alternative view on a study with a poor trial protocol.
Stouten B.

https://journals.sagepub.com/doi/full/10.1177/1359105317707531

 

Wow. This picture speaks way, way more than 1,000 words and most of those words are profanity. As fraud goes, this makes the MMR paper and everything that followed it seem honest and good.

 

Just noticed that the changes made to the Cochrane review are not normally described as an update (which would include a new literature search and inclusion of new studies/data) but as an amendment. I have now changed this to the overviews I posted.

 

Maybe cherry picked to help (still) support their findings? What difference would their have been without them?

Were they within their remit to add these?

 

@Sly Saint What do you mean by 3 added studies?

As far as I know, no studies have been added to the data.

In their discussion section, they mention 2 relevant RCT's (GETSET and Marques et al. 2015) that have been published since their literature search but the data of these aren't included in the meta-analysis.

 

See quoted section in @Sly Saint's post ...

https://www.s4me.info/threads/cochr...version-october-2019.11564/page-4#post-205847

 

Do you mean this: "We have added three new studies in this updated review (Jason 2007; Wearden 2010; White 2011)"?

This is a statement that was already in the previous version.

These studies (Wearden et al. 2010 is the FINE trial and White et al. 2011 the PACE trial) were added to the review done by Edmonds et al. back in 2004. So Larun et al. did an update of that version.

After problems were noted with the Larun et al. update, they have made an amendment to correct for some mistakes, but no new studies/data were added to the meta-analysis as far as I can see.

 

Those studies were also in the 2014 version of the review, but were new compared to the review before that.

 

Thank you all who are working on the Cochrane case.
I'm not able to read much or word a substantial critique.

However, I think the easiest and quickest measures Cochrane now could undertake as a first setp to take evidence-based medicine and patients seriously were to add the editor-in-chief's comment on the review, so that it's immediately visible to anyone looking at both the review and its abstract.

I thought to write to the new editor in chief something around the following lines:

Dear ......
Thank you very much for your comment on the publication of the revised edition of the Cochrane review on exercise therapy for 'Chronic Fatigue Syndrome'.

Even though some amendments of the revised review have to be acknowledged, to us and other critics the review still appears strongly misleading, the evidence of the included studies is still overstated. Moreover, the authors' conclusions do not sufficiently reflect the restrictions worded in other parts of the review. For example, we consider it a grave neglect hat the review doesn't specify the inclusion criteria for patients more precisely than "able to participate in exercise therapy".

The plans announced for a full update sound promising. This process will take considerable time though, during which the still misleading parts of the review will continue to inform health care decisions with a high potential to harm patients. Given the high impact of Cochrane reviews for health care decisions around the globe, we think immediate action is needed.

We propose as a first step to add your announcement of the plans for a full update to the revised review, so that it's immediately visible to anyone looking at both the review and its abstract.

(add some formal e-mail language I'm currently not able to word.)

Edited to add: Maybe add some more criticsm about the "certainty of evidence" and announce that comments will be submitted as soon as possible, but a visible notice by the editor-in-chief would at least acknowledge that there are serious issues with the current review that have to be dealt with also in Cochrane's view.)

I'm aware this would still be far from ideal, but...

Is this something S4ME would like to consider setting up and sending?

@Andy

 

Michiel is now even more famous than he was before

http://www.virology.ws/2019/10/09/trial-by-error-more-on-the-revised-cochrane-exercise-review/