The critiques of PACE were published in different journals.
-
New Forum Software Installed (Still a few issues but reopening the forum) More details.
Cognitive Behavioural Therapy for chronic fatigue and CFS: outcomes from a specialist clinic in the UK (2020) Adamson, Wessely, Chalder
- Thread starter Esther12
- Start date
True, but that only happened once the data were released and it was clear that the claims about PACE were true. And then Keith G wrote an excellent commentary and David Marks at JHP took it on as an issue. But PACE was a major, high-profile study using a lot of public funding, published in the leading medical journal and having had a big impact on policy, or at least reinforcing existing policy.
This is basically a low-quality trade association journal publishing a stupid and egregiously flawed article. But it is ultimately of little consequence and will have no impact on anything except in the minds of those already inclined to share that perspective. Except for the fact that Wessely and Chalder are co-authors, and this shows that they're still up to their regular tricks, I think it would be of no interest at all.
Having said that, if there's a chance of getting it published officially somewhere, that would be great.
Andy
Retired committee member
Blog: Brian Hughes: Our response to that controversial study on CBT outcomes in chronic fatigue has now been formally published, https://thesciencebit.net/2021/04/1...onic-fatigue-has-now-been-formally-published/
Response to Adamson et al. (2020): ‘Cognitive behavioural therapy for chronic fatigue and chronic fatigue syndrome: Outcomes from a specialist clinic in the UK’, 2021, Hughes and Tuller
Abstract
In a paper published in the Journal of the Royal Society of Medicine, Adamson et al. (2020) interpret data as showing that cognitive behavioural therapy leads to improvement in patients with chronic fatigue syndrome and chronic fatigue. Their research is undermined by several methodological limitations, including: (a) sampling ambiguity; (b) weak measurement; (c) survivor bias; (d) missing data and (e) lack of a control group. Unacknowledged sample attrition renders statements in the published Abstract misleading with regard to points of fact. That the paper was approved by peer reviewers and editors illustrates how non-rigorous editorial processes contribute to systematic publication bias.
Open access, https://journals.sagepub.com/doi/full/10.1177/13591053211008203
Response to Adamson et al. (2020): ‘Cognitive behavioural therapy for chronic fatigue and chronic fatigue syndrome: Outcomes from a specialist clinic in the UK’, 2021, Hughes and Tuller
Abstract
In a paper published in the Journal of the Royal Society of Medicine, Adamson et al. (2020) interpret data as showing that cognitive behavioural therapy leads to improvement in patients with chronic fatigue syndrome and chronic fatigue. Their research is undermined by several methodological limitations, including: (a) sampling ambiguity; (b) weak measurement; (c) survivor bias; (d) missing data and (e) lack of a control group. Unacknowledged sample attrition renders statements in the published Abstract misleading with regard to points of fact. That the paper was approved by peer reviewers and editors illustrates how non-rigorous editorial processes contribute to systematic publication bias.
Open access, https://journals.sagepub.com/doi/full/10.1177/13591053211008203
Wonderful
Kalliope
Senior Member (Voting Rights)
Trial By Error by David Tuller: Journal of Health Psychology Published Hughes-Tuller Critique of Wessely-Chalder CBT Claims
Quote:
We were delighted that the Journal of Health Psychology was interested in reviewing our critique, and ultimately publishing it. As can be seen by comparing the version we posted on a pre-print server with the one released by the journal, we have added a strong critique of the deficient peer review process to which the Wessely-Chalder study was subjected. The authors have done themselves a real disservice by publishing their findings in a journal with such close ties to one of them, if that association was a factor in the inadequacy of the review process. As a result, the good professors and the journal have all scored an own goal.
Quote:
We were delighted that the Journal of Health Psychology was interested in reviewing our critique, and ultimately publishing it. As can be seen by comparing the version we posted on a pre-print server with the one released by the journal, we have added a strong critique of the deficient peer review process to which the Wessely-Chalder study was subjected. The authors have done themselves a real disservice by publishing their findings in a journal with such close ties to one of them, if that association was a factor in the inadequacy of the review process. As a result, the good professors and the journal have all scored an own goal.
Kalliope
Senior Member (Voting Rights)
The Faculty Lounge What's Going on at the Journal of the Royal Society of Medicine? by Steve Lubet
Quote:
My occasional coauthors David Tuller (UC Berkeley) and Brian Hughes (National University of Ireland) have an important new article in the Journal of Health Psychology. They provide a devastating critique of an ME/CFS study recently published in the Journal of the Royal Society of Medicine. As Tuller and Hughes explain, the underlying study suffered from “serious methodological flaws and the highly misleading way in which its results had been presented,” and “key statistical statements in the paper’s Abstract were factually inaccurate.” But that is not why I am posting about it.
Most alarming, from an academic perspective, was the way in which the Royal Society of Medicine first accepted a deeply problematic paper, and then refused to publish a subsequent critique.
Quote:
My occasional coauthors David Tuller (UC Berkeley) and Brian Hughes (National University of Ireland) have an important new article in the Journal of Health Psychology. They provide a devastating critique of an ME/CFS study recently published in the Journal of the Royal Society of Medicine. As Tuller and Hughes explain, the underlying study suffered from “serious methodological flaws and the highly misleading way in which its results had been presented,” and “key statistical statements in the paper’s Abstract were factually inaccurate.” But that is not why I am posting about it.
Most alarming, from an academic perspective, was the way in which the Royal Society of Medicine first accepted a deeply problematic paper, and then refused to publish a subsequent critique.
daftasabrush
Senior Member (Voting Rights)
So much for Wessely's "retirement". Were the vast financial conflicts of interest disclosed?
I would be interested to hear of they cited any studies showing CBT didn't help. The FINE trial for instance.
I would be interested to hear of they cited any studies showing CBT didn't help. The FINE trial for instance.
Andy
Retired committee member
CFS patients remain severely disabled after specialist treatment with CBT in the UK, 2021, Vink and Vink-Niese
We read the study entitled ‘Cognitive behavioural therapy for chronic fatigue and chronic fatigue syndrome: outcomes from a specialist clinic in the UK’ with interest. Adamson et al.1 concluded that cognitive behavioural therapy is an effective treatment for chronic fatigue syndrome because fatigue and physical functioning scores improved significantly and 90% were happy with it.
However, the results for chronic fatigue and chronic fatigue syndrome were not reported separately which might have inflated their chronic fatigue syndrome results.
Also, there were only physical functioning scores at baseline for 77.2% even though it was a primary outcome which also biased the results.
According to the study, 31% dropped out, yet at follow-up there were only scores for 50.5% (fatigue), 50.9% (social adjustment), 40.6% (physical functioning) and 36.7% (global improvement). Consequently, up to 63.3% dropped out, which further biased the results as most patients only drop out if therapy is harmful or not effective, making it unlikely that 90% were happy with cognitive behavioural therapy.
The study noted that those who dropped out were more ill (worse physical functioning and Work and Social Adjustment scores). Consequently, dropout was non-random, which artificially inflated the results. Despite this, the scores after ‘effective’ treatment were only 18.6 (fatigue) and 58.5 (physical functioning). According to both scores, patients were still ill enough to enter the PACE (Pacing, graded Activity, and Cognitive behaviour therapy: a randomised Evaluation) trial2 by the corresponding author of Adamson et al.
Moreover, according to the literature, a physical functioning score of 65 or less equates to severe disability.3
Why the study did not use objective outcomes is unclear as PACE2 showed that cognitive behavioural therapy doesn’t lead to objective improvement (step test and 6-min walking test).
Conclusion
Patients remained severely disabled after treatment with cognitive behavioural therapy in a specialist clinic in the UK. Therefore, it cannot be said that cognitive behavioural therapy is an effective treatment for chronic fatigue syndrome.
Open access, https://journals.sagepub.com/eprint/ZQR9DFAJUXMF6THEDBGR/full
We read the study entitled ‘Cognitive behavioural therapy for chronic fatigue and chronic fatigue syndrome: outcomes from a specialist clinic in the UK’ with interest. Adamson et al.1 concluded that cognitive behavioural therapy is an effective treatment for chronic fatigue syndrome because fatigue and physical functioning scores improved significantly and 90% were happy with it.
However, the results for chronic fatigue and chronic fatigue syndrome were not reported separately which might have inflated their chronic fatigue syndrome results.
Also, there were only physical functioning scores at baseline for 77.2% even though it was a primary outcome which also biased the results.
According to the study, 31% dropped out, yet at follow-up there were only scores for 50.5% (fatigue), 50.9% (social adjustment), 40.6% (physical functioning) and 36.7% (global improvement). Consequently, up to 63.3% dropped out, which further biased the results as most patients only drop out if therapy is harmful or not effective, making it unlikely that 90% were happy with cognitive behavioural therapy.
The study noted that those who dropped out were more ill (worse physical functioning and Work and Social Adjustment scores). Consequently, dropout was non-random, which artificially inflated the results. Despite this, the scores after ‘effective’ treatment were only 18.6 (fatigue) and 58.5 (physical functioning). According to both scores, patients were still ill enough to enter the PACE (Pacing, graded Activity, and Cognitive behaviour therapy: a randomised Evaluation) trial2 by the corresponding author of Adamson et al.
Moreover, according to the literature, a physical functioning score of 65 or less equates to severe disability.3
Why the study did not use objective outcomes is unclear as PACE2 showed that cognitive behavioural therapy doesn’t lead to objective improvement (step test and 6-min walking test).
Conclusion
Patients remained severely disabled after treatment with cognitive behavioural therapy in a specialist clinic in the UK. Therefore, it cannot be said that cognitive behavioural therapy is an effective treatment for chronic fatigue syndrome.
Open access, https://journals.sagepub.com/eprint/ZQR9DFAJUXMF6THEDBGR/full
Thanks @Mark Vink, excellent.
Andy
Retired committee member
Response from the authors.
Cognitive behaviour therapy for chronic fatigue syndrome: Authors' reply, naturalistic outcomes paper, 2021, Chalder et al
"We previously reported on routine clinical outcomes after cognitive behaviour therapy for chronic fatigue syndrome in an NHS clinic.1 We found that fatigue, physical functioning and social adjustment all significantly improved, providing some evidence that results from randomised controlled trials can be extrapolated to everyday clinical settings.
Vink, Vink-Niese2 and Tack3 raised a number of issues with our paper, which we will respond to. Although the NHS clinic sees patients with both chronic fatigue and chronic fatigue syndrome, all patients included in this evaluation met NICE criteria for chronic fatigue syndrome and were assessed by an experienced clinician prior to treatment.
A proportion of scores on the SF-36 physical functioning scale were missing. This was not related to dropout but due to the measure being introduced two years after the other routine outcome data collection had started. However, the sample size was reasonable. We included this measure as it is routinely used in trials of behavioural treatments for chronic fatigue syndrome."
More at open access https://journals.sagepub.com/doi/10.1177/01410768211017775
I note that, of course, they ignore the issues raised by @dave30th et al elsewhere.
Cognitive behaviour therapy for chronic fatigue syndrome: Authors' reply, naturalistic outcomes paper, 2021, Chalder et al
"We previously reported on routine clinical outcomes after cognitive behaviour therapy for chronic fatigue syndrome in an NHS clinic.1 We found that fatigue, physical functioning and social adjustment all significantly improved, providing some evidence that results from randomised controlled trials can be extrapolated to everyday clinical settings.
Vink, Vink-Niese2 and Tack3 raised a number of issues with our paper, which we will respond to. Although the NHS clinic sees patients with both chronic fatigue and chronic fatigue syndrome, all patients included in this evaluation met NICE criteria for chronic fatigue syndrome and were assessed by an experienced clinician prior to treatment.
A proportion of scores on the SF-36 physical functioning scale were missing. This was not related to dropout but due to the measure being introduced two years after the other routine outcome data collection had started. However, the sample size was reasonable. We included this measure as it is routinely used in trials of behavioural treatments for chronic fatigue syndrome."
More at open access https://journals.sagepub.com/doi/10.1177/01410768211017775
I note that, of course, they ignore the issues raised by @dave30th et al elsewhere.
Hoopoe
Senior Member (Voting Rights)
Yes, but what does this mean exactly? The authors presumably want to express that the treatment works in the real world too. However the clinical trial has not actually shown that the treatment works because it failed to control for nonspecific effects, and the real world data is even less reliable in that regard. That results of the clinical trial are similar to clinical practice therefore means nothing as far as this aspect is concerned.
This response confuses objective biomarkers with objective outcomes in clinical trials. Many objective outcomes can and have been included in studies. They consistently show a lack of effect.
It's not a fact that any improvement actually occurred because bias affecting the subjective outcomes hasn't been controlled for. Not in the clinical trials, even less so in practice.
The authors appear unwilling to discuss the issue of systematic bias.
Last edited:
Mary McCarthy famously said of Lillian Hellman that "every word she writes is a lie, including 'and' and 'the.'"
I'm not going to say that about any investigators involved in this example of atrocious research--Hellman sued McCarthy over the statement, although Hellman died before the suit could be heard--but I will just leave the sentence out there to speak for itself and people can make their own assumptions.
ADD: I have no position on the veracity of the McCarthy claim that Hellman was a terrible and dishonest writer, although it was disgraceful that Hellman sued her over what was clearly an expression of allowed opinion and not a phrase meant to be taken literally.
I'm not going to say that about any investigators involved in this example of atrocious research--Hellman sued McCarthy over the statement, although Hellman died before the suit could be heard--but I will just leave the sentence out there to speak for itself and people can make their own assumptions.
ADD: I have no position on the veracity of the McCarthy claim that Hellman was a terrible and dishonest writer, although it was disgraceful that Hellman sued her over what was clearly an expression of allowed opinion and not a phrase meant to be taken literally.
ME/CFS Skeptic
Senior Member (Voting Rights)
I also submitted a letter to the editor. The submitted version is available here:
https://www.researchgate.net/public..._chronic_fatigue_-_a_comment_on_Adamson_et_al
I had hoped that by focusing on specific questions, the authors would have to clarify some issues that weren't clear before. Unfortunately, I haven't become much wiser after their response.
The authors said there was a dropout rate of 31% which refers to patients who did not complete any questionnaires at discharge or follow-up. But the proportion filled out each outcome measure was much higher. In other words, patients filled out one but not other questionnaires and it still isn't clear why. In their response, the authors only mention that they started using the physical functioning subscale after a while which explains the lower number of participants for this scale at baseline (something Vink & Vink-Niese highlighted). This doesn't explain why 49% of the data on fatigue and social adjustment at three months follow-up, is missing.
I also asked how many patients the 90% who were satisfied with their treatment represents because the sample and the drop-out rate seem to differ substantially for each outcome measure. The authors responded that "these figures represent all patients who completed self-report measures at discharge and are therefore commensurate with all other reported figures at discharge" which is not very enlightening. Why couldn't they just give the number of participants who filled in this questionnaire?
I also asked: "the self-reported global improvement scale used in this study had only six instead of seven options, as is usually the case. Could the authors explain why the option ‘much worse’ was not available in this questionnaire?" but they seemed to have ignored this issue completely.
https://www.researchgate.net/public..._chronic_fatigue_-_a_comment_on_Adamson_et_al
I had hoped that by focusing on specific questions, the authors would have to clarify some issues that weren't clear before. Unfortunately, I haven't become much wiser after their response.
The authors said there was a dropout rate of 31% which refers to patients who did not complete any questionnaires at discharge or follow-up. But the proportion filled out each outcome measure was much higher. In other words, patients filled out one but not other questionnaires and it still isn't clear why. In their response, the authors only mention that they started using the physical functioning subscale after a while which explains the lower number of participants for this scale at baseline (something Vink & Vink-Niese highlighted). This doesn't explain why 49% of the data on fatigue and social adjustment at three months follow-up, is missing.
I also asked how many patients the 90% who were satisfied with their treatment represents because the sample and the drop-out rate seem to differ substantially for each outcome measure. The authors responded that "these figures represent all patients who completed self-report measures at discharge and are therefore commensurate with all other reported figures at discharge" which is not very enlightening. Why couldn't they just give the number of participants who filled in this questionnaire?
I also asked: "the self-reported global improvement scale used in this study had only six instead of seven options, as is usually the case. Could the authors explain why the option ‘much worse’ was not available in this questionnaire?" but they seemed to have ignored this issue completely.
ME/CFS Skeptic
Senior Member (Voting Rights)
As usual, I received support from S4ME members who provided feedback and fact-checked before I submitted the comment.
Many thanks, really appreciate the help!
Many thanks, really appreciate the help!
Jonathan Edwards
Senior Member (Voting Rights)
It is interesting to see Simon Wessely publicly putting his name to such ignorant remarks.
Effectively, they are saying 'yes, just to be explicit, we really have no clue about how to gather reliable evidence'.
rvallee
Senior Member (Voting Rights)
These people are completely unserious, but at this point there is at least equal blame on the journals who simply don't care about inaccuracies and contradictions in research they publish. They never actually address the flaws, systematically ignore them, and it's that the journals don't even care that is the problem. No one can function without oversight and here this is the function the journals are supposed to play, by simply refusing to do their job they are complicit in misconduct and unprofessional behavior.
rvallee
Senior Member (Voting Rights)
Entirely their choice. Explicitly choosing data going back that far in time is their problem. They chose this time frame, no one forced them to. That the editors of the journal are OK with such BS answers clearly shows they have entirely waived off their role in BPS papers, they merely publish stuff, they don't do any vetting and don't even care what's in it.