CBT combined with music therapy for chronic fatigue following Epstein-Barr virus infection in adolescents: a feasibility study, 2020, Wyller et al

Here’s the original version of the article: https://bmjpaedsopen.bmj.com/content/4/1/e000620.responses

Here’s my comment on that version of the article: https://bmjpaedsopen.bmj.com/content/4/1/e000620.responses

Here’s the retraction notice: https://bmjpaedsopen.bmj.com/content/4/1/e000620ret

Here’ the republished version: https://bmjpaedsopen.bmj.com/content/4/1/e000797


It would be good if we could make an overview of the most significant changes between the retracted and republished version. I don’t see many.

• The title now says ‘a randomized controlled trial’ instead of ‘a feasibility study’. In other sentences the word ‘feasible’/’feasibility’ was replaced with ‘useful’/’usefulness’, we ‘explored’ was changed into we ‘studied’ etc.
• In the abstract and conclusion the following has been added: “participants’ concern regarding school absence should be properly addressed to secure recruitment.”
• Instead of highlighting trends the abstract now simply says that there were also no significant differences among the two groups. In the results and discussion section, however, the authors still highlight “most symptom scores tended to favour the intervention group” and “there was a trend towards higher recovery rate with a total of 8 (62 %) recovered individuals in the treatment group” and “we observed a concurrent tendency of improvement of many symptom scores, including fatigue and postexertional malaise, in the intervention group.”
• While the original conclusion read: “The tendencies towards positive effects on patients’ symptoms and recovery might justify a full-scale clinical trial”, the rewritten conclusion reads: “A fully powered trial is needed to evaluate efficacy”. So without highlighting tendencies towards positive effects.
• The abstract makes clear that there was a low response rate: “A total of 91 individuals with postinfectious CF were eligible, and a total of 43 were included (21 intervention group, 22 control group).” In the original version they simply said: “A total of 43 individuals… without mention that 91 patients were eligible.

Did anyone find other significant changes?

 

Yes, another example of how important your work is, @Michiel Tack and @dave30th .Thank you to everyone involved.

Yes. As far as I can see tha editors don't disclose what their error actually was. In my view, the journal's error was to not reject the paper.

[Edit: ] It would be a precedent if they did admit that.

Instead, they let the authors republish the paper with an amended title.

So it seems in protecting the authors they try to protect themselves, too.


Haven't read the republished paper yet but in addition to the points highlighted by Midnattsol and Michiel I saw that it still refers to the old version of the Cochrane review when it states:

"graded exercise therapy (GET) in itself is shown to have beneficial effects in CFS."

And of course, the abstract's first sentence still is:

"Cognitive–behavioural therapy (CBT) is effective in chronic fatigue (CF) syndrome."(*)

Actually, it's slightly amended: It now even more clearly implies the authors' understanding of CFS of just being a syndrome of "CF".

Previous version: "Cognitive–behavioural therapy (CBT) is effective in chronic fatigue syndrome."

I hope that also Retraction Watch will follow up the editors' understanding of transparency and research integrity.

They quote one of the editors:

"I want to make it clear that the authors were not responsible for this error. Chronic fatigue syndrome is still not fully understood and dogged by controversy, so it’s important to add to the knowledge base, hence the decision to invite the authors to resubmit their original study."

So the editors' understanding of adding to the knowledge-base is to publish a still highly miselading paper that references other highly misleading work in a misleading way that equally wasn't retracted because editors preferred to protect the authors instead of research integrity.

In addition to the misleading reference to the Cochrane review hey also approvingly refrerence the lightning process study when they claim that:

"Preliminary evidence suggests that a multidisciplinary approach combining CBT with other elements might improve effectiveness as compared with CBT alone in patients suffering from CFS".

(Edited for clarity.)

(*) Edited to add: In the full text, the authors write:
"Evidence suggests a beneficial effect of cognitive–behavioural therapy (CBT) in adults5 and adolescents6 with chronic fatigue syndrome (CFS), as well as in CF from well-defined causes such as cancer7 and diabetes.8"

The references for CFS are the PACE trial and FITNET; the papers referenced for CF in cancer and diabetes are both co-authored by Hans Knoop.

 

I have not previously seen that

chronic fatigue (CF) syndrome

That means that subsequent use of "CF" refers to "chronic fatigue" and not "chronic fatigue syndrome". The paper seems very vague on this point

 

In the republished version the authors have also included supplemental file 2 - which is the statistical analysis plan: https://bmjpaedsopen.bmj.com/conten.../bmjpo-2020-000797supp002_data_supplement.pdf

 

I'm afraid I won't be looking but have you tried using the 'compare document' function that is available on word processing software such as Word or Google Docs? You may know of it already, but thought I would suggest it just in case.

 

I remember talking to the late great Doug Altman in May 2018. I had been pestering him and colleagues on and off to look at PACE, and the Crawley research, and he knew I was really worked up about it, but he didn't have time (or I think inclination) to get involved. But on that occasion over lunch at a conference/meeting, he told me that he was considering strengthening his vocabulary when writing about the problems in research to include "lying". I realised this was a big thing for him to say, and I sensed he was quite upset. For despite calling out the travesties, particularly regarding the misuse and poor reporting of statistics in research, he would never name and shame - he never had a bad word to say about anyone.

 

In the first version submitted, the authors didn’t use the term “feasibility study” but “exploratory randomized trial” in the title and they stated their conclusion more firmly as:

“A mental training program combining music therapy and cognitive behavior therapy lowered the level of physical activity, improved symptoms of post-exertional malaise, led to higher recovery rate over time, did not cause harm, and appeared feasible in adolescents suffering from chronic fatigue after Epstein-Barr virus infection.”
https://bmjpaedsopen.bmj.com/content/bmjpo/4/1/e000620.draft-revisions.pdf
​

That was, of course, all inappropriate because the small sample size doesn’t allow to make such conclusions.

The strange thing about this study is that the number of participants the authors tried to recruit (50) is much lower than the number the power analysis suggested was needed to detect a difference of moderate effect size (120). So it was powered only to detect large effect sizes. The authors justified this by saying that since "CBT alone is documented to have a moderate effect size in CFS/ME, only a substantial effect size is of direct clinical interest. Analogously, only a substantial treatment effect is of interest regarding markers of pathophysiology.” In the protocol, they also argue that “the FITNET study suggests that larger treatment effects might be assumed in adolescent CFS/ME patients as compared to adults.”

So when they presented their results, this reasoning was ignored, and instead, good acceptability and feasibility of the intervention were emphasized. That the trends towards improvement were not statistically significant was disregarded because this was only a very small trial and the positive trends were said to argue for a fully powered randomized trial.

This was quite confusing. One reviewer, Maria Loades, made the following remark about this:

“I struggle to understand from the aims of the study and the way the study is described whether this was intended as a feasibility study – i.e. to look at feasibility (can this be done?), acceptability (how do participants experience it?) and to give some indication of potential effect sizes to power a future larger scale trial, or whether this was intended as a fully powered trial. Throughout, I think this needs to be clarified for the reader and interpretations/conclusions drawn in light of what the aim was.”​

To which the authors responded:

“Thank you. We agree – this study should be regarded a feasibility study, and the manuscript has been rephrased accordingly.”
​

It could be that editors suggested this as well: that given the very low sample size and power, this study looks more like a feasibility study and that it should be presented as such. Even if this is true and there was such a suggestion by the editors, the authors still rephrased and published the study as a feasibility study, knowing that this wasn’t how it was designed or presented in their protocol and statistical analysis plan. So the statements by editor Imti Choonara and the BMJ retraction notice that “the authors were not responsible for this error” or that “this was not due to error on behalf of the authors” are very likely untrue.

The main issue with rephrasing the study as a feasibility study is that the authors could escape the conclusion that the intervention wasn’t as effective as they thought. If the intervention did show significant improvements (as can happen by chance) the authors would have probably touted it as an important finding. If the intervention didn’t show the improvements hoped for (as was the case) then they could highlight acceptability and feasibility and say that trends toward improvement deserve further study (as they did). So either the intervention is highly effective or it needs further study. This methodology should be avoided as it increases the chance of false findings and the risk that scarce funds are diverted to research ideas that do not merit those investments.

The fact that the authors have rephrased their paper and avoided the use of the word ‘feasibility’, changes little to all of this. They are still avoiding the main conclusion that can be drawn from this study, namely that the intervention failed to show improvements of a large effect size compared to the control group.

 

Some of the issues raised such as the addition of post-exertional malaise as an outcome when it wasn't listed as one of the secondary outcomes or the highlighting of recovery rates even though these do not take dropouts into account, and the difference only appear after the treatment.

I think the paper is still misleading.

In addition, the republished version does not make clear at all that there were issues with this paper that justified a retraction (and my comment is no longer there, so the issues that weren't addressed have now become free of criticism).

 

It’s a rather unbalanced situation: if the intervention shows large effect sizes then it’s effective. When it doesn’t a full-scale trial is urgently needed. There’s no risk of your hypothesis being disproven.

 

Did they submit a CONSORT checklist with their manuscript?? Is there a flow diagram? https://www.equator-network.org/reporting-guidelines/consort/
I will have a look when I get the chance...

 

It is simply unbelievable (except of course it happens all the time) that the trial registration (https://clinicaltrials.gov/ct2/show/NCT02499302) doesn't mention PEM yet they still report it in the paper. "Interestingly, we observed a concurrent tendency of improvement of many symptom scores, including fatigue and postexertional malaise, in the intervention group"

Also, where are the reviewers reports??

 

Academic grifters of the highest order.

 

https://www.diffchecker.com/

edit: I've somehow messed this post up while writing it... give me a few minutes to sort it out!

Lots of changes of 'feasibility' to 'usefulness'.

eg:

to

Looking through these changes makes the claim that this was just a problem caused by the journal's editorial mishap, rather than a problem with the authors, really odd.

to

to

There are significant differences and similarities in the discussion section:

to

There were other small changes too.

 

Should we be sympathetic to the journal for the seeming white-wash on the authors?

Does anyone know what sort of legal pressure journals can come under?

Or is this more likely to be BMJ BS of caring more about protecting bad researchers from the mob of unruly ME patients who refuse to be grateful to Wyller & co for all they've done for us?

 

Methods Adolescents (12–20 years old) participating in a postinfectious cohort study who developed CF 6 months after an acute EBV infection were eligible for the present study

Does this make sense to anyone? Why would participants have developed CF six months after infection? Are there established criteria for CF which specify this period. Are they not merely confusing this with criteria for CFS, which is ,of course not what they purport to be studying? Not at all. One would presume that they merely continued to suffer fatigue after infection. Where are the criteria for CF which specify the six months delay?

 

There are still so many severe flaws in the paper and in the first open and the 2nd, not (yet) "opened" review process, that I find it difficult to track which of them are the most relevant and also which of them have been already addressed by whom.

So apologies if my comments are redundant, but...

Does anyone know whether ""exploratory randomized trial" is an established term in trial methodology?

(see also my previous post here.)

 

I read this as the editor, probably with input from editorial board (which includes Esther Crawley), seeing this as a convenient face-saving way of not upsetting any of their contributors. An editorial mistake is just a slight oversight. It does not put any pressure on any authors or on any referees who might have provided muddled opinions - or indeed suggested an idea to the authors that on reflection wasn't so hot.