Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

If I can be of any help, do please let me know. I have lots of time, and a reasonable amount of energy. Need direction though, or I'll tend to go off piste.

 

I did a search with "myalgic encephalomyelitis" in the title, abstract or keyword. There were 63 records under the trials tab, but 4 of the 1st five records are clearly not trials. I haven't looked at the full list. One of the first five looks like it might be a trial, but there's no abstract. All the CENTRAL records are of papers, not studies. So for PACE there are many papers published at different times, so there should be several biblographic records in the Cochrane database. In theory they will be linked together. Just for fun, I will check if there is a record of all the PACE papers are in CENTRAL, and if they are linked together. I am doubtful they will be, but you never know.

 

you and me both! ;-)

 

yes sorry, have deleted my post as it was incorrect.
I should know better than to be looking at things that require my brain to work at that time of day. Am not quite there yet but will have another play with it later.

eta: had a play, watched a few training videos but now have lost sight of what I was doing it for

 

If these are quotes from Hilda Bastian then I am concerned (although maybe this was in her sitting on the fence days).

The issue has nothing to do with the review being out of date or with new trials being added or whatever. It has to do with the fact that a complaint was lodged pointing out that the review was incompetent, followed by lots of other sources saying the same and Cochrane experts themselves deciding it was not up to standard.

 

Indeed. I think these are Hilda Bastian quotes. They seem to have changed the withdrawal policy at the same time as all this kerfuffle, and it makes withdrawal a bigger deal - akin to a retraction. So I guess now it's even more difficult to get the authors to agree to it than before. What I don't understand is why the authors have any say in the matter...unless it's Cochrane's "duty of care" to its volunteers (and its image) - never mind the patients. Not retracting it and instead distracting us with a promise of a shiny new review (which will take FOREVER) is, as you say, completely missing the point. The review should be retracted and a full apology given to patients.

 

In my experience not understanding why people make certain decisions is often down to misunderstanding what motivates them; understand the motive and then things can make much more sense. In this case my feeling is the decision is not motivated by science, but far more likely motivated by career politics and self interests. Just my opinion of course.

 

The decision is nothing to do with science. It is everything to do with self-interest of Cochrane as an institution which will never take any responsibility for anything - even its own incompetence. They seem to have made a (hopefully reversible) decision not to retract this review but instead have amended it to as watered down a version as they can "defend" without admitting their incompetence in publishing it in the first place. At the same time they protect the authors' reputations and careers. A win-win. Except for patients of course.

 

You did a a great search strategy - more sophisticated than mine which was to see how many studies in the Cochrane database used the term Myalgic Encephalomyelitis. I was surprised that a large proportion appeared not to be trials in a trials database! Well done Cochrane. I guess this is because there is no review group (aside from the inappropriate mental disorders group) maintaining the database in this area of research. Can you send that strategy to me and I will run it and put the results in excel and send it to you if you like. Or I could have a look myself and for a start identify what proportion are actually trials. Would be interesting. Maybe a basis for a study register which could include surveillance studies and other types of evidence, especially of harm.

 

It wasn't anything amazing; just all trials with Myalgic Encephalomyelitis, and other names in the title/abstract, combined with mention of 'harms' 'adverse' 'adverse events' in the full text.
It came up with 146 entries:

 

Great - I might lose the limits on the mention of harms to see how many there are in total. Would be interesting to see how many more records there are. Looking at the results they do seem to be all trials...did you limit the search to records with trial in the title? Perhaps I was being unfair about the trials database being full of non-trials!

 

1159 Trials matching "#1 - ("myalgic encephalomyelitis"):ti,ab,kw OR ("myalgic encephalopathy"):ti,ab,kw OR ("chronic fatigue syndrome"):ti,ab,kw OR ("CFS"):ti,ab,kw" (Word variations have been searched)

there are some odd ones that come up that do not look particularly relevent but on closer inspection they do mention cfs, ME etc in the abstract
(the adverse events, harms etc were all boolean OR, and done on a separate search which I then combined with the one above.).

Am getting sidetracked again as there are some interesting old papers from when the illness was taken seriously.

eta:
on closer inspection some uses of CFS are not for Chronic Fatigue Syndrome.
so the total figure will be less than above.

eta2: I've been through the first 100 and several are nothing to do with ME/CFS as per my comment above. There are also quite a few that are not trials, many with no abstract, and also somewhere along the line Fibro and chronic fatigue are being included (might have something to do with how keywords and Mesh keywords have been set up; eg

although I didn't use MeSH in my search (?)

egs of dodgy entries

eta: forgot to say that I sorted the initial search by date first published Old to New.

 

From what I remember from when working for Cochrane, there are regular searches of EMBASE and Medline which dump lots of records in CENTRAL and then the review groups go through them and clean out the non trials. But this system obviously doesn't work. And especially not in this area. What would be much more useful for Cochrane to do would be to create a study-based database (like the ALOIS database for dementia trials https://alois.medsci.ox.ac.uk/search - but for everything). Instead they have carried on maintaining CENTRAL (badly) which is a bibliographic database and much less useful as a result. Having a database with meta data about characteristics of trials - eg. outcomes (both measured, and reported), participant numbers, treatment characteristics etc, plus information about funding and other potential conflicts of interest - would be much more useful and save a huge amount of effort when conducting reviews, or other research - not least in identifying good and bad trial conduct, AND most importantly non-methods-related sources of bias (eg. triallists having reputational and ideological interest in the results of the trial going a particular way - very common, even in non-drug trials). It could be done in advance, at one time. Cochrane are the only organisation who could do this - it would be a huge benefit to society to map the research landscape in this way. But each Cochrane group reinvents this wheel every new review they do. The characteristics of the included studies don't include anything about funding or other conflicts, and are stuck in a table in the review in pdf format. Often characteristics from the same study could be extracted by several different review teams because of an overlap in the review question (eg. CBT for CFS and Exercise for CFS both include the PACE Trial)- this is avoidable waste.

 

"It was created in 2008 thanks to a grant from the Alzheimer's Association."
I imagine that any group that 'takes on' ME/CFS would require funding to undertake a similar set up and although it would be great I doubt if there would be any current UK ME charity willing to stump up the cash. No idea how much it would be.
Also given the politics, I am not sure if there would be consensus as to who should oversee it's initial creation and what should and should not be included. eg 'research' on 'chronic fatigue'.

But from what I have seen of the existing set up it is a real dogs dinner.

Also, although I've looked at many papers before, it really struck me how really badly so many of them are in terms of trying to categorise them into any kind of order. (But maybe that's just me and my former profession as an Analyst programmer).

 

Yes, it is a dogs dinner. I used to work for the Cochrane dementia group on a public engagement project which involved recruiting and training volunteers (we initially targetted carers and former carers of people with dementia) to populate the ALOIS database. I could see the huge potential of extending the scope of this citizen science to cover other disease areas (ie. other Cochrane review groups would adopt the same approach and get patients involved in the work.) I even did a poster about it for a Cochrane conference...It would change Cochrane too much and potentially give too much power to patients and the public so the idea never got off the ground

 

That doesn't inspire confidence in the next review of exercise therapy for ME/CFS.

Even if well meaning people are involved, in the end I think Cochrane will maintain control and ensure that nonpatient interests aren't harmed too much.

Does anyone think Cochrane will admit that unblinded clinical trials with subjective outcomes and sometimes also patient indoctrination are worthless? I doubt it because they would encounter too much resistance from people whose careers and income depends on the emperor wearing the most wonderful garments even if that's not true.

I think the end result will be some sort of political compromise (a middle ground between opposing views) but not strict commitment to rigorous science.

 

unfortunately you could then end up in a 'Wiki situation' and it will always be the overseers who have the final word.

Also I think (inspite of all the rhetoric about getting the public engaged in science) that the journals, a lot of researchers and Cochrane, like to keep the 'mysticism' and complexity around research because it keeps them in control.

If they standardised and enforced how research papers etc were presented from registration onwards, ie key pieces of information re funding, coi s, protocols etc (in set fields on a standardised form) with a view to the data being easily transferable into whatever data retrieval method might be used, it would make it all much more transparent/easier to analyse.

But then that would be far too logical.

 

Absolutely nail on the head. I would be pleasantly surprised if the new review looked (primarily) at objective outcomes and concluded that GET and CBT are ineffective. Careers, reputation, and income for bigwigs come before patients for sure.

 

Absolutely spot on about them wanting to preserve the mystique to stay in control. I am all for standardization - it really isn't rocket science - my 14 year old daughter did the coding task. But yes, Cochrane were very scared to hand over such a difficult task to the public. But at the same time do a crap job themselves - evidenced by the dogs dinner that is CENTRAL.