Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

I agree nothing researchers or I have to deal with here is as bad as the effect of this illness on your life. I am listening, and reading a lot about people's experiences of the illness and concerns about the Cochrane review, and will continue to.

I don't think Cochrane reviewers generally believe just because a medical journal published a trial that it must be good, but unfortunately many others do, I agree. I agree that people with ME/CFS have not been treated fairly in many ways, and I will do my best to make this particular process better.

 

That is it in a nutshell. So very many lives destroyed by deeply flawed research, due to arrogantly held ideologies still held and promoted to this day, with no end in sight. And it has deeply influenced the way that medical professionals across the whole spectrum perceive and treat pwME with such disdain and callousness in some cases - not always, but often. It's incredibly depressing and demoralising.

ME/CFS is a condition that is incredibly difficult for outsiders to comprehend, and I very much deem most researchers to be outsiders - if not then they would have far greater insights than they do. Laboratory experiments and questionnaires are profoundly ineffective at gleaning anything approaching an adequate picture, as I see it. I truly believe that if genuine, sincere ME/CFS researchers want to really understand more about ME/CFS, then as good a crash course as any would be to spend a week or so with a pwME.

In the case of someone like my wife, they would get to realise that she is not deconditioned and never has been, and that research and treatments based on this assumption are just plain wrong and potentially dangerous. And that within her limits my wife is not in the least bit fearful of exercise (far far from it), and pushes her physical and mental boundaries hard, but within sane limits. They would realise how bl**dy insulting it is to presume such rubbish, insulting not just to her, but to me her husband, and to our children, who all know so well how she strives endlessly against the odds - odds that are far worse than they should be thanks to the aforementioned rubbish. And they would realise just how morally imperative, important and urgent it is to set the scientific record straight; clearly, openly and unambiguously.

If the watchkeepers cannot be relied upon to keep watch, then who the hell is there to rely on?

I do genuinely appreciate there are good, sincere researchers that get caught up on the wrong side of much of this, but if they were offered a swap, I very very much doubt they would take it. If they truly saw the reality of the lives of pwME, they might feel more motivated.

Not feeling terribly diplomatic today (see above "incredibly depressing and demoralising").

 

Thank you for this information, @Hilda. Very helpful

CDC has contracted a group to conduct an ME/CFS treatment review (all treatments, not just GET) and has stated they will provide an opportunity for public review of the draft report. The time and process for this has not yet been announced.

Does Cochrane ever do that? Given the community concerns with previous reviews, the harms that have been done to patients in the name of Cochrane, and the depth of community knowledge on the GET evidence base, this would seem warranted.

For background:
The protocol for the current CDC review is: https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=142805

The review is being conducted by the same group that did the 2016 review that found evidence of harms but insufficient evidence of efficacy for GET once Oxford studies were excluded. https://www.ncbi.nlm.nih.gov/books/NBK379582/

By excluding the three trials using the Oxford (Sharpe, 1991) case definition for inclusion, there would be insufficient evidence of the effectiveness of GET on any outcome (1 trial, n=49).​

 

I am grateful for your interest and contribution to the peer review process here, @Hilda Bastian. I agree with almost everything you say but the quote above unsettles me a bit.

I am not quite sure why I should be grateful for your interest in an illness that I never worked on or have in my family but I guess there may be three answers. One is Bob. As I think Tovey realised, if we forget Bob we may as well wash our hands of the whole business. 'Due process' does not cut it. A related one is whatever makes people decide to march for George Floyd. I guess that must be why I hang around this forum. The third is a feeling of disgust and embarrassment at the behaviour of people who consider themselves my scientific colleagues (one of whom has accused me of 'disloyalty', as if I should be loyal to them rather than patients).

I started out as an invited advisor on research projects in ME/CFS. I tried to get myself boned up on the immunology and metabolism that were relevant. In the process I attended a meeting where I was introduced to the PACE trial by Peter White. I had little interest in psychological therapies but was interested to get a rounded view. I have listened to thousands of colleagues present their research. His presentation was something I had never encountered before. It came across as a deliberate and disingenuous manipulation designed to discredit patients in order to protect poor science. Within two minutes I felt I was being scammed. A couple of slides were flashed up to prove that PACE proved CBT and GET worked (with no mention of methodology or even a valid Y axis) and the rest of the fifteen minute talk was abuse of patients.

The claim was made that ad hominem 'impugning of character' was anti-science. But this was a transparent conflation of two different issues. A psychiatrist who gives a lecture every time a patient insults them is unlikely to have much time for clinical work. It is part of the job and has no effect on research practice or reputation. What damages scientific reputation, and rightly so if justified, is pointing out that the research is poor quality. That was what White was upset about. And the pointing out was justified. Adding to that 'impugning of character' seems not unreasonable since it has derived not from a 'difference of opinion' but from there being every reason to think that someone is acting out of self interest.

Over the last five years I have gradually come to learn just how appalling the execution of these studies has been. Not only do they break basic rules for generating reliable evidence but we know from FOI requests how the massaging of results came about and has been repeatedly covered up (with vast sums spent on refusing to share the facts). I keep thinking that I have heard the worst only to find that I haven't. The patients opened my eyes. Some may oversimplify but most of those are just expressing the only way they can understand things and it makes sense. And as a community S4ME produces level of scientific debate that puts most academic meetings to shame.

This is not an area where there is reasonable difference of opinion. I presented my analysis of the situation to the UCL Division of Medicine Grand Round and not a single person raised disagreement when I asked for a show of hands. The one person, who initially abstained, on hearing the case said it was of utmost importance that the weakness of the studies was brought to the notice of NICE. Only people with a vested interest disagree with the principle that open label studies with subjective endpoints are essentially worthless. The problem I perceive is that in the 'methodology/quality control' business vested interests come in a remarkable range of flavours. (We have seen some strange goings on recently with shift in the Risk of Bias Tool.)

I understand the process that has been set in motion and that it is hard to see another way forward. But I think the patients have every reason to think that they are being let down. You only have to look at what is going on in Norway at present with the Lightning Process to see how much the rot is still there. I think we all owe it to Bob to do better. If we can't at least try to do that medical science ceases to have legitimacy.

 

Thanks! Cochrane doesn't, but I agree, and I am advocating for that. (My 30 May blog post: https://absolutelymaybe.plos.org/20...bout-bias-in-protocols-of-systematic-reviews/ .) Systematically looking at the concerns that have already been expressed about the Cochrane review will also be one of the starting points for the advisory group.

(The lead author for that update of the AHRQ review is one of the coordinating editors of Cochrane's Back and Neck Group. Another Cochrane editor chaired the recent NHMRC process on ME/CFS.)

 

This is a bit off-topic but I wanted to say this is completely true, and I say that as a non-patient examining this from a professional perspective. My colleagues at Berkeley and elsewhere had simply believed what they read. When I brought them the PACE trial, they were flabbergasted at the methodological and ethical lapses. That's why PACE has been used by my epidemiology colleagues as a case study of terrible research in graduate seminars. But until you look at the trial, it is hard to understand that someone--in this case, the Lancet--published a study in which participants met outcome thresholds at baseline. And you still have The Lancet and every academic and medical authority in the UK defending this trial-including Cochrane. it is incredible.

 

I agree about Robert Courtney. And I have thoroughly expressed my great anger about how he and Tom, and their concerns and published comments, were mistreated. That's why a key starting point for me is to revisit those comments now (as well as those of others). The only remedies available to me are, as you say, to ensure it's done better this time; and to retrospectively properly respect their important contribution in multiple ways, even though it is too late.

I agree people with ME/CFS were badly let down by the previous process, and have good reason for expecting no better as a result. This early in the process all I can do is acknowledge that, and point to signals that it's not business as usual. I think the editor-in-chief choosing me for this role is one signal - she knew that was embracing disruption and high levels of engagement, and you will see the first results of that soon. There will still be disputed issues - everyone isn't going to agree about everything. While that's inevitable, it won't be because there's no listening, serious consideration, and response. Secondly, new protocol with new authors and editorial process is another signal that this is a re-start, not a continuation.

 

Thank you for that, @Hilda. I had not seen that before. Good blog

On public comment... Unlike AHRQ, CDC is only providing for public comment on the draft review, not the draft protocol. There was a key informants panel, which I was on, that provided input on the key questions and other points as part of the protocol development process. But while it included members of the public, it was not open to everyone

Regarding the point about "How this intervention might work" - it's frustrating that in postulating how GET might work, the authors ignored evidence of biological impairment that would undermine that theory. If Cochrane could require a fuller accounting of such evidence in the answer to that question, it would help those assessing the findings and conclusions.

 

Beautifully stated. Truthfully stated. Clearly stated. Goes straight to the heart of the matter. So grateful for your perseverance and presence here with us and on behalf of ME patients around the globe. Many, many thanks Dr. Edwards.

 

Thank you, Jonathan. That's a concise and moving post.

 

It is good that Hilda is approaching this the way she is doing. This will give credibility to the new review.

My own conclusion is that a difference of opinion doesn't have the explanatory power to explain what is happening. It looks like the PACE trial authors and some of their allies are willing to do whatever it takes, even hurting patients, to advance certain interests. One doesn't start a professionally conducted smear campaign against a patient group over differences in opinion. One doesn't spend a quarter million pounds for lawyers to hide clinical trial data over differences in opinion. One doesn't define recovery in a way that allows severely impaired patients to get worse and be counted as recovered without a clear intent to obscure the truth.

Their behaviour has many similarities to that of industry financed scientists that want to shape public perception of issues like climate change and health effects of smoking. I don't think it's a coincidence that all three principal investigators in the PACE trial had ties to the health insurance industry (ties they failed to declare to the PACE trial participants). They are dishonest, manipulative and well financed. It's a battle where dirty tricks are used to ensure certain views prevail over the truth.

What interests and views are being advanced exactly? There are several. One is that of the health insurance industry who doesn't want society to believe that ME/CFS is a serious medical problem because it would cut into their profits. They want ME/CFS to be seen a problem of fatigue, and it classified as mental health problem or psychosomatic disorder. Another commercial interest group is that of the CBT industry and various clinics. This group wants society to believe that CBT/GET is effective. I've been told by a patient that one of the Durch CBT proponents works in a clinic where a rehabilitation course can cost up to something like €45000, paid for by taxpayers (I don't remember the exact amount because my memory is bad but it was about that much). Finally, another major interest group are government agencies responsible for welfare benefits. They want to reduce expenses and therefore have much the same interests as the health insurance industry.

There of course also a lot of people who are biased towards exercise and therapy being good and safe. They are perfectly happy to believe in a story about irrational activists attacking good science because they have only very superficial knowledge.

Some doctors like ME/CFS being viewed as "not a medical problem" because they find these cases frustrating to deal with, and would like them to be someone else's problem.

The reason these interest groups are prevailing is perhaps mainly because of a lack of effective opposition.

The pharmaceutical industry is absent in all this. If there was a pharmaceutical treatment, I suspect they would put effort and money into changing public perception and combating underdiagnosis.

Patients struggle to assert themselves for several reasons. The illness incapacitates them which makes it hard to do effective advocacy. Patients are poor which makes it hard to fund pilot studies that could develop diagnostic tests and find objective markers of the illness which would end a lot of nonsense. Most patients are not even diagnosed. The patients community is fragmented (eg. the ME vs CFS debate). Patients don't look as sick as they are, standard blood tests look completely normal and their disability doesn't fit with how society pictures disability. The CBT/GET model also directly attacks the credibility of patients as witness to their own condition.

 

Thanks for the comments @Hilda Bastian.

I am actually more optimistic than some here. I see reasons to think that by the end of this year there may be some positive outcomes on long standing issues.

 

Sure. I am unfairly speculating because of the timing. Under the previous looser policy, Cochrane withdrew Chinese medicine for CFS review in October 2018 because they were doing a strategic review of the portfolio, including the CFS reviews.

Reason for withdrawal for publication

Supporting better health for those receiving care is central to Cochrane’s mission to promote evidence‐informed health decision‐making by producing high quality, relevant, accessible systematic reviews and other synthesized research evidence.

The Cochrane Common Mental Disorders Group is currently undertaking a strategic portfolio assessment of all existing and planned reviews and protocols, including those relating to Chronic Fatigue Syndrome (CFS). During the course of our portfolio assessment, we are aiming to ensure that the most up to date evidence is available in the Cochrane Library to inform decision‐making. This latest version of the review was published in 2009. The Cochrane Common Mental Disorders Group are therefore temporarily withdrawing this review, with the full support of the author team, to allow time to determine whether the evidence is up‐to‐date.

The editorial group responsible for this previously published document have withdrawn it from publication.

It would make sense to withdraw the CBT for CFS review as it is even more out of date than the Chinese medicine one. However, now there is a policy in place to justify it staying on the library with no obvious plans to update it. I'll write to Cochrane to find out a bit more about the reasoning, process and the timing of the new policy, especially as it now allows very out of date reviews to remain on the library. Maybe there is some discretion when a review is so seriously out of date. As you say, I'm sure there was a lengthy internal process leading up to such a significant change in policy. I think it's reasonable to ask for some information about the process which led to it, so I will do that. I realise FOI doesn't extend to charities, so there's no obligation for Cochrane to make this information public, but it's always worth asking! Even Fiona Godlee has said that paradoxically retraction is a good thing. https://www.cbc.ca/news/health/bmj-fiona-godlee-science-1.3541769. Yes, it would be interesting to see what PubMed are doing these days.

 

That's easy to answer: the status you're discussing from October 2018 has been overtaken by events. Being out of date is no longer a justification for withdrawing a Cochrane review, so it wouldn't apply. The responsibility for CFS is being transferred away the Common Mental Disorders Group, so their portfolio issues are irrelevant as well. We have discussed it recently, and it's clear that decisions that will be made for the exercise review will have implications for that review over and above the issue of its outdated evidence base. Don't want to divert focus from the exercise review at this particular critical stage, as that would slow it down, and it's a priority. But the implications for the other review are high in people's minds, and once we've established some basics, it will be easier to tackle it.

 

Thank you for this. All we want is someone to look honestly, without prejudice, at the evidence because there is no way an honest person could look at all the evidence without seeing the flaws and upholding our case.

 

I agree so much Trish. Clicking "like" for some of these posts does not give a proper sense of the gratitude I feel. For years, decades, no one outside our community seemed able to see the flaws in what was being said about us. It was surreal and I could not understand it. The fact that people of intelligence and status have looked at the evidence and seen what we could see cannot be underestimated.

Not only does it give me hope for my future and that of my children (since there is a high familial risk) but it has (almost) got rid of the feeling of how could I be right in what I felt about my life and how I experienced my illness when no one else acknowledged it. That undermining of self may be one of the worst things done to me over the years. Many disease have little that can be done for them but to be told it is all imagination destroys you.