Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

Then you're in luck!

This is literally what the PACE authors recently published in their own journal (the board of this journal is basically the BPS folks responsible for most of the ME-BPS literature): https://www.s4me.info/threads/patte...ndrome-2020-chalder-sharpe-white-et-al.15313/. And they plan on publishing more. We have no idea why. It doesn't appear to serve any purpose. But this is typical of BPS experiments.

In a recent CBT experiment by Wyller, one of the 2 peer reviewers plainly stated they only read the abstract and when pointed out BMJ finds no issue with that. PACE was sold on the promise that it would have objective outcomes and no one but us found issue with dropping that even though it was the whole point. After years of being pressed to justify this change (the real reason is that another trial had a null result) they simply stated that they prefer this outcome because it aligns with their expectations. I don't know what we're supposed to do with something like that but it's wrong.

In a recent trial of CBT for "dissociative seizures", which for years promised only the objective outcome of number of seizures would be significant, and yet lo and behold, they recently announced success in their endeavor even though the much-promised primary outcome, which they assured is the only outcome that counts, was null. There is simply no actual rigor in this field, only the appearance of. It's bona fide alternative medicine using the technical language of science but none of the substance.

 

@rvallee the correct term is pseudoscience. It looks sort of like science but it's actually a belief system.

 

I wrote about that here: https://www.virology.ws/2019/08/31/...nes-new-bias-guideline-is-lp-study-co-author/

One of the authors is a co-author of many studies with Professor Crawley. One was the Lightning Process study, which violated all sorts of rules designed to protect against bias. The other was the school absence study, in which the investigators interviewed more than 100 minors and their families and then published it without ANY ethical review by declaring it to be "service evaluation."

 

I’ve only managed to a read a few posts in this thread, so apologies if I’m misunderstanding or repeating anything that has been said.

First, thanks to @Hilda Bastian for taking on this task and engaging in this forum.

I apologise if I have missed the explanation in this thread but please can you you clarify exactly what it is that you disagree about and why?

I don’t see a conflict here. I want patients’ voices about our care to be heard. I want doctors and researchers to listen and take heed of my account of my symptoms and experience. And I want my accounts of my symptoms to be recorded in clinical trials. But, having listened to patients, I want scientists to test their hypothesis by designing clinical trials which produce useful, interpretable results. Where trials cannot be blinded that means using at least one objective outcome measure against which more subjective measures can be compared.

The Rituximab trials clearly demonstrated how difficult it can be to interpret the results from unblinded trials in ME/CFS. The results from the unblinded trials produced far better results than PACE, but the subsequent double-blind RCT showed the drug treatment to be no better than the placebo.

If we accept that subjectivity is a scale, can we agree there is a point on that scale where results in unblinded trials become uninterpretable (if they are not supported by at least one objective outcome measure)?

I’m guessing you know about this Hilda, but if not please read @Lucibee ’s blog: on “Whatever happened to Actigraphy?” Snow Leopard uses diplomatic language. To me the minutes provide convincing evidence that the investigators were not interested in genuinely testing the effectiveness of the therapies but rather seeking to design a trial that produced results which supported their preconceptions.

This view is supported by the correspondence in BMC Psychology in which Sharpe et al confessed: “We prefer the definitions of recovery we used to those used by Wilshire et al. [those specified in the protocol] as they give absolute rates more consistent both with the literature, and with our clinical experience.” (https://bmcpsychology.biomedcentral.com/articles/10.1186/s40359-019-0288-x)

I appreciate the genuine difficulties of trying to test the effectiveness of treatments which cannot be blinded for conditions which are difficult to assess objectively, but that does not mean that we should accept bad methodology. And PACE was worse than that because the investigators appear to have deliberately chosen to drop objective outcome measures and diverted from the protocol in order to produce results which are “more consistent with [their] clinical experience.”

[edit - missing word "tiral" and changed "interpretable" to "uninterpretable"]

 

There is also the worrying role the Sciene Media Centre (UK) played in presenting the Lightning Process trial (SMILE):

https://www.virology.ws/2017/12/18/trial-by-error-my-questions-for-the-science-media-centre/

What I don't recall now is whether the trial investigators reported how school attendance was measuerd. On the SMC webiste it's said it was an objective measure:

https://www.sciencemediacentre.org/expert-reaction-to-controversial-treatment-for-cfsme/

 

This is slightly complicated. The protocol promised they would have self-reported school attendance and they would vet that against official records. In the trial, they used self-reported school attendance and forgot to mention anything about the promise to vet those findings against official data. That didn't matter for the former primary outcome--school attendance at six months. That outcome had null results as a self-reported outcome, but the news was buried. In contrast, school attendance at 12 months had a positive result, so they highlighted that in promoting the study-without ever mentioning that they did not check that against official school records. The whole thing was a methodological and ethical disaster that has been thoroughly whitewashed by BMJ.

 

The change in question was the lowering of the secondary-outcome-newly-promoted-to-primary-outcome SF-36 from 85 down to 60 as the threshold for recovery. The cutoff point to participate in PACE was SF-36 <65, so 13% of participants met the definition for recovery of SF-36 >=60 published in the paper on the day they entered the trial (although this is something that fluctuates a lot so single scores are unreliable). This major issue has been dismissed by the authors with sophistry and no one but us seems to find issue with overlapping subjective entry and recovery criteria changed mid-way in an open label trial of behavioral manipulation.

Peter White, the primary investigator, has previously defined his own threshold of a SF-36 below 75 qualifying as severe impairment. Obviously 60 is significantly below White's own threshold of 75 to qualify as severe impairment, yet he published a threshold of 60 in PACE as satisfying a notion of recovery. 60 is about the average score for an octogenarian population.

Despite this, people keep claiming that PACE "proved" ME is psychological. A pragmatic randomized trial, which cannot infer cause. Even though it had null results. Which is dismissed by saying many small trials show the same. Even though PACE was sold as the definitive trial of those smaller prior trials and its interpretation modified to align with prior expectations, which the trial was meant to test, as admitted by the authors themselves.

A later PACE paper acknowledged there was an actual increase in disability benefits and that it is not curative, only possibly beneficial to some. The authors and Richard Horton went on a PR tour to tout the CBT/GET treatments as a cure, for those who want it, based on the PACE results. There is a wide gap between the basic facts and how they are presented, published and discussed. And they all bias in the same direction, without fail.

 

Hi Hilda, I hope you weren't pressurised and I would not like to think you are here or performing this task for any reason other than its your free choice.

Presumably one gets to be in a position like yours from a general desire and motivation to promote better science for the greater good.

I'd be interested to know at what point this becomes "sticking your neck out" , or why that would be a consideration.

 

That's a very different discussion. I was talking about scientific weakness of an argument. It really doesn't bother me personally when you label me an idiot who doesn't understand the scientific method.

 

Oh yes, it was totally my choice to enter the discussion about ME/CFS and the Cochrane review, and there was no pressure or incentive to agree to lead the IAG.

It was sticking my neck out because it would make me a target for prolific aggressive behavior, both directed at me (for not agreeing with everything or acceding to demands for what I should do), and because I'd be tagged into a flood of conversation that included it. Longterm commitment to an issue means you can't manage that so easily (eg by muting/blocking/diverting to spam, particular words like the issue's name and people/email accounts). That's a drag on your communication media & psyche, and a disproportionate time cost, when there are so many other incredibly worthwhile and more gratifying things to do that don't carry those productivity costs. (And productivity loss when you're a freelancer is also economic loss.)

The reason I got involved, knowing my taking a strong public position was likely to result in longterm engagement over the Cochrane review, was the general desire you raise, plus thinking at the time that the others who could make a difference didn't look like they were going to do it.

 

If it wasn't clear, I'm not referring to you, but the hypothetical people who "clock out", as soon as we mention points of "everyone knowing" about the importance of participant blinding in controlling biases and other methodological flaws leading to high risk of bias.

 

If the claim that started this discussion was always framed as you phrased it here, then it wouldn't have been a good example of what I mean, so the discussion wouldn't have taken place. But that's not the statement I've been taking issue with. Nor is the statement I mentioned an unusual aberration. The discourse around the Cochrane review often includes variants on the argument that all unblinded trials with subjective endpoints are known by everyone to be valueless - often citing/linking to the Students4BestEvidence blog post since that came out. (And it's just one example of a weak argument that doesn't do justice to a very strong case with plenty of very strong arguments that doesn't need to overplay anything.)

The thread about "clocking out" came from me saying I was one of the people who clocked out because of claims that I could see were problematic. So whether you intended to include me in the category or not, that was a logical conclusion to draw from the thread. (Thanks for saying you didn't mean that.)

It does no justice to serious complex topics, I know, but that's life in a time of information overload. When you're a casual bystander to debates about a gazillion topics, you inevitably form quick impressions about the strength of a case based on the strength of the arguments you see made for it, and that can drive whether or not it's something you want to learn more about or not. It's superficial, it's unfair, but no one can give every single subject the attention it deserves.

 

Thanks! I'm really sorry to everyone who picks this up at a later point, but while I'll continue to read this thread for the long haul, I genuinely don't think I can make my point of view on the very narrow issues I was discussing any clearer than I already have.

There's going to be plenty of opportunity in the coming months, though, to thoroughly discuss issues as they pertain specifically to evidence around ME/CFS and the Cochrane review. And I'm really grateful to all of you for contributing your knowledge and time to this.

 

it should be a long tradition. Maybe anybody can tell me where psychiatry is located, as it doesn´t seem to be psychology nor neurology? Or is it a technical issue, like neurology for degenerative diseases, and psychiatry for functional diseases in the brain? At least this were understandable.

But as I see it, psychiatrists are sitting on their tradition with people or patients not taken seriously in the wider past and defend anything mysterious. But maybe it´s simply too difficult to investigate dysfunctions? And it´s simply much easier to work with any CBT from the psychology, claiming that deeds influence the function of the nerves than to investigate the machinery where all deeds and believes need to rely upon? An additional problem is that medicals are not well understood and are potentially harmful of course (which may be, of course, difficult to weight against the benefit).

The point would be that psychiatrists would not be driven by themselves to clarify their position, as it would diminish the freedom, the room of possibilities (including not justified ones).

 

Hi Hilda,

I don't know whether I will be able to reply to some of your and others valuable posts within reasonable time, but I'd like to let some non-substantial thoughts here.

I wrote this over the last days, so apologies if some points are now redundant.

To facilitate reading, I will post this in two parts.

First, I'd like to second others comments expressing their appreciation for your engagement with us on the forum.

Also, apologies for my mix-up of questions and addressees in my previous comments.

Open exchange and taken seriously as a patient and at the same time as an individual wanting and being capable to roughly understand the reasoning behind the research on the illness I suffer from and the healthcare I have been offered is something I very rarely encountered in the medical world since I was diagnosed with ME/CFS 22 years ago.

From a certain network of medical and other health care professionals that invented or adhere to the cognitive-behavioral approach to ME and all 'MUS', people with ME and their advocates are confronted with prejudices about what they collectively think, what their motivations are to engage in ME research, that they don't know the real business of psychiatry, psychology, trial design etc. and therefore their (our) criticism is of no value.

Cochrane acted as a protector of this network and still does. I see that you and maybe even Cochrane's editor-in-chief can't do much in this regard even if you wanted, because there is lack of consensus at Cochrane. We patients have to live or die with the consequences for at least two more years and at the same time the only chance to change things is to engage with the IAG even if there still is no clear signal we could trust Cochrane.

It's difficult to believe how much irrationality, factual errors, and blaming is applied by some people of the CBT-approach-to-ME-network to disdain criticism by patient organizations and by simply all people, be it civil scientists or professionals, challenging their work.

Many of the critics are S4ME forum members. I find it particularly unjustified if they are presented as opponents of psychology per se, given that on S4ME all ME research is being discussed and criticized, and also much if not most biomedical ME research is assessed as lacking robust evidence or over-interpreting their findings, whereas the little proportion assessed as good research, including psychological, is both highly appreciated and constructively criticized (often people find crucial things could be done in a better way even in 'good' research).

Mostly, there is no urgent need to publicly criticize bad biomedical research on ME by pwME on other platforms than here because, except by some fringe practitioners, no general treatment suggestions are derived from that research.

Also, many S4ME members are at the same time staff of other international, national or regional patient organizations, and these don't always have the same stance on many issues. I think "agree to disagree" is daily practice on S4ME and it's worthwhile to engage in a discussion to be clear about what exactly we disagree.

I'm aware that there is much irrationality and hate out there on social media in general, and patients and their advocates are no exception to this general phenomenon. I don't mean to defend those. It's a not-organized minority without any institutional affiliation though, and I find it irresponsible and much more harmful if some highly influential healthcare professionals and their allies in some medical journals and certain media outlets associate all well-founded criticism with the general phenomenon of internet trolls or adherents of diverse anti-science or pseudo-science stances. Healthcare professionals' or other academics' and civil scientists' professional expertise is being belittled or ignored if they challenge the cognitive-behavioral approach to ME.

I want to understand and even though my illness and related personal circumstances restrict the speed and scope of what I'm able to take in, I still want and am able to learn. S4ME is a place where this happens. We all have different professional backgrounds and expertise, including medical professors and other scientists, doctors, psychologists , and we learn from each other, also and often from controversies.

I appreciate the forum culture that encourages everyone to ask and point to potential misunderstandings before they accumulate to a communication barrier. I'm always happy when people point me to factual or logical errors in my comments.

Unfortunately, when I see such an accumulation of potential misunderstanding emerging but feel not up to commenting, I sometimes become impatient and comment nevertheless. Apologies if that results sometimes in muddled posts and multiple editing.

 

(Continued from my previous post.)

So what I basically wanted to say or ask is:

1. It's good that you're here. I see that the language we use regarding blinding and subjective and (potential) objective outcomes/ endpoints etc. could be more precise to avoid misunderstandings. From the forum discussion I also gained some new insights regarding these issues and that I used some terms in a misleading or maybe even erroneous way.

However, I don't know if I will be able to take part in the discussion in the future, and anyway, I'm not one of the forum's experts with regard to trial design and the methodology of systematic reviews and relevant related issued. But maybe I'm not alone in genuinely wanting to understand the point you made here in response to Jonathan Edwards:

I think you referred to Jo's sentence "The use of subjective outcomes (alone) from unblinded trials to judge the usefulness of a treatment is valueless".

In my opinion, it would be extremely helpful to work with this sentence and to know with which amendments of this sentence you could agree.(*)

2. Many forum members never got genuine responses to their criticism of the Cochrane reviews on exercise and Cochrane's review methodology in general. Most of this criticism is posted and being discussed on other threads.

In addition, many forum members, due to their illness and related personal circumstances, aren't on the forum regularly, a couple of members have to refrain completely from posting for an unforeseeable period of time. Also, I can't omit to once again mention the late Robert Courtney. Other knowledgeable forum members recovered or found out that they were misdiagnosed and understandably now are busy with other things. So, most people who contributed to the discussion before the IAG was constituted, won't be able to engage with the IAG.

So I'd like to ask you: Are you interested in reading what forum members think is the most relevant of the criticism posted and discussed on S4ME and would you like to comment here on some points you find crucial?

(I'm not a staff member of S4ME and only speak for myself.)

(*)Edited to temporarily remove my own suggestion of amendments.

 

Thanks - and no worries: the misunderstandings will work themselves out. It's an inevitable process when someone from a different perspective arrives. The misunderstandings should be sifted from the genuine disagreements before anything is finalized - but also trying to ensure that this doesn't stretch out to a very long time.

 

Thanks, again! Point 1: no, it was a different statement by Jonathan that I discussed, and then a version of it from someone else who dropped the "primary" endpoint part. I think I've understood what the points were that people were making - although for some of them it took a while (like realizing that some thought systematic reviews could only ever be of primary endpoints of trials, as though systematic reviews were themselves hypothesis testing as trials are). I'll note this, and will come back at a later point and follow up on your suggestion of a list of points, carefully expressed - great idea!

Your point 2:

Many forum members never got genuine responses to their criticism of the Cochrane reviews on exercise and Cochrane's review methodology in general. Most of this criticism is posted and being discussed on other threads.

In addition, many forum members, due to their illness and related personal circumstances, aren't on the forum regularly, a couple of members have to refrain completely from posting for an unforeseeable period of time. Also, I can't omit to once again mention the late Robert Courtney. Other knowledgeable forum members recovered or found out that they were misdiagnosed and understandably now are busy with other things. So, most people who contributed to the discussion before the IAG was constituted, won't be able to engage with the IAG.

I agree there never was a proper response to Robert Courtney and others, and I find that egregious (and I've communicated my anger about that very clearly). The issues you raise here are why my starting point is gathering the published critiques, and beginning the IAG's work with a proper discussion of the issues raised in them. My plan is to have a public draft for discussion to see if we have missed or misinterpreted anything, and then we can build on that. It's one way to make sure people's contributions to date won't be lost, with proper follow-through and acknowledgment.