Trial By Error: A Plea to Fiona Godlee on a Familiar Topic

http://www.virology.ws/2019/05/16/trial-by-error-a-plea-to-fiona-godlee-on-a-familiar-topic/

 

Thank you @dave30th .

 

Can somebody point me to the editor’s note on the original LP paper in the Archives of Disease in Childhood, please. Thanks.

 

https://adc.bmj.com/content/103/2/155.responses

 

Hi, there are many good arguments against the trial, including the ones you raise. The ones I have focused on are ones that even the journal has to agree are violations and warrant retraction. They've acknowledged as much in the editor's note. So I'm just pushing them to take the step based on their own acknowledgement of these specific flaws.

 

I don't understand how a completely opaque treatment is allowed. Nobody knows what's in it because it's a commercial product. It's a complete black box to anyone interpreting the results. That makes a complete mockery of the scientific method. Those claims are on the same level as scientology. BMJ are risking their entire reputation over this. How could anyone take a medical journal seriously after having allowed that?

 

They seem paralyzed by the fact that being outside the UK, David can't be bullied away, which is what usually happens to anyone questioning this type of research. It usually does go away. This time it won't and after decades of a complete void in accountability where nobody challenges obvious pseudoscience, they don't know how to deal with it in the normal process of actual science. This stuff is usually rubber-stamped and defended without consideration for substance.

 

Thank you - much appreciated

 

Even Professor Dorothy Bishop, who has previously supported BPS approach to ME was very lukewarm in her response to SMILE in the SMC briefing.
http://www.sciencemediacentre.org/expert-reaction-to-controversial-treatment-for-cfsme/

SEPTEMBER 20, 2017
expert reaction to controversial treatment for CFS/ME

Prof. Dorothy Bishop, Professor of Developmental Neuropsychology, University of Oxford, said:

“The gains for patients in this study do seem solid, however, I am still rather uneasy because while the patient allocation and statistical analysis of the trial appear to be done to a high standard, the intervention that was assessed is commercial and associated with a number of warning signs. The Lightning Process appears based on neurolinguistic programming, which, despite its scientific-sounding name, has long been recognised as pseudoscience.

“I am sympathetic to the authors’ decision to evaluate the Lightning Process (LP), given that they had patients who had used it and reported favourably on it, and it could be argued that to fail to do so would indicate a degree of closed-mindedness. But the commercial nature of LP really creates problems. We cannot tell which aspect of LP is responsible for the gains in patients who took part.

“I noticed, for instance, that LP involves group sessions, whereas the comparison group undergoing standard medical care were treated individually. So it may be that the benefits derive from interacting with other children with chronic fatigue syndrome/ME, rather than the specific exercises and training. This is, of course, something that could be investigated in future research but meanwhile the concern is that this report will in effect act as positive publicity for a programme that is being proposed for a wide range of physical conditions (including chronic pain, low self-esteem, multiple sclerosis, and depression, to name just a few) and has to date been promoted largely through celebrity endorsements.”

 

That is a constant with this type of research. They appear solid, yet systematically crumble under minimal scrutiny. Merely declaring it high quality and low bias does not make it so. If it appears to be solid yet obviously based on pseudoscience, that's telling you something!

There is a good reason why subjective research is normally soundly rejected in medicine. Creating an exception based entirely out of prejudice is not an appropriate justification. Psychosomatic models of peptic ulcers also appeared solid. So did phrenology, to some anyway, and the cancer personality nonsense. Which is why it is critical to look at the substance, not the style. This whole body of research is built on the mere appearance of science, wholly lacking in even the barest minimum of substance.

That's supposed to be a gigantic red flag. This mistake not only happened many times in the past, it is at the very center of the crisis of replicability. All of it will inevitably end up collapsing under the weight of its massive incompetence. Meanwhile it's millions of lives being toyed with by mediocre careerists who place their ego above millions begging for help and pleading for sanity.

Until there are serious consequences this will keep happening over and over again. None of this is innocent, these people are recklessly ruining millions of lives over a gut feeling and are supported by institutions who don't expect to pay any price for enabling obvious garbage pseudoscience.

 

The comments by Bishop are interesting because they reveal a serious degree of blinkering. She is happy to be sceptical because the treatment is commercial. Everyone can be sceptical for that reason. But she seems to think the methodology was OK.

She picks patient allocation and statistics as her yardstick of methodology. So why not pick subjective outcomes and the risk of bias. Presumably because she finds it uncomfortable to raise this, just as the referee of my JHP paper did, because it might cast doubt on a lot of stuff - maybe some of hers.

She rightly points out that we have no idea which bit of the lightning process is effective. But she misses the point that we have no idea if any of it is effective. Moreover, we have no idea which bit of any psychotherapy is effective - and very likely no idea if any of it is effective.

The Pied Piper theme rises again. Psychotherapy is excellent for rats but hang on - something all too similar seems to be carrying off all the children and thousands of other people with unexplained illnesses. Could it be that Dr Bishop is a bit worried that psychology will lose business?

 

https://vadamagazine.com/lifestyle/health/esther-crawley-claims-harassment-university-no-record (I think Adam is a member here) ...

https://www.coyneoftherealm.com/201...nal-providers-in-esther-crawleys-smile-trial/ ...

Same quote from Christine Fenton being reported. As a retired Head Teacher I would think she is well placed to be making that statement.

 

Be fair. This was a SMC Expert Reaction. She may merely have been making the points which science journalists might understand and be likely to report. Who knows?

 

Do we know that the adults who had access to the children were properly screened? Is this described anywhere? When I did work involving access to papers about children ( I never actually saw a child in this role), I had to get police screened etc, Have forgotten the acronyms for the process.

Were children interviewed/ talked to about this? If they were, or if my experience is valid, then there should have been formal processes followed by anyone who interviewed them.

The charity I was working for followed the usual procedures. Did this research do that? There is clearly an issue about children being taught to lie about what was happening to them @Trish but were the normal procedures about access to children followed?

ETA: used to be known as CRB checks, which I was subject to

Now DBS checks https://crbdirect.org.uk/?gclid=EAIaIQobChMI39LkqNGg4gIVibPtCh0OSQjdEAAYASAAEgJk0fD_BwE

 

Does anybody understand what Sharpe wanted to say here (as one of the experts asked by the SMC)?

The first sentence seems just strangely worded, but the follwing sentences? Perhaps I don't understand the meaning of the words "however" and "likely" in this context?

(The preceding sentence is: "This is a robust study because patient were allocated to one of the two treatments at random ensuring that any difference seen in outcome between these treatments, is not due to pre-existing differences in the patients.")

And that's how he describes LP:

http://www.sciencemediacentre.org/expert-reaction-to-controversial-treatment-for-cfsme/

[edited -- typos]

 

I think there must be a misprint somewhere. He must have meant unlikely.

The unblinded rituximab study is interesting here because the spurious positive effect lasted for three years. Sharpe and others have been trying to deny bias on the basis that it is unlikely to affect longer term results. But there is no solid basis for that.

It actually looks as if Bishop was primed by Sharpe (who would have thought it). Patient allocation and statistics are what matter - not the fact that patients had been trained to say the right thing.