David Tuller: Trial by Error: My exchange with Professor Bishop

Indigophoton

Indigophoton

Senior Member (Voting Rights)
I recently wrote to Oxford University neuropsychologist Dorothy Bishop, who had provided a statement to the Science Media Centre about the Lightning Process study. Although she had expressed concerns about the pseudo-scientific nature of the intervention, she found it to be generally well conducted and noted that the findings appeared to be solid. In my e-mail, which I posted on Virology Blog, I noted the study’s methodological anomalies and asked her to review the documentation and consider whether she wanted to revise her remarks.
Click to expand...

http://www.virology.ws/2018/06/25/trial-by-error-my-exchange-with-professor-bishop/
 
Although she had expressed concerns about the pseudo-scientific nature of the intervention, she found it to be generally well conducted and noted that the findings appeared to be solid.
Click to expand...
So much fail :emoji_face_palm:
So if i did a trial on homeopathy, played with the trial protocol and reported its effective thats now good science? :emoji_rolling_eyes:

If i use a magic 8 ball enough times it will eventually say "All signs point to yes"
 
Prof Bishop does come across as sincere. And she seems to genuinely believe that Crawley was just testing LP out of some sort of commitment to the scientific process, in spite of having doubts about it (we know that story is false). I think she probably genuinely believes that these researchers are ethical and its just a bunch of crazy patients making a great big fuss over nothing.

But then aren't these people the real problem? These people who claim to be standing up for a better, more open science? The people who ought to look more carefully, whose opinion carries weight, but they just gloss over stuff, believe the lines the authors spin, and remain complacent? I find myself getting quite angry about it.

Bishop's quite outspoken about the horrors of psychodynamic approaches to autism (the 'refrigerator mothers' stuff). But similar harms are being done to PwMEs, and she dismisses us as no better than climate change deniers.

I'm not sure what to make of Prof Bishop's intellectual abilities. She seems to do fairly solid work and lots of it. But razor sharp? Not sure. She asked a question on twitter once which kind of shocked me. She'd done this study that involved comparing two groups that were selected to be different on some measure (I think it might have been scores on a reading or language test). In her study, she compared how these two groups performed on some experimental task. But then she combined the two groups together and ran some correlations.

This is a super no-no. If you choose two groups on the basis of their distinctness from one another, you can't then mix them all up and perform correlations. The reason is that you have taken out a lot of the natural variance by selecting people that were maximally different for your groups in the first place. This will have the effect of inflating any correlations that you find.

Anyway, she was criticised by a reviewer for doing this combined correlation, and she asked on twitter whether the reviewer was right or not. She had never heard of that before. I was a little shocked by that! I mean, kudos to her for actually having the courage to ask I suppose. But still a a bit scary.
 
She's a reactionary skeptic. She once pounced on a tweet of mine in which I was suggesting a book for a popular science prize, and she derided it as being too pop sciency(!) It was clear she had neither read the book in question or understood the research behind it, even though it should have been her area (developmental neuroscience). [Although, maybe it was because it was her area and she disapproved.] I steered well clear of her after that.
 
Last edited:
Woolie said:
Anyway, she was criticised by a reviewer for doing this combined correlation, and she asked on twitter whether the reviewer was right or not. She had never heard of that before. I was a little shocked by that! I mean, kudos to her for actually having the courage to ask I suppose. But still a a bit scary.
Click to expand...

As you point out @Woolie, it is not something you need to have 'heard of before'. It is barn door common sense. Anyone who does something like this simply does not understand how the world works. Academia is full of them, as you know. Again, it seems that the quality police are so often people of not particular quality.
 
If PACE taught me anything, besides unblinded trials relying on questionnaire being unreliable, it's that science is hard and even the pros make a lot of mistakes. Many of these mistakes are probably due to a lack of time. Having a good understanding of a problem is very time consuming.
 
That said it is very disappointing that Prof. Bishop doesn't seem to have recognized LP as exploitative pseudoscience. Kids being instructed by a guru figure to stand in "magical circles" and telling their illness to stop, at the cost of several hundred pounds. Or perhaps she really has bought fully into the delusional somatization disorder narrative which seems to make this kind of exploitation suddenly OK in the eyes of some.
 
Last edited:
strategist said:
That said it is very disappointing that Prof. Bishop doesn't seem to have recognized LP as exploitative pseudoscience. Kids being instructed by a guru figure to stand in "magical circles" and telling their illness to stop, for several hundred pounds. Or perhaps she really has bought fully into the delusional somatization disorder narrative which seems to make this kind of exploitation suddenly OK in the eyes of some.
Click to expand...

Maybe I am being hopelessly naive for asking this.

But is there a chance that some of the CBT proponents who support LP don't even know how the LP actually functions?

It honestly wasn't until the Buzzfeed article last year that I found out just how crazy the LP actually was. I thought it was ''just'' an intense form of CBT, and not as it turned out, an inbred form of CBT on acid.
 
Last edited:
Yea. My take on the Bishop reply is that she is kind of uncomfortable with the fringey-ness of LP, but can see absolutely nothing wrong with the way the trial was conducted.

This enables her maintain a position that Schrodinger would be proud of: able to support her follow SMC member, while at the same time preserving her reputation as a "proper" scientist.
 
Woolie said:
Yea. My take on the Bishop reply is that she is kind of uncomfortable with the fringey-ness of LP, but can see absolutely nothing wrong with the way the trial was conducted.

This enables her maintain a position that Schrodinger would be proud of: able to support her follow SMC member, while at the same time preserving her reputation as a "proper" scientist.
Click to expand...

While, sadly, making a fool of herself on both counts.

I wish there was some way to get through to people supporting research on LP the harm they are doing by not exposing the completely unethical practises it involves - secrecy, telling lies to patients, telling them to lie about their symptoms, pyramid selling... Something seems to have gone horribly wrong with ethics committees.
 
Trish said:
While, sadly, making a fool of herself on both counts.

I wish there was some way to get through to people supporting research on LP the harm they are doing by not exposing the completely unethical practises it involves - secrecy, telling lies to patients, telling them to lie about their symptoms, pyramid selling... Something seems to have gone horribly wrong with ethics committees.
Click to expand...
I agree that these are the main issues specific to LP. The technical flaws in the methodology of the study are a secondary issue and speak more generally to the quality of work being done by specific individuals. I think if you want to really put someone on the spot just ask them why they think the salutary findings of a study investigating the effects of brainwashing are reliable. The 'sterile' technical flaws - however egregious they may be - provide a bit of an escape inasmuch as they are fertile ground for distracting haggling and prevarication. Even if every facet of the study external to the specific machinations of the lightning process were done as well as possible, it would still be neither science, nor ethical.
 
Woolie said:
Bishop's quite outspoken about the horrors of psychodynamic approaches to autism (the 'refrigerator mothers' stuff). But similar harms are being done to PwMEs, and she dismisses us as no better than climate change deniers.
Click to expand...

She doesn't just dismiss us she propagates and promotes that myth.

Woolie said:
This enables her maintain a position that Schrodinger would be proud of: able to support her follow SMC member, while at the same time preserving her reputation as a "proper" scientist.
Click to expand...

But here comments should reflect badly on her reputation as a proper scientist in that she is supporting outcome switching.

I'm not impressed with her from the little I've read its not well reasoned arguments but tends to be quite superficial. I suspect she only quickly read the paper.

Those in positions of power and influence should be careful on how they use there reputation as trust in them can be quickly lost and reputations ruined.
 
I have serious concerns about the ethics involved with LP that I haven't seen mentioned. Parents are desperate for their children to get better so it is not like making a decision about a new car, say. Someone saying "this will cure your child and give them their life back" - parents will go to moneylenders to pay for it.

On another forum we had a mother who was upset when we didn't back her taking her daughter to it. This was someone living off benefits.

A peer reviewed "evidence based" trial finding LP gets better results than standard care only feeds into this desperation. I do not think the extra pressure parents feel is taken into account.

Saw a TV drama where unscrupulous people were selling quack cures to parents of children with terminal cancer; these so called doctors are doing the same thing.
 
Rick Sanchez said:
It honestly wasn't until the Buzzfeed article last year that I found out just how crazy the LP actually was.
Click to expand...
But the name: LIGHTNING process. It sounds religious-esoterical to me. Alone this makes LP very uninteresting for me, and I didn't even care to check about it were it not for clinical trials about it.

Sometimes it feels like they think "crap science for a crap disease".
 
Sean said:
Quite apart from the sheer idiocy and danger of it, there is something profoundly disrespectful about it, isn't there.
Click to expand...
Absolutely.

And I didn't mean all those people who care about applying real research. Those people obviously respect people with ME.
 
James Morris-Lent said:
The 'sterile' technical flaws - however egregious they may be - provide a bit of an escape inasmuch as they are fertile ground for distracting haggling and prevarication. Even if every facet of the study external to the specific machinations of the lightning process were done as well as possible, it would still be neither science, nor ethical.
Click to expand...
This is completely true. However, I'm a pragmatist and believe in pressing where you can get the most traction. The methodological flaws are egregious and should doom the study before even having to think about the ethics of the LP itself. Pointing out these flaws also makes it clear that the investigators manipulated their data and engaged in what could be considered academic or scientific fraud. The trial registration violation alone should lead to immediate retraction. The outcome swapping, and the fact that all these shenanigans were not disclosed in the paper, makes is much worse. Now, we don't yet know what will happen. But BMJ has been unable to answer the questions for almost six months now, so clearly editors there know they've been caught and are trying to figure out what to do.
 
You must log in or register to reply here.
Back
Top Bottom