David Tuller: Trial By Error: BMJ Still "Looking Into" Lightning Process Paper

Kalliope

Kalliope

Senior Member (Voting Rights)
BMJ Still "Looking Into Lightning Process Paper"

Tuller has received via CC an e-mail from dr. Brown, editor of Archives of Disease in Childhood, stating that they had previously acknowledged receipt of his complaints, are looking into them and will respond fully when ready.

Tuller has therefore sent yet another e-mail to dr. Brown.

So why, after more than four months of investigation, have you still not managed to confirm that the Lightning Process study violated BMJ’s own policy about trial registration? Why have you not acknowledged that the investigators biased their findings by swapping outcome measures based on results from more than half the participants? Why have you not informed your readers of these methodological anomalies? Why has Dr Godlee herself not stepped in to make the appropriate determinations, since you and your editorial team appear unable to handle your responsibilities in a timely manner? Would you agree that this study is likely to impact medical treatment and public policy involving children, and that addressing the issues quickly is therefore critical?
 
I had not taken too much notice of the reference to the Lightning Process in the NICE scoping document. However, it does make clear just how close papers like this are to altering clinical practice. I have serious concerns about the SMILE study, greater than pretty much anything else, and I agree with David that it needs proper formal investigation. We cannot afford to have these concerns swept under the carpet.
 
There are lots of reasons to be concerned about the SMILE study, starting with the LP itself. However, it turned out that you don't even need to criticize the LP itself in order to show that the study is bogus. Just pointing out these flaws should be enough in any normal circumstances for a full retraction. You simply can't do what was done here. The fact that the investigators got research ethics committee approval is no excuse. The REC obviously made a big mistake in approving these maneuvers. The journal should have rejected it for publication.
 
LOL - this is the key part imo:

It would literally take no more than a few minutes to determine the accuracy of my “complaints,” as you call them, so this delay is inexplicable. All you need to do is review the paper itself, the trial registration, and the research ethics committee application in which the investigators sought approval to include the feasibility study participants in the full trial while simultaneously swapping primary and secondary outcomes based on the early results. If you have trouble accessing any of these documents, I would be happy to forward them.
Click to expand...

Their journal clearly requires trials to have been prospectively registered - how can anyone claim SMILE was prospectively registered? How can it take them months to realise SMILE was not prospectively registered?
 
Trish said:
@dave30th and @Jonathan Edwards, do you think it would be a good idea to spell out to the NICE people just what LP actually involves - just how unethical it is, especially with children. I hope they would have the sense to be horrified and to remove it immediately from consideration.
Click to expand...

Thank you for saying this. It is disturbing when children receive "treatment" where they are taught that they must stop saying how ill they are feeling.

It is the people that advocate this kind of treatment that need psychological help, so they can start bearing the harsh reality and no longer feel the need to silence the sufferer.
 
Trish said:
@dave30th and @Jonathan Edwards, do you think it would be a good idea to spell out to the NICE people just what LP actually involves - just how unethical it is, especially with children. I hope they would have the sense to be horrified and to remove it immediately from consideration.
Click to expand...

I think it has been flagged up in such a way that the people who matter will make sure they educate themselves about the problems. One thing I think is clear is that the people at NICE involved with the guidelines are not going to drift through this without making sure they know what needs to be known.
 
Sasha said:
I think this is very important. We forget that other people don't know what it involves.
Click to expand...

Sometimes I just say to myself. I'm a hypnotherapist. I've created a cure for ME/CFS, fibromyalgia and many other diseases without any evidence on any aspect of it. But, hey, I just know it will work. One day a pediatric psychologist will perform a study on my treatment. The psychologist's control for an adult telling a child to stand on a piece of paper in a circle of other children, telling him to tell symptoms to 'stop', will be standard medical care. There's nothing wrong with that. The children, half of whom were groomed for selection 2 years in advance, will self-report changes in symptoms. The researcher will be allowed to look at results and swap primary and secondary outcomes half way through.

Then I think, nah, that's too crazy it will never happen.
 
You forgot the part where said researcher, in response to criticism, then goes on a journey through academia to convince the rest of the world that she is a morally good person that stands up for children and good science.
 
Last edited:
Kalliope said:
https://adc.bmj.com/content/103/2/155.responses#editors-note
I've added it to my previous comment
Click to expand...

Thanks!

But when I googled on the title of the paper, I get this link, to an earlier version of the paper where the editor's note doesn't appear at all:

https://adc.bmj.com/content/early/2017/09/20/archdischild-2017-313375

and this latest version, where I also can't see the editor's note:

https://adc.bmj.com/content/early/2017/09/21/archdischild-2017-313375

So if the editor's note is invisible, what's the point of it?

@dave30th

An issue for all papers where the journal didn't do its initial job well and uncritically published a paper with misleading results is how they'll manage to reach all the people who've read the paper and have no reason to go back to the journal site and read it again. I think the journal should have to make an announcement in their latest issue, so as to get the attention of those readers.

Does that go on in any journal?
 
Since publication, the study has been criticised for failing to meet ICMJE and BMJ policy on trial registration and for not fully adhering to CONSORT guidance on trial reporting. The journal has been criticised for not detecting these issues during editorial and peer review. We have acknowledged these comments and reviewed our processes in relation to this paper and relating to EQUATOR guidance in general. In addition, we have received clarifications from the authors which are under editorial consideration.
Click to expand...

So they say they have carried out the review - surely if they had carried it out properly they would have found the problems and should have promptly retracted the paper, not just added an ambiguous notice. I suppose they do have to give the author a chance to explain - but why would that take time to consider - I have no idea how the problems could be explained away.
 
You must log in or register to reply here.
Back
Top Bottom