BMJ Archives of Diseases in Childhood: ''Editor’s note on correction to Crawley et al. (2018)'', 2019, Nick Brown. (SMILE LP Trial)

Kalliope

Senior Member (Voting Rights)


BMJ has "corrected" the Lightning Process study but has allowed its conclusions to stand, as far as I can tell. I will have more to say about this. Apparently, the authors have convinced the journal that the outcome swapping had nothing to do with the fact that the revised primary outcome had positive results: "The process has additionally involved seeking assurance from the authors that the change in primary outcome was not influenced by (positive) findings in the feasibility phase."

Everything was an innocent mistake after all!

https://adc.bmj.com/content/early/2019/07/11/archdischild-2017-313375ednote?fbclid=IwAR3-LBW7tdYPwycVC0p0ozmkjdT6bzMx9nz0XVrrfPk97Rp59e1vafba3P0
 
Last edited:
It shouldn't matter whether or not the editor believes it was all an innocent mistake, what matters is that the BMJ should never have published it.

"The process has additionally involved seeking assurance from the authors that the change in primary outcome was not influenced by (positive) findings in the feasibility phase."

Would a pharmaceutical company be allowed to offer such a gentlemanly assurance?
 
Out of curiosity I had a look at all the threads we have on David Tuller's efforts to get this dealt with. Here are all the threads I found:

Dec 2017
Tuller / Trial By Error: The Crawley Chronicles, Resumed

Blog: David Tuller, "Trial By Error: My Questions for the Science Media Centre"

January 2018
David Tuller: Trial By Error: Professor Crawley’s Bogus BuzzFeed Claims - 17th January 2018

Trial By Error: A Letter to Archives of Disease in Childhood

February 2018
Trial By Error: A Letter to BMJ Open

March 2018
David Tuller - Trial By Error: My Exchange With Archives of Disease in Childhood

April 2018
Tuller, Trial By Error: Our Latest Tango with BMJ Open…

May 2018
David Tuller: Trial By Error: NICE's Consideration of the Lightning Process

June 2018
David Tuller: Trial By Error: BMJ Still "Looking Into" Lightning Process Paper

David Tuller: Trial By Error: My Letter to MP Monaghan About BMJ Studies

David Tuller: Trial By Error: A Letter to Health Officials About BMJ’s Lax Editorial Standards

David Tuller: Trial By Error: My Letter to the Science Media Centre about BMJ Study

David Tuller: Trial By Error: More Letters About BMJ’s Flawed Pediatric Studies

David Tuller: Trial by Error: My exchange with Professor Bishop

July 2018
David Tuller: Trial By Error: My Letter To Fiona Godlee

David Tuller: Trial By Error: Waiting for Godlee

David Tuller: Trial By Error: The Contentless “Editor’s Note” About the Lightning Process Trial

August 2018
David Tuller: Trial By Error: Another Letter to BMJ's Dr. Godlee

October 2018
David Tuller: Trial By Error: My Latest Letter to Archives of Disease in Childhood

David Tuller: Trial By Error: Where's My Apology, BMJ Open?

November 2018
David Tuller: Trial By Error: Yet Another Letter About the Lightning Process Study

January 2019
David Tuller: Trial By Error: My Letter to LP's Study's Senior Author

Trial By Error: Bristol Investigating Crawley Papers

February 2019
Trial By Error: Letters to Fiona Godlee and Nigel Hawkes

April 2019
Trial By Error: My Latest Letter to Bristol’s Legal Department

May 2019
David Tuller: Trial By Error: My Most Recent Exchange with Bristol

Trial By Error: A Plea to Fiona Godlee on a Familiar Topic

Trial By Error: The Lightning Process Is “Effective”? Really?

June 2019
David Tuller: Trial By Error: Time to Retract the LP Study; Letter to Archives of Disease in Childhood

Trial By Error: Another Review Mentions LP Study and Prompts More Letters

Trial By Error: An Update about the Pediatric MUS Systematic Review

July 2019
Trial By Error: FOI Response from Bristol about LP Study; Correction in BJGP about MUS
 
wow! what kind of an insane person would devote that kind of energy on such a piece of garbage??

And the really annoying thing is, now they tell us that while you were doing all that work, they were quietly getting on with a process of editing and reviewing. They could have informed you of that at any stage, and saved you all that trouble.

And after all that, they cravenly gave in to Crawley et al's ridiculous claim that the outcome switching was not influenced by the results of the pilot.

Piffle I say.
 
BMJ has "corrected" the Lightning Process study but has allowed its conclusions to stand, as far as I can tell. I will have more to say about this. Apparently, the authors have convinced the journal that the outcome swapping had nothing to do with the fact that the revised primary outcome had positive results: "The process has additionally involved seeking assurance from the authors that the change in primary outcome was not influenced by (positive) findings in the feasibility phase.

How exactly have the BMJ issued a correction in this case?

Which protocol now appears as the original one on the paper and how is history revision permitted in such circumstances?

If they have outlined the correction to show the original protocol doesn't that change the "positive results" when matched to the outcome swapping?
 
Last edited:
It looks like the original version of the article is still available on PMC, and I archived a copy here in case it's useful: https://web.archive.org/web/20190711172758/https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5865512/

I tried to compare the text here ( bit of a mess - I had trouble getting this to work and I didn't compare the tables - anyone want to try to clean things up? I fear my poor attempt could lead to people missing important changes): https://www.diffchecker.com/AvCo4xsy

Edit: I'm an idiot and wasted my time - they listed the changes here - what follows is just the bits I pulled out on my own so I'd recommend reading their info: https://adc.bmj.com/content/early/2019/07/11/archdischild-2017-313375corr1

It looks like they removed this:

LP practitioners have completed a diploma through the Phil Parker Training Institute in Neurolinguistic Programming, Life Coaching and Clinical Hypnotherapy. This diploma is examined through written and practical examinations and is accredited by the British Institute of Hypnotherapy and NLP. Following the diploma, LP practitioners undertake a further course to learn the tools and delivery required for the LP after which they must pass both a practical and written examination. Practitioners undertake supervision and continuous professional development in order to further develop their skills and knowledge. They are regulated by the register of LP practitioners, adhere to a code of conduct and there is a Professional Conduct Committee that oversees complaints and professional practice issues.

Added stuff:

We tested the feasibility of running this trial with a feasibility phase (29 September 2010 to 18 September 2012). We determined the trial was feasible in June 2012 and registered the full trial (31 July 2012). We applied for an amendment to recruit children into the full trial as opposed to a feasibility trial (see online supplementary table 1 for detailed description of amendments). Full trial first randomisation was on 19 September 2012. We continued seamlessly with participant recruitment without any interim between-group comparison of participant outcome data from the feasibility phase. Children from both phases (feasibility and full) were analysed.

The primary outcome was the SF-36 physical function subscale (SF-36-PFS)26 analysed as a continuous variable collected at 6 months after randomisation. We chose the SF-36 based on qualitative work conducted in the feasibility phase of the study.24 We have reported that parents and participants ‘commented that the school attendance primary outcome did not accurately reflect what they were able to do, particularly if they were recruited during, or had transitioned to, A levels during the study.’ In addition, ‘we were aware of some participants who had chosen not to increase school attendance despite increased activity.’ We therefore concluded that: ‘trials involving 17 and 18 year olds should consider alternative primary outcome measures to school attendance as it is difficult to assess for those transitioning from General Certificate of Secondary Education to A levels, and may not be appropriate for those who do not consider school attendance their primary goal.’ At this stage, our recommendation was that a ‘full study uses other primary outcomes, such as the SF-36 or the Chalder Fatigue Scale and uses school attendance as a secondary outcome.’ These findings informed our application for our ethical amendment to a full study in 2011 (see online supplementary table 1) and were published in our feasibility paper in 2013.24

Qualitative interviews with Specialist Medical Intervention and Lightning Evaluation (SMILE) participants then formed part of a larger study which described the conceptual model for paediatric CFS/ME.27 In this study, physical activity (or disability) is described by children as being pivotal because of the impacts on social participation and emotional well-being. While school was deemed to be an important contextual factor, these qualitative results led us to choose the SF-36-PFS as a primary outcome with school attendance as a secondary outcome. There was no analysis of any outcome data during or after the feasibility phase until the entire trial was completed.

Secondary outcomes were the SF-36-PFS at 3 and 12 months, and school attendance (days per week), the Chalder Fatigue Scale,28 pain (visual analogue scale), Hospital Anxiety and Depression Scale (HADS),29 Spence Children’s Anxiety Scale (SCAS)30 and quality-adjusted life-years (QALY, derived from the EQ-5D-Y)31 at 3, 6 and 12 months. At 3, 6 and 12 months parents completed an adapted four-item Work Productivity and Activity Impairment: General Health questionnaire (V2.0)32 and a resource use questionnaire assessing their child’s health service use (eg, general practitioner or specialist care), educational service use (eg, school counsellor), health-related travel and other family costs.

136 consented to receiving further information and 100 were randomised: 49 to SMC only and 51 to SMC+LP. Fifty-six of these participants were included in the report of whether it was feasible to conduct this randomised controlled trial.24

Finally we get info on their original primary outcome... they didn't have 'capacity' to collect it:

We did not have capacity to check school attendance using school records, but this could have provided an objective outcome. Further unpublished work suggests this is highly correlated with the self report measure we used. Only 36 (70%) of those allocated LP attended the full course prior to the 6-month follow-up but we estimated the effect in all those who completed the full LP course. The study was originally planned as a feasibility study and although randomised was not registered on a trial registry, since the aim was to investigate feasibility rather than effectiveness of the intervention. After establishing feasibility, we applied to register the full trial in June 2012. At this time, the results of our feasibility work suggested we could use either SF-36-PFS or Chalder Fatigue Scale or both which we registered as primary outcomes. We decided to use just the SF-36-PFS and published this in 2013 and in our analyses plan. We did not update the ISTCRN site until 2018, however, we uploaded the relevant publications in 2016 and the study website had the updated analyses plan.

Are they planning to correct the ISRCTN (they got the acronym wrong) page which still falsely claims that the trial was prospectively registered?
 
Last edited:
This is pathetic. How is any of this seriously worth damaging BMJ's reputation like this? I know they expect no consequences for it but inevitably there will be. This would never be tolerated in a non-controversial disease and once the veil of controversy is lifted there will be hard questions.

Basically BMJ's position, like PLOS's position a few years earlier, is that their rules have a secret "or whatever" clause where things that are required can be waived off for inexplicable reasons. Those rules are subject to arbitrary suspension whenever someone feels like it. Just a suggestion, no big deal. Scientific method, shientific shmethod.

The whole study becomes even more meaningless given this. And it remains, its extraordinary conclusions placed on the same level as any large double-blinded placebo-controlled trial, because... reasons. The political demand for this ideology is something to watch. No matter the cost, no matter the consequences, this insane project will be bullied through.
 
Last edited:
So this stands for NICE' s consideration. Wonderful....
Well this is the thing, at what point does a government body decide to consider whether private publications like the BMJ and the Lancet need to be adjudicated over for their own bias?

Or do they go on forever and a day just ignoring that private publications can get things appallingly wrong and then just dig their heels in knowing that they can hide behind, "governments should not interfere with science."

What happens when private publications have made a complete hash of science and realise they have to double down, obfuscate, deny, ignore and worse simply to protect their own reputation.

What happens when they are just incompetent idiots?

Do government agencies that affect public health care just have to ignore the elephant in the room and claim they have to issue advice based on the "scientific evidence" published in the private domain especially when that evidence has been bought and paid for with public funds.

Its such a cluster fuck of the public paying for their own abuse via collection of public funds by the government who are supposedly there to serve the people.

It's pathetic when it becomes even clearer that the, "scientific outcomes" often start out as political desires.

Its an unmerry-go-round.
 
Last edited:
It's still an unblinded trial with subjective outcome measures, so not scientifically useful.

And there is no long term follow up (LTFU) data as far as I know. Other trials such as PACE and FINE with transient improvements shown on subjective measures in the short term found the effects had gone by LTFU.

We also now have the data, which looks to me like a mess with lots of missing scores and some meaningless scores. @JohnTheJack I hope someone is looking closely at the data for possible re-analysis.
 
O. M. G. This is a car crash. A mega-wreck. Just look at it! LOOK AT IT!!!!

They've been allowed to correct it to admit to all sorts of shenanigans. And still keep the same Conclusions. Flippin 'eck!

Cross-linking with my analysis on the other thread.

I'm going to look at this in more detail, and judging by the timeline from the editor's note, they only released the data once re-publication was assured. It should have "expressions of concern" all over it. Like my face at the moment.

@dave30th said:
wow! what kind of an insane person would devote that kind of energy on such a piece of garbage??

lol
 
Last edited:
From the editor's note;
"statistical review and review by an experienced ADC editor not involved with handling the article previously"
Would be interesting to see these reviews as well as the "further unpublished work" that "suggests [school attendance using school records] highly correlated with the self report measure we used" and "could have provided an objective outcome" if only we had the "capacity to check" this. (EC & et al 2018)

(Edited to remove some muddled thoughts).
 
Last edited:
Back
Top Bottom