BMJ Archives of Diseases in Childhood: ''Editor’s note on correction to Crawley et al. (2018)'', 2019, Nick Brown. (SMILE LP Trial)

Retraction Watch has mentioned it in their weekly news summary

Weekend reads: Researcher resigns following questions about ties to China; grad student's suicide sparks misconducts investigation; study of chronic fatigue syndrome corrected

• “BMJ policy requires prospective registration of randomised trials but we do not consider a failure to enforce that policy grounds for retraction.” A BMJ journal corrects a paper about a method of treating chronic fatigue syndrome in children.

 

Maybe. But surely that would have shown up in their feasibility sutdy, prior to converting to a full trial. And if that was the case shouldn't they have reported it rather than claimed "that self-reported school attendance lined up very well with the schools’ records of attendance" [as reported at Buzzfeed]?

 

Feasible for what? Getting the intended results?

As per their "clarification" (aka admission) here for instance, that the trial was indeed not prospectively registered.

Clarification of an infringement does not somehow make it acceptable.

 

Which is definitely something that would be checked beforehand.

Otherwise it's like beginning work on a bridge and just hoping whatever municipality has jurisdiction on each side will say yes once you start digging. Makes no sense, it's a clear design flaw that should have disqualified the trial as it would be entirely useless without reliable data.

Then again being entirely useless did not stop it from being published so it sounds more like a case of "we really, really wanted to but shucks, well, we gave it our best so this will have to do".

Easy to forget that this is not serious research and instead is merely an exercise in fabricating evidence for conclusions that have been accepted as axiomatic.

 

That's a standard that is found throughout the psychosocial body of research: "clarifying" something by merely describing it in dry terms. It's not theft, it's removal of property without informed consent. Pretty much all the "clarifications" offered for PACE follow this model. Well, that and "we just prefer this outcome so whatever". It's quite similar to handling conflicts of interests by merely declaring them (often after the fact) and saying it probably has no impact. Basically it's hand-waving, shows that they are exempted from normal rules and standards.

It's used a lot by dishonest politicians. It's sad to see this being normalized in medical research, this carving out of exceptions from the norm by simply not thinking they are required because it has been pre-determined that we're not even sick and so the normal ethical rules don't apply here. Same as patients' rights don't apply if you're not considered a patient.

The editors' note itself even states that pre-registration is a "requirement", a word that is as unambiguous as it gets, but that here it's somehow not required. It's required but it does not matter. There's an odd exception but it's not detailed or even acknowledged beyond "whatever". Typical of people who know they are above accountability.

 

Aha. On the Parental Consent to Study form, there is a box for "I agree that my child's school attendance records may be checked."

However, I'm not sure how easy this would have been to do given that the questionnaires were anonymised. They'd have to break the code to check the school attendance record. And each school probably has a different policy on releasing such data. Given the number of students and datapoints involved, and having to marry up dates, it would likely be a monstrous amount of work, if possible at all.

My feeling is still that they checked a small sample at baseline (the easiest to date-match) to check whether it seemed to tally. Or even that they used data from another similar study, such as the School Absence Study. But that's just supposition...

 

In which case the feasibility ‘stage’ should have picked up on the problems which should then have been reflected in the main study protocol, rather than emerging in a very confusing post publication edit when concerns had been formally raised.

 

There's another article on that Retraction Watch page that seems relevant to me!

Terence Stephenson: Reducing questionable research practices and biases
July 4, 2019
The greatest threat to medical science is not fabrication of results but “presentational fraud,” it is also the hardest to deal with, says Terence Stephenson

Whole article is quite an interesting read.

 

Although not necessarily. They were clear that they didn't look at the data during the feasibility stage ["There was no analysis of any outcome data during or after the feasibility phase until the entire trial was completed"], so they won't have looked at potential problems with the data. They say that the feasibility phase was all about whether it was feasible to run the trial, mainly in terms of whether it was possible to recruit enough kids into the trial, randomise them, and collect the [questionnaire] data. There was no analysis of any outcome data.

It does seem that the decision to change SA to a secondary measure was based on qualitative feedback from participants, as they said:

However, there is also evidence of conflict between the interventions, where the emphasis of LP seems to be to get the participants back to full SA as soon as possible, whereas SMC is more about improvement, however much or little that may be:

So while some had chosen not to increase SA despite increased activity, it may have been the reverse in the LP group, due to the intervention telling them something different.

They of course neglect the possibility that the intervention messages will also have an affect on the reporting of the SF36-PF subscale that could make it divert from actual activity/severity.

 

It would be rather strange if they didn't look at any outcome data. Isn't it part of the feasibility trial to see if they can collect the outcome measure they have in mind for the full trial?

 

Judging by the BMJ response they would correct a given paper down so much that it read as the full McDonalds menu and they would still publish it.

Amazing what you could come up with with an infinitive amount of monkeys and an infinite amount of typewriters.

 

They did what they could, I think. They collected the SR SA data OK, and got consent to check school records (as a 'sensitivity' check), but presumably had problems further down the line that they couldn't have anticipated at the feasibility stage because they weren't focussed on that. And once they'd made the decision to downgrade it to a secondary outcome, it becomes less important, I guess.

 

Terence Stephenson is speakng at EBM live next week @Caroline Struthers Mon at 13.30.
Reducing Questionable Research Practices and Bias

Terence Stephenson

We are all familiar with straightforward fraud. The Lancet paper by Andrew Wakefield alleging evidence of a link between measles, mumps and rubella immunisation and autism is perhaps the most high profile example. The potential consequences of research fraud can be seen from the impact on measles uptake. More than half a million children in Britain have been left unprotected against measles in the past decade. In the first three months of this year more than 110,000 measles cases have been reported worldwide, up a quarter on the same period last year.

Much more difficult is what I would describe as ‘presentational fraud’. In these cases, there has not been true fabrication and the data which are published are not false – but not all the data are published or they are presented at research meetings in a misleading format.

The English word morality comes from the Latin mores meaning the essential or characteristic customs and conventions of a community. To me ‘Presentational fraud’ is immoral in breaching the expected conventions of scientific integrity. Wolfgang Gaissmeier has drawn attention to examples from the field of prostate cancer and Muir Gray has noted the use of the ‘framing effect’ in contrasting benefits and risks, especially in relation to drug trials.

Methodological failings in study design can introduce bias into research which can be unwitting or deliberate. These should be picked up by peer review and good editorial processes but peer review has become less robust with the explosion in online journals. Also, good peer review is time consuming but brings few rewards in modern academia which means obtaining good refereeing can be difficult. If allied to editorial eagerness, this can be a toxic mix.

Research data obtained with a lack of proper informed consent may be scientifically correct but this is unethical. The subsequent use of such data poses a particular problem.

 

7 March 2018




Our Nuffield Professor of Child Health, Professor Terence Stephenson, was recognised in the New Year Honours list for his services to healthcare and children's health. (2018)

Terence is Nuffield Professor of Child Health at the UCL GOS Institute of Child Health and Honorary Consultant Paediatrician at UCL Hospitals NHS Foundation Trust & Great Ormond Street Hospital for Children NHS Foundation Trust. He is also Chair of the General Medical Council (GMC) and before that he was Chair of the Academy of Medical Royal Colleges and President of the Royal College of Paediatrics and Child Health.

Professor Rosalind Smyth, Director of the UCL Great Ormond Street Institute of Child Health said: "Terence has made an enormous contribution throughout his career to children's health and also to the integrity of clinical care more widely in the medical profession. We are delighted that his contribution to healthcare and children's health services has been recognised in this way."

 

It's also impossible to verify and dubious considering that there is no blinding and there is constant interaction with the participants.

It's as credible as saying a weight loss trial did not weigh anyone so they could not possibly know in advance whether there was any change. It's a context in which it's easy to guesstimate and have a good idea of what the data will show.

Hence the issue with unblinded trials. Assurances like that become meaningless, especially in people who clearly use controversial methodology and have been accused many times of cheating and reinterpreting their data. Crawley is simply not a credible source for this level of trust.

 

Trial By Error blog post by David Tuller:
Our Exchange with BMJ Journal about "Correction" of LP Study

Last Thursday, Professor Racaniello and I received an e-mail from Dr Brown. He alerted us to the news that a “corrected” version of the Lightning Process study had just been posted, along with a lengthy correction notice and a new editor’s note explaining the changes. He thanked us for bringing the issues to the journal’s attention. I have posted that message below, followed by our response, which Professor Racaniello sent this morning.

I consider it a major victory to have Archives of Disease in Childhood acknowledge that everything I documented is true. The correction notice itself is unusual in its length and in the sheer number of points it needs to cover. But allowing the investigators to re-affirm their original reported findings despite the study’s disqualifying methodological and ethical violations represents a perplexing abrogation of BMJ’s editorial responsibilities.