Anomalies in the review process and interpretation of the evidence in the NICE guideline for (CFS & ME), 2023, White et al

(From the no discussion thread with the responses)

Oh that's a point I missed and an important one, even though I made it in the past. They tested their own product here. They literally invented this, then were handed millions to "test" whether their own product was any good.

This is exactly like Theranos having been handed millions in public money to "test" their own technology, in private, reporting with hidden data that it works, then somehow having the shamelessness to argue that any criticism of their own product, and testing, is invalid. After reanalysis of the data, obtained through litigation, showed the effect was not only massively inflated, but used absurd changes that made starting and end points overlap.

Then attacking regulators finding so, trying to corrupt the process behind the scenes, now documented. In that scenario, with a private company involved, someone would have gone to prison. Careers would have been ended. I don't think there is as much evidence behind the Stanford president's own scandal. It's just so much corruption.

I never understood how that was valid. It's so nakedly corrupt and incompatible with academic rigor to test the thing you invented yourself. In preliminary testing, sure, obviously. Which they did. Dozens of times over. Always small trials with no validity. But "the definitive trial" of it? Somehow presented as independent testing by the editor-in-chief of a reputable journal who made a promotional tour for it? WTH?

 

I was having a look at the JNNP site and came across a reference to a patient editorial board.
Would there be any benefit in sending the responses many patients have written to the patient editorial board? If they want patient engagement, let's give it.

https://jnnp.bmj.com/content/90/4/369


Editorial commentary

Patient Editorial Board for JNNP
FREE

1. Sara Riggare,
2. Rick Thurbon,
3. Sharon Roman,
4. Tessa Richards,
5. Matthew C Kiernan.
   1. Correspondence to Professor Matthew C Kiernan, Journal of Neurology, Neurosurgery and Psychiatry Editorial Office, BMJ Publishing Group, London, UK; jnnp@bmj.com

"JNNP strives to improve patient engagement in the studies that we publish
At a recent BMJ Editor’s retreat, as formal proceedings ended and the dinner began, a late arrival and the need to squeeze in one more seat at the table resulted in the new direction you see, with the launch of a Patient Editorial Board. Following wide ranging discussions about patient involvement across all aspects of clinical research, a commitment was made to address ways to improve patient engagement in the studies that we publish. Establishment of an Editorial Board comprised solely of patients to help guide the journal, is part of a growing trend for greater consumer involvement across all walks of life in an attempt to refine and drive more meaningful exchanges and outcomes. Of relevance to neurology practice, a number of patient support groups have reacted to a perceived lack of progress in relation to discovery of new treatments and investigation of rare neurological diseases. To date, rare diseases seem to have been left in the wake of the more prominent conditions. This has become increasingly apparent in the realm of neurodegenerative disease where patient groups have broadly questioned the direction of research. Specifically, questions have arisen about the suitability of clinical trial models particularly for conditions where there is significant heterogeneity, even if the diagnosis is the same. In contrast, specialties such as oncology are accelerating towards targeted and individualised precision approaches, while for neurodegenerative disease such approaches are languishing. Patients complain that their voices are disappearing into large multicentre trials, many of which still result in negative outcomes. Patients have argued that the clinical trial model, built around incremental refinements to treat large patient populations, seems poorly designed to tackle many of the diseases we encounter as neurologists. Separately, discussions are also beginning to focus on concepts around perception of clinical significance—what a scientific manuscript suggests as statistically significant, individual patients may challenge as being of no use to them in day-to-day life.

Increasingly, patient support groups are encouraging clinicians to focus on the issues that confront them as patients, and further, to turn their attention to more ambitious projects, rather than incremental change. Criticism has been levelled that too much time is being spent on securing grant renewals and flawed metrics like impact factor and individual H index, rather than real impact. To guide this process in a more positive direction, there is a move towards involving patients throughout research projects in addition to the clinical trial design process, with ongoing communication and collaboration setting the benchmarks. In doing so, patients are advocating for scientists, medical publishers, charities and public funding institutions to join together to promote more meaningful dialogue and change......"

edit: bolding in the text, not heading, mine.

 

Good spot

I shall try and send mine over tomorrow.

If anyone gets to this before me please do pass on my rapid response and details

 

I have just gotten around to reading this and found it very good.

Just to nit-pick a bit:

The PACE Trial was approved for funding in May 2003. It was well underway by the time the NICE guidelines were published in August 2007. So I wouldn’t be inclined to make this point.

 

I just checked: there is also an extension for Firefox. They are also working on an update for Safari.

 

Researcher allegiance in psychotherapy outcome research: an overview of reviews
Thomas Munder et al. Clin Psychol Rev. 2013 Jun
https://pubmed.ncbi.nlm.nih.gov/23500154/

The term researcher allegiance can be useful when making points like those in Joan Crawford’s eletter.

 

To be honest, I think a lot of the eletters contained a lot of rhetoric and discussion of the authors that you don’t tend to see in published papers. While there can be some relaxation for eletters, if you are not a doctor, you are probably at a disadvantage and editors might be less willing to give leeway.

I hope most of the authors of the eletters resubmit their letters taking out parts they think could have been problematic and/or toning down some of the language. It would be disappointing if there was only one rapid response posted in reply to this very problematic editorial.

I do think these editors are biased but it may be best to try to give them no reason to turn down a response.

 

Carolyn Wilshire discusses this in this paper:
The problem of bias in behavioural intervention studies: Lessons from the PACE trial
Carolyn Wilshire. J Health Psychol. 2017 Aug.
Abstract
Geraghty's recent editorial on the PACE trial for chronic fatigue syndrome has stimulated a lively discussion. Here, I consider whether the published claims are justified by the data. I also discuss wider issues concerning trial procedures, researcher allegiance and participant reporting bias. Cognitive behavioural therapy and graded exercise therapy had modest, time-limited effects on self-report measures, but little effect on more objective measures such as fitness and employment status. Given that the trial was non-blinded, and the favoured treatments were promoted to participants as 'highly effective', these effects may reflect participant response bias. In non-blinded trials, the issue of reporting biases deserves greater attention in future.
https://journals.sagepub.com/doi/10...=ori:rid:crossref.org&rfr_dat=cr_pub 0pubmed

Though it is focusing on authors so it might jeopardise an eletter if included in this situation.

 

Another paper discussing researcher allegiance is:
Investigator bias and the PACE trial
Steven Lubet
Volume 22, Issue 9
https://doi.org/10.1177/1359105317697324

• 
  Abstract
  The PACE investigators reject Geraghty’s suggestion that the cognitive behavior therapy/graded exercise therapy trial could have been better left to researchers with no stake in the theories under study. The potential sources and standards for determining researcher bias are considered, concluding that the PACE investigators “impartiality might reasonably be questioned.”
  https://journals.sagepub.com/doi/full/10.1177/1359105317697324

 

Minor update from MEAction UK.
https://www.meaction.net/2023/08/21/meaction-uk-interim-update-on-jnnp-complaint/

We've had our complaint acknowledged but not responded to yet, have submitted a Subject Access Request and are looking at further steps.
I.e. it's ongoing and we're continuing to fight the removal post publication.

 

I think it's more a request for what information someone holds about you, where they got it from, how they use it, and whether they've shared it with anyone else.

They're often requested by individuals. For instance, my GP practice suggested it was the best way to get the medical records I needed for my PIP application, as they had a well established protocol for processing SARs.

 

Yes, Subject Access Requests are like FOI requests, but specifically requesting information about <yourself> (it comes under GDPR).

• You can't make a request about another person.
• FOI only really applies to public bodies so we can't apply that here as the BMJ and JNNP are a private organisation.
• On the off chance that any of us were named in communications from editors/lawyers/others then this should bring it to light.
• They have 30 days to respond unless exceptional circumstances, otherwise the ICO get involved.

If we aren't named personally then it won't come to light unfortunately. But this is just one tool in our box though, we have other options open if the response to our complaint isn't satisfactory

 

So, is there anything we can do to pressure JNNP over their overt censorship of responses? They are clearly in violation of BMJ policies, their BS disclaimer is worthless here as none of the responses made public have any such thing, they are matter-of-fact and respectful.

This should be appealed to BMJ directly, given that two of the signatories are on the editorial board, they're unlikely to do anything on their own. They have clearly shown their colors, choosing to censor basic facts.

It's very annoying that NICE hasn't actually said anything. It suggests things happening in secret behind closed doors that are bad for us.

I strongly suspect that the only response was published because it was co-authored with Jonathan Edwards. Even though there is no requirement to have an academic or professional chaperone.

 

I’ve just emailed NICE to ask where and when its “detailed response” to the JNNP article will be published.

Will post if I receive a reply.

 

Yep I think we've seen a complete split/move happen now where at least some of these journals, and it being much more predominant in certain specialties/subjects, are basically being used as/operating as 'media' rather than 'academic' and the various expectations relating to what is involved with publishing and the purpose of an academic literature in academia.

And ergo the positions of control at such are more like what you might see editorially of certain types of press publications etc rather than them being about maintaining rigour for their subject and ergo prestige and respect for the publication. Or maybe we should think of it like a newsletter or 'BPS weekly' level of thing for a business sector ie more about showcasing 'research' from certain angles etc.

They can be a soapbox pretty much, particularly where a paywall means it is abstracts that don't have the std format so can say things quite different to the data findings in them, or what from a level of original data or methods means what would be more accurately termed a manifesto can be labelled as 'research' etc ?

This surely means that for those specialties where there is no longer this proper peer review process then another form of independent RAE quality assessment needs to be reintroduced given remits have shifted around in this way?

 

On Twitter,
"MEAction UK are delighted to announce that the @JNNP_BMJ are publishing an edited version of our rapid response. Follow the link for more info: https://www.meaction.net/2023/09/19/jnnp-to-publish-edited-rapid-response-by-meaction-uk/"

 

https://www.meaction.net/2023/09/19/jnnp-to-publish-edited-rapid-response-by-meaction-uk/

The JNNP have now agreed to reinstate, an edited version of MEAction UK's rapid response. We agreed to their suggested changes, and removal of the notice as we think it's important to remain on record alongside the original paper. With the other published Rapid Response (Dom Salisbury, Robert H Saunders and Jonathan CW Edwards) we feel that this at least gives some long term rebuttal to the "paper".

1. No reason was given for initially approving our response before removing it, just pointing to their terms and conditions.
   
2. Our Subject Access Request also yielded no further useful information.
3. We don't have a date for the updated response to be published, but at a guess it would be in the next few days.

Why other responses weren't published or even acknowledged I have no idea, but I suspect the JNNP if challenged, would just say they don't have to and it's the editors decision.

 

@Andy beat me to it