Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

Discussion in '2021 Cochrane Exercise Therapy Review' started by Lucibee, Feb 13, 2020.

  1. MEMarge

    MEMarge Senior Member (Voting Rights)

    Messages:
    2,926
    Location:
    UK
  2. Caroline Struthers

    Caroline Struthers Senior Member (Voting Rights)

    Messages:
    966
    Location:
    Oxford UK
  3. Adrian

    Adrian Administrator Staff Member

    Messages:
    6,563
    Location:
    UK
    Are review boards needed. Could some tools be applied to perfom a standard checking. I would say that this would need a formal specification of the trial and then use some form of model checker. But what could chatGPT do to find flaws (I've seen it being used to find security issues in code) could it be used to find methodological flaws in a trial protocol?
     
    FMMM1, Medfeb, Peter Trewhitt and 2 others like this.
  4. Caroline Struthers

    Caroline Struthers Senior Member (Voting Rights)

    Messages:
    966
    Location:
    Oxford UK
    My proposal is to systematically produce template trial protocols for the use of funders. The design and methods review would be done in advance of the research being funded. This would mean research groups would bid to conduct pre-designed and approved trial protocols, produced to fill evidence gaps and explicitly avoid mistakes of previous trials, and including meaningful input from patients. So it's not about finding flaws in unsolicited trial protocols, but more about ensuring trial protocols are sensible in the first place.
     
  5. Adrian

    Adrian Administrator Staff Member

    Messages:
    6,563
    Location:
    UK
    I was thinking in terms of tools that if you have a well specified protocol then you could use the tool to find potential issues (and show how they are being addressed). Things like the quality of outcome measures, blinding also randomization protocols etc. I also think the same should be applied to stats analysis (where I think the code should be prewritten) which could for example check for preconditions for summary stats and tests.

    Sets of heuristics could be build up, from knowledge of things that have gone wrong in the past. It could help improve protocols and if you force someone to go through a process and justify where there are issues then than can help prevent knowingly bad trial protocols.

    Its the kind of thing that is done in other areas.

    I think having mechanisms and rule sets can aid a process as you are less reliant on someone doing careful reviews.

    But I was also serious about looking at could ChatGPT do some reviews and find problems.
     
  6. rvallee

    rvallee Senior Member (Voting Rights)

    Messages:
    13,661
    Location:
    Canada
    Just a reminder that ChatGPT is basically a child's toy compared to what's coming this year, which will also be a child's toy compared to the next generation, probably the next year, if not before. It's not about what ChatGPT can do, it's about how much more the next iteration can do compared to ChatGPT than what ChatGPT can do better than what came before. It's a race and it has so many competitors, all racing to be the first. It's going to be a multi-trillion industry and the efforts are happening accordingly.

    Usually technological progress requires a downtime period, having to deal with manufacturing process yields, supply chains and logistics, then distribution and sales. Products have to go through a lengthy process of being actually built and delivered to people. And manufactured products need to sell before a better version gets commercialized, improvements staggered to maximize profits caused by the need to produce actual real-life items and getting them in the hands of paying customers.

    Services like AI, where the delivery infrastructure is already fully built, can be deployed for use in days once they're ready, they can go through generational leaps in a span of months, the pace is seriously mind-blowing when you've been following it for a while. And the second a better version comes around, it can displace the big players, so the need to push is there and constant.
     
    Ariel, Pustekuchen, Amw66 and 6 others like this.
  7. Sly Saint

    Sly Saint Senior Member (Voting Rights)

    Messages:
    9,924
    Location:
    UK
    there should be a compulsory section on 'limitations of this research' which in published papers often only appears tucked away somewhere in the text or simply not mentioned at all.
    Having it as a a key feature in the protocol might help to focus attention on how sound the methodology of the research proposal is.
     
  8. FMMM1

    FMMM1 Senior Member (Voting Rights)

    Messages:
    2,812
    If a public body is looking for someone to run/carry out a specified study then publish the protocol. E.g. PACE seemed to be intended to get objective evidence [Actimetry - FitBIT], if that was set out in the spec, and you didn't deliver, then no money.

    If you're applying to a public body for funding [researcher is suggesting that this research should be funded] and the public body approves it, in principle, then it e.g. published the protocol and ask for comments.

    But yes, most of the existing problems are systemically poor trial designs - lack of objective outcome criteria. Those could largely be dealt with by guidance.
     
  9. Andy

    Andy Committee Member

    Messages:
    23,034
    Location:
    Hampshire, UK
  10. FMMM1

    FMMM1 Senior Member (Voting Rights)

    Messages:
    2,812
    Caroline just thinking that the APPG might be a useful way in i.e. to deliver/help to deliver this

    E.g. could ask the APPG to ask NIHR (research umbrella body?) to adopt this policy --
     
    alktipping, Ariel and Peter Trewhitt like this.
  11. Caroline Struthers

    Caroline Struthers Senior Member (Voting Rights)

    Messages:
    966
    Location:
    Oxford UK
    I will be putting in an application for funding to explore the feasibility of this policy to the NIHR "Research for patient benefit" funding stream. They have launched a "highlight" series to help under-represented researchers, starting with nurses and midwives. But they will also be highlighting methodologists, and allied health professionals (https://www.nihr.ac.uk/documents/rf...-nurses-and-midwives-call-specification/32050). Technically, I am a methodologist as my main job with the UK EQUATOR Centre is focused on improving transparency and reporting of research. Not sure when the deadline will be, but hopefully sometime in the summer.
     
    mango, Solstice, Mithriel and 20 others like this.
  12. Caroline Struthers

    Caroline Struthers Senior Member (Voting Rights)

    Messages:
    966
    Location:
    Oxford UK
    This is the latest correspondence with Cochrane after I wrote to say I was preparing a complaint on 3 January. Getting hold of the email address of the new CEO sped things up considerably.



    So I now have to get on and submit the complaint itself. To remind everyone, it will be specifically about Karla allowing the authors of the exercise review to ignore Guyatt's instruction that they explicitly state that exercise has no meaningful positive effect on fatigue.
     
    Last edited: Jan 20, 2023
  13. Trish

    Trish Moderator Staff Member

    Messages:
    55,414
    Location:
    UK
    Thanks for persevering with this, Caroline.
     
  14. rvallee

    rvallee Senior Member (Voting Rights)

    Messages:
    13,661
    Location:
    Canada
    Well, here we are, 3 years to the day. Not even a shadow of an attempt of an effort, not even a whisper. I expected very little from this, given how the odds were stacked, and it still managed to massively underdeliver. Impressive, in a way. The whole process began almost 7 years ago, Cochrane has been aware all this time that their shoddy review is invalid and causing harm. Clearly they couldn't care less what happens to patients, how their product is used, how many suffer and die needlessly because of it.

    Cochrane is gonna Cochrane. I can't believe for a second that there wasn't influence from the likes of Paul Garner, politics rule here. Cochrane has always been a private club for self-promotion. As the flag-bearer of the failed evidence-based medicine paradigm, the end of this rotten organization will mark a major shift towards progress in medicine. It didn't take hindsight to predict that an evidence paradigm built on opinion and biased agendas was ever going to work out, especially without a single bit of science to it. Some math, sure, on fake numbers. Not science. The only parts of medicine that work are founded on science. Nothing in evidence-based medicine meets scientific criteria, and it's terrible. A child can see through the flaws in this logic. But as with most things, until it happens to them, people don't understand the life experience of others. Even when it's their freaking job, it's not even close.

    We still regularly see deaths in the community. Many in their early adulthood. Suicides, early deaths, despair. That we see, there's a lot more we never hear about. All enabled and promoted by Cochrane. Among others, sure, but you made this dystopian nightmare, helped impose it on us, coercively, making a mockery of informed consent. All predictable stuff, an entire discipline built on exemptions to the rule and zero accountability is basically choosing violence.

    The disappointment is infinite, but I really did not expect much anyway. The fault still lies with Cochrane, their treachery and corruption is at the core of what they do. This is what everything about us, in secret, behind closed doors amounts to. No one ever thinks of the other half: first do no harm, or else what. Well it's not the other half, it's not even considered. It should, but it says everything that nothing ever happens, no matter how much monstrous harm is inflicted on millions. Medicine fails the waiter test too often, when they owe us nothing, they do nothing.

    [​IMG]
     
    Chezboo, EzzieD, Mithriel and 20 others like this.
  15. Trish

    Trish Moderator Staff Member

    Messages:
    55,414
    Location:
    UK
    This must surely be crushingly disappointing too to the patients and carers and decent clinicians who volunteered to participate in this fiasco.
     
    Chezboo, EzzieD, Mithriel and 21 others like this.
  16. Hutan

    Hutan Moderator Staff Member

    Messages:
    29,374
    Location:
    Aotearoa New Zealand
    Yes. Wasn't this supposed to be a shining example of patient engagement? Surely even Cochrane itself has to recognise it has utterly failed with that?

    In fact, it has utterly failed generally. The harm (caused by the Cochrane stamp of authority for exercise treatments for ME/CFS) continues, precisely at the time when there is the most global need for it to be gone.

    I assume that some of the 'patient and carers and decent clinicians' involved have applied enormous efforts to get the old review removed or replaced, but we have seen precious little evidence of that effort. How long is it reasonable for them to continue to provide legitimacy to the process in the hope of 'changing things from the inside'? Surely there is a limit to how long it is ethical to patiently and quietly wait, given the harm that delay causes including, as @rvallee says, deaths caused by despair?

    I would like to hear from the patients, carers and decent clinicians, some of whom are members here, and @Hilda Bastian herself, as to what their plan is. In the meantime, this thread stands as a sad record of the failure.
     
    Chezboo, EzzieD, Mithriel and 21 others like this.
  17. Jonathan Edwards

    Jonathan Edwards Senior Member (Voting Rights)

    Messages:
    15,175
    Location:
    London, UK
    It reminds me of that extra bit of material attached to a jeans pocket below the stitching that carries on just as two flaps of material going nowhere that slides down next to your leg when you put the jeans on. That seems to be Cochrane now - a continuation that no longer does anything and is invisible.
     
  18. Hutan

    Hutan Moderator Staff Member

    Messages:
    29,374
    Location:
    Aotearoa New Zealand
    :) Well, that prompted a google search that contributes to my search history surely defying any categorisation. I still have no idea what extra bit of material Jonathan means. Regardless, I don't think it is a perfect analogy, because Cochrane does continue to be visible, and in fact highly influential, not least with respect to propping up the idea of exercise as a therapy for ME/CFS.
     
    NelliePledge, Solstice, Joh and 9 others like this.
  19. Sean

    Sean Moderator Staff Member

    Messages:
    8,064
    Location:
    Australia
    It is now indisputable that this process has been a complete and utter failure.

    I am grateful to those who tried to stop the rot and reform the institution, from without or within, and am sorry about your wasted efforts, but it is clearly beyond repair and should be shut down.

    Everything the BPS crowd touches dies.
     
    Last edited: Feb 14, 2023
    MEMarge, ukxmrv, bobbler and 12 others like this.
  20. Caroline Struthers

    Caroline Struthers Senior Member (Voting Rights)

    Messages:
    966
    Location:
    Oxford UK
    Chezboo, EzzieD, MEMarge and 19 others like this.

Share This Page