Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

Discussion in '2021 Cochrane Exercise Therapy Review' started by Lucibee, Feb 13, 2020.

  1. Hutan

    Hutan Moderator Staff Member

    Messages:
    27,828
    Location:
    Aotearoa New Zealand
    So, is this potentially the process?

    Writing team makes a protocol
    Protocol is peer-reviewed (peer-reviewers potentially including a team from S4ME)
    Protocol is released to wider public for comment
    Protocol is finalised

    Review is prepared (S4ME could run a somewhat parallel process to consider each study in the review, so we are ready to compare findings against the review)
    Peer-review?
    Draft review is released for public comment
    Review finalised



    BPS proponents appear to have been frantically producing papers lately. Having new evidence turning up could potentially set back a review process, and allow some bypassing of quality control measures. Should there not be some date, for example when the protocol is finalised, when no further evidence can be considered?
     
    Kitty, Michelle, alktipping and 8 others like this.
  2. Sly Saint

    Sly Saint Senior Member (Voting Rights)

    Messages:
    9,626
    Location:
    UK
    Kitty, Michelle, alktipping and 11 others like this.
  3. Trish

    Trish Moderator Staff Member

    Messages:
    53,402
    Location:
    UK
    I don't agree that experience of severe ME should be a key criterion for being on the review author group. To me, with only 2 patient/carers on the writing team, it's much more important to be competent to conduct a systematic review of clinical trials and to both understand and be prepared to speak up in the group about the need for things like therapist effect and placebo effect on unblinded trials to be understood, and the rest of the serious flaws in the trials under study.

    But I do agree the process should have some oversight and input by people with experience of severe ME to help the reviewers understand, for example, the significance of using the Oxford definition, and that ME/CFS is not the same as idiopathic chronic fatigue. Also the issue over the studies not properly tracking patients who drop out or report harms. Reviewers who don't understand that PEM is not just about being a bit tired the next day may think PEM as a side effect is mild enough to be ignored. Having someone on the IAG who can tell them first hand about the long term harm caused by GET from personal experience could help them take that problem seriously.

    I don't know what severity the current appointees to the IAG have experienced. I'm not asking for personal medical details of individuals, more a policy question for the IAG appointment process.

    But still, if there's only one place left on the IAG, my preference is that it be a patient with expertise in reviewing GET studies, whether formally as part of an author team, or more informally in, for example, forum discussions.

    Educating the review group about severe and very severe ME and the impact GET has had on some patients ending up with much more severe ME after doing GET could be fed into the process by inviting someone like Emma Shorter to speak to the review panel, or getting them to watch some of the good videos that feature severe ME in the GET context.

    Edited to clarify a half finished sentence.
     
    Last edited: Jun 6, 2021
    Kitty, Michelle, alktipping and 10 others like this.
  4. cassava7

    cassava7 Senior Member (Voting Rights)

    Messages:
    1,007
    It is important to note that Dr Newton recommends activity-based approaches such as graded exercise and advice on deconditioning as a first-line intervention for ME/CFS.

    She recommends the Lightning Process, CBT and mindfulness (27:46 timestamp in the video below).

    She has also made negative statements about patients that echo the psychiatric view of medically unexplained symptoms and malingering, but that are not supported by the evidence (30:24):
    Many patients will become:

    - Agorophobic/social phobic (negative peer feedback (especially teenagers)
    - Trapped in negative feedback loop
    - Secondary depression/anxiety
    - Bed-bound = disaster!
    - Often due to complicity of family; secondary gain​

    @Hilda Bastian I understand that the pandemic prevented performing an in-depth review of participants for a lack of time, but unsubstantiated patient-blaming views should have precluded anyone from being considered for the author team (let alone a strong positive bias towards the intervention being reviewed).

    https://www.youtube.com/watch?v=_i5OrKQsdG4


     
    Last edited: Jun 6, 2021
    bobbler, lycaena, Kirsten and 22 others like this.
  5. Ariel

    Ariel Senior Member (Voting Rights)

    Messages:
    1,059
    Location:
    UK
    Completely agree with this post, as the person needs to really understand the issues as opposed to just being an arbitrary patient, however careful one is in the selection.

    One would hope that everyone involved understands the problems about the case definitions and that ME/CFS is not idiopathic chronic fatigue. It's also very important to understand what PEM is and how severe it can be - but I have noticed a lack of understanding of the basics of what it is. (Delayed effect, etc). PEM is characteristic of the condition and everyone involved must understand these basics for the chance of an outcome that is not based on error and confusion.

    It is vital to understand how the methodological issues interplay with the above misconceptions. Expertise and experience is crucial, because this isn't easy or obvious even to people familiar with the discussions. The interplay of methodological issues and inaccurate, inadequate conceptions of ME/CFS should really be at the heart of any serious consideration of this subject.

    I am writing this post because I did not mean to suggest otherwise via the suggestion that a severe patient might be recruited - I was just very worried that the perspectives of severe patients would not be adequately heard - but of course this would not help address the above issues.

    <Personal experience>
    Experience with looking at literature and training is really key. Analytical skills are also important and speaking personally, they are often something one has to train for to do well. When I got very ill over a decade ago I was a graduate student and a lot of what we did was to analyze technical papers in seminars. Over and over and over again! What is this author saying? What is right with the paper? What is wrong with the paper? (Always something!) What are the key assumptions of the paper and what are the relationships with the conclusions, if any? It changed the way I approach any published literature; skills need to be honed over time and can't just materialize immediately. Practice is so important. That is why it's a shame if people who are more actively involved in analyzing this stuff are excluded. They have a key skill-set and important contextual knowledge.

    I really respect everyone here so much for continuing to do this, especially given that many of us are very ill. I have learned a lot from reading the posts here. I wish others could see and read them, too!
     
    bobbler, Kitty, Michelle and 9 others like this.
  6. Hilda Bastian

    Hilda Bastian Guest Guest

    Messages:
    181
    Location:
    Australia
    Excellent - thanks! My tunnel vision had me assuming it was a PSP related to NICE from a while ago & it didn't click. So now I'm guessing you mean the James Lind Alliance? https://www.psp-me.co.uk/ Yes, they do that great, and have great resources for it. A great example for us to include when we get to that stage - thank you! Have added it to the list - if you mean another one, please let me know (no rush).
     
    bobbler, Kitty, Michelle and 2 others like this.
  7. Hutan

    Hutan Moderator Staff Member

    Messages:
    27,828
    Location:
    Aotearoa New Zealand
    Yes.

    But surely the patient/carer representatives on the writing team should have this knowledge? If those representatives don't know the significance of ME/CFS definitions, or know that dropouts are a common problem in GET trials to be watched for, or can't explain PEM to the other reviewers, or have a good idea how bad severe ME/CFS can be, or know the problems with outcome measurement tools, or can't pull up videos or arrange a zoom call with an expert to help the review team understand if it is needed, then why they are there?

    I don't understand what the IAG actually does or how it can work. I mean, sure, if there is one, definitely have people with ME/CFS on it, preferably at least one patient with a very good understanding of GET and the impact of exercise, and hopefully Cochrane is paying them something, so that's nice. But, in this unwieldy process, how do they make any sort of difference? There seems to be a lot of overlaps between the roles of the patient/carer on the writing team, the IAG, and the peer reviewers. I think the subject matter expertise must be embedded in the review team, so that it can be applied as it is needed.


    Hilda, given it is difficult (although necessary at times) to talk about specific individuals in the ME/CFS review roles, perhaps we revisit the homeopathy example but think about it instead as a review of the Lightning Process?

    So, imagine Cochrane is running a 'Lightning Process for multiple sclerosis' review in some country, and there has been quite a bit of controversy about whether it helps this particular illness, with most patients saying it's useless but some being very keen, and most LP practitioners adamant it's terrific and that the government could save lots of money by replacing drugs with the LP. The government too is excited at the potential savings and very interested.

    So, would Cochrane typically have an 8 person review team that included:
    • a civil servant with very mild MS who thinks that the LP has kept them quite well but has never said so in social media, doesn't have any background in trial analysis, MS or otherwise, and has not been active in a MS advocacy community
    • a person who has been providing LP courses for years, and who specialises in providing the treatment to people with MS.
    Isn't the review a matter of finding the trials, looking at them carefully to identify issues that might affect the validity of the findings, aggregating the findings, considering harms, and coming to some conclusion about whether the intervention works and what research is needed? Assuming you have appointed a competent MS subject matter expert, and experienced reviewers with integrity, what would the two people on the review team that I mentioned contribute - what are some examples of how their presence would make the process better? Is there not a risk that their presence could make the process worse? Wouldn't it be better to have the LP provider as a peer reviewer, rather than embedded in the process?
     
    bobbler, lycaena, Michelle and 8 others like this.
  8. Hilda Bastian

    Hilda Bastian Guest Guest

    Messages:
    181
    Location:
    Australia
    s
    (a) Sorry - I'm not going to pre-empt the June report here. (But when you think about this, don't forget the IAG is part of the process.)

    (b) And please don't consider yourself limited by me in any way in coming up with proposals for peer review - just don't ask me to commit to anything, because there will be IAG recommendations to the editors (not personal ones from me).

    (c) The protocol and bypassing quality control measures: for now, the broad answer is, won't bypass me/IAG. There'll be more information on detail in the next report. (Sorry about being a broken record on this.)
     
    Kitty and Michelle like this.
  9. InfiniteRubix

    InfiniteRubix Senior Member (Voting Rights)

    Messages:
    818
    Location:
    Earth, in a fractal universe
    It is extremely important to remember that assuming that Cochrane, NICE, CDC or local medical processes operate in a vacuum is incorrect. Their scope and actions are bound by the law. Full stop. It is the fundamental driver and arbitrator, unless we wish to avoid highlighting it to appease insistent law breakers and purveyors of malpractice.

    The various reviews are mere details WITHIN their lawful limits. Not addressing that fact opens up waste of energy and effort, allowing discussion of possibilities that are not legally sustainable anyway. Plus, most legal advice will fail to be informed correctly and come terminally incorrect conclusions.

    How this reality is referred to or plays out in different contexts has to be different, sure. But the fact remains. Neither Cochrane, NICE nor the CDC define the law. And us giving any implicit space to such thinking is us unnecessarily gifting away rights and obligations for the short to medium term. Again.

    https://www.s4me.info/threads/compl...nce-underwriting-etc-vs-me.20482/#post-349054

     
    Kitty, Michelle and alktipping like this.
  10. Trish

    Trish Moderator Staff Member

    Messages:
    53,402
    Location:
    UK
    Of course the ones on the review writing team should have a wide understanding of all levels of severity and be able to convey them to the writing group where relevant, for example in taking harms seriously, but if the IAG is to serve any purpose, presumably part of that is to be able to feed into the reviewing discussions information that the review panel haven't understood properly. A sort of back up to the patients on the writing team.
     
    Kitty, Michelle, alktipping and 3 others like this.
  11. Trish

    Trish Moderator Staff Member

    Messages:
    53,402
    Location:
    UK
    Can you put in plain English what you are referring to here. I'm a bit lost.
     
    Kitty, Michelle, alktipping and 2 others like this.
  12. Hilda Bastian

    Hilda Bastian Guest Guest

    Messages:
    181
    Location:
    Australia
    All I can say is the same thing again: the group includes people with different perspectives. I understand some are opposed to that concept completely. (I did not say the pandemic prevented reviewing people in-depth.)
     
    Kitty likes this.
  13. Hilda Bastian

    Hilda Bastian Guest Guest

    Messages:
    181
    Location:
    Australia
    All the IAG members do not discuss their health in their bios; some do. That's here.
     
    Kitty likes this.
  14. Caroline Struthers

    Caroline Struthers Senior Member (Voting Rights)

    Messages:
    882
    Location:
    Oxford UK
    I know the Vink and Vink-Niese review is not a systematic review and doesn't claim to be. Did you consider inviting them to lead (or be part of) the Cochrane review author team to develop the new Cochrane protocol, search strategy etc. within the Cochrane process? This is what I meant by Cochranizing their work. It would be good to know if you considered it, and if you did, the reasons why it wasn't taken forward as an option.
     
    Kitty, Sly Saint, Michelle and 8 others like this.
  15. InfiniteRubix

    InfiniteRubix Senior Member (Voting Rights)

    Messages:
    818
    Location:
    Earth, in a fractal universe
    Any organisation's work, output or guidelines that facilitates/creates unlawful consequences is either 1) to be ignored as a requirement of the law or 2) is itself open to legal challenge.

    None of these organisations can transform unlawful decisions/acts into lawful behaviour.

    Put more crudely, garbage in, unlawful behaviour out.
     
    Kitty, Skycloud, Michelle and 5 others like this.
  16. Hilda Bastian

    Hilda Bastian Guest Guest

    Messages:
    181
    Location:
    Australia
    2 systematic reviewers from Cochrane Response were allocated to that work: I never considered that they weren't well-enough skilled and positioned to do it, or that it wasn't a good way to do this. I'm not going to discuss reasons for any individual not being part of the review team.
     
  17. InfiniteRubix

    InfiniteRubix Senior Member (Voting Rights)

    Messages:
    818
    Location:
    Earth, in a fractal universe
    The Twitter thread summarises the primary points
     
    alktipping and Trish like this.
  18. Trish

    Trish Moderator Staff Member

    Messages:
    53,402
    Location:
    UK
    Kitty, alktipping and InfiniteRubix like this.
  19. InfiniteRubix

    InfiniteRubix Senior Member (Voting Rights)

    Messages:
    818
    Location:
    Earth, in a fractal universe
    Ignoring the illiterate is effectively a requirement of law in this context.
     
    alktipping and oldtimer like this.
  20. Jonathan Edwards

    Jonathan Edwards Senior Member (Voting Rights)

    Messages:
    13,971
    Location:
    London, UK
    This causes me serious concern @Hilda Bastian.

    What we need from the exercise review is the best possible judgment on whether or not a set of trials reflect some real beneficial effect specific to certain treatments. Whether or not they do have an effect is a matter of fact, even if very difficult to ascertain. I do not like the word objective, which tends to get muddled in post-modernist type language but I think in this context we can see that there is an objective fact (hard to ascertain) as to whether exercise is beneficial in ME - and various subsidiary related facts of detail. There is therefore an objective fact of the matter as to whether the trials reflect a real effect.

    My understanding is that Cochrane is morally and probably legally obliged to report on the evidence entirely on the a basis of the best possible rational case for whether the trials can be taken to reliably reflect a true effect. The reliability is also an objective fact just as the reliability of the weather forecast on wind speed and precipitation is an objective fact (but easier to ascertain). The lack of reliability of open trials with subjective endpoints in identifying a real effect as also based in good evidence.

    To me it would be shocking and I think a breach of human rights to allow that points of view, or sociological factors other than simply the best possible attempt to ascertain objective truth here should have any influence at all. To do so would be to admit that it was legitimate for Cochrane to favour a treatment even if the was no good evidence that it worked or to discourage a treatment even if there was evidence that it did work. There is absolutely no case for having people with 'different points of view' authoring a review.

    I hope I have misunderstood what you were suggesting but it does sound tantamount to an admission that Cochrane does not follow the policy it advertises and which it is very likely bound in law to stick to.

    Would anyone with an official position in Cochrane endorse the view I think is being suggested?
     
    bobbler, Louie41, Chezboo and 25 others like this.

Share This Page