Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

Discussion in '2021 Cochrane Exercise Therapy Review' started by Lucibee, Feb 13, 2020.

  1. Adrian

    Adrian Administrator Staff Member

    Messages:
    6,511
    Location:
    UK
    So perhaps you can describe the recruitment process a bit more? Prior to interviewing what did the skills and marking matrix look like? Who else was on the interview panel, what additional checks were done (for example prior to interviewing people I always try to read some papers or other material (and even look at open source proects in github!)). What style of interview (standard questions in which case share, technical presentations etc). How follow up interviews happened to deal with any potential issues or check skills areas that didn't get adaquately tested during the panel?
     
  2. MSEsperanza

    MSEsperanza Senior Member (Voting Rights)

    Messages:
    2,881
    Location:
    betwixt and between

    Apologies for not checking / mentioning the dates:

    The comment is from 2000, Henschke joined the author team in 2010--

    now edited my post to point to the dates:

    didn't check the revisions so no idea how the author team from 2010 (when Henschke joined them) responded to it, also don't know Henschke's current stance on that.

    As the reply to the comment still is linked to the review and no reply added, it still appears to be valid for the current review, though.

    Again, apologies for causing extra confusion.
     
    Last edited: Jun 5, 2021
  3. Adrian

    Adrian Administrator Staff Member

    Messages:
    6,511
    Location:
    UK
    They have not just failed people with ME they have demonstrated a lack of competance in backing a review that was not up to any standard and demonstrated they don't care about accuracy by leaving that review on the books for years dispite knowing the problems. Its a systemic issue with Cochrane not just an issue with a ME/CFS review
     
    bobbler, alktipping, Chezboo and 14 others like this.
  4. MSEsperanza

    MSEsperanza Senior Member (Voting Rights)

    Messages:
    2,881
    Location:
    betwixt and between
    see my addition here.

    I agree the editor's reply to the comment is appalling.

    I think it would be good if forum members could collect some points about the review authors' work that we would like to discuss and of which it would be good to know the authors' current stance.

    Editedt to add: I realize that's what we're doing already.

    It would be much easier for me if we could do that in a structured manner, e.g. on a subforum or similar. Also understand if people prefer to collect and discuss in the current thread -- just won't be able to contribute much or maybe not at all in that case.

    (Edited to replace quote of deleted message.)
     
    Last edited: Jun 5, 2021
    Michelle and alktipping like this.
  5. Invisible Woman

    Invisible Woman Senior Member (Voting Rights)

    Messages:
    10,280
    I'm still catching up with the thread, so please forgive me if this has been raised.

    Whose position is entrenched, here?

    From where I'm standing it's not ours. I don't really care whether ME is eventually classed as a psychiatric, psychological, behavioural, autoimmune, or whatever disease.

    Most of us here have already faced some fairly extreme discrimination and lost an awful lot of the things that mattered to us and everyone else takes for granted in life.

    I don't see how asking, repeatedly, for as often and as long as is necessary, for high quality research with good sound methodology and results presented honestly and without spin.

    as a community we at least want effective treatment, while we wait and support researchers working for a cure. That treatment doesn't exist at the moment because of the over reliance on spun data from research with very poor methodology.

    This narrative of entrenched views and opposing camps is an extension of the unreasonable, militant activist narrative.

    As a patient group we have time and again raised valid objections and queries on the science of studies. Even when our input is acknowledged as valid authors of papers have simply been able to decline to make corrections or withdraw their paper.

    You make it sound as though there are two opposing and equally powerful forces with your team in the middle here. That is simply not the case.
     
    bobbler, Simone, Blueskytoo and 23 others like this.
  6. MSEsperanza

    MSEsperanza Senior Member (Voting Rights)

    Messages:
    2,881
    Location:
    betwixt and between
    Am too slow to catch up with the thread and wording my points takes too much time so apologies for my odd posting behaviour.

    When re-reading through the therad a bit there are so many points others and I have made already and I think still apply.

    On the other hand, ist seems that the links to reviews and comments I posted last year and re-posted today had been missed by most people back then.

    That's why I think a subforum would make sense.

    It would also be much easier for me if we had the space to post Hilda's spreadsheets on a therad to check which criticism the IAG is already aware of and to which we could refer if we wanted to ask the authors on their current stance on those issues.
     
  7. Jonathan Edwards

    Jonathan Edwards Senior Member (Voting Rights)

    Messages:
    13,971
    Location:
    London, UK
    This is a valid argument Hilda raised some time ago.

    But the other side of it is this: There is no point in doing a systematic review of evidence of efficacy of a treatment unless you think you may find some reliable evidence of efficacy. If you don't find any there is no reason to suggest the therapy is used. If all the studies available fall below a minimum requirement for reliability, as for exercise for ME, then looking at them for other interesting bits of information seems pointless.

    The again, it would be reasonable to say that you might want to do a systematic review, particularly for a treatment that has got itself into routine use without proper testing as non-pharmacological treatments tend to, in order to raise the possibility that you might be wanting to report that people should stop using the treatment because it causes harm. I guess that for vaccines and supplements and a variety of other things a systematic review might have detection of harm as its primary purpose. But in those cases it is usually an issue of raising harms in the context of reasonably likely or accepted benefits. For exercise for ME no such likely or accepted benefit is known. Moreover, the most usual route to documenting a problem with harm is to gather open data first and if it is worrying to look at it in a formal study, if that is justified by reliable evidence of benefit.
     
    bobbler, Kirsten, Michelle and 13 others like this.
  8. Jonathan Edwards

    Jonathan Edwards Senior Member (Voting Rights)

    Messages:
    13,971
    Location:
    London, UK
    Yes, I am rather hoping that Hilda realises she should never have raised that!
     
    bobbler, Blueskytoo, Barry and 14 others like this.
  9. Hutan

    Hutan Moderator Staff Member

    Messages:
    27,828
    Location:
    Aotearoa New Zealand
    Why not? Research methodologists who deal mostly with drug interventions rather than behavioural interventions would probably understand the issues with PACE -- surely Cochrane have some. Expert patients who understand the methodological issues would understand the issues with PACE very well. I don't see any difficulty in having people who can deal with the sorts of issues in PACE apart from having a willingness to have those people in the team.

    The Cochrane reviewers have an interest in not criticising methods that they've marked as acceptable on other reviews.

    Julia Newton has an interest in not criticising methods she has used, or assumptions that she has made, in her research. Have you read much of Dr Newton's work? I actually argued before that most people understand that exercise doesn't fix everything, but Newton's paper is pretty strongly pro-exercise. I suggest there is a bit there to suggest that she is ideologically opposed to criticism of the universal utility of exercise.
    As noted below about Kay, we don't have enough information to know if she has conflicts of interest. She lists 'Navy' as a conflict of interest, so presumably she sees some potential for conflict there.


    How can you tell that Kay is a 'fairly good pick'? What do you know about her skills in research methodology, or explaining to others the issues with GET trials, or standing up to other members of the panel about the problems with BPS methodology? How can you be sure that she does not have an association of some sort with BPS people? How can you be sure that Kay doesn't attribute her 'high-functioning status' to the fact that she does yoga and keeps active, and thinks that people with lower levels of function need some help in the form of GET to get motivated and active too? Of course I'm speculating and she probably doesn't, but I don't think we know.

    I've deleted my post about Henschke as it didn't seem quite fair given the issue with timing.
     
    Simone, Barry, alktipping and 7 others like this.
  10. Jonathan Edwards

    Jonathan Edwards Senior Member (Voting Rights)

    Messages:
    13,971
    Location:
    London, UK
    Technically speaking I think the philosophy behind combined use of objective and subjective measures for arthritis is that the subjective measures of pain, stiffness, swelling and tenderness (subjective either to the patient or the assessor) are primary but need to be corroborated by objective measures to ensure they have a high probability of being reliable indicators of something a treatment might be modifying.

    The situation is very complex. Even the combined outcome measures are not reliable in open studies. They are largely used in blinded studies in order to guard against rather different problems from that of suggestibility, placebo effect, collusion etc. They may be more relevant to things like regression to mean and systematic shifts with time unrelated to treatment.
     
    bobbler, Simone, Michelle and 10 others like this.
  11. Hilda Bastian

    Hilda Bastian Guest Guest

    Messages:
    181
    Location:
    Australia
    I was answering your first question: if it's on the list, that's the answer to the question of has it been analyzed, because everything on the list has been analyzed.

    I didn't reply to the other question, because to repeat again that there will be a new protocol, prepared by the author team as announced, is so clearly redundant: that it means it won't be the one from the CDC or NICE review or any other review that already exists (including the current Cochrane review). (Vink and Vink-Niese, while a very thorough critique of the Cochrane review and re-analysis of studies, is not itself a systematic review and doesn't claim to be - no search strategy, specified methods, etc.)
     
  12. Jonathan Edwards

    Jonathan Edwards Senior Member (Voting Rights)

    Messages:
    13,971
    Location:
    London, UK
    I agree with Hutan. In fact I would go much further. Anyone not fully conversant with the reasons why PACE is no good is not competent to do this review. You don't ask people who do not understand plumbing to service a central heating system. I could muster up ten people who are conversant straight away - members of my old academic rheumatology department, including trainees.

    This stuff is absolutely basic, which is why it is so shocking to hear founder members of Cochrane like Chalmers, Gotzsche, Garner or Glasziou coming out with the remarks they do.
     
    bobbler, Simone, Blueskytoo and 29 others like this.
  13. Caroline Struthers

    Caroline Struthers Senior Member (Voting Rights)

    Messages:
    882
    Location:
    Oxford UK
    Oh I didn't realise. It's only the NICE review that has not yet been analysed.
     
    Simone, tomnext, FMMM1 and 2 others like this.
  14. Hutan

    Hutan Moderator Staff Member

    Messages:
    27,828
    Location:
    Aotearoa New Zealand
    Yes, we are covering old ground on this. Of course there can be valid reasons for having subjective outcomes in a trial; having some is not a disqualifying feature of a good trial. And applying some thought to what people have measured is always a good move. I don't think anyone ever thought otherwise. We do have to use a short label when referring to the issue.

    But, outcomes of surveys of fatigue or activity levels or overall wellness in GET trials aren't reliable indicators of a treatment benefit.

    MSESpe - Yes, a subforum could be good. The moderation team are working on what set up could help.

    Yes. They are allowed a bit of time to look at it if they haven't seen it before :), but if they don't have the capacity to understand the problems with it, then they should not be on the writing team.
     
    bobbler, Simone, Blueskytoo and 13 others like this.
  15. Robert 1973

    Robert 1973 Senior Member (Voting Rights)

    Messages:
    1,352
    Location:
    UK
    I’m a bit puzzled why none of this was included in monthly progress reports from March 2020, as decisions were being made. That is what I had assumed would happen. That way, we could have scrutinised these decisions and processes before people were appointed.

    When we heard nothing for 14+ months I assumed that work had stalled. Instead, it appears that a great deal of work was going on but for some reason we weren’t being told about it. Please can you clarify why that was, Hilda?
     
    Blueskytoo, JaneL, Michelle and 14 others like this.
  16. ukxmrv

    ukxmrv Senior Member (Voting Rights)

    Messages:
    867
    Thank you Hilda Bastien for engaging so thoroughly in this process and talking to patients here.

    Last night I went to sleep dreaming I was in a Grisham courtroom drama. We have a trial "the evidence" and a jury needs to be selected. In true Grisham style we get the story of how important it is to pick the right jury and how trading and shenanigans (favourite forum word) go on behind the scenes to ensure that the right duty is selected for whatever outcome is desired by both sides.

    Cochrane have gone one better by engaging you, Hilda as the "front". A champion of a certain style of consumer rights and a formidable person with lots of guts and experience. Hilda, I would like to suggest that maybe it is this old fashioned style of consumer rights dispute / evidence analysis that could potentially be the problem here. I could be wrong. However, I think it needs to be mentioned as a possibility.

    We have seen a succession of Inquiries here in the UK into a range of different concerns where evidence needs to be heard and assessed. The pattern is for the first Inquiry to be controlled by the Establishment and to return a verdict that leaves people injured or affected unhappy. Then over time the Inquires become more and more focussed on the rights and the interests of the people affected as the original establishment is allowed to retire, die to disappear.

    With Cochrane we are at the first stage here.

    Cochrane have bought the type of court the case will be held in (the use the Grisham court room analogy), they selected someone with a firm view of how it should be done (which appears to be outmoded and wrong at least to UK sensibilities) and the jury is stacked with the wrong people.

    It's no longer acceptable in England for people affected to be labelled as "activists" and for their input to be disregarded for that reason. It's recognised that unless patient interests are not being heard and addressed the problem doesn't go away. It is recognised that people affected by an issue deserve to be heard. It's no longer acceptable to put a group of people together with patient representatives that don't reflect the views and an understanding of the issues that brought this Inquiry together. It is recognised that it is normal for anyone severely affected by an issue to be informed and to speak out.

    A brief look at Kay's internet presence should have raised some red flags that said wrong person and deeply "Establishment"

    https://civilservice.blog.gov.uk/2019/05/03/personal-disability-stories-10-kays-story/

    1. In this part she speaks about her experience of being mildly affected and exercise "Some types of exercise can be detrimental if not done the right way – I am, by necessity, more yoga than parkrun!"

    The pro GET lobby use the same language. It's "exercise" not done in the "right way". Wrong person.

    2. Here she says "I have always had exceptional support through Defence Medical Services.".

    So what is this exceptional support on offer that the rest of us can only dream about. A ME patient would want to know. Are they truly "exceptional" offering things that are different to the original NICE Guideline and is so, why don't we know about them and have access to them.

    The answer is of course that it's the same crap the rest of us are getting with more employer type support. If she is happy with her medical care then she will not understand the battle with NICE and with Cochrane that the rest of us are fighting. Wrong person.

    3. She makes non-evidence based attributes to her own health problems here "Initially, I reacted physically to the increased cortisol levels firing round my body, and I ended up in an early menopause, aged 41."

    We have had threads on the forum that look at the claims for cortisol and what evidence there is. Kay has obviously not seen these threads or read any medical papers on the topic. It's pure behavioural BS i.e. stress = physical problem.

    Who told her this and why does she believe them? Wrong person.

    4. I've also had a look through her Twitter feed and seen "likes" for particular views that are controversial areas for people with ME and pain. Personally I would not have "liked" them and I would have recognised where they come from. Wrong person.

    Hilda, you should have seen all of this and recognised it.

    It's looks like a disaster and deeply unprofessional, patronising and well out of date with attitudes and methods.
     
    Last edited by a moderator: Jun 5, 2021
    bobbler, Hutan, Simone and 19 others like this.
  17. Adrian

    Adrian Administrator Staff Member

    Messages:
    6,511
    Location:
    UK
    My belief is there wasn't a formal process just Hilda talking with a few people thinking they might help or I don't like them. But this is why being really transparent about approach and process is so important. And yes if the skills required and process for recruitment was published prior to recruitment then that would have allowed people to give feedback about what is needed.
     
    Last edited: Jun 5, 2021
    bobbler, Hutan, Simone and 10 others like this.
  18. Adrian

    Adrian Administrator Staff Member

    Messages:
    6,511
    Location:
    UK
    Given it is gas I suspect doing so would be illegal in the UK.
    Unlike handling devices with gas those doing reviews and giving medical advice don't need to demonstrate competence.
     
    bobbler, Kirsten, JaneL and 10 others like this.
  19. Mithriel

    Mithriel Senior Member (Voting Rights)

    Messages:
    2,816
    Nothing, no type of evidence, demonstration of error in any part of the process of research, patient experience, expert testimony, committee reports, judicial judgements, anything you can think of has changed the minds of the BPS people.

    They respond to everything by making statements which are untrue, like claiming patients are terrorists then using their eminence to have special branch keep tabs on ME "activists" (that must have been boring!), making out everyone criticising their work is anti science (A committee set up by Wessley gave him an award for standing up for science against the awful ME patients).

    They also use the law against patient organisations and parents who do not want BPS treatment for their children. Again, I could go on as I am trying to summarize forty years of our history.

    When the PACE trial was laid bare in it's methodological, misleading awfulness they ... well they just ignored it. They just say the results showed GET and CBT are evidence based treatments for ME.

    When they thought Cochrane were going to react to the problems with the last review they released a statement through Reuters stating that Cochrane had caved in to threats from patients. For the next week and every time it is revisited there are discussions of how dreadful it is that patients (small groups of activist ones of course) can influence what solid findings are accepted with heartfelt sorrow for those poor patients who won't be able to get proper treatment if they win.

    Oh, and they have quiet words with editors of journals and scientists find their head of departments have had complaints.

    What they never, ever do is answer the criticisms, though they claim they have done so to anyone who will listen.

    If this review finds that exercise is not shown to be useful for ME I will be shocked if there is not a resurgence of newspaper reports of terrorist patients and handwringing of how far Cochrane has fallen.
     
    Sean, Simone, geminiqry and 19 others like this.
  20. Adrian

    Adrian Administrator Staff Member

    Messages:
    6,511
    Location:
    UK
    Shouldn't an early stage of any review be to look at the reliability of the evidence in each study found. Looking at the strength of controls and how they may interact with the measurement system (i.e. looking for things like do the different trial arms control for the right things; would patients see treatments as equal (and blinding comes in here) and whether the measures taken will be impacted by the issues in comparing the different trial arms (which could be subjective outcomes but also other issues with even more objective measures). I realize to analyse a trial in sufficient detail to really understand the issues is a big task but it feels necessary. But I get the impression that systematic reviews don't go as far as really assessing the full data just combining things with a rough quality judgement. Maybe that would work if reports were totally honest and peer review good but that is not the case.

    To my mind any protocol should specify a minimum standard for the acceptability of evidence in terms of its reliability when taking account of possible trial issues. Then the review may say 'we found no trials of sufficient standard'. I guess there is another thing that can be done of looking of secondary outcomes that may be more reliable - but these need vetting for example, it was only after analysis from patients that the PACE group admitted that they didn't do the 6mwt correctly and they are still cherry picking results in that they haven't published the step test (secondary outcome).
     
    bobbler, Simone, JaneL and 10 others like this.

Share This Page