1. Sign our petition calling on Cochrane to withdraw their review of Exercise Therapy for CFS here.
    Dismiss Notice
  2. Guest, the 'News in Brief' for the week beginning 18th March 2024 is here.
    Dismiss Notice
  3. Welcome! To read the Core Purpose and Values of our forum, click here.
    Dismiss Notice

RoB 2: a revised tool for assessing risk of bias in randomised trials (2019) Sterne et al.

Discussion in 'Research methodology news and research' started by ME/CFS Skeptic, Aug 29, 2019.

  1. Hoopoe

    Hoopoe Senior Member (Voting Rights)

    Messages:
    5,234
    Going from memory, in PACE the expectations for GET and APT were not different, but this was at baseline. Using this to claim that expectation bias was not a concern implies also claiming that the different treatments received later infused exactly the same expectations in participants, which seems rather dubious.

    At the time, CBT/GET was already recommended by health authorities as most effective treatment and it promised a cure. The illness model of CBT/GET describes reducing activities in response to symptoms (an integral part of pacing) as maladaptive behaviour. Therapists would probably favor CBT/GET simply because it provided more job opportunities (pacing is usually not therapist delivered, and I think at the time CBT/GET was already available on the NHS).
     
    Last edited: Sep 25, 2019
  2. ME/CFS Skeptic

    ME/CFS Skeptic Senior Member (Voting Rights)

    Messages:
    3,495
    Location:
    Belgium
    Could you give an example or do you mean that the more a tool tries to specify with rules how bias should be assessed, the more it gives the impression that no judgement is needed?

    I don't think anyone would disagree with that, it's just that they have chosen to use only 3 possible outcomes for the risk of bias assessment: low risk of bias, some concerns or high risk of bias.

    In a sense, that means that a trial can be downgraded in only two steps to a high risk of bias. I think that it explains why it is now more difficult to rate a trial as high risk of bias for a particular domain. Do you think they should have used more than three outcomes?
     
    Annamaria likes this.
  3. ME/CFS Skeptic

    ME/CFS Skeptic Senior Member (Voting Rights)

    Messages:
    3,495
    Location:
    Belgium
    This was only for blinding though.

    For selective reporting, it seems that deviations from the protocol are taken less seriously in the new tool. In the old tool a trial had to follow the protocol in reporting the outcome. It reads:
    In the new tool, this is assessed at question 5.1. In the elaboration they write:
    So that seems a problem, as others have already noted. They should have probably maintained the old description, as a failure to follow the protocol, would only result in some concerns, not a high risk of bias.
     
  4. ME/CFS Skeptic

    ME/CFS Skeptic Senior Member (Voting Rights)

    Messages:
    3,495
    Location:
    Belgium
    Caroline Struthers likes this.
  5. ME/CFS Skeptic

    ME/CFS Skeptic Senior Member (Voting Rights)

    Messages:
    3,495
    Location:
    Belgium
    Barry likes this.
  6. Jonathan Edwards

    Jonathan Edwards Senior Member (Voting Rights)

    Messages:
    13,278
    Location:
    London, UK
    Precisely. It is non sequitur and deliberately so.
     
    Annamaria, Amw66 and Barry like this.
  7. Barry

    Barry Senior Member (Voting Rights)

    Messages:
    8,385
    I think they may be confusing risk with consequence. There are many scenarios where the consequence of any one event occurring, out of a set of possible events, will be as serious as any of the other events. e.g. Any one of the following can kill you: being hit by lightning, falling off a cliff, being shot, being struck by flying debris, etc. The worst-case consequence of any one of them is effectively the same as all of them happening to you - you die. It's a logical OR.

    But the risk of being killed by such events is clearly less for the person who is simply in a thunderstorm, compared to another person who is walking along a cliff top in a thunderstorm, with gales blowing stuff (including the person) all over the place, and being threatened by a gunman into the bargain.

    Risk of an event occurring is distinct from the consequence of an event occurring. I'm not sure they appreciate that. Or maybe they do, but want to mislead people into thinking risk can be treated that way.

    ETA: Corrected "It's a logical AND" to "It's a logical OR". Wrote it in too much of a hurry. Any one of the events means you are dead; more events does not make you more dead.
     
    Last edited: Sep 25, 2019
    Simbindi, Annamaria, rvallee and 2 others like this.
  8. Amw66

    Amw66 Senior Member (Voting Rights)

    Messages:
    6,263
    Simple risk assessment - risk x likelihood of adverse effect It would no doubt be illuminating to see any of those for BPS input.
     
    Mithriel, Annamaria and Barry like this.
  9. Snow Leopard

    Snow Leopard Senior Member (Voting Rights)

    Messages:
    3,823
    Location:
    Australia
    I think Larun's argument is ridiculous because the patients who really don't like GET would never consent to participate in such a trial in the first place.

    Participants who are willing to participate in such a trial likely have "an open mind" if you know what I mean and even if they have preconceived ideas, participants will still be subject to the usual set of biases as with any trial. But notably, these therapies were explicitly designed to challenge cognitions hence GET and CBT are inherently more subject to self reporting biases than APT that made patients more conscious of their limitations and lack of improvement.
     
    Sean, Annamaria, rvallee and 3 others like this.
  10. Mithriel

    Mithriel Senior Member (Voting Rights)

    Messages:
    2,815
    They struggles to get patients to enter the trial which is why they changed the entry score from 60 to 65 and put out word to doctors that people with fibro could be referred.

    So people who did not want to do GET never went near the trial in the first place.
     
    Annamaria, rvallee, lycaena and 3 others like this.
  11. Caroline Struthers

    Caroline Struthers Senior Member (Voting Rights)

    Messages:
    821
    Location:
    Oxford UK
    ladycatlover, Roy S, Amw66 and 9 others like this.
  12. Caroline Struthers

    Caroline Struthers Senior Member (Voting Rights)

    Messages:
    821
    Location:
    Oxford UK
    Finally finally after my having to remove any reference to the "P" word or the problems with the Exercise Review, and chase a couple of times, my not very rapid Rapid Response got published
    https://www.bmj.com/content/366/bmj.l4898/rr-3
    woohoo!!!!!
     
  13. Lucibee

    Lucibee Senior Member (Voting Rights)

    Messages:
    1,483
    Location:
    Mid-Wales
  14. Trish

    Trish Moderator Staff Member

    Messages:
    51,890
    Location:
    UK
    Very good, @Caroline Struthers, it's good to see it there along with the responses by Michiel Tack and Jonathan Edwards.
    Is there any chance anyone will take any notice of these responses?
     
  15. Caroline Struthers

    Caroline Struthers Senior Member (Voting Rights)

    Messages:
    821
    Location:
    Oxford UK
    Errr. Probably not. I think they get sent to at least the first author....
     
  16. Barry

    Barry Senior Member (Voting Rights)

    Messages:
    8,385
    Irrespective of whether these things get heeded or not, I think it is so very important they get into the written record, as your response has done. Hopefully the day may come where people really do start to take note and heed what is going on.
     
  17. Sean

    Sean Moderator Staff Member

    Messages:
    7,046
    Location:
    Australia
    This.
     
    rvallee, Barry, Amw66 and 2 others like this.
  18. Lucibee

    Lucibee Senior Member (Voting Rights)

    Messages:
    1,483
    Location:
    Mid-Wales
    Yes. Very important for future historians of medicine to know what happened in the early 21st century. Unfortunately, journal correspondence (whether in print or online) does very little to change anything if the original authors don't want to make corrections. And it's very, very rare for the journal to insist that they do so.

    Today, I've just been looking back at the huge amount of correspondence that a certain article in the Journal of the Royal College of GPs generated 30 years ago...
     
  19. chrisb

    chrisb Senior Member (Voting Rights)

    Messages:
    4,602
    That "certain article" doesn't want to download. At present it must be considered an indefinite article.

    EDIT fixed now
     
    Last edited: Oct 25, 2019
    MSEsperanza, Barry and Lucibee like this.
  20. chrisb

    chrisb Senior Member (Voting Rights)

    Messages:
    4,602
    Tha t certai article shows the dangers inherent in the use an d misuse of language. It is strange how they "propose a model" but then use the indicative mood to express it, rather than offering some degree of conditionality. I am sure the perpetrators passed their "Use of English" exams and knew what they were doing.
     
    MSEsperanza and Lucibee like this.

Share This Page