RoB 2: a revised tool for assessing risk of bias in randomised trials (2019) Sterne et al.

Going from memory, in PACE the expectations for GET and APT were not different, but this was at baseline. Using this to claim that expectation bias was not a concern implies also claiming that the different treatments received later infused exactly the same expectations in participants, which seems rather dubious.

At the time, CBT/GET was already recommended by health authorities as most effective treatment and it promised a cure. The illness model of CBT/GET describes reducing activities in response to symptoms (an integral part of pacing) as maladaptive behaviour. Therapists would probably favor CBT/GET simply because it provided more job opportunities (pacing is usually not therapist delivered, and I think at the time CBT/GET was already available on the NHS).

 

Could you give an example or do you mean that the more a tool tries to specify with rules how bias should be assessed, the more it gives the impression that no judgement is needed?

I don't think anyone would disagree with that, it's just that they have chosen to use only 3 possible outcomes for the risk of bias assessment: low risk of bias, some concerns or high risk of bias.

In a sense, that means that a trial can be downgraded in only two steps to a high risk of bias. I think that it explains why it is now more difficult to rate a trial as high risk of bias for a particular domain. Do you think they should have used more than three outcomes?

 

This was only for blinding though.

For selective reporting, it seems that deviations from the protocol are taken less seriously in the new tool. In the old tool a trial had to follow the protocol in reporting the outcome. It reads:

In the new tool, this is assessed at question 5.1. In the elaboration they write:

So that seems a problem, as others have already noted. They should have probably maintained the old description, as a failure to follow the protocol, would only result in some concerns, not a high risk of bias.

 

Thanks

I'm going to link to your submitted comment, published on the other thread, in case some might have missed it:
https://www.s4me.info/threads/bias-due-to-a-lack-of-blinding-a-discussion.11429/#post-203732

 

What we would normally understand under risk of bias due to lack of blinding (the expectations, hopes and response bias of patients) is assessed at Domain 4: Risk of bias in measurement of the outcome, question 4.3 to 4.5.

More info in this long post: https://www.s4me.info/threads/rob-2...ls-2019-sterne-et-al.11025/page-5#post-203995

 

Precisely. It is non sequitur and deliberately so.

 

I think they may be confusing risk with consequence. There are many scenarios where the consequence of any one event occurring, out of a set of possible events, will be as serious as any of the other events. e.g. Any one of the following can kill you: being hit by lightning, falling off a cliff, being shot, being struck by flying debris, etc. The worst-case consequence of any one of them is effectively the same as all of them happening to you - you die. It's a logical OR.

But the risk of being killed by such events is clearly less for the person who is simply in a thunderstorm, compared to another person who is walking along a cliff top in a thunderstorm, with gales blowing stuff (including the person) all over the place, and being threatened by a gunman into the bargain.

Risk of an event occurring is distinct from the consequence of an event occurring. I'm not sure they appreciate that. Or maybe they do, but want to mislead people into thinking risk can be treated that way.

ETA: Corrected "It's a logical AND" to "It's a logical OR". Wrote it in too much of a hurry. Any one of the events means you are dead; more events does not make you more dead.

 

Simple risk assessment - risk x likelihood of adverse effect It would no doubt be illuminating to see any of those for BPS input.

 

I think Larun's argument is ridiculous because the patients who really don't like GET would never consent to participate in such a trial in the first place.

Participants who are willing to participate in such a trial likely have "an open mind" if you know what I mean and even if they have preconceived ideas, participants will still be subject to the usual set of biases as with any trial. But notably, these therapies were explicitly designed to challenge cognitions hence GET and CBT are inherently more subject to self reporting biases than APT that made patients more conscious of their limitations and lack of improvement.

 

They struggles to get patients to enter the trial which is why they changed the entry score from 60 to 65 and put out word to doctors that people with fibro could be referred.

So people who did not want to do GET never went near the trial in the first place.

 

Finally I submitted the rapid response to the BMJ and they didn't publish it!! I spent hours and hours honing it, and got some excellent feedback which improved it immensely, so that was a bit annoying. So I posted it on my blog...and tagged @jonathanasterne.
https://healthycontrolblog.wordpres...-assessing-risk-of-bias-in-randomised-trials/

 

Finally finally after my having to remove any reference to the "P" word or the problems with the Exercise Review, and chase a couple of times, my not very rapid Rapid Response got published
https://www.bmj.com/content/366/bmj.l4898/rr-3
woohoo!!!!!

 

Link seems to be not working, but this one is: https://www.bmj.com/content/366/bmj.l4898/rapid-responses

 

Errr. Probably not. I think they get sent to at least the first author....

 

Irrespective of whether these things get heeded or not, I think it is so very important they get into the written record, as your response has done. Hopefully the day may come where people really do start to take note and heed what is going on.

 

Yes. Very important for future historians of medicine to know what happened in the early 21st century. Unfortunately, journal correspondence (whether in print or online) does very little to change anything if the original authors don't want to make corrections. And it's very, very rare for the journal to insist that they do so.

Today, I've just been looking back at the huge amount of correspondence that a certain article in the Journal of the Royal College of GPs generated 30 years ago...

 

That "certain article" doesn't want to download. At present it must be considered an indefinite article.

EDIT fixed now

 

Tha t certai article shows the dangers inherent in the use an d misuse of language. It is strange how they "propose a model" but then use the indicative mood to express it, rather than offering some degree of conditionality. I am sure the perpetrators passed their "Use of English" exams and knew what they were doing.