Can we influence trial and review methodology, eg open-label trials with subjective primary outcomes?

Sasha

Sasha

Senior Member (Voting Rights)
This post has been copied and following posts moved from:
2025: The 2019/24 Cochrane Larun review Exercise Therapy for CFS - including IAG, campaign, petition, comments and articles

alktipping said:
but they are not the right sort of people .Being that signatories are not the people who have power and influence to damage their careers.
Click to expand...

I think you make a very good point there! To Cochrane, we probably look like 15,000 patients in straitjackets complaining about being forced to take their meds.

And abuse of a group of patients on this scale is so rare that when patients push back, we're going to look like the weird ones, not the perpetrators - especially since they've managed to paint us as having dodgy thinking.

@Jonathan Edwards @dave30th - What do you think about starting up a petition or very widely circulated open letter for only clinicians/researchers to sign, in their tens of thousands? Probably one not specifically about the Larun review but about the use of open-label trials with subjective primary measures, which would take Larun (and possibly Cochrane) down with it?

This practice needs driving out of medicine. It should never have got in through the door. It's time for the doctors to start the revolution!

Are clinicians/researchers likely to be too trapped in the system to feel free to sign? If so, there's surely a ton of retired ones ready to be unleashed...
 
Last edited by a moderator:
Interesting idea. Maybe an open letter to Cochrane addressing the question of relying on subjective outcomes in open label trials and the flaw in the Grade system of only downgrading rather than rejecting such evidence in reviews.
 
Trish said:
Interesting idea. Maybe an open letter to Cochrane addressing the question of relying on subjective outcomes in open label trials and the flaw in the Grade system of only downgrading rather than rejecting such evidence in reviews.
Click to expand...

I think the target of the kind of letter/petition that I'm proposing needs thinking about very carefully.

If we want big numbers, and we do, it might be better to either target nobody specific, or to target the full range of major organisations that perpetuate this shoddy, dangerous research - so not just Cochrane but the major funding bodies, the major journals, the health services, government bodies that assess research (such as NICE), etc. etc.

I think a lot more potential signatories could get excited about driving this insanity out of the whole of medicine, rather than just Cochrane. This needs to be a movement, IMO.
 
Last edited:
Sasha said:
Are clinicians/researchers likely to be too trapped in the system to feel free to sign?
Click to expand...
If this is a problem in the whole field you are working in, you are setting yourself up to not receive funding or be taken seriously. And giving credence to anyone who say what you work with is all rubbish. I'm sure this will influence people in different ways, but as a dietitian we've been taught to reply to "difficult" patients that all we do is evidence-based. If you stop believing that, what have you really got?

I'd like to get students involved, I've met too many that buy into this because we are taught it's just the way it is.
 
Midnattsol said:
If this is a problem in the whole field you are working in, you are setting yourself up to not receive funding or be taken seriously.
Click to expand...

The people I imagine signing this would be doctors/researchers who don't work in the behavioural fields. @Jonathan Edwards said:

Jonathan Edwards said:
It may be worth reminding people that when I presented my thoughts about the PACE trial to the University College London Division of Medicine (cardiologists, endocrinologists, haematologists, clinical pharmacologists etc.) nearly ten years ago now the audience unanimously agreed that PACE was a non-starter methodologically because it was open label nd with subjective outcomes. This isn't a minor issue. It is the bedrock of trial design. In other branches of medicine we simply do not take any notice of trials like this.
Click to expand...

So those are the kind of people who can see the issue clearly. @Jonathan Edwards, are they likely to feel afraid of signing a petition/open letter?

I'd like to get students involved, I've met too many that buy into this because we are taught it's just the way it is.
Click to expand...

Students signing wouldn't carry much weight but getting students involved in circulating the petition/letter would be a great way to educate them.
 
Sasha said:
The people I imagine signing this would be doctors/researchers who don't work in the behavioural fields. So those are the kind of people who can see the issue clearly.
Click to expand...
Those of us in behavioral fields are taught methodology together with those who go on to be in different fields, I must admit I don't see those other fields as being better informed than "us". Being able to see that a study is not a double blind randomized controlled trial doesn't mean someone can say why it's bad that it's not.

Edit to add: An example of how fields that rely on (blinded) RCT still struggle with methodology is how it is claimed masks don't work against infection despite evidence from other fields (physics) show mask material filter infectious particles. Sure we can talk about if people is unable to use the masks properly and that the filtering may not be 100% similar to in a lab, but that is a long way from saying we need an RCT to prove if masks work.
 
Last edited:
Midnattsol said:
Those of us in behavioral fields are taught methodology together with those who go on to be in different fields, I must admit I don't see those other fields as being better informed than "us". Being able to see that a study is not a double blind randomized controlled trial doesn't mean someone can say why it's bad that it's not.
Click to expand...
I completely agree, but I was addressing your point that if you're in a behavioural field and speak out, your funding may be much more likely to be at risk if you sign such a letter/petition than someone from another field.
 
Sasha said:
I completely agree, but I was addressing your point that if you're in a behavioural field and speak out, your funding may be much more likely to be at risk if you sign such a letter/petition than someone from another field.
Click to expand...
But we work together (which is good for funding: it's multidisciplinary!). And having other fields being critical of behavior research makes it easy to argue to your in-group "those people just don't get what we do" and ignore the criticism like before.
 
Midnattsol said:
But we work together (which is good for funding: it's multidisciplinary!). And having other fields being critical of behavior research makes it easy to argue to your in-group "those people just don't get what we do" and ignore the criticism like before.
Click to expand...
I think I must be missing your point! I wasn't arguing that we should only suggest that people from non-behavioural areas should sign, only that those in behavioural areas might feel under particular pressure not to.

Are we at cross-purposes?
 
Sasha said:
I think I must be missing your point! I wasn't arguing that we should only suggest that people from non-behavioural areas should sign, only that those in behavioural areas might feel under particular pressure not to.

Are we at cross-purposes?
Click to expand...
No, I just think I reacted to who you defined as the people who could see the issue and didn't intend to quarrel on the rest :) I'm in a bit of a bad mood as I've had the pleasure of being told off in healthcare with "all our decisions are evidence based" and "we follow the latest research", while the person ignored what I said and went on to give me outdated advice from my own field. If someone don't want to listen they won't regardless the expertise of the one talking.
 
Sasha said:
Are clinicians/researchers likely to be too trapped in the system to feel free to sign? If so, there's surely a ton of retired ones ready to be unleashed...
Click to expand...

I have come to think that physicians have always been witch-doctors at heart, over history. In the immediate post-war years of 1950-80 scientists became leaders within practical medicine, whereas a generation before the scientists with medical degrees mostly went into science.

The payoff came in the period 1980-2000, with huge advances in all sort of fields, but already by then the witch-doctors had taken back control. I was at the tail end of the scientific era. Now the Royal Colleges are run by compromisers and empire builders.

That doesn't stop the ME/CFS community bucking the system but I don't see any signatures on a petition any time soon.
 
Jonathan Edwards said:
I have come to think that physicians have always been witch-doctors at heart, over history. In the immediate post-war years of 1950-80 scientists became leaders within practical medicine, whereas a generation before the scientists with medical degrees mostly went into science.

The payoff came in the period 1980-2000, with huge advances in all sort of fields, but already by then the witch-doctors had taken back control. I was at the tail end of the scientific era. Now the Royal Colleges are run by compromisers and empire builders.

That doesn't stop the ME/CFS community bucking the system but I don't see any signatures on a petition any time soon.
Click to expand...
But what about all your colleagues who unanimously thought PACE was rubbish and clearly saw the issue with open-label/subjective-measures trials? That suggests that the vast majority of doctors, at least outside the behavioural fields, aren't witch doctors.

Have doctors ever signed a petition or open letter en masse? If so, on what sort of topics?
 
I suspect that changing an entire paradigm across medicine as a whole though desirable is too ambitious at present. I think chipping away at specific issues is the realistic option.

Interestingly the Magenta study did not solely rely on the open label subjective outcomes. Was this the result of years of criticism of the earlier bad research? Is change being effective at least in some areas?
 
Peter Trewhitt said:
I suspect that changing an entire paradigm across medicine as a whole though desirable is too ambitious at present. I think chipping away at specific issues is the realistic option.
Click to expand...

Chipping away at specific issues could very much narrower the amount of professional support that we could expect to get, though. It's a good question where the right balance lies but I think the 'Go big' option is worth seriously considering.
 
I have also wondered if there are indirect things - not ME/CFS specific but that have a wider support base - we could lend our support to that would move things along in a beneficial direction for us. One example: if journals & funders insisted that anonymised raw data had to be uploaded to a publicly accessible data repository. There is already an open data movement but it would be unusually beneficial for us because of the low quality of trials & ideological capture in the ME/CFS field - psychobehavioural triallists would have to be much more honest in their analyses if they know their data would have to be made freely & openly available. Reanalyses would probably become more common, fraud would be harder to get away with; it'd be a boon for robustness in general.

On ethics, too - some of the things in the 2024 Helsinki declaration could make it more difficult to perform "patient-unfriendly" behavioural research - as long as ethics committees & triallists are actually held to those standards. Pretty much any effort aiming to increase methodological robustness, patient involvement in research & open data is something we should learn about & support as it will disproportionately benefit us.
 
You must log in or register to reply here.
Back
Top Bottom