New Zealand: Dr Matthew Phillips, neurologist

[rvallee]

This is wild to me. There is so much gray area in medical research, so many unknowns. Why in the name of all that is holy does trial design remain so muddled? Surely that is something that can be pinned down. Seems like such a massive drag on progress.

The purpose of a system is what it does, there is no point in claiming that the purpose of a system is to do what it constantly fails to do.

There just isn't any interest in rigorous trials that give clear "no" answers to almost every question. It might be the stated purpose of those systems, but it's not what they actually do.

 

[bobbler]

I have tried to look up the current education programmes for medical doctors in Norway. I have not been able to found any direct mentions of trial methodology. I also hear from people that have gone trough the training or worked/studied with those people that they really don’t learn anything about it.

It might be mentioned in some lectures, but I’m fairly certain it’s not emphasised based on those anecdotes, at least not to the extent that it should.

It also aligns with my experience from economics. Our methodology class for our master’s thesis essentially told us you have to write that something might be this way instead of that it is this way, but qualitative research was presented as a perfectly valid method and not as one with substantial limitations.

Agreed. there is spoon-feeding to exams in most subjects and sadly for example when learning statistics they are taught to be more concerned about ticking the box on the p-number than on why that p is even used in the first place. Or who needs to be in that sample for there even to be any point about calculating it.

I don't think a single behaviourist for example would be able to explain what external validity of their sample (vs a population) is.

I do think that such things should be known and it should be not just a source of shame/embarrassment when they get it wrong or make a mistake that wasn't an honest I feel so mortified typo error done every so often rather than trying it on, and should have that written all over the literature.

But there was no shame when their therapist-delivered literature was analysed as low and very low quality by Nice, just them playing the victim and focusing on targeting patients - again - with slander instead of any intention to be any better themselves at what they claim to be capable of doing.

 

[rvallee]

And a big part of this ongoing production of poor quality trials must be that these designs produce positive outcomes, even for ineffective treatments. If you already believe the treatment works and just want a trial result to promote it

It's become so normalized that published protocols can simply state it and no one cares, they'll still pretend there is any "equipoise". From Efficacy of blended digital and face-to-face psychotherapy compared to enhanced psychotherapy for patients with [SSD] (iSOMA+): 2026 Jutzi et al:

Nonetheless, the superiority of blended psychotherapy is assumed, given the multimedial and interactive design as well as further persuasive, that is, engagement-enhancing, features of iSOMA (i.e., automatic reminders, progress bar, unlocking achievements), compared to passive bibliotherapy

It's become a game of persuasion, it's not even subtle anymore. It's become normalized to try to get people to change how they think, then asking them what they think, which might not even be what they actually think, but it looks good enough on a biased questionnaire, as long as no one thinks about it.

Like I say often, this is far bigger than just us, or even neglected/discriminated illnesses. It's become the purpose of the system to fake positives. It "gives people hope", or whatever.

In almost every other industry this would be considered fraudulent practices. In health care it's just become "the way things are done". It's all part of the same game of persuasion, and now is more persuaded of this than the people who control, influence and work in those systems.

 

[bobbler]

Agreed. there is spoon-feeding to exams in most subjects and sadly for example when learning statistics they are taught to be more concerned about ticking the box on the p-number than on why that p is even used in the first place. Or who needs to be in that sample for there even to be any point about calculating it.

I don't think a single behaviourist for example would be able to explain what external validity of their sample (vs a population) is.

I do think that such things should be known and it should be not just a source of shame/embarrassment when they get it wrong or make a mistake that wasn't an honest I feel so mortified typo error done every so often rather than trying it on, and should have that written all over the literature.

But there was no shame when their therapist-delivered literature was analysed as low and very low quality by Nice, just them playing the victim and focusing on targeting patients - again - with slander instead of any intention to be any better themselves at what they claim to be capable of doing.

Then there is the fact that even basics which would get you a fail on an undergrad project class or stats exam of not checking whether data is parametric or non-parametric when choosing which test to use - and those aren't final year tests or knowledge either - ended up being used by one big project didn't it without anyone pulling it up or checking

I wonder whether those reading it were understanding enough of what it meant to just pass it off as 'ahh stats, I don't like it, a test is a test, they are close enough' or the issue was so basic that noone checked? and because of this assumption the working is not required to be spelled out so it is easily checked without a reader downloading the data and having to do all the working themselves to see?

 

[rvallee]

It was not something you got in lectures or text books. It was something you absorbed from day to day involvement with evidence analysis. Maybe now that has been replaced by 'pathways'.

It seems to place the change there, then. It's learning through culture, rather than formal, and the culture has moved to want to lower standards. Which seems kind of expected, frankly. Something this critical being informally taught is bound to cause issues like this.

Now, promoting a small % of positive anecdotes, presented with 'hope' and "mind over matter" magic as "evidence-based" gets applause at conferences and awards. The culture has spoken in favor of this, and with nothing formal to stop, or even slow, the decline, well that explains the race to the bottom. There's something natural about people describing the emperor's new clothes yet again that to make their descriptions interesting, they will only get more flowery over time, otherwise there just isn't much to say.

It's an influencer economy now, where controversy is currency. Pseudoscience like this used to get dismissed without giving it a first thought. Now we get absurd nonsense about sniffing chocolate while lifting weights and firmly-held beliefs that we must, MUST, somehow expect bad things to happen, even if we don't, because that's what the models say, or whatever.