3 Levers to pull to solve a disease

[Siebe]

This was originally written as a comment on the new podcast about medical innovation called Hard Drugs by Works in Progress.

The last couple of days I've been thinking about how to solve a disease in general, and which levers there are for patients & policymakers. I wish there was a playbook!

I identify 3 major levers:

1. Research quantity/funding. This is the obvious lever that a lot of advocacy focuses on. Simple to understand, but hard to convince politicians of! What does the shape of the input-output curve look like, and when do diminishing returns really set in? (ME/CFS funding is like 10x below average, so obviously there's a lot of focus on funding)

2. Research quality. I have the impression that this matters greatly, and spans many factors or even orders of magnitude. You can have heaps of money, but with a toxic field you don't get far (think Alzheimer's). For ME/CFS, 89% of studies in the last decade did not even mandate subjects to have the defining symptom of the disease (Post-Exertional Malaise), which gives a general sense of how bad the field is. imagine a lot comes down to peer review quality (I'm excited for Al improving the process), and perhaps some diseases offer more prestige so attract more talent? think patients can affect this, but it's hard. I've also seen IFP innovating the grant-making process, and would like to add that patients have a pretty unique perspective and e.g. the Patient-Led Research Collaborative had a pretty good regranting program. I imagine a couple of separate entities, one run by patients, would be far more effective than trying to involve patients in in the legacy process where they're often not empowered.

3. Market incentives. Others have said a lot about this. FDA/ EMA processes have pushed the cost of drug development >$1 billion. Earlier market access while still forcing rigorous Phase 3 trials eventually seems like a good policy. In my uninformed impression, patient advocacy groups don't focus much on this, and don't understand the pharma industry well. I wish they would collaborate more to pressure reform here.

More generally, I wonder which theory of change is more accurate/cost-effective:

A) Understand first. Figure out how a disease works through non-drug research, then try to solve that
B) Trial-and-error. Run trials trying to understand and maybe coincidentally hit something that works.

I feel like A is often the assumed model, but B would probably work better, IF drugs were easier to get to market.

 

[Yann04]

Welcome @Siebe !
Nice to see you here

 

[Utsikt]

Welcome!
The is an interesting topic.

2. Research quality

imagine a lot comes down to peer review quality (I'm excited for Al improving the process)

I’m not sure better peer review would increase the quality of published studies, unless you were able to increase the quality across the entire board. Mostly because if one journal becomes harder to publish in, the researchers will just go to another journal. We already see this where many researchers only publish in journals that are sympathetic to their views.

I’m also curious about how AI could improve the peer reviews? What are your thoughts here?

perhaps some diseases offer more prestige so attract more talent?

Prestige could also attract people that like to play the prestige game. And there are easier ways to gain prestige than to solve really hard problems.

 

[Siebe]

I’m also curious about how AI could improve the peer reviews? What are your thoughts here?

My uninformed impression is that a lot of ME/CFS peer review (as well as many other fields) is bad because researchers that aren't familiar with the field are asked to review it. I think AI can currently do better than ~80% of peer reviewers, and with good system prompting and tooling probably 95-98%

 

[Siebe]

Prestige could also attract people that like to play the prestige game. And there are easier ways to gain prestige than to solve really hard problems.

Sure there are downsides to prestige, but at the current stage I think more prestige would clearly be a net benefit

 

[butter.]

This was originally written as a comment on the new podcast about medical innovation called Hard Drugs by Works in Progress.

The last couple of days I've been thinking about how to solve a disease in general, and which levers there are for patients & policymakers. I wish there was a playbook!

I identify 3 major levers:

1. Research quantity/funding. This is the obvious lever that a lot of advocacy focuses on. Simple to understand, but hard to convince politicians of! What does the shape of the input-output curve look like, and when do diminishing returns really set in? (ME/CFS funding is like 10x below average, so obviously there's a lot of focus on funding)

2. Research quality. I have the impression that this matters greatly, and spans many factors or even orders of magnitude. You can have heaps of money, but with a toxic field you don't get far (think Alzheimer's). For ME/CFS, 89% of studies in the last decade did not even mandate subjects to have the defining symptom of the disease (Post-Exertional Malaise), which gives a general sense of how bad the field is. imagine a lot comes down to peer review quality (I'm excited for Al improving the process), and perhaps some diseases offer more prestige so attract more talent? think patients can affect this, but it's hard. I've also seen IFP innovating the grant-making process, and would like to add that patients have a pretty unique perspective and e.g. the Patient-Led Research Collaborative had a pretty good regranting program. I imagine a couple of separate entities, one run by patients, would be far more effective than trying to involve patients in in the legacy process where they're often not empowered.

3. Market incentives. Others have said a lot about this. FDA/ EMA processes have pushed the cost of drug development >$1 billion. Earlier market access while still forcing rigorous Phase 3 trials eventually seems like a good policy. In my uninformed impression, patient advocacy groups don't focus much on this, and don't understand the pharma industry well. I wish they would collaborate more to pressure reform here.

More generally, I wonder which theory of change is more accurate/cost-effective:

A) Understand first. Figure out how a disease works through non-drug research, then try to solve that
B) Trial-and-error. Run trials trying to understand and maybe coincidentally hit something that works.

I feel like A is often the assumed model, but B would probably work better, IF drugs were easier to get to market.

On a quite fundamental level it's impossible to 'really know' what's 'best to do' to solve a disease. Most people will (by necessity) reason by analogy and not from first principles and logic.

People will say disease X gets so and so much funding hence looking at disease burden ME/CFS should get so and so much, but in reality that's simply not how it works very often. People will say disease Y has been solved by applying this and that technique and so we should use this and that technique, but in reality that's not very useful very often.
....

The most reasonable approach from an advocacy POV for the vast majority of advocates (there are exceptions of course) is to use their energy to increase funding one way or the other. Getting healthy people to advocate for more funding is probably the greatest and most underutilized force multiplier atm. You say there is a focus on funding already, I would disagree with that. There is hardly any serious fr for ME/CFS. It seems quite likely funding will further decrease over the next years globally, simply because US institutional funding is trending towards zero (yes!).

One of the very few things we can confidently say maybe is that more energy and intelligence will be needed to solve this problem. Money is a 'storage' and exchange medium for energy and in extension intelligence. So we need more of it.

 

[poetinsf]

I've been wondering, could ME/CFS be solved if trillions of dollars were available? Despite LC making the splash recently, ME/CFS is still not conceived as serious or visceral as cancer or Altheimer's in popular imagination. Add to that the notorious difficulty of ME/CFS research, you could be thinking you are more likely to get the Nobel prize for a breakthrough in cancer or Altheimer than ME/CFS if you are an ambitious researcher. Which means money may attract more quantity than quality.

More generally, I wonder which theory of change is more accurate/cost-effective:

A) Understand first. Figure out how a disease works through non-drug research, then try to solve that
B) Trial-and-error. Run trials trying to understand and maybe coincidentally hit something that works.

We had a debate about it a while ago. I was advocating doing B as well since we haven't made much headway with A. But I do understand why getting the drug out to the market is difficult and why off-level use is restricted to special cases only. It's probably not realistic to do B in wider scale.

 

[Utsikt]

My uninformed impression is that a lot of ME/CFS peer review (as well as many other fields) is bad because researchers that aren't familiar with the field are asked to review it. I think AI can currently do better than ~80% of peer reviewers, and with good system prompting and tooling probably 95-98%

Where do you get those numbers from?

And as I mentioned earlier, why would better peer review in some journals improve the standards across the field?

We already have editors that are willing to print studies that an undergrad in an unrelated field could pick apart in minutes. You’re assuming that everyone involved wants the science to be as good as possible. That is very rarely the case, as can be illustrated by e.g. the refusal to retract PACE and Walitt et al by the Lancet and Nature.

Sure there are downsides to prestige, but at the current stage I think more prestige would clearly be a net benefit

Why? Do you have any reason to believe that more prestige in itself would attract researchers that have the ability to make a substantial difference to the field of ME/CFS the next decade?

Who are those researchers, why do they care about prestige, and how would ME/CFS become prestigious enough to make them swap from what you would have to assume are already quite prestigious fields?