Do more sensitive methods tend to lead to medical breakthroughs?

[EndME]

In regards to LC and ME/CFS research I often see researchers making the claim that current methods are not sensitive enough to detect certain problems, but that their new study using very similar methods that hundreds of others have used before would now be able to show differences because their methods are more sensitive.

That applies to researchers claiming to have to measure inflammatory markers using SIMOA assays rather than ELISA assays or using SIMOA assays rather than ELISA to detect viral antigens and a whole bunch of other technologies and methodologies. Especially in the viral persistence field this seems to be a massive hope.

It seems to me that in the past medical breakthroughs have mostly come through novel ideas and novel technological inventions rather than some technological advancement that shifts your measurement by a decimal place. I would think that in many cases such a technological advancement would not be strictly necessary if one could just circumvent accuracy problems by increasing the sample size.

Typically I cannot judge whether there is much truth to such hopes or whether this is just the type of stuff one typically can easily get grants for. Of course I don't discount the fact that small advancements over multiple decades can end up leading to large jumps and breakthroughs nor that the majority of research tends to be incremental rather than anything else.

I'm just wandering whether there are sensible historical accounts or reasonable arguments in regards to pathology why one should put much hope into an study that looks at the same things many other studies have looked at before but does so using a slightly better tool?

 

[Jonathan Edwards]

The electron microscope allowed us to diagnose minimal change glomerulonephritis, but in general I agree. Sensitivity of test is rarely important.

 

[Yann04]

This really makes me think of Jarred Younger. Who said that by the end of the year with some studies he’s doing in Microgliya with very sensitive CT scan and MRI we would likely have definite proof of neuroinflammation.

I guess we’ll see over the next year if that holds…

 

[EndME]

This really makes me think of Jarred Younger. Who said that by the end of the year with some studies he’s doing in Microgliya with very sensitive CT scan and MRI we would likely have definite proof of neuroinflammation.

I guess we’ll see over the next year if that holds…

Unfortunately, that makes me think of similar things I have heard before where subsequently I often didn't even see any results published at all. It seems that Younger hasn't published on ME/CFS (or LC) in the past few years. Perhaps he has a solid hypothesis and only time can tell. From what I can tell his activated microglia ideas revolving around these releasing cytokines are largely based on his well-cited 2019 study Evidence of widespread metabolite abnormalities in ME/CFS: assessment with whole-brain magnetic resonance spectroscopy which after correcting for multiple comparisons however only had one significant result. That paper doesn't seem to have raised much discussion over here, let's hope it'll be positively different for the follow-up.

 

[Sean]

Could be sensitivity. Or could be not using the appropriate measurement technique, or not using it in the right way, for the particular problem.

Using the finest microscope available is pointless if you are not looking at the appropriate tissue sample, for example.

 

[FMMM1]

In regards to LC and ME/CFS research I often see researchers making the claim that current methods are not sensitive enough to detect certain problems, but that their new study using very similar methods that hundreds of others have used before would now be able to show differences because their methods are more sensitive.

If you look at NIH metabolomics talk (Dr Li - Jackson Laboratories) metabolite coverage is low in ME/CFS - most of the data is from Metabolon's "standard" package.
Couple of thoughts:
1) GWAS (DecodeME) either via standard data analysis, or via PrecisionLife*, may identify potential genes/pathways - that would help to focus the metabolite search - bioarkers;
2) looking at a wider range of metabolites may yield biomarkers.

It seems to me that in the past medical breakthroughs have mostly come through novel ideas and novel technological inventions rather than some technological advancement that shifts your measurement by a decimal place. I would think that in many cases such a technological advancement would not be strictly necessary if one could just circumvent accuracy problems by increasing the sample size.

I recall (correctly?) Jonathan posting that:

• most recent developments have been via genetic studies [like GWAS] i.e. not hypothesis drive;
• Angela Vincent said that there'd need to be a new discovery/technology to solve ME/CFS. The sort of thing that occurs to me is if ME/CFS is a signal [e.g. TGFbeta**] which leads to your brain telling you you're unwell - you can't measure that brain talk - you might be able to piece the mechanism together though from other clues e.g. GWAS or even identify the signal (talk to Brian Walitt!).

I'm just wandering whether there are sensible historical accounts or reasonable arguments in regards to pathology why one should put much hope into an study that looks at the same things many other studies have looked at before but does so using a slightly better tool?

If you look at metabolomics (my first comment in this post) I think a very general study (looking at as many metabolites as possible) versus a more focused study - e.g. looking at pathways turned up via DecodeME - might be very different i.e. due to higher sensitivity. Some things really stand out - others not so e.g. B12 deficiency can be identified via measuring methylmalonic acid; in humans it requires greater sensitivity (i.e. compared to cattle, sheep etc.).

*https://twitter.com/user/status/1791044092224208929


**https://www.s4me.info/threads/remarkable-researchers-hunting-for-‘something-in-the-blood’-of-people-with-me.38429/page-2#post-531401

 

[EndME]

If you look at NIH metabolomics talk (Dr Li - Jackson Laboratories) metabolite coverage is low in ME/CFS - most of the data is from Metabolon's "standard" package.
Couple of thoughts:
1) GWAS (DecodeME) either via standard data analysis, or via PrecisionLife*, may identify potential genes/pathways - that would help to focus the metabolite search - bioarkers;
2) looking at a wider range of metabolites may yield biomarkers.


I recall (correctly?) Jonathan posting that:

• most recent developments have been via genetic studies [like GWAS] i.e. not hypothesis drive;
• Angela Vincent said that there'd need to be a new discovery/technology to solve ME/CFS. The sort of thing that occurs to me is if ME/CFS is a signal [e.g. TGFbeta**] which leads to your brain telling you you're unwell - you can't measure that brain talk - you might be able to piece the mechanism together though from other clues e.g. GWAS or even identify the signal (talk to Brian Walitt!).

If you look at metabolomics (my first comment in this post) I think a very general study (looking at as many metabolites as possible) versus a more focused study - e.g. looking at pathways turned up via DecodeME - might be very different i.e. due to higher sensitivity. Some things really stand out - others not so e.g. B12 deficiency can be identified via measuring methylmalonic acid; in humans it requires greater sensitivity (i.e. compared to cattle, sheep etc.).

*https://twitter.com/user/status/1791044092224208929


**https://www.s4me.info/threads/remarkable-researchers-hunting-for-‘something-in-the-blood’-of-people-with-me.38429/page-2#post-531401

Option 1) would be an argument against more sensitivity and an argument for targeted research based on previous results.
Option 2) I find hard to say anything about. Is there any precedence for wider than standard metabolic research being able to differentiate a signal from noise in an illness/disease where previously nothing of note had been detected? Anecdotally, I've found a lot of the metabolimics research to be similar to microbiome research where it's hard to tell if anything possibly could resemble a signal, hopefully time will show that I'm wrong.

 

[Kitty]

Is there any precedence for wider than standard metabolic research being able to differentiate a signal from noise in an illness/disease where previously nothing of note had been detected? Anecdotally, I've found a lot of the metabolimics research to be similar to microbiome research where it's hard to tell if anything possibly could resemble a signal, hopefully time will show that I'm wrong.

My impression is that the field is at such an early stage of development it's hard to know what any of the results mean. Doesn't mean it's not a promising avenue, just that researchers probably still have a lot to learn about how to manage and interpret the vast mass of data it produces.

 

[FMMM1]

Option 1) would be an argument against more sensitivity and an argument for targeted research based on previous results.

Option 1 - focused metabolomics i.e. post DecodeME - basically hypothesis free metabolomics will have reduced sensitivity (lower detection limits) - since you're looking as widely as possible. More focused metabolomics should have better detection limits and the levels will be more accurate (closer to true value). So that offers the potential to identify a biomarker which wasn't picked up in less focused metabolomics.

Option 2) I find hard to say anything about. Is there any precedence for wider than standard metabolic research being able to differentiate a signal from noise in an illness/disease where previously nothing of note had been detected? Anecdotally, I've found a lot of the metabolimics research to be similar to microbiome research where it's hard to tell if anything possibly could resemble a signal, hopefully time will show that I'm wrong.

Option 2 - higher metabolite coverage - basically your hypothesis free and looking for something that stands out e.g. something that is increased/decreased - biomarker. I worked in a lab (lowest level of technician) and I recall a PhD student (now a professor of biochemistry) pointing out a mysterious peak - only present in the test samples (not the healthy controls). He was right of course, but his discovery was "transferred" to another up and coming individual --- way of the world (science). But yes, strange findings (I think Jonathan spoke of counterintuitive findings) can be informative!

All of the above is based on limited knowledge!

 

[FMMM1]

I recall Jonathan posting that disease tends to stand out like a barn door - so yes - suggests that you need to change the strategy i.e. rather than looking again at "Y" with 2X sensitivity. That's basically what GWAS DecodeME aims to do --- move away from testing/retesting hypothesis.
I still think metabolomics is worth pursuing - it's actually about measuring everything i.e. new things - rather than looking again at "Y" with 2X sensitivity. Also, if DecodeME doesn't provide clues [in dementia multiple studies the size of DecodeME were required to find clues] then metabolomics may be one of the remaining hypothesis free options.

@EndME
"Anecdotally, I've found a lot of the metabolimics research to be similar to microbiome research where it's hard to tell if anything possibly could resemble a signal, hopefully time will show that I'm wrong."
Agree but we may not have a better/ideal option!

 

[Trish]

I recall Jonathan posting that disease tends to stand out like a barn door - so yes - suggests that you need to change the strategy i.e. rather than looking again at "Y" with 2X sensitivity.

Can you link the post you're referring to please, so we can see the context.

 

[FMMM1]

Can you link the post you're referring to please, so we can see the context.

"In the meantime, the drug company scientists could see barn door that the data had to be significant because they used objective measures back up by pharmacodynamic profiles that could not conceivably have any other explanation. "
https://www.s4me.info/threads/check...l-and-pilot-2023-mol-et-al.37953/#post-524799

 

[Jonathan Edwards]

Can you link the post you're referring to please, so we can see the context.

I think I would be happy to say that in general in medical science once you have hit on the right thing to measure the difference you are looking for isn't hard to spot. There are exceptions. Using PCR to find micro-organisms in blood is a good example. You won't see them down a microscope unless you culture the blood for hours. But by and large you already know the thing is there, the more sensitive test just makes it more oratorical to find in a routine clinical situation.

 

[FMMM1]

My initial

In regards to LC and ME/CFS research I often see researchers making the claim that current methods are not sensitive enough to detect certain problems, but that their new study using very similar methods that hundreds of others have used before would now be able to show differences because their methods are more sensitive.

That applies to researchers claiming to have to measure inflammatory markers using SIMOA assays rather than ELISA assays or using SIMOA assays rather than ELISA to detect viral antigens and a whole bunch of other technologies and methodologies. Especially in the viral persistence field this seems to be a massive hope.

It seems to me that in the past medical breakthroughs have mostly come through novel ideas and novel technological inventions rather than some technological advancement that shifts your measurement by a decimal place. I would think that in many cases such a technological advancement would not be strictly necessary if one could just circumvent accuracy problems by increasing the sample size.

Typically I cannot judge whether there is much truth to such hopes or whether this is just the type of stuff one typically can easily get grants for. Of course I don't discount the fact that small advancements over multiple decades can end up leading to large jumps and breakthroughs nor that the majority of research tends to be incremental rather than anything else.

I'm just wandering whether there are sensible historical accounts or reasonable arguments in regards to pathology why one should put much hope into an study that looks at the same things many other studies have looked at before but does so using a slightly better tool?

Reminds me of @Snow Leopard annoyance at researchers repeatedly looking in the same place and --- spoiler alert --- finding nothing (again!).
In terms of sensitivity (as per comment above) there are times when it's relevant e.g. I think blood sodium potassium ratio is critical - so you'd need to measure both accurately. As per my comments above, I'm hoping that DecodeME (GWAS) and/or whole genome sequence/rare variant studies [EDIT - I'm hoping NIH will fund one - Research Roadmap] will identify relevant genes and that will focus attention on genes /pathways - identification of biomarkers. Limits of detection/sensitivity can be remarkably low - problem is we don't know where to look!