Assessing cellular energy dysfunction in CFS/ME using a commercially available laboratory test, 2019, Morten, Newton et al

[Hoopoe]

I just thought it was the decent think to do, given the damning statements in this paper about the Accumen test. I don't ever remember reading such strong language in a Scientific paper. The UK produces very little biomedical research as it is, why antagonise further......... now we have "press releases" rather than some quiet phone calls to resolve differences.

Of course, Myhill/McLaren-Howard should reach out too. But they have been "slapped in the face" hard, and in public, and I would expect them to be very upset. I would, if it was my work.

One way to look at the situation is that patients were being sold an expensive test that isn't fit for purpose. Maybe the replication attempt is flawed and the test works but it's not a good sign that there is disagreement. The possibility that the test isn't fit for purpose must be seriously considered.

It's painful but if this kind of questioning didn't occur, real progress would be delayed even further (there would at best be an illusion of progress).

It's better finding out that something doesn't work after a few years than finding out after 20.

That any replication attempts are being made in general is a good sign. It would be worse if there were no attempts.

 

[InitialConditions]

I hope this isn't just left as it is now. This sort of discrepancy does occur quite a lot in science. And to the outsider it can seem frustrating because one naturally asssumes that one party is correct and the other is wrong. Sometimes, it's not that simple - especially when it comes down to methedology, which often isn't definite, but a series of approximations and assumptions, with 'best' practice rather than 'correct' practice.

 

[Amw66]

This, presumably, is Dr Myhill's clinic in Wales? So the blood samples would need to be sent by post, unless patients had the blood draw done separately at Biolab in London.




So, JMH was "blinded to the *Ability*" (a mutually agreed number on the Bell Ability Scale, between patient and SM), but, surely, if the samples have been obtained in different ways and at different times, some by himself and others by the private medical clinic, he would know what was what. This wasn't a blinded study.

Can we assume control subjects attended Biolab in London, in person? Or might they have sent samples by post?


Hmm, I'm not sure about this... Were is the information on storage time, for patient samples vs controls?


Just adding that info for completeness.

when we did the tests, blood had to be at Biolab for 9am the next morning, with a recommendation that blood be drawn in the previous afternoon to minimise timeframe. It was a We good bloods drawn at 4.10pm and was in the post at 4.30pm.
There is, I believe, a viability test done re blood quality and if not suitable ( assuming a level pf degradation) the test is not run.
The curious bit is the accurate correlation with severity.

Norman Booth was passionate about replication. Given the current situation, I agree with @wigglethemouse - get the labs working together to bottom out/ test differences - it could be a very positive thing for an illness where there are so few positives.

 

[Jonathan Edwards]

I do find it hard to believe that if Mc/My have found regular ME/control differences it's all down to processing time. Just seems bizarre that that would occur.

Except that in my experience that sort of artefact happens all the time in lab work. It may not be down to one thing. Systematic errors in research labs are everywhere.

 

[richie]

Except that in my experience that sort of artefact happens all the time in lab work. It may not be down to one thing. Systematic errors in research labs are everywhere.

I suppose it's possible but I wonder as to mechanism.
E.g. I was wondering if Mc/My ran a test on controls vs ME and did the ME batch after the controals, thus occasioning ME to produce worse mito function due to delay, or whether they got lots of ME diagnosed samples from clinics, kept some longer than others due to circumstance (workload, postage etc,), found they were worse - due to delay -, then "confirmed" them as ME while fresher samples with better function were labelled as not or less severe ME. There must have been some means for producing this artefact, if that is what it is.
Perhaps this is covered in the critical paper but I couldn't take it all in.

 

[Hoopoe]

The authors of the present paper have some data that suggests cells change over time. It's hard to understand the technical details but the meaning is clear.

Figure 4B is the 1 hour PBMC fraction from a Histopaque™ separation and the scatter plot shows no neutrophil like cells. Figure 4C is the 1 hour neutrophil layer from a Histopaque™ separation and it shows contamination with lymphocytes and other cellular debris. Figure 4D–F depict cells that were isolated from blood that had spent 24 hrs in heparin at room temperature. The whole white cell fractions in (A) < 1 hr isolation and (D) 24 hr isolation show different proportions of neutrophils and lymphocytes in the respective quadrants suggesting that the longer processing time has altered cell properties. This could change the properties of the cells in the samples isolated at 24 hrs so they no longer reflect cellular function at the time of harvesting. This further complicates the interpretation of the data as the assumption is made that both controls and ME/CFS patients cells behave in the same way after incubation in a blood tube for 24 hrs. In addition, in the Histopaque™ gradients the 24 hr cell populations neutrophil fraction (F) looks quite different to the 1 hr samples (C) with a large drop in neutrophil numbers. Overall, Fig. 4 highlights the importance of isolating the cells from fresh blood. If blood has been left for an extended period of time cells are lost with further processing using Histopaque™, with additional changes in cell morphology possibly due to activation.

If the Myhill group used as controls samples that were fresh, while patient samples were shipped by mail and at least 24 hours old, that could plausibly produce consistently different results between sick and healthy patients.

 

[richie]

The authors of the present paper have some data that suggests cells change over time. It's hard to understand the technical details but the meaning is clear.


If the Myhill group used as controls samples that were fresh, while patient samples were shipped by mail and at least 24 hours old, that could plausibly produce consistently different results between sick and healthy patients.

Such questions need to be asked.

 

[Jonathan Edwards]

I suppose it's possible but I wonder as to mechanism.

Are you familiar with supervising research assistants and doctoral students in a lab? Or with colleagues running experiments in a lab?

 

[Hoopoe]

If the Myhill group used as controls samples that were fresh, while patient samples were shipped by mail and at least 24 hours old, that could plausibly produce consistently different results between sick and healthy patients.

I'm reading one of the papers on the test now https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2680051/

They write

The samples from the patient group and the normal (control) group were processed in the same way.

It doesn't specifically say that the delay between blood draw and testing was the same for both groups. If the authors believed that it wasn't relevant, they wouldn't consider there to be any differences in the processing even with different delay in both groups.

Such a hypothetical difference in processing wouldn't be able to explain this nice correlation between illness severity and their mitochondrial energy score.

 

[Andy]

Of course, Myhill/McLaren-Howard should reach out too. But they have been "slapped in the face" hard, and in public, and I would expect them to be very upset. I would, if it was my work.

Well hopefully they will send in a response to the publishing journal so that it is published alongside this paper. Not only is that the way, as I understand it, the process is meant to work, but it also increases the chances of someone else being interested in investigating this further.

ETA: a missing "s" bugged me enough that I had to edit it in.

 

[hinterland]

I'm reading one of the papers on the test now https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2680051/

They write

It doesn't specifically say that the delay between blood draw and testing was the same for both groups. If the authors believed that it wasn't relevant, they wouldn't consider there to be any differences in the processing even with different delay in both groups.

Such a hypothetical difference in processing wouldn't be able to explain this nice correlation between illness severity and their mitochondrial energy score.
View attachment 8064

The trend between CFS Ability scale and mitochondrial energy score is intriguing, and the Ability score was blinded to testing. Perhaps there is something there, and this should be explored further but the test is a long way from standardisation and commercial application.

What would also be good to see is a plot of sample processing time against MES.

Or, a new experiment with control and patient samples repeatedly tested over incremental time points, where the venepucture was done on site, so the first time point is zero.

 

[richie]

Are you familiar with supervising research assistants and doctoral students in a lab? Or with colleagues running experiments in a lab?

McL is a senior scientist and it is reasonable not to expect such errors, though I accept your implication that in real life labs they do occur. Timing problems may be most likely but cannot be simply assumed and I think the points and questions should be put to Dr McL-H, which I hope he would respond to. The critical paper may be right as to results but wrong as to what went wrong at Akumen and that should be elucidated.
I am not nor have I ever been involved in lab work. I express my position without expertise, have a go at guessing what might have gone wrong in line with the critical paper's surmise, ask for questions to be put to Drs My/Mc and hope my inexpert knowledge will then be advanced with everyone else's. If I express no doubt, I get nowhere. I am happy with that and it is the approach of many intelligent sufferers - the little people must fight for themselves and not be shy about showing annoying naivety, if the aim is to learn - because the big uns have let us down too often and disagree among themselves anyway.
A fair criticism of work can be accompanied by an unfair or inaccurate surmise as to what went wrong. There is nothing wrong with criticising the criticism.
Those who are familiar with lab work may have better ideas as to the mechanism of Mc/My's assumed error which I hope will be discussed. What will be said if McL has records showing timing was not the issue?

 

[Jonathan Edwards]

I am not nor have I ever been involved in lab work. I express my position without expertise, have a go at guessing what might have gone wrong in line with the critical paper's surmise, ask for questions to be put to Drs My/Mc and hope my inexpert knowledge will then be advanced with everyone else's.

OK, that is useful to know. I accept your viewpoint from that position.

McL is a senior scientist and it is reasonable not to expect such errors

It might be reasonable but sadly it isn't. Unconscious cherry picking bedevils all work of this sort. As Richard Feynman said, the easiest person to fool is yourself. That is why we need replication.

The graph of mitochondrial score against severity might look convincing because it is so clear cut. However, from my perspective as a lab biologist this graph shouts artefact at me. The scatters don't look right to me. If the scores in the severe cases were really that low then it should be a trivial exercise to identify what is wrong and it should be replicable by any lab. If the scores reflected some general metabolic problem rather than just some quirk of neutrophils (which would be the interest of the study) then we would expect the person to be in intensive care with little chance of survival for 24 hours. It just does not make biological sense to me.

 

[Hoopoe]

The mitochondrial energy score is constructed from several different measured parameters. Maybe that's why it doesn't look like a natural thing, because it's a human construct.

The biochemical measurements in the “ATP profile” separate the energy generation and recycling processes into 5 steps. As in any multistep process, for example electrical power production or an assembly line, the efficiency of the overall process is the product of the efficiencies of the individual steps. Any suggestion of relative weighting is irrelevant; it only results in an overall normalization factor. The product of ATP and ATP Ratio is the cellular concentration of ATP complexed with magnesium and this is the available energy supply of ATP. Ox Phos is the efficiency of the ETC which converts ADP into ATP. However, for the recycling of ADP to make more energy available the Translocator protein must efficiently have its binding site facing out to collect ADP (TL OUT) and alternately facing in (TL IN) to efficiently transmit ATP from the mitochondria into the cytosol where its energy can be used.

We have found it useful to calculate the product of the five factors, the overall mitochondrial energy-producing relative efficiency, and call it the Mitochondrial Energy Score. We just multiply the 5 factors together for each patient and each control. The minimum value for the controls is 0.182 fmol/cell. We have chosen this as our normalisation point so we divide all the Energy Scores (for both patients and controls) by this value. Thus all controls have Mitochondrial Energy Score ≥ 1.00.

Edit: I'm inclined to agree that very low mitochondrial energy scores are suspicious because it implies that in the respective patient, at least one step of ATP generation is extremely impaired. That's how I understand it anyway.

 

[richie]

OK, that is useful to know. I accept your viewpoint from that position.



It might be reasonable but sadly it isn't. Unconscious cherry picking bedevils all work of this sort. As Richard Feynman said, the easiest person to fool is yourself. That is why we need replication.

Agreed

The graph of mitochondrial score against severity might look convincing because it is so clear cut. However, from my perspective as a lab biologist this graph shouts artefact at me. The scatters don't look right to me. If the scores in the severe cases were really that low then it should be a trivial exercise to identify what is wrong and it should be replicable by any lab.

From my position of ignorance your point sounds reasonable (no sarcasm intended btw).

If the scores reflected some general metabolic problem rather than just some quirk of neutrophils (which would be the interest of the study) then we would expect the person to be in intensive care with little chance of survival for 24 hours. It just does not make biological sense to me.

That's the kind of issue which it would be interesting to elucidate with McL/My. I have myself always wondered to what extent any findings on compromised mitochondrial function - if valid- reflect a local or systemic problem. Another problem is that if mito compromise is really found in a systemic illness, which is not classically an illness of the mitochondria, we still don't know how to interpret such findings.

I don't personally know whether My/Mc ever presented their findings as a diagnostic test, but phrases such as "mitochondrial not psychological" were bandied about, which in itself is a contentious statement, assuming not only reflection of test result in symptoms - now at issue- but also implying that mitochondrial is an antonym of psychological - easy meat for Sharpe, Wessely etc - then it may be a quick call to the SMC to show how naive we are about mental illness and hence unable to recognise our own "mental" illness...........

 

[adambeyoncelowe]

The mitochondrial energy score is constructed from several different measured parameters. Maybe that's why it doesn't look like a natural thing, because it's a human construct.



Edit: I'm inclined to agree that very low mitochondrial energy scores are suspicious because it implies that in the respective patient, at least one step of ATP generation is extremely impaired. That's how I understand it anyway.

ETA: This is wrong, so don't waste spoons reading it. (Moved this upfront as a civil service. )

Also, the numbers are multiplied together not averaged, so you'll always end up with a low number and this may exaggerate the effect of delayed testing.

Let's say you score almost full function for everything at 0.8 (out of 1.0). So let's convert that to an MES. 0.8^5 is just under 0.33 (~30% on the Bell Disability Scale), which is severe ME.

If you think of it that way, only slight deviations from the maximum score give you low results. 0.7^5 is just under 0.17. Therefore most patients must actually be scoring somewhere between 0.9 (averages at ~60% on the Bell Scale) and 0.8 (~30%, as above). Few will score lower than 0.75 (~0.24/20% Bell Score).

Could that be an artefact of delayed sampling time or lab error? It's plausible. Patients are already mobility impaired and have other factors potentially interfering with or complicating sampling. So it may take longer to get the draw, or you might have to have blood drawn at home. The more impaired the patient, the more this may be an issue (remember that blog by Anil where it took ages to find a vein for an IV drip?).

That extra time before running the tests on samples might be enough to see a drop from an average of 1.0 in all measures to 0.9 or 0.8. Especially if, as the MEA study shows, the samples seem to have lower scores the longer they're left.

ETA: This could, in theory, be tested rather easily.
ETA2: This is wrong. I misread the y axes.

 

[Hoopoe]

Let's say you score almost full function for everything at 0.8 (out of 1.0). So let's convert that to an MES. 0.8^5 is just under 0.33 (~30% on the Bell Disability Scale), which is severe ME.

If you think of it that way, only slight deviations from the maximum score give you low results. 0.7^5 is just under 0.17. Therefore most patients must actually be scoring somewhere between 0.9 (averages at ~60% on the Bell Scale) and 0.8 (~30%, as above). Few will score lower than 0.75 (~0.24/20% Bell Score).

These are the five factors which as I understand are multiplied and then divided by 0.182 to obtain the mitochondrial energy score (so your calculations would be incorrect). Here are these five factors, together with the CFS ability Scale scores.

 

[Amw66]

Also, the numbers are multiplied together not averaged, so you'll always end up with a low number and this may exaggerate the effect of delayed testing.

Let's say you score almost full function for everything at 0.8 (out of 1.0). So let's convert that to an MES. 0.8^5 is just under 0.33 (~30% on the Bell Disability Scale), which is severe ME.

If you think of it that way, only slight deviations from the maximum score give you low results. 0.7^5 is just under 0.17. Therefore most patients must actually be scoring somewhere between 0.9 (averages at ~60% on the Bell Scale) and 0.8 (~30%, as above). Few will score lower than 0.75 (~0.24/20% Bell Score).

Could that be an artefact of delayed sampling time or lab error? It's plausible. Patients are already mobility impaired and have other factors potentially interfering with or complicating sampling. So it may take longer to get the draw, or you might have to have blood drawn at home. The more impaired the patient, the more this may be an issue (remember that blog by Anil where it took ages to find a vein for an IV drip?).

That extra time before running the tests on samples might be enough to see a drop from an average of 1.0 in all measures to 0.9 or 0.8. Especially if, as the MEA study shows, the samples seem to have lower scores the longer they're left.

ETA: This could, in theory, be tested rather easily.

0.35 and 0.11 scores

 

[adambeyoncelowe]

These are the five factors which as I understand are multiplied and then divided by 0.182 to obtain the mitochondrial energy score (so your calculations would be incorrect). Here are these five factors, together with the CFS ability Scale scores.

View attachment 8074

Ah, my mistake. I missed the division and didn't realise the numbers didn't all run 0-1. Is the first score up to 3? And another looks like it tops out at 0.8? I can't read it too well. The others look like they're still 0-1. So it looks like you can safely ignore me.

But couldn't the multiplication still magnify any difference? There are still a lot of times you're multiplying by less than one.

 

[Amw66]

It is a couple of years since I read the papers, but from what I recall ( correct if wrong), there was also the identification of potential subgroups related to ox phos mechanisms.

My daughter fell into one of these, where ATP was being chucked out of the cell faster than it could be replenished - didn' t have a clue what it could be then , but now it sounds like purinergic signalling.