Treating patients suffering from myalgic encephalopathy/chronic fatigue syndrome (ME/CFS) with sodium dichloroacetate, Comhaire 2018

I wonder whether you would include such investigations and statements about patients with other chronic physical illnesses. For example, with patients with, say multiple sclerosis, Parkinsons Disease, Rheumatoid Arthritis, do you consider it relevant or appropriate to delve into the patient's past psychosocial history in order to provide 'explanations' of causative factors?

People with ME have suffered for decades from psychiatrists and other doctors and therapists inappropriately attributing our physical symptoms to our past. There is no good evidence that any of this is correct or appropriate. Published papers based on these ideas are so flawed as to be meaningless.

Have you seen the film Unrest? In that a neurologist 'explained' the film's director, Jen Brea's symptoms as Conversion disorder caused by a past trauma she could not remember. As a result, she tested this psychological diagnosis by walking the 2 miles home from the consultation, and collapsed and was unable to get out of bed for months, went from mild to severe ME, and still has not recovered to her previous level years later. This is not an isolated incident, the story is all too familiar. Attributing symptoms of ME to past events can be dangerous. And even if not dangerous, it is inappropriate, inaccurate and unhelpful to patients.

I do not understand this. Why does it have to be a food supplement to be tested? If you are not able to get permission to test it as a drug treatment, surely the most responsible thing to do to help patients would be to cancel your patent on the use of DCA in treating ME/CFS and free it up for other doctors to test its efficacy in a double blind trial.

 

But when generating a formula like this based on the data you need separate training and test sets otherwise the regression will just pick up on random variations within the given test set based - that is what the regression optimization is designed to do. The way that you know if it works is to test the equation generated on new unseen data.

Ideally you need three data sets one for training, one for choosing hyper parameters and one for testing.

 

Its a case of methodology and understanding the algorithms. When training a classifier on data you can give very good results on training data with low error it doesn't mean that the classifier will generalize at all to any unseen data or it that it hasn't picked up on irrelevant features in the data.

Your methodology wasn't clear in your paper but given what I read I believe you are wrong in the assertion that about significance.

For example, there is a quite famous case of a classifier that learned to distinguish cats and dogs but when someone looked at it they discovered that in the training set (and test set) all dogs were on grass and all cats were inside and the generalization performance on other data sets reflected this. So what they had was an on-grass detector and they were mistaken in believing it was a cat and dog recognition system. In the same way its easy to believe that you have a reactor/non-reactor classifier but really its just picking up on some small feature on the training set (without even having a separate test set).

 

I have been experimenting putting different patterns of answers in the FSS questionnaire according to my own symptoms at different stages of my illness, and the sort of pattern I would expect in different circumstances, and calculating the p values according to the two different versions of the so called prediction formulae given by Dr C.
Here are some of my observations:
The first version, which included duration of illness was, as Dr C found out when he tried it on patients with longer duration of illness than the range covered by his first sample, completely useless.
For example, for the same set of FSS scores, a severely affected patient increased their chances of improvement from under 10 % to over 99% over the durations from 5 to 30 years. So of course that formula was a dud.

Using the second formula, and comparing it with the first formula set at 10 years duration, there was still a significant difference in p values between the two formulae. For example, filling it in for myself when moderately affected after 10 years, the first formula gave me a 79% chance of improvement, the second formula gave me a 10% chance of improvement.

There was a general trend for both formulae to give a better chance of improvement for the more mildly affected across the board, so someone who filled in the maximum 7 on every question was given a 12% or 6% chance of recovery, whereas 4's across all questions gave 99% and 82%

Finally I had a go at filling in the questionnaire on the basis that fatigue was a purely physical problem and compared it with a more depression type fatigue with worse scores on motivation, social and role responsibilities.
The difference here was stark.
Physically based fatigue gave an almost 100% chance of improvement.
Depression based fatigue gave a zero chance of improvement.

So in summary - the first formula was useless because it was based on a sample with too narrow a duration range, so failed when patients with higher duration were included.

Both formulae may be reflecting a problem with diagnosis, with patients with depression related fatigue not responding to the treatment, and patients with physically based fatigue, and more likely to actually have ME/CFS responding.

In other words, the magic formula is simply an artefact of an ill diagnosed sample with containing two distinct groups of patients, and the formula is simply separating the parts of the questionnaire that best separate the patients into those two different diagnostic categories.

So my theory is, the drug is 'working' on patients with ME/CFS and failing on patients with depression based fatigue.

We then come to interpretation.

I can see two interpretations:

1. The drug works for ME/CFS

2. The drug doesn't work for ME/CFS, but the placebo effect is operating on the ME/CFS patients differently from the depressed patients. This seems just as plausible, given that it is an open label trial, so the patients know they are getting a drug that seems promising, and those with ME/CFS will be naturally hopeful. They may also, in that hopeful state, and knowing they are taking part in a trial, be extra careful with pacing during the trial, in order not to confound the effects of the drug, crashing less often as a result of pacing, and with a combination of hope and more stable symptoms, fill in the end of trial questionnaire more positively. Compare that with the depressed group, who, being depressed, are not hopeful, and feel just as depressed at the end of the trial, so show no subjective improvement.

You may wonder why I have spent time doing all this. The answer is simple. I was bored and needed something to occupy my mind. And I was curious as to how such an outlandish seeming claim could be made about an apparently nonsensical formula.

I still maintain, as @Adrian has explained, that these are retrospective formulae based on small samples, and only applying to those samples. The value of the second formula as a predictive formula will only be known and should only be claimed to be valid if it has been tested on a completely new cohort of patients not including any of the original sample. And even if it does turn out to be predictive, it may simply be an artefact of poor initial diagnosis, and a tool (FSS) that picks up the effects of that poor diagnosis.

Well that was fun.

Edit to add: It is also perfectly possible that I have filled in the formulae wrongly on my spreadsheet, so don't take all this too seriously.

 

I will probably be a cognitive vegetable tomorrow.

I am one of the 'lucky' ME folk whose cognitive symptoms are at the mild end of the spectrum, whereas my physical symptoms are at the severe end. So I can spend an hour doing 'thinking' stuff so long as I'm lying in bed. I couldn't maintain it all day. That's my thinking done for the day!

And of course, as I said, I may have got something completely wrong - it's all too easy to enter a few numbers wrongly on a spreadsheet.