I think a balance should be struck between skepticism, which is the essence of scientific rigour, and cynicism which can hold back scientific progress.
We should be skeptical of this paper. It is an n=10 uncontrolled, open label trial conducted by a conflicted person so we cannot be certain of the reliability of any of its results. It is partly based on subjective reports of fatigue, which is open to bias - but we can compare the level of bias to other trials.
We should also be skeptical of arguments that these results are due to the placebo effect (and similar effects). I want to point out a few features that suggest these results aren't purely placebo.
Firstly, the effect size is huge. The worst patient went from a subjectively rated physical score of 5 (meaning they answered "limited a lot" for all but one question on physical limitations) to 55 (they answered "limited a little" for all but one question and "not limited at all" to one question) after 2 weeks. After 2 months, that patient further improved to 85 (answered "not at all limited" for 7 questions, "limited a little" for 3 questions and "limited a lot" for 1 question). These questions include bathing yourself, the ability to walk a mile, playing golf and doing vigorous exercise.
So someone who couldn't climb one flight of stairs is now walking a mile and doing mild exercise with no problem, but probably can't do vigorous exercise. This is a dramatic recovery. This is a recovery of 80 points on a 100 point scale.
All the other most severely affected patients also recorded very large jumps in physical score and in vitality score. All but one patient (who recorded the worst physical score) recorded large jumps in social score (and all improved). It's over-egging it to call them cured (as the paper does) but its definitely a miracle level improvement. Indeed, the fact the paper describes them as 'cured' is probably the most misleading thing in the paper. It uses the same trick as PACE, setting a definition of cured that meant that 40% of them had "cured" levels of physical fatigue before the trial started!
Secondly, this effect size dwarfs the effects seen in other biased studies. Consider
this retrospective, uncontrolled and open label "trial" of Abilify.
Among responders to Abilify, the symptom of fatigue fell from 5.76 to 2.86 (on a 10 point scale) on average. Among both responders and non-responders, the symptoms from from 5.78 to 3.50 on average on a top point scale. By contrast, the SGB improved patients from an average physical score of 39 to 79. If we invert the Abilify scale for all patients, this is a 54% improvement on the baseline compared to a 102% improvement from SGB.
In other words, SGB was twice as effective as Abilify (noting that the latter had n=86 compared to n=10 for SGB) and all patients were responders (unlike Abilify where only 77% responded, again noting the small sample size).
Thirdly, objective improvements matched the subjective improvement in physical and social function, and vitality. There were significant improvements in POTS and cognitive impairment that were objectively measured.
For completeness, I do not that some subjectively reported improvement was not matched by objective measurement. Sleep quality and some cognitive impairment improvements were subjectively reported that did not quite match the objective data.
Fourthly, PEM improved (but only disappeared in one patient). If a placebo can get rid of my PEM, then sign me up.
For these reasons, we cannot be certain this is producing a true result. It would have been nice if OMF had ensured the protocol for this trial was more rigorous prior to funding it though. But it seems promising enough to test in a proper RCT if a procedure can be devised. Let's hope OMF actually enforces some rigour next time.
