A proposal for ME Action: a commitment to evidence-based medicine

Perhaps you are thinking of something like this as far as medical evidence is concerned?



Is this what you mean by status?

I’m not sure where LDN and CCI fit in terms of the above hierarchy but perhaps someone more knowledgeable than me will confirm (I’d guess near or at the bottom of the pyramid?).

 

"Status" is an example of double meanings through pwME glasses, worrying that status should not equal eminence - a reasonable concern.

This is why my point is so critical. The meanings of words or concepts are not always the same to 1) a new ME researcher or a third party vs 2) loaded meanings for a pwME community steeped in injustice and rightly paranoid about the past. It allows some of us, and we might all do it sometimes or one day, to forget that scientific principles don't care about our history. History is relevant, but it is social and political, not scientific. Any word can be misinterpreted, but there will also always be some people who just don't care enough about scientific process, regardless of misinterpretation.

So I'll use your word, "crap". It's a very good one to define the spirit tbh and cut though the crap . Every scientific finding has a status and that spectrum could range from:

• Crap vs Not crap
  
• Worth further confirming its not crap
• Just an idea that needs confirmation that it's not crap

That's the principle that is being placed in doubt that is worrying me.

Also, if nearly all ME research is stuck at the "just an idea" end, so be it. We need to know this. As patients. As doctors. As funders. Etc.

 

Exactly! Whatever the version or choice of terms /granularity, it is the spirit of this hierarchy that is being placed into doubt so often. That is why I have become so worried about what has been happening.

I agree that LDN, CCI or most things are not going to be high up that pyramid. I am not sure where banana fits in the list But, that's a second order issue, which I will defer to others and contribute to as and when I have a thought. But not in this thread.

Another observation, this hierarchy is for scientific findings overall, of which treatments are just a part, and would also include observed phenomena, etc. All findings are subject to some version of this hierarchy, whether we like it or not.

 

Does anyone still have any discomfort with this?

Is this issue still in the air?

It goes to the heart of CCI and MEpedia conversations, S4ME credibility and @Michiel Tack 's letter. If we don't all agree, it needs urgent clarification IMO.

 

I'd also ask @Jonathan Edwards or anyone else who should be tagged from the community (please tag!) to ensure that this position is expressing a fundamentally boring and banal point without holes.

If it has holes, they need filling. But the banal shouldn't be as disputable as it has been.

 

The most useful pyramid needs defining. Maybe with less detail for this purpose. But the principle is remains.

 

Put otherwise, which pyramid is the second order question. Too many posts seemed to dispute there was any pyramid at all. That is the entirety of my worry.

 

That pyramid needs to be shared a lot more!

 

I think the pyramid is a dumb idea because it makes people think there is an easy method to determine what is reliable and what is not.

It reduces critical thinking when more critical thinking is needed.

The reliability of evidence depends on many things, not just a classification. There is probably also an arms race phenomenon whereby the people wishing to promote their evidence as reliable are racing against the people trying to suppress false claims. Due to this arms race it will be difficult to ever come up with a simple set of rules that determine whether something should be considered reliable or not.

The pyramid is at least useful to convey the idea that there are varying levels of reliability.

 

Breaking down the pyramid, in terms of ME/CFS treatments:

1. Meta-analyses & systematic reviews: none on ME/CFS drugs.
2. Randomized controlled trials: should probably split further into phase 1, 2 or 3 trials. Some phase 1 trials, nothing close to phase 3 after the rituximab failure.
3-5. Cohort, case-control and cross sectional studies: on drugs used by ME/CFS patients? None that I'm aware of.
6. Animal trials & in vitro studies: some drugs tested on animal model of ME/CFS, likely of no use in humans. In vitro studies? Lots done against pathogens, also likely not of that much use.
7. Case reports, opinion papers, and letters: yeah, those exist.

Based on this, I would agree the pyramid is right now not too useful in terms of assessing ME/CFS treatments since most of the current evidence lands in the bottom two sections and nothing immediate is happening further up in the pyramid. But the point of @InfiniteRubix is important I reckon.

 

Varying levels of reliability exist, agreed. But that is an outcome of research. Variability is a finding, or axiomatic observation at a minimum. But it's not the process.

I totally agree that pyramid might not be the best list of stages to use for a process - I defer to others about which pyramid. But a logical progression/regression of stages for a finding has to be part of our general understanding. Otherwise our contributions are not implementable at scale institutionally or commercially. They will require others to come in, not to sanity test our assumptions, but to actually provide some.

If you are deciding which treatments you want to recommend. If you're deciding which research projects you are going to invest in. If you're deciding which pharmaceuticals are a good bet commercially for R&D. Any of those scenarios require some decision to be made. If the decision isn't made systemically, with a framework of progression, with critical thinking in that process of course, how else is it to be made? It can only be made on a subjective or political basis otherwise. Policy institutions and businesses don't work that way officially, so that means that others would reinjext that discipline for us, in private.

I believe that discipline would be great to come from here, but perhaps I am being too aspirational. But, at a minimum, the idea that a research idea has a journey and can be compared or prioritised shouldn't be revolutionary. It's so banal to me when considering how to make things actionable and ensure things don't go off the rails. It's how things work.....

This is always been my worry, because I'm not just coming at it from a perspective of scientific process and probity. But also from a policy perspective. And also from a commercial perspective.

 

I am not a scientist; I'm a long term (29 years) pwME. I think this discussion is critically important.

When I first joined s4ME in October 2017, I felt confident that I was getting good information. That appears to have changed over the past year. Now it seems that many people are talking as if snake oil will cure us, and science based information has no more value to many commenters than personal anecdote.

One of the reasons I joined s4ME was because other fora seemed not to value science. I hope s4ME will continue to be a place where robust science-based examination of ideas will continue, and where pwME can feel confident in the information they gain on the site.

 

I really don't think it's fair to say that other fora dont value science period. It's also an inflammatory statement. I also don't think its true.
Most people have a poor estimation of how rational and empirical they are, and cognitive bias/error is basically part of being human. So when I see any community or person make claims to be more scientific or rational than other communities I take those claims with a grain of salt.

I would say that s4me has more of a parsimonious focus on peer reviewed literature than other forums. I don't think that necessarily means its more scientific. Observation, speculation, and self experiment are all part of the scientific process. I have been frustrated sometimes about how on some other forums theres less vigorous and skeptical discussion of science than here, but I think all these forums have something to offer and having major splits between the user base in one and the other leads to less productive dialogue. Just my 2c

 

This thread is worrying me profoundly - even more than it was. I am trying very hard to word that 'appropriately'. I'm worried enough to use energy that I don't have for some days now, going round in circles to make the same point in a myriad of ways in the hope that one sticks. I see it as critically important for our futures and for any of our congenital successors. Basic boring scientific 'first principles' are in doubt. This site is unique - and instead of this specific thread encouraging best principles elsewhere, it seems like the opposite is happening reverse. I would rhetorically ask why.

Humans are biased and subjective creatures. I've been trying to show that this is the problem that we are meant to be controlling - and doing it with scientific principles. The reverse seems to be happening... using the problem to undermine scientific principles. The idea that empiricism is an aspiration is not a reason to abandon empiricism.

Some cultures have the very everyday phrase of "mental masturbation". I worry that this thread is distracting from it's point, @Michiel Tack 's important letter, perhaps 'successfully'.

Nobody is telling me that I am misunderstanding what's happening here - I'd love to be wrong.

 

This.

 

If one is to apply a level of rigor in critiquing scientific papers one should apply the same level of rigor to critiquing scientific principles--the result should end up being similar in terms of the rigor and criticality helping to greatly clarify things.

 

Someone said on this board that the constant criticality and negativity that is part of the scientific process may be upsetting to people. I would say maybe the same is true about criticality in philosophy of science.