1. The latest crowdfunder to support David Tuller's work has opened. To donate click here.
    Dismiss Notice
  2. Guest, the 'News in Brief' for the week beginning 18th October 2021 is here.
    Dismiss Notice
  3. Welcome! To read the Core Purpose and Values of our forum, click here.
    Dismiss Notice

A charter to improve ME/CFS research

Discussion in 'BioMedical ME/CFS News' started by Michiel Tack, Oct 10, 2021.

  1. Michiel Tack

    Michiel Tack Senior Member (Voting Rights)

    Messages:
    2,668
    Location:
    Belgium
    I recently followed an online webinar about patient involvement in the Dutch ME/CFS program. It looks like the idea is to involve patients in as many aspects of the research project as possible, which is a great initiative. If I recall correctly, ME/CFS patients were already involved in ensuring quality of research in Norway. Patients can also have an influence on research through charities that provide research funding.

    With this in mind, I thought it might be useful to write down a few general principles that might help to improve the quality of ME/CFS research without overburdening scientists too much. Here’s what I got thus far, hope others can complement.

    1) Pre-registration: publish a protocol
    Every scientific study involving ME/CFS participants should be well-thought in advance and publish a protocol that specifies hypotheses, experimental design, power calculation, outcome measures, etc. This prevents p-hacking and cherry-picking of results and makes a research project more transparent towards patients and other researchers. Without a pre-specific protocol, it is impossible to know how many analyses were conducted or whether highlighted findings were selected post-hoc. The Open Science Framework (OSF) makes it easy to pre-register a study protocol.

    2) Involve ME/CFS patients in the design, and implementation of your study
    ME/CFS researchers should involve patients in the design and implementation of their research. Patients can help point out practical difficulties in trial design such as problems with transportation or ambiguity in questionnaires. In addition, patients have a lived experience that offers a unique perspective on ME/CFS. In the past, research has resulted in dismissive, patronizing, and simplistic theories about ME/CFS that likely would not have taken this form if patients were actively involved in research. Lastly, engaging patients in research also helps to make sure that results are well-disseminated in the ME/CFS patient community.

    3) Use recent ME/CFS diagnostic criteria correctly
    It is recommended that research uses recent diagnostic criteria for ME/CFS such as the Canadian Consensus Criteria (CCC), the International Consensus Criteria (ICC), or the National Academy of Medicine case definition. When older definitions such as the 1994 Fukuda criteria are used for comparability, it is advised to measure symptoms and characteristics (such as orthostatic intolerance) that allow the percentage of patients meeting recent criteria, to be calculated.

    When some aspects of ME/CFS case definitions are not fulfilled, it is recommended to clarify this in the publication title and description. For example, studies that did not include a clinical examination of patients are usually referred to as studying ME/CFS-like symptoms rather than ME/CFS.

    4) Use the Common Data Elements for ME/CFS
    It is advised that researchers use the Common Data Elements for ME/CFS research. These recommendations provide an overview of the most adequate tools and questionnaires to measure symptoms and quality of life in patients with ME/CFS. They help to make sure that research is harmonized and comparable.

    5) Follow relevant reporting guidelines
    ME/CFS research should be reported clearly and transparently. ME/CFS publications should follow relevant reporting guidelines to make sure studies can be replicated by other researchers and their findings are easily understood by doctors and patients. The Equator network provides a useful overview of reporting guidelines for many different study types.

    6) Focus on effect size estimates and their uncertainty rather than statistical significance
    The findings of ME/CFS research are ideally reported in terms of effect size and their uncertainty, rather than statistical significance. The latter is often an arbitrary threshold and findings that are statistically significant can be so small that they are clinically irrelevant. A focus on effect sizes and their precision also help the interpretation of replication studies.

    7) Make your findings open-access
    The findings of ME/CFS research should be available to patients, doctors, policymakers, and other researchers. If results are not published in an open-access journal or on a pre-print server, it is recommended that authors publish their submitted version on a personal website or academic server such as ResearchGate. Most scientific publishers allow the Author’s Original Manuscript (AOM), the version of a paper before submission and peer review, to be freely distributed.

    8) Publish your data, not just the results
    Researchers should make the raw data and statistical analyses publicly available in a way that ensures the anonymity and privacy of study participants. Publishing full datasets enables others to check publications for errors or conduct additional analyses. It also ensures a lasting impact of a study because the individual patient data can be used in future meta-analyses. Scientific studies are often expensive and require big efforts from ill patients. Withholding valuable information from these studies blocks scientific progress and should be viewed as lamentable, regardless of current norms on sharing data in the scientific community.
     
    Michelle, lycaena, MEMarge and 33 others like this.
  2. Michiel Tack

    Michiel Tack Senior Member (Voting Rights)

    Messages:
    2,668
    Location:
    Belgium
    Another suggestion:

    Use adequate controls
    It is recommended to not only use healthy controls but patients with other chronic illnesses. Otherwise, it is difficult to determine if reported differences are due to being ill or due to having ME/CFS.
     
    Michelle, lycaena, MEMarge and 21 others like this.
  3. Michiel Tack

    Michiel Tack Senior Member (Voting Rights)

    Messages:
    2,668
    Location:
    Belgium
    Some may think that I've missed the main issues in ME/CFS research but I've deliberately focused as little as possible on specifics because these depend on the particular study in question and it is ultimately the researchers who decide how to tackle a scientific problem.

    Most of the suggestions listed above are about transparency and being up to date with modern practices. They help to make the most out of research funding without taking much time and effort of researchers. They help to prevent research waste and are therefore in the interest of the ME/CFS patient community in my view.

    Most of the issues are about old norms in the research community that are not up to date with the age of the internet. They date back to a time where the space for printing findings was restricted and statistical calculation power was limited. I hope that patient advocates and organizations can help to raise these standards and make sure that ME/CFS research is at the forefront of this evolution.
     
    Michelle, cfsandmore, Simone and 10 others like this.
  4. Peter Trewhitt

    Peter Trewhitt Senior Member (Voting Rights)

    Messages:
    2,230
    This might be too specific for your intentions, but I would want to reiterate ‘wherever possible use objective outcome measures’, ‘in unblinded trials never rely solely on subjective outcomes’.
     
    Michelle, lycaena, MEMarge and 12 others like this.
  5. Trish

    Trish Moderator Staff Member

    Messages:
    36,342
    Location:
    UK
    Is there a way of stating that both researchers and patient groups involved in ME research should not be ones that specifically promote a particular treatment. (I'm thinking of LP, where research is supported by groups that claim success with LP).

    Also I think something on outcome measures must include objective measures that are meaningful to patients, and all outcome measures should relate to improvements in core symptoms and/or long term ability to function.
     
    Michelle, MEMarge, Barry and 16 others like this.
  6. Medfeb

    Medfeb Senior Member (Voting Rights)

    Messages:
    393
    Good list. Thank you for doing this, @Michiel Tack

    Some food for thought...

    Consider adding that patients should be involved in the strategy/priority setting before specific projects are selected. That will help drive the overall research agenda to focus on what matters the most to patients. It's too late to drive that once projects start.

    On the criteria, IMO, Fukuda should no longer be used at all because it's so non-specific that its hard to trust the research. And I'm skeptical that studies will report enough of the data to be able to make sense of it. If Fukuda is used, it should only be when cohorts are required to meet both Fukuda and one of more of CCC, IOM, or ME-ICC.
    We are seeing Long COVID researchers using Fukuda to identify "ME/CFS" cases in their research. This could generate a great deal of uninterpretable garbage.

    Maybe think about adding the need to agree to methods for how common data element instruments are applied. The CDE initiatives produced some useful guidance on instruments. But it was not scoped to say what criteria should be used (or even that PEM should be mandatory) or to define how the selected instruments should be applied. For example, what dimensions of SF-36 have to be impaired to satisfy the functional impairment criteria in the various definitions? The CDEs do not say. But one mid-2000s CDC study required impairment on any one dimension, including just emotional function. So patients could satisfy the definition's impairment criteria even without any physical impairment.

    The other thought is to recommend the use of study design methods that include severely ill patients and other populations often marginalized in research and health care.
     
    Michelle, MEMarge, Simone and 13 others like this.
  7. Andy

    Andy Committee Member (& Outreach when energy allows)

    Messages:
    14,345
    Location:
    Hampshire, UK
    Like other points made, this may be too specific, but it might be worth including a note that any patients involved in any way will not be able to work sustainably at the pace of researchers. This was an adjustment that was made to the internal processes of DecodeME in order to facilitate the on going participation of our patient reps.
     
  8. Ravn

    Ravn Senior Member (Voting Rights)

    Messages:
    1,440
    In principle I'm strongly in favour of involving patients in all stages of research. There's just one curly question: which patients?

    Trish touched on the risk of groups with major COI infiltrating the process. The other risk is well-meaning but naive patients being used for whatever the equivalent term for green-washing is. And there seem to be a lot of patients who are so grateful to anyone doing any vaguely biomedical-looking research into ME they seem to lose their critical thinking ability.

    To really make a positive difference you need patients with at least a basic understanding of the scientific method* and who are prepared to be critical grumps (of the cooperative sort). Don't know how to phrase that more neutrally.

    *Can't remember the details but a while ago I read somewhere about increasing patient involvement in research (not ME related) and there was a suggestion of patients being given a short intro course into scientific research methods. That could be a way of selecting patients with at least some commitment to engaging properly with scientific questions. Still wouldn't screen out the COI types though.
     
  9. Hutan

    Hutan Moderator Staff Member

    Messages:
    16,582
    Location:
    New Zealand
    Thanks @Michiel Tack, this is very helpful. I'd like to see us draft something on the forum. I'd ideally like to see ME/CFS advocacy groups signing up to some international agreement, where they undertake not to support research and researchers not meeting the guidelines.

    Some more points off the top of my head:

    Trial design should take into account the following characteristics of ME/CFS:
    • high rates of recovery and improvement during the first two years
    • substantial fluctuations in the severity of symptoms during the day, weekly and longer term, making it difficult for patients to accurately assess minor subjective improvements
    • exertion greatly affects symptom levels
    • many people with ME/CFS wake later than healthy controls and have low activity levels

    Taking these characteristics into account in trials may require:
    • careful use of controls (people with ME/CFS; people with chronic health conditions placing similar restrictions on their lives)
    • inclusion of objective outcomes in unblinded treatment trials
    • assessment periods that involve months rather than weeks
    • assessment of symptoms using repeated real time data collection
    • standardisation of exertion levels for several days prior to, and during, a measurement event such as a repeated CPET
     
    Michelle, lycaena, MEMarge and 18 others like this.
  10. Kitty

    Kitty Senior Member (Voting Rights)

    Messages:
    2,262
    Location:
    UK
    Some really good ideas here.

    I realise it's not always feasible, especially in early exploratory studies, but we need to work towards better-sized cohorts too. With such a heterogeneous condition, can you read anything much from a study of fewer than 20 patients? (Especially when it's a mixed-sex group too, as there may be metabolic differences that we don't yet understand.)
     
    Michelle, cfsandmore, EzzieD and 10 others like this.
  11. Milo

    Milo Senior Member (Voting Rights)

    Messages:
    1,550
    Do you mean conflict of interest and bias? I think research ethics boards usually take care of this kind of bias, like the recent Norwegian LP trial that has failed ethics review.
     
  12. Milo

    Milo Senior Member (Voting Rights)

    Messages:
    1,550
    Agreed. However tissue and body fluids that have been in freezers for a few decades may only come with a Fukuda criteria- dr Peterson’s patients come to mind, which lead to using this definition when that cohort was used for the Chronic Fatigue Initiative
     
    Michelle, MEMarge, Louie41 and 6 others like this.
  13. Milo

    Milo Senior Member (Voting Rights)

    Messages:
    1,550
    Definitely would be nice, however you need to understand that new researchers entering the field do need a ‘proof of concept’ or pilot study, usually with less than 20 patients to get funded for a larger study. The funding is a compounding problem, seed grants are usually barebones.
     
    MEMarge, Louie41, Robert 1973 and 3 others like this.
  14. Obermann

    Obermann Senior Member (Voting Rights)

    Messages:
    109
    Location:
    Stockholm
    Thank you, Michiel, for bringing the subject up and thank you for providing the list. I would like to rephrase point 1. All confirmative studies of ME/CFS should be pre-registered, but there must also be room for explorative studies and confirmative studies that are complemented with explorative observations. It must be clear from both the abstract and the conclusions section in a study which observations/conclusions are confirmative and which ones are explorative in nature.
     
    Michelle, MEMarge, Louie41 and 9 others like this.
  15. Ravn

    Ravn Senior Member (Voting Rights)

    Messages:
    1,440
    That sounds like a really good project. Patient involvement is starting to become a bit of a thing in medical research so it would be good to get in early on and steer the process in a productive direction so it doesn't become just a box ticking exercise.
     
    Michelle, Louie41, Trish and 8 others like this.
  16. strategist

    strategist Senior Member (Voting Rights)

    Messages:
    3,890
    Most of these items seem to be in response to poor clinical trials of CBT/GET studies.

    We need more than just stopping poor CBT/GET studies to move the science forward.

    I think we need more emphasis on replication, adequate sample size, adequate controls and on designing studies that can tell us about possible subtypes of ME/CFS. The assumption that it's just one illness could be wrong.

    Low funding seems to have a lot do with these problems so hopefully things will improve.
     
    Michelle, Ravn, MEMarge and 7 others like this.
  17. Suffolkres

    Suffolkres Senior Member (Voting Rights)

    Messages:
    1,018
    Have in place agreed Key Performance Indicators (KPIs) and Measure against these during annual required Quality Control Audits of Service Performance which should be made public and published.
     
    Ravn, Louie41, Sean and 1 other person like this.
  18. Michiel Tack

    Michiel Tack Senior Member (Voting Rights)

    Messages:
    2,668
    Location:
    Belgium
    Was not the intention. In that case, I would have focused more on things like objective outcomes and bias due to lack of blinding. It's mostly inspired by developments in the field of psychology following the "replication crisis".

    In some way, one could argue that clinical trials are one step ahead of many other medical studies because the idea to pre-register, perform power-calculations are more common.
     
    Michelle, Ravn, MEMarge and 8 others like this.
  19. Hutan

    Hutan Moderator Staff Member

    Messages:
    16,582
    Location:
    New Zealand
    I don't think they are. I had in mind exercise physiology, cytokine and cortisol studies and trials of supplements and medical devices when I thought of the items I suggested.

    Lack of blinding and lack of objective outcomes are issues with a whole range of studies.
     
    Michelle, Ravn, MEMarge and 6 others like this.
  20. Trish

    Trish Moderator Staff Member

    Messages:
    36,342
    Location:
    UK
    True, but the Crawley LP trial happened, and I assume had her tame fan club as her patient/carer group approving, and an ethics committee approving it. There needs to be a stronger system that disallows trials that include persuasion to ignore physical symptoms.
     
    Michelle, lycaena, EzzieD and 10 others like this.

Share This Page