Estimating Prevalence, Demographics and Costs of ME/CFS Using Large Scale Medical Claims Data and Machine Learning, 2018, Valdez, Proskauer et al

Discussion in 'ME/CFS research' started by Trish, Dec 18, 2018.

  1. ME/CFS Skeptic

    ME/CFS Skeptic Senior Member (Voting Rights)

    Messages:
    4,002
    Location:
    Belgium
    Yes. I basically ignored the results of ME + CFS diagnoses because of the problems with the ICD-CM codes for CFS and I focused on ME-results only, for the reasons you mention.

    As you explained the ME in this paper is not ICC-defined ME but something else. Nonetheless I think it’s a relatively reliable selection, considering the enormous problems with diagnosis in the field of ME/CFS. There is a supplement to the paper that explains this a little further.
    • In the ICD-CM 9 ME appeared in the index only of the code 323.9, Unspecified causes of encephalitis, myelitis, and encephalomyelitis.
    • In the ICD-CM 10 ME falls under G93.3 which is labelled Postviral fatigue syndrome.
    So there is a danger that other diagnoses such as unspecified causes of encephalitis or a passing PVFS have been given the same code as ME, confusing matters for the analysis. But the authors looked at continuous enrollment of 2 to 4 years, so the patients apparently had a chronic condition.

    So I suppose it's reasonable to assume that most of the 14.000 patients who received this label, had what we call ME/CFS. Other prevalence estimates that we currently have, also had seriously flaws such as using the Fukuda criteria. So I think this study provides us useful information.

    I see it as a different way of looking at ME/CFS and coming up with more or less the same estimated prevalence. The direct prevalence of the ME- ICD codes was 0,12%. I was quite surprised how high this number was because this is most certainly an underestimation as few physicians make the diagnosis of ME or use these ICD-codes. Because of the large dataset, the authors were able to estimate the number of ME patients that have not (yet) been diagnosed. Total prevalence then increased to around 0,85%.

    So somewhere between 0,12 and 0,85% That is in agreement with community-based prevalence studies (using Fukuda-criteria) of 0,24-0,42% and with the data from the UK biobank (0,44%).

    The percentage of patients being female (60%) is a bit lower than the community-based prevalence studies found higher rates (83% for Wichita and 71% for Chicago), the UK biobank study (70%), and a 2016 Australian prevalence study (around 75%), but other studies have found even lower rates such as Naculs prevalence study (51%).
     
  2. Webdog

    Webdog Senior Member (Voting Rights)

    Messages:
    2,265
    Location:
    Holodeck #2
    Just recently, Kaiser Permanente decided to start using ME G93.3 and slowly move away from CFS R53.82. Eventually CFS will should be abandoned by Kaiser. This paper supports that decision.
     
    Last edited: Jan 8, 2019
    Hutan, Inara, andypants and 9 others like this.
  3. Webdog

    Webdog Senior Member (Voting Rights)

    Messages:
    2,265
    Location:
    Holodeck #2
    Some cost quotes from the study, suggesting that healthcare providers can actually reduce costs by training doctors and providing more effective care.
     
  4. Medfeb

    Medfeb Senior Member (Voting Rights)

    Messages:
    585
    Agree with both @Michiel Tack and @Webdog's posts. The ME cohort is certainly better than the CFS cohort. And Kaiser's decision is encouraging

    But I'm not so sure that we can conclude that most in the "ME" cohort have what we call ME/CFS especially since the US medical community has not used the term ME. If I remember correctly, I've heard at least one of our disease experts use the term PVFS when the duration was not long enough or when some other criteria were not met. So I expect at least some of these cases could have resolved shortly and not been the kind of chronic illness seen in ME.

    That said, its all we have at this point and we need to exploit whatever insights we can gain.
     
    Hutan, andypants, MEMarge and 4 others like this.
  5. Webdog

    Webdog Senior Member (Voting Rights)

    Messages:
    2,265
    Location:
    Holodeck #2
    Essentially, the approach that Kaiser Permanente is taking is to map IOM/SEID criteria to ME G93.3. Though nothing should prevent a doctor from using ICC if they prefer.

    It seems a reasonable way to proceed at this time. Certainly better than the current practice of mapping everyone to CFS R53.82.
     
    Hutan, andypants, MEMarge and 4 others like this.
  6. ME/CFS Skeptic

    ME/CFS Skeptic Senior Member (Voting Rights)

    Messages:
    4,002
    Location:
    Belgium
    Didn't the fact that they looked at codes with continuous enrollment from 2 to 4 years partially solve that problem of PVFS cases resolving in a short timespan, or am I misinterpreting this?

    You also have to compare it to what we currently had as estimates. You can have Fukuda CFS and recover after a few months. Data from the epidemiological study in Wichita, Kansas indicated that at three year follow-up almost 80% no longer fulfilled the diagnostic criteria. And if I remember correctly, in Dubbo-like studies 10% of infectious cases meet Fukuda-criteria at 6 months, but this decreased to 4% as time went on. So 60% of the original CFS sample no longer had CFS at long-term follow up.

    So what we call ME/CFS (unfortunatedly) is highly variable, uncertain and ill defined.
     
  7. Medfeb

    Medfeb Senior Member (Voting Rights)

    Messages:
    585
    @Michiel Tack - good point about continuous enrollment. That would certainly help that.

    And you are right about the US epi studies - remarkably few kept a CFS diagnosis even two years in a row in that study.
     
    andypants, MEMarge, JaimeS and 3 others like this.
  8. JaimeS

    JaimeS Senior Member (Voting Rights)

    Messages:
    1,248
    Location:
    Stanford, CA
    Whoa, seriously?

    That's out-and-out weird. Any rationale?
     
    andypants and MEMarge like this.
  9. Snow Leopard

    Snow Leopard Senior Member (Voting Rights)

    Messages:
    3,860
    Location:
    Australia
    "roughly 2.8 million" is only a guess because it fails to account for demographics, but 0.8% was the pooled prevalence of "clinically assessed" cases worldwide in this systematic analysis:
    https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3616604/

    I'd expect the USA to have lower prevalence due to lower infectious burden than other countries, but still be in the same ballpark.

    The key problem with all of these estimates is the same as the debate about the various case criteria - there is no strict demarcation, leading to slippage of estimates if researchers or clinicians are sloppy. If all you need is unexplained long term fatigue, then estimates blow out to over 3%.
     
    andypants and ME/CFS Skeptic like this.
  10. Forbin

    Forbin Senior Member (Voting Rights)

    Messages:
    1,581
    Location:
    USA
    I don't think it accounts for everything, but... Leonard Jason's figure of 422/100,000 (.422%) is only intended as an estimate of the adult US population with CFS. If this current study is not confined to adults, you'd expect it to come up with a somewhat larger number - though a doubling of the previous estimate seems unlikely to be due to this factor alone.
     
  11. JaimeS

    JaimeS Senior Member (Voting Rights)

    Messages:
    1,248
    Location:
    Stanford, CA
    No, I'm wondering why Kaiser won't use the ME coding.
     
  12. JaimeS

    JaimeS Senior Member (Voting Rights)

    Messages:
    1,248
    Location:
    Stanford, CA
    But we've never used Oxford in the US. The only reason unexplained fatigue could be categorized as "CFS" is sheer laziness here. Our weakest criteria is Fukuda, which requires at least four of eight symptoms no matter how you slice it.
     
    andypants, MEMarge and ME/CFS Skeptic like this.
  13. Adrian

    Adrian Administrator Staff Member

    Messages:
    6,563
    Location:
    UK
    I don't know much about the US insurance system but could the 20% coverage of people with medicare bias the results they get in that it covers individuals with disabilities which could include people with ME. I think the question is are there more likely to be people with ME in the data set due to this?
     
  14. Medfeb

    Medfeb Senior Member (Voting Rights)

    Messages:
    585
    But most of those 8 symptoms are not unusual or distinctive and there are 163 different combinations of them when you specify any 4 of 8 symptoms. Nacul pointed out that only 20% (if I remember correctly) of those combinations require PEM. So in practice, lots of conditions can be stuffed into the Fukuda bucket as long as they have "medically unexplained chronic fatigue" - the only thing actually required.
     
    Inara, JaimeS, TiredSam and 3 others like this.
  15. Adrian

    Adrian Administrator Staff Member

    Messages:
    6,563
    Location:
    UK
    I'm still reading through the paper. But what I think they have done from the ML perspective is used the ME diagnostic codes from a subset as labels for a supervised training algorithm (a boosted tree). Then they use the model (after testing that it works on a test set (and selecting the model hyper parameters using a verification set) on features from the full data set which is based on other codes associated with patients along with the age, gender etc.

    This gives them estimated for those who have similar patterns. But I'm not sure how good their model is they quote some figures but I would have liked an analysis of some of the errors around any primary diagnosis code on false positives as they could be picking up on other chronic conditions and this (as relatively rare things) being hidden in quite good overall results. This would result in over estimates. But I could be wrong as I've not read everything carefully yet.

    The other thing that struck me was they will pick up those people who are seeing doctors but not those patients who have given up on doctors as they don't do anything. There is no temporal model in their data (I think) so they are unlikely to pickup on such trends where one year a patient sees the doctor then gives up.

    The other thing that struck me was the struggled to model ME+CFS which could just be a feature of their ML modelling but it suggests there is a certain diversity or noise in the wider data set. It could be due to better diagnostics with ME but it could be due to some more knowledgeable doctors using the ME code and others with less experience using the CFS code.
     
    Hutan, Amw66, andypants and 4 others like this.
  16. Adrian

    Adrian Administrator Staff Member

    Messages:
    6,563
    Location:
    UK
    From what I understand these aren't necessarily using any particular criteria but are really down to a doctors judgement and preference for particular codes. But the fact they can model the ME codes but struggled with ME+CFS does suggest something about the consistency and usage of the codes. What would be nice would be to see some sort of cluster analysis of their data to see if there are groups of common patterns of their selected features.
     
    andypants, JaimeS, Andy and 1 other person like this.
  17. Simon M

    Simon M Senior Member (Voting Rights)

    Messages:
    995
    Location:
    UK
    I read this for another purpose and thought I'd post my comments here.


    It makes sense to explore the value of the huge insurance claims database, with thousands of cases of "CFS" and "ME", But both the symptoms/factors selected by machine learning and the demographics look implausible for ME/CFS.

    The code for CFS includes the dustbin diagnosis of “chronic fatigue, unspecified”, which means it isn't much use. And it’s unsurprising that they failed to construct a machine learning model for it (that was the basis for claiming it was “less homogeneous“ than ME).

    ME/CFS diagnosis is not taught in medical schocyol. As the authors comment, those familiar with ME/CFS are probably the ones using the ME label to cover both ME and CFS. It’s likely that other physicians are putting all sorts of crap under the same label.

    The prevalence for ME alone is 0.12%, which is a more credible rate than the headline 0.8% for CFS and ME combined. However, the details cast doubt on the accura of diagnosis:

    The top symptoms/factors selected by machine learning include disorders of the urinary system, vitamin D deficiency, and diagnostic procedures for abdomen. Breathing difficulties are on a par with headaches. Neither post-exertional malaise nor disabling fatigue are collected symptoms. This does not look like ME/CFS. The authors acknowledge that machine learning did not validate the diagnosis.

    It’s not surprising that the demographics look so weird:

    IMG_0009.jpg

    Other data and clinical experience indicates that the gender difference emerges during adolescence and remains at around 3:1, and that peak incidence years are the teens and the 30s. ME/CFS is rarely diagnosed in people over the age of 65 because there are so many other potential causes of the symptoms.

    By contrast, the graph implies* peak incidence from age 60, not in adolescent years and people‘s 30s. And only a modest female bias.
    * The big jumps in prevalence indicate high rates of incidence

    I think that, unfortunately, this is a case of garbage in, garbage out.
     
    Amw66, Inara, ScottTriGuy and 4 others like this.
  18. ME/CFS Skeptic

    ME/CFS Skeptic Senior Member (Voting Rights)

    Messages:
    4,002
    Location:
    Belgium
    Wasn't this a problem of the database and ICD-codes rather than of the diagnosis itself? Could be that doctors used these symptoms to make the diagnosis but that this was not recored in the data.

    Good point. This is weird. Could doctors be using the PVFS code for old people not recovering from a virus?

    Maybe it could be useful to take a sample of the 14.000 patients who got the ME/PVFS code and perform a diagnostic physical examination on them. It could tell us if this kind of approach is usefull or deceiving. This is probably not easy to do because of privacy reasons.
     
    Last edited: Jan 11, 2019
    Andy likes this.
  19. Trish

    Trish Moderator Staff Member

    Messages:
    55,414
    Location:
    UK
    Is it new diagnoses, or all people who have an ME diagnosis? I am in my 60's and still have an ME/CFS diagnosis, even though that diagnosis was first made nearly 30 years ago.
     
    Mij, Andy and NelliePledge like this.
  20. Medfeb

    Medfeb Senior Member (Voting Rights)

    Messages:
    585
    Interesting point. Thanks for posting

    I'd think that up to certain age, the high rate in prevalence in older patients could also reflect the poor prognosis/low rates of recovery - that is, patients at age 60 include those who became ill at 20, 30, 40, etc as well as 60.

    But if that were the explanation, I would have expected the rates to fall off in the 70s and 80s especially if the disease is associated with an earlier morbidity as a few studies suggested. The continued rise into the 70s and 80s is weird.
     
    Snow Leopard, ukxmrv, Webdog and 2 others like this.

Share This Page