(Masters thesis) Can Natural Language Processing Reveal Doctors’ Attitudes toward Specific Medical Conditions?, 2020, Brooke Scoles

[Dolphin]

https://www.mortengroup.org.uk/publications-presentations/h-b-scoles-thesis

Can Natural Language Processing Reveal Doctors’ Attitudes toward Specific Medical Conditions?
H. Brooke Scoles



https://www.mortengroup.org.uk/app/download/12285222/Scoles-Masters-Thesis.pdf
Adobe Acrobat document [1.9 MB]

Abstract

In recent decades, an increasing quantity of economics research has been conducted on biased beliefs and their impact on labor markets, education outcomes, housing and other economic activities.

The discrimination that results due to biased beliefs is difficult to observe.

Still, a combination of econometric techniques and inventive experimental design have provided convincing evidence that such bias exists.

Research into biased beliefs and discrimination has mostly focused on gender and race, while very little has been done on disability.

Within existing studies of disability there is almost nothing on "hidden" disabilities or disease, with no papers examining the influence of bias on patients.

Systemic biases in the judgment of patients, if they exist, can potentially influence their care.

Hence, biases are difficult to measure in a lab setting because this environment obscures behavior that is not easily detectable or that is intentionally hidden.

One way to approach such an exploration is to look for evidence in language in anonymous settings.

Natural language approaches, including LASSO-logistic regression and emotion dictionaries, provide the tools for this mode of analysis.

The analysis was focused on a specific disease, ME/CFS, because of characteristics that make it a good candidate for researching attitudes and biased beliefs.

Using this approach, it appears that medical decisions regarding treatment are not entirely objective and that they are influenced by incorrect beliefs.

The language used by medical professionals shows that doctors’ attitudes towards patients are not consistent but vary in line with different diseases.

Such differences have economic implications, potentially lowering the quality of care, worsening health outcomes and lowering labor market productivity or participation.

 

[Hutan]

This is an interesting Masters thesis. It takes an approach that has been used to analyse natural language for gender and racial bias and applies it to analyse discrimination related to disease type. There's a remarkable example from an economics employment forum where words like 'hotter' and 'attractive' were most predictive of female pronouns in posts and 'motivated' and 'philosopher' were most predictive of male pronouns. This thesis takes data from the Medical Reddit.

There are lots of great quotes and Brooke has done well to understand ME politics. They have done slightly less well in understanding medical issues related to ME, accepting findings from preliminary research as material facts, for example:

Lastly, it is implied throughout the threads that the patients are seeking secondary gain in the form of disability benefits or accommodations. In the case of ME/CFS, there is physiological evidence available if doctors run the correct tests. From this sample of comments, it appears that many doctors do not seem to know this. As mentioned above, there are tests for metabolites that show severe ME/CFS patients as much as 16 standard deviations away from healthy controls

Interesting bits and pieces:

Thus, it is possible that the language used by the medical community represents views that have real impact on patient care at the clinical level as well as research priorities at the policy level.

Through analyzing text data from anonymous discussions by medical providers, I found evidence that doctors appear to be more judgmental when discussing ME/CFS relative to other diseases.

Such a conflict would apply particularly to insurance companies whose policies have not included coverage for mental illness. An Unum internal report on ME/CFS stated that ME/CFS has been caused by "failure of coping mechanisms" combined with "entitlement philosophy" and an inability to keep up in an increasingly demanding economy. The report authors speculated that as "the American Dream is well out of reach for many," stress was increasing instances of ME/CFS which was made up of, "highly educated professionals slipping into self-imposed oblivion! [emphasis in original]" The firm noted that their "dollar exposure [to ME/CFS claims] is significant" and that they would “lose millions if we do not move quickly to address this increasing problem”. The firm consequently resolved to present the disease as “neurosis with a new banner" and push treatments that would "increase [patients’] motivation to return to work" (Jackson, 1995). If this attempt at rebranding has been successful, it should be visible in the data within the terms medical professionals use to speak about the disease. When analyzing these attitudes, it is thus useful to keep in mind the misalignment of incentives between insurance companies and social welfare that may be contributing to incorrect bias.

Language that implies opacity, such as "complex" or "vague," does not imply outright negative stereotypes the same way as words like "lazy" or "malinger," but it can still have an impact on investment [in research].

Section 2.2. has illustrations of posts. There's really nasty stuff there.

There's a number of analyses - all coming to the conclusion that ME/CFS is a stigmatised disease. Here are the words that differentiated ME/CFS from other selected diseases:


What is clear from that is that current issues (in this case the Afflicted TV program) impacted on what specific words were predictive. It was commented that diseases like MS had a lot less stigma words that were highly predictive and a lot more specifically medical words like drugs. There are probably a few factors at work there, not just less stigma. ME/CFS has few drugs that would be mentioned frequently in a discussion about it.

So yeah. Interesting. I hope we hear more from Brooke about ME/CFS. It would be good to see more of this kind of analysis of language used by medical practitioners about different diseases.

 

[Adam pwme]

Very interesting. I wondered if this approach could be used to analyse online harrasment of researchers. But I expect there would be ethical issues.

 

[Andy]

Worth noting that Brooke is part of Karl Morten's team at Oxford - thread, https://www.s4me.info/threads/karl-...ign-understanding-the-causes-of-me-cfs.13991/

ETA: Added Tweet below.

 

[Hoopoe]

According to this analysis, depression is far less stigmatized than ME/CFS. Funny. Who said that ME/CFS patients just want to avoid the stigma of mental illness?

I'm not sure I agree with all interpretations. That treatments are being discussed far more in other diseases may just reflect the fact that there aren't any good guidelines on treatments due to lack of research showing us what treatments can help.

I like the connection that the author makes here:

The results from the previous section reveal that doctors’ attitudes appear to be different for different diseases. Negative attitudes resulting in the delay or denial of treatment, or delaying investigation into symptoms, can prevent improvements in these patients’ health. Patients whose disease could be kept under control can experience more severe symptoms and more severe limitations to their activities. In addition to delaying treatment, biased attitudes may lead to inefficiencies by discouraging investment in biomedical research. Stereotyping a particular condition as mysterious could also lead to a culture of hopelessness regarding the condition in question, so deterring researchers from entering the field. It could also lead to an underestimation of the marginal benefit of funding the disease if policymakers or health researchers mistakenly believe that nothing can be done about the condition. This outcome has welfare implications for the quality of life and mortality of a population, as well as economic implications for labor market participation and GDP. Diseases like ME/CFS cost the US and UK economies billions of dollars in medical bills, care-taking costs and lost labor market productivity each year.

 

[mariovitali]

I am really happy to see that NLP applications are getting more attention. Remember, NLP does not mean always Neuro-Linguistic Programming ;-)


The same techniques can be applied to patient notes on thoughts, symptoms, positivity and negativity of sentiment using speech-to-text recognition (i've been giving similar ideas to one ME organisation since 2018 but no luck) and then using NLP to extract concepts of symptoms, sentiment etc for a given day.


Some techniques used are indeed -in my opinion- chosen correctly (e.g LASSO to select features) but it would be interesting to see bigrams (two consecutive words) , trigrams (3 consecutive words) as well in the input data. The author says :

.....while the words with the greatest predictive power for comments about male economists included "philosopher," "keen," "textbook," and "motivated" (Wu, 2017). One criticism of this technique is that breaking text down into individual words loses context. For example, if a word or phrase has the word "not" in front of it, the text actually has the opposite meaning from that implied by the word on its own

Although not related with analysing PUBMED abstracts , this is very interesting work indeed.

 

[Hoopoe]

The psychosomatic researchers that publish research that supports these negative attitudes towards patients should ask themselves what role they are playing in all this and what harm they have caused to millions of people over the course of the last 30 years.

In my opinion the language used by the Medicine group on reddit has somewhat improved in recent years. There are comments that express the possibility that this might just be a real disease.

 

[Dolphin]

 

[Mike Dean]

Brooke has a seriously disabling condition, which makes this study even more remarkable

 

[wigglethemouse]

What is clear from that is that current issues (in this case the Afflicted TV program) impacted on what specific words were predictive.

Jamison with a score of 3.62 seems to indicate that the Netflix Afflicted documentary may have had a big effect.

 

[rvallee]

Jamison with a score of 3.62 seems to indicate that the Netflix Afflicted documentary may have had a big effect.

Yeah I remember that thread. /r/medicine doesn't care about ME but doesn't miss an occasion to do a two minutes of hate on us if it's neatly packaged to be exactly that. Afflicted should eventually be considered hate speech, it was made for that.

Irony is that they were conned by a scam deliberately framed to be misleading. Doesn't exactly inspire confidence in what medical training teaches about critical thinking and seeing through obvious BS.