Questionnaires - design, validation and use in ME/CFS research - discussion thread

Discussion in 'Subjective outcome measures (questionnaires)' started by Trish, Jan 30, 2022.

  1. Ravn

    Ravn Senior Member (Voting Rights)

    Messages:
    2,191
    Location:
    Aotearoa New Zealand
    @rvallee posted this in another thread but this BMJ opinion piece is also relevant here.

    Using condition specific patient reported outcome measures for long covid
    https://www.bmj.com/content/376/bmj.o257
     
    Ash, alktipping, Hutan and 9 others like this.
  2. Ravn

    Ravn Senior Member (Voting Rights)

    Messages:
    2,191
    Location:
    Aotearoa New Zealand
    This blog was linked elsewhere but the quoted bit is relevant here. It's unclear to me if the part I bolded below means they're only going to analyse existing questionnaires or develop new ones. Either way, are any of the Canadians here involved in this? If not, can they muscle in?
    https://www.eleanorsteinmd.ca/blog/ICanCME
     
    alktipping, Hutan, Snowdrop and 6 others like this.
  3. MSEsperanza

    MSEsperanza Senior Member (Voting Rights)

    Messages:
    2,981
    Location:
    betwixt and between
    Apologies, not able to read through the thread, but stumbled across this study on fatigue in Lupus. Not sure if it aimed to be a validation study, but they used actimetry/ accelerometers to examine the correlation between subjective and objective measures and also tested the internal consistency of the subjective measures (patient reported outcomes).

    Fatigue, patient reported outcomes, and objective measurement of physical activity in systemic lupus erythematosus. Lupus. 2016 Oct ;25(11):1190-9. https://pubmed.ncbi.nlm.nih.gov/26869353/

    Forum thread here.

    Haven't read beyond the the abstract so don't know how the idea that physical activation could decrease fatigue impacts the authors' overall reasoning.



    I think both the conclusion and how they present the results doesn't suggest the study showed more than a strong correlation between fatigue and physical activity?

    That means the subjective measures were consistent with the objective measures.

    Don't know however if this is what they wanted to find out.

    What interests me in this regard is:

    Is the risk of bias in a validation study comparable to the risk of bias in a clinical trial?

    If a validation study shows a strong consistency between subjective and objective measures, does that mean there would be a similar consistency when using the same measures in a clinical trial?

    Edit:
    I don't think there is a comparable risk of bias in both sorts of studies so don't think that in an unblinded trial even the best validated subjective measure could make an additional objective measure redundant.
     
    Last edited: Feb 20, 2022
    Ravn, alktipping, Michelle and 2 others like this.
  4. Snowdrop

    Snowdrop Senior Member (Voting Rights)

    Messages:
    2,134
    Location:
    Canada
    I've just found this thread.

    If there is anyone in Toronto who might know about this it would be @ScottTriGuy.

    But His name is not coming up as available so I don't know if he reads anything here.
     
  5. Trish

    Trish Moderator Staff Member

    Messages:
    55,951
    Location:
    UK
    This looks relevant to this thread.
    https://twitter.com/user/status/1502152885160660994

    The heading is:
    Assessment of body-focussed Beck Anxiety and Beck Depression Inventory items in chronic illness: A preliminary analysis of emotional versus physical causes. by Jill Albertson et al.
     
    Last edited: Mar 12, 2022
    Lou B Lou, Ravn, alktipping and 9 others like this.
  6. Trish

    Trish Moderator Staff Member

    Messages:
    55,951
    Location:
    UK
    Lou B Lou, Ravn, alktipping and 4 others like this.
  7. JemPD

    JemPD Senior Member (Voting Rights)

    Messages:
    4,507
    I wonder if there is a link to something other than an unreadable photograph? That looks very useful.

    Also, whats with the cucumber hanging over that woman's head? :D
     
    Lou B Lou, Ravn, alktipping and 2 others like this.
  8. SNT Gatchaman

    SNT Gatchaman Senior Member (Voting Rights)

    Messages:
    5,998
    Location:
    Aotearoa New Zealand
    I've just completed round 2 of a Delphi Consensus for long COVID evaluation. See https://www.pc-cos.org. I only just came across this on Twitter last night and hadn't seen it discussed previously. I rushed and just managed to register in as a "health professional with Long Covid" before close today and the process took a couple of hours.

    Many of the questionnaires discussed here were being assessed. I hope to have rated appropriately as discussed above and elsewhere. Apart from the ridiculous overload of anxiety/depression/psych instruments, they only offered Fukuda for ME type evaluation. I zero-rated and suggested ICC.

    Cognitive instruments also focused on dementia/AD or depression. Physical symptoms seemed overly focused on "lung damage" rather than eg dysautonomia/metabolic failure. I made some suggestions there too, but mucked up at the end when I intended to add the NASA lean test. There was no ability to return and review from the penultimate page, grrr.
     
  9. Trish

    Trish Moderator Staff Member

    Messages:
    55,951
    Location:
    UK
    Intersting, thanks, @SNT Gatchaman. Is it possible for you to write to the researchers explaining your concerns?
     
    Lou B Lou, Ravn, Missense and 5 others like this.
  10. SNT Gatchaman

    SNT Gatchaman Senior Member (Voting Rights)

    Messages:
    5,998
    Location:
    Aotearoa New Zealand
    It's a shame I didn't catch this for the initial intake — and others here could have been involved too, or perhaps S4ME as an organisation. Suggests it can't have been placed on the radars of the large ME groups?

    Highly surprising (and also not) that Fukuda was the only one on offer. I presume CCC/ICC weren't offered up by those initially involved. If that is so, I am doubly glad I came across it. There will be two more surveys that I have to complete.

    I'll see and report back how things progress, but yes it might require further direct correspondence.
     
  11. Snow Leopard

    Snow Leopard Senior Member (Voting Rights)

    Messages:
    3,933
    Location:
    Australia
    Are they including patient experts in this round, or is this a case of the blind leading the blind? (so to speak, no offence to blind people) There are no "long-COVID experts" for a Delphi Consensus process to be valid...
     
    Last edited: Jun 30, 2022
  12. SNT Gatchaman

    SNT Gatchaman Senior Member (Voting Rights)

    Messages:
    5,998
    Location:
    Aotearoa New Zealand
    As I recall, the options I could choose definitely included "patient with Long Covid" or words to that effect; as well as healthcare professionals who are or are not involved in caring for pwLC, or who themselves have LC; and researchers into LC. I think there were other categories also (eg perhaps patient support group rep). I had to hustle to make the deadline, so I didn't spend too much time looking at these preliminaries.

    Hopefully, I'll get a sense of the contributors from the responses when they come through, but I guess there will be varying degrees of relative expertise (and the inevitable self-assumed expertise). As you point out, no-one can be an expert in a condition that has yet to be explained. Let's hope the process can make something reasonable, although I expect it will inevitably be well short of ideal.
     
  13. SNT Gatchaman

    SNT Gatchaman Senior Member (Voting Rights)

    Messages:
    5,998
    Location:
    Aotearoa New Zealand
    Ravn, Trish, Hutan and 1 other person like this.
  14. SNT Gatchaman

    SNT Gatchaman Senior Member (Voting Rights)

    Messages:
    5,998
    Location:
    Aotearoa New Zealand
    I've just completed round 2 of the Delphi process. This is the part where you can change your responses depending on your review of the spread of others' responses. They divided into: patients with LC, medics/researchers with LC, and medics/researchers without LC. Groups 2 and 3 seemed broadly similar as I was going through.

    Perhaps unsurprisingly, I was at significant variance with the majority of my responses. Those I was happy with tended to also be favoured by others, but where I disliked the instrument, most favoured. Eg various* fatigue scales were included, which I had independently rated at 2, or occasionally 1. Many respondees were clustered at 7-9. Now that I review commentary here and in other threads, I think I probably managed to represent our thoughts, but suspect it'll end up as a minority opinion.

    * You know.
     
    Ravn, rvallee, alktipping and 9 others like this.
  15. SNT Gatchaman

    SNT Gatchaman Senior Member (Voting Rights)

    Messages:
    5,998
    Location:
    Aotearoa New Zealand
    Just to expand on that. The medics/researchers with or without LC looked to have a similar mode +/- mean, but the deviation was more in those with LC. As I was going through I wondered if this meant that only some of those with lived experience were overcoming their professional biases and seeing the deficiencies of the existing instruments (and paradigm). I may be trying to read too much into it.
     
  16. Hutan

    Hutan Moderator Staff Member

    Messages:
    29,851
    Location:
    Aotearoa New Zealand
    I was doing a bit of work on something else, and came across this paper:
    I won't make a separate thread for it because it's pain, not ME/CFS related, but it's a terrific example of how the problems of poorly conceived measures are then compounded by data analysis techniques that take the data out of context. And then there is no one with common sense and decency in the report drafting, or editing or peer review stages to stop really appalling conclusions causing harm.

    Ethnic- and sex-related differences in pain characteristics, psychological distress and pain-related disability in patients attending a New Zealand teaching hospital pain service, 2018, Burri et al )

    This study took the data of 798 people who attended a hospital pain service over a few years. Each person was required to fill out a battery of surveys to assess the usual, depression, anxiety and so on, and also 'pain catastophising'. The results of these surveys were used to help work out what sort of help the person needed (which I think was probably mostly thought to be helping them to learn to live with the pain - but that's another story). As part of an initiative to demonstrate the utility of analysis of patient records, some researchers came along and analysed these surveys.

    One of the conclusions was “Pacific Islanders had the highest tendencies to catastrophise about their pain followed by Maori and Asians”. Which is just staggering. As background, many Pacific Islanders in Auckland are newly arrived from the islands. These immigrants don't have much capital and they end up working in the low paid physical jobs - cleaning, bus-driving, labouring. There maybe some English language issues and certainly a hesitancy about interacting with institutions. People pay to see the GP, so, if money is tight, health care gets put off.

    So, Strike 1 is that the scale is called the Pain Catastrophising Scale, when what it is actually is a Pain-related worry scale. Put in those terms, it's probably useful for a clinician to get an idea of how much concern pain is causing someone. For example, a cleaner who may not be able to keep doing their job, and who then probably could not keep covering the costs of rent and food for their family is likely to be worrying a lot more than someone who is finding it difficult to keep playing tennis on the weekend. I guess if someone seems to be worrying a lot about their pain, out of proportion to the difficulties they face, maybe reassurance that what they have isn't cancer, just the wear and tear of age, something like that, could be part of the clinical response. But the survey doesn't say anything about the difficulties someone is facing, so it can't tell you how reasonable the worry is.

    Strike 2 is the data analysis, that takes that data and uses it out of context, reporting the results still with that 'catastrophising' label. It's worse than that though. The paper says that Pacific Islanders constituted 10% of the hospital catchment population, but only 2.6% of the people attending the pain clinic were Pacific Islanders. To be clear, that is 21 people out of the sample of 798 people that they are basing this abstract-worthy conclusion on. It's extremely likely that the average pain level of the Pacific Islanders making it to the pain service was higher than most, even if it isn't showing up in the results. And 21 people is not a big enough sample to make conclusions about a whole ethnicity (in fact, 'Pacific Islanders' covers multiple ethnicities). There were only 30 Māori people and 44 Asian people in the sample also.

    I don't think the p-values in the chart showing the results make sense. It's not at all clear to me what comparisons the p-values actually relate to.

    Screen Shot 2023-01-23 at 11.19.09 am.png


    Strike 3 is that no one at any point in getting the paper into the New Zealand Medical journal thought, hang on, that sounds pretty offensive. It might cause harm saying that, are we really really sure? It went though peer review. A fair proportion of the doctors in New Zealand probably read that abstract and went, 'yeah, those Pacific Islanders, they catastrophise, they make a big deal of their pain, not like me'. I can't see any record of anyone writing in to the journal saying 'I don't think this is right'.
     
    shak8, alktipping, MEMarge and 11 others like this.
  17. Ravn

    Ravn Senior Member (Voting Rights)

    Messages:
    2,191
    Location:
    Aotearoa New Zealand
    Maybe all our doctors are so overworked and burnt out they don't bother to read the New Zealand Medical Journal? Or maybe they quickly worked out that this sort of nonsense is not worth wasting reading time on? It's not like this article provides any actionable information when faced with an individual patient. Even if the data was rock solid - which almost certainly it is not (though I've only read the abstract) - it's hardly relevant if men as a group on average have "less pain acceptance an willingness" or whether as a group on group average "Middle Eastern/Latin/African patients showed highest levels of kinesiophobia".

    Doesn't explain though why anyone thought this was a good way to study pain experiences, why anyone thought it was a good study to fund, or why anyone thought it was worth publishing. Sigh
     
    alktipping, Hutan, MEMarge and 4 others like this.
  18. Ravn

    Ravn Senior Member (Voting Rights)

    Messages:
    2,191
    Location:
    Aotearoa New Zealand
    alktipping, Hutan, MEMarge and 6 others like this.
  19. Trish

    Trish Moderator Staff Member

    Messages:
    55,951
    Location:
    UK
    As part of the 2021 NICE ME/CFS guideline there are some recommendations for research. They include this one:
    Individual research recommendation details
    Recommendation ID
    NG206/02
    Question

    A core outcome set: What core set of relevant health outcome measures should be used for trials of treatments for ME/CFS and managing symptoms of ME/CFS?

    Any explanatory notes
    (if applicable)
    For a short explanation of why the committee made this recommendation, see the rationale section on managing ME/CFS. Full details of the evidence and the committee's discussion are in evidence review G: nonpharmacological management of ME/CFS.

    Source guidance details
    Comes from guidance
    Myalgic encephalomyelitis (or encephalopathy)/chronic fatigue syndrome: diagnosis and management
    Number
    NG206
    Date issued
    October 2021
    Other details
    Is this a recommendation for the use of a technology only in the context of research? No
    Is it a recommendation that suggests collection of data or the establishment of a register? No
    ______________________

    There is a new research project under way that is discussed on this thread:
    UK:ME Association funds research for a new clinical assessment toolkit in NHS ME/CFS specialist services, 2023
     
  20. Kitty

    Kitty Senior Member (Voting Rights)

    Messages:
    7,067
    Location:
    UK
    It reads like a variant of patient-blaming, only with racist overtones.

    Even if "catastrophising" actually exists (which is not a given, despite the researchers' peculiar lack of curiosity about it), so what? The patients would still be in pain even if they stopped.

    I wonder if some respondents haven't gained insight into some of the issues yet? For patients at least, it can take a long time to develop an appreciation of the differences between fatigue, fatiguability, and weakness, and even then it can be hard to make clear distinctions. But an awareness of the differences does at least highlight the problems with some fatigue scales, which don't offer opportunities to rate them separately or even acknowledge they exist.
     
    RedFox, Sean, alktipping and 3 others like this.

Share This Page