Assessment at clinics

[PhysiosforME]

Thanks everyone for helping me think more clearly about this matter. I've got a lot more thinking to do yet.

Some quick thoughts.

The "evaluation" is an overall evaluation of the clinic's approach, not of the individual patient. Prior to using any assessment, it would be tested over a period of time with ME patients, and some sort of measure of variability obtained. Minor changes like a pyjama day or not would vanish under those natural variations. After all, if a clinic is claiming success, it needs to be well above any natural variations.

The reason for only asking for details for one day is to reduce load and aim at accuracy. There would have to be some sort of spiel about perhaps choosing a couple of "normal" days in advance to complete the clock chart, and perhaps use the second day as the one to answer the questions.

I did think about electronic methods, such as fitbits etc., but decided firstly that they were too inaccurate, and secondly that they did not easily select energy-sapping tasks, such as driving or trying to arrange house insurance: the emphasis was on physical movement. One problem that reared its head when I was trying to work out why PACE had ordered so few trackers for their 640 patients was the sheer complexity of getting them returned, the data uploaded, and ready for the next patient. When you start to factor in weekends, holidays, delays in the post etc. it gets very messy.

It's interesting, Jon, that you were able to discuss the difficulties of assessing success in teaching maths with your daughter: I had envisaged having to explain the complexities. I also think that Peter's list is very relevant. But surely the point is not so much that we have to measure "success" with such a difficult condition, in which normal ideas of success are inappropriate, but that we need to measure effectiveness, for which Peter's list is very important. I think that there could be two important outcomes from such an analysis. The first is that clinics stop claiming major success in treating the condition, and the second that unhelpful practices or attitudes are highlighted.

As head of maths, even knowing that the analysis was difficult and potentially misleading, I spent a lot of time on it each year. It helped me pinpoint areas that needed improvement. Obviously it had to be combined with other sources of information and professional knowledge, but it kept me focused on the task of making the department as good as it could be. Do specialists running departments have that pressure and do they have suitable methods of looking at that?

This is a fab thread and one I am really interested in. I have a few thoughts but they are really just ideas at the moment so will watch with interest.

 

[Trish]

My thoughts would be to produce a set of ME-patient approved assessments for centres to use to measure any effectiveness.

Just one more thought from me. How can clinics measure effectiveness of their therapy when there is no effective therapy to measure? All they will be measuring is natural fluctuations, and in the case of GET and directive CBT, a combination of short term wishful thinking, and long term harms.

 

[Graham]

Just one more thought from me. How can clinics measure effectiveness of their therapy when there is no effective therapy to measure? All they will be measuring is natural fluctuations, and in the case of GET and directive CBT, a combination of short term wishful thinking, and long term harms.

True enough, but suppose their aim was to give patients an appropriate diagnosis, teach them about pacing and boom and bust, offer help with pain medication, possibly look at sleep problems, perhaps look at useful massage techniques, discuss the possible problems with diet (and the bigger problems with a diet fetish), make a few suggestions about relaxation techniques and the sort of CBT that encourages an efficient use of energies (mainly by being able to say no to things), and, if there is one, put them in contact with a good local support group. That would be valuable, wouldn't it? And wouldn't it show some benefit to relative newbies?

Effectiveness doesn't mean curing: it means doing what you set out to do effectively. If their aims are made clear to the patients, and the limitations, then surely they can be effective. I don't have to get every pupil up to A-level maths standard to be an effective one.

There are just one or two centres in the country that do that: the others push the improve/recover through CBT line.

 

[WillowJ]

I don't require my doctor to be trying experimental treatments in order to be "effective."

I require that

• They have some understanding that I'm very ill even if they haven't got a useful clinical test to show this yet.
• They keep a watch out for problems other than ME and run tests or offer referrals for other stuff when indicated --eta, and treatments as indicated /eta
  
• They chart signs, symptoms, and practical difficulties so there's a record of what's going on
• They offer basic symptomatic relief as appropriate
• They offer guiadance or support (whichever is appropriate for my level of patient experience) for self-management techniques like pacing or dietary symptom manangement
• They offer whatever services, supports, and DME as are appropriate and would be offered to someone with a different diagnosis who was facing the same problems I am facing

 

[Trish]

True enough, but suppose their aim was to give patients an appropriate diagnosis, teach them about pacing and boom and bust, offer help with pain medication, possibly look at sleep problems, perhaps look at useful massage techniques, discuss the possible problems with diet (and the bigger problems with a diet fetish), make a few suggestions about relaxation techniques and the sort of CBT that encourages an efficient use of energies (mainly by being able to say no to things), and, if there is one, put them in contact with a good local support group. That would be valuable, wouldn't it? And wouldn't it show some benefit to relative newbies?

I agree, but I thought the point of this thread was to measure 'effectiveness' using things like activity levels.

If we are talking about clinics giving pacing and lifestyle advice and support, all very valuable, then how would knowing whether patients can get dressed or go to work show effectiveness?

We know pacing is a management strategy, it's not expected to lead to measurable improvements in activity levels, more to stabilise them. So do you count 'no change' as success or failure? It may in fact be that, if someone is pushing too hard to keep going when they arrive at the clinic, actually doing less at the end of a series of clinic visits is a measure of success.

 

[Graham]

Good point Trish. As I said, the aim was to straighten out my thinking, and you are all certainly doing that. Sincere thanks!

Thinking back to my early days, pacing and avoiding boom and bust did produce a definite improvement on what I could comfortably sustain. Perhaps more emphasis on what can be comfortably sustained rather than painfully achieved is key here.

Willow's point about watching out for other problems is also a key concern. Unfortunately, here in the UK people tend to get referred to their ME clinic/specialist (if they have one locally), where they may have one, or perhaps a dozen sessions, then that's it. From then on the General Practitioner takes over. That often, sadly, means that nothing more is done! At our local ME centre, I believe they referred patients still see the specialist for perhaps 20 to 30 minutes for a diagnosis, then are passed on to a support team for explanations and CBT type treatment in a series of sessions. No-one watches out for progression of symptoms etc. and you never get called back to see how you are doing. As for offering services or support (I'm guessing DME is disability medical equipment), patients have to chase that up themselves.

 

[WillowJ]

I'm guessing DME is disability medical equipment

oh sorry, durable medical equipment, but yes it would include disability items like a shower transfer bench (a very awkard item and the problem of getting over the tub rim ought to be solved in a different way, imo), wheelchairs of all types, grab bars, and so on.

It doesn't include items that get used up like gloves and protective garments, which I don't know the category for.

 

[WillowJ]

As far as "how yer doin'" questionnaires, I think at this point these are most useful for keeping a record of signs, sympoms, and practical difficulties, but I also think it's a good thought to develop ME-specific questionnaires so they can be used in efficacy assessments at a future time, or as secondary measures during drug development/testing.

My doctor never does give me a routine pain sheet or symptom survey (I get one when establishing with a new doctor, but never for follow-up), and I think it would be logical if I had one every visit. But it would need to be brief.

 

[WillowJ]

I had another thought.

Sometimes after an appointment I get a letter in the mail, or nowadays a link to a webpage, with a survey from a third party assessor. They want to know about my clinic visit and they ask all sorts of things like:

How easy was it to get an appointment?
How long did you have to wait for an appointment?
Were the appointment staff polite?

How long did you have to wait after you arrived at the clinic?
Were you informed of any delays?
Were the staff polite?
...

Was your doctor well informed about your health condition?
Had your doctor read your chart?
Did your doctor listen to you?
Did you doctor explain things well?
Did your doctor understand your needs?

etc.

Maybe that kind of questionnaire could be informative at this point in time. I stopped answering mine as I don't see it as mostly the individual doctor's problem that they don't know anything about my conditions, but I used to write up mine in the comments spot (after the answer where you filled in "a little bit" versus "a lot" or "not applicable" or whatever else, there's a freeform section to write anything you like) that nobody was educated and it wasn't the doctors' fault, but still they didn't know about my health condition, and I would give an informative source.

ETA: sometimes they'll take the freeform answers from the doctor section and post them online (the best places will post everything, good and bad) as physician reviews.

 

[Hutan]

I agree WillowJ that that kind of assessment is very useful.

In my experience with the health system, the people/groups who actively ask for feedback tend to be the ones offering the best service. I expect there's a two way relationship going on there:
1. the feedback helps them improve their service;
2. the ones willing to deal with feedback are also the ones who care most about ensuring their patients have a good experience.

 

[Peter T]

I agree WillowJ that that kind of assessment is very useful.

In my experience with the health system, the people/groups who actively ask for feedback tend to be the ones offering the best service. I expect there's a two way relationship going on there:
1. the feedback helps them improve their service;
2. the ones willing to deal with feedback are also the ones who care most about ensuring their patients have a good experience.

When giving patients feed back forms, I found enormous variation in the responses depending on when they were given the form. Invariably feed back forms filled in immediately following a session/event were uniformly positive and so provided little constructive information to act on. Unless something very obvious had gone wrong, when the clinicians were also very aware of the issues anyway. We would get much more helpful critical feedback after a few weeks or even longer, though obviously there was a much lower return rate.

I expect feedback being collected from by an independent third party might also be less biased by the hopefully positive relationship between the clinician and the service user. Also it is very hard to give negative feedback to some one you have had face to face contact with. Certainly when many years ago I felt obliged to give very negative feedback about someone leading a two day manager trailing course, it got very unpleasant. [added - though I did also follow up the negative feedback with a formal complaint.] An important consideration is the power relationship influencing any feedback, and generally there is an unavoidable power imbalance between a medical professional and service users.

Though I wonder how helpful feedback forms would be in ME. A form after a session would be a nightmare. With something taken home, I put feedback forms on the non essential pile, and invariably they lose out to taking a shower or preparing a meal. Something on line that can be done lying in bed on an iPad might not fare so badly.

 

[Graham]

I think it is vital to keep everything to a minimum, in order to make it as easy as possible for the patient to respond.

This is my thinking, so far, pinching all your thoughts of course! Before, after, and follow-up assessments: a few days before the first session, a week or so after the first few sessions (or one session if that's all there was), and a follow-up after a year, even if sessions are still going on.

1. A short "state of mind" set of questions, obviously subjective, asking whether patients think they have ME, understand pacing, etc.: the sort of thing that well-explained support from most centres should give. This is not to seek out depression etc. but to check that the centre has got the basic ground information across, and that it treated its patients with respect. It could also ask about help with pain, sleep etc.

2. A short factual set of questions about what the patient did the day before: the aim of this is to decide on the severity – severe, very severe, or unbelievably severe. The questions should aim at extending the low end of ability so that you don't get the bottom end clumping of scores like the Chalder Fatigue Scale does. It should be robust enough to cover our usual day-to-day fluctuations, although I do appreciate that some people experience quite large changes. It needs some sort of let-out clause.

3. A short online cognitive test along the lines mentioned. We were discussing this today at our local support group meeting, and many of us felt that a key part of ME is the way in which the level of brainfog matches the level of physical disability, whereas in other conditions the brainfog is generally less (for example, members with EDS were in wheelchairs: they experienced brainfog but were capable of driving reasonable distances, much further than others with ME who were not in wheelchairs).

My belief is that newbie patients could find that pacing, and understanding ME, along with the relief of getting a diagnosis, will create a small improvement in what patients can do and in brainfog: a key question is whether that would be maintained.

My reasons behind this are that (1) could provide some validation that the centres are helping: that (2) could provide better measures of the range of severity of ME, and that it is unlikely to register significant improvement in general: and (3) may turn out to be a reasonably reliable measure of the severity of ME, but will need to be compared with (2).

Of course this would be pretty irrelevant to the way-beyond unbelievably severe patients, but who but an idiot would want to put them through some form of assessment? If anything is found to help them, it would be pretty obvious.