UK:ME Association funds research for a new clinical assessment toolkit in NHS ME/CFS specialist services, 2023

It says they follow AMRCguidance

https://www.amrc.org.uk/listing/category/amrcs-principles-of-expert-review

 

I don't think it is as simple as that. For a charity of this size the only ongoing 'expert' may be Charles. He will then make use of expert reviewers. So we know the name - Charles Shepherd. The quoted bit is vague because it does not specific the level of involvement or relate that to charity size. Individual peer reviewers are generally not named. That tradition of anonymity may be open to question but in this particular context I think it is right. I think review comments on journal articles would be better openly authored but for review that involves decisions about money - and in effect researchers' careers - it is unreasonable to expect reviewers' names to be known.

Life just isn't this simple.

If we followed your arguments, however reasonable they may sound, we should shut down the MRC and Versus Arthritis and the NIH for completely screwing up 90% of funding for the last 50 years.

If people in science had the critical skills of people here then all biomedical problems would have been solved long ago. They don't. Most really aren't up to the job but have got themselves into a career and need funding. Peer reviewers almost by definition believe in the old stuff that science is there to show needs replacing by new stuff. If everybody else could see why the new stuff is better then it would already be the old stuff. No single argument explains why everything is a mess but put them all together and you begin to understand why most advances in science had actually been thought of 200 years earlier but didn't catch on because the community wasn't bright enough to see how it worked.

 

This

 

Yeah and who does the reviewing of the reviewing of the reviewing?
The circular nature of the problem is something budding researchers get to understand pretty early on. As an old boss of mine said 'The system stinks'. But who decides how to unstink it if not the system?

You cannot expect referees to be named. A relatively junior person who is expert in a field may be asked for an opinion and if it is negative and their name is known you can be pretty sure that is the end of their career. Remember those little messages sent by a certain psychiatrist to employers of junior researchers in the past - one of whom nearly go fired and another of whom got hauled up in from of a committee to explain their behaviour.

No, referees who are asked to give an honest opinion on something that may matter £5M to a senior heavyweight, have to be anonymous. And if they are, everyone has to be.

 

The MEA has historically taken the approach that avoiding direct conflict with the establishment will achieve more than sacrificing a good working relationship. It can be disputed whether this is always the right course of action, or whether at certain times a line needs to be drawn, but there are good arguments on both sides. At present BACME represent ‘the establishment’ in relation to UK specialist ME/CFS services.

Also though many of us would like to see the current approach of UK specialist services providing very selective short term rehabilitation to a narrow cross section of people with ME/CFS (ie only the mildly or moderately impaired) replaced with a radically different approach that addresses ME in its entirety (ie addressing the full symptom range and practical implications of the condition across the full spectrum of severity), it is very unlikely that in the short term we will see any substantial change in models of service delivery. Consequently there are good arguments for a close working relationship with BACME trying to influence for change from within.

So I don’t necessarily criticise the MEA for initially giving this grant if they felt it could be a way in to promote change in the current UK services, however I am not optimistic that the current research team are open to such change and so far there are no signs that the MEA will achieve via this grant any significant impact on services beyond getting them to modify their vocabulary choices to mask deviation from the ‘new’ NICE guidelines.

 

Broadly, there seem to be two problems:

• The MEA doesn't appear to accept the potential for harm if this tool goes into use as it is.
  
  
• The MEA hasn't addressed the issue of a researcher they funded asking for patient input, then making unprofessional accusations and refusing to engage further when she received it.

Perhaps these should be the main focus of any questions to them?

The second is interesting. Unless this project is an unusual setup, the grant giving body would have a relationship with Sarah as the researcher, but they wouldn't be managing the project or her work on it. There would be an explicit agreement that the project is delivered as described, on schedule, and on budget (unless variations were agreed mutually, and it's quite normal for that to happen), and an explicit or implicit expectation that it would be conducted in a professional manner and not bring the funding body into disrepute.

The MEA's options in respect of managing the accusations depend on their general policies and the specific conditions of this grant. If these don't contain anything useful about conduct, it might be impractical for them to insist, for instance, that Sarah apologises. The only real option might be the nuclear one, withdrawal of funding on the grounds of loss of confidence; that's a tough choice and may not be justified here.

But in that case, they should engage with us. The work has their badge on it, reports of the researcher's behaviour are in the public domain, and if I were leading the MEA I'd be very concerned about its credibility being undermined. It's not even an especially difficult issue to deal with it, it only needs a willingness to be open to conversation.

 

I think it’s reasonable to ask them for the info the AMRC states in its guidance should be available to the public, because MEA states they follow that guidance.

 

It's important for places who don't have specialist services . If starting from scratch this doesn't bode well

 

Agreed. I struggle for words that will get this across to others with the right nuance but there are certain positions and certain aspects that will necessitate people having to focus on 'defending' whether it be a decision or themselves and so on. There are other questions or issues where the focus can be on 'shall we do or note or ask x, y, z'. Obviously these areas interlink because the former issues might want to be discussions of lessons learned and if there is any clever thing that could be tightened up to avoid such actions (I'm not necessarily talking about something MEA did wrong, nor that they were being allowed to 'anticipate' until you've proof it happens etc).

If it seems like we are criticising 'both' then you end up with a reply that defends eg Sarah's actions being more likely. Of course the longer it goes on then it all becomes unavoidable circle because you have to criticise the failure to have pulled someone up.

As @Peter Trewhitt says it is worth trying things. Sometimes giving people good faith. The issue is when that isn't honoured having systems that make sure just because you did so you aren't lumbered with such a model then claiming it is 'backed by MEA and patients' - both things that can be flagged even if the funding doesn't provide any options for change (and they could do and would have the option of even if they hadn't funded it, and then it turned out to not be as expected/described). BUt it feels like the funding should provide such options on the basis of consultation being part of it.

The worry is that whilst all of this gets allowed to be pretended to be argument - and ignoring is going to allow it to make it look like that because it necessitates the other person has to then repeat themselves and so on - all the time is being lost where change to probably one of the most important things because it is a measure, isn't being discussed in a grown-up way. And I'm so bored of that game/tactic being played on us.

So what if those doing the project are just now non-communicative to changing having done what they've fancied or done, just because MEA backed it initially certainly does not leave them without options or with the only option being to try and get a little movement from something that is intransigent. If it has turned out not to be their project then they can actually step away and say things that are critiques and that is important they remember those choices. And we need to emphasise we are giving space to them of those options and are curious why it isn't happening.

 

I can see an argument that the clinics may be expected by the NHS to develop these tools, and if they do, the odds on them doing it really badly are high—so why don't we pre-empt it by developing a decent one and handing it to them?

It's a strategy I've used many a time in my professional life. Okay, mine were often contracts and project partnership agreements, but the principle's the same. Drafting them is a hassle, it takes skill and experience, and if someone offers you an oven-ready one that looks thorough and fair, it's tempting to accept it. It's not really about manipulation (though there can be a bit of that in the shaping and direction of the thing), it's more about saving yourself weeks of tiresome negotiation if the other party offers up something unworkable.

 

Agreed there are interesting aspects to this. I just don't know how this question can be asked at the same time as the others and it not leading to us not getting an answer to the other questions which potentially something can be done about at this stage.

Because if it is worded as 'why was this project funded' then necessarily the person being asked is having to work with said person and defend in order to defend their own position - all half way house answers to this if asked right now will close down the options on the other questions.

It will also be a question that gets pointed to the person who did in the first place, who will tend to justify and answer that literally in that temporal moment (not a 'when we did we thought x, but now we realise it is y'), because it is basically a question of 'justify', rather than a request to 'fix it' or 'can x be changed' or 'do you still believe in/have confidence in/back..' or a question that might go to a team or others who could look at whether it is 'on track'. They also can't say 'we thought we'd shape it' or whatever because that will end up precluding the 'working with' option.

I think we also have to be aware that time is of the essence and whilst people are working on responses they aren't working on other things (failure demand issue), so too many questions means the one with the biggest implications to them is going to have to come ahead of the next one down, which might be the ones with the tighter deadlines.

Of course what gets complicated is that I know you are rightly asking these questions because if some of these are quite specific objectives that means certain bits can't change or put boundaries in or list what is the purpose/to be achieved in order of priority then that explains and gives clues as to what solutions are possible and other 'needs' are having to be met for it to be useful. I do know that I'd feel more confident if the first step was a pause or something similar so that procrastination doesn't lead to the status quo inadvertently getting signed off. But I guess we'd need to think through and game out how that could get 'used' / be a problem. I can think of all sorts of potential reactions and things that could be going on or suggested behind the scenes that are nudging things certain ways too.

I just don't know how to word things clearly in a question to tip toe around these in a way that isn't going to get misconstrued (other than in back of my head there is something not gamed out about asking for either a debrief to inform future once it has been completed, but how that line makes sure it doesn't close down the bits that might need to be asked if relevant to the 'solutions' part - but even the existence of that means things might need to have 'one eye to' if that has more implications for them). That doesn't mean there isn't one.

 

I would suggest that there is no such thing and never will be.
Any assessment is for a purpose. There are myriad purposes and each will be suited best by a different assessment. The whole thing is a mirage.

 

if we have an idea more specifically of what we think is needed /could be useful, then perhaps phrasing a question along those terms is the angle/starter for ten? or a potential one for the list to choose from

 

Oh, I agree. I'm just trying to second guess the thinking, and the idea of producing something focused on patients' needs and realities might have looked appealing.

Many of us have probably started conversations like that at some point, but when the idea's pulled apart it becomes clear it's a non-starter. But if you don't do that, or do it without inviting input from the parties involved, you risk ending up with something like this.