I don't agree that experience of severe ME should be a key criterion for being on the review author group. To me, with only 2 patient/carers on the writing team, it's much more important to be competent to conduct a systematic review of clinical trials and to both understand and be prepared to speak up in the group about the need for things like therapist effect and placebo effect on unblinded trials, and the rest of the serious flaws in the trials under study.
Yes.
But I do agree the process should have some oversight and input by people with experience of severe ME to help the reviewers understand, for example, the significance of using the Oxford definition, and that ME/CFS is not the same as idiopathic chronic fatigue. Also the issue over the studies not properly tracking patients who drop out or report harms. Reviewers who don't understand that PEM is not just about being a bit tired the next day may think PEM as a side effect is mild enough to be ignored. Having someone on the IAG who can tell them first hand about the long term harm caused by GET from personal experience could help them take that problem seriously.
But surely the patient/carer representatives on the writing team should have this knowledge? If those representatives don't know the significance of ME/CFS definitions, or know that dropouts are a common problem in GET trials to be watched for, or can't explain PEM to the other reviewers, or have a good idea how bad severe ME/CFS can be, or know the problems with outcome measurement tools, or can't pull up videos or arrange a zoom call with an expert to help the review team understand if it is needed, then why they are there?
I don't understand what the IAG actually does or how it can work. I mean, sure, if there is one, definitely have people with ME/CFS on it, preferably at least one patient with a very good understanding of GET and the impact of exercise, and hopefully Cochrane is paying them something, so that's nice. But, in this unwieldy process, how do they make any sort of difference? There seems to be a lot of overlaps between the roles of the patient/carer on the writing team, the IAG, and the peer reviewers. I think the subject matter expertise must be embedded in the review team, so that it can be applied as it is needed.
And then, if you were conducting a review of whether homeopathy helped ME, you would not need to have a clinician who is a big fan of homeopathy on the review panel, nor would you need to have a person with ME who likes homeopathy on there either, to explain to the others what ME is like. What are they going to say, to add to the process? Probably, 'in my experience, homeopathy is great'. You'd just need a few people who are good at analysing intervention studies and writing up a review.
Hilda, given it is difficult (although necessary at times) to talk about specific individuals in the ME/CFS review roles, perhaps we revisit the homeopathy example but think about it instead as a review of the Lightning Process?
So, imagine Cochrane is running a 'Lightning Process for multiple sclerosis' review in some country, and there has been quite a bit of controversy about whether it helps this particular illness, with most patients saying it's useless but some being very keen, and most LP practitioners adamant it's terrific and that the government could save lots of money by replacing drugs with the LP. The government too is excited at the potential savings and very interested.
So, would Cochrane typically have an 8 person review team that included:
- a civil servant with very mild MS who thinks that the LP has kept them quite well but has never said so in social media, doesn't have any background in trial analysis, MS or otherwise, and has not been active in a MS advocacy community
- a person who has been providing LP courses for years, and who specialises in providing the treatment to people with MS.
Isn't the review a matter of finding the trials, looking at them carefully to identify issues that might affect the validity of the findings, aggregating the findings, considering harms, and coming to some conclusion about whether the intervention works and what research is needed? Assuming you have appointed a competent MS subject matter expert, and experienced reviewers with integrity, what would the two people on the review team that I mentioned contribute - what are some examples of how their presence would make the process better? Is there not a risk that their presence could make the process worse? Wouldn't it be better to have the LP provider as a peer reviewer, rather than embedded in the process?
