Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

Yes. This is a political problem of why standards are not being followed for ME, and are indeed actively being diluted and discarded. It isn't a technical problem with the normal standards, that everybody else has to follow.

 

This.

We may not be able to adequately blind all measures in all situations, but we can use much more objective measures of the real-world practical consequences of the self-reported benefits, including the critical testing of correlations with self-report measures.

Patterns of changes in activity levels after therapy are certainly objectively measurable across a wide domain. Assuming, of course, that increasing those levels is the end point of therapy, which the highest of authority states is so:

 

We have very little information about Kay and specifically about her knowledge of ME/CFS issues, but the information that we do have suggests to me that she has not spent much time becoming familiar with the politics or the science of ME/CFS. I don't see how she can be effective in guiding the members of the review team who are not familiar with ME/CFS research.

I'm concerned about the process that Hilda applied in order to select the sole person with ME/CFS for the review team. There are many people with ME/CFS with good skills in research analysis who would have made valuable members of the review team.

I am seeing a process that almost could not have been designed better to make people with ME/CFS familiar with the issues with GET be frustrated and despairing. I have no doubt that our expressions of frustration will be turned back on us by some, to suggest that we aren't grateful when people try to help us, that we are a very difficult community and that is why so few people want to be associated with ME/CFS. But, a straight-forward evidence review that was staffed by qualified unbiased people would not be causing these problems.

I hope Hilda that you will understand what I am saying here. It's a car crash in motion. I will try to look away soon, but I'm concerned about the damage to the ME/CFS community that will be left at the end. Please just start again, aiming to make the review simpler and more robust.

edited

 

I agree that the information we have about the patient on the review panel tells us nothing about her particular suitability for the role. I think an explanation of this surprising appointment lies in the criteria spelled out in Hilda's report:
https://community.cochrane.org/orga...ofile-reviews-pilot/cochrane-exercise-and#Two

Nothing in the criteria suggests the need for people with proven knowledge, skills and experience with in depth reviewing of clinical trials.

I apologise if this has already been answered, but I would like to know from @Hilda Bastian
- who came up with these criteria,
- what role people selected to fit these criteria are expected to play in the technical process of reviewing research, including data analysis, and understanding and being able to contribute to decisions on what to conclude from the evidence including being able to explain why some data is unfit for purpose (eg the Chalder Fatigue Questionnaire).

There is only one of the above criteria which hints at a useful background, but it is so vague as to be meaningless:
'A history of interest in research on ME/CFS'

Edited.

 

But to be clear that is a priority choice to prioritize pandemic response things.

[edit]
Cochrane is also an organization with many involved so I assume additional resources could have been found to help.

 

What bothers me is that Hilda was searching online for candidates and then appointing them rather than running an open (and perferably transparant) process. If we compare this to say how the PSP recruited people by setting up an initial small committee (including the CMRC PAG representatives and @MEMarge nominated from here) and using that to advertise and operate a recuritment process. That got good candidates and this is at the heart of progressing.

Isn't this meant to be a template process for other reviews and including 'consumers' and hence it is worth reflecting on the process, how it is run, how it would be reproduced and how it compares to other processes that seek to get patients involved (such as the PSP).

Updates on this overall recruitment process could have been given over time as they were happening. Things like opening up for applications with a close date; what was being looked for for applicants (reflecting the range of skills and knowledge that would be valued). This didn't seem to happen (although I admit I've been very busy doing other stuff so it may have slipped by).

And I noticed on Hilda's report that

So I would have hoped that a project manager from Cochrane would have pushed for a better process.

 

One thing I think is important is this

It is really important that patients get to comment on the protocol as that is at the heart of what the review will say and whether it will be any good. I am worried about the brief comment as patients need time to digests if brief is say just a week that is insufficient time for a good response given peoples illness. I don't see why it can't be done at the same time as peer review.

I also think it is really important that comments are taken into account and coherent reasons (not just we will have to differ here like happended with comments on the previous cochrane review) should be given for the major criticisms. Also I'm concerned that putting a peer reviewed protocol out for further comment could lead to difficulties in getting changes as I can see comments 'but peer reviewers said it was fine' even though peer review is a really poor process (as people don't have time for detailed analysis).

So it would be good to have the full process layed out now in terms of proposed time scales and any back and forth that will be expected with the IAG, peer reviewers, the patient community etc. Lets have a process defined (it can change if necessary) rather than doing it in an adhoc manner.

 

Will some peer reviewers be drawn from the patient community?

 

So maybe more controvertial in terms of peer review could S4ME be a peer reviewer as a group? I know that goes againt the normal model. I assume there would be confidentiality concerns but we could put together a group of patients to review and have a private forum to work in an collate comments (I think we did this pulling together comments for the NICE long covid guidence as it was confidential). That may take away from some individual patients as reviewers but may allow those whose health is not up to doing a full review (or who have prefered to stay anonymous in contributing to previous work) a chance to apply their limited time/energy to where they have skills.

 

So the PSP recruitment happened during the pandemic (it is only UK wide) and I do realize the issues with working with the pandemic. Its not unusual for me to spend all day on zoom and then have to spend my evenings and weekends catching up with work (which is what I should be doing now!).

But if this is meant to be a template for the future it is really important to think through process and what is robust and repeatable.