Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

Oh yeah - I remember now.

 

Programs like this will not only make Cochrane absolete but an enormous percentage of white collar jobs. My brother used to train helpdesk-employees, he'd have to build training programs so that people could actually give solid advice within a couple courses. The jobs on either side of that equation will be obsolete not too long from now as it wouldn't be too hard to hook up ChatGPT to a bunch of user manuals, have it extrapolate useful information and then answer the phone for you. Or just answer online.

My work pre-ME was as a mid-level accountant I think you could say(3 years of study). I've worked for a company that did the books for self-employed people and small businesses. At an internship some woman was still writing stuff down manually whereas we were thought in excel, which was much quicker at the time. Excel got replaced by an accounting-program which got replaced by a better one and so on and so fort, tremendously increasing output of a single accountant. I'd imagine the coming evolutions would eliminate the job almost completely.

We're already able to scan just about anything with our phones, we can pay with our phones too. It's a very small step from that to having your phone immediately log it in your accounts. Government being disastrous at IT will make the final hurdle a bit harder to climb, but once you can log everything into your accounts at once it should also be very doable to get a summary at the end of each year which you just have to gloss over before sending it to the inland revenue.

With machine-learning the easy to spot errors can already be found in research papers, once this thing blows up further it can likely review thousands if not millions of papers at once and spot every tiny thing wrong with them.

But as you said, the algorithms need training. The thing is it gets easier to train with each new iteration and we likely won't be dependent on Cochrane for setting something up. Some angry scientist who has a buddy that specializes in automation could likely set it up somewhere in the future, from my limited understanding.

 

I need to see a movie or mini-series about this happening, please.

 

Given what happened to postmasters and postmistresses in the UK because of accountancy software with bugs in it I think the prospect is terrifying.

https://en.wikipedia.org/wiki/British_Post_Office_scandal

 

“if scientists come to rely on AI programs to prepare literature reviews or summarise their findings, the proper context of the work and the deep scrutiny that results deserve could be lost”.
Cochrane is so good at that "deep scrutiny" isn't it?! https://www.theguardian.com/science...ban-listing-of-chatgpt-as-co-author-on-papers

 

An Ianucci sketch on this might have more effect ....

 

or reviewers of published papers who only read the abstract....

 

Thanks Caroline.
My initial reaction is that it's terribly convenient for Cochrane - they're politically clever enough to realise this [puberty blockers and hormones for adolescents] is risky territory. OK to misrepresent those with ME/CFS of course --- less controversial target.

Great idea to use this against Cochrane!

 

The weird thing about this is it doesn't seem to understand that AI-generated content can, and should, be checked by humans.

This is basically the same as not trusting mechanical calculators because human computers, it used to be a job title, are much better at it.

It's not as if the content has to be used as is, without even checking. It's a multiplier, it doesn't remove human intelligence from the loop. But it does have the potential to remove bias, and that is basically disastrous for psychology and everything that relies on their methods, like EBM, where they need bias, otherwise they have nothing.

But on the part of Cochrane, I don't really see their product, the reviews, as their main thing. It's a club, it's about people connecting with other like-minded people and using that influence. The reviewers are the real product, not the reviews. If all that's left is the reviews, the entire organization is superfluous, as they can be produced far better, faster and cheaper without the bloat of their private influencers club.

 

It's the basic trainee level scrutiny I find a bit lacking.

 

Speaking of which, I saw a few people experimenting with ChatGPT to review some of their old papers.

Lots of work ahead but it seemed to do better at the most basic stuff than the obscure, which is pretty much the opposite we see with humans, who can argue forcefully about semantic trivia while being perfectly OK with overlapping too-disabled-to-work and recovered-and-fully-healthy thresholds.

Or somehow being OK with open label trials with subjective outcomes from biased researchers, even when they're not even allowed. Bots won't accept the "pseudoscience is Ok for pseudo-illness" nonsense unless it's explicitly written in their instructions, which I wouldn't put past organizations like Cochrane.

Not sure what an editor would think of a bot review that says "didn't bother reading past the abstract", though.

Only a matter of time before it does better than humans, although of course better than humans is not desirable to some, and basically a death sentence for the BPS ideology.

For further discussion go to this thread:
OpenAi's new ChatGPT

 

Back on topic.

Trial By Error: What the Hell is Going on with Cochrane’s Long-Delayed Re-Review of “Exercise Therapies”?

"What is going on with Cochrane’s much-delayed re-review of exercise therapy for the illness the organization is these days calling ME/CFS? It is more than three years since Cochrane announced plans to develop this completely new review, but no one outside the organization seems to have much idea of what’s happening with that project.

Maybe no one inside really does, either.

Patients are frustrated and angry—and rightly so. The existing review, as flawed as it is, still gets cited as evidence that exercise therapies have been shown to be effective treatments. Recently, for example, the authors of a propaganda piece presented in the guise of a “research agenda for post-COVID-19 fatigue” referenced the contested Cochrane review to assert this exact point and suggest the same interventions would be helpful for the new global cohort of long Covid patients. However, in a telling omission that highlights the weakness of their argument, the authors failed to mention the current ME/CFS guidelines from the UK’s National Institute for Health and Care Excellence (NICE). These were published in 2021 and reversed NICE’s prior exercise therapy recommendation; the agency assessed the quality of the evidence in favor of these interventions as uniformly “low” or “very low.”"

More at https://www.virology.ws/2023/03/21/...long-delayed-re-review-of-exercise-therapies/

 

thanks!

 

Agree it is interesting. But is it a good example?

I think the situation is hardly comparable -- only able to skim-read but if i understood correctly, the NICE commissioned an external reviewer to evaluate the evidence who found the evidence base not strong enough to issue an actual position:

"The evidence reviews were published in March 2021. Unfortunately, the available evidence was not deemed strong enough to form the basis of a policy position."

https://cass.independent-review.uk/nice-evidence-reviews/

https://www.england.nhs.uk/commissi...mme/implementing-advice-from-the-cass-review/

From skimming the bmj article, the references and also the responses, I don't have the impression the mentioned bmj article gives a fair account, neither of some details regarding the evidence nor the involved parties' disagreement on interpreting the evidence.

The evidence on exercise for ME/CFS I think is pretty conclusive as is the current NICE guidance on ME/CFS -- exercise is not an effective treatment for ME/CFS.

Yet, for decades, proponents of exercise for ME/CFS both have done bad trials and have misrepresented the evidence, and they continue to do so.

And the bmj has played its part in misrepresenting the evidence and also how NICE arrived at its conclusions in the new ME/CFS guideline.

Anticipated apologies if I misunderstood or misrepresented anything, but I think we need to be very cautious with referring to material and decisions that seem to suit our case without taking a closer look.

Also, I'm reminded of the NICE guidance on chronic pain (discussed here) -- would we want Cochrane to adopt NICE' evidence review of this guidance, too?


(Edit to replace centuries with decades )
Edit 2 to remove redundant part.
Edit 3 to remove another redundant part and to clarify the point about the bmj's role in the tangled story of ME/CFS

 

I echo this.

No answer yet.

 

For me what is relevant is not whether the ME exercise situation or the use of puberty blockers have any specific parallels, but rather that it is a precedent of Cochrane using NICE reviews to inform their own action. Obviously Cochrane should not be bound to slavishly adopt any NICE position, but I think Cochrane now needs a clear justification for not withdrawing their flawed exercise review completely and why they need to commence a new review, in the light of the seemingly clear conclusions from the NICE evidence review and the consequent new NICE ME/CFS guidelines.

 

I realized that bit but the reasoning behind this officially seems to be there is no sufficiently strong evidence base to do a review. That is a step before doing the actual assessment of the evidence.

And again, just to follow NICE's decisions per se seems not a good point -- see the old NICE guideline (based mainly on the same evidence base but different criteria in the assessment process) and also the current chronic pain guideline.

Not defending Cochrane in any way just don't think this is a good point on our behalf.