Article on replication crisis

[Alvin]

If the PACE trial was replicated then I expect that we would end up with similar results: CBT/GET leading to slight improvements in subjective self-report outcomes, but not more objective outcomes (aside from people in GET who focus on walking having a slight improvement in walking speed). The problem with PACE was more that it's design had a high risk of bias built in which meant that these results did not show that CBT/GET led to real improvements in participant's levels of fatigue and physical functioning, and could just be down to bias.

The problems with PACE were more down to poor design and spun results and replication wouldn't do much to help us understand the problems there.

Any attempt to conduct a 'new' PACE would need a quite different design, and given that even with the inbuilt biases of the PACE trial CBT/GET only led to small and transitory changes in how people report their symptoms, I don't think spending £8 million on a 'new PACE' will be seen as a wise use of limited resources.

Thats the thing a debate on whether or not to reproduce fraud would be a fiasco. Also that would stimulate great "revelations" and the methods used and the biases and broken protocols would be argued greatly. This benefits us, mainstream scientists arguing en masse whether or not to commit fraud would make PACE untenable. Today its a small group arguing the science, if it became a huge public spectacle we would win. And thats what we want i assume.

It also used oxford criteria and rewrote improvement criteria and boasted in newsletters to influence patients and doctors pressured patients to claim improvement. Let that become part of the very huge public spectacle

I understand the urge to stay below the radar but thats not working. So lets go big and get results. Fortune favours the bold.

 

[JohnTheJack]

But its fraud thats on the books that harms us, so how do we move on?
I'm not saying replication is the only way to get it retracted but if it will work where other things have failed that would be great.
If other things will work such as parliament hearing, a lawsuit, public shaming or any other method that would be great as well

Just to clarify: they proposed that CBT and GET are effective. They conducted small, poor trials and found evidence to support their claim. They then attempted to replicate those findings in a bigger trial and failed. There is no need for further replication. Conducting another large trial on CBT-GET to try to see if the null finding in PACE is replicated is a waste of time and money. As far as the science is concerned we need to move on.

As far as the politics etc, the making sure the null finding is accepted, the recognition that their small crap trials did not replicate and there is no evidence for their claims, all that still needs to be done. But that's a different problem.

 

[Alvin]

Just to clarify: they proposed that CBT and GET are effective. They conducted small, poor trials and found evidence to support their claim. They then attempted to replicate those findings in a bigger trial and failed. There is no need for further replication. Conducting another large trial on CBT-GET to try to see if the null finding in PACE is replicated is a waste of time and money.

I don't disagree

As far as the politics etc, the making sure the null finding is accepted, the recognition that their small crap trials did not replicate and there is no evidence for their claims, all that still needs to be done. But that's a different problem.

All roads lead to Rome, If it were "replicated" which would became a fiasco it would put huge public pressure on this issue. The Lancet does not care about facts or reality otherwise it would already be gone. Whether its because the leader has an irrational belief in CBT/GET, they are afraid of more bad press (perhaps from the Wakefield fiasco) or any other reason it really doesn't matter.

Obviously no one is interested in replicating PACE and i'll bet money its not going to happen. That said if there was an avenue to suggest studies to be replicated i would nominate it.

 

[Trish]

They may not be replicating PACE, but terrible studies like Magenta and Fitnet rumble on. With just as bad or worse methodology to PACE.

 

[Andy]

Replication failures in psychology not due to differences in study populations

A large-scale effort to replicate results in psychology research has rebuffed claims that failures to reproduce social-science findings might be down to differences in study populations.

The drive recruited labs around the world to try to replicate the results of 28 classic and contemporary psychology experiments. Only half were reproduced successfully using a strict threshold for significance that was set at P < 0.0001 (the P value is a common test for judging the strength of scientific evidence).

The initiative sampled populations from across six continents, and the team behind the effort says that its overall findings suggest that the culture or setting of the group of participants is not an important factor in whether results can be replicated.

https://www.nature.com/articles/d41586-018-07474-y