Reversals in psychology, 2020, blog by argmin gravitas

[ME/CFS Skeptic]

I've finally found a useful overview of findings in psychology that were once in high esteem but are now refuted. See: https://www.gleech.org/psych

Many of the things were in psychology textbooks quite recently, including:

• Macbeth effect
• Pygmalion effect
• Ego depletion
• Power posing
• Stanford prison ‘experiment’ and the Milgram experiment
• Multiple intelligences

If I understand correctly, the change in psychology towards replication and critical reviewing popular theories and findings started after Daryl Bem published a paper on precognition in 2012. It used the same methods and statistical techniques respected researchers use, but to 'demonstrate' an absurd point namely that undergraduates can improve their memory test performance by studying after the test.

The overview notes about this paper:

"This one is fun because Bem’s statistical methods were “impeccable” in the sense that they were what everyone else was using. He is Patient Zero in the replication crisis, and has done us all a great service."​

The replication crisis in psychology helps me to understand and grasp what has happened in ME/CFS where scientific research seems to have made some awful mistakes. Science seems to have been using standards that are way too low. it is difficult to explain to outsiders how experts and researchers could be so wrong. In my experience, the replication crisis in psychology helps to explain things because it is very recent history.

This might be a provocative idea but perhaps quacks such as Parker and Gupta will show the way forward by using the same methods of GET/CBT researchers but with absurd conclusions. Much like what happened after Bem, it may become obvious that standard researcher methods used in the GET/CBT literature aren't reliable. Perhaps homeopathy proponents should start setting up non-blinded trials with subjective outcomes and a waiting-list control group?

 

[Hoopoe]

I don't see why we need any new clinical trials. There already are good examples showing how self-reported health is easily influenced by sham treatments.

Psychological medicine just doesn't seem ready to accept that there are problems with their standard methods.

It's an interesting question what it would take for the field to start doubting themselves.

 

[Peter T]

I've finally found a useful overview of findings in psychology that were once in high esteem but are now refuted. See: https://www.gleech.org/psych

Many of the things were in psychology textbooks quite recently, including:

• Macbeth effect
• Pygmalion effect
• Ego depletion
• Power posing
• Stanford prison ‘experiment’ and the Milgram experiment
• Multiple intelligences

If I understand correctly, the change in psychology towards replication and critical reviewing popular theories and findings started after Daryl Bem published a paper on precognition in 2012. It used the same methods and statistical techniques respected researchers use, but to 'demonstrate' an absurd point namely that undergraduates can improve their memory test performance by studying after the test.

The overview notes about this paper:

"This one is fun because Bem’s statistical methods were “impeccable” in the sense that they were what everyone else was using. He is Patient Zero in the replication crisis, and has done us all a great service."​

The replication crisis in psychology helps me to understand and grasp what has happened in ME/CFS where scientific research seems to have made some awful mistakes. Science seems to have been using standards that are way too low. it is difficult to explain to outsiders how experts and researchers could be so wrong. In my experience, the replication crisis in psychology helps to explain things because it is very recent history.

This might be a provocative idea but perhaps quacks such as Parker and Gupta will show the way forward by using the same methods of GET/CBT researchers but with absurd conclusions. Much like what happened after Bem, it may become obvious that standard researcher methods used in the GET/CBT literature aren't reliable. Perhaps homeopathy proponents should start setting up non-blinded trials with subjective outcomes and a waiting-list control group?

Can’t be certain that this would work.

I had thought that Prof Crawley’s SMILE study would be the reductio ad absurdum of this approach, and lead people to question this whole use of subjective outcomes in unblinded trials relating to GET/CBT, but low and behold seeming rational intelligent scientists started taking seriously the use [of] an obviously quack intervention, namely the Lightening Process.

 

[Hoopoe]

What do proponents of these clinical trials really believe about the bias affecting subjective outcomes?

If we understood what they believe, it might be possible to design an experiment that shows they are wrong. The response by Sharpe that placebo effects were not an issue in the PACE trial is absurd and reveals that either they managed to avoid thinking at all about this important issue or deliberately exploited it and are feigning stupidity in response to being called out on this.

(My recollection of what Sharpe said, was that the questionnaires aren't susceptible to reporting bias and that placebo effects (in ME/CFS?) don't last long enough to affect the results. Both of these ideas can be demonstrated to be incorrect).

Perhaps what they really believe is that placebos are curative and that patients with psychologically-maintained health problems can be treated with sham treatments because what matters is that the patient decides to get well. Therefore there is no need to control for placebo effects.

But that interpretation doesn't really make sense either because if this were true then placebo treatments would cure ME/CFS for real, with impressive objective improvement. The opposite can be observed.

 

[Jonathan Edwards]

I had thought that Prof Crawley’s SMILE study would be the reductio ad absurdum of this approach

This is the salient problem. I agree with Michiel in principle but reality has trumped theory.

 

[rvallee]

What do proponents of these clinical trials really believe?

If we understood what they believe, it would be possible to design an experiment that shows they are wrong. The response by Sharpe that placebo effects were not an issue in the PACE trial is absurd and reveals that either they managed to avoid thinking at all about this important issue or deliberately exploited it and are feigning stupidity in response to being called out on this.

Perhaps what they really believe is that placebos are curative and that patients with psychologically-maintained health problems can be treated with sham treatments because what matters is that the patient decides to get well. Therefore there is no need to control for placebo effects.

But that interpretation doesn't really make sense either because if this were true then placebo treatments would cure ME/CFS for real, with impressive objective improvement. The opposite can be observed.

Simon Wessely summed it up this way, paraphrased: "the placebo is one of the most powerful interventions we have". They believe their effect is a placebo effect. They think the placebo effect is powerful and that our experience is a powerful nocebo. It's circular logic lacking any evidence but this is the crux and why they think their trials "work": they simply define work as nothing, since they defined the problem as there being no problem it's "consistent" in a twisted way where consistent means not consistent.

Essentially they are answering their own questions, which have nothing to do with the problem, but nevertheless they only ask those questions and none other.

 

[Jonathan Edwards]

What do proponents of these clinical trials really believe?

I am not sure that these people believe anything. They see that following a fashion is their best way to get ahead. They very much give the impression that they do not have any ability to judge what is true and what is not so work on the principle that it doesn't matter. I have come to think that being able to judge whether one theory holds up better than another is actually quite a rare talent amongst scientists. It is reasonably common amongst people in general - at least people who take an interest in science - but maybe those people self-select in a different way from scientists.

 

[DokaGirl]

Interesting that when study PARTICIPANTS answer GET/CBT study questionnaires, presumably favorably, ( possibly, or likely due to various influences), the participants are believed.

However, when PATIENTS describe ME as a physiological disease, a living hell that is worsened with exertion of any kind, these patients are delusional, fabricating, lazy etc.

Interesting that study participants "tell the truth", and ME patients "lie".

 

[Hutan]

From the blog - might be useful.

Reason to be cautious about mindfulness for mental health. Most studies are low quality and use inconsistent designs, there’s higher heterogeneity than other mental health treatments, and there’s strong reason to suspect reporting bias. None of the 36 meta-analyses before 2016 mentioned publication bias. The hammer may fall.

 

[Sean]

because what matters is that the patient decides to get well.

This will always be their fallback position: It only works if the patient wants to get better.

The big plus of this position being, of course, that it is completely unfalsifiable. It is impossible for a patient to prove they do want to get better and are trying hard enough.

 

[Trish]

This reminds me of Professor Brian Hughes book 'Psychology in Crisis'. It's a while since I read it (published in 2018) but I think it covers much of the same sort of problems raised here.

 

[lycaena]

There is also the popular science book "Science fictions" by Ritchie Stuart about the replication crisis. https://www.sciencefictions.org/

 

[Invisible Woman]

I think there's also a fundamental laziness that creeps in.

If you aren't all that invested in what you're doing - it puts food on the table and pays the mortgage - and everyone else thinks "xyz" then "xyz" is the way to go.

No effort on your part, you have the defence that it's just what everyone else thinks, plus you don't have to disagree or stand up to anyone more powerful than yourself.

 

[ME/CFS Skeptic]

So we might need something more absurd than the Lightning Process?

 

[rvallee]

So we might need something more absurd than the Lightning Process?

I doubt there is a limit to how absurd the "treatment" can be, it's not judged this way, it's judged based on perception that the problem is silly and therefore it's worth trying anything, even the same silly thing over and over again without regard for any process or common sense.

 

[Hoopoe]

So we might need something more absurd than the Lightning Process?

I think we need a biomarker so that one can simply show a lack of improvement of the illness after psychotherapy that is "shown" to be effective with the usual flawed trials.

It will be up open minded researchers to conduct such a study because psychotherapy proponents do not want to find out if their therapy really works.

We already have a few studies showing a lack of improvement on objective outcomes but that doesn't seem to have convinced any of the psychotherapy proponents that psychotherapy might have no effect on the illness.

 

[Jonathan Edwards]

So we might need something more absurd than the Lightning Process?

Well there's always the dead parrot sketch.

 

[Sean]

Well there's always the dead parrot sketch.

And it was a Norwegian Blue.

 

[MEMarge]

So we might need something more absurd than the Lightning Process?

If we could get across the absurdity and dangers of LP, it might be enough.

Anyone know any actors or comedians who could produce a sketch about it?

 

[rvallee]

If we could get across the absurdity and dangers of LP, it might be enough.

Anyone know any actors or comedians who could produce a sketch about it?

No kidding but a well-made animation showing what it actually is would probably get more across than hundreds of scientific papers.

Just what it is. Nothing more. This is one of those things that are maximally absurd when you just neutrally list what it is without any emphasis, the absurdity speaks for itself.