Recipe for study that will give good results

This post has been copied from the thread "Lightning Process study in Norway - Given Ethics Approval February 2022"

This article may belong in the News from Scandinavia where I have collected a few others in the same series, but it was very relevant here. How to design your study to give a positive result, and I think most will recognize the study design of the LP study

Recipe for study that will give good results

• Let the study be carried out by researchers who have a lot to gain from good results for the form of treatment being tested. It tends to affect the interpretation of the results.
• Carry out the treatment in the studies in a way that maximizes the hope and expectation of the participants, not necessarily the way the patients will receive the therapy in reality.
• Not controlled for the placebo effect! Rather, set up the experiment so that you compare the treatment with something that might make the patients worse. This is how it looks like your treatment is having a great effect.
• Bring a few participants. This increases the chance of incidental positive findings, even when the treatment does not really have an effect.
• Measure many possible changes to the treatment. If the method doesn't seem to work on what you were supposed to investigate in the first place, some positive changes may appear on the other targets. And then you can just focus on them, while not talking so much about what didn't work.
• And should everything still end in negative results, you can always put the study in the drawer, instead of making the bankruptcy public, write Cuijpers and Cristea.

The article also features a short discussion on the problems of wait list controls, with a link to a more in-depth article on the topic

Noe er galt i forskningen på psykologi
Something is wrong in psychology research

 

The rest in the series at Forskning.no (google translate web is at time of writing this not dealing well with the forskning.no site, so only including the Norwegian links with a translation of the titles):

Hvorfor er forskerne så uenige om hvor godt psykoterapi virker? / Why are researchers disagreeing on how well psychotherapy works?
Dag Wollebæk gikk inn i teoriene bak psykologisk behandling: – Jeg har kastet bort nok tid på å gå i terapi / Dag Wollebæk read up on the theories behind psychological treatment: I've wasted enough time on therapy *
Mellom 5 og 10 prosent blir sykere etter å ha snakket med psykolog. Noen behandlingsformer gir større risiko for skade enn andre. / Between 5 and 10 percent become worse after talking to a psychologist. Some forms of treatment have higher risks of harm than others.
Hva er problemet med venteliste-studier? / What is the problem with wait-list studies?
Det teoretiske grunnlaget for psykologien vakler. Ingen vet om teoriene er riktige

*Wollebæk has been critical of the Lightning Process study at the Norwegian university NTNU due to bad methodology, and voiced support for pwME in social media

Forskning.no also include this disclaimer "for readers with psychological problems":
"In this series, we ask questions about research into psychotherapy. Several researchers speak out critically, and there is much disagreement. But the researchers agree on one thing: if you have mental health problems, there are generally better chances of recovery if you go to therapy. And if you get treatment that you feel is good for you, there is no reason to stop."

I think the last sentence is unfortunate as it does not acknowledge that one can be harmed from treatment despite feeling it is "good".