A Waste of 1,000 Research Papers

[Andy]

Decades of early research on the genetics of depression were built on nonexistent foundations. How did that happen?

In 1996, a group of European researchers found that a certain gene, called SLC6A4, might influence a person’s risk of depression.

It was a blockbuster discovery at the time. The team found that a less active version of the gene was more common among 454 people who had mood disorders than in 570 who did not. In theory, anyone who had this particular gene variant could be at higher risk for depression, and that finding, they said, might help in diagnosing such disorders, assessing suicidal behavior, or even predicting a person’s response to antidepressants.

Back then, tools for sequencing DNA weren’t as cheap or powerful as they are today. When researchers wanted to work out which genes might affect a disease or trait, they made educated guesses, and picked likely “candidate genes.” For depression, SLC6A4 seemed like a great candidate: it’s responsible for getting a chemical called serotonin into brain cells, and serotonin had already been linked to mood and depression. Over two decades, this one gene inspired at least 450 research papers.

But a new study—the biggest and most comprehensive of its kind yet—shows that this seemingly sturdy mountain of research is actually a house of cards, built on non-existent foundations.

https://www.theatlantic.com/science/archive/2019/05/waste-1000-studies/589684/

 

[ScottTriGuy]

I like this comparison...made me chuckle...

"...It’s as if they’d been “describing the life cycle of unicorns, what unicorns eat, all the different subspecies of unicorn, which cuts of unicorn meat are tastiest, and a blow-by-blow account of a wrestling match between unicorns and Bigfoot,...”

 

[NelliePledge]

Perhaps Ed Yong could do a follow up article about PACE

 

[Hutan]

Dorothy Bishop from the University of Oxford argues that institutions and funders that supported candidate-gene work in depression should also be asking themselves some hard questions. “They need to recognize that even those who think they are elite are not immune to poor reproducibility, which leads to a huge amount of waste,” she says.

That would be the same Dorothy Bishop who found the SMILE study to be
"generally well conducted and noted that the findings appeared to be solid."

 

[Hip]

I am reminded of Prof Paul Ewald's point of view, that the whole genetic causes of disease bandwagon has been vastly overplayed for decades. From Wikipedia:

Ewald says that "chronic diseases, if they are common and damaging, must be powerful eliminators of any genetic instruction that may cause them." One example of this is schizophrenia; patients with this mental illness rarely reproduce. Ewald argues that, just by evolutionary pressures, schizophrenia would have already been eliminated if its causes were strictly genetic; he suggests that in the future, an infectious cause of schizophrenia will be discovered.

Further evidence for a non-genetic etiology of diseases like schizophrenia, Ewald also points out, comes from concordance studies on identical twins, which measure the percentage of identical twins who both develop a disease. A concordance of 100% indicates a primarily genetic disease, which is not really influenced by environmental factors like infection, nutrition, or toxins.

Huntington's disease, for example, has a concordance rate of 100%, indicating a predominately genetic etiology. However, when the concordance rate is lower, this indicates environmental factors like infectious microbes or toxin exposure are playing a causal role. Schizophrenia's concordance is approximately 35-60%, suggesting, says Ewald, that microbes are etiologically involved.

Another example is breast cancer: Ewald notes that in the case of identical twins, when one twin develops breast cancer, the other twin has only a 10% to 20% chance of developing the disease, and this concordance rate of just 20% again indicates that environmental factors like infectious microbes or toxins are likely playing large causal roles in breast cancer.

 

[Sly Saint]

“Any evidence that the results might not be reliable was simply not what many people wanted to hear.”

“We’re told that science self-corrects, but what the candidate gene literature demonstrates is that it often self-corrects very slowly, and very wastefully, even when the writing has been on the wall for a very long time,”

“There’s an unwillingness to part with a previous hypothesis,” he says. “It’s hard to wrap your head around the fact that maybe you were on a wild goose chase for years.”

“Those who don’t learn from the past are doomed to repeat it.”

MS and SW take note

 

[Arnie Pye]

“We’re told that science self-corrects, but what the candidate gene literature demonstrates is that it often self-corrects very slowly, and very wastefully, even when the writing has been on the wall for a very long time,”

And when it is advantageous for a powerful group of people - e.g. governments, the obscenely wealthy, big business - for science NOT to change there are plenty of scientists who can be bought to achieve that end and delay progress for decades.

 

[Snow Leopard]

What about our own backyard?

There are candidate gene association studies in ME or CFS that are likely spurious, but that doesn't stop many people from believing the findings.

 

[Snow Leopard]

That would be the same Dorothy Bishop who found the SMILE study to be
"generally well conducted and noted that the findings appeared to be solid."

She said that via email to David Tuller?

Here is what she said on the SMC site:

“The gains for patients in this study do seem solid, however, I am still rather uneasy because while the patient allocation and statistical analysis of the trial appear to be done to a high standard, the intervention that was assessed is commercial and associated with a number of warning signs. The Lightning Process appears based on neurolinguistic programming, which, despite its scientific-sounding name, has long been recognised as pseudoscience.

....

“I noticed, for instance, that LP involves group sessions, whereas the comparison group undergoing standard medical care were treated individually. So it may be that the benefits derive from interacting with other children with chronic fatigue syndrome/ME, rather than the specific exercises and training. This is, of course, something that could be investigated in future research but meanwhile the concern is that this report will in effect act as positive publicity for a programme that is being proposed for a wide range of physical conditions (including chronic pain, low self-esteem, multiple sclerosis, and depression, to name just a few) and has to date been promoted largely through celebrity endorsements.”

Calling it psuedoscience and having concern it is just publicity for a commercial company...

 

[Alvin]

No surprise
Are we ready to accept the implications?

 

[Sly Saint]

The replication crisis and its zombies are a student’s worst nightmare

The Atlantic magazine recently published another article about the replication crisis, focusing on why disproven research is still studied. The fields of science are littered with what Steven Poole refers to as “intellectual zombies,” disproven ideas that refuse to die.

Graduate students are uniquely harmed by these unreplicatable experiments. While there are real fiscal and societal costs, there are also unseen personal costs. The “waste of 1000 studies” is a waste of many more years that graduate students spent trying to make these experiments work.

full blog here
https://massivesci.com/notes/replic...-zombie-ideas-graduate-student-mental-health/