Should research teams include some engineers?

[wigglethemouse]

Some of the discussion seems to be about how the engineering industry uses checks, tests, and reviews to make sure projects stay on track and are reliable. Does this seem to be missing in some medical research areas?

In Engineering I even had a project cancelled and re-specified which I thought was great rather than proceed and produce something that wouldn't sell well.

 

[jnmaciuch]

Some of the discussion seems to be about how the engineering industry uses checks, tests, and reviews to make sure projects stay on track and are reliable. Does this seem to be missing in some medical research areas?

Nope, in many cases you’re even required by the funding body to provide proof of progress and research plans to get the next installment of funding for the upcoming year. Not to mention department performance reviews, study sections for grant awards, IRB protocols, research ethics committees, peer review, peer feedback at conferences/poster sessions, and all the other places where your work is scrutinized.

I think it’s easy to assume, based on the presence of so much bad science, that these review mechanisms must not be as robust as in other fields. But honestly they're not any less comprehensive than the reviews all my close friends describe being employed in the tech industry. Even more stringent, in many ways, judging by the sheer amount of work to keep up with reporting requirements.

The realization I came to was just that most of the mechanisms any rational person can dream up to Fix Science often don’t seem to actually prevent bad science, and not for any one easily understandable reason.

 

[wigglethemouse]

So perhaps at the end of the day it comes down to people. There are a relatively large amount of not very competent people in every walk of life, or said another way, a bunch of very average people who we'd prefer they think a bit more about what they are doing and what it means.

The best bosses I've seen are the ones that can get to the real information / issue within 3 questions. Those people are pretty rare. Maybe we just don't have enough people asking the right questions, and the ones doing the work that can provide an answer to satisfy such a boss.

I'd love to have a Dr Maureen Hanson as a boss if I was a young Scientist. In fact I've heard other aspiring researchers say the same thing. Mentorship, direction,good decision making, and making sure I stay on track. Pretty rare.

Not to mention department performance reviews, study sections for grant awards, IRB protocols, research ethics committees, peer review, peer feedback at conferences/poster sessions, and all the other places where your work is scrutinized.

Yes, I've heard the same. In some cases perhaps too much bureaucracy preventing progress.

 

[jnmaciuch]

So perhaps at the end of the day it comes down to people. There are a relatively large amount of not very competent people in every walk of life, or said another way, a bunch of very average people who we'd prefer they think a bit more about what they are doing and what it means.

The best bosses I've seen are the ones that can get to the real information / issue within 3 questions. Those people are pretty rare. Maybe we just don't have enough people asking the right questions, and the ones doing the work that can provide an answer to satisfy such a boss.

Yup, I'm with you on that. It's also a thing that seems really hard to gauge for yourself. Tons of people delude themselves every day into thinking they're the pinnacle of rationality and efficiency, even manage to convince others that they are gods gift to critical thinking—scientists are probably especially prone to skewed self assessments.

I think you’re right on the money that it is a rare skill. Probably one that can be cultivated to some extent, but maybe not reliably.

 

[Hutan]

So perhaps at the end of the day it comes down to people. There are a relatively large amount of not very competent people in every walk of life, or said another way, a bunch of very average people who we'd prefer they think a bit more about what they are doing and what it means.

I guess this is it, really. It's true that a lot of medical research is surprisingly bad, even when it's not about ME/CFS. Incompetence, and also the bias created by the prospect of financial returns.

There are probably some ME/CFS specific issues too.

It is not a prestige area. If you are a bright medical scientist, there are questions to answer that will see you better paid, better resourced and better recognised, questions to do with cancer for example. Only those with a personal connection to the disease or at least a good understanding of the need are likely to plow on with ME/CFS regardless. On average, our researchers probably aren't as good as the ones addressing more prestigious diseases. I think that also probably applies a bit to our funders and charities. In many cases, it's just whoever puts their hand up to do the (volunteer/poorly paid/low status) work.

Money to some extent could counter the lack of prestige - but, we don't have a lot of that, as ME/CFS hits young people of working age, and hits them with physical incapacity and shame-inducing stigma. People with ME/CFS and their families have tended to lose any power - financial, influential - that they had. People like many here at S4ME who call for things to be better tend to struggle to gain decision-making roles, even within our own charities. We don't have the well-resourced institutions like the cancer research institutes.

There is also prejudice. That affects the questions ME/CFS researchers want to answer, and to a large extent get paid to answer - e.g. Is it perfectionism? Will a mindfulness course fix it? How can we implicate cortisol in the pathology? Why don't we measure cytokines in blood again?

Sorry, off topic. To answer the question - yes, engineers might be useful for some research questions. But, there are skills and personal qualities more important than what degree the person did when they were young.

 

[poetinsf]

I think that would be useful. Immunology and neurology in particular are control systems in some ways, sometimes complete with feedback loop. I'm sure people in those fields have the knowledge of systems, if not a formal training in systems engineering. But medical people are often so focused on the molecular level that they may fail to see the obvious. Hence they propose theories like muscle anomaly or virus or energy production dysfunction etc. for ME/CFS without ever explaining PEM.

Delayed response to, and recovery from, exertion is everywhere. Why not take the existing process, fit it to ME/CFS and see if it explains/predicts? This is one place they could use engineer's perspective, at macro level. If the model works, then a theory can be formed in accordance with the correspondence principle. Further biological research could follow to confirm.

 

[Hoopoe]

I think it’s easy to assume, based on the presence of so much bad science, that these review mechanisms must not be as robust as in other fields. But honestly they're not any less comprehensive than the reviews all my close friends describe being employed in the tech industry. Even more stringent, in many ways, judging by the sheer amount of work to keep up with reporting requirements.

My views have been influenced by the PACE trial, the difficulty replicating various findings in ME/CFS, by how widespread manipulation is in research, and the rigidity and inefficiency of the world of scientific publishing. I'm looking at these things from the outside and thinking there are big problems.

I think that emphasis on reliability would make research more useful for humanity. Not because it leads to better hypotheses, more creativity or deeper insight, but because it makes the results more reliably true and more replicable. This should ultimately reduce waste.

Bioinformatics is closer to the IT sector than other sciences and I suspect it has adopted some of its culture.