Open data and the role of citizen scientists in ME/CFS research

Discussion in 'Other research methodology topics' started by Cheshire, Jan 21, 2018.

  1. Cheshire

    Cheshire Moderator Staff Member

    Messages:
    4,675
    This post has been copied and following discussion moved from:
    COFFI - The international collaborative on fatigue following infection

    p19:
     
    Last edited by a moderator: Oct 2, 2023
  2. Valentijn

    Valentijn Guest

    Messages:
    2,275
    Location:
    Netherlands
    Why restrict the anonymized data to "bona fide" researchers, when it was collected by unethical hacks with no respect for the scientific process? Surely if us filth-encrusted citizen-scientists get something wrong, the real scientists (not these ones) can set us straight

    Confidentiality of what, the conflicting results? The data itself is anonymized, so is not confidential.

    What's the excuse for prioritizing access to their buddies? Is it too difficult for them to email the same data to two different people?

    I'm still amazed anyone is willing to hire or fund these jokers.
     
    janice, Awol, Jan and 13 others like this.
  3. petrichor

    petrichor Senior Member (Voting Rights)

    Messages:
    321
    I personally think that's pretty fair. Almost every scientist would tell you that people without the necessary training and experience usually have very great overconfidence in their scientific skills and analytical ability, and when you have too many citizen scientists coming in with their own opinions and takes things can turn into a mess. There are now so many bona fide researchers that represent ME/CFS patients, that I don't have a problem as long as the data can make its way into their hands.
     
  4. Esther12

    Esther12 Senior Member (Voting Rights)

    Messages:
    4,393
    This was being run by Peter White. He, as a 'bona fide' researcher, will protect us from the overconfidence of a citizen scientists like Alem Matthees turning things into a mess?

    Who gets to decide who is a 'bona fide' researcher anyway?

    Most researchers don't get paid to pick apart the work of other researchers, and that's partly why it was citizen scientists who had to lead the way in pointing out the problems with White's work.
     
    Hutan, MSEsperanza, janice and 19 others like this.
  5. Snow Leopard

    Snow Leopard Senior Member (Voting Rights)

    Messages:
    3,853
    Location:
    Australia
    They say "bona fide" but we know they really mean people screened not to poke holes in their work. I bet if a researcher like Carolyn Wilshire, Jonathan Edwards etc tried to get ahold of the data, they'd simply come up with excuses.

    Their policy is very different from the policies of the "open science" movement.
     
    Hutan, inox, MSEsperanza and 17 others like this.
  6. Cheshire

    Cheshire Moderator Staff Member

    Messages:
    4,675
    Are you aware that the release of the PACE data which led to the reanalysis of the trial was achieved thanks to Allem Matthees, a patient? If we had had to wait for Sharpe and co to gently release the data to other scientists with different views, we'd be still waiting.
     
    Hutan, MSEsperanza, janice and 20 others like this.
  7. petrichor

    petrichor Senior Member (Voting Rights)

    Messages:
    321
    The greatest concern is whether they allow other researchers like Carolyn Wilshire, Jonathan Edwards, Ron Davis, Jose Montoya, Ian Lipkin, etc. to access the data. Most of those researchers would be quite concerned about the data in this collaboration. That, though, is a different question to whether they release the data to citizen scientists, which is less important, I think. If possible it's better for properly trained researchers familiar with the research community to lead the way. (You'd find that, although we can think citizen scientists are great, very few in the research community take them seriously.)

    The problem with PACE was different, which was that they weren't releasing the data to anyone at all. That's a different problem. The great success of Allem Mathees was getting them to release the data in the first place, not the fact that he allowed citizen scientists to access the data.

    So, as long as they release the data to other researchers, like the ones I listed, I don't think there is a problem.
     
  8. Cheshire

    Cheshire Moderator Staff Member

    Messages:
    4,675
    That's pretty ironic to defend academics in a disease where academics have failed us constantly.

    Of course, there are patients who think of themselves as scientific genius while talking BS. (I could also say that about a lot of the so-called experts that have been in charge of ME CFS these last decades.) But that is certainly not a reason to put all patients in that box, some were scientists before getting sick, others acquired a great and respectable knowledge afterwards. Everybody must be judged on their own merits.

    If you took the time to just dig a little bit into the history of ME/CFS, you'd find that nothing, ever, would have happened without these patients.
     
    Last edited: Jan 22, 2018
    Hutan, m.patriarca, janice and 18 others like this.
  9. petrichor

    petrichor Senior Member (Voting Rights)

    Messages:
    321
    We have good scientists representing us that can lead the way. If possible, it's far more effective for them to lead the way, rather than trying to do it ourselves. You're very right that patients play an extremely important role in making things happen, but that comes from activism and influencing researchers to do the right thing, rather than our scientific skills.
     
  10. Cheshire

    Cheshire Moderator Staff Member

    Messages:
    4,675
    Without patients' scientific skills the criticism of the BPS view would have been impossible.
     
    Hutan, janice, ScottTriGuy and 18 others like this.
  11. petrichor

    petrichor Senior Member (Voting Rights)

    Messages:
    321
    That came from activism and influencing researchers to do the right thing, not our scientific skills. And that's irrelevant now, because we've got lots of researchers that can represent us.
     
  12. Valentijn

    Valentijn Guest

    Messages:
    2,275
    Location:
    Netherlands
    My experience is that having a university degree has caused a great many researchers and clinicians to grossly overestimate their scientific skills and analytical ability. Those are skills which have to be actively learned, yet they are not learning them, at least in some degrees (psychology seems especially remiss). It's a huge problem which is resulting in a shocking amount of research that is completely useless.

    We are not coming in with "opinions". We are coming in with evaluations of the research, based on scientific principles which are at least nominally universal. To say that our contributions are lesser or problematic due to our lack of status is a logical fallacy: Appeal to Authority. If a patient is getting stuff wrong, other people (including patients) can point that out and discuss it. Ideas are not superior based on their source, and must be evaluated on their own merits.

    The data isn't making its way into their hands. As you'll notice in the anti-patient message you approve of, they are prioritizing access to people they work with. Since an email is as easily sent to 2 people as to 1, what they are really proposing is selective withholding of data from other researchers, especially those who they do not work with. Not-so-coincidentally, those are also the researchers who would be willing to point out flaws with the original interpretation of data.

    Is this your own special "bona fide" list? Because other researchers at universities have asked for data and been denied it. And perhaps we can directly ask @Carolyn Wilshire and @Jonathan Edwards what they think about limiting the access of that data to people like themselves.

    That approach resulted in diddly-squat happening for decades. It was patients' refusal to be inflicted with bad science which changed the situation. This was spear-headed by patients with no formal education such as Tom Kindlon and Alem Mathees. "Properly trained researchers" are involved now because our involvement paved the way for them. Why should we abandon an approach which is very effective in favor of an approach which was a complete failure?

    If a "researcher" judges content solely based on who produced it, who gives a damn what they think? They're obviously third-rate hacks who can only form a good opinion of themselves if they place all of their stock in their credentials. The amazing researchers working with the ME community have repeatedly said how impressed and inspired they are by what patients have managed to accomplish in the scientific realm :inlove: Those are the ones whose opinions I care about.

    They were releasing PACE data - to people they work with, and no one else. Their approach to PACE data is exactly what they want to apply to all of their data. Fortunately the courts and the vast majority of the scientific community are now adamantly opposed to hiding data, especially when it's publicly funded.

    And there's plenty of men out there ready to mansplain for us, so us little ladies can retire to the kitchen to make them dinner while they look after our civil rights for us like we asked them to :rolleyes:

    Even if we could completely depend upon other people to represent us, why should we? Those researchers and academics speak for themselves in the realm of science, though it usually is very helpful to us, but we should also be permitted to speak for ourselves. We have different brains, different thoughts, and different valid points to make.

    In summary, I think you're missing two very major points:
    The first is that science is not a private club. In fact, it's the opposite. It's used to make research accessible, understandable, and reproducible. It does not require a degree to develop a thorough comprehension of scientific methodology, not does having a degree guarantee such a comprehension. This is especially the case among doctors, who have a professional degree, not a scientific degree. People are not going to comprehend good research practices unless they deliberately learn about them - and everyone has the access to learn about them.

    The second point is that there is no excuse for hiding data. The alarmist claims from the PACE crew and their buddies are soundly rejected by the rest of the scientific community, as well as the judiciary. No one can be identified from anonymized data, and any poor or inappropriate interpretations of the data will be corrected by other patients or academics.
     
    Hutan, janice, ScottTriGuy and 21 others like this.
  13. Jonathan Edwards

    Jonathan Edwards Senior Member (Voting Rights)

    Messages:
    13,964
    Location:
    London, UK
    Sorry, but being in the middle of this I am quite clear that the patients have set the agenda. The work of Matthees and Kindlon is just as much science as that of Lipkin and Fluge. And it has been much more important in taking us forward. Most of the lab research out there is still second rate. It will only become first rate because of the patients. The international research community only came into existence because of IiME - run by carers and patients.

    The scientists have completely screwed up on this on their own.
    The bit about citizen scientists is specifically in there so that Peter White can protect himself from the truth.
    It took me a couple of years to learn the reality of this game. It is not pretty.
     
    Last edited: Jan 22, 2018
  14. Adrian

    Adrian Administrator Staff Member

    Messages:
    6,511
    Location:
    UK
    I would think of it very differently. Not everyone can just set themselves up as s 'citizen scientist' but many patients have a lot of scientific skills and experience some in the related subject and some in others. But that range of experience gives a perspective on the research. Having worked (and run) multi-disciplinary research projects there is something extra you get from considering a wide range of experiences and inputs.

    One issue that scientists (and many other groups) can have in closed communities is group think. And this is another reason it is so important to get a range of views and inputs. It seems to me that the community around CBT/GET in ME is really quite closed to the extent that they don't address issues than many others raise.

    In terms of data analysis there are many data sets that get released into the community to be looked at and examined. The stats and machine learning communities are always keen to see interesting data sets to test new techniques or look at the validity of the existing ones. Its a normal part of science these days.

    The problems that are coming are due to very tight control on data along with changes to protocols and the use of inappropriate statistics. That allows people like the PACE authors to manipulate data and when limited data was released this became blatantly obvious. In fact it was obvious that something was wrong just reading the papers.

    Maybe I have a different perspective because I work in a field (Computer security) where untrained people pop up and demonstrate vulnerabilities, attacks (and new defenses). They will often go for bug bounties or publish code so people can verify. But they do not get ignored because if they have found an attack then so may others so it gets looked at and fixed. It is a very different attitude where people are paid to find issues and no one thinks we can keep things out of the hands of others in the way some groups are trying to control data and hence the message.
     
  15. Adrian

    Adrian Administrator Staff Member

    Messages:
    6,511
    Location:
    UK
    Remember patients have scientific skills and could be researchers or have been researchers before they were ill. Patients are too often dismissed as an ignorant group as if once ill any previous skills are lost.
     
  16. petrichor

    petrichor Senior Member (Voting Rights)

    Messages:
    321
    Well, the points that everyone have made are reasonable. I still think that many patients greatly overestimate their scientific ability, and it's extremely important that patients themselves don't fall into group think, but I see the validity in everyone's points.

    If patients do want to have input on research, they need to make sure they hold themselves to the same scientific standards that they hold others to, that they don't fall into personally attacking people, or that they don't agree with particular arguments solely for the fact it supports their position. Those are the standards that the scientific community tries (and fails, very often, I know) to hold themselves to, and patients ought to try to hold themselves to those standards too.
     
  17. Jonathan Edwards

    Jonathan Edwards Senior Member (Voting Rights)

    Messages:
    13,964
    Location:
    London, UK
    I am afraid the 'scientific community' does not even try any more. Things are dire. Which is one of the reasons I retired at 60. Creative science has almost come to a halt. It has been replaced by self interest. Patients have an excuse for muddled thinking or emotional bias - scientists have no excuse.
     
    janice, inox, ScottTriGuy and 18 others like this.
  18. Adrian

    Adrian Administrator Staff Member

    Messages:
    6,511
    Location:
    UK
    I think many of us here are trying to hold ourselves to standards of carefully assessing methodology and understanding results. It's not just the CBT studies that get examined and criticized (but they are some of the weakest so get pulled apart the most).

    Researchers often have favored theories which they push because they believe in them. That is necessary as without it the momentum wouldn't develop to further the research but this is why it is important to listen and take on board criticism from other groups. That can lead to improvements in the work and hence better research.
     
  19. Snow Leopard

    Snow Leopard Senior Member (Voting Rights)

    Messages:
    3,853
    Location:
    Australia
    I disagree, it is about partnership. Nothing about us without us.
    A great example is Lenny Jason who has long been willing to listen to the community and even collaborate.

    Also, as an aside, a surprising number of people on this forum come from scientific backgrounds, careers of which have been limited or cut short due to illness.

    I've published multiple times in mainstream journals, had my pubmed comments pointing out errors "featured". Would you say I have some "scientific ability"?
     
  20. Liv aka Mrs Sowester

    Liv aka Mrs Sowester Senior Member (Voting Rights)

    Messages:
    1,201
    Yes, that is a very important point; ME doesn't discriminate, the 'gods in white coats' get it too. And when they do they are often horrified to realise how wrong they were.
     
    inox, janice, Dolphin and 13 others like this.

Share This Page