1. Sign our petition calling on Cochrane to withdraw their review of Exercise Therapy for CFS here.
    Dismiss Notice
  2. Guest, the 'News in Brief' for the week beginning 18th March 2024 is here.
    Dismiss Notice
  3. Welcome! To read the Core Purpose and Values of our forum, click here.
    Dismiss Notice

Methodology over metrics: Current scientific standards are a disservice to patients and society, 2021, Van Calster et al

Discussion in 'Research methodology news and research' started by Andy, Jun 2, 2021.

  1. Andy

    Andy Committee Member

    Messages:
    21,810
    Location:
    Hampshire, UK
    Highlights
    • The overall quality of medical research remains poor, despite longstanding criticisms.
    • The scientific enterprise is business-like and consistently undervalues its own backbone, methodology.
    • Despite great initiatives to improve research quality, progress is modest.
    • Top-down action from journals, funding agencies, universities and governments is needed to break the cycle. These actions should give methodology a central place in funding acquisition as well as study design, conduct, and reporting.
    Abstract
    Covid-19 research made it painfully clear that the scandal of poor medical research, as denounced by Altman in 1994, persists today. The overall quality of medical research remains poor, despite longstanding criticisms. The problems are well known, but the research community fails to properly address them. We suggest most problems stem from an underlying paradox: although methodology is undeniably the backbone of qualitative and responsible research, science consistently undervalues methodology. The focus remains more on the destination (research claims and metrics) than on the journey. Notwithstanding, research should serve society more than the reputation of those involved. While we notice that many initiatives are being established to improve components of the research cycle, these initiatives are too disjointed. The overall system is monolithic and slow to adapt. We assert that a top-down action is needed from journals, universities, funders and governments to break the cycle and put methodology first. These actions should involve the widespread adoption of registered reports, balanced research funding between innovative, incremental and methodological research projects, full recognition and demystification of peer review, mandatory statistical review of reports, adherence to reporting guidelines, and investment in methodological education and research. Currently, the scientific enterprise is doing a major disservice to patients and society.

    Open access, https://www.jclinepi.com/article/S0895-4356(21)00170-0/fulltext
     
  2. Hutan

    Hutan Moderator Staff Member

    Messages:
    26,534
    Location:
    Aotearoa New Zealand
    Maybe Cochrane could lead the way by not using studies that have no scientific value in their reviews e.g. those relying on subjective outcomes in unblinded studies to prove a treatment works. Cochrane is a long way from being a leader for good science right now though. @Hilda Bastian
     
  3. Snowdrop

    Snowdrop Senior Member (Voting Rights)

    Messages:
    2,134
    Location:
    Canada
    I think I have mentioned this before. It is never ever an issue of just one study or even a group of studies.

    The problems mentioned above are systemic and are fatally injuring science in a way that will have horrifying repercussions that go beyond medicine. Why trust science at all if it can say anything you like, arrive at any destination you please for political or economic reasons.

    I realise that people involved in scientific endeavours will have much to say that they think is nuanced and well reasoned but there is a lot of sophistry and justification of compromise (to accommodate the realities of politics and economics probably) that do not serve to illuminate but to obfuscate.

    Society suffers.
     
  4. alktipping

    alktipping Senior Member (Voting Rights)

    Messages:
    1,197
    a necessary fix would require a government watchdog that has real teeth . Academic and scientific fraud should have real world legal consequences ie lengthy prison sentences and unlimited fines for those who refuse to do their due diligence in filtering out this blight . at present the whole industry seems to be run as some sort of club with zero accountability for the massive amount of harm caused .
     
  5. Sean

    Sean Moderator Staff Member

    Messages:
    7,045
    Location:
    Australia
    It is all about methodology. The rest is toxic crap.
     
    Michelle, Invisible Woman and MEMarge like this.
  6. ME/CFS Skeptic

    ME/CFS Skeptic Senior Member (Voting Rights)

    Messages:
    3,495
    Location:
    Belgium
    The 1994 paper by Altman that the authors refer to is also worth a read. it starts:
    Van Calster and colleagues comment on this paper, saying it "could have been written today, without changing a single word."
     
    Last edited: Jun 3, 2021
    Snow Leopard, Chezboo, Sean and 9 others like this.

Share This Page