Article: Highlight negative results to improve science

Discussion in 'Research methodology news and research' started by Andy, Oct 4, 2019.

  1. Andy

    Andy Committee Member

    Messages:
    22,308
    Location:
    Hampshire, UK
    https://www.nature.com/articles/d41586-019-02960-3
     
  2. rvallee

    rvallee Senior Member (Voting Rights)

    Messages:
    12,919
    Location:
    Canada
    Gonna have to work on step 0 first: do not allow negative results to be fraudulently manipulated into false positive ones.

    Most people would assume this is not possible in this day and age. Sadly, most people would be wrong about that.
     
  3. Barry

    Barry Senior Member (Voting Rights)

    Messages:
    8,386
    Only publishing 'positive' results is itself a form of selection bias. As absurd as publishing the UK °C temperatures, but discarding all negative values.

    Feels like there should be a very different publishing infrastructure in place. One where a trial's preregistration includes undertakings to publish - by the scientists, by their institutions, and by the publishers - ensuring results are published without fear or favour, wherever they fall. If this crucial step not followed, then the trial does not get funded. But probably a pipe dream.

    I fully appreciate that funders have hugely vested interests, and therefore this sort of bias is exactly what they expect and demand. If the trial they funded shows their new magic potion helps millions of people, then they obviously intend as many people to know about it. If it didn't, or worse if maimed half the people (thalidomide?) then they clearly want to squash it. The bottom line is their thing, not philanthropy.
     
    Annamaria, Cheshire, obeat and 3 others like this.
  4. Sean

    Sean Moderator Staff Member

    Messages:
    7,493
    Location:
    Australia
    a failed experiment

    An experiment is only a failure if it is not conducted rigorously. A negative result is not a failure in itself.
     
  5. Amw66

    Amw66 Senior Member (Voting Rights)

    Messages:
    6,515
    Negatives can be as, if not more, illuminating as positives.

    Often in design people know what they don't want rather than what they do. Often you can get to a solution by knowing what not to do. It's an elimination process as well as a creative one.

    The literature is seriously incomplete without honestly reported null results.

    How much funding could be garnered and / or targeted if we had a fuller picture.
     
  6. Peter Trewhitt

    Peter Trewhitt Senior Member (Voting Rights)

    Messages:
    3,814
    As information technology develops options for sharing information grows.

    Historically journals were by necessity selective, with, at their best, the aims of selecting the most informative and reliable studies progressing understanding in their specific field. However [as said] this can have the unintended side effects of focusing on positive results and on research within the current prevailing prejudices.

    This is not to say there is not still a place for high profile journals promoting by selective publication practices specific potentially important studies. However there is also the opportunity now for total blanket publication of all grant funded research, even those with negative or ambiguous results. It would be possible for academic institutions or specific research fields to publish on line write ups of all research undertaken that was grant funded or required ethical approval.

    Presumably the researchers/authors would at some point make a decision with specific research to go down the traditional journal route or to prepare write ups for this fall back blanket publication system. This system could note all prospectively registered research, grant awards or ethical approval requests, then either provide links to subsequent journal articles, host write ups of those studies that did not make the conventional journal publication pathways or provide statements about research that for whatever reason was not completed or not written up.

    Because of the costs involved, such a system might not be able to have any editorial oversight or peer review, so research published this way might require more critical examination than that going through the current journal system, but, as readers here will know, editorial oversight and peer review does not always guarantee unbiased or accurate reporting. However it would ensure access to negative or inconclusive results, that may be as important to understanding an issue as the more hyped positive findings.
     
    Last edited: Oct 5, 2019
    rvallee, Mithriel, Andy and 1 other person like this.
  7. Sean

    Sean Moderator Staff Member

    Messages:
    7,493
    Location:
    Australia
    IOW, the point of an experiment/observation is to get a clear (or clearer) result. It doesn't matter if the result is negative or positive, or if it confirms or refutes the hypothesis, as long as it is an accurate result.

    Clarity (removing or reducing ambiguity) in experimental/observational outcomes is the aim. Lack of clarity is the failure.

    Say there is indirect evidence that there may be a lake at X location. So you go to X, carefully survey the area, and find there is no lake.

    That negative result is not a failed experiment/observation, it is in fact a highly successful one, because it answered the question unambiguously.
     
  8. Amw66

    Amw66 Senior Member (Voting Rights)

    Messages:
    6,515
    How many unpublished trials are there for ME,?
    From their hypotheses/ subject matter what does this say?

    I had hoped Amalok Bansal' s glucocorticoid receptor trial would be published as the theory made a lot of sense from our experience.

    Then of course are BPS ones - non publication may have interesting indications.

    Is there a list of unpublished trials?
     
    rvallee and Andy like this.
  9. Andy

    Andy Committee Member

    Messages:
    22,308
    Location:
    Hampshire, UK
    No, not that I'm aware of.
     

Share This Page