Editorial Comment: The Proposal to Lower P Value Thresholds to .005, 2018, Ioannidis

No idea if this is of interest to anybody, all way over my head but I know some people here like to discuss P values.

https://jamanetwork.com/journals/jama/fullarticle/2676503

 

Only a poor tradesman blames their tools (in this case p-value).

The problem is people making poor generalisations based on limited evidence, not the alpha itself.

 

I wonder how far this idea will be taken. I can see circumstances where it would make life a lot harder for patients. This whole post relies on a very poor understanding of statistics (by me).

Imagine the following scenario...

1) A blood testing laboratory re-evaluates its reference range for serum vitamin B12 (for example). Their current range, based on subjects who are assumed to be healthy, is 200 - 800 ng/L. When looking at the entire result set for their subjects they discarded the lowest and highest 2.5% of the values, leaving the range they currently have, which is the middle 95% of the results. One can think of the reference range as a hypothesis. The null hypothesis is that "normal" serum vitamin B12 is in the range 200 - 800 ng/L and that a patient with a level in this range is healthy. The alternative hypothesis is that the patient's B12 level is not healthy. The alternative hypothesis is proven when the patient's B12 is less than 200 or higher than 800.

2) When re-evaluating, they test the level of serum vitamin B12 again, and this time they only exclude the top and bottom 1% of the results to give a range which covers the middle 99% of their (supposedly) healthy subjects. Surely this would mean that people with extremely low B12 levels would be classed as healthy?

I don't know if this is how reference ranges for all possible results are actually set. I would hope that clinical experience would be useful as well. But it does mean that modern medicine (theoretically) always believes that 95% of people are healthy. An obvious example of when this fails is when looking at statistics for body weight. There are lots more than 2.5% of the population who are overweight.

Edit : Clarification