In combination with low power this leads to serious problems, see for example Button et al 2016 in Nature. These practices contribute on the whole to a bias in the literature through too many published false positive and too few published true negative results. Only statistically significant results, non-significant ones end up in the To experiment until a significant result turns up Results as the ones that were searched for in the first place The Results are Known): describing in hindsight statistically significant But it appears that science in general has room for improvement concerning reproducibility, see for example Baker 2016 in Nature.

These were much discussed in the media because they contain to a varying degree elements of fraud such cases are in reality rather rare. Impressive examples of misuse of data and of the massive damage this can inflict on science and society are the examples of Andrew Wakefield, Diedrik Stapel or Brian Wansink. That come back with significant results.” This isĭone by performing many statistical tests on the data and only reporting those So, what is p-hacking? Searching for p-hacking on Wikipedia leads to the article on data dredging that starts like this:ĭata snooping, data butchery, and p-hacking) is the misuse of data analysis toįind patterns in data that can be presented as statistically significant, thusĭramatically increasing and understating the risk of false positives. “Why Most Published Research Findings Are False”, Ioannidis 2005 in PLOS Medicine.“The scandal of poor medical research”, Altman 1994 in BMJ.“The Meaning of “Significance” for Different Types of Research”, de Groot 1956, originally in Dutch, here translated by E.J.But the awareness of the issue is not new, for example: P-hacking, among others also called data dredging or fishing for significance, is one of several questionable research practices, that is denounced and fought against increasingly in the last few years. Eva Furrer, Center for Reproducible Science, UZH

Data dredging in clinical trials