Editing Uncomfortable science

'''Uncomfortable science''', as identified by [[statistician]] [[John Tukey]],<ref>{{Cite journal | doi = 10.1038/msb.2011.70| title = The self-assessment trap: Can we all be better than average?| journal = Molecular Systems Biology| volume = 7| year = 2011| last1 = Norel | first1 = R. | last2 = Rice | first2 = J. J. | last3 = Stolovitzky | first3 = G. | pages=537 | pmid=21988833 | pmc=3261704}}</ref><ref>{{cite book |editor=Hoaglin, D.C |title=Exploring Data Tables Trends and Shapes |publisher=Wiley |isbn=0-471-09776-4 |quote=Much of science also falls under John Tukey's label "uncomfortable science," because real repetition is not feasible or practical. |display-editors=etal |url-access=registration |url=https://archive.org/details/exploringdatatab0000unse }}</ref> comprises situations in which there is a need to draw an [[statistical inference|inference]] from a limited [[sample (statistics)|sample]] of [[data]], where further samples influenced by the same [[cause system]] will not be available. More specifically, it involves the analysis of a finite natural phenomenon for which it is difficult to overcome the problem of using a common sample of [[data]] for both [[exploratory data analysis]] and [[confirmatory data analysis]]. This leads to the danger of [[systematic bias]] through [[testing hypotheses suggested by the data]].

A typical example is [[Titius-Bode law|Bode's law]], which provides a simple numerical rule for the distances of the [[planet]]s in the [[Solar System]] from the [[Sun]]. Once the rule has been derived, through the [[trial and error]] matching of various rules with the observed [[data]] (exploratory data analysis), there are not enough planets remaining for a rigorous and independent test of the [[hypothesis]] (confirmatory data analysis). We have exhausted the natural [[phenomenon|phenomena]]. The agreement between data and the numerical rule should be no surprise, as we have deliberately chosen the rule to match the data.  If we are concerned about what Bode's law tells us about the cause system of planetary distribution then we demand confirmation that will not be available until better information about other planetary systems becomes available.

==See also==
*[[Cosmic variance]] for an extreme example of this phenomenon
*[[Data mining]]

==Bibliography==
*{{cite book |last=Diaconis |first=P. |author-link=P. Diaconis |editor-last=Hoaglin |editor-first=D.C |chapter=Theories of data analysis: from magical thinking through classical statistics |title=Exploring Data Tables Trends and Shapes |publisher=Wiley |isbn=0-471-09776-4 |date=1985 |display-editors=etal |url-access=registration |url=https://archive.org/details/exploringdatatab0000unse }}

==References==
{{Reflist}}

[[Category:Philosophy of statistics]]
[[Category:Statistical hypothesis testing]]