Editing Experiment (section)

==Observational studies {{anchor|Contrast with observational study}}==
[[File:Blackbox3D-obs.png|thumb|The [[black box|black box model]] for observation (input and output are ''observables''). When there are a [[feedback]] with some observer's control, as illustrated, the observation is also an experiment.]]

An [[observational study]] is used when it is impractical, unethical, cost-prohibitive (or otherwise inefficient) to fit a physical or social system into a laboratory setting, to completely control confounding factors, or to apply random assignment. It can also be used when confounding factors are either limited or known well enough to analyze the data in light of them (though this may be rare when social phenomena are under examination). For an observational science to be valid, the experimenter must know and account for [[confounding]] factors. In these situations, observational studies have value because they often suggest hypotheses that can be tested with randomized experiments or by collecting fresh data.{{cn|date=April 2025}}

Fundamentally, however, observational studies are not experiments. By definition, observational studies lack the manipulation required for [[Baconian method|Baconian experiments]]. In addition, observational studies (e.g., in biological or social systems) often involve variables that are difficult to quantify or control. Observational studies are limited because they lack the statistical properties of randomized experiments. In a randomized experiment, the method of randomization specified in the experimental protocol guides the statistical analysis, which is usually specified also by the experimental protocol.<ref name="Hinkelmann, Klaus and Kempthorne, Oscar 2008">{{cite book
|last1=Hinkelmann|first1= Klaus |author-link2=Oscar Kempthorne |last2=Kempthorne |first2=Oscar
|year=2008
|title=Design and Analysis of Experiments, Volume I: Introduction to Experimental Design
|edition= Second
|publisher=Wiley
|isbn=978-0-471-72756-9
}}</ref> Without a statistical model that reflects an objective randomization, the statistical analysis relies on a subjective model.<ref name="Hinkelmann, Klaus and Kempthorne, Oscar 2008"/> Inferences from subjective models are unreliable in theory and practice.<ref>{{cite book|last1=Freedman|first1=David|last2=Pisani|first2=Robert|last3=Purves|first3=Roger|author1-link=David A. Freedman|title=Statistics|date=2007|publisher=Norton|location=New York|isbn=978-0-393-92972-0|edition= 4th}}</ref> In fact, there are several cases where carefully conducted observational studies consistently give wrong results, that is, where the results of the observational studies are inconsistent and also differ from the results of experiments. For example, epidemiological studies of colon cancer consistently show beneficial correlations with broccoli consumption, while experiments find no benefit.<ref>{{cite book|last1=Freedman|first1=David A.|title=Statistical models : theory and practice|date=2009|publisher=Cambridge University Press|location=Cambridge|isbn=978-0-521-74385-3|edition= Revised}}</ref>

A particular problem with observational studies involving human subjects is the great difficulty attaining fair comparisons between treatments (or exposures), because such studies are prone to [[selection bias]], and groups receiving different treatments (exposures) may differ greatly according to their covariates (age, height, weight, medications, exercise, nutritional status, ethnicity, family medical history, etc.). In contrast, randomization implies that for each covariate, the mean for each group is expected to be the same. For any randomized trial, some variation from the mean is expected, of course, but the randomization ensures that the experimental groups have mean values that are close, due to the [[central limit theorem]] and [[Markov's inequality]]. With inadequate randomization or low sample size, the systematic variation in covariates between the treatment groups (or exposure groups) makes it difficult to separate the effect of the treatment (exposure) from the effects of the other covariates, most of which have not been measured. The mathematical models used to analyze such data must consider each differing covariate (if measured), and results are not meaningful if a covariate is neither randomized nor included in the model.

To avoid conditions that render an experiment far less useful, physicians conducting medical trials—say for U.S. [[Food and Drug Administration]] approval—quantify and randomize the covariates that can be identified. Researchers attempt to reduce the biases of observational studies with [[matching (statistics)|matching]] methods such as [[propensity score matching]], which require large populations of subjects and extensive information on covariates. However, propensity score matching is no longer recommended as a technique because it can increase, rather than decrease, bias.<ref>{{Cite journal|last1=King|first1=Gary|last2=Nielsen|first2=Richard|date=October 2019|title=Why Propensity Scores Should Not Be Used for Matching|journal=Political Analysis|language=en|volume=27|issue=4|pages=435–454|doi=10.1017/pan.2019.11|hdl=1721.1/128459|issn=1047-1987|doi-access=free|hdl-access=free}}</ref> Outcomes are also quantified when possible (bone density, the amount of some cell or substance in the blood, physical strength or endurance, etc.) and not based on a subject's or a professional observer's opinion. In this way, the design of an observational study can render the results more objective and therefore, more convincing.