Editing Instrumental variables estimation

{{Short description|Technique in statistics}}
In [[statistics]], [[econometrics]], [[epidemiology]] and related disciplines, the method of '''instrumental variables''' ('''IV''') is used to estimate [[Causal inference|causal relationships]] when [[controlled experiment]]s are not feasible or when a treatment is not successfully delivered to every unit in a randomized experiment.<ref name=Imbens:00>{{cite journal |last1=Imbens |first1=G. |first2=J. |last2=Angrist |year=1994 |title=Identification and estimation of local average treatment effects |journal=[[Econometrica]] |volume=62 |issue=2 |pages=467–476 |jstor=2951620 |doi=10.2307/2951620 |s2cid=153123153 |url=https://www.nber.org/papers/t0118 }}</ref> Intuitively, IVs are used when an explanatory variable of interest is correlated with the error term (endogenous), in which case [[ordinary least squares]] and [[ANOVA]] give [[bias (statistics)|biased]] results. A valid instrument induces changes in the explanatory variable (is correlated with the endogenous variable) but has no independent effect on the dependent variable and is not correlated with the error term,  allowing a researcher to uncover the causal effect of the explanatory variable on the dependent variable.

Instrumental variable methods allow for [[consistent estimator|consistent]] estimation when the [[dependent and independent variables|explanatory variables]] (covariates) are [[correlation|correlated]] with the [[errors and residuals in statistics|error terms]] in a [[regression analysis|regression]] model. Such correlation may occur when:
# changes in the dependent variable change the value of at least one of the [[covariate]]s ("reverse" causation),
# there are [[Omitted-variable bias|omitted variables]] that affect both the dependent and explanatory variables, or
# the [[Errors-in-variables models|covariates are subject to measurement error]].
Explanatory variables that suffer from one or more of these issues in the context of a regression are sometimes referred to as [[endogeneity (econometrics)|endogenous]]. In this situation, [[ordinary least squares]] produces biased and inconsistent estimates.<ref name=Bullock:00>{{cite journal |last1=Bullock |first1=J. G. |last2=Green |first2=D. P. |last3=Ha |first3=S. E. |year=2010 |title=Yes, But What's the Mechanism? (Don't Expect an Easy Answer) |journal=[[Journal of Personality and Social Psychology]] |volume=98 |issue=4 |pages=550–558 |doi=10.1037/a0018933 |pmid=20307128 |citeseerx=10.1.1.169.5465 |s2cid=7913867 }}</ref> However, if an ''instrument'' is available, consistent estimates may still be obtained. An instrument is a variable that does not itself belong in the explanatory equation but is correlated with the [[endogeneity (econometrics)|endogenous]] explanatory variables, conditionally on the value of other covariates.

In linear models, there are two main requirements for using IVs:
* The instrument must be correlated with the endogenous explanatory variables, conditionally on the other covariates. If this correlation is strong, then the instrument is said to have a '''strong first stage'''. A weak correlation may provide misleading inferences about parameter estimates and standard errors.<ref>https://www.stata.com/meeting/5nasug/wiv.pdf{{full|date=June 2021}}</ref><ref>{{Cite journal|last=Nichols|first=Austin|date=2006-07-23|title=Weak Instruments: An Overview and New Techniques|url=https://econpapers.repec.org/paper/bocasug06/3.htm}}</ref>
* The instrument cannot be correlated with the error term in the explanatory equation, conditionally on the other covariates. In other words, the instrument cannot suffer from the same problem as the original predicting variable. If this condition is met, then the instrument is said to satisfy the '''exclusion restriction'''.

== Example ==
Informally, in attempting to estimate the causal effect of some variable ''X'' ("covariate" or "explanatory variable") on another ''Y'' ("dependent variable"), an ''instrument'' is a third variable ''Z'' which affects ''Y'' only through its effect on&nbsp;''X''.  

For example, suppose a researcher wishes to estimate the causal effect of smoking (''X'') on general health (''Y'').<ref name="Leigh:00">{{cite journal |last1=Leigh |first1=J. P. |last2=Schembri |first2=M. |year=2004 |title=Instrumental Variables Technique: Cigarette Price Provided Better Estimate of Effects of Smoking on SF-12 |journal=[[Journal of Clinical Epidemiology]] |volume=57 |issue=3 |pages=284–293 |doi=10.1016/j.jclinepi.2003.08.006 |pmid=15066689}}</ref>  Correlation between smoking and health does not imply that smoking causes poor health because other variables, such as depression, may affect both health and smoking, or because health may affect smoking.  It is not possible to conduct controlled experiments on smoking status in the general population.  The researcher may attempt to estimate the causal effect of smoking on health from observational data by using the tax rate for tobacco products (''Z'') as an instrument for smoking.  The tax rate for tobacco products is a reasonable choice for an instrument because the researcher assumes that it can only be correlated with health through its effect on smoking.  If the researcher then finds tobacco taxes and state of health to be correlated, this may be viewed as evidence that smoking causes changes in health.

==History==
The first use of an instrument variable occurred in a 1928 book by [[Philip Green Wright|Philip G. Wright]], best known for his excellent description of the production, transport and sale of vegetable and animal oils in the early 1900s in the United States.<ref>{{cite journal |last1=Epstein |first1=Roy J. |title=The Fall of OLS in Structural Estimation |journal=Oxford Economic Papers |date=1989 |volume=41 |issue=1 |pages=94–107 |doi=10.1093/oxfordjournals.oep.a041930 |jstor=2663184 }}</ref><ref name="stock:trebbi03">{{cite journal |last1=Stock |first1=James H. |last2=Trebbi |first2=Francesco  |year=2003 |title=Retrospectives: Who Invented Instrumental Variable Regression?  |journal=[[Journal of Economic Perspectives]] |volume=17 |issue=3 |pages=177–194 |doi=10.1257/089533003769204416 |doi-access=free }}<!-- ref is helpful because the authorship of the appendix B was disputed and this references attempts to settle the dispute --></ref> In 1945, [[Olav Reiersøl]] applied the same approach in the context of [[errors-in-variables models]] in his dissertation, giving the method its name.<ref>{{cite book |first=Olav |last=Reiersøl |title=Confluence Analysis by Means of Instrumental Sets of Variables |series=Arkiv for Mathematic, Astronomi, och Fysik |volume=32A |year=1945 |location=Uppsala |publisher=Almquist & Wiksells |oclc=793451601 }}</ref>

Wright attempted to determine the supply and demand for butter using [[panel data]] on prices and quantities sold in the United States. The idea was that a regression analysis could produce a demand or supply curve because they are formed by the path between prices and quantities demanded or supplied. The problem was that the observational data did not form a demand or supply curve as such, but rather a cloud of point observations that took different shapes under varying market conditions. It seemed that making deductions from the data remained elusive.

The problem was that price affected both supply and demand so that a function describing only one of the two could not be constructed directly from the observational data. Wright correctly concluded that he needed a variable that correlated with either demand or supply but not both – that is, an instrumental variable.

After much deliberation, Wright decided to use regional rainfall as his instrumental variable: he concluded that rainfall affected grass production and hence milk production and ultimately butter supply, but not butter demand. In this way he was able to construct a regression equation with only the instrumental variable of price and supply.<ref name="Wool">Wooldridge, J.: ''Introductory Econometrics''. South-Western, Scarborough, Kanada, 2009.</ref>

Formal definitions of instrumental variables, using counterfactuals and graphical criteria, were given by [[Judea Pearl]] in 2000.<ref name=Pearl:00/>  [[Joshua Angrist|Angrist]] and [[Alan Krueger|Krueger]] (2001) present a survey of the history and uses of instrumental variable techniques.<ref name="angrist:00">{{cite journal |last1=Angrist |first1=J. |last2=Krueger |first2=A. |year=2001 |title=Instrumental Variables and the Search for Identification: From Supply and Demand to Natural Experiments |journal=[[Journal of Economic Perspectives]] |volume=15 |issue=4 |pages=69–85 |doi=10.1257/jep.15.4.69 |doi-access=free|hdl=1721.1/63775 |hdl-access=free }}</ref> Notions of causality in econometrics, and their relationship with instrumental variables and other methods, are discussed by [[James Heckman|Heckman]] (2008).<ref name=Heckman:00>{{cite journal |last=Heckman|first=J. |year=2008 |title=Econometric Causality |journal=[[International Statistical Review]] |volume=76 |issue=1 |pages=1–27 |doi=10.1111/j.1751-5823.2007.00024.x }}</ref>

== Theory==
While the ideas behind IV extend to a broad class of models, a very common context for IV is in [[linear regression]]. Traditionally,<ref name=bowden:turkington84>{{cite book|last1=Bowden|first1=R.J.|last2=Turkington|first2=D.A.|title=Instrumental Variables|date=1984|publisher=Cambridge University Press|location=Cambridge, England}}</ref> an instrumental variable is defined
as a variable <math>Z</math> that is correlated with the independent variable 
<math>X</math> and uncorrelated with the "error term" U in the linear equation

: <math>Y = X \beta + U </math>

<math>Y</math> is a vector. <math>X</math> is a matrix, usually with a column of ones and perhaps with additional columns for other covariates. Consider how an instrument allows <math>\beta</math> to be recovered. Recall that [[ordinary least squares|OLS]] solves for <math> \widehat{\beta }</math> such that <math>\operatorname{cov}(X,\widehat U) = 0</math> (when we minimize the sum of squared errors, <math>\min_\beta (Y- X\beta)'(Y- X\beta) </math>, the first-order condition is exactly <math> X' (Y- X\widehat{\beta}) = X' \widehat{U} = 0 </math>.) If the true model is believed to have <math>\operatorname{cov}(X,U) \neq 0</math> due to any of the reasons listed above—for example, if there is an [[Omitted-variable bias|omitted variable]] which affects both <math>X</math> and <math>Y</math> separately—then this [[ordinary least squares|OLS]] procedure will ''not'' yield the causal impact of <math>X</math> on <math>Y</math>. OLS will simply pick the parameter that makes the resulting errors appear uncorrelated with <math>X</math>.

Consider for simplicity the single-variable case. Suppose we are considering a regression with one variable and a constant (perhaps no other covariates are necessary, or perhaps we have [[Frisch–Waugh–Lovell theorem|partialed out]] any other relevant covariates):

:<math>y=\alpha + \beta x + u</math>

In this case, the coefficient on the regressor of interest is given by <math> \widehat{\beta }= \frac{\operatorname{cov}(x,y)}{\operatorname{var}(x)} </math>. Substituting for <math>y</math> gives

: <math>
\begin{align}
\widehat{\beta} & = \frac{\operatorname{cov}(x,y)}{\operatorname{var}(x)} = \frac{\operatorname{cov}(x,\alpha + \beta x + u)}{\operatorname{var}(x)} \\[6pt]
& =\frac{\operatorname{cov}(x, \alpha +\beta x)}{\operatorname{var}(x)} +\frac{\operatorname{cov}(x,u)}{\operatorname{var}(x)}= \beta^* + \frac{\operatorname{cov}(x,u)}{\operatorname{var}(x)},
\end{align}
</math>

where <math>\beta^*</math> is what the estimated coefficient vector would be if 
<math>\operatorname{cov}(x,u) = 0</math>. In this case, it can be shown that <math>\beta^*</math> is an unbiased estimator of <math>\beta</math>.
If <math>\operatorname{cov}(x,u) \neq 0</math> in the underlying model that we believe, then [[ordinary least squares|OLS]] gives an inconsistent estimate which does ''not'' reflect the underlying causal effect of interest. IV helps to fix this problem by identifying the parameters <math>{\beta}</math> not based on whether <math>x</math> is uncorrelated with <math>u</math>, but based on whether another variable <math>z</math> is uncorrelated with <math>u</math>. If theory suggests that <math>z</math> is related to <math>x</math> (the first stage) but uncorrelated with <math>u</math> (the exclusion restriction), then IV may identify the causal parameter of interest where OLS fails. Because there are multiple specific ways of using and deriving IV estimators even in just the linear case (IV, 2SLS, GMM), we save further discussion for the [[#Estimation|Estimation]] section below.

==Graphical definition==
IV techniques have been developed among a much broader class of non-linear models. General definitions of instrumental variables, using counterfactual and graphical formalism, were given by Pearl (2000; p.&nbsp;248).<ref name=Pearl:00>{{cite book |last=Pearl |first=J.|title=Causality: Models, Reasoning, and Inference |url=https://archive.org/details/causalitymodelsr0000pear |url-access=registration |publisher=[[Cambridge University Press]] |year=2000 |location=New York |isbn=978-0-521-89560-6}}</ref> The graphical definition requires that ''Z'' satisfy the following conditions:

: <math>(Z \perp\!\!\!\perp Y)_{G_{\overline{X}}} \qquad(Z \not\!\!{\perp\!\!\!\perp} X)_G </math>

where <math>\perp\!\!\!\perp</math> stands for [[Bayesian network#d-separation|''d''-separation]] and <math>G_{\overline{X}}</math> stands for the [[Bayesian network|graph]] in which all arrows entering ''X'' are cut off.

The counterfactual definition requires that ''Z'' satisfies

: <math>(Z \perp\!\!\!\perp Y_x)\qquad (Z \not\!\!{\perp\!\!\!\perp} X)</math>

where ''Y''<sub>''x''</sub> stands for the value that ''Y'' would attain had ''X'' been ''x'' and <math>\perp\!\!\!\perp</math> stands for independence.

If there are additional covariates ''W'' then the above definitions are modified so that ''Z'' qualifies as an instrument if the given criteria hold conditional on ''W''.

The essence of Pearl's definition is:
# The equations of interest are "structural", not "regression".
# The error term ''U'' stands for all exogenous factors that affect ''Y'' when ''X'' is held constant.
# The instrument ''Z'' should be independent of ''U''.
# The instrument ''Z'' should not affect ''Y'' when ''X'' is held constant (exclusion restriction).
# The instrument ''Z'' should not be independent of ''X''.

These conditions do not rely on specific functional
form of the equations and are applicable therefore to
nonlinear equations, where ''U'' can be non-additive
(see Non-parametric analysis). They are also applicable to a system of multiple
equations, in which ''X'' (and other factors) affect ''Y'' through
several intermediate variables. An instrumental variable need not be
a cause of ''X''; a proxy of such cause may also be
used, if it satisfies conditions 1–5.<ref name=Pearl:00/> The exclusion restriction (condition 4) is redundant; it follows from conditions 2 and 3.

===Selecting suitable instruments===

Since ''U'' is unobserved, the requirement that ''Z'' be independent of ''U'' cannot be inferred from data and must instead be determined from the model structure, i.e., the data-generating process. [[Causal graphs]] are a representation of this structure, and the graphical definition given above can be used to quickly determine whether a variable ''Z'' qualifies as an instrumental variable given a set of covariates ''W''. To see how, consider the following example.

Suppose that we wish to estimate the effect of a university tutoring program on grade point average ([[Grading in education|GPA]]).  The relationship between attending the tutoring program and GPA may be confounded by a number of factors.  Students who attend the tutoring program may care more about their grades or may be struggling with their work.  This confounding is depicted in the Figures 1–3 on the right through the bidirected arc between Tutoring Program and GPA.  If students are assigned to dormitories at random, the proximity of the student's dorm to the tutoring program is a natural candidate for being an instrumental variable.

<gallery mode="packed">
File:Instrumental Variable Example Effect of Tutoring 1.png|Figure 1: Proximity qualifies as an instrumental variable given Library Hours
File:Instrumental Variable Example Effect of Tutoring (Edge Deleted)_2.png|Figure 2: <math>G_{\overline{X}}</math>, which is used to determine whether Proximity is an instrumental variable.
File:Instrumental Variable Example Effect of Tutoring 2.png|Figure 3: Proximity does not qualify as an instrumental variable given Library Hours
File:Instrumental Variable Example Effect of Tutoring 3.png|Figure 4: Proximity qualifies as an instrumental variable, as long as we do not include Library Hours as a covariate.
</gallery>

However, what if the tutoring program is located in the college library?  In that case, Proximity may also cause students to spend more time at the library, which in turn improves their GPA (see Figure 1).  Using the causal graph depicted in the Figure 2, we see that Proximity does not qualify as an instrumental variable because it is connected to GPA through the path Proximity <math> \rightarrow </math> Library Hours <math> \rightarrow </math>  GPA in <math>G_{\overline{X}}</math>.  However, if we control for Library Hours by adding it as a covariate then Proximity becomes an instrumental variable, since Proximity is separated from GPA given Library Hours in <math>G_{\overline{X}}</math>{{Citation needed|date=April 2019}}.

Now, suppose that we notice that a student's "natural ability" affects his or her number of hours in the library as well as his or her GPA, as in Figure 3.  Using the causal graph, we see that Library Hours is a collider and conditioning on it opens the path Proximity <math>\rightarrow</math> Library Hours <math>\leftrightarrow</math> GPA.  As a result, Proximity cannot be used as an instrumental variable.

Finally, suppose that Library Hours does not actually affect GPA because students who do not study in the library simply study elsewhere, as in Figure 4.  In this case, controlling for Library Hours still opens a spurious path from Proximity to GPA.  However, if we do not control for Library Hours and remove it as a covariate then Proximity can again be used an instrumental variable.

==Estimation==

We now revisit and expand upon the mechanics of IV in greater detail. Suppose the data are generated by a process of the form

: <math> y_i = X_i \beta + e_i, </math>

where
* ''i'' indexes observations,
* <math>y_i</math> is the ''i''-th value of the dependent variable,
* <math>X_i</math> is a vector of the ''i''-th values of the independent variable(s) and a constant,
* <math>e_i</math> is the ''i''-th value of an unobserved error term representing all causes of <math>y_i</math> other than <math>X_i</math>, and
* <math>\beta</math> is an unobserved parameter vector.

The parameter vector <math>\beta</math> is the causal effect on <math>y_i</math> of a one unit change in each element of <math>X_i</math>, holding all other causes of <math>y_i</math> constant.  The econometric goal is to estimate <math>\beta</math>.  For simplicity's sake assume the draws of ''e'' are uncorrelated and that they are drawn from distributions with the same [[variance]] (that is, that the errors are serially uncorrelated and [[homoskedastic]]).

Suppose also that a regression model of nominally the same form is proposed.  Given a random sample of ''T'' observations from this process, the [[least squares|ordinary least squares]] estimator is

: <math> \widehat{\beta}_\mathrm{OLS} = (X^\mathrm T X)^{-1} X^\mathrm T y = (X^\mathrm T X)^{-1} X^\mathrm T (X \beta + e) = \beta + (X^\mathrm T X)^{-1} X^\mathrm T e</math>

where ''X'', ''y'' and ''e'' denote column vectors of length ''T''. This equation is similar to the equation involving <math> \operatorname{cov}(X,y) </math> in the introduction (this is the matrix version of that equation). When ''X'' and ''e'' are [[correlation|uncorrelated]], under certain regularity conditions the second term has an expected value conditional on ''X'' of zero and converges to zero in the limit, so the estimator is [[estimator bias|unbiased]] and consistent.  When ''X'' and the other unmeasured, causal variables collapsed into the ''e'' term are correlated, however, the OLS estimator is generally biased and inconsistent for&nbsp;''β''.  In this case, it is valid to use the estimates to predict values of ''y'' given values of ''X'', but the estimate does not recover the causal effect of ''X'' on&nbsp;''y''.

To recover the underlying parameter <math> \beta </math>, we introduce a set of variables ''Z'' that is highly correlated with each [[endogeneity (econometrics)|endogenous]] component of ''X'' but (in our underlying model) is not correlated with&nbsp;''e''. For simplicity, one might consider ''X'' to be a ''T'' × 2  matrix composed of a column of constants and one endogenous variable, and ''Z'' to be a ''T'' × 2 consisting of a column of constants and one instrumental variable. However, this technique generalizes to ''X'' being a matrix of a constant and, say, 5 endogenous variables, with ''Z'' being a matrix composed of a constant and 5 instruments. In the discussion that follows, we will assume that ''X'' is a ''T'' × ''K'' matrix and leave this value ''K'' unspecified. An estimator in which ''X'' and ''Z'' are both ''T'' × ''K'' matrices is referred to as [[Identifiability|just-identified]] .

Suppose that the relationship between each endogenous component ''x''<sub>''i''</sub> and the instruments is given by

: <math> x_i = Z_i \gamma + v_i, </math>

The most common IV specification uses the following estimator:

:<math> \widehat{\beta}_\mathrm{IV} = (Z^\mathrm T X)^{-1} Z^\mathrm T y </math>

This specification approaches the true parameter as the sample gets large, so long as <math> Z^\mathrm T e = 0 </math> in the true model:

:<math> \widehat{\beta}_\mathrm{IV} = (Z^\mathrm T X)^{-1} Z^\mathrm T y = (Z^\mathrm T X)^{-1} Z^\mathrm T X \beta + (Z^\mathrm T X)^{-1} Z^\mathrm T e  \rightarrow \beta    </math>

As long as <math> Z^\mathrm T e = 0 </math> in the underlying process which generates the data, the appropriate use of the IV estimator will identify this parameter. This works because IV solves for the unique parameter that satisfies <math> Z^\mathrm T e = 0 </math>, and therefore hones in on the true underlying parameter as the sample size grows.

Now an extension: suppose that there are more instruments than there are covariates in the equation of interest, so that ''Z'' is a ''T × M'' matrix with ''M > K''. This is often called the '''over-identified''' case. In this case, the [[generalized method of moments]] (GMM) can be used. The GMM IV estimator is

:<math> \widehat{\beta}_\mathrm{GMM} = (X^\mathrm T P_Z X)^{-1}X^\mathrm T P_Z y,</math>

where <math>P_Z</math> refers to the [[projection matrix]] <math>P_Z=Z(Z^\mathrm T Z)^{-1}Z^\mathrm T</math>.

This expression collapses to the first when the number of instruments is equal to the number of covariates in the equation of interest. The over-identified IV is therefore a generalization of the just-identified IV.

{{hidden begin|style=border:1px solid #aaa|title={{center|Proof that  β<sub>GMM</sub> collapses to β<sub>IV</sub> in the just-identified case}}}}
Developing the <math>\beta_\text{GMM}</math> expression:
:<math> \widehat{\beta}_\mathrm{GMM} = (X^\mathrm{T} Z(Z^\mathrm{T} Z)^{-1}Z^\mathrm{T} X)^{-1}X^\mathrm{T} Z(Z^\mathrm{T} Z)^{-1}Z^\mathrm{T} y</math>

In the just-identified case, we have as many instruments as covariates, so that the dimension of ''X'' is the same as that of&nbsp;''Z''. Hence, <math>X^\mathrm{T} Z, Z^\mathrm{T} Z</math> and <math>Z^\mathrm{T}X</math> are all squared matrices of the same dimension. We can expand the inverse, using the fact that, for any invertible ''n''-by-''n'' matrices '''A''' and '''B''', ('''AB''')<sup>−1</sup> = '''B'''<sup>−1</sup>'''A'''<sup>−1</sup> (see [[Invertible matrix#Properties]]):
:<math>
\begin{align}
\widehat{\beta}_\mathrm{GMM} &= (Z^\mathrm{T} X)^{-1}(Z^\mathrm{T} Z)(X^\mathrm{T} Z)^{-1}X^\mathrm{T} Z(Z^\mathrm{T} Z)^{-1}Z^\mathrm{T} y\\
&=  (Z^\mathrm{T} X)^{-1}(Z^\mathrm{T} Z)(Z^\mathrm{T} Z)^{-1}Z^\mathrm{T} y\\
&=(Z^\mathrm{T} X)^{-1}Z^\mathrm{T}y \\
&=\widehat{\beta}_\mathrm{IV}
\end{align}
</math>
Reference: see Davidson and Mackinnnon (1993)<ref>{{cite book|last1=Davidson|first1=Russell|first2=James|last2=Mackinnon|title=Estimation and Inference in Econometrics|year=1993|publisher=Oxford University Press|location=New York|isbn=978-0-19-506011-9}}</ref>{{rp|218}}
{{hidden end}}

There is an equivalent [[Parameter identification problem|under-identified]] estimator for the case where ''m < k''. Since the parameters are the solutions to a set of linear equations, an under-identified model using the set of equations <math> Z'v = 0 </math> does not have a unique solution.

==Interpretation as two-stage least squares==

One computational method which can be used to calculate IV estimates is two-stage least squares (2SLS or TSLS).  In the first stage, each explanatory variable that is an endogenous covariate in the equation of interest is regressed on all of the exogenous variables in the model, including both exogenous covariates in the equation of interest and the excluded instruments.  The predicted values from these regressions are obtained:

'''Stage 1:''' Regress each column of '''X''' on '''Z''', (<math> X = Z \delta + \text{errors} </math>):

: <math>\widehat{\delta}=(Z^\mathrm{T} Z)^{-1}Z^\mathrm{T}X, \,</math>

and save the predicted values:

: <math>\widehat{X}= Z\widehat{\delta} = {\color{ProcessBlue}Z(Z^\mathrm{T} Z)^{-1}Z^\mathrm{T}}X = {\color{ProcessBlue}P_Z} X.\, </math>

In the second stage, the regression of interest is estimated as usual, except that in this stage each endogenous covariate is replaced with the predicted values from the first stage:

'''Stage 2: ''' Regress '''Y''' on the predicted values from the first stage:

: <math> Y = \widehat X \beta + \mathrm{noise},\,</math>

which gives
: <math> \beta_\text{2SLS} = \left(X^\mathrm{T}{\color{ProcessBlue}P_Z} X\right)^{-1} X^\mathrm{T}{\color{ProcessBlue}P_Z}Y.</math>

This method is only valid in linear models. For categorical endogenous covariates, one might be tempted to use a different first stage than ordinary least squares, such as a [[probit model]] for the first stage followed by OLS for the second. This is commonly known in the econometric literature as the ''forbidden regression'',<ref>Wooldridge, J. (2010). Econometric Analysis of Cross Section and Panel Data. Econometric Analysis of Cross Section and Panel Data. MIT Press.{{pn|date=June 2021}}</ref> because second-stage IV parameter estimates are consistent only in special cases.<ref>{{cite thesis |last1=Lergenmuller |first1=Simon |title=Two-stage predictor substitution for time-to-event data |date=2017 |hdl=10852/57801 }}</ref>

{{hidden begin|style=border:1px solid #aaa|title={{center|Proof: computation of the 2SLS estimator}}}}
The usual OLS estimator is: <math> (\widehat X^\mathrm{T}\widehat X)^{-1}\widehat X^\mathrm{T}Y</math>.
Replacing <math> \widehat X = P_Z X</math> and noting that <math>P_Z </math> is a symmetric and [[Idempotence|idempotent]] matrix, so that <math>  P_Z^\mathrm{T}P_Z=P_Z P_Z = P_Z</math>
: <math> \beta_\text{2SLS} = (\widehat X^\mathrm{T}\widehat X)^{-1}\widehat X^\mathrm{T} Y = \left(X^\mathrm{T}P_Z^\mathrm{T}P_Z X\right)^{-1} X^\mathrm{T}P_Z^\mathrm{T}Y=\left(X^\mathrm{T}P_Z X\right)^{-1} X^\mathrm{T}P_ZY.</math>
{{hidden end}}

The resulting estimator of <math>\beta</math> is numerically identical to the expression displayed above. A small correction must be made to the sum-of-squared residuals in the second-stage fitted model in order that the covariance matrix of <math>\beta</math> is calculated correctly.

==Non-parametric analysis==

When the form of the structural equations is unknown, an instrumental variable <math>Z</math> can still be defined through the equations:

:<math>x = g(z,u) \, </math>
:<math>y = f(x,u) \, </math>

where <math>f</math> and <math>g</math> are two arbitrary functions and <math>Z</math> is independent of <math>U</math>. Unlike linear models, however, measurements of <math>Z, X</math> and <math>Y</math> do not allow for the identification of the average causal effect of <math>X</math> on <math>Y</math>, denoted ACE
:<math>\text{ACE} = \Pr(y\mid \text{do}(x)) = \operatorname{E}_u[f(x,u)].</math>
Balke and Pearl [1997] derived tight bounds on ACE and showed that these can provide valuable information on the sign and size of ACE.<ref name=balke:pearl97>{{cite journal |last1=Balke |first1=A. |last2=Pearl |first2=J. |title=Bounds on treatment effects from studies with imperfect compliance |journal=[[Journal of the American Statistical Association]] |volume=92 |issue=439 |pages=1172–1176 |year=1997 |doi=10.1080/01621459.1997.10474074 |citeseerx=10.1.1.26.3952 |s2cid=18365761 }}</ref>

In linear analysis, there is no test to falsify the assumption the <math>Z</math> is instrumental relative to the pair <math>(X,Y)</math>. This is not the case when <math>X</math> is discrete. Pearl (2000) has shown that, for all <math>f</math> and <math>g</math>, the following constraint, called "Instrumental Inequality" must hold whenever <math>Z</math> satisfies the two equations above:<ref name=Pearl:00 />
:<math>\max_x \sum_y [\max_z \Pr(y,x\mid z)]\leq 1.</math>

==Interpretation under treatment effect heterogeneity==

The exposition above assumes that the causal effect of interest does not vary across observations, that is, that <math>\beta</math> is a constant.  Generally, different subjects will respond in different ways to changes in the "treatment" ''x''.  When this possibility is recognized, the average effect in the population of a change in ''x'' on ''y'' may differ from the effect in a given subpopulation.  For example, the average effect of a job training program may substantially differ across the group of people who actually receive the training and the group which chooses not to receive training.  For these reasons, IV methods invoke implicit assumptions on behavioral response, or more generally assumptions over the correlation between the response to treatment and propensity to receive treatment.<ref>{{cite journal |last=Heckman |first=J. |year=1997 |title=Instrumental variables: A study of implicit behavioral assumptions used in making program evaluations |journal=[[Journal of Human Resources]] |volume=32 |issue=3 |pages=441–462 |jstor=146178 |doi=10.2307/146178 }}</ref>

The standard IV estimator can recover [[local average treatment effect]]s (LATE) rather than [[average treatment effects]] (ATE).<ref name=Imbens:00 />  Imbens and Angrist (1994) demonstrate that the linear IV estimate can be interpreted under weak conditions as a weighted average of local average treatment effects, where the weights depend on the elasticity of the endogenous regressor to changes in the instrumental variables. Roughly, that means that the effect of a variable is only revealed for the subpopulations affected by the observed changes in the instruments, and that subpopulations which respond most to changes in the instruments will have the largest effects on the magnitude of the IV estimate.

For example, if a researcher uses presence of a land-grant college as an instrument for college education in an earnings regression, she identifies the effect of college on earnings in the subpopulation which would obtain a college degree if a college is present but which would not obtain a degree if a college is not present.  This empirical approach does not, without further assumptions, tell the researcher anything about the effect of college among people who would either always or never get a college degree regardless of whether a local college exists.

==Weak instruments problem==

As Bound, [[David A. Jaeger|Jaeger]], and Baker (1995) note, a problem is caused by the selection of "weak" instruments, instruments that are poor predictors of the endogenous question predictor in the first-stage equation.<ref>{{Cite journal | doi = 10.1080/01621459.1995.10476536| title = Problems with Instrumental Variables Estimation when the Correlation between the Instruments and the Endogenous Explanatory Variable is Weak| journal = Journal of the American Statistical Association| volume = 90| issue = 430| pages = 443| year = 1995| last1 = Bound | first1 = J. | last2 = Jaeger | first2 = D. A. | last3 = Baker | first3 = R. M. }}</ref> In this case, the prediction of the question predictor by the instrument will be poor and the predicted values will have very little variation. Consequently, they are unlikely to have much success in predicting the ultimate outcome when they are used to replace the question predictor in the second-stage equation.

In the context of the smoking and health example discussed above, tobacco taxes are weak instruments for smoking if smoking status is largely unresponsive to changes in taxes.  If higher taxes do not induce people to quit smoking (or not start smoking), then variation in tax rates tells us nothing about the effect of smoking on health.  If taxes affect health through channels other than through their effect on smoking, then the instruments are invalid and the instrumental variables approach may yield misleading results.  For example, places and times with relatively health-conscious populations may both implement high tobacco taxes and exhibit better health even holding smoking rates constant, so we would observe a correlation between health and tobacco taxes even if it were the case that smoking has no effect on health.  In this case, we would be mistaken to infer a causal effect of smoking on health from the observed correlation between tobacco taxes and health.

===Testing for weak instruments===
The strength of the instruments can be directly assessed because both the endogenous covariates and the instruments are observable.<ref name=Stock:00>{{cite journal |last1=Stock |first1=J. |first2=J. |last2=Wright |first3=M. |last3=Yogo |year=2002 |title=A Survey of Weak Instruments and Weak Identification in Generalized Method of Moments |journal=[[Journal of the American Statistical Association]] |volume=20 |issue=4 |pages=518–529 |doi=10.1198/073500102288618658 |citeseerx=10.1.1.319.2477 |s2cid=14793271 }}</ref>  A common rule of thumb for models with one endogenous regressor is: the [[F-test|F-statistic]] against the [[Null hypothesis|null]] that the excluded instruments are irrelevant in the first-stage regression should be larger than 10.

==Statistical inference and hypothesis testing==

When the covariates are exogenous, the small-sample properties of the OLS estimator can be derived in a straightforward manner by calculating moments of the estimator conditional on ''X''.  When some of the covariates are endogenous so that instrumental variables estimation is implemented, simple expressions for the moments of the estimator cannot be so obtained.  Generally, instrumental variables estimators only have desirable asymptotic, not finite sample, properties, and inference is based on asymptotic approximations to the [[sampling distribution]] of the estimator.  Even when the instruments are uncorrelated with the error in the equation of interest and when the instruments are not weak, the finite sample properties of the instrumental variables estimator may be poor.  For example, exactly identified models produce finite sample estimators with no moments, so the estimator can be said to be neither biased nor unbiased, the nominal size of test statistics may be substantially distorted, and the estimates may commonly be far away from the true value of the parameter.<ref name="Nelson 1990:00">{{cite journal |last1=Nelson |first1=C. R. |first2=R. |last2=Startz |year=1990 |title=Some Further Results on the Exact Small Sample Properties of the Instrumental Variable Estimator |journal=[[Econometrica]] |volume=58 |issue=4 |pages=967–976 |jstor=2938359 |doi=10.2307/2938359 |s2cid=119872226 |url=https://www.nber.org/papers/t0068 }}</ref>

==Testing the exclusion restriction==

The assumption that the instruments are not correlated with the error term in the equation of interest is not testable in exactly identified models.  If the model is overidentified, there is information available which may be used to test this assumption.  The most common test of these ''overidentifying restrictions'', called the [[Sargan–Hansen test]], is based on the observation that the residuals should be uncorrelated with the set of exogenous variables if the instruments are truly exogenous.<ref>{{cite book |first=Fumio |last=Hayashi |chapter=Testing Overidentifying Restrictions |title=Econometrics |location=Princeton |publisher=Princeton University Press |year=2000 |isbn=978-0-691-01018-2 |pages=217–221 |chapter-url=https://books.google.com/books?id=QyIW8WUIyzcC&pg=PA217 }}</ref>  The Sargan–Hansen [[test statistic]] can be calculated as <math>TR^2</math> (the number of observations multiplied by the [[coefficient of determination]]) from the OLS regression of the residuals onto the set of exogenous variables.  This statistic will be asymptotically chi-squared with ''m''&nbsp;−&nbsp;''k'' degrees of freedom under the null that the error term is uncorrelated with the instruments.

==See also==
* {{annotated link|Control function (econometrics)}}
* {{annotated link|Optimal instruments}}

==References==
{{Reflist|30em}}

==Further reading==
* {{cite book |last=Greene |first=William H. |author-link=William Greene (economist) |title=Econometric Analysis |url=https://archive.org/details/econometricanaly00gree_641 |url-access=limited |location=Upper Saddle River |publisher=Pearson Prentice-Hall |year=2008 |edition=Sixth |isbn=978-0-13-600383-0 |pages=[https://archive.org/details/econometricanaly00gree_641/page/n351 314]–353 }}
* {{cite book |last1=Gujarati |first1=Damodar N. |author-link=Damodar N. Gujarati |last2=Porter |first2=Dawn C.|author2-link=Dawn C. Porter |title=Basic Econometrics |url=https://archive.org/details/basiceconometric05edguja |url-access=registration |location=New York |publisher=McGraw-Hill Irwin |edition=Fifth |year=2009 |isbn=978-0-07-337577-9 |pages=[https://archive.org/details/basiceconometric05edguja/page/711 711]–736 }}
* Keane, Michael P.; Neal, Timothy (2024). "[[doi:10.1146/annurev-economics-092123-111021|A Practical Guide to Weak Instruments]]". ''Annual Review of Economics''. '''16''': 185–212.
* {{cite book |first=Denis |last=Sargan |author-link=Denis Sargan |title=Lectures on Advanced Econometric Theory |location=Oxford |publisher=Basil Blackwell |year=1988 |isbn=978-0-631-14956-9 |pages=42–67 }}
* {{cite book |last=Wooldridge |first=Jeffrey M. |author-link=Jeffrey Wooldridge |year=2013 |title=Introductory Econometrics: A Modern Approach |location=Mason, OH |publisher=South-Western |edition=Fifth international |pages=490–528 |isbn=978-1-111-53439-4 }}

== Bibliography ==
* Wooldridge, J. (1997): Quasi-Likelihood Methods for Count Data, Handbook of Applied Econometrics, Volume 2, ed. M. H. Pesaran and P. Schmidt, Oxford, Blackwell, pp.&nbsp;352–406
* Terza, J. V. (1998): "Estimating Count Models with Endogenous Switching: Sample Selection and Endogenous Treatment Effects." ''Journal of Econometrics'' (84), pp.&nbsp;129–154
* Wooldridge, J. (2002): "Econometric Analysis of Cross Section and Panel Data", ''MIT Press'', Cambridge, Massachusetts.

==External links==
* [http://emlab.berkeley.edu/users/mcfadden/e240b_f01/ch4.pdf Chapter] from [[Daniel McFadden]]'s textbook
* {{YouTube|id=Kb4LvSguwjg&list=PLD15D38DC7AA3B737&index=12|title=Econometrics lecture (topic: instrumental variable)}} by [[Mark Thoma]].
* {{YouTube|id=D5lt9bhOshc&list=PLD15D38DC7AA3B737&index=15#t=54m09s|title=Econometrics lecture (topic: two-stages least square)}} by Mark Thoma

{{Authority control}}

[[Category:Regression analysis]]
[[Category:Simultaneous equation methods (econometrics)]]