Editing Simultaneous equations model (section)

== Estimation ==

=== Two-stage least squares (2SLS) ===
The simplest and the most common estimation method for the simultaneous equations model is the so-called [[two-stage least squares]] method,<ref name="Greene 2003 loc=p. 399">{{cite book
  | last = Greene | first = William H.
  | title = Econometric analysis
  | publisher = Prentice Hall
  | year = 2002 | edition = 5th
  | isbn = 0-13-066189-9
  | pages = 398–99
  }}</ref> developed independently by {{harvtxt|Theil|1953}} and {{harvtxt|Basmann|1957}}.<ref>{{cite report|first=H.|last=Theil|title=Estimation and Simultaneous Correlation in Complete Equation Systems|type=Memorandum|publisher=Central Planning Bureau|year=1953}} Reprinted in ''Henri Theil’s Contributions to Economics and Econometrics'' (Springer, 1992), {{doi|10.1007/978-94-011-2546-8_6}}.</ref><ref>{{cite journal
  | last = Basmann | first = R. L. |author-link=Robert Basmann
  | title = A generalized classical method of linear estimation of coefficients in a structural equation
  | year = 1957
  | journal = [[Econometrica]]
  | volume = 25 | issue = 1
  | pages = 77–83
  | jstor = 1907743
  | doi=10.2307/1907743
  }}</ref><ref>{{cite book
  | last = Theil | first = Henri |author-link=Henri Theil
  | title = Principles of Econometrics
  | url = https://archive.org/details/principlesofecon0000thei | url-access = registration | year = 1971
  | publisher = John Wiley
  | location = New York
  | isbn = 978-0-471-85845-4 }}</ref> It is an equation-by-equation technique, where the endogenous regressors on the right-hand side of each equation are being instrumented with the regressors ''X'' from all other equations. The method is called “two-stage” because it conducts estimation in two steps:<ref name="Greene 2003 loc=p. 399" />
: ''Step 1'': Regress ''Y<sub>−i</sub>'' on ''X'' and obtain the predicted values <math style="vertical-align:-.2em">\scriptstyle\hat{Y}_{\!-i}</math>;
: ''Step 2'': Estimate ''γ<sub>i</sub>'', ''β<sub>i</sub>'' by the [[ordinary least squares]] regression of ''y<sub>i</sub>'' on <math style="vertical-align:-.2em">\scriptstyle\hat{Y}_{\!-i}</math> and ''X<sub>i</sub>''.

If the ''i''<sup>th</sup> equation in the model is written as
: <math>
    y_i = \begin{pmatrix}Y_{-i} & X_i\end{pmatrix}\begin{pmatrix}\gamma_i\\\beta_i\end{pmatrix} + u_i
        \equiv Z_i \delta_i + u_i,
  </math>
where ''Z<sub>i</sub>'' is a ''T×''(''n<sub>i</sub> + k<sub>i</sub>'') matrix of both endogenous and exogenous regressors in the ''i''<sup>th</sup> equation, and ''δ<sub>i</sub>'' is an (''n<sub>i</sub> + k<sub>i</sub>'')-dimensional vector of regression coefficients, then the 2SLS estimator of ''δ<sub>i</sub>'' will be given by<ref name="Greene 2003 loc=p. 399"/>
: <math>
    \hat\delta_i = \big(\hat{Z}'_i\hat{Z}_i\big)^{-1}\hat{Z}'_i y_i
                 = \big( Z'_iPZ_i \big)^{-1} Z'_iPy_i,
  </math>
where {{nowrap|''P'' {{=}} ''X'' (''X'' ′''X'')<sup>−1</sup>''X'' ′}} is the projection matrix onto the linear space spanned by the exogenous regressors ''X''.

=== Indirect least squares ===
Indirect least squares is an approach in [[econometrics]] where the [[coefficient]]s in a simultaneous equations model are estimated from the [[reduced form]] model using [[ordinary least squares]].<ref>Park, S-B. (1974) "On Indirect Least Squares Estimation of a Simultaneous Equation System", ''The Canadian Journal of Statistics / La Revue Canadienne de Statistique'', 2 (1), 75–82 {{JSTOR|3314964}}</ref><ref>{{cite journal | last1 = Vajda | first1 = S. | last2 = Valko | first2 = P. | last3 = Godfrey | first3 = K.R. | year = 1987 | title = Direct and indirect least squares methods in continuous-time parameter estimation | journal = Automatica | volume = 23 | issue = 6| pages = 707–718 | doi = 10.1016/0005-1098(87)90027-6 }}</ref>  For this, the structural system of equations is transformed into the reduced form first. Once the coefficients are estimated the model is put back into the structural form.

=== Limited information maximum likelihood (LIML) ===
The “limited information” maximum likelihood method was suggested by [[Meyer Abraham Girshick|M. A. Girshick]] in 1947,<ref>First application by {{cite journal |first1=M. A. |last1=Girshick |first2=Trygve |last2=Haavelmo |title=Statistical Analysis of the Demand for Food: Examples of Simultaneous Estimation of Structural Equations |journal=[[Econometrica]] |volume=15 |issue=2 |year=1947 |pages=79–110 |doi= 10.2307/1907066|jstor=1907066 }}</ref> and formalized by [[Theodore Wilbur Anderson|T. W. Anderson]] and [[Herman Rubin|H. Rubin]] in 1949.<ref>{{cite journal
  | last1 = Anderson | first1 = T.W.
  | last2 = Rubin    | first2 = H.
  | title = Estimator of the parameters of a single equation in a complete system of stochastic equations
  | year = 1949
  | journal = [[Annals of Mathematical Statistics]]
  | volume = 20 | issue = 1
  | pages = 46–63
  | jstor = 2236803
  | doi=10.1214/aoms/1177730090
  | doi-access = free
  }}</ref> It is used when one is interested in estimating a single structural equation at a time (hence its name of limited information), say for observation i:

: <math>   y_i = Y_{-i}\gamma_i  +X_i\beta_i+ u_i \equiv Z_i \delta_i + u_i  </math>

The structural equations for the remaining endogenous variables Y<sub>−i</sub> are not specified, and they are given in their reduced form:
: <math>   Y_{-i} = X \Pi + U_{-i} </math>

Notation in this context is different than for the simple [[Instrumental variable|IV]] case. One has:
* <math>Y_{-i}</math>: The endogenous variable(s).
* <math>X_{-i}</math>: The exogenous variable(s)
* <math>X</math>: The instrument(s) (often denoted <math>Z</math>)

The explicit formula for the LIML is:<ref>{{cite book
  | last = Amemiya
  | first = Takeshi
  | title = Advanced Econometrics
  | year = 1985
  | publisher = Harvard University Press
  | location = Cambridge, Massachusetts
  | isbn = 0-674-00560-0
  | page = [https://archive.org/details/advancedeconomet00amem/page/235 235]
  | url-access = registration
  | url = https://archive.org/details/advancedeconomet00amem
  }}</ref>
: <math>
    \hat\delta_i = \Big(Z'_i(I-\lambda M)Z_i\Big)^{\!-1}Z'_i(I-\lambda M)y_i,
  </math>
where {{nowrap|''M'' {{=}} ''I − X'' (''X'' ′''X'')<sup>−1</sup>''X'' ′}}, and ''λ'' is the smallest characteristic root of the matrix:
: <math>
    \Big(\begin{bmatrix}y_i\\Y_{-i}\end{bmatrix} M_i \begin{bmatrix}y_i&Y_{-i}\end{bmatrix} \Big)
    \Big(\begin{bmatrix}y_i\\Y_{-i}\end{bmatrix} M \begin{bmatrix}y_i&Y_{-i}\end{bmatrix} \Big)^{\!-1}
  </math>

where, in a similar way,  {{nowrap|''M<sub>i</sub>'' {{=}} ''I − X<sub>i</sub>'' (''X<sub>i</sub>''′''X<sub>i</sub>'')<sup>−1</sup>''X<sub>i</sub>''′}}.

In other words, ''λ'' is the smallest solution of the [[Generalized eigenvalue problem#Generalized eigenvalue problem|generalized eigenvalue problem]], see {{harvtxt|Theil|1971|loc=p. 503}}:

: <math>
\Big|\begin{bmatrix}y_i&Y_{-i}\end{bmatrix}' M_i \begin{bmatrix}y_i&Y_{-i}\end{bmatrix} -\lambda
\begin{bmatrix}y_i&Y_{-i}\end{bmatrix}' M \begin{bmatrix}y_i&Y_{-i}\end{bmatrix} \Big|=0
  </math>

==== K class estimators ====
The LIML is a special case of the K-class estimators:<ref name="DavidsonMacKinnon649">{{cite book
  | last1 = Davidson | first1 = Russell
  | last2 = MacKinnon | first2 = James G.
  | title = Estimation and inference in econometrics
  | year = 1993
  | publisher = Oxford University Press
  | isbn = 0-19-506011-3
  | page=649
  }}</ref>
: <math>
    \hat\delta = \Big(Z'(I-\kappa M)Z\Big)^{\!-1}Z'(I-\kappa M)y,
  </math>

with:
* <math> \delta = \begin{bmatrix} \beta_i & \gamma_i\end{bmatrix} </math>
* <math> Z = \begin{bmatrix} X_i & Y_{-i}\end{bmatrix} </math>
Several estimators belong to this class:
* κ=0: [[Ordinary least squares|OLS]]
* κ=1: 2SLS. Note indeed that in this case, <math> I-\kappa M = I-M= P </math> the usual projection matrix of the 2SLS
* κ=λ: LIML
* κ=λ - α / (n-K): {{harvtxt|Fuller|1977}} estimator.<ref>{{cite journal
  | last = Fuller | first = Wayne |author-link=Wayne Fuller
  | title = Some Properties of a Modification of the Limited Information Estimator
  | year = 1977
  | journal = Econometrica
  | volume = 45 |issue=4
  | pages = 939–953
  | doi=10.2307/1912683
  | jstor = 1912683 }}</ref> Here K represents the number of instruments, n the sample size, and α a positive constant to specify. A value of α=1 will yield an estimator that is approximately unbiased.<ref name="DavidsonMacKinnon649" />

=== Three-stage least squares (3SLS) ===
The three-stage least squares estimator was introduced by {{harvtxt|Zellner|Theil|1962}}.<ref>{{cite journal
  | last1 = Zellner | first1 = Arnold |author-link1=Arnold Zellner
  | last2 = Theil   | first2 = Henri |author-link2=Henri Theil
  | title = Three-stage least squares: simultaneous estimation of simultaneous equations
  | year = 1962
  | journal = Econometrica
  | volume = 30 | issue = 1
  | pages = 54–78
  | jstor = 1911287
  | doi=10.2307/1911287
  }}</ref><ref>{{cite book |first=Jan |last=Kmenta |chapter=System Methods of Estimation |title=Elements of Econometrics |location=New York |publisher=Macmillan |edition=Second |year=1986 |pages=695–701 |isbn=9780023650703 |chapter-url=https://books.google.com/books?id=Bxq7AAAAIAAJ&pg=PA695 }}</ref>  It can be seen as a special case of multi-equation [[Generalized method of moments|GMM]] where the set of [[instrumental variable]]s is common to all equations.<ref>{{cite book |first=Fumio |last=Hayashi |chapter=Multiple-Equation GMM |title=Econometrics |publisher=Princeton University Press |year=2000 |pages=276–279 |isbn=1400823838 |chapter-url=https://books.google.com/books?id=QyIW8WUIyzcC&pg=PA276 }}</ref> If all regressors are in fact predetermined, then 3SLS reduces to [[seemingly unrelated regressions]] (SUR). Thus it may also be seen as a combination of [[2SLS|two-stage least squares]] (2SLS) with SUR.