Editing Gauss–Newton algorithm (section)

==Solving overdetermined systems of equations==

The Gauss-Newton iteration
<math display="block">\mathbf{x}^{(k+1)} = \mathbf{x}^{(k)} - J(\mathbf{x}^{(k)})^\dagger\mathbf{f}(\mathbf{x}^{(k)}) \,,\quad k=0,1,\ldots</math>
is an effective method for solving  [[overdetermined system]]s of equations in the form of <math>\mathbf{f}(\mathbf{x})=\mathbf{0}</math> with
<math display="block">\mathbf{f}(\mathbf{x}) = \begin{bmatrix} f_1(x_1,\ldots,x_n) \\ \vdots \\ f_m(x_1,\ldots,x_n) \end{bmatrix}</math>
and <math>m>n</math> where <math>J(\mathbf{x})^\dagger</math> is the [[Moore-Penrose inverse]] (also known as [[pseudoinverse]]) of the [[Jacobian matrix]] <math>J(\mathbf{x})</math> of <math>\mathbf{f}(\mathbf{x})</math>. 
It can be considered an extension of [[Newton's method]] and enjoys the same local quadratic convergence <ref name=DenSch></ref> toward isolated regular solutions. 

If the solution doesn't exist but the initial iterate <math>\mathbf{x}^{(0)}</math> is near a point <math>\hat{\mathbf{x}} = (\hat{x}_1,\ldots,\hat{x}_n)</math> at which the sum of squares <math display="inline">\sum_{i=1}^m |f_i(x_1,\ldots,x_n)|^2 \equiv \|\mathbf{f}(\mathbf{x})\|_2^2</math> reaches a small local minimum, the Gauss-Newton iteration linearly converges to <math>\hat{\mathbf{x}}</math>. The point <math>\hat{\mathbf{x}}</math> is often called a [[least squares]] solution of the overdetermined system.