Editing Gauss–Newton algorithm (section)

== Related algorithms ==

In a [[quasi-Newton method]], such as that due to [[Davidon–Fletcher–Powell formula|Davidon, Fletcher and Powell]] or Broyden–Fletcher–Goldfarb–Shanno ([[BFGS method]]) an estimate of the full Hessian <math display="inline">\frac{\partial^2 S}{\partial \beta_j \partial\beta_k}</math> is built up numerically using first derivatives <math display="inline">\frac{\partial r_i}{\partial\beta_j}</math> only so that after ''n'' refinement cycles the method closely approximates to Newton's method in performance. Note that quasi-Newton methods can minimize general real-valued functions, whereas Gauss–Newton, Levenberg–Marquardt, etc. fits only to nonlinear least-squares problems.

Another method for solving minimization problems using only first derivatives is [[gradient descent]]. However, this method does not take into account the second derivatives even approximately. Consequently, it is highly inefficient for many functions, especially if the parameters have strong interactions.