Editing Nonlinear programming (section)

=== Numeric methods ===
In most realistic cases, it is very hard to solve the KKT conditions analytically, and so the problems are solved using [[Numerical analysis|numerical methods]]. These methods are iterative: they start with an initial point, and then proceed to points that are supposed to be closer to the optimal point, using some update rule. There are three kinds of update rules:<ref name=":0" />{{Rp|location=5.1.2}}

* Zero-order routines - use only the values of the objective function and constraint functions at the current point;
* First-order routines - use also the values of the ''[[gradient]]s'' of these functions;
* Second-order routines - use also the values of the [[Hessian matrix|''Hessians'']] of these functions.

Third-order routines (and higher) are theoretically possible, but not used in practice, due to the higher computational load and little theoretical benefit.