Editing Descent direction (section)

In [[optimization (mathematics)|optimization]], a '''descent direction''' is a vector <math>\mathbf{p}\in\mathbb R^n</math> that points towards a local minimum <math>\mathbf{x}^*</math> of an objective function <math>f:\mathbb R^n\to\mathbb R</math>.

Computing <math>\mathbf{x}^*</math> by an iterative method, such as [[line search]] defines a descent direction <math>\mathbf{p}_k\in\mathbb R^n</math> at the <math>k</math>th iterate to be any <math>\mathbf{p}_k</math> such that <math>\langle\mathbf{p}_k,\nabla f(\mathbf{x}_k)\rangle < 0</math>, where <math> \langle , \rangle </math> denotes the [[inner product]]. The motivation for such an approach is that small steps along <math>\mathbf{p}_k</math> guarantee that <math>\displaystyle f</math> is reduced, by [[Taylor's theorem]].

Using this definition, the negative of a non-zero gradient is always a
descent direction, as <math> \langle -\nabla f(\mathbf{x}_k), \nabla f(\mathbf{x}_k) \rangle = -\langle \nabla f(\mathbf{x}_k), \nabla f(\mathbf{x}_k) \rangle < 0 </math>.

Numerous methods exist to compute descent directions, all with differing merits, such as [[gradient descent]] or the [[conjugate gradient method]].

More generally, if <math>P</math> is a [[positive definite]] matrix, then
<math>p_k = -P \nabla f(x_k)</math> is a descent direction at <math>x_k</math>.<ref name="?">{{cite book | author =  J. M. Ortega and W. C. Rheinbold | title = Iterative Solution of Nonlinear Equations in Several Variables | pages = 243 | year = 1970 | doi = 10.1137/1.9780898719468
 | isbn = 978-0-89871-461-6 }}</ref> This generality is used in [[preconditioned gradient descent]] methods.