Descent direction
In optimization, a descent direction is a vector <math>\mathbf{p}\in\mathbb R^n</math> that points towards a local minimum <math>\mathbf{x}^*</math> of an objective function <math>f:\mathbb R^n\to\mathbb R</math>.
Computing <math>\mathbf{x}^*</math> by an iterative method, such as line search defines a descent direction <math>\mathbf{p}_k\in\mathbb R^n</math> at the <math>k</math>th iterate to be any <math>\mathbf{p}_k</math> such that <math>\langle\mathbf{p}_k,\nabla f(\mathbf{x}_k)\rangle < 0</math>, where <math> \langle , \rangle </math> denotes the inner product. The motivation for such an approach is that small steps along <math>\mathbf{p}_k</math> guarantee that <math>\displaystyle f</math> is reduced, by Taylor's theorem.
Using this definition, the negative of a non-zero gradient is always a descent direction, as <math> \langle -\nabla f(\mathbf{x}_k), \nabla f(\mathbf{x}_k) \rangle = -\langle \nabla f(\mathbf{x}_k), \nabla f(\mathbf{x}_k) \rangle < 0 </math>.
Numerous methods exist to compute descent directions, all with differing merits, such as gradient descent or the conjugate gradient method.
More generally, if <math>P</math> is a positive definite matrix, then <math>p_k = -P \nabla f(x_k)</math> is a descent direction at <math>x_k</math>.<ref name="?">Template:Cite book</ref> This generality is used in preconditioned gradient descent methods.