Editing Iterative method (section)

{{Short description|Algorithm in which each approximation of the solution is derived from prior approximations}}

In [[computational mathematics]], an '''iterative method''' is a [[Algorithm|mathematical procedure]] that uses an initial value to generate a sequence of improving approximate solutions for a class of problems, in which the ''i''-th approximation (called an "iterate") is derived from the previous ones. 

A specific implementation with [[Algorithm#Termination|termination]] criteria for a given iterative method like [[gradient descent]], [[hill climbing]], [[Newton's method]], or [[Quasi-Newton method|quasi-Newton methods]] like [[Broyden–Fletcher–Goldfarb–Shanno algorithm|BFGS]], is an [[algorithm]] of an iterative method or a '''method of successive approximation'''. An iterative method is called ''[[Convergent series|convergent]]'' if the corresponding sequence converges for given initial approximations. A mathematically rigorous convergence analysis of an iterative method is usually performed; however, [[heuristic]]-based iterative methods are also common.

In contrast, '''direct methods''' attempt to solve the problem by a finite sequence of operations. In the absence of [[rounding error]]s, direct methods would deliver an exact solution (for example, solving a linear system of equations <math>A\mathbf{x}=\mathbf{b}</math> by [[Gaussian elimination]]). Iterative methods are often the only choice for [[nonlinear equation]]s. However, iterative methods are often useful even for linear problems involving many variables (sometimes on the order of millions), where direct methods would be prohibitively expensive (and in some cases impossible) even with the best available computing power.<ref>{{Cite journal|doi=10.1016/j.jcp.2015.09.040|title=Recycling Krylov subspaces for CFD applications and a new hybrid recycling solver|year=2015|last1=Amritkar|first1=Amit|last2=de Sturler|first2=Eric|last3=Świrydowicz|first3=Katarzyna|last4=Tafti|first4=Danesh|last5=Ahuja|first5=Kapil|journal=Journal of Computational Physics|volume=303|page=222|arxiv=1501.03358|bibcode=2015JCoPh.303..222A}}</ref>