Editing Secant method

{{Short description|Root-finding method}}{{Use American English|date = March 2019}}
[[Image:Secant method.svg|thumb|300px|The first two iterations of the secant method. The red curve shows the function ''f'', and the blue lines are the secants. For this particular case, the secant method will not converge to the visible root.]]

In [[numerical analysis]], the '''secant method''' is a [[root-finding algorithm]] that uses a succession of [[Root of a function|roots]] of [[secant line]]s to better approximate a root of a [[Function (mathematics)|function]] ''f''.  The secant method can be thought of as a [[finite-difference]] approximation of [[Newton's method]], so it is considered a [[quasi-Newton method]]. Historically, it is as an evolution of the [[Regula falsi|method of false position]], which predates Newton's method by over 3000 years.<ref>{{Cite journal|last1=Papakonstantinou|first1=Joanna|last2=Tapia|first2=Richard|date=2013|title=Origin and evolution of the secant method in one dimension|url=https://www.jstor.org/stable/10.4169/amer.math.monthly.120.06.500|journal=American Mathematical Monthly|volume=120|issue=6|pages=500–518|doi=10.4169/amer.math.monthly.120.06.500|jstor=10.4169/amer.math.monthly.120.06.500 |s2cid=17645996 |url-access=subscription}}</ref>

==The method==
The secant method is an iterative numerical method for finding a zero of a function {{mvar|f}}. Given two initial values {{math|''x''<sub>0</sub>}} and {{math|''x''<sub>1</sub>}}, the method proceeds according to the [[recurrence relation]]
:<math>
x_n
 = x_{n-1} - f(x_{n-1}) \frac{x_{n-1} - x_{n-2}}{f(x_{n-1}) - f(x_{n-2})}
 = \frac{x_{n-2} f(x_{n-1}) - x_{n-1} f(x_{n-2})}{f(x_{n-1}) - f(x_{n-2})}.
</math>

This is a nonlinear second-order recurrence that is well-defined given {{mvar|f}} and the two initial values {{math|''x''<sub>0</sub>}} and {{math|''x''<sub>1</sub>}}. Ideally, the initial values should be chosen close to the desired zero.

==Derivation of the method==
Starting with initial values {{math|''x''<sub>0</sub>}} and {{math|''x''<sub>1</sub>}}, we construct a line through the points {{math|(''x''<sub>0</sub>, ''f''(''x''<sub>0</sub>))}} and {{math|(''x''<sub>1</sub>, ''f''(''x''<sub>1</sub>))}}, as shown in the picture above. In point–point form,<ref>{{cite book |last1=Marsden |first1=Jerrold |title=Calculus I |date=1985 |publisher=Springer-Verlag New York Inc. |isbn=978-1-4612-5024-1 |pages=31 |url=https://link.springer.com/book/10.1007/978-1-4612-5024-1}}</ref> the equation of this line is

:<math>y = \frac{f(x_1) - f(x_0)}{x_1 - x_0}(x - x_1) + f(x_1).</math>
    
The root of this linear function, that is the value of {{mvar|x}} such that {{math|''y'' {{=}} 0}} is

:<math>x = x_1 - f(x_1) \frac{x_1 - x_0}{f(x_1) - f(x_0)}.</math>

We then use this new value of {{mvar|x}} as {{math|''x''<sub>2</sub>}} and repeat the process, using {{math|''x''<sub>1</sub>}} and {{math|''x''<sub>2</sub>}} instead of {{math|''x''<sub>0</sub>}} and {{math|''x''<sub>1</sub>}}. We continue this process, solving for {{math|''x''<sub>3</sub>}}, {{math|''x''<sub>4</sub>}}, etc., until we reach a sufficiently high level of precision (a sufficiently small difference between {{math|''x''<sub>''n''</sub>}} and {{math|''x''<sub>''n''−1</sub>}}):

:<math>
\begin{align}
x_2 & = x_1 - f(x_1) \frac{x_1 - x_0}{f(x_1) - f(x_0)}, \\[6pt]
x_3 & = x_2 - f(x_2) \frac{x_2 - x_1}{f(x_2) - f(x_1)}, \\[6pt]
& \,\,\,\vdots \\[6pt]
x_n & = x_{n-1} - f(x_{n-1}) \frac{x_{n-1} - x_{n-2}}{f(x_{n-1}) - f(x_{n-2})}.
\end{align}
</math>

==Convergence==

The iterates <math>x_n</math> of the secant method converge to a root of <math>f</math> if the initial values <math>x_0</math> and <math>x_1</math> are sufficiently close to the root and <math>f</math> is well-behaved. When <math>f</math> is twice continuously differentiable and the root in question is a simple root, i.e., it has multiplicity 1, the [[order of convergence]] is the [[golden ratio]] <math>\varphi = (1+\sqrt{5})/2 \approx 1.618.</math><ref name=":0">{{Cite web |last=Chanson |first=Jeffrey R. |date=October 3, 2024 |title=Order of Convergence |url=https://math.libretexts.org/Bookshelves/Applied_Mathematics/Numerical_Methods_(Chasnov)/02%3A_Root_Finding/2.04%3A_Order_of_Convergence |access-date=October 3, 2024 |website=LibreTexts Mathematics}}</ref> This convergence is superlinear but subquadratic.

If the initial values are not close enough to the root or <math>f</math> is not well-behaved, then there is no guarantee that the secant method converges at all. There is no general definition of "close enough", but the criterion for convergence has to do with how "wiggly" the function is on the interval between the initial values. For example, if <math>f</math> is differentiable on that interval and there is a point where <math>f' = 0</math> on the interval, then the algorithm may not converge.

==Comparison with other root-finding methods==

The secant method does not require or guarantee that the root remains bracketed by sequential iterates, like the [[bisection method]] does, and hence it does not always converge. The [[false position method]] (or {{lang|la|regula falsi}}) uses the same formula as the secant method. However, it does not apply the formula on <math>x_{n-1}</math> and <math>x_{n-2}</math>, like the secant method, but on <math>x_{n-1}</math> and on the last iterate <math>x_k</math> such that <math>f(x_k)</math> and <math>f(x_{n-1})</math> have a different sign. This means that the [[false position method]] always converges; however, only with a linear order of convergence. Bracketing with a super-linear order of convergence as the secant method can be attained with improvements to the false position method (see [[Regula falsi#Improvements in ''regula falsi''|Regula falsi § Improvements in ''regula falsi'']]) such as the [[ITP Method|ITP method]] or the [[Illinois Method|Illinois method]].

The recurrence formula of the secant method can be derived from the formula for [[Newton's method]]
:<math>x_n = x_{n-1} - \frac{f(x_{n-1})}{f'(x_{n-1})}</math>
by using the [[finite-difference]] approximation, for a small <math>\epsilon =x_{n-1} - x_{n-2} </math>:

<math>f'(x_{n-1}) = \lim_{\epsilon \rightarrow 0} \frac {f(x_{n-1})-f(x_{n-1} - \epsilon)}{\epsilon } \approx \frac{f(x_{n-1}) - f(x_{n-2})}{x_{n-1} - x_{n-2}}</math>

The secant method can be interpreted as a method in which the derivative is replaced by an approximation and is thus a [[quasi-Newton method]].

If we compare Newton's method with the secant method, we see that Newton's method converges faster (order 2 against order the [[golden ratio]] ''φ''&nbsp;≈&nbsp;1.6).<ref name=":0" /> However, Newton's method requires the evaluation of both <math>f</math> and its derivative <math>f'</math> at every step, while the secant method only requires the evaluation of <math>f</math>. Therefore, the secant method may sometimes be faster in practice. For instance, if we assume that evaluating <math>f</math> takes as much time as evaluating its derivative and we neglect all other costs, we can do two steps of the secant method (decreasing the logarithm of the error by a factor ''φ''<sup>2</sup>&nbsp;≈&nbsp;2.6) for the same cost as one step of Newton's method (decreasing the logarithm of the error by a factor of&nbsp;2), so the secant method is faster. In higher dimensions, the full set of [[Partial derivative|partial derivatives]] required for Newton's method, that is, the [[Jacobian matrix and determinant|Jacobian matrix]], may become much more expensive to calculate than the function itself.  If, however, we consider parallel processing for the evaluation of the derivative or derivatives, Newton's method can be faster in clock time though still costing more computational operations overall.

==Generalization==

[[Broyden's method]] is a generalization of the secant method to more than one dimension.

The following graph shows the function ''f'' in red and the last secant line in bold blue.  In the graph, the ''x'' intercept of the secant line seems to be a good approximation of the root of ''f''.

[[Image:Secant method example code result.svg|center]]

==Computational example==
Below, the secant method is implemented in the [[Python (programming language)|Python]] programming language. 

It is then applied to find a root of the function {{math|''f''(''x'') {{=}} ''x''<sup>2</sup> − 612}} with initial points <math>x_0 = 10</math> and <math>x_1 = 30</math>

<syntaxhighlight lang="python3">
def secant_method(f, x0: int, x1: int, iterations: int) -> float:
    """Return the root calculated using the secant method."""
    for i in range(iterations):
        x2 = x1 - f(x1) * (x1 - x0) / float(f(x1) - f(x0))
        x0, x1 = x1, x2
        # Apply a stopping criterion here (see below)
    return x2

def f_example(x):
    return x ** 2 - 612

root = secant_method(f_example, 10, 30, 5)

print(f"Root: {root}")  # Root: 24.738633748750722

</syntaxhighlight>

It is very important to have a good stopping criterion above, otherwise, due to limited numerical precision of floating point numbers, the algorithm can return inaccurate results if running for too many iterations. For example, the loop above can stop when one of these is reached first: {{Mono|abs(x0 - x1) < tol}}, or {{Mono|abs(x0/x1-1) < tol}}, or {{Mono|abs(f(x1)) < tol}}. <ref>{{cite web | url=https://www.cfm.brown.edu/people/dobrush/am33/Matlab/ch3/secant.html | title=MATLAB Tutorial for the First Course. Part 1.3: Secant Methods }}</ref>

==Notes==
{{reflist}}

== See also ==
* [[False position method]]

==References==
* {{cite book |first=Mordecai |last=Avriel |title=Nonlinear Programming: Analysis and Methods |publisher=Prentice Hall |year=1976 |isbn=0-13-623603-0 |pages=220–221 }}
* {{cite book| title=Numerical analysis for applied science|first1=Myron B. |last1=Allen |first2=Eli L. |last2=Isaacson | pages=188–195| isbn=978-0-471-55266-6| year=1998| publisher=[[John Wiley & Sons]]| url=https://books.google.com/books?id=PpB9cjOxQAQC}}

==External links==
* [http://numericalmethods.eng.usf.edu/topics/secant_method.html Secant Method] Notes, PPT, Mathcad, Maple, Mathematica, Matlab at [http://numericalmethods.eng.usf.edu Holistic Numerical Methods Institute]
* {{MathWorld|urlname=SecantMethod|title=Secant Method}}

{{Root-finding algorithms}}

[[Category:Quasi-Newton methods]]