Editing Logistic function (section)

== Mathematical properties ==
<!--{{move section portions from|Pierre François Verhulst|date=February 2014}} Please see next section-->
The '''{{visible anchor|standard logistic function}}''' is the logistic function with parameters <math>k = 1</math>, <math>x_0 = 0</math>, <math>L = 1</math>, which yields

<math display="block">f(x) = \frac{1}{1 + e^{-x}} = \frac{e^x}{e^x + 1} = \frac{e^{x/2}}{e^{x/2} + e^{-x/2}}.</math>

In practice, due to the nature of the [[exponential function]] <math>e^{-x}</math>, it is often sufficient to compute the standard logistic function for <math>x</math> over a small range of real numbers, such as a range contained in [−6,&nbsp;+6], as it quickly converges very close to its saturation values of 0 and 1.

===Symmetries===
The logistic function has the symmetry property that

<math display="block">1 - f(x) = f(-x).</math>

This reflects that the growth from 0 when <math>x</math> is small is symmetric with the decay of the gap to the limit (1) when <math>x</math> is large.

Further, <math>x \mapsto f(x) - 1/2</math> is an [[odd function]].

The sum of the logistic function and its reflection about the vertical axis, <math>f(-x)</math>, is

<math display="block">\frac{1}{1 + e^{-x}} + \frac{1}{1 + e^{-(-x)}} = \frac{e^x}{e^x + 1} + \frac{1}{e^x + 1} = 1.</math>

The logistic function is thus rotationally symmetrical about the point (0,&nbsp;1/2).<ref>{{cite book |title=Neural Networks – A Systematic Introduction |author=Raul Rojas |url=http://page.mi.fu-berlin.de/rojas/neural/chapter/K11.pdf |access-date=15 October 2016}}</ref>

===Inverse function===
The logistic function is the inverse of the natural [[logit]] function

<math display="block"> \operatorname{logit} p = \log \frac p {1-p} \quad \text{ for }\, 0<p<1 </math>

and so converts the logarithm of [[odds]] into a [[probability]].  The conversion from the [[log-likelihood ratio]] of two alternatives also takes the form of a logistic curve.

===Hyperbolic tangent===

The logistic function is an offset and scaled [[hyperbolic tangent]] function:
<math display="block">f(x) = \frac12 + \frac12 \tanh\left(\frac{x}{2}\right),</math>
or
<math display="block">\tanh(x) = 2 f(2x) - 1.</math>

This follows from
<math display="block">
\begin{align}
\tanh(x) & = \frac{e^x - e^{-x}}{e^x + e^{-x}} = \frac{e^x \cdot \left(1 - e^{-2x}\right)}{e^x \cdot \left(1 + e^{-2x}\right)} \\
         &= f(2x) - \frac{e^{-2x}}{1 + e^{-2x}} = f(2x) - \frac{e^{-2x} + 1 - 1}{1 + e^{-2x}} = 2f(2x) - 1.
\end{align}
</math>

The hyperbolic-tangent relationship leads to another form for the logistic function's derivative:

<math display="block">\frac{d}{dx} f(x) = \frac14 \operatorname{sech}^2\left(\frac{x}{2}\right),</math>

which ties the logistic function into the [[logistic distribution]].

Geometrically, the hyperbolic tangent function is the [[hyperbolic angle]] on the [[unit hyperbola]] <math>x^2 - y^2 = 1</math>, which factors as <math>(x + y)(x - y) = 1</math>, and thus has asymptotes the lines through the origin with slope {{tmath|-1}} and with slope {{tmath|1}}, and vertex at {{tmath|(1, 0)}} corresponding to the range and midpoint ({{tmath|0/1 = 0}}) of tanh. Analogously, the logistic function can be viewed as the hyperbolic angle on the hyperbola <math>xy - y^2 = 1</math>, which factors as <math>y(x - y) = 1</math>, and thus has asymptotes the lines through the origin with slope {{tmath|0}} and with slope {{tmath|1}}, and vertex at {{tmath|(2, 1)}}, corresponding to the range and midpoint ({{tmath|1/2}}) of the logistic function.

Parametrically, [[hyperbolic cosine]] and [[hyperbolic sine]] give coordinates on the unit hyperbola:{{efn|Using {{tmath|t}} for the parameter and {{tmath|(x, y)}} for the coordinates.}} <math>\left( (e^t + e^{-t})/2, (e^t - e^{-t})/2\right)</math>, with quotient the hyperbolic tangent. Similarly, <math>\bigl(e^{t/2} + e^{-t/2}, e^{t/2}\bigr)</math> parametrizes the hyperbola <math>xy - y^2 = 1</math>, with quotient the logistic function. These correspond to [[linear transformations]] (and rescaling the parametrization) of [[Hyperbola#As_an_affine_image_of_the_hyperbola_y_=_1/x|the hyperbola <math>xy = 1</math>]], with parametrization <math>(e^{-t}, e^t)</math>: the parametrization of the hyperbola for the logistic function corresponds to <math>t/2</math> and the linear transformation <math>\bigl( \begin{smallmatrix}
1 & 1\\ 0 & 1 \end{smallmatrix} \bigr)</math>, while the parametrization of the unit hyperbola (for the hyperbolic tangent) corresponds to the linear transformation <math>\tfrac{1}{2}\bigl( \begin{smallmatrix}
1 & 1\\ -1 & 1 \end{smallmatrix} \bigr)</math>.

=== Derivative ===
[[File:Logistic function derivatives.png|class=skin-invert-image|thumb|The logistic function and its first 3 derivatives]]

The standard logistic function has an easily calculated [[derivative]]. The derivative is known as the density of the [[logistic distribution]]:

<math display="block">f(x) = \frac{1}{1 + e^{-x}} = \frac{e^x}{1 + e^x},</math>

<math display="block">\begin{align}
\frac{d}{dx} f(x) &= \frac{e^x \cdot (1 + e^x) - e^x \cdot e^x}{{\left(1 + e^x\right)}^2} \\[1ex]
&= \frac{e^x}{{\left(1 + e^x\right)}^2} \\[1ex]
&= \left(\frac{e^x}{1 + e^x}\right) \left(\frac{1}{1 + e^x}\right) \\[1ex]
&= \left(\frac{e^x}{1 + e^x}\right) \left(1-\frac{e^x}{1 + e^x}\right) \\[1.2ex]
&= f(x)\left(1 - f(x)\right)
\end{align}</math>from which all higher derivatives can be derived algebraically. For example, <math>f'' = (1-2f)(1-f)f </math>.

The logistic distribution is a [[location–scale family]], which corresponds to parameters of the logistic function. If {{tmath|1=L = 1}} is fixed, then the midpoint {{tmath|x_0}} is the location and the slope {{tmath|k}} is the scale.

=== Integral ===
Conversely, its [[antiderivative]] can be computed by the [[Integration by substitution|substitution]] <math>u = 1 + e^x</math>, since

<math display=block>f(x) = \frac{e^x}{1 + e^x} = \frac{u'}{u},</math>

so (dropping the [[constant of integration]])

<math display="block">\int \frac{e^x}{1 + e^x}\,dx = \int \frac{1}{u}\,du = \ln u = \ln (1 + e^x).</math>

In [[artificial neural network]]s, this is known as the ''[[softplus]]'' function and (with scaling) is a smooth approximation of the [[ramp function]], just as the logistic function (with scaling) is a smooth approximation of the [[Heaviside step function]].

=== Taylor series ===
The standard logistic function is [[Analytic function|analytic]] on the whole real line since <math>f : \mathbb{R} \to \mathbb{R}</math>, <math>f(x) = \frac{1}{1+e^{-x}} = h(g(x)) </math> where <math>g : \mathbb{R} \to \mathbb{R}</math>, <math>g(x) = 1 + e^{-x}</math> and <math>h : (0, \infty) \to (0, \infty)</math>, <math>h(x) = \frac{1}{x}</math>  are analytic on their domains, and the composition of analytic functions is again analytic.

A formula for the ''n''th derivative of the standard logistic function is

<math display="block">\frac{d^n f}{dx^n} = \sum_{i=1}^n \frac{\left(\sum_{j=1}^n {\left(-1\right)}^{i+j} \binom{i}{j} j^n\right) e^{-ix}}{{\left(1+e^{-x}\right)}^{i+1}} </math> 

therefore its [[Taylor series]] about the point <math>a </math> is

<math display="block">f(x) = f(a)(x-a) + \sum_{n=1}^{\infty} \sum_{i=1}^n \frac{\left(\sum_{j=1}^n {\left(-1\right)}^{i+j} \binom{i}{j} j^n\right) e^{-ix}}{{\left(1 + e^{-x}\right)}^{i+1}} \frac{{\left(x-a\right)}^n}{n!} . </math>

=== Logistic differential equation ===
The  unique standard logistic function is the solution of the simple first-order non-linear [[ordinary differential equation]]

<math display="block">\frac{d}{dx}f(x) = f(x)\big(1 - f(x)\big)</math>

with [[boundary condition]] <math>f(0) = 1/2</math>. This equation is the continuous version of the [[logistic map]]. Note that the reciprocal logistic function is solution to a simple first-order ''linear'' ordinary differential equation.<ref>{{cite journal |last1=Kocian |first1=Alexander |last2=Carmassi |first2=Giulia|last3=Cela |first3=Fatjon |last4=Incrocci|first4=Luca|last5=Milazzo|first5=Paolo|last6=Chessa|first6=Stefano |title=Bayesian Sigmoid-Type Time Series Forecasting with Missing Data for Greenhouse Crops |journal= Sensors|date=7 June 2020 |volume=20 |issue=11 |page=3246 |doi=10.3390/s20113246 |pmid=32517314 |pmc=7309099 |bibcode=2020Senso..20.3246K |doi-access=free }}</ref>

The qualitative behavior is easily understood in terms of the [[Phase line (mathematics)|phase line]]: the derivative is 0 when the function is 1; and the derivative is positive for <math>f</math> between 0 and 1, and negative for <math>f</math> above 1 or less than 0 (though negative populations do not generally accord with a physical model). This yields an unstable equilibrium at 0 and a stable equilibrium at 1, and thus for any function value greater than 0 and less than 1, it grows to 1.<!--

The above equation can be rewritten in the following steps:

<math display="block">\frac{d}{dx}f(x) = f(x)(1-f(x)) </math>
<math display="block">\frac{dy}{dx} = y(1-y) </math>
<math display="block">\frac{dy}{dx} = y - y^2 </math>
<math display="block">\frac{dy}{dx} - y = -y^2 </math> trivial algebraic manipulation-->

The logistic equation is a special case of the [[Bernoulli differential equation]] and has the following solution:

<math display="block">f(x) = \frac{e^x}{e^x + C}.</math>

Choosing the constant of integration <math>C = 1</math> gives the other well known form of the definition of the logistic curve:

<math display="block">f(x) = \frac{e^x}{e^x + 1} = \frac{1}{1 + e^{-x}}.</math>

More quantitatively, as can be seen from the analytical solution, the logistic curve shows early [[exponential growth]] for negative argument, which reaches to linear growth of slope 1/4 for an argument near 0, then approaches 1 with an exponentially decaying gap.

The differential equation derived above is a special case of a general differential equation that only models the sigmoid function for <math>x > 0</math>. In many modeling applications, the more ''general form''<ref>Kyurkchiev, Nikolay, and Svetoslav Markov. "Sigmoid functions: some approximation and modelling aspects". LAP LAMBERT Academic Publishing, Saarbrucken (2015).</ref>  
<math display="block">\frac{df(x)}{dx} = \frac{k}{L} f(x)\big(L - f(x)\big), \quad f(0) = \frac {L} {1 + e^{k x_0}}</math> 
can be desirable. Its solution is the shifted and scaled [[sigmoid function]] <math>L \sigma \big(k(x - x_0)\big) = \frac {L} {1 + e^{-k(x - x_0)}}</math>.