Bernstein polynomial

File:Bernstein Approximation.gif

Bernstein polynomials approximating a curve

In the mathematical field of numerical analysis, a Bernstein polynomial is a polynomial expressed as a linear combination of Bernstein basis polynomials. The idea is named after mathematician Sergei Natanovich Bernstein.

Polynomials in Bernstein form were first used by Bernstein in a constructive proof for the Weierstrass approximation theorem. With the advent of computer graphics, Bernstein polynomials, restricted to the interval [0, 1], became important in the form of Bézier curves.

A numerically stable way to evaluate polynomials in Bernstein form is de Casteljau's algorithm.

File:Bernstein Polynomials.svg

Bernstein basis polynomials for 4th degree curve blending

Definition

Bernstein basis polynomials

The <math>\ n + 1\ </math> Bernstein basis polynomials of degree <math>\ n\ </math> are defined as

<math>\ b_{\nu,n}(x)\ \equiv\ \binom{n}{\nu}\ x^{\nu} \left( 1 - x \right)^{n - \nu}\ , ~~</math> for <math>~~ \nu = 0\ ,\ \ldots\ , n\ ,</math>

where <math>\ \tbinom{n}{\nu}\ </math> is a binomial coefficient.

So, for example, <math>\ b_{2,5}(x)\ =\ \tbinom{5}{2}x^2(1-x)^3\ =\ 10x^2(1-x)^3 ~.</math>

The first few Bernstein basis polynomials for blending Template:Math or Template:Math values together are:

<math>

\begin{align} b_{0,0}(x) & = 1\ , \\ b_{0,1}(x) & = 1 - x\ , & b_{1,1}(x) & = x \\ b_{0,2}(x) & = (1 - x)^2\ , & b_{1,2}(x) & = 2x(1 - x)\ , & b_{2,2}(x) & = x^2 \\ b_{0,3}(x) & = (1 - x)^3\ , & b_{1,3}(x) & = 3x(1 - x)^2\ , & b_{2,3}(x) & = 3x^2(1 - x)\ , & b_{3,3}(x) & = x^3 ~. \end{align} </math>

The Bernstein basis polynomials of degree <math>\ n\ </math> form a basis for the vector space <math>\ \Pi_n\ </math> of polynomials of degree at most <math>\ n\ ,</math> all with real coefficients.

Bernstein polynomials

A linear combination of Bernstein basis polynomials

<math>\ B_n(x)\ \equiv\ \sum_{\nu=0}^{n} \beta_{\nu} b_{\nu,n}(x)\ </math>

is called a Bernstein polynomial or polynomial in Bernstein form of degree <math>\ n ~.</math><ref name="Lorentz">Template:Harvnb</ref> The coefficients <math>\ \beta_\nu\ </math> are called Bernstein coefficients or Bézier coefficients.

The first few Bernstein basis polynomials from above in monomial form are:

<math>

\begin{align} b_{0,0}(x) & = 1\ , \\ b_{0,1}(x) & = 1 - 1x\ , & b_{1,1}(x) & = 0 + 1x \\ b_{0,2}(x) & = 1 - 2x + 1x^2, & b_{1,2}(x) & = 0 + 2x - 2x^2\ , & b_{2,2}(x) & = 0 + 0x + 1x^2 \\ b_{0,3}(x) & = 1 - 3x + 3x^2 - 1x^3\ , & b_{1,3}(x) & = 0 + 3x - 6x^2 + 3x^3\ , & b_{2,3}(x) & = 0 + 0x + 3x^2 - 3x^3, & b_{3,3}(x) & = 0 + 0x + 0x^2 + 1x^3 ~. \end{align} </math>

Properties

The Bernstein basis polynomials have the following properties:

• <math>\ b_{\nu, n}\!(x) \equiv 0\ ,</math> if <math>\ \nu < 0\ </math> or if <math>\ \nu > n ~.</math>
• <math>\ b_{\nu, n}\!(x) \ge 0\ </math> for <math>\ x \in [0,\ 1] ~.</math>
• <math>\ b_{\nu, n}\!\left( 1 - x \right) = b_{n - \nu, n}\!(x) ~.</math>
• <math>\ b_{\nu, n}\!(0) = \delta_{\nu, 0}\ </math> and <math>\ b_{\nu, n}\!(1) = \delta_{\nu, n}\ </math> where <math>\ \delta_{i,j}\ </math> is the Kronecker delta function: <math>\ \delta_{ij} \equiv \begin{cases}

0 &\text{if } i \neq j\ , \\ 1 &\text{if } i=j ~.

\end{cases}</math>

• <math>\ b_{\nu, n}\!(x)\ </math> has a root with multiplicity <math>\ \nu\ </math> at point <math>\ x = 0\ </math> (note: when <math>\ \nu = 0\ ,</math> there is no root at Template:Math).
• <math>\ b_{\nu, n}\!(x)\ </math> has a root with multiplicity <math>\ \left( n - \nu \right)\ </math> at point <math>\ x = 1\ </math> (note: if <math>\ \nu = n\ ,</math> there is no root at Template:Math).
• The derivative can be written as a combination of two polynomials of lower degree: <math display="block">\ b_{\nu, n}'\!(x) = n \bigl[\ b_{\nu - 1, n - 1}\!(x)\ -\ b_{\nu, n - 1}\!(x)\ \bigr] ~.</math>
• The Template:Mvar-th derivative at Template:Math: <math display="block">\ b_{\nu, n}^{(k)}\!(0)\ =\ \frac{n!}{(n - k)!} \binom{k}{\nu} (-1)^{\nu + k} ~.</math>
• The Template:Mvar-th derivative at 1: <math display="block">\ b_{\nu, n}^{(k)}(1)\ =\ (-1)^k b_{n - \nu, n}^{(k)}(0) ~.</math>
• The transformation of the Bernstein polynomial to monomials is <math display="block">\ b_{\nu,n}\!(x)\ =\ \binom{n}{\nu}\sum_{k=0}^{n-\nu} \binom{n-\nu}{k}(-1)^{n-\nu-k} x^{\nu+k}\ =\ \sum_{\ell=\nu}^n \binom{n}{\ell}\binom{\ell}{\nu}(-1)^{\ell-\nu}x^\ell\ ,</math> and by the inverse binomial transformation, the reverse transformation is<ref>Template:Cite arXiv</ref> <math display="block">\ x^k\ =\ \sum_{i=0}^{n-k} \frac{ \binom{n-k}{i} }{ \binom{n}{i} } b_{n-i,n}\!(x)\ =\ \frac{1}{\binom{n}{k}} \sum_{j=k}^n \binom{j}{k}b_{j,n}\!(x) ~.</math>
• The indefinite integral is given by <math display="block">\ \int b_{\nu, n}\!(x)\ \operatorname{d} x = \frac{1}{n+1} \sum_{j=\nu+1}^{n+1} b_{j, n+1}\!(x) ~.</math>
• The definite integral is constant for a given Template:Mvar: <math display="block">\ \int_0^1 b_{\nu, n}\!(x)\ \operatorname{d} x = \frac{1}{n+1} ~~</math> for all <math>~~ \nu = 0, 1,\ \dots\ , n ~.</math>
• If <math>\ n \ne 0\ , ~</math> then <math>~~ b_{\nu, n}\!(x)\ </math> has a unique local maximum on the interval <math>\ [0,\, 1]\ </math> at <math>\ x = \frac{\nu}{n} ~.</math> This maximum takes the value <math display="block">\ \nu^\nu n^{-n} \left( n - \nu \right)^{n - \nu} {n \choose \nu} ~.</math>
• The Bernstein basis polynomials of degree <math>\ n\ </math> form a partition of unity: <math display="block">\ \sum_{\nu = 0}^n b_{\nu, n}(x)\ =\ \sum_{\nu = 0}^n {n \choose \nu} x^\nu \left(1 - x\right)^{n - \nu}\ =\ \left(x + \left( 1 - x \right) \right)^n = 1 ~.</math>
• By taking the first <math>x</math>-derivative of <math>\ (x + y)^n\ ,</math> treating <math>\ y\ </math> as constant, then substituting the value <math>\ y = 1-x\ ,</math> it can be shown that <math display="block">\ \sum_{\nu=0}^{n} \nu\ b_{\nu, n}\!(x) = n\ x ~.</math>
• Similarly the second <math>\ x\ </math>-derivative of <math>\ (x+y)^n\ ,</math> with <math>\ y\ </math> then again substituted <math>\ y = 1-x\ ,</math> shows that <math display="block">\ \sum_{\nu=1}^{n} \nu \left( \nu-1 \right)\ b_{\nu, n}\!(x) = n\left( n-1 \right)\ x^2 ~.</math>
• A Bernstein polynomial can always be written as a linear combination of polynomials of higher degree: <math display="block">\ b_{\nu, n - 1}\!(x)\ =\ \left( \frac{n - \nu}{n}\right)\ b_{\nu, n}\!(x)\ +\ \left( \frac{\nu + 1}{n}\right)\ b_{\nu + 1, n}\!(x) ~.</math>
• The expansion of the Chebyshev Polynomials of the First Kind into the Bernstein basis is<ref>Template:Cite journal</ref> <math display="block">\ T_n\!(u)\ =\ (2n-1)!!\ \sum_{k=0}^n \frac{~ (-1)^{n-k}\ }{\ (2k-1)!!\ (2n-2k-1)!!\ }\ b_{k,n}\!(u) ~.</math>

Approximating continuous functions

Let ƒ be a continuous function on the interval [0, 1]. Consider the Bernstein polynomial

<math>B_n(f)(x) = \sum_{\nu = 0}^n f\left( \frac{\nu}{n} \right) b_{\nu,n}(x).</math>

It can be shown that

<math>\lim_{n \to \infty}{ B_n(f) } = f </math>

uniformly on the interval [0, 1].<ref name=Nat6>Natanson (1964) p. 6</ref><ref name="Lorentz"/><ref name="Feller 1966">Template:Harvnb</ref><ref name="Beals 2004">Template:Harvnb</ref>

Bernstein polynomials thus provide one way to prove the Weierstrass approximation theorem that every real-valued continuous function on a real interval [a, b] can be uniformly approximated by polynomial functions over <math>\mathbb R</math>.<ref name=Nat3>Natanson (1964) p. 3</ref>

A more general statement for a function with continuous k^th derivative is

<math>{\left\| B_n(f)^{(k)} \right\|}_\infty \le \frac{ (n)_k }{ n^k } \left\| f^{(k)} \right\|_\infty \quad\ \text{and} \quad\ \left\| f^{(k)}- B_n(f)^{(k)} \right\|_\infty \to 0,</math>

where additionally

<math>\frac{ (n)_k }{ n^k } = \left( 1 - \frac{0}{n} \right) \left( 1 - \frac{1}{n} \right) \cdots \left( 1 - \frac{k - 1}{n} \right)</math>

is an eigenvalue of B_n; the corresponding eigenfunction is a polynomial of degree k.

Probabilistic proof

This proof follows Bernstein's original proof of 1912.<ref>Template:Harvnb</ref> See also Feller (1966) or Koralov & Sinai (2007).<ref>Template:Cite book</ref><ref name="Feller 1966"/>

Motivation

We will first give intuition for Bernstein's original proof. A continuous function on a compact interval must be uniformly continuous. Thus, the value of any continuous function can be uniformly approximated by its value on some finite net of points in the interval. This consideration renders the approximation theorem intuitive, given that polynomials should be flexible enough to match (or nearly match) a finite number of pairs <math>(x, f(x))</math>. To do so, we might (1) construct a function close to <math>f</math> on a lattice, and then (2) smooth out the function outside the lattice to make a polynomial.

The probabilistic proof below simply provides a constructive method to create a polynomial which is approximately equal to <math>f</math> on such a point lattice, given that "smoothing out" a function is not always trivial. Taking the expectation of a random variable with a simple distribution is a common way to smooth. Here, we take advantage of the fact that Bernstein polynomials look like Binomial expectations. We split the interval into a lattice of n discrete values. Then, to evaluate any f(x), we evaluate f at one of the n lattice points close to x, randomly chosen by the Binomial distribution. The expectation of this approximation technique is polynomial, as it is the expectation of a function of a binomial RV. The proof below illustrates that this achieves a uniform approximation of f. The crux of the proof is to (1) justify replacing an arbitrary point with a binomially chosen lattice point by concentration properties of a Binomial distribution, and (2) justify the inference from <math>x \approx X</math> to <math>f(x) \approx f(X)</math> by uniform continuity.

Bernstein's proof

Suppose K is a random variable distributed as the number of successes in n independent Bernoulli trials with probability x of success on each trial; in other words, K has a binomial distribution with parameters n and x. Then we have the expected value <math>\operatorname{\mathcal E}\left[\frac{K}{n}\right] = x\ </math> and

<math>p(K) = {n \choose K} x^{K} \left( 1 - x \right)^{n - K} = b_{K,n}(x)</math>

By the weak law of large numbers of probability theory,

<math>\lim_{n \to \infty}{ P\left( \left| \frac{K}{n} - x \right|>\delta \right) } = 0</math>

for every δ > 0. Moreover, this relation holds uniformly in x, which can be seen from its proof via Chebyshev's inequality, taking into account that the variance of Template:Frac K, equal to Template:Frac x(1−x), is bounded from above by Template:Frac irrespective of x.

Because ƒ, being continuous on a closed bounded interval, must be uniformly continuous on that interval, one infers a statement of the form

<math>\lim_{n \to \infty}{ P\left( \left| f\left( \frac{K}{n} \right) - f\left( x \right) \right| > \varepsilon \right) } = 0</math>

uniformly in x for each <math>\epsilon > 0</math>. Taking into account that ƒ is bounded (on the given interval) one finds that

<math>\lim_{n \to \infty}{ \operatorname{\mathcal E}\left( \left| f\left( \frac{K}{n} \right) - f\left( x \right) \right| \right) } = 0</math>

uniformly in x. To justify this statement, we use a common method in probability theory to convert from closeness in probability to closeness in expectation. One splits the expectation of <math>\left| f\left( \frac{K}{n} \right) - f\left( x \right) \right|</math> into two parts split based on whether or not <math>\left| f\left( \frac{K}{n} \right) - f\left( x \right) \right| < \epsilon</math>. In the interval where the difference does not exceed ε, the expectation clearly cannot exceed ε. In the other interval, the difference still cannot exceed 2M, where M is an upper bound for |ƒ(x)| (since uniformly continuous functions are bounded). However, by our 'closeness in probability' statement, this interval cannot have probability greater than ε. Thus, this part of the expectation contributes no more than 2M times ε. Then the total expectation is no more than <math>\epsilon + 2M\epsilon</math>, which can be made arbitrarily small by choosing small ε.

Finally, one observes that the absolute value of the difference between expectations never exceeds the expectation of the absolute value of the difference, a consequence of Holder's Inequality. Thus, using the above expectation, we see that (uniformly in x)

<math>\lim_{n \to \infty}{ \left| \operatorname{\mathcal E}f\left( \frac{K}{n} \right) - \operatorname{\mathcal E}f\left( x \right) \right| } \leq \lim_{n \to \infty}{ \operatorname{\mathcal E}\left( \left| f\left( \frac{K}{n} \right) - f\left( x \right) \right| \right) } = 0</math>

Noting that our randomness was over K while x is constant, the expectation of f(x) is just equal to f(x). But then we have shown that <math>\operatorname{\mathcal E_x}f\left( \frac{K}{n} \right)</math> converges to f(x). Then we will be done if <math>\operatorname{\mathcal E_x}f\left( \frac{K}{n} \right)</math> is a polynomial in x (the subscript reminding us that x controls the distribution of K). Indeed it is:

<math>\operatorname{\mathcal E_x}\left[f\left(\frac{K}{n}\right)\right] = \sum_{K=0}^n f\left(\frac{K}{n}\right) p(K) = \sum_{K=0}^n f\left(\frac{K}{n}\right) b_{K,n}(x) = B_n(f)(x)</math>

Uniform convergence rates between functions

In the above proof, recall that convergence in each limit involving f depends on the uniform continuity of f, which implies a rate of convergence dependent on f 's modulus of continuity <math>\omega.</math> It also depends on 'M', the absolute bound of the function, although this can be bypassed if one bounds <math>\omega</math> and the interval size. Thus, the approximation only holds uniformly across x for a fixed f, but one can readily extend the proof to uniformly approximate a set of functions with a set of Bernstein polynomials in the context of equicontinuity.

Elementary proof

The probabilistic proof can also be rephrased in an elementary way, using the underlying probabilistic ideas but proceeding by direct verification:<ref>Template:Harvnb</ref><ref name="Beals 2004"/><ref>Template:Harvnb</ref><ref>Template:Harvnb</ref><ref>Template:Harvnb</ref>

The following identities can be verified:

1. <math> \sum_k {n \choose k} x^k (1-x)^{n-k} = 1</math> ("probability")
2. <math> \sum_k {k\over n} {n \choose k} x^k (1-x)^{n-k} = x</math> ("mean")
3. <math> \sum_k \left( x -{k\over n}\right)^2 {n \choose k} x^k (1-x)^{n-k} = {x(1-x)\over n}. </math> ("variance")

In fact, by the binomial theorem

<math display="block">(1+t)^n = \sum_k {n \choose k} t^k,</math>

and this equation can be applied twice to <math>t\frac{d}{dt}</math>. The identities (1), (2), and (3) follow easily using the substitution <math>t = x/ (1 - x)</math>.

Within these three identities, use the above basis polynomial notation

<math> b_{k,n}(x) = {n\choose k} x^k (1-x)^{n-k},</math>

and let

<math> f_n(x) = \sum_k f(k/n)\, b_{k,n}(x).</math>

Thus, by identity (1)

<math>f_n(x) - f(x) = \sum_k [f(k/n) - f(x)] \,b_{k,n}(x), </math>

so that

<math>|f_n(x) - f(x)| \le \sum_k |f(k/n) - f(x)| \, b_{k,n}(x).</math>

Since f is uniformly continuous, given <math>\varepsilon > 0</math>, there is a <math>\delta > 0</math> such that <math>|f(a) - f(b)| < \varepsilon</math> whenever <math>|a-b| < \delta</math>. Moreover, by continuity, <math>M= \sup |f| < \infty</math>. But then

<math> |f_n(x) - f(x)| \le \sum_{|x -{k\over n}|< \delta} |f(k/n) - f(x)|\, b_{k,n}(x) + \sum_{|x -{k\over n}|\ge \delta} |f(k/n) - f(x)|\, b_{k,n}(x) .</math>

The first sum is less than ε. On the other hand, by identity (3) above, and since <math>|x - k/n| \ge \delta</math>, the second sum is bounded by <math>2M</math> times

<math>\sum_{|x - k/n|\ge \delta} b_{k,n}(x) \le \sum_k \delta^{-2} \left(x -{k\over n}\right)^2 b_{k,n}(x) = \delta^{-2} {x(1-x)\over n} < {1\over4} \delta^{-2} n^{-1}.</math>

(Chebyshev's inequality)

It follows that the polynomials f_n tend to f uniformly.

Generalizations to higher dimension

Bernstein polynomials can be generalized to Template:Math dimensions – the resulting polynomials have the form Template:Math.<ref name="Lorentz"/> In the simplest case only products of the unit interval Template:Math are considered; but, using affine transformations of the line, Bernstein polynomials can also be defined for products Template:Math. For a continuous function Template:Math on the Template:Math-fold product of the unit interval, the proof that Template:Math can be uniformly approximated by

<math>\sum_{i_1} \sum_{i_2} \cdots \sum_{i_k} {n_1\choose i_1} {n_2\choose i_2} \cdots {n_k\choose i_k}

f\left({i_1\over n_1}, {i_2\over n_2}, \dots, {i_k\over n_k}\right) x_1^{i_1} (1-x_1)^{n_1-i_1} x_2^{i_2} (1-x_2)^{n_2-i_2} \cdots x_k^{i_k} (1-x_k)^{n_k - i_k} </math>

is a straightforward extension of Bernstein's proof in one dimension. <ref>Template:Citation</ref>

See also

• Polynomial interpolation
• Newton form
• Lagrange form
• Binomial QMF (also known as Daubechies wavelet)

Notes

<references />

References

• Template:Citation, English translation
• Template:Citation
• Template:Citation, Russian edition first published in 1940
• Template:Citation
• Template:Citation
• Template:Cite journal
• Template:Springer
• Template:Cite book
• Template:Citation
• Template:Citation

External links

• Template:Cite journal
• Template:Cite journal
• Template:Cite book
• Template:Cite journal
• Template:Cite journal
• {{#invoke:citation/CS1|citation

|CitationClass=web }} from University of California, Davis. Note the error in the summation limits in the first formula on page 9.

• Template:Cite journal
• {{#invoke:citation/CS1|citation

|CitationClass=web }} Feature Column from American Mathematical Society

• Template:Cite journal
• Template:Cite journal
• Template:Cite journal
• Template:Cite journal
• Template:Mathworld
• {{#if:

| This article incorporates material from the following PlanetMath articles, which are licensed under the Creative Commons Attribution/Share-Alike License: {{#if: BernsteinPolynomial | properties of Bernstein polynomial | {{#if: | properties of Bernstein polynomial | [{{{sourceurl}}} properties of Bernstein polynomial] }} }}, {{#if: | {{{title2}}} | {{#if: | {{{title2}}} | [{{{sourceurl2}}} {{{title2}}}] }} }}{{#if: | , {{#if: | {{{title3}}} | {{#if: | {{{title3}}} | [{{{sourceurl3}}} {{{title3}}}] }} }} }}{{#if: | , {{#if: | {{{title4}}} | {{#if: | {{{title4}}} | [{{{sourceurl4}}} {{{title4}}}] }} }} }}{{#if: | , {{#if: | {{{title5}}} | {{#if: | {{{title5}}} | [{{{sourceurl5}}} {{{title5}}}] }} }} }}{{#if: | , {{#if: | {{{title6}}} | {{#if: | {{{title6}}} | [{{{sourceurl6}}} {{{title6}}}] }} }} }}{{#if: | , {{#if: | {{{title7}}} | {{#if: | {{{title7}}} | [{{{sourceurl7}}} {{{title7}}}] }} }} }}{{#if: | , {{#if: | {{{title8}}} | {{#if: | {{{title8}}} | [{{{sourceurl8}}} {{{title8}}}] }} }} }}{{#if: | , {{#if: | {{{title9}}} | {{#if: | {{{title9}}} | [{{{sourceurl9}}} {{{title9}}}] }} }} }}. | This article incorporates material from {{#if: BernsteinPolynomial | properties of Bernstein polynomial | properties of Bernstein polynomial}} on PlanetMath, which is licensed under the Creative Commons Attribution/Share-Alike License. }}