Logarithmically concave function

Template:Short description In convex analysis, a non-negative function Template:Math is logarithmically concave (or log-concave for short) if its domain is a convex set, and if it satisfies the inequality

<math>

   f(\theta x + (1 - \theta) y) \geq f(x)^{\theta} f(y)^{1 - \theta}
 </math>

for all Template:Math and Template:Math. If Template:Math is strictly positive, this is equivalent to saying that the logarithm of the function, Template:Math, is concave; that is,

<math>

   \log  f(\theta x + (1 - \theta) y) \geq \theta \log f(x) + (1-\theta) \log f(y)
 </math>

for all Template:Math and Template:Math.

Examples of log-concave functions are the 0-1 indicator functions of convex sets (which requires the more flexible definition), and the Gaussian function.

Similarly, a function is log-convex if it satisfies the reverse inequality

<math>

   f(\theta x + (1 - \theta) y) \leq f(x)^{\theta} f(y)^{1 - \theta}
 </math>

for all Template:Math and Template:Math.

Properties

• A log-concave function is also quasi-concave. This follows from the fact that the logarithm is monotone implying that the superlevel sets of this function are convex.<ref name=":0" />
• Every concave function that is nonnegative on its domain is log-concave. However, the reverse does not necessarily hold. An example is the Gaussian function Template:Math = Template:Math which is log-concave since Template:Math = Template:Math is a concave function of Template:Math. But Template:Math is not concave since the second derivative is positive for |Template:Math| > 1:

<math>f(x)=e^{-\frac{x^2}{2}} (x^2-1) \nleq 0</math>

• From above two points, concavity <math>\Rightarrow</math> log-concavity <math>\Rightarrow</math> quasiconcavity.
• A twice differentiable, nonnegative function with a convex domain is log-concave if and only if for all Template:Math satisfying Template:Math,

<math>f(x)\nabla^2f(x) \preceq \nabla f(x)\nabla f(x)^T</math>,<ref name=":0">Template:Cite book</ref>

i.e.

<math>f(x)\nabla^2f(x) - \nabla f(x)\nabla f(x)^T</math> is

negative semi-definite. For functions of one variable, this condition simplifies to

<math>f(x)f(x) \leq (f'(x))^2</math>

Operations preserving log-concavity

• Products: The product of log-concave functions is also log-concave. Indeed, if Template:Math and Template:Math are log-concave functions, then Template:Math and Template:Math are concave by definition. Therefore

<math>\log\,f(x) + \log\,g(x) = \log(f(x)g(x))</math>

is concave, and hence also Template:Math is log-concave.

• Marginals: if Template:Math : Template:Math is log-concave, then

<math>g(x)=\int f(x,y) dy</math>

is log-concave (see Prékopa–Leindler inequality).

• This implies that convolution preserves log-concavity, since Template:Math = Template:Math is log-concave if Template:Math and Template:Math are log-concave, and therefore

<math>(f*g)(x)=\int f(x-y)g(y) dy = \int h(x,y) dy</math>

is log-concave.

Log-concave distributions

Log-concave distributions are necessary for a number of algorithms, e.g. adaptive rejection sampling. Every distribution with log-concave density is a maximum entropy probability distribution with specified mean μ and Deviation risk measure D.<ref name="Grechuk1">Template:Cite journal</ref> As it happens, many common probability distributions are log-concave. Some examples:<ref name=":1">See Template:Cite journal</ref>

• the normal distribution and multivariate normal distributions,
• the exponential distribution,
• the uniform distribution over any convex set,
• the logistic distribution,
• the extreme value distribution,
• the Laplace distribution,
• the chi distribution,
• the hyperbolic secant distribution,
• the Wishart distribution, if n ≥ p + 1,<ref name="prekopa">Template:Cite journal</ref>
• the Dirichlet distribution, if all parameters are ≥ 1,<ref name="prekopa"/>
• the gamma distribution if the shape parameter is ≥ 1,
• the chi-square distribution if the number of degrees of freedom is ≥ 2,
• the beta distribution if both shape parameters are ≥ 1, and
• the Weibull distribution if the shape parameter is ≥ 1.

Note that all of the parameter restrictions have the same basic source: The exponent of non-negative quantity must be non-negative in order for the function to be log-concave.

The following distributions are non-log-concave for all parameters:

• the Student's t-distribution,
• the Cauchy distribution,
• the Pareto distribution,
• the log-normal distribution, and
• the F-distribution.

Note that the cumulative distribution function (CDF) of all log-concave distributions is also log-concave. However, some non-log-concave distributions also have log-concave CDF's:

• the log-normal distribution,
• the Pareto distribution,
• the Weibull distribution when the shape parameter < 1, and
• the gamma distribution when the shape parameter < 1.

The following are among the properties of log-concave distributions:

• If a density is log-concave, so is its cumulative distribution function (CDF).
• If a multivariate density is log-concave, so is the marginal density over any subset of variables.
• The sum of two independent log-concave random variables is log-concave. This follows from the fact that the convolution of two log-concave functions is log-concave.
• The product of two log-concave functions is log-concave. This means that joint densities formed by multiplying two probability densities (e.g. the normal-gamma distribution, which always has a shape parameter ≥ 1) will be log-concave. This property is heavily used in general-purpose Gibbs sampling programs such as BUGS and JAGS, which are thereby able to use adaptive rejection sampling over a wide variety of conditional distributions derived from the product of other distributions.
• If a density is log-concave, so is its survival function.<ref name=":1" />
• If a density is log-concave, it has a monotone hazard rate (MHR), and is a regular distribution since the derivative of the logarithm of the survival function is the negative hazard rate, and by concavity is monotone i.e.

<math>\frac{d}{dx}\log\left(1-F(x)\right) = -\frac{f(x)}{1-F(x)}</math> which is decreasing as it is the derivative of a concave function.

See also

• logarithmically concave sequence
• logarithmically concave measure
• logarithmically convex function
• convex function

References

• Template:Cite book
• Template:Cite book

• Template:Cite book

• Template:Cite book