Phi-entropy inequality

A probability measure $\mu$ satisfies a $\Phi$-entropy inequality with constant $C>0$ and for a given smooth convex function $\Phi$ on a interval $I$ if for every $I$-valued smooth function $f$,

$$\mathrm{Ent}^\Phi_\mu(f)=\int \Phi(f)d\mu -\Phi\left(\int fd\mu\right)\leq C\int \Phi''(f)|\nabla f|^2 d\mu.$$

The quantity $\mathrm{Ent}^\Phi_\mu(f)$, called the $\Phi$-entropy of $f$ for $\mu$, is non-negative since $\Phi$ is convex, by virtue of Jensen's inequality. The case where $\Phi$ is affine is trivial since the $\Phi$-entropy is then identically zero.

Both Poincaré, logarithmic Sobolev and Beckner inequalities are particular cases of the $\Phi$-entropy inequality, corresponding to the functions $\Phi$ given by $u\mapsto u^2$, $u\mapsto u\log(u)$ and $u\mapsto u^p$ with $1<p\leq 2$ respectively. If $\Phi$ is not an affine function then the $\Phi$-entropy inequality implies a Poincaré inequality with constant $C$ (take $f=1+\epsilon g$ and let $\epsilon$ goes to $0$).

Open problems

$\Phi$-entropy inequalities allow to interpolate between a Poincaré inequality and logarithmic Sobolev inequality, which is stronger. Also, one can ask these two natural questions:

Role of convexity

Many available methods used to derive $\Phi$-entropy inequalities rely crucially on additional assumptions on $\Phi$ related to convexity. It is shown in [C2, Theorem 4.4] that the following statements are equivalent:

The convexity of $C^\Phi$ gives $\Phi$-entropy inequalities for diffusions while the convexity of $A^\Phi$ gives modified versions for discrete space processes such as Poisson space and Lévy processes. The $B^\Phi$ transform appears naturally in a de Bruijn like identity for the Poisson process, and in the exponential decay of the $\Phi$-entropy along certain discrete space processes such as the M/M/$\infty$ queue. The tensorization property of the $\Phi$-entropy functional can be used to obtain $\Phi$-entropy inequalities for Gauss and Poisson laws from the two-point space, as Gross did for the logarithmic Sobolev inequality in his seminal paper [G], see [C2] for more details.

A bit of history

A general framework for $\Phi$-entropy inequalities was proposed in [C1, C2] to unify and generalize a bunch of results concerning entropic inequalities. The notion of $\Phi$-entropy, called $\Phi$-divergence or Jensen divergence, goes back at least to I. Csiszár [Cs]. The quantity $A^\Phi(u,v)$ is known in convex analysis as a Bregman distance between $u$ and $u+v$ [Br]. The convexity condition involving the $t$ parameter appears in a work of R. Lataƚa and K. Oleszkiewicz [LO], see also [H]. The variational formula for $\Phi$-entropies and its usage for model selection via concentration was considered by S. Boucheron, O. Bousquet, G. Lugosi, and P. Massart [M, BBLM]. Various $\Phi$-entropic inequalities for diffusions where already known by D. Bakry and others, see e.g. [H, Ba, AMTU]. Some additional aspects can be found in the more recent works [MC, RZ, ABD, GI, BG].

Examples

The left hand side of the $\Phi$-entropy inequality above involves the $C^\Phi$ transform of $\Phi$. If $C^\Phi$ is convex, it is shown in [C1] (see also [H, Ba, AMTU]) that the $\Phi$-entropy inequality holds in many Gaussian like situations, including: Gauss law, uniformly log-concave laws, law at time $t$ of diffusions with positive $\Gamma_{\!\!2}$-curvature (i.e. local inequalities) (includes the Ornstein-Uhlenbeck process), as well as Wiener measure of Brownian motion on manifold with bounded below positive $\Gamma_{\!\!2}$-curvature. This last result generalizes [CHL]. See also [MC] and [BI] for more recent advances. Only Gaussian like laws satisfy to all $\Phi$ entropy inequalities for all convex $\Phi$ such that $C^\Phi$ is convex. For families of laws between the exponential law and the Gaussian law, only a limited subset of $\Phi$ entropy inequalities are possible, see e.g. [BCR1, BCR2].

Concentration, Orlicz hypercontractivity, isoperimetry

For all these aspects, see e.g. [LO, M, BBLM, BCR1, BCR2] and references therein.

Stabilities

The tensorization formula of the $\Phi$-entropy mentioned above implies that the $\Phi$-entropy inequality is stable by tensor product as soon as $C^\Phi$ is convex. This means that $\mu^{\otimes n}$ satisfies the same inequality with the same constant, for every $n$. As for the Poincaré inequality and the logarithmic Sobolev inequality, the $\Phi$-entropy inequality is also stable by the action of the translation/orthogonal group on $\mu$, by the action of Lipschitz maps on $\mu$, and also by bounded perturbations on the log-density of $\mu$. These stabilities do not require that $C^\Phi$ is convex. For more details, see e.g. [C1]. Some additional stabilities are considered in [BLW].

Discrete setting

By replacing the $C^\Phi$ transform by the $A^\Phi$ transform in the right hand side and the continuous gradient by a finite difference operator, the corresponding $\Phi$-entropy inequality holds true for the Poisson law, the paths space of the Poisson process, as well as for Poisson space and many Lévy processes and the corresponding infinitely divisible law. This generalizes [W]. See [GI] for further generalizations in the same spirit. It is shown in [C2] that such $\Phi$-entropy inequaities hold true for Binomial-Poisson laws and the M/M/$\infty$ queue, which appears as a discrete space analogue of the Ornstein-Uhlenbeck process. This generalizes a result of [BL].

Variance and entropy as extremal cases

The functions $u\mapsto u\log(u)$ and $u\mapsto u^2$ appear as extremal cases of the convexity assumptions on $C^\Phi$. The set of convex $\Phi$ functions for which $C^\Phi$ is convex is a convex cone, see [C1] and [C2, Section 4] for more details.

More original works

See for instance [W2].

Bibliography

Some mistakes in [C1] are corrected in [C2]. Some aspects are also developed in [C3].

CategoryEncyclopedia

Phi-entropy inequality (last edited 2010-05-30 21:43:09 by DjalilChafai)