We calculate the average differential entropy of a $q$-component Gaussian mixture in $\mathbb R^n$. For simplicity, all components have covariance matrix $\sigma^2 { \mathbf 1}$, while the means $\{ \mathbf{ W}_i\}_{ i=1}^{ q}$ are i.i.d. Gaussian vectors with zero mean and covariance $s^2 { \mathbf 1}$. We obtain a series expansion in $\mu=s^2/\sigma^2$ for the average differential entropy up to order $\mathcal{ O}(\mu^2)$, and we provide a recipe to calculate higher order terms. Our result provides an analytic approximation with a quantifiable order of magnitude for the error, which is not achieved in previous literature.

Categories: cs.IT math.IT

Average entropy of Gaussian mixtures