A New Estimator of Kullback--Leibler Divergence via Shannon Entropy
We examine the estimation of the Kullback-Leibler (KL) divergence and the use of the goodness-of-fit test for multivariate continuous distributions. Our starting point is the maximum entropy principle for Shannon entropy: among all distributions with a fixed mean vector and covariance matrix, the multivariate Gaussian distributions uniquely maximize entropy. As a result, the KL divergence from a moment-matched Gaussian distribution to an unknown density can then be written as the \emph{entropy difference}, which is a suitable information-theoretic measure of divergence from the Gaussian…
Comments and reviews
No comments yet.
API
curl /api/v1/records/arxiv/2603.06759
curl -X POST /api/v1/records/arxiv/2603.06759/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
-d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2603.06759/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
-d '{"body": "…", "review": true}'