Latent MI (LMI) approximation is a method for estimating mutual information in high dimensions, using the idea that real world high-dimensional data has underlying low-dimensional structure. For more details, see our manuscript (NeurIPS 2024 spotlight 🌟). latentmi is our Python implementation of LMI approximation, built with the hope that practicioners find it pleasant to use :)
$ pip install latentmifrom latentmi import lmi
Xs = # some samples of a high dimensional variable
Ys = # some samples of a high dimensional variable
pmis, embedding, model = lmi.estimate(Xs, Ys)
MI_estimate = np.nanmean(pmis) # voila !More detailed instructions can be found in the demo notebook.
latentmi was created by Gokul Gowri. It is licensed under the terms of the MIT license.
latentmi was created with cookiecutter and the py-pkgs-cookiecutter template.
The implementation of the ksg estimator is adapted from Greg Ver Steeg's Nonparametric Entropy Estimation Toolbox.