Advanced Search

Browse by Discipline

Scientific Societies

E-print Alerts

Add E-prints

E-print Network

  Advanced Search  

Auxiliary Variational Information Maximization for Dimensionality Reduction

Summary: Auxiliary Variational Information Maximization
for Dimensionality Reduction
Felix Agakov1
and David Barber2
University of Edinburgh, 5 Forrest Hill, EH1 2QL Edinburgh, UK
felixa@inf.ed.ac.uk, www.anc.ed.ac.uk
IDIAP, Rue du Simplon 4, CH-1920 Martigny Switzerland,
Abstract. Mutual Information (MI) is a long studied measure of in-
formation content, and many attempts to apply it to feature extrac-
tion and stochastic coding have been made. However, in general MI is
computationally intractable to evaluate, and most previous studies re-
define the criterion in forms of approximations. Recently we described
properties of a simple lower bound on MI, and discussed its links to
some of the popular dimensionality reduction techniques. Here we in-
troduce a richer family of auxiliary variational bounds on MI, which
generalizes our previous approximations. Our specific focus then is on
applying the bound to extracting informative lower-dimensional projec-


Source: Agakov, Felix - Institute for Adaptive and Neural Computation, School of Informatics, University of Edinburgh


Collections: Computer Technologies and Information Sciences