Summary: Temporal Hidden Hopfield Models
Felix V. Agakov and David Barber
Division of Informatics, University of Edinburgh, Edinburgh EH1 2QL, UK
felixa@anc.ed.ac.uk, dbarber@anc.ed.ac.uk
http://anc.ed.ac.uk
November 5, 2002
Abstract
Many popular probabilistic models for temporal sequences assume simple hidden dy­
namics or low­dimensionality of discrete variables. For higher dimensional discrete hidden
variables, recourse is often made to approximate mean field theories, which to date have been
applied to models with only simple hidden unit dynamics. We consider a class of models in
which the discrete hidden space is defined by parallel dynamics of densely connected high­
dimensional stochastic Hopfield networks. For these Hidden Hopfield Models (HHMs), mean
field methods are derived for learning discrete and continuous temporal sequences. We dis­
cuss applications of HHMs to classification and reconstruction of non­stationary time series.
We also demonstrate a few problems (e.g. learning of incomplete binary sequences and re­
construction of 3D occupancy graphs) where distributed discrete hidden space representation
may be useful.
1 Markovian Dynamics for Temporal Sequences
Dynamic Bayesian networks are popular tools for modeling temporally correlated patterns. In­

Source: Agakov, Felix - Institute for Adaptive and Neural Computation, School of Informatics, University of Edinburgh

Collections: Computer Technologies and Information Sciences