GPDBN: A Smooth Generative Model of Shape with Uncertainty Propagation

Alessandro Di Martino
University of Bath
Erik Bodin
University of Bristol
Carl Henrik Ek
University of Bristol
Neill D.F. Campbell
University of Bath

Please try the interative demo below!

Overview

The shape of an object is an important characteristic for many vision problems such as segmentation, detection and tracking. Being independent of appearance, it is possible to generalize to a large range of objects from only small amounts of data. However, shapes represented as silhouette images are challenging to model due to complicated likelihood functions leading to intractable posteriors. In this paper we present a generative model of shapes which provides a low dimensional latent encoding which importantly resides on a smooth manifold with respect to the silhouette images. The proposed model propagates uncertainty in a principled manner allowing it to learn from small amounts of data and pro- viding predictions with associated uncertainty. We provide experiments that show how our proposed model provides favorable quantitative results compared with the state-of-the-art while simultaneously providing a representation that resides on a low-dimensional interpretable manifold.

Model

...
The GPDBN model structure. Graphical representations of the (a) Restricted Boltzmann Machine (RBM), (b) Deep Belief Network (DBN) and (c) the Shape Boltzmann Machine (SBM) of [Eslami et al]. (d) The GPDBN combines the benefits of a Gaussian Process Latent Variable Model (GPLVM) with a trainable, stochastic likelihood from a DBN; here X represents the latent variables, H the Gaussian activations, and V the observed (data) space.
...
An example GPDBN shape manifold with comparisons to other approaches. (a) The manifold learned by the GPDBN model on the Weizmann horse dataset. Moving over the manifold changes the pose of the horse with smooth paths in the manifold producing smooth transitions in silhouette pose. The heat map encodes the predictive variance of the model with darker regions indicating higher uncertainty and lower confidence in the silhouette estimates. (b) Qualitative comparison of silhouettes generated from low variance manifold areas by the different GP models (images manually ordered by visual similarity).

Interactive GPDBN and GPLVM Manifold Demonstration

Please drag the red and white circle around the heat map to explore the manifold!

Select Model:    

Select Zoom:    


Unlikely Probability Likely

Different datasets and models may be selected from the drop-down menu. There are two models:

The manifold is a probabilistic embedding and the heat-map provides an indication of the probability of finding a reasonable shape (measures the predictive variance). Regions in white have a low predictive variance (high probability) while regions in black have a high variance (low probability) and are less likely to produce representative shapes. The darkest black regions (away from the manifold) will return the an average shape. The filckering in the results is due to the model displaying an average obtained over a number of random samples propagated through the model.

Please note that the manifolds have been restricted to two dimensions for visualisation. The natural dimension of some of the manifolds may be higher and this can lead to separated islands. When performing other tasks with the manifold the natural dimensionality would be used. The blue dots indicate the embedded locations of the original training data.

Browser Compatibility: We have tested with the latest versions of Safari (7.0.1) and Chrome (32.0.1700.77).

Publication

accv18_gpdbn
Gaussian Process Deep Belief Networks: A Smooth Generative Model of Shape with Uncertainty Propagation,
Alessandro Di Martino, Erik Bodin, Carl Henrik Ek and Neill D. F. Campbell,
Asian Conf. on Computer Vision (ACCV), 2018
[pdf] [code]

Acknowledgements

This work was supported by the EPSRC CAMERA (EP/M023281/1) grant and the Royal Society.