Spherical Cauchy VAEs Offer New Way to Model Heavy-Tailed Posteriors on Spheres

Variational autoencoders (VAEs) on spherical data have lacked a satisfying solution for modeling heavy-tailed posteriors. Existing approaches force researchers to choose between computational complexity and mathematical convenience, but a new framework based on the spherical Cauchy distribution sidesteps both limitations.

The Problem with Current Methods#

Standard approaches to spherical VAEs each carry drawbacks. The Von Mises-Fisher distribution requires modified Bessel functions and rejection sampling—adding computational overhead. The Power Spherical distribution achieves closed-form solutions only by constraining its density to zero at the antipode, an artificial restriction. Neither option provides the flexibility that Student-family distributions offer for standard Euclidean VAEs.

How Spherical Cauchy Works#

The spherical Cauchy distribution transforms into a multivariate Student distribution via stereographic projection, a classical geometric mapping. A Möbius transformation—another geometric operation—converts random draws from a uniform sphere into exact posterior samples using only inner products, norms, and basic arithmetic. This elegant connection means the Kullback-Leibler (KL) divergence to a uniform prior reduces to a scalar expectation that terminates exactly in even dimensions, requiring just one logarithm and a polynomial with finitely many correction terms. Odd dimensions admit certified truncation strategies.

The resulting KL divergence exhibits desirable mathematical properties: it increases monotonically and remains convex in the concentration parameter, and the same function computes pairwise KL between two such posteriors.

Performance Gains#

When tuned for equal local precision, the spherical Cauchy distribution combines broader angular tails with a smaller KL penalty than both Von Mises-Fisher and Power Spherical alternatives. Speed benchmarks show substantial improvements: on CPU, the method runs 1.5× faster than Power Spherical and 4.2× faster than robust Von Mises-Fisher implementations per latent-layer step. GPU acceleration widens the gap to 1.6× and 5.4× respectively.

Empirically, across multiple seeds the spherical Cauchy variant achieves the lowest reconstruction loss on MNIST at every tested dimensionality, and improves held-out viewpoint-gap negative log-likelihood on smallNORB by 3.6 percent.