Gaussian process $f$ — random function with jointly Gaussian marginals.

Characterized by

• a mean function $m(x) = \E(f(x))$,
• a kernel (covariance) function $k(x, x') = \Cov(f(x), f(x'))$.

Notation: $f \~ \f{GP}(m, k)$.

The kernel $k$ must be positive (semi-)definite (non-trivial requirement).

Takes

• data $(x_1, y_1), .., (x_n, y_n) \in X \x \R$,
• and a prior Gaussian Process $\f{GP}(m, k)$

giving the posterior (conditional) Gaussian process $\f{GP}(\hat{m}, \hat{k})$.

The functions $\hat{m}$ and $\hat{k}$ may be explicitly expressed in terms of $m$ and $k$.

$$ \htmlData{class=fragment fade-out,fragment-index=6}{ \footnotesize \mathclap{ k_{\nu, \kappa, \sigma^2}(x,x') = \sigma^2 \frac{2^{1-\nu}}{\Gamma(\nu)} \del{\sqrt{2\nu} \frac{\abs{x-x'}}{\kappa}}^\nu K_\nu \del{\sqrt{2\nu} \frac{\abs{x-x'}}{\kappa}} } } \htmlData{class=fragment d-print-none,fragment-index=6}{ \footnotesize \mathclap{ k_{\infty, \kappa, \sigma^2}(x,x') = \sigma^2 \exp\del{-\frac{\abs{x-x'}^2}{2\kappa^2}} } } $$ $\sigma^2$: variance $\kappa$: length scale $\nu$: smoothness
$\nu\to\infty$: RBF kernel (Gaussian, Heat, Diffusion)

$\nu = 1/2$

$\nu = 3/2$

$\nu = 5/2$

$\nu = \infty$

Manifolds: not PSD for some $\kappa$ (in some cases—for all) unless the manifold is isometric to $\mathbb{R}^d$.

Feragen et al. (CVPR 2015) and Da Costa et al. (SIMODS 2025).

Graph nodes: not PSD for some $\kappa$ unless can be isometrically embedded into a Hilbert space.

Schoenberg (Trans. Am. Math. Soc. 1938).

Not PSD on the square graph.

Not PSD on the circle manifold.

$$ \htmlData{class=fragment,fragment-index=0}{ \underset{\t{Matérn}}{\undergroup{\del{\frac{2\nu}{\kappa^2} - \Delta}^{\frac{\nu}{2}+\frac{d}{4}} f = \c{W}}} } $$ $\Delta$: Laplacian $\c{W}$: Gaussian white noise $d$: dimension

The solution is a Gaussian process with kernel $$ \htmlData{fragment-index=2,class=fragment}{ k_{\nu, \kappa, \sigma^2}(x,x') = \frac{\sigma^2}{C_{\nu, \kappa}} \sum_{n=0}^\infty \Phi_{\nu, \kappa}(\lambda_n) f_n(x) f_n(x') } $$

$\lambda_n, f_n$: eigenvalues and eigenfunctions of the Laplace–Beltrami operator

Sphere $\mathbb{S}_2$

Torus $\mathbb{T}^2 = \bb{S}_1 \x \bb{S}_1$

Projective plane $\mathrm{RP}^2$

Driven by generalized random phase Fourier features (GRPFF).

Let $\mathcal{P}$ denote the heat kernel, the fundamental solution of $ \frac{\partial u}{\partial t} = \Delta_x u . $

Then, on $\mathbb{R}^d$: $$ \begin{aligned} \htmlData{fragment-index=3,class=fragment}{ k_{\infty, \kappa, \sigma^2}(x, x') } & \htmlData{fragment-index=3,class=fragment}{ \propto \mathcal{P}(\kappa^2/2, x, x'), } \\ \htmlData{fragment-index=4,class=fragment}{ k_{\nu, \kappa, \sigma^2}(x, x') } & \htmlData{fragment-index=4,class=fragment}{ \propto \sigma^2 \int_{0}^{\infty} t^{\nu-1+d/2}e^{-\frac{2 \nu}{\kappa^2} t} \mathcal{P}(t, x, x') \mathrm{d} t } \end{aligned} $$

Alternative generalization: use the above as the definition.

☺ Always positive semi-definite + Interpretable inductive bias.

☺ Usually equivalent to SPDE-based with no SPDE machinery.

☹ Implicit, requires solving the equation.

For symmetric spaces like $\bb{H}_d$,  $\mathrm{SPD}(d)$ harmonic analysis generalizes well.

This allows to generalize random Fourier features to this setting.

— a few technical challenges arise though.

— see «Stationary Kernels and Gaussian Processes on Lie Groups and their Homogeneous Spaces II», (JMLR 2024).

Geometry-aware vs Euclidean

SPDE turns into a stochastic linear system. The solution has kernel $$ \htmlData{fragment-index=2,class=fragment}{ k_{\nu, \kappa, \sigma^2}(i, j) = \frac{\sigma^2}{C_{\nu, \kappa}} \sum_{n=0}^{\abs{V}-1} \Phi_{\nu, \kappa}(\lambda_n) \mathbf{f_n}(i)\mathbf{f_n}(j) } $$

$\lambda_n, \mathbf{f_n}$: eigenvalues and eigenvectors of the Laplacian matrix

https://geometric-kernels.github.io/