One of the most influential websites for me was John Baez’s This Week’s Finds. I learned many things from these blog posts and it helped me realize which scientific or mathematical topics interested me most. Lately, I have been spending quite some time studying several topics, usually a few weeks each, and it occurred to me that it might be fun and interesting to write about them (even though everything hasn’t been worked out yet).

In a future post, I am going to talk a bit more about the mathematics of financial markets (I am working in finance after all). One of the important theorems there is that of Girsanov’s theorem. During the study of this topic, however, I stumbled upon a related result (a predecessor to be exact), the Cameron–Martin formula. This post will introduce some notions and generalizations that occur when studying translations of Gaussian measures.

Overview

This post is structured as follows:

  • Short introduction to Gaussian measures.
  • Some necessary ideas from linear algebra and topology.
  • Doing probability theory on abstract vector spaces.
  • Some extras and applications.
Gaussian Measures

One of the most archetypical probability distributions1 is the (standard) normal distribution (also called Gaussian distribution). In 1D, it is given by the following probability density function:

\[\mathcal{N}(x) = \frac{1}{\sqrt{2\pi}}\exp\left(-\frac{x^2}{2}\right)\,.\]

In higher dimensions $(n\in\mathbb{N})$, the generalization is given by:

\begin{gather} \label{normal} \mathcal{N}(\vec{x}) = \frac{1}{(2\pi)^{n/2}}\exp\left(-\frac{|\vec{x}|^2}{2}\right)\,. \end{gather}

A common question in physics and mathematics is what the transformations are that leave a certain structure invariant (cf. symmetry transformations as in this post). Since the (centered2) normal distribution only depends on the norm of the point $\vec{x}$, it is clearly invariant under orthogonal transformations (i.e. rotations). However, next to rotations, also translations

\[\vec{x}\mapsto\vec{x}+\vec{a}\]

are of major importance in the study of vector spaces. However, a famous result by Haar implies that, up to a constant factor, the standard volume measure $\lambda$ (i.e. Lebesgue measure) is the only measure on $\mathbb{R}^n$ that is invariant under translations. Nonetheless, the measures $\lambda$ and $a_*\lambda$, where the latter is the pushforward of $\lambda$ under the translation defined above, are equivalent in that they are absolutely continuous with respect to each other. The (classical) Cameron–Martin theorem states that the Radon–Nikodym derivative in this case is given by the following formula:

\begin{gather} \label{cameron_martin} \frac{\mathrm{d}a_*\lambda}{\mathrm{d}\lambda}(\vec{x}) = \exp\left(\vec{x}\cdot\vec{a}-\frac{|a|^2}{2}\right)\,. \end{gather}

The essence of this month’s blog post will be to work through the extension of this theorem to Gaussian measures on so-called Banach and Hilbert spaces.

Some Linear Algebra (and Topology)

We first take some steps back and look at the algebraic structure of $\mathbb{R}^n$ (or any other $n$-dimensional vector space since these are linearly isomorphic). Euclidean space comes equipped with the structure of an inner product space and, consequently, that of a normed space. These are formally defined as follows.

A vector space $V$ equipped with a bilinear operation $\langle\cdot\mid\cdot\rangle:V\times V\rightarrow\mathbb{R}$ such that the following conditions are satisfied:

  1. Nondegeneracy: $\forall v\in V:\langle v\mid w\rangle=0\implies w=0$.
  2. Positive-definiteness: $\forall v\in V:\langle v\mid v\rangle\geq 0$.
  3. Symmetry: $\forall v,w\in V:\langle v\mid w\rangle = \langle w\mid v\rangle$.

A vector space $V$ equipped with a nonnegative function $\lVert\cdot\rVert:V\rightarrow\mathbb{R}^+$ such that the following conditions are satisfied:

  1. Nondegeneracy: $\lVert v\rVert=0\implies v=0$.
  2. Positive-homogeneity: $\forall v\in V,\lambda\in\mathbb{R}:\lVert\lambda v\rVert=\lvert\lambda\rvert\lVert v\rVert$.
  3. Triangle inequality: $\forall v,w\in V:\lVert v+w\rVert\leq\lVert v\rVert+\lVert w\rVert$.

Every inner product induces an associated norm by the following formula:

\[\lVert v\rVert := \sqrt{\langle v\mid v\rangle}\,.\]

Using this convention, any finite-dimensional inner-product space gives rise to a canonical Gaussian measure as in Eq. \eqref{normal}.

In the realm of infinite-dimensional vector spaces, as usually treated in functional analysis, everything gets slightly more tricky due to the presence of topological subtleties. For the purpose of this post, three notions will be of interest: Separability, Banach spaces and Hilbert spaces. (The first one is purely topological, the latter two are also algebraic.)

A topological space is said to be separable if it admits a countable dense subset.

A complete normed space $V$, i.e. a normed space $(V,\lVert\cdot\rVert)$ such that $V$ with the induced metric

\[d(v,w) := \lVert v-w\rVert\]

is a complete metric space.

An inner product space that is also a Banach space.

A very important result in functional analysis states that every separable Hilbert space is either

  1. finite dimensional, or
  2. isomorphic to $\ell^2$, the space of square-summable sequences.
Gaussian Measures on Banach Spaces

Assume that Eq. \eqref{normal} would make sense on a separable infinite-dimensional Hilbert space $\mathcal{H}$ and choose an orthonormal basis $(e_{n})_{n\in\mathbb{N}}$. Under the would-be Gaussian measure $\gamma_{\mathcal{H}}$, the random variables

\[X_n:\mathcal{H}\rightarrow\mathbb{R}:x\mapsto\langle x\mid e_n\rangle\]

would be independent standard normal random variables. Now, by the strong law of large numbers, the norm of the random vector $x=\sum_{n\in\mathbb{N}}X_n(x)e_n$ would diverge a.e.:

\[\lVert x\rVert^2 = \sum_{n\in\mathbb{N}}X_n(x)^2\longrightarrow +\infty\,.\]

Of course, this cannot be true. The idea, introduced by Gross, is that $\mathcal{H}$ is actually too small to support a Gaussian measure and one should endow $\mathcal{H}$ with a less conservative norm and consider its associated completion.

To this end, one needs to consider a suitable $\sigma$-algebra on a (separable) vector space. Consider a vector space $V$ and its continuous dual3 $V$’. (In fact, any dual pairing $(X,Y)$ suffices.) For every finite collection ${\phi_1,\ldots,\phi_n}\subset V’$, consider the sets

\begin{gather} \label{cylinder} C_{\phi_1,\ldots,\phi_n}(B) := \{v\in V\mid\bigl(\phi_1(v),\ldots,\phi_n(v)\bigr)\in B\}\,, \end{gather}

where $B$ ranges over the Borel sets in $\mathbb{R}^n$. Sets of this form are called cylinder sets. Note that the collection of all cylinder sets, ranging over both indexing functionals and Borel sets, does not form a $\sigma$-algebra. Only when keeping the indexing functionals fixed, a $\sigma$-algebra $\mathcal{C}_{\phi_1,\ldots,\phi_n}(V)$ is obtained. However, the $\sigma$-algebra generated by all cylinder sets is called the cylindrical $\sigma$-algebra $\mathcal{C}(V)$. For separable Banach spaces, it was shown by Mourier that the cylindrical algebra coincides with the natural Borel algebra $\mathcal{B}(V)$.

Now, a large part of my time invested in understanding this topic was spent on the following property. In the seminal paper by Gross, and some subsequent work such as by Dudley, a different convention for the cylinder sets was used:

\begin{gather} \label{dual_cylinder} C_F(\tilde{B}) := \{v\in V\mid\pi_F(v)\in\tilde{B}\} = \tilde{B} + F^\perp\,, \end{gather}

where $F$ is a finite-dimensional subspace of $V’$, $\tilde{B}$ is a Borel set in $F^*\cong V/F^\perp\cong F$ and $\pi_F:V\rightarrow V/F^\perp$ is the canonical projection with the complement $F^\perp$ defined as follows:

\[F^\perp := \{v\in V\mid\forall\phi\in F:\phi(v)=0\}\,.\]

To actually see the equivalence of these two constructions, I needed quite some pen-and-paper work, so let us go through it together.

Proof
For the cylinder set $C_{\phi_1,\ldots,\phi_n}(B)$ in Eq. \eqref{cylinder}, it is clear that adding any element of $\langle\phi_1,\ldots,\phi_n\rangle^\perp$, where $\langle\cdots\rangle$ indicates the linear span of a set of vectors, would leave the set invariant. As such, one obtains \begin{align*} C_{\phi_1,\ldots,\phi_n}(B) &= \{v\in V/\langle\phi_1,\ldots,\phi_n\rangle^\perp\mid\bigl(\phi_1(v),\ldots,\phi_n(v)\bigr)\in B\} + \langle\phi_1,\ldots,\phi_n\rangle^\perp\\ &= \{v\in \langle\phi_1,\ldots,\phi_n\rangle^\ast\mid\bigl(\phi_1(v),\ldots,\phi_n(v)\bigr)\in B\} + \langle\phi_1,\ldots,\phi_n\rangle^\perp\,. \end{align*} Since the map $(\phi_1,\ldots,\phi_n)$ is linear (and acts between finite-dimensional spaces), it is also continuous and, hence, the set in the first term is equal to some Borel set $\tilde{B}\in\mathcal{B}(\langle\phi_1,\ldots,\phi_n\rangle^*)$. Accordingly, it is of the form of Eq. \eqref{dual_cylinder}.

Conversely, consider any basis $\{\phi_1,\ldots,\phi_n\}$ of the finite-dimensional subspace $F\subseteq V'$. By reversing the steps above, it is not hard to see that $C_F$ is equivalent to $C_{\phi_1,\ldots,\phi_n}$.
$\Box$

Given a cylindrical algebra $\mathcal{C}(V)$, a cylindrical measure (or cylinder set measure) is a nonnegative, finitely additive set function $\mu:\mathcal{C}(V)\rightarrow\mathbb{R}^+$ that restricts to a proper measure on every sub-$\sigma$-algebra $\mathcal{C}_{\phi_1,\ldots,\phi_n}(V)$.

Although the Gaussian measure in Eq. \eqref{normal} does not make sense as a measure on $\mathcal{C}(\mathcal{H})$, with $\mathcal{H}$ again a separable, infinite-dimensional Hilbert space, it does give rise to a cylindrical measure $\mu$. To obtain a proper measure, the Hilbert norm $\lVert\cdot\rVert_{\mathcal{H}}$ is now replaced by a norm that interacts nicely with respect to $\mu$. A (continuous) norm $\lVert\cdot\rVert$ on $\mathcal{H}$ is said to be measurable (with respect to $\mu$) if, for every $\varepsilon>0$, there exists a finite-dimensional subspace $V_\varepsilon\subset\mathcal{H}$ such that the following relation holds for all finite-dimensional subspaces $V\subset\mathcal{H}$:

\[V\perp V_\varepsilon\implies\mu\bigl(\{v\in V\mid\lVert v\rVert>\varepsilon\}\bigr)<\varepsilon,.\]

Now, choose such a measurable norm (e.g. any norm induced by an injective Hilbert–Schmidt operator4) and consider the (Banach) completion $\mathbb{B}$ of $\mathcal{H}$ with respect to this norm. The triple $(\mathcal{H},\mathbb{B},\mu)$ is called an abstract Wiener space. The cylindrical measure $\mu$ can be simply extended to $\mathbb{B}$ as follows:

\[\mu(B) := \mu(B\cap\mathcal{H})\]

for all $B\in\mathcal{B}(\mathbb{B})$. It can now be shown that $\mu$ can actually be enhanced to a proper probability measure $\gamma$ on $\mathbb{B}$ (as in the Kolmogorov–Daniell extension theorem).5 Given this measure $\gamma$, one can define the moments of functionals:

\begin{align*} \mathrm{E}[\phi] &:= \int_{\mathbb{B}}\phi(x)\,d\gamma(x)\,,\\ \mathrm{Cov}[\phi,\psi] &:= \int_{\mathbb{B}}\bigl(\phi(x)-\mathrm{E}[\phi]\bigr)\bigl(\psi(x)-\mathrm{E}[\psi]\bigr)\,d\gamma(x)\,. \end{align*}

Using these operators, one can express the characteristic function of $\gamma$ as follows:

\[\tilde{\gamma}(\phi) = \exp\left(i\mathrm{E}[\phi] - \frac{\mathrm{Cov}[\phi,\phi]}{2}\right)\,.\]

On $\mathbb{B}$, one can now consider the following norm:

\[\lVert x\rVert_\gamma := \sup\{\phi(x)\mid\phi\in\mathbb{B}'\land\mathrm{Cov}[\phi,\phi]\leq 1\}\,.\]

Note that when the Gaussian measure was extended through the choice of a measurable norm $\lVert\cdot\rVert$, then

\[\lVert x\rVert = \lVert x\rVert_\gamma\]

for all $x\in\mathbb{B}$. Moreover, in this case, it can be shown that the subspace

\[\mathcal{H}_\gamma := \{x\in\mathbb{B}\mid\lVert x\rVert_\gamma<+\infty\}\]

is isomorphic to $\mathcal{H}$. $\mathcal{H}$ is also called the Cameron–Martin space of the abstract Wiener space $(\mathbb{B},\mathcal{H},\gamma)$ for the following reason. The translated measure $h_\ast\gamma$, where $h\in\mathcal{H}$, is absolutely continuous with respect to $\gamma$ with Radon–Nikodym derivative

\[\frac{\mathrm{d}h_*\lambda}{\mathrm{d}\lambda}(x) = \exp\left(\langle x\mid h\rangle-\frac{\|h\|_\gamma^2}{2}\right)\,.\]

Comparing this expression to Eq. \eqref{cameron_martin} should explain the terminology.

As a last note, it should be said that although it is the Cameron–Martin space $\mathcal{H}$ that essentially gives all the structure to the abstract Wiener space $(\mathcal{H},\mathbb{B},\gamma)$, it is ‘very small’ in that

\[\gamma(\mathcal{H})=0\]

whenever $\dim(\mathcal{H})=+\infty$.

References
Footnotes

  1. For a refresher on measure and probability theory, see this appendix. ↩

  2. Centered distributions are those with mean 0. ↩

  3. For finite-dimensional spaces, this is simply the linear dual $V^\ast$. However, for general topological vector spaces, the continuous dual $V’$ is the subset of the linear dual $V^\ast$ consisting of continuous functionals. ↩

  4. See e.g. the paper by Sato: “Gaussian measure on a Banach space and abstract Winer measure”. ↩

  5. It was shown by Sato that the new norm $\lVert\cdot\rVert$ actually does not have to be measurable. This is only a sufficient condition. Only for Hilbert norms, i.e. those induced by an inner product, do ‘admissibility’ and ‘measurability’ coincide. ↩