Notation#
This page summarizes the notations used throughout the theory pages of this chapter; the same notation is used in the accompanying research report. Symbols not listed here are explained where they first appear.
Notation |
Meaning |
|---|---|
\(\N\) |
the set of natural numbers (including \(0\)) |
\(\R\) |
the set of real numbers |
\(\R^d\) |
the \(d\)-dimensional real vector space (of column vectors) |
\(\# S\) |
the cardinality of a finite set \(S\) |
\(\Pi_{\Omega}\) |
the projection map onto a convex set \(\Omega \subseteq \R^d\) |
\([K]\) |
the set \(\{1, 2, \ldots, K\}\), where \(K \in \N_{> 0}\) |
\(\theta\) |
the model parameters, \(\theta \in \R^d\) |
\(\col(\theta_1, \ldots, \theta_K)\) |
the column vector stacking \(\theta_1, \ldots, \theta_K\) vertically in order |
\(\operatorname{dom}(f)\) |
the domain of the function \(f\), i.e. \(\{\theta \in \R^d ~|~ f(\theta) < +\infty\}\) |
\(\nabla f(\cdot)\) |
the gradient of \(f\) |
\(\partial f(\cdot)\) |
the subdifferential (the set of subgradients) of \(f\) |
\(\mathcal{D}(x, y)\) |
the (joint) distribution of the data, where \(x\) is the feature and \(y\) is the label |
\(I_n\) or \(I\) |
the identity matrix (of order \(n\)) |
\(O_n\) or \(O\) |
the zero matrix (of order \(n\)) |
\(A^{\mathrm{T}}\) |
the transpose of a matrix \(A\) |
\(A^{-1}\) |
the inverse of a matrix \(A\) |
\(\lVert \cdot \rVert_p\) |
the \(p\)-norm of a vector or a matrix (\(0 \leqslant p \leqslant +\infty\)); when the subscript \(p\) is omitted, the \(2\)-norm is meant |
\(\lVert \cdot \rVert_F\) |
the Frobenius norm of a matrix |
\(\mathbb{E}[\cdot]\) |
the mathematical expectation |
\(\mathcal{N}(\mu, \sigma^2)\) |
the univariate normal distribution with mean \(\mu \in \R\) and variance \(\sigma^2 \in \R\) |
\(\mathcal{N}(v, \Sigma)\) |
the \(d\)-dimensional normal distribution with mean \(v \in \R^d\) and covariance matrix \(\Sigma \in \R^{d \times d}\) |