Gauss Linear Model

The Gauss statistical model says that the ⚠ $(x_n,y_n)\in\mathbb{R}^p\times\mathbb{R}$ are generated as follows:

• there are no restrictions on the way ⚠ $x_n$ are generated;
• given ⚠ $x_n$, the ⚠ $y_n$ are generated from ⚠ $y_n = w\cdot x_n + \xi_n$, where ⚠ $w\in\mathbb{R}^p$ is a vector of parameters, ⚠ $\xi_n$ is distributed as ⚠ $N(0,\sigma^2)$, and ⚠ $\sigma>0$ is another parameter.

See Section 8.5 of Vovk et al. (2005) and Vovk et al. (2009) for the formulation of this model as an on-line compression model.

The most basic version of this model is where there are no ⚠ $x$s, and the model is ⚠ $y_n\sim N(0,\sigma^2)$. The summary of ⚠ $y_1,\ldots,y_n$ is ⚠ $t_n:=y_1^2+\cdots+y_n^2$ and the Gauss repetitive structure postulates that the distribution of ⚠ $y_1,\ldots,y_n$ is uniform on the sphere of radius ⚠ $t_n^{1/2}$. Borel (1914) noticed that the Gauss statistical model (used by Maxwell as a model in statistical physics) is equivalent to the Gauss repetitive structure (used for a similar purpose by Gibbs). For further historical comments, see Vovk et al. (2005), Section 8.8, and Diaconis and Freedman (1987), Section 6.

Bibliography

• Persi Diaconis and David Freedman (1987). A dozen de Finetti-style results in search of a theory. Annales de l'Institut Henri Poincare B 23:397-423.
• Vladimir Vovk, Alexander Gammerman and Glenn Shafer (2005). Algorithmic learning in a random world. Springer, New York.
• Vladimir Vovk, Ilia Nouretdinov, and Alexander Gammerman (2009). On-line predictive linear regression. Annals of Statistics 37:1566-1590.