Statistical And On-line Compression Modelling

In standard statistical modelling Reality is modelled as a family probability measures $\{P_{\theta} \mid \theta\in\Theta\}$. In on-line compression modelling Reality is modelled as a 5-tuple whose key elements are the forward functions and backward kernels. With each on-line compression model $M$ we can associate the statistical model $\phi(M)$ defined as the extreme points of the probability measures on $\mathbf{Z}^{\infty}$ (where $\mathbf{Z}$ is the example space) that agree with the on-line compression model. Natural questions (some rather vague) are:

  • What are the statistical models that can be obtained in this way (are of the form $\phi(M)$ for some $M$)?
  • Characterize the on-line compression models $M$ for which there is no "loss of information" in replacing $M$ by $\phi(M)$.
  • Does $\phi$ establish a bijection between some wide and natural classes of on-line compression and statistical models?

Some work in this direction has been done by Martin-Lof, Lauritzen, and other authors for repetitive structures (models very closely related to on-line compression models). See Vovk et al. (2005), Section 8.8.