Randomness Assumption (IID assumption)

The randomness assumption (also known as the IID assumption) is that the observations in a sequence are generated independently from the same probability distribution $Q$ on the space of possible observations $\mathbf{Z}$ (often $\mathbf{Z}=\mathbf{X}\times\mathbf{Y}$). A weaker (for a wide class of $\mathbf{Z}$, according to de Finetti's theorem) assumption is that of exchangeability.

The randomness assumption is used in stochastic prediction and conformal prediction. It is a standard assumption in machine learning. In applications, algorithms developed under this assumption (such as SVM) are often applied when the assumption is violated. However, if the observations $z_1,z_2,\ldots$, $z_i=(x_i,y_i)$, are coming from a stationary measure on $\mathbf{Z}^{\infty}$, the IID assumption can be often made "almost satisfied" by extending the objects $x_i$. For example, in the case of time series we may add the pre-history of $y_i$ to $x_i$ (and this will work very well if the time series is Markov).