Calibration-cum-resolution

Calibration-cum-resolution is the property of forecasts that unites the calibration and resolution properties. Let the sequence of outcomes be {⚠ $y_n$} (assumed binary), the sequence of forecasts be {⚠ $\gamma_n$}, and let {⚠ $x_n$} be the signal used in forecasting {⚠ $y_n$}. The forecasts have this property if

{⚠ $\frac{\sum_{n=1,\dots,N: (\gamma_n,x_n) \approx (\gamma^*,x^*)} (y_n - \gamma_n) } { \sum_{n=1,\dots,N: (\gamma_n,x_n) \approx (\gamma^*,x^*)} 1 }\approx 0,$}

for all forecasts {⚠ $\gamma^*$} and all signals {⚠ $x^*$}. A convenient (and easier to formalize) restatement of this property is: a prediction algorithm achieves asymptotic calibration-cum-resolution if

{⚠ $\lim\limits_{N \to \infty} \frac{1}{N} \sum_{n=1}^N (y_n - \gamma_n) f(\gamma_n, x_n) = 0$}

for all continuous functions {⚠ $f: [0,1] \times X \to \mathbb{R}$} from some class. Calibration corresponds to the case where {⚠ $f=f(\gamma,x)$} does not depend on {⚠ $x$}, and resolution to the case where {⚠ $f$} does not depend on {⚠ $\gamma$}. In case of weather forecasts, calibration-cum-resolution means that forecaster is good in predicting of the probability of rain (it was raining in 70% of the days, when the forecaster predicted 70% probability of rain), and he is also good in predicting the weather "for Thursdays" (or for any other days, if we assess his forecasts only for these days).

Bibliography

• Vladimir Vovk, Non-asymptotic calibration and resolution. Theoretical Computer Science (Special Issue devoted to ALT 2005) 387, 77–89 (2007).