Weak teachers represent a class of prediction algorithms for the case of the relaxation of the on-line protocol - the case when Reality provides true labels of examples with a delay or only occasionally, for a subset of trials (or both).
Teaching schedule - a function defined on an infinite set , and satisfying
for all and .
The teaching schedule describes the way the data is disclosed: after the trial , Reality provides the lable for the object .
Weak teacher or -taught version of a confidence predictor is
Ideal teacher (TCM). If and for each , then .
Slow teacher. If lag: is an increasing function, , and then is a predictor that learns the true label for each object but with a delay equal to .
Lazy teacher. If and for each , then is given the true lables immediately but not for every object.
In case of weak teachers there is no validity in the strongest possible way (conservative validity). However, the following weaker types of validity can be defined:
- weak validity;
- strong validity;
- validity in the sense of the law of the iterated algorithm.
All the statements in the section are given under the randomness assumption.
A randomized confidence predictor is asymptotically exact in probability if, for all significance levels and all probability distributions on ,
where is the random variable defined as follows:
if ; otherwise.
Similarly, a confidence predictor is asymptotically conservative in probability if, for all significance levels and all probability distributions on ,
Theorem Let be a teaching schedule with domain , .
- If , any -taught smoothed conformal predictor is asymptotically exact in probability.
- Otherwise, there exists an -taught smoothed conformal predictor which is not asymptotically exact in probability.
Corollary If , any -taught conformal predictor is asymptotically conservative in probability.