Generalized Linear Models

Let the outcome belong to $ [Y_1,Y_2] $. We say that Expert $\theta$'s prediction at step $t$ is denoted $\xi^\theta_t$ and is equal to

$$\xi_t^\theta = Y_1+(Y_2-Y_1)\sigma(\theta'x_t).$$

Here $\sigma: \mathbb{R}\to\mathbb{R}$ is a fixed \emph{activation function}. We have $\sigma:\mathbb{R}\to [0,1]$ in all the cases except linear regression (see below). If the range of the function $\sigma$ is $ [0,1] $, the experts necessarily give predictions from $ [Y_1,Y_2] $.

  • Logistic activation function is $\sigma(z) = \frac{1}{1+e^{-z}}$.
  • Probit activation function is
    $$\sigma(z) = \Phi(z) = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^z e^{-v^2\slash{}2} dv,$$

where $\Phi$ is the cumulative distribution function of the normal distribution with zero mean and unit variance.

  • Complementary log-log (comlog) activation function $\sigma(z) = 1-\exp (-\exp(z))$.

Generalized Linear Models are often used for classification purposes.

Bibliography

  • McCullagh, P., Nelder, J. Generalized Linear Models. Chapman & Hall/CRC, second edn. 1989.