Generalized Linear Models

Let the outcome belong to ⚠ $ [Y_1,Y_2] $. We say that Expert ⚠ $\theta$'s prediction at step ⚠ $t$ is denoted ⚠ $\xi^\theta_t$ and is equal to

⚠ $\displaystyle{\xi_t^\theta = Y_1+(Y_2-Y_1)\sigma(\theta'x_t).}$

Here ⚠ $\sigma: \mathbb{R}\to\mathbb{R}$ is a fixed \emph{activation function}. We have ⚠ $\sigma:\mathbb{R}\to [0,1]$ in all the cases except linear regression (see below). If the range of the function ⚠ $\sigma$ is ⚠ $ [0,1] $, the experts necessarily give predictions from ⚠ $ [Y_1,Y_2] $.

  • Logistic activation function is ⚠ $\sigma(z) = \frac{1}{1+e^{-z}}$.
  • Probit activation function is
    ⚠ $\displaystyle{\sigma(z) = \Phi(z) = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^z e^{-v^2/2} dv,}$

where ⚠ $\Phi$ is the cumulative distribution function of the normal distribution with zero mean and unit variance.

  • Complementary log-log (comlog) activation function ⚠ $\sigma(z) = 1-\exp (-\exp(z))$.

Generalized Linear Models are often used for classification purposes.

Bibliography

  • Mc Cullagh, P., Nelder, J. Generalized Linear Models. Chapman & Hall/CRC, second edn. 1989.