In the GLM framework, the link function plays a deceptively simple but profoundly important role: it transforms the expected value of the response into a quantity that can be modeled as a linear combination of predictors.

Consider the challenge: when modeling probabilities, the mean must lie in (0,1); when modeling counts, the mean must be non-negative; when modeling durations, the mean must be strictly positive. Yet our linear predictor $\eta = \mathbf{x}^\top \boldsymbol{\beta}$ can take any real value. How do we reconcile these incompatible domains?

The link function $g(\cdot)$ is the mathematical bridge. It maps the constrained mean space to the entire real line, allowing us to say $g(\mu) = \eta$ for any $\eta \in \mathbb{R}$. The properties of this bridge—its shape, its derivatives, its interpretability—profoundly affect the behavior and meaning of our model.

Not every function can serve as a link. For a function $g: \mathcal{M} \to \mathbb{R}$ to be a valid link function, it must satisfy several mathematical requirements.

Requirement 1: Defined on the Mean Space

The link function must be defined on the set $\mathcal{M}$ of possible mean values for the chosen distribution:

For example, the log link $g(\mu) = \log(\mu)$ is appropriate when $\mathcal{M} = (0, \infty)$ but not when negative means are possible.

Requirement 2: Monotonicity

The link function must be strictly monotonic (either strictly increasing or strictly decreasing throughout $\mathcal{M}$). This ensures a one-to-one correspondence between $\mu$ and $\eta$.

Monotonicity guarantees:

• For each linear predictor value $\eta$, there exists exactly one mean $\mu = g^{-1}(\eta)$
• The model is identifiable—different parameter values produce different predictions
• The score equations have unique solutions (under regularity conditions)

Requirement 3: Differentiability

The link function should be twice differentiable on the interior of $\mathcal{M}$. This is needed for:

• Deriving the score function in maximum likelihood estimation
• Computing the Fisher information matrix
• Applying Newton-Raphson or Fisher scoring algorithms

The first derivative $g'(\mu)$ appears in the weight matrix of iteratively reweighted least squares: $$w_i = \frac{1}{V(\mu_i) [g'(\mu_i)]^2}$$

Requirement 4: Range Coverage

The link function must map $\mathcal{M}$ onto the entire real line $\mathbb{R}$: $$g: \mathcal{M} \to \mathbb{R} \text{ is surjective}$$

This ensures that for any possible linear predictor value, there exists a valid mean. Without this, certain predictor combinations might produce undefined predictions.

The identity link is the simplest possible link function:

$$g(\mu) = \mu \qquad g^{-1}(\eta) = \eta$$

With the identity link, the mean equals the linear predictor directly: $$\mu_i = \mathbf{x}_i^\top \boldsymbol{\beta}$$

Properties

Derivative: $g'(\mu) = 1$ (constant)

Interpretation: Parameters have direct, additive effects on the mean: $$\frac{\partial \mu}{\partial X_j} = \beta_j$$

A one-unit increase in $X_j$ increases the expected response by exactly $\beta_j$ units, regardless of the current value of $X_j$ or other predictors.

When It's Appropriate

The identity link is the canonical link for the Normal distribution. It's also appropriate when:

• The mean can take any real value (no constraints)
• Predictor effects are genuinely additive on the response scale
• The relationship between predictors and response is approximately linear

The logit link (also called the log-odds link) is the canonical link for the Bernoulli and binomial distributions:

$$g(p) = \log\left(\frac{p}{1-p}\right) = \text{logit}(p) \qquad g^{-1}(\eta) = \frac{e^\eta}{1 + e^\eta} = \frac{1}{1 + e^{-\eta}}$$

The inverse function $g^{-1}$ is the logistic (or sigmoid) function, which maps the entire real line to (0,1).

Mathematical Properties

Domain: $p \in (0, 1)$ Range: $\eta \in (-\infty, +\infty)$

Derivative: $$g'(p) = \frac{1}{p(1-p)}$$

Note that $g'(p)$ is large near 0 and 1 (where small changes in probability correspond to large changes in log-odds) and smallest at $p = 0.5$.

Symmetry: $$\text{logit}(p) = -\text{logit}(1-p)$$

This symmetry means that the effect of predictors on the probability of success equals (in magnitude but opposite sign) the effect on the probability of failure.

Interpretation: The Odds Ratio

The logit link gives rise to one of the most important quantities in epidemiology and social science: the odds ratio.

Recall that for a binary outcome, the odds of success are: $$\text{odds} = \frac{p}{1-p}$$

With the logit link: $$\log\left(\frac{p}{1-p}\right) = \beta_0 + \beta_1 X_1 + \cdots + \beta_p X_p$$

Exponentiating both sides: $$\frac{p}{1-p} = e^{\beta_0} \cdot e^{\beta_1 X_1} \cdot \ldots \cdot e^{\beta_p X_p}$$

Now consider what happens when $X_j$ increases by 1 unit (holding others constant):

$$\frac{\text{odds after}}{\text{odds before}} = \frac{e^{\beta_0 + \beta_1 X_1 + \cdots + \beta_j(X_j+1) + \cdots}}{e^{\beta_0 + \beta_1 X_1 + \cdots + \beta_j X_j + \cdots}} = e^{\beta_j}$$

Thus $e^{\beta_j}$ is the multiplicative change in odds per unit increase in $X_j$—the odds ratio.

The probit link is an alternative to the logit for binary response data:

$$g(p) = \Phi^{-1}(p) \qquad g^{-1}(\eta) = \Phi(\eta)$$

where $\Phi$ is the cumulative distribution function (CDF) of the standard normal distribution:

$$\Phi(z) = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^{z} e^{-t^2/2} , dt$$

Motivation: The Latent Variable Interpretation

Probit regression has an elegant interpretation through latent variables. Suppose there exists an unobserved continuous variable $Y^*_i$ such that:

$$Y^*_i = \mathbf{x}_i^\top \boldsymbol{\beta} + \varepsilon_i, \qquad \varepsilon_i \sim \mathcal{N}(0, 1)$$

We observe $Y_i = 1$ if $Y^*_i > 0$ and $Y_i = 0$ otherwise. Then:

$$P(Y_i = 1) = P(Y^*_i > 0) = P(\mathbf{x}_i^\top \boldsymbol{\beta} + \varepsilon_i > 0) = P(\varepsilon_i > -\mathbf{x}_i^\top \boldsymbol{\beta})$$

Since $\varepsilon_i \sim \mathcal{N}(0,1)$ is symmetric: $$P(Y_i = 1) = \Phi(\mathbf{x}_i^\top \boldsymbol{\beta})$$

This latent variable interpretation is widely used in economics (discrete choice theory) and psychometrics.

Logit vs. Probit: A Comparison

In practice, logit and probit give nearly identical predictions for most datasets. The relationship between them is approximately:

$$\text{logit}(p) \approx \frac{\pi}{\sqrt{3}} \cdot \Phi^{-1}(p) \approx 1.81 \cdot \Phi^{-1}(p)$$

Thus, probit coefficients are roughly 1.8 times smaller than logit coefficients for the same data.

The main difference is in the tails:

• Logit has heavier tails (logistic distribution has kurtosis 6, vs. 3 for normal)
• Probit approaches 0 and 1 more quickly as $|\eta|$ increases
• For extreme predictions, logit is more conservative (higher uncertainty)

When to choose:

• Use logit when odds ratio interpretation is valuable, or for consistency with common practice
• Use probit when the latent variable interpretation is meaningful, or when convention in your field dictates

The log link is the canonical link for the Poisson distribution and commonly used for Gamma regression:

$$g(\mu) = \log(\mu) \qquad g^{-1}(\eta) = e^\eta$$

Properties

Domain: $\mu \in (0, \infty)$ Range: $\eta \in (-\infty, +\infty)$

Derivative: $g'(\mu) = 1/\mu$

Key Feature: The inverse link $\mu = e^\eta$ is always positive, regardless of $\eta$. This automatically satisfies the positivity constraint for counts and positive continuous responses.

Interpretation: Multiplicative Effects

With the log link: $$\log(\mu) = \beta_0 + \beta_1 X_1 + \cdots + \beta_p X_p$$

Exponentiating: $$\mu = e^{\beta_0} \cdot e^{\beta_1 X_1} \cdot \ldots \cdot e^{\beta_p X_p}$$

Predictor effects are multiplicative on the mean. When $X_j$ increases by 1:

$$\frac{\mu_{\text{after}}}{\mu_{\text{before}}} = e^{\beta_j}$$

The quantity $e^{\beta_j}$ is the rate ratio or incidence rate ratio (IRR) in epidemiology.

The Log Link for Multiplicative Phenomena

The log link is appropriate when effects combine multiplicatively rather than additively. Many phenomena exhibit this pattern:

• Population growth: Each factor multiplies the growth rate
• Compound interest: Returns combine multiplicatively
• Epidemiology: Risk factors often multiply baseline risk
• Economics: Percentage effects are multiplicative
• Biology: Many biological processes involve cascading multiplicative effects

In these contexts, the log link provides:

1. Natural interpretation: Effects as percentage changes
2. Positivity guarantee: Predictions can never be negative
3. Proportional effects: A 10% increase is 10% whether the baseline is 5 or 5,000

The complementary log-log (cloglog) link is an asymmetric alternative to logit and probit for binary data:

$$g(p) = \log(-\log(1-p)) \qquad g^{-1}(\eta) = 1 - \exp(-\exp(\eta))$$

Properties and Interpretation

Unlike the symmetric logit and probit, the cloglog link is asymmetric:

• It approaches 0 relatively slowly as $\eta \to -\infty$
• It approaches 1 rapidly as $\eta \to +\infty$

Connection to Extreme Value Theory

The cloglog link arises naturally in survival analysis and extreme value theory. If we have a binary outcome arising from whether an event occurs before a fixed time point, and the underlying hazard is constant (exponential survival), the appropriate link is cloglog.

Specifically, if $T \sim \text{Exponential}(\lambda)$ and we observe $Y = I(T \leq t_0)$:

$$P(Y = 1) = P(T \leq t_0) = 1 - e^{-\lambda t_0}$$

If $\log(\lambda t_0) = \eta$, then $P(Y=1) = 1 - e^{-e^\eta}$, which is the cloglog inverse link.

When to Use Cloglog

• Grouped survival data: When binary outcomes represent "event occurred within period"
• Proportion of infected/diseased: When a latent continuous process has an extreme-value distribution
• Asymmetric responses: When the probability curve should be asymmetric (steep on one side, gradual on the other)

The Inverse (Reciprocal) Link

The inverse link is the canonical link for the Gamma distribution:

$$g(\mu) = \frac{1}{\mu} \qquad g^{-1}(\eta) = \frac{1}{\eta}$$

Properties:

• Domain: $\mu > 0$
• Range: $\eta > 0$ (when used canonically) or all reals if we allow $\mu = 1/\eta$ with appropriate sign handling

Interpretation: Not intuitive. A unit increase in $X_j$ changes $1/\mu$ by $\beta_j$. This makes the inverse link unpopular despite being canonical—practitioners often prefer the log link for Gamma regression.

The Inverse Squared Link

The inverse squared link is canonical for the Inverse Gaussian distribution:

$$g(\mu) = \frac{1}{\mu^2} \qquad g^{-1}(\eta) = \frac{1}{\sqrt{\eta}}$$

The Square Root Link

The square root link is sometimes used for Poisson data:

$$g(\mu) = \sqrt{\mu} \qquad g^{-1}(\eta) = \eta^2$$

This link ensures $\mu > 0$ only for $\eta > 0$, and has variance-stabilizing properties for Poisson data (the square root transformation makes Poisson variance approximately constant).

Selecting an appropriate link function involves balancing several considerations:

Decision Framework

Step 1: Ensure Validity The link must map the mean space to ℝ. For binary data, you need a link that maps (0,1) to ℝ; for count data, one that maps (0,∞) to ℝ.

Step 2: Consider the Canonical Link The canonical link has nice mathematical properties (sufficient statistics, concavity). Start with the canonical unless you have reasons to deviate.

Step 3: Prioritize Interpretability Coefficients should be meaningful for your application:

• Need odds ratios → Logit
• Need multiplicative effects → Log
• Need additive effects → Identity (if valid)

Step 4: Consider Domain Knowledge Does theory or prior research suggest a particular relationship? In pharmacokinetics, log-linear relationships are standard; in psychometrics, probit has theoretical justification.

Comparing Links Empirically

When unsure, you can compare models with different links using:

1. AIC/BIC: Lower is better, but only comparable within the same response distribution
2. Residual plots: Check for patterns indicating systematic misfit
3. Prediction accuracy: Cross-validation error on held-out data
4. Link test: A formal test that the link is correctly specified (Pregibon's link test adds $\hat{\eta}^2$ as a predictor; if significant, the link may be wrong)

However, often different links give very similar fits, and the choice comes down to interpretation and convention.

We've explored link functions—the critical bridge between constrained means and unbounded linear predictors. Let's consolidate the key concepts:

What's Next:

In the next page, we'll explore the exponential family of distributions—the mathematical foundation that makes the GLM framework possible. Understanding the exponential family reveals why certain distribution-link combinations work well and provides the tools for deriving new GLMs.