Experimental Design

How data are collected determines what conclusions they can support. Good experimental design is what lets epidemiologists move from “associated with” toward “causes,” and recognizing sources of bias is essential when a true experiment is impossible.

Figure 1. Left: randomization balances confounders. In an observational study an unmeasured confounder

U

feeds both the exposure

X

and the outcome

Y

, biasing the

X\to Y

estimate; randomly assigning

X

severs the

U\to X

arrow, so the remaining

X\to Y

path is the causal effect. Right: blocking groups similar units (by age or site) and randomizes treatment within each block, removing that nuisance variation from the comparison.

Experimental vs. observational studies#

In an experiment, the investigator actively assigns the exposure or treatment (ideally at random). This breaks the link between exposure and confounders and licenses causal claims.
In an observational study, exposure is merely observed as it occurs. Associations may be real but can be distorted by confounding, so causal interpretation demands caution.

Principles of good experiments#

Randomization — random assignment balances known and unknown confounders across groups in expectation, so differences in outcome can be attributed to the treatment (Figure 1).
Control — a comparison group (placebo or standard care) isolates the treatment effect from background trends.
Replication — multiple independent units per condition let us estimate variability and gain power.
Blocking — grouping similar units (e.g., by age band or site) and randomizing within blocks removes nuisance variation and sharpens comparisons.

Observational designs#

Retrospective (e.g., case-control): start from outcome and look back at exposures.
Prospective (e.g., cohort): enroll by exposure status and follow forward for outcomes.
Sample survey: measure a population cross-section at a point in time (see survey sampling).

Sources of bias#

Non-response bias — those who decline differ systematically from responders.
Self-selection bias — participants choose their own exposure/group.
Measurement error — mismeasured exposure or outcome (recall, instrument).
Cohort effects — birth-cohort differences masquerade as age or time effects.
Omitted-variable bias / confounding — a common cause of exposure and outcome is ignored.
Temporality — unclear whether exposure preceded outcome.
Spatial dependence — nearby units share unmeasured influences, violating independence.
Spillover / interference — one unit’s treatment affects another’s outcome (e.g., herd immunity).
Compliance — assigned treatment differs from treatment received.

Randomization in code#

Randomly assign $n=10$ subjects to treatment vs. control.

R#

set.seed(1)
subjects <- 1:10
assignment <- sample(rep(c("treat", "control"), each = 5))  # random labels
data.frame(subject = subjects, group = assignment)

Python#

Python

import numpy as np
rng = np.random.default_rng(1)
labels = np.array(["treat"] * 5 + ["control"] * 5)
rng.shuffle(labels)                     # random assignment
print(list(zip(range(1, 11), labels)))

[(1, np.str_('control')), (2, np.str_('treat')), (3, np.str_('control')), (4, np.str_('treat')), (5, np.str_('treat')), (6, np.str_('treat')), (7, np.str_('control')), (8, np.str_('control')), (9, np.str_('control')), (10, np.str_('treat'))]

Julia#

Julia

using Random
Random.seed!(1)
labels = vcat(fill("treat", 5), fill("control", 5))
shuffle!(labels)                        # random assignment
foreach(println, zip(1:10, labels))

Why it matters for statistics#

Design decides which biases the analysis can and cannot fix: no statistical adjustment recovers information a flawed design never captured. Randomization and control are what make inferred effects causal, and naming the biases of observational studies guides both honest interpretation and better analysis.