Mixed models 1
A conceptual introduction
Dr Massimiliano Canzi
From linear regression to mixed models
Did you know: naked mole-rats live underground in East
Africa, in colonies of about 70 animals on average, and up to 295. Only
one female, the queen, breeds.
Source: Smithsonian’s National Zoo
Today
- Repeated measures and the independence assumption
- Old solutions: averaging over participants or items
- Mixed models: varying intercepts and varying slopes
- Partial pooling: borrowing strength across groups
- Interpreting random effects, and the
lme4syntax - Why varying slopes matter
- What is a random effect, anyway?
So far
Lectures 4 to 9: lm(), with one or several predictors,
and how to test them.
Every one of those models assumed that the data points are independent: each one brings new information.
In linguistics, that is almost never true.
Repeated measures
Data have repeated measures when the same units, such as participants, speakers, words or schools, are measured more than once. A participant responds to many words, each word is read by many participants, and a speaker produces many tokens of the same vowel.
Measurements from the same unit tend to be more alike than measurements from different units. That breaks the independence assumption of linear regression.
The data
A simulated lexical decision experiment: 20 participants each respond to the same 40 words.
set.seed(21)
n_subj <- 20
n_item <- 40
z0 <- rnorm(n_subj)
z1 <- rnorm(n_subj)
participants <- tibble(participant = sprintf("P%02d", 1:n_subj),
u0 = 70 * z0, # own speed
u1 = 18 * (-0.6 * z0 + sqrt(1 - 0.36) * z1)) # own frequency effect
words <- tibble(word = sprintf("w%02d", 1:n_item),
log_freq = round(runif(n_item, 1, 6), 2),
w0 = rnorm(n_item, 0, 30)) # own difficulty
lexdec <- crossing(participants, words) |>
mutate(freq_c = log_freq - mean(words$log_freq),
rt = round(650 + u0 + w0 + (-30 + u1) * freq_c + rnorm(n(), 0, 50))) |>
select(participant, word, log_freq, freq_c, rt)The true frequency effect is −30 ms per log unit. But participants differ in speed and in the size of their frequency effect, and words differ in difficulty.
800 rows, but not 800 independent observations
# A tibble: 800 × 5
participant word log_freq freq_c rt
<chr> <chr> <dbl> <dbl> <dbl>
1 P01 w01 3.61 -0.169 657
2 P01 w02 5.95 2.17 515
3 P01 w03 1.32 -2.46 722
4 P01 w04 5.31 1.53 587
# ℹ 796 more rows# A tibble: 3 × 2
participant n
<chr> <int>
1 P01 40
2 P02 40
3 P03 40Everyone together
One cloud of 800 points, one line. This is what
lm(rt ~ freq_c) sees.
One participant at a time
Every participant has their own level (fast or slow) and their own slope (a strong or weak frequency effect).
The independence assumption
Did you know: each naked mole-rat colony has its own
dialect. Researchers analysed over 30,000 soft
chirps from seven colonies: the animals answered calls in their own
colony’s dialect and largely ignored the others.
Sources: Barker et al. (2021), Science; Science News
What goes wrong with lm()?
# A tibble: 2 × 5
term estimate std.error statistic p.value
<chr> <dbl> <dbl> <dbl> <dbl>
1 (Intercept) 662. 3.50 189. 0
2 freq_c -29.5 2.31 -12.8 4.22e-34The estimate is fine. The standard error is computed as if we had 800 independent observations.
But we only have 20 participants. If we ran the experiment again with 20 new participants, the slope could easily change by more than this SE suggests.
The residuals give it away
The residuals of one participant are mostly positive or mostly negative. Knowing who produced a data point tells us something about its residual: they are not independent.
Two sources of non-independence
Participants differ: some are faster than others, and some show bigger effects. Items differ too: some words are harder than others, for everyone.
Clark (1973) called ignoring item variation the language-as-fixed-effect fallacy: if our conclusions should hold for words in general, not just the 40 we tested, the words are a sample too.
Discussion
Think of a study you would like to run, or one you have read.
- Which units does it measure more than once: speakers, words, texts, classes, recording sessions?
- Should the conclusions hold beyond the particular units in the study?
Design and averaging
Did you know: naked mole-rats can live for more than 30
years. In records of over 3,000 animals, their risk of dying did not rise with age, unlike in any other mammal
studied.
Source: Ruby, Smith & Buffenstein (2018), eLife
The traditional fix: averaging
Collapse the data so that each unit contributes one number. A by-participant (F1) analysis: one frequency slope per participant, then test the 20 slopes.
by_participant <- lexdec |>
summarise(slope = coef(lm(rt ~ freq_c))[2], .by = participant)
t.test(by_participant$slope)$conf.int[1] -38.36409 -20.65403
attr(,"conf.level")
[1] 0.95By-item analysis
A by-item (F2) analysis: average over participants, one RT per word.
by_word <- lexdec |> summarise(rt = mean(rt), freq_c = first(freq_c), .by = word)
tidy(lm(rt ~ freq_c, data = by_word))[2, ]# A tibble: 1 × 5
term estimate std.error statistic p.value
<chr> <dbl> <dbl> <dbl> <dbl>
1 freq_c -29.5 2.93 -10.1 2.88e-12Problems with averaging
- Two separate analyses, which can disagree: which one do we report?
- Averaging throws away information: variation within participants is lost
- With missing data or unbalanced designs, averages are based on different numbers of observations
- It is hard to include predictors that vary within a unit, such as trial number
Mixed models handle participants and items, in one model, on the raw data.
Varying intercepts and slopes
Did you know: the dialects are learned: pups raised in another colony took on that
colony’s dialect. When a queen died, her colony’s dialect became more
variable, until a new queen took over.
Sources: Barker et al. (2021), Science; Science News
Fixed and random effects
A linear mixed effects model extends linear regression to repeated measures. It mixes two kinds of terms:
| Fixed effects | Random effects | |
|---|---|---|
| What they are | the effects of the predictors we are interested in | adjustments to those effects for each participant or word |
| Here | the frequency effect,
freq_c |
each participant’s speed and frequency effect, each word’s difficulty |
| Read like | the coefficients of lm() |
how much the grouping units vary |
Three ways to pool
Complete pooling ignores participants and is overconfident. No pooling treats each participant as unrelated to the others, which makes every line noisy. Partial pooling treats participants as different but similar, drawn from one population.
Varying intercepts
Each participant gets their own intercept: the overall intercept plus an adjustment.
b₀,b₁: the fixed effects, shared by everyoneu₀ⱼ: participant j’s adjustment, drawn from a normal distribution with mean 0 and an estimated SD
Varying intercepts, visually
Parallel lines: participants differ in speed but share the same frequency effect.
Varying slopes
Each participant also gets their own slope:
Now the lines fan out: some participants are much more sensitive to frequency than others.
What the model estimates
The model does not estimate 20 separate intercepts and 20 separate slopes as free parameters. It estimates:
- the fixed effects: the average intercept and slope
- the SD of the intercepts across participants
- the SD of the slopes across participants
- the correlation between intercepts and slopes
- the residual SD
The participants’ individual adjustments are then predicted from these, and from each participant’s own data.
Partial pooling = shrinkage
Pink: each participant’s own slope (no pooling). Blue: the mixed model’s estimate. Extreme slopes are pulled, shrunk, towards the average (dashed): the model borrows strength from the other participants.
Interpreting random effects
Did you know: naked mole-rat queens show no menopause.
They stay fertile past the age of 30.
Source: Ruby, Smith & Buffenstein (2018), eLife
Random effect SDs
m_full <- lmer(rt ~ freq_c + (1 + freq_c | participant) + (1 | word), data = lexdec)
VarCorr(m_full) Groups Name Std.Dev. Corr
word (Intercept) 25.758
participant (Intercept) 79.043
freq_c 18.192 -0.630
Residual 49.735 | Row | Varies with an SD of about |
|---|---|
participant (Intercept) |
79 ms: participants’ average RTs |
participant freq_c |
18 ms per log unit: their frequency slopes |
word (Intercept) |
26 ms: how hard the words are |
Residual |
50 ms: what is left, trial by trial |
What an SD of 18 means
The average slope is -29.5 ms per log unit. With an SD of 18 across participants, about 95% of participants should have slopes within ±2 SD:
[1] -65.892736 6.874614A large random-effect SD means that participants differ a lot, even after accounting for frequency. A small one means they behave much alike.
Random effect correlations
The model estimates a correlation of about -0.63: slower participants show stronger (more negative) frequency effects. Correlations like this can be theoretically interesting in their own right.
The lme4 syntax
Did you know: naked mole-rats can survive at least 18 minutes without any oxygen, by running their brain
and heart on fructose instead of glucose.
Sources: Park et al. (2017), Science; Max Delbrück Center
Random effects in lmer()
The {lme4} package: lmer() works like
lm(), plus random effects in brackets,
(effects | group).
| You write | Each group gets |
|---|---|
(1 \| participant) |
its own intercept |
(1 + freq_c \| participant) |
its own intercept and slope, and their correlation |
(1 + freq_c \|\| participant) |
its own intercept and slope, without a correlation |
(0 + freq_c \| participant) |
only its own slope |
(1 \| participant) + (1 \| word) |
crossed effects: every participant sees every word |
(1 \| school / class) |
nested effects: each class belongs to one school |
Our model
The fixed effects, rt ~ freq_c, are written exactly as
in lm(). Then each participant gets their own intercept and
frequency slope, and each word its own intercept.
Why no (1 + freq_c | word)? Each word has one frequency.
A slope needs variation in the predictor within the group: frequency
varies within participants, but not within words.
A first look at the output
(Intercept) freq_c
661.97000 -29.50906 The fixed effects read like lm() coefficients: an
average RT of 662 ms at average frequency, and -29.5 ms per log unit of
frequency.
Next week: the full output, significance testing, likelihood ratio tests, and what to do when models do not converge.
Why varying slopes matter
Same estimate, different certainty
models <- list(
"lm(rt ~ freq_c)" = lm(rt ~ freq_c, data = lexdec),
"+ (1 | participant)" = lmer(rt ~ freq_c + (1 | participant), data = lexdec),
"+ (1 + freq_c | participant)" = lmer(rt ~ freq_c + (1 + freq_c | participant), data = lexdec),
"+ (1 + freq_c | participant) + (1 | word)" = m_full
)| model | estimate | SE |
|---|---|---|
| lm(rt ~ freq_c) | -29.51 | 2.31 |
| + (1 | participant) | -29.51 | 1.45 |
| + (1 + freq_c | participant) | -29.51 | 4.23 |
| + (1 + freq_c | participant) + (1 | word) | -29.51 | 5.01 |
The estimate is the same in every model. The standard error is not:
random intercepts alone make it smaller than lm(), and
varying slopes make it about three times larger than with random
intercepts alone.
Why?
With random intercepts only, the model assumes every participant has the same frequency effect. All 800 trials then count as evidence about one slope.
But participants differ in their slopes. To generalise to new participants, the model must know how much slopes vary, and that uncertainty belongs in the SE.
Barr et al. (2013) showed that random-intercept-only models can have Type I error rates well above 5%, and recommend the maximal random effects structure justified by the design for confirmatory tests.
Compare with averaging
| analysis | SE |
|---|---|
| By-participant (F1) analysis | 4.23 |
| Mixed model, (1 + freq_c | participant) | 4.23 |
With varying slopes, the mixed model is about as cautious as the by-participant analysis, while keeping all the data, and adding words in the same model.
Different interpretations
What is a random effect?
There is no single answer. Gelman (2005) lists five definitions in use, summarised by Coretta (2021):
- Constant vs varying: fixed coefficients are constant across individuals; random coefficients vary (Kreft & de Leeuw, 1998)
- Researcher’s interest: effects are fixed if the research question is about them; random if the interest lies in the underlying population (Searle, Casella & McCulloch)
- Sampling: “When a sample exhausts the population, the corresponding variable is fixed”; when the sample is a negligible part of it, the variable is random (Green & Tukey, 1960)
What is a random effect?
- Random variable: “If an effect is assumed to be a realized value of a random variable, it is called a random effect” (LaMotte, 1983)
- Estimation: fixed effects are estimated by least squares or maximum likelihood; random effects are estimated with shrinkage, that is, partial pooling (Snijders & Bosker)
The definitions often disagree. Is “word” random because the words are a sample (3), because they vary (1), or because their estimates are shrunk (5)?
So which is it?
- Gelman recommends avoiding “fixed” and “random” altogether: effects are constant or varying across groups
- In Bayesian work, population-level and group-level effects are increasingly common terms
- In practice: a grouping factor gets random effects when its levels are a sample from a larger population (participants, words, speakers), and you want to generalise beyond them
Whatever the label, ask: which units are repeated in my data, and which effects can vary across them?
Your turn
Which random effects?
For each design, name the grouping factors, and the varying slopes the design allows:
- 30 speakers each read 50 sentences, half of them questions and half statements. Outcome: final pitch rise.
- 24 children are recorded at ages 2, 3 and 4. Outcome: vocabulary size.
- 40 participants rate 60 words for emotional valence. Each word has one frequency and one length.
Possible answers
- Speakers and sentences.
(1 + type | speaker) + (1 | sentence): type varies within speakers, but each sentence is one type - Children.
(1 + age | child): age varies within each child - Participants and words.
(1 + frequency + length | participant) + (1 | word): frequency and length vary within participants, not within words
Summary
- Repeated measures violate independence: data points from the same unit are alike
- Ignoring this makes SEs too small; averaging throws information away
- Mixed models add varying intercepts and varying slopes for each grouping factor
- They estimate the SD of these adjustments, and shrink individual estimates towards the average
(1 + x | participant) + (1 | item): slopes only for predictors that vary within the group- “Random effect” has several meanings: think constant vs varying
Slides and code
These slides
mcanzi.de/ws26_statistics/
Code, data and all lectures
github.com/mcanzi/ws26_statistics