A Lead Time Is Not a Detection

The finding

A small transformer learning modular addition memorises its training set within a few hundred steps, then sits at chance on held-out data for thousands more, then suddenly generalises. Several signals have been proposed that are supposed to warn you this is coming.

I fixed the rule for what counts as a warning before looking at any of the results: a signal has to move five standard deviations away from its own early-training baseline and stay there for three consecutive checkpoints.

Under that rule, the three Fourier measures warned before the jump on all eight seeds, with a median lead of roughly 2,600 steps. Weight norm, which is widely cited, never fired once. Effective embedding rank fired on five of eight seeds only after generalisation had already happened, which is not a warning at all.

Why it matters

A lead time is a number you can always produce. Take any curve, pick the moment it crosses some line, and subtract. You get a positive number, and a positive number looks like a result.

If the threshold is chosen after seeing where the jump was, the number measures your freedom to choose, not the signal. The gap between the two is the whole subject of the paper: one of these signals survives a rule it did not get to pick, and the others do not.

Method

A one-layer transformer, 128-dimensional model width, four heads, trained on (a + b) mod 97 with 30% of the pairs used for training. 223,360 parameters, 12,000 steps, weight decay 1.0, on CPU. The sweep above fixed the weight decay and training fraction, because grokking does not happen everywhere in that space.

Six signals were tracked at every hundredth step: weight norm, embedding norm, effective embedding rank, and three Fourier measures of the embedding spectrum. The firing rule was written down first and applied to all of them identically.

Results

Seed

01234567

SignalFourier top-k powerFourier entropyFourier effective frequenciesEmbedding normEffective embedding rankWeight norm

train accuracy test accuracy signal fires test accuracy crosses 0.5

2,500 steps of lead on seed 0: fired at 1,600, grokking at 4,100.

Weight norm fired on zero of eight seeds. It is the signal most often quoted, and under a rule it did not choose it detects nothing.

The three Fourier measures each fired on eight of eight, with median leads of 2,550, 2,650 and 2,650 steps. Embedding norm also fired on all eight, with a shorter median lead of 1,700.

Effective embedding rank is the interesting failure. It fired on eight seeds, so a table of firing counts would make it look successful, but on five of those the firing step came after the test-accuracy crossing. Averaging its lead times without checking the sign would have produced a positive mean and a false claim.

What I checked that could have broken it

The obvious worry is that the result is specific to addition, so I repeated the whole measurement on modular subtraction with five fresh seeds. Weight norm again fired on zero of five. The Fourier measures again fired on five of five, with longer median leads of 4,600 to 4,900 steps.

The second worry is the definition of grokking itself. The step at which a run counts as grokked depends on the accuracy threshold, and the lead times shift with it. The lead-time study uses a test accuracy of 0.5, and both thresholds are recorded in the run files so the choice is visible rather than buried.

Limits

One task family, one architecture, one model size. Modular arithmetic is where grokking is easiest to produce and study, and nothing here shows the same signals behave this way on a real dataset.

Eight seeds is enough to separate zero from eight. It is not enough to put a confident interval on a median lead time, and the spread across seeds is wide: the standard deviation on the Fourier leads is over 1,100 steps.

The Fourier measures are also the signals most obviously related to the mechanism people believe drives grokking on this task. That they work here is a smaller claim than it might look: they were the best guess going in, and they were tested on the task that guess was made for.