# Bad run or bad manager? Spotting a real change with CUSUM

Source: https://www.footballdatascience.co.uk/learn/bad-run-or-bad-manager
Published: 2026-10-12

> A run of ten games five points below what the odds expected turns up in 56% of Scottish top-flight seasons, and luck alone produces one in 55%. Clubs then go back to the level the odds expected. An alarm built to catch a real change shows why boards can't tell a bad manager from a bad run in time.

**On the terraces:** Lose a few on the bounce and the fans want the manager out. But a run that bad turns up in about half of all seasons by luck alone. This piece checks what happens after one, and whether any alarm can tell a bad manager from a bad run.

## The football question

**Ten games, five points short of where you should be. Is it a bad run or a bad manager?**

Every board faces it. Sack too early and you lose a good manager to a bad month. Wait too long and a bad appointment costs you the season. Fans, pundits and directors all look at the same run of results and see different things in it.

Statistics has a tool built for exactly this: watching a stream of results and deciding, as early as possible, whether something has really changed. It's called **CUSUM**, and it was invented for factory lines in the 1950s. Before using it, though, you need to know how much a run of results can wobble by luck alone.

## The concept

A run of results only means something against what the team **should** have got. Ten games against Celtic, Rangers and the top six aren't the same as ten against the bottom half, and five points from ten is a crisis at one club and normal at another.

The bookmakers' odds already price all of that in: the opponent, home or away, the squad, injuries, form. Turn the three prices into chances (taking out the bookmaker's margin, as in [the bookmaker odds piece](/research/bookmaker-odds)) and you get each team's **expected points** for the match:

$$\begin{aligned} \text{expected points} = \;&3 \times P(\text{win}) \\ &+ 1 \times P(\text{draw}) \end{aligned}$$

<div class="plain" markdown="1">
In plain football

- A side given a 50% chance of winning and a 25% chance of a draw expects 3 × 0.5 + 0.25 = **1.75 points**.
- Over a run, add up the points actually won and take away the points the odds expected. A side **5 points below the odds** got five points fewer than its matches were worth.
</div>

Even a side doing exactly what the odds say won't land on its expected points. Over 24 seasons of Scottish top-flight matches, one game's result sits **1.19 points** either side of what the odds expected, on average. Over ten games those wobbles partly cancel, but not much: about **3.8 points** either side ([standard deviation](/learn/normal-distribution), √10 × 1.19). A run of ten games five points below the odds is only a little more than one ordinary wobble.

## How often does luck alone produce a bad run?

Take every real team-season from 2002/03 to 2025/26, 288 of them, keep the real fixtures and the real odds, and replay each season twenty times with every result drawn from the odds' own chances. Every simulated team is **exactly as good as the bookmakers thought**. Then look for a run of ten games at least five points below the odds.

| Seasons with a ten-game run 5+ points below the odds | Share |
|---|---|
| Results drawn from the odds (luck alone) | 55% |
| Real seasons | 56% |

More than half of seasons hold a run like that **by luck alone**, and real seasons have them about as often. Most bad runs need no bad manager to explain them.

## What happens after a bad run?

If a run reveals a real problem, it should carry on. Taking every time a club fell 5+ points below the odds over ten games, **186 runs**, here are the club's next ten league games, whoever was in charge by then (following the manager instead would leave out the ones who were sacked, and keep only the survivors, the same trap as in [the substitutions piece](/research/do-substitutions-work)):

| Points a game | All clubs (186) | Without Old Firm (168) |
|---|---|---|
| Odds expected, before the run | 1.32 | 1.23 |
| Odds expected, next ten | 1.28 | 1.19 |
| Actually won, next ten | **1.31** | **1.21** |

After a bad run, clubs went straight back to about the level the odds expected **before** it. Against the odds, the next ten games came out **+0.31 points** (luck margin ± 0.50), nothing at all. The bookmakers barely moved their view either (1.32 to 1.28), and they were right not to. Without Celtic and Rangers the picture is the same.

That's **regression to the mean**: a run picked because it was bad is mostly bad luck, and bad luck doesn't carry on. The same pattern shows up in [form against underlying performance](/myth-or-maths/form-vs-underlying), and it's why the [new manager bounce](/myth-or-maths/new-manager-bounce) mostly happens to clubs that kept their manager too.

## CUSUM: an alarm for a real change

If a ten-game window can't tell bad luck from a bad manager, perhaps a smarter alarm can. **CUSUM** (cumulative sum, from E. S. Page's 1954 paper) keeps a running tally of how far results fall short, forgives a little each game so ordinary wobbles fade away, and sounds when the tally gets too big:

$$\begin{aligned} S_n = \max\big(0,\; &S_{n-1} + \text{expected}_n \\ &- \text{points}_n - k\big) \end{aligned}$$

<div class="plain" markdown="1">
In plain football

- **S** is the tally of points dropped below the odds. After each game, add what the odds expected and take off what you actually won.
- **k** is the forgiveness: a quarter of a point a game here. It stops the tally creeping up from a side that's only a fraction below par.
- **max(0, ...)** means the tally never goes below zero. A good run wipes the slate; it doesn't bank credit.
- The alarm sounds when **S** reaches a level you set in advance, say **4 points**.
</div>

## A football example

A made-up side, expected **1.4 points** in every game, wins its first match and then has a sticky spell:

<figure class="rank-chart">
<div role="img" aria-label="The CUSUM tally after each of eight results: win 0, loss 1.15, loss 2.3, draw 2.45, loss 3.6, win 1.75, loss 2.9, loss 4.05, which crosses the alarm at 4 points.">

</div>
<figcaption>The tally after each result for a made-up side expected 1.4 points a game, forgiving a quarter of a point each game. It crosses the 4-point alarm at the eighth game.</figcaption>
</figure>

- **Game 1, a win:** 1.4 − 3 − 0.25 is below zero, so the tally stays at **0**.
- **Game 2, a loss:** 1.4 − 0 − 0.25 = 1.15 dropped. Tally **1.15**.
- **Game 4, a draw:** 1.4 − 1 − 0.25 = 0.15. Tally **2.45**.
- **Game 6, a win:** takes 1.85 off. Tally **1.75**.
- **Game 8, a loss:** tally **4.05**. The alarm sounds.

Seven points from eight games when the odds expected 11.2. That feels like a crisis, but it's 4.2 points short, and luck alone spreads eight games by about 3.4 points either side. The alarm has gone off on a side that could easily be exactly as good as its odds.

## Would it have caught the sacked managers?

Using the 158 managers from [how long a manager lasts](/learn/how-long-does-a-manager-last), **68** were sacked or left by mutual consent after at least ten league games with Bet365 prices.

They really were below par: **75%** finished below the odds over their whole time in the job, against **45%** of the other managers. But not by much. The typical sacked manager was **0.09 points a game** below the odds, about three and a half points over a season, and **3.4 points** below in his last ten games: less than one ordinary ten-game wobble.

So run the CUSUM alarm (forgiving 0.25 a game) through every sacked manager's real results. Then, for a fair comparison, replay **the same fixtures** with results drawn from the odds, as if each manager had been exactly par, and see how often the alarm goes off anyway.

<figure class="rank-chart">
<div role="img" aria-label="Share flagged at each alarm level. At 3 points, 85% of sacked managers against 73% on the same fixtures at par. At 4 points, 71% against 54%. At 5 points, 43% against 38%. At 6 points, 25% against 26%.">

</div>
<figcaption>Sacked managers flagged by the CUSUM alarm (mint) against managers exactly as good as their odds on the same fixtures (moss), at four alarm levels.</figcaption>
</figure>

At **4 points** the alarm catches **71%** of the managers who were later sacked. It would also have gone off on **54%** of them if they had been doing exactly what the odds expected. Make it stricter and it catches fewer, while the par managers keep pace: at 6 points the two are level. Give a par side two full seasons and the 4-point alarm sounds on **84%** of them.

There's no setting that catches most bad managers without also condemning most ordinary ones. The gap between a sacked manager and a par one, about a tenth of a point a game, is small next to luck, and a few dozen games aren't enough to see it.

## Why it matters

CUSUM is used where catching a change early matters: a machine drifting out of tolerance, a hospital's surgical outcomes, a rise in cases of a disease. It works there because the change it hunts is large next to the noise, or the stream is long. Football results give neither. A manager who is a tenth of a point a game worse than par needs hundreds of games to show it, and nobody gets hundreds of games.

For a board, that means a bad run on its own is weak evidence. It's worth asking what the run is measured against (the odds, not the league table), whether something else has changed (injuries, the chances being created), and what the club would have done if the same results had come from luck. The same question in a single match, whether to change things at 1–0 down, is in [the substitutions piece](/research/do-substitutions-work): most of the pick-up after a change comes anyway. And when a new manager does start well, [Bayesian thinking](/learn/new-manager-better) shows how much a handful of games can tell you.

## Limitations

- **The odds aren't neutral about the manager.** A manager who improves a squad over a season raises its odds, and the alarm then judges him against his own good work. CUSUM against the odds measures surprise, not quality.
- **Sacked isn't the same as bad.** Some sacked managers were let go for reasons that aren't results; some kept their jobs while results slipped. The comparison asks only whether results could have told them apart.
- **One alarm design.** Other forgiveness values move the trade-off, but they don't open a gap: forgiving nothing catches 94% of the sacked managers at 4 points and 81% of par ones.
- **Bet365 prices from 2002/03 only**, so managers before then are left out, and the tenures carry the limits listed in [Statistics #13](/learn/how-long-does-a-manager-last).
- **No manager is named.** The data can't single out who was unlucky and who was poor, so it doesn't try.

## Reproduce the analysis

Save [manager-tenures-spfl.csv](/static/data/manager-tenures-spfl.csv) and the Premiership results files (SC0) from football-data.co.uk, which the [download script](/data#get-the-files) fetches under the names the snippet expects, in one folder. It needs only Python 3 and takes about a second:

```python
# From Football Data Science by Bryan McGuire. Free to use with credit.
# https://www.footballdatascience.co.uk/learn/bad-run-or-bad-manager
import csv
import random
from collections import defaultdict
from datetime import date, datetime
from math import sqrt
from statistics import mean, median, pstdev

rnd = random.Random(2026)  # fixed, so the simulated seasons repeat exactly
OLD_FIRM = ("Celtic", "Rangers")

# every league match from both sides: the odds' chances (win, draw, loss) and the points won
matches = []
for y in range(2002, 2026):  # Bet365 prices start in 2002/03
    with open(f"SC0_{y % 100:02d}{(y + 1) % 100:02d}.csv", encoding="latin-1") as f:
        for r in csv.DictReader(f):
            if r.get("FTR") not in ("H", "D", "A") or not r.get("B365H"):
                continue
            d = datetime.strptime(r["Date"], "%d/%m/%Y" if len(r["Date"]) == 10 else "%d/%m/%y").date()
            h, dr, a = (1 / float(r[k]) for k in ("B365H", "B365D", "B365A"))
            s = h + dr + a  # take out the bookmaker's margin
            home = {"H": 3, "D": 1, "A": 0}[r["FTR"]]
            matches.append((d, y, r["HomeTeam"], (h / s, dr / s, a / s), home))
            matches.append((d, y, r["AwayTeam"], (a / s, dr / s, h / s), 1 if home == 1 else 3 - home))
matches.sort(key=lambda m: m[0])

def expected(p):  # expected points from the odds
    return 3 * p[0] + p[1]

def by_luck(p):  # a result drawn from the odds' own chances
    u = rnd.random()
    return 3 if u < p[0] else 1 if u < p[0] + p[1] else 0

# 1. how much luck is in a run of results?
spread = mean(9 * p[0] + p[1] - expected(p) ** 2 for *_, p, _ in matches)
print(f"{len(matches)} team-matches; luck: ± {sqrt(spread):.2f} points a game, ± {sqrt(10 * spread):.1f} over ten")

# 2. how often does a season hold a run of ten games 5+ points below the odds?
seasons = defaultdict(list)
for d, y, team, p, pts in matches:
    seasons[(team, y)].append((p, pts))

def worst_ten(gaps):
    return min(sum(gaps[i:i + 10]) for i in range(len(gaps) - 9))

real = [worst_ten([pts - expected(p) for p, pts in g]) <= -5 for g in seasons.values()]
luck = [worst_ten([by_luck(p) - expected(p) for p, _ in g]) <= -5 for g in seasons.values() for _ in range(20)]
print(f"seasons with a ten-game run 5+ below the odds: real {mean(real):.0%} of {len(real)}, "
      f"results drawn from the odds {mean(luck):.0%}")

# 3. after such a run, the club's next ten league games (whoever the manager)
clubs = defaultdict(list)
for d, y, team, p, pts in matches:
    clubs[team].append((expected(p), pts))
for label, keep in (("all clubs", lambda t: True), ("without Celtic and Rangers", lambda t: t not in OLD_FIRM)):
    before, odds_after, after, gap = [], [], [], []
    for team, g in clubs.items():
        i = 10
        while keep(team) and i + 10 <= len(g):
            if sum(pts - e for e, pts in g[i - 10:i]) <= -5:
                nxt = g[i:i + 10]
                before.append(mean(e for e, _ in g[i - 10:i]))
                odds_after.append(mean(e for e, _ in nxt))
                after.append(mean(pts for _, pts in nxt))
                gap.append(sum(pts - e for e, pts in nxt))
                i += 20  # the next run starts after these ten
            else:
                i += 1
    print(f"{label}: {len(gap)} runs; next ten {mean(after):.2f} points a game, the odds expected "
          f"{mean(before):.2f} before the run and {mean(odds_after):.2f} after it; "
          f"against the odds {mean(gap):+.2f} ± {2 * pstdev(gap) / sqrt(len(gap)):.2f} points")

# 4. sacked managers: how far below the odds were they?
tenures = []
with open("manager-tenures-spfl.csv", encoding="utf-8") as f:
    for r in csv.DictReader(f):
        start, end = date.fromisoformat(r["appointed"]), date.fromisoformat(r["until"])
        games = [(p, pts) for d, y, team, p, pts in matches if team == r["club"] and start < d <= end]
        if len(games) >= 10:
            pushed = r["left"] == "1" and r["how"] in ("Sacked", "Mutual consent", "Contract terminated")
            tenures.append({"sacked": pushed, "games": games})
sacked = [t["games"] for t in tenures if t["sacked"]]
rest = [t["games"] for t in tenures if not t["sacked"]]
gap = lambda g: sum(pts - expected(p) for p, pts in g)
print(f"\n{len(sacked)} sacked managers with 10+ games: {mean(gap(g) < 0 for g in sacked):.0%} below the odds "
      f"over their whole time (the other {len(rest)}: {mean(gap(g) < 0 for g in rest):.0%}); "
      f"median {median(gap(g) / len(g) for g in sacked):+.2f} a game, "
      f"{median(gap(g[-10:]) for g in sacked):+.1f} in their last ten")

# 5. CUSUM: a running tally of points dropped below the odds, forgiving a quarter-point a game
def alarm(gaps, alarm_at, forgive=0.25):
    tally = 0
    for game, g in enumerate(gaps, 1):
        tally = max(0, tally - g - forgive)
        if tally >= alarm_at:
            return game
    return None

print("alarm at   sacked managers   the same fixtures, results drawn from the odds")
for alarm_at in (3, 4, 5, 6):
    hit = mean(alarm([pts - expected(p) for p, pts in g], alarm_at) is not None for g in sacked)
    par = mean(alarm([by_luck(p) - expected(p) for p, _ in g], alarm_at) is not None for g in sacked for _ in range(200))
    print(f"  {alarm_at} points   {hit:9.0%}   {par:9.0%}")
hit = mean(alarm([pts - expected(p) for p, pts in g], 4, forgive=0) is not None for g in sacked)
par = mean(alarm([by_luck(p) - expected(p) for p, _ in g], 4, forgive=0) is not None for g in sacked for _ in range(200))
print(f"forgiving nothing, alarm at 4 points: sacked managers {hit:.0%}, the same fixtures at par {par:.0%}")
two = [g + seasons[(team, y + 1)] for (team, y), g in seasons.items() if (team, y + 1) in seasons]
par = mean(alarm([by_luck(p) - expected(p) for p, _ in g], 4) is not None for g in two for _ in range(20))
print(f"a side exactly as good as its odds, two full seasons: the 4-point alarm goes off {par:.0%} of the time")
```

## Further reading

- [CUSUM](https://en.wikipedia.org/wiki/CUSUM), Wikipedia. The method, E. S. Page's 1954 paper that introduced it, and how the tally and the alarm level are set.
- [Regression toward the mean](https://en.wikipedia.org/wiki/Regression_toward_the_mean), Wikipedia. Why anything picked for being extreme tends to look more ordinary next time.
- [Change detection](https://en.wikipedia.org/wiki/Change_detection), Wikipedia. The wider family of methods for spotting the moment a stream of data changes.