Sacked after a bad run, poached after a good one. Event studies

The football question

A manager is sacked and the next six games go far better: +0.56 points a game. A manager is poached by a bigger club and the next six go worse: −0.29. Put the two together and it looks like proof that managers matter. Get rid of a bad one and you improve; lose a good one and you suffer.

Is that what's going on?

The concept

An event study lines up every case by the moment the event happened, here the day the manager left, and looks at the results in each game before and after. Instead of one before-and-after number, you see the whole shape: how things were going, the change, and what came next.

The most useful part is the part people skip: the games before. They show why the event happened. If clubs only change manager after a slump, the games before will show the slump, and anything that happens afterwards has to be read against it.

Two kinds of departure

The site's manager-changes data records why each in-season change happened. Two kinds stand out:

• Pushed out: sacked, resigned, left by mutual consent or contract terminated. These follow how things were going. There are 81 with six league games either side in the same season.
• Poached: signed by another club (or, once, by Scotland). These follow how well things were going: clubs come for managers whose sides are doing well. There are 16.

The two lines are mirror images.

Pushed out. The sides were already sinking: 1.12 points a game six games out, down to 0.35 in the last game before the change. Over the six games before, 0.84 a game. After, 1.39. A bounce of +0.56, well outside its luck margin of ±0.17.

Poached. The sides were flying: 2.12 points a game two games before, and 1.57 over the six games before. After the manager left, 1.28. A fall of −0.29, inside its luck margin of ±0.35, because 16 changes is a small number.

The level they came back to

The key comparison is with each club's own normal level: its points a game over the rest of the same season, outside those twelve games.

Both groups end up close to where the club usually is. The sacked sides weren't 0.84 sides; they were around 1.3 sides having a bad spell. The poached sides weren't 1.57 sides; they were around 1.3 sides having a good one. After the change, both went back. That's regression to the mean: an extreme run is partly luck, and luck doesn't last, whoever is in the dugout.

Why it matters

Event studies began in finance, lining up share prices around company announcements, and they're now used wherever something happens at a known moment. Their strength is the picture: a reader can see at a glance whether things were already changing before the event. A flat line before and a jump after is evidence. A dip before and a recovery after is a warning.

This is the same trap as Part 1, seen in time. To estimate what the change itself did, you need a comparison: sides on the same kind of run that kept their manager. That's exactly what the new manager bounce does, matching each change to sides with the same recent points, and it finds sides that kept their manager bounced just as much. The event study shows why that matching is needed, and the next part of this series is about matching itself.

Limitations

• Only 16 poached managers. The fall after a poaching is inside its luck margin, so on its own it isn't proof of anything. It's the mirror image of the sackings, and that's the point.
• Points a game swing with the fixtures. Six games can include the Old Firm or not, so single games jump around; the rest-of-season level evens that out but isn't a perfect "normal".
• The categories are taken from the records. "Mutual consent" can hide a sacking, and a resignation can be pushed; they're grouped with the sackings because they usually follow bad results.
• Six games in the same season. Changes too near the start or end of a season are left out, and there's no table for 2010/11.

Reproduce the analysis

Save the Premiership results files (SC0) from football-data.co.uk and the site's manager-changes-spfl.csv in one folder; the download script fetches the results files under the names the snippet expects. It needs only Python 3 and takes about a second:

# From Football Data Science by Bryan McGuire. Free to use with credit.
# https://www.footballdatascience.co.uk/learn/event-studies-managers
import csv
from collections import defaultdict
from datetime import datetime
from math import sqrt
from statistics import mean, pstdev

N = 6  # games either side of the change
PUSHED = ("Sacked", "Resigned", "Mutual consent", "Contract terminated")  # left because of how things were going
POACHED = ("Signed by", "Joined")  # taken by another club

games = defaultdict(list)  # (season, club) -> (date, points) for each league game
for y in range(2000, 2026):
    s = f"{y % 100:02d}{(y + 1) % 100:02d}"
    with open(f"SC0_{s}.csv", encoding="latin-1") as f:
        for r in csv.DictReader(f):
            if r.get("FTR") not in ("H", "D", "A"):
                continue
            d = datetime.strptime(r["Date"], "%d/%m/%Y" if len(r["Date"]) == 10 else "%d/%m/%y")
            x, z = int(r["FTHG"]), int(r["FTAG"])
            games[s, r["HomeTeam"]].append((d, 3 * (x > z) + (x == z)))
            games[s, r["AwayTeam"]].append((d, 3 * (z > x) + (x == z)))

with open("manager-changes-spfl.csv", encoding="utf-8") as f:
    changes = list(csv.DictReader(f))

def events(kinds):
    """Each change of this kind with N games either side in the same season: the 2N results, and the rest of the season."""
    out = []
    for c in changes:
        key = (c["season"][2:4] + c["season"][5:7], c["club"])
        if not c["left"].startswith(kinds) or key not in games:
            continue
        season = sorted(games[key])
        i = sum(d <= datetime.fromisoformat(c["date"]) for d, _ in season)  # games played before the change
        pts = [p for _, p in season]
        if i >= N and len(pts) - i >= N:
            out.append((pts[i - N:i + N], pts[:i - N] + pts[i + N:]))
    return out

def average(values):  # mean and luck margin
    return mean(values), 2 * pstdev(values) / sqrt(len(values))

for label, kinds in (("pushed out (sacked, resigned, mutual consent)", PUSHED), ("poached by another club", POACHED)):
    ev = events(kinds)
    before, after = average([mean(w[:N]) for w, _ in ev]), average([mean(w[N:]) for w, _ in ev])
    change = average([mean(w[N:]) - mean(w[:N]) for w, _ in ev])
    print(f"{label}: {len(ev)} changes")
    print("  game by game: " + ", ".join(f"{k - N if k < N else k - N + 1:+d} {mean(w[k] for w, _ in ev):.2f}" for k in range(2 * N)))
    print(f"  six before {before[0]:.2f}, six after {after[0]:.2f}: a change of {change[0]:+.2f} (luck margin ± {change[1]:.2f})")
    print(f"  the same clubs' rest of the season: {mean(mean(r) for _, r in ev):.2f} points a game")

Further reading

• Event study, Wikipedia. The method as it began in finance, measuring "abnormal" returns around an announcement against what was expected.
• Regression toward the mean, Wikipedia. Why anything picked for being extreme tends to look more ordinary next time.