Best eleven or best combination? Cohort analysis and the midfield partnership

The football question

Pick your best eleven players, or the eleven that work best together?

A manager with 25 players has to choose eleven. A creative midfielder may be wasted without someone doing his running. Two good holding players may get in each other's way. And everyone has heard that a new partnership needs time to gel.

Those are two different questions, and statistics has a different tool for each. Does the mix matter? compares groups that share a make-up. Do partnerships get better? follows groups from the day they start. The second is cohort analysis, and it hides one of the commonest traps in data.

This piece answers both from every starting line-up in a full season of four of Europe's top leagues.

Data: StatsBomb open data. The analysis is ours, not StatsBomb's.

The concept

Segment analysis (also called subgroup analysis) splits things into groups that share a trait and compares the groups: midfields with two holding players against midfields with one.

Cohort analysis groups things by a shared starting point and follows each group over time. In business it's every customer who signed up in January, tracked month by month to see how many are still paying. In football it can be every midfield pair who first started together in the autumn, tracked game by game.

Both go wrong in the same way. Who ends up in a group isn't random. Strong teams pick different midfields from weak ones, and a manager keeps a partnership going only while it's working. Compare the groups as they stand and you're partly comparing the teams and the manager's choices, not the thing you set out to measure.

StatsBomb's open data records every event in the 2015/16 Premier League, La Liga, Serie A and Ligue 1: 1,517 matches (Ligue 1 is three short of its 380), with each side's starting eleven, the position each player started in, and every shot with its expected goals (xG: how often a chance like that goes in). The measure of a match here is the xG difference, a side's xG minus its opponent's, which tracks how well a side played more closely than the score does.

Sorting the midfielders

To compare a creative midfielder with a holding one, each player needs a type. The 340 midfielders with at least 10 starts in central midfield and 900 minutes all season are sorted by two numbers per 90 minutes: defending (tackles plus interceptions) and creating (passes that set up a shot, plus completed passes into the box).

Counted as they stand, those numbers carry the team inside them. A midfielder whose side rarely has the ball gets far more chances to tackle, and one at a side that keeps it gets more chances to create. So each count is scaled to what it would be with the ball shared evenly:

$$\begin{aligned} &\text{defending, adjusted} \\ &= \text{per 90} \times \frac{0.5}{\text{their share of the ball}} \end{aligned}$$

That adjustment matters: 60 of the 340 change type once the ball's share is allowed for.

Does the mix matter?

That leaves 1,279 matches where a side started a three-man central midfield with all three players typed. Straight averages flatter whoever the strong teams pick, so each match is also judged like for like, against what the two sides' strength and home advantage predicted:

$$\begin{aligned} &\text{expected xG difference} \\ &= 0.87 \times \text{strength gap} \\ &\quad \pm 0.33 \text{ (home or away)} \end{aligned}$$

The most common mixes:

The textbook three, one to sit, one to create and one to get about, is the most common midfield by a distance, but like for like it's +0.06, inside its luck margin. No mix stands clear of luck. The best and worst like-for-like figures sit 0.24 xG a match apart, and their margins overlap.

The trap shows best in the question fans argue about most, whether two holding midfielders are one too many:

Count tackles as they stand and two holding midfielders look disastrous: 0.64 xG a match worse than none, nearly two goals every three games. Most of that was the label. "Defensive" midfielders were partly just midfielders at sides without the ball, and those sides are weaker. Allow for the ball and the raw gap falls to 0.09. Like for like, either way, it's 0.1 to 0.2 xG a match, and once the ball is allowed for, the luck margin (± 0.13) is as big as the gap.

So the honest answer is: a second holding midfielder might cost a little, around a tenth of an xG a match, but one season can't tell that from luck. Big raw gaps like the one on the left of the chart come from who plays two holders, not the two holders themselves. The same trap, a confounder riding along with the thing you're measuring, is in do football stats mislead?

Do partnerships get better with time?

Now the cohort question. Every pair of central midfielders who started a match together for the same side is a partnership: 1,859 of them in the season. Group each one by when it first started, and count how many were still being picked together:

That's a cohort table: one row per starting point, followed along the columns. Later cohorts thin out faster partly because the season ran out on them: a pair formed in March can't reach 20 games, and 6% of the autumn pairs did. They're censored, still going when the watching stopped, and comparing the rows as if they'd had the same time would be unfair to the later ones.

The table also shows how quickly managers break things up. A third of the autumn partnerships didn't last three games.

Now the obvious question. How do partnerships play, like for like, by how many games they've had together?

It looks like partnerships need a game or two to settle: below par in their first game together, about par from then on. But look at who is in each row. Every row holds only the partnerships still going. The first row has all 1,859; the 4th-to-6th row is down to 748, having lost every pair that was broken up early, and managers break up the ones that aren't working.

The test is to go back to the beginning and split the partnerships by how long they went on to last:

The partnerships that lasted were already better in their first three games: +0.04 for those that went on to last 4 to 9 games and +0.10 for 10 to 19, against −0.16 for the ones broken up within three. (The 64 that reached 20 are too few to say much, ± 0.18.) Some short partnerships are stand-ins covering an injury, a suspension or a rest, and a side missing regulars is weaker than its season average.

Grouping by how long they lasted uses what happened next, so it can't be used to judge partnerships fairly; it's here only to show who drops out. The fair test of gelling is to follow the same partnerships all the way through. 130 lasted at least 15 games:

No sign of gelling. The partnerships that lasted were a little above par from their first game and stayed there. The low first row came from the poor ones being in it, and the later rows look better because they've left, not because anyone got better.

That's survivorship bias, and cohort analysis is where it most often hides. A business chart of "average spend per active customer" rises month after month even if nobody spends a penny more, because the small spenders cancel first. The safe habit is the one used here: keep the cohort fixed, follow the same members, and ask who left before you believe a curve.

Why it matters

Back to the manager's question. Neither the mix of a midfield nor the time a partnership has had together stands out once the comparison is fair. That doesn't mean combinations don't matter. It means the differences are smaller than the noise in one season, and much smaller than the raw numbers suggest.

To see them you'd need what basketball has: years of line-ups, with players moving between many different combinations so each one's effect can be pulled apart from his team-mates'. That's adjusted plus-minus, and even there the estimates for single line-ups are notoriously noisy.

The traps travel well beyond football. A segment looks good because of who's in it (selection bias), a label carries the thing you meant to measure (tackles and possession), and a curve rises because the weak drop out (survivorship). A close relative, collider bias, creates a link when the filter depends on both of the things you're comparing. The same "who left?" question sits behind how long a manager lasts, and a board judging its manager by a run of results faces it in bad run or bad manager?

Limitations

• One season. 2015/16 is the only season StatsBomb has released in full for these leagues, so every margin is wide. With five or ten seasons the small effects here might become clear, either way.
• Partnerships older than the data. A pair's "first game together" is its first in this season. Some had played together for years, which blurs the early games, but it can't explain why the pairs that lasted were better from the start.
• Types come from the same season. Each player's type is worked out from the matches being judged, and the cut-offs between types are a choice (clearly more of one than the other, 0.75 standard deviations). StatsBomb's positions are slots in a formation, so a "central midfielder" is where he started, not what he did.
• Like for like is approximate. It allows for each side's season strength, not who was missing on the day, so a weakened side counts as below par. That's part of what the short partnerships show.
• The matches aren't independent. One match gives up to three partnerships, and one side's matches share a lot. The luck margins come from resampling whole teams, which allows for that.
• xG, not points. xG difference is steadier than results, but it isn't the result. Points per game for each midfield are in the snippet's output and follow the same pattern.

Reproduce the analysis

You'll need StatsBomb's open data on your computer, about 15 GB unpacked (the download is compressed). This fetches only the events and the match lists:

mkdir -p ~/statsbomb-open-data && curl -sL https://codeload.github.com/statsbomb/open-data/tar.gz/refs/heads/master \
  | tar xz -C ~/statsbomb-open-data --strip-components=1 --wildcards 'open-data-master/data/competitions.json' \
    'open-data-master/data/matches/*' 'open-data-master/data/events/*'

Then this reads the four leagues once and prints every number in the piece. It needs only Python 3 and takes about a minute:

# From Football Data Science by Bryan McGuire. Free to use with credit.
# https://www.footballdatascience.co.uk/learn/midfield-partnerships-cohort-analysis
import json
import os
import random
from collections import Counter, defaultdict
from statistics import mean, pstdev

DATA = os.path.expanduser("~/statsbomb-open-data/data")
LEAGUES = ("Premier League", "La Liga", "Serie A", "Ligue 1")
MIDFIELD = ("Defensive Midfield", "Center Midfield", "Attacking Midfield")  # central slots, not wide ones
rnd = random.Random(2026)  # fixed, so the luck margins repeat exactly

# 1. every 2015/16 match in the four leagues: who started where, each side's xG, the points
matches = []
player = defaultdict(Counter)  # per player: minutes, tackles + interceptions, chances made, and possession
for c in json.load(open(f"{DATA}/competitions.json")):
    if c["competition_name"] not in LEAGUES or c["season_name"] != "2015/2016" or c["competition_gender"] != "male":
        continue
    for m in json.load(open(f"{DATA}/matches/{c['competition_id']}/{c['season_id']}.json")):
        events = json.load(open(f"{DATA}/events/{m['match_id']}.json"))
        sides = {m["home_team"]["home_team_id"]: m["home_score"], m["away_team"]["away_team_id"]: m["away_score"]}
        xg, xi, passes = Counter(), {}, Counter()
        end = max(e["minute"] for e in events if e["period"] <= 2) + 1
        on, off = {}, {}
        for e in events:
            kind, team = e["type"]["name"], e["team"]["id"]
            if kind == "Starting XI":
                xi[team] = [(p["player"]["id"], p["position"]["name"]) for p in e["tactics"]["lineup"]]
                on.update({p["player"]["id"]: (0, team) for p in e["tactics"]["lineup"]})
            elif kind == "Substitution":
                off[e["player"]["id"]] = e["minute"]
                on[e["substitution"]["replacement"]["id"]] = (e["minute"], team)
            elif kind in ("Bad Behaviour", "Foul Committed"):
                card = e.get(kind.lower().replace(" ", "_"), {}).get("card", {}).get("name", "")
                if card in ("Red Card", "Second Yellow"):
                    off[e["player"]["id"]] = e["minute"]
            elif kind == "Shot":
                xg[team] += e["shot"]["statsbomb_xg"]
            if "player" not in e:
                continue
            p = player[e["player"]["id"]]
            if kind == "Interception" or (kind == "Duel" and e["duel"]["type"]["name"] == "Tackle"):
                p["defend"] += 1
            elif kind == "Pass":
                passes[team] += 1
                ps = e["pass"]
                into_box = ps["end_location"][0] >= 102 and 18 <= ps["end_location"][1] <= 62 and e["location"][0] < 102
                p["create"] += bool(ps.get("shot_assist") or ps.get("goal_assist") or (into_box and "outcome" not in ps))
        for pid, (start, team) in on.items():  # minutes played, and his side's share of the passes in those matches
            mins = max(0, off.get(pid, end) - start)
            share = passes[team] / sum(passes.values())
            player[pid].update({"minutes": mins, "ball": mins * share})
        (a, ga), (b, gb) = sides.items()
        for us, them, gf, gc in ((a, b, ga, gb), (b, a, gb, ga)):
            matches.append({"league": c["competition_name"], "date": m["match_date"], "team": us, "opp": them,
                            "home": us == m["home_team"]["home_team_id"], "xgd": xg[us] - xg[them],
                            "points": 3 if gf > gc else 1 if gf == gc else 0,
                            "mid": [pid for pid, pos in xi[us] if any(s in pos for s in MIDFIELD)]})
print(f"{len(matches) // 2} matches, {len(matches)} team-matches")

# 2. midfielders by type: 10+ central-midfield starts and 900+ minutes; actions per 90, adjusted for possession
starts = Counter(pid for t in matches for pid in t["mid"])
pool = [pid for pid in starts if starts[pid] >= 10 and player[pid]["minutes"] >= 900]

def per90(pid, key, adjust):  # defending needs the other side to have the ball, creating needs your side to have it
    p = player[pid]
    share = p["ball"] / p["minutes"]
    return p[key] * 90 / p["minutes"] * (0.5 / (1 - share if key == "defend" else share) if adjust else 1)

def z(values):
    mu, sd = mean(values), pstdev(values)
    return [(v - mu) / sd for v in values]

def types(adjust):
    d, cr = z([per90(p, "defend", adjust) for p in pool]), z([per90(p, "create", adjust) for p in pool])
    return {pid: "creative" if c - x > 0.75 else "defensive" if x - c > 0.75 else "all-rounder"
            for pid, x, c in zip(pool, d, cr)}

plain, kind = types(False), types(True)
print(f"{len(pool)} midfielders:", ", ".join(f"{n} {k}" for k, n in Counter(kind.values()).most_common()),
      f"({sum(plain[p] != kind[p] for p in pool)} change type once possession is allowed for)")

# 3. like for like: what the two sides' strength and home advantage predict for the match
total, games = Counter(), Counter()
for t in matches:
    total[t["team"]] += t["xgd"]
    games[t["team"]] += 1
for t in matches:  # each side's xG difference a game in its other matches that season
    t["gap"] = (total[t["team"]] - t["xgd"]) / (games[t["team"]] - 1) - (total[t["opp"]] + t["xgd"]) / (games[t["opp"]] - 1)
    t["side"] = 1 if t["home"] else -1
sgg, sgs, sss = (sum(t[a] * t[b] for t in matches) for a, b in (("gap", "gap"), ("gap", "side"), ("side", "side")))
sgy, ssy = sum(t["gap"] * t["xgd"] for t in matches), sum(t["side"] * t["xgd"] for t in matches)
w_gap = (sgy * sss - ssy * sgs) / (sgg * sss - sgs ** 2)
w_home = (ssy - w_gap * sgs) / sss
for t in matches:
    t["vs_expected"] = t["xgd"] - w_gap * t["gap"] - w_home * t["side"]
print(f"expected xG difference = {w_gap:.2f} x strength gap {w_home:+.2f} at home ({-w_home:+.2f} away)")

def margin(rows, stat):
    """Luck margin (about 2 standard errors) on stat(rows), resampling whole teams: a team's matches aren't independent."""
    by_team = defaultdict(list)
    for r in rows:
        by_team[r["team"]].append(r)
    teams = list(by_team)
    return 2 * pstdev(stat([r for tm in rnd.choices(teams, k=len(teams)) for r in by_team[tm]]) for _ in range(500))

def avg(key):
    return lambda rows: mean(r[key] for r in rows)

# 4. three-man midfields with every player typed: raw, then like for like
for kind_of, title in ((plain, "Types from raw counts"), (kind, "Types allowing for possession")):
    trios = [t for t in matches if len(t["mid"]) == 3 and all(p in kind_of for p in t["mid"])]
    for t in trios:
        t["holding"] = sum(kind_of[p] == "defensive" for p in t["mid"])
        t["shape"] = ", ".join(sorted(kind_of[p] for p in t["mid"]))
    base = mean(t["vs_expected"] for t in trios)  # typed trios belong to slightly better line-ups than average
    print(f"\n{title}: {len(trios)} three-man midfields\nmidfield                             matches  raw xGD  points  like for like")
    shapes = Counter(t["shape"] for t in trios)
    groups = [(s, [t for t in trios if t["shape"] == s]) for s, n in shapes.most_common() if n >= 80]
    groups += [(f"{k} holding", [t for t in trios if t["holding"] == k]) for k in (0, 1, 2)]
    for name, g in groups:
        print(f"{name:36} {len(g):5} {mean(t['xgd'] for t in g):+8.2f} {mean(t['points'] for t in g):7.2f} "
              f"{mean(t['vs_expected'] for t in g) - base:+8.2f} ± {margin(g, avg('vs_expected')):.2f}")
    for key, label in (("xgd", "raw"), ("vs_expected", "like for like")):
        gap = lambda rows: mean(r[key] for r in rows if r["holding"] == 2) - mean(r[key] for r in rows if r["holding"] == 0)
        print(f"two holding midfielders against none, {label}: {gap(trios):+.2f} ± {margin(trios, gap):.2f} xG a match")

# 5. partnerships: every pair of central midfielders who started together, followed game by game
matches.sort(key=lambda t: t["date"])
together, rows = Counter(), []
for t in matches:
    for i, a in enumerate(t["mid"]):
        for b in t["mid"][i + 1:]:
            pair = (t["team"], min(a, b), max(a, b))
            together[pair] += 1
            rows.append({"pair": pair, "n": together[pair], "team": t["team"], "date": t["date"], "vs_expected": t["vs_expected"]})
lasted = dict(together)
print(f"\n{len(lasted)} partnerships, {len(rows)} pair-matches")

# the cohort table: partnerships grouped by when they first started together, still going after n games
first = {r["pair"]: r["date"][:7] for r in reversed(rows) if r["n"] == 1}
cohorts = (("Aug-Oct", ("2015-08", "2015-09", "2015-10")), ("Nov-Jan", ("2015-11", "2015-12", "2016-01")),
           ("Feb-May", ("2016-02", "2016-03", "2016-04", "2016-05")))
print("first together   pairs   still together after 3, 10, 20 games")
for name, months in cohorts:
    pairs = [p for p in lasted if first[p] in months]
    print(f"{name:14} {len(pairs):7}   " + "  ".join(f"{mean(lasted[p] >= k for p in pairs):4.0%}" for k in (3, 10, 20)))

def show(label, keep):
    g = [r for r in rows if keep(r)]
    print(f"  {label:28} {mean(r['vs_expected'] for r in g):+.2f} ± {margin(g, avg('vs_expected')):.2f}"
          f"  ({len({r['pair'] for r in g})} partnerships, {len(g)} pair-matches)")

print("like-for-like xG difference, every partnership still going, by games together:")
for a, b in ((1, 1), (2, 3), (4, 6), (7, 10), (11, 15), (16, 99)):
    show(f"games {a}-{b}" if a != b else f"game {a}", lambda r: a <= r["n"] <= b)
print("their first three games together, by how long the partnership went on to last:")
for a, b in ((1, 3), (4, 9), (10, 19), (20, 99)):
    show(f"lasted {a}-{b} games" if b < 99 else f"lasted {a}+ games", lambda r: r["n"] <= 3 and a <= lasted[r["pair"]] <= b)
same = [p for p in lasted if lasted[p] >= 15]
print(f"the same {len(same)} partnerships, every one lasting 15+ games:")
for a, b in ((1, 5), (6, 10), (11, 15)):
    show(f"games {a}-{b}", lambda r: lasted[r["pair"]] >= 15 and a <= r["n"] <= b)

Further reading

• Cohort analysis, Wikipedia. Cohorts as business analytics uses them: customers grouped by when they started, followed over time.
• Survivorship bias, Wikipedia. The error of judging from the ones that made it, including Abraham Wald's bombers in the Second World War.
• Plus–minus, Wikipedia. How basketball and ice hockey credit players with the score while they're on, and the adjusted version that separates a player from his team-mates.
• StatsBomb open data, on GitHub (now under Hudl, which owns StatsBomb): the free event data used here, with documentation of every field, including starting line-ups and positions.