# Better finishers, or luck? The standard error and the luck margin

Source: https://www.footballdatascience.co.uk/learn/standard-error-luck-margin
Published: 2026-10-26

> Hearts scored with 34.7% of their shots on target last season and Dundee United with 28.8%. How far luck alone can move a rate like that, how to put a margin round it, and why most of the gap in finishing between clubs is luck.

**On the terraces:** Your side scores more of its chances than anyone else, so they must be the best finishers? Over one season, luck alone can move that figure a long way. This piece shows how far, and how to tell a real gap from a lucky one.

## The football question

Last season Hearts scored with **34.7%** of their shots on target. Dundee United scored with **28.8%**. That's six points, about one goal in every 17 shots on target.

**Are Hearts the better finishers, or could luck alone open a gap like that?**

The same question sits behind every comparison of two rates on this site, from left-footed penalties to bookings for away sides. The answer is always the same tool: the **luck margin**.

## The concept

Think of a team's season as a long run of shots on target, each one going in or not. Even if the team's real finishing stayed exactly the same, a different season with the same chances would give a slightly different rate. Some shots that went in would be saved, and some saves would go in.

The **standard error** measures how far the rate typically moves between those seasons. For a rate p worked out from n tries:

$$\text{SE} = \sqrt{\frac{p\,(1 - p)}{n}}$$

<div class="plain" markdown="1">
In plain football

- \(p\) is the rate: 0.347 for Hearts, goals per shot on target.
- \(n\) is how many shots it rests on: 193 for Hearts last season.
- \(p(1 - p)\) is largest when a shot is a coin toss and smaller when nearly everything goes in or nearly nothing does, the same idea as the [Bernoulli distribution](/learn/bernoulli-distribution).
- Dividing by \(n\) under the square root is the key part: more shots, smaller standard error.
</div>

The **luck margin** is two standard errors either side. About 95 seasons in 100 land within it by chance alone. Statisticians call the rate plus or minus this margin a **95% confidence interval**.

$$\begin{aligned} \text{margin} &= 2 \times \sqrt{\frac{0.347 \times 0.653}{193}} \\ &= 2 \times 0.034 \\ &= 0.069 \end{aligned}$$

<div class="plain" markdown="1">
In plain football

- Hearts' 34.7% could easily have been anything from about 27.8% to 41.6% with the same finishing.
- At 193 shots on target, that's about 13 goals either way over the season.
</div>

## Every club last season

Here is every Premiership club's 2025/26 finishing, with its luck margin as a line. The dashed line is the league's rate, 32.1%.

<figure class="rank-chart"><div role="img" aria-label="Goals per shot on target for the twelve Premiership clubs in 2025/26, each with its luck margin. Eleven clubs' lines cross the league rate of 32.1%; St Mirren's, at 21.7%, sits clear of it.">

</div><figcaption>Goals per shot on target, 2025/26 Premiership, with each club's luck margin (two standard errors). Gold: further from the league rate than luck alone would put it.</figcaption></figure>

**Eleven of the twelve lines cross the league's rate.** From Falkirk and Hearts at 34.7% down to Dundee United at 28.8%, every one of them could be an average finishing side having a slightly lucky or unlucky season.

**St Mirren are the exception.** They scored 30 from 138 shots on target, **21.7%**, more than ten points below the league, and further than luck alone would put them. They weren't one bad spell either: 16 from 68 in the first half of the season and 14 from 70 in the second, 23.5% and 20.0%.

## Is Hearts against Dundee United a real gap?

To compare two clubs, add up both clubs' uncertainty. Each rate has its own standard error, and the gap between them carries both:

$$\begin{aligned} \text{gap margin} &= 2\sqrt{\text{SE}_\text{H}^2 + \text{SE}_\text{DU}^2} \\ &= 2\sqrt{0.034^2 + 0.035^2} \\ &= 0.098 \end{aligned}$$

<div class="plain" markdown="1">
In plain football

- \(\text{SE}_\text{H}\) and \(\text{SE}_\text{DU}\) are Hearts' and Dundee United's standard errors, from their own rates and shots.
- Squaring them, adding and taking the square root is how two separate bits of luck combine. It's less than adding the two margins, because both clubs are rarely at the far end of their luck, in opposite directions, at the same time.
- The answer, **±9.8 points**, is bigger than the 5.9-point gap.
</div>

So the gap is well inside the luck margin. Hearts may well be better finishers, but one season of shots can't show it. This is the formula behind the [left-footed penalties myth](/myth-or-maths/left-footed-penalties), where 72.7% against 74.2% was well inside its ±5.8.

## The square-root rule

The margin shrinks with the square root of the number of shots. **Four times the shots halves the margin.** At last season's league rate:

| Shots on target | Luck margin |
|---|---|
| 42 (about 10 games) | ±14.4 points |
| 161 (a season) | ±7.4 points |
| 644 (four seasons) | ±3.7 points |

That's why the finishing table after ten games tells you almost nothing: a club could sit 14 points above or below its real rate by luck alone. Even a full season leaves ±7 points, wider than most of the gaps between clubs.

## Does the formula tell the truth?

Two checks, one made up and one real.

**A simulation.** Take Hearts' 193 shots on target and give every one the league's 32.1% chance of going in. Replay that season 10,000 times. The simulated rates spread by **3.35 points** around 32.1%; the formula says **3.36**. And **94.8%** of the simulated seasons land inside the luck margin, as the formula promises.

**Real seasons split in half.** If a club's finishing really stays the same through a season, its two halves should differ only by luck. Across all 312 Premiership team-seasons since 2000/01, **291 (93.3%)** of the half-to-half gaps fall inside their luck margin, close to the 95% that luck alone predicts. Finishing barely changes within a season; the small shortfall could be injuries, transfers or a change of manager.

## How much of finishing is luck?

Clubs' finishing rates spread out by about **4.2 points** from their season's rate, across all 312 team-seasons. Luck alone, from the number of shots each club had, would spread them by **3.6 points**. Compare the two as variances, the squares, and luck accounts for about **73%** of the spread.

What's left for real differences in finishing is small: a typical club's true rate sits about **2.2 points** from its season's average. Real differences exist, and St Mirren last season look like one, but they're smaller than the table makes them look. Most of what looks like clinical finishing in one season is luck.

Each club is measured against its own season because the rates moved: about **27%** of shots on target went in a season up to 2012/13, about **32%** since, when fewer shots started to be recorded as on target (see [linear regression](/learn/linear-regression), where the same change shows up).

## Why it matters

Almost every comparison on this site carries a luck margin: [left- against right-footed penalties](/myth-or-maths/left-footed-penalties), [do substitutions work?](/research/do-substitutions-work), [do referees favour the home side?](/research/referees-home-side), the [new manager bounce](/myth-or-maths/new-manager-bounce). It's the same two standard errors each time. A gap smaller than its margin could be luck. A gap bigger than it might still be luck occasionally, about one time in 20, which is why a piece that checks many splits treats one near the edge as a hint, not a finding.

The [Beta distribution](/learn/beta-distribution) asks the same question the Bayesian way: how sure can you be of a penalty taker's real rate after eight from ten? It also lets what you believed beforehand count. The luck margin uses only the data in front of you.

## Limitations

- **The margin assumes every shot is a separate try.** A rebound straight after a save is really part of one chance, which makes the true uncertainty a little bigger than the formula says.
- **Shots on target aren't all alike.** A club that creates better chances will score more of them, so a high rate can be good chances, not good finishing. The margin says whether a gap is luck, not where it comes from.
- **Two standard errors is a convention.** It catches about 95% of what luck can do, so about one season in 20 lands outside it by chance alone. Check twelve clubs and finding one outside it isn't proof on its own; St Mirren's two low halves make their case stronger.
- **The formula is an approximation.** It works well with dozens of shots or more and a rate not close to 0 or 1. For a handful of kicks, such as a single taker's penalties, the [Beta distribution](/learn/beta-distribution) is the better tool.
- **Goals include own goals**, which aren't anyone's shot on target. That's why 75 team-matches show more goals than shots on target. It barely moves the rates.

## Reproduce the analysis

Save the Premiership results files (SC0) from football-data.co.uk in one folder; the [download script](/data#get-the-files) fetches them under the names the snippet expects. It needs only Python 3 and takes under a second:

```python
# From Football Data Science by Bryan McGuire. Free to use with credit.
# https://www.footballdatascience.co.uk/learn/standard-error-luck-margin
import csv
import random
from collections import defaultdict
from math import sqrt
from statistics import mean, pstdev, pvariance

rnd = random.Random(17)  # fixed, so the simulation repeats exactly

def margin(p, n):  # the luck margin: two standard errors of a rate p from n tries
    return 2 * sqrt(p * (1 - p) / n)

# every team's goals and shots on target, match by match, for each Premiership season
games = defaultdict(list)  # (season, team) -> [(goals, shots on target), ...] in date order
for y in range(2000, 2026):
    with open(f"SC0_{y % 100:02d}{(y + 1) % 100:02d}.csv", encoding="latin-1") as f:
        for r in csv.DictReader(f):
            if not r.get("HST") or not r.get("FTHG"):
                continue
            games[y, r["HomeTeam"]].append((int(r["FTHG"]), int(float(r["HST"]))))
            games[y, r["AwayTeam"]].append((int(r["FTAG"]), int(float(r["AST"]))))

total = {k: (sum(g for g, _ in v), sum(s for _, s in v)) for k, v in games.items()}
league = defaultdict(lambda: [0, 0])  # each season's own rate: shot recording changed over the years
for (y, _), (g, s) in total.items():
    league[y][0] += g
    league[y][1] += s
rate = {y: g / s for y, (g, s) in league.items()}
print(f"{len(total)} team-seasons from {sum(len(v) for v in games.values()) // 2:,} matches")
print(f"league rate: {rate[2000]:.1%} in 2000/01, {mean(rate[y] for y in range(2000, 2013)):.1%} a season to 2012/13, "
      f"{mean(rate[y] for y in range(2013, 2026)):.1%} since")

print("\n2025/26: goals per shot on target, with the luck margin")
p = rate[2025]
print(f"league {p:.1%}")
for (y, team), (g, s) in sorted(total.items(), key=lambda kv: -kv[1][0] / kv[1][1]):
    if y == 2025:
        r = g / s
        flag = "  outside the margin" if abs(r - p) > margin(p, s) else ""
        print(f"{team:14} {g:3}/{s:3} = {r:.1%} ± {margin(r, s):.1%}{flag}")

(gh, sh), (gd, sd) = total[2025, "Hearts"], total[2025, "Dundee United"]
a, b = gh / sh, gd / sd
gap_margin = 2 * sqrt(a * (1 - a) / sh + b * (1 - b) / sd)
print(f"\nHearts v Dundee United: gap {a - b:+.1%}, luck margin ± {gap_margin:.1%}")
g1, s1 = map(sum, zip(*games[2025, "St Mirren"][:19]))
g2, s2 = map(sum, zip(*games[2025, "St Mirren"][19:]))
print(f"St Mirren by half: {g1}/{s1} = {g1 / s1:.1%}, then {g2}/{s2} = {g2 / s2:.1%}")

print("\nthe square-root rule, at last season's rate")
for label, n in (("about 10 games", 42), ("a season", 161), ("four seasons", 644)):
    print(f"{label:15} {n:4} shots on target: ± {margin(p, n):.1%}")

# check the formula by simulation: replay Hearts' 193 shots at the league rate, 10,000 times
sims = [sum(rnd.random() < p for _ in range(sh)) / sh for _ in range(10_000)]
print(f"\nsimulated spread for {sh} shots: {pstdev(sims):.2%}; the formula says {sqrt(p * (1 - p) / sh):.2%}")
inside = sum(abs(x - p) <= margin(p, sh) for x in sims) / len(sims)
print(f"simulated seasons inside the luck margin: {inside:.1%}")

# how much of the spread between clubs is luck? compare it with what luck alone would give
gaps = [g / s - rate[y] for (y, _), (g, s) in total.items()]
luck = mean(rate[y] * (1 - rate[y]) / s for (y, _), (g, s) in total.items())
print(f"\nspread between team-seasons: {pstdev(gaps):.1%}; luck alone: {sqrt(luck):.1%}; "
      f"luck's share of the variance: {luck / pvariance(gaps):.0%}")
print(f"what's left for real differences in finishing: a typical club sits {sqrt(pvariance(gaps) - luck):.1%} from its season's rate")

# is the margin honest? split every team-season in two and compare the halves
within = 0
for k, v in games.items():
    half = len(v) // 2
    (ga, sa), (gb, sb) = (map(sum, zip(*part)) for part in (v[:half], v[half:]))
    both = (ga + gb) / (sa + sb)
    within += abs(ga / sa - gb / sb) <= 2 * sqrt(both * (1 - both) * (1 / sa + 1 / sb))
print(f"halves inside their luck margin: {within} of {len(games)} = {within / len(games):.1%} (luck alone: about 95%)")
own = sum(1 for v in games.values() for g, s in v if g > s)
print(f"team-matches with more goals than shots on target (own goals): {own}")
```

## Further reading

- [Standard error](https://en.wikipedia.org/wiki/Standard_error), Wikipedia. What it measures, and why the square root of the sample size sits in its formula.
- [Binomial proportion confidence interval](https://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval), Wikipedia. The interval used here (the normal approximation) and better ones for small samples, such as Wilson's.
- [68–95–99.7 rule](https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule), Wikipedia. Why two standard errors either side covers about 95%.