# Does kicking first win shootouts? Randomised experiments, and how many you need

Source: https://www.footballdatascience.co.uk/learn/randomised-experiments-shootouts
Published: 2026-10-26

> A coin toss decides who kicks first in a shootout, which makes it a real experiment. In StatsBomb's open data the side kicking first won 18 of 36, exactly half. Why that settles nothing, and how many shootouts it would take to tell.

**On the terraces:** Win the toss and kick first, everyone says, because the pressure lands on the other lot. In 36 shootouts the side going first won exactly half. This piece shows why 36 is nowhere near enough to know, and how many you'd need.

## The football question

Before a penalty shootout the referee tosses a coin, and the captain who wins it chooses which side kicks first. Nearly every captain chooses to go first. The idea is that the side kicking second is always catching up, so the pressure lands on them.

Researchers have reported that the side kicking first wins about six shootouts in ten, and in 2017 football's lawmakers, IFAB, took it seriously enough to trial a new order, "ABBA", like a tennis tie-break, so neither side would always be the one catching up. UEFA tried it at its under-17 championships. In November 2018 IFAB dropped the idea.

**So does kicking first really win shootouts?**

## The concept

Every earlier part of this series needed a stand-in for the counterfactual, the result that didn't happen: the Championship for the Premiership without VAR in [Part 3](/learn/difference-in-differences-var), the side in seventh for the side in sixth in [Part 4](/learn/regression-discontinuity-split). Each stand-in came with a worry: were the two groups really alike?

A **randomised experiment** removes the worry. When a coin decides who gets the treatment, here kicking first, the coin can't know which side is better, braver or has the better keeper. So the two groups are alike on everything, measured or not, apart from the treatment. Any difference in results is the treatment's doing, or luck.

That's why medicine tests new drugs with randomised trials, and why the shootout coin toss is a gift: a real experiment, run by referees, at every tournament. It's the only design in this series with no hidden-cause problem at all.

## What the data shows

StatsBomb's free open data includes **36 penalty shootouts**, from men's and women's tournaments (seven of them in the women's game). In each one, the side that kicked first is recorded.

**The side kicking first won 18 and lost 18.** Exactly half.

<div class="flex flex-wrap items-center gap-4 rounded-md bg-chalk px-4 py-3" style="margin:1.5rem 0"><img src="/static/img/statsbomb-logo.png" alt="StatsBomb" width="180" height="29" class="h-auto w-44"><p class="text-sm text-ink">Data: StatsBomb open data. The analysis is ours, not StatsBomb's.</p></div>

So the first-kicker edge is a myth? Not so fast. With 36 shootouts, the [luck margin](/learn/standard-error-luck-margin) around 50% is **±16.7 points**: anything from about 33% to 67% could happen by luck alone. A true edge of 60% sits comfortably inside that. If the side kicking first really won 60% of the time, a result of 18 or fewer from 36 would still turn up **15% of the time**, about one study in seven.

<figure class="rank-chart"><div role="img" aria-label="10,000 make-believe studies of 36 shootouts. When the truth is 50%, most studies find 16 to 23 wins for the side kicking first. When the truth is 60%, most find 20 to 27. The two sets of bars overlap heavily, and our result, 18, sits where both are common.">

</div><figcaption>10,000 make-believe studies of 36 shootouts each, when kicking first truly wins half the time (moss) and when it wins 60% (mint). The two overlap so much that one study of 36 can't tell them apart.</figcaption></figure>

The simulation makes it plain. Run 10,000 make-believe studies of 36 shootouts in a world with no edge, and 10,000 in a world where kicking first wins 60%. The two piles of results overlap heavily. Our 18 is common in both.

## How many would you need?

The chance that a study spots an effect that's really there is called its **statistical power**. It depends on how big the effect is and how much data there is:

| Shootouts | Luck margin | Spots a 60% edge | Spots a 55% edge |
|---|---|---|---|
| 36 (ours) | ±16.7 | 16% | 6% |
| 100 | ±10.0 | 46% | 14% |
| 200 | ±7.1 | 79% | 26% |
| 300 | ±5.8 | 93% | 39% |
| 500 | ±4.5 | 99% | 59% |

<div class="plain" markdown="1">
In plain football

- "Spots" means the result lands outside the luck margin around 50%, so you'd call the edge real.
- With our 36 shootouts, a real 60% edge would be spotted only about one time in six.
- To be fairly sure of spotting it, about four times in five, you need around 200 shootouts. A smaller edge, 55%, needs more than 500.
</div>

That's more shootouts than most studies have, which is why the research disagrees: some studies report a strong edge for the side kicking first, others find little or none. A few hundred shootouts can only just tell a 60% edge from no edge, and a small edge from none at all hardly ever. It was a close call for a rule change.

## Why it matters

A perfect design isn't enough on its own. **A small experiment that finds nothing hasn't shown there's nothing there.** It has shown only that the effect, if there is one, is smaller than the study could see. "No evidence of an effect" and "evidence of no effect" are different things, and mixing them up is one of the commonest mistakes in football analysis, from "the new system made no difference" after six games to "the sports scientist's plan doesn't work" after one season.

The question to ask of any "it made no difference" is: **how big a difference could this data have seen?** Here, with 36 shootouts, nothing smaller than about 17 points either way.

For the kicks themselves, left foot against right and in shootouts against open play, see [are left-footed penalty takers worse?](/myth-or-maths/left-footed-penalties) And for where to aim once you're on the spot, [the Nash equilibrium of a penalty](/learn/penalties-nash-equilibrium).

## Limitations

- **The toss isn't recorded, only who kicked first.** Nearly every captain who wins the toss chooses to go first, so who kicks first is almost as random as the toss itself, but not quite.
- **StatsBomb's open data isn't a random sample** of shootouts. It's the tournaments StatsBomb has chosen to release: World Cups, European Championships and others, men's and women's.
- **Men's and women's shootouts are counted together**, because seven women's shootouts are far too few to look at on their own.
- **The published studies aren't quoted here.** The 60% figure is the one IFAB's proposal was reported to rest on; this piece uses it only as the edge to test against, not as a finding.

## Reproduce the analysis

You'll need StatsBomb's open data on your computer, about 15 GB unpacked. This fetches only the events and the match lists:

```bash
mkdir -p ~/statsbomb-open-data && curl -sL https://codeload.github.com/statsbomb/open-data/tar.gz/refs/heads/master \
  | tar xz -C ~/statsbomb-open-data --strip-components=1 --wildcards 'open-data-master/data/competitions.json' \
    'open-data-master/data/matches/*' 'open-data-master/data/events/*'
```

Then this finds every shootout and prints every number in the piece. It takes about half a minute, because it skips matches without a shootout before reading them:

```python
# From Football Data Science by Bryan McGuire. Free to use with credit.
# https://www.footballdatascience.co.uk/learn/randomised-experiments-shootouts
import glob
import json
import os
import random
from collections import Counter
from math import comb, sqrt

DATA = os.path.expanduser("~/statsbomb-open-data/data")
rnd = random.Random(5)  # fixed, so the simulation repeats exactly

with open(f"{DATA}/competitions.json", encoding="utf-8") as f:  # men's or women's, for each competition and season
    gender = {(c["competition_id"], c["season_id"]): c["competition_gender"] for c in json.load(f)}
women = set()
for path in glob.glob(f"{DATA}/matches/*/*.json"):
    with open(path, encoding="utf-8") as f:
        women |= {m["match_id"] for m in json.load(f)
                  if gender.get((m["competition"]["competition_id"], m["season"]["season_id"])) == "female"}

# every shootout: which side kicked first, and whether it won (period 5 is the shootout)
shootouts = []
for path in glob.glob(f"{DATA}/events/*.json"):
    with open(path, encoding="utf-8") as f:
        raw = f.read()
    if '"period" : 5' not in raw:  # skip the matches without one before reading them properly
        continue
    kicks = [e for e in json.loads(raw) if e["period"] == 5 and e["type"]["name"] == "Shot"]
    scored = Counter()
    for e in kicks:
        scored[e["team"]["name"]] += e["shot"]["outcome"]["name"] == "Goal"
    first = kicks[0]["team"]["name"]
    second = next(t for t in scored if t != first)
    shootouts.append({"first_won": scored[first] > scored[second], "women": int(os.path.basename(path)[:-5]) in women})

n, won = len(shootouts), sum(s["first_won"] for s in shootouts)
print(f"{n} shootouts ({sum(s['women'] for s in shootouts)} in the women's game); the side kicking first won {won} ({won / n:.0%})")

def margin(n):  # the luck margin around 50%: two standard errors
    return 2 * sqrt(0.25 / n)

def power(n, true):  # the chance a study of n shootouts lands outside 50% plus or minus its luck margin
    return sum(comb(n, k) * true ** k * (1 - true) ** (n - k) for k in range(n + 1) if abs(k / n - 0.5) > margin(n))

print(f"luck margin with {n}: ± {margin(n):.1%}")
at_most = sum(comb(n, k) * 0.6 ** k * 0.4 ** (n - k) for k in range(won + 1))
print(f"if the first side really won 60%, {won} or fewer wins from {n} would happen {at_most:.0%} of the time")

print("\nshootouts  luck margin  chance of spotting a 60% edge  a 55% edge")
for size in (36, 100, 200, 300, 500):
    print(f"{size:9}  ± {margin(size):5.1%}       {power(size, 0.6):4.0%}                     {power(size, 0.55):4.0%}")

# 10,000 make-believe studies of 36 shootouts: what they show when the truth is 50%, and when it's 60%
print("\nshare of 10,000 studies of 36 shootouts, by the first side's win rate they found")
for true in (0.5, 0.6):
    found = Counter(min(sum(rnd.random() < true for _ in range(n)) // 4, 8) for _ in range(10_000))
    print(f"truth {true:.0%}: " + ", ".join(f"{4 * b}-{4 * b + 3 if b < 8 else n} wins {found[b] / 100:.0f}%" for b in range(9)))
```

## Further reading

- [Penalty shoot-out](https://en.wikipedia.org/wiki/Penalty_shoot-out_(association_football)), Wikipedia. The rules, the ABBA trial, and the research on whether kicking first helps, which the page itself calls ambiguous.
- [Randomized controlled trial](https://en.wikipedia.org/wiki/Randomized_controlled_trial), Wikipedia. Why randomly deciding who gets a treatment makes the two groups alike.
- [Power of a test](https://en.wikipedia.org/wiki/Power_of_a_test), Wikipedia. Statistical power, and how it depends on the size of the effect and the amount of data.
- [StatsBomb open data](https://github.com/statsbomb/open-data), on GitHub (now under Hudl, which owns StatsBomb): the free event data used here.