# Are TV games duller? Hypothesis tests and p-values

Source: https://www.footballdatascience.co.uk/learn/hypothesis-tests-tv-games
Published: 2026-10-27

> Premiership games moved from Saturday 3pm, mostly for television, have more goals, and a shuffle test says the gap is no fluke. How a hypothesis test works, what a p-value does and doesn't tell you, and why this one still isn't about the cameras.

**On the terraces:** Everybody knows the live game on the telly is always a stinker. The goals say the opposite, and a test says that's no fluke. This piece shows why it still has nothing to do with the cameras.

## The football question

Ask anyone who watches a lot of Scottish football on television and they'll tell you: the live game is always the dull one. Pick the 12:00 kick-off on a Sunday and you've picked the 0–0.

In the UK no live football is shown at 3pm on a Saturday, so the games on television are the ones moved to another slot: Saturday lunchtime, Friday night, Sunday. The results files record every kick-off time from 2019/20, so the belief can be tested.

**Do games moved from Saturday 3pm have fewer goals?**

Across 1,031 Premiership games played Friday to Sunday since 2019/20 (2020/21 left out: no crowds, and every game was streamed), the answer is the opposite:

| Kick-off | Games | Goals a game |
|---|---|---|
| Saturday 3pm | 634 | 2.63 |
| Moved | 397 | 3.04 |

The moved games had **0.42 more goals a game**, about one extra goal every two and a half games. Could that be luck? That's the question a **hypothesis test** answers.

## The concept

A hypothesis test starts by assuming the dull answer: kick-off time makes no difference at all. That assumption is the **null hypothesis**. Then it asks how surprising the real gap would be if the null hypothesis were true.

The plainest way to find out is a **shuffle test**, also called a permutation test. If kick-off time makes no difference, the labels "moved" and "Saturday 3pm" are just stickers on the games. So:

1. Keep every game's goals where they are.
2. Peel the 397 "moved" stickers off and stick them back on 397 games chosen at random.
3. Work out the gap again. It's a gap made by luck alone.
4. Do it 20,000 times.

The pile of 20,000 shuffled gaps shows everything luck can do with these games. The **p-value** is the share of shuffles that produced a gap at least as big as the real one, in either direction:

$$\begin{aligned} p &= \frac{\text{shuffles with a gap as big}}{\text{number of shuffles}} \\ &= \frac{2}{20{,}000} \\ &= 0.0001 \end{aligned}$$

<div class="plain" markdown="1">
In plain football

- Only 2 of the 20,000 shuffles produced a gap of 0.42 goals or more, either way.
- If kick-off time made no difference, a gap this big would turn up about once in 10,000 tries.
- By convention, a p-value under 0.05 (one in 20) is called **statistically significant**: too big a gap to put down to luck with any comfort.
</div>

The shuffled gaps spread by ±0.21 goals either side of zero, 95 times in 100. That's exactly the [luck margin](/learn/standard-error-luck-margin) from the formula in Statistics #17: 0.42 ± 0.21. The shuffle and the formula agree, as they should. The shuffle needs no formula, which is why it's a good way to see what a test is doing.

So the gap is real. Television games in the Premiership do have more goals. But that's not the end of it.

## What a p-value isn't

A p-value of 0.0001 is easy to misread. It does **not** mean:

- **"There's a 0.01% chance the gap is luck."** It's the chance of a gap this big *if* there were no difference. It says nothing directly about how likely the null hypothesis is; for that you need [Bayes' theorem](/learn/bayes-theorem) and a view beforehand.
- **"The gap is big."** With enough games, a tiny gap gets a tiny p-value. The size of the gap, 0.42 goals, is a separate question from whether it's luck.
- **"Moving the game causes the goals."** The test only says the two groups differ by more than luck. It doesn't say why.

That last one is the trap here.

## The catch: who plays

Television companies don't pick games at random. They pick the ones people want to watch, and in Scotland that mostly means Celtic and Rangers.

| | Moved | Saturday 3pm |
|---|---|---|
| Celtic or Rangers playing | 57% | 14% |

And games with Celtic or Rangers in them have more goals wherever they're played: **3.28 a game**, against **2.57** in the rest. Mismatches bring goals.

So the moved slots are packed with the high-scoring games for a reason that has nothing to do with the time on the clock. The Old Firm is a [confounder](/research/correlation-causation): it pushes a game towards a moved slot *and* towards more goals. In the language of [causal diagrams](/learn/causal-diagrams-home-advantage), it's a fork, and the fix is to hold it fixed.

## Like with like

Run the same shuffle test twice: once on games with Celtic or Rangers in them, once on the rest.

| | Goals a game, moved v 3pm | p-value |
|---|---|---|
| Old Firm (316) | 3.30 v 3.22 | 0.71 |
| The rest (715) | 2.70 v 2.53 | 0.20 |

Within each group, moved games and Saturday 3pm games score about the same. The gaps, +0.08 and +0.17, are the kind luck turns up all the time: 71% and 20% of shuffles did as well.

<figure class="rank-chart"><div role="img" aria-label="Two histograms of 20,000 shuffled gaps. For all games, the real gap of +0.42 goals sits beyond almost every shuffle. Without Celtic and Rangers, the real gap of +0.17 sits inside the pile.">

</div><figcaption>Each pile is 20,000 shuffles: the gap in goals a game that luck alone gives these games. Gold lines: the real gap, and the same size the other way. Top: all games, real gap outside the pile. Bottom: without Celtic and Rangers, real gap inside it.</figcaption></figure>

Same games, same test, opposite answer. The first test was right that the groups differ. It just couldn't say why.

## Not significant isn't the same as no effect

Without Celtic and Rangers, the gap is **+0.17 ± 0.27**. That margin runs from about 0.10 *fewer* goals in moved games to 0.44 *more*.

So the data doesn't show that television games are duller. It doesn't show they're livelier either. It shows that 171 moved games without the Old Firm are too few to tell a small difference from luck, the same lesson as the [square-root rule](/learn/standard-error-luck-margin). "Not significant" means "can't tell", never "proved no difference".

What it does rule out is the version on the terraces. If moved games were really the stinkers people remember, say half a goal fewer, the margin would sit well below zero. Anything duller than about 0.1 goals fewer a game is outside it.

## Test enough things and something turns up

One more trap. A p-value under 0.05 happens by luck about one time in 20. So if you test enough questions, some will come out "significant" with nothing behind them.

To see it, split the same 1,031 games in two by a coin toss, 1,000 times, and test each split as if it meant something. **49 of the 1,000 (4.9%)** came out significant at the 5% level, with nothing behind any of them but the coin.

That's why the like-for-like tests here are the two the Old Firm explanation called for, and no more. Search through Sunday 12:00, Friday night, Saturday 12:30 and every other slot until one gives a small p-value, and something would look odd by chance.

## Why it matters

Every "is this gap real?" on the site is a hypothesis test, whether it's called one or not: the [luck margin](/learn/standard-error-luck-margin), the reshuffles in [do referees favour the home side?](/research/referees-home-side), the log-rank test in [how long does a manager last?](/learn/how-long-does-a-manager-last). The same three questions apply to all of them. Could it be luck? If not, how big is it? And what else changed between the groups?

The first question is the one a p-value answers. The other two are the ones that decide what the gap means. The [midweek myth](/myth-or-maths/midweek) asks the same kind of question about midweek rounds, and finds no difference there either.

## Limitations

- **The files don't say which games were on television.** "Moved from Saturday 3pm" stands in for it. Many moved games are television picks, but some are moved because a club played in Europe on the Thursday.
- **Goals aren't the only measure of a dull game.** A 0–0 with chances at both ends can be better to watch than a 3–0 that was over at half-time. Goals are simply what the files record for every game.
- **Celtic and Rangers are a rough way to compare like with like.** Other mismatches end up on television too. Matching on the pre-match odds, as in [matching half-time leads](/learn/matching-half-time-leads), would be more careful, with fewer games in each group.
- **Six seasons is not long.** Kick-off times only go back to 2019/20, which is why the like-for-like gap has such a wide margin.
- **The test is two-sided.** It asks whether moved games differ in either direction, which is fair when the belief and the first look point opposite ways.

## Reproduce the analysis

Save the Premiership results files (SC0) from football-data.co.uk in one folder; the [download script](/data#get-the-files) fetches them under the names the snippet expects. It needs only Python 3 and takes about 15 seconds:

```python
# From Football Data Science by Bryan McGuire. Free to use with credit.
# https://www.footballdatascience.co.uk/learn/hypothesis-tests-tv-games
import csv
import random
from datetime import datetime
from math import sqrt
from statistics import mean, pvariance

rnd = random.Random(18)  # fixed, so the shuffles repeat exactly
OLD_FIRM = {"Celtic", "Rangers"}

# Friday-to-Sunday Premiership games with a kick-off time (the files have them from 2019/20 on).
# 2020/21 is left out: no crowds, and every game was streamed, so no game was "the TV game".
games = []  # (moved from Saturday 3pm?, Celtic or Rangers playing?, goals)
for y in range(2019, 2026):
    if y == 2020:
        continue
    with open(f"SC0_{y % 100:02d}{(y + 1) % 100:02d}.csv", encoding="latin-1") as f:
        for r in csv.DictReader(f):
            if not r.get("FTHG") or not r.get("Time"):
                continue
            day = datetime.strptime(r["Date"], "%d/%m/%Y").strftime("%a")
            if day not in ("Fri", "Sat", "Sun"):
                continue  # midweek rounds are another question: see the midweek myth
            moved = not (day == "Sat" and r["Time"] == "15:00")
            games.append((moved, bool({r["HomeTeam"], r["AwayTeam"]} & OLD_FIRM), int(r["FTHG"]) + int(r["FTAG"])))


def gap(labels, goals):  # goals a game in moved games minus goals a game at Saturday 3pm
    moved = [g for m, g in zip(labels, goals) if m]
    sat = [g for m, g in zip(labels, goals) if not m]
    return mean(moved) - mean(sat)


def margin(labels, goals):  # the luck margin round that gap: two standard errors (Statistics #17)
    moved = [g for m, g in zip(labels, goals) if m]
    sat = [g for m, g in zip(labels, goals) if not m]
    return 2 * sqrt(pvariance(moved) / len(moved) + pvariance(sat) / len(sat))


def shuffle_test(rows, name, shuffles=20_000):
    """Deal the 'moved' labels out again at random, keeping the goals where they are, many times.
    The p-value is the share of shuffles with a gap at least as big as the real one, either way."""
    labels, goals = [m for m, _, _ in rows], [g for *_, g in rows]
    real = gap(labels, goals)
    fake = []
    for _ in range(shuffles):
        rnd.shuffle(labels)
        fake.append(gap(labels, goals))
    p = sum(abs(x) >= abs(real) for x in fake) / shuffles
    labels = [m for m, _, _ in rows]
    moved = [g for m, _, g in rows if m]
    sat = [g for m, _, g in rows if not m]
    print(f"\n{name}: moved {len(moved)} games {mean(moved):.2f} goals a game, Saturday 3pm {len(sat)} games {mean(sat):.2f}")
    print(f"  gap {real:+.2f} ± {margin(labels, goals):.2f}; shuffled gaps spread ± {2 * sqrt(pvariance(fake)):.2f}; "
          f"p = {p:.4f} ({sum(abs(x) >= abs(real) for x in fake)} of {shuffles:,} shuffles as big)")
    bins = [sum(-0.5 + i / 20 <= x < -0.45 + i / 20 for x in fake) for i in range(20)]  # 0.05-goal bins, -0.5 to +0.5
    print(f"  shuffled gaps by 0.05-goal bin from -0.5: {bins}")
    return real, p


print(f"{len(games)} Friday-to-Sunday games, 2019/20 to 2025/26 without 2020/21")
shuffle_test(games, "All games")
moved_of = mean(of for m, of, _ in games if m)
sat_of = mean(of for m, of, _ in games if not m)
print(f"\nCeltic or Rangers playing: {moved_of:.0%} of moved games, {sat_of:.0%} of Saturday 3pm games")
for of in (True, False):
    goals = [g for _, o, g in games if o == of]
    print(f"{'with' if of else 'without'} Celtic or Rangers: {mean(goals):.2f} goals a game over {len(goals)} games")

# like for like: compare moved and Saturday 3pm games among Old Firm games, then among the rest
shuffle_test([r for r in games if r[1]], "Old Firm games only")
shuffle_test([r for r in games if not r[1]], "Without Celtic and Rangers")

# testing many things: 1,000 made-up splits of the same games, each by a coin toss
goals = [g for *_, g in games]
hits = 0
for _ in range(1000):
    labels = [rnd.random() < 0.5 for _ in goals]
    hits += abs(gap(labels, goals)) > margin(labels, goals)
print(f"\n1,000 coin-toss splits: {hits} ({hits / 10:.1f}%) look 'significant' at the 5% level")
```

## Further reading

- [Permutation test](https://en.wikipedia.org/wiki/Permutation_test), Wikipedia. The shuffle test in general, and how it compares with tests built on formulas.
- [p-value](https://en.wikipedia.org/wiki/P-value), Wikipedia. The definition, with the conventions around 0.05.
- [Misuse of p-values](https://en.wikipedia.org/wiki/Misuse_of_p-values), Wikipedia. The common misreadings, including the American Statistical Association's 2016 statement on them.
- [Multiple comparisons problem](https://en.wikipedia.org/wiki/Multiple_comparisons_problem), Wikipedia. Why testing many things turns up false alarms, and the corrections used for it.