Why do teams give the ball back? Repeated games and the shadow of the future

The football question

A player goes down injured. His team kicks the ball out so he can be treated, and when play restarts, the other side throws it straight back. Nobody has to. A team that kept the ball would be ahead, at least for that moment. So why does almost everyone give it back?

Part 3 showed how a deal that's good for both sides can fall apart: in the prisoner's dilemma, each side does better by breaking it, whatever the other does. Giving the ball back looks like that kind of deal. The difference is that football teams don't meet once. They meet again and again, and that changes everything.

The concept

A repeated game is the same game played over and over by the same players. What a player does today can be answered tomorrow, so each choice has a cost later on as well as a payoff now.

That future is often called the shadow of the future. If two sides are likely to meet again, breaking a deal today buys a one-off gain and a run of punishment after it. If the shadow is long enough, keeping the deal pays, even for a player who cares only about himself.

A football example

Take Part 3's wage race again: two rival clubs, each holding its wage bill or raising it. Here's each club's profit a season in £m (made-up numbers, as before), Club A's first:

Played once, raising is best whatever the other club does, so both raise and make £2m each, when holding would have given both £5m.

Now suppose the two clubs face the same choice every season, and each season there's a chance p that they'll both still be in the league to play it again next season. Club B follows a simple rule: hold, unless A has ever raised, and then raise for ever. Should A hold?

Holding every season earns 5 a season for as long as the game goes on. Raising once earns 8 this season, then 2 a season after that, because B never trusts it again. Adding up what each is worth over the seasons to come, A should keep holding when

$$\begin{aligned} \frac{5}{1 - p} &\ge 8 + \frac{2p}{1 - p} \\ p &\ge \frac{8 - 5}{8 - 2} = 0.5 \end{aligned}$$

Two clubs in the same league are very likely to meet again, so the shadow of the future is long. The same arithmetic says why a club facing relegation, or a manager about to leave, is more tempted to break unwritten rules: for them, p is small.

Tit-for-tat

Around 1980, the political scientist Robert Axelrod invited experts to send in strategies for a repeated prisoner's dilemma and played them all against each other in a computer tournament. The winner was the simplest entry, sent in by the mathematical psychologist Anatol Rapoport: tit-for-tat. It cooperates first, then copies whatever the opponent did last time. Axelrod ran a second tournament, with everyone knowing the result of the first, and tit-for-tat won again.

Here's a small tournament of the same kind on the wage race. Six strategies each play every strategy, themselves included, over ten seasons:

The striking thing is how tit-for-tat wins. It never beats a single opponent head to head. Against always-raise it loses 19 to 26, because it holds once in the first season and gets taken. Against always-hold it draws 50 to 50. It wins the tournament because it does well with everyone: it rewards a club that holds, punishes one that raises, and forgives as soon as the other side comes back. Axelrod summed up the winning strategies as nice (never the first to break a deal), provocable (hit back), forgiving and not envious.

Always-raise wins every head-to-head it can, 80 to 10 against always-hold, and still finishes near the bottom, because nobody will cooperate with it.

Giving the ball back

Kicking the ball out for an injured player, and getting it back, is a deal of this kind. Keeping the ball is the temptation. Everyone giving it back is the deal both sides prefer. And football teams, players and managers meet again and again, with everyone watching. A team that broke the custom would find it broken against them next time, and would be booed, written about and remembered.

The Laws of the Game now handle part of this: when the referee stops play for an injury with the ball in play, the restart is a dropped ball for the team that last touched it (or for the defending keeper, inside his penalty area). But when a team kicks the ball out itself, giving it back is still only a custom.

The best-known test of it came on 13 February 1999, in an FA Cup fifth-round tie between Arsenal and Sheffield United at 1–1. Sheffield United's keeper Alan Kelly kicked the ball out so a teammate could be treated. Ray Parlour threw it back towards him, but Nwankwo Kanu, ten minutes into his Arsenal debut and apparently unaware of the custom, ran onto it and crossed for Marc Overmars to score. Arsenal won 2–1, and Arsène Wenger at once offered to replay the match. The FA agreed, and ten days later Arsenal won the replay 2–1. Game theory explains why the custom holds. The replay shows how much a club valued being seen to keep it.

Repeated games with no deal to keep

Not every repeated game has a deal in it. A penalty is pure competition, as Part 1 showed: every goal the taker gains, the keeper loses. Playing it again and again doesn't make room for cooperation. It adds just one lesson: leave no pattern. A taker whose next kick can be predicted from his last few will be read.

Professionals seem to manage it. Palacios-Huerta, whose numbers Part 1 used, found that a player's next choice couldn't be predicted from his previous ones. Keepers may be weaker at it. A 2014 study of 361 kicks in World Cup and European Championship shootouts from 1976 to 2012, by Erman Misirlisoy and Patrick Haggard, reported that after a run of kicks to one side, keepers became more likely to dive the other way, the gambler's fallacy: believing a run must end. Takers didn't seem to exploit it. A re-analysis in 2019 by Simcha Avugos and colleagues found the pattern after two or three kicks in a row to the same side, but whether it counts as real depends on how wide "the middle" of the goal is drawn. After three in a row, keepers went the other way in 71% to 83% of cases, depending on that choice, but from fewer than 20 such cases. Treat it as suggestive, not settled.

Why it matters

Many of football's unwritten rules are deals like this, and giving the ball back is the best known. Repeated games explain why they can survive without referees, and when they break: when a side doesn't expect to meet the other again, or the stakes of one match are too big. In the pure contests, like a penalty, repetition teaches the opposite lesson: never be predictable. Part 6 turns to the people who set the rules, and how changing them changes what players do.

Limitations

• The wage race is made up, and real clubs don't follow one strategy for ever.
• Tournament results depend on who's in it. Tit-for-tat won Axelrod's two tournaments, and it tops this small one, but strategies built to beat it can do better in other line-ups. It's a strong strategy, not the best in every company.
• The 0.5 threshold is for these numbers and this punishment. A softer punishment such as tit-for-tat needs a longer shadow of the future to keep the deal.
• Customs aren't only about payoffs. Players give the ball back partly because it's the decent thing to do. Game theory explains why the custom is stable, not why each player keeps it.
• The shootout evidence is mixed, as described above.

Try it yourself

Add a strategy to the code below, say "tit-for-two-tats", which only raises after the other side has raised twice in a row. Does it beat tit-for-tat in this tournament? Then add always-raise twice more and see whether tit-for-tat still wins.

Reproduce the analysis

This works out the threshold, runs the tournament and the head-to-head matches, and prints every number in the article. Nothing to download.

# Part 3's wage race (made up): each club holds its wages or raises them, profit in £m a season.
PAY = {("hold", "hold"): (5, 5), ("hold", "raise"): (1, 8), ("raise", "hold"): (8, 1), ("raise", "raise"): (2, 2)}
T, R, P, S = 8, 5, 2, 1   # temptation to raise, reward for both holding, both raising, holding while the other raises

# Strategies see the other club's past moves and choose this season's.
strategies = {
    "Always hold": lambda theirs: "hold",
    "Always raise": lambda theirs: "raise",
    "Tit-for-tat": lambda theirs: theirs[-1] if theirs else "hold",       # hold first, then copy them
    "Grudger": lambda theirs: "raise" if "raise" in theirs else "hold",    # hold until they raise once, then never forgive
    "Suspicious tit-for-tat": lambda theirs: theirs[-1] if theirs else "raise",
    "Alternator": lambda theirs: "hold" if len(theirs) % 2 == 0 else "raise",
}


def play(a, b, seasons=10):
    """Two strategies play the wage race for a number of seasons; returns each one's total profit."""
    seen_a, seen_b, total_a, total_b = [], [], 0, 0   # seen_a: B's past moves, as A sees them
    for _ in range(seasons):
        ma, mb = strategies[a](seen_a), strategies[b](seen_b)
        pa, pb = PAY[ma, mb]
        total_a, total_b = total_a + pa, total_b + pb
        seen_a.append(mb)
        seen_b.append(ma)
    return total_a, total_b


# Once: whatever the other does, raising pays more, so both raise (Part 3).
print(f"Once: both raise, {P} each; both holding would give {R} each")
# Again and again: holding pays if the chance of meeting again next season, p, is high enough that
# R every season beats T once and then P every season after: R/(1-p) >= T + p*P/(1-p).
print(f"Holding is worth it against a grudger if p is at least (T - R) / (T - P) = {(T - R) / (T - P):.2f}")
for p in [0.4, 0.5, 0.6]:
    keep, cheat = R / (1 - p), T + p * P / (1 - p)
    print(f"  p = {p}: keep holding {keep:5.2f}, raise once {cheat:5.2f}")

# A round-robin tournament: every strategy plays every strategy, itself included, for 10 seasons.
names = list(strategies)
score = {n: 0 for n in names}
for i, a in enumerate(names):
    for b in names[i:]:
        pa, pb = play(a, b)
        score[a] += pa
        if a != b:
            score[b] += pb
print()
print("Tournament, average profit a season (£m):")
for n in sorted(names, key=lambda n: -score[n]):
    print(f"  {n:<24}{score[n] / (10 * len(names)):.2f}")
print()
print("Head to head, 10 seasons:")
for a, b in [("Tit-for-tat", "Always raise"), ("Tit-for-tat", "Always hold"), ("Always raise", "Always hold")]:
    print(f"  {a} {play(a, b)[0]}, {b} {play(a, b)[1]}")
wins = [b for b in names if b != "Tit-for-tat" and play("Tit-for-tat", b)[0] > play("Tit-for-tat", b)[1]]
print(f"Opponents tit-for-tat outscores head to head: {len(wins)} of {len(names) - 1}")

It prints:

Once: both raise, 2 each; both holding would give 5 each
Holding is worth it against a grudger if p is at least (T - R) / (T - P) = 0.50
  p = 0.4: keep holding  8.33, raise once  9.33
  p = 0.5: keep holding 10.00, raise once 10.00
  p = 0.6: keep holding 12.50, raise once 11.00

Tournament, average profit a season (£m):
  Tit-for-tat             4.27
  Grudger                 4.00
  Always hold             3.93
  Alternator              3.90
  Always raise            3.70
  Suspicious tit-for-tat  3.47

Head to head, 10 seasons:
  Tit-for-tat 19, Always raise 26
  Tit-for-tat 50, Always hold 50
  Always raise 80, Always hold 10
Opponents tit-for-tat outscores head to head: 0 of 5

Each strategy is a one-line rule that looks only at the other club's past moves, so adding your own takes one line.

Further reading

• The Evolution of Cooperation, Wikipedia. Axelrod's tournaments, tit-for-tat and the properties of winning strategies.
• Tit for tat, Wikipedia. The strategy, where it comes from, and its weaknesses.
• Goal center width, how to count sequences, and the gambler's fallacy in soccer penalty shootouts, Avugos, Azar, Gavish, Sher and Bar-Eli, Judgment and Decision Making (2019). The re-analysis of the shootout study, free to read.
• Repeated game, Wikipedia. The general theory, including the "folk theorem" on which deals can last.