Folarin Balogun and the consistency vs common sense trade-off

Do you prefer the right outcome with the wrong process, or the wrong outcome with the right process?

One of the big VAR controversies of the current World Cup has involved Folarin Balogun. He was sent off in the USA’s last-32 match against Bosnia and Herzegovina, meaning he should have been suspended for their last-16 match against Belgium.

The red card was widely seen as being a harsh decision, but no appeal is possible against straight red cards, so it looked like that was that. However, after a request from Donald Trump, FIFA invoked Article 27 of their rules that allows them to suspend a disciplinary measure, and Balogun was able to play against Belgium.

It is a highly unusual intervention, and one that has prompted a great deal of controversy. Even though the USA lost their match against Belgium (with Balogun in the team), the precedent it has set is likely to outlast their participation in the tournament. Already, there are suggestions that France and England are attempting to get FIFA to overturn sanctions on their players.

This entire episode illustrates many of the themes I wrote about in my 2024 book, I Can’t Stop Thinking About VAR. At its core is a conflict between two different ways of making decisions: the letter of the law and the spirit of the law.

1. The letter of the law says that we should prioritise a consistent set of rules and a clear process. We should stick to that process even if sometimes it delivers odd outcomes, because consistency and clarity are important end goals in and of themselves.

2. The spirit of the law says we should prioritise getting to the right decision. If our rules and processes deliver the wrong decision, what is the point of them? Sometimes you need some common sense to fix obviously absurd outcomes.

This tension is present in modern football, but it’s present throughout history. It’s a major theme of the New Testament. It’s constantly cropping up in Shakespeare’s plays. You can see it today in political debates about the balance of power between the executive and the judiciary. It’s also a significant debate in the development of artificial intelligence systems. And it matters a lot for how we assess students, which is our particular specialism at No More Marking.

The right process and the wrong outcome

But to return to football, over the last few years, VAR has prioritised the letter of the law, resulting in many decisions which seem lacking in common sense. The original decision to send off Balogun was a good example. Balogun was not sent off by the referee in real-time – in fact, the referee did not even award a free kick to begin with. It was only after he saw a slow-motion replay that he changed his mind.

But we know that slow-motion video replays change the way we perceive intent and impact. I wrote about this in chapter 2 of my book: it is well-documented in the academic literature, and it’s been a big problem with VAR from the early days. It’s such a big problem that referees are actually told that they cannot use a slow-motion replay to determine intensity or intent, only to determine facts like whether there is impact or who touches a ball first. Of course, it is exceptionally hard, maybe impossible, to ask a human to view a slow-motion replay but not let it affect their perception of intensity.

The Balogun incident also exemplifies another problem with VAR: whether the red card is technically right or wrong, it is highly unlikely to be awarded in a pre-VAR world. VAR is therefore not just implementing the rules: it is changing the way they are interpreted, and changing them in ways that lack legitimacy and consent. This has been a particular issue with the way the handball law has changed since the introduction of VAR.

VAR’s fixation with the correct process at the expense of sensible outcomes has become easy to mock. Phrases like “great process, lads” have become memes. So you might expect that FIFA’s intervention to overturn this decision and allow Balogun to play would be welcomed by everyone as a sign of common sense reasserting itself.

The wrong process and the right outcome

Except, of course, that has not happened. The decision to overturn the red card has generated as much, if not more, controversy as the original decision and has reminded a footballing public who are fed up of process of why it remains important.

Yes, we have learnt the hard way in the past few years that it is possible to follow the correct process and get the wrong outcome. But the Balogun decision is a reminder that the opposite error is just as damaging. If you get the right decision after following the wrong process, you create all kinds of new problems.

Balogun’s suspension was overturned using Article 27, which is a kind of FIFA superpower clause that lets them over-ride the consequences of any on-field decision. Now, what everyone wants to know is, how does this clause get activated? Can any player get any punishment overturned using Article 27? If so, then red and yellow cards are going to lose a huge amount of their deterrent power. If every player can’t get a card suspended, then who is making the decision about who can, and what are the criteria? Are there some guidelines? Is there an application process? Who gets to phone FIFA with the request? As the England manager Thomas Tuchel said, “Where does this start and where does this end? It’s my question. I don’t have an answer.”

People want to know what the process is for abandoning process. Suddenly, you realise why clarity and consistency matter so much, because if you don’t have them there is the perception of corruption. After Belgium beat the USA in the last-16 match, the Belgian FA posted “Overturn this” on social media.

In chapter 13 of my book, I predicted the problems that will occur if football starts to see decisions as provisional and open to endless revision – either from an excess of process, or from the lack of it. The Balogun incident is an illustration of these risks.

If football were run by the populist post-modernists, it would be a tremendously exciting spectacle. The refereeing would be atrociously incompetent, and horribly biased towards the powerful teams. Everyone would watch it because it was so compelling, but you would come away from it feeling unclean.

If football were run by the academic post-modernists, it would be a tremendously dull spectacle. The game would stop every minute for a five-minute review of all the footage from fifteen different angles. League table point tallies would be provisional, with confidence intervals, and subject to rolling legal challenges, statistical revisions and methodological updates. Nobody would watch it because it was so dull.

In both cases, the actual football would be a secondary consideration. The way to win trophies would not be to play better football or develop more innovative tactics. The way to win trophies would be to bribe (or more politely, ‘lobby’) the right official or statistician.

In football, and maybe in other areas too, we have a lot of leaders who are keen to stick rigidly to processes that aren’t working. We also have a lot of leaders who don’t care about processes at all. What we really need – and don’t have - are leaders who care a lot about designing and reforming processes so they are more likely to reliably lead to the right outcome.

OK, so how do we fix this problem?

The specific issue with Balogun should have been quite easy to avoid. Instead of a FIFA superpower clause, the World Cup needed a formal appeals system, like the ones that exist in many domestic leagues. If the USA could have appealed using a formal process, open to all teams, much of this controversy could have been avoided.

The wider problem with VAR is harder to fix. The tension between the letter and spirit of the law has existed for millennia. It is easy to say we need both, but how?

In chapter 7 of my book, I put forward a proposal for how AI can help us resolve the tension. Since writing that chapter, I have become even more convinced that it’s the answer – not just for football, but for many other walks of life. Last month, I wrote an article for AI Policy Perspectives about why deep learning was the perfect technology to solve VAR’s consistency-common sense dilemma.

The earliest AI methods were rules-based: experts tried to write down every rule a skilled human might apply when making a decision, in much the same way that football’s rulemakers now attempt to write down every possible way a ball might make contact with a hand.

These rule sets to create early AI multiplied, yet they couldn’t capture what a human was actually doing. The breakthrough came not from mor e or better human-crafted rules, but from giving the machine vast quantities of data and letting it intuit the rules itself. AI models trained in this way proved better at capturing and representing the tacit knowledge that experts hold but find difficult to represent in words.

We set up this Substack in 2023 to chart our experiments with AI assessment, where we deal with many of the same consistency-common sense conflicts. As you will know if you’ve followed us, we started out being fairly sceptical, but have gradually become more and more convinced.

I have had moments where I’ve found the power of the AI models to be quite frightening. The first time we ran a big AI assessment, I manually reviewed every major human-AI disagreement. They were all the result of human error. The AI was right about every one. Of course, since then we have encountered some AI errors, but I still remain astonished at what is possible now compared to just a few years ago.

We have known for a long time that machines are great at making fast and reproducible decisions at scale. But generally speaking, that speed has involved a loss of the kind of nuance and subtlety of the best human judgements. So it is quite uncanny to see machines making sophisticated, seemingly human judgements, but at a superhuman scale and with no human fatigue or carelessness. I can understand why Presidents and Popes care so much about this technology! The impact on the economy and productivity could be profound, but the impact on laws, institutions and human judgement could be equally transformative.

For now, though, we have a World Cup to complete under our old human rules. The letter and the spirit of the law are not done fighting yet.