Why Is The Hot Hand Fallacy Really A Fallacy?

The hot hand fallacy is one of those phenomena that divides both mainstream folks and scientific researchers alike. Rest assured that we will be focusing on the scientific side of things in this essay. Over the past decades, there have been quite a few plot twists in this field of research.

I will start this essay by giving a brief introduction to the notion of the hot hand. Then, I will be covering the history of research in this field that shook the scientific world. Finally, we will be arriving at the state-of-the-art research that re-shook the already shaken scientific plane. In the hope that this essay will help stabilize the ground rather than shaking it up more, let us begin.

This essay is supported by Generatebg

What is the Hot Hand?

When I was a child, I used to play cricket with the neighbourhood kids. For the uninitiated, cricket is a sport that involves a bat, a ball, and eleven players per team (two teams go at it); this level of knowledge suffices for our purposes here.

Neither was I particularly good at the game nor did I command any sort of special respect among my fellow teammates. If anything, I was one of the weaker players. However, on one particular day, I just happened to be in the zone. I was seeing the ball clearly and my hand-eye coordination was well tuned.

I was facing a highly-respected bowler from the opponent team. As I unleashed, I hit two consecutive balls cleanly, and one of them went out of the park! My team’s captain ran over with visible excitement towards my teammate who was standing on my opposite end, and said the following:

“No matter what we do, we have to make sure that Hemanth gets to face as many balls as possible. I’ve never seen him hit like this before. We have to make the best of it while he’s in the zone!”

I was quite surprised and intrigued to hear this. But I really felt something different this time — this sense of something more than myself activating. I knew that I was not a particularly good player. But at this moment, I was one of the best in the park. I had what we would call the hot hand and everyone seemed to sense this.

You might have had similar experiences where you had a short run of above-average performance. It need not have been a game; anything that tracks performance like a test at school would fit the scenario. Almost everyone intuitively understands and/or has experienced the hot hand.

---

What is the Hot Hand Fallacy?

In1985, Thomas Gilovich, Robert Vallone, and Amos Tversky published a paper titled “The hot hand in basketball: On the misperception of random sequences” (linked in the references section at the end of the essay). The research material in this paper targeted the sport of basketball and claimed that the notion of the hot hand was a fallacy!

Among the researchers involved, Amos Tversky is arguably the most well-known (primarily for his work in behavioural psychology). His co-researcher Daniel Kahneman went on to win the Nobel Prize for a behavioural psychology theory that he co-developed with Tversky. Had Tversky been alive at the time of this recognition, he would have also gotten the Prize as well.

Nonetheless, Amos Tversky is a well-established name in the social sciences, especially as a researcher who critically exposed human fallacies and cognitive biases. So, it was no wonder that this paper caused quite the stir. Their research seemed quite rigorous.

The Research Behind the Hot Hand Fallacy

Gilovich et al. considered the basket shooting records of the Philadelphia 76ers from the 1980–81 season, free throw records of Boston Celtics, and a controlled experiment featuring Cornell’s varsity teams. What they wanted to see was if the so-called hot hand streaks from players sufficiently differentiated themselves from events of pure chance.

In other words, they were looking for statistical significance that rejects the null-hypothesis that the notion of the hot hand does not exist. For an in-depth discussion on statistical significance, check out my essay: How to Really Understand Statistical Significance?

They took strings of hit/miss information and chopped them into blocks of 4. Then, they classified the blocks into bad (0 or 1 hits), moderate (2 hits), and good (3 or 4 hits) respectively. If you try to build a binomial model out of this information, you’ll arrive at 16 possible combinations.

For a shooter with 50% success rate, under the assumption that the null hypothesis (that hot hand does not exist) is true, we could expect that each of these combinations is equally likely. That is, you’d expect 31.25% of the shooter’s four-shot sequences to be bad, 37.5% to be moderate, and 31.25% to be good.

Gilovich et al. then went on to ask the following question:

If the null hypothesis is true, would it be extremely unlikely that we observe the results we observe?

The answer turned out to be a strong “no” (that is, no statistical significance). And so, they established that the notion of hot hand did not exist and that it was a cognitive bias that made human beings observe patterns in randomness.

They further went on to argue that by believing in the hot hand, players were potentially taking more risks by passing to one particular player rather than passing to objectively better players.

“…the belief in the “hot hand” is not just erroneous, it could also be costly.”

– Gilovich et al.

---

The Hot Hand Fallacy is a Fallacy!

After Gilovich et al. published their work, the research world was ablaze with heated arguments and discussions. All the while, Amos Tversky managed to defend their research work and point out sufficient flaws in methods followed by adversaries.

However, long after Tversky’s time, in 2015, Joshua Miller and Adam Sanjurjo published a paper titled “Surprised by the Gambler’s and Hot Hand Fallacies? A Truth in the Law of Small Numbers” (also linked in the references section at the end of the essay).

This was by far not the first paper to challenge the work of Gilovich et al. But this particular paper gained fame because it exposed a fundamental flaw with the original methods followed by Gilovich et al. after all these decades.

Miller and Sanjurjo showed that there was a subtle issue with chopping long strings of data into blocks of 4. I’ll explain this using a simplified example. Consider a situation where we are looking to predict the what comes after each hit in three-block-shots. Using a binomial model with 50% chance of hit or miss, we would arrive at 8 possible combinations of three-block-shots.

First, we note down the immediate result after each hit in each combination Then, we compute the proportion of hits that result from each combination. Finally, we compute the overall proportion of hits from the total bag. Based on our binomial model with a 50% hit or miss chance, we should expect a ratio of 0.5. But as you can see in the tabular illustration above, we get a ratio that is less than 0.5.

Gilovich et al. considered four-shot-blocks instead of three. If we extend this logic to four-shot blocks, we would get a ratio of around 0.4 as well. So, it is clear that chopping off long strings of information into blocks of four introduces a bias that tends towards misses than hits.

Miller and Sanjurjo showed that after considering this bias and correcting for it, the data from Gilovich et al. indeed showed statistical significance that supported the hypothesis that the hot hand exists.

Furthermore, Kevin Korb and Michael Stillwell showed in a 2003 paper (also linked in the references section at the end of the essay) that the statistical method used by Gilovich et al. lacked the resolution needed to capture statistical significance.

The Future of The Hot Hand Fallacy

Inthe end, the story of the hot hand fallacy still rages on. Researchers continue to collect data and test for statistical significance and the results seem to be divided.

There is more and more drive towards defining the phenomenon with higher resolution. For example, some researchers claim that it is not just a hit that counts as part of the hot hand, but also the speed with which the hit was made. So, the requirement for data collection also becomes more challenging with such resolutions in definition.

As far as the paper from Gilovich et al. is concerned, it is ironical that Tversky fell into the same trap that he (and Kahneman) exposed time and again. This is the fact that people expect smaller data sets to behave like larger data sets (imposing the law of large numbers on small data sets).

Richard Guy published a hilarious paper ridiculing this phenomenon. If you are interested in learning more about this, check out my essay on the strong law of small numbers.

I’ll conclude by saying that this story showcases the fundamental issue with the interface points that I talked about in my essay on statistical significance. The definition of the null-hypothesis and the interpretation of the term “statistical significance” are two areas causing a lot of agony in the statistical world.

---

Epilogue

Remember the story I told you about my childhood hot hand? After my team’s captain left the scene, I could sense the opponent team tensing up.

I was ready. The very next ball was angled behind my legs. I pulled off the finest flick I’d ever come up with in my life. I almost couldn’t believe it.

As I was enjoying the moment, I saw the athletic keeper behind me launching into the most acrobatic dive I’ve ever seen to take a spectacular catch. With disbelief, I had to walk back. My hot hand streak had come to an end. ‘Reversion to the mean’ took another victim that day.

---

References: Gilovich et al. (scientific paper), J. Miller and A. Sanjurjo (scientific paper), and K. Korb and M. Stillwell (scientific paper), and Jordon Ellenberg.

If you’d like to get notified when interesting content gets published here, consider subscribing.

Further reading that might interest you: How To Perfectly Predict Improbable Events? and The Bell Curve Performance Review System Is Actually Flawed.

If you would like to support me as an author, consider contributing on Patreon.