Newcomb’s Paradox – A Challenge To Game Theory

Newcomb’s paradox arises out of a simple money game between two players. The paradox was originally formulated by William Newcomb (the great-grandnephew of astronomer Simon Newcomb). Later on, the paradox gained popularity in the philosophical and mathematical circles due to the works of people such as Robert Nozick and Martin Gardner.

In this essay, we will cover the game setup that leads to the paradox first. Then, we will move on to analysing the game using game theory. Finally, we will touch upon the implications of Newcomb’s paradox on higher-level concepts such as free will and consciousness.

This essay is supported by Generatebg

The Rules of the Game

Imagine a game involving two players — you and a superior Artificial General Intelligence (AGI). Such an AGI is far more intelligent and computationally capable than any human being on the planet.

You are now presented with two boxes in front of you: Box A and Box B.

Box A is transparent and has $1000 placed inside of it. Box B is opaque and it is unclear as to what is inside at first. You then find out that Box B could contain either $1,000,000 or $0 based on the AGI’s prediction of your choice.

You have the following choices:

Choice 1: You can choose Box A and Box B together.

Choice 2: You can choose Box B only.

Now, the AGI’s actions don’t affect Box A or its contents whatsoever. However, if the AGI predicts that you would choose both Box A and Box B together (Choice 1), then it places $0 inside Box B. If it predicts that you would go for Box B only (Choice 2) instead, it places $1,000,000 inside Box B.

These are the rules of the game. Now that you know the rules, do you fancy a round?

---

What Would You Choose?

When I first encountered this game, the following thought occurred to me:

“This is a no-brainer; the choice is so obvious here!”

If you are like me, you likely have a strong preference lined up in your mind as well. If so, just hold onto that thought at the back of your mind for a moment.

When I studied the research papers and survey data behind this game, I realized that I could not have been more wrong! The data suggested that people were almost equally divided between the two choices.

To prove this point, check out the following statistical information from a 2020-survey conducted on 714 professional philosophers. Only a slight majority voted for Choice-1 in comparison to Choice-2 (39.03% vs. 31.19%).

So, what is going on here? To understand further, let us analyse how this game could play out using a game-theoretic approach.

Newcomb’s Paradox — A Challenge to Game Theory

To start our analysis, let us first breakdown the array of possible outcomes:

There are two approaches from game theory that we could take: the Expected Utility Theory and the Strategic Dominance Principle.

The expected utility theory encourages a decision based on probabilities. If you consider the AGI’s prediction capabilities as very high (close to certain = probability tending to 1), choosing both A and B sets the ‘expected’ winnings at about $1000 per game.

This indicates that the better choice is to choose Box B only (choice 2). So, this line of thought aims to statistically maximise your winnings and suggests you to go for Choice 2, setting the winnings at about $1,000,000 per game.

On the other hand, the strategic dominance principle does not consider the abilities of the AGI. According to this approach, you should choose the strategy that is ALWAYS better. Choosing both A and B (Choice 1) will always yield $1000 more than choosing only B (Choice 2).

Again, choosing both the boxes sets the winnings at about $1000 per game.

To summarise:

1. Expected Utility Theory — You should choose Box B only (Choice 2).

2. Strategic Dominance Principle — You should choose both boxes (Choice 1).

So, it appears that game theory is divided. Depending upon your inclination, one of these choices appears “obvious” to you. However, as a whole, humanity appears to be divided on the best possible approach. That is essentially the paradox.

Now that we have covered the details of the paradox, let us explore how it leads to more complex challenges.

---

Newcomb’s Paradox — Problems of Free Will and Consciousness

Free Will

The trouble with Newcomb’s paradox starts with the existence of the superior AGI. If we ‘generalise’ such a general intelligence, it leads to the existence of an intelligence that can deterministically predict human behaviour.

This opens up a whole can of worms. If it is possible to predict human behaviour deterministically, then the following question emerges:

“Does free will even exist?”

If we consider that free will exists, your human choice in the game would be ‘random enough’ for the AGI to not be able to predict your choice correctly every time. If the AGI can somehow successfully predict your behaviour, your behaviour could be related to the AGI’s choice causally in either direction. This line of thought leads to another can of worms in the form of the ‘chicken and egg’ problem or the grandfather paradox.

I am not going into further detail into this rabbit hole, but it suffices if you know that such messy after-effects exist if one chooses to investigate Newcomb’s paradox further.

Consciousness

Newcomb’s paradox also poses the challenge of machine consciousness. The AGI’s prediction model has to simulate human consciousness in some manner for it to predict correctly. If the AGI is capable of doing this successfully, you would not be able to tell if you are really standing in front of the boxes or are inside a simulation within the AGI’s virtual environment.

In this scenario, the argument is that as the “virtual” chooser, you would “tell” the AGI which choice you are going to make (refer to the essay: Are we living in a simulation?). As a result, you can conveniently go for Choice 2 (Box B only) every time you play the game.

Final Thoughts

Inthe end, what started out as a simple game turns out to be a mire of deep philosophical and mathematical questions. There appears to be no straightforward ‘right’ or ‘wrong’. So, it is no wonder that humanity is divided with its choice in the game.

I personally went for choice 2 (Box B only). My logic was closely aligned with the notion of the “virtual” chooser “telling” the AGI what the choice is going to be. This almost makes me think that the AGI is a wish-granting Genie.

There are some scientific arguments stating that Newcomb’s paradox is structurally isomorphic to Braess’s paradox (check out my essay on Braess’s paradox for more details) and hence, a counter-intuitive solution should be possible. There are other approaches that try to split the problem into stages and solve it.

All in all, the scientific community is still divided on this topic today and the paradox remains!

---

References: Robert Nozick (research article), 2020 Philpapers Survey, and A.D. Irvine (research article).

I hope you found this article interesting and useful. If you’d like to get notified when interesting content gets published here, consider subscribing.

Further reading that might interest you: Can You Really Solve The Staircase Paradox? and How To Run A Math Hotel With Infinite Infinities?