Correlation (II): How To Spot The Surrogate Endpoint Problem? - A cartoon illustration showing two stick figures. On the left, a happy stick figure says, "To complete the study, we have to wait for you to die!" On the right, a sad stick figure with a green face seems uncomfortable with the conversation.

When it comes to the notion of correlation, the surrogate endpoint problem presents a very persistent challenge that no one has solved until today. It subtly catches researchers off guard and leads to false conclusions and decisions.

In my previous essay on correlation, I covered Francis Galton’s genius historical discovery and some of the traps in analysing correlation such as intransitivity and non-linear relationships between correlated variables.

In this essay, I will be covering the much more challenging issue of the surrogate endpoint problem. We will begin by setting up a simple correlation scenario between two binary variables.

Then, we will shortly cover how this simple setup leads to statistical traps. Following this, we will proceed to see how the surrogate endpoint problem presents itself naturally. Without any further ado, let us begin.

This essay is supported by Generatebg

A product with a beautiful background featuring the sponsor: Generatebg - a service that generates high-resolution backgrounds in just one click. The description says "No more costly photographers" and displays a "Get Started" button beneath the description.

Correlation Between Binary Variables

Consider the hypothetical example that a vegan diet is positively correlated with COVID-19 infection. Both of these factors could be treated as binary variables. That is, both variables could be answers to yes or no questions.

A person can either follow a complete vegan diet or not. Similarly, a person can be infected with COVID-19 or not. Based on the positive correlation, we could say that a vegan person is more likely than the average person to get infected by COVID-19.

This is the same as saying that a person infected by COVID-19 is more likely than the average person to be vegan. Do you realise that both of these are logically equivalent statements?

If not, why don’t we look at their mathematical expressions? The first statement can be expressed mathematically as follows:

(Vegans with COVID-19) / (All vegans) > (All people with COVID-19) / (All people)

Similarly, the mathematical expression for the second statement is as follows:

(Vegans with COVID-19) / (All people with COVID-19) > (All vegans) / (All people)

To see the logical equivalence more clearly, multiply the first expression by [(All vegans)*(All people)] on both sides. Similarly, multiply the second expression by [(All people with COVID-19)*(All people)] on both sides. In both cases, you will get the following expression:

(Vegans with COVID-19) * (All people) > (All people with COVID-19) * (All vegans)

So, now that we have established the logical equivalence, let us move on to the statistical treatment of correlation.

Statistics and Correlation

When we consider the last expression, one prominent problem pops up. When we consider any meaningful sample size, there is only a very slim chance that the left-hand side of the expression would be exactly equal to the right-hand side.

Correlation (II): How To Spot The Surrogate Endpoint Problem?— An illustration featuring a comic, where a stick figure standing behind an announcer table is pointing to a presentation of what appears like linear regression data. The stick figure says, “Here, we see correlation between correlation and causation!”
Correlation presentation cartoon— illustrative art created by the author

What this means is that these two variables are going to be correlated one way or another. And there is nothing special about being vegan or suffering from COVID. You can expect gender or blood type or any binary variable to be positively or negatively correlated with COVID risk.

So, how do we solve this problem? That is where the statisticians come in. When you read reports on correlation based on scientific studies, what you get are statistically significant correlations.

The concept of statistical significance leads to a whole host of issues that I have covered in a dedicated essay. But the issues don’t stop there. Even with statistically significant correlation, causal misinterpretation shows up again!

Causal Misinterpretation from Correlation

Let us say that we indeed managed to establish in a statistically significant fashion that being vegan is positively correlated with mortal COVID-19 infections. Consequently, the following statement is in order:

“If you are vegan, you are more likely than the average person to get infected mortally by COVID-19.”

This statement states factually what we know so far. But it is missing the flair and finesse that mainstream media generally looks for.

So, you will often find the following statement instead of the above one in mainstream media:

“If you were not vegan, you would be less likely to be mortally infected with COVID-19.”

The difference is subtle, but the implications are huge. While the first statement was factual, the second statement implies a causal link between the two variables.

We have in no way proved that there is a causal link between the variables from the statistically significant correlation alone. This issue directly leads us to the main issue we are covering in this essay.

The Surrogate Endpoint Problem

The surrogate endpoint problem arises naturally from many correlation scenarios. Consider the previous example of the correlation between being vegan and mortal COVID-19 infections.

It is very resource intensive and time-consuming to invest in scientific studies that quantitatively measure mortal COVID-19 risk from veganism. The researchers have to wait for vegans to die from COVID-19.

Correlation (II): How To Spot The Surrogate Endpoint Problem? — A cartoon illustration showing two stick figures. On the left, a happy stick figure says, “To complete the study, we have to wait for you to die!” On the right, a sad stick figure with a green face seems uncomfortable with the conversation.
Correlation study cartoon — illustration created by the author

So, instead of waiting, the researchers try to find a surrogate endpoint. This could be some biomarker such as blood oxygen levels.

If the blood oxygen level of a vegan infected with COVID-19 drops below a threshold number, the researchers might declare the situation a mortal risk.

The surrogate endpoint, then, is a proxy that takes the place of a much more complex phenomenon.

Not only might the actual phenomenon and the surrogate endpoint not be causally linked, but they might be consequences of some factor that we never even tracked.

As a result, our analysis might lead to false conclusions and erroneous decisions.

How to Spot the Surrogate Endpoint Problem

Whenever you read phrases like “This spray increases cancer risk in users…” or “Eating this food increases cardiovascular health risk…”, pay attention to what was actually measured to make such conclusions.

Phrases such as “cardiovascular health risk” or “cancer risk” are usually quantified by some surrogate/proxy. It need not always lead to the surrogate endpoint problem, but the issue always lurks around such studies.

More than 130 years after Galton’s discovery of the notion of correlation, we still struggle to thread our way between correlation and causation thanks to issues such as statistical significance and the surrogate endpoint problem.

For all we know, the root cause of this issue could be our (human) nature rather than our (lack of) scientific progress. But that does not mean that we should stop trying to solve such problems.

Such is our strong drive for scientific progress and evolution as a species!

Reference and credit: Jordon Ellenberg.

If you’d like to get notified when interesting content gets published here, consider subscribing.

Further reading that might interest you:

If you would like to support me as an author, consider contributing on Patreon.

Street Science

Explore humanity's most curious questions!

Sign up to receive more of our awesome content in your inbox!

Select your update frequency:

We don’t spam! Read our privacy policy for more info.