Entropy And Heat: The Hidden Connection Behind Time Flow

The concepts of entropy and heat share a subtle relationship that plays an important role in the uni-directional flow of time. While pop-science culture often refers to entropy as ‘disorder’ (which leads to misunderstandings), the relationship between entropy and heat almost never gets any love.

It is perhaps the very technical nature of this relationship that makes it hard for mainstream pop-science to grasp it. My aim with this essay is to deconstruct this technical relationship into simpler language that is accessible to non-technical folks as well.

In my essay on the origins of entropy, I began by narrating the story of how Rudolf Clausius and co. discovered entropy whilst trying the improve the efficiency of the old steam engine. To understand the subtle relationship between entropy and heat, we will be picking up from this discussion.

For those who are short on time, I will recap the most important bits about the origins of entropy in this essay. But if you wish to gain a holistic grasp of these concepts, I highly recommend you read the origins before proceeding with this essay. Without any further ado, let us begin.

This essay is supported by Generatebg

Recap — How Was Entropy Born?

In the mid-1800s, the steam engine was at the centre of the booming industrial revolution. This beautiful invention enabled quick and efficient transportation of not just people but also industrial raw materials and goods.

At the core of it, however, the steam engine had a fundamental problem. You see, the steam engine lost up to 95% of the heat produced by burning its fuel (such as wood/coal) as waste to the environment. This blatant inefficiency drove many scientists and engineers crazy.

Whole generations of researchers and tinkerers such as Lazarre Carnot and Sadi Carnot (L. Carnot’s son) threw themselves into solving this problem. As one of the by-products, the field of thermodynamics popped out (not a big deal, right?).

L. Carnot figured out the notion of work loss in machines due to shocks and accelerations. S. Carnot developed the crucial concept of thermodynamic reversibility. Rudolf Clausius built upon these concepts and came up with the notion of irreversible heat loss, which he later termed entropy.

Recap — How is the Steam Engine Related to Entropy?

In the context of the steam engine, the dictum goes as follows:

The process of burning fuel to run the steam engine will always produce irreversible heat waste — something that leads to degradation.

Firstly, this formulation is very crude. Secondly, it lacks the statistical finesse that Ludwig Blotzmann and co. brought to the discussion much later.

But I wish to linger on this crude statement, for it holds the key to understanding the subtle relationship between heat and entropy.

Let us start by asking the following question:

Why is heat waste unavoidable and irreversible in a steam engine?

---

The Steam Engine for Dummies: A Crash Course

The Steam engine works using the heat generated by burning its fuel. This heat, in turn, boils water and turns it into steam. This steam expands and creates pressure that pushes a piston down a snugly fit barrel.

After expansion, the steam cools down (either actively or passively), and the piston returns to its original position. At this moment, a fresh dose of steam awaits to push the piston down the barrel again.

This cycle creates a uni-axial to-and-fro motion of the piston. Innovators of the time converted this uni-axial motion into rotational motion using contraptions such as gears to drive the locomotive wheels forwards or backwards.

Now that we know how the steam engine works, why is heat waste unavoidable during its operation? We are close to answering this question, but we are still missing one important piece of the puzzle.

A Childhood Experience to Remember

When I was a child, I loved playing with the iron; I just liked its form factor for some reason. My father, of course, did not approve of this. He actively discouraged me and warned me that I would eventually get hurt.

To no one’s surprise, one fine day, I put my hand against the hot surface of the iron. It probably took a few milliseconds before I felt a burning pain in my palm. I remember screaming in pain.

There are some things children need to experience first-hand to really grasp; words are simply not sufficient.

In any case, why did my hand hurt? Well, the hot temperature of the iron burnt my hand. But that is just a surface-level explanation. Let’s get more specific.

We can define the temperature of a body as the average speed of its molecules; it is a measure of the amount of kinetic energy within the body.

The Nature of Heat

The molecules on the surface of the iron were moving/vibrating at a higher average speed than those of my hand. When my hand came in contact with the iron, the high-speed molecules of the iron collided with the slower-moving molecules in my hand, causing their speed to go up as well.

As soon as my brain sensed this increased average molecular speed, it registered it as heat/pain. Note that during my interaction with the iron, something flowed from the iron to my hand.

In olden times, people imagined an invisible substance known as “caloric” that flowed from hot bodies to colder bodies. Today, any lay person would say that it was “heat” that flowed from the iron to my hand.

In reality, no physical “matter” flowed from the iron to my hand, but a property of that matter, namely the average molecular speed, did. This is what the lay person would refer to as the flow of heat.

What Do Heat and Entropy Have in Common?

In the origins of entropy, I explained how a high entropy macrostate of a system features an awesome number of possible micro-configurations that represent the said macrostate.

Conversely, a low entropy macrostate of a system features a relatively lower number of possible micro-configurations, which represent the said macrostate.

When the iron is hot, its average molecular speed is high; the possible number of micro-configurations that lead to the same macrostate are also high. Therefore, the iron’s entropy is high.

On the contrary, my hand has a lower temperature and lower average molecular speed. Therefore, my hand has lower entropy.

When my hand touched the iron’s hot surface, the faster moving iron-molecules collided with the slower-moving hand molecules. In the process, the average molecular speed of my hand also increased. Now, ask yourself what happens to entropy during this interaction.

---

The Hidden Connection Between Heat and Entropy

By now, we have made it clear that the (hotter) iron has higher entropy in comparison to my (colder) hand. But during the speed transfer (touch), the iron’s temperature decreased a little.

As a result, its entropy also decreased a little. Consequently, my hand’s temperature and entropy went up a little.

What does this mean? Well, it means that for heat to flow from the iron to my hand, entropy needs to flow from the iron to my hand as well. Conversely, for entropy to flow from one body to another, heat needs to flow as well.

In short, entropy rides heat flow. This is the subtle relationship between entropy and heat. Now that we have come thus far, we are finally ready to tackle the question that sent us on this journey:

Why is heat waste unavoidable and irreversible in a steam engine?

On Course to Explain the Steam Engine’s Heat Waste Production

Remember that the steam engine’s piston returns to its original position after each cycle? For this to occur, the entire engine’s macrostate must also return to its original form (or an equivalent state).

In other words, if the steam engine’s macrostate is reset at the end of each cycle, it means that its entropy must also be reset! In the previous section, we just saw that entropy rides the flow of heat.

So, if the steam engine resets its entropy at the end of each cycle, it MUST release heat to the environment at the end of each cycle as well. This realisation explains why heat is inevitable when operating the steam engine. But it still does not explain ‘heat waste’.

Think about it. The steam engine receives heat from the burning fuel and releases heat at the end of each cycle. So, why should it produce heat waste?

A Thought Experiment

To understand why, imagine two room heaters: one kept in a cold room and another identical one kept in a warmer room. Let us now assume that both room heaters produce the same amount of heat.

In the cold room, the molecules of air have a lower average molecular-speed and hence posses lower entropy. As soon as the room heater starts heating this room, the air molecules collide and start moving faster and disperse all over the room.

Compare that to the warmer room. The air molecules there have a higher average molecular speed, and hence, higher entropy. So, the identical space heater increases the molecular speed only by a little in comparison to the colder room.

In other words, with identical heaters, the entropy increase in the colder room is higher than in the warmer room. The takeaway from this thought experiment is this: the difference in environmental temperature affects entropy flow.

Great! Back to our steam engine now.

---

The Mystery of the Steam Engine’s Heat Waste

Tounderstand why the steam engine’s heat waste production is unavoidable, we were missing a vital link. This link is the fact the there is a temperature difference between the burning fuel and the steam engine’s environment.

Because of this temperature difference, the steam engine can release all of the entropy that it collected from the burning fuel by expelling only part of the heat it collected to the (cooler) environment.

The steam engine, then, uses the remaining heat to thrust the piston back and forth. This is what we call useful heat. We call the heat that the steam engine releases to the environment as ‘heat waste’.

Should it not be Impossible to Reduce Entropy?

The real beauty of the steam engine is that it constantly reduces its entropy at the end of each cycle during its operation. Compare that to what the second law of thermodynamics states:

The entropy of an isolated system that is spontaneously evolving can never decrease.

So, what gives? How does the steam engine reduce entropy? Well, the key to this lies in understanding the term ‘isolated system’.

For the second law to apply, we must not only consider the steam engine itself, but all the systems and sub-systems it interacts with such as the environmental air molecules, the leaves they touch, etc.

If we do a careful accounting of the entropy of the whole system, we will note that the entropy of the whole system does indeed go up. It was the same scenario with the iron and my hand as well. The iron’s entropy went down a little as my hand’s entropy increased a little.

But if we tallied the entropy of the entire environment such as air, table, chair, etc., we would note that the net entropy of the system does goes up.

---

The Link Between the Steam Engine and Time Flow

In origins, I mentioned that entropy lies at the heart of why time flows in one direction and time-reversal is not possible. We will get a glimpse of this phenomenon in this section. But for now, let us get back to the steam engine.

The steam engine is not immune to the second law. There is indeed a natural drive for its entropy to increase. But we, human beings, have designed it such that it fights this natural drive. How does the steam engine actually fight the natural drive to increase entropy?

Well, it does so by transferring the entropy it accumulates to its surroundings. And to do this, it MUST release heat waste to its surroundings as well. This notion is not exclusive to the steam engine.

Any man-made apparatus faces the natural drive to increase its entropy; the natural drive to breakdown and wither away. So, the said man-made apparatus maintains its structural integrity by locally reducing entropy. And all this comes at the cost of unavoidable heat waste.

To link all of this with time flow, we just need to ask a simple question: if devices that locally reduce entropy actually contribute a global entropy increase, where is all this entropy headed? The answer is in one direction: forward time.

Final comments

Asyou ponder upon the fascinating concepts that I have shared with you in this essay, I will conclude by reiterating a vital point:

All the physical systems that we know of will, with extraordinarily great likelihood, evolve from states of lower entropy to those of higher entropy over time.

So, where does the limit lie? The answer to that question requires a discussion on its own. I’ll continue this discussion in future essays on this topic. For now, I hope you enjoyed reading this one!

---

Reference: Brian Greene.

If you’d like to get notified when interesting content gets published here, consider subscribing.

Further reading that might interest you: 

• Why Is The Hot Hand Fallacy Really A Fallacy? 
• How Imagination Helps You Get Good At Mental Math?
• How To Easily Outperform High IQ Individuals.

If you would like to support me as an author, consider contributing on Patreon.