Creativity from Chaos: Like a Diligent, Discerning Demon
Lessons in the creative process from a 150-year old thought experiment in physics
"Odd how the creative power at once brings the whole universe to order."
— Virginia Woolf
For a few months now, I have been on a mission to find the source of this seemingly divine creative power. I’d imagined it to be a portal through which brilliant ideas, insights, and essays might spontaneously flow, not unlike the legends of Jack Kerouac’s frenzied caffeine-fueled typing of On the Road in three weeks on a 120-foot-long unpunctuated scroll. Unexpectedly, everything I read and experienced pointed elsewhere. Towards something simultaneously more ubiquitous, empowering, and earthly. So earthly, it seemed straight out of a physics textbook.
In 1867, Scottish physicist James Maxwell imagined a tiny demon chasing a deeply relatable yet impossible wish: creating order from noise without exerting any effort. Maxwell’s demon challenged the most fundamental laws of physics. Hence, his thought experiment set off decades of research to demystify the illusion that form can emerge without performing work. Doing so required physicists to define the essential nature of how random noise can turn into ordered information. Remarkably, those lessons echo the same secrets that many creative artists also seem to have arrived at, through their own solitary struggles in trying to produce coherent, beautiful, insightful works.
I first encountered Maxwell’s Demon fifteen years ago as a graduate student who had recently stepped into the kaleidoscopes of America and its university education. It was to be the start of a new journey. One that I hoped would lead me to gates that unlock the vast possibilities unknown to me. But I stumbled almost as soon as I began.
Choices abounded. The menu of sliced bread options at the grocery store was exceeded only by the prospective list of my department’s electives. I had not yet realized the range of ways in which I could microscopically sub-specialize as a protein structure biochemist, a theoretical evolutionary biologist, a biomaterials engineer, or at least a dozen other equally fascinating careers. I could also, instead, head down new alleys and rediscover myself as an intellectual property lawyer, pharma executive, or science writer. And all of these possibilities came before even getting to the more intimidating decisions on nurturing friendships, romances, and the soul. etc.
As I was struggling to restore direction to my quest, which now risked looking more like a random walk, Maxwell’s demon was indefatigably winning at its game.
In Maxwell’s thought experiment, a gas is distributed across two connected chambers, initially at the same temperature. The second law of thermodynamics dictates that the two chambers will maintain their state with gas molecules freely bouncing between them, unless an external force acts to change the status quo. The random motion of its molecules will prevent any pockets of higher or lower temperature.
But Maxwell envisioned a scenario that might challenge this system’s entropy — a miniature demon guiding a small frictionless door between the chambers. From its position, it could precisely observe and sort individual molecules, opening the door only for fast-moving ones, passing them through to one side and keeping the slower-moving ones on the other. Over time, one chamber, with more of the faster molecules, would heat up and the other cool down. And if the door was smoother than ice, the demon wouldn’t have expended any energy. The demon’s diligent yet seemingly effortless practice would forge directionality from the noise.
The demon’s achievement struck me as both inspiring and enviable. While I was drowning in the deluge of choices, here was Maxwell's demon, easily organizing its world, making an endless series of yes and no decisions, turning chaos into order — all while seemingly violating the laws of physics. Marvelous! That defiant demon became my hero. The demon that refused to be bound by thermodynamics.
Surely, it was too magical to be true. The demon couldn’t really defy physics. It must be exerting extraordinary effort to drive the two chambers to different temperatures. But how?
Almost as soon as Maxwell described the demon, physicists knew something was missing. Entropy can’t be defeated in a closed system like that. Physicists from Leo Szilard to Richard Feynman spent the next century disproving Maxwell to show that the demon couldn’t, in fact, turn a random collection of air particles into an organized form without expending any energy.
The Effort in Discernment
Although the door between the chambers had well-oiled hinges that didn't consume any energy, the demon was still working hard. Even before touching the door, it needed to sense the speed of each molecule. And gathering that information was no negligible task. In 1929, Szilard codified the idea that information was energy. He defined a “bit” as a unit of information about the location of each particle, in one or the other chamber. To realize the value of that bit, he proved, the demon must expend enormous energy. The key was not to focus just on the door — all the work done was in the discernment. The demon’s real superpower was knowing exactly which molecules were worth opening the door for, and which ones to hold back.
Ideas inspired by Szilard’s work on Maxwell’s demon still shape our lives in unseen ways. Two decades after Szilard, mathematician Claude Shannon used the equivalence of energy and information to develop a new theory of computation, which now powers all digital communication. And it laid the groundwork for artificial intelligence. The physics underlying Szilard’s own calculations is central to how quantum computers, millions of times faster than today’s machines, are being designed.
But even aside from those incredible inspirations for technology, lessons learned from the undoing of Maxwell’s demon propelled me almost as much as its magic had entranced me. I thought about the demon’s ability to discern, to evaluate, to judge. Was a specific course of action worth taking or not? Not unlike the demon which was making a set of rapid calls about which molecules to let pass, I too was facing many binary decisions: did I really need to read that 700-page textbook, should I join that student group that meets on Friday evenings, and which side of the river should I live on?
Studying Maxwell’s Demon showed me that getting answers is almost never easy. Discernment takes effort. Not only was it okay for me to not know all the answers, but spending time and energy trying to figure them out was the only way. All I had to do was make one decision at a time, weigh its two sides, and let it land one way or another. And then move on to the next such hoop to jump through in life.
And I didn’t even have to land on both feet, navigating every decision. I might opt for the nice-looking shoes and later find out that they are too uncomfortable to wear. But on the other hand, rushing across town at lunchtime to tour my dream apartment might mean that I get a chance to rent it before anyone else. So, just as long as most of the flips landed the right way, I would be filtering out some of the chaos and moving towards order.
Soon, thinking about decisions like that became a habit. Most of the choices overwhelming me were just a form of noise, which could eventually be overcome through diligent discernment. With that, I thought I had internalized most of what Maxwell’s demon had to offer. And I didn’t think of it again for years — until recently.
Tapping the Noise
“The way to get good ideas is to get lots of ideas, and throw the bad ones away.”
― Linus Pauling
This Linus Pauling quote was meant to be inspiring. A well-intended colleague shared it with me at a time when I was struggling to reboot my creative writing practice. I knew I desperately wanted to write. I made myself a Substack, joined an online writing course, and thought I was marching towards a promised peak. But I stumbled once again. I just couldn’t decide what to write about. I felt more like I was in a lake, and didn't even know how to tread water. Pauling’s advice was a distant shore. I couldn’t find one single idea. How was I supposed to come up with lots of them to throw away?
Of course, many blocked creators similarly struggle with the very first few steps, and common portrayals of creativity in pop culture certainly don’t help either.
It appears that ideas, beauty, and talents tend to descend from the heavens. Jack Kerouac’s legend is far from alone. It also happened with Neo discovering his unique connection to the Matrix’s Source, Harry Potter’s exceptional aptitude for defence against the Dark Arts, and with Pushkin’s Mozart channeling fully formed divine melodies directly onto the piano. And here I was, failing to strike just a couple of sparks for my non-fiction writing term paper.
Perhaps I needed to follow Thoraeu’s path to Walden Pond, shut myself out from the noise of the world, and wait for inspiration to emerge? But tapping my feet while facing a blank wall in my basement study didn’t work either, and I was adrift once again. What next? Weren’t noise, chaos, and the outside world the enemies to be defeated?
Or perhaps just the opposite. More recently, I read The Artist’s Way, and was intrigued by the role subconscious processing plays in nurturing creativity. Together, creators like Julia Cameron and psychologists like Timothy Wilson introduced me to a much more realistic view of creativity: one where the artist’s mind is not so much a lightning rod waiting to catch a wondrous bolt but is more akin to a pot of soil, nourished and filled with a variety of seedlings eager to sprout and flower1. All they need from the artist’s conscious mind is a chance to be noticed. And for the weeds to be plucked away.
In this conception, the seeds and the fertilizers are all around us, in what we read, who we talk to, where we work, who we live with, love, and to whom we lend a hand.
Although most of On the Road was typed in those three weeks in 1951, Jack Kerouac was tapping into a vast set of travel experiences, observations, and reflections diligently documented in his journals over the previous decade. Even his spontaneous style was a deliberate, cultivated choice. Kerouac studied James Joyce for years and expressed a desire for his own work to read like Ulysses. The intensely focused sessions were a means for Kerouac to tap into what he had been sowing within himself. His raw draft reportedly reads like a memoir. And it was the perfect substrate to be edited, structured, fictionalized, and eventually published more than six years later.
Even Mozart, as historians more accurately know now, turned into a musical savant only after absorbing years of sophisticated music as a toddler. His diligent studying of great composers like Haydn and Bach is a fact that just doesn’t make for great cinema.
The challenge, I’ve realized, is not to seal shut all doors on the noise but instead to channel Maxwell’s demon once again — this time to sift through the sand for glistening pieces of sea glass. It is the subconscious demon at work whenever a mastermind is hit with a brilliant flash of invention while on a mundane walk. Every poet piecing together a sparkling verse from the same words we all speak every day is also doing the deliberate work of the same demon.
Submerged in the static of social media, incendiary news bulletins, and humdrum daily duties is a universe of quiet epiphanies. But how do we keep parsing the pandemonium? I am still figuring that out. Though when I took a couple of hours one morning to let myself wander the halls of a museum, I was unexpectedly inspired by one of those beautiful, timeless kernels of truth we all seek.
"Invention, it must be humbly admitted, does not consist in creating out of void, but out of chaos; the materials must, in the first place, be afforded: it can give form to dark, shapeless substances, but cannot bring into being the substance itself.”
— Mary Shelly in Frankenstein
The purpose of life and consciousness, of illuminating enduring complexity and beauty from dark noise, then becomes a threefold task. First, to notice the air particles flying in the chamber and catalog the material of our experiences. Next, to filter all those ideas, develop a sense of taste strong enough to know how to “throw the bad ones away.” Third, and perhaps most importantly, to keep doing that over and over again. The repeated work of writing it all up and winnowing it down, then crafting the outlines to showcase what remains, refining the paragraphs, and selecting the words. The fodder for our works is all around us. Artistic expression just seems to demand unsparing documentation, discerning refinement, and unrelenting diligence.
Thanks to , , and for valuable advice in editing this essay.
From the archives
For a beautiful elaboration of the pot-of-soil analogy, check out ’s post “Where Do Ideas Come From?”
Wow, thanks Gairik for the insights into this Maxwell's experiment. I liked your take on drawing parallels between Maxwell's experiment and the challenges we face in taking decisions in real life. Also, your unique writing style draws the reader in and reflects the depths of your creative writing!
That was so interesting! i didn't know about Maxwell's experiment to start with, but it's such a good way to talk about current era noise!