Dark Sector 2: Dark Energy

After years of research, astronomers and cosmologists have determined that we don’t understand more than 95% of the contents of the Universe. Less than 5% is composed of the same kind of matter as us - made of atoms and their component protons, electrons, neutrons and other subatomic particles, known as baryonic matter. Despite our colossal ignorance of the remainder, we can constrain some of its properties, and crucially have determined that it comes in two flavours, dubbed dark matter and dark energy. Collectively known as the dark sector, this vast amount of invisible content lies at the cutting edge of astrophysics and has captured the imagination of both science fiction readers and writers.

Last time in Cosmic Stories I considered the role of dark matter in science fiction. In this second of two entries, I’m going to take a look at a still greater puzzle: dark energy.

An Accelerating Universe

When Albert Einstein formulated his theory of general relativity (GR), he encountered a problem. The equations which fell naturally out of his model of the interrelationship between time, space, energy and mass all had solutions which implied that the Universe was not static - instead it should be constantly in a state of expansion or contraction. Since cosmologists of the time assumed the Universe had been eternal and in equilibrium, Einstein introduced a cosmological constant, lambda () - an extra term describing excess energy to counteract the effect of gravity (like keeping something inflated by using heat energy to warm it up). Later, when evidence was found for an expanding universe in the 1920s and 30s, the cosmological constant was quietly dropped, with Einstein describing its invention as the worst blunder of his life - both in terms of accuracy and in terms of lacking the intellectual courage to accept the implication of his work.

And there the matter rested until observations of distant supernovae and then of the cosmic microwave background radiation in the late 1990s once again challenged the same equations. While GR implied the Universe was expanding, it also implied that it should be slowing down (decelerating), in the same way that an object dropped onto a rubber sheet will be slowing down even as it’s pushing its way ever deeper into the sheet, and will eventually come to rest. Indeed, if there was enough matter in the universe, above a certain critical density, the expansion might come to a stop entirely and start to contract again - with gravity acting like the elastic rubber in the sheet bouncing the ball back upwards. However the observations showed that the expansion of the Universe was, in fact, accelerating instead. This can’t be done with gravity, and implied something was pushing outwards instead - in fact operating in exactly the same way as Einstein’s cosmological constant. This means that instead of snapping back into a Big Crunch, the structure of space-time (our rubber sheet) may instead continue being stretched until eventually (trillions of years into the future) we encounter a “Big Rip” instead.

Since the early 2000s, astronomers have largely accepted a ‘concordance’ model in which less than five percent (a twentieth) of the mass-energy density in the Universe is made up of baryonic matter like ourselves, twenty five percent (a quarter) or more is made up of dark matter, and the remaining seventy percent is made up of this repulsive force: dark energy. What’s more the total of these three quantities at the current time is very close to the critical density, suggesting that our universe will neither tear apart nor collapse back to a singularity in the very distant future, but instead eventually settle to something in between, very close to the static universe Einstein first envisaged. 

As is the case with dark matter, although the evidence for its existence is robust and quantifiably real, there is no clear and easy explanation for what dark energy is, although several theories have been proposed. In its simplest form it may have been unchanged throughout the history of the Universe - acting as a constant still known as lambda - and only becoming dominant as the universe grew and matter has spread out. In its more complex form, there are hints that dark energy may be changing with cosmic time (very tentative hints from supernova analyses, which will become clearer over the next few years), or that it might have still more complex properties. Meanwhile, terms associated with individual theories regarding its characteristics and nature, such as vacuum energy, or quintessence, have become synonymous with dark energy itself.

 Filling a vacuum

The idea that vacuums could be far from empty started appearing in science fiction well before the modern era of the cosmological constant, as an integral part of quantum mechanics, which requires that the lowest energy state of anything is non-zero. The Casimir Effect, first observed in the 1930s, demonstrated that vacuum can spontaneously generate particle-antiparticle pairs when constrained between metal plates sufficiently close together.

Thus vacuum energy appears relatively early in science fiction, often as a source of energy for spaceship drives or other technologies, as for example in Rogueworld by Charles Sheffield (short story, F&SF, May 1983).

An interestingly philosophical approach is taken by Geoffrey Landis in his 1988 short story Vacuum States (Asimov’s, July 1988). This describes an experiment to extract zero point energy from the vacuum, but focuses on a moral dilemma: if there’s even a tiny chance that this could trigger a collapse in the Universe to a different quantum state (thus destroying all life), should the experiment proceed?

A far more comedic take is that of Take me to the Pilot (Short story, di Fillipo, Asimov’s, August 1995) in which two characters step into a chaotic ‘oververse’ where events and objects reflect those in the real universe. The grass on which they walk represents the vacuum energy, with each crushed blade a minor adjustment to the energy-particle balance of the universe.

While work in particle physics has confirmed the existence of vacuum energy and its role in the laws of quantum mechanics, it could not (or did not try to) explain its connection to the cosmos as a whole. To do this robustly would require a grand unified theory (or GUT) that unites quantum and relativistic physics, and which still remains an unresolved challenge in today’s physics. Stories which exploit vacuum energy usually assume that the problem of devising a GUT has been solved. However the discovery of the acceleration of the Universe’s expansion suggest a connection to another form of energy that seems to be intrinsic to the fabric of space-time: dark energy.

Glimpses of Darkness

Naturally, science fiction has followed developments in this field with interest.

Published in Analog magazine in February 1996, David Brin’s short story An Ever-Reddening Glow looks at the cosmological expansion as a form of pollution generated by the most common stardrive of sentient species. While it doesn’t name dark energy, it picks up on the early indications that the expansion might be accelerating and suggests that at least half of the expansion might be attributable not to the Big Bang but instead to a form of propulsion that generates and collapses microscopic black holes to cause ripples in space time a craft can surf upon.

"While you speed ahead, you cause the distance from point A to point B to increase, making it marginally harder for the next voyager to make the same crossing."

I laughed. "Marginally is right! It would take millions of ships . . . millions of millions . . . to begin to appreciably affect interstellar distances, which are already increasing anyway, due to the cosmological expansion—"

The star-probe cut in. "And where do you think that expansion comes from?"

The protagonist, one of a group of environmentally-aware colonists, faces the same dilemma as so many when confronted with a call to give up speed and convenience for the common good:

“Alas, it is hard to practice self-control when you are young, and so full of a will to see and do things as fast as possible. Besides, everyone else is doing it. What difference will our measly contribution make to the mighty expansion of the Universe? It's not as if we'd help matters much, if we alone stopped.”

The idea that the great mysteries of the dark sector might be resolved by alien action appeared elsewhere. The Missing Mass (short story, Analog, Dec 2000) is one of Larry Niven’s series of stories describing the peculiar conversations to be had amongst the interstellar clientele of the Draco Tavern - where the human barkeeper and his compatriots represent one of the least advanced species.

While the title is a reference to the Missing Mass problem that led to the discovery of dark matter, the story’s main focus is on the suspicion that the advanced alien Chirpsithra may be tapping the vacuum energy for their interstellar engines. As such the discussion swirls around human cosmology, including the evidence from type Ia supernovae that the expansion of the Universe may be accelerating (the key evidence for dark energy). While the Chirpsithra present deny they possess any such drive, they describe ancient beings who could convert vacuum energy to mass, explaining the origin of galaxies and why the mass and energy of the Universe appear inconsistent now.

However this was still early days for dark energy, and the discussion in The Missing Mass comes across as a little muddled.

By the time Stephen Baxter’ wrote his short story In the Abyss of Time (Asimov’s, August 2006), the observational evidence was clearer and a much more familiar view of modern dark energy emerges. The narrative describes an experimental time travel mission which dives into the far future, with the genius inventor and his physicist colleague taking along a reporter for the experience. Having watched the Milky Way’s collision with Andromeda and travelled twenty-five billion years into the future, the travellers look out for the Big Crunch (a victory of mass over energy) which fortunately doesn’t come to pass. As the journalist character is told,

“We knew the universe had come barrelling out of the Big Bang, and gravity controlled the future. If the mass density of the universe was too high, if gravity was too strong, then the universe would reach some maximum radius and start to fall back on itself. Otherwise the universe would expand forever. Big Crunch, or endless dissipation. But that simple picture fell apart when those anomalous distant-supernova results turned up in the 1990s. And now the answer to that epochal question about the universe’s ultimate fate depends on the properties of dark energy, which are unknown.”

Indeed the genius is explicit about his goals:

"Now you know my objective," Elstead said. “To observe directly our cosmological future—to see which of many possible outcomes we must endure—and thereby, incidentally, to confirm various models of fundamental physics by direct inspection of their far-future consequences. What a goal it is!”

Having ruled out a lot of different models, concluding that dark energy was indeed likely to be constant, and watched the victory of dark energy in infinite expansion, the crew witness a predictable real-world consequence of dark energy - a gradual fading out of distant galaxies as the Universe begins to expand faster than their light can reach us. Ultimately the crew of the Spacetime Bathyscape watch the shrinking of space-time horizons to a single sun and the decay of protons … before finding themselves unexpectedly attacked and fleeing home to their own time. The story ends with many unanswered questions, included amongst them the question of whether it’s possible to see so far into the death of the Universe and remain sane.

Perhaps the best short story to explore the nature of dark energy (in my own opinion) can be found in The Burst by C W Johnson, which appeared in Asimov’s, January 2012. This describes the work of a young graduate student, Cayla, who is taken on by a famous, elderly professor. Determined to live up to him, she works intense, long hours and pores over observations of the Large Magellanic Cloud, looking for new phenomena. Having found a series of astrophysical transients - bursts a trillionth as powerful as gamma ray bursts, she constructs a meticulous and plausible explanation that links the observations with quantum physics and suggests that the bursts are releases of energy caused by an unstable non-linearity in the Schrodinger equation of quantum mechanics, associated with the budding of alternate universes:

“You heard me talk about dark energy?”

“Talk, yes. Understand, no. All I remember is, dark energy something something expanding universe something something accelerating something something, or something.”

Cayla smiled. “Actually, that’s a reasonable summary. We know the universe is expanding, we know the rate of expansion is accelerating, and something must cause that acceleration. But no one knows what that something something is. Or knew.” Her smile grew into a grin.

“Let me guess ”

“The energy from these bursts, my bursts, roughly equals the kinetic energy added to the expansion of the universe. Hard to dismiss it as a coincidence.”

In this narrative, Cayla has found the origin of dark energy. The supervision and research interaction here is rather old fashioned, with more students today working more closely with their supervisor or as part of larger teams. It is still recognisable nonetheless. However, as well as capturing the uncertainties, pressure and relentless imposter syndrome of a PhD student in astrophysics, the story also demonstrates that the cosmic questions pale in the face of human ones - both in the form of a health scare for her boyfriend and a bereavement for her mentor.

End Times

Perhaps the greatest fascination and area for speculation that arises from dark energy is in the area of eschatology - a study of the eventual fate and end of the Universe [1].

Stephen Baxter, who has a fascination with hard science, has explored a range of possible futures, as well as the desolation described in In the Abyss of Time. His earlier short story The Gravity Mine (Asimov’s, April 2000), looks at the far future of humanity, many trillions of years in the future, at a time so remote that the current era of luminous stars burning baryonic matter is no more than a dim memory. Both the decay of baryons (protons themselves are unstable over cosmological timescales) and the ongoing expansion of the universe has left the last handful of ultra-massive black holes as the only remaining energy source. The story focuses on the efforts of the last post-human descendants to survive and go on amidst their smouldering embers.

Baxter returned to the use of dark energies repeatedly throughout his career, including in Firstborn, his collaborative novel written with Arthur C Clarke, in which it is a mechanism by which time and space can be manipulated. While most of his fiction involves sweeping cosmological scales and events, his short story Last Contact (published in The Solaris Book of New Science Fiction, ed. Mann, in 2007) is notable for bringing the big questions down to a quieter, suburban human scale. Discovering that the Big Rip is approaching far more rapidly than anyone previously suspected, an astrophysicist visits her widowed mother’s garden repeatedly during the last few months as the stars and galaxies fade beyond the horizon, before sharing the last moments with her as the Sun fades out and the Earth itself is torn apart.

A more rarified view was taken by Gregory Benford in The Final Now (short story, published by Tor.com in 2010). This envisages male and female deities existing throughout time and creating an Other (implied to be humanity) to challenge and entertain them. The story follows the Other’s insistence on striving for immortality through to the very end, when the accelerating, expanding universe tears apart in a great rip, still under the watchful eyes of the deities.

By contrast David Moles’ short story The Metric (Asimov’s, May/June 2021) is more practical. Here the dark energy is not the focus of the story, but rather provides the context. In a very distant future, where only two ancient cities are left on Earth, a messenger arrives from the long-abandoned stars, squeezing its way through an ever more distorted space-time. It carries the news that the metric - an ancient structure built by humans to combat the accelerating expansion of the Universe and keep civilization going - is failing, and that the end of the Universe in a Big Rip is imminent [2].

“The history Petal knew at least in outline: that once, when the quintessence had been weak, there had been stars in the sky, and worlds around those stars, worlds found, or made, as the people of those primordial times had gone out from this Earth or one much like it; that they had built ships, magical ships the size of cities, to sail between those stars, almost as fast as light, and then faster. That as the quintessence grew stronger and threatened to tear those stars from one another they had knit and forged the metric out of space-time itself to hold them together. And that it had held, for an age as long as all the time before, linking the worlds of humanity by secret fast ways, even as the sky went dark.
But in the end, Petal also knew, even the metric had not been enough - the quintessence, pulling apart the nodes of the metric and pulling the metric itself ever tighter, had prevailed.”

Now the residual effects of the construct will prevent the universe ending cleanly, and hence the birth of the next, and it must be deactivated… by destroying the Earth. The bulk of the story is a coming-of-age tale about the young girl, Petal, running away from home to carry this news to the other surviving city, and the subsequent travails of both her and her family. In this sense, it is interesting that just two decades after it was accepted back into astrophysics, the existence of dark energy has become normalised to the extent where it no longer needs to be explored or solved, but simply becomes the context against which human lives are lived and ordeals survived.

The Cosmological Constant

In January 1933 an article in the early science fiction pulp magazine Wonder Stories reprinted an editorial originally appearing in the New York Times in November 1932. Comparing fashions in cosmology to those in Paris fashion houses it laments:

“But early this year Einstein, de Sitter et Cie. upset the whole Rue de la Cosmogonie by snipping out of their expanding universe its principal glory; in other words, its gorgeous lambda, its “cosmological constant,” which was originally introduced by Einstein to satisfy an esthetic need, but which ended by making the universal crinoline expand. The crinoline is still curved, but whether the curve is convex or concave Einstein, de Sitter et Cie. are unable to say.”

The previous December, the same magazine had printed an editorial from Hugo Gernsback discussing the discovery of the Universe’s expansion at the very end of the 1920s and the mystery of its fate, concluding that:

“The subject is so vast, and our knowledge is so primitive and childlike, that it will be thousands of years before anything worthwhile knowing about this fascinating subject will have been disclosed.”

As insightful and foresighted as Gernsback could be, he had no idea of the huge advances in observational cosmology that would be made over the near-century that has passed. The ability to observe ever fainter objects, and particularly to observe from above the atmosphere, opened up the measurements of both distant supernovae and the cosmic microwave background.

The influence and impact of dark energy are now being measured through a range of different phenomena, each of which provides complementary information. These firmly favour models including dark energy over any without. They now also slightly favour a dark energy that evolves over time, based on results from the Dark Energy Spectroscopic Instrument (DESI), although this result is still tentative. Over the next few years, the Legacy Survey of Space and Time at the Vera Rubin Observatory is likely to be amongst the most pivotal tests for its properties, finding more and more distant supernovae than has previously been possible. We might yet find evidence that the cosmological constant is far from the unchanging fixture of the Universe that its common name suggests and that the fate of the Universe differs from the current expectations of a Big Rip as articulated by Last Contact, The Final Now and The Metric.

The record of science fiction shows that the mystery of that end state remains a fascinating topic for speculation

As the laments of 1933 demonstrate, the cosmological paradigm can shift abruptly, on the weight of the evidence, as previous assumptions are challenged, tested and revised in light of new data. This is the scientific method in practice, and the speed with which the Lambda Cold Dark Matter (i.e. dark energy and dark matter) universe model became canonical around the turn of this century reflects the strength of the evidence supporting it. While it may yet be replaced, that replacement will only occur when a new model is able to explain more data, and in more detail, than the preceding one - just as LCDM itself did. As of yet, there is no sign of such a replacement model emerging.

Even so, astronomers keep an open mind. The young astrophysicist in The Burst is convinced that she must be mistaken - that such leaps in understanding are not possible and could not come from her. The reality, as her advisors demonstrate, is that astronomers true are human - driven by human emotion and insight - and humans, whether the mundane creatures of the present or the superintelligences of The Final Now, are capable of remarkable things.

In the meantime, as was the case for dark matter, science fiction tests have recorded the evolution of ideas, and the drift of dark energy from the wildest of ideas and a philosophical conundrum through to a structured and widely-accepted framework on which our understanding of the Universe - and ourselves - can be hung.