(This post constitutes my contribution to the Carnival of Cosmology, which I am hosting at Galileo’s Pendulum. The Carnival will appear here tomorrow, so stay tuned for everyone else’s contributions! If you’re interested in joining in, you have until midnight tonight, US Eastern time, to submit a post to me.)

Cosmology has more than its share of big mysteries, in the form of two unsolved riddles: dark matter and dark energy. I’ve written a lot about both: see my posts “Is Cosmology in Shambles?” (answer: no) and “Observation of a Matter Bridge” for dark matter, and “Universe in a Box” for dark energy. The names “dark matter” and “dark energy” both are placeholders for our ignorance: while the observational evidence in favor of both is very strong, we don’t know exactly what either of these substances are. However, in many ways the evidence for dark energy is even stronger than for dark matter—even though dark energy itself seems more puzzling.

Review: What Do We Mean by “Dark Energy”?

Since the 1920s, we’ve had strong evidence that the Universe is expanding. If the Universe contained only matter (both ordinary and dark matter) and light, then its expansion would gradually slow down, but it doesn’t. In 1998, two groups of astronomers discovered that the rate of expansion is increasing, a discovery for which three of the project leaders—Saul Permutter, Adam Riess, and Brian Schmidt—received the 2011 Nobel Prize in physics. From my blog post commemorating the award:

If the expansion of the universe was decelerating, then there would be slightly more supernovas at smaller distances than at larger. However, the supernova search teams found that there were slightly more supernovas at larger distances, indicating that the rate of expansion is growing. The universe is accelerating, and nobody has a good answer as to why. The name given to quantify our ignorance is “dark energy”; observations of the cosmic microwave background show that it comprises 72% of the energy content of the universe, compared with about 5% for ordinary matter and 23% for dark matter.

(For more on the cosmic microwave background (CMB), see another earlier post. I’ll try to stop cross-linking soon!)

Dark energy wasn’t entirely unexpected: cosmologists including Steven Weinberg, Michael Turner, and Lawrence Krauss had been talking about restoring Einstein’s “cosmological constant” idea prior to 1998, but giving it the opposite property. While Einstein proposed an additional component to the Universe to prevent expansion or contraction, these other cosmologists argued from the basis of quantum theory that the energy of the vacuum (more on that shortly!) could drive expansion of the Universe. However, they recognized the simplest calculation was wrong, even if their general conclusions were probably right. Nevertheless, the two groups of astronomers who discovered cosmic acceleration observationally—the High-z Supernova Search team and the Supernova Cosmology Project—weren’t looking for acceleration, so it was a major surprise to them to find it. The first draft of one of the papers even included “deceleration” in the title!

This is what we know about dark energy: its general effect on the expansion of spacetime, and roughly how much of it there is compared to the other contents of the Universe. Now let’s consider what we don’t know….

Ignorance is Not Bliss

To understand why dark energy is weird, let’s think about how it differs from the more ordinary stuff in the Universe. As I mentioned earlier, a Universe containing only matter and radiation will slow its rate of expansion over time: as the Universe expands, the atoms, dark matter particles, and photons get more spread out, meaning the density of energy they supply to drive growth decreases. While this might sound like pressure is driving expansion, it’s not true: at the temperatures and quantities of matter particles present in an average chunk of the Universe, the pressure they contribute is effectively zero.

Dark energy, on the other hand, acts as though it has negative pressure. The figure on the right illustrates that concept: while ordinary gas pushes back when you try to compress it, dark energy pushes back less the more it is compressed. This explains why dark energy drives the acceleration of the Universe: the bigger the volume in which it’s contained, the more pressure—and energy—it supplies to expansion.
This picture is fairly clear, whatever substance dark energy turns out to be.

However, the easy view breaks down when we go a little beyond that. If you’ll forgive the inclusion of a simple equation, it might help us out:

where p is the pressure, d is the density (amount of stuff in a fixed volume), and w is known as the “equation of state”, though that’s more appropriate for the whole formula. For dark matter and ordinary matter under cosmological conditions, w is nearly zero; for light or particles (like neutrinos) moving close to the speed of light, w is 1/3. (In chemistry of physics class, you probably learned the ideal gas law; if we write d as the number of moles divided by volume, then w depends only on temperature. In other words, the ideal gas law is another version of the same equation, covering the case when temperatures and densities are relatively high.)

Dark energy, on the other hand, has a negative value for w, and a lot of effort is being expended to determine what value it should have. We have several possibilities:

• If dark energy is vacuum energy, then it’s the energy of empty space. That’s not as weird as it sounds: from quantum field theory, we know that spacetime is a cauldron of particles that only potentially exist. The properties of particles are determined through their interactions with this quantum vacuum, which is full of fields—including the Higgs field, which gives mass to many particles. (The Higgs boson is a manifestation of this field, if enough energy is present to make one.) This means the more vacuum there is, the more vacuum energy, and the more pressure. This is the simplest case, and it corresponds to a wvalue of -1.Since this is mathematically the same thing as Einstein’s original idea, vacuum energy is also called the cosmological constant, since its contribution to the density of energy is constant in both space and time; similarly, w is always -1. The problem with this simplest model is that nobody quite knows how to calculate the actual value of the cosmological constant; the most straightforward prediction from quantum field theory (first performed, as far as I know, by Steven Weinberg) leads to a number that’s far too large, compared with observation. Despite this major drawback, most cosmologists think this option is best, and I’m inclined to agree in the absence of better (simple) ideas.
• If dark energy is a fluid of a fundamentally different type than any other we know about, then w is likely to be between -1/3 and -1. This idea corresponds to a veritable plethora of possible models, and I admit I don’t know much about most of them. They go under names like quintessence, and they are significantly more complicated than the vacuum energy scenario. For example, w changes with time in quintessence models: its value today is not what it was in the past, and may be different again in the future. Our current observations can’t rule that possibility out completely, though the amount of change in w seems to be fairly small for the regime where we have good data.
• Maybe w is less than -1, an option known as phantom energy. Katie Mack’s contribution to the Carnival of Cosmology will cover that idea, but suffice to say it’s kind of a nightmare, both theoretically and in terms of what it means for the future of the Universe. Thankfully, it also seems to be ruled out by observational data.

Beyond these options, there are several others that obtain w and dark energy behavior through entirely other means.

• The cyclic universe model of Paul Steinhardt and Neil Turok describes our Universe as part of a coupled pair that interact. What we perceive as dark energy is a measure of the energy being pumped into our Universe as the bond between it and the shadow universe stretches. (I may be getting some of the details wrong; my apologies if that’s so.) I admit, I’m kind of fond of this scenario for its sheer creativity and the testable predictions it makes, but I don’t believe a word of it. I’m not sure what it predicts about the value(s) of w, either.
• Perhaps our beloved theory of gravity, general relativity, requires modification to fit cosmological observations. General relativity is very well tested on the scale of the Solar System and in strong-gravity cases (including binary neutron stars), but perhaps what we see as dark energy is a reflection of its breakdown on the very largest scales. A number of alternative gravitation theories are out there, many of which are fanciful or far more complicated than any well-tested theory we have. I also like this idea, though I haven’t found a particular alternative gravity model that satisfies both observational data and my own high standards. (Crackpots send me one of these about once a month as well, but there are plenty of reasonable people who work in this area too.)

The problem as usual when we get beyond our standard, accepted physics, we are in a situation akin to tabulating all non-mammalian animals. Our well-tested models are a subset of all possible ideas out there, and some of them may be valid, but the sheer number is prohibitive to cataloging. So, let’s leave the tabulation behind and focus on observations, which have driven our knowledge of dark energy up till now, and will likely continue to do so for a few more years.

What We Are Doing About Our Ignorance

As part of the research for my book-in-progress, Back Roads, Dark Skies, I traveled to Fermilab, where a new instrument known as the Dark Energy Camera (DECam) was assembled and tested. Unfortunately by the time of my visit, the camera had already gone—shipped to Cerro Tololo, where it has been installed on the very same telescope where dark energy was discovered. The camera is the main detector for the upcoming Dark Energy Survey (DES), which seeks to image over 300 million galaxies. By finding their distance from Earth (via their cosmic redshift), astronomers should be able to pin down what value w has—including whether its value changes in time. Comparing the observed galaxies to extensive computer simulations, we can rule out a large number of possible theories, and maybe isolate the most correct one.

DES is one of a number of similar projected cosmological surveys, showing yet again that the observers are ahead of us theorists on the race to characterize dark energy. I don’t begrudge them their lead: after all, for many years theorists could pretty much predict whatever they wanted, since the observers’ equipment wasn’t good enough to test their ideas. They’ve brought us back to reality quite well, which is a positive thing. However, observational data alone won’t tell us exactly what dark energy is, no matter how well it describes what it does, so we theorists need to keep working. The stakes are nothing less than an understanding of our Universe: its past, present, and future.