Back in 1998, two groups of astrophysicists studying supernovae made one of the most important experimental discoveries of the 20th century: They found that empty space, vacuum, is not entirely empty. Each cubic centimeter of empty space contains about 10-29 grams of invisible matter, or, equivalently, vacuum energy. This is almost nothing, 29 orders of magnitude smaller than the mass of matter in a cubic centimeter of water, 5 orders of magnitude smaller than the proton mass. If the whole Earth would be made of such matter, it would weight less than a gram.
If vacuum energy is so small, how do we even know that it is there? Just try to put 10-29 grams on a most sensitive scale, at it will show nothing at all. At first, many people were skeptical about it, but then combined efforts of cosmologists who studied cosmic microwave background radiation and large scale structure of the universe not only confirmed this discovery, but allowed to measure energy density of vacuum with a few percent accuracy. Doubts and disbelief were replaced by acceptance, enthusiasm, and, finally, by the Nobel Prizes received in 2011 by Saul Perlmutter, Brian Schmidt and Adam Riess.
The news has shaken physicists all over the word. But is it much ado about nothing? If something is so hard to find, maybe it is irrelevant and not newsworthy?
Vacuum energy is extremely small indeed, but in fact it is comparable to the average energy density of normal matter in the universe. Prior to this discovery, astronomers believed that density of matter constituted only 30 percent of the density corresponding to a flat universe. This would mean that the universe is open, contrary to the prediction of the inflationary theory of the origin of the universe. The unexpected discovery of vacuum energy added the required 70 percent to the sum, thus confirming one of the most important predictions of inflationary cosmology.
The tiny vacuum energy is large enough to make our universe slowly accelerate. It will take more about 10 billion years for the universe to double in size, but if this expansion continues, in about 150 billion years all distant galaxies will run away from our galaxy so far that they will forever disappear from our view. That was quite a change from our previous expectations that in the future we are going to see more and more…
The possibility that vacuum may have energy was discussed almost a century ago by Einstein, but then he discarded the idea. Particle physicists re-introduced it again, but their best estimates of the vacuum energy density were way too large to be true. For a long time they were trying to find a theory explaining why vacuum energy must be zero, but all such attempts failed. Explaining why it is not zero, but incredibly, excruciatingly small, is a much greater challenge.
And there is an additional problem: At present, vacuum energy is comparable with the average energy density of matter in the universe. In the past, the universe was small, and vacuum energy was negligibly small compared to the energy density of normal matter. In the future, the universe will grow and density of normal matter will become exponentially small. Why do we live exactly at the time when the energy of empty space is comparable to the energy of normal matter?
Thirty years ago, well before the discovery of the energy of nothing, Steven Weinberg and several other scientists started to argue that observing a small value of the vacuum energy would not be too surprising: A universe with a large negative vacuum energy would collapse before life could have any chance to emerge, whereas a large positive vacuum energy would not allow galaxies to form. Thus we can only live in a universe with a sufficiently small absolute value of vacuum energy. But this anthropic argument by itself was not quite sufficient. We used to think that all parameters of the theory of fundamental interactions, such as vacuum energy, are just numbers, which are given to us and cannot change. That is why the vacuum energy was also called the cosmological constant. But if the vacuum energy is a true constant that cannot change, anthropic considerations cannot help.
The only presently known way to solve this problem was found in the context of the theory of inflationary multiverse and string theory landscape, which claims that our universe consists of many parts with different properties and with different values of vacuum energy. We can live only in those parts of the universe where the vacuum energy is small enough, which explains why the vacuum energy is so small in the part of the world where we live.
Some people are critical to this way of thinking, but during the 18 years since the discovery of the vacuum energy nobody came with a convincing alternative solution of this problem. Many others are very excited, including Steven Weinberg, who exclaimed: "Now we may be at a new turning point, a radical change in what we accept as a legitimate foundation for a physical theory."
This explanation of a small vacuum energy has an unexpected twist to it: According to this scenario, all vacua of our type are not stable, but metastable. This means that, in a distant future, our vacuum is going to decay, destroying life as we know it in our part of the universe, while recreating it over and over again it other parts of the world.
It is too early to say whether these conclusions are here to stay, or they will experience significant modifications in the future. In any case, it is amazing that the news of a seemingly inconsequential discovery of an incredibly small energy of empty space may have groundbreaking consequences for cosmology, string theory, scientific methodology, and even for our views of the ultimate fate of the universe.