3
With his fundamental papers published between 1925 and 1927, Dirac had proved himself a quantum wizard, a leading physicist of the quantum-mechanical revolution. However, although his contributions were invariably original and recognized to be highly significant, in almost all cases other European physicists obtained the same results—and in some cases, they even scooped him. Dirac felt that he still lived in Heisenberg’s shadow and had not yet produced a deep and really novel theory. The Nobel Committee in Stockholm even concluded that “Dirac is in the front rank of the group of researchers who have set themselves the task to realize Heisenberg’s bold thought,” but noted that “Dirac is a successor in relation to Heisenberg.”
The deep and really novel theory that he dreamed of came unexpectedly to him at the end of 1927. The eponymous Dirac equation is no less fundamental than the better-known Schrödinger equation, and its consequences even more amazing. The memorial stone for Dirac at Westminster Abbey reproduces the equation in a compact form familiar to modern physicists: iγ∂ψ = mψ. (Appropriately, the stone is placed near the tomb of Isaac Newton). To understand the origin and significance of this mysteriously-looking equation we need to go back to the spring of 1926.
When Schrödinger introduced his wave equation to the world of physics, he used it to calculate the energy levels of the hydrogen atom and derive the same spectral lines that Bohr had obtained from the old quantum theory in 1913. The hydrogen spectrum was a triumph of wave mechanics, but Schrödinger realized that the triumph was incomplete: the spectral lines were not sharp but consisted of narrowly spaced doublets, a phenomenon known as “fine structure.” The separation of the doublet lines, as given by the difference in frequency, is very small and can be expressed by a number known as the fine-structure constant. This number would play an important role in Dirac’s physics. Within the framework of the Bohr model, the fine structure could be accounted for by taking into regard the theory of relativity, but by 1926 Bohr’s orbital model was dead. Schrödinger was acutely aware that his equation did not satisfy the requirements of relativity and, probably, for this reason, was unable to explain hydrogen’s fine structure.
The Schrödinger wave equation contains terms that correspond to a particle’s energy E and its momentum p. According to relativity theory, energy is associated with time (t) while the momentum is associated with the space coordinates (x in one dimension). However, energy enters the quantum wave equation linearly (as E ~ t) and momentum enters as the square (as p2 ~ x2), which means that the equation is not symmetrical in time and space. To Schrödinger’s despair, when he modified the equation according to the space-time symmetry requirement, it was still unable to reproduce the correct fine structure spectrum. There was something wrong, but he could not figure out what it was.
Nonetheless, in the summer of 1926 Schrödinger published the relativistic version of his wave equation and so did half a dozen other physicists. Pauli called it “the equation with the many fathers.” Because two of the “fathers” were Oskar Klein and Walter Gordon, a Swedish and German physicist respectively, it became known as the Klein-Gordon equation. It was a nice but apparently useless formula. The problem with relativity was not exclusive to wave mechanics for it also appeared in matrix mechanics and Dirac’s q-number algebra. After all, the three formulations were just different versions of the same theory, quantum mechanics.
There was another and possibly related problem. To characterize the behavior of an electron, it must be ascribed a “spin” quantum number that can attain only two values, + ½ or − ½. In the classical picture, it corresponds to a spherical charge that rotates around its axis in either of the two directions. Spin was discovered in the summer of 1925 and initially seemed foreign to the new quantum mechanics. It could be grafted upon the quantum equations but not derived from them. Physicists vaguely realized that spin and relativity were somewhat connected, but nobody could tell in what way or how they fit into the formalism of quantum mechanics. Attempts to make them fit were ad hoc and did not appeal to Dirac at all. He was slowly getting interested in the spin puzzle, and in December of 1926, when he stayed with Heisenberg in Copenhagen, the two physicists made a bet of when spin would be properly explained. Dirac thought three months, Heisenberg, at least, three years. They were both wrong, but Heisenberg more than Dirac. (The process took one year).
A little more than three months after the bet, Pauli came up with a quantum-mechanical theory of spin, if not a proper explanation. His idea was to extend Schrödinger’s wave function ψ from one to two components. An electron would then be characterized by (ψ1, ψ2) where the two functions represented the electron’s two spin states by means of new variables. The variables were 2 × 2 “Pauli matrices” with two rows and two columns. Pauli’s theory made sense of the spin within the framework of ordinary quantum mechanics, but since it was not relativistic, Dirac considered it to be merely a provisional solution. The easy way would be to integrate spin and the Klein-Gordon equation, but this turned out not to be possible. It was Dirac who solved the problem and perhaps won the bet with Heisenberg. However, the solution came in a roundabout way. “I was not interested in bringing the spin of the electron into the wave equation,” he recalled. “It was a great surprise to me when I later discovered that the simplest possible case did involve the spin.” Dirac’s discovery was an example of what is called serendipity, the almost accidental discovery of something the scientist is not looking for.
Having no faith in the Klein-Gordon equation, at the end of 1927 Dirac decided to find a better solution for a wave equation in accordance with the theory of relativity. Based on the relativity requirement and the general structure of quantum mechanics, he was convinced that the equation he looked for must be linear not only in energy, as in the ordinary Schrödinger equation, but also in momentum. This conviction brought him face to face with a purely mathematical problem, namely how to write the square root of a sum of four squares as a linear combination. Take the square root of (a2 + b2 + c2 + d2) and try to write it as n1a + n2b + n3c + n4d, where the n’s are some coefficients. That’s not an easy problem, but Dirac needed to solve it. Guided by Pauli’s spin matrices and his own mathematical intuition, he realized that the trick could be done if the n-coefficients were 4 × 4 matrices, that is, quantities with four rows and columns comprising 16 numbers. With these “Dirac matrices,” he could straightforwardly write down the new wave equation for a free electron. As a consequence of the 4 × 4 matrices, the wave function had not only two components, as in Pauli’s theory, but four: ψ = (ψ1, ψ2, ψ3, ψ4).
The crucial step in Dirac’s derivation was the reduction of a physical problem to a mathematical one. The method was characteristic of his style of physics. “A great deal of my work is just playing with equations and seeing what they give,” Dirac said in a 1963 interview. “I think it’s a peculiarity of myself that I like to play about with equations, just looking for beautiful relations which maybe can’t have any physical meaning at all. Sometimes they do.” In this case, they did.
And what about the spin? Dirac had originally ignored the problem by considering, for reasons of simplicity, an electron without spin, knowing that such a particle did not exist. But when he extended his wave equation of the free electron to one where the electron interacted with an electromagnetic field, he discovered that the correct spin appeared, almost mysteriously. Without introducing the spin in advance, Dirac was able to deduce the electron’s spin from the first principles upon which his equation was built. In a certain if somewhat unhistorical sense, had spin not been discovered experimentally, it would have turned up deductively in Dirac’s theory. This was a great and unexpected triumph. It was less unexpected, but still very satisfactory that the equation could account in detail for the fine structure of the hydrogen spectrum.
Dirac submitted his paper on the new electron theory on the first day of 1928, and it appeared in print a month later. It came as a bombshell to the physics community. Dirac had worked alone, almost secretly, not for reasons of priority but because it was his habit. His senior colleague Charles Darwin, a professor at the University of Edinburgh and a grandson of the “real Darwin,” was one of the few who knew in advance what was going on. In a letter to Bohr of December 26, 1927, he wrote: “I was at Cambridge a few days ago and saw Dirac. He has now got a completely new system of equations for the electron, which does the spin right in all cases and seems to be ‘the thing.’ His equations are first order, not second, differential equations!” Dirac’s equation was indeed the thing. It was received with equal measures of surprise and enthusiasm—even by the physicists who had come close to solving the problem themselves. Among them was Jordan, who, in this case, lost to Dirac. A physicist who at the time worked in Göttingen recalled: “The general feeling was that Dirac had had more than he deserved! Doing physics in that way was not done! … [The Dirac equation] was immediately seen as the solution. It was regarded really as an absolute wonder.”
The new relativistic theory of the electron had a revolutionary effect on quantum physics, both pure and applied. It was as if it had a life of its own, full of surprises and subtleties undreamed of even by Dirac when he worked it out. For example, the 4 × 4 Dirac matrices attracted much interest among the pure mathematicians who eagerly studied the properties of the matrices and other mathematical objects related to them. This branch of mathematical physics eventually developed into a minor industry and is still an active field of research. Right after Dirac’s theory appeared, it seemed that although it had great explanatory power, there was no particular predictive power. It explained the electron’s spin and hydrogen’s fine structure most beautifully, but no predictions of novel phenomena followed from it. This situation soon changed. Within a few years, Dirac’s equation and the electron theory based on it proved successful over a wide range of physical areas, including high-energy scattering processes and the mysterious cosmic rays. Most dramatically, it led to the successful prediction of a new class of elementary particles, which Dirac called antiparticles.
The revolutionary concept of antimatter or antiparticles had its roots in conceptual problems of the Dirac equation noted at an early date. In the four-component wave function (ψ1, ψ2, ψ3, ψ4) two of the components, say ψ1 and ψ2, refer to the two spin states of an ordinary electron, just as in Pauli’s theory. What do ψ3 and ψ4 refer to? They enter because Dirac’s theory is a quantum translation of the energy-momentum relation in relativity theory, which involves the squares of energy and momentum. But Dirac’s equation, following the basic rules of quantum mechanics, was linear in energy. It involved E and not E2. Now, when you take the square root of mc2 you don’t get just the energy E but also the negative quantity − E. After all, (− E)2 = E2. Formally ψ3 and ψ4 refer to the two spin states of a negatively charged electron with negative energy. We can avoid negative-energy electrons, but in that case, the two wave functions need to be interpreted as belonging to electrons of positive charge. As Dirac pointed out in 1930, “an electron with negative energy moves in an external field as though it carries a positive charge.”
Both interpretations invited trouble. Take the hypothetical negative-energy (and negative-charge) electrons. Since energy is equivalent to mass by E = mc2, the mass of a negative-energy electron at rest will also be negative. It can easily be shown that the energy of such a particle, when set in motion, will decrease the faster it moves; moreover, when a force is impressed on the particle, it will accelerate in the opposite direction of the force. Absurd! The Russian physicist George Gamow had first met Dirac in 1927, and the two established a lasting friendship. Gamow called the negative-energy electrons formally occurring in Dirac’s theory “donkey electrons”—the harder they were pushed, the slower they would move. From the standpoint of physics, by 1930 the positive-electron interpretation was no better, for the charge of electrons was known to be negative. Earlier speculations about positive electrons had turned out to be just that—speculations. The only known positive particle—or “positive electron”—was the proton, which is nearly 2,000 times heavier than an electron. According to the paradigm governing physics at the time, there were no other massive elementary particles than the electron and the proton (the photon has no mass). The dilemma was this: either one had to postulate electrons with negative energy that behaved absurdly or the existence of a particle for which there was not the slightest empirical evidence. As a third possibility, one might have discarded Dirac’s theory in toto, but this was an option no one seriously considered. There was no way back.
The intertwined problems of the energy and charge of Dirac’s electron were much debated in the physics community. Heisenberg, Jordan, and others were deeply worried about what they called the “± difficulty.” For example, in 1928 Jordan referred to “the murky problem of the asymmetry of the forms of electricity, that is, the inequality of mass for positive and negative electrons.” In relation to this problem, he judged that Dirac’s “theory is entangled in temporarily insoluble difficulties.” Heisenberg concurred: “The saddest chapter of modern physics is and remains the Dirac theory.” Dirac was worried too, but in the late autumn of 1929, he thought to have found an answer to the ± difficulty. His hypothesis was as original as it was audacious—vintage Dirac. He first reported it in letters to Bohr, through which it became known to other physicists, and publicly announced his “Theory of Electrons and Protons” in a paper written in early 1930.
Dirac realized that particles with negative energy could have no reality in nature, and he consequently focused on the ± difficulty in the sense of the hypothetical positive electrons. This is not to say that he just ignored the negative energy problem, but he made the negative-energy electrons invisible, so to speak. Dirac boldly assumed a world of negative-energy states uniformly occupied by an infinite number of electrons with energies extending from – mc2 to minus infinity. If this distribution of negative-energy electrons in the “Dirac sea” were exactly uniform, they would be unobservable, merely serving to define a state of normal electrification. Dirac identified it with the vacuum. As he wrote to Bohr, “On my new theory the state of negative energy has a physical meaning, but the electron cannot jump down into it because it is already occupied.” This was a result of Pauli’s fundamental exclusion principle, according to which two electrons cannot occupy the same quantum state. Pauli suggested the principle in 1925 to explain the periodic system of the chemical elements, and in 1945, he was belatedly awarded the Nobel Prize for his discovery.
Had Dirac just postulated his world to be filled up with negative-energy electrons and governed by the Pauli principle, it would have been nothing but a metaphysical idea: it would have had no measurable consequences. But he further assumed that a few of the negative-energy states might be unoccupied, and in this case, it would be possible for a real, positive-energy electron to “jump” into one of the unoccupied states, and fill it. The vacant states or “holes” would appear as real physical entities: “Only the small departures from exact uniformity, brought about by some of the negative-energy states being unoccupied, can we hope to observe,” he noted. “These holes will be things of positive energy and will therefore be in this respect like ordinary particles.”
The idea of holes enabled Dirac to account for the negative-energy solutions that formally turned up in his theory without having to introduce observable negative-energy particles. But what were these holes or things? Dirac thought at first that the holes were protons—or, for that matter, that electrons were holes in a negative-energy proton sea. The two particles were basically the same, the proton being an electron in disguise and the electron a proton in disguise. “We are,” he wrote in 1930, “led to the assumption that the holes in the distribution of negative-energy electrons are the protons.”
By identifying the proton with a vacant negative-energy state, he had effectively reduced all elementary particles known at the time to just one fundamental entity, the electron. This was a unifying idea that appealed greatly to Dirac, as he made clear in an address he gave in September of 1930. It has always been “the dream of philosophers” to explain all matter in terms of just one entity, Dirac said, boldly suggesting that it was no longer just a dream. “There are … reasons for believing that the electron and proton are really not independent, but are just two manifestations of one elementary kind of particle.” He ended Principles of Quantum Mechanics, published a few months earlier, with the same dramatic message.
Dirac was almost alone in defending the electron-proton hypothesis, which most of his colleagues in physics considered wildly speculative and unreasonable. While some criticized it on technical grounds, others simply wrote it off as nonsense. The grand and beautiful hypothesis was not nonsense, but it was wrong: it faced two serious empirical problems, which neither Dirac nor other physicists were able to escape.
According to the hypothesis, one would expect that an ordinary electron would occasionally make a quantum transition to fill a hole. In that case, it would annihilate together with the hole, or proton, and the mass of the two particles would transform into radiation energy in accordance with Einstein’s E =mc2 formula. Annihilation processes of this kind, e– + p+→ radiation, had been considered by several astronomers as a possible source of stellar energy, but they had never been observed in either nature or the laboratory. Dirac was well aware that the process was without experimental evidence, and yet he optimistically stated that “There seems to be no reason why such processes should not actually occur somewhere in the world.” The lack of evidence for electron-proton annihilation indicated that the frequency of such processes was exceedingly small, and perhaps zero. Unfortunately, when physicists calculated the probability of proton-electron annihilation on the basis of Dirac’s theory, they came up with large and not small numbers. According to Oppenheimer, if Dirac were right, then all matter would long ago have disappeared and left a universe filled entirely by radiation.
The other problem was more obvious, namely that the proton is much heavier than the electron. Heisenberg had pointed out as early as December of 1929 that in Dirac’s theory the two particles—the electron and the hole/proton—must have the same mass; his objection was confirmed by calculations made by other physicists. Dirac tried for a while to explain the embarrassing mass difference, but in the end, he was forced to conclude that his unitary hypothesis was wrong. He did it with regret, especially because he found the hypothesis very attractive.
The capitulation did not mean that Dirac abandoned his imagery of an infinitely deep sea filled (or almost filled) with negative-energy electrons. He just revised it, offering a new candidate for the holes. He did this in a remarkable paper of June 1931. The groundbreaking idea of antiparticles was only dealt with casually, for the main subject of the paper was the possible existence of magnetic monopoles. After admitting that the earlier electron-proton idea was no longer tenable, Dirac wrote: “A hole, if there were one, would be a new kind of particle, unknown to experimental physics, having the same mass and opposite charge to an electron. We may call such a particle an anti-electron.” He further suggested that antielectrons were probably very rare in nature and might never be found, but “if they could be produced experimentally in high vacuum they would be quite stable and amenable to observation.” There is no doubt that Dirac considered the antielectron to be a real particle, as he made clear in a talk he gave at Princeton in the fall of 1931, emphasizing that anti-electrons “are not to be considered as a mathematical fiction.” As a means of detection, he introduced the notion of what has become known as pair production, the formation of a particle-antiparticle pair out of radiation energy.
Within the space of one page, Dirac predicted two new elementary particles in addition to the two already known empirically. The proton was no longer the antiparticle of the electron and therefore supposedly had its own mirror particle: “Presumably the protons will have their own negative-energy states, all of which normally are occupied, an unoccupied one appearing as an anti-proton.” That was all, but in some of his later papers, Dirac elaborated on the idea of matter made up of antiparticles, by which he meant antielectrons and antiprotons. Although the neutron was discovered in 1932, he never mentioned the antineutron (which has the same zero charge as the neutron, but differs from it in other respects). Dirac, the paragon of a rational physicist, was not a stranger to speculation, as illustrated in his 1933 Nobel lecture. On this occasion, he suggested that it might be a coincidence that our solar system consists almost solely of ordinary protons and electrons. “It is quite probable,” he said, “that for some of the stars it is the other way about, these stars being built up mainly of positrons and negative protons. In fact, there may be half the stars of each kind. The two kinds of stars would both show the same spectra, and there would be no way of distinguishing them by present astronomical methods.” Dirac stopped short of speculating about anti-humans.
Many physicists considered Dirac’s prediction of a positive electron to be as poorly justified as his earlier proton hypothesis. Both ideas were based on the theory of holes, which was widely regarded with skepticism or as merely a provisional picture. Bohr and Pauli sided against the hole theory and its idea of antiparticles as vacancies in a sea of negative energy states. Their dislike persisted even after the positive electron was discovered. After all, even though Dirac had predicted a positive electron from the hole theory, it did not mean that the discovery of the particle proved the theory right. There were other ways through which the new particle could be better justified.
In April of 1932, Bohr’s institute in Copenhagen celebrated its 10th anniversary in the presence of a large number of quantum physicists, including Pauli, Dirac, Ehrenfest, Gamow, Klein, Darwin, and Bohr himself. The participants staged a parody of Goethe’s Faust, adapted to the situation in theoretical physics as it was at that time. Part of the memorable play dealt with Dirac’s hole theory and the criticism of it. Dirac’s part was as follows:
That donkey-electrons should wander
Quite aimless through space, is a slander,
That only with articles
On hole-like particles
Could be said to have found a defender.
To which Bohr replied:
But the point of the fact is remaining
That we cannot refrain from complaining,
That such a caprice
Will reveal the malice
Of devouring the world it’s sustaining.
Physics textbooks often portray the actual discovery of the positive electron as a classical example of the fruitful interplay between theory and experiment. A theory predicts a novel object or phenomenon in nature, which is subsequently investigated experimentally and actually found to exist. As theory guides the discovery, so the discovery confirms the theory. The authentic discovery history of the positive electron is more complex and less morally suited for textbooks. The brief version is that in 1932, Carl Anderson, a California experimental physicist, found tracks in cloud chambers in the upper atmosphere that he interpreted as positive particles in the cosmic rays. He did not originally suggest they were positive electrons, which he only postulated in a paper the following year, where he also proposed the convenient name “positron” for the new particle (and the logically reasonable but less successful “negatron” for the ordinary electron).
Anderson discovered the positron in 1933 but not Dirac’s antielectron. Many years later, Anderson remarked: “Despite the fact that Dirac’s relativistic theory of the electron was an adequate theory of the positron, and despite the fact that the existence of this theory was well known to nearly all physicists, it played no part whatsoever in the discovery of the positron.” It was only after further cosmic rays experiments carried out at about the same time by the London physicist Patrick Blackett and his Italian collaborator Giuseppe Occhialini that it was realized that Anderson’s positron was Dirac’s antielectron. Not only was Blackett aware of Dirac’s theory, but while working in Cambridge, he also discussed his experiments and their theoretical significance with Dirac. In their 1933 paper, Blackett and Occhialini referred explicitly to the antielectron, suggesting that the positive electron supported Dirac’s theory. Anderson and Blackett would both be awarded a Nobel Prize for their work on cosmic rays particles, the first in 1936 and the latter in 1948.
After 1933, the fundamental idea of antiparticles was generally accepted, especially after it was shown that antiparticles could be accounted for without relying on Dirac’s somewhat fanciful picture of a sea of unobservable negative-energy particles. With or without this picture, the prediction of antiparticles was considered a marvel. Heisenberg once called it “perhaps the biggest jump of all big jumps in physics of our century.” This was an apt description, for a jump it was. With the benefit of hindsight, Dirac later said that the idea of antimatter “really follows directly from Einstein’s special theory of relativity when it is combined with the quantum mechanics of Heisenberg.” But he downplayed his own genius and the creative element in the discovery of antimatter. If it had been as simple as he said, antimatter would have been discovered by one of the dozen physicists who sought a relativistic version of quantum mechanics before Dirac, perhaps by Schrödinger, Klein, or Pauli. But it wasn’t. To an almost unparalleled extent in the history of science, antimatter was due to the creative mind of just one scientist.
For a long time, antiparticles meant positrons, whereas the antiproton (or negative proton) was rarely discussed. But with the entrance of still more powerful accelerators into the elementary particle physics, the antiproton also turned out to be real. It was first detected in high-energy experiments in 1955, and today antiprotons are produced routinely and in large numbers. The antineutron followed in 1956. As Dirac had speculated just over two decades earlier, matter consisting purely of antiparticles might actually exist somewhere in the universe. Whether or not cosmic antimatter exists, since 1996 the simplest form of antimatter has been produced in the laboratory. The short-lived anti-hydrogen atom is made up of a positron revolving round an antiproton. Anti-helium nuclei (but not atoms) have also been detected.
By 1950, it was recognized that the particle-antiparticle symmetry is a fundamental law of nature. We call the positron the antiparticle of the electron, but we might just as well have called the electron the antiparticle of the positron. Matter has no priority over antimatter, except that our universe has much more matter than antimatter. The perfect symmetry between the two kinds of matter remained a mystery for a long time. Could Dirac’s speculation of 1933 be more than just a wild idea? More than 30 years later, there had been attempts to establish a “plasma cosmology” on the assumption that the universe originally consisted of equal amounts of matter and antimatter, the Big Bang being a gigantic explosion of matter annihilating with antimatter. Plasma cosmology, still cultivated by a few physicists, attracted only limited scientific interest and Dirac never mentioned the hypothesis.
With the recognition of the hot Big Bang model in the 1960s, the challenge no longer consisted of explaining why there was so little antimatter, but in elucidating why there was some at all. If antimatter were as abundant as matter in the very early and dense universe, almost all antiparticles would have annihilated with particles, leaving a universe consisting of photons and only insignificant traces of matter. Obviously, this “annihilation catastrophe” did not occur—if it had, we would not exist. How could an initially symmetrical universe have evolved into the world like the one we know? Only at the time of Dirac’s death in 1984 did physicists believe to have found an answer. It was vaguely related to another of Dirac’s ideas, namely that the laws of physics are not static but have evolved over time. The kind of “grand unified” law that governed the very early universe allowed a slight preponderance of matter over antimatter, and thereby, prevented the annihilation catastrophe. This explanation essentially solved the mystery.
The antielectron was originally a pretty wild idea based on an abstract theory of quantum mechanics, and the positron was an elusive and exotic particle of interest only to pure physics. Today, many hospitals are equipped with PET scanners, an acronym for positron-electron tomography. The patient is injected with a radioactive isotope, which emits positrons, and the positrons annihilate with the electrons in the tissue. As a result of the annihilation, a pair of well-defined photons is emitted. By registering the photonic signals, a computer image of some inner part of the patient is produced. The first versions of PET scanning technology were developed when Dirac was still alive. Had he known about the technology (which he probably did not), he might have been pleased to see how his old prediction had morphed into a useful medical imaging instrument. But it is more likely that he wouldn’t have cared—medical imaging by means of positrons is not fundamental physics.