6
An independent and original thinker, Dirac never cared about the fashions of the physics community. He basically worked in fields that he found to be fundamental and interesting, unconcerned with whether they were judged to belong to mainstream physics. Or he simply invented new fields, as he did in the case of monopole and positron physics. Any idea of joining popular trends, or otherwise bending his ideas to adapt to majority views was totally foreign to him. It may have been in this connection that Bohr once remarked: “of all physicists, Dirac has the purest soul.” There was a price to pay for the purity, both scientifically and socially, but Dirac was willing to pay it.
Although Dirac was not blind to the value of experimental research—on one or two occasions he even engaged in experiments—he favored methods based on pure mathematical reasoning over the more standard empirical-inductive approach. The latter method became increasingly popular after World War II when advanced apparatus (in the form of accelerators and detectors, for example) stimulated a close integration of theory and experiment. Theoretical progress relied on and was guided ever more closely by new experimental data. On the organizational side, teams involving both theoretical and experimental physicists became the rule rather than the exception. Individual research was out, and teamwork was in.
In a talk he gave in 1972, Dirac referred with regret to the then-dominant method of physics, namely to “keep close to the experimental results, hear about all the latest information that the experimenters obtain and then proceed to set up a theory to account for them.” This approach, he continued, “might develop somewhat into a rat-race. Of course, it needs rather intelligent rats to take part in it.” As far as fundamental physics was concerned, Dirac much preferred to rely on mathematics and his own basic beliefs, without paying too much attention to experimental results. He explained: “It’s just that one feels that nature is constructed in a certain way and one hangs onto the idea rather like one might hang onto a religious belief.”
As Dirac did not follow the trends in method, he did not follow the trends in popularity among different subjects of physics either. At any given time, some subjects attract more students, money, and glamor than other subjects, and not necessarily for good reasons. In the first decades after the end of World War II, nuclear physics, elementary particle physics, and solid-state physics were by far the most popular and fast-growing areas of research. Dirac showed no interest in any of them. Instead, he had become fascinated with the electron and felt no need to consider the multitude of new particles discovered in the cosmic rays or in the ever more powerful high-energy accelerators. One or two elementary particles were enough for the ascetic Cambridge physicist.
Only on one occasion did Dirac consider another elementary particle, and then it was a close relative of the electron, the 207 times heavier muon. Discovered in 1936, the muon, originally called a “meson” or “mesotron,” is radioactive and decays spontaneously to an electron and a neutrino. In 1962, while revising his classical theory of the electron, Dirac suggested that instead of being a point-particle, “if one supposes the electron to have a finite size, … one can assume that the lowest excited state is the muon.” On this basis, he was able to calculate the mass of the muon to 53 times the electron’s mass, not an impressive agreement. The discrepancy did not worry Dirac much, as his model did not incorporate spin and thus could not be expected to correspond to a real particle. In any case, his theory of the finite-size electron and its cousin, the muon, was completely out of tune with the ideas of particle physicists at the time.
While the new generation of competitive particle physicists tended to dismiss Dirac as a cantankerous old man holding on to the past, he himself considered particle hunting as an unappealing enterprise. Dirac’s preference for cultivating subjects according to his own taste led him on several occasions, and especially after the mid-1930s, into areas that were far from mainstream physics. His work on cosmology and the constants of nature was definitely unconventional, and the same was the case with much of his research related to quantum theory. On the other hand, he never crossed the fine line between science and pseudo-science.
Contrary to most of his younger colleagues, Dirac could afford the luxury of working only on subjects that pleased him. Not only was he a highly reputed Nobel laureate with few students, but he was also a theorist in no need of collaboration or input from teams of experimenters. A leading expert in general relativity characterized him as “one of the very few scientists who could work even on a lonely island if he had a library and could perhaps even do without books and journals.” According to another contemporary physicist, “Dirac is a man who could never, between his great discoveries, do any sort of bread and butter problem.” This was not totally accurate, but it was close enough to the truth. Dirac’s view concerning mainstream physics and the methods of physics was original, but not exceptional. It was to a large extent shared by Einstein, who similarly placed himself on the sideline of mainstream science. Indeed, after about 1950, Dirac found himself in a position not unlike that of Einstein by challenging the conventional wisdom of quantum physics, both men isolated themselves. However, their objections to the state of affairs in fundamental physics were based on entirely different reasons. They had only one thing in common: they made almost no impact on the physics community.
At about the time when Dirac proposed his theory of the anti-electron, soon to become a theory of the positron, the foundation of quantum mechanics was a subject of worry to the physics community. In 1930, Bohr suggested to Dirac that “the solution to the present troubles will not be reached without a revision of our general physical ideas still deeper than that contemplated in the present quantum mechanics.” The troubles that Bohr referred to related to the attempts to establish a consistent quantum theory of electromagnetic interactions—a quantum field theory—in agreement with the theory of relativity. A few years later, Oppenheimer described the situation in quantum physics more pointedly: the existing theory was simply “in a hell of a way.” The responses to what was generally perceived as a crisis varied. Several prominent physicists, including Heisenberg, Pauli, and Dirac, believed that the problems could not be solved within existing theory but should be exploited in constructing a radically different theory of the future. Other physicists preferred a more conservative and pragmatic approach that would build on improvements of existing quantum theory.
To get an impression of the sense of crisis in the physics community during Dirac’s lifetime, we just need to consider the electron. Although described with admirable precision by Dirac’s equation of 1928, when the electron’s interaction with its own field was taken into account, it behaved strangely, almost perversely. Several problems turned up, the nastiest one being that the electron’s mass became infinite. Infinities are not necessarily bad news if they are of a mathematical nature only. But if they are concerned with real and measurable quantities such as the electron’s mass, which obviously cannot be infinite, then they are bad news. Dirac had created a large part of the foundation of quantum electrodynamics and quantum field theory and naturally felt committed to the development of these areas of fundamental physics. He was no less worried about these issues than Bohr and Oppenheimer. In a valiant attempt to remedy some of the problems, he published in 1934 a complicated field theory of electrons and positrons. The theory’s orgy of mathematical equations was impressive, but the infinities remained.
This was the beginning of Dirac’s gradual move into non-mainstream areas of physics as far as quantum mechanics was concerned. Of course, what is defined as mainstream is a sociological and not a scientific question. In the years after 1934, Dirac focused on finding better quantum equations, improving the mathematical basis of quantum mechanics, or by some other means solving the problems that haunted existing theory. For example, whereas his relativistic wave equation of 1928 only described electrons and other possible particles with half-integral spin, in 1936 he produced a generalization that was presumably valid also for other elementary particles.
Also, the same year, Dirac, who was concerned with the riddles of the quantum theory of fields and particles, suggested abandoning the whole “so-called quantum electrodynamics.” Without offering a proper alternative, he considered for a brief while to replace it with a theory in which energy was not strictly conserved. Although violation of energy conservation was restricted to atomic processes, it was a radical proposal. The law of absolute energy conservation is very fundamental. As a consequence of his temporal disbelief in energy conservation, he felt justified in questioning the existence of the neutrino, an elementary particle that Pauli had introduced to understand beta radioactivity without violating energy conservation. Although the neutrino was hypothetical (it was only detected in 1956), by 1936 almost all physicists accepted it as a real particle. But not Dirac, who was willing to sacrifice it along with the theory of quantum electrodynamics.
Dirac’s drastic and somewhat desperate proposal was broadly criticized, and within a year, he had returned to the safe ground of energy conservation. In the meantime, some physicists opposed to the Bohr-Heisenberg mainstream interpretation of quantum mechanics, prominent among them Einstein and Schrödinger, welcomed Dirac’s heterodoxy. They mistakenly thought it was a revolt against the Copenhagen view of quantum mechanics. “I am very happy that one of the real adepts now argues for the abandonment of the awful ‘quantum electrodynamics’,” Einstein commented. However, at the time, Dirac did not really question the Copenhagen view, and he expressed no interest at all in the alternative ideas of Einstein and Schrödinger. He would eventually change his mind, but only much later.
“I really spent my life trying to find better equations for quantum electrodynamics,” Dirac said in 1979, looking back on a long career in physics. Forty years earlier he thought that it might be worthwhile first to find better equations for the classical electron and then formulate the classical equations in terms of quantum mechanics. In this way, he hoped to get rid of the infinities. The result of this line of thinking was an important theory of the classical electron but not, unfortunately, one that could be successfully transformed into a quantum theory.
The classical electron theory was only one of several ideas that flowed from Dirac’s fertile mind. Another idea was to reconsider the concept of probability that lies at the heart of quantum mechanics. The probability that something happens, say, that a radioactive atom decays in the next minute, is a number between zero and one. But why not extend the meaning of probability to negative probabilities—numbers smaller than zero? Perhaps it sounds crazy, but in 1941 Dirac presented to the Royal Society a new theory of quantum mechanics based on this idea. The ever-critical Pauli, at first, found the theory promising, but eventually reached the conclusion that it had nothing to do with the real world of physics—it was, he said, a “mathematical fiction.” Neither this nor other theories brought Dirac closer to the goal, a mathematically consistent and physically sensible theory of quantum electrodynamics.
The goal was finally achieved in 1947 when two young American theorists, Richard Feynman and Julian Schwinger, developed their versions of a new formalism of quantum electrodynamics. A different and slightly earlier version of the same theory was independently proposed by Sin-Itiro Tomonaga in Japan. “Renormalization quantum electrodynamics” is a terrible name, but that is what the theory is called. In a nutshell, based on the existing framework of quantum mechanics and relativity theory, the two Americans developed schemes that allowed them to calculate measurable quantities (such as the mass of the electron) and obtain finite answers. The infinities did not really disappear in the new theory, but they were tamed in the sense that they did not show up in the final result. By means of a clever subtraction procedure, a finite number was obtained by subtracting an infinite number from another infinite number! The great advantage of the renormalization theory was that it worked. It was quickly used to make detailed calculations of experimental phenomena that hitherto had escaped explanation, and the calculations agreed impressively with data.
Dirac was not directly involved in the creation of what was heralded as a new paradigm in quantum theory, but indirectly he was—some of his early papers served as important sources of inspiration for the three founders of the paradigm. At any rate, having digested the theory, Dirac came to the conclusion that it could not possibly be correct. The hostile attitude towards renormalization quantum electrodynamics remained throughout the rest of his life. It turned him into an outsider in the science that he had done more than anyone else to create. While the new generation of quantum physicists celebrated the end of the more than decade-long war against the infinities and happily went on with their calculations, Dirac thought that the battle had been won using foul means. To him, it was not a genuine victory.
“What we need and shall strive after,” Dirac said in a talk he gave in 1949, “is a change in the fundamental concepts, analogous to the change in 1925 from Bohr to Heisenberg and Schrödinger, which will sweep away the present difficulties automatically.” He was too modest to mention his own role in this dramatic phase in the history of quantum physics. Dirac hoped to repeat the success of his youth when quantum mechanics had been discovered almost by accident. Relativity and quantum mechanics had been established through a series of revolutionary steps; Dirac was convinced that a new revolution was needed. The breakthrough of quantum electrodynamics in the late 1940s, on the other hand, was fundamentally conservative. It was not a radically new theory but, rather, the old theory dressed up in fancy new clothes.
At the bottom of Dirac’s resistance to post-war quantum theory were questions about the meaning and values of physics. The outlook of Schwinger, Feynman, and others of the new generation of quantum physicists was pragmatic and one-sidedly focused on getting answers from theory that agreed with experiment. Physics, they said, was a matter of calculation and comparison between calculated and experimental results—no more and no less. With the new U.S. style of physics, questions that related to interpretation or even worse—to philosophy, were avoided or denigrated. Conceptual scrutiny of the foundations of physics tended to be seen as an unnecessary luxury. This attitude to fundamental physics, still popular among many modern physicists, has been summarized in the sentence “shut up and calculate!”
Dirac was not a stranger to the “shut up and calculate” philosophy, which he shared to some extent in the early part of his career. Physical theory, he said, was basically a formal instrument that allowed the calculation of experimental results. This was also the implicit message of his great textbook in quantum mechanics. One reviewer of the book paraphrased, not unfairly, Dirac’s view as follows: “A mathematical machine is set up, and without asserting or believing that it is the same as Nature’s machine, we put in data at one end and take out results at the other. As long as these results tally with those of Nature … we regard the machine as a satisfactory theory.” As late as 1967, in the fourth edition of Principles of Quantum Mechanics, Dirac subscribed to a view of this kind, effectively limiting physics to mathematical manipulations of symbols related to observable quantities. “Only questions about the results of experiments have a real significance,” he wrote, “and it is only such questions that theoretical physics has to consider.”
However, Dirac’s attitude to the pragmatic “shut up and calculate” view changed under the impact of the disagreeable renormalization quantum theory. Along with several other physicists of the pre-war generation, Dirac disliked the new style of physics. In 1981, near the end of his life, Dirac delivered an address entitled “Does Renormalization Make Sense?” It was a rhetorical question. “Some physicists may be happy to have a set of working rules leading to results in agreement with observation,” he pointed out. “They may think that this is the goal of physics. But it is not enough. One wants to understand how Nature works.” To Feynman and his kindred souls, understanding could be reduced to empirically successful calculations. In another address from the same period, Dirac pointed out that even a wrong theory might produce results in agreement with observation. His favorite example was the kind of atomic theory he had met in his youth—Bohr’s orbital model of the atom. “The Bohr theory … gave very good answers, but still the Bohr theory had the wrong concepts,” Dirac noted. “Correspondingly, the renormalized kind of quantum theory with which physicists are working nowadays is not justifiable by agreement with experiments under certain conditions.”
Dirac admitted that renormalization quantum electrodynamics was highly successful in terms of agreement between theory and experiment, but he thought that the price for the success was much too high. Not only did the theory not lead to a proper understanding of nature, but it also built on working rules without a foundation in logic and what he considered to be “sensible mathematics.” According to Dirac, the infinities had not disappeared, they had been neglected. “Sensible mathematics involves neglecting a quantity when it is small—not neglecting it just because it is infinitely great and you do not want it!” Dirac was not alone in criticizing the new quantum electrodynamics, but one of the recurring elements in his uncompromising opposition was particular to him. This was the emphasis of the theory’s lack of aesthetical quality. Since the 1930s, Dirac had been increasingly concerned by the role of what he called “beautiful mathematics” in physics, and he could find no beauty at all in the new quantum electrodynamics. On the contrary, it was “complicated and ugly,” he said.
To most practicing physicists in the post-war period, the issue of consistency in quantum theory was just a pedantic problem they did not need to address. Nor did the generation of Schwinger and Feynman appreciate the aesthetic values that Dirac and his contemporaries associated with physics. By 1950, Dirac was no longer a young man. He may have recalled the Faust parody staged at Bohr’s Institute nearly two decades earlier, when he recited in German a passage translated into English:
Age is, of course, a fever chill
That every physicist must fear.
He’s better dead than living still,
When once he’s past his thirtieth year.
Or he may have recalled how he congratulated the slightly older Heisenberg on his 30th birthday: “You are now past 30 and you are no longer a physicist.”
During the last decade of his life, Dirac concentrated on two separate research projects, both of which were well outside mainstream science. One was aimed at a new foundation of quantum theory, and the other was his idea that the constant of gravitation varied in time. His attempts to improve quantum theory in general and quantum electrodynamics, in particular, took many directions, some more unorthodox than others. One of them was a proposal of reintroducing the ether, the medium that had played such a predominant role in Victorian physics but vanished from the scene with the acceptance of relativity theory. However, Dirac’s ether was different from the classical version as it was based on quantum mechanics and in agreement with the theory of relativity. It could not be ascribed a definite velocity and, for this reason, it did not justify the absolute space and time associated with the Newtonian world picture.
Nevertheless, Dirac considered his ether as physically real as his Victorian predecessors had done. He even speculated, much like some of the ether physicists of the fin-de-siècle period, that his ether might be “a very light and tenuous form of matter.” Dirac thought that the new quantum ether might serve as an ally in his continual fight against the fashions of current quantum theory, but nothing useful came out of the idea. While the press showed interest in it, the physicists did not.
Indeed, none of Dirac’s many alternatives were even remotely successful or attracted more than polite attention in the physics community. It seemed that the quantum wizard had lost the magic wand of his youth. As the eminent Russian theoretical physicist Lev Landau viciously quipped to a colleague in 1957, “Dirac is the greatest living physicist and he has done nothing of importance since 1930.”