Heads or Tails


IF you put a kettle on a nice hot fire, will the water freeze? ‘Certainly not,’ says common sense, indignantly. ‘Probably not,’ says physics, hesitantly. According to physics, if every member of the human race put a kettle on the fire every day for the next million million years (during which, according to Jeans, the world is to remain habitable), it is not unlikely that sooner or later the water in one of these kettles would freeze instead of boiling. Unfortunately it is impossible to know in advance when this is going to happen, otherwise the man to whom it happened could get himself revered as a magician.

The kettle that freezes when put on the fire is only one of many kinds of occurrence which in practice we regard as impossible, but which in theory are only enormously improbable. If you let a drop of ink fall into a glass of water, the ink will diffuse itself throughout the water, discoloring the whole; it is theoretically possible that after doing this the ink should collect itself again into a drop, but any man who thought he saw this happening would conclude that his eyes had played him a trick. Eddington, in his book. The Nature of the Physical World, gives many examples of such improbabilities, which are nevertheless not theoretically impossible. For example, suppose you have a vessel divided into two portions with a trapdoor between the two: one portion is filled with air, the other is a vacuum; you open the trapdoor and the air streams into the portion that was previously empty; if the trapdoor is left open, it is theoretically possible that at some future moment all the molecules of air will have collected themselves again into the first compartment, leaving the second empty. It is also theoretically possible that an army of monkeys strumming on typewriters may accidentally produce all the books in the British Museum, but Eddington points out that this is less unlikely than the reassembling of the air in the one compartment.

Are we to ignore altogether such wild improbabilities? Dare we treat them in theoretical physics as we do in practical life, and assume that they will never happen? In old-fashioned physics it was thought that if we were sufficiently clever we could calculate everything and deal only in mathematical certainties. It cannot be said that this view is now known to be false, but there is an increasing tendency to throw doubt upon it and to hold that laws which are merely concerned with probability have to be brought in to supplement the laws that are concerned with certainty.

The chief of these laws concerned with probabilities goes by the somewhat imposing title of ‘The Second Law of Thermodynamics.’ This is nothing like so alarming an affair as it sounds; it states, broadly speaking, that the universe is becoming gradually less and less improbable. To take again the drop of ink in the glass of water, it seems improbable that all the ink should be in one place and all the water in another — such a state of affairs, we feel, requires an explanation; but if the ink is uniformly diffused throughout the water, that seems only natural, and we do not look with the same conviction for some reason that it should be so. We may say, in this sense, that the universe grows every day less surprising. It is like a pack of cards, which comes from the makers arranged in proper order, but after being in use for some time shows no trace of its original arrangement. This again is only probable; it might happen that by casual shuffling of a pack of cards you got it back into its original order, but you would require a very unusual reputation for probity before you could get anybody to believe that this had happened. Modern physics suggests that the world is getting gradually more and more completely shuffled, so that the traces of the original order are gradually disappearing.

Imagine some tidy old gentleman’s library, with all the volumes beautifully classified according to subject and size; imagine that in his absence children get into the library and throw the books at each other’s heads, while next morning the housemaid puts them back into the shelves higgledy-piggledy; gradually the traces of orderly arrangement disappear, and the books are found haphazard throughout the shelves.

This is the sort of thing that physics says is happening to the universe. For some reason, which I do not fully understand, this theory is welcomed by theologians as evidence of the divine government of the world. It is arguable that the theory affords evidence of a creation, but it affords the very reverse of evidence that any design has guided the subsequent course of affairs. I am by no means persuaded that the theory as it stands is valid; I think that both theologians and their opponents will be ill-advised if they treat it as the last word in theoretical physics.

Physics is a subject which has been changing with extraordinary rapidity in recent years, and there is no sort of ground for supposing that it has arrived at a stable phase. One reason for thinking this is that the concept of probability is wrapped in obscurity and affords indeed the chief scandal of modern logic. Nobody knows what is meant by saying that an event is improbable; nobody knows in what circumstances we are justified in assuming that an improbable event will not happen. At every moment the most wildly improbable events occur. Why do we accept some of these quite calmly, while others cause amazement? Nobody really knows. I have no satisfactory theory to offer, and the extent of what I have to say in the present article is to make people a little wary of all theories that depend upon this rather hazy notion of probability.


But let us first consider a little more in detail what probability in physics means in practice. If you have a family of two, they may consist of a boy and a girl, or of two boys, or of two girls; it is twice as probable that you will have a boy and a girl as it is that you will have two boys, and twice as probable as it is that you will have two girls. In this case probability has, or may have, a very simple meaning. If you were to enumerate all the families of two children existing in the world, you would find that about half consist of a boy and a girl, while about a quarter consist of two boys and about a quarter consist of two girls. At any rate, we may suppose that this is the case for purposes of illustration. The second law of thermodynamics would suggest that originally all families had consisted either only of boys or only of girls, but that gradually families evenly divided between boys and girls had come more and more to preponderate. This particular illustration, if taken literally, would, of course, be misleading, but it suffices to show the sort of thing that is meant.

In the sun there is a great deal of energy, while in interstellar space there is very little. The second law of thermodynamics suggests that gradually there will be less and less difference between the amount of energy in one place and the amount of energy in another, so that the universe will become more and more homogeneous and democratic. Everything pleasant is associated with some transition toward democracy, and becomes impossible when democracy has been completely achieved.

Life, both animal and vegetable, is an incident in this transition, since it consists in the utilization of solar energy: first in the building up of those chemical compounds that are essential to life, and ultimately in the dissipation of their energy throughout the universe in the form of heat. The warmth of the human body is derived from the energy of the sun; it warms the surrounding air, and the warmth of the surrounding air is dissipated by radiation. Thus the warmth of the sun, after keeping us alive for a little while, becomes diffused throughout space in a form which is no longer available for any useful purpose. This train of thought is to be commended to those who imagine that the evolution of life is the purpose of the universe. If modern physics is to be believed, the purpose of the universe is to make the emptiest and coldest portions of the universe slightly less cold, though not warm enough to be suitable even for an Eskimo.

This train of thought rests on probability, and probability rests on muddle.

I do not mean to deny that all this will probably happen; I only mean to assert that people have not the least idea what they mean when they say so.

Before we plunge into philosophy, let us consider the application of probability to daily life. At every moment, as I remarked before, the most wildly improbable things are happening. Let us take some illustrations. Suppose you hire a taxi, and suppose its number is M-Z.102348. What could be more improbable than that you should have hit just upon this number? You were bound to hit upon some number, but the odds were so strongly against the number which in fact you did hit upon that any sensible man would have dismissed this occurrence as practically impossible. And yet it occurred. Or again, take a simple question like your height. If you are asked how tall you are and you reply, say, ‘Five feet, ten inches,’ the answer does not seem at all improbable. But suppose you are a person addicted to mathematical accuracy, and you reply that your height is 5 feet, 10.321 inches; you have now made an assertion which may be true, but which is exceedingly improbable. The moment your height is measured to a thousandth of an inch, it becomes very improbable that you are the height you are, although at the same time it is quite certain that you are the height you are.

Again: how many square miles are there in the United States? I do not know, but if you suggest some number I shall be justified in saying that it is very unlikely to be exactly that, and yet there is some number that it is, although this is so improbable. What all these illustrations indicate is that every actual occurrence is wildly improbable as soon as it is accurately described. It does not astonish us unless the improbability remains when it is described only vaguely. If I say I met a man whose height is 5 feet, 10.321 inches, people will say, ‘What of it?’ although I have asserted a marvel. But if I say I have met a man over twenty feet high, they will be either incredulous or amazed, because I have said something which remains improbable in spite of being vague.

Two things emerge: first, that the improbable is not always astonishing, and second, that even the most wildly improbable things do happen. The things that do not often happen are astonishing things, but this is a law concerning our emotions, not concerning the world, for obviously what happens often does not astonish us. Bishop Butler said that probability is the guide of life, but I doubt whether he had thought out the implications of his statement, any more than the modern physicists have who say that probability is the guide of physics.

If the views of modern physicists were accepted in daily life, they would have a somewhat bewildering effect. Take, for example, some quite simple question, such as ‘What is a red flag?’ Some people say that it is an illegal symbol of revolution; others that it is a mark of a steam roller. Such views, according to Schrödinger, are shallow; according to him, it is an undulatory distribution of statistical probabilities of certain kinds of quantum transitions in atoms. Everything that we see is nothing but a distribution of probabilities. We do not see what is there, but only what is more or less likely to be there, and we see it just as much when it is not there as when it is there, provided the probability is right.

This view of the world certainly outdoes Bishop Berkeley at his own game. You are not obliged to accept it, for you may instead believe with Heisenberg that a red flag consists of some billions of infinite rectangles of integers, but few people would find this view any more comforting than Schrödinger’s, though I confess that I am one of the few.


It is time to ask ourselves what, if anything, is meant by probability. There are two theories on this subject, neither of which, to my mind, is satisfactory. One is known as the frequency theory; the other is the theory whose ablest advocate is Mr. Keynes, known to publicists by his Economic Consequences of the Peace, but to the learned world by his Treatise on Probability. Mr. Keynes holds that probability is an ultimate notion not further definable, consisting of a certain relation between premises and conclusion. Premises may prove a conclusion; in that case the conclusion is certain in relation to those premises. But they may only make the conclusion more or less probable; for example, if you are in contact with scarlet fever, you are not sure to catch scarlet fever, but there is a certain degree of probability that you will do so. The premise ‘I am in contact with scarlet fever’ does not demonstrate the conclusion ‘I shall catch scarlet fever,’ but the conclusion has to the premise a certain relation of probability. If this view of probability is accepted, we cannot be asked what we mean by the word, since the word is not definable in any other terms. It is clear that since words can only be defined by means of other words, any system of definitions must start with certain words that are not defined, and Mr. Keynes proposes to put the word ‘probability’ among these primitive notions.

Mr. Keynes is, I think, almost certainly right in holding that probability does not attach to a proposition in itself, but only in relation to certain premises, and that its probability in relation to certain premises may be quite different from its probability in relation to certain other premises. For example, if all you know about a man is that he lives in the British Isles, there is a certain definitely ascertainable probability that his name is William Williams. This probability is measured by dividing the number of males in the British Isles called William Williams by the total number of males in the British Isles. But if you know further that the man lives in Wales, the probability of his being called William Williams is very greatly increased. This does not mean that the probability that you had obtained before was wrong; it merely means that it was relative to different data. Given sufficient data, any statement is either certainly true or certainly false, so that probability only arises in relation to insufficient data. That is why probability is specially useful in regard to the future, as to which our data are always insufficient. So far I think we ought all to agree with Mr. Keynes.

But when he maintains that probability is indefinable, he is on more doubtful ground. I do not profess to have a satisfactory definition to offer, but I think that with sufficient ingenuity a satisfactory definition could be found. When a word is said to be indefinable, it is necessary to maintain that one knows what it means without the help of a definition. There are a number of words of which this is true; we know quite well what we mean by such words as ‘red’ and ‘blue’ and ‘sweet’ and ‘sour,’ and ‘up’ and ‘down.’ After we have learned the definition of these words, we do not understand them any better than we did before. But where probability is concerned, I at any rate have no such feeling; I wish to be told what it means, and until I am told I am in doubt. I do not think that Mr. Keynes’s theory can be refuted: it is a self-consistent theory which cannot be proved to be untrue. But it leaves one with a certain intellectual dissatisfaction and a feeling that a difficult problem has been evaded.

The so-called frequency theory does not have this defect, though in its traditional form it has others. The frequency theory in its simplest form states that when you know that a certain object belongs to a certain class, and that a certain proportion of that class has a certain characteristic, then that proportion measures the probability that the object in question has the said characteristic. We had an instance of this just now with the man called William Williams. In all statistical applications of probability, this definition is satisfactory. It is adequate, for example, to the use of probability in connection with life insurance. It is adequate also to the use of probability in modern physics, and in the doings of bookmakers. It is adequate, in a word, wherever exact numerical estimates of probabilities are possible.

But unfortunately this definition leaves us in the lurch as soon as we come to the question of induction, and we generally do come up against this question when we try to use probability as a guide in life. Take, for example, life insurance, no longer as mathematics, but as a practical proposition. Life-insurance companies have generally argued that people will die at the same rate in the future as in the past. In fact the rate at which people have died has been diminishing ever since life insurance came into vogue. The result has been that the insurance companies have grown rich; since this gave them no ground of complaint, the intellectual fallacy at the basis of their calculations has not worried them. If the purchase of annuities had been commoner than life insurance, the companies concerned would have been led to bankruptcy by their intellectual errors, and would, in that case, have become more acutely aware of them. They might, at least theoretically, have corrected their errors by a wider induction; but, however wide an induction may be, it can never lead to certainty.

The trouble is that the kind of probability derived from an induction does not seem to be the kind dealt with by the frequency theory. When a certain number of observations seem to have established a law of nature, every man of science knows that further observations may prove the supposed law to be incorrect. The law is therefore at best probable. But when we ask, ‘How probable is it?’ and hope to get a definite numerical answer, such as ‘The odds in its favor are a thousand to one,’ we are disappointed. Mr. Keynes maintains — and I for one do not know how to refute his arguments — that even the best-grounded inductions are not more likely to be true than false. Will the sun rise to-morrow? We none of us feel any serious doubt on this subject, and yet the best view seems to be that it is not more likely to rise than not to rise.

There is no way of dealing with the situation on the frequency theory. If we were to take all the natural laws that have ever been believed, and ask ourselves what proportion of them can still be regarded as tenable, I am afraid the result would be far from encouraging.

Moreover the frequency theory does not give any reason why a rational man in action should act upon the greater probability where certainties are unobtainable. And yet that is what we all really want probability to do for us. We all of us do act upon probabilities, not only in a remote and ultimate sense, but in a perfectly obvious everyday sense. Every glass of water that we drink, every mouthful of food that we eat, may contain bacteria or poison, but we disregard this probability except when for some reason it is unusually large. When men are found guilty of murder or theft, there is always some chance that they may be innocent, but when the chance is very small we behave as if it did not exist.

So long as a probability is numerically measurable, this seems sensible enough, but the most important probabilities are too vague to come under this head. What is the likelihood, for example, that psychical research may demonstrate a future life? Clearly we have not the data required for a numerical estimate, yet some kind of estimate has to be made by any man who is free to decide whether he shall or shall not devote some time to psychical research. What is the likelihood that universal education will improve the condition of mankind? Nobody knows, and if it were worth while a very good case could be made out for the negative. Yet we all cheerfully act upon the assumption that education is desirable, to the extent of paying out good money for the purpose.

The degree of probability that we demand depends, however, upon the nature of the case. Before executing a man for murder, we demand a much higher degree of evidence of his guilt than we do of his innocence before voting for him as President.


I have been speaking lately of probabilities without adducing any definite premises, which might seem contrary to what was said in connection with Mr. Keynes. In all such cases, and generally wherever one is considering a probability in relation to practice, there is a tacit reference to all relevant knowledge as constituting the premises of the probability. Take, for example, the man accused of murder: from his own point of view the matter is certain, because he has knowledge which the judge and the jury do not possess, but when one speaks of the likelihood of his guilt without specifically adducing premises, one means the likelihood in relation to what is known to the judge and jury. This is clearly the sort of probability with which practical men are concerned, but unfortunately it is a sort about which theorists are, as a rule, very hazy.

This whole matter of probability in relation to conduct is extraordinarily unsatisfactory. If men were rational animals, it would be even more so, since they would be paralyzed in action by their theoretical insecurity. As a matter of fact we act upon our passions, among which credulity is by no means the least. Without credulity we should be brought to a standstill, not only in love and marriage, politics and business, but even in science, since induction has no hitherto discovered rational basis. The civilized man differs from the savage, not in the extent of his credulity, but in its character. His credulity is systematized and organized, and can in the last resort be confined to a few great principles; when these have been swallowed, all the rest of his procedure becomes comparatively rational. The savage, on the contrary, has a large number of disconnected credulities — some connected with the wind, others with the rain, some with the sea, and others with the mountains. In this respect, as in others, it is system, organization, and interrelatedness that distinguish civilized life. But to suppose that these things have beneficent results is itself part of the civilized man’s credulity. Ethical and metaphysical postulates, however irrational, usually go hand in hand. The man who holds that the world is of such and such a kind almost always holds also that it is our duty to make it so. The civilized man, conversely, holding that it is our duty to make the world orderly and interrelated, holds also that the world is orderly and interrelated. On this basis it is possible to find a metaphysical justification for induction, as has been shown by Bergson and Dr. Whitehead.

For my part, I see no reason to believe that the world must be convenient for the man of science. It may be that all the law and order that we seem to perceive in the world is due to our own selective apperception. It is amazing to what an extent preconceptions can falsify the testimony of experience. Ignorant people believe that when you pour hot water into a glass it is more likely to crack if the glass is thin than if it is thick; educated people believe the opposite. Each side appeals with equal confidence to experience. A rational man would conclude that the laws of nature are different in the kitchen from what they are in the laboratory. But this is not the conclusion that either party does in fact draw.

Why should the man who has a laboratory so confidently dismiss the testimony of his cook? He is persuaded that she is a bad observer, misled by her prejudices, and he believes that he is quite impartial in his observation of phenomena. But there are plenty of facts which go to show that the man of science is not so very different from the cook. When I was young there was a universally accepted mathematical proof that the action of gravitation must be practically instantaneous. When Einstein came along, it became necessary to hold that gravitational action is propagated with the velocity of light, so the previously accepted proof to the contrary was found to be fallacious. I am not for a moment denying that it was fallacious; I am only saying that such instances should make us somewhat skeptical as to what arguments should still pass as valid.

It is a mistake to confuse probability with uncertainty. Everything that passes for knowledge is more or less uncertain — that is to say, there is a greater or less risk of error in regard to it. But this applies to knowledge concerning probability as well as to other knowledge. When we say that the chance of a coin coming heads is a half, we are not saying that we have uncertain knowledge that it will come heads, we are saying that we have practically certain knowledge that it is as likely to come heads as not to. Some knowledge concerning probabilities is as nearly certain as any knowledge that we possess.

Probability, therefore, is not to be associated with fallibility and our liability to error; probability is a definite branch of knowledge, but at present a very unsatisfactory branch. I think that its further study requires a clear separation of those cases where an exact numerical estimate is possible from those others that we considered in connection with induction. Where an exact numerical estimate is possible, the frequency theory is satisfactory, but in relation to such problems as that of induction this theory fails us.

Perhaps a mistake has been made in applying the one word ‘probability’ to the two classes of cases; perhaps two radically different concepts are involved. If so, a very large realm maybe rescued from the vague and unsatisfactory condition of the general theory of probability. Even so, however, the problems that remain are serious, important, and difficult. I do not know of any branch of logic where new ideas are so urgently called for.