Selected Quotes about Probability
Here is a selection of quotes (and references) about probability concepts, encountered during my attempts to understand them better. I am rounding them up here as I encounter them so that I do not forget them when I have to actually explain things to people.
Definitions of Probability
The probability of an event is defined as the number of cases favorable to the event, divided by the total number of possible cases, provided that the latter are equally likely to occur.
There is a difficulty of a logical order in this definition of probability, due to the occurrence of the words equally likely as applied to the possible cases. Equally likely cases would seem to be those whose probabilities are equal, thus making the definition of probability depend on the idea of probability itself, a clear-cut example of a vicious circle. This is a little like defining a cow as a cowlike animal. It means that we must have some notion of what equally likely cases are before we tackle any concrete problem.
So, if a sample space has n outcomes, each of which are equally likely to occur when the experiment is performed once, then each outcome has as a probability 1/n of occurring...Hence, Pr(event) = (number of outcomes in the event)/(number of outcomes in the sample space)
We will define the probability of an event as the long-run relative frequency of the event in repeated trials under similar conditions. Every word of this "frequentist" definition has been chosen carefully, so read it again.
If we can identify equally likely cases, then calculating probabilities amounts simply to enumerating equally likely cases - not always easy, but straightforward in principle. However, identifying equally likely cases requires more thought.
Sometimes we estimate probabilities from data. The probability of our precious observing run being clouded out is estimated by (number of cloudy nights last year)/365 but two issues arise. One is the limited data - we suspect that 10 years' worth of data would give a different, more accurate result. The second issue is simply the identification of the 'equally likely' cases. Not all nights are equally likely to be cloudy, some student of these matters tells us; it's much more likely to be cloudy in winter. What is 'winter', then? A set of nights equally likely to be cloudy?
The notion we adopt for the present is that probability is a numerical formalisation of our degree or intensity of belief.
Subjectivity of Probability
Ascribing an apparently subjective meaning to probability in this way needs careful justification. After all, one person's degree of belief is another person's certainty, depending on what is known. We can only reason as best we can with the information we have; if our probabilities turn out to be wrong, the deficiency is in what we know, not the definition of probability. We just need to be sure that two people with the same information will arrive at the same probabilities.
Also in drawing one card from a deck of fifty-two cards we say that one card is as likely to appear as another, provided that the deck has been thoroughly shuffled, and the faces of the cards are hidden. This instance brings to light a very important consideration: a priori probabilities depend on the amount of knowledge assumed as given; they change when the assumed amount of pertinent knowledge is changed. If you are able to see some of the cards when making the draw, it is no longer possible to assume that all fifty-two cards are on the same footing. We cannot be sure that you are not influenced one way or another by what you see. Furthermore, the proviso that the deck be shuffled has no importance whatever unless you have some idea in advance of the order of the cards, as you would, for instance, if the deck were new.
Probability depends, therefore, on our knowledge and on our ability to make estimates. In effect, our common sense! Fortunately, there is a certain amount of agreement in the common sense of many things, so that different people will make the same estimate. Probabilities need not, however, be "absolute" numbers. Since they depend on our ignorance, the may become different if our knowledge changes.