The probability of an event is a measure of the chance that the event occurs. Two measures are commonly used: the a priori and relative frequency definitions of probability.
The relative frequency of an event E is defined as the proportion of n trials which result in E. If the number of trials n is large, the proportion of trials resulting in E is a good estimate of the true probability that E will occur. The following applet illustrates the principles.
In this applet, the user can set the true probability of an event E (denoted by p) and the number of trials (denoted by n). By default, p = 0.5 and n = 100. As n increases, the estimate of p, denoted by p hat, should converge to p. The simulation is more effective if it is not too fast. Click on the left side of the slider (under Slow) once or twice to slow the simulation.
Suppose the true probability of an event E is 0.4, i.e., P(E) = 0.4, but you don't actually know this probability. For example, you are handed a tack, but you are not told the true probability the tack will land pointing down in a single toss, i.e., you don't know p = P(Tack Down). How would you proceed? You would run an experiment in which you toss the tack a number of times and then record the proportion of the tosses that land down. How would you decide the number of tosses, i.e, the number of trials n, to give you a good estimate of p? The answer to this question can be made precise when you study the binomial probability distribution, but until then we will use an ad hoc approach.
Would n = 20 be enough tosses to get a precise estimate of p? Change p to 0.4 and n to 20. We will obtain 10 estimates and see how well they estimate p. My 10 estimates are: 0.60, 0.40, 0.30, 0.30, 0.35, 0.40, 0.35, 0.60, 0.50, and 0.45. (You should obtain your own estimates.) The mean of these 10 estimates is 0.425 which is not too far from 0.4. However, we need to know how the estimates vary since in real situations you will only be able to get one estimate. The standard deviation of the estimates is 0.111 which indicates fairly high variability. Would it be unusual to get an estimate of 0.5 or greater or of 0.3 or less? The answer is nothese values can and do occur as seen above.
Is n = 100 enough? Change n to 100 and obtain 10 estimates. My 10 estimates are: 0.34, 0.43, 0.42, 0.41, 0.48, 0.36, 0.45, 0.47, 0.43, and 0.39. (Again, obtain your own estimates.) These estimates seem a lot better and none are 0.3 or below, or 0.5 or above. The mean of these estimates is 0.418 and the standard deviation is 0.045. Overall, these estimates are much better than those for n = 20, but they still vary. It is not unusual to have estimates of 0.45 or above or 0.35 or below. (We can make this statement precise when we study the binomial distribution.)
You can get as much accuracy as you want by increasing n. Try n = 200 and n = 500. At 500, you will get accurate estimates, but some variation is still present. In real life situations, you only need so much accuracy. For example, how accurate do presidential polls need to be? Since the true proportion of voters supporting candidate A changes continually, it is wasteful to get a precise estimate at a specific time.
The a priori definition of probability allows probabilities to be computed in special cases without experimentation. Most notably, probabilities can be computed in games of chance. A priori probabilities are most commonly computed for equally likely outcomes which in turn often depends on symmetry. For example, a die exhibits a type of symmetry and as a result the sides (outcomes) are equally likely. Thus, we know that the probability of getting an even outcome is 1/2 since 3 out of 6 possible outcomes are even.
We can now give an operational a priori definition of probability. If an experiment has n equally like outcomes and r of the outcomes are in the event E, then the probability of E, i.e., P(E), is r/n.
Example #1 demonstrates the relative frequency and a priori definitions of probability using an urn with 10 balls.
Exercise #1 demonstrates the relative frequency and a priori definitions of probability using an urn with 10 balls.
Exercise #2 demonstrates the relative frequency and a priori definitions of probability using an urn with 100 balls.
Exercise #3 demonstrates the relative frequency and a priori definitions of probability using an urn with 25 balls.