Chapter 11: Probability

We see probabilities almost every day in our lives. When you pick up the newspaper or read the news on the internet, you may encounter statements such as "there is a 60% chance of rain today" or "a poll shows that 52% of voters approve of the president's job performance." Probabilities are essential in sports, games, and gambling establishments, but probabilities are also used to make business decisions, figure out insurance premiums, and determine the price of raffle tickets. In its most general sense, probability provides a way to measure the chance or likelihood that something will happen.

Probability Terminology

Before discussing how to find probabilities, we need to familiarize ourselves with some basic terminology. When studying probability, we consider a random experiment to be an activity or operation that gives a result that can be observed but not predicted ahead of time. If we roll a pair of dice, pick a card from a deck of playing cards, spin a spinner, or randomly select a person and observe their hair color, we are executing an experiment and observing the result.

Any possible result of conducting an experiment is called an outcome. For the experiment of flipping a coin, there are only two outcomes: heads or tails. For the experiment of rolling a single die, there are six outcomes: $1$ , $2$ , $3$ , $4$ , $5$ , or $6$ . For an experiment, this collection of all possible outcomes is called the sample space.

An event is a collection of outcomes from an experiment. In some instances events contain only one outcome, while at other times an event may contain more than one outcome. Consider the experiment of rolling a single die. The event "rolling a 3" contains only the outcome ${3}$ , while the event "rolling an even number" contains the outcomes ${2, 4, 6}$ .

Definition: Probability Terminology

A random experiment is an activity or operation with a result that cannot be predicted ahead of time.

Any result from conducting an experiment is called an outcome.

The sample space of an experiment is the set of all its possible outcomes.

An event is a subset of the sample space and describes a collection of outcomes.

Example: Rolling a Die

Consider an experiment of rolling a single die. When we roll it, only one outcome will occur, but we are unsure which outcome. There are six possible outcomes, so the sample space is

$S = {1, 2, 3, 4, 5, 6} .$

"Rolling a 2" is an event that contains only one outcome: ${2}$ .
"Rolling a number greater than 2" is another event that contains multiple outcomes: ${3, 4, 5, 6}$ .

Example: Tossing Two Coins

Two pennies are tossed at the same time. Both pennies may land heads up (which we write as HH), or the first penny might land heads up and the second one tails up (which we write as HT), and so on. Write the sample space for the experiment and list the outcomes in the event "getting at least one heads."

The sample space for this experiment is

$S = {HH, H T, T H, TT} .$

If we define event $A$ as "getting at least one heads," the outcomes in event $A$ can be written as

$A = {HH, H T, T H} .$

Exercise

Gabe performs an experiment of flipping a coin and then rolling a regular six-sided die.

Give the sample space for how the coin and die could land.
Give the outcomes in event $A$ : rolls an odd number.
Give the outcomes in event $B$ : gets tails and rolls an even number.

Answer:

${H 1, H 2, H 3, H 4, H 5, H 6, T 1, T 2, T 3, T 4, T 5, T 6}$
${H 1, H 3, H 5, T 1, T 3, T 5}$
${T 2, T 4, T 6}$

Probability is one way to measure the chance or likelihood that an event will occur. Probability is usually denoted in function notation by $P$ , and the event is denoted by a capital letter such as $A$ , $B$ , or $C$ . The mathematical notation that indicates the probability that event $A$ happens is $P (A)$ .

Definition: Probability

Probability is a numerical measure of the chance or likelihood that an event will occur.

Definition: Theoretical Probability

A theoretical probability is based on a mathematical model where the number of outcomes in the event is compared with the number of outcomes in the sample space of an experiment. If the outcomes are equally likely, a formula for the theoretical probability of event $E$ is

$P (E) = \frac{number of outcomes in event E}{number of outcomes in sample space S} .$

Let's apply this formula in some relatively simple examples.

Example: Rolling a Six-Sided Die

Consider the experiment of rolling a regular six-sided die. Find the probability of each event:

rolling a 5
rolling an even number
rolling a number greater than 4
rolling a 7
rolling a number less than 7

There are $6$ possible equally likely outcomes in the sample space:

$S = {1, 2, 3, 4, 5, 6} .$

There is only one outcome in the event "rolling a 5": ${5}$ . Thus, $P (rolling a 5) = \frac{1}{6} .$
There are three outcomes in the event "rolling an even number": ${2, 4, 6}$ . Thus, $P (rolling an even number) = \frac{3}{6} = \frac{1}{2} .$
There are two outcomes in the event "rolling a number greater than 4": ${5, 6}$ . Thus, $P (rolling a number greater than 4) = \frac{2}{6} = \frac{1}{3} .$
There are no outcomes in the event "rolling a 7": ${}$ . Thus, $P (rolling a 7) = \frac{0}{6} = 0.$
There are six outcomes in the event "rolling a number less than 7": ${1, 2, 3, 4, 5, 6}$ . Thus, $P (rolling a number less than 7) = \frac{6}{6} = 1.$

The previous example illustrates some important properties about values that can be legitimate probabilities.

The number of outcomes in an event can never be lower than $0$ . So, the smallest a probability can be is $0$ . If the probability of an event is $0$ , we say that event is impossible.
The number of outcomes in an event can never be more than the number of outcomes in the sample space. Therefore, the largest a probability can be is $1$ . If the probability of an event is $1$ , we say that event is certain.
The probability of any event must always fall between $0$ and $1$ , inclusive. In the course of this chapter, if you compute a probability and get an answer that is negative or greater than $1$ , you have made a mistake and should recheck your work.

Definition: Probability Properties

An event that cannot occur has probability $0$ . This event is impossible.

An event that must occur has probability $1$ . This event is certain.

The probability of any event must be between $0$ and $1$ , inclusive. That is,

$0 \leq P (E) \leq 1.$

Example: Probabilities with Two Coins

Recall the experiment in which two pennies are tossed simultaneously and how they land is recorded. Find the probability of getting each result:

exactly two heads
exactly one head
at least one head
more than two heads

The sample space for this experiment is $S = {HH, H T, T H, TT}$ .

The event "exactly two heads" occurs in one outcome, ${HH}$ : $P (exactly two heads) = \frac{1}{4} .$
The event "exactly one head" occurs in two outcomes, ${H T, T H}$ : $P (exactly one head) = \frac{2}{4} = \frac{1}{2} .$
The event "at least one head" occurs in three outcomes, ${HH, H T, T H}$ : $P (at least one head) = \frac{3}{4} .$
The event "more than two heads" occurs in zero outcomes, ${}$ : $P (more than two heads) = \frac{0}{4} = 0.$

Definition: Theoretical and Empirical Probability

A theoretical probability is based on a mathematical model where all outcomes are equally likely to occur.

An empirical probability is based on collected data and is the relative frequency of the event occurring.

Example: Theoretical and Empirical Probability

Lawrence is playing with a standard 52-card deck and wants to find the probability of selecting a queen from the deck.

The theoretical probability that Lawrence pulls a single queen is

$\frac{4}{52} = \frac{1}{13} \approx 0.0769,$

or about $7.69%$ .

If Lawrence decides to try it out $25$ times and pulls a queen at random $3$ times in $25$ trials of "pull a card, record it, put it back," the empirical probability of pulling a queen is

$\frac{3}{25} = 0.12,$

or about $12%$ .

The Law of Large Numbers

Flipping a coin is often used to randomly make a decision when there are only two choices. For example, you may flip a coin to decide whether you have steak or fish for dinner. Or, a referee uses a coin flip to decide which football team receives the ball prior to kickoff. The reason why a coin flip seems fair in these circumstances is that most of us agree that the probability of getting heads (and tails) on a coin is $\frac{1}{2} = 0.5 = 50%$ . But what does this mean in practice?

Does that mean if we flip a coin twice we will get heads exactly once? If a coin is tossed $10$ times, will we necessarily get heads five times? Most of us intuitively know the answer is no. Indeed, if we flip a coin $10$ times we might find that it lands on heads $7$ times. So what does it mean to say that the probability of heads on a fair coin is $\frac{1}{2}$ ?

To investigate this question, consider the table showing results that may happen when a coin is tossed several times. The top row shows the number of times the coin has been tossed. The next row shows the number of heads that have occurred. The bottom row shows the empirical probability, which is the ratio of the number of heads observed to the number of trials.


Number of Trials	10	20	30	40	50
Number of Heads Observed	7	13	17	22	26
Empirical Probability of Heads	$\frac{7}{10}$	$\frac{13}{20}$	$\frac{17}{30}$	$\frac{22}{40}$	$\frac{26}{50}$

Notice that as the number of trials increases, the empirical probability gets closer to $0.5$ , which is what we expect to happen theoretically. In fact, if we kept increasing the number of trials, we would find that the empirical probability would eventually be very close to $\frac{1}{2} = 0.5$ .

This relationship between empirical probability and theoretical probability can be summarized by the Law of Large Numbers. The probability of an event applies to a large number of trials, not a single trial or a few trials. We should not be surprised that the empirical probability calculated from only a few trials is different from the theoretical probability. It is only empirical probability calculated over the long run that gives an accurate probability.

Definition: The Law of Large Numbers

The Law of Large Numbers states that we can only expect the empirical probability of an event to approximate its true probability when the number of trials of the experiment is large.

Keyboard shortcuts

Mathematics Brush-up for Data Science

Chapter 11: Probability

Probability Terminology

The Law of Large Numbers