Content

Binomial random variables

Define \(X\) to be the number of successes in \(n\) independent Bernoulli trials, each with probability of success \(p\). We now derive the probability function of \(X\). First we note that \(X\) can take the values \(0,1,2,\dots,n\). Hence, there are \(n+1\) possible discrete values for \(X\). We are interested in the probability that \(X\) takes a particular value \(x\). If there are \(x\) successes observed, then there must be \(n-x\) failures observed. One way for this to happen is for the first \(x\) Bernoulli trials to be successes, and the remaining \(n-x\) to be failures. The probability of this is given by \[ \underbrace{p \times p \times \dots \times p}_{x \text{ of these}} \times \underbrace{(1-p) \times (1-p) \times \dots \times (1-p)}_{n-x \text{ of these}} = p^x (1-p)^{n-x}. \]

However, this is only one of the many ways that we can obtain \(x\) successes and \(n-x\) failures. They could occur in exactly the opposite order: all \(n-x\) failures first, then \(x\) successes. This outcome, which has the same number of successes as the first outcome but with a different arrangement, also has probability \(p^x (1-p)^{n-x}\). There are many other ways to arrange \(x\) successes and \(n-x\) failures among the \(n\) possible positions. Once we position the \(x\) successes, the positions of the \(n-x\) failures are inevitable.

The number of ways of arranging \(x\) successes in \(n\) trials is equal to the number of ways of choosing \(x\) objects from \(n\) objects, which is equal to \[ \dbinom{n}{x} = \dfrac{n!}{x!(n-x)!} = \dfrac{n \times (n-1) \times (n-2) \times \dots \times (n-x+2) \times (n-x+1)}{x \times (x-1) \times (x-2) \times \dots \times 2 \times 1}. \] (This is discussed in the module The binomial theorem.) Recall that \(0!\) is defined to be 1, and that \(\binom{n}{n} = \binom{n}{0} = 1\). We can now determine the total probability of obtaining \(x\) successes. Each outcome with \(x\) successes and \(n-x\) failures has individual probability \(p^x (1-p)^{n-x}\), and there are \(\binom{n}{x}\) possible arrangements. Hence, the total probability of \(x\) successes in \(n\) independent Bernoulli trials is \[ \dbinom{n}{x}\, p^x (1-p)^{n-x}, \quad \text{for } x = 0,1,2,\dots,n. \]

This is the binomial distribution, and it is arguably the most important discrete distribution of all, because of the range of its application.

If \(X\) is the number of successes in \(n\) independent Bernoulli trials, each with probability of success \(p\), then the probability function \(p_X(x)\) of \(X\) is given by \[ p_X(x) = \Pr(X=x) = \dbinom{n}{x}\, p^x (1-p)^{n-x}, \quad \text{for } x = 0,1,2,\dots,n, \] and \(X\) is a binomial random variable with parameters \(n\) and \(p\). We write \(X \stackrel{\mathrm{d}}{=} \mathrm{Bi}(n,p)\). Notes.

• The probability of \(n\) successes out of \(n\) trials equals \(\binom{n}{n}\, p^n (1-p)^0 = p^n\), which is the answer we expect from first principles. Similarly, the probability of zero successes out of \(n\) trials is the probability of obtaining \(n\) failures, which is \((1-p)^n\).
• Consider the two essential properties of any probability function \(p_X(x)\):
  1. \(p_X(x) \geq 0\), for every real number \(x\).
  2. \(\sum p_X(x) = 1\), where the sum is taken over all values \(x\) for which \(p_X(x) > 0\).
  In the case that \(X\) is a binomial random variable, we clearly have \(p_X(x) \geq 0\), since \(0 < p < 1\). It looks like a more challenging task to verify that \(\sum p_X(x) = 1\). However, we can use the binomial theorem: \[ (a + b)^n = \sum_{r=0}^n \dbinom{n}{r}\, a^{n-r}b^r. \] (See the module The binomial theorem.) Note the form of each summand, and the similarity to the probability function of the binomial distribution. It follows by the binomial theorem that \[ \sum p_X(x) = \sum_{x=0}^n \dbinom{n}{x}\, p^x (1-p)^{n-x} = \bigl((1-p) + p\bigr)^n = 1. \]
• If we view the formula \[ p_X(x) = \dbinom{n}{x}\, p^x (1-p)^{n-x} \] in isolation, we may wonder whether \(p_X(x) \leq 1\). After all, the number \(\binom{n}{x}\) can be very large: for example, \(\binom{300}{150} \approx 10^{89}\). But in these circumstances, the number \(p^x (1-p)^{n-x}\) is very small: for example, \(0.4^{150}\times 0.6^{150} \approx 10^{-93}\). So it seems that, in the formula for \(p_X(x)\), we may be multiplying a very large number by a very small number. Can we be sure that the product \(p_X(x)\) does not exceed one? The simplest answer is that we know that \(p_X(x) \leq 1\) because we derived \(p_X(x)\) from first principles as a probability. Alternatively, since \(\sum p_X(x) = 1\) and \(p_X(x) \geq 0\), for all possible values of \(X\), it follows that \(p_X(x) \leq 1\).

In general, it is easier to calculate binomial probabilities using technology. Microsoft Excel provides a function for this, called \(\sf \text{BINOM.DIST}\). To use this function to calculate \(p_X(x)\) for \(X \stackrel{\mathrm{d}}{=} \mathrm{Bi}(n,p)\), four arguments are required: \(x,\ n,\ p,\ \sf \text{FALSE}\). For example, to find the value of \(p_X(2)\) for \(X \stackrel{\mathrm{d}}{=} \mathrm{Bi}(5,\frac{1}{4})\), enter the formula \[ \sf \text{=BINOM.DIST(2, 5, 0.25, FALSE)} \]

in a cell; this should produce the result 0.2637 (to four decimal places). The somewhat off-putting argument \(\sf \text{FALSE}\) is required to get the actual probability function \(p_X(x)\). If \(\sf \text{TRUE}\) is used instead, then the result is the cumulative probability \(\sum_{k=0}^{x} p_X(k)\), which we do not consider here.

It is useful to get a general picture of the binomial distribution. figure 2 shows nine binomial distributions, each with \(n=20\); the values of \(p\) range from 0.01 to 0.99.

Detailed description

Figure 2: Nine binomial distributions with parameters \(n=20\) and \(p\) as labelled.

Interactive 1   Interactive 2

A number of features of this figure are worth noting:

• First, we may wonder: Where are all the other probabilities? We know that a binomial random variable can take any value from 0 to \(n\). Here \(n=20\), so there are 21 possible values. But in all of the graphs, there are far fewer than 21 spikes showing. Why? The reason is simply that many of the probabilities are too small to be visible on the scale shown. The smallest visible probabilities are about 0.01; many of the probabilities here are 0.001 or smaller, and therefore invisible. For example (taking an extreme case that is easy to work out), for the top-left graph where \(X \stackrel{\mathrm{d}}{=} \mathrm{Bi}(20,0.01)\), we have \(\Pr(X=20) = 0.01^{20} = 10^{-40}\).
• When \(p\) is close to 0 or 1, the distribution is not symmetric. When \(p\) is close to 0.5, it is closer to being symmetric. When \(p = 0.5\), it is symmetric.
• Consider the pairs of distributions for \(p = \theta\) and \(p = 1-\theta\): \(p = 0.01\) and \(p=0.99\); \(p = 0.05\) and \(p=0.95\); \(p=0.2\) and \(p=0.8\); \(p=0.35\) and \(p=0.65\). Look carefully at the two distributions for any one of these pairs. The two distributions are mirror images, reflected about \(x=10\). This follows from the nature of the binomial distribution.
  
  According to the definition of the binomial distribution, we count the number of successes. Since each of the Bernoulli trials has only two possible outcomes, it must be equivalent, in some sense, to count the number of failures. If we know the number of successes is equal to \(x\), then clearly the number of failures is just \(n-x\). It follows that, if \(X \stackrel{\mathrm{d}}{=} \mathrm{Bi}(n,p)\) and \(Y\) is the number of failures, then \(Y = n-X\) and \(Y \stackrel{\mathrm{d}}{=} \mathrm{Bi}(n,1-p)\). Note that \(X\) and \(Y\) are not independent.
• Finally, the spread of the binomial distribution is smaller when \(p\) is close to 0 or 1, and the spread is greatest when \(p = 0.5\).

Next page - Content - Mean and variance