For the rest of this lecture, we're going to be focused on counting. There are a couple of rules we leverage heavily when counting to make it easier for us to deal with larger and more complex sets. Both of these rules should be pretty intuitive, but they're good to be aware of and be confident that they are correct.
Suppose you are ordering an ice cream cone. There are three flavors of ice cream (chocolate, vanilla, strawberry), and two types of cones (cake and waffle). How many different orders are possible?
Intuitively, it's $3\cdot 2 = 6$.
To see this in an organized way, you can draw a tree, like in Figure 1.4 on page 10 of [BH].
We could formalize the thinking for this ice cream problem via a decision process. A decision process is nothing more than a sequence of decisions that leads to a particular outcome. If a decision process has stages have $d_1, d_2, \cdots, d_n$ options at each level then the tree has $d_1 \cdot d_2 \cdots d_k$ leaves, and hence so does the set we're trying to count.
To be a bit more precise, we state the principle as the following theorem.
Suppose a decision process with $k$ steps produces a tree with the following properties:
An immediate application of decision processes is called the multiplication rule.
If $A_1, A_2, \dots, A_k$ are finite sets, then $$|A_1 \times A_2 \times \cdots \times A_k| = |A_1| \cdot |A_2| \cdots |A_k|.$$
Suppose I have a bin with balls labeled $1$ to $10$, and another with balls labeled $11$ to $15$. How many outcomes are possible if I draw a ball out of each bin?
The set of outcomes for bin $1$ is $B_1 = \{1, 2, ..., 10\}$, and the set of outcomes for bin $2$ is $B_2 = \{11, 12, ..., 15\}$, so the set of total outcomes is $B_1 \times B_2$. By the multiplication rule, we learn that $|B_1 \times B_2| = |B_1| \cdot |B_2| = 10 \cdot 5 = \boxed{50}$.
If $A_1, A_2, \dots, A_k$ are disjoint finite sets, then $$|A_1 \cup A_2 \cup \cdots \cup A_k| = |A_1| + |A_2| + |A_k|.$$
Suppose I have a bin with balls labeled $1$ to $10$, and another with balls labeled $11$ to $15$. How many outcomes are possible if I draw a ball out of one of the bins?
The set of outcomes for bin $1$ is $B_1 = \{1, 2, ..., 10\}$, and the set of outcomes for bin $2$ is $B_2 = \{11, 12, ..., 15\}$, so the set of total outcomes is $B_1 \cup B_2$. By the addition rule, we learn that $|B_1 \cup B_2| = |B_1| + |B_2| = 10 + 5 = \boxed{15}$.
We'll solve a lot of problems using this approach, and also test the limits of this approach — not everything can be counted this way!
We start with some examples.
How many two-digit strings are there whose digits are different?
We create a decision process for this as follows:
How many two-digit numbers are there whose digits are different?
We create a decision process for this as follows:
How many ways are there to arrange $6$ people into a line?
We create a decision process for this as follows:
The decreasing product of positive integers from some $n$ down to $1$ is referred to as $n$ factorial or $n!$, with the factorial by the exclamation point. For example, $$5! = 5 \cdot 4 \cdot 3 \cdot 2 \cdot 1.$$ For notational reasons, it is also convenient to define $0!=1$.
The Multiplication Rule gets applied in ad hoc ways to all sorts of problems where one is making a sequence of decisions (like our ice cream cones). There are two patterns that are so common in probability and combinatorics that they are given names: Sampling with and without replacement.
If we make $k$ choices, one after the other, from a set of $n$ objects, and we allow the same object to be chosen multiple times, then there are $n^k$ possible outcomes.
This theorem is just the Multiplication Rule with all of the sets $A_1,\ldots,A_k$ taken as the same set of objects.
If we make $k$ choices, one after the other, from a set of $n$ objects, and we do not allow the same object to be chosen multiple times, then there are $n\cdot(n-1)\cdots (n-k+1)$ possible outcomes.
Let $A$ be the set of $n$ objects. The decision process is:
In other words, at the $i$-th step, one chooses $a_i$ under the constraint that is different from $a_1,\ldots,a_{i-1}$.
There are clearly $n$ options at the first step. In the second step, there are $n-1$ options, since $a_1$ is not allowed. Continuing, in the $i$-th step there are $n-(i-1)$ options remaining. At the last step, there $n-(k-1) = n-k+1$ options. We can check that each outcome appears exactly once, similar to before. The formula now follows from decision process counting.
Another way to write $n\cdot(n-1)\cdots (n-k+1)$ is $$ \frac{n!}{(n-k)!}. $$ Later this notation will be more convenient, and it's also easier to remember since you don't have to mess with the $n-k+1$ factor each time.
Prove that $|\mathcal{P}(S)| = 2^{|S|}$.
If we denote $S=\{s_1, s_2, ..., s_n\}$, we note that the power set consists of every possible subset of $S$. To construct any subset of $S$, we can use the following decision process:
Then we have $n$ total decisions to make, each of which has two possible choices, for a total of $2^{|S|}$ possibilities.
Let's try a new one. Say I have my same bin of $10$ labeled balls, but I want to pull out three at once. In other words, I don't really care about the order of the three balls I'm getting - I just want to know which three I got.
Using what we've learned so far, we might try to make another decision process. We could try to select the first ball ($10$ options) and then select the second ball ($9$ options), and finally select the third ball ($8$ options). This may seem tempting, but it's actually incorrect!
Let's take a closer look at what happened: because we only care about the balls we drew and not the order we drew them in, this decision process actually counts the same outcome multiple times! For example, in one run I may draw $4$, then $7$, then $1$, but in a different run I may draw $1$, then $4$, then $7$. These are the same outcome if I don't care about the order I drew them in, but the decision process doesn't account for that! In fact, for each $3$-ball group, we can use a decision process to find that there are $3 \cdot 2 \cdot 1$ different orderings in which we can get the same outcome. Knowing that, we can simply divide the number of outcomes we have found through our decision process by $6$, giving us a final answer of $(10 \cdot 9 \cdot 8) /(3 \cdot 2 \cdot 1)$.
It is often the case that we are interested in the number of subsets of size $k$ exist in a set of size $n$. So often, in fact, that we will introduce special notation for it:
Let $n,k$ be integers with $n\geq 0$. We define the notation $\binom{n}{k}$ (pronounced "$n$ choose $k$") to be the number of subsets of size $k$ from a set of size $n$. More formally, $$ \binom{n}{k} = \left|\{ S \subseteq \{1,\ldots,n\} \ : \ |S| = k\}\right|. $$ These numbers are called binomial coefficients.
Binomial coefficients are useful in a great many ways in combinatorics and other areas of math. Observe that if $k$ is negative or greater than $n$, then $\binom{n}{k}=0$ since there are never sets of those sizes. When $k$ is between $0$ and $n$, we have the following formula that generalizes the pattern above:
Let $n,k$ be integers with $n\geq 0$ and $0 \leq k \leq n$. Then $$ \binom{n}{k} = \frac{n!}{(n-k)!\cdot k!}. $$ (We use the convention $0!=1$ in the notation above.)
This will be our first example of a combinatorial proof, which is a perfectly rigorous way to establish formulas like this, but it might at first feel loose and fast compared to more concrete proofs. (Indeed, you can prove this via some sort of induction if you prefer.)
The number $n!/(n-k)!$ counts the ways to pick $k$ elements from $\{1,\ldots,n\}$ with order. Amongst these, each subset of size $k$ will have its elements listed exactly $k!$ times. Therefore $$ k!\cdot \binom{n}{k} = \frac{n!}{(n-k)!}. $$ It's worth noticing that this argument works when $n=0$ and/or when $k=0$ or $k=n$.
We have actually proved something non-obvious: For all $n$ and $k$ in theorem, $n!$ is divisible by $(n-k)!\cdot k!$. A priori it was not be clear why this should be the case. The choose function does not have to be used on its own either — it can often show up as one step of a decision process.
How many "words" can we make from the letters in LALALALA? It has $8$ letters, but the answer is not $8!$ since some of those arrangements give the same word (e.g. swapping the A's around won't change the word).
We can create a decision process for this problem as follows: Begin with eight blanks, and then
There are $d_1 = \binom{8}{4}$ ways to the make the first choice, and $d_2 = 1$ way to make the second choice. The answer is thus $d_1\cdot d_2 = \binom{8}{4}$.
We could have selected positions from the A's first without changing our answer.