Conditional Probability

Key Definitions

Def. (Conditional Probability)

The conditional probability of event $A$ given event $B$ is defined as:

$P(A|B) = \frac{P(A \cap B)}{P(B)}$

provided that $P(B) > 0$. This represents the probability of $A$ occurring given that we know $B$ has occurred.

Def. (Bayes' Rule)

Bayes' Rule (or Bayes' Theorem) provides a way to reverse conditional probabilities:

$P(A|B) = \frac{P(B|A)P(A)}{P(B)}$

This is fundamental for updating probabilities based on new evidence.

Def. (Law of Total Probability)

Let $B_1, B_2, \ldots, B_n$ be a partition of the sample space $S$ (i.e., they are mutually exclusive and $\bigcup_{i=1}^{n} B_i = S$), and let $A$ be any event. Then:

$P(A) = \sum_{i=1}^{n} P(A|B_i)P(B_i)$

This is useful when we can express $A$ in terms of simpler conditional probabilities.

Def. (Partition of Sample Space)

A collection of events $B_1, B_2, \ldots, B_n$ forms a partition of the sample space $S$ if:

1. The events are mutually exclusive: $B_i \cap B_j = \emptyset$ for $i \neq j$
2. The events are exhaustive: $\bigcup_{i=1}^{n} B_i = S$
3. Each event has positive probability: $P(B_i) > 0$ for all $i$

Def. (Independence of Events)

Two events $A$ and $B$ are independent if the occurrence of one does not affect the probability of the other. Formally:

$P(A \cap B) = P(A)P(B)$

Equivalently, if $P(B) > 0$, then $A$ and $B$ are independent if and only if $P(A|B) = P(A)$.

Multiple events: Events $A_1, A_2, \ldots, A_n$ are:

• Pairwise independent if $P(A_i \cap A_j) = P(A_i)P(A_j)$ for all $i \neq j$
• Mutually independent if for every subset $\{i_1, i_2, \ldots, i_k\}$: $P(A_{i_1} \cap A_{i_2} \cap \cdots \cap A_{i_k}) = P(A_{i_1})P(A_{i_2})\cdots P(A_{i_k})$

Note: Mutual independence is stronger than pairwise independence.

Lecture materials and examples will be posted on Canvas.