Discrete Probability Distributions

Overview

This section covers discrete random variables and their probability distributions.

Key Definitions

Def. (Random Variable)

A random variable is a function that maps outcomes in the sample space to real numbers. Formally, $X: S \to \mathbb{R}$.

Def. (Discrete Random Variable)

A random variable $X$ is discrete if it takes on a countable number of distinct values.

Def. (Probability Mass Function (PMF))

For a discrete random variable $X$, the PMF is a function $p_X(x) = P(X = x)$ that gives the probability that $X$ takes the value $x$.

Def. (Cumulative Distribution Function (CDF))

The CDF of a random variable $X$ is defined as $F_X(x) = P(X \leq x)$ for all $x \in \mathbb{R}$.

Def. (Expected Value)

The expected value (or mean) of a discrete random variable $X$ is defined as $E[X] = \sum_{x} x \cdot P(X = x)$, provided the sum converges.

Def. (Variance)

The variance of a random variable $X$ is defined as $\text{Var}(X) = E[(X - E[X])^2] = E[X^2] - (E[X])^2$.

The variance measures the spread or dispersion of the distribution around the mean.

Def. (Standard Deviation)

The standard deviation of a random variable $X$ is the square root of the variance:

$\sigma_X = \sqrt{\text{Var}(X)}$

The standard deviation has the same units as $X$ and provides a measure of spread in the original units.

Def. (Independence of Random Variables)

Random variables $X$ and $Y$ are independent if for all values $x$ and $y$:

$P(X = x, Y = y) = P(X = x) \cdot P(Y = y)$

Equivalently, knowing the value of $X$ provides no information about $Y$.

Properties:

• If $X$ and $Y$ are independent, then $E[XY] = E[X]E[Y]$
• If $X$ and $Y$ are independent, then $\text{Var}(X + Y) = \text{Var}(X) + \text{Var}(Y)$

Multiple random variables: $X_1, X_2, \ldots, X_n$ are mutually independent if for all values $x_1, x_2, \ldots, x_n$: $P(X_1 = x_1, X_2 = x_2, \ldots, X_n = x_n) = P(X_1 = x_1) \cdot P(X_2 = x_2) \cdots P(X_n = x_n)$

Common Discrete Distributions

Bernoulli Distribution

Def. (Bernoulli Distribution)

A random variable $X$ follows a Bernoulli distribution with parameter $p \in [0,1]$, denoted $X \sim \text{Bernoulli}(p)$, if $P(X = 1) = p$ and $P(X = 0) = 1-p$.

• Parameter: $p$ = probability of success
• Support: $\{0, 1\}$
• Applications: Binary outcomes in CS (success/failure, true/false)

Binomial Distribution

Def. (Binomial Distribution)

A random variable $X$ follows a Binomial distribution with parameters $n$ and $p$, denoted $X \sim \text{Binomial}(n, p)$, if it represents the number of successes in $n$ independent Bernoulli$(p)$ trials. The PMF is:

$P(X = k) = \binom{n}{k} p^k (1-p)^{n-k} \text{ for } k = 0, 1, \ldots, n$

• Parameters: $n$ = number of trials, $p$ = probability of success per trial
• Support: $\{0, 1, 2, \ldots, n\}$
• Relationship: Sum of $n$ independent Bernoulli$(p)$ random variables

Geometric Distribution

Def. (Geometric Distribution)

A random variable $X$ follows a Geometric distribution with parameter $p$, denoted $X \sim \text{Geometric}(p)$, if it represents the number of trials needed to get the first success in a sequence of independent Bernoulli$(p)$ trials. The PMF is:

$P(X = k) = (1-p)^{k-1} p \text{ for } k = 1, 2, 3, \ldots$

• Parameter: $p$ = probability of success per trial
• Support: $\{1, 2, 3, \ldots\}$
• Memoryless Property: $P(X > n+m | X > n) = P(X > m)$

Poisson Distribution

Def. (Poisson Distribution)

A random variable $X$ follows a Poisson distribution with parameter $\lambda > 0$, denoted $X \sim \text{Poisson}(\lambda)$, if its PMF is:

$P(X = k) = \frac{\lambda^k e^{-\lambda}}{k!} \text{ for } k = 0, 1, 2, \ldots$

• Parameter: $\lambda$ = average rate of occurrence
• Support: $\{0, 1, 2, \ldots\}$
• Applications: Network traffic, system failures, rare events

Key Formulas

Expected Value

For a discrete random variable $X$:

$E[X] = \sum_{x} x \cdot P(X = x)$

Variance

$\text{Var}(X) = E[X^2] - (E[X])^2$