Continuous Probability Distributions

Overview

This section covers continuous random variables and continuous probability distributions.

Key Definitions

Def. (Continuous Random Variable)

A random variable $X$ is continuous if there exists a non-negative function $f(x)$, called the probability density function (PDF), such that for any interval $[a, b]$, we have:

$P(a \leq X \leq b) = \int_a^b f(x) dx$

Def. (Probability Density Function (PDF))

For a continuous random variable $X$, the PDF $f(x)$ is a non-negative function satisfying:

1. $f(x) \geq 0$ for all $x$
2. $\int_{-\infty}^{\infty} f(x) dx = 1$
3. $P(a \leq X \leq b) = \int_a^b f(x) dx$

Note: For continuous random variables, $P(X = x) = 0$ for any specific value $x$.

Def. (Cumulative Distribution Function (Continuous RV))

For a continuous random variable $X$ with PDF $f(x)$, the CDF is:

$F(x) = P(X \leq x) = \int_{-\infty}^{x} f(t) dt$

Def. (Expected Value (Continuous RV))

The expected value of a continuous random variable $X$ with PDF $f(x)$ is:

$E[X] = \int_{-\infty}^{\infty} x f(x) dx$

provided the integral converges.

Common Continuous Distributions

Uniform Distribution

Def. (Uniform Distribution)

A continuous random variable $X$ follows a Uniform distribution on $[a, b]$, denoted $X \sim \text{Uniform}(a, b)$, if its PDF is:

$f(x) = \begin{cases} \frac{1}{b-a} & \text{if } x \in [a,b] \\ 0 & \text{otherwise} \end{cases}$

• Parameters: $a, b \in \mathbb{R}$ with $a < b$
• Support: $[a, b]$
• Applications: Random number generation, simulations

Exponential Distribution

Def. (Exponential Distribution)

A continuous random variable $X$ follows an Exponential distribution with rate $\lambda > 0$, denoted $X \sim \text{Exp}(\lambda)$, if its PDF is:

$f(x) = \begin{cases} \lambda e^{-\lambda x} & \text{if } x \geq 0 \\ 0 & \text{otherwise} \end{cases}$

• Parameter: $\lambda > 0$ (rate parameter)
• Support: $[0, \infty)$
• Memoryless Property: $P(X > s+t | X > s) = P(X > t)$
• Applications: Time between events in a Poisson process, system lifetimes

Normal (Gaussian) Distribution

Def. (Normal Distribution)

A continuous random variable $X$ follows a Normal distribution with mean $\mu$ and variance $\sigma^2$, denoted $X \sim N(\mu, \sigma^2)$, if its PDF is:

$f(x) = \frac{1}{\sigma\sqrt{2\pi}} e^{-\frac{(x-\mu)^2}{2\sigma^2}} \text{ for } x \in \mathbb{R}$

• Parameters: $\mu \in \mathbb{R}$ (mean), $\sigma > 0$ (standard deviation)
• Support: $(-\infty, \infty)$

Def. (Standard Normal Distribution)

The standard normal distribution is $N(0, 1)$, with PDF:

$\phi(z) = \frac{1}{\sqrt{2\pi}} e^{-z^2/2}$

Standardization: If $X \sim N(\mu, \sigma^2)$, then $Z = \frac{X-\mu}{\sigma} \sim N(0,1)$

Def. (Z-Score (Standard Score))

The z-score of a value $x$ from a distribution with mean $\mu$ and standard deviation $\sigma$ is:

$z = \frac{x - \mu}{\sigma}$

The z-score represents the number of standard deviations $x$ is from the mean. It allows comparison of values from different normal distributions.

Interpretation:

• $z > 0$: value is above the mean
• $z < 0$: value is below the mean
• $|z| > 2$: value is more than 2 standard deviations from the mean (unusual)

Power-Law Distribution

Def. (Power-Law Distribution)

A continuous random variable $X$ follows a Power-Law distribution with parameters $\alpha > 1$ and $x_{\min} > 0$, denoted $X \sim \text{PowerLaw}(\alpha, x_{\min})$, if its PDF is:

$f(x) = \begin{cases} \frac{\alpha-1}{x_{\min}} \left(\frac{x}{x_{\min}}\right)^{-\alpha} & \text{if } x \geq x_{\min} \\ 0 & \text{otherwise} \end{cases}$

• Parameters: $\alpha > 1$ (exponent), $x_{\min} > 0$ (minimum value)
• Support: $[x_{\min}, \infty)$
• Heavy Tails: Probability decreases as a power of $x$
• Applications: Network degree distributions, file sizes, word frequencies