Applied Engineering Stochastic Processes Cheat Sheet

This cheat sheet covers key concepts in applied engineering stochastic processes, focusing on multiple random variables, probability distributions, estimation techniques, and their applications.

Core Principles

Stochastic processes involve random variables and their evolution over time, crucial for modeling real-world phenomena.
Joint probability distributions (CDF, PMF, PDF) describe the behavior of multiple random variables and their relationships.
Marginal distributions focus on individual random variables within a joint distribution.
Independence of random variables simplifies analysis, where joint probabilities factorize into individual probabilities.
Covariance and correlation quantify the linear relationship between random variables.
Estimation techniques, like Minimum Mean Square Error (MMSE) and Linear MMSE (LMMSE), provide methods to predict or estimate unknown values based on available information.

Formulas

$P(A|B) = P(AB) / P(B)$
$F_{X|B}(x) = P[X \leq x | B]$
$P_{X|B}(x) = P[X = x | B]$
$f_{X|B}(x) = dF_{X|B}(x) / dx$
$P_{X|B}(x) = P_X(x) / P[B]$ (Discrete)
$f_{X|B}(x) = f_X(x) / P[B]$ (Continuous)
$E[W] = \sum g(x,y)P_{X,Y}(x,y)$ (Discrete)
$E[W] = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} g(x,y)f_{X,Y}(x,y) dx dy$ (Continuous)
$E[X+Y] = E[X] + E[Y]$
$Var[X+Y] = Var[X] + Var[Y] + 2E[(X-\mu_X)(Y-\mu_Y)]$
$Cov[X,Y] = E[(X-\mu_X)(Y-\mu_Y)]$
$\rho_{X,Y} = Cov[X,Y] / (\sigma_X \sigma_Y)$
$f_{X,Y}(x,y) = f_X(x)f_Y(y)$ (Independent Continuous)
$P_{X,Y}(x,y) = P_X(x)P_Y(y)$ (Independent Discrete)
$\hat{X} = E[X]$ (Blind Estimate)
$\hat{X}_A = E[X|A]$
$\hat{X}_L(Y) = a^*Y + b^*$ (Linear MMSE)
$a^* = Cov[X,Y] / Var[Y]$
$b^* = E[X] - a^*E[Y]$
$e_L^* = Var[X] - (Cov[X,Y])^2 / Var[Y]$
$\hat{X}_L(Y) = C_{XY} C_Y^{-1} (Y - E[Y]) + E[X]$
$e_L^* = Var[X] - C_{XY} C_Y^{-1} C_{YX}'$