Core Probability & Statistics Formulas

This cheat sheet provides essential formulas and concepts for probability and statistics, covering counting principles, basic probability, random variables, and key distributions. It serves as a quick reference for core concepts in probability theory.

Core Principles

• Understand the sample space (Ω) and event probabilities.
• Apply the addition and multiplication rules for probabilities.
• Differentiate between discrete and continuous random variables.
• Calculate expectation and variance for various distributions.
• Recognize and apply common probability distributions.

Action Steps

• 1. Identify the sample space and events.
• 2. Determine if events are independent or dependent.
• 3. Choose the appropriate probability rules (addition, multiplication, conditional).
• 4. Identify the type of random variable (discrete/continuous).
• 5. Select the correct probability distribution (Binomial, Poisson, Normal, etc.).
• 6. Calculate expectation and variance using the relevant formulas.
• 7. Apply inequalities like Chebyshev's for bounds.

Formulas

• $P(E \cup F) = P(E) + P(F) - P(E \cap F)$
• $P(E \cap F) = P(E|F)P(F)$
• $E[X] = \sum x_i p(x_i)$
• $Var(X) = E[X^2] - (E[X])^2$
• $F(x) = P(X \leq x) = \int_{-\infty}^{x} f(t) dt$
• $E[X] = \int_{-\infty}^{\infty} x f(x) dx$
• $Cov(X, Y) = E[XY] - E[X]E[Y]$
• $P(|X - \mu| \geq k\sigma) \leq 1/k^2$

Key Terms

• Sample Space (Ω): The set of all possible outcomes of an experiment.
• Event: A subset of the sample space.
• Conditional Probability: The probability of an event occurring given that another event has already occurred.
• Random Variable: A variable whose value is a numerical outcome of a random phenomenon.
• Expectation (E[X]): The weighted average of all possible values of a random variable.
• Variance (Var(X)): A measure of the spread or dispersion of a random variable's values.
• PMF (Probability Mass Function): Gives the probability that a discrete random variable is exactly equal to some value.
• PDF (Probability Density Function): Describes the relative likelihood for a continuous random variable to take on a given value.
• CDF (Cumulative Distribution Function): Gives the probability that a random variable is less than or equal to a certain value.

Pro Tips

• Use the complement rule P(EC) = 1 - P(E) for complex probabilities.
• Recognize that for continuous variables, P(X=x) = 0.
• Covariance is zero for independent variables, but the converse is not always true.
• Chebyshev's inequality provides a bound even without knowing the exact distribution.
• The Central Limit Theorem is powerful for approximating distributions with the Normal distribution.

Pitfalls to Avoid

• Confusing permutations (order matters) with combinations (order doesn't matter).
• Assuming independence when events are actually dependent.
• Incorrectly applying conditional probability formulas.
• Mistaking PMF for PDF or CDF.
• Forgetting the conditions for applying theorems like the Law of Large Numbers or CLT.

Real World Examples

• Quality control in manufacturing: Using Binomial or Poisson distributions to model defects.
• Financial modeling: Applying Normal or Log-Normal distributions to asset prices.
• Reliability engineering: Using Exponential or Gamma distributions to model component lifetime.
• Polling and surveys: Using concepts of sampling distributions and the Central Limit Theorem.

Timeline

• 17th Century: Development of basic probability theory by Pascal and Fermat.
• 18th Century: Work by De Moivre on the Normal distribution and Central Limit Theorem.
• 19th Century: Contributions by Gauss, Chebyshev, and Markov on inequalities and distributions.
• Early 20th Century: Formalization of probability theory by Kolmogorov.
• Mid-20th Century onwards: Expansion into stochastic processes, Bayesian statistics, and computational methods.

People

• Blaise Pascal: Pioneered probability theory.
• Pierre de Fermat: Co-founder of probability theory.
• Abraham de Moivre: Developed the Normal distribution approximation.
• Pafnuty Chebyshev: Known for Chebyshev's inequality.
• Andrey Markov: Known for Markov chains.
• Andrei Kolmogorov: Formalized modern probability theory.