Statistical Concepts Cheat Sheet

This cheat sheet covers fundamental statistical concepts, from data collection and variable types to measures of central tendency, dispersion, probability, and hypothesis testing. It provides definitions, formulas, and key principles for understanding and analyzing data.

Core Principles

Statistics is the science of collecting, organizing, summarizing, and analyzing data.
Descriptive statistics summarize sample data; inferential statistics generalize to populations.
Population is the entire group; a sample is a subset.
Variables can be qualitative (categorical) or quantitative (numeric).
Levels of measurement include Nominal, Ordinal, Interval, and Ratio.
Observational studies observe; designed experiments manipulate variables.
Bias occurs when a sample does not represent the target population.
Measures of central tendency (mean, median, mode) describe the center of data.
Measures of dispersion (range, standard deviation, variance) describe data spread.
Probability deals with the likelihood of events, with basic rules for calculation.
Hypothesis testing involves formulating null and alternative hypotheses to test claims about parameters.

Action Steps

Identify research objectives clearly.
Choose an appropriate sampling or experimental design.
Collect data systematically.
Describe data using descriptive statistics (tables, graphs).
Perform inference to test hypotheses or estimate parameters.
Identify variable types (qualitative/quantitative, discrete/continuous).
Determine the appropriate level of measurement (Nominal, Ordinal, Interval, Ratio).
Select the correct statistical test based on data type and research question.
Interpret results in the context of the problem, considering statistical significance and practical importance.

Formulas

$ \bar{x} = \frac{1}{n}\sum_{i=1}^{n} x_i \quad \text{(sample mean)} $
$ \mu = \frac{1}{N}\sum_{i=1}^{N} X_i \quad \text{(population mean)} $
$ R = \max(x_i) - \min(x_i) \quad \text{(range)} $
$ \sigma = \sqrt{\frac{1}{N}\sum_{i=1}^{N}(X_i - \mu)^2} \quad \text{(population standard deviation)} $
$ s = \sqrt{\frac{1}{n-1}\sum_{i=1}^{n}(x_i - \bar{x})^2} \quad \text{(sample standard deviation)} $
$ P(E \cup F) = P(E) + P(F) \quad \text{(disjoint events)} $
$ P(E \cup F) = P(E) + P(F) - P(E \cap F) \quad \text{(general addition rule)} $
$ P(E^c) = 1 - P(E) \quad \text{(complement rule)} $
$ P(E \cap F) = P(E) \, P(F) \quad \text{(independent events)} $
$ P(F \mid E) = \frac{P(E \cap F)}{P(E)} \quad \text{(conditional probability)} $
$ ^nP_r = \frac{n!} {(n-r)!} \quad \text{(permutations)} $
$ ^nC_r = \frac{n!} {r!(n-r)!} \quad \text{(combinations)} $
$ \mu = E(X) = \sum x \, P(X = x) \quad \text{(expected value)} $
$ \sigma = \sqrt{\sum (x - \mu)^2 \, P(X = x)} \quad \text{(standard deviation for discrete random variable)} $
$ P(X = x) = \binom{n}{x} p^{x} (1-p)^{\,n-x} \quad \text{(binomial probability mass function)} $
$ Z = \frac{x - \mu}{\sigma} \quad \text{(z-score)} $
$ \sigma_{\bar{X}} = \frac{\sigma}{\sqrt{n}} \quad \text{(standard error of the mean)} $
$ E = z_{\alpha/2}\sqrt{\frac{\hat{p}(1-\hat{p})}{n}} \quad \text{(margin of error for proportion)} $
$ E = t_{\alpha/2, n-1}\frac{s}{\sqrt{n}} \quad \text{(margin of error for mean, unknown \sigma)} $
$ z = \frac{\hat{p} - p_0}{\sqrt{\dfrac{p_0(1-p_0)}{n}}} \quad \text{(z-test for proportion)} $
$ t = \frac{\bar{x} - \mu_0}{s/\sqrt{n}} \quad \text{(t-test for mean)} $

Key Terms

Statistics: The science of collecting, organizing, summarizing, and analyzing data to draw conclusions with confidence.
Descriptive Statistics: Summarizes sample data using methods like tables and graphs.
Inferential Statistics: Uses a sample to make statements or generalizations about a larger population.
Population: The entire group of interest in a study.
Sample: A subset of individuals or items selected from a population.
Parameter: A characteristic of a population (often denoted by Greek letters like \mu or \sigma).
Statistic: A characteristic of a sample (often denoted by Roman letters like \bar{x} or s).
Qualitative Variable: A variable that describes non-numeric attributes (e.g., color, gender).
Quantitative Variable: A variable that measures numeric quantities (e.g., height, age).
Discrete Variable: A quantitative variable whose values can only take on a finite or countably infinite number of values (e.g., number of siblings).
Continuous Variable: A quantitative variable that can take on any value within a given range (e.g., temperature, height).
Nominal Level: Categories with no inherent order (e.g., blood type).
Ordinal Level: Ordered categories where the differences are not meaningful (e.g., satisfaction ratings).
Interval Level: Ordered data with meaningful differences but no true zero (e.g., Celsius temperature).
Ratio Level: Ordered data with meaningful differences and a true zero, allowing meaningful ratios (e.g., height, weight).
Observational Study: Researchers observe subjects and measure variables without assigning treatments.
Designed Experiment: Researchers manipulate treatments and assign them to experimental units to observe effects.
Bias: Systematic error that causes a sample statistic to differ from the population parameter.
Sampling Bias: Bias introduced by the sampling method, leading to undercoverage or overcoverage.
Non-response Bias: Bias resulting from systematic differences between respondents and non-respondents.
Response Bias: Bias introduced by the way questions are asked or how interviewers interact with respondents.
Mean: The average of a dataset; calculated by summing all values and dividing by the number of values.
Median: The middle value in an ordered dataset.
Mode: The most frequently occurring value in a dataset.
Range: The difference between the maximum and minimum values in a dataset.
Standard Deviation: A measure of the average distance of data points from the mean.
Variance: The square of the standard deviation.
Z-score: Measures how many standard deviations a data point is from the mean.
Percentile: The value below which a certain percentage of observations fall.
Quartiles: Values that divide the data into four equal parts (Q1, Median, Q3).
IQR: Interquartile Range (Q3 - Q1), a measure of spread resistant to outliers.
Scatter Diagram: A graph plotting pairs of data points (x, y) to visualize relationships.
Correlation Coefficient (r): Measures the strength and direction of a linear relationship between two variables (-1 to +1).
Regression Line: A line that best fits the data points in a scatter plot, used for prediction.
Residual: The difference between an observed value and its predicted value from the regression line.
R-squared (R^2): The proportion of the variance in the dependent variable that is predictable from the independent variable(s).
Simpson's Paradox: A phenomenon where an association between two variables reverses or disappears when a third variable is introduced.
Random Process: A process whose outcomes are uncertain but long-run proportions are predictable.
Law of Large Numbers: As the number of trials increases, the observed proportion of an event converges to its true probability.
Empirical Probability: Calculated based on observed frequencies from experiments or data.
Classical Probability: Calculated based on equally likely outcomes (e.g., rolling a die).
Subjective Probability: Based on personal belief or judgment.
Disjoint Events: Events that cannot occur at the same time (mutually exclusive).
Independent Events: The occurrence of one event does not affect the probability of the other.
Conditional Probability: The probability of an event occurring given that another event has already occurred.
Permutation: An arrangement of items where order matters and repetition is not allowed.
Combination: A selection of items where order does not matter.
Discrete Random Variable: A variable that can only take on a finite or countably infinite number of values.
Probability Distribution: Assigns probabilities to each possible value of a random variable.
Binomial Distribution: Models the number of successes in a fixed number of independent Bernoulli trials.
Poisson Distribution: Models the number of rare events occurring in a fixed interval of time or space.
Normal Distribution: A symmetric, bell-shaped distribution defined by its mean and standard deviation.
Standard Normal Distribution: A normal distribution with a mean of 0 and a standard deviation of 1 (Z-distribution).
Sampling Distribution: The probability distribution of a statistic (e.g., sample mean) calculated from all possible samples of a given size.
Central Limit Theorem: States that the sampling distribution of the sample mean becomes approximately normal as sample size increases, regardless of population distribution.
Point Estimate: A single value used to estimate a population parameter.
Confidence Interval (CI): A range of values likely to contain the population parameter, with a certain level of confidence.
Margin of Error: The amount added and subtracted from the point estimate to form the confidence interval.
Hypothesis Testing: A statistical method used to make decisions or draw conclusions about a population based on sample data.
Null Hypothesis (H0): A statement of no effect or no difference, typically containing an equality.
Alternative Hypothesis (H1): A statement that contradicts the null hypothesis, representing the effect or difference we seek evidence for.
Type I Error: Rejecting the null hypothesis when it is actually true (false positive).
Type II Error: Failing to reject the null hypothesis when it is actually false (false negative).
P-value: The probability of observing a test statistic as extreme as, or more extreme than, the one calculated from the sample, assuming the null hypothesis is true.
Significance Level (\alpha): The probability threshold for rejecting the null hypothesis (commonly 0.05).

Timeline

Early 18th Century: Development of probability theory (Pascal, Fermat).
18th-19th Century: Foundations of statistical inference laid by mathematicians like Gauss and Laplace.
Early 20th Century: Development of key statistical methods by Fisher, Pearson, and Neyman (ANOVA, chi-square, hypothesis testing).
Mid-20th Century: Rise of computational statistics and the development of the t-distribution and bootstrap methods.
Late 20th - 21st Century: Explosion of data analysis with computing power, machine learning, and big data techniques.

People

Blaise Pascal: Co-founder of probability theory.
Pierre de Fermat: Co-founder of probability theory.
Carl Friedrich Gauss: Developed the normal distribution and methods for analyzing data.
Pierre-Simon Laplace: Extended probability theory and introduced Bayesian concepts.
Sir Ronald Fisher: Pioneer of modern statistics; developed ANOVA, maximum likelihood estimation.
Karl Pearson: Developed the chi-square test and correlation coefficient.
Jerzy Neyman: Developed confidence intervals and hypothesis testing framework.
William Sealy Gosset (Student): Developed the t-distribution.
Bradley Efron: Developed the bootstrap method.