Correlation and Regression Essentials

Understand and apply correlation and regression to analyze relationships between variables, determining the strength, direction, and predictive power of these associations.

Core Principles

• Correlation measures the linear relationship between two variables.
• Regression models the relationship to predict one variable based on another.
• The Pearson product-moment correlation coefficient (r) quantifies linear correlation.
• The coefficient of determination (r²) indicates the proportion of variance explained.
• Linear regression finds the 'line of best fit' to model data.
• Correlation does not imply causation.

Action Steps

• Identify the independent (x) and dependent (y) variables.
• Collect paired data for both variables.
• Calculate the sums: Σx, Σy, Σx², Σy², Σxy.
• Use the Pearson correlation formula to find 'r'.
• Interpret 'r' based on its value (-1 to +1).
• Calculate the coefficient of determination (r²).
• Interpret r² to understand the proportion of variance explained.
• Calculate the slope (b) and y-intercept (a) for the regression line.
• Formulate the linear regression equation: ŷ = a + bx.
• Use the equation to predict y-values for given x-values.

Formulas

• $r_{xy} = \frac{\sum_{i=1}^{n} (x_i - \bar{x})(y_i - \bar{y})}{\sqrt{\sum_{i=1}^{n} (x_i - \bar{x})^2 \sum_{i=1}^{n} (y_i - \bar{y})^2}}$
• $r = \frac{n(\sum xy) - (\sum x)(\sum y)}{\sqrt{[n\sum x^2 - (\sum x)^2][n\sum y^2 - (\sum y)^2]}}$
• $r^2 = \text{Coefficient of Determination}$
• $\hat{y} = a + bx$
• $b = \frac{n \sum xy - (\sum x)(\sum y)}{n \sum x^2 - (\sum x)^2}$
• $a = \frac{\sum y}{n} - b \left(\frac{\sum x}{n}\right)$

Key Terms

• Correlation: A statistical measure that describes the extent to which two variables change together.
• Regression: A statistical method used to predict the value of a dependent variable based on an independent variable.
• Pearson Product-Moment Correlation Coefficient (r): A measure of the strength and direction of the linear relationship between two quantitative variables, ranging from -1 to +1.
• Coefficient of Determination (r²): The proportion of the variance in the dependent variable that is predictable from the independent variable(s).
• Line of Best Fit: The line that best represents the trend in a scatter plot, minimizing the distance between the line and the data points.
• Scatter Plot: A graphical representation of the relationship between two variables, where each point represents a pair of values.
• Positive Correlation: As one variable increases, the other variable tends to increase.
• Negative Correlation: As one variable increases, the other variable tends to decrease.
• Zero Correlation: No discernible linear relationship between the two variables.
• Slope (b): In a linear regression equation, it represents the change in the dependent variable for a one-unit change in the independent variable.
• Y-intercept (a): In a linear regression equation, it represents the predicted value of the dependent variable when the independent variable is zero.

Timeline

• 1857-1936: Life of Karl Pearson, influential English mathematician and biostatician who contributed significantly to statistics.
• 1911: Karl Pearson founded the world's first university statistics department at University College of London.