EECS 442: Computer Vision & ML Fundamentals
Master core computer vision and machine learning techniques by understanding Fourier transforms, image filtering, KNN, backpropagation, model fitting, and generative architectures, focusing on practical application and common pitfalls.
Core Principles
- Fourier Transform Duality: Convolution in space equals multiplication in frequency; small features in space become large in frequency, and closer stripes in space correspond to dots farther out in frequency.
- Model Fitting Balance: Overfitting occurs with high training accuracy but low validation accuracy (gap), requiring more data or regularization; underfitting means low accuracy on both, necessitating a larger model or more features.
- Backpropagation Chain Rule: Compute gradients by propagating backward from the output (grad=1) through the computational graph, multiplying by local derivatives and summing for shared variables.
- Diffusion Model Training: Train a neural network to predict the noise component added to an image at various timesteps, then iteratively denoise from pure noise to generate images during sampling.
- Generative Adversarial Networks (GANs): A generator creates fake data to fool a discriminator, which learns to distinguish real from fake, iteratively improving both models through a minimax game.
Action Steps
- Debug PyTorch Shapes: 1) Write down the tensor shape after every layer. 2) Ensure `nn.Linear(in, out)`'s `in` parameter matches the last dimension of the input. 3) Resolve "(AxB and CxD)" errors by ensuring B=C.
- Perform Backpropagation: 1) Name every intermediate variable. 2) Execute forward pass (top-to-bottom). 3) Set output gradient to 1. 4) Execute backward pass (reverse order), calculating `grad = local_derivative × upstream_gradient`. 5) Add gradients for variables used multiple times.
- Prevent Data Leakage: Split data into distinct training, validation, and test sets; only use the test set once after hyperparameter tuning is complete to ensure unbiased performance evaluation.
Formulas
- Convolution Output Size (no padding): $$(N - K + 2P) / S + 1$$
- HOG Magnitude: $$m = \sqrt{Gx^2 + Gy^2}$$
- HOG Unsigned Orientation (same signs): $$\theta = \arctan\left(\frac{|Gy|}{|Gx|}\right)$$
- HOG Unsigned Orientation (opposite signs): $$\theta = 180^\circ - \arctan\left(\frac{|Gy|}{|Gx|}\right)$$
- Diffusion Forward Process: $$x_t = \sqrt{\bar{\alpha}_t}x_0 + \sqrt{1 - \bar{\alpha}_t}\epsilon$$
- Classifier-Free Guidance (CFG): $$\epsilon = \epsilon_u + s(\epsilon_c - \epsilon_u)$$
- GAN Objective (Minimax): $$\min_G \max_D \mathbb{E}_{x \sim p_{data}(x)}[\log D(x)] + \mathbb{E}_{z \sim p_z(z)}[\log(1 - D(G(z)))]$$
- 3D Homogeneous Coordinate Projection: $$x' = fX/Z, \quad y' = fY/Z$$
Key Terms
- L2 Regularization (λ): A technique where increasing λ simplifies the model (increases bias, decreases variance) and decreasing λ makes it more complex, controlling model capacity.
- Convolutional Kernel: A small matrix (e.g., Box ÷9, Gauss ÷16, Laplacian) applied across an image to perform operations like blurring, sharpening, or edge detection by weighted summation.
- HOG (Histogram of Oriented Gradients): A feature descriptor that counts occurrences of gradient orientation in localized portions of an image, used for object detection by capturing shape information.
- Classifier-Free Guidance (CFG): In diffusion models, a parameter `s` that scales the difference between conditional (prompted) and unconditional (empty prompt) noise predictions to control adherence to the prompt, with higher `s` leading to stronger guidance.
- Autoencoder: A neural network that learns an efficient data encoding (code z) by attempting to reconstruct its input from a compressed representation, useful for dimensionality reduction and feature learning.
Pro Tips
- Fourier Gallery Rule: Spectrum is the image structure rotated 90°; closer stripes in space correspond to dots farther out in frequency, while small features in space become large in frequency.
- Vectorize KNN: Use compiled array operations instead of Python loops for KNN calculations to achieve the same result much faster by avoiding per-iteration overhead.
- SGD for Optimization: Employ Stochastic Gradient Descent (SGD) for cheap, frequent updates, which helps escape saddle points and local minima, fitting memory better than Batch Gradient Descent (BGD).
- Median Filter Advantage: Use median filters to effectively remove impulse noise (spikes) and preserve image edges, unlike Gaussian filters which smear details, by sorting window values and taking the middle element.
Pitfalls to Avoid
- Incorrect K in KNN: Choosing a small K leads to "jagged" (overfit) models, while a large K results in "smooth" (underfit) models, both reducing generalization performance.
- Forward Noising in Diffusion Sampling: Attempting to generate images by forward noising (adding noise) instead of reverse sampling (iteratively denoising) will not produce desired outputs, wasting computational resources.
- Test Set Leakage: Using the test set for hyperparameter tuning or model selection leads to over-optimistic scores that do not reflect true generalization performance, causing misleading model evaluation.
Related cheat sheets
More like this