Image Representation and Description

This cheat sheet covers key concepts in image representation and description, focusing on shape representation using chain codes, Fourier descriptors, and texture analysis techniques like co-occurrence matrices and Law's texture energy measures. It also delves into Local Binary Patterns (LBP) for texture-based detection and recognition, and Principal Component Analysis (PCA) for dimensionality reduction and face recognition.

Core Principles

• Image representation aims to describe information in a more suitable form than the image itself.
• Boundaries and regions are key elements for shape representation.
• Chain codes represent object boundaries as sequences of line segments.
• Fourier descriptors represent boundary points as complex numbers and use Fourier transforms for analysis.
• Texture analysis involves quantifying image surface characteristics.
• Co-occurrence matrices capture spatial relationships between pixel values.
• Law's texture energy measures use convolution filters to extract texture features.
• Local Binary Patterns (LBP) are robust texture descriptors used in face recognition and other applications.
• PCA reduces dimensionality by identifying principal components (eigenvectors) with the largest variance.
• Camera calibration determines intrinsic and extrinsic parameters to relate 3D world points to 2D image points.
• Stereo vision uses two images to infer depth information through triangulation and disparity calculation.

Action Steps

• To represent image information, convert it into forms more suitable than the raw image.
• Utilize chain codes to represent object boundaries as sequences of directional segments.
• Apply Fourier descriptors by treating boundary points as complex numbers and using Fourier transforms.
• Analyze texture by examining pixel neighborhoods and their arrangements.
• Compute co-occurrence matrices to capture spatial relationships between pixel values.
• Use Law's texture energy measures with convolution filters for feature extraction.
• Employ Local Binary Patterns (LBP) for robust texture analysis and recognition tasks.
• Reduce dimensionality using PCA by identifying and retaining principal components with the most variance.
• Calibrate cameras by estimating intrinsic and extrinsic parameters to establish world-to-image coordinate transformations.
• Recover depth information in stereo vision by finding correspondences and using triangulation.

Formulas

• $s(k) = x(k) + jy(k)$
• $a(u) = \frac{1}{K} \sum_{k=0}^{K-1} s(k)e^{-2\pi juk / K}$
• $s(k) = \frac{1}{K} \sum_{u=0}^{K-1} a(u)e^{2\pi juk / K}$
• $\chi^2(h_i, h_j) = \frac{1}{2} \sum_{m=1}^{K} \frac{[h_i(m) - h_j(m)]^2}{h_i(m) + h_j(m)}$
• $N_d(i, j) = \frac{C_d(i, j)}{\sum_{i,j} C_d(i, j)}$
• $S_d(i, j) = C_d(i, j) + C_{-d}(i, j)$
• $Energy = \sum_{a,b} P^2(a,b)$
• $Entropy = -\sum_{a,b} P(a,b) \log P(a,b)$
• $Contrast = \sum_{a,b} |a-b|^\kappa P(a,b)$
• $Inverse difference moment = \sum_{a,b: a \ne b} \frac{1}{|a-b|^\kappa} P(a,b)$
• $Correlation = \frac{\sum_{a,b} [ab P(a,b)] - \mu_x \mu_y}{\sigma_x \sigma_y}$
• $L_3 = (1, 2, 1)$
• $E_3 = (-1, 0, 1)$
• $S_3 = (-1, 2, -1)$
• $D(S, M) = \sum_{b=1}^{B} \min(S_b, M_b)$
• $L(S, M) = -\sum_{b=1}^{B} S_b \log M_b$
• $\chi^2(S, M) = \sum_{b=1}^{B} \frac{(S_b - M_b)^2}{S_b + M_b}$
• $x = f \frac{X}{Z}$
• $y = f \frac{Y}{Z}$
• $p = M_{int} P$
• $p = M_{ext} P_w$
• $p = M P_w$
• $Z = \frac{Tf}{d}$

Key Terms

• Chain Codes: Represent an object boundary by a connected sequence of straight line segments of specified length and direction.
• Fourier Descriptor: A method to represent boundary points using Fourier transforms, capturing shape information.
• Co-occurrence Matrix: A 2D array representing the frequency of pairs of pixel values occurring in specific spatial relationships.
• Law's Texture Energy Measures: A set of convolution filters used to assess gray level, edges, spots, ripples, and waves in textures.
• Local Binary Pattern (LBP): A texture descriptor that characterizes texture using the distribution of local pixel value comparisons.
• Principal Component Analysis (PCA): A technique for dimensionality reduction that identifies orthogonal components capturing maximum variance.
• Eigenfaces: The principal components derived from PCA applied to face images, used for face recognition.
• Camera Calibration: The process of estimating intrinsic and extrinsic camera parameters.
• Stereo Vision: A technique that infers 3D structure and depth from two or more 2D images taken from different viewpoints.
• Disparity: The difference in the positions of corresponding points in two stereo images, used to calculate depth.
• Epipolar Constraint: A geometric constraint that limits the search for corresponding points in stereo vision to an epipolar line.

Pro Tips

• Normalize chain codes to account for starting point variations.
• The first difference of a chain code is invariant to rotation.
• Use multiple displacement vectors for co-occurrence matrices to capture richer texture information.
• Uniform LBP patterns reduce feature vector dimensionality and improve robustness.
• PCA is effective for dimensionality reduction and feature extraction in face recognition (Eigenfaces).
• Camera calibration is crucial for accurately mapping 3D world points to 2D image points.
• Stereo vision relies on finding correspondences between images to determine depth.
• Disparity is inversely proportional to depth in stereo vision systems.

Pitfalls to Avoid

• Chain codes are sensitive to starting points and rotation if not normalized.
• Texture analysis can be complex; simple histogram comparisons might not capture all nuances.
• Overfitting can occur in neural networks if not properly regularized or if the training set is too small.
• In stereo vision, incorrect correspondences lead to significant depth errors.
• The epipolar constraint helps reduce search space but doesn't eliminate all ambiguity in correspondence matching.

Myth vs Reality

• A single image contains all necessary depth information.: A single 2D image inherently loses depth information; stereo vision or other depth cues are required.
• All features are equally important for classification.: Feature selection is crucial; irrelevant or redundant features can degrade performance and increase computational cost.

Real World Examples

• Face Recognition: Using PCA (Eigenfaces) to create a lower-dimensional representation of faces for efficient matching.
• Texture Classification: Using LBP histograms to model and classify different textures in images.
• 3D Reconstruction: Employing stereo vision techniques to determine the depth map of a scene from two camera views.

Statistics

• Natural images are approximately uniform LBP: 90%
• Number of features in a 24x24 sub-window: ~160,000

People

• Rafael C. Gonzalez and Richard E. Wood: Authors of 'Digital Image Processing', cited for chain code examples.
• Malik: Cited for Chi-square distance formula.
• Ahonen, Matas, He, Pietikäinen: Researchers cited for dealing with rotation in texture analysis.
• Ahonen, Hadid, Pietikäinen: Researchers cited for LBP face descriptor and concatenation.
• Viola and Jones: Developed a real-time face detection algorithm using Haar features, integral images, and AdaBoost.
• M. Turk, A. Pentland: Pioneers of Eigenfaces for face recognition.
• Fausett, L.: Author of 'Fundamentals of Neural Networks', cited for multi-layer neural network details.