Interactive visualization of Principal Component Analysis, covariance ellipses, and eigenvectors for understanding dimensionality reduction through geometric interpretation
Measures how variables vary together. For centered data: Σ = (1/n)XᵀX. Diagonal elements are variances, off-diagonal are covariances.
Principal directions of maximum variance. Orthogonal vectors that define the axes of the covariance ellipse. First eigenvector points in direction of maximum variance.
Amount of variance explained by each eigenvector. Larger eigenvalue means more variance in that direction. The squared lengths of the covariance ellipse semi-axes.
Visual representation of the covariance matrix. Shows the shape and orientation of data distribution. Semi-axes aligned with eigenvectors, lengths proportional to √eigenvalues.
Subtracting the mean from each dimension: x_centered = x - μ. Essential for PCA to find directions of maximum variance around the mean.
Keeping only top-k principal components reduces dimensions while preserving maximum variance. Reconstruction error = sum of discarded eigenvalues.
For centered data matrix X, Σ = (1/n)XᵀX
Σ can be decomposed as Σ = QΛQᵀ where Q contains eigenvectors and Λ is diagonal matrix of eigenvalues
Projects data onto principal components (rotation and possibly projection)
Reconstructs data using only k principal components
Fraction of total variance explained by first principal component
Parametric equation for covariance ellipse at 1σ (multiply by k for kσ ellipse)
Subtract the mean from each dimension: x_centered = x - μ. This shifts the data to be centered at the origin.
Calculate Σ = (1/n)XᵀX where X is the centered data matrix. This captures how variables vary together.
Solve Σv = λv. Sort eigenvectors by eigenvalues in descending order. Larger eigenvalues indicate directions of more variance.
Transform data: z = Qᵀ(x - μ). This rotates the coordinate system to align with principal directions.
Keep only top-k components: z_k = Q_kᵀ(x - μ). This reduces dimensions while preserving maximum variance.
Reconstruct from k components: x̂ = Q_k z_k + μ. Reconstruction error = sum of discarded eigenvalues.
Project high-dimensional data to 2D or 3D for visualization while preserving as much variance as possible. Essential for exploratory data analysis.
Extract compact feature representations for machine learning. Used in face recognition (Eigenfaces), handwriting recognition, and more.
Remove noise by reconstructing with fewer components. Noise typically captured by smaller eigenvalues (later PCs).
Compress images by keeping top-k principal components. Achieve significant compression while preserving main features.
Detect outliers by measuring reconstruction error. Anomalies have high reconstruction error when using few PCs.
Handle correlated features in regression analysis. PCA transforms to orthogonal (uncorrelated) components.
Covariance ellipse becomes a circle (or axis-aligned). No preferred direction. Equal variance in all directions. Eigenvalues are equal.
Data trends upward. Covariance ellipse tilts at 45°. First eigenvector points in direction of trend.
Data trends downward. Covariance ellipse tilts at -45°. Inverse relationship between variables.
Degenerate ellipse becomes a line. One eigenvalue approaches zero. Data is essentially 1D. Perfect reconstruction with 1 PC.
High noise increases both eigenvalues equally. Makes ellipse more circular. Reduces the advantage of dimensionality reduction.
Eigenvectors are directions that don't change direction under the linear transformation. They're the 'natural axes' of the data distribution.