Matrix Approximation using SVD

Any real matrix A can be decomposed as A = UΣVᵀ, where U and V are orthogonal matrices and Σ is a diagonal matrix of singular values σ₁ ≥ σ₂ ≥ ... ≥ σᵣ ≥ 0. The matrix can be written as a sum of rank-1 matrices:

The rank-k approximation Aₖ keeps only the first k terms: Aₖ = Σᵢ₌₁ᵏ σᵢ uᵢ vᵢᵀ. By the Eckart–Young theorem, this is the best rank-k approximation of A in both the spectral and Frobenius norms.

For a color image, the process is applied independently to each of the R, G, and B channels. Each channel yields its own set of singular values and singular vectors, and the three reconstructed channels are combined to form the color approximation. Because the three channels have similar but not identical singular value spectra, the rank-k truncation may affect color fidelity before it affects overall image structure.

Each term σᵢ uᵢ vᵢᵀ is itself a grayscale image (a rank-1 matrix). The early terms capture the overall brightness and large-scale structure, while the later terms add fine details and edges. Because σᵢ decreases rapidly, a small k can produce a recognizable image with far fewer values than the original matrix.