Vector Projection onto a Line
Interactive demonstration of the geometric meaning of the dot product as a projection, from Mathematics for Machine Learning (Deisenroth, Faisal & Ong), Section 3.8.1. A vector x is projected onto the 1-dimensional subspace (line) spanned by a unit vector b. The dot product bTx is the signed length of the projection; multiplying by b recovers the projected vector.
Mathematical Background
Given a unit vector \(\mathbf{b}\) (so \(\mathbf{b}^\top \mathbf{b} = 1\)), the line through the origin in the direction of \(\mathbf{b}\) is the 1-dimensional subspace \(U = \{\lambda \mathbf{b} : \lambda \in \mathbb{R}\}\). The projection of a vector \(\mathbf{x}\) onto this line is the point on the line closest to \(\mathbf{x}\).
The key insight is that the dot product \(\mathbf{b}^\top \mathbf{x} = \langle \mathbf{b}, \mathbf{x} \rangle\) measures exactly the signed length of the projection of \(\mathbf{x}\) onto the direction of \(\mathbf{b}\). Because \(\mathbf{b}\) is a unit vector, the projected vector is obtained by scaling \(\mathbf{b}\) by this scalar:
The projection can also be written as a matrix multiplication. Since the coefficient \(\mathbf{b}^\top \mathbf{x}\) is a scalar, we can write
where \(\mathbf{P} = \mathbf{b}\mathbf{b}^\top\) is the projection matrix onto the line. The residual \(\mathbf{x} - \pi_\mathbf{b}(\mathbf{x})\) is orthogonal to \(\mathbf{b}\), which is why the projection is the closest point on the line. Drag \(\mathbf{x}\) and \(\mathbf{b}\) in the figure above to see how the dot product \(\mathbf{b}^\top \mathbf{x}\) changes and how it determines the projected vector.
Reference: Deisenroth, M.P., Faisal, A.A., Ong, C.S. Mathematics for Machine Learning, Section 3.8.1 "Projection onto 1-Dimensional Subspaces (Lines)", Equations 3.40–3.41, Figures 3.8 and 3.10(a). Cambridge University Press, 2020. Freely available at mml-book.github.io.