In the standard geometrical interpretation, we view the OLS problem in the 
-dimensional vector space 
. For our purpose, we restate this
as follows: The OLS fitted vector is the solution of the quadratic program
where the column space of X, denoted 
, is the subspace
of 
spanned by the column vectors of X:
and distance in 
is measured by the Euclidean norm
The geometry of the solution to this problem is familiar to many students of
econometrics and other statistical disciplines. We can view the vector y
as a point above a plane representing 
. The quadratic
program seeks to find the point 
in the plane closest to y.
Intuition immediately suggests the solution: 
is the unique point
in the plane such that the line joining y and 
is perpendicular
to the plane. In Projection, we give the formal
justification of this intuition. We prove that 
where 
is
the unique orthogonal projector onto 
and that 
if X is full column rank.
The term ``orthogonal projection'' is an accurate description of the mapping
from y to 
. One can envision 
as the shadow projected
onto 
by y when the light travels along the direction
orthogonal to 
. This is depicted in Figure 2, where we have placed a spot light directly
overhead a vector. We will use this same geometric intuition to describe the
Gauss-Markov Theorem.
