OLS as Orthogonal Projection

Let and note that . As a result, is orthogonal to every element of : for all ,

Therefore, because ,

so that minimizes over all .