Spectral theorem: Eigenvalue decomposition for symmetric matrices
Spectral theorem
We can decompose any symmetric matrix with the symmetric eigenvalue decomposition (SED)
where the matrix of is orthogonal (that is, ), and contains the eigenvectors of , while the diagonal matrix contains the eigenvalues of . |
Proof: The proof is by induction on the size of the matrix . The result is trivial for . Now let and assume the result is true for any matrix of size .
Consider the function of , . From the basic properties of the determinant, it is a polynomial of degree , called the characteristic polynomial of . By the fundamental theorem of algebra, any polynomial of degree has (possibly not distinct) complex roots; these are called the eigenvalues of . We denote these eigenvalues by .
If is an eigenvalue of , that is, , then must be non-invertible (see here). This means that there exists a non-zero real vector such that . We can always normalize so that . Thus, is real. That is, the eigenvalues of a symmetric matrix are always real.
Now consider the eigenvalue and an associated eigenvector . Using the Gram-Schmidt orthogonalization procedure, we can compute a matrix such that is orthogonal. By induction, we can write the symmetric matrix as, where is a matrix of eigenvectors, and are the eigenvalues of . Finally, we define the matrix . By construction the matrix is orthogonal.
We have
where we have exploited the fact that , and .
We have exhibited an orthogonal matrix such that is diagonal. This proves the theorem.