Spectral theorem: Eigenvalue decomposition for symmetric matrices
Spectral theorem
We can decompose any symmetric matrix
where the matrix of |
Proof: The proof is by induction on the size of the matrix
. The result is trivial for
. Now let
and assume the result is true for any matrix of size
.
Consider the function of ,
. From the basic properties of the determinant, it is a polynomial of degree
, called the characteristic polynomial of
. By the fundamental theorem of algebra, any polynomial of degree
has
(possibly not distinct) complex roots; these are called the eigenvalues of
. We denote these eigenvalues by
.
If is an eigenvalue of
, that is,
, then
must be non-invertible (see here). This means that there exists a non-zero real vector
such that
. We can always normalize
so that
. Thus,
is real. That is, the eigenvalues of a symmetric matrix are always real.
Now consider the eigenvalue and an associated eigenvector
. Using the Gram-Schmidt orthogonalization procedure, we can compute a
matrix
such that
is orthogonal. By induction, we can write the
symmetric matrix
as,
where
is a
matrix of eigenvectors, and
are the
eigenvalues of
. Finally, we define the
matrix
. By construction the matrix
is orthogonal.
We have
where we have exploited the fact that , and
.
We have exhibited an orthogonal matrix
such that
is diagonal. This proves the theorem.