The Spectral Theorem

Theorem. (Schur) If A is an $n \times n$ matrix, then there is a unitary matrix U such that $UAU^{-1}$ is upper triangular. (Recall that a matrix is upper triangular if the entries below the main diagonal are 0.)

Proof. Use induction on n, the size of A. If A is $1 \times 1$ , it's already upper triangular, so there's nothing to do.

Take $n > 1$ , and assume the result is true for $(n - 1) \times (n - 1)$ matrices.

Over the complex numbers, the characteristic polynomial of A must have at least one root, so A has an eigenvalue $\lambda$ and a corresponding eigenvector $\vec{v}$ :

$$A\vec{v} = \lambda\vec{v}.$$

By dividing $\vec{v}$ by its length, I can assume $\|\vec{v}\| = 1$ .

Build a basis with $\vec{v}$ as the first vector, then use Gram-Schmidt to construct an orthonormal basis with $\vec{v}$ as the first vector:

$${\cal B} = \{\vec{v}, \bvec{u_2}, \ldots, \bvec{u_n}\}.$$

Let

$$U = \left[\matrix{\uparrow & \uparrow & & \uparrow \cr \vec{v} & \bvec{u_2} & \ldots & \bvec{u_n} \cr \downarrow & \downarrow & & \downarrow \cr}\right].$$

Since the columns are orthonormal, U is unitary. Then

$$U^*AU = [{\rm std} \to {\cal B}]\cdot A\cdot [{\cal B} \to {\rm std}] = \left[\matrix{\lambda & \hbox{junk} \cr \matrix{0 \cr \vdots \cr 0 \cr} & \left[\matrix{& \vphantom{x} & \cr \hphantom{x} & B & \hphantom{x} \cr & \vphantom{x} & \cr}\right] \cr}\right].$$

The issue here is why the first column of the last matrix is what it is. To see this, notice that $[{\rm std} \to {\cal B}]\cdot A\cdot [{\cal B} \to {\rm std}]$ is the matrix $[T]_{{\cal B},{\cal B}}$ of the linear transformation $T(\vec{x}) = A\vec{x}$ relative to ${\cal B}$ . But

$$T(\vec{v}) = A\vec{v} = \lambda\vec{v} = (\lambda,0\ldots,0)_{\cal B},$$

so this is the first column of $[T]_{{\cal B},{\cal B}}$ .

The matrix B is $(n - 1) \times (n - 1)$ , so by induction I can find a unitary matrix V such that $V^*BV$ is upper triangular. Then

$$\left[\matrix{1 & \matrix{0 & \cdots & 0 \cr} \cr \matrix{0 \cr \vdots \cr 0 \cr} & \left[\matrix{& \vphantom{x} & \cr \hphantom{x} & V & \hphantom{x} \cr & \vphantom{x} & \cr}\right] \cr}\right]$$

is unitary, and

$$\left[\matrix{1 & \matrix{0 & \cdots & 0 \cr} \cr \matrix{0 \cr \vdots \cr 0 \cr} & \left[\matrix{& \vphantom{x} & \cr \hphantom{x} & V & \hphantom{x} \cr & \vphantom{x} & \cr}\right] \cr}\right]^*U^*AU \left[\matrix{1 & \matrix{0 & \cdots & 0 \cr} \cr \matrix{0 \cr \vdots \cr 0 \cr} & \left[\matrix{& \vphantom{x} & \cr \hphantom{x} & V & \hphantom{x} \cr & \vphantom{x} & \cr}\right] \cr}\right]$$

is upper triangular.

This completes the induction step.

Theorem. ( The Spectral Theorem) If A is Hermitian, then there is a unitary matrix U and a diagonal matrix D such that

$$U^* A U = D.$$

(Note that since U is unitary, $U^* = U^{-1}$ .)

Proof. Find a unitary matrix U such that $U^*AU = T$ , where T is upper triangular. Then since $A^* = A$ ,

$$(U^*AU)^* = T^*, \quad U^*A^*U = T^*, \quad U^*AU = T^*.$$

But then $T = T^*$ . T is upper triangular, $T^*$ (the conjugate transpose) is lower triangular, so T must be diagonal.

Corollary. ( The Principal Axis Theorem) If A is a real symmetric matrix, there is an orthogonal matrix O and a diagonal matrix D such that

$$O^T A O = D.$$

(Note that since O is orthogonal, $O^T = O^{-1}$ .)

Proof. Real symmetric matrices are Hermitian and real orthogonal matrices are unitary, so the result follows from the Spectral Theorem.

I showed earlier that for a Hermitian matrix (or in the real case, a symmetric matrix), eigenvectors corresponding to different eigenvalues are perpendicular. Consequently, if I have an $n \times n$ Hermitian matrix (or in the real case, an $n \times n$ symmetric matrix) with n different eigenvalues, the corresponding eigenvectors form an orthogonal basis. I can get an orthonormal basis --- and hence, a unitary diagonalizing matrix (or in the real case, an orthogonal diagonalizing matrix) --- by simply dividing each vector by its length.

Things are a little more complicated if I have fewer than n eigenvalues. The Spectral Theorem guarantees that I'll have n independent eigenvectors, but some eigenvalues will have several eigenvectors. In this case, I'd need to use Gram-Schmidt on the eigenvectors for each eigenvalue to get an orthogonal set of eigenvectors for each eigenvalue. Eigenvectors corresponding to different eigenvalues are still perpendicular by the result cited earlier, so the orthogonal sets for the eigenvalues fit together to form an orthogonal basis. As before, I get an orthonormal basis by dividing each vector by its length.

To keep the computations simple, I'll stick to the first case (n different eigenvalues) in the examples below.

Example. Let

$$A = \left[\matrix{ -1 & 2 & 0 \cr 2 & 2 & 0 \cr 0 & 0 & 3 \cr}\right].$$

Find an orthogonal matrices O which diagonalizes A. Find $O^{-1}$ and the corresponding diagonal matrix.

The characteristic polynomial is

$$(x - 3)[(x - 2)(x + 1) - (2)(2)] = -(x - 3)^2 (x + 2).$$

The eigenvalues are $x = 3$ and $x = -2$ .

For $x = 3$ , the eigenvector matrix is

$$A - 3 I = \left[\matrix{ -4 & 2 & 0 \cr 2 & -1 & 0 \cr 0 & 0 & 0 \cr}\right] \quad\to\quad \left[\matrix{ 2 & -1 & 0 \cr 0 & 0 & 0 \cr 0 & 0 & 0 \cr}\right]$$

This gives the independent eigenvectors $(1, 2, 0)$ and $(0, 0, 1)$ . Dividing them by their lengths, I get $\dfrac{1}{\sqrt{5}} (1, 2, 0)$ and $(0, 0, 1)$ .

For $x = -2$ , the eigenvector matrix is

$$A + 2 I = \left[\matrix{ 1 & 2 & 0 \cr 2 & 4 & 0 \cr 0 & 0 & 5 \cr}\right] \quad\to\quad \left[\matrix{ 1 & 2 & 0 \cr 0 & 0 & 1 \cr 0 & 0 & 0 \cr}\right]$$

This gives the independent eigenvector $(-2, 1, 0)$ . Dividing it by its length, I get $\dfrac{1}{\sqrt{5}} (-2, 1, 0)$ .

Thus, the orthogonal diagonalizing matrix is

$$O = \left[\matrix{ \dfrac{1}{\sqrt{5}} & 0 & -\dfrac{2}{\sqrt{5}} \cr \noalign{\vskip2pt} \dfrac{2}{\sqrt{5}} & 0 & \dfrac{1}{\sqrt{5}} \cr \noalign{\vskip2pt} 0 & 1 & 0 \cr}\right].$$

Then

$$O^{-1} = O^T = \left[\matrix{ \dfrac{1}{\sqrt{5}} & \dfrac{2}{\sqrt{5}} & 0 \cr \noalign{\vskip2pt} 0 & 0 & 1 \cr \noalign{\vskip2pt} -\dfrac{2}{\sqrt{5}} & \dfrac{1}{\sqrt{5}} & 0 \cr}\right].$$

The diagonal matrix is

$$O^T A O = \left[\matrix{ 3 & 0 & 0 \cr 0 & 3 & 0 \cr 0 & 0 & -2 \cr}\right].\quad\halmos$$

Example. Let

$$A = \left[\matrix{3 & 1 - 2i \cr 1 + 2i & -1 \cr}\right].$$

Find a unitary matrix U which diagonalizes A.

Since A is Hermitian, the Spectral Theorem applies.

The characteristic polynomial is

$$\det (A - \lambda I) = \det\left[\matrix{3 - \lambda & 1 - 2i \cr 1 + 2i & -1 - \lambda \cr}\right] = \lambda^2 - 2\lambda - 8 = (\lambda - 4)(\lambda + 2).$$

The eigenvalues are 4 and -2.

For $\lambda = 4$ , partial row reduction gives

$$A - 4I = \left[\matrix{-1 & 1 - 2i \cr 1 + 2i & -5 \cr}\right] \to \left[\matrix{-1 & 1 - 2i \cr 0 & 0 \cr}\right].$$

(Since this is a $2 \times 2$ $A - \lambda I$ matrix, I know that the second row must be a multiple of the first.)

An eigenvector $(a,b)$ must satisfy

$$(-1)a + (1 - 2i)b = 0.$$

I can get a solution by swapping the -1 and the $1 - 2i$ and negating the -1 to give 1: $(a,b) = (1 - 2i,1)$ is an eigenvector for $\lambda = 4$ .

For $\lambda = -2$ , partial row reduction gives

$$A + 2I = \left[\matrix{5 & 1 - 2i \cr 1 + 2i & 1 \cr}\right] \to \left[\matrix{5 & 1 - 2i \cr 0 & 0 \cr}\right].$$

Using the same technique as I used for $\lambda = 4$ , I see that $(a,b) = (1 - 2i,-5)$ is an eigenvector for $\lambda = -2$ .

The result I proved earlier says that these eigenvectors are automatically perpendicular. Check by taking their complex dot product:

$$(1 - 2i,1)\cdot (1 - 2i,-5) = (1 - 2i)(1 + 2i) + (1)(-5) = 5 - 5 = 0.$$

Find the lengths of the eigenvectors:

$$\|(1 - 2i,1)\| = \sqrt{6}, \quad \|(1 - 2i,-5)\| = \sqrt{30}.$$ Next, divide each eigenvector by its length:

$$\dfrac{1}{\sqrt{6}}(1 - 2i,1), \quad \dfrac{1}{\sqrt{30}}(1 - 2i,-5).$$

Finally, U is constructed using these normalized vectors as the columns:

$$U = \left[\matrix{\dfrac{1 - 2i}{\sqrt{6}} & \dfrac{1 - 2i}{\sqrt{30}} \cr \noalign{\vskip2pt} \dfrac{1}{\sqrt{6}} & -\dfrac{5}{\sqrt{30}} \cr}\right].\quad\halmos$$

Send comments about this page to: Bruce.Ikenaga@millersville.edu.

Bruce Ikenaga's Home Page

Copyright 2014 by Bruce Ikenaga