Rayleigh quotients

Alicia Y. Tsai; Authors: Laurent EI Ghaoui; Contributors: Phan Hoang, Tony Tin, Ha To Tam, Nguyen Van Kep, Le Dinh Nam, Juliette Decugis, Nguyen Quoc Cuong.; Giuseppe C. Calafiore

Rayleigh quotients

Theorem

For a symmetric matrix $A$ , we can express the smallest and largest eigenvalues, $\lambda_{\min} (A)$ and $\lambda_{\max} (A)$ , as

$\begin{align*} \lambda_{\min}(A) &= \min\limits_{x} \{x^TAx: x^Tx = 1\}, \end{align*}$

$\begin{align*} \lambda_{\max}(A) &= \max\limits_{x} \{x^TAx: x^Tx = 1\}. \end{align*}$

Proof: The proof of the expression above derives from the SED of the matrix, and the invariance of the Euclidean norm constraint under orthogonal transformations. We show this only for the largest eigenvalue; the proof for the expression for the smallest eigenvalue follows similar lines. Indeed, with $A= U\Lambda U^T$ , we have

$\begin{align*} \max\limits_{x} \{x^TAx: x^Tx = 1\} &= \max\limits_{x} \{x^TU\Lambda U^Tx: x^Tx = 1\}. \end{align*}$

Now we can define the new variable $\tilde{x} = U^Tx$ , so that $x = U \tilde{x} = u_i$ , and express the problem as

$\begin{align*} \max\limits_{x} \{x^TAx: x^Tx = 1\} &= \max\limits_{\hat{x}} \{\hat{x}^T\Lambda \hat{x}: \hat{x}^T\hat{x} = 1\} = \max\limits_{\hat{x}}\left\{\sum\limits_{i=1}^n \lambda_i \tilde{x}_i^2: \sum\limits_{i=1}^n \tilde{x}_i^2 =1 \right\}. \end{align*}$

Clearly, the maximum is less than $\lambda_{\max}$ . That upper bound is attained, with $\tilde{x}_i =1$ for an index $i$ such that $\lambda_i = \lambda_{\max}$ , and $\tilde{x}_i =0$ for $j \neq i$ . This proves the result. This corresponds to setting $x = U \tilde{x} = u_i$ , where $u_i$ is the eigenvector corresponding to $\lambda_i = \lambda_{\max}$ .

License

Icon for the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License

Theorem

License

Share This Book