Linear algebra II

Lecture 23

Dr. Greg Chism

University of Arizona
INFO 511 - Fall 2024

Eigenvalues + Eigenvectors

Eigenvalues and Eigenvectors

• Eigenvalues and eigenvectors decompose a matrix into its fundamental components.
• Eigenvalue equation: \(\mathbf{A} \mathbf{v} = \lambda \mathbf{v}\)

Calculating Eigenvalues and Eigenvectors

We will be using Python for this…

import numpy as np
from numpy.linalg import eig
A = np.array([[1, 2], [4, 5]])
eigenvals, eigenvecs = eig(A)
print("EIGENVALUES:", eigenvals)
print("EIGENVECTORS:", eigenvecs)

EIGENVALUES: [-0.46410162  6.46410162]
EIGENVECTORS: [[-0.80689822 -0.34372377]
 [ 0.59069049 -0.9390708 ]]

Eigen decomposition

• Decomposition formula: \(\mathbf{A} = \mathbf{Q} \mathbf{L} \mathbf{Q}^{-1}\)

• \(\mathbf{Q}\) is the matrix of eigenvectors, \(\mathbf{L}\) is the diagonal matrix of eigenvalues, and \(\mathbf{Q}^{-1}\) is the inverse of \(\mathbf{Q}\)

Recomposing matrices

from numpy import diag
from numpy.linalg import inv
Q = eigenvecs
L = diag(eigenvals)
R = inv(Q)
B = Q @ L @ R
print(B)

[[1. 2.]
 [4. 5.]]

Applications in Data Science and Machine Learning

• Principal Component Analysis (PCA): Reduces dimensionality while preserving variance.

• Eigenvalues in system stability: Determine stability in control systems and differential equations.

Special Types of Matrices

Identity Matrix: Diagonal of 1s, other elements are 0s.

\[ \mathbf{I} = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} \]

Diagonal Matrix: Non-zero elements only on the diagonal.

\[ \mathbf{D} = \begin{bmatrix} 4 & 0 & 0 \\ 0 & 5 & 0 \\ 0 & 0 & 6 \end{bmatrix} \]

Triangular Matrix: Triangular shape of non-zero elements (upper \(\mathbf{U}\), lower \(\mathbf{L}\)).

\[ \mathbf{U} = \begin{bmatrix}1 & 2 & 3 \\0 & 4 & 5 \\0 & 0 & 6\end{bmatrix} \\ \mathbf{L} = \begin{bmatrix} 1 & 0 & 0 \\ 2 & 3 & 0 \\ 4 & 5 & 6 \end{bmatrix} \]

ae-16-pca

Principal Component Analysis