Eigenvalues and Eigenvectors

For a square matrix $A$ , an eigenvector is a non-zero vector that the matrix only stretches or shrinks — its direction is preserved (up to sign) under multiplication by $A$ . The factor by which it gets scaled is the corresponding eigenvalue.

These pairs reveal how the linear map $\mathbf{x} \mapsto A\mathbf{x}$ behaves along its “natural axes”: every other vector twists toward or away from these eigendirections, but the eigenvectors themselves only get rescaled.

Definition

For a square matrix $A \in \mathbb{R}^{n \times n}$ , a non-zero vector $\mathbf{v} \in \mathbb{R}^n$ is an eigenvector of $A$ with eigenvalue $\lambda \in \mathbb{R}$ if:

A \mathbf{v} = \lambda \mathbf{v}

The pair $(\lambda, \mathbf{v})$ is called an eigenpair of $A$ . The requirement $\mathbf{v} \neq \mathbf{0}$ is essential — the equation is trivially satisfied by the zero vector for any $\lambda$ , so without it the definition would be vacuous.

The scalar $\lambda$ may be complex even when $A$ has real entries, but for the symmetric matrices most often encountered in this material — Hessian matrices, covariance matrices, and so on — every eigenvalue is guaranteed to be real.

Finding Eigenvalues

Start from the eigenvalue equation $A\mathbf{v} = \lambda \mathbf{v}$ and move everything to one side:

A\mathbf{v} - \lambda \mathbf{v} = \mathbf{0} \quad \Longrightarrow \quad (A - \lambda I) \mathbf{v} = \mathbf{0}

(The identity matrix sneaks in because $\lambda \mathbf{v} = \lambda I \mathbf{v}$ — it’s the only way to subtract a scalar from a matrix and keep the equation matrix-shaped.)

The key observation: we’re looking for a non-zero $\mathbf{v}$ that the matrix $A - \lambda I$ sends to $\mathbf{0}$ . A healthy, invertible matrix never does that — multiplying any non-zero vector by an invertible matrix gives back another non-zero vector. So $A - \lambda I$ has to be non-invertible — it must collapse some direction down to $\mathbf{0}$ . From the determinant properties, this is exactly the condition

\det(A - \lambda I) = 0

which gives a single equation in the unknown $\lambda$ .

The characteristic polynomial of a square matrix $A \in \mathbb{R}^{n \times n}$ is

p_A(\lambda) = \det(A - \lambda I)

Its roots are exactly the eigenvalues of $A$ . For an $n \times n$ matrix, $p_A$ is a polynomial of degree $n$ in $\lambda$ , so $A$ has at most $n$ eigenvalues (counted with multiplicity).

Picture $\lambda$ as a tuning knob you can dial. For most settings, the matrix $A - \lambda I$ is perfectly healthy — its determinant is some non-zero number, and the matrix has an inverse. But at a few special, isolated values of $\lambda$ — exactly the eigenvalues — the matrix collapses, its determinant snaps to zero, and a direction in space gets crushed to $\mathbf{0}$ . The characteristic polynomial $p_A(\lambda)$ is just the determinant of $A - \lambda I$ written as a function of $\lambda$ , and “finding eigenvalues” is the same as hunting for the values where this polynomial crosses zero.

In practice the recipe is three steps:

Form $A - \lambda I$ by subtracting $\lambda$ from every diagonal entry of $A$ (off-diagonal entries stay put).
Compute its determinant. Expanding gives a polynomial in $\lambda$ — the characteristic polynomial $p_A(\lambda)$ .
Solve $p_A(\lambda) = 0$ for $\lambda$ . Each root is an eigenvalue.

Finding Eigenvectors

Once you have an eigenvalue $\lambda$ , the matching eigenvectors are the directions that $A - \lambda I$ flattens to zero — and you already know at least one such direction must exist, because that collapsing-a-direction property is exactly what made $\lambda$ an eigenvalue in the first place. To pin it down, plug the value of $\lambda$ back into $A - \lambda I$ and solve

(A - \lambda I) \mathbf{v} = \mathbf{0}

for $\mathbf{v}$ . This is just a linear system: for a $2 \times 2$ matrix it boils down to a single linear relation between $v_1$ and $v_2$ , leaving one free parameter to slide along.

One thing to expect: if $\mathbf{v}$ is an eigenvector, so is $2\mathbf{v}$ , $-\mathbf{v}$ , or any other non-zero scalar multiple — multiplying both sides of $A\mathbf{v} = \lambda \mathbf{v}$ by a constant doesn’t change anything. That’s not a flaw in the recipe; it just means the eigenvector’s direction is what’s pinned down, not its length. By convention you pick a clean representative with simple integer entries (or unit length).

Worked Example

Find the eigenvalues and eigenvectors of $A = \begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix}$ .

Step 1. Form $A - \lambda I$ by subtracting $\lambda$ from each diagonal entry:

A - \lambda I = \begin{pmatrix} 4 - \lambda & 1 \\ 2 & 3 - \lambda \end{pmatrix}

Step 2. Compute its determinant — this is the characteristic polynomial:

p_A(\lambda) = \det(A - \lambda I) = (4 - \lambda)(3 - \lambda) - 1 \cdot 2 = \lambda^2 - 7\lambda + 10

Step 3. Set $p_A(\lambda) = 0$ and solve. Factoring is easiest here:

\lambda^2 - 7\lambda + 10 = (\lambda - 5)(\lambda - 2) = 0 \quad \Longrightarrow \quad \lambda_1 = 5,\ \lambda_2 = 2

So $A$ has two eigenvalues, $5$ and $2$ .

Step 4. For each eigenvalue, plug it back in and solve $(A - \lambda I)\mathbf{v} = \mathbf{0}$ to find the matching eigenvector.

For $\lambda_1 = 5$ :

\begin{pmatrix} -1 & 1 \\ 2 & -2 \end{pmatrix} \begin{pmatrix} v_1 \\ v_2 \end{pmatrix} = \mathbf{0} \quad \Longrightarrow \quad v_1 = v_2 \quad \Longrightarrow \quad \mathbf{v}_1 = \begin{pmatrix} 1 \\ 1 \end{pmatrix}

For $\lambda_2 = 2$ :

\begin{pmatrix} 2 & 1 \\ 2 & 1 \end{pmatrix} \begin{pmatrix} v_1 \\ v_2 \end{pmatrix} = \mathbf{0} \quad \Longrightarrow \quad 2v_1 + v_2 = 0 \quad \Longrightarrow \quad \mathbf{v}_2 = \begin{pmatrix} 1 \\ -2 \end{pmatrix}

Sanity check — both pairs satisfy $A\mathbf{v} = \lambda \mathbf{v}$ :

A \mathbf{v}_1 = \begin{pmatrix} 4 \cdot 1 + 1 \cdot 1 \\ 2 \cdot 1 + 3 \cdot 1 \end{pmatrix} = \begin{pmatrix} 5 \\ 5 \end{pmatrix} = 5 \mathbf{v}_1

A \mathbf{v}_2 = \begin{pmatrix} 4 \cdot 1 + 1 \cdot (-2) \\ 2 \cdot 1 + 3 \cdot (-2) \end{pmatrix} = \begin{pmatrix} 2 \\ -4 \end{pmatrix} = 2 \mathbf{v}_2

The same recipe extends to larger matrices — for $3 \times 3$ matrices the characteristic polynomial is cubic in $\lambda$ and yields up to three eigenvalues, each with its own eigenvector(s) found by the same plug-and-solve step.

Sum and Product of Eigenvalues

For any square matrix, the eigenvalues come paired with two simple bookkeeping identities — they sum to the trace, and they multiply to the determinant.

For a square matrix $A$ with eigenvalues $\lambda_1, \ldots, \lambda_n$ (counted with multiplicity), the trace equals the sum of the eigenvalues:

\operatorname{tr}(A) = \lambda_1 + \lambda_2 + \cdots + \lambda_n.

For a square matrix $A$ with eigenvalues $\lambda_1, \ldots, \lambda_n$ (counted with multiplicity), the determinant equals the product of the eigenvalues:

\det(A) = \lambda_1 \cdot \lambda_2 \cdots \lambda_n.

Both follow from a single observation: the characteristic polynomial can be expanded directly or read off its roots, and matching coefficients recovers the identities at once. The $2 \times 2$ case spells it out.

Expanding $\det(A - \lambda I)$ for $A = \begin{pmatrix} a & b \\ c & d \end{pmatrix}$ gives

\det(A - \lambda I) = (a - \lambda)(d - \lambda) - bc = \lambda^2 - (a + d)\lambda + (ad - bc).

The same polynomial, with $\lambda_1, \lambda_2$ as its roots, also factors as

(\lambda - \lambda_1)(\lambda - \lambda_2) = \lambda^2 - (\lambda_1 + \lambda_2)\lambda + \lambda_1 \lambda_2.

Matching coefficients (Vieta’s formulas) reads off both identities at once:

\lambda_1 + \lambda_2 = a + d = \operatorname{tr}(A), \qquad \lambda_1 \lambda_2 = ad - bc = \det(A).

For an $n \times n$ matrix the picture is the same — match coefficients of the degree- $n$ characteristic polynomial with its fully-factored form.

Diagonal Matrices: A Free Lunch

For a diagonal matrix the entire computation collapses to inspection — no characteristic polynomial needed.

If $D \in \mathbb{R}^{n \times n}$ is diagonal with entries $d_1, d_2, \ldots, d_n$ along its diagonal, then:

The eigenvalues of $D$ are exactly the diagonal entries: $\lambda_i = d_i$ for $i \in \{1, \ldots, n\}$ .
The corresponding eigenvectors are the standard basis vectors: $D \mathbf{e}_i = d_i \mathbf{e}_i$ .

The reason is mechanical: multiplying $D$ by $\mathbf{e}_i$ picks out the $i$ -th column of $D$ , and since $D$ is diagonal, that column is $d_i \mathbf{e}_i$ . So $\mathbf{e}_i$ maps to itself, scaled by the matching diagonal entry — the very definition of an eigenpair.

The characteristic polynomial confirms it directly: $D - \lambda I$ is also diagonal, with entries $d_i - \lambda$ along its diagonal, and the determinant of a diagonal matrix is the product of its diagonal entries, so

p_D(\lambda) = \det(D - \lambda I) = \prod_{i=1}^{n} (d_i - \lambda)

which is already in factored form — its roots are visibly $d_1, \ldots, d_n$ .

D = \begin{pmatrix} 3 & 0 & 0 \\ 0 & -1 & 0 \\ 0 & 0 & 7 \end{pmatrix}

has eigenvalues $\lambda_1 = 3,\ \lambda_2 = -1,\ \lambda_3 = 7$ with corresponding eigenvectors $\mathbf{e}_1, \mathbf{e}_2, \mathbf{e}_3$ . The characteristic polynomial is

p_D(\lambda) = (3 - \lambda)(-1 - \lambda)(7 - \lambda)

with roots reading straight off the diagonal.