Inner Product

The inner product is a fundamental operation that pairs two vectors and returns a scalar, capturing how much the two vectors “align” with each other. It is defined abstractly on any vector space — but throughout this material we work in Rn\mathbb{R}^n with the standard inner product, which has its own well-known name.

Inner Product vs Dot Product

The two terms are often used interchangeably, but they are not quite the same:

  • Inner product is the general concept: any operation ⟨⋅,⋅⟩\langle \cdot, \cdot \rangle on a vector space that is symmetric, linear in each argument, and positive-definite.
  • Dot product is the specific inner product on Rn\mathbb{R}^n defined by component-wise multiply-then-sum.

So every dot product is an inner product, but not every inner product is a dot product. In Rn\mathbb{R}^n — the only setting we encounter in this material — the standard inner product is the dot product, so the two names refer to the exact same operation. The notation differs by convention:

  • ⟨u,v⟩\langle \mathbf{u}, \mathbf{v} \rangle — angle brackets, the general inner-product notation
  • u⋅v\mathbf{u} \cdot \mathbf{v} — center dot, the specific dot-product notation

Both appear in this material, sometimes within the same expression. They mean the same thing in Rn\mathbb{R}^n.

Algebraic Definition

The inner product of u=(u1,…,un)⊤\mathbf{u} = (u_1, \ldots, u_n)^\top and v=(v1,…,vn)⊤\mathbf{v} = (v_1, \ldots, v_n)^\top in Rn\mathbb{R}^n is:

⟨u,v⟩=u⋅v=u⊤v=∑i=1nuivi\langle \mathbf{u}, \mathbf{v} \rangle = \mathbf{u} \cdot \mathbf{v} = \mathbf{u}^\top \mathbf{v} = \sum_{i=1}^{n} u_i v_i

In Rn\mathbb{R}^n, this specific inner product is also called the dot product or scalar product.

This is simply a component-wise multiply-then-sum.

In R3\mathbb{R}^3:

⟨(1,2,3)⊤, (4,5,6)⊤⟩=1⋅4+2⋅5+3⋅6=4+10+18=32\langle (1, 2, 3)^\top,\, (4, 5, 6)^\top \rangle = 1 \cdot 4 + 2 \cdot 5 + 3 \cdot 6 = 4 + 10 + 18 = 32

Geometric Interpretation

The same operation has a clean geometric meaning:

For u,v∈Rn\mathbf{u}, \mathbf{v} \in \mathbb{R}^n, the inner product equals:

⟨u,v⟩=∥u∥ ∥v∥cos⁡θ\langle \mathbf{u}, \mathbf{v} \rangle = \|\mathbf{u}\| \, \|\mathbf{v}\| \cos \theta

where θ∈[0,π]\theta \in [0, \pi] is the angle between the two vectors and ∥⋅∥\|\cdot\| denotes the Euclidean norm.

This form makes three key facts immediately visible:

  • Orthogonality: ⟨u,v⟩=0\langle \mathbf{u}, \mathbf{v} \rangle = 0 if and only if θ=90°\theta = 90° — the vectors are perpendicular.
  • Sign: the inner product is positive when θ<90°\theta < 90° (vectors point in roughly the same direction), zero at 90°90°, and negative when θ>90°\theta > 90° (vectors point away from each other).
  • Self-pairing: ⟨v,v⟩=∥v∥2\langle \mathbf{v}, \mathbf{v} \rangle = \|\mathbf{v}\|^2, so the norm of a vector is ∥v∥=⟨v,v⟩\|\mathbf{v}\| = \sqrt{\langle \mathbf{v}, \mathbf{v} \rangle}.

Projection

The geometric form ⟨a,b⟩=∥a∥ ∥b∥cos⁡θ\langle \mathbf{a}, \mathbf{b} \rangle = \|\mathbf{a}\| \, \|\mathbf{b}\| \cos\theta makes the inner product the natural tool for projecting one vector onto another — extracting how much of a\mathbf{a} points along b\mathbf{b}.

The scalar projection of a∈Rn\mathbf{a} \in \mathbb{R}^n onto a non-zero b∈Rn\mathbf{b} \in \mathbb{R}^n — the signed length of the shadow a\mathbf{a} casts on the direction of b\mathbf{b} — is

compba=∥a∥cos⁡θ=⟨a,b⟩∥b∥\text{comp}_\mathbf{b} \mathbf{a} = \|\mathbf{a}\| \cos\theta = \frac{\langle \mathbf{a}, \mathbf{b} \rangle}{\|\mathbf{b}\|}

It is positive when the angle is acute, zero when the vectors are perpendicular, and negative when the angle is obtuse.

Read compba\text{comp}_\mathbf{b} \mathbf{a} as “the component of a\mathbf{a} along b\mathbf{b}” — and note this is a single (signed) number, not a vector. It is a signed length in the range [−∥a∥,∥a∥][-\|\mathbf{a}\|, \|\mathbf{a}\|] — hitting +∥a∥+\|\mathbf{a}\| when a\mathbf{a} is parallel to b\mathbf{b}, −∥a∥-\|\mathbf{a}\| when antiparallel, and zero when perpendicular.

The vector projection of a∈Rn\mathbf{a} \in \mathbb{R}^n onto a non-zero b∈Rn\mathbf{b} \in \mathbb{R}^n — the actual displacement along b\mathbf{b} that a\mathbf{a} shadows onto — is the scalar projection scaled by the unit vector b^=b/∥b∥\hat{\mathbf{b}} = \mathbf{b}/\|\mathbf{b}\|:

projba=⟨a,b⟩∥b∥2 b\text{proj}_\mathbf{b} \mathbf{a} = \frac{\langle \mathbf{a}, \mathbf{b} \rangle}{\|\mathbf{b}\|^2} \, \mathbf{b}

Read projba\text{proj}_\mathbf{b} \mathbf{a} as “the projection of a\mathbf{a} onto b\mathbf{b}” — and note this is a vector, parallel to b\mathbf{b} (or anti-parallel, if the angle is obtuse), with length ∣compba∣|\text{comp}_\mathbf{b} \mathbf{a}|. It is the literal arrow you’d draw by dropping a perpendicular from the tip of a\mathbf{a} down to the line through b\mathbf{b}.

Reading the inner product as a weighted projection. The inner product splits as

⟨a,b⟩=∥a∥cos⁡θ⋅∥b∥=compba⋅∥b∥\langle \mathbf{a}, \mathbf{b} \rangle = \|\mathbf{a}\| \cos\theta \cdot \|\mathbf{b}\| = \text{comp}_\mathbf{b} \mathbf{a} \cdot \|\mathbf{b}\|

— the scalar projection of a\mathbf{a} onto b\mathbf{b}, scaled by the length of b\mathbf{b}. So whenever we take an inner product, we are measuring “how much a\mathbf{a} aligns with b\mathbf{b}” weighted by the magnitude of b\mathbf{b} — useful whenever b\mathbf{b} has its own physical meaning, e.g. a surface normal whose length encodes an area element.

A common confusion: ⟨a,b⟩\langle \mathbf{a}, \mathbf{b} \rangle is not the scalar projection. Both contain a cos⁡θ\cos\theta factor and both feel like a measure of “alignment”, but they differ by a length:

⟨a,b⟩=compba⋅∥b∥.\langle \mathbf{a}, \mathbf{b} \rangle = \text{comp}_\mathbf{b} \mathbf{a} \cdot \|\mathbf{b}\|.

The two coincide exactly when ∥b∥=1\|\mathbf{b}\| = 1 — dotting against a unit vector (a basis vector ei\mathbf{e}_i, a unit normal n^\hat{\mathbf{n}}) makes the dot product equal to the scalar projection. Most early dot-product practice happens against unit basis vectors, where a⋅ei=ai\mathbf{a} \cdot \mathbf{e}_i = a_i is both the ii-th coordinate and the scalar projection onto ei\mathbf{e}_i — easy to over-generalize from. For any non-unit b\mathbf{b}, the inner product carries the extra ∥b∥\|\mathbf{b}\| factor.

Two sanity checks that the two operations are different:

  • ⟨a,b⟩=⟨b,a⟩\langle \mathbf{a}, \mathbf{b} \rangle = \langle \mathbf{b}, \mathbf{a} \rangle is symmetric, but compba≠compab\text{comp}_\mathbf{b} \mathbf{a} \neq \text{comp}_\mathbf{a} \mathbf{b} in general — the projection picks out one vector as the “direction onto which we project”.
  • The scalar projection is a length; the inner product is length-times-length (when a,b\mathbf{a}, \mathbf{b} carry length units). The units don’t match.

Key Properties

For all u,v,w∈Rn\mathbf{u}, \mathbf{v}, \mathbf{w} \in \mathbb{R}^n and α∈R\alpha \in \mathbb{R}:

  • Commutativity: ⟨u,v⟩=⟨v,u⟩\langle \mathbf{u}, \mathbf{v} \rangle = \langle \mathbf{v}, \mathbf{u} \rangle
  • Linearity: ⟨αu+w, v⟩=α⟨u,v⟩+⟨w,v⟩\langle \alpha\mathbf{u} + \mathbf{w},\, \mathbf{v} \rangle = \alpha \langle \mathbf{u}, \mathbf{v} \rangle + \langle \mathbf{w}, \mathbf{v} \rangle
  • Positive-definiteness: ⟨v,v⟩≥0\langle \mathbf{v}, \mathbf{v} \rangle \geq 0, with equality only when v=0\mathbf{v} = \mathbf{0}