A conversation prompted by Simon’s last post reminded me of an analogy that’s too excellent to be buried in a comments thread. It must be very well-known, but I’ll go ahead and describe it anyway.
The analogy is between complex numbers and linear operators on an inner
product space. Its best feature is that it makes important properties
of complex numbers correspond to important properties of operators:
The title of this post refers to Sheldon Axler’s beautiful book Linear
Algebra Done Right, which I’ve written about before. Most of what
I’ll say can be found in Chapter 7. It’s one of those texts that feels
like a piece of category theory even though it’s not actually about categories.
Today, all vector spaces are over ℂ and finite-dimensional.
Most (all?) of what I’ll say can be done in more sophisticated
functional-analytic settings, but I’ll stick to this most basic of
situations.
Fix a vector space X equipped with an inner
product. By an operator on X, I mean a linear map X→X.
Here’s how the analogy goes.
Complex numbers are like operators
This is the basis of everything that follows.
There’s not much
substance to this statement yet. For now, let’s just observe that both the
complex numbers and the operators on X form rings. I’ll write End(X) for the ring of operators on X, following the usual categorical custom.
(“End” stands for “endomorphisms”.)
The two rings ℂ and End(X) don’t seem very similar.
Unlike ℂ, the ring End(X) isn’t commutative and usually has
nontrivial zero-divisors. (Indeed, as long as dim(X)≥2, there is
some T∈End(X) with T≠0 but T2=0.) Perhaps surprisingly,
these differences don’t prevent the development of this useful analogy.
In some loose sense, we can pass back and forth between ℂ and End(X). In one direction, starting with a complex number λ, we get the
operator x↦λx. In elementary texts, this operator is
often written as λI, but I’ll almost always write it as just λ.
In the opposite direction, starting with an operator on X, we get not just a single complex
number but a collection of them — namely, its eigenvalues.
Complex conjugates are like adjoints
Every complex number z has a complex conjugate z*. Taking complex
conjugates defines a self-inverse automorphism of the ring ℂ.
Every linear map T:X→Y of inner product spaces has an adjoint T*:Y→X, characterized by the equation ⟨Tx,y⟩=⟨x,T*y⟩. In particular, every operator T on X has an adjoint T*, also an operator on X.
It’s almost true that taking adjoints
defines a self-inverse automorphism of End(X). The only obstruction is
that taking adjoints reverses the order of composition: (TS)*=S*T*. So actually, taking adjoints defines a pair of mutually
inverse ring isomorphisms
End(X)op←⟶End(X)
where End(X)op is the ring End(X) with its order of multiplication
reversed.
What about those back-and-forth passages between complex numbers and
operators?
First, start with a complex number λ; then the adjoint of the
operator λI is λ*I. That is, (λI)*=λ*I. This is why I’m writing z* for the complex conjugate of z, rather than the more common z¯.
Second, start with an operator T. Then the eigenvalues of T* are
exactly the conjugates of the eigenvalues of T. Why? Because taking
the adjoint defines an isomorphism of rings, so T−λ is invertible iff (T−λ)*=T*−λ* is.
Real numbers are like self-adjoint operators
A complex number z is real if and only if z=z*. By definition, an
operator T is self-adjoint if and only if T=T*.
Again, let’s look at the passages back and forth between ℂ and End(X). First, let λ∈ℂ. As long as X is
nontrivial, the operator λ is self-adjoint iff λ is real.
Second, if T is a self-adjoint operator then all its eigenvalues are
real. The converse isn’t true: an operator can have all real eigenvalues
without being self-adjoint. We’ll come back to that.
Any even half-serious endeavour involving self-adjoint operators makes use
of the theorem that classifies them, the spectral theorem. Loosely put,
this states that every self-adjoint operator is an orthogonal sum of
self-adjoint operators of the most simple kind: scalar multiplication by a
real number.
Precisely: given any self-adjoint operator T, there is a unique
orthogonal decomposition X=⨁λ∈ℝXλ such that for each λ, the restriction of T to Xλ is
multiplication by λ. Of course, all but finitely many of these
subspaces Xλ are trivial, the nontrivial ones are those for which λ is an eigenvalue, and Xλ is the eigenspace ker(T−λ).
Nonnegative real numbers are like positive operators
For a complex number z, the following are equivalent:
- (1) z is nonnegative, i.e. real and ≥0
- (2) z=w*w for some complex w
- (3) z=w*w (=w2) for some real w
- (4) z=w*w (=w2) for some nonnegative w
- (5) z=w*w (=w2) for a unique nonnegative w.
I’ll follow custom and say that an operator T is positive if it is
self-adjoint and each eigenvalue is ≥0. (Other names are “positive
semidefinite” and “nonnegative definite”. As we were recently discussing, the terminology around positive/nonnegative is a bit of a mess.) Note
that by definition, “positive” includes “self-adjoint”. This is just like
the convention that when we call a complex number “nonnegative”, we tacitly
include the condition “real”.
For an operator T on X, the following are equivalent:
- (1) T is positive, i.e. self-adjoint and each eigenvalue is ≥0
- (1.5) T is self-adjoint and ⟨Tx,x⟩≥0 for all x∈X
- (2) T=S*S for some inner product space Y and linear map S:X→Y
- (2.5) T=S*S for some operator S on X
- (3) T=S*S (=S2) for some self-adjoint operator S
- (4) T=S*S (=S2) for some positive operator S
- (5) T=S*S (=S2) for a unique positive operator S.
The implications 5⇒4⇒⋯⇒1 are all either
trivial or easy. The remaining implication, 1⇒5, follows
from the spectral theorem, using 1⇒5 of the result on
nonnegativity of numbers.
In particular, given λ∈ℂ, the operator λ is
positive iff the number λ is nonnegative (assuming that X is nontrivial).
And given an operator T, if T is positive then each eigenvalue of T is nonnegative (but not conversely).
The modulus of a complex number is like… the modulus of an operator?
What is the modulus of a complex number? Let’s answer this carefully, using
the theorem above on nonnegativity of complex numbers. Let z∈ℂ. By the theorem, z*z is nonnegative, so by the
theorem again, there is a unique nonnegative m such that z*z=m*m (=m2). This m is, of course, ∣z∣, the modulus of z.
What is the analogue for operators? Let’s use the theorem above on
positivity of operators. Let T∈End(X). By the theorem, T*T is positive, so by the theorem again, there is a unique positive M such
that T*T=M*M (=M2). I’ll
call M the modulus of T and write it as ∣T∣. I don’t
know whether the term “modulus” is standard here, and I’m pretty sure the
notation ∣T∣ isn’t — it’s risky, given the potential for
confusion with a norm. But I’ll use it anyway, to emphasize the analogy.
Complex numbers of unit modulus are like isometries
A complex
number z has unit modulus if and only if z*z=1, if and only if zz*=1. An operator T is an isometry if and only if T*T=1, if and only if TT*=1 (if and only if T preserves inner
products, if and only if T preserves distances). Isometries are more
often called unitary operators, but I find the term “isometry” more vivid.
Now that we have a definition of “modulus” for operators, we can ask: which operators are literally “of unit modulus”? In other words, which operators T satisfy ∣T∣=1? Here 1 is the identity operator. Certainly 1 is positive, so ∣T∣=1 if and only if T*T=1*1, if and only if T is an isometry. So the different parts of the analogy hang together nicely.
Once again, let’s go back and forth between complex numbers and operators.
Given λ∈ℂ, the operator λ is an isometry iff
the number λ is of unit modulus (again, assuming that X is
nontrivial). Given an operator T, if T is an isometry then all its
eigenvalues are of unit modulus. Again, the converse is false, and again,
we’ll come back to that.
Polar decomposition of complex numbers and operators
Any complex number z can be expressed as a product
z=up
where u is of unit modulus and p is nonnegative. Moreover, this p is uniquely determined as ∣z∣, and if z≠0 then u is uniquely determined by z too. (If z=0 then many choices of u are possible.)
Similarly, it’s a theorem that any operator T can be expressed as a composite
T=UP
where U is an isometry and P is positive. Moreover, this P is
uniquely determined as ∣T∣, and if T is invertible then U is uniquely determined by T too. (If T is not invertible then many
choices of U are possible.)
In the case where T is just multiplication by a scalar z, the second theorem (polar decomposition of operators) reduces to the first (polar decomposition of complex numbers).
If you prefer, you can decompose an operator in the other order too: an isometry
followed by a positive operator. To see this, decompose T* as UP; then T=P*U*=PU*. But U* is an isometry, since the
adjoint of an isometry is again an isometry — just as the conjugate
of a complex number of unit modulus is again of unit modulus.
And that’s the analogy.
Normal operators, and the fraying of the analogy
Like all analogies, this one eventually frays. Right at the start, we
noted a big difference between complex numbers and operators: multiplying
complex numbers is commutative, but composing operators isn’t. And another
one: there are no nonzero nilpotent complex numbers, but there are nonzero nilpotent operators.
I’ll explain the trouble this causes by talking about operators T that
satisfy the equation T*T=TT*. In a fit of no inspiration,
someone once called such operators normal, and the name stuck.
Now, all complex numbers z are “normal”, in the sense that z*z=zz*, but not all operators T are normal — for example, any
nonzero nilpotent is “abnormal”. So this is a wrinkle in the analogy. You
might conclude from this that the correct analogue for the complex numbers
is not the set of all operators, but just the normal ones. This idea has
in its favour that all self-adjoint operators and isometries (“real
numbers” and “numbers of unit modulus”) are normal — because an
operator commutes with both itself and its inverse.
However, the normal operators don’t form a ring, at
least, not under the usual operations. The class of normal operators is
closed under taking polynomials in one variable, but not under composition.
Indeed, the polar decomposition theorem implies that by composing two
normal operators, we can obtain any operator we like.
The normal operators are nevertheless a useful class, giving
further depth to the analogy. I clearly remember the first time I saw the
definition of normal operator: I was overwhelmed by the feeling that it was
an awful hack. “Someone,” I thought to myself, “simply wants a definition
that includes both self-adjoint operators and isometries, and they’ve
written down the first thing that came into their head.” Oh young, foolish
self; I was wrong. Here’s why:
Normal operators are exactly the right context for the spectral
theorem.
Recall that for an operator T, the spectral theorem says that X is the
orthogonal sum of the eigenspaces of T. This statement isn’t true for
all operators. Earlier on, I stated that it was true for all
self-adjoint operators, and that in that case, all the eigenvalues are
real. But there are certainly non-self-adjoint operators such that X is the orthogonal sum of the eigenspaces — multiplication by any non-real scalar is an example.
So which operators is the spectral theorem true for? Exactly the normal
ones. In other words:
Spectral theorem Let T be an operator on X. Then X is the orthogonal sum of the eigenspaces of T if and only if T is normal.
This says that multiplication by a scalar is a normal operator, that the
class of normal operators is closed under orthogonal sums, and that
combining these two constructions generates all possible normal operators. ‘Only if’ is easy; it’s ‘if’ that takes work. You can find a
proof in Linear Algebra Done Right.
We can read off two corollaries, both supporting the claim that “complex
numbers are like normal operators” is a better analogy than “complex
numbers are like operators”.
Corollary Let T be a normal operator. Then (i) T is
self-adjoint if
and only if all eigenvalues of T are real, and (ii) T is an isometry if and
only if all eigenvalues of T are of unit modulus.
We saw earlier that without the normality, the “only if” parts are true but the “if” parts fail.
Fundamental theorem of algebra for normal operators Let p be
a nonconstant polynomial over ℂ, and let T be a normal
operator. Then there exists a normal operator S such that p(S)=T.
For both proofs, all we have to do is observe that the class of operators T for which the result holds contains all operators of the form “multiply
by a scalar” and is closed under orthogonal sums. That’s all there is to
it!
As a longtime believer that
Matt Welch 
Juggalo Night Court [is] a daily ceremony
designed to smooth out trivial festival disputes under the
jurisprudence of Judge Upchuck the Clown and bailiff/professional
wrestler Mad Man Pondo.