 Projection (linear algebra)

"Orthogonal projection" redirects here. For the technical drawing concept, see orthographic projection. For a concrete discussion of orthogonal projections in finitedimensional linear spaces, see vector projection.
In linear algebra and functional analysis, a projection is a linear transformation P from a vector space to itself such that P^{2} = P. It leaves its image unchanged.^{[1]} Though abstract, this definition of "projection" formalizes and generalizes the idea of graphical projection. One can also consider the effect of a projection on a geometrical object by examining the effect of the projection on points in the object.
Contents
Simple example
Orthogonal projection
For example, the function which maps the point (x, y, z) in threedimensional space R^{3} to the point (x, y, 0) is a projection onto the xy plane. This function is represented by the matrix
The action of this matrix on an arbitrary vector is
and
therefore P = P^{2}, proving that P is indeed a projection.
Oblique projection
An example of a simple nonorthogonal (oblique) projection (for definition see below) is
Via matrix multiplication, one sees that
proving that P is indeed a projection.
The projection P is orthogonal if and only if α = 0.
Classification
Assume the underlying vector space is finite dimensional (therefore issues such as continuity of a projection need not be considered).
As stated in the introduction, a projection P is a linear transformation that is idempotent, meaning that P^{2} = P.
Let W be an underlying vector space. Suppose the subspaces U and V are the range and null space of P respectively. Then we have these basic properties:
 P is the identity operator I on U:
 We have a direct sum W = U ⊕ V. This means that every vector x may be decomposed uniquely in the manner x = u + v, where u is in U and v is in V. The decomposition is given by
The range and kernel of a projection are complementary, as are P and Q = I − P. The operator Q is also a projection and the range and kernel of P become the kernel and range of Q and viceversa.
We say P is a projection along V onto U (kernel/range) and Q is a projection along U onto V.
Decomposition of a vector space into direct sums is not unique in general. Therefore, given a subspace V, in general there are many projections whose range (or kernel) is V.
The spectrum of a projection is contained in {0, 1}, as . Only 0 and 1 can be an eigenvalue of a projection, the corresponding eigenspaces are the range and kernel of the projection.
If a projection is nontrivial it has minimal polynomial X^{2} − X = X(X − I), which factors into distinct roots, and thus P is diagonalizable.
Orthogonal projections
If the underlying vector space is endowed with an inner product, orthogonality and its attendant notions (such as the selfadjointness of a linear operator) become available. An orthogonal projection is a projection for which the range U and the null space V are orthogonal subspaces. A projection is orthogonal if and only if it is selfadjoint, which means that, in the context of real vector spaces, the associated matrix is symmetric relative to an orthonormal basis: P = P^{T} (for the complex case, the matrix is hermitian: P = (P^{*})^{T}). Indeed, if x and y are vectors in the domain of the projection, then Px ∈ U and y − Py ∈ V, and
where is the positivedefinite scalar product, so Px and y − Py are orthogonal for all x and y if and only if P = P^{T}P, which is equivalent to ( P = P^{T} and P = P^{2} ).^{[2]}
The simplest case is where the projection is an orthogonal projection onto a line. If u is a unit vector on the line, then the projection is given by
This operator leaves u invariant, and it annihilates all vectors orthogonal to u, proving that it is indeed the orthogonal projection onto the line containing u.^{[3]} A simple way to see this is to consider an arbitrary vector x as the sum of a component on the line (i.e. the projected vector we seek) and another perpendicular to it, . Applying projection, we get by the properties of the dot product of parallel and perpendicular vectors.
This formula can be generalized to orthogonal projections on a subspace of arbitrary dimension. Let u_{1}, ..., u_{k} be an orthonormal basis of the subspace U, and let A denote the nbyk matrix whose columns are u_{1}, ..., u_{k}. Then the projection is given by
 ^{[4]}
The matrix A^{T} is the partial isometry that vanishes on the orthogonal complement of U and A is the isometry that embeds U into the underlying vector space. The range of P_{A} is therefore the final space of A. It is also clear that A^{T}A is the identity operator on U.
The orthonormality condition can also be dropped. If u_{1}, ..., u_{k} is a (not necessarily orthonormal) basis, and A is the matrix with these vectors as columns, then the projection is
 ^{[5]}
The matrix A still embeds U into the underlying vector space but is no longer an isometry in general. The matrix (A^{T}A)^{−1} is a "normalizing factor" that recovers the norm. For example, the rank1 operator uu^{T} is not a projection if u ≠ 1. After dividing by u^{T}u = u^{2}, we obtain the projection u(u^{T}u)^{−1}u^{T} onto the subspace spanned by u.
If a matrix is nonsingular and A^{T} B = 0 (i.e., B is the null space matrix of A)^{[6]}, the following holds:
If the orthogonal condition is enhanced to A^{T} W B = 0 with W nonsingular and symmetric, the following holds:
All these formulas also hold for complex inner product spaces, provided that the conjugate transpose is used instead of the transpose.
Oblique projections
The term oblique projections is sometimes used to refer to nonorthogonal projections. These projections are also used to represent spatial figures in twodimensional drawings (see oblique projection), though not as frequently as orthogonal projections.
Oblique projections are defined by their range and null space. A formula for the matrix representing the projection with a given range and null space can be found as follows. Let the vectors u_{1}, ..., u_{k} form a basis for the range of the projection, and assemble these vectors in the nbyk matrix A. The range and the null space are complementary spaces, so the null space has dimension n − k. It follows that the orthogonal complement of the null space has dimension k. Let v_{1}, ..., v_{k} form a basis for the orthogonal complement of the null space of the projection, and assemble these vectors in the matrix B. Then the projection is defined by
This expression generalizes the formula for orthogonal projections given above.^{[7]}
Canonical forms
Any projection P = P^{2} on a vector space of dimension d over a field is a diagonalizable matrix, since its minimal polynomial is x^{2} − x, which splits into distinct linear factors. Thus there exists a basis in which P has the form
where r is the rank of P. Here I_{r} is the identity matrix of size r, and 0_{d−r} is the zero matrix of size d − r. If the vector space is complex and equipped with an inner product, then there is an orthonormal basis in which the matrix of P is ^{[8]}
 .
where σ_{1} ≥ σ_{2} ≥ ... ≥ σ_{k} > 0. The integers k, s, m and the real numbers σ_{i} are uniquely determined. Note that 2k + s + m = d. The factor I_{m} ⊕ 0_{s} corresponds to the maximal invariant subspace on which P acts as an orthogonal projection (so that P itself is orthogonal if and only if k = 0) and the σ_{i}blocks correspond to the oblique components.
Projections on normed vector spaces
When the underlying vector space X is a (not necessarily finitedimensional) normed vector space, analytic questions, irrelevant in the finitedimensional case, need to be considered. Assume now X is a Banach space.
Many of the algebraic notions discussed above survive the passage to this context. A given direct sum decomposition of X into complementary subspaces still specifies a projection, and vice versa. If X is the direct sum X = U ⊕ V, then the operator defined by P(u + v) = u is still a projection with range U and kernel V. It is also clear that P^{2} = P. Conversely, if P is projection on X, i.e. P^{2} = P, then it is easily verified that (I − P)^{2} = (I − P). In other words, (I − P) is also a projection. The relation I = P + (I − P) implies X is the direct sum Ran(P) ⊕ Ran(I − P).
However, in contrast to the finitedimensional case, projections need not be continuous in general. If a subspace U of X is not closed in the norm topology, then projection onto U is not continuous. In other words, the range of a continuous projection P must be a closed subspace. Furthermore, the kernel of a continuous projection (in fact, a continuous linear operator in general) is closed. Thus a continuous projection P gives a decomposition of X into two complementary closed subspaces: X = Ran(P) ⊕ Ker(P) = Ran(P) ⊕ Ran(I − P).
The converse holds also, with an additional assumption. Suppose U is a closed subspace of X. If there exists a closed subspace V such that X = U ⊕ V, then the projection P with range U and kernel V is continuous. This follows from the closed graph theorem. Suppose x_{n} → x and Px_{n} → y. One needs to show Px = y. Since U is closed and {Px_{n}} ⊂ U, y lies in U, i.e. Py = y. Also, x_{n} − Px_{n} = (I − P)x_{n} → x − y. Because V is closed and {(I − P)x_{n}} ⊂ V, we have x − y ∈ V, i.e. P(x − y) = Px − Py = Px − y = 0, which proves the claim.
The above argument makes use of the assumption that both U and V are closed. In general, given a closed subspace U, there need not exist a complementary closed subspace V, although for Hilbert spaces this can always be done by taking the orthogonal complement. For Banach spaces, a onedimensional subspace always has a closed complementary subspace. This is an immediate consequence of Hahn–Banach theorem. Let U be the linear span of u. By Hahn–Banach, there exists a bounded linear functional Φ such that φ(u) = 1. The operator P(x) = φ(x)u satisfies P^{2} = P, i.e. it is a projection. Boundedness of φ implies continuity of P and therefore Ker(P) = Ran(I − P) is a closed complementary subspace of U.
Applications and further considerations
Projections (orthogonal and otherwise) play a major role in algorithms for certain linear algebra problems:
 QR decomposition (see Householder transformation and Gram–Schmidt decomposition);
 Singular value decomposition
 Reduction to Hessenberg form (the first step in many eigenvalue algorithms).
 Linear regression
As stated above, projections are a special case of idempotents. Analytically, orthogonal projections are noncommutative generalizations of characteristic functions. Idempotents are used in classifying, for instance, semisimple algebras, while measure theory begins with considering characteristic functions of measurable sets. Therefore, as one can imagine, projections are very often encountered in the context operator algebras. In particular, a von Neumann algebra is generated by its complete lattice of projections.
Generalizations
More generally, given a map between normed vector spaces one can analogously ask for this map to be an isometry on the orthogonal complement of the kernel: that be an isometry; in particular it must be onto. The case of an orthogonal projection is when W is a subspace of V. In Riemannian geometry, this is used in the definition of a Riemannian submersion.
See also
 Centering matrix, which is an example of a projection matrix.
 Orthogonalization
 Invariant subspace
Notes
 ^ Meyer, pp 386+387
 ^ Meyer, p. 433
 ^ Meyer, p. 431
 ^ Meyer, equation (5.13.4)
 ^ Meyer, equation (5.13.3)
 ^ see also Properties of the leastsquares estimators in Linear least squares
 ^ Meyer, equation (7.10.39)
 ^ Doković, D. Ž. (August 1991). "Unitary similarity of projectors". Aequationes Mathematicae 42 (1): 220–224. doi:10.1007/BF01818492. http://www.springerlink.com/content/w3r57501226447m6/.
References
 Dunford, N.; Schwartz, J. T. (1958). Linear Operators, Part I: General Theory. Interscience.
 Meyer, Carl D. (2000). Matrix Analysis and Applied Linear Algebra. Society for Industrial and Applied Mathematics. ISBN 9780898714548. http://www.matrixanalysis.com/.
External links
 MIT Linear Algebra Lecture on Projection Matrices at Google Video, from MIT OpenCourseWare
 Planar Geometric Projections Tutorial  a simpletofollow tutorial explaining the different types of planar geometric projections.
Topics related to linear algebra Scalar · Vector · Vector space · Vector projection · Linear span · Linear map · Linear projection · Linear independence · Linear combination · Basis · Column space · Row space · Dual space · Orthogonality · Rank · Minor · Kernel · Eigenvalues and eigenvectors · Least squares regressions · Outer product · Inner product space · Dot product · Transpose · Gram–Schmidt process · Matrix decompositionCategories: Functional analysis
 Linear algebra
 Linear operators
Wikimedia Foundation. 2010.
Look at other dictionaries:
Linear algebra — R3 is a vector (linear) space, and lines and planes passing through the origin are vector subspaces in R3. Subspaces are a common object of study in linear algebra. Linear algebra is a branch of mathematics that studies vector spaces, also called … Wikipedia
Reflection (linear algebra) — In linear algebra, a reflection is a linear transformation that squares to the identity ( R 2 = I , where R is in K dimensional space), also known as an involution in the general linear group. In addition to reflections across hyperplanes, the… … Wikipedia
Trace (linear algebra) — In linear algebra, the trace of an n by n square matrix A is defined to be the sum of the elements on the main diagonal (the diagonal from the upper left to the lower right) of A, i.e., where aii represents the entry on the ith row and ith column … Wikipedia
Rank (linear algebra) — The column rank of a matrix A is the maximum number of linearly independent column vectors of A. The row rank of a matrix A is the maximum number of linearly independent row vectors of A. Equivalently, the column rank of A is the dimension of the … Wikipedia
Basis (linear algebra) — Basis vector redirects here. For basis vector in the context of crystals, see crystal structure. For a more general concept in physics, see frame of reference. In linear algebra, a basis is a set of linearly independent vectors that, in a linear… … Wikipedia
Minor (linear algebra) — This article is about a concept in linear algebra. For the unrelated concept of minor in graph theory, see Minor (graph theory). In linear algebra, a minor of a matrix A is the determinant of some smaller square matrix, cut down from A by… … Wikipedia
Cone (linear algebra) — In linear algebra, a (linear) cone is a subset of a vector space that is closed under multiplication by positive scalars. In other words, a subset C of a real vector space V is a cone if and only if λx belongs to C for any x in C and any positive … Wikipedia
Theorems and definitions in linear algebra — This article collects the main theorems and definitions in linear algebra. Vector spaces A vector space( or linear space) V over a number field² F consists of a set on which two operations (called addition and scalar multiplication, respectively) … Wikipedia
List of linear algebra topics — This is a list of linear algebra topics. See also list of matrices glossary of tensor theory. Contents 1 Linear equations 2 Matrices 3 Matrix decompositions 4 … Wikipedia
Projection (mathematics) — Commutativity of this diagram is the universality of projection π, for any map f and set X. Generally speaking, in mathematics, a projection is a mapping of a set (or of a mathematical structure) which is idempotent, which means that a projection … Wikipedia