Row and column spaces

File:Matrix Rows.svg

The row vectors of a matrix. The row space of this matrix is the vector space spanned by the row vectors.

File:Matrix Columns.svg

The column vectors of a matrix. The column space of this matrix is the vector space spanned by the column vectors.

In linear algebra, the column space (also called the range or image) of a matrix A is the span (set of all possible linear combinations) of its column vectors. The column space of a matrix is the image or range of the corresponding matrix transformation.

Let <math>F</math> be a field. The column space of an Template:Math matrix with components from <math>F</math> is a linear subspace of the m-space <math>F^m</math>. The dimension of the column space is called the rank of the matrix and is at most Template:Math.<ref name="ReferenceA">Linear algebra, as discussed in this article, is a very well established mathematical discipline for which there are many sources. Almost all of the material in this article can be found in Lay 2005, Meyer 2001, and Strang 2005.</ref> A definition for matrices over a ring <math>R</math> is also possible.

The row space is defined similarly.

The row space and the column space of a matrix Template:Mvar are sometimes denoted as Template:Math and Template:Math respectively.<ref>Template:Cite book</ref>

This article considers matrices of real numbers. The row and column spaces are subspaces of the real spaces <math>\R^n</math> and <math>\R^m</math> respectively.<ref>Template:Harvtxt</ref>

Overview

Let Template:Mvar be an Template:Mvar-by-Template:Mvar matrix. Then

• Template:Math,<ref>Template:Harvtxt</ref>
• Template:Math = number of pivots in any echelon form of Template:Mvar,
• Template:Math = the maximum number of linearly independent rows or columns of Template:Mvar.<ref>Template:Harvtxt</ref>

If the matrix represents a linear transformation, the column space of the matrix equals the image of this linear transformation.

The column space of a matrix Template:Mvar is the set of all linear combinations of the columns in Template:Mvar. If Template:Math, then Template:Math.

Given a matrix Template:Mvar, the action of the matrix Template:Mvar on a vector Template:Math returns a linear combination of the columns of Template:Mvar with the coordinates of Template:Math as coefficients; that is, the columns of the matrix generate the column space.

Example

Given a matrix Template:Mvar:

<math>

 J =
 \begin{bmatrix}
   2 & 4 & 1 & 3 & 2\\
   -1 & -2 & 1 & 0 & 5\\
   1 & 6 & 2 & 2 & 2\\
   3 & 6 & 2 & 5 & 1
 \end{bmatrix}

</math> the rows are <math>\mathbf{r}_1 = \begin{bmatrix} 2 & 4 & 1 & 3 & 2 \end{bmatrix}</math>, <math>\mathbf{r}_2 = \begin{bmatrix} -1 & -2 & 1 & 0 & 5 \end{bmatrix}</math>, <math>\mathbf{r}_3 = \begin{bmatrix} 1 & 6 & 2 & 2 & 2 \end{bmatrix}</math>, <math>\mathbf{r}_4 = \begin{bmatrix} 3 & 6 & 2 & 5 & 1 \end{bmatrix}</math>. Consequently, the row space of Template:Mvar is the subspace of <math>\R^5</math> spanned by Template:Math. Since these four row vectors are linearly independent, the row space is 4-dimensional. Moreover, in this case it can be seen that they are all orthogonal to the vector Template:Math (Template:Math is an element of the kernel of Template:Mvar ), so it can be deduced that the row space consists of all vectors in <math>\R^5</math> that are orthogonal to Template:Math.

Column space

Definition

Let Template:Mvar be a field of scalars. Let Template:Math be an Template:Math matrix, with column vectors Template:Math. A linear combination of these vectors is any vector of the form

<math>c_1 \mathbf{v}_1 + c_2 \mathbf{v}_2 + \cdots + c_n \mathbf{v}_n,</math>

where Template:Math are scalars. The set of all possible linear combinations of Template:Math is called the column space of Template:Mvar. That is, the column space of Template:Mvar is the span of the vectors Template:Math.

Any linear combination of the column vectors of a matrix Template:Mvar can be written as the product of Template:Mvar with a column vector:

<math>\begin{array} {rcl}

A \begin{bmatrix} c_1 \\ \vdots \\ c_n \end{bmatrix} & = & \begin{bmatrix} a_{11} & \cdots & a_{1n} \\ \vdots & \ddots & \vdots \\ a_{m1} & \cdots & a_{mn} \end{bmatrix} \begin{bmatrix} c_1 \\ \vdots \\ c_n \end{bmatrix} = \begin{bmatrix} c_1 a_{11} + \cdots + c_{n} a_{1n} \\ \vdots \\ c_{1} a_{m1} + \cdots + c_{n} a_{mn} \end{bmatrix} = c_1 \begin{bmatrix} a_{11} \\ \vdots \\ a_{m1} \end{bmatrix} + \cdots + c_n \begin{bmatrix} a_{1n} \\ \vdots \\ a_{mn} \end{bmatrix} \\ & = & c_1 \mathbf{v}_1 + \cdots + c_n \mathbf{v}_n \end{array}</math>

Therefore, the column space of Template:Mvar consists of all possible products Template:Math, for Template:Math. This is the same as the image (or range) of the corresponding matrix transformation.

Example

If <math>A = \begin{bmatrix} 1 & 0 \\ 0 & 1 \\ 2 & 0 \end{bmatrix}</math>, then the column vectors are Template:Math and Template:Math. A linear combination of v₁ and v₂ is any vector of the form <math display="block">c_1 \begin{bmatrix} 1 \\ 0 \\ 2 \end{bmatrix} + c_2 \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} = \begin{bmatrix} c_1 \\ c_2 \\ 2c_1 \end{bmatrix}</math> The set of all such vectors is the column space of Template:Mvar. In this case, the column space is precisely the set of vectors Template:Math satisfying the equation Template:Math (using Cartesian coordinates, this set is a plane through the origin in three-dimensional space).

Basis

The columns of Template:Mvar span the column space, but they may not form a basis if the column vectors are not linearly independent. Fortunately, elementary row operations do not affect the dependence relations between the column vectors. This makes it possible to use row reduction to find a basis for the column space.

For example, consider the matrix

<math>A = \begin{bmatrix} 1 & 3 & 1 & 4 \\ 2 & 7 & 3 & 9 \\ 1 & 5 & 3 & 1 \\ 1 & 2 & 0 & 8 \end{bmatrix}.</math>

The columns of this matrix span the column space, but they may not be linearly independent, in which case some subset of them will form a basis. To find this basis, we reduce Template:Mvar to reduced row echelon form:

<math>\begin{bmatrix} 1 & 3 & 1 & 4 \\ 2 & 7 & 3 & 9 \\ 1 & 5 & 3 & 1 \\ 1 & 2 & 0 & 8 \end{bmatrix}

\sim \begin{bmatrix} 1 & 3 & 1 & 4 \\ 0 & 1 & 1 & 1 \\ 0 & 2 & 2 & -3 \\ 0 & -1 & -1 & 4 \end{bmatrix} \sim \begin{bmatrix} 1 & 0 & -2 & 1 \\ 0 & 1 & 1 & 1 \\ 0 & 0 & 0 & -5 \\ 0 & 0 & 0 & 5 \end{bmatrix} \sim \begin{bmatrix} 1 & 0 & -2 & 0 \\ 0 & 1 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \end{bmatrix}.</math><ref>This computation uses the Gauss–Jordan row-reduction algorithm. Each of the shown steps involves multiple elementary row operations.</ref> At this point, it is clear that the first, second, and fourth columns are linearly independent, while the third column is a linear combination of the first two. (Specifically, Template:Math.) Therefore, the first, second, and fourth columns of the original matrix are a basis for the column space:

<math>\begin{bmatrix} 1 \\ 2 \\ 1 \\ 1\end{bmatrix},\;\;

\begin{bmatrix} 3 \\ 7 \\ 5 \\ 2\end{bmatrix},\;\; \begin{bmatrix} 4 \\ 9 \\ 1 \\ 8\end{bmatrix}.</math> Note that the independent columns of the reduced row echelon form are precisely the columns with pivots. This makes it possible to determine which columns are linearly independent by reducing only to echelon form.

The above algorithm can be used in general to find the dependence relations between any set of vectors, and to pick out a basis from any spanning set. Also finding a basis for the column space of Template:Mvar is equivalent to finding a basis for the row space of the transpose matrix Template:Math.

To find the basis in a practical setting (e.g., for large matrices), the singular-value decomposition is typically used.

Dimension

{{#invoke:Labelled list hatnote|labelledList|Main article|Main articles|Main page|Main pages}} The dimension of the column space is called the rank of the matrix. The rank is equal to the number of pivots in the reduced row echelon form, and is the maximum number of linearly independent columns that can be chosen from the matrix. For example, the 4 × 4 matrix in the example above has rank three.

Because the column space is the image of the corresponding matrix transformation, the rank of a matrix is the same as the dimension of the image. For example, the transformation <math>\R^4 \to \R^4</math> described by the matrix above maps all of <math>\R^4</math> to some three-dimensional subspace.

The nullity of a matrix is the dimension of the null space, and is equal to the number of columns in the reduced row echelon form that do not have pivots.<ref>Columns without pivots represent free variables in the associated homogeneous system of linear equations.</ref> The rank and nullity of a matrix Template:Mvar with Template:Mvar columns are related by the equation:

<math>\operatorname{rank}(A) + \operatorname{nullity}(A) = n.\,</math>

This is known as the rank–nullity theorem.

Relation to the left null space

The left null space of Template:Mvar is the set of all vectors Template:Math such that Template:Math. It is the same as the null space of the transpose of Template:Mvar. The product of the matrix Template:Math and the vector Template:Math can be written in terms of the dot product of vectors:

<math>A^\mathsf{T}\mathbf{x} = \begin{bmatrix} \mathbf{v}_1 \cdot \mathbf{x} \\ \mathbf{v}_2 \cdot \mathbf{x} \\ \vdots \\ \mathbf{v}_n \cdot \mathbf{x} \end{bmatrix},</math>

because row vectors of Template:Math are transposes of column vectors Template:Math of Template:Mvar. Thus Template:Math if and only if Template:Math is orthogonal (perpendicular) to each of the column vectors of Template:Mvar.

It follows that the left null space (the null space of Template:Math) is the orthogonal complement to the column space of Template:Mvar.

For a matrix Template:Mvar, the column space, row space, null space, and left null space are sometimes referred to as the four fundamental subspaces.

For matrices over a ring

Similarly the column space (sometimes disambiguated as right column space) can be defined for matrices over a ring Template:Mvar as

<math>\sum\limits_{k=1}^n \mathbf{v}_k c_k</math>

for any Template:Math, with replacement of the vector Template:Mvar-space with "right free module", which changes the order of scalar multiplication of the vector Template:Math to the scalar Template:Math such that it is written in an unusual order vector–scalar.<ref>Important only if Template:Mvar is not commutative. Actually, this form is merely a product Template:Math of the matrix Template:Mvar to the column vector Template:Math from Template:Math where the order of factors is preserved, unlike the formula above.</ref>

Row space

Definition

Let Template:Mvar be a field of scalars. Let Template:Mvar be an Template:Math matrix, with row vectors Template:Math. A linear combination of these vectors is any vector of the form

<math>c_1 \mathbf{r}_1 + c_2 \mathbf{r}_2 + \cdots + c_m \mathbf{r}_m,</math>

where Template:Math are scalars. The set of all possible linear combinations of Template:Math is called the row space of Template:Mvar. That is, the row space of Template:Mvar is the span of the vectors Template:Math.

For example, if

<math>A = \begin{bmatrix} 1 & 0 & 2 \\ 0 & 1 & 0 \end{bmatrix},</math>

then the row vectors are Template:Math and Template:Math. A linear combination of Template:Math and Template:Math is any vector of the form

<math>c_1 \begin{bmatrix}1 & 0 & 2\end{bmatrix} + c_2 \begin{bmatrix}0 & 1 & 0\end{bmatrix} = \begin{bmatrix}c_1 & c_2 & 2c_1\end{bmatrix}.</math>

The set of all such vectors is the row space of Template:Mvar. In this case, the row space is precisely the set of vectors Template:Math satisfying the equation Template:Math (using Cartesian coordinates, this set is a plane through the origin in three-dimensional space).

For a matrix that represents a homogeneous system of linear equations, the row space consists of all linear equations that follow from those in the system.

The column space of Template:Mvar is equal to the row space of Template:Math.

Basis

The row space is not affected by elementary row operations. This makes it possible to use row reduction to find a basis for the row space.

For example, consider the matrix

<math>A = \begin{bmatrix} 1 & 3 & 2 \\ 2 & 7 & 4 \\ 1 & 5 & 2\end{bmatrix}.</math>

The rows of this matrix span the row space, but they may not be linearly independent, in which case the rows will not be a basis. To find a basis, we reduce Template:Mvar to row echelon form:

Template:Math, Template:Math, Template:Math represents the rows.

<math>

\begin{align} \begin{bmatrix} 1 & 3 & 2 \\ 2 & 7 & 4 \\ 1 & 5 & 2\end{bmatrix} &\xrightarrow{\mathbf{r}_2-2\mathbf{r}_1 \to \mathbf{r}_2} \begin{bmatrix} 1 & 3 & 2 \\ 0 & 1 & 0 \\ 1 & 5 & 2\end{bmatrix} \xrightarrow{\mathbf{r}_3-\,\,\mathbf{r}_1 \to \mathbf{r}_3} \begin{bmatrix} 1 & 3 & 2 \\ 0 & 1 & 0 \\ 0 & 2 & 0\end{bmatrix} \\ &\xrightarrow{\mathbf{r}_3-2\mathbf{r}_2 \to \mathbf{r}_3} \begin{bmatrix} 1 & 3 & 2 \\ 0 & 1 & 0 \\ 0 & 0 & 0\end{bmatrix} \xrightarrow{\mathbf{r}_1-3\mathbf{r}_2 \to \mathbf{r}_1} \begin{bmatrix} 1 & 0 & 2 \\ 0 & 1 & 0 \\ 0 & 0 & 0\end{bmatrix}. \end{align} </math> Once the matrix is in echelon form, the nonzero rows are a basis for the row space. In this case, the basis is Template:Math. Another possible basis Template:Math comes from a further reduction.<ref name="example">The example is valid over the real numbers, the rational numbers, and other number fields. It is not necessarily correct over fields and rings with non-zero characteristic.</ref>

This algorithm can be used in general to find a basis for the span of a set of vectors. If the matrix is further simplified to reduced row echelon form, then the resulting basis is uniquely determined by the row space.

It is sometimes convenient to find a basis for the row space from among the rows of the original matrix instead (for example, this result is useful in giving an elementary proof that the determinantal rank of a matrix is equal to its rank). Since row operations can affect linear dependence relations of the row vectors, such a basis is instead found indirectly using the fact that the column space of Template:Math is equal to the row space of Template:Mvar. Using the example matrix Template:Mvar above, find Template:Math and reduce it to row echelon form:

<math>

A^{\mathrm{T}} = \begin{bmatrix} 1 & 2 & 1 \\ 3 & 7 & 5 \\ 2 & 4 & 2\end{bmatrix} \sim \begin{bmatrix} 1 & 2 & 1 \\ 0 & 1 & 2 \\ 0 & 0 & 0\end{bmatrix}. </math>

The pivots indicate that the first two columns of Template:Math form a basis of the column space of Template:Math. Therefore, the first two rows of Template:Mvar (before any row reductions) also form a basis of the row space of Template:Mvar.

Dimension

{{#invoke:Labelled list hatnote|labelledList|Main article|Main articles|Main page|Main pages}} The dimension of the row space is called the rank of the matrix. This is the same as the maximum number of linearly independent rows that can be chosen from the matrix, or equivalently the number of pivots. For example, the 3 × 3 matrix in the example above has rank two.<ref name="example"/>

The rank of a matrix is also equal to the dimension of the column space. The dimension of the null space is called the nullity of the matrix, and is related to the rank by the following equation:

<math>\operatorname{rank}(A) + \operatorname{nullity}(A) = n,</math>

where Template:Mvar is the number of columns of the matrix Template:Mvar. The equation above is known as the rank–nullity theorem.

Relation to the null space

The null space of matrix Template:Mvar is the set of all vectors Template:Math for which Template:Math. The product of the matrix Template:Mvar and the vector Template:Math can be written in terms of the dot product of vectors:

<math>A\mathbf{x} = \begin{bmatrix} \mathbf{r}_1 \cdot \mathbf{x} \\ \mathbf{r}_2 \cdot \mathbf{x} \\ \vdots \\ \mathbf{r}_m \cdot \mathbf{x} \end{bmatrix},</math>

where Template:Math are the row vectors of Template:Mvar. Thus Template:Math if and only if Template:Math is orthogonal (perpendicular) to each of the row vectors of Template:Mvar.

It follows that the null space of Template:Mvar is the orthogonal complement to the row space. For example, if the row space is a plane through the origin in three dimensions, then the null space will be the perpendicular line through the origin. This provides a proof of the rank–nullity theorem (see dimension above).

The row space and null space are two of the four fundamental subspaces associated with a matrix Template:Mvar (the other two being the column space and left null space).

Relation to coimage

If Template:Mvar and Template:Mvar are vector spaces, then the kernel of a linear transformation Template:Math is the set of vectors Template:Math for which Template:Math. The kernel of a linear transformation is analogous to the null space of a matrix.

If Template:Mvar is an inner product space, then the orthogonal complement to the kernel can be thought of as a generalization of the row space. This is sometimes called the coimage of Template:Mvar. The transformation Template:Mvar is one-to-one on its coimage, and the coimage maps isomorphically onto the image of Template:Mvar.

When Template:Mvar is not an inner product space, the coimage of Template:Mvar can be defined as the quotient space Template:Math.

See also

• Euclidean subspace

Further reading

• Template:Citation
• Template:Citation
• Template:Citation
• Template:Citation
• Template:Citation
• Template:Citation
• Template:Citation
• Template:Citation
• Template:Citation

External links

• {{#invoke:Template wrapper|{{#if:|list|wrap}}|_template=cite web

|_exclude=urlname, _debug, id |url = https://mathworld.wolfram.com/{{#if:RowSpace%7CRowSpace.html}} |title = Row Space |author = Weisstein, Eric W. |website = MathWorld |access-date = |ref = Template:SfnRef }}

• {{#invoke:Template wrapper|{{#if:|list|wrap}}|_template=cite web

|_exclude=urlname, _debug, id |url = https://mathworld.wolfram.com/{{#if:ColumnSpace%7CColumnSpace.html}} |title = Column Space |author = Weisstein, Eric W. |website = MathWorld |access-date = |ref = Template:SfnRef }}

• Template:Aut, MIT Linear Algebra Lecture on the Four Fundamental Subspaces at Google Video, from MIT OpenCourseWare
• Khan Academy video tutorial
• Lecture on column space and nullspace by Gilbert Strang of MIT
• Row Space and Column Space

{{#invoke:Navbox|navbox}}