Matrix Arithmetic

Most of linear algebra involves mathematical objects called matrices. A matrix is a finite rectangular array of numbers:

$$\left[\matrix{1 & -3 & 0 & 5 \cr 3 & 0 & -17 & 9 \cr -1 & -2 & 4 & -3 \cr}\right] \quad \left[\matrix{0 \cr 5 \cr -5 \cr 3 \cr -3.14 \cr}\right] \quad \left[\matrix{\dfrac{17}{5} & -4 & 0 & \dfrac{1}{2} \cr}\right] \quad \left[\matrix{0 & 0 \cr 0 & 0 \cr}\right]$$

In this case, the numbers are elements of $\rational$ (or $\real$ ). In general, the entries will be elements of some commutative ring or field.

I'll explain operations with matrices in the following examples. I'll discuss and prove some of the properties of these operations later on.


Example. ( Dimensions of matrices) An $m \times
   n$ matrix is a matrix with m rows and n columns. Sometimes this is expressed by saying that the dimensions of the matrix are $m \times n$ .

$$\matrix{\left[\matrix{1 & 2 & 3 \cr 4 & 5 & 6 \cr}\right] & \left[\matrix{0 & 0 \cr 0 & 0 \cr 0 & 0 \cr 0 & 0 \cr}\right] & \left[\matrix{\pi \cr}\right] \cr \hbox{a } 2 \times 3 \hbox{ matrix} & \hbox{a } 4 \times 2 \hbox{ matrix} & \hbox{a } 1 \times 1 \hbox{ matrix} \cr}$$

A $1 \times n$ matrix is called an n-dimensional row vector. For example, here's a 3-dimensional row vector:

$$\left[\matrix{4 & \sqrt{17} - 5 & 42 \cr}\right]$$

Likewise, an $n \times 1$ matrix is called an n-dimensional column vector. Here's a 3-dimensional column vector:

$$\left[\matrix{0 \cr 17 - e^2 \cr {\root 3 \of {617}} \cr}\right]$$

Two matrices are equal if they have the same dimensions and the corresponding entries are equal. For example, if

$$\left[\matrix{7 & 0 & 3 \cr 5 & -4 & 10 \cr}\right] = \left[\matrix{a & b & 3 \cr 5 & -4 & c \cr}\right],$$

then $a = 7$ , $b = 0$ , and $c = 10$ .


Definition. If R is a commutative ring, then $M(n,R)$ is the set of $n \times n$ matrices with entries in R.

For example, $M(2,\real)$ is the set of $2 \times 2$ matrices with real entries. $M(3,\integer_5)$ is the set of $3 \times 3$ matrices with entries in $\integer_5$ .


Example. ( Addition and subtraction) In this example, I'll assume that the matrices have entries in $\real$ .

You can add (or subtract) matrices by adding (or subtracting) corresponding entries.

$$\left[\matrix{1 & -1 & 0 \cr 2 & -4 & 6 \cr}\right] + \left[\matrix{7 & 1 & 3 \cr 0 & 3 & -6 \cr}\right] = \left[\matrix{8 & 0 & 3 \cr 2 & -1 & 0 \cr}\right].$$

If you are adding several matrices, you can group them any way you wish:

$$\left(\left[\matrix{1 & -2 \cr 5 & -11 \cr}\right] + \left[\matrix{0 & 2 \cr 9 & 8 \cr}\right]\right) + \left[\matrix{4 & 3 \cr -7 & 6 \cr}\right] = \left[\matrix{1 & 0 \cr 14 & -3 \cr}\right] + \left[\matrix{4 & 3 \cr -7 & 6 \cr}\right] = \left[\matrix{5 & 3 \cr 7 & 3 \cr}\right],$$

$$\left[\matrix{1 & -2 \cr 5 & -11 \cr}\right] + \left(\left[\matrix{0 & 2 \cr 9 & 8 \cr}\right] + \left[\matrix{4 & 3 \cr -7 & 6 \cr}\right]\right) = \left[\matrix{1 & -2 \cr 5 & -11 \cr}\right] + \left[\matrix{4 & 5 \cr 2 & 14 \cr}\right] = \left[\matrix{5 & 3 \cr 7 & 3 \cr}\right].$$

$$\left[\matrix{1 & -3 \cr 2 & 0 \cr 4 & \sqrt{2} \cr}\right] - \left[\matrix{6 & 3 \cr 2 & \pi \cr 0 & 0 \cr}\right] = \left[\matrix{-5 & -6 \cr 0 & -\pi \cr 4 & \sqrt{2} \cr}\right].$$

Note that matrix addition is commutative:

$$\left[\matrix{7 & 1 & 3 \cr 0 & 3 & -6 \cr}\right] + \left[\matrix{1 & -1 & 0 \cr 2 & -4 & 6 \cr}\right] = \left[\matrix{8 & 0 & 3 \cr 2 & -1 & 0 \cr}\right].$$

Symbolically, if A and B are matrices with the same dimensions, then

$$A + B = B + A.$$

Of course, matrix subtraction is not commutative.

You can only add or subtract matrices with the same dimensions.

$$\left[\matrix{1 & 2 \cr 3 & 4 \cr}\right] \quad\hbox{and}\quad \left[\matrix{0 & 0 & 0 \cr 0 & 0 & 0 \cr}\right] \quad\hbox{can't be added}.\quad\halmos$$


Example. ( Adding or subtracting matrices over $\integer_n$ ) You add or subtract matrices over $\integer_n$ by adding or subtracting corresponding entries, but all the arithmetic is done in $\integer_n$ .

For example, here are some matrix computations over $\integer_5$ :

$$\left[\matrix{1 & 2 & 4 \cr 0 & 3 & 3 \cr}\right] + \left[\matrix{4 & 3 & 0 \cr 2 & 1 & 3 \cr}\right] = \left[\matrix{1 + 4 & 2 + 3 & 4 + 0 \cr 0 + 2 & 3 + 1 & 3 + 3 \cr}\right] = \left[\matrix{0 & 0 & 4 \cr 2 & 4 & 1 \cr}\right].$$

$$\left[\matrix{3 & 4 \cr 2 & 0 \cr}\right] - \left[\matrix{2 & 1 \cr 4 & 1 \cr}\right] = \left[\matrix{3 - 2 & 4 - 1 \cr 2 - 4 & 0 - 1 \cr}\right] = \left[\matrix{1 & 3 \cr -2 & -1 \cr}\right] = \left[\matrix{1 & 3 \cr 3 & 4 \cr}\right].$$

Note that in the second example, there were some negative numbers in the middle of the computation, but the final answer was expressed entirely in terms of elements of $\integer_5 = \{0, 1, 2, 3, 4\}$ .


Example. A zero matrix $\zeromatrix$ is a matrix all of whose entries are 0. If you add the $m \times n$ zero matrix to another $m \times n$ matrix A, you get A:

$$\left[\matrix{31 & 97 \cr 24 & -53 \cr}\right] + \left[\matrix{0 & 0 \cr 0 & 0 \cr}\right] = \left[\matrix{31 & 97 \cr 24 & -53 \cr}\right].$$

In symbols, if $\zeromatrix$ is a zero matrix and A is a matrix of the same size, then

$$A + \zeromatrix = A \quad\hbox{and}\quad \zeromatrix + A = A.$$

A zero matrix is said to be an identity element for matrix addition.


Example. You can multiply a matrix by a number by multiplying each entry by the number. Here is an example with real numbers:

$$7\cdot \left[\matrix{3 & -3 \cr 4 & -1 \cr 0 & 2 \cr}\right] = \left[\matrix{21 & -21 \cr 28 & -7 \cr 0 & 14 \cr}\right].$$

Things work in the same way over $\integer_n$ , but all the arithmetic is done in $\integer_n$ . Here is an example over $\integer_5$ :

$$3\cdot \left[\matrix{2 & 1 \cr 4 & 1 \cr}\right] = \left[\matrix{3\cdot 2 & 3\cdot 1 \cr 3\cdot 4 & 3\cdot 1 \cr}\right] = \left[\matrix{1 & 3 \cr 2 & 3 \cr}\right].\quad\halmos$$


Example. To compute the product $AB$ of two matrices, take the dot products of the rows of A with the columns of B. In this example, assume all the matrices have real entries.

$$\left[\matrix{{\bf 2} & {\bf 1} & {\bf 4} \cr -1 & 0 & 3 \cr}\right] \left[\matrix{{\bf 1} & 6 \cr {\bf 5} & 1 \cr {\bf 2} & -2 \cr}\right] = \left[\matrix{(2 + 5 + 8 = 15) & \cdot \cr \cdot & \cdot \cr}\right]$$

$$\left[\matrix{{\bf 2} & {\bf 1} & {\bf 4} \cr -1 & 0 & 3 \cr}\right] \left[\matrix{1 & {\bf 6} \cr 5 & {\bf 1} \cr 2 & {\bf -2} \cr}\right] = \left[\matrix{15 & (12 + 1 - 8 = 5) \cr \cdot & \cdot \cr}\right]$$

$$\left[\matrix{2 & 1 & 4 \cr {\bf -1} & {\bf 0} & {\bf 3} \cr}\right] \left[\matrix{{\bf 1} & 6 \cr {\bf 5} & 1 \cr {\bf 2} & -2 \cr}\right] = \left[\matrix{15 & 5 \cr (-1 + 0 + 6 = 5) & \cdot \cr}\right]$$

$$\left[\matrix{2 & 1 & 4 \cr {\bf -1} & {\bf 0} & {\bf 3} \cr}\right] \left[\matrix{1 & {\bf 6} \cr 5 & {\bf 1} \cr 2 & {\bf -2} \cr}\right] = \left[\matrix{15 & 5 \cr 5 & (-6 + 0 - 6 = -12) \cr}\right]$$

$$\left[\matrix{2 & 1 & 4 \cr -1 & 0 & 3 \cr}\right] \left[\matrix{1 & 6 \cr 5 & 1 \cr 2 & -2 \cr}\right] = \left[\matrix{15 & 5 \cr 5 & -12 \cr}\right].$$

In order for the multiplication to work, the matrices must have compatible dimensions: The number of columns in A should equal the number of rows of B. Thus, if A is an $m \times n$ matrix and B is an $n \times p$ matrix, $AB$ will be an $m
   \times p$ matrix.

Here are two more examples, again using matrices with real entries:

$$\left[\matrix{-2 & 3 & 0 \cr}\right] \left[\matrix{2 & 0 \cr 9 & -4 \cr 1 & 5 \cr}\right] = \left[\matrix{23 & -12 \cr}\right].$$

$$\left[\matrix{4 & -2 \cr 1 & 1 \cr}\right] \left[\matrix{1 & 1 \cr 0 & -3 \cr}\right] = \left[\matrix{4 & 10 \cr 1 & -2 \cr}\right].$$

Here is an example with matrices in $M(2,\integer_3)$ . Remember that all the arithmetic is done in $\integer_3$ .

$$\left[\matrix{2 & 1 \cr 0 & 2 \cr}\right] \left[\matrix{1 & 2 \cr 2 & 1 \cr}\right] = \left[\matrix{1 & 2 \cr 1 & 2 \cr}\right].\quad\halmos$$


Example. If you multiply a matrix by a zero matrix, you get a zero matrix:

$$\left[\matrix{0 & 0 \cr 0 & 0 \cr}\right] \left[\matrix{9 & -3 & 43 \cr 5 & 0 & -1.2 \cr}\right] = \left[\matrix{0 & 0 & 0 \cr 0 & 0 & 0 \cr}\right].$$

In symbols, if $\zeromatrix$ is a zero matrix and A is a matrix compatible with it for multiplication, then

$$A\cdot \hbox{\bf 0} = \hbox{\bf 0} \quad\hbox{and}\quad \hbox{\bf 0}\cdot A = \hbox{\bf 0}.$$

(The zero matrices here may have different sizes.)


Example. Write the system of linear equations

$$\matrix{ x & + & 4y & - & 6z & = & 3 \cr -2x & + & & & 10z & = & 3 \cr x & + & y & + & z & = & 5 \cr } $$

as a matrix multiplication equation.

$$\left[\matrix{1 & 4 & -6 \cr -2 & 0 & 10 \cr 1 & 1 & 1 \cr}\right] \left[\matrix{x \cr y \cr z \cr}\right] = \left[\matrix{3 \cr 3 \cr 5 \cr}\right]. \quad\halmos$$


Example. Write the system of equations which correspond to the matrix equation

$$\left[\matrix{2 & 3 & 0 \cr -1 & 5 & 17 \cr}\right] \left[\matrix{a \cr b \cr c \cr}\right] = \left[\matrix{13 \cr 0 \cr}\right].$$

Multiply out the left side:

$$\left[\matrix{2a + 3b \cr -a + 5b + 17c \cr}\right] = \left[\matrix{13 \cr 0 \cr}\right].$$

Equate corresponding entries:

$$\matrix{2a & + & 3b & & & = & 13 \cr -a & + & 5b & + & 17c & = & 0 \cr}\quad\halmos$$


Example. ( Identity matrices) There are special matrices which serve as identities for multiplication: The $n \times n$ identity matrix is the square matrices with 1's down the main diagonal --- the diagonal running from northwest to southeast --- and 0's everywhere else. For example, the $3 \times 3$ identity matrix is

$$I = \left[\matrix{1 & 0 & 0 \cr 0 & 1 & 0 \cr 0 & 0 & 1 \cr}\right].$$

If I is the $n \times n$ identity and A is a matrix which is compatible for multiplication with A, then

$$AI = A \quad\hbox{and}\quad IA = A.$$

For example,

$$\left[\matrix{3 & -7 & 10 \cr \pi & \dfrac{1}{2} & -1.723 \cr 0 & 19 & 712 \cr}\right] \left[\matrix{1 & 0 & 0 \cr 0 & 1 & 0 \cr 0 & 0 & 1 \cr}\right] = \left[\matrix{3 & -7 & 10 \cr \pi & \dfrac{1}{2} & -1.723 \cr 0 & 19 & 712 \cr}\right].\quad\halmos$$


Example. Matrix multiplication obeys some of the algebraic laws you're familiar with. For example, matrix multiplication is associative: If A, B, and C are matrices and their dimensions are compatible for multiplication, then

$$(AB)C = A(BC).$$

However, matrix multiplication is not commutative in general. That is, $AB \ne BA$ for all matrices A, B.

One trivial way to get a counterexample is to let A be $3 \times 5$ and let B be $5 \times 3$ . Then $AB$ is $3 \times
   3$ while $BA$ is $5 \times
   5$ .

However, it's easy to come up with counterexamples even when $AB$ and $BA$ have the same dimensions. For example, consider the following matrices in $M(2,\real)$ :

$$A = \left[\matrix{1 & -1 \cr 2 & 3 \cr}\right] \quad\hbox{and}\quad B = \left[\matrix{1 & 0 \cr 4 & 1 \cr}\right].$$

Then

$$AB = \left[\matrix{-3 & -1 \cr 14 & 3 \cr}\right] \quad\hbox{but}\quad BA = \left[\matrix{1 & -1 \cr 6 & -1}\right].\quad\halmos$$


Example. If A is a matrix, the transpose $A^T$ of A is obtained by swapping the rows and columns of A. For example,

$$\left[\matrix{1 & 2 & 3 \cr 4 & 5 & 6 \cr}\right]^T = \left[\matrix{1 & 4 \cr 2 & 5 \cr 3 & 6 \cr}\right].$$

Notice that the transpose of an $m \times n$ matrix is an $n \times m$ matrix.


Example. Consider the following matrices with real entries:

$$A = \left[\matrix{1 & 0 & 1 & 2 \cr 2 & 1 & -1 & 0 \cr}\right], \quad B = \left[\matrix{c & c \cr 1 & 1 \cr}\right], \quad C = \left[\matrix{0 & 1 \cr 2 & -1 \cr 0 & 0 \cr 1 & 1 \cr}\right].$$

(a) Compute $CB + A^T$ .

$$CB + A^T = \left[\matrix{0 & 1 \cr 2 & -1 \cr 0 & 0 \cr 1 & 1 \cr}\right] \left[\matrix{c & c \cr 1 & 1 \cr}\right] + \left[\matrix{1 & 0 & 1 & 2 \cr 2 & 1 & -1 & 0 \cr}\right]^T = \left[\matrix{1 & 1 \cr 2c - 1 & 2c - 1 \cr 0 & 0 \cr c + 1 & c + 1 \cr}\right] + \left[\matrix{1 & 2 \cr 0 & 1 \cr 1 & -1 \cr 2 & 0 \cr}\right] = \left[\matrix{2 & 3 \cr 2c - 1 & 2c \cr 1 & -1 \cr c + 3 & c + 1 \cr}\right].\quad\halmos$$

(b) Compute $AC - 2B$ .

$$AC - 2B = \left[\matrix{1 & 0 & 1 & 2 \cr 2 & 1 & -1 & 0 \cr}\right] \left[\matrix{0 & 1 \cr 2 & -1 \cr 0 & 0 \cr 1 & 1 \cr}\right] - 2 \left[\matrix{c & c \cr 1 & 1 \cr}\right] = \left[\matrix{2 & 3 \cr 2 & 1 \cr}\right] - \left[\matrix{2c & 2c \cr 2 & 2 \cr}\right] = \left[\matrix{2 - 2c & 3 - 2c \cr 0 & -1 \cr}\right].\quad\halmos$$


Example. The inverse of an $n \times n$ matrix A is a matrix $A^{-1}$ which satisfies

$$AA^{-1} = I \quad\hbox{and}\quad A^{-1}A = I.$$

There is no such thing as matrix division in general, because some matrices do not have inverses. But if A has an inverse, you can simulate division by multiplying by $A^{-1}$ . This is often useful in solving matrix equations.

For example, if

$$A = \left[\matrix{a & b \cr c & d \cr}\right]$$

is a $2 \times 2$ matrix, the inverse of A --- if there is one --- turns out to be

$$A^{-1} = (ad - bc)^{-1} \left[\matrix{d & -b \cr -c & a \cr}\right].$$

To show that this is the inverse of A, I have to check that $AA^{-1} = I$ and $A^{-1}A
   = I$ :

$$AA^{-1} = \left[\matrix{a & b \cr c & d \cr}\right]\cdot (ad - bc)^{-1} \left[\matrix{d & -b \cr -c & a \cr}\right] = (ad - bc)^{-1} \left[\matrix{ad - bc & 0 \cr 0 & ad - bc}\right] = \left[\matrix{1 & 0 \cr 0 & 1 \cr}\right],$$

$$A^{-1}A = (ad - bc)^{-1} \left[\matrix{d & -b \cr -c & a \cr}\right] \left[\matrix{a & b \cr c & d \cr}\right] = (ad - bc)^{-1} \left[\matrix{ad - bc & 0 \cr 0 & ad - bc \cr}\right] = \left[\matrix{1 & 0 \cr 0 & 1 \cr}\right].$$

This proves that the formula gives the inverse of a $2 \times 2$ matrix.

Here's an example for a matrix with real entries:

$$\left[\matrix{3 & 5 \cr -2 & -1 \cr}\right]^{-1} = \dfrac{1}{7} \left[\matrix{-1 & -5 \cr 2 & 3 \cr}\right].$$

If $ad - bc = 0$ , you can't use this formula; in fact, a matrix which satisfies this condition does not have an inverse. A matrix which does not have an inverse is called singular.

For example,

$$\left[\matrix{2 & 6 \cr 1 & 3 \cr}\right] \quad \hbox{is singular}.\quad\halmos$$


Example. For what values of x is the following real matrix singular?

$$\left[\matrix{x - 1 & 3 \cr 4 & x - 5 \cr}\right]$$

The matrix is singular --- not invertible --- if

$$(x - 1)(x - 5) - (3)(4) = 0.$$

Solve for x:

$$(x^2 - 6x + 5) - 12 = 0, \quad x^2 - 6x - 7 = 0, \quad (x - 7)(x + 1) = 0, \quad x = 7 \quad\hbox{or}\quad x = -1.$$

The matrix is singular for $x = 7$ and for $x = -1$ .


Example. ( Solving a system of equations) Here's how to use inverses and matrix multiplication to solve a system of equations. Suppose I want to solve the following system over $\real$ for x and y:

$$\matrix{x & + & 3y & = & 7 \cr 2x & - & 2y & = & -2 \cr}$$

Write the equation in matrix form:

$$\left[\matrix{1 & 3 \cr 2 & -2 \cr}\right] \left[\matrix{x \cr y \cr}\right] = \left[\matrix{7 \cr -2 \cr}\right].$$

Multiply both sides by the inverse of the square matrix:

$$-\dfrac{1}{8} \left[\matrix{-2 & -3 \cr -2 & 1 \cr}\right] \left[\matrix{1 & 3 \cr 2 & -2 \cr}\right] \left[\matrix{x \cr y \cr}\right] = -\dfrac{1}{8} \left[\matrix{-2 & -3 \cr -2 & 1 \cr}\right] \left[\matrix{7 \cr -2 \cr}\right].$$

On the left, the square matrix and its inverse cancel, since they multiply to I. On the right,

$$-\dfrac{1}{8} \left[\matrix{-2 & -3 \cr -2 & 1 \cr}\right] \left[\matrix{7 \cr -2 \cr}\right] = \left[\matrix{1 \cr 2 \cr}\right].$$

Therefore,

$$\left[\matrix{x \cr y \cr}\right] = \left[\matrix{1 \cr 2 \cr}\right].$$

The solution is $x = 1$ , $y = 2$ .


Contact information

Bruce Ikenaga's Home Page

Copyright 2016 by Bruce Ikenaga