Linear Algebra

Linear Algebra is the mathematics of systems of linear equations.  Mainly, it is the study of matrices.  Mathematica has several built in commands that work with matrices.

Systems and Matrices

Definition: Let (Subscript[a, ij]), (Subscript[b, i])∈ℝ for 1⩽ i ⩽ m, 1 ⩽ j ⩽ n, be fixed numbers.  Then the following is a system of linear equations in the unknown variables (Subscript[x, 1]),..., (Subscript[x, n]).

(Subscript[a, 11])(Subscript[x, 1])+(Subscript[a, 12])(Subscript[x, 2])+...+(Subscript[a, 1n])(Subscript[x, n])=(Subscript[b, 1])

(Subscript[a, 21])(Subscript[x, 1])+(Subscript[a, 22])(Subscript[x, 2])+...+(Subscript[a, 2n])(Subscript[x, n])=(Subscript[b, 2])

⋮

(Subscript[a, m1])(Subscript[x, 1])+(Subscript[a, m2])(Subscript[x, 2])+...+(Subscript[a, mn])(Subscript[x, n])=(Subscript[b, m])

Of course, we don't have to use real numbers for the constants.  Complex numbers, rational, or other choices of objects for the (Subscript[a, ij])'s and (Subscript[b, i])'s are acceptable.  In an effort to ease the process of solving systems of linear equations, matrices were developed.

Definition: A matrix is a rectangular array of numbers (or other objects).  Thus, a matrix, A, could be represented as

A=MatrixForm[Array[(Subscript[a, ##])&,{3,4}]]

In this case we have displayed a matrix with 3 rows and 4 columns.  Thus the size of this matrix is 3×4 (read "three-by-four").  We often refer to a particular element in the matrix by which row and column the element appears.  So the i,(j^th) entry would be (Subscript[a, ij]).

As with other algebraic objects we can develop a set of arithmetic rules that operate on matrices.  In Mathematica the standard operations of +, -, *, and / perform element-by-element operations.  For example

M1={{a,b,c},{d,e,f}}
M2={{1,2,3},{4,5,6},{7,8,9}}
M3={{α,β,γ},{δ,ϵ,ζ}}

These are three different matrices.  To see them in the form of a matrix we use the MatrixForm command.

MatrixForm[M1]
MatrixForm[M2]
MatrixForm[M3]

Now notice the results of performing standard operations:

MatrixForm[M1+M3]

MatrixForm[M1*M2]

Notice that this last operation did not work.  This is because the two matrices are different sizes.  However, there is an additional operation, called the "Dot product" that does allow us to "multiply" these two matrices together:

MatrixForm[M1.M2]

Notice that the result of this type of product proceeds as follows.  The i,(j^th) entry in the result is the sum of the term-by-term multiplication of the (i^th) row of the first matrix with the (j^th) column of the second matrix.  Thus, we can apply this type of product when the number of columns in the first matrix matches the number of rows in the second matrix.  That is, we can do the following: (Subscript[A, m×k]) . (Subscript[B, k×n]) where the result will be an m×n matrix.

With this matrix (Dot) multiplication, we can use matrix notation to rewrite systems of equations.  In the following example we with use Mathematica to solve a linear system of equations.

Example 1: consider the following system of equations:

x + y + z = 6, x - y + z = 2, and 2x - 3y + z = -1.

Note that the following Mathematica commands show that the left-hand-side of the system can be written using a coefficient matrix, a column vector of variables, and matrix multiplication.

A={{1,1,1},{1,-1,1},{2,-3,1}}
v={{x},{y},{z}}
MatrixForm[A.v]

We can now define the right-hand-side of the equations as a vector of values and then use the command LinearSolve to solve the system of equations:

b={6,2,-1}
sols=LinearSolve[A,b]

Note that in this case the solution to the system of linear equations is x = 1, y = 2, and z = 3.

Example 2: consider the system: 2x + 3y - z = 5, x - y + z = 2, and 3x - 2y + 3z = 7.  The solution to this system is:

sols2=LinearSolve[{{2,3,-1},{1,-1,1},{3,-2,3}}, {5,2,7}]

We can access the individual parts of this solution as follows:

sols2[[1]]

sols2[[3]]

sols2[[{1,2}]]

Example 3: consider the system w + 3x - 2y = 1, 3w + 10x - 7y + z = 2, and -5w - 5x + 3y + 7z = 3.  Then we get:

A={{1,3,-2,0},{3,10,-7,1},{-5,-5,3,7}};
MatrixForm[A]

b={1,2,3};
sols3=LinearSolve[A,b]

Note that this system of linear equations has more variables than equations.  In this case it is likely that more than one solutions exists.  The LinearSolve command only finds a solution, not always all solutions.  To see this we present two alternative ways to solve the system of equations which does indicate that solutions to this system are not unique.

{w,x,y,z}/.First[Solve[Thread[A.{w,x,y,z}==b],{w,x,y}]]

In this case, z is a parameter (any real number) that we will call t, and the solution to the system is w = 2t - 2, x = 5, y = t + 6, z = t.  Alternately we may row-reduce an augmented coefficient matrix:

AugA={{1,3,-2,0,1},{3,10,-7,1,2},{-5,-5,3,7,3}};
MatrixForm[AugA]
MatrixForm[RowReduce[AugA]]

We can read this as an augmented coefficient matrix for a system of linear equations that has the same solution set.  That is, each row of this reduced matrix corresponds to one of the equations given above.

Eigenvalues and Eigenvectors

One other topic of interest in this course is that of eigenvalues and eigenvectors.  Eigenvalues and eigenvectors appear in mathematics across disciplines.  They are use to analyze problems in dynamics, mathematical biology, physics, and business in the context of linear and nonlinear problems.  First a definition:

Definition: Let λ ∈ ℂ and nonzero vector v satisfy the equation Av = λv. Then we say that λ is an eigenvalue and v is an eigenvector of the matrix A.

Even if the matrix A is a real matrix, it is still possible to obtain complex eigenvalues and eigenvectors.  We often write the eigenvalue and eigenvector as an ordered pair ( λ, v ), the eigenpair.  Mathematica has built-in commands to calculate the eigenpair for a matrix:

A={{2,3,-1},{1,-1,1},{3,-2,3}};
MatrixForm[A]
Eigenvalues[N[A]]
Eigenvectors[N[A]]

The N command causes Mathematica to seek numerical approximations.  Note that although the matrix has real entries, there are both eigenvalues and eigenvectors that are complex (and appear in complex conjugate pairs).  To see the eigenpairs, Mathematica has a command:

{evals,evecs}=Eigensystem[N[A]]

So we can get at the individual eigenpairs by indexing into these lists:

evals[[1]]
evecs[[1]]

evals[[3]]
evecs[[3]]

Example 4: consider the following matrix:

A={{1,2,3,4},{0,1,2,3},{0,0,-2,1},{0,0,0,Pi}};
MatrixForm[A]

Find the eigenpairs.

{eval4,evec4}=Eigensystem[A]

Note that the eigenvalues are π, -2, and 1, where the eigenvalue 1 has multiplicity 2.  Also note that there are only three non-zero eigenvectors.  Thirdly, note that if the matrix is upper (or lower) triangular, then the eigenvalues are the diagonal elements.

Exercise 3.

(1) Use matrix equations to solve the following systems of equations.

A.

x − 2y + z = 2

3x − 4y + 5z = 15

2x + y − 3z = 1

B.

x + y + z + w = 0

x − y + z − w = 6

2x − y + 2z − w = 3

x + y + 2z = 4

C.

a + b + c = 5

2a − b − c = 4

a − 2b − 2c = −1

D.

s + 2t − u = 2

3s − t + u = 1

s − 5t + 3u = 3

(2) Find the eigenvalues and eigenvectors of the coefficient matrix for the system in (1B).