7.3 Systems of Linear Algebraic Equations; Linear Independence, Eigenvalues, Eigenvectors

These are supporting notes and examples for section 7.3 of the book.  Please read through the section in the book before looking at these examples.

Systems of Linear Algebraic Equations and Linear Independence

Example 1:

x+2y+3z=3, 2x-y-z=2, and x-3y+2z=11.

Writing this system of algebraic equation in the form A (OverVector[v])=(OverVector[b]), we have the coefficient matrix is

A={{1,2,3},{2,-1,-1},{1,-3,2}};
MatrixForm[A]

The right hand side vector is

b={{3},{2},{11}};
MatrixForm[b]

and the variable vector is (OverVector[v])=(((x, y, z))^T).  We can solve this system using matrix inverses: (OverVector[v])=(A^(-1))(OverVector[b]).  In Mathematica this is

v=Inverse[A].b//MatrixForm

So x=1, y=-2, and z=2.  RowReduce can also give us this solution.  We augment the coefficient matrix with the right hand side vector and apply the command:

AAug={{1,2,3,3},{2,-1,-1,2},{1,-3,2,11}};
MatrixForm[AAug]
RowReduce[AAug]//MatrixForm

Notice that the right most column gives the solution.  That is, the first row tells us that 1x+0y+0z=1 or x=1, the second row gives y=-2 and the third row gives z=2.

In this example, we say the system of equations is consistent.  The rows (and columns) of the coefficient matrix form a linearly independent set of vectors and (A^(-1)) exists.

The modern calculators can also be used to preform row reduction.  Calculators vary, but for the TI-84 you enter the augments matrix via the edit tab under Matrix (or List) and then use rref command in the Math tab under Matrix.  See the manufacturer’s web site for how to use your calculator.  In the calculators the rref is short for “reduced row echelon form”, so you could google your make of calculator and “reduced row echelon form” to get instructional information on how to use your calculator.

Example 2

x+y+z=4, x-2y+3z=5 and x+4y-z=3.

Let’s look at the reduced row echelon form of the augmented matrix:

Clear[A,AAug]

AAug={{1,1,1,4},{1,-2,3,5},{1,4,-1,3}};
MatrixForm[AAug]
RowReduce[AAug]//MatrixForm

The first row of this means that x+(5/3)z=(13/3) and the second row says y-(2/3)z=-(1/3).  The third row says 0=0, which is not really helpful.  This system is called dependent.  In this system, z can be thought of as a parameter and there are an infinite number of solutions (dependent on the value of z).  We would write the solutions in parametric form as follows: x=(13/3)-(5/3)t, y=(2/3)t-(1/3), and z=t.

If we look at the coefficient matrix and try to take it’s inverse, then:

A={{1,1,1},{1,-2,3},{1,4,-1}};
Inverse[A]//MatrixForm

we see that the coefficient matrix is singular.  That is, (A^(-1)) does not exist.  The rows (and columns) of the coefficient matrix form a linearly dependent set of vectors.  For instance, in this case, the third row = 2(the first row) - (the second row).

Example 3

x+y+z=4, x-2y+3z=1 and x+4y-z=3.

Notice that this is the same system as example 2 with a change in the right hand side of the second equation.  Do the rref:

Clear[A,AAug]

AAug={{1,1,1,4},{1,-2,3,1},{1,4,-1,3}};
MatrixForm[AAug]
RowReduce[AAug]//MatrixForm

Notice that the last row says 0=1, which is false.  This system is called inconsistent and has no solutions.  Of course, the coefficient matrix is the same as in example 2, so it’s inverse does not exist.

Determinants

The determinant of a matrix is a function that assigns to each square matrix a real (or complex) number.  We use absolute value symbols to denote the determinant of a matrix.  For a 2×2 matrix, the determinant is

|(({{a, b}, {c, d}}))|=a d-b c.

For higher dimensional matrices determinants get a bit more complicated.  Here is the 3×3 case:

|(({{({{a, b}, {d, e}}), ({{c}, {f}})}, {({{g, h}}), i}}))|=a|(({{e, f}, {h, i}}))|-b|(({{d, f}, {g, i}}))|+c|(({{d, e}, {g, h}}))|.

This is called expansion by cofactors.  There are lots of resources on the internet covering this topic.  If you are interested, just google “determinant expansion by cofactors” to see explanations and examples.

We are interested in the relationship between determinants, inverses and the linear independence of row (and column) vectors.  We saw that (A^(-1)) exists ⇔ the rows (and columns) form a linearly independent set of vectors ⇔ A (OverVector[v])=(OverVector[b]) is a consistent system of equations and has a unique solution no matter the (OverVector[b]).  Here is how determinants fit in: A is singular ((A^(-1)) does not exist) ⇔ |A|=0.  That is, a square matrix is invertible if and only if the determinant of the matrix is nonzero.

Here is how Mathematica does determinants:  we will use the coefficient matrices from above.  From example 1:

Det[{{1,2,3},{2,-1,-1},{1,-3,2}}]

From examples 2 and 3:

Det[{{1,1,1},{1,-2,3},{1,4,-1}}]

Again, |A|=0 ⇔ A (OverVector[v])=(OverVector[b]) is either inconsistent or dependent; |A|≠0 ⇔ A (OverVector[v])=(OverVector[b]) is consistent (has a unique solution for each right hand side vector).

Eigenvalues and Eigenvectors

Square matrices can be thought of as a linear transformation of (R^n) to (R^n).  That is, multiplying a matrix times a vector will result in a vector that has been dilated in fixed directions.  Eigenvectors are the directions of the dilations, eigenvalues are the amount of the dilation.  For a square matrix A, we have

A (OverVector[v])=λ (OverVector[v])

where λ is an eigenvalue and (OverVector[v]) is the corresponding eigenvector.  Note that we can rewrite the above as

(A-λ I)(OverVector[v])=(OverVector[0])

where I is the identity matrix.  This is a system of equations.  This system always has the zero vector as a solution, but that does not interest us.  We are interested in nontrivial solution to this system.  That is, we need to find values of λ and (OverVector[v]), that are not zero.  If A-λ I is invertible, then the only solution would be the zero vector, so we need to find values of λ such that A-λ I is singular.

Definition: the characteristic polynomial for A is |A-λ I|.

The roots of the characteristic polynomial are the eigenvalues for the matrix.  Then we plug in the eigenvalues into the above system to find the corresponding eigenvectors.

Example 4

Use the characteristics polynomial to find the eigen-stuff for the following matrix.

A=(({{4, 1}, {1, 4}}))

First the characteristic polynomial:

P(λ)=|(({{4-λ, 1}, {1, 4-λ}}))|=(((4-λ))^2)-1=(λ^2)-8λ+15=(λ-5)(λ-3).

This has roots of λ=5 and λ=3.  These are the eigenvalues.  Let’s look at the first one, λ=5.  To find the corresponding eigenvector we need to solve (A-5I)(OverVector[v])=(OverVector[0]) for (OverVector[v]).  Note that we expect this system to be dependent (have an infinite number of solutions), because the coefficient matrix is singular.  Do the rref for this system:

Clear[AAug]

AAug={{-1,1,0},{1,-1,0}};
MatrixForm[AAug]
RowReduce[AAug]//MatrixForm

Remember that the last row of the reduced row echelon form says that 0=0.  The first row say that (Subscript[v, 1])-(Subscript[v, 2])=0.  So (Subscript[v, 1])=(Subscript[v, 2]).  So if we let (Subscript[v, 2])=t, then (Subscript[v, 1])=t also.  This means that the corresponding family of eigenvectors is:

(OverVector[v])=t (({{1}, {1}})),

for any real value of t.  While we can choose any value of t that we like, it is common to choose an “esthetically pleasing” value and say that one of the eigen-pair for this matrix is (5, (((1,1))^T)).  That is λ=5 and (OverVector[v])=(((1,1))^T).
Now find the corresponding eigenvector for the eigenvalue λ=3.

Clear[AAug]

AAug={{1,1,0},{1,1,0}};
MatrixForm[AAug]
RowReduce[AAug]//MatrixForm

Here (Subscript[v, 1])+(Subscript[v, 2])=0, so (Subscript[v, 1])=-(Subscript[v, 2]) and

(OverVector[v])=t (({{-1}, {1}}))

and the eigen-pair is (3,(((-1,1))^T)).  Here is how Mathematica does it:

estuff=Eigensystem[{{4,1},{1,4}}]

Notice the form that Mathematica uses for its output.  It lists the eigenvalues:

estuff[[1]]

and then the corresponding eigenvectors:

estuff[[2]]

So the first eigen-pair is:

{estuff[[1,1]],estuff[[2,1]]}

and the second eigen-pair is:

{estuff[[1,2]],estuff[[2,2]]}

Final thoughts

We will use eigenvalues and eigenvectors to analyze, and sometimes solve, systems of ODE’s.