Eigenvalue Problems

1.

Eigenvalues and eigenvectors

2.

Vector spaces

3.

Linear transformations

4.

Matrix diagonalization

The Eigenvalue Problem



Consider a nxn matrix A



Vector equation: Ax = x

» Seek solutions for x and 
»  satisfying the equation are the eigenvalues
» Eigenvalues can be real and/or imaginary;

distinct and/or repeated

» x satisfying the equation are the eigenvectors



Nomenclature

» The set of all eigenvalues is called the spectrum
» Absolute value of an eigenvalue:

» The largest of the absolute values of the

eigenvalues is called the spectral radius

2

2

b

a

ib

a

j

j















Determining Eigenvalues



Vector equation

» Ax = x  (A-x = 0
» A- is called the characteristic matrix



Non-trivial solutions exist if and only if:

» This is called the characteristic equation



Characteristic polynomial

» nth-order polynomial in 
» Roots are the eigenvalues {

1

, 

2

, …, 

n

}

0

)

det(

2

1

2

22

21

1

12

11





















nn

n

n

n

n

a

a

a

a

a

a

a

a

a















I

A

Eigenvalue Example



Characteristic matrix



Characteristic equation



Eigenvalues: 

1

= -5, 

2

= 2





























































4

3

2

1

1

0

0

1

4

3

2

1

I

A

0

10

3

)

3

)(

2

(

)

4

)(

1

(

2































I

A

Eigenvalue Properties



Eigenvalues of A and A

T

are equal



Singular matrix has at least one zero

eigenvalue



Eigenvalues of A

-1

: 1/

1

, 1/

2

, …, 1/

n



Eigenvalues of diagonal & triangular matrices

are equal to the diagonal elements



Trace



Determinant







n

j

j

Tr

1

)

(



A







n

j

j

1



A

Determining Eigenvectors



First determine eigenvalues: {

1

, 

2

, …,



n

}



Then determine eigenvector

corresponding to each eigenvalue:



Eigenvectors determined up to scalar

multiple



Distinct eigenvalues

» Produce linearly independent eigenvectors



Repeated eigenvalues

» Produce linearly dependent eigenvectors
» Procedure to determine eigenvectors more

complex (see text)

» Will demonstrate in Matlab

0

)

(

0

)

(











k

k

x

I

A

x

I

A





Eigenvector Example



Eigenvalues



Determine eigenvectors: Ax = x



Eigenvector for 

1

= -5



Eigenvector for 

1

= 2









































3

1

or

9487

.

0

3162

.

0

0

3

0

2

6

1

1

2

1

2

1

x

x

x

x

x

x









































1

2

or

4472

.

0

8944

.

0

0

6

3

0

2

2

2

2

1

2

1

x

x

x

x

x

x

2

5

4

3

2

1

2

1



























A

0

)

4

(

3

0

2

)

1

(

4

3

2

2

1

2

1

2

2

1

1

2

1























x

x

x

x

x

x

x

x

x

x









Matlab Examples

>> A=[ 1 2; 3 -4];
>> e=eig(A)
e =
2
-5
>> [X,e] = eig(A)
X =
0.8944 -0.3162
0.4472 0.9487
e =
2 0
0 -5

>> A=[2 5; 0 2];
>> e=eig(A)
e =
2
2
>> [X,e]=eig(A)
X =
1.0000 -1.0000
0 0.0000
e =
2 0
0 2

Vector Spaces



Real vector space V

» Set of all n-dimensional vectors with real

elements

» Often denoted R

n

» Element of real vector space denoted



Properties of a real vector space

» Vector addition

» Scalar multiplication

V



x

0

a

a

w

v

u

w

v

u

a

0

a

a

b

b

a



























)

(

)

(

)

(

a

a

a

a

a

a

a

b

a

b

a

















1

)

(

)

(

)

(

)

(

k

c

k

c

ck

k

c

c

c

c

Vector Spaces cont.



Linearly independent vectors

» Elements:
» Linear combination:
» Equation satisfied only for c

j

= 0



Basis

» n-dimensional vector space V contains

exactly n linearly independent vectors

» Any n linearly independent vectors form a

basis for V

» Any element of V can be expressed as a

linear combination of the basis vectors



Example: unit basis vectors in R

3

0

2

1









(m)

(2)

(1)

a

a

a

m

c

c

c



V



(m)

(2)

(1)

a

a

a

,

,

,

































































































3

2

1

3

2

1

3

3

2

1

1

0

0

0

1

0

0

0

1

c

c

c

c

c

c

c

c

c

)

(

(2)

(1)

a

a

a

x

Inner Product Spaces



Inner product



Properties of an inner product space



Two vectors with zero inner product are called

orthogonal



Relationship to vector norm

» Euclidean norm

» General norm

» Unit vector: ||a|| = 1





















n

k

n

n

k

k

T

b

a

b

a

b

a

b

a

1

2

2

1

1

)

,

(



b

a

b

a

b

a

0

if

only

and

if

0

)

,

(

0

)

,

(

)

,

(

)

,

(

)

,

(

)

,

(

)

,

(

2

1

2

1















a

a

a

a

a

a

c

c

a

c

b

c

a

c

b

a

q

q

q

q

2

2

2

2

1

)

,

(

n

T

a

a

a















a

a

a

a

a

b

a

b

a

b

a

b

a









)

,

(

Linear Transformation



Properties of a linear operator F

» Linear operator example: multiplication by a matrix

» Nonlinear operator example: Euclidean norm



Linear transformation



Invertible transformation

» Often called a coordinate transformation

)

(

)

(

)

(

)

(

)

(

x

x

x

v

x

v

cF

c

F

F

F

F









Ax

y

A

y

x









tion

Transforma

Operator

,

Elements

xn

m

m

n

R

R

R

y

A

x

Ax

y

A

y

x

1

x

tion

Transforma

Inverse

tion

Transforma

,

,

Dimensions













n

n

n

n

R

R

R

Orthogonal Transformations



Orthogonal matrix

»

A square matrix satisfying: A

T

= A

-1

»

Determinant has value +1 or -1

»

Eigenvalues are real or complex conjugate pairs

with absolute value of unity

»

A square matrix is orthonormal if:



Orthogonal transformation

»

y = Ax where A is an orthogonal matrix

»

Preserves the inner product between any two

vectors

»

The norm is also invariant to orthogonal

transformation

b

a

v

u

Ab

v

Aa

u











 ,













k

j

k

j

k

T

j

if

1

if

0

a

a

Similarity Transformations



Eigenbasis

» If a nxn matrix has n distinct eigenvalues,

the eigenvectors form a basis for R

n

» The eigenvectors of a symmetric matrix

form an orthonormal basis for R

n

» If a nxn matrix has repeated eigenvalues,

the eigenvectors may not form a basis for
R

n

(see text)



Similar matrices

» Two nxn matrices are similar if there exists

a nonsingular nxn matrix P such that:

» Similar matrices have the same eigenvalues
» If x is an eigenvector of A, then y = P

-1

x is

an eigenvector of the similar matrix

AP

P

A

1

ˆ





Matrix Diagonalization



Assume the nxn matrix A has an eigenbasis



Form the nxn modal matrix X with the

eigenvectors of A as column vectors: X = [x

1

,

x

2

, …, x

n

]



Then the similar matrix D = X

-1

AX is diagonal

with the eigenvalues of A as the diagonal

elements



Companion relation: XDX

-1

= A



























































n

nn

n

n

n

n

a

a

a

a

a

a

a

a

a



































0

0

0

0

0

0

2

1

1

2

1

2

22

21

1

12

11

AX

X

D

A

Matrix Diagonalization

Example



























































































































































































2

0

0

5

2

15

4

5

1

3

2

1

7

1

1

3

2

1

4

3

2

1

1

3

2

1

7

1

1

3

2

1

7

1

1

3

2

1

1

2

,

2

3

1

,

5

4

3

2

1

1

1

1

2

1

2

2

1

1

AX

X

D

AX

X

D

X

x

x

X

x

x

A





Matlab Example

>> A=[-1 2 3; 4 -5 6; 7 8 -9];
>> [X,e]=eig(A)
X =
-0.5250 -0.6019 -0.1182
-0.5918 0.7045 -0.4929
-0.6116 0.3760 0.8620
e =
4.7494 0 0
0 -5.2152 0
0 0 -14.5343
>> D=inv(X)*A*X
D =
4.7494 -0.0000 -0.0000
-0.0000 -5.2152 -0.0000
0.0000 -0.0000 -14.5343