Diagonalisation problem & Applications

We are now going to diagonalise a square matrix

 

 

The problem of diagonalising a matrix may be stated as:

- Given an NxN matrix A, can we find an invertible NxN matrix P and an NxN diagonal matrix D such that

- A = PDP^(−1)

- The task of diagonalising A is simply to find matrices P and D such that A can be written as PDP^(−1)

- Not all square matrices can be diagonalised.!!

 

Condition of diagonalising

- Yes, we can diagonalise the NxN matrix A, that is, we can write A = PDP^(−1) , if and only if A has N linearly independent eigenvectors.

- If A does not have N linearly independent eigenvectors, it is not diagonalisable.!!

bcs : P행렬을 구성할 수 없다!

- P행렬은 linearly independent한 N개의 X1, X2, ..., XN들이 (N차)열벡터로 자리 잡으면서 만든 행렬

- P = (X1, X2, ... , XN-1, XN) ---> NxN matrix임!! Don't Confuse it!!

- X1 : 1st Eigenvector, X2 : 2nd Eigenvector... ,XN : Nth Eigenvector 

- D = (λ1

                λ2

                      .

                        .

                              λN-1

                                      λN) 

- λj is the eigenvalue which gives the eigenvector Xj

- X1 Eigenvector를 만드는 eigenvalue : λ1, X2 Eigenvector를 만드는 eigenvalue : λ2....

 

Clues to decide whether to diagonalise or not

- Eigenvectors (non-trivial ones) that come from distinct eigenvalues of a square matrix are linearly independent

- If an NxN matrix A has N distinct eigenvalues, we can select one eigenvector from each eigenvalue to form N linearly independent eigenvectors.! ---> Such a matrix A is diagonalisable.!! 

- if an NxN matrix A has less than N distinct eigenvalues, it may or may not be diagonalisable depending on whether N linearly independent vectors can be found.

 

Symmetric Matrix's Diagonalisation

- If A is symmetric, it may be possible to construct P in such a way that P^(−1) can be easily found 

- If A = A^(T) then A is said to be symmetric.!

 

Here is a theorem for a symmetric matrix

- If A is an NxN symmetric matrix whose elements are real numbers then A has only real eigenvalues.

- Furthermore, if the symmetric matrix A has N distinct eigenvalues λ1 , λ2 , …, λN−1 and λN with unit norm(vector의 절댓값 크기가 1) eigenvectors X1 , X2 , …, XN−1 and XN respectively, then we can form P=(X1 X2 … XN ) such that P^(−1) = P^(T) . Hence, we can write A = PDP^(T)

 

How is the diagonal matrix used?

<Example>

- If a square matrix A can be diagonalised, it is relatively easy to compute A^(M) for high power M,

- 여기서 A^(M)을 쉽게 구하는 것이 왜 중요하냐..? ----> Deep Neural Network에서는 layer가 많아서 행렬곱의 연산이 빨리 될수록 더 효율적인 AI 알고리즘이 된다..!(Diagonalisation은 행렬곱을 빨리 할 수 있는, 특히 특정행렬에 여러번 곱을 하는 걸 빨리 할 수 있는 장점이 있다!!)

-  A = PDP^(-1)  ∵ a square matrix A can be diagonalised

-  A^(5) = PDP^(-1)PDP^(-1)PDP^(-1)PDP^(-1)PDP^(-1)

            = P D I D I D I D I D P^(-1) = PD^(5)P^(-1)

- In General:

A^(M) = PD^(M)P^(-1)

- D^(M) is easy to calculate if D is a diagonal matrix!!

- E.g. 

- D = (a 0 0

          0 b 0     

          0 0 c)

-----> D^(20) = (a^(20)    0    0

                            0    b^(20) 0

                            0         0   c^(20))

- Conclusion : A를 M번 곱하는 건 매우 복잡하고 어렵지만 대각화를 한 다음 곱하는 건(즉, D를 M번 곱하는 건) 매우 쉽다!!