Machine Learning

Appendix A

Linear Algebra

Chapter Outline

A.1 Properties of Matrices 1013

Matrix inversion lemmas 1014

Matrix derivatives 1014

A.2 Positive Definite and Symmetric Matrices 1015

A.3 Wirtinger Calculus 1016

References 1017

A.1 Properties of Matrices

Let A,B,C, and D be matrices of appropriate sizes. Invertibility is always assumed, whenever a matrix inversion is performed. The following properties hold true.

• (AB)^T = B^TA^T.

• (AB)⁻¹ = B⁻¹A⁻¹.

• (A^T)⁻¹ = (A^T)⁻¹.

• trace{AB} = trace{BA}.

• From the previous, we readily get

$trace\{ABC\}=trace\{CAB\}=trace\{BCA\}.$

• det(AB) = det(A)det(B), where det(⋅) denotes the determinant of a square matrix. As ...