Matrix Operations

Addition $A + B$

Requires equal shapes; result stays $m \times n$ (element-wise).

Scalar multiple $cA$

Same shape $m \times n$; scales every entry.

Matrix product $AB$

Needs $n = p$; result is $m \times q$ (inner dims cancel).

Hadamard $A \odot B$

Requires equal shapes; element-wise product $m \times n$.

Transpose $A^\top$

Shape becomes $n \times m$; $(A^\top)_{ij} = A_{ji}$.

Product transpose

$(AB)^\top = B^\top A^\top$ (order reverses).

Non-commutative

In general $AB \ne BA$, even when both are defined.

Identity $I_n$

$n \times n$ with $1$s on the diagonal; $AI = IA = A$.

Inverse $A^{-1}$

Only for square $A$ with $\det(A) \ne 0$; then $A A^{-1} = I$.

Inverse of product

$(AB)^{-1} = B^{-1} A^{-1}$ (order reverses).

Singular matrix

$\det(A) = 0$: no inverse; columns are linearly dependent.

Symmetric

$A = A^\top$; covariance and Gram matrices are symmetric.

Orthogonal

$A^\top A = I$, so $A^{-1} = A^\top$; preserves lengths/angles.

Trace $\mathrm{tr}(A)$

Sum of diagonal entries (square $A$); $\mathrm{tr}(AB) = \mathrm{tr}(BA)$.

Determinant $\det(A)$

Scalar (square $A$); signed volume scaling of the linear map.

Det = 0

Map collapses volume to zero $\Rightarrow$ not invertible.

Rank

Number of linearly independent rows/columns; $\le \min(m, n)$.