Hadamard product (matrices)

For two matrices A and B of the same dimension m × n, the Hadamard product ${\displaystyle A\circ B}$ (or ${\displaystyle A\odot B}$[3][4][5]) is a matrix of the same dimension as the operands, with elements given by

${\displaystyle (A\circ B)_{ij}=(A\odot B)_{ij}=(A)_{ij}(B)_{ij}.}$

For matrices of different dimensions (m × n and p × q, where m ≠ p or n ≠ q) the Hadamard product is undefined.

For example, the Hadamard product for a 3 × 3 matrix A with a 3 × 3 matrix B is

${\displaystyle {\begin{bmatrix}a_{11}&a_{12}&a_{13}\\a_{21}&a_{22}&a_{23}\\a_{31}&a_{32}&a_{33}\end{bmatrix}}\circ {\begin{bmatrix}b_{11}&b_{12}&b_{13}\\b_{21}&b_{22}&b_{23}\\b_{31}&b_{32}&b_{33}\end{bmatrix}}={\begin{bmatrix}a_{11}\,b_{11}&a_{12}\,b_{12}&a_{13}\,b_{13}\\a_{21}\,b_{21}&a_{22}\,b_{22}&a_{23}\,b_{23}\\a_{31}\,b_{31}&a_{32}\,b_{32}&a_{33}\,b_{33}\end{bmatrix}}.}$

The Hadamard product of two positive-semidefinite matrices is positive-semidefinite.[7] This is known as the Schur product theorem,[6] after German mathematician Issai Schur. For two positive-semidefinite matrices A and B, it is also known that the determinant of their Hadamard product is greater than or equal to the product of their respective determinants:[7]

${\displaystyle \det(\mathbf {A} \circ \mathbf {B} )\geq \det(\mathbf {A} )\det(\mathbf {B} ).}$

Hadamard multiplication is built into certain programming languages under various names. In MATLAB, GNU Octave, GAUSS and HP Prime, it is known as array multiplication, or in Julia broadcast multiplication, with the symbol .*.[10] In Fortran, R,[11] APL, J and Wolfram Language (Mathematica), it is done through simple multiplication operator *, whereas the matrix product is done through the function matmul, %*%, +.×, +/ .* and the . operators, respectively. In Python with the NumPy numerical library or the SymPy symbolic library, multiplication of array objects as a1*a2 produces the Hadamard product, but otherwise multiplication as a1@a2 or matrix objects m1*m2 will produce a matrix product. The Eigen C++ library provides a cwiseProduct member function for the Matrix class (a.cwiseProduct(b)), while the Armadillo library uses the operator % to make compact expressions (a % b; a * b is a matrix product).

The Hadamard product appears in lossy compression algorithms such as JPEG. The decoding step involves an entry-for-entry product, in other words the Hadamard product.

It is used in the machine learning literature, for example to describe the architecture of recurrent neural networks as GRUs or LSTMs.

Other Hadamard operations are also seen in the mathematical literature,[12] namely the Hadamard root and Hadamard power (which are in effect the same thing because of fractional indices), defined for a matrix such that:

For

${\displaystyle {\begin{aligned}\mathbf {B} &=\mathbf {A} ^{\circ 2}\\B_{ij}&={A_{ij}}^{2}\end{aligned}}}$

and for

${\displaystyle {\begin{aligned}\mathbf {B} &=\mathbf {A} ^{\circ {\frac {1}{2}}}\\B_{ij}&={A_{ij}}^{\frac {1}{2}}\end{aligned}}}$

The Hadamard inverse reads:[12]

${\displaystyle {\begin{aligned}\mathbf {B} &=\mathbf {A} ^{\circ -1}\\B_{ij}&={A_{ij}}^{-1}\end{aligned}}}$

A Hadamard division is defined as:[13][14]

${\displaystyle {\begin{aligned}\mathbf {C} &=\mathbf {A} \oslash \mathbf {B} \\C_{ij}&={\frac {A_{ij}}{B_{ij}}}\end{aligned}}}$

• Pointwise product
• Kronecker product