Gramian matrix

In linear algebra, the Gram matrix (Gramian matrix or Gramian) of a set of vectors $v_{1},\dots ,v_{n}$ in an inner product space is the Hermitian matrix of inner products, whose entries are given by $G_{ij}=\langle v_{i},v_{j}\rangle$ .^[1]

An important application is to compute linear independence: a set of vectors are linearly independent if and only if the Gram determinant (the determinant of the Gram matrix) is non-zero.

It is named after Jørgen Pedersen Gram.

Examples

For finite-dimensional real vectors with the usual Euclidean dot product, the Gram matrix is simply $G=V^{\mathrm {T} }V$ (or $G={\overline {V}}^{\mathrm {T} }V$ for complex vectors using the conjugate transpose), where $V$ is a matrix whose columns are the vectors $v_{k}$ .

Most commonly, the vectors are elements of a Euclidean space, or are functions in an L² space, such as continuous functions on a compact interval $[a,b]$ (which are a subspace of $L^{2}([a,b])$ ).

Given real-valued functions $\{\ell _{i}(\cdot ),\,i=1,\dots ,n\}$ on the interval $[t_{0},t_{f}]$ , the Gram matrix $G=[G_{{ij}}]$ is given by the standard inner product on functions:

G_{ij}=\int _{t_{0}}^{t_{f}}\ell _{i}(\tau ){\bar {\ell _{j}}}(\tau )\,d\tau .

For a general bilinear form $B$ on a finite-dimensional vector space over any field we can define a Gram matrix $G$ attached to a set of vectors $v_{1},\dots ,v_{n}$ by $G_{ij}=B(v_{i},v_{j})$ . The matrix will be symmetric if the bilinear form $B$ is symmetric.

Applications

In Riemannian geometry, given an embedded $k$ -dimensional Riemannian manifold $M\subset \mathbb {R} ^{n}$ and a coordinate chart $\phi :U\to \mathbb {R} ^{n}$ for $(x_{1},\ldots ,x_{k})\in U\subset \mathbb {R} ^{k}$ , the volume form $\omega$ on $M$ induced by the embedding may be computed using the Gramian of the coordinate tangent vectors:

\omega ={\sqrt {\det G}}\ dx_{1}\cdots dx_{k},\quad G=\left[\left\langle {\tfrac {\partial \phi }{\partial x_{i}}},{\tfrac {\partial \phi }{\partial x_{j}}}\right\rangle \right].

This generalizes the classical surface integral of a parametrized surface $\phi :U\to S\subset \mathbb {R} ^{3}$ for $(x,y)\in U\subset \mathbb {R} ^{2}$ :

\int _{S}f\ dA\ =\ \iint _{U}f(\phi (x,y))\,\left|{\tfrac {\partial \phi }{\partial x}}\,{\times }\,{\tfrac {\partial \phi }{\partial y}}\right|\,dx\,dy.

If the vectors are centered random variables, the Gramian is approximately proportional to the covariance matrix, with the scaling determined by the number of elements in the vector.
In quantum chemistry, the Gram matrix of a set of basis vectors is the overlap matrix.
In control theory (or more generally systems theory), the controllability Gramian and observability Gramian determine properties of a linear system.
Gramian matrices arise in covariance structure model fitting (see e.g., Jamshidian and Bentler, 1993, Applied Psychological Measurement, Volume 18, pp. 79–94).
In the finite element method, the Gram matrix arises from approximating a function from a finite dimensional space; the Gram matrix entries are then the inner products of the basis functions of the finite dimensional subspace.
In machine learning, kernel functions are often represented as Gram matrices.^[2]

Properties

Positive-semidefiniteness

The Gramian matrix is positive-semidefinite, and every positive symmetric semidefinite matrix is the Gramian matrix for some set of vectors. Further, in finite-dimensions it determines the vectors up to isomorphism, i.e. any two sets of vectors with the same Gramian matrix must be related by a single unitary matrix. These facts follow from taking the spectral decomposition of any positive-semidefinite matrix $P$ , so that $P=UDU^{\mathrm {H} }=(U{\sqrt {D}})(U{\sqrt {D}})^{\mathrm {H} }$ and so $P$ is the Gramian matrix of the rows of $U{\sqrt {D}}$ . The Gramian matrix of any orthonormal basis is the identity matrix. The infinite-dimensional analog of this statement is Mercer's theorem.

Derivation of positive-semidefiniteness

The fact that the Gramian matrix is positive-semidefinite can be seen from the following simple derivation:

x^{\mathrm {T} }{\mathbf {G}}x=\sum _{i,j}\langle v_{i},v_{j}\rangle x_{i}x_{j}=\sum _{i,j}\langle v_{i}x_{i},v_{j}x_{j}\rangle =\langle \sum _{i}v_{i}x_{i},\sum _{j}v_{j}x_{j}\rangle =\langle \sum _{i}v_{i}x_{i},\sum _{i}v_{i}x_{i}\rangle \geq 0.

The first equality follows from the definition of matrix multiplication, the second and third from the bi-linearity of the inner-product, and the last from the positive definiteness of the inner product. Note that this also shows that the Gramian matrix is positive definite if and only if the vectors $v_{i}$ are linearly independent.

Change of basis

Under change of basis, to an orthogonal basis, represented by an invertible orthogonal matrix $P$ , the Gram matrix will change by a matrix congruence to $P^{\mathrm {T} }GP$ .

Gram determinant

The Gram determinant or Gramian is the determinant of the Gram matrix:

G(x_{1},\dots ,x_{n})={\begin{vmatrix}\langle x_{1},x_{1}\rangle &\langle x_{1},x_{2}\rangle &\dots &\langle x_{1},x_{n}\rangle \\\langle x_{2},x_{1}\rangle &\langle x_{2},x_{2}\rangle &\dots &\langle x_{2},x_{n}\rangle \\\vdots &\vdots &\ddots &\vdots \\\langle x_{n},x_{1}\rangle &\langle x_{n},x_{2}\rangle &\dots &\langle x_{n},x_{n}\rangle \end{vmatrix}}.

Geometrically, the Gram determinant is the square of the volume of the parallelotope formed by the vectors. In particular, the vectors are linearly independent if and only if the Gram determinant is nonzero (if and only if the Gram matrix is nonsingular).

The Gram determinant can also be expressed in terms of the exterior product of vectors by

G(x_{1},\dots ,x_{n})=\|x_{1}\wedge \cdots \wedge x_{n}\|^{2}.

References

↑ Horn & Johnson 2013, p. 441
Theorem 7.2.10 Let $v_{1},\ldots ,v_{m}$ be vectors in an inner product space $V$ with inner product $\langle {\cdot ,\cdot }\rangle$ and let $G=[\langle {v_{j},v_{i}}\rangle ]_{i,j=1}^{m}\in M_{m}$ . Then
(a) $G$ is Hermitian and positive-semidefinite
(b) $G$ is positive-definite if and only if the vectors $v_{1},\ldots ,v_{m}$ are linearly-independent.
(c) $\operatorname {rank} G=\dim \operatorname {span} \{v_{1},\ldots ,v_{m}\}$
↑ Lanckriet, G. R. G.; Cristianini, N.; Bartlett, P.; Ghaoui, L. E.; Jordan, M. I. (2004). "Learning the kernel matrix with semidefinite programming". Journal of Machine Learning Research. 5: 27–72 [p. 29].

Horn, Roger A.; Johnson, Charles R. (2013). "7.2 Characterizations and Properties". Matrix Analysis (Second Edition). Cambridge University Press. ISBN 978-0-521-83940-2.

External links

Hazewinkel, Michiel, ed. (2001) [1994], "Gram matrix", Encyclopedia of Mathematics, Springer Science+Business Media B.V. / Kluwer Academic Publishers, ISBN 978-1-55608-010-4
Volumes of parallelograms by Frank Jones

This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.

[1] Horn & Johnson 2013, p. 441
Theorem 7.2.10 Let $v_{1},\ldots ,v_{m}$ be vectors in an inner product space $V$ with inner product $\langle {\cdot ,\cdot }\rangle$ and let $G=[\langle {v_{j},v_{i}}\rangle ]_{i,j=1}^{m}\in M_{m}$ . Then
(a) $G$ is Hermitian and positive-semidefinite
(b) $G$ is positive-definite if and only if the vectors $v_{1},\ldots ,v_{m}$ are linearly-independent.
(c) $\operatorname {rank} G=\dim \operatorname {span} \{v_{1},\ldots ,v_{m}\}$

[2] Lanckriet, G. R. G.; Cristianini, N.; Bartlett, P.; Ghaoui, L. E.; Jordan, M. I. (2004). "Learning the kernel matrix with semidefinite programming". Journal of Machine Learning Research. 5: 27–72 [p. 29].

Matrix classes
Explicitly constrained entries	(0,1) Alternant Anti-diagonal Anti-Hermitian Anti-symmetric Arrowhead Band Bidiagonal Binary Bisymmetric Block-diagonal Block Block tridiagonal Boolean Cauchy Centrosymmetric Conference Complex Hadamard Copositive Diagonally dominant Diagonal Discrete Fourier Transform Elementary Equivalent Frobenius Generalized permutation Hadamard Hankel Hermitian Hessenberg Hollow Integer Logical Markov Metzler Monomial Moore Nonnegative Partitioned Parisi Pentadiagonal Permutation Persymmetric Polynomial Positive Quaternionic Sign Signature Skew-Hermitian Skew-symmetric Skyline Sparse Sylvester Symmetric Toeplitz Triangular Tridiagonal Unitary Vandermonde Walsh Z
Constant	Exchange Hilbert Identity Lehmer Of ones Pascal Pauli Redheffer Shift Zero
Conditions on eigenvalues or eigenvectors	Companion Convergent Defective Diagonalizable Hurwitz Positive-definite Stability Stieltjes
Satisfying conditions on products or inverses	Congruent Idempotent or Projection Invertible Involutory Nilpotent Normal Orthogonal Orthonormal Singular Unimodular Unipotent Totally unimodular Weighing
With specific applications	Adjugate Alternating sign Augmented Bézout Carleman Cartan Circulant Cofactor Commutation Coxeter Derogatory Distance Duplication Elimination Euclidean distance Fundamental (linear differential equation) Generator Gramian Hessian Householder Jacobian Moment Payoff Pick Random Rotation Seifert Shear Similarity Symplectic Totally positive Transformation Wedderburn X–Y–Z
Used in statistics	Bernoulli Centering Correlation Covariance Design Dispersion Doubly stochastic Fisher information Hat Precision Stochastic Transition
Used in graph theory	Adjacency Biadjacency Degree Edmonds Incidence Laplacian Seidel adjacency Skew-adjacency Tutte
Used in science and engineering	Cabibbo–Kobayashi–Maskawa Density Fundamental (computer vision) Fuzzy associative Gamma Gell-Mann Hamiltonian Irregular Overlap S State transition Substitution Z (chemistry)
Related terms	Jordan canonical form Linear independence Matrix exponential Matrix representation of conic sections Perfect matrix Pseudoinverse Quaternionic matrix Row echelon form Wronskian
List of matrices Category:Matrices