Generalized singular value decomposition

The generalized singular value decomposition (GSVD) is a matrix decomposition more general than the singular value decomposition. It was introduced by Van Loan [1] in 1976 and later developed by Paige and Saunders. The SVD and the GSVD, as well as some other possible generalizations of the SVD [2][3][4] , are extensively used in the study of the conditioning and regularization of linear systems with respect to quadratic semi-norms

Let ${\displaystyle \mathbb {F} =\mathbb {R} }$, or ${\displaystyle \mathbb {F} =\mathbb {C} }$. Given matrices ${\displaystyle A\in \mathbb {F} ^{m\times n}}$ and ${\displaystyle B\in \mathbb {F} ^{p\times n}}$, their GSVD is given by [5]

${\displaystyle A=U\Sigma _{1}[X,0]Q^{*}}$

and

${\displaystyle B=V\Sigma _{2}[X,0]Q^{*}}$

where ${\displaystyle U\in \mathbb {F} ^{m\times m},V\in \mathbb {F} ^{p\times p}}$, and ${\displaystyle Q\in \mathbb {F} ^{n\times n}}$ are unitary matrices, and ${\displaystyle X\in \mathbb {F} ^{r\times r}}$ is non-singular, where ${\displaystyle r=rank([A^{*},B^{*}])}$. Also, ${\displaystyle \Sigma _{1}\in \mathbb {F} ^{m\times r}}$ is non-negative diagonal, and ${\displaystyle \Sigma _{2}\in \mathbb {F} ^{p\times r}}$ is non-negative block-diagonal, with diagonal blocks; ${\displaystyle \Sigma _{2}}$ is not always diagonal. It holds that ${\displaystyle \Sigma _{1}^{T}\Sigma _{1}=\lceil \alpha _{1}^{2},\dots ,\alpha _{r}^{2}\rfloor }$ and ${\displaystyle \Sigma _{2}^{T}\Sigma _{2}=\lceil \beta _{1}^{2},\dots ,\beta _{r}^{2}\rfloor }$, and that ${\displaystyle \Sigma _{1}^{T}\Sigma _{1}+\Sigma _{2}^{T}\Sigma _{2}=I_{r}}$. This implies ${\displaystyle 0\leq \alpha _{i},\beta _{i}\leq 1}$. The ratios ${\displaystyle \sigma _{i}=\alpha _{i}/\beta _{i}}$ are called the generalized singular values of ${\displaystyle A}$ and ${\displaystyle B}$. If ${\displaystyle B}$ is square and invertible, then the generalized singular values are the singular values, and ${\displaystyle U}$ and ${\displaystyle V}$ are the matrices of singular vectors, of the matrix ${\displaystyle AB^{-1}}$. Further, if ${\displaystyle B=I}$, then the GSVD reduces to the singular value decomposition, explaining the name.

The weighted version of the generalized singular value decomposition (GSVD) is a constrained matrix decomposition with constraints imposed on the left and right singular vectors of the singular value decomposition.[6][7][8] This form of the GSVD is an extension of the SVD as such. Given the SVD of an m×n real or complex matrix M

${\displaystyle M=U\Sigma V^{*}\,}$

where

${\displaystyle U^{*}W_{u}U=V^{*}W_{v}V=I.}$

Where I is the identity matrix and where ${\displaystyle U}$ and ${\displaystyle V}$ are orthonormal given their constraints (${\displaystyle W_{u}}$ and ${\displaystyle W_{v}}$). Additionally, ${\displaystyle W_{u}}$ and ${\displaystyle W_{v}}$ are positive definite matrices (often diagonal matrices of weights). This form of the GSVD is the core of certain techniques, such as generalized principal component analysis and Correspondence analysis.

The weighted form of the GSVD is called as such because, with the correct selection of weights, it generalizes many techniques (such as multidimensional scaling and linear discriminant analysis)[9]

The GSVD, formulated as a comparative spectral decomposition,[10] has been successfully applied to signal processing and data science, e.g., in genomic signal processing.[11][12][13]

These applications inspired several additional comparative spectral decompositions, i.e., the higher-order GSVD (HO GSVD)[14] and the tensor GSVD.[15][16]