Singular Value Decomposition

The singular value decomposition, or SVD, factors a finite-dimensional linear map into orthonormal input directions, nonnegative stretching factors, and orthonormal output directions.

Unlike eigenvalue diagonalization, SVD applies to every finite matrix, including rectangular matrices and matrices that are not diagonalizable. In quantum mechanics it appears in Schmidt decomposition, operator approximations, numerical conditioning, and the analysis of maps between finite Hilbert spaces.

Statement

Let

$$A:V\to W$$

be a linear map between finite-dimensional complex inner-product spaces. If the rank of $A$ is $k$, then there are orthonormal vectors

$$\{v_1,\ldots,v_k\}\subset V, \qquad \{u_1,\ldots,u_k\}\subset W,$$

and positive numbers

$$s_1\ge s_2\ge\cdots\ge s_k>0$$

such that

$$A = \sum_{r=1}^k s_r\, \lvert u_r\rangle\langle v_r\rvert.$$

The numbers $s_r$ are the nonzero singular values of $A$. The vectors $v_r$ are right singular vectors, and the vectors $u_r$ are left singular vectors.

In matrix form, after choosing orthonormal bases,

$$A=U\Sigma V^\dagger.$$

Here $U$ and $V$ are unitary matrices, and $\Sigma$ is a rectangular diagonal matrix whose diagonal entries are the singular values, padded with zeros if necessary.

How the Pieces Are Found

The positive operator

$$A^\dagger A:V\to V$$

is Hermitian and positive semidefinite. Its eigenvalues are nonnegative. If

$$A^\dagger A\,v_r=s_r^2 v_r, \qquad s_r>0,$$

then

$$u_r=\frac{1}{s_r}Av_r$$

is a unit vector in $W$. The resulting vectors satisfy

$$Av_r=s_r u_r, \qquad A^\dagger u_r=s_r v_r.$$

Thus the SVD is closely related to the Spectral Decomposition of $A^\dagger A$. The numerical lesson is more subtle: one should not usually compute an SVD by explicitly diagonalizing $A^\dagger A$, because forming $A^\dagger A$ squares the condition number and can lose small singular directions.

Compact and Full Forms

The sum

$$A = \sum_{r=1}^k s_r\, \lvert u_r\rangle\langle v_r\rvert$$

is the compact SVD. It keeps only the nonzero singular values.

The full matrix form extends $\{v_r\}$ to an orthonormal basis of $V$ and $\{u_r\}$ to an orthonormal basis of $W$. The extra basis vectors account for the kernel and the zero singular values.

If $A$ is an $m\times n$ matrix, then $\Sigma$ has size $m\times n$. Its nonzero diagonal entries are $s_1,\ldots,s_k$, where

$$k=\operatorname{rank}A.$$

Rank, Kernel, and Range

The rank of $A$ is the number of positive singular values:

$$\operatorname{rank}A = \#\{r:s_r>0\}.$$

The null space is spanned by right singular vectors with zero singular value. Equivalently,

$$Av=0 \quad \Longleftrightarrow \quad A^\dagger A\,v=0.$$

The range of $A$ is spanned by the left singular vectors with positive singular value:

$$\operatorname{im}A = \operatorname{span}\{u_1,\ldots,u_k\}.$$

These statements are often the cleanest way to find the effective support of a linear map.

Geometric Meaning

For a real matrix acting between Euclidean spaces, the SVD says:

1. rotate or reflect the input coordinates using $V^\dagger$;
2. stretch orthogonal axes by $s_1,\ldots,s_k$ using $\Sigma$;
3. rotate or reflect the result using $U$.

Over complex Hilbert spaces, replace rotations and reflections by unitary changes of orthonormal basis. The singular values are still the principal stretching factors:

$$\lVert Av_r\rVert=s_r.$$

The largest singular value gives the operator norm:

$$\lVert A\rVert = s_1$$

for the standard Hilbert-space norm.

SVD Versus Diagonalization

Eigenvalue diagonalization concerns a square operator $A:V\to V$ and asks whether there is a basis of eigenvectors:

$$A=PDP^{-1}.$$

SVD concerns a map $A:V\to W$ and uses two orthonormal bases, one in the domain and one in the codomain:

$$A=U\Sigma V^\dagger.$$

Important differences:

• SVD exists for every finite matrix; diagonalization does not.
• SVD works for rectangular maps; ordinary diagonalization is a square-matrix notion.
• singular values are nonnegative real numbers; eigenvalues may be complex.
• right and left singular vectors may live in different spaces.
• for normal positive semidefinite matrices, singular values coincide with eigenvalues; in general they do not.

The ordinary diagonalization story is Diagonalization, and the finite-dimensional normal-operator case is Normal Operators.

Quantum Use

The most important quantum use is the Schmidt decomposition. A bipartite vector

$$\lvert\Psi\rangle = \sum_{i,j}C_{ij} \lvert i\rangle_A\lvert j\rangle_B$$

has a coefficient matrix $C$. Applying the SVD to $C$ gives the Schmidt coefficients and Schmidt bases. The physics-facing theorem is Schmidt Decomposition.

For the coefficient-matrix derivation itself, see Schmidt Decomposition as Linear Algebra.

SVD also appears when estimating numerical rank, truncating a state or operator to its dominant components, and diagnosing ill-conditioned calculations. If small singular values are physically meaningful, truncating them is an approximation that must be justified by the model, not merely by convenience.

Pseudoinverse

If

$$A = \sum_{r=1}^k s_r\, \lvert u_r\rangle\langle v_r\rvert,$$

then the Moore-Penrose pseudoinverse is

$$A^+ = \sum_{r=1}^k \frac{1}{s_r}\, \lvert v_r\rangle\langle u_r\rvert.$$

It inverts $A$ on the supported singular directions and ignores the zero singular directions. This is useful in least-squares problems and in numerical linear algebra. In quantum calculations, the same warning applies as elsewhere: dividing by very small singular values can amplify noise and discretization error.

Quantum Linear Algebra owns the access-aware use of these objects in the quantum linear-systems problem—including the normalized solution-state, success, error, conditioning, and readout contract—while this page retains the SVD and Moore–Penrose pseudoinverse definitions and derivations.

Worked Example

Let

$$A = \begin{pmatrix} 1 & 1\\ 0 & 0 \end{pmatrix}.$$

Then

$$A^\dagger A = \begin{pmatrix} 1 & 1\\ 1 & 1 \end{pmatrix}.$$

The eigenvalues of $A^\dagger A$ are $2$ and $0$. A normalized eigenvector for eigenvalue $2$ is

$$v_1 = \frac{1}{\sqrt2} \begin{pmatrix} 1\\ 1 \end{pmatrix}.$$

The positive singular value is

$$s_1=\sqrt2.$$

The corresponding left singular vector is

$$u_1 = \frac{1}{s_1}Av_1 = \begin{pmatrix} 1\\ 0 \end{pmatrix}.$$

Therefore the compact SVD is

$$A = \sqrt2\, \lvert u_1\rangle\langle v_1\rvert.$$

In words, $A$ keeps only the input direction proportional to $(1,1)^T$, stretches it by $\sqrt2$, and sends it to the first output basis vector.

Common Mistakes

• Treating singular values as eigenvalues of $A$ rather than square roots of eigenvalues of $A^\dagger A$.
• Forgetting that SVD uses two bases, one for the domain and one for the codomain.
• Assuming the left and right singular vectors are the same for a non-normal matrix.
• Computing singular values by forming $A^\dagger A$ in a sensitive numerical problem.
• Calling tiny nonzero singular values exactly zero without estimating numerical error.
• Treating an SVD truncation as exact physics without stating the approximation.
• Forgetting that degenerate singular values leave freedom to rotate the corresponding singular subspaces.

Cross-Links

• Matrices as Linear Maps
• Orthonormal Bases
• Spectral Decomposition
• Diagonalization
• Normal Operators
• Tensor Products
• Schmidt Decomposition as Linear Algebra
• Matrix Diagonalization
• Schmidt Decomposition

References

• G. H. Golub and C. F. Van Loan, Matrix Computations, 4th ed., Johns Hopkins University Press, 2013.
• R. A. Horn and C. R. Johnson, Matrix Analysis, 2nd ed., Cambridge University Press, 2012.
• L. N. Trefethen and D. Bau, Numerical Linear Algebra, SIAM, 1997.
• G. Strang, Linear Algebra and Its Applications, 4th ed., Brooks/Cole, 2006.
• M. A. Nielsen and I. L. Chuang, Quantum Computation and Quantum Information, Cambridge University Press, 2010.

Exercises

1. Find the singular values of

$$A = \begin{pmatrix} 3 & 0\\ 0 & 1 \end{pmatrix}.$$

Solution

Here

$$A^\dagger A = \begin{pmatrix} 9 & 0\\ 0 & 1 \end{pmatrix}.$$

The singular values are the square roots of the eigenvalues:

$$s_1=3, \qquad s_2=1.$$

2. Show that the rank of a matrix equals the number of positive singular values.

Solution

In the compact SVD,

$$A = \sum_{r=1}^k s_r\, \lvert u_r\rangle\langle v_r\rvert$$

with all $s_r>0$. The range is spanned by $u_1,\ldots,u_k$, so its dimension is $k$. Therefore $\operatorname{rank}A=k$, the number of positive singular values.

3. Let

$$A = \begin{pmatrix} 1 & 1\\ 0 & 0 \end{pmatrix}.$$

Find a normalized vector in $\ker A$.

Solution

The equation $Ax=0$ gives $x_1+x_2=0$. A normalized vector in the kernel is

$$\frac{1}{\sqrt2} \begin{pmatrix} 1\\ -1 \end{pmatrix}.$$

It is the right singular vector associated with the zero singular value.

4. Why is SVD more appropriate than eigenvalue diagonalization for a rectangular coefficient matrix in a bipartite state?

Solution

A rectangular matrix does not define an operator from one space to itself, so ordinary eigenvalue diagonalization is not the right tool. SVD applies to maps between different finite-dimensional inner-product spaces and supplies orthonormal bases on both sides. That is exactly the structure needed for the Schmidt decomposition of a bipartite state.