Tensor Products

The tensor product $V\otimes W$ converts bilinear dependence on two vector spaces into ordinary linear dependence on one larger vector space. Its distinguished elements are the simple tensors $v\otimes w$, but the construction also contains arbitrary linear combinations of simple tensors.

For finite-dimensional spaces,

$$\dim(V\otimes W) =\dim V\,\dim W.$$

That multiplicative dimension is the first clue that a tensor product is not an ordered pair or a direct sum. This page develops the mathematical construction. The physical rule assigning tensor-product Hilbert spaces to distinguishable composite systems belongs to Tensor Products of Hilbert Spaces.

Why Bilinearity Is the Starting Point

Suppose a construction should combine $v\in V$ and $w\in W$ while remaining linear in either input when the other is fixed. It must obey

$$\begin{aligned} (v+v')\otimes w &=v\otimes w+v'\otimes w,\\ v\otimes(w+w') &=v\otimes w+v\otimes w',\\ (\alpha v)\otimes w &=\alpha(v\otimes w)\\ &=v\otimes(\alpha w). \end{aligned}$$

These relations have immediate consequences:

$$0\otimes w=0, \qquad v\otimes0=0.$$

They also show that a scalar can be moved from one factor to the other:

$$(\alpha v)\otimes w =\alpha(v\otimes w) =v\otimes(\alpha w).$$

Consequently, the factors of a nonzero simple tensor are not unique. For $\alpha\ne0$,

$$(\alpha v)\otimes(\alpha^{-1}w) =v\otimes w.$$

The symbol $\otimes$ therefore does not package two vectors as a literal ordered pair. It records their bilinear combination subject to these relations.

Algebraic Construction

One concrete construction begins with the free vector space generated by formal symbols $[v,w]$ for all $(v,w)\in V\times W$. Quotient by the subspace generated by the bilinearity relations

$$\begin{aligned} [v+v',w]-[v,w]-[v',w],\\ [v,w+w']-[v,w]-[v,w'],\\ [\alpha v,w]-\alpha[v,w],\\ [v,\alpha w]-\alpha[v,w]. \end{aligned}$$

The equivalence class of $[v,w]$ is denoted $v\otimes w$. This quotient is the algebraic tensor product $V\otimes W$.

The construction is characterized without reference to formal symbols by its universal property. Let

$$\begin{aligned} \tau&:V\times W\longrightarrow V\otimes W,\\ \tau(v,w)&=v\otimes w. \end{aligned}$$

For every bilinear map $\beta:V\times W\to X$, there is a unique linear map $\widetilde\beta:V\otimes W\to X$ such that

$$\beta=\widetilde\beta\circ\tau.$$

Equivalently,

$$\widetilde\beta(v\otimes w)=\beta(v,w)$$

on simple tensors, and linearity determines its value on every tensor. This property is what makes the tensor product canonical: any two constructions with it are uniquely isomorphic in a way that preserves simple tensors.

Product Bases and Dimension

Let

$$\mathcal E=(e_1,\ldots,e_m), \qquad \mathcal F=(f_1,\ldots,f_n)$$

be bases of $V$ and $W$. Then

$$\begin{aligned} \mathcal E\otimes\mathcal F &=\{e_i\otimes f_j\},\\ 1\le i&\le m, \qquad 1\le j\le n. \end{aligned}$$

is a basis of $V\otimes W$. It has $mn$ elements, so

$$\dim(V\otimes W)=mn.$$

If

$$v=\sum_i x_i e_i, \qquad w=\sum_j y_j f_j,$$

bilinearity gives

$$v\otimes w =\sum_{i,j}x_i y_j\, e_i\otimes f_j.$$

Every tensor $T\in V\otimes W$ has a unique coordinate expansion

$$T=\sum_{i,j}c_{ij}\, e_i\otimes f_j.$$

The coefficients $c_{ij}$ depend on the chosen product basis, just as vector coordinates depend on a basis. The tensor itself does not.

Simple and Nonsimple Tensors

A nonzero tensor is simple, decomposable, or rank one when it can be written as $v\otimes w$. Most tensors are not simple.

In chosen bases, collect the coefficients of $T$ into the $m\times n$ matrix

$$C_T=(c_{ij}).$$

If $T=v\otimes w$, then

$$c_{ij}=x_i y_j, \qquad C_T=xy^T,$$

so $C_T$ has matrix rank one. Conversely, every rank-one coefficient matrix factors as $xy^T$ and therefore defines a simple tensor. Thus, for nonzero $T$,

$$T\text{ is simple} \quad\Longleftrightarrow\quad \operatorname{rank}C_T=1.$$

For example,

$$\begin{aligned} T &=e_1\otimes f_1 +2e_1\otimes f_2\\ &\quad +3e_2\otimes f_1 +6e_2\otimes f_2 \end{aligned}$$

has coefficient matrix

$$\begin{pmatrix} 1&2\\ 3&6 \end{pmatrix} = \begin{pmatrix} 1\\3 \end{pmatrix} \begin{pmatrix} 1&2 \end{pmatrix}.$$

Therefore

$$T=(e_1+3e_2)\otimes(f_1+2f_2).$$

By contrast, $e_1\otimes f_1+e_2\otimes f_2$ has coefficient matrix $I_2$ and is not simple. The minimum number of simple tensors needed in a bipartite decomposition equals the rank of $C_T$. This is the entry point to Singular Value Decomposition and Schmidt Decomposition as Linear Algebra.

Inner Products

If $V$ and $W$ are finite-dimensional inner-product spaces, define the inner product first on simple tensors by

$$\langle v_1\otimes w_1, v_2\otimes w_2 \rangle = \langle v_1,v_2\rangle_V \langle w_1,w_2\rangle_W,$$

and extend sesquilinearly. In particular,

$$\lVert v\otimes w\rVert =\lVert v\rVert\,\lVert w\rVert.$$

If $\{e_i\}$ and $\{f_j\}$ are orthonormal, then

$$\langle e_i\otimes f_j, e_k\otimes f_\ell \rangle =\delta_{ik}\delta_{j\ell}.$$

The product basis is therefore orthonormal. For

$$T=\sum_{i,j}c_{ij}\, e_i\otimes f_j,$$

one has

$$\lVert T\rVert^2 =\sum_{i,j}\lvert c_{ij}\rvert^2.$$

In finite dimensions no further completion is needed. In infinite dimensions, the algebraic tensor product is generally incomplete in this norm; its completion is the Hilbert-space tensor product.

Tensor Products of Linear Maps

Let

$$A:V\to V', \qquad B:W\to W'$$

be linear maps. Their tensor product is the unique linear map

$$A\otimes B: V\otimes W\longrightarrow V'\otimes W'$$

defined by

$$(A\otimes B)(v\otimes w) =Av\otimes Bw.$$

Well-definedness follows from the universal property because $(v,w)\mapsto Av\otimes Bw$ is bilinear.

Compatible tensor-product maps obey

$$\begin{aligned} (A\otimes B)(C\otimes D) &=(AC)\otimes(BD),\\ (A\otimes B)^\dagger &=A^\dagger\otimes B^\dagger,\\ I_V\otimes I_W &=I_{V\otimes W}. \end{aligned}$$

If $A$ and $B$ are invertible, unitary, Hermitian, or positive, then $A\otimes B$ inherits the corresponding property, with the qualification that the tensor product of two Hermitian operators is Hermitian but need not be positive unless both spectra have compatible signs.

Local-factor maps commute:

$$\begin{aligned} (A\otimes I_W)(I_V\otimes B) &=A\otimes B,\\ (I_V\otimes B)(A\otimes I_W) &=A\otimes B. \end{aligned}$$

This identity is algebraic; it does not assert that all physical operations on separated subsystems can be implemented independently.

Spectral and Trace Identities

If $Av_i=a_i v_i$ and $Bw_j=b_jw_j$, then

$$(A\otimes B)(v_i\otimes w_j) =a_i b_j(v_i\otimes w_j).$$

When eigenbases exist, the eigenvalues of $A\otimes B$ are all products $a_i b_j$, counted with multiplicity. In finite dimensions,

$$\begin{aligned} \operatorname{rank}(A\otimes B) &=\operatorname{rank}A\, \operatorname{rank}B,\\ \operatorname{tr}(A\otimes B) &=(\operatorname{tr}A) (\operatorname{tr}B). \end{aligned}$$

If $A$ is $m\times m$ and $B$ is $n\times n$, then

$$\det(A\otimes B) =(\det A)^n(\det B)^m.$$

These formulas include algebraic multiplicities. They are particularly useful for checking symbolic and numerical constructions.

Kronecker Product Matrices

Choose ordered bases of $V$ and $W$, then choose an ordering of the product basis. With the lexicographic convention

$$\left( \begin{gathered} e_1\otimes f_1,\ldots,e_1\otimes f_n,\\ e_2\otimes f_1,\ldots,e_m\otimes f_n \end{gathered} \right),$$

the matrix of $A\otimes B$ is the Kronecker product

$$[A\otimes B] = \begin{pmatrix} a_{11}B&\cdots&a_{1m}B\\ \vdots&\ddots&\vdots\\ a_{m1}B&\cdots&a_{mm}B \end{pmatrix}.$$

For example, if

$$A= \begin{pmatrix} 1&2\\ 0&-1 \end{pmatrix}, \qquad B= \begin{pmatrix} 0&1\\ 1&0 \end{pmatrix},$$

then

$$A\otimes B = \begin{pmatrix} 0&1&0&2\\ 1&0&2&0\\ 0&0&0&-1\\ 0&0&-1&0 \end{pmatrix}.$$

Changing the product-basis ordering conjugates this matrix by a permutation matrix. The abstract operator is unchanged, but coordinate arrays and code must use the same convention.

Factor Order and Canonical Isomorphisms

Tensor products are associative and symmetric up to canonical isomorphism, not literal equality. The associator sends

$$(u\otimes v)\otimes w \longmapsto u\otimes(v\otimes w),$$

and the swap map sends

$$S_{V,W}(v\otimes w) =w\otimes v.$$

These maps justify suppressing parentheses when factor order is otherwise clear. They do not justify silently reversing factors. In coordinates, the swap is represented by a nontrivial permutation matrix, often called a commutation matrix.

Tensor products distribute over direct sums:

$$(V\oplus W)\otimes X \cong (V\otimes X)\oplus(W\otimes X).$$

But $V\otimes W$ is not generally isomorphic to $V\oplus W$: their dimensions are multiplicative and additive, respectively. See Direct Sums.

Dual Spaces and Linear Maps

In finite dimensions there are natural isomorphisms

$$\begin{aligned} (V\otimes W)^* &\cong V^*\otimes W^*,\\ \operatorname{Hom}(V,W) &\cong W\otimes V^*. \end{aligned}$$

Under the second isomorphism, a simple tensor $w\otimes\varphi$ corresponds to the rank-one map

$$v\longmapsto\varphi(v)w.$$

This identification explains why outer products, rank-one operators, and two-index coefficient arrays are manifestations of the same tensor structure. In infinite dimensions the analogous identifications require topological qualifications and are not automatic for arbitrary bounded operators.

Subsystem Notation

When factors are labeled $A$ and $B$, write

$$\mathcal H_A\otimes\mathcal H_B$$

and keep labels on ambiguous vectors and identities:

$$\lvert i\rangle_A\otimes\lvert j\rangle_B, \qquad X_A\otimes I_B.$$

Compact notation such as

$$\lvert ij\rangle \equiv \lvert i\rangle_A\otimes\lvert j\rangle_B$$

is safe only after the factor order has been declared. Likewise, writing $X_A$ as shorthand for $X_A\otimes I_B$ is convenient only when the ambient space is unambiguous.

The notation is mathematical. The statement that a physical composite has state space $\mathcal H_A\otimes\mathcal H_B$ is a composition postulate; see Tensor Products in Core Formalism. Site-wide ordering conventions are collected in Tensor-Product Ordering.

Numerical Representation

Tensor-product dimensions grow multiplicatively. If $k$ factors each have dimension $d$, the full dimension is $d^k$, and a dense operator has $d^{2k}$ entries. This growth is structural, not an implementation accident.

For reliable computation:

• record the factor order and product-basis order explicitly;
• distinguish row-major and column-major reshaping conventions;
• test a Kronecker implementation on basis vectors;
• exploit sparse and structured tensor products;
• apply local operators by reshaping and contracting when possible instead of forming the full matrix;
• compare abstract identities before comparing flattened arrays;
• track whether complex conjugation is required by a dual or adjoint.

A reshape is not itself a basis-independent operation. It becomes meaningful only after dimensions and index ordering have been specified.

Common Mistakes

• Treating $v\otimes w$ as an ordered pair $(v,w)$.
• Assuming every tensor is simple.
• Expanding $(v+v')\otimes(w+w')$ but omitting the cross terms.
• Forgetting that scalar factors can move between tensor factors.
• Confusing the abstract tensor product with one Kronecker coordinate matrix.
• Reversing factor order without applying the swap isomorphism.
• Writing $A$ on a composite space without the required identity factors.
• Comparing flattened arrays produced by different basis-order conventions.
• Assuming a direct sum and a tensor product describe the same composition.
• Using the finite-dimensional dual-space identities without topological qualifications in infinite dimensions.

Exercises

1. Let $V$ have basis $(e_1,e_2)$ and $W$ have basis $(f_1,f_2,f_3)$. Expand

   $$(2e_1-e_2)\otimes(f_1+3f_3)$$

   in the product basis and give its coefficient matrix.

Solution

Set $X=(2e_1-e_2)\otimes(f_1+3f_3)$. Bilinearity gives

$$\begin{aligned} X &=2e_1\otimes f_1 +6e_1\otimes f_3\\ &\quad -e_2\otimes f_1 -3e_2\otimes f_3. \end{aligned}$$

With rows indexed by $(e_1,e_2)$ and columns by $(f_1,f_2,f_3)$, the coefficient matrix is

$$\begin{pmatrix} 2&0&6\\ -1&0&-3 \end{pmatrix}.$$

It has rank one, as expected for a simple tensor.

2. Determine which of the following tensors are simple:

   $$\begin{aligned} T_1 &=e_1\otimes f_1 +e_1\otimes f_2\\ &\quad +2e_2\otimes f_1 +2e_2\otimes f_2,\\ T_2 &=e_1\otimes f_1 +e_2\otimes f_2. \end{aligned}$$

Solution

Their coefficient matrices are

$$C_1= \begin{pmatrix} 1&1\\ 2&2 \end{pmatrix}, \qquad C_2= \begin{pmatrix} 1&0\\ 0&1 \end{pmatrix}.$$

The first matrix has rank one and factors as

$$C_1= \begin{pmatrix} 1\\2 \end{pmatrix} \begin{pmatrix} 1&1 \end{pmatrix}.$$

Hence

$$T_1=(e_1+2e_2)\otimes(f_1+f_2).$$

The second matrix has rank two, so $T_2$ is not simple.

3. Prove that both local-factor products equal $A\otimes B$:

   $$\begin{aligned} (A\otimes I_W)(I_V\otimes B) &=A\otimes B,\\ (I_V\otimes B)(A\otimes I_W) &=A\otimes B. \end{aligned}$$

   by checking simple tensors. Why is that sufficient?

Solution

For every $v\in V$ and $w\in W$, set $x=v\otimes w$ and abbreviate

$$\begin{aligned} L&=(A\otimes I_W)(I_V\otimes B),\\ R&=(I_V\otimes B)(A\otimes I_W). \end{aligned}$$

Then

$$\begin{aligned} Lx &=(A\otimes I_W)(v\otimes Bw)\\ &=Av\otimes Bw. \end{aligned}$$

In the reverse order,

$$\begin{aligned} Rx &=(I_V\otimes B)(Av\otimes w)\\ &=Av\otimes Bw. \end{aligned}$$

The two linear maps agree on every simple tensor. Simple tensors span $V\otimes W$, so the maps agree everywhere.

4. Let $A$ have eigenvalues $2$ and $-1$, and let $B$ have eigenvalues $3$, $4$, and $5$, with eigenbases in both spaces. List the eigenvalues of $A\otimes B$, compute its trace, and verify the trace-product identity.

Solution

The product eigenvectors have eigenvalues

$$6,\ 8,\ 10,\ -3,\ -4,\ -5.$$

Their sum is

$$\begin{aligned} \operatorname{tr}(A\otimes B) &=24-12\\ &=12. \end{aligned}$$

Separately,

$$\operatorname{tr}A=1, \qquad \operatorname{tr}B=12,$$

so

$$\operatorname{tr}(A\otimes B) =(\operatorname{tr}A)(\operatorname{tr}B) =12.$$

References

• S. Axler, Linear Algebra Done Right, 3rd ed., Springer, 2015.
• S. Roman, Advanced Linear Algebra, 3rd ed., Springer, 2008.
• W. H. Greub, Multilinear Algebra, 2nd ed., Springer, 1978.
• R. A. Horn and C. R. Johnson, Topics in Matrix Analysis, Cambridge University Press, 1991.
• M. A. Nielsen and I. L. Chuang, Quantum Computation and Quantum Information, Cambridge University Press, 2010.