Variance and Covariance

Variance measures the spread of one random variable. Covariance measures how two random variables fluctuate together.

For a real random variable $X$ with mean

$$\mu_X=\mathbb E[X],$$

the variance is

$$\operatorname{Var}(X) = \mathbb E[(X-\mu_X)^2].$$

The standard deviation is

$$\sigma_X = \sqrt{\operatorname{Var}(X)}.$$

In quantum mechanics, standard deviations become the $\Delta A$ appearing in uncertainty relations. The operator-specific treatment is Variance and Standard Deviation; this page supplies the probability language underneath it.

Finite Moments

Variance is defined only when the required expectations exist. A random variable can have a well-defined distribution but no finite variance.

For a discrete random variable with values $x_i$ and probabilities $p_i$,

$$\mathbb E[X] = \sum_i x_i p_i, \qquad \mathbb E[X^2] = \sum_i x_i^2p_i.$$

For a continuous random variable with density $f_X(x)$,

$$\mathbb E[X] = \int_{-\infty}^{\infty}x f_X(x)\,dx, \qquad \mathbb E[X^2] = \int_{-\infty}^{\infty}x^2 f_X(x)\,dx,$$

when the integrals converge in the required sense.

Finite normalization does not imply finite variance. A heavy-tailed distribution can have total probability one but divergent second moment.

Variance Formula

Expanding the square gives the computational form

$$\operatorname{Var}(X) = \mathbb E[X^2] - \mathbb E[X]^2.$$

Derivation:

$$\begin{aligned} \mathbb E[(X-\mu_X)^2] &= \mathbb E[X^2-2\mu_X X+\mu_X^2]\\ &= \mathbb E[X^2] -2\mu_X\mathbb E[X] +\mu_X^2\\ &= \mathbb E[X^2]-\mu_X^2. \end{aligned}$$

Variance is nonnegative:

$$\operatorname{Var}(X)\ge0,$$

because it is the expectation of a square. It has the squared units of $X$, while $\sigma_X$ has the same units as $X$.

Standard Deviation

The standard deviation

$$\sigma_X=\sqrt{\operatorname{Var}(X)}$$

is the most common quoted measure of spread because it has the same units as the measured quantity.

It is not the same thing as an experimental error. In probability theory it describes the spread of the distribution. In quantum mechanics, $\Delta A$ describes the spread of ideal measurement outcomes predicted by a state, even before apparatus imperfections are considered.

Two-Outcome Example

Let $X$ take values $+1$ and $-1$ with

$$\mathbb P(X=+1)=q, \qquad \mathbb P(X=-1)=1-q.$$

Then

$$\mathbb E[X] = 2q-1.$$

Since $X^2=1$ always,

$$\mathbb E[X^2]=1.$$

Therefore

$$\operatorname{Var}(X) = 1-(2q-1)^2 = 4q(1-q).$$

The variance is zero when one outcome is certain and maximal when $q=1/2$.

Gaussian Example

For a normal density with mean $\mu$ and standard deviation $s$,

$$f(x) = \frac{1}{\sqrt{2\pi}\,s} \exp\!\left[ - \frac{(x-\mu)^2}{2s^2} \right],$$

one has

$$\mathbb E[X]=\mu, \qquad \operatorname{Var}(X)=s^2.$$

Different wavefunction conventions sometimes use different width parameters. The variance is the reliable way to identify the physical width of the probability density.

For the broader Gaussian toolkit, including multivariate covariance matrices and Gaussian integrals, see Gaussian Distributions.

Covariance

For two real random variables $X$ and $Y$ on the same probability space, the covariance is

$$\operatorname{Cov}(X,Y) = \mathbb E[(X-\mu_X)(Y-\mu_Y)].$$

Equivalently,

$$\operatorname{Cov}(X,Y) = \mathbb E[XY] - \mathbb E[X]\mathbb E[Y].$$

Covariance is positive when $X$ and $Y$ tend to fluctuate in the same direction, negative when they tend to fluctuate in opposite directions, and zero when their linear fluctuation is absent.

The units of $\operatorname{Cov}(X,Y)$ are the units of $X$ times the units of $Y$.

Joint Distributions Are Required

Covariance depends on the joint distribution of $X$ and $Y$, not only on their separate distributions.

For a joint density $f_{X,Y}(x,y)$,

$$\mathbb E[XY] = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} xy\,f_{X,Y}(x,y)\,dx\,dy.$$

The marginal densities

$$f_X(x) = \int_{-\infty}^{\infty} f_{X,Y}(x,y)\,dy, \qquad f_Y(y) = \int_{-\infty}^{\infty} f_{X,Y}(x,y)\,dx$$

do not determine the covariance by themselves.

This point matters in quantum mechanics because incompatible observables generally do not come with a single classical joint distribution of pre-existing values; see Classical Probability versus Quantum Probability.

Correlation Coefficient

When $\sigma_X>0$ and $\sigma_Y>0$, the correlation coefficient is

$$\rho_{XY} = \frac{\operatorname{Cov}(X,Y)}{\sigma_X\sigma_Y}.$$

It is dimensionless and satisfies

$$-1\le \rho_{XY}\le 1.$$

The bound follows from the Cauchy–Schwarz inequality applied to the centered variables $X-\mu_X$ and $Y-\mu_Y$.

Correlation rescales covariance so that variables with different units can be compared. It measures linear association, not all possible dependence.

Uncorrelated Does Not Mean Independent

If $X$ and $Y$ are independent and have finite second moments, then

$$\operatorname{Cov}(X,Y)=0.$$

The converse is false. For example, let $X$ be uniformly distributed on $[-1,1]$ and let

$$Y=X^2.$$

Then

$$\mathbb E[X]=0, \qquad \mathbb E[XY] = \mathbb E[X^3]=0,$$

so $\operatorname{Cov}(X,Y)=0$. But $Y$ is fully determined by $X$, so they are not independent.

Zero covariance means no linear covariance, not no relationship.

Covariance Matrices

For random variables $X_1,\ldots,X_n$, the covariance matrix is

$$\Sigma_{ij} = \operatorname{Cov}(X_i,X_j).$$

It is symmetric:

$$\Sigma_{ij}=\Sigma_{ji}.$$

It is also positive semidefinite. For any real vector $\mathbf a$,

$$\mathbf a^T\Sigma\mathbf a = \operatorname{Var} \left( \sum_i a_iX_i \right) \ge0.$$

This property is the probability-theory ancestor of many positivity constraints in quantum mechanics, including uncertainty inequalities and positivity of density operators.

Quantum Preview

For a quantum observable $A$ in a state, the outcome variance is the classical variance of the Born-rule measurement distribution:

$$(\Delta A)^2 = \langle A^2\rangle - \langle A\rangle^2.$$

For two commuting observables with a joint measurement, covariance has the ordinary interpretation as covariance of two jointly measured outcomes.

For noncommuting observables, one must be more careful. A common symmetrized quantum covariance is

$$\operatorname{Cov}_\psi(A,B) = \frac12 \left\langle \{\widetilde A,\widetilde B\} \right\rangle,$$

where

$$\widetilde A=A-\langle A\rangle I, \qquad \widetilde B=B-\langle B\rangle I,$$

and

$$\{\widetilde A,\widetilde B\} = \widetilde A\widetilde B+\widetilde B\widetilde A.$$

This is not the same as assuming that $A$ and $B$ have simultaneous hidden classical values. It is an operator expression defined by the state and observables.

The general uncertainty relation includes both spread and noncommutativity. A basic form is

$$\Delta A\,\Delta B \ge \frac12 \left\lvert \langle[A,B]\rangle \right\rvert.$$

The full discussion belongs in General Uncertainty Relations.

Common Mistakes

• Treating variance as the mean absolute deviation.
• Forgetting to subtract the square of the mean in $\mathbb E[X^2]-\mathbb E[X]^2$.
• Assuming zero mean implies zero variance.
• Forgetting that variance has squared units.
• Computing covariance from marginal distributions without a joint distribution.
• Treating zero covariance as independence.
• Reading quantum uncertainty as ordinary apparatus error.
• Assigning covariance to noncommuting observables without specifying the measurement or operator convention.

Cross-Links

• Random Variables
• Probability Densities
• Expectation Values
• Inner Products
• Conditional Probability
• Characteristic Functions
• Gaussian Distributions
• Monte Carlo Basics
• Classical Probability versus Quantum Probability
• Variance and Standard Deviation
• General Uncertainty Relations
• Commutators and Anticommutators

References

• W. Feller, An Introduction to Probability Theory and Its Applications, Volume I, 3rd ed., Wiley, 1968.
• P. Billingsley, Probability and Measure, 3rd ed., Wiley, 1995.
• R. Durrett, Probability: Theory and Examples, 5th ed., Cambridge University Press, 2019.
• R. Shankar, Principles of Quantum Mechanics, 2nd ed., Springer, 1994.
• J. J. Sakurai and J. Napolitano, Modern Quantum Mechanics, 3rd ed., Cambridge University Press, 2020.

Exercises

1. A random variable takes values $0$ and $1$ with $\mathbb P(X=1)=q$. Compute its variance.

Solution

Here

$$\mathbb E[X]=q, \qquad \mathbb E[X^2]=q,$$

because $X^2=X$ for values $0$ and $1$. Therefore

$$\operatorname{Var}(X) = q-q^2 = q(1-q).$$

2. Let $X$ take values $+1$ and $-1$ with equal probability. Compute $\mathbb E[X]$, $\mathbb E[X^2]$, and $\operatorname{Var}(X)$.

Solution

The mean is

$$\mathbb E[X] = \frac12(1)+\frac12(-1) = 0.$$

Since $X^2=1$ always,

$$\mathbb E[X^2]=1.$$

Thus

$$\operatorname{Var}(X) = 1-0^2 = 1.$$

3. Suppose $\operatorname{Var}(X)=4$, $\operatorname{Var}(Y)=9$, and $\operatorname{Cov}(X,Y)=3$. Find the correlation coefficient.

Solution

The standard deviations are $\sigma_X=2$ and $\sigma_Y=3$. Therefore

$$\rho_{XY} = \frac{3}{2\cdot3} = \frac12.$$

4. Show that adding a constant does not change variance: $\operatorname{Var}(X+c)=\operatorname{Var}(X)$.

Solution

The mean of $X+c$ is $\mu_X+c$. Therefore

$$X+c-(\mu_X+c)=X-\mu_X.$$

So

$$\operatorname{Var}(X+c) = \mathbb E[(X-\mu_X)^2] = \operatorname{Var}(X).$$

5. For centered variables with $\mathbb E[X]=\mathbb E[Y]=0$, what does $\operatorname{Cov}(X,Y)$ reduce to?

Solution

Using

$$\operatorname{Cov}(X,Y) = \mathbb E[XY]-\mathbb E[X]\mathbb E[Y],$$

and the centered assumptions, one gets

$$\operatorname{Cov}(X,Y) = \mathbb E[XY].$$