Conditioning and Stability

Conditioning measures how sensitive a mathematical problem is to small changes in its input. Stability measures whether a numerical algorithm introduces more error than the problem itself forces.

These are different questions. Error Estimates explains how these diagnostics enter an explicit uncertainty budget.

Question	Concept
Does the exact answer change a lot when the input changes slightly?	conditioning
Does the algorithm behave like the exact solution of a nearby problem?	stability
Is the computed answer close to the exact answer for the original data?	forward accuracy
Does the computed answer satisfy the original equations nearly?	residual

A poor result can come from an ill-conditioned problem, an unstable algorithm, an insufficient discretization, or a coding mistake. The purpose of this page is to separate those possibilities.

Why Quantum Mechanics Needs It

Quantum calculations routinely ask for quantities that are sensitive:

• small energy splittings between nearly degenerate states;
• eigenvectors inside a nearly degenerate eigenspace;
• wavefunctions expanded in a nearly linearly dependent basis;
• tunneling amplitudes that are exponentially small;
• scattering phase shifts near resonances;
• long-time time evolution;
• expectation values obtained from subtracting large terms;
• inverse or reconstruction problems from incomplete data.

In such cases, “the code ran” and “the residual is small” are not enough. One must ask whether the problem is well conditioned at the requested accuracy and whether the algorithm is stable for that problem class.

Conditioning as Sensitivity

Let a problem be represented abstractly as a map

$$y=f(x),$$

where $x$ is the input data and $y$ is the exact answer. A condition number estimates how much the answer can change when the input is perturbed.

For a scalar differentiable function with $f(x)\ne0$, a relative condition number is

$$\kappa_f(x) = \left\lvert \frac{x f'(x)}{f(x)} \right\rvert.$$

It means that a small relative perturbation in $x$ can produce, to first order,

$$\frac{\lvert\Delta y\rvert}{\lvert y\rvert} \approx \kappa_f(x) \frac{\lvert\Delta x\rvert}{\lvert x\rvert}.$$

Large $\kappa_f$ means the problem itself is sensitive. No algorithm can recover more information than the data determine.

Matrix Condition Numbers

For a linear system

$$Ax=b,$$

the condition number of an invertible matrix is

$$\kappa(A) = \lVert A\rVert\,\lVert A^{-1}\rVert.$$

In the Euclidean norm, this can be written using singular values:

$$\kappa_2(A) = \frac{\sigma_{\max}(A)}{\sigma_{\min}(A)}.$$

If $\sigma_{\min}(A)$ is small, the matrix is close to singular. Then small perturbations in $A$ or $b$ can cause large changes in $x$.

A typical first-order warning is

$$\frac{\lVert \Delta x\rVert}{\lVert x\rVert} \lesssim \kappa(A) \frac{\lVert \Delta b\rVert}{\lVert b\rVert},$$

when the perturbation is only in $b$ and the bound is interpreted in the appropriate norm and small-perturbation regime.

This is why Singular Value Decomposition is more than a linear-algebra ornament: singular values diagnose near-null directions and loss of numerical information.

Quantum Linear Algebra owns how intrinsic $\kappa_2(A)$ and access-induced block conditioning enter QLSP success, error, and resource claims; this page retains general problem sensitivity, residuals, forward and backward error, and classical stability.

Forward Error, Backward Error, and Residuals

The forward error compares the computed answer $\tilde y$ with the exact answer $y$:

$$\text{forward error} = \lVert \tilde y-y\rVert.$$

The backward error asks how much the input data would need to change to make $\tilde y$ exact:

$$\tilde y=f(x+\Delta x).$$

An algorithm is backward stable when the required perturbation $\Delta x$ is comparable to the unavoidable rounding scale:

$$\frac{\lVert \Delta x\rVert}{\lVert x\rVert} = O(u),$$

up to moderate constants and problem-size factors.

For an equation such as $Ax=b$, the residual is

$$r=b-A\tilde x.$$

A small residual says that $\tilde x$ nearly satisfies the equation. But if $A$ is ill conditioned, a small residual may still correspond to a large forward error because

$$x-\tilde x = A^{-1}r.$$

Residuals are essential diagnostics, but they should be interpreted together with conditioning.

Stable Algorithms Do Not Fix Ill Conditioning

A backward stable algorithm gives an answer that is exact for a nearby problem. That is usually the right standard. But if the original problem is ill conditioned, the nearby problem may have a noticeably different answer.

Thus stability and conditioning combine schematically as

$$\text{forward error} \approx \text{condition number} \times \text{backward error}.$$

For a problem with condition number $10^8$, double-precision rounding at the $10^{-16}$ scale can become a $10^{-8}$ scale forward error even with a stable algorithm. This may be excellent or unacceptable depending on the physical question.

Eigenvalue Sensitivity

Hermitian eigenvalue problems are much better behaved than general non-Hermitian eigenvalue problems. If

$$\widetilde H=H+E$$

with $H$ and $E$ Hermitian, the eigenvalues shift by at most the perturbation size in spectral norm:

$$\lvert \widetilde E_n-E_n\rvert \le \lVert E\rVert_2$$

after matching the ordered eigenvalues.

Eigenvectors are more sensitive. If an eigenvalue is separated from the rest of the spectrum by a gap $g$, then the angle between the exact and perturbed eigendirections is controlled roughly by

$$\sin\theta \lesssim \frac{\lVert E\rVert_2}{g}.$$

When $g$ is small, individual eigenvectors can rotate strongly under tiny perturbations. The physically meaningful object may be the whole nearly degenerate subspace, not a particular basis vector inside it.

This distinction is central in Matrix Diagonalization: eigenvalue residuals, orthogonality, symmetry labels, and subspace comparisons are all part of validation.

Nonorthogonal Bases and Overlap Matrices

Many variational and computational methods use a nonorthogonal basis $\{\phi_i\}$. The overlap matrix is

$$S_{ij} = \langle \phi_i,\phi_j\rangle.$$

The generalized eigenvalue problem has the form

$$Hc=ESc.$$

If the basis functions are nearly linearly dependent, then $S$ has very small singular values and is ill conditioned. The generalized eigenvalue problem may amplify roundoff, produce spurious states, or make coefficients $c_i$ enormous while the represented wavefunction remains moderate.

The fix is not merely to ask for more digits in the printed output. One may need to remove near-dependent basis vectors, orthogonalize carefully, use SVD thresholds, rescale the basis, or reformulate the problem.

Small Splittings and Physical Claims

Suppose a calculation reports

$$\Delta E=E_2-E_1.$$

If $\Delta E$ is much smaller than the absolute uncertainty in $E_1$ and $E_2$, then the splitting has not been resolved. This is common in tunneling, weak symmetry breaking, fine structure, avoided crossings, and finite-size effects.

Useful checks include:

• compare $\Delta E$ with eigenpair residuals and estimated discretization error;
• vary basis size, grid size, and precision;
• check whether a symmetry predicts exact degeneracy;
• compare with an asymptotic or exactly solvable limit;
• track the subspace rather than individual eigenvectors near a crossing.

A small number is a physical result only after it survives these checks.

Stability in Time Evolution

For exact closed-system quantum mechanics, time evolution is unitary:

$$\lVert \psi(t)\rVert = \lVert \psi(0)\rVert.$$

A numerical time-stepping method should be judged partly by whether it respects or accurately approximates this structure. An algorithm can be locally accurate for short times but drift in norm or energy over long times.

Stability questions include:

• Does the method preserve norm for a time-independent Hermitian Hamiltonian?
• Does reducing the time step improve the answer at the expected rate?
• Does the error grow linearly, diffusively, or exponentially with time?
• Is the evolution problem stiff because the Hamiltonian has widely separated scales?

For long-time dynamics, qualitative stability may matter as much as a small one-step error.

Choosing Tolerances

A tolerance should be tied to a scale. The statement

$$\lVert r\rVert \lt 10^{-10}$$

is incomplete unless the scale of $A$, $x$, and $b$ is known. A relative residual such as

$$\frac{\lVert b-A\tilde x\rVert} {\lVert A\rVert\,\lVert \tilde x\rVert+\lVert b\rVert}$$

is often more meaningful because it compares the residual with the size of the terms being balanced.

Tolerances should also respect the problem’s conditioning. Asking for $10^{-12}$ relative accuracy from a problem with condition number $10^8$ in double precision may be unrealistic without extended precision or a reformulation.

Practical Checklist

For a numerical quantum calculation, ask:

• What is the mathematical problem map from data to answer?
• Which input perturbations are physically or numerically plausible?
• What residual was checked, and in what norm?
• Is the condition number or spectral gap known or estimated?
• Does the algorithm have a backward-stability guarantee for this problem class?
• Are small physical conclusions larger than estimated numerical uncertainty?
• Is the result stable under basis, grid, time-step, and precision changes?

When the answer changes under harmless numerical choices, treat that as a diagnostic signal rather than a nuisance.

Common Mistakes

• Calling an algorithm unstable when the problem is simply ill conditioned.
• Calling a problem well solved because the residual is small, without checking conditioning.
• Trusting individual eigenvectors inside a nearly degenerate subspace.
• Reporting tiny energy splittings without an absolute error estimate.
• Using a nonorthogonal basis without checking the overlap matrix.
• Choosing tolerances with no scale or units.
• Assuming double precision is enough because the final answer prints many digits.
• Ignoring symmetry constraints that could distinguish numerical noise from real effects.

Cross-Links

• Floating-Point Arithmetic
• Error Estimates
• Discretization
• Matrix Diagonalization
• Time-Stepping Methods
• Norms and Metrics
• Matrices as Linear Maps
• Singular Value Decomposition
• Hermitian Operators
• Small Parameters and Error Estimates

References

• N. J. Higham, Accuracy and Stability of Numerical Algorithms, 2nd ed., SIAM, 2002.
• G. H. Golub and C. F. Van Loan, Matrix Computations, 4th ed., Johns Hopkins University Press, 2013.
• L. N. Trefethen and D. Bau, Numerical Linear Algebra, SIAM, 1997.
• J. H. Wilkinson, The Algebraic Eigenvalue Problem, Oxford University Press, 1965.
• J. M. Thijssen, Computational Physics, 2nd ed., Cambridge University Press, 2007.

Exercises

1. Let $f(x)=1/(1-x)$ for $x\ne1$. Compute the relative condition number.

Solution

We have

$$f'(x) = \frac{1}{(1-x)^2}.$$

Thus

$$\kappa_f(x) = \left\lvert \frac{x f'(x)}{f(x)} \right\rvert = \left\lvert \frac{x}{1-x} \right\rvert.$$

The condition number becomes large near $x=1$, where the function is very sensitive to input perturbations.

2. A backward stable algorithm is applied to a problem with condition number $10^8$ in double precision. Estimate the possible scale of forward relative error from rounding alone.

Solution

Double precision has unit roundoff around $10^{-16}$. A rough estimate is

$$\text{forward relative error} \sim 10^8\times10^{-16} = 10^{-8}.$$

The exact constant depends on the problem and algorithm, but the estimate shows that backward stability does not guarantee sixteen accurate digits when the problem is ill conditioned.

3. Why can eigenvectors be unreliable near degeneracy even when eigenvalues are accurate?

Solution

Hermitian eigenvalues have good absolute perturbation bounds, but eigenvector directions depend on spectral gaps. If the gap $g$ to nearby eigenvalues is small, a perturbation of size $\lVert E\rVert_2$ can rotate the eigenvector by an angle controlled roughly by $\lVert E\rVert_2/g$. Near degeneracy, the individual basis vectors inside the eigenspace are not stable; the subspace is the better object to compare.

4. Let $A=\operatorname{diag}(1,\epsilon)$ with $0\lt\epsilon\ll1$. If a computed solution has residual $r=(0,\rho)^T$, what is the corresponding error $e=A^{-1}r$?

Solution

Since

$$A^{-1} = \operatorname{diag}(1,\epsilon^{-1}),$$

the error is

$$e=A^{-1}r = \begin{pmatrix} 0\\ \rho/\epsilon \end{pmatrix}.$$

A small residual component $\rho$ can produce a large error if $\epsilon$ is tiny. This is the residual-versus-conditioning lesson in its simplest form.

5. In a nonorthogonal variational basis, why is a small singular value of the overlap matrix $S$ a warning sign?

Solution

A small singular value means that some nonzero coefficient vector produces a very small represented vector. The basis is nearly linearly dependent. In the generalized eigenvalue problem $Hc=ESc$, this near-null direction can amplify roundoff and produce unstable coefficients or spurious eigenvalues. One should consider orthogonalization, SVD truncation, rescaling, or basis repair.