Imaginary-Time Projection Notebook
This notebook guide turns imaginary-time evolution into a controlled ground-state algorithm. A finite-difference harmonic oscillator supplies a sparse Hamiltonian, exact continuum energies, and a trusted discrete eigensolver benchmark. A deliberately contaminated trial state makes the decay of excited components measurable rather than merely visible.
The operator-semigroup and finite-temperature meaning of
belongs to Imaginary Time, while Euclidean and Imaginary-Time Path Integrals owns its kernel and path-integral representation. This page owns the deterministic numerical workflow: apply the damping operator, normalize after every step, diagnose convergence, compare with exact diagonalization, and expose the overlap and symmetry conditions under which projection fails.
Purpose
Section titled “Purpose”The notebook should demonstrate that:
- normalized imaginary-time evolution selects the lowest-energy component present in the trial state;
- the spectral gap and initial overlaps determine the convergence rate;
- per-step normalization prevents underflow but does not create missing ground-state overlap;
- an exponential-action method and an implicit approximation have different time-step errors;
- exact diagonalization validates the projection on the same discrete Hamiltonian;
- grid error, finite-box error, projection error, and time-step error must be reported separately;
- energy convergence alone is weaker than fidelity, residual, variance, and symmetry checks.
The algorithm finds the ground state of the discretized problem. Agreement with the continuum ground state is a second question requiring spatial convergence.
Imaginary-Time Filtering
Section titled “Imaginary-Time Filtering”Let
and expand a trial state as
Unnormalized imaginary-time evolution gives
If and the ground state is nondegenerate, the normalized state
approaches . Factoring out the ground energy makes the rate explicit:
The overall factor disappears under normalization. The relative excited amplitudes do not:
Power-Method Interpretation
Section titled “Power-Method Interpretation”For a fixed step , define
Its eigenvalues are
Repeated application of followed by normalization is a power method. The ground-state eigenvalue has the largest magnitude because is the smallest energy. The ratio
controls suppression per step. The reference energy changes the common scale of all but not their ratios.
This interpretation explains both success and failure. Projection cannot create a component in an eigenvector whose coefficient is exactly zero, and a small spectral gap makes convergence slow.
Discrete Harmonic-Oscillator Benchmark
Section titled “Discrete Harmonic-Oscillator Benchmark”Use the dimensionless oscillator
with
The continuum energies are
Place interior points on :
Impose homogeneous Dirichlet conditions at . The centered second difference gives
Thus is real symmetric and tridiagonal, with
and
The boundary condition and finite box are part of the discrete model. They must be identical in the projector and eigensolver routes.
Discrete Normalization Convention
Section titled “Discrete Normalization Convention”If an array stores samples , the continuum norm is approximated by
A standard matrix eigensolver instead normalizes vectors with
These conventions are reconciled by storing weighted samples
Then
Because is a constant factor on a uniform grid, the same matrix acts on . Choose either convention and use it everywhere. Mixing them produces incorrect overlaps and wavefunction amplitudes even when eigenvalues look correct.
Controlled Trial State
Section titled “Controlled Trial State”Let , , and be the first three normalized continuum oscillator wavefunctions. Use
with real
Sample this state on the grid and normalize it with the chosen discrete convention. In the continuum benchmark, its initial ground-state fidelity is
The trial state contains substantial odd and even excited contamination, so both parity sectors are tested.
Analytic Contamination Law
Section titled “Analytic Contamination Law”After factoring out the ground-state damping, the three amplitudes are proportional to
The exact continuum ground-state fidelity is
The excited-state contamination is
The energy error is
At late time, the term dominates:
This gives a predicted logarithmic slope of . The discrete eigensystem supplies a slightly different gap and overlap, allowing the notebook to separate continuum discretization error from projector error.
Normalized Projection Step
Section titled “Normalized Projection Step”For each step, compute
Record the pre-normalization scale
then normalize:
Normalization prevents exponential underflow and keeps diagnostics on a stable scale. It does not change the relative eigenstate damping.
For an exact exponential action, changing a constant changes only the discarded normalization factor. For an approximate step, the shift can change truncation error, so it must be recorded.
Near convergence to an eigenstate, the recorded scale estimates the energy:
Compare this estimate with the Rayleigh quotient and exact diagonalization. Agreement among them is a useful implementation check.
Exponential-Action Method
Section titled “Exponential-Action Method”The preferred projection route applies the matrix exponential to a vector without forming the dense matrix exponential:
For a sparse tridiagonal , an exponential-action algorithm uses matrix-vector products and adaptive polynomial information internally. It preserves the exact imaginary-time map up to the action algorithm’s numerical tolerance, so the user-selected normalization interval is not a first-order time discretization.
Validate this route on a smaller grid by comparing it with the spectral expression
constructed from a full dense eigendecomposition.
Implicit and Explicit Comparisons
Section titled “Implicit and Explicit Comparisons”A backward-Euler step is
For an energy eigencomponent, its amplification factor is
With a safe shift, high-energy components are strongly damped. The method is first-order accurate in , so repeat it with smaller steps and compare against exponential action.
Forward Euler gives
Its amplification factor is
High finite-difference energies scale as , making this method severely step-size restricted. Include it only as a controlled failure demonstration. Per-step renormalization can hide its explosive high-mode amplification while the state converges to the wrong vector.
Exact-Diagonalization Benchmark
Section titled “Exact-Diagonalization Benchmark”Compute the lowest several eigenpairs of the same :
Validate:
and
Also compare low energies with
Projection fidelity must first be measured against , the exact ground vector of the discrete matrix. Only after projection has converged should and be compared with their continuum targets under box and grid refinement.
For a real symmetric matrix, use a Hermitian sparse eigensolver and request the smallest algebraic eigenvalues. Check residuals explicitly; a solver status alone is not a validation record.
Energy, Residual, and Variance
Section titled “Energy, Residual, and Variance”For a normalized state, define the Rayleigh quotient
The energy variance is
Thus the variance is also the squared eigenpair residual. It vanishes for any exact eigenstate, not only the ground state. Combine it with energy and symmetry information.
For continuously normalized imaginary-time flow,
one finds
The exact normalized energy is monotone. A numerical increase beyond tolerance indicates a step, solver, normalization, or inner-product problem.
Fidelity and Phase Alignment
Section titled “Fidelity and Phase Alignment”With the discrete ground vector available, compute
For wavefunction plots, align the arbitrary phase:
Then compare with . For a real calculation this reduces to a sign choice. Comparing signed arrays without alignment can report a large error for physically identical eigenvectors.
Symmetry and Overlap Traps
Section titled “Symmetry and Overlap Traps”The oscillator ground state is even. An exactly odd trial state obeys
Because preserves parity on a symmetric grid, imaginary-time evolution remains in the odd sector and approaches the lowest odd state, , rather than the global ground state.
Run three trials:
- the mixed state with and ;
- an even state with , whose leading contamination is ;
- the odd state , which has zero ground overlap.
The even trial should converge faster because its first allowed gap is . The odd trial should have vanishing ground fidelity while its energy and variance converge cleanly to the first excited state.
Floating-point or solver errors can leak a tiny even component into an intended odd calculation. At very long imaginary time, that tiny lower-energy component may eventually dominate. If projection is meant to stay in a symmetry sector, enforce or monitor the symmetry explicitly.
Degeneracy and Excited States
Section titled “Degeneracy and Excited States”If the ground energy is degenerate, normalized projection approaches the normalized projection of the trial state onto the ground eigenspace. It does not select a unique basis vector inside that subspace.
To target an excited state, project out already known lower states after every step:
then renormalize. This is an extension, not part of the baseline ground-state algorithm. Orthogonalization errors and near-degeneracy require additional care.
Baseline Parameters
Section titled “Baseline Parameters”Use
Take for the positive oscillator Hamiltonian. Record diagnostics at every step and store a smaller set of states for plots.
For the continuum three-level benchmark,
The finite-difference result should approach the corresponding discrete prediction based on measured overlaps and gaps. It should not be forced to equal the continuum number before spatial refinement.
Notebook Workflow
Section titled “Notebook Workflow”Organize the notebook into these stages:
- define physical and grid parameters;
- build the sparse tridiagonal with explicit boundary conditions;
- check symmetry and Hermiticity;
- compute several lowest eigenpairs and validate residuals;
- construct and discretely normalize the mixed trial state;
- measure its eigenbasis overlaps;
- apply exponential-action projection and normalize every step;
- record pre-normalization scales, energy, variance, residual, parity, and fidelity;
- compare contamination with analytic continuum and discrete spectral predictions;
- repeat with backward Euler and step refinement;
- run the even and odd symmetry trials;
- refine box, grid, and projection parameters separately;
- run assertions before producing plots.
Record package versions, sparse formats, eigensolver settings, exponential-action options, random seeds for random trial states, and all tolerances.
Validation Checks
Section titled “Validation Checks”The minimum validation suite is:
| Check | Target |
|---|---|
| Hamiltonian | real symmetric with the intended Dirichlet boundary condition |
| Eigensolver | low eigenpair residuals and orthonormality pass |
| Continuum spectrum | low approach under spatial refinement |
| Initial overlap | measured agrees with the sampled trial state |
| Per-step norm | normalized state has unit norm in the declared convention |
| Shift invariance | exact-action normalized states agree for different constant |
| Energy monotonicity | does not increase beyond tolerance |
| Variance | approaches zero |
| Fidelity | approaches one for the mixed trial |
| Decay rate | late-time contamination slope matches the lowest populated gap |
| Norm-ratio energy | agrees with Rayleigh quotient near convergence |
| Backward Euler | converges to exponential action as decreases |
| Odd trial | approaches the first odd state, not the ground state |
| Spatial convergence | box enlargement and grid refinement are tested separately |
Do not stop only because successive energies differ by a small amount. A biased integrator can plateau, and a symmetry-restricted excited state can have a stable energy.
Convergence Study
Section titled “Convergence Study”Separate four limits:
- Projection time. Increase at fixed method and discrete Hamiltonian.
- Projection step. Refine for backward or forward Euler. Exponential action should be insensitive to normalization interval up to algorithmic tolerance.
- Grid spacing. Increase at fixed and verify the expected second-order finite-difference regime.
- Box size. Increase at approximately fixed and monitor boundary amplitude.
The highest discrete energy grows like . Forward-Euler stability therefore becomes more restrictive as the spatial grid is refined. A time step that worked on a coarse grid is not automatically valid on a fine one.
Plot contamination, energy error, and variance on logarithmic scales. Report where each quantity reaches a discretization or floating-point floor.
Expected Results
Section titled “Expected Results”For the mixed baseline state:
- the initial discrete ground fidelity is close to ;
- energy decreases monotonically toward ;
- excited contamination follows the measured spectral-gap prediction;
- the late-time log slope approaches ;
- fidelity reaches approximately by before other errors are considered;
- energy variance and eigenpair residual approach zero;
- the projected state agrees with the discrete ground eigenvector after phase alignment;
- the even trial converges with the larger even-sector gap;
- the odd trial converges to the first excited state and never acquires ground overlap in the symmetry-preserving calculation;
- backward Euler approaches exponential action under step refinement;
- continuum agreement improves only under box and grid refinement.
Common Numerical Failures
Section titled “Common Numerical Failures”- Assuming normalization creates ground overlap. It rescales existing components only.
- Comparing the projector directly with the continuum state before validating the discrete ground state.
- Mixing Euclidean and ordinary vector normalization on the coordinate grid.
- Renormalizing without recording the pre-normalization scale. This discards an energy diagnostic.
- Using forward Euler with a step chosen independently of the highest grid energy.
- Calling backward Euler exact because it is stable. It has first-order step error.
- Forming a dense matrix exponential for a large sparse Hamiltonian. Apply the exponential to the vector.
- Using energy change as the only stopping criterion. Check variance, residual, fidelity, and symmetry.
- Ignoring parity. An odd trial state correctly projects to the lowest odd state.
- Breaking a symmetry accidentally and interpreting late-time sector leakage as physical.
- Using a reference-energy shift without recording it. Approximate propagators can depend on the shift.
- Refining while keeping an unstable explicit imaginary-time step.
- Treating a degenerate ground space as a unique eigenvector.
Extensions
Section titled “Extensions”- Project the quartic oscillator ground state and compare with sparse diagonalization.
- Target the first few excited states by repeated orthogonalization.
- Compare exponential action with a symmetric split-operator imaginary-time step.
- Use inverse iteration or shift-invert eigensolvers and compare convergence mechanisms.
- Project within a fixed parity sector using an explicit symmetry projector.
- Continue from pure-state projection to thermal traces and Euclidean correlation functions.
Cross-Links
Section titled “Cross-Links”- Euclidean and Imaginary-Time Path Integrals
- From Euclidean Time to Euclidean QFT
- Quantum Harmonic Oscillator
- Finite Difference Methods
- Matrix Diagonalization
- Sparse Eigensolvers
- Matrix Exponentials Numerically
- Convergence Tests
- Benchmark Problems
- Validation Tests
References
Section titled “References”- D. J. Tannor, Introduction to Quantum Mechanics: A Time-Dependent Perspective, University Science Books, 2007.
- J. M. Thijssen, Computational Physics, 2nd ed., Cambridge University Press, 2007.
- Y. Saad, Numerical Methods for Large Eigenvalue Problems, 2nd ed., Society for Industrial and Applied Mathematics, 2011.
- A. H. Al-Mohy and N. J. Higham, “Computing the Action of the Matrix Exponential, with an Application to Exponential Integrators,” SIAM Journal on Scientific Computing 33, 488–511 (2011), doi:10.1137/100788860.
- SciPy Developers, expm_multiply documentation, consulted for sparse exponential action.
- SciPy Developers, eigsh documentation, consulted for Hermitian sparse eigenvalue settings and residual validation.
Exercises
Section titled “Exercises”1. Derive the contamination law
Section titled “1. Derive the contamination law”For the three-level trial state, derive and the energy error.
Solution
After removing the common ground factor, the unnormalized amplitudes are
The squared norm is proportional to
The normalized ground probability is therefore
Using and ,
2. Prove monotone energy descent
Section titled “2. Prove monotone energy descent”For normalized imaginary-time flow, derive .
Solution
The normalized flow and its adjoint are
and
For time-independent ,
3. Derive the norm-ratio energy estimator
Section titled “3. Derive the norm-ratio energy estimator”Assume the normalized state has converged to an exact eigenstate. Derive the estimate from .
Solution
For ,
Its norm is
Taking the logarithm gives
Before convergence, the norm combines several spectral components and the expression is only an effective-energy diagnostic.
4. Explain the odd-state trap
Section titled “4. Explain the odd-state trap”Why does an odd trial state converge to the first excited oscillator state rather than the ground state?
Solution
The ground state is even, so its overlap with an odd trial state vanishes. The symmetric oscillator Hamiltonian commutes with parity, and imaginary-time evolution preserves the odd subspace. Within that subspace the lowest energy belongs to the state. Projection therefore suppresses higher odd states relative to but cannot generate an even ground-state component.
5. Derive the forward-Euler restriction
Section titled “5. Derive the forward-Euler restriction”Let the shifted discrete energies satisfy . Find a condition ensuring that every forward-Euler amplification factor has magnitude at most one.
Solution
For
the condition requires
The upper bound is automatic for nonnegative shifted energies. The lower bound at the largest energy gives
Because grows like for a finite-difference Hamiltonian, the allowed explicit step shrinks like .
6. Separate projection and discretization error
Section titled “6. Separate projection and discretization error”Design a test showing that a projected state has converged to the discrete ground state even if the discrete ground energy has not converged to the continuum value.
Solution
At fixed and , compute and by exact diagonalization. Run projection until the fidelity with is near one and the residual with is below tolerance. This establishes projection convergence for that matrix. Then repeat the entire matrix construction at larger and smaller . Changes in and the phase-aligned wavefunction now measure spatial discretization and box error, not incomplete imaginary-time projection.