Gram–Schmidt Orthogonalisation

Q: Should the orthogonal vectors be recomputed after every basis update?

No — that would dominate the cost. LLL updates the affected mu coefficients and norms incrementally after each swap or size reduction, touching only the entries that can have changed.

Q: What causes the loss of orthogonality?

Catastrophic cancellation. When a vector is nearly in the span of its predecessors, the subtraction removes almost all of its magnitude and the remainder consists largely of rounding error, so the computed orthogonal vector points in a direction that is essentially arbitrary.

Q: Is Cholesky an alternative?

Yes. Factoring the Gram matrix by Cholesky yields the same information without forming the orthogonal vectors explicitly, and is often preferred when only the mu coefficients and norms are needed — which is the case inside LLL.

Executive summary

The quantities every reduction algorithm is defined in terms of

Gram–Schmidt turns a basis into an orthogonal family by subtracting from each vector its projections onto the previous ones. The projection coefficients μ_ij and the squared lengths of the orthogonal vectors are precisely the quantities in which the LLL reduction conditions are stated. The orthogonal vectors themselves are almost never lattice vectors — they are a measuring instrument, not an output.

Learning objectives

State the Gram–Schmidt recurrence and the meaning of its coefficients.
Explain why the orthogonal family is not a lattice basis.
Identify the numerical failure mode of the classical procedure.
Choose between floating-point and exact integral variants.
Connect the μ coefficients to the LLL reduction conditions.

Section 01The procedure

b*_i = b_i − ∑_j<i μ_ij b*_j, μ_ij = ⟨b_i, b*_j⟩ / ‖b*_j‖²

The family {b*_i} is orthogonal and spans the same real subspace, and the determinant is preserved: det L = ∏ ‖b*_i‖. In matrix terms this is the QR decomposition, with the μ forming the unit lower-triangular factor.

The orthogonal vectors are not lattice vectors

The μ coefficients are rational, so b*_i generally lies outside the lattice. Reduction algorithms use the b*_i only to measure the basis; every vector they actually manipulate is an integer combination of the original basis.

Section 02Numerical behaviour

The classical procedure, implemented exactly as written, loses orthogonality catastrophically when the basis is ill-conditioned — which is precisely the situation in which reduction is needed. The subtractions cancel almost entirely, and the surviving digits are rounding noise.

Variants and their behaviour
Variant	Stability	Cost	When to use
Classical Gram–Schmidt	Poor	2n³/3	Never, in floating point
Modified Gram–Schmidt	Much better	Same	Default floating-point choice — project one vector at a time
Householder QR	Excellent	Higher	When orthogonality must be near machine precision
Exact rational / integral	Exact by construction	Highest	Verification, and small dimensions where exactness is affordable

Modified is not a refinement, it is the version to use

Modified Gram–Schmidt subtracts each projection immediately rather than accumulating them, so later projections are computed against already-corrected vectors. The operation count is identical; only the rounding behaviour differs. There is no reason to implement the classical form.

Section 03Exact variants and the LLL connection

Working with exact rationals is safe but slow, since the μ have rapidly growing denominators. The integral variant clears these denominators by scaling with the leading principal minors d_i = ∏_j≤i ‖b*_j‖², which are integers for an integer lattice. All arithmetic then stays in ℤ.

The reduction conditions of LLL are stated entirely in these terms:

|μ_ij| ≤ 1/2 for j < i (size reduction)

‖b*_i‖² ≥ (δ − μ_i,i−1²) ‖b*_i−1‖² (Lovász condition)

The standard engineering compromise

Production LLL implementations run Gram–Schmidt in floating point for speed, and re-verify the conditions in exact arithmetic whenever a test comes out close to the threshold. This gives near floating-point speed with exact-arithmetic guarantees — and is the reason floating point is acceptable inside an otherwise exact subject.

ReferenceFrequently asked questions

Should the orthogonal vectors be recomputed after every basis update?

No — that would dominate the cost. LLL updates the affected mu coefficients and norms incrementally after each swap or size reduction, touching only the entries that can have changed.

What causes the loss of orthogonality?

Catastrophic cancellation. When a vector is nearly in the span of its predecessors, the subtraction removes almost all of its magnitude and the remainder consists largely of rounding error, so the computed orthogonal vector points in a direction that is essentially arbitrary.

Is Cholesky an alternative?

Yes. Factoring the Gram matrix by Cholesky yields the same information without forming the orthogonal vectors explicitly, and is often preferred when only the mu coefficients and norms are needed — which is the case inside LLL.

NavigateContinue in this stream

Curated next steps from this page. The site also surfaces algorithmically related reading below.

ProvenanceSources and further reading

This page is an original KEVOS explanatory article. It presents the underlying mathematics — definitions, algorithms, complexity results and selection criteria — in KEVOS editorial voice. No text is reproduced from any copyrighted source. Where numerical tables are relevant, KEVOS links to live authoritative databases rather than republishing static values.

Handbook application: from concept to controlled practice

Purpose. This expanded section turns the original page into a practical handbook. It preserves the supplied material and adds a repeatable way to apply, check and review Gram–Schmidt Orthogonalisation. It does not replace a contract, legislation, a controlled standard, competent engineering judgement or specialist advice.

The operating aim is to turn a compact mathematical statement into a usable chain of definitions, claims, examples and checks. Read the original explanation first, then use the workflow and checks below to convert knowledge into evidence.

Treat Gram–Schmidt Orthogonalisation as a network of definitions and implications, not as a list of formulas. The working vocabulary on this page—numerical, section, orthogonalisation, gram-schmidt, procedure—should be made explicit before any proof or computation begins. Record the ambient set or structure, the permitted operations and the equality or equivalence relation in use. A compact theorem often changes meaning when the base field, finiteness condition, commutativity assumption or direction of an action changes.

For a proof, write the hypotheses as a checklist and mark the line at which each one is used. For a computation, state the representation of the input, the arithmetic model, the termination condition and the output invariant. For a classification problem, distinguish existence from uniqueness and distinguish an object from its representation. These separations prevent a correct local calculation from being mistaken for the general result.

A useful worked example should be small enough to inspect completely but rich enough to exercise the main mechanism. Compute the result in two ways where practical: symbolically and by substitution, structurally and numerically, or directly and through a normal form. Then include one near-miss example in which a hypothesis fails. The contrast explains why the theorem is shaped as it is and gives the reader a diagnostic pattern for later problems.

Verification is part of the mathematics. Check domains and codomains, substitute proposed solutions, test identity and zero cases, compare dimensions or cardinalities, and confirm that maps respect the required operations. In numerical work, report precision, conditioning and a residual rather than digits alone. In algorithmic work, separate mathematical correctness from implementation complexity and resource limits.

Step-by-step operating method

Fix the setting. State the objects, ambient structure, notation and assumptions before manipulating symbols.
Separate claims. Distinguish definitions, hypotheses, conclusions, equivalent conditions and consequences.
Choose a method. Select proof, construction, calculation or algorithm according to the question actually asked.
Work a small case. Use the smallest non-trivial example to expose the mechanism and test edge behaviour.
Verify independently. Substitute back, check invariants, test boundary cases or use an alternative derivation.

Worked-example protocol

Illustrative method—not a source theorem. Start with a small admissible input and list the definitions it must satisfy. Carry out each transformation on a separate line, citing the property that permits it. Preserve exact values until approximation is necessary. At the end, verify the output against the original definition and one invariant such as dimension, degree, determinant, order, norm or residual. Then alter one hypothesis and observe which step ceases to be valid. This protocol creates a reusable example without inventing a theorem-specific numerical answer.

Stage	Record	Quality check
Input	Objects, domain, notation, assumptions	Every symbol is defined
Method	Permitted operation or cited result at each step	All hypotheses hold
Output	Exact result and representation	Correct type, domain and form
Verification	Substitution, invariant or alternative derivation	Independent agreement
Boundary test	Zero, identity, degenerate or failed hypothesis	Scope is understood

Common failure modes and recovery actions

1. Watch for

Using a theorem without checking every hypothesis.

Recovery: Return to the governing definition or requirement and restate the decision in one sentence.

2. Watch for

Treating a suggestive example as a proof of the general case.

Recovery: Separate evidence from assumption, assign an owner and set a date for validation.

3. Watch for

Changing notation or conventions part-way through an argument.

Recovery: Run a small counterexample, boundary test, pilot or independent check before proceeding.

4. Watch for

Hiding a division-by-zero, convergence, finiteness or commutativity assumption.

Recovery: Record the consequence, decision and rationale, then update the controlled baseline.

5. Watch for

Reporting a computed result without a residual, substitution or structural check.

Recovery: Escalate when the issue affects safety, compliance, acceptance, material value or an agreed tolerance.

Review checklist

Can every symbol be traced to a definition or prior result?
Which hypothesis does each major step use?
Does the method cover zero, identity, degenerate and boundary cases?
Can the conclusion be checked by a second representation or calculation?
Are mandatory requirements distinguished from recommendations and illustrative values?
Are sources, assumptions, units, dates and versions recorded closely enough to reproduce the decision?
Have safety, legal, ethical, stakeholder and operational consequences been considered at the appropriate level?
Is there a named owner and a trigger for review, escalation, change or retirement?

Questions for deeper application

What is the most important distinction a practitioner must preserve when applying Gram–Schmidt Orthogonalisation?