KEVOS
ArticlesServicesCase studiesAboutContact
ArticlesServicesCase studiesAboutContact
← ArticlesExpectation and VarianceEngineering · Engineering MathematicsLesson 575/887← PrevNext →
GuidePublished 7 Aug 2026Updated 13 Aug 20267 min readBy Kevin Jogin
On this page

Ask about this page

KEVOS AIExpectation and Variance

KEVOS knowledge first · trusted web sources when needed

Engineering  /  Mathematics  — Discrete Probability

Expectation and Variance

Expectation, variance, their algebraic properties and their use in analysing randomised algorithms.

Page KV-MATH-0344Reading time 3 minReviewed 2026-08-07Author Kevin Jogin

Executive summary

Expectation is the weighted average of a random variable and is linear without qualification. Variance measures spread and is additive only under independence.

The asymmetry between these two facts governs how randomised algorithm analyses are structured.

Learning objectives

  1. Compute expectation and variance from a distribution.
  2. Apply linearity of expectation to sums of indicators.
  3. State the conditions for variance additivity.

01Expectation

Definition

Expectation

E[X] = Σ_s s · P(X = s), equivalently Σ_{ω} X(ω) P(ω).

Theorem

Linearity

E[aX + bY] = aE[X] + bE[Y] for all random variables X, Y on the same space and all constants a, b, with no independence assumption.

A frequently useful alternative form for a non-negative integer variable is the tail sum.

E[X] = Σ_{k ≥ 1} P(X ≥ k)   for X taking values in {0, 1, 2, ...}

This form is often easier to evaluate, because tail probabilities are what an algorithm analysis naturally produces — the probability that a loop runs at least k times.

02Variance

Definition

Variance and standard deviation

Var[X] = E[(X − E[X])²] = E[X²] − E[X]².

The standard deviation is √Var[X], in the same units as X.

Variance rules
PropertyStatementCondition
ScalingVar[aX] = a²Var[X]Always
TranslationVar[X + c] = Var[X]Always
AdditivityVar[X+Y] = Var[X]+Var[Y]X, Y uncorrelated
Sum of n termsVar[ΣXᵢ] = ΣVar[Xᵢ]Pairwise independent
Caution
Variance is not linear. Var[X + X] = 4Var[X], not 2Var[X]. The additivity rule requires the variables to be uncorrelated, and applying it without checking is a common error that produces a variance too small by a factor that can be arbitrarily large.

03Standard distributions

  1. Bernoulli(p)E = p, Var = p(1−p)A single indicator; the atom of most analyses
  2. Binomial(n,p)E = np, Var = np(1−p)Successes in n independent trials
  3. Geometric(p)E = 1/p, Var = (1−p)/p²Trials until first success; unbounded
  4. Uniform on 1..nE = (n+1)/2, Var = (n²−1)/12Random index selection

The geometric distribution is the one that appears whenever an algorithm repeats until success — random prime generation, rejection sampling, finding a quadratic non-residue. Its expectation 1/p is what converts a per-trial success probability into an expected running time.

04Frequently asked questions

Why use E[X²] − E[X]² rather than the definition?

Because it needs only two moments and both are usually easier to compute directly, especially for sums of indicators where the square expands into pairwise products that pairwise independence handles.

Does a finite expectation guarantee finite variance?

No. Distributions with heavy tails can have finite mean and infinite variance. For the bounded variables arising from terminating algorithms the question does not arise, but it matters for unbounded loops.

Is expected running time the right measure?

It is the standard one but it hides tail behaviour. An algorithm with good expectation may still be slow with uncomfortable probability, which is why concentration inequalities are applied on top rather than instead.

Related pages

  • Markov's and Chebyshev's Inequalities
  • Random Variables

Sources and method

Structural reference: Victor Shoup, A Computational Introduction to Number Theory and Algebra, Version 1, Cambridge University Press, 2005 — book pages 111-117.

This page carries the durable method layer only: definitions, constructions, algorithms, complexity results and selection criteria, authored originally for KEVOS. No text is transcribed or paraphrased from the source, and no numeric tables or benchmark data are reproduced — these are routed to live authoritative sources instead.

Author: Kevin Jogin. Last reviewed 2026-08-07.

Handbook application: from concept to controlled practice

Purpose. This expanded section turns the original page into a practical handbook. It preserves the supplied material and adds a repeatable way to apply, check and review Expectation and Variance. It does not replace a contract, legislation, a controlled standard, competent engineering judgement or specialist advice.

The operating aim is to turn a compact mathematical statement into a usable chain of definitions, claims, examples and checks. Read the original explanation first, then use the workflow and checks below to convert knowledge into evidence.

Treat Expectation and Variance as a network of definitions and implications, not as a list of formulas. The working vocabulary on this page—expectation, variance, algebraic, properties, analysing—should be made explicit before any proof or computation begins. Record the ambient set or structure, the permitted operations and the equality or equivalence relation in use. A compact theorem often changes meaning when the base field, finiteness condition, commutativity assumption or direction of an action changes.

For a proof, write the hypotheses as a checklist and mark the line at which each one is used. For a computation, state the representation of the input, the arithmetic model, the termination condition and the output invariant. For a classification problem, distinguish existence from uniqueness and distinguish an object from its representation. These separations prevent a correct local calculation from being mistaken for the general result.

A useful worked example should be small enough to inspect completely but rich enough to exercise the main mechanism. Compute the result in two ways where practical: symbolically and by substitution, structurally and numerically, or directly and through a normal form. Then include one near-miss example in which a hypothesis fails. The contrast explains why the theorem is shaped as it is and gives the reader a diagnostic pattern for later problems.

Verification is part of the mathematics. Check domains and codomains, substitute proposed solutions, test identity and zero cases, compare dimensions or cardinalities, and confirm that maps respect the required operations. In numerical work, report precision, conditioning and a residual rather than digits alone. In algorithmic work, separate mathematical correctness from implementation complexity and resource limits.

Step-by-step operating method

  1. Fix the setting. State the objects, ambient structure, notation and assumptions before manipulating symbols.
  2. Separate claims. Distinguish definitions, hypotheses, conclusions, equivalent conditions and consequences.
  3. Choose a method. Select proof, construction, calculation or algorithm according to the question actually asked.
  4. Work a small case. Use the smallest non-trivial example to expose the mechanism and test edge behaviour.
  5. Verify independently. Substitute back, check invariants, test boundary cases or use an alternative derivation.

Worked-example protocol

Illustrative method—not a source theorem. Start with a small admissible input and list the definitions it must satisfy. Carry out each transformation on a separate line, citing the property that permits it. Preserve exact values until approximation is necessary. At the end, verify the output against the original definition and one invariant such as dimension, degree, determinant, order, norm or residual. Then alter one hypothesis and observe which step ceases to be valid. This protocol creates a reusable example without inventing a theorem-specific numerical answer.

StageRecordQuality check
InputObjects, domain, notation, assumptionsEvery symbol is defined
MethodPermitted operation or cited result at each stepAll hypotheses hold
OutputExact result and representationCorrect type, domain and form
VerificationSubstitution, invariant or alternative derivationIndependent agreement
Boundary testZero, identity, degenerate or failed hypothesisScope is understood

Common failure modes and recovery actions

1. Watch for

Using a theorem without checking every hypothesis.

Recovery: Return to the governing definition or requirement and restate the decision in one sentence.

2. Watch for

Treating a suggestive example as a proof of the general case.

Recovery: Separate evidence from assumption, assign an owner and set a date for validation.

3. Watch for

Changing notation or conventions part-way through an argument.

Recovery: Run a small counterexample, boundary test, pilot or independent check before proceeding.

4. Watch for

Hiding a division-by-zero, convergence, finiteness or commutativity assumption.

Recovery: Record the consequence, decision and rationale, then update the controlled baseline.

5. Watch for

Reporting a computed result without a residual, substitution or structural check.

Recovery: Escalate when the issue affects safety, compliance, acceptance, material value or an agreed tolerance.

Review checklist

  • Can every symbol be traced to a definition or prior result?
  • Which hypothesis does each major step use?
  • Does the method cover zero, identity, degenerate and boundary cases?
  • Can the conclusion be checked by a second representation or calculation?
  • Are mandatory requirements distinguished from recommendations and illustrative values?
  • Are sources, assumptions, units, dates and versions recorded closely enough to reproduce the decision?
  • Have safety, legal, ethical, stakeholder and operational consequences been considered at the appropriate level?
  • Is there a named owner and a trigger for review, escalation, change or retirement?

Questions for deeper application

What is the most important distinction a practitioner must preserve when applying Expectation and Variance?

Answer with a fact or cited source where available. Where evidence is incomplete, record the assumption, consequence, responsible owner and next validation action.

Which assumption about expectation would change the result most if it proved false?

Answer with a fact or cited source where available. Where evidence is incomplete, record the assumption, consequence, responsible owner and next validation action.

What evidence would allow an independent reviewer to reproduce or challenge the conclusion?

Answer with a fact or cited source where available. Where evidence is incomplete, record the assumption, consequence, responsible owner and next validation action.

Which boundary, exception or failure case has not yet been tested?

Answer with a fact or cited source where available. Where evidence is incomplete, record the assumption, consequence, responsible owner and next validation action.

What must be handed over, monitored or reviewed after the immediate work is complete?

Answer with a fact or cited source where available. Where evidence is incomplete, record the assumption, consequence, responsible owner and next validation action.

Authoritative references and use notes

The sources below were selected as institutional or primary guidance for the broader practice. They support the handbook method; they do not imply that every statement or clause in a source applies to every project. Confirm the current edition, jurisdiction, contract and application before treating any requirement as mandatory.

  • MIT OpenCourseWare — Introduction to Probability and Statistics — Massachusetts Institute of Technology. Used for probability, inference, hypothesis testing and regression. Accessed 2026-08-13.
  • NIST Digital Library of Mathematical Functions — National Institute of Standards and Technology. Used for mathematical notation, numerical methods, asymptotics and special functions. Accessed 2026-08-13.

Continue learning

Random VariablesGuide · Engineering MathematicsNEXT LESSON →Markov's and Chebyshev's InequalitiesGuide · Engineering MathematicsConditional Probability and IndependenceGuide · Engineering MathematicsThe Birthday ParadoxGuide · Engineering Mathematics
KEVOS · Engineering, manufacturing and project improvement
ArticlesServicesCase studiesAboutContact
© 2026 KEVOS®