Policy Search

There's a more direct route to a good decision rule. Don't compute values and derive a policy from them — write the policy down with a few dials, and turn the dials until it works.

Everything so far reasoned about the value of states and then acted greedily. Policy search sidesteps values entirely. You express a decision rule as a parameterised policy — a formula with adjustable settings, such as "if schedule variance exceeds θ₁, add resources; if open risk exposure exceeds θ₂, escalate" — and you search directly for the parameter values that make the policy perform best. When you can simulate a project but can't model it cleanly enough for value iteration, this is often the most practical path.

1Judging a policy by simulation

To search for good parameters you need to score any candidate. That's approximate policy evaluation: run the parameterised policy through many simulated projects and average the return it earns. The average is noisy — each simulation is one roll of the dice — but with enough runs it ranks candidates reliably. The whole method rests on being able to simulate outcomes, not on having tidy transition probabilities.

Figure 1. Each dot is a candidate policy, scored by simulated return. The search samples, evaluates, and moves toward higher-performing settings — climbing a landscape it can measure but never sees in full.

2Ways to search the dials

Local search nudges the current parameters and keeps changes that improve the simulated return — simple hill-climbing over settings.
Genetic algorithms keep a population of policies, combine and mutate the better ones, and let good settings propagate.
The cross-entropy method maintains a probability distribution over settings, samples a batch, keeps the best-performing fraction, and re-centres the distribution on them — tightening toward good regions each round.
Evolution strategies perturb the current parameters in many random directions, then step toward the perturbations that scored well — robust and easy to parallelise.

Key idea

Policy search optimises the decision rule directly against simulated performance, never computing the value of a single state. If you can simulate your project and describe your response as adjustable settings, you can tune it without a model.

What it means in practice

Write your risk-response strategy as a rule with explicit thresholds and dials, then let simulation tell you where to set them — rather than arguing over trigger levels in a meeting. Because it needs only the ability to simulate outcomes, policy search suits messy, real-world response logic that resists clean mathematical modelling. The output is directly usable: a tuned, transparent set of thresholds your team can read and follow.

Handbook application: from concept to controlled practice

Purpose. This expanded section turns the original page into a practical handbook. It preserves the supplied material and adds a repeatable way to apply, check and review Policy Search. It does not replace a contract, legislation, a controlled standard, competent engineering judgement or specialist advice.

The operating aim is to convert the subject into a governed decision, owned work, usable evidence and a reviewable outcome. Read the original explanation first, then use the workflow and checks below to convert knowledge into evidence.

Use Policy Search as a decision instrument rather than an administrative form. The subject terms—policy, search, simulation, searching, directly—need an explicit connection to the project objective, business value and stakeholder commitments. Before completing the artefact, write one sentence stating who will use it, what decision it supports and when that decision is required.

Apply a disciplined information model. Separate facts supported by evidence, forecasts derived from a method, assumptions awaiting validation, constraints that limit choice, risks that may occur, issues that already exist and actions assigned to people. Each material entry should have an owner, date, status and next review point. Where probability or impact scores are used, define the scale so different reviewers interpret it consistently.

A baseline is useful only when changes are visible. Give the artefact an identifier, version, approval state and effective date. Define which changes require reapproval, how superseded versions are retained and where supporting evidence is stored. During reviews, focus on exceptions, decisions and trends rather than reading every field aloud. Record the decision and rationale, not merely that a meeting occurred.

Close the loop beyond delivery. Confirm acceptance criteria, unresolved items, transferred responsibilities and operational ownership. Where benefits are expected, identify the outcome measure, baseline, target, observation period and owner who remains accountable after the project team disbands. Lessons should describe the condition, consequence and reusable action; a generic statement such as “communicate better” cannot improve the next project.

Step-by-step operating method

Clarify the decision. Name the outcome, sponsor, affected stakeholders and decision that this work must enable.
Set boundaries. Record scope, assumptions, constraints, dependencies, tolerances and escalation conditions.
Plan the evidence. Define deliverables, measures, owners, due dates and acceptance criteria before execution.
Control delivery. Compare actual performance with the baseline, assess changes and manage risks and issues explicitly.
Close the loop. Confirm acceptance, transfer ownership, capture lessons and track benefits beyond handover.

Completion and governance protocol

Start with a short drafting workshop involving the accountable owner and the people who hold the evidence. Complete high-consequence fields first: objective, scope, owner, baseline, acceptance, dependencies and escalation. Mark unknowns as assumptions or actions rather than hiding them behind vague prose. Circulate a review draft, resolve conflicting interpretations, baseline the approved version and place the next review date in an owned schedule.

Information type	Minimum useful content	Review test
Outcome	Observable change and intended recipient	Not merely a deliverable or activity
Measure	Definition, baseline, target, frequency and source	Two reviewers would calculate it the same way
Ownership	One accountable role plus contributors and approver	Authority matches responsibility
Uncertainty	Assumption, risk or issue with response and trigger	Status reflects current reality
Control	Version, approval, review date and change rule	Current baseline is identifiable

Common failure modes and recovery actions

1. Watch for

Producing a document with no named decision or accountable owner.

Recovery: Return to the governing definition or requirement and restate the decision in one sentence.

2. Watch for

Mixing risks, current issues, assumptions and actions in one unstructured list.

Recovery: Separate evidence from assumption, assign an owner and set a date for validation.

3. Watch for

Measuring activity or output while leaving the intended outcome undefined.

Recovery: Run a small counterexample, boundary test, pilot or independent check before proceeding.

4. Watch for

Accepting changes without evaluating effects on value, scope, schedule, cost and risk.

Recovery: Record the consequence, decision and rationale, then update the controlled baseline.

5. Watch for

Closing the project at delivery even though benefit ownership has not transferred.

Recovery: Escalate when the issue affects safety, compliance, acceptance, material value or an agreed tolerance.