Project Management›Project Risk Management›Algorithms for Decision Making›Chapter 21
21Part IV · State Uncertainty
Online Belief-State Planning
Plan from where your belief actually is, right now — simulate a few action-and-observation steps ahead, take the best move, then re-plan as new signals arrive.
There is the same escape here as in Part II: don't solve the whole belief space. From your current belief, work out the best next move by looking a little way ahead — then do it again once the next noisy signal lands.
Online belief-state planning is the partially-observable counterpart of the online planning in Chapter 8. Instead of precomputing a policy over all beliefs, it plans at decision time from the belief you're actually in: it looks ahead over the actions you might take and the observations you might then receive, estimates which first move fares best, commits to it, and re-plans from the updated belief after the real observation arrives. It scales by reasoning only from where you are.
1Looking ahead through actions and observations
The lookahead tree now alternates two kinds of branching. From the current belief you branch on each candidate action; from each action you branch on each possible observation, and each observation leads to a different updated belief. Values are backed up from the leaves to rank the first actions. Because observations multiply the branching, the practical methods keep it manageable — sampling a limited set of observations, and pruning lines that can't win.
2Monte Carlo planning over histories
The standout practical method extends the tree search of Chapter 8 to partial observability. It represents the current belief with a swarm of sampled state hypotheses (a particle set) and grows a search tree over sequences of actions and observations, running simulations to the horizon and backing up the results — concentrating effort on the most promising lines. This lets online methods handle POMDPs far larger than any exact approach, precisely because they never reason about beliefs they'll never reach.
Plan from the belief you're actually in, not from all possible beliefs. Simulate a few action-and-observation steps ahead, commit to the best next move, and re-plan when reality answers back — trading stored policy for fresh, focused computation each time.
This is real-time decision-making under imperfect visibility. From your current read on a project — uncertainty and all — reason forward not just about what you might do, but about what you might then learn, and choose the move that fares best across those futures. Then, when the next report or test result arrives, update your belief and decide again. Crucially, factor in that some actions are worth taking because of what they'll reveal, not only what they'll achieve — the value of an early probe is often the information it returns.