Risk and Uncertainty in R&D Financial Analysis
The dividing line in this 1984 framing is whether a probability distribution of outcomes exists at all — and the source then widens uncertainty to cover a second case, distributions so skewed that statistics stops helping. The measurement regime that feeds both is on the same page.
Two words that are not synonyms
Most working documents use risk and uncertainty interchangeably. R3 does not, and the distinction determines which machinery applies. Getting it wrong does not produce a slightly worse analysis; it produces a confident analysis of the wrong kind of situation.
Two things follow. Risk is the default by preference rather than by evidence — the paper says so. And the extension turns the category from an epistemic one into an operational one: a skewed distribution is knowable, and the reason it is filed under uncertainty is that the project is escaping control.
The two terms, compressed
- Risk
- The situation in which a distribution of outcomes can be calculated or inferred. Handled by quantification: means, standard deviations, coefficients of variation, portfolio effects.
- Uncertainty
- The residual: a non-probabilistic distribution of outcomes is likely or feared, or the distribution is excessively skewed. These are the projects escaping management control, or threatening to.
- Risk, as measured for hurdles
- The amount by which anticipated profit is likely to vary with the economic cycle.
Where the risk comes from, and who is averse to it
R3 traces every source of risk to one cause — imperfect knowledge of the future — then splits it by side of the ledger.
The cost side
- Estimates of development and product costs have inherent inaccuracies of a predictable and measurable sort.
- They may also be biased up or down owing to the frailties of human nature, and owing to weaknesses in the R&D management and control procedures themselves.
- So part of the bias is a property of the organisation, not of the estimator — and more statistics will not remove it.
The benefit side
- Forecasts of future markets; market shares that indicate expected sales; margins specifying rates of return.
- All stated to be equally subject to variation.
- The paper does not treat the commercial forecast as firmer ground than the cost estimate.
The common denominator is stated plainly: all classifications of the sources of risk reduce to the potential for unforeseen variations, up and down, in predicted cash flows both inward and outward. Another paper in this set decomposes uncertainty by origin rather than by ledger side — see Sources of Uncertainty in R&D Projects.
WHO WANTS WHICH RISK, AND WHY
| Party | Disposition | The reason the source gives |
|---|---|---|
| The financial director of a firm | Tends to be risk averse | The concern is bankruptcy from a succession of unexpectedly large negative cash flows |
| The individual entrepreneur and the venture capitalist | May be risk seeking | Prepared to lose a small stake for potentially large returns if all goes to plan or better |
| Most corporations | Tend to be risk averse | Even though stockholders have limited liability, they expect profits and are disturbed by losses |
This is why a risk adjustment has to be declared rather than embedded: the same number reads as prudent or as timid depending on who is looking at it.
The machinery risk calls for
Where a distribution exists, R3 works with the standard statistical apparatus, and its metric of choice is deliberately dimensionless. The coefficient of variation is defined as the ratio of the dispersion — the standard deviation — to the mean, and the paper treats it as a measure of technical risk. In the idealised case the mean of outcomes would be exactly the preestimated value, with dispersion about the mean depending on the quality of R&D management. That is the textbook picture.
R3 also warns off the obvious repair. It is tempting to weight for bias in the preestimates plus the coefficient of variation of costs and benefits — but that move is stated to ignore both the portfolio of projects and the risk aversion of management. Those two omissions are what the rest of the machinery handles.
The portfolio treatment, in order
Treat the projects as a portfolio
Because R&D has the characteristics of an investment, a group of projects can be considered in the same terms as a portfolio of tangible investments.
Claim the diversification benefit, conditionally
Unless all projects are positively correlated, the level of relative risk should be lessened by diversification. The condition does real work — projects that all depend on the same technology or market are not diversified in this sense.
Measure concentration
One concrete portfolio-risk metric is offered: the fraction of the portfolio represented by its largest project. It is a rule proposed by the author with no threshold value, so it tells you the direction of a change, not whether a level is acceptable.
Set the offsetting force against it
Most corporations tend to be risk averse. Diversification pushes the assessment one way and risk aversion the other, and both belong in the same adjustment.
R3 then studies the two offsetting factors using a power-series expansion — defined in the paper as the sum of coefficients multiplied by powers of the independent variable, for powers from zero to infinity. Because coefficients of variation take usual small values, limiting the series to quadratic form is asserted to be reasonable. That step is an analytical rule the author states rather than demonstrates.
The machinery uncertainty calls for
When the distribution is non-probabilistic or excessively skewed, the objective changes. The R&D manager facing projects that escape control is stated to be less interested in developing or applying a mathematical theory than in obtaining a means of bringing a deteriorating situation under control.
The tripwire is a rule of thumb the author proposes with no supporting data: the three standard deviation limit provides a workable cutoff separating probabilistic inadvertence from impending disaster. Below it, the deviation is noise in a system still behaving statistically. Above it, the assumption that a distribution is operating has failed.
- Distribution holds — treat as risk
- Deviation beyond three standard deviations
- The distribution assumption fails
- Objective shifts to regaining control
What goes wrong when the wrong one is assumed
Both errors are common and they fail differently. One produces false precision; the other abandons the quantification the paper argues for throughout.
Two misclassifications and their consequences
The classification also decides which selection technique can carry the case. Every technique on Financial Techniques for R&D Project Selection assumes a forecastable cash flow, and none survives a project that has stopped behaving like a distribution.
Project measurement and control, which feeds both
The risk apparatus needs a history, and R3's fourth tool group supplies it. Measurement runs on two dimensions — budgets and projects — and the firm is budget oriented, measuring deviations in spending and in applied time.
What makes the treatment financial rather than administrative is one requirement: projects must be measured on the basis of past performance as well as current, in order to permit the calculation of risk and to provide guidance for estimating. Measurement is the source of the distributions everything above depends on. The separate literature on structured retrospection is treated on Post-Project Reviews in R&D.
- Actual development cost versus the objective.
- Development cost as a fraction of the R&D budget.
- Actual product cost versus the objective.
- Time elapsed to completion versus the objective.
Those four are extracted for every past full project or separately structured task. With a sufficiently large sample, techniques of statistical inference are stated to be available to determine quantitatively which factors are significant.
THE DERIVED MEASURES AND WHAT THEY ACTUALLY MEASURE
| Measure | Form | What it is really measuring |
|---|---|---|
| The most important management measurement | Mean and standard deviation of past development and product cost outcomes per subordinate supervisor, as a percentage of expected outcome | Technical personnel risk and pure technical risk together — filtered by the supervisor, modelled as a communications channel with characteristic distortion |
| The R&D contribution to commercial risk | Mean and standard deviation of past outcomes of product cost estimates | The only commercial-risk input assigned to R&D. Beyond it, the source says leave commercial risk evaluation to product and marketing management |
| Additional technical risk factors | One or two beyond the core set | A stopping rule, not a measure. Beyond that, further effort is said not to be justified by the refinement in accuracy |
The supervisor-as-channel modelling matters: low variance in a supervisor's reported outcomes may mean a well-run section or a well-filtered one, and the measure cannot tell them apart.
Two rules govern their use. Current project performance must not only be measured but fed back to development teams with minimal delay, and used to forecast the potential outcomes of projects critically. And objectives must be set at attainable levels while goals stay high, with the research manager raising the organisation's standards progressively from the level the initial measurements reveal.
Where the whole apparatus stops working
R3 states its own boundaries, and they are specific enough to test against your situation first.
Preconditions the source states
- The number of projects is large, and their probability of random outcomes is large — the stated scope condition on the whole apparatus
- You are not at a point of great discontinuity: past experience is said to dimension risk in cost estimates except at such points
- You are not a start-up: past experience is said to dimension risk except in start-up companies
- The data can be obtained, at a time and cost worth paying against the department's other opportunity costs
- The measurements can be used practically in selection and evaluation, not merely computed
- The linear simplification is labelled as an approximation, since the two quadratic terms will not exactly cancel
The first three bite hardest. A discontinuity is precisely where the historical baseline fails, and also where the stakes are highest — the source elsewhere calls technological discontinuity risk often the difference between survival and ruin. The method is weakest exactly where the decision matters most, which is a limitation to state in your own analysis rather than discover in review.
What to carry forward
- Risk means a distribution you can calculate or infer. Uncertainty means you cannot — or the distribution is so skewed that the project is escaping control. The two words select different machinery.
- Risk is the source's default by preference, because the whole article is a push toward quantification. Know that when you inherit the classification.
- Real cost distributions are stated to be one-tailed with the preestimate off the mean, and marketing and engineering estimates are biased in opposite directions. Symmetric ranges are misspecified.
- Risk aversion and diversification enter as quadratic terms of opposite sign, so the recommendation is the crude linear adjustment — declared to management as risk-averse, not hidden inside the number.
- Three standard deviations is the proposed tripwire between noise and impending disaster: a rule of thumb offered without supporting data.
- Measurement supplies the distributions — four measurements per past project, plus cost outcomes by supervisor, with the supervisor modelled as a distorting channel.
- Performance measures how well objectives were achieved, never whether they were right. The apparatus also fails at discontinuities and in start-ups, by the source's own statement.
Frequently asked questions
What exactly is the difference between risk and uncertainty here?
Risk is used where a distribution of outcomes can be calculated or inferred. Uncertainty is reserved for the residual cases where a non-probabilistic distribution is likely or feared, and is later extended to cover distributions that are excessively skewed. The second half of that definition is operational rather than statistical: those are the projects escaping management control.
Why does the source recommend a crude linear risk adjustment?
Because it argues that risk aversion and the benefits of portfolio diversification enter the analysis as quadratic terms of opposite sign, so an elaborate treatment of both largely offsets itself. It is careful to say they will not exactly cancel in all instances. The instruction that goes with the shortcut is to tell management the adjustment is basically risk-averse.
Is the three standard deviation rule a standard?
No. It is a heuristic proposed by the author with no supporting data given, offered as a workable cutoff separating probabilistic inadvertence from impending disaster. Treat it as a tripwire you have chosen rather than as an established threshold, and state it as such in any document that uses it.
What should I do once a project crosses into uncertainty?
The source says the objective changes — from measuring the situation to bringing a deteriorating one under control — and states that game-theoretic strategies can handle uncertainty. It names no strategy and works no example, so the method has to be sourced elsewhere and should not be attributed to this paper.
Why measure cost outcomes by supervisor rather than by project?
Because the R&D manager operates through a level of supervision, so the mean and standard deviation of past cost outcomes for each supervisor's projects, as a percentage of preestimate, is the most useful management measurement available. The source notes it captures technical personnel risk and pure technical risk together, filtered by the supervisor as a channel with characteristic distortion.
Does good performance on these measures mean the project was worth doing?
No, and the source is explicit about it. Performance is actual results compared with objectives or preestimates, and it does not take into account the correctness of the objectives. A project can hit every target and still have been the wrong project, which is why selection and measurement have to stay separate activities.
References and source attribution
- R3 - viewing R&D projects financially. Practitioner review and tutorial article in a journal for research management, March-April 1984; 6 printed pages. The risk and uncertainty definitions, the coefficient-of-variation and portfolio treatment, the three standard deviation cutoff, and the measurement and control regime are drawn from its risk and its measurement sections.
- Figure reproduced in outline from R3: a two-panel probability distribution of outcomes of development or product cost, contrasting an idealised symmetric case centred on the preestimate with a realistic skewed one-tailed case in which the preestimate is off-centre. The horizontal axis is annotated as running from good to bad R&D management.
- Extraction limitation: several sentences in the risk section of the available extract fall in damaged columns - the threshold at which statistical methods are said to become meaningless, and part of the passage on directional bias in estimates. Those passages are reported here as incomplete rather than reconstructed.
- Eleven copyrighted journal articles on R&D project management, supplied as a reading set for a literature review and profiled for this library. Front matter, abstracts, framework sections, tables and figures were read; article bodies were not reproduced, and all content here is paraphrase.
- Supplied teaching source for this library (research methods and research process materials). Used here for page conventions and voice only; it does not treat R&D project management.
Suggested questions for Ask KEVOS
- Is my project a risk problem or an uncertainty problem under these definitions?
- Build the four past-project measurements into a template I can apply retrospectively.
- Show me what a skewed one-tailed cost distribution does to a symmetric confidence range.
- How should I declare a risk-averse adjustment to management rather than hiding it in the estimate?
- What concentration does our largest project represent, and what does that tell me without a threshold?
- Which of the stated preconditions does my organisation fail, and what follows from that?