Chaos theory and weather prediction / personal lessonsLesson 14 of 14
Chaos and weather · Lesson 14

Audit a weather-dependent decision

Translate uncertainty into a decision with explicit costs and limits.

By the end: Combine event definition, forecast evidence, and loss assumptions.

Start with the action and its consequences

A fictional event organizer can protect equipment for a fixed cost of eight units. If left unprotected, the specified rain event causes forty units of damage. Protection is assumed to eliminate all that damage and costs eight whether rain occurs or not. The raw ensemble fraction is 25%.

Expected loss without protection is probability times damage, or ten units. Protection costs eight, so it has lower expected loss under these assumptions. The break-even probability is cost divided by damage, or 20%. This model does not prescribe real emergency or financial decisions.

Derive the action threshold from a loss table

A decision table separates what the atmosphere might do from what the decision maker can do. Consider a fictional outdoor workshop that can cover materials for six accounting units. Without covering, the defined rain event damages materials by thirty units. Covering prevents all that damage, and the six-unit cost is paid whether rain occurs or not. There are two actions and two outcomes. The table contains consequences, not probabilities. It remains the same if the rain forecast changes, provided the materials, cover, and definition of damaging rain stay fixed for this workshop.

Let p be the probability of that damaging event during the workshop. The non-event probability is one minus p because these are the only two outcomes. Waiting has expected loss p times thirty plus one minus p times zero. Covering has expected loss p times six plus one minus p times six, which reduces to six. Expected loss is the probability-weighted average of possible losses. It need not equal the loss on the actual occasion. Waiting leads to either zero or thirty units, never to its intermediate expected value on a single workshop day.

Workshop action losses

ActionNo damaging eventDamaging event
Wait0 units30 units
Cover6 units6 units
Avoided loss before cost0 units30 units
This separate fictional workshop pays six units whenever it covers materials. The event concerns its operating window only. Full damage prevention is assumed; neither event probability nor actual weather measurements appear in this consequence table.

At p equal to 0.4, waiting has expected loss twelve units, while covering costs six. The expected saving from covering is therefore six units. If the day turns out dry, covering still costs six and waiting would have cost zero. That outcome does not retroactively change the expected-loss comparison that used information available beforehand. A good decision under uncertainty can have a worse realized outcome on one occasion. Evaluating the decision rule requires repeated relevant cases or an explicit loss criterion, rather than treating each unlucky outcome as proof of a mistaken rule.

The threshold follows from ordinary algebra. Set the covering cost six equal to the expected waiting loss thirty times p. Dividing both sides by thirty gives p equal to 0.2. Above this probability, the expected damage avoided exceeds the covering cost. Below it, waiting has lower expected loss. At equality, both actions have the same expected loss, so the criterion alone gives no strict preference. The ratio uses quantities measured in the same accounting units. It is a probability, not a temperature, rain amount, or score assigned to forecast quality.

Expected waiting loss
W=p×30+(1−p)×0=30×p

This paper workshop has event loss 30 accounting units and covering cost 6 units. p is the damaging-event probability; W and C are expected losses in accounting units.

Expected covering loss
C=p×6+(1−p)×6=6 units
Solve for equality
p*=630=0.2

The ratio cancels accounting units. Above p* covering has lower expected loss; equality gives no strict preference under this criterion.

The straight waiting-loss line is exact only because this example assigns the same thirty-unit loss to every event and zero loss to every non-event. If damage varied with rain intensity, a single event probability might not contain enough information. Light rain and severe rain could both count as events while having different consequences. Then expected loss would require additional outcome categories or a distribution of damage. A binary event is useful when its distinction matches the decision's consequences, not merely because the forecast happens to provide a convenient percentage for it.

Workshop expected losses

Expected loss, accounting units

Workshop expected losses0153000.51

Damaging-event probability

  • Solid: Wait
  • Dashed: Cover
These are exact straight expected-loss relations for one workshop under full prevention, fixed six-unit cost, and thirty-unit event damage. Connected points interpolate those algebraic relations, not a weather trajectory or a sequence of realized losses.

A fixed cost also needs a clear interpretation. Six units might represent supplies and labor under an agreed conversion into one common measure. If covering has unpriced inconvenience, blocks access, or creates another hazard, the table omits consequences relevant to the choice. The arithmetic remains correct for its entries while the recommendation may become unsuitable. Naming the cost and loss units makes that omission visible. It also prevents treating a calculation for replaceable materials as a complete model for safety decisions, where constraints and consequences may not reduce to a single monetary average.

Audit the probability before optimizing

Define rain at the site, the 24-hour window, and the threshold that damages the equipment. Confirm that the forecast probability addresses that event. Ask whether historical verification supports using raw ensemble fractions and whether important uncertainties are missing.

A 25% estimate versus a 20% threshold is close enough that probability and cost errors can change the decision. Recalculate under plausible alternatives rather than displaying one action as certain. Expected loss is one criterion; risk tolerance, safety constraints, and unequal impacts may require others.

Match the forecast to the decision event

A probability must refer to the same event that appears in the loss table. For the workshop, define damaging rain as at least a chosen amount at its site during its operating window. A forecast for any rain across a large region over twenty-four hours answers another question. Neither percentage can replace the other without additional information. The event's location, accumulation window, and inclusive threshold determine what counts as an occurrence. A carefully optimized action based on the wrong event can be worse than a simpler rule based on the event that actually causes damage.

Invent a two-stage comparison to expose the mismatch. Suppose the probability of measurable rain over a full day is 0.6. Given such rain, suppose the probability that enough reaches the workshop during its operating hours is 0.25. If workshop damage requires that full-day rain event, the probability of damaging workshop rain is 0.6 times 0.25, or 0.15. The multiplication uses a conditional probability, not independence. At the workshop's 0.2 action threshold, these assumptions favor waiting. Substituting the broader 0.6 probability would favor covering, so event matching changes the recommendation.

Match the narrower event
P(D)=P(R)×P(D|R)

D is stipulated to be a subset of full-day rain R. This product uses a conditional probability, not independence.

Use the conditional fraction
P(D)=0.6×0.25=0.15<0.2

The broader rain probability cannot replace the workshop event probability in its loss table.

Broader and narrower events

QuantityAssumed probabilityRole
Full-day rain0.6Broader event
Workshop damage given rain0.25Conditional fraction
Workshop damage0.15Product under nesting
Workshop action threshold0.2Cost divided by loss
The damage event is stipulated to be a subset of full-day rain, so its probability is 0.6 times 0.25. These are invented workshop assumptions, not a temporal or spatial conversion rule for real forecasts.

The conditional value 0.25 is an extra assumption, not something you can infer from the full-day percentage. Site conditions, timing, and storm movement could alter it. Even a well-calibrated regional product might not be calibrated for the workshop's narrower event. Calibration means that cases assigned a particular probability have a corresponding event frequency over relevant repeated cases. Checking calibration therefore requires both stored predictions and observations of the matching event. Relabeling a forecast percentage with a more convenient event name does not transfer the evidence that supported the original percentage.

An ensemble fraction is another possible input whose interpretation needs support. If seven of twenty invented members reach an event threshold, the raw fraction is 0.35. That calculation is exact for the twenty members. It does not establish that the matching event occurs in about 35 percent of relevant real cases assigned that probability. Members can share model errors, and a finite set can miss outcomes. Copying all twenty members produces fourteen successes out of forty but the same information. More runs help only if their design represents relevant uncertainty, not because a larger denominator itself certifies the percentage.

Forecast verification and decision value use different comparisons. A Brier score averages squared probability errors across a specified set of events. The action table instead assigns losses to decisions and outcomes. Two users with different covering costs can act differently on the same verified probability. A score improvement concentrated far from one user's action threshold may have little effect on that user's losses, while an improvement near the threshold can change actions. This does not make the score meaningless. It means a general measure of probability quality is not a complete measure of every user's utility.

Probability ranges and action changes

Probability rangeExpected waiting lossComparison with cost 6
0.12 to 0.183.6 to 5.4 unitsWait throughout
0.15 to 0.304.5 to 9 unitsAction changes
Exactly 0.206 unitsEqual expected loss
All rows use thirty-unit workshop damage and full prevention for a six-unit cost. The probability ranges are stipulated sensitivity cases, not estimated confidence intervals. They show whether uncertainty can cross this decision threshold.

Probability uncertainty can be represented as a range when a single well-supported value is unavailable. Suppose reasonable inputs put the workshop's event probability between 0.12 and 0.18. Waiting's expected loss lies between 3.6 and 5.4 units, below covering's six throughout. If the range instead spans 0.15 to 0.3, the lower-loss action changes within it. The first recommendation is robust to that stated uncertainty range; the second depends on resolving uncertainty or selecting another criterion. Neither range is automatically a confidence interval unless a method gives it that statistical interpretation.

Inspect the rules and their result

Break-even probability
p*=CL

Cost is paid in every protected case. Loss occurs only if the event happens unprotected. Protection eliminates that loss in this example.

Probability changes retain cost; cost changes retain probability. Unprotected event damage stays 40 units. Protection costs the chosen amount in every case and eliminates damage. These are invented loss assumptions. The thin dotted line marks the selected probability.

Expected losses for both actions across event probabilitiesExpected loss, units40200050100Event probability, percent
Solid blue: Wait. Dashed amber: Protect, cost 8.
ActionExpected loss at selected probability
Wait10.00
Protect8.00
Threshold 20%. Protect under expected-loss minimization. At equality neither action is strictly better.
ActionNo damaging rainDamaging rainExpected loss at p = 0.25
Wait04010
Protect888
A complete conditional decision calculation.

Explore the lab without changing its assumptions

The interactive lab uses a different fixed damage amount from the workshop calculation. Its unprotected damaging event always costs forty units. The probability slider runs from zero to one hundred percent, while the cost buttons choose eight, twelve, or twenty units. Changing probability retains cost; changing cost retains probability. Both action lines remain visible, and the vertical marker shows the selected probability. The horizontal axis describes alternative probability assumptions for the same event, not elapsed time. Moving the marker cannot make rain occur or reveal which outcome will happen in any real situation.

Choose Cost 20 and move the probability slider to 40 percent. Waiting has expected loss sixteen units because 0.4 times forty is sixteen. Protection costs twenty, so waiting has lower expected loss by four units. Move only the probability to 60 percent. Waiting now has expected loss twenty-four units, so protection has lower expected loss by four. The action changes because the probability crosses the fifty-percent threshold for this cost. Nothing about the stipulated damage or effectiveness changed during that comparison, which isolates the role of the probability input.

Lab comparison at cost twenty

Expected loss, units

Lab comparison at cost twenty02040050100

Event probability, percent

  • Solid: Wait
  • Dashed: Protect
Samples use the lab loss of forty and cost twenty for one specified event window. Connected segments are exact linear expectation relations under full prevention. Percent on this axis must be divided by one hundred before multiplying damage.

At Cost 20 and probability 50 percent, both expected losses are twenty units. The equality readout is meaningful rather than a rounding failure. One action pays twenty in every outcome, whereas waiting pays either zero or forty. Their averages agree but their distributions differ. A decision maker who dislikes large losses might prefer protection at equality, while another constraint could favor waiting. The lab intentionally does not settle such preferences. Its equality statement reports only the result of expected-loss minimization under the specified consequence table and the selected event probability.

A useful sensitivity measure is the difference between expected waiting loss and protection cost. With forty-unit damage and cost twenty, that difference equals forty times p minus twenty. Raising p by 0.05 raises the difference by two units. This relation explains why a probability estimate near the threshold demands more attention than one far away, if all other entries are trusted. It does not imply probability precision is the only uncertainty. An error in damage or intervention effectiveness can also move the threshold, even if the numerical probability remains unchanged in the display.

The three cost settings also show why one probability does not prescribe one universal action. At 40 percent, waiting's expected loss remains sixteen for all three settings. Protection costs eight, twelve, or twenty, so its advantage compared with waiting is eight, four, or negative four units. These are comparisons across hypothetical cost assumptions, not three simultaneous services with verified prices. The retained probability makes the comparison easy to isolate. If a real intervention's cost changes with the forecast or expected severity, that dependence needs a new calculation outside the lab's fixed-cost model.

Cost comparisons at forty percent

Protection costWaiting lossSaving from protection
8 units16 units8 units
12 units16 units4 units
20 units16 units−4 units
Each row keeps event probability at forty percent and unprotected event damage at forty units. Savings mean expected waiting loss minus certain protection cost. A negative saving favors waiting under this criterion, not a prediction that damage cannot occur.

Read the graph and table together. The graph shows where the selected protection-cost line crosses the waiting line over the full range of probabilities. Changing cost moves that crossing. The table evaluates the two actions only at the selected marker. Neither display verifies the input probability or estimates actual material damage. There is no forecasting system, observation stream, or clock behind the slider. Its purpose is to separate the algebra of a conditional choice from the evidence needed to justify its inputs. An exact graph can support a conditional statement while leaving the real decision unresolved.

Explain both the recommendation and its limits

A useful final explanation identifies the rule, state uncertainty, numerical assumptions, event definition, verification benchmark, and consequences. Better observations can improve initial conditions. Better numerical schemes can reduce approximation error. Neither alone removes all forecast uncertainty.

For real hazards, use current local meteorological warnings and emergency guidance. This offline course contains no current forecasts and is not a warning service. The goal is to interpret expert explanations and ask precise questions, not to qualify as a forecaster.

Test incomplete protection and alternative criteria

Full prevention makes the lab threshold simple, but some actions only reduce damage. Return to the fictional workshop with thirty-unit unprotected event loss and six-unit covering cost. Suppose covering leaves ten units of damage when the event occurs. Then its non-event loss is six and its event loss is sixteen. Expected covering loss becomes six plus ten times p. Waiting still costs thirty times p in expectation. Protection now avoids twenty units of event damage before paying its cost. The relevant comparison uses avoided damage, not the original damage amount alone.

Set thirty times p equal to six plus ten times p. Subtract ten times p from both sides to obtain twenty times p equal to six, so the new threshold is 0.3. At p equal to 0.25, waiting's expected loss is 7.5 units and partial protection's expected loss is 8.5. At p equal to 0.4, the corresponding losses are twelve and ten. Lower effectiveness raises the probability needed to justify the same cost because each protected event yields less avoided damage. This extension is a paper calculation, not an effectiveness control available in the lab.

Include residual damage
C=6+10×p; W=30×p

Covering costs 6 units and leaves 10 units of event damage. W and C are expected losses in accounting units; p is event probability.

Set expected losses equal
30×p=6+10×p⇒20×p=6
Divide by avoided damage
p*=620=0.3
Compare at p = 0.4
W=30×0.4=12 units; C=6+4=10 units

Covering reduces expected loss but still costs 16 units if the damaging event occurs.

Partial prevention in the workshop

ActionNo eventEvent
Wait0 units30 units
Partial cover6 units16 units
Damage avoided before cost0 units20 units
Covering always costs six units and leaves ten units of event damage. Thus its event loss includes both cost and residual damage. The event window is one workshop; these invented consequences are not intervention-effectiveness measurements.

Robustness can involve more than a probability range. Suppose workshop probability lies between 0.2 and 0.4, while the residual protected damage lies between zero and ten units. Waiting loss spans six to twelve units. Covering loss spans six to ten units, but comparing these separate ranges alone is insufficient because both depend on the same p. At p equal to 0.2 with residual ten, waiting is cheaper. At p equal to 0.4 with residual ten, covering is cheaper. The action is not uniformly preferred across the stipulated joint assumptions.

Expected loss treats one additional unit of loss as equally important at every starting wealth level. That may be an unsuitable preference for a small organization unable to absorb the full thirty-unit damage. A hard budget constraint or a limit on unacceptable outcomes asks a different question from minimizing an average. Such a constraint must be stated, not hidden inside a revised probability. The atmosphere's event probability does not become larger because a user is vulnerable. Vulnerability changes consequences or the decision rule while the meteorological event definition and probability can remain exactly the same.

Another criterion is worst-case loss among the specified outcomes. Under the partial-protection workshop table, waiting has worst-case loss thirty, and covering has worst-case loss sixteen. A rule minimizing that worst case chooses covering without using p. This rule differs from expected-loss minimization and can pay for protection even when the event probability is low. It is not universally superior; it sacrifices average performance in some cases to limit the largest listed loss. An omitted outcome can invalidate either criterion's practical conclusion, so changing the criterion does not remove the need for a credible consequence table.

Two criteria on the same workshop

CriterionWaitPartial cover
Expected loss at p = 0.13 units7 units
Expected loss at p = 0.412 units10 units
Worst listed outcome30 units16 units
The probability-dependent rows average over one workshop's two outcomes. The worst-case row does not average and ignores probability. Different preferred actions can follow from the same losses when the criterion or probability changes.

A final decision statement should say which comparison it actually supports. For the partial-protection workshop at p equal to 0.4, covering reduces expected loss from twelve to ten units under the specified damage amounts and fixed cost. The remaining event loss is sixteen, so covering does not eliminate risk. That statement preserves the probability, consequence table, and criterion without promising a safe outcome. Real weather-related safety decisions also require current official guidance and appropriate expertise. This offline calculation teaches conditional reasoning; it supplies neither present hazard information nor authorization to disregard warnings or safety obligations.

Exercises

Test a more expensive action

If protection costs twelve instead of eight, with probability 25% and damage forty, what changes?

Hint

Compare twelve with 0.25 times forty.

Worked answer
Protection costWait lossThresholdLower expected loss
81020%Protect
121030%Wait
Same probability, different consequences.

Waiting has expected loss ten; protection costs twelve. Waiting minimizes expected loss in this simplified model. The new threshold is 12/40 = 30%. The forecast did not change, but the preferred action did.

Audit a confident headline

A headline says: "All members agree, so flooding is impossible." Name two problems and one useful next check.

Hint

Ask about shared model error and whether rainfall is the same event as flooding.

Worked answer
Headline inferenceMissing check
Agreement means certaintyCalibration, coverage, shared model errors
Rainfall means flood outcomeHydrology and exposure
Offline toy means present safetyCurrent official local guidance
Connect each inference to the missing evidence.

Agreement does not exclude shared bias or omitted extremes. Flooding also depends on runoff, soil saturation, drainage, and upstream water, so rain-member agreement is not a complete flood forecast. Check a verified flood product for the actual basin and time window, together with current official warnings.

Sources and further reading

Selected reference passages support the definitions and modeling distinctions below. The examples, explanations, and implementations are original. Terminology and assumptions are stated within each lesson; source review is not an independent audit of this course. Optional links are not runtime dependencies.

  • Forecast verification methods, CAWCR

    Selected sections on reliability, resolution, Brier scores, skill, and forecast value. The eight-case sample is deliberately too small to establish real calibration.

  • ECMWF: Quantifying forecast uncertainty

    Initial-condition and model perturbations. Operational configurations change; the twelve invented rain members are not ECMWF output.

  • ECMWF: Modelling and prediction

    Atmospheric dynamics, physical processes, model limitations, and probabilistic forecasting. No current operational resolution or skill claim is inferred.