Independent events leave each other’s probabilities unchanged. Exclusive events cannot happen together.
By the end: Check independence algebraically and identify plausible dependence in repeated observations.
Events A and B are independent when the probability of both equals the product of their individual probabilities. If B has positive probability, this is equivalent to P(A | B) = P(A). Learning B then leaves the probability of A unchanged. Independence is relative to a particular model and available information.
Mutually exclusive events have no overlap. If both have positive probability, they cannot be independent: learning one happened makes the other impossible. On one die, getting one and getting two are exclusive. On two independent dice, first die one and second die two can happen together.
Suppose a population contains 100 equally weighted cases. Event A occurs in forty cases and event B in thirty. If they are independent, learning A should leave B's fraction at 30%. Therefore twelve of the forty A cases must also be B cases, since 40 times 0.30 = 12. Among the sixty cases outside A, eighteen must be B cases, since 60 times 0.30 = 18. Together these overlaps recover the thirty B cases in the whole population.
The four disjoint groups are then twelve in both events, twenty-eight in A only, eighteen in B only, and forty-two in neither. Independence has fixed a particular overlap, rather than declaring that there is no overlap. Check the reverse direction too. Among the thirty B cases, twelve are A cases, giving 12/30 = 40%, the original A fraction. The same joint table supports both directions because 0.40 times 0.30 gives the same product as 0.30 times 0.40.
Now keep the margins but make the events mutually exclusive. All forty A cases must sit outside B, and all thirty B cases outside A. The remaining thirty are neither. Learning A changes the probability of B from 30% to zero. This is a strong dependence enforced by the exclusion rule. The product 0.40 times 0.30 is still 0.12, while the actual joint probability is zero. Events with positive probabilities cannot satisfy both exclusivity and independence under the same probability model.
These two tables also explain why the ordinary word 'unrelated' is too imprecise. Events can concern separate objects but share a cause. Conversely, events concerning one object can be independent in a particular model. On a fair six-sided die, let A mean an even result and B mean a result of five or six. Their probabilities are one half and one third. Only six satisfies both, with probability one sixth, exactly the product. Independence concerns the weights assigned to these specified events.
Use that die example as a paper exploration. List which of the six equally likely faces remain after learning that the result is even. They are two, four, and six. One of these three faces satisfies B, so its conditional probability remains one third. Replace B with 'greater than three.' Two of the three even faces now qualify, whereas three of six qualify initially. The new conditional probability is two thirds rather than one half. Changing the event changes whether independence holds, even on the same fair die.
A limiting case prevents overgeneralization. If A has probability zero, its joint probability with any B is also zero, which equals the product of their probabilities. The product definition therefore declares A independent of B. But conditioning on A through elementary division is undefined, since its denominator is zero. Likewise an event of probability one imposes no restriction on the other event's weights. The usual comparison of conditional and unconditional probabilities needs a positive denominator, while the product definition handles these boundary cases directly.
| B event | P(B) | Even faces in B | P(B given even) |
|---|---|---|---|
| Five or six | 2/6 = 1/3 | Six only | 1/3 |
| Greater than three | 3/6 = 1/2 | Four and six | 2/3 |
Two servers sharing a power supply can fail together more often than the product of their separate failure probabilities predicts. Two measurements of the same contaminated sample may share an error. Multiplying separate error rates as if every check were fresh evidence can greatly overstate reliability.
Conditional independence means independence after fixing a specified condition. Test results may be independent given true fault status even while correlated across a mixed population. Faulty items produce more positives on both tests. Conversely, a common sensor defect can preserve dependence even after fault status is fixed.
Consider a constructed population with 20% faulty items and 80% sound items. Two tests each flag 80% of faulty items and 10% of sound items. Assume the tests are independent after fixing whether the item is faulty. This means that within faulty items, learning the first result leaves the second test's flag probability at 80%. Within sound items, it stays at 10%. The assumption concerns each state-specific population, not the mixture of states encountered when items arrive for testing.
For a single test, the overall flag probability is 0.20 times 0.80 plus 0.80 times 0.10 = 0.24. For two flags, calculate within the states first. Faulty items contribute 0.20 times 0.80 times 0.80 = 0.128. Sound items contribute 0.80 times 0.10 times 0.10 = 0.008. Adding gives 0.136, or 13.6%. Simply squaring the overall 24% flag rate would give 5.76%, which is wrong because the tests are not independent across the mixed population.
Why does the mixture produce dependence? A first flag makes faulty items more common among the remaining possibilities. The second test flags faulty items more often, so its chance of flagging rises in this selected group. Specifically, the probability of a second flag given a first flag is 0.136/0.24, about 56.67%, not 24%. The first measurement need not physically alter the second measurement. The information changes the mix of item states over which the second result is predicted.
Conditional independence also tells us exactly which likelihoods can be multiplied when updating evidence. Given fault, two flags have likelihood 0.80 times 0.80 = 0.64. Given soundness, their likelihood is 0.10 times 0.10 = 0.01. These are probabilities of evidence under stipulated states, not probabilities of states after evidence. Including the prior weights gives a posterior fault probability of 0.128/0.136, about 94.12%. The prior and the likelihoods play different roles even though both enter the same calculation.
Now change the mechanism. Suppose a fraction of sound items carry a contaminant that makes both tests flag. Even after soundness is fixed, learning the first result can reveal contamination and change the probability of the second. The original conditional-independence assumption has failed. Different instruments do not automatically avoid this problem if they examine the same contaminated material. Separate measurements require a joint error model or evidence about the shared causes. Having two collection times is not itself evidence that the errors are independent.
Before multiplying two state-specific error rates, write the full condition in words. 'Among sound items of this type, knowing the first result does not change the distribution of the second result.' Ask whether shared preparation, temperature, operator behavior, or item difficulty makes that claim implausible. Independence given a richer condition can hold when independence given state alone fails. A model that accounts for both state and contamination must average over contamination to recover the state-only prediction, rather than discard it after multiplying convenient marginal rates.
| Among sound items | Independent errors | Shared contamination |
|---|---|---|
| First test flags | 10% | 10% |
| Second test flags | 10% | 10% |
| Both flag | 1% | 10% |
| Second flags given first flag | 10% | 100% |
| Neither flags | 81% | 90% |
| Bits | A: first 1 | B: second 1 | C: match |
|---|---|---|---|
| 00 | No | No | Yes |
| 01 | No | Yes | No |
| 10 | Yes | No | No |
| 11 | Yes | Yes | Yes |
Imagine two components, each with a 10% probability of failing during a specified hour. These individual probabilities do not determine the probability of both failures. One model treats their failures as independent. A second model says a shared power failure disables both with probability 10%, and otherwise both work. A third model allows exactly one component to fail, with probability 10% for each component and 80% for neither. All three models preserve each component's individual failure probability while giving different joint outcomes.
Under independence, both fail with probability 0.10 times 0.10 = 0.01. First-only failure has probability 0.10 times 0.90 = 0.09, and second-only failure also has probability 0.09. Neither fails with probability 0.90 times 0.90 = 0.81. These four probabilities sum to one. Under the shared-power model, both fail with probability 0.10 and neither fails with probability 0.90. Under the exclusive-failure model, both fail with probability zero. The individual rates therefore allow a tenfold difference, or elimination, of the simultaneous-failure probability.
The event 'at least one fails' changes in the opposite direction. It has probability 19% in the independent model, 10% in the shared-power model, and 20% in the exclusive model. Stronger positive dependence can concentrate failures into fewer hours while making those hours more severe. Whether this is better or worse depends on the question. A service that survives one component failure cares about both failing. A process that stops after any failure cares about at least one. Dependence is not itself a loss measurement.
| Outcome in one hour | Independent | Shared failure | Exclusive failures |
|---|---|---|---|
| Both fail | 1% | 10% | 0% |
| First only fails | 9% | 0% | 10% |
| Second only fails | 9% | 0% | 10% |
| Neither fails | 81% | 90% | 80% |
| At least one fails | 19% | 10% | 20% |
Redundancy calculations must therefore begin with the service rule. If either working component suffices, independent components yield a 1% outage probability in this model. Adding a common outage cause creates a lower limit that independent component improvements cannot remove. For example, suppose common power loss has probability 2%. When power is available, each component fails independently with probability 10%. The total service outage probability is 0.02 plus 0.98 times 0.01 = 0.0298, or 2.98%. The branches are disjoint, so their probabilities add.
Compare this with multiplying the new individual component rates. Each component fails with probability 0.02 plus 0.98 times 0.10 = 0.118. Squaring 0.118 gives 0.013924, or 1.3924%, well below the correct 2.98% outage probability. Both component failures include the same power-loss branch. Treating those marginal rates as independent effectively treats a shared cause as two unrelated chances. The error is in the joint model, not in multiplication itself. Conditioning on available power restores the stipulated independence of the component-specific failures.
For a guided design comparison, set component-specific failure to zero while leaving the 2% power-loss probability fixed. The outage probability becomes 2%, not zero. Alternatively, remove power loss but retain the 10% independent component risks, and outage becomes 1%. These counterfactual calculations expose which modeled mechanism each improvement addresses. They do not establish that a real backup power system will remove the common cause. Evidence about the intervention and its possible new failures is separate from evaluating the arithmetic under a stipulated mechanism.
For three or more events, pairwise independence does not guarantee that every combination factors. Toss two independent fair bits. Let A be that the first is one, B that the second is one, and C that the bits match. Every pair is independent, yet A and B together determine C.
For a sequence modeled as independent trials, independence concerns all relevant combinations, not just neighboring correlations. Use the mechanism of data collection as well as observed associations. A small sample can fail to reveal dependence; zero correlation alone is also weaker than independence.
The two-bit example has only four equally likely outcomes, so every independence claim can be checked by enumeration. A occurs in outcomes 10 and 11. B occurs in 01 and 11. C, the event that the bits match, occurs in 00 and 11. Each event therefore has probability one half. Every pair overlaps in exactly one outcome, giving probability one quarter, equal to one half times one half. All three pairwise independence checks pass in this probability model.
But learning A and B together identifies outcome 11, so C becomes certain. The triple overlap has probability one quarter, whereas multiplying the three individual probabilities gives one eighth. Mutual independence would require this triple factorization as well as the pairwise factorizations. The failure matters whenever a calculation concerns several events jointly. Testing each pair cannot justify multiplying all three probabilities, even when every pairwise check is exact rather than an estimate from a small sample. The omitted combination can carry a deterministic restriction.
Compare with three genuinely independent fair bits, whose eight equally likely outcomes have probability one eighth each. If the events are 'first bit is one,' 'second bit is one,' and 'third bit is one,' all three happen only in 111. Their joint probability is one eighth as the product requires. The two-bit construction instead defines its third event from the first two bits. It can imitate the pairwise fractions of three independent bits while having fewer possible combinations and a different probability for the all-true event.
Dependence may also escape a simple numerical correlation check. Correlation summarizes whether larger values of one numerical variable tend to accompany larger or smaller values of another in a linear pattern. Consider an equally likely choice among the numbers negative one, zero, and one. Record that number and its square. Positive and negative departures balance, giving zero correlation, yet the square is determined by the number. Learning that the square is zero identifies the number as zero. Independence would forbid that change in probability.
In observed data, even the simpler pairwise fractions are estimates. A small register can show an overlap close to the product by chance, whether the generating events are independent or weakly dependent. A mechanism that generates fresh fair bits gives a reason for an independence model; an observed overlap supplies a check on that model, with limited precision. Neither a failure to notice a pattern nor a rounded correlation of zero establishes every joint factorization needed for a long sequence of evidence or reliability calculations.
When reading a claim based on many observations, identify what counts as an observation and which combinations the calculation uses. Five sources may share an original dataset even if their authors never communicate. Multiple instruments may share a calibration error even if their random electronic noise is independent. A suitable model can represent both shared and separate variation. Its predictions then follow from the stated joint probabilities. The number of reports, measurements, or pairwise checks alone cannot establish the independence assumptions required to treat every result as fresh evidence.
Two positive-probability alarms cannot occur together under a protocol. Can their activations be independent?
Compare the joint probability with the product.
| Quantity | Value |
|---|---|
| P(A and B) | 0 |
| P(A) × P(B) | Positive |
| Equal? | No |
No. Their joint probability is zero, while the product of their positive marginal probabilities is positive. Exclusivity enforces dependence. An event with probability zero is a degenerate exception to this argument.
A test has a 5% false-positive rate. A program displays the same stored result twice. Is the false-positive chance for two positive displays 0.25%?
The second display contains no new measurement.
| Display pair for a sound item | Probability |
|---|---|
| Positive, positive | 5% |
| Negative, negative | 95% |
| Disagreement | 0% |
No. Given the first display, the second is certain to match. Among sound items, the probability of two positive displays remains 5%. Squaring 0.05 would require two independent false-positive events conditional on soundness, not two views of one event.
The explanations and exercises are complete without these optional links. The references distinguish probability identities from inference procedures and reporting guidance. The examples are constructed teaching models, not empirical estimates or professional decision advice.
Addition, multiplication, conditional probability, and the role of independence.