Experiments and implementations · Intermediate

Checking conditional probability through sampling

Simulate an invented screening process with a fixed seed and compare sensitivity with abnormality among positive results.

Ask Logeia about this articleTry the three questionsReview your answers

Updated

On this page
  1. Calculate before simulating
  2. Prepare a reproducible run
  3. Count observations without mixing conditions
  4. Interpret the deviations
  5. Change one parameter at a time
  6. Return to the contingency table
  7. Ask Logeia about this articleCheck your understanding

Calculate before simulating

Use an invented process with abnormality probability 0.1, positive probability 0.9 among abnormal items, and positive probability 0.1 among normal items. The overall positive probability is 0.18, so abnormality among positives is 0.09/0.18=0.5. Sensitivity is 0.9, which answers a different question. Write these expectations before running the program. This is a random-data model, not a measured screening system, and its parameters should not be applied to real tests. A four-cell contingency table supplies an independent way to check the denominators later used in code.

Prepare a reproducible run

Save the code as sample.py and run it with Python 3. It requires only the standard library and makes no network requests or external data reads. A fixed seed supports reproduction in a recorded environment; it does not force observed proportions to equal their expectations. The first draw determines abnormality, and the second draw uses a probability conditional on that result. Using 0.1 for both second-stage cases would implement a different model. Cleanup consists only of removing the script if desired; no service, account, or database state is created.

Count observations without mixing conditions

The counters track all abnormal items, all positive items, and their intersection. Posterior uses the positive count as denominator; sensitivity uses the abnormal count. Their shared numerator does not make them interchangeable. The code generates one hundred thousand observations and checks both proportions against broad expected neighborhoods. These assertions help detect a misimplemented process; they are not confidence intervals or a formal model-validation test. Recalculate the two ratios from the printed counts so that the numerical output is independently connected to the conditional-probability definitions.

python
import random
rng = random.Random(20261028)
n = 100_000
abnormal = positive = true_positive = 0
for _ in range(n):
    a = rng.random() < 0.1
    b = rng.random() < (0.9 if a else 0.1)
    abnormal += a
    positive += b
    true_positive += a and b
posterior = true_positive / positive
sensitivity = true_positive / abnormal
print(n, abnormal, positive, true_positive)
print(round(posterior, 4), round(sensitivity, 4))
assert abs(posterior - 0.5) < 0.03
assert abs(sensitivity - 0.9) < 0.03

Interpret the deviations

Finite samples need not produce exactly 0.5 and 0.9. Both denominator counts fluctuate because the population is generated randomly. First check that denominators are positive and that true positives do not exceed either marginal count. Changing the seed changes the observed estimates. Reproducing one seeded result does not prove a theorem, and one random discrepancy does not disprove Bayes’ rule. Interpret observations in two layers: whether the code generated the stated process, and how much sampling fluctuation remains within that process. The printed counts make these checks more transparent than percentages alone.

Change one parameter at a time

Change only abnormality prevalence to 0.01. The theoretical posterior becomes 0.009 divided by 0.108, about 0.0833, while conditional sensitivity remains 0.9. Update the assertion target accordingly; still requiring a posterior near 0.5 would test an obsolete model. With very small samples, there may be no abnormal observations, so division requires an explicit nonzero-denominator check. A separate experiment can vary the false-positive rate while preserving prevalence. Changing one mechanism at a time makes it possible to explain which probability changed and why the displayed posterior moved.

Return to the contingency table

Recover the other cells using abnormal minus true positives, positive minus true positives, and n minus abnormal minus positive plus true positives. All four counts must be nonnegative and sum to n. For the original and one-percent-prevalence settings, report the table, both conditional proportions, and their deviations from theory. Completion means explaining each denominator and identifying the conditions held fixed, not matching one reference output digit for digit. If that explanation is difficult, revisit the contingency-table guide before adding more random simulations or increasing the sample size.

Reading and practice are recorded separately in this browser, for each language.

Learn this in order

Prepare for AI systems · STEP 10

Choose a learning path