Skip to content

Quickstart

Seven commands. The corpus needs no credentials and the whole thing trains on a laptop in about a minute.

1. Install and initialise

pip install "prodrome[dev]"
prodrome init

init writes every shipped configuration into configs/. Each command below defaults to configs/default.yaml, so --config is left out.

2. Get the corpus

prodrome data fetch
prodrome data prepare

prepared corpus
  patients         40,336
  rows             1,552,210
  septic patients  2,932 (7.3%)
  per site         A: 20,336, B: 20,000
  corpus sha256    ef46d3e6ec2497f7279ef352acdc32521cf54d42bb785ba2f3df4f2e179d854f

That digest is computed over the sorted per-patient content hashes. Every report carries it, and the gate refuses to compare two reports that disagree on it, because they did not measure the same thing.

3. Train

prodrome train

Fits on the training patients of site A, holding a slice back internally for early stopping so the calibration patients stay untouched by model fitting.

4. Calibrate, and read what it promises

prodrome calibrate

alert policy for default
  target precision      0.30
  alert budget          4.0 per patient-day
  confidence            95%
  threshold             0.06080
  calibration patients  4,067
  alerted patients      592
  observed precision    0.3429
  lower bound           0.3106
  observed alert rate   0.174 per patient-day

With probability at least 95% over the calibration draw, at most 4.0 alerts are raised
per patient-day and at least 30% of alerted patients go on to develop sepsis.

The threshold is not a default. It is the lowest cut whose precision, at its exact Clopper-Pearson lower bound, clears the floor and whose alert rate stays inside the budget. When nothing qualifies, the policy says so and alerts on nothing. See Calibrated abstention.

5. Evaluate, on both sites

prodrome eval

You get one report per test site. The one that matters is site B, which the model never trained on and was never calibrated on:

default on site B (external)
  patients                       20,000 (1,142 septic)
  utility                        0.2468
  auprc                          0.072
  auroc                          0.802
  detection rate (ever in time)  0.456
  detection rate (in window)     0.153
  median lead time               33.0 h
  alerts per patient-day         0.11
  precision among alerted        0.305
  guarantee                      held

Two detection rates, and the gap between them is the honest part. The generous one counts any alert that arrived in time, which on a long stay can mean an alarm that fired two days early. The strict one counts only alerts inside the window the utility score rewards.

6. Find out what it is actually reading

prodrome ablation

Refits the same model on restricted feature sets and prints a verdict. On this corpus it says the model is substantially reading the workup rather than the patient. See What the model reads.

7. Make it worse on purpose and watch the gate refuse

prodrome eval --out reports/good.json
prodrome gate --baseline reports/good.json --candidate reports/degraded.json

roll back: default@A against default@A
- utility dropped by 0.1500 (baseline 0.3338, candidate 0.1838); the allowed drop is 0.0000.

Exit code 1. Wire that into CI and a change that makes the system worse cannot be merged by accident. See Evaluate and gate.

Next