The report

One JSON file per configuration per test site. It is the unit the gate compares and the unit the leaderboard prints, so it carries everything needed to decide whether a change was an improvement, plus the provenance needed to know whether the comparison is even legitimate.

{
  "prodrome_version": "0.1.0",
  "config_name": "default",
  "config_hash": "9169351c8f363d78",
  "corpus_sha256": "ef46d3e6ec2497f7...",
  "split_seed": 7,
  "train_sites": ["A"],
  "test_site": "B",
  "split": "test",
  "n_patients": 20000,
  "n_hours": 761995,
  "n_septic_patients": 1142,
  "utility": {...}, "discrimination": {...}, "timing": {...},
  "calibration": {...}, "operational": {...}, "guarantee": {...},
  "subgroups": [...], "notes": {...}
}

Provenance

Field	Why it is there
corpus_sha256	the gate refuses to compare reports that disagree on it
split_seed	two seeds scored different patients
train_sites, test_site	external is derived from them, and that is the result that counts
config_hash	covers only fields that change a number, so a rename is not a new experiment
notes.environment	system, architecture, Python, CPU count; timings only compare within one kind of machine

utility

The challenge score, plus the four totals it normalises, because the ratio hides how much of the ceiling was reachable and a report that cannot show its working is one nobody can check.

Field	Meaning
normalized	1 is alerting through the whole beneficial window, 0 is never alerting
observed, best, inaction, worst	the raw totals

A negative normalised score means the alerts cost more than they were worth.

timing

Two detection rates, and the gap between them is the honest part.

Field	Meaning
detection_rate	septic stays ever alerted at or before the late cutoff. Generous: on a long stay this can be an alarm that fired two days early
detection_rate_in_window	alerted inside the window the utility score rewards. Strict, and the one to quote
lead_time_median_h, _q25_h, _q75_h	measured from the recorded clinical onset, which sits six hours after the label turns on

On the measured data the two rates differ by a factor of three.

operational

What the ward experiences.

Field	Meaning
alerts_per_patient_day	rising edges, not alerting hours: a ward sees one alarm and then a flagged patient
share_patients_alerted	how much of the unit gets flagged at all
precision_among_alerted_patients	the quantity the guarantee bounds

guarantee

Field	Meaning
stated	the promise in one sentence
attainable	false means no threshold could be certified, so the system alerts on nothing
precision_held, budget_held	whether it survived on these patients. Both null when nothing was alerted, because a vacuous true would be a lie

notes.threshold_source

calibrated, or alerts on nothing (no threshold could be certified).

Where they live

reports/<config>-site<X>-<timestamp>.json and reports/latest/<config>-site<X>.json. The site is part of the name, so two sites of one configuration cannot overwrite each other's baseline. The timestamped ones are git-ignored and the latest ones are committed, so the repository carries the history of quality without carrying every run.