Skip to content

Causality

Every metric Prodrome reports is a claim about what would have been known at a particular hour. If one feature column is computed from a later row, the claim is false, the lead times are fiction, and no care taken downstream recovers it.

Leakage is the most dangerous class of bug in this kind of work because it does not raise. It just makes the numbers better.

The rule

The value of any feature at hour t is a function of hours start through t of that stay, and of nothing else. Not the stay's later rows, not another stay, not any statistic computed over the population.

The last clause is enforced by the type system rather than by discipline. A FeatureSet is stateless and fits nothing. Anything that needs a population statistic, such as a scaler or an imputation median, belongs to the RiskModel and is fitted on the training split alone. If feature building cannot see the population, it cannot leak the population, and the only remaining question is whether it can see a stay's future.

The test

That question is answered directly, by truncation.

  1. Build the features for a whole stay.
  2. Build them again for only its first k hours.
  3. Compare row for row and column for column.

If a single value differs, some feature read an hour that had not happened yet, and the test names the column and the first differing value. Seven truncation points are used on every stay in the fixture.

A second test rebuilds each stay alone and compares it against the same stay inside a crowd. That catches the other failure mode: a rolling window that runs off the start of a stay into the previous patient, which a truncation test on its own would never see. A third reorders the patients and checks nothing moved.

Proving the test works

A test that has never failed proves nothing. Both tests were verified by injecting a deliberate leak, one feature reading a single row ahead, and confirming they fail and name the column. The first attempt at that injection was itself instructive: the leak was written into a column that a later line overwrote, so the tests passed and the negative control was misplaced rather than the tests weak. That near miss is why the feature builder now refuses to write the same column twice.

The primitives

The window functions are written in numpy over contiguous stays rather than borrowed from pandas, where a group boundary is a convention rather than an assertion. They are small enough to read:

  • group_ffill carries the last observation forward and resets at each stay. A row before the first observation stays missing, because there is nothing to carry, and inventing a population mean there would be exactly the leak this module prevents.
  • group_shift looks backwards only and returns nothing when the stay is younger than the offset.
  • rolling_mean uses a cumulative sum with a reset, so a partial window at the start of a stay is a partial window rather than a window borrowed from the previous patient.

The corpus has one row per ICU hour with no gaps, which the parser checks, so shifting by k rows is shifting by k hours.

Where you can see it working

The websocket endpoint is the same guarantee made visible. The service holds the stay's history, appends the hour you just sent, rebuilds the features over the history it has, and answers. It has never been sent the next hour, because there is not one yet. See Serve it.