Event-driven recalibration — can a cross-modal drift statistic beat a fixed clock?

Executive summary

A Wi-Fi crowd-counting model does not stay accurate: move the furniture, and the radio "fingerprint" it learned drifts until the count is wrong. This experiment asks whether a cheap, always-on signal can tell you when the model has actually broken — so you recalibrate on demand instead of on a fixed timer. The signal we test is cross-modal disagreement (xmodal): the gap between what the Wi-Fi channel thinks the occupancy is and what a second, independently-calibrated BLE (Bluetooth) channel thinks. When the two physically-different radios start disagreeing, something real has changed.

The headline, stated honestly: the disagreement signal detects prediction-relevant drift very well (Spearman ρ = 0.93, 95% CI [0.72, 1.0]) and stays quiet when the room merely changes without breaking the model. But acting on it does not make the count more accurate than a fixed recalibration schedule — the payoff is budget: it reaches the same accuracy while recalibrating about four times less often across an estate where most zones never drift. Detectability is confirmed in-silico; the "beats the clock on accuracy" claim is refuted. All of this is simulated — a real Wi-Fi/BLE capture (IP-106) is the standing gate before any deployment claim.

The problem, in plain words

Imagine you calibrate a bathroom scale while standing on a soft carpet. Later someone swaps the carpet for tile. The scale now reads wrong — not because your weight changed, but because its environment did. Wi-Fi crowd counting has the same disease: the model learns "this pattern of signal strength ≈ 5 people," but the pattern depends on where the walls, furniture, and reflective surfaces are. Rearrange the room and the model silently drifts. This is a textbook case of concept drift — the input–output relationship changing over time — which is a whole research field of its own (gama2014 ↗).

The naive fix is to recalibrate on a timer ("every week, re-measure with a known number of people"). That works, but it is expensive and wasteful: most rooms are stable most of the time, so you pay for recalibrations you did not need, and you can still be caught out between timer ticks. The smarter idea — an event-driven trigger — only recalibrates when a monitor says the model has actually broken. The hard part is building that monitor without ground truth: at runtime you do not know the real headcount, so you cannot directly measure your own error.

What we are trying to prove

• Hypothesis (falsifiable): the cross-modal disagreement xmodal — computable online with no ground truth — ranks time-periods by true occupancy error (high Spearman ρ) and stays below threshold on periods where the environment shifted but the model did not break. If it fires on those error-irrelevant shifts, it is a naive change-detector, not a drift-relevant trigger, and the idea fails.
• Second, sceptical hypothesis: acting on the trigger lowers count error versus a fixed cadence at the same recalibration budget. We expected this to be the weak claim, and it is — see results.
• What a null means: if xmodal did not out-track single-modality detectors, it would mean the BLE channel adds nothing as a drift reference and the fusion premise weakens. It does out-track them — the value of BLE here is not as a people-counter (it is weak at that too) but as a second, differently-drifting physical witness.

How the experiment works (plain method)

1. Build a controlled drift corpus (csi-layout-drift): a synthetic corridor is ray-traced under ordered furniture displacements, with a deliberate split — a losin arm where displacement genuinely grows count error, and a losperp arm where the signal amplitude shifts ~27 dB but the count error stays flat (environment change without model breakage). Co-registered BLE is solved from the same scene.
2. Compute four candidate drift monitors online: csi-ks (Wi-Fi self-monitoring), ble-ks (BLE channel shift), xmodal (the cross-modal disagreement), and an idealised oracle-counter ceiling.
3. Detectability test: does each monitor's value rank-correlate with the true occupancy error, and does it stay quiet on the error-irrelevant arm?
4. Policy test: simulate deployment — over the drifted states, compare "never recalibrate" / "fixed cadence" / "trigger[xmodal]" / "oracle-error" on both error and recalibration spend, at matched budget.

What we've found so far (honest, across campaigns)

Policy at matched budget (signal arm): fixed cadence 0.399 error / 8.0 estate-wide recals; trigger[xmodal] 0.407 / 2.0 recals. Even the perfect-knowledge trigger (0.421) does not beat fixed cadence on error — because recalibrating on schedule also fixes below-threshold drift the trigger deliberately ignores. So: detectability confirmed, accuracy-policy refuted, budget-parity is the real result.

How to read the figures

• fig_drift_trigger_detect — four scatter panels (one per monitor), statistic on x, true occupancy error on y, with the per-panel Spearman ρ in the title. A good monitor makes the cloud rise left-to-right; note xmodal's tight rise vs csi-ks's scatter. n = 18 cells/arm, so read the rank trend, not individual points.
• fig_drift_trigger_policy — the small-multiples bar chart of error and recals per policy; the honest reading is overlapping error bars between trigger and fixed, with the recal-count bar far lower for the trigger.
• fig_drift_trigger_curves — per-arm displacement→error curves; the seed spread is the story (seed-1 placements happen to be drift-robust; the big excursion is seed-0).

Review panel

Each voice is a prepared expert with a one-line stance and the literature it argues from. Verdicts are about this experiment and its current evidence, not the idea in the abstract.

Optional further voice for future rounds — 🛡️ Responsible-Sensing / Ethics reviewer (privacy of device-free occupancy, consent, dual-use), grounded in renet-responsible-sensing. Not yet run for this card.

Key references

• gama2014 ↗ — the drift-detection framing this whole experiment inherits.
• jung2025 — real multi-zone CSI occupancy; grounds the drift-on-layout premise.
• zou2018 ↗, zou2017 — device-free CSI counting baselines.
• demrozi2021 ↗ — BLE occupancy, the second modality's real-world basis.
• zhang2026 — the reproducibility bar the SWE voice invokes.
• huang2025 — a public real-CSI anchor for the red-team's cross-check demand.