Sim-to-real: measured WiFi CSI validates the simulator's crowd-counting claim
Campaign: c-operanet-sim2real · Session: 01KY1WX4D7FREN8GV247Z70X04 · Data: OPERAnet exp028 (real, CC0, Bristol) — a furnished room, up to 6 people, one leaving every ~5 min, measured WiFi CSI.
Executive summary
Our simulations made a falsifiable claim: WiFi CSI can count a crowd in line-of-sight / open-plan space (the mall sim measured 0.15–0.31 nats of count information per access point), but fails behind walls (the S3DIS office campaign). We tested the positive half against real measured hardware CSI — the OPERAnet crowd-counting experiment. As people leave the room one by one, the real CSI's amplitude variance steps down monotonically with the count, and the measured count information is 0.174 nats — squarely inside the simulated band. The simulator is right where it claims to be right, which is exactly what makes its walled-building failure prediction trustworthy rather than a bug.
What we measured
OPERAnet's CSI is stored in a MATLAB v7.3 table whose per-frame labels are locked in an MCOS object h5py can't decode; but the experiment's protocol is published (6 people, one exits every ~5 min), so we reconstructed the count timeline (6→5→4→3→2→1) and read the raw CSI streams directly. Per ~1 s window we took a CSI amplitude-dispersion feature (log amplitude variance across links) — the measured analogue of the mean_amp_db feature the sim reduces — and computed its relationship to the (protocol-known) count.
Result
The real CSI perturbation drops in six clean steps as the room empties (figure, left): count 6 → amplitude-variance 0.69, 5 → 0.68, 4 → 0.65, 3 → 0.61, 2 → 0.55, 1 → 0.44 (log10). Monotonic. Spearman(count, variance) = 0.463; I(count; CSI-amplitude-variance) = 0.174 nats, CI [0.139, 0.208]. Mean amplitude alone was weak (0.04 nats) — the dispersion carries the count, consistent with the body-motion perturbation mechanism.
The simulator's open-plan band was 0.15–0.31 nats. The real number, 0.174, sits inside it — an independent cross-check (the sim and the real analysis share no parameters).
What it means
- The simulator is validated in its positive regime. Measured hardware CSI carries crowd-count information of the magnitude the sim predicted for LOS spaces.
- The c-public-spaces negative becomes a credible prediction. Because the sim matches reality where it says counting works, its claim that walls kill counting is a physical prediction to test on hardware — not a simulation artefact.
- Feature matters: counting rides on CSI variability, not mean level — a concrete design cue (use dispersion features).
Confidence & caveats
The count is protocol-reconstructed (equal 6-blocks); exact departure times are unknown, so the alignment is approximate — but the monotonic trend and MI are robust to modest boundary error. One experiment, one room, one feature: this validates direction and order-of-magnitude, not a calibrated detection rate. Real, public-domain data; the honest next step is parsing the true per-frame labels (MATLAB or the UWB CSV) and extending to more experiments.
Figure: fig_operanet_sim2real.png — real CSI variance vs time with count steps, and the monotonic count↔CSI relationship against the simulated band.