Sim-to-real count transfer — does synthetic-impaired CSI pretraining beat real-only at a small real budget?

Executive summary

The dream behind the whole simulation programme is simple: train a Wi-Fi crowd counter on cheap synthetic data, then make it work in a real building with only a handful of real measurements. If a ray-traced simulator can manufacture the "how does a crowd bend the radio channel in this room" structure that a small real dataset is too sparse to contain, then synthetic pretraining should beat real-only training at the same tiny real budget. This card is the platform's honest test of that dream — deliberately gated on a public real-CSI cross-check, because the campaign-rigor audit's top recommendation was "prove it zero-shot on a public dataset before any transfer claim travels."

The headline, stated honestly, in two parts:

1. The count-transfer test itself was never run. Its campaign (c-sim2real-count-transfer) has two sessions; both sealed with zero runs, blocked at preflight. The 24-cell synthetic corpus was never generated (one probe cell at most), and the transfer reduction that computes "sim-pretrain vs real-only at k-shot" existed only as a self-tested harness on planted synthetic data. So the central hypothesis is open and untested — a wiring gap, not a scientific result.

2. A sibling calibration probe did run (c-operanet-sim2real-fingerprint) and it delivers a real-anchored, cautionary finding: the ray-tracer over-codes the environment. Synthetic CSI identifies which floor you are on at 0.98 where real OPERAnet CSI identifies the room at 0.83, and a synthetic count model transfers across floors at 0.57 retention where a real one keeps 0.73. The sim's rooms are too distinguishable — so a model pretrained on it would read optimistically and generalise worse than reality, not better. Crucially, hardware-noise impairment did not fix this (it is structural geometry, not a noise floor); only scene diversity (furniture/material variation) softened the synthetic fingerprint toward the real 0.83.

Integrity caveat up front: the c-operanet-sim2real-fingerprint numbers live in the campaign note as a local fan-out (2026-07-13); there is no sealed S3 session for that campaign (sim_campaign_sessions returns none). They are the operator's recorded results, with the operator's own wide-CI caveats (2 floors, 8 runs/floor, leave-one-run-out fold std 0.30-0.47) — directional, not audited through the session-manifest chain. Read them as a strong hint, not a closed result. Everything here is simulation against public real targets; a first-party real Wi-Fi/BLE capture (IP-106) is the standing gate.

The problem, in plain words

Suppose you want to teach a computer to count people in a room from nothing but the Wi-Fi signal bouncing around it. To learn well it needs lots of labelled examples — "this wobble in the signal means 3 people, this one means 7" — across many different rooms, because the wobble depends heavily on where the walls, furniture and metal cabinets are. Collecting that real, labelled data is slow and expensive: someone has to stand in the room with a clipboard counting heads.

The tempting shortcut is a simulator: a physics engine that ray-traces radio waves through a 3-D model of a building with virtual people walking through it, spitting out synthetic Wi-Fi channel measurements (CSI) for free, in any quantity, perfectly labelled. If we pretrain on a mountain of that synthetic data and then fine-tune on just a few real measurements, maybe we get a great counter for almost no real-world effort. That is the classic sim-to-real promise, and it is exactly the promise domain-adaptation research keeps testing across wireless sensing (chen2023_5cbd; wang2026_2758).

The catch — the reason this needs a hard test rather than a hopeful assumption — is the reality gap. A simulator is only as honest as its physics and its 3-D scenes. If its rooms are cleaner, more geometric, and more distinct than real rooms, a model trained on it learns to lean on cues that do not exist in the messy real world. It will look brilliant in simulation and fall over on the first real Pi. So the only trustworthy way to judge synthetic pretraining is to score it against real, public measurements — here OPERAnet (bocus2022_ce7f ↗) and WiMANS (huang2025_060d).

What we are trying to prove

• Primary hypothesis (falsifiable): a crowd counter pretrained on synthetic-impaired CSI and fine-tuned on a small real budget (k = 5/10/20 examples per count) achieves lower held-out real count error than a real-only model trained on the same small budget — measured leave-one-environment-out on WiMANS. If it does not beat real-only, the ray-tracer's occupancy structure is not real-representative and the sim programme's licensing premise weakens.
• The honesty gate (non-negotiable): always report zero-shot (sim-only → real, no fine-tuning) and the clean-vs-impaired ablation, so any gain is attributed to genuine transfer rather than to the Doppler representation alone. That representation already reaches LOEO R² ≈ 0.47 on real WiMANS with no sim at all — so a naive "look, it works" is not evidence of transfer.
• What a null means: if synthetic pretraining does not help at small k, the finding is that our ray-traced occupancy structure differs from real occupancy structure in a way fine-tuning cannot repair — which is itself a publishable, defect-localising result (it points the fix at scene diversity, materials, or the body-scatterer prior, not at "more data").

How the experiment works (plain method)

The card spans two coupled campaigns — one that makes and audits the synthetic data, and one that tests the transfer:

1. Fingerprint / calibration probe (c-operanet-sim2real-fingerprint): simulate the same occupancy signal across several distinct real-building floor geometries (ResPlan floors), then ask three real-anchored questions. (a) Can a classifier tell the floors apart from synthetic CSI — and is that separability comparable to the real OPERAnet room-ID of 0.83, or does the sim over-code it? (b) Does a synthetic occupancy model trained on one floor still work on another (cross-floor retention), the way the real one keeps 0.73 across rooms? (c) Which knob — hardware-noise impairment or scene diversity — moves the synthetic fingerprint toward reality? This is framed as a calibration probe, not a hypothesis test: a match validates the sim as a sim-to-real substrate; a mismatch tells us exactly what to fix.
2. Transfer test (c-sim2real-count-transfer): generate a class-balanced synthetic corpus (JuPedSim crowd → Sionna ray-traced CSI, 0–5 agents × 4 seeds, with a commodity-NIC impairment layer so it is not unrealistically clean), turn each run into a Doppler spectrogram, and then compare, on held-out real WiMANS: real-only(k) vs synthetic-pretrain-then-k-shot vs zero-shot vs real-full. The claim lands only if synthetic-pretrain beats real-only at small k, after the honesty gate rules out the representation-only explanation.

What we've found so far (honest, across campaigns)

The transfer test never executed (c-sim2real-count-transfer)

Both sessions of this campaign sealed with zero runs launched, zero CPU-hours, zero tokens spent on compute — blocked at the mandated preflight gate, correctly and cheaply, before orphaning an unsupervisable sweep:

So: the impairment contract is verified on a single cell, the reduction harness is authored and self-tests on planted synthetic data, and the working grid is recorded for a hand-off. None of the three success criteria (corpus, transfer effect, honesty gate) is resolved, and the central hypothesis has never touched real data. This is a wiring gap, not a negative result — but it must not be reported as anything more than "designed and unblocked, not run."

The fingerprint probe ran and warns us (c-operanet-sim2real-fingerprint)

Recorded from a local deterministic fan-out (31 static + coupled crowd runs), in the campaign note, without a sealed S3 session — treat as directional:

Three things the probe actually taught us, each with a real anchor:

1. The sim over-codes the environment. Synthetic floor-ID (0.98) beats the real room-ID (0.83): the ray-tracer's rooms are too distinguishable because they lack real nuisance (hardware drift, furniture change, temporal variation). The direction matches reality (environment is the hard generalisation axis) but the magnitude is optimistic — a model pretrained on this would over-fit the environment cue.
2. Occupancy needs motion, and even then transfers worse than real. A static placement snapshot carries no count signal (0.13, at chance); the coupled JuPedSim→Sionna temporal chain recovers it (0.60), mirroring the real OPERAnet lesson that activity/occupancy lives in temporal dynamics. But cross-floor retention is 0.57 vs the real 0.73 — a steeper drop, exactly what over-coding predicts.
3. Hardware noise is the wrong knob; scene diversity is the right one. Adding a commodity-NIC impairment layer (SNR 20 dB, CFO, STO, IQ, phase noise) left the fingerprint at ~1.0 and degraded occupancy retention (0.71 → 0.62) — moving away from real. Randomising furniture/material per run dropped synthetic floor-ID from ~1.0 toward 0.67. The over-coding is structural geometry, not a noise floor. This reframes the exp-csi-calibration refit as a scene-diversity fit, not a noise-model fit — a concrete, useful outcome.

Net: the one arm that ran does not confirm the transfer dream — it produces a red flag for it (the sim is too clean/too distinct), plus a specific fix. That is a healthy, honest state for a sim-to-real card to be in.

How to read the figures

These figures are named and described in the c-operanet-sim2real-fingerprint campaign note; they are not attached to a sealed session, so read them as the operator's recorded outputs, not session-audited artefacts.

• operanet_sim2real.png — two bars: (a) environment-ID accuracy, real OPERAnet room-ID (0.83) vs synthetic floor-ID, with the chance line; the synthetic bar rising above the real one is the "over-codes" story. (b) synthetic occupancy sitting at the chance line in the static config — the null that motivated the coupled re-run.
• operanet_crossfloor_graded.png — within-floor occupancy (0.60 and 0.72, both ≫ chance) vs cross-floor (0.37) and the derived retention 0.57, against the real 0.73 reference line. The honest reading is the gap below 0.73, not the point value — fold std ≈ 0.30-0.39 on 2 floors.
• operanet_scene_diversity.png — synthetic floor-ID under empty scene (~1.0), + impairment (~1.0, no change), and + metal-furniture diversity (0.67), against the real 0.83. The message is which bar moves: noise does not, scene diversity does.

There are no figures for the transfer test — it did not run.

Review panel

Each voice is a prepared expert with a one-line stance and the literature it argues from. Verdicts are about this experiment and its current evidence — one arm producing a cautionary fingerprint, one arm unexecuted — not about the idea in the abstract.

Key references

• chen2023_5cbd — the domain-gap framing this whole card inherits; environment as the hard axis.
• wang2026_2758 — where and why sims flatter cross-environment generalisation.
• bocus2022_ce7f ↗ — the real anchor: room-ID 0.83, cross-room retention 0.73.
• huang2025_060d — the real count anchor for the (unrun) transfer test.
• hou2023_bf83, cha2025_cd6e ↗ — the few-shot real-only baselines the transfer must beat.
• vishwakarma2021_fe49 ↗ — sim-augments-real precedent (radar), and the warning that a learned bridge may be needed.
• jiang2018_77f6 ↗ — separating environment-invariant signal from nuisance (identifiability).
• brunello2025_d781 — how easily CSI metrics drift, inflating apparent effects (statistician).
• cominelli2023_e6ee, xie2015_0389 ↗, gringoli2019_68e7 ↗ — real commodity-hardware CSI structure and extraction (field / deployment).
• zhang2026_ccac, guarino2026_e72c ↗ — the reproducibility bar the SWE voice invokes against the broken artifact chain.