Participant phone timing, observations and recovery

Current assessment (2026-09-15)

The July implementation report does not certify the current app or its handset acceptance gates. Monitor-mode injection subsequently resolved the old infrastructure pacing limitation in EXP-011. Phone illumination is therefore an optional acquisition mode, while timestamped participant observations and interruption recovery remain necessary instrument questions.

Backgrounding, suspension and termination are distinct states; a compile or a background-mode declaration cannot establish continuous traffic in all three. Beacon observations do not guarantee background residency. A schedule or zone observation carries uncertainty and is not continuous headcount truth. The current quests use surveyed markers and the deployed fleet; the old demand for an urgent ESP32/iBeacon rollout is not a current dependency.

Next evidence needed: Follow the refreshed EXP-P1 acceptance path. Record a handset run across interruptions and closing clock alignment, then declare supported operating states from the observations. Compare optional phone illumination with the current monitor injector.

Basis: Thesis Narrative, the September 10 synthesis, and the September 15 register review. See the review record for evidence sources and outstanding checks.

Earlier protocol and dated results

The methods and observations below retain their original scope. Read their readiness statements and proposed next steps as historical. The current assessment above records what has changed; frozen thresholds have not been retrospectively revised.

Original hypothesis (before this review): A commodity smartphone — iOS first — running the MonadCount app while backgrounded or terminated, associated to a fleet AX210 AP and emitting a commanded-rate unicast UDP stream, delivers a uniform CSI sounding source at the observer node (inter-arrival CV ≲ 0.5 at ≥ 100 Hz) where the broadcast illuminator of EXP-P2 delivered only 30–72 % at CV 1.6–2.5; and the same backgrounded app witnesses surveyed iBeacon anchors well enough to carry zone-level ground truth without any foreground interaction.

Design constraints and what they cost

What we are trying to prove

Four falsifiable gates, reordered so that residency is proven before illumination. Each null becomes an inherited constraint rather than a failure.

• H1 — Background residency (iOS). An iPhone with the app backgrounded, screen off, in a pocket sustains a beacon-region session and an outbound UDP stream for ≥ 30 minutes with < 1 % emitted-packet loss and no gap in the anchor log longer than 30 s. Null: if iOS suspends the emitter despite an active location session, then a phone cannot be an illuminator in the field, and uniform illumination must come from a mains-powered injector node instead — a topology constraint discovered on the bench rather than during a field day.
• H2 — Uniform illumination. Under the H1 condition, a phone associated to monad01's AP and commanded to emit at rate R produces at monad02 a CSI record stream with inter-arrival CV ≲ 0.5 for R ∈ {25, 50, 100, 200} Hz, and delivered rate within 10 % of R — against EXP-P2's broadcast baseline of 30–72 % at CV 1.6–2.5 on the identical rig. Null: phone uplink as bursty as broadcast ⇒ no phone-carried source can serve EXP-007 Packet-Rate Floor for Motion Sensing.
• H3 — Clock. Phone-stamped events and csid records place on one timeline with residual offset < 5 ms over a 30-minute session — twenty times inside the EXP-P1 H2 sub-100 ms gate. Protocol specified below. Null: if a backgrounded phone's monotonic clock cannot be disciplined to that level, phone timestamps become intervals and the analysis may only use estimators robust to the measured width — written down before the field run, not after.
• H4 — Anchor legibility. A backgrounded (and separately, a terminated) iOS app receives region enter/exit events from surveyed iBeacon anchors within 30 s of the crossing, and receives RSSI samples at ≥ 0.2 Hz per in-range anchor while backgrounded. Null: if terminated-state relaunch is unreliable in practice, participants must at least launch the app once per session, and "terminated" drops out of the supported states.

A fifth item is a decision, not a gate: the app almost certainly does not need to advertise BLE at all. With anchors as iBeacons and the phone as witness, the counting direction reverses — infrastructure no longer counts phones, phones report which zone they are in. That removes the Android-only, iOS-hostile peripheral role from the design. Confirm by measurement in P3.8 before deleting the idea permanently.

Why iOS is the harder platform — and the concrete API map

Leading with iOS is the right call precisely because its background model is the binding constraint: anything that survives iOS backgrounded will survive Android with a foreground service, while the reverse is false.

Three of these are already in place on v0.2.0 — association, the entitlement, and the local-network purpose string — which is why iOS-first is cheaper than it looks. Everything in the "stay alive" and "see an anchor" rows is greenfield.

Clock discipline — the protocol (resolved)

The rule is: the collector is the clock master, the phone never rewrites its own timestamps, and the offset is stored as data.

Estimator. SNTP-style four-timestamp exchange over UDP to the collector, on the same socket and interface as the data stream:

• t1 — phone monotonic ns at send
• t2 — collector receive (its reference clock)
• t3 — collector send
• t4 — phone monotonic ns at receive

then offset = ((t2 − t1) + (t3 − t4)) / 2 and delay = (t4 − t1) − (t3 − t2). Fire a burst of ~20 exchanges and keep the sample with the minimum delay — the standard NTP/Cristian filter, and the reason this works at all on a jittery wireless link: the minimum-delay sample is the one least contaminated by queueing, and on a local Wi-Fi link it lands in the low single-digit milliseconds.

Skew. Run the burst at session start, midpoint and end. A linear fit of offset against monotonic time gives skew in ppm and lets the analysis interpolate the offset at any instant rather than step it.

The data stream is the sync stream. Every emitted data packet already carries {session_id, seq, t_mono_ns}. The collector stamps arrival with the reference clock. That means offset drift is continuously observable for free over the whole session — the periodic bursts are only there to bootstrap and to bound the estimate; the packet stream refines it. This also gives the delivered-rate timeline H2 needs from the same records. One mechanism, three products.

What is stored. In the session sidecar: {boot_id, t_mono_ns, offset_ns, rtt_ns, skew_ppm} per exchange, plus the platform clock source used. Every phone-side event carries t_mono_ns and boot_id; wall-clock is recorded alongside but is never load-bearing. Analysis applies the offset, so the correction is auditable and reversible — the same discipline csid already follows on the observer side.

Hazards.

• Sleep. Use the sleep-continuous monotonic clock on each platform (CLOCK_MONOTONIC_RAW on Darwin, elapsedRealtimeNanos() on Android). The uptime-style clocks that stop during sleep would silently compress a backgrounded session's timeline.
• Reboot. A boot_id (or process-start UUID) invalidates continuity across restarts; a session that spans a reboot is two sessions.
• Interface change. If the phone leaves the experiment AP, the offset estimate to the collector is no longer over the same path. Record interface transitions as session events and re-run the burst after each.

Acceptance for H3: residual |offset| < 5 ms at the session midpoint when checked against a held-out burst, and skew stable within ± 20 ppm across the session.

What the instrument is now (built 2026-07-27)

The card's original audit — of v0.2.0, before any of this work — is preserved in the git history of repos/monad-app; what follows is the state after the build. Built is not measured. Everything below compiles and, for the API, boots; nothing below has produced a reading from a radio.

What was removed rather than added

The build was a replacement, not a layer. Deleted as superseded: the fire-and-forget UdpService/UdpSender POC, the wifi_test_v2 screens, BleSensingService + BleAdvertisementRepository + the BleAdvertisementRecord table (migration 5 drops it), BleDataExportService, and QuestData{Export,Flush}Service. The export → upload → flush-anyway sequence had been written three times — in the active-quest model and both end screens — and the copies had drifted; all three deleted local data on paths where the upload had failed. It now exists once, in QuestSessionCoordinator, with the rule inverted to upload-then-delete.

Two build defects were fixed on the way, both of which made the repository unbuildable or dishonest: the Firebase plugins were applied unconditionally, so a fresh clone failed at :processDebugGoogleServices before compiling a line of Kotlin (now opt-in on google-services.json); and minSdk was 24 while the Wi-Fi code was already @RequiresApi(Q), promising a compatibility that never existed (now 29).

The collection side (repos/monad-backend)

The instrument is not only the phone. The API that serves the lab bundle and accepts sessions was migrated in the same pass:

• Storage moved to the project's Hetzner bucket, and AWS was removed from every layer. Not merely repointed: the AWS SDK is gone (replaced by async-aws/s3, two focused packages instead of a ~400-service monolith), the AWS Terraform for EC2/ECR is deleted, and the environment variables are the same HETZNER_S3_* names monad-knowledge already uses, so one ansible-vault secret set now covers the fleet captures, the simulation artefacts and the phone sessions alike. The endpoint is required — an absent endpoint previously meant "talk to Amazon", which is the class of default that ships research data to the wrong provider unnoticed. Path-style addressing is mandatory, since Hetzner issues no wildcard certificate for virtual-hosted bucket names.
• Key layout replaced. The old experiments/{y}/{m}/{d}/{userId}/{enrollmentId}/ partitioned by upload date: a retry across midnight split one session across prefixes, and nothing could be joined to a csid capture, which is addressed by session. Now datasets/monad-app-sessions/{participant}/{session}/.
• GET /api/lab/config serves the operator-authored bundle (a file, not a Doctrine entity — it describes physical reality and is edited next to the hardware).
• Observability. The container exports OTLP to Alloy on the existing monad network (IP-051 LGTM). Beyond HTTP/Doctrine spans, each uploaded sidecar becomes instrument metrics mirroring these gates: monad.lab.traffic.interval_cv, monad.lab.traffic.delivered, monad.lab.clock.offset_ms, and monad.lab.session.unpinned — the count of sessions whose socket was not pinned, i.e. the silent failure this design exists to prevent. A non-zero rate there invalidates the runs it covers.
• Deployment as api.monad.dubec.dev on the project host, behind the nginx that already serves monad.dubec.dev. Production runtime changed from symfony server:start (a development server) to FrankenPHP, and /api/health was made public — it had required a JWT, so every healthcheck read as "down". Host recon then caught three more: the container's port collided with Headscale (moved to 8084), the vhost used an http2 on; directive this host's nginx 1.24 rejects, and the ACME challenge root was wrong.

The collector — collectord (repos/csid/crates/collector)

The receiving end, without which none of H2 or H3 can produce a number: the phone knows what it commanded and handed to the kernel, but only the collector knows what actually arrived and when. A fourth crate in the csid workspace, so it inherits that repository's config idiom, session shape, sync machinery, systemd conventions and Ansible role rather than growing a second set.

• Kernel arrival timestamps (SO_TIMESTAMPNS). This is why it is a Rust service on the node and not a script somewhere convenient: a userspace stamp taken after the scheduler wakes the process carries jitter of the same order as the quantity being measured. Whether kernel stamping was active is recorded per session, so a userspace-stamped run can never be silently pooled with the others.
• Answers the four-timestamp exchange using the kernel arrival stamp as t2, and echoes the request's sequence number — the phone rejects a mismatched reply, because a stale one looks fast and would bias its minimum-delay filter the wrong way.
• Sessions close on silence. A phone never announces the end of a session; it runs out of battery or walks out of range. After an idle timeout the sidecar flips receiving → complete, which is what makes it shippable, to datasets/monad-app-sessions/<participant>/<session>/collector/ — beside the phone's own artefacts for the same link.
• Protocol addition: SESSION_HELLO, sent at session start and repeated with every clock burst, so the collector learns the participant without widening every packet at 200 Hz.

Verified end-to-end against synthetic phone traffic: session opened and identified from the announcement, 5/5 clock exchanges answered with echoed sequence numbers and valid reference stamps, 197 packets recorded, and three deliberately-skipped sequence numbers correctly reported as lost: 3. Not yet run on monad01.

Deployment state (2026-07-27)

Both halves are authored and locally verified; neither is on the hardware yet, and the order matters.

The API is deployed (2026-07-27, ansible-playbook playbooks/monad-api.yml). api.monad.dubec.dev serves /api/health as {"status":"healthy","database":true} over a Let's Encrypt certificate; the lab bundle endpoint is live and correctly requires a token. Runbook, preflight gates and the user-creation commands are in infrastructure.

Two check-mode defects surfaced during the dry run and were fixed rather than worked around, both the same class: --check reports template tasks as changed without writing anything, so everything downstream that acts on what they would have written — a symlink source, a chdir target, a notified handler — has to be skipped in check mode or the dry run dies on the absence of something a real run would own. The role was then audited for the whole class instead of the two instances.

The collector still waits. csid@drift-overnight-illum is capturing on monad02 right now, fed by the illuminator on monad01. The fleet role will not restart a running capture — that handler deliberately only warns, and a running process keeps its old inode — but adding crates/collector to the role's hash paths triggers an on-node cargo build --release, and a 4-core Pi compiling while a netlink RX loop and a 25 Hz paced sender are running is contention with exactly the quantities the drift arm measures. Deploy after it closes:

ansible-playbook playbooks/fleet.yml --limit monad01 --tags csid -e csid_build_jobs=1

--limit monad01 keeps the play off the capture node entirely.

Two facts settled by reading the live fleet rather than assuming:

• The collector's address is 10.99.0.1:9998 — monad01's wlp1s0 on the experiment AP subnet, which is the only address a phone joined to monad-illum-24 can reach, since that SSID routes nowhere else. It must be a literal IPv4: the phone pins its socket to the Wi-Fi interface, where no DNS answers.
• Port 9998, not 9999. The illuminator broadcasts to 10.99.0.255:9999; nothing listens there, so the collector would have bound successfully and then ingested 25 Hz of illuminator frames as malformed datagrams.

The plan

Status after the 2026-07-27 build: the software steps (P3.1–P3.4, P3.6, and the console) are written and compile; the steps that need hardware or firmware (P3.0, P3.5, and the measurement halves of P3.2) are open. Ordered so the binding constraint is tested first and the expensive steps are gated by cheap ones. iOS actual first in every step; Android actual follows.

P3.0 — Anchor identity: switch to iBeacon (1–2 days, ESP32 side). ⬜ NOT STARTED — now the single blocker for witnessing. Reflash the _knowledge/hardware/esp32-c6 anchors to advertise an iBeacon frame with a lab UUID, major = floor/zone, minor = anchor index, at a fixed advertising interval, and survey their positions into PostGIS. This is now on the critical path for iOS witnessing, and it also cleans up the vault story — an anchor's identity becomes a surveyed record rather than a device name. Acceptance: iOS CLLocationManager ranges all anchors in a room; Android Kable still reads them (iBeacon is ordinary manufacturer data there).

P3.1 — Pin the socket to the AP. ✅ BUILT, unverified on hardware. iOS: NWConnection with NWParameters.requiredInterfaceType = .wifi, as a Kotlin expect/actual (ktor's shared socket cannot express interface pinning). Android: Network.bindSocket(). Acceptance: a datagram sent while joined to monad-illum-24 appears in tcpdump on monad01 with cellular data enabled — that is the whole test.

P3.2 — Background residency. ✅ BUILT / ⬜ UNMEASURED (H1, H4). UIBackgroundModes: location + Always authorization + a CLBeaconRegion monitoring and ranging session; bluetooth-central and CoreBluetooth state restoration if a CoreBluetooth path is also wanted. Android: foreground service + wake lock + battery-optimisation exemption prompt. Acceptance: H1 and H4 measured on the operator's iPhone — 30 minutes, screen off, in a pocket, with a log the app itself writes.

P3.3 — TrafficGenerator. ✅ BUILT. Commanded rate, payload size, duration; monotonic pacing with drift correction (schedule the n-th packet at t0 + n/R, never delay(1/R) in a loop); per-packet header {session_id, seq, t_mono_ns, boot_id}; local log of commanded-vs-actual send times. Runs inside the P3.2 background session. Acceptance: on-phone CV of send times ≲ 0.2 at 100 Hz while backgrounded.

P3.4 — Clock. ✅ BUILT. The protocol above: SNTP burst at start/mid/end on the same socket, offset stored not applied, continuous refinement from the data stream. Acceptance: H3.

P3.5 — H2 measurement on the EXP-P2 rig (one afternoon). ⬜ THE NEXT THING TO DO. iPhone joined to monad01, csid capturing on monad02, rate ladder 25/50/100/200 Hz × 5 minutes, phone backgrounded and stationary. Compare delivered rate and inter-arrival CV against the EXP-P2 broadcast baseline on the identical rig. This is the experiment that decides whether the phone is an instrument or merely a subject.

P3.6 — Session durability + sidecar. ✅ BUILT, both ends. Upload-then-delete: local data flushed only after the server confirms receipt, with an explicit "unsynced sessions" screen. Emit a metadata.json mirroring the CSIQ session block (identity / radio / environment / lifecycle / summary) to datasets/monad-app-sessions/<participant>/<session>/ so a phone session is a first-class source alongside csid:<host>/<session> (zhang2026_ccac ↗ — a phone is the most device-dependent radio in the room, so its provenance block matters most).

P3.7 — Make the schedule executable. ⬜ OPEN — the three stub step types are still stubs. Implement connect_to_ap (join + verify + start the generator), walk_to (target cell from PostGIS, arrival confirmed by beacon-region entry), and a new emit_traffic step. At that point the quest engine is the field-experiment scheduler and an experiment protocol is a database row, not a person with a stopwatch.

P3.8 — Decide phone-side BLE advertising by measurement. ⬜ OPEN. With P3.0–P3.7 done, check whether beacon-region witnessing meets EXP-F1's ± 1 person / 90 %-of-windows target on a five-person dry run. Build advertising only if it does not — the expectation is that it never gets built.

Scale-up path — two people to a hundred students

The user-facing arc is also a methodological one: available truth degrades as the crowd grows, and the design must change with it.

The design consequence of S2 is worth stating early: with 100 unscripted students you will never know the true headcount continuously, and you should stop trying. You will have a large set of exactly-known partial observations — "at 13:42:07 participant 31 entered zone major=3, minor=7" — plus a continuous CSI record. That is a semi-supervised setting, and it matches how the thesis's calibration story actually works: a few trustworthy anchors correcting a drifting continuous model, which is the periodic-calibration hypothesis restated in the truth domain rather than the signal domain.

Gamification should therefore be tuned to produce anchors, not engagement for its own sake: reward zone coverage, reward off-peak visits (when drift data is scarcest), reward returning on consecutive days. And because the app is backgrounded, most of this accrues passively — the student opens the app to see a leaderboard, not to collect data.

Privacy and ethics — decide now, not at S2

Three obligations become formal the moment the cohort stops being colleagues, and all three have a code consequence:

1. Scan filtering is a privacy control. Today the allow-list is device names; after P3.0 it becomes a beacon UUID. Either way, the app must record only lab anchors — a participant's phone must never become a passive collector of bystanders' device addresses in a public university building. Enforce it at the schema, not in a filter a future refactor can drop.
2. Pseudonymous participant IDs. The upload path is keyed by enrolment ID; nothing downstream should join that to an account e-mail in stored data. The account exists for the game; the dataset carries only the pseudonym.
3. Consent and withdrawal. A student who deletes the app must be able to have their sessions removed — which requires sessions findable by pseudonym, which the P3.6 S3 layout gives for free and an ad-hoc layout would not.

Worth a short written protocol before S2 and an ethics check on what FIIT STU requires for a study of this shape.

On accelerating the ESP32 work

Asked directly, and the answer changed with the iOS-first constraint. Split it:

• As surveyed iBeacon anchors — yes, and now urgently. This moved from "nice, cheap" to P3.0, on the critical path: a backgrounded iOS app cannot witness anything else, and it is the mechanism that keeps the app alive for the illuminator role. It is also modest work — advertising firmware, a UUID scheme, and a PostGIS survey. Check first whether the AX210's onboard Bluetooth 5.3 is reachable on the Pi 5 M.2 HAT carriers (_knowledge/hardware/intel-ax210 notes BT is "a separate USB endpoint on most carriers"); if it is, the fleet can beacon and the ESP32 anchor role becomes redundant.
• As distributed CSI receivers — still not urgent. The receiver-density question is real, but the AX210 nodes already produce richer CSI than an ESP32-C5 will, and adding a second extractor toolchain before the first has produced a field dataset multiplies the debugging surface at the wrong moment.

Key references

• cominelli2023_e6ee ↗ — why an associated-station HE link is worth chasing: subcarrier count and antenna configuration dominate what the channel can reveal, and both improve when the frame is unicast rather than group-addressed.
• zhang2026_ccac ↗ — device-dependent distortion makes raw measurements incomparable without recorded provenance; the argument for the P3.6 sidecar.
• zeng2021_1e4f ↗ — the cross-chain ratio that makes a 2×2 uplink worth more than twice a 2×1 broadcast.
• longo2019_b72f ↗ · demrozi2021_bf55 ↗ — the BLE-counting baseline this card deliberately inverts: phones report zones rather than infrastructure counting phones.
• sadowski2020_f2ba ↗ — the path-loss envelope (n ∈ [1.8, 3.5], σ ∈ [3, 7] dB) any phone-side RSSI must be sanity-checked against.

Related

• Wireless Laboratory Framework — the role contracts this card implements (illuminator + witness on one device)
• EXP-P1 — sensing plane; its sub-100 ms sync gate is inherited and tightened here
• EXP-P2 — the measurement that motivates this card, and the rig P3.5 reuses
• EXP-007 Packet-Rate Floor for Motion Sensing · EXP-008 Active Injection vs Passive Sniffing — the two sampling experiments a paced phone source makes executable on hardware
• EXP-F1 · EXP-F2 · EXP-F3 — the field series gated by this bring-up
• _knowledge/hardware/mobile-app — instrument note, corrected against this audit