W03 - Capturing CSI: records and bytes

Week 2 ended with a tensor: tones by chains by time. This week opens the file that tensor came from. The card estimates the channel for its own equaliser and discards it, so every toolchain in this week is some way of catching that estimate on its way out. What arrives is a stream of records: a fixed header followed by a payload of integers. Every step from radio to file can corrupt, drop or relabel a record, and most corruptions leave a clean-looking array.

The companion deck is primary. It carries one live widget per section below, and each widget at its default state shows the number this note quotes. The workbook workbook.ipynb computes every number here with numpy only.

All simulated states in this week are rebuilt from a seed with a 32-bit linear congruential generator and Box-Muller normals, so a widget and the workbook agree to the printed digits.

1. From radio to file

Idea. A record passes eight stages: transmitter, air, NIC firmware, driver, socket, logger, file, parser. Each can change it.

The driver hands CSI to the socket that registered for it. A second program that wants the same radio's CSI cannot share it, so a running capture cannot be watched by a second tool on the same radio. Plan for one consumer per radio.

Derivation: the record counter audit. The driver stamps each record it delivers with a counter that increases by 1 and is one byte wide, so it wraps at 256. For consecutive stored counters a and b let

Then d = 1 is clean, d = 0 is a duplicate, and d > 1 hides d - 1 missing records.

In words: the counter turns silent loss into a visible gap, but only while the hole is shorter than one period. A hole of h records gives d = (h + 1) \bmod 256, so holes of up to 254 records are measured exactly, a hole of 255 gives d = 0 and reads as a duplicate, and longer holes alias.

Worked numbers (workbook section 1, seed 3). A logger receives 1000 records and loses each with probability 0.02 and duplicates each survivor with probability 0.005. It truly dropped 27 and stored 979 records. The audit finds 27 missing and 6 duplicates, so the counter recovered the loss exactly. Now let a burst of 300 consecutive records vanish. The counter difference across the hole is 301 \bmod 256 = 45, so the audit reports 44 missing and the other 256 are invisible.

Concept check. A capture logs 200 records per second. How long a stall can the counter audit measure without ambiguity? (A hole of at most 254 records, so a stall of about 1.27 s. From 255 records on, the audit under-reports or reads a duplicate.)

2. Toolchains as workflows

Idea. The eight toolchains below differ in the questions a study cares about: how many receive chains, how wide a channel, how dense the tone grid, how many bits per component, and whether you may read the source. Widths and tone counts come from the papers cited in the reading list. The Nexmon count is the 242 data tones of an 80 MHz channel: of the 256 sub-channels, Nexmon returns only the data ones (meneghello2023_0a93 ↗). The 1992 tones at 160 MHz are the HE160 grid of the AX200/AX210 silicon that IAX, FeitCSI and PicoScenes share.

Workflow facts worth knowing before you buy hardware. The Atheros tool needs the transmitter to set a sounding flag, so it needs the cooperation of both ends. Nexmon and AX-CSI extract only in monitor mode, so the card cannot keep a connection while it captures. IAX reads CSI in station, access-point and monitor modes and needs nothing from the transmitter. The ESP32 has a single receive chain, which limits what its CSI can resolve. PicoScenes is closed source, which is its main obstacle to reuse.

Derivation: tone density and record size. A wider channel carries more tones, and the tone grid gets denser as the subcarrier spacing shrinks. For IAX at 160 MHz the spacing is 160\,\text{MHz}/2048 = 78.125 kHz, and

In words: each coefficient costs 4 B, two chains double it, and the header is a small constant on top. The IAX paper reports 16 208 B per measurement for this configuration, which matches.

Worked numbers (workbook section 2). Tones per MHz at the maximum width: Intel 5300 0.75, Atheros 2.85, ESP32-C5 2.65, Nexmon 3.025, IAX/FeitCSI/PicoScenes 12.45, AX-CSI 12.8. Between the 5300 and IAX the grid is 16.6 times denser. The ESP32-C5 payload for one 53-tone record is 53 \times 2\ \text{B} = 106 B.

Two gotchas that move with the tool. Tone order in the buffer differs: an ESP32-C5 buffer starts at the DC tone, then the positive tones, then the negative ones (53 entries, the DC null included). An ascending-frequency plot needs an fftshift; workbook section 2 shows DC at index 0 in the raw buffer and index 26 after np.fft.fftshift. The DC entry is a literal zero, and 20\log_{10}0 = -\infty, so mask it before any decibel conversion. The AX210 path used in this course stores 52 tones with no DC entry at all.

Concept check. Why is a device that reaches 160 MHz on paper not automatically the right tool for a 20 MHz office study? (Record size, storage and rate grow with tone count; a 20 MHz study needs 52 to 242 tones and pays 16 208 B per record for 1992.)

3. Anatomy of one record

Idea. On the AX210 path a record is a 272-byte header followed by the CSI payload. The payload holds n_\text{rx}\, n_\text{tx} contiguous chain blocks of n_\text{tone} coefficients. A coefficient is two signed 16-bit little-endian words with the imaginary word first.

In words: skip c whole chain blocks, then i coefficients, two words per coefficient; the second word is the real part and the first is the imaginary part.

Header fields. These are the fields established for the reader used here, with little-endian offsets.

Profiles and sizes (workbook section 3):

At a sustained 608 records per second, 688 B records accumulate 608 \times 688 \times 3600 = 1.506 GB per hour.

Worked decode. Take the stored words (4, 3) at payload bytes 0 to 3, the first coefficient of chain 0. The payload starts at byte 272, so that coefficient sits at bytes 272 to 275. The correct reader gives H = 3 + 4j, |H| = 5, phase 0.9273 rad (53.13^\circ), at tone index 0, which is k = -26. The swapped reader gives 4 + 3j, phase 0.6435 rad. It equals j\,\overline{H} = j(3 - 4j) = 4 + 3j. The magnitude is the same, and that is why amplitude plots never expose the mistake.

For the index formula take chain 1, tone index 10, legacy-52: \text{index} = 2(52 + 10) = 124, so the imaginary word begins at byte 272 + 2 \times 124 = 520, and k = -16, which is -5.0 MHz from the centre.

The tone-major mistake. A reader that assumes tone-major storage pairs stored coefficient p as chain p \bmod 2, tone \lfloor p/2 \rfloor. For stored pair 1 the truth is chain 0, tone 1, and the tone-major reader says chain 1, tone 0. For pair 62 the truth is chain 1, tone 10, and the tone-major reader says chain 0, tone 31. Nothing crashes.

A round trip in the workbook packs a synthetic record with struct, parses it back with numpy, and finds a maximum mismatch of 0 and a payload length equal to n_\text{rx}\, n_\text{tx}\, n_\text{tone} \times 4.

Compression note. In a survey of 653 thousand headers the payload compressed only 1.81 times and the header 21.2 times. The header is redundant; the payload is close to noise. The payload is already 60.5 % of a raw legacy-52 record, and because it barely compresses, its share of a compressed file is larger still.

Concept check. A file has n_\text{tone} = 56 and two chains. Give the payload length in bytes and the byte offset of the imaginary word of chain 1, tone 3. (448 B; 272 + 2 \times 2(56 + 3) = 272 + 236 = 508.)

4. Two clocks in one record

Idea. A record carries a fast counter and a slow one. The baseband counter runs at 320 MHz, one tick 3.125 ns, in a 32-bit field, so it wraps. The host stamps arrival on its own clock, which is absolute but jitters.

In words: the counter gives intra-capture timing to nanoseconds but forgets which lap it is on; the host clock knows the date but not the nanoseconds.

The firmware microsecond field wraps too, at 2^{32}\ \mu\text{s} = 71.58 min. The host monotonic clock in the header is the only one that does not wrap during a capture.

Unwrapping. Count a wrap each time the counter decreases, then

This works only if consecutive records are less than one T_\text{wrap} apart. A true gap of 15.0 s is read as 15.0 \bmod 13.4218 = 1.5782 s.

Worked numbers (workbook section 4, seed 7). Records arrive at 200 Hz, so consecutive counters differ by 1 600 000 ticks. The counter starts at 3\times 10^9 ticks, so the first wrap falls at record 810 (4.05 s) and the second at record 3494 (17.47 s). Record 4000 has counter value 810 065 408 and, after two wraps, unwrapped time 20.000 s. The host jitter model is \sigma |z| with \sigma = 1 ms; at record 4000 the host stamp deviates by -0.9986 ms from the counter timeline (relative to the first record), the jitter median is 0.658 ms and the 99.9th percentile 3.117 ms.

A measured contrast: on AX210 monitor captures the host stamps showed a median jitter of 19 microseconds and a 99.9th percentile of 5.4 ms, and 49 thousand records arrived with none out of order. The host monotonic clock is what counts the laps: it unwraps the counter and exposes a stall; the wall clock dates the capture. The rule that follows: analyse timing on the counter, anchor the wall clock on the host stamp.

Concept check. A capture pauses for 20 s and resumes. What does the unwrapped counter report for the pause, and how do you detect the error? (20 mod 13.42 = 6.58 s; compare with the host stamps, which differ by about 20 s.)

5. Record classes on ambient air

Idea. Rate flags classify each frame: CCK, legacy OFDM, HT, VHT or HE. CCK and DSSS frames carry no CSI. The OFDM classes carry different tone grids at 20 MHz: legacy 52, HT 56, VHT 56, HE 242. An ambient capture is therefore a mixture of record classes, and the record length is a class marker.

In words: only the OFDM share of the air becomes records; count every frame as a record and the rate is inflated by 1/(1-f_\text{CCK}) - 1.

Measured anchors.

• A passive AX210 monitor on 2.4 GHz channel 11 delivered 74 % legacy 52-tone and 26 % HT 56-tone records.
• One hour of a passive AX210 monitor on 6 GHz (centre 5975 MHz, 20 MHz) delivered 100 % legacy 52-tone records. That is one hour at one site, a pilot, and not a verdict about the band. The firmware does deliver HE CSI: other captures with the same receiver model hold 40 154 records at 242 tones across 23 capture sessions.

Worked numbers (workbook section 5, seed 1, 2000 frames per scenario). The scenario mixes are assumed and only the CSI-bearing part of the quiet 2.4 GHz mix is anchored to the measurement above.

The busy scenario yields three distinct payload sizes (416, 448 and 1936 B), the quiet one two, the 6 GHz one a single size. A pooled per-tone average across classes mixes different tone grids, so filter by class before you average.

Concept check. A 2.4 GHz injection at a group address is set up at a requested 6 Mb/s but the frames go out as 1 Mb/s DSSS. What will the receiver report? (Frames seen and no CSI records; a low yield is DSSS traffic, not a quiet channel.)

6. Empty and stale records

Idea. On the AX210 path about 15 to 16 % of records are all-zero in the captures surveyed; the ESP32-C5 emits none. The fraction is a property of the capture, not a constant: the 6 GHz hour of section 5 had 4.6 % empty records. Measure it on every file. Separately, a chain whose RSSI reads -127 dBm has no measurement, and its block is a stale duplicate of an earlier one. -127 dBm is about 26 dB below the thermal noise floor of a 20 MHz channel, so it is a sentinel, not a weak signal.

Derivation. With a zero fraction z and mean amplitude \mu over the non-zero records, the unfiltered mean over all records is

In words: zeros pull the mean amplitude down by exactly the zero fraction. For chain B with independent fluctuations of variance \sigma^2, the mean squared successive difference is 2\sigma^2. A stale record repeats the previous block, contributing a difference of 0, so with a stale fraction s the statistic shrinks to 2\sigma^2 (1 - s).

Worked numbers (workbook section 6, seed 5). Model: 200 records, chain A amplitude 20 + 2z, chain B 18 + 2z. At z = 0.15 the model has 30 zero records, a useful yield of 85.0 %. With an assumed stale fraction s = 0.10 of the non-zero records (17 records), chain B is usable in 76.5 % of the records.

• Mean amplitude, unfiltered 16.6714, filtered 19.6134, reference (no zero records) 19.6178. Bias: -15.02\ \% unfiltered, -0.02\ \% filtered.
• Chain B mean squared successive difference: 6.5935 with stale records left in, 7.3309 filtered. The ratio 0.8994 sits next to 1 - s = 0.90.
• The law \bar A_\text{unfiltered}/\bar A_\text{filtered} = 1 - z holds exactly at z = 0.10, 0.15, 0.30 (0.900, 0.850, 0.700).

Concept check. You estimate a motion statistic from chain B and see it fall 10 % without any change in the room. Name two possible causes of that drop in the record stream. (A stale fraction near 10 %, or more zero records; check the sentinel RSSI and the all-zero test before interpreting.)

7. Verify the parse with causality

Idea. A wrong parse can keep |H| unchanged. The guard is a property of the physics: an impulse response is causal. Energy arrives after the transmission time, so the delay-domain energy sits at n \ge 0 when the parse is right.

Derivation. Reading the words in the wrong order, real first, maps H \to j\,\overline{H}. Conjugation in frequency is time reversal in delay:

So |h'[n]| = |h[-n]|: the energy at delay n moves to delay -n, and early taps become late ones. A tone-major reader is a different mistake: it pairs the wrong coefficients, so the impulse response is smeared instead of mirrored.

In words: a mirrored phase looks fine tone by tone but arrives from the future, and no amplitude plot can see it.

Worked numbers (workbook section 7, seed 2). Channel: three taps at 100, 210 and 360 ns with gains 1, 0.6, 0.35; 52 legacy tones; 25 dB SNR; the words are quantised to imaginary-first int16 pairs and then parsed three ways. Delay bins are 50 ns wide, early energy is n \ge 0 and late energy n < 0. The 64-point inverse DFT is circular, so bins n = -32 \dots -1 are the same as bins 32 \dots 63: the late half of the circle, which a causal channel with a short delay spread leaves nearly empty.

The maximum |\,|H|_\text{correct} - |H|_\text{swapped}\,| is 0.0, and the swapped values equal j\overline{H} exactly. The product of the two ratios is 1.78, not 1: bin 0 and bin -32 map to themselves under time reversal.

The tone-major reader escapes the early/late test (27.19) and is caught by compactness (0.5884 against 0.8145). It also changes what a "chain ratio" means. The two-chain ratio |H_A/H_B| becomes the ratio of tone i to tone i+1 of one chain: in the workbook the magnitudes agree with the adjacent-tone ratio to a maximum difference of 0.0, the median ratio falls from 0.604 dB to -0.030 dB and its spread from 5.64 dB to 2.22 dB.

Measured. On real AX210 captures the impulse response concentrated 21.3 times more energy at early delays than late with the correct parse, and 0.48 times with the swapped parse, an anti-causal channel. The measured window is the reader's own and differs from the one in this workbook, so compare the direction and not the digits. Reading chain-major storage chain-major gave a more compact impulse response in 99.4 % of 5186 records. In a separate check on three sample recordings (200 two-chain records each), a ratio computed by a tone-major reader matched the adjacent-tone ratio of one chain exactly in all 600 records.

Limits. The test assumes the timing reference sits before the first arrival. A large packet-detection offset shifts the delay axis and can push a correct parse into the late half, so correct the linear phase slope first when you have one.

Concept check. Your |H| plots look right and the early/late ratio is 30. Is the parse verified? (It excludes the I/Q swap; a tone-major reshape can still pass, so also check the impulse response compactness and the payload arithmetic.)

Exam questions

1. Why can two programs not read CSI from the same radio at once, and what design consequence follows for monitoring a running capture? Model answer. The driver delivers CSI to the one socket that registered for it, so there is one consumer per radio. A second tool either receives nothing or takes the stream from the first. Monitor a running capture through the logger's own counters and status output, or dedicate another radio to observation.

2. A flat int16 array has n_\text{tone} = 52 and two chains. Give its length, the index of (c,i) = (1, 10), the byte offset in the record, and H in terms of the words. Model answer. The payload has 2 \times 52 \times 2 = 208 words (416 B). The index is 2(1 \cdot 52 + 10) = 124, the byte offset 272 + 2\times 124 = 520, and H = iq[125] + j\,iq[124], at k = -16, which is -5.0 MHz.

3. A counter starts at 3\times 10^9 ticks and records arrive at 200 Hz. At which record does it first wrap, and what is the unwrapped time of record 4000? Model answer. Ticks to the wrap: 2^{32} - 3\times 10^9 = 1\,294\,967\,296, which is 4.0468 s, so record 810 at 200 Hz. Record 4000 arrives at 20 s. Its counter is 3\times 10^9 + 4000 \times 1\,600\,000 = 9.4\times 10^9, minus two periods of 4 294 967 296 gives 810 065 408, and adding the two wraps back recovers 20.000 s.

4. A colleague states that a passive 2.4 GHz capture delivered 2000 CSI records from 2000 frames. In that channel 30 % of frames are CCK. What do they overstate? Model answer. CCK frames yield no CSI. Only 1400 frames can be records (70 %). Counting all 2000 overstates the rate by 1/0.7 - 1 = 42.9\ \%. The workbook draw of 2000 frames gives 1406 records and an inflation of 42.2 %.

5. Your phase-derived delay profile is anti-causal but every amplitude plot looks fine. Give the algebra, and name one parse mistake that the early/late test does not catch. Model answer. Reading real-first maps H \to j\overline{H}. |H| is unchanged, the phase is mirrored, and conjugation in frequency is time reversal in delay, so energy moves from delay n to -n. The early/late ratio inverts (32.95 to 0.054 in the workbook). A tone-major reshape of chain-major data keeps the ratio near 27 but halves the compactness measure (0.5884 against 0.8145), so check compactness and payload length as well.

6. Zero records are 15 % of a stream. Compute the bias of the unfiltered mean amplitude. A stale chain covers 10 % of records; what happens to a successive-difference variance on that chain? Model answer. The unfiltered mean is (1 - 0.15) = 0.85 times the filtered mean, a bias of -15\ \% (workbook: -15.02\ \% against -0.02\ \% after filtering). Stale records repeat the previous block, so their differences are 0 and the statistic shrinks by the factor 1 - s = 0.90 (workbook ratio 0.8994).

Further reading

• guarino2026_e72c ↗: tool comparison for 802.11n, ac and ax, and where reproducibility fails at collection.
• zhang2025_72c4 ↗: the IAX tool, its 272-byte header plus matrix layout, and its comparison with the other tools.
• gringoli2022_1c5c ↗: PHY table layout of a Broadcom 802.11ax extractor, 8192 values per table.
• gringoli2019_68e7 ↗: Nexmon CSI, its per-tone 4-byte floating-point format and a validation of the extracted data.
• zhang2026_ccac ↗: a protocol view of how capture choices decide reproducibility.
• meneghello2023_0a93 ↗: a Nexmon dataset at 80 MHz, and why it holds 242 of the 256 sub-channels.

Provenance (instructor only)

First-party numbers on this page and the files they were read from this session:

• Header layout, chain-major, imaginary-first, RSSI -127 sentinel, counter, rate flags, 320 MHz clock and wrap, header/matrix compression, 15 to 16 % zero records, one consumer per radio: .claude/memory/csid-capture-service.md
• Header field offsets and widths, rate flag layout (raw stream reader source): repos/csid/crates/csiq/src/raw.rs
• Chain-major and imaginary-first reader and tests, early/late ratios, 99.4 % of 5186 records: repos/csid/python/csiq/reader.py, repos/csid/python/tests/test_consumer_rules.py
• Discrepancy to resolve: the reader source and its test docstring say 21.5, the memory page .claude/memory/csid-capture-service.md says 21.3 (the 2026-09-24 diary does not state the ratio). This page uses 21.3 as the course specification directs.
• 6 GHz hour with 4.6 % empty records: obsidian/monad-knowledge/diary/2026-09-03 - The 6 GHz band answered on the third try at 200 records a second, and the lake's decoder is two days behind the fleet.md
• 16 208 B per 160 MHz measurement, Atheros sounding flag, IAX in station, AP and monitor modes: zhang2025_72c4 (search_vault passages, sections "F. Limitations", "A. CSI Extractor", "B. Atheros CSI Tool"). Nexmon 242 of 256 sub-channels at 80 MHz: meneghello2023_0a93.
• Tone-major reading and 600 records: obsidian/monad-knowledge/diary/2026-09-23 - The lake's ratio columns divide neighbouring tones of one chain, not chain A by chain B.md
• ESP32-C5 53 entries, 8-bit pairs, DC first ordering, no all-zero records: obsidian/monad-knowledge/diary/2026-09-24 - CSI teaching needs explicit vendor layouts and visual steps.md, .claude/memory/esp32-and-photon.md
• Baseband 320 MHz, 3.125 ns; host jitter p50 19 us and p99.9 5.4 ms; 6 GHz channel byte reads 48: .claude/memory/csid-capture-service.md, .claude/memory/ax210-pi5-driver.md
• 74 % legacy and 26 % HT on a channel-11 ambient capture: obsidian/monad-knowledge/edu/courses/WS501-wifi-csi-sensing/week-05-preprocessing/lecture.md
• 6 GHz hour, 100 % legacy 52-tone, 40 154 HE records at 242 tones across 23 sessions, pilot status: obsidian/monad-knowledge/diary/2026-09-03 - The 6 GHz band answered on the third try at 200 records a second, and the lake's decoder is two days behind the fleet.md, obsidian/monad-knowledge/diary/2026-09-04 - The 6 GHz verdict was not supportable, and the lake tools were racing on one connection.md
• DSSS fallback of a 2.4 GHz group-addressed injection: .claude/memory/illumination-and-injector.md
• 608 records per second sustained: .claude/memory/csid-capture-service.md
• Reference math ported into the deck widgets: monad_knowledge/web/frontend/src/lib/csi-school.js, monad_knowledge/web/frontend/src/islands/ByteInspector.svelte
• Assumptions flagged in this page: csi_len drawn 4 B wide, 3 chains for the Intel 5300 and Atheros cards, 16-bit for PicoScenes and FeitCSI, the stale fraction 10 %, the scenario frame mixes (only the 74/26 CSI-bearing split of the quiet 2.4 GHz mix is anchored), the synthetic channel taps.