Collecting CSI off commodity hardware: capture modes, toolchains, formats, and the gotchas that bite before any algorithm runs

Where We Are

Week 2 derived the object a Wi-Fi card hands you: a per-packet table of complex numbers, one per antenna-pair-and-tone, written as \hat H_{r,t,k} = a_{r,t,k}\,e^{j\theta_{r,t,k}}. We catalogued the hardware fingerprints baked into the sizes — the switched-off centre tone, the quieter band edges, the per-packet AGC volume knob, and tone averaging — and noted that the angles carry their own offsets we have not yet removed. We ended on a measured result from the lab's clean channel: about 27.5 dB of weakening per person, a Rician-K slope that rises with occupancy and sign-flips under mild noise.

All of that assumed the table was already on disk. This week is the missing prequel: how does the table get out of the card and into a file at all, and what breaks on the way? It is the least glamorous week of the course and the one that wastes the most PhD-months when skipped. A model trained on a mis-ordered subcarrier axis, or on a capture too slow to see a walking person, fails for reasons no amount of model tuning can fix — because the damage happened during collection, before the data was ever read.

A few terms recur, so fix them up front. Firmware is the program running on the Wi-Fi chip itself, below the operating system. A driver is the operating-system code that talks to the chip. Monitor mode is a card configuration that reports every packet it overhears, not just packets addressed to it. Injection is transmitting hand-crafted packets to force the channel to be measured on demand. PRR (packet reception rate) is how many CSI-bearing packets per second you actually capture. Ground truth is the trusted record of what physically happened — who was where, doing what — that a CSI trace is labelled against. Each is defined again, in context, the first time it does real work below.

1. The Collection Pipeline, and Why the Card Fights You

Recall from Week 2 that the card estimates the channel on every packet during the preamble — it computes \hat H(k) = Y_k / X_k from the known training field just to equalise the data that follows. That estimate is internal scratch work. Once the packet is decoded, the card overwrites it. From the chip's point of view, CSI is a disposable intermediate, not an output.

So the whole game of CSI collection is catching a value the hardware intends to throw away. That single fact explains why there is no "CSI mode" you can switch on in a normal driver, and why every tool in Section 3 is some flavour of firmware or driver surgery. The pipeline a real study runs looks like this:

transmitter → radio channel → receiving NIC front-end → patched firmware/driver that copies CSI out before it is discarded → user-space logger → on-disk file → parser → the tensor \mathbf{H}(t) from Week 2

Each arrow is a place a study can quietly break. The patched firmware may report tones in a different order than your parser assumes (Section 4). The logger may drop packets under load, thinning your effective sample rate below what the motion needs (Section 5). The file format may pack the centre-tone null as a real zero that your dB conversion turns into -\infty (Section 4). None of these is a modelling problem, and none is visible once the data is a clean-looking NumPy array — which is exactly why they survive into published results.

Workbook §3.1 renders this pipeline as an annotated diagram and marks the three stages — firmware copy-out, logging, parsing — where the gotchas in Sections 4–5 enter.

2. Two Ways to Collect: Active Injection vs Passive Sniffing

Before choosing a tool, choose a stance. There are two, and they trade control for deployability.

Active sensing means you own both ends. A dedicated transmitter injects packets at a rate you set, and a dedicated receiver logs the CSI. You control the packet rate, the bandwidth, the antennas, and the geometry. This is how almost every benchmark dataset was built, because a controlled, high, steady packet rate is the single most valuable property a CSI trace can have (Section 5 explains why). The cost is that you have deployed two pieces of dedicated hardware that do nothing but sense.

Passive sensing means you sniff packets that some other device — a phone streaming video, a laptop on a call — is already sending. You add no traffic; you just put a card in monitor mode and log the CSI of whatever flies past. This is the deployable, real-world stance: no instrumentation of the people being sensed, no extra spectrum used (cf. the lab's interest in leveraging ambient traffic, sharma2024_c8a2 ↗). The cost is that you get the packet rate the environment happens to produce — bursty, uneven, and often far too slow when nobody is actively using the network. A passive capture during a quiet meeting can collapse to a few packets per second, and Section 5 shows that a few packets per second cannot see a walking person.

Most of this course's downstream tasks assume an active, high-rate capture, because that is what the public datasets provide. But the honest deployment target — and the lab's own crowd-sensing direction — is passive. Keep the gap in mind: a method validated on a steady 1000 Hz active trace may be invisible on a bursty 20 Hz passive one, and that is a collection problem, not a model problem.

3. The Five Toolchains, as Workflows

Week 2 compared three cards on what they can see (tone count, bit depth, bandwidth). This section compares five toolchains on what it takes to run them: which firmware they patch, whether they can inject, which Wi-Fi generation they reach, and what they write to disk. These are the practical axes that decide whether a collection campaign is a weekend or a month.

Intel 5300 — Linux 802.11n CSI Tool (Halperin 2011)

The field's most-cited tool, and a modified firmware plus a modified iwlwifi driver (halperin2011_2cbc ↗, intel-5300). You flash custom firmware, load the patched driver, and a user-space log_to_file utility writes a compact binary .dat of 30 grouped subcarriers at 8-bit resolution. Injection is supported via a connected-AP or ad-hoc setup. The chipset is discontinued and 802.11n-only (40 MHz ceiling), but because most public datasets — Widar3.0 included — were built on it, its .dat format and 30-group decimation are facts you must parse correctly even if you never own the card.

Atheros CSI Tool — ath9k (Xie 2015)

A patched open-source ath9k driver, no firmware blob to flash (atheros-csi-tool, xie2015_0389 ↗). It reports the full 56-tone grid at 10-bit resolution and, crucially, runs on cheap OpenWrt routers as well as x86 hosts — the first tool to make a sub-$50 CSI sniffer plausible. Still 802.11n.

Nexmon CSI — Broadcom (Gringoli 2019)

A C-based firmware-patching framework for Broadcom chips, with the nexmon_csi extension exposing per-frame CSI (gringoli2019_68e7 ↗, nexmon-csi). Its headline property is the hardware it unlocks: the Broadcom radio inside the Raspberry Pi 3B+/4 and consumer routers, at one-tenth the cost of an Intel rig, and the first commodity path to 802.11ac (80 MHz, 256 subcarriers). It writes CSI inside pcap files — convenient, but the source of the single most common parsing bug in the field (Section 4). The 80 MHz public dataset of meneghello2023_0a93 ↗ was collected this way.

PicoScenes — unified platform (Jiang 2021)

Not a single patch but a platform that wraps Atheros QCA9300, Intel 5300, and Intel AX200/AX210 behind one MATLAB/Python API (picoscenes). It is the first practical route to 802.11ax (Wi-Fi 6) CSI on AX200/AX210, supports up to 27 concurrent NICs and rich frame injection, and emits a self-describing struct carrying CSI, timestamps, RSSI, and SNR together. The trade-off is weight: it is a large dependency to install and learn, and historically not fully open. It is the reference tool for modern ISAC (integrated sensing-and-communication) experiments.

FeitCSI — open-source 802.11ax (KuskoSoft, 2023+)

The newest entrant, and the one the curriculum singles out: the first free, open-source tool to extract and inject CSI for all formats (802.11a/g/n/ac/ax) and all bandwidths (20/40/80/160 MHz) on Intel AX200/AX210 (feitcsi). It captures up to 512 subcarriers at 16-bit precision across 2.4/5/6 GHz, exposes Fine Timing Measurement, and can be driven over a UDP socket from MATLAB. In effect it gives the AX200/AX210 most of PicoScenes' capability without the licensing or footprint — which is why a CSI study started in 2026 on Wi-Fi 6 hardware will likely reach for FeitCSI or PicoScenes rather than the discontinued Intel 5300.

The table below compares the five on the workflow axes, not the capability axes of Week 2.

Workbook §3.2 renders this matrix as a figure and prints, for each tool, the one workflow fact that most often surprises a newcomer (e.g. "Nexmon CSI lives inside pcap; you parse packets, not a CSI file").

One unifying mercy: the community parser CSIKit reads Atheros, Intel 5300, Nexmon, ESP32, FeitCSI, and PicoScenes formats behind one Python interface. It does not, however, save you from the semantic gotchas of Section 4 — it hands you an array; whether the axis means what you think is still on you.

4. The Parsing Gotchas That Corrupt a Study Silently

These are the bugs that produce a plausible-looking array and a wrong result. Each is shown in data in the workbook.

Subcarrier ordering (the fftshift trap). A card's firmware reports subcarriers in FFT order — index 0 is the centre frequency, then the positive half, then the negative half wraps around at the end: [0, 1, \dots, N/2-1, -N/2, \dots, -1]. Human intuition (and most plots) wants them in natural order, [-N/2, \dots, -1, 0, 1, \dots, N/2-1], with DC in the middle. Converting between the two is a single fftshift, but forget it and the DC null sits at the edge of your plot instead of the centre, the band-edge roll-off appears in the middle, and any geometry you read off the subcarrier pattern is scrambled. This is the most common Nexmon-pcap bug precisely because pcap hands you raw firmware order.

The DC-null placeholder. Week 2's switched-off centre tone is reported as a literal 0+0j (or a tiny placeholder). Convert amplitude to decibels with 20\log_{10}|H_k| and that zero becomes -\infty, which then poisons every mean, normalisation, and plot axis downstream. The fix is to treat the DC index as missing data and exclude or interpolate it before any dB conversion — never after.

Byte order and bit width. The Intel .dat packs signed 8-bit components; Atheros packs 10-bit; FeitCSI 16-bit; Nexmon stores floats in pcap payloads. Read a 10-bit stream as 8-bit, or little-endian as big-endian, and you get an array of the right shape full of garbage — no error, just noise that looks like a bad channel. Trust the parser (CSIKit) over hand-rolled struct.unpack unless you have a reason not to.

Dropped and duplicate packets. A logger under load drops packets; a flaky link retransmits and duplicates them. Either way the nominal packet rate you configured is not the effective rate on disk, and the gaps are uneven. If you assume a uniform time axis — as every spectrogram and STFT does — uneven sampling smears the Doppler features Week 5 depends on. Always reconstruct the real time axis from the per-packet timestamps and check the gap distribution before trusting any frequency-domain feature.

Workbook §3.3 plots the same synthetic CSI in raw firmware order and after fftshift, so the DC-null jumping from edge to centre is unmistakable; §3.4 builds a packet-timestamp stream with realistic drops and duplicates and plots the inter-packet-interval histogram, marking the long tail that breaks the uniform-sampling assumption.

5. Packet Rate Is a First-Class Experimental Variable

Of all the collection parameters, packet rate is the one most often left to chance and the one that most directly caps what you can sense. The reason is Nyquist, and it is worth deriving because it converts an abstract "collect fast enough" into a hard number.

A person moving changes the channel over time. From packet to packet, a path whose length changes induces a phase rotation — a Doppler shift f_D. Your stream of packets samples that time-varying channel at the packet reception rate, PRR. Sampling theory is unforgiving: a signal at frequency f_D can only be recovered without aliasing if you sample at more than twice it. So the fastest Doppler your capture can represent is

A Doppler shift maps to a radial speed v through the carrier wavelength \lambda = c/f_c. For motion along the dominant reflection path the shift reaches f_D = 2v/\lambda — the same two-way geometry as a radar return, and the conservative upper bound for the bistatic Wi-Fi case (where the true factor is \cos\alpha + \cos\beta \le 2). Setting the Nyquist ceiling equal to this maximum, f_{\max} = 2v_{\max}/\lambda, gives the speed ceiling

Put numbers in. On a 5.32 GHz channel, \lambda = c/f_c = 3\times10^8 / 5.32\times10^9 \approx 5.6 cm. Then:

• PRR = 1000 Hz → v_{\max} \approx 0.056 \times 1000 / 4 = 14 m/s. Comfortably covers any human motion, including a fast hand gesture.
• PRR = 100 Hz → v_{\max} \approx 1.4 m/s. Exactly normal walking speed — you are at the edge, and a brisk walk aliases.
• PRR = 20 Hz (a plausible quiet-network passive capture) → v_{\max} \approx 0.28 m/s. A walking person is now folded back into the low-frequency band as an artefact; the model sees motion that is not there and misses the motion that is.

This is the punchline of the week: the choice you make at collection time — set a 1000 Hz active injection, or accept a 20 Hz passive trickle — sets a ceiling on every downstream task, and no preprocessing or model in Weeks 4–10 can raise it. A walking-speed activity recogniser validated on Widar3.0's high active rate is defined to fail on a slow passive capture, and the failure looks like a model bug while being a collection bug.

Workbook §3.5 plots v_{\max} against PRR for the 2.4 and 5 GHz bands, marks the walking-speed and gesture-speed thresholds, and shades the region where common passive capture rates cannot see a walking person. The printed line reads PRR=100 Hz @ 5.32 GHz -> v_max = 1.41 m/s.

6. Ground Truth Is the Scarce Resource, Not CSI

CSI is cheap once a toolchain works — a card left running overnight produces gigabytes. What is expensive, and what actually bounds the field, is the trusted label: the record of who was where doing what, time-aligned to the trace. Two collection designs bracket the cost spectrum.

Widar3.0 — exhaustive, hand-labelled, multi-domain. The canonical cross-domain gesture dataset (zheng2019_5389 ↗, widar) was built by recording volunteers performing a fixed gesture vocabulary, deliberately varied across the factors that break CSI models: multiple rooms, multiple positions within a room, multiple body orientations, and six simultaneous receivers for spatial diversity. The combinatorial product of those factors is what the dataset calls its ~75 "domains," and the total runs to roughly 258,000 gesture instances over thousands of minutes. The labels came from a rigid, scripted collection protocol — every instance was commanded, so the label is known by construction. That rigour is also the limitation: scripted gestures are not spontaneous behaviour, and the labelling cost scales with the combinatorial domain explosion, which is why so few datasets cover it.

The Widar3.0 design also gave the field its most-cited domain-independent feature, the BVP (Body-coordinate Velocity Profile) — a 20\times20\times T representation of velocity components in a body-centred frame, derived from the multi-receiver Doppler so that the same gesture yields the same BVP regardless of where the person stands or faces. BVP is the payoff of collecting from six receivers at once; with one receiver it cannot be formed. Weeks 5 and 7 build on it.

The lab's BLE-assisted ground truth — automated, continuous, occupancy-scale. At the opposite end, the lab's own direction (EXP-001) trades the scripted-gesture rigour for automated labelling: a periodic BLE beacon signal provides a continuous, machine-read occupancy/position ground truth alongside the CSI, removing the human-in-the-loop labelling that caps Widar3.0's scale. The bet is that an automated label, even if coarser than a scripted gesture command, can be collected continuously and at occupancy scale — which is the regime crowd-sensing actually needs.

The contrast is the lesson. Widar3.0 shows what exhaustive hand-labelling buys (clean cross-domain splits) and what it costs (combinatorial explosion, scripted-not-spontaneous behaviour). The BLE-assisted approach shows the alternative bet (automate the label, scale the collection). The curriculum's recurring "28 datasets, one with synchronised video" thread (Week 11) is the field-level symptom of the same scarcity: CSI is everywhere, trustworthy time-aligned ground truth is almost nowhere.

Workbook §3.6 reproduces the Widar3.0 domain arithmetic — the combinatorial product of rooms × positions × orientations that yields the ~75-domain figure — and plots how labelling cost grows with each added factor, making the explosion visible rather than asserted.

Key Results and Where to Reproduce Them

Bridge to Week 4

This week got the CSI table onto disk honestly: the right tool for the hardware, the right packet rate for the motion, the axis in the right order, and the dropped packets accounted for. What it did not do is clean the table — the AGC volume knob still scales each packet differently, the band-edge roll-off still colours the amplitudes, and the phase still carries the CFO, SFO, and packet-boundary offsets Week 2 flagged and Section 4 left in place. Week 4 is preprocessing: amplitude sanitisation (AGC removal, band-edge normalisation, denoising) and the phase-sanitisation tricks that make the angle usable at all — the CSI-ratio and conjugate-multiplication methods that cancel the synchronisation offsets between antennas. The gotchas catalogued here are exactly the inputs Week 4's pipeline is built to repair, so a faithful collection is the precondition for everything that follows.

Further Reading

• Yang, Zhang, Chi & Zhang 2022, Hands-on Wireless Sensing with Wi-Fi: A Tutorial (Tsinghua TNS; arXiv:2206.09532) — the primary source for this week. Its "CSI Data Collection" section is the canonical walk-through of the active/passive distinction and the Intel/Atheros/PicoScenes toolchains; the companion site hosts the Widar3.0 release. https://tns.thss.tsinghua.edu.cn/wst/
• Halperin et al. 2011, Tool Release: Gathering 802.11n Traces with Channel State Information — the original Intel 5300 toolchain; the .dat format and 30-group decimation most public datasets inherit. halperin2011_2cbc ↗
• Xie et al. 2015, Precise Power Delay Profiling with Commodity WiFi — the Atheros CSI tool context; full 56-tone, 10-bit, OpenWrt-deployable capture. xie2015_0389 ↗
• Gringoli et al. 2019, Free Your CSI — the Nexmon CSI extractor; pcap-based capture on Raspberry Pi and consumer routers, and a side-by-side characterisation of the major tools. gringoli2019_68e7 ↗
• Meneghello et al. 2023, A CSI Dataset for Wireless Human Sensing on 80 MHz Wi-Fi Channels — a Nexmon-collected 80 MHz dataset; a worked example of an active collection at the wide-band, high-rate end. meneghello2023_0a93 ↗
• Zheng et al. 2019, Zero-Effort Cross-Domain Gesture Recognition with Wi-Fi — the Widar3.0 collection design and the BVP feature; the reference for what exhaustive multi-domain ground truth costs and buys. zheng2019_5389 ↗
• Bocus et al. 2022, OPERAnet — a rare multi-sensor dataset with synchronised RF and vision ground truth; the exception that proves the scarcity rule. bocus2022_ce7f ↗
• Ma et al. 2020, WiFi Sensing with Channel State Information — surveys collection practice and the phase-offset sources (CFO, SFO) Week 4 will remove. ma2020_4782 ↗
• Guarino et al. 2026, A survey on CSI-based Wi-Fi sensing datasets and models with a focus on reproducibility — the most current toolchain comparison, covering PicoScenes, FeitCSI, and AX-CSI for 802.11ax, and the reproducibility failures that trace back to collection. guarino2026_e72c ↗
• Sharma et al. 2024, Towards Energy Efficient Wireless Sensing by Leveraging Ambient Wi-Fi Traffic — the passive-sensing stance and the packet-rate problem it inherits. sharma2024_c8a2 ↗