CSI Model Drift Measurement Over Time

Motivation

The core problem this thesis addresses: CSI sensing models degrade over time as the environment changes. This is well-documented anecdotally but poorly quantified for crowd counting specifically.

Billah & Campbell (2021) ↗ measured BLE-based occupancy model degradation: LSTM F1 dropped from 92.2% to 70.3% in just 7 days. Guarino et al. (2026) ↗ and Wang et al. (2026) ↗ both identify environmental robustness as the critical open problem. Koo et al. (2026) ↗ calls for "lightweight self-calibration routines."

No paper in the vault systematically measures CSI crowd counting drift with controlled environmental variables over an extended period. This experiment fills that gap and establishes the baseline that EXP-003 (calibration campaigns) will improve upon.

Setup

Prerequisites

• EXP-001 BLE-Assisted CSI Ground Truth Collection must be validated (BLE ground truth pipeline working)

Device Deployment

• Same physical setup as EXP-001 (CSI TX/RX pairs + BLE listeners)
• Devices must remain in fixed positions for the entire 30-day period
• Power: continuous operation (wall power, not battery)

Environmental Monitoring

Log daily:

• Furniture positions (photograph with timestamp)
• Door/window open/closed state
• Number of additional electronic devices in room (laptops, monitors)
• Temperature and humidity (sensor or manual reading)
• Any maintenance events (cleaning, renovation)

Procedure

Day 0: Model Training

1. Collect 2 hours of CSI data with BLE ground truth (incremental occupancy 0-10 people)
2. Extract CSI features per Wi-CaL ↗ approach:
   • Amplitude variance per subcarrier (time window = 1s)
   • Phase difference across antenna pairs
   • RSS standard deviation
3. Train crowd counting model:
   • Model A: Random Forest (baseline, lightweight)
   • Model B: LSTM on raw CSI amplitude sequences
   • Model C: CNN on CSI amplitude images (subcarrier x time)
4. Record Day 0 accuracy on held-out test set
5. Freeze all models — no retraining for 30 days

Days 1-30: Drift Monitoring

1. Run continuous CSI collection during working hours (8:00-18:00)
2. BLE ground truth runs in parallel (automated)
3. Every day, evaluate frozen models on that day's data:
   • MAE (mean absolute error) of people count
   • Classification accuracy for occupancy levels (0, 1-2, 3-5, 6-10)
   • F1 score per occupancy level
4. Log environmental changes daily

Controlled Perturbations (optional, if naturally occurring changes are insufficient)

• Day 7: Move 1 chair
• Day 14: Rearrange furniture (swap table and chair positions)
• Day 21: Add a large object (whiteboard, monitor) near a TX-RX link
• Day 28: Return to original layout

Expected Outputs

• dataset/exp002/daily_accuracy.csv — MAE, F1, accuracy per model per day
• dataset/exp002/environment_log.csv — daily environmental state
• dataset/exp002/csi/day_XX/ — raw CSI data per day
• Degradation curve: accuracy vs. days since training
• Correlation analysis: accuracy drop vs. environmental change magnitude

Analysis Plan

Primary Metrics

1. Degradation half-life: days until accuracy drops to 50% of Day 0 performance
2. Degradation rate: slope of accuracy-vs-time regression
3. Environmental sensitivity: correlation between furniture changes and accuracy drops
4. Model robustness ranking: which model architecture degrades slowest?

Visualizations

• Line plot: accuracy over 30 days, one line per model
• Scatter plot: accuracy drop vs. number of environmental changes since Day 0
• Heatmap: per-subcarrier amplitude variance drift over time

Success Criteria

• 30 consecutive days of data collection (allowing for weekend gaps)
• Measurable accuracy degradation documented (>10% drop from Day 0)
• At least one model shows statistically significant correlation between environmental change and accuracy drop
• Results are sufficient to motivate the recalibration approach in EXP-003

Related Work in Vault

• BLE Can See ↗ — F1 drops from 92.2% to 70.3% in 7 days (BLE, not CSI)
• CSI Sensing Datasets Survey ↗ — "environmental robustness and generalization" is the #1 open issue
• WiFi Sensing Generalizability Survey ↗ — cross-environment generalization taxonomy
• Green Wireless Sensing Survey ↗ — "recalibration burden differs across model families"
• Santos et al. 2024 ↗ — environment dependence in WiFi CSI crowd counting
• Meneghello et al. 2023 ↗ — domain adaptation for CSI sensing

Dependencies

• EXP-001 (validated BLE ground truth)
• 30 days of continuous device operation
• Stable power supply and network connectivity

Notes

This experiment produces the problem statement evidence for the thesis. The degradation curve is the central motivation figure: "without recalibration, CSI crowd counting becomes unreliable within X days."