Description
The Transformer architecture replaces recurrence with multi-head self-attention, letting every token attend to every other in O(N²) and producing strong long-range models for sequences. In WiFi sensing, Transformers and attention modules are increasingly used as the backbone for cross-domain CSI sensing — particularly for fusing multiple antennas / subcarrier groups and for sequence-level pretraining tasks. Plain attention modules also drop into CNN pipelines as a re-weighting step.
When it's used
- Cross-domain / cross-environment CSI sensing models
- Long sequence aggregation (gait, multi-action recognition)
- Multi-modal fusion (CSI + RSSI + IMU)
- Pretrained backbones for
self-supervised-learning
Limitations
- Quadratic memory cost in sequence length without linear-attention tricks
- Data-hungry; small CSI datasets struggle without augmentation
- Less inductive bias than CNN/LSTM for short windows