Knowledge distillation is a model compression and transfer technique in which a smaller "student" network is trained to mimic the output distributions, intermediate representations, or behavioral patterns of a larger, more capable "teacher" network, rather than being trained solely on hard ground-truth labels. In WiFi CSI-based human sensing, it matters because it enables the deployment of lightweight models on resource-constrained edge devices while preserving much of the predictive accuracy achieved by complex deep learning architectures, and it also facilitates cross-domain adaptation by transferring learned feature representations from data-rich source domains to data-scarce target domains. Key variants relevant to this field include response-based distillation, which matches final output logits or probability distributions, feature-based distillation, which aligns intermediate layer activations, and self-distillation or few-shot distillation schemes that are particularly useful in scenarios like DASECount where labeled target-domain samples are extremely limited.
Source Papers
- DASECount: Domain-Agnostic Sample-Efficient Wireless Indoor Crowd Counting via Few-Shot Learning ↗ — DASECount: Domain-Agnostic Sample-Efficient Wireless Indoor
- Deep Learning-Enhanced Human Sensing with Channel State Information: A Survey ↗ — Deep Learning-Enhanced Human Sensing with Channel State Info