Description
Classifying signs of a sign language (typically ASL, with smaller-scale work on other sign systems) from wireless signal perturbations of hand and arm motion. Sign language recognition is a structured, large-vocabulary cousin of gesture-recognition — it brings sequence modeling and language-modeling concerns alongside the underlying CSI sensitivity question. It is also a direct accessibility application: a deployable wireless ASL recognizer could replace or supplement camera-based interpretation systems.
Why it's hard
- Vocabulary size: even basic deployable systems target 100+ signs, far beyond typical gesture sets.
- Continuous signing has no clean inter-sign boundaries; segmentation is itself a research problem.
- Two-handed and facial-expression components are essentially invisible to commodity WiFi.
- Cross-signer transfer is poor; same domain-shift problem as activity-recognition but more severe.
Common approaches
- Sequence-to-sequence models (encoder-decoder, Transformer) over CSI feature sequences.
- Connectionist temporal classification for unsegmented training.
- Transfer from larger gesture datasets via pretraining + fine-tuning.