SAMoSA: Sensing Activities with Motion and Subsampled Audio

Despite advances in audio- and motion-based human activity recognition (HAR) systems, a practical, power-efficient, and privacy-sensitive activity recognition approach has remained elusive. State-of-the-art activity recognition systems generally require power-hungry and privacy-invasive audio data. This is especially challenging for resource-constrained wearables, such as smartwatches. In contrast, our approach Senses Activities using Motion and Subsampled Audio (SAMoSA) using a smartwatch. We use audio rates ≤1kHz, rendering spoken content unintelligible, while also reducing power consumption. Our multimodal deep learning model achieves a recognition accuracy of 92.2% across 26 daily activities in four indoor environments.

---

Citation

Vimal Mollyn, Karan Ahuja, Dhruv Verma, Chris Harrison, and Mayank Goel. 2022. SAMoSA: Sensing Activities with Motion and Subsampled Audio. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 6, 3, Article 132 (September 2022), 19 pages.