Not There Yet: Feasibility and Challenges of Mobile Sound Recognition to Support Deaf and Hard-of-Hearing People
Published in ASSETS, 2023
Abstract
Sound awareness systems help DHH users recognize different sounds in their surroundings. We conducted a three-week longitudinal field study of SoundWatch, a smartwatch-based sound recognition app, with 10 deaf and hard of hearing (DHH) participants to assess its real-world feasibility. While the app increased environmental awareness and supported everyday tasks, challenges such as background noise, sound variability, and misidentification highlight the need for improved model reliability, user customization, and collective model enhancement.
My Contributions
- Initially tested the usability of SoundWatch, ensuring it was functioning correctly before deployment in the field to 10 DHH participants.
- Helped refine the interview scripts for the introductory interviews with participants.
- Conducted exit interviews with participants and analyzed user feedback using qualitative coding.
- Wrote Python scripts to scrape user log files and analyzed participants’ usage trends of the system over time.
Awards
🏆 Best Student Paper Honorable Mention
Recommended citation: Jeremy Zhengqi Huang, Hriday Chhabria, and Dhruv Jain. (2023). "Not There Yet: Feasibility and Challenges of Mobile Sound Recognition to Support Deaf and Hard-of-Hearing People." ASSETS 2023. ACM, Article 15, 1–14. 🏆 Best Student Paper Honorable Mention.
Download Paper