Abstract
Sound and speech AI has made remarkable progress in recognizing, separating, describing, and generating audio. Much of this progress is measured through model-centric metrics: accuracy, error rates, localization performance, and other benchmarks that tell us how well an algorithm performs a predefined task. But the systems we ultimately build do not end at the model output. They end with a person interpreting that output, acting on it, and using it to navigate the world. In this talk, I argue that human experience should not be treated merely as a downstream outcome of a sound AI system, but as part of the system itself. Drawing on a series of projects spanning sound recognition, accessibility, human-AI collaboration, and personalized audio, I will show how human context, goals, behavior, and interaction can reveal limitations that are invisible in benchmark evaluations—and can fundamentally change how these systems should be designed. A person may provide contextual knowledge that a model lacks, have needs that shift across situations, or value an output that is not captured by conventional notions of correctness. These examples motivate a more holistic view of sound AI in which model performance remains important, but is considered as one part of a larger human-AI system. I will discuss how this perspective can help us design technologies that are not only more accurate but also more adaptive, useful, and effective in the lives of the people they are intended to support.
More info