Signal Processing · Embedded Intelligence · 2026 · Engineering Project
WYSH
What You See is What You Hear
Enhancing the audio of the object in the FoV in real-time using adaptive beamforming.
1st Place
Result
2025
Year
Beamforming
Domain
Overview
Audio-Visual Zooming is an intelligent acoustic focusing system designed to enhance a selected sound source in real time. Unlike conventional microphones that capture every sound within their range, the system uses microphone arrays and adaptive beamforming techniques to selectively enhance audio from a desired direction while suppressing surrounding noise.
The project combines embedded signal processing with spatial audio techniques to create an 'audio zoom' experience that complements visual zoom, enabling clearer speech capture and improved situational awareness in applications such as conferencing, surveillance, and assistive technologies.
Problem
Traditional microphones capture all surrounding sounds equally, making it difficult to isolate a specific speaker or sound source in noisy environments. While cameras provide optical zoom to focus visually, there is no equivalent mechanism for selectively focusing audio. This project addresses that challenge by developing an adaptive beamforming system capable of enhancing a chosen sound source while reducing unwanted background interference in real time.
Process
Array capture
A microphone array captures multi-channel audio in sync across the room, giving the system a spatial view of where sound is coming from, not just what sound exists.
Adaptive beamforming
Adaptive beamforming algorithms dynamically steer the microphone array's sensitivity toward a selected sound source, allowing the system to enhance desired audio while suppressing unwanted signals from other directions.
Real-time embedded pipeline
The signal-processing pipeline was designed for real-time execution on embedded hardware, enabling low-latency audio enhancement suitable for interactive applications.
Coordinated audio-visual output
The enhanced audio output is synchronized with visual zoom interactions, providing a seamless user experience where both audio and video focus together on the selected subject.
