A CONFORMER-BASED ENSEMBLE APPROACH FOR SOUND EVENT LOCALIZATION AND DETECTION FOR STEREO DATA

Arjun Bahuguna¹, Rahul Peter^2
1Universitat Pompeu Fabra, Dept. of Engineering, Barcelona, 08018, Spain, ²Aalto University, Electrical Engineering Dept., Espoo, 20150, Finland

Abstract

This report presents our approach to task 3 of the DCASE Challenge 2025, which focuses on the localization and detection of stereo sound events (SELD) in regular video content. We propose a three-part ensemble model that operates in the audio domain and outperforms the official baseline. To address class imbalance in the STARSS23 dataset, we explore synthetic data generation using SpatialScaper and apply data augmentation techniques such as channel-swapping and time-domain remixing. Our proposed system achieves an F-score of 28%, DOA error of 17.3°, and relative distance error of 0.43 on the development data set. We conclude by suggesting possible future enhancements.

EXPLOITING STEREO SPATIAL PROPERTIES WITH RESNET-CONFORMERS FOR ROBUST EVENT DETECTION AND LOCALIZATION

Banerjee Mohor¹, Nagisetty Srikanth², Han Boon Teo^2
1Nanyang Technological University, ²Panasonic R&D Center Singapore

Abstract

This technical report presents our submission for Task 3, Track A of the DCASE 2025 Challenge: Stereo Sound Event Localization and Detection (SELD) in regular video content. Our system focuses exclusively on the audio-only track, leveraging stereo audio inputs for precise SELD. At the heart of our framework lies a ResNet-Conformer architecture augmented with pre-trained ONE-PEACE embeddings. This powerful combination processes a rich set of spatial and spectral features, including Mel spectrograms, Interaural Phase Differences (IPDs), Interaural Level Differences (ILDs), and Generalized Cross-Correlation with Phase Transform (GCC-PHAT). To optimize performance across multiple SELD tasks - namely Direction of Arrival (DoA) estimation, sound event detection (SED), and sound distance estimation (SDE) - we employ a modular training strategy: separate modules are trained for DoA estimation and SDE, with their outputs fused through a joint prediction scheme. The ONE-PEACE embeddings are integrated alongside the ResNet-Conformer outputs and jointly processed to enhance downstream task performance further. We evaluate our system on the DCASE 2025 Task 3 dev-test set, derived from the Sony-TAu Realistic Spatial Soundscapes 2023 (STARSS23) dataset. Our method achieves an F-score of 48.2%, representing a substantial 25.42% improvement over the official baseline, thereby demonstrating the effectiveness of our approach.

Stereo Sound Event Localization and Detection with Onscreen/offscreen Classification

Kazuki Shimada¹, Archontis Politis², Iran R. Roman³, Parthasaarathy Sudarsanam², David Diaz-Guerra², Ruchi Pandey², Kengo Uchida¹, Yuichiro Koyama⁴, Naoya Takahashi⁵, Takashi Shibuya¹, Shusuke Takahashi⁴, Tuomas Virtanen², Yuki Mitsufuji^6,7
1Sony AI, Japan, ²Audio Research Group, Tampere University, Finland, ³Queen Mary University of London, UK, ⁴Sony Group Corporation, Japan, ⁵Sony AI, Switzerland, ⁶Sony AI, USA, ⁷Sony Group Corporation, USA

SPATIAL AND SEMANTIC EMBEDDING INTEGRATION FOR STEREO SOUND EVENT LOCALIZATION AND DETECTION IN REGULAR VIDEOS

Davide Berghi, Philip J. B. Jackson
CVSSP, University of Surrey, U.K.

Abstract

This report presents our systems submitted to the audio-only and audio-visual tracks of the DCASE2025 Task 3 Challenge: Stereo Sound Event Localization and Detection (SELD) in Regular Video Content. SELD is a complex task that combines temporal event classification with spatial localization, requiring reasoning across spatial, temporal, and semantic dimensions. The last is arguably the most challenging to model. Traditional SELD architectures rely on multichannel input, which limits their ability to leverage large-scale pre-training due to data constraints. To address this, we enhance standard SELD architectures with semantic information by integrating pre-trained, contrastive language-aligned models: CLAP for audio and OWL-ViT for visual inputs. These embeddings are incorporated into a modified Conformer module tailored for multimodal fusion, which we refer to as the Cross-Modal Conformer. Additionally, we incorporate autocorrelation-based acoustic features to improve distance estimation. We pre-train our models on curated synthetic audio and audio-visual datasets and apply a left-right channel swapping augmentation to further increase the training data. Both our audio-only and audio-visual systems substantially outperform the challenge baselines on the development set, demonstrating the effectiveness of our strategy. Performance is further improved through model ensembling and a visual post-processing step based on human keypoints. Future work will investigate the contribution of each modality and explore architectural variants to further enhance results.

MULTI-ACCDOA-BASED SELD IN STEREO AUDIO: FEATURE EXTRACTION AND DATA AUGMENTATION STRATEGIES

Bingnan Duan, Yinhuan Dong, Liuyuan Na
The University of Edinburgh, School of Engineering, Edinburgh, UK

Abstract

This technical report describes the proposed system submitted to the DCASE2025 Task3: Stereo sound event localization and detection in regular video content (Track A: Audio-only inference). To improve SELD performance, we replace the convolutional blocks in the baseline model with ResNet blocks, extract a 3-channel input feature consisting of log-mel spectrograms and short-term power of autocorrelation (stpACC), and employ two data augmentation techniques: Time Masking and Frame Shuffle. Our system uses the Multi-ACCDOA output representation with an ADPIT loss function to support overlapping sound events. Evaluated on the development dataset, our proposed method achieves significant improvements over the official baseline across F1-score, DOA error, and relative distance error.

THE SYSTEM FOR DCASE 2025 SOUND EVENT LOCALIZATION AND DETECTION CHALLENGE

Chengnuo Sun, Lijian Gao
Jiangsu University, Zhenjiang, China

Abstract

This technical report gives an overview of our system for task3 with audiovisual of the DCASE 2025 challenge. We propose a Sound Event Localization and Detection (SELD) system for stereo sound event localization and detection in regular video content. Compared with the baseline, the proposed method pays more attention to the temporal relationship between modalities. We evaluated our methods on the dev-test set of the Sony-TAu Realistic Spatial Soundscapes 2023 (STARSS23) dataset and we achieve significant improvements over the baseline method.

THE NERC-SLIP SYSTEM FOR STEREO SOUND EVENT LOCALIZATION AND DETECTION IN REGULAR VIDEO CONTENT OF DCASE 2025 CHALLENGE

Qing Wang¹, Hengyi Hong¹, Ruoyu Wei³, Lin Li², Yuxuan Dong¹, Mingqi Cai², Xin Fang², Jiangzhao Wu³, Jun Du^1
1University of Science and Technology of China, Hefei, China, ²iFLYTEK, Hefei, China, ³National Intelligent Voice Innovation Center, Hefei, China

Abstract

This technical report details our submission system for Task 3 of the DCASE 2025 Challenge, which focuses on sound event localization and detection (SELD) in regular video content with stereo audio. In addition to estimating the direction of arrival (DOA) and distance of sound sources, the audio-visual SELD task requires predicting whether the sound source is on-screen. For the audio-only track, we used two-channel log-Mel spectrogram features from stereo audio as model inputs. We adapted the audio-visual pixel swapping (AVPS) technique from first-order Ambisonics (FOA) to stereo format through left-right channel swapping coupled with horizontal video pixel transposition, effectively doubling the training data. Our architecture implemented three specialized models for DOA, distance, and source coordinates estimation tasks, subsequently integrated through a joint prediction framework. The audio-visual track utilized a ResNet-50 model pre-trained on ImageNet for visual feature extraction, enhanced by a teacher-student learning paradigm for cross-modal knowledge distillation. To improve on-screen event detection, we developed a novel two-stage visual post-processing method. Our methods were evaluated using the development set of the DCASE 2025 Task 3.

STEREO SOUND EVENT LOCALIZATION AND DETECTION BASED ON PSELDNET PRETRAINING AND BIMAMBA SEQUENCE MODELING

Wenmiao Gao¹, Yang Xiao^2
1Denmark Technical University, 2800 Kgs. Lyngby, Denmark, ²The University of Melbourne, Melbourne, Australia

Abstract

Pre-training methods have achieved significant performance improvements in sound event localization and detection (SELD) tasks, but existing Transformer-based models suffer from high computational complexity. In this work, we propose a stereo sound event localization and detection system based on pre-trained PSELDnet and bidirectional Mamba sequence modeling. We replace the Conformer module with a BiMamba module and introduce asymmetric convolutions to more effectively model the spatiotemporal relationships between time and frequency dimensions. Experimental results demonstrate that the proposed method achieves significantly better performance than the baseline and the original PSELDnet with Conformer decoder architecture on the DCASE2025 Task 3 development dataset, while also reducing computational complexity. These findings highlight the effectiveness of the BiMamba architecture in addressing the challenges of the SELD task.

GISP@HEU'S SUBMISSION TO THE DCASE 2025 CHALLENGE: STEREO SELD TASK

Congyi Fan¹, Shitong Fan¹, Feiyang Xiao¹, Wenbo Wang², Xinyi Che³, Qiaoxi Zhu⁴, Jian Guan^1
1Group of Intelligent Signal Processing (GISP), College of Computer Science and Technology, Harbin Engineering University, Harbin, China, ²Faculty of Computing, Harbin Institute of Technology, Harbin, China, ³Sichuan University, ⁴University of Technology Sydney, Ultimo, Australia

Abstract

This technical report presents our submission to Task 3 of the DCASE 2025 Challenge. To enhance the model's generalization ability, we adopt the official synthetic data generation pipeline to expand the training set. In addition, SpecAugment is applied for data augmentation to improve event recognition performance. To address the challenges of ambiguous localization and long-range temporal dependencies inherent in stereo SELD, we use the Mamba architecture, which effectively captures both local and global temporal dynamics, thereby improving overall system performance.

STEREO SOUND EVENT LOCALIZATION AND DETECTION WITH SOURCE DISTANCE ESTIMATION USING DATA-DRIVEN RESNET-CONFORMER ENSEMBLE

Changjiang He^1,2, Jian Chen¹, Siyao Cheng^1,2, Jiahua Bao^1,2, Jie Liu^1,2
1Harbin Institute of Technology, Faculty of Computing, China, ²State Key Laboratory of Smart Farm Technologies and Systems, China

Abstract

This technical report presents our submitted system for Task 3 of the Detection and Classification of Acoustic Scenes and Events (DCASE) 2025 Challenge: Stereo Sound Event Localization and Detection in Regular Video Content (SELD). The DCASE task 3 includes two tracks, and we participate exclusively in the audio-only track. First, we perform data augmentation by employing audio channel swapping (ACS) and data simulation techniques, expanding the dataset to 3.7 times its original size. Subsequently, a single ResNet-Conformer model is used to perform SELD predictions. To further optimize the model and submit multiple model ensemble solutions, we fine-tuned it on the original dataset after training it on the augmented dataset. During model ensemble, we integrate two models—SED-DoA and SED-SDE. Our approach is evaluated on the development test set of the dataset.

SOUND EVENT LOCALIZATION AND DETECTION MODEL WITH ATTENTION-BASED NEURAL NETWORKS AND DATA MODELING

Gwantae Kim
Samsung Electronics, Suwon, South Korea

Abstract

The technical report presents our submission system for task 3 of the DCASE 2025 Challenge, which tackles sound event localization and detection(SELD) problems in regular video content and stereo audio contents. In this report, we introduce data preparation and augmentation method, neural network structure, post-processing, and model ensemble strategy.

ResNet-Conformer for Stereo Sound Event Localization and Distance Estimation in DCASE 2025 task3

Jehyun Park, Hyeonuk Nam, Yong-Hwa Park
Korea Advanced Institute of Science and Technology, South Korea

Abstract

In the DCASE 2025 Task 3 Track A challenge, we propose a ResNet-Conformer architecture for stereo sound event localization and detection (SELD) with integrated sound distance estimation (SDE). We develop two complementary systems. The first system follows a two-stage training strategy, where the model is initially trained to perform sound event detection (SED) and direction-of-arrival (DOA) estimation, and then fine-tuned to also predict source distance. The second system is based on a dual-branch ensemble that combines a model trained for SED and DOA with another model trained for SED and SDE. Both systems share a common backbone consisting of a ResNet-based convolutional encoder followed by an 8-layer Conformer stack, with separate output branches for SED (sigmoid), DOA (tanh), and SDE (ReLU). To enhance robustness, we apply audio channel swapping (ACS) and FilterAugment as data augmentation techniques. Evaluation on the DCASE 2025 Task 3 development set demonstrates that the proposed ensemble system improves overall SELD performance.

A MULTI-LEVEL FEATURE EXTRACTION NETWORK FOR SOUND EVENT LOCALIZATION AND DETECTION IN DCASE 2025 TASK 3

QingJing Wan^1,2, Ying Hu^1,2, Jie Liu^1,2, Qiong Wu^1,2, Qin Yang^1,2, WenTao Zhou^1,2, Tianqing Zhou^1,2, Nannan Teng^1,2, Fangxu Chen^1,2, Zijun Chen^1,2
1Xinjiang University, School of Information Science and Engineering, Urumqi, China, ²Key Laboratory of Signal Detection and Processing in Xinjiang, Urumqi, China

Abstract

This technical report describes our submission system for Task 3 of the DCASE 2025 Challenge: stereo sound event localization and detection (SELD) in regular video content. We participate in the audio-only track. Our system adopts a Multi-Level Feature Extraction Network, which consists of three main components. First, a Feature Extraction Enhancement module(FEEM) is used to extract fine-grained and meaningful features at multiple hierarchical levels, improving the model's ability to handle both sub-tasks: Direction of Arrival (DOA) estimation and Sound Event Detection (SED). Second, a Feature Fusion module(FFM) is employed to integrate multi-level features, further enhancing the representational capacity of the network. Finally, several data augmentation strategies are applied to improve the robustness of the network. Experimental results on the DCASE 2025 Task 3 stereo SELD dataset demonstrate the effectiveness of the proposed system.

A STEREO SOUND EVENT LOCALIZATION AND DETECTION METHOD BASED ON FEATURE FUSION AND TWO-STAGE TRAINING

Digao Wu, Ming Zhu
Huazhong University of Science and Technology, School of Electronic Information and Communications, Wuhan, China

Abstract

This technical report presents our system for Task 3 of the DCASE 2025 Challenge: Stereo Sound Event Localization and Detection in Regular Video Content. The task requires predicting the activity, azimuth, and distance of sound events using stereo audio. We participate in the audio-only track. We propose a stereo SELD model based on the ResNet-Conformer structure, integrating channel-wise attention and feature fusion, with outputs represented in the ACCDOA format. To enhance model performance, we augment the training data with additional stereo audio segments sampled from the official DCASE 2024 synthetic dataset. We apply several data augmentation techniques and adopt a two-stage training strategy to improve generalization and performance on real data. A dynamic thresholding method is also introduced during inference to further boost the prediction accuracy. The experimental results on the official development dataset show that our proposed system outperforms the baseline in all evaluation metrics.

IMPROVING STEREO 3D SOUND EVENT LOCALIZATION AND DETECTION: PERCEPTUAL FEATURES, STEREO-SPECIFIC DATA AUGMENTATION, AND DISTANCE NORMALIZATION

Jun-Wei Yeow, Ee-Leng Tan, Santi Peksi, Woon-Seng Gan
Smart Nation TRANS Lab, Nanyang Technological University, Singapore

Abstract

This technical report presents our submission to Task 3 of the DCASE 2025 Challenge: Stereo Sound Event Localization and Detection (SELD) in Regular Video Content. We address the audio-only task in this report and introduce several key contributions. First, we design perceptually-motivated input features that improve event detection, sound source localization, and distance estimation. Second, we adapt augmentation strategies specifically for the intricacies of stereo audio, including channel swapping and time-frequency masking. We also incorporate the recently proposed FilterAugment technique that has yet to be explored for SELD work. Lastly, we apply a distance normalization approach during training to stabilize regression targets. Experiments on the stereo STARSS23 dataset demonstrate consistent performance gains across all SELD metrics. Code to replicate our work is available in this repository.

THE MAMBA-BASED SYSTEM FOR DCASE 2025 SOUND EVENT LOCALIZATION AND DETECTION CHALLENGE

Rendong Pi, Xiang Yu
The Hong Kong Polytechnic University, Mechanical Engineering Dept., Hung Hom, Kowloon, Hong Kong, China

Abstract

This technical report gives an overview of our system for task 3 in DCASE 2025 challenge. We propose a stereo sound event loclaization and detection (SELD) system using the mel spectral and intensity vector as the input features. We construct a Mamba-based network and achieve the significant improvements compared to the beseline system with few data augmentation methods. We conduct the performance evaluation on the dev-test of the Stereo SELD dataset extracted from the Sony-TAu Realistic Spatial Soundscapes 2023 (STARSS23) dataset.

ENHANCING STEREO SOUND EVENT LOCALIZATION AND DETECTION THROUGH PRETRAINED AUDIO REPRESENTATIONS AND HYBRID ARCHITECTURES

Tianbo Zhao, Zerui Han, Mengmei Liu
Xiaomi Corporation, MITC-Multimodal generation, Beijing, China

Abstract

The technical report presents our submission system for Task 3 of the DCASE 2025 Challenge: Stereo sound event localization and detection (SELD) in regular video content. This year we participate in the audio-only track. We propose a method that decomposes the SELD task into two sub-tasks. For the detection task, we employ the pre-trained Dasheng model, which is a high-performing audio encoder. For the localization task, we utilize the ResNet-Conformer architecture, which has demonstrated excellent performance in recent years' DCASE tasks. We evaluated our method on the dev-test set of the development dataset. The results show that our approach outperforms the baseline.