Data-Efficient Low-Complexity Acoustic Scene Classification


Challenge results

Task description

The goal of the acoustic scene classification task is to classify recordings into one of the ten predefined acoustic scene classes. This task continues the Acoustic Scene Classification tasks from previous editions of the DCASE Challenge, with a slight shift of focus. This year, the task concentrates on three challenging aspects: (1) a recording device mismatch, (2) low-complexity constraints, and (3) the limited availability of labeled data for training.

More detailed task description can be found in the task description page

Teams ranking

Submission information Rank Entry ranks per split
(maximum among entries)
Accuracies per split
(maximum among entries)
Rank Submission label Name Technical
Report
Official
rank
Rank
score
Split
5%
Split
10%
Split
25%
Split
50%
Split
100%
Split
5%
Split
10%
Split
25%
Split
50%
Split
100%
Auzanneau_CEA Yolovote Auzanneau2024 18 37.38 38 38 38 38 38 33.07 35.31 37.69 39.76 41.08
BAI_JLESS BAI_T1_S1 Bai2024 14 52.39 31 28 23 27 29 44.75 49.31 54.85 55.77 57.29
Cai_XJTLU TFSN t=3 Cai2024 4 56.35 9 9 5 8 1 48.91 53.16 58.09 59.47 62.12
Chen_GXU SENet Chen2024 15 52.12 29 23 26 32 31 45.20 49.82 53.90 54.72 56.96
Chen_SCUT BC-PACN-64 Chen2024a 12 52.74 33 30 28 21 16 43.94 48.89 53.76 56.96 60.14
Gao_UniSA base_fl Gao2024 11 52.82 27 25 31 22 24 45.91 49.45 53.27 56.74 58.73
Han_SJTUTHU Lin_c96c64 Bing2024 1 58.46 1 1 1 2 2 54.35 56.69 59.09 60.38 61.82
MALACH24_JKU CR_P_B_CPM David2024 2 57.19 4 4 6 4 7 51.95 54.46 58.01 60.05 61.51
OO_NTUPRDCSG MofleRes1 Oo2024 7 54.83 14 18 16 14 17 48.52 51.43 55.87 58.42 59.91
Park_KT Quant Park2024 6 55.40 11 12 12 17 15 48.86 52.44 57.30 58.09 60.32
DCASE2024 baseline Baseline 17 50.73 32 36 37 34 33 44.00 46.95 51.47 54.40 56.84
Shao_NEPUMSE NEPUMSE Shao2024 3 57.15 6 7 4 1 3 51.38 53.75 58.31 60.61 61.71
Surkov_ITMO Large Surkov2024 13 52.65 18 20 25 33 36 48.03 51.18 54.29 54.63 55.15
Tan_CISS NTUCISS_T1 Tan2024 16 51.71 22 27 35 35 35 46.50 49.32 52.62 54.03 56.07
Truchan_LUH Recursiv Truchan2024 9 53.52 25 24 21 18 20 46.02 49.64 55.15 57.43 59.39
Werning_UPBNT UPBNT Werning2024 8 54.35 8 11 18 26 26 49.21 52.51 55.49 56.20 58.34
Yan_NPU Yan_NPU Yan2024 10 52.94 21 29 33 25 23 47.51 49.07 52.97 56.36 58.79
Yeo_NTU KDEnsemble Yeo2024 5 56.12 10 8 10 9 6 48.88 53.29 57.47 59.36 61.59

Systems ranking

Submission information Overall rank Ranks per split Complexity ranks Evaluation dataset / Accuracy with 95% confidence interval Development dataset / Accuracy
Rank Submission label Name Technical
Report
Rank Rank score Split
5%
Split
10%
Split
25%
Split
50%
Split
100%
Memory
rank
MACs
rank
Split
5%
Split
10%
Split
25%
Split
50%
Split
100%
Split
5%
Split
10%
Split
25%
Split
50%
Split
100%
Auzanneau_CEA_task1_1 Yolovote Auzanneau2024 38 37.4 38 38 38 38 38 6 3 33.1 (32.8 - 33.3) 35.3 (35.0 - 35.6) 37.7 (37.4 - 38.0) 39.8 (39.5 - 40.0) 41.1 (40.8 - 41.4) 31.4 34.4 38.4 39.0 41.3
BAI_JLESS_task1_1 BAI_T1_S1 Bai2024 32 52.0 31 35 24 27 30 16 12 44.7 (44.5 - 45.0) 47.8 (47.5 - 48.1) 54.5 (54.3 - 54.8) 55.8 (55.5 - 56.1) 57.1 (56.9 - 57.4) 42.6 47.1 54.0 57.0 59.4
BAI_JLESS_task1_2 BAI_T1_S2 Bai2024 37 50.7 34 37 36 36 34 16 12 43.8 (43.6 - 44.1) 46.9 (46.6 - 47.2) 52.0 (51.8 - 52.3) 53.9 (53.6 - 54.2) 56.7 (56.5 - 57.0) 42.9 46.9 51.7 56.2 59.3
BAI_JLESS_task1_3 BAI_T1_S3 Bai2024 30 52.1 35 28 23 31 29 16 12 43.7 (43.4 - 43.9) 49.3 (49.0 - 49.6) 54.9 (54.6 - 55.1) 55.2 (54.9 - 55.4) 57.3 (57.0 - 57.6) 42.2 49.2 54.1 56.4 60.3
Cai_XJTLU_task1_1 TFSN t=1 Cai2024 12 55.4 20 14 14 8 14 15 16 47.5 (47.3 - 47.8) 52.4 (52.1 - 52.7) 57.0 (56.8 - 57.3) 59.5 (59.2 - 59.8) 60.6 (60.3 - 60.9) 47.4 52.1 57.5 61.1 62.4
Cai_XJTLU_task1_2 TFSN t=3 Cai2024 10 56.0 9 15 5 12 1 15 16 48.9 (48.6 - 49.2) 52.0 (51.8 - 52.3) 58.1 (57.8 - 58.4) 59.0 (58.8 - 59.3) 62.1 (61.8 - 62.4) 49.0 52.3 57.9 60.7 62.9
Cai_XJTLU_task1_3 TFSN t=12 Cai2024 11 55.9 17 9 8 11 12 15 16 48.4 (48.1 - 48.6) 53.2 (52.9 - 53.4) 57.6 (57.3 - 57.8) 59.2 (58.9 - 59.5) 61.1 (60.8 - 61.4) 47.9 52.3 57.5 60.1 62.8
Chen_GXU_task1_1 SENet Chen2024 29 52.1 29 23 26 32 31 19 15 45.2 (44.9 - 45.5) 49.8 (49.5 - 50.1) 53.9 (53.6 - 54.2) 54.7 (54.4 - 55.0) 57.0 (56.7 - 57.2) 43.2 48.7 53.8 56.8 57.3
Chen_SCUT_task1_1 BC-PACN-64 Chen2024a 28 52.5 33 34 28 21 16 8 4 43.9 (43.7 - 44.2) 47.8 (47.5 - 48.1) 53.8 (53.5 - 54.0) 57.0 (56.7 - 57.2) 60.1 (59.9 - 60.4) 47.1 52.4 58.0 60.9 63.7
Chen_SCUT_task1_2 BC-5-PACB-48 Chen2024a 35 51.7 37 32 32 28 28 10 2 43.1 (42.8 - 43.3) 48.2 (48.0 - 48.5) 53.3 (53.0 - 53.5) 55.7 (55.4 - 56.0) 58.1 (57.8 - 58.3) 46.5 53.1 57.0 60.2 61.5
Chen_SCUT_task1_3 BC-PACN-48 Chen2024a 31 52.0 36 30 29 23 27 1 1 43.1 (42.9 - 43.4) 48.9 (48.6 - 49.2) 53.6 (53.4 - 53.9) 56.5 (56.2 - 56.8) 58.1 (57.8 - 58.4) 46.4 52.6 57.4 59.8 62.0
Gao_UniSA_task1_1 base_fl Gao2024 25 52.8 27 25 31 22 24 11 17 45.9 (45.6 - 46.2) 49.5 (49.2 - 49.7) 53.3 (53.0 - 53.6) 56.7 (56.5 - 57.0) 58.7 (58.5 - 59.0) 44.3 47.9 53.1 56.3 59.6
Han_SJTUTHU_task1_1 Agp_c64 Bing2024 3 58.1 3 2 3 2 8 17 21 53.4 (53.1 - 53.7) 56.4 (56.1 - 56.7) 58.6 (58.4 - 58.9) 60.4 (60.1 - 60.7) 61.5 (61.2 - 61.7) 52.2 55.8 58.9 61.4 62.2
Han_SJTUTHU_task1_2 Lin_c96c64 Bing2024 1 58.2 1 3 2 3 2 14 13 54.3 (54.1 - 54.6) 55.8 (55.5 - 56.1) 58.9 (58.7 - 59.2) 60.2 (60.0 - 60.5) 61.8 (61.5 - 62.1) 52.7 55.8 58.7 60.4 61.7
Han_SJTUTHU_task1_3 Linear_c96 Bing2024 2 58.2 2 1 1 5 9 18 20 54.0 (53.7 - 54.2) 56.7 (56.4 - 57.0) 59.1 (58.8 - 59.4) 60.0 (59.7 - 60.3) 61.3 (61.1 - 61.6) 53.0 56.9 59.1 61.0 61.3
MALACH24_JKU_task1_1 CR_B_CPM David2024 5 57.0 5 4 6 7 7 11 15 51.6 (51.3 - 51.8) 54.5 (54.2 - 54.7) 58.0 (57.7 - 58.3) 59.7 (59.4 - 60.0) 61.5 (61.2 - 61.8) 49.7 53.7 57.5 60.2 62.2
MALACH24_JKU_task1_2 CR_P_B_CPM David2024 6 57.0 4 5 9 4 10 11 15 52.0 (51.7 - 52.2) 54.0 (53.7 - 54.3) 57.5 (57.3 - 57.8) 60.0 (59.8 - 60.3) 61.3 (61.0 - 61.5) 50.5 53.1 56.8 59.2 60.6
MALACH24_JKU_task1_3 CR_M_CPM David2024 7 56.5 7 6 11 13 13 11 15 51.2 (50.9 - 51.4) 54.0 (53.7 - 54.3) 57.3 (57.0 - 57.6) 59.0 (58.7 - 59.3) 60.9 (60.6 - 61.2) 49.5 53.5 56.9 59.6 61.5
OO_NTUPRDCSG_task1_1 MofleRes1 Oo2024 17 54.8 14 18 17 15 18 7 14 48.5 (48.2 - 48.8) 51.4 (51.1 - 51.7) 55.8 (55.5 - 56.1) 58.4 (58.1 - 58.6) 59.9 (59.6 - 60.1) 47.6 51.3 56.7 60.0 62.2
OO_NTUPRDCSG_task1_2 MofleRes2 Oo2024 15 54.8 16 19 16 14 17 7 14 48.4 (48.1 - 48.7) 51.3 (51.1 - 51.6) 55.9 (55.6 - 56.2) 58.4 (58.1 - 58.7) 59.9 (59.6 - 60.2) 47.6 51.5 57.2 60.1 62.5
Park_KT_task1_1 Quant Park2024 20 53.7 11 13 27 24 32 3 11 48.9 (48.6 - 49.1) 52.4 (52.1 - 52.7) 53.8 (53.5 - 54.1) 56.4 (56.1 - 56.7) 56.9 (56.6 - 57.2) 49.7 52.4 55.8 57.6 58.8
Park_KT_task1_2 Small Park2024 13 55.4 12 12 12 17 15 20 10 48.7 (48.4 - 48.9) 52.4 (52.2 - 52.7) 57.3 (57.0 - 57.6) 58.1 (57.8 - 58.4) 60.3 (60.0 - 60.6) 49.7 53.2 58.4 59.4 61.9
Park_KT_task1_3 KD Park2024 16 54.8 15 16 15 20 19 20 10 48.5 (48.2 - 48.8) 51.9 (51.6 - 52.1) 56.6 (56.4 - 56.9) 57.2 (56.9 - 57.5) 59.8 (59.5 - 60.1) 47.8 53.1 57.6 59.2 61.6
DCASE2024 baseline Baseline 36 50.7 32 36 37 34 33 11 15 44.0 (43.7 - 44.3) 46.9 (46.7 - 47.2) 51.5 (51.2 - 51.8) 54.4 (54.1 - 54.7) 56.8 (56.6 - 57.1) 42.4 45.3 50.3 53.2 57.0
Shao_NEPUMSE_task1_1 NEPUMSE Shao2024 4 57.2 6 7 4 1 3 5 6 51.4 (51.1 - 51.7) 53.8 (53.5 - 54.0) 58.3 (58.0 - 58.6) 60.6 (60.3 - 60.9) 61.7 (61.4 - 62.0) 51.1 53.5 59.7 62.3 64.6
Shao_NEPUMSE_task1_2 NEPUMSE Shao2024 8 56.1 13 10 7 6 5 9 7 48.5 (48.2 - 48.8) 53.0 (52.7 - 53.3) 57.7 (57.4 - 58.0) 59.8 (59.6 - 60.1) 61.6 (61.3 - 61.9) 48.7 53.1 59.5 62.1 65.1
Shao_NEPUMSE_task1_3 NEPUMSE Shao2024 14 55.2 24 17 13 10 4 13 5 46.2 (45.9 - 46.5) 51.8 (51.5 - 52.1) 57.2 (56.9 - 57.4) 59.2 (58.9 - 59.5) 61.7 (61.4 - 61.9) 46.6 51.4 58.3 62.2 65.1
Surkov_ITMO_task1_1 Large Surkov2024 27 52.7 18 20 25 33 36 11 8 48.0 (47.7 - 48.3) 51.2 (50.9 - 51.5) 54.3 (54.0 - 54.6) 54.6 (54.3 - 54.9) 55.2 (54.9 - 55.4) 46.2 49.6 53.5 54.7 55.8
Surkov_ITMO_task1_2 Small Surkov2024 33 51.9 19 21 34 37 37 11 8 47.8 (47.5 - 48.1) 50.3 (50.0 - 50.5) 52.7 (52.5 - 53.0) 53.9 (53.6 - 54.1) 55.0 (54.7 - 55.3) 46.5 49.0 51.8 54.0 55.6
Tan_CISS_task1_1 NTUCISS_T1 Tan2024 34 51.7 22 27 35 35 35 11 15 46.5 (46.2 - 46.8) 49.3 (49.0 - 49.6) 52.6 (52.3 - 52.9) 54.0 (53.7 - 54.3) 56.1 (55.8 - 56.3) 46.0 49.5 53.9 56.8 59.1
Truchan_LUH_task1_1 Isotropic Truchan2024 22 53.1 28 33 22 18 20 4 19 45.3 (45.0 - 45.6) 48.1 (47.9 - 48.4) 55.0 (54.7 - 55.3) 57.4 (57.1 - 57.7) 59.4 (59.1 - 59.7) 44.2 47.7 55.5 58.4 61.0
Truchan_LUH_task1_2 Recursiv Truchan2024 26 52.7 25 24 30 29 22 4 19 46.0 (45.7 - 46.3) 49.6 (49.4 - 49.9) 53.4 (53.1 - 53.7) 55.5 (55.2 - 55.8) 58.8 (58.5 - 59.1) 45.8 50.5 52.7 55.4 60.0
Truchan_LUH_task1_3 RSC Truchan2024 24 52.9 30 31 21 19 25 4 19 44.9 (44.7 - 45.2) 48.9 (48.6 - 49.2) 55.1 (54.9 - 55.4) 57.2 (56.9 - 57.5) 58.4 (58.1 - 58.7) 44.8 48.8 54.9 56.9 58.9
Werning_UPBNT_task1_1 UPBNT Werning2024 18 54.4 8 11 18 26 26 11 15 49.2 (48.9 - 49.5) 52.5 (52.2 - 52.8) 55.5 (55.2 - 55.8) 56.2 (55.9 - 56.5) 58.3 (58.1 - 58.6) 46.7 50.3 53.8 55.0 60.0
Yan_NPU_task1_1 Yan_NPU Yan2024 23 52.9 21 29 33 25 23 12 18 47.5 (47.2 - 47.8) 49.1 (48.8 - 49.4) 53.0 (52.7 - 53.3) 56.4 (56.1 - 56.6) 58.8 (58.5 - 59.1) 45.7 48.3 52.1 55.7 59.8
Yeo_NTU_task1_1 24BCBL Yeo2024 21 53.1 23 26 20 30 21 2 9 46.3 (46.0 - 46.5) 49.4 (49.1 - 49.7) 55.2 (55.0 - 55.5) 55.4 (55.1 - 55.7) 59.3 (59.1 - 59.6) 43.7 48.7 53.4 55.5 57.6
Yeo_NTU_task1_2 KDEnsemble Yeo2024 9 56.1 10 8 10 9 6 11 15 48.9 (48.6 - 49.2) 53.3 (53.0 - 53.6) 57.5 (57.2 - 57.8) 59.4 (59.1 - 59.6) 61.6 (61.3 - 61.9) 48.0 53.0 56.8 59.8 61.7
Yeo_NTU_task1_3 TFS Yeo2024 19 54.2 26 22 19 16 11 11 15 46.0 (45.7 - 46.3) 50.2 (49.9 - 50.5) 55.4 (55.1 - 55.7) 58.3 (58.0 - 58.6) 61.1 (60.8 - 61.4) 44.9 50.0 55.6 59.7 62.2

Split 5%

Submission information Evaluation dataset Development dataset
Rank Submission label Name Technical
Report
Split 5
Rank
Memory
rank
MACs
rank
Accuracy
with 95% confidence interval
Accuracy
Auzanneau_CEA_task1_1 Yolovote Auzanneau2024 38 6 3 33.1 (32.8 - 33.3) 31.4
BAI_JLESS_task1_1 BAI_T1_S1 Bai2024 31 16 12 44.7 (44.5 - 45.0) 42.6
BAI_JLESS_task1_2 BAI_T1_S2 Bai2024 34 16 12 43.8 (43.6 - 44.1) 42.9
BAI_JLESS_task1_3 BAI_T1_S3 Bai2024 35 16 12 43.7 (43.4 - 43.9) 42.2
Cai_XJTLU_task1_1 TFSN t=1 Cai2024 20 15 16 47.5 (47.3 - 47.8) 47.4
Cai_XJTLU_task1_2 TFSN t=3 Cai2024 9 15 16 48.9 (48.6 - 49.2) 49.0
Cai_XJTLU_task1_3 TFSN t=12 Cai2024 17 15 16 48.4 (48.1 - 48.6) 47.9
Chen_GXU_task1_1 SENet Chen2024 29 19 15 45.2 (44.9 - 45.5) 43.2
Chen_SCUT_task1_1 BC-PACN-64 Chen2024a 33 8 4 43.9 (43.7 - 44.2) 47.1
Chen_SCUT_task1_2 BC-5-PACB-48 Chen2024a 37 10 2 43.1 (42.8 - 43.3) 46.5
Chen_SCUT_task1_3 BC-PACN-48 Chen2024a 36 1 1 43.1 (42.9 - 43.4) 46.4
Gao_UniSA_task1_1 base_fl Gao2024 27 11 17 45.9 (45.6 - 46.2) 44.3
Han_SJTUTHU_task1_1 Agp_c64 Bing2024 3 17 21 53.4 (53.1 - 53.7) 52.2
Han_SJTUTHU_task1_2 Lin_c96c64 Bing2024 1 14 13 54.3 (54.1 - 54.6) 52.7
Han_SJTUTHU_task1_3 Linear_c96 Bing2024 2 18 20 54.0 (53.7 - 54.2) 53.0
MALACH24_JKU_task1_1 CR_B_CPM David2024 5 11 15 51.6 (51.3 - 51.8) 49.7
MALACH24_JKU_task1_2 CR_P_B_CPM David2024 4 11 15 52.0 (51.7 - 52.2) 50.5
MALACH24_JKU_task1_3 CR_M_CPM David2024 7 11 15 51.2 (50.9 - 51.4) 49.5
OO_NTUPRDCSG_task1_1 MofleRes1 Oo2024 14 7 14 48.5 (48.2 - 48.8) 47.6
OO_NTUPRDCSG_task1_2 MofleRes2 Oo2024 16 7 14 48.4 (48.1 - 48.7) 47.6
Park_KT_task1_1 Quant Park2024 11 3 11 48.9 (48.6 - 49.1) 49.7
Park_KT_task1_2 Small Park2024 12 20 10 48.7 (48.4 - 48.9) 49.7
Park_KT_task1_3 KD Park2024 15 20 10 48.5 (48.2 - 48.8) 47.8
DCASE2024 baseline Baseline 32 11 15 44.0 (43.7 - 44.3) 42.4
Shao_NEPUMSE_task1_1 NEPUMSE Shao2024 6 5 6 51.4 (51.1 - 51.7) 51.1
Shao_NEPUMSE_task1_2 NEPUMSE Shao2024 13 9 7 48.5 (48.2 - 48.8) 48.7
Shao_NEPUMSE_task1_3 NEPUMSE Shao2024 24 13 5 46.2 (45.9 - 46.5) 46.6
Surkov_ITMO_task1_1 Large Surkov2024 18 11 8 48.0 (47.7 - 48.3) 46.2
Surkov_ITMO_task1_2 Small Surkov2024 19 11 8 47.8 (47.5 - 48.1) 46.5
Tan_CISS_task1_1 NTUCISS_T1 Tan2024 22 11 15 46.5 (46.2 - 46.8) 46.0
Truchan_LUH_task1_1 Isotropic Truchan2024 28 4 19 45.3 (45.0 - 45.6) 44.2
Truchan_LUH_task1_2 Recursiv Truchan2024 25 4 19 46.0 (45.7 - 46.3) 45.8
Truchan_LUH_task1_3 RSC Truchan2024 30 4 19 44.9 (44.7 - 45.2) 44.8
Werning_UPBNT_task1_1 UPBNT Werning2024 8 11 15 49.2 (48.9 - 49.5) 46.7
Yan_NPU_task1_1 Yan_NPU Yan2024 21 12 18 47.5 (47.2 - 47.8) 45.7
Yeo_NTU_task1_1 24BCBL Yeo2024 23 2 9 46.3 (46.0 - 46.5) 43.7
Yeo_NTU_task1_2 KDEnsemble Yeo2024 10 11 15 48.9 (48.6 - 49.2) 48.0
Yeo_NTU_task1_3 TFS Yeo2024 26 11 15 46.0 (45.7 - 46.3) 44.9

Split 10%

%%

Submission information Evaluation dataset Development dataset
Rank Submission label Name Technical
Report
Split 10
Rank
Memory
rank
MACs
rank
Accuracy
with 95% confidence interval
Accuracy
Auzanneau_CEA_task1_1 Yolovote Auzanneau2024 38 6 3 35.3 (35.0 - 35.6) 34.4
BAI_JLESS_task1_1 BAI_T1_S1 Bai2024 35 16 12 47.8 (47.5 - 48.1) 47.1
BAI_JLESS_task1_2 BAI_T1_S2 Bai2024 37 16 12 46.9 (46.6 - 47.2) 46.9
BAI_JLESS_task1_3 BAI_T1_S3 Bai2024 28 16 12 49.3 (49.0 - 49.6) 49.2
Cai_XJTLU_task1_1 TFSN t=1 Cai2024 14 15 16 52.4 (52.1 - 52.7) 52.1
Cai_XJTLU_task1_2 TFSN t=3 Cai2024 15 15 16 52.0 (51.8 - 52.3) 52.3
Cai_XJTLU_task1_3 TFSN t=12 Cai2024 9 15 16 53.2 (52.9 - 53.4) 52.3
Chen_GXU_task1_1 SENet Chen2024 23 19 15 49.8 (49.5 - 50.1) 48.7
Chen_SCUT_task1_1 BC-PACN-64 Chen2024a 34 8 4 47.8 (47.5 - 48.1) 52.4
Chen_SCUT_task1_2 BC-5-PACB-48 Chen2024a 32 10 2 48.2 (48.0 - 48.5) 53.1
Chen_SCUT_task1_3 BC-PACN-48 Chen2024a 30 1 1 48.9 (48.6 - 49.2) 52.6
Gao_UniSA_task1_1 base_fl Gao2024 25 11 17 49.5 (49.2 - 49.7) 47.9
Han_SJTUTHU_task1_1 Agp_c64 Bing2024 2 17 21 56.4 (56.1 - 56.7) 55.8
Han_SJTUTHU_task1_2 Lin_c96c64 Bing2024 3 14 13 55.8 (55.5 - 56.1) 55.8
Han_SJTUTHU_task1_3 Linear_c96 Bing2024 1 18 20 56.7 (56.4 - 57.0) 56.9
MALACH24_JKU_task1_1 CR_B_CPM David2024 4 11 15 54.5 (54.2 - 54.7) 53.7
MALACH24_JKU_task1_2 CR_P_B_CPM David2024 5 11 15 54.0 (53.7 - 54.3) 53.1
MALACH24_JKU_task1_3 CR_M_CPM David2024 6 11 15 54.0 (53.7 - 54.3) 53.5
OO_NTUPRDCSG_task1_1 MofleRes1 Oo2024 18 7 14 51.4 (51.1 - 51.7) 51.3
OO_NTUPRDCSG_task1_2 MofleRes2 Oo2024 19 7 14 51.3 (51.1 - 51.6) 51.5
Park_KT_task1_1 Quant Park2024 13 3 11 52.4 (52.1 - 52.7) 52.4
Park_KT_task1_2 Small Park2024 12 20 10 52.4 (52.2 - 52.7) 53.2
Park_KT_task1_3 KD Park2024 16 20 10 51.9 (51.6 - 52.1) 53.1
DCASE2024 baseline Baseline 36 11 15 46.9 (46.7 - 47.2) 45.3
Shao_NEPUMSE_task1_1 NEPUMSE Shao2024 7 5 6 53.8 (53.5 - 54.0) 53.5
Shao_NEPUMSE_task1_2 NEPUMSE Shao2024 10 9 7 53.0 (52.7 - 53.3) 53.1
Shao_NEPUMSE_task1_3 NEPUMSE Shao2024 17 13 5 51.8 (51.5 - 52.1) 51.4
Surkov_ITMO_task1_1 Large Surkov2024 20 11 8 51.2 (50.9 - 51.5) 49.6
Surkov_ITMO_task1_2 Small Surkov2024 21 11 8 50.3 (50.0 - 50.5) 49.0
Tan_CISS_task1_1 NTUCISS_T1 Tan2024 27 11 15 49.3 (49.0 - 49.6) 49.5
Truchan_LUH_task1_1 Isotropic Truchan2024 33 4 19 48.1 (47.9 - 48.4) 47.7
Truchan_LUH_task1_2 Recursiv Truchan2024 24 4 19 49.6 (49.4 - 49.9) 50.5
Truchan_LUH_task1_3 RSC Truchan2024 31 4 19 48.9 (48.6 - 49.2) 48.8
Werning_UPBNT_task1_1 UPBNT Werning2024 11 11 15 52.5 (52.2 - 52.8) 50.3
Yan_NPU_task1_1 Yan_NPU Yan2024 29 12 18 49.1 (48.8 - 49.4) 48.3
Yeo_NTU_task1_1 24BCBL Yeo2024 26 2 9 49.4 (49.1 - 49.7) 48.7
Yeo_NTU_task1_2 KDEnsemble Yeo2024 8 11 15 53.3 (53.0 - 53.6) 53.0
Yeo_NTU_task1_3 TFS Yeo2024 22 11 15 50.2 (49.9 - 50.5) 50.0

Split 25%

Submission information Evaluation dataset Development dataset
Rank Submission label Name Technical
Report
Split 25
Rank
Memory
rank
MACs
rank
Accuracy
with 95% confidence interval
Accuracy
Auzanneau_CEA_task1_1 Yolovote Auzanneau2024 38 6 3 37.7 (37.4 - 38.0) 38.4
BAI_JLESS_task1_1 BAI_T1_S1 Bai2024 24 16 12 54.5 (54.3 - 54.8) 54.0
BAI_JLESS_task1_2 BAI_T1_S2 Bai2024 36 16 12 52.0 (51.8 - 52.3) 51.7
BAI_JLESS_task1_3 BAI_T1_S3 Bai2024 23 16 12 54.9 (54.6 - 55.1) 54.1
Cai_XJTLU_task1_1 TFSN t=1 Cai2024 14 15 16 57.0 (56.8 - 57.3) 57.5
Cai_XJTLU_task1_2 TFSN t=3 Cai2024 5 15 16 58.1 (57.8 - 58.4) 57.9
Cai_XJTLU_task1_3 TFSN t=12 Cai2024 8 15 16 57.6 (57.3 - 57.8) 57.5
Chen_GXU_task1_1 SENet Chen2024 26 19 15 53.9 (53.6 - 54.2) 53.8
Chen_SCUT_task1_1 BC-PACN-64 Chen2024a 28 8 4 53.8 (53.5 - 54.0) 58.0
Chen_SCUT_task1_2 BC-5-PACB-48 Chen2024a 32 10 2 53.3 (53.0 - 53.5) 57.0
Chen_SCUT_task1_3 BC-PACN-48 Chen2024a 29 1 1 53.6 (53.4 - 53.9) 57.4
Gao_UniSA_task1_1 base_fl Gao2024 31 11 17 53.3 (53.0 - 53.6) 53.1
Han_SJTUTHU_task1_1 Agp_c64 Bing2024 3 17 21 58.6 (58.4 - 58.9) 58.9
Han_SJTUTHU_task1_2 Lin_c96c64 Bing2024 2 14 13 58.9 (58.7 - 59.2) 58.7
Han_SJTUTHU_task1_3 Linear_c96 Bing2024 1 18 20 59.1 (58.8 - 59.4) 59.1
MALACH24_JKU_task1_1 CR_B_CPM David2024 6 11 15 58.0 (57.7 - 58.3) 57.5
MALACH24_JKU_task1_2 CR_P_B_CPM David2024 9 11 15 57.5 (57.3 - 57.8) 56.8
MALACH24_JKU_task1_3 CR_M_CPM David2024 11 11 15 57.3 (57.0 - 57.6) 56.9
OO_NTUPRDCSG_task1_1 MofleRes1 Oo2024 17 7 14 55.8 (55.5 - 56.1) 56.7
OO_NTUPRDCSG_task1_2 MofleRes2 Oo2024 16 7 14 55.9 (55.6 - 56.2) 57.2
Park_KT_task1_1 Quant Park2024 27 3 11 53.8 (53.5 - 54.1) 55.8
Park_KT_task1_2 Small Park2024 12 20 10 57.3 (57.0 - 57.6) 58.4
Park_KT_task1_3 KD Park2024 15 20 10 56.6 (56.4 - 56.9) 57.6
DCASE2024 baseline Baseline 37 11 15 51.5 (51.2 - 51.8) 50.3
Shao_NEPUMSE_task1_1 NEPUMSE Shao2024 4 5 6 58.3 (58.0 - 58.6) 59.7
Shao_NEPUMSE_task1_2 NEPUMSE Shao2024 7 9 7 57.7 (57.4 - 58.0) 59.5
Shao_NEPUMSE_task1_3 NEPUMSE Shao2024 13 13 5 57.2 (56.9 - 57.4) 58.3
Surkov_ITMO_task1_1 Large Surkov2024 25 11 8 54.3 (54.0 - 54.6) 53.5
Surkov_ITMO_task1_2 Small Surkov2024 34 11 8 52.7 (52.5 - 53.0) 51.8
Tan_CISS_task1_1 NTUCISS_T1 Tan2024 35 11 15 52.6 (52.3 - 52.9) 53.9
Truchan_LUH_task1_1 Isotropic Truchan2024 22 4 19 55.0 (54.7 - 55.3) 55.5
Truchan_LUH_task1_2 Recursiv Truchan2024 30 4 19 53.4 (53.1 - 53.7) 52.7
Truchan_LUH_task1_3 RSC Truchan2024 21 4 19 55.1 (54.9 - 55.4) 54.9
Werning_UPBNT_task1_1 UPBNT Werning2024 18 11 15 55.5 (55.2 - 55.8) 53.8
Yan_NPU_task1_1 Yan_NPU Yan2024 33 12 18 53.0 (52.7 - 53.3) 52.1
Yeo_NTU_task1_1 24BCBL Yeo2024 20 2 9 55.2 (55.0 - 55.5) 53.4
Yeo_NTU_task1_2 KDEnsemble Yeo2024 10 11 15 57.5 (57.2 - 57.8) 56.8
Yeo_NTU_task1_3 TFS Yeo2024 19 11 15 55.4 (55.1 - 55.7) 55.6

Split 50%

Submission information Evaluation dataset Development dataset
Rank Submission label Name Technical
Report
Split 50
Rank
Memory
rank
MACs
rank
Accuracy
with 95% confidence interval
Accuracy
Auzanneau_CEA_task1_1 Yolovote Auzanneau2024 38 6 3 39.8 (39.5 - 40.0) 39.0
BAI_JLESS_task1_1 BAI_T1_S1 Bai2024 27 16 12 55.8 (55.5 - 56.1) 57.0
BAI_JLESS_task1_2 BAI_T1_S2 Bai2024 36 16 12 53.9 (53.6 - 54.2) 56.2
BAI_JLESS_task1_3 BAI_T1_S3 Bai2024 31 16 12 55.2 (54.9 - 55.4) 56.4
Cai_XJTLU_task1_1 TFSN t=1 Cai2024 8 15 16 59.5 (59.2 - 59.8) 61.1
Cai_XJTLU_task1_2 TFSN t=3 Cai2024 12 15 16 59.0 (58.8 - 59.3) 60.7
Cai_XJTLU_task1_3 TFSN t=12 Cai2024 11 15 16 59.2 (58.9 - 59.5) 60.1
Chen_GXU_task1_1 SENet Chen2024 32 19 15 54.7 (54.4 - 55.0) 56.8
Chen_SCUT_task1_1 BC-PACN-64 Chen2024a 21 8 4 57.0 (56.7 - 57.2) 60.9
Chen_SCUT_task1_2 BC-5-PACB-48 Chen2024a 28 10 2 55.7 (55.4 - 56.0) 60.2
Chen_SCUT_task1_3 BC-PACN-48 Chen2024a 23 1 1 56.5 (56.2 - 56.8) 59.8
Gao_UniSA_task1_1 base_fl Gao2024 22 11 17 56.7 (56.5 - 57.0) 56.3
Han_SJTUTHU_task1_1 Agp_c64 Bing2024 2 17 21 60.4 (60.1 - 60.7) 61.4
Han_SJTUTHU_task1_2 Lin_c96c64 Bing2024 3 14 13 60.2 (60.0 - 60.5) 60.4
Han_SJTUTHU_task1_3 Linear_c96 Bing2024 5 18 20 60.0 (59.7 - 60.3) 61.0
MALACH24_JKU_task1_1 CR_B_CPM David2024 7 11 15 59.7 (59.4 - 60.0) 60.2
MALACH24_JKU_task1_2 CR_P_B_CPM David2024 4 11 15 60.0 (59.8 - 60.3) 59.2
MALACH24_JKU_task1_3 CR_M_CPM David2024 13 11 15 59.0 (58.7 - 59.3) 59.6
OO_NTUPRDCSG_task1_1 MofleRes1 Oo2024 15 7 14 58.4 (58.1 - 58.6) 60.0
OO_NTUPRDCSG_task1_2 MofleRes2 Oo2024 14 7 14 58.4 (58.1 - 58.7) 60.1
Park_KT_task1_1 Quant Park2024 24 3 11 56.4 (56.1 - 56.7) 57.6
Park_KT_task1_2 Small Park2024 17 20 10 58.1 (57.8 - 58.4) 59.4
Park_KT_task1_3 KD Park2024 20 20 10 57.2 (56.9 - 57.5) 59.2
DCASE2024 baseline Baseline 34 11 15 54.4 (54.1 - 54.7) 53.2
Shao_NEPUMSE_task1_1 NEPUMSE Shao2024 1 5 6 60.6 (60.3 - 60.9) 62.3
Shao_NEPUMSE_task1_2 NEPUMSE Shao2024 6 9 7 59.8 (59.6 - 60.1) 62.1
Shao_NEPUMSE_task1_3 NEPUMSE Shao2024 10 13 5 59.2 (58.9 - 59.5) 62.2
Surkov_ITMO_task1_1 Large Surkov2024 33 11 8 54.6 (54.3 - 54.9) 54.7
Surkov_ITMO_task1_2 Small Surkov2024 37 11 8 53.9 (53.6 - 54.1) 54.0
Tan_CISS_task1_1 NTUCISS_T1 Tan2024 35 11 15 54.0 (53.7 - 54.3) 56.8
Truchan_LUH_task1_1 Isotropic Truchan2024 18 4 19 57.4 (57.1 - 57.7) 58.4
Truchan_LUH_task1_2 Recursiv Truchan2024 29 4 19 55.5 (55.2 - 55.8) 55.4
Truchan_LUH_task1_3 RSC Truchan2024 19 4 19 57.2 (56.9 - 57.5) 56.9
Werning_UPBNT_task1_1 UPBNT Werning2024 26 11 15 56.2 (55.9 - 56.5) 55.0
Yan_NPU_task1_1 Yan_NPU Yan2024 25 12 18 56.4 (56.1 - 56.6) 55.7
Yeo_NTU_task1_1 24BCBL Yeo2024 30 2 9 55.4 (55.1 - 55.7) 55.5
Yeo_NTU_task1_2 KDEnsemble Yeo2024 9 11 15 59.4 (59.1 - 59.6) 59.8
Yeo_NTU_task1_3 TFS Yeo2024 16 11 15 58.3 (58.0 - 58.6) 59.7

Split 100%

Submission information Evaluation dataset Development dataset
Rank Submission label Name Technical
Report
Official
system rank
Memory
rank
MACs
rank
Accuracy
with 95% confidence interval
Accuracy
Auzanneau_CEA_task1_1 Yolovote Auzanneau2024 38 6 3 41.1 (40.8 - 41.4) 41.3
BAI_JLESS_task1_1 BAI_T1_S1 Bai2024 30 16 12 57.1 (56.9 - 57.4) 59.4
BAI_JLESS_task1_2 BAI_T1_S2 Bai2024 34 16 12 56.7 (56.5 - 57.0) 59.3
BAI_JLESS_task1_3 BAI_T1_S3 Bai2024 29 16 12 57.3 (57.0 - 57.6) 60.3
Cai_XJTLU_task1_1 TFSN t=1 Cai2024 14 15 16 60.6 (60.3 - 60.9) 62.4
Cai_XJTLU_task1_2 TFSN t=3 Cai2024 1 15 16 62.1 (61.8 - 62.4) 62.9
Cai_XJTLU_task1_3 TFSN t=12 Cai2024 12 15 16 61.1 (60.8 - 61.4) 62.8
Chen_GXU_task1_1 SENet Chen2024 31 19 15 57.0 (56.7 - 57.2) 57.3
Chen_SCUT_task1_1 BC-PACN-64 Chen2024a 16 8 4 60.1 (59.9 - 60.4) 63.7
Chen_SCUT_task1_2 BC-5-PACB-48 Chen2024a 28 10 2 58.1 (57.8 - 58.3) 61.5
Chen_SCUT_task1_3 BC-PACN-48 Chen2024a 27 1 1 58.1 (57.8 - 58.4) 62.0
Gao_UniSA_task1_1 base_fl Gao2024 24 11 17 58.7 (58.5 - 59.0) 59.6
Han_SJTUTHU_task1_1 Agp_c64 Bing2024 8 17 21 61.5 (61.2 - 61.7) 62.2
Han_SJTUTHU_task1_2 Lin_c96c64 Bing2024 2 14 13 61.8 (61.5 - 62.1) 61.7
Han_SJTUTHU_task1_3 Linear_c96 Bing2024 9 18 20 61.3 (61.1 - 61.6) 61.3
MALACH24_JKU_task1_1 CR_B_CPM David2024 7 11 15 61.5 (61.2 - 61.8) 62.2
MALACH24_JKU_task1_2 CR_P_B_CPM David2024 10 11 15 61.3 (61.0 - 61.5) 60.6
MALACH24_JKU_task1_3 CR_M_CPM David2024 13 11 15 60.9 (60.6 - 61.2) 61.5
OO_NTUPRDCSG_task1_1 MofleRes1 Oo2024 18 7 14 59.9 (59.6 - 60.1) 62.2
OO_NTUPRDCSG_task1_2 MofleRes2 Oo2024 17 7 14 59.9 (59.6 - 60.2) 62.5
Park_KT_task1_1 Quant Park2024 32 3 11 56.9 (56.6 - 57.2) 58.8
Park_KT_task1_2 Small Park2024 15 20 10 60.3 (60.0 - 60.6) 61.9
Park_KT_task1_3 KD Park2024 19 20 10 59.8 (59.5 - 60.1) 61.6
DCASE2024 baseline Baseline 33 11 15 56.8 (56.6 - 57.1) 57.0
Shao_NEPUMSE_task1_1 NEPUMSE Shao2024 3 5 6 61.7 (61.4 - 62.0) 64.6
Shao_NEPUMSE_task1_2 NEPUMSE Shao2024 5 9 7 61.6 (61.3 - 61.9) 65.1
Shao_NEPUMSE_task1_3 NEPUMSE Shao2024 4 13 5 61.7 (61.4 - 61.9) 65.1
Surkov_ITMO_task1_1 Large Surkov2024 36 11 8 55.2 (54.9 - 55.4) 55.8
Surkov_ITMO_task1_2 Small Surkov2024 37 11 8 55.0 (54.7 - 55.3) 55.6
Tan_CISS_task1_1 NTUCISS_T1 Tan2024 35 11 15 56.1 (55.8 - 56.3) 59.1
Truchan_LUH_task1_1 Isotropic Truchan2024 20 4 19 59.4 (59.1 - 59.7) 61.0
Truchan_LUH_task1_2 Recursiv Truchan2024 22 4 19 58.8 (58.5 - 59.1) 60.0
Truchan_LUH_task1_3 RSC Truchan2024 25 4 19 58.4 (58.1 - 58.7) 58.9
Werning_UPBNT_task1_1 UPBNT Werning2024 26 11 15 58.3 (58.1 - 58.6) 60.0
Yan_NPU_task1_1 Yan_NPU Yan2024 23 12 18 58.8 (58.5 - 59.1) 59.8
Yeo_NTU_task1_1 24BCBL Yeo2024 21 2 9 59.3 (59.1 - 59.6) 57.6
Yeo_NTU_task1_2 KDEnsemble Yeo2024 6 11 15 61.6 (61.3 - 61.9) 61.7
Yeo_NTU_task1_3 TFS Yeo2024 11 11 15 61.1 (60.8 - 61.4) 62.2

Logloss

Submission information Overall rank Evaluation dataset / Logloss Development dataset / Logloss
Rank Submission label Name Technical
Report
Rank Rank score Split
5%
Split
10%
Split
25%
Split
50%
Split
100%
Split
5%
Split
10%
Split
25%
Split
50%
Split
100%
Auzanneau_CEA_task1_1 Yolovote Auzanneau2024 38 37.4 17.463 17.168 16.348 13.357 10.569
BAI_JLESS_task1_1 BAI_T1_S1 Bai2024 32 52.0 1.607 1.554 1.297 1.294 1.319
BAI_JLESS_task1_2 BAI_T1_S2 Bai2024 37 50.7 1.630 1.554 1.449 1.424 1.451
BAI_JLESS_task1_3 BAI_T1_S3 Bai2024 30 52.1 1.647 1.462 1.287 1.402 1.341
Cai_XJTLU_task1_1 TFSN t=1 Cai2024 12 55.4 1.474 1.318 1.192 1.139 1.137 1.467 1.325 1.169 1.085 1.069
Cai_XJTLU_task1_2 TFSN t=3 Cai2024 10 56.0 1.397 1.312 1.141 1.123 1.041 1.387 1.314 1.139 1.085 1.009
Cai_XJTLU_task1_3 TFSN t=12 Cai2024 11 55.9 1.435 1.269 1.157 1.129 1.093 1.459 1.295 1.152 1.089 1.034
Chen_GXU_task1_1 SENet Chen2024 29 52.1 1.658 1.592 1.453 1.389 1.254 1.763 1.624 1.453 1.731 1.215
Chen_SCUT_task1_1 BC-PACN-64 Chen2024a 28 52.5 1.671 1.703 1.397 1.292 1.243 1.494 1.330 1.181 1.088 1.068
Chen_SCUT_task1_2 BC-5-PACB-48 Chen2024a 35 51.7 1.758 1.651 1.393 1.353 1.339
Chen_SCUT_task1_3 BC-PACN-48 Chen2024a 31 52.0 1.729 1.563 1.391 1.306 1.307
Gao_UniSA_task1_1 base_fl Gao2024 25 52.8 1.546 1.508 1.311 1.215 1.150
Han_SJTUTHU_task1_1 Agp_c64 Bing2024 3 58.1 1.295 1.198 1.124 1.078 1.053
Han_SJTUTHU_task1_2 Lin_c96c64 Bing2024 1 58.2 1.261 1.206 1.119 1.077 1.049
Han_SJTUTHU_task1_3 Linear_c96 Bing2024 2 58.2 1.272 1.195 1.117 1.088 1.058
MALACH24_JKU_task1_1 CR_B_CPM David2024 5 57.0 1.364 1.269 1.168 1.118 1.066 1.388 1.274 1.169 1.104 1.051
MALACH24_JKU_task1_2 CR_P_B_CPM David2024 6 57.0 1.440 1.402 1.256 1.203 1.174 1.462 1.413 1.268 1.213 1.185
MALACH24_JKU_task1_3 CR_M_CPM David2024 7 56.5 1.370 1.282 1.180 1.131 1.080 1.388 1.277 1.175 1.111 1.058
OO_NTUPRDCSG_task1_1 MofleRes1 Oo2024 17 54.8 1.486 1.395 1.295 1.197 1.139
OO_NTUPRDCSG_task1_2 MofleRes2 Oo2024 15 54.8 1.487 1.390 1.278 1.188 1.129
Park_KT_task1_1 Quant Park2024 20 53.7 1.666 1.806 1.729 1.717 1.462
Park_KT_task1_2 Small Park2024 13 55.4 1.740 1.754 1.538 1.486 1.420
Park_KT_task1_3 KD Park2024 16 54.8 1.674 1.733 1.494 1.596 1.469
DCASE2024 baseline Baseline 36 50.7 1.661 1.625 1.522 1.477 1.447
Shao_NEPUMSE_task1_1 NEPUMSE Shao2024 4 57.2 1.455 1.387 1.216 1.151 1.122
Shao_NEPUMSE_task1_2 NEPUMSE Shao2024 8 56.1 1.595 1.450 1.306 1.217 1.154
Shao_NEPUMSE_task1_3 NEPUMSE Shao2024 14 55.2 1.622 1.437 1.276 1.219 1.143
Surkov_ITMO_task1_1 Large Surkov2024 27 52.7 1.440 1.344 1.303 1.279 1.282 1.467 1.384 1.302 1.267 1.263
Surkov_ITMO_task1_2 Small Surkov2024 33 51.9 1.452 1.384 1.354 1.345 1.330 1.477 1.426 1.380 1.340 1.302
Tan_CISS_task1_1 NTUCISS_T1 Tan2024 34 51.7 1.522 1.430 1.353 1.294 1.248
Truchan_LUH_task1_1 Isotropic Truchan2024 22 53.1 1.722 1.592 1.351 1.286 1.228 1.770 1.600 1.320 1.230 1.140
Truchan_LUH_task1_2 Recursiv Truchan2024 26 52.7 1.684 1.575 1.395 1.346 1.192 1.680 1.530 1.410 1.350 1.140
Truchan_LUH_task1_3 RSC Truchan2024 24 52.9 1.736 1.590 1.406 1.277 1.225 1.710 1.600 1.410 1.250 1.190
Werning_UPBNT_task1_1 UPBNT Werning2024 18 54.4 1.510 1.423 1.358 1.280 1.323 1.572 1.478 1.403 1.312 1.235
Yan_NPU_task1_1 Yan_NPU Yan2024 23 52.9 1.723 1.625 1.493 1.305 1.221
Yeo_NTU_task1_1 24BCBL Yeo2024 21 53.1 1.690 1.598 1.496 1.485 1.438
Yeo_NTU_task1_2 KDEnsemble Yeo2024 9 56.1 1.421 1.275 1.196 1.136 1.086
Yeo_NTU_task1_3 TFS Yeo2024 19 54.2 1.557 1.454 1.365 1.305 1.276

System complexity

Submission information Rank Accuracy / Evaluation dataset Acoustic model System
Rank Submission label Technical
Report
System
rank
Rank score Split
5%
Split
10%
Split
25%
Split
50%
Split
100%
MACS Parameters Complexity
management
Auzanneau_CEA_task1_1 Auzanneau2024 38 37.38 33.1 35.3 37.7 39.8 41.1 13509354 114988 Pruning, Quantization-aware training
BAI_JLESS_task1_1 Bai2024 32 51.99 44.7 47.8 54.5 55.8 57.1 28893268 126952 knowledge distillation,weight quantization
BAI_JLESS_task1_2 Bai2024 37 50.70 43.8 46.9 52.0 53.9 56.7 28893268 126952 knowledge distillation,weight quantization
BAI_JLESS_task1_3 Bai2024 30 52.05 43.7 49.3 54.9 55.2 57.3 28893268 126952 knowledge distillation,weight quantization
Cai_XJTLU_task1_1 Cai2024 12 55.40 47.5 52.4 57.0 59.5 60.6 29419648 126858 knowledge distillation, network design, weight quantization
Cai_XJTLU_task1_2 Cai2024 10 56.04 48.9 52.0 58.1 59.0 62.1 29419648 126858 knowledge distillation, network design, weight quantization
Cai_XJTLU_task1_3 Cai2024 11 55.87 48.4 53.2 57.6 59.2 61.1 29419648 126858 knowledge distillation, network design, weight quantization
Chen_GXU_task1_1 Chen2024 29 52.12 45.2 49.8 53.9 54.7 57.0 29419156 63900 precision_16, network design
Chen_SCUT_task1_1 Chen2024a 28 52.52 43.9 47.8 53.8 57.0 60.1 16591488 117870 knowledge distillation,precision_8, network design,weight quantization
Chen_SCUT_task1_2 Chen2024a 35 51.66 43.1 48.2 53.3 55.7 58.1 11210976 122294 knowledge distillation,precision_8, network design,weight quantization
Chen_SCUT_task1_3 Chen2024a 31 52.05 43.1 48.9 53.6 56.5 58.1 10068192 69782 knowledge distillation,precision_8, network design,weight quantization
Gao_UniSA_task1_1 Gao2024 25 52.82 45.9 49.5 53.3 56.7 58.7 29428372 61148 precision_16
Han_SJTUTHU_task1_1 Bing2024 3 58.05 53.4 56.4 58.6 60.4 61.5 29982132 63748 precision_16, network design, knowledge distillation, pruning
Han_SJTUTHU_task1_2 Bing2024 1 58.23 54.3 55.8 58.9 60.2 61.8 29221122 63215 precision_16, network design, knowledge distillation, pruning
Han_SJTUTHU_task1_3 Bing2024 2 58.22 54.0 56.7 59.1 60.0 61.3 29840890 63875 precision_16, network design, knowledge distillation, pruning
MALACH24_JKU_task1_1 David2024 5 57.05 51.6 54.5 58.0 59.7 61.5 29419156 61148 precision_16, network design, knowledge distillation
MALACH24_JKU_task1_2 David2024 6 56.96 52.0 54.0 57.5 60.0 61.3 29419156 61148 precision_16, network design, knowledge distillation
MALACH24_JKU_task1_3 David2024 7 56.47 51.2 54.0 57.3 59.0 60.9 29419156 61148 precision_16, network design, knowledge distillation
OO_NTUPRDCSG_task1_1 Oo2024 17 54.79 48.5 51.4 55.8 58.4 59.9 29407896 116842 Quantization Aware Training, network design
OO_NTUPRDCSG_task1_2 Oo2024 15 54.79 48.4 51.3 55.9 58.4 59.9 29407896 116842 Quantization Aware Training, network design
Park_KT_task1_1 Park2024 20 53.67 48.9 52.4 53.8 56.4 56.9 28568272 87672 weight quantization
Park_KT_task1_2 Park2024 13 55.36 48.7 52.4 57.3 58.1 60.3 26480312 63992 precision_16
Park_KT_task1_3 Park2024 16 54.79 48.5 51.9 56.6 57.2 59.8 26480312 63992 precision_16
DCASE2024 baseline 36 50.73 44.0 46.9 51.5 54.4 56.8 29419156 61148 precision_16, network design
Shao_NEPUMSE_task1_1 Shao2024 4 57.15 51.4 53.8 58.3 60.6 61.7 16911324 107457 knowledge distillation, network design, weight quantization
Shao_NEPUMSE_task1_2 Shao2024 8 56.13 48.5 53.0 57.7 59.8 61.6 17272736 121925 knowledge distillation, network design, weight quantization
Shao_NEPUMSE_task1_3 Shao2024 14 55.21 46.2 51.8 57.2 59.2 61.7 16785856 126410 knowledge distillation, network design, weight quantization
Surkov_ITMO_task1_1 Surkov2024 27 52.65 48.0 51.2 54.3 54.6 55.2 21896340 61148 precision_16, network design, knowledge distillation
Surkov_ITMO_task1_2 Surkov2024 33 51.92 47.8 50.3 52.7 53.9 55.0 21896340 61148 precision_16, network design, knowledge distillation
Tan_CISS_task1_1 Tan2024 34 51.71 46.5 49.3 52.6 54.0 56.1 29419156 61148 int8, network design
Truchan_LUH_task1_1 Truchan2024 22 53.06 45.3 48.1 55.0 57.4 59.4 29747914 47946 precision_16, network design
Truchan_LUH_task1_2 Truchan2024 26 52.68 46.0 49.6 53.4 55.5 58.8 29747914 47946 precision_16, network design
Truchan_LUH_task1_3 Truchan2024 24 52.91 44.9 48.9 55.1 57.2 58.4 29747914 47946 precision_16, network design
Werning_UPBNT_task1_1 Werning2024 18 54.35 49.2 52.5 55.5 56.2 58.3 29419156 61148 precision_16, network design, knowledge distillation
Yan_NPU_task1_1 Yan2024 23 52.94 47.5 49.1 53.0 56.4 58.8 29675722 31290 precision_32, network design
Yeo_NTU_task1_1 Yeo2024 21 53.13 46.3 49.4 55.2 55.4 59.3 22649568 35062 precision_16, network design
Yeo_NTU_task1_2 Yeo2024 9 56.12 48.9 53.3 57.5 59.4 61.6 29419156 61148 precision_16, network design, knowledge distillation
Yeo_NTU_task1_3 Yeo2024 19 54.19 46.0 50.2 55.4 58.3 61.1 29419156 61148 precision_16, network design, knowledge distillation


Generalization performance

All results with evaluation dataset.

Devices

Per split accuracy
Submission information Overall Rank score Split 5% Split 10% Split 25% Split 50% Split 100%
Rank Submission label Technical
Report
System
rank
Rank score Unseen Seen Unseen Seen Unseen Seen Unseen Seen Unseen Seen Unseen Seen
Auzanneau_CEA_task1_1 Auzanneau2024 38 37.38 34.12 40.10 31.45 34.43 33.37 36.93 34.45 40.40 35.50 43.30 35.82 45.47
BAI_JLESS_task1_1 Bai2024 32 51.99 48.49 54.91 42.82 46.35 44.75 50.28 51.29 57.25 51.81 59.06 51.76 61.61
BAI_JLESS_task1_2 Bai2024 37 50.70 47.05 53.73 42.20 45.21 43.76 49.56 49.06 54.54 48.81 58.20 51.42 61.18
BAI_JLESS_task1_3 Bai2024 30 52.05 48.57 54.96 41.48 45.47 46.48 51.66 51.81 57.39 50.75 58.85 52.32 61.44
Cai_XJTLU_task1_1 Cai2024 12 55.40 52.90 57.49 45.77 49.03 50.17 54.21 54.39 59.27 56.77 61.72 57.40 63.23
Cai_XJTLU_task1_2 Cai2024 10 56.04 53.71 57.98 46.76 50.69 50.19 53.59 56.02 59.81 56.12 61.46 59.45 64.35
Cai_XJTLU_task1_3 Cai2024 11 55.87 53.51 57.83 47.17 49.35 51.50 54.55 54.90 59.77 56.28 61.60 57.73 63.88
Chen_GXU_task1_1 Chen2024 29 52.12 49.30 54.48 42.95 47.08 47.71 51.58 51.95 55.54 50.61 58.14 53.26 60.05
Chen_SCUT_task1_1 Chen2024a 28 52.52 46.50 57.53 39.16 47.92 42.87 51.88 47.31 59.12 50.05 62.71 53.10 66.00
Chen_SCUT_task1_2 Chen2024a 35 51.66 45.86 56.50 38.65 46.74 43.56 52.14 46.72 58.70 49.08 61.25 51.30 63.68
Chen_SCUT_task1_3 Chen2024a 31 52.05 46.34 56.81 38.35 47.13 44.32 52.70 47.42 58.83 49.96 61.95 51.64 63.44
Gao_UniSA_task1_1 Gao2024 25 52.82 49.91 55.25 43.84 47.64 46.86 51.61 50.08 55.94 53.59 59.36 55.17 61.70
Han_SJTUTHU_task1_1 Bing2024 3 58.05 55.30 60.35 50.80 55.54 54.08 58.35 55.65 61.12 57.34 62.92 58.62 63.84
Han_SJTUTHU_task1_2 Bing2024 1 58.23 55.63 60.40 52.79 55.65 52.81 58.34 56.34 61.10 57.66 62.39 58.57 64.52
Han_SJTUTHU_task1_3 Bing2024 2 58.22 55.58 60.42 52.26 55.39 54.14 58.81 56.43 61.31 56.68 62.80 58.41 63.78
MALACH24_JKU_task1_1 David2024 5 57.05 55.48 58.35 50.94 52.08 53.25 55.46 56.63 59.15 57.43 61.58 59.14 63.48
MALACH24_JKU_task1_2 David2024 6 56.96 55.69 58.02 51.38 52.43 53.01 54.86 56.39 58.49 58.22 61.57 59.46 62.75
MALACH24_JKU_task1_3 David2024 7 56.47 54.88 57.80 50.39 51.80 52.65 55.08 55.72 58.65 56.85 60.83 58.77 62.65
OO_NTUPRDCSG_task1_1 Oo2024 17 54.79 51.64 57.41 46.59 50.13 48.99 53.46 52.21 58.74 54.83 61.29 55.56 63.42
OO_NTUPRDCSG_task1_2 Oo2024 15 54.79 51.72 57.35 46.67 49.89 48.95 53.33 52.42 58.74 54.85 61.40 55.74 63.39
Park_KT_task1_1 Park2024 20 53.67 50.37 56.42 46.98 50.42 50.90 53.67 49.33 57.49 52.43 59.76 52.20 60.77
Park_KT_task1_2 Park2024 13 55.36 51.84 58.29 45.77 51.06 49.88 54.57 54.00 60.05 53.19 62.18 56.37 63.61
Park_KT_task1_3 Park2024 16 54.79 51.15 57.83 46.18 50.41 49.13 54.14 52.94 59.73 52.62 60.98 54.87 63.87
DCASE2024 baseline 36 50.73 48.13 52.90 42.41 45.33 43.83 49.54 49.48 53.12 51.05 57.20 53.88 59.30
Shao_NEPUMSE_task1_1 Shao2024 4 57.15 53.28 60.38 48.88 53.46 49.68 57.15 54.19 61.75 56.46 64.07 57.20 65.48
Shao_NEPUMSE_task1_2 Shao2024 8 56.13 51.73 59.79 44.63 51.78 48.92 56.40 52.97 61.60 55.22 63.69 56.91 65.51
Shao_NEPUMSE_task1_3 Shao2024 14 55.21 50.54 59.11 42.06 49.67 47.81 55.15 52.47 61.08 53.98 63.58 56.37 66.07
Surkov_ITMO_task1_1 Surkov2024 27 52.65 49.73 55.09 46.56 49.26 48.30 53.57 51.49 56.62 51.61 57.14 50.69 58.87
Surkov_ITMO_task1_2 Surkov2024 33 51.92 48.85 54.49 46.11 49.16 46.94 53.03 49.73 55.26 50.56 56.60 50.90 58.38
Tan_CISS_task1_1 Tan2024 34 51.71 48.80 54.13 44.33 48.31 47.51 50.83 49.05 55.61 50.93 56.61 52.19 59.30
Truchan_LUH_task1_1 Truchan2024 22 53.06 51.10 54.69 43.95 46.43 45.74 50.14 53.19 56.55 55.46 59.06 57.17 61.25
Truchan_LUH_task1_2 Truchan2024 26 52.68 50.53 54.47 44.59 47.21 46.71 52.08 51.81 54.72 53.21 57.46 56.31 60.90
Truchan_LUH_task1_3 Truchan2024 24 52.91 50.82 54.65 43.20 46.40 46.25 51.05 53.46 56.55 54.90 59.09 56.30 60.15
Werning_UPBNT_task1_1 Werning2024 18 54.35 52.09 56.24 48.06 50.18 50.79 53.95 53.61 57.06 53.10 58.79 54.89 61.22
Yan_NPU_task1_1 Yan2024 23 52.94 49.91 55.46 45.86 48.88 46.02 51.60 49.82 55.59 53.10 59.07 54.77 62.15
Yeo_NTU_task1_1 Yeo2024 21 53.13 50.61 55.22 44.32 47.89 47.49 50.94 52.80 57.29 51.74 58.46 56.72 61.54
Yeo_NTU_task1_2 Yeo2024 9 56.12 54.02 57.87 47.84 49.75 50.67 55.46 55.83 58.84 56.81 61.49 58.93 63.81
Yeo_NTU_task1_3 Yeo2024 19 54.19 51.82 56.16 44.70 47.04 48.17 51.85 52.69 57.66 55.29 60.81 58.25 63.46

Cities

Per split accuracy
Submission information Overall Rank score Split 5% Split 10% Split 25% Split 50% Split 100%
Rank Submission label Technical
Report
System
rank
Rank score Unseen Seen Unseen Seen Unseen Seen Unseen Seen Unseen Seen Unseen Seen
Auzanneau_CEA_task1_1 Auzanneau2024 38 37.38 36.28 37.63 33.88 32.92 35.75 35.24 35.73 38.11 37.47 40.25 38.60 41.62
BAI_JLESS_task1_1 Bai2024 32 51.99 52.00 52.01 45.94 44.52 48.51 47.63 53.90 54.70 55.10 55.93 56.55 57.28
BAI_JLESS_task1_2 Bai2024 37 50.70 50.47 50.76 45.24 43.57 47.23 46.87 52.03 52.07 53.12 54.12 54.75 57.17
BAI_JLESS_task1_3 Bai2024 30 52.05 51.61 52.17 44.83 43.43 49.36 49.31 54.07 55.04 54.00 55.43 55.78 57.63
Cai_XJTLU_task1_1 Cai2024 12 55.40 56.00 55.31 49.67 47.14 54.31 52.00 57.60 56.96 58.96 59.61 59.48 60.84
Cai_XJTLU_task1_2 Cai2024 10 56.04 56.88 55.89 51.37 48.42 53.63 51.75 58.74 57.98 59.45 58.98 61.22 62.34
Cai_XJTLU_task1_3 Cai2024 11 55.87 56.73 55.72 50.06 48.03 54.88 52.84 58.34 57.42 59.72 59.10 60.68 61.19
Chen_GXU_task1_1 Chen2024 29 52.12 52.42 52.08 45.84 45.08 51.96 49.40 53.51 54.01 54.96 54.69 55.80 57.22
Chen_SCUT_task1_1 Chen2024a 28 52.52 51.48 52.76 44.61 43.84 47.57 47.85 52.25 54.10 55.40 57.31 57.58 60.71
Chen_SCUT_task1_2 Chen2024a 35 51.66 50.58 51.92 42.88 43.12 49.04 48.10 51.45 53.66 53.60 56.19 55.95 58.52
Chen_SCUT_task1_3 Chen2024a 31 52.05 51.29 52.24 43.92 43.00 48.83 48.93 52.42 53.94 55.67 56.70 55.60 58.63
Gao_UniSA_task1_1 Gao2024 25 52.82 53.20 52.77 47.70 45.56 51.05 49.14 53.59 53.23 55.99 56.92 57.66 58.97
Han_SJTUTHU_task1_1 Bing2024 3 58.05 58.08 58.08 54.67 53.14 56.87 56.34 57.82 58.83 59.74 60.54 61.28 61.54
Han_SJTUTHU_task1_2 Bing2024 1 58.23 58.28 58.25 55.18 54.20 55.91 55.83 58.05 59.15 59.76 60.37 62.52 61.70
Han_SJTUTHU_task1_3 Bing2024 2 58.22 58.49 58.19 55.69 53.64 57.57 56.54 58.96 59.15 59.69 60.11 60.55 61.53
MALACH24_JKU_task1_1 David2024 5 57.05 57.34 57.01 52.05 51.48 55.10 54.35 58.65 57.90 59.43 59.78 61.48 61.55
MALACH24_JKU_task1_2 David2024 6 56.96 57.27 56.93 53.06 51.75 54.54 53.94 57.85 57.50 60.23 60.04 60.67 61.41
MALACH24_JKU_task1_3 David2024 7 56.47 57.07 56.38 52.44 50.92 54.88 53.81 58.22 57.16 58.83 59.09 60.98 60.90
OO_NTUPRDCSG_task1_1 Oo2024 17 54.79 54.38 54.89 50.36 48.17 52.31 51.27 54.79 56.00 57.04 58.65 57.39 60.38
OO_NTUPRDCSG_task1_2 Oo2024 15 54.79 54.63 54.85 50.13 48.09 52.41 51.14 54.73 56.13 57.36 58.67 58.49 60.23
Park_KT_task1_1 Park2024 20 53.67 52.73 53.89 49.48 48.75 53.13 52.29 52.15 54.13 54.37 56.86 54.49 57.39
Park_KT_task1_2 Park2024 13 55.36 54.82 55.50 50.15 48.38 52.39 52.47 56.63 57.46 56.92 58.35 58.02 60.82
Park_KT_task1_3 Park2024 16 54.79 54.39 54.90 50.63 48.07 52.22 51.81 55.42 56.92 55.89 57.48 57.79 60.22
DCASE2024 baseline 36 50.73 51.48 50.60 46.65 43.48 48.58 46.63 52.28 51.33 54.49 54.41 55.42 57.16
Shao_NEPUMSE_task1_1 Shao2024 4 57.15 56.90 57.23 53.10 51.05 54.35 53.65 57.90 58.42 59.43 60.88 59.73 62.15
Shao_NEPUMSE_task1_2 Shao2024 8 56.13 55.11 56.36 48.90 48.47 52.45 53.12 56.52 57.94 58.23 60.20 59.45 62.07
Shao_NEPUMSE_task1_3 Shao2024 14 55.21 54.29 55.42 46.68 46.13 51.54 51.88 55.88 57.45 57.40 59.62 59.94 62.04
Surkov_ITMO_task1_1 Surkov2024 27 52.65 52.94 52.63 50.46 47.57 51.90 51.06 53.77 54.42 54.16 54.76 54.42 55.33
Surkov_ITMO_task1_2 Surkov2024 33 51.92 51.93 51.96 50.06 47.34 51.18 50.10 51.55 53.02 52.98 54.07 53.89 55.24
Tan_CISS_task1_1 Tan2024 34 51.71 51.82 51.72 46.18 46.59 50.15 49.17 54.18 52.34 53.24 54.24 55.33 56.26
Truchan_LUH_task1_1 Truchan2024 22 53.06 52.70 53.15 47.13 44.95 48.79 48.03 53.64 55.33 56.37 57.67 57.57 59.79
Truchan_LUH_task1_2 Truchan2024 26 52.68 52.38 52.76 47.97 45.64 50.62 49.45 53.00 53.51 53.65 55.93 56.65 59.28
Truchan_LUH_task1_3 Truchan2024 24 52.91 52.34 53.05 45.77 44.79 48.60 48.93 54.94 55.21 56.15 57.42 56.23 58.87
Werning_UPBNT_task1_1 Werning2024 18 54.35 55.30 54.18 51.33 48.81 55.01 52.02 56.90 55.22 56.35 56.19 56.90 58.67
Yan_NPU_task1_1 Yan2024 23 52.94 52.92 52.97 48.96 47.23 50.39 48.81 52.46 53.09 55.54 56.55 57.22 59.15
Yeo_NTU_task1_1 Yeo2024 21 53.13 53.07 53.17 47.60 46.01 49.23 49.42 55.03 55.32 55.22 55.47 58.28 59.60
Yeo_NTU_task1_2 Yeo2024 9 56.12 56.19 56.13 49.45 48.78 54.15 53.13 57.05 57.58 58.86 59.49 61.42 61.66
Yeo_NTU_task1_3 Yeo2024 19 54.19 54.65 54.12 45.94 45.99 51.58 49.91 55.39 55.42 59.02 58.18 61.33 61.07

Class-wise performance

Split 5%

Overall Split
Rank Submission label Technical
Report
System
rank
Rank score Accuracy Airport Bus Metro Metro
station
Park Public
square
Shopping
mall
Street
pedestrian
Street
traffic
Tram
Auzanneau_CEA_task1_1 Auzanneau2024 38 37.38 33.1 21.5 46.5 33.2 25.5 56.2 11.6 38.9 17.1 60.7 19.5
BAI_JLESS_task1_1 Bai2024 32 51.99 44.7 26.8 56.7 37.1 39.1 65.4 26.1 59.3 30.7 63.0 43.2
BAI_JLESS_task1_2 Bai2024 37 50.70 43.8 31.0 58.6 42.2 34.2 74.9 16.3 52.6 27.9 68.4 32.4
BAI_JLESS_task1_3 Bai2024 30 52.05 43.7 21.5 47.3 43.8 39.1 67.1 20.0 63.5 29.0 63.9 41.3
Cai_XJTLU_task1_1 Cai2024 12 55.40 47.5 27.7 56.8 41.2 40.7 76.2 28.2 67.5 22.8 70.1 44.4
Cai_XJTLU_task1_2 Cai2024 10 56.04 48.9 27.0 59.6 32.2 40.4 75.5 29.2 70.5 21.4 69.8 63.5
Cai_XJTLU_task1_3 Cai2024 11 55.87 48.4 25.1 62.8 31.7 50.2 73.1 32.4 65.7 25.5 70.7 46.4
Chen_GXU_task1_1 Chen2024 29 52.12 45.2 24.7 60.2 38.2 37.6 72.7 22.6 47.5 29.5 70.0 49.2
Chen_SCUT_task1_1 Chen2024a 28 52.52 43.9 28.6 62.4 37.0 44.1 66.6 22.3 47.0 16.2 63.8 51.3
Chen_SCUT_task1_2 Chen2024a 35 51.66 43.1 26.6 75.9 29.6 44.0 66.4 20.0 43.2 17.3 60.8 46.7
Chen_SCUT_task1_3 Chen2024a 31 52.05 43.1 25.9 69.0 34.3 40.4 62.0 20.3 43.2 22.2 61.1 52.9
Gao_UniSA_task1_1 Gao2024 25 52.82 45.9 28.9 56.6 43.7 43.0 78.3 26.8 43.6 33.6 60.7 43.9
Han_SJTUTHU_task1_1 Bing2024 3 58.05 53.4 27.7 62.1 53.8 50.7 79.9 36.9 56.3 37.3 75.1 53.9
Han_SJTUTHU_task1_2 Bing2024 1 58.23 54.3 32.1 58.2 45.3 52.4 77.9 32.2 67.4 38.7 77.9 61.2
Han_SJTUTHU_task1_3 Bing2024 2 58.22 54.0 35.2 62.4 51.9 50.9 75.0 36.8 60.7 36.1 77.7 52.8
MALACH24_JKU_task1_1 David2024 5 57.05 51.6 34.6 63.5 47.8 41.7 73.1 28.6 64.4 38.2 74.0 49.6
MALACH24_JKU_task1_2 David2024 6 56.96 52.0 36.9 65.2 48.7 43.3 76.7 27.5 63.6 32.0 77.5 48.2
MALACH24_JKU_task1_3 David2024 7 56.47 51.2 34.3 61.0 51.0 41.8 72.7 27.2 65.1 36.7 73.9 47.9
OO_NTUPRDCSG_task1_1 Oo2024 17 54.79 48.5 18.8 61.9 52.1 39.8 75.0 24.9 67.6 31.1 67.7 46.3
OO_NTUPRDCSG_task1_2 Oo2024 15 54.79 48.4 18.2 61.3 51.8 41.3 74.4 25.1 69.3 29.6 67.8 45.5
Park_KT_task1_1 Park2024 20 53.67 48.9 23.2 74.5 45.1 30.2 74.6 28.3 59.5 42.4 71.8 38.9
Park_KT_task1_2 Park2024 13 55.36 48.7 33.4 67.4 56.3 31.8 68.9 29.5 48.2 31.5 77.4 42.3
Park_KT_task1_3 Park2024 16 54.79 48.5 26.9 50.3 52.9 36.5 70.7 33.9 57.3 31.1 70.7 54.5
DCASE2024 baseline 36 50.73 44.0 30.2 47.4 38.9 36.3 70.4 23.6 49.1 31.2 69.2 43.8
Shao_NEPUMSE_task1_1 Shao2024 4 57.15 51.4 27.5 66.3 50.9 44.4 77.2 26.9 59.5 36.2 74.0 51.0
Shao_NEPUMSE_task1_2 Shao2024 8 56.13 48.5 30.5 62.6 45.8 44.5 75.0 21.8 55.3 29.4 69.5 50.8
Shao_NEPUMSE_task1_3 Shao2024 14 55.21 46.2 27.1 51.1 49.8 41.5 69.4 21.7 55.4 30.3 74.4 41.4
Surkov_ITMO_task1_1 Surkov2024 27 52.65 48.0 24.7 63.1 48.6 33.0 70.4 27.5 66.5 35.3 69.2 41.9
Surkov_ITMO_task1_2 Surkov2024 33 51.92 47.8 23.2 64.2 47.6 34.0 69.7 28.2 63.2 37.3 70.2 40.1
Tan_CISS_task1_1 Tan2024 34 51.71 46.5 34.5 76.3 27.4 37.5 70.4 17.1 57.8 31.2 68.4 44.5
Truchan_LUH_task1_1 Truchan2024 22 53.06 45.3 32.5 56.9 41.0 34.2 69.6 31.1 49.8 30.0 66.8 41.1
Truchan_LUH_task1_2 Truchan2024 26 52.68 46.0 32.6 57.6 43.0 34.7 67.8 28.2 51.5 33.5 67.7 43.5
Truchan_LUH_task1_3 Truchan2024 24 52.91 44.9 32.8 56.2 41.6 34.0 69.1 24.5 49.4 33.3 67.0 41.5
Werning_UPBNT_task1_1 Werning2024 18 54.35 49.2 31.3 57.7 42.0 44.3 73.4 25.0 59.8 36.7 70.6 51.5
Yan_NPU_task1_1 Yan2024 23 52.94 47.5 30.5 51.1 48.3 39.7 77.0 23.9 56.2 34.5 69.1 44.8
Yeo_NTU_task1_1 Yeo2024 21 53.13 46.3 31.1 65.4 35.9 39.7 80.1 25.2 47.8 21.0 72.7 43.6
Yeo_NTU_task1_2 Yeo2024 9 56.12 48.9 35.0 64.5 47.7 53.6 74.3 20.1 60.3 19.3 75.2 38.7
Yeo_NTU_task1_3 Yeo2024 19 54.19 46.0 28.5 59.4 39.2 56.0 64.6 26.0 50.5 21.5 72.4 41.8

Split 10%

Overall Split
Rank Submission label Technical
Report
System
rank
Rank score Accuracy Airport Bus Metro Metro
station
Park Public
square
Shopping
mall
Street
pedestrian
Street
traffic
Tram
Auzanneau_CEA_task1_1 Auzanneau2024 38 37.38 35.3 25.0 47.1 31.8 31.4 63.8 15.4 36.9 21.7 54.2 25.6
BAI_JLESS_task1_1 Bai2024 32 51.99 47.8 27.8 59.8 53.7 37.6 78.1 24.9 59.6 29.3 62.5 44.5
BAI_JLESS_task1_2 Bai2024 37 50.70 46.9 35.2 66.0 57.1 29.5 77.8 22.8 48.2 29.0 68.3 35.4
BAI_JLESS_task1_3 Bai2024 30 52.05 49.3 37.2 62.8 48.5 42.2 81.5 21.4 53.8 32.6 64.0 49.1
Cai_XJTLU_task1_1 Cai2024 12 55.40 52.4 31.3 58.0 52.9 47.7 77.7 32.5 63.6 34.8 69.0 56.2
Cai_XJTLU_task1_2 Cai2024 10 56.04 52.0 33.1 62.8 47.4 49.0 77.2 32.4 67.2 30.1 71.6 49.5
Cai_XJTLU_task1_3 Cai2024 11 55.87 53.2 33.0 60.2 46.5 46.7 77.0 31.7 70.5 34.0 72.1 60.0
Chen_GXU_task1_1 Chen2024 29 52.12 49.8 41.3 65.6 48.3 41.5 76.1 32.4 51.4 25.3 61.7 54.7
Chen_SCUT_task1_1 Chen2024a 28 52.52 47.8 31.4 77.8 43.4 42.9 70.4 30.2 43.2 22.6 62.7 53.3
Chen_SCUT_task1_2 Chen2024a 35 51.66 48.2 29.1 75.4 41.1 49.6 70.8 30.5 47.5 21.3 65.8 51.2
Chen_SCUT_task1_3 Chen2024a 31 52.05 48.9 33.5 73.5 49.1 45.8 68.2 27.4 51.7 21.6 69.7 48.3
Gao_UniSA_task1_1 Gao2024 25 52.82 49.5 27.0 64.1 45.1 37.2 75.9 30.9 45.0 42.7 70.2 56.5
Han_SJTUTHU_task1_1 Bing2024 3 58.05 56.4 42.0 64.7 50.4 48.8 80.6 36.1 68.4 31.9 78.0 63.3
Han_SJTUTHU_task1_2 Bing2024 1 58.23 55.8 32.9 65.0 54.0 54.3 83.0 30.3 62.9 36.1 78.9 60.9
Han_SJTUTHU_task1_3 Bing2024 2 58.22 56.7 40.7 65.7 58.3 46.8 84.0 35.1 63.2 34.0 77.5 61.5
MALACH24_JKU_task1_1 David2024 5 57.05 54.5 34.6 66.5 53.4 46.9 76.0 32.6 65.9 37.0 77.2 54.5
MALACH24_JKU_task1_2 David2024 6 56.96 54.0 34.6 70.2 50.4 43.4 80.6 29.7 68.0 35.4 78.9 49.0
MALACH24_JKU_task1_3 David2024 7 56.47 54.0 36.9 64.3 54.7 44.4 74.3 33.0 64.9 36.4 76.0 54.9
OO_NTUPRDCSG_task1_1 Oo2024 17 54.79 51.4 24.0 68.6 53.7 48.5 76.1 29.1 64.0 31.3 73.2 45.8
OO_NTUPRDCSG_task1_2 Oo2024 15 54.79 51.3 24.9 68.2 54.3 48.8 75.8 29.5 62.2 30.5 73.6 45.7
Park_KT_task1_1 Park2024 20 53.67 52.4 33.3 69.1 56.0 45.6 69.7 34.1 67.8 22.7 70.0 55.8
Park_KT_task1_2 Park2024 13 55.36 52.4 31.0 69.0 56.4 43.9 73.6 33.0 60.0 32.7 66.5 58.2
Park_KT_task1_3 Park2024 16 54.79 51.9 34.3 67.2 55.3 39.8 75.4 33.5 56.8 35.1 65.7 55.6
DCASE2024 baseline 36 50.73 46.9 25.0 53.1 45.2 43.2 80.4 25.6 53.8 28.9 68.1 46.2
Shao_NEPUMSE_task1_1 Shao2024 4 57.15 53.8 29.3 65.8 55.4 51.0 82.1 30.5 54.1 39.3 71.7 58.2
Shao_NEPUMSE_task1_2 Shao2024 8 56.13 53.0 28.7 64.5 57.0 46.3 81.9 27.6 59.3 38.0 71.5 55.2
Shao_NEPUMSE_task1_3 Shao2024 14 55.21 51.8 31.2 67.4 52.1 51.3 77.7 26.1 55.4 36.1 71.4 49.3
Surkov_ITMO_task1_1 Surkov2024 27 52.65 51.2 26.5 63.4 53.7 38.8 76.9 34.4 63.0 34.6 71.5 48.8
Surkov_ITMO_task1_2 Surkov2024 33 51.92 50.3 27.8 62.5 49.1 41.4 78.6 27.8 57.2 34.2 68.3 55.6
Tan_CISS_task1_1 Tan2024 34 51.71 49.3 31.1 85.4 37.4 37.7 70.5 23.4 65.7 26.1 71.3 44.5
Truchan_LUH_task1_1 Truchan2024 22 53.06 48.1 29.8 56.3 42.0 38.2 75.5 32.9 53.5 35.1 68.2 50.0
Truchan_LUH_task1_2 Truchan2024 26 52.68 49.6 34.4 66.1 46.1 37.4 75.8 29.7 53.3 34.6 68.7 50.3
Truchan_LUH_task1_3 Truchan2024 24 52.91 48.9 31.0 63.5 48.5 41.8 73.0 28.0 52.5 36.4 67.9 46.1
Werning_UPBNT_task1_1 Werning2024 18 54.35 52.5 30.8 58.1 51.7 47.0 76.1 34.6 54.9 42.6 75.0 54.3
Yan_NPU_task1_1 Yan2024 23 52.94 49.1 27.2 55.7 53.3 39.2 79.2 29.4 55.3 34.2 69.4 47.7
Yeo_NTU_task1_1 Yeo2024 21 53.13 49.4 34.8 70.4 50.2 32.1 74.9 27.8 54.0 26.5 78.6 44.3
Yeo_NTU_task1_2 Yeo2024 9 56.12 53.3 29.6 65.6 51.1 52.8 79.2 31.5 60.8 35.1 75.2 52.0
Yeo_NTU_task1_3 Yeo2024 19 54.19 50.2 29.1 60.5 48.5 49.3 75.2 33.6 52.6 30.7 72.3 50.1

Split 25%

Overall Split
Rank Submission label Technical
Report
System
rank
Rank score Accuracy Airport Bus Metro Metro
station
Park Public
square
Shopping
mall
Street
pedestrian
Street
traffic
Tram
Auzanneau_CEA_task1_1 Auzanneau2024 38 37.38 37.7 23.1 43.3 36.8 22.8 72.5 17.1 36.1 23.2 53.8 48.1
BAI_JLESS_task1_1 Bai2024 32 51.99 54.5 43.1 77.2 43.6 49.5 85.9 29.6 68.7 24.7 64.6 58.4
BAI_JLESS_task1_2 Bai2024 37 50.70 52.0 45.3 72.8 54.1 42.7 86.2 30.6 50.3 23.8 65.3 49.5
BAI_JLESS_task1_3 Bai2024 30 52.05 54.9 49.1 73.6 48.7 48.9 83.3 33.8 53.4 28.4 69.4 60.0
Cai_XJTLU_task1_1 Cai2024 12 55.40 57.0 41.2 73.1 53.7 47.1 75.0 40.5 60.6 35.3 76.5 67.4
Cai_XJTLU_task1_2 Cai2024 10 56.04 58.1 44.6 70.4 54.7 50.8 77.8 37.0 67.4 34.9 78.5 64.8
Cai_XJTLU_task1_3 Cai2024 11 55.87 57.6 38.8 67.4 51.7 53.0 78.5 35.1 62.5 41.2 76.2 71.1
Chen_GXU_task1_1 Chen2024 29 52.12 53.9 40.4 71.1 60.1 45.4 79.8 35.7 60.2 29.4 66.6 50.4
Chen_SCUT_task1_1 Chen2024a 28 52.52 53.8 42.7 80.9 53.3 41.3 75.6 34.4 45.0 31.1 70.9 62.2
Chen_SCUT_task1_2 Chen2024a 35 51.66 53.3 44.3 77.6 47.1 39.9 77.9 33.2 51.5 30.6 67.2 63.3
Chen_SCUT_task1_3 Chen2024a 31 52.05 53.6 39.8 79.6 46.2 47.3 79.3 32.1 52.7 31.8 68.5 59.3
Gao_UniSA_task1_1 Gao2024 25 52.82 53.3 31.6 67.1 45.8 41.5 78.1 35.3 59.1 40.8 72.7 60.6
Han_SJTUTHU_task1_1 Bing2024 3 58.05 58.6 43.6 73.0 53.2 49.1 82.1 33.2 65.8 41.2 78.6 66.6
Han_SJTUTHU_task1_2 Bing2024 1 58.23 58.9 46.3 67.6 52.4 52.4 87.3 31.4 70.0 34.4 78.2 69.3
Han_SJTUTHU_task1_3 Bing2024 2 58.22 59.1 48.2 70.8 48.6 55.0 85.0 34.0 66.9 34.8 76.8 70.8
MALACH24_JKU_task1_1 David2024 5 57.05 58.0 45.7 71.8 54.1 51.7 80.9 32.7 67.7 35.6 79.8 60.0
MALACH24_JKU_task1_2 David2024 6 56.96 57.5 42.6 72.5 53.0 53.1 82.1 30.2 70.2 32.6 80.8 58.2
MALACH24_JKU_task1_3 David2024 7 56.47 57.3 45.5 71.0 53.7 51.3 79.5 32.5 65.4 36.5 78.8 59.1
OO_NTUPRDCSG_task1_1 Oo2024 17 54.79 55.8 30.3 72.9 54.5 55.0 80.6 33.8 64.6 33.3 69.6 63.1
OO_NTUPRDCSG_task1_2 Oo2024 15 54.79 55.9 30.3 73.0 54.4 55.1 80.7 33.6 65.4 33.3 69.8 63.2
Park_KT_task1_1 Park2024 20 53.67 53.8 36.7 73.2 55.0 45.7 78.4 27.3 60.0 36.3 67.1 58.2
Park_KT_task1_2 Park2024 13 55.36 57.3 39.4 74.0 54.8 50.7 81.0 33.8 61.5 38.3 74.3 65.3
Park_KT_task1_3 Park2024 16 54.79 56.6 37.4 74.9 58.3 49.6 80.5 35.0 58.0 35.1 72.4 65.2
DCASE2024 baseline 36 50.73 51.5 37.3 63.6 41.2 34.1 72.1 42.5 60.2 32.9 77.1 53.8
Shao_NEPUMSE_task1_1 Shao2024 4 57.15 58.3 37.3 76.6 57.1 62.5 84.5 29.0 59.0 41.0 73.6 62.3
Shao_NEPUMSE_task1_2 Shao2024 8 56.13 57.7 36.4 69.9 64.7 55.7 85.6 28.7 60.4 37.9 74.9 62.7
Shao_NEPUMSE_task1_3 Shao2024 14 55.21 57.2 32.4 75.1 60.9 55.0 84.9 26.7 64.5 36.8 74.4 61.1
Surkov_ITMO_task1_1 Surkov2024 27 52.65 54.3 36.9 68.6 51.4 44.6 83.1 30.9 58.3 35.0 72.7 61.4
Surkov_ITMO_task1_2 Surkov2024 33 51.92 52.7 38.6 68.9 47.7 43.1 80.7 29.5 57.2 31.1 71.1 59.6
Tan_CISS_task1_1 Tan2024 34 51.71 52.6 41.1 82.3 41.7 46.9 78.6 25.9 46.0 38.6 64.4 60.8
Truchan_LUH_task1_1 Truchan2024 22 53.06 55.0 42.5 74.2 48.6 39.4 79.9 32.8 63.9 33.9 78.3 56.6
Truchan_LUH_task1_2 Truchan2024 26 52.68 53.4 42.4 60.0 45.1 38.8 78.0 36.2 65.2 36.9 73.6 57.8
Truchan_LUH_task1_3 Truchan2024 24 52.91 55.1 42.9 70.5 50.9 42.4 76.7 36.3 62.2 34.9 74.4 60.2
Werning_UPBNT_task1_1 Werning2024 18 54.35 55.5 41.5 68.1 48.2 49.7 85.0 27.7 57.3 40.0 76.3 61.1
Yan_NPU_task1_1 Yan2024 23 52.94 53.0 33.0 59.7 47.9 46.6 81.8 30.0 56.8 42.7 72.2 58.9
Yeo_NTU_task1_1 Yeo2024 21 53.13 55.2 40.1 66.4 48.0 43.8 83.1 35.5 60.8 39.8 72.8 62.2
Yeo_NTU_task1_2 Yeo2024 9 56.12 57.5 43.2 67.2 55.5 50.5 76.5 37.3 67.2 36.4 80.2 60.7
Yeo_NTU_task1_3 Yeo2024 19 54.19 55.4 41.7 61.8 54.0 54.7 72.7 39.0 59.2 32.5 78.4 59.9

Split 50%

Overall Split
Rank Submission label Technical
Report
System
rank
Rank score Accuracy Airport Bus Metro Metro
station
Park Public
square
Shopping
mall
Street
pedestrian
Street
traffic
Tram
Auzanneau_CEA_task1_1 Auzanneau2024 38 37.38 39.8 33.8 42.4 40.8 32.8 64.4 16.8 37.2 18.7 58.7 52.0
BAI_JLESS_task1_1 Bai2024 32 51.99 55.8 41.2 74.6 53.7 51.7 86.4 31.7 60.2 28.5 70.2 59.6
BAI_JLESS_task1_2 Bai2024 37 50.70 53.9 43.7 77.7 55.3 47.7 86.8 29.2 47.8 26.3 64.4 60.5
BAI_JLESS_task1_3 Bai2024 30 52.05 55.2 45.0 73.9 51.1 48.0 84.9 29.3 55.2 27.0 72.1 65.1
Cai_XJTLU_task1_1 Cai2024 12 55.40 59.5 36.9 78.0 56.9 57.6 85.0 31.4 64.4 40.4 75.6 68.5
Cai_XJTLU_task1_2 Cai2024 10 56.04 59.0 33.3 75.1 58.7 49.7 85.5 36.4 66.6 37.1 75.1 72.9
Cai_XJTLU_task1_3 Cai2024 11 55.87 59.2 40.6 78.8 54.5 53.6 79.0 37.2 63.0 37.2 78.7 69.2
Chen_GXU_task1_1 Chen2024 29 52.12 54.7 33.6 70.9 45.9 54.4 88.1 25.2 56.4 44.9 59.6 68.1
Chen_SCUT_task1_1 Chen2024a 28 52.52 57.0 46.6 80.2 56.1 52.3 77.1 33.9 51.5 34.7 73.6 63.5
Chen_SCUT_task1_2 Chen2024a 35 51.66 55.7 47.9 82.2 55.1 49.3 77.6 29.8 48.4 32.4 73.3 61.1
Chen_SCUT_task1_3 Chen2024a 31 52.05 56.5 51.2 78.9 51.9 49.1 77.1 36.1 49.0 32.1 71.6 67.9
Gao_UniSA_task1_1 Gao2024 25 52.82 56.7 40.6 75.4 51.5 50.8 79.9 39.7 53.6 35.7 80.0 60.4
Han_SJTUTHU_task1_1 Bing2024 3 58.05 60.4 47.6 76.6 59.0 55.5 85.0 32.0 63.3 39.8 79.7 65.2
Han_SJTUTHU_task1_2 Bing2024 1 58.23 60.2 59.1 76.2 56.3 51.8 82.9 34.4 64.6 34.6 80.5 61.8
Han_SJTUTHU_task1_3 Bing2024 2 58.22 60.0 49.4 77.1 57.7 52.0 84.6 37.1 61.5 32.9 82.9 64.8
MALACH24_JKU_task1_1 David2024 5 57.05 59.7 44.9 74.9 57.3 53.5 82.1 34.2 67.5 38.8 78.6 65.2
MALACH24_JKU_task1_2 David2024 6 56.96 60.0 50.9 77.8 57.1 49.4 84.4 33.2 64.2 39.1 79.6 64.8
MALACH24_JKU_task1_3 David2024 7 56.47 59.0 45.7 73.8 56.5 51.8 80.8 33.1 67.7 40.0 78.4 62.4
OO_NTUPRDCSG_task1_1 Oo2024 17 54.79 58.4 39.6 73.0 56.2 55.9 81.9 32.9 63.0 36.7 76.0 68.3
OO_NTUPRDCSG_task1_2 Oo2024 15 54.79 58.4 39.9 73.5 54.8 55.2 82.7 32.7 64.0 36.9 76.0 68.5
Park_KT_task1_1 Park2024 20 53.67 56.4 37.0 64.4 68.8 54.1 78.8 26.1 69.3 33.9 71.6 60.3
Park_KT_task1_2 Park2024 13 55.36 58.1 37.7 77.9 58.6 49.0 81.6 29.3 62.1 41.2 74.0 69.5
Park_KT_task1_3 Park2024 16 54.79 57.2 41.3 71.4 57.9 55.2 79.9 31.6 58.6 36.2 73.2 66.5
DCASE2024 baseline 36 50.73 54.4 29.9 71.8 45.6 46.0 79.5 34.1 54.1 41.8 77.4 63.8
Shao_NEPUMSE_task1_1 Shao2024 4 57.15 60.6 41.2 78.4 61.7 57.0 84.2 33.1 59.6 40.4 78.9 71.7
Shao_NEPUMSE_task1_2 Shao2024 8 56.13 59.8 41.5 76.5 60.9 60.2 85.6 28.7 60.1 39.2 78.0 67.7
Shao_NEPUMSE_task1_3 Shao2024 14 55.21 59.2 39.2 78.9 60.5 60.6 88.1 26.0 61.3 38.7 75.8 63.0
Surkov_ITMO_task1_1 Surkov2024 27 52.65 54.6 41.0 64.9 57.6 46.9 83.0 28.4 57.0 31.5 72.5 63.5
Surkov_ITMO_task1_2 Surkov2024 33 51.92 53.9 37.9 70.1 53.6 45.1 82.2 28.4 54.6 33.8 71.3 61.7
Tan_CISS_task1_1 Tan2024 34 51.71 54.0 44.4 80.8 43.5 39.7 80.7 25.4 64.0 33.4 66.2 62.2
Truchan_LUH_task1_1 Truchan2024 22 53.06 57.4 46.2 69.7 60.0 44.6 76.6 32.6 62.5 38.9 80.8 62.3
Truchan_LUH_task1_2 Truchan2024 26 52.68 55.5 36.4 71.0 55.3 39.4 75.3 32.7 62.0 50.8 76.4 56.1
Truchan_LUH_task1_3 Truchan2024 24 52.91 57.2 48.7 70.4 52.7 46.9 71.0 43.1 57.7 39.2 77.7 64.5
Werning_UPBNT_task1_1 Werning2024 18 54.35 56.2 34.8 74.3 46.6 52.8 78.7 26.6 58.8 45.4 76.9 67.2
Yan_NPU_task1_1 Yan2024 23 52.94 56.4 41.8 61.1 57.7 54.3 85.6 29.0 61.9 35.5 74.4 62.4
Yeo_NTU_task1_1 Yeo2024 21 53.13 55.4 42.3 79.1 44.4 49.7 83.4 32.1 43.2 38.3 75.2 66.4
Yeo_NTU_task1_2 Yeo2024 9 56.12 59.4 42.1 72.4 60.9 51.2 79.8 33.3 68.8 39.0 80.5 65.7
Yeo_NTU_task1_3 Yeo2024 19 54.19 58.3 51.1 68.9 63.4 56.5 73.6 39.4 53.4 38.7 78.0 60.0

Split 100%

Overall Split
Rank Submission label Technical
Report
System
rank
Rank score Accuracy Airport Bus Metro Metro
station
Park Public
square
Shopping
mall
Street
pedestrian
Street
traffic
Tram
Auzanneau_CEA_task1_1 Auzanneau2024 38 37.38 41.1 29.3 48.5 42.4 30.4 67.0 25.5 35.8 26.4 50.3 55.3
BAI_JLESS_task1_1 Bai2024 32 51.99 57.1 40.8 74.6 61.2 54.4 88.8 33.9 52.3 30.8 69.0 65.6
BAI_JLESS_task1_2 Bai2024 37 50.70 56.7 42.3 77.3 60.2 54.6 82.4 31.1 47.8 31.2 74.1 66.5
BAI_JLESS_task1_3 Bai2024 30 52.05 57.3 49.2 80.1 57.7 54.1 87.4 33.0 52.3 27.1 70.1 62.1
Cai_XJTLU_task1_1 Cai2024 12 55.40 60.6 48.6 84.9 53.5 57.3 80.3 36.8 59.7 38.1 77.8 68.8
Cai_XJTLU_task1_2 Cai2024 10 56.04 62.1 52.6 82.9 56.9 56.9 83.0 36.7 65.3 36.6 79.2 71.2
Cai_XJTLU_task1_3 Cai2024 11 55.87 61.1 46.8 77.0 63.7 57.1 80.1 33.1 59.3 41.4 80.5 71.8
Chen_GXU_task1_1 Chen2024 29 52.12 57.0 38.1 71.5 65.3 53.9 85.9 32.6 53.8 35.0 74.2 59.4
Chen_SCUT_task1_1 Chen2024a 28 52.52 60.1 50.4 78.1 62.8 59.1 77.6 38.9 52.2 36.2 79.3 66.8
Chen_SCUT_task1_2 Chen2024a 35 51.66 58.1 44.1 78.2 59.0 55.8 75.4 41.5 52.6 33.5 74.1 66.4
Chen_SCUT_task1_3 Chen2024a 31 52.05 58.1 45.9 81.7 56.3 51.4 73.1 40.4 54.9 31.6 77.3 68.2
Gao_UniSA_task1_1 Gao2024 25 52.82 58.7 41.6 78.2 53.9 51.7 85.5 36.1 53.9 42.0 75.9 68.6
Han_SJTUTHU_task1_1 Bing2024 3 58.05 61.5 55.6 74.4 61.8 59.4 84.0 33.0 60.2 38.3 83.3 64.8
Han_SJTUTHU_task1_2 Bing2024 1 58.23 61.8 51.6 75.6 61.1 58.6 83.0 35.1 61.3 38.5 84.0 69.4
Han_SJTUTHU_task1_3 Bing2024 2 58.22 61.3 49.6 71.3 60.2 56.0 82.9 35.4 61.4 41.3 84.2 71.1
MALACH24_JKU_task1_1 David2024 5 57.05 61.5 46.0 78.5 59.8 55.2 83.1 35.5 66.9 42.7 79.7 67.5
MALACH24_JKU_task1_2 David2024 6 56.96 61.3 47.6 79.9 58.6 47.4 82.8 34.5 70.0 43.0 80.9 67.9
MALACH24_JKU_task1_3 David2024 7 56.47 60.9 45.8 77.1 59.7 55.4 83.2 33.7 68.3 40.6 79.3 65.7
OO_NTUPRDCSG_task1_1 Oo2024 17 54.79 59.9 43.1 77.8 61.6 56.9 78.1 31.0 62.5 44.0 75.1 68.4
OO_NTUPRDCSG_task1_2 Oo2024 15 54.79 59.9 44.1 77.3 61.7 57.7 78.0 31.1 63.1 43.3 74.5 68.3
Park_KT_task1_1 Park2024 20 53.67 56.9 35.4 74.4 61.8 59.7 71.5 26.7 63.8 38.8 76.4 60.2
Park_KT_task1_2 Park2024 13 55.36 60.3 43.0 77.3 59.3 57.1 82.6 35.2 61.2 43.8 76.2 67.5
Park_KT_task1_3 Park2024 16 54.79 59.8 40.6 78.3 63.2 56.5 81.8 34.9 60.7 37.8 75.2 68.7
DCASE2024 baseline 36 50.73 56.8 43.1 75.8 51.6 41.8 77.4 41.4 65.4 34.2 77.5 60.2
Shao_NEPUMSE_task1_1 Shao2024 4 57.15 61.7 42.3 78.8 65.1 63.8 86.4 28.4 58.7 43.6 78.6 71.3
Shao_NEPUMSE_task1_2 Shao2024 8 56.13 61.6 40.6 77.4 66.6 62.3 87.7 31.6 64.1 37.3 77.7 70.7
Shao_NEPUMSE_task1_3 Shao2024 14 55.21 61.7 44.6 81.0 65.7 59.5 90.6 29.6 59.2 39.5 78.2 68.6
Surkov_ITMO_task1_1 Surkov2024 27 52.65 55.2 39.9 67.6 58.0 48.9 82.3 29.3 55.9 34.2 72.8 62.6
Surkov_ITMO_task1_2 Surkov2024 33 51.92 55.0 36.8 74.8 57.2 49.0 84.1 26.8 57.7 31.0 72.1 60.2
Tan_CISS_task1_1 Tan2024 34 51.71 56.1 49.8 80.9 46.1 45.1 81.3 28.8 51.3 38.0 71.5 67.9
Truchan_LUH_task1_1 Truchan2024 22 53.06 59.4 55.3 75.1 58.8 48.3 80.4 34.9 62.2 40.4 76.5 62.1
Truchan_LUH_task1_2 Truchan2024 26 52.68 58.8 45.8 75.8 54.7 49.2 81.2 31.4 63.4 42.3 77.9 66.3
Truchan_LUH_task1_3 Truchan2024 24 52.91 58.4 46.0 68.4 59.6 46.9 76.9 34.4 55.3 53.8 80.9 61.9
Werning_UPBNT_task1_1 Werning2024 18 54.35 58.3 48.9 72.9 60.5 44.7 85.4 31.5 58.4 35.6 78.2 67.3
Yan_NPU_task1_1 Yan2024 23 52.94 58.8 43.7 74.1 58.5 53.0 84.4 27.7 57.6 43.0 75.4 70.6
Yeo_NTU_task1_1 Yeo2024 21 53.13 59.3 50.8 72.5 58.4 46.4 83.7 40.0 57.9 38.9 77.6 67.3
Yeo_NTU_task1_2 Yeo2024 9 56.12 61.6 51.3 73.2 58.6 57.6 81.0 36.8 61.8 41.0 81.9 72.8
Yeo_NTU_task1_3 Yeo2024 19 54.19 61.1 50.1 71.1 63.7 58.2 79.2 42.1 57.0 40.3 81.6 67.8

Device-wise performance

Split 5%

Overall Split
Overall Unseen devices Seen devices
Rank Submission label Technical
Report
System
rank
Rank score Accuracy Accuracy /
Unseen
Accuracy /
Seen
D S7 S8 S9 S10 A B C S1 S2 S3
Auzanneau_CEA_task1_1 Auzanneau2024 38 37.38 33.1 31.4 34.4 32.0 33.9 33.0 28.2 30.1 46.6 30.1 41.2 28.5 26.8 33.3
BAI_JLESS_task1_1 Bai2024 32 51.99 44.7 42.8 46.3 41.5 45.0 48.0 38.9 40.6 56.2 44.7 50.5 43.3 38.7 44.5
BAI_JLESS_task1_2 Bai2024 37 50.70 43.8 42.2 45.2 42.3 44.5 43.7 39.8 40.6 53.8 41.8 47.9 43.0 39.9 44.8
BAI_JLESS_task1_3 Bai2024 30 52.05 43.7 41.5 45.5 40.5 43.2 46.0 37.5 40.2 56.0 42.0 48.9 43.4 38.8 43.7
Cai_XJTLU_task1_1 Cai2024 12 55.40 47.5 45.8 49.0 40.7 50.5 45.9 46.0 45.8 55.6 46.7 50.0 48.3 45.1 48.5
Cai_XJTLU_task1_2 Cai2024 10 56.04 48.9 46.8 50.7 46.1 50.4 47.6 45.0 44.7 55.9 49.5 51.9 48.8 47.7 50.3
Cai_XJTLU_task1_3 Cai2024 11 55.87 48.4 47.2 49.4 43.7 51.0 46.2 46.8 48.2 53.8 48.7 50.1 48.7 45.8 49.1
Chen_GXU_task1_1 Chen2024 29 52.12 45.2 42.9 47.1 42.5 45.1 46.2 40.2 40.7 57.8 46.0 50.6 41.8 40.4 45.9
Chen_SCUT_task1_1 Chen2024a 28 52.52 43.9 39.2 47.9 21.5 48.4 45.4 37.0 43.5 50.7 45.6 48.9 47.3 44.5 50.5
Chen_SCUT_task1_2 Chen2024a 35 51.66 43.1 38.6 46.7 17.2 47.2 46.2 39.0 43.7 42.8 43.2 47.2 49.0 46.7 51.6
Chen_SCUT_task1_3 Chen2024a 31 52.05 43.1 38.4 47.1 17.9 48.3 47.3 36.5 41.8 45.8 46.1 48.1 46.8 45.4 50.5
Gao_UniSA_task1_1 Gao2024 25 52.82 45.9 43.8 47.6 41.6 46.1 44.6 43.8 43.1 56.7 45.9 49.5 46.0 41.2 46.5
Han_SJTUTHU_task1_1 Bing2024 3 58.05 53.4 50.8 55.5 44.5 54.6 51.4 52.6 50.9 63.6 52.8 57.5 53.2 50.8 55.3
Han_SJTUTHU_task1_2 Bing2024 1 58.23 54.3 52.8 55.6 47.9 56.1 52.7 54.4 52.8 64.8 53.4 56.4 54.2 49.7 55.3
Han_SJTUTHU_task1_3 Bing2024 2 58.22 54.0 52.3 55.4 48.7 55.3 52.8 53.1 51.5 64.3 53.2 56.4 53.7 49.2 55.6
MALACH24_JKU_task1_1 David2024 5 57.05 51.6 50.9 52.1 49.5 52.9 50.7 51.6 50.0 57.8 51.0 52.5 50.5 48.1 52.6
MALACH24_JKU_task1_2 David2024 6 56.96 52.0 51.4 52.4 49.5 54.2 50.9 52.0 50.2 58.0 50.0 52.7 51.3 49.0 53.6
MALACH24_JKU_task1_3 David2024 7 56.47 51.2 50.4 51.8 49.2 52.7 49.6 51.5 49.1 57.9 50.6 52.2 50.1 48.0 52.0
OO_NTUPRDCSG_task1_1 Oo2024 17 54.79 48.5 46.6 50.1 42.4 50.1 49.1 46.4 44.9 59.1 48.4 52.4 47.4 44.5 49.0
OO_NTUPRDCSG_task1_2 Oo2024 15 54.79 48.4 46.7 49.9 42.4 50.4 48.0 46.8 45.7 58.9 47.5 51.5 47.8 44.7 48.9
Park_KT_task1_1 Park2024 20 53.67 48.9 47.0 50.4 40.4 51.8 50.9 46.6 45.2 56.3 49.3 49.9 48.6 47.2 51.2
Park_KT_task1_2 Park2024 13 55.36 48.7 45.8 51.1 39.7 50.4 50.3 44.6 43.8 59.0 47.6 53.8 48.5 47.8 49.8
Park_KT_task1_3 Park2024 16 54.79 48.5 46.2 50.4 37.8 49.7 50.1 47.7 45.5 57.7 49.0 51.9 48.3 45.8 49.8
DCASE2024 baseline 36 50.73 44.0 42.4 45.3 42.3 46.0 42.6 39.3 41.8 56.3 43.9 47.5 41.3 37.9 45.2
Shao_NEPUMSE_task1_1 Shao2024 4 57.15 51.4 48.9 53.5 44.3 53.8 49.2 47.6 49.4 61.3 52.5 55.2 50.9 48.0 52.9
Shao_NEPUMSE_task1_2 Shao2024 8 56.13 48.5 44.6 51.8 39.4 50.5 50.2 38.3 44.7 61.4 51.4 54.2 48.2 44.9 50.6
Shao_NEPUMSE_task1_3 Shao2024 14 55.21 46.2 42.1 49.7 36.4 48.2 45.0 36.1 44.5 60.3 49.3 54.0 45.6 41.6 47.1
Surkov_ITMO_task1_1 Surkov2024 27 52.65 48.0 46.6 49.3 47.0 48.7 49.6 44.2 43.2 62.5 46.2 55.8 43.4 38.5 49.1
Surkov_ITMO_task1_2 Surkov2024 33 51.92 47.8 46.1 49.2 47.2 47.7 48.1 43.2 44.2 61.5 46.4 54.7 43.6 38.7 50.1
Tan_CISS_task1_1 Tan2024 34 51.71 46.5 44.3 48.3 41.9 46.2 44.2 44.1 45.2 53.2 45.7 52.0 45.2 45.8 47.9
Truchan_LUH_task1_1 Truchan2024 22 53.06 45.3 43.9 46.4 42.2 47.0 43.0 43.7 43.9 52.7 45.4 47.6 44.6 42.6 45.7
Truchan_LUH_task1_2 Truchan2024 26 52.68 46.0 44.6 47.2 41.5 47.7 43.2 45.4 45.1 54.2 45.9 48.5 45.2 42.9 46.6
Truchan_LUH_task1_3 Truchan2024 24 52.91 44.9 43.2 46.4 41.9 45.8 41.1 43.4 43.8 52.6 45.0 47.2 44.8 42.1 46.7
Werning_UPBNT_task1_1 Werning2024 18 54.35 49.2 48.1 50.2 49.4 49.5 49.5 47.3 44.6 62.0 49.8 52.9 45.2 42.9 48.2
Yan_NPU_task1_1 Yan2024 23 52.94 47.5 45.9 48.9 41.1 49.0 49.2 46.2 43.8 59.3 48.0 53.0 46.1 40.9 46.0
Yeo_NTU_task1_1 Yeo2024 21 53.13 46.3 44.3 47.9 41.5 46.4 46.3 43.4 44.1 53.1 44.8 50.3 46.0 44.1 48.9
Yeo_NTU_task1_2 Yeo2024 9 56.12 48.9 47.8 49.7 47.9 50.9 49.5 45.3 45.7 55.4 48.4 51.8 48.1 44.8 50.0
Yeo_NTU_task1_3 Yeo2024 19 54.19 46.0 44.7 47.0 43.9 47.6 44.1 43.6 44.2 52.2 46.5 48.6 45.3 43.0 46.6

Split 10%

Overall Split
Overall Unseen devices Seen devices
Rank Submission label Technical
Report
System
rank
Rank score Accuracy Accuracy /
Unseen
Accuracy /
Seen
D S7 S8 S9 S10 A B C S1 S2 S3
Auzanneau_CEA_task1_1 Auzanneau2024 38 37.38 35.3 33.4 36.9 32.5 36.2 36.2 29.4 32.5 48.0 33.8 43.2 31.2 30.1 35.3
BAI_JLESS_task1_1 Bai2024 32 51.99 47.8 44.8 50.3 38.9 48.6 50.2 41.5 44.6 59.2 47.2 51.9 47.4 46.6 49.4
BAI_JLESS_task1_2 Bai2024 37 50.70 46.9 43.8 49.6 41.2 47.3 49.2 38.7 42.4 58.7 45.9 51.9 47.6 44.4 48.8
BAI_JLESS_task1_3 Bai2024 30 52.05 49.3 46.5 51.7 42.2 50.1 51.9 42.7 45.5 60.4 47.8 53.3 50.0 46.9 51.7
Cai_XJTLU_task1_1 Cai2024 12 55.40 52.4 50.2 54.2 44.8 54.0 50.6 50.6 50.9 60.0 52.7 54.7 53.5 51.0 53.3
Cai_XJTLU_task1_2 Cai2024 10 56.04 52.0 50.2 53.6 47.4 55.0 50.7 48.3 49.6 59.8 50.6 54.4 52.8 49.5 54.5
Cai_XJTLU_task1_3 Cai2024 11 55.87 53.2 51.5 54.6 48.9 55.8 51.2 50.4 51.3 61.0 52.5 56.0 53.5 50.1 54.3
Chen_GXU_task1_1 Chen2024 29 52.12 49.8 47.7 51.6 45.2 51.4 51.2 43.8 47.0 61.7 48.8 54.2 47.7 45.0 52.1
Chen_SCUT_task1_1 Chen2024a 28 52.52 47.8 42.9 51.9 21.0 51.8 52.4 43.8 45.3 49.6 46.8 51.7 53.6 54.6 55.0
Chen_SCUT_task1_2 Chen2024a 35 51.66 48.2 43.6 52.1 20.2 52.2 53.7 45.9 45.8 51.2 46.4 52.6 53.3 53.8 55.5
Chen_SCUT_task1_3 Chen2024a 31 52.05 48.9 44.3 52.7 24.8 53.6 53.6 44.4 45.3 53.9 47.6 52.8 54.3 53.2 54.3
Gao_UniSA_task1_1 Gao2024 25 52.82 49.5 46.9 51.6 43.3 50.5 50.3 44.4 45.8 61.5 50.7 53.3 46.6 46.2 51.3
Han_SJTUTHU_task1_1 Bing2024 3 58.05 56.4 54.1 58.3 50.2 57.7 55.0 53.4 54.1 67.6 56.4 59.2 55.1 53.3 58.3
Han_SJTUTHU_task1_2 Bing2024 1 58.23 55.8 52.8 58.3 46.0 57.0 53.9 54.1 53.0 67.0 56.6 59.5 55.3 54.1 57.6
Han_SJTUTHU_task1_3 Bing2024 2 58.22 56.7 54.1 58.8 48.6 57.7 56.0 54.8 53.5 66.5 56.5 60.0 56.2 55.2 58.5
MALACH24_JKU_task1_1 David2024 5 57.05 54.5 53.3 55.5 50.3 56.2 53.0 53.3 53.5 60.8 53.3 54.5 54.7 52.8 56.6
MALACH24_JKU_task1_2 David2024 6 56.96 54.0 53.0 54.9 49.3 56.1 52.2 53.9 53.6 59.8 51.9 53.6 55.0 52.3 56.6
MALACH24_JKU_task1_3 David2024 7 56.47 54.0 52.6 55.1 49.4 55.8 51.9 52.8 53.3 59.9 53.1 54.4 54.1 52.6 56.4
OO_NTUPRDCSG_task1_1 Oo2024 17 54.79 51.4 49.0 53.5 40.6 54.2 50.5 50.7 48.9 61.9 52.9 54.4 51.4 48.5 51.6
OO_NTUPRDCSG_task1_2 Oo2024 15 54.79 51.3 48.9 53.3 39.9 54.0 50.5 50.9 49.4 61.5 52.4 54.1 51.6 48.5 51.8
Park_KT_task1_1 Park2024 20 53.67 52.4 50.9 53.7 46.8 54.7 54.0 51.9 47.0 61.7 52.3 54.6 52.5 49.2 51.7
Park_KT_task1_2 Park2024 13 55.36 52.4 49.9 54.6 41.5 55.0 54.6 49.7 48.7 63.0 52.6 55.9 52.2 49.8 53.9
Park_KT_task1_3 Park2024 16 54.79 51.9 49.1 54.1 40.5 54.7 54.0 49.2 47.2 62.1 52.5 53.9 52.3 50.1 53.8
DCASE2024 baseline 36 50.73 46.9 43.8 49.5 38.6 47.9 45.5 43.0 44.2 60.2 48.1 51.3 45.8 43.0 48.9
Shao_NEPUMSE_task1_1 Shao2024 4 57.15 53.8 49.7 57.1 40.1 55.5 53.7 48.7 50.3 65.8 56.5 58.5 54.8 51.9 55.4
Shao_NEPUMSE_task1_2 Shao2024 8 56.13 53.0 48.9 56.4 39.2 55.0 54.4 47.4 48.5 65.5 54.9 57.9 53.4 51.4 55.2
Shao_NEPUMSE_task1_3 Shao2024 14 55.21 51.8 47.8 55.1 37.8 55.2 53.2 43.1 49.7 64.1 54.5 55.9 52.5 49.2 54.7
Surkov_ITMO_task1_1 Surkov2024 27 52.65 51.2 48.3 53.6 46.7 51.3 50.7 46.9 45.9 64.4 50.5 56.1 50.0 46.7 53.7
Surkov_ITMO_task1_2 Surkov2024 33 51.92 50.3 46.9 53.0 46.4 49.6 49.2 46.5 43.1 63.3 50.1 54.5 48.8 47.9 53.5
Tan_CISS_task1_1 Tan2024 34 51.71 49.3 47.5 50.8 46.2 50.3 47.4 44.1 49.5 52.6 47.8 54.4 49.9 47.9 52.4
Truchan_LUH_task1_1 Truchan2024 22 53.06 48.1 45.7 50.1 40.9 50.2 43.6 46.6 47.4 56.9 49.0 50.7 48.8 46.3 49.1
Truchan_LUH_task1_2 Truchan2024 26 52.68 49.6 46.7 52.1 39.8 51.2 46.8 47.2 48.5 58.8 51.3 52.4 50.4 48.1 51.6
Truchan_LUH_task1_3 Truchan2024 24 52.91 48.9 46.2 51.1 41.5 50.7 45.1 46.5 47.4 57.5 49.7 51.7 49.4 47.6 50.4
Werning_UPBNT_task1_1 Werning2024 18 54.35 52.5 50.8 54.0 49.4 52.7 49.5 51.4 50.9 64.0 52.7 53.9 51.1 48.3 53.7
Yan_NPU_task1_1 Yan2024 23 52.94 49.1 46.0 51.6 41.1 49.3 48.4 44.8 46.6 62.6 50.2 52.3 48.4 46.2 49.9
Yeo_NTU_task1_1 Yeo2024 21 53.13 49.4 47.5 50.9 42.7 50.5 49.9 46.9 47.4 57.9 47.4 54.6 49.8 45.8 50.1
Yeo_NTU_task1_2 Yeo2024 9 56.12 53.3 50.7 55.5 46.6 55.0 49.6 50.8 51.3 61.9 53.3 55.2 54.1 53.2 55.1
Yeo_NTU_task1_3 Yeo2024 19 54.19 50.2 48.2 51.9 45.3 51.7 47.0 48.8 48.1 57.2 49.6 51.8 50.0 50.5 52.0

Split 25%

Overall Split
Overall Unseen devices Seen devices
Rank Submission label Technical
Report
System
rank
Rank score Accuracy Accuracy /
Unseen
Accuracy /
Seen
D S7 S8 S9 S10 A B C S1 S2 S3
Auzanneau_CEA_task1_1 Auzanneau2024 38 37.38 37.7 34.4 40.4 32.0 37.1 37.9 30.6 34.5 50.4 39.2 44.1 34.9 35.7 38.2
BAI_JLESS_task1_1 Bai2024 32 51.99 54.5 51.3 57.2 49.4 54.9 55.2 48.4 48.6 66.1 55.7 59.6 53.4 53.2 55.4
BAI_JLESS_task1_2 Bai2024 37 50.70 52.0 49.1 54.5 44.2 52.6 54.9 45.8 47.7 63.0 53.2 57.7 51.3 48.1 53.9
BAI_JLESS_task1_3 Bai2024 30 52.05 54.9 51.8 57.4 48.3 55.3 55.8 48.9 50.8 66.8 55.4 60.6 53.0 52.3 56.2
Cai_XJTLU_task1_1 Cai2024 12 55.40 57.0 54.4 59.3 49.0 59.2 54.1 53.6 56.1 64.8 57.9 59.4 57.9 56.2 59.3
Cai_XJTLU_task1_2 Cai2024 10 56.04 58.1 56.0 59.8 51.2 61.0 54.8 55.4 57.7 65.7 58.7 60.0 58.6 56.6 59.4
Cai_XJTLU_task1_3 Cai2024 11 55.87 57.6 54.9 59.8 50.6 59.9 55.0 53.7 55.4 65.9 59.0 59.6 58.0 56.5 59.6
Chen_GXU_task1_1 Chen2024 29 52.12 53.9 51.9 55.5 45.7 57.5 54.8 50.7 51.0 66.5 56.0 58.0 52.1 47.5 53.2
Chen_SCUT_task1_1 Chen2024a 28 52.52 53.8 47.3 59.1 25.5 57.0 57.0 45.2 51.8 55.8 56.9 60.4 60.4 60.1 61.2
Chen_SCUT_task1_2 Chen2024a 35 51.66 53.3 46.7 58.7 25.9 55.9 56.2 44.9 50.7 57.4 57.4 60.4 58.1 58.9 60.0
Chen_SCUT_task1_3 Chen2024a 31 52.05 53.6 47.4 58.8 30.3 56.6 55.0 43.9 51.3 57.9 56.9 59.1 58.8 59.4 60.8
Gao_UniSA_task1_1 Gao2024 25 52.82 53.3 50.1 55.9 45.1 53.5 50.0 49.2 52.6 65.1 56.4 55.4 53.9 50.4 54.5
Han_SJTUTHU_task1_1 Bing2024 3 58.05 58.6 55.6 61.1 49.1 59.7 57.6 54.9 56.9 69.7 61.2 62.8 56.8 56.0 60.1
Han_SJTUTHU_task1_2 Bing2024 1 58.23 58.9 56.3 61.1 52.2 60.1 58.4 55.6 55.5 69.4 60.9 62.3 56.5 57.2 60.3
Han_SJTUTHU_task1_3 Bing2024 2 58.22 59.1 56.4 61.3 50.1 60.0 57.6 57.0 57.6 69.2 61.1 62.9 57.8 56.1 60.7
MALACH24_JKU_task1_1 David2024 5 57.05 58.0 56.6 59.1 50.8 60.6 56.5 57.1 58.2 64.5 56.6 59.2 58.4 56.0 60.3
MALACH24_JKU_task1_2 David2024 6 56.96 57.5 56.4 58.5 50.1 60.5 56.1 57.3 57.8 64.1 55.1 58.7 58.3 55.0 59.7
MALACH24_JKU_task1_3 David2024 7 56.47 57.3 55.7 58.6 49.5 60.0 55.4 56.2 57.5 63.8 56.5 58.1 57.8 56.3 59.4
OO_NTUPRDCSG_task1_1 Oo2024 17 54.79 55.8 52.2 58.7 42.3 57.9 56.2 52.2 52.3 66.1 57.7 59.6 56.5 54.9 57.6
OO_NTUPRDCSG_task1_2 Oo2024 15 54.79 55.9 52.4 58.7 42.8 57.5 56.4 52.8 52.6 66.1 57.9 59.6 56.5 54.6 57.7
Park_KT_task1_1 Park2024 20 53.67 53.8 49.3 57.5 42.4 57.1 55.4 44.4 47.3 65.1 57.6 58.3 55.2 52.2 56.4
Park_KT_task1_2 Park2024 13 55.36 57.3 54.0 60.1 49.9 59.4 57.0 50.7 53.0 67.0 58.8 60.7 58.2 55.5 60.1
Park_KT_task1_3 Park2024 16 54.79 56.6 52.9 59.7 42.9 60.3 58.4 51.4 51.6 66.4 59.1 61.4 56.1 55.7 59.7
DCASE2024 baseline 36 50.73 51.5 49.5 53.1 46.5 52.7 48.8 46.3 53.2 63.1 54.0 54.3 50.1 44.9 52.4
Shao_NEPUMSE_task1_1 Shao2024 4 57.15 58.3 54.2 61.7 45.9 60.2 57.3 52.9 54.6 69.0 61.4 62.4 59.3 57.1 61.3
Shao_NEPUMSE_task1_2 Shao2024 8 56.13 57.7 53.0 61.6 43.8 58.2 57.1 51.4 54.3 70.0 61.8 62.8 59.0 55.4 60.5
Shao_NEPUMSE_task1_3 Shao2024 14 55.21 57.2 52.5 61.1 44.1 58.5 56.8 50.0 53.0 68.5 61.2 62.1 58.4 55.7 60.6
Surkov_ITMO_task1_1 Surkov2024 27 52.65 54.3 51.5 56.6 50.4 54.7 53.1 49.9 49.3 65.2 55.9 58.6 52.8 50.9 56.4
Surkov_ITMO_task1_2 Surkov2024 33 51.92 52.7 49.7 55.3 47.7 52.3 52.9 48.3 47.5 63.8 54.9 57.3 50.6 50.0 55.0
Tan_CISS_task1_1 Tan2024 34 51.71 52.6 49.0 55.6 40.2 53.0 50.9 49.1 52.1 53.3 56.1 59.9 53.7 54.1 56.5
Truchan_LUH_task1_1 Truchan2024 22 53.06 55.0 53.2 56.5 46.3 57.7 52.2 54.1 55.6 62.4 55.3 56.7 54.7 53.2 57.0
Truchan_LUH_task1_2 Truchan2024 26 52.68 53.4 51.8 54.7 47.8 56.6 49.2 51.7 53.6 60.8 54.8 54.3 53.7 50.2 54.6
Truchan_LUH_task1_3 Truchan2024 24 52.91 55.1 53.5 56.6 48.7 57.7 52.8 52.9 55.3 62.8 55.5 56.6 55.2 52.5 56.7
Werning_UPBNT_task1_1 Werning2024 18 54.35 55.5 53.6 57.1 51.1 56.5 54.4 52.8 53.3 66.4 57.2 56.9 52.9 52.2 56.7
Yan_NPU_task1_1 Yan2024 23 52.94 53.0 49.8 55.6 40.6 55.3 53.1 48.1 52.0 65.1 56.8 57.7 52.1 48.5 53.4
Yeo_NTU_task1_1 Yeo2024 21 53.13 55.2 52.8 57.3 48.6 55.7 55.4 52.4 52.0 63.5 55.8 58.3 55.5 53.5 57.2
Yeo_NTU_task1_2 Yeo2024 9 56.12 57.5 55.8 58.8 52.3 60.1 53.6 56.3 56.9 65.2 56.2 58.5 58.4 55.5 59.3
Yeo_NTU_task1_3 Yeo2024 19 54.19 55.4 52.7 57.7 46.9 58.1 51.1 54.0 53.4 63.5 56.6 57.5 55.9 54.7 57.7

Split 50%

Overall Split
Overall Unseen devices Seen devices
Rank Submission label Technical
Report
System
rank
Rank score Accuracy Accuracy /
Unseen
Accuracy /
Seen
D S7 S8 S9 S10 A B C S1 S2 S3
Auzanneau_CEA_task1_1 Auzanneau2024 38 37.38 39.8 35.5 43.3 29.7 39.0 41.1 32.8 34.9 52.4 42.1 48.4 39.4 35.0 42.5
BAI_JLESS_task1_1 Bai2024 32 51.99 55.8 51.8 59.1 42.3 56.8 56.3 51.0 52.6 67.4 58.5 61.8 56.0 54.0 56.6
BAI_JLESS_task1_2 Bai2024 37 50.70 53.9 48.8 58.2 39.5 54.4 56.1 44.6 49.4 66.9 58.3 61.6 54.9 51.0 56.4
BAI_JLESS_task1_3 Bai2024 30 52.05 55.2 50.7 58.8 43.7 56.4 55.7 49.3 48.6 67.1 58.6 61.9 56.4 52.8 56.4
Cai_XJTLU_task1_1 Cai2024 12 55.40 59.5 56.8 61.7 50.7 61.1 57.9 56.4 57.8 67.2 59.7 61.7 60.6 59.5 61.7
Cai_XJTLU_task1_2 Cai2024 10 56.04 59.0 56.1 61.5 48.7 60.0 57.4 57.7 56.8 67.3 59.6 61.3 60.5 59.0 61.1
Cai_XJTLU_task1_3 Cai2024 11 55.87 59.2 56.3 61.6 49.5 60.5 55.6 57.6 58.2 66.8 59.7 61.8 60.7 59.5 61.2
Chen_GXU_task1_1 Chen2024 29 52.12 54.7 50.6 58.1 48.0 55.0 54.5 45.7 49.9 67.5 58.7 60.3 53.8 53.6 55.0
Chen_SCUT_task1_1 Chen2024a 28 52.52 57.0 50.0 62.7 29.4 60.0 58.7 47.3 54.9 60.9 60.9 63.7 63.3 63.5 64.0
Chen_SCUT_task1_2 Chen2024a 35 51.66 55.7 49.1 61.2 30.0 57.5 58.6 46.9 52.4 60.2 60.0 63.3 61.8 60.1 62.0
Chen_SCUT_task1_3 Chen2024a 31 52.05 56.5 50.0 61.9 36.8 58.2 57.9 45.0 51.9 60.7 59.9 63.6 62.1 62.1 63.2
Gao_UniSA_task1_1 Gao2024 25 52.82 56.7 53.6 59.4 46.8 58.0 55.0 53.3 54.9 67.4 59.8 58.8 57.2 55.1 58.0
Han_SJTUTHU_task1_1 Bing2024 3 58.05 60.4 57.3 62.9 48.1 62.3 59.5 58.1 58.6 71.0 62.5 63.6 60.2 58.0 62.2
Han_SJTUTHU_task1_2 Bing2024 1 58.23 60.2 57.7 62.4 52.7 62.0 58.1 56.6 59.0 70.3 62.0 63.7 59.5 57.6 61.3
Han_SJTUTHU_task1_3 Bing2024 2 58.22 60.0 56.7 62.8 48.5 61.0 58.0 57.3 58.7 70.4 62.6 63.7 60.0 58.8 61.3
MALACH24_JKU_task1_1 David2024 5 57.05 59.7 57.4 61.6 49.3 61.4 57.6 59.4 59.6 66.6 59.2 61.2 61.2 59.3 62.0
MALACH24_JKU_task1_2 David2024 6 56.96 60.0 58.2 61.6 51.0 62.4 58.1 59.6 60.0 66.6 58.7 60.8 61.6 59.5 62.3
MALACH24_JKU_task1_3 David2024 7 56.47 59.0 56.9 60.8 48.0 61.0 56.9 58.7 59.6 66.1 58.3 60.2 60.7 58.5 61.2
OO_NTUPRDCSG_task1_1 Oo2024 17 54.79 58.4 54.8 61.3 46.1 60.6 57.4 55.0 55.1 67.8 60.7 62.5 59.6 56.9 60.2
OO_NTUPRDCSG_task1_2 Oo2024 15 54.79 58.4 54.8 61.4 46.7 60.6 56.9 54.8 55.3 67.6 61.0 63.3 59.2 56.8 60.6
Park_KT_task1_1 Park2024 20 53.67 56.4 52.4 59.8 46.6 59.4 57.6 49.6 48.9 67.1 57.8 60.3 59.1 56.0 58.2
Park_KT_task1_2 Park2024 13 55.36 58.1 53.2 62.2 49.5 58.9 57.6 51.4 48.6 69.2 61.6 62.2 59.6 59.8 60.8
Park_KT_task1_3 Park2024 16 54.79 57.2 52.6 61.0 45.8 58.4 56.8 49.7 52.4 68.6 59.8 61.5 59.1 57.0 59.9
DCASE2024 baseline 36 50.73 54.4 51.1 57.2 44.4 55.1 53.3 48.4 54.1 66.6 58.6 58.0 53.8 50.0 56.2
Shao_NEPUMSE_task1_1 Shao2024 4 57.15 60.6 56.5 64.1 49.6 61.6 59.7 54.8 56.6 71.0 64.4 65.0 61.2 59.3 63.4
Shao_NEPUMSE_task1_2 Shao2024 8 56.13 59.8 55.2 63.7 45.7 61.9 59.3 52.5 56.7 70.1 63.1 64.3 61.6 59.9 63.1
Shao_NEPUMSE_task1_3 Shao2024 14 55.21 59.2 54.0 63.6 45.7 60.0 57.1 51.0 56.1 70.4 62.6 64.3 61.7 60.3 62.1
Surkov_ITMO_task1_1 Surkov2024 27 52.65 54.6 51.6 57.1 50.6 54.8 54.5 50.5 47.7 66.1 56.5 59.6 53.5 51.6 55.5
Surkov_ITMO_task1_2 Surkov2024 33 51.92 53.9 50.6 56.6 49.9 53.5 53.7 48.5 47.3 64.9 56.7 58.7 52.7 50.5 56.1
Tan_CISS_task1_1 Tan2024 34 51.71 54.0 50.9 56.6 44.9 54.6 51.8 49.9 53.4 55.1 58.1 61.0 54.8 53.9 56.8
Truchan_LUH_task1_1 Truchan2024 22 53.06 57.4 55.5 59.1 49.8 59.9 54.6 56.0 56.9 65.0 58.3 59.1 57.4 56.1 58.4
Truchan_LUH_task1_2 Truchan2024 26 52.68 55.5 53.2 57.5 48.2 58.0 50.7 52.8 56.3 64.3 57.0 57.1 55.0 54.4 57.0
Truchan_LUH_task1_3 Truchan2024 24 52.91 57.2 54.9 59.1 50.2 59.0 54.0 54.1 57.2 64.8 58.5 59.0 58.2 55.9 58.1
Werning_UPBNT_task1_1 Werning2024 18 54.35 56.2 53.1 58.8 45.4 57.4 55.7 52.9 54.1 67.2 58.9 59.6 56.2 54.2 56.6
Yan_NPU_task1_1 Yan2024 23 52.94 56.4 53.1 59.1 44.6 58.5 54.6 52.9 54.8 68.3 58.2 58.5 57.3 53.9 58.2
Yeo_NTU_task1_1 Yeo2024 21 53.13 55.4 51.7 58.5 45.6 55.0 54.7 50.4 53.1 62.3 57.0 60.3 55.7 57.3 58.1
Yeo_NTU_task1_2 Yeo2024 9 56.12 59.4 56.8 61.5 53.0 61.7 55.8 57.2 56.3 67.4 59.9 61.6 59.7 59.3 61.0
Yeo_NTU_task1_3 Yeo2024 19 54.19 58.3 55.3 60.8 46.0 60.8 54.2 58.0 57.3 65.6 59.2 60.2 60.3 58.5 61.1

Split 100%

Overall Split
Overall Unseen devices Seen devices
Rank Submission label Technical
Report
System
rank
Rank score Accuracy Accuracy /
Unseen
Accuracy /
Seen
D S7 S8 S9 S10 A B C S1 S2 S3
Auzanneau_CEA_task1_1 Auzanneau2024 38 37.38 41.1 35.8 45.5 30.5 40.3 41.4 34.0 33.0 54.0 47.0 48.7 41.3 38.2 43.6
BAI_JLESS_task1_1 Bai2024 32 51.99 57.1 51.8 61.6 42.9 59.3 57.1 48.5 51.0 68.6 61.2 63.7 58.9 57.2 60.1
BAI_JLESS_task1_2 Bai2024 37 50.70 56.7 51.4 61.2 41.8 57.8 59.2 47.8 50.4 68.7 61.1 62.6 59.2 55.9 59.5
BAI_JLESS_task1_3 Bai2024 30 52.05 57.3 52.3 61.4 43.7 58.9 57.6 50.9 50.5 68.1 61.5 63.1 59.3 56.7 60.0
Cai_XJTLU_task1_1 Cai2024 12 55.40 60.6 57.4 63.2 50.3 61.9 57.6 57.3 59.9 68.6 62.6 62.8 61.8 61.0 62.5
Cai_XJTLU_task1_2 Cai2024 10 56.04 62.1 59.4 64.4 54.6 64.1 60.8 57.7 59.9 69.5 63.2 64.4 63.0 62.5 63.5
Cai_XJTLU_task1_3 Cai2024 11 55.87 61.1 57.7 63.9 51.7 62.6 58.2 57.4 58.8 69.5 62.0 63.6 62.6 62.1 63.5
Chen_GXU_task1_1 Chen2024 29 52.12 57.0 53.3 60.0 42.3 59.0 58.2 52.2 54.6 67.8 60.5 59.9 57.8 54.8 59.5
Chen_SCUT_task1_1 Chen2024a 28 52.52 60.1 53.1 66.0 36.4 62.3 60.7 53.4 52.7 67.5 65.0 66.2 65.7 65.5 66.0
Chen_SCUT_task1_2 Chen2024a 35 51.66 58.1 51.3 63.7 32.9 60.8 59.6 52.9 50.4 63.3 62.0 64.3 63.9 63.6 65.0
Chen_SCUT_task1_3 Chen2024a 31 52.05 58.1 51.6 63.4 31.9 61.2 60.2 53.4 51.6 63.1 61.8 64.1 63.7 63.7 64.2
Gao_UniSA_task1_1 Gao2024 25 52.82 58.7 55.2 61.7 47.4 58.6 58.0 54.6 57.2 69.4 62.6 61.0 59.4 57.8 59.9
Han_SJTUTHU_task1_1 Bing2024 3 58.05 61.5 58.6 63.8 51.4 62.6 60.6 59.2 59.4 71.8 63.6 64.6 61.4 59.1 62.7
Han_SJTUTHU_task1_2 Bing2024 1 58.23 61.8 58.6 64.5 52.4 62.3 60.9 58.3 58.9 71.9 64.0 65.3 62.2 60.5 63.2
Han_SJTUTHU_task1_3 Bing2024 2 58.22 61.3 58.4 63.8 51.1 62.8 60.0 59.4 58.7 71.6 63.1 65.1 61.3 58.9 62.7
MALACH24_JKU_task1_1 David2024 5 57.05 61.5 59.1 63.5 51.8 63.4 58.6 60.5 61.4 68.3 60.4 62.4 63.4 62.1 64.2
MALACH24_JKU_task1_2 David2024 6 56.96 61.3 59.5 62.7 53.4 63.9 58.5 60.9 60.6 67.5 59.3 62.0 63.2 60.6 63.9
MALACH24_JKU_task1_3 David2024 7 56.47 60.9 58.8 62.7 50.4 63.0 58.7 60.4 61.2 67.2 60.0 61.6 62.5 61.4 63.3
OO_NTUPRDCSG_task1_1 Oo2024 17 54.79 59.9 55.6 63.4 50.5 61.4 57.7 54.8 53.4 69.5 62.9 63.9 61.6 60.3 62.3
OO_NTUPRDCSG_task1_2 Oo2024 15 54.79 59.9 55.7 63.4 51.1 61.4 57.7 55.1 53.3 69.7 62.7 63.9 61.7 59.9 62.5
Park_KT_task1_1 Park2024 20 53.67 56.9 52.2 60.8 45.8 59.3 54.2 49.2 52.5 66.2 60.1 60.5 60.3 57.4 60.1
Park_KT_task1_2 Park2024 13 55.36 60.3 56.4 63.6 53.1 62.6 59.1 52.3 54.7 68.8 62.9 63.3 62.3 60.8 63.6
Park_KT_task1_3 Park2024 16 54.79 59.8 54.9 63.9 48.0 62.6 58.9 52.1 52.8 69.0 63.5 63.7 62.8 61.4 62.9
DCASE2024 baseline 36 50.73 56.8 53.9 59.3 47.0 58.4 56.0 53.1 54.9 66.7 61.1 60.3 56.5 54.2 57.0
Shao_NEPUMSE_task1_1 Shao2024 4 57.15 61.7 57.2 65.5 44.9 64.0 60.3 58.3 58.6 71.2 65.0 65.2 64.1 62.8 64.5
Shao_NEPUMSE_task1_2 Shao2024 8 56.13 61.6 56.9 65.5 47.9 62.9 60.3 55.8 57.7 71.8 64.9 66.1 64.0 62.4 63.9
Shao_NEPUMSE_task1_3 Shao2024 14 55.21 61.7 56.4 66.1 47.2 64.3 59.8 53.4 57.1 72.2 65.2 66.0 64.5 63.6 64.9
Surkov_ITMO_task1_1 Surkov2024 27 52.65 55.2 50.7 58.9 48.9 54.7 53.4 49.7 46.7 65.5 59.2 60.5 56.7 53.9 57.4
Surkov_ITMO_task1_2 Surkov2024 33 51.92 55.0 50.9 58.4 47.7 55.1 54.6 50.3 46.8 65.9 58.6 60.6 56.1 53.3 55.7
Tan_CISS_task1_1 Tan2024 34 51.71 56.1 52.2 59.3 45.0 56.9 54.9 47.8 56.5 59.1 58.5 64.0 57.7 56.5 60.0
Truchan_LUH_task1_1 Truchan2024 22 53.06 59.4 57.2 61.2 51.6 61.0 57.1 58.1 58.1 66.5 60.6 60.0 60.4 58.2 61.7
Truchan_LUH_task1_2 Truchan2024 26 52.68 58.8 56.3 60.9 51.0 61.3 55.4 56.0 57.9 65.4 60.6 60.4 60.0 57.7 61.3
Truchan_LUH_task1_3 Truchan2024 24 52.91 58.4 56.3 60.2 50.3 60.8 54.6 56.1 59.6 65.6 59.3 59.3 59.3 56.9 60.4
Werning_UPBNT_task1_1 Werning2024 18 54.35 58.3 54.9 61.2 46.9 57.7 58.7 53.1 58.0 68.5 61.6 61.4 59.0 56.8 60.0
Yan_NPU_task1_1 Yan2024 23 52.94 58.8 54.8 62.1 44.8 60.2 59.0 55.5 54.2 69.6 62.0 62.2 59.7 58.0 61.5
Yeo_NTU_task1_1 Yeo2024 21 53.13 59.3 56.7 61.5 52.6 60.4 58.1 56.2 56.4 66.6 59.6 62.7 60.2 58.7 61.5
Yeo_NTU_task1_2 Yeo2024 9 56.12 61.6 58.9 63.8 53.6 63.8 58.0 60.1 59.1 69.2 61.9 63.5 62.2 62.2 63.8
Yeo_NTU_task1_3 Yeo2024 19 54.19 61.1 58.2 63.5 50.7 63.3 58.6 59.3 59.4 68.8 61.8 62.3 61.7 62.5 63.7

System characteristics

General characteristics

Rank Submission label Technical
Report
Rank Rank score Sampling
rate
Data
augmentation
Features
Auzanneau_CEA_task1_1 Auzanneau2024 38 37.4 22.05kHz sound division mel spectrogram
BAI_JLESS_task1_1 Bai2024 32 52.0 32kHz freq-mixstyle, FMix, frequency masking log-mel energies
BAI_JLESS_task1_2 Bai2024 37 50.7 32kHz freq-mixstyle, mixup, frequency masking log-mel energies
BAI_JLESS_task1_3 Bai2024 30 52.1 32kHz freq-mixstyle, fmix, frequency masking log-mel energies
Cai_XJTLU_task1_1 Cai2024 12 55.4 32kHz mixup, freq-mixstyle, dir augmentation log-mel energies
Cai_XJTLU_task1_2 Cai2024 10 56.0 32kHz mixup, freq-mixstyle, dir augmentation log-mel energies
Cai_XJTLU_task1_3 Cai2024 11 55.9 32kHz mixup, freq-mixstyle, dir augmentation log-mel energies
Chen_GXU_task1_1 Chen2024 29 52.1 32kHz freq-mixstyle, pitch shifting, time rolling log-mel energies
Chen_SCUT_task1_1 Chen2024a 28 52.5 44.1kHz mix-style, specaugment, spectrum modulation log-mel energies
Chen_SCUT_task1_2 Chen2024a 35 51.7 44.1kHz mix-style, specaugment, spectrum modulation log-mel energies
Chen_SCUT_task1_3 Chen2024a 31 52.0 44.1kHz mix-style, specaugment, spectrum modulation log-mel energies
Gao_UniSA_task1_1 Gao2024 25 52.8 32kHz freq-mixstyle, time rolling log-mel energies
Han_SJTUTHU_task1_1 Bing2024 3 58.1 32kHz freq-mixstyle, frequency masking, time masking, time rolling log-mel energies
Han_SJTUTHU_task1_2 Bing2024 1 58.2 32kHz freq-mixstyle, frequency masking, time masking, time rolling log-mel energies
Han_SJTUTHU_task1_3 Bing2024 2 58.2 32kHz freq-mixstyle, frequency masking, time masking, time rolling log-mel energies
MALACH24_JKU_task1_1 David2024 5 57.0 32kHz freq-mixstyle, frequency masking, time rolling, DIR augmentation log-mel energies
MALACH24_JKU_task1_2 David2024 6 57.0 32kHz freq-mixstyle, frequency masking, time rolling, DIR augmentation, FilterAugment log-mel energies
MALACH24_JKU_task1_3 David2024 7 56.5 32kHz freq-mixstyle, frequency masking, time rolling, DIR augmentation log-mel energies
OO_NTUPRDCSG_task1_1 Oo2024 17 54.8 32kHz freq-mixstyle, dir augmentation, time rolling log-mel energies
OO_NTUPRDCSG_task1_2 Oo2024 15 54.8 32kHz freq-mixstyle, dir augmentation, time rolling log-mel energies
Park_KT_task1_1 Park2024 20 53.7 32kHz frequency masking, freq-mixstyle, dir augmentation, time rolling log-mel energies
Park_KT_task1_2 Park2024 13 55.4 32kHz frequency masking, freq-mixstyle, dir augmentation, time rolling log-mel energies
Park_KT_task1_3 Park2024 16 54.8 32kHz frequency masking, freq-mixstyle, dir augmentation, time rolling log-mel energies
DCASE2024 baseline 36 50.7 32kHz freq-mixstyle, pitch shifting, time rolling log-mel energies
Shao_NEPUMSE_task1_1 Shao2024 4 57.2 32kHz freq-mixstyle, dir augmentation, time rolling, playback log-mel energies
Shao_NEPUMSE_task1_2 Shao2024 8 56.1 32kHz freq-mixstyle, dir augmentation, time rolling, playback log-mel energies
Shao_NEPUMSE_task1_3 Shao2024 14 55.2 32kHz freq-mixstyle, dir augmentation, time rolling, playback log-mel energies
Surkov_ITMO_task1_1 Surkov2024 27 52.7 16kHz freq-mixstyle log-mel spectrogram
Surkov_ITMO_task1_2 Surkov2024 33 51.9 16kHz freq-mixstyle log-mel spectrogram
Tan_CISS_task1_1 Tan2024 34 51.7 44.1kHz freq-mixstyle, device impulse response, Specaug log-mel energies
Truchan_LUH_task1_1 Truchan2024 22 53.1 32kHz freq-mixstyle, time rolling, dir augmentation log-mel energies
Truchan_LUH_task1_2 Truchan2024 26 52.7 32kHz freq-mixstyle, time rolling, dir augmentation log-mel energies
Truchan_LUH_task1_3 Truchan2024 24 52.9 32kHz freq-mixstyle, time rolling, dir augmentation log-mel energies
Werning_UPBNT_task1_1 Werning2024 18 54.4 32kHz freq-mixstyle, pitch shifting, time rolling log-mel energies
Yan_NPU_task1_1 Yan2024 23 52.9 32kHz freq-mixstyle, frequency masking, time rolling log-mel energies
Yeo_NTU_task1_1 Yeo2024 21 53.1 44.1kHz freq-mixstyle, mixup, device impulse response augmentation log-mel energies
Yeo_NTU_task1_2 Yeo2024 9 56.1 32kHz freq-mixstyle, mixup, device impulse response augmentation, frequency masking log-mel energies
Yeo_NTU_task1_3 Yeo2024 19 54.2 32kHz freq-mixstyle, mixup, device impulse response augmentation, frequency masking log-mel energies



Machine learning characteristics

Rank Code Technical
Report
Rank Rank score External
data usage
External
data sources
Model
complexity
Model
MACS
Classifier Decision
making
Framework Pipeline Split adaptations System adaptations
Auzanneau_CEA_task1_1 Auzanneau2024 38 37.4 114988 13509354 CNN Yolov8 pytorch training batch size division of sound samples, vote
BAI_JLESS_task1_1 Bai2024 32 52.0 pre-trained model 126952 28893268 Dymn,CP-Mobile pytorch training
BAI_JLESS_task1_2 Bai2024 37 50.7 pre-trained model 126952 28893268 Dymn,CP-Mobile pytorch training
BAI_JLESS_task1_3 Bai2024 30 52.1 pre-trained model 126952 28893268 Dymn,CP-Mobile pytorch training
Cai_XJTLU_task1_1 Cai2024 12 55.4 dataset AudioSet, MicIRP 126858 29419648 TF-SepNet pytorch-lightning train teachers, ensemble teachers, train student using knowledge distillation, post-training static quantization lr knowledge distillation
Cai_XJTLU_task1_2 Cai2024 10 56.0 dataset AudioSet, MicIRP 126858 29419648 TF-SepNet pytorch-lightning train teachers, ensemble teachers, train student using knowledge distillation, post-training static quantization lr knowledge distillation
Cai_XJTLU_task1_3 Cai2024 11 55.9 dataset AudioSet, MicIRP 126858 29419648 TF-SepNet pytorch-lightning train teachers, ensemble teachers, train student using knowledge distillation, post-training static quantization lr knowledge distillation
Chen_GXU_task1_1 Chen2024 29 52.1 63900 29419156 RF-regularized CNN pytorch training
Chen_SCUT_task1_1 Chen2024a 28 52.5 117870 16591488 RF-regularized CNN,Transformer pytorch train teachers,ensemble teachers,train student using knowledge distillation,quantization-aware training lr, the weight coefficient and temperature coefficient of knowledge distillation architecture,model size
Chen_SCUT_task1_2 Chen2024a 35 51.7 122294 11210976 RF-regularized CNN,Transformer pytorch train teachers,ensemble teachers,train student using knowledge distillation,quantization-aware training lr, the weight coefficient and temperature coefficient of knowledge distillation architecture,model size
Chen_SCUT_task1_3 Chen2024a 31 52.0 69782 10068192 RF-regularized CNN,Transformer pytorch train teachers,ensemble teachers,train student using knowledge distillation,quantization-aware training lr, the weight coefficient and temperature coefficient of knowledge distillation architecture,model size
Gao_UniSA_task1_1 Gao2024 25 52.8 61148 29428372 MobileNet pytorch training batch size, lr
Han_SJTUTHU_task1_1 Bing2024 3 58.1 pre-trained model PaSST 63748 29982132 CNN pytorch train teachers, ensemble teachers, train student using knowledge distillation, pruning, train student using knowledge distillation
Han_SJTUTHU_task1_2 Bing2024 1 58.2 pre-trained model PaSST 63215 29221122 CNN pytorch train teachers, ensemble teachers, train student using knowledge distillation, pruning, train student using knowledge distillation, pruning, train student using knowledge distillation
Han_SJTUTHU_task1_3 Bing2024 2 58.2 pre-trained model PaSST 63875 29840890 CNN pytorch train teachers, ensemble teachers, train student using knowledge distillation, pruning, train student using knowledge distillation
MALACH24_JKU_task1_1 David2024 5 57.0 dataset MicIRP, AudioSet 61148 29419156 RF-regularized CNN pytorch, pytorch-lightning pretrain student on AudioSet, train pretrained student, train teachers, ensemble teachers, train student using knowledge distillation knowledge distillation
MALACH24_JKU_task1_2 David2024 6 57.0 pre-trained model, dataset MicIRP, AudioSet, PaSST 61148 29419156 RF-regularized CNN, Transformer pytorch, pytorch-lightning pretrain student and PaSST on AudioSet, train pretrained student, train teachers, ensemble teachers, train student using knowledge distillation lr, epochs, warmup steps knowledge distillation, PaSST, data augmentations
MALACH24_JKU_task1_3 David2024 7 56.5 dataset MicIRP, AudioSet, PaSST 61148 29419156 RF-regularized CNN pytorch, pytorch-lightning pretrain student and on AudioSet, train pretrained student, train teachers, ensemble teachers, train student using knowledge distillation knowledge distillation
OO_NTUPRDCSG_task1_1 Oo2024 17 54.8 116842 29407896 CNN, Ensemble pytorch training, quantization-aware training, enssemble
OO_NTUPRDCSG_task1_2 Oo2024 15 54.8 116842 29407896 CNN, Ensemble pytorch training, quantization-aware training, enssemble
Park_KT_task1_1 Park2024 20 53.7 dataset MicIRP 87672 28568272 GhostRes2Net pytorch training weight quantization
Park_KT_task1_2 Park2024 13 55.4 dataset MicIRP 63992 26480312 GhostRes2Net pytorch training model_size
Park_KT_task1_3 Park2024 16 54.8 dataset, pre-trained model EfficientAT, PaSST, MicIRP 63992 26480312 GhostRes2Net pytorch alternately train teacher and student model at each training step (online knowledge distillation) knowledge distillation
DCASE2024 baseline 36 50.7 61148 29419156 RF-regularized CNN pytorch training
Shao_NEPUMSE_task1_1 Shao2024 4 57.2 107457 16911324 Mamba, Inverted residual block pytorch train student using knowledge distillation, quantization-aware training
Shao_NEPUMSE_task1_2 Shao2024 8 56.1 121925 17272736 Mamba, Inverted residual block pytorch train student using knowledge distillation, quantization-aware training
Shao_NEPUMSE_task1_3 Shao2024 14 55.2 126410 16785856 Mamba, Inverted residual block pytorch train student using knowledge distillation, quantization-aware training
Surkov_ITMO_task1_1 Surkov2024 27 52.7 dataset, pre-trained model EfficientAT, AudioSet 61148 21896340 CNN, CRNN pytorch, pytorch-lightning train teacher using mean-teacher SSL, train student using knowledge distillation external unlabeled data size
Surkov_ITMO_task1_2 Surkov2024 33 51.9 dataset, pre-trained model EfficientAT, AudioSet 61148 21896340 CNN, CRNN pytorch, pytorch-lightning train teacher using mean-teacher SSL, train student using knowledge distillation external unlabeled data size
Tan_CISS_task1_1 Tan2024 34 51.7 61148 29419156 CNN pytorch training
Truchan_LUH_task1_1 Truchan2024 22 53.1 MicIRP 47946 29747914 CNN pytorch training architecture
Truchan_LUH_task1_2 Truchan2024 26 52.7 MicIRP 47946 29747914 CNN pytorch training architecture
Truchan_LUH_task1_3 Truchan2024 24 52.9 MicIRP 47946 29747914 CNN pytorch training, discard dominant features representation learning
Werning_UPBNT_task1_1 Werning2024 18 54.4 dataset, pre-trained model, embeddings AudioSet 61148 29419156 RF-regularized CNN pytorch dataset pruning, knowledge distillation, fine-tuning
Yan_NPU_task1_1 Yan2024 23 52.9 31290 29675722 CNN pytorch training
Yeo_NTU_task1_1 Yeo2024 21 53.1 MicIRP MicIRP 35062 22649568 RF-regularized CNN pytorch training
Yeo_NTU_task1_2 Yeo2024 9 56.1 MicIRP, pre-trained model MicIRP, AudioSet 61148 29419156 RF-regularized CNN, PaSST pytorch train teachers, ensemble teachers, train student using knowledge distillation
Yeo_NTU_task1_3 Yeo2024 19 54.2 MicIRP, pre-trained model MicIRP, AudioSet 61148 29419156 RF-regularized CNN, PaSST pytorch train teachers, ensemble teachers, train student using knowledge distillation, train student using FocusNet knowledge distillation, FocusNet

Technical reports

Low-Complexity Classification of Acoustic Scenes Based on Reduced Sound Duration and Voting

Fabrice Auzanneau
CEA List, Palaiseau, France

Abstract

This report describes our approach to the DCASE (Detection and Classification of Acoustic Scenes and Events) Challenge for Task 1 ”Low-Complexity Acoustic Scene Classification” [1]. The task 1 of the DCASE challenge aims to classify acoustic scenes using devices with low computational power and memory. This task involves a combination of precision and complexity, which encourages participants to build efficient systems for acoustic scene classification (ASC). This year, an additional challenging real-world situation has been added: the limited availability of labeled data. The systems must take into account five scenarios that progressively limit the amount of training data. The largest set corresponds to the entire training data set, while the smallest contains only 5% of the audio clips in the training data set. Our approach is a combination of the use of a deep neural network and statistical processing. A network based on the YoloV8 topology is pruned to ensure that it meets the memory constraints of the challenge. The network is trained on half-length data (500ms), then quantized to reduce its size. During inference, each one-second sound is divided into two 500ms parts. Each half is used to make an inference, and both results are combined to improve the classification result. In the event of disagreement, a voting strategy is applied to decide on the correct category. Classification performance is thus improved by 2% for the smallest subset and by over 5% for the largest one.

System characteristics
Sampling rate 22.05kHz
Data augmentation sound division
Features mel spectrogram
Classifier CNN Yolov8
PDF

Hierarchical Acoustic Scene Classification with Knowledge Distillation and Pre-Trained Dynamic Networks

Jisheng Bai1, Mou Wang2, Ee-Leng Tan3, Jin Jie Sean Yeo3, Jun Wei Yeow3, Santi Peksi3, Dongyuan Shi3, Woon-Seng Gan3 and Jianfeng Chen1
1School of Marine Science and Technology, Northwestern Polytechnical University, Xi'an, China, 2Chinese Academy of Sciences, Institute of Acoustics, Beijing, China, 3Nanyang Technological University, Smart Nation TRANS Lab, Singapore

Abstract

Previous acoustic scene classification (ASC) tasks in the DCASE challenge focused on two important aspects: recording device mismatch and low-complexity systems. However, implementing ASC systems in real-life applications remains challenging due to the time-consuming process of collecting large amounts of labeled data for system development. DCASE2024 Task 1 is proposed to explore possible solutions that can efficiently utilize varying ratios of available training data while maintaining ASC performance. In this paper, we propose a hierarchical learning-based method to develop ASC systems using knowledge distillation and pre-trained dynamic networks. Specifically, we fine-tune the dynamic networks, which are pre-trained on Audioset, with an additional classification task and various data augmentation methods. We then employ an ensemble of fine-tuned dynamic networks to teach CP-Mobile networks. Finally, we fine-tune the CP-Mobile networks using quantization-aware training to achieve low-complexity models. The experimental results demonstrate that the proposed systems outperform the baseline system.

System characteristics
Sampling rate 32kHz
Data augmentation freq-mixstyle, FMix, frequency masking; freq-mixstyle, mixup, frequency masking; freq-mixstyle, fmix, frequency masking
Features log-mel energies
Classifier Dymn,CP-Mobile
PDF

Data-Efficient Acoustic Scene Classification via Ensemble Teachers Distillation and Pruning

Han Bing1, Huang Wen1, Chen Zhengyang1, Jiang Anbai2, Chen Xie1, Fan Pingyi2, Lu Cheng3, Lv Zhiqiang4, Liu Jia2, Zhang Wei-Qiang2 and Qian Yanmin1
1Shanghai Jiao Tong University, Shanghai, China, 2Tsinghua University, Beijing, China, 3North China Electric Power University, Beijing, China, 4Huakong AI, Beijing, China

Abstract

The goal of the acoustic scene classification task is to classify recordings into one of the ten predefined acous- tic scene classes. In this report, we describe the SJTU- THU team’s submission for Task 1 Data-Efficient Low- Complexity Acoustic Scene Classification of the DCASE 2024 challenge. Firstly, we design a new architecture named SSCP-Mobile (spatially separable) by enhancing the CP- Mobile with spatially separable convolution structure and achieve lower computation expenses and better performance. Then we adopt several pretrained PaSST models as ensemble teachers to teach CP-Mobile with knowledge distillation. Af- ter that, We use model pruning techniques to trim the model to meet the computational and parameter requirements of the competition. Finally, we will use knowledge distillation tech- niques again to fine tune the pruned model and further im- prove its performance.

System characteristics
Sampling rate 32kHz
Data augmentation freq-mixstyle, frequency masking, time masking, time rolling
Features log-mel energies
Classifier CNN
PDF

DCASE2024 Task1 Submission: Data-Efficient Acoustic Scene Classification with Self-Supervised Teachers

Yiqiang Cai1, Minyu Lin1, Shengchen Li1 and Xi Shao2
1School of Advanced Technology, Xi'an Jiaotong-Liverpool University, Suzhou, China, 2College of Tellecommunications and Information Engineering, Nanjing University of Posts and Telecommunications, Nanjing, China

Abstract

The task 1 of the DCASE Challenge 2024 focuses on developing low-complexity acoustic scene classification (ASC) systems with limited labeled training data. This technical report details the systems we submitted. We firstly use self-supervised learning (SSL) techniques to pre-train large teacher models on AudioSet. The self-supervised teachers are then fine-tuned on ASC dataset using different weight-freezing strategies. Knowledge distillation is employed to transfer the self-supervised knowledge to a low-complexity ASC model. The student model, TF-SepNet-64, is designed to meet the upper complexity limit of the challenge requirements. To mitigate the device shift problem, we used Freq-MixStyle and device impulse response augmentation. In experiments, our best system, trained on 5 given subsets, achieves an average accuracy of 56.6%.

System characteristics
Sampling rate 32kHz
Data augmentation mixup, freq-mixstyle, dir augmentation
Features log-mel energies
Classifier TF-SepNet
PDF

Data-Efficient Low-Complexity Acoustic Scene Classification with Curriculum Learning and Se-Layer

Xuanyan Chen and Wei Xie
School of Computer and Electronic Information, Guangxi University (GXU) Guangxi, Guangxi, China

Abstract

This technical report presents a data-efficient and low-complexity acoustic scene classification (ASC) system developed for Task 1 of the DCASE2024 Challenge. The primary objective is to create ASC models that perform effectively with limited labeled data and minimal computational resources, addressing practical constraints in real-world applications. Our proposed system integrates Squeeze-and-Excitation (SE) layers within the baseline network and employs a curriculum learning approach for training. SE layers enhance feature representation by recalibrating channelwise feature responses, while curriculum learning structures the training process by progressively introducing more complex examples, facilitating better model generalization and robustness. Experimental results demonstrate significant improvements in classification accuracy across various data splits, with our system outperforming the baseline by up to 7% on the development dataset. The approach promises to advance the accessibility and scalability of ASC technologies in resource-constrained environments.

System characteristics
Sampling rate 32kHz
Data augmentation freq-mixstyle, pitch shifting, time rolling
Features log-mel energies
Classifier RF-regularized CNN
PDF

Data-Efficient Low-Complexity Acoustic Scene Classification Using Parallel Attention Broad-Cast-Residual Network

Guoqing Chen and Yanxiong Li
School of Electronic and Information Engineering, South China University of Technology, Guangzhou, China

Abstract

This technical report describes our proposed system for Task 1 of Detection and Classification of Acoustic Scenes and Events (DCASE) 2024. We propose a data-efficient low-complexity acoustic scene classification method, which utilizes a parallel attention broad-residual network that consists of four parts (i.e., the modules of pre-processing, fusion, global and local contextual information extraction). We integrate the broadcast residual learning into the network to enhance its ability for extracting local contextual information. To further improve accuracy and reduce complexity, we integrate other techniques into our method, such as knowledge distillation, data augmentation, adaptive residual normalization, and quantization-aware training. There are five training subsets that contain approximately 5%, 10%, 25%, 50%, and 100% of the audio snippets in the training dataset. Using a subset of the five training subsets above as training data to construct a system, we obtain five systems. The accuracy scores obtained by these five systems on the evaluation samples of the development dataset are 47.14%, 52.38%, 58.04%, 60.88%, and 63.7% respectively.

System characteristics
Sampling rate 44.1kHz
Data augmentation mix-style, specaugment, spectrum modulation
Features log-mel energies
Classifier RF-regularized CNN,Transformer
PDF

Data-Efficient Acoustic Scene Classification with Pre-Trained Cp-Mobile

Nadrchal David, Rostamza Aida and Schilcher Patrick
Johannes Kepler University (JKU) Linz, Linz, Austria

Abstract

This report presents our submission for Task 1: Data-Efficient Low-Complexity Acoustic Scene Classification in the DCASE2024 challenge. Drawing inspiration from the top-ranked system in the 2023 edition, our approach is based on a Knowledge Distillation training routine: we employ ensembles of fine-tuned CP-ResNet and PaSST as teachers for each subset, with a modified version of the CP-Mobile baseline model serving as the student. A key improvement in our methodology is pre-training the student on both AudioSet and the corresponding training subset before knowledge distillation, which significantly enhances its performance. To improve device generalization, we use various data augmentation techniques, including Freq-MixStyle, Device impulse response augmentation, FilterAugment, frequency masking, and time rolling. Our results demonstrate substantial improvements in test accuracy compared to the baseline system, validating the effectiveness of our approach for each subset.

System characteristics
Sampling rate 32kHz
Data augmentation freq-mixstyle, frequency masking, time rolling, DIR augmentation; freq-mixstyle, frequency masking, time rolling, DIR augmentation, FilterAugment
Features log-mel energies
Classifier RF-regularized CNN; RF-regularized CNN, Transformer
PDF

Data-Efficient Low-Complexity Acoustic Scene Classification Using Mobilenet Variant

Wei Gao and Ivan Lee
UniSA STEM, University of South Australia, Adelaide, Australia

Abstract

This report describes our submission on the task of low-complexity acoustic scene classification of the DCASE 2024 challenge. To meet the system complexity limitations of the task, we trained a single MobileNet variant fitting for all five pre-defined data folds. The training was optimised towards a focal loss function that helped on hard misclassified samples. The models were deployed with FP16 precision for the sake of efficient inference.

System characteristics
Sampling rate 32kHz
Data augmentation freq-mixstyle, time rolling
Features log-mel energies
Classifier MobileNet
PDF

Low Complexity Acoustic Scene Classification with Moflenet

Yifei Oo1 and Nagisetty Srikanth2
1Nanyang Technological University, Singapore, 2Institute of Computational Perception (CP), Johannes Kepler University (JKU) Linz, Linz, Austria

Abstract

This technical report details our approach to Task 1: Low-Complexity Acoustic Scene Classification for the DCASE 2024 challenge. We introduced a novel architecture, MofleNet, featuring shuffle channels and residual inverted bottleneck blocks. For the challenge submission, we ensemble this new model with CPResNet. To enhance cross-device generalization performance, Freq-MixStyle and Device Impulse Response (DIR) augmentation are applied during training. To meet the constraint of keeping the model size under 128kB, both models are fine-tuned using Quantization Aware Training to perform computations in 8-bit precision. This ensemble method achieved an average accuracy improvement of 6% on the TAU Urban Acoustic Scenes 2022 Mobile development dataset compared to the baseline model of DCASE 2024 Task 1.

System characteristics
Sampling rate 32kHz
Data augmentation freq-mixstyle, dir augmentation, time rolling
Features log-mel energies
Classifier CNN, Ensemble
PDF

Kt Submission: Periodic Activation and Knowledge Distillation for Data-Efficient Low-Complexity Acoustic Scene Classification

JaeHan Park1, Taesoo Kim2, Daniel Rho3, Jiwon Kim3 and Gahui Lee3
1AI Tech Lab, Korea Telecom Corporation, Seoul, South Korea, 2AI Tech Lab, Korea Telecom Coporation, Seoul, South Korea, 3AI Tech Lab, Korea Telecom Corporation, Seoul Korea

Abstract

This technical report describes our approach to participating in DCASE 2024 Challenge Task 1: Data-Efficient Low-Complexity Acoustic Scene Classification. Our main contribution to this work is the use of an improved backbone model, a modified BC-Res2Net with the GhostNet module. In addition, in order to improve generalization performance even with a limited amount of training data, we adopted the snake activation function, which is known to be robust to unseen data due to its extrapolation capabilities. Through experiments, we demonstrated that our model significantly improves acoustic scene classification performance, especially when the number of training samples is limited.

System characteristics
Sampling rate 32kHz
Data augmentation frequency masking, freq-mixstyle, dir augmentation, time rolling
Features log-mel energies
Classifier GhostRes2Net
PDF

Low-Complexity Acoustic Scene Classification with Limited Training Data

Yun-Fei Shao, Peng Jiang and Wei Li
The School of Mechanical Science and Engineering, Northeast Petroleum University, Daqing, China

Abstract

This report details the architecture we used to address task 1 of the DCASE2024 challenge. In addition to dealing with device mismatches and low complexity limitations, this year's tasks have also added training data limitations. The architecture we propose is based on Mamba, which is a selection state space model with the ability to establish long-term dependencies. Specifically, we designed a variable parallel mamba architecture to further reduce parameters and combined it with the inverted residual block in MobilenetV2 to address training data constraints by adjusting the number of mamba modules and the number of parallels. In addition, we also enhanced the impulse response of audio with energy values greater than the average. Freq-MixStyle (FMS) and audio playback data augmentation methods were used. We apply two model compression schemes: Quantization-Aware Training (QAT), and Knowledge distillation. The proposed system achieves higher classification accuracies than the baseline system. After model compression, our model achieves an average accuracy of 50.1% within the 107.46 K parameters size, 8-bit quantization, and MMACs 16.9 M.

System characteristics
Sampling rate 32kHz
Data augmentation freq-mixstyle, dir augmentation, time rolling, playback
Features log-mel energies
Classifier Mamba, Inverted residual block
PDF

Efficient Acoustic Scene Classification Using Mean-Teacher and Knowledge Distillation

Maxim Surkov
ITMO University, Saint Petersburg, Russia

Abstract

This technical report describes submission for the DCASE 2024 Task 1 ”Data-Efficient Low-Complexity Acoustic Scene Classifica- tion”. Pretrained Efficient-AT was fine-tuned using a mean-teacher self supervised learning algorithm on labeled data, presented by the authors of the task, and on external unlabeled data taken from AudioSet. Current state-of-the-art efficient neural network CP-Mobile was then trained on the same data using knowledge distillation from fine-tuned Efficient-AT model. Proposed model consists of 61K pa- rameters and requires 22M MACs. Using sufficient amount of external data in pair with knowledge distillation improve the results by around 4% in accuracy in compare with the baseline approach in cases with the small amount of labeled data.

System characteristics
Sampling rate 16kHz
Data augmentation freq-mixstyle
Features log-mel spectrogram
Classifier CNN, CRNN
PDF

Acoustic Scene Classification Using Convolution Neural Network with Limited Data

Ee-Leng Tan1, Bai Jisheng2, Jun Wei Yeo3, Santi Peksi3 and Woon-Seng Gan3
1EEE, Nanyang Technological University, Singapore, 2School of Marine Science and Technology, Northwestern Polytechnical University, Xi’an, China, 3EEE, Nanyang Technological Univeristy, Singapore, Singapore

Abstract

This technical report describes the SNTL-NTU team’s submission for Task 1 Low-Complexity Acoustic Scene Classification of the Detection and Classification of Acoustic Scenes and Events (DCASE) 2024 challenge [1]. The proposed CNN model is trained using the TAU Urban Acoustic Scene 2022 Mobile development dataset [2]. For the model's input, each audio sample is transformed into log-mel energies. The model has a memory usage of 88.2 KB and requires 25.9M multiply-and-accumulate (MAC) operations. Using the development dataset, the proposed model achieved an accuracy of 46.02%, 49.54%, 53.89%, 56.43%, and 59.08% for 5%, 10%, 25%, 50%, and 100% of the development dataset, respectively.

System characteristics
Sampling rate 44.1kHz
Data augmentation freq-mixstyle, device impulse response, Specaug
Features log-mel energies
Classifier CNN
PDF

Ascdomain: Domain Invariant Device-Adversarial Isotropic Convolutional Neural Architecture

Hubert Truchan1, Tien Hung Ngo1 and Zahra Ahmadi2
1L3S Research Center, Leibniz University Hannover, Hanover, Germany, 2Peter L. Reichertz Medical Informatics Institute, Hannover Medical School, Hanover, Germany

Abstract

The ongoing advancement of deep learning in the classification of acoustic scenes (ASC) showcases the expanded versatility of handling complex auditory environments. However, the imperative for efficient real-time decision-making in practical applications requires models that are adaptive and computationally frugal. Addressing these constraints, we introduce four innovative architectures: a scalable isotropic convolutional network, a recursive columnar architecture, an adversarial version to unlearn domain-specific features, and a representation self-challenging method. Each architecture is designed for low-complexity and data-efficient ASC, incorporating advanced techniques such as device impulse response enhancement and two-dimensional signal embedding to enhance robustness against device mismatch. Our approaches are validated in the DCASE 2024 Task 1 challenge using the TAU Urban Acoustic Scenes 2022 Mobile dataset, achieving state-of-the-art performance and significantly improving domain generalization.

System characteristics
Sampling rate 32kHz
Data augmentation freq-mixstyle, time rolling, dir augmentation
Features log-mel energies
Classifier CNN
PDF

Upb-Nt Submission to Dcase24: Dataset Pruning for Targeted Knowledge Distillation

Alexander Werning and Reinhold Haeb-Umbach
Department of Communications Engineering (NT), Paderborn University (UPB), Paderborn, Germany

Abstract

In this technical report, we describe our submission for Task 1 Data-Efficient Low-Complexity Acoustic Scene Classification. We adopt the baseline model and add a specialised knowledge distillation process before proceeding with the baseline training process. Our model was distilled on a pruned subset of the AudioSet dataset using large pretrained models. The pruning of the dataset is based on the similarity of the data to the targeted challenge dataset.

System characteristics
Sampling rate 32kHz
Data augmentation freq-mixstyle, pitch shifting, time rolling
Features log-mel energies
Classifier RF-regularized CNN
PDF

Submission for DCASE 2024 Task1: An Asymmetric Residual Deep Neural Network for Low-Complexityacoustic Scene Classification

Chenhong Yan, Yang Yu and Xiaohang Xiong
School of Marine Science and Technology, Northwestern Polytechnical Universi University, Xi’an, China

Abstract

This technical report describes the Sunshine Team24’s submission for DCASE 2024 ASC Task 1, data-efficient low-complexity acoustic scene classification(ASC). Compared to Task 1 in the DCASE 2023 Challenge, the following aspects change in the 2024 edition: (a) Training sets of different sizes are provided. These train subsets contain approximately 5%, 10%, 25%, 50%, and 100% of the audio snippets in the train split provided in Task 1 of the DCASE 2023 challenge. A system must only be trained on the specified subset and the explicitly allowed external resources. (b)The model complexity is not part of the ranking system. The model's complexity is limited in terms of hard constraints. To this end, the memory requirement for model parameters is restricted to 128 kB, and the maximum number of multiply-accumulate operations (MACs) is restricted to 30 million. In this report, we present low-complexity systems for ASC that follow the rules intended for the task.

System characteristics
Sampling rate 32kHz
Data augmentation freq-mixstyle, frequency masking, time rolling
Features log-mel energies
Classifier CNN
PDF

Data Efficient Acoustic Scene Classification using Teacher-Informed Confusing Class Instruction

Sean Yeo1, Ee-Leng Tan1, Jisheng Bai2, Santi Peksi1 and Woon-Seng Gan1
1Smart Nation TRANS Lab (SNTL), Nanyang Technological University, Singapore, 2Marine Science and Technology, Northwestern Polytechnical University, Xi'an, China

Abstract

In this technical report, we describe the CISS-NTU team’s submission for Task 1 Data-Efficient Low-Complexity Acoustic Scene Classification of the detection and classification of acoustic scenes and events (DCASE) 2024 challenge. Three systems are introduced to tackle training subsets of different sizes. For small training subsets, we explored reducing the complexity of the provided baseline model by reducing the number of base channels. We introduce data augmentation in the form of mixup to increase the diversity of training samples. For the larger training subsets, we use FocusNet to provide confusing class information to an ensemble of multiple Patchout faSt Spectrogram Transformer (PaSST) models and baseline models trained on the original sampling rate of 44.1 kHz. We use Knowledge Distillation to distill the ensemble model to the baseline student model. Training the systems on the TAU Urban Acoustic Scene 2022 Mobile development dataset yielded the highest average testing accuracy of (62.21, 59.82, 56.81, 53.03, 47.97)% on split (100, 50, 25, 10, 5)% respectively over the three systems

System characteristics
Sampling rate 44.1kHz; 32kHz
Data augmentation freq-mixstyle, mixup, device impulse response augmentation; freq-mixstyle, mixup, device impulse response augmentation, frequency masking
Features log-mel energies
Classifier RF-regularized CNN; RF-regularized CNN, PaSST
PDF