Statement of interest
You can drop us an email at dcasechallenge@gmail.com if you are interested of participating to any of the tasks. However, it is not necessary, and all challenge submissions will be accepted.
Technical report
Because the challenge is premised upon the sharing of ideas and results, all participants are required to submit a technical report about the submitted system to help us and the community better understand how the algorithm works. The technical report must describe in sufficient detail the system. Maximum length for the document is 5 pages. The technical report is submitted at the same time as the results.
Detailed information on required content can be found on the submission page.
Cross-validation setup
For obtaining fully comparable results between participants, a file list containing train/test folds is provided for all tasks. Use the provided setup for reporting development set results. If you are using this dataset for research, please also publish results using the given setup, this way it is possible to compare fairly the performance using different methods and approaches.
Challenge rules
- Manipulation of provided training and development data is allowed. Task specific modifications of this rule are indicated in the individual task pages.
- Participants are not allowed to make subjective judgments of the evaluation data, nor to annotate it.
- Evaluation dataset cannot be used to train the submitted system, except for Task 3. The use of statistics about the evaluation data in the decision making is also forbidden.
- The system outputs that do not respect the challenge rules will be evaluated on request, but they will not be officially included in the challenge rankings.
- Persons from the institutes of DCASE organizing teams are eligible to participate the challenge, if they can have an arrangement which prevents them from having any unfair advantage in the challenge. This arrangement should be briefly documented in the technical report describing the submission.
- Other task specific rules may apply. These are indicated and highlighted in the individual task pages.
Use of external data
Use of external data is not allowed for system development, except for Task 1C, Task 2, Task 3, and Task 5. The use of external data in the mentioned tasks must obey the following conditions:
- The used external datasets must be publicly available without cost.
- Participants should inform and suggest task organizers external datasets they are going to use. The datasets will be listed on the task page, so all competitors know about them and have equal opportunity to use them.
- Once the evaluation set is published, the list of external datasets allowed is locked (no further external sources allowed).
Data from another task is considered external data.
Submission
Participants are requested to submit results in the required submission package. Participants are allowed to train their system using any subset or complete set of the available development dataset.
Multiple system outputs can be submitted (maximum 4 per participant). If submitting multiple systems, the individual system output files should be packaged into same submission package. Please follow instruction carefully when forming the submission package.
Evaluation
The evaluation of the submitted results will be done by the organizers. The metric used for ranking of submissions is different for each task; it is described on each of the task pages.
The evaluation results will be announced online, and discussed at DCASE 2018 Workshop.