Technical Guidelines

Scheduling

As soon as you are notified of your assigned paper, please check all of them to make sure that:

• there is no obvious conflict of interest (see section on Conflict of Interest below)
• you are qualified to review the papers assigned

Please notify the Program Committee immediately if any issues arise regarding these points.

Evaluation Criteria

You will be asked to provide an evaluation of the papers assigned to you according to the following criteria:

• Relevance of the topic to DCASE
• Originality/importance of the paper
• Potential of the paper to impact the DCASE community
• Experimental validation
• Clarity of presentation

Keep in mind that minor flaws can be corrected, and should not be a reason to reject a paper. Similarly, failing to obtain state of the art results should not be perceived as lowering the novelty of a paper in terms of originality. However, accepted papers must be technically sound and make an original and substantial contribution to the field.

When deciding your recommendation for a paper, do not be shy. Use the whole spectrum of evaluation scores: if you think a paper is outstanding, give it the highest score. Similarly, if you think a paper is really bad (and can convincingly support your opinion), then give it the lowest score.

Ensure that your scores are consistent with your comments to the authors. In particular, receiving good comments and a poor score is frustrating, and often causes the authors to request clarifications or rebuttals.

Comments for the Authors

Your comments for the authors are probably the most important part of your reviews. They will be returned to the authors, so you should include any specific feedback which can help improve the papers. Thorough comments also help the Program Committee decide which papers to accept, sometimes more than your score.

Short reviews are not helpful to both the authors and the Program Committee. Please be as specific and detailed as you can. When discussing related work and references, simply saying “this is well known” or “this has been common practice for years” is not sufficient. You should cite publications, or other public disclosures of techniques, which can support your statements. Be specific also when you suggest improvements in the writing. If there is a particular passage in the text that is unclear, point it out and give suggestions for improvements. On the other hand, if the whole paper is poorly written, you are not expected to rewrite it for the authors.

Be generous about providing new ideas for improvement. You may suggest different techniques or tools to be used in the applications presented in a paper. You may also suggest the authors a new application area that might benefit from their work. You may suggest them a generalization of their concept, which they have not considered.

If you think that the paper has merits but does not exactly match the topics of the DCASE workshop, please do not simply reject the paper but communicate this to the Program Committee. These papers will then be looked at and discussed with specific care. Suggestions for alternative publication options that are more appropriate in your opinion (journals, conferences, workshops) are welcome.

Ethical Guidelines

Commitment and Respect

Remember that academic careers and reputations rely on scientific publications. Therefore you have to be seriously committed to your work as a reviewer. A sketchy or casual review is a lack of respect to the authors who have seriously submitted their paper, and in the long run damages the reputation of the workshop. If you have agreed to review a paper, you should devote enough time to write a thoughtful and detailed review. If you think you cannot review properly your assigned papers because you are too busy, please communicate this to the Program Committee as soon as possible so that the papers can be reassigned in time. Acting that way you are helping much more than doing a superficial review.

Keep in mind that belittling or sarcastic comments are not appropriate. Even if you think that a paper is really bad, you should be constructive and still provide feedback to the authors. If you give a paper a low score, it is essential that you justify the reason for that score in detail. Just saying “I do not like this approach because I am a guru in this area” is not constructive.

Confidentiality and Anonymity

As a reviewer you have the responsibility to protect the confidentiality of the ideas represented in the papers you review. In order to comply with confidentiality requirements:

• you should not show your assigned papers (or their accompanying material) to anyone else, including colleagues or students
• you should not use ideas from your assigned papers to develop new ones until the paper has been made public
• after completing your reviews, you should keep all copies of your assigned papers and accompanying material strictly confidential; also you should not use implementations you may have written and results you may have obtained to evaluate the ideas in the papers until they have been published and are properly citable.

Although some reviewers like to disclose their identity to authors, it is advisable not to do so. One of the most common ways of inadvertently disclosing your identity is asking the authors to cite your past work. This should be avoided. Besides, this attitude may have a negative effect on your review: it may be seen as if you just want to gain more citations, and may ultimately result in the authors just ignoring your review (and possibly the Program Committee too).

Conflicts of Interest

Even though you would judge impartially any paper assigned to you, there has to be no doubt about the impartiality of your reviews. Therefore, if there is a potential conflict of interest with one of your assigned papers, you should inform the Program Committee. Although in general you should use your judgment, examples of situations of potential conflicts of interest are the following:

• you work in the same research group as one of the authors;
• you have been involved in the work and will be credited in some way (e.g. you have hosted one of the authors in your lab, to carry out work related to the paper);
• you have formally collaborated (e.g., written a paper together, or been awarded a joint grant) with one of the authors in the past three years (more or less);
• you were the M.Sc./Ph.D. advisor (or advisee) of one of the authors: this is often considered to be a lifetime conflict of interest;
• you have reasons to believe that others might see a conflict of interest, even though there is none (e.g., you and one of the authors work for the same multinational corporation, although you work in different departments on different continents and have never met before).

In case you have any doubt about a potential conflict of interest, then rather decide that there is such a conflict and contact the Program Committee.

Acknowledgements

This document closely follows the reviewer guidelines of the ISMIR conferences.