In this course, students will be reading and reviewing papers for each class. During the class, some students will be presenting the papers for the week, they and the rest of the class (including me) will be discussing these papers and our take on them. There is also a term-long project, which is worth the biggest chunk of your grade. Following is the grade breakdown:
- Project 45%
- Presentations 20%
- Paper Reviews 20%
- Class Participation 15%
Project
The research project could be any of the following:
- New research idea: A prototype implementation of a new research idea that addresses one of the drawbacks or limitations of an existing research work, or a completely new research idea that is inspired by any of your readings.
- Experimental Study: An experimental comparison and evaluation of existing work in a specific research topic. Students are not supposed to reimplement all of the existing solution. Rather, they should be able to reuse an existing code base with minor changes to run the benchmark. The main contribution in the benchmark is to give insights that did not exist in the systems used in the evaluation.
- Survey: With the extensive research efforts in the topics covered in this course, a survey paper should summarize and categorize the major research contributions in a specific area. The survey should not be a mere summarization of existing papers, rather, the students should provide their own insight on the surveyed body of work. For example, they can provide a categorization or a taxonomy that highlights that major research directions in that area. Students can also identify the open research problems that were hardly addressed in the literature.
- System Implementation and Reproducibility: I have a number of systems I would like implemented. Your project could be choosing any of them, and implement and reproduce the results reported.
The project can be done individually or in groups. However, the assessment will take into consideration how many students are in the group. E.g., if one student demonstrates contributions in her/his project that is equal to the contributions for a team of three students, students should expect a high variance in grades.
The project deliverables will be:
- Project Proposal: This should be a maximum of two-pages proposal (including references) in ACM Proceedings Format (Latex is mandatory). To write a good proposal, I strongly suggest reading Jennifer Widom's tips for writing introductions. I also strongly suggest reading the whole document as it is helpful for writing research papers in general. This proposal is due on February 25. If you have a solid idea that you would like to submit before the deadline to get better feedback and give yourself more time to work on the project, early submissions of the proposal are STRONGLY encouraged.
- Project Paper: Again, in ACM Proceedings Format. This should be at least 7 pages including references. Depending on the size of the group and contributions, the paper could be longer. So, there is no page-limit. Due date for the project paper is April 12 (11:59 PM). Late submissions are allowed for two more weeks with a hard deadline for submission on April 26th.
- Source Code: Your source code is expected to be publically available on github. The github link for your project should be in the project paper. Please write a good README that clearly describe how to run your code. Due date for the project source code is the same as for the project paper.
Presentations
There will be 22 presentations throughout the term. This workload may not be evenly distributed over the students doing this class. Therefore, the student who presents one more presentation than average will get a bonus. Each presentation should be 30 to 45 minutes long, followed by a 30 to 45 minutes of discussion of the paper. The presenter should not only present the details of the paper, but also suggest the discussion points at the end of his/her presentation.
Paper Reviews
The paper reviews are due at 11:00 AM on the day of the class. The format for the review is fixed: Summary of the paper, three or more strong points, three or more weak points, and any additional comments you may have on this paper. The number of fields required is small, but you are expected to be elaborative. Theoretically, if your review is written in a Word document, it should be at least one page long in 12 pt. Your two worst reviews will not count towards your grade.
Here are a few comments to consider when you write your reviews:
- Don't copy/paste sentences from the paper. Write down your own understanding of the paper.
- When listing the strong and weak points, please enumerate them and don't write a single big paragraph with all the points. The writing of the paper cannot be one of the main strong or weak points.
- Don't just say that one of the strong points is that this paper used a deep learning approach. This is not a strong point. A strong point would be to say why you think using deep learning in this case is a good approach. Not why deep learning is good in general, but why in this case.
- In general, when you choose one point as a strong or weak point. Elaborate on why you think it's strong or weak. I can't read minds to know what you had in mind when you chose "using templates to answer questions" (btw, that's a superficial answers. Not shaming anyone).
- Please get over the writing of the paper when it comes to enumerating strong or weak points. Focus on the real beef of the paper rather than presentation. I'd accept presentation as a fourth strong or weak point, not one of the main three.
- Don't count something that is natural for the authors to do as a strong point. For example, using a benchmark with complex question in the evaluation. Well, this isn't a strong point. If they didn't do that, it would have been a real shitty paper. Another example would be beating the state-of-the-art systems. If they didn't, they wouldn't publish a paper.
- Please don't use the future work as an answer for the sequel for the paper. That's not the point. The point is that maybe this paper gave you an idea for a new project. What would this idea be? I changed the wording of the question in the review to reflect that.
Some References:
- Reading a Computer Science Paper.
- Example of a bad review.
- How to Read a Paper.
- IMPORTANT: Here is a sample review for a rejected paper (real review). Please have a look at it. It will show you how reviews look like in real life. You'll see that strong points are not elaborated on. That's OK for this kind of review because they're reviewing the paper for acceptance or rejection. If it's accepted, they won't elaborate more on what's obvious. On the other hand, you'll find they tend to elaborate more on weak points. Sometimes the three (or more) weak points are brief, but that's because they take their time and space in the detailed comments section. Since we don't have that, you're expected to elaborate where you mention the three strong or weak points. Please read this review carefully to understand the level of feedback expected in a review. This is what's expected from you in your reviews throughout this course.
Paper Review Submission Link
Class Participation
This is a seminar-based class, meaning that your participation in the class is essential. You are encouraged to ask questions, answer other students questions, give comments over the papers we discuss, etc.
