Advanced Deep Learning (Adv DL) – CISC889011 – Spring 2023 – Purnell Hall 324B
Course Description: This course will cover advanced deep-learning topics. Out of many possible topics, the course will cover generative adversarial networks, latent variable models (VAEs), graph neural networks, deep reinforcement learning, and foundation models (transformers). The class will include lecture sessions (covering the methods and examples), followed by paper reading sessions (covering papers picked by the students).
Topics
| Date | Topic |
|---|---|
| 2/7 | Logistics |
| 2/9 | Latent Variable Models – Theory |
| 2/14 | Latent Variable Models – Basic VAEs |
| 2/16 | Latent Variable Models - Paper Review 1 |
| 2/21 | Latent Variable Models – VAE Types |
| 2/23 | Latent Variable Models - Paper Review 2 |
| 2/28 | Generative Models - Theory |
| 3/2 | No Class |
| 3/7 | Generative Models - Variations |
| 3/9 | Generative Models - Applications |
| 3/14 | Generative Models - Paper Review 1 |
| 3/16 | Generative Models - Paper Review 2 |
| 3/21 | Graph Neural Networks - Theory |
| 3/23 | Graph Neural Networks - Variations and Applications |
| 3/28 | No Class - Spring Break |
| 3/30 | No Class - Spring Break |
| 4/4 | Graph Neural Networks - Paper Review 1 |
| 4/6 | Graph Neural Networks - Paper Review 2 |
| 4/11 | Deep Reinforcement Learning - Theory |
| 4/13 | Deep Reinforcement Learning - Variations and Applications |
| 4/18 | Deep Reinforcement Learning - Paper Review 1 |
| 4/20 | Deep Reinforcement Learning - Paper Review 2 |
| 4/25 | Transformer-based Models - Theory |
| 4/27 | Transformer-based Models - Variations |
| 5/2 | Transformer-based Models - Paper Review 1 |
| 5/4 | Transformer-based Models - Paper Review 2 |
| 5/9 | Project Preparation - No Class |
| 5/11 | Project Presentation |
| 5/16 | Project Presentation |
Learning Objectives
- For the five major deep learning topics covered in class, be familiar with the following aspects:
- Theoretical (math) basis
- Algorithmic implementation and considerations
- Major applications and success stories
- Current SOTA methods
- Any possible societal impacts and implications
- Future ways of extension and research
Papers
Projects
Grading
| Category | Percentage | Notes | |
|---|---|---|---|
| Reading assignments | 40 | 2 paper of your choice per topic | |
| Project | Code (+ short repot) | 15 | |
| Presentation | 10 | ||
| Participation | 20 | ||
| Knowledge retention mini-exam | 5 | 1 PM, May 22. In Class, Open Book | |
| Peer-review | 10 | For grading classmates |
| Grade | Interval | Grade | Interval | Grade | Interval |
|---|---|---|---|---|---|
| A | 90 and over | B- | 70 to 75 | D+ | 50 to 55 |
| A- | 85 to 90 | C+ | 65 to 70 | D | 45 to 50 |
| B+ | 80 to 85 | C | 60 to 65 | D- | 40 to 45 |
| B | 75 to 80 | C- | 55 to 60 | F | Below 40 |
Ed Discussion
We will use Ed for ALL of the class communications. The system is highly catered to getting you to help fast and efficiently from classmates, the TA, and myself. Rather than emailing questions to the teaching staff, I encourage you to post your questions on Ed. Check for existing questions, and consider answering your classmates’ questions.