- ES Español (Latinoamérica)

- EN English

7.54. Deep Learning (Mandatory)
- Semester: 9th Sem. Credits: 5
- Hour of this course: Theory: 3 hours; Laboratory: 4 hours;
- Syllabus:
- htmlonly

Español (Latinoamérica)

English - Prerrequisites:
- AI263 Introduction to Machine Learning (8th Sem)
7.54.1. Justification ↑ Back to top
This course covers modern deep learning fundamentals, including convolutional networks, transformer architectures, and training techniques for advanced AI systems.
7.54.2. Generales Goals ↑ Back to top
- Implement deep neural networks using modern frameworks (PyTorch/TensorFlow)
- Design architectures for specific domains (vision, language, etc.)
- Analyze ethical limitations of deep models
7.54.3. Contribution to Outcomes ↑ Back to top
- ABET-2) An ability to formulate or design a system, process, procedure, or program for the intended purpose. (Usage)
- ABET-3) An ability to develop and conduct experiments or test hypotheses, analyze and interpret data, and use scientific judgment to draw conclusions. (Assessment)
7.54.4. Content ↑ Back to top
7.54.4.1. Neural Network Training and Optimization (10 hours) [Skills ABET-2,ABET-3] ↑ Back to top
Bibliography: (Goodfellow et al., 2016b)
Topics
- Modern backpropagation: computational graphs and automatic differentiation
- Advanced optimizers: momentum, RMSProp, Adam, and learning-rate schedules
Learning Outcomes
- Implement a deep neural network from scratch, including its backpropagation pass [Apply]
- Select and tune an optimizer and its learning-rate schedule for training stability [Apply]
7.54.4.2. Regularization Techniques for Deep Networks (5 hours) [Skills ABET-2,ABET-3] ↑ Back to top
Bibliography: (Goodfellow et al., 2016b)
Topics
- Dropout regularization
- Batch normalization
Learning Outcomes
- Diagnose overfitting in a training run using regularization diagnostics [Analyze]
- Apply dropout and batch normalization to improve a model's generalization [Apply]
7.54.4.3. Convolutional and Residual Network Architectures (9 hours) [Skills ABET-2,ABET-3] ↑ Back to top
Bibliography: (He et al., 2016)
Topics
- Deep convolutional neural network architectures
- Residual connections and ResNets
Learning Outcomes
- Design a deep CNN architecture for a given vision task [Apply]
- Adapt a residual architecture to a specific vision domain [Apply]
7.54.4.4. Transformer Architectures and Generative Models (16 hours) [Skills ABET-2,ABET-3] ↑ Back to top
Bibliography: (Vaswani et al., 2017a)
Topics
- Transformers and the attention mechanism
- Generative models: variational autoencoders (VAEs) and generative adversarial networks (GANs)
Learning Outcomes
- Train a transformer model on a real dataset [Apply]
- Analyze and contrast VAE and GAN generative-model architectures [Analyze]
7.54.5. Bibliography ↑ Back to top
Goodfellow, I., Bengio, Y., and Courville, A. (2016b). Deep Learning. MIT Press.
He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. CVPR, pages 770–778.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A., Kaiser, L., and Polosukhin, I. (2017a). Attention is all you need. Advances in Neural Information Processing Systems, 30.