MIT 6.S191: Recurrent Neural Networks, Transformers, and Attention
MIT Introduction to Deep Learning 6.S191: Lecture 2
Recurrent Neural Networks
Lecturer: Ava Amini
2023 Edition
For all lectures, slides, and lab materials: http://introtodeeplearning.com
Lecture Outline
0:00 - Introduction
3:07 - Sequence modeling
5:09 - Neurons with recurrence
12:05 - Recurrent neural networks
13:47 - RNN intuition
15:03 - Unfolding RNNs
18:57 - RNNs from scratch
21:50 - Design criteria for sequential modeling
23:45 - Word prediction example
29:57 - Backpropagation through time
32:25 - Gradient issues
37:03 - Long short term memory (LSTM)
39:50 - RNN applications
44:50 - Attention fundamentals
48:10 - Intuition of attention
50:30 - Attention and search relationship
52:40 - Learning attention with neural networks
58:16 - Scaling attention and applications
1:02:02 - Summary
Subscribe to stay up to date with new deep learning lectures at MIT, or follow us @MITDeepLearning on Twitter and Instagram to stay fully-connected!!
MIT Introduction to Deep Learning 6.S191: Lecture 2
Recurrent Neural Networks
Lecturer: Ava Amini
2023 Edition
For all lectures, slides, and lab materials: http://introtodeeplearning.com
Lecture Outline
0:00 – Introduction
3:07 – Sequence modeling
5:09 – Neurons with recurrence
12:05 – Recurrent neural networks
13:47 – RNN intuition
15:03 – Unfolding RNNs
18:57 – RNNs from scratch
21:50 – Design criteria for sequential modeling
23:45 – Word prediction example
29:57 – Backpropagation through time
32:25 – Gradient issues
37:03 – Long short term memory (LSTM)
39:50 – RNN applications
44:50 – Attention fundamentals
48:10 – Intuition of attention
50:30 – Attention and search relationship
52:40 – Learning attention with neural networks
58:16 – Scaling attention and applications
1:02:02 – Summary
Subscribe to stay up to date with new deep learning lectures at MIT, or follow us @MITDeepLearning on Twitter and Instagram to stay fully-connected!!
