Distribution Augmentation for Generative Modeling
This video explains a recent paper from OpenAI exploring how to improve generative models with data augmentation. DistAug conditions models on the transformation in a multi-task learning way. This results in improved performance, particularly with more parameters, more augmentations, and less dropout! Thanks for watching! Please Subscribe!
Paper Links:
DistAug: https://proceedings.icml.cc/static/paper_files/icml/2020/6095-Paper.pdf
ImageGPT: https://openai.com/blog/image-gpt/
A Survey on Image Data Augmentation: https://link.springer.com/article/10.1186/s40537-019-0197-0
Training GANs with Limited Data: https://arxiv.org/abs/2006.06676
Chapters
0:00 Beginning
1:36 Data Augmentation in Computer Vision
4:20 Challenge of Data Aug in Generative Modeling
6:24 DistAug, condition on augmentation embedding
8:20 Start of Sequence token embedding
11:20 Data-Dependent Regularization and Multi-Task Learning
12:32 Examples of Generated Images and Nearest Neighbors in the original dataset
13:34 Benefits of Scale - ImageGPT-2?
15:23 Regularizing Deep Learning, Data Augmentation vs. Dropout / Weight Decay / L2 Regularization
16:53 Why does Rotation augmentation work better than Jigsaw?
This video explains a recent paper from OpenAI exploring how to improve generative models with data augmentation. DistAug conditions models on the transformation in a multi-task learning way. This results in improved performance, particularly with more parameters, more augmentations, and less dropout! Thanks for watching! Please Subscribe!
Paper Links:
DistAug: https://proceedings.icml.cc/static/paper_files/icml/2020/6095-Paper.pdf
ImageGPT: https://openai.com/blog/image-gpt/
A Survey on Image Data Augmentation: https://link.springer.com/article/10.1186/s40537-019-0197-0
Training GANs with Limited Data: https://arxiv.org/abs/2006.06676
Chapters
0:00 Beginning
1:36 Data Augmentation in Computer Vision
4:20 Challenge of Data Aug in Generative Modeling
6:24 DistAug, condition on augmentation embedding
8:20 Start of Sequence token embedding
11:20 Data-Dependent Regularization and Multi-Task Learning
12:32 Examples of Generated Images and Nearest Neighbors in the original dataset
13:34 Benefits of Scale – ImageGPT-2?
15:23 Regularizing Deep Learning, Data Augmentation vs. Dropout / Weight Decay / L2 Regularization
16:53 Why does Rotation augmentation work better than Jigsaw?
