Get 20% off at https://shortform.com/artem

In this video we explore Boltzmann Machines – one of the first generative models that learns probability distribution of data, leveraging stochastic rules and latent representations.

Socials:
X/Twitter: https://x.com/ArtemKRSV
Patreon (to view the extended script): https://www.patreon.com/artemkirsanov

OUTLINE:
00:00 Introduction
01:56 Goal of Boltzmann Machines
05:26 Boltzmann Distribution
13:29 Stochastic Update Rule
17:39 Contrastive Hebbian Rule
25:41 Hidden Units
28:25 Restricted Boltzmann Machines
29:38 Conclusion & Outro

References:
1. Ackley, D., Hinton, G. & Sejnowski, T. A learning algorithm for boltzmann machines. Cognitive Science 9, 147–169 (1985).
2. Downing, K. L. Gradient Expectations: Structure, Origins, and Synthesis of Predictive Neural Networks. (The MIT Press, Cambridge, Massachusetts, 2023).
3. Hinton, G. E. & Salakhutdinov, R. R. Reducing the Dimensionality of Data with Neural Networks. Science 313, 504–507 (2006).
4. Hinton, G. E. A Practical Guide to Training Restricted Boltzmann Machines. in Neural Networks: Tricks of the Trade (eds. Montavon, G., Orr, G. B. & Müller, K.-R.) vol. 7700 599–619 (Springer Berlin Heidelberg, Berlin, Heidelberg, 2012).