THE FUTURE IS HERE

How to Create a Neural Network (and Train it to Identify Doodles)

Exploring how neural networks learn by programming one from scratch in C#, and then attempting to teach it to recognize various doodles and images.

Source code: https://github.com/SebLague/Neural-Network-Experiments
Demo: https://sebastian.itch.io/neural-network-experiment

If you’d like to support me in creating more videos (and get early access to new stuff), you can join my patreon here: https://www.patreon.com/SebastianLague

The data is from:
Digits http://yann.lecun.com/exdb/mnist/
Fashion https://github.com/zalandoresearch/fashion-mnist
Doodles https://github.com/googlecreativelab/quickdraw-dataset
Cifar10 https://www.cs.toronto.edu/~kriz/cifar.html

Chapters
0:00 Introduction
2:39 The decision boundary
3:49 Weights
5:42 Biases
6:45 Hidden layers
7:45 Programming the network
9:57 Activation functions
12:42 Cost
15:07 Gradient descent example
18:22 The cost landscape
19:55 Programming gradient descent
21:10 It’s learning! (slowly)
23:21 Calculus example
27:34 The chain rule
29:50 Some partial derivatives
33:14 Backpropagation
39:25 Digit recognition
43:56 Drawing our own digits
47:37 Fashion
48:25 Doodles
52:00 The final challenge

Music:
Cosmic Waves – Michael FK
Amber – The Stolen Orchestra
Beyond the Horizon – Sounds Like Sander
Air – Assaf Ayalon
Purest Form – Sounds Like Sander
Hear Wide Open – Sounds Like Sander
Universal Wonder – Moments
Roman P – Moments
All In Good Time – Shimmer
It Will Come Back – The Stolen Orchestra
Frontier – Shimmer
New Moon – Cloud Wave
Sunflower – Cody Martin
Inner Peace – Moments
Enchanted – Cody Martin
Just Around The Corner – Shimmer