AI Learns to Park – Deep Reinforcement Learning

Share it with your friends Like

Thanks! Share it with your friends!

Close

An AI learns to park a car in a parking lot in a 3D physics simulation. The simulation was implemented using Unity’s ML-Agents framework (https://unity3d.com/machine-learning). The AI consists of a deep Neural Network with 3 hidden layers of 128 neurons each. It is trained with the Proximal Policy Optimization (PPO) algorithm, which is a Reinforcement Learning approach.

Basically, the input of the Neural Network are the readings of eight depth sensors, the car’s current speed and position, as well as its relative position to the target. The outputs of the Neural Network are interpreted as engine force, braking force and turning force. These outputs can be seen at the top right corner of the zoomed out camera shots.

The AI starts off with random behaviour, i.e. the Neural Network is initialized with random weights. It then gradually learns to solve the task by reacting to environment feedback accordingly. The environment tells the AI whether it is doing good or bad with positive or negative reward signals.
In this project, the AI is rewarded with small positive signals for getting closer to the parking spot, which is outlined in red, and gets a larger reward when it actually reaches the parking spot and stops there. The final reward for reaching the parking spot is dependent on how parallel the car stops in relation to the actual parking position. If the car stops in a 90° angle to the actual parking direction for instance, the AI will only be rewarded a very small amount, relative to the amount it would get for stopping completely parallel to the actual direction.
The AI is penalized with a negative reward signal, when it either drives further away from the parking spot or if it crashes into any obstacles.

The training process shown in this video took about 23 hours on a computer with an i5 (7th or 8th gen) and a GTX 1070 with 100x simulation speed.

Subscribe for more content like this:
https://www.youtube.com/channel/UC_eerU4SleeptEbD2AA_nDw?sub_confirmation=1

Follow me on Twitter for more frequent updates on my projects:
https://twitter.com/SamuelArzt

Also check out my other videos related to this Project:

Two AI fight for the same Parking Spot:
https://www.youtube.com/watch?v=CqYKhbyHFtA

Neural Networks Explained in a Minute:
https://www.youtube.com/watch?v=rEDzUT3ymw4

Cars learn to maneuver Parcour with Genetic Algorithm:
https://www.youtube.com/watch?v=Aut32pR5PQA

Music from Bensound.com:
Timelapse Music: “The Elevator Bossa Nova”
Comedic Background: “Jazz Comedy”
Outro: “All That”

#ArtificialIntelligence #MachineLearning #ReinforcementLearning #AI #NeuralNetworks

Comments

Samuel Arzt says:

Watch two AI Agents fight for the same parking spot:


raj kumar says:

Do you have to watch each attempt?
Im a beginner in ml, so just curious.

Beat Saber Man says:

Aight bro now just put this into a real car and bam! We have cars that can park themselves

Eugene Dzhonov says:

Ai parks in the front because it does not need to drive out)

Die Offenbarung says:

Gta 6 look awesome.

Michael Hughes says:

peugeot drivers be like

RvN says:

When you tell your wife to park the car

Haitham AlHazmy says:

until 200k it is called AS (artificial stupidity)

Seryan Black says:

everytime i think AI got it… AI hits a lamp xD

Дмитрий Паймуллин says:

Вот проблема ИИ – нет доказательства, что машина когда-нибудь не врежется даже после 300000 попыток и постоянных успешных тестов.

Marcelo Cavalcante says:

After 30 million dollars of damage our little boy learned to park

pramod surya says:

I like 20003 attempt. I always wanted to park like that

Ray says:

Literally every girl at Walmart

Robin Spanier says:

And now lets see how he behaves with different locations.. bad demonstration

Ann Drei says:

Can you share the source code for this project?
Thank you!

ETB says:

Пьяный батя паркует свой жигуль

David Gaudette says:

I think it might learn faster if there was a negative consequence to hitting objects and other cars as well as a positive one for reaching the goal in the least amount of time or whatever.

Gustavo Orsi says:

0:46 when you almost finish your math exam but the last question is too hard

Bess Snow says:

Sooo. Your saying I’m smarter than a.i., cause I learned how to park a car in less than 4 tries.

Brim Cap says:

Nobody:

Not a single soul:

That girl you see when you look out of the window: 2:46

Write a comment

*