How to Deploy a Machine Learning Model to Google Cloud for 20% Software Engineers (CS329s tutorial)

Share it with your friends Like

Thanks! Share it with your friends!


It’s time to reveal the magician’s secrets behind deploying machine learning models! In this tutorial, I go through an example machine learning deployment scenario using Google Cloud and an image recognition app called Food Vision 🍔👁.

Get all the code on GitHub –
Slides –
Full CS329s syllabus –
Learn ML (my beginner-friendly ML course) –

Connect elsewhere:
Web –
Get email updates on my work –

0:00 – Intro/hello
1:42 – Presentation start (what we’re going to cover)
6:00 – Food Vision 🍔👁 (the app we’re building) recipe
11:16 – The end goal we’re working towards (data flywheel)
13:07 – The data flywheel: the holy grail of ML apps
14:57 – Tesla’s data flywheel
17:02 – Food Vision’s data flywheel
18:24 – Deploying a model on the cloud outline
21:14 – Steps we’re going to go through to deploy our app
27:06 – Question: “How do you identify hard samples in your data?”
37:53 – Creating a bucket on Google Storage
45:51 – Uploading to Google Storage from Google Colab
48:02 – Deploying a model to AI Platform
52:50 – Creating an AI Platform Prediction version
58:10 – Creating a Service Account to access our model on Google Cloud
1:02:32 – Authenticating our app with our private Service Account key
1:09:19 – What happens when we run make gcloud-deploy
1:11:27 – Problems you’ll run into when deploying your models
1:20:12 – Extensions you could perform on this tutorial
1:20:49 – Part 2 start (tutorial overtime)
1:28:43 – Dealing with different data shapes
1:32:35 – An error you might run into when using the example app (3 total models deployed)
1:33:20 – Dealing with data size restrictions
1:38:48 – Stepping through the make gcloud-deploy command
1:51:00 – Summary and wrap up



Daniel Bourke says:

*Update (20 May 2021):* After Google's recent I/O event, AI Platform has now changed its name to Vertex AI, so some of the terminology/steps on the GCP platform may be named differently/have slightly different steps, but the main principles of deploying a model should remain.

Happy Valentine's Day team ❤️! Sending love to you all. Happy Machine Learning!

Maria Kalimeri says:

Hi Daniel and many thanks for the effort! Great video!

Am I wrong in the following? (This is from the part where you use the app locally with the model deployed on GCP) When you created the model version, right at the bottom of the setup page (57:02), there was the option to choose a service account which you ended up not specifying at that point (neither later?). Then you created a service account, generated a key for it and saved it locally for the app to use when making a request. I did the same and managed to get it worked but only when I don't explicitly specify the sa. When I do specify the sa (during model version creation) I get "Failed to add IAM policy binding" error and model is not deployed. By playing around, I figured that the credentials for the specific sa don't even matter, GCP apparently has some default sa that it uses when one doesn't specify a custom sa.

rex says:

Dan mate are u sing the apple m1 or just a regular mac cus my m1 is having issues installing all this shit

Deekshitha DP says:

Hey Daniel,

First of all thank you so so much for providing such informative content!! I sincerely appreciate all your efforts and hardwork.
By watching this video, I have gathered plenty of information. I am going to continue watching all your videos in the coming days!!

I am having few doubts. I will be greatful to you if you could guide me and suggest me your thoughts on this.

1. At 1:17:56 you mentioned about one class containing 11 classes while another class with random non-food images. What I understand from here is that you have got 2 main classes, one of them having 11 different classes, which makes it binary classification. How can I train this kind of image directory?
I am having similar use case; I want to classify oral and non-oral images, where a non-oral class contaiins any random images of different categories.
Could you please help me with this problem? How can classify my images?

2. Could you please let me what are the requirements/preparations that I need beforehand to deploy my trained model into GCP to use it in APP development?

Thank you very much once again for such a valuable content!! ❤️ ❤️ ❤️

Keep rising and surprising people with more videos!!!!!

Nam Phan Hoàng says:

Thank you for your instruction. I really love it. Have you figured out how to deal with the timeout problem in google cloud build steps? I'm considering ditching the docker on local and go full cloudbuilds.yaml. I will post my result here when it's done.

Talha Khursheed says:

How much will it cost me to deploy my ML model on GCP?

Golden water says:

can you shrink the video length for the future? It is too long ..

Mohamed Abdelrehem says:

i am bored on the train going home so i found your video best 2 hours of my life <3 <3

Jorge Felipe Gaviria Fierro says:

Thank you so much for this, really great content 🙂

hamzakivideo says:

Thank you sharing this! Quick question, how should I run tensorboard; I ran %tensorboard –logdir logs/scalar – but I keep getting an error of dataset not active

Travis Vitello says:

I almost never comment on YouTube videos, but here I wanted to break that trend in order to kindly thank you for this tutorial. This was a MAJOR lifesaver for me in my SYS6016 (Machine Learning) course at UVA. Absolutely awesome stuff.

002_Aarish Alam says:

converting floating points to int16 is giving weird results.

Sơn says:

That's great video. Thanks you.
–From Viet Nam with love–

Ajay Vasisht says:

I absolutely loved this! I've been following your channel since watching your Udacity reviews since I've been wanting to learn more ML / DL. Unrelatedly I came across Chip's class through her twitter. When worlds collide! Really great lesson, Daniel – super useful to see things actually work, and love your energy! Keep it up 🙂

One thing – can you explain how you went about going from model 1 (without donut class) to going to model 2 (with donut class) in GCP? Trying to understand the flywheel better!

Alex Chan says:

47:20 my exact reaction whenever anything goes right

RisuAnetto says:

How can I use Pytorch model? Do I need a custom container? What container is useful for a BERT model?

Nicholai says:

Is there any diff. between ML and AI engineers, does AI engineer(job role) really exist?

Kesava E says:

3:41 I loved the way how he used kite image for small apps and plane for large apps 😂

Lauren Zhu says:

thanks for coming to our class!!! my favorite guest lecture so far!!

tsogoo bayartsogt says:

44:28 This is one way, but if you already connected with your Google Drive, then you'd be able to use GCP for that particular account (which I think a lot more easier)

Ameen Bello says:

Does anyone have the link to his twitch channel, can't seem to fnd it anywhere

Aritxa Troyano says:

Great video Daniel. Keep Going!!!

Jithendra Yenugula says:

You using Google instead of duckduckgo? 😐 It's from your videos I started using duckduckgo 😅
Anyway great tutorial…Thanks for making it public🤟

Write a comment


Area 51