Welcome to this Step-by-Step Guide on getting started with building deep learning model, serve it as REST API with Flask and deploy it using Docker and Kubernetes on Google Cloud Platform (GCP). Throughout this post, the focus will be on steps to successfully deploy the AI Model.

This is a guide for anyone out there who has heard about Docker and Kubernetes but hasn’t tried it out yet. To know more about DevOps Pipeline Workflow to Deploy AI Models with Docker and Kubernetes, read the related post at An Overview of Pipeline Workflow to Deploy AI Models with Docker and Kubernetes
As this guide uses Docker and Kubernetes from GCP, you do not need get into the hassle of installing Docker and Kubernetes on your system.
Step 0: Need Google Account for GCP

Step 0.1: Creation of Account on Google
To get started we need a Google account. Once our google account is ready, we need to setup GCP.
Once our GCP is ready, we need to create a virtual machine on GCP to run out project.
Step 0.2: Creation of VM Instance on GCP
To create a VM instance you need to navigate to Compute Engine from the navigation bar on the top left corner of your screen and then select VM instances.
Then choose “Create”. Next, we will choose the operating system, the computational power, disk size and a few more specifications.


And now click on “Create”.
Step 0.3: Accessing the VM Instance using SSH Terminal.
Let’s get started with setting up the VM Instance for deploying using Docker. Click on ‘SSH’ to open SSH terminal.

Step 1: Docker Set Up on VM Instance
Step 1.1:
Firstly, we will remove the existing Docker from our VM Instance to do a clean installation of Docker. Note: if you choose OS other than CentOS 7, these commands may vary.
sudo yum remove docker docker-client docker-client-latest docker-common docker-latest docker-latest-logrotate docker-logrotate docker-selinux docker-engine-selinux docker-engine
Step 1.2:
Install Docker
sudo yum install -y yum-utils device-mapper-persistent-data lvm2
sudo yum-config-manager –add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo yum install docker-ce
Step 1.3:
Start Docker & Check if Docker is running properly. Once Docker installation is completed, we will start Docker
Check if docker is running properly
sudo systemctl start docker
sudo docker run hello-world
If you get the output as the one below, you are ready to move to next step:
Hello from Docker!
This message shows that your installation appears to be working correctly.To generate this message, Docker took the following steps: 1. The Docker client contacted the Docker daemon. 2. The Docker daemon pulled the “hello-world” image from the Docker Hub. (amd64) 3. The Docker daemon created a new container from that image which runs the executable that produces the output you are currently reading. 4. The Docker daemon streamed that output to the Docker client, which sent it to your terminal.
Step 1.4: Including the Flask API for Deep Learning Model on this VM Docker Instance
First, create a new directory called “docker_project”
Make sure to change directory and move into that folder
Command:
mkdir docker_project
cd docker_project
Step 1.4.1: Create app.py
Now, we will create app.py which will contain our deep learning model served as REST API on Flask.
Make sure, when calling app.run() specify the host as ‘0.0.0.0’ like app.run(host=’0.0.0.0’). It makes our app available on localhost as well as on external IP.
vim app.py
Copy paste the below code into app.py
from keras.applications import ResNet50 from keras.preprocessing.image import img_to_array from keras.applications import imagenet_utils from PIL import Image import numpy as np import flask import io import tensorflow as tf # initialize our Flask application and the Keras model app = flask.Flask(__name__) model = None def load_model(): # load the pre-trained Keras model (here we are using a model # pre-trained on ImageNet and provided by Keras, but you can # substitute in your own networks just as easily) global model model = ResNet50(weights="imagenet") global graph graph = tf.get_default_graph() def prepare_image(image, target): # if the image mode is not RGB, convert it if image.mode != "RGB": image = image.convert("RGB") # resize the input image and preprocess it image = image.resize(target) image = img_to_array(image) image = np.expand_dims(image, axis=0) image = imagenet_utils.preprocess_input(image) # return the processed image return image @app.route("/predict", methods=["POST"]) def predict(): # initialize the data dictionary that will be returned from the # view data = {"success": False} # ensure an image was properly uploaded to our endpoint if flask.request.method == "POST": if flask.request.files.get("image"): # read the image in PIL format image = flask.request.files["image"].read() image = Image.open(io.BytesIO(image)) # preprocess the image and prepare it for classification image = prepare_image(image, target=(224, 224)) # classify the input image and then initialize the list # of predictions to return to the client with graph.as_default(): preds = model.predict(image) results = imagenet_utils.decode_predictions(preds) data["predictions"] = [] # loop over the results and add them to the list of # returned predictions for (imagenetID, label, prob) in results[0]: r = {"label": label, "probability": float(prob)} data["predictions"].append(r) # indicate that the request was a success data["success"] = True # return the data dictionary as a JSON response return flask.jsonify(data) # if this is the main thread of execution first load the model and # then start the server if __name__ == "__main__": print(("* Loading Keras model and Flask starting server..." "please wait until server has fully started")) load_model() app.run(host='0.0.0.0')
Once you have copied the above code, you exit edit mode by hitting the “Esc” key. You then save and close the file by typing “:x”
Step 1.4.2: Create requirements.txt
To install dependencies required for our app, we need to create a file called requirements.txt.
This file will contain the packages our code needs to run, e.g., keras, flask, and others. This way no matter where we ship our Docker container, the underlying server will be able to install our code’s required dependencies.
vim requirements.txt
Copy the following into requirements.txt and save and close as you did before for app.py file
keras==2.2.4
tensorflow==1.14.0
flask
gevent
pillow
requests
Step 1.4.3: Create Dockerfile
A Dockerfile is a text file that contains a list of commands that the Docker daemon calls while creating an image. The Dockerfile contains all the information that Docker needs to know to run the app — a base Docker image to run from, location of your project code, any dependencies it has, and what commands to run at start-up. It is a simple way to automate the image creation process. The best part is that the commands you write in a Dockerfile are almost identical to their equivalent Linux commands. This means you don’t really have to learn new syntax to create your own Dockerfile.
Copy the following into Dockerfile. Save and close as you did before (using ‘Esc’ followed by :x”)
Docker reads this file to build and run the project.
vim Dockerfile
FROM python:3.6.10
COPY . /app
WORKDIR /app
RUN pip install -r requirements.txt
EXPOSE 5000
CMD [“python”, “app.py”]
Step 2: Build New Docker Image
Once your Dockerfile is ready. you can go ahead build your docker container. To build our Docker container, run the following command.
This instructs Docker to build a container for the code located in our current working director “docker_project”. With this command, python and the packages listed in requitements.txt are installed in our Docker Container.
sudo docker build -t docker_project:latest .
Step 3: Run Tests
Step 3.1: Once building the Docker Image is built, we will run this docker image and check if it works.
Check the status of your container
sudo docker run -p 5000:5000 docker_project
sudo docker ps -a
Step 3.2 : Testing out model.
- With our model running, it’s time to test it. Select “View gcloud command” from SSH Options

- Select Run in CLOUD SHELL.
- Execute the first command as it is.

And run the following command:
curl -X POST -F image=@tiger.jpg ‘http://localhost:5000/predict’
Make sure tiger.jpg is in your current directory
You should see result like this:

We can see that our model correctly classified image of a tiger, as a TIGER. Bravo! You have successfully run a trained a deep learning model with Keras, served it with Flask, and wrapped it with Docker. The hard part is over.
But, we are not yet done here. You will ONLY be able to access this docker image through the same VM machine.
What if you want to run the prediction from our deep learning model from anywhere in the world? This is where Kubernetes comes into play.
Now, let’s deploy this container with Kubernetes.
Initiate Deployment on KUBERNETES
Step 4: To get started with Kubernetes, we/you need Docker Hub Account.
The reason we do this is that we won’t physically move our container to our Kubernetes cluster. Instead, we will instruct Kubernetes to install our container from a centrally hosted server, i.e., Docker Hub.
Now, we need to login to Docker Account from the same SSH
Step 4.1: Docker Hub Login
If you see message as : “Login Succeeded”, you are successfully logged into your Docker account, you can move to next step.
Step 4.2: Tagging Docker Container
Now, we need to tag our container before we upload it. For this we locate the image id .
sudo docker login
Command:
sudo docker images
Output will look like this:

Step 4.2: Tagging Docker Container (contd..)
Now, let’s tag our container . NOTE: Change the highlighted text as per your image ID , your Docker Hub ID and project name
sudo docker tag 79bf849d84c7 <Docker Hub ID>/docker_project
Step 4.3: Once our docker container is tagged, we can push our container to Docker Hub.
If you navigate to your Docker Hub Account, you should see the docker_project app in your repository
sudo docker push <Docker Hub ID>/docker_project
Step 5: Create Kubernetes Cluster for Deployment
Step 5.1: Navigate to Kubernetes Engine in GCP

Step 5.2: Create a new Kubernetes Cluster

After that click “Create”.
To connect to this cluster, click on “Connect”.

Click on “Run in Cloud Shell”. It will bring up a new console for Kubernetes Cluster.

Step 5.3: Now, We run our Docker container in Kubernetes
Let’s create a deployment of our app.
kubectl create deployment docker-project –image=<Docker Hub ID>/docker_project
Step 6: Run Kubernetes Service
Once the deployment is created, we can run our app. We’ll specify with –port that we want to run our app on port 5000
kubectl run docker-project –image=<Docker Hub ID>/docker_project –port 5000
Step 7: Expose the Deployment
We need to expose out pod on port 80 to outside world.
kubectl expose deployment docker-project –type=LoadBalancer –port 80 –target-port 5000

Open terminal on any PC and run the following command:
curl -X POST -F image=@tiger.jpg ‘http://<External IP of your VM instance>/predict’
By executing, the above steps you have now successfully build a deep learning model, served it as REST API with Flask and deployed it using Docker and Kubernetes on Google Cloud Platform (GCP).
At StrataHive, Our Core Capability is in Artificial Intelligence-based Software Component Development and Integration with other Systems/Application. We assist our Clients in adopting new Architectures and to operationalize their AI and ML Initiatives.
Know more about StrataHive’s Solution Offerings at this link.
Thanks for reading through this post. Any suggestions for further improving this would be cheerfully solicited.