A Step-by-Step Guide for Deploying AI Models using Docker and Kubernetes on Google Cloud Platform

Welcome to this Step-by-Step Guide on getting started with building deep learning model, serve it as REST API with Flask and deploy it using Docker and Kubernetes on Google Cloud Platform (GCP). Throughout this post, the focus will be on steps to successfully deploy the AI Model.

This is a guide for anyone out there who has heard about Docker and Kubernetes but hasn’t tried it out yet. To know more about DevOps Pipeline Workflow to Deploy AI Models with Docker and Kubernetes, read the related post  at An Overview of Pipeline Workflow to Deploy AI Models with Docker and Kubernetes

As this guide uses Docker and Kubernetes from GCP, you do not need get into the hassle of installing Docker and Kubernetes on your system.

Step 0.1: Creation of Account on Google

To get started we need a Google account. Once our google account is ready, we need to setup GCP.

Once our GCP is ready, we need to create a virtual machine on GCP to run out project.

Step 0.2: Creation of VM Instance on GCP

To create a VM instance you need to navigate to Compute Engine from the navigation bar on the top left corner of your screen and then select VM instances.

Then choose “Create”. Next, we will choose the operating system, the computational power, disk size and a few more specifications.

And now click on “Create”.

Step 0.3: Accessing the VM Instance using SSH Terminal.

Let’s get started with setting up the VM Instance for deploying using Docker. Click on ‘SSH’ to open SSH terminal.

If you get the output as the one below, you are ready to move to next step:

Hello from Docker!
This message shows that your installation appears to be working correctly.To generate this message, Docker took the following steps: 1. The Docker client contacted the Docker daemon. 2. The Docker daemon pulled the “hello-world” image from the Docker Hub. (amd64) 3. The Docker daemon created a new container from that image which runs the executable that produces the output you are currently reading. 4. The Docker daemon streamed that output to the Docker client, which sent it to your terminal.

Copy paste the  below code into app.py

from keras.applications import ResNet50
from keras.preprocessing.image import img_to_array
from keras.applications import imagenet_utils
from PIL import Image
import numpy as np
import flask
import io
import tensorflow as tf

# initialize our Flask application and the Keras model
app = flask.Flask(__name__)
model = None

def load_model():
	# load the pre-trained Keras model (here we are using a model
	# pre-trained on ImageNet and provided by Keras, but you can
	# substitute in your own networks just as easily)
	global model
	model = ResNet50(weights="imagenet")
	global graph
	graph = tf.get_default_graph()

def prepare_image(image, target):
	# if the image mode is not RGB, convert it
	if image.mode != "RGB":
		image = image.convert("RGB")

	# resize the input image and preprocess it
	image = image.resize(target)
	image = img_to_array(image)
	image = np.expand_dims(image, axis=0)
	image = imagenet_utils.preprocess_input(image)

	# return the processed image
	return image

@app.route("/predict", methods=["POST"])
def predict():
	# initialize the data dictionary that will be returned from the
	# view
	data = {"success": False}

	# ensure an image was properly uploaded to our endpoint
	if flask.request.method == "POST":
		if flask.request.files.get("image"):
			# read the image in PIL format
			image = flask.request.files["image"].read()
			image = Image.open(io.BytesIO(image))

			# preprocess the image and prepare it for classification
			image = prepare_image(image, target=(224, 224))

			# classify the input image and then initialize the list
			# of predictions to return to the client
			with graph.as_default():
				preds = model.predict(image)
				results = imagenet_utils.decode_predictions(preds)
				data["predictions"] = []

				# loop over the results and add them to the list of
				# returned predictions
				for (imagenetID, label, prob) in results[0]:
					r = {"label": label, "probability": float(prob)}
					data["predictions"].append(r)

				# indicate that the request was a success
				data["success"] = True

	# return the data dictionary as a JSON response
	return flask.jsonify(data)

# if this is the main thread of execution first load the model and
# then start the server
if __name__ == "__main__":
	print(("* Loading Keras model and Flask starting server..."
		"please wait until server has fully started"))
	load_model()
	app.run(host='0.0.0.0')

Once your Dockerfile is ready.  you can go ahead build your docker container. To build our Docker container, run the following command.

This instructs Docker to build a container for the code located in our current working director “docker_project”. With this command, python and the packages listed in requitements.txt are installed in our Docker Container.

Step 3.1: Once building the Docker Image is built, we will run this docker image and check if it works.

Check the status of your container

Step 3.2 : Testing out model.

• With our model running, it’s time to test it. Select “View gcloud command” from SSH Options

• Select Run in CLOUD SHELL.
• Execute the first command as it is.

And run the following command:

curl -X POST -F image=@tiger.jpg ‘http://localhost:5000/predict’

Make sure tiger.jpg is in your current directory

You should see result like this:

We can see that our model correctly classified image of a tiger, as a TIGER. Bravo! You have successfully run a trained a deep learning model with Keras, served it with Flask, and wrapped it with Docker.  The hard part is over.

But, we are not yet done here. You will ONLY be able to access this docker image through the same VM machine.

What if you want to run the prediction from our deep learning model from anywhere in the world? This is where Kubernetes comes into play.

 Now, let’s deploy this container with Kubernetes.

Step 4: To get started with Kubernetes, we/you need Docker Hub Account.

The reason we do this is that we won’t physically move our container to our Kubernetes cluster. Instead, we will instruct Kubernetes to install our container from a centrally hosted server, i.e., Docker Hub.

Now, we need to login to Docker Account from the same SSH

Step 4.1: Docker Hub Login

If you see message as : “Login Succeeded”, you are successfully logged into your Docker account, you can move to next step.

Step 4.2: Tagging Docker Container

Now, we need to tag our container before we upload it. For this we locate the image id .

Output will look like this:

Step 4.2: Tagging Docker Container (contd..)

Now, let’s tag our container . NOTE: Change the highlighted text as per your image ID , your Docker Hub ID and project name

Step 5.1: Navigate to Kubernetes Engine in GCP

Step 5.2: Create a new Kubernetes Cluster

After that click “Create”.

To connect to this cluster, click on “Connect”.

Click on “Run in Cloud Shell”. It will bring up a new console for Kubernetes Cluster.

Step 5.3: Now, We run our Docker container in Kubernetes

Let’s create a deployment of our app.

Once the deployment is created, we can run our app. We’ll specify with –port that we want to run our app on port 5000

We need to expose out pod on port 80 to outside world.

Open terminal on any PC and run the following command:

By executing, the above steps you have now successfully build a deep learning model, served it as REST API with Flask and deployed it using Docker and Kubernetes on Google Cloud Platform (GCP).

At StrataHive, Our Core Capability is in Artificial Intelligence-based Software Component Development and Integration with other Systems/Application. We assist our Clients in adopting new Architectures and to operationalize their AI and ML Initiatives.

Know more about StrataHive’s Solution Offerings at this link.

Thanks for reading through this post. Any suggestions for further improving this would be cheerfully solicited.