Deploy and generate predictions

Once you have successfully trained a model using model training integration with Vertex AI, you can deploy the model to an endpoint for inference.

Deploy your model to an endpoint on Vertex

Go to Vertex AI console, select the Models page and click on the model that you want to deploy.

Click on the ellipsis and select “Deploy to endpoint”.

Fill out the deployment config and press deploy.

Run model inference on new data with the deployed model endpoint.

import base64
from io import BytesIO

from google.cloud import aiplatform
from PIL import Image, ImageDraw

# Specify model endpoint (must be deployed to an endpoint)
endpoint_name = "YOUR_ENDPOINT_NAME"
endpoint = aiplatform.Endpoint.list(filter=f'display_name="{endpoint_name}"')[0]

# Preprocess new data for inference
with open('dog.jpg', "rb") as f:
    image_bytes = f.read()
# Note that the max content size is 1.5mb
b64_bytes = base64.b64encode(image_bytes).decode("utf-8")

# Do inference on new data
result = endpoint.predict(
  instances=[{
    'content': b64_bytes
}],
  parameters={
    'confidenceThreshold': 0.5,
    'maxPredictions': 5
  })

# Visualize results
im = Image.open(BytesIO(image_bytes))
w, h = im.size
d = ImageDraw.Draw(im)
for prediction in result.predictions:
    for bbox in prediction['bboxes']:
        d.rectangle([bbox[0] * w, bbox[2] * h, bbox[1] * w, bbox[3] * h])
im.show()