Set up Google Vertex AutoML service

Use this guide to integrate your model training pipeline with GCP Vertex AI. Vertex AI an ML platform where you can build a model training pipeline, train and deploy models. Support for other cloud providers coming soon.

Requirements

1. Install docker-compose.

2. install gcloud client

# install gcloud cli
curl https://sdk.cloud.google.com | bash
# load env vars
source ~/.<bash_profile/zshrc/bashrc> 
# login
gcloud auth login
# Set the correct gcloud project
gcloud config set project PROJECT_NAME

3. Create gcloud service account. Create gcloud service account keys if you don't have one already.

gcloud iam service-accounts create ACCOUNT_NAME \
    --description="account for configuring model training" \
    --display-name="model-training"

4. Assign gcloud service account your account permissions.

    # assign gcloud service account permissions
gcloud iam service-accounts add-iam-policy-binding \
    SERVICE_ACCOUNT@PROJECT_ID.iam.gserviceaccount.com \
    --member="user:YOUR_ACCOUNT_EMAIL" \
    --role="roles/iam.serviceAccountUser"

5. Set path for your service account credentials.

export GOOGLE_APPLICATION_CREDENTIALS=/Path/to/credentials/.config/gcloud/<name>.json

6. Create the service account credentials

gcloud iam service-accounts keys create $GOOGLE_APPLICATION_CREDENTIALS \
    --iam-account=ACCOUNT_NAME@PROJECT_ID.iam.gserviceaccount.com

7. Connect docker to Google Container Registry (GCR)

gcloud auth configure-docker

Deploy model training service to cloud platform

1. Clone the Labelbox model training repo:

git clone https://github.com/Labelbox/model-training.git
cd model-training

2. Create a .env file to keep track of the following env vars (copy .env.example to get started):

   • GCS_BUCKET
     • GCS bucket to store all of the artifacts. It must be a unique name. If the bucket doesn't exist it will automatically be created.
   • GOOGLE_PROJECT
     • Google cloud project name.
   • SERVICE_SECRET
     • You will have to use the same secret when making a request to the service from your Labelbox UI.
   • GOOGLE_APPLICATION_CREDENTIALS
     • Path to the application credentials.
   • LABELBOX_API_KEY
     • Your Labelbox API key
   • GOOGLE_SERVICE_ACCOUNT
     • Google service account. Will have the following format: <name>@<project>.iam.gserviceaccount.com.
   • DEPLOYMENT_NAME
     • This is the name that all of the google resources will use. You can deploy separate coordinator services (such as prod-training-service for production and dev-training-service for development)

3. Once the .env file has the correct values or if you update it later, load the env vars again. Deploy the service to the cloud. This step might take a while.

source .env
./deployment/deploy.sh

When this script completes, copy the IP address printed in the console. This will be the ip address of your coordinator service. You will need to use it to configure the model training integration in Labelbox UI later.

4. Test that it is running with:

curl http://<ip>:8000/ping

The remote ip will be printed to the console when you run the deployment script. The server will respond with pong if the deployment was successful

5. (Recommended) Reserve a static IP address on Google by following this [guide] so that the IP address of your coordinator service won't change after a few days. (https://cloud.google.com/compute/docs/ip-addresses/reserve-static-external-ip-address)