Create a model run

A model run is a model training experiment within a model directory. Each model run provides a versioned data snapshot of the data rows, annotations, and data splits for that model run. You can upload predictions to the model run and compare its performance against other model runs in a model directory.

Overview

Once you create a model and choose the labeled data rows for training, you can add a model run to keep track of your model training experiment. It is important to put your model run into the model directory with the correct data rows and ontology configured in the previous steps.

📘

Multiple experiments

If you are iterating multiple model experiments on a machine learning task, it is best practice to put your model runs under the same model directory. This allows you to visualize and compare the performance of the different model runs.

During the model run creation process, you can specify how you want to split the selected data rows into training, validation, and testing sets. To learn more, see Curate data splits.

Labelbox versions your data rows, annotations, and data splits at the time of training, making it easy for you to reproduce your model and compare models trained on different data versions.

Create a model run via the UI

1. First, select the model you just created under the Model tab. Make sure to choose the right model directory, because the model run will access the data rows and ontology defined by the model in the previous step.

2. If you already have model runs in this model directory, you should see the previous model runs' evaluation results. In this case, we are going to create our first model run for the newly created model.

To create a new model run, click + New model run. This will lead you to configure a new model experiment and create data splits.

Create a model run using the SDK

First, follow the instructions in Create a model.

Each model run requires you to specify a list of ground truth label IDs. You can access the ground truth label IDs by exporting labels from your project.

# scripts omitted for creating a model lb_model
lb_model_run = lb_model.create_model_run("model interation 1")

# Add ground truth labels (list of label IDs) to model run. Example: ['cl2mebix4ci3y10ad96kgfnpg', 'cl2mebj9ici4010ad8frvfh5j']
lb_model_run.upsert_labels(ground_truth_labels)

client = labelbox.Client(LB_API_KEY)
project = client.get_project(LABELBOX_PROJECT_ID)

labels_gen = project.label_generator().as_list()
ground_truth_labels = labels_gen._data

### Prepare list of ground truth label IDs: ['cl2mebix4ci3y10ad96kgfnpg', 'cl2mebj9ici4010ad8frvfh5j']
ground_truth_labels_list = [label.uid for label in ground_truth_labels]

run_name = '{}.{}'.format(0,1)
model_name = "My model"

model = client.create_model(name = model_name, ontology_id = project.ontology().uid)
model_run = model.create_model_run(run_name)
model_run.upsert_labels(ground_truth_labels_list)

Performance and limitations

A model run can contain up to 1 million data rows.