Model Run

A Model Run is a model training experiment within a Model directory. Each Model Run has its data snapshot (Data Rows, annotations, and data splits) versioned. You can upload predictions to a Model Run, and compare its performance against other Model Runs in the Model directory.

Get all Model Runs inside a Model

model_runs = model.model_runs()

Create a Model Run

Creates a model run belonging to this model.

model_run_name = "<your_model_run_name>"
example_config = {
    "learning_rate": 0.001, 
    "batch_size": 32, 
}
model_run = model.create_model_run(name=model_run_name, config=example_config)

Get Model Run

model_run_id = "<your_model_run_id>"
model_run = client.get_model_run(model_run_id=get_model_run)

model_run_data = model_run.model_run_data_rows()
model_run_config = model_run.get_config())

Add data rows to a Model Run and assign split

Add data rows to a model run without any associated labels, and assign them to a split -- one of "TRAINING", "VALIDATION", "TEST".

Note that assign_data_rows_to_split only works on data rows or labels that are already in a model run.

client.enable_experimental=True

dataset = None # Your training dataset 
data_row_ids = [data_row.uid for data_row in dataset.export_data_rows()]

model_run.upsert_data_rows(data_row_ids)
model_run.assign_data_rows_to_split(
  data_row_ids=data_row_ids,
  split="TRAINING",
)

# Turn on the experimental mode of the SDK
client.enable_experimental=True

# For new data rows
split="TRAINING" # should be one of "TRAINING", "VALIDATION", "TEST", "UNASSIGNED"
model_run_data_rows_ids = ["<data_row_id_1>","<data_row_id_2>", ...]
model_run.upsert_data_rows(data_row_ids)

# assign_data_rows_to_split only works 
model_run.assign_data_rows_to_split(
  data_row_ids=data_row_ids,
  split=split
)

Add labels to a Model Run

Adds data rows and labels to a model run.

label_ids = ["<label_id_1>","<label_id_2>", ...]
model_run.upsert_labels(label_ids)

Export labels from a Model Run

# Turn on the experimental mode of the SDK
client.enable_experimental=True

# If download=False, this returns the URLs of the data files associated with this ModelRun’s labels.
download = False
model_run.export_labels(download=download)
# If download=True, this instead returns the contents as NDJSON format.
download = True
model_run.export_labels(download=download)

Delete data rows from a Model Run

data_row_ids = ["<data_row_id_1>","<data_row_id_2>", ...]
model_run.delete_model_run_data_rows(data_row_ids=data_row_ids)

Create, modify, and delete model run config to track your hyperparameters.

example_config = {
    "learning_rate": 0.001, 
    "checkpoint_path": "/path/to/checkpoint/file",
    "early_stopping": False,
    "batch_size": 32, 
    "optimizer": { 
      "adam": {
        "beta1": 0.899999976158,
        "beta2": 0.999000012875,
        "epsilon": 9.99999993923e-9
      }
    }, 
    "ngpu": 1, 
    }

model_run_1 = model.create_model_run(name="run 1", config=example_config)
# You can also create a model with config specified, see above.
# Here is how to create a model run first and update the model config field.
model_run_2 = model.create_model_run(name="run 2")
#The update will repace the previous model run config with the new json input.
model_run_2.update_config(example_config)

Get model run config

model_run_parameters = model_run.get_config()

Delete the model run config

model_run.reset_config()

Delete Model Run

model_run.delete()