A Model Run is a model training experiment within a Model directory. Each Model Run has its data snapshot (Data Rows, annotations, and data splits) versioned. You can upload predictions to a Model Run, and compare its performance against other Model Runs in the Model directory.
Get all Model Runs inside a Model
model_runs = model.model_runs()
Create a Model Run
Creates a model run belonging to this model.
model_run_name = "<your_model_run_name>"
example_config = {
"learning_rate": 0.001,
"batch_size": 32,
}
model_run = model.create_model_run(name=model_run_name, config=example_config)
Get Model Run
model_run_id = "<your_model_run_id>"
model_run = client.get_model_run(model_run_id=get_model_run)
model_run_data = model_run.model_run_data_rows()
model_run_config = model_run.get_config())
Add data rows to a Model Run and assign split
Add data rows to a model run without any associated labels, and assign them to a split -- one of "TRAINING", "VALIDATION", "TEST".
Note that assign_data_rows_to_split
only works on data rows or labels that are already in a model run.
client.enable_experimental=True
dataset = None # Your training dataset
data_row_ids = [data_row.uid for data_row in dataset.export_data_rows()]
model_run.upsert_data_rows(data_row_ids)
model_run.assign_data_rows_to_split(
data_row_ids=data_row_ids,
split="TRAINING",
)
# Turn on the experimental mode of the SDK
client.enable_experimental=True
# For new data rows
split="TRAINING" # should be one of "TRAINING", "VALIDATION", "TEST", "UNASSIGNED"
model_run_data_rows_ids = ["<data_row_id_1>","<data_row_id_2>", ...]
model_run.upsert_data_rows(data_row_ids)
# assign_data_rows_to_split only works
model_run.assign_data_rows_to_split(
data_row_ids=data_row_ids,
split=split
)
Add labels to a Model Run
Adds data rows and labels to a model run.
label_ids = ["<label_id_1>","<label_id_2>", ...]
model_run.upsert_labels(label_ids)
Export labels from a Model Run
# Turn on the experimental mode of the SDK
client.enable_experimental=True
# If download=False, this returns the URLs of the data files associated with this ModelRun’s labels.
download = False
model_run.export_labels(download=download)
# If download=True, this instead returns the contents as NDJSON format.
download = True
model_run.export_labels(download=download)
Delete data rows from a Model Run
data_row_ids = ["<data_row_id_1>","<data_row_id_2>", ...]
model_run.delete_model_run_data_rows(data_row_ids=data_row_ids)
Create, modify, and delete model run config to track your hyperparameters.
example_config = {
"learning_rate": 0.001,
"checkpoint_path": "/path/to/checkpoint/file",
"early_stopping": False,
"batch_size": 32,
"optimizer": {
"adam": {
"beta1": 0.899999976158,
"beta2": 0.999000012875,
"epsilon": 9.99999993923e-9
}
},
"ngpu": 1,
}
model_run_1 = model.create_model_run(name="run 1", config=example_config)
# You can also create a model with config specified, see above.
# Here is how to create a model run first and update the model config field.
model_run_2 = model.create_model_run(name="run 2")
#The update will repace the previous model run config with the new json input.
model_run_2.update_config(example_config)
Get model run config
model_run_parameters = model_run.get_config()
Delete the model run config
model_run.reset_config()
Delete Model Run
model_run.delete()