Add model run config to track hyperparameters

Overview

You can use model run config to create, version, and track your hyperparameters and any training-related configurations for a model run.

It is common for ML teams to kick off multiple model runs with different sets of hyperparameters and compare their performance. Therefore, having the model configs logged and their final performance compared is crucial to troubleshoot a model, share best practices or reproduce model results.

Create and modify model run config in the UI

In a model run page, click on the Settings icon, select Model run config.
If you have not created a model run config for this model run before, there will be a pre-populated template, with common hyperparameters and their default values of None.

You can modify the values (must be booleans, strings, or numbers) and save them. The format of model run config must be a valid json.

You can load a previous model run's config by using the slider user another model run's config as a template and modify from it as template.

30103010

You can also click the trash icon to reset all content, the copy icon to copy the content of the model run config in your clipboard.

To export and download the model run config, you can click the download icon.

Compare model run configs between model runs

In model comparison mode, you can compare the model performances as well as the model run configs to understand how are hyperparameters affecting your model.

30223022

Create and modify model run config in SDK

You can create the model run with the config specified.

example_config = {
    "learning_rate": 0.001, 
    "checkpoint_path": "/path/to/checkpoint/file",
    "early_stopping": False,
    "batch_size": 32, 
    "optimizer": { 
      "adam": {
        "beta1": 0.899999976158,
        "beta2": 0.999000012875,
        "epsilon": 9.99999993923e-9
      }
    }, 
    "ngpu": 1, 
    }

model = client.get_model("<MODEL_ID>")
# create a model with the config 
model_run = model.create_model_run(name="run 1", config=example_config)

# alternatively, create a model run first and update the model config field.
model_run = model.create_model_run(name="run 2")
model_run.update_config(example_config)

Get model run config

model_run_parameters = model_run.get_config()

Update the model run config

The update will repace the previous model run config with the new json input.

model_run = client.get_model_run("<MODEL_RUN_ID>")
model_run.update_config({"batch_size": 16})

Delete the model run config

model_run.reset_config()

Did this page help you?