Migrating to workflows

What are workflows?

Many AI labeling teams struggle to prioritize the right data to label and end up spending more money on data labeling than they should. In the past few months, we released two features – the data rows tab and batches – to help teams better navigate and queue data for labeling.

Now, we’re excited to introduce a third feature called workflows. Workflows give you more granular control over how your Data Rows are reviewed. You can use workflows to create rule-based review tasks and multi-step sequences to reduce costs and increase the quality and efficiency of your labeling operations.

Watch these videos to see how it works.

Batches + data rows tab + workflows

This video explains how workflows, batches, and the data row tab work together.

Rollout plan: batches + data rows tab + workflows

New projects

Over the next few months, Labelbox will automatically configure all new projects with batches + data rows tab + workflows for new projects. These changes will happen on a rolling basis.

The table below indicates the cutoff date after which all new projects will be automatically configured with batches + data row tab + workflows.

Old projects

The table below indicates the planned migration deadlines for all old projects to be automatically migrated to use Batches + Data Row tab + Workflows.

📘

Migrating old projects

We will begin migrating old projects to the new batches + data row tab + workflow paradigm starting on 3/31/23. Soon, we will be sharing a migration schedule for each customer tier. For now, no action is required for your old projects.

Features being replaced

For clarity, here is a table that states which features are being replaced by newer, better features.

Comparison: Old paradigm vs new paradigm

Below are the advantages of the new Workflow + Data Rows tab + Batches paradigm.

Limitations

• The current limitation for workflows is 10K data rows. Coming soon: 25K data row limit.
• Currently, only new label imports are supported.

How to submit feedback

If you would like to submit feedback about workflow, data rows tab, or batches, please use this feedback form. Our product team reviews this feedback regularly.

FAQ

Why are we making this change?

Having worked with hundreds of AI teams, we recognized the need for more granular control over labeling workflows. In order to streamline and improve the creation, maintenance, and quality control of data rows, we’re introducing a new way for teams to queue and review.

How do these changes affect me?

Rather than queueing an entire dataset, we strongly encourage batch-based queueing for more flexibility and control over your workflow. With batches, you can:

• Prioritize slices of data by adding batches to a project in priority
• Manage batches & view batch history
• Enable active learning workflows to identify the most high-impact data rows for labeling

Changes for Free/EDU only (before 11/21)

🚧

Free/EDU only

Our Free and EDU customers received an early version of the batches + data row tab + workflow paradigm. These changes only apply to you if you are a free or EDU customer. And these changes outlined from here to the end of this page will be superseded by the changes outlined in the sections above.

If you’re creating a project through the UI

1. You will be prompted to select a quality setting (benchmark or consensus) that will determine your project’s queueing mode.
2. You cannot change the quality setting or queueing mode after a project is created.
3. Benchmark projects will default to batch-based queuing.
4. Consensus projects will default to dataset-based queueing.

📘

Benchmarks projects

Workflows are available for benchmark projects only (consensus not yet supported). New benchmark projects will have an initial review task automatically set up. For existing projects, please go into the Workflow tab and create a new review task by clicking the New Task button.

If you’re creating a project through the SDK

1. When creating a project, you must specify a queueing mode and quality mode (the backend will no longer infer defaults since these values cannot be changed after a project has been created). Learn more here on how to set up your queueing and quality modes.
   a. Projects configured with benchmarks will default to batch queuing mode.
   b. Projects configured with consensus will default to dataset-queueing mode.
   c. You cannot update the quality setting or queueing mode after a project has been created.
   d. Datasets cannot be attached to batch projects.
   e. Batches cannot be attached to dataset projects.
2. All new projects will require media type upon creation. Learn how to set media type here.
3. Batches have new arguments for configuring quality settings.

Queue mode / quality settings combinations

Use the table below to understand the supported combinations and default configurations available when configuring the queue mode and quality mode for your project.

📘

Python SDK

See our reference to learn how to set the queueing mode when setting up a project.