Move to workflows, data rows tab, and batches
A guide for switching to workflows, data rows tab, and batch-based queueing.
Over the past few months, we’ve released workflows on a rolling basis to help teams create highly customizable quality and review processes based on project needs. You can use workflows to create rule-based review tasks and multi-step sequences to reduce costs and increase the quality and efficiency of your labeling operations.
Releasing workflows also meant that all new projects created were automatically configured with batch-based queueing and the Data Rows tab, representing a more efficient way to queue and review your data. We will soon be sunsetting dataset-based queueing, the Labels tab, and the Review step in favor of this new way to queue and review your data.
Watch these videos to see how Workflow works.
Batches + data rows tab + workflows
This video explains how workflows, batches, and the data row tab work together.
Rollout plan: batches + data rows tab + workflows
New projects
We have now rolled out the batches, Data Rows tab, and workflows experience for all customers. You should now be able to create new projects that are automatically configured with batches, the Data Rows tab, and workflows.
What happens to old projects created before workflows?
We will automatically migrate all legacy / old projects that were created before we introduced workflows. These projects are distinguished by the ‘legacy’ tag in Annotate.
Labelbox will automatically be migrating your old / existing projects to the new paradigm (batch-based queuing, Data Rows tab, and workflows). This will happen on Labelbox’s backend and no action is required on your end.
You will receive an email specifying your migration date. We will be sending out expected migration dates during the week of March 6th, 2023 – please check your email to see when your expected migration email will take place.
After the migration date specified in the email, your legacy projects will no longer have access to:
- Dataset-queue mode
- Replaced by batch-based queueing
- Labels tab
- Replaced by the Data Rows tab
- Review step
- Replaced by workflows
- Project-level consensus
- Replaced by consensus at the batch-level
In the meantime, please refer to this page to learn more about the logistics and timing of the upcoming migration.
Comparison: Old paradigm vs new paradigm
Below are the advantages of the new Workflow + Data Rows tab + Batches paradigm.
| Function | Labels tab + review step + dataset-based queueing | Data rows tab + workflows + batches |
|---|---|---|
| Multi-step review | No | Up to 10 review steps per workflow |
| Customizable review step | No | All 10 steps in the workflow are customizable |
| Review history | No | Audit log shows all actions on a Data Row |
| Bulk actions | Yes | Yes |
| Ad-hoc review | Yes | Yes |
| Voting | Thumbs up/thumbs down | Approve/reject |
| Filters | Limited/manual | Automated quick filters by data row status (unlabeled, in review, in rework, etc) |
| Rework labels | Manually delete & requeue | All rejected data rows are automatically sent to Rework task in Workflow |
| Re-review | Limited | Select data rows and click Move to task |
Limitations
- The current limitation for bulk actions with workflows is 10K data rows, including Move to step.
- Currently, only new label imports are supported.
How to submit feedback
If you would like to submit feedback about workflow, data rows tab, or batches, please use this feedback form. Our product team reviews this feedback regularly.
FAQ
Why are we making this change?
Having worked with hundreds of AI teams, we recognized the need for more granular control over labeling workflows. In order to streamline and improve the creation, maintenance, and quality control of data rows, we’re introducing a new way for teams to queue and review.
How do these changes affect me?
Rather than queueing an entire dataset, we strongly encourage batch-based queueing for more flexibility and control over your workflow. With batches, you can:
- Prioritize slices of data by adding batches to a project in priority
- Manage batches & view batch history
- Enable active learning workflows to identify the most high-impact data rows for labeling
To learn more about the migration process and other FAQs, visit this page.
Python SDK
See our reference to learn how to set the queueing mode when setting up a project.
Updated 4 months ago