Migration FAQ

To add a batch to a project, am I still required to upload a dataset to Labelbox?
Yes. The concept of uploading a dataset to Labelbox is not going away. After you upload a dataset, the dataset will live in Catalog. From Catalog, you can add data rows to a project via batches. You could send the entire dataset as a batch to a project, if needed. Labelbox is currently working on scaling batches to 100,000 data rows. There is no limit to how many batches you can add to a project.

If I am not planning on using quality settings, which configuration is recommended: benchmarks or consensus?
Labelbox recommends selecting benchmarks (when setting up a project via the SDK, the default configuration, if you don’t pick a quality setting, is batches + benchmarks). However, is there a possibility that you may need to use consensus in the future, then select consensus and set it to 0%.

How do I set consensus at the batch level?
When you add a batch to a project (after you already selected consensus for your project), you will be prompted to configure consensus for that batch.

If you are in a consensus project, you will see 3 configuration settings:

  • A toggle to enable/disable consensus for that batch
  • A slider to set the coverage percentage and
  • A place to enter the number of labels.

Items #2 and #3 in the list above replace the old labeling parameter overrides (LPO) feature as it enables you to customize the assets in the queue at the batch level. In the data row tab, you’ll be able to see the number of existing labels and the number of expected labels for each consensus data row.

🚧

Note

Note: Once you add a batch to a project, you cannot change the number of labels setting. In the future, we may support dynamic consensus settings.

Are the data row statuses visible in the export?
Data row workflow history is shown in the export (the tasks the data row has traversed through). This is shown in reverse chronological order.

Can I use the rework task instead of delete-and-requeue?
Delete and requeue will still be supported for a little while longer. Eventually, rework will replace delete-and-reqeueue when the asynchronous functionality in the rework task is at parity with Delete-and-requeue.

Will I still be able to “save label as template”?
Yes. This same functionality will be provided when you send a data row to the rework task. Labelbox will preserve the existing annotations on the data row so the next person who opens the data row as a rework task will be able to use the existing annotations as a starting point.

I have dataset + batch mode enabled for my project which is causing me to not be able to delete-and-requeue. What should I do?
Create a new project configured with batch mode. Labelbox will be offering a migration path for existing projects. However, this migration path will not be available until the end of Q4 2022.

I want to set up a consensus project, but the consensus + batches configuration is not yet supported. What should I do?
Toward the end of December, we will be releasing a migration path only after batches + consensus configuration is supported. No customers will be expected to migrate until the batches + consensus configuration is supported (coming soon).

Can I enable benchmarks AND consensus on a project?
It is not supported yet, but we will be releasing that functionality soon.

Will I have access to a master record of all actions on a labeled data row, not just createdAt?
We will be expanding on the audit log functionality in early 2023 which should provide a master record of all actions. However, this is not exposed in today’s version. In the future, we may expand the workflow webhooks to include a more comprehensive master record.

What is the migration plan if I have a programmatic integration with datasets?
The end state for every project is the batches + data row tab + workflow paradigm. Whether you set up a project via the SDK or UI, every project will be treated the same.

You can still create a project, upload a dataset, and send a batch for labeling via the SDK.

I don’t have a need for batches or workflows. What do I do?
You can still add your entire dataset to a batch and send the batch to a project for labeling. We will be releasing a Python notebook to demonstrate how to do this soon.