Rework with labels

📘

Moving to Workflow

In order to provide a less involved process of re-working labels during QA, the Workflow (beta) feature will automatically place rejected data rows in a specialized Rework task (queue). Learn more here.

Re-working labels are one of the best ways to keep your labeling costs low while radically improving its quality. A lot of customers come to Labelbox after experiencing pure outsourcing services, and they are delighted by experiencing the ability to fully control the label lifecycle in Labelbox. Rework with labels can be perhaps best understood with an analogy: imagine, if you had to ask your software programmers to write software from scratch every time. Clearly, we don't do that. Even when software is outsourced, it is done so using git-based collaborative cloud platforms.

Below are a few common ways you can rework with Labels.

Process of re-queueing assets

When you delete a label on an asset, Labelbox automatically re-enters that asset into the labeling queue to get relabeled. You can choose to use the existing label as a template for the future labeler.

Re-queueing through Data rows tab

Follow these steps to relabel an asset:

  1. Go to the Data rows tab.

  2. Select the data row(s) you wish to relabel. This will enable a button on the top right to say N selected where N denotes the number of data rows that have been selected.

  3. Click the highlighted button to show the dropdown and select Delete and requeue from the options. This will prompt you to confirm whether to use deleted labels as templates.

    • If you select the option of using a template and the dataset or batch is still attached to the project, the asset along with its label template will be added back to the label queue.
    • If the template option isn't selected, only the asset will be added to the label queue.
  4. Click Start labeling to load the asset in the Editor and label it again.

Re-queueing on the Data rows tabRe-queueing on the Data rows tab

Re-queueing on the Data rows tab

Re-queueing through Labels tab

Follow these steps to relabel an asset:

  1. Go to the Labels tab.

  2. Select the data row you wish to relabel and click Delete labels.

  3. Check the box to use the deleted label as a template (starting point) the next time the asset is loaded into the Editor. As long as your dataset is attached to your project, the asset along with its label template will be added back to the label queue. If your dataset is not attached to your project, re-enqueued assets and label templates will not be available until the dataset is reattached to the project.

  4. Click Start labeling to load the asset in the Editor and label it again.

Filter, inspect and create an issue for resolution

Driven by any hypothesis about the data, you can filter labels or data rows and create an issue on the label to be resolved.


Did this page help you?