Review step

📘

Workflows is replacing the review step

The review step is a legacy feature that is being replaced by workflows. We are rolling out workflows for customers on a rolling basis. To learn more about migrating to workflows, visit this migration guide.

Labelbox has a separate interface and queueing system for reviewing Labels. When you set up your team in Labelbox, you have the option to designate members of your team as Reviewers. Reviewers may approve or deny a Label, open an issue on a Label, and make modifications to the Label. See permission settings for more details.

You can configure how many Data Rows get sent to the review queue by specifying the percentage of Data Rows in the Review step. The review queue abides by the following rules:

• Only submitted or skipped assets can be entered into the review queue.

• With additional configuration, an asset in the review queue may be reviewed by more than one user, but never twice by the same user.

Members with the Reviewer role will see these assets in a special review interface. Reviewers are able to modify the annotations and submit their reviews by approving or rejecting the labeled assets.

How to enable the Review step

Enable the Review step by doing the following:

1. Select a project.

2. Go to the Settings tab.

3. Click Quality.

4. Toggle on the Review step.

5. Set the Coverage and click Confirm changes.

6. Go to the Labels tab.

7. Click Review labels to enter review mode.

How to view Reviews

The Reviews section in the Project overview tab contains a simple metric for understanding the general trends in user reviews.

📘

Note

Labels that have received reviews and have an equal count of upvotes and downvotes are currently not filterable.

How are the assets queued for Review Labels

Review queue is built based on first in first out from the submitted labels.