Review step

Labelbox has a separate interface and queueing system for reviewing Labels. When you set up your team in Labelbox, you have the option to designate members of your team as Reviewers. Reviewers may approve or deny a Label, open an issue on a Label, and make modifications to the Label. See permission settings for more details.

You can configure how many Data Rows get sent to the review queue by specifying the percentage of Data Rows in the Review step. The review queue abides by the following rules:

  • Only submitted or skipped assets can be entered into the review queue.

  • With additional configuration, an asset in the review queue may be reviewed by more than one user, but never twice by the same user.

Members with the Reviewer role will see these assets in a special review interface. Reviewers are able to modify the annotations and submit their reviews by approving or rejecting the labeled assets.

How to enable the Review step

Enable the Review step by doing the following:

  1. Select a project.

  2. Go to the Settings tab.

  3. Click Quality.

  4. Toggle on the Review step.

  5. Set the Coverage and click Confirm changes.

  6. Go to the Labels tab.

  7. Click Review labels to enter review mode.

How to view Reviews

The Reviews section in the Project overview tab contains a simple metric for understanding the general trends in user reviews.



Labels that have received reviews and have an equal count of upvotes and downvotes are currently not filterable.



Un-reviewed labels

No reviews at all

Accepted labels

Labels where the majority upvoted

Declined labels

Labels where the majority downvoted

Issue count

Number of issues in the project (see Issues & comments )

Did this page help you?