Opening the data rows activity

You can navigate to the Data row Activity table by going to the Data rows tab in the project page. In the table you will see all the data rows either labeled or queued in the project listed. Each data row will have some information such as status (see below for more details), labeling time, review time, dataset and if there any open issues.

Status definitions

Status	Definition
Queued	Data row is currently queued for labeling.
Submitted	A label has been submitted for the data row but has not yet been reviewed (Approved or Rejected).
Skipped	The data row was skipped during labeling.
Approved	The last review on the label is a thumbs up or approve. Benchmark data rows will always have a status of Approved.
Rejected	The last review on the label is a thumbs down or reject.
Ongoing	This is a special status only shown when Consensus is used. This status means that not all the consensus submissions have been submitted yet.

Gallery view

If you want to view what the data rows look like, select the gallery view option in the top right corner of the table. This will take you to a preview mode where we will show a thumbnail of the data row.

For images, we also will render any bounding box annotations in the preview.

Filtering the data

You can filter the list of data rows to the relevant context by applying a filter. To do so, select the filter icon in the top left hand corner of the table. A new filter pane will appear where you can apply the relevant filters for your query.

You can filter your data by the following fields:

Annotations
Dataset
External ID / Asset ID
Label ID
Labeler
Reviewer
Metadata
Labeled Date
Reviewed Date
Functions

Bulk approving or rejecting labels

When reviewing labeled data rows in this table, you can also select multiple labels for bulk approval or bulk rejection. This can be done in either the gallery or list view of the table, though most commonly will be done from the gallery view.

In the example below, you can see how the user has selected several data rows for images of dogs that have been incorrectly annotated with cat that need to be rejected in bulk.

Hiding sensitive labels from labelers

If during the review process you realize that a specific data row contains sensitive or inappropriate content that you want to hide from labelers, you can select that data row and mark it as "Hide from labelers".

This action will make it such that all labelers in the project will be unable to view the data row in the editor from that point on. When they try to access the image, they will instead see an "unauthorized to view this data row" message.

Viewing and editing labels in the data row browser

You can view more details for a labeled data row by clicking on it in the Activity Table to open up the Data Row Browser. In this view, you will be able to perform the following actions:

Update the status of a labeled data row to Approve or Reject using the buttons on the top
To edit the annotations, you can hit the "edit" button in the top right corner to enter an editing mode for changes you want to apply. This option will prevent unintended edits when in the data row browser.
To view all the labels for Benchmark or Consensus data row, you must hit the ">" option in the left side panel for the data row in question.