Queue customization

Python Tutorial

Github

Google Colab

Queue management

Open in Github

Open in Google Colab

Labelbox uses a queue system to organize and distribute data to labelers and reviewers in your project. Each project has one Label queue and one optional Review queue.

How the label queue works

Label queue

Review queue

Contains unlabeled assets

Contains submitted and skipped labels

Queue customization supported

Queue customization not supported

Assets are reserved upon distribution to labelers

Labeled assets are not reserved upon distribution to reviewers

View enqueued assets under Labels tab > Queue

No way to view enqueued labeled assets in the Labelbox UI

Automatically enabled for all projects

Must be manually enabled in the project settings

Labelbox distributes assets from the Label queue in batches to each active individual labeler. When a batch of assets is distributed to an active labeler, Labelbox “checks out” those assets to that labeler. Once the individual’s queue of reserved assets empties, Labelbox automatically fills it with a new batch of unlabeled assets.

If a user starts labeling an asset, then is idle for more than 60 minutes, Labelbox will automatically log the user out and assign the asset to another active labeler. Any unsubmitted work done by the original labeler will be lost.

To avoid lost work, we recommend that labelers consult their administrator if they anticipate being away from their computer for longer than 60 minutes. Work should also be limited to one browser tab; use of multiple tabs should be avoided.

You can view your queued assets, select a project, go to the Labels tab, and click on the Queue sub-tab.

Batch queue

Batch queue mode allows you to use Catalog to send Data Rows to Projects for labeling. You can use this feature to have more control over the data you queue for labeling.

Labeling parameters (LPO)

Number of Labels

By default, each asset in the Label queue will be labeled once. However, if you need a specific asset to be labeled more than once, you can use the GraphQL API to target an individual asset and specify the number of times it should get labeled. Labelbox will automatically re-enter that asset into the Label queue and redistribute it to active labelers until it has received the number of Labels specified.

Priority

If you need your assets to appear in the queue in a certain order, you can assign them individual priority scores. The order of non-prioritized assets in the queue is not guaranteed.

In this prioritization example, asset E will be labeled first, A will be second, and D will be third.

If you skip numbers when you set the prioritization, the Label queue will default to the next asset in the priority order. In this example, asset E will be labeled first, A will be second, and D will be third.

When you assign multiple assets the same priority number without rebuilding the queue, they will be ordered lexicographically. In this example, asset B will be labeled first, E will be second, A will be third, and D will be fourth.


Did this page help you?