Labelbox uses a queue system to organize and distribute data to labelers and reviewers in your project. Each project has one Label queue and one optional Review queue.
Contains unlabeled assets
Contains submitted and skipped labels
Queue customization supported
Queue customization not supported
Assets are reserved upon distribution to labelers
Labeled assets are not reserved upon distribution to reviewers
View enqueued assets under Labels tab > Queue
No way to view enqueued labeled assets in the Labelbox UI
Automatically enabled for all projects
Must be manually enabled in the project settings
Labelbox distributes assets from the Label queue in batches to each active individual labeler. When a batch of assets is distributed to an active labeler, Labelbox “checks out” those assets to that labeler. Once the individual’s queue of reserved assets empties, Labelbox automatically fills it with a new batch of unlabeled assets.
If a user starts labeling an asset, then is idle for more than 60 minutes, Labelbox will automatically log the user out and assign the asset to another active labeler. Any unsubmitted work done by the original labeler will be lost.
To avoid lost work, we recommend that labelers consult their administrator if they anticipate being away from their computer for longer than 60 minutes. Work should also be limited to one browser tab; use of multiple tabs should be avoided.
You can view your queued assets, select a project, go to the Labels tab, and click on the Queue sub-tab.
Batch queue mode allows you to use Catalog to send Data Rows to Projects for labeling. You can use this feature to have more control over the data you queue for labeling.
By default, each asset in the Label queue will be labeled once. However, if you need a specific asset to be labeled more than once, you can use the GraphQL API to target an individual asset and specify the number of times it should get labeled. Labelbox will automatically re-enter that asset into the Label queue and redistribute it to active labelers until it has received the number of Labels specified.
If you need your assets to appear in the queue in a certain order, you can assign them individual priority scores. The order of non-prioritized assets in the queue is not guaranteed.
In this prioritization example, asset E will be labeled first, A will be second, and D will be third.
If you skip numbers when you set the prioritization, the Label queue will default to the next asset in the priority order. In this example, asset E will be labeled first, A will be second, and D will be third.
When you assign multiple assets the same priority number without rebuilding the queue, they will be ordered lexicographically. In this example, asset B will be labeled first, E will be second, A will be third, and D will be fourth.
Updated about 1 month ago