Task distribution

Learn how the label queue and reservations work.

Overview

Once data rows are attached to a project, they will be queued by Labelbox for labeling. During labeling, Labelbox will distribute data rows from the queue in batches to each active individual labeler. When data rows are distributed to an active labeler, Labelbox “reserves” those assets to that labeler so that they cannot be labeled by any other team member.

When a labeler starts an active labeling session, Labelbox will fetch two data rows from the labeler's reserved assets to the browser to label: one will be served up immediately to the editor while the second will be cached in memory for a smoother experience after the first data row is submitted. Our system will continue to pre-fetch the next queued asset during an active labeling session.

Once data rows have been labeled, they are removed from the queue and can be found in the Label Activity Table

Reservations

Once a labeler starts labeling, our queue will reserve up to 10 assets for the labeler. As they work down their reservations, our queue will continue to replenish their reservations from the queue. This ensures that the labeler will have sufficient work to avoid unintended downtime due to latency spikes or changes in the label queue.

Labelbox will keep reservations active for 90 minutes after a user goes idle, after which we will clear out all the reservations for other labelers to pick up.

To avoid lost work, we recommend that labelers consult their administrator if they anticipate being away from their computer for longer than 90 minutes. Any unsubmitted work done by the original labeler will be lost. Work should also be limited to one browser tab. Duplicate labels may be unintentionally created when the user has multiple tabs open.

📘

If you have a project that needs a lower reservation count than 10, please reach out to your account manager.

Labelbox uses a queue system to organize and distribute data to labelers and reviewers in your project. Each project has one Label queue and one optional Review queue.

Labeling Parameter Overrides (LPO)

Number of Labels

By default, each asset in the Label queue will be labeled once. However, if you need a specific asset to be labeled more than once, you can use the GraphQL API to target an individual asset and specify the number of times it should get labeled. Labelbox will automatically re-enter that asset into the Label queue and redistribute it to active labelers until it has received the number of Labels specified.

Priority

If you need your assets to appear in the queue in a certain order, you can assign them individual priority scores. The order of non-prioritized assets in the queue is not guaranteed.

In this prioritization example, asset E will be labeled first, A will be second, and D will be third.

If you skip numbers when you set the prioritization, the Label queue will default to the next asset in the priority order. In this example, asset E will be labeled first, A will be second, and D will be third.

When you assign multiple assets the same priority number without rebuilding the queue, they will be ordered lexicographically. In this example, asset B will be labeled first, E will be second, A will be third, and D will be fourth.

Complete Python SDK tutorial

Python Tutorial

Github

Google Colab

Queue management

Open in Github

Open in Google Colab


Did this page help you?