How the labeling queue and reservations works

Learn how the label queue and reservations work.

When a dataset or batch is added to a project, Labelbox will enqueue the data rows in that dataset or batch to the labeling queue. Once the data rows enter the labeling queue, Labelbox distributes them to each active labeler. When a labelers begins labeling, Labelbox “reserves” several data rows for that labeler—meaning those assets cannot be labeled by any other team member. This is to ensure that we do not have contention resulting in duplicate labeling and we can pre-load assets for a faster labeling experience.

940940

To view your assets that are queued for labeling, go to the data row tab in your project and click on the "To Label" filter as seen below. In this tab, you can easily explore your queued data to see what type of data is still in the labeling queue.

12421242

Reservations

Once a labeler starts labeling, Labelbox will reserve a certain number of assets for the labeler, depending on the Quality setting for the project. The number of assets reserved will be 10 for projects that use Consensus and 3 for projects that don't use Consensus. As they work down their reservations, our queue will continue to replenish their reservations from the queue. This ensures that the labeler will have sufficient work to avoid unintended downtime due to latency spikes or changes in the label queue.

Labelbox will keep reservations active for 90 minutes. This is renewed every 10 minutes while a user has the asset open in their tab. After 90 minutes of idle time, the reservations will be cleared for other labelers to pick up.

To easily view the active reservations, please navigate to Project > Settings > Queue

24722472

Best practices

  • To avoid lost work, we recommend that labelers consult their administrator if they anticipate being away from their computer for longer than 90 minutes. Any unsubmitted work done by the original labeler will be lost.
  • Work should also be limited to one browser tab. Duplicate labels may be unintentionally created when the user has multiple tabs open and is actively labeling.

Manage the queue via the Python SDK

Python Tutorial

Github

Google Colab

Queue management

Open in Github

Open in Google Colab


Did this page help you?