Reservations

When you send a batch to a project, Labelbox will enter those data rows into the project's labeling queue. When a batch of data rows enters a project's labeling queue, Labelbox automatically distributes the data rows among each active labeler.

Labelbox “reserves” several data rows for that labeler—meaning no other team member can label an asset that is reserved by another team member. This is to ensure that we do not have contention resulting in duplicate labeling and we can pre-load assets for a faster labeling experience.

To view your assets that are queued for labeling, go to the data row tab in your project and click on the To label filter as seen below. In this tab, you can easily explore your queued data to see what remains in the labeling queue.

Once a labeler starts labeling, Labelbox will reserve a certain number of assets for the labeler, depending on the quality setting for the project. The number of assets reserved will be 10 for projects that use consensus and 3 for projects that don't use consensus. As they work down their reservations, our queue will continue to replenish their reservations from the queue. This ensures that the labeler will have sufficient work to avoid unintended downtime due to latency spikes or changes in the label queue.

Labelbox will keep reservations active for 90 minutes. This is renewed every 10 minutes while a user has the asset open in their tab. After 90 minutes of idle time, the reservations will be cleared for other labelers to pick up.

To easily view the active reservations, please navigate to Project > Settings > Queue.

Clearing Reservations

You can manually clear out reservations for a user by navigating to Project > Settings > Queue. Find the user in the queue that you want to clear reservations for, click the 3 dots icon, and select Clear all reservations for this user.

Note: this will clear all reservations across all projects for this user.

Best practices

• To avoid lost work, we recommend that labelers consult their administrator if they anticipate being away from their computer for longer than 90 minutes. Any unsubmitted work done by the original labeler will be lost.
• Our timer will stop counting review time and label time if our system detects 5 minutes of inactivity. When we detect user activity again, we start the timer back up. This is something Labelbox does intentionally to record only active screen time on the application.
• Work should also be limited to one browser tab. Duplicate labels may be unintentionally created when the user has multiple tabs open and is actively labeling.