The labeling queue is the engine that feeds data to your labeling team. It determines which data assets are worked on and in what order. Understanding how to manage this queue is essential for running an efficient labeling operation, ensuring that your most critical data is prioritized, and preventing bottlenecks that can slow down your project.
By default, the queue operates on a First-In, First-Out (FIFO) basis—the first data rows you add to the project will be the first ones presented to your labelers. However, Labelbox provides you with powerful tools to override this default behavior and take precise control over the flow of work.
This guide covers the three core pillars of queue management:
- Setting data row priority to control the order of labeling.
- Managing reservations to handle in-progress work.
- Re-queuing data to send assets back for rework.
Data row priority
The most direct way to manage your queue is by assigning a priority level to your data rows. This allows you to push urgent or important data to the front of the queue, ensuring it gets labeled next.
Priority is set on a scale from 1 (highest) to 5 (lowest). A data row with a priority of 1 will be labeled before any data row with a priority of 2, and so on.
There are three ways you can control data row priority:
- At batch creation: You’ll be prompted to set the priority when you create a batch to be sent to a project. The default priority level is 5.
- In the Data Rows tab: Open a project and go to the Data Rows tab to select a set of data rows and update priority.
- Using the Python SDK: If you need more granular control (i.e., non-integer values) when setting the priority, reference our batches developer guide.
How to update data row priority
You can update the priority on one or more data rows at once using the Data Rows tab.
- Navigate to the Data Rows tab in your project.
- Use the filters on the left to select the data you want to prioritize. For example, you might filter by metadata, batch name, or upload date.
- Select the data rows using the checkboxes. You can select individual rows or click the top checkbox to select all rows that match your filter.
- A bulk actions menu will appear at the top of the table.
- Choose the desired priority level (1-5) from the dropdown menu. Select Change priority. You can adjust it later via the SDK (see the Batches developer guide).
- Click Confirm. A notification banner acknowledges the changes and includes a link to the notification center, where you can follow progress.
The queue will immediately re-order itself. The next labeler to request a task will receive the highest-priority data row that is not currently reserved.
Reservations
When a labeler begins working on a data row, Labelbox automatically creates a reservation. This “locks” the asset to that specific labeler, preventing anyone else from working on it at the same time and avoiding duplicated effort.
To easily view the active reservations per team member, navigate to Project → Settings → Queue.
How to update reservation count and duration
By default, once a labeler starts labeling, Labelbox reserves 5 data rows for the labeler, and these reservations expire after 4 hours of idle time. The labeling queue replenishes reservations for labelers once they finish labeling a reserved data row.
Once a reservation expires, the associated data rows become available to other labelers, but the reservations still show in the original project settings queue until other labelers claim the data rows.
To update reservation settings:
- Go to the project in Annotate.
- Go to Settings → Queue.
- To update the number of data rows that can be reserved by a single user, update the slider for Reservation count. Value range is 1 - 5.
- To update the number of hours that a data row can be reserved to a single user, update the slider for Reservation duration (hours). Value range is 1 - 24.
Labelers who pause their work on a data row may lose unsubmitted work if they exceed their data row reservation time out.
How to clear reservations
- Go to the project in Annotate.
- Go to Settings → Queue. There you will see the reservation queue: which Data Rows are reserved, by which user, and which are only queued.
- Filter by the Queue state.
- Select Clear all reservations to drop all current holds and re-serve rows based on current priorities.
Clearing reservations can impact unsubmitted work, depending on the situation:
- During active labeling: Labelers may lose any unsubmitted work when clearing the reservation, unless they later take the same asset, which is uncommon and not guaranteed.
- For expired reservations: Clearing expired reservations doesn’t affect the availability of the data row, but the row must be claimed before the queue updates.
Real-time labeling
When you enable real-time labeling, it allows your team to label continuously. This can be helpful if you have labeling time constraints or if you need to improve your labeling team’s efficiency by enabling a continuous flow in the labeling queue.
To enable real-time labeling
- Select a project and go to Settings → Automation.
- Toggle on Real-time labeling.
- When all assets in the queue have been labeled, the labeler will see a button that says Wait for next asset. When a labeler clicks on Wait for next asset, Labelbox will display a real-time screen that will automatically display the next task that enters the queue.
Best practices
- To avoid lost work, Labelbox recommends that labelers consult their administrator if they anticipate being away from their computer for over 24 hours. Any unsubmitted work done by the original labeler will be lost if you clear the reservations.
- The timer starts counting label time after labelers click Edit. It stops counting review and label times if our system detects 5 minutes of inactivity. When we detect user activity again, we restart the timer. This is something Labelbox does intentionally to record only active screen time on the application.
- All labeling work should be limited to one browser tab, and no mobile devices should be used. Duplicate labels may be unintentionally created when a user has multiple tabs open and is actively labeling.
