A new view of the Performance tab, the Performance Board, is in beta. It lets you ask questions about your project’s performance in plain language and pin the answers to a personal board. The rest of this page describes the original dashboard, which is still available.
- Individual
- TEQ (Throughput, Efficiency, and Quality)
- Participation
- Instruction Quiz (see Quiz)
To read these numbers from your own tools, see the Performance API.
Performance Board (beta)
The Performance Board is in beta and is available to select workspaces. To request access, contact your Labelbox account team.
How it works
When you turn on the new view, the Performance tab replaces the Individual, TEQ, Participation, and Instruction Quiz sub-tabs with two tabs: Analyze and Performance Board.Analyze
Analyze is a chat window where you ask questions about the project’s throughput, efficiency, quality, and contributors. It has no filter bar. Instead, you describe what you want in the question, including the time range, the people, or the metric.- Suggested questions. Before you start, Analyze shows sample questions chosen for your project. Click one to send it. If a question doesn’t apply to your project, it’s shown greyed out with the reason. For example, consensus questions don’t appear on projects without consensus.
- Finding first. The first sentence of every answer is the finding, with its number. For example, if you ask “What are the bottlenecks in my project?”, the answer might be: “Initial Review is the bottleneck. 3,878 rows are waiting, about eight days of work at the recent review pace.” The conversation stays on the left, and the chart or table appears on the right with a short interpretation. You can switch between the chart and the table.
- Show the work. Every answer includes a panel that lists the metric, the date window and time zone, the scope, the number of members or data rows included, and anything that was excluded and why. It also shows when the data was last updated.
- Check the supporting data. Click Open in Legacy View to open the related legacy charts or tables in a new browser tab. Some analyses don’t have an exact equivalent in the legacy dashboard; the note beside the link explains the differences.
- Activity and Stop. While the agent works, you can expand its activity to see which steps it’s running and how long it has been working. You can stop a run at any time and ask another question in the same conversation.
- Conversation history. Your conversations are saved, so you can reopen a past conversation and continue it. Your conversations are private to you.
Performance Board
The Performance Board is a personal board of answers you want to revisit. A new board starts with a few default cards, and you can add your own by pinning answers from Analyze.- Cards keep the question, not a snapshot. Each card re-asks its question when the board refreshes. Its date window is relative, such as “last 14 days,” and moves forward to the current date on each refresh.
- Refreshing. The board refreshes when you open it, if the last refresh was more than an hour ago. You can also click Refresh now. Each person can refresh a project’s board up to 4 times per day (UTC). The board shows when it last refreshed and how many refreshes are left.
- Change notices. When a card’s finding changes in a meaningful way, such as a trend reversing or a different bottleneck, the card tells you what changed. Click Compare before and after to see the before and after.
- Managing cards. You can rename, reorder, and remove cards, and switch each card between chart and table views. Open a card pinned from a conversation to return to that conversation.
Data freshness
Most performance metrics update hourly. Live workflow counts, where available, are labeled separately. Every answer and card states when its data was last updated.How it differs from the legacy Performance tab
Get started
1
Open the Performance tab
Open your project and select the Performance tab.
2
Turn on the new view
Click Try out — new Performance Board. Labelbox remembers this choice for you on this project. Turning it on doesn’t change the view for anyone else.
3
Ask a question
On the Analyze tab, click a suggested question or type your own, then click Send. The agent checks the project context and runs the analysis. The first answer in a new conversation can take a couple of minutes while the agent starts. Follow-up questions are faster.
4
Review the answer
Read the finding in the conversation, then review the chart or table on the right. Open Show the work to see what the answer covers, or click Open in Legacy View to check the supporting data.
5
Pin the answer to your board
Click Add to Performance Board. Enter a name (by default, the question you asked), choose whether the card shows the chart or the table, and confirm. You’re saving the question, so the card can show a new finding as the project progresses.
6
Check your board
Select the Performance Board tab to see your cards. Open the board whenever you want to check in, or click Refresh now to update it.
Example questions
- What are the bottlenecks in my project?
- How does my output this week compare with last week?
- How has throughput changed over the last 14 days?
- Who are the quality outliers?
- Who agrees least with consensus?
- What is the approval rate by reviewer?
Permissions
The new view uses the same permissions as the legacy Performance tab. If you can’t see the Performance tab today, you can’t see it in either view.
On projects shared with another workspace, members of the guest workspace can use the new view, subject to the same project permissions.
Usage limits
The following limits apply per person unless noted. Daily limits reset at midnight UTC.
Chat and board refreshes share the uncached analysis limits. An uncached analysis reads the underlying data instead of reusing a cached result; cache hits don’t consume these limits.
Throughput, efficiency, and quality (TEQ)
This section provides three separate views to help you understand the performance of your labeling operation: Throughput, Efficiency, and Quality.Throughput view
The throughput view provides insight into the amount of labeling work being produced and helps you answer questions like:- How many assets were labeled in the last 30 days?
- What is the review time and rework time for labeled assets?
- What is the average amount of labeling work being produced?
Forecast
The forecast feature shows your average daily throughput and completed work (data rows, labels, reviews, annotations) and uses that to forecast completion dates. To see this, enable the Forecast toggle.Average daily throughput
The average daily throughput is calculated by dividing the total units completed in the selected date range by the number of calendar days in that range. Where:- Total units completed = sum of the relevant throughput metric over the selected period (e.g., total labels created, total reviews, etc)
- Number of calendar days = inclusive of days between start and end date (e.g., Jan 1–Jan 7 → 7 days). This aligns the “average per day” with how the forecast projects completion over calendar time.
- For Labels created / Reviews received / Annotations created, average daily throughput is in units like “labels/day”, “reviews/day”, “annotations/day”.
- For time-based metrics, the forecast uses those as inputs, but the primary “average daily throughput” surfaced is for the count-based metrics (data rows/labels/reviews/annotations), per the spec and launch comms.
Efficiency view
The efficiency view helps you visualize the time spent per unit of work, per labeled asset, or per review. These metrics help answer questions such as:- What is the average amount of time spent labeling an asset?
- How can I reduce the time spent per labeled asset?
“Skipped” time is excluded from labeling time, and “abandoned” (unsaved) work is counted toward labeling time. You can use filters to include or exclude time spent on “deleted” labels.
Quality view
The quality view helps you understand the accuracy and consistency of the labeling work being produced. These metrics answer questions like:- What is the average quality of a labeled asset?
- How can I ensure label quality is more consistent across the team?
Individual member performance
Click Individual in the Performance dashboard to view individual metrics for each team member that has worked on the project. The performance metrics are separated by intent (labeling and reviewing) and are shown as distinct views in the table. Team members are listed individually in separate rows and will only appear if they have actively performed tasks during the selected period.Labeling metrics
Here are the metrics displayed in the Individual → Labeling view.Reviewing metrics
Here are the metrics displayed in the Individual → Reviewing view.Participation histogram
The Participation histogram is a chart in the Performance Dashboard that visualizes the distribution of labeler activity across a project. Specifically, it shows:- How many labelers fall into different activity buckets (e.g., by number of tasks completed or labels created)
- It helps admins see at a glance whether participation is concentrated among a few heavy contributors or spread evenly across the workforce
- Labelers who haven’t participated at all
- Labelers who are highly active vs. barely active
- Overall workforce engagement health for a project
Timer details
To help you measure efficiency, the timer log tracks how much time is spent on each data row. The time is broken down into three categories: labeling, reviewing, and reworking.How time is tracked in your project
A data row goes through many different states, which are tracked in the timer log.Important notes
- Inactivity Pause: If a user is inactive for more than five minutes, the timer will automatically pause. It will resume as soon as they become active again.
- Consensus and Benchmarks: If your project uses consensus or benchmarks, the time recorded for a data row will be the total time spent across all labels created for that data row.
How importing labels impacts the timer
Importing labels affects labeling time. When you import:- Ground truth labels: No label time is recorded on the Data rows tab or the performance dashboard for that label. Label time is displayed as zero (0). When team members modify the label, time is recorded as review time.
- Model-assisted learning (MAL) pre-labels: No label time is recorded on the Data rows tab or the performance dashboard for that label. Label time is displayed as zero (0). When team members open the data row in the editor and click Edit, the time spent before selecting Skip or Submit is recorded as labeling time.