Monitor the performance of your labeling project and labeling service workforce.
Metric | Description |
---|---|
Project count | The number of projects the member contributed to in the period selected. Only available on Member Performance. |
Member count | The number of members contributing to the project in the period selected. Only available on Project Performance. |
Labels created | The number of labeled data rows over a specified time period, including deleted labels by default. |
Annotations created | The number of annotations (features) created over a specified time period, including deleted annotations by default. |
Reviews received | The count of approve and reject actions performed on labeled data rows within a project. |
Total time | The total time spent on labeling, reviewing, and reworking data rows. |
Labeling time | The total time spent labeling data rows. Time increments when a labeler skips or submits an asset in the labeling queue. |
Review time | The total time spent reviewing labeled data rows. For reviewers, time increments when they view an asset in the review queue or in the data row browser view. For labelers, time increments when they view submitted labels. |
Rework time | The total time spent on reworking labeled data rows. Time increments when a labeler submits an asset in the rework queue, navigates to the review queue to make edits, or approves/rejects the asset in the data row browser view. |
Approval | The percentage of data rows labeled by the member that received an Approve action in the period selected. |
Metric | Description |
---|---|
Avg time per label | The average labeling time spent per label. Avg time per label = Total labeling time / number of labels submitted |
Avg review time | The average review time per data row. Avg review time = Total review time / number of data rows reviewed |
Avg rework time | The average rework time per data row. Avg rework time = Total rework time/ number of data rows reworked |
Metric | Description |
---|---|
Benchmark score | The average benchmark scores on labeled data rows within a specified time frame. |
Consensus score | The average consensus score of labeled assets over a selected period. This is the average agreement score of consensus labels within themselves (for a given data row). |