Issues and comments

Labeling data is an inherently collaborative process that requires continuous feedback between labelers and reviewers to ensure high-quality outcomes. In the Labelbox editor, you can facilitate this collaboration throughout the labeling and reviewing process by creating an issue on the asset and opening it up for discussion in the comments section.

How it works

Once a label is submitted or skipped, the issues on that labeled asset will become visible to reviewers on that project. Team members can then comment on those issues or create new issues. Once an issue has been fully addressed, you can change the status to resolved, signaling to other users that the issue has been fixed or the question has been answered.

📘
Issues created on unsubmitted assets
Issues made on assets that have not been submitted nor skipped will not be visible to other team members, nor will they appear in the Issues tab. Make sure to click Submit or Skip in the editor for issues to be visible.
Resolved issues are hidden by default.

Issues and comments operate at the project level, not at the organizational level. Permissions for creating and modifying issues vary depending on the user's role. These same permissions apply to workforce user roles but only for labeled assets created by users in that workforce organization.

Role	Permissions
Labeler	- Can create issues on assets they have labeled in the project. - Can update and delete their own issues on assets they have labeled. - Can resolve and reopen their own issues. - Can create, update, and delete issues on assets they have labeled.
Reviewer or Admin	- Can create issues on any labeled asset. - Can delete their own issues on any labeled asset. - Can resolve and reopen issues on any labeled asset. - Can create, update, and delete comments on any labeled asset.

As an example, a common feedback loop between reviewers and labelers is:

A labeler submits a label on an asset.
The reviewer reviews the label and notices an issue. The reviewer creates an issue on the label.
The labeler can find any issues on their labels in the Issues tab. Clicking on the issue will take them directly to the label with the issue.
The labeler can either correct the issue and resolve the issue or comment on the issue to get more clarity from the reviewer.

Create an issue

In the Editor, click on the issues icon in the top left corner to activate the tool.
Click on the image where you would like to apply the Issue.
Enter the content for the issue (e.g., a description of a problem, a question about an annotation, etc.) in the text box.
Click Enter key to create the issue.

View all issues for a project

Select a project and go to the Issues tab.
Filter the issues by state by selecting All, Open, or Resolved.

Create a comment on an issue

Go to the Issues tab and select an Issue OR click on an issue in the editor.
Enter your response to the Issue in the Reply text box.
Click Enter key to create the comment.

Edit, resolve, or delete an issue

On the Issue, click on the more options menu.
Select Edit to change the content of the issue/comment, Resolve to close the issue, or Delete to remove it entirely. Access to these actions will vary based on permissions.
If you are the author of an Issue, you can drag and drop it to move its location on the image.