Labeling editors

Labelbox provides several native editor interfaces, each designed for specific use cases. Explore the links below to learn which editor best fits your use case.

• Computer vision:

• Language:

• Large language models:

• Model evaluation:

​

Editor interface

Labelbox automatically adjusts the editor interface based on the asset type selected for the project. However, there are some global editor settings and enhancements shared across all of the editor interfaces. These global editor settings are designed to provide an optimal labeling experience for your team through increased customization and quick accessibility.

Additionally, there are some data enrichment capabilities and supplementary tools available for use in the editor only with certain data modalities.

Features such as attachments, instructions, and the data row information panel are available in the editor regardless of the type of asset being labeled.

​

Attachments

Attachments can be used to provide supplementary content to any asset to help provide additional context for the labeling team. An attachment applies to an individual asset and may comprise an image, video, text, or HTML content. Markdown rendering is supported for attachments, allowing you to use to format text, add links, and include images within attachments. Multiple attachments can be linked to an singular data row.

​

Supported attachment types

Below are the types of attachment data that we support in the editor.

* The HTML attachment type can take a text or HTML file and render it in an iframe in the attachments panel. Markdown and HTML can also be used with this type to style the content.

​

Import attachments

When creating an attachment on a data row, you must provide the following information.

For details on how to create attachments or import data rows with attachments using the Python SDK, please view the documentation here.

JSON format

If uploading a JSON file of data rows through the SDK, use the below format to include attachments. Note how multiple attachments link to an individual data row.

[
    {
      "row_data": "https://lb-test-data.s3.us-west-1.amazonaws.com/image-samples/sample-image-1.jpg",
      "global_key": "https://lb-test-data.s3.us-west-1.amazonaws.com/image-samples/sample-image-1.jpg",
      "media_type": "IMAGE",
      "metadata_fields": [{"schema_id": "cko8s9r5v0001h2dk9elqdidh", "value": "tag_string"}],
      "attachments": [
            {
                "type": "RAW_TEXT",
                "value": "IOWA, Zone 2232, June 2022 [Text string]"
            },
            {
                "type": "RAW_TEXT",
                "value": "https://storage.googleapis.com/labelbox-sample-datasets/Docs/text_attachment.txt"
            },
            {
                "type": "TEXT_URL",
                "value": "https://storage.googleapis.com/labelbox-sample-datasets/Docs/text_attachment.txt"
            },
            {
                "type": "IMAGE",
                "value": "https://storage.googleapis.com/labelbox-sample-datasets/Docs/disease_attachment.jpeg"
            },
            {
                "type": "VIDEO",
                "value": "https://storage.googleapis.com/labelbox-sample-datasets/Docs/drone_video.mp4"
            },
            {
                "type": "HTML",
                "value": "https://storage.googleapis.com/labelbox-sample-datasets/Docs/windy.html"
            }
        ]
    }
]

​

Viewing attachments

When any attachments have been provided for an asset, labelers will see the Attachment button appear in the top left of the editor.

You can also use the hotkey Shift + ? to open the attachments panel.

​

Labeling instructions

Labeling instructions (or Annotation guidelines) are essential for any large-scale labeling operation. Whether your labeling team is in-house or outsourced, machine learning teams can leverage labeling instructions to communicate best practices with their labelers.

The following template shows one way to create labeling instructions; a PDF download is also available.

​

Add labeling instructions to an ontology

Each ontology can have one labeling instructions document. You can attach instructions whenever editing an ontology by clicking the Instructions tab, and you can update the instructions at any time. For details, see Add/update instructions.

Keep in mind that ontologies can be connected to multiple projects. Updating the instructions for one project impacts all projects sharing the ontology.

If instructions vary between projects, create individual copies of your ontology for each unique project or set of instructions.

​

Data row information panel

Access to additional context and information at the data row level can be extremely helpful to labelers as they label data. Users can click to view a side panel that brings up data row information, providing teams with ample context and easy access to information related to the particular data row being served.

Labelers can view information such as the global key, the dataset, and when the data row was created. Based on data type, labelers can also view specific media attributes, such as an image’s width and height, the number of pages in a PDF document, the frame rate and duration of a video, and more. Additionally, labelers can also see any metadata attached to the data row.

​

Supplementary layers and tools

Some complementary features are only available for use in the editor when labeling specific data modalities.

​

Image overlay

Image overlay can be used to provide labelers with additional view options for the image being labeled. For example, if you have additional cameras capturing images of your subject matter in different formats (greyscale, thermal, etc.), you may want to provide these images as contextual layers to the primary image.

Layers are applied to individual images, and a maximum of 10 layers may be added per image. Image layers are a visualization tool designed to help you view the asset to be labeled in different ways. You may change visual settings on the image layers (e.g., transparency, brightness) and still create annotations on the data row.

Import image overlay layers

Image overlays are imported as attachments to a data row. You must include the following information for each image layer.

For details on how to create image layers or import data rows with image layers using the Python SDK, please view the documentation here.

JSON format

If uploading a JSON file of data rows through the SDK, use the below format to include image layers. Note how the image overlay layers are imported as attachments on an individual data row.

[
    {
      "row_data": "https://lb-test-data.s3.us-west-1.amazonaws.com/image-samples/sample-image-1.jpg",
      "global_key": "https://lb-test-data.s3.us-west-1.amazonaws.com/image-samples/sample-image-1.jpg",
      "media_type": "IMAGE",
      "attachments": [
            {
                "type": "IMAGE_OVERLAY",
                "value": "https://storage.googleapis.com/labelbox-sample-datasets/Docs/rgb.jpg",
                "name": "RGB"
            },
            {
                "type": "IMAGE_OVERLAY",
                "value": "https://storage.googleapis.com/labelbox-sample-datasets/Docs/cir.jpg",
                "name": "CIR"
            },
            {
                "type": "IMAGE_OVERLAY",
                "value": "https://storage.googleapis.com/labelbox-sample-datasets/Docs/weeds.jpg",
                "name": "Weeds"
            }
        ]
    }
]

View image overlay layers

When layers have been provided for an image, you will see an Image overlay button appear in the top bar of the editor directly to the left of the zoom options.

Clicking on this button opens a dropdown menu where you can select an image to overlay. Alternatively, you can use the keyboard shortcut Option/Alt + Layer # to quickly switch between layers.

​

Smart labeling tools

Labelbox provides various AI-based tools to help you label faster. By eliminating some of your most manual work, these tools allow you to take advantage of the latest AI technologies to move fast and achieve higher throughput.

​

Auto-segment

This tool is embedded into the segmentation mask tool. To use auto-segment, select a segmentation mask tool, toggle on auto-segment by selecting the magic wand icon or using the hotkey R, and draw a box around an object.

Labelbox will automatically draw a segmentation mask on the object inside the box. Then, you can make edits, as usual, using the segmentation mask’s pen tools. For more details, please view the documentation here.

​

Bounding box tracking (beta)

This tool is embedded into the bounding box tool and is only available when labeling video assets. When you draw a bounding box around an object, click Track Object to activate bounding box tracking.

Labelbox will automatically track the object for a specified number of frames. Edit the results by selecting and removing any desired keyframes. For more details, please view the documentation here.

​

Supported annotations

The table below indicates which top-level annotation types you can use to label each asset type.