A segmentation mask is an object-type annotation that labels pixels in an image with a particular class.

Import

Python SDK

Mask is a kind of ObjectAnnotation

Definition

Mask(mask=MaskData(arr=array), color=(R, G, B))

Parameter

Value

mask

A wrapper of MaskData to specify the data to extract. Multiple Masks can use the same MaskData to improve upload performance.

color

Color in RGB tuple: (R,G,B)

MaskData

MaskData is mask data in a NumPy array. There are numerous ways to generate MaskData.

Parameter and methods

Value

arr

NumPy uint8 array of [H, W, 3]

url

optional path to the array

MaskData.from_2D_array(arr)

construct array from 2D rather than 3D array

Convert Polygon to MaskData
color = (255, 255, 255)
np_mask = polygon_annotation.value.draw(height=im.size[1],
                                        width=im.size[0],
                                        color=color)
mask_annotation = Mask(mask=MaskData(arr=np_mask), color=color)
# Build a combined mask from polygons
eye_color, nose_color = (255, 255, 255), (255, 0, 0)
h, w = np_data.shape[:2]
eyes = [
    Polygon(points=[
        Point(x=x, y=y) for x, y in [[82, 180], [83, 184], [88, 184], [86, 180]]
    ]),
    Polygon(points=[
        Point(x=x, y=y)
        for x, y in [[97, 182], [99, 184], [102, 183], [101, 180], [98, 180]]
    ]),
]
eye_masks = np.max([eye.draw(height=h, width=w) for eye in eyes], axis=0)
nose = Polygon(points=[
    Point(x=x, y=y) for x, y in [[95, 192], [93, 197], [96, 198], [100, 197],
                                 [100, 194], [100, 192], [96, 192]]
])
nose_mask = nose.draw(height=h, width=w, color=nose_color)
# Picks the brighter color if there is overlap.
# If you don't want overlap then just simply create separate masks
np_seg_mask = np.max([nose_mask, eye_masks], axis=0)

##Create two mask annotations from this one image
mask_data = MaskData(arr=np_seg_mask)
eye_mask = Mask(mask=mask_data, color=eye_color)
nose_mask = Mask(mask=mask_data, color=nose_color)

#Calling mask.draw() will return a mask with pixels equal to the specified color
eye_raster = eye_mask.draw()
nose_raster = nose_mask.draw()

Supported data types

Data type

Supported

Image

Yes

ObjectAnnotation(
  name = "mask",
  value = Mask(mask=MaskData(arr=np_mask), color=color),
)

Format convertors

Method

Usage

Comments

.draw()

MaskData.draw()

This will return a mask with pixels equal to the specified color

NDJSON

NDJSON format is recommended if an annotation type is not yet supported in Python SDK or if you are unable to use the Python environment.

Definition

Parameter

Asset type

Required

Description

uuid

Image

Yes

A user-generated UUID for each annotation. If you import an annotation to a Data Row and there is already an imported annotation with the same uuid on that Data Row, the latest import will override the previous one. The uuid must be 128 bits (32 characters). The following formats are supported:

  • A0EEBC99-9C0B-4EF8-BB6D-6BB9BD380A11

  • {a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11}

  • a0eebc999c0b4ef8bb6d6bb9bd380a11

  • a0ee-bc99-9c0b-4ef8-bb6d-6bb9-bd38-0a11

  • {a0eebc99-9c0b4ef8-bb6d6bb9-bd380a11}

schemaId

Image

Yes

The ID of the schema that contains all of the information needed for rendering your annotation.

dataRow.id

Image

Yes

The ID of the Data Row where you want to attach the imported annotations.

mask.instanceURI

Image

Yes

Cloud-hosted mask (can be hosted on any cloud provider). If you are importing multiple mask annotations on a single Data Row, each mask should reference the same instanceURI. See example below.

mask.colorRGB

Image

Yes

An array of RGB values from 0 to 255 that indicates which color represents each given mask. Only 3-channel RGB colors are supported.

Supported data types

Data type

Supported

Image

Yes

Format

{
    "uuid": "45b15f9d-7884-4bb7-ac01-3567e8ed6c36",
    "schemaId": "ck68grts29n7w0890wv344dif",
    "dataRow": {
        "id": "cjxav5aa07r1g0dsq70t9eveg"
    },
    "mask": {
        "instanceURI": "https://api.labelbox.com/masks/feature/ckmuuw2tq00053g68rbwjpoim?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiJjazUycnZ4MWtxYXpiMDc3MDBtcTI3eDRsIiwib3JnYW5pemF0aW9uSWQiOiJjazUycnZ4MG1wdzRnMDc2NndncXZqdGw5IiwiaWF0IjoxNjE3MDM4NDAwLCJleHAiOjE2MTk2MzA0MDB9.5YaORL6mWpPqgAb6IbWChm4MQo14obOU8LFWbfCEHR0",
        "colorRGB": [
            255,
            255,
            255
        ]
    }
}{
    "uuid": "3a95ddcd-3ad0-4dc5-a24e-c05004b4b4d5",
    "schemaId": "ck7wi85rnd1050757aac5ba4d",
    "dataRow": {
        "id": "cjxav5aa07r1g0dsq70t9eveg"
    },
    "mask": {
        "instanceURI": "https://api.labelbox.com/masks/feature/ckmuuw2tq00053g68rbwjpoim?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiJjazUycnZ4MWtxYXpiMDc3MDBtcTI3eDRsIiwib3JnYW5pemF0aW9uSWQiOiJjazUycnZ4MG1wdzRnMDc2NndncXZqdGw5IiwiaWF0IjoxNjE3MDM4NDAwLCJleHAiOjE2MTk2MzA0MDB9.5YaORL6mWpPqgAb6IbWChm4MQo14obOU8LFWbfCEHR0",
        "colorRGB": [
            255,
            0,
            0
        ]
    }
}{
    "uuid": "f8284cbc-ecf3-4363-9e10-138501daf5f7",
    "schemaId": "ck7wi85pr1xz6079026yc0hch",
    "dataRow": {
        "id": "cjxav5aa07r1g0dsq70t9eveg"
    },
    "mask": {
        "instanceURI": "https://api.labelbox.com/masks/feature/ckmuuw2tq00053g68rbwjpoim?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiJjazUycnZ4MWtxYXpiMDc3MDBtcTI3eDRsIiwib3JnYW5pemF0aW9uSWQiOiJjazUycnZ4MG1wdzRnMDc2NndncXZqdGw5IiwiaWF0IjoxNjE3MDM4NDAwLCJleHAiOjE2MTk2MzA0MDB9.5YaORL6mWpPqgAb6IbWChm4MQo14obOU8LFWbfCEHR0",
        "colorRGB": [
            0,
            0,
            0
        ]
    }
}

Export

When you export your Segmentation mask annotations from Labelbox, the export file will contain the following information for each Segmentation mask.

Python SDK

The export format of the polygon is similar to the import format.

Learn more about exporting annotations using the SDK here

JSON

You will receive a JSON file when you generate an export from the app.

Parameter

Asset type supported

Description

schemaId

Image
Video (beta)

The ID of the schema containing all of the information needed for rendering your annotation.

featureId

Image
Video (beta)

ID of the annotation.

title

Image
Video (beta)

Name of the annotation in the ontology.

color

Image
Video (beta)

Color of the annotation in the ontology.

keyframe

Video (beta)

When true, it means that a labeler created or made an adjustment to the annotation on that frame. When false, it means the annotation was auto-populated or interpolated on that frame.

instanceURI

Image
Video (beta)

Annotation information hosted on Labelbox servers.

Format

{
    "featureId": "ck9bmetz800340za5k073bm7w",
    "schemaId": "ck9blmq1lnlxz08892qqlmxto",
    "title": "Orange bird",
    "value": "orange_bird",
    "color": "#2BFF00",
    "instanceURI": "https://api.labelbox.com/masks/feature/ckmuuw2tq00053g68rbwjpoim?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiJjazUycnZ4MWtxYXpiMDc3MDBtcTI3eDRsIiwib3JnYW5pemF0aW9uSWQiOiJjazUycnZ4MG1wdzRnMDc2NndncXZqdGw5IiwiaWF0IjoxNjE3MDM4NDAwLCJleHAiOjE2MTk2MzA0MDB9.5YaORL6mWpPqgAb6IbWChm4MQo14obOU8LFWbfCEHR0"
}
{
    "featureId": "ck9bmetz800340za5k073bm7w",
    "schemaId": "ck9blmq1lnlxz08892qqlmxto",
    "title": "Defect",
    "value": "defect",
    "color": "#FFD7D2",
    "keyframe": "true",
    "instanceURI": "https://api.labelbox.com/masks/feature/ckmuuw2tq00053g68rbwjpoim?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiJjazUycnZ4MWtxYXpiMDc3MDBtcTI3eDRsIiwib3JnYW5pemF0aW9uSWQiOiJjazUycnZ4MG1wdzRnMDc2NndncXZqdGw5IiwiaWF0IjoxNjE3MDM4NDAwLCJleHAiOjE2MTk2MzA0MDB9.5YaORL6mWpPqgAb6IbWChm4MQo14obOU8LFWbfCEHR0"
}
{ // This is from the frame data of the DICOM file for one of the axes
    "frameNumber": 172,
    "classifications": [],
    "objects": [
        {
            "featureId": "cl2y5hdq200053g6cklrdauj7",
            "schemaId": "ckzohkul206400z976bjrguy3",
            "title": "Tool 1",
            "value": "tool_1",
            "color": "#1CE6FF",
            "keyframe": true,
            "instanceURI": "https://api.labelbox.com/masks/feature/cl2y5hdq200053g6cklrdauj7/172",
            "classifications": []
        }
    ],
    "relationships": []
}