Mask

A segmentation mask is an object-type annotation that labels pixels in an image with a particular class.

Import

Python SDK

Mask is a kind of ObjectAnnotation

Definition

Mask(mask=MaskData(arr=array), color=(R, G, B))

Parameter	Value
`mask`	A wrapper of MaskData to specify the data to extract. Multiple Masks can use the same MaskData to improve upload performance.
`color`	Color in RGB tuple: (R,G,B)

MaskData

MaskData is mask data in a NumPy array. There are numerous ways to generate MaskData.

Parameter and methods	Value
`arr`	NumPy uint8 array of [H, W, 3]
`url`	optional path to the array
`MaskData.from_2D_array(arr)`	construct array from 2D rather than 3D array

Convert Polygon to MaskData

color = (255, 255, 255)
np_mask = polygon_annotation.value.draw(height=im.size[1],
                                        width=im.size[0],
                                        color=color)
mask_annotation = Mask(mask=MaskData(arr=np_mask), color=color)

# Build a combined mask from polygons
eye_color, nose_color = (255, 255, 255), (255, 0, 0)
h, w = np_data.shape[:2]
eyes = [
    Polygon(points=[
        Point(x=x, y=y) for x, y in [[82, 180], [83, 184], [88, 184], [86, 180]]
    ]),
    Polygon(points=[
        Point(x=x, y=y)
        for x, y in [[97, 182], [99, 184], [102, 183], [101, 180], [98, 180]]
    ]),
]
eye_masks = np.max([eye.draw(height=h, width=w) for eye in eyes], axis=0)
nose = Polygon(points=[
    Point(x=x, y=y) for x, y in [[95, 192], [93, 197], [96, 198], [100, 197],
                                 [100, 194], [100, 192], [96, 192]]
])
nose_mask = nose.draw(height=h, width=w, color=nose_color)
# Picks the brighter color if there is overlap.
# If you don't want overlap then just simply create separate masks
np_seg_mask = np.max([nose_mask, eye_masks], axis=0)

##Create two mask annotations from this one image
mask_data = MaskData(arr=np_seg_mask)
eye_mask = Mask(mask=mask_data, color=eye_color)
nose_mask = Mask(mask=mask_data, color=nose_color)

#Calling mask.draw() will return a mask with pixels equal to the specified color
eye_raster = eye_mask.draw()
nose_raster = nose_mask.draw()

Supported data types

Data type	Supported
Image	Yes

ObjectAnnotation(
  name = "mask",
  value = Mask(mask=MaskData(arr=np_mask), color=color),
)

Format convertors

Method	Usage	Comments
`.draw()`	`MaskData.draw()`	This will return a mask with pixels equal to the specified color

NDJSON

NDJSON format is recommended if an annotation type is not yet supported in Python SDK or if you are unable to use the Python environment.

Definition

Parameter	Asset type	Required	Description
`uuid`	Image	Yes	A user-generated UUID for each annotation. If you import an annotation to a Data Row and there is already an imported annotation with the same uuid on that Data Row, the latest import will override the previous one. The uuid must be 128 bits (32 characters). The following formats are supported: - `A0EEBC99-9C0B-4EF8-BB6D-6BB9BD380A11` - `{a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11}` - `a0eebc999c0b4ef8bb6d6bb9bd380a11` - `a0ee-bc99-9c0b-4ef8-bb6d-6bb9-bd38-0a11` - `{a0eebc99-9c0b4ef8-bb6d6bb9-bd380a11}`
`schemaId`	Image	Yes	The ID of the schema that contains all of the information needed for rendering your annotation.
`dataRow.id`	Image	Yes	The ID of the Data Row where you want to attach the imported annotations.
`mask.instanceURI`	Image	Yes	Cloud-hosted mask (can be hosted on any cloud provider). If you are importing multiple mask annotations on a single Data Row, each mask should reference the same `instanceURI`. See example below.
`mask.colorRGB`	Image	Yes	An array of RGB values from 0 to 255 that indicates which color represents each given mask. Only 3-channel RGB colors are supported.

Supported data types

Data type	Supported
Image	Yes

Format

{
    "uuid": "45b15f9d-7884-4bb7-ac01-3567e8ed6c36",
    "schemaId": "ck68grts29n7w0890wv344dif",
    "dataRow": {
        "id": "cjxav5aa07r1g0dsq70t9eveg"
    },
    "mask": {
        "instanceURI": "https://api.labelbox.com/masks/feature/ckmuuw2tq00053g68rbwjpoim?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiJjazUycnZ4MWtxYXpiMDc3MDBtcTI3eDRsIiwib3JnYW5pemF0aW9uSWQiOiJjazUycnZ4MG1wdzRnMDc2NndncXZqdGw5IiwiaWF0IjoxNjE3MDM4NDAwLCJleHAiOjE2MTk2MzA0MDB9.5YaORL6mWpPqgAb6IbWChm4MQo14obOU8LFWbfCEHR0",
        "colorRGB": [
            255,
            255,
            255
        ]
    }
}{
    "uuid": "3a95ddcd-3ad0-4dc5-a24e-c05004b4b4d5",
    "schemaId": "ck7wi85rnd1050757aac5ba4d",
    "dataRow": {
        "id": "cjxav5aa07r1g0dsq70t9eveg"
    },
    "mask": {
        "instanceURI": "https://api.labelbox.com/masks/feature/ckmuuw2tq00053g68rbwjpoim?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiJjazUycnZ4MWtxYXpiMDc3MDBtcTI3eDRsIiwib3JnYW5pemF0aW9uSWQiOiJjazUycnZ4MG1wdzRnMDc2NndncXZqdGw5IiwiaWF0IjoxNjE3MDM4NDAwLCJleHAiOjE2MTk2MzA0MDB9.5YaORL6mWpPqgAb6IbWChm4MQo14obOU8LFWbfCEHR0",
        "colorRGB": [
            255,
            0,
            0
        ]
    }
}{
    "uuid": "f8284cbc-ecf3-4363-9e10-138501daf5f7",
    "schemaId": "ck7wi85pr1xz6079026yc0hch",
    "dataRow": {
        "id": "cjxav5aa07r1g0dsq70t9eveg"
    },
    "mask": {
        "instanceURI": "https://api.labelbox.com/masks/feature/ckmuuw2tq00053g68rbwjpoim?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiJjazUycnZ4MWtxYXpiMDc3MDBtcTI3eDRsIiwib3JnYW5pemF0aW9uSWQiOiJjazUycnZ4MG1wdzRnMDc2NndncXZqdGw5IiwiaWF0IjoxNjE3MDM4NDAwLCJleHAiOjE2MTk2MzA0MDB9.5YaORL6mWpPqgAb6IbWChm4MQo14obOU8LFWbfCEHR0",
        "colorRGB": [
            0,
            0,
            0
        ]
    }
}

Export

When you export your Segmentation mask annotations from Labelbox, the export file will contain the following information for each Segmentation mask.

Python SDK

The export format of the polygon is similar to the import format.

Learn more about exporting annotations using the SDK here

JSON

You will receive a JSON file when you generate an export from the app.

Parameter	Asset type supported	Description
`schemaId`	Image Video (beta)	The ID of the schema containing all of the information needed for rendering your annotation.
`featureId`	Image Video (beta)	ID of the annotation.
`title`	Image Video (beta)	Name of the annotation in the ontology.
`color`	Image Video (beta)	Color of the annotation in the ontology.
`keyframe`	Video (beta)	When `true`, it means that a labeler created or made an adjustment to the annotation on that frame. When `false`, it means the annotation was auto-populated or interpolated on that frame.
`instanceURI`	Image Video (beta)	Annotation information hosted on Labelbox servers.

Format

{
    "featureId": "ck9bmetz800340za5k073bm7w",
    "schemaId": "ck9blmq1lnlxz08892qqlmxto",
    "title": "Orange bird",
    "value": "orange_bird",
    "color": "#2BFF00",
    "instanceURI": "https://api.labelbox.com/masks/feature/ckmuuw2tq00053g68rbwjpoim?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiJjazUycnZ4MWtxYXpiMDc3MDBtcTI3eDRsIiwib3JnYW5pemF0aW9uSWQiOiJjazUycnZ4MG1wdzRnMDc2NndncXZqdGw5IiwiaWF0IjoxNjE3MDM4NDAwLCJleHAiOjE2MTk2MzA0MDB9.5YaORL6mWpPqgAb6IbWChm4MQo14obOU8LFWbfCEHR0"
}

{
    "featureId": "ck9bmetz800340za5k073bm7w",
    "schemaId": "ck9blmq1lnlxz08892qqlmxto",
    "title": "Defect",
    "value": "defect",
    "color": "#FFD7D2",
    "keyframe": "true",
    "instanceURI": "https://api.labelbox.com/masks/feature/ckmuuw2tq00053g68rbwjpoim?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiJjazUycnZ4MWtxYXpiMDc3MDBtcTI3eDRsIiwib3JnYW5pemF0aW9uSWQiOiJjazUycnZ4MG1wdzRnMDc2NndncXZqdGw5IiwiaWF0IjoxNjE3MDM4NDAwLCJleHAiOjE2MTk2MzA0MDB9.5YaORL6mWpPqgAb6IbWChm4MQo14obOU8LFWbfCEHR0"
}

{ // This is from the frame data of the DICOM file for one of the axes
    "frameNumber": 172,
    "classifications": [],
    "objects": [
        {
            "featureId": "cl2y5hdq200053g6cklrdauj7",
            "schemaId": "ckzohkul206400z976bjrguy3",
            "title": "Tool 1",
            "value": "tool_1",
            "color": "#1CE6FF",
            "keyframe": true,
            "instanceURI": "https://api.labelbox.com/masks/feature/cl2y5hdq200053g6cklrdauj7/172",
            "classifications": []
        }
    ],
    "relationships": []
}