A segmentation mask is an object-type annotation that labels pixels in an image with a particular class.

Import

Python SDK

Mask is a kind of ObjectAnnotation

Definition

Mask(mask=MaskData(arr=array), color=(R, G, B))

ParameterValue
maskA wrapper of MaskData to specify the data to extract. Multiple Masks can use the same MaskData to improve upload performance.
colorColor in RGB tuple: (R,G,B)

MaskData

MaskData is mask data in a NumPy array. There are numerous ways to generate MaskData.

Parameter and methodsValue
arrNumPy uint8 array of [H, W, 3]
urloptional path to the array
MaskData.from_2D_array(arr)construct array from 2D rather than 3D array
Convert Polygon to MaskData
color = (255, 255, 255)
np_mask = polygon_annotation.value.draw(height=im.size[1],
                                        width=im.size[0],
                                        color=color)
mask_annotation = Mask(mask=MaskData(arr=np_mask), color=color)
# Build a combined mask from polygons
eye_color, nose_color = (255, 255, 255), (255, 0, 0)
h, w = np_data.shape[:2]
eyes = [
    Polygon(points=[
        Point(x=x, y=y) for x, y in [[82, 180], [83, 184], [88, 184], [86, 180]]
    ]),
    Polygon(points=[
        Point(x=x, y=y)
        for x, y in [[97, 182], [99, 184], [102, 183], [101, 180], [98, 180]]
    ]),
]
eye_masks = np.max([eye.draw(height=h, width=w) for eye in eyes], axis=0)
nose = Polygon(points=[
    Point(x=x, y=y) for x, y in [[95, 192], [93, 197], [96, 198], [100, 197],
                                 [100, 194], [100, 192], [96, 192]]
])
nose_mask = nose.draw(height=h, width=w, color=nose_color)
# Picks the brighter color if there is overlap.
# If you don't want overlap then just simply create separate masks
np_seg_mask = np.max([nose_mask, eye_masks], axis=0)

##Create two mask annotations from this one image
mask_data = MaskData(arr=np_seg_mask)
eye_mask = Mask(mask=mask_data, color=eye_color)
nose_mask = Mask(mask=mask_data, color=nose_color)

#Calling mask.draw() will return a mask with pixels equal to the specified color
eye_raster = eye_mask.draw()
nose_raster = nose_mask.draw()

Supported data types

Data typeSupported
ImageYes
ObjectAnnotation(
  name = "mask",
  value = Mask(mask=MaskData(arr=np_mask), color=color),
)

Format convertors

MethodUsageComments
.draw()MaskData.draw()This will return a mask with pixels equal to the specified color

NDJSON

NDJSON format is recommended if an annotation type is not yet supported in Python SDK or if you are unable to use the Python environment.

Definition

ParameterAsset typeRequiredDescription
uuidImageYesA user-generated UUID for each annotation. If you import an annotation to a Data Row and there is already an imported annotation with the same uuid on that Data Row, the latest import will override the previous one. The uuid must be 128 bits (32 characters). The following formats are supported:

- A0EEBC99-9C0B-4EF8-BB6D-6BB9BD380A11

- {a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11}

- a0eebc999c0b4ef8bb6d6bb9bd380a11

- a0ee-bc99-9c0b-4ef8-bb6d-6bb9-bd38-0a11

- {a0eebc99-9c0b4ef8-bb6d6bb9-bd380a11}
schemaIdImageYesThe ID of the schema that contains all of the information needed for rendering your annotation.
dataRow.idImageYesThe ID of the Data Row where you want to attach the imported annotations.
mask.instanceURIImageYesCloud-hosted mask (can be hosted on any cloud provider). If you are importing multiple mask annotations on a single Data Row, each mask should reference the same instanceURI. See example below.
mask.colorRGBImageYesAn array of RGB values from 0 to 255 that indicates which color represents each given mask. Only 3-channel RGB colors are supported.

Supported data types

Data typeSupported
ImageYes

Format

{
    "uuid": "45b15f9d-7884-4bb7-ac01-3567e8ed6c36",
    "schemaId": "ck68grts29n7w0890wv344dif",
    "dataRow": {
        "id": "cjxav5aa07r1g0dsq70t9eveg"
    },
    "mask": {
        "instanceURI": "https://api.labelbox.com/masks/feature/ckmuuw2tq00053g68rbwjpoim?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiJjazUycnZ4MWtxYXpiMDc3MDBtcTI3eDRsIiwib3JnYW5pemF0aW9uSWQiOiJjazUycnZ4MG1wdzRnMDc2NndncXZqdGw5IiwiaWF0IjoxNjE3MDM4NDAwLCJleHAiOjE2MTk2MzA0MDB9.5YaORL6mWpPqgAb6IbWChm4MQo14obOU8LFWbfCEHR0",
        "colorRGB": [
            255,
            255,
            255
        ]
    }
}{
    "uuid": "3a95ddcd-3ad0-4dc5-a24e-c05004b4b4d5",
    "schemaId": "ck7wi85rnd1050757aac5ba4d",
    "dataRow": {
        "id": "cjxav5aa07r1g0dsq70t9eveg"
    },
    "mask": {
        "instanceURI": "https://api.labelbox.com/masks/feature/ckmuuw2tq00053g68rbwjpoim?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiJjazUycnZ4MWtxYXpiMDc3MDBtcTI3eDRsIiwib3JnYW5pemF0aW9uSWQiOiJjazUycnZ4MG1wdzRnMDc2NndncXZqdGw5IiwiaWF0IjoxNjE3MDM4NDAwLCJleHAiOjE2MTk2MzA0MDB9.5YaORL6mWpPqgAb6IbWChm4MQo14obOU8LFWbfCEHR0",
        "colorRGB": [
            255,
            0,
            0
        ]
    }
}{
    "uuid": "f8284cbc-ecf3-4363-9e10-138501daf5f7",
    "schemaId": "ck7wi85pr1xz6079026yc0hch",
    "dataRow": {
        "id": "cjxav5aa07r1g0dsq70t9eveg"
    },
    "mask": {
        "instanceURI": "https://api.labelbox.com/masks/feature/ckmuuw2tq00053g68rbwjpoim?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiJjazUycnZ4MWtxYXpiMDc3MDBtcTI3eDRsIiwib3JnYW5pemF0aW9uSWQiOiJjazUycnZ4MG1wdzRnMDc2NndncXZqdGw5IiwiaWF0IjoxNjE3MDM4NDAwLCJleHAiOjE2MTk2MzA0MDB9.5YaORL6mWpPqgAb6IbWChm4MQo14obOU8LFWbfCEHR0",
        "colorRGB": [
            0,
            0,
            0
        ]
    }
}

Export

When you export your Segmentation mask annotations from Labelbox, the export file will contain the following information for each Segmentation mask.

Python SDK

The export format of the polygon is similar to the import format.

Learn more about exporting annotations using the SDK here

JSON

You will receive a JSON file when you generate an export from the app.

ParameterAsset type supportedDescription
schemaIdImage
Video (beta)
The ID of the schema containing all of the information needed for rendering your annotation.
featureIdImage
Video (beta)
ID of the annotation.
titleImage
Video (beta)
Name of the annotation in the ontology.
colorImage
Video (beta)
Color of the annotation in the ontology.
keyframeVideo (beta)When true, it means that a labeler created or made an adjustment to the annotation on that frame. When false, it means the annotation was auto-populated or interpolated on that frame.
instanceURIImage
Video (beta)
Annotation information hosted on Labelbox servers.

Format

{
    "featureId": "ck9bmetz800340za5k073bm7w",
    "schemaId": "ck9blmq1lnlxz08892qqlmxto",
    "title": "Orange bird",
    "value": "orange_bird",
    "color": "#2BFF00",
    "instanceURI": "https://api.labelbox.com/masks/feature/ckmuuw2tq00053g68rbwjpoim?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiJjazUycnZ4MWtxYXpiMDc3MDBtcTI3eDRsIiwib3JnYW5pemF0aW9uSWQiOiJjazUycnZ4MG1wdzRnMDc2NndncXZqdGw5IiwiaWF0IjoxNjE3MDM4NDAwLCJleHAiOjE2MTk2MzA0MDB9.5YaORL6mWpPqgAb6IbWChm4MQo14obOU8LFWbfCEHR0"
}
{
    "featureId": "ck9bmetz800340za5k073bm7w",
    "schemaId": "ck9blmq1lnlxz08892qqlmxto",
    "title": "Defect",
    "value": "defect",
    "color": "#FFD7D2",
    "keyframe": "true",
    "instanceURI": "https://api.labelbox.com/masks/feature/ckmuuw2tq00053g68rbwjpoim?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiJjazUycnZ4MWtxYXpiMDc3MDBtcTI3eDRsIiwib3JnYW5pemF0aW9uSWQiOiJjazUycnZ4MG1wdzRnMDc2NndncXZqdGw5IiwiaWF0IjoxNjE3MDM4NDAwLCJleHAiOjE2MTk2MzA0MDB9.5YaORL6mWpPqgAb6IbWChm4MQo14obOU8LFWbfCEHR0"
}
{ // This is from the frame data of the DICOM file for one of the axes
    "frameNumber": 172,
    "classifications": [],
    "objects": [
        {
            "featureId": "cl2y5hdq200053g6cklrdauj7",
            "schemaId": "ckzohkul206400z976bjrguy3",
            "title": "Tool 1",
            "value": "tool_1",
            "color": "#1CE6FF",
            "keyframe": true,
            "instanceURI": "https://api.labelbox.com/masks/feature/cl2y5hdq200053g6cklrdauj7/172",
            "classifications": []
        }
    ],
    "relationships": []
}