Upload geospatial predictions

Open In Colab

GitHub

Supported predictions

To upload predictions in Labelbox, you need to create the prediction payload. This section provides this payload for every prediction type. Labelbox supports two formats for the predictions payload:

Python annotation types (recommended)
NDJSON

Both are described below.

Uploading confidence scores is optional

If you do not specify a confidence score, the prediction will be treated as if it had a confidence score of 1.

Point

point_prediction = lb_types.ObjectAnnotation(
  name = "point_geo",
  confidence=0.4,
  value = lb_types.Point(x=-99.20647859573366, y=19.40018029091072),
)

point_prediction_ndjson = {
    "name": "point_geo",
    "confidence": 0.4,
    "point": {
         "x": -99.20647859573366,
         "y": 19.40018029091072
     }
}

Polyline

# Coordinates
coords = [
            [
                -99.20842051506044,
                19.40032196622975
            ],
            [
                -99.20809864997865,
                19.39758963475322
            ],
            [
                -99.20758366584778,
                19.39776167179227
            ],
            [
                -99.20728325843811,
                19.3973265189299
            ]
        ]

line_points = []

for sub in coords: 
  line_points.append(lb_types.Point(x=sub[0], y=sub[1]))

# Python Annotation 
polyline_prediction = lb_types.ObjectAnnotation(
  name = "polyline_geo",
  confidence=0.5,
  value = lb_types.Line(points=line_points),
)

coords = [
            [
                -99.20842051506044,
                19.40032196622975
            ],
            [
                -99.20809864997865,
                19.39758963475322
            ],
            [
                -99.20758366584778,
                19.39776167179227
            ],
            [
                -99.20728325843811,
                19.3973265189299
            ]
        ]

line_points_ndjson = []

for sub in coords: 
  line_points_ndjson.append({"x":sub[0], "y":sub[1]})
  
polyline_prediction_ndjson = {
    "name": "polyline_geo",
    "confidence": 0.5,
    "line": line_points_ndjson
}

Polygon

polygon_points = []

for sub in coords_polygon: 
  polygon_points.append(lb_types.Point(x=sub[0], y=sub[1]))

polygon_prediction = lb_types.ObjectAnnotation(
  name = "polygon_geo",
  confidence=0.5,
  value = lb_types.Polygon(points=polygon_points),
)

polygon_points_ndjson = []

for sub in coords_polygon:
  polygon_points_ndjson.append({"x":sub[0], "y":sub[1]})

polygon_prediction_ndjson = {
    "name": "polygon_geo",
    "confidence": 0.5,
    "polygon": polygon_points_ndjson
}

coords_polygon = [
    [
        -99.21042680740356,
        19.40036244486966
    ],
    [
        -99.2104160785675,
        19.40017017124035
    ],
    [
        -99.2103409767151,
        19.400008256428897
    ],
    [
        -99.21014785766603,
        19.400008256428897
    ],
    [
        -99.21019077301027,
        19.39983622176518
    ],
    [
        -99.21022295951845,
        19.399674306621385
    ],
    [
        -99.21029806137086,
        19.39951239131646
    ],
    [
        -99.2102873325348,
        19.399340356128437
    ],
    [
        -99.21025514602663,
        19.399117722085677
    ],
    [
        -99.21024441719057,
        19.39892544698541
    ],
    [
        -99.2102336883545,
        19.39874329141769
    ],
    [
        -99.21021223068239,
        19.398561135646027
    ],
    [
        -99.21018004417421,
        19.398399219233365
    ],
    [
        -99.21011567115785,
        19.39822718286836
    ],
    [
        -99.20992255210878,
        19.398136104719125
    ],
    [
        -99.20974016189577,
        19.398085505725305
    ],
    [
        -99.20957922935487,
        19.398004547302467
    ],
    [
        -99.20939683914186,
        19.39792358883935
    ],
    [
        -99.20918226242067,
        19.39786286996558
    ],
    [
        -99.20899987220764,
        19.397822390703805
    ],
    [
        -99.20891404151918,
        19.397994427496787
    ],
    [
        -99.20890331268312,
        19.398176583902874
    ],
    [
        -99.20889258384706,
        19.398368859888045
    ],
    [
        -99.20889258384706,
        19.398540896103246
    ],
    [
        -99.20890331268312,
        19.39872305189756
    ],
    [
        -99.20889258384706,
        19.39890520748796
    ],
    [
        -99.20889258384706,
        19.39907724313608
    ],
    [
        -99.20889258384706,
        19.399259398329956
    ],
    [
        -99.20890331268312,
        19.399431433603585
    ],
    [
        -99.20890331268312,
        19.39961358840092
    ],
    [
        -99.20890331268312,
        19.399785623300048
    ],
    [
        -99.20897841453552,
        19.399937418648214
    ],
    [
        -99.20919299125673,
        19.399937418648214
    ],
    [
        -99.2093861103058,
        19.39991717927664
    ],
    [
        -99.20956850051881,
        19.39996777770086
    ],
    [
        -99.20961141586305,
        19.40013981222548
    ],
    [
        -99.20963287353517,
        19.40032196622975
    ],
    [
        -99.20978307724,
        19.4004130431554
    ],
    [
        -99.20996546745302,
        19.40039280384301
    ],
    [
        -99.21019077301027,
        19.400372564528084
    ],
    [
        -99.21042680740356,
        19.40036244486966
    ]

]

Bounding box

bbox_top_left = lb_types.Point(x= -99.20746564865112, y=19.39799442829336)
bbox_bottom_right = lb_types.Point(x=-99.20568466186523, y=19.39925939999194)

bbox_prediction = lb_types.ObjectAnnotation(
  name = "bbox_geo",
  confidence=0.5,
  value = lb_types.Rectangle(start=bbox_top_left, end=bbox_bottom_right)
)

coord_object =  {
    "coordinates" : [[
            [
                -99.20746564865112,
                19.39799442829336
            ],
            [
                -99.20746564865112,
                19.39925939999194
            ],
            [
                -99.20568466186523,
                19.39925939999194
            ],
            [
                -99.20568466186523,
                19.39799442829336
            ],
            [
                -99.20746564865112,
                19.39799442829336
            ]
        ]]
}

bbox_prediction_ndjson = {
    "name" : "bbox_geo",
    "confidence": 0.5,
    "bbox" : {
        'top': coord_object["coordinates"][0][1][1],
        'left': coord_object["coordinates"][0][1][0],
        'height': coord_object["coordinates"][0][3][1] - coord_object["coordinates"][0][1][1],
        'width': coord_object["coordinates"][0][3][0] - coord_object["coordinates"][0][1][0]
    }
}

Classification: Radio (single choice)

radio_prediction = lb_types.ClassificationAnnotation(
    name="radio_question_geo",
    value=lb_types.Radio(answer=lb_types.ClassificationAnswer(name="first_radio_answer", confidence=0.5))
)

radio_prediction_ndjson = {
    "name": "radio_question_geo",
    "answer": { "name": "first_radio_answer", "confidence": 0.5}
}

Bounding box with nested checklist classification

bbox_with_checklist_subclass = lb_types.ObjectAnnotation(
    name="bbox_checklist_geo",
    confidence=0.5,
    value=lb_types.Rectangle(
        start=lb_types.Point(x=-99.210266, y=19.39540372195134), # Top left
        end=lb_types.Point(x=-99.20621067903966, y=19.396901), # Bottom right
    ),
    classifications=[
        lb_types.ClassificationAnnotation(
            name="checklist_class_name",
            value=lb_types.Checklist(
                answer=[lb_types.ClassificationAnswer(name="first_checklist_answer", confidence=0.5)]
            )
        )
    ]
)
)

coord_object_checklist = {
    "coordinates": [
       [
            [
                -99.210266,
                19.39540372195134
            ],
            [
                -99.210266,
                19.396901
            ],
            [
                -99.20621067903966,
                19.396901
            ],
            [
                -99.20621067903966,
                19.39540372195134
            ],
            [
                -99.210266,
                19.39540372195134
            ]
      ]
    ]
}

bbox_with_checklist_subclass_ndjson = {
    "name": "bbox_checklist_geo",
    "confidence": 0.5,
    "classifications": [{
        "name": "checklist_class_name",
        "answer": [
            { "name":"first_checklist_answer", "confidence": 0.5}
        ]
    }],
    "bbox": {
        'top': coord_object_checklist["coordinates"][0][1][1],
        'left': coord_object_checklist["coordinates"][0][1][0],
        'height': coord_object_checklist["coordinates"][0][3][1] - coord_object_checklist["coordinates"][0][1][1],
        'width': coord_object_checklist["coordinates"][0][3][0] - coord_object_checklist["coordinates"][0][1][0]
    }
}

Bounding box with nested free-text classification

bbox_with_free_text_subclass = lb_types.ObjectAnnotation(
    name="bbox_text_geo",
    value=lb_types.Rectangle(
        start=lb_types.Point(x=-99.21019613742828, y=19.397447957052933), # Top left
        end=lb_types.Point(x=-99.20986354351044, y=19.39772119262215), # Bottom right
    ),
    classifications=[
        lb_types.ClassificationAnnotation(
            name="free_text_geo",
            value=lb_types.Text(answer="sample text")
        )
    ]
)

coord_object_text ={
    "coordinates": [
      [
        [
            -99.21019613742828,
            19.397447957052933
        ],
        [
            -99.21019613742828,
            19.39772119262215
        ],
        [
            -99.20986354351044,
            19.39772119262215
        ],
        [
            -99.20986354351044,
            19.397447957052933
        ],
        [
            -99.21019613742828,
            19.397447957052933
        ]
      ]
    ]
}

bbox_with_free_text_subclass_ndjson = {
    "name":"bbox_text_geo",
    "confidence": 0.5,
    "classifications": [{
        "name": "free_text_geo",
        "confidence": 0.5,
        "answer": "sample text"
    }],
    "bbox": {
        'top': coord_object_text["coordinates"][0][1][1],
        'left': coord_object_text["coordinates"][0][1][0],
        'height': coord_object_text["coordinates"][0][3][1] - coord_object_text["coordinates"][0][1][1],
        'width': coord_object_text["coordinates"][0][3][0] - coord_object_text["coordinates"][0][1][0]
    }
}

Classification: Checklist (multi-choice)

checklist_prediction = lb_types.ClassificationAnnotation(
    name="checklist_question_geo",
    confidence = 0.5,
    value=lb_types.Checklist(answer = [
        lb_types.ClassificationAnswer(name = "first_checklist_answer", confidence = 0.5),
        lb_types.ClassificationAnswer(name = "second_checklist_answer", confidence = 0.5),
        lb_types.ClassificationAnswer(name = "third_checklist_answer", confidence = 0.5)
    ])
)

checklist_prediction_ndjson = {
  'name': 'checklist_question_geo',
  "confidence": 0.5,
  'answer': [
    {'name': 'first_checklist_answer', "confidence": 0.5},
    {'name': 'second_checklist_answer', "confidence": 0.5},
    {'name': 'third_checklist_answer', "confidence": 0.5},
  ]
}

Classification: Nested radio

nested_radio_prediction = lb_types.ClassificationAnnotation(
name="nested_radio_question",
value=lb_types.Radio(
    answer=lb_types.ClassificationAnswer(
    name="first_radio_answer",
    confidence=0.5,
    classifications=[
        lb_types.ClassificationAnnotation(
        name="sub_radio_question",
        value=lb_types.Radio(
            answer=lb_types.ClassificationAnswer(
            name="first_sub_radio_answer",
            confidence=0.2
            )
        )
        )
    ]
    )
)
)

nested_radio_prediction_ndjson= {
  "name": "nested_radio_question",
  "answer": {
      "name": "first_radio_answer",
      "confidence": 0.2,
      "classifications": [{
          "name":"sub_radio_question",
          "answer": { "name" : "first_sub_radio_answer", "confidence": 0.3}
        }]
    }
}

Classification: Nested checklist

nested_checklist_prediction = lb_types.ClassificationAnnotation(
  name="nested_checklist_question",
  value=lb_types.Checklist(
    answer=[lb_types.ClassificationAnswer(
      name="first_checklist_answer",
      confidence=0.5,
      classifications=[
        lb_types.ClassificationAnnotation(
          name="sub_checklist_question",
          value=lb_types.Checklist(
            answer=[lb_types.ClassificationAnswer(
            name="first_sub_checklist_answer",
            confidence=0.5
          )]
        ))
      ]
    )]
  )
)

nested_checklist_prediction_ndjson = {
  "name": "nested_checklist_question",
  "answer": [{
      "name": "first_checklist_answer",
      "confidence":0.5,
      "classifications" : [
        {
          "name": "sub_checklist_question",
          "answer": {"name": "first_sub_checklist_answer", "confidence": 0.5}
        }
      ]
  }]
}

End-to-end example: Upload predictions to a model run

Follow the steps below to upload predictions to a model run.

Before you start

You will need to import these libraries to use the code examples in this section.

import os
import uuid
import numpy as np
from PIL import Image
import cv2

import labelbox as lb
import labelbox.types as lb_types

Replace the value of API_KEY with a valid API key to connect to the Labelbox client.

API_KEY = None
client = lb.Client(API_KEY)

Step 1: Import data rows into Catalog

top_left_bound = lb_types.Point(x=-99.21052827588443, y=19.400498983095076)
bottom_right_bound = lb_types.Point(x=-99.20534818927473, y=19.39533555271248)

epsg = lb_types.EPSG.EPSG4326
bounds = lb_types.TiledBounds(epsg=epsg, bounds=[top_left_bound, bottom_right_bound])
global_key = "mexico_city"

tile_layer = lb_types.TileLayer(
    url="https://s3-us-west-1.amazonaws.com/lb-tiler-layers/mexico_city/{z}/{x}/{y}.png"
)

tiled_image_data = lb_types.TiledImageData(tile_layer=tile_layer,
                                  tile_bounds=bounds,
                                  zoom_levels=[17, 23])

asset = {
    "row_data": tiled_image_data.asdict(),
    "global_key": global_key,
    "media_type": "TMS_GEO"
}

dataset = client.create_dataset(name="geo_demo_dataset")
task= dataset.create_data_rows([asset])
print("Errors:",task.errors)
print("Failed data rows:", task.failed_data_rows)

Step 2: Create/select an ontology for your model predictions

Your model run should have the correct ontology set up with all the tools and classifications supported for your predictions. Here is an example of creating an ontology programmatically for all the example predictions above:

ontology_builder = lb.OntologyBuilder(
    tools=[
        lb.Tool(tool=lb.Tool.Type.POINT, name="point_geo"),
        lb.Tool(tool=lb.Tool.Type.LINE, name="polyline_geo"),
        lb.Tool(tool=lb.Tool.Type.POLYGON, name="polygon_geo"),
        lb.Tool(tool=lb.Tool.Type.POLYGON, name="polygon_geo_2"),
        lb.Tool(tool=lb.Tool.Type.BBOX, name="bbox_geo"),
        lb.Tool(
          tool=lb.Tool.Type.BBOX,
          name="bbox_checklist_geo",
          classifications=[
                lb.Classification(
                    class_type=lb.Classification.Type.CHECKLIST,
                    name="checklist_class_name",
                    options=[
                      lb.Option(value="first_checklist_answer")
                    ]
                ),
            ]
          ),
        lb.Tool(
          tool=lb.Tool.Type.BBOX,
          name="bbox_text_geo",
          classifications=[
                lb.Classification(
                    class_type=lb.Classification.Type.TEXT,
                    name="free_text_geo"
                ),
            ]
          )
      ],
      classifications = [
          lb.Classification(
              class_type=lb.Classification.Type.CHECKLIST,
              name="checklist_question_geo",
              options=[
                  lb.Option(value="first_checklist_answer"),
                  lb.Option(value="second_checklist_answer"),
                  lb.Option(value="third_checklist_answer")
              ]
          ),
          lb.Classification(
              class_type=lb.Classification.Type.RADIO,
              name="radio_question_geo",
              options=[
                  lb.Option(value="first_radio_answer")
              ]
          ),

        lb.Classification(
          class_type=lb.Classification.Type.RADIO,
          name="nested_radio_question",
          options=[
            lb.Option(value="first_radio_answer",
              options=[
                  lb.Classification(
                    class_type=lb.Classification.Type.RADIO,
                    name="sub_radio_question",
                    options=[
                      lb.Option(value="first_sub_radio_answer")
                    ]
                ),
              ]
            ),
          ],
        ),
        lb.Classification(
          class_type=lb.Classification.Type.CHECKLIST,
          name="nested_checklist_question",
          options=[
              lb.Option("first_checklist_answer",
                options=[
                  lb.Classification(
                      class_type=lb.Classification.Type.CHECKLIST,
                      name="sub_checklist_question",
                      options=[lb.Option("first_sub_checklist_answer")]
                  )
              ]
            )
          ]
      )
    ]

)

ontology = client.create_ontology("Ontology Geospatial Annotations",
ontology_builder.asdict(),
media_type=lb.MediaType.Geospatial_Tile)

Step 3: Create a Model and model run

# create Model
model = client.create_model(name="geospatial_model_run_" + str(uuid.uuid4()),
                            ontology_id=ontology.uid)
# create model run
model_run = model.create_model_run("iteration 1")

Step 4: Send data rows to the model run

model_run.upsert_data_rows(global_keys=[global_key])

Step 5: Create the predictions payload

Create the annotations payload using the snippets of code shown above. Labelbox supports two formats for the annotations payload: NDJSON and Python annotation types. Both approaches are described below with instructions to compose annotations into Labels attached to the data rows. The resulting labels and ndjson_labels from each approach will include every annotation (created above) supported by the respective method.

labels =[]
labels.append(
    lb_types.Label(
         data={
            "global_key": global_key,
            "tile_layer": tile_layer,
            "tile_bounds":bounds,
            "zoom_levels": [12, 20]
        },
        annotations = [
            point_prediction,
            polyline_prediction,
            polygon_prediction,
            bbox_prediction,
            radio_prediction,
            bbox_with_checklist_subclass,
            bbox_with_free_text_subclass,
            checklist_prediction,
            nested_checklist_prediction,
            nested_radio_prediction
        ]
    )
)

label_ndjson = []
for prediction in [
    radio_prediction_ndjson,
    checklist_prediction_ndjson,
    bbox_with_free_text_subclass_ndjson,
    bbox_with_checklist_subclass_ndjson,
    bbox_prediction_ndjson,
    point_prediction_ndjson,
    polyline_prediction_ndjson,
    polygon_prediction_ndjson,
    nested_checklist_prediction_ndjson,
    nested_radio_prediction_ndjson
]:
  prediction.update({
      'dataRow': {'globalKey': global_key},
  })
  label_ndjson.append(prediction)

Step 6: Upload the predictions payload to the Model Run

# Upload the prediction label to the Model Run
upload_job_prediction = model_run.add_predictions(
    name="prediction_upload_job"+str(uuid.uuid4()),
    predictions=labels)

# Errors will appear for annotation uploads that failed.

print("Errors:", upload_job_prediction.errors)
print("Status of uploads: ", upload_job_prediction.statuses)

Step 7: Send annotations to the Model Run (Optional)

# 7.1. Create a labelbox project

# Create a Labelbox project
project = client.create_project(name="geospatial_prediction_demo",                                    
                                    queue_mode=lb.QueueMode.Batch,
                                    media_type=lb.MediaType.Geospatial_Tile)
project.connect_ontology(ontology)

# 7.2. Create a batch to send to the project

project.create_batch(
  "batch_geospatial_prediction_demo", # Each batch in a project must have a unique name
  global_keys=[global_key], # A list of data rows or data row ids
  priority=5 # priority between 1(Highest) - 5(lowest)
)


# 7.3 Create the annotations payload as explained in:
# https://docs.labelbox.com/reference/import-geospatial-annotations#supported-annotations
point_annotation...
polyline_annotation...
polygon_annotation...
bbox_annotation...
radio_annotation...
bbox_with_checklist_subclass...
bbox_with_free_text_subclass...
checklist_annotation...
nested_radio_annotation...
nested_checklist_annotation...

# 7.4 Create the label object 
labels =[]
labels.append(
    lb_types.Label(
        data={
            "global_key": global_key,
            "tile_layer": tile_layer,
            "tile_bounds":bounds,
            "zoom_levels": [12, 20]
        },
        annotations = [
            point_annotation,
            polyline_annotation,
            polygon_annotation,
            bbox_annotation,
            radio_annotation,
            bbox_with_checklist_subclass,  
            bbox_with_free_text_subclass,
            checklist_annotation,
            nested_checklist_annotation, 
            nested_radio_annotation
        ]
    )
) 

# 7.5 Upload annotations to the project using Label Import
upload_job_annotation = lb.LabelImport.create_from_objects(
    client = client,
    project_id = project.uid,
    name="geospatial_annotations_import_" + str(uuid.uuid4()),
    labels=labels)

upload_job_annotation.wait_until_done()
# Errors will appear for annotation uploads that failed.
print("Errors:", upload_job_annotation.errors)
print("Status of uploads: ", upload_job_annotation.statuses)

# 7.6. Send the annotations to the Model Run

# get the labels id from the project
model_run.upsert_labels(project_id=project.uid)

Getting started

Annotate

Model

Catalog

Schema

Import/Export

Integrations

Manage Team

Examples

Upload geospatial predictions

Open In Colab

GitHub

Supported predictions

Uploading confidence scores is optional

Point

Polyline

Polygon

Bounding box

Classification: Radio (single choice)

Bounding box with nested checklist classification

Bounding box with nested free-text classification

Classification: Checklist (multi-choice)

Classification: Nested radio

Classification: Nested checklist

End-to-end example: Upload predictions to a model run

Before you start

Step 1: Import data rows into Catalog

Step 2: Create/select an ontology for your model predictions

Step 3: Create a Model and model run

Step 4: Send data rows to the model run

Step 5: Create the predictions payload

Step 6: Upload the predictions payload to the Model Run

Step 7: Send annotations to the Model Run (Optional)

Open In Colab

GitHub

​Supported predictions

​Uploading confidence scores is optional

​Point

​Polyline

​Polygon

​Bounding box

​Classification: Radio (single choice)

​Bounding box with nested checklist classification

​Bounding box with nested free-text classification

​Classification: Checklist (multi-choice)

​Classification: Nested radio

​Classification: Nested checklist

​End-to-end example: Upload predictions to a model run

​Before you start

​Step 1: Import data rows into Catalog

​Step 2: Create/select an ontology for your model predictions

​Step 3: Create a Model and model run

​Step 4: Send data rows to the model run

​Step 5: Create the predictions payload

​Step 6: Upload the predictions payload to the Model Run

​Step 7: Send annotations to the Model Run (Optional)

Supported predictions

Uploading confidence scores is optional

Point

Polyline

Polygon

Bounding box

Classification: Radio (single choice)

Bounding box with nested checklist classification

Bounding box with nested free-text classification

Classification: Checklist (multi-choice)

Classification: Nested radio

Classification: Nested checklist

End-to-end example: Upload predictions to a model run

Before you start

Step 1: Import data rows into Catalog

Step 2: Create/select an ontology for your model predictions

Step 3: Create a Model and model run

Step 4: Send data rows to the model run

Step 5: Create the predictions payload

Step 6: Upload the predictions payload to the Model Run

Step 7: Send annotations to the Model Run (Optional)