> ## Documentation Index
> Fetch the complete documentation index at: https://docs.labelbox.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Import document annotations
> Developer guide for importing annotations on document (PDF) data and sample import formats.
## Overview
To import annotations in Labelbox, you need to create an annotations payload. In this section, we provide this payload for every supported annotation type.
### Annotation payload types
Labelbox supports two formats for the annotations payload:
* Python annotation types (recommended)
* Provides a seamless transition between third-party platforms, machine learning pipelines, and Labelbox.
* Allows you to build annotations locally with local file paths, numpy arrays, or URLs
* Easily convert Python Annotation Type format to NDJSON format to quickly import annotations to Labelbox
* Supports one-level nested classification (radio, checklist, or free-form text) under a tool or classification annotation.
* JSON
* Skips formatting annotation payload in the Labelbox Python annotation type
* Supports any levels of nested classification (radio, checklist, or free-form text) under a tool or classification annotation.
### Label import types
Labelbox additionally supports two types of label imports:
* [Model-assisted labeling (MAL)](/docs/model-assisted-labeling)
* This workflow allows you to import computer-generated predictions (or simply annotations created outside of Labelbox) as pre-labels on an asset.
* [Ground truth](https://www.google.com/url?q=https%3A%2F%2Fdocs.labelbox.com%2Fdocs%2Fimport-ground-truth)
* This workflow functionality allows you to bulk import your ground truth annotations from an external or third-party labeling system into Labelbox *Annotate*. Using the label import API to import external data is a useful way to consolidate and migrate all annotations into Labelbox as a single source of truth.
## Supported annotations
The following annotations are supported for a document data row:
* Radio
* Checklist
* Free-form text
* Bounding box
* Entity
* Relationship
### Annotations are global and page based
Classification annotations are only supported globally while page-specific tool annotations are supported at a per page basis.
## Classifications
### Radio (single-choice, global)
```python Python annotation theme={null}
radio_annotation = lb_types.ClassificationAnnotation(
name="radio_question",
value=lb_types.Radio(answer =
lb_types.ClassificationAnswer(name = "first_radio_answer")
)
)
```
```python NDJSON theme={null}
radio_annotation_ndjson = {
"name": "radio_question",
"answer": {"name": "first_radio_answer"}
}
```
### Checklist (multi-choice, global)
```python Python annotation theme={null}
checklist_annotation = lb_types.ClassificationAnnotation(
name="checklist_question",
value=lb_types.Checklist(answer = [
lb_types.ClassificationAnswer(name = "first_checklist_answer"),
lb_types.ClassificationAnswer(name = "second_checklist_answer")
])
)
```
```python NDJSON theme={null}
checklist_annotation_ndjson = {
"name": "checklist_question",
"answer": [
{"name": "first_checklist_answer"},
{"name": "second_checklist_answer"}
]
}
```
### Free-form text (global)
```python Python annotation theme={null}
text_annotation = lb_types.ClassificationAnnotation(
name="free_text", # must match your ontology feature"s name
value=lb_types.Text(answer="sample text")
)
```
```python NDJSON theme={null}
text_annotation_ndjson = {
"name": "free_text",
"answer": "sample text"
}
```
### Nested Classifications (global)
```python Python annotation theme={null}
nested_checklist_annotation = lb_types.ClassificationAnnotation(
name="nested_checklist_question",
value=lb_types.Checklist(
answer=[lb_types.ClassificationAnswer(
name="first_checklist_answer",
classifications=[
lb_types.ClassificationAnnotation(
name="sub_checklist_question",
value=lb_types.Checklist(
answer=[lb_types.ClassificationAnswer(
name="first_sub_checklist_answer"
)]
))
]
)]
)
)
nested_radio_annotation = lb_types.ClassificationAnnotation(
name="nested_radio_question",
value=lb_types.Radio(
answer=lb_types.ClassificationAnswer(
name="first_radio_answer",
classifications=[
lb_types.ClassificationAnnotation(
name="sub_radio_question",
value=lb_types.Radio(
answer=lb_types.ClassificationAnswer(
name="first_sub_radio_answer"
)
)
)
]
)
)
)
```
```python NDJSON theme={null}
nested_checklist_annotation_ndjson = {
"name": "nested_checklist_question",
"answer": [{
"name": "first_checklist_answer",
"classifications" : [
{
"name": "sub_checklist_question",
"answer": {"name": "first_sub_checklist_answer"}
}
]
}]
}
nested_radio_annotation_ndjson = {
"name": "nested_radio_question",
"answer": {
"name": "first_radio_answer",
"classifications": [{
"name":"sub_radio_question",
"answer": { "name" : "first_sub_radio_answer"}
}]
}
}
```
## Tools
### Bounding box (page specific)
When importing a bounding box annotation, you need to specify its `DocumentRectangle` value, which defines its precise area on a document page. You can choose from the following `RectangleUnit` options, which determine the measurement unit used to define the dimensions and coordinates of the bounding box:
* INCHES
* PIXELS
* POINTS
* PERCENT
```python Python annotation theme={null}
bbox_annotation = lb_types.ObjectAnnotation(
name="bounding_box", # must match your ontology feature's name
value=lb_types.DocumentRectangle(
start=lb_types.Point(x=102.771, y=135.3), # x = left, y = top
end=lb_types.Point(x=518.571, y=245.143), # x = left + width , y = top + height
page=0,
unit=lb_types.RectangleUnit.POINTS
)
)
```
```python NDJSON theme={null}
bbox_annotation_ndjson = {
"name": "bounding_box",
"bbox": {
"top": 135.3,
"left": 102.771,
"height": 109.843,
"width": 415.8
},
"page": 0,
"unit": "POINTS"
}
```
### Entity (page specific)
`textSelections` is the payload required for each entity annotation. Each`textSelections` item in the list requires the following fields:
* The`groupId` associated with a group of words.
* A list of `tokenIds` for each word in the group of words.
* The `page` of the document (1-indexed).
Both `tokenIds` and `groupdId` can extracted from the text layer URL attached to the data row. For more information on text layers, visit our [import document data guide](/reference/import-document-data).
```python Python annotation theme={null}
entities_annotations = lb_types.ObjectAnnotation(
name="named_entity",
value= lb_types.DocumentEntity(
name="named_entity",
textSelections=[
lb_types.DocumentTextSelection(
token_ids=[],
group_id="",
page=1
)
]
)
)
```
```python NDJSON theme={null}
entities_annotations_ndjson = {
"name": "named_entity",
"textSelections": [
{
"tokenIds": [
"", ## ids associated with each word in a group
],
"groupId": "", ## id associated with a group of words
"page": 1,
}
]
}
```
### Tool with nested classifications (page specific)
```python Bounding Box Example theme={null}
bbox_with_radio_subclass_annotation = lb_types.ObjectAnnotation(
name="bbox_with_radio_subclass",
value=lb_types.DocumentRectangle(
start=lb_types.Point(x=317.271, y=226.757), # x = left, y = top
end=lb_types.Point(x=566.657, y=420.986), # x = left + width , y = top + height
unit=lb_types.RectangleUnit.POINTS,
page=1
),
classifications=[
lb_types.ClassificationAnnotation(
name="sub_radio_question",
value=lb_types.Radio(
answer=lb_types.ClassificationAnswer(
name="first_sub_radio_answer",
classifications=[
lb_types.ClassificationAnnotation(
name="second_sub_radio_question",
value=lb_types.Radio(
answer=lb_types.ClassificationAnswer(
name="second_sub_radio_answer"
)
)
)
]
)
)
)
]
)
```
```python Entity Example theme={null}
ner_with_checklist_subclass_annotation = lb_types.ObjectAnnotation(
name="ner_with_checklist_subclass",
value=lb_types.DocumentEntity(
name="ner_with_checklist_subclass",
text_selections=[
lb_types.DocumentTextSelection(
token_ids=[],
group_id="",
page=1
)
]
),
classifications=[
lb_types.ClassificationAnnotation(
name="sub_checklist_question",
value=lb_types.Checklist(
answer=[lb_types.ClassificationAnswer(name="first_sub_checklist_answer")]
)
)
]
)
```
```python NDJSON theme={null}
bbox_with_radio_subclass_annotation_ndjson = {
"name": "bbox_with_radio_subclass",
"classifications": [
{
"name": "sub_radio_question",
"answer": {
"name": "first_sub_radio_answer",
"classifications": [
{
"name": "second_sub_radio_question",
"answer": {
"name": "second_sub_radio_answer"}
}
]
}
}
],
"bbox": {
"top": 226.757,
"left": 317.271,
"height": 194.229,
"width": 249.386
},
"page": 1,
"unit": "POINTS"
}
```
## Relationships (page specific)
```python Bounding Box Example theme={null}
bbox_source = lb_types.ObjectAnnotation(
name="bounding_box",
value=lb_types.DocumentRectangle(
start=lb_types.Point(x=188.257, y=68.875), # x = left, y = top
end=lb_types.Point(x=270.907, y=149.556), # x = left + width , y = top + height
unit=lb_types.RectangleUnit.POINTS,
page=1
),
)
bbox_target = lb_types.ObjectAnnotation(
name="bounding_box",
value=lb_types.DocumentRectangle(
start=lb_types.Point(x=96.424, y=66.251),
end=lb_types.Point(x=179.074, y=146.932),
unit=lb_types.RectangleUnit.POINTS,
page=1
),
)
bbox_relationship = lb_types.RelationshipAnnotation(
name="relationship",
value=lb_types.Relationship(
source=bbox_source, # Alternatively, specify a source_ontology_name
target=bbox_target,
type=lb_types.Relationship.Type.UNIDIRECTIONAL,
))
```
```python Entity Example theme={null}
entity_source = lb_types.ObjectAnnotation(
name="named_entity",
value= lb_types.DocumentEntity(
name="named_entity",
textSelections=[
lb_types.DocumentTextSelection(
token_ids=[],
group_id="",
page=1
)
]
)
)
entity_target = lb_types.ObjectAnnotation(
name="named_entity",
value=lb_types.DocumentEntity(
name="named_entity",
textSelections=[
lb_types.DocumentTextSelection(
token_ids=[],
group_id="",
page=1
)
]
)
)
entity_relationship = lb_types.RelationshipAnnotation(
name="relationship",
value=lb_types.Relationship(
source=entity_source, # Alternatively, specify a source_ontology_name
target=entity_target,
type=lb_types.Relationship.Type.UNIDIRECTIONAL,
))
```
```python NDJSON theme={null}
uuid_source = str(uuid.uuid4())
uuid_target = str(uuid.uuid4())
entity_source_ndjson = {
"name": "named_entity",
"uuid": uuid_source,
"textSelections": [
{
"tokenIds": [
""
],
"groupId": "",
"page": 1
}
]
}
entity_target_ndjson = {
"name": "named_entity",
"uuid": uuid_target,
"textSelections": [
{
"tokenIds": [
""
],
"groupId": "",
"page": 1
}
]
}
ner_relationship_annotation_ndjson = {
"name": "relationship",
"relationship": {
"source": uuid_source, # UUID reference to the source annotation.
"target": uuid_target, # UUID reference to the target annotation
"type": "unidirectional"
}
}
```
### Read-only relationships
### This is a [beta](/docs/product-release-phases#beta) feature
Admins can mark relationships as read\_only to prevent other roles from deleting them:
```python Python theme={null}
bbox_source = lb_types.ObjectAnnotation(
name="bounding_box",
value=lb_types.DocumentRectangle(
start=lb_types.Point(x=188.257, y=68.875), # x = left, y = top
end=lb_types.Point(x=270.907, y=149.556), # x = left + width , y = top + height
unit=lb_types.RectangleUnit.POINTS,
page=1
),
)
bbox_target = lb_types.ObjectAnnotation(
name="bounding_box",
value=lb_types.DocumentRectangle(
start=lb_types.Point(x=96.424, y=66.251),
end=lb_types.Point(x=179.074, y=146.932),
unit=lb_types.RectangleUnit.POINTS,
page=1
),
)
bbox_relationship = lb_types.RelationshipAnnotation(
name="relationship",
value=lb_types.Relationship(
source=bbox_source, # Alternatively, specify a source_ontology_name
target=bbox_target,
type=lb_types.Relationship.Type.UNIDIRECTIONAL,
readonly=True,
))
```
### Relationships between classifications and objects
### This is a [beta](/docs/product-release-phases#beta) feature
```python Linking multiple objects to classifications expandable theme={null}
radio_annotation = lb_types.ClassificationAnnotation(
name="radio_pdf",
value=lb_types.Radio(
answer=lb_types.ClassificationAnswer(name="second_radio_answer")
),
)
checklist_annotation = lb_types.ClassificationAnnotation(
name="checklist_pdf", # must match your ontology feature's name
value=lb_types.Checklist(
answer=[
lb_types.ClassificationAnswer(name="first_checklist_answer"),
lb_types.ClassificationAnswer(name="second_checklist_answer"),
]
),
)
text_annotation = lb_types.ClassificationAnnotation(
name="text_pdf", value=lb_types.Text(answer="sample text")
)
bbox_annotation = lb_types.ObjectAnnotation(
name="bounding_box",
value=lb_types.DocumentRectangle(
start=lb_types.Point(x=0.5, y=0.5), # x = left, y = top
end=lb_types.Point(x=0.7, y=0.7), # x= left + width , y = top + height
page=1,
unit=RectangleUnit.PERCENT,
),
)
bbox_annotation_2 = lb_types.ObjectAnnotation(
name="bounding_box",
value=lb_types.DocumentRectangle(
start=lb_types.Point(x=0.5, y=0.5),
end=lb_types.Point(x=0.7, y=0.7),
page=2,
unit=RectangleUnit.PERCENT,
),
)
entity_annotation = lb_types.ObjectAnnotation(
name="named_entity",
value=lb_types.DocumentEntity(
name="named_entity",
textSelections=[
lb_types.DocumentTextSelection(
token_ids=["9763b68b-9ea7-472d-af40-20ab69fbee71"],
group_id="ee108a07-059a-4302-910f-b16eb148f123",
page=1,
)
],
),
)
text_bbox_relationship = lb_types.RelationshipAnnotation(
name="relationship",
value=lb_types.Relationship(
source=text_annotation, # Alternatively, specify a source_ontology_name
target=bbox_annotation,
type=lb_types.Relationship.Type.UNIDIRECTIONAL,
),
)
radio_bbox_relationship = lb_types.RelationshipAnnotation(
name="relationship",
value=lb_types.Relationship(
source=radio_annotation, # Alternatively, specify a source_ontology_name
target=bbox_annotation,
type=lb_types.Relationship.Type.UNIDIRECTIONAL,
),
)
checklist_bbox_relationship = lb_types.RelationshipAnnotation(
name="relationship",
value=lb_types.Relationship(
source=checklist_annotation, # Alternatively, specify a source_ontology_name
target=bbox_annotation,
type=lb_types.Relationship.Type.UNIDIRECTIONAL,
),
)
text_entity_relationship = lb_types.RelationshipAnnotation(
name="relationship",
value=lb_types.Relationship(
source=text_annotation, # Alternatively, specify a source_ontology_name
target=entity_annotation,
type=lb_types.Relationship.Type.UNIDIRECTIONAL,
),
)
bbox_bbox2_relationship = lb_types.RelationshipAnnotation(
name="relationship",
value=lb_types.Relationship(
source=bbox_annotation, # Alternatively, specify a source_ontology_name
target=bbox_annotation_2,
type=lb_types.Relationship.Type.UNIDIRECTIONAL,
),
)
```
```python NDJSON example theme={null}
uuid_radio = str(uuid.uuid4())
uuid_checklist = str(uuid.uuid4())
uuid_text = str(uuid.uuid4())
uuid_bbox_1 = str(uuid.uuid4())
uuid_bbox_2 = str(uuid.uuid4())
uuid_entity = str(uuid.uuid4())
# Radio annotation
radio_annotation_ndjson = {
"name": "radio_pdf",
"radio": {
"answer": {
"name": "second_radio_answer"
}
}
}
# Checklist annotation
checklist_annotation_ndjson = {
"name": "checklist_pdf",
"checklist": {
"answer": [
{"name": "first_checklist_answer"},
{"name": "second_checklist_answer"}
]
}
}
# Text annotation
text_annotation_ndjson = {
"name": "text_pdf",
"text": {
"answer": "sample text"
}
}
# Bounding box annotation 1
bbox_annotation_ndjson = {
"name": "bounding_box",
"documentRectangle": {
"start": {"x": 0.5, "y": 0.5},
"end": {"x": 0.7, "y": 0.7},
"page": 1,
"unit": "PERCENT"
}
}
# Bounding box annotation 2
bbox_annotation_2_ndjson = {
"name": "bounding_box",
"documentRectangle": {
"start": {"x": 0.5, "y": 0.5},
"end": {"x": 0.7, "y": 0.7},
"page": 2,
"unit": "PERCENT"
}
}
# Named entity annotation
entity_annotation_ndjson = {
"name": "named_entity",
"documentEntity": {
"name": "named_entity",
"textSelections": [
{
"tokenIds": ["9763b68b-9ea7-472d-af40-20ab69fbee71"],
"groupId": "ee108a07-059a-4302-910f-b16eb148f123",
"page": 1
}
]
}
}
# Relationships
text_bbox_relationship_ndjson = {
"name": "relationship",
"relationship": {
"source": uuid_text,
"target": uuid_bbox_1,
"type": "unidirectional"
}
}
radio_bbox_relationship_ndjson = {
"name": "relationship",
"relationship": {
"source": uuid_radio,
"target": uuid_bbox_1,
"type": "unidirectional"
}
}
checklist_bbox_relationship_ndjson = {
"name": "relationship",
"relationship": {
"source": uuid_checklist,
"target": uuid_bbox_1,
"type": "unidirectional"
}
}
text_entity_relationship_ndjson = {
"name": "relationship",
"relationship": {
"source": uuid_text,
"target": uuid_entity,
"type": "unidirectional"
}
}
bbox_bbox2_relationship_ndjson = {
"name": "relationship",
"relationship": {
"source": uuid_bbox_1,
"target": uuid_bbox_2,
"type": "unidirectional"
}
}
```
## Creating text entity annotation
For importing entity annotations, you can use your own `text_layer_url` or a Labelbox-generated `text_layer_url`.
You can get the Labelbox-generated `text_layer_url` by exporting the data row. The below code snippet demonstrates this process by exporting from a data row.
```python Python theme={null}
# Export data row
task = lb.DataRow.export(client=client,global_keys=[global_key])
task.wait_till_done()
if task.has_result():
stream = task.get_buffered_stream()
text_layer = ""
for output in stream:
output_json = output.json
text_layer = output_json['media_attributes']['text_layer_url']
print(text_layer)
```
```python Python theme={null}
import requests
import json
# Helper method for updating text selections
def update_text_selections(annotation, group_id, list_tokens, page):
return annotation.update({
"textSelections": [
{
"groupId": group_id,
"tokenIds": list_tokens,
"page": page
}
]
})
# Fetch the content of the text layer
res = requests.get(text_layer)
# Phrases that we want to annotation obtained from the text layer url
content_phrases = ["Metal-insulator (MI) transitions have been one of the" ,
"T. Sasaki, N. Yoneyama, and N. Kobayashi",,
"Organic charge transfer salts based on the donor",
"the experimental investigations on this issue have not"]
# Parse the text layer
text_selections = []
text_selections_ner = []
text_selections_source = []
text_selections_target = []
for obj in json.loads(res.text):
for group in obj["groups"]:
if group["content"] == content_phrases[0]:
list_tokens = [x["id"] for x in group["tokens"]]
# build text selections for Python Annotation Types
document_text_selection = lb_types.DocumentTextSelection(groupId=group["id"], tokenIds=list_tokens, page=1)
text_selections.append(document_text_selection)
# build text selection for the NDJson annotations
update_text_selections(annotation=entities_annotations_ndjson,
group_id=group["id"], # id representing group of words
list_tokens=list_tokens, # ids representing individual words from the group
page=1)
if group["content"] == content_phrases[1]:
list_tokens_2 = [x["id"] for x in group["tokens"]]
# build text selections for Python Annotation Types
ner_text_selection = lb_types.DocumentTextSelection(groupId=group["id"], tokenIds=list_tokens_2, page=1)
text_selections_ner.append(ner_text_selection)
# build text selection for the NDJson annotations
update_text_selections(annotation=ner_with_checklist_subclass_annotation_ndjson,
group_id=group["id"], # id representing group of words
list_tokens=list_tokens_2, # ids representing individual words from the group
page=1)
if group["content"] == content_phrases[2]:
relationship_source = [x["id"] for x in group["tokens"]]
# build text selections for Python Annotation Types
text_selection_entity_source = lb_types.DocumentTextSelection(groupId=group["id"], tokenIds=relationship_source, page=1)
text_selections_source.append(text_selection_entity_source)
# build text selection for the NDJson annotations
update_text_selections(annotation=entity_source_ndjson,
group_id=group["id"], # id representing group of words
list_tokens=relationship_source, # ids representing individual words from the group
page=1)
if group["content"] == content_phrases[3]:
relationship_target = [x["id"] for x in group["tokens"]]
# build text selections for Python Annotation Types
text_selection_entity_target = lb_types.DocumentTextSelection(group_id=group["id"], tokenIds=relationship_target, page=1)
text_selections_target.append(text_selection_entity_target)
# build text selections forthe NDJson annotations
update_text_selections(annotation=entity_target_ndjson,
group_id=group["id"], # id representing group of words
list_tokens=relationship_target, # ids representing individual words from the group
page=1)
```
Re-write the Python annotations to include text selections (only required for Python annotation types)
```python theme={null}
# re-write the entity annotation with text selections
entities_annotation_document_entity = lb_types.DocumentEntity(name="named_entity", textSelections = text_selections)
entities_annotation = lb_types.ObjectAnnotation(name="named_entity",value=entities_annotation_document_entity)
# re-write the entity annotation + subclassification with text selections
classifications = [
lb_types.ClassificationAnnotation(
name="sub_checklist_question",
value=lb_types.Checklist(
answer=[lb_types.ClassificationAnswer(name="first_sub_checklist_answer")]
)
)
]
ner_annotation_with_subclass = lb_types.DocumentEntity(name="ner_with_checklist_subclass", textSelections= text_selections_ner)
ner_with_checklist_subclass_annotation = lb_types.ObjectAnnotation(name="ner_with_checklist_subclass",
value=ner_annotation_with_subclass,
classifications=classifications)
# re-write the entity source and target annotations withe text selectios
entity_source_doc = lb_types.DocumentEntity(name="named_entity", text_selections= text_selections_source)
entity_source = lb_types.ObjectAnnotation(name="named_entity", value=entity_source_doc)
entity_target_doc = lb_types.DocumentEntity(name="named_entity", text_selections=text_selections_target)
entity_target = lb_types.ObjectAnnotation(name="named_entity", value=entity_target_doc)
# re-write the entity relationship with the re-created entities
entity_relationship = lb_types.RelationshipAnnotation(
name="relationship",
value=lb_types.Relationship(
source=entity_source,
target=entity_target,
type=lb_types.Relationship.Type.UNIDIRECTIONAL,
))
print(f"entities_annotations_ndjson={entities_annotations_ndjson}")
print(f"entities_annotation={entities_annotation}")
print(f"nested_entities_annotation_ndjson={ner_with_checklist_subclass_annotation_ndjson}")
print(f"nested_entities_annotation={ner_with_checklist_subclass_annotation}")
print(f"entity_source_ndjson={entity_source_ndjson}")
print(f"entity_target_ndjson={entity_target_ndjson}")
print(f"entity_source={entity_source}")
print(f"entity_target={entity_target}")
```
## Example: Import pre-labels or ground truths
The steps to import annotations as pre-labels (machine-assisted learning) are similar to those to import annotations as ground truth labels. However, they vary slightly, and we will describe the differences for each scenario.
### Before you start
The below imports are needed to use the code examples in this section.
```python Python theme={null}
import uuid
import json
import labelbox as lb
import labelbox.types as lb_types
```
Replace the value of `API_KEY` with a valid [API key](/reference/create-api-key) to connect to the Labelbox client.
```python Python theme={null}
API_KEY = None
client = lb.Client(API_KEY)
```
### Step 1: Import data rows
Data rows must first be uploaded to **Catalog** to attach annotations.
This example shows how to create a data row in **Catalog** by attaching it to a [dataset](/reference/dataset) .
```python Python theme={null}
global_key = "0801.3483.pdf"
img_url = {
"row_data": {
"pdf_url": "https://storage.googleapis.com/labelbox-datasets/arxiv-pdf/data/99-word-token-pdfs/0801.3483.pdf",
},
"global_key": global_key
}
dataset = client.create_dataset(name="pdf_demo_dataset")
task = dataset.create_data_rows([img_url])
task.wait_till_done()
print(f"Failed data rows: {task.failed_data_rows}")
print(f"Errors: {task.errors}")
```
### Step 2: Set up ontology
Your project ontology should support the tools and classifications required by your annotations. To ensure accurate schema feature mapping, the value used as the `name` parameter should match the value of the `name` field in your annotation.
For example, when we created an annotation above, we provided a name`annotation_name`. Now, when we set up our ontology, we must ensure that the name of our bounding box tool is also `anotations_name`. The same alignment must hold true for the other tools and classifications we create in our ontology.
This example shows how to create an ontology containing all supported [annotation types](#supported-annotations) .
```python Python expandable theme={null}
ontology_builder = lb.OntologyBuilder(
classifications=[ # List of Classification objects
lb.Classification(
class_type=lb.Classification.Type.RADIO,
name="radio_question",
scope = lb.Classification.Scope.GLOBAL,
options=[
lb.Option(value="first_radio_answer"),
lb.Option(value="second_radio_answer")
]
),
lb.Classification(
class_type=lb.Classification.Type.CHECKLIST,
name="checklist_question",
scope = lb.Classification.Scope.GLOBAL,
options=[
lb.Option(value="first_checklist_answer"),
lb.Option(value="second_checklist_answer")
]
),
lb.Classification(
class_type=lb.Classification.Type.TEXT,
name="free_text",
scope = lb.Classification.Scope.GLOBAL
),
lb.Classification(
class_type=lb.Classification.Type.RADIO,
name="nested_radio_question",
scope = lb.Classification.Scope.GLOBAL,
options=[
lb.Option("first_radio_answer",
options=[
lb.Classification(
class_type=lb.Classification.Type.RADIO,
name="sub_radio_question",
options=[lb.Option("first_sub_radio_answer")]
)
])
]
),
lb.Classification(
class_type=lb.Classification.Type.CHECKLIST,
name="nested_checklist_question",
scope = lb.Classification.Scope.GLOBAL,
options=[
lb.Option("first_checklist_answer",
options=[
lb.Classification(
class_type=lb.Classification.Type.CHECKLIST,
name="sub_checklist_question",
options=[lb.Option("first_sub_checklist_answer")]
)
])
]
),
],
tools=[ # List of Tool objects
lb.Tool( tool=lb.Tool.Type.BBOX,name="bounding_box"),
lb.Tool(tool=lb.Tool.Type.NER, name="named_entity"),
lb.Tool(tool=lb.Tool.Type.RELATIONSHIP,name="relationship"),
lb.Tool(tool=lb.Tool.Type.NER,
name="ner_with_checklist_subclass",
classifications=[
lb.Classification(
class_type=lb.Classification.Type.CHECKLIST,
name="sub_checklist_question",
options=[
lb.Option(value="first_sub_checklist_answer")
]
)
]),
lb.Tool( tool=lb.Tool.Type.BBOX,
name="bbox_with_radio_subclass",
classifications=[
lb.Classification(
class_type=lb.Classification.Type.RADIO,
name="sub_radio_question",
options=[
lb.Option(
value="first_sub_radio_answer" ,
options=[
lb.Classification(
class_type=lb.Classification.Type.RADIO,
name="second_sub_radio_question",
options=[lb.Option("second_sub_radio_answer")]
)]
)]
)]
)]
)
ontology = client.create_ontology("Document Annotation Import Demo",
ontology_builder.asdict(),
media_type=lb.MediaType.Document)
```
### Step 3: Set Up a Labeling Project
```python Python theme={null}
# Create a Labelbox project
project = client.create_project(name="PDF_annotation_demo",
queue_mode=QueueMode.Batch,
media_type=lb.MediaType.Document)
project.connect_ontology(ontology)
```
### Step 4: Send Data Rows to Project
```python Python theme={null}
project.create_batch(
"PDF_annotation_batch", # Each batch in a project must have a unique name
global_keys=[global_key] , # a list of global keys, data rows, or data row ids
priority=5 # priority between 1(highest) - 5(lowest)
)
```
### Step 5: Create annotation payloads
For help understanding annotation payloads, see [overview](#overview). To declare payloads, you can use Python annotation types (*preferred*) or NDJSON objects.
These examples demonstrate each format and how to compose annotations into labels attached to data rows.
```python Python annotation payload theme={null}
# create a Label
labels = []
labels.append(
lb_types.Label(
data={"global_key" : global_key },
annotations = [
entities_annotation,
checklist_annotation,
nested_checklist_annotation,
text_annotation,
radio_annotation,
nested_radio_annotation,
bbox_annotation,
bbox_with_radio_subclass_annotation,
ner_with_checklist_subclass_annotation,
entity_source,
entity_target,
entity_relationship,# Only supported for MAL imports
bbox_source,
bbox_target,
bbox_relationship # Only supported for MAL imports
]
)
)
```
```python NDJSON payload theme={null}
label_ndjson = []
for annot in [
entities_annotations_ndjson,
checklist_annotation_ndjson,
nested_checklist_annotation_ndjson,
text_annotation_ndjson,
radio_annotation_ndjson,
nested_radio_annotation_ndjson,
bbox_annotation_ndjson,
bbox_with_radio_subclass_annotation_ndjson,
ner_with_checklist_subclass_annotation_ndjson,
entity_source_ndjson,
entity_target_ndjson,
ner_relationship_annotation_ndjson, # Only supported for MAL imports
bbox_source_ndjson,
bbox_target_ndjson,
bbox_relationship_annotation_ndjson # Only supported for MAL imports
]:
annot.update({
"dataRow": {"globalKey": global_key},
})
label_ndjson.append(annot)
```
### Step 6: Import annotation payload
For prelabeled (model-assisted labeling) scenarios, pass your payload as the value of the `predictions` parameter. For ground truths, pass the payload to the `labels` parameter.
### Warning
Relationship annotations are not supported for ground truth import jobs.
#### Option A: Upload as [prelabels (model assisted labeling)](/docs/model-assisted-labeling)
This option is helpful for speeding up the initial labeling process and reducing the manual labeling workload for high-volume datasets.
```python MAL import theme={null}
# Upload MAL label for this data row in project
upload_job = lb.MALPredictionImport.create_from_objects(
client = client,
project_id = project.uid,
name="mal_job"+str(uuid.uuid4()),
predictions=label
)
print(f"Errors: {upload_job.errors}", )
print(f"Status of uploads: {upload_job.statuses}")
```
#### Option B: Upload to a labeling project as [ground truth](/docs/import-ground-truth)
This option is helpful for loading high-confidence labels from another platform or previous projects that just need review rather than manual labeling effort.
```python Label import theme={null}
# Upload label for this data row in project
upload_job = lb.LabelImport.create_from_objects(
client = client,
project_id = project.uid,
name="label_import_job"+str(uuid.uuid4()),
labels=label
)
print(f"Errors: {upload_job.errors}", )
print(f"Status of uploads: {upload_job.statuses}")
```