Open In Colab
GitHub
Export JSON annotations
# Set the export params to include/exclude certain fields.
export_params= {
"attachments": True,
"metadata_fields": True,
"data_row_details": True,
"project_details": True,
"label_details": True,
"performance_details": True,
"interpolated_frames": True
}
# Note: Filters follow AND logic, so typically using one filter is sufficient.
filters= {
"last_activity_at": ["2000-01-01 00:00:00", "2050-01-01 00:00:00"],
"workflow_status": "<wkf-status>"
}
export_task = project.export(params=export_params, filters=filters)
export_task.wait_till_done()
# Stream the export using a callback function
def json_stream_handler(output: labelbox.BufferedJsonConverterOutput):
print(output.json)
export_task.get_buffered_stream(stream_type=labelbox.StreamType.RESULT).start(stream_handler=json_stream_handler)
# Collect all exported data into a list
export_json = [data_row.json for data_row in export_task.get_buffered_stream()]
print("file size: ", export_task.get_total_file_size(stream_type=lb.StreamType.RESULT))
print("line count: ", export_task.get_total_lines(stream_type=lb.StreamType.RESULT))
Annotation export formats
Classification - Radio
{
"feature_id": "clws3nkhh00043b6p7qktjffs",
"feature_schema_id": "clws3n1yu055907wlg1uh1i8g",
"name": "Context subject",
"value": "context_subject",
"message_id": "clws2arr200073b6p7ehwkoh4",
"conversational_radio_answer": {
"feature_id": "clws3nkhh00033b6ps99i1d56",
"feature_schema_id": "clws3n1yu055a07wl2axp39nq",
"name": "Car",
"value": "car",
"classifications": []
}
}
Classification - Free-form text
{
"feature_id": "clws3or5q00093b6p00oxtkve",
"feature_schema_id": "clws3n1yv055v07wlbnd57c9h",
"name": "Additional comment",
"value": "additional_comment",
"text_answer": {
"content": "Color: white"
}
}
Classification - Checklist
{
"feature_id": "clpiyav9m00053b6mpe2qk0im",
"feature_schema_id": "clpiya8h507dx072efo9kb97j",
"name": "Evaluations done",
"checklist_answers": [
{
"feature_id": "clpiyav9m00043b6mvhur1p19",
"name": "Usefulness",
"classifications": []
},
{
"feature_id": "clpiyax4d00073b6mqyyt0504",
"name": "Toxicity",
"classifications": []
}
]
}
Message Ranking
{
"feature_id": "clwrt25ar00093b6po9g7fiu3",
"feature_schema_id": "clwkfbcnn01kl07z68er64ffi",
"name": "Rank given prompt",
"annotation_kind": "MessageRanking",
"classifications": [],
"ranked_messages": {
"parent_message_id": "clwrt16zz00033b6p6ud7w233",
"ranked_messages": [
{
"message_id": "clwrt1fw700063b6pe8zpo1xp",
"order": 1,
"model_config_id": "e606593e-def3-4662-9737-3fc4ac412dd3"
},
{
"message_id": "clwrt1fw700073b6p8k3vkh1x",
"order": 2,
"model_config_id": "8fcda464-a0c7-4300-a9fa-c9a5cf82cab2"
}
]
}
}
Message selection
{
"feature_id": "clwrt2dru000b3b6pkj3i5zxk",
"feature_schema_id": "clwknckmp030v07xp6kow09xd",
"name": "Subject relevance",
"annotation_kind": "MessageMultiSelection",
"classifications": [],
"selected_messages": {
"parent_message_id": "clwrt16zz00033b6p6ud7w233",
"messages": [
{
"message_id": "clwrt1fw700063b6pe8zpo1xp",
"model_config_id": "e606593e-def3-4662-9737-3fc4ac412dd3"
}
]
}
}
Message step reasoning
{
"feature_id": "cm3gp5d39000k3b6tpet38ui9",
"feature_schema_id": "cm3ffq1mm04t407we109o1hyi",
"name": "my message step task",
"annotation_kind": "StepReasoning",
"classifications": [],
"steps": {
"parent_message_id": "cm3gp54f500073b6t26gyznra",
"steps": [
{
"step_id": "cm3gp54q6000a3b6t4563k4ba",
"content": "Here's a simple way to make a cup of tea:",
"variant_id": 0,
"variant": "correct"
},
{
"step_id": "cm3gp54q6000b3b6tsuiaudjb",
"content": "1. Boil water: Fill a kettle with water and bring it to a boil.",
"variant_id": 0,
"variant": "correct"
},
{
"step_id": "cm3gp54q6000d3b6tozd9neiq",
"content": "2. Add tea: Once the water is boiling, empty your now preheated teacup and add your tea. This can be a tea bag or loose tea leaves in a tea infuser. The amount of tea you add will depend on how strong you like your tea.",
"variant_id": 0,
"variant": "correct"
},
{
"step_id": "cm3gp54q6000e3b6t8y4k7grw",
"content": "3. Pour hot water: Pour the boiling water into the cup, over the tea.",
"variant_id": 0,
"variant": "correct"
},
{
"step_id": "cm3gp54q6000f3b6tddynpg1y",
"content": "4. Steep: Let the tea steep for a few minutes. The exact time will depend on the type of tea and how strong you like it. A general guide is 3-5 minutes. After the tea has steeped, remove the tea bag or infuser from the cup.",
"variant_id": 0,
"variant": "correct"
},
{
"step_id": "cm3gp54q6000h3b6t7mopupfy",
"content": "5. Add extras: If you like, you can now add extras such as milk, sugar, honey, or lemon. ",
"variant_id": 0,
"variant": "correct"
}
],
"root_step_id": "cm3gp54q6000a3b6t4563k4ba"
}
},
{
"feature_id": "cm3gp8byj00103b6t9904n65h",
"feature_schema_id": "cm3ffq1mm04t407we109o1hyi",
"name": "my message step task",
"annotation_kind": "StepReasoning",
"classifications": [],
"steps": {
"parent_message_id": "cm3gp82di000o3b6tr13dwupc",
"steps": [
{
"step_id": "cm3gp82mx000p3b6tlkd0c8a6",
"content": "Sure, here's more detail on step 5:",
"variant_id": 0,
"variant": "correct"
},
{
"step_id": "cm3gp82mx000q3b6tptjgz0ol",
"content": "5. Add extras: ",
"variant_id": 0,
"variant": "correct"
},
{
"step_id": "cm3gp82mx000r3b6t7mqcilx6",
"content": " - Milk: Some people prefer to add a splash of milk to their tea to make it creamier and to soften the flavor. The amount of milk can vary based on personal preference. Some teas, like black teas, often pair well with milk. ",
"variant_id": 0,
"variant": "correct"
},
{
"step_id": "cm3gp82mx000s3b6tntnvhpfc",
"content": " - Sugar: If you prefer your tea sweet, you can add sugar. Start with a teaspoon and then adjust to taste. You can also use sweeteners like stevia or sucralose if you're trying to avoid sugar.",
"variant_id": 2,
"variant": "corrected",
"alternative_steps": [
{
"content": " - Agave: If you prefer your tea sweet, you can add agave syrup. Start with a teaspoon and then adjust to taste. Agave is a natural sweetener that is sweeter than sugar, so you might need less of it.",
"variant_id": 0,
"variant": "neutral"
},
{
"content": " - Maple Syrup: If you prefer your tea sweet, you can add maple syrup. Start with a teaspoon and then adjust to taste. Maple syrup can add a unique, slightly smoky sweetness to your tea.",
"variant_id": 0,
"variant": "correct"
},
{
"content": " - Dates: If you prefer your tea sweet, you can blend dates and add the paste to your tea. Dates are a natural sweetener and can also add a bit of texture to your tea.",
"variant_id": 1,
"variant": "neutral"
}
],
"rewritten_content": "Sugar: If you prefer sweet tea, add sugar or other sweeteners.",
"justification": "The response provided recommendations that pose health risks we should avoid."
},
{
"step_id": "cm3gpcrij00133b6tz9766jfw",
"content": "- Sugar: Start with a teaspoon of sugar, stir it in and then taste. Add more if needed. Remember, you can always add more sugar but you can't take it away once it's added.",
"variant_id": 0,
"variant": "correct"
},
{
"step_id": "cm3gpcrij00153b6t0woknwgo",
"content": "- Lemon: A slice of lemon or a splash of lemon juice can add a refreshing citrus note to your tea. This is often paired with black tea.",
"variant_id": 0,
"variant": "correct"
},
{
"step_id": "cm3gpcrij00163b6tcetum5w1",
"content": "- Herbs and Spices: Some people like to add herbs and spices to their tea. For example, mint leaves, a cinnamon stick, or a few cardamom pods can add a unique flavor to your tea.",
"variant_id": 0,
"variant": "correct"
}
],
"root_step_id": "cm3gp82mx000p3b6tlkd0c8a6"
}
}
Prompt rating
{
"feature_id": "cm46ljjhs000i3b6tan025id0",
"feature_schema_id": "cm46l1i4n02yp07s99isra6v9",
"name": "Rate",
"annotation_kind": "PromptIssue",
"classifications": [
{
"feature_id": "cm46ljm67000k3b6t34goxv5r",
"feature_schema_id": "cm46l1i4n02yq07s9g32o677i",
"name": "Rate the prompt",
"value": "rate_the_prompt",
"checklist_answers": [
{
"feature_id": "cm46ljm67000j3b6tpr2rvaip",
"feature_schema_id": "cm46l1i4n02yt07s9haoj7aek",
"name": "This prompt contains a false, offensive, or controversial premise (eg. \u201cwhy does 1+1=3\u201d?)",
"value": "this_prompt_contains_a_false_offensive_or_controversial_premise_eg_why_does_1_1_3",
"classifications": []
}
]
}
],
"prompt_message_id": "cm46li41v000d3b6tdqzc9nv2"
},
{
"feature_id": "cm46lk2pr000q3b6ti5c02fbb",
"feature_schema_id": "cm46l1i4n02yp07s99isra6v9",
"name": "Rate",
"annotation_kind": "PromptIssue",
"classifications": [
{
"feature_id": "cm46lk3pv000s3b6tzlpvifq5",
"feature_schema_id": "cm46l1i4n02yq07s9g32o677i",
"name": "Rate the prompt",
"value": "rate_the_prompt",
"checklist_answers": [
{
"feature_id": "cm46lk3pv000r3b6t0vueiz56",
"feature_schema_id": "cm46l1i4n02yv07s9ccti4qv9",
"name": "This prompt is not self-contained, i.e. the prompt cannot be understood without additional context about previous turns, account information or images.",
"value": "this_prompt_is_not_self_contained_i_e_the_prompt_cannot_be_understood_without_additional_context_about_previous_turns_account_information_or_images",
"classifications": []
}
]
}
],
"prompt_message_id": "cm46ljwcy000l3b6tmk9oua72"
}
Fact-checking
{
"feature_id": "cm46myu0200193b6ttjhe11n5",
"feature_schema_id": "cm46m7j0k02t707uaesr2fhjq",
"name": "Fact-checking",
"annotation_kind": "FactChecking",
"classifications": [],
"steps": {
"parent_message_id": "cm46mvig0000z3b6t4c1qzqbr",
"steps": [
{
"content": "The Triassic period, which occurred approximately 252 to 201 million years ago, was a time of significant evolutionary change. While many species from that era have gone extinct, some lineages have persisted to the present day. Here are a few examples of animals and plants from the Triassic period that have descendants or close relatives still existing today:",
"variant": ""
},
{
"content": "1. **Crocodilians**: The ancestors of modern crocodiles and alligators first appeared in the Triassic period. While the specific species from that time are extinct, their descendants have survived and evolved into the crocodilians we know today.",
"variant_id": 4,
"variant": "cant_confidently_assess"
},
{
"content": "2. **Turtles**: Turtles also have ancient origins, with some of their earliest ancestors appearing in the late Triassic period. Modern turtles are part of this long lineage.",
"variant_id": 1,
"variant": "inaccurate",
"justification": "Modern turtles are different from Eunotosaurus in the Triassic period."
},
{
"content": "3. **Lepidosaurs**: This group includes modern lizards and snakes. Their ancestors were present during the Triassic period, and they have since diversified into the many species we see today.",
"variant_id": 4,
"variant": "cant_confidently_assess"
},
{
"content": "4. **Horsetails and Ferns**: These plants were common during the Triassic period and have survived to the present day. While the specific species from that time are not the same, the groups themselves have persisted.",
"variant_id": 4,
"variant": "cant_confidently_assess"
},
{
"content": "5. **Ginkgo**: The Ginkgo tree, often referred to as a \"living fossil,\" has ancient origins dating back to the Triassic period. The modern Ginkgo biloba is the only surviving species of this once-diverse group.",
"variant_id": 0,
"variant": "accurate",
"justification": "Ginkgo first appeared over 290 million years ago, which is the Triassic period."
},
{
"content": "6. **Conifers**: These gymnosperms were already present during the Triassic period and have continued to thrive, with many species existing today.",
"variant_id": 4,
"variant": "cant_confidently_assess"
},
{
"content": "These examples illustrate the resilience and adaptability of certain lineages that have managed to survive through multiple mass extinctions and environmental changes over millions of years.",
"variant_id": 3,
"variant": "unsupported"
}
]
}
}
Sample project export
{
"data_row": {
"id": "clwrm93488cpd0734f740r9lr",
"global_key": "clwrm92sb0grg07wncr6dhihw",
"row_data": "{\"type\":\"application/vnd.labelbox.conversational.model-chat-evaluation\",\"version\":2,\"actors\":{\"clwrt16zz00023b6pg6hlhtub\":{\"role\":\"human\",\"metadata\":{\"name\":\"User\"}},\"clwrt17a900043b6py2tepemd\":{\"role\":\"model\",\"metadata\":{\"modelConfigId\":\"e606593e-def3-4662-9737-3fc4ac412dd3\"}},\"clwrt17a900053b6p0xtyv1im\":{\"role\":\"model\",\"metadata\":{\"modelConfigId\":\"8fcda464-a0c7-4300-a9fa-c9a5cf82cab2\"}}},\"messages\":{\"clwrt16zz00033b6p6ud7w233\":{\"actorId\":\"clwrt16zz00023b6pg6hlhtub\",\"content\":[{\"type\":\"text\",\"content\":\"Is there a car in this image?\"},{\"type\":\"fileData\",\"fileUri\":\"https://c7.staticflickr.com/3/2447/3799589580_4b73283f92_o.jpg\",\"mimeType\":\"image/jpeg\"}],\"childMessageIds\":[\"clwrt1fw700063b6pe8zpo1xp\",\"clwrt1fw700073b6p8k3vkh1x\"]},\"clwrt1fw700063b6pe8zpo1xp\":{\"actorId\":\"clwrt17a900043b6py2tepemd\",\"content\":[{\"type\":\"text\",\"content\":\"Yes, there is a car in this image. It appears to be a white hatchback parked in a parking lot.\"}],\"childMessageIds\":[]},\"clwrt1fw700073b6p8k3vkh1x\":{\"actorId\":\"clwrt17a900053b6p0xtyv1im\",\"content\":[{\"type\":\"text\",\"content\":\" Yes, there is a white Volkswagen Golf in the image.\"}],\"childMessageIds\":[]}},\"rootMessageIds\":[\"clwrt16zz00033b6p6ud7w233\"]}",
"details": {
"dataset_id": "clwkf82ul001l0755mhckcmuy",
"dataset_name": "Model evaluation 101",
"created_at": "2024-05-29T09:20:43.342+00:00",
"updated_at": "2024-05-29T12:31:52.027+00:00",
"last_activity_at": "2024-05-29T12:33:21.203+00:00",
"created_by": "[email protected]"
}
},
"media_attributes": {
"asset_type": "conversational",
"mime_type": "application/vnd.labelbox.conversational.model-chat-evaluation",
"labelable_ids": [
"clwrt16zz00033b6p6ud7w233",
"clwrt1fw700063b6pe8zpo1xp",
"clwrt1fw700073b6p8k3vkh1x"
],
"message_count": 3
},
"attachments": [],
"metadata_fields": [],
"projects": {
"clwrm92jc0grc07wncq654q5s": {
"name": "Multi modal chat evalution",
"labels": [
{
"label_kind": "Default",
"version": "1.0.0",
"id": "clwrsyrdp02mr07x5378j3ysr",
"label_details": {
"created_at": "2024-05-29T12:33:21.000+00:00",
"updated_at": "2024-05-29T12:33:21.000+00:00",
"created_by": "[email protected]",
"content_last_updated_at": "2024-05-29T12:33:20.787+00:00",
"reviews": []
},
"performance_details": {
"seconds_to_create": 250,
"seconds_to_review": 0,
"skipped": false
},
"annotations": {
"objects": [
{
"feature_id": "clwrt25ar00093b6po9g7fiu3",
"feature_schema_id": "clwkfbcnn01kl07z68er64ffi",
"name": "Rank given prompt",
"annotation_kind": "MessageRanking",
"classifications": [],
"ranked_messages": {
"parent_message_id": "clwrt16zz00033b6p6ud7w233",
"ranked_messages": [
{
"message_id": "clwrt1fw700063b6pe8zpo1xp",
"order": 1,
"model_config_id": "e606593e-def3-4662-9737-3fc4ac412dd3"
},
{
"message_id": "clwrt1fw700073b6p8k3vkh1x",
"order": 2,
"model_config_id": "8fcda464-a0c7-4300-a9fa-c9a5cf82cab2"
}
]
}
},
{
"feature_id": "clwrt2dru000b3b6pkj3i5zxk",
"feature_schema_id": "clwknckmp030v07xp6kow09xd",
"name": "Subject relevance",
"annotation_kind": "MessageMultiSelection",
"classifications": [],
"selected_messages": {
"parent_message_id": "clwrt16zz00033b6p6ud7w233",
"messages": [
{
"message_id": "clwrt1fw700063b6pe8zpo1xp",
"model_config_id": "e606593e-def3-4662-9737-3fc4ac412dd3"
}
]
}
}
],
"classifications": [],
"relationships": []
}
}
],
"project_details": {
"ontology_id": "clwkn6leb034b07waeolfb7tl",
"task_id": "0c392f64-bc65-44e6-8848-ae5a12000328",
"task_name": "Initial review task",
"batch_id": "c2267a60-1d9c-11ef-9edf-ed6f8eede535",
"batch_name": "batch_clwrm92jc0grc07wncq654q5s_clwrm9eph006d0750b3s5b2cr",
"workflow_status": "IN_REVIEW",
"priority": 5,
"consensus_expected_label_count": 2,
"workflow_history": [
{
"action": "Move",
"created_at": "2024-05-29T12:33:21.160+00:00",
"created_by": "[email protected]",
"previous_task_name": "Initial labeling task",
"previous_task_id": "ebf41e2b-e231-0e43-adee-92252c1858dd",
"next_task_name": "Initial review task",
"next_task_id": "0c392f64-bc65-44e6-8848-ae5a12000328"
},
{
"action": "Move",
"created_at": "2024-05-29T12:33:21.151+00:00",
"created_by": "[email protected]",
"next_task_name": "Initial labeling task",
"next_task_id": "ebf41e2b-e231-0e43-adee-92252c1858dd"
}
]
},
"project_model_configs": {
"models": {
"5d9647e7-aafe-4275-944d-c2e9b122105a": {
"name": "Google Gemini Pro Vision",
"created_at": "2023-12-13T21:51:42.657+00:00"
},
"036aeb03-b0b4-442e-a6e1-161382bc1228": {
"name": "OpenAI GPT-4o",
"created_at": "2024-05-23T23:56:03.078+00:00"
}
},
"model_configs": {
"e606593e-def3-4662-9737-3fc4ac412dd3": {
"name": "GPT_4o_multi_modal",
"model_id": "036aeb03-b0b4-442e-a6e1-161382bc1228",
"inference_params": {
"top_p": 1,
"detail": "high",
"temperature": 0,
"use_text_only": false,
"max_new_tokens": 1024,
"use_image_only": false,
"use_attachments": false
}
},
"8fcda464-a0c7-4300-a9fa-c9a5cf82cab2": {
"name": "Gemini_pro_vison_multi_modal",
"model_id": "5d9647e7-aafe-4275-944d-c2e9b122105a",
"inference_params": {
"use_attachments": false
}
}
},
"project_model_configs": [
{
"id": "clwrsxr1y009i07v93w2h45qm",
"model_id": "036aeb03-b0b4-442e-a6e1-161382bc1228",
"model_config_id": "e606593e-def3-4662-9737-3fc4ac412dd3"
},
{
"id": "clwrsyext00ar07v5czpo7w2a",
"model_id": "5d9647e7-aafe-4275-944d-c2e9b122105a",
"model_config_id": "8fcda464-a0c7-4300-a9fa-c9a5cf82cab2"
}
]
},
"project_tags": []
}
}