> ## Documentation Index
> Fetch the complete documentation index at: https://docs.labelbox.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Import prompt and response annotations

> Developer guide for importing annotations on prompt and response projects.

<CardGroup cols={2}>
  <Card title="Open In Colab" icon="infinity" iconType="solid" href="https://colab.research.google.com/github/Labelbox/labelbox-notebooks/blob/main/annotation_import/prompt_response.ipynb" horizontal />

  <Card title="GitHub" icon="github" iconType="solid" href="https://github.com/Labelbox/labelbox-notebooks/tree/main/annotation_import/prompt_response.ipynb" horizontal />
</CardGroup>

## Overview

To import annotations in Labelbox, you need to create an annotations payload. In this section, we provide this payload for every supported annotation type.

### Annotation payload types

Labelbox supports two formats for the annotations payload:

* Python annotation types (recommended)

  * Provides a seamless transition between third-party platforms, machine learning pipelines, and Labelbox.
  * Allows you to build annotations locally with local file paths, numpy arrays, or URLs.
  * Supports easy conversion to NDJSON format to quickly import annotations to Labelbox.
  * Supports one-level nested classification (radio, checklist, or free-form text) under a tool or classification annotation.

* JSON

  * Skips formatting annotation payload in the Labelbox Python annotation type.
  * Supports any levels of nested classification (radio, checklist, or free-form text) under a tool or classification annotation.

### Label Import Types

Labelbox supports two types of label imports:

* [Model-assisted labeling (MAL)](/docs/model-assisted-labeling) allows you to import computer-generated predictions and simple annotations created outside of Labelbox as pre-labels on an asset.
* [Ground truth](/docs/import-ground-truth) allows you to bulk import ground truth annotations from an external or third-party labeling system into Labelbox *Annotate*. Using the label import API to import external data can consolidate and migrate all annotations into Labelbox as a single source of truth.

## Supported Annotations

Prompt and response generated projects support the following annotations data row:

* Prompt and response creation projects

  * Prompt text
  * Radio
  * Checklist
  * Response text

* Prompt creation projects

  * Prompt text

* Response creation projects

  * Radio
  * Checklist
  * Response text

## Prompt

### Free-form text

<Info>
  ### Information

  Only one prompt annotation is allowed per label.
</Info>

<CodeGroup>
  ```python Python annotation theme={null}
  prompt_annotation = lb_types.PromptClassificationAnnotation(
      name = "Follow the prompt and select answers",
      value = lb_types.PromptText(answer = "This is an example of a prompt")
  )
  ```

  ```json NDJSON theme={null}
  prompt_annotation_ndjson = {
    "name": "Follow the prompt and select answers",
    "answer": "This is an example of a prompt"
  }
  ```
</CodeGroup>

## Responses

### Radio (single choice)

<CodeGroup>
  ```python Python annotation theme={null}
  response_radio_annotation = lb_types.ClassificationAnnotation(
      name="response radio feature",
      value=lb_types.Radio(answer =
          lb_types.ClassificationAnswer(name = "first_radio_answer")
      )
  )
  ```

  ```json NDJSON theme={null}
  response_radio_annotation_ndjson = {
    "name": "response radio feature",
    "answer": {
        "name": "first_radio_answer"
      }
  }
  ```
</CodeGroup>

### Checklist (multiple choice)

<CodeGroup>
  ```python Python annotation theme={null}
  response_checklist_annotation = lb_types.ClassificationAnnotation(
      name="response checklist feature",
      value=lb_types.Checklist(answer = [
          lb_types.ClassificationAnswer(name = "option_1"),
          lb_types.ClassificationAnswer(name = "option_2"),
      ])
    )
  ```

  ```json NDJSON theme={null}
  response_checklist_annotation_ndjson = {
    "name": "response checklist feature",
    "answer": [
      {
        "name": "option_1"
      },
      {
        "name": "option_2"
      }
    ]
  }
  ```
</CodeGroup>

### Free-form text

<CodeGroup>
  ```python Python annotation theme={null}
  response_text_annotation = lb_types.ClassificationAnnotation(
      name = "Provide a reason for your choice",
      value = lb_types.Text(answer = "This is an example of a response text")
  )
  ```

  ```json NDJSON theme={null}
  response_text_annotation_ndjson = {
    "name": "Provide a reason for your choice",
    "answer": "This is an example of a response text"
  }
  ```
</CodeGroup>

### Nested classification

<CodeGroup>
  ```python Python annotation theme={null}
  nested_response_radio_annotation = lb_types.ClassificationAnnotation(
  name="nested_response_radio_question",
  value=lb_types.Radio(
    answer=lb_types.ClassificationAnswer(
      name="first_radio_answer",
      classifications=[
        lb_types.ClassificationAnnotation(
          name="sub_radio_question",
          value=lb_types.Radio(
            answer=lb_types.ClassificationAnswer(
              name="first_sub_radio_answer"
            )
          )
        )
      ]
    )
  )
  )

  nested_response_checklist_annotation = lb_types.ClassificationAnnotation(
  name="nested_response_checklist_question",
  value=lb_types.Checklist(
    answer=[lb_types.ClassificationAnswer(
      name="first_checklist_answer",
      classifications=[
        lb_types.ClassificationAnnotation(
          name="sub_checklist_question",
          value=lb_types.Checklist(
            answer=[lb_types.ClassificationAnswer(
            name="first_sub_checklist_answer"
          )]
        ))
      ]
    )]
  )
  )
  ```

  ```json NDJSON theme={null}
  nested_response_radio_annotation_ndjson = {
    "name": "nested_radio_question",
    "answer": [{
        "name": "first_radio_answer",
        "classifications" : [
          {
            "name": "sub_radio_question",
            "answer": {"name": "first_sub_radio_answer"}
          }
        ]
    }]
  }

  nested_response_checklist_annotation_ndjson = {
    "name": "nested_checklist_question",
    "answer": [{
        "name": "first_checklist_answer",
        "classifications" : [
          {
            "name": "sub_checklist_question",
            "answer": {"name": "first_sub_checklist_answer"}
          }
        ]
    }]
  }
  ```
</CodeGroup>

## Example: Import pre-labels or ground truths

The steps to import annotations as pre-labels (machine-assisted learning) are similar to the steps to import annotations as ground truth labels, and we will describe the slight differences for each scenario.

### Before you start

The below imports are needed to use the code examples in this section.

<CodeGroup>
  ```python Python theme={null}
  import labelbox as lb
  import labelbox.types as lb_types
  import uuid
  import time
  ```
</CodeGroup>

Replace the value of `API_KEY` with a valid [API key](/reference/create-api-key) to connect to the Labelbox client.

<CodeGroup>
  ```python Python theme={null}
  API_KEY = None
  client = lb.Client(API_KEY)
  ```
</CodeGroup>

### Step 1: Import or generate data rows and create projects

Each type of the prompt and response generation project requires different setup. See [prompt and response project](/reference/prompt-and-response-projects) for more details on the differences.

#### Prompt response and prompt creation

A prompts and responses creation project automatically generates empty data rows upon creation. You will then need to obtain either the `global_keys` or `data_row_ids` attached to the generated data rows by exporting them from the created project or obtaining them directly on the [data row tab](/docs/data-rows-activity) using the UI.

<CodeGroup>
  ```python Prompt and response creation theme={null}
  prompt_response_project = client.create_prompt_response_generation_project(
      name="Demo prompt response project",
      media_type=lb.MediaType.LLMPromptResponseCreation,
      dataset_name="Demo prompt response dataset",
      data_row_count=1,
  )
  ```

  ```python Prompt creation theme={null}
  prompt_project = client.create_prompt_response_generation_project(
      name="Demo prompt project",
      media_type=lb.MediaType.LLMPromptCreation,
      dataset_name="Demo prompt dataset",
      data_row_count=1,
  )
  ```
</CodeGroup>

#### Response creation

For response creation projects, text data rows are used and are not generated upon project creation. The following steps create a dataset with a text data row attached, create a response creation project, and batch the created data row towards the project.

<CodeGroup>
  ```python Python theme={null}
  # Create dataset with text data row
  global_key = "lorem-ipsum.txt"
  text_asset = {
      "row_data": "https://storage.googleapis.com/labelbox-sample-datasets/nlp/lorem-ipsum.txt",
      "global_key": global_key,
      "media_type": "TEXT",
      }

  dataset = client.create_dataset(name="text_annotation_import_demo_dataset")
  task = dataset.create_data_rows([text_asset])
  task.wait_till_done()

  print("Errors:",task.errors)
  print("Failed data rows:", task.failed_data_rows)

  # Create response creation project

  project = client.create_response_creation_project(
    name="Demo response project",
  )

  # Create a batch of data rows for newly created project

  batch = project.create_batch(
    name="Demo response batch", # each batch in a project must have a unique name
    global_keys=[global_key], # paginated collection of data row objects, list of data row ids or global keys
    priority=1 # priority between 1(highest) - 5(lowest)
  )
  ```
</CodeGroup>

### Step 2: Set up ontology

Your project ontology needs to support the classifications required by your annotations. To ensure accurate schema feature mapping, the value used as the `name` parameter needs to match the value of the `name` field in your annotation.

For example, if you provide a name `annotation_name` for your created annotation, you need to name the bounding box tool as `anotations_name` when setting up your ontology. The same alignment must hold true for the other tools and classifications that you create in the ontology.

This example shows how to create an ontology containing all supported by prompt and response project types [annotation types](#supported-annotations).

<CodeGroup>
  ```python Prompt and response creation expandable theme={null}
  ontology_builder = lb.OntologyBuilder(
      tools=[],
      classifications=[
          lb.PromptResponseClassification(
              class_type=lb.PromptResponseClassification.Type.PROMPT,
              name="prompt text",
              character_min = 1, # Minimum character count of prompt field (optional)
              character_max = 20, # Maximum character count of prompt field (optional)
          ),
          lb.PromptResponseClassification(
              class_type=lb.PromptResponseClassification.Type.RESPONSE_CHECKLIST,
              name="response checklist feature",
              options=[
                  lb.ResponseOption(value="option_1", label="option_1"),
                  lb.ResponseOption(value="option_2", label="option_2"),
              ],
          ),
          lb.PromptResponseClassification(
              class_type=lb.PromptResponseClassification.Type.RESPONSE_RADIO,
              name="response radio feature",
              options=[
                  lb.ResponseOption(value="first_radio_answer"),
                  lb.ResponseOption(value="second_radio_answer"),
              ],
          ),
          lb.PromptResponseClassification(
              class_type=lb.PromptResponseClassification.Type.RESPONSE_TEXT,
              name="response text",
              character_min = 1, # Minimum character count of response text field (optional)
              character_max = 20, # Maximum character count of response text field (optional)
          ),
          lb.PromptResponseClassification(
              class_type=lb.PromptResponseClassification.Type.RESPONSE_RADIO,
              name="nested_response_radio_question",
              options=[
                  lb.ResponseOption("first_radio_answer",
                                    options=[
                                      lb.PromptResponseClassification(
                                        class_type=lb.PromptResponseClassification.RESPONSE_RADIO,
                                        name="sub_radio_question",
                                        options=[lb.ResponseOption("first_sub_radio_answer")]
                                      )
                                    ])
              ],
          ),
          lb.PromptResponseClassification(
              class_type=lb.PromptResponseClassification.Type.RESPONSE_CHECKLIST,
              name="nested_response_checklist_question",
              options=[
                  lb.ResponseOption("first_checklist_answer",
                                    options=[
                                      lb.PromptResponseClassification(
                                        class_type=lb.PromptResponseClassification.RESPONSE_CHECKLIST,
                                        name="sub_checklist_question",
                                        options=[lb.ResponseOption("first_sub_checklist_answer")]
                                      )
                                    ])
              ],
          ),
      ],
  )

  # Create ontology
  ontology = client.create_ontology(
      "Prompt and response ontology",
      ontology_builder.asdict(),
      media_type=lb.MediaType.LLMPromptResponseCreation,
  )

  # Connect ontology
  prompt_response_project.connect_ontology(ontology)
  ```

  ```python Prompt creation theme={null}
  ontology_builder = lb.OntologyBuilder(
      tools=[],
      classifications=[
          lb.PromptResponseClassification(
              class_type=lb.PromptResponseClassification.Type.PROMPT,
              name=f"prompt text",
              character_min = 1, # Minimum character count of prompt field (optional)
              character_max = 20, # Maximum character count of prompt field (optional)
          )
      ],
  )

  # Create ontology
  ontology = client.create_ontology(
      "Prompt ontology",
      ontology_builder.asdict(),
      media_type=lb.MediaType.LLMPromptCreation,
  )

  # Connect ontology
  prompt_project.connect_ontology(ontology)
  ```

  ```python Response creation theme={null}
  ontology_builder = lb.OntologyBuilder(
      tools=[],
      classifications=[
          lb.PromptResponseClassification(
              class_type=lb.PromptResponseClassification.Type.RESPONSE_CHECKLIST,
              name="response checklist feature",
              options=[
                  lb.ResponseOption(value="option 1", label="option 1"),
                  lb.ResponseOption(value="option 2", label="option 2"),
              ],
          ),
          lb.PromptResponseClassification(
              class_type=lb.PromptResponseClassification.Type.RESPONSE_RADIO,
              name="response radio feature",
              options=[
                  lb.ResponseOption(value="first_radio_answer"),
                  lb.ResponseOption(value="second_radio_answer"),
              ],
          ),
          lb.PromptResponseClassification(
              class_type=lb.PromptResponseClassification.Type.RESPONSE_TEXT,
              name="response text",
              character_min = 1, # Minimum character count of response text field (optional)
              character_max = 20, # Maximum character count of response text field (optional)
          ),
          lb.PromptResponseClassification(
              class_type=lb.PromptResponseClassification.Type.RESPONSE_RADIO,
              name="nested_response_radio_question",
              options=[
                  lb.ResponseOption("first_radio_answer",
                                    options=[
                                      lb.PromptResponseClassification(
                                        class_type=lb.PromptResponseClassification.RESPONSE_RADIO
                                        name="sub_radio_question",
                                        options=[lb.ResponseOption("first_sub_radio_answer")]
                                      )
                                    ])
              ],
          ),
          lb.PromptResponseClassification(
              class_type=lb.PromptResponseClassification.Type.RESPONSE_CHECKLIST,
              name="nested_response_checklist_question",
              options=[
                  lb.ResponseOption("first_checklist_answer",
                                    options=[
                                      lb.PromptResponseClassification(
                                        class_type=lb.PromptResponseClassification.RESPONSE_CHECKLIST
                                        name="sub_checklist_question",
                                        options=[lb.ResponseOption("first_sub_checklist_answer")]
                                      )
                                    ])
              ],
          ),
      ],
  )

  # Create ontology
  ontology = client.create_ontology(
      "Response ontology",
      ontology_builder.asdict(),
      media_type=lb.MediaType.Text,
      ontology_kind=lb.OntologyKind.ResponseCreation
  )

  # Connect ontology
  response_project.connect_ontology(ontology)
  ```
</CodeGroup>

### Step 3: Export for global\_keys

For prompt response creation and prompt creation projects you will need to obtain either the `global_keys` or `data_row_ids` attached to the generated data rows by exporting them from the created project. Since the generation of data rows is an async process you will need to wait for the project data rows to be completed before exporting.

<CodeGroup>
  ```python Prompt and response export theme={null}
  time.sleep(20)

  export_task = prompt_response_project.export()
  export_task.wait_till_done()

  # Stream the export using a callback function

  def json_stream_handler(output: labelbox.BufferedJsonConverterOutput):
    print(output.json)

  export_task.get_buffered_stream(stream_type=labelbox.StreamType.RESULT).start(stream_handler=json_stream_handler)

  # Collect all exported data into a list

  export_json = [data_row.json for data_row in export_task.get_buffered_stream()]

  # Obtain global keys to be used later on

  global_keys = [dr.json["data_row"]["global_key"] for dr in stream]

  ```

  ```python Prompt export theme={null}
  time.sleep(20)

  export_task = prompt_project.export()
  export_task.wait_till_done()

  # Stream the export using a callback function
  def json_stream_handler(output: labelbox.BufferedJsonConverterOutput):
    print(output.json)

  export_task.get_buffered_stream(stream_type=labelbox.StreamType.RESULT).start(stream_handler=json_stream_handler)

  # Collect all exported data into a list
  export_json = [data_row.json for data_row in export_task.get_buffered_stream()]

  # Obtain global keys to be used later on
  global_keys = [dr.json["data_row"]["global_key"] for dr in stream]
  ```
</CodeGroup>

### Step 4: Import Annotations

For prelabeled (model-assisted labeling) scenarios, pass your payload as the value of the `predictions` parameter. For ground truths, pass the payload to the `labels` parameter.

#### Payload

Depending on the type of prompt and response project you are using your payload might look different. For the response payload, you can also use `is_benchmark_reference` to specify [benchmarks](/docs/benchmark).

<CodeGroup>
  ```python Prompt and response payload theme={null}
  # Python annotation objects
  label = []
  annotations = [
    prompt_annotation,
    response_radio_annotation,
    response_checklist_annotation,
    response_text_annotation,
    nested_response_radio_annotation,
    nested_response_checklist_annotation
  ]
  label.append(
      lb_types.Label(data={"global_key" : global_keys[0] },
                     annotations=annotations)
              )

  # NDJSON
  label_ndjson = []
  annotations = [
    prompt_annotation_ndjson,
    response_radio_annotation_ndjson,
    response_checklist_annotation_ndjson,
    response_text_annotation_ndjson,
    nested_response_radio_annotation_ndjson,
    nested_response_checklist_annotation_ndjson
  ]
  for annotation in annotations:
      annotation.update({
          "dataRow": {
              "globalKey": global_keys[0]
          },
      })
      label_ndjson.append(annotation)
  ```

  ```python Prompt payload theme={null}
  # Python annotation objects
  label = []
  annotations = [
    prompt_annotation
  ]
  label.append(
      lb_types.Label(data={"global_key" : global_keys[0] },
                     annotations=annotations)
              )

  # NDJSON
  label_ndjson = []
  annotations = [
    prompt_annotation_ndjson
  ]
  for annotation in annotations:
      annotation.update({
          "dataRow": {
              "globalKey": global_keys[0]
          },
      })
      label_ndjson.append(annotation)
  ```

  ```python Response payload theme={null}
  # Python annotation objects
  label = []
  annotations = [
    response_radio_annotation,
    response_checklist_annotation,
    response_text_annotation,
    nested_response_radio_annotation,
    nested_response_checklist_annotation
  ]
  label.append(
      lb_types.Label(data={"global_key" : global_key },
                     annotations=annotations
                     # Optional: set the label as a benchmark
                     # Only supported for groud truth imports
                     is_benchmark_reference = True)
              )

  # NDJSON
  label_ndjson = []
  annotations = [
    response_radio_annotation_ndjson,
    response_checklist_annotation_ndjson,
    response_text_annotation_ndjson,
    nested_response_radio_annotation_ndjson,
    nested_response_checklist_annotation_ndjson
  ]
  for annotation in annotations:
      annotation.update({
          "dataRow": {
              "globalKey": global_key
          },
      })
      label_ndjson.append(annotation)
  ```
</CodeGroup>

#### Option A: Upload as [prelabels (model assisted labeling)](/docs/model-assisted-labeling)

This option is helpful for speeding up the initial labeling process and reducing the manual labeling workload for high-volume datasets.

<CodeGroup>
  ```python MAL import theme={null}
  # Upload MAL label for this data row in project
  upload_job = lb.MALPredictionImport.create_from_objects(
      client = client,
      project_id = prompt_response_project.uid, # Replace with project of different prompt and response project types
      name = "mal_job"+str(uuid.uuid4()),
      predictions = label
  )

  print(f"Errors: {upload_job.errors}", )
  print(f"Status of uploads: {upload_job.statuses}")

  ```
</CodeGroup>

#### Option B: Upload to a labeling project as [ground truth](/docs/import-ground-truth)

This option is helpful for loading high-confidence labels from another platform or previous projects that just need review rather than manual labeling effort.

<CodeGroup>
  ```python Label import theme={null}
  # Upload label for this data row in project
  upload_job = lb.LabelImport.create_from_objects(
      client = client,
      project_id = prompt_response_project.uid, # Replace with project of different prompt and response project types
      name = "label_import_job" + str(uuid.uuid4()),
      labels = label
  )

  print(f"Errors: {upload_job.errors}", )
  print(f"Status of uploads: {upload_job.statuses}")
  ```
</CodeGroup>