Run Foundry apps

Foundry apps can streamline traditional model runs by automating the process.

There's more than one way to run a Foundry app. You can:

• Run one using the traditional Foundry workflow
• Run one using a REST endpoint
• You can run Foundry apps in more than one as part of the traditional Foundry workflow, using REST API, or using the SDK.

Here, we discuss each approach in detail.

🚧

This page describes features currently in beta. Some improvements may not yet be documented.

Each data row processed during the model run appears in the app's Prediction tab, which lets you inspect the history of use and results.

Run app from Catalog

To run a Foundry app:

1. Use Catalog to select the data rows to be processed by your Foundry app.

2. When prompted to select a model, select the Apps tab and then select your Foundry app.

   

3. From here, you can generate previews or submit the model run as you would in any other Foundry model run.

Run app using REST API

You can use a REST API to run a Foundry app, which lets you generate data row predictions from external workflows.

You can use REST endpoints to inference (predict labels for):

• Data rows stored in Labelbox Catalog datasets
• Raw data assets not stored in Labelbox.

The Labelbox app helps define REST queries for common tasks, including:

• Creating a prediction job for an existing data row.
• Creating a prediction job for a raw data asset.
• Getting the result of a previous prediction job.
• Using a search filter to locate a previous prediction job.

The app shows the verb to use and provides sample request bodies when needed.

To use the app to define a REST query for one of these tasks:

1. Use Model to open your Foundry app and then select the Developers tab.

   

2. REST authentication is handled through an API key. If you have not already generated an API key, select Generate API key and then follow the prompts. (Remember to save your API key in a safe place.)

3. Select the endpoint appropriate for the task you want to perform.

4. Use the available samples to define your REST query. When you copy endpoints to your device Clipboard, remember that you get only the URI. (The verb is specified separately; that process varies according to your REST client.)

5. Endpoint examples include a sample request body when necessary. Use these to help define your REST query.

Predict existing data row

To use a Foundry app to create a prediction job for an existing data row, specify the data row ID in the request body of your REST query:

curl --location 'https://app.labelbox.com/api/v1/foundry-app/:appId/predict' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <API_KEY>' \
--data '{
    "type": "dataRow",
    "id": "<data_row_id>"
}'

Note that type is set to dataRow and that <data_row_id> specifies the appropriate value.

Predict raw asset

You can also use Foundry apps to generate predictions for raw data assets that aren't stored in Labelbox Catalog.

curl --location 'https://app.labelbox.com/api/v1/foundry-app/:appId/predict' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <API_KEY>' \
--data '{
    "type": "text | image",
    "url": "<asset_url>",
    "createDataRow": "true", 
    "addToModelRun": "true"
}'

curl --location 'https://app.labelbox.com/api/v1/foundry-app/:appId/predict' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <API_KEY>' \
--data '{
    "type": "text",
    "text": "<text content>",
    "createDataRow": "true", 
    "addToModelRun": "true"
}'

curl --location 'https://app.labelbox.com/api/v1/foundry-app/:appId/predict' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <API_KEY>' \
--data '{
    "type": "image",
    "base64": "<base64 encoding of the image>",
    "createDataRow": "true", 
    "addToModelRun": "true"
}'

In the request body:

• Set typeto either text or image (currently the only supported values).
• Specify the raw data assets in one of the following ways:
  • Use url to specify a public URL pointing to the asset
  • Set text to the text value of the asset
  • Use base64 to specify a Base 64-encoded string that defines an image
• Set createDataRow to true (default) to create a new data row
• Set addToModelRunto true (default) to add the data row and prediction to a model run

When createDataRow and addToModelRun are true, the data row is added to a dataset named App {Name} - inference data and the prediction is added to a model run named App {Name } - inferences, where {Name} is the name of your Foundry app.

The request response confirms the start of the prediction job.

{
    "id": "449f0b35-38c6-4fb1-bb47-144e958d08a4",   <--- Job ID
    "organizationId": "clah7z9b610d707wpfpf35066",
    "userId": "clah7z9br10d807wpboxt920r",
    "modelAppId": "bfe3927c-1f60-44ec-9114-c6a5f6ff0de2",
    "dataRowId": "clqbkkn656s6s07975dhqnotz",
    "status": "in_progress",
    "createdAt": "2024-03-25T19:04:44.850617+00:00"
}

The id field includes the Job ID, which can be used to retrieve results when the job completes.

Get prediction results

Prediction jobs run asynchronously, which means you need a second query to retrieve results. This query requires the Job ID provided in the initial request response.

curl --location 'https://app.labelbox.com/api/v1/foundry-app/prediction/:id' \
--header 'Authorization: Bearer <API_KEY>'

Prediction results appear in the response:

{
    "id": "449f0b35-38c6-4fb1-bb47-144e958d08a4",
    "organizationId": "clah7z9b610d707wpfpf35066",
    "userId": "clah7z9br10d807wpboxt920r",
    "modelAppId": "bfe3927c-1f60-44ec-9114-c6a5f6ff0de2",
    "dataRowId": "clqbkkn656s6s07975dhqnotz",
    "status": "success",
    "annotations": [
        {
            "objects": [],
            "relationships": [],
            "classifications": [
                {
                    "name": "describe_photo",
                    "feature_id": "af4e3cd30b904392a370559bf0a92064",
                    "text_answer": {
                        "content": "N/A"
                    }
                },
                {
                    "name": "has_people",
                    "feature_id": "69bda23bc9df4dcab86ace5c95cb6503",
                    "radio_answer": {
                        "name": "no",
                        "feature_id": "c0c57d73d3ba4884af88357bf6c3a025"
                    }
                },
                {
                    "name": "has_chairs",
                    "feature_id": "88f84d30ea4745be98a66baa23d685ed",
                    "radio_answer": {
                        "name": "no",
                        "feature_id": "289e63b18ff746c899a1eb6c0c358fe3"
                    }
                }
            ]
        }
    ],
    "price": 0,
    "createdAt": "2024-03-25T19:04:44.850617+00:00",
    "finishedAt": "2024-03-25T19:05:01.704949+00:00"
}

You can also use the Predictions tab of the Foundry app.

Find prediction job

There are several ways to locate earlier prediction jobs, you can find them using:

• Model App ID, which is displayed on the details of your Foundry app.
• Job status, which supported the following values: in_progress, success, error, canceled, and retried.
• Creation date, which supports a range of dates specified in ISO 8601 format.

Samples of each are shown here:

curl --location 'https://app.labelbox.com/api/v1/foundry-app/:appId/predictions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <API_KEY>' \
--data '{
  "filters": {
    "modelRunId": "string",
  },
  "pagination": {
    "limit": "number",
    "cursor": "string"
  }
}'

curl --location 'https://app.labelbox.com/api/v1/foundry-app/:appId/predictions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <API_KEY>' \
--data '{
  "filters": {
    "status": "in_progress|success|error|canceled|retried",
  },
  "pagination": {
    "limit": "number",
    "cursor": "string"
  }
}'

curl --location 'https://app.labelbox.com/api/v1/foundry-app/:appId/predictions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <API_KEY>' \
--data '{
  "filters": {
    "createdFrom": "string (ISO8601 date format)",
    "createdTo": "string (ISO8601 date format)"
  },
  "pagination": {
    "limit": "number",
    "cursor": "string"
  }
}'

Run app using SDK

You can use the SDK to run Foundry apps and process prediction results. This means, for example, you use a script to send predictions to Annotate for human review.

Use the Developers tab of a Foundry app to view a sample Python script that automates data workflow.

The initial example shows how to:

• Use a slice to select a set of data rows
• Run a Foundry app against a set of data rows
• Retrieve prediction results
• Send prediction results to Annotate for human review

The sample shows the overall, but cannot be run as specified. You need to customize the example to suit your individual needs.

To use this in your own projects, you'll need to customize the script with a number of ID values, which can be found using the Labelbox app. For example, the App ID appears when you update your Foundry app.