​
June 3, 2025

Release notes

​
App

Added

  • When you create a project and select the Multi-modal chat editor, you’ll be prompted to select from a dropdown of preconfigured settings for your project. If needed, you can update these settings in the advanced project settings section.
  • New models added to Foundry
    • Amazon Nova Sonic
    • Amazon Nova Premier
    • Open AI GPT-4o Image Generation
    • Meta Llama 3.3
    • Anthropic Claude 4

Changed

  • The “start transcribing” button in the audio editor will be disabled unless your organization has access to Foundry.
  • The issues creation flow in the Multi-modal chat editor has been slightly updated to display a more discreet issues icon (caution icon). Clicking this icon allows you to create, resolve, and reopen issues in the task.

Removed

  • Removed these models from Foundry
    • Google Gemini 2.0 Flash Experimental
    • xAI Grok Beta

Future

​
Python SDK

To view the latest updates to our Python SDK, see the What’s new section in the developer guides.

​
May 5, 2025

Release notes

​
App

Added

  • The new form-based UI in the Multi-modal chat editor makes the labeling experience more intuitive, displays instructions at each labeling task, and makes the tools menu more accessible.
  • In the Multi-modal chat editor, you’ll see a minimap on the right that indicates the location of the errors to address before the task can be submitted. Selecting the red markers in the minimap will bring you to each error in the conversation. To learn more, read this blog post.
  • We added turn-based audio and video support to the Multi-modal chat editor to enable labelers to generate video training data on real-world examples. You can enable this in the project settings.
  • We added real-time audio and video support to the Multi-modal chat editor to enable labelers to submit video as a response to the model in real time. You can enable this in the project settings.
  • The new “Model selection” task option in the Multi-modal chat editor makes it more intuitive for labelers to select which model response is better, indicate that both are bad, or are “good”. You can configure this task to function as a likert scale. To configure this task at the global level, go to Schema → Ontologies → Create → Model chat evaluation → Model selection tasks → Global → Add. To configure this task at the turn-level, go to Schema → Ontologies → Create → Model chat evaluation → Model selection tasks → Per turn → Add.
  • Foundry now supports the following models:
    • Gemini 2.5 Pro
    • Meta Llama 4.0 Maverick
    • Grok 3
    • OpenAI GPT 4.1
    • OpenAI GPT-4o Transcribe
    • OpenAI GPT-4o mini Transcribe
    • OpenAI GPT-4o mini TTS
    • OpenAI o3
    • OpenAI o4-mini

Changed

  • In the Data Rows tab, you can now expand column widths and reorder columns in the table.
  • Within a project, the “Automation” tab has now been renamed to the “Import labels” tab. Here you can find your import jobs used for MAL predictions, ground truth, or benchmarks. There is also helpful code recommended that includes information from the ontology assigned to the project.

Removed

  • April 30, 2025, we started introducing changes to the Python SDK that will break compatibility with Export v2 non-streamable methods in version 3.66 and earlier. To see which methods will be impacted by this change, visit our Deprecations page.

Future

​
Python SDK

To view the latest updates to our Python SDK, see the What’s new section in the developer guides.

​
April 2, 2025

Release notes

​
App

Added

  • We have rolled out an improved experience for managing workspace members and groups to all tiers. To learn more about these changes, read Manage members and permissions.

Changed

  • When you select the Workflows tab in a project, you’ll notice the UI has been updated. This new Workflow editor allows you more flexibility when creating and arranging the steps in your project workflow. To learn more, see our docs on Workflows.
  • If you have MFA enabled for your organization, you will see a checkbox upon login that offers to “Remember this device for 30 days”.
  • Integrating AWS S3 via IAM now requires you to create a custom trust policy instead of selecting the Labelbox AWS account. This change gives you more granular control, making the integration more flexible and customizable. See our docs on AWS S3 integration to learn more.

Future

  • On April 30, 2025, we will introduce changes to the Python SDK that will break compatibility with Export v2 non-streamable methods in version 3.66 and earlier. To see which methods will be impacted by this change, visit our Deprecations page.
  • On June 30, 2025, we will sunset the Census integration due to lack of usage. If you have any questions or concerns, please contact support@labelbox.com.

​
Python SDK

To view the latest updates to our Python SDK, see the What’s new section in the developer guides.

​
March 3, 2025

Release notes

​
App

Added

  • In the image editor, you can now record an audio clip as a response, and the editor will automatically transcribe and save the audio recording as a classification. To learn more, read our Image editor guide.
  • In the Services tab, we introduced an NLP search option to make it easier to find candidates that have certain skills or specializations. To use the NLP search, go to the Services tab, select the “Natural language” filter, and enter one or more keywords to narrow down the search results.
  • When you are labeling code in the Multi-modal chat editor, you now have access to a full VSCode IDE web instance and a remote host environment, allowing you to work on entire code repositories, run CLI tools, execute code, use debuggers, write tests, install additional VSCode extensions, and use Clari copilot to write/edit code blocks. To use this feature, create a project with the Multi-modal chat editor, edit the code block, and select the “Edit in coder” button to open the IDE web instance.
  • The following models are now supported in the Model tab (see “Hosted by Labelbox” section):
    • Anthropic Claude 3.7 Sonnet
    • Anthropic Claude 3.7 Sonnet Think
    • Google Gemini 2.0 Flash
    • Google Gemini 2.0 Flash-Lite
    • Google Gemini 2.0 Pro

Changed

  • The home page was improved recently to highlight our labeling services and to make our core features easier to find.
  • We are gradually rolling out an improved experience for managing workspace members and groups. To ensure a smooth transition, these changes will be rolled out tier by tier. To learn more about these changes, read New member and group management experience.

Removed

  • On February 13th, we disabled the “Reporting” tab (AKA Enterprise Dashboard) for all customers. The Reporting tab has been replaced by the new Monitor tab.
  • On February 28th, we disabled the Catalog cluster view + Smart select feature for all customers.

Future

  • On April 24, 2025, we will sunset exports v2 non-streamable for all customers using Python SDK versions 3.66 or earlier.

​
Python SDK

  • To view the latest updates to our Python SDK, see the What’s new section in the developer guides.
​
February 4, 2025

Release notes

​
App

Added

  • The following models are now hosted and available in the Model tab:
    • OpenAI o1 & o3-mini
    • OpenAI Whisper
    • Google Gemini 2.0 Flash Thinking
    • Google Gemini 2.0 Flash Experimental
    • Amazon Nova Micro, Lite, and Pro
  • To increase both the speed and accuracy of extracting text from a PDF, labelers can now draw bounding boxes around text in a document, and the PDF editor will automatically extract the text from the bounding box.
  • The Audio editor now automatically transcribes speech to text to make labeling audio files easier.

Changed

  • The Multi-modal chat editor now includes an improved experience for editing and running code. To learn more, see our Code runner docs.
  • The AI critic in the Multi-modal chat editor now has an improved experience for getting suggestions on coding tasks.
  • The Services tab is now easier to navigate and has an improved experience for filtering results.

Future

  • On February 13, 2025, we will sunset the “Reporting” tab (AKA Enterprise Dashboard) for all customers. The Reporting tab has been replaced by the new Monitor tab. To learn more about this change, refer to this guide.

​
Python SDK

To view the latest updates to our Python SDK, see the What’s new section in the developer guides.

​
January 6, 2025

Release notes

​
App

Added

  • The Multi-modal chat editor now has a built-in code runner that runs and detects code blocks. This feature allows labelers and reviewers to execute code directly in the editor and see the results. Code execution results are persisted and included in exports. To learn more, see our Code runner docs. This feature is currently in beta.
  • Polygons, polylines, bounding boxes, and points now have a z-order and are rendered so that those in front occlude or block out those behind. Their z-order can be changed using a new view in the Objects panel. To learn more, see our docs on Editor settings.
  • The new “Polygon snapping” feature allows points on lines, polygons, and bounding boxes to align or attach to polygon edges. You can toggle on polygon snapping in the Editor settings.
  • To encourage labeler understanding of project instructions, labelers will be enforced to read and acknowledge project instructions before working on a project. If project instructions are modified, labelers will be required to re-read instructions before they can resume labeling in the project.Enforced labeling instructions only impacts customers who opt into using Alignerr labeling services for their projects.
  • In the PDF editor, you can now create bounding boxes on the page and use the relationship tool to connect the bounding boxes to classifications. This feature is available upon request (email support@labelbox.com to request access).
  • In the new labeling services marketplace, you can view Alignerr labeler profiles and select labelers for your next project based on their expertise and availability. To browse Alignerr labeler profiles, select “Services” from the left nav bar. To learn more, see our docs on Alignerr Connect.

Changed

  • After a series of bug fixes, the custom roles feature now has an improved user experience. Fixes mostly include changes to make permission settings more reliable.
  • You can now download the history of changes made to a user group as a CSV file. To do this, go to Workspace settings > Groups > Export > Export history.
  • The latest timer mechanism has now been rolled out to all remaining organizations. This new timer mechanism offers more accurate measurements for labeling, reviewing, and reworking time.

Future

  • On February 13, 2025, we will sunset the “Reporting” tab (AKA Enterprise Dashboard) for all customers. The Reporting tab has been replaced by the new Monitor tab. To learn more about this change, refer to this guide.

​
Python SDK

To view the latest updates to our Python SDK, see the What’s new section in the developer guides.

​
December 3, 2024

Release notes

​
App

Added

  • In-editor suggestions are now available in the Multi-modal chat and Prompt & response editors. You can use this AI-powered tool to double-check your grammar and wording to ensure your prompts are written correctly. This feature is enabled on a project-by-project basis.
  • The new “step-by-step reasoning” task in the Multi-modal chat editor allows labelers to classify each step in the model response as “correct” or “incorrect”. Then, the model regenerates its response starting from the incorrect segment while preserving the previous responses. To learn more about how this works, read this guide.
  • The new “fact-checking” task in the Multi-modal chat editor allows labelers to easily rate any step in the response as “accurate”, “inaccurate”, “disputed”, “unsupported”, etc.
  • The new “prompt-rating” task in the Multi-modal chat editor allows labelers to select from a pre-set list of toggles to signal that the content should not be labeled for a specific reason (e.g., offensive content, contains PII, not understandable, etc).

Removed

  • On November 7th 2024, we removed the native support for YOLO models in Foundry. If you would like to set up your own YOLO model for inferencing, refer to our custom model integration docs.
  • On November 7th 2024, we disabled the model fine-tuning feature, meaning the image fine-tuning is no longer available. We may re-enable this feature in the future.
  • On November 25th 2024, we sunsetted the DICOM editor in our platform for all customers. Labeling DICOM data is no longer supported in Labelbox.

Future

  • On February 13th 2025, we will sunset the “Reporting” tab (AKA Enterprise Dashboard) for all customers. The Reporting tab has been replaced by the new Monitor tab. To learn more about this change, refer to this guide.

​
Python SDK

To view the latest updates to our Python SDK, see the What’s new section in the developer guides.

​
November 4, 2024

Release notes

​
App

Added

  • In the video editor, classifications can now be arbitrarily nested. Nested classifications can also be displayed in the timeline.
  • When you are configuring classifications in an ontology, you will see an option to toggle on/off the Likert scale option. Enabling the Likert scale option will automatically populate labeler classification responses with the first response option listed.
  • Now, when you enter a response in the multi-modal chat editor, you’ll see a button for “Get suggestions”. When you enable this AI-powered tool, you will get suggestions (code or grammar) to catch any mistakes you may have overlooked in your response. This feature is currently in beta.
  • When you are in review mode, you will now see Consensus scores next to the annotation names in the Tools menu.
  • You can now enable email notifications to stay up-to-date on changes related to project assignments, batch updates, and issue activity. To learn more, see our docs on Notifications.

Fixed

  • The Usage page got some UI improvements and additional filters for better usability. Changes include fixes to LBU calculation for custom dates, graphing capabilities, and LBU display on modal dialogs.
  • The toggle-off and backspace issues in the video editor have also been fixed.

Future

  • On November 7th 2024, Labelbox will sunset native support for YOLO models in Foundry. If you would like to set up your own YOLO model for inferencing, refer to our custom model integration docs.
  • On November 7th 2024, Labelbox will disable YOLOv8 for the model fine-tuning feature. This means that image fine-tuning will not be usable. We may re-enable image fine-tuning in the future.

​
Python SDK

  • To view the latest updates to our Python SDK, see the What’s new section in the developer guides.
​
October 1, 2024

Release notes

Added

  • The new Labelbox Microsoft Entra application allows you to install a verified version of the Labelbox Enterprise application into your Azure tenant. To learn how this works, see these docs.
  • The Performance tab in Annotate now contains an option to enable outlier detection metrics. These metrics highlight and bucket labelers who are outliers in key areas for quick remediation.
  • Customers who want to connect their own models to Labelbox now have a self-serve, product-integrated UI to bring their own models for LLM, classification, text, and NER use cases. Bounding boxes and masks will still require manual onboarding for the time being. To learn more, see our docs on Custom model integration.
  • The new Monitor page is now available for Enterprise customers. The Monitor contains charts, tables, and filters to display project and member performance across workspaces. It also contains new bulk actions and outlier detection metrics. To learn more, see our Monitor docs.

Changed

  • Asset proxy (the feature that hides raw data row URLs from being exposed in the platform) is now supported for all project types. To request access to this feature, contact our support team.

​
Python SDK

To view the latest updates to our Python SDK, see the What’s new section in the developer guides.

​
September 5th, 2024

Release notes

​
App

Added

  • Now, you can subscribe to email notifications for the following: 1) when labelers are assigned to (or unassigned from) a project, 2) when an issue is created, resolved, or commented on and 3) when a batch of data rows is added to a project or has been labeled. To manage email notifications, go to your user profile in the app UI.

Changed

  • The Multimodal chat editor has a new and improved UI. Key changes include: 1) When ranking, selecting, and classifying, the turn remains horizontal and 2) Instead of a left side panel, each turn is self-contained with its own tasks and classifications.
  • When you create new workflow tasks, you have more task filters to choose from: Issue category, Dataset, Batch, Metadata, Model prediction, Labeling time, Review time, Natural language, and Label feedback.
  • When you add task filters to workflow tasks, you can select “Match: Any” to apply OR logic or select “Match: All” to apply AND logic to the filters.
  • You will see an updated flow when requesting labeling services via the UI.

Removed

  • Export v1 has been sunsetted for all customers.

​
Python SDK

To view the latest updates to our Python SDK, see the What’s new section in the developer guides.

​
August 5, 2024

Release notes

​
App

Added

  • The multimodal chat editor now supports offline multimodal chat evaluation projects. To learn how this works, see our Multi-modal chat evaluation docs.
  • The Multimodal chat evaluation editor now supports LaTeX math expressions. To learn more, see our Multi-modal chat evaluation docs.
  • Enterprise customers will see an updated flow for requesting labeling services in the app UI.
  • You can now use the Benchmarks & Consensus tools for prompt & response generation projects.
  • You can now use the Benchmarks & Consensus tools for video projects.
  • Benchmarks & Consensus are now enabled for the following workflows: Send data rows from Model to Annotate, send data rows from Annotate to a model experiment, and Bulk classification in Catalog.
  • In the Data rows tab in Annotate, you will see an improved search and filtering experience for data rows with Benchmarks.
  • The audio editor now supports temporal classifications and a timeline, so you can assign classifications to specific points in the audio timeline.
  • When you compare model runs in the metrics and cluster view in Model, it will only display the metrics calculations for the data rows that appear in both model runs.

Changed

  • You can now configure your projects to use Benchmarks & Consensus simultaneously. If you have a project that has already been set up with Benchmarks, you can enable Consensus later in the project settings. However, it is still not possible to disable once set.
  • In the Data row browser, you can now set a Consensus label as a Benchmark label. You can do so by selecting a data row in the Data rows tab that has a Consensus label and select Add as Benchmark. This will calculate both the Benchmark and Consensus score for the data row.
  • The left side panel is now resizable across all editors.
  • The default reservation count for the labeling queue has been updated from 10 to 4. Admins can adjust the queue parameters in the project settings.
  • The Auto-segment tool in the image editor has been upgraded to use SAM 2.

Removed

  • The custom editor has been sunsetted for all customers.
  • The following data connector libraries have been archived: labelbase, labelspark, labelpandas, labelsnow, labelbox-bigquery. To learn more, see our Deprecations page.
  • The External workforce tab has been removed from the project settings.

​
Python SDK

The latest version of our Python SDK is v3.76.0. See our full changelog in Github for more details on what was added recently.

​
Version 3.76.0 (2024-07-29)

Added

  • Added Project get_labeling_service(), request_labeling_service() and get_labeling_service_status()
  • Added project and ontology creation for prompt response projects: Client create_prompt_response_generation_project(), create_response_creation_project()
  • Added is_benchmark_enabled, is_consensus_enabled to Project

Updated

  • Made Project quality modes a list to allow combining more than 1 quality mode per project

Notebooks

  • Added back in export migration guide
  • Added correct data param to video notebook

Other

  • Use connection pool for all http and graphql requests

​
Version 3.75.1 (2024-07-16)

Removed

  • Project MEDIA_TYPE JSON

​
Version 3.75.0 (2024-07-10)

Added

  • Added Project set_project_model_setup_complete() method
  • Added user group management
  • Refactor Dataset create_data_rows_sync to use upsert
  • Added upload_type to Project
  • Added prompt classification for python object support
  • Alias wait_xxx functions

Fixed

  • Predictions missing during Catalog slice Export
  • Prevented adding batches to live chat evaluation projects
  • Added missing media types
  • Deprecate Project setup_editor and add Project connect_ontology
  • Bumped dateutil max version
  • Bumped version rye
  • Updated create ontology for project setup
​
July 5, 2024

Release notes

​
App

Added

  • For video projects, you can now assign free-form text classifications to individual frames or a range of frames.
  • The Usage tab in Workspace settings now contains LBU usage per product and data modality. It also displays usage and project billing costs for Boost Workforce and Boost Workforce Express projects. To learn more, see our docs on Account details.
  • Enterprise and pay-as-you-go customers will see an improved setup flow when configuring a data warehouse integration. When you initiate a new integration in the Integrations tab, you’ll see step-by-step instructions to guide you through the setup process. To learn more, see our Census integration docs.
  • Admins can define their own user roles with any set of permissions that Labelbox provides and allow those roles to be assigned to any user. To learn more, see our docs on Member roles.
  • Benchmarks and Consensus are now supported for projects created using the “Prompt and response generation” editor. This is only supported when you select the “Humans generate responses to uploaded prompts” option.
  • Claude 3.5 Sonnet is now available in Foundry.
  • Images are now supported for Claude 3 Opus, Claude 3.5 Sonnet, and Claude 3 Haiku in Foundry.

Changed

  • You can now configure your project settings to use the Benchmarks and Consensus tools at the same time.
  • Contract-based customers will see an updated flow when requesting Boost workforce services.
  • We made some improvements to our annotation import formats to make them more intuitive. To see the updated import formats, click through the links at the top of the Import ground truth doc.

Fixed

  • A fix that blocks concurrent edits to the same label has addressed the platform issues causing duplicate classifications, multiple segmentation groups, and options with deleted questions.
  • The root cause of certain data integrity issues, such as missing pixels in raster segmentation masks and multiple/duplicate classifications, has been addressed.
  • A fix in our queueing system has boosted Export v2’s overall performance, significantly improving export performance on larger scales.

Removed

  • Claude Instant, Claude 2, and Claude 3 Sonnet have been removed from Foundry due to lack of usage.
  • The custom editor has been sunset for all customers. Custom editor projects are no longer accessible (exporting is also disabled) via the app and the SDK. Custom editor projects will no longer appear in your project list.

​
Python SDK

The latest version of our Python SDK is v3.74.0. See our full changelog in Github for more details on what was added recently.

​
Version 3.74.0 (2024-06-24)

Added

  • Include predictions in export
  • Upsert label feedback method Client upsert_label_feedback()

​
Version 3.74.0 (2024-06-24)

Added

  • Include predictions in export
  • Upsert label feedback method Client upsert_label_feedback()

Removed

  • Removed deprecated class LabelList

​
Version 3.73.0 (2024-06-20)

Added

  • Conversational data row checks
  • UI ontology mode support
  • Empty data row validation

Fixed

  • Numpy semver locked to < 2.0.0

​
Version 3.72.2 (2024-06-10)

Added

  • SLSA provenance generation

​
Version 3.72.1 (2024-06-06)

Fixed

  • Fix client.get_project() for LLM projects
  • Throw user-readable errors when creating a custom embedding
​
June 5, 2024

Release notes

​
App

Added

  • Now you can browse video assets (as well as their annotations and predictions) in the detailed view within Catalog, Model, and Annotate.
  • Our Foundry product is now available as GA. Foundry enables you to use powerful foundation models to prelabel your data and integrate directly with Annotate workflows. To learn more, read our docs on Foundry.
  • Now, when you create an API key, you can specify its scope to a particular permission level. See our docs on API keys to learn more.
  • You can now clone projects and their settings (ontology, members, name, tags, issue categories). To do this, go to Annotate, select a project, click on the project name at the top, and select Duplicate project.
  • We introduced a new role called tenant admin. This tenant admin role has the permission to create new workspaces and invite team members to any workspace. See our docs on Tenant Admin to learn more.
  • OpenAI’s GPT-4o, as well as Anthropic’s Claude 3 Sonnet and Claude 3 Haiku models, are now available in Foundry.
  • When you open an experiment from Model, you’ll see a new view called “List view”. This new view displays each data row as a row in a list. Each row displays a preview of the asset and any annotation and/or prediction information for the data row. See our docs on Model runs to learn more.
  • Now, in the data rows tab, you can filter data rows by agreement score at the feature level. To do this, go to the Data rows tab, filter by “Consensus average,” and select “Feature”. You can further narrow down your results by feature schema. To learn more, see our docs on the Concensus.
  • We released a new live multimodal chat evaluation editor that enables you to enter a prompt, get a response from multiple models, and use an ontology to rank and select the best model responses.

Removed

  • On March 16, 2024, we sunset Export v1 for pro, standard, and select enterprise customers.

Future

  • On June 30, 2024, we will sunset the custom editor for all customers.

​
Python SDK

The latest version of our Python SDK is v3.72.0. See our full changelog in Github for more details on what was added recently.

​
Version 3.72.0 (2024-06-04)

Added

  • Update Dataset create_data_rows to allow upload of an unlimited number of data rows
  • New Dataset methods for iam_integraton: add_iam_integration, remove_iam_integration

Notebooks

  • Added model evaluation SDK method notebook
  • Added quick start notebook geared towards new users

​
Version 3.71.0 (2024-05-28)

Added

  • project.get_overview() to be able to retrieve project details
  • project.clone() to be able to clone projects
  • ExportTask.get_buffered_stream to replace ExportTask.get_stream

Fixed

  • ExportTask.result / ExportTask.errors parsing content incorrectly
  • Lack of exceptions related to updating model config

​
Version 3.70.0 (2024-05-20)

Added

  • Added chat model evaluation support
    • client.create_model_config()
    • ModelConfig project_model_configs()
    • ModelConfig add_model_config()
    • ModelConfig delete_project_model_config()
    • ProjectModelConfig delete()
    • client.create_model_evaluation_project()
  • Update existing methods to support chat model evaluation project
    • client.create_ontology()
    • client.create_ontology_from_feature_schemas()
  • Coco deprecation message

Fixed

  • Fixed error reporting for client.create_project()
  • Do not retry http 422 errors

Notebooks

  • Send_to_annotate_from_catalog functionalities outside Foundry

Fixed in Notebooks

  • Fixed meta notebook
  • Modified queue_management.ipynb to remove some parameters
  • Update_huggingface.ipynb

​
Python SDK

Added

  • Dark mode (sun/moon icon on the top right-hand side)

Fixed

  • All example notebooks from our SDK repository are now in the Python tutorials tab
​
May 7, 2024

Release notes

​
App

Added

  • Our new warehouse integration p rovides an easy, no-code solution to keep your datasets in Labelbox in sync with the tables in your data warehouse. You can use this integration to connect over 25 different data sources, including Big Query, Databricks, Snowflake, and Google Sheets. To learn more, see our Census integration docs.
  • You can use the new fine-tuning capability (beta) to fine-tune a YoloV8 object detection model with custom features. To run fine-tuning, you’ll need to create a model experiment, import data rows and ground truth, and add an ontology to use for fine-tuning. Then, select the “fine-tune model” button to configure the training job. Docs on this feature are coming soon.
  • You can now include custom and auto-generated embeddings when you export your data rows via Export v2. You can find an example of exported embeddings on this page: Export image annotations.
  • When you create an issue, you now have the option to assign the issue to a category. Issue categories can be viewed and managed in the Issues tab in a project. To learn more, see our docs on Issues.
  • The following model + annotation types are now supported in Foundry:
    • Video bounding box detection via GroundingDINO model
    • Video segmentation mask detection via GroundingDINO + SAM model
    • Video frame-based classification via Google Gemini 1.5 Pro (Beta) model
    • Video global classification via X-Clip, Gemini Pro Vision, Gemini 1.5 Pro (Beta) models
  • We introduced new ways to create model experiments in the UI: 1) From the Manage selection dropdown in Catalog, you can send selected data rows to a new experiment or an existing experiment. 2) From Model, you can now click Create -> Experiment to create a new experiment. 3) From an existing experiment, you can now click + New Model Run to append data rows to the experiment.
  • LlaMa v3 is now supported in Foundry.

Changed

  • Our updated home page provides you with key actions, featured reads, and entry points to make it easier to get started on projects.
  • For projects that are configured with Boost Workforce Express, labeler instructions are now added as part of the ontology. We also enabled hyperlinking in the ontology Instructions to allow you to link a Google doc as labeler instructions.
  • With the new Catalog search experience, you can quickly search your data in Catalog by typing into the data search filter bar. You no longer need to specify a filter to find the data you are looking for. See Filters for more details.
  • The rate limit for Gemini 1.5 Pro increased from 5 to 600 requests per minute for Labelbox Foundry.

Removed

  • On April 19, we sunsetted Export v1 for free, edu, and starter customers. To learn more about this deprecation, see our migration guide.

Future

  • On May 15, we will begin sunsetting Export v1 for pro, standard, and enterprise customers. To learn more, see our migration guide.
  • On June 30, we will be sunsetting the custom editor for all customers.

​
Python SDK

The latest version of our Python SDK is v3.69.1. See our full changelog in Github for more details on what was added recently.

​
Version 3.69.1 (2024-05-01)

Fixed- Fixed a bug with certain types of content not being returned as a result of ExportTask.result or ExportTask.errors

​
Version 3.69.0 (2024-04-25)

Added

  • Support to export embeddings from the SDK

Fixed

  • Used OpenCV’s headless library in replacement of OpenCV’s default library

​
Version 3.68.0 (2024-04-16)

Added

  • Added support for embeddings.
  • Introduced the use of ‘rye’ as a package manager for SDK contributors.
  • Implemented a unified create method for AnnotationImport, MEAPredictionImport, and MALPredictionImport.
  • Enhanced annotation upload functionality to accept data row IDs, global keys, or external IDs directly for labelbox.data.annotation_types.label

Fixed

  • Ensure items in dataset.upsert_data_rows are not empty
  • Streamable export fix to report export_v2 errors as list of dictionaries, compatible with older releases

​
Version 3.67.0 (2024-04-05)

Added

  • Added SECURITY.md file
  • Made export_v2 methods use streamable backend
  • Added support for custom embeddings to dataset create data row(s) methods
  • Added ability to upsert data rows via dataset.upsert_data_rows() method
  • Added AssetAttachment with an ability to update() and delete()

Updated

  • Added check for 5000 labels per annotation per data row

Fixed

  • Errors and Failed data rows are included in the task.result for dataset.create_data_rows()
  • Fixed 500 error handling and reporting

Notebooks

  • Updated import notebook for image data
  • Added attachment PDF example, removed requirements around text_layer_url
  • Included the get_catalog() method to the export notebook
  • Added workflow status filter to export_data notebook for projects
  • Send predictions to a project demo
  • Removed model diagnostic notebooks
​
April 2, 2024

Release notes

​
App

Added

  • You can now filter and sort by custom metrics at the prediction level in the Model runs view. This new filter capability supports autogenerated metrics as well as custom metrics at the prediction level. To learn more, see Filters.
  • When you enter the Schema tab, you’ll see a new tab for Embeddings. In this part of the app, you can view all of your autogenerated and custom embeddings for your organization. Here, you can also create custom embeddings via the UI. See Embeddings to learn more about the Embeddings subtab.
  • You can now add PDFs as attachments to data rows via the PDF_URL attachment type. To view an example of importing a PDF as an attachment, see Attachments.
  • If you are a new user and do not have any data in the Catalog yet, Labelbox will show a zero-state screen with guidance on how to add data rows.
  • If you are a new user and do not have any projects in Annotate yet, Labelbox will show a zero-state screen with guidance on how to create your first project.
  • Foundry now includes support for Claude 3 Opus and Google Gemini 1.5 Pro.
  • The new inferencing endpoints (beta) enable you to run any Foundry app to generate predictions on your data via a REST API. You can use this REST API to 1) pass raw data without creating data rows or a dataset, 2) pass raw data and specify which dataset to create the data row, or 3) pass a specific data row that has already been created in Labelbox. To learn more, see our docs on Foundry apps.

Changed

  • When you import PDFs as data rows, you no longer need to specify the text_layer_url as Labelbox now automatically generates the text layers when you import PDF assets. This means you no longer need to create the text layers and include them when importing PDF assets.
  • We now render the image overlay attachments in the order they were created. This means they are no longer randomly ordered when you view them in the editor.
  • Now when you click on a relationship edge in the editor, the relationship will be selected in the objects panel. If the relationship has a subclassification, the subclassification will open in the left panel. This improvement impacts the text and conversational text editors. To learn more, see our docs on Text relationships.

Fixed

  • The issue where data rows were stuck in the “To label” queue has been resolved. Now data rows that have been labeled or skipped should be moved to the next workflow step, reliably.
  • The issue causing some assets to not load properly in the data row browser has been resolved.
  • We implemented a fix to ensure feature names with underscores are automatically formatted with surrounding spaces for accurate model processing.

Future

  • On April 20, 2024, we will sunset Export v1 for Free, EDU, and Starter tiers. On April 27, 2024, we will sunset Export v1 for Standard, Pro, and Enterprise tiers. To learn more, see the Export v1 to v2 migration guide.
  • On Jun 30, 2024, we will be sunsetting our custom editor for all customers.

​
Python SDK

The latest version of our Python SDK is v3.66.0. See our full changelog in Github for more details on what was added recently.

​
Version 3.66.0 (2024-03-20)

Added

  • Added support for Python 3.11, 3.12
  • Added update method to attachments

Notebooks

  • Improved notebooks for integration and model diagnostics
  • Removed databricks integrations notebooks

Updated

  • Updated README for clarity and contribution guidelines

Removed

  • Removed support Python 3.7 as it has been end of life since June 2023
​
March 5, 2024

Release notes

​
App

Added

  • Smart select (beta) in Catalog offers three different ways to curate your data within a dataset: random, ordered, and cluster-based. To access this feature, go to Catalog, select a dataset that is larger than 100 data rows, and select “Smart select”.
  • Cluster view (beta) in Catalog is a new interface that helps you discover and find similar data rows as well as labeling or model mistakes. See Cluster view for more details.
  • Natural language search and similarity search in Catalog now support video and audio assets.
  • You can now 1) import custom metrics per prediction and 2) filter/sort by prediction-level custom metrics. These added functionalities make it easier to use our model error analysis tools in the Model product. To learn more about prediction-level metrics on model runs, see Filters.
  • You can now invoke Foundry models from within a project to generate pre-labels. In the project overview, you’ll see “suggested models” to help you select the best model for generating predictions. This feature is in beta.
  • The new Audit trail feature provides a way for you to view all operations (create, update, and delete) on an annotation. This feature makes monitoring changes to annotations over time easier for auditing purposes.
  • The new enhanced video player (beta) in Catalog offers new capabilities to help you curate, evaluate, and visualize video data. These new capabilities include basic play/pause controls, support for predictions, pre-labels, ground truth (bounding boxes, polylines, and points), and support for comparing model runs.
  • If you export from an image or video labeling project that contains data rows labeled with segmentation masks, you will see a new property in the export file called composite_mask. Composite masks are a grouping of all segmentation masks on a label. To learn more, see Export image annotations.
  • If you export from a model run that contains segmentation mask predictions, you will see a new property in the export file called composite_mask. To learn more, see Export image annotations.
  • You can now have a workflow filter based on consensus average. For example, you can use this filter to send data rows with a consensus score over 80% straight to “Done”. To do this, create a new workflow step and select consensus average from the workflow filters.
  • Boost express makes it easier for you to request help labeling data. Once you’ve set up a project, you can request up to fifteen labelers to work on your data. See our docs on Boost express to learn more.
  • The billing tab in your Workspace settings now has a button that enables you to easily unsubscribe from Foundry.

Changed

  • When you select a project in Annotate, you will see a new project overview UI that makes navigating your project easier.
  • We have disabled the ability to download videos from the preview mode for all users.
  • When you are in a project, you will now see a “Start” button with a dropdown menu containing options for labeling, reviewing, and any other tasks you have configured for your project.

Fixed

  • Google SSO users that have a profile picture on their account can now log in as expected.
  • The issue causing large numbers of data rows to be stuck in the “To Label” task has been fixed.
  • The Enterprise dashboard now displays the correct information for your organization.
  • In the Data row details panel, the row data can no longer be displayed with the security token.

Future

​
Python SDK

The latest version of our Python SDK is v3.65.0. See our full changelog in Github for more details on what was added recently.

​
Version 3.65.0 (2024-03-05)

Notes

  • Rerelease of 3.64.0

​
Version 3.64.0 (2024-02-29)

Added

  • Client.get_catalog Add catalog schema class. Catalog exports can now be made without creating a slice first
  • last_activity_at filter added to export_v2, allowing users to specify a datetime window without a slice

Removed

  • Review related WebhookDataSource topics

Notebooks

  • Added get_catalog notebook
  • Update custom metrics notebook
  • Update notebooks for video and image annotation import

​
Version 3.63.0 (2024-02-19)

Added

  • Ability for users to install and use sdk with pydantic v.2. while still maintaining support for pydantic v1.
  • ModelRun export() and export_v2() add model_run_details to support splits

Notebooks

  • Add composite mask notebook

​
Version 3.62.0 (2024-02-12)

Added

  • Support custom metrics for predictions (all applicable annotation classes)
  • FoundryClient.run_app Add data_row identifier validation for running foundry app
  • Client.get_error_status_code Default to 500 error if a server error is unparseable instead of throwing an exception

Updated

  • DataRowMetadata, DataRowMetadataBatchResponse, _UpsertBatchDataRowMetadata Make data_row_id and global_key optional in all schema types

Fixed

  • ExportTask.__str__ Fix returned type in ExportTask instance representation

Removed

  • Project.upsert_review_queue

Notebooks

  • Update notebooks to new export methods
  • Add model slice notebook
  • Added support for annotation import with img bytes
​
February 5, 2024

Release notes

​
App

Added

  • When you select a project in Annotate, you will now see a count for “Skipped” data rows in the project overview.
  • The text search filter in Catalog now supports text attachments.
  • The new Cluster view (beta) in Catalog is a visual tool that allows you to explore relationships between data rows, identify edge cases and outliers, select for pre-labeling or human review, and quickly classify large datasets in bulk. Read our docs on Cluster view (beta) to learn more.
  • The new Cloud bucket sync (beta) provides a simple way for you to sync the data in your cloud buckets to Labelbox (supported for AWS S3 buckets, Google Cloud Storage, and Microsoft Azure Blob Storage). You must have IAM delegated access configured in order to use the cloud bucket sync (beta) in Catalog.
  • Foundry now has the following improvements:
    • GPT4v will now support PDF files
    • Gemini Pro Vision supports video classification use cases
    • Improved speed for preview mode for GPT-like models and llava\llama
    • Increased the speed on the submit job mode

Changed

  • We removed the technical limitation that prevented customers from exporting over 700K data rows at a time. Read our docs on streamable exports and Limits to learn more about the new export limits.
  • Our new flat-rate LBU pricing system makes understanding your usage in Labelbox easier. To learn more about the recent LBU pricing changes, see our Manage account and billing docs.
  • When you select a project in Annotate, you will see a new project overview UI that shows labeling progress, workflow tasks, and other key configurations for your project.
  • We made some improvements to increase the reliability of routing data rows within Annotate workflows. This improvement mostly helps large-scale organizations that rely heavily on review workflows and workflow history for their operations.

Fixed

  • When you update the metadata field on a data row, the change should be reflected in the “Last activity” filter in Catalog.
  • Creating multiple relationships between points and other vector annotations in the editor now works as expected.
  • The issue where tracked bounding boxes appear outside the editor canvas has been fixed.
  • The issue of the editor not allowing you to unhide a single annotation when the whole annotation group is hidden has been fixed. You can now hide/unhide individual annotations as expected.
  • The hotkeys for zooming in and out of an asset in the editor have been fixed.
  • Now, when a user navigates back to a skipped data row in the editor, the user can’t navigate to the next asset without filling in all required fields or discarding the changes.
  • The sequence of undoing/redoing annotations and then creating new annotations now works as expected.

Future

​
Python SDK

The latest version of our Python SDK is v3.61.2. See our full changelog in Github for more details on what was added recently.

​
Version 3.61.2 (2024-01-29)

Added

  • ModelSlice.get_data_row_identifiers for Foundry data rows

Fixed

  • ModelSlice.get_data_row_identifiers scoping by model run id

​
Version 3.61.1 (2024-01-25)

Fixed

  • Removed export API limit (5000)

​
Version 3.61.0 (2024-01-22)

Added

  • ModelSlice.get_data_row_identifiers
    • Fetches all data row ids and global keys for the model slice
    • NOTE Foundry model slices are note supported yet

Updated

  • Updated exports v1 deprecation date to April 30th, 2024
  • Remove streamable param from export_v2 methods

​
Version 3.60.0 (2024-01-17)

Added

  • Get resource tags from a project
  • Method to CatalogSlice to get data row identifiers (both uids and global keys)
  • Added deprecation notice for the upsert_review_queue method in project

Notebooks

  • Update notebook for Project move_data_rows_to_task_queue
  • Added notebook for model foundry
  • Added notebook for migrating from Exports V1 to V2

​
Version 3.59.0 (2024-01-05)

Added

  • Support set_labeling_parameter_overrides for global keys
  • Support bulk_delete of data row metadata for global keys
  • Support bulk_export of data row metadata for global keys

Fixed

  • Stop overwriting class annotations on prediction upload
  • Prevent users from uploading video annotations over the API limit (5000)
  • Make description optional for foundry app

Notebooks

  • Update notebooks for Project set_labeling_parameter_overrides add support for global keys
​
January 4, 2024

Release notes

​
App

Added

  • Foundry is now available to all users. Foundry enables you to use foundational models to predict annotations for your data and to create model runs to compare and diagnose the behavior of different models against your data and requirements. To learn more, read our docs.
  • In the Catalog, there is now an easy way to send predictions straight to Annotate. Once you’ve generated predictions with Foundry, you can send them to Annotate either as prelabels or as labels for human review, either as prelabels or as annotations associated with a specific labeling step. To learn more, read these docs.
  • Catalog similarity search now works on up to 1 billion data rows at a time.
  • We’ve updated our Create dataset workflow to highlight the following import options: Python SDK script, Cloud buckets integration, Data warehouse sync, or local file upload. Go to Catalog and select +New to see the new import flow.

Fixed

  • When you navigate to Model —> Create —> Experiment, the Experiment page now opens as expected.
  • For ontologies containing required radio and/or checklist per message classifications, if a user attempts to remove the classification answer, they will be shown an error message and the classification value will be restored.
  • In the Data Rows tab, if a user inserts a value for Media attribute —> Attribute value —> Duration, the filter will now be applied correctly and the correct results will be shown.
  • Undo/redo behavior in the editor is limited to the current asset only.
  • When an Issue has resolved status, the Resolve button now has “Reopen” text instead, as it reopens a resolved Issue.

​
Python SDK

The latest version of our Python SDK is v3.58.1. See our full changelog in Github for more details on what was added recently.

​
Version 3.58.1 (2023-12-15)

Added

  • Support to export all projects and all model runs to export_v2 for a dataset and a slice

Notebooks

  • Update exports v2 notebook to include methods that return ExportTask

​
Version 3.58.0 (2023-12-11)

Added

  • ontology_id to the model app instantiation
  • LLM data generation label types
  • run_foundry_app to support running model foundry apps
  • Two methods for sending data rows to any workflow task in a project, that can also include predictions from a model run, or annotations from a different project

Fixed

  • Documentation index for identifiables

Removed

  • Project.datasets and Datasets.projects methods as they have been deprecated

Notebooks

  • Added note books for Human labeling(GT/MAL/MEA) + data generation (GT/MAL)
  • Remove relationship annotations from text and conversational imports
​
Welcome to Labelbox

Welcome to the developer hub and documentation for Labelbox!