August 5th, 2024

​

App

• The multimodal chat editor now supports offline multimodal chat evaluation projects. To learn how this works, see our Multi-modal chat evaluation docs.
• The Multimodal chat evaluation editor now supports LaTeX math expressions. To learn more, see our Multi-modal chat evaluation docs.
• Enterprise customers will see an updated flow for requesting labeling services in the app UI.
• You can now use the Benchmarks & Consensus tools for prompt & response generation projects.
• You can now use the Benchmarks & Consensus tools for video projects.
• Benchmarks & Consensus are now enabled for the following workflows: Send data rows from Model to Annotate, send data rows from Annotate to a model experiment, and Bulk classification in Catalog.
• In the Data rows tab in Annotate, you will see an improved search and filtering experience for data rows with Benchmarks.
• The audio editor now supports temporal classifications and a timeline, so you can assign classifications to specific points in the audio timeline.
• When you compare model runs in the metrics and cluster view in Model, it will only display the metrics calculations for the data rows that appear in both model runs.

• You can now configure your projects to use Benchmarks & Consensus simultaneously. If you have a project that has already been set up with Benchmarks, you can enable Consensus later in the project settings. However, it is still not possible to disable once set.
• In the Data row browser, you can now set a Consensus label as a Benchmark label. You can do so by selecting a data row in the Data rows tab that has a Consensus label and select Add as Benchmark. This will calculate both the Benchmark and Consensus score for the data row.
• The left side panel is now resizable across all editors.
• The default reservation count for the labeling queue has been updated from 10 to 4. Admins can adjust the queue parameters in the project settings.
• The Auto-segment tool in the image editor has been upgraded to use SAM 2.

• The custom editor has been sunsetted for all customers.
• The following data connector libraries have been archived: labelbase, labelspark, labelpandas, labelsnow, labelbox-bigquery. To learn more, see our Deprecations page.
• The External workforce tab has been removed from the project settings.

​

Python SDK

​

Version 3.76.0 (2024-07-29)

• Added Project get_labeling_service(), request_labeling_service() and get_labeling_service_status()
• Added project and ontology creation for prompt response projects: Client create_prompt_response_generation_project(), create_response_creation_project()
• Added is_benchmark_enabled, is_consensus_enabled to Project

• Made Project quality modes a list to allow combining more than 1 quality mode per project

• Added back in export migration guide
• Added correct data param to video notebook

• Use connection pool for all http and graphql requests

​

Version 3.75.1 (2024-07-16)

• Project MEDIA_TYPE JSON

​

Version 3.75.0 (2024-07-10)

• Added Project set_project_model_setup_complete() method
• Added user group management
• Refactor Dataset create_data_rows_sync to use upsert
• Added upload_type to Project
• Added prompt classification for python object support
• Alias wait_xxx functions

• Predictions missing during Catalog slice Export
• Prevented adding batches to live chat evaluation projects
• Added missing media types
• Deprecate Project setup_editor and add Project connect_ontology
• Bumped dateutil max version
• Bumped version rye
• Updated create ontology for project setup