Search and view compatibility

A summary of search and view capabilities available in Catalog by data type.

Below are the supported search filters that Catalog supports out of the box. You can extend the usability of these core offerings by uploading your own custom metadata and embeddings, then combining them with the supported filters below to accomplish your search and curation objectives.

Supported search filters (basic)

To learn more about the supported filters, see Filters.

Asset typeAnnotationDatasetMetadataProjectMedia attributeBatchData row
Image
Video
TextOnly mime type
HTMLOnly mime type
DocumentOnly mime type
Tiled imageryOnly mime type
DICOMOnly mime type
Audio
ConversationalOnly mime type

Enrich your data with custom metadata

As noted above, Labelbox supports metadata on any data type. To learn how to use custom metadata and these Catalog filters to enrich your data, read this blog post on how to make your data queryable using foundation models.

Supported search filters (advanced)

Asset typeFind textNatural language
Image-✔*
Video--
Text✔**
HTML✔**
Document✔***
Tiled imagery-✔*
DICOM--
Audio--
Conversational✔**

* Uses the off-the-shelf CLIP-ViT-B-32 vision model (512 dimensions).

** Uses the off-the-shelf all-mpnet-base-v2 text model (768 dimensions), based on the first 64k characters.

*** Users can pick between off-the-shelf CLIP-ViT-B-32 vision model (512 dimensions) and all-mpnet-base-v2 text model (768 dimensions, based on the first 64k characters).

Similarity (embeddings)

Asset typeOff-the-shelf embeddingsCustom embeddings
ImageCLIP-ViT-B-32 (512 dimensions)Up to 2048 dimensions per embedding; up to 100 custom embeddings per workspace.
Video-Up to 2048 dimensions per embedding; up to 100 custom embeddings per workspace.
Textall-mpnet-base-v2 (768 dimensions)Up to 2048 dimensions per embedding; up to 100 custom embeddings per workspace.
HTMLall-mpnet-base-v2 (768 dimensions)Up to 2048 dimensions per embedding; up to 100 custom embeddings per workspace.
DocumentCLIP-ViT-B-32 (512 dimensions)

and

all-mpnet-base-v2 (768 dimensions)
Up to 2048 dimensions per embedding; up to 100 custom embeddings per workspace.
Tiled imageryCLIP-ViT-B-32 (512 dimensions)Up to 2048 dimensions per embedding; up to 100 custom embeddings per workspace.
DICOM-Up to 2048 dimensions per embedding; up to 100 custom embeddings per workspace.
Audio-Up to 2048 dimensions per embedding; up to 100 custom embeddings per workspace.
Conversational-Up to 2048 dimensions per embedding; up to 100 custom embeddings per workspace.

Enhance similarity search with custom embeddings

As noted above, Labelbox supports custom embeddings on any data type. Powerful embeddings can be generated using foundational models and easily uploaded to Labelbox. You can then use these embeddings in combination with any of the above filters to accomplish your data search goals.

For an example of how to get started, check out this guide on how to generate custom embeddings using foundational models and upload them to Labelbox.

Options for viewing your data

Asset typeThumbnail viewDetail viewAnnotations overlay (thumbnail)Annotations overlay (detail)
Image
Video-Classifications only
Text
HTML-
DocumentClassifications onlyClassifications only
Tiled imagery
DICOM-
Audio---
Conversational---Classifications only