Labelbox documentation

FAQ

General FAQ

What is Labelbox?

Labelbox is the industry-leading labeling and training data management platform.

Does Labelbox only sell labeling services?

Labelbox is primarily a training data platform that offers the following

  • A user-friendly, customizable platform for labeling images, video, text, and geospatial data.

  • A model-assisted labeling workflow.

  • Extensive team collaboration tools.

  • Quality and performance management tools.

  • Dataset management.

  • A labeling workforce that you can integrate into your plan.

How do I set up a labeling project?

Once you have created a Labelbox account, see Create a project. To learn how to set up a project programmatically see this Python tutorial.

Do I need to be a data scientist to use Labelbox?

You do not need to be a data scientist to label data in the Labelbox application. We designed our user-friendly platform to require minimal technical knowledge and expertise such that anyone can annotate data using our out-of-the-box application. However, it may be beneficial to have someone experienced in data science on your team to help ensure that your machine learning project is successfully developed and deployed.

What kind of data can I label with Labelbox?

With our out-of-the-box labeling application, you can label the following:

  • Images

  • Videos

  • Text

  • Geospatial data

To see a list of supported file formats, see our Data model reference.

How do people use the labels they create in Labelbox?

In most cases, Labelbox is used by individuals, organizations, and companies in need of high-quality training data to develop and deploy robust machine learning models. People choose Labelbox to create training data for various applications, ranging from independent, small-scale projects to enterprise-scale applications that leverage artificial intelligence to gain a competitive edge.

What is the difference between the new Editor, the Tiled Imagery Editor, and the Custom Editor?

When you create a project, you have the option to choose from the following labeling interfaces:

  • Editor — also known as the “Image editor”, “Pictor”, or the "Segmentation editor" — is the most recent Labelbox labeling interface released at the end of 2019. This latest version provides the most flexibility for creating semantic segmentation and vector-based annotations. It also supports bulk importing predictions as well as more robust quality assurance tools.

  • Legacy Editor — also referred to as "v4" or the “object detection editor” — is the previous version of the label editor.

  • Custom Editor allows you to create a completely customized template with just HTML and JavaScript. To get started, check out our out-of-the-box custom label editor templates. Visit our Custom Editor SDK in Github to learn more.

Where can I find the Custom Editor SDK?

If you find that the out-of-the-box Editor does not have the functionality you need, you can build your own custom labeling interface with the Custom Editor SDK.

Billing and pricing FAQ

How much does Labelbox cost?

For organizations building expert artificial intelligence systems, we offer Pro & Enterprise licenses. For smaller projects, we offer a Developer license which is free for evaluation, individuals, and small projects with a limit of 2500 labels per year. Check out labelbox.com/pricing for more information on the Developer, Pro, and Enterprise license options.

Does Labelbox have a discount for nonprofit organizations or academic users?

If you will be using Labelbox for educational or non-commercial purposes, you may be eligible for an Education license. To apply, visit labelbox.com/education or contact us at support@labelbox.com.

How much does the Labelbox Workforce cost?

The cost of a labeling workforce familiar with the Labelbox application is included with all Enterprise contracts. Please visit this page or reach out to support@labelbox.com for more information.

Account management FAQ

What happens to my labeled data if I migrate from a paid plan to the Developer Tier?

If you migrate from a paid plan to the developer (free) tier, Labelbox will not delete any data from your account. If you have already submitted 2,500 labels in the past 12 months, you will no longer be able to export labels.

If my account has more than 5 users, will any users be deactivated when I migrate from a paid account to the Developer Tier?

No users will be deactivated, but you will not be able to add additional users.

Labeling workforce FAQ

Can I outsource a team to label my data?

If you would like to outsource a workforce to label your data in Labelbox, you have two options:

  • Use our Labelbox workforce when you sign up for a Pro or Enterprise account.

  • Find a third party workforce and teach them how to use Labelbox.

Can I still use Labelbox if I already have a labeling team?

Yes. Whether your labeling team is outsourced, internal, or both, Labelbox allows your labeling team to annotate data while giving you the tools to measure their label quality and performance. If needed, you have the ability to combine the efforts of your internal team and an outsourced labeling workforce on the same projects.

Support FAQ

Does Labelbox provide support?

Labelbox provides support to all Labelbox users. We have a team of dedicated machine learning support engineers based in San Francisco from 9 am - 5 pm PST. However, you are welcome to send us a support request outside of these hours and one of our support engineers will respond at their earliest availability. Enterprise accounts will receive a dedicated account manager for additional support as included in the contract. The best way to reach our support team is via the chat feature at the bottom right corner of the web page. You can also email us at support@labelbox.com.

Quality assurance FAQ

What is the difference between Consensus and Benchmark?

The Consensus tool allows you to compare the label of one member against the labels of other members to automatically calculate the overall consensus. You can configure the percentage of your dataset in a project that will be selected to be labeled more than once by distinct members. Because this calculation compares a labeler's work to that of other labelers, you can see which labelers are producing labels that tend not to agree with the majority. You can also track the consensus of your labeled dataset against the entire labeling team to ensure high label quality. Poor consensus may also be an indicator of a need to clarify the ontology or classification questions in order to boost the consensus scores.

The Benchmarks tool allows you to set a “gold standard” annotation and compare your labelers' annotations against it. The benchmark tool works by interspersing data to be labeled, for which there is a benchmark label, to each person labeling. The accuracy score for each label is calculated based on its comparison to its respective benchmark.

Will other labelers see the data I am labeling?

If Consensus is turned on and configured for your project, yes. Otherwise, no.

Can I compare model-assisted predictions against human-made labels?

With Labelbox, you can connect your machine learning model and compare your model’s performance against your team of labelers. To learn how to import model predictions to your project, see How to import annotations via Model-assisted labeling.

Technical FAQ

Does Labelbox have an API or SDK?

Labelbox offers a Python SDK for those who wish to use Labelbox programmatically using Python instead of by writing GraphQL queries and mutations. Visit our Python SDK docs to learn more.

Access to the Labelbox GraphQL API is included in the Pro and Enterprise tiers. Visit our GraphQL API docs to learn more.

How do I create an API key?

API Keys can only be created by organization administrators. To create an API key in the Labelbox UI, go to app.labelbox.com/account/api-keys.

Importing & exporting data FAQ

I already have training data. Can I import it into Labelbox?

Whether you are uploading model-assisted predictions or human-labeled data, you can import your labels via the app or programmatically. To learn more, see our docs for Model-assisted labeling.Model-assisted labeling

Which label export formats does Labelbox support?

Your labeled data can be exported in JSON or CSV format, which should allow you to import it into any standard machine learning framework. Labelbox does not offer any native export functionality for COCO or PASCAL VOC.

Does Labelbox work with on-prem data?

Labelbox works with source data hosted on-premises or on a private cloud. For on-prem configurations, the source data is accessed directly from the client computer and is never shared by (or accessible to) Labelbox. To learn more about our cloud integrations and on-prem configurations, see our docs on Labelbox deployment configurations.

Does Labelbox work with cloud data such as AWS or Google Cloud Storage?

Yes. To learn how to connect your cloud data to Labelbox, see How to connect your data via IAM delegated access (recommended) or How to connect your data via customer-secured URLs.

Can I import images from my smartphone?

Notably (but not exclusively), photos taken with smartphones have orientation stored as an EXIF tag in the image’s metadata. However, Labelbox does not read this EXIF tag and will render the image in its original, unrotated state. You may upload photos from a smartphone if you use an external image editor to save your image in the correct orientation first.

Accessibility FAQ

Does Labelbox work in all countries?

Yes, unless your network has a restrictive firewall or you have internet connectivity issues. We currently only support the English language on our platform.

Do I need an internet connection to use Labelbox?

You must have an internet connection to use Labelbox. We recommend Google Chrome as a web browser. Due to known compatibility issues, we do not recommend Internet Explorer or Safari.

What if I lose internet connection while I am labeling? Will I lose all of my work?

Your work will be saved as long as you have an internet connection. In the event that you lose internet connection, a message will appear on your screen notifying you that your internet connection is compromised. If this happens, make sure your connection is restored before you continue working to ensure that your work is saved.

Can I use Labelbox on a mobile device or iPad?

Labelbox is not supported on mobile browsers so using Labelbox on a mobile device or iPad will not work.

Privacy & security FAQ

How is data stored at rest?

Data is encrypted at rest using AES256.

How is data secured when outsourcing to the workforce?

The outsourced labeling team connects directly to a Labelbox project to label and review, but cannot access the data (including training data) directly. In addition, the workforce providers are Labelbox consultants bound by strict confidentiality provisions regarding your data.

What is the security practice of the labeling workforce?

Our Labelbox annotation services delivery center is SOC2 Type1 certified. All the work is done in a supervised environment by annotators who are employed by Labelbox’s labeling services consulting company. We don’t crowdsource and don’t work with any freelancers.

In order to keep up with customer needs during the COVID-19 quarantine, some of our workforce teams have been allowed to work remotely. However, part of our workforce is still working from its premises.