Labelbox documentation

FAQ

General FAQ

What is Labelbox?

Labelbox is the industry-leading labeling and training data management platform.

Does Labelbox only sell labeling services?

In addition to labeling services, Labelbox offers:

  • A user-friendly, customizable platform for labeling images, video, text, and geospatial data.

  • Extensive collaboration management tools.

  • Quality and performance management tools.

How do I set up a labeling project?

Once you have created a Labelbox account, see Create a project.

Do I need to be a data scientist to use Labelbox?

You do not need to be a data scientist to label data in the Labelbox application. We designed our user-friendly platform to require minimal technical knowledge and expertise such that anyone can annotate data using our out-of-the-box application. However, it may be beneficial to have someone experienced in data science on your team to help ensure that your machine learning project is successfully developed and deployed.

What kind of data can I label with Labelbox?

With our out-of-the-box labeling application, you can label the following:

  • Images

  • Videos

  • Text

  • Geospatial data

To see a list of supported file formats, see our Data model reference.

How do people use the labels they create in Labelbox?

In most cases, Labelbox is used by individuals, organizations, and companies in need of high-quality training data to develop and deploy robust machine learning models. People choose Labelbox to create training data for various applications, ranging from independent, small-scale projects to enterprise-scale applications that leverage artificial intelligence to gain a competitive edge.

What is the difference between the new Editor, the Tiled Imagery Editor, and the Custom Editor?

When you create a project, you have the option to choose from the following labeling interfaces:

  • Editor — also known as the “Image editor”, “Pictor”, or the "Segmentation editor" — is the most recent Labelbox labeling interface released at the end of 2019. This latest version provides the most flexibility for creating semantic segmentation and vector-based annotations. It also supports bulk importing predictions as well as more robust quality assurance tools.

  • Tiled Imagery Editor — also referred to as the “Legacy editor”, "v4", or the “object detection editor” — is the previous version of the label editor. This is the labeling interface to select if you need to label geospatial data.

  • Custom Editor allows you to create a completely customized template with just HTML and JavaScript. To get started, check out our out-of-the-box custom label editor templates. See our docs on building a custom interface to learn more.

Billing and pricing FAQ

How much does Labelbox cost?

For organizations building expert artificial intelligence systems, we offer Pro & Enterprise licenses. For smaller projects, we offer a Developer license which is free for evaluation, individuals, and small projects with a limit of 2500 labels per year. Check out labelbox.com/pricing for more information on the Developer, Pro, and Enterprise license options.

Does Labelbox have a discount for nonprofit organizations or academic users?

If you will be using Labelbox for educational or non-commercial purposes, you may be eligible for an Education license. To apply, visit labelbox.com/education or contact us at support@labelbox.com.

How much does the Labelbox Workforce cost?

The cost of a labeling workforce familiar with the Labelbox application is included with all Enterprise contracts. Please visit this page or reach out to support@labelbox.com for more information.

Account management FAQ

What happens to my labeled data if I migrate from a paid plan to the Developer Tier?

Labelbox will not delete any data from your account. If you have already submitted 2,500 labels in the past 12 months, you will no longer be able to export labels.

If my account has more than 5 users, will any users be deactivated when I migrate from a paid account to the Developer Tier?

No users will be deactivated, but you will not be able to add additional users.

Labeling workforce FAQ

Can I outsource a team to label my data?

If you would like to outsource a workforce to label your data in Labelbox, you have two options:

  • Use our Labelbox workforce when you sign up for a Pro or Enterprise account.

  • Find a third party workforce and teach them how to use Labelbox.

Can I still use Labelbox if I already have a labeling team?

Yes. Whether your labeling team is outsourced, internal, or both, Labelbox allows your labelers to annotate data while giving you the tools to measure their label quality and performance. If needed, you have the ability to combine the efforts of your internal team and an outsourced labeling workforce on the same projects.

Support FAQ

Does Labelbox provide support?

Labelbox provides support to all Labelbox users. We have a team of dedicated machine learning support engineers based in San Francisco from 9 am - 5 pm PST. However, you are welcome to send us a support request outside of these hours and one of our support engineers will respond at their earliest availability. Enterprise accounts will receive a dedicated account manager for additional support as included in the contract. The best way to reach our support team is via the chat feature at the bottom right corner of the web page. You can also email us at support@labelbox.com.

Quality assurance FAQ

What is the difference between Consensus and Benchmark?

The Consensus tool allows you to compare the label of one member against the labels of other members to automatically calculate the overall consensus. You can configure the percentage of your dataset in a project that will be selected to be labeled more than once by distinct members. Because this calculation compares a labeler's work to that of other labelers, you can see which labelers are producing labels that tend not to agree with the majority. You can also track the consensus of your labeled dataset against the entire labeling team to ensure high label quality. Poor consensus may also be an indicator of a need to clarify the ontology or classification questions in order to boost the consensus scores.

The Benchmarks tool allows you to set a “gold standard” annotation and compare your labelers' annotations against it. The benchmark tool works by interspersing data to be labeled, for which there is a benchmark label, to each person labeling. The accuracy score for each label is calculated based on its comparison to its respective benchmark.

Will other labelers see the data I am labeling?

If Consensus is turned on and configured, yes. Otherwise, no.

Can I compare model-assisted predictions against human-made labels?

With Labelbox you can connect your machine learning model and compare your model’s performance against your team of labelers. To learn how to import model predictions to your project, see our Guide to importing annotations.

Technical FAQ

Does Labelbox have an API?

Yes. All of the functionality offered in our application interface can also be programmatically configured with our GraphQL API. Access to the Labelbox API is included in the Pro and Enterprise tiers.

Labelbox also offers a Python SDK for those who wish to use Labelbox programmatically using Python instead of by writing GraphQL queries and mutations.

How do I create an API key?

API Keys can only be created by organization administrators. To create an API key in the Labelbox UI, go to app.labelbox.com/account/api-keys.

Importing & exporting data FAQ

I already have training data. Can I import it into Labelbox?

Whether you are uploading model-assisted predictions or human-labeled data, you can import your labels via the app or programmatically. To learn more, see our docs for Model-assisted labeling.Model-assisted labeling

Which label export formats does Labelbox support?

Your labeled data can be exported in JSON or CSV format, which should allow you to import it into any standard machine learning framework. Labelbox does not offer any native export functionality for COCO or PASCAL VOC.

Does Labelbox work with on-prem data?

Labelbox works with source data hosted on-premises or on a private cloud. For on-prem configurations, the source data is accessed directly from the client computer and is never shared by (or accessible to) Labelbox. To learn more about our cloud integrations and on-prem configurations, see our docs on Importing data.Import data

Does Labelbox work with cloud data such as AWS or Google Cloud Storage?

Yes. To learn how to connect your cloud data to Labelbox, see our docs on Importing data.Import data

Can I import images from my smartphone?

Notably (but not exclusively), photos taken with smartphones have orientation stored as an EXIF tag in the image’s metadata. However, Labelbox does not read this EXIF tag and will render the image in its original, unrotated state. You may upload photos from a smartphone if you use an external image editor to save your image in the correct orientation.

Accessibility FAQ

Does Labelbox work in all countries?

Yes, unless your network has a restrictive firewall or you have internet connectivity issues. We currently only support the English language on our platform.

Do I need an internet connection to use Labelbox?

You must have an internet connection to use Labelbox. We recommend Google Chrome as a web browser. Due to known compatibility issues, we do not recommend Internet Explorer or Safari.

What if I lose internet connection while I am labeling? Will I lose all of the work that I've completed?

Your work will be saved as long as you have an internet connection. In the event that you lose internet connection, a message will appear on your screen notifying you that your internet connection is compromised. If this happens, make sure your connection is restored before you continue working to ensure that your work is saved.

Can I use Labelbox on a mobile device or iPad?

Labelbox is not supported on mobile browsers so using Labelbox on a mobile device or iPad will not work.

Privacy & security FAQ

How is data stored at rest?

Data is encrypted at rest using AES256.

How is data secured when outsourcing to the workforce?

The outsourced labeling team connects directly to a Labelbox project to label and review, but cannot access the data (including training data) directly. In addition, the workforce providers are Labelbox consultants bound by strict confidentiality provisions regarding your data.

What is the security practice of the labeling workforce?

Our Labelbox annotation services delivery center is SOC2 Type1 certified. All the work is done in a supervised environment by annotators who are employed by Labelbox’s labeling services consulting company. We don’t crowdsource and don’t work with any freelancers. We also don’t allow any remote logins from home.

In order to keep up with customer needs during the COVID-19 quarantine, some of our workforce teams have been allowed to work remotely. However, part of our workforce is still working from its premises.