Images

The image editor lets you label images with various supported annotation types, attachments, and editor settings.

​

Set up image annotation projects

To set up an image annotation project:

1. Create an image dataset and load the images you want to label. Alternatively, use the SDK to import images.
2. On the Annotate projects page, click the + New project button.
3. Select Image. Add a Name and an optional Description for your project.
4. Click Save. The system then creates the project and redirects you to the project overview page.
5. Click Add data. Then select your image dataset. Click Sample to sample your dataset, or you can manually select data rows and click Queue batch.

​

Set up an ontology

After setting up an audio annotation project, you can add an ontology based on how you want to label the data. The image editor supports the following annotation types that you can include in your ontology:

Feature	Import annotation	Export annotation
Bounding box	See payload	See payload
Segmentation mask	See payload	See payload
Polygon	See payload	See payload
Polyline	See payload	See payload
Point	See payload	See payload
Relationship	See payload	See payload
Radio classification	See payload	See payload
Checklist classification	See payload	See payload
Free-form text classification	See payload	See payload

​

Audio-to-text for text classification

You can enable free-text classification to allow labelers to record audio instead of typing text manually. The system transcribes the recorded audio into text using the integrated OpenAI Whisper model. To set up and use audio-to-text for text classification:

1. When creating the ontology, add a Text classification feature and enable Record audio for transcription.
2. In the editor, select the text classification annotation and click Record audio. Each recording can last up to 30 minutes and automatically stops when it reaches the limit.
3. Click Stop recording when you finish speaking. The system transcribes the audio into text and adds the recording as an attachment to the annotation.
4. Edit the transcribed text, remove the attached audio file, or click Record audio to re-record if needed.
5. Click Submit to save the annotation.

​

Bounding box

Create a bounding box by starting at one corner and dragging your cursor to create the shape around an object in the image. You can also click and drag to reposition the bounding box on the image.

​

Cuboid

Create a cuboid by starting at one corner and dragging your cursor to create a box shape around an object in the image. Once you release it, it will automatically become a cuboid. You can use the various levers on the tool to adjust its rotation along the x, y, and z axes. Along the center of the top bar, you will find buttons to switch to Rotate mode,Move mode, or Scale mode. You can also directly input the rotation into the modal that appears in the ontology pane once the cuboid is initially drawn or selected for editing. The fields available to edit are:

• Scale: This controls the cuboid size in each dimension. The unit is in pixels.
• Rotate: This controls the rotation of the cuboid along the three axes. For more details on the rotation, please refer to the diagram below.

​

Segmentation mask

The segmentation mask annotation supports different tools based on the ontology creation time.

​

AutoSegment 2.0

Ontologies created after June 13, 2023, by default, use a new AutoSegment 2.0 tool powered by Meta’s Segment Anything model. You can use the AutoSegment 2.0 tool to generate mask predictions for individual objects in your image. The Auto Segment 2.0 tool is designed for instance segmentation, meaning you can quickly label individual objects as it predicts one class at a time. Adjusting the contrast and brightness of the image does not affect the accuracy of the SAM model. When selecting a segmentation tool, choose the AutoSegment option from the top navigation bar. There are two modes available:

• Box mode: allows you to draw a box around an object to generate a mask on it.

• Hover and Click mode: allows you to visualize masks as you move the cursor around, click to generate, and further modify by adding positive and negative points.

​

AutoSegment (Legacy)

Only ontologies created before June 13, 2023 use this is the older version of AutoSegment. When a segmentation tool is selected, choose the AutoSegment option from the top navigation bar, and draw a box around the object you want to label. For best results, draw the boundary of the box close to the object edges. You can also adjust the brightness and contrast settings in the Adjustments menu in the top navigation bar to further improve results. Creating more contrast between the object you are labeling and its background will help the model detect the object more accurately.

​

Brush tool

Ontologies created after June 13, 2023 can access the brush tool. Use the brush tool to draw a freehand mask as though you are painting the canvas. You can choose from a circle or square-shaped brush and adjust the size in pixels. Click the brush icon with the minus sign to use the eraser tool. It behaves similarly but removes mask pixels as you move the mouse.

​

Pen tool

Use the pen tool to outline the item in the image. Hold the cursor down to draw freehand or let the cursor go to draw straight lines between points. The pen tool is only available when creating segmentation annotations. Click the pen icon with the plus sign to use the pen tool.

​

Erase tool

You can use the erase tool to clean up the edges of a segmentation mask. Click the pen icon with the minus sign to use the erase tool.

​

Fill tool

You can use the fill tool to easily label backgrounds or, otherwise, assign a segmentation annotation to all other pixels in the image that have not already been labeled. The fill tool is only enabled for segmentation annotations. Click the droplet icon to use the fill tool.

​

Polygon

Create a polygon annotation by clicking to create each point in the shape. Click the first point to close the polygon.

​

Polyline

Use the polyline tool to label lines in an image. Click on the last point to complete the shape.

​

Point

Use the point tool to label precise locations on the image.

​

Relationship

To create a relationship between annotations, select a relationship tool and hover over the annotation where you want the relationship to start to reveal the annotation’s anchor points. Click an anchor point to create the starting point of the relationship, then bring your mouse over to the annotation you want to relate it to, hovering over it to reveal its anchor points. Finally, click one of the anchor points to complete the relationship. Right-click a relationship to change its direction, make it bi-directional, or delete it from the asset.

​

Editor settings

The image editor has the following settings for object-type annotations:

• Overlay object titles: Displays the names of annotation objects on the image.
• Show annotations: Displays created annotation objects on the image.
• Enable annotations occlusion: Allows annotation objects to overlap based on z-order (front objects occlude those behind them).
• Polygon snapping: Allows points on lines, polygons, and bounding boxes to align or attach to polygon edges.

​

Setting object positions

When the Enable annotations occlusion setting is active, you can adjust the z-order of annotation objects to control how they stack visually. The OBJECTS panel provides two modes for managing object positions:

• Universal arrangement: Adjust the stacking order incrementally by selecting and repositioning individual objects.

• List-based arrangement: Use a drag-and-drop interface to reorder objects directly in a list.