Videos

Guide for labeling video data.

Overview

When you attach a video to a project, the Labelbox Editor will automatically adjust the Editor interface for video labeling.

For more information on importing videos to Labelbox, see our docs on Video import format.

This is what the Editor will look like when you are labeling video data.This is what the Editor will look like when you are labeling video data.

This is what the Editor will look like when you are labeling video data.

Video editor components

Editor component

Description

Tools menu

Contains the list of annotation classes (AKA ontology) for your project.

Annotations menu

Lists all of the annotations that have been created so far.

Progress bar

A visual indicator of the temporal progression of the frame sequence. You can hover over the progress bar to quickly preview any frame in the video.

Classification panel

Appears only for frame classification-type annotations (e.g., Radio). Here you can apply select answer(s) on a classification question and toggle the classification on/off.

Timeline

Displays a temporal visualization of all created annotations on the video clip. Indicates the frames on which the annotation was created. You can hover over the timeline to preview any frame in the video.

Keyframe

A keyframe is a frame in the sequence where an annotation starts, ends, or is modified in some way. A keyframe is indicated on an annotation track as a small circle. The frames between two keyframes are called interpolated (tweened) frames. This is because Labelbox uses interpolation to incrementally adjust the size and position of an annotation on frames in between keyframes. Note: interpolation is not applied to classifications.

Volume

Sometimes, to make a certain classification in a video, you need to be able to hear and see the content in the frame sequence. When a video has audio, your labeling team will be able to hear the audio during video playback (but not when hovering or stepping through with arrow keys). Labelers can adjust the volume level and click the volume button to mute/unmute audio playback. You can use the Video editor to classify videos with both audio and visual content or audio content alone. To use the Video editor to label audio only, please reach out to Support for details.

Navigate the video

Use the progress bar to navigate the video in its entirety, or use the timeline to navigate the current window of the video. The width of the current window can be adjusted from the player's settings to the right of the timestamp.

You can scrub a preview of any part of the video by hovering over a certain part of the progress bar or timeline. Clicking within the progress bar or timeline will bring the playhead to the clicked position.

Press play/pause or use Shift + Space to control playback of the video. The Left and Right arrow keys can be used to move backwards one frame and forwards one frame, respectively.

To navigate to a specific frame click on the current frame number, replace it with the frame number you want to navigate to, and press Enter.

Hover over the top edge of the timeline or annotations menu, and click and drag to change the height of the timeline.

Supported annotation types

Below are the annotation types you may include in your ontology when you are labeling video data.

Annotation type

Import

Export

Bounding box

See reference

See reference

Polyline

Coming soon

See reference

Point

Coming soon

See reference

Segmentation mask (beta)
(Pro, Enterprise, and Education licenses only)

Coming soon

See reference

Radio classification

See reference (frames only)

See reference

Checklist classification

See reference (frames only)

See reference

📘

Scope of classifications

Classifications in the video editor can only be applied at the global or frame level (nesting not yet supported).

Bounding box

To create a bounding box, use your cursor to place the top left and bottom right corners of the shape. By default, the shape will be created in the current frame and all frames following. You can use the toggle Annotations menu (or the hotkey T) to adjust the length of the object instance.

To reposition the bounding box, simply click + hold then use your mouse or trackpad to reposition the annotation on the frame. You can also resize the bounding box by clicking on the corners and dragging them to their new position. Whenever an object is created or edited on a video frame, a keyframe is created, and all frames between keyframes will interpolate. For a refresher on interpolation, see here.

Shortcut: In the Tools panel, you will see a numerical hotkey next to the name of the annotation. Use the specified number hotkey (e.g., 1, 2, 3) in the Tools panel to activate the bounding box tool.

To create another instance of the bounding box, press the number hotkey again to activate the tool, then create another bounding box. Once all instances have been created, press E to submit your label.

Polyline

To create a polyline, use your cursor to place each vertex of the line and click the last point again (or press F to end the shape. By default, the shape will be created in the current frame and all frames following. You can use the toggle Annotations menu (or the hotkey T) to adjust the length of the object instance.

To reposition the polyline, simply click + drag each vertex to reposition it on the frame. You can also click and drag the polyline to change its position. Whenever an object is created or edited on a video frame, a keyframe is created, and all frames between keyframes will interpolate. For a refresher on interpolation, see here.

Shortcut: In the Tools panel, you will see a numerical hotkey next to the name of the annotation. Use the specified number hotkey (e.g., 1, 2, 3) in the Tools panel to activate the polyline tool.

To create another instance of the polyline, press the number hotkey again to activate the tool, then create another polyline. Once all instances have been created, press E to submit your label.

Point

To create a point, put your cursor where you want to place the point and click and release to create it. By default, the shape will be created in the current frame and all frames following. You can use the toggle Annotations menu (or the hotkey T) to adjust the length of the object instance.

To reposition the point, simply click + drag it to reposition it on the frame. Whenever an object is created or edited on a video frame, a keyframe is created, and all frames between keyframes will interpolate. For a refresher on interpolation, see here.

Shortcut: In the Tools panel, you will see a numerical hotkey next to the name of the annotation. Use the specified number hotkey (e.g., 1, 2, 3) in the Tools panel to activate the point tool.

To create another instance of the point, press the number hotkey again to activate the tool, then create another point. Once all instances have been created, press E to submit your label.

Segmentation mask

To create a segmentation mask, click and drag to free-hand draw a shape. You can also click and release to place vertices with straight edges in between. Unlike other shapes, segmentation masks can only be created on individual frames. Multiple masks may be created on the same frame.

To edit a segmentation mask, you can use the add/ remove tools in the top navigation bar to add to and remove from a segmentation mask. Segmentation masks cannot be repositioned.

Shortcut: In the Tools panel, you will see a numerical hotkey next to the name of the annotation. Use the specified number hotkey (e.g., 1, 2, 3) in the Tools panel to activate the segmentation tool.

To create another instance of the mask, press the number hotkey again to activate the tool, then create another mask. Once all instances have been created, press E to submit your label.

❗️

Limited view during playback

Please note that segmentation masks currently will not display during video playback due to performance constraints. Please use the arrow keys or navigate to a frame directly to review a mask.

Frame classification

Create a frame classification by activating the classification question and inputting the answer value(s). Only radio and checklist classification types are supported. Once all classifications have been completed, press E to submit your label.

Shortcut: In the Tools panel, you will see a numerical hotkey next to the name of the annotation. Use the specified number hotkey (e.g., 1, 2, 3) in the Tools panel to activate the classification tool.

Global classification

Create a global classification by activating the classification question and inputting the answer value(s). Global classifications are values that apply to the entire video, rather than at the frame level, so they do not show in the timeline. Only radio and checklist classification types are supported. Once all classifications have been completed, press E to submit your label.

Shortcut: In the Tools panel, you will see a numerical hotkey next to the name of the annotation. Use the specified number hotkey (e.g., 1, 2, 3) in the Tools panel to activate the global classification tool.

Video specific hotkeys

Function

Hotkey

Description

Play/Pause

Shift + Space

Play or pause the video playback.

Move backward one frame

Left

Move back one frame in the sequence.

Move forward one frame

Right

Move forward one frame in the sequence.

Add/remove a keyframe

K

Add/remove a keyframe for the selected annotation on the selected frame.

Add/remove an object

T

Toggle the presence of an object on or off in the video timeline.

Select multiple contiguous frames

Shift + Select

Select multiple, contiguous frames in the video.

Select multiple discontiguous frames

Shift + Cmd + Select

Select multiple, discontiguous frames in the video.

Advance to the previous keyframe

Opt + Left

Jump to the previous keyframe on an annotation instance track.

Advance to the next keyframe

Opt + Right

Jump to the next keyframe on an annotation instance track.


Did this page help you?