Manual Annotation

• 1: Annotation modes
• 1.1: Single shape
• 1.2: Attribute annotation mode
• 1.3: 3D object annotation
• 1.4: Annotation with tags
• 2: Annotation with shapes
• 2.1: Annotation with rectangles
• 2.2: Shape mode
• 2.3: Annotation with polygons
• 2.3.1: Manual drawing
• 2.3.2: Drawing using automatic borders
• 2.3.3: Edit polygon
• 2.3.4: Track mode with polygons
• 2.3.5: Creating masks
• 2.4: Annotation with polylines
• 2.5: Track mode
• 2.6: Annotation with points
• 2.6.1: Points in shape mode
• 2.6.2: Linear interpolation with one point
• 2.7: Annotation with ellipses
• 2.8: Annotation with cuboids
• 2.8.1: Creating the cuboid
• 2.8.2: Editing the cuboid
• 2.9: Annotation with skeletons
• 2.10: Annotation with brush tool
• 2.11: Types of shapes
• 2.12: Shape grouping
• 3: Editing & Utility Tools
• 3.1: Join and slice tools
• 3.2: Shapes converter
• 3.3: Contextual images
• 3.4: Filter

1 - Annotation modes

CVAT provides several editor-level modes that change how the annotation workspace behaves. These modes control what actions are available to the user, which tools can be used, and how objects can be created or modified.

Use this section to understand when to switch modes and how each mode supports a specific step of the annotation workflow.

1.1 - Single shape

The CVAT Single Shape annotation mode accelerates the annotation process and enhances workflow efficiency for specific scenarios.

By using this mode you can label objects with a chosen annotation shape and label when an image contains only a single object. By eliminating the necessity to select tools from the sidebar and facilitating quicker navigation between images without the reliance on hotkeys, this feature makes the annotation process significantly faster.

See:

• Single Shape mode annotation interface
• Annotating in Single Shape mode
• Query parameters
• Video tutorial

Single Shape mode annotation interface

A set of controls in the interface of the Single Shape annotation mode may vary depending on different settings.

Images below displays the complete interface, featuring all available fields; as mentioned above, certain fields may be absent depending on the scenario.

For instance, when annotating with rectangles, the Number of points field will not appear, and if annotating a single class, the Labels selector will be omitted.

To access Single Shape mode, open the job, navigate to the top right corner, and from the drop-down menu, select Single Shape.

The interface will be different if the shape type was set to Any in the label Constructor:

The Single Shape annotation mode has the following fields:

Annotating in Single Shape mode

To annotate in Single Shape mode, follow these steps:

1. Open the job and switch to Single Shape mode.
2. Annotate the image based on the selected shape. For more information on shapes, see Annotation Tools.
3. (Optional) If the image does not contain any objects to annotate, click Skip at the top of the right panel.
4. Submit your work.

Query parameters

Also, we introduced additional query parameters, which you may append to the job link, to initialize the annotation process and automate workflow:

You can combine these parameters to customize the workspace for an annotator, for example:

/tasks/<tid>/jobs/<jid>?defaultWorkspace=single_shape&defaultLabel=dog&defaultPointsCount=10

Will open the following job:

Video tutorial

For a better understanding of how Single Shape mode operates, we recommend watching the following tutorial.

1.2 - Attribute annotation mode

• In this mode, you can edit attributes with fast navigation between objects and frames using a keyboard. Open the drop-down list in the top panel and select Attribute annotation.

  

• In this mode, objects panel change to a special panel:

  

• The active attribute will be red. In this case, it is gender. Look at the bottom side panel to see all possible shortcuts for changing the attribute. Press key 2 on your keyboard to assign a value (female) for the attribute or select from the drop-down list.

  

• Press Up Arrow/Down Arrow on your keyboard or select the buttons in the UI to go to the next/previous attribute. In this case, after pressing Down Arrow you will be able to edit the Age attribute.

  

• Use Right Arrow/Left Arrow keys to move to the previous/next image with annotation.

To display all the hot keys available in the attribute annotation mode, press F2.

It is possible to handle lots of objects on the same frame in the mode.

It is more convenient to annotate objects of the same type. In this case you can apply the appropriate filter. For example, the following filter will hide all objects except person: label=="Person".

To navigate between objects (person in this case), use the following buttons switch between objects in the frame on the special panel:

or shortcuts:

• Tab — go to the next object
• Shift+Tab — go to the previous object.

In order to change the zoom level, go to settings (press F3) in the workspace tab and set the value Attribute annotation mode (AAM) zoom margin in px.

1.3 - 3D object annotation

Use the 3D Annotation tool for labeling 3D objects and scenes, such as vehicles, buildings, landscapes, and others.

Check out:

• Navigation
• Annotation with cuboids
  • Annotation with shapes
  • Tracking with cuboids

Navigation

The 3D annotation canvas looks like the following:

For information on the available tools, consult Controls sidebar.

You can navigate, using the mouse, or navigation keys:

You can also use keyboard shortcuts to navigate:

Annotation with cuboids

There are two options available for 3D annotation:

• Shape: for tasks like object detection.
• Track: uses interpolation to predict the position of objects in subsequent frames. A unique ID will be assigned to each object and maintained throughout the sequence of images.

Annotation with shapes

To add a 3D shape:

1. On the objects pane, select Draw new cuboid > select the label from the drop-down list > Shape.

   

2. The cursor will be followed by a cuboid. Place the cuboid on the 3D scene.

   

3. Use projections to adjust the cuboid. Click and hold the left mouse button to edit the label shape on the projection.

   

4. (Optional) Move one of the four points to change the size of the cuboid.

   

5. (Optional) To rotate the cuboid, select the middle point and then drag the cuboid up/down or to left/right.

   

Tracking with cuboids

To track with cuboids:

1. On the objects pane, select Draw new cuboid > select the label from the drop-down list > Track.

2. The cursor will be followed by a cuboid. Place the cuboid on the 3D scene.

3. Use projections to adjust the cuboid. Select and hold the left mouse button to edit the label shape on the projection.

   

4. (Optional) Move one of the four points to change the size of the cuboid.

   

5. (Optional) To rotate the cuboid, click on the middle point and then drag the cuboid up/down or to left/right.

   

6. Move several frames forward. You will see the cuboid you’ve added in frame 1. Adjust it, if needed.

7. Repeat to the last frame with the presence of the object you are tracking.

For more information about tracking, consult Track mode.

As well as 2D-task objects, 3D-task objects support the ability to change appearance, attributes, properties and have an action menu. Read more in objects sidebar section.

Moving an object

If you hover the cursor over a cuboid and press Shift+N, the cuboid will be cut, so you can paste it in other place (double-click to paste the cuboid).

Copying

As well as in 2D task you can copy and paste objects by Ctrl+C and Ctrl+V, but unlike 2D tasks you have to place a copied object in a 3D space (double click to paste).

Image of the projection window

You can copy or save the projection-window image by left-clicking on it and selecting a “save image as” or “copy image”.

Cuboid orientation

The feature enables or disables the display of cuboid orientation arrows in the 3D space. It is controlled by a checkbox located in the appearance block. When enabled, arrows representing the cuboid’s axis orientation (X - red, Y - green, Z - blue) are displayed, providing a visual reference for the cuboid’s alignment within the 3D environment. This feature is useful for understanding the spatial orientation of the cuboid.

Cuboid size input

The size input feature allows users to manually specify the dimensions of a cuboid in the 3D space. This feature is accessible through the objects sidebar - details panel, where you can input precise values for the width, height, and length (X - width, Y - height, Z - length) of the cuboid. By entering these values, the cuboid’s size is adjusted accordingly to its orientation, providing greater control and accuracy when annotating objects in 3D tasks.

1.4 - Annotation with tags

It is used to annotate frames, tags are not displayed in the workspace. Before you start, open the drop-down list in the top panel and select Tag annotation.

The objects sidebar will be replaced with a special panel for working with tags. Here you can select a label for a tag and add it by clicking on the Plus button. You can also customize hotkeys for each label.

If you need to use only one label for one frame, then enable the Automatically go to the next frame checkbox, then after you add the tag the frame will automatically switch to the next.

Tags will be shown in the top left corner of the canvas. You can show/hide them in the settings.

2 - Annotation with shapes

This section describes all shapes you can create in CVAT. Each shape type supports its own tools, editing options, and use cases.

Review these pages to choose the correct shape tool for your annotation task.

2.1 - Annotation with rectangles

To learn more about annotation using a rectangle, see the sections:

• Shape mode (basics)
• Track mode (basics)

Rotation rectangle

To rotate the rectangle, pull on the rotation point. Rotation is done around the center of the rectangle. To rotate at a fixed angle (multiple of 15 degrees), hold shift. In the process of rotation, you can see the angle of rotation.

Annotation with rectangle by 4 points

It is an efficient method of bounding box annotation, proposed here. Before starting, you need to make sure that the drawing method by 4 points is selected.

Press Shape or Track for entering drawing mode. Click on four extreme points: the top, bottom, left- and right-most physical points on the object. Drawing will be automatically completed right after clicking the fourth point. Press Esc to cancel editing.

2.2 - Shape mode

Usage examples:

• Create new annotations for a set of images.
• Add/modify/delete objects for existing annotations.

1. You need to select Rectangle on the controls sidebar:

   

   Before you start, select the correct Label (should be specified by you when creating the task) and Drawing Method (by 2 points or by 4 points):

   

2. Creating a new annotation in Shape mode:

   • Create a separate Rectangle by selecting Shape.

     

   • Choose the opposite points. Your first rectangle is ready!

     

   • To learn more about creating a rectangle read here.

   • It is possible to adjust boundaries and location of the rectangle using a mouse. The rectangle’s size is shown in the top right corner, you can check it by selecting any point of the shape. You can also undo your actions using Ctrl+Z and redo them with Shift+Ctrl+Z or Ctrl+Y.

3. You can see the Object card in the objects sidebar or open it by right-clicking on the object. You can change the attributes in the details section. You can perform basic operations or delete an object by selecting on the action menu button.

   

4. The following figure is an example of a fully annotated frame with separate shapes.

   

Occluded Occlusion is an attribute used if an object is occluded by another object or isn’t fully visible on the frame. Use Q shortcut to set the property quickly.

Example: the three cars on the figure below should be labeled as occluded.

If a frame contains too many objects and it is difficult to annotate them due to many shapes placed mostly in the same place, it makes sense to lock them. Shapes for locked objects are transparent, and it is easy to annotate new objects. Besides, you can’t change previously annotated objects by accident. Shortcut: L.

2.3 - Annotation with polygons

2.3.1 - Manual drawing

It is used for semantic / instance segmentation.

Before starting, you need to select Polygon on the controls sidebar and choose the correct Label.

• Click Shape to enter drawing mode. There are two ways to draw a polygon: either create points by clicking or by dragging the mouse on the screen while holding Shift.

Clicking points	Holding Shift+Dragging

• When Shift isn’t pressed, you can zoom in/out (when scrolling the mouse wheel) and move (when clicking the mouse wheel and moving the mouse), you can also delete the previous point by right-clicking on it.
• You can use the Selected opacity slider in the Objects sidebar to change the opacity of the polygon. You can read more in the Objects sidebar section.
• Press N again or click the Done button on the top panel for completing the shape.
• After creating the polygon, you can move the points or delete them by right-clicking and selecting Delete point or clicking with pressed Alt key in the context menu.

2.3.2 - Drawing using automatic borders

You can use auto borders when drawing a polygon. Using automatic borders allows you to automatically trace the outline of polygons existing in the annotation.

• To do this, go to settings -> workspace tab and enable Automatic Bordering or press Ctrl while drawing a polygon.

  

• Start drawing / editing a polygon.

• Points of other shapes will be highlighted, which means that the polygon can be attached to them.

• Define the part of the polygon path that you want to repeat.

  

• Click on the first point of the contour part.

  

• Then click on any point located on part of the path. The selected point will be highlighted in purple.

  

• Click on the last point and the outline to this point will be built automatically.

  

Besides, you can set a fixed number of points in the Number of points field, then drawing will be stopped automatically. To enable dragging you should right-click inside the polygon and choose Switch pinned property.

Below you can see results with opacity and black stroke:

If you need to annotate small objects, increase Image Quality to 95 in Create task dialog for your convenience.

2.3.3 - Edit polygon

To edit a polygon you have to click on it while holding Shift, it will open the polygon editor.

• In the editor you can create new points or delete part of a polygon by closing the line on another point.

• When Intelligent polygon cropping option is activated in the settings, CVAT considers two criteria to decide which part of a polygon should be cut off during automatic editing.

  • The first criteria is a number of cut points.
  • The second criteria is a length of a cut curve.

  If both criteria recommend to cut the same part, algorithm works automatically, and if not, a user has to make the decision. If you want to choose manually which part of a polygon should be cut off, disable Intelligent polygon cropping in the settings. In this case after closing the polygon, you can select the part of the polygon you want to leave.

  

• You can press Esc to cancel editing.

  

2.3.4 - Track mode with polygons

Polygons in the track mode allow you to mark moving objects more accurately other than using a rectangle.

1. To create a polygon in the track mode, click the Track button.

   

2. Create a polygon the same way as in the case of Annotation with polygons. Press N or click the Done button on the top panel to complete the polygon.

3. Pay attention to the fact that the created polygon has a starting point and a direction, these elements are important for annotation of the following frames.

4. After going a few frames forward press Shift+N, the old polygon will disappear and you can create a new polygon. The new starting point should match the starting point of the previously created polygon (in this example, the top of the left mirror). The direction must also match (in this example, clockwise). After creating the polygon, press N and the intermediate frames will be interpolated automatically.

   

5. If you need to change the starting point, right-click on the desired point and select Set starting point. To change the direction, right-click on the desired point and select switch orientation.

   

There is no need to redraw the polygon every time using Shift+N, instead you can simply move the points or edit a part of the polygon by pressing Shift+Click.

2.3.5 - Creating masks

Cutting holes in polygons

Currently, CVAT does not support cutting transparent holes in polygons. However, it is possible to generate holes in exported instance and class masks. To do this, one needs to define a background class in the task and draw holes with it as additional shapes above the shapes needed to have holes:

The editor window:

Remember to use z-axis ordering for shapes by [-] and [+, =] keys.

Exported masks:

Notice that it is currently impossible to have a single instance number for internal shapes (they will be merged into the largest one and then covered by “holes”).

Creating masks

There are several formats in CVAT that can be used to export masks:

• Segmentation Mask (PASCAL VOC masks)
• CamVid
• MOTS
• ICDAR
• COCO (RLE-encoded instance masks, guide)
• Datumaro

An example of exported masks (in the Segmentation Mask format):

Important notices:

• Both boxes and polygons are converted into masks
• Grouped objects are considered as a single instance and exported as a single mask (label and attributes are taken from the largest object in the group)

Class colors

All the labels have associated colors, which are used in the generated masks. These colors can be changed in the task label properties:

Label colors are also displayed in the annotation window on the right panel, where you can show or hide specific labels (only the presented labels are displayed):

A background class can be:

• A default class, which is implicitly-added, of black color (RGB 0, 0, 0)
• background class with any color (has a priority, name is case-insensitive)
• Any class of black color (RGB 0, 0, 0)

To change background color in generated masks (default is black), change background class color to the desired one.

2.4 - Annotation with polylines

It is used for road markup annotation etc.

Before starting, you need to select the Polyline. You can set a fixed number of points in the Number of points field, then drawing will be stopped automatically.

Click Shape to enter drawing mode. There are two ways to draw a polyline — you either create points by clicking or by dragging a mouse on the screen while holding Shift. When Shift isn’t pressed, you can zoom in/out (when scrolling the mouse wheel) and move (when clicking the mouse wheel and moving the mouse), you can delete previous points by right-clicking on it. Press N again or click the Done button on the top panel to complete the shape. You can delete a point by clicking on it with pressed Ctrl or right-clicking on a point and selecting Delete point. Click with pressed Shift will open a polyline editor. There you can create new points(by clicking or dragging) or delete part of a polygon closing the red line on another point. Press Esc to cancel editing.

2.5 - Track mode

Usage examples:

• Create new annotations for a sequence of frames.
• Add/modify/delete objects for existing annotations.
• Edit tracks, merge several rectangles into one track.

1. Like in the Shape mode, you need to select a Rectangle on the sidebar, in the appearing form, select the desired Label and the Drawing method.

   

2. Creating a track for an object (look at the selected car as an example):

   • Create a Rectangle in Track mode by selecting Track.

     

   • In Track mode, the rectangle will be automatically interpolated on the next frames.

   • The cyclist starts moving on frame #2270. Let’s mark the frame as a key frame. You can press K for that or select the star button (see the screenshot below).

     

   • If the object starts to change its position, you need to modify the rectangle where it happens. It isn’t necessary to change the rectangle on each frame, simply update several keyframes and the frames between them will be interpolated automatically.

   • Let’s jump 30 frames forward and adjust the boundaries of the object. See an example below:

     

   • After that the rectangle of the object will be changed automatically on frames 2270 to 2300:

     

3. When the annotated object disappears or becomes too small, you need to finish the track. You have to choose Outside Property, shortcut O.

   

4. If the object isn’t visible on a couple of frames and then appears again, you can use the Merge feature to merge several individual tracks into one.

   

   • Create tracks for moments when the cyclist is visible:

     

   • Select Merge button or press key M and select on any rectangle of the first track and on any rectangle of the second track and so on:

     

   • Select Merge button or press M to apply changes.

     

   • The final annotated sequence of frames in Interpolation mode can look like the clip below:

     

Shapes that were created in the track mode, have extra navigation buttons.

• These buttons help to jump to the previous/next keyframe.

  

• The button helps to jump to the initial frame and to the last keyframe.

  

You can use the Split function to split one track into two tracks:

2.6 - Annotation with points

2.6.1 - Points in shape mode

It is used for face, landmarks annotation etc.

Before you start you need to select the Points. If necessary you can set a fixed number of points in the Number of points field, then drawing will be stopped automatically.

Click Shape to entering the drawing mode. Now you can start annotation of the necessary area. Points are automatically grouped — all points will be considered linked between each start and finish. Press N again or click the Done button on the top panel to finish marking the area. You can delete a point by clicking with pressed Ctrl or right-clicking on a point and selecting Delete point. Clicking with pressed Shift will open the points shape editor. There you can add new points into an existing shape. You can zoom in/out (when scrolling the mouse wheel) and move (when clicking the mouse wheel and moving the mouse) while drawing. You can drag an object after it has been drawn and change the position of individual points after finishing an object.

2.6.2 - Linear interpolation with one point

You can use linear interpolation for points to annotate a moving object:

1. Before you start, select the Points.

2. Linear interpolation works only with one point, so you need to set Number of points to 1.

3. After that select the Track.

   

4. Click Track to enter the drawing mode left-click to create a point and after that shape will be automatically completed.

   

5. Move forward a few frames and move the point to the desired position, this way you will create a keyframe and intermediate frames will be drawn automatically. You can work with this object as with an interpolated track: you can hide it using the Outside, move around keyframes, etc.

   

6. This way you’ll get linear interpolation using the Points.

   

2.7 - Annotation with ellipses

It is used for road sign annotation etc.

First of all you need to select the ellipse on the controls sidebar.

Choose a Label and click Shape or Track to start drawing. An ellipse can be created the same way as a rectangle, you need to specify two opposite points, and the ellipse will be inscribed in an imaginary rectangle. Press N or click the Done button on the top panel to complete the shape.

You can rotate ellipses using a rotation point in the same way as rectangles.

Annotation with ellipses video tutorial

2.8 - Annotation with cuboids

It is used to annotate 3 dimensional objects such as cars, boxes, etc… Currently the feature supports one point perspective and has the constraint where the vertical edges are exactly parallel to the sides.

2.8.1 - Creating the cuboid

Before you start, you have to make sure that Cuboid is selected and choose a drawing method from rectangle or by 4 points.

Drawing cuboid by 4 points

Choose a drawing method by 4 points and select Shape to enter the drawing mode. There are many ways to draw a cuboid. You can draw the cuboid by placing 4 points, after that the drawing will be completed automatically. The first 3 points determine the plane of the cuboid while the last point determines the depth of that plane. For the first 3 points, it is recommended to only draw the 2 closest side faces, as well as the top and bottom face.

A few examples:

Drawing cuboid from rectangle

Choose a drawing method from rectangle and select Shape to enter the drawing mode. When you draw using the rectangle method, you must select the frontal plane of the object using the bounding box. The depth and perspective of the resulting cuboid can be edited.

Example:

2.8.2 - Editing the cuboid

The cuboid can be edited in multiple ways: by dragging points, by dragging certain faces or by dragging planes. First notice that there is a face that is painted with gray lines only, let us call it the front face.

You can move the cuboid by simply dragging the shape behind the front face. The cuboid can be extended by dragging on the point in the middle of the edges. The cuboid can also be extended up and down by dragging the point at the vertices.

To draw with perspective effects it should be assumed that the front face is the closest to the camera. To begin simply drag the points on the vertices that are not on the gray/front face while holding Shift. The cuboid can then be edited as usual.

If you wish to reset perspective effects, you may right click on the cuboid, and select Reset perspective to return to a regular cuboid.

The location of the gray face can be swapped with the adjacent visible side face. You can do it by right clicking on the cuboid and selecting Switch perspective orientation. Note that this will also reset the perspective effects.

Certain faces of the cuboid can also be edited, these faces are: the left, right and dorsal faces, relative to the gray face. Simply drag the faces to move them independently from the rest of the cuboid.

You can also use cuboids in track mode, similar to rectangles in track mode (basics) or

2.9 - Annotation with skeletons

In this guide, we delve into the efficient process of annotating complex structures through the implementation of Skeleton annotations.

Skeletons serve as annotation templates for annotating complex objects with a consistent structure, such as human pose estimation or facial landmarks.

A Skeleton is composed of numerous points (also referred to as elements), which may be connected by edges. Each point functions as an individual object, possessing unique attributes and properties like color, occlusion, and visibility.

Skeletons can be exported in two formats: CVAT for image and COCO Keypoints.

See:

• Adding Skeleton manually
  • Skeleton Configurator
  • Configuring Skeleton points
  • Adding Skeleton labels manually
• Adding Skeleton labels from the model
• Annotation with Skeletons
• Automatic annotation with Skeletons
• Editing skeletons on the canvas
• Editing skeletons on the sidebar

Adding Skeleton manually

To start annotating using skeletons, you need to set up a Skeleton task in Configurator:

To open Configurator, when creating a task, click on the Setup skeleton button if you want to set up the skeleton manually, or From model if you want to add skeleton labels from a model.

Skeleton Configurator

The skeleton Configurator is a tool to build skeletons for annotation. It has the following fields:

Configuring Skeleton points

You can name labels, set attributes, and change the color of each point of the skeleton.

To do this, right-click on the skeleton point and select Configure:

In the opened menu, you can change the point setting. It is similar to adding labels and attributes of the regular task:

A Skeleton point can only exist within its parent Skeleton.

Adding Skeleton labels manually

To create the Skeleton task, do the following:

1. Open Configurator.
2. (Optional) Upload background image.
3. In the Label name field, enter the name of the label.
4. (Optional) Add attribute
   Note: you can add attributes exclusively to each point, for more information, see Configuring Skeleton points
5. Use Add point to add points to the Drawing area.
6. Use Add edge to add edges between points.
7. Upload files.
8. Click:
   • Submit & Open to create and open the task.
   • Submit & Continue to submit the configuration and start creating a new task.

Adding Skeleton labels from the model

To add points from the model, and annotate do the following:

1. Open Basic configurator.

2. On the Constructor tab, click From model.

3. From the Select a model to pick labels select the Human pose estimation model or others if available.

4. Click on the model’s labels, you want to use.
   Selected labels will become gray.

   

5. (Optional) If you want to adjust labels, within the label, click the Update attributes icon.
   The Skeleton configurator will open, where you can configure the skeleton.
   Note: Labels cannot be adjusted after the task/project is created.

6. Click Done. The labels, that you selected, will appear in the labels window.

7. Upload data.

8. Click:

   • Submit & Open to create and open the task.
   • Submit & Continue to submit the configuration and start creating a new task.

Annotation with Skeletons

To annotate with Skeleton, do the following

1. Open job.

2. On the tools panel select Draw new skeleton.

3. Select Track to annotate with tracking or Shape to annotate without tracking.

   

4. Draw a skeleton on the image.

Automatic annotation with Skeletons

To automatically annotate with Skeleton, do the following

1. Open the job and on the tools panel select AI Tools > Detectors

2. From the drop-down list select the model. You will see a list of points to match and the name of the skeleton on the top of the list.

   

3. (Optional) By clicking on the Bin icon, you can remove any mapped item:

   • A skeleton together with all points.
   • Certain points from two mapped skeletons.

4. Click Annotate.

Editing skeletons on the canvas

A drawn skeleton is encompassed within a bounding box, it allows you to manipulate the skeleton as a regular bounding box, enabling actions such as dragging, resizing, or rotating:

Upon repositioning a point, the bounding box adjusts automatically, without affecting other points:

Additionally, Shortcuts are applicable to both the skeleton as a whole and its elements:

• To use a shortcut to the entire skeleton, hover over the bounding box and push the shortcut keyboard key. This action is applicable for shortcuts like the lock, occluded, pinned, keyframe, and outside for skeleton tracks.
• To use a shortcut to a specific skeleton point, hover over the point and push the shortcut keyboard key. The same list of shortcuts is available, with the addition of outside, which is also applicable to individual skeleton shape elements.

Editing skeletons on the sidebar

In CVAT, the sidebar offers an alternative method for setting up skeleton properties and attributes.

This approach is similar to that used for other object types supported by CVAT, but with a few specific alterations:

An additional collapsible section is provided for users to view a comprehensive list of skeleton parts.

Skeleton points can have properties like Outside, Occluded, and Hidden.

Both Outside and Hidden make a skeleton point invisible.

• Outside property is part of annotations. Use it when part of the object is out of frame borders.

• Hidden makes a point hidden only for the annotator’s convenience, this property will not be saved between different sessions.

• Occluded keeps the point visible on the frame and usually means that the point is still on a frame, just hidden behind another object.

2.10 - Annotation with brush tool

With a brush tool, you can create masks for disjoint objects, that have multiple parts, such as a house hiding behind trees, a car behind a pedestrian, or a pillar behind a traffic sign. The brush tool has several modes, for example: erase pixels, change brush shapes, and polygon-to-mask mode.

Use brush tool for Semantic (Panoptic) and Instance Image Segmentation tasks.
For more information about segmentation masks in CVAT, see Creating masks.

See:

• Brush tool menu
• Annotation with brush
• Annotation with polygon-to-mask
• Remove underlying pixels
• AI Tools
• Import and export

Brush tool menu

The brush tool menu appears on the top of the screen after you click Shape:

It has the following elements:

Element	Description
	Save mask saves the created mask. The saved mask will appear on the object sidebar
	Save mask and continue adds a new mask to the object sidebar and allows you to draw a new one immediately.
	Brush adds new mask/ new regions to the previously added mask).
	Eraser removes part of the mask.
	Polygon selection tool. Selection will become a mask.
	Remove polygon selection subtracts part of the polygon selection.
	Brush size in pixels. Note: Visible only when Brush or Eraser are selected.
	Brush shape with two options: circle and square. Note: Visible only when Brush or Eraser are selected.
	Remove underlying pixels. When you are drawing or editing a mask with this tool, pixels on other masks that are located at the same positions as the pixels of the current mask are deleted.
	Hide mask. When drawing or editing a mask, you can enable this feature to temporarily hide the mask, allowing you to see the objects underneath more clearly.
	Label that will be assigned to the newly created mask
	Move. Click and hold to move the menu bar to the other place on the screen

Annotation with brush

To annotate with brush, do the following:

1. From the controls sidebar, select Brush .

2. In the Draw new mask menu, select label for your mask, and click Shape.
   The Brush tool will be selected by default.

   

3. With the brush, draw a mask on the object you want to label.
   To erase selection, use Eraser

   

4. After you applied the mask, on the top menu bar click Save mask
   to finish the process (or N on the keyboard).

5. Added object will appear on the objects sidebar.

To add the next object, repeat steps 1 to 5. All added objects will be visible on the image and the objects sidebar.

To save the job with all added objects, on the top menu, click Save .

Annotation with polygon-to-mask

To annotate with polygon-to-mask, do the following:

1. From the controls sidebar, select Brush .

2. In the Draw new mask menu, select label for your mask, and click Shape.

   

3. In the brush tool menu, select Polygon .

4. With the Polygon tool, draw a mask for the object you want to label.
   To correct selection, use Remove polygon selection .

5. Use Save mask (or N on the keyboard)
   to switch between add/remove polygon tools:

   

6. After you added the polygon selection, on the top menu bar click Save mask
   to finish the process (or N on the keyboard).

7. Click Save mask again (or N on the keyboard).
   The added object will appear on the objects sidebar.

To add the next object, repeat steps 1 to 5.

All added objects will be visible on the image and the objects sidebar.

To save the job with all added objects, on the top menu, click Save .

Remove underlying pixels

Use Remove underlying pixels tool when you want to add a mask and simultaneously delete the pixels of
other masks that are located at the same positions. It is a highly useful feature to avoid meticulous drawing edges twice between two different objects.

AI Tools

You can convert AI tool masks to polygons. To do this, use the following AI tool menu:

1. Go to the Detectors tab.
2. Switch toggle Masks to polygons to the right.
3. Add source and destination labels from the drop-down lists.
4. Click Annotate.

Import and export

For export, see Export dataset

Import follows the general import dataset procedure, with the additional option of converting masks to polygons.

To use it, when uploading the dataset, switch the Convert masks to polygon toggle to the right:

2.11 - Types of shapes

There are several shapes with which you can annotate your images:

• Rectangle or Bounding box
• Polygon
• Polyline
• Points
• Ellipse
• Cuboid
• Cuboid in 3D task
• Skeleton
• Tag

And there is what they look like:

Tag - has no shape in the workspace, but is displayed in objects sidebar.

2.12 - Shape grouping

This feature allows us to group several shapes.

You may use the Group Shapes button or shortcuts:

• G — start selection / end selection in group mode
• Esc — close group mode
• Shift+G — reset group for selected shapes

You may select shapes clicking on them or selecting an area.

Grouped shapes will have group_id filed in dumped annotation.

Also you may switch color distribution from an instance (default) to a group. You have to switch Color By Group checkbox for that.

Shapes that don’t have group_id, will be highlighted in white.

Shapes grouping video tutorial

3 - Editing & Utility Tools

This section covers tools that help modify shapes, manage large annotations, and streamline the annotation workflow.

Use these guides to learn how to join or split shapes, group them, filter objects, and work with contextual images and frame operations.

3.1 - Join and slice tools

In CVAT you can modify shapes by either joining multiple shapes into a single label or slicing a single label into several shapes.

This document provides guidance on how to perform these operations effectively.

See:

• Joining masks
• Slicing polygons and masks

Joining masks

The Join masks tool (), is specifically designed to work with mask annotations.

This tool is useful in scenarios where a single object in an image is annotated with multiple shapes, and there is a need to merge these shapes into a single one.

To join masks, do the following:

1. From the Edit block, select Join masks .
2. Click on the canvas area, to select masks that you want to join.
3. (Optional) To remove the selection click the mask one more time.
4. Click again on Join masks (J) to execute the join operation.

Upon completion, the selected masks will be joined into a single mask.

Slicing polygons and masks

The Slice mask/polygon () is compatible with both mask and polygon annotations.

This tool is useful in scenarios where multiple objects in an image are annotated with one shape, and there is a need to slice this shape into multiple parts.

To slice mask or polygon, do the following:

1. From the Edit block, select Slice mask/polygon .
2. Click on the shape you intend to slice. A black contour will appear around the selected shape.
3. Set an initial point for slicing by clicking on the contour.
4. Draw a line across the shape to define the slicing path.
   Hold Shift to add points automatically on cursor movement.
   Note: The line cannot cross itself.
   Note: The line cannot cross the contour more than twice.
5. (Optional)> Right-click to cancel the latest point.
6. Click on the contour (Alt+J) (outside the contour) to finalize the slicing.

3.2 - Shapes converter

The shapes converter is a feature that enables bulk actions on filtered shapes. It allows you to perform mutual conversion between masks, polygons and rectangles.

See:

• Run actions menu
• Convert shapes
• Convert shapes video tutorial

Run actions menu

Annotations actions can be accessed from the annotation menu. To access it, click on the burger icon and then select Run actions.

You will see the following dialog:

With the following fields:

Convert shapes

Recommended Precautions Before Running Annotation Actions

• Saving changes: It is recommended to save all changes prior to initiating the annotation action. If unsaved changes are detected, a prompt will advise to save these changes to avoid any potential loss of data.

• Disable auto-save: Prior to running the annotation action, disabling the auto-save feature is advisable. A notification will suggest this action if auto-save is currently active.

• Committing changes: Changes applied during the annotation session will not be committed to the server until the saving process is manually initiated. This can be done either by the user or through the auto-save feature, should it be enabled.

To convert shapes, do the following:

1. Annotate your dataset.

   

2. Set up filters.

   

3. From the burger menu, select Run actions.

4. Choose the action you need from the Select action drop-down list.

5. (Optional) In the Starting from frame field, enter the frame number where the action should begin, and in the up to frame field, specify the frame number where the action should end.

6. (Optional) Select an option from Or choose one of the predefined options to apply the action.

7. Click Run.
   A progress bar will appear. You may abort the process by clicking Cancel until the process commits modified objects at the end of pipeline.

   

Convert shapes video tutorial

3.3 - Contextual images

Contextual images (or related images) are additional images that provide context or additional information related to the primary image.

Use them to add extra contextual about the object to improve the accuracy of annotation.

Contextual images are available for 2D and 3D tasks.

See:

• Folder structure
• Contextual images

Folder structure

To add contextual images to the task, you need to organize the images folder into one of the supported file layouts. A task with contextual images can be created both from an archive or from raw files.

Example for 2D tasks:

1. In the folder with the images for annotation, create a folder: related_images.
2. Add to the related_images a subfolder with the same name as the primary image to which it should be linked.
3. Place the contextual image(s) within the subfolder created in step 2.
4. Add folder to the archive.
5. Create task.

Supported file layouts for 2D and 3D tasks:

• 2D task
• 3D option 1
• 3D option 2
• 3D task KITTI format
• 3D task Supervisely format

root_directory/
  image_1_to_be_annotated.jpg
  image_2_to_be_annotated.jpg
  related_images/
    image_1_to_be_annotated_jpg/
      context_image_for_image_1.jpg
    image_2_to_be_annotated_jpg/
      context_image_for_image_2.jpg
    subdirectory_example/
      image_3_to_be_annotated.jpg
        related_images/
        image_3_to_be_annotated_jpg/
            context_image_for_image_3.jpg

root_directory/
  pointcloud1.pcd
  pointcloud1.jpg
  pointcloud2.pcd
  pointcloud2.png
  ...

root_directory/
  pointcloud1/
    pointcloud1.pcd
    pointcloud1_ri1.png
    pointcloud1_ri2.jpg
    ...
  pointcloud2/
    pointcloud2.pcd
    pointcloud2_ri1.bmp

image_00/
  data/
    0000000000.png
    0000000001.png
    0000000002.png
    0000000003.png
image_01/
  data/
    0000000000.png
    0000000001.png
    0000000002.png
    0000000003.png
image_N/
  data/
    0000000000.png
    0000000001.png
    0000000002.png
    0000000003.png
velodyne_points/
  data/
    0000000000.bin
    0000000001.bin
    0000000002.bin
    0000000003.bin

root_directory/
  pointcloud/
    pointcloud1.pcd
    pointcloud2.pcd
  related_images/
    pointcloud1_pcd/
      context_image_for_pointclud1.jpg
    pointcloud2_pcd/
      context_image_for_pointcloud2.jpg

For more general information about 3D data formats, see 3D data formats.

Contextual images

The maximum amount of contextual images is twelve.

By default they will be positioned on the right side of the main image.

When you add contextual images to the set, small toolbar will appear on the top of the screen, with the following elements:

Element	Description
	Fit views. Click to restore the layout to its original appearance. If you’ve expanded any images in the layout, they will returned to their original size. This won’t affect the number of context images on the screen.
	Add new image. Click to add context image to the layout.
	Reload layout. Click to reload layout to the default view. Note, that this action can change the number of context images resetting them back to three.

Each context image has the following elements:

3.4 - Filter

There are some reasons to use the feature:

1. When you use a filter, objects that don’t match the filter will be hidden.
2. The fast navigation between frames which have an object of interest. Use the Left Arrow / Right Arrow keys for this purpose or customize the UI buttons by right-clicking and select switching by filter. If there are no objects which correspond to the filter, you will go to the previous / next frame which contains any annotated objects.

To apply filters you need to click on the button on the top panel.

Create a filter

It will open a window for filter input. Here you will find two buttons: Add rule and Add group.

Rules

The Add rule button adds a rule for objects display. A rule may use the following properties:

Supported properties for annotation

• Supported properties for projects list
• Supported properties for tasks list
• Supported properties for jobs list
• Supported properties for cloud storages list

Supported operators for properties

== - Equally; != - Not equal; > - More; >= - More or equal; < - Less; <= - Less or equal;

Any in; Not in - these operators allow you to set multiple values in one rule;

Is empty; is not empty – these operators don’t require to input a value.

Between; Not between – these operators allow you to choose a range between two values.

Like - this operator indicate that the property must contain a value.

Starts with; Ends with - filter by beginning or end.

Some properties support two types of values that you can choose:

You can add multiple rules, to do so click the add rule button and set another rule. Once you’ve set a new rule, you’ll be able to choose which operator they will be connected by: And or Or.

All subsequent rules will be joined by the chosen operator. Click Submit to apply the filter or if you want multiple rules to be connected by different operators, use groups.

Groups

To add a group, click the Add group button. Inside the group you can create rules or groups.

If there is more than one rule in the group, they can be connected by And or Or operators. The rule group will work as well as a separate rule outside the group and will be joined by an operator outside the group. You can create groups within other groups, to do so you need to click the add group button within the group.

You can move rules and groups. To move the rule or group, drag it by the button. To remove the rule or group, click on the Delete button.

If you activate the Not button, objects that don’t match the group will be filtered out. Click Submit to apply the filter. The Cancel button undoes the filter. The Clear filter button removes the filter.

Once applied filter automatically appears in Recent used list. Maximum length of the list is 10.

Sort and filter lists

On the projects, task list on the project page, tasks, jobs, and cloud storage pages, you can use sorting and filters.

Sort by

You can sort by the following parameters:

• Jobs list: ID, assignee, updated date, stage, state, task ID, project ID, task name, project name.
• Tasks list or tasks list on project page: ID, owner, status, assignee, updated date, subset, mode, dimension, project ID, name, project name.
• Projects list: ID, assignee, owner, status, name, updated date.
• Cloud storages list: ID, provider type, updated date, display name, resource, credentials, owner, description.

To apply sorting, drag the parameter to the top area above the horizontal bar. The parameters below the horizontal line will not be applied. By moving the parameters you can change the priority, first of all sorting will occur according to the parameters that are above.

Pressing the Sort button switches Ascending sort/Descending sort.

Quick filters

Quick Filters contain several frequently used filters:

• Assigned to me - show only those projects, tasks or jobs that are assigned to you.
• Owned by me - show only those projects or tasks that are owned by you.
• Not completed - show only those projects, tasks or jobs that have a status other than completed.
• AWS storages - show only AWS cloud storages
• Azure storages - show only Azure cloud storages
• Google cloud storages - show only Google cloud storages

Date and time selection

When creating a Last updated rule, you can select the date and time by using the selection window.

You can select the year and month using the arrows or by clicking on the year and month. To select a day, click on it in the calendar, To select the time, you can select the hours and minutes using the scrolling list. Or you can select the current date and time by clicking the Now button. To apply, click Ok.