Documentation

• 1: Getting started
• 1.1: Product tour
• 1.1.1: CVAT intro
• 1.1.2: Introduction to CVAT and Datumaro
• 2: Integrations
• 2.1: FiftyOne
• 3: Frequently asked questions
• 4: Manual
• 4.1: Basics
• 4.1.1: Registration
• 4.1.2: Creating an annotation task
• 4.1.3: Creating multi tasks
• 4.1.4: Jobs page
• 4.1.5: Tasks page
• 4.1.6: Task details
• 4.1.7: Interface of the annotation tool
• 4.1.8: Basic navigation
• 4.1.9: Top Panel
• 4.1.10: Controls sidebar
• 4.1.11: Objects sidebar
• 4.1.12: Workspace
• 4.1.13: 3D task workspace
• 4.1.14: Standard 3D mode (basics)
• 4.1.15: Settings
• 4.1.16: Types of shapes
• 4.1.17: Shape mode (basics)
• 4.1.18: Track mode (basics)
• 4.1.19: 3D Object annotation (basics)
• 4.1.20: Attribute annotation mode (basics)
• 4.1.21: Vocabulary
• 4.1.22: Cloud storages page
• 4.1.23: Attach cloud storage
• 4.2: Advanced
• 4.2.1: Projects page
• 4.2.2: Organization
• 4.2.3: Search
• 4.2.4: Shape mode (advanced)
• 4.2.5: Track mode (advanced)
• 4.2.6: 3D Object annotation (advanced)
• 4.2.7: Attribute annotation mode (advanced)
• 4.2.8: Annotation with rectangles
• 4.2.9: Annotation with polygons
• 4.2.9.1: Manual drawing
• 4.2.9.2: Drawing using automatic borders
• 4.2.9.3: Edit polygon
• 4.2.9.4: Track mode with polygons
• 4.2.9.5: Creating masks
• 4.2.10: Annotation with polylines
• 4.2.11: Annotation with points
• 4.2.11.1: Points in shape mode
• 4.2.11.2: Linear interpolation with one point
• 4.2.12: Annotation with ellipses
• 4.2.13: Annotation with cuboids
• 4.2.13.1: Creating the cuboid
• 4.2.13.2: Editing the cuboid
• 4.2.14: Annotation with skeletons
• 4.2.14.1: Creating the skeleton
• 4.2.14.2: Editing the skeleton
• 4.2.15: Annotation with brush tool
• 4.2.16: Annotation with tags
• 4.2.17: Models
• 4.2.18: AI Tools
• 4.2.19: OpenCV tools
• 4.2.20: Automatic annotation
• 4.2.21: Backup Task and Project
• 4.2.22: Frame deleting
• 4.2.23: Export/import datasets and upload annotation
• 4.2.24: Formats
• 4.2.24.1:
• 4.2.24.2:
• 4.2.24.3:
• 4.2.24.4:
• 4.2.24.5:
• 4.2.24.6:
• 4.2.24.7:
• 4.2.24.8:
• 4.2.24.9:
• 4.2.24.10:
• 4.2.24.11:
• 4.2.24.12:
• 4.2.24.13:
• 4.2.24.14:
• 4.2.24.15:
• 4.2.24.16:
• 4.2.24.17:
• 4.2.24.18:
• 4.2.24.19:
• 4.2.25: Task synchronization with a repository
• 4.2.26: XML annotation format
• 4.2.27: Shortcuts
• 4.2.28: Filter
• 4.2.29: Review
• 4.2.30: Context images for 2d task
• 4.2.31: Shape grouping
• 4.2.32: Analytics Monitoring
• 4.2.33: Dataset Manifest
• 4.2.34: Data preparation on the fly
• 4.2.35: Serverless tutorial
• 5: Administration
• 5.1: Basics
• 5.1.1: Installation Guide
• 5.1.2: Superuser registration
• 5.1.3: Social auth configuration
• 5.1.4: AWS-Deployment Guide
• 5.1.5: REST API guide
• 5.2: Advanced
• 5.2.1: CVAT deployment on Kubernetes with Helm
• 5.2.2: Semi-automatic and Automatic Annotation
• 5.2.3: Installation Analytics
• 5.2.4: Mounting cloud storage
• 5.2.5: LDAP Backed Authentication
• 5.2.6: Backup guide
• 5.2.7: Upgrade guide
• 5.2.8: IAM: system roles
• 5.2.9: Webhooks
• 6: API & SDK
• 6.1: Server API
• 6.2: CVAT Python SDK
• 6.2.1: SDK API Reference
• 6.2.1.1: APIs
• 6.2.1.1.1: AnalyticsApi class reference
• 6.2.1.1.2: AssetsApi class reference
• 6.2.1.1.3: AuthApi class reference
• 6.2.1.1.4: CloudstoragesApi class reference
• 6.2.1.1.5: CommentsApi class reference
• 6.2.1.1.6: EventsApi class reference
• 6.2.1.1.7: GuidesApi class reference
• 6.2.1.1.8: InvitationsApi class reference
• 6.2.1.1.9: IssuesApi class reference
• 6.2.1.1.10: JobsApi class reference
• 6.2.1.1.11: LabelsApi class reference
• 6.2.1.1.12: LambdaApi class reference
• 6.2.1.1.13: MembershipsApi class reference
• 6.2.1.1.14: OrganizationsApi class reference
• 6.2.1.1.15: ProjectsApi class reference
• 6.2.1.1.16: QualityApi class reference
• 6.2.1.1.17: SchemaApi class reference
• 6.2.1.1.18: ServerApi class reference
• 6.2.1.1.19: TasksApi class reference
• 6.2.1.1.20: UsersApi class reference
• 6.2.1.1.21: WebhooksApi class reference
• 6.2.1.2: Models
• 6.2.1.2.1: About class reference
• 6.2.1.2.2: AcceptInvitationRead class reference
• 6.2.1.2.3: AnalyticsReport class reference
• 6.2.1.2.4: AnalyticsReportCreateRequest class reference
• 6.2.1.2.5: AnalyticsReportTargetEnum class reference
• 6.2.1.2.6: AnnotationConflict class reference
• 6.2.1.2.7: AnnotationConflictTypeEnum class reference
• 6.2.1.2.8: AnnotationFileRequest class reference
• 6.2.1.2.9: AnnotationGuideRead class reference
• 6.2.1.2.10: AnnotationGuideWriteRequest class reference
• 6.2.1.2.11: AnnotationId class reference
• 6.2.1.2.12: AnnotationIdShapeType class reference
• 6.2.1.2.13: AnnotationIdTypeEnum class reference
• 6.2.1.2.14: AnnotationsRead class reference
• 6.2.1.2.15: AssetRead class reference
• 6.2.1.2.16: Attribute class reference
• 6.2.1.2.17: AttributeRequest class reference
• 6.2.1.2.18: AttributeVal class reference
• 6.2.1.2.19: AttributeValRequest class reference
• 6.2.1.2.20: BackupWriteRequest class reference
• 6.2.1.2.21: BasicOrganization class reference
• 6.2.1.2.22: BasicUser class reference
• 6.2.1.2.23: BasicUserRequest class reference
• 6.2.1.2.24: BinaryOperation class reference
• 6.2.1.2.25: ChunkType class reference
• 6.2.1.2.26: ClientEvents class reference
• 6.2.1.2.27: ClientEventsRequest class reference
• 6.2.1.2.28: ClientEventsRequestPreviousEvent class reference
• 6.2.1.2.29: CloudStorageContent class reference
• 6.2.1.2.30: CloudStorageRead class reference
• 6.2.1.2.31: CloudStorageReadOwner class reference
• 6.2.1.2.32: CloudStorageWriteRequest class reference
• 6.2.1.2.33: CommentRead class reference
• 6.2.1.2.34: CommentsSummary class reference
• 6.2.1.2.35: CommentWriteRequest class reference
• 6.2.1.2.36: CredentialsTypeEnum class reference
• 6.2.1.2.37: DataFrame class reference
• 6.2.1.2.38: DataMetaRead class reference
• 6.2.1.2.39: DataRequest class reference
• 6.2.1.2.40: DatasetFileRequest class reference
• 6.2.1.2.41: DatasetFormat class reference
• 6.2.1.2.42: DatasetFormats class reference
• 6.2.1.2.43: DatasetWriteRequest class reference
• 6.2.1.2.44: DefaultViewEnum class reference
• 6.2.1.2.45: Event class reference
• 6.2.1.2.46: EventRequest class reference
• 6.2.1.2.47: Events class reference
• 6.2.1.2.48: EventsEnum class reference
• 6.2.1.2.49: FileInfo class reference
• 6.2.1.2.50: FileInfoTypeEnum class reference
• 6.2.1.2.51: FrameMeta class reference
• 6.2.1.2.52: FrameSelectionMethodEnum class reference
• 6.2.1.2.53: FunctionCall class reference
• 6.2.1.2.54: FunctionCallParams class reference
• 6.2.1.2.55: FunctionCallRequest class reference
• 6.2.1.2.56: GranularityEnum class reference
• 6.2.1.2.57: InputTypeEnum class reference
• 6.2.1.2.58: InvitationRead class reference
• 6.2.1.2.59: InvitationWriteRequest class reference
• 6.2.1.2.60: IssueRead class reference
• 6.2.1.2.61: IssuesSummary class reference
• 6.2.1.2.62: IssueWriteRequest class reference
• 6.2.1.2.63: JobAnnotationsUpdateRequest class reference
• 6.2.1.2.64: JobRead class reference
• 6.2.1.2.65: JobReadAssignee class reference
• 6.2.1.2.66: JobReadTargetStorage class reference
• 6.2.1.2.67: JobsSummary class reference
• 6.2.1.2.68: JobStage class reference
• 6.2.1.2.69: JobStatus class reference
• 6.2.1.2.70: JobType class reference
• 6.2.1.2.71: JobWriteRequest class reference
• 6.2.1.2.72: Label class reference
• 6.2.1.2.73: LabeledData class reference
• 6.2.1.2.74: LabeledDataRequest class reference
• 6.2.1.2.75: LabeledImage class reference
• 6.2.1.2.76: LabeledImageRequest class reference
• 6.2.1.2.77: LabeledShape class reference
• 6.2.1.2.78: LabeledShapeRequest class reference
• 6.2.1.2.79: LabeledTrack class reference
• 6.2.1.2.80: LabeledTrackRequest class reference
• 6.2.1.2.81: LabelMappingEntryRequest class reference
• 6.2.1.2.82: LabelsSummary class reference
• 6.2.1.2.83: LocationEnum class reference
• 6.2.1.2.84: LoginSerializerExRequest class reference
• 6.2.1.2.85: MembershipRead class reference
• 6.2.1.2.86: MetaUser class reference
• 6.2.1.2.87: Metric class reference
• 6.2.1.2.88: MetricGranularity class reference
• 6.2.1.2.89: NullEnum class reference
• 6.2.1.2.90: OnlineFunctionCallRequest class reference
• 6.2.1.2.91: OperationStatus class reference
• 6.2.1.2.92: OperatorEnum class reference
• 6.2.1.2.93: OrganizationRead class reference
• 6.2.1.2.94: OrganizationWriteRequest class reference
• 6.2.1.2.95: PaginatedAnnotationConflictList class reference
• 6.2.1.2.96: PaginatedCloudStorageReadList class reference
• 6.2.1.2.97: PaginatedCommentReadList class reference
• 6.2.1.2.98: PaginatedInvitationReadList class reference
• 6.2.1.2.99: PaginatedIssueReadList class reference
• 6.2.1.2.100: PaginatedJobReadList class reference
• 6.2.1.2.101: PaginatedLabelList class reference
• 6.2.1.2.102: PaginatedMembershipReadList class reference
• 6.2.1.2.103: PaginatedMetaUserList class reference
• 6.2.1.2.104: PaginatedOrganizationReadList class reference
• 6.2.1.2.105: PaginatedProjectReadList class reference
• 6.2.1.2.106: PaginatedQualityReportList class reference
• 6.2.1.2.107: PaginatedQualitySettingsList class reference
• 6.2.1.2.108: PaginatedTaskReadList class reference
• 6.2.1.2.109: PaginatedWebhookDeliveryReadList class reference
• 6.2.1.2.110: PaginatedWebhookReadList class reference
• 6.2.1.2.111: PasswordChangeRequest class reference
• 6.2.1.2.112: PasswordResetConfirmRequest class reference
• 6.2.1.2.113: PasswordResetSerializerExRequest class reference
• 6.2.1.2.114: PatchedAnnotationGuideWriteRequest class reference
• 6.2.1.2.115: PatchedCloudStorageWriteRequest class reference
• 6.2.1.2.116: PatchedCommentWriteRequest class reference
• 6.2.1.2.117: PatchedDataMetaWriteRequest class reference
• 6.2.1.2.118: PatchedInvitationWriteRequest class reference
• 6.2.1.2.119: PatchedIssueWriteRequest class reference
• 6.2.1.2.120: PatchedJobWriteRequest class reference
• 6.2.1.2.121: PatchedLabeledDataRequest class reference
• 6.2.1.2.122: PatchedLabelRequest class reference
• 6.2.1.2.123: PatchedMembershipWriteRequest class reference
• 6.2.1.2.124: PatchedOrganizationWriteRequest class reference
• 6.2.1.2.125: PatchedProjectWriteRequest class reference
• 6.2.1.2.126: PatchedProjectWriteRequestTargetStorage class reference
• 6.2.1.2.127: PatchedQualitySettingsRequest class reference
• 6.2.1.2.128: PatchedTaskWriteRequest class reference
• 6.2.1.2.129: PatchedTaskWriteRequestTargetStorage class reference
• 6.2.1.2.130: PatchedUserRequest class reference
• 6.2.1.2.131: PatchedWebhookWriteRequest class reference
• 6.2.1.2.132: Plugins class reference
• 6.2.1.2.133: ProjectFileRequest class reference
• 6.2.1.2.134: ProjectRead class reference
• 6.2.1.2.135: ProjectReadTargetStorage class reference
• 6.2.1.2.136: ProjectWriteRequest class reference
• 6.2.1.2.137: ProviderTypeEnum class reference
• 6.2.1.2.138: QualityEnum class reference
• 6.2.1.2.139: QualityReport class reference
• 6.2.1.2.140: QualityReportCreateRequest class reference
• 6.2.1.2.141: QualityReportSummary class reference
• 6.2.1.2.142: QualityReportTarget class reference
• 6.2.1.2.143: QualitySettings class reference
• 6.2.1.2.144: RegisterSerializerEx class reference
• 6.2.1.2.145: RegisterSerializerExRequest class reference
• 6.2.1.2.146: RestAuthDetail class reference
• 6.2.1.2.147: RoleEnum class reference
• 6.2.1.2.148: RqId class reference
• 6.2.1.2.149: RqStatus class reference
• 6.2.1.2.150: RqStatusStateEnum class reference
• 6.2.1.2.151: SeverityEnum class reference
• 6.2.1.2.152: ShapeType class reference
• 6.2.1.2.153: SigningRequest class reference
• 6.2.1.2.154: SortingMethod class reference
• 6.2.1.2.155: Storage class reference
• 6.2.1.2.156: StorageMethod class reference
• 6.2.1.2.157: StorageRequest class reference
• 6.2.1.2.158: StorageType class reference
• 6.2.1.2.159: Sublabel class reference
• 6.2.1.2.160: SubLabeledShape class reference
• 6.2.1.2.161: SubLabeledShapeRequest class reference
• 6.2.1.2.162: SubLabeledTrack class reference
• 6.2.1.2.163: SubLabeledTrackRequest class reference
• 6.2.1.2.164: SublabelMappingEntryRequest class reference
• 6.2.1.2.165: SublabelRequest class reference
• 6.2.1.2.166: TaskAnnotationsUpdateRequest class reference
• 6.2.1.2.167: TaskAnnotationsWriteRequest class reference
• 6.2.1.2.168: TaskFileRequest class reference
• 6.2.1.2.169: TaskRead class reference
• 6.2.1.2.170: TasksSummary class reference
• 6.2.1.2.171: TaskWriteRequest class reference
• 6.2.1.2.172: Token class reference
• 6.2.1.2.173: TrackedShape class reference
• 6.2.1.2.174: TrackedShapeRequest class reference
• 6.2.1.2.175: Transformation class reference
• 6.2.1.2.176: TransformationBinary class reference
• 6.2.1.2.177: User class reference
• 6.2.1.2.178: WebhookContentType class reference
• 6.2.1.2.179: WebhookDeliveryRead class reference
• 6.2.1.2.180: WebhookRead class reference
• 6.2.1.2.181: WebhookType class reference
• 6.2.1.2.182: WebhookWriteRequest class reference
• 6.2.2: Low-level API
• 6.2.3: High-level API
• 6.2.4: Developer guide
• 6.3: Command line interface (CLI)
• 7: Contributing to this project
• 7.1: Development environment
• 7.2: Setup additional components in development environment
• 7.3: JavaScript/Typescript coding style
• 7.4: Branching model
• 7.5: Using the issue tracker
• 7.6: Bug reports
• 7.7: Feature requests
• 7.8: Pull requests
• 7.9: How to add a new annotation format support
• 7.10: Running tests

1 - Getting started

This section contains basic information and links to sections necessary for a quick start.

Installation

First step is to install CVAT on your system:

• Installation on Ubuntu
• Installation on Windows 10
• Installation on Mac OS

To learn how to create a superuser and log in to CVAT, go to the authorization section.

Getting started in CVAT

To create a task, go to Tasks section. Click Create new task to go to the task creation page.

Set the name of the future task.

Set the label using the constructor: first click Add label, then enter the name of the label and choose the color.

You need to upload images or videos for your future annotation. To do so, simply drag and drop the files.

To learn more, go to creating an annotation task

Annotation

Basic

When the task is created, you will see a corresponding message in the top right corner. Click the Open task button to go to the task page.

Once on the task page, open a link to the job in the jobs list.

Choose a correct section for your type of the task and start annotation.

Advanced

In CVAT there is the possibility of using automatic and semi-automatic annotation what gives you the opportunity to speed up the execution of the annotation:

• OpenCV tools - tools included in CVAT by default.
• AI tools - tools requiring installation.
• Automatic annotation - automatic annotation with using DL models.

Export dataset

1. To download the annotations, first you have to save all changes. Click the Save button or press Ctrl+Sto save annotations quickly.

2. After you saved the changes, click the Menu button.

3. Then click the Export dataset button.

4. Lastly choose a format of the dataset. Exporting is available in formats from the list of supported formats.

To learn more, go to export/import datasets section.

1.1 - Product tour

1.1.1 - CVAT intro

In this video, we show how to use CVAT - how to sign up, upload your data, annotate it, and download it. This video is the first one in a series that’ll guide you on our product and its many features.

1.1.2 - Introduction to CVAT and Datumaro

We are excited to introduce the first video in our course series designed to help you annotate data faster and better using CVAT. In this introductory 4 minute video, we walk through:

1. what problems CVAT and Datumaro solve,
2. how they can speed up your model training process, and
3. some resources you can use to learn more about how to use them.

2 - Integrations

2.1 - FiftyOne

FiftyOne is an open-source tool for building high-quality datasets and computer vision models. FiftyOne supercharges your machine learning workflows by enabling you to visualize datasets and interpret models faster and more effectively.

FiftyOne provides an API to create tasks and jobs, upload data, define label schemas, and download annotations using CVAT, all programmatically in Python. All of the following label types are supported, for both image and video datasets:

• Classifications
• Detections
• Instance segmentations
• Polygons and polylines
• Keypoints
• Scalar fields
• Semantic segmentation

3 - Frequently asked questions

How to migrate data from CVAT.org to CVAT.ai

Please follow the export tasks and projects guide to download an archive with data which corresponds to your task or project. The backup for a project will have all tasks which are inside the project. Thus you don’t need to export them separately.

Please follow the import tasks and projects guide to upload your backup with a task or project to a CVAT instance.

See a quick demo below. It is really a simple process. If your data is huge, it may take some time. Please be patient.

How to upgrade CVAT

Before upgrading, please follow the backup guide and backup all CVAT volumes.

Follow the upgrade guide.

Kibana app works, but no logs are displayed

Make sure there aren’t error messages from Elasticsearch:

docker logs cvat_elasticsearch

If you see errors like this:

lood stage disk watermark [95%] exceeded on [uMg9WI30QIOJxxJNDiIPgQ][uMg9WI3][/usr/share/elasticsearch/data/nodes/0] free: 116.5gb[4%], all indices on this node will be marked read-only

You should free up disk space or change the threshold, to do so check: Elasticsearch documentation.

How to change default CVAT hostname or port

To change the hostname, simply set the CVAT_HOST environemnt variable

export CVAT_HOST=<YOUR_HOSTNAME_OR_IP>

NOTE, if you’re using docker-compose with sudo to run CVAT, then please add the -E (or --preserve-env) flag to preserve the user environment variable which set above to take effect in your docker containers:

sudo -E docker-compose up -d

If you want to change the default web application port, change the ports part of traefik service configuration in docker-compose.yml

services:
  traefik:
    ...
    ...
    ports:
      - <YOUR_WEB_PORTAL_PORT>:8080
      - 8090:8090

Note that changing the port does not make sense if you are using HTTPS - port 443 is conventionally used for HTTPS connections, and is needed for Let’s Encrypt TLS challenge.

How to configure connected share folder on Windows

Follow the Docker manual and configure the directory that you want to use as a shared directory:

• Docker toolbox manual
• Docker for windows (see FILE SHARING section)

After that, it should be possible to use this directory as a CVAT share:

version: '3.3'

services:
  cvat:
    volumes:
      - cvat_share:/home/django/share:ro

volumes:
  cvat_share:
    driver_opts:
      type: none
      device: /d/my_cvat_share
      o: bind

How to make unassigned tasks not visible to all users

Set reduce_task_visibility variable to True.

Where are uploaded images/videos stored

The uploaded data is stored in the cvat_data docker volume:

volumes:
  - cvat_data:/home/django/data

Where are annotations stored

Annotations are stored in the PostgreSQL database. The database files are stored in the cvat_db docker volume:

volumes:
  - cvat_db:/var/lib/postgresql/data

How to mark job/task as completed

The status is set by the user in the Info window of the job annotation view. There are three types of status: annotation, validation or completed. The status of the job changes the progress bar of the task.

How to install CVAT on Windows 10 Home

Follow this guide.

I do not have the Analytics tab on the header section. How can I add analytics

You should build CVAT images with ‘Analytics’ component.

How to upload annotations to an entire task from UI when there are multiple jobs in the task

You can upload annotation for a multi-job task from the Dasboard view or the Task view. Uploading of annotation from the Annotation view only affects the current job.

How to specify multiple hostnames

To do this, you will need to edit traefik.http.<router>.cvat.rule docker label for both the cvat and cvat_ui services, like so (see the documentation on Traefik rules for more details):

  cvat:
    labels:
      - traefik.http.routers.cvat.rule=(Host(`example1.com`) || Host(`example2.com`)) &&
          PathPrefix(`/api/`, `/git/`, `/opencv/`, `/analytics/`, `/static/`, `/admin`, `/documentation/`, `/django-rq`)

  cvat_ui:
    labels:
      - traefik.http.routers.cvat-ui.rule=Host(`example1.com`) || Host(`example2.com`)

How to create a task with multiple jobs

Set the segment size when you create a new task, this option is available in the Advanced configuration section.

How to transfer CVAT to another machine

Follow the backup/restore guide.

How to load your own DL model into CVAT

See the information here in the Serverless tutorial.

My server uses a custom SSL certificate and I don’t want to check it.

You can call control SSL certificate check with the --insecure CLI argument. For SDK, you can specify ssl_verify = True/False in the cvat_sdk.core.client.Config object.

4 - Manual

4.1 - Basics

4.1.1 - Registration

To start to annotate in CVAT, you need to create an account or log in to the existing account.

This section describes App CVAT, that is sutable for small personal projects, that do not require user management. It is also ok to use if you just want to try what is CVAT.

While it is easy to use, it has some limitations. For example, in App CVAT you cannot create a superuser (admin account) or administer user roles. All these features are available for Admin user in local version of CVAT.

See:

• User registration
• Account access

To creata account or log in, go to the App CVAT login page:

Note: By default authentication and registration with Google and GitHub work only for App CVAT.
If you want to use Google and GitHub authentication on a local installation, see Social auth configuration.

User registration

To register as a non-admin user, do the following:

1. Click Create an account.

   

2. Fill in all blank fields, accept terms of use, and click the Create an account button.

A username generates from the email automatically. You can edit it if needed.

To register with Google or Github, click the button with the name of the service, and folow instructions on the screen.

Account access

To access your account, do the following:

1. Go to the login page.
2. Enter username or email. The password field will appear.
3. Enter the password and click Next.

To log in with Google or Github, click the button with the name of the service.

4.1.2 - Creating an annotation task

Create an annotation task pressing + button and select Create new task on the tasks page or on the project page.

Notice that the task will be created inside the organization that you had selected at the time of creation. Read more about organizations.

Specify parameters of the task:

Basic configuration

Name

The name of the task to be created.

Projects

The project that this task will be related with.

Labels

There are two ways of working with labels (available only if the task is not related to the project):

• The Constructor is a simple way to add and adjust labels. To add a new label click the Add label button.

  

  You can set a name of the label in the Label name field and choose a color for each label.

  

  If necessary you can add an attribute and set its properties by clicking Add an attribute:

  

  The following actions are available here:

  1. Set the attribute’s name.
  2. Choose the way to display the attribute:
     • Select — drop down list of value
     • Radio — is used when it is necessary to choose just one option out of few suggested.
     • Checkbox — is used when it is necessary to choose any number of options out of suggested.
     • Text — is used when an attribute is entered as a text.
     • Number — is used when an attribute is entered as a number.
  3. Set values for the attribute. The values could be separated by pressing Enter. The entered value is displayed as a separate element which could be deleted by pressing Backspace or clicking the close button (x). If the specified way of displaying the attribute is Text or Number, the entered value will be displayed as text by default (e.g. you can specify the text format).
  4. Checkbox Mutable determines if an attribute would be changed frame to frame.
  5. You can delete the attribute by clicking the close button (x).

  Click the Continue button to add more labels. If you need to cancel adding a label - press the Cancel button. After all the necessary labels are added click the Done button. After clicking Done the added labels would be displayed as separate elements of different colour. You can edit or delete labels by clicking Update attributes or Delete label.

• The Raw is a way of working with labels for an advanced user. Raw presents label data in json format with an option of editing and copying labels as a text. The Done button applies the changes and the Reset button cancels the changes.

Select files

Press tab My computer to choose some files for annotation from your PC. If you select tab Connected file share you can choose files for annotation from your network. If you select Remote source , you’ll see a field where you can enter a list of URLs (one URL per line). If you upload a video or dataset with images and select Use cache option, you can attach a manifest.jsonl file. You can find how to prepare it here. If you select the Cloud Storage tab, you can select a cloud storage (for this type the cloud storage name), after that choose the manifest file and select the required files. For more information on how to attach cloud storage, see attach cloud storage

Data formats for a 3D task

To create a 3D task, you must prepare an archive with one of the following directory structures:

• Velodyne
• 3D pointcloud
• 3D Option 1
• 3D Option 2

    VELODYNE FORMAT
    Structure:
      velodyne_points/
        data/
          image_01.bin
          IMAGE_00 # unknown dirname,
                   # generally image_01.png can be under IMAGE_00, IMAGE_01, IMAGE_02, IMAGE_03, etc
      data/
        image_01.png
  

    3D POINTCLOUD DATA FORMAT
    Structure:
      pointcloud/
        00001.pcd
      related_images/
        00001_pcd/
          image_01.png # or any other image
  

    3D, DEFAULT DATAFORMAT Option 1
    Structure:
      data/
        image.pcd
        image.png
  

    3D, DEFAULT DATAFORMAT Option 2
    Structure:
      data/
        image_1/
            image_1.pcd
            context_1.png # or any other name
            context_2.jpg
  

You can’t mix 2D and 3D data in the same task.

Advanced configuration

Sorting method

Option to sort the data. It is not relevant for videos. For example, the sequence 2.jpeg, 10.jpeg, 1.jpeg after sorting will be:

• lexicographical: 1.jpeg, 10.jpeg, 2.jpeg
• natural: 1.jpeg, 2.jpeg, 10.jpeg
• predefined: 2.jpeg, 10.jpeg, 1.jpeg

Use zip/video chunks

Force to use zip chunks as compressed data. Cut out content for videos only.

Use cache

Defines how to work with data. Select the checkbox to switch to the “on-the-fly data processing”, which will reduce the task creation time (by preparing chunks when requests are received) and store data in a cache of limited size with a policy of evicting less popular items. See more here.

Image Quality

Use this option to specify quality of uploaded images. The option helps to load high resolution datasets faster. Use the value from 5 (almost completely compressed images) to 100 (not compressed images).

Overlap Size

Use this option to make overlapped segments. The option makes tracks continuous from one segment into another. Use it for interpolation mode. There are several options for using the parameter:

• For an interpolation task (video sequence). If you annotate a bounding box on two adjacent segments they will be merged into one bounding box. If overlap equals to zero or annotation is poor on adjacent segments inside a dumped annotation file, you will have several tracks, one for each segment, which corresponds to the object.
• For an annotation task (independent images). If an object exists on overlapped segments, the overlap is greater than zero and the annotation is good enough on adjacent segments, it will be automatically merged into one object. If overlap equals to zero or annotation is poor on adjacent segments inside a dumped annotation file, you will have several bounding boxes for the same object. Thus, you annotate an object on the first segment. You annotate the same object on second segment, and if you do it right, you will have one track inside the annotations. If annotations on different segments (on overlapped frames) are very different, you will have two shapes for the same object. This functionality works only for bounding boxes. Polygons, polylines, points don’t support automatic merge on overlapped segments even the overlap parameter isn’t zero and match between corresponding shapes on adjacent segments is perfect.

Segment size

Use this option to divide a huge dataset into a few smaller segments. For example, one job cannot be annotated by several labelers (it isn’t supported). Thus using “segment size” you can create several jobs for the same annotation task. It will help you to parallel data annotation process.

Start frame

Frame from which video in task begins.

Stop frame

Frame on which video in task ends.

Frame Step

Use this option to filter video frames. For example, enter 25 to leave every twenty fifth frame in the video or every twenty fifth image.

Chunk size

Defines a number of frames to be packed in a chunk when send from client to server. Server defines automatically if empty.

Recommended values:

• 1080p or less: 36
• 2k or less: 8 - 16
• 4k or less: 4 - 8
• More: 1 - 4

Dataset Repository

URL link of the repository optionally specifies the path to the repository for storage (default: annotation / <dump_file_name> .zip). The .zip and .xml file extension of annotation are supported. Field format: URL [PATH] example: https://github.com/project/repos.git [1/2/3/4/annotation.xml]

Supported URL formats :

• https://github.com/project/repos[.git]
• github.com/project/repos[.git]
• git@github.com:project/repos[.git]

After the task is created, the synchronization status is displayed on the task page.

If you specify a dataset repository, when you create a task, you will see a message about the need to grant access with the ssh key. This is the key you need to add to your github account. For other git systems, you can learn about adding an ssh key in their documentation.

Use LFS

If the annotation file is large, you can create a repository with LFS support.

Issue tracker

Specify full issue tracker’s URL if it’s necessary.

Source storage

Specify source storage for import resources like annotations and backups. It can be a local or a cloud storage. If the task is created in the project, then the Use project source storage switch will determine whether to use the default values ​​or specify new ones.

Target storage

Specify target storage for export resources like annotations and backups. It can be a local or a cloud storage. If the task is created in the project, then the Use project target storage switch will determine whether to use the default values ​​or specify new ones.

To save and open task click on Submit & Open button. Also you can click on Submit & Continue button for creating several tasks in sequence. Then, the created tasks will be displayed on a tasks page.

4.1.3 - Creating multi tasks

In case when you have a few videos and you don’t want to set up CVAT tasks one-by-one manually, you can use the multi tasks option.

This feature is available for videos only.

Create several tasks pressing + button and select Create multi tasks on the tasks page or on the project page.

Specify the task parameters:

• In the Name text field, there is a template you can use for quick and simple task naming.

  

  In the template you can manage the following:

  • some_text - any text that will be entered into the template will be the same for all created tasks.
  • {{index}} - index file in set (sets a number of a task starting from 0).
  • {{file_name}} - name of file (sets a task name from a filename).

  You can find out a prompt by hovering the mouse over the text When forming the name, a template is used.

  

• Project, labels, select files and advanced configuration are filled in the same way as when creating an annotation task.

It is not possible yet to use cloud storages for multi tasks creation.

1. Let’s specify in the Name field the Road_annotation-{{index}}-{{file_name}} string, where Road_annotation is “any text” (let it match to the task type name). The task index number and the video file name will be specified after it.

2. After selecting files, file names will be shown under the Select files section. In case of there are more than 4 files, the total number of selected files will be displayed only.

   

3. The Submit button indicates how many tasks will be created.

After clicking on the Submit button, information about tasks creation process will be displayed.

If there were no errors during the process of creating tasks, we can complete this by clicking on Ok.

This is how the created tasks will look like.

Errors

You can face with the following errors during multi tasks creation:

• If where were no video files selected, this warning will be shown.

  

• If during tasks creation there was not possible to process with some videos, the names of such videos will be displayed in the Failed files list. In this case, you can try to create tasks for these videos one more time by clicking on the Retry failed tasks button, or you can click on the Ok button to skip these videos.

  

4.1.4 - Jobs page

On the jobs page, users (for example, with the worker role) can see the jobs that are assigned to them without having access to the task page, as well as track progress, sort and apply filters to the job list.

On the job page there is a list of jobs presented in the form of tiles, where each tile is one job. Each element contains:

• job ID
• dimension 2D or 3D
• preview
• stage and state
• when hovering over an element, you can see:
  • size
  • assignee
• menu to navigate to a task, project, or bug tracker.

To open the job in a new tab, click on the job by holding Ctrl.

In the upper left corner there is a search bar, using which you can find the job by assignee, stage, state, etc. In the upper right corner there are sorting, quick filters and filter.

Filter

Applying filter disables the quick filter.

The filter works similarly to the filters for annotation, you can create rules from properties, operators and values and group rules into groups. For more details, see the filter section. Learn more about date and time selection.

For clear all filters press Clear filters.

Supported properties for jobs list

4.1.5 - Tasks page

The tasks page contains elements and each of them relates to a separate task. They are sorted in creation order. Each element contains: task name, preview, progress bar, button Open, and menu Actions. Each button is responsible for a in menu Actions specific function:

• Export task dataset — download annotations or annotations and images in a specific format. More information is available in the export/import datasets section.
• Upload annotation upload annotations in a specific format. More information is available in the export/import datasets section.
• Automatic Annotation — automatic annotation with OpenVINO toolkit. Presence depends on how you build the CVAT instance.
• Backup task — make a backup of this task into a zip archive. Read more in the backup section.
• Move to project — Moving a task to a project (can be used to move a task from one project to another). Note that attributes reset during the moving process. In case of label mismatch, you can create or delete necessary labels in the project/task. Some task labels can be matched with the target project labels.
• Delete — delete task.

In the upper left corner there is a search bar, using which you can find the task by assignee, task name etc. In the upper right corner there are sorting, quick filters and filter.

Filter

Applying filter disables the quick filter.

The filter works similarly to the filters for annotation, you can create rules from properties, operators and values and group rules into groups. For more details, see the filter section. Learn more about date and time selection.

For clear all filters press Clear filters.

Supported properties for tasks list

Push Open button to go to task details.

4.1.6 - Task details

Task details is a task page which contains a preview, a progress bar and the details of the task (specified when the task was created) and the jobs section.

• The next actions are available on this page:

  1. Change the task’s title.

  2. Open Actions menu.

  3. Change issue tracker or open issue tracker if it is specified.

  4. Change labels (available only if the task is not related to the project). You can add new labels or add attributes for the existing labels in the Raw mode or the Constructor mode. By clicking Copy you will copy the labels to the clipboard.

  5. Assigned to — is used to assign a task to a person. Start typing an assignee’s name and/or choose the right person out of the dropdown list. In the list of users, you will only see the users of the organization where the task is created.

  6. Dataset Repository

     • Repository link
     • Synchronization status with dataset repository. When you click on the status, the current annotation will be sent. It has several states:
       • Synchronized - task synchronized, that is, created a pull of requisites with an actual annotation file.
       • Merged - merged pull request with up-to-date annotation file.
       • Synchronize - highlighted in red, annotations are not synced.
     • Use a format drop-down list of formats in which the annotation can be synchronized.
     • Support for large file enabling the use of LFS.

• Jobs — is a list of all jobs for a particular task. Here you can find the next data:

  • Jobs name with a hyperlink to it.
  • Frames — the frame interval.
  • A stage of the job. The stage is specified by a drop-down list. There are three stages: annotation, validation or acceptance. This value affects the task progress bar.
  • A state of the job. The state can be changed by an assigned user in the menu inside the job. There are several possible states: new, in progress, rejected, completed.
  • Started on — start date of this job.
  • Duration — is the amount of time the job is being worked.
  • Assignee is the user who is working on the job. You can start typing an assignee’s name and/or choose the right person out of the dropdown list.
  • Reviewer – a user assigned to carry out the review, read more in the review section.
  • Copy. By clicking Copy you will copy the job list to the clipboard. The job list contains direct links to jobs.

  You can filter or sort jobs by status, as well as by assigner or reviewer.

Follow a link inside Jobs section to start annotation process. In some cases, you can have several links. It depends on size of your task and Overlap Size and Segment Size parameters. To improve UX, only the first chunk of several frames will be loaded and you will be able to annotate first images. Other frames will be loaded in background.

4.1.7 - Interface of the annotation tool

Main user interface

The tool consists of:

• Header - pinned header used to navigate CVAT sections and account settings;

• Top panel — contains navigation buttons, main functions and menu access;

• Workspace — space where images are shown;

• Controls sidebar — contains tools for navigating the image, zoom, creating shapes and editing tracks (merge, split, group);

• Objects sidebar — contains label filter, two lists: objects (on the frame) and labels (of objects on the frame) and appearance settings.

Pop-up messages

In CVAT, you’ll receive pop-up messages in the upper-right corner, on any page. Pop-up messages can contain useful information, links, or error messages.

Information message

Informational messages inform about the end of the auto-annotation process. Learn more about auto-annotation.

Jump Suggestion Messages

Open a task

After creating a task, you can immediately open it by clicking Open task. Learn more about creating a task.

Continue to the frame on which the work on the job is finished

When you open a job that you previously worked on, you will receive pop-up messages with a proposal to go to the frame that was visited before closing the tab.

Error Messages

If you perform impossible actions, you may receive an error message. The message may contain information about the error or a prompt to open the browser console (shortcut F12) for information. If you encounter a bug that you can’t solve yourself, you can create an issue on GitHub.

4.1.8 - Basic navigation

1. Use arrows below to move to the next/previous frame. Use the scroll bar slider to scroll through frames. Almost every button has a shortcut. To get a hint about a shortcut, just move your mouse pointer over an UI element.

   

2. To navigate the image, use the button on the controls sidebar. Another way an image can be moved/shifted is by holding the left mouse button inside an area without annotated objects. If the Mouse Wheel is pressed, then all annotated objects are ignored. Otherwise the a highlighted bounding box will be moved instead of the image itself.

   

3. You can use the button on the sidebar controls to zoom on a region of interest. Use the button Fit the image to fit the image in the workspace. You can also use the mouse wheel to scale the image (the image will be zoomed relatively to your current cursor position).

   

4.1.9 - Top Panel

Menu button

It is the main menu of the annotation tool. It can be used to download, upload and remove annotations.

Button assignment:

• Upload Annotations — uploads annotations into a task.

• Export as a dataset — download a data set from a task in one of the supported formats. You can also enter a Custom name and enable the Save images checkbox if you want the dataset to contain images.

• Remove Annotations — calls the confirmation window if you click Delete, the annotation of the current job will be removed, if you click Select range you can remove annotation on range frames, if you activate checkbox Delete only keyframe for tracks then only keyframes will be deleted from the tracks, on the selected range.

  

• Open the task — opens a page with details about the task.

• Change job state - changes the state of the job (new, in progress, rejected, completed).

• Finish the job/Renew the job - changes the job stage and state to acceptance and completed / annotation and new correspondingly.

Save Work

Saves annotations for the current job. The button has an indication of the saving process.

Undo-redo buttons

Use buttons to undo actions or redo them.

Done

Used to complete the creation of the object. This button appears only when the object is being created.

Block

Used to pause automatic line creation when drawing a polygon with OpenCV Intelligent scissors. Also used to postpone server requests when creating an object using AI Tools. When blocking is activated, the button turns blue.

Player

Go to the first /the latest frames.

Go to the next/previous frame with a predefined step. Shortcuts: V — step backward, C — step forward. By default the step is 10 frames (change at Account Menu —> Settings —> Player Step).

The button to go to the next / previous frame has the customization possibility. To customize, right-click on the button and select one of three options:

1. The default option - go to the next / previous frame (the step is 1 frame).
2. Go to the next / previous frame that has any objects (in particular filtered). Read the filter section to know the details how to use it.
3. Go to the next / previous frame without annotation at all. Use this option in cases when you need to find missed frames quickly.

Shortcuts: D - previous, F - next.

Play the sequence of frames or the set of images. Shortcut: Space (change at Account Menu —> Settings —> Player Speed).

Go to a specific frame. Press ~ to focus on the element.

Fullscreen Player

The fullscreen player mode. The keyboard shortcut is F11.

Info

Open the job info.

Overview:

• Assignee - the one to whom the job is assigned.
• Reviewer – a user assigned to carry out the review, read more in the review section.
• Start Frame - the number of the first frame in this job.
• End Frame - the number of the last frame in this job.
• Frames - the total number of all frames in the job.

Annotations statistics:

This is a table number of created shapes, sorted by labels (e.g. vehicle, person) and type of annotation (shape, track). As well as the number of manual and interpolated frames.

UI switcher

Switching between user interface modes.

4.1.10 - Controls sidebar

Navigation

Navigation block - contains tools for moving and rotating images.

Icon	Description
	Cursor (Esc)- a basic annotation pedacting tool.
	Move the image- a tool for moving around the image without the possibility of editing.
	Rotate- two buttons to rotate the current frame a clockwise (Ctrl+R) and anticlockwise (Ctrl+Shift+R). You can enable Rotate all images in the settings to rotate all the images in the job

Zoom

Zoom block - contains tools for image zoom.

Icon	Description
	Fit image- fits image into the workspace size. Shortcut - double click on an image
	Select a region of interest- zooms in on a selected region. You can use this tool to quickly zoom in on a specific part of the frame.

Shapes

Shapes block - contains all the tools for creating shapes.

Icon	Description	Links to section
	AI Tools	AI Tools
	OpenCV	OpenCV
	Rectangle	Shape mode; Track mode; Drawing by 4 points
	Polygon	Annotation with polygons; Track mode with polygons
	Polyline	Annotation with polylines
	Points	Annotation with points
	Ellipses	Annotation with ellipses
	Cuboid	Annotation with cuboids
	Brushing tools	Annotation with brushing
	Tag	Annotation with tags
	Open an issue	Review (available only in review mode)

Edit

Edit block - contains tools for editing tracks and shapes.

Icon	Description	Links to section
	Merge Shapes(M) - starts/stops the merging shapes mode.	Track mode (basics)
	Group Shapes (G) - starts/stops the grouping shapes mode.	Shape grouping
	Split - splits a track.	Track mode (advanced)

4.1.11 - Objects sidebar

Hide objects sidebar

Hide - the button hides the object’s sidebar.

Objects

Filter input box

The way how to use filters is described in the advanced guide here.

List of objects

• Switch lock property for all - switches lock property of all objects in the frame.
• Switch hidden property for all - switches hide property of all objects in the frame.
• Expand/collapse all - collapses/expands the details field of all objects in the frame.
• Sorting - sort the list of objects: updated time, ID - accent, ID - descent

In the objects sidebar you can see the list of available objects on the current frame. The following figure is an example of how the list might look like:

Shape mode	Track mode

Objects on the side bar

The type of a shape can be changed by selecting Label property. For instance, it can look like shown on the figure below:

Object action menu

The action menu calls up the button:

The action menu contains:

• Create object URL - puts a link to an object on the clipboard. After you open the link, this object will be filtered.

• Make a copy- copies an object. The keyboard shortcut is Ctrl + C Ctrl + V.

• Propagate - Copies the form to several frames, invokes a dialog box in which you can specify the number of copies or the frame onto which you want to copy the object. The keyboard shortcut Ctrl + B.

  

• To background - moves the object to the background. The keyboard shortcut -,_.

• To foreground - moves the object to the foreground. The keyboard shortcut +,=.

• Change instance color- choosing a color using the color picker (available only in instance mode).

  

• Remove - removes the object. The keyboard shortcut Del,Shift+Del.

A shape can be locked to prevent its modification or moving by an accident. Shortcut to lock an object: L.

A shape can be Occluded. Shortcut: Q. Such shapes have dashed boundaries.

You can change the way an object is displayed on a frame (show or hide).

Switch pinned property - when enabled, a shape cannot be moved by dragging or dropping.

Tracker switcher - enable/disable tracking for the object.

By clicking on the Details button you can collapse or expand the field with all the attributes of the object.

Labels

In this tab you can lock or hide objects of a certain label. To change the color for a specific label, you need to go to the task page and select the color by clicking the edit button, this way you will change the label color for all jobs in the task.

Fast label change

You can change the label of an object using hot keys. In order to do it, you need to assign a number (from 0 to 9) to labels. By default numbers 1,2…0 are assigned to the first ten labels. To assign a number, click on the button placed at the right of a label name on the sidebar.

After that you will be able to assign a corresponding label to an object by hovering your mouse cursor over it and pressing Ctrl + Num(0..9).

In case you do not point the cursor to the object, pressing Ctrl + Num(0..9) will set a chosen label as default, so that the next object you create (use N key) will automatically have this label assigned.

Appearance

Color By options

Change the color scheme of annotation:

• Instance — every shape has random color

  

• Group — every group of shape has its own random color, ungrouped shapes are white

  

• Label — every label (e.g. car, person) has its own random color

  

  You can change any random color pointing to a needed box on a frame or on an object sidebar.

Fill Opacity slider

Change the opacity of every shape in the annotation.

Selected Fill Opacity slider

Change the opacity of the selected object’s fill. It is possible to change opacity while drawing an object in the case of rectangles, polygons and cuboids.

Outlines borders checkbox

You can change a special shape border color by clicking on the Eyedropper icon.

Show bitmap checkbox

If enabled all shapes are displayed in white and the background is black.

Show projections checkbox

Enables / disables the display of auxiliary perspective lines. Only relevant for cuboids

4.1.12 - Workspace

This is the main field in which drawing and editing objects takes place. In addition the workspace also has the following functions:

• Right-clicking on an object calls up the Object card - this is an element containing the necessary controls for changing the label and attributes of the object, as well as the action menu.

  

• Right-clicking a point deletes it.

  

• Z-axis slider - Allows you to switch annotation layers hiding the upper layers (slider is enabled if several z layers are on a frame). This element has a button for adding a new layer. When pressed, a new layer is added and switched to it. You can move objects in layers using the + and - keys.

  

• Image settings panel -  used to set up the grid and set up image brightness contrast saturation.

  • Show Grid, change grid size, choose color and transparency:

  

  • Adjust Brightness/Contrast/Saturation of too exposed or too dark images using color settings (it affects only how a user sees the image, not the image itself).

  

  • Reset color settings to default values.

4.1.13 - 3D task workspace

If the related_images folder contains any images, a context image will be available in the perspective window. The contextual image could be compared to 3D data and would help to identify the labels of marked objects.

Perspective – a main window for work with objects in a 3D task.

Projections - projections are tied to an object so that a cuboid is in the center and looks like a rectangle. Projections show only the selected object.

• Top – a projection of the view from above.
• Side – a projection of the left side of the object.
• Front - a frontal projection of the object.

4.1.14 - Standard 3D mode (basics)

Standard 3d mode - Designed to work with 3D data. The mode is automatically available if you add PCD or Kitty BIN format data when you create a task. read more

You can adjust the size of the projections, to do so, simply drag the boundary between the projections.

4.1.15 - Settings

To open the settings open the user menu in the header and select the settings item or press F2.

Settings have two tabs:

In tab Player you can:

• Control step of C and V shortcuts.
• Control speed of Space/Play button.
• Select canvas background color. You can choose a background color or enter manually (in RGB or HEX format).
• Reset zoom Show every image in full size or zoomed out like previous (it is enabled by default for interpolation mode and disabled for annotation mode).
• Rotate all images checkbox — switch the rotation of all frames or an individual frame.
• Smooth image checkbox — smooth image when zoom-in it.

  smoothed	pixelized

In tab Workspace you can:

• Enable auto save checkbox — turned off by default.

• Auto save interval (min) input box — 15 minutes by default.

• Show all interpolation tracks checkbox — shows hidden objects on the side panel for every interpolated object (turned off by default).

• Always show object details - show text for an object on the canvas not only when the object is activated:

  

• Content of a text - setup of the composition of the object details:

  • ID - object identifier.
  • Attributes - attributes of the object.
  • Label - object label.
  • Source- source of creating of objects MANUAL, AUTO or SEMI-AUTO.
  • Descriptions - description of attributes.

• Position of a text - text positioning mode selection:

  • Auto - the object details will be automatically placed where free space is.
  • Center - the object details will be embedded to a corresponding object if possible.

• Font size of a text - specifies the text size of the object details.

• Automatic bordering - enable automatic bordering for polygons and polylines during drawing/editing. For more information To find out more, go to the section annotation with polygons.

• Intelligent polygon cropping - activates intelligent cropping when editing the polygon (read more in the section edit polygon

• Show tags on frame - shows/hides frame tags on current frame

• Attribute annotation mode (AAM) zoom margin input box — defines margins (in px) for shape in the attribute annotation mode.

• Control points size — defines a size of any interactable points in the tool (polygon’s vertexes, rectangle dragging points, etc.)

• Default number of points in polygon approximation With this setting, you can choose the default number of points in polygon. Works for serverless interactors and OpenCV scissors.

• Click Save to save settings (settings will be saved on the server and will not change after the page is refreshed). Click Cancel or press F2 to return to the annotation.

4.1.16 - Types of shapes

There are several shapes with which you can annotate your images:

• Rectangle or Bounding box
• Polygon
• Polyline
• Points
• Ellipse
• Cuboid
• Cuboid in 3d task
• Skeleton
• Tag

And there is how they all look like:

Tag - has no shape in the workspace, but is displayed in objects sidebar.

4.1.17 - Shape mode (basics)

Usage examples:

• Create new annotations for a set of images.
• Add/modify/delete objects for existing annotations.

1. You need to select Rectangle on the controls sidebar:

   

   Before you start, select the correct Label (should be specified by you when creating the task) and Drawing Method (by 2 points or by 4 points):

   

2. Creating a new annotation in Shape mode:

   • Create a separate Rectangle by clicking on Shape.

     

   • Choose the opposite points. Your first rectangle is ready!

     

   • To learn more about creating a rectangle read here.

   • It is possible to adjust boundaries and location of the rectangle using a mouse. Rectangle’s size is shown in the top right corner , you can check it by clicking on any point of the shape. You can also undo your actions using Ctrl+Z and redo them with Shift+Ctrl+Z or Ctrl+Y.

3. You can see the Object card in the objects sidebar or open it by right-clicking on the object. You can change the attributes in the details section. You can perform basic operations or delete an object by clicking on the action menu button.

   

4. The following figure is an example of a fully annotated frame with separate shapes.

   

   Read more in the section shape mode (advanced).

4.1.18 - Track mode (basics)

Usage examples:

• Create new annotations for a sequence of frames.
• Add/modify/delete objects for existing annotations.
• Edit tracks, merge several rectangles into one track.

1. Like in the Shape mode, you need to select a Rectangle on the sidebar, in the appearing form, select the desired Label and the Drawing method.

   

2. Creating a track for an object (look at the selected car as an example):

   • Create a Rectangle in Track mode by clicking on Track.

     

   • In Track mode the rectangle will be automatically interpolated on the next frames.

   • The cyclist starts moving on frame #2270. Let’s mark the frame as a key frame. You can press K for that or click the star button (see the screenshot below).

     

   • If the object starts to change its position, you need to modify the rectangle where it happens. It isn’t necessary to change the rectangle on each frame, simply update several keyframes and the frames between them will be interpolated automatically.

   • Let’s jump 30 frames forward and adjust the boundaries of the object. See an example below:

     

   • After that the rectangle of the object will be changed automatically on frames 2270 to 2300:

     

3. When the annotated object disappears or becomes too small, you need to finish the track. You have to choose Outside Property, shortcut O.

   

4. If the object isn’t visible on a couple of frames and then appears again, you can use the Merge feature to merge several individual tracks into one.

   

   • Create tracks for moments when the cyclist is visible:

     

   • Click Merge button or press key M and click on any rectangle of the first track and on any rectangle of the second track and so on:

     

   • Click Merge button or press M to apply changes.

     

   • The final annotated sequence of frames in Interpolation mode can look like the clip below:

     

     Read more in the section track mode (advanced).

4.1.19 - 3D Object annotation (basics)

Navigation

To move in 3D space you can use several methods:

• Select the move the image tool so you can move the camera using the mouse:

  

  • Hold down the left mouse button in the perspective window to turn the camera around the conditional point.
  • Hold down the right mouse button in the perspective window to move the camera inside the 3D space.
  • Move the mouse while holding down the wheel to zoom in/out in the perspective window.
  • Scroll the wheel to zoom in/out (works both in perspective and in projections).

• Move using the keys in the perspective window

  

You can move around by pressing the corresponding buttons:

• To rotate the camera use: Shift+arrrowup/Shift+arrrowdown/Shift+arrrowleft/Shift+arrrowright.
• To move left/right use: Allt+J/Alt+L.
• To move up/down use: Alt-U/Alt+O.
• To zoom in/out use: Alt+K/Alt+I.

Creating a cuboid

To create a cube in a 3D task you need to click the appropriate icon on the control sidebar, select the label of the future object and click shape.

After that the cursor will be followed by a cube. In the creation process you can rotate and move the camera only using the keys. Left double-click will create an object. You can place an object only near the dots of the point cloud.

To adjust the size precisely, you need to edit the cuboid on the projections, for this change Cursor on control sidebar or press Esc. In each projection you can:

Move the object in the projection plane - to do this, hover over the object, press the left mouse button and move the object.

Move one of the four points - you can change the size of the cuboid by dragging the points in the projection.

Rotate the cuboid in the projection plane – to rotate the cuboid you should click on the appropriate point and then drag it up/down or to the left/right.

4.1.20 - Attribute annotation mode (basics)

• In this mode you can edit attributes with fast navigation between objects and frames using a keyboard. Open the drop-down list in the top panel and select Attribute annotation Mode.

  

• In this mode objects panel change to a special panel :

  

• The active attribute will be red. In this case it is gender . Look at the bottom side panel to see all possible shortcuts for changing the attribute. Press key 2 on your keyboard to assign a value (female) for the attribute or select from the drop-down list.

  

• Press Up Arrow/Down Arrow on your keyboard or click the buttons in the UI to go to the next/previous attribute. In this case, after pressing Down Arrow you will be able to edit the Age attribute.

  

• Use Right Arrow/Left Arrow keys to move to the previous/next image with annotation.

To see all the hot keys available in the attribute annotation mode, press F2. Read more in the section attribute annotation mode (advanced).

4.1.21 - Vocabulary

Label

Label is a type of an annotated object (e.g. person, car, vehicle, etc.)

Attribute

Attribute is a property of an annotated object (e.g. color, model, quality, etc.). There are two types of attributes:

Unique

Unique immutable and can’t be changed from frame to frame (e.g. age, gender, color, etc.)

Temporary

Temporary mutable and can be changed on any frame (e.g. quality, pose, truncated, etc.)

Track

Track is a set of shapes on different frames which corresponds to one object. Tracks are created in Track mode

Annotation

Annotation is a set of shapes and tracks. There are several types of annotations:

• Manual which is created by a person
• Semi-automatic which is created mainly automatically, but the user provides some data (e.g. interpolation)
• Automatic which is created automatically without a person in the loop

Approximation

Approximation allows you to reduce the number of points in the polygon. Can be used to reduce the annotation file and to facilitate editing polygons.

Trackable

Trackable object will be tracked automatically if the previous frame was a latest keyframe for the object. More details in the section trackers.

Mode

Interpolation

Mode for video annotation, which uses track objects. Only objects on keyframes are manually annotation, and intermediate frames are linearly interpolated.

Related sections:

• Track mode

Annotation

Mode for images annotation, which uses shape objects.

Related sections:

• Shape mode

Dimension

Depends on the task data type that is defined when the task is created.

2D

The data format of 2d tasks are images and videos. Related sections:

• Creating an annotation task

3D

The data format of 3d tasks is a cloud of points. Data formats for a 3D task

Related sections:

• 3D task workspace
• Standard 3D mode
• 3D Object annotation

State

State of the job. The state can be changed by an assigned user in the menu inside the job. There are several possible states: new, in progress, rejected, completed.

Stage

Stage of the job. The stage is specified with the drop-down list on the task page. There are three stages: annotation, validation or acceptance. This value affects the task progress bar.

Subset

A project can have subsets. Subsets are groups for tasks that make it easier to work with the dataset. It could be test, train, validation or custom subset.

Credentials

Under credentials is understood Key & secret key, Account name and token, Anonymous access, Key file. Used to attach cloud storage.

Resource

Under resource is understood bucket name or container name. Used to attach cloud storage.

4.1.22 - Cloud storages page

The cloud storages page contains elements, each of them relating to a separate cloud storage.  Each element contains: preview, cloud storage name, provider, creation and update info, status, ? button for displaying the description and the actions menu.

Each button in the action menu is responsible for a specific function:

• Update — update this cloud storage
• Delete — delete cloud storage.

This preview will appear when it is impossible to get a real preview (e.g. storage is empty or invalid credentials were used).

In the upper left corner there is a search bar, using which you can find the cloud storage by display name, provider, etc. In the upper right corner there are sorting, quick filters and filter.

Filter

Applying filter disables the quick filter.

The filter works similarly to the filters for annotation, you can create rules from properties, operators and values and group rules into groups. For more details, see the filter section. Learn more about date and time selection.

For clear all filters press Clear filters.

Supported properties for cloud storages list

Click the + button to attach a new cloud storage.

4.1.23 - Attach cloud storage

In CVAT you can use AWS-S3, Azure Blob Container and Google cloud storages to store image datasets for your tasks.

Using AWS-S3

Create AWS account

First, you need to create an AWS account, to do this, register of 5 steps following the instructions (even if you plan to use a free basic account you may need to link a credit card to verify your identity).

To learn more about the operation and benefits of AWS cloud, take a free AWS Cloud Practitioner Essentials course, which will be available after registration.

Create a bucket

After the account is created, go to console AWS-S3 and click Create bucket.

You’ll be taken to the bucket creation page. Here you have to specify the name of the bucket, region, optionally you can copy the settings of another bucket by clicking on the choose bucket button. Checkbox block all public access can be enabled as we will use access key ID and secret access key to gain access. In the following sections, you can leave the default settings and click create bucket. After you create the bucket it will appear in the list of buckets.

Create user and configure permissions

To access bucket you will need to create a user, to do this, go IAM and click add users. You need to choose AWS access type, have an access key ID and secret access key.

After pressing next button to configure permissions, you need to create a user group. To do this click create a group, input the group name and select permission policies add AmazonS3ReadOnlyAccess using the search (if you want the user you create to have write rights to bucket select AmazonS3FullAccess).

You can also add tags for the user (optional), and look again at the entered data. In the last step of creating a user, you will be provided with access key ID and secret access key, they will need to be used in CVAT when adding cloud storage.

Upload dataset

Prepare dataset

For example, let’s take The Oxford-IIIT Pet Dataset:

• Download the archive with images.

• Unpack the archive into the prepared folder and create a manifest file as described in prepare manifest file section:

  python <cvat repository>/utils/dataset_manifest/create.py --output-dir <yourfolder> <yourfolder>
  

Upload

• When the manifest file is ready, open the previously prepared bucket and click Upload:

  

• Drag the manifest file and image folder on the page and click Upload:

  

Now you can attach new cloud storage into CVAT.

Using Azure Blob Container

Create Microsoft account

First, create a Microsoft account by registering, or you can use your GitHub account to log in. After signing up for Azure, you’ll need to choose a subscription plan, you can choose a free 12-month subscription, but you’ll need to enter your credit card details to verify your identity. To learn more about Azure, read documentation.

Create a storage account

After registration, go to Azure portal. Hover over the resource groups and click create in the window that appears.

Enter a name for the group and click review + create, check the entered data and click create. After the resource group is created, go to the resource groups page and navigate to the resource group that you created. Click create for create a storage account.

• Basics

  Enter storage account name (will be used in CVAT to access your container), select a region, select performance in our case will be standard enough, select redundancy enough LRS more about redundancy. Click next to go to the advanced section.

  

• Advanced

  In the advanced section, you can change public access by disabling enable blob public access to deny anonymous access to the container. If you want to change public access you can find this switch in the configuration section of your storage account.

  After that, go to the review section, check the entered data and click create.

  

You will be reached to the deployment page after the finished, navigate to the resource by clicking on go to resource.

Create a container

Go to the containers section and create a new container. Enter the name of the container (will be used in CVAT to access your container) and select container in public access level.

SAS token

Using the SAS token, you can securely transfer access to the container to other people by preconfiguring rights, as well as the date/time of the starting and expiration of the token. To generate a SAS token, go to Shared access signature section of your storage account. Here you should enable Blob in the Allowed services, Container and Object in the Allowed resource types, Read and List in the Allowed permissions, HTTPS and HTTP in the Allowed protocols, also here you can set the date/time of the starting and expiration for the token. Click Generation SAS token. and copy SAS token (will be used in CVAT to access your container).

For personal use, you can enter the Access Key from the your storage account in the SAS Token field, access key can be found in the security + networking section. Click show keys to show the key.

Upload dataset

Prepare the dataset as in the point prepare dataset.

• When the dataset is ready, go to your container and click upload.

• Click select a files and select all images from the images folder in the upload to folder item write the name of the folder in which you want to upload images in this case “images”.

  

• Click upload, when the images are loaded you will need to upload a manifest file. When loading a manifest, you need to make sure that the relative paths specified in the manifest file match the paths to the files in the container. Click select a file and select manifest file, in order to upload file to the root of the container leave blank upload to folder field.

Now you can attach new cloud storage into CVAT.

Using Google Cloud Storage

Create Google account

First, create a Google account, go to account login page and click Create account. After, go to the Google Cloud page, click Get started, enter the required data and accept the terms of service (you’ll need credit card information to register).

Create a bucket

Your first project will be created automatically, you can see it on the cloud resource manager page. To create a bucket, go to the cloud storage page and press Create bucket. Next, enter the name of the bucket, add labels if necessary, select the type of location for example region and the location nearest to you, select storage class, when selecting access control you can enable Enforce public access prevention on this bucket (if you plan to have anonymous access to your bucket, it should be disabled) you can select Uniform or Fine-grained access control, if you need protection of your object data select protect object data type. When all the information is entered click Create to create the bucket.

Upload

Prepare the dataset as in the point prepare dataset.

To upload files, you can simply drag and drop files and folders into a browser window or use the upload folder and/or upload files.

Access permissions

To access Google Cloud Storage from CVAT you will need a Project ID you can find it by going to cloud resource manager page

Create a service account and key file

To access your bucket you need a key file and a service account. To create a service account, go to IAM & Admin/Service Accounts and press Create Service Account. Enter your account name and click Create And Continue. Select a role for example Basic/Viewer. Next, you can give access rights to the service account, to complete click Done.

The account you created will appear in the service accounts list, open it and go to the Keys tab. To create a key, click ADD and select Create new key, next you need to choose the key type JSON and select Create. The key file will be downloaded automatically.

Learn more about creating keys.

Anonymous access

To configure anonymous access, open your bucket and go to the permissions tab click ADD to add new principals. In new principals field specify allUsers, select role for example Cloud Storage Legacy/Storage Legacy Bucket Reader and press SAVE.

Now you can attach new cloud storage into CVAT.

Attach new cloud storage

After you upload the dataset and manifest file to AWS-S3, Azure Blob Container or Google Cloud Storage you will be able to attach a cloud storage. To do this, press the + button on the Cloud storages page and fill out the following form:

• Display name - the display name of the cloud storage.

• Description (optional) - description of the cloud storage, appears when you click on the ? button of an item on cloud storages page.

• Provider - choose provider of the cloud storage:

  • AWS-S3:

    • Bucket - cloud storage bucket name.

    • Authorization type:

      • Key id and secret access key pair - available on IAM to obtain an access key and a secret key, create a user using IAM and grant the appropriate rights learn more.

        • ACCESS KEY ID
        • SECRET ACCESS KEY ID

      • Anonymous access - for anonymous access, you need to enable public access to bucket.

    • Region - here you can choose a region from the list or add a new one. To get more information click on ?.

    
    

  • Azure Blob Container:

    • Container name - name of the cloud storage container.

      • Authorization type:

        • Account name and SAS token:

          • Account name - storage account name.
          • SAS token - is located in the Shared access signature section of your Storage account learn more.

        • Anonymous access - for anonymous access enable blob public access in the configuration section of your storage account. in this case, you only need the storage account name to gain anonymous access.

          • Account name - storage account name.
    
    

  • Google Cloud:

    • Bucket name - cloud storage bucket name, you can find the created bucket on the storage browser page.

    • Authorization type:

      • Key file - you can drag a key file to the area attach a file or click on the area to select the key file through the explorer. If the environment variable GOOGLE_APPLICATION_CREDENTIALS is specified for an environment with a deployed CVAT instance, then it will be used if you do not attach the key file (more about GOOGLE_APPLICATION_CREDENTIALS).

      • Anonymous access - for anonymous access, you need to enable public access to bucket.

    • Prefix - used to filter data from the bucket.

    • Project ID - you can find the created project on the cloud resource manager page, note that the project name does not match the project ID.

    • Location - here you can choose a region from the list or add a new one. To get more information click on ?.

    
    

• Manifest - the path to the manifest file on your cloud storage. You can add multiple manifest files using the Add manifest button. You can find on how to prepare dataset manifest here. If you have data on the cloud storage and don’t want to download content locally, you can mount your cloud storage as a share point according to that guide and prepare manifest for the data.

To publish the cloud storage, click submit, after which it will be available on the Cloud storages page.

Using AWS Data Exchange

Subscribe to data set

You can use AWS Data Exchange to add image datasets. For example, consider adding a set of datasets 500 Image & Metadata Free Sample. Go to browse catalog and use the search to find 500 Image & Metadata Free Sample, open the dataset page and click continue to subscribe, you will be taken to the page complete subscription request, read the information provided and click send subscription request to provider.

Export to bucket

After that, this dataset will appear in the list subscriptions. Now you need to export the dataset to Amazon S3. First, let’s create a new one bucket similar to described above. To export one of the datasets to a new bucket open it entitled data select one of the datasets, select the corresponding revision and click export to Amazon S3 (please note that if bucket and dataset are located in different regions, export fees may apply). In the window that appears, select the created bucket and click export.

Prepare manifest file

Now you need to prepare a manifest file. I used AWS cli and script for prepare manifest file. Perform the installation using the manual aws-shell, I used aws-cli 1.20.49 Python 3.7.9 Windows 10. You can configure credentials by running aws configure. You will need to enter Access Key ID and Secret Access Key as well as region.

aws configure
Access Key ID: <your Access Key ID>
Secret Access Key: <your Secret Access Key>

Copy the content of the bucket to a folder on your computer:

aws s3 cp <s3://bucket-name> <yourfolder> --recursive

After copying the files, you can create a manifest file as described in preapair manifest file section:

python <cvat repository>/utils/dataset_manifest/create.py --output-dir <yourfolder> <yourfolder>

When the manifest file is ready, you can upload it to aws s3 bucket. If you gave full write permissions when you created the user, run:

aws s3 cp <yourfolder>/manifest.jsonl <s3://bucket-name>

If you have given read-only permissions, use the download through the browser, click upload, drag the manifest file to the page and click upload.

Now you can attach new cloud storage using the dataset 500 Image & Metadata Free Sample.

4.2 - Advanced

4.2.1 - Projects page

Projects page

On this page you can create a new project, create a project from a backup, and also see the created projects.

In the upper left corner there is a search bar, using which you can find the project by project name, assignee etc. In the upper right corner there are sorting, quick filters and filter.

Filter

Applying filter disables the quick filter.

The filter works similarly to the filters for annotation, you can create rules from properties, operators and values and group rules into groups. For more details, see the filter section. Learn more about date and time selection.

For clear all filters press Clear filters.

Supported properties for projects list

Create a project

At CVAT, you can create a project containing tasks of the same type. All tasks related to the project will inherit a list of labels.

To create a project, go to the projects section by clicking on the Projects item in the top menu. On the projects page, you can see a list of projects, use a search, or create a new project by clicking on the + button and select Create New Project.

Note that the project will be created in the organization that you selected at the time of creation. Read more about organizations.

You can change: the name of the project, the list of labels (which will be used for tasks created as parts of this project) and a skeleton if it’s necessary. In advanced configuration also you can specify: a link to the issue, source and target storages. Learn more about creating a label list, creating the skeleton and attach cloud storage.

To save and open project click on Submit & Open button. Also you can click on Submit & Continue button for creating several projects in sequence

Once created, the project will appear on the projects page. To open a project, just click on it.

Here you can do the following:

1. Change the project’s title.

2. Open the Actions menu. Each button is responsible for a specific function in the Actions menu:

   • Export dataset/Import dataset - download/upload annotations or annotations and images in a specific format. More information is available in the export/import datasets section.
   • Backup project - make a backup of the project read more in the backup section.
   • Delete - remove the project and all related tasks.

3. Change issue tracker or open issue tracker if it is specified.

4. Change labels and skeleton. You can add new labels or add attributes for the existing labels in the Raw mode or the Constructor mode. You can also change the color for different labels. By clicking Setup skeleton you can create a skeleton for this project.

5. Assigned to — is used to assign a project to a person. Start typing an assignee’s name and/or choose the right person out of the dropdown list.

6. Tasks — is a list of all tasks for a particular project, with the ability to search, sort and filter for tasks in the project. Read more about search. Read more about sorting and filter It is possible to choose a subset for tasks in the project. You can use the available options (Train, Test, Validation) or set your own.

4.2.2 - Organization

Personal workspace

Your Personal workspace will display the tasks and projects you’ve created.

Create a new organization

To create an organization, open the user menu, go to Organization and click Create.

Fill in the required information to create your organization. You need to enter a Short name of the organization, which will be displayed in the menu. You can specify other fields: Full Name, Description and the organization contacts. Of them will be visible on the organization settings page.

Organization page

To go to the organization page, open the user menu, go to Organization and click Settings.

Invite members into organization

To add members, click Invite members. In the window that appears, enter the email of the user you want to add and select the role (the role defines a set of rules):

• Worker - workers have only access to tasks, projects, and jobs, assigned to them.
• Supervisor - this role allows you to create and assign jobs, tasks and projects to members of the organization.
• Maintainer - a member with this role has all the capabilities of the role supervisor, sees all the tasks and the projects created by other members of the organization, has full access to the Cloud Storages feature, and can modify members and their roles.
• Owner - a role assigned to the creator of the organization with maximum capabilities.

In addition to roles, there are groups of users that are configured on the Admin page. Read more about the roles in IAM system roles section.

After you add members, they will appear on your organization settings page, with each member listed and information about invitation details. You can change a member’s role or remove a member at any time.

The member can leave the organization on his own by clicking Leave organization on the organization settings page.

Remove organization

You can remove an organization that you created. Deleting an organization will delete all related resources (annotations, jobs, tasks, projects, cloud storages, ..). In order to remove an organization, click Remove organization, you will be asked to confirm the deletion by entering the short name of the organization.

4.2.3 - Search

There are several options how to use the search.

• Search within all fields (owner, assignee, task name, task status, task mode). To execute enter a search string in search field.
• Search for specific fields. How to perform:
  • owner: admin - all tasks created by the user who has the substring “admin” in his name
  • assignee: employee - all tasks which are assigned to a user who has the substring “employee” in his name
  • name: training - all tasks with the substring “training” in their names
  • mode: annotation or mode: interpolation - all tasks with images or videos.
  • status: annotation or status: validation or status: completed - search by status
  • id: 5 - task with id = 5.
• Multiple filters. Filters can be combined (except for the identifier) ​​using the keyword AND:
  • mode: interpolation AND owner: admin
  • mode: annotation and status: annotation

The search is case insensitive.

4.2.4 - Shape mode (advanced)

Basic operations in the mode were described in section shape mode (basics).

Occluded Occlusion is an attribute used if an object is occluded by another object or isn’t fully visible on the frame. Use Q shortcut to set the property quickly.

Example: the three cars on the figure below should be labeled as occluded.

If a frame contains too many objects and it is difficult to annotate them due to many shapes placed mostly in the same place, it makes sense to lock them. Shapes for locked objects are transparent, and it is easy to annotate new objects. Besides, you can’t change previously annotated objects by accident. Shortcut: L.

4.2.5 - Track mode (advanced)

Basic operations in the mode were described in section track mode (basics).

Shapes that were created in the track mode, have extra navigation buttons.

• These buttons help to jump to the previous/next keyframe.

  

• The button helps to jump to the initial frame and to the last keyframe.

  

You can use the Split function to split one track into two tracks:

4.2.6 - 3D Object annotation (advanced)

As well as 2D-task objects, 3D-task objects support the ability to change appearance, attributes, properties and have an action menu. Read more in objects sidebar section.

Moving an object

If you hover the cursor over a cuboid and press Shift+N, the cuboid will be cut, so you can paste it in other place (double-click to paste the cuboid).

Copying

As well as in 2D task you can copy and paste objects by Ctrl+C and Ctrl+V, but unlike 2D tasks you have to place a copied object in a 3D space (double click to paste).

Image of the projection window

You can copy or save the projection-window image by left-clicking on it and selecting a “save image as” or “copy image”.

4.2.7 - Attribute annotation mode (advanced)

Basic operations in the mode were described in section attribute annotation mode (basics).

It is possible to handle lots of objects on the same frame in the mode.

It is more convenient to annotate objects of the same type. In this case you can apply the appropriate filter. For example, the following filter will hide all objects except person: label=="Person".

To navigate between objects (person in this case), use the following buttons switch between objects in the frame on the special panel:

or shortcuts:

• Tab — go to the next object
• Shift+Tab — go to the previous object.

In order to change the zoom level, go to settings (press F3) in the workspace tab and set the value Attribute annotation mode (AAM) zoom margin in px.

4.2.8 - Annotation with rectangles

To learn more about annotation using a rectangle, see the sections:

• Shape mode (basics)
• Track mode (basics)
• Shape mode (advanced)
• Track mode (advanced)

Rotation rectangle

To rotate the rectangle, pull on the rotation point. Rotation is done around the center of the rectangle. To rotate at a fixed angle (multiple of 15 degrees), hold shift. In the process of rotation, you can see the angle of rotation.

Annotation with rectangle by 4 points

It is an efficient method of bounding box annotation, proposed here. Before starting, you need to make sure that the drawing method by 4 points is selected.

Press Shape or Track for entering drawing mode. Click on four extreme points: the top, bottom, left- and right-most physical points on the object. Drawing will be automatically completed right after clicking the fourth point. Press Esc to cancel editing.

4.2.9 - Annotation with polygons

4.2.9.1 - Manual drawing

It is used for semantic / instance segmentation.

Before starting, you need to select Polygon on the controls sidebar and choose the correct Label.

• Click Shape to enter drawing mode. There are two ways to draw a polygon: either create points by clicking or by dragging the mouse on the screen while holding Shift.

Clicking points	Holding Shift+Dragging

• When Shift isn’t pressed, you can zoom in/out (when scrolling the mouse wheel) and move (when clicking the mouse wheel and moving the mouse), you can also delete the previous point by right-clicking on it.
• You can use the Selected opacity slider in the Objects sidebar to change the opacity of the polygon. You can read more in the Objects sidebar section.
• Press N again or click the Done button on the top panel for completing the shape.
• After creating the polygon, you can move the points or delete them by right-clicking and selecting Delete point or clicking with pressed Alt key in the context menu.

4.2.9.2 - Drawing using automatic borders

You can use auto borders when drawing a polygon. Using automatic borders allows you to automatically trace the outline of polygons existing in the annotation.

• To do this, go to settings -> workspace tab and enable Automatic Bordering or press Ctrl while drawing a polygon.

  

• Start drawing / editing a polygon.

• Points of other shapes will be highlighted, which means that the polygon can be attached to them.

• Define the part of the polygon path that you want to repeat.

  

• Click on the first point of the contour part.

  

• Then click on any point located on part of the path. The selected point will be highlighted in purple.

  

• Click on the last point and the outline to this point will be built automatically.

  

Besides, you can set a fixed number of points in the Number of points field, then drawing will be stopped automatically. To enable dragging you should right-click inside the polygon and choose Switch pinned property.

Below you can see results with opacity and black stroke:

If you need to annotate small objects, increase Image Quality to 95 in Create task dialog for your convenience.

4.2.9.3 - Edit polygon

To edit a polygon you have to click on it while holding Shift, it will open the polygon editor.

• In the editor you can create new points or delete part of a polygon by closing the line on another point.

• When Intelligent polygon cropping option is activated in the settings, CVAT considers two criteria to decide which part of a polygon should be cut off during automatic editing.

  • The first criteria is a number of cut points.
  • The second criteria is a length of a cut curve.

  If both criteria recommend to cut the same part, algorithm works automatically, and if not, a user has to make the decision. If you want to choose manually which part of a polygon should be cut off, disable Intelligent polygon cropping in the settings. In this case after closing the polygon, you can select the part of the polygon you want to leave.

  

• You can press Esc to cancel editing.

  

4.2.9.4 - Track mode with polygons

Polygons in the track mode allow you to mark moving objects more accurately other than using a rectangle (Tracking mode (basic); Tracking mode (advanced)).

1. To create a polygon in the track mode, click the Track button.

   

2. Create a polygon the same way as in the case of Annotation with polygons. Press N or click the Done button on the top panel to complete the polygon.

3. Pay attention to the fact that the created polygon has a starting point and a direction, these elements are important for annotation of the following frames.

4. After going a few frames forward press Shift+N, the old polygon will disappear and you can create a new polygon. The new starting point should match the starting point of the previously created polygon (in this example, the top of the left mirror). The direction must also match (in this example, clockwise). After creating the polygon, press N and the intermediate frames will be interpolated automatically.

   

5. If you need to change the starting point, right-click on the desired point and select Set starting point. To change the direction, right-click on the desired point and select switch orientation.

   

There is no need to redraw the polygon every time using Shift+N, instead you can simply move the points or edit a part of the polygon by pressing Shift+Click.

4.2.9.5 - Creating masks

Cutting holes in polygons

Currently, CVAT does not support cutting transparent holes in polygons. However, it is poissble to generate holes in exported instance and class masks. To do this, one needs to define a background class in the task and draw holes with it as additional shapes above the shapes needed to have holes:

The editor window:

Remember to use z-axis ordering for shapes by [-] and [+, =] keys.

Exported masks:

Notice that it is currently impossible to have a single instance number for internal shapes (they will be merged into the largest one and then covered by “holes”).

Creating masks

There are several formats in CVAT that can be used to export masks:

• Segmentation Mask (PASCAL VOC masks)
• CamVid
• MOTS
• ICDAR
• COCO (RLE-encoded instance masks, guide)
• TFRecord (over Datumaro, guide):
• Datumaro

An example of exported masks (in the Segmentation Mask format):

Important notices:

• Both boxes and polygons are converted into masks
• Grouped objects are considered as a single instance and exported as a single mask (label and attributes are taken from the largest object in the group)

Class colors

All the labels have associated colors, which are used in the generated masks. These colors can be changed in the task label properties:

Label colors are also displayed in the annotation window on the right panel, where you can show or hide specific labels (only the presented labels are displayed):

A background class can be:

• A default class, which is implicitly-added, of black color (RGB 0, 0, 0)
• background class with any color (has a priority, name is case-insensitive)
• Any class of black color (RGB 0, 0, 0)

To change background color in generated masks (default is black), change background class color to the desired one.

4.2.10 - Annotation with polylines

It is used for road markup annotation etc.

Before starting, you need to select the Polyline. You can set a fixed number of points in the Number of points field, then drawing will be stopped automatically.

Click Shape to enter drawing mode. There are two ways to draw a polyline — you either create points by clicking or by dragging a mouse on the screen while holding Shift. When Shift isn’t pressed, you can zoom in/out (when scrolling the mouse wheel) and move (when clicking the mouse wheel and moving the mouse), you can delete previous points by right-clicking on it. Press N again or click the Done button on the top panel to complete the shape. You can delete a point by clicking on it with pressed Ctrl or right-clicking on a point and selecting Delete point. Click with pressed Shift will open a polyline editor. There you can create new points(by clicking or dragging) or delete part of a polygon closing the red line on another point. Press Esc to cancel editing.

4.2.11 - Annotation with points

4.2.11.1 - Points in shape mode

It is used for face, landmarks annotation etc.

Before you start you need to select the Points. If necessary you can set a fixed number of points in the Number of points field, then drawing will be stopped automatically.

Click Shape to entering the drawing mode. Now you can start annotation of the necessary area. Points are automatically grouped — all points will be considered linked between each start and finish. Press N again or click the Done button on the top panel to finish marking the area. You can delete a point by clicking with pressed Ctrl or right-clicking on a point and selecting Delete point. Clicking with pressed Shift will open the points shape editor. There you can add new points into an existing shape. You can zoom in/out (when scrolling the mouse wheel) and move (when clicking the mouse wheel and moving the mouse) while drawing. You can drag an object after it has been drawn and change the position of individual points after finishing an object.

4.2.11.2 - Linear interpolation with one point

You can use linear interpolation for points to annotate a moving object:

1. Before you start, select the Points.

2. Linear interpolation works only with one point, so you need to set Number of points to 1.

3. After that select the Track.

   

4. Click Track to enter the drawing mode left-click to create a point and after that shape will be automatically completed.

   

5. Move forward a few frames and move the point to the desired position, this way you will create a keyframe and intermediate frames will be drawn automatically. You can work with this object as with an interpolated track: you can hide it using the Outside, move around keyframes, etc.

   

6. This way you’ll get linear interpolation using the Points.

   

4.2.12 - Annotation with ellipses

It is used for road sign annotation etc.

First of all you need to select the ellipse on the controls sidebar.

Choose a Label and click Shape or Track to start drawing. An ellipse can be created the same way as a rectangle, you need to specify two opposite points, and the ellipse will be inscribed in an imaginary rectangle. Press N or click the Done button on the top panel to complete the shape.

You can rotate ellipses using a rotation point in the same way as rectangles.

4.2.13 - Annotation with cuboids

It is used to annotate 3 dimensional objects such as cars, boxes, etc… Currently the feature supports one point perspective and has the constraint where the vertical edges are exactly parallel to the sides.

4.2.13.1 - Creating the cuboid

Before you start, you have to make sure that Cuboid is selected and choose a drawing method ”from rectangle” or “by 4 points”.

Drawing cuboid by 4 points

Choose a drawing method “by 4 points” and click Shape to enter the drawing mode. There are many ways to draw a cuboid. You can draw the cuboid by placing 4 points, after that the drawing will be completed automatically. The first 3 points determine the plane of the cuboid while the last point determines the depth of that plane. For the first 3 points, it is recommended to only draw the 2 closest side faces, as well as the top and bottom face.

A few examples:

Drawing cuboid from rectangle

Choose a drawing method “from rectangle” and click Shape to enter the drawing mode. When you draw using the rectangle method, you must select the frontal plane of the object using the bounding box. The depth and perspective of the resulting cuboid can be edited.

Example:

4.2.13.2 - Editing the cuboid

The cuboid can be edited in multiple ways: by dragging points, by dragging certain faces or by dragging planes. First notice that there is a face that is painted with gray lines only, let us call it the front face.

You can move the cuboid by simply dragging the shape behind the front face. The cuboid can be extended by dragging on the point in the middle of the edges. The cuboid can also be extended up and down by dragging the point at the vertices.

To draw with perspective effects it should be assumed that the front face is the closest to the camera. To begin simply drag the points on the vertices that are not on the gray/front face while holding Shift. The cuboid can then be edited as usual.

If you wish to reset perspective effects, you may right click on the cuboid, and select Reset perspective to return to a regular cuboid.

The location of the gray face can be swapped with the adjacent visible side face. You can do it by right clicking on the cuboid and selecting Switch perspective orientation. Note that this will also reset the perspective effects.

Certain faces of the cuboid can also be edited, these faces are: the left, right and dorsal faces, relative to the gray face. Simply drag the faces to move them independently from the rest of the cuboid.

You can also use cuboids in track mode, similar to rectangles in track mode (basics and advanced) or Track mode with polygons

4.2.14 - Annotation with skeletons

Skeletons should be used as annotations templates when you need to annotate complex objects sharing the same structure (e.g. human pose estimation, facial landmarks, etc.). A skeleton consist of any number of points (also called as elements), joined or not joined by edges. Any point itself is considered like an individual object with its own attributes and properties (like color, occluded, outside, etc). At the same time a skeleton point can exist only within the parent skeleton.

Any skeleton elements can be hidden (by marking them outside) if necessary (for example if a part is out of a frame). Currently there are two formats which support exporting skeletons: CVAT & COCO.

4.2.14.1 - Creating the skeleton

Initial skeleton setup

Unlike other CVAT objects, to start annotating using skeletons, first of all you need to setup a skeleton. You can do that in the label configurator during creating a task/project, or later in created instances.

So, start by clicking Setup skeleton option:

Below the regular label form where you need to add a name, and setup attributes if necessary, you will see a drawing area with some buttons aside:

• PUT AN IMAGE AS A BACKGROUND - is a helpful feature you can use to draw a skeleton template easier, seeing an example - object you need to annotate in the future.
• PUT NEW SKELETON POINTS - is activated by default. It is a mode where you can add new skeleton points clicking the drawing area.
• DRAW AN EDGE BETWEEN TWO POINTS - in this mode you can add an edge, clicking any two points, which are not joined yet.
• REMOVE A DRAWN SKELETON POINTS - in this mode clicking a point will remove the point and all attached edges. You can also remove an edge only, it will be highligted as red on hover.
• DOWNLOAD DRAWN TEMPLATE AS AN .SVG - you can download setup configuration to use it in future
• UPLOAD A TEMPLATE FROM AN .SVG FILE - you can upload previously downloaded configuration

Let’s draw an exampe skeleton - star. After the skeleton is drawn, you can setup each its point. Just hover the point, do right mouse click and click Configure:

Here you can setup a point name, its color and attributes if necessary like for a regular CVAT label:

Press Done button to finish editing the point. Press Continue button to save the skeleton. Continue creating a task/project in a regular way.

For an existing task/project you are not allowed to change a skeleton configuration for now. You can copy/insert skeletons configuration using Raw tab of the label configurator.

Drawing a skeleton from rectangle

In opened job go to left sidebar and find Draw new skeleton control, hover it:

If the control is absent, be sure you have setup at least one skeleton in the corresponding task/project. In a pop-up dropdown you can select between a skeleton Shape and a skeleton Track, depends on your task. Draw a skeleton as a regular bounding box, clicking two points on a canvas:

Well done, you’ve just created the first skeleton.

4.2.14.2 - Editing the skeleton

Editing skeletons on the canvas

A drawn skeleton is wrapped by a bounding box for a user convenience. Using this wrapper the user can edit the skeleton as a regular bounding box, by dragging, resizing, or rotating:

Moreover, each the skeleton point can be dragged itself. After dragging, the wrapping bounding box is adjusted automatically, other points are not affected:

You can use Shortcuts on both a skeleton itself and its elements.

• Hover the mouse cursor over the bounding box to apply a shortcut on the whole skeleton (like lock, occluded, pinned, keyframe and outside for skeleton tracks)
• Hover the mouse cursor over one of skeleton points to apply a shortcut to this point (the same shortcuts list, but outside is available also for a skeleton shape elements)

Editing skeletons on the sidebar

Using the sidebar is another way to setup skeleton properties, and attributes. It works a similar way, like for other kinds of objects supported by CVAT, but with some changes:

• A user is not allowed to switch a skeleton label
• Outside property is always available for skeleton elements (it does not matter if they are tracks or not)
• Additional collapse is available for a user, to see a list of skeleton parts

4.2.15 - Annotation with brush tool

With a brush tool, you can create masks for disjoint objects, that have multiple parts, such as a house hiding behind trees, a car behind a pedestrian, or a pillar behind a traffic sign. The brush tool has several modes, for example: erase pixels, change brush shapes, and polygon-to-mask mode.

Use brush tool for Semantic (Panoptic) and Instance Image Segmentation tasks.
For more information about segmentation masks in CVAT, see Creating masks.

See:

• Brush tool menu
• Annotation with brush
• Annotation with polygon-to-mask
• Remove underlying pixels
• AI Tools
• Import and export

Brush tool menu

The brush tool menu appears on the top of the screen after you click Shape:

It has the following elements:

Element	Description
	Save mask saves the created mask. The saved mask will appear on the object sidebar
	Save mask and continue adds a new mask to the object sidebar and allows you to draw a new one immediately.
	Brush adds new mask/ new regions to the previously added mask).
	Eraser removes part of the mask.
	Polygon selection tool. Selection will become a mask.
	Remove polygon selection substracts part of the polygon selection.
	Brush size in pixels. Note: Visible only when Brush or Eraser are selected.
	Brush shape with two options: circle and square. Note: Visible only when Brush or Eraser are selected.
	Remove underlying pixels. When you are drawing or editing a mask with this tool, pixels on other masks that are located at the same positions as the pixels of the current mask are deleted.
	Label that will be assigned to the newly created mask
	Move. Click and hold to move the menu bar to the other place on the screen

Annotation with brush

Prerequisites: Create task, upload the image, add labels, and open the workspace.

Do the following:

1. From the controls sidebar, select Brush .

2. In the Draw new mask menu, select label for your mask, and click Shape.
   The Brush tool will be selected by default.

   

3. With the brush, draw a mask on the object you want to label.
   To erase selection, use Eraser

   

4. After you applied the mask, on the top menu bar click Save mask
   to finish the process (or N on the keyboard).

5. Added object will appear on the objects sidebar.

To add the next object, repeat steps 1 to 5. All added objects will be visible on the image and the objects sidebar.

To save the job with all added objects, on the top menu click Save .

Annotation with polygon-to-mask

Prerequisites: Create task, upload the image,
add labels, and open the workspace.

Do the following:

1. From the controls sidebar, select Brush .

2. In the Draw new mask menu, select label for your mask, and click Shape.

   

3. In the brush tool menu, select Polygon .

4. With the Polygon tool, draw a mask for the object you want to label.
   To correct selection, use Remove polygon selection .

5. Use Save mask (or N on the keyboard)
   to switch between add/remove polygon tools:

   

6. After you added the polygon selection, on the top menu bar click Save mask
   to finish the process (or N on the keyboard).

7. Click Save mask again (or N on the keyboard).
   The added object will appear on the objects sidebar.

To add the next object, repeat steps 1 to 5.

All added objects will be visible on the image and the objects sidebar.

To save the job with all added objects, on the top menu click Save .

Remove underlying pixels

Use Remove underlying pixels tool when you want to add a mask and simultaneously delete the pixels of
other masks that are located at the same positions. It is a highly useful feature to avoid meticulous drawing edges twice between two different objects.

AI Tools

You can convert AI tool masks to polygons. To do this, use the following AI tool menu:

1. Go to the Detectors tab.
2. Switch toggle Masks to polygons to the right.
3. Add source and destination labels from the drop-down lists.
4. Click Annotate.

Import and export

For export, see Export dataset

Import follows the general import dataset procedure, with the additional option of converting masks to polygons.

Note: This option is available for formats that work with masks only.

To use it, when uploading the dataset, switch the Convert masks to polygon toggle to the right:

4.2.16 - Annotation with tags

It is used to annotate frames, tags are not displayed in the workspace. Before you start, open the drop-down list in the top panel and select Tag annotation.

The objects sidebar will be replaced with a special panel for working with tags. Here you can select a label for a tag and add it by clicking on the Plus button. You can also customize hotkeys for each label.

If you need to use only one label for one frame, then enable the Automatically go to the next frame checkbox, then after you add the tag the frame will automatically switch to the next.

Tags will be shown in the top left corner of the canvas. You can show/hide them in the settings.

4.2.17 - Models

To deploy the models, you will need to install the necessary components using Semi-automatic and Automatic Annotation guide. To learn how to deploy the model, read Serverless tutorial.

The Models page contains a list of deep learning (DL) models deployed for semi-automatic and automatic annotation. To open the Models page, click the Models button on the navigation bar. The list of models is presented in the form of a table. The parameters indicated for each model are the following:

• Framework the model is based on
• model Name
• model Type:
  • detector - used for automatic annotation (available in detectors and automatic annotation)
  • interactor - used for semi-automatic shape annotation (available in interactors)
  • tracker - used for semi-automatic track annotation (available in trackers)
  • reid - used to combine individual objects into a track (available in automatic annotation)
• Description - brief description of the model
• Labels - list of the supported labels (only for the models of the detectors type)

4.2.18 - AI Tools

The tool is designed for semi-automatic and automatic annotation using DL models. The tool is available only if there is a corresponding model. For more details about DL models read the Models section.

Interactors

Interactors are used to create a polygon semi-automatically. Supported DL models are not bound to the label and can be used for any objects. To create a polygon usually you need to use regular or positive points. For some kinds of segmentation negative points are available. Positive points are the points related to the object. Negative points should be placed outside the boundary of the object. In most cases specifying positive points alone is enough to build a polygon. A list of available out-of-the-box interactors is placed below.

• Before you start, select the magic wand on the controls sidebar and go to the Interactors tab. Then select a label for the polygon and a required DL model. To view help about each of the models, you can click the Question mark icon.

  

• Click Interact to enter the interaction mode. Depending on the selected model, the method of markup will also differ. Now you can place positive and/or negative points. The IOG model also uses a rectangle. Left click creates a positive point and right click creates a negative point. After placing the required number of points (the number is different depending on the model), the request will be sent to the server and when the process is complete a polygon will be created. If you are not satisfied with the result, you can set additional points or remove points. To delete a point, hover over the point you want to delete, if the point can be deleted, it will enlarge and the cursor will turn into a cross, then left-click on the point. If you want to postpone the request and create a few more points, hold down Ctrl and continue (the Block button on the top panel will turn blue), the request will be sent after the key is released.

  

• In the process of drawing, you can select the number of points in the polygon using the switch.

  

• You can use the Selected opacity slider in the Objects sidebar to change the opacity of the polygon. You can read more in the Objects sidebar section.

• To finish interaction, click on the Done button on the top panel or press N on your keyboard.

• When the object is finished, you can edit it like a polygon. You can read about editing polygons in the Annotation with polygons section.

Deep extreme cut (DEXTR)

This is an optimized version of the original model, introduced at the end of 2017. It uses the information about extreme points of an object to get its mask. The mask then converted to a polygon. For now this is the fastest interactor on CPU.

Feature backpropagating refinement scheme (f-BRS)

The model allows to get a mask for an object using positive points (should be left-clicked on the foreground), and negative points (should be right-clicked on the background, if necessary). It is recommended to run the model on GPU, if possible.

High Resolution Net (HRNet)

The model allows to get a mask for an object using positive points (should be left-clicked on the foreground), and negative points (should be right-clicked on the background, if necessary). It is recommended to run the model on GPU, if possible.

Inside-Outside-Guidance

The model uses a bounding box and inside/outside points to create a mask. First of all, you need to create a bounding box, wrapping the object. Then you need to use positive and negative points to say the model where is a foreground, and where is a background. Negative points are optional.

Detectors

Detectors are used to automatically annotate one frame. Supported DL models are suitable only for certain labels.

• Before you start, click the magic wand on the controls sidebar and select the Detectors tab. You need to match the labels of the DL model (left column) with the labels in your task (right column). Then click Annotate.

  

• Some of models supports attributes annotation (like facial emotions, for example: serverless/openvino/omz/intel/face-detection-0205). In this case you can also match attributes of the DL model with the attributes of a CVAT label.

  

• This action will automatically annotates one frame. In the Automatic annotation section you can read how to make automatic annotation of all frames.

Mask RCNN

The model generates polygons for each instance of an object in the image.

Faster RCNN

The model generates bounding boxes for each instance of an object in the image. In this model, RPN and Fast R-CNN are combined into a single network.

Trackers

Trackers are used to automatically annotate an object using bounding box. Supported DL models are not bound to the label and can be used for any objects.

• Before you start, select the magic wand on the controls sidebar and go to the Trackers tab. Then select a Label and Tracker for the object and click Track. Then annotate the desired objects with the bounding box in the first frame.

  

• All annotated objects will be automatically tracked when you move to the next frame. For tracking, use Next button on the top panel or the F button to move on to the next frame.

  

• You can enable/disable tracking using tracker switcher on sidebar.

  

• Trackable objects have indication on canvas with a model indication.

  

• You can monitoring the process by the messages appearing at the top. If you change one or more objects, before moving to the next frame, you will see a message that the objects states initialization is taking place. The objects that you do not change are already on the server and therefore do not require initialization. After the objects are initialized, tracking will occur.

  

SiamMask

Fast online Object Tracking and Segmentation. Tracker is able to track different objects in one server request. Trackable object will be tracked automatically if the previous frame was a latest keyframe for the object. Have tracker indication on canvas. SiamMask tracker supported CUDA.

If you plan to track simple non-overlapping objects consider using fast client-side TrackerMIL from OpenCV.

4.2.19 - OpenCV tools

The tool based on Open CV Computer Vision library which is an open-source product that includes many CV algorithms. Some of these algorithms can be used to simplify the annotation process.

First step to work with OpenCV is to load it into CVAT. Click on the toolbar icon, library will be downloaded automatically.

Once it is loaded, the tool’s functionality will be available.

Intelligent scissors

Intelligent scissors is an CV method of creating a polygon by placing points with automatic drawing of a line between them. The distance between the adjacent points is limited by the threshold of action, displayed as a red square which is tied to the cursor.

• First, select the label and then click on the intelligent scissors button.

  

• Create the first point on the boundary of the allocated object. You will see a line repeating the outline of the object.

• Place the second point, so that the previous point is within the restrictive threshold. After that a line repeating the object boundary will be automatically created between the points.

  

  To increase or lower the action threshold, hold Ctrl and scroll the mouse wheel. Increasing action threshold will affect the performance. During the drawing process you can remove the last point by clicking on it with the left mouse button.

• You can also create a boundary manually (like when creating a polygon) by temporarily disabling the automatic line creation. To do that, switch blocking on by pressing Ctrl.

• In the process of drawing, you can select the number of points in the polygon using the switch.

  

• You can use the Selected opacity slider in the Objects sidebar to change the opacity of the polygon. You can read more in the Objects sidebar section.

• Once all the points are placed, you can complete the creation of the object by clicking on the Done button on the top panel or press N on your keyboard. As a result, a polygon will be created (read more about the polygons in the annotation with polygons).

Histogram Equalization

Histogram equalization is an CV method that improves contrast in an image in order to stretch out the intensity range. This method usually increases the global contrast of images when its usable data is represented by close contrast values. It is useful in images with backgrounds and foregrounds that are both bright or both dark.

• First, select the image tab and then click on histogram equalization button.

  

• Then contrast of current frame will be improved. If you change frame, it will be equalized too. You can disable equalization by clicking histogram equalization button again.

  

TrackerMIL

Trackers are used to automatically annotate an object on video. The TrackerMIL model is not bound to labels and can be used for any object.

• Before you start, select the OpenCV tools on the controls sidebar and go to the Trackers tab. Then select a Label and Tracker for the object and click Track. Then annotate the desired objects with the bounding box in the first frame.

  

• All annotated objects will be automatically tracked when you move to the next frame. For tracking, use Next button on the top panel or the F button to move on to the next frame.

  

• You can enable/disable tracking using tracker switcher on sidebar.

  

• Trackable objects have indication on canvas with a model indication.

  

• You can follow the tracking by the messages appearing at the top.

  

4.2.20 - Automatic annotation

Automatic Annotation is used for creating preliminary annotations. To use Automatic Annotation you need a DL model that can be deployed by a CVAT administrator. You can find the list of available models in the Models section.

1. To launch automatic annotation, you should open the dashboard and find a task which you want to annotate. Then click the Actions button and choose option Automatic Annotation from the dropdown menu.

   

2. In the dialog window select a model you need. DL models are created for specific labels, e.g. the Crossroad model was taught using footage from cameras located above the highway and it is best to use this model for the tasks with similar camera angles. If it’s necessary select the Clean old annotations checkbox. Adjust the labels so that the task labels will correspond to the labels of the DL model. For example, let’s consider a task where you have to annotate labels “car” and “person”. You should connect the “person” label from the model to the “person” label in the task. As for the “car” label, you should choose the most fitting label available in the model - the “vehicle” label. If the chosen model supports automatic attributes detection (like facial expressions, for example: serverless/openvino/omz/intel/face-detection-0205), you can also map attributes between the DL model and your task. The task requires to annotate cars only and choosing the “vehicle” label implies annotation of all vehicles, in this case using auto annotation will help you complete the task faster. Click Submit to begin the automatic annotation process.

   

3. At runtime - you can see the percentage of completion. You can cancel the automatic annotation by clicking on the Cancelbutton.

   

4. The end result of an automatic annotation is an annotation with separate rectangles (or other shapes)

   

5. You can combine separate bounding boxes into tracks using the Person reidentification model. To do this, click on the automatic annotation item in the action menu again and select the model of the ReID type (in this case the Person reidentification model). You can set the following parameters:

   • Model Threshold is a maximum cosine distance between objects’ embeddings.
   • Maximum distance defines a maximum radius that an object can diverge between adjacent frames.

   

6. You can remove false positives and edit tracks using Split and Merge functions.

   

4.2.21 - Backup Task and Project

Overview

In CVAT you can backup tasks and projects. This can be used to backup a task or project on your PC or to transfer to another server.

Create backup

To backup a task or project, open the action menu and select Backup Task or Backup Project.

You can backup a project or a task locally on your PC or using an attached cloud storage.

(Optional) Specify the name in the Custom name text field for backup, otherwise the file of backup name will be given by the mask project_<project_name>_backup_<date>_<time>.zip for the projects and task_<task_name>_backup_<date>_<time>.zip for the tasks.

If you want to save a backup to a specific attached cloud storage, you should additionally turn off the switch Use default settings, select the Cloud storage value in the Target storage and select this storage in the list of the attached cloud storages.

Create backup APIs

• endpoints:
  • /tasks/{id}/backup
  • /projects/{id}/backup
• method: GET
• responses: 202, 201 with zip archive payload

Upload backup APIs

• endpoints:
  • /api/tasks/backup
  • /api/projects/backup
• method: POST
• Content-Type: multipart/form-data
• responses: 202, 201 with json payload

Create from backup

To create a task or project from a backup, go to the tasks or projects page, click the Create from backup button and select the archive you need.

As a result, you’ll get a task containing data, parameters, and annotations of the previously exported task.

Backup file structure

As a result, you’ll get a zip archive containing data, task or project and task specification and annotations with the following structure:

• Task Backup Structure
• Project Backup Structure

    .
    ├── data
    │   └── {user uploaded data}
    ├── task.json
    └── annotations.json
  

    .
    ├── task_{id}
    │   ├── data
    │   │   └── {user uploaded data}
    │   ├── task.json
    │   └── annotations.json
    └── project.json
  

4.2.22 - Frame deleting

Delete frame

You can delete current frame from a task. This frame will not be presented either in the UI or in the exported annotation. Thus, it is possible to mark corrupted frames that are not subject to annotation.

1. Go to the Job annotation view and than click on the Delete frame button.

   

2. After that you will be asked to confirm frame deleting. Important note: all annotations from that frame will be deleted, unsaved annotations will be saved and the frame will be invisible in the annotation view (Until you make it visible in the settings). If there is some overlap in the task and the deleted frame falls within this interval, then this will cause this frame to become unavailable in another job as well.

3. When you delete a frame in a job with tracks, you may need adjust some tracks manually. Common adjustments are:

   • Add keyframes at the edges of the deleted interval in order for the interpolation to look correct;
   • Move keyframe start or end keyframe to the correct side of the deleted interval.

Configurate deleted frames visability and navigation

If you need to enable showing the deleted frames, you can do it in the settings.

1. Go to the settings and chose Player settings.

   

2. Click on the Show deleted frames checkbox. And close the settings dialog.

   

3. Then you will be able to navigate through deleted frames. But annotation tools will be unavailable. Deleted frames differ in the corresponding overlay.

4. There are view ways to navigate through deleted frames without enabling this option:

   • Go to the frame via direct navigation methods: navigation slider or frame input field,
   • Go to the frame via direct link.

5. Navigation with step will not count deleted frames.

Restore deleted frame

You can also restore deleted frames in the task.

1. Turn on deleted frames visability, as it was told in the previous part, and go to the deleted frame you want to restore.

   

2. Click on the Restore icon. The frame will be restored immediately.

4.2.23 - Export/import datasets and upload annotation

Export dataset

You can export a dataset to a project, task or job.

1. To download the latest annotations, you have to save all changes first. Click the Save button. There is a Ctrl+S shortcut to save annotations quickly.

   

2. After that, click the Menu button. Exporting and importing of task and project datasets takes place through the Action menu.

3. Press the Export task dataset button.

   

4. Choose the format for exporting the dataset. Exporting and importing is available in:

   • Standard CVAT formats:

     • CVAT for video choose if the task is created in interpolation mode.

     • CVAT for images choose if a task is created in annotation mode.

       
       

   • And also in formats from the list of annotation formats supported by CVAT.

   • For 3D tasks, the following formats are available:

     • Kitti Raw Format 1.0
     • Sly Point Cloud Format 1.0 - Supervisely Point Cloud dataset
   
   

5. To download images with the dataset, enable the Save images option.

6. (Optional) To name the resulting archive, use the Custom name field.

7. You can choose a storage for dataset export by selecting a target storage Local or Cloud storage. The default settings are the settings that had been selected when the project was created (for example, if you specified a local storage when you created the project, then by default, you will be prompted to export the dataset to your PC). You can find out the default value by hovering the mouse over the ?. Learn more about attach cloud storage.

Import dataset

You can import dataset only to a project. In this case, the data will be split into subsets. To import a dataset, do the following on the Project page:

• Open the Actions menu.
• Press the Import dataset button.
• Select the dataset format (if you did not specify a custom name during export, the format will be in the archive name).
• Drag the file to the file upload area or click on the upload area to select the file through the explorer.

• You can also import a dataset from an attached cloud storage. Here you should select the annotation format, then select a cloud storage from the list or use default settings if you have already specified required cloud storage for task or project and specify a zip archive to the text field File name.

During the import process, you will be able to track the progress of the import.

Upload annotations

In the task or job you can upload an annotation. For this select the item Upload annotation in the menu Action of the task or in the job Menu on the Top panel select the format in which you plan to upload the annotation and select the annotation file or archive via explorer.

Or you can also use the attached cloud storage to upload the annotation file.

4.2.24 - Formats

CVAT supported the following formats:

• CVAT
• Datumaro
• LabelMe
• MOT
• MOTS
• COCO
• PASCAL VOC and mask
• YOLO
• TF detection API
• ImageNet
• CamVid
• WIDER Face
• VGGFace2
• Market-1501
• ICDAR13/15
• Open Images
• Cityscapes
• KITTI
• LFW

4.2.24.1 -

CVAT

This is the native CVAT annotation format. It supports all CVAT annotations features, so it can be used to make data backups.

• supported annotations CVAT for Images: Rectangles, Polygons, Polylines, Points, Cuboids, Skeletons, Tags, Tracks

• supported annotations CVAT for Videos: Rectangles, Polygons, Polylines, Points, Cuboids, Skeletons, Tracks

• attributes are supported

• Format specification

CVAT for images export

Downloaded file: a ZIP file of the following structure:

taskname.zip/
├── images/
|   ├── img1.png
|   └── img2.jpg
└── annotations.xml

• tracks are split by frames

CVAT for videos export

Downloaded file: a ZIP file of the following structure:

taskname.zip/
├── images/
|   ├── frame_000000.png
|   └── frame_000001.png
└── annotations.xml

• shapes are exported as single-frame tracks

CVAT loader

Uploaded file: an XML file or a ZIP file of the structures above

4.2.24.2 -

Datumaro format

Datumaro is a tool, which can help with complex dataset and annotation transformations, format conversions, dataset statistics, merging, custom formats etc. It is used as a provider of dataset support in CVAT, so basically, everything possible in CVAT is possible in Datumaro too, but Datumaro can offer dataset operations.

• supported annotations: any 2D shapes, labels
• supported attributes: any

Import annotations in Datumaro format

Uploaded file: a zip archive of the following structure:

<archive_name>.zip/
└── annotations/
    ├── subset1.json # fully description of classes and all dataset items
    └── subset2.json # fully description of classes and all dataset items

JSON annotations files in the annotations directory should have similar structure:

{
  "info": {},
  "categories": {
    "label": {
      "labels": [
        {
          "name": "label_0",
          "parent": "",
          "attributes": []
        },
        {
          "name": "label_1",
          "parent": "",
          "attributes": []
        }
      ],
      "attributes": []
    }
  },
  "items": [
    {
      "id": "img1",
      "annotations": [
        {
          "id": 0,
          "type": "polygon",
          "attributes": {},
          "group": 0,
          "label_id": 1,
          "points": [1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0],
          "z_order": 0
        },
        {
          "id": 1,
          "type": "bbox",
          "attributes": {},
          "group": 1,
          "label_id": 0,
          "z_order": 0,
          "bbox": [1.0, 2.0, 3.0, 4.0]
        },
        {
          "id": 2,
          "type": "mask",
          "attributes": {},
          "group": 1,
          "label_id": 0,
          "rle": {
            "counts": "d0d0:F\\0",
            "size": [10, 10]
          },
          "z_order": 0
        }
      ]
    }
  ]
}

Export annotations in Datumaro format

Downloaded file: a zip archive of the following structure:

taskname.zip/
├── annotations/
│   └── default.json # fully description of classes and all dataset items
└── images/ # if the option `save images` was selected
    └── default
        ├── image1.jpg
        ├── image2.jpg
        ├── ...

4.2.24.3 -

LabelMe

LabelMe export

Downloaded file: a zip archive of the following structure:

taskname.zip/
├── img1.jpg
└── img1.xml

• supported annotations: Rectangles, Polygons (with attributes)

LabelMe import

Uploaded file: a zip archive of the following structure:

taskname.zip/
├── Masks/
|   ├── img1_mask1.png
|   └── img1_mask2.png
├── img1.xml
├── img2.xml
└── img3.xml

• supported annotations: Rectangles, Polygons, Masks (as polygons)

4.2.24.4 -

MOT sequence

MOT export

Downloaded file: a zip archive of the following structure:

taskname.zip/
├── img1/
|   ├── image1.jpg
|   └── image2.jpg
└── gt/
    ├── labels.txt
    └── gt.txt

# labels.txt
cat
dog
person
...

# gt.txt
# frame_id, track_id, x, y, w, h, "not ignored", class_id, visibility, <skipped>
1,1,1363,569,103,241,1,1,0.86014
...

• supported annotations: Rectangle shapes and tracks
• supported attributes: visibility (number), ignored (checkbox)

MOT import

Uploaded file: a zip archive of the structure above or:

taskname.zip/
├── labels.txt # optional, mandatory for non-official labels
└── gt.txt

• supported annotations: Rectangle tracks

4.2.24.5 -

MOTS PNG

MOTS PNG export

Downloaded file: a zip archive of the following structure:

taskname.zip/
└── <any_subset_name>/
    |   images/
    |   ├── image1.jpg
    |   └── image2.jpg
    └── instances/
        ├── labels.txt
        ├── image1.png
        └── image2.png

# labels.txt
cat
dog
person
...

• supported annotations: Rectangle and Polygon tracks

MOTS PNG import

Uploaded file: a zip archive of the structure above

• supported annotations: Polygon tracks

4.2.24.6 -

MS COCO Object Detection

• Format specification

COCO export

Downloaded file: a zip archive with the structure described here

• supported annotations: Polygons, Rectangles
• supported attributes:
  • is_crowd (checkbox or integer with values 0 and 1) - specifies that the instance (an object group) should have an RLE-encoded mask in the segmentation field. All the grouped shapes are merged into a single mask, the largest one defines all the object properties
  • score (number) - the annotation score field
  • arbitrary attributes - will be stored in the attributes annotation section

Support for COCO tasks via Datumaro is described here For example, support for COCO keypoints over Datumaro:

1. Install Datumaro pip install datumaro
2. Export the task in the Datumaro format, unzip
3. Export the Datumaro project in coco / coco_person_keypoints formats datum export -f coco -p path/to/project [-- --save-images]

This way, one can export CVAT points as single keypoints or keypoint lists (without the visibility COCO flag).

COCO import

Uploaded file: a single unpacked *.json or a zip archive with the structure described here (without images).

• supported annotations: Polygons, Rectangles (if the segmentation field is empty)

MS COCO Keypoint Detection

• Format specification

COCO export

Downloaded file: a zip archive with the structure described here

• supported annotations: Skeletons
• supported attributes:
  • is_crowd (checkbox or integer with values 0 and 1) - specifies that the instance (an object group) should have an RLE-encoded mask in the segmentation field. All the grouped shapes are merged into a single mask, the largest one defines all the object properties
  • score (number) - the annotation score field
  • arbitrary attributes - will be stored in the attributes annotation section

COCO import

Uploaded file: a single unpacked *.json or a zip archive with the structure described here (without images).

• supported annotations: Skeletons

How to create a task from MS COCO dataset

1. Download the MS COCO dataset.

   For example val images and instances annotations

2. Create a CVAT task with the following labels:

   person bicycle car motorcycle airplane bus train truck boat "traffic light" "fire hydrant" "stop sign" "parking meter" bench bird cat dog horse sheep cow elephant bear zebra giraffe backpack umbrella handbag tie suitcase frisbee skis snowboard "sports ball" kite "baseball bat" "baseball glove" skateboard surfboard "tennis racket" bottle "wine glass" cup fork knife spoon bowl banana apple sandwich orange broccoli carrot "hot dog" pizza donut cake chair couch "potted plant" bed "dining table" toilet tv laptop mouse remote keyboard "cell phone" microwave oven toaster sink refrigerator book clock vase scissors "teddy bear" "hair drier" toothbrush
   

3. Select val2017.zip as data (See Creating an annotation task guide for details)

4. Unpack annotations_trainval2017.zip

5. click Upload annotation button, choose COCO 1.1 and select instances_val2017.json annotation file. It can take some time.

4.2.24.7 -

Pascal VOC

• Format specification

• supported annotations:

  • Rectangles (detection and layout tasks)
  • Tags (action- and classification tasks)
  • Polygons (segmentation task)

• supported attributes:

  • occluded (both UI option and a separate attribute)
  • truncated and difficult (should be defined for labels as checkbox -es)
  • action attributes (import only, should be defined as checkbox -es)
  • arbitrary attributes (in the attributes section of XML files)

Pascal VOC export

Downloaded file: a zip archive of the following structure:

taskname.zip/
├── JPEGImages/
│   ├── <image_name1>.jpg
│   ├── <image_name2>.jpg
│   └── <image_nameN>.jpg
├── Annotations/
│   ├── <image_name1>.xml
│   ├── <image_name2>.xml
│   └── <image_nameN>.xml
├── ImageSets/
│   └── Main/
│       └── default.txt
└── labelmap.txt

# labelmap.txt
# label : color_rgb : 'body' parts : actions
background:::
aeroplane:::
bicycle:::
bird:::

Pascal VOC import

Uploaded file: a zip archive of the structure declared above or the following:

taskname.zip/
├── <image_name1>.xml
├── <image_name2>.xml
└── <image_nameN>.xml

It must be possible for CVAT to match the frame name and file name from annotation .xml file (the filename tag, e. g. <filename>2008_004457.jpg</filename> ).

There are 2 options:

1. full match between frame name and file name from annotation .xml (in cases when task was created from images or image archive).

2. match by frame number. File name should be <number>.jpg or frame_000000.jpg. It should be used when task was created from video.

Segmentation mask export

Downloaded file: a zip archive of the following structure:

taskname.zip/
├── labelmap.txt # optional, required for non-VOC labels
├── ImageSets/
│   └── Segmentation/
│       └── default.txt # list of image names without extension
├── SegmentationClass/ # merged class masks
│   ├── image1.png
│   └── image2.png
└── SegmentationObject/ # merged instance masks
    ├── image1.png
    └── image2.png

# labelmap.txt
# label : color (RGB) : 'body' parts : actions
background:0,128,0::
aeroplane:10,10,128::
bicycle:10,128,0::
bird:0,108,128::
boat:108,0,100::
bottle:18,0,8::
bus:12,28,0::

Mask is a png image with 1 or 3 channels where each pixel has own color which corresponds to a label. Colors are generated following to Pascal VOC algorithm. (0, 0, 0) is used for background by default.

• supported shapes: Rectangles, Polygons

Segmentation mask import

Uploaded file: a zip archive of the following structure:

  taskname.zip/
  ├── labelmap.txt # optional, required for non-VOC labels
  ├── ImageSets/
  │   └── Segmentation/
  │       └── <any_subset_name>.txt
  ├── SegmentationClass/
  │   ├── image1.png
  │   └── image2.png
  └── SegmentationObject/
      ├── image1.png
      └── image2.png

It is also possible to import grayscale (1-channel) PNG masks. For grayscale masks provide a list of labels with the number of lines equal to the maximum color index on images. The lines must be in the right order so that line index is equal to the color index. Lines can have arbitrary, but different, colors. If there are gaps in the used color indices in the annotations, they must be filled with arbitrary dummy labels. Example:

q:0,128,0:: # color index 0
aeroplane:10,10,128:: # color index 1
_dummy2:2,2,2:: # filler for color index 2
_dummy3:3,3,3:: # filler for color index 3
boat:108,0,100:: # color index 3
...
_dummy198:198,198,198:: # filler for color index 198
_dummy199:199,199,199:: # filler for color index 199
...
the last label:12,28,0:: # color index 200

• supported shapes: Polygons

How to create a task from Pascal VOC dataset

1. Download the Pascal Voc dataset (Can be downloaded from the PASCAL VOC website)

2. Create a CVAT task with the following labels:

   aeroplane bicycle bird boat bottle bus car cat chair cow diningtable
   dog horse motorbike person pottedplant sheep sofa train tvmonitor
   

   You can add ~checkbox=difficult:false ~checkbox=truncated:false attributes for each label if you want to use them.

   Select interesting image files (See Creating an annotation task guide for details)

3. zip the corresponding annotation files

4. click Upload annotation button, choose Pascal VOC ZIP 1.1

   and select the zip file with annotations from previous step. It may take some time.

4.2.24.8 -

YOLO

• Format specification
• supported annotations: Rectangles

YOLO export

Downloaded file: a zip archive with following structure:

archive.zip/
├── obj.data
├── obj.names
├── obj_<subset>_data
│   ├── image1.txt
│   └── image2.txt
└── train.txt # list of subset image paths

# the only valid subsets are: train, valid
# train.txt and valid.txt:
obj_<subset>_data/image1.jpg
obj_<subset>_data/image2.jpg

# obj.data:
classes = 3 # optional
names = obj.names
train = train.txt
valid = valid.txt # optional
backup = backup/ # optional

# obj.names:
cat
dog
airplane

# image_name.txt:
# label_id - id from obj.names
# cx, cy - relative coordinates of the bbox center
# rw, rh - relative size of the bbox
# label_id cx cy rw rh
1 0.3 0.8 0.1 0.3
2 0.7 0.2 0.3 0.1

Each annotation *.txt file has a name that corresponds to the name of the image file (e. g. frame_000001.txt is the annotation for the frame_000001.jpg image). The *.txt file structure: each line describes label and bounding box in the following format label_id cx cy w h. obj.names contains the ordered list of label names.

YOLO import

Uploaded file: a zip archive of the same structure as above It must be possible to match the CVAT frame (image name) and annotation file name. There are 2 options:

1. full match between image name and name of annotation *.txt file (in cases when a task was created from images or archive of images).

2. match by frame number (if CVAT cannot match by name). File name should be in the following format <number>.jpg . It should be used when task was created from a video.

How to create a task from YOLO formatted dataset (from VOC for example)

1. Follow the official guide(see Training YOLO on VOC section) and prepare the YOLO formatted annotation files.

2. Zip train images

zip images.zip -j -@ < train.txt

1. Create a CVAT task with the following labels:

   aeroplane bicycle bird boat bottle bus car cat chair cow diningtable dog
   horse motorbike person pottedplant sheep sofa train tvmonitor
   

   Select images. zip as data. Most likely you should use share functionality because size of images. zip is more than 500Mb. See Creating an annotation task guide for details.

2. Create obj.names with the following content:

   aeroplane
   bicycle
   bird
   boat
   bottle
   bus
   car
   cat
   chair
   cow
   diningtable
   dog
   horse
   motorbike
   person
   pottedplant
   sheep
   sofa
   train
   tvmonitor
   

3. Zip all label files together (we need to add only label files that correspond to the train subset)

   cat train.txt | while read p; do echo ${p%/*/*}/labels/${${p##*/}%%.*}.txt; done | zip labels.zip -j -@ obj.names
   

4. Click Upload annotation button, choose YOLO 1.1 and select the zip

   file with labels from the previous step.

4.2.24.9 -

TFRecord

TFRecord is a very flexible format, but we try to correspond the format that used in TF object detection with minimal modifications.

Used feature description:

image_feature_description = {
    'image/filename': tf.io.FixedLenFeature([], tf.string),
    'image/source_id': tf.io.FixedLenFeature([], tf.string),
    'image/height': tf.io.FixedLenFeature([], tf.int64),
    'image/width': tf.io.FixedLenFeature([], tf.int64),
    # Object boxes and classes.
    'image/object/bbox/xmin': tf.io.VarLenFeature(tf.float32),
    'image/object/bbox/xmax': tf.io.VarLenFeature(tf.float32),
    'image/object/bbox/ymin': tf.io.VarLenFeature(tf.float32),
    'image/object/bbox/ymax': tf.io.VarLenFeature(tf.float32),
    'image/object/class/label': tf.io.VarLenFeature(tf.int64),
    'image/object/class/text': tf.io.VarLenFeature(tf.string),
}

TFRecord export

Downloaded file: a zip archive with following structure:

taskname.zip/
├── default.tfrecord
└── label_map.pbtxt

# label_map.pbtxt
item {
	id: 1
	name: 'label_0'
}
item {
	id: 2
	name: 'label_1'
}
...

• supported annotations: Rectangles, Polygons (as masks, manually over Datumaro)

How to export masks:

1. Export annotations in Datumaro format
2. Apply polygons_to_masks and boxes_to_masks transforms

datum transform -t polygons_to_masks -p path/to/proj -o ptm
datum transform -t boxes_to_masks -p ptm -o btm

1. Export in the TF Detection API format

datum export -f tf_detection_api -p btm [-- --save-images]

TFRecord import

Uploaded file: a zip archive of following structure:

taskname.zip/
└── <any name>.tfrecord

• supported annotations: Rectangles

How to create a task from TFRecord dataset (from VOC2007 for example)

1. Create label_map.pbtxt file with the following content:

item {
    id: 1
    name: 'aeroplane'
}
item {
    id: 2
    name: 'bicycle'
}
item {
    id: 3
    name: 'bird'
}
item {
    id: 4
    name: 'boat'
}
item {
    id: 5
    name: 'bottle'
}
item {
    id: 6
    name: 'bus'
}
item {
    id: 7
    name: 'car'
}
item {
    id: 8
    name: 'cat'
}
item {
    id: 9
    name: 'chair'
}
item {
    id: 10
    name: 'cow'
}
item {
    id: 11
    name: 'diningtable'
}
item {
    id: 12
    name: 'dog'
}
item {
    id: 13
    name: 'horse'
}
item {
    id: 14
    name: 'motorbike'
}
item {
    id: 15
    name: 'person'
}
item {
    id: 16
    name: 'pottedplant'
}
item {
    id: 17
    name: 'sheep'
}
item {
    id: 18
    name: 'sofa'
}
item {
    id: 19
    name: 'train'
}
item {
    id: 20
    name: 'tvmonitor'
}

1. Use create_pascal_tf_record.py

to convert VOC2007 dataset to TFRecord format. As example:

python create_pascal_tf_record.py --data_dir <path to VOCdevkit> --set train --year VOC2007 --output_path pascal.tfrecord --label_map_path label_map.pbtxt

1. Zip train images

   cat <path to VOCdevkit>/VOC2007/ImageSets/Main/train.txt | while read p; do echo <path to VOCdevkit>/VOC2007/JPEGImages/${p}.jpg  ; done | zip images.zip -j -@
   

2. Create a CVAT task with the following labels:

   aeroplane bicycle bird boat bottle bus car cat chair cow diningtable dog horse motorbike person pottedplant sheep sofa train tvmonitor
   

   Select images. zip as data. See Creating an annotation task guide for details.

3. Zip pascal.tfrecord and label_map.pbtxt files together

   zip anno.zip -j <path to pascal.tfrecord> <path to label_map.pbtxt>
   

4. Click Upload annotation button, choose TFRecord 1.0 and select the zip file

   with labels from the previous step. It may take some time.

4.2.24.10 -

ImageNet

ImageNet export

Downloaded file: a zip archive of the following structure:

# if we save images:
taskname.zip/
├── label1/
|   ├── label1_image1.jpg
|   └── label1_image2.jpg
└── label2/
    ├── label2_image1.jpg
    ├── label2_image3.jpg
    └── label2_image4.jpg

# if we keep only annotation:
taskname.zip/
├── <any_subset_name>.txt
└── synsets.txt

• supported annotations: Labels

ImageNet import

Uploaded file: a zip archive of the structure above

• supported annotations: Labels

4.2.24.11 -

WIDER Face

WIDER Face export

Downloaded file: a zip archive of the following structure:

taskname.zip/
├── labels.txt # optional
├── wider_face_split/
│   └── wider_face_<any_subset_name>_bbx_gt.txt
└── WIDER_<any_subset_name>/
    └── images/
        ├── 0--label0/
        │   └── 0_label0_image1.jpg
        └── 1--label1/
            └── 1_label1_image2.jpg

• supported annotations: Rectangles (with attributes), Labels
• supported attributes:
  • blur, expression, illumination, pose, invalid
  • occluded (both the annotation property & an attribute)

WIDER Face import

Uploaded file: a zip archive of the structure above

• supported annotations: Rectangles (with attributes), Labels
• supported attributes:
  • blur, expression, illumination, occluded, pose, invalid

4.2.24.12 -

CamVid

CamVid export

Downloaded file: a zip archive of the following structure:

taskname.zip/
├── labelmap.txt # optional, required for non-CamVid labels
├── <any_subset_name>/
|   ├── image1.png
|   └── image2.png
├── <any_subset_name>annot/
|   ├── image1.png
|   └── image2.png
└── <any_subset_name>.txt

# labelmap.txt
# color (RGB) label
0 0 0 Void
64 128 64 Animal
192 0 128 Archway
0 128 192 Bicyclist
0 128 64 Bridge

Mask is a png image with 1 or 3 channels where each pixel has own color which corresponds to a label. (0, 0, 0) is used for background by default.

• supported annotations: Rectangles, Polygons

CamVid import

Uploaded file: a zip archive of the structure above

• supported annotations: Polygons

4.2.24.13 -

VGGFace2

VGGFace2 export

Downloaded file: a zip archive of the following structure:

taskname.zip/
├── labels.txt # optional
├── <any_subset_name>/
|   ├── label0/
|   |   └── image1.jpg
|   └── label1/
|       └── image2.jpg
└── bb_landmark/
    ├── loose_bb_<any_subset_name>.csv
    └── loose_landmark_<any_subset_name>.csv
# labels.txt
# n000001 car
label0 <class0>
label1 <class1>

• supported annotations: Rectangles, Points (landmarks - groups of 5 points)

VGGFace2 import

Uploaded file: a zip archive of the structure above

• supported annotations: Rectangles, Points (landmarks - groups of 5 points)

4.2.24.14 -

Market-1501

Market-1501 export

Downloaded file: a zip archive of the following structure:

taskname.zip/
├── bounding_box_<any_subset_name>/
│   └── image_name_1.jpg
└── query
    ├── image_name_2.jpg
    └── image_name_3.jpg
# if we keep only annotation:
taskname.zip/
└── images_<any_subset_name>.txt
# images_<any_subset_name>.txt
query/image_name_1.jpg
bounding_box_<any_subset_name>/image_name_2.jpg
bounding_box_<any_subset_name>/image_name_3.jpg
# image_name = 0001_c1s1_000015_00.jpg
0001 - person id
c1 - camera id (there are totally 6 cameras)
s1 - sequence
000015 - frame number in sequence
00 - means that this bounding box is the first one among the several

• supported annotations: Label market-1501 with attributes (query, person_id, camera_id)

Market-1501 import

Uploaded file: a zip archive of the structure above

• supported annotations: Label market-1501 with attributes (query, person_id, camera_id)

4.2.24.15 -

ICDAR13/15

ICDAR13/15 export

Downloaded file: a zip archive of the following structure:

# word recognition task
taskname.zip/
└── word_recognition/
    └── <any_subset_name>/
        ├── images
        |   ├── word1.png
        |   └── word2.png
        └── gt.txt
# text localization task
taskname.zip/
└── text_localization/
    └── <any_subset_name>/
        ├── images
        |   ├── img_1.png
        |   └── img_2.png
        ├── gt_img_1.txt
        └── gt_img_1.txt
#text segmentation task
taskname.zip/
└── text_localization/
    └── <any_subset_name>/
        ├── images
        |   ├── 1.png
        |   └── 2.png
        ├── 1_GT.bmp
        ├── 1_GT.txt
        ├── 2_GT.bmp
        └── 2_GT.txt

Word recognition task:

• supported annotations: Label icdar with attribute caption

Text localization task:

• supported annotations: Rectangles and Polygons with label icdar and attribute text

Text segmentation task:

• supported annotations: Rectangles and Polygons with label icdar and attributes index, text, color, center

ICDAR13/15 import

Uploaded file: a zip archive of the structure above

Word recognition task:

• supported annotations: Label icdar with attribute caption

Text localization task:

• supported annotations: Rectangles and Polygons with label icdar and attribute text

Text segmentation task:

• supported annotations: Rectangles and Polygons with label icdar and attributes index, text, color, center

4.2.24.16 -

Open Images

• Format specification

• Supported annotations:

  • Rectangles (detection task)
  • Tags (classification task)
  • Polygons (segmentation task)

• Supported attributes:

  • Labels

    • score (should be defined for labels as text or number). The confidence level from 0 to 1.

  • Bounding boxes

    • score (should be defined for labels as text or number). The confidence level from 0 to 1.
    • occluded (both UI option and a separate attribute). Whether the object is occluded by another object.
    • truncated (should be defined for labels as checkbox -es). Whether the object extends beyond the boundary of the image.
    • is_group_of (should be defined for labels as checkbox -es). Whether the object represents a group of objects of the same class.
    • is_depiction (should be defined for labels as checkbox -es). Whether the object is a depiction (such as a drawing) rather than a real object.
    • is_inside (should be defined for labels as checkbox -es). Whether the object is seen from the inside.

  • Masks

    • box_id (should be defined for labels as text). An identifier for the bounding box associated with the mask.
    • predicted_iou (should be defined for labels as text or number). Predicted IoU value with respect to the ground truth.

Open Images export

Downloaded file: a zip archive of the following structure:

└─ taskname.zip/
    ├── annotations/
    │   ├── bbox_labels_600_hierarchy.json
    │   ├── class-descriptions.csv
    |   ├── images.meta  # additional file with information about image sizes
    │   ├── <subset_name>-image_ids_and_rotation.csv
    │   ├── <subset_name>-annotations-bbox.csv
    │   ├── <subset_name>-annotations-human-imagelabels.csv
    │   └── <subset_name>-annotations-object-segmentation.csv
    ├── images/
    │   ├── subset1/
    │   │   ├── <image_name101.jpg>
    │   │   ├── <image_name102.jpg>
    │   │   └── ...
    │   ├── subset2/
    │   │   ├── <image_name201.jpg>
    │   │   ├── <image_name202.jpg>
    │   │   └── ...
    |   ├── ...
    └── masks/
        ├── subset1/
        │   ├── <mask_name101.png>
        │   ├── <mask_name102.png>
        │   └── ...
        ├── subset2/
        │   ├── <mask_name201.png>
        │   ├── <mask_name202.png>
        │   └── ...
        ├── ...

Open Images import

Uploaded file: a zip archive of the following structure:

└─ upload.zip/
    ├── annotations/
    │   ├── bbox_labels_600_hierarchy.json
    │   ├── class-descriptions.csv
    |   ├── images.meta  # optional, file with information about image sizes
    │   ├── <subset_name>-image_ids_and_rotation.csv
    │   ├── <subset_name>-annotations-bbox.csv
    │   ├── <subset_name>-annotations-human-imagelabels.csv
    │   └── <subset_name>-annotations-object-segmentation.csv
    └── masks/
        ├── subset1/
        │   ├── <mask_name101.png>
        │   ├── <mask_name102.png>
        │   └── ...
        ├── subset2/
        │   ├── <mask_name201.png>
        │   ├── <mask_name202.png>
        │   └── ...
        ├── ...

Image ids in the <subset_name>-image_ids_and_rotation.csv should match with image names in the task.

4.2.24.17 -

Cityscapes

• Format specification

• Supported annotations

  • Polygons (segmentation task)

• Supported attributes

  • ‘is_crowd’ (boolean, should be defined for labels as checkbox -es) Specifies if the annotation label can distinguish between different instances. If False, the annotation id field encodes the instance id.

Cityscapes export

Downloaded file: a zip archive of the following structure:

.
├── label_color.txt
├── gtFine
│   ├── <subset_name>
│   │   └── <city_name>
│   │       ├── image_0_gtFine_instanceIds.png
│   │       ├── image_0_gtFine_color.png
│   │       ├── image_0_gtFine_labelIds.png
│   │       ├── image_1_gtFine_instanceIds.png
│   │       ├── image_1_gtFine_color.png
│   │       ├── image_1_gtFine_labelIds.png
│   │       ├── ...
└── imgsFine  # if saving images was requested
    └── leftImg8bit
        ├── <subset_name>
        │   └── <city_name>
        │       ├── image_0_leftImg8bit.png
        │       ├── image_1_leftImg8bit.png
        │       ├── ...

• label_color.txt a file that describes the color for each label

# label_color.txt example
# r g b label_name
0 0 0 background
0 255 0 tree
...

• *_gtFine_color.png class labels encoded by its color.
• *_gtFine_labelIds.png class labels are encoded by its index.
• *_gtFine_instanceIds.png class and instance labels encoded by an instance ID. The pixel values encode class and the individual instance: the integer part of a division by 1000 of each ID provides class ID, the remainder is the instance ID. If a certain annotation describes multiple instances, then the pixels have the regular ID of that class

Cityscapes annotations import

Uploaded file: a zip archive with the following structure:

.
├── label_color.txt # optional
└── gtFine
    └── <city_name>
        ├── image_0_gtFine_instanceIds.png
        ├── image_1_gtFine_instanceIds.png
        ├── ...

Creating task with Cityscapes dataset

Create a task with the labels you need or you can use the labels and colors of the original dataset. To work with the Cityscapes format, you must have a black color label for the background.

Original Cityscapes color map:

[
    {"name": "unlabeled", "color": "#000000", "attributes": []},
    {"name": "egovehicle", "color": "#000000", "attributes": []},
    {"name": "rectificationborder", "color": "#000000", "attributes": []},
    {"name": "outofroi", "color": "#000000", "attributes": []},
    {"name": "static", "color": "#000000", "attributes": []},
    {"name": "dynamic", "color": "#6f4a00", "attributes": []},
    {"name": "ground", "color": "#510051", "attributes": []},
    {"name": "road", "color": "#804080", "attributes": []},
    {"name": "sidewalk", "color": "#f423e8", "attributes": []},
    {"name": "parking", "color": "#faaaa0", "attributes": []},
    {"name": "railtrack", "color": "#e6968c", "attributes": []},
    {"name": "building", "color": "#464646", "attributes": []},
    {"name": "wall", "color": "#66669c", "attributes": []},
    {"name": "fence", "color": "#be9999", "attributes": []},
    {"name": "guardrail", "color": "#b4a5b4", "attributes": []},
    {"name": "bridge", "color": "#966464", "attributes": []},
    {"name": "tunnel", "color": "#96785a", "attributes": []},
    {"name": "pole", "color": "#999999", "attributes": []},
    {"name": "polegroup", "color": "#999999", "attributes": []},
    {"name": "trafficlight", "color": "#faaa1e", "attributes": []},
    {"name": "trafficsign", "color": "#dcdc00", "attributes": []},
    {"name": "vegetation", "color": "#6b8e23", "attributes": []},
    {"name": "terrain", "color": "#98fb98", "attributes": []},
    {"name": "sky", "color": "#4682b4", "attributes": []},
    {"name": "person", "color": "#dc143c", "attributes": []},
    {"name": "rider", "color": "#ff0000", "attributes": []},
    {"name": "car", "color": "#00008e", "attributes": []},
    {"name": "truck", "color": "#000046", "attributes": []},
    {"name": "bus", "color": "#003c64", "attributes": []},
    {"name": "caravan", "color": "#00005a", "attributes": []},
    {"name": "trailer", "color": "#00006e", "attributes": []},
    {"name": "train", "color": "#005064", "attributes": []},
    {"name": "motorcycle", "color": "#0000e6", "attributes": []},
    {"name": "bicycle", "color": "#770b20", "attributes": []},
    {"name": "licenseplate", "color": "#00000e", "attributes": []}
]

Upload images when creating a task:

images.zip/
    ├── image_0.jpg
    ├── image_1.jpg
    ├── ...

After creating the task, upload the Cityscapes annotations as described in the previous section.

4.2.24.18 -

KITTI

• Format specification for KITTI detection

• Format specification for KITTI segmentation

• supported annotations:

  • Rectangles (detection task)
  • Polygon (segmentation task)

• supported attributes:

  • occluded (both UI option and a separate attribute). Indicates that a significant portion of the object within the bounding box is occluded by another object
  • truncated supported only for rectangles (should be defined for labels as checkbox -es). Indicates that the bounding box specified for the object does not correspond to the full extent of the object
  • ‘is_crowd’ supported only for polygons (should be defined for labels as checkbox -es). Indicates that the annotation covers multiple instances of the same class

KITTI annotations export

Downloaded file: a zip archive of the following structure:

└─ annotations.zip/
    ├── label_colors.txt # list of pairs r g b label_name
    ├── labels.txt # list of labels
    └── default/
        ├── label_2/ # left color camera label files
        │   ├── <image_name_1>.txt
        │   ├── <image_name_2>.txt
        │   └── ...
        ├── instance/ # instance segmentation masks
        │   ├── <image_name_1>.png
        │   ├── <image_name_2>.png
        │   └── ...
        ├── semantic/ # semantic segmentation masks (labels are encoded by its id)
        │   ├── <image_name_1>.png
        │   ├── <image_name_2>.png
        │   └── ...
        └── semantic_rgb/ # semantic segmentation masks (labels are encoded by its color)
            ├── <image_name_1>.png
            ├── <image_name_2>.png
            └── ...

KITTI annotations import

You can upload KITTI annotations in two ways: rectangles for the detection task and masks for the segmentation task.

For detection tasks the uploading archive should have the following structure:

└─ annotations.zip/
    ├── labels.txt # optional, labels list for non-original detection labels
    └── <subset_name>/
        ├── label_2/ # left color camera label files
        │   ├── <image_name_1>.txt
        │   ├── <image_name_2>.txt
        │   └── ...

For segmentation tasks the uploading archive should have the following structure:

└─ annotations.zip/
    ├── label_colors.txt # optional, color map for non-original segmentation labels
    └── <subset_name>/
        ├── instance/ # instance segmentation masks
        │   ├── <image_name_1>.png
        │   ├── <image_name_2>.png
        │   └── ...
        ├── semantic/ # optional, semantic segmentation masks (labels are encoded by its id)
        │   ├── <image_name_1>.png
        │   ├── <image_name_2>.png
        │   └── ...
        └── semantic_rgb/ # optional, semantic segmentation masks (labels are encoded by its color)
            ├── <image_name_1>.png
            ├── <image_name_2>.png
            └── ...

All annotation files and masks should have structures that are described in the original format specification.

4.2.24.19 -

LFW

• Format specification available here

• Supported annotations: tags, points.

• Supported attributes:

  • negative_pairs (should be defined for labels as text): list of image names with mismatched persons.
  • positive_pairs (should be defined for labels as text): list of image names with matched persons.

Import LFW annotation

The uploaded annotations file should be a zip file with the following structure:

<archive_name>.zip/
    └── annotations/
        ├── landmarks.txt # list with landmark points for each image
        ├── pairs.txt # list of matched and mismatched pairs of person
        └── people.txt # optional file with a list of persons name

Full information about the content of annotation files is available here

Export LFW annotation

Downloaded file: a zip archive of the following structure:

<archive_name>.zip/
    └── images/ # if the option save images was selected
    │    ├── name1/
    │    │   ├── name1_0001.jpg
    │    │   ├── name1_0002.jpg
    │    │   ├── ...
    │    ├── name2/
    │    │   ├── name2_0001.jpg
    │    │   ├── name2_0002.jpg
    │    │   ├── ...
    │    ├── ...
    ├── landmarks.txt
    ├── pairs.txt
    └── people.txt

Example: create task with images and upload LFW annotations into it

This is one of the possible ways to create a task and add LFW annotations for it.

• On the task creation page:
  • Add labels that correspond to the names of the persons.
  • For each label define text attributes with names positive_pairs and negative_pairs
  • Add images using zip archive from local repository:

images.zip/
    ├── name1_0001.jpg
    ├── name1_0002.jpg
    ├── ...
    ├── name1_<N>.jpg
    ├── name2_0001.jpg
    ├── ...

• On the annotation page: Upload annotation -> LFW 1.0 -> choose archive with structure that described in the import section.

4.2.25 - Task synchronization with a repository

Notice: this feature works only if a git repository was specified when the task was created.

1. At the end of the annotation process, a task is synchronized by clicking Synchronize on the task page. If the synchronization is successful, the button will change to Sychronized in blue:

   

2. The annotation is now in the repository in a temporary branch. The next step is to go to the repository and manually create a pull request to the main branch.

3. After merging the PR, when the annotation is saved in the main branch, the button changes to Merged and is highlighted in green.

   

If annotation in the task does not correspond annotations in the repository, the sync button will turn red:

4.2.26 - XML annotation format

When you want to download annotations from Computer Vision Annotation Tool (CVAT) you can choose one of several data formats. The document describes XML annotation format. Each format has X.Y version (e.g. 1.0). In general the major version (X) is incremented when the data format has incompatible changes and the minor version (Y) is incremented when the data format is slightly modified (e.g. it has one or several extra fields inside meta information). The document will describe all changes for all versions of XML annotation format.

Version 1.1

There are two different formats for images and video tasks at the moment. The both formats have a common part which is described below. From the previous version flipped tag was added. Also original_size tag was added for interpolation mode to specify frame size. In annotation mode each image tag has width and height attributes for the same purpose.

<?xml version="1.0" encoding="utf-8"?>
<annotations>
  <version>1.1</version>
  <meta>
    <task>
      <id>Number: id of the task</id>
      <name>String: some task name</name>
      <size>Number: count of frames/images in the task</size>
      <mode>String: interpolation or annotation</mode>
      <overlap>Number: number of overlapped frames between segments</overlap>
      <bugtracker>String: URL on an page which describe the task</bugtracker>
      <flipped>Boolean: were images of the task flipped? (True/False)</flipped>
      <created>String: date when the task was created</created>
      <updated>String: date when the task was updated</updated>
      <labels>
        <label>
          <name>String: name of the label (e.g. car, person)</name>
          <type>String: any, bbox, cuboid, cuboid_3d, ellipse, polygon, polyline, points, skeleton, tag</type>
          <attributes>
            <attribute>
              <name>String: attribute name</name>
              <mutable>Boolean: mutable (allow different values between frames)</mutable>
              <input_type>String: select, checkbox, radio, number, text</input_type>
              <default_value>String: default value</default_value>
              <values>String: possible values, separated by newlines
ex. value 2
ex. value 3</values>
            </attribute>
          </attributes>
          <svg>String: label representation in svg, only for skeletons</svg>
          <parent>String: label parent name, only for skeletons</parent>
        </label>
      </labels>
      <segments>
        <segment>
          <id>Number: id of the segment</id>
          <start>Number: first frame</start>
          <stop>Number: last frame</stop>
          <url>String: URL (e.g. http://cvat.example.com/?id=213)</url>
        </segment>
      </segments>
      <owner>
        <username>String: the author of the task</username>
        <email>String: email of the author</email>
      </owner>
      <original_size>
        <width>Number: frame width</width>
        <height>Number: frame height</height>
      </original_size>
    </task>
    <dumped>String: date when the annotation was dumped</dumped>
  </meta>
  ...
</annotations>

Annotation

Below you can find description of the data format for images tasks. On each image it is possible to have many different objects. Each object can have multiple attributes. If an annotation task is created with z_order flag then each object will have z_order attribute which is used to draw objects properly when they are intersected (if z_order is bigger the object is closer to camera). In previous versions of the format only box shape was available. In later releases polygon, polyline, points, skeletons and tags were added. Please see below for more details:

<?xml version="1.0" encoding="utf-8"?>
<annotations>
  ...
  <image id="Number: id of the image (the index in lexical order of images)" name="String: path to the image"
    width="Number: image width" height="Number: image height">
    <box label="String: the associated label" xtl="Number: float" ytl="Number: float" xbr="Number: float" ybr="Number: float" occluded="Number: 0 - False, 1 - True" z_order="Number: z-order of the object">
      <attribute name="String: an attribute name">String: the attribute value</attribute>
      ...
    </box>
    <polygon label="String: the associated label" points="x0,y0;x1,y1;..." occluded="Number: 0 - False, 1 - True"
    z_order="Number: z-order of the object">
      <attribute name="String: an attribute name">String: the attribute value</attribute>
      ...
    </polygon>
    <polyline label="String: the associated label" points="x0,y0;x1,y1;..." occluded="Number: 0 - False, 1 - True"
    z_order="Number: z-order of the object">
      <attribute name="String: an attribute name">String: the attribute value</attribute>
      ...
    </polyline>
    <polyline label="String: the associated label" points="x0,y0;x1,y1;..." occluded="Number: 0 - False, 1 - True"
    z_order="Number: z-order of the object">
      <attribute name="String: an attribute name">String: the attribute value</attribute>
      ...
    </polyline>
    <points label="String: the associated label" points="x0,y0;x1,y1;..." occluded="Number: 0 - False, 1 - True"
    z_order="Number: z-order of the object">
      <attribute name="String: an attribute name">String: the attribute value</attribute>
      ...
    </points>
    <tag label="String: the associated label" source="manual or auto">
      <attribute name="String: an attribute name">String: the attribute value</attribute>
      ...
    </tag>
    <skeleton label="String: the associated label" occluded="Number: 0 - False, 1 - True" z_order="Number: z-order of the object">
      <points label="String: the associated label" occluded="Number: 0 - False, 1 - True" outside="Number: 0 - False, 1 - True" points="x0,y0;x1,y1" z_order="0">
        <attribute name="String: an attribute name">String: the attribute value</attribute>
      </points>
      ...
      <attribute name="String: an attribute name">String: the attribute value</attribute>
      ...
    </skeleton>
    ...
  </image>
  ...
</annotations>

Example:

<?xml version="1.0" encoding="utf-8"?>
<annotations>
  <version>1.1</version>
  <meta>
    <task>
      <id>4</id>
      <name>segmentation</name>
      <size>27</size>
      <mode>annotation</mode>
      <overlap>0</overlap>
      <bugtracker></bugtracker>
      <flipped>False</flipped>
      <created>2018-09-25 11:34:24.617558+03:00</created>
      <updated>2018-09-25 11:38:27.301183+03:00</updated>
      <labels>
        <label>
          <name>car</name>
          <attributes>
          </attributes>
        </label>
        <label>
          <name>traffic_line</name>
          <attributes>
          </attributes>
        </label>
        <label>
          <name>wheel</name>
          <attributes>
          </attributes>
        </label>
        <label>
          <name>plate</name>
          <attributes>
          </attributes>
        </label>
        <label>
          <name>s1</name>
          <type>skeleton</type>
          <attributes>
          </attributes>
          <svg>&lt;line x1="36.87290954589844" y1="47.732025146484375" x2="86.87290954589844" y2="10.775501251220703" stroke="black" data-type="edge" data-node-from="2" stroke-width="0.5" data-node-to="3"&gt;&lt;/line&gt;&lt;line x1="25.167224884033203" y1="22.64841079711914" x2="36.87290954589844" y2="47.732025146484375" stroke="black" data-type="edge" data-node-from="1" stroke-width="0.5" data-node-to="2"&gt;&lt;/line&gt;&lt;circle r="1.5" stroke="black" fill="#b3b3b3" cx="25.167224884033203" cy="22.64841079711914" stroke-width="0.1" data-type="element node" data-element-id="1" data-node-id="1" data-label-name="1"&gt;&lt;/circle&gt;&lt;circle r="1.5" stroke="black" fill="#b3b3b3" cx="36.87290954589844" cy="47.732025146484375" stroke-width="0.1" data-type="element node" data-element-id="2" data-node-id="2" data-label-name="2"&gt;&lt;/circle&gt;&lt;circle r="1.5" stroke="black" fill="#b3b3b3" cx="86.87290954589844" cy="10.775501251220703" stroke-width="0.1" data-type="element node" data-element-id="3" data-node-id="3" data-label-name="3"&gt;&lt;/circle&gt;</svg>
        </label>
        <label>
          <name>1</name>
          <type>points</type>
          <attributes>
          </attributes>
          <parent>s1</parent>
        </label>
        <label>
          <name>2</name>
          <type>points</type>
          <attributes>
          </attributes>
          <parent>s1</parent>
        </label>
        <label>
          <name>3</name>
          <type>points</type>
          <attributes>
          </attributes>
          <parent>s1</parent>
        </label>
      </labels>
      <segments>
        <segment>
          <id>4</id>
          <start>0</start>
          <stop>26</stop>
          <url>http://localhost:8080/?id=4</url>
        </segment>
      </segments>
      <owner>
        <username>admin</username>
        <email></email>
      </owner>
    </task>
    <dumped>2018-09-25 11:38:28.799808+03:00</dumped>
  </meta>
  <image id="0" name="filename000.jpg" width="1600" height="1200">
    <box label="plate" xtl="797.33" ytl="870.92" xbr="965.52" ybr="928.94" occluded="0" z_order="4">
    </box>
    <polygon label="car" points="561.30,916.23;561.30,842.77;554.72,761.63;553.62,716.67;565.68,677.20;577.74,566.45;547.04,559.87;536.08,542.33;528.40,520.40;541.56,512.72;559.10,509.43;582.13,506.14;588.71,464.48;583.23,448.03;587.61,434.87;594.19,431.58;609.54,399.78;633.66,369.08;676.43,294.52;695.07,279.17;703.84,279.17;735.64,268.20;817.88,264.91;923.14,266.01;997.70,274.78;1047.04,283.55;1063.49,289.04;1090.90,330.70;1111.74,371.27;1135.86,397.59;1147.92,428.29;1155.60,435.97;1157.79,451.32;1156.69,462.28;1159.98,491.89;1163.27,522.59;1173.14,513.82;1199.46,516.01;1224.68,521.49;1225.77,544.52;1207.13,568.64;1181.91,576.32;1178.62,582.90;1177.53,619.08;1186.30,680.48;1199.46,711.19;1206.03,733.12;1203.84,760.53;1197.26,818.64;1199.46,840.57;1203.84,908.56;1192.88,930.49;1184.10,939.26;1162.17,944.74;1139.15,960.09;1058.01,976.54;1028.40,969.96;1002.09,972.15;931.91,974.35;844.19,972.15;772.92,972.15;729.06,967.77;713.71,971.06;685.20,973.25;659.98,968.86;644.63,984.21;623.80,983.12;588.71,985.31;560.20,966.67" occluded="0" z_order="1">
    </polygon>
    <polyline label="traffic_line" points="462.10,0.00;126.80,1200.00" occluded="0" z_order="3">
    </polyline>
    <polyline label="traffic_line" points="1212.40,0.00;1568.66,1200.00" occluded="0" z_order="2">
    </polyline>
    <points label="wheel" points="574.90,939.48;1170.16,907.90;1130.69,445.26;600.16,459.48" occluded="0" z_order="5">
    </points>
    <tag label="good_frame" source="manual">
    </tag>
    <skeleton label="s1" occluded="0" source="manual" points="" rotation="0.00" z_order="0">
      <points label="1" occluded="0" source="manual" outside="0" points="54.47,94.81" z_order="0">
      </points>
      <points label="2" occluded="0" source="manual" outside="0" points="68.02,162.34" z_order="0">
      </points>
      <points label="3" occluded="0" source="manual" outside="0" points="125.87,62.85" z_order="0">
      </points>
    </skeleton>
  </image>
</annotations>

Interpolation

Below you can find description of the data format for video tasks. The annotation contains tracks. Each track corresponds to an object which can be presented on multiple frames. The same object cannot be presented on the same frame in multiple locations. Each location of the object can have multiple attributes even if an attribute is immutable for the object it will be cloned for each location (a known redundancy).

<?xml version="1.0" encoding="utf-8"?>
<annotations>
  ...
  <track id="Number: id of the track (doesn't have any special meeting)" label="String: the associated label" source="manual or auto">
    <box frame="Number: frame" xtl="Number: float" ytl="Number: float" xbr="Number: float" ybr="Number: float" outside="Number: 0 - False, 1 - True" occluded="Number: 0 - False, 1 - True" keyframe="Number: 0 - False, 1 - True">
      <attribute name="String: an attribute name">String: the attribute value</attribute>
      ...
    </box>
    <polygon frame="Number: frame" points="x0,y0;x1,y1;..." outside="Number: 0 - False, 1 - True" occluded="Number: 0 - False, 1 - True" keyframe="Number: 0 - False, 1 - True">
      <attribute name="String: an attribute name">String: the attribute value</attribute>
    </polygon>
    <polyline frame="Number: frame" points="x0,y0;x1,y1;..." outside="Number: 0 - False, 1 - True" occluded="Number: 0 - False, 1 - True" keyframe="Number: 0 - False, 1 - True">
      <attribute name="String: an attribute name">String: the attribute value</attribute>
    </polyline>
    <points frame="Number: frame" points="x0,y0;x1,y1;..." outside="Number: 0 - False, 1 - True" occluded="Number: 0 - False, 1 - True" keyframe="Number: 0 - False, 1 - True">
      <attribute name="String: an attribute name">String: the attribute value</attribute>
    </points>
    ...
  </track>
  <track id="Number: id of the track (doesn't have any special meeting)" label="String: the associated label" source="manual or auto">
    <skeleton frame="Number: frame" outside="Number: 0 - False, 1 - True" occluded="Number: 0 - False, 1 - True" keyframe="Number: 0 - False, 1 - True">
    </skeleton>
    ...
    <track id="Number: id of the track (doesn't have any special meeting)" label="String: the associated label" source="manual or auto">
      <points frame="Number: frame" outside="Number: 0 - False, 1 - True" occluded="Number: 0 - False, 1 - True" keyframe="Number: 0 - False, 1 - True" points="x0,y0;x1,y1">
      </points>
      ...
    </track>
    ...
  </track>
  ...
</annotations>

Example:

<?xml version="1.0" encoding="utf-8"?>
<annotations>
  <version>1.1</version>
  <meta>
    <task>
      <id>5</id>
      <name>interpolation</name>
      <size>4620</size>
      <mode>interpolation</mode>
      <overlap>5</overlap>
      <bugtracker></bugtracker>
      <flipped>False</flipped>
      <created>2018-09-25 12:32:09.868194+03:00</created>
      <updated>2018-09-25 16:05:05.619841+03:00</updated>
      <labels>
        <label>
          <name>person</name>
          <attributes>
          </attributes>
        </label>
        <label>
          <name>car</name>
          <attributes>
          </attributes>
        </label>
        <label>
          <name>s1</name>
          <type>skeleton</type>
          <attributes>
          </attributes>
          <svg>&lt;line x1="36.87290954589844" y1="47.732025146484375" x2="86.87290954589844" y2="10.775501251220703" stroke="black" data-type="edge" data-node-from="2" stroke-width="0.5" data-node-to="3"&gt;&lt;/line&gt;&lt;line x1="25.167224884033203" y1="22.64841079711914" x2="36.87290954589844" y2="47.732025146484375" stroke="black" data-type="edge" data-node-from="1" stroke-width="0.5" data-node-to="2"&gt;&lt;/line&gt;&lt;circle r="1.5" stroke="black" fill="#b3b3b3" cx="25.167224884033203" cy="22.64841079711914" stroke-width="0.1" data-type="element node" data-element-id="1" data-node-id="1" data-label-name="1"&gt;&lt;/circle&gt;&lt;circle r="1.5" stroke="black" fill="#b3b3b3" cx="36.87290954589844" cy="47.732025146484375" stroke-width="0.1" data-type="element node" data-element-id="2" data-node-id="2" data-label-name="2"&gt;&lt;/circle&gt;&lt;circle r="1.5" stroke="black" fill="#b3b3b3" cx="86.87290954589844" cy="10.775501251220703" stroke-width="0.1" data-type="element node" data-element-id="3" data-node-id="3" data-label-name="3"&gt;&lt;/circle&gt;</svg>
        </label>
        <label>
          <name>1</name>
          <type>points</type>
          <attributes>
          </attributes>
          <parent>s1</parent>
        </label>
        <label>
          <name>2</name>
          <type>points</type>
          <attributes>
          </attributes>
          <parent>s1</parent>
        </label>
        <label>
          <name>3</name>
          <type>points</type>
          <attributes>
          </attributes>
          <parent>s1</parent>
        </label>
      </labels>
      <segments>
        <segment>
          <id>5</id>
          <start>0</start>
          <stop>4619</stop>
          <url>http://localhost:8080/?id=5</url>
        </segment>
      </segments>
      <owner>
        <username>admin</username>
        <email></email>
      </owner>
      <original_size>
        <width>640</width>
        <height>480</height>
      </original_size>
    </task>
    <dumped>2018-09-25 16:05:07.134046+03:00</dumped>
  </meta>
  <track id="0" label="car">
    <polygon frame="0" points="324.79,213.16;323.74,227.90;347.42,237.37;371.11,217.37;350.05,190.00;318.47,191.58" outside="0" occluded="0" keyframe="1">
    </polygon>
    <polygon frame="1" points="324.79,213.16;323.74,227.90;347.42,237.37;371.11,217.37;350.05,190.00;318.47,191.58" outside="1" occluded="0" keyframe="1">
    </polygon>
    <polygon frame="6" points="305.32,237.90;312.16,207.90;352.69,206.32;355.32,233.16;331.11,254.74" outside="0" occluded="0" keyframe="1">
    </polygon>
    <polygon frame="7" points="305.32,237.90;312.16,207.90;352.69,206.32;355.32,233.16;331.11,254.74" outside="1" occluded="0" keyframe="1">
    </polygon>
    <polygon frame="13" points="313.74,233.16;331.11,220.00;359.53,243.16;333.21,283.16;287.95,274.74" outside="0" occluded="0" keyframe="1">
    </polygon>
    <polygon frame="14" points="313.74,233.16;331.11,220.00;359.53,243.16;333.21,283.16;287.95,274.74" outside="1" occluded="0" keyframe="1">
    </polygon>
  </track>
  <track id="1" label="s1" source="manual">
    <skeleton frame="0" outside="0" occluded="0" keyframe="1" points="" z_order="0">
    </skeleton>
    <skeleton frame="1" outside="1" occluded="0" keyframe="1" points="" z_order="0">
    </skeleton>
    <skeleton frame="6" outside="0" occluded="0" keyframe="1" points="" z_order="0">
    </skeleton>
    <skeleton frame="7" outside="1" occluded="0" keyframe="1" points="" z_order="0">
    </skeleton>
    <skeleton frame="13" outside="0" occluded="0" keyframe="0" points="" z_order="0">
    </skeleton>
    <skeleton frame="14" outside="1" occluded="0" keyframe="1" points="" z_order="0">
    </skeleton>
    <track id="0" label="1" source="manual" task_id="20" subset="default_1">
      <points frame="0" outside="0" occluded="0" keyframe="1" points="112.07,258.59" z_order="0">
      </points>
      <points frame="1" outside="1" occluded="0" keyframe="1" points="112.07,258.59" z_order="0">
      </points>
      <points frame="6" outside="0" occluded="0" keyframe="0" points="120.07,270.59" z_order="0">
      </points>
      <points frame="7" outside="1" occluded="0" keyframe="1" points="120.07,270.59" z_order="0">
      </points>
      <points frame="13" outside="0" occluded="0" keyframe="0" points="112.07,258.59" z_order="0">
      </points>
      <points frame="14" outside="1" occluded="0" keyframe="1" points="112.07,258.59" z_order="0">
      </points>
    </track>
    <track id="1" label="2" source="manual" task_id="20" subset="default_1">
      <points frame="0" outside="0" occluded="0" keyframe="1" points="127.87,333.23" z_order="0">
      </points>
      <points frame="1" outside="1" occluded="0" keyframe="1" points="127.87,333.23" z_order="0">
      </points>
      <points frame="6" outside="0" occluded="0" keyframe="0" points="140.87,350.23" z_order="0">
      </points>
      <points frame="7" outside="1" occluded="0" keyframe="1" points="140.87,350.23" z_order="0">
      </points>
      <points frame="13" outside="0" occluded="0" keyframe="0" points="127.87,333.23" z_order="0">
      </points>
      <points frame="14" outside="1" occluded="0" keyframe="1" points="127.87,333.23" z_order="0">
      </points>
    </track>
    <track id="2" label="3" source="manual" task_id="20" subset="default_1">
      <points frame="0" outside="0" occluded="0" keyframe="1" points="195.37,223.27" z_order="0">
      </points>
      <points frame="1" outside="1" occluded="0" keyframe="1" points="195.37,223.27" z_order="0">
      </points>
      <points frame="6" outside="0" occluded="0" keyframe="0" points="210.37,260.27" z_order="0">
      </points>
      <points frame="7" outside="1" occluded="0" keyframe="1" points="210.37,260.27" z_order="0">
      </points>
      <points frame="13" outside="0" occluded="0" keyframe="0" points="195.37,223.27" z_order="0">
      </points>
      <points frame="14" outside="1" occluded="0" keyframe="1" points="195.37,223.27" z_order="0">
      </points>
    </track>
  </track>
</annotations>