This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Enterprise Deployment

Deployment options and infrastructure guides for CVAT Enterprise.

1: Deployment with Docker Compose
2: Deployment on Kubernetes

1 - Deployment with Docker Compose

Instructions for deploying CVAT Enterprise using Docker Compose.

AWS requirements to create, run and manage VMs

IAM user should have the following permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeInstances",
                "ec2:DescribeImages",
                "ec2:DescribeInstanceTypes",
                "ec2:DescribeKeyPairs",
                "ec2:DescribeVpcs",
                "ec2:DescribeSubnets",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeSecurityGroupRules",
                "ec2:DescribeVolumes",
                "ec2:DescribeAvailabilityZones",
                "ec2:CreateSecurityGroup",
                "ec2:AuthorizeSecurityGroupIngress",
                "ec2:AuthorizeSecurityGroupEgress",
                "ec2:RevokeSecurityGroupIngress",
                "ec2:RevokeSecurityGroupEgress",
                "ec2:ModifySecurityGroupRules",
                "ec2:UpdateSecurityGroupRuleDescriptionsIngress",
                "ec2:UpdateSecurityGroupRuleDescriptionsEgress",
                "ec2:CreateKeyPair",
                "ec2:CreateTags",
                "ec2:DescribeAddresses",
                "ec2:AllocateAddress",
                "ec2:AssociateAddress",
                "ec2:RunInstances",
                "elasticfilesystem:DescribeFileSystems",
                "elasticfilesystem:CreateFileSystem",
                "elasticfilesystem:TagResource",
                "elasticfilesystem:CreateMountTarget",
                "elasticfilesystem:DescribeMountTargets"
            ],
            "Resource": "*"
        }
    ]
}

Currently, the Amazon EC2 Describe* API actions do not support resource-level permissions, so you cannot restrict which individual resources users can view. However, you can apply resource-level permissions on the ec2:RunInstances API action to restrict which resources users can use to launch an instance. The launch fails if users select options that they are not authorized to use. See this guide for details.

Prepare the environment to obtain CVAT Enterprise images (Customer side actions).

How to configure the environment in case non AWS EC2 instance

Install AWS CLI. See this guide for additional details. Role name may be like CvatEnterpriseCustomer or any name which you prefer.

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

Install docker

Create aws config folder and config file (Note: the current file will be overwritten):

mkdir -p ~/.aws
cat <<EOF > ~/.aws/config
[profile CvatEnterpriseCustomer]
role_arn = <MUST BE PROVIDED BY CVAT TEAM>
source_profile=CvatEnterpriseCustomer
external_id = <MUST BE PROVIDED BY CVAT TEAM>

EOF

Create aws credentials file (Note: the current file will be overwritten):

cat <<EOF > ~/.aws/credentials
[CvatEnterpriseCustomer]
aws_access_key_id = <MUST BE PROVIDED BY CVAT TEAM>
aws_secret_access_key = <MUST BE PROVIDED BY CVAT TEAM>

EOF

Verify that the Docker login command succeeds by running the following command:

aws ecr get-login-password --region eu-west-1 --profile CvatEnterpriseCustomer | docker login --username AWS --password-stdin <MUST BE PROVIDED BY CVAT TEAM>

How to configure the environment in case AWS EC2 instance

Create an IAM instance profile for Amazon EC2 instance with the following see this guide for additional details.

Trusted entities:

   {
"Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Statement1",
      "Effect": "Allow",
      "Principal": {
        "Service": "ec2.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Policy

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowToAssumeCrossAccountRole",
      "Effect": "Allow",
      "Action": "sts:AssumeRole",
      "Resource": "<MUST BE PROVIDED BY CVAT TEAM>"
    }
  ]
}

Provide CVAT team role arn (arn:aws:iam::123456789000:role/AnyPreferableRoleName) to provide access to the CVAT ECR.
Create an EC2 instance and attach the IAM role from the step above to the EC2 instance.
Login to the instance with SSH.
Install AWS CLI. See this guide for additional details. Role name may be like CvatEnterpriseCustomer or any name which you prefer.
```
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
```
Install docker

Create aws config folder and config file:

mkdir ~/.aws
cat <<EOF > ~/.aws/config
[profile CvatEnterpriseCustomer]
role_arn = <MUST BE PROVIDED BY CVAT TEAM>
credential_source = Ec2InstanceMetadata

EOF

Verify that the Docker login command succeeds by running the following command:

aws ecr get-login-password --region eu-west-1 --profile CvatEnterpriseCustomer | docker login --username AWS --password-stdin <MUST BE PROVIDED BY CVAT TEAM>

Steps how to deploy CVAT on customer managed instance (both EC2 and non EC2)

Customer side actions

Clone CVAT repo

git clone https://github.com/cvat-ai/cvat.git && cd cvat

Place the docker-compose.enterprise.yml configuration file that you should receive from CVAT team.

Modify docker-compose.enterprise.yml if needed (e.g. to change default directory to store all CVAT related data). Please consult with CVAT team if you have any questions.

volumes:
  cvat_data:
    driver_opts:
      type: none
      device: /mnt/cvat/data
      o: bind

  cvat_db:
    driver_opts:
      type: none
      device: /mnt/cvat/db
      o: bind

  cvat_keys:
    driver_opts:
      type: none
      device: /mnt/cvat/keys
      o: bind

  cvat_logs:
    driver_opts:
      type: none
      device: /mnt/cvat/logs
      o: bind

  cvat_events:
    driver_opts:
      type: none
      device: /mnt/cvat/events
      o: bind

  cvat_cache_db:
    driver_opts:
      type: none
      device: /mnt/cvat/cache
      o: bind

To simplify deploying, use the following shell script. Don’t forget to change the CVAT_HOST variable in the file (it should be FQDN).

#!/usr/bin/env bash
set -e

aws ecr get-login-password --region eu-west-1 --profile CvatEnterpriseCustomer | docker login --username AWS --password-stdin <MUST BE PROVIDED BY CVAT TEAM>

export CVAT_HOST=\<CUSTOM_DOMAIN\>
export ACME_EMAIL=support@cvat.ai
export CVAT_VERSION='v2.41.0'

git fetch origin
git checkout ${CVAT_VERSION}

docker compose \
-f docker-compose.yml \
-f docker-compose.enterprise.yml \
"$@"

Run cvat with the following command:
```
./docker-compose.sh up -d
```

Create a superuser:

docker exec -it cvat_server bash -ic 'python3 ~/manage.py createsuperuser'

2 - Deployment on Kubernetes

Guide for deploying CVAT Enterprise on a Kubernetes cluster.

Prerequisites

A Kubernetes cluster kubeVersion >= 1.23.0-0
kubectl and Helm installed and configured to use the cluster.
The CVAT Enterprise Docker images (server and UI).
A private Docker registry, accessible by the Kubernetes cluster. It will be referred to as registry.example below.
RWX StorageClass must be configured in the cluster.

Deployment steps

Download the Docker images for the desired release.

Install AWS CLI. See this guide for additional details. Role name may be like CvatEnterpriseCustomer or any name which you prefer.

 curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
 unzip awscliv2.zip
 sudo ./aws/install

Install docker

Create aws config folder and config file (Note: the current file will be overwritten)

 mkdir -p ~/.aws
 cat <<EOF > ~/.aws/config
 [profile CvatEnterpriseCustomer]
 role_arn = <MUST BE PROVIDED BY CVAT TEAM>
 source_profile=CvatEnterpriseCustomer
 external_id = <MUST BE PROVIDED BY CVAT TEAM>

 EOF

Create aws credentials file (Note: the current file will be overwritten):

cat <<EOF > ~/.aws/credentials
[CvatEnterpriseCustomer]
aws_access_key_id = <MUST BE PROVIDED BY CVAT TEAM>
aws_secret_access_key = <MUST BE PROVIDED BY CVAT TEAM>

EOF

Verify that the Docker login command succeeds by running the following command:

aws ecr get-login-password --region eu-west-1 --profile CvatEnterpriseCustomer | docker login --username AWS --password-stdin <MUST BE PROVIDED BY CVAT TEAM>

Pull server and ui images

docker pull <MUST BE PROVIDED BY CVAT TEAM>/cvat/server_ent:vX.Y.Z
docker pull <MUST BE PROVIDED BY CVAT TEAM>/cvat/ui_ent:vX.Y.Z

Upload the images to the registry:

docker login registry.example

docker tag <MUST BE PROVIDED BY CVAT TEAM>/cvat/server_ent:vX.Y.Z registry.example/cvat/server_ent:vX.Y.Z
docker tag <MUST BE PROVIDED BY CVAT TEAM>/cvat/ui_ent:vX.Y.Z registry.example/cvat/ui_ent:vX.Y.Z
docker push registry.example/cvat/server_ent:vX.Y.Z
docker push registry.example/cvat/ui_ent:vX.Y.Z

Check out the open source CVAT repository at the tag corresponding to the desired release.
Unpack the archive with the enterprise chart:
```
tar -xvzf cvat_enterprise-*.tgz
```
Create a file named values.override.yml with the necessary settings, which must contain at least the override for CVAT image registry, as shown in the example below. All supported settings can be obtained in the community version of values.yaml, which is used as a subchart of the enterprise chart, and in the enterprise chart archive, which you should obtain from the CVAT team along with these instructions.

Add Enterprise-specific settings to values.override.yml:

cvat:
  cvat:
    backend:
      image: registry.example/cvat/server_ent
      imagePullPolicy: IfNotPresent

    frontend:
      image: registry.example/cvat/ui_ent
      imagePullPolicy: IfNotPresent

Deploy a Helm release using the chart in the CVAT repository and values.override.yml:

helm upgrade -n dev-ent dev-ent -i --create-namespace ./cvat_enterprise -f ./values.override.yaml

Create an auth_config.yml file as described in SSO configuration.

Upload it as a secret to the Kubernetes cluster:

kubectl create secret generic cvat-auth-config --from-file=auth_config.yml

Deploy as in the previous section, but with the following settings added to values.override.yml:

cvat:
  cvat:
    backend:
      permissionFix:
        enabled: false

      server:
        additionalVolumes:
        - name: auth-config
          secret:
            secretName: cvat-auth-config
        additionalVolumeMounts:
        - mountPath: /home/django/auth_config.yml
          name: auth-config
          subPath: auth_config.yml

Enterprise Deployment

1 - Deployment with Docker Compose

AWS requirements to create, run and manage VMs

Prepare the environment to obtain CVAT Enterprise images (Customer side actions).

How to configure the environment in case non AWS EC2 instance

How to configure the environment in case AWS EC2 instance

Steps how to deploy CVAT on customer managed instance (both EC2 and non EC2)

Customer side actions

2 - Deployment on Kubernetes

Prerequisites

Deployment steps

Enabling social account authentication or SSO