Development environment

Installing a development environment for different operating systems.

Setup the dependencies:

  • Install necessary dependencies:

    Ubuntu 18.04

    sudo apt-get update && sudo apt-get --no-install-recommends install -y build-essential curl git redis-server python3-dev python3-pip python3-venv python3-tk libldap2-dev libsasl2-dev
    
    # Install Node.js 16 and yarn
    curl -sL https://deb.nodesource.com/setup_16.x | sudo -E bash -
    sudo apt-get install -y nodejs
    sudo npm install --global yarn
    

    MacOS 10.15

    brew install git python pyenv redis curl openssl node sqlite3 geos
    

    Arch Linux

    # Update the system and AUR (you can use any other AUR helper of choice) first:
    sudo pacman -Syyu
    pikaur -Syu
    
    # Install the required dependencies:
    sudo pacman -S base-devel curl git redis cmake gcc python python-pip tk libldap libsasl pkgconf ffmpeg geos openldap
    
    # CVAT supports only Python 3.9, so install it if you don’t have it:
    pikaur -S python39
    
    # Install Node.js 16, yarn and npm
    sudo pacman -S nodejs-lts-gallium yarn npm
    
  • Install Chrome

  • Install VS Code.

  • Install the following VScode extensions:

  • Make sure to use Python 3.9.0 or higher

    python3 --version
    
  • Install CVAT on your local host:

    git clone https://github.com/cvat-ai/cvat
    cd cvat && mkdir logs keys
    python3 -m venv .env
    . .env/bin/activate
    pip install -U pip wheel setuptools
    pip install -r cvat/requirements/development.txt
    

    Note that the .txt files in the cvat/requirements directory have pinned dependencies intended for the main target OS/Python version (the one used in the main Dockerfile). If you’re unable to install those dependency versions, you can substitute the corresponding .in files instead. That way, you’re more likely to be able to install the dependencies, but their versions might not correspond to those used in production.

    Note for Mac users

    If you have any problems with installing dependencies from cvat/requirements/*.txt, you may need to reinstall your system python In some cases after system update it can be configured incorrectly and cannot compile some native modules

    Make sure Homebrew lib path is in DYLD_LIBRARY_PATH. For Apple Silicon: export DYLD_LIBRARY_PATH=/opt/homebrew/lib:$DYLD_LIBRARY_PATH

    Homebrew will install FFMpeg 5.0 by default, which does not work, so you should install 4.X. You can install older 4.X FFMpeg using Homebrew like that:

     cd "$(brew --repo homebrew/core)"
     git checkout addd616edc9134f057e33694c420f4900be59db8
     brew unlink ffmpeg
     HOMEBREW_NO_AUTO_UPDATE=1 brew install ffmpeg
     git checkout master
    

    if you are still facing error Running setup.py install for av ... error, you may try more radical variant

     cd "$(brew --repo homebrew/core)"
     git checkout addd616edc9134f057e33694c420f4900be59db8
     brew uninstall ffmpeg --force
     HOMEBREW_NO_AUTO_UPDATE=1 brew install ffmpeg
     git checkout master
    

    If you faced with error Failed building wheel for h5py, you may need install hdf5

    brew install hdf5
    export HDF5_DIR="$(brew --prefix hdf5)"
    pip install --no-binary=h5py h5py
    

    If you faced with error OSError: Could not find library geos_c or load any of its variants ['libgeos_c.so.1', 'libgeos_c.so']. You may fix this using

    sudo ln -s /opt/homebrew/lib/libgeos_c.dylib /usr/local/lib
    

    On Mac with Apple Silicon (M1) in order to install TensorFlow you will have to edit cvat/requirements/base.txt. Change tensorflow to tensorflow-macos May need to downgrade version Python to 3.9.* or upgrade version tensorflow-macos

    Note for Arch Linux users:

    Because PyAV as of version 10.0.0 already works with FFMPEG5, you may consider changing the av version requirement in /cvat/cvat/requirements/base.txt to 10.0.0 or higher.

    Perform this action before installing cvat requirements from the list mentioned above.

  • Install Docker Engine and Docker-Compose

  • Pull and run Open Policy Agent docker image:

     docker run -d --rm --name cvat_opa_debug -p 8181:8181 openpolicyagent/opa:0.45.0-rootless \
     run --server --set=decision_logs.console=true --set=services.cvat.url=http://host.docker.internal:7000 \
     --set=bundles.cvat.service=cvat --set=bundles.cvat.resource=/api/auth/rules
    
  • Pull and run PostgreSQL docker image:

    docker run --name cvat_db_debug -e POSTGRES_HOST_AUTH_METHOD=trust -e POSTGRES_USER=root \
    -e POSTGRES_DB=cvat -p 5432:5432 -d postgres
    

    Note: use docker start/stop cvat_db_debug commands to start and stop the container. If it is removed, data will be removed together with the container.

  • Apply migrations and create a super user for CVAT:

    python manage.py migrate
    python manage.py collectstatic
    python manage.py createsuperuser
    
  • Install npm packages for UI (run the following command from CVAT root directory):

    yarn --frozen-lockfile
    

    Note for Mac users

    If you faced with error

    Node Sass does not yet support your current environment: OS X 64-bit with Unsupported runtime (57)

    Read this article Node Sass does not yet support your current environment

Run CVAT

  • Start npm UI debug server (run the following command from CVAT root directory):

    • If you want to run CVAT in localhost:
      yarn run start:cvat-ui
      
    • If you want to access CVAT from outside of your host:
      CVAT_UI_HOST='<YOUR_HOST_IP>' yarn run start:cvat-ui
      
  • Open a new terminal window.

  • Run VScode from the virtual environment (run the following command from CVAT root directory):

    source .env/bin/activate && code
    
  • Inside VScode, Open CVAT root dir

  • Select server: debug configuration and run it (F5) to run REST server and its workers

  • Make sure that Uncaught Exceptions option under breakpoints section is unchecked

  • If you choose to run CVAT in localhost: Select server: chrome configuration and run it (F5) to open CVAT in Chrome

  • Alternative: If you changed CVAT_UI_HOST just enter <YOUR_HOST_IP>:3000 in your browser.

You have done! Now it is possible to insert breakpoints and debug server and client of the tool. Instructions for running tests locally are available here.

Note for Windows users

You develop CVAT under WSL (Windows subsystem for Linux) following next steps.

  • Install WSL using this guide.

  • Following this guide install Ubuntu 18.04 Linux distribution for WSL.

  • Run Ubuntu using start menu link or execute next command

    wsl -d Ubuntu-18.04
    
  • Run all commands from this installation guide in WSL Ubuntu shell.

  • You might have to manually start the redis server in wsl before you can start the configuration inside Visual Studio Code. You can do this with sudo service redis-server start. Alternatively you can also use a redis docker image instead of using the redis-server locally.

Note for Mac users

  • You might have to manually start the redis server. You can do this with redis-server. Alternatively you can also use a redis docker image instead of using the redis-server locally.

Note for Arch Linux users

  • You need to start redis and docker services manually in order to begin debugging/running tests:
    sudo systemctl start redis.service
    sudo systemctl start docker.service
    

CVAT Analytics Ports

In case you cannot access analytics, check if the following ports are open:

cvat_vector:
    ports:
      - '8282:80'

  cvat_clickhouse:
    ports:
      - '8123:8123'

In addition, you can completely disable analytics if you don’t need it by deleting the following data from launch.json:

  "DJANGO_LOG_SERVER_HOST": "localhost",
  "DJANGO_LOG_SERVER_PORT": "8282"

Analytics on GitHub: Analytics Components