TFRecord

The TFRecord format is tightly integrated with TensorFlow and is commonly used for training models within the TensorFlow ecosystem.

TFRecord is an incredibly flexible data format. We strive to align our implementation with the format employed by the TensorFlow Object Detection API, making only minimal changes as necessary.

For more information, see:

• TFRecord
• Dataset examples

This format does not have a fixed structure, so in CVAT the following structure is used:

image_feature_description = {
    'image/filename': tf.io.FixedLenFeature([], tf.string),
    'image/source_id': tf.io.FixedLenFeature([], tf.string),
    'image/height': tf.io.FixedLenFeature([], tf.int64),
    'image/width': tf.io.FixedLenFeature([], tf.int64),
    # Object boxes and classes.
    'image/object/bbox/xmin': tf.io.VarLenFeature(tf.float32),
    'image/object/bbox/xmax': tf.io.VarLenFeature(tf.float32),
    'image/object/bbox/ymin': tf.io.VarLenFeature(tf.float32),
    'image/object/bbox/ymax': tf.io.VarLenFeature(tf.float32),
    'image/object/class/label': tf.io.VarLenFeature(tf.int64),
    'image/object/class/text': tf.io.VarLenFeature(tf.string),
}

TFRecord export

For export of images:

• Supported annotations: Bounding Boxes, Polygons (as masks, manually over Datumaro)
• Attributes: Not supported.
• Tracks: Not supported.

The downloaded file is a .zip archive with the following structure:

taskname.zip/
├── default.tfrecord
└── label_map.pbtxt

# label_map.pbtxt
item {
	id: 1
	name: 'label_0'
}
item {
	id: 2
	name: 'label_1'
}
...

How to export masks:

1. Export annotations in Datumaro format.

2. Apply polygons_to_masks and boxes_to_masks transforms:

   datum transform -t polygons_to_masks -p path/to/proj -o ptm
   datum transform -t boxes_to_masks -p ptm -o btm
   

3. Export in the TF Detection API format:

   datum export -f tf_detection_api -p btm [-- --save-images]
   

TFRecord import

Uploaded file: a zip archive of following structure:

taskname.zip/
└── <any name>.tfrecord

• supported annotations: Rectangles

How to create a task from TFRecord dataset (from VOC2007 for example)

1. Create label_map.pbtxt file with the following content:

item {
    id: 1
    name: 'aeroplane'
}
item {
    id: 2
    name: 'bicycle'
}
item {
    id: 3
    name: 'bird'
}
item {
    id: 4
    name: 'boat'
}
item {
    id: 5
    name: 'bottle'
}
item {
    id: 6
    name: 'bus'
}
item {
    id: 7
    name: 'car'
}
item {
    id: 8
    name: 'cat'
}
item {
    id: 9
    name: 'chair'
}
item {
    id: 10
    name: 'cow'
}
item {
    id: 11
    name: 'diningtable'
}
item {
    id: 12
    name: 'dog'
}
item {
    id: 13
    name: 'horse'
}
item {
    id: 14
    name: 'motorbike'
}
item {
    id: 15
    name: 'person'
}
item {
    id: 16
    name: 'pottedplant'
}
item {
    id: 17
    name: 'sheep'
}
item {
    id: 18
    name: 'sofa'
}
item {
    id: 19
    name: 'train'
}
item {
    id: 20
    name: 'tvmonitor'
}

1. Use create_pascal_tf_record.py

to convert VOC2007 dataset to TFRecord format. As example:

python create_pascal_tf_record.py --data_dir <path to VOCdevkit> --set train --year VOC2007 --output_path pascal.tfrecord --label_map_path label_map.pbtxt

1. Zip train images

   cat <path to VOCdevkit>/VOC2007/ImageSets/Main/train.txt | while read p; do echo <path to VOCdevkit>/VOC2007/JPEGImages/${p}.jpg  ; done | zip images.zip -j -@
   

2. Create a CVAT task with the following labels:

   aeroplane bicycle bird boat bottle bus car cat chair cow diningtable dog horse motorbike person pottedplant sheep sofa train tvmonitor
   

   Select images. zip as data. See Creating an annotation task guide for details.

3. Zip pascal.tfrecord and label_map.pbtxt files together

   zip anno.zip -j <path to pascal.tfrecord> <path to label_map.pbtxt>
   

4. Click Upload annotation button, choose TFRecord 1.0 and select the zip file

   with labels from the previous step. It may take some time.