Object tracking with UDF#

Object tracking#

Object tracking is supported by DL Streamer through gvatrack element which assigns ids to each uniquely identified objects. It is typically inserted into a video pipeline after gvadetect element.

There are several tracking types supported, for example, short-term-imageless, zero-term, zero-term-imageless. By default, zero-term tracking is enabled which requires object detection to be run on every frame.

More details object tracking, tracking types, performance considerations, examples can be found here.

Usage#

Note: By default, tracking is disabled.

To enable object tracking in a video pipeline that has a UDFLoader element, two elements gvapython and gvatrack need to be inserted after the udfloader element in the pipeline. When tracking is enabled, the metadata will include the object ids starting from sequentially from 1 for each detection.

  1. gvapython element adds detected ROIs to video frame. This is required by the tracking element to assign object IDs.

  2. gvatrack adds tracking info.

To disable tracking, the above elements (if present) following the udfloader element need to be removed from the pipeline. When tracking is disabled, the metadata will include the object id as None for each detection.

Here is an example for enabling tracking for Pallet Defect Detection pipeline that uses Geti UDF. In Geti UDF, metadata_converter should be set to geti_to_dcaas.

"pipeline": "multifilesrc loop=TRUE stop-index=0 location=/home/pipeline-server/resources/videos/warehouse.avi name=source ! h264parse ! decodebin ! queue max-size-buffers=10 ! videoconvert ! video/x-raw,format=RGB ! udfloader name=udfloader ! gvapython class=AddDetectionRoi function=process module=/home/pipeline-server/gvapython/detection/add_roi.py name=add_roi ! gvatrack tracking-type=short-term-imageless ! appsink name=destination",

   "udfs": {
      "udfloader": [
        {
          "name": "python.geti_udf.geti_udf",
          "type": "python",
          "device": "CPU",
          "visualize": "true",
          "deployment": "./resources/models/geti/pallet_defect_detection/deployment",
          "metadata_converter": "geti_to_dcaas"
        }
      ]
    }

Specific metadata format is expected out of the UDF for it work with the gvapython that adds ROIs.

Recommendations#

  • Tracking accuracy is reliant on model’s accuracy in detecting objects. For skipped detections, other tracking types like short-term-imageless may be explored which does not expect detection on every frame.

  • When running multiple pipelines with tracking enabled, the object ids would still sequentially start from 1 for each pipeline. It can be coupled with either pipeline tag or unique published topic identifying results associated with a specific pipeline.

Known Issues#

Currently, visualizer does not show the object IDs along with the overlayed detection/labels.