UDF Writing Guide

• Overview

• How-To Guide for Writing UDF

  • Steps for Writing Python UDFs

  • Python APIs for writing UDF

Overview

An User Defined Function (UDF) is a chunk of user code that can transform video frames and/or manipulate metadata. For example, a UDF can act as filter, preprocessor, classifier or a detector. These User Defined Functions can be developed in Python. DL Streamer Pipeline Server provides a GStreamer plugin - udfloader using which users can configure and load arbitrary UDFs. These UDFs are then called once for each video frame.

Gstreamer udfloader plugin supports loading and executing of python UDFs. Metadata/inference results generated by the UDFs are attached to GST buffer as GVAJSONMeta. A single instance of udfloader plugin can load and execute a sequence of UDFs, thus enabling chaining multiple UDFs together. Pipelines can also have multiple udfloader elements.

How-To Guide for Writing UDF

This document describes step-wise instruction for writing a User defined Function (UDF) in Python to be used with the gst udfloader element.

Ability to run arbitrary User Defined Function (UDFs) as a part of the GStreamer pipeline is one of the cardinal feature of DL Streamer Pipeline Server. It enables users to adjoin any pre-processing or post-processing logic to the video analytics pipeline. Currently UDFs can be implemented using Python.

The order in which the UDFs are defined in pipeline definition file is the order in which data will flow across them. Currently there is no support for demux/mux the data flow to/fro the UDFs.

Steps for Writing Python UDFs

This section describes the process of writing a Python UDF. As discussed in the aforementioned scenario, it also has two aspects to it.

• Writing the actual UDF.

• Adding configuration details to the gst udfloader element for deploying it

Python APIs for writing UDF

• Initialization

  In case of Python the initialization callback is also same as the native case. User must create a Udf class and the __init__() function defined in class act as a initialization routine for custom UDF. Following is the dummy constructor code:

  class Udf:
  """Example UDF
  """
  def __init__(self):
      """Constructor
      """
      # Add the initialization code in this method.
  

• Process Actual Data

  The API used to process the actual frame is as follows:

  process(self, frame, metadata):
      # Process the frame in this method
      # metadata can be used to return inference result
  

  Argument:

  frame: Image frame in numpy’s ndarray format

  metadata: An empty dictionary. Inference results can be inserted in this data structure.

  Return value:

  This function returns three values.

  1st Value : Represents if the frame Need to be dropped or Not. It is boolean in nature. In case of failure user can return True in this positional return value.

  2nd Value : It represents the actual modified frame if at all it has been modified. Hence the type is numpy’s ndarray. If the frame is not modified user can return a None in this place.

  3rd Value : Metadata is returned in this place. Hence the type is dict. In general user can return the passed argument as part of this function.