Interaction with other EIS services#

Other EIS services can use metadata and blobs generated from EVAM over secure gRPC. See Interface section.

DataStore#

The DataStore microservice stores video and time-series data, supporting two types of data: JSON (InfluxDB) and blobs (MinIO object storage). EVAM publishes both frame and metadata generated during pipeline execution from supported pipeline elements, as well as any additional data from EVAM, using secure gRPC communication. When configured, DataStore can receive these data and store it.

To learn more about DataStore, click here.

Edge Data Collection#

Edge Data Collection is a microservice that allows users to annotate data and export them. It relies on DataStore as a primary data source to fetch images and annotate them. Users can add new annotations/review existing ones and save this information back to DataStore. The data can also be exported to a storage location. Depending upon the type of data, users can choose to export annotated data from DataStore in a trainable data format e.g. COCO or ImageNet for object detection and classification tasks. Data can be stored in a mounted disk or Azure.

Compatible meta data#

Features like Annotation and Remote Storage provided by Edge Data Collection requires certain fields to be present in the metadata to operate. These fields carry annotation information, track data export status and other important details. Since User Defined Functions (UDFs) are usually attached with ingestion source such as EdgeVideoAnalyticsMicroservice (EVAM), we let UDFs add these fields along with any other fields it already adds as part of its functionality. Users need to add these compatible UDFs then to EVAM’s configuration file.

Mandatory Fields#

In case you want to add your custom UDF and make it compatible with Edge Data Collection, ensure that the following fields are present in the metadata published by the UDF. Any missing key here will result in an error. Keys with sample values.

# In python UDF, process() method
"last_modified": time_ns()
"export_code": 0
"annotation_type": "auto"
"annotations": {"objects":[]}

last_modified: Mandatory. Refers to the last time the metadata was modified. The value should be in nanoseconds.

export_code: Mandatory. Refers to the status of the data export. The following export codes are supported:

  • 0: Data has not been exported

  • 1: Data has been exported

annotation_type: Mandatory. Refers to the type of annotation. The following annotation types are supported:

  • auto: for auto annotation

  • human: for human annotation

annotations: Mandatory. Refers to the annotations for objects detection in the image.

objects: A list of objects detected in the image. If present, each object should have the following keys:

  • label: The label of the object

  • score: The confidence score of the object

  • bbox: The bounding box of the object. It is list of bbox coordinates in top-left and bottom-right format. For example, [x1, y1, x2, y2]

It is important that the objects within the annotations dictionary have bounding box dimensions in top-left and bottom-right format. Some models may have bounding box predicted as top-left coordinates, width and height. In that case, the UDF must convert it to the expected top-left and bottom-right format to avoid incorrect bounding box dimensions during data export or visualization.

Classification label fields#

The annotations key value pair only carries bounding box information from an object detection model. However, if the UDF is a classification model, ensure that a classification label key field with score value is also present in the metadata published by the UDF in addition to the mandatory ones above. For example: A UDF that classifies an image to be of Person class with a confidence score of 0.45, the published metadata must have the key-value pair- "Person": 0.45.

Sample compatible UDFs#

Certain sample UDFs are already provided with EVAM that adds these metadata fields making them Edge Data Collection compatible.

  • Anomalib UDF

  • Person Detection UDF

To learn more about Edge Data Collection, click here.

Visualizer#

Visualizer is a streaming microservice to visualize live frames(annotated/unannotated) published by EVAM in a web browser To learn more about Visualizer, click here.