Model Registry#

Overview#

This document provides instructions on how to get started with the model registry microservice, utilize its REST API, and configure EVAM to interact with it.

The model registry microservice provides a centralized repository that can be accessed by different applications, services, or developers to store, and access models. It is an essential tool for deploying machine learning models, as it streamlines model management, fosters collaboration, and ultimately, aids in improving model deployments.

In the current release of EVAM, the following two workflows are supported.

Model download during initialization phase of EVAM from the model registry microservice. Based on user inputs in configuration file of EVAM, model is downloaded and deployed.
Model download through REST API during EVAM’s runtime. Once the model download is completed, users can use EVAM’s REST API call to STOP any pipeline and launch new pipeline with newly downloaded model. You can also automatically stop an existing pipeline instance, and start a new one with the new model after it is downloaded.

Get Started Guide#

Download the model_registry.zip file via the Intel Edge Platform Catalog
Follow the instructions in the Model Registry’s Get Started Guide to download the modules, and run the microservice.
Complete the following tutorials:

Tutorial 1: Storing and Retrieving Models in the Registry

EVAM Integration#

Pre-requisites#

In order to successfully, store models received from the model registry microservice within the context of EVAM, the following steps are required before starting the Docker* container for EVAM:

Standalone Mode#

Go to the working directory
```
cd <EVAM_WORKDIR>
```
Execute the init_mr_dir.sh utility script to create the mr_models directory with the EVAM_USER ownership.
```
cd <EVAM_WORKDIR>/utils
sudo ./init_mr_dir.sh
```

Configuration (.env)#

Required#

EVAM requires a password to authenticate successfully with the model registry microservice.

Set the value for MR_USER_PASSWORD
- MR_USER_PASSWORD (String): The password for the default account.
  - Note: This value must be the same value assigned to the USER_PASSWORD environment variable set for the model registry microservice.
    - The value for the USER_PASSWORD environment variable can be found in the model registry microservice’s .env file.

HTTPS and HTTP mode#

Model registry microservice supports both HTTPS and HTTP protocols. HTTPS is enabled by default. We need to run the following steps for HTTPS to work. When enabled in HTTPS_MODE, EVAM will attempt to verify its SSL certificate using the file(s) in the /run/secrets/ModelRegistry_Server directory within the Docker container by default.

Note: For HTTP mode (not recommended for production), ensure Model registry microservice has the environment variable ENABLE_HTTPS_MODE is set to "false" before deploying. The following steps to generate and copy certificates can be skipped in that case.

Create the Certificates/model_registry/ directory in the parent directory of the docker directory
```
cd [PARENT_DIR_of_docker_DIR]

mkdir -p Certificates/model_registry
```
- Note: The /run/secrets/ModelRegistry_Server directory in the container is mounted to the local Certificates/model_registry directory on the host system as defined in the docker-compose.yml file.
- MR_VERIFY_CERT (String): Controls whether SSL certificate verification is performed during HTTPS requests to the model registry microservice.
  - Valid options are True, False, and </path/to/CA_Bundle_file>.
    - True causes EVAM to validate the model registry’s certificate’s chain of trust, checks its expiration date and verify its hostname.
    - False causes EVAM to ignore verifying the SSL certificate. This may be useful during testing, but not advised for production.
    - </path/to/CA_Bundle_file> specifies the path to a CA_BUNDLE file.
  - Example: MR_VERIFY_CERT=False
  - Default Value: /run/secrets/ModelRegistry_Server/ca-bundle.crt
Navigate to the model registry’s Certificates/ssl directory
```
cd <path/to>/Certificates/ssl
```
Create a CA BUNDLE file from the model registry’s server-ca.crt and server.crt files in its Certificates/ssl directory
```
sudo cat server-ca.crt server.crt > ca-bundle.crt
```
Move (DO NOT Copy) the newly created ca-bundle.crt file from the model registry’s Certificates/ssl directory to EVAM’s Certificates/model_registry/ directory.
- Note: By default, EVAM requires the ca-bundle.crt file when sending requests to the model registry to verify its SSL certificate.
- The ca-bundle.crt file is required for EVAM and should not be kept in the model registry’s Certificates/ssl directory when its containers are started. It will lead to SSL certificate verification issues between the model registry and its dependent containers.
```
sudo mv ca-bundle.crt <path/to>/Certificates/model_registry/
```

Configuration (config.json)#

EVAM requires the following configuration properties to search, retrieve and store a model locally from the model registry microservice: The following configuration applies to both the supported protocols HTTPS(default) and HTTP. Replace the <PROTOCOL> references in steps as https or http accordingly based on the protocol supported by model registry microservice at the time of deployment i.e. env variable ENABLE_HTTPS_MODE is set to "true" or "false".

model_registry (Object): The properties used to connect to the model registry microservice and the directory to save models locally within the context of EVAM.
- Location: Within the config object.
- Supported sub-properties:
  - url (String): The service’s IP address or hostname and port of the running model registry microservice.
    - Example: "url": "<PROTOCOL>://10.101.10.101:32002"
  - request_timeout (Number, Optional): The maximum amount of time in seconds that requests involving the model registry microservice are allowed to take.
    - More details: If not provided, a default value of 300 seconds will be applied.
  - saved_models_dir (String): The directory where models are saved when retrieved from the model registry microservice.
    - More details: If this directory does not exist, it will be created when a model is being saved for the first time.
    - Example: "saved_models_dir": "./mr_models"
model_params (Object): The properties used to retrieve a model stored in the model registry microservice.
- Location: Within an object in the "config.pipelines" list.
- Supported sub-properties:
  - name (String, Optional): The name associated to a model.
    - Example: "name": "pallet-detection-FP32-YOLO"
  - project_name (String, Optional): The name of the project associated to a model.
    - Example: "project_name": "pallet-detection"
  - version (String, Optional): The version of a model.
    - Example: "version": "v1.0"
  - category (String, Optional): The category of a model.
    - Example: "category": "Detection"
  - architecture (String, Optional): The architecture of a model.
    - Example: "model_type": "YOLO"
  - precision (String, Optional): The precision of a model.
    - Example: "precision": "FP32"
- Note: The query performed is an AND search if more than 1 sub-property is provided. Despite all the sub-properties being optional, EVAM requires at least 1 sub-property to execute a query.
deployment (String): The path to the deployment directory for the saved model retrieved from the model registry microservice.
- More details: The directory path should be defined as {config.model_registry.saved_models_dir}/{config.pipelines[...].name}_m-{model_params.version}_{model_params.precision}/deployment
- Location: Within an object in the "config.pipelines[...].udfs.udfloader[...]" list.
- Example: "deployment": "./mr_models/pallet_detection_m-v1_fp32/deployment"

Example#

Replace the <PROTOCOL> accordingly

{
    "config": {
        "model_registry": {
            "url": "<PROTOCOL>://10.101.10.101:32002",
            "request_timeout": 300,
            "saved_models_dir": "./mr_models"
        },
        "pipelines": [
            {
                "name": "pallet_detection",
                "model_params": {
                    "project_name": "pallet-detection",
                    "version": "v1",
                    "category": "Detection",
                    "architecture": "YOLO",
                    "precision": "FP32"
                },
                "udfs": {
                    "udfloader": [
                        {
                            "deployment": "./mr_models/pallet_detection_m-v1_fp32/deployment",
                        }
                    ]
                }
            }
        ]
    },
    "interfaces": {
    }
}

Note: The above example is for educational purposes. It denotes the properties relevant to EVAM’s interactions with a running instance of the model registry microservice. This is not a functional JSON file.

REST API Endpoint#

The REST API endpoint for downloading files associated with a specific model from the model registry microservice into EVAM is /pipelines/{name}/{version}/{instance_id}/models. This endpoint can also stop an existing pipeline instance and start a new one using the specified model. For more information, visit the REST Endpoints Reference Guide.