Using Predictors

In Three Simple Steps

Computer vision is perhaps the most common use case for machine learning. ML allows for solving some interesting vision tasks, like image classification, detection, segmentation, and so on. Across all of these tasks, the NatML workflow is relatively uniform:

Creating a Predictor

First, we create a model:

// Fetch the SSD Lite object detection model
var model = await MLEdgeModel.Create("@natsuite/ssd-lite");

We can log the model to see what data it consumes and produces:

// Inspect the model
Debug.Log(model);
// Prints:
// MLEdgeModel
// Input: input.1: (1, 3, 300, 300) System.Single <-- INPUT IMAGE
// Output: 836: (1, 3000, 21) System.Single
// Output: 863: (1, 3000, 4) System.Single

Then we instantiate the predictor class for the model:

// Create the SSD Lite predictor
var predictor = new SSDLitePredictor(model);

When using models from NatML Hub, the predictor page will typically provide a link to a predictor package that you can import and use in your Unity project.

If you are using your own model file, you will have to write a predictor class implementation for it.

You will always want to create the model and predictor once, usually in your Start method, because it is a time-consuming process.

Making Predictions

To make a prediction, we simply need to call Predict on the predictor:

// Given an image...
Texture2D image = ...;
// Make a prediction
var results = predictor.Predict(imageFeature);

Some predictors might require multiple inputs. See the README of your predictor for what features to pass in.

Disposing the Predictor

Some predictors require explicit disposal when they are no longer needed. These predictors define a Dispose method:

// Dispose the predictor
predictor.Dispose(); // this may or may not exist, so refer to your predictor README

All predictors (i.e. all types that inherit IMLPredictor) are actually disposable. But most predictors will hide the Dispose method if it is not needed.

Once the predictor is disposed, the model MUST be disposed.

// Dispose the model
model.Dispose();