NatML
Search…
Fetching Models
Keeping your App Binary Small
The very first step in using ML in your app is fetching an ML model. NatML supports fetching models from different sources, and this functionality is encapsulated in the MLModelData class.

Fetching from Hub

NatML Hub is a model delivery service built specifically for NatML. Hub uses high-speed edge networks to deliver ML models to your users when and where they need it:
1
// Fetch model data from NatML Hub
2
var modelData = await MLModelData.FromHub("@natsuite/tiny-yolo-v3");
3
// Create a model from model data
4
var model = modelData.Deserialize();
Copied!
Models are loaded from Hub and cached on device, so your users only ever have to download the model once.
NatML Hub provides several benefits over using ML model files. First, it allows you to distribute smaller app binaries and stay under app store size limits, because most ML models can be anywhere from 20MB-200MB. Furthermore, Hub is able to deliver models that are specifically optimized for the device that requests them.

Importing in Unity

NatML also supports importing .onnx files into Unity, and will create MLModelData instances from them. Simply drop an .onnx file in your project and Unity will import it accordingly:
Model importer for a MobileNet ONNX file.
Once you have an MLModelData instance, you can create an MLModel from it:
1
// Create a model from model data
2
var model = modelData.Deserialize();
Copied!

Fetching from File

You can fetch model data from a local .onnx file at runtime. You can do so using an absolute path:
1
// Fetch model data from a local file
2
var modelData = await MLModelData.FromFile("/path/to/model.onnx");
3
// Create a model
4
var model = modelData.Deserialize();
Copied!
You can also fetch model data from a file in your app's StreamingAssets directory:
1
// Fetch model data from a relative path in StreamingAssets
2
var modelData = await MLModelData.FromStreamingAssets("model.onnx");
Copied!

Using Supplemental Data

The MLModelData class is more versatile than simply loading models. Usually, supplemental data is needed to properly use an ML model. For example, when using a classification model, you will need a list of class labels which correspond to the model's output probabilities. The MLModelData class encapsulates all of the information needed to both load a model, and make predictions with it.
Supplemental data is only available for models loaded from NatML Hub.

Class Labels

Object classification and detection models output probabilities corresponding to a set of predefined class labels. The MLModelData class can provide these class labels, depending on how it is fetched:
1
// Create a classification model and get its corresponding labels
2
MLModel model = modelData.Deserialize();
3
string[] labels = modelData.labels;
4
// Create a classification predictor using the model's class labels
5
var predictor = new MLClassificationPredictor(model, labels);
Copied!

Image Normalization

Some vision models require input images to have certain normalization, which is a mathematical transformation of the input data. This normalization information can be used by an MLImageFeature when creating input data for a model:
1
// Create an input feature from an image
2
Texture2D image = ...;
3
var feature = new MLImageFeature(image);
4
// Apply normalization coefficients from model data
5
(feature.mean, feature.std) = modelData.normalization;
Copied!

Image Aspect Mode

Certain vision models, like object detectors, require input images to be cropped in a specific way so as to preserve all the information in the image while making a prediction. This aspect mode can be used by an MLImageFeature when creating input data for a model:
1
// Create an input feature from an image
2
Texture2D image = ...;
3
var feature = new MLImageFeature(image);
4
// Apply the proper aspect mode expected by the model
5
feature.aspectMode = modelData.aspectMode;
Copied!
Last modified 3mo ago