IMLAsyncPredictor

interface IMLAsyncPredictor<TOutput>

In NatML, async predictors are lightweight primitives that make Hub (server-side) predictions with one or more MLModel instances. Predictors play a very crucial role in using ML models, because of their two primary purposes:

• Predictors provide models with input data for prediction.

• Predictors convert model outputs to a form that is usable by developers.

You will typically never have to implement IMLAsyncPredictor yourself. Instead, discover existing Hub predictors on NatML Hub.

Defining the Predictor

All Hub predictors must implement this interface. The predictor has a single generic type argument, TOutput, which is a developer-friendly type that is returned when a prediction is made. For example, a ResNet18HubPredictor for the ResNet18 image classifier model will use a tuple for its output type:

// The ResNet18 classification predictor returns a (label, score) tuple
class ResNet18HubPredictor implements IMLAsyncPredictor<[string, number]> { ... }

Hub predictor class names should always end with "HubPredictor".

Writing the Constructor

All Hub predictors must define one or more constructors that accept one or more MLModel instances, along with any other predictor data needed to make predictions with the model(s). For example:

/**
 * Create a predictor
 * @param model ML model used to make predictions.
 */
public constructor (model: MLModel) { ... }

Within the constructor, the model should store a readonly reference to the model(s). The type of this reference should be MLHubModel:

// Define the `model` member
private readonly model: MLHubModel;
// And in the constructor...
constructor (model: MLModel) {
    this.model = model as MLHubModel;
}

The MLHubModel class extends MLModel, and exposes a predict method for making Hub predictions.

Making Predictions

All Hub predictors must implement a public predict method which accepts a variadic MLFeature[] and returns a Promise<TOutput>:

/**
 * Make a prediction on one or more input features.
 * @param inputs Input features.
 * @returns Prediction output.
 */
public async predict (...inputs: MLFeature[]): Promise<TOutput>;

Within the predict method, the predictor should do three things:

Input Checking

The predictor should check that the client has provided the correct number of input features, and that the features have the model's expected types.

If these checks fail, an appropriate exception should be thrown. Do this instead of returning an un-initialized output.

Prediction

INCOMPLETE

Marshaling

Once you have raw output features from the model, you can then marshal the feature data into a more developer-friendly type. This is where most of the heavy-lifting happens in a predictor:

// Marshal the output feature data into a developer-friendly type
const outputArrayFeature = new MLArrayFeature<Float32Array>(rawOutputFeatures[0]);
// Do stuff with this data...
...

Finally, return your predictor's output:

// Create the prediction result from the output data
const result: TOutput = ...;
// Return it
return result;

Disposing the Predictor

Hub predictors may define a dispose method. This method should be used to dispose any explicitly-managed resources used by the predictor, like recurrent state for recurrent models.

The predictor must not dispose any models provided to it. This is the responsibility of the client.