IMLAsyncPredictor
interface IMLAsyncPredictor<TOutput>
In NatML, async predictors are lightweight primitives that make Hub (server-side) predictions with one or more MLModel
instances. Predictors play a very crucial role in using ML models, because of their two primary purposes:
Predictors provide models with input data for prediction.
Predictors convert model outputs to a form that is usable by developers.
You will typically never have to implement IMLAsyncPredictor
yourself. Instead, discover existing Hub predictors on NatML Hub.
Defining the Predictor
All Hub predictors must implement this interface. The predictor has a single generic type argument, TOutput
, which is a developer-friendly type that is returned when a prediction is made. For example, a ResNet18HubPredictor
for the ResNet18 image classifier model will use a tuple for its output type:
Hub predictor class names should always end with "HubPredictor"
.
Writing the Constructor
All Hub predictors must define one or more constructors that accept one or more MLModel
instances, along with any other predictor data needed to make predictions with the model(s). For example:
Within the constructor, the model should store a readonly
reference to the model(s). The type of this reference should be MLHubModel
:
The MLHubModel
class extends MLModel
, and exposes a predict
method for making Hub predictions.
Making Predictions
All Hub predictors must implement a public predict
method which accepts a variadic MLFeature[]
and returns a Promise<TOutput>
:
Within the predict
method, the predictor should do three things:
Input Checking
The predictor should check that the client has provided the correct number of input features, and that the features have the model's expected types.
If these checks fail, an appropriate exception should be thrown. Do this instead of returning an un-initialized output.
Prediction
INCOMPLETE
Marshaling
Once you have raw output features from the model, you can then marshal the feature data into a more developer-friendly type. This is where most of the heavy-lifting happens in a predictor:
Finally, return your predictor's output:
Disposing the Predictor
Hub predictors may define a dispose
method. This method should be used to dispose any explicitly-managed resources used by the predictor, like recurrent state for recurrent models.
The predictor must not dispose
any models provided to it. This is the responsibility of the client.
Last updated