fp16) or integers. Though model quantization is not a silver bullet for performance, it typically results in better performance on certain accelerators.
MLModelDatais fetched from Hub, Hub has several choices to make which can highly influence the runtime performance of the model. Some of these choices include:
fp16) or integer quantization.
MLFeatureimplementations already provide highly-optimized routines for converting input features into prediction-ready data. If your predictor cannot benefit from these routines, then you should attempt write highly-parallelized conversion code. For this, we highly recommend using Unity's Burst compiler or other SIMD routines.
MLFeature, whether by using an implicit conversion or by using a specific feature constructor, no computation is expected to happen. As such, features are very lightweight objects that can be used liberally.
MLAsyncPredictorwhich is a wrapper around any existing predictor for this purpose:
Disposethe predictor when you are done with it, so as not to leave threads and other resources dangling.