MLPredictorExtensions

class NatSuite.ML.Extensions.MLPredictorExtensions

This class contains extension methods for working with predictors.

Async Predictions

/// <summary>
/// Create an async predictor from a predictor.
/// This typically results in significant performance improvements,
/// as predictions are run on a worker thread.
/// </summary>
/// <param name="predictor">Backing predictor to create an async predictor with.</param>
/// <returns>Async predictor which runs predictions on a worker thread.</returns>
static MLAsyncPredictor<TOutput> ToAsync<TOutput> (this IMLPredictor<TOutput> predictor);

This extension method converts any predictor to an MLAsyncPredictor.

Vision Extensions

NatML provides common extensions for working with vision models.

Non Maximum Suppression

/// <summary>
/// Perform non-max suppression on a set of candidate boxes.
/// </summary>
/// <param name="rects">Candidate boxes.</param>
/// <param name="scores">Candidate scores.</param>
/// <param name="maxIoU">Maximum IoU for preserving overlapping boxes.</param>
/// <returns>Indices of boxes to keep.</returns>
static int[] NonMaxSuppression (
IReadOnlyList<Rect> rects,
IReadOnlyList<float> scores,
in float maxIoU
);

INCOMPLETE.

Aspect Ratio Rectification

/// <summary>
/// Rectify a detection rectangle for a given aspect mode.
/// </summary>
/// <param name="rect">Input rectangle in normalized coordinates.</param>
/// <param name="featureType">Feature image type used for aspect scaling.</param>
/// <param name="imageWidth">Original image width before aspect scaling.</param>
/// <param name="imageHeight">Original image height before aspect scaling.</param>
/// <param name="aspectMode">Aspect mode used for scaling.</param>
/// <returns>Rectified detection rectangle in the original image space.</returns>
static Rect RectifyAspect (
this in Rect rect,
in MLImageType featureType,
in int imageWidth,
in int imageHeight,
in MLImageFeature.AspectMode aspectMode
);

INCOMPLETE.

Image Feature Inspection

/// <summary>
/// Get the image size for a feature.
/// </summary>
/// <param name="width">Output image width.</param>
/// <param name="height">Output image height.</param>
/// <return>Whether the feature is an image and has a valid image size.</returns>
static bool GetImageSize (
this MLFeature feature,
out int width,
out int height
);

INCOMPLETE.

Hub Predictions

NatML supports making server-side predictions for ML models that require heavier computation than is available on devices. These predictors are called Hub Predictors, and they work by serializing and transmitting feature data to the Hub cloud service. NatML provides common serialization functions for feature data:

Serializing Audio

/// <summary>
/// Serialize an audio sample buffer to a waveform.
/// This method is thread-safe and can be used from any thread.
/// </summary>
/// <param name="sampleBuffer">Linear PCM sample buffer interleaved by channel.</param>
/// <param name="sampleRate">Audio sample rate.</param>
/// <param name="channelCount">Audio channel count.</param>
/// <returns>Serialized waveform data.</returns>
static byte[] SerializeAudio (
in float[] sampleBuffer,
in int sampleRate,
in int channelCount
);

INCOMPLETE.

Serializing Images

/// <summary>
/// Serialize a pixel buffer to a PNG.
/// This method is thread-safe and can be used from any thread.
/// </summary>
/// <param name="pixelBuffer">Pixel buffer with RGBA8888 layout.</param>
/// <param name="width">Image width.</param>
/// <param name="height">Image height.</param>
/// <returns>Serialized PNG data.</returns>
static byte[] SerializeImage<T> (
in T[] pixelBuffer,
in int width,
in int height
) where T : unmanaged;

INCOMPLETE.