Augmented Reality

Augmented Reality + Machine Learning = 🥰🥰🥰

NatML has first-class integration with Unity ARFoundation for adding machine learning to augmented reality applications. We do so by providing ML feature implementations for working with AR camera and depth images:

Making Predictions in AR

Making predictions on the AR camera image closely mirrors the steps we've outlined above. The only difference comes in how we create an image feature for prediction. Instead of using the MLImageFeature class, we will instead use the MLXRCpuImageFeature class. First, we need to acquire the current XRCpuImage from ARFoundation:

// Get the ARCameraManager
ARCameraManager cameraManager = ...;
// Get the latest camera image
XRCpuImage image;
cameraManager.TryAcquireLatestCpuImage(out image);

Next, we create an MLXRCpuImageFeature from the XRCpuImage:

// Create an AR image feature
var imageFeature = new MLXRCpuImageFeature(image);

From here, the rest of the prediction workflow remains the same.

Working with AR Depth

• AR depth