Core Concepts

All You Need to Know 🤓
You do not need to be an expert in AI research to deploy ML models. NatML focuses on making ML deployment as painless as possible for developers. Before diving in, it is crucial to understand a few core concepts, and how they interact with one another. These concepts are:
  • Models
  • Features
  • Predictors
  • Endpoints


An ML model is a 'black box' which receives one or more features and outputs one or more features. For example, a vision classifier receives an image feature and produces a probability distribution feature, telling you how likely the image is one label or another.
Under the hood, an ML model is a computation graph. This graph is what empowers ML models with the ability to learn and make predictions on new data.
At NatML, we prefer to refer to a raw ML models as a "Graph" for clarity.


A feature is any data that can be consumed or produced by a model. For example, you will use a lot of image features when working with vision models.


On NatML, a predictor fully describes an ML model, along with any data and code required to use the aforementioned model. As a result, predictors form the foundation of NatML, serving as the most fundamental unit.
We distinguish predictors based on where their predictions are actually run:

Edge Predictors

Edge predictors are predictors that execute the ML graph on the local device. These predictors are commonly used in interactive media applications, like in augmented reality apps built with Unity Engine.
To use an edge predictor, you will need a Machine Learning Runtime (MLRT) which knows how to run the computations defined within the ML graph. NatML is strongly opinionated about which MLRT to use on which platform:
NatML Graph Format
​TensorFlow Lite​
NatML provides infrastructure for converting between ONNX, TFLITE, and COREML graph formats while performing optimizations for runtime performance.
NatML provides a cross-platform machine learning runtime for Unity Engine. See the docs.

Cloud Predictors

Cloud predictors are predictors that execute the ML graph server-side, on NatML's servers. These predictors are suited for using models that might be too large for use on local devices (e.g. LLMs); or for applications that require higher accuracy predictions. Cloud predictors enable predictions through their Endpoints.


On NatML, a prediction endpoint defines the server, code, GPUs, networking infrastructure, and all other resources that are used to host a cloud predictor at a given URL. NatML is designed to drastically simplify the process of creating and managing these cloud resources.
Endpoints are created from Jupyter notebooks that define a predict function, like so:
def predict (input_a: float, input_b: bool=True) -> str:
A predictor that returns a friendly message to the user.
input_a (float): An unused float parameter.
input_b (bool): Another unused parameter, this time a boolean.
str: A friendly message :)
return "it works"
NatML handles provisioning servers, GPUs, feature serialization and deserialization, and much more. All you have to do is bring your notebook!