Recording Textures
Just One More Step
For some more advanced use cases, you might want to record video frames from a Texture
in Unity. Typically, recording from a Texture
requires you to first read the pixel data from the GPU before committing the data to the recorder.
Unity provides two ways to readback pixel data from a Texture
into system memory. They differ based on whether the client is willing to accept a deferred completion of the readback.
Synchronous Readbacks
A synchronous readback will request pixel data from the GPU and block until the request has been completed.
Once the above code executes, the pixel data will be accessible through the Texture2D
.
Due to the concurrent nature of GPU's, such a readback will typically result in a pipeline stall. This means that both the CPU and GPU cannot do any other work until the request is complete. Pipeline stalls are typically very expensive, and will cause a very noticeable frame rate hit in most cases.
Though expensive, synchronous readbacks exhibit very predictable memory consumption patterns.
Asynchronous Readbacks
Unity provides the AsyncGPUReadback
API for performing an asynchronous readback.
This readback will request the data from the GPU but will not wait for the GPU to complete the request. When the transfer is complete, Unity will invoke the provided callback. We can then commit the pixel buffer to the recorder within this callback.
The advantage with this approach is that it provides much better performance over synchronous readbacks. There are a few disadvantages with this approach:
It adds latency on the order of a few frames
It has less predictable memory consumption patterns than synchronous readbacks.
It is not supported on all platforms and devices, so you must check at runtime.
CameraInput
and ScreenInput
will default to asynchronous readbacks on devices that support it.
Texture Inputs
NatCorder provides primitives that implement both synchronous and asynchronous readbacks, so that you don't have to. The TextureInput
class handles synchronous readbacks:
Similarly for asynchronous readbacks, NatCorder provides AsyncTextureInput
:
Last updated