Media Devices

NatCorder seamlessly integrates with our cross-platform media device API for Unity Engine, NatDevice. NatDevice provides a lightweight API for accessing and using hardware cameras and microphones. It has the same cross-platform support as NatCorder, and has the same minimum requirements.
GitHub - natmlx/NatDevice: High performance, cross-platform camera and microphone streaming for Unity Engine. Register at https://hub.natml.ai
GitHub
NatDevice on GitHub.
Recording the Camera
Unity's API for using the hardware camera, WebCamTexture, has several limitations that prevent it from being used in a production environment.
First and foremost, the camera preview is not corrected for orientation on mobile platforms. This means that when the camera is started and displayed, it will be rotated wrongly. Correcting this orientation for display is a non-trivial process. But correcting the orientation for recording is orders of magnitude more difficult, as you will have to manually rotate each pixel buffer in memory. NatDevice handles all these gory details automatically.
Beyond the above point, NatDevice provides an expansive API for camera control. NatDevice allows for controlling the camera exposure, focus, flash, zoom, torch, white balance mode, and so on.
The typical NatDevice camera workflow is to first find a camera device to stream the camera preview from:
1
// Get a camera device
2
var query = new MediaDeviceQuery(MediaDeviceCriteria.CameraDevice);
3
var device = query.current as CameraDevice;
Copied!
Then we can start streaming the camera preview:
1
// Set the preview resolution to full HD
2
device.previewResolution = (1920, 1080);
3
// Then start streaming the preview
4
var textureOutput = new TextureOutput();
5
device.StartRunning(textureOutput);
6
var previewTexture = await textureOutput;
Copied!
Back on the NatCorder side, we will first create a recorder. Since we are recording the camera preview, we need to make sure that the frame size of the recorder is set to that of the preview texture.
1
// Create a recorder
2
var recorder = new MP4Recorder(previewTexture.width, previewTexture.height, 30);
Copied!
We can now commit a bunch of frames from the camera preview into our recording.
1
// Record a bunch of frames
2
var clock = new RealtimeClock();
3
for (var i = 0; i < 300; ++i) {
4
    recorder.CommitFrame(previewTexture.GetPixels32(), clock.timestamp);
5
    await Task.Yield();
6
}
Copied!
And once we are done recording camera preview frames, we can stop recording.
1
// Finish recording
2
var path = await recorder.FinishWriting();
Copied!
Recording the Microphone
If you plan to record user audio through the microphone, you will want to use NatDevice. Unity's Microphone API is known to often produce glitchy audio.
We created our NatMic API because we couldn't get around Unity's glitchy microphone audio. NatMic was later merged with NatCam to create NatDevice.
NatDevice on the other hand will stream audio sample buffers directly from the microphone to your code in .NET. To start, get a microphone:
1
// Get a microphone
2
var query = new MediaDeviceQuery(MediaDeviceCriteria.AudioDevice);
3
var device = query.current as AudioDevice;
Copied!
Then when you start recording, stream audio samples from the NatDevice AudioDevice to the recorder:
1
// Create a recorder
2
var recorder = new WAVRecorder(device.sampleRate, device.channelCount);
3
// Start the microphone
4
device.StartRunning(audioBuffer => {
5
    // And pass sample buffers directly to the recorder
6
    recorder.CommitSamples(audioBuffer.sampleBuffer.ToArray());
7
});
Copied!
When finishing recording, stop the audio device before stopping the recording:
1
// Stop streaming audio samples to the recorder
2
device.StopRunning();
3
// Then stop recording
4
var path = await recorder.FinishWriting();
Copied!
And that's it!

AudioInput

Next - Integrations

Augmented Reality

Last modified 14d ago

Copy link

Contents

Recording the Camera

Recording the Microphone