Unfortunately, the vMix Recorder does not support a suitable format for recording videos with an alpha channel.
This is the method I use. There may be other or better options, but I do not currently know a better way.

This guide explains how to record an NDI output from vMix using NDI Studio Monitor, resulting in a MOV video with an alpha channel. This MOV video can then be converted to a WEBM video with an alpha channel using FFmpeg.

The goal is to create a video with transparency, also known as an alpha channel.

The workflow is:

1. Create a transparent background in vMix. (Add Input, Color, Transparent)
2. Place the desired element over this transparent background using a keyer.
3. Output the vMix signal via NDI.
4. Record this output with NDI Studio Monitor, which is part of the NDI Tools.
5. The resulting .mov file contains the alpha channel.
6. This file can then be converted to .webm using FFmpeg.

The finished WEBM file can be used, for example, in browsers or imported back into vMix.

One advantage of WEBM is the significantly smaller file size. In many cases, the WEBM file is approximately 90% smaller than the original MOV file.

You need:

1. vMix
2. NDI Tools with NDI Studio Monitor
3. a vMix project containing the element that should be recorded with transparency
4. optionally FFmpeg for later conversion to WEBM

First, a transparent background is needed in vMix.

Create a new input:

1. Click Add Input in vMix.
2. Select a Color Input.
3. Choose Transparent as the color or background.
4. Add the transparent input.

This transparent input will be used as the base image for the output.

Now place the transparent input on the desired output.

To do this:

1. Select the transparent Color Input.
2. Place this input on the desired output.
3. At this point, the output contains only a transparent image.

Make sure that you do not accidentally use a black, white, or differently colored background. The background must really be transparent.

Now place the element that should be visible in the final video over the transparent background.

A keyer is used for this.

Depending on the vMix version, a different number of keyers may be available:

1. older versions: up to 4 keyers
2. newer versions: up to 8 keyers

Procedure:

1. Keep the transparent input on the output.
2. Place the desired element on this output using a free keyer.
3. Adjust the position, size, and appearance of the element as needed.

The element to be keyed is now placed over the transparent background.

The output therefore consists of:

1. visible element
2. transparent background
3. alpha channel for transparency

For NDI Studio Monitor to receive the signal, the desired vMix output must be available as an NDI source.

In vMix, the corresponding output must be made available through the NDI output settings.

Depending on the vMix version and configuration, this may be done through the Output or NDI settings.

The important point is that the output containing the transparent input and the keyed element is available as an NDI source.

Now start NDI Studio Monitor.

Select the vMix output there:

1. Open NDI Studio Monitor.
2. In the source selection menu, choose the desired vMix NDI source.
3. Select the output that contains the transparent image with the keyer.

The element should now be visible in Studio Monitor. Depending on the display, the background may appear black, transparent, or as a checkerboard pattern. What matters is that the recorded signal contains the alpha channel.

Start the recording in NDI Studio Monitor.

To do this:

1. Select the correct vMix NDI output.
2. Start the recording function in NDI Studio Monitor.
3. Play or output the desired sequence.
4. Stop the recording.

NDI Studio Monitor saves the recording as a video file.

The resulting file has the extension:

.mov

This MOV file contains the video image including the alpha channel.

After recording, you will have a MOV file.

This file contains:

1. the visible element
2. the transparent background
3. the alpha channel, also called the key

This makes the MOV video suitable as an intermediate format for further processing or conversion to a smaller format.

The recorded MOV file can then be converted to a WEBM video with an alpha channel using FFmpeg.
This is described in the next article.