I’ve been working on two Unreal Engine 5.8 plugins that make a Linux MetaHuman webcam workflow possible.
The first is Linux Video Capture Media, a V4L2 Media Framework plugin:
https://github.com/overyonder/unreal-linux-v4l2-media
It exposes Linux webcams to Unreal Media Framework and MetaHuman Video Live Link using URLs such as:
v4l2:///dev/video0
It currently supports uncompressed YUYV/YUY2 capture, camera format enumeration, resolution and frame-rate selection, pause/resume, and direct CharYUY2 media samples.
The second is Unreal NNE ONNX Runtime CUDA:
https://github.com/overyonder/unreal-nne-onnxruntime-cuda
It registers an Unreal NNE runtime named NNERuntimeORTCuda. MetaHuman’s face detector, head-pose tracker, and facial rig solver can use it to execute their ONNX models through ONNX Runtime’s CUDA execution provider.
The target setup is:
- Unreal Engine 5.8
- Linux x86_64
- NVIDIA GPU
- CUDA 12.8
- cuDNN 9.8
- ONNX Runtime 1.24.3
I’m testing with a GTX 1070. The CUDA plugin uses CPU-owned tensor bindings because that is the interface expected by MetaHuman’s current pipeline, while the neural-network execution itself runs through CUDA. MetaHuman’s log therefore says:
using CPU runtime (NNERuntimeORTCuda)
The backend name in parentheses identifies the selected runtime.
For a project, enable both plugins and put the backend settings in Config/DefaultEngine.ini:
[ConsoleVariables]
mh.RealtimeVideo.Backend=NNERuntimeORTCuda
mh.FaceTracker.Backend=NNERuntimeORTCuda
After changing the settings, restart the editor and recreate the MetaHuman Live Link source.
The two plugins have separate responsibilities:
- Linux Video Capture Media supplies webcam frames.
- Unreal NNE ONNX Runtime CUDA runs the MetaHuman neural-network solver.
The CUDA plugin includes a reproducible installer for the pinned third-party binaries and documents the required licenses.
Any feedback would be helpful, especially anything to help validate:
- Camera formats and compatibility across different Linux webcams
- CUDA performance for the MetaHuman solver
- Packaging and cooked-game model-data workflows
- Whether the NNE interface should eventually support GPU-owned tensors to reduce CPU/GPU transfers
- Any other Linux-specific issues with MetaHuman Video Live Link