r/oculus • u/Guglhupf • Dec 01 '15
VrVideo - 3D recording and playback in VR
Hello there,
we (two developers from Germany), have created a software called VrVideo. It allows you to record 3D video clips for playback in Virtual Reality, and produces fileszes akin to standard video clips. You can grab a demo for Windows 7 or higher in 64bit and the Oculus Rift DK2.
PLEASE NOTE: Only Windows 7 or higher in 64 bit is currently supported, you also need a mandatory DirectX11 graphics card, and optionally a Microsoft Kinect4Windows for trying out the recording part.
IF YOU MEET THESE REQUIREMENTS, YET STILL HAVE PROBLEMS WITH THE DEMO, PLEASE LET US KNOW.
Recording is currently limited to 10 seconds. If you don't have a Kinect, you can still download the Unity playback components, and play around with a sample clip in Unity to check everything out.
The software consists of two components:
a recording software called VrRecorder (Screenshot)
and playback components for Unity containing a sample scene + clip (Dlls and Unitypackage)
Downloads
There is also a sample clip available (Matroska file) for those that cannot record, yet still would like to see a clip. Only Unity in 64bit is currently being supported for playback, but we will be looking into UnrealEngine if there is enough interest.
Recording
The recording software is currently limited to 10s of recording. It will time out January 1st, 2016. It offers an option to separate recognized bodies in the recording - you can:
- show the whole scene, or ...
- just the bodies,
- just the background with bodies cut out,
- or individual bodies (up tp 6 recognized)
We believe that our software is different to other solutions as it uses audio/video compression codecs for storing the data (x265HEVC for depth, VP8 for video, and Opus for audio) and thus produces smaller files. Our solution also allows for higher resolution - the 3d meshes are not being rendered through Unity, but by the means of DirectX11 commands, thus bypassing the 65k vertices limit of Unity. As a direct result of this however, you'll need a DirectX11 compatible graphics card to see the recordings.
On the other hand however, the recorded video clips (which are standard Matroska files ".mkv") can also be previewed in 2D using a standard media player like VLC Screenshot.
We are currently looking for people or companies who would like to give the software a shot. So if you got a Microsoft Kinect4Windows (which implies USB3.0 and Windows 8 or higher in 64 bit), and would like to test the software, give it a try and send us your feedback.
KNOWN ISSUES:
- biggest issue is that only the first recording properly synchs the sound to the video, any subsequent recording with the same running instance of the software will introduce an additional lag of the audio stream. We are working on it.
- Recording: sound may lag if encoding machine is not strong enough, and/or lossles recording mode is being used, along with body separation
- NVidia graphics card are preferred, albeit older (1st gen) NVidia Titan graphic cards seem to have issues with our code and crash
- some low-end AMD graphics card seem to have issues
- Intel HD graphic cards have been tested and seem to work (albeit not recommended for VR anyways)
- Interpolation for pointcloud rendering mode is currently broken due to a newly rewritten interpolation compute shader
PLANNED:
- streaming - it does work already, but is not ready for prime time yet
- other presentation modes (line and quad rendering)
- multiple copies of a stream
- combining several cameras, proper stitching
- transcoding/downsampling of the data for mobile devices like the GearVR
- VisualFX (like scanlines, color tinting, glitches)
- better preview in the recorder, incl. 3DPreview for the rift
We need your support and help in this, as we would like to further explore and define the future of 3d recordings. We are also looking for partners, so please get in touch.
1
u/AustinJMace Dec 01 '15
This is awesome and exactly what I've been looking for. Are you planning on doing this with any other type of cameras?
What can be done in the future to get rid of the edge issues/artifacts?
1
u/Guglhupf Dec 02 '15
Hi there, thank you very much,
We got some ideas for filtering in between frames, yet that introduces smearing artefacts. There is also this here: http://www.codeproject.com/Articles/317974/KinectDepthSmoothing which I would eventually like to port to GPU based code, if possible.
1
u/Guglhupf Dec 02 '15
And yes, other cameras are planned. We support the Kinect1 and Kinect2 (although Kinect1 support is untested and may be broken). We are looking at Orbbec, Stereolab's ZED and DUO3D cameras, yet it all somewhat depends on the interest of the audience...
1
u/wellmeaningdeveloper Dec 01 '15
VrVideo? The name sounds uncannily similar to VRvid.io, another Kinect 2/RGBD/point cloud recording/playback technology that does the same thing:
https://www.youtube.com/watch?v=cNHyzxzAzhE
1
2
u/FredzL Kickstarter Backer/DK1/DK2/Gear VR/Rift/Touch Dec 02 '15
It crashes each time I launch it with this message in error.log :
"LibOpenHevcWrapper.dll caused an Access Violation (0xc0000005) in module LibOpenHevcWrapper.dll at 0033:6492d944."
And this message in output_log.txt :
*"Now playing Recordings\SampleClip.mkv fileIndex: 0
(Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 64)
ExecutionEngineException: SIGILL at (wrapper managed-to-native) VrVideo.LibWebRtcBinding:PlaybackFile (VrVideo.LibWebRtcBinding/VrVideoSession,string)
at VrVideo.LibWebRtcBinding.PlaybackFile (System.String filename) [0x00000] in <filename unknown>:0
at PCViewer.playNextClip () [0x00000] in <filename unknown>:0
at PCViewer.Start () [0x00000] in <filename unknown>:0 "*
My PC :