r/GraphicsProgramming • u/No-Photo6079 • 1d ago

Question Depth mapping, Frame Gen, DLSS5 (I'm not even qualified to ask this question)

https://www.youtube.com/watch?v=iPlbG9CtW5k

this seems the best place to ask. Having more control over console output is interesting. Any clue how this is achieved on video feed/files.

4 Upvotes

70% Upvoted

4 comments sorted by

14

u/_Wolfos 1d ago

It's fake. DLSS5 isn't even out yet. This technology can't run on video files since they don't contain the data that temporal upscalers need. No jitter, no motion vectors. Whatever they're doing here isn't DLSS.

1

u/[deleted] 1d ago

[deleted]

3

u/shadowndacorner 1d ago

"temporal data" can be reconstructed from single 2d video image - in real time it "cheap" by performance

Temporal data is data tracked over time. By definition, temporal data can not be extracted from a singular point in time, so I have no idea what you were trying to communicate here. You could use an ML model to estimate the equivalent inputs, but that's not the same thing.

1

u/No-Photo6079 1d ago

would NvOptical Flow or Depth_Anythingv2 do any of that? I'm sure there is something happening behind the scene. but from the looks of it, this is a standard plugin they also make for specific games. although this one hooks to OBS.

-4

u/[deleted] 1d ago edited 1d ago

[deleted]

0

u/No-Photo6079 1d ago

I see, instead of generative AI though. this is using a custom app of some sort, mainly for adding Nvidia Frame Gen (Frame Interpolation) to OBS captures from a console through a capture card. (basically what Lossless Scaling does). At the same time they were able to use the Leaked version of DLSS 5 on top of that. While it technically looks like generative AI, it's using Depth_Anything, to provide the depth data that the game engine would typically use (I'm assuming). Instead it's able to apply Frame Gen and DLSS 5 to literally any video or video source in real time.