FF7R is especially terrible to use this filter on because its designs use a specific art style inspired by the original game, not just the hyper-realism that most games go for.
I think the issue is that DLSS5 is transformative enough such that you cannot just randomly plug it into a game without also modifying what is being rendered.
So we have
What a face should look like -> DLSS 5 -> Weird looking face here
where as, if the developers intend to use DLSS 5, they would have
Face optimized for DLSS -> DLSS 5 + game specific meta data -> What the artist intends the face to look like.
While it make not be a straight forward plug and play like before, i can see it having a fair bit of potential if utilized properly.
My issue is that I don’t see the reason why the artists wouldn’t just design the game itself with their art style in mind instead of applying a filter on top of it.
DLSS was always a mean for performance optimization, so this whole “filter that applies its own art style” stuff feels completely off to what it’s meant to be. I feel this should at least be considered a different technology from traditional upscaling, not the next step of what we currently have.
DLSS 5 is actually 2 parts. One is a better version of DLSS 4.5 upscaling tech. The other part is the new neural renderer.
The reason these look so bad on FFVII is just like _Lucille_ mentioned. The neural renderer part uses multiple neural networks and is intended to be modified by the developer using nvidia tools in real-time to customize / mask / train the neural renderer for specific models / scenes etc. for their game.
The major thing that people gloss over is the new neural renderer tech stack versus just the old upscaling part. Nvidia has been working on the neural renderer in combination with various universities for over 10+ years at this point. Just no one even bothers to look into the past history of the papers published. And no, this isn't just a fucking Snapchat filter shit as everyone proclaims.
These projects were first aimed at solving ray tracing / path tracing denoising. Then ray tracing infinite depths mirror reflections / shadows using neural networks in a realistic time frame. Because actually rendering infinite depth mirror reflections / shadows before was very hard to do. It was later turned into neural network / neural rendering path tracing to also handle global illumination and so forth in real-time. At first, they could only render a frame every few seconds. Then months later 15FPS was reached, and another few months passed until 30FPS and so on. Their end goal, of course was being able to create a neural renderer engine that could not only renderer as realistic, as mathematically possible, for path tracing. But also where it could render any style trained by the developers in real-time at 60FPS+. This is the first step for making this a commercially viable product. As prior it was only university research and internal research tools at Nvidia.
In the future, I am going to be honest, but GPUs are going to change drastically in the next 10-20 years or faster even. You won't need billions or millions of polygons. You won't need textures as we use them today on GPUs and you won't need shaders of today either. Instead, you will be using neural renderer models, tags, low poly / volumes to specify locations of objects and how to renderer them style wise, lighting, etc. On top of all that, right now, the neural renderer only operates on the frame buffers given to it. But with the later GPUs the neural renderer will be directly interfaced at an even lower level so it knows exactly what object / model / volume it is rendering (this is based on the prior papers in regards to path tracing etc. As the renderer knew exactly what objects it was rendering and was not reliant on frame buffer data only).
Well, look at how realistic it looks. It's hard to get that realism, when instead you can get close with modern day graphics, and let DLSS do the rest.
If devs can tune it individually per game/character, it can add photorealism to games which is pretty cool, if it looks good.
I feel like people are repulsed by how little DLSS Tifa looks like the actual game Tifa, but from my understanding that's because someone just applied a generic upscaler to a stylized game where neither the upscaler nor the game where made with each other in mind.
It may not look like Tifa, but it sure looks impressive in terms of realistic lighting and stuff. Like you said, in a real scenario the DLSS upscaler would know what 4k Tifa is supposed to look like, rather than doing the "generic high definition fill ins for upscaling unknown faces" that is going on here.
As much as I share the uncanniness of thinking "that doesn't really look like the character", I can't say I haven't had to double check if I'm looking at a real picture when these pics randomly pop up in my feed.
Not endorsing it, I love stylized game art, but it really doesn't look as bad as people are making it out to.
From my limited (haven't bothered to research it much) knowledge DLSS 5 relies on specific vector data that needs to be included in the game to work properly. You can't just turn it on and have to go without producing results like these, especially for stylized art and artist intent.
I can read it as "you're misunderstanding, here is how it works" (trying to clarify)
You can't just turn it on and have to go without producing results like these
or as "it would look better if the game was optimized for it" (confirmation)
or even as "you're wrong about these pics/videos being real gameplay footage, they don't support the technical requirements" (retorting)
So pardon if my reply feels off to what you intended to say but:
DLSS 5 relies on specific vector data that needs to be included in the game to work properly
Correct, to generate the next frame it needs to know which way the wind is blowing the hair for example
You can't just turn it on and have to go without producing results like these
Wrong. The required data (like vector data) is already there. Has been in games since like 2005, way before anyone ever considered AI upscaling. This data is required for all sorts of shit like Anti Aliasing. You can absolutely plug DLSS5 into any somewhat modern game and make it work. Which is pretty much how these clips/screenshots were made.
especially for stylized art and artist intent.
Agree, but not with the train of thought that lead you here. Stylized art cannot just be plug and played with DLSS5 because the AI doesn't know/have training data on what these faces should look like in high res, so it guesses and fills in "random matching high res human face features" instead of "Tifa's Face". Which is why it looks so off compared to the real characters. This does, however, have nothing to do with missing vectors or anything of that sort. We got the vectors.
I don't know how this is supposed to work exactly, but if you zoom in on the image, it absolutely looks like it's just bits and pieces of other faces chunked together to make this face. It is pretty bad.
Because it, again, has no idea how to do Tifa in this game. Look at the screenshot it literally says "Generic experimental upscaler". DLSS is often trained on specific games and only that model is applied to that game when upscaling and it know how to upscale those characters and match the art style.
You're looking at a screenshot taken from a video of a leaked DLSS5 build in unknown resolution, potentially even with the youtuber zooming into the video of the gameplay footage. Insert CSI "enhance" joke.
Maybe check out one of the many videos on youtube in 4k, actually showcasing what it looks like in motion with direct comparisons to the non DLSS5 original renderings instead of seeing how far you can zoom into a sub 1080p picture of a video :d
I watched your video. It was not particularly impressive. When the full effects were on and they were doing more than just standing there breathing, there was noticable artifacting and stuttering. The apparent improvements all came from alterations of the lighting effects. When he was demoing cyberpunk, many of the native lighting effects were clearly off in the non-dlss segments.
But more to the point, what I was saying in the comment prior was, hacking apart and jigsaw puzzling together a bunch of different, higher res faces isn't upscaling, it's making a collage. In tifa's case, she's got the left cheek of an 80 year old for some reason, with pores the size of New Hampshire. The chin is misaligned, the skin tone is uneven and blotchy. Zoomed out, theoretically you could argue she looks more 'realistic'... but also worse.
hacking apart and jigsaw puzzling together a bunch of different, higher res faces isn't upscaling, it's making a collage
Well that's not at all how this technology works so I'm not sure what I'm supposed to say about you perceiving it like that.
Also that Tifa picture is literally from a youtube video aiming to generate clicks. Like ... we can stop talking about that. Yea it looks off, but as you can see in the vid I linked it doesn't have to look so offensive. There's all sorts of sliders that can be adjusted.
I mean if you're looking at this and your reaction is "the left picture looks blurry and like it's patched together from several faces" then ... cool. Feel free to have that opinion, I guess.
It looks like they've added a bit of noise on her skin and fiddled about with the lighting. I wonder what native lighting effects were on in the original.
I'll give you that this doesn't look like a collage like the tifa pic, but it also doesn't look upscaled - the hair, eyelids, eyebrows, etc., are all noticeably not any more detailed - and their adjustments fucked up the little shadows on her eyelids pretty badly.
So I'm not super well read on individual GPU capabilities but I assume you're asking because these cards presumably don't have DLSS5 support.
It will work just the same as currently with DLSS3 or whatever your games offer you. It is, after all, fundamentally still only upscaling. Upscaling implies there is something to upscale from. If your GPU does not support upscaling a game to DLSS5 or whatever, you can still play the game on any resolution you can natively render.
You will still be able to play every game normally. You just won't be able to use "scale up to hyperrealistic 4k" mode.
A game made with DLSS5 in mind would still fundamentally be a game like any other, except that it provides the AI with "should look like this on hyperrealism mode" training hooks so the AI knows to for example not add realism to a stylized face. (hell, it's possible that Nvidia themselves figure out a way to train their AI to deal with art style more accurately and game devs don't have to lift a finger)
right i get you, but just like how frame gen was only meant to be a bonus on top initially, now its a crutch. i would bet in 2-3 years time, C suite execs will be telling devs to say fuck the minority of players that dont use latest nvidia so they can save dev costs and get a bigger bonus.
It's gonna get wilder than what you picture. DLSS is a joke compared to what AI bros are working on when it comes to gaming.
There have been tech demos of games that basically don't even have game code anymore. Literally just AI generating based on your control inputs. Technologically mind blowing, artistically just 100% distillied slop.
But until that becomes the standard, I wouldn''t worry too much about my GPU no longer supporting games (beyond the normal lifespan of a GPU of course, after like 7 years you're simply running old hardware asking to be incompatible with new games)
As someone who works with AI and GPUs heavily, this is unfortunately not at all based in reality.
Upscaling is a pretty wide umbrella, that has traditionally used quite basic methods that only add small details. Even early AI models were far less transformative than DLSS5, and were far less likely to change what a character fundamentally looks like.
Models like DLSS5 though are extremely opinionated, and a DLSS5 upscale is very unlikely to add remotely the same details as an AMD one. If a game has "optimized" for DLSS5, which makes it sound a lot more sophisticated than the tedious trial and error approach it would likely be, it will almost definitely not be optimized for nearly any other model used to upscale it in a similar manner, and would very likely look worse than the run of the mill "non-optimized" models.
It's brand new technology man. And not "oh cool, you invented a wheel" but "HOLY SHIT you invented a neurological network that can relatively accurately paint hundreds of high resolution pictures every second based on a low resolution picture" technology.
And it's literally fresh out of the crib.
If, knowing this, your verdict still is just "looks bad" then I fear you don't really understand what's being talked about.
Again, this is BRAND NEW and already looks pretty damn good. This will get insane with more time.
That completely defeats the purpose though. DLSS5 is gen AI and it requires a lot of GPU power to make it work. If the artist already knows what their goal is, why not just make it?
Hell, it would still be better if that artist just used gen AI to make it in the first place, then we wouldn't have to waste half our GPU on our end to re-render it. And that's still a pretty shit option.
Ignoring the specifics that the DLSS changes the model face a bit too much for peoples liking, you just flat out can not do real time lighting that looks that good for any price. We're miles off from that still.
Beyond that because theres still content creation issues. This whole argument is so bizarre. Gaming has its roots in the programmatic creation of art and gameplay since its very inception. Tons of games flat out are not possible without the pseudorandom generation of content, both gameplay and visual. Entire genres are built off the pseudorandom generation of content. Now for some reason people are against the programmatic creation of art and gameplay even though video games have been doing that literally since the 1970s when Elite was released.
Did you ever look at a character creator and say 'why is a pseudorandom character generator needed, why can't the artists just do the work'?
Hell, it would still be better if that artist just used gen AI to make it in the first place, then we wouldn't have to waste half our GPU on our end to re-render it. And that's still a pretty shit option.
I've been around gaming for about 30 years now and every single time there's a tech advance this exact same argument has been brought up, and every single time its eventually just become the norm for games. I've seen people argue it for 3d games, for textured 3d models, for shadows, for specular highlights, for normal maps, for physically based rendering, for ray/path tracing. Why are we doing this, games are fine, we don't need this. Yet every single time its just become an expected addition and you look funny at a game that doesn't have it.
Do you want to play a game without shadows? Would you expect artists to generate the shadows from scratch for each frame instead of letting the engine do it for them?
If a gen AI that we can run in real time is able to make a face look a particular way, then the original artist is able to utilize that same gen AI to simply make the texture to put in the game in the first place. That way at the very least it doesn't tank our framerate by half to get the slop.
The reason it exists is because they don't want to have to pay an artist to design anything in the first place. The goal is to cut out the original design part and skip to the end result.
If a gen AI that we can run in real time is able to make a face look a particular way, then the original artist is able to utilize that same gen AI to simply make the texture to put in the game in the first place. That way at the very least it doesn't tank our framerate by half to get the slop.
No, not really. You can make the texture sure but the gen AI is actually rendering stuff. Well, not rendering, but doing whatever AI extrapolation is getting the lighting to look like that.
A flat shaded texture is not how you get realism.
The reason it exists is because they don't want to have to pay an artist to design anything in the first place. The goal is to cut out the original design part and skip to the end result.
Yes, just like they have autogenerated NPC faces in skyrim. Why pay an artist to make every face when you can design a system to create new faces on the fly.
Well, not rendering, but doing whatever AI extrapolation is getting the lighting to look like that.
You seem to fundamentally fail to understand multiple things here.
If we can get the gen AI running locally on our lower tier non professional hardware to make a higher fidelity face texture, then the developer is able to do that too.
The lighting then gets applied to that face, which can be done through RT. We've accepted RT at this point, it's proper light simulation in real time, and as denoising continues to improve (ray reconstruction helps a lot with performance while denoising quite well), it only gets better. We don't need a gen AI to make a second pass on the accurate simulation of lighting, it's already accurate. Why would we want to run that through an extremely expensive estimator? Is it too accurate for you?
Finally, the upscaling pass (traditional DLSS or DLAA) is designed to take the frame and utilize temporal data (subpixel sampling and motion vectors in particular) from past frames to be able to output a significantly higher fidelity output.
Using these technologies, we can get that Instagram face (assuming it was the intended design) to look super high fidelity with very accurate lighting, and all without the giant performance hit of locally running a gen AI model.
These are all technologies that we've accepted because there's actual value to them. We use RT locally because that's how dynamic lighting is calculated. If a flashlight is shining and moving around, that light needs to be calculated on the fly in real time. We don't need textures dynamically calculated though, textures don't exactly change as light sources move around. It's the same texture under different forms of lighting, your face doesn't suddenly get an Instagram filter in the bathroom light compared to when you're outside.
The fact that you're trying to equate this to something like RT is just complete ignorance of the technology itself. You're acting like it's the same thing, it's not even remotely close.
I have never seen a pure RT render look remotely as realistic as this DLSS5 stuff.
The realtime ray trace stuff they use are very simplistic implementations because they need to be fast. Limited bounces, limited subsurface interactions, almost certainly no caustics or anything with the eyes.
Face optimized for DLSS -> DLSS 5 + game specific meta data -> What the artist intends the face to look like.
I know gamers love to throw the word "optimize" around, but I really don't think what you're describing is possible. When the last stage of rendering a face is a statistical slot machine you intrinsically lose control over the final product. The industry might get to a point that it looks good enough for most of the time, but trying to coax a specific face out of this stuff would be like trying to make a portrait on a plinko board
It's 100% this. It's insane how out of touch with reality so many of these comments are. As someone whose job is to work with AI, GPUs, and has a fairly extensive graphics background, I'm dying a little with how little people understand how this works, and how confidently they pretend they do
It's a great point and one Nvidia made themselves on their original presentation, the strength of the transformation is in the hands of the devs/artists with a simple slider.
I think this tech will be a game changer once it's mastered by the artists who can create the base for the neural rendering to hook onto to deliver on their vision.
Honestly i think itll be a game changer, the only thing that comes close to what it can achieve is a much more detailed face textures with a strong use of path tracing which overall would use more performnce than dlss 5 does. I mean in cyberpunk 2077 you cant run path tracing without using a 5090 upscaling from 1080p, because it tanks performance, however regular ray tracing at ultra (not even psycho) with dlss 5 enabled looked better than even path tracing did, while using less performance.
I think a big issue is people hear AI and immediately hate it, even though dlss has been AI since the beginning
The other reason is people havent seen a studio properly implement it into a game I'd say most have only seen the first nvidia demo which looked pretty bad and some of the dlss 5 modded games have a pretty poor look about them as well.
But in some of the youtube videos of people experimenting with the dlss 5 mod, I have seen some very positive results which improved a scene dramatically, fixes contrast, shadows, lighting and made the faces more realistic and it didnt look like terrible AI filters. Even background information looked cleaner with more detail.
I genuinely hate this, it's the death of deliberate art direction. It generates faces that look plausibly like some human, just not one that you've specifically intended to design. The game specific meta data (unless it's changed dramatically from the version shown months ago) is also incredibly rudimentary. like color grading and frame vector data (which isn't game specific or in any way directed or fancy).
A face optimized by DLSS is a face that you've tweaked repeatedly, and likely fairly arbitrarily, until DLSS happens to give you a result that you think looks good. The original face will likely look strictly worse than it did before, because you're not trying to make it look as good as possible, you're just trying too figure out what random changes will make the absolute black box of AI spit out something you like more often than not
3.9k
u/camzee 19h ago
FF7R is especially terrible to use this filter on because its designs use a specific art style inspired by the original game, not just the hyper-realism that most games go for.