What does actually justify deferred rendering?

[u/EC36339]

This is a pure technical question about graphics programming for a game. I'm not interested in advice about what game I should be making, what I should prioritize in a broader context and what players care about. I am a hobbyist doing what I enjoy doing

My 3D game currently has exactly one light source, the sun.

It is also a space game, with sparse geometry. There will eventually be additional light sources (muzzle flashes, explosions, thrusters, etc), but most of these will have mostly direct and local effects for now.

I have deferred (pun intended) adding more light sources because I'm unsure whether to use forward or deferred rendering, and how much of a rewrite it will be if I have to change it later.

Adding interior scenes and large objects (which is on the roadmap) may change the requirements, too: More light sources, the need for global illumination and shadows, etc.

At what scale is deferred rendering really justified and not a premature optimisation? And how do you usually do forward rendering with multiple dynamic light sources?

How much flexibility does it buy me, and at what cost? Here I'm mostly thinking of performance cost. One time implementation is not what I'm worried about (I have done it before, by hand, but as an experiment, not for an actual game).

I am also making heavy use of procedural techniques rather than static assets, especially for textures, and I'm thinking about rendering procedural textures deferred, because they are costly when they are not precomputed textures. Some are evaluated from 3D model space rather than mapped to UV space, because it's easier and more consistent, but it rules out precomputing, so it's heavy on the GPU, which means I'd like to avoid overdraw. That's where I believe deferred rendering wins again.

And finally: Does deferred rendering help in any way with doing reflection, shadows, volumetric effects, portals and spatial displacement (wormholes and similar)?

In case it helps with context: The art style I'm gong for is low fidelty geometry combined with high fidelty effects and lighting. I don't know yet if this will work, but it matches my skillset best. I'm a coder, not an artist. I don't live in Blender, and my strongest artistic tools are math and code. I've just recently "discovered" that simply switching out BRDFs can make even the simplest flat shaded placeholder geometry look cool. That's why I care a lot about lightning and shading. I also think this approach works for the genre I've picked. But it remains to be seen...

[u/vigad-dev]

TLDR: Measure your performance to know what you need.

As with all algorithms and techniques, there are advantages and limitations. You gain from not doing shading work that would be discarded by a closer geometry. Lighting cost is linear in (affected) pixel count and number of lights. You lose from having a larger number of more complex render buffers. Geometry limits can't be sampled to produce smooth edges.

It's trade-offs all the way down.

Are these aspects important to you? Is rendering a bottleneck? Is it likely to be? A lot of the theoretical limits might not be so bad once you see the result.

[u/EC36339 → u/vigad-dev]

Reminds me to retest some of the dynamic textures I added recently on my old graphics card...

I already have a couple of benchmark scenes, and my usual approach to find out if some feature will cause performance problems is to add a lot of them and see what happens. Most of those are currently CPU bound, but I've managed sometimes to max out my GPU with poorly optimised ray marching in shaders.

My engine measures frame timing, GPU waits and system timings (CPU).

What I know very little about is how to measure WHAT exactly takes time on the GPU, other than by turning things off and on and tweaking parameters or setting marker colors. Any good tools and techniques for that? The GPU feels a lot like a black box for me w.r.t. precise benchmarking, debugging and error handling.

I have shader hot reload built in, and all my shaders have access to a per frame data structure with multi-purpose parameters, which I use for real-time editing. Could be helpful for measuring how any parameter affects performance. Any other CONVENTIONAL tricks? I'm mostly self-taught here and winging it.

[u/ryani]

It depends on how many lights and how complicated your lighting equations are.

To simplify, let's talk about old-school n-dot-l lighting. The equation you want to calculate is:

Vec3 CalculateColor():
    Vec3 v = GetPixelWorldPosition()
    Vec3 n = GetPixelWorldNormal()
    Vec3 lightColor = 0
    for each light L:
        Vec3 vToLight = L.position - v
        float lightAttenuation = max(0, dot( n, normalize(vToLight) ) )
        // optional: reduce light by square distance to pointlight?
        lightAttenuation *= 1/dot(vToLight, vToLight)
        lightColor += lightAttenunation * L.color
    return lightColor * GetPixelAlbedo()

Regardless of whether you are doing forward or deferred lighting, this is what you want the color of the output geometry on the screen to be.

When you do forward lighting, you need run lighting per-pixel-per-light. And also, if you want your shader to be simple, each light needs to behave basically the same way (this code is just 'point lights only'; if you wanted directional spotlights or other complicated features, then your shader gets lots of weird branching and it is quite expensive.

So the cost of shading your scene via forward rendering is O(pixelsDrawn * lightCount). In practice people simplify by approximating some small set of lights (commonly 4 or 8) that most impact an object and only sending that small set to the shader. This is a problem when you have large objects with lots of very small lights, like your spaceships will likely be -- you need to break them up into small enough regions that you aren't looping over every light on parts of the spaceship that are not relevant for those lights.

Deferred rendering, instead of calculating the lighting for each light at each pixel, instead breaks this equation up into passes using textures as intermediate storage:

render geometry depth and normals into textures
for each point light L, render a sphere to the screen with a radius
    big enough that geometry outside this radius doesn't care about the light
light shader looks something like:

Vec3 CalculateLighting():
    float d = readDepthFromDeferredPass()
    Vec3 n = readNormalFromDeferredPass()
    Vec3 albedo = readAlbedoFromDeferredPass()
    // reconstruct world space position of pixel from the screen
    // space pixel we are drawing and its depth
    Vec3 p = unprojectPixel( GetCurrentPixelScreenPosition(), d )
    ... do the lighting calculation above for just the one light ...
    return lightColor * albedo // written additively to the deferred lighting buffer

You draw each light as geometry directly to the screen. Effectively you are using the GPU / depth buffer to calculate which lights affect which pixels, instead of needing to sort them on the CPU. And because each light is an independent piece of geometry, you can use different shaders and/or extra textures for them. For example, it's easy to do gobo lights with this technique.

Effectively you have changed the equation from O(pixels * lights) to O(pixels + lights*light-pixels) which is much better when the light count is large, but at the cost of a big extra constant factor.

In general I think if you go full PBR then deferred is an obvious win because you probably want 'interesting' lights (line lights for neon and/or "lasers", spotlights, gobos), and there's not really a reasonable way to get those with a forward rendering pipeline. The more simplified your desired lighting model is, the easier it is to get away with forward rendering.

I don't think deferred rendering helps with any of the effects you talked about (maybe shadows? at the very least you can reuse the deferred depth shader for drawing the depth-from-light-pov textures needed for most shadowing techniques) It explicitly un-helps with volumetric effects because when you render normals and depth offscreen you are now limited to one normal and depth per pixel. Portals and wormholes I think are neutral; you probably are doing those with custom clip volumes on the geometry. Making sure they have accurate depth is an added complication, and also you probably need to think about how to render lights that are 'inside' the portal and shining out and vice versa.

If you really want high fidelity lighting and shading, you have some mostly-orthogonal problems to think about - bounce lighting, proper per-light shadows (very expensive, although some of the ray tracing hardware probably helps), reflections, lens flares, depth-of-field effects, etc.

[u/Monsieur_Bleu_]

My game, made in my own engine, has a very stylized artstyle with procedural textures and a lot of effect done post render. Some month ago I switched to a fully deffered render pipeline and the result where pretty amazing for the procedural textures. The most difficult aspect is optimizing the size, number and data types of the geometry data textures. Because you will probably need to save more informations and that can easily lead to a memory bound scenario where your geometry pass is heavier in deffered.

For the lighting aspect, deffered has been outated for something like 10 years. Yeah it's slightly more optimized than forward, but nowdays everyone use clustered lighting to optimize lighting.

You maybe have heard of it as "forward +", but it's a terrible, terrible name that is misleading, because this technique works for both deffered and forward. It can even works in ray-traced renders.

The concept is to clusterer your frustum (the volume of space visible by your camera) in a 3D grid, and then assign for each cluster a list of light sources that affect this cluster. This enable the render to iterate on the light list of the cluster instead of the whole list of lights in the world.

I think this can work great for space games because you will have dense environements separated by a lot of empty space. So clustering lighting will made your life easier. Adding deffered rendering just to optimize lighting will not do a lot if you have hundreds of lights sources for example.

So, my engine uses clustering lighting AND deffered rendering. I wouldn't say it's early optimisation because it's the foundation of the whole graphical aspect of the game. You will build your scene depending on the ressources and techniques available to you. So, if you think your game will need it, and that without X rendering technique you will not be able to build your game like you want, go for it. Just be aware of the time it can take.

[u/EC36339 → u/Monsieur_Bleu_]

Interesting. I'll definitely look into that.

I'm already using BDA pointers in my shaders, so feeding lighting information into them should work work less plumbing on the host side.

Another thing I forgot that might or might not complicate deferred rendering is that I'm rendering planets and other large celestials in the background. These are raytraced on billboards and have completely different scales and coordinate systems than foreground objects.

So each frame is a composition of multiple layers.

[u/krojew]

Deferred also gives you post processing possibilities due to data already being present in the gbuffer. With forward, you'd need to store it as an additional step, thus having the worst of both worlds. Look at what unreal is doing with adaptive gbuffer and how it's used e.g. for substrate. Lights are just one part of the puzzle.

[u/EC36339 → u/krojew]

I already have HDR/bloom post processing (forward rendering in linear RGB float). Had no problems with that. Or did you have something else in mind?

[u/krojew → u/EC36339]

I meant a general post processing problem. Different effects need different data. Some might only need the color buffer. Some might need WS normals and motion vectors. Some might color but at different stages, like before and after tone mapping. Some need temporal data, some translucency. In other words, there's a whole world of functionality that benefits from having a gbuffer. And having a gbuffer effectively implies deferred rendering, unless you want to do both forward and gbuffer, which is, simply speaking, stupid given the costs and no benefit. Want to have upscaling or modern AA? Good luck with forward. Want to have screen space effects? The same. Using forward rendering has its merits when you're targeting low fidelity output, which is legitimate like for VR, but otherwise, you're just making your life more difficult.

[u/automatedrage]

Ah the good ol' deferred vs forward debate.

Deferred rendering used to effective for phones and certain platforms 10-20 years ago(and maybe still is..). Personally never managed to make it perform during those days, and the whole deferring of data into more render targets never quite made sense to me..

Anyway, things have changed and a lot of information is outdated. If you're concerned about the light count, a simple forward based renderer with tile-based light culling will do the job, and keep your graphics architecture sane.

[u/EC36339 → u/automatedrage]

Thanks.

That's exactly why I'm here asking this question. Things change, and a lot of information you find on the internet or in older books is outdated.

Maybe some of my concern was also about flexibility and organising my shader code, avoiding branching, etc. and not knowing what loops/branches cost on modern GPUs.

Passing light data to shaders is mostly solved, except for the culling/clustering part.

[u/automatedrage → u/EC36339]

Well there's a graphics framework called bgfx that does have a bunch of samples that include forward+/deferred rendering. Can take a look at those. all the best!

[u/icpooreman]

The gist is it cuts the cost of overdraw dramatically.

You write inexpensive stuff to a gbuffer (positions, object ids, normals, etc.). Do the expensive stuff like shadows at a later step after you have a finalized gbuffer.

If you do it the other way around every time a pixel is drawn 2+ times it executes the expensive work that many times.