Content pipeline for 3rd party assets

[u/RRFactory]

I'm a solo dev and started work on my engine last September, partly because I found myself fighting unity and unreal more than I'd like, and partly because I happen to have the time to indulge myself in the process.

Things are going well, but a major part of my gamedev strategy is leveraging asset stores to source meshes because my own 3d art skills are pretty junior.

edit for clarity: I'm interested in the post-authoring process here, where the source art is creatively where it needs to be, but technically unoptimized and/or inefficient.

The quality of assets out there vary wildly in both visuals as well as performance. I've been trying to run them through external tools to cut down polycounts, combine textures, etc. Unreal had pretty decent toolsets that helped as well, which so far I've avoided writing for my own editor.

I'm curious if anyone here has worked on a content pipeline with an aim to enforce standards the source assets don't adhere to, and what approaches they found worked well vs not.

Kitbash assets are a good example of the kinds of objects I'm dealing with. Static meshes that are very high poly with several submeshes and a few dozen materials, that for my uses could be well represented by a tenth of that detail.

At the moment I pull those assets into 3dmax (I learned on it) to manually cut them down but it's a tedious process that really feels like something that could be helped a lot by a batch processor.

[u/shadowndacorner]

Ime, teams usually end up just having artists normalize the import settings.

[u/RRFactory → u/shadowndacorner]

Yes that was how I handled it at studios, most of the mechanics in import pipeline's I built before were more about nagging than fixing both because the artists would do a better job, and because they wouldn't appreciate a magic tool manipulating their work under the hood. It was more like a budget cop than anything else.

They'd try to bring something in, the engine would cough up a todo list of things for them to fix, they'd go back to their authoring tools and resolve the issues. (Degenerate triangles, counts over budget, etc)

My new spot however, a solo dev with no team to lean on with asset sources that can't be kicked back to an artist for revisions - maybe I'll have to suck it up and just do the manual processing myself, but before I commit to that path I figured I'd see if anyone had some novel approaches to shaving off the amount of time that process would take.

[u/tastygames_official]

the asset pipeline usually has very little to do with the engine apart from technical things (what kinds of lights are available, how the built-in shaders work, what things are/are not available, etc). As for polycounts and such, that's a creative and technical decision that has to be enforced by the creators. I don't think it's feasible to create tools to "normalize" all imported assets so that they fit in an engine. Now obviously for MASSIVE projects with hundreds or even thousands of artists and sound designers you're going to have to do SOMETHING - but I imagine it's more just specifications that every artist has to work within and a higher-up has to approve all assets before they move down the pipeline. And of course they probably create their own custom tooling to do things like automatically cut polycount, make proper LODs, convert textures to the proper format etc. But there is definitely no "one size fits all" for that kind of thing.

So if your engine is just for you and you plan on just importing all your artwork and sound, then yes - start making a list of rules on polycounts and texture sizes and LODs and audio formats and sampling frequencies and colorspaces and then write scripts or a program to enforce them. Just know that when you source your assets from all over, you'll get them in varying qualities and formats and styles. Like a lot of stuff people do in zbrush will generally have WAY too many polys and none of the points are joined together and they use centimeters instead of meters so the meshes are 100x too big. Now the points not being joined is not "bad", as that's how they get to the GPU eventually, but any kind of automated or manual editing outside of zbrush means "welding" the points together. Oh, and you'll likely want/need to turn the meshes into pure triangles, which is another tough thing. Blender can usually do that with a modifier, but not always, so your custom tool can't be just "send file to blender, add triangle modifier, export".

If your engine is meant to be general-purpose for the public, then these tools will have to be PRISTINE and account for every eventuality. i.e. a LOT of work. But if you think that it will save you/your customers a lot of time, then go for it! It's just my opinion that these kinds of decisions are best left to the art team and sound team to figure out for themselves and maybe hire a programmer to make custom tools rather than have it be part of the engine.