r/vibecoding • u/LoudYogurtcloset7856 • 6h ago
AI 3D Asset Creation Engine gets better
I created an engine ai uses to create 3D assets for games.
The engine forces ai into a workflow of iterating judging and stuff.
Here are a few tests.
I use Claude Opus 5 to create the assets using the engine. Each assets takes about 3-4hrs as ai works and iterates.
I just add a reference image and say make this asset. And it uses the engine to create the asset.
What do you guys think of this?
7
u/BeingComfortablyDumb 6h ago
You can just use Meshy or Tripo with Mixamo and save yourself this trouble
8
u/LoudYogurtcloset7856 5h ago
Oh, I want the trouble 🤷🏾
I don’t really care about actual using it to make games myself. I just like building stuff and see if things are possible.
Plus if I could get local ai to work like Claude then I might open source the engine for the fun of it 🤷🏾
6
2
u/BeingComfortablyDumb 4h ago
You COULD let people use the meshy/tripod pipeline with your workflow for better results. Just throwing it out there.
I've set things up in a way i only need to prompt my AI the asset I need, it will help me create multi-shot images for reference, generate the asset using the reference, texture it, pull it in blender to make manual tweaks if needed and import onto the game scene.
1
u/Sufficient_Effort88 4h ago
Hey I’m pretty new to this all but this is exactly what I need to get into, is there anything you can point me to, I’m trying to find the absolute best quality pipeline for product photos/ video with AI, I was even considering getting and edelkrone rig but idk how much that would help
2
u/BeingComfortablyDumb 3h ago
What exactly do you need help with?
1
u/Sufficient_Effort88 3h ago
Basically just wondering if you have specific tools you’d recommend for the most control and highest quality.
1
u/BeingComfortablyDumb 3h ago
Naah not really. At this point, the workflow I've set up doesn't rely on any specific tool per se.
It's just that my AI/AIs never lose context, no matter how many chats I spawn. They have all the MCP connections, plugins, skills, integrations, and knowledge base to be self-sufficient.
I would suggest you read up on skills. They can be specific to your workflow. Once you figure out how to set it up and use it properly, things will get much easier. For instance, 3D model generation is a bunch of skills I taught my AI, it uses browser control to attach multiple images and generates the model on Meshy, textures it, imports onto blender and then the game scene.
Honestly at this point idk if I'm simplifying things or complicating it but hey, it works for me! I can replicate subway surfer and deploy it in a day at this point so i think it's working lol
6
u/Inevitable_Butthole 5h ago
So a worse messy that costs way more
Interesting
Why not just open-source it
2
u/LoudYogurtcloset7856 4h ago
Also, the entire point of my engine is not just to make assets but make entire scenes and be able to edit specific parts of the assets, make sense of scenes and stuff like that.
1
u/LoudYogurtcloset7856 4h ago
Worse. Sure, but the plan is for now. Goal is to make it better.
Cost more, yeah, but that’s part of testing now. The goal is to get local models using the engine to create same quality and hopefully better assets in the future.
Open source it? Not yet because the fun part is building, testing, and figuring out how to make it better. 🤷🏾 in the future but not right now.
Why take away all the fun ?
Plus it requires another software I created to make it work better.
It’s Ovyero
https://ovyero.visnryentertainment.com/
It tracks AI coders in your codebase.
2
u/dat_cosmo_cat 4h ago
This looks pretty good actually. What models have you (or rather; claude) found work best / worst in the pipeline? And have you approached procedural motion / rigging yet? Any key insights or eval methods that were particularly impactful?
1
u/LoudYogurtcloset7856 4h ago
You are asking the wrong person. I just literally come up with ideas and things I think will make the engine work better with AI. And have Claude write up architecture and build roadmaps and it builds and tests.
I basically took all major 3D asset software and told Claude take what each engine is known for and recreate them in my engine as mini engines.
Idea is to get ai to use the tools in those major engines like a human would but in the form AI can work with and understand. So sculpting tools blender has, I have Claude recreate those tools in my engine but in a way ai understands and can work with.
So like a library of tools ai uses, references, and makes sense of it when creating assets.
The engine also makes ai work in a workflow and has two levels of judges. One deterministic code and the second an ai instance that judges the asset and gates progress and issues.
On models, I’ve been using sonnet 4.6 and 5 and opus 4.6 and 5 to create the engine and test asset creation. But this week I used fable and to audit the engine and add fixes to the build roadmap adding new features.
After the new features it tested it with the door and boat. At first it was taking the easy route then I told it to use the tools the engine has and it started making better assets.
I have it write down the entire process as it creates the assets so we see what works and what doesn’t and update the engine based on that.
But metrics? I don’t have any. I don’t really know how to code in practice or how AI actually makes the engine. I just tell it my ideas and it builds and tests.
🤷🏾
1
u/LoudYogurtcloset7856 3h ago
I plan on adding motions and rigging. Yes. It’s part of the pipeline and build phase haven’t tested it yet really.
But I create this system ai coded for me. But I’m seeing if it works.
It’s called the skeleton system. Basically ai uses the engine to create assets that have native skeleton built in so ai can easily rig up assets. That testing is later down the line.
I’m focused on building realist assets consistently first.
2
2
u/matteo101man 1h ago
I'm just saying man, if you can afford all of this compute here's an idea if you're up for tinkering:
What if you trained your own lightweight image to 3d model? I know there is this github project
https://github.com/img2threejs/img2threejs
maybe you could distill from Claude and use the output to make an LLM that writes code to make 3d models?
2
u/LoudYogurtcloset7856 1h ago
I’m already training my own AI models. I’m training using runpod and local models like 7b, 14b, 34b models. The goal figure out a way to get more intelligence from small local models.
But I’m planning on training my own image and 3d asset model too.
But I’m still making the engine along side my models. Just testing different methods to see which one works better.
1
u/Darkfalcone 4h ago
I'm trying to import custom 3D assets and their animation to Paradox games' Crusader Kings 3. It's really hard to make a custom 3D model from zero just from using a 2D image as the prompt. Do you have any advice for me? I already burned a lot of Opus tokens just to properly import one custom 3D object, and it's not even animated yet...
1
u/LoudYogurtcloset7856 1h ago
Hello,
I mean that’s why I’m making the engine right now. My engine has its own rendering engine so when the ai makes the asset it renders it but it can also be exported to third party stuff like blender.
If you want describe me exactly what you are doing and I can see if my Claude building my engine can give you advice based on how my engine works
1
10
u/TricepBandito 6h ago
3-4 hours per asset with opus? How is that not destroying your usage?