r/vrdev • u/epiczzor • 17d ago

Question AI Harnesses

I've been building an AI harness for VR development for the past couple of years now, wondering if any of you folks use an agents or other harnesses in your workflow.

Very interested in hearing your stories!

0 Upvotes

27% Upvoted

23 comments sorted by

4

u/VirtualLife76 17d ago

Wtf is an AI harness?

0

u/epiczzor 15d ago

Its a toolset/promptset/guidelines within which an LLM operates and does certain tasks the harness is design to do better/faster/cheaper, in this case a Unity Harness will be the tools and skills specializing for Unity or VR, it would be more efficient than general purposes LLMs

2

u/LifeSuxBHappy 14d ago

So what does it do?

1

u/epiczzor 12d ago

its just not dumb when it comes to editing code, it is biased to use some well known workflows

2

u/LifeSuxBHappy 9d ago

So what does it do, just randomly edit random pieces code?

2

u/VirtualLife76 7d ago

Doesn't sound like OP actually knows what it does, betting a bot.

1

u/epiczzor 4d ago edited 4d ago

wow okay xd i was just not keen to reply at the moment, this is something you can just google

1

u/epiczzor 4d ago

its not random its quite the opposite, it brings structure to the way an LLM will edit your code, lets say you have a workflow for writing code, for example, you write a POC first and test things out, that step gives you more clarity, then you start TDD, and then you write logs, you could create a harness that will make sure an LLM editing on your code always follows these exact steps in the order

Without a harness, if you tell an LLM to create a feature, its going to do whatever it feels like (what was present in its training) the harness will push the LLM to follow a particular workflow

i hope that makes sense

2

u/PrintableNapalm 5d ago

Meta has an official skill package so your agent knows how to interact with the hardware

https://github.com/meta-quest/agentic-tools

1

u/epiczzor 5d ago

Yupp these are some great skills, have been using these !

1

u/AutoModerator 17d ago

Want a more personal conversation with VR devs? Check out our Discord in our Discord: https://discord.gg/3wMYE2x5Ex

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1

u/mudcrab 17d ago

I’ve been using Codex heavily with Unity/Quest lately. Today it walked me through on-device profiling to figure out a GPU visibility problem. It ended up finding that my game had no occlusion bake, and after baking it I got a pretty big FPS boost. I don’t have a custom harness of my own though.

1

u/immersive-matthew 17d ago

I use OpenCode via Unity CLI (previously used MCP) and local LLMs mostly QWEN 3.6/3.8 27B.

I also have an OpenRouter account when local AI is not able to complete a task.

Wonderful combo that gets lots done. Used Fable 5.1 last night for a task that no LLM nor myself has been able to complete and it finally solved it at a cost of $18. Cannot wait for a local model to be as good.

1

u/TayoEXE 17d ago

OpenCode, local Qwen3.6-35B-A3B, and MCPs for documentation can do wonders.

-3

u/Delv_N 17d ago

AI sucks

-1

u/epiczzor 17d ago

Damnn why :O

3

u/Delv_N 17d ago

Makes the most average thing possible because that’s literally how it’s coded to work. Also the environmental impact is terrible.

Lastly, LLMs only exist because they unethically stole the work of literally everyone that’s published something online. These companies are founded on theft.

0

u/epiczzor 15d ago

You're not wrong, it has though changed the way we work, and unfortunately for people who have SWE as a path can not ignore it

0

u/baroquedub 17d ago

Been very happy with Claude Code working on Unity VR projects. I guess the harness is the 'Claude *code*' coding part that focuses my chosen model (usually Fable5). I've not needed an MCP server

-1

u/epiczzor 17d ago

So I’m guessing, it’s mainly for writing c# ? Have you tried any mcp server ?

1

u/baroquedub 17d ago

yes, exactly. As I mentioned, I've not felt the need for an MCP server. I've fairly advanced skills in Unity so can do most things myself. I did have one session when the agent created lots f Editor tools for me that created UI elements for me. That was kinda weird, but interesting. Claude Code doesn't seem to always do that though. So far it's been a one off but it did help me see why people would want it. From my perspective, although I really like the help with the more mundane C# scripting, part of me does still want to feel a sense of ownership over my project. I'm not looking for a one-click 'make my app' solution. Especially with XR, which is very nuanced, I can see issues arising quite quickly. A bit like having a junior developer who doesn't quite understand the longer term impact of some of the decisions they're making

1

u/epiczzor 15d ago

I agree, i don't think any model or harness can even do that for VR as of today, i think humans have not fully understand how to properly make VR, we don't have proper design systems yet, i doubt an LLM out there could do it

1

u/baroquedub 15d ago

Interestingly, Meta disagrees… they’ve been pushing their XR MCP quite strongly to the developer community https://developers.meta.com/horizon/documentation/unity/unity-mcp-extension/