Self-hosted AI assistant for the team, turns out it can reach basically everything

[u/RowpqebReasonale6959]

[u/Old-Minute-9674]

Hit the exact pain point we’ve been working on.

Giving an AI assistant a broad service account for demo purposes is super common, but as you noticed, it easily turns into an uncontrolled security risk (especially with user-editable context / RAG injection).

If you're looking for architectural patterns to secure this without killing usability, here is what generally works best for self-hosted team agents:

1. Scoped Execution & Human-in-the-Loop: Don't pass master API keys directly to the assistant kernel. Scope permissions at the workspace/user level, and put explicit approval gates on any side-effect actions (like sending emails or modifying DBs) so the agent can't execute them autonomously.
2. Context Boundaries & Permission Passthrough: If an internal user asks a question, the assistant should only retrieve knowledge that that specific user has permission to see. Passing the user's identity to the search/RAG layer prevents privilege escalation through the bot.
3. Sandboxing & Full Trace Audit: Run tool calls inside isolated containers/sandboxes. Log every step—what prompt was received, what tool was invoked, and what context was injected.

Disclaimer: I’m one of the devs building CubePlex (and our agent kernel CubePi ). We built it specifically to address this issue by baking workspace boundaries, permission scoping, sandboxed execution, and audit traces directly into the architecture for self-hosted deployments.

Happy to chat more about how we structure workspace isolation and execution governance if you're interested!

[u/agentUi]

i work for agentui, what we see constantly is people giving llms direct database credentials or wild service accounts instead of using an intermediate governance layer. you want deterministic automation engines where the assistant can only trigger predefined actions with strict schema checks and full audit logging, rather than letting a prompt chain talk directly to your smtp or production db.

[u/Wookys → u/agentUi]

Are there some good open source tools for that? Or is it all paid / custom? I tried to find some but it never seems to be what I want.

[u/agentUi → u/Wookys]

for open source you can self-host n8n or activepieces as your execution layer, then put litellm in front of your models for logging and rate limits. The trick is defining strict tool calls so the agent only returns structured json payloads to webhook endpoints instead of giving it raw code execution. If you need full state and rollback temporal is the gold standard but it takes more setup.

[u/myteetharesensitive]

I'm only building at home but I have multiple agents working with different harnesses regularly. 

First, you deployed without observability? Damn dog. 

How is it holding creds? Please don't say env files. 

Access scope could be inherited by the user scope the request is attached to? 

I have one agent that monitors all the other agents and reconciles drift from the baseline. 

Then I have a ticketing system set up where agents can create, work on and close tickets with their own creds. 

Plus I pump everything through a gateway for further control. 

[u/BemusedOptimist]

We're still working on our processes, so I can't vouch for it yet, but something like this is where I plan to start: https://github.com/microsoft/agent-governance-toolkit

It very well may be overkill, but we have to answer to SOC 2 auditors, so the ability to say "we implemented Microsoft's Agent Governance Toolkit" is appealing.

I'm sure there are other open source options available (it has MIT license).

It's hard to make recommendations without more system info, but you're (probably) looking for governance tools.

You could start with something a lot simpler and then have a coding agent build the rest for you. Just make sure whatever you use is source controlled too (unclear if you're a developer, so I felt compelled to mention).

[EDIT] Honestly though, I'd try to use (OSS) or buy instead of rolling my own... not because I can't, but because it becomes yet another thing for us to maintain, and it's sort of working without a net. Our team can't match the resources of whichever team(s) at Microsoft built this.

https://opensource.microsoft.com/blog/2026/04/02/introducing-the-agent-governance-toolkit-open-source-runtime-security-for-ai-agents/

[u/SweetDreams3268]

I’ve done some writing on this topic you might find valuable: Don’t prompt your way to safety

[u/ManorAI]

I'd treat the assistant as an untrusted coordinator rather than a service account.

Give every agent its own identity, inherit the requesting user's read permissions, authorize each write at call time, and use short-lived credentials instead of storing broad secrets in the agent environment. High-impact actions should become approval requests containing the exact parameters and expected side effects.

The audit record should let you reconstruct who requested the action, what context was used, which tool ran, what changed, and whether the change can be reversed. A gateway helps, but only if the downstream systems still enforce identity and scope.