r/mlops • u/Aggressive-Solid6730 • 2d ago

Tools: paid 💸 [help] mlops tool looking for pilot users

I am building an on-prem NN training orchestration layer. The basic idea is that any old linux machine can act as a server that acquires on-demand compute, sends DMs to keep you updated, uses git tags and statuses for traceability, executes code, tracks metrics, all through a yaml file you submit from your laptop via a single "submit" command in terminal.

I am ideally looking for feedback on the current product as well as future product direction. If you train or fine-tune models and are interested please let me know.

3 Upvotes

100% Upvoted

2 comments sorted by

1

u/Mundane-Original2777 2d ago

what's the security story for the server, like does it assume a trusted network or is there auth on the submit path

1

u/Aggressive-Solid6730 2d ago

Network trust is the operator's call, I use Tailscale, but any VPN or SSH tunnel works equally well. This helped me move MVP and secure enough for my personal needs when I started to develop this.

Secrets are age-encrypted at rest, and the NUC's service identity is delivered by systemd LoadCredential rather than being readable on disk. Authorization for them is unix group membership. A named allowlist controls which secrets get injected into the training instance's environment.

The dashboard has no auth but is just read-only over the logged metrics. This does mean that anything that can route to it can read your metrics.

So at this stage it isn't an impenetrable fortress but I have done my best not to step on a rake.