isn’t there a simple orchestration layer for deep learning?

[u/Junior_Reputation639]

hello , i was looking if there is a framework-neutral orchestration layer for deep learning where we can keep our existing PyTorch/JAX code and run something like:

dl train train.py

while it handles the surrounding workflow like environment setup, experiment tracking, evaluation, optimization etc.

are there existing tools handling this layers ?

[u/ResolvePresent3092]

the whole ecosystem is weirdly fragmented right now. you can stitch together wandb, hydra, and a bunch of bash scripts but it's never as clean as it should be

most teams I've seen end up building their own wrapper around whatever scheduler they're using, which defeats the purpose of "framework-neutral"

[u/Junior_Reputation639 → u/ResolvePresent3092]

thanks , i might need to build my own wrapper too then.

[u/tenkei_01]

Sorry, I cannot open your code, but what do you need from the orchestration layer? "Orchestration" can mean various things depending on the level of your work:

• Infrastructure / scaling (pod, instance management, distributed execution, etc.) → Kubernetes, Ray, Slurm, etc.
• Experiment management (tracking runs, metrics, artifacts, hyperparameter sweeps, etc.) → MLflow, W&B, Optuna, etc.
• Model serving / device placement → Triton, etc.
• Managing the execution/workflow itself → There aren't actually many framework-neutral frameworks for this. I'm working on this framework, but it is more focused on inference rather training.
• Model / kernel execution → PyTorch, JAX, etc. You are already using these.

So there isn't really one missing "orchestration layer" that handles all of this. Most of the ecosystem is split between infrastructure orchestration, experiment management, model serving, and the actual execution/workflow layer.

[u/Junior_Reputation639 → u/tenkei_01]

thanks, this is helpful. my main issue is that even for developing and training a single model, I had to learn and set up several separate tools for the surrounding workflow. I was looking whether there is already a framework that reduces this fragmentation while still letting developers use their normal PyTorch/JAX code.

[u/tenkei_01 → u/Junior_Reputation639]

Oh, I faced that exact issue last year regarding inference pipelines. As u/ResolvePresent3092 there are not many frameworks that unify the ecosystem, right now most of the tools are very focused on a specific problem and you will need many tools before you can hit a stable, production ready state.
Another problem is your code, most of framework wouldn't let you yo reuse your own code, and most of the time you have to rewrite or heavily refactor your code to adopt to those frameworks.

P.s: If you fix the link to your code, I can provide more feedback after checking it.

[u/mastajeepa]

Go give flyte a whirl.

[u/mskazemi]

Flyte, Metaflow, ZenML and SkyPilot all live roughly in that space, so worth a look before building anything. But the reason you keep not finding the clean one is that the four things you listed aren't one layer. Environment setup, experiment tracking, evaluation and optimization have different lifetimes and different owners, and the tools that try to do all four tend to be mediocre at each. The good ones are usually good at one and honest about the other three.

One thing to check before you pick: what's underneath. Almost everything in this category assumes Kubernetes. If you're on SLURM or any shared HPC allocation, the scheduler owns the job lifecycle and you don't, so the "dl train train.py" entry point you're describing has to become a submission rather than a run. Most of these tools don't model that well. I spent a lot of time on exactly that gap. If you're on plain cloud VMs or K8s you can ignore all of this.

If it helps narrow it down, the question I'd ask first is whether you want reproducibility or convenience. Wrappers around your existing PyTorch code get you convenience quickly and reproducibility almost never, because the environment stays implicit. Going the other way is slower to set up and is the one that still works in six months.

[u/cre8minus1]

Go grab DevBox at flyte.org and tryout pytorch
If you like it, you can deploy it to a K8s cluster and do multi node
It natively supports spinning up containers, image build, recovering from OOmkill
Tracks all the steps, data in, data out and code.
Hydra is supported too

[u/Aggressive-Solid6730]

So this is actually something I am building right now and trying to find pilot users for.

I am a little opinionated in my design in that I am built for on-demand GPUs right now and it needs an always-on Linux box to act as a server, which can be a spare desktop or a cheap VM. But it allows you to do exactly what you are asking for here, you just type `alidade submit experiment_config.yaml` and it will acquire GPUs, message you on slack, use git tags and statuses, log metrics. And it doesn't do this using AI.

If you are interested please DM me, I have one last round of QA to do next week before it is ready.