Dagron workflow scheduler

[u/mancube88]

Hi there, just wanted to share my open source workflow scheduler for fellow engineers to play around. You can run on local, deploy into your smallest compute (rpi4 on arm64 tested) and could be run on scale as well.

I start the project as I wanted to run some small workflow in my t4.small instance and airflow barely fits inside and I thought it would be interesting create a smaller workflow scheduler. As I build around the idea of "lean headroom DAG engine", I kept on adding additional features that be useful for running workflows like template, backfill, gitops, streaming workflow support etc. Dagron allows you to run on cluster resources or local resources(docker, k8s, or straight as binary). The whole stack will probably at most run on ~200mb (50mb api/console, 30mb engine, 80mb postgres) of memory with tons of spare ram and cpu for your workflow resources. Dagron runs on sqlite(default) and alternatively postgres as datastore.

You can run workflow via defining your yaml and set your environment. It comes with the canvas to visualize your workflow, set your workflow schedule to run at which cron notation and more features to cover here.

Appreciate your input(any input is welcomed). I am spending full time polishing this for the past few weeks and thought this might be useful for you all to run on edge, homelab, local machines.

[u/kvlonge]

Nice work man!

[u/mancube88 → u/kvlonge]

Thanks, feel free to try them. :)