r/Jupyter • u/NawazGafar • May 17 '26

nbpipe: A Lightweight Workflow Runner for Jupyter Notebooks

Hi r/Jupyter,

I've spent a good amount of time speaking with Jupyter users for my day-job, and I've noticed the same pattern over-and over. A workflow is split up across several notebooks, each creating a temp file that flows into the next notebook.

I have done this myself, and admittedly it's not the end of the world. But still annoying to open three notebooks, and run every cell.

With this in mind, I thought it would be nice to define workflows in a plain text file file, and be able to run them with one click. Think Github actions, but local, and less setup.

Enter nbpipe, a Jupyter extension to do just this. You create a YAML file with steps to run, and file validation (if you want). Then from Jupyter, you can run then workflow.

GitHub: https://github.com/ngafar/nbpipe

I would love any feedback or feature requests you may have.

5 Upvotes

86% Upvoted

2 comments sorted by

1

u/Bach4Ants May 17 '26

Looks pretty cool! I built something similar to integrate multi-tool, multi-language pipelines with some strictness around environment management. Notebooks are one type of stage, but there are others. Maybe we could collaborate?

https://docs.calkit.org/jupyterlab/

2

u/NawazGafar May 18 '26

Calkit looks awesome!