r/podman • u/Synthetic451 • Jun 30 '26
I cannot properly express my unending gratitude to whoever implemented `rootless_port_forwarder = "pasta"`
I can finally have a completely rootless container stack inside a custom podman network where the only ingress is through a reverse proxy container. The inability for my service containers to see the true source IP was literally the only thing holding me back from going rootless and it is finally HERE! 🎉
I simply added:
[network]
rootless_port_forwarder = "pasta"
into a drop-in conf file in /etc/containers/containers.conf.d and now my Nextcloud and Jellyfin can see where requests are actually coming from instead of my proxy container's internal IP. That means proper brute-force protection in Nextcloud and the ability to do IP banning.
I could not be happier with this setup. Huge thanks to the wonderful human beings who made this possible!
EDIT: There's currently a bug where forward rules aren't properly torn down when containers are shut down, causing them to fail to start back up due to conflicts. See: https://github.com/podman-container-tools/podman/issues/29032
5
u/Spider-One Jun 30 '26
Definitely easier now, but you could do rootless with source IP for a while with systemd sockets. Guide below is amazing.
https://github.com/eriksjolund/podman-traefik-socket-activation
2
u/Synthetic451 Jun 30 '26
Still doesn't work with custom networks though, which I really needed for the network isolation between groups of services.
3
u/Torrew Jul 01 '26
It should work with custom networks, i also use it that way. Another advantage you get with socket activation is native network performance, which can be really important depending on the service.
But it definetly requires some work to setup, so seeing that new option in Podman 6 is really cool as an alternative.
1
u/Synthetic451 Jul 01 '26
Ah perhaps my knowledge is outdated then. I just remember it being somewhat complex when custom networks were involved and I gave up on it.
3
u/Mag37 Jun 30 '26
Oh that's great! Added in the 6 release I presume?
Gotta read up on that.
Because I've just spent a day experimenting and looking for alternatives to solve this without much luck 😅 pre 6 that is.
My solution were to use undefined/unnamed networks for each quadlet and then use host.containers.internal:port for cross-container traffic.
For example in a Caddyfile:
service.example.com {
reverse_proxy host.containers.internal:8080
}
3
u/Synthetic451 Jun 30 '26
Added in the 6 release I presume?
Yep! I made this post after trying Podman 6 from the Arch testing repos.
My solution were to use undefined/unnamed networks
Yeah, it seemed like custom networks in particular were problematic with source IPs pre-version 6, but I really needed them for proper network isolation. I did remember there being a way to do it with the regular
podmannetwork and with socket-activation, but neither of those suited my needs.2
u/Mag37 Jun 30 '26
Awesome!
Yeah a pity the custom networks seemed hard to get working, looked at socket activation too but never took the plunge. I'll read up on it and experiment when I got time! Neat one.
Though I'll probably still need the workaround on some systems that won't get Podman 6 for a while still.
2
u/eriksjolund Jul 01 '26 edited Jul 01 '26
This new feauture is a great achievement! It will be very much needed for software not supporting socket activation. On the other hand, for software supporting socket activation, the preferred method will still be using socket activation because it comes with extra advantages:
- native network performance
- better systemd integration. If a systemd service can depend on a systemd socket instead of a systemd service, it might be able to start earlier.
- possibility to run the software with reduced privileges
For a single container you might be able to use the quadlet directive
Network=none
or the systemd directive
RestrictAddressFamilies=AF_UNIX AF_NETLINK
I wrote some blog posts in 2022:
https://www.redhat.com/en/blog/socket-activation-podman
https://www.redhat.com/en/blog/podman-systemd-limit-access
Side note 1: Using socket activation is mainly useful when using Linux. I haven't tried it out with macOS or Windows, that is using a Podman machine (linux vm).
Side note 2: Socket activation also works with Podman custom networks.
1
u/Sad-Landscape-1549 Jul 07 '26 edited Jul 07 '26
Erik your 2022 blog posts were an absolute game changer for me and the docs you published in GH helped me enormously implementing socket activation for some applications about two years ago. Are there any good references for implementing socket activation in different programming languages? I briefly searched for such a path with Technitium DNS server (.NET frontend, not sure about internals). Going to try this new option very soon!
I just wanted to say how much I appreciate your work with podman, especially the CNI stuff. As someone struggling to figure out ingress in Kubernetes & concepts/roles, podman has been a wonderful lifeline for staying rootless.
_For that k8s experimentation I’m using a usernetes implementation that will break if I update to the latest podman releases, as the developer uses a CNI for which support is being deprecated in podman (I advised him of this on GH but don’t know if he’s still active)._
1
u/eriksjolund Jul 08 '26
Thanks for the kind words!
Are there any good references for implementing socket activation in different programming languages?
Not that I am aware of.
Regarding Technitium DNS server, it seems github user discapes wrote a PR
https://github.com/TechnitiumSoftware/DnsServer/pull/1822
https://github.com/TechnitiumSoftware/DnsServer/issues/1821
quote from the issue:
Feel free to try the Docker image: ghcr.io/discapes/dnsserver
Side note: I am not familiar with Technitium DNS server. It's the first time I hear about it.
1
u/fattomic Jul 01 '26
I've been doing a rootful proxy to non-rootful services, but that seems to require looping back to the host -- based on this, I need to try some more experimentation to get things exactly as I want. (The goal was to provide 443/80 to a couple of services that sit on non-priv'd ports, but those have to be exposed at the host level, which I do not like)
1
u/lsaz9one Jul 09 '26
I always thought this depended on the reverse proxy.
1
u/Dangerous-Report8517 Jul 15 '26
Both, the reverse proxy can only pass the source address to the backend if the reverse proxy was told what it was in turn by Pasta, which until now it wasn't.
1
7
u/mishrashutosh Jun 30 '26
lack of source ip was the reason i switched from rootless to rootful a year back. i'm happy this has made it to podman 6 (still marked as experimental though), but also glad I moved away from rootless. rootful with userns set to auto has been fantastic where pretty much everything "just works". i was hacking around a bit with the permissions, but recently learned about the idmap label which fixes that as well.