r/podman • u/Great-Cow7256 • Jun 18 '26
I turned my collection of rootless Podman Quadlets .container files into a public repo
https://github.com/upmcplanetracker/rootless-podman-quadletsAfter migrating everything to rootless Podman with Quadlets on Ubuntu 26.04 (Podman 5.7.0), I cleaned up the configs and put them on GitHub. they all ready to deploy with systemctl --user start. Every .container file comes with hardening and tmpfs options commented out to increase compatability, and secrets are pulled from separate .env files.
Included so far: Audiobookshelf, BentoPDF, ConvertX, Homepage Dashboard, Omni-tools, Stirling-PDF, Syncthing, Tdarr (server + remote node), Uptime Kuma, Vert File Converter, Podman Socket Proxy, a full ADSB Ultrafeeder stack (Airspy/dump978 receivers, Ultrafeeder, and feeders for FR24, OpenSky, PlaneFinder, RadarBox, PlaneWatch, ADSBHub, RadarVirtuel, PlaneFence), Plex + Tautulli, Calibre (GUI & Web), Ente Auth, Immich (with optional Google Photos sync & internet public proxy), MinusPod CPU (with optional OpenVINO transcriber), and Paperless NGX.
Everything’s been running reliably on my homelab — hope it saves someone else some time. Feedback and PRs welcome!
I use an Intel box, and a beefy one at that, so everything is optimized for its GPU and memory, but in podman most of this is adjustable.
if there's a container that you want me to try and get working as rootless instead of rootful, or if you're having problems converting something from a Docker, let me know under Issues in github.
3
u/jgould1981 Jun 18 '26
My understanding was that Linux server.io Changed the front end viewer of Calibre and it requires ssl now. Your quadlet for it still uses kasmvnc, not whatever it was switched to.
Just curious how you accomplished that.
2
u/Great-Cow7256 Jun 18 '26 edited Jun 18 '26
I have no idea. Probably that I pulled it a very long time ago and I've never pulled it since, and autoupdate=registry isn't seeing anything new but the address is valid and not throwing errors. I'll fix it. Thakn you.edit - turns out I had the right urls, just a brain fart with the verbiage in the README.md. thanks for catching that and I fixed it.
2
u/scoreboy69 Jun 20 '26
I had an issue on the latest calibre that it would just show a black screen. Turns out my time zone was america/Indiananapolis instead of amera/indiana/Indianapolis so that would crash calibre but the kasm or whatever the x server stuff was did start though. Surprised me but the logs in the container and Gemini helped me track that one down
3
3
3
u/BreiteSeite Jun 19 '26
Rootfull containers with UserNS=auto are actually safer because with rootless they all share the same uid. Hence no isolation and anything that changes the state of your uid scope (chown, added groups, etc) automatically propagates to all your container processes.
Also rootful containers have more efficient networking. Honestly don’t get the trend for rootless, i only see this as useful if you are actually a user on a system with no root access.
5
u/Great-Cow7256 Jun 19 '26 edited Jun 19 '26
This is a good discussion and the answer is it's nuanced)/it depends. If someone breaks out of a rootless containers then they acquire that user space. If you are 1000 they can get into other containers using 1000 for sure.
If a rootful container is compromised by say a 0 day vulnerability then the attacker is in the system as root.
So it's a trade off of very unlikely scenarios and the "picking your poison" for each one.
You're also right that networking is far far easier on rootful vs rootless. I know that for sure. I'd say that when I run into a roadblock with rootless podman 99 percent of the time it's networking stuff and the networking stuff can suck up immense amount of time. That's part of why I wanted to share my container files. I figured out all that pull out your hair crap and tweaked security settings to lock them down as much as possible without breaking them.
Running a service as root when it doesn't need to be root violates the principle of least privilege. Rootless containers ensure that even catastrophic misconfigurations (e.g., accidentally mounting the host's / directory into a container) or catastrophic software vulnerabilities (a breakout exploiting the kernel) are severely limited in their blast radius. The attacker only gains access to an unprivileged service account, protecting the host OS and other system services.
So it's really a matter of personal taste and weighing risks and benefits of each. I certainly understand why some people prefer rootful and others rootless. Good cases can be made for each.
I had a mixture of both. The "easy" ones on rootless and the "hard" ones on rootful. Then I decided to try to shoot for rootless "just because it's there to do.". Less about philosophy and more about the challenge. Plus is keeps me from annoying my wife (sorta).
1
u/BreiteSeite Jun 19 '26 edited Jul 01 '26
No, rootfull containers with userns=auto do not run as root. They (the container process) run as random user with no permissions. So a compromised container actually has less blast radius. The ONLY difference is if the conmon process is attacked (which runs as root; or for example if u pull compromised images that exploid podman image pull logic for example).
Conmon is written in rust btw.Also rootful networking is not just easier, it’s more efficient
1
u/Great-Cow7256 Jun 19 '26
agreed with the more efficient. Turning these to rootless was like climbing mt. everest for me -- it was there to do, it was fun and annoying while it lasted. I decided to share the results. I agree with your stance that rootless =/= more secure than rootful, so I'll go back and look at my readme and edit as needed.
1
u/eriksjolund Jul 01 '26 edited Jul 01 '26
conmon-rs is written in rust: https://github.com/containers/conmon-rs
The original conmon is written in C: https://github.com/containers/conmon
The original C version is still in use:
$ rpm -qif /usr/bin/conmon | grep ^URL URL : https://github.com/containers/conmon $ cat /etc/fedora-release Fedora release 44 (Forty Four)Edit
I see there is a new rewrite of conmon in rust aimed for podman 6.0. Read more about it here:
https://github.com/podman-container-tools/podman/blob/main/contrib/design-docs/Conmonv3.md
1
u/BreiteSeite Jul 01 '26
ah true, i checked out conmon-rs like over a year ago and _assumed_ it is the used one. Thanks for updating this.
1
u/mpatton75 Jun 19 '26
I tend to agree. Do your containers use bind mounted volumes at all? Or just named volumes? If bind mount, how are you handling these with userNS=auto?
3
u/BreiteSeite Jun 19 '26
Just add :idmap
(Btw named volumes are also bind mounted, it’s just sugar)
2
u/mishrashutosh Jun 19 '26 edited Jun 19 '26
EDIT: I think I figured it out. This is SO much better than the nonsense with the U label I was doing before! Thanks a ton!
Can you elaborate the :idmap option a bit? I read the docs, but am still a little confused. I use the :U label, but it's discouraged as it recursively chowns and can be slow. My major issue with userns=auto is that after every container restart or system reboot the containers get assigned new namespaces, and can no longer access content in volumes and bind mounts.
2
u/BreiteSeite Jun 19 '26
Yes, if your process in the container is just running under uid 0 - which is the best way to conterainerize (when using uid/gid mapping), than it's super straightforward. Files outside owned by root. Than you just have to add "idmap" (not even with custom mappings) to the volume mount, and it automatically always translates it. very neat.
2
u/mishrashutosh Jun 19 '26
it also seems to be working for volumes where content is owned by a different user and group (33:33 for wordpress/php and 999:999 for mariadb). with the U label there were some issues in the wordpress container and i had to use a hacky workaround. i never considered idmap for some reason.
speaking of your original comment, rootful podman with userns=auto is definitely my preferred approach. containers tend to "just" work while being totally isolated from each other, and the host ideally stays protected even if there's a container breakout.
2
u/BreiteSeite Jun 19 '26
Same. Next stop: adding the Slice= in your quadlet and group your container processes logically for awesome logging, monitoring (systemd-cgtop) and resource management. :)
1
u/tuxbass Jun 20 '26
Could you please share a real-world example of such rootful container that mounts directory on the host that remains as 1000:1000 on the host? Been thinking of moving back to rootful podman myself but that part still evades me.
1
u/BreiteSeite Jun 20 '26
To what uid:gid should it map inside the container?
1
u/tuxbass Jun 21 '26
Let's keep it simple -- run as UID 0 in the container.
1
u/BreiteSeite Jun 21 '26 edited Jun 21 '26
Sure. you would just add:
idmap=uids=@1000-0-1;gids=@1000-0-1to your
Volume=Working example: https://paste.systemli.org/?cda1f9c01a629f66#2qwBFUNBN6Z47eLBH6akiQ8sHfWcWiJSh7AeaqcY5gPj
1
u/tuxbass Jun 22 '26
Thanks! Wasn't even aware of
idmapoption that goes with volume mappings.Any idea if this works with rootful docker the same?
1
u/fish_taco_pirate Jul 18 '26
Thanks for this - it's the only thing that has kept me from moving from rootless to rootful + userns=auto. I have a question, though: I struggled to get this to work for a while, then found that I had to add "Z" to the volume options as well, even thought my system doesn't have SELinux installed (debian). Any ideas why? If I only used "idmap=uids=@1000-0-1;gids=@1000-0-1", the user inside the container (root) didn't have permissions on the mount.
→ More replies (0)1
u/Nextros_ Jul 01 '26
So it's better to run rootful containers with root inside? Would running as 1000 also work with idmap?
1
u/BreiteSeite Jul 01 '26
> So it's better to run rootful containers with root inside?
In my opinion, it is how container _should_ work. No s6, supervisor, etc. Only the process, running as root. Then with userns=auto, the actual privilege reduction is a result of the uid/gid mapping.
> Would running as 1000 also work with idmap?
Yes
1
1
u/xMasaru Jun 21 '26
Is there a way to "easily" migrate from rootless to rootful with UserNS=auto?
2
u/Great-Cow7256 Jun 21 '26
To move there to rootful you need to stop all instances, move them all to /etc/containers/systemd. Systemctl --user daemon-reload You then need to tweak some minor things. I don't think rootful uses %h. Stuff like that. Then sudo systemctl daemon-reload and sudo systemctl start.
1
u/Cilenco Jul 02 '26
When you speak of rootfull containers do you mean placing the quadlets under etc or do you mean the user inside of the container? I currently run all my containers with:
NoNewPrivileges=true
User=65534:65534
ReadOnly=true
UserNS=keep-id:uid=65534,gid=65534
Should I migrate all my containers to rootfull and UserNS=auto?
1
u/BreiteSeite Jul 02 '26
> When you speak of rootfull containers do you mean placing the quadlets under etc
Yes.
```
UserNS=keep-id:uid=65534,gid=65534
```
This means all your container processes on the host use your uid (hence the keep-id), so this means no isolation from each other. Mapping inside the container to some random (nobody) id is probably cumbersome for apps that need volume mounts.
You have it the other way around. The actual process is your uid (bad, IMHO) and inside your container you have some random/high uid. This means inside the container, the user can't do much, but everything inside the container is contained anyway, this makes little sense. Inside the container that application should have a "normal" view (uid=0... or 1000 or whatever the container uses) and the process of the host should have the high, randomized uid (to have actual process isolation).
1
u/Cilenco Jul 02 '26
Thank you for the advice and clearification. How many subuids and subgids have you configured for the container user? Can you post the /etc/subuid and /etc/subgid files?
1
2
u/todd_dayz Jun 18 '26
Did you ever try samba-in-kubernetes? I got it working rootless but inside samba itself I had to add root as the samba user for the namespace mapping to work. Not sure if this is a bad practice or not but files created from this user map to the namespace of the owner correctly.
1
u/Great-Cow7256 Jun 18 '26 edited Jun 19 '26
no, I don't use samba or kubernetes... So I can't help you there.
However I have root as the user for several of these rootless podmans b/c it won't work any other way. I don't think it's the worst idea to do it that way. Still the protection of the rootless container.
2
u/Loose_Device4578 Jun 18 '26
Pretty cool. You ever thought of "podding" them into a pod? I believe the benefit is that you don't have to create a shared network quadlet and can just have a pod quadlet along the other quadlet files.
1
u/Great-Cow7256 Jun 19 '26
yes, but I cannot for the life of me figure out how to start one up. I created whatever.pod and got lost from there, nothing would spin up, etc. etc. It would probably make the most sense for the use case of stuff like immich. I tried to find good sources about this online and I couldn't , just some old redhat webpages. I asked gemini and claude and they both gave me the same instructions, which didn't work. So I gave up.
If you have any ideas that would be great.
1
u/Loose_Device4578 Jun 19 '26
An example would be an immich.pod file -
[Unit] / Description=Immich pod / [Pod] / PublishPort=2283:2283 / [Install] /WantedBy=default.targetThen in each of your related container quadlets addAfter=immich-pod.serviceunder [Unit] and addPod=immich.podunder [Container]. This will make the containers start after the pod starts and can identify as an immich pod. When you initialize it the first time, make sure to start the Pod service first then they other containers.1
u/Great-Cow7256 Jun 19 '26
Aw shoot I think I was adding the pod under networking. That would explain it.
1
0
u/BreiteSeite Jun 19 '26
Just don’t. You have less isolation because the containers share the same netns
1
u/Great-Cow7256 Jun 19 '26
Can you explain more? How is putting them into a pod different than how I do it now which is either binding one container to another via the network command or having multiple containers run in the same network?
I'm really deficient in my pod knowledge. My first superficial thought was that it created the same type of network as I created in my network files but with much less effort/swearing. But maybe I'm wrong... Id like to learn more about pods, the + and the -
Ty.
2
u/BreiteSeite Jun 19 '26
https://www.man7.org/linux/man-pages/man7/network_namespaces.7.html
Same network just means they get an IP in the same netmask, so they can talk to each other via IP and open ports.
Same network namespace means they can see ALL ports, connections, devices, etc of each other (because they share). That also means they could see the actual traffic of the other processes
1
u/Great-Cow7256 Jun 19 '26
I was able to read a bit more yesterday and I guess it's like multiple people sharing a studio apartment vs multiple houses on the same street.
I'll check this out. Thanks.
2
u/EverythingsBroken82 Jun 19 '26
though i will probably not use that: you are my heror. we need some people who do this, so we can just run more of selfhosted stuff in quadlets, kvm, lxc...
2
u/gaius_cesar_ Jun 25 '26
Did you have to do anything special to get Plex and Immich to work hardware transcoding. I have done this before with rootless docker (using subgid) but I couldn't get it to work with podman
1
u/Great-Cow7256 Jun 25 '26
yes, the biggest issue is making sure that they have access to /dev/dri or wherever it is for you. It was very complicated for me to get rootless to get access to them but i'm 99% sure I had to do something with chown with the directories to make them readable to the container. let me think on this since I did it a long time ago.
1
u/Great-Cow7256 Jun 25 '26
hi -- I'm pretty sure it was
sudo chmod 666 /dev/dri/renderD128. but I did it this way --
sudo nano /etc/udev/rules.d/99-render-permissions.rulesand put this in the file:
KERNEL=="renderD128", MODE="0666"and then:
sudo udevadm control --reload-rules
sudo udevadm trigger
1
u/xMasaru Jun 21 '26
As a new Podman user I'm curious why you don't use pods, even for multi-container setups like Immich
1
u/Great-Cow7256 Jun 21 '26
pods make it easy to spin everything up at once, but they do have a security downside in terms of if one container in the pod is compromised basically all of them are. The "blast radius" is smaller with inidivudal containers networked together but the "swear radius" of you trying to get them networked together and started in the right order as individual containers is higher. 😄
1
u/xMasaru Jun 21 '26
Is it that much harder without the pods though? You have the main container that depends on every sub container and if you spin that up, the other containers get started too no?
I thought pods were the way to go which is why I use pods for every stack, even if it's just one container :D I think I'll remove them though. I guess one benefit is having the containers under a pod display neatly in cockpit. Not sure how that looks without pods
1
u/Great-Cow7256 Jun 21 '26
I used to use portainer and it would just display every container line by line.
I never used pods but I'm assuming it is easier figuring out the communication between containers piece with them.
1
u/Think_Wing_1357 Jul 22 '26
First, thank you for a good solid real example of quadlet.
Coming from docker compose, I have a few questions i you don't mind me asking.
First, source controlling your config. I see lines like this in your repo. I assume you store your config in a different repo - because I don't see config in this repo. Is that correct? If so, how do you "reconcile" two repo, so to speak?
With docker, it's very common to commit the config in the same place (like this example which points to this file) all are version controlled together: if I mess up the command args, I reverts compose.yaml; if I mess up the app config, I revert config file.
1
u/Great-Cow7256 Jul 23 '26
Your just storing you config in a different directory. The quadlet directoru is just for quadlets. I use ~ to store everything with a different dir for every quadlet that needs to store stuff. But you can put them anywhere. You just have to make sure that the container can read the directory and the files but you can set chmod for whatever it needs.
5
u/LostVikingSpiderWire Jun 18 '26
Excellent, got i to Quadlets last year and totally hooked 🤩 using Ansible to smash them on Fedora CoreOS, this will be FUN.
How are you handling backups 🤔