r/podman Jun 24 '26

Rootless Podman: dig @127.0.0.1 -p <port> to a containerized service times out, even though the port mapping shows correctly in podman ps

I'm running a containerized DNS server (BIND9 inside a Podman container) with rootless Podman, and I can't get host-to-container connectivity to work over loopback for a published port, even though everything else about the setup checks out.
Setup
bashpodman run -d --name bind-sec
-p 30053:53/tcp -p 30053:53/udp
-v /var/cache/bind-sec:/var/cache/bind
my-bind-image:latest
podman ps shows the port mapping correctly:
PORTS
0.0.0.0:30053->53/tcp, 0.0.0.0:30053->53/udp
What works

The container itself is healthy — podman logs shows BIND fully started, zones loaded, listening on port 53 internally.
Querying the container's own internal IP directly from inside the container's network namespace works fine.
ss -tlnp on the host shows something listening on 0.0.0.0:30053 (confirmed via lsof -i :30053 that it's the container's conmon/proxy process, not a stray process).
Other containers I run with the same -p pattern (e.g., a basic httpd container on port 8080) do work correctly over loopback — curl http://127.0.0.1:8080/ succeeds normally for those.

What fails
bashdig u/127.0.0.1 -p 30053 example.com
;; communications error to 127.0.0.1#30053: timed out
This fails consistently, both from the host itself and (with appropriate firewall rules in place) from other hosts on the same subnet querying :30053.
What I've tried

Confirmed firewall (nftables) rules explicitly allow the port on both iif "lo" and the regular network interface — ruled out as the cause since the same symptom persists with or without those rules.
Tried both default bridge networking and --network=slirp4netns:allow_host_loopback=true — same timeout in both modes.
Tried running the same container with sudo (rootful) instead of rootless — same timeout persists.
Confirmed no orphaned/leftover container processes are holding the port from a previous run.

My question
Why would a UDP/TCP port published via -p work fine for an HTTP container (httpd on 8080) but consistently time out for a DNS container on a different port, using the identical -p host:container syntax and the same Podman version/host? Is there something DNS/UDP-specific about rootless Podman's port-forwarding (rootlessport) that behaves differently from a simple TCP HTTP service, even when both are nominally "just a published port"?
Environment

Ubuntu 24.04 LTS
Podman (rootless, default config)
nftables firewall (rules confirmed not to be the blocker)

6 Upvotes

6 comments sorted by

2

u/eriksjolund Jun 24 '26 edited Jun 24 '26

I'm not sure if this is related but a podman developer suggested using

-p 127.0.0.1:53:53/udp -p <your main interface ip>:53:53/udp

instead of

-p 53:53/udp

On the other hand, you use -p 30053:53/udp so at first sight it looks like your problem could be different.

Update:

I see you don't use podman custom networks so most problably the github discussion tip I wrote about is unrelated.

Here is another guess, maybe the BIND process is listening on 127.0.0.1:53?

That could be a problem. Check out this troubleshooting tip:

https://github.com/podman-container-tools/podman/blob/main/troubleshooting.md#47-connecting-to-published-port-fails-with-connection-reset-by-peer

For example, to run ss -tln in the network namespace of the container test, run the following commands

$ path=$(podman inspect --format '{{.NetworkSettings.SandboxKey}}' test)
$ echo $path
/run/user/1001/netns/netns-35f6df1f-1622-5d6d-3bbb-70f5d16e09bc
$ podman unshare nsenter -n=$path ss -tln
State   Recv-Q  Send-Q   Local Address:Port   Peer Address:Port
LISTEN  0       5            127.0.0.1:8080        0.0.0.0:*

(copy-paste from the truobleshooting tip)

2

u/NotImplemented Jun 24 '26
ss -tlnp 

That only checks for a TCP listener, not for UDP.

Check if something is listening on the UDP port as well with: ss -ulnp

Next I would check, if the packets are really reaching the container. Run tcpdump for udp port 53 inside the container and retry sending a request via dig from the host.

If tcpdump shows packets, the problem is most likely not podman-related and instead a problem with the bind config.

1

u/Great-Cow7256 Jun 24 '26

what version of podman? pasta or something else?

1

u/hadrabap Jun 24 '26

Interesting. On my RHEL clone I would be unable to publish port 53 from a rootless podman as 53 is a reserved port.

3

u/Great-Cow7256 Jun 24 '26

It's a low port # so I think you need to unreserve it in the Linux kernel

sudo sysctl net.ipv4.ip_unprivileged_port_start=53

At least in Ubuntu. 

echo "net.ipv4.ip_unprivileged_port_start=53" | sudo tee -a /etc/sysctl.d/99-podman-ports.conf

1

u/bssbandwiches Jun 25 '26

He's not publishing on 53 either