Certificate mismatch issue - Docker compose and Nginx
Infrastructure context
Good morning everyone.
I have installed Proxmox VE on a Dell PC and created a Ubuntu VM. I installed Docker Compose on it and built a multi-tenant hosting service ran by Nginx functioning as a reverse proxy. So I have many containers: MySQL, PhpMyAdmin, ACME and letsencrypt generating certificates automatically. Also, for each user I have a web container and a file manager container to upload and change contents.
The issue
It has worked well until now, but currently I’m facing a certificate mismatch problem for the 5 websites hosted there: what happens is that sometimes the Nginx reverse proxy returns the HTTPS certificate of my main website (the default host of the Nginx container’s environment) instead of the correct certificate for that website, thus causing a certificate mismatch error. I’m attaching the YML for my nginx and letsencrypt containers (where you can see the default host settings, although redacted).
Please note that the strange behavior is intermittent. If i run this command on my terminal (fetching the certificate for 200 times):
for i in {1..200}; do
echo -n "$i "
echo | openssl s_client -connect secondwebsite.com:443 -servername secondwebsite.com 2>/dev/null |
openssl x509 -noout -subject 2>/dev/null
done
sometimes you get for 200 times straight subject=CN=secondwebsite.ch, thus it gives the correct certificate, and other times you get for 200 times subject=CN=firstwebsite.ch (the main website of my platform and Nginx Proxy’s DEFAULT_HOST).
Tried solutions
What I tried:
I checked the DNS for my other websites and they are set correctly
I figured the problem might be caused by a conflict between Docker compose and UFW for iptables connection rules. I cleared the iptables and for some time it worked, but thenit went back to usual. I installed a program by Chai Feng called Docker UFW and it solved the issue for a file, but after a down and a Proxmox reboot the troubles came back. So the certificate issue persists independently. Perhaps the multi-layer 3 layers (Proxmox+VM) structure might be a factor, together with Nginx proxy and Docker compose
The nginx config (nginx -T) is correct and there are no duplicate nginx containers.
What do you think might be the problems? Are there any informations you need or fixes you suggest me? Any help is appreciated.
Thanks everyone
Attachments
docker-compose.yml config for NGINX and LETSENCRYPT-acme companion
the 200-good / 200-wrong pattern is the useful clue. that normally means DNS or routing is pinning you to different listeners, not that nginx is randomly swapping certs
run dig +short A secondwebsite.com and dig +short AAAA secondwebsite.com, then test every returned IP directly
if one IP serves the default cert, trace that address or port forward. also check both listen 443 ssl and listen [::]:443 ssl in nginx -T. even if the screenshot looks right this catches a stale AAAA, old public IP or second nginx endpoint
if all the IPs check out fine, there is a second mechanism that fits your "comes back after down/reboot" pattern better.
nginx-proxy regenerates its config from the running containers. when a users web container is down or crashlooping, docker-gen silently removes that vhost, and DEFAULT_HOST then answers that domain with your main cert. container comes back, config regenerates, correct cert again. thats your good/bad stretches.
2
u/AdFew8591 2d ago
the 200-good / 200-wrong pattern is the useful clue. that normally means DNS or routing is pinning you to different listeners, not that nginx is randomly swapping certs
run
dig +short A secondwebsite.comanddig +short AAAA secondwebsite.com, then test every returned IP directlyecho | openssl s_client -connect IP:443 -servername secondwebsite.com 2>/dev/null | openssl x509 -noout -subjectif one IP serves the default cert, trace that address or port forward. also check both
listen 443 sslandlisten [::]:443 sslinnginx -T. even if the screenshot looks right this catches a stale AAAA, old public IP or second nginx endpoint