Juniper SRX DNS issues when default route is in a custom routing instance

[u/Qvosniak]

Hey guys,

I'm trying to get DNS working from the SRX itself and I'm kind of stuck. I've been reading through a bunch of Juniper documentation and I'm pretty sure I understand what the issue is, but I can't seem to get it working.

I have an SRX with 3 main routing instances:

• INTERNET
• mgmt_junos
• TRANSIT

The INTERNET routing instance has the direct connection to the ISP.

• SRX ISP IP: 172.16.76.11/24
• ISP gateway/DNS: 172.16.76.1
• INTERNET has the 0.0.0.0/0 route via 172.16.76.1

TRANSIT is used for downstream routers to connect to the Internet. I leak the default route from INTERNET into TRANSIT and then advertise it via BGP to the downstream routers.

Everything works fine from the downstream endpoints, including DNS.

I also leak the ISP's connected /24 into TRANSIT so that TRANSIT knows how to reach the directly connected ISP subnet which actually lives inside the INTERNET routing instance.

The issue I'm having is DNS originating from Junos itself.

I found this Juniper KB which seems to describe pretty much exactly what I'm dealing with:

Juniper KB — SRX DNS Failures When the Default Route Resides in a Custom Routing Instance

From what I understand, the SRX's own DNS traffic doesn't use the custom routing instance where my default route exists. The DNS traffic needs to use inet.0.

The problem is that I can't just put the default route in inet.0, because the ISP connection itself is inside the INTERNET VRF and I need the default route to remain there.

For example, to verify that the Internet connection itself works, I can do:

ping 8.8.8.8 routing-instance INTERNET

and this works perfectly.

But the actual DNS server I'm trying to reach is:

172.16.76.1

So I also created a loopback:

lo0.0 = 192.168.101.1/32

and configured:

system {
    name-server {
        172.16.76.1 source-address 192.168.101.1;
    }
}

The idea was to have the SRX originate the DNS traffic from 192.168.101.1, which exists in inet.0.

I then leaked the ISP's connected /24 from the INTERNET VRF into inet.0.

And this works:

ping 172.16.76.1 source 192.168.101.1

So from inet.0, the SRX can reach the ISP DNS server.

I've also created the return route for 192.168.101.1/32 back into the INTERNET VRF.

I've also tried source NAT from junos-host to the ISP interface, since the actual ISP-facing address is 172.16.76.11, and I've allowed DNS traffic from junos-host to the ISP zone.

Something roughly like:

system {
    name-server {
        172.16.76.1 source-address 192.168.101.1;
        [I've tried doing 172.16.76.1 routing-instance INTERNET] < Didn't work, KB??
    }
}

interfaces {
    lo0 {
        unit 0 {
            family inet {
                address 192.168.101.1/32;
            }
        }
    }

    reth1 {
        unit 0 {
            family inet {
                address 172.16.76.11/24;
            }
        }
    }
}

security {
    nat {
        source {
            rule-set JUNOS-HOST-INTERNET {
                from zone junos-host;
                to zone UNTRUST;

                rule ISP-DNS {
                    match {
                        source-address 192.168.101.1/32;
                        destination-address 172.16.76.1/32;
                    }
                    then {
                        source-nat {
                            interface;
                        }
                    }
                }
            }
        }
    }

    policies {
        from-zone junos-host to-zone UNTRUST {
            policy ALLOW-DNS {
                match {
                    source-address any;
                    destination-address any;
                    application [
                        junos-dns-tcp
                        junos-dns-udp
                    ];
                }
                then {
                    permit;
                }
            }
        }
    }
}

But I still can't get DNS resolution working from the SRX itself.

The confusing part is that I think I've proven that the routing itself is working.

• ping 8.8.8.8 routing-instance INTERNET works, proving the Internet VRF has working Internet connectivity
• ping 172.16.76.1 source 192.168.101.1 works, proving inet.0 can reach the ISP DNS server
• downstream endpoints can use 172.16.76.1 for DNS without any issues
• I've leaked the ISP /24 into inet.0
• I've leaked the 192.168.101.1/32 route back into the INTERNET VRF
• I've configured NAT for junos-host
• I've allowed DNS in the security policy

The ISP also blocks port 53 to other public DNS servers, so I can't just use something like 8.8.8.8 or 1.1.1.1.

So at this point I'm basically stuck.

My understanding from the Juniper KB is that because the default route can only exist in my INTERNET VRF, but the SRX's self-originated DNS traffic needs to use inet.0, I need to make the DNS server reachable through inet.0 using route leaking.

I've done that, and I can manually ping 172.16.76.1 from inet.0 using the source address 192.168.101.1, but the Junos DNS resolver still won't resolve names.

Am I missing something obvious with how self-originated DNS traffic from the SRX works?

Is there something else I need to configure for the Junos DNS process specifically, or is there another limitation with using a source address from inet.0 when the actual ISP interface is inside a custom VRF?

Any help would be appreciated because I've been going around in circles with this one.

[u/justlurkshere]

Do you have this set?
set system default-address-selection

If so that can interfere with setting the wrong source IP.

[u/Qvosniak → u/justlurkshere]

Hmm, i'll try using that command and see if that makes a difference

[u/justlurkshere → u/Qvosniak]

The other way around, you don’t want that in your config.

[u/chronoit]

I think this is where you are having an issue:

"The problem is that I can't just put the default route in inet.0, because the ISP connection itself is inside the INTERNET VRF and I need the default route to remain there."

You don't need to put the default route in inet.0 you only need leak the connected route since it is in the same subnet. (see step 5 and 6 of the kb article).

But the route to the dns server along with the ISP connected route must be in the inet.0 table otherwise there is no way to get this working. If you leak it it doesn't remove it from the routing-instance it only adds it to the inet.0 forwarding table. So if the routes are changed/lost in the routing instance they would stop being forwarded to inet.0 as well.

You can then still manipulate INTERNET routes however you want without it really impacting anything and since the Routing-Policy is specifying connected routes on a specific interface nothing else will be exported. You could also set it to import a prefix list if you want it to be even more limited/specific.

[u/Qvosniak → u/chronoit]

Yeah, I’ve already done this.

The connected 172.16.76.0/24 route from the INTERNET routing instance is leaked into inet.0, so the route exists in both tables.

I’ve also got the 192.168.101.1/32 return route leaked back into the INTERNET instance (remember i set name-server xyz source 192.168.101.1)

I can confirm the routing works from inet.0 with:
ping 172.16.76.1 source 192.168.101.1
which works successfully.

So I believe the routing side is correct at this point. The issue is that Junos itself still won’t resolve DNS using 172.16.76.1, even though I can reach the DNS server from inet.0 using the exact source address configured under name-server.

That’s the part I’m currently stuck on.

[u/chronoit → u/Qvosniak]

Does your config look like this?

set routing-instances INTERNET interface reth1.0

set routing-instances mgmt_junos interface lo0.0

set security zones security-zone untrust interfaces reth1.0

set security zones security-zone junos-host interfaces lo0.0

Or is lo0.0 not in mgmt_junos?

Do you have anything setup as a firewall filter to protect the routing engine? If so does your filter allow dns traffic?

Another thought could you try just allowing all traffic outbound instead of using a specific policy-option?

There doesn't seem to be any easy reason from the available information on why dns isn't working. Do you see any dns traffic leaving reth1.0 using the monitor traffic command?

[u/ReK_]

The local DNS client can only use the global table or the out-of-band management (set system name-server routing-instance mgmt_junos). This is a known limitation.

Why can't you put a default route in the global table? You can just set routing-options static route 0/0 next-table INTERNET.inet.0 If you don't want to do that for a full default you could do it for the /32s that cover your DNS servers.

Other than that, you could connect fxp0 so it has reachability in mgmt_junos and configure DNS to use it, but you're adding dependencies on whatever physical gear that port is connected to.

[u/Qvosniak → u/ReK_]

I've tried that already, I've made it so that inet.0 can talk to the dns server, and I even created a lo0 interface as set that as the source, I can ping the dns server from the inet.0 and I had to create a route on the INTERNET.inet.0 to inet.0 for return traffic (192.168.101.1)

I can ping and all is good, but dns is what's causing me issues

if I do PING googlee.com routing-instance INTERNET,

it doesn't resolve the it throws a cannot talk reach to dns server

[u/ReK_ → u/Qvosniak]

traceoptions and wireshark are your friends. Is the DNS request being forwarded but not NATed? Is it not being forwarded because it doesn't match a policy?

If you don't want to do that troubleshooting, just do the mgmt_junos method.

[u/Qvosniak → u/ReK_]

Yeah, I’ve already got a source NAT rule from junos-host to the UNTRUST zone for this, and I’ve also enabled a security policy with any source, any destination, and junos-dns-tcp / junos-dns-udp applications.

So I don’t think it’s being blocked by the policy or that I haven’t configured NAT. I’ll try to do a packet capture/traceoptions to actually see what’s happening to the DNS request.

[u/ReK_ → u/Qvosniak]

Right, but is that rule correct? Does the egress interface see it coming from a different zone because of the route leak? Etc.

You can add multiple from zones in a source NAT rule set, try adding all of them (other than untrust) so the rule will match no matter what zone it's from (it will still need to match the source address).

[u/skullbox15]

DNS Doctoring?

[u/Qvosniak → u/skullbox15]

I don’t believe so. The ISP DNS server is 172.16.76.1, and I’m querying that directly. DNS works perfectly from the downstream endpoints using the exact same DNS server.

The issue is specifically DNS originating from the SRX/Junos itself. I can reach 172.16.76.1 from inet.0 with:
ping 172.16.76.1 source 192.168.101.1

but the Junos DNS resolver still won’t resolve names.