Hairpin NAT issue. #437

Closed
opened 2025-11-13 12:00:28 -06:00 by GiteaMirror · 8 comments
Owner

Originally created by @lthieman on GitHub (Jun 15, 2025).

Context: I have a domain name registered with Cloudflare and have a wildcard A record pointed to the public IP of a UCG Max router and ports 80, 443 and 51820 forwarded to my reverse proxy which is on a 192.168.69.xxx subnet while the rest of the house is on the 192.168.1.xxx subnet. The proxy then points to my services (audiobookshelf, etc) which are also on the 69 subnet.

Issue: This all worked perfectly without issue when I was using Nginx Proxy Manager for my reverse proxy. But I wanted to switch to Pangolin to make use of its pin code auth (I don’t need the tunneling with this setup) and everything works perfectly from the internet side but no device on the LAN side of the router can get to Pangolin or any of the services behind it. This would seem like a router issue, except it worked with Nginx Proxy Manager and I haven't been able to solve the issue with any settings in the router.

Note: I'm not using any tunneling with this setup, just a typical local reverse proxy config.

I hope this makes sense to you lovely people and you have some idea why it works with one reverse proxy and not the other. Thanks!

Originally created by @lthieman on GitHub (Jun 15, 2025). Context: I have a domain name registered with Cloudflare and have a wildcard A record pointed to the public IP of a UCG Max router and ports 80, 443 and 51820 forwarded to my reverse proxy which is on a 192.168.69.xxx subnet while the rest of the house is on the 192.168.1.xxx subnet. The proxy then points to my services (audiobookshelf, etc) which are also on the 69 subnet. Issue: This all worked perfectly without issue when I was using Nginx Proxy Manager for my reverse proxy. But I wanted to switch to Pangolin to make use of its pin code auth (I don’t need the tunneling with this setup) and everything works perfectly from the internet side but no device on the LAN side of the router can get to Pangolin or any of the services behind it. This would seem like a router issue, except it worked with Nginx Proxy Manager and I haven't been able to solve the issue with any settings in the router. Note: I'm not using any tunneling with this setup, just a typical local reverse proxy config. I hope this makes sense to you lovely people and you have some idea why it works with one reverse proxy and not the other. Thanks!
Author
Owner

@ex-aequo-et-bono commented on GitHub (Jun 15, 2025):

This is likely a DNS issue. Do you use a local DNS resolver (pihole/adguard/technitium)? If so, you'll need to create A/CNAME records in your local resolver to point to the Pangolin instance.

@ex-aequo-et-bono commented on GitHub (Jun 15, 2025): This is likely a DNS issue. Do you use a local DNS resolver (pihole/adguard/technitium)? If so, you'll need to create A/CNAME records in your local resolver to point to the Pangolin instance.
Author
Owner

@lthieman commented on GitHub (Jun 15, 2025):

No Pihole or the like. Just standard Unifi configuration. I didn't have to do anything special with Nginx Proxy Manager but even setting up specific DNS records for internal domains in Unifi does not work with Pangolin.

@lthieman commented on GitHub (Jun 15, 2025): No Pihole or the like. Just standard Unifi configuration. I didn't have to do anything special with Nginx Proxy Manager but even setting up specific DNS records for internal domains in Unifi does not work with Pangolin.
Author
Owner

@oschwartz10612 commented on GitHub (Jun 15, 2025):

What is the error when you try to reach pangolin or a resource. Does it just time out in the browser? Could you do a tcpdump on the pangolin server and make some requests and see if any packets make it?

@oschwartz10612 commented on GitHub (Jun 15, 2025): What is the error when you try to reach pangolin or a resource. Does it just time out in the browser? Could you do a tcpdump on the pangolin server and make some requests and see if any packets make it?
Author
Owner

@lthieman commented on GitHub (Jun 15, 2025):

In the browser it seems to just time out. The Audiobookshelf app says it cannot ping the server. Unfortunately doing a TCPdump is a little over my head, but if you have a page that explains how I can try.

@lthieman commented on GitHub (Jun 15, 2025): In the browser it seems to just time out. The Audiobookshelf app says it cannot ping the server. Unfortunately doing a TCPdump is a little over my head, but if you have a page that explains how I can try.
Author
Owner

@lthieman commented on GitHub (Jun 15, 2025):

The physical hosts are at my parent's house so this is a little challenging to diagnose remotely since it works fine from my side being outside the house. But I managed to set up a VM inside their network to test with. I installed tcpdump on my Pangolin instance and then tried to reach my Audiobookshelf server from the VM. I can see in the tcpdump output where that machine hit Pangolin, but I'm not knowledgeable enough to know what I'm looking for. Are there any particular things I should be looking for about the packets that would tell you what's going on?

@lthieman commented on GitHub (Jun 15, 2025): The physical hosts are at my parent's house so this is a little challenging to diagnose remotely since it works fine from my side being outside the house. But I managed to set up a VM inside their network to test with. I installed tcpdump on my Pangolin instance and then tried to reach my Audiobookshelf server from the VM. I can see in the tcpdump output where that machine hit Pangolin, but I'm not knowledgeable enough to know what I'm looking for. Are there any particular things I should be looking for about the packets that would tell you what's going on?
Author
Owner

@oschwartz10612 commented on GitHub (Jun 15, 2025):

Could you post a snippet of the dump? If you are getting traffic thats a good sign. We would want to know the source ip of the traffic.

One thing to check: do you have cloduflare proxy on?

@oschwartz10612 commented on GitHub (Jun 15, 2025): Could you post a snippet of the dump? If you are getting traffic thats a good sign. We would want to know the source ip of the traffic. One thing to check: do you have cloduflare proxy on?
Author
Owner

@lthieman commented on GitHub (Jun 15, 2025):

I had Cloudflare proxy on when using Nginx Proxy Manager, but Pangolin did not work at all with it on so that is off now.

5:d6:e0:59:52.800b, length 36
15:43:50.743223 IP6 fe80::be24:11ff:fecf:6b4b.58776 > ff12::8384.21027: UDP, length 259
15:43:50.743230 IP bendebian.localdomain.45216 > 192.168.1.255.21027: UDP, length 259
15:43:51.580428 IP pangolin.home.lee.http > bendebian.localdomain.37508: Flags [S.], seq 4023027006, ack 3812388584, win 65160, options [mss 1240,sackOK,TS val 3316576746 ecr 1375474920,nop,wscale 7], length 0
15:43:51.580432 IP pangolin.home.lee.http > bendebian.localdomain.37494: Flags [S.], seq 3735180830, ack 2069717061, win 65160, options [mss 1240,sackOK,TS val 3316576746 ecr 1375474920,nop,wscale 7], length 0
15:43:51.836470 IP pangolin.home.lee.http > bendebian.localdomain.37518: Flags [S.], seq 2959763528, ack 735311790, win 65160, options [mss 1240,sackOK,TS val 3316577002 ecr 1375475170,nop,wscale 7], length 0
15:43:52.521659 STP 802.1w, Rapid STP, Flags [Learn, Forward, Agreement], bridge-id 8000.9c:05:d6:e0:59:52.800b, length 36
15:43:54.521696 STP 802.1w, Rapid STP, Flags [Learn, Forward, Agreement], bridge-id 8000.9c:05:d6:e0:59:52.800b, length 36
15:43:55.773025 IP bendebian.localdomain.37494 > pangolin.home.lee.http: Flags [S], seq 2069717060, win 64240, options [mss 1460,sackOK,TS val 1375490217 ecr 0,nop,wscale 7], length 0
15:43:55.773043 IP bendebian.localdomain.37508 > pangolin.home.lee.http: Flags [S], seq 3812388583, win 64240, options [mss 1460,sackOK,TS val 1375490217 ecr 0,nop,wscale 7], length 0
15:43:55.773070 IP pangolin.home.lee.http > bendebian.localdomain.37494: Flags [S.], seq 3735180830, ack 2069717061, win 65160, options [mss 1240,sackOK,TS val 3316580938 ecr 1375474920,nop,wscale 7], length 0
15:43:55.773074 IP pangolin.home.lee.http > bendebian.localdomain.37508: Flags [S.], seq 4023027006, ack 3812388584, win 65160, options [mss 1240,sackOK,TS val 3316580938 ecr 1375474920,nop,wscale 7], length 0
15:43:56.028930 IP bendebian.localdomain.37518 > pangolin.home.lee.http: Flags [S], seq 735311789, win 64240, options [mss 1460,sackOK,TS val 1375490473 ecr 0,nop,wscale 7], length 0
15:43:56.028969 IP pangolin.home.lee.http > bendebian.localdomain.37518: Flags [S.], seq 2959763528, ack 735311790, win 65160, options [mss 1240,sackOK,TS val 3316581194 ecr 1375475170,nop,wscale 7], length 0
15:43:56.038091 IP unifi.localdomain.2647 > 224.0.37.42.2647: UDP, length 10
15:43:56.038099 IP 192.168.30.1.2647 > 224.0.37.42.2647: UDP, length 10
15:43:56.038100 IP 192.168.58.1.2647 > 224.0.37.42.2647: UDP, length 10
15:43:56.038126 IP 192.168.69.1.2647 > 224.0.37.42.2647: UDP, length 10<

It doesn't seem to be using IP's at all for this exchange. So "home.lee" is just what I've always used when installing operating systems and it asks for a made up hostname, it's not my public domain name. "bendebian" is the name of the VM I used to attempt accessing Audiobookshelf behind Pangolin. Now I would expect the debian VM to appear as "bendebian.home.lee" but the Unifi router is using "localdomain" so maybe the router switched that as the packet went through? I don't know how this would affect our phones though, which I've never set any kind of host domain for. Could it be that the host OS that Pangolin is installed on needs to be "pangolin.localdomain" to match what the router is expecting?

@lthieman commented on GitHub (Jun 15, 2025): I had Cloudflare proxy on when using Nginx Proxy Manager, but Pangolin did not work at all with it on so that is off now. > 5:d6:e0:59:52.800b, length 36 15:43:50.743223 IP6 fe80::be24:11ff:fecf:6b4b.58776 > ff12::8384.21027: UDP, length 259 15:43:50.743230 IP bendebian.localdomain.45216 > 192.168.1.255.21027: UDP, length 259 15:43:51.580428 IP pangolin.home.lee.http > bendebian.localdomain.37508: Flags [S.], seq 4023027006, ack 3812388584, win 65160, options [mss 1240,sackOK,TS val 3316576746 ecr 1375474920,nop,wscale 7], length 0 15:43:51.580432 IP pangolin.home.lee.http > bendebian.localdomain.37494: Flags [S.], seq 3735180830, ack 2069717061, win 65160, options [mss 1240,sackOK,TS val 3316576746 ecr 1375474920,nop,wscale 7], length 0 15:43:51.836470 IP pangolin.home.lee.http > bendebian.localdomain.37518: Flags [S.], seq 2959763528, ack 735311790, win 65160, options [mss 1240,sackOK,TS val 3316577002 ecr 1375475170,nop,wscale 7], length 0 15:43:52.521659 STP 802.1w, Rapid STP, Flags [Learn, Forward, Agreement], bridge-id 8000.9c:05:d6:e0:59:52.800b, length 36 15:43:54.521696 STP 802.1w, Rapid STP, Flags [Learn, Forward, Agreement], bridge-id 8000.9c:05:d6:e0:59:52.800b, length 36 15:43:55.773025 IP bendebian.localdomain.37494 > pangolin.home.lee.http: Flags [S], seq 2069717060, win 64240, options [mss 1460,sackOK,TS val 1375490217 ecr 0,nop,wscale 7], length 0 15:43:55.773043 IP bendebian.localdomain.37508 > pangolin.home.lee.http: Flags [S], seq 3812388583, win 64240, options [mss 1460,sackOK,TS val 1375490217 ecr 0,nop,wscale 7], length 0 15:43:55.773070 IP pangolin.home.lee.http > bendebian.localdomain.37494: Flags [S.], seq 3735180830, ack 2069717061, win 65160, options [mss 1240,sackOK,TS val 3316580938 ecr 1375474920,nop,wscale 7], length 0 15:43:55.773074 IP pangolin.home.lee.http > bendebian.localdomain.37508: Flags [S.], seq 4023027006, ack 3812388584, win 65160, options [mss 1240,sackOK,TS val 3316580938 ecr 1375474920,nop,wscale 7], length 0 15:43:56.028930 IP bendebian.localdomain.37518 > pangolin.home.lee.http: Flags [S], seq 735311789, win 64240, options [mss 1460,sackOK,TS val 1375490473 ecr 0,nop,wscale 7], length 0 15:43:56.028969 IP pangolin.home.lee.http > bendebian.localdomain.37518: Flags [S.], seq 2959763528, ack 735311790, win 65160, options [mss 1240,sackOK,TS val 3316581194 ecr 1375475170,nop,wscale 7], length 0 15:43:56.038091 IP unifi.localdomain.2647 > 224.0.37.42.2647: UDP, length 10 15:43:56.038099 IP 192.168.30.1.2647 > 224.0.37.42.2647: UDP, length 10 15:43:56.038100 IP 192.168.58.1.2647 > 224.0.37.42.2647: UDP, length 10 15:43:56.038126 IP 192.168.69.1.2647 > 224.0.37.42.2647: UDP, length 10< It doesn't seem to be using IP's at all for this exchange. So "home.lee" is just what I've always used when installing operating systems and it asks for a made up hostname, it's not my public domain name. "bendebian" is the name of the VM I used to attempt accessing Audiobookshelf behind Pangolin. Now I would expect the debian VM to appear as "bendebian.home.lee" but the Unifi router is using "localdomain" so maybe the router switched that as the packet went through? I don't know how this would affect our phones though, which I've never set any kind of host domain for. Could it be that the host OS that Pangolin is installed on needs to be "pangolin.localdomain" to match what the router is expecting?
Author
Owner

@lthieman commented on GitHub (Jun 16, 2025):

I believe I got this figured out. Something about the Unifi firewall rules got screwed up when they changed to the new zone based system and it wasn't allowing any return traffic from the DMZ. Not sure what Nginx was doing to get around it, but I was able to get it working with Pangolin by fixing the firewall rules. Thank you for your time!

@lthieman commented on GitHub (Jun 16, 2025): I believe I got this figured out. Something about the Unifi firewall rules got screwed up when they changed to the new zone based system and it wasn't allowing any return traffic from the DMZ. Not sure what Nginx was doing to get around it, but I was able to get it working with Pangolin by fixing the firewall rules. Thank you for your time!
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/pangolin#437