[GH-ISSUE #1442] Rules are not working #3855

Closed
opened 2026-04-20 08:02:19 -05:00 by GiteaMirror · 23 comments
Owner

Originally created by @TehMaat on GitHub (Sep 10, 2025).
Original GitHub issue: https://github.com/fosrl/pangolin/issues/1442

As title says, allow rules are not working and I lost a lot of data because of that.

Originally created by @TehMaat on GitHub (Sep 10, 2025). Original GitHub issue: https://github.com/fosrl/pangolin/issues/1442 As title says, allow rules are not working and I lost a lot of data because of that.
GiteaMirror added the stale label 2026-04-20 08:02:19 -05:00
Author
Owner

@TehMaat commented on GitHub (Sep 10, 2025):

I'm not behind Cloudflare or anything.
Pangolin is in a VPS on netcup.

Maybe because of ipv6?

<!-- gh-comment-id:3273762651 --> @TehMaat commented on GitHub (Sep 10, 2025): I'm not behind Cloudflare or anything. Pangolin is in a VPS on netcup. Maybe because of ipv6?
Author
Owner

@oschwartz10612 commented on GitHub (Sep 10, 2025):

How did you loose data! Sorry about that!

In what way are the rules not working? Are things not getting blocked? Could you enable debug logs in https://docs.digpangolin.com/self-host/advanced/config-file#param-log-level and reproduce? We will see from there if its an ipv6 issue.

<!-- gh-comment-id:3276593721 --> @oschwartz10612 commented on GitHub (Sep 10, 2025): How did you loose data! Sorry about that! In what way are the rules not working? Are things not getting blocked? Could you enable debug logs in https://docs.digpangolin.com/self-host/advanced/config-file#param-log-level and reproduce? We will see from there if its an ipv6 issue.
Author
Owner

@TehMaat commented on GitHub (Sep 10, 2025):

Rules are not working as it always prompt me to auth.

So I don't think it's ipv6 but I think it's a NAT problem. Here's the logs (sorry for the screen but I'm about to sleep and I'm not at my pc)
image

<!-- gh-comment-id:3276642756 --> @TehMaat commented on GitHub (Sep 10, 2025): Rules are not working as it always prompt me to auth. So I don't think it's ipv6 but I think it's a NAT problem. Here's the logs (sorry for the screen but I'm about to sleep and I'm not at my pc) ![image](https://github.com/user-attachments/assets/d189789b-98c5-418f-9b7f-ddc2aa8594cc)
Author
Owner

@TehMaat commented on GitHub (Sep 24, 2025):

Hello?
Can't anybody help me ?
It's still not working.

<!-- gh-comment-id:3328177288 --> @TehMaat commented on GitHub (Sep 24, 2025): Hello? Can't anybody help me ? It's still not working.
Author
Owner

@miloschwartz commented on GitHub (Sep 27, 2025):

Are your allow rules for IP matching? If so, the logs in your screenshot show the request IP as 172.18.0.1. Is this expected?

<!-- gh-comment-id:3342101551 --> @miloschwartz commented on GitHub (Sep 27, 2025): Are your allow rules for IP matching? If so, the logs in your screenshot show the request IP as 172.18.0.1. Is this expected?
Author
Owner

@TehMaat commented on GitHub (Sep 28, 2025):

It's not expected, docker I think is natting the traffic and i don't know why. I always created the pangolin's network with the docker compose provided and now it does that.
If you can guide me fine, otherwise I'll just look into it whenever I have time

<!-- gh-comment-id:3342923393 --> @TehMaat commented on GitHub (Sep 28, 2025): It's not expected, docker I think is natting the traffic and i don't know why. I always created the pangolin's network with the docker compose provided and now it does that. If you can guide me fine, otherwise I'll just look into it whenever I have time
Author
Owner

@TehMaat commented on GitHub (Oct 17, 2025):

I'm reopening with hopes someone will help me.

<!-- gh-comment-id:3416399979 --> @TehMaat commented on GitHub (Oct 17, 2025): I'm reopening with hopes someone will help me.
Author
Owner

@oschwartz10612 commented on GitHub (Oct 17, 2025):

Yeah this would occur if docker is natting the source traffic before it reaches Traefik itself.

Could you try to disable v6:

networks:
  default:
    driver: bridge
    name: pangolin
    enable_ipv6: false

Also I know @AstralDestiny might have some thoughts on this one.

<!-- gh-comment-id:3416430407 --> @oschwartz10612 commented on GitHub (Oct 17, 2025): Yeah this would occur if docker is natting the source traffic before it reaches Traefik itself. Could you try to disable v6: ``` networks: default: driver: bridge name: pangolin enable_ipv6: false ``` Also I know @AstralDestiny might have some thoughts on this one.
Author
Owner

@AstralDestiny commented on GitHub (Oct 17, 2025):

I would recommend having ipv6 enabled if you're docker host on the vps is ipv6 compatible else you'll get incoming ip's formatted as random v4 ranges.. Can you put traefik into trace and down and up and try to connect and see what it says? Are you using docker engine or docker desktop? and does your config deviate at all? Has the host been tampered to "fix" anything docker too?

<!-- gh-comment-id:3417217420 --> @AstralDestiny commented on GitHub (Oct 17, 2025): I would recommend having ipv6 enabled if you're docker host on the vps is ipv6 compatible else you'll get incoming ip's formatted as random v4 ranges.. Can you put traefik into trace and down and up and try to connect and see what it says? Are you using docker engine or docker desktop? and does your config deviate at all? Has the host been tampered to "fix" anything docker too?
Author
Owner

@TehMaat commented on GitHub (Oct 18, 2025):

@AstralDestiny
I'm on ubuntu on arm, every config/compose is as per docs. And what do you mean with has the host been tampered?
Right now I even flushed nftables and I still see the Natted IP.

2025-10-18T09:53:35Z DBG github.com/traefik/traefik/v3/pkg/server/service/loadbalancer/wrr/wrr.go:173 > Service selected by WRR: https://100.89.128.4:53620
2025-10-18T09:53:35Z DBG fmt/print.go:305 > Badger: Valid session module=github.com/fosrl/badger plugin=plugin-badger runtime=
2025-10-18T09:53:35Z DBG github.com/traefik/traefik/v3/pkg/server/service/loadbalancer/wrr/wrr.go:173 > Service selected by WRR: https://100.89.128.4:53620

Here's the log from Traefik.

I'm inclined to not disable ipv6 as I'm paying for it to be honest

<!-- gh-comment-id:3418147972 --> @TehMaat commented on GitHub (Oct 18, 2025): @AstralDestiny I'm on ubuntu on arm, every config/compose is as per docs. And what do you mean with `has the host been tampered`? Right now I even flushed nftables and I still see the Natted IP. ``` 2025-10-18T09:53:35Z DBG github.com/traefik/traefik/v3/pkg/server/service/loadbalancer/wrr/wrr.go:173 > Service selected by WRR: https://100.89.128.4:53620 2025-10-18T09:53:35Z DBG fmt/print.go:305 > Badger: Valid session module=github.com/fosrl/badger plugin=plugin-badger runtime= 2025-10-18T09:53:35Z DBG github.com/traefik/traefik/v3/pkg/server/service/loadbalancer/wrr/wrr.go:173 > Service selected by WRR: https://100.89.128.4:53620 ``` Here's the log from Traefik. I'm inclined to not disable ipv6 as I'm paying for it to be honest
Author
Owner

@AstralDestiny commented on GitHub (Oct 19, 2025):

Sorry where are you seeing it on your backend host behind all this? Are you trusting newt to get the X-Forwarded-For headers?

<!-- gh-comment-id:3419745788 --> @AstralDestiny commented on GitHub (Oct 19, 2025): Sorry where are you seeing it on your backend host behind all this? Are you trusting newt to get the X-Forwarded-For headers?
Author
Owner

@TehMaat commented on GitHub (Oct 19, 2025):

Can you rephrase it? I'm not understanding what you're saying sorry.

<!-- gh-comment-id:3419907469 --> @TehMaat commented on GitHub (Oct 19, 2025): Can you rephrase it? I'm not understanding what you're saying sorry.
Author
Owner

@AstralDestiny commented on GitHub (Oct 19, 2025):

Where are you seeing the docker ip on a backend service? or just looking for X-Real-IP headers..? Or the X-Forwarded-For?

<!-- gh-comment-id:3420046167 --> @AstralDestiny commented on GitHub (Oct 19, 2025): Where are you seeing the docker ip on a backend service? or just looking for X-Real-IP headers..? Or the X-Forwarded-For?
Author
Owner

@TehMaat commented on GitHub (Oct 20, 2025):

I'm seeing it everywhere(?)

Image Image
<!-- gh-comment-id:3420809652 --> @TehMaat commented on GitHub (Oct 20, 2025): I'm seeing it everywhere(?) <img width="1194" height="402" alt="Image" src="https://github.com/user-attachments/assets/ec4893c3-d9a9-495e-a946-2931b20dd8a4" /> <img width="1194" height="402" alt="Image" src="https://github.com/user-attachments/assets/11e5c75c-4480-4c55-94f2-65fe6452db1f" />
Author
Owner

@TehMaat commented on GitHub (Nov 2, 2025):

Are there any news for me?

<!-- gh-comment-id:3478088950 --> @TehMaat commented on GitHub (Nov 2, 2025): Are there any news for me?
Author
Owner

@AstralDestiny commented on GitHub (Nov 7, 2025):

Sorry for the delay can you show the in pangolin rules area if you have touched them or not?

<!-- gh-comment-id:3501159597 --> @AstralDestiny commented on GitHub (Nov 7, 2025): Sorry for the delay can you show the in pangolin rules area if you have touched them or not?
Author
Owner

@TehMaat commented on GitHub (Nov 8, 2025):

Right now are disabled for almost everything as i can't have something that broke my home lab.
What do you need to see?

<!-- gh-comment-id:3506869261 --> @TehMaat commented on GitHub (Nov 8, 2025): Right now are disabled for almost everything as i can't have something that broke my home lab. What do you need to see?
Author
Owner

@AstralDestiny commented on GitHub (Nov 9, 2025):

Well you said the rules aren't working but unless we can replicate the same issue it could be numerous things. So screenshot of the rules and if the domain is "protected" or not

<!-- gh-comment-id:3507844428 --> @AstralDestiny commented on GitHub (Nov 9, 2025): Well you said the rules aren't working but unless we can replicate the same issue it could be numerous things. So screenshot of the rules and if the domain is "protected" or not
Author
Owner

@TehMaat commented on GitHub (Nov 9, 2025):

I'm gonna be a bit rude because I'm kinda fed up with this issue. Sorry (but not sorry) for that
If you read all the comments you can find all your answers.
Every request will be natted with the gateway of the pangolin docker network. And if you can, you can see it in every screenshot I shared in this almost prehistoric thread.
In the comment made 20 days ago, you can see a keycloak with request that has arrived with that IP. And that IP it's in every header created by human kind. Keycloack is behind pangolin.

The network has been created with pangolin docker compose found in the docs, and NO CHANGES have been made on iptables/nftables or else in the host configuration.

This is happening since I migrated pangolin from an ipv4 only VPS to an ipv6 ready one.

You are welcome to tell me "we don't know to help you" or "this is a docker problem sorry" but please, please don't keep asking me things I've already explained.

<!-- gh-comment-id:3507986043 --> @TehMaat commented on GitHub (Nov 9, 2025): I'm gonna be a bit rude because I'm kinda fed up with this issue. Sorry (but not sorry) for that If you read all the comments you can find all your answers. Every request will be natted with the gateway of the pangolin docker network. And if you can, you can see it in every screenshot I shared in this almost prehistoric thread. In the comment made 20 days ago, you can see a keycloak with request that has arrived with that IP. And that IP it's in every header created by human kind. Keycloack is behind pangolin. The network has been created with pangolin docker compose found in the docs, and NO CHANGES have been made on iptables/nftables or else in the host configuration. This is happening since I migrated pangolin from an ipv4 only VPS to an ipv6 ready one. You are welcome to tell me "we don't know to help you" or "this is a docker problem sorry" but please, please don't keep asking me things I've already explained.
Author
Owner

@AstralDestiny commented on GitHub (Nov 12, 2025):

No need to apologize I'm just still not sure, I can't replicate that's why I keep asking,

Even if I get a word wall I'd be pretty content if it explains everything stack wise.

for example, Internet > pangolin(Traefik)(this host is dual stack) > Gerbil > Newt > pingvin
pingvin trusts newt so it gets to see the real ip.

pingvin-share-1  | [Nest] 63  - 11/12/2025, 1:33:07 PM     LOG [AuthService] Failed login attempt for user astral from IP 1052:XXXX:XXXX:XXX::2
pingvin-share-1  | [Nest] 63  - 11/12/2025, 1:33:11 PM     LOG [AuthService] Successful password login for user test@example.com from IP 1052:XXXX:XXXX:XXX::2 (changed the ip but it's showing the proper ip) 

Same route but instead newt > traefik(internal) > whoami

(External traefik)

2025-11-12T08:36:57-05:00 DBG github.com/traefik/traefik/v3/pkg/server/service/loadbalancer/wrr/wrr.go:176 > Service selected by WRR: http://pangolin:3000
2025-11-12T08:36:58-05:00 DBG fmt/print.go:305 > Badger: Valid session module=github.com/fosrl/badger plugin=plugin-badger runtime=
2025-11-12T08:36:58-05:00 DBG github.com/traefik/traefik/v3/pkg/server/service/loadbalancer/wrr/wrr.go:176 > Service selected by WRR: https://100.89.128.4:56397
2025-11-12T08:36:58-05:00 DBG fmt/print.go:305 > Badger: Valid session module=github.com/fosrl/badger plugin=plugin-badger runtime=
2025-11-12T08:36:58-05:00 DBG github.com/traefik/traefik/v3/pkg/server/service/loadbalancer/wrr/wrr.go:176 > Service selected by WRR: https://100.89.128.4:56397

the 100.89.128.4 is the newt client's ip which when it arrives at the newt ip newt then reflects it out to 192.168.202.1 where traefik is waiting. Both hit this same host the top one just went right to pingvin this one is routing through another traefik.

(Internal traefik picks it up,
2025-11-12T08:36:12-05:00 DBG github.com/traefik/traefik/v3/pkg/server/service/loadbalancer/wrr/wrr.go:175 > Service selected by WRR: http://172.23.0.24:80)

X-Forwarded-For: 1052:XXXX:XXXX:XXX::2
X-Forwarded-Host: whoami.example.com
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: 41acc5cbd3c4
X-Real-Ip: 1052:XXXX:XXXX:XXX::2

And within badger,

2025-11-12T14:19:32+00:00 [debug]: Verify session: Badger sent {"sessions":{},"originalRequestURL":"https://test.domain.com/","scheme":"","host":"test.domain.com","path":"/","method":"GET","tls":true,"requestIp":"[XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX]:50612","headers":{"Accept":"*/*","Authorization":"Basic blah","User-Agent":"curl/8.14.1","X-Forwarded-Host":"test.domain.com","X-Forwarded-Port":"443","X-Forwarded-Proto":"https","X-Forwarded-Server":"b09d5ab1ed44","X-Real-Ip":"XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX"}}

This last one was done via, curl -6 -u "user:pass" https://test.domain.com

Anyways I need a lot more info I get your angry and upset but I need to be able to replicate even if it means asking for more info, Give me your entire stack including application locations, If you've already shown that I'm sorry I guess I'm blind in the end, Though one of your routes doesn't have a AAAA record in play don't know about the others. Though for stuff like X-Real-IP don't rely on that it's an era old thing.

Or if you like bug me on the discord, I'm pretty sure I recall talking with you in the past but I could be wrong. Pretty sure I recall a user floating about with star wars naming.

Either way this is likely either a dns level issue or something went funky with the move to the dual stack host.. Again I don't know unless I can replicate the behavior.

Anyways no I'd rather fix this as it is a learning experience to see what to look out for in the future. :)

I'm sorry I was blind to which levels you were referring to originally.

<!-- gh-comment-id:3522286441 --> @AstralDestiny commented on GitHub (Nov 12, 2025): No need to apologize I'm just still not sure, I can't replicate that's why I keep asking, Even if I get a word wall I'd be pretty content if it explains everything stack wise. for example, Internet > pangolin(Traefik)(this host is dual stack) > Gerbil > Newt > pingvin pingvin trusts newt so it gets to see the real ip. ``` pingvin-share-1 | [Nest] 63 - 11/12/2025, 1:33:07 PM LOG [AuthService] Failed login attempt for user astral from IP 1052:XXXX:XXXX:XXX::2 pingvin-share-1 | [Nest] 63 - 11/12/2025, 1:33:11 PM LOG [AuthService] Successful password login for user test@example.com from IP 1052:XXXX:XXXX:XXX::2 (changed the ip but it's showing the proper ip) ``` Same route but instead newt > traefik(internal) > whoami (External traefik) ``` 2025-11-12T08:36:57-05:00 DBG github.com/traefik/traefik/v3/pkg/server/service/loadbalancer/wrr/wrr.go:176 > Service selected by WRR: http://pangolin:3000 2025-11-12T08:36:58-05:00 DBG fmt/print.go:305 > Badger: Valid session module=github.com/fosrl/badger plugin=plugin-badger runtime= 2025-11-12T08:36:58-05:00 DBG github.com/traefik/traefik/v3/pkg/server/service/loadbalancer/wrr/wrr.go:176 > Service selected by WRR: https://100.89.128.4:56397 2025-11-12T08:36:58-05:00 DBG fmt/print.go:305 > Badger: Valid session module=github.com/fosrl/badger plugin=plugin-badger runtime= 2025-11-12T08:36:58-05:00 DBG github.com/traefik/traefik/v3/pkg/server/service/loadbalancer/wrr/wrr.go:176 > Service selected by WRR: https://100.89.128.4:56397 ``` the 100.89.128.4 is the newt client's ip which when it arrives at the newt ip newt then reflects it out to 192.168.202.1 where traefik is waiting. Both hit this same host the top one just went right to pingvin this one is routing through another traefik. (Internal traefik picks it up, `2025-11-12T08:36:12-05:00 DBG github.com/traefik/traefik/v3/pkg/server/service/loadbalancer/wrr/wrr.go:175 > Service selected by WRR: http://172.23.0.24:80)` ``` X-Forwarded-For: 1052:XXXX:XXXX:XXX::2 X-Forwarded-Host: whoami.example.com X-Forwarded-Port: 443 X-Forwarded-Proto: https X-Forwarded-Server: 41acc5cbd3c4 X-Real-Ip: 1052:XXXX:XXXX:XXX::2 ``` And within badger, `2025-11-12T14:19:32+00:00 [debug]: Verify session: Badger sent {"sessions":{},"originalRequestURL":"https://test.domain.com/","scheme":"","host":"test.domain.com","path":"/","method":"GET","tls":true,"requestIp":"[XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX]:50612","headers":{"Accept":"*/*","Authorization":"Basic blah","User-Agent":"curl/8.14.1","X-Forwarded-Host":"test.domain.com","X-Forwarded-Port":"443","X-Forwarded-Proto":"https","X-Forwarded-Server":"b09d5ab1ed44","X-Real-Ip":"XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX"}}` This last one was done via, `curl -6 -u "user:pass" https://test.domain.com` Anyways I need a lot more info I get your angry and upset but I need to be able to replicate even if it means asking for more info, Give me your entire stack including application locations, If you've already shown that I'm sorry I guess I'm blind in the end, Though one of your routes doesn't have a AAAA record in play don't know about the others. Though for stuff like X-Real-IP don't rely on that it's an era old thing. Or if you like bug me on the [discord](https://discord.gg/MZtgvEfNCc), I'm pretty sure I recall talking with you in the past but I could be wrong. Pretty sure I recall a user floating about with star wars naming. Either way this is likely either a dns level issue or something went funky with the move to the dual stack host.. Again I don't know unless I can replicate the behavior. Anyways no I'd rather fix this as it is a learning experience to see what to look out for in the future. :) I'm sorry I was blind to which levels you were referring to originally.
Author
Owner

@TehMaat commented on GitHub (Nov 13, 2025):

Hello Astral,
Thank you for taking the time to explain things a bit more, and I am happy that you want to understand more on what's going on. I'll try to bug you on discord in the next days.

<!-- gh-comment-id:3529702912 --> @TehMaat commented on GitHub (Nov 13, 2025): Hello Astral, Thank you for taking the time to explain things a bit more, and I am happy that you want to understand more on what's going on. I'll try to bug you on discord in the next days.
Author
Owner

@TehMaat commented on GitHub (Nov 27, 2025):

Hello Astral I fixed it myself.
It was a nftables compatibility with docker. I asked Claude another thing (I was having problem with certs and it was a internet reachability problem for pangolin) and it told me to do this

Image

And now I can see the real ip, the certs are renewed and everything works

<!-- gh-comment-id:3586708045 --> @TehMaat commented on GitHub (Nov 27, 2025): Hello Astral I fixed it myself. It was a nftables compatibility with docker. I asked Claude another thing (I was having problem with certs and it was a internet reachability problem for pangolin) and it told me to do this <img width="1718" height="674" alt="Image" src="https://github.com/user-attachments/assets/cb20be24-6bd8-4527-8d81-d8675b1dc413" /> And now I can see the real ip, the certs are renewed and everything works
Author
Owner

@github-actions[bot] commented on GitHub (Dec 12, 2025):

This issue has been automatically marked as stale due to 14 days of inactivity. It will be closed in 14 days if no further activity occurs.

<!-- gh-comment-id:3644377352 --> @github-actions[bot] commented on GitHub (Dec 12, 2025): This issue has been automatically marked as stale due to 14 days of inactivity. It will be closed in 14 days if no further activity occurs.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/pangolin#3855