github-starred/pangolin
2
0
Fork 0
mirror of https://github.com/fosrl/pangolin.git synced 2026-08-27 06:47:02 -05:00
Code Issues 1.6k Packages Projects Releases 79 Wiki Activity

Docker containers losing network connectivity amongst themselves #734

New Issue
Open
opened 2025-11-13 12:09:31 -06:00 by GiteaMirror · 17 comments
GiteaMirror commented 2025-11-13 12:09:31 -06:00
Owner
Copy Link
Copy Source

Originally created by @clanger81 on GitHub (Oct 20, 2025).

Describe the Bug

Sorry, I didn't really know how to title this problem accurately. I've been having this issue intermittently for awhile now and it's becoming pretty frustrating. From the logs of each container, it looks like both gerbil and traefik lose connection to the pangolin container, there doesn't appear to be a specific reason, seems to happen completely randomly. Before running Pangolin, I had a single wireguard tunnel back to my local server VLAN, traefik was configured manually with rules from a file provider. I also had an instance of Crowdsec running alongside. I'd really like to continue using Pangolin as it makes setup of sub-domains significantly easier than doing it all by hand.

Ultimately Gerbil logs will spit out the following at some stage:
INFO: 2025/10/20 10:44:55 Failed to report peer bandwidth: API returned non-OK status: 408 Request Timeout
INFO: 2025/10/20 10:46:52 Failed to report peer bandwidth: failed to send bandwidth data: Post "http://pangolin:3001/api/v1/gerbil/receive-bandwidth": dial tcp: lookup pangolin: i/o timeout
INFO: 2025/10/20 10:49:00 Failed to report peer bandwidth: failed to send bandwidth data: Post "http://pangolin:3001/api/v1/gerbil/receive-bandwidth": dial tcp: lookup pangolin: i/o timeout
INFO: 2025/10/20 10:49:47 Failed to report peer bandwidth: failed to send bandwidth data: Post "http://pangolin:3001/api/v1/gerbil/receive-bandwidth": dial tcp: lookup pangolin on 127.0.0.11:53: read udp 127.0.0.1:56696->127.0.0.11:53: i/o timeout
INFO: 2025/10/20 10:50:47 Failed to report peer bandwidth: failed to send bandwidth data: Post "http://pangolin:3001/api/v1/gerbil/receive-bandwidth": dial tcp: lookup pangolin: i/o timeout
INFO: 2025/10/20 10:52:45 Failed to report peer bandwidth: failed to send bandwidth data: Post "http://pangolin:3001/api/v1/gerbil/receive-bandwidth": dial tcp: lookup pangolin: i/o timeout
INFO: 2025/10/20 10:54:14 Failed to report peer bandwidth: failed to send bandwidth data: Post "http://pangolin:3001/api/v1/gerbil/receive-bandwidth": dial tcp: lookup pangolin: i/o timeout
INFO: 2025/10/20 10:57:52 Failed to report peer bandwidth: failed to send bandwidth data: Post "http://pangolin:3001/api/v1/gerbil/receive-bandwidth": dial tcp: lookup pangolin: i/o timeout
INFO: 2025/10/20 11:00:14 Failed to report peer bandwidth: failed to send bandwidth data: Post "http://pangolin:3001/api/v1/gerbil/receive-bandwidth": dial tcp: lookup pangolin: i/o timeout
INFO: 2025/10/20 11:06:44 Failed to report peer bandwidth: failed to send bandwidth data: Post "http://pangolin:3001/api/v1/gerbil/receive-bandwidth": dial tcp: lookup pangolin: i/o timeout
INFO: 2025/10/20 11:11:17 Failed to report peer bandwidth: failed to send bandwidth data: Post "http://pangolin:3001/api/v1/gerbil/receive-bandwidth": dial tcp: lookup pangolin: i/o timeout
INFO: 2025/10/20 11:20:15 Fetching remote config from http://pangolin:3001/api/v1/gerbil/get-config
ERROR: 2025/10/20 11:20:15 Error fetching remote config http://pangolin:3001/api/v1/gerbil/get-config: Post "http://pangolin:3001/api/v1/gerbil/get-config": dial tcp 172.18.0.3:3001: connect: connection refused
ERROR: 2025/10/20 11:20:15 Failed to load configuration: Post "http://pangolin:3001/api/v1/gerbil/get-config": dial tcp 172.18.0.3:3001: connect: connection refused
INFO: 2025/10/20 11:20:20 Fetching remote config from http://pangolin:3001/api/v1/gerbil/get-config
ERROR: 2025/10/20 11:20:20 Error fetching remote config http://pangolin:3001/api/v1/gerbil/get-config: Post "http://pangolin:3001/api/v1/gerbil/get-config": dial tcp 172.18.0.3:3001: connect: connection refused
ERROR: 2025/10/20 11:20:20 Failed to load configuration: Post "http://pangolin:3001/api/v1/gerbil/get-config": dial tcp 172.18.0.3:3001: connect: connection refused

And Traefik logs spit this out:
{"level":"error","providerName":"http","error":"cannot fetch configuration data: do fetch request: Get "http://pangolin:3001/api/v1/traefik-config": dial tcp 172.18.0.4:3001: connect: connection refused","time":"2025-10-20T05:40:10Z","message":"Provider error, retrying in 695.480607ms"}
{"level":"error","providerName":"http","error":"cannot fetch configuration data: do fetch request: Get "http://pangolin:3001/api/v1/traefik-config": dial tcp 172.18.0.4:3001: connect: connection refused","time":"2025-10-20T05:40:11Z","message":"Provider error, retrying in 856.717352ms"}
ERROR: CrowdsecBouncerTraefikPlugin: 2025/10/20 05:40:11 handleMetricsTicker:reportMetrics reportMetrics:query crowdsecQuery:unreachable url:http://crowdsec:8080/v1/usage-metrics Post "http://crowdsec:8080/v1/usage-metrics": dial tcp 172.18.0.3:8080: connect: connection refused
{"level":"error","providerName":"http","error":"cannot fetch configuration data: do fetch request: Get "http://pangolin:3001/api/v1/traefik-config": dial tcp 172.18.0.4:3001: connect: connection refused","time":"2025-10-20T05:40:12Z","message":"Provider error, retrying in 1.576982697s"}
{"level":"error","providerName":"http","error":"cannot fetch configuration data: do fetch request: Get "http://pangolin:3001/api/v1/traefik-config": dial tcp 172.18.0.4:3001: connect: connection refused","time":"2025-10-20T05:40:13Z","message":"Provider error, retrying in 2.509794383s"}
ERROR: CrowdsecBouncerTraefikPlugin: 2025/10/20 05:40:23 appsecQuery:unreachable
ERROR: CrowdsecBouncerTraefikPlugin: 2025/10/20 05:40:23 appsecQuery:unreachable
{"level":"error","providerName":"http","error":"cannot fetch configuration data: do fetch request: Get "http://pangolin:3001/api/v1/traefik-config": context deadline exceeded (Client.Timeout exceeded while awaiting headers)","time":"2025-10-20T10:40:37Z","message":"Provider error, retrying in 424.080591ms"}
{"level":"error","providerName":"http","error":"cannot fetch configuration data: do fetch request: Get "http://pangolin:3001/api/v1/traefik-config": context deadline exceeded (Client.Timeout exceeded while awaiting headers)","time":"2025-10-20T10:40:53Z","message":"Provider error, retrying in 818.761466ms"}
{"level":"error","providerName":"http","error":"cannot fetch configuration data: do fetch request: Get "http://pangolin:3001/api/v1/traefik-config": context deadline exceeded (Client.Timeout exceeded while awaiting headers)","time":"2025-10-20T10:41:16Z","message":"Provider error, retrying in 701.622439ms"}
{"level":"error","providerName":"http","error":"cannot fetch configuration data: do fetch request: Get "http://pangolin:3001/api/v1/traefik-config": context deadline exceeded (Client.Timeout exceeded while awaiting headers)","time":"2025-10-20T10:43:16Z","message":"Provider error, retrying in 262.435427ms"}
{"level":"error","providerName":"http","error":"cannot fetch configuration data: do fetch request: Get "http://pangolin:3001/api/v1/traefik-config": context deadline exceeded (Client.Timeout exceeded while awaiting headers)","time":"2025-10-20T10:44:36Z","message":"Provider error, retrying in 313.481504ms"}
{"level":"error","providerName":"http","error":"cannot fetch configuration data: do fetch request: Get "http://pangolin:3001/api/v1/traefik-config": context deadline exceeded (Client.Timeout exceeded while awaiting headers)","time":"2025-10-20T10:49:43Z","message":"Provider error, retrying in 634.789881ms"}
{"level":"error","providerName":"http","error":"cannot fetch configuration data: do fetch request: Get "http://pangolin:3001/api/v1/traefik-config": context deadline exceeded (Client.Timeout exceeded while awaiting headers)","time":"2025-10-20T10:58:08Z","message":"Provider error, retrying in 257.667696ms"}

The Pangolin logs from what I can tell are completely clean and don't really demonstrate any issues. I've had them on debug for the past couple of weeks to try and troubleshoot this but there's nothing in them that really demonstrates a problem.

A few things about my setup:

  • My instance is behind Cloudflare proxy, I have not tried exposing it yet to see if that makes a difference or not. Certs are being pulled through Cloudflare.
  • I am running the crowdsec plugin. Those logs also look clean, no errors from what I can tell.
  • Local network IPs are whitelisted in my config along with Cloudflare DNS IPs setup for trusted header forwarding.
  • I'm using Defguard as my OpenID Auth provider.

Environment

  • OS Type & Version: Ubuntu 24.04
  • Pangolin Version: 1.9.4
  • Gerbil Version: 1.2.2
  • Traefik Version: 3.5.3
  • Newt Version: 1.5.2 (on two nodes)

To Reproduce

No idea, the issue is completely random as far as I can tell. Only way I know it's down is I try to access one of my sub-domains and they don't work. A restart of the whole stack resolves the problem. I've had it go down after an hour or two, lately it's generally stable for two to three days. I'll note that this has been a consistent issue over several versions of Pangolin to date.

Expected Behavior

The Pangolin stack stays stable for a consistent period of time.

Originally created by @clanger81 on GitHub (Oct 20, 2025). ### Describe the Bug Sorry, I didn't really know how to title this problem accurately. I've been having this issue intermittently for awhile now and it's becoming pretty frustrating. From the logs of each container, it looks like both gerbil and traefik lose connection to the pangolin container, there doesn't appear to be a specific reason, seems to happen completely randomly. Before running Pangolin, I had a single wireguard tunnel back to my local server VLAN, traefik was configured manually with rules from a file provider. I also had an instance of Crowdsec running alongside. I'd really like to continue using Pangolin as it makes setup of sub-domains significantly easier than doing it all by hand. Ultimately Gerbil logs will spit out the following at some stage: INFO: 2025/10/20 10:44:55 Failed to report peer bandwidth: API returned non-OK status: 408 Request Timeout INFO: 2025/10/20 10:46:52 Failed to report peer bandwidth: failed to send bandwidth data: Post "http://pangolin:3001/api/v1/gerbil/receive-bandwidth": dial tcp: lookup pangolin: i/o timeout INFO: 2025/10/20 10:49:00 Failed to report peer bandwidth: failed to send bandwidth data: Post "http://pangolin:3001/api/v1/gerbil/receive-bandwidth": dial tcp: lookup pangolin: i/o timeout INFO: 2025/10/20 10:49:47 Failed to report peer bandwidth: failed to send bandwidth data: Post "http://pangolin:3001/api/v1/gerbil/receive-bandwidth": dial tcp: lookup pangolin on 127.0.0.11:53: read udp 127.0.0.1:56696->127.0.0.11:53: i/o timeout INFO: 2025/10/20 10:50:47 Failed to report peer bandwidth: failed to send bandwidth data: Post "http://pangolin:3001/api/v1/gerbil/receive-bandwidth": dial tcp: lookup pangolin: i/o timeout INFO: 2025/10/20 10:52:45 Failed to report peer bandwidth: failed to send bandwidth data: Post "http://pangolin:3001/api/v1/gerbil/receive-bandwidth": dial tcp: lookup pangolin: i/o timeout INFO: 2025/10/20 10:54:14 Failed to report peer bandwidth: failed to send bandwidth data: Post "http://pangolin:3001/api/v1/gerbil/receive-bandwidth": dial tcp: lookup pangolin: i/o timeout INFO: 2025/10/20 10:57:52 Failed to report peer bandwidth: failed to send bandwidth data: Post "http://pangolin:3001/api/v1/gerbil/receive-bandwidth": dial tcp: lookup pangolin: i/o timeout INFO: 2025/10/20 11:00:14 Failed to report peer bandwidth: failed to send bandwidth data: Post "http://pangolin:3001/api/v1/gerbil/receive-bandwidth": dial tcp: lookup pangolin: i/o timeout INFO: 2025/10/20 11:06:44 Failed to report peer bandwidth: failed to send bandwidth data: Post "http://pangolin:3001/api/v1/gerbil/receive-bandwidth": dial tcp: lookup pangolin: i/o timeout INFO: 2025/10/20 11:11:17 Failed to report peer bandwidth: failed to send bandwidth data: Post "http://pangolin:3001/api/v1/gerbil/receive-bandwidth": dial tcp: lookup pangolin: i/o timeout INFO: 2025/10/20 11:20:15 Fetching remote config from http://pangolin:3001/api/v1/gerbil/get-config ERROR: 2025/10/20 11:20:15 Error fetching remote config http://pangolin:3001/api/v1/gerbil/get-config: Post "http://pangolin:3001/api/v1/gerbil/get-config": dial tcp 172.18.0.3:3001: connect: connection refused ERROR: 2025/10/20 11:20:15 Failed to load configuration: Post "http://pangolin:3001/api/v1/gerbil/get-config": dial tcp 172.18.0.3:3001: connect: connection refused INFO: 2025/10/20 11:20:20 Fetching remote config from http://pangolin:3001/api/v1/gerbil/get-config ERROR: 2025/10/20 11:20:20 Error fetching remote config http://pangolin:3001/api/v1/gerbil/get-config: Post "http://pangolin:3001/api/v1/gerbil/get-config": dial tcp 172.18.0.3:3001: connect: connection refused ERROR: 2025/10/20 11:20:20 Failed to load configuration: Post "http://pangolin:3001/api/v1/gerbil/get-config": dial tcp 172.18.0.3:3001: connect: connection refused And Traefik logs spit this out: {"level":"error","providerName":"http","error":"cannot fetch configuration data: do fetch request: Get \"http://pangolin:3001/api/v1/traefik-config\": dial tcp 172.18.0.4:3001: connect: connection refused","time":"2025-10-20T05:40:10Z","message":"Provider error, retrying in 695.480607ms"} {"level":"error","providerName":"http","error":"cannot fetch configuration data: do fetch request: Get \"http://pangolin:3001/api/v1/traefik-config\": dial tcp 172.18.0.4:3001: connect: connection refused","time":"2025-10-20T05:40:11Z","message":"Provider error, retrying in 856.717352ms"} ERROR: CrowdsecBouncerTraefikPlugin: 2025/10/20 05:40:11 handleMetricsTicker:reportMetrics reportMetrics:query crowdsecQuery:unreachable url:http://crowdsec:8080/v1/usage-metrics Post "http://crowdsec:8080/v1/usage-metrics": dial tcp 172.18.0.3:8080: connect: connection refused {"level":"error","providerName":"http","error":"cannot fetch configuration data: do fetch request: Get \"http://pangolin:3001/api/v1/traefik-config\": dial tcp 172.18.0.4:3001: connect: connection refused","time":"2025-10-20T05:40:12Z","message":"Provider error, retrying in 1.576982697s"} {"level":"error","providerName":"http","error":"cannot fetch configuration data: do fetch request: Get \"http://pangolin:3001/api/v1/traefik-config\": dial tcp 172.18.0.4:3001: connect: connection refused","time":"2025-10-20T05:40:13Z","message":"Provider error, retrying in 2.509794383s"} ERROR: CrowdsecBouncerTraefikPlugin: 2025/10/20 05:40:23 appsecQuery:unreachable ERROR: CrowdsecBouncerTraefikPlugin: 2025/10/20 05:40:23 appsecQuery:unreachable {"level":"error","providerName":"http","error":"cannot fetch configuration data: do fetch request: Get \"http://pangolin:3001/api/v1/traefik-config\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)","time":"2025-10-20T10:40:37Z","message":"Provider error, retrying in 424.080591ms"} {"level":"error","providerName":"http","error":"cannot fetch configuration data: do fetch request: Get \"http://pangolin:3001/api/v1/traefik-config\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)","time":"2025-10-20T10:40:53Z","message":"Provider error, retrying in 818.761466ms"} {"level":"error","providerName":"http","error":"cannot fetch configuration data: do fetch request: Get \"http://pangolin:3001/api/v1/traefik-config\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)","time":"2025-10-20T10:41:16Z","message":"Provider error, retrying in 701.622439ms"} {"level":"error","providerName":"http","error":"cannot fetch configuration data: do fetch request: Get \"http://pangolin:3001/api/v1/traefik-config\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)","time":"2025-10-20T10:43:16Z","message":"Provider error, retrying in 262.435427ms"} {"level":"error","providerName":"http","error":"cannot fetch configuration data: do fetch request: Get \"http://pangolin:3001/api/v1/traefik-config\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)","time":"2025-10-20T10:44:36Z","message":"Provider error, retrying in 313.481504ms"} {"level":"error","providerName":"http","error":"cannot fetch configuration data: do fetch request: Get \"http://pangolin:3001/api/v1/traefik-config\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)","time":"2025-10-20T10:49:43Z","message":"Provider error, retrying in 634.789881ms"} {"level":"error","providerName":"http","error":"cannot fetch configuration data: do fetch request: Get \"http://pangolin:3001/api/v1/traefik-config\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)","time":"2025-10-20T10:58:08Z","message":"Provider error, retrying in 257.667696ms"} The Pangolin logs from what I can tell are completely clean and don't really demonstrate any issues. I've had them on debug for the past couple of weeks to try and troubleshoot this but there's nothing in them that really demonstrates a problem. A few things about my setup: - My instance is behind Cloudflare proxy, I have not tried exposing it yet to see if that makes a difference or not. Certs are being pulled through Cloudflare. - I am running the crowdsec plugin. Those logs also look clean, no errors from what I can tell. - Local network IPs are whitelisted in my config along with Cloudflare DNS IPs setup for trusted header forwarding. - I'm using Defguard as my OpenID Auth provider. ### Environment - OS Type & Version: Ubuntu 24.04 - Pangolin Version: 1.9.4 - Gerbil Version: 1.2.2 - Traefik Version: 3.5.3 - Newt Version: 1.5.2 (on two nodes) ### To Reproduce No idea, the issue is completely random as far as I can tell. Only way I know it's down is I try to access one of my sub-domains and they don't work. A restart of the whole stack resolves the problem. I've had it go down after an hour or two, lately it's generally stable for two to three days. I'll note that this has been a consistent issue over several versions of Pangolin to date. ### Expected Behavior The Pangolin stack stays stable for a consistent period of time.
GiteaMirror commented 2025-11-13 12:09:32 -06:00
Author
Owner
Copy Link
Copy Source

@scroguard commented on GitHub (Oct 23, 2025):

i too am experiencing this issue, it started after updating to the ee-latest branch. mine happens roughly ever 24-48 hours. a restart of the entire stack brings everything back online.

OS: Debian 13
Pangolin Version: ee-latest
Gerbil Version: latest
Traefik Version: 3.5.2
Newt Version: latest on most nodes, otherwise 1.2.1

Snippet from logs from the last crash:

traefik | 50.193.208.181 - - [23/Oct/2025:11:00:07 +0000] "GET /agent.ashx HTTP/1.1" 404 19 "-" "-" 6182 "-" "-" 0ms
traefik | 50.193.208.177 - - [23/Oct/2025:11:00:08 +0000] "POST /webGui/include/NetworkInfo.php HTTP/2.0" 200 3 "-" "-" 6183 "93-unraid-2-router@http" "https://100.89.128.4:40610" 48ms
traefik | 50.193.208.177 - - [23/Oct/2025:11:00:13 +0000] "POST /webGui/include/NetworkInfo.php HTTP/2.0" 200 3 "-" "-" 6184 "93-unraid-2-router@http" "https://100.89.128.4:40610" 52ms
traefik | 50.193.208.177 - - [23/Oct/2025:11:00:18 +0000] "POST /webGui/include/NetworkInfo.php HTTP/2.0" 200 3 "-" "-" 6185 "93-unraid-2-router@http" "https://100.89.128.4:40610" 51ms
traefik | 50.193.208.177 - - [23/Oct/2025:11:00:23 +0000] "POST /webGui/include/NetworkInfo.php HTTP/2.0" 200 3 "-" "-" 6186 "93-unraid-2-router@http" "https://100.89.128.4:40610" 56ms
traefik | 50.193.208.177 - - [23/Oct/2025:11:00:28 +0000] "POST /webGui/include/NetworkInfo.php HTTP/2.0" 200 3 "-" "-" 6187 "93-unraid-2-router@http" "https://100.89.128.4:40610" 48ms
traefik | 50.193.208.177 - - [23/Oct/2025:11:00:33 +0000] "POST /webGui/include/NetworkInfo.php HTTP/2.0" 200 3 "-" "-" 6188 "93-unraid-2-router@http" "https://100.89.128.4:40610" 48ms
traefik | 50.193.208.177 - - [23/Oct/2025:11:00:38 +0000] "POST /webGui/include/NetworkInfo.php HTTP/2.0" 200 3 "-" "-" 6189 "93-unraid-2-router@http" "https://100.89.128.4:40610" 47ms
traefik | 50.193.208.177 - - [23/Oct/2025:11:00:44 +0000] "POST /webGui/include/NetworkInfo.php HTTP/2.0" 200 3 "-" "-" 6190 "93-unraid-2-router@http" "https://100.89.128.4:40610" 50ms
traefik | 50.193.208.177 - - [23/Oct/2025:11:00:49 +0000] "POST /webGui/include/NetworkInfo.php HTTP/2.0" 200 3 "-" "-" 6191 "93-unraid-2-router@http" "https://100.89.128.4:40610" 49ms
traefik | 50.193.208.177 - - [23/Oct/2025:11:00:54 +0000] "POST /webGui/include/NetworkInfo.php HTTP/2.0" 200 3 "-" "-" 6192 "93-unraid-2-router@http" "https://100.89.128.4:40610" 47ms
traefik | 50.193.208.177 - - [23/Oct/2025:11:00:59 +0000] "POST /webGui/include/NetworkInfo.php HTTP/2.0" 200 3 "-" "-" 6193 "93-unraid-2-router@http" "https://100.89.128.4:40610" 47ms
traefik | 50.193.208.177 - - [23/Oct/2025:11:01:04 +0000] "POST /webGui/include/NetworkInfo.php HTTP/2.0" 200 3 "-" "-" 6194 "93-unraid-2-router@http" "https://100.89.128.4:40610" 46ms
traefik | 2025-10-23T11:01:08Z INF I have to go...
traefik | 2025-10-23T11:01:08Z INF Stopping server gracefully
traefik | 50.193.208.177 - - [23/Oct/2025:08:00:52 +0000] "GET /api/v1/ws?clientType=newt&token=rcn2ok4ntdzpcoqxn76shobcjho2eyms HTTP/1.1" 0 0 "-" "-" 23 "api-router@file" "http://pangolin:3000" 10815690ms
traefik | 50.193.208.177 - - [23/Oct/2025:08:00:57 +0000] "GET /api/v1/ws?token=evyvk6gmaxapubx7vdxo2jqvizigsljc HTTP/1.1" 0 0 "-" "-" 35 "api-router@file" "http://pangolin:3000" 10810683ms
traefik | 50.193.208.177 - - [23/Oct/2025:08:00:57 +0000] "GET /api/v1/ws?token=3bed44bqxkhnac3i5ta3eg4rvpidfmi3 HTTP/1.1" 0 0 "-" "-" 34 "api-router@file" "http://pangolin:3000" 10810693ms
traefik | 74.48.158.10 - - [23/Oct/2025:08:00:54 +0000] "GET /api/v1/ws?token=fxphi4xiaatkgcufxopl3hvkyck53lwp HTTP/1.1" 0 0 "-" "-" 31 "api-router@file" "http://pangolin:3000" 10813316ms
traefik | 50.193.208.177 - - [23/Oct/2025:08:00:52 +0000] "GET /api/v1/ws?clientType=newt&token=ilugazzicmzsg4wenqdln4yiquz7qt6w HTTP/1.1" 0 0 "-" "-" 28 "api-router@file" "http://pangolin:3000" 10815475ms
traefik | 50.193.208.177 - - [23/Oct/2025:08:00:52 +0000] "GET /api/v1/ws?clientType=newt&token=dt2tdvjslyegjoixnphhuurkz66vbbum HTTP/1.1" 0 0 "-" "-" 26 "api-router@file" "http://pangolin:3000" 10815582ms
traefik | 50.193.208.177 - - [23/Oct/2025:08:00:52 +0000] "GET /api/v1/ws?clientType=newt&token=jvll2hexjkojvuxikprk4hq6wlgjx5ye HTTP/1.1" 0 0 "-" "-" 25 "api-router@file" "http://pangolin:3000" 10815596ms
traefik | 50.193.208.177 - - [23/Oct/2025:08:00:52 +0000] "GET /api/v1/ws?clientType=newt&token=onzcu5papj2wrc5l2vcmk3s4btewy2cn HTTP/1.1" 0 0 "-" "-" 24 "api-router@file" "http://pangolin:3000" 10815602ms
traefik | 50.193.208.177 - - [23/Oct/2025:09:30:09 +0000] "GET /api/v1/ws?clientType=newt&token=txgs6jjlgwqffday54alejwsfch5nxxe HTTP/1.1" 0 0 "-" "-" 1402 "api-router@file" "http://pangolin:3000" 5459053ms
traefik | 50.193.208.177 - - [23/Oct/2025:08:01:43 +0000] "GET /api/v1/ws?token=isoijbnzegemoyuvxp3eaoqr2nic7p6b HTTP/1.1" 0 0 "-" "-" 52 "api-router@file" "http://pangolin:3000" 10764905ms
traefik | 50.193.208.177 - - [23/Oct/2025:08:00:58 +0000] "GET /api/v1/ws?token=e3rsysz3upvb4sgxofwqfznv74ejtcek HTTP/1.1" 0 0 "-" "-" 37 "api-router@file" "http://pangolin:3000" 10809594ms
traefik | 2025-10-23T11:01:13Z ERR Provider error, retrying in 687.251414ms error="cannot fetch configuration data: do fetch request: Get "http://pangolin:3001/api/v1/traefik-config": dial tcp: lookup pangolin on 127.0.0.11:53: read udp 127.0.0.1:55986->127.0.0.11:53: read: connection refused" providerName=http
traefik | 2025-10-23T11:01:13Z ERR Provider error, retrying in 837.042586ms error="cannot fetch configuration data: do fetch request: Get "http://pangolin:3001/api/v1/traefik-config": dial tcp: lookup pangolin on 127.0.0.11:53: read udp 127.0.0.1:44739->127.0.0.11:53: read: connection refused" providerName=http
traefik | 2025-10-23T11:01:14Z ERR Provider error, retrying in 654.74774ms error="cannot fetch configuration data: do fetch request: Get "http://pangolin:3001/api/v1/traefik-config": dial tcp: lookup pangolin on 127.0.0.11:53: read udp 127.0.0.1:37185->127.0.0.11:53: read: connection refused" providerName=http
traefik | 2025-10-23T11:01:15Z ERR Provider error, retrying in 2.496583882s error="cannot fetch configuration data: do fetch request: Get "http://pangolin:3001/api/v1/traefik-config": dial tcp: lookup pangolin on 127.0.0.11:53: read udp 127.0.0.1:56039->127.0.0.11:53: read: connection refused" providerName=http
traefik | 2025-10-23T11:01:17Z ERR Provider error, retrying in 3.319319448s error="cannot fetch configuration data: do fetch request: Get "http://pangolin:3001/api/v1/traefik-config": dial tcp: lookup pangolin on 127.0.0.11:53: read udp 127.0.0.1:49298->127.0.0.11:53: read: connection refused" providerName=http
traefik | 50.193.208.177 - - [23/Oct/2025:08:10:08 +0000] "GET /sub/temperature?last_event_id=1761206406%3A0 HTTP/1.1" 0 0 "-" "-" 194 "93-unraid-2-router@http" "https://100.89.128.4:40610" 10270192ms
traefik | 50.193.208.177 - - [23/Oct/2025:08:08:28 +0000] "GET /sub/session,var,notify?last_event_id=1761203419%3A-%2C-%2C%5B0%5D HTTP/1.1" 0 0 "-" "-" 166 "93-unraid-2-router@http" "https://100.89.128.4:40610" 10369753ms
traefik | 50.193.208.177 - - [23/Oct/2025:08:10:04 +0000] "GET /graphql HTTP/1.1" 0 0 "-" "-" 192 "1-stash-organizer-router@http" "http://100.89.128.16:65332" 10273936ms
traefik | 50.193.208.177 - - [23/Oct/2025:08:10:04 +0000] "GET /graphql HTTP/1.1" 0 0 "-" "-" 191 "93-unraid-2-router@http" "https://100.89.128.4:40610" 10274256ms
traefik | 2025-10-23T11:01:18Z INF Server stopped
traefik | 2025-10-23T11:01:18Z INF Shutting down
traefik | 2025-10-23T11:01:18Z ERR Cannot retrieve data error="context canceled" providerName=http
traefik | 98.160.214.233 - - [23/Oct/2025:08:01:22 +0000] "GET /:/eventsource/notifications?X-Plex-Token=FbAKo7XN_PhfZ6itiQ_R HTTP/1.0" 200 23716 "-" "-" 43 "41-plex-passthrough-router@http" "https://100.89.128.8:55372" 10795765ms

@scroguard commented on GitHub (Oct 23, 2025): i too am experiencing this issue, it started after updating to the ee-latest branch. mine happens roughly ever 24-48 hours. a restart of the entire stack brings everything back online. OS: Debian 13 Pangolin Version: ee-latest Gerbil Version: latest Traefik Version: 3.5.2 Newt Version: latest on most nodes, otherwise 1.2.1 Snippet from logs from the last crash: traefik | 50.193.208.181 - - [23/Oct/2025:11:00:07 +0000] "GET /agent.ashx HTTP/1.1" 404 19 "-" "-" 6182 "-" "-" 0ms traefik | 50.193.208.177 - - [23/Oct/2025:11:00:08 +0000] "POST /webGui/include/NetworkInfo.php HTTP/2.0" 200 3 "-" "-" 6183 "93-unraid-2-router@http" "https://100.89.128.4:40610" 48ms traefik | 50.193.208.177 - - [23/Oct/2025:11:00:13 +0000] "POST /webGui/include/NetworkInfo.php HTTP/2.0" 200 3 "-" "-" 6184 "93-unraid-2-router@http" "https://100.89.128.4:40610" 52ms traefik | 50.193.208.177 - - [23/Oct/2025:11:00:18 +0000] "POST /webGui/include/NetworkInfo.php HTTP/2.0" 200 3 "-" "-" 6185 "93-unraid-2-router@http" "https://100.89.128.4:40610" 51ms traefik | 50.193.208.177 - - [23/Oct/2025:11:00:23 +0000] "POST /webGui/include/NetworkInfo.php HTTP/2.0" 200 3 "-" "-" 6186 "93-unraid-2-router@http" "https://100.89.128.4:40610" 56ms traefik | 50.193.208.177 - - [23/Oct/2025:11:00:28 +0000] "POST /webGui/include/NetworkInfo.php HTTP/2.0" 200 3 "-" "-" 6187 "93-unraid-2-router@http" "https://100.89.128.4:40610" 48ms traefik | 50.193.208.177 - - [23/Oct/2025:11:00:33 +0000] "POST /webGui/include/NetworkInfo.php HTTP/2.0" 200 3 "-" "-" 6188 "93-unraid-2-router@http" "https://100.89.128.4:40610" 48ms traefik | 50.193.208.177 - - [23/Oct/2025:11:00:38 +0000] "POST /webGui/include/NetworkInfo.php HTTP/2.0" 200 3 "-" "-" 6189 "93-unraid-2-router@http" "https://100.89.128.4:40610" 47ms traefik | 50.193.208.177 - - [23/Oct/2025:11:00:44 +0000] "POST /webGui/include/NetworkInfo.php HTTP/2.0" 200 3 "-" "-" 6190 "93-unraid-2-router@http" "https://100.89.128.4:40610" 50ms traefik | 50.193.208.177 - - [23/Oct/2025:11:00:49 +0000] "POST /webGui/include/NetworkInfo.php HTTP/2.0" 200 3 "-" "-" 6191 "93-unraid-2-router@http" "https://100.89.128.4:40610" 49ms traefik | 50.193.208.177 - - [23/Oct/2025:11:00:54 +0000] "POST /webGui/include/NetworkInfo.php HTTP/2.0" 200 3 "-" "-" 6192 "93-unraid-2-router@http" "https://100.89.128.4:40610" 47ms traefik | 50.193.208.177 - - [23/Oct/2025:11:00:59 +0000] "POST /webGui/include/NetworkInfo.php HTTP/2.0" 200 3 "-" "-" 6193 "93-unraid-2-router@http" "https://100.89.128.4:40610" 47ms traefik | 50.193.208.177 - - [23/Oct/2025:11:01:04 +0000] "POST /webGui/include/NetworkInfo.php HTTP/2.0" 200 3 "-" "-" 6194 "93-unraid-2-router@http" "https://100.89.128.4:40610" 46ms traefik | 2025-10-23T11:01:08Z INF I have to go... traefik | 2025-10-23T11:01:08Z INF Stopping server gracefully traefik | 50.193.208.177 - - [23/Oct/2025:08:00:52 +0000] "GET /api/v1/ws?clientType=newt&token=rcn2ok4ntdzpcoqxn76shobcjho2eyms HTTP/1.1" 0 0 "-" "-" 23 "api-router@file" "http://pangolin:3000" 10815690ms traefik | 50.193.208.177 - - [23/Oct/2025:08:00:57 +0000] "GET /api/v1/ws?token=evyvk6gmaxapubx7vdxo2jqvizigsljc HTTP/1.1" 0 0 "-" "-" 35 "api-router@file" "http://pangolin:3000" 10810683ms traefik | 50.193.208.177 - - [23/Oct/2025:08:00:57 +0000] "GET /api/v1/ws?token=3bed44bqxkhnac3i5ta3eg4rvpidfmi3 HTTP/1.1" 0 0 "-" "-" 34 "api-router@file" "http://pangolin:3000" 10810693ms traefik | 74.48.158.10 - - [23/Oct/2025:08:00:54 +0000] "GET /api/v1/ws?token=fxphi4xiaatkgcufxopl3hvkyck53lwp HTTP/1.1" 0 0 "-" "-" 31 "api-router@file" "http://pangolin:3000" 10813316ms traefik | 50.193.208.177 - - [23/Oct/2025:08:00:52 +0000] "GET /api/v1/ws?clientType=newt&token=ilugazzicmzsg4wenqdln4yiquz7qt6w HTTP/1.1" 0 0 "-" "-" 28 "api-router@file" "http://pangolin:3000" 10815475ms traefik | 50.193.208.177 - - [23/Oct/2025:08:00:52 +0000] "GET /api/v1/ws?clientType=newt&token=dt2tdvjslyegjoixnphhuurkz66vbbum HTTP/1.1" 0 0 "-" "-" 26 "api-router@file" "http://pangolin:3000" 10815582ms traefik | 50.193.208.177 - - [23/Oct/2025:08:00:52 +0000] "GET /api/v1/ws?clientType=newt&token=jvll2hexjkojvuxikprk4hq6wlgjx5ye HTTP/1.1" 0 0 "-" "-" 25 "api-router@file" "http://pangolin:3000" 10815596ms traefik | 50.193.208.177 - - [23/Oct/2025:08:00:52 +0000] "GET /api/v1/ws?clientType=newt&token=onzcu5papj2wrc5l2vcmk3s4btewy2cn HTTP/1.1" 0 0 "-" "-" 24 "api-router@file" "http://pangolin:3000" 10815602ms traefik | 50.193.208.177 - - [23/Oct/2025:09:30:09 +0000] "GET /api/v1/ws?clientType=newt&token=txgs6jjlgwqffday54alejwsfch5nxxe HTTP/1.1" 0 0 "-" "-" 1402 "api-router@file" "http://pangolin:3000" 5459053ms traefik | 50.193.208.177 - - [23/Oct/2025:08:01:43 +0000] "GET /api/v1/ws?token=isoijbnzegemoyuvxp3eaoqr2nic7p6b HTTP/1.1" 0 0 "-" "-" 52 "api-router@file" "http://pangolin:3000" 10764905ms traefik | 50.193.208.177 - - [23/Oct/2025:08:00:58 +0000] "GET /api/v1/ws?token=e3rsysz3upvb4sgxofwqfznv74ejtcek HTTP/1.1" 0 0 "-" "-" 37 "api-router@file" "http://pangolin:3000" 10809594ms traefik | 2025-10-23T11:01:13Z ERR Provider error, retrying in 687.251414ms error="cannot fetch configuration data: do fetch request: Get \"http://pangolin:3001/api/v1/traefik-config\": dial tcp: lookup pangolin on 127.0.0.11:53: read udp 127.0.0.1:55986->127.0.0.11:53: read: connection refused" providerName=http traefik | 2025-10-23T11:01:13Z ERR Provider error, retrying in 837.042586ms error="cannot fetch configuration data: do fetch request: Get \"http://pangolin:3001/api/v1/traefik-config\": dial tcp: lookup pangolin on 127.0.0.11:53: read udp 127.0.0.1:44739->127.0.0.11:53: read: connection refused" providerName=http traefik | 2025-10-23T11:01:14Z ERR Provider error, retrying in 654.74774ms error="cannot fetch configuration data: do fetch request: Get \"http://pangolin:3001/api/v1/traefik-config\": dial tcp: lookup pangolin on 127.0.0.11:53: read udp 127.0.0.1:37185->127.0.0.11:53: read: connection refused" providerName=http traefik | 2025-10-23T11:01:15Z ERR Provider error, retrying in 2.496583882s error="cannot fetch configuration data: do fetch request: Get \"http://pangolin:3001/api/v1/traefik-config\": dial tcp: lookup pangolin on 127.0.0.11:53: read udp 127.0.0.1:56039->127.0.0.11:53: read: connection refused" providerName=http traefik | 2025-10-23T11:01:17Z ERR Provider error, retrying in 3.319319448s error="cannot fetch configuration data: do fetch request: Get \"http://pangolin:3001/api/v1/traefik-config\": dial tcp: lookup pangolin on 127.0.0.11:53: read udp 127.0.0.1:49298->127.0.0.11:53: read: connection refused" providerName=http traefik | 50.193.208.177 - - [23/Oct/2025:08:10:08 +0000] "GET /sub/temperature?last_event_id=1761206406%3A0 HTTP/1.1" 0 0 "-" "-" 194 "93-unraid-2-router@http" "https://100.89.128.4:40610" 10270192ms traefik | 50.193.208.177 - - [23/Oct/2025:08:08:28 +0000] "GET /sub/session,var,notify?last_event_id=1761203419%3A-%2C-%2C%5B0%5D HTTP/1.1" 0 0 "-" "-" 166 "93-unraid-2-router@http" "https://100.89.128.4:40610" 10369753ms traefik | 50.193.208.177 - - [23/Oct/2025:08:10:04 +0000] "GET /graphql HTTP/1.1" 0 0 "-" "-" 192 "1-stash-organizer-router@http" "http://100.89.128.16:65332" 10273936ms traefik | 50.193.208.177 - - [23/Oct/2025:08:10:04 +0000] "GET /graphql HTTP/1.1" 0 0 "-" "-" 191 "93-unraid-2-router@http" "https://100.89.128.4:40610" 10274256ms traefik | 2025-10-23T11:01:18Z INF Server stopped traefik | 2025-10-23T11:01:18Z INF Shutting down traefik | 2025-10-23T11:01:18Z ERR Cannot retrieve data error="context canceled" providerName=http traefik | 98.160.214.233 - - [23/Oct/2025:08:01:22 +0000] "GET /:/eventsource/notifications?X-Plex-Token=FbAKo7XN_PhfZ6itiQ_R HTTP/1.0" 200 23716 "-" "-" 43 "41-plex-passthrough-router@http" "https://100.89.128.8:55372" 10795765ms
GiteaMirror commented 2025-11-13 12:09:32 -06:00
Author
Owner
Copy Link
Copy Source

@clanger81 commented on GitHub (Oct 24, 2025):

To add to this, I had thought my stack was running the latest pangolin container but it turns out it wasn't. I also noticed some of the healthcheck config changed from the default compose file that the installer puts together. Updated my compose to match the repository example and updated Pangolin to v1.11.0 and the stack was stable for about 36 hours before losing network connectivity again. I'm really not sure what to do at this stage... My wife gets seriously upset when she loses connectivity to Bitwarden... heh.

@clanger81 commented on GitHub (Oct 24, 2025): To add to this, I had thought my stack was running the latest pangolin container but it turns out it wasn't. I also noticed some of the healthcheck config changed from the default compose file that the installer puts together. Updated my compose to match the repository example and updated Pangolin to v1.11.0 and the stack was stable for about 36 hours before losing network connectivity again. I'm really not sure what to do at this stage... My wife gets seriously upset when she loses connectivity to Bitwarden... heh.
GiteaMirror commented 2025-11-13 12:09:32 -06:00
Author
Owner
Copy Link
Copy Source

@Lokowitz commented on GitHub (Oct 24, 2025):

Hi @clanger81,

cloud you please post your docker-compose.yml ? Maybe it is not up to date anymore, there were some updates in the last releases.
And why are you just upgrading to 1.11.0 and not 1.11.1 ? :)

@Lokowitz commented on GitHub (Oct 24, 2025): Hi @clanger81, cloud you please post your docker-compose.yml ? Maybe it is not up to date anymore, there were some updates in the last releases. And why are you just upgrading to 1.11.0 and not 1.11.1 ? :)
GiteaMirror commented 2025-11-13 12:09:32 -06:00
Author
Owner
Copy Link
Copy Source

@clanger81 commented on GitHub (Oct 24, 2025):

Heh, I did pull 1.11.1 this morning restarting my containers because they were down. Compose file below:

name: pangolin
networks:
  default:
    driver: bridge
    enable_ipv6: true
    name: pangolin

services:
  crowdsec:
    command: -t
    container_name: crowdsec
    environment:
      COLLECTIONS: crowdsecurity/traefik crowdsecurity/appsec-virtual-patching crowdsecurity/appsec-generic-rules crowdsecurity/linux crowdsecurity/iptables
      ENROLL_INSTANCE_NAME: pangolin-crowdsec
      ENROLL_TAGS: docker
      GID: "1000"
      PARSERS: crowdsecurity/whitelists
    healthcheck:
      interval: 10s
      retries: 15
      test:
        - CMD
        - cscli
        - capi
        - status
      timeout: 10s
    image: docker.io/crowdsecurity/crowdsec:latest
    labels:
      - traefik.enable=false
    ports:
      - 6060:6060
    restart: unless-stopped
    volumes:
      - ./config/crowdsec:/etc/crowdsec
      - ./config/crowdsec/db:/var/lib/crowdsec/data
      - ./config/traefik/logs:/var/log/traefik:ro
      - /var/log/journal:/var/log/host:ro

  gerbil:
    cap_add:
      - NET_ADMIN
      - SYS_MODULE
    command:
      - --reachableAt=http://gerbil:3004
      - --generateAndSaveKeyTo=/var/config/key
      - --remoteConfig=http://pangolin:3001/api/v1/
    container_name: gerbil
    depends_on:
      pangolin:
        condition: service_healthy
    image: docker.io/fosrl/gerbil:latest
    ports:
      - 51820:51820/udp
      - 21820:21820/udp
      - 443:443
      - 80:80
    restart: unless-stopped
    volumes:
      - ./config/:/var/config
  pangolin:
    container_name: pangolin
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3001/api/v1/"]
      interval: "3s"
      timeout: "3s"
      retries: 15
    image: docker.io/fosrl/pangolin:latest
    restart: unless-stopped
    volumes:
      - ./config:/app/config
      - pangolin-data:/var/certificates
      - pangolin-data:/var/dynamic

  traefik:
    command:
      - --configFile=/etc/traefik/traefik_config.yml
    container_name: traefik
    depends_on:
      crowdsec:
        condition: service_healthy
      pangolin:
        condition: service_healthy
    image: docker.io/traefik:v3.5
    network_mode: service:gerbil
    restart: unless-stopped
    environment:
      CLOUDFLARE_DNS_API_TOKEN: "MY_API_KEY"
    volumes:
      - ./config/traefik:/etc/traefik:ro
      - ./config/letsencrypt:/letsencrypt
      - ./config/traefik/logs:/var/log/traefik
      - pangolin-data:/var/certificates:ro
      - pangolin-data:/var/dynamic:ro

volumes:
  pangolin-data: null
@clanger81 commented on GitHub (Oct 24, 2025): Heh, I did pull 1.11.1 this morning restarting my containers because they were down. Compose file below: ``` name: pangolin networks: default: driver: bridge enable_ipv6: true name: pangolin services: crowdsec: command: -t container_name: crowdsec environment: COLLECTIONS: crowdsecurity/traefik crowdsecurity/appsec-virtual-patching crowdsecurity/appsec-generic-rules crowdsecurity/linux crowdsecurity/iptables ENROLL_INSTANCE_NAME: pangolin-crowdsec ENROLL_TAGS: docker GID: "1000" PARSERS: crowdsecurity/whitelists healthcheck: interval: 10s retries: 15 test: - CMD - cscli - capi - status timeout: 10s image: docker.io/crowdsecurity/crowdsec:latest labels: - traefik.enable=false ports: - 6060:6060 restart: unless-stopped volumes: - ./config/crowdsec:/etc/crowdsec - ./config/crowdsec/db:/var/lib/crowdsec/data - ./config/traefik/logs:/var/log/traefik:ro - /var/log/journal:/var/log/host:ro gerbil: cap_add: - NET_ADMIN - SYS_MODULE command: - --reachableAt=http://gerbil:3004 - --generateAndSaveKeyTo=/var/config/key - --remoteConfig=http://pangolin:3001/api/v1/ container_name: gerbil depends_on: pangolin: condition: service_healthy image: docker.io/fosrl/gerbil:latest ports: - 51820:51820/udp - 21820:21820/udp - 443:443 - 80:80 restart: unless-stopped volumes: - ./config/:/var/config pangolin: container_name: pangolin healthcheck: test: ["CMD", "curl", "-f", "http://localhost:3001/api/v1/"] interval: "3s" timeout: "3s" retries: 15 image: docker.io/fosrl/pangolin:latest restart: unless-stopped volumes: - ./config:/app/config - pangolin-data:/var/certificates - pangolin-data:/var/dynamic traefik: command: - --configFile=/etc/traefik/traefik_config.yml container_name: traefik depends_on: crowdsec: condition: service_healthy pangolin: condition: service_healthy image: docker.io/traefik:v3.5 network_mode: service:gerbil restart: unless-stopped environment: CLOUDFLARE_DNS_API_TOKEN: "MY_API_KEY" volumes: - ./config/traefik:/etc/traefik:ro - ./config/letsencrypt:/letsencrypt - ./config/traefik/logs:/var/log/traefik - pangolin-data:/var/certificates:ro - pangolin-data:/var/dynamic:ro volumes: pangolin-data: null ```
GiteaMirror commented 2025-11-13 12:09:33 -06:00
Author
Owner
Copy Link
Copy Source

@Lokowitz commented on GitHub (Oct 24, 2025):

Hmpf I think config is looking good.
What kind of system do you have? (cpu, ram, disk space)
Did you check your ram usage? Crowdsec can be very ram hungry.
Did you already checked if it is working without crowdsec?

@Lokowitz commented on GitHub (Oct 24, 2025): Hmpf I think config is looking good. What kind of system do you have? (cpu, ram, disk space) Did you check your ram usage? Crowdsec can be very ram hungry. Did you already checked if it is working without crowdsec?
GiteaMirror commented 2025-11-13 12:09:33 -06:00
Author
Owner
Copy Link
Copy Source

@Lokowitz commented on GitHub (Oct 24, 2025):

And could you please check if the following lines are included in /config/traefik/dynamic_config.yml

http:
  middlewares:
    default-whitelist: # Whitelist middleware for internal IPs
      ipWhiteList:  # Internal IP addresses
        sourceRange:  # Internal IP addresses
        - "10.0.0.0/8"  # Internal IP addresses
        - "192.168.0.0/16" # Internal IP addresses
        - "172.16.0.0/12" # Internal IP addresses
    crowdsec:
      plugin:
        crowdsec:
          forwardedHeadersTrustedIPs: # Forwarded headers trusted IPs
            - "0.0.0.0/0" # All IP addresses are trusted for forwarded headers (CHANGE MADE HERE)
          clientTrustedIPs: # Client trusted IPs (CHANGE MADE HERE)
            - "10.0.0.0/8" # Internal LAN IP addresses
            - "172.16.0.0/12" # Internal LAN IP addresses
            - "192.168.0.0/16" # Internal LAN IP addresses
            - "100.89.137.0/20" # Internal LAN IP addresses
@Lokowitz commented on GitHub (Oct 24, 2025): And could you please check if the following lines are included in `/config/traefik/dynamic_config.yml` ``` http: middlewares: default-whitelist: # Whitelist middleware for internal IPs ipWhiteList: # Internal IP addresses sourceRange: # Internal IP addresses - "10.0.0.0/8" # Internal IP addresses - "192.168.0.0/16" # Internal IP addresses - "172.16.0.0/12" # Internal IP addresses crowdsec: plugin: crowdsec: forwardedHeadersTrustedIPs: # Forwarded headers trusted IPs - "0.0.0.0/0" # All IP addresses are trusted for forwarded headers (CHANGE MADE HERE) clientTrustedIPs: # Client trusted IPs (CHANGE MADE HERE) - "10.0.0.0/8" # Internal LAN IP addresses - "172.16.0.0/12" # Internal LAN IP addresses - "192.168.0.0/16" # Internal LAN IP addresses - "100.89.137.0/20" # Internal LAN IP addresses ```
GiteaMirror commented 2025-11-13 12:09:33 -06:00
Author
Owner
Copy Link
Copy Source

@Lokowitz commented on GitHub (Oct 24, 2025):

And are you doing docker compose restart or docker compose down && docker compose up -d?
So if you are just using restart please switch to down and up to completely recreate the docker network.

@Lokowitz commented on GitHub (Oct 24, 2025): And are you doing `docker compose restart` or docker `compose down && docker compose up -d`? So if you are just using restart please switch to down and up to completely recreate the docker network.
GiteaMirror commented 2025-11-13 12:09:33 -06:00
Author
Owner
Copy Link
Copy Source

@clanger81 commented on GitHub (Oct 24, 2025):

Server is a VPS running on 2 cores (host is Xeon E5-2689 V4) 3GB of RAM and 2Tb of storage (19Gb used). Average CPU load is about 4% with 1.9Gb of memory usage. Per my first post, I used to run a single WG tunnel to my home server VLAN with 3 crowdsec bouncers (traefik, cloudflare, fw) with no issues whatsoever.

Always use docker compose up/down with a pull in between to ensure latest containers.

dynamic_config.yml below:

http:
  middlewares:
    crowdsec:
      plugin:
        crowdsec:
          clientTrustedIPs:
            - 10.0.0.0/8
            - 172.16.0.0/12
            - 192.168.0.0/16
            - 100.89.137.0/20
          crowdsecAppsecBodyLimit: 10485760
          crowdsecAppsecEnabled: true
          crowdsecAppsecFailureBlock: true
          crowdsecAppsecHost: crowdsec:7422
          crowdsecAppsecUnreachableBlock: true
          crowdsecLapiHost: crowdsec:8080
          crowdsecLapiKey: MY_CROWDSEC_API_KEY
          crowdsecLapiScheme: http
          crowdsecMode: live
          defaultDecisionSeconds: 15
          enabled: true
          forwardedHeadersTrustedIPs:
            - 10.0.0.0/8
            - 172.16.0.0/12
            - 192.168.0.0/16
            - 100.89.137.0/20
            - 173.245.48.0/20
            - 103.21.244.0/22
            - 103.22.200.0/22
            - 103.31.4.0/22
            - 141.101.64.0/18
            - 108.162.192.0/18
            - 190.93.240.0/20
            - 188.114.96.0/20
            - 197.234.240.0/22
            - 198.41.128.0/17
            - 162.158.0.0/15
            - 104.16.0.0/13
            - 104.24.0.0/14
            - 172.64.0.0/13
            - 131.0.72.0/22
          httpTimeoutSeconds: 10
          logLevel: INFO
          updateIntervalSeconds: 15
          updateMaxFailure: 0
    default-whitelist:
      ipWhiteList:
        sourceRange:
          - 10.0.0.0/8
          - 192.168.0.0/16
          - 172.16.0.0/12
    redirect-to-https:
      redirectScheme:
        scheme: https
    security-headers:
      headers:
        contentTypeNosniff: true
        customFrameOptionsValue: SAMEORIGIN
        customResponseHeaders:
          Server: ""
          X-Forwarded-Proto: https
          X-Powered-By: ""
        forceSTSHeader: true
        hostsProxyHeaders:
          - X-Forwarded-Host
        referrerPolicy: strict-origin-when-cross-origin
        sslProxyHeaders:
          X-Forwarded-Proto: https
        stsIncludeSubdomains: true
        stsPreload: true
        stsSeconds: 63072000
  routers:
    api-router:
      entryPoints:
        - websecure
      middlewares:
        - security-headers
      rule: Host(`pangolin.MY_DOMAIN_1`) && PathPrefix(`/api/v1`)
      service: api-service
      tls:
        certResolver: letsencrypt
    main-app-router-redirect:
      entryPoints:
        - web
      middlewares:
        - redirect-to-https
      rule: Host(`pangolin.MY_DOMAIN_1`)
      service: next-service
    next-router:
      entryPoints:
        - websecure
      middlewares:
        - security-headers
      rule: Host(`pangolin.MY_DOMAIN_1`) && !PathPrefix(`/api/v1`)
      service: next-service
      tls:
        certResolver: letsencrypt
        domains:
          - main: "MY_DOMAIN_1"
            sans:
              - "*.MY_DOMAIN_1"
          - main: "MY_DOMAIN_2"
            sans:
              - "*.MY_DOMAIN_2"
    ws-router:
      entryPoints:
        - websecure
      middlewares:
        - security-headers
      rule: Host(`pangolin.MY_DOMAIN_1`)
      service: api-service
      tls:
        certResolver: letsencrypt
  services:
    api-service:
      loadBalancer:
        servers:
          - url: http://pangolin:3000
    next-service:
      loadBalancer:
        servers:
          - url: http://pangolin:3002
@clanger81 commented on GitHub (Oct 24, 2025): Server is a VPS running on 2 cores (host is Xeon E5-2689 V4) 3GB of RAM and 2Tb of storage (19Gb used). Average CPU load is about 4% with 1.9Gb of memory usage. Per my first post, I used to run a single WG tunnel to my home server VLAN with 3 crowdsec bouncers (traefik, cloudflare, fw) with no issues whatsoever. Always use docker compose up/down with a pull in between to ensure latest containers. dynamic_config.yml below: ``` http: middlewares: crowdsec: plugin: crowdsec: clientTrustedIPs: - 10.0.0.0/8 - 172.16.0.0/12 - 192.168.0.0/16 - 100.89.137.0/20 crowdsecAppsecBodyLimit: 10485760 crowdsecAppsecEnabled: true crowdsecAppsecFailureBlock: true crowdsecAppsecHost: crowdsec:7422 crowdsecAppsecUnreachableBlock: true crowdsecLapiHost: crowdsec:8080 crowdsecLapiKey: MY_CROWDSEC_API_KEY crowdsecLapiScheme: http crowdsecMode: live defaultDecisionSeconds: 15 enabled: true forwardedHeadersTrustedIPs: - 10.0.0.0/8 - 172.16.0.0/12 - 192.168.0.0/16 - 100.89.137.0/20 - 173.245.48.0/20 - 103.21.244.0/22 - 103.22.200.0/22 - 103.31.4.0/22 - 141.101.64.0/18 - 108.162.192.0/18 - 190.93.240.0/20 - 188.114.96.0/20 - 197.234.240.0/22 - 198.41.128.0/17 - 162.158.0.0/15 - 104.16.0.0/13 - 104.24.0.0/14 - 172.64.0.0/13 - 131.0.72.0/22 httpTimeoutSeconds: 10 logLevel: INFO updateIntervalSeconds: 15 updateMaxFailure: 0 default-whitelist: ipWhiteList: sourceRange: - 10.0.0.0/8 - 192.168.0.0/16 - 172.16.0.0/12 redirect-to-https: redirectScheme: scheme: https security-headers: headers: contentTypeNosniff: true customFrameOptionsValue: SAMEORIGIN customResponseHeaders: Server: "" X-Forwarded-Proto: https X-Powered-By: "" forceSTSHeader: true hostsProxyHeaders: - X-Forwarded-Host referrerPolicy: strict-origin-when-cross-origin sslProxyHeaders: X-Forwarded-Proto: https stsIncludeSubdomains: true stsPreload: true stsSeconds: 63072000 routers: api-router: entryPoints: - websecure middlewares: - security-headers rule: Host(`pangolin.MY_DOMAIN_1`) && PathPrefix(`/api/v1`) service: api-service tls: certResolver: letsencrypt main-app-router-redirect: entryPoints: - web middlewares: - redirect-to-https rule: Host(`pangolin.MY_DOMAIN_1`) service: next-service next-router: entryPoints: - websecure middlewares: - security-headers rule: Host(`pangolin.MY_DOMAIN_1`) && !PathPrefix(`/api/v1`) service: next-service tls: certResolver: letsencrypt domains: - main: "MY_DOMAIN_1" sans: - "*.MY_DOMAIN_1" - main: "MY_DOMAIN_2" sans: - "*.MY_DOMAIN_2" ws-router: entryPoints: - websecure middlewares: - security-headers rule: Host(`pangolin.MY_DOMAIN_1`) service: api-service tls: certResolver: letsencrypt services: api-service: loadBalancer: servers: - url: http://pangolin:3000 next-service: loadBalancer: servers: - url: http://pangolin:3002 ```
GiteaMirror commented 2025-11-13 12:09:34 -06:00
Author
Owner
Copy Link
Copy Source

@scroguard commented on GitHub (Oct 27, 2025):

to answer the question regarding the server specs, here are mine:

VM inside proxmox running Debian 13 with the following settings:
4GB RAM
2 CPU cores
64GB allocated virtual disk

The VM usually hovers between 930 and 1.3GB of RAM usage

@scroguard commented on GitHub (Oct 27, 2025): to answer the question regarding the server specs, here are mine: VM inside proxmox running Debian 13 with the following settings: 4GB RAM 2 CPU cores 64GB allocated virtual disk The VM usually hovers between 930 and 1.3GB of RAM usage
GiteaMirror commented 2025-11-13 12:09:34 -06:00
Author
Owner
Copy Link
Copy Source

@HomelabHaven commented on GitHub (Oct 28, 2025):

To be honest I am glad I am not the only one experiencing this issue.
My compose and Traefik configs are basically identical to @clanger81

Things that I have tested/tried which did not fix the issue:

  • every conceivable "depends_on" configuration and also completely without "depends_on" in the compose stack
  • disabling container updates through Watchtower and now through Komodo
  • removing the "docker-volume-backup.stop-during-backup=true" label I have set to stop before backups

I have noticed that when things randomly go down I can sometimes get away with restarting the Traefik container for the services hosted through Pangolin to become available again but restarting the stack is the same amount of work and is what I usually do.

It is particularly annoying when the Komodo remote ("periphery") connection that is connected via a Pangolin resource is unreachable because then I can't manage it remotely with Komodo. ^_^

@HomelabHaven commented on GitHub (Oct 28, 2025): To be honest I am glad I am not the only one experiencing this issue. My compose and Traefik configs are basically identical to @clanger81 Things that I have tested/tried which did not fix the issue: - every conceivable "depends_on" configuration and also completely without "depends_on" in the compose stack - disabling container updates through Watchtower and now through Komodo - removing the "docker-volume-backup.stop-during-backup=true" label I have set to stop before backups I have noticed that when things randomly go down I can sometimes get away with restarting the Traefik container for the services hosted through Pangolin to become available again but restarting the stack is the same amount of work and is what I usually do. It is particularly annoying when the Komodo remote ("periphery") connection that is connected via a Pangolin resource is unreachable because then I can't manage it remotely with Komodo. ^_^
GiteaMirror commented 2025-11-13 12:09:34 -06:00
Author
Owner
Copy Link
Copy Source

@clanger81 commented on GitHub (Oct 28, 2025):

To be honest I am glad I am not the only one experiencing this issue. My compose and Traefik configs are basically identical to @clanger81

Things that I have tested/tried which did not fix the issue:

  • every conceivable "depends_on" configuration and also completely without "depends_on" in the compose stack
  • disabling container updates through Watchtower and now through Komodo
  • removing the "docker-volume-backup.stop-during-backup=true" label I have set to stop before backups

I have noticed that when things randomly go down I can sometimes get away with restarting the Traefik container for the services hosted through Pangolin to become available again but restarting the stack is the same amount of work and is what I usually do.

It is particularly annoying when the Komodo remote ("periphery") connection that is connected via a Pangolin resource is unreachable because then I can't manage it remotely with Komodo. ^_^

I have also noticed that a restart of the traefik container seems to resolve things though I'm not sure why this is the case given Gerbil is also complaining about not being able to connect to the pangolin container.

One thing I did forget to note, though I doubt is the solution, I don't have port 21820 open via ufw as I don't use the VPN client functionality of the Pangolin stack. I've never seen Gerbil complain about this but maybe there's a potential issue I'm missing here? Not sure how that would affect inter-container networking though.

@clanger81 commented on GitHub (Oct 28, 2025): > To be honest I am glad I am not the only one experiencing this issue. My compose and Traefik configs are basically identical to [@clanger81](https://github.com/clanger81) > > Things that I have tested/tried which did not fix the issue: > > * every conceivable "depends_on" configuration and also completely without "depends_on" in the compose stack > * disabling container updates through Watchtower and now through Komodo > * removing the "docker-volume-backup.stop-during-backup=true" label I have set to stop before backups > > I have noticed that when things randomly go down I can sometimes get away with restarting the Traefik container for the services hosted through Pangolin to become available again but restarting the stack is the same amount of work and is what I usually do. > > It is particularly annoying when the Komodo remote ("periphery") connection that is connected via a Pangolin resource is unreachable because then I can't manage it remotely with Komodo. ^_^ I have also noticed that a restart of the traefik container seems to resolve things though I'm not sure why this is the case given Gerbil is also complaining about not being able to connect to the pangolin container. One thing I did forget to note, though I doubt is the solution, I don't have port 21820 open via ufw as I don't use the VPN client functionality of the Pangolin stack. I've never seen Gerbil complain about this but maybe there's a potential issue I'm missing here? Not sure how that would affect inter-container networking though.
GiteaMirror commented 2025-11-13 12:09:34 -06:00
Author
Owner
Copy Link
Copy Source

@HomelabHaven commented on GitHub (Oct 28, 2025):

I don't think it has to do with the "olm" port (21820:21820/udp) seeing as I have it exposed.

@HomelabHaven commented on GitHub (Oct 28, 2025): I don't think it has to do with the "olm" port (21820:21820/udp) seeing as I have it exposed.
GiteaMirror commented 2025-11-13 12:09:35 -06:00
Author
Owner
Copy Link
Copy Source

@HomelabHaven commented on GitHub (Oct 28, 2025):

@clanger81 Did you follow the community guide for setting up crowdsec as I did?
https://docs.pangolin.net/self-host/community-guides/crowdsec

@HomelabHaven commented on GitHub (Oct 28, 2025): @clanger81 Did you follow the community guide for setting up crowdsec as I did? https://docs.pangolin.net/self-host/community-guides/crowdsec
GiteaMirror commented 2025-11-13 12:09:35 -06:00
Author
Owner
Copy Link
Copy Source

@clanger81 commented on GitHub (Oct 28, 2025):

@clanger81 Did you follow the community guide for setting up crowdsec as I did? https://docs.pangolin.net/self-host/community-guides/crowdsec

I did follow the community guide for crowdsec

@clanger81 commented on GitHub (Oct 28, 2025): > [@clanger81](https://github.com/clanger81) Did you follow the community guide for setting up crowdsec as I did? https://docs.pangolin.net/self-host/community-guides/crowdsec I did follow the community guide for crowdsec
GiteaMirror commented 2025-11-13 12:09:35 -06:00
Author
Owner
Copy Link
Copy Source

@scroguard commented on GitHub (Oct 30, 2025):

it happened again this morning, here is a snippet from the docker logs:

traefik | 192.168.11.8 - - [30/Oct/2025:08:00:04 +0000] "POST /api/v1/auth/newt/get-token HTTP/2.0" 502 11 "-" "-" 8378 "api-router@file" "http://pangolin:3000" 16ms
traefik | 2025-10-30T08:00:05Z ERR Provider error, retrying in 666.83104ms error="cannot fetch configuration data: do fetch request: Get "http://pangolin:3001/api/v1/traefik-config": dial tcp 172.18.0.2:3001: connect: connection refused" providerName=http
traefik | 2025-10-30T08:00:05Z ERR Provider error, retrying in 648.057315ms error="cannot fetch configuration data: do fetch request: Get "http://pangolin:3001/api/v1/traefik-config": dial tcp: lookup pangolin on 127.0.0.11:53: read udp 127.0.0.1:53251->127.0.0.11:53: read: connection refused" providerName=http
traefik | 2025-10-30T08:00:06Z ERR Provider error, retrying in 635.696754ms error="cannot fetch configuration data: do fetch request: Get "http://pangolin:3001/api/v1/traefik-config": dial tcp: lookup pangolin on 127.0.0.11:53: read udp 127.0.0.1:41181->127.0.0.11:53: read: connection refused" providerName=http
traefik | 2025-10-30T08:00:07Z ERR Provider error, retrying in 2.455847922s error="cannot fetch configuration data: do fetch request: Get "http://pangolin:3001/api/v1/traefik-config": dial tcp: lookup pangolin on 127.0.0.11:53: read udp 127.0.0.1:54831->127.0.0.11:53: read: connection refused" providerName=http
traefik | 2025-10-30T08:00:09Z ERR Provider error, retrying in 2.449075756s error="cannot fetch configuration data: do fetch request: Get "http://pangolin:3001/api/v1/traefik-config": dial tcp: lookup pangolin on 127.0.0.11:53: read udp 127.0.0.1:59354->127.0.0.11:53: read: connection refused" providerName=http
traefik | 2025-10-30T08:00:12Z ERR Provider error, retrying in 5.225875397s error="cannot fetch configuration data: do fetch request: Get "http://pangolin:3001/api/v1/traefik-config": dial tcp: lookup pangolin on 127.0.0.11:53: read udp 127.0.0.1:42410->127.0.0.11:53: read: connection refused" providerName=http
traefik | 98.160.214.233 - - [29/Oct/2025:23:58:33 +0000] "GET /:/eventsource/notifications?X-Plex-Token=FbAKo7XN_PhfZ6itiQ_R HTTP/1.0" 200 63536 "-" "-" 44 "41-plex-passthrough-router@http" "https://100.89.128.8:55372" 28900752ms
traefik | 50.193.208.177 - - [30/Oct/2025:05:12:14 +0000] "GET /notifications/hub?access_token=eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJuYmYiOjE3NjE4MDExMzQsImV4cCI6MTc2MTgwODMzNCwiaXNzIjoiaHR0cDovL2xvY2FsaG9zdHxsb2dpbiIsInN1YiI6ImE0NGQwOGYwLWU0YTAtNDdkYS04M2ViLTNiNGQyNGU1MWM2YSIsInByZW1pdW0iOnRydWUsIm5hbWUiOiJqdXN0aW4gYWxicmVjaHQiLCJlbWFpbCI6Imp1c3RpbkBqdXN0aW5hbGJyZWNodC5jb20iLCJlbWFpbF92ZXJpZmllZCI6dHJ1ZSwic3N0YW1wIjoiMDEzNDUzN2MtNThlNC00ZjY0LWI1ZGEtMDI5MThlNjdiYTllIiwiZGV2aWNlIjoiNDQ5MTkxYTAtZWExNC00ZWQyLWI0MzYtNjk4YTRjMDYzZDEyIiwiZGV2aWNldHlwZSI6IkZpcmVmb3ggRXh0ZW5zaW9uIiwiY2xpZW50X2lkIjoiYnJvd3NlciIsInNjb3BlIjpbImFwaSIsIm9mZmxpbmVfYWNjZXNzIl0sImFtciI6WyJBcHBsaWNhdGlvbiJdfQ.J7EXlPyMmQLI6P_VzTIB3FBnVWoBnNR8R9JNVRB34jJjJxi2aFLn2KkSfuvg4nsm7TK54o1r09CtD3kO_N6BdVHnCKBRcJ2KCy3NxBZnzCXhBqdwnccxkDBtuYdOZb0dxFOiFgUfnAFEMtAvTDru7azNvakjbQ9rvyZK_N-QcYh9b4KK8mf54sPh_HLZtB-BsNe8fw8GvmUptgVzSjF4l6lAmS6rWKRk4GZQ_bPwEl0E9vMjaaCn5dpQCNwaDT_8cbwdKmqk5p-aEaVAfNg_Z9ZgSKaEGCPYK5Rcu-tuLCI7msoPAyPLQBxBF-LNf0MiYH4r9k2e_L3lM4T7KfvWBQ HTTP/1.1" 0 0 "-" "-" 6399 "36-vault-albrechthome-net-router@http" "http://100.89.128.36:43256" 10080601ms
traefik | 2025-10-30T08:00:14Z INF Server stopped

as others have stated, restarting traefik fixes the issue.

for now i have created a simple bash script that restarts the traefik container every morning to hopefully avoid this issue and keep everything online until an official fix has been found.

@scroguard commented on GitHub (Oct 30, 2025): it happened again this morning, here is a snippet from the docker logs: traefik | 192.168.11.8 - - [30/Oct/2025:08:00:04 +0000] "POST /api/v1/auth/newt/get-token HTTP/2.0" 502 11 "-" "-" 8378 "api-router@file" "http://pangolin:3000" 16ms traefik | 2025-10-30T08:00:05Z ERR Provider error, retrying in 666.83104ms error="cannot fetch configuration data: do fetch request: Get \"http://pangolin:3001/api/v1/traefik-config\": dial tcp 172.18.0.2:3001: connect: connection refused" providerName=http traefik | 2025-10-30T08:00:05Z ERR Provider error, retrying in 648.057315ms error="cannot fetch configuration data: do fetch request: Get \"http://pangolin:3001/api/v1/traefik-config\": dial tcp: lookup pangolin on 127.0.0.11:53: read udp 127.0.0.1:53251->127.0.0.11:53: read: connection refused" providerName=http traefik | 2025-10-30T08:00:06Z ERR Provider error, retrying in 635.696754ms error="cannot fetch configuration data: do fetch request: Get \"http://pangolin:3001/api/v1/traefik-config\": dial tcp: lookup pangolin on 127.0.0.11:53: read udp 127.0.0.1:41181->127.0.0.11:53: read: connection refused" providerName=http traefik | 2025-10-30T08:00:07Z ERR Provider error, retrying in 2.455847922s error="cannot fetch configuration data: do fetch request: Get \"http://pangolin:3001/api/v1/traefik-config\": dial tcp: lookup pangolin on 127.0.0.11:53: read udp 127.0.0.1:54831->127.0.0.11:53: read: connection refused" providerName=http traefik | 2025-10-30T08:00:09Z ERR Provider error, retrying in 2.449075756s error="cannot fetch configuration data: do fetch request: Get \"http://pangolin:3001/api/v1/traefik-config\": dial tcp: lookup pangolin on 127.0.0.11:53: read udp 127.0.0.1:59354->127.0.0.11:53: read: connection refused" providerName=http traefik | 2025-10-30T08:00:12Z ERR Provider error, retrying in 5.225875397s error="cannot fetch configuration data: do fetch request: Get \"http://pangolin:3001/api/v1/traefik-config\": dial tcp: lookup pangolin on 127.0.0.11:53: read udp 127.0.0.1:42410->127.0.0.11:53: read: connection refused" providerName=http traefik | 98.160.214.233 - - [29/Oct/2025:23:58:33 +0000] "GET /:/eventsource/notifications?X-Plex-Token=FbAKo7XN_PhfZ6itiQ_R HTTP/1.0" 200 63536 "-" "-" 44 "41-plex-passthrough-router@http" "https://100.89.128.8:55372" 28900752ms traefik | 50.193.208.177 - - [30/Oct/2025:05:12:14 +0000] "GET /notifications/hub?access_token=eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJuYmYiOjE3NjE4MDExMzQsImV4cCI6MTc2MTgwODMzNCwiaXNzIjoiaHR0cDovL2xvY2FsaG9zdHxsb2dpbiIsInN1YiI6ImE0NGQwOGYwLWU0YTAtNDdkYS04M2ViLTNiNGQyNGU1MWM2YSIsInByZW1pdW0iOnRydWUsIm5hbWUiOiJqdXN0aW4gYWxicmVjaHQiLCJlbWFpbCI6Imp1c3RpbkBqdXN0aW5hbGJyZWNodC5jb20iLCJlbWFpbF92ZXJpZmllZCI6dHJ1ZSwic3N0YW1wIjoiMDEzNDUzN2MtNThlNC00ZjY0LWI1ZGEtMDI5MThlNjdiYTllIiwiZGV2aWNlIjoiNDQ5MTkxYTAtZWExNC00ZWQyLWI0MzYtNjk4YTRjMDYzZDEyIiwiZGV2aWNldHlwZSI6IkZpcmVmb3ggRXh0ZW5zaW9uIiwiY2xpZW50X2lkIjoiYnJvd3NlciIsInNjb3BlIjpbImFwaSIsIm9mZmxpbmVfYWNjZXNzIl0sImFtciI6WyJBcHBsaWNhdGlvbiJdfQ.J7EXlPyMmQLI6P_VzTIB3FBnVWoBnNR8R9JNVRB34jJjJxi2aFLn2KkSfuvg4nsm7TK54o1r09CtD3kO_N6BdVHnCKBRcJ2KCy3NxBZnzCXhBqdwnccxkDBtuYdOZb0dxFOiFgUfnAFEMtAvTDru7azNvakjbQ9rvyZK_N-QcYh9b4KK8mf54sPh_HLZtB-BsNe8fw8GvmUptgVzSjF4l6lAmS6rWKRk4GZQ_bPwEl0E9vMjaaCn5dpQCNwaDT_8cbwdKmqk5p-aEaVAfNg_Z9ZgSKaEGCPYK5Rcu-tuLCI7msoPAyPLQBxBF-LNf0MiYH4r9k2e_L3lM4T7KfvWBQ HTTP/1.1" 0 0 "-" "-" 6399 "36-vault-albrechthome-net-router@http" "http://100.89.128.36:43256" 10080601ms traefik | 2025-10-30T08:00:14Z INF Server stopped as others have stated, restarting traefik fixes the issue. for now i have created a simple bash script that restarts the traefik container every morning to hopefully avoid this issue and keep everything online until an official fix has been found.
GiteaMirror commented 2025-11-13 12:09:35 -06:00
Author
Owner
Copy Link
Copy Source

@HomelabHaven commented on GitHub (Oct 30, 2025):

Just FYI: The problem is not specific to traefik:v3.5 as I've been naughty and had the traefik container image set to "latest" for a while.

@HomelabHaven commented on GitHub (Oct 30, 2025): Just FYI: The problem is **not** specific to traefik:v3.5 as I've been naughty and had the traefik container image set to "latest" for a while.
GiteaMirror commented 2025-11-13 12:09:35 -06:00
Author
Owner
Copy Link
Copy Source

@clanger81 commented on GitHub (Nov 11, 2025):

Either of you have any luck sorting this out? Pangolin seems a bit more stable with subsequent releases but I'm still having this issue creep up every 4-5 days. Thinking I may need to ditch Pangolin and go back to manual Traefik configuration.

@clanger81 commented on GitHub (Nov 11, 2025): Either of you have any luck sorting this out? Pangolin seems a bit more stable with subsequent releases but I'm still having this issue creep up every 4-5 days. Thinking I may need to ditch Pangolin and go back to manual Traefik configuration.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
api
authentication
bug
config
dependencies
docker
documentation
enhancement
good first issue
help wanted
Improvement
Look Into
needs investigating
networking
new feature
non-critical bug
potential bug
pull-request
Mirrored from GitHub Pull Request
 question
reverse proxy
Security
stale
ui
wontfix
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
GiteaMirror ninjasurge (Kaehlen Rog)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/pangolin#734