github-starred/pangolin
2
0
Fork 0
mirror of https://github.com/fosrl/pangolin.git synced 2026-07-25 06:36:53 -05:00
Code Issues 1.6k Packages Projects Releases 79 Wiki Activity

[GH-ISSUE #2375] Newt cannot connect to the Pangolin network. #6954

New Issue
Closed
opened 2026-04-25 15:56:46 -05:00 by GiteaMirror · 9 comments
GiteaMirror commented 2026-04-25 15:56:46 -05:00
Owner
Copy Link
Copy Source

Originally created by @jack9603301 on GitHub (Jan 29, 2026).
Original GitHub issue: https://github.com/fosrl/pangolin/issues/2375

Describe the Bug

This is a very strange problem, and the following characteristics have been observed:

  1. Communication between 51820, 21820, and the target site is working normally based on socat testing.
  2. newt cannot connect.
  3. Connecting using a server works normally, even with different servers.
  4. I tested it for two days a couple of days ago and everything was working perfectly. I don't know what I changed, but now it's showing an error.

pangolin vps log(docker compose logs -f)

traefik  | 2026-01-29T16:12:39Z WRN Traefik can reject some encoded characters in the request path.When your backend is not fully compliant with [RFC 3986](https://datatracker.ietf.org/doc/html/rfc3986),it is recommended to set these options to `false` to avoid split-view situation.Refer to the documentation for more details: https://doc.traefik.io/traefik/v3.6/migrate/v3/#encoded-characters-configuration-default-values
pangolin  | 
pangolin  | > @fosrl/pangolin@0.0.0 start
pangolin  | > ENVIRONMENT=prod node dist/migrations.mjs && ENVIRONMENT=prod NODE_ENV=development node --enable-source-maps dist/server.mjs
pangolin  | 
pangolin  | Starting migrations from version 1.15.1
pangolin  | Migrations to run: 
pangolin  | All migrations completed successfully
pangolin  | 2026-01-29T16:12:26+00:00 [info]: Pangolin gathers anonymous usage data to help us better understand how the software is used and guide future improvements and feature development. You can find more details, including instructions for opting out of this anonymous data collection, at: https://docs.pangolin.net/telemetry
pangolin  | 2026-01-29T16:12:26+00:00 [info]: API server is running on http://localhost:3000
pangolin  | 2026-01-29T16:12:26+00:00 [info]: Internal server is running on http://localhost:3001
pangolin  | 2026-01-29T16:12:31+00:00 [info]: Next.js server is running on http://localhost:3002
pangolin  | 2026-01-29T16:12:38+00:00 [info]: Updated exit node reachableAt to http://gerbil:3004
pangolin  | 2026-01-29T16:12:44+00:00 [info]: Establishing websocket connection
pangolin  | 2026-01-29T16:12:44+00:00 [info]: Client added to tracking - NEWT ID: 0d0ofv150tn4ou4, Connection ID: 44834769-229c-4692-8885-ce8f197d7411, Total connections: 1, Config version: 0
pangolin  | 2026-01-29T16:12:44+00:00 [info]: WebSocket connection fully established and ready - NEWT ID: 0d0ofv150tn4ou4
pangolin  | 2026-01-29T16:12:44+00:00 [info]: Handling ping request newt message!
pangolin  | 2026-01-29T16:12:44+00:00 [info]: Sticking with previously connected node: Exit Node NXQAkMwa (0 ms), latency diff = 0.0ms / NaN%.
pangolin  | 2026-01-29T16:12:44+00:00 [info]: Adding peer with public key aH54nHQpkRfV3/FDkKp/PqQTgquE5dvT+gbSOjlmnzM= to exit node 1
pangolin  | 2026-01-29T16:13:54+00:00 [info]: All connections removed for NEWT ID: 0d0ofv150tn4ou4
pangolin  | 2026-01-29T16:13:54+00:00 [info]: Client disconnected - NEWT ID: 0d0ofv150tn4ou4
pangolin  | 2026-01-29T16:13:55+00:00 [info]: Rate limit service cleanup completed
pangolin  | 2026-01-29T16:13:55+00:00 [info]: WebSocket cleanup completed
pangolin  | npm notice
pangolin  | npm notice New minor version of npm available! 11.6.2 -> 11.8.0
pangolin  | npm notice Changelog: https://github.com/npm/cli/releases/tag/v11.8.0
pangolin  | npm notice To update run: npm install -g npm@11.8.0
pangolin  | npm notice
pangolin  | 
pangolin  | > @fosrl/pangolin@0.0.0 start
pangolin  | > ENVIRONMENT=prod node dist/migrations.mjs && ENVIRONMENT=prod NODE_ENV=development node --enable-source-maps dist/server.mjs
pangolin  | 
pangolin  | Starting migrations from version 1.15.1
pangolin  | Migrations to run: 
pangolin  | All migrations completed successfully
pangolin  | 2026-01-29T16:16:33+00:00 [info]: Pangolin gathers anonymous usage data to help us better understand how the software is used and guide future improvements and feature development. You can find more details, including instructions for opting out of this anonymous data collection, at: https://docs.pangolin.net/telemetry
pangolin  | 2026-01-29T16:16:34+00:00 [info]: API server is running on http://localhost:3000
pangolin  | 2026-01-29T16:16:34+00:00 [info]: Internal server is running on http://localhost:3001
pangolin  | 2026-01-29T16:16:38+00:00 [info]: Next.js server is running on http://localhost:3002
pangolin  | 2026-01-29T16:16:46+00:00 [info]: Updated exit node reachableAt to http://gerbil:3004
pangolin  | 2026-01-29T16:16:53+00:00 [info]: Establishing websocket connection
pangolin  | 2026-01-29T16:16:53+00:00 [info]: Client added to tracking - NEWT ID: 0d0ofv150tn4ou4, Connection ID: f7939089-a4d2-4a63-90f3-07fb6c0b60cd, Total connections: 1, Config version: 0
pangolin  | 2026-01-29T16:16:53+00:00 [info]: WebSocket connection fully established and ready - NEWT ID: 0d0ofv150tn4ou4
traefik   | 2026-01-29T16:12:39Z INF Traefik version 3.6.7 built on 2026-01-14T14:04:03Z version=3.6.7
traefik   | 2026-01-29T16:12:39Z INF Version check is enabled.
traefik   | 2026-01-29T16:12:39Z INF Traefik checks for new releases to notify you if your version is out of date.
traefik   | 2026-01-29T16:12:39Z INF It also collects usage data during this process.
traefik   | 2026-01-29T16:12:39Z INF Check the documentation to get more info: https://doc.traefik.io/traefik/contributing/data-collection/
traefik   | 2026-01-29T16:12:39Z INF 
traefik   | Stats collection is disabled.
traefik   | Help us improve Traefik by turning this feature on :)
traefik   | More details on: https://doc.traefik.io/traefik/contributing/data-collection/
traefik   | 
traefik   | 2026-01-29T16:12:39Z INF Loading plugins... plugins=["badger"]
traefik   | 2026-01-29T16:12:40Z INF Plugins loaded. plugins=["badger"]
traefik   | 2026-01-29T16:12:40Z INF Starting provider aggregator *aggregator.ProviderAggregator
traefik   | 2026-01-29T16:12:40Z INF Starting provider *file.Provider
traefik   | 2026-01-29T16:12:40Z INF Starting provider *traefik.Provider
traefik   | 2026-01-29T16:12:40Z INF Starting provider *http.Provider
traefik   | 2026-01-29T16:12:40Z INF Starting provider *acme.ChallengeTLSALPN
traefik   | 2026-01-29T16:12:40Z INF Starting provider *acme.Provider
traefik   | 2026-01-29T16:12:40Z INF Testing certificate renew... acmeCA=https://acme-v02.api.letsencrypt.org/directory providerName=letsencrypt.acme
traefik   | 2026-01-29T16:12:46Z INF Register... providerName=letsencrypt.acme
traefik   | 2026-01-29T16:12:52Z ERR Unable to obtain ACME certificate for domains error="unable to generate a certificate for the domains [dashboard.pangolin.qhjack.top]: error: one or more domains had a problem:\n[dashboard.pangolin.qhjack.top] invalid authorization: acme: error: 403 :: urn:ietf:params:acme:error:unauthorized :: 2a01:7e03::f03c:94ff:fefa:f120: Invalid response from http://dashboard.pangolin.qhjack.top/.well-known/acme-challenge/FToJdC_C6pQ7YOCrUU5IIm5vYCoDSAX8TQyuXX22o3s: 404\n" ACME CA=https://acme-v02.api.letsencrypt.org/directory acmeCA=https://acme-v02.api.letsencrypt.org/directory domains=["dashboard.pangolin.qhjack.top"] providerName=letsencrypt.acme routerName=next-router@file rule="Host(`dashboard.pangolin.qhjack.top`) && !PathPrefix(`/api/v1`)"
traefik   | 2026-01-29T16:13:44Z INF I have to go...
traefik   | 2026-01-29T16:13:44Z INF Stopping server gracefully
traefik   | 2026-01-29T16:13:54Z INF Server stopped
traefik   | 2026-01-29T16:13:54Z INF Shutting down
traefik   | 2026-01-29T16:16:47Z WRN Traefik can reject some encoded characters in the request path.When your backend is not fully compliant with [RFC 3986](https://datatracker.ietf.org/doc/html/rfc3986),it is recommended to set these options to `false` to avoid split-view situation.Refer to the documentation for more details: https://doc.traefik.io/traefik/v3.6/migrate/v3/#encoded-characters-configuration-default-values
traefik   | 2026-01-29T16:16:47Z INF Traefik version 3.6.7 built on 2026-01-14T14:04:03Z version=3.6.7
traefik   | 2026-01-29T16:16:47Z INF Version check is enabled.
traefik   | 2026-01-29T16:16:47Z INF Traefik checks for new releases to notify you if your version is out of date.
traefik   | 2026-01-29T16:16:47Z INF It also collects usage data during this process.
traefik   | 2026-01-29T16:16:47Z INF Check the documentation to get more info: https://doc.traefik.io/traefik/contributing/data-collection/
traefik   | 2026-01-29T16:16:47Z INF 
traefik   | Stats collection is disabled.
traefik   | Help us improve Traefik by turning this feature on :)
traefik   | More details on: https://doc.traefik.io/traefik/contributing/data-collection/
traefik   | 
traefik   | 2026-01-29T16:16:47Z INF Loading plugins... plugins=["badger"]
traefik   | 2026-01-29T16:16:48Z INF Plugins loaded. plugins=["badger"]
traefik   | 2026-01-29T16:16:48Z INF Starting provider aggregator *aggregator.ProviderAggregator
traefik   | 2026-01-29T16:16:48Z INF Starting provider *file.Provider
traefik   | 2026-01-29T16:16:48Z INF Starting provider *traefik.Provider
traefik   | 2026-01-29T16:16:48Z INF Starting provider *http.Provider
traefik   | 2026-01-29T16:16:48Z INF Starting provider *acme.ChallengeTLSALPN
traefik   | 2026-01-29T16:16:48Z INF Starting provider *acme.Provider
traefik   | 2026-01-29T16:16:48Z INF Testing certificate renew... acmeCA=https://acme-v02.api.letsencrypt.org/directory providerName=letsencrypt.acme
traefik   | 2026-01-29T16:16:55Z ERR Unable to obtain ACME certificate for domains error="unable to generate a certificate for the domains [dashboard.pangolin.qhjack.top]: error: one or more domains had a problem:\n[dashboard.pangolin.qhjack.top] invalid authorization: acme: error: 403 :: urn:ietf:params:acme:error:unauthorized :: 2a01:7e03::f03c:94ff:fefa:f120: Invalid response from http://dashboard.pangolin.qhjack.top/.well-known/acme-challenge/9yflhQ68xqvbAYh1Nlr4bVbuS2-oO7d1wS0ybGRPMcs: 404\n" ACME CA=https://acme-v02.api.letsencrypt.org/directory acmeCA=https://acme-v02.api.letsencrypt.org/directory domains=["dashboard.pangolin.qhjack.top"] providerName=letsencrypt.acme routerName=api-router@file rule="Host(`dashboard.pangolin.qhjack.top`) && PathPrefix(`/api/v1`)"
gerbil    | INFO: 2026/01/29 16:12:37 Fetching remote config from http://pangolin:3001/api/v1/gerbil/get-config
gerbil    | INFO: 2026/01/29 16:12:38 Created WireGuard interface wg0
gerbil    | INFO: 2026/01/29 16:12:38 Assigned IP address 100.89.128.1/24 to interface wg0
gerbil    | INFO: 2026/01/29 16:12:38 Attempting to delete existing MSS clamping rule for chain INPUT
gerbil    | INFO: 2026/01/29 16:12:38 Attempting to delete existing MSS clamping rule for chain OUTPUT
gerbil    | INFO: 2026/01/29 16:12:38 Attempting to delete existing MSS clamping rule for chain FORWARD
gerbil    | INFO: 2026/01/29 16:12:38 Adding MSS clamping rule for chain INPUT
gerbil    | INFO: 2026/01/29 16:12:38 Successfully added and verified MSS clamping rule for chain INPUT
gerbil    | INFO: 2026/01/29 16:12:38 Adding MSS clamping rule for chain OUTPUT
gerbil    | INFO: 2026/01/29 16:12:38 Successfully added and verified MSS clamping rule for chain OUTPUT
gerbil    | INFO: 2026/01/29 16:12:38 Adding MSS clamping rule for chain FORWARD
gerbil    | INFO: 2026/01/29 16:12:38 Successfully added and verified MSS clamping rule for chain FORWARD
gerbil    | INFO: 2026/01/29 16:12:38 WireGuard interface wg0 created and configured
gerbil    | INFO: 2026/01/29 16:12:38 Peer aH54nHQpkRfV3/FDkKp/PqQTgquE5dvT+gbSOjlmnzM= added successfully
gerbil    | INFO: 2026/01/29 16:12:38 Received initial mappings: {"mappings":{}}
gerbil    | INFO: 2026/01/29 16:12:38 Loaded 0 initial proxy mappings
gerbil    | INFO: 2026/01/29 16:12:38 UDP server listening on :21820
gerbil    | INFO: 2026/01/29 16:12:38 Starting HTTP server on :3004
gerbil    | INFO: 2026/01/29 16:12:44 Clearing connections for added peer with WG IP: 100.89.128.4
gerbil    | INFO: 2026/01/29 16:12:44 Cleared 0 connections for WG IP: 100.89.128.4
gerbil    | INFO: 2026/01/29 16:12:44 Peer aH54nHQpkRfV3/FDkKp/PqQTgquE5dvT+gbSOjlmnzM= added successfully
gerbil    | 2026/01/29 16:13:54 Stopping SNI Proxy...
gerbil    | 2026/01/29 16:13:54 All connections closed gracefully
gerbil    | 2026/01/29 16:13:54 SNI Proxy stopped
gerbil    | INFO: 2026/01/29 16:13:54 Context cancelled, shutting down
gerbil    | panic: close of closed channel
gerbil    | 
gerbil    | goroutine 1 [running]:
gerbil    | github.com/fosrl/gerbil/relay.(*UDPProxyServer).Stop(0xc0000e0640)
gerbil    | 	/app/relay/relay.go:237 +0x99
gerbil    | main.main()
gerbil    | 	/app/main.go:420 +0x217a
gerbil    | INFO: 2026/01/29 16:16:46 Fetching remote config from http://pangolin:3001/api/v1/gerbil/get-config
gerbil    | INFO: 2026/01/29 16:16:46 Created WireGuard interface wg0
gerbil    | INFO: 2026/01/29 16:16:46 Assigned IP address 100.89.128.1/24 to interface wg0
gerbil    | INFO: 2026/01/29 16:16:46 Attempting to delete existing MSS clamping rule for chain INPUT
gerbil    | INFO: 2026/01/29 16:16:46 Attempting to delete existing MSS clamping rule for chain OUTPUT
gerbil    | INFO: 2026/01/29 16:16:46 Attempting to delete existing MSS clamping rule for chain FORWARD
gerbil    | INFO: 2026/01/29 16:16:46 Adding MSS clamping rule for chain INPUT
gerbil    | INFO: 2026/01/29 16:16:46 Successfully added and verified MSS clamping rule for chain INPUT
gerbil    | INFO: 2026/01/29 16:16:46 Adding MSS clamping rule for chain OUTPUT
gerbil    | INFO: 2026/01/29 16:16:46 Successfully added and verified MSS clamping rule for chain OUTPUT
gerbil    | INFO: 2026/01/29 16:16:46 Adding MSS clamping rule for chain FORWARD
gerbil    | INFO: 2026/01/29 16:16:46 Successfully added and verified MSS clamping rule for chain FORWARD
gerbil    | INFO: 2026/01/29 16:16:46 WireGuard interface wg0 created and configured
gerbil    | INFO: 2026/01/29 16:16:46 Peer aH54nHQpkRfV3/FDkKp/PqQTgquE5dvT+gbSOjlmnzM= added successfully
gerbil    | INFO: 2026/01/29 16:16:46 Received initial mappings: {"mappings":{}}
gerbil    | INFO: 2026/01/29 16:16:46 Loaded 0 initial proxy mappings
gerbil    | INFO: 2026/01/29 16:16:46 UDP server listening on :21820
gerbil    | INFO: 2026/01/29 16:16:46 Starting HTTP server on :3004
pangolin  | 2026-01-29T16:17:14+00:00 [info]: All connections removed for NEWT ID: 0d0ofv150tn4ou4
pangolin  | 2026-01-29T16:17:14+00:00 [info]: Client disconnected - NEWT ID: 0d0ofv150tn4ou4
pangolin  | 2026-01-29T16:17:22+00:00 [info]: Establishing websocket connection
pangolin  | 2026-01-29T16:17:22+00:00 [info]: Client added to tracking - NEWT ID: 0d0ofv150tn4ou4, Connection ID: f17ab364-1c81-4133-b64f-f234de40df28, Total connections: 1, Config version: 0
pangolin  | 2026-01-29T16:17:22+00:00 [info]: WebSocket connection fully established and ready - NEWT ID: 0d0ofv150tn4ou4
pangolin  | 2026-01-29T16:17:23+00:00 [info]: Handling ping request newt message!
pangolin  | 2026-01-29T16:17:23+00:00 [warn]: handleGetConfigMessage: Site 1 last hole punch is too old, skipping
pangolin  | 2026-01-29T16:17:23+00:00 [info]: Sticking with previously connected node: Exit Node NXQAkMwa (0 ms), latency diff = 0.0ms / NaN%.
pangolin  | 2026-01-29T16:17:23+00:00 [info]: Public key mismatch. Deleting old peer...
pangolin  | 2026-01-29T16:17:23+00:00 [info]: Deleting peer with public key aH54nHQpkRfV3/FDkKp/PqQTgquE5dvT+gbSOjlmnzM= from exit node 1
gerbil    | INFO: 2026/01/29 16:17:23 Clearing connections for removed peer with WG IP: 100.89.128.4
gerbil    | INFO: 2026/01/29 16:17:23 Cleared 0 connections for WG IP: 100.89.128.4
gerbil    | INFO: 2026/01/29 16:17:23 Peer aH54nHQpkRfV3/FDkKp/PqQTgquE5dvT+gbSOjlmnzM= removed successfully
pangolin  | 2026-01-29T16:17:23+00:00 [info]: Adding peer with public key v02BP1H27dBZlfRgUG6uNDI8lZj13SeTkQvlqXudbX0= to exit node 1
gerbil    | INFO: 2026/01/29 16:17:23 Clearing connections for added peer with WG IP: 100.89.128.4
gerbil    | INFO: 2026/01/29 16:17:23 Cleared 0 connections for WG IP: 100.89.128.4
gerbil    | INFO: 2026/01/29 16:17:23 Peer v02BP1H27dBZlfRgUG6uNDI8lZj13SeTkQvlqXudbX0= added successfully
pangolin  | 2026-01-29T16:17:24+00:00 [info]: Sticking with previously connected node: Exit Node NXQAkMwa (0 ms), latency diff = 0.0ms / NaN%.
pangolin  | 2026-01-29T16:17:24+00:00 [info]: Adding peer with public key v02BP1H27dBZlfRgUG6uNDI8lZj13SeTkQvlqXudbX0= to exit node 1
gerbil    | INFO: 2026/01/29 16:17:24 Clearing connections for added peer with WG IP: 100.89.128.4
gerbil    | INFO: 2026/01/29 16:17:24 Cleared 0 connections for WG IP: 100.89.128.4
gerbil    | INFO: 2026/01/29 16:17:24 Peer v02BP1H27dBZlfRgUG6uNDI8lZj13SeTkQvlqXudbX0= added successfully
gerbil    | INFO: 2026/01/29 16:17:25 Cleared 0 sessions for WG IP: 10.20.245.1
gerbil    | INFO: 2026/01/29 16:17:25 Updated 0 proxy mappings: 100.89.128.4:57558 -> 100.89.128.4:59046
gerbil    | INFO: 2026/01/29 16:17:25 Cleared 0 sessions for WG IP: 10.20.245.1
gerbil    | INFO: 2026/01/29 16:17:27 Updated 0 proxy mappings: 100.89.128.4:59046 -> 100.89.128.4:59046
gerbil    | INFO: 2026/01/29 16:17:28 Cleared 0 sessions for WG IP: 10.20.245.1
gerbil    | INFO: 2026/01/29 16:17:29 Updated 0 proxy mappings: 100.89.128.4:59046 -> 100.89.128.4:59046
gerbil    | INFO: 2026/01/29 16:17:31 Updated 0 proxy mappings: 100.89.128.4:59046 -> 100.89.128.4:59046
gerbil    | INFO: 2026/01/29 16:17:32 Cleared 0 sessions for WG IP: 10.20.245.1
gerbil    | INFO: 2026/01/29 16:17:33 Updated 0 proxy mappings: 100.89.128.4:59046 -> 100.89.128.4:59046
gerbil    | INFO: 2026/01/29 16:17:35 Updated 0 proxy mappings: 100.89.128.4:59046 -> 100.89.128.4:59046
pangolin  | 2026-01-29T16:17:37+00:00 [warn]: handleGetConfigMessage: Site 1 last hole punch is too old, skipping
pangolin  | 2026-01-29T16:17:39+00:00 [warn]: handleGetConfigMessage: Site 1 last hole punch is too old, skipping
gerbil    | INFO: 2026/01/29 16:17:40 Cleared 0 sessions for WG IP: 10.20.245.1
gerbil    | INFO: 2026/01/29 16:17:41 Updated 0 proxy mappings: 100.89.128.4:59046 -> 100.89.128.4:59046
gerbil    | INFO: 2026/01/29 16:17:56 Cleared 0 sessions for WG IP: 10.20.245.1
pangolin  | (node:18) [DEP0169] DeprecationWarning: `url.parse()` behavior is not standardized and prone to errors that have security implications. Use the WHATWG URL API instead. CVEs are not issued for `url.parse()` vulnerabilities.
pangolin  | (Use `node --trace-deprecation ...` to show where the warning was created)
gerbil    | INFO: 2026/01/29 16:18:28 Cleared 0 sessions for WG IP: 10.20.245.1


pangolin  | 2026-01-29T16:20:22+00:00 [info]: Handling ping request newt message!
pangolin  | 2026-01-29T16:20:22+00:00 [info]: Sticking with previously connected node: Exit Node NXQAkMwa (0 ms), latency diff = 0.0ms / NaN%.
pangolin  | 2026-01-29T16:20:22+00:00 [info]: Adding peer with public key v02BP1H27dBZlfRgUG6uNDI8lZj13SeTkQvlqXudbX0= to exit node 1
gerbil    | INFO: 2026/01/29 16:20:22 Clearing connections for added peer with WG IP: 100.89.128.4
gerbil    | INFO: 2026/01/29 16:20:22 Cleared 0 connections for WG IP: 100.89.128.4
gerbil    | INFO: 2026/01/29 16:20:22 Peer v02BP1H27dBZlfRgUG6uNDI8lZj13SeTkQvlqXudbX0= added successfully
pangolin  | 2026-01-29T16:22:45+00:00 [info]: Handling ping request newt message!
pangolin  | 2026-01-29T16:22:45+00:00 [info]: Sticking with previously connected node: Exit Node NXQAkMwa (0 ms), latency diff = 0.0ms / NaN%.
pangolin  | 2026-01-29T16:22:45+00:00 [info]: Adding peer with public key v02BP1H27dBZlfRgUG6uNDI8lZj13SeTkQvlqXudbX0= to exit node 1
gerbil    | INFO: 2026/01/29 16:22:45 Clearing connections for added peer with WG IP: 100.89.128.4
gerbil    | INFO: 2026/01/29 16:22:45 Cleared 0 connections for WG IP: 100.89.128.4
gerbil    | INFO: 2026/01/29 16:22:45 Peer v02BP1H27dBZlfRgUG6uNDI8lZj13SeTkQvlqXudbX0= added successfully
pangolin  | 2026-01-29T16:25:07+00:00 [info]: Handling ping request newt message!
pangolin  | 2026-01-29T16:25:07+00:00 [info]: Sticking with previously connected node: Exit Node NXQAkMwa (0 ms), latency diff = 0.0ms / NaN%.
pangolin  | 2026-01-29T16:25:07+00:00 [info]: Adding peer with public key v02BP1H27dBZlfRgUG6uNDI8lZj13SeTkQvlqXudbX0= to exit node 1
gerbil    | INFO: 2026/01/29 16:25:07 Clearing connections for added peer with WG IP: 100.89.128.4
gerbil    | INFO: 2026/01/29 16:25:07 Cleared 0 connections for WG IP: 100.89.128.4
gerbil    | INFO: 2026/01/29 16:25:07 Peer v02BP1H27dBZlfRgUG6uNDI8lZj13SeTkQvlqXudbX0= added successfully

newt log:

WARN: 2026/01/30 00:16:49 Periodic ping failed (6 consecutive failures): all 4 ping attempts failed, last error: failed to read ICMP packet: i/o timeout
INFO: 2026/01/30 00:16:51 Server version: 1.15.1
INFO: 2026/01/30 00:16:53 Failed to report peer bandwidth: failed to send bandwidth data: not connected
INFO: 2026/01/30 00:16:53 Websocket connected
WARN: 2026/01/30 00:16:58 Ping attempt 23 failed: failed to read ICMP packet: i/o timeout

WARN: 2026/01/30 00:17:13 Ping attempt 24 failed: failed to read ICMP packet: i/o timeout
^CINFO: 2026/01/30 00:17:14 Closing clients...
INFO: 2026/01/30 00:17:14 Stopping direct UDP relay
INFO: 2026/01/30 00:17:14 Hole punch manager stopped
INFO: 2026/01/30 00:17:14 UDP hole punch goroutine ended for all exit nodes
INFO: 2026/01/30 00:17:14 Released shared UDP bind
INFO: 2026/01/30 00:17:14 WGTester Server stopped
INFO: 2026/01/30 00:17:14 Stopping health check monitor with 0 targets
INFO: 2026/01/30 00:17:14 Health check monitor stopped
INFO: 2026/01/30 00:17:14 Exiting...
jack@JACK-PC ~/work/ota> newt --id 0d0ofv150tn4ou4 --secret uxkjrsfa51ehegxgiueahgs92k2q5z609ur59fg3b9d9c7m5 --endpoint https://dashboard.pangolin.qhjack.top
INFO: 2026/01/30 00:17:14 Newt version 1.9.0
INFO: 2026/01/30 00:17:19 Server version: 1.15.1
INFO: 2026/01/30 00:17:23 Websocket connected
INFO: 2026/01/30 00:17:24 Connecting to endpoint: dashboard.pangolin.qhjack.top
INFO: 2026/01/30 00:17:43 SendMessageInterval timed out after 10 attempts for message type: newt/wg/get-config
WARN: 2026/01/30 00:17:55 Initial reliable ping failed, but continuing: all 5 ping attempts failed, last error: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:18:00 Ping attempt 1 failed: failed to read ICMP packet: i/o timeout
INFO: 2026/01/30 00:18:00 Stopping ping check
INFO: 2026/01/30 00:18:00 Connecting to endpoint: dashboard.pangolin.qhjack.top
WARN: 2026/01/30 00:18:00 Ping attempt 2 failed: failed to read ICMP packet: ping read: endpoint is closed for receive
WARN: 2026/01/30 00:18:00 Failed to start hole punch: hole punch already running
WARN: 2026/01/30 00:18:31 Initial reliable ping failed, but continuing: all 5 ping attempts failed, last error: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:18:36 Ping attempt 1 failed: failed to read ICMP packet: i/o timeout
INFO: 2026/01/30 00:18:36 Client connectivity setup. Ready to accept connections from clients!
ERROR: 2026/01/30 00:18:36 Failed to ensure WireGuard interface: failed to bring up WireGuard device: use of closed network connection
ERROR: 2026/01/30 00:18:36 Clients functionality will be disabled until the interface can be created
ERROR: 2026/01/30 00:18:36 Failed to ensure WireGuard interface: failed to bring up WireGuard device: SharedBind reference count went negative
ERROR: 2026/01/30 00:18:36 Clients functionality will be disabled until the interface can be created
ERROR: 2026/01/30 00:18:36 Failed to ensure WireGuard interface: failed to bring up WireGuard device: SharedBind reference count went negative
ERROR: 2026/01/30 00:18:36 Clients functionality will be disabled until the interface can be created
ERROR: 2026/01/30 00:18:36 Failed to ensure WireGuard interface: failed to bring up WireGuard device: SharedBind reference count went negative
ERROR: 2026/01/30 00:18:36 Clients functionality will be disabled until the interface can be created
ERROR: 2026/01/30 00:18:36 Failed to ensure WireGuard interface: failed to bring up WireGuard device: SharedBind reference count went negative
ERROR: 2026/01/30 00:18:36 Clients functionality will be disabled until the interface can be created
ERROR: 2026/01/30 00:18:36 Failed to ensure WireGuard interface: failed to bring up WireGuard device: SharedBind reference count went negative
ERROR: 2026/01/30 00:18:36 Clients functionality will be disabled until the interface can be created
WARN: 2026/01/30 00:18:41 Ping attempt 2 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:18:48 Ping attempt 3 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:18:55 Ping attempt 4 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:19:02 Ping attempt 5 failed: failed to read ICMP packet: i/o timeout
INFO: 2026/01/30 00:19:02 Increasing ping retry delay to 3s
WARN: 2026/01/30 00:19:10 Ping attempt 6 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:19:18 Ping attempt 7 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:19:20 Periodic ping failed (2 consecutive failures): all 2 ping attempts failed, last error: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:19:26 Ping attempt 8 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:19:34 Ping attempt 9 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:19:42 Ping attempt 10 failed: failed to read ICMP packet: i/o timeout
INFO: 2026/01/30 00:19:42 Increasing ping retry delay to 4.5s
WARN: 2026/01/30 00:19:51 Periodic ping failed (3 consecutive failures): all 2 ping attempts failed, last error: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:19:52 Ping attempt 11 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:20:01 Ping attempt 12 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:20:11 Ping attempt 13 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:20:20 Ping attempt 14 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:20:22 Periodic ping failed (4 consecutive failures): all 2 ping attempts failed, last error: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:20:22 Connection to server lost after 4 failures. Continuous reconnection attempts will be made.
INFO: 2026/01/30 00:20:22 Stopping ping check
INFO: 2026/01/30 00:20:22 Connecting to endpoint: dashboard.pangolin.qhjack.top
WARN: 2026/01/30 00:20:23 Failed to start hole punch: hole punch already running
WARN: 2026/01/30 00:20:54 Initial reliable ping failed, but continuing: all 5 ping attempts failed, last error: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:20:59 Ping attempt 1 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:21:04 Ping attempt 2 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:21:11 Ping attempt 3 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:21:18 Ping attempt 4 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:21:25 Ping attempt 5 failed: failed to read ICMP packet: i/o timeout
INFO: 2026/01/30 00:21:25 Increasing ping retry delay to 3s
WARN: 2026/01/30 00:21:33 Ping attempt 6 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:21:41 Ping attempt 7 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:21:43 Periodic ping failed (2 consecutive failures): all 2 ping attempts failed, last error: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:21:49 Ping attempt 8 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:21:57 Ping attempt 9 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:22:05 Ping attempt 10 failed: failed to read ICMP packet: i/o timeout
INFO: 2026/01/30 00:22:05 Increasing ping retry delay to 4.5s
WARN: 2026/01/30 00:22:14 Periodic ping failed (3 consecutive failures): all 2 ping attempts failed, last error: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:22:15 Ping attempt 11 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:22:24 Ping attempt 12 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:22:34 Ping attempt 13 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:22:43 Ping attempt 14 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:22:45 Periodic ping failed (4 consecutive failures): all 2 ping attempts failed, last error: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:22:45 Connection to server lost after 4 failures. Continuous reconnection attempts will be made.
INFO: 2026/01/30 00:22:45 Stopping ping check
INFO: 2026/01/30 00:22:45 Connecting to endpoint: dashboard.pangolin.qhjack.top
WARN: 2026/01/30 00:22:46 Failed to start hole punch: hole punch already running
WARN: 2026/01/30 00:23:16 Initial reliable ping failed, but continuing: all 5 ping attempts failed, last error: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:23:21 Ping attempt 1 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:23:26 Ping attempt 2 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:23:33 Ping attempt 3 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:23:40 Ping attempt 4 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:23:47 Ping attempt 5 failed: failed to read ICMP packet: i/o timeout
INFO: 2026/01/30 00:23:47 Increasing ping retry delay to 3s
WARN: 2026/01/30 00:23:55 Ping attempt 6 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:24:03 Ping attempt 7 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:24:06 Periodic ping failed (2 consecutive failures): all 2 ping attempts failed, last error: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:24:11 Ping attempt 8 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:24:19 Ping attempt 9 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:24:27 Ping attempt 10 failed: failed to read ICMP packet: i/o timeout
INFO: 2026/01/30 00:24:27 Increasing ping retry delay to 4.5s
WARN: 2026/01/30 00:24:36 Periodic ping failed (3 consecutive failures): all 2 ping attempts failed, last error: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:24:37 Ping attempt 11 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:24:46 Ping attempt 12 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:24:56 Ping attempt 13 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:25:05 Ping attempt 14 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:25:07 Periodic ping failed (4 consecutive failures): all 2 ping attempts failed, last error: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:25:07 Connection to server lost after 4 failures. Continuous reconnection attempts will be made.
INFO: 2026/01/30 00:25:07 Stopping ping check
INFO: 2026/01/30 00:25:07 Connecting to endpoint: dashboard.pangolin.qhjack.top
WARN: 2026/01/30 00:25:08 Failed to start hole punch: hole punch already running
WARN: 2026/01/30 00:25:39 Initial reliable ping failed, but continuing: all 5 ping attempts failed, last error: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:25:44 Ping attempt 1 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:25:49 Ping attempt 2 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:25:56 Ping attempt 3 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:26:03 Ping attempt 4 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/01/30 00:26:10 Ping attempt 5 failed: failed to read ICMP packet: i/o timeout
INFO: 2026/01/30 00:26:10 Increasing ping retry delay to 3s
WARN: 2026/01/30 00:26:18 Ping attempt 6 failed: failed to read ICMP packet: i/o timeout

Environment

The topology uses a two-layer proxy topology:
VPS -> NPM -> LXD (Docker) -> Pangolin

To Reproduce

The investigation is still ongoing, but he should be able to connect to the network normally. I've been testing it for two days, and the port hasn't been blocked.

Expected Behavior

Newt cannot connect to the Pangolin network.

Originally created by @jack9603301 on GitHub (Jan 29, 2026). Original GitHub issue: https://github.com/fosrl/pangolin/issues/2375 ### Describe the Bug This is a very strange problem, and the following characteristics have been observed: 1. Communication between 51820, 21820, and the target site is working normally based on socat testing. 2. newt cannot connect. 3. Connecting using a server works normally, even with different servers. 4. I tested it for two days a couple of days ago and everything was working perfectly. I don't know what I changed, but now it's showing an error. pangolin vps log(docker compose logs -f) ``` traefik | 2026-01-29T16:12:39Z WRN Traefik can reject some encoded characters in the request path.When your backend is not fully compliant with [RFC 3986](https://datatracker.ietf.org/doc/html/rfc3986),it is recommended to set these options to `false` to avoid split-view situation.Refer to the documentation for more details: https://doc.traefik.io/traefik/v3.6/migrate/v3/#encoded-characters-configuration-default-values pangolin | pangolin | > @fosrl/pangolin@0.0.0 start pangolin | > ENVIRONMENT=prod node dist/migrations.mjs && ENVIRONMENT=prod NODE_ENV=development node --enable-source-maps dist/server.mjs pangolin | pangolin | Starting migrations from version 1.15.1 pangolin | Migrations to run: pangolin | All migrations completed successfully pangolin | 2026-01-29T16:12:26+00:00 [info]: Pangolin gathers anonymous usage data to help us better understand how the software is used and guide future improvements and feature development. You can find more details, including instructions for opting out of this anonymous data collection, at: https://docs.pangolin.net/telemetry pangolin | 2026-01-29T16:12:26+00:00 [info]: API server is running on http://localhost:3000 pangolin | 2026-01-29T16:12:26+00:00 [info]: Internal server is running on http://localhost:3001 pangolin | 2026-01-29T16:12:31+00:00 [info]: Next.js server is running on http://localhost:3002 pangolin | 2026-01-29T16:12:38+00:00 [info]: Updated exit node reachableAt to http://gerbil:3004 pangolin | 2026-01-29T16:12:44+00:00 [info]: Establishing websocket connection pangolin | 2026-01-29T16:12:44+00:00 [info]: Client added to tracking - NEWT ID: 0d0ofv150tn4ou4, Connection ID: 44834769-229c-4692-8885-ce8f197d7411, Total connections: 1, Config version: 0 pangolin | 2026-01-29T16:12:44+00:00 [info]: WebSocket connection fully established and ready - NEWT ID: 0d0ofv150tn4ou4 pangolin | 2026-01-29T16:12:44+00:00 [info]: Handling ping request newt message! pangolin | 2026-01-29T16:12:44+00:00 [info]: Sticking with previously connected node: Exit Node NXQAkMwa (0 ms), latency diff = 0.0ms / NaN%. pangolin | 2026-01-29T16:12:44+00:00 [info]: Adding peer with public key aH54nHQpkRfV3/FDkKp/PqQTgquE5dvT+gbSOjlmnzM= to exit node 1 pangolin | 2026-01-29T16:13:54+00:00 [info]: All connections removed for NEWT ID: 0d0ofv150tn4ou4 pangolin | 2026-01-29T16:13:54+00:00 [info]: Client disconnected - NEWT ID: 0d0ofv150tn4ou4 pangolin | 2026-01-29T16:13:55+00:00 [info]: Rate limit service cleanup completed pangolin | 2026-01-29T16:13:55+00:00 [info]: WebSocket cleanup completed pangolin | npm notice pangolin | npm notice New minor version of npm available! 11.6.2 -> 11.8.0 pangolin | npm notice Changelog: https://github.com/npm/cli/releases/tag/v11.8.0 pangolin | npm notice To update run: npm install -g npm@11.8.0 pangolin | npm notice pangolin | pangolin | > @fosrl/pangolin@0.0.0 start pangolin | > ENVIRONMENT=prod node dist/migrations.mjs && ENVIRONMENT=prod NODE_ENV=development node --enable-source-maps dist/server.mjs pangolin | pangolin | Starting migrations from version 1.15.1 pangolin | Migrations to run: pangolin | All migrations completed successfully pangolin | 2026-01-29T16:16:33+00:00 [info]: Pangolin gathers anonymous usage data to help us better understand how the software is used and guide future improvements and feature development. You can find more details, including instructions for opting out of this anonymous data collection, at: https://docs.pangolin.net/telemetry pangolin | 2026-01-29T16:16:34+00:00 [info]: API server is running on http://localhost:3000 pangolin | 2026-01-29T16:16:34+00:00 [info]: Internal server is running on http://localhost:3001 pangolin | 2026-01-29T16:16:38+00:00 [info]: Next.js server is running on http://localhost:3002 pangolin | 2026-01-29T16:16:46+00:00 [info]: Updated exit node reachableAt to http://gerbil:3004 pangolin | 2026-01-29T16:16:53+00:00 [info]: Establishing websocket connection pangolin | 2026-01-29T16:16:53+00:00 [info]: Client added to tracking - NEWT ID: 0d0ofv150tn4ou4, Connection ID: f7939089-a4d2-4a63-90f3-07fb6c0b60cd, Total connections: 1, Config version: 0 pangolin | 2026-01-29T16:16:53+00:00 [info]: WebSocket connection fully established and ready - NEWT ID: 0d0ofv150tn4ou4 traefik | 2026-01-29T16:12:39Z INF Traefik version 3.6.7 built on 2026-01-14T14:04:03Z version=3.6.7 traefik | 2026-01-29T16:12:39Z INF Version check is enabled. traefik | 2026-01-29T16:12:39Z INF Traefik checks for new releases to notify you if your version is out of date. traefik | 2026-01-29T16:12:39Z INF It also collects usage data during this process. traefik | 2026-01-29T16:12:39Z INF Check the documentation to get more info: https://doc.traefik.io/traefik/contributing/data-collection/ traefik | 2026-01-29T16:12:39Z INF traefik | Stats collection is disabled. traefik | Help us improve Traefik by turning this feature on :) traefik | More details on: https://doc.traefik.io/traefik/contributing/data-collection/ traefik | traefik | 2026-01-29T16:12:39Z INF Loading plugins... plugins=["badger"] traefik | 2026-01-29T16:12:40Z INF Plugins loaded. plugins=["badger"] traefik | 2026-01-29T16:12:40Z INF Starting provider aggregator *aggregator.ProviderAggregator traefik | 2026-01-29T16:12:40Z INF Starting provider *file.Provider traefik | 2026-01-29T16:12:40Z INF Starting provider *traefik.Provider traefik | 2026-01-29T16:12:40Z INF Starting provider *http.Provider traefik | 2026-01-29T16:12:40Z INF Starting provider *acme.ChallengeTLSALPN traefik | 2026-01-29T16:12:40Z INF Starting provider *acme.Provider traefik | 2026-01-29T16:12:40Z INF Testing certificate renew... acmeCA=https://acme-v02.api.letsencrypt.org/directory providerName=letsencrypt.acme traefik | 2026-01-29T16:12:46Z INF Register... providerName=letsencrypt.acme traefik | 2026-01-29T16:12:52Z ERR Unable to obtain ACME certificate for domains error="unable to generate a certificate for the domains [dashboard.pangolin.qhjack.top]: error: one or more domains had a problem:\n[dashboard.pangolin.qhjack.top] invalid authorization: acme: error: 403 :: urn:ietf:params:acme:error:unauthorized :: 2a01:7e03::f03c:94ff:fefa:f120: Invalid response from http://dashboard.pangolin.qhjack.top/.well-known/acme-challenge/FToJdC_C6pQ7YOCrUU5IIm5vYCoDSAX8TQyuXX22o3s: 404\n" ACME CA=https://acme-v02.api.letsencrypt.org/directory acmeCA=https://acme-v02.api.letsencrypt.org/directory domains=["dashboard.pangolin.qhjack.top"] providerName=letsencrypt.acme routerName=next-router@file rule="Host(`dashboard.pangolin.qhjack.top`) && !PathPrefix(`/api/v1`)" traefik | 2026-01-29T16:13:44Z INF I have to go... traefik | 2026-01-29T16:13:44Z INF Stopping server gracefully traefik | 2026-01-29T16:13:54Z INF Server stopped traefik | 2026-01-29T16:13:54Z INF Shutting down traefik | 2026-01-29T16:16:47Z WRN Traefik can reject some encoded characters in the request path.When your backend is not fully compliant with [RFC 3986](https://datatracker.ietf.org/doc/html/rfc3986),it is recommended to set these options to `false` to avoid split-view situation.Refer to the documentation for more details: https://doc.traefik.io/traefik/v3.6/migrate/v3/#encoded-characters-configuration-default-values traefik | 2026-01-29T16:16:47Z INF Traefik version 3.6.7 built on 2026-01-14T14:04:03Z version=3.6.7 traefik | 2026-01-29T16:16:47Z INF Version check is enabled. traefik | 2026-01-29T16:16:47Z INF Traefik checks for new releases to notify you if your version is out of date. traefik | 2026-01-29T16:16:47Z INF It also collects usage data during this process. traefik | 2026-01-29T16:16:47Z INF Check the documentation to get more info: https://doc.traefik.io/traefik/contributing/data-collection/ traefik | 2026-01-29T16:16:47Z INF traefik | Stats collection is disabled. traefik | Help us improve Traefik by turning this feature on :) traefik | More details on: https://doc.traefik.io/traefik/contributing/data-collection/ traefik | traefik | 2026-01-29T16:16:47Z INF Loading plugins... plugins=["badger"] traefik | 2026-01-29T16:16:48Z INF Plugins loaded. plugins=["badger"] traefik | 2026-01-29T16:16:48Z INF Starting provider aggregator *aggregator.ProviderAggregator traefik | 2026-01-29T16:16:48Z INF Starting provider *file.Provider traefik | 2026-01-29T16:16:48Z INF Starting provider *traefik.Provider traefik | 2026-01-29T16:16:48Z INF Starting provider *http.Provider traefik | 2026-01-29T16:16:48Z INF Starting provider *acme.ChallengeTLSALPN traefik | 2026-01-29T16:16:48Z INF Starting provider *acme.Provider traefik | 2026-01-29T16:16:48Z INF Testing certificate renew... acmeCA=https://acme-v02.api.letsencrypt.org/directory providerName=letsencrypt.acme traefik | 2026-01-29T16:16:55Z ERR Unable to obtain ACME certificate for domains error="unable to generate a certificate for the domains [dashboard.pangolin.qhjack.top]: error: one or more domains had a problem:\n[dashboard.pangolin.qhjack.top] invalid authorization: acme: error: 403 :: urn:ietf:params:acme:error:unauthorized :: 2a01:7e03::f03c:94ff:fefa:f120: Invalid response from http://dashboard.pangolin.qhjack.top/.well-known/acme-challenge/9yflhQ68xqvbAYh1Nlr4bVbuS2-oO7d1wS0ybGRPMcs: 404\n" ACME CA=https://acme-v02.api.letsencrypt.org/directory acmeCA=https://acme-v02.api.letsencrypt.org/directory domains=["dashboard.pangolin.qhjack.top"] providerName=letsencrypt.acme routerName=api-router@file rule="Host(`dashboard.pangolin.qhjack.top`) && PathPrefix(`/api/v1`)" gerbil | INFO: 2026/01/29 16:12:37 Fetching remote config from http://pangolin:3001/api/v1/gerbil/get-config gerbil | INFO: 2026/01/29 16:12:38 Created WireGuard interface wg0 gerbil | INFO: 2026/01/29 16:12:38 Assigned IP address 100.89.128.1/24 to interface wg0 gerbil | INFO: 2026/01/29 16:12:38 Attempting to delete existing MSS clamping rule for chain INPUT gerbil | INFO: 2026/01/29 16:12:38 Attempting to delete existing MSS clamping rule for chain OUTPUT gerbil | INFO: 2026/01/29 16:12:38 Attempting to delete existing MSS clamping rule for chain FORWARD gerbil | INFO: 2026/01/29 16:12:38 Adding MSS clamping rule for chain INPUT gerbil | INFO: 2026/01/29 16:12:38 Successfully added and verified MSS clamping rule for chain INPUT gerbil | INFO: 2026/01/29 16:12:38 Adding MSS clamping rule for chain OUTPUT gerbil | INFO: 2026/01/29 16:12:38 Successfully added and verified MSS clamping rule for chain OUTPUT gerbil | INFO: 2026/01/29 16:12:38 Adding MSS clamping rule for chain FORWARD gerbil | INFO: 2026/01/29 16:12:38 Successfully added and verified MSS clamping rule for chain FORWARD gerbil | INFO: 2026/01/29 16:12:38 WireGuard interface wg0 created and configured gerbil | INFO: 2026/01/29 16:12:38 Peer aH54nHQpkRfV3/FDkKp/PqQTgquE5dvT+gbSOjlmnzM= added successfully gerbil | INFO: 2026/01/29 16:12:38 Received initial mappings: {"mappings":{}} gerbil | INFO: 2026/01/29 16:12:38 Loaded 0 initial proxy mappings gerbil | INFO: 2026/01/29 16:12:38 UDP server listening on :21820 gerbil | INFO: 2026/01/29 16:12:38 Starting HTTP server on :3004 gerbil | INFO: 2026/01/29 16:12:44 Clearing connections for added peer with WG IP: 100.89.128.4 gerbil | INFO: 2026/01/29 16:12:44 Cleared 0 connections for WG IP: 100.89.128.4 gerbil | INFO: 2026/01/29 16:12:44 Peer aH54nHQpkRfV3/FDkKp/PqQTgquE5dvT+gbSOjlmnzM= added successfully gerbil | 2026/01/29 16:13:54 Stopping SNI Proxy... gerbil | 2026/01/29 16:13:54 All connections closed gracefully gerbil | 2026/01/29 16:13:54 SNI Proxy stopped gerbil | INFO: 2026/01/29 16:13:54 Context cancelled, shutting down gerbil | panic: close of closed channel gerbil | gerbil | goroutine 1 [running]: gerbil | github.com/fosrl/gerbil/relay.(*UDPProxyServer).Stop(0xc0000e0640) gerbil | /app/relay/relay.go:237 +0x99 gerbil | main.main() gerbil | /app/main.go:420 +0x217a gerbil | INFO: 2026/01/29 16:16:46 Fetching remote config from http://pangolin:3001/api/v1/gerbil/get-config gerbil | INFO: 2026/01/29 16:16:46 Created WireGuard interface wg0 gerbil | INFO: 2026/01/29 16:16:46 Assigned IP address 100.89.128.1/24 to interface wg0 gerbil | INFO: 2026/01/29 16:16:46 Attempting to delete existing MSS clamping rule for chain INPUT gerbil | INFO: 2026/01/29 16:16:46 Attempting to delete existing MSS clamping rule for chain OUTPUT gerbil | INFO: 2026/01/29 16:16:46 Attempting to delete existing MSS clamping rule for chain FORWARD gerbil | INFO: 2026/01/29 16:16:46 Adding MSS clamping rule for chain INPUT gerbil | INFO: 2026/01/29 16:16:46 Successfully added and verified MSS clamping rule for chain INPUT gerbil | INFO: 2026/01/29 16:16:46 Adding MSS clamping rule for chain OUTPUT gerbil | INFO: 2026/01/29 16:16:46 Successfully added and verified MSS clamping rule for chain OUTPUT gerbil | INFO: 2026/01/29 16:16:46 Adding MSS clamping rule for chain FORWARD gerbil | INFO: 2026/01/29 16:16:46 Successfully added and verified MSS clamping rule for chain FORWARD gerbil | INFO: 2026/01/29 16:16:46 WireGuard interface wg0 created and configured gerbil | INFO: 2026/01/29 16:16:46 Peer aH54nHQpkRfV3/FDkKp/PqQTgquE5dvT+gbSOjlmnzM= added successfully gerbil | INFO: 2026/01/29 16:16:46 Received initial mappings: {"mappings":{}} gerbil | INFO: 2026/01/29 16:16:46 Loaded 0 initial proxy mappings gerbil | INFO: 2026/01/29 16:16:46 UDP server listening on :21820 gerbil | INFO: 2026/01/29 16:16:46 Starting HTTP server on :3004 pangolin | 2026-01-29T16:17:14+00:00 [info]: All connections removed for NEWT ID: 0d0ofv150tn4ou4 pangolin | 2026-01-29T16:17:14+00:00 [info]: Client disconnected - NEWT ID: 0d0ofv150tn4ou4 pangolin | 2026-01-29T16:17:22+00:00 [info]: Establishing websocket connection pangolin | 2026-01-29T16:17:22+00:00 [info]: Client added to tracking - NEWT ID: 0d0ofv150tn4ou4, Connection ID: f17ab364-1c81-4133-b64f-f234de40df28, Total connections: 1, Config version: 0 pangolin | 2026-01-29T16:17:22+00:00 [info]: WebSocket connection fully established and ready - NEWT ID: 0d0ofv150tn4ou4 pangolin | 2026-01-29T16:17:23+00:00 [info]: Handling ping request newt message! pangolin | 2026-01-29T16:17:23+00:00 [warn]: handleGetConfigMessage: Site 1 last hole punch is too old, skipping pangolin | 2026-01-29T16:17:23+00:00 [info]: Sticking with previously connected node: Exit Node NXQAkMwa (0 ms), latency diff = 0.0ms / NaN%. pangolin | 2026-01-29T16:17:23+00:00 [info]: Public key mismatch. Deleting old peer... pangolin | 2026-01-29T16:17:23+00:00 [info]: Deleting peer with public key aH54nHQpkRfV3/FDkKp/PqQTgquE5dvT+gbSOjlmnzM= from exit node 1 gerbil | INFO: 2026/01/29 16:17:23 Clearing connections for removed peer with WG IP: 100.89.128.4 gerbil | INFO: 2026/01/29 16:17:23 Cleared 0 connections for WG IP: 100.89.128.4 gerbil | INFO: 2026/01/29 16:17:23 Peer aH54nHQpkRfV3/FDkKp/PqQTgquE5dvT+gbSOjlmnzM= removed successfully pangolin | 2026-01-29T16:17:23+00:00 [info]: Adding peer with public key v02BP1H27dBZlfRgUG6uNDI8lZj13SeTkQvlqXudbX0= to exit node 1 gerbil | INFO: 2026/01/29 16:17:23 Clearing connections for added peer with WG IP: 100.89.128.4 gerbil | INFO: 2026/01/29 16:17:23 Cleared 0 connections for WG IP: 100.89.128.4 gerbil | INFO: 2026/01/29 16:17:23 Peer v02BP1H27dBZlfRgUG6uNDI8lZj13SeTkQvlqXudbX0= added successfully pangolin | 2026-01-29T16:17:24+00:00 [info]: Sticking with previously connected node: Exit Node NXQAkMwa (0 ms), latency diff = 0.0ms / NaN%. pangolin | 2026-01-29T16:17:24+00:00 [info]: Adding peer with public key v02BP1H27dBZlfRgUG6uNDI8lZj13SeTkQvlqXudbX0= to exit node 1 gerbil | INFO: 2026/01/29 16:17:24 Clearing connections for added peer with WG IP: 100.89.128.4 gerbil | INFO: 2026/01/29 16:17:24 Cleared 0 connections for WG IP: 100.89.128.4 gerbil | INFO: 2026/01/29 16:17:24 Peer v02BP1H27dBZlfRgUG6uNDI8lZj13SeTkQvlqXudbX0= added successfully gerbil | INFO: 2026/01/29 16:17:25 Cleared 0 sessions for WG IP: 10.20.245.1 gerbil | INFO: 2026/01/29 16:17:25 Updated 0 proxy mappings: 100.89.128.4:57558 -> 100.89.128.4:59046 gerbil | INFO: 2026/01/29 16:17:25 Cleared 0 sessions for WG IP: 10.20.245.1 gerbil | INFO: 2026/01/29 16:17:27 Updated 0 proxy mappings: 100.89.128.4:59046 -> 100.89.128.4:59046 gerbil | INFO: 2026/01/29 16:17:28 Cleared 0 sessions for WG IP: 10.20.245.1 gerbil | INFO: 2026/01/29 16:17:29 Updated 0 proxy mappings: 100.89.128.4:59046 -> 100.89.128.4:59046 gerbil | INFO: 2026/01/29 16:17:31 Updated 0 proxy mappings: 100.89.128.4:59046 -> 100.89.128.4:59046 gerbil | INFO: 2026/01/29 16:17:32 Cleared 0 sessions for WG IP: 10.20.245.1 gerbil | INFO: 2026/01/29 16:17:33 Updated 0 proxy mappings: 100.89.128.4:59046 -> 100.89.128.4:59046 gerbil | INFO: 2026/01/29 16:17:35 Updated 0 proxy mappings: 100.89.128.4:59046 -> 100.89.128.4:59046 pangolin | 2026-01-29T16:17:37+00:00 [warn]: handleGetConfigMessage: Site 1 last hole punch is too old, skipping pangolin | 2026-01-29T16:17:39+00:00 [warn]: handleGetConfigMessage: Site 1 last hole punch is too old, skipping gerbil | INFO: 2026/01/29 16:17:40 Cleared 0 sessions for WG IP: 10.20.245.1 gerbil | INFO: 2026/01/29 16:17:41 Updated 0 proxy mappings: 100.89.128.4:59046 -> 100.89.128.4:59046 gerbil | INFO: 2026/01/29 16:17:56 Cleared 0 sessions for WG IP: 10.20.245.1 pangolin | (node:18) [DEP0169] DeprecationWarning: `url.parse()` behavior is not standardized and prone to errors that have security implications. Use the WHATWG URL API instead. CVEs are not issued for `url.parse()` vulnerabilities. pangolin | (Use `node --trace-deprecation ...` to show where the warning was created) gerbil | INFO: 2026/01/29 16:18:28 Cleared 0 sessions for WG IP: 10.20.245.1 pangolin | 2026-01-29T16:20:22+00:00 [info]: Handling ping request newt message! pangolin | 2026-01-29T16:20:22+00:00 [info]: Sticking with previously connected node: Exit Node NXQAkMwa (0 ms), latency diff = 0.0ms / NaN%. pangolin | 2026-01-29T16:20:22+00:00 [info]: Adding peer with public key v02BP1H27dBZlfRgUG6uNDI8lZj13SeTkQvlqXudbX0= to exit node 1 gerbil | INFO: 2026/01/29 16:20:22 Clearing connections for added peer with WG IP: 100.89.128.4 gerbil | INFO: 2026/01/29 16:20:22 Cleared 0 connections for WG IP: 100.89.128.4 gerbil | INFO: 2026/01/29 16:20:22 Peer v02BP1H27dBZlfRgUG6uNDI8lZj13SeTkQvlqXudbX0= added successfully pangolin | 2026-01-29T16:22:45+00:00 [info]: Handling ping request newt message! pangolin | 2026-01-29T16:22:45+00:00 [info]: Sticking with previously connected node: Exit Node NXQAkMwa (0 ms), latency diff = 0.0ms / NaN%. pangolin | 2026-01-29T16:22:45+00:00 [info]: Adding peer with public key v02BP1H27dBZlfRgUG6uNDI8lZj13SeTkQvlqXudbX0= to exit node 1 gerbil | INFO: 2026/01/29 16:22:45 Clearing connections for added peer with WG IP: 100.89.128.4 gerbil | INFO: 2026/01/29 16:22:45 Cleared 0 connections for WG IP: 100.89.128.4 gerbil | INFO: 2026/01/29 16:22:45 Peer v02BP1H27dBZlfRgUG6uNDI8lZj13SeTkQvlqXudbX0= added successfully pangolin | 2026-01-29T16:25:07+00:00 [info]: Handling ping request newt message! pangolin | 2026-01-29T16:25:07+00:00 [info]: Sticking with previously connected node: Exit Node NXQAkMwa (0 ms), latency diff = 0.0ms / NaN%. pangolin | 2026-01-29T16:25:07+00:00 [info]: Adding peer with public key v02BP1H27dBZlfRgUG6uNDI8lZj13SeTkQvlqXudbX0= to exit node 1 gerbil | INFO: 2026/01/29 16:25:07 Clearing connections for added peer with WG IP: 100.89.128.4 gerbil | INFO: 2026/01/29 16:25:07 Cleared 0 connections for WG IP: 100.89.128.4 gerbil | INFO: 2026/01/29 16:25:07 Peer v02BP1H27dBZlfRgUG6uNDI8lZj13SeTkQvlqXudbX0= added successfully ``` newt log: ``` WARN: 2026/01/30 00:16:49 Periodic ping failed (6 consecutive failures): all 4 ping attempts failed, last error: failed to read ICMP packet: i/o timeout INFO: 2026/01/30 00:16:51 Server version: 1.15.1 INFO: 2026/01/30 00:16:53 Failed to report peer bandwidth: failed to send bandwidth data: not connected INFO: 2026/01/30 00:16:53 Websocket connected WARN: 2026/01/30 00:16:58 Ping attempt 23 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:17:13 Ping attempt 24 failed: failed to read ICMP packet: i/o timeout ^CINFO: 2026/01/30 00:17:14 Closing clients... INFO: 2026/01/30 00:17:14 Stopping direct UDP relay INFO: 2026/01/30 00:17:14 Hole punch manager stopped INFO: 2026/01/30 00:17:14 UDP hole punch goroutine ended for all exit nodes INFO: 2026/01/30 00:17:14 Released shared UDP bind INFO: 2026/01/30 00:17:14 WGTester Server stopped INFO: 2026/01/30 00:17:14 Stopping health check monitor with 0 targets INFO: 2026/01/30 00:17:14 Health check monitor stopped INFO: 2026/01/30 00:17:14 Exiting... jack@JACK-PC ~/work/ota> newt --id 0d0ofv150tn4ou4 --secret uxkjrsfa51ehegxgiueahgs92k2q5z609ur59fg3b9d9c7m5 --endpoint https://dashboard.pangolin.qhjack.top INFO: 2026/01/30 00:17:14 Newt version 1.9.0 INFO: 2026/01/30 00:17:19 Server version: 1.15.1 INFO: 2026/01/30 00:17:23 Websocket connected INFO: 2026/01/30 00:17:24 Connecting to endpoint: dashboard.pangolin.qhjack.top INFO: 2026/01/30 00:17:43 SendMessageInterval timed out after 10 attempts for message type: newt/wg/get-config WARN: 2026/01/30 00:17:55 Initial reliable ping failed, but continuing: all 5 ping attempts failed, last error: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:18:00 Ping attempt 1 failed: failed to read ICMP packet: i/o timeout INFO: 2026/01/30 00:18:00 Stopping ping check INFO: 2026/01/30 00:18:00 Connecting to endpoint: dashboard.pangolin.qhjack.top WARN: 2026/01/30 00:18:00 Ping attempt 2 failed: failed to read ICMP packet: ping read: endpoint is closed for receive WARN: 2026/01/30 00:18:00 Failed to start hole punch: hole punch already running WARN: 2026/01/30 00:18:31 Initial reliable ping failed, but continuing: all 5 ping attempts failed, last error: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:18:36 Ping attempt 1 failed: failed to read ICMP packet: i/o timeout INFO: 2026/01/30 00:18:36 Client connectivity setup. Ready to accept connections from clients! ERROR: 2026/01/30 00:18:36 Failed to ensure WireGuard interface: failed to bring up WireGuard device: use of closed network connection ERROR: 2026/01/30 00:18:36 Clients functionality will be disabled until the interface can be created ERROR: 2026/01/30 00:18:36 Failed to ensure WireGuard interface: failed to bring up WireGuard device: SharedBind reference count went negative ERROR: 2026/01/30 00:18:36 Clients functionality will be disabled until the interface can be created ERROR: 2026/01/30 00:18:36 Failed to ensure WireGuard interface: failed to bring up WireGuard device: SharedBind reference count went negative ERROR: 2026/01/30 00:18:36 Clients functionality will be disabled until the interface can be created ERROR: 2026/01/30 00:18:36 Failed to ensure WireGuard interface: failed to bring up WireGuard device: SharedBind reference count went negative ERROR: 2026/01/30 00:18:36 Clients functionality will be disabled until the interface can be created ERROR: 2026/01/30 00:18:36 Failed to ensure WireGuard interface: failed to bring up WireGuard device: SharedBind reference count went negative ERROR: 2026/01/30 00:18:36 Clients functionality will be disabled until the interface can be created ERROR: 2026/01/30 00:18:36 Failed to ensure WireGuard interface: failed to bring up WireGuard device: SharedBind reference count went negative ERROR: 2026/01/30 00:18:36 Clients functionality will be disabled until the interface can be created WARN: 2026/01/30 00:18:41 Ping attempt 2 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:18:48 Ping attempt 3 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:18:55 Ping attempt 4 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:19:02 Ping attempt 5 failed: failed to read ICMP packet: i/o timeout INFO: 2026/01/30 00:19:02 Increasing ping retry delay to 3s WARN: 2026/01/30 00:19:10 Ping attempt 6 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:19:18 Ping attempt 7 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:19:20 Periodic ping failed (2 consecutive failures): all 2 ping attempts failed, last error: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:19:26 Ping attempt 8 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:19:34 Ping attempt 9 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:19:42 Ping attempt 10 failed: failed to read ICMP packet: i/o timeout INFO: 2026/01/30 00:19:42 Increasing ping retry delay to 4.5s WARN: 2026/01/30 00:19:51 Periodic ping failed (3 consecutive failures): all 2 ping attempts failed, last error: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:19:52 Ping attempt 11 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:20:01 Ping attempt 12 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:20:11 Ping attempt 13 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:20:20 Ping attempt 14 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:20:22 Periodic ping failed (4 consecutive failures): all 2 ping attempts failed, last error: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:20:22 Connection to server lost after 4 failures. Continuous reconnection attempts will be made. INFO: 2026/01/30 00:20:22 Stopping ping check INFO: 2026/01/30 00:20:22 Connecting to endpoint: dashboard.pangolin.qhjack.top WARN: 2026/01/30 00:20:23 Failed to start hole punch: hole punch already running WARN: 2026/01/30 00:20:54 Initial reliable ping failed, but continuing: all 5 ping attempts failed, last error: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:20:59 Ping attempt 1 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:21:04 Ping attempt 2 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:21:11 Ping attempt 3 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:21:18 Ping attempt 4 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:21:25 Ping attempt 5 failed: failed to read ICMP packet: i/o timeout INFO: 2026/01/30 00:21:25 Increasing ping retry delay to 3s WARN: 2026/01/30 00:21:33 Ping attempt 6 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:21:41 Ping attempt 7 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:21:43 Periodic ping failed (2 consecutive failures): all 2 ping attempts failed, last error: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:21:49 Ping attempt 8 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:21:57 Ping attempt 9 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:22:05 Ping attempt 10 failed: failed to read ICMP packet: i/o timeout INFO: 2026/01/30 00:22:05 Increasing ping retry delay to 4.5s WARN: 2026/01/30 00:22:14 Periodic ping failed (3 consecutive failures): all 2 ping attempts failed, last error: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:22:15 Ping attempt 11 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:22:24 Ping attempt 12 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:22:34 Ping attempt 13 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:22:43 Ping attempt 14 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:22:45 Periodic ping failed (4 consecutive failures): all 2 ping attempts failed, last error: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:22:45 Connection to server lost after 4 failures. Continuous reconnection attempts will be made. INFO: 2026/01/30 00:22:45 Stopping ping check INFO: 2026/01/30 00:22:45 Connecting to endpoint: dashboard.pangolin.qhjack.top WARN: 2026/01/30 00:22:46 Failed to start hole punch: hole punch already running WARN: 2026/01/30 00:23:16 Initial reliable ping failed, but continuing: all 5 ping attempts failed, last error: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:23:21 Ping attempt 1 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:23:26 Ping attempt 2 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:23:33 Ping attempt 3 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:23:40 Ping attempt 4 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:23:47 Ping attempt 5 failed: failed to read ICMP packet: i/o timeout INFO: 2026/01/30 00:23:47 Increasing ping retry delay to 3s WARN: 2026/01/30 00:23:55 Ping attempt 6 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:24:03 Ping attempt 7 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:24:06 Periodic ping failed (2 consecutive failures): all 2 ping attempts failed, last error: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:24:11 Ping attempt 8 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:24:19 Ping attempt 9 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:24:27 Ping attempt 10 failed: failed to read ICMP packet: i/o timeout INFO: 2026/01/30 00:24:27 Increasing ping retry delay to 4.5s WARN: 2026/01/30 00:24:36 Periodic ping failed (3 consecutive failures): all 2 ping attempts failed, last error: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:24:37 Ping attempt 11 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:24:46 Ping attempt 12 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:24:56 Ping attempt 13 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:25:05 Ping attempt 14 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:25:07 Periodic ping failed (4 consecutive failures): all 2 ping attempts failed, last error: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:25:07 Connection to server lost after 4 failures. Continuous reconnection attempts will be made. INFO: 2026/01/30 00:25:07 Stopping ping check INFO: 2026/01/30 00:25:07 Connecting to endpoint: dashboard.pangolin.qhjack.top WARN: 2026/01/30 00:25:08 Failed to start hole punch: hole punch already running WARN: 2026/01/30 00:25:39 Initial reliable ping failed, but continuing: all 5 ping attempts failed, last error: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:25:44 Ping attempt 1 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:25:49 Ping attempt 2 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:25:56 Ping attempt 3 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:26:03 Ping attempt 4 failed: failed to read ICMP packet: i/o timeout WARN: 2026/01/30 00:26:10 Ping attempt 5 failed: failed to read ICMP packet: i/o timeout INFO: 2026/01/30 00:26:10 Increasing ping retry delay to 3s WARN: 2026/01/30 00:26:18 Ping attempt 6 failed: failed to read ICMP packet: i/o timeout ``` ### Environment The topology uses a two-layer proxy topology: VPS -> NPM -> LXD (Docker) -> Pangolin ### To Reproduce The investigation is still ongoing, but he should be able to connect to the network normally. I've been testing it for two days, and the port hasn't been blocked. ### Expected Behavior Newt cannot connect to the Pangolin network.
GiteaMirror added the stale label 2026-04-25 15:56:46 -05:00
GiteaMirror commented 2026-04-25 15:56:47 -05:00
Author
Owner
Copy Link
Copy Source

@jack9603301 commented on GitHub (Jan 29, 2026):

I reinstalled it, simplified the topology, and changed two ports. Now it can connect to the network via newt, but the downside is that another problem has arisen: the client cannot connect.

Log:

[warn]: Client last hole punch is too old and we have sites to send; skipping this register

<!-- gh-comment-id:3819988244 --> @jack9603301 commented on GitHub (Jan 29, 2026): I reinstalled it, simplified the topology, and changed two ports. Now it can connect to the network via newt, but the downside is that another problem has arisen: the client cannot connect. Log: [warn]: Client last hole punch is too old and we have sites to send; skipping this register
GiteaMirror commented 2026-04-25 15:56:48 -05:00
Author
Owner
Copy Link
Copy Source

@bazziebaz commented on GitHub (Feb 7, 2026):

Having similar issues on Oracle free VPS. All security list are rules in place (TCP 80+443 UDP 51820+21820), netcat tests from newt docker container all ok, but no luck.

Now, did this test: on VPS security list: open up all IPs and all protocols -> SUCCESS! As if there is one more undocumented port which is also needed? After site is up I can remove the all-rule and it seems to keep on working.

<!-- gh-comment-id:3864392433 --> @bazziebaz commented on GitHub (Feb 7, 2026): Having similar issues on Oracle free VPS. All security list are rules in place (TCP 80+443 UDP 51820+21820), netcat tests from newt docker container all ok, but no luck. Now, did this test: on VPS security list: open up all IPs and all protocols -> SUCCESS! As if there is one more undocumented port which is also needed? After site is up I can remove the all-rule and it seems to keep on working.
GiteaMirror commented 2026-04-25 15:56:48 -05:00
Author
Owner
Copy Link
Copy Source

@geek1o commented on GitHub (Feb 8, 2026):

I have exactly the same problem as bazziebaz, the only thing that helps is completely disabling the firewall, although the necessary ports are open.

<!-- gh-comment-id:3868413941 --> @geek1o commented on GitHub (Feb 8, 2026): I have exactly the same problem as [bazziebaz](https://github.com/bazziebaz), the only thing that helps is completely disabling the firewall, although the necessary ports are open.
GiteaMirror commented 2026-04-25 15:56:49 -05:00
Author
Owner
Copy Link
Copy Source

@Xerionty commented on GitHub (Feb 10, 2026):

Came here because I experience the same issue.

INFO: 2026/02/10 11:43:32 Server version: 1.15.2
INFO: 2026/02/10 11:43:32 Websocket connected
INFO: 2026/02/10 11:43:32 Connecting to endpoint: [removed]
INFO: 2026/02/10 11:43:52 SendMessageInterval timed out after 10 attempts for message type: newt/wg/get-config
WARN: 2026/02/10 11:44:03 Initial reliable ping failed, but continuing: all 5 ping attempts failed, last error: failed to read ICMP packet: i/o timeout
WARN: 2026/02/10 11:44:08 Ping attempt 1 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/02/10 11:44:13 Ping attempt 2 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/02/10 11:44:20 Ping attempt 3 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/02/10 11:44:27 Ping attempt 4 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/02/10 11:44:34 Ping attempt 5 failed: failed to read ICMP packet: i/o timeout
INFO: 2026/02/10 11:44:34 Increasing ping retry delay to 3s
WARN: 2026/02/10 11:44:42 Ping attempt 6 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/02/10 11:44:50 Ping attempt 7 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/02/10 11:44:52 Periodic ping failed (2 consecutive failures): all 2 ping attempts failed, last error: failed to read ICMP packet: i/o timeout
WARN: 2026/02/10 11:44:58 Ping attempt 8 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/02/10 11:45:06 Ping attempt 9 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/02/10 11:45:14 Ping attempt 10 failed: failed to read ICMP packet: i/o timeout
INFO: 2026/02/10 11:45:14 Increasing ping retry delay to 4.5s


gerbil    | INFO: 2026/02/10 10:43:06 Peer [removed]= removed successfully
pangolin  | 2026-02-10T10:43:06+00:00 [info]: Exit node request successful: {"method":"DELETE","url":"http://gerbil:3004/peer?public_key=[removed]%3D","status":"Peer removed successfully"}
pangolin  | 2026-02-10T10:43:06+00:00 [info]: Adding peer with public key [removed] to exit node 1
gerbil    | INFO: 2026/02/10 10:43:06 Clearing connections for added peer with WG IP: 100.89.128.4
gerbil    | INFO: 2026/02/10 10:43:06 Cleared 0 connections for WG IP: 100.89.128.4
gerbil    | INFO: 2026/02/10 10:43:06 Peer [removed] added successfully
pangolin  | 2026-02-10T10:43:06+00:00 [info]: Exit node request successful: {"method":"POST","url":"http://gerbil:3004/peer","status":"Peer added successfully"}
pangolin  | 2026-02-10T10:43:17+00:00 [info]: All connections removed for NEWT ID: j0jto96hgfc5zng
pangolin  | 2026-02-10T10:43:17+00:00 [info]: Client disconnected - NEWT ID: j0jto96hgfc5zng
pangolin  | 2026-02-10T10:43:32+00:00 [info]: Establishing websocket connection
pangolin  | 2026-02-10T10:43:32+00:00 [info]: Client added to tracking - NEWT ID: j0jto96hgfc5zng, Connection ID: 5ee73c83-3e76-4bae-b4fc-d50198dd649e, Total connections: 1
pangolin  | 2026-02-10T10:43:32+00:00 [info]: WebSocket connection established - NEWT ID: j0jto96hgfc5zng
pangolin  | 2026-02-10T10:43:32+00:00 [info]: Handling ping request newt message!
pangolin  | 2026-02-10T10:43:32+00:00 [info]: Public key mismatch. Deleting old peer...
pangolin  | 2026-02-10T10:43:32+00:00 [info]: Deleting peer with public key [removed] from exit node 1
gerbil    | INFO: 2026/02/10 10:43:32 Clearing connections for removed peer with WG IP: 100.89.128.4
gerbil    | INFO: 2026/02/10 10:43:32 Cleared 0 connections for WG IP: 100.89.128.4
gerbil    | INFO: 2026/02/10 10:43:32 Peer [removed] removed successfully
pangolin  | 2026-02-10T10:43:32+00:00 [info]: Exit node request successful: {"method":"DELETE","url":"http://gerbil:3004/peer?public_key=[removed]","status":"Peer removed successfully"}
pangolin  | 2026-02-10T10:43:32+00:00 [info]: Adding peer with public key [removed] to exit node 1
gerbil    | INFO: 2026/02/10 10:43:32 Clearing connections for added peer with WG IP: 100.89.128.4
gerbil    | INFO: 2026/02/10 10:43:32 Cleared 0 connections for WG IP: 100.89.128.4
gerbil    | INFO: 2026/02/10 10:43:32 Peer [removed] added successfully
pangolin  | 2026-02-10T10:43:32+00:00 [info]: Exit node request successful: {"method":"POST","url":"http://gerbil:3004/peer","status":"Peer added successfully"}
pangolin  | 2026-02-10T10:45:53+00:00 [info]: Handling ping request newt message!```
<!-- gh-comment-id:3876845728 --> @Xerionty commented on GitHub (Feb 10, 2026): Came here because I experience the same issue. ```INFO: 2026/02/10 11:43:32 Newt version 1.9.0 INFO: 2026/02/10 11:43:32 Server version: 1.15.2 INFO: 2026/02/10 11:43:32 Websocket connected INFO: 2026/02/10 11:43:32 Connecting to endpoint: [removed] INFO: 2026/02/10 11:43:52 SendMessageInterval timed out after 10 attempts for message type: newt/wg/get-config WARN: 2026/02/10 11:44:03 Initial reliable ping failed, but continuing: all 5 ping attempts failed, last error: failed to read ICMP packet: i/o timeout WARN: 2026/02/10 11:44:08 Ping attempt 1 failed: failed to read ICMP packet: i/o timeout WARN: 2026/02/10 11:44:13 Ping attempt 2 failed: failed to read ICMP packet: i/o timeout WARN: 2026/02/10 11:44:20 Ping attempt 3 failed: failed to read ICMP packet: i/o timeout WARN: 2026/02/10 11:44:27 Ping attempt 4 failed: failed to read ICMP packet: i/o timeout WARN: 2026/02/10 11:44:34 Ping attempt 5 failed: failed to read ICMP packet: i/o timeout INFO: 2026/02/10 11:44:34 Increasing ping retry delay to 3s WARN: 2026/02/10 11:44:42 Ping attempt 6 failed: failed to read ICMP packet: i/o timeout WARN: 2026/02/10 11:44:50 Ping attempt 7 failed: failed to read ICMP packet: i/o timeout WARN: 2026/02/10 11:44:52 Periodic ping failed (2 consecutive failures): all 2 ping attempts failed, last error: failed to read ICMP packet: i/o timeout WARN: 2026/02/10 11:44:58 Ping attempt 8 failed: failed to read ICMP packet: i/o timeout WARN: 2026/02/10 11:45:06 Ping attempt 9 failed: failed to read ICMP packet: i/o timeout WARN: 2026/02/10 11:45:14 Ping attempt 10 failed: failed to read ICMP packet: i/o timeout INFO: 2026/02/10 11:45:14 Increasing ping retry delay to 4.5s gerbil | INFO: 2026/02/10 10:43:06 Peer [removed]= removed successfully pangolin | 2026-02-10T10:43:06+00:00 [info]: Exit node request successful: {"method":"DELETE","url":"http://gerbil:3004/peer?public_key=[removed]%3D","status":"Peer removed successfully"} pangolin | 2026-02-10T10:43:06+00:00 [info]: Adding peer with public key [removed] to exit node 1 gerbil | INFO: 2026/02/10 10:43:06 Clearing connections for added peer with WG IP: 100.89.128.4 gerbil | INFO: 2026/02/10 10:43:06 Cleared 0 connections for WG IP: 100.89.128.4 gerbil | INFO: 2026/02/10 10:43:06 Peer [removed] added successfully pangolin | 2026-02-10T10:43:06+00:00 [info]: Exit node request successful: {"method":"POST","url":"http://gerbil:3004/peer","status":"Peer added successfully"} pangolin | 2026-02-10T10:43:17+00:00 [info]: All connections removed for NEWT ID: j0jto96hgfc5zng pangolin | 2026-02-10T10:43:17+00:00 [info]: Client disconnected - NEWT ID: j0jto96hgfc5zng pangolin | 2026-02-10T10:43:32+00:00 [info]: Establishing websocket connection pangolin | 2026-02-10T10:43:32+00:00 [info]: Client added to tracking - NEWT ID: j0jto96hgfc5zng, Connection ID: 5ee73c83-3e76-4bae-b4fc-d50198dd649e, Total connections: 1 pangolin | 2026-02-10T10:43:32+00:00 [info]: WebSocket connection established - NEWT ID: j0jto96hgfc5zng pangolin | 2026-02-10T10:43:32+00:00 [info]: Handling ping request newt message! pangolin | 2026-02-10T10:43:32+00:00 [info]: Public key mismatch. Deleting old peer... pangolin | 2026-02-10T10:43:32+00:00 [info]: Deleting peer with public key [removed] from exit node 1 gerbil | INFO: 2026/02/10 10:43:32 Clearing connections for removed peer with WG IP: 100.89.128.4 gerbil | INFO: 2026/02/10 10:43:32 Cleared 0 connections for WG IP: 100.89.128.4 gerbil | INFO: 2026/02/10 10:43:32 Peer [removed] removed successfully pangolin | 2026-02-10T10:43:32+00:00 [info]: Exit node request successful: {"method":"DELETE","url":"http://gerbil:3004/peer?public_key=[removed]","status":"Peer removed successfully"} pangolin | 2026-02-10T10:43:32+00:00 [info]: Adding peer with public key [removed] to exit node 1 gerbil | INFO: 2026/02/10 10:43:32 Clearing connections for added peer with WG IP: 100.89.128.4 gerbil | INFO: 2026/02/10 10:43:32 Cleared 0 connections for WG IP: 100.89.128.4 gerbil | INFO: 2026/02/10 10:43:32 Peer [removed] added successfully pangolin | 2026-02-10T10:43:32+00:00 [info]: Exit node request successful: {"method":"POST","url":"http://gerbil:3004/peer","status":"Peer added successfully"} pangolin | 2026-02-10T10:45:53+00:00 [info]: Handling ping request newt message!```
GiteaMirror commented 2026-04-25 15:56:49 -05:00
Author
Owner
Copy Link
Copy Source

@jack9603301 commented on GitHub (Feb 11, 2026):

I am testing on another server that is located in the same country as me, so I think it should be normal. Let me test it. Perhaps the previous error was caused by interference with UDP traffic or poor network quality conditions

<!-- gh-comment-id:3885599938 --> @jack9603301 commented on GitHub (Feb 11, 2026): I am testing on another server that is located in the same country as me, so I think it should be normal. Let me test it. Perhaps the previous error was caused by interference with UDP traffic or poor network quality conditions
GiteaMirror commented 2026-04-25 15:56:49 -05:00
Author
Owner
Copy Link
Copy Source

@jack9603301 commented on GitHub (Feb 11, 2026):

I am testing on another server that is located in the same country as me, so I think it should be normal. Let me test it. Perhaps the previous error was caused by interference with UDP traffic or poor network quality conditions

<!-- gh-comment-id:3885600056 --> @jack9603301 commented on GitHub (Feb 11, 2026): I am testing on another server that is located in the same country as me, so I think it should be normal. Let me test it. Perhaps the previous error was caused by interference with UDP traffic or poor network quality conditions
GiteaMirror commented 2026-04-25 15:56:50 -05:00
Author
Owner
Copy Link
Copy Source

@schtinkafinga commented on GitHub (Feb 14, 2026):

Having similar issues on Oracle free VPS. All security list are rules in place (TCP 80+443 UDP 51820+21820), netcat tests from newt docker container all ok, but no luck.

Now, did this test: on VPS security list: open up all IPs and all protocols -> SUCCESS! As if there is one more undocumented port which is also needed? After site is up I can remove the all-rule and it seems to keep on working.

In Cloud console you have to set your firewall ingress to allow all UDP ports

<!-- gh-comment-id:3901715025 --> @schtinkafinga commented on GitHub (Feb 14, 2026): > Having similar issues on Oracle free VPS. All security list are rules in place (TCP 80+443 UDP 51820+21820), netcat tests from newt docker container all ok, but no luck. > > Now, did this test: on VPS security list: open up all IPs and all protocols -> SUCCESS! As if there is one more undocumented port which is also needed? After site is up I can remove the all-rule and it seems to keep on working. In Cloud console you have to set your firewall ingress to allow all UDP ports
GiteaMirror commented 2026-04-25 15:56:50 -05:00
Author
Owner
Copy Link
Copy Source

@github-actions[bot] commented on GitHub (Mar 1, 2026):

This issue has been automatically marked as stale due to 14 days of inactivity. It will be closed in 14 days if no further activity occurs.

<!-- gh-comment-id:3978725052 --> @github-actions[bot] commented on GitHub (Mar 1, 2026): This issue has been automatically marked as stale due to 14 days of inactivity. It will be closed in 14 days if no further activity occurs.
GiteaMirror commented 2026-04-25 15:56:50 -05:00
Author
Owner
Copy Link
Copy Source

@github-actions[bot] commented on GitHub (Mar 15, 2026):

This issue has been automatically closed due to inactivity. If you believe this is still relevant, please open a new issue with up-to-date information.

<!-- gh-comment-id:4061824732 --> @github-actions[bot] commented on GitHub (Mar 15, 2026): This issue has been automatically closed due to inactivity. If you believe this is still relevant, please open a new issue with up-to-date information.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
api
authentication
bug
config
dependencies
docker
documentation
enhancement
good first issue
help wanted
Improvement
Look Into
needs investigating
networking
new feature
non-critical bug
potential bug
pull-request
Mirrored from GitHub Pull Request
 question
reverse proxy
Security
stale
ui
wontfix
No labels stale
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
GiteaMirror ninjasurge (Kaehlen Rog)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/pangolin#6954