[GH-ISSUE #2815] Newt unable to reconnect after interface drops #15134

Closed
opened 2026-05-16 03:27:15 -05:00 by GiteaMirror · 18 comments
Owner

Originally created by @Frost-4X on GitHub (Apr 9, 2026).
Original GitHub issue: https://github.com/fosrl/pangolin/issues/2815

Describe the Bug

I have a router that I've set to randomise the MAC address every 3 days. When that happens the interface gets dropped for a moment to apply the changes. When it comes back online, Newt fails to reconnect, and does its consecutive pings.
WARN: 2026/04/09 00:31:22 Periodic ping failed (15 consecutive failures): all 4 ping attempts failed, last error: failed to read ICMP packet: i/o timeout

I checked with to make sure it had nothing to do with the MAC randomisation, and sure enough, a simple router reboot causes Newt to go into stasis.

I have two servers running on the same network, with different Newt versions, they both go into stasis.

Environment

  • OS Type & Version: DietPi 0.6.2
  • Pangolin Version: 1.17.0 (happened on 1.15.x as well)
  • Newt Version: 1.9.0 and 1.11.0

To Reproduce

Have a working instance of pangolin, with Newt running, then restart the interface that Newt is connected to.

Expected Behavior

Newt should be able to identify the interface has probably restarted, and attempt to initiate some kind of reconnection.

Originally created by @Frost-4X on GitHub (Apr 9, 2026). Original GitHub issue: https://github.com/fosrl/pangolin/issues/2815 ### Describe the Bug I have a router that I've set to randomise the MAC address every 3 days. When that happens the interface gets dropped for a moment to apply the changes. When it comes back online, Newt fails to reconnect, and does its consecutive pings. `WARN: 2026/04/09 00:31:22 Periodic ping failed (15 consecutive failures): all 4 ping attempts failed, last error: failed to read ICMP packet: i/o timeout` I checked with to make sure it had nothing to do with the MAC randomisation, and sure enough, a simple router reboot causes Newt to go into stasis. I have two servers running on the same network, with different Newt versions, they both go into stasis. ### Environment - OS Type & Version: DietPi 0.6.2 - Pangolin Version: 1.17.0 (happened on 1.15.x as well) - Newt Version: 1.9.0 and 1.11.0 ### To Reproduce Have a working instance of pangolin, with Newt running, then restart the interface that Newt is connected to. ### Expected Behavior Newt should be able to identify the interface has probably restarted, and attempt to initiate some kind of reconnection.
GiteaMirror added the needs investigating label 2026-05-16 03:27:15 -05:00
Author
Owner

@Frost-4X commented on GitHub (Apr 9, 2026):

I've turned of my randomisation schedule, so this will affect me less. However, its still an inconvenience if my router ever gets power cycled, I lose my Pangolin connections. Then I have to connect to my server via other tunnelling services, and restart both instances of Newt.

<!-- gh-comment-id:4210677511 --> @Frost-4X commented on GitHub (Apr 9, 2026): I've turned of my randomisation schedule, so this will affect me less. However, its still an inconvenience if my router ever gets power cycled, I lose my Pangolin connections. Then I have to connect to my server via other tunnelling services, and restart both instances of Newt.
Author
Owner

@oschwartz10612 commented on GitHub (Apr 9, 2026):

Would you be able to do a power cycle and provide the whole newt log?

<!-- gh-comment-id:4210909108 --> @oschwartz10612 commented on GitHub (Apr 9, 2026): Would you be able to do a power cycle and provide the whole newt log?
Author
Owner

@Frost-4X commented on GitHub (Apr 9, 2026):

Power cycle what?

I assume you mean the router. I'll attach a log later.

<!-- gh-comment-id:4210959707 --> @Frost-4X commented on GitHub (Apr 9, 2026): Power cycle what? I assume you mean the router. I'll attach a log later.
Author
Owner

@github-actions[bot] commented on GitHub (Apr 24, 2026):

This issue has been automatically marked as stale due to 14 days of inactivity. It will be closed in 14 days if no further activity occurs.

<!-- gh-comment-id:4309620962 --> @github-actions[bot] commented on GitHub (Apr 24, 2026): This issue has been automatically marked as stale due to 14 days of inactivity. It will be closed in 14 days if no further activity occurs.
Author
Owner

@AstralDestiny commented on GitHub (Apr 24, 2026):

Curious why the mac randomization? Don't want isp to track you or something even though no matter how much time the public ip changes they have something that will always point to you as a customer and your past given ip's handed out..

As for newt put it into --log-level debug and restart and let us know what the logs say. Just know logs might contain your domain and other stuff.

<!-- gh-comment-id:4309643116 --> @AstralDestiny commented on GitHub (Apr 24, 2026): Curious why the mac randomization? Don't want isp to track you or something even though no matter how much time the public ip changes they have something that will always point to you as a customer and your past given ip's handed out.. As for newt put it into --log-level debug and restart and let us know what the logs say. Just know logs might contain your domain and other stuff.
Author
Owner

@Frost-4X commented on GitHub (Apr 24, 2026):

My router is a subnet router under another router that's connecting to the WAN. By Randomising, it makes the traffic seem like different devices, though its pretty evident where its coming from. It just something I setup once and forgot to disable. The problem has nothing to do with randomisation, that's just the trigger.

I'll try remember to get around to posting some debug logs, pretty busy at present.

<!-- gh-comment-id:4309665587 --> @Frost-4X commented on GitHub (Apr 24, 2026): My router is a subnet router under another router that's connecting to the WAN. By Randomising, it makes the traffic seem like different devices, though its pretty evident where its coming from. It just something I setup once and forgot to disable. The problem has nothing to do with randomisation, that's just the trigger. I'll try remember to get around to posting some debug logs, pretty busy at present.
Author
Owner

@dougmaitelli commented on GitHub (May 2, 2026):

I been experiencing (what I assume to be) the same issue for about 2 months now.
Everything works great, then at some point over the weeks running non-stop newt will start throwing these logs non-stop:

WARN: 2026/05/02 02:01:46 Periodic ping failed (1131 consecutive failures): all 4 ping attempts failed, last error: failed to read ICMP packet: i/o timeout

At that point I won't be able to reach any public resource from outside the network anymore.
As soon as I restart newt everything goes back to work.

I had this happen 3 times in a 2 months period now.

<!-- gh-comment-id:4362751470 --> @dougmaitelli commented on GitHub (May 2, 2026): I been experiencing (what I assume to be) the same issue for about 2 months now. Everything works great, then at some point over the weeks running non-stop newt will start throwing these logs non-stop: ``` WARN: 2026/05/02 02:01:46 Periodic ping failed (1131 consecutive failures): all 4 ping attempts failed, last error: failed to read ICMP packet: i/o timeout ``` At that point I won't be able to reach any public resource from outside the network anymore. As soon as I restart newt everything goes back to work. I had this happen 3 times in a 2 months period now.
Author
Owner

@AstralDestiny commented on GitHub (May 2, 2026):

Container bound or host bound?

<!-- gh-comment-id:4362953251 --> @AstralDestiny commented on GitHub (May 2, 2026): Container bound or host bound?
Author
Owner

@dougmaitelli commented on GitHub (May 2, 2026):

Container bound or host bound?

Not sure what you mean, but I am running newt inside docker using compose:

  newt:
    container_name: newt
    image: fosrl/newt:1.12.0
    environment:
      - PANGOLIN_ENDPOINT=https://pangolin.${DOMAIN}
      - NEWT_ID=${NEWT_ID}
      - NEWT_SECRET=${NEWT_SECRET}
    restart: unless-stopped
<!-- gh-comment-id:4363527845 --> @dougmaitelli commented on GitHub (May 2, 2026): > Container bound or host bound? Not sure what you mean, but I am running newt inside docker using compose: ``` newt: container_name: newt image: fosrl/newt:1.12.0 environment: - PANGOLIN_ENDPOINT=https://pangolin.${DOMAIN} - NEWT_ID=${NEWT_ID} - NEWT_SECRET=${NEWT_SECRET} restart: unless-stopped ```
Author
Owner

@AstralDestiny commented on GitHub (May 2, 2026):

You got any logs prior to the issue?

also can you add

services:
  newt:
    container_name: newt
    image: fosrl/newt:1.12.0
    environment:
      - PANGOLIN_ENDPOINT=https://pangolin.${DOMAIN}
      - NEWT_ID=${NEWT_ID}
      - NEWT_SECRET=${NEWT_SECRET}
      - LOG_LEVEL=debug # This 
    restart: unless-stopped

..?

<!-- gh-comment-id:4364119360 --> @AstralDestiny commented on GitHub (May 2, 2026): You got any logs prior to the issue? also can you add ```yml services: newt: container_name: newt image: fosrl/newt:1.12.0 environment: - PANGOLIN_ENDPOINT=https://pangolin.${DOMAIN} - NEWT_ID=${NEWT_ID} - NEWT_SECRET=${NEWT_SECRET} - LOG_LEVEL=debug # This restart: unless-stopped ``` ..?
Author
Owner

@dougmaitelli commented on GitHub (May 2, 2026):

You got any logs prior to the issue?

also can you add

services:
newt:
container_name: newt
image: fosrl/newt:1.12.0
environment:
- PANGOLIN_ENDPOINT=https://pangolin.${DOMAIN}
- NEWT_ID=${NEWT_ID}
- NEWT_SECRET=${NEWT_SECRET}
- LOG_LEVEL=debug # This
restart: unless-stopped

..?

I have all the logs still, but I don't see anything on them besides the normal startup messages and then followed by thousands of repeats of this same warning. I can upload them if needed but I did go over it twice already and there is nothing there.

I will add the LOG_LEVEL now and let's see how long it will take for it to fail again, last time took 4 days.

EDIT: Worth mentioning, I am running newt 1.12.0 and I did not update it yet so we can try to get the logs for the original issue without risking an update to fix it

<!-- gh-comment-id:4364650214 --> @dougmaitelli commented on GitHub (May 2, 2026): > You got any logs prior to the issue? > > also can you add > > services: > newt: > container_name: newt > image: fosrl/newt:1.12.0 > environment: > - PANGOLIN_ENDPOINT=https://pangolin.${DOMAIN} > - NEWT_ID=${NEWT_ID} > - NEWT_SECRET=${NEWT_SECRET} > - LOG_LEVEL=debug # This > restart: unless-stopped > > ..? I have all the logs still, but I don't see anything on them besides the normal startup messages and then followed by thousands of repeats of this same warning. I can upload them if needed but I did go over it twice already and there is nothing there. I will add the LOG_LEVEL now and let's see how long it will take for it to fail again, last time took 4 days. EDIT: Worth mentioning, I am running newt 1.12.0 and I did not update it yet so we can try to get the logs for the original issue without risking an update to fix it
Author
Owner

@AstralDestiny commented on GitHub (May 7, 2026):

Take it it's been happy so far then?

<!-- gh-comment-id:4399809405 --> @AstralDestiny commented on GitHub (May 7, 2026): Take it it's been happy so far then?
Author
Owner

@dougmaitelli commented on GitHub (May 7, 2026):

Take it it's been happy so far then?

So far yeah, still monitoring it every day to wait for it to happen again

<!-- gh-comment-id:4400633164 --> @dougmaitelli commented on GitHub (May 7, 2026): > Take it it's been happy so far then? So far yeah, still monitoring it every day to wait for it to happen again
Author
Owner

@dougmaitelli commented on GitHub (May 8, 2026):

Take it it's been happy so far then?

I was able to induce a failure by restarting the ISP modem. I assume that what happens is that when my ISP rotates my external IP or my external link has a downtime and newt loses connection it is never able to recover on it's own unless it is manually restarted.

DEBUG: 2026/05/08 10:22:52 Ping attempt 3/4 failed: failed to read ICMP packet: i/o timeout
DEBUG: gerbil-wireguard: 2026/05/08 10:22:54 peer(RlYV…sJDc) - Sending handshake initiation
DEBUG: gerbil-wireguard: 2026/05/08 10:22:59 peer(RlYV…sJDc) - Sending handshake initiation
DEBUG: gerbil-wireguard: 2026/05/08 10:23:04 peer(RlYV…sJDc) - Sending handshake initiation
DEBUG: 2026/05/08 10:23:09 Ping attempt 4/4 failed: failed to read ICMP packet: i/o timeout
WARN: 2026/05/08 10:23:09 Periodic ping failed (16 consecutive failures): all 4 ping attempts failed, last error: failed to read ICMP packet: i/o timeout
DEBUG: gerbil-wireguard: 2026/05/08 10:23:09 peer(RlYV…sJDc) - Sending handshake initiation
DEBUG: gerbil-wireguard: 2026/05/08 10:23:14 peer(RlYV…sJDc) - Sending handshake initiation
DEBUG: gerbil-wireguard: 2026/05/08 10:23:19 peer(RlYV…sJDc) - Sending handshake initiation
DEBUG: 2026/05/08 10:23:24 Ping attempt 1/4 failed: failed to read ICMP packet: i/o timeout
DEBUG: gerbil-wireguard: 2026/05/08 10:23:24 peer(RlYV…sJDc) - Sending handshake initiation
DEBUG: gerbil-wireguard: 2026/05/08 10:23:29 peer(RlYV…sJDc) - Sending handshake initiation
DEBUG: gerbil-wireguard: 2026/05/08 10:23:34 peer(RlYV…sJDc) - Sending handshake initiation
DEBUG: gerbil-wireguard: 2026/05/08 10:23:39 peer(RlYV…sJDc) - Sending handshake initiation
DEBUG: 2026/05/08 10:23:40 Ping attempt 2/4 failed: failed to read ICMP packet: i/o timeout
<!-- gh-comment-id:4405667312 --> @dougmaitelli commented on GitHub (May 8, 2026): > Take it it's been happy so far then? I was able to induce a failure by restarting the ISP modem. I assume that what happens is that when my ISP rotates my external IP or my external link has a downtime and newt loses connection it is never able to recover on it's own unless it is manually restarted. ``` DEBUG: 2026/05/08 10:22:52 Ping attempt 3/4 failed: failed to read ICMP packet: i/o timeout DEBUG: gerbil-wireguard: 2026/05/08 10:22:54 peer(RlYV…sJDc) - Sending handshake initiation DEBUG: gerbil-wireguard: 2026/05/08 10:22:59 peer(RlYV…sJDc) - Sending handshake initiation DEBUG: gerbil-wireguard: 2026/05/08 10:23:04 peer(RlYV…sJDc) - Sending handshake initiation DEBUG: 2026/05/08 10:23:09 Ping attempt 4/4 failed: failed to read ICMP packet: i/o timeout WARN: 2026/05/08 10:23:09 Periodic ping failed (16 consecutive failures): all 4 ping attempts failed, last error: failed to read ICMP packet: i/o timeout DEBUG: gerbil-wireguard: 2026/05/08 10:23:09 peer(RlYV…sJDc) - Sending handshake initiation DEBUG: gerbil-wireguard: 2026/05/08 10:23:14 peer(RlYV…sJDc) - Sending handshake initiation DEBUG: gerbil-wireguard: 2026/05/08 10:23:19 peer(RlYV…sJDc) - Sending handshake initiation DEBUG: 2026/05/08 10:23:24 Ping attempt 1/4 failed: failed to read ICMP packet: i/o timeout DEBUG: gerbil-wireguard: 2026/05/08 10:23:24 peer(RlYV…sJDc) - Sending handshake initiation DEBUG: gerbil-wireguard: 2026/05/08 10:23:29 peer(RlYV…sJDc) - Sending handshake initiation DEBUG: gerbil-wireguard: 2026/05/08 10:23:34 peer(RlYV…sJDc) - Sending handshake initiation DEBUG: gerbil-wireguard: 2026/05/08 10:23:39 peer(RlYV…sJDc) - Sending handshake initiation DEBUG: 2026/05/08 10:23:40 Ping attempt 2/4 failed: failed to read ICMP packet: i/o timeout ```
Author
Owner

@oschwartz10612 commented on GitHub (May 8, 2026):

Are you running the latest 1.12.4? We introduced some fixes

<!-- gh-comment-id:4408756158 --> @oschwartz10612 commented on GitHub (May 8, 2026): Are you running the latest 1.12.4? We introduced some fixes
Author
Owner

@oschwartz10612 commented on GitHub (May 8, 2026):

https://github.com/fosrl/newt/issues/284

<!-- gh-comment-id:4408759282 --> @oschwartz10612 commented on GitHub (May 8, 2026): https://github.com/fosrl/newt/issues/284
Author
Owner

@dougmaitelli commented on GitHub (May 10, 2026):

Yeah, I was keeping it on 1.12.0 in order to test / debug the behavior if necessary.
I checked the issue you linked and it looks like the same problem I have. I updated it yesterday to 1.12.4 and I could not reproduce the issue now anymore.

<!-- gh-comment-id:4416420458 --> @dougmaitelli commented on GitHub (May 10, 2026): Yeah, I was keeping it on 1.12.0 in order to test / debug the behavior if necessary. I checked the issue you linked and it looks like the same problem I have. I updated it yesterday to 1.12.4 and I could not reproduce the issue now anymore.
Author
Owner

@LaurenceJJones commented on GitHub (May 11, 2026):

Classing as complete, if you hit anymore issues please comment on the linked newt issue 👍🏻

<!-- gh-comment-id:4419353150 --> @LaurenceJJones commented on GitHub (May 11, 2026): Classing as complete, if you hit anymore issues please comment on the linked newt issue 👍🏻
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/pangolin#15134