[GH-ISSUE #2631] Memory leak in Node.js server caused by rapid UDP endpoint roaming (LTE/CGNAT) #10962

Closed
opened 2026-05-06 15:38:39 -05:00 by GiteaMirror · 5 comments
Owner

Originally created by @cmmrandau on GitHub (Mar 11, 2026).
Original GitHub issue: https://github.com/fosrl/pangolin/issues/2631

Describe the Bug

The Pangolin Node.js process is experiencing a memory leak, eventually consuming almost all available host RAM (reached 5.8 GiB in my case).

The issue appears to be tied to endpoint roaming/port changes. I have a newt client running on an LTE network in a Docker container. Because of the cellular provider's aggressive CGNAT, the UDP connection drops exactly every 60 seconds, and the client reconnects on a new port.

While WireGuard natively handles this endpoint roaming fine, the Pangolin server appears to be holding onto every single dead connection state instead of garbage-collecting them. Over 5 days, at 1,440 port changes a day, the Node process memory slowly balloons until the system runs out of RAM.

(please forgive me for using AI to write the report)

Environment

  • OS Type & Version: Ubuntu 24.04
  • Pangolin Version: 1.16.2 EE
  • Gerbil Version: 1.30
  • Traefik Version: 3.69
  • Newt Version:1.10.2

To Reproduce

Run a newt client behind an aggressive CGNAT or LTE connection that forces a port change frequently (or artificially force the client to change its source port every 60 seconds).

Monitor the Pangolin server logs; observe the constant [info]: Site X endpoint changed from IP:PORT to IP:NEW_PORT messages.

On the newt client side: lots of INFO: 2026/03/11 01:10:25 TCP Forwarder: Successfully connected to 172.18.0.7:3553, starting bidirectional copy

Monitor the Pangolin node process RAM usage via top or btop.

Observe the memory usage steadily climbing without ever releasing.

Expected Behavior

When a WireGuard endpoint roams and changes its port, Pangolin should update the tunnel's active endpoint and immediately discard/garbage-collect the old connection state from memory.

Originally created by @cmmrandau on GitHub (Mar 11, 2026). Original GitHub issue: https://github.com/fosrl/pangolin/issues/2631 ### Describe the Bug The Pangolin Node.js process is experiencing a memory leak, eventually consuming almost all available host RAM (reached 5.8 GiB in my case). The issue appears to be tied to endpoint roaming/port changes. I have a newt client running on an LTE network in a Docker container. Because of the cellular provider's aggressive CGNAT, the UDP connection drops exactly every 60 seconds, and the client reconnects on a new port. While WireGuard natively handles this endpoint roaming fine, the Pangolin server appears to be holding onto every single dead connection state instead of garbage-collecting them. Over 5 days, at 1,440 port changes a day, the Node process memory slowly balloons until the system runs out of RAM. (please forgive me for using AI to write the report) ### Environment - OS Type & Version: Ubuntu 24.04 - Pangolin Version: 1.16.2 EE - Gerbil Version: 1.30 - Traefik Version: 3.69 - Newt Version:1.10.2 ### To Reproduce Run a newt client behind an aggressive CGNAT or LTE connection that forces a port change frequently (or artificially force the client to change its source port every 60 seconds). Monitor the Pangolin server logs; observe the constant [info]: Site X endpoint changed from IP:PORT to IP:NEW_PORT messages. On the newt client side: lots of INFO: 2026/03/11 01:10:25 TCP Forwarder: Successfully connected to 172.18.0.7:3553, starting bidirectional copy Monitor the Pangolin node process RAM usage via top or btop. Observe the memory usage steadily climbing without ever releasing. ### Expected Behavior When a WireGuard endpoint roams and changes its port, Pangolin should update the tunnel's active endpoint and immediately discard/garbage-collect the old connection state from memory.
GiteaMirror added the stale label 2026-05-06 15:38:39 -05:00
Author
Owner

@cmmrandau commented on GitHub (Mar 11, 2026):

Image
<!-- gh-comment-id:4035520976 --> @cmmrandau commented on GitHub (Mar 11, 2026): <img width="1920" height="1200" alt="Image" src="https://github.com/user-attachments/assets/d2f8f6b4-e869-4e24-9460-dd392738c90c" />
Author
Owner

@xylcro commented on GitHub (Mar 12, 2026):

This may be related to #2120

<!-- gh-comment-id:4049501873 --> @xylcro commented on GitHub (Mar 12, 2026): This may be related to #2120
Author
Owner

@cmmrandau commented on GitHub (Mar 12, 2026):

Yes I've seen that. I've removed the CGNAT site as well as traefik dashboard and crowdsec manager to see if the "vanilla" pangolin suite also leaks memory, in which case it's likely the same bug. Never seen it before, though, before I added the CGNAT site and the log spam began.

<!-- gh-comment-id:4049869506 --> @cmmrandau commented on GitHub (Mar 12, 2026): Yes I've seen that. I've removed the CGNAT site as well as traefik dashboard and crowdsec manager to see if the "vanilla" pangolin suite also leaks memory, in which case it's likely the same bug. Never seen it before, though, before I added the CGNAT site and the log spam began.
Author
Owner

@github-actions[bot] commented on GitHub (Mar 27, 2026):

This issue has been automatically marked as stale due to 14 days of inactivity. It will be closed in 14 days if no further activity occurs.

<!-- gh-comment-id:4139240772 --> @github-actions[bot] commented on GitHub (Mar 27, 2026): This issue has been automatically marked as stale due to 14 days of inactivity. It will be closed in 14 days if no further activity occurs.
Author
Owner

@github-actions[bot] commented on GitHub (Apr 10, 2026):

This issue has been automatically closed due to inactivity. If you believe this is still relevant, please open a new issue with up-to-date information.

<!-- gh-comment-id:4218926806 --> @github-actions[bot] commented on GitHub (Apr 10, 2026): This issue has been automatically closed due to inactivity. If you believe this is still relevant, please open a new issue with up-to-date information.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/pangolin#10962