[GH-ISSUE #1724] Unable to create Sites #10753

Closed
opened 2026-05-06 14:53:02 -05:00 by GiteaMirror · 32 comments
Owner

Originally created by @Lokowitz on GitHub (Oct 21, 2025).
Original GitHub issue: https://github.com/fosrl/pangolin/issues/1724

Describe the Bug

Notices today that i am not able to create a new site anymore.
Newt and Basic WG are grayed out not selectable, just Local.
So sounds like a connection problem to Gerbil. But my existing Sites and Resources are working still fine.
Maybe something related to the Gerbil default port change?

Only interesting from the logs, Gerbil is spamming all the time:

ERROR: 2025/10/21 16:08:28 No proxy mapping found for 92.210.193.252:17407
ERROR: 2025/10/21 16:08:33 No proxy mapping found for 92.210.193.252:17407
ERROR: 2025/10/21 16:08:38 No proxy mapping found for 92.210.193.252:17407
ERROR: 2025/10/21 16:08:43 No proxy mapping found for 92.210.193.252:17407
ERROR: 2025/10/21 16:08:49 No proxy mapping found for 92.210.193.252:17407
ERROR: 2025/10/21 16:08:54 No proxy mapping found for 92.210.193.252:17407
ERROR: 2025/10/21 16:08:59 No proxy mapping found for 92.210.193.252:17407

Pangolin looks good:

> @fosrl/pangolin@0.0.0 start
> ENVIRONMENT=prod node dist/migrations.mjs && ENVIRONMENT=prod NODE_ENV=development node --enable-source-maps dist/server.mjs

Starting migrations from version 1.11.1
Migrations to run:
All migrations completed successfully
2025-10-21T13:50:26+02:00 [info]: Started offline checker interval
2025-10-21T13:50:26+02:00 [warn]: Email SMTP configuration is missing. Emails will not be sent.
2025-10-21T13:50:27+02:00 [warn]: Server admin exists. Setup token generation skipped.
2025-10-21T13:50:27+02:00 [info]: Pangolin now gathers anonymous usage data to help us better understand how the software is used and guide future improvements and feature development. You can find more details, including instructions for opting out of this anonymous data collection, at: https://docs.pangolin.net/telemetry
2025-10-21T13:50:27+02:00 [info]: API server is running on http://localhost:3000
2025-10-21T13:50:27+02:00 [info]: Internal server is running on http://localhost:3001
2025-10-21T13:50:29+02:00 [info]: Next.js server is running on http://localhost:3002
2025-10-21T13:50:33+02:00 [info]: Updated exit node with reachableAt to http://gerbil:3004
2025-10-21T13:50:37+02:00 [info]: Establishing websocket connection
2025-10-21T13:50:37+02:00 [info]: Client added to tracking - NEWT ID: <HIDE>, Connection ID: df4fa1e0-89c2-4fc8-ba64-70733742156c, Total connections: 1
2025-10-21T13:50:37+02:00 [info]: WebSocket connection established - NEWT ID: <HIDE>
2025-10-21T13:50:37+02:00 [info]: Establishing websocket connection
2025-10-21T13:50:37+02:00 [info]: Client added to tracking - OLM ID: <HIDE>, Connection ID: 9e16fac2-9143-48db-853c-7baa4ffed7e1, Total connections: 1
2025-10-21T13:50:37+02:00 [info]: WebSocket connection established - OLM ID: <HIDE>
2025-10-21T13:50:37+02:00 [info]: Establishing websocket connection
2025-10-21T13:50:37+02:00 [info]: Client added to tracking - NEWT ID: 89fyx1zrf1hsmio, Connection ID: f00d4994-3792-4042-98d8-642de4bfba58, Total connections: 1
2025-10-21T13:50:37+02:00 [info]: WebSocket connection established - NEWT ID: <HIDE>
2025-10-21T13:50:38+02:00 [info]: Handling ping request newt message!
2025-10-21T13:50:39+02:00 [info]: Adding peer with public key <HIDE> to exit node 1
2025-10-21T13:50:39+02:00 [info]: Exit node request successful: {"method":"POST","url":"http://gerbil:3004/peer","status":"Peer added successfully"}
2025-10-21T13:50:39+02:00 [info]: Handling healthcheck status message
2025-10-21T13:50:39+02:00 [info]: Handling ping request newt message!
2025-10-21T13:50:39+02:00 [info]: Adding peer with public key <HIDE> to exit node 1
2025-10-21T13:50:39+02:00 [info]: Exit node request successful: {"method":"POST","url":"http://gerbil:3004/peer","status":"Peer added successfully"}
2025-10-21T13:50:40+02:00 [info]: Handling healthcheck status message
2025-10-21T13:50:45+02:00 [info]: Establishing websocket connection
2025-10-21T13:50:45+02:00 [info]: Client added to tracking - NEWT ID: <HIDE>, Connection ID: 85569e1f-7553-4b34-9d0b-55bf7c37e9ce, Total connections: 1
2025-10-21T13:50:45+02:00 [info]: WebSocket connection established - NEWT ID: <HIDE>

Environment

  • OS Type & Version: Debian 12
  • Pangolin Version: 1.11.1
  • Gerbil Version: 1.2.2
  • Traefik Version: 3.5
  • Newt Version: 1.5.2

To Reproduce

Not sure if this can be reproduced easily. My Setup is just regulary upgraded since beta phase.

Expected Behavior

Able to add new Sites

Originally created by @Lokowitz on GitHub (Oct 21, 2025). Original GitHub issue: https://github.com/fosrl/pangolin/issues/1724 ### Describe the Bug Notices today that i am not able to create a new site anymore. `Newt` and `Basic WG` are grayed out not selectable, just `Local`. So sounds like a connection problem to Gerbil. But my existing Sites and Resources are working still fine. Maybe something related to the Gerbil default port change? Only interesting from the logs, Gerbil is spamming all the time: ``` ERROR: 2025/10/21 16:08:28 No proxy mapping found for 92.210.193.252:17407 ERROR: 2025/10/21 16:08:33 No proxy mapping found for 92.210.193.252:17407 ERROR: 2025/10/21 16:08:38 No proxy mapping found for 92.210.193.252:17407 ERROR: 2025/10/21 16:08:43 No proxy mapping found for 92.210.193.252:17407 ERROR: 2025/10/21 16:08:49 No proxy mapping found for 92.210.193.252:17407 ERROR: 2025/10/21 16:08:54 No proxy mapping found for 92.210.193.252:17407 ERROR: 2025/10/21 16:08:59 No proxy mapping found for 92.210.193.252:17407 ``` Pangolin looks good: ``` > @fosrl/pangolin@0.0.0 start > ENVIRONMENT=prod node dist/migrations.mjs && ENVIRONMENT=prod NODE_ENV=development node --enable-source-maps dist/server.mjs Starting migrations from version 1.11.1 Migrations to run: All migrations completed successfully 2025-10-21T13:50:26+02:00 [info]: Started offline checker interval 2025-10-21T13:50:26+02:00 [warn]: Email SMTP configuration is missing. Emails will not be sent. 2025-10-21T13:50:27+02:00 [warn]: Server admin exists. Setup token generation skipped. 2025-10-21T13:50:27+02:00 [info]: Pangolin now gathers anonymous usage data to help us better understand how the software is used and guide future improvements and feature development. You can find more details, including instructions for opting out of this anonymous data collection, at: https://docs.pangolin.net/telemetry 2025-10-21T13:50:27+02:00 [info]: API server is running on http://localhost:3000 2025-10-21T13:50:27+02:00 [info]: Internal server is running on http://localhost:3001 2025-10-21T13:50:29+02:00 [info]: Next.js server is running on http://localhost:3002 2025-10-21T13:50:33+02:00 [info]: Updated exit node with reachableAt to http://gerbil:3004 2025-10-21T13:50:37+02:00 [info]: Establishing websocket connection 2025-10-21T13:50:37+02:00 [info]: Client added to tracking - NEWT ID: <HIDE>, Connection ID: df4fa1e0-89c2-4fc8-ba64-70733742156c, Total connections: 1 2025-10-21T13:50:37+02:00 [info]: WebSocket connection established - NEWT ID: <HIDE> 2025-10-21T13:50:37+02:00 [info]: Establishing websocket connection 2025-10-21T13:50:37+02:00 [info]: Client added to tracking - OLM ID: <HIDE>, Connection ID: 9e16fac2-9143-48db-853c-7baa4ffed7e1, Total connections: 1 2025-10-21T13:50:37+02:00 [info]: WebSocket connection established - OLM ID: <HIDE> 2025-10-21T13:50:37+02:00 [info]: Establishing websocket connection 2025-10-21T13:50:37+02:00 [info]: Client added to tracking - NEWT ID: 89fyx1zrf1hsmio, Connection ID: f00d4994-3792-4042-98d8-642de4bfba58, Total connections: 1 2025-10-21T13:50:37+02:00 [info]: WebSocket connection established - NEWT ID: <HIDE> 2025-10-21T13:50:38+02:00 [info]: Handling ping request newt message! 2025-10-21T13:50:39+02:00 [info]: Adding peer with public key <HIDE> to exit node 1 2025-10-21T13:50:39+02:00 [info]: Exit node request successful: {"method":"POST","url":"http://gerbil:3004/peer","status":"Peer added successfully"} 2025-10-21T13:50:39+02:00 [info]: Handling healthcheck status message 2025-10-21T13:50:39+02:00 [info]: Handling ping request newt message! 2025-10-21T13:50:39+02:00 [info]: Adding peer with public key <HIDE> to exit node 1 2025-10-21T13:50:39+02:00 [info]: Exit node request successful: {"method":"POST","url":"http://gerbil:3004/peer","status":"Peer added successfully"} 2025-10-21T13:50:40+02:00 [info]: Handling healthcheck status message 2025-10-21T13:50:45+02:00 [info]: Establishing websocket connection 2025-10-21T13:50:45+02:00 [info]: Client added to tracking - NEWT ID: <HIDE>, Connection ID: 85569e1f-7553-4b34-9d0b-55bf7c37e9ce, Total connections: 1 2025-10-21T13:50:45+02:00 [info]: WebSocket connection established - NEWT ID: <HIDE> ``` ### Environment - OS Type & Version: Debian 12 - Pangolin Version: 1.11.1 - Gerbil Version: 1.2.2 - Traefik Version: 3.5 - Newt Version: 1.5.2 ### To Reproduce Not sure if this can be reproduced easily. My Setup is just regulary upgraded since beta phase. ### Expected Behavior Able to add new Sites
GiteaMirror added the stale label 2026-05-06 14:53:02 -05:00
Author
Owner

@baldemar-wuda commented on GitHub (Oct 21, 2025):

Same thing for me.

<!-- gh-comment-id:3427420141 --> @baldemar-wuda commented on GitHub (Oct 21, 2025): Same thing for me.
Author
Owner

@Lokowitz commented on GitHub (Oct 21, 2025):

And I tried to go back to older versions:
Pangolin 1.10.3 and 1.11.0
Gerbil 1.2.0 and 1.2.1

But no combination solved the issue.

<!-- gh-comment-id:3427777371 --> @Lokowitz commented on GitHub (Oct 21, 2025): And I tried to go back to older versions: Pangolin 1.10.3 and 1.11.0 Gerbil 1.2.0 and 1.2.1 But no combination solved the issue.
Author
Owner

@oschwartz10612 commented on GitHub (Oct 21, 2025):

e447549de1 should fix this when 1.11.1 is
released again.

<!-- gh-comment-id:3428560477 --> @oschwartz10612 commented on GitHub (Oct 21, 2025): e447549de13d942ed1eda3e6372c1b7370bb1a4f should fix this when 1.11.1 is released again.
Author
Owner

@oschwartz10612 commented on GitHub (Oct 21, 2025):

Could you check the latest 1.11.1 release?

<!-- gh-comment-id:3429084468 --> @oschwartz10612 commented on GitHub (Oct 21, 2025): Could you check the latest 1.11.1 release?
Author
Owner

@Tuphal commented on GitHub (Oct 22, 2025):

Still not possible to add a new tunnel.

Image
<!-- gh-comment-id:3430519032 --> @Tuphal commented on GitHub (Oct 22, 2025): Still not possible to add a new tunnel. <img width="1285" height="678" alt="Image" src="https://github.com/user-attachments/assets/7d9c8c8b-3c53-4529-b7b8-053adf01b541" />
Author
Owner

@Lokowitz commented on GitHub (Oct 22, 2025):

Also for me not working with the new image.

<!-- gh-comment-id:3430567387 --> @Lokowitz commented on GitHub (Oct 22, 2025): Also for me not working with the new image.
Author
Owner

@Lokowitz commented on GitHub (Oct 22, 2025):

Gerbil spamming is caused through olm holepunch, I turned it off for now.
Should be related to my CGNAT IPv4, hope support for IPv6 is coming :)

<!-- gh-comment-id:3432341665 --> @Lokowitz commented on GitHub (Oct 22, 2025): Gerbil spamming is caused through olm holepunch, I turned it off for now. Should be related to my CGNAT IPv4, hope support for IPv6 is coming :)
Author
Owner

@oschwartz10612 commented on GitHub (Oct 22, 2025):

Hum this is caused when there is not an exit node in the database to
assign a site to. This is usually when there is no gerbil registered but
the update should not have caused this but who knows...

Couple of questions:

  • Are you on enterprise or community

  • If you restart the whole stack docker compose down and docker compose up to get gerbil to register does it work?

  • If not when this occurs could you check the console logs and network
    tab in the browser debugger. Looking for an error message or a 500/400
    error call to pick-site-defaults in the network tab. LMK if this is the
    case.

  • If you have a backup database, roll back to that and then upgrade
    again and see if it occurs. Make sure it still works on your old database.

  • Run the following commands

sudo apt install sqlite3

sqlite3 config/db/db.sqlite "SELECT * FROM exitNodes"

And then let me know what the output is

<!-- gh-comment-id:3433411732 --> @oschwartz10612 commented on GitHub (Oct 22, 2025): Hum this is caused when there is not an exit node in the database to assign a site to. This is usually when there is no gerbil registered but the update should not have caused this but who knows... Couple of questions: * Are you on enterprise or community * If you restart the whole stack `docker compose down` and `docker compose up` to get gerbil to register does it work? * If not when this occurs could you check the console logs and network tab in the browser debugger. Looking for an error message or a 500/400 error call to pick-site-defaults in the network tab. LMK if this is the case. * If you have a backup database, roll back to that and then upgrade again and see if it occurs. Make sure it still works on your old database. * Run the following commands sudo apt install sqlite3 sqlite3 config/db/db.sqlite "SELECT * FROM exitNodes" And then let me know what the output is
Author
Owner

@Tuphal commented on GitHub (Oct 22, 2025):

  • Are you on enterprise or community

Community edition

  • If you restart the whole stack docker compose down and docker compose up to get gerbil to register does it work?

Gebril is connected. The first site is working without issues

  • If not when this occurs could you check the console logs and network
    tab in the browser debugger. Looking for an error message or a 500/400
    error call to pick-site-defaults in the network tab. LMK if this is the
    case.

This is pointing to a right direction..

Image

The API shows: {"data":null,"success":false,"error":true,"message":"No available subnets","status":500,"stack":null}

I could not find any config variable to change the subnet. The org-subnet should be enough big for a second site.

Image Image

Edit: I used DB Browser for my downloaded database. Here is the exitNodes entry:

Image

Edit 2: Should the subnet from the site not be part of the adress?

Subnet 100.89.128.8/30
Adress 100.89.128.1/24

Image
<!-- gh-comment-id:3433984329 --> @Tuphal commented on GitHub (Oct 22, 2025): > * Are you on enterprise or community Community edition > * If you restart the whole stack `docker compose down` and `docker compose up` to get gerbil to register does it work? Gebril is connected. The first site is working without issues > * If not when this occurs could you check the console logs and network > tab in the browser debugger. Looking for an error message or a 500/400 > error call to pick-site-defaults in the network tab. LMK if this is the > case. This is pointing to a right direction.. <img width="973" height="178" alt="Image" src="https://github.com/user-attachments/assets/0b712eec-3128-44f8-928b-27320b7f0bab" /> The API shows: `{"data":null,"success":false,"error":true,"message":"No available subnets","status":500,"stack":null}` I could not find any config variable to change the subnet. The org-subnet should be enough big for a second site. <img width="1427" height="345" alt="Image" src="https://github.com/user-attachments/assets/6f56b5e1-84a9-46c6-a154-c8c3b6ea3ae4" /> <img width="600" height="432" alt="Image" src="https://github.com/user-attachments/assets/a33cdecc-7cb2-4071-a138-db6db984b1db" /> Edit: I used DB Browser for my downloaded database. Here is the exitNodes entry: <img width="1301" height="239" alt="Image" src="https://github.com/user-attachments/assets/dc77a8ff-d4ba-4bd2-9639-1368badcbe61" /> Edit 2: Should the subnet from the site not be part of the adress? `Subnet 100.89.128.8/30` `Adress 100.89.128.1/24` <img width="1440" height="169" alt="Image" src="https://github.com/user-attachments/assets/8c9eff03-e26e-492f-8ef6-de14a9172518" />
Author
Owner

@Lokowitz commented on GitHub (Oct 23, 2025):

I got deeper in the database and found some inconsistencies in ip addresses and subnets.
I updated the database and now It is working again, maybe it was caused through a wrong migration task and/or outdated configs.

1. Check config/config.yml

I still had the subnet group configured and this is not required anymore -> so remove

orgs:
  block_size: 24
  subnet_group: 100.89.138.0/20

2. Check database table orgs

Image

I have 2 orgs, one really old one (since beta phase) and one is maybe 2-3 months old
The old org has subnet 100.90.128.0/24 and the newer one has 100.89.138.0/20, the new default should be 100.90.128.0/24, but is matching the value in config.yml.
Line 3 is just a test org to get the default values.

3. Check database table sites

Image

Subnet for my first site (local) is set to 0.0.0.0/0 a new created is getting 0.0.0.0/32.
The other newt sites have 100.89.128.4/30, 100.89.128.8/30, 100.89.128.12/30 what is looking good so far.
SiteId 1, 4, 5 is associated to org 1, SiteId 9 to org 2 and SiteId 12 to org 3 (test).
And now you see that address of SiteId 9 is not matching the subnet of org 2.

Steps to fix it

  1. Set all subnets of orgs to the same 100.90.128.0/24
  2. Set subnet for local sites to 0.0.0.0/32
  3. Set address for local sites to NULL
  4. Set address for newt sites to 100.90.128.x/24 to match the org subnet (increase x for each site in the same org)
Image Image
<!-- gh-comment-id:3435455019 --> @Lokowitz commented on GitHub (Oct 23, 2025): I got deeper in the database and found some inconsistencies in ip addresses and subnets. I updated the database and now It is working again, maybe it was caused through a wrong migration task and/or outdated configs. ### 1. Check `config/config.yml` I still had the subnet group configured and this is not required anymore -> so remove ``` orgs: block_size: 24 subnet_group: 100.89.138.0/20 ``` ### 2. Check database table orgs <img width="432" height="256" alt="Image" src="https://github.com/user-attachments/assets/a6aa1b63-fae0-4e0d-a311-49472ea44f13" /> I have 2 orgs, one really old one (since beta phase) and one is maybe 2-3 months old The old org has subnet `100.90.128.0/24` and the newer one has `100.89.138.0/20`, the new default should be `100.90.128.0/24`, but is matching the value in `config.yml`. Line 3 is just a test org to get the default values. ### 3. Check database table sites <img width="826" height="360" alt="Image" src="https://github.com/user-attachments/assets/ffcef22c-6b29-491b-8844-a0fd578d4ec1" /> Subnet for my first site (local) is set to `0.0.0.0/0` a new created is getting `0.0.0.0/32`. The other newt sites have `100.89.128.4/30`, `100.89.128.8/30`, `100.89.128.12/30` what is looking good so far. SiteId 1, 4, 5 is associated to org 1, SiteId 9 to org 2 and SiteId 12 to org 3 (test). And now you see that address of SiteId 9 is not matching the subnet of org 2. ## Steps to fix it 1. Set all subnets of orgs to the same `100.90.128.0/24` 2. Set subnet for local sites to `0.0.0.0/32` 3. Set address for local sites to `NULL` 4. Set address for newt sites to `100.90.128.x/24` to match the org subnet (increase x for each site in the same org) <img width="434" height="274" alt="Image" src="https://github.com/user-attachments/assets/66e3b30c-1643-4a94-afc0-084352697ec0" /> <img width="996" height="358" alt="Image" src="https://github.com/user-attachments/assets/4f467e5b-e66b-4ee2-be37-8afceb903def" />
Author
Owner

@Tuphal commented on GitHub (Oct 24, 2025):

Thank you ver much @Lokowitz for the steps.

These worked perfectly !!!

In my case I only had to follow step 2 and 3

<!-- gh-comment-id:3441265379 --> @Tuphal commented on GitHub (Oct 24, 2025): Thank you ver much @Lokowitz for the steps. These worked perfectly !!! In my case I only had to follow step 2 and 3
Author
Owner

@elcajon commented on GitHub (Oct 28, 2025):

Also run into this today.
Changing the local subnet from /0 to /32 fixed it (address was already empty)

Image

Database migration table content for reference:

Image
<!-- gh-comment-id:3457392706 --> @elcajon commented on GitHub (Oct 28, 2025): Also run into this today. Changing the local subnet from /0 to /32 fixed it (address was already empty) <img width="192" height="145" alt="Image" src="https://github.com/user-attachments/assets/044f5057-e318-478b-9971-d367766cf7e6" /> Database migration table content for reference: <img width="207" height="265" alt="Image" src="https://github.com/user-attachments/assets/0b0030d3-c976-4bf8-ba72-285331c4945f" />
Author
Owner

@intersectRaven commented on GitHub (Nov 4, 2025):

Also run into this today. Changing the local subnet from /0 to /32 fixed it (address was already empty)
Image

Database migration table content for reference:
Image

I experienced this just yesterday and this fixed it.

<!-- gh-comment-id:3483613308 --> @intersectRaven commented on GitHub (Nov 4, 2025): > Also run into this today. Changing the local subnet from /0 to /32 fixed it (address was already empty) > <img alt="Image" width="192" height="145" src="https://private-user-images.githubusercontent.com/52052494/506659616-044f5057-e318-478b-9971-d367766cf7e6.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NjIyMjcwNzMsIm5iZiI6MTc2MjIyNjc3MywicGF0aCI6Ii81MjA1MjQ5NC81MDY2NTk2MTYtMDQ0ZjUwNTctZTMxOC00NzhiLTk5NzEtZDM2Nzc2NmNmN2U2LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTExMDQlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUxMTA0VDAzMjYxM1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWRiZGJkZDFkMmQ0OWE1M2M5NDQxOWNjODZjOTZiYjRlYTc2NGE2MDg0MjJhMDRjOTIwN2I3MjE0MzNlNjBkMjMmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.8T24zXLr5BrvUbWR3O1RNv8UTCNL4j7DGWJFWR4c_mk"> > > Database migration table content for reference: > <img alt="Image" width="207" height="265" src="https://private-user-images.githubusercontent.com/52052494/506665027-0b0030d3-c976-4bf8-ba72-285331c4945f.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NjIyMjcwNzMsIm5iZiI6MTc2MjIyNjc3MywicGF0aCI6Ii81MjA1MjQ5NC81MDY2NjUwMjctMGIwMDMwZDMtYzk3Ni00YmY4LWJhNzItMjg1MzMxYzQ5NDVmLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTExMDQlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUxMTA0VDAzMjYxM1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWVlMDIyZTI1ODM2ZDQ3OTJhNGQ0MzgzZmRjOTE5NmU2NDRhYmJiODU3NWJmOWY0Yjk2Yjg4MGIxMzExZTUzNGYmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.tS0DO_9cnNtE0i1-CAwXQqH9Y1V788ZCf7XfTZFoDjg"> I experienced this just yesterday and this fixed it.
Author
Owner

@oschwartz10612 commented on GitHub (Nov 4, 2025):

Thanks so much @Lokowitz that was indeed an issue on an older version. I double checked and this should not occur on latest versions.

<!-- gh-comment-id:3487484165 --> @oschwartz10612 commented on GitHub (Nov 4, 2025): Thanks so much @Lokowitz that was indeed an issue on an older version. I double checked and this should not occur on latest versions.
Author
Owner

@github-actions[bot] commented on GitHub (Nov 19, 2025):

This issue has been automatically marked as stale due to 14 days of inactivity. It will be closed in 14 days if no further activity occurs.

<!-- gh-comment-id:3550023303 --> @github-actions[bot] commented on GitHub (Nov 19, 2025): This issue has been automatically marked as stale due to 14 days of inactivity. It will be closed in 14 days if no further activity occurs.
Author
Owner

@ALERTua commented on GitHub (Dec 17, 2025):

The issue persists on Pangolin v1.13.1.
The workaround still helps.

<!-- gh-comment-id:3665382946 --> @ALERTua commented on GitHub (Dec 17, 2025): The issue persists on Pangolin v1.13.1. The workaround still helps.
Author
Owner

@cpuks commented on GitHub (Dec 19, 2025):

Yeah v1.13.1 still has this bug and fix works

<!-- gh-comment-id:3675730320 --> @cpuks commented on GitHub (Dec 19, 2025): Yeah v1.13.1 still has this bug and fix works
Author
Owner

@dr-jeleise commented on GitHub (Dec 28, 2025):

May I ask a question for my better understanding of the architecture.
In the famous example of Lokowitz he shows an example of the table sites. Why ist the subnet of the entries (100.89.128.x) different from the corresponding adresses (100.90.128.x)? How do the 89 and 90 match?

<!-- gh-comment-id:3694745002 --> @dr-jeleise commented on GitHub (Dec 28, 2025): May I ask a question for my better understanding of the architecture. In the famous example of Lokowitz he shows an example of the table sites. Why ist the subnet of the entries (100.89.128.x) different from the corresponding adresses (100.90.128.x)? How do the 89 and 90 match?
Author
Owner

@nlsrchtr commented on GitHub (Feb 24, 2026):

Hi @oschwartz10612,

many thanks for your work on this wonderful piece of software. It helped me a lot so far!

I was hit by this error message today as well, but the details are bit different. I just want to post here, so others might see it as well, since it's related. My highest subnet is "100.89.128.252/30" and Pangolin doesn't allow me to create new subnets anymore. (Yes, there are around 60 sites connected).

What is a good way forward to extend the available subnets for Pangolin, so it doesn't get "confused" and the connected sites are still not able to talk to each other?

Thanks for any help on this!

Regards,

<!-- gh-comment-id:3954407562 --> @nlsrchtr commented on GitHub (Feb 24, 2026): Hi @oschwartz10612, many thanks for your work on this wonderful piece of software. It helped me a lot so far! I was hit by this error message today as well, but the details are bit different. I just want to post here, so others might see it as well, since it's related. My highest subnet is "100.89.128.252/30" and Pangolin doesn't allow me to create new subnets anymore. (Yes, there are around 60 sites connected). What is a good way forward to extend the available subnets for Pangolin, so it doesn't get "confused" and the connected sites are still not able to talk to each other? Thanks for any help on this! Regards,
Author
Owner

@oschwartz10612 commented on GitHub (Feb 25, 2026):

@nlsrchtr You can open up the subnet on the gerbil config: https://docs.pangolin.net/self-host/advanced/config-file#param-subnet-group

<!-- gh-comment-id:3961829638 --> @oschwartz10612 commented on GitHub (Feb 25, 2026): @nlsrchtr You can open up the subnet on the gerbil config: https://docs.pangolin.net/self-host/advanced/config-file#param-subnet-group
Author
Owner

@nlsrchtr commented on GitHub (Feb 26, 2026):

Hi @oschwartz10612,

thanks for coming back on this. Since there is nothing defined in my config, I assume the default value is the "Example" value (10.0.0.0/8) from the documentation, right?

If I see this correctly, this subnet defines IP ranges from 10.0.0.1 up to 10.255.255.254 which is more than enough for me.

But Pangolin seems to automatically assign subnets only with 100.89.128.X/30 and also blocks the creation of new sites in the UI as well.

Is there anything I'm missing here?

Thank you for looking into this.

<!-- gh-comment-id:3964810414 --> @nlsrchtr commented on GitHub (Feb 26, 2026): Hi @oschwartz10612, thanks for coming back on this. Since there is nothing defined in my config, I assume the default value is the "Example" value (`10.0.0.0/8`) from the documentation, right? If I see [this](https://jodies.de/ipcalc?host=10.0.0.0&mask1=8&mask2=) correctly, this subnet defines IP ranges from `10.0.0.1` up to `10.255.255.254` which is more than enough for me. But Pangolin seems to automatically assign subnets only with `100.89.128.X/30` and also blocks the creation of new sites in the UI as well. Is there anything I'm missing here? Thank you for looking into this.
Author
Owner

@oschwartz10612 commented on GitHub (Feb 26, 2026):

We should update the documentation. The default is 100.89.137.0/20

<!-- gh-comment-id:3968520795 --> @oschwartz10612 commented on GitHub (Feb 26, 2026): We should update the documentation. The default is 100.89.137.0/20
Author
Owner

@nlsrchtr commented on GitHub (Mar 2, 2026):

Thanks for the information. With this netmask there are 4094 hosts available. The IPs range is from 100.89.128.1 upto 100.89.143.254.

I have 60 sites connected and only the 100.89.128.X is "full", but wouldn't Pangolin take another IP address from the range above? An IP from the next range 100.89.129.1 for example?

It seems to me, I'm missing something here? Could you enlighten me once more, @oschwartz10612?

<!-- gh-comment-id:3984300588 --> @nlsrchtr commented on GitHub (Mar 2, 2026): Thanks for the information. With this netmask there are [4094 hosts](https://jodies.de/ipcalc?host=100.89.137.0&mask1=20&mask2=) available. The IPs range is from `100.89.128.1` upto `100.89.143.254`. I have 60 sites connected and only the `100.89.128.X` is "full", but wouldn't Pangolin take another IP address from the range above? An IP from the next range `100.89.129.1` for example? It seems to me, I'm missing something here? Could you enlighten me once more, @oschwartz10612?
Author
Owner

@nlsrchtr commented on GitHub (Mar 9, 2026):

Hi @oschwartz10612,

just wanted to ping you on this issue here again, since I'm still wondering if this situation is a bug or a limitation in my understanding of the subnets: If the default defined subnet of 100.89.137.0/20 is offering 4094 hosts, why is Pangolin not using something outside of 100.89.137.0 - 100.89.137.255, when that part of the IP range is already used?

Do you need any debugging, logs, configs or something else to dive into this deeper?

<!-- gh-comment-id:4024110662 --> @nlsrchtr commented on GitHub (Mar 9, 2026): Hi @oschwartz10612, just wanted to ping you on this issue here again, since I'm still wondering if this situation is a bug or a limitation in my understanding of the subnets: If the default defined subnet of `100.89.137.0/20` is offering 4094 hosts, why is Pangolin not using something outside of `100.89.137.0 - 100.89.137.255`, when that part of the IP range is already used? Do you need any debugging, logs, configs or something else to dive into this deeper?
Author
Owner

@oschwartz10612 commented on GitHub (Mar 26, 2026):

@nlsrchtr perhaps you have a different default. We may have had a /24 in there for some time which could be causing your issue. Could you try to set the subnet group to a larger subnet and restart gerbil and pangolin and test again.

Before doing that you could try to do the following commands to see what the actual value is that gerbil is assigning sites from:

$ sudo apt install sqlite3
$ sqlite3 config/db/db.sqlite
> select * from exitNodes;

And let me know the output

<!-- gh-comment-id:4131370068 --> @oschwartz10612 commented on GitHub (Mar 26, 2026): @nlsrchtr perhaps you have a different default. We may have had a /24 in there for some time which could be causing your issue. Could you try to set the [subnet group](https://docs.pangolin.net/self-host/advanced/config-file#param-subnet-group) to a larger subnet and restart gerbil and pangolin and test again. Before doing that you could try to do the following commands to see what the actual value is that gerbil is assigning sites from: ``` $ sudo apt install sqlite3 $ sqlite3 config/db/db.sqlite > select * from exitNodes; ``` And let me know the output
Author
Owner

@nlsrchtr commented on GitHub (Mar 26, 2026):

Hi @oschwartz10612,

thanks for getting back to me! You are right - in the output there is a /24 net defined: 100.89.128.1/24

1|Exit Node XXXX|100.89.128.1/24|XXXX|XXXX|51820|http://gerbil:3003||1||gerbil|

What would be a stable way out of this situation? Is it just setting the subnet group to 100.89.137.0/20 and hoping all site will reconnect and everything works out? That would be great.

Regards,

<!-- gh-comment-id:4135835782 --> @nlsrchtr commented on GitHub (Mar 26, 2026): Hi @oschwartz10612, thanks for getting back to me! You are right - in the output there is a /24 net defined: `100.89.128.1/24` `1|Exit Node XXXX|100.89.128.1/24|XXXX|XXXX|51820|http://gerbil:3003||1||gerbil|` What would be a stable way out of this situation? Is it just setting the subnet group to `100.89.137.0/20` and hoping all site will reconnect and everything works out? That would be great. Regards,
Author
Owner

@oschwartz10612 commented on GitHub (Mar 26, 2026):

I would set it to 100.89.128.1/20 and you should be good I think! That way the existing sites are still included in the range.

<!-- gh-comment-id:4136146274 --> @oschwartz10612 commented on GitHub (Mar 26, 2026): I would set it to 100.89.128.1/20 and you should be good I think! That way the existing sites are still included in the range.
Author
Owner

@nlsrchtr commented on GitHub (Mar 26, 2026):

I'm sorry, but this didn't worked out. I set:

gerbil:
  start_port: 51820
  base_endpoint: "example.com"
  subnet_group: "100.89.128.1/20"

and even rebooted the whole installation. But after that, I was still not able to add new sites. Looking into the SQlite DB, the entry is still 100.89.128.1/24. Do I need to update that manually in the database?

<!-- gh-comment-id:4136747482 --> @nlsrchtr commented on GitHub (Mar 26, 2026): I'm sorry, but this didn't worked out. I set: ``` gerbil: start_port: 51820 base_endpoint: "example.com" subnet_group: "100.89.128.1/20" ``` and even rebooted the whole installation. But after that, I was still not able to add new sites. Looking into the SQlite DB, the entry is still `100.89.128.1/24`. Do I need to update that manually in the database?
Author
Owner

@oschwartz10612 commented on GitHub (Mar 26, 2026):

Ahh this was my mistake. It does not copy it in. Apologies. Yes, you may need to set this manually.

$ sqlite3 config/db/db.sqlite
> update exitNodes set address = '100.89.128.1/20';

If this does not fix it then we could be having issues with the subnets and for the private resources which we can investigate.

<!-- gh-comment-id:4137349668 --> @oschwartz10612 commented on GitHub (Mar 26, 2026): Ahh this was my mistake. It does not copy it in. Apologies. Yes, you may need to set this manually. ``` $ sqlite3 config/db/db.sqlite > update exitNodes set address = '100.89.128.1/20'; ``` If this does not fix it then we could be having issues with the subnets and for the private resources which we can investigate.
Author
Owner

@nlsrchtr commented on GitHub (Mar 26, 2026):

It looks way better, I can create sites now!

But I observe a strange behaviour:

  1. Sites can be created
  2. Newly creates sites are marked as "Online"
  3. The health check on public resources created for those sites is working as expected
  4. A HTTP request to those public resources runs into a "Gateway Timeout"
  5. In the log of the site devices I can find the entry "Initial reliable ping failed, but continuing: all 5 ping attempts failed, last error: failed to read ICMP packet: i/o timeout"

I can observe this only with newly created sites, existing sites are up and running. That's why I suspect, this might have to do with our changes.

<!-- gh-comment-id:4138945892 --> @nlsrchtr commented on GitHub (Mar 26, 2026): It looks way better, I can create sites now! But I observe a strange behaviour: 1. Sites can be created 2. Newly creates sites are marked as "Online" 3. The health check on public resources created for those sites is working as expected 4. A HTTP request to those public resources runs into a "Gateway Timeout" 5. In the log of the site devices I can find the entry "Initial reliable ping failed, but continuing: all 5 ping attempts failed, last error: failed to read ICMP packet: i/o timeout" I can observe this only with newly created sites, existing sites are up and running. That's why I suspect, this might have to do with our changes.
Author
Owner

@oschwartz10612 commented on GitHub (Mar 27, 2026):

This likely is because gerbil is using the old subnet on the interface.
Try to restart gerbil again now.

<!-- gh-comment-id:4144567154 --> @oschwartz10612 commented on GitHub (Mar 27, 2026): This likely is because gerbil is using the old subnet on the interface. Try to restart gerbil again now.
Author
Owner

@nlsrchtr commented on GitHub (Mar 29, 2026):

Hi @oschwartz10612, you are right! After rebooting the gerbil service, everything was up and running!

Thanks a lot for your support, your patience and this wonderful piece of software!

Please keep up the good work on it!

<!-- gh-comment-id:4149559867 --> @nlsrchtr commented on GitHub (Mar 29, 2026): Hi @oschwartz10612, you are right! After rebooting the gerbil service, everything was up and running! Thanks a lot for your support, your patience and this wonderful piece of software! _Please keep up the good work on it!_
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/pangolin#10753