[GH-ISSUE #1241] 404 on all Ressources after update (1.4.0 -> 1.8.0) #27442

Closed
opened 2026-06-10 22:31:39 -05:00 by GiteaMirror · 4 comments
Owner

Originally created by @Antoine2tt on GitHub (Aug 8, 2025).
Original GitHub issue: https://github.com/fosrl/pangolin/issues/1241

Hello !

I'm having a 404 error on all ressources after update (from 1.4.0 to 1.8.0)

The only thing i'm seeing in log, is that traefik (1.4.5) is saying me this :

ERR Provider error, retrying in 664.618717ms error="cannot decode configuration data: field not found, node: sticky" providerName=http

For information here's some config file

Docker Compose
name: pangolin
services:
  pangolin:
    image: fosrl/pangolin:1.8.0
    container_name: pangolin
    restart: unless-stopped
    volumes:
      - ./config:/app/config
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3001/api/v1/"]
      interval: "3s"
      timeout: "3s"
      retries: 15

  gerbil:
    image: fosrl/gerbil:1.1.0
    container_name: gerbil
    restart: unless-stopped
    depends_on:
      pangolin:
        condition: service_healthy
    command:
      - --reachableAt=http://gerbil:3003
      - --generateAndSaveKeyTo=/var/config/key
      - --remoteConfig=http://pangolin:3001/api/v1/gerbil/get-config
      - --reportBandwidthTo=http://pangolin:3001/api/v1/gerbil/receive-bandwidth
    volumes:
      - ./config/:/var/config
    cap_add:
      - NET_ADMIN
      - SYS_MODULE
    ports:
      - 51820:51820/udp
      - 21820:21820/udp
      - 443:443 # Port for traefik because of the network_mode
      - 80:80 # Port for traefik because of the network_mode
      - 25565:25565
      - 25566:25566

  traefik:
    image: traefik:v3.4.5
    container_name: traefik
    restart: unless-stopped
    network_mode: service:gerbil # Ports appear on the gerbil service
    depends_on:
      pangolin:
        condition: service_healthy
    command:
      - --configFile=/etc/traefik/traefik_config.yml
    volumes:
      - ./config/traefik:/etc/traefik:ro # Volume to store the Traefik configuration
      - ./config/letsencrypt:/letsencrypt # Volume to store the Let's Encrypt certificates

networks:
  default:
    driver: bridge
    name: pangolin
    enable_ipv6: true
pangolin config.yml
app:
  dashboard_url: https://dash.url.com
  log_level: info
  save_logs: false
domains:
  domain1:
    base_domain: url.com
    cert_resolver: letsencrypt
  domain2:
    base_domain: url2.com
    cert_resolver: letsencrypt
  domain3:
    base_domain: url3.com
    cert_resolver: letsencrypt
server:
  external_port: 3000
  internal_port: 3001
  next_port: 3002
  internal_hostname: pangolin
  session_cookie_name: p_session_token
  resource_access_token_param: p_token
  resource_access_token_headers:
    id: P-Access-Token-Id
    token: P-Access-Token
  resource_session_request_param: p_session_request
  secret: top secret ;) 
  cors:
    origins:
      - https://dash.url.com
    methods:
      - GET
      - POST
      - PUT
      - DELETE
      - PATCH
    headers:
      - X-CSRF-Token
      - Content-Type
    credentials: false
traefik:
  cert_resolver: letsencrypt
  http_entrypoint: web
  https_entrypoint: websecure
gerbil:
  start_port: 51820
  base_endpoint: dash.url.com
  use_subdomain: false
  block_size: 24
  site_block_size: 30
  subnet_group: 100.89.137.0/20
rate_limits:
  global:
    window_minutes: 1
    max_requests: 500
users:
  server_admin:
    email: secret@email.com
    password: secretpassword
flags:
  require_email_verification: false
  disable_signup_without_invite: true
  disable_user_create_org: true
  allow_raw_resources: true
  allow_base_domain_resources: true
email:
  smtp_host: redacted
  smtp_port: 587
  smtp_user: redacted
  smtp_pass: redacted
traefik dynamic config
http:
  middlewares:
    redirect-to-https:
      redirectScheme:
        scheme: https

  routers:
    # HTTP to HTTPS redirect router
    main-app-router-redirect:
      rule: "Host(`dash.url.com`)"
      service: next-service
      entryPoints:
        - web
      middlewares:
        - redirect-to-https

    # Next.js router (handles everything except API and WebSocket paths)
    next-router:
      rule: "Host(`dash.url.com`) && !PathPrefix(`/api/v1`)"
      service: next-service
      entryPoints:
        - websecure
      tls:
        certResolver: letsencrypt

    # API router (handles /api/v1 paths)
    api-router:
      rule: "Host(`dash.url.com`) && PathPrefix(`/api/v1`)"
      service: api-service
      entryPoints:
        - websecure
      tls:
        certResolver: letsencrypt

    # WebSocket router
    ws-router:
      rule: "Host(`dash.url.com`)"
      service: api-service
      entryPoints:
        - websecure
      tls:
        certResolver: letsencrypt

  services:
    next-service:
      loadBalancer:
        servers:
          - url: "http://pangolin:3002"  # Next.js server

    api-service:
      loadBalancer:
        servers:
          - url: "http://pangolin:3000"  # API/WebSocket server
traefik config
api:
  insecure: true
  dashboard: true

providers:
  http:
    endpoint: "http://pangolin:3001/api/v1/traefik-config"
    pollInterval: "5s"
  file:
    filename: "/etc/traefik/dynamic_config.yml"

experimental:
  plugins:
    badger:
      moduleName: "github.com/fosrl/badger"
      version: "v1.2.0"
log:
  level: "INFO"
  format: "common"

certificatesResolvers:
  letsencrypt:
    acme:
      httpChallenge:
        entryPoint: web
      email: "still@mysecretemail.com"
      storage: "/letsencrypt/acme.json"
      caServer: "https://acme-v02.api.letsencrypt.org/directory"

entryPoints:
  web:
    address: ":80"
  websecure:
    address: ":443"
  tcp-25565:
    address: ":25565/tcp"
  tcp-25566:
    address: ":25566/tcp"
    transport:
      respondingTimeouts:
        readTimeout: "30m"
    http:
      tls:
        certResolver: "letsencrypt"

serversTransport:
  insecureSkipVerify: true

I already tried traefik v 2.5.0, not working, my "Sites" are online and using last newt version

Thanks for any help !
Have a good day!

Originally created by @Antoine2tt on GitHub (Aug 8, 2025). Original GitHub issue: https://github.com/fosrl/pangolin/issues/1241 Hello ! I'm having a 404 error on all ressources after update (from 1.4.0 to 1.8.0) The only thing i'm seeing in log, is that traefik (1.4.5) is saying me this : `ERR Provider error, retrying in 664.618717ms error="cannot decode configuration data: field not found, node: sticky" providerName=http ` For information here's some config file <details> <summary>Docker Compose</summary> ``` name: pangolin services: pangolin: image: fosrl/pangolin:1.8.0 container_name: pangolin restart: unless-stopped volumes: - ./config:/app/config healthcheck: test: ["CMD", "curl", "-f", "http://localhost:3001/api/v1/"] interval: "3s" timeout: "3s" retries: 15 gerbil: image: fosrl/gerbil:1.1.0 container_name: gerbil restart: unless-stopped depends_on: pangolin: condition: service_healthy command: - --reachableAt=http://gerbil:3003 - --generateAndSaveKeyTo=/var/config/key - --remoteConfig=http://pangolin:3001/api/v1/gerbil/get-config - --reportBandwidthTo=http://pangolin:3001/api/v1/gerbil/receive-bandwidth volumes: - ./config/:/var/config cap_add: - NET_ADMIN - SYS_MODULE ports: - 51820:51820/udp - 21820:21820/udp - 443:443 # Port for traefik because of the network_mode - 80:80 # Port for traefik because of the network_mode - 25565:25565 - 25566:25566 traefik: image: traefik:v3.4.5 container_name: traefik restart: unless-stopped network_mode: service:gerbil # Ports appear on the gerbil service depends_on: pangolin: condition: service_healthy command: - --configFile=/etc/traefik/traefik_config.yml volumes: - ./config/traefik:/etc/traefik:ro # Volume to store the Traefik configuration - ./config/letsencrypt:/letsencrypt # Volume to store the Let's Encrypt certificates networks: default: driver: bridge name: pangolin enable_ipv6: true ``` </details> <details> <summary> pangolin config.yml </summary> ``` app: dashboard_url: https://dash.url.com log_level: info save_logs: false domains: domain1: base_domain: url.com cert_resolver: letsencrypt domain2: base_domain: url2.com cert_resolver: letsencrypt domain3: base_domain: url3.com cert_resolver: letsencrypt server: external_port: 3000 internal_port: 3001 next_port: 3002 internal_hostname: pangolin session_cookie_name: p_session_token resource_access_token_param: p_token resource_access_token_headers: id: P-Access-Token-Id token: P-Access-Token resource_session_request_param: p_session_request secret: top secret ;) cors: origins: - https://dash.url.com methods: - GET - POST - PUT - DELETE - PATCH headers: - X-CSRF-Token - Content-Type credentials: false traefik: cert_resolver: letsencrypt http_entrypoint: web https_entrypoint: websecure gerbil: start_port: 51820 base_endpoint: dash.url.com use_subdomain: false block_size: 24 site_block_size: 30 subnet_group: 100.89.137.0/20 rate_limits: global: window_minutes: 1 max_requests: 500 users: server_admin: email: secret@email.com password: secretpassword flags: require_email_verification: false disable_signup_without_invite: true disable_user_create_org: true allow_raw_resources: true allow_base_domain_resources: true email: smtp_host: redacted smtp_port: 587 smtp_user: redacted smtp_pass: redacted ``` </details> <details> <summary> traefik dynamic config </summary> ``` http: middlewares: redirect-to-https: redirectScheme: scheme: https routers: # HTTP to HTTPS redirect router main-app-router-redirect: rule: "Host(`dash.url.com`)" service: next-service entryPoints: - web middlewares: - redirect-to-https # Next.js router (handles everything except API and WebSocket paths) next-router: rule: "Host(`dash.url.com`) && !PathPrefix(`/api/v1`)" service: next-service entryPoints: - websecure tls: certResolver: letsencrypt # API router (handles /api/v1 paths) api-router: rule: "Host(`dash.url.com`) && PathPrefix(`/api/v1`)" service: api-service entryPoints: - websecure tls: certResolver: letsencrypt # WebSocket router ws-router: rule: "Host(`dash.url.com`)" service: api-service entryPoints: - websecure tls: certResolver: letsencrypt services: next-service: loadBalancer: servers: - url: "http://pangolin:3002" # Next.js server api-service: loadBalancer: servers: - url: "http://pangolin:3000" # API/WebSocket server ``` </details> <details> <summary> traefik config </summary> ``` api: insecure: true dashboard: true providers: http: endpoint: "http://pangolin:3001/api/v1/traefik-config" pollInterval: "5s" file: filename: "/etc/traefik/dynamic_config.yml" experimental: plugins: badger: moduleName: "github.com/fosrl/badger" version: "v1.2.0" log: level: "INFO" format: "common" certificatesResolvers: letsencrypt: acme: httpChallenge: entryPoint: web email: "still@mysecretemail.com" storage: "/letsencrypt/acme.json" caServer: "https://acme-v02.api.letsencrypt.org/directory" entryPoints: web: address: ":80" websecure: address: ":443" tcp-25565: address: ":25565/tcp" tcp-25566: address: ":25566/tcp" transport: respondingTimeouts: readTimeout: "30m" http: tls: certResolver: "letsencrypt" serversTransport: insecureSkipVerify: true ``` </details> I already tried traefik v 2.5.0, not working, my "Sites" are online and using last newt version Thanks for any help ! Have a good day!
Author
Owner

@marcschaeferger commented on GitHub (Aug 8, 2025):

Hi @Antoine2tt,

The reason why all resources are returning 404 errors is that the configuration automatically generated by Pangolin for Traefik contains deprecated or outdated fields after your updates. Specifically, the field sticky as indicated in the log is no longer supported in newer Traefik versions (previously used for load balancers). When Traefik encounters unknown or invalid fields like this, it fails to parse that configuration completely. As a result, no routes are registered, and Traefik responds with 404 for all the routes that are in the config that can't be parsed.

How to Confirm:

You can inspect the automatically generated Traefik configuration directly from the Pangolin container by running the following commands:

docker exec -it pangolin sh
curl http://pangolin:3001/api/v1/traefik-config

To help pinpoint the exact issue, you can specifically look for the sticky field or other deprecated keys in the generated config:

curl http://pangolin:3001/api/v1/traefik-config | grep sticky
<!-- gh-comment-id:3169154182 --> @marcschaeferger commented on GitHub (Aug 8, 2025): Hi @Antoine2tt, **The reason why all resources are returning 404 errors is that the configuration automatically generated by Pangolin for Traefik contains deprecated or outdated fields after your updates.** Specifically, the field `sticky` as indicated in the log is no longer supported in newer Traefik versions (previously used for load balancers). When Traefik encounters unknown or invalid fields like this, it fails to parse that configuration completely. As a result, **no routes are registered, and Traefik responds with 404 for all the routes that are in the config that can't be parsed**. ## How to Confirm: You can inspect the automatically generated Traefik configuration directly from the Pangolin container by running the following commands: ```sh docker exec -it pangolin sh curl http://pangolin:3001/api/v1/traefik-config ``` To help pinpoint the exact issue, you can specifically look for the `sticky` field or other deprecated keys in the generated config: ```sh curl http://pangolin:3001/api/v1/traefik-config | grep sticky ```
Author
Owner

@Antoine2tt commented on GitHub (Aug 8, 2025):

Hello @marcschaeferger

Thanks for your answer, i did what you asked, and i saw "sticky" in the return of

curl http://pangolin:3001/api/v1/traefik-config | grep sticky

At the end of the output :

"sticky":{"ipStrategy":{"depth":0,"sourcePort":true}}}}}}}

Is there anything i can do to fix that ?
Thanks !

<!-- gh-comment-id:3169172662 --> @Antoine2tt commented on GitHub (Aug 8, 2025): Hello @marcschaeferger Thanks for your answer, i did what you asked, and i saw "sticky" in the return of `curl http://pangolin:3001/api/v1/traefik-config | grep sticky` At the end of the output : `"sticky":{"ipStrategy":{"depth":0,"sourcePort":true}}}}}}}` Is there anything i can do to fix that ? Thanks !
Author
Owner

@marcschaeferger commented on GitHub (Aug 8, 2025):

@Antoine2tt,

The "sticky" field found in the generated Traefik configuration is stored within Pangolin's database alongside all resources and other configuration data. In your case, since your Pangolin setup does not specify an external database like PostgreSQL, it is very likely using SQLite, which is the default database if not explicitly overridden in the Pangolin app configuration.

The .db SQLite file is stored on your Docker host's filesystem. Based on your docker-compose.yml, the database file is most likely located in the ./config folder alongside your compose file, and the specific file is typically named something like db.sqlite.


Step-by-Step Solution

1. Backup the SQLite Database

Before making any changes, it's critical to back up the SQLite database to ensure you can easily recover if something goes wrong.

Run (Example):

cp ./config/db/db.sqlite /home/pangolin.sqlite.bak

2. Copy the File for Editing

Create a working copy of this file, preferably in a temporary folder like /tmp:

cp ./config/db/db.sqlite /tmp/pangolin-edit.sqlite

3. Edit the SQLite Database

To remove the "sticky" entry, open the copied database with a database editor. Depending on your preference, use either:

  • GUI Tool:
    Tools like DBeaver allow you to edit SQLite databases with a user-friendly interface.

  • CLI Tool:
    Use the built-in SQLite CLI tool:

    sqlite3 /tmp/pangolin-edit.db
    

    After opening the database, you can explore tables and locate the problematic configuration.

4. Locate the sticky Field

Pangolin stores the resource configurations in specific tables. To identify where the sticky field is located, you can run SQL queries. For example:

SELECT r.resourceId, r.fullDomain, r.stickySession, s.siteId
FROM resources r
JOIN sites s ON s.siteId = r.siteId;

See https://github.com/fosrl/pangolin/blob/main/server/routers/traefik/getTraefikConfig.ts

Once you've identified the sticky-related setting, update its value. From my database, I can confirm that the sticky value should be set to 0. If you're able to wait, I can test this process using an older version later and provide more details. Alternatively, you could reach out to the maintainer for further clarification if this is a supported way.

5. Replace the Original Database

After editing, stop Pangolin (if a short Downtime is feasable) to swap out the database file:

docker compose down

Copy the edited database back to its original location:

cp -f /tmp/pangolin-edit.db ./config/db/db.sqlite

Finally, restart Pangolin to load the corrected data:

docker compose up -d

Important Notes

  • Editing SQLite files is safe and reversible as long as you keep proper backups.
  • After restarting Pangolin, the "sticky" field should no longer appear in the generated Traefik config, and the 404 error should be resolved.

Summary

  1. Backup the database file (db.sqlite).
  2. Open the file with a GUI tool like DBeaver or with the sqlite3 CLI and remove references to "sticky".
  3. Replace the database on the host filesystem.
  4. Restart Pangolin using docker compose.

Let me know if you need help locating the database file or editing its contents!

<!-- gh-comment-id:3169311496 --> @marcschaeferger commented on GitHub (Aug 8, 2025): @Antoine2tt, The `"sticky"` field found in the generated Traefik configuration is stored within Pangolin's database alongside all resources and other configuration data. In your case, since your Pangolin setup does not specify an external database like PostgreSQL, it is very likely using **SQLite**, which is the default database if not explicitly overridden in the Pangolin app configuration. The `.db` SQLite file is stored on your **Docker host's filesystem**. Based on your `docker-compose.yml`, the database file is most likely located in the `./config` folder alongside your compose file, and the specific file is typically named something like `db.sqlite`. --- ### Step-by-Step Solution #### 1. **Backup the SQLite Database** Before making any changes, it's critical to back up the SQLite database to ensure you can easily recover if something goes wrong. Run (Example): ```sh cp ./config/db/db.sqlite /home/pangolin.sqlite.bak ``` #### 2. **Copy the File for Editing** Create a working copy of this file, preferably in a temporary folder like `/tmp`: ```sh cp ./config/db/db.sqlite /tmp/pangolin-edit.sqlite ``` #### 3. **Edit the SQLite Database** To remove the `"sticky"` entry, open the copied database with a database editor. Depending on your preference, use either: - **GUI Tool:** Tools like [DBeaver](https://dbeaver.io/) allow you to edit SQLite databases with a user-friendly interface. - **CLI Tool:** Use the built-in SQLite CLI tool: ```sh sqlite3 /tmp/pangolin-edit.db ``` After opening the database, you can explore tables and locate the problematic configuration. #### 4. **Locate the `sticky` Field** Pangolin stores the resource configurations in specific tables. To identify where the `sticky` field is located, you can run SQL queries. For example: ```sql SELECT r.resourceId, r.fullDomain, r.stickySession, s.siteId FROM resources r JOIN sites s ON s.siteId = r.siteId; ``` See https://github.com/fosrl/pangolin/blob/main/server/routers/traefik/getTraefikConfig.ts Once you've identified the sticky-related setting, update its value. From my database, I can confirm that the sticky value should be set to 0. If you're able to wait, I can test this process using an older version later and provide more details. Alternatively, you could reach out to the maintainer for further clarification if this is a supported way. #### 5. **Replace the Original Database** After editing, stop Pangolin (if a short Downtime is feasable) to swap out the database file: ```sh docker compose down ``` Copy the edited database back to its original location: ```sh cp -f /tmp/pangolin-edit.db ./config/db/db.sqlite ``` Finally, restart Pangolin to load the corrected data: ```sh docker compose up -d ``` --- ### Important Notes - Editing SQLite files is safe and reversible as long as you keep proper backups. - After restarting Pangolin, the `"sticky"` field should no longer appear in the generated Traefik config, and the 404 error should be resolved. --- ### Summary 1. Backup the database file (`db.sqlite`). 2. Open the file with a GUI tool like DBeaver or with the `sqlite3` CLI and remove references to `"sticky"`. 3. Replace the database on the host filesystem. 4. Restart Pangolin using `docker compose`. Let me know if you need help locating the database file or editing its contents!
Author
Owner

@Antoine2tt commented on GitHub (Aug 8, 2025):

@marcschaeferger

Thanks for puting me on the right way !!
I had something with "sticky" option enable in my "Ressources"

I just delete it, restart and everything is working again !Thanks again

Have a good night !

<!-- gh-comment-id:3169457003 --> @Antoine2tt commented on GitHub (Aug 8, 2025): @marcschaeferger Thanks for puting me on the right way !! I had something with "sticky" option enable in my "Ressources" I just delete it, restart and everything is working again !Thanks again Have a good night !
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/pangolin#27442