github-starred/pangolin
2
0
Fork 0
mirror of https://github.com/fosrl/pangolin.git synced 2026-11-13 06:14:30 -06:00
Code Issues 1.6k Packages Projects Releases 79 Wiki Activity

[GH-ISSUE #1613] Unable to deploy with rootless podman #14846

New Issue
Closed
opened 2026-05-16 02:50:59 -05:00 by GiteaMirror · 18 comments
GiteaMirror commented 2026-05-16 02:50:59 -05:00
Owner
Copy Link
Copy Source

Originally created by @Froggy232 on GitHub (Oct 4, 2025).
Original GitHub issue: https://github.com/fosrl/pangolin/issues/1613

Describe the Bug

Hi,

I try to deploy pangolin rootless on a Hetzner VPS that run Fedora CoreOS.

So far, I have created a new user zone, permitted unprivileged user to open restricted ports, and have redacted theses quadlets files :

Everything start successfully, except that gerbil and traefik seems to not be able to connect to the app, the logs of traefik tell me 2025-10-03T13:50:51Z ERR Provider error, retrying in 732.775521ms error="cannot fetch configuration data: do fetch request: Get \"http://app:3001/api/v1/traefik-config\": dial tcp 127.0.0.1:3001: connect: connection refused" providerName=http and gerbil logs are here.

Thanks you a lot for your help, of course, feel free to ask me any other informations as config files, I can post them too.
Have a nice day

Environment

  • OS Type & Version: Fedora CoreOS 42, pangolin is deployed through rootless podman & quadlet

To Reproduce

Try to deploy pangolin with podman in rootless mode

Expected Behavior

Traefik and gerbil should communicate with the app as they are all in the same pod. When I type podman ps, the app container seems to not have theses ports opened, it's probably related?
Thanks you a lot for your help, I would really love to switch from caddy + wireguard to Pangolin, but I fail to solve this issue.

Originally created by @Froggy232 on GitHub (Oct 4, 2025). Original GitHub issue: https://github.com/fosrl/pangolin/issues/1613 ### Describe the Bug Hi, I try to deploy pangolin rootless on a Hetzner VPS that run Fedora CoreOS. So far, I have created a new user zone, permitted unprivileged user to open restricted ports, and have redacted theses quadlets files : - [app.container](https://privatebin.alrhome.net/?5c3920d7b6da2504#8A53C3cJV1ykjTMrdDhegbuQKtZudUL88i4tWnYc1Wik) - [gerbil.container](https://privatebin.alrhome.net/?893a648cc84bda4f#351BtdB1v284xcaFtAJWBQe8wpn7u1Fskio8J4Lu6qXZ) - [traefik.container](https://privatebin.alrhome.net/?28e46f2204050cad#CLDaiafLcDJ4cuV3oH5WGWwNe2R1FXvNDkWz5i5gbRxW) - [pangolin.pod](https://privatebin.alrhome.net/?a5bdbca9c068a7e2#BAuSvXPD2styybEccRM6nkbW8TxhPyVouPVKfkHScT8n) Everything start successfully, except that gerbil and traefik seems to not be able to connect to the app, the logs of traefik tell me `2025-10-03T13:50:51Z ERR Provider error, retrying in 732.775521ms error="cannot fetch configuration data: do fetch request: Get \"http://app:3001/api/v1/traefik-config\": dial tcp 127.0.0.1:3001: connect: connection refused" providerName=http` and gerbil logs are [here](https://privatebin.alrhome.net/?42ad806ca24b5863#EW4g47Rqi9wuTGqRmp665UDesDaPgjUNp3uT7xNsuzJx). Thanks you a lot for your help, of course, feel free to ask me any other informations as config files, I can post them too. Have a nice day ### Environment - OS Type & Version: Fedora CoreOS 42, pangolin is deployed through rootless podman & quadlet ### To Reproduce Try to deploy pangolin with podman in rootless mode ### Expected Behavior Traefik and gerbil should communicate with the app as they are all in the same pod. When I type `podman ps`, the app container seems to not have theses ports opened, it's probably related? Thanks you a lot for your help, I would really love to switch from caddy + wireguard to Pangolin, but I fail to solve this issue.
GiteaMirror commented 2026-05-16 02:51:00 -05:00
Author
Owner
Copy Link
Copy Source

@allentd commented on GitHub (Oct 4, 2025):

Hi, you might want to create podman network to connect the container through their hostname.

Got the same error when testing Pangolin recently on rootless podman without custom network.

<!-- gh-comment-id:3368497205 --> @allentd commented on GitHub (Oct 4, 2025): Hi, you might want to create podman network to connect the container through their hostname. Got the same error when testing Pangolin recently on rootless podman without custom network.
GiteaMirror commented 2026-05-16 02:51:01 -05:00
Author
Owner
Copy Link
Copy Source

@Froggy232 commented on GitHub (Oct 4, 2025):

Hi,
Thanks for your input, I will try that!
Are you sure it's necessary if they are inside the same container though? My others services works like that (they are grouped per service, one per pod) and everything works well, except for Pangolin.
I will try that right now regardless, thanks you a lot!

EDIT : Thanks you so much, I think it worked! Now, gerbil seems to be good, but I still have a traefik error : 2025-10-04T23:03:32Z ERR Provider error, retrying in 1.269850554s error="cannot fetch configuration data: do fetch request: Get \"http://app:3001/api/v1/traefik-config\": dial tcp 192.168.30.3:3001: connect: connection refused" providerName=http
I don't understand what it means, I will try to do research but if someone has an idea, I would be happy to hear it!
Thanks again

<!-- gh-comment-id:3368589166 --> @Froggy232 commented on GitHub (Oct 4, 2025): Hi, Thanks for your input, I will try that! Are you sure it's necessary if they are inside the same container though? My others services works like that (they are grouped per service, one per pod) and everything works well, except for Pangolin. I will try that right now regardless, thanks you a lot! EDIT : Thanks you so much, I think it worked! Now, gerbil seems to be good, but I still have a traefik error : `2025-10-04T23:03:32Z ERR Provider error, retrying in 1.269850554s error="cannot fetch configuration data: do fetch request: Get \"http://app:3001/api/v1/traefik-config\": dial tcp 192.168.30.3:3001: connect: connection refused" providerName=http` I don't understand what it means, I will try to do research but if someone has an idea, I would be happy to hear it! Thanks again
GiteaMirror commented 2026-05-16 02:51:01 -05:00
Author
Owner
Copy Link
Copy Source

@Froggy232 commented on GitHub (Oct 4, 2025):

Traefik seems to sometime not be able to connect to the app container, but after relaunching it manually it seems to work.
Though, if I type pangolin.mydomain.tld in the address bar, I get an ECONNRESET error from chrome or firefox.
Someone has an idea on that error? I don't have anything in the log.
I feel like I'm closer than ever to get it working haha; thanks again @allentd

<!-- gh-comment-id:3368599975 --> @Froggy232 commented on GitHub (Oct 4, 2025): Traefik seems to sometime not be able to connect to the app container, but after relaunching it manually it seems to work. Though, if I type `pangolin.mydomain.tld` in the address bar, I get an ECONNRESET error from chrome or firefox. Someone has an idea on that error? I don't have anything in the log. I feel like I'm closer than ever to get it working haha; thanks again @allentd
GiteaMirror commented 2026-05-16 02:51:02 -05:00
Author
Owner
Copy Link
Copy Source

@allentd commented on GitHub (Oct 5, 2025):

About the errors, I'm still seeing 2, 3 lines of it even after adding After= and Requires= or BindsTo= to traefik.container on innitial startup with a working setup.

As for browsers ECONNRESET, it will be wild guess without any logs. Maybe configs issue?

Can try enabling traefik log in traefik.yaml to check it.

<!-- gh-comment-id:3369075617 --> @allentd commented on GitHub (Oct 5, 2025): About the errors, I'm still seeing 2, 3 lines of it even after adding After= and Requires= or BindsTo= to traefik.container on innitial startup with a working setup. As for browsers ECONNRESET, it will be wild guess without any logs. Maybe configs issue? Can try enabling traefik log in traefik.yaml to check it.
GiteaMirror commented 2026-05-16 02:51:03 -05:00
Author
Owner
Copy Link
Copy Source

@Froggy232 commented on GitHub (Oct 5, 2025):

Sorry about the logs, I will check and post them but I think it's maybe a certificates problem? I have removed all the firewalls but it seems traefik still can't obtain them, it stays indefinitely on this step : 2025-10-05T17:46:02Z INF Testing certificate renew... acmeCA=https://acme-v02.api.letsencrypt.org/directory providerName=letsencrypt.acme and I don't see anything in the certificates volume, maybe I'm wrong?
I will add log in traefik and report back, thanks again

<!-- gh-comment-id:3369214032 --> @Froggy232 commented on GitHub (Oct 5, 2025): Sorry about the logs, I will check and post them but I think it's maybe a certificates problem? I have removed all the firewalls but it seems traefik still can't obtain them, it stays indefinitely on this step : `2025-10-05T17:46:02Z INF Testing certificate renew... acmeCA=https://acme-v02.api.letsencrypt.org/directory providerName=letsencrypt.acme` and I don't see anything in the certificates volume, maybe I'm wrong? I will add log in traefik and report back, thanks again
GiteaMirror commented 2026-05-16 02:51:03 -05:00
Author
Owner
Copy Link
Copy Source

@allentd commented on GitHub (Oct 5, 2025):

Didn't see any issue in the logs, at least to my untrained eyes. New to Traefik, even newer to Pangolin, might have to wait for other to chime in. Sorry.

But the logs is exposing your domain name in there.

Just random thing come to my mind, how about firewall? Is VPS IP accessible through browser? It will just shows Traefik 404 page even on my working setup.

<!-- gh-comment-id:3369284388 --> @allentd commented on GitHub (Oct 5, 2025): Didn't see any issue in the logs, at least to my untrained eyes. New to Traefik, even newer to Pangolin, might have to wait for other to chime in. Sorry. But the logs is exposing your domain name in there. Just random thing come to my mind, how about firewall? Is VPS IP accessible through browser? It will just shows Traefik 404 page even on my working setup.
GiteaMirror commented 2026-05-16 02:51:03 -05:00
Author
Owner
Copy Link
Copy Source

@hhftechnology commented on GitHub (Oct 5, 2025):

Corrected Quadlet Pack

app.container

[Unit]
Description=Pangolin App Service

[Container]
Pod=pangolin.pod
ContainerName=app
Image=docker.io/fosrl/pangolin
Volume=%h/config/:/app/config:z
# Ensure the app binds to all interfaces, not just localhost
Exec='--listen=0.0.0.0:3001'

[Service]
Restart=always

[Install]
WantedBy=default.target

gerbil.container

[Unit]
Description=Gerbil Service
After=app.service
Requires=app.service

[Container]
Pod=pangolin.pod
AutoUpdate=registry
AddCapability=NET_ADMIN SYS_MODULE
ContainerName=gerbil
Image=docker.io/fosrl/gerbil
Network=host
Volume=%h/config/:/var/config:z
Exec='--reachableAt=http://127.0.0.1:3003' \
     '--generateAndSaveKeyTo=/var/config/key' \
     '--remoteConfig=http://app:3001/api/v1/' \
     '--sni-port=443'

[Service]
Restart=always

[Install]
WantedBy=default.target

traefik.container

[Unit]
Description=Traefik Reverse Proxy
After=app.service
Requires=app.service

[Container]
Pod=pangolin.pod
ContainerName=traefik
Image=docker.io/traefik
Exec='--configFile=/etc/traefik/traefik_config.yml'
Volume=%h/config/traefik:/etc/traefik:ro,z
Volume=%h/config/letsencrypt:/letsencrypt:z
Volume=%h/config/traefik/logs:/var/log/traefik:z
Volume=%h/certificates:/var/certificates:ro,Z
Volume=%h/dynamic:/var/dynamic:ro,Z

[Service]
Restart=always

[Install]
WantedBy=default.target

pangolin.pod

[Pod]
PublishPort=80:80/tcp
PublishPort=443:443/tcp
PublishPort=443:443/udp
PublishPort=51820:51820/udp
PublishPort=21820:21820/udp

Key Fixes

  • App container now binds to 0.0.0.0:3001 → Traefik and Gerbil can reach it inside the pod.
  • Unit ordering (After= + Requires=) → Traefik and Gerbil won’t start before the app is ready.
  • Pod ports → Explicitly expose 80/443 for ACME HTTP‑01 challenges and TLS traffic.
  • Volumes → ACME storage (/letsencrypt/acme.json) is persisted so certificates don’t vanish on restart.

Next Steps you can try

  1. Drop these files into ~/.config/containers/systemd/.
  2. Run: 
    systemctl --user daemon-reload
systemctl --user enable --now pangolin.pod
  3. Verify app is listening: 
    podman exec -it app ss -tlnp | grep 3001
    Should show 0.0.0.0:3001.
  4. Test Traefik can fetch config: 
    podman exec -it traefik curl -v http://app:3001/api/v1/traefik-config
  5. Hit your domain on port 80/443 and watch Traefik logs for ACME success.

traefik_config.yml (static)

api:
  insecure: true
  dashboard: true

log:
  level: INFO
  format: common

entryPoints:
  web:
    address: ":80"
  websecure:
    address: ":443"
    http:
      tls:
        certResolver: letsencrypt

providers:
  http:
    endpoint: "http://app:3001/api/v1/traefik-config"
    pollInterval: "5s"
  file:
    filename: "/etc/traefik/dynamic_config.yml"

certificatesResolvers:
  letsencrypt:
    acme:
      email: "admin@example.com"
      storage: "/letsencrypt/acme.json"
      httpChallenge:
        entryPoint: web

Notes:

  • endpoint points to http://app:3001/... (since in your pod the container is named app).
  • ACME is enabled with HTTP‑01 on port 80.
  • Dashboard is exposed on :8080 by default (you can add an entryPoint if you want to expose it externally).

dynamic_config.yml (dynamic)

http:
  middlewares:
    redirect-to-https:
      redirectScheme:
        scheme: https

  routers:
    redirect:
      rule: "Host(`your.domain.com`)"
      entryPoints:
        - web
      middlewares:
        - redirect-to-https
      service: next-service

    next-router:
      rule: "Host(`your.domain.com`) && !PathPrefix(`/api/v1`)"
      entryPoints:
        - websecure
      service: next-service
      tls:
        certResolver: letsencrypt

    api-router:
      rule: "Host(`your.domain.com`) && PathPrefix(`/api/v1`)"
      entryPoints:
        - websecure
      service: api-service
      tls:
        certResolver: letsencrypt

    ws-router:
      rule: "Host(`your.domain.com`)"
      entryPoints:
        - websecure
      service: api-service
      tls:
        certResolver: letsencrypt

  services:
    next-service:
      loadBalancer:
        servers:
          - url: "http://app:3002"   # Next.js server
    api-service:
      loadBalancer:
        servers:
          - url: "http://app:3000"   # API/WebSocket server

Notes:

  • Replace your.domain.com with actual domain.
  • Services point to http://app:3000 and http://app:3002 (since they’re in the same pod, DNS app works).

How yopu should use this

  1. Drop both files into the mounted traefik config volume: 
    ~/.config/containers/systemd/traefik_config.yml
~/.config/containers/systemd/dynamic_config.yml
    (or wherever %h/config/traefik is mapped).
  2. Restart the pod: 
    systemctl --user restart pangolin.pod
  3. Watch Traefik logs for:
    • ... obtained ACME certificate successfully
    • Configuration received from provider http
<!-- gh-comment-id:3369300289 --> @hhftechnology commented on GitHub (Oct 5, 2025): --- ## Corrected Quadlet Pack ### `app.container` ```ini [Unit] Description=Pangolin App Service [Container] Pod=pangolin.pod ContainerName=app Image=docker.io/fosrl/pangolin Volume=%h/config/:/app/config:z # Ensure the app binds to all interfaces, not just localhost Exec='--listen=0.0.0.0:3001' [Service] Restart=always [Install] WantedBy=default.target ``` --- ### `gerbil.container` ```ini [Unit] Description=Gerbil Service After=app.service Requires=app.service [Container] Pod=pangolin.pod AutoUpdate=registry AddCapability=NET_ADMIN SYS_MODULE ContainerName=gerbil Image=docker.io/fosrl/gerbil Network=host Volume=%h/config/:/var/config:z Exec='--reachableAt=http://127.0.0.1:3003' \ '--generateAndSaveKeyTo=/var/config/key' \ '--remoteConfig=http://app:3001/api/v1/' \ '--sni-port=443' [Service] Restart=always [Install] WantedBy=default.target ``` --- ### `traefik.container` ```ini [Unit] Description=Traefik Reverse Proxy After=app.service Requires=app.service [Container] Pod=pangolin.pod ContainerName=traefik Image=docker.io/traefik Exec='--configFile=/etc/traefik/traefik_config.yml' Volume=%h/config/traefik:/etc/traefik:ro,z Volume=%h/config/letsencrypt:/letsencrypt:z Volume=%h/config/traefik/logs:/var/log/traefik:z Volume=%h/certificates:/var/certificates:ro,Z Volume=%h/dynamic:/var/dynamic:ro,Z [Service] Restart=always [Install] WantedBy=default.target ``` --- ### `pangolin.pod` ```ini [Pod] PublishPort=80:80/tcp PublishPort=443:443/tcp PublishPort=443:443/udp PublishPort=51820:51820/udp PublishPort=21820:21820/udp ``` --- ## Key Fixes - **App container now binds to `0.0.0.0:3001`** → Traefik and Gerbil can reach it inside the pod. - **Unit ordering (`After=` + `Requires=`)** → Traefik and Gerbil won’t start before the app is ready. - **Pod ports** → Explicitly expose 80/443 for ACME HTTP‑01 challenges and TLS traffic. - **Volumes** → ACME storage (`/letsencrypt/acme.json`) is persisted so certificates don’t vanish on restart. --- ## Next Steps you can try 1. Drop these files into `~/.config/containers/systemd/`. 2. Run: ```bash systemctl --user daemon-reload systemctl --user enable --now pangolin.pod ``` 3. Verify app is listening: ```bash podman exec -it app ss -tlnp | grep 3001 ``` Should show `0.0.0.0:3001`. 4. Test Traefik can fetch config: ```bash podman exec -it traefik curl -v http://app:3001/api/v1/traefik-config ``` 5. Hit your domain on port 80/443 and watch Traefik logs for ACME success. --- ## `traefik_config.yml` (static) ```yaml api: insecure: true dashboard: true log: level: INFO format: common entryPoints: web: address: ":80" websecure: address: ":443" http: tls: certResolver: letsencrypt providers: http: endpoint: "http://app:3001/api/v1/traefik-config" pollInterval: "5s" file: filename: "/etc/traefik/dynamic_config.yml" certificatesResolvers: letsencrypt: acme: email: "admin@example.com" storage: "/letsencrypt/acme.json" httpChallenge: entryPoint: web ``` Notes: - `endpoint` points to `http://app:3001/...` (since in your pod the container is named `app`). - ACME is enabled with HTTP‑01 on port 80. - Dashboard is exposed on `:8080` by default (you can add an entryPoint if you want to expose it externally). --- ## `dynamic_config.yml` (dynamic) ```yaml http: middlewares: redirect-to-https: redirectScheme: scheme: https routers: redirect: rule: "Host(`your.domain.com`)" entryPoints: - web middlewares: - redirect-to-https service: next-service next-router: rule: "Host(`your.domain.com`) && !PathPrefix(`/api/v1`)" entryPoints: - websecure service: next-service tls: certResolver: letsencrypt api-router: rule: "Host(`your.domain.com`) && PathPrefix(`/api/v1`)" entryPoints: - websecure service: api-service tls: certResolver: letsencrypt ws-router: rule: "Host(`your.domain.com`)" entryPoints: - websecure service: api-service tls: certResolver: letsencrypt services: next-service: loadBalancer: servers: - url: "http://app:3002" # Next.js server api-service: loadBalancer: servers: - url: "http://app:3000" # API/WebSocket server ``` Notes: - Replace `your.domain.com` with actual domain. - Services point to `http://app:3000` and `http://app:3002` (since they’re in the same pod, DNS `app` works). --- ## How yopu should use this 1. Drop both files into the mounted `traefik` config volume: ``` ~/.config/containers/systemd/traefik_config.yml ~/.config/containers/systemd/dynamic_config.yml ``` (or wherever `%h/config/traefik` is mapped). 2. Restart the pod: ```bash systemctl --user restart pangolin.pod ``` 3. Watch Traefik logs for: - `... obtained ACME certificate successfully` - `Configuration received from provider http` ---
GiteaMirror commented 2026-05-16 02:51:04 -05:00
Author
Owner
Copy Link
Copy Source

@Froggy232 commented on GitHub (Oct 6, 2025):

Thanks you a lot!!!
I will try that right now and then report, thanks a lot!

<!-- gh-comment-id:3370308155 --> @Froggy232 commented on GitHub (Oct 6, 2025): Thanks you a lot!!! I will try that right now and then report, thanks a lot!
GiteaMirror commented 2026-05-16 02:51:05 -05:00
Author
Owner
Copy Link
Copy Source

@Froggy232 commented on GitHub (Oct 6, 2025):

It doesn't seems to work unfortunally, I now have that in the app logs : node: bad option: --listen=0.0.0.0:3001
Thanks again

<!-- gh-comment-id:3370344153 --> @Froggy232 commented on GitHub (Oct 6, 2025): It doesn't seems to work unfortunally, I now have that in the app logs : node: `bad option: --listen=0.0.0.0:3001` Thanks again
GiteaMirror commented 2026-05-16 02:51:06 -05:00
Author
Owner
Copy Link
Copy Source

@Froggy232 commented on GitHub (Oct 6, 2025):

Very weirdly, I just tried again without the --listen=0.0.0.0:3001 and without the custom network, and everything works now!
I can post my quadlet files if someone want them to deploy Pangolin on quadlet, as reference, if it's helpful to someone.
Thanks you a lot, I can now begin to migrate!
Have a nice day, and thanks again

<!-- gh-comment-id:3371189997 --> @Froggy232 commented on GitHub (Oct 6, 2025): Very weirdly, I just tried again without the `--listen=0.0.0.0:3001` and without the custom network, and everything works now! I can post my quadlet files if someone want them to deploy Pangolin on quadlet, as reference, if it's helpful to someone. Thanks you a lot, I can now begin to migrate! Have a nice day, and thanks again
GiteaMirror commented 2026-05-16 02:51:06 -05:00
Author
Owner
Copy Link
Copy Source

@Froggy232 commented on GitHub (Oct 6, 2025):

So sorry, I was thinking it was working but gerbil still doesn't work it seems? I can access the dashboard but when I try to deploy a new site, I can only select local.
Also, gerbil logs tell me :

INFO: 2025/10/06 13:17:30 Fetching remote config from http://app:3001/api/v1/gerbil/get-config
ERROR: 2025/10/06 13:17:37 Error fetching remote config http://app:3001/api/v1/gerbil/get-config: Post "http://app:3001/api/v1/gerbil/get-config": dial tcp: lookup app on 127.0.0.53:53: server misbehaving
ERROR: 2025/10/06 13:17:37 Failed to load configuration: Post "http://app:3001/api/v1/gerbil/get-config": dial tcp: lookup app on 127.0.0.53:53: server misbehaving
INFO: 2025/10/06 13:17:42 Fetching remote config from http://app:3001/api/v1/gerbil/get-config

I fail to understand the problem, but as I said, I have deployed it without the Exec=--listen=0.0.0.0:3001 line, so maybe it's related? Sorry to request help so much, but it seems it's now very close to working.
Thanks again

<!-- gh-comment-id:3371648590 --> @Froggy232 commented on GitHub (Oct 6, 2025): So sorry, I was thinking it was working but gerbil still doesn't work it seems? I can access the dashboard but when I try to deploy a new site, I can only select local. Also, gerbil logs tell me : > INFO: 2025/10/06 13:17:30 Fetching remote config from http://app:3001/api/v1/gerbil/get-config > ERROR: 2025/10/06 13:17:37 Error fetching remote config http://app:3001/api/v1/gerbil/get-config: Post "http://app:3001/api/v1/gerbil/get-config": dial tcp: lookup app on 127.0.0.53:53: server misbehaving > ERROR: 2025/10/06 13:17:37 Failed to load configuration: Post "http://app:3001/api/v1/gerbil/get-config": dial tcp: lookup app on 127.0.0.53:53: server misbehaving > INFO: 2025/10/06 13:17:42 Fetching remote config from http://app:3001/api/v1/gerbil/get-config I fail to understand the problem, but as I said, I have deployed it without the `Exec=--listen=0.0.0.0:3001` line, so maybe it's related? Sorry to request help so much, but it seems it's now very close to working. Thanks again
GiteaMirror commented 2026-05-16 02:51:07 -05:00
Author
Owner
Copy Link
Copy Source

@Froggy232 commented on GitHub (Oct 9, 2025):

Hi again,

So, I continue to investigate, and the more I look at it the more it seems impossible to solve :

  • If I remove the option Network=host to the gerbil quadlet file, it fail few seconds after starting with the error : Failed to start proxy: failed to listen on port 443: listen tcp :443: bind: address already in use because obviously it's used by the pangolin pod and the traefik container (I don't even understand why gerbil need to listen on that port?)
  • If I add the option Network=host it also fails to start because it can't connect to the app container anymore (probably because they are not in the same network anymore? The logs are in the message before this one).

So far, I'm tempted to manually manage the VPN tunnel between my Pangolin instance, my clients and my homeserver manually, but it's pretty sad to not being able to use Pangolin and Newt for that, especially for the clients parts.

Again, if anyone can help I would accept it very heartfully, I would like so much to switch to pangolin...
Thanks again!

<!-- gh-comment-id:3385605044 --> @Froggy232 commented on GitHub (Oct 9, 2025): Hi again, So, I continue to investigate, and the more I look at it the more it seems impossible to solve : - If I remove the option `Network=host` to the gerbil quadlet file, it fail few seconds after starting with the error : `Failed to start proxy: failed to listen on port 443: listen tcp :443: bind: address already in use` because obviously it's used by the pangolin pod and the traefik container (I don't even understand why gerbil need to listen on that port?) - If I add the option `Network=host` it also fails to start because it can't connect to the app container anymore (probably because they are not in the same network anymore? The logs are in the message before this one). So far, I'm tempted to manually manage the VPN tunnel between my Pangolin instance, my clients and my homeserver manually, but it's pretty sad to not being able to use Pangolin and Newt for that, especially for the clients parts. Again, if anyone can help I would accept it very heartfully, I would like so much to switch to pangolin... Thanks again!
GiteaMirror commented 2026-05-16 02:51:07 -05:00
Author
Owner
Copy Link
Copy Source

@Froggy232 commented on GitHub (Oct 9, 2025):

I have some updates : if I remove the line sni 443 from gerbil quadlet file, everything seems to work this time! Dashboard is up, I have no error in logs and gerbil seems to work, I can add some VMs in the dashboard through wireguard or newt, but when I try to add a new site I always have an Internal Server Error.
Would someone have an idea? I will update this post with logs and screenshot soon.
Thanks again!

<!-- gh-comment-id:3386222377 --> @Froggy232 commented on GitHub (Oct 9, 2025): I have some updates : if I remove the line `sni 443` from gerbil quadlet file, everything seems to work this time! Dashboard is up, I have no error in logs and gerbil seems to work, I can add some VMs in the dashboard through wireguard or newt, but when I try to add a new site I always have an `Internal Server Error`. Would someone have an idea? I will update this post with logs and screenshot soon. Thanks again!
GiteaMirror commented 2026-05-16 02:51:08 -05:00
Author
Owner
Copy Link
Copy Source

@remogatto commented on GitHub (Oct 9, 2025):

I was struggling with this issue too but I think I have a fix. It's a timing issue. You need to wait for pangolin container to go healthy in order to successfully connect the other containers (gerbil, traefik). The fundamental change was adding Notify=healthy in the pangolin.container configuration file, together with a bunch of Health* keys. My quadlets pack follows:

proxy.pod

[Pod]
PodName=proxy
PublishPort=51820:51820/udp
PublishPort=21820:21820/udp
PublishPort=443:443
PublishPort=80:80

pangolin.container

[Unit]
Description=Pangolin

[Container]
Pod=proxy.pod
ContainerName=pangolin
Image=docker.io/fosrl/pangolin:latest
Volume=%h/srv/volumes/pangolin/config:/app/config:z
Volume=pangolin-data:/var/certificates
Volume=pangolin-data:/var/dynamic
Notify=healthy
HealthCmd="/usr/bin/curl -f http://localhost:3001/api/v1/"
HealthInterval=3s
HealthRetries=15
HealthTimeout=15s
AutoUpdate=registry

[Service]
Restart=always

[Install]
WantedBy=default.target

gerbil.container

[Unit]
Description=Gerbil

[Container]
Pod=proxy.pod
ContainerName=gerbil
AddCapability=NET_ADMIN SYS_MODULE
Image=docker.io/fosrl/gerbil:latest
Volume=%h/srv/volumes/pangolin/config:/var/config:z
Exec='--reachableAt=http://gerbil:3003' \
     '--generateAndSaveKeyTo=/var/config/key' \
     '--remoteConfig=http://pangolin:3001/api/v1/'
AutoUpdate=registry

[Service]
Restart=always

[Install]
WantedBy=default.target

traefik.container

[Container]
Pod=proxy.pod
AutoUpdate=registry
ContainerName=traefik
Exec='--configFile=/etc/traefik/traefik_config.yml'
Image=docker.io/traefik
Volume=%h/srv/volumes/pangolin/config/traefik:/etc/traefik:ro,z
Volume=%h/srv/volumes/pangolin/config/letsencrypt:/letsencrypt:z
Volume=%h/srv/volumes/pangolin/config/traefik/logs:/var/log/traefik:z
Volume=pangolin-data:/var/certificates
Volume=pangolin-data:/var/dynamix

[Service]
Restart=always

[Install]
WantedBy=default.target

Let me know if this solves your issue.

Note

  • I'm using podman 5.6.2 on Fedora 42.

Reference

  1. https://docs.podman.io/en/latest/markdown/podman-systemd.unit.5.html#notify-defaults-to-false
<!-- gh-comment-id:3386384322 --> @remogatto commented on GitHub (Oct 9, 2025): I was struggling with this issue too but I think I have a fix. It's a timing issue. You need to wait for pangolin container to go healthy in order to successfully connect the other containers (gerbil, traefik). The fundamental change was adding `Notify=healthy` in the `pangolin.container` configuration file, together with a bunch of Health* keys. My quadlets pack follows: ## proxy.pod ```ini [Pod] PodName=proxy PublishPort=51820:51820/udp PublishPort=21820:21820/udp PublishPort=443:443 PublishPort=80:80 ``` ## pangolin.container ```ini [Unit] Description=Pangolin [Container] Pod=proxy.pod ContainerName=pangolin Image=docker.io/fosrl/pangolin:latest Volume=%h/srv/volumes/pangolin/config:/app/config:z Volume=pangolin-data:/var/certificates Volume=pangolin-data:/var/dynamic Notify=healthy HealthCmd="/usr/bin/curl -f http://localhost:3001/api/v1/" HealthInterval=3s HealthRetries=15 HealthTimeout=15s AutoUpdate=registry [Service] Restart=always [Install] WantedBy=default.target ``` ## gerbil.container ```ini [Unit] Description=Gerbil [Container] Pod=proxy.pod ContainerName=gerbil AddCapability=NET_ADMIN SYS_MODULE Image=docker.io/fosrl/gerbil:latest Volume=%h/srv/volumes/pangolin/config:/var/config:z Exec='--reachableAt=http://gerbil:3003' \ '--generateAndSaveKeyTo=/var/config/key' \ '--remoteConfig=http://pangolin:3001/api/v1/' AutoUpdate=registry [Service] Restart=always [Install] WantedBy=default.target ``` ## traefik.container ```ini [Container] Pod=proxy.pod AutoUpdate=registry ContainerName=traefik Exec='--configFile=/etc/traefik/traefik_config.yml' Image=docker.io/traefik Volume=%h/srv/volumes/pangolin/config/traefik:/etc/traefik:ro,z Volume=%h/srv/volumes/pangolin/config/letsencrypt:/letsencrypt:z Volume=%h/srv/volumes/pangolin/config/traefik/logs:/var/log/traefik:z Volume=pangolin-data:/var/certificates Volume=pangolin-data:/var/dynamix [Service] Restart=always [Install] WantedBy=default.target ``` Let me know if this solves your issue. ## Note * I'm using podman 5.6.2 on Fedora 42. ## Reference 1. https://docs.podman.io/en/latest/markdown/podman-systemd.unit.5.html#notify-defaults-to-false
GiteaMirror commented 2026-05-16 02:51:09 -05:00
Author
Owner
Copy Link
Copy Source

@Froggy232 commented on GitHub (Oct 9, 2025):

Thanks you a lot, we have very similar setup so I hope it will work.
I will try right now, and report back.
Thanks again!

<!-- gh-comment-id:3386590966 --> @Froggy232 commented on GitHub (Oct 9, 2025): Thanks you a lot, we have very similar setup so I hope it will work. I will try right now, and report back. Thanks again!
GiteaMirror commented 2026-05-16 02:51:10 -05:00
Author
Owner
Copy Link
Copy Source

@Froggy232 commented on GitHub (Oct 10, 2025):

Hi,
Sorry for the delay, it seems everything works!!!
Thanks you so much, it seems that the hetzner vps self hosted installation is working fully, but when I try to deploy on my own server I get an internal server error everytime I try to access a ressource, I don't know why yet.
I will continue to investigate, but it seems like the final run haha.
Thanks again

<!-- gh-comment-id:3390732594 --> @Froggy232 commented on GitHub (Oct 10, 2025): Hi, Sorry for the delay, it seems everything works!!! Thanks you so much, it seems that the hetzner vps self hosted installation is working fully, but when I try to deploy on my own server I get an `internal server error` everytime I try to access a ressource, I don't know why yet. I will continue to investigate, but it seems like the final run haha. Thanks again
GiteaMirror commented 2026-05-16 02:51:12 -05:00
Author
Owner
Copy Link
Copy Source

@Froggy232 commented on GitHub (Oct 10, 2025):

I have a lot of http: TLS handshake error from 82.67.85.247:58756: read tcp 192.168.100.2:443->82.67.85.247:58756: read: connection reset by peer, maybe it's related? So far, I fail to understand why it works on my test VPS but not on my real server.

<!-- gh-comment-id:3390846254 --> @Froggy232 commented on GitHub (Oct 10, 2025): I have a lot of `http: TLS handshake error from 82.67.85.247:58756: read tcp 192.168.100.2:443->82.67.85.247:58756: read: connection reset by peer`, maybe it's related? So far, I fail to understand why it works on my test VPS but not on my real server.
GiteaMirror commented 2026-05-16 02:51:14 -05:00
Author
Owner
Copy Link
Copy Source

@Froggy232 commented on GitHub (Oct 11, 2025):

Ok, I had a problem with DNS but now everything works!
Thanks you a lot, pangolin seems very great!

<!-- gh-comment-id:3392991906 --> @Froggy232 commented on GitHub (Oct 11, 2025): Ok, I had a problem with DNS but now everything works! Thanks you a lot, pangolin seems very great!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
api
authentication
bug
config
dependencies
docker
documentation
enhancement
good first issue
help wanted
Improvement
Look Into
needs investigating
networking
new feature
non-critical bug
potential bug
pull-request
Mirrored from GitHub Pull Request
 question
reverse proxy
Security
stale
ui
wontfix
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
GiteaMirror ninjasurge (Kaehlen Rog)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/pangolin#14846