HTTPS Host header not forwarded #89

Closed
opened 2025-11-13 11:49:37 -06:00 by GiteaMirror · 20 comments
Owner

Originally created by @JochenFroehlich on GitHub (Feb 15, 2025).

I'm running a traefik behind the newt tunnel.

On my traefik behind the newt tunnel i get the following errors:
2025-02-15T16:05:16+01:00 DBG log/log.go:245 > http: TLS handshake error from 172.16.100.1:44880: tls: no certificates configured
2025-02-15T16:05:17+01:00 DBG github.com/traefik/traefik/v3/pkg/tls/tlsmanager.go:216 > TLS: strict SNI enabled - No certificate found for domain: "", closing connection

It seams that pangolin ist not forwading the https host header.

As my traefik is configured for SNI and needs the hostname to decide to which backend the traffic needs to be send.
Other services behind the newt tunnel without SNI are working.

Originally created by @JochenFroehlich on GitHub (Feb 15, 2025). I'm running a traefik behind the newt tunnel. On my traefik behind the newt tunnel i get the following errors: 2025-02-15T16:05:16+01:00 DBG log/log.go:245 > http: TLS handshake error from 172.16.100.1:44880: tls: no certificates configured 2025-02-15T16:05:17+01:00 DBG github.com/traefik/traefik/v3/pkg/tls/tlsmanager.go:216 > TLS: strict SNI enabled - No certificate found for domain: "", closing connection It seams that pangolin ist not forwading the https host header. As my traefik is configured for SNI and needs the hostname to decide to which backend the traffic needs to be send. Other services behind the newt tunnel without SNI are working.
GiteaMirror added the stale label 2025-11-13 11:49:37 -06:00
Author
Owner

@oschwartz10612 commented on GitHub (Feb 15, 2025):

It looks like Traefik is passing the host header along by default: https://doc.traefik.io/traefik/routing/services/#pass-host-header

Here is an excerpt I just tested using the whoami container.

Hostname: db3b364a2cf4
IP: 127.0.0.1
IP: ::1
IP: 172.17.0.4
RemoteAddr: 172.17.0.1:34250
GET / HTTP/1.1
Host: test.mydomain.net
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:131.0) Gecko/20100101 Firefox/131.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/png,image/svg+xml,*/*;q=0.8
Accept-Encoding: gzip, br
Accept-Language: en-US,en;q=0.5
Alt-Used: test.mydomain.net
Priority: u=0, i
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: same-site
Sec-Fetch-User: ?1
Upgrade-Insecure-Requests: 1
X-Forwarded-For: 172.70.43.61
X-Forwarded-Host: test.mydomain.net
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: 11afa51cd854
X-Real-Ip: 172.70.43.61

Maybe you need to use the host that is hitting Pangolin to make this work?

@oschwartz10612 commented on GitHub (Feb 15, 2025): It looks like Traefik is passing the host header along by default: https://doc.traefik.io/traefik/routing/services/#pass-host-header Here is an excerpt I just tested using the whoami container. ``` Hostname: db3b364a2cf4 IP: 127.0.0.1 IP: ::1 IP: 172.17.0.4 RemoteAddr: 172.17.0.1:34250 GET / HTTP/1.1 Host: test.mydomain.net User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:131.0) Gecko/20100101 Firefox/131.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/png,image/svg+xml,*/*;q=0.8 Accept-Encoding: gzip, br Accept-Language: en-US,en;q=0.5 Alt-Used: test.mydomain.net Priority: u=0, i Sec-Fetch-Dest: document Sec-Fetch-Mode: navigate Sec-Fetch-Site: same-site Sec-Fetch-User: ?1 Upgrade-Insecure-Requests: 1 X-Forwarded-For: 172.70.43.61 X-Forwarded-Host: test.mydomain.net X-Forwarded-Port: 443 X-Forwarded-Proto: https X-Forwarded-Server: 11afa51cd854 X-Real-Ip: 172.70.43.61 ``` Maybe you need to use the host that is hitting Pangolin to make this work?
Author
Owner

@JochenFroehlich commented on GitHub (Feb 24, 2025):

The DNS-Name which I see under Host and X-Forwarded-Host) in the WhoAmI Container is the original one called by the client, not the one I enter in the pangolin dashboard below ressources -> connectivity -> "IP / Hostname".

@JochenFroehlich commented on GitHub (Feb 24, 2025): The DNS-Name which I see under Host and X-Forwarded-Host) in the WhoAmI Container is the original one called by the client, not the one I enter in the pangolin dashboard below ressources -> connectivity -> "IP / Hostname".
Author
Owner

@oschwartz10612 commented on GitHub (Feb 24, 2025):

Ahh I see. Yes I think this is expected behavior of a reverse proxy. The forwarded header is not the destination but the original request because downstream you want to know what was originally requested.

Perhaps we can look at in the future allowing rewriting of the host header. For now could you check out the following Traefik documentation and see if you can edit your config to meet your needs?

https://doc.traefik.io/traefik/middlewares/http/headers/

@oschwartz10612 commented on GitHub (Feb 24, 2025): Ahh I see. Yes I think this is expected behavior of a reverse proxy. The forwarded header is not the destination but the original request because downstream you want to know what was originally requested. Perhaps we can look at in the future allowing rewriting of the host header. For now could you check out the following Traefik documentation and see if you can edit your config to meet your needs? https://doc.traefik.io/traefik/middlewares/http/headers/
Author
Owner

@JochenFroehlich commented on GitHub (Feb 24, 2025):

On my local traefik i'll have no chance to work with the headers as they are encrypted, only after hitting the correct endpoint in the traefik i'll be able to analyze the headers further. That's why i rely on the SNI headers, as they are unencrypted.

@JochenFroehlich commented on GitHub (Feb 24, 2025): On my local traefik i'll have no chance to work with the headers as they are encrypted, only after hitting the correct endpoint in the traefik i'll be able to analyze the headers further. That's why i rely on the SNI headers, as they are unencrypted.
Author
Owner

@oschwartz10612 commented on GitHub (Feb 24, 2025):

You could look at editing the header config in the Traefik instance running in Pangolin.

@oschwartz10612 commented on GitHub (Feb 24, 2025): You could look at editing the header config in the Traefik instance running in Pangolin.
Author
Owner

@crocofied commented on GitHub (Mar 7, 2025):

Similiar issue here with headers, cant get OnlyOffice working with Nextcloud. We should definetly be able to edit the Headers

@crocofied commented on GitHub (Mar 7, 2025): Similiar issue here with headers, cant get OnlyOffice working with Nextcloud. We should definetly be able to edit the Headers
Author
Owner

@4nx commented on GitHub (Mar 31, 2025):

Same for me with every service I run behind an own traefik reverse proxy in my homelab and which I try to reach over pangolin. The host header is not forwarded and so the traefik reverse proxy in front of my authentik, unifi, paperless etc could not be made available behind pangolin.

The log says like @JochenFroehlich:

2025-03-31T17:44:32Z DBG github.com/traefik/traefik/v3/pkg/tls/tlsmanager.go:216 > TLS: strict SNI enabled - No certificate found for domain: "", closing connection
2025-03-31T17:44:32Z DBG log/log.go:245 > http: TLS handshake error from 192.168.2.81:50858: tls: no certificates configured

Because of the dynamic configurations of targets within pangolin you can also not workaround this via manual setups within the pangolin traefik configuration. So we need the feature of forwarding at least host headers or maybe even better custom headers within the ressources configuration of pangolin.

@4nx commented on GitHub (Mar 31, 2025): Same for me with every service I run behind an own traefik reverse proxy in my homelab and which I try to reach over pangolin. The host header is not forwarded and so the traefik reverse proxy in front of my authentik, unifi, paperless etc could not be made available behind pangolin. The log says like @JochenFroehlich: 2025-03-31T17:44:32Z DBG github.com/traefik/traefik/v3/pkg/tls/tlsmanager.go:216 > TLS: strict SNI enabled - No certificate found for domain: "", closing connection 2025-03-31T17:44:32Z DBG log/log.go:245 > http: TLS handshake error from 192.168.2.81:50858: tls: no certificates configured Because of the dynamic configurations of targets within pangolin you can also not workaround this via manual setups within the pangolin traefik configuration. So we need the feature of forwarding at least host headers or maybe even better custom headers within the ressources configuration of pangolin.
Author
Owner

@4nx commented on GitHub (Mar 31, 2025):

If I can see this correct there should be a "passHostHeader: true" within the generated loadBalancer configurations here:

https://github.com/fosrl/pangolin/blob/4be1d87602eef4d9580bf54339191cab5251ccfd/server/routers/traefik/getTraefikConfig.ts#L228

config_output.http.services![serviceName] = {
    loadBalancer: {
        passHostHeader: true, // <-- like this
        servers: targets
            .filter((target: Target) => {
                if (!target.enabled) {
                    return false;
                }
                if (
                    site.type === "local" ||
                    site.type === "wireguard"
                ) {
@4nx commented on GitHub (Mar 31, 2025): If I can see this correct there should be a "passHostHeader: true" within the generated loadBalancer configurations here: [https://github.com/fosrl/pangolin/blob/4be1d87602eef4d9580bf54339191cab5251ccfd/server/routers/traefik/getTraefikConfig.ts#L228](https://github.com/fosrl/pangolin/blob/4be1d87602eef4d9580bf54339191cab5251ccfd/server/routers/traefik/getTraefikConfig.ts#L228) ``` config_output.http.services![serviceName] = { loadBalancer: { passHostHeader: true, // <-- like this servers: targets .filter((target: Target) => { if (!target.enabled) { return false; } if ( site.type === "local" || site.type === "wireguard" ) { ```
Author
Owner

@4nx commented on GitHub (Mar 31, 2025):

@oschwartz10612 maybe you could have a look if my guess is correct.

@4nx commented on GitHub (Mar 31, 2025): @oschwartz10612 maybe you could have a look if my guess is correct.
Author
Owner

@oschwartz10612 commented on GitHub (Mar 31, 2025):

@4nx I think by default the passHostHeader is set to true? https://doc.traefik.io/traefik/routing/services/#pass-host-header

@oschwartz10612 commented on GitHub (Mar 31, 2025): @4nx I think by default the `passHostHeader` is set to true? https://doc.traefik.io/traefik/routing/services/#pass-host-header
Author
Owner

@4nx commented on GitHub (Mar 31, 2025):

@oschwartz10612 You are right. I verified that this is not the problem. The problem seems to be that there should be a serversTransports so services will be called with the right SNI names.

Should be added here:

https://github.com/fosrl/pangolin/blob/4be1d87602eef4d9580bf54339191cab5251ccfd/server/routers/traefik/getTraefikConfig.ts#L96

const config_output: any = {
    http: {
        middlewares: { ... },
        routers: { ... },
        services: { ... },
        serversTransports: {} // <-- here
    }
};

and again here without the passHostHeader:

https://github.com/fosrl/pangolin/blob/4be1d87602eef4d9580bf54339191cab5251ccfd/server/routers/traefik/getTraefikConfig.ts#L228

const serversTransportName = `transport-${serviceName}`;

config_output.http.services![serviceName] = {
    loadBalancer: {
        serversTransport: serversTransportName, // <-- NEW
        servers: targets
            .filter((target: Target) => {
                if (!target.enabled) {
                    return false;
                }
                if (site.type === "local" || site.type === "wireguard") {
                    if (!target.ip || !target.port || !target.method) {
                        return false;
                    }
                } else if (site.type === "newt") {
                    if (!target.internalPort || !target.method) {
                        return false;
                    }
                }
                return true;
            })
            .map((target: Target) => {
                if (site.type === "local" || site.type === "wireguard") {
                    return {
                        url: `${target.method}://${target.ip}:${target.port}`
                    };
                } else if (site.type === "newt") {
                    const ip = site.subnet.split("/")[0];
                    return {
                        url: `${target.method}://${ip}:${target.internalPort}`
                    };
                }
            })
    }
};

config_output.http.serversTransports![serversTransportName] = {
    insecureSkipVerify: true,
    serverName: fullDomain // <-- hostname for SNI
};

Something like this. Be advised that I reviewed this via AI because I am not good in TS.

@4nx commented on GitHub (Mar 31, 2025): @oschwartz10612 You are right. I verified that this is not the problem. The problem seems to be that there should be a serversTransports so services will be called with the right SNI names. Should be added here: [https://github.com/fosrl/pangolin/blob/4be1d87602eef4d9580bf54339191cab5251ccfd/server/routers/traefik/getTraefikConfig.ts#L96](https://github.com/fosrl/pangolin/blob/4be1d87602eef4d9580bf54339191cab5251ccfd/server/routers/traefik/getTraefikConfig.ts#L96) ``` const config_output: any = { http: { middlewares: { ... }, routers: { ... }, services: { ... }, serversTransports: {} // <-- here } }; ``` and again here without the passHostHeader: [https://github.com/fosrl/pangolin/blob/4be1d87602eef4d9580bf54339191cab5251ccfd/server/routers/traefik/getTraefikConfig.ts#L228](https://github.com/fosrl/pangolin/blob/4be1d87602eef4d9580bf54339191cab5251ccfd/server/routers/traefik/getTraefikConfig.ts#L228) ``` const serversTransportName = `transport-${serviceName}`; config_output.http.services![serviceName] = { loadBalancer: { serversTransport: serversTransportName, // <-- NEW servers: targets .filter((target: Target) => { if (!target.enabled) { return false; } if (site.type === "local" || site.type === "wireguard") { if (!target.ip || !target.port || !target.method) { return false; } } else if (site.type === "newt") { if (!target.internalPort || !target.method) { return false; } } return true; }) .map((target: Target) => { if (site.type === "local" || site.type === "wireguard") { return { url: `${target.method}://${target.ip}:${target.port}` }; } else if (site.type === "newt") { const ip = site.subnet.split("/")[0]; return { url: `${target.method}://${ip}:${target.internalPort}` }; } }) } }; config_output.http.serversTransports![serversTransportName] = { insecureSkipVerify: true, serverName: fullDomain // <-- hostname for SNI }; ``` Something like this. Be advised that I reviewed this via AI because I am not good in TS.
Author
Owner

@achtnullzwei commented on GitHub (Apr 1, 2025):

+1 to be able to add serverName in serversTransport. I have most of my homelab services running on docker and exposing them via Traefik initially to leverage LE certificates. So running into the same issue at the moment. Would love to move away from Cloudflare Tunnels eventually. :)

@achtnullzwei commented on GitHub (Apr 1, 2025): +1 to be able to add serverName in serversTransport. I have most of my homelab services running on docker and exposing them via Traefik initially to leverage LE certificates. So running into the same issue at the moment. Would love to move away from Cloudflare Tunnels eventually. :)
Author
Owner

@4nx commented on GitHub (Apr 2, 2025):

@oschwartz10612 should we write a feature request cause the subject of this issue isn't really connected to the content anymore?

@4nx commented on GitHub (Apr 2, 2025): @oschwartz10612 should we write a feature request cause the subject of this issue isn't really connected to the content anymore?
Author
Owner

@miloschwartz commented on GitHub (Apr 2, 2025):

@4nx Yes, please do. That will be the most helpful for our documentation purposes Thanks!

@miloschwartz commented on GitHub (Apr 2, 2025): @4nx Yes, please do. That will be the most helpful for our documentation purposes Thanks!
Author
Owner

@4nx commented on GitHub (Apr 2, 2025):

done

@4nx commented on GitHub (Apr 2, 2025): done
Author
Owner

@asychev commented on GitHub (Apr 8, 2025):

I think I might hit the same issue, when target is a service behind Nginx Proxy manager:

{"level":"debug","error":"remote error: tls: unrecognized name","time":"2025-04-07T22:08:09Z","caller":"[github.com/traefik/traefik/v3/pkg/proxy/httputil/proxy.go:121](http://github.com/traefik/traefik/v3/pkg/proxy/httputil/proxy.go:121)","message":"502 Bad Gateway"}

@asychev commented on GitHub (Apr 8, 2025): I think I might hit the same issue, when target is a service behind Nginx Proxy manager: `{"level":"debug","error":"remote error: tls: unrecognized name","time":"2025-04-07T22:08:09Z","caller":"[github.com/traefik/traefik/v3/pkg/proxy/httputil/proxy.go:121](http://github.com/traefik/traefik/v3/pkg/proxy/httputil/proxy.go:121)","message":"502 Bad Gateway"}`
Author
Owner

@nulldoubt commented on GitHub (Apr 9, 2025):

Some apps, like Ollama, don't work properly when the host header is different. Usually one can pass --http-host-header="somehost:someport" when using a cloudflare tunnel and it'd fix that issue. Is there a way to change this in the traefik config, without modifying the code?

@nulldoubt commented on GitHub (Apr 9, 2025): Some apps, like Ollama, don't work properly when the host header is different. Usually one can pass `--http-host-header="somehost:someport"` when using a cloudflare tunnel and it'd fix that issue. Is there a way to change this in the traefik config, without modifying the code?
Author
Owner

@nulldoubt commented on GitHub (Apr 9, 2025):

Ok, nevermind. I found a solution to my specific issue. Ollama wasn't happy with the reverse proxy at first but binding it to 0.0.0.0 helped. It had nothing to do with traefik or Pangolin, sorry for disturbing.

@nulldoubt commented on GitHub (Apr 9, 2025): Ok, nevermind. I found a solution to my specific issue. Ollama wasn't happy with the reverse proxy at first but binding it to `0.0.0.0` helped. It had nothing to do with traefik or Pangolin, sorry for disturbing.
Author
Owner

@github-actions[bot] commented on GitHub (Apr 26, 2025):

This issue has been automatically marked as stale due to 14 days of inactivity. It will be closed in 14 days if no further activity occurs.

@github-actions[bot] commented on GitHub (Apr 26, 2025): This issue has been automatically marked as stale due to 14 days of inactivity. It will be closed in 14 days if no further activity occurs.
Author
Owner

@github-actions[bot] commented on GitHub (May 11, 2025):

This issue has been automatically closed due to inactivity. If you believe this is still relevant, please open a new issue with up-to-date information.

@github-actions[bot] commented on GitHub (May 11, 2025): This issue has been automatically closed due to inactivity. If you believe this is still relevant, please open a new issue with up-to-date information.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/pangolin#89