github-starred/pangolin
2
0
Fork 0
mirror of https://github.com/fosrl/pangolin.git synced 2026-05-07 21:30:36 -05:00
Code Issues 564 Packages Projects Releases 72 Wiki Activity

[GH-ISSUE #2418] HTTP Resources return 418 - TCP proxy not listening on internal ports via WireGuard #6960

New Issue
Closed
opened 2026-04-25 15:57:05 -05:00 by GiteaMirror · 1 comment
GiteaMirror commented 2026-04-25 15:57:05 -05:00
Owner
Copy Link

Originally created by @remibardalen on GitHub (Feb 5, 2026).
Original GitHub issue: https://github.com/fosrl/pangolin/issues/2418

Environment

  • Pangolin Server: 1.15.1
  • Newt: 1.9.0
  • Traefik: 3.6.7
  • Newt running with --network host

Description

HTTP Resources configured with sso=0 (public access) return HTTP 418 instead of reaching the backend. The TCP proxy starts according to Newt logs, but connections to the internal ports via WireGuard are refused.

Steps to Reproduce

  1. Create HTTPS Resource in Pangolin Dashboard
  2. Set target to internal server (e.g., 192.168.1.100:80)
  3. Disable "Use Platform SSO" (sso=0 in database)
  4. Access resource externally via browser or curl
  5. Result: HTTP 418 with empty body

Expected Behavior

Request should pass through WireGuard tunnel to backend and return content from the target server.

Actual Behavior

  • Badger middleware validates successfully (Traefik debug log: Badger: Valid session)
  • Traefik load balancer selects backend: http://100.89.128.4:<internalPort>
  • TCP connection to WireGuard IP on internal port is refused
  • HTTP 418 returned to client (empty body)

Debugging Done

1. WireGuard tunnel connectivity

# From Gerbil container - WORKS
ping 100.89.128.4
# 64 bytes from 100.89.128.4: seq=0 ttl=64 time=12ms

2. Pangolin API verify-session

curl -X POST 'http://localhost:3001/api/v1/badger/verify-session' \
  -H 'Content-Type: application/json' \
  -d '{"scheme":"https","host":"app.example.com","path":"/","method":"GET"}'
# Returns: {"valid":true,"message":"Access allowed"}

3. Newt logs show TCP proxy starting

INFO: Started tcp proxy to 192.168.1.100:80
INFO: Started tcp proxy to 192.168.1.100:80

4. But connection to internal port fails

# From Gerbil container - FAILS
wget http://100.89.128.4:<internalPort>
# wget: can't connect to remote host: Connection refused

5. Database configuration verified correct

SELECT name, sso, blockAccess, enabled FROM resources;
-- myapp|0|0|1

SELECT name, ip, port, internalPort FROM targets JOIN resources...;
-- myapp|192.168.1.100|80|57803

Attempted Fixes (none worked)

  • Restarted all containers (Pangolin, Gerbil, Traefik, Newt)
  • Recreated Newt container from scratch
  • Deleted and recreated HTTP resources in Dashboard
  • Toggled targets enabled=0 then enabled=1 to force resync
  • Verified Newt is latest version (1.9.0)

Additional Notes

  • Site Resources work correctly - VPN-style Layer 3 access functions as expected
  • Only HTTP Resources exhibit this behavior
  • The TCP proxy appears to start but doesn't bind to the WireGuard userspace netstack properly
  • Internal access (bypassing Pangolin, going directly to backend) works perfectly
Originally created by @remibardalen on GitHub (Feb 5, 2026). Original GitHub issue: https://github.com/fosrl/pangolin/issues/2418 ## Environment - Pangolin Server: 1.15.1 - Newt: 1.9.0 - Traefik: 3.6.7 - Newt running with `--network host` ## Description HTTP Resources configured with `sso=0` (public access) return HTTP 418 instead of reaching the backend. The TCP proxy starts according to Newt logs, but connections to the internal ports via WireGuard are refused. ## Steps to Reproduce 1. Create HTTPS Resource in Pangolin Dashboard 2. Set target to internal server (e.g., 192.168.1.100:80) 3. Disable "Use Platform SSO" (sso=0 in database) 4. Access resource externally via browser or curl 5. Result: HTTP 418 with empty body ## Expected Behavior Request should pass through WireGuard tunnel to backend and return content from the target server. ## Actual Behavior - Badger middleware validates successfully (Traefik debug log: `Badger: Valid session`) - Traefik load balancer selects backend: `http://100.89.128.4:<internalPort>` - TCP connection to WireGuard IP on internal port is refused - HTTP 418 returned to client (empty body) ## Debugging Done ### 1. WireGuard tunnel connectivity ```bash # From Gerbil container - WORKS ping 100.89.128.4 # 64 bytes from 100.89.128.4: seq=0 ttl=64 time=12ms ``` ### 2. Pangolin API verify-session ```bash curl -X POST 'http://localhost:3001/api/v1/badger/verify-session' \ -H 'Content-Type: application/json' \ -d '{"scheme":"https","host":"app.example.com","path":"/","method":"GET"}' # Returns: {"valid":true,"message":"Access allowed"} ``` ### 3. Newt logs show TCP proxy starting ``` INFO: Started tcp proxy to 192.168.1.100:80 INFO: Started tcp proxy to 192.168.1.100:80 ``` ### 4. But connection to internal port fails ```bash # From Gerbil container - FAILS wget http://100.89.128.4:<internalPort> # wget: can't connect to remote host: Connection refused ``` ### 5. Database configuration verified correct ```sql SELECT name, sso, blockAccess, enabled FROM resources; -- myapp|0|0|1 SELECT name, ip, port, internalPort FROM targets JOIN resources...; -- myapp|192.168.1.100|80|57803 ``` ## Attempted Fixes (none worked) - Restarted all containers (Pangolin, Gerbil, Traefik, Newt) - Recreated Newt container from scratch - Deleted and recreated HTTP resources in Dashboard - Toggled targets enabled=0 then enabled=1 to force resync - Verified Newt is latest version (1.9.0) ## Additional Notes - **Site Resources work correctly** - VPN-style Layer 3 access functions as expected - Only HTTP Resources exhibit this behavior - The TCP proxy appears to start but doesn't bind to the WireGuard userspace netstack properly - Internal access (bypassing Pangolin, going directly to backend) works perfectly
GiteaMirror commented 2026-04-25 15:57:07 -05:00
Author
Owner
Copy Link

@remibardalen commented on GitHub (Feb 5, 2026):

Root Cause Found & Solution

After extensive debugging, the issue is now resolved.

The Problem

When HTTP Resource targets point to a reverse proxy (like Traefik) that does HTTP→HTTPS redirects:

  1. VPS Traefik terminates SSL and makes HTTP request to backend via WireGuard
  2. Local Traefik returns 301 Moved Permanently to HTTPS
  3. VPS Traefik tries to connect via HTTPS to the WireGuard IP (100.89.128.x)
  4. Local Traefik expects proper SNI (whoami.example.com) but receives the IP address
  5. TLS error: "remote error: tls: unrecognized name"
  6. Result: 418 I'm a teapot or 502 Bad Gateway

The Solution

Point targets directly to backend services instead of through a local reverse proxy:

-- Instead of pointing to local Traefik (port 80/443):
UPDATE targets SET port = 9000, method = 'http' WHERE resourceId = <auth-resource>;
UPDATE targets SET port = 8080, method = 'http' WHERE resourceId = <app-resource>;

For services only available in Docker networks, expose them on a host port:

docker run -d --name myapp -p 8080:80 myapp:latest

Confirmed working from both desktop and mobile.

Suggestions for Future Improvements

1. Better Error Messages

The 418 I'm a teapot response doesn't indicate what's wrong. Consider returning 502 Bad Gateway with a descriptive error body, or log the actual TLS/connection error.

2. Dashboard Warnings

Add warnings in the resource configuration UI when target method is http and port is 80 or 443 (common reverse proxy ports), suggesting users point directly to their application.

3. Documentation

Add a troubleshooting section about:

  • Why HTTP resources might fail with 418/502
  • The HTTP→HTTPS redirect gotcha when using local reverse proxies
  • Best practice: Point targets directly to backend services

4. SNI Passthrough Option

Consider adding an option to set custom SNI when connecting to HTTPS backends.

5. Health Check Enhancement

When a target is configured, perform a connectivity test that detects redirect responses and TLS errors before saving.

Thanks for a great product!

<!-- gh-comment-id:3853393640 --> @remibardalen commented on GitHub (Feb 5, 2026): ## Root Cause Found & Solution After extensive debugging, the issue is now **resolved**. --- ### The Problem When HTTP Resource targets point to a reverse proxy (like Traefik) that does HTTP→HTTPS redirects: 1. VPS Traefik terminates SSL and makes HTTP request to backend via WireGuard 2. Local Traefik returns `301 Moved Permanently` to HTTPS 3. VPS Traefik tries to connect via HTTPS to the WireGuard IP (100.89.128.x) 4. Local Traefik expects proper SNI (`whoami.example.com`) but receives the IP address 5. TLS error: `"remote error: tls: unrecognized name"` 6. Result: `418 I'm a teapot` or `502 Bad Gateway` --- ### The Solution **Point targets directly to backend services instead of through a local reverse proxy:** ```sql -- Instead of pointing to local Traefik (port 80/443): UPDATE targets SET port = 9000, method = 'http' WHERE resourceId = <auth-resource>; UPDATE targets SET port = 8080, method = 'http' WHERE resourceId = <app-resource>; ``` For services only available in Docker networks, expose them on a host port: ```bash docker run -d --name myapp -p 8080:80 myapp:latest ``` **Confirmed working from both desktop and mobile.** --- ## Suggestions for Future Improvements ### 1. Better Error Messages The `418 I'm a teapot` response doesn't indicate what's wrong. Consider returning `502 Bad Gateway` with a descriptive error body, or log the actual TLS/connection error. ### 2. Dashboard Warnings Add warnings in the resource configuration UI when target method is `http` and port is `80` or `443` (common reverse proxy ports), suggesting users point directly to their application. ### 3. Documentation Add a troubleshooting section about: - Why HTTP resources might fail with 418/502 - The HTTP→HTTPS redirect gotcha when using local reverse proxies - Best practice: Point targets directly to backend services ### 4. SNI Passthrough Option Consider adding an option to set custom SNI when connecting to HTTPS backends. ### 5. Health Check Enhancement When a target is configured, perform a connectivity test that detects redirect responses and TLS errors before saving. --- Thanks for a great product!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
api
authentication
bug
config
dependencies
docker
documentation
enhancement
good first issue
help wanted
Improvement
Look Into
needs investigating
networking
new feature
non-critical bug
potential bug
pull-request
Mirrored from GitHub Pull Request
 question
reverse proxy
Security
stale
ui
wontfix
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/pangolin#6960
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?