Compare commits

...

75 Commits
1.3.0 ... 1.4.1

Author SHA1 Message Date
Owen
bad244d0ea Merge branch 'main' into dev 2025-08-13 14:56:02 -07:00
Owen
0047b54e94 Dont override ENV
Fixes #101
2025-08-12 20:44:34 -07:00
Owen
f0c8d2c7c7 Change permissions to 0600
Fixes #104
2025-08-11 08:15:36 -07:00
Owen
151d0e38e6 Stop sending requests when you get a terminate 2025-08-03 14:47:36 -07:00
Owen Schwartz
3ccd755d55 Merge pull request #95 from fosrl/dependabot/go_modules/prod-patch-updates-e08645070f
Bump github.com/docker/docker from 28.3.2+incompatible to 28.3.3+incompatible in the prod-patch-updates group
2025-07-29 23:24:19 -07:00
Owen Schwartz
a0f0b674e8 Merge pull request #96 from firecat53/main
Update flake.nix to 1.4.0
2025-07-29 23:24:03 -07:00
Owen
9e675121d3 Dont reset dns 2025-07-29 22:42:54 -07:00
Owen
45d17da570 Fix the bind problem by just recreating the dev
TODO: WHY CANT WE REBIND TO A PORT - WE NEED TO FIX THIS BETTER
2025-07-29 20:58:48 -07:00
Owen
dfba35f8bb Use the tunnel ip 2025-07-29 16:31:42 -07:00
Scott Hansen
9e73aab21d Update flake.nix to 1.4.0 2025-07-29 14:14:42 -07:00
dependabot[bot]
e1ddad006a Bump github.com/docker/docker in the prod-patch-updates group
Bumps the prod-patch-updates group with 1 update: [github.com/docker/docker](https://github.com/docker/docker).


Updates `github.com/docker/docker` from 28.3.2+incompatible to 28.3.3+incompatible
- [Release notes](https://github.com/docker/docker/releases)
- [Commits](https://github.com/docker/docker/compare/v28.3.2...v28.3.3)

---
updated-dependencies:
- dependency-name: github.com/docker/docker
  dependency-version: 28.3.3+incompatible
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod-patch-updates
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-07-29 16:39:02 +00:00
Owen
29567d6e0b Dont print private key 2025-07-28 20:07:13 -07:00
Owen
47321ea9ad Update readme: env 2025-07-28 12:34:38 -07:00
Owen
abfc9d8efc Update readme: cli 2025-07-28 12:12:40 -07:00
Owen
c6929621e7 Merge branch 'main' into dev 2025-07-28 12:02:22 -07:00
Owen
46993203a3 Update readme 2025-07-28 12:02:10 -07:00
Owen
8306084354 SSH not ready 2025-07-28 12:02:09 -07:00
Owen
02c1e2b7d0 Compute kind of works now!? 2025-07-28 12:02:09 -07:00
Owen
ae7e2a1055 Clean up operation 2025-07-28 12:02:09 -07:00
Owen Schwartz
88f1335cff Merge pull request #93 from Lokowitz/sync-go-version
Sync go version
2025-07-28 11:59:10 -07:00
Owen
8bf9c9795b Netstack working 2025-07-27 10:25:34 -07:00
Marvin
5d343cd420 modified: go.mod
modified:   go.sum
2025-07-26 13:25:52 +00:00
Marvin
d1473b7e22 go.mod aktualisieren 2025-07-26 10:32:20 +02:00
Marvin
2efbd7dd6a Dockerfile aktualisieren 2025-07-26 10:31:53 +02:00
Marvin
82a3a39a1f .go-version aktualisieren 2025-07-26 10:31:35 +02:00
Marvin
df09193834 cicd.yml aktualisieren 2025-07-26 10:31:20 +02:00
Marvin
b2fe4e3b03 test.yml aktualisieren 2025-07-26 10:31:05 +02:00
Owen
e14d53087f Starting to work on option 2025-07-25 16:16:33 -07:00
Owen
3583270f73 Adding option for netstack 2025-07-25 16:16:00 -07:00
Owen
f5be05c55a Add flag 2025-07-25 16:14:25 -07:00
Owen
d09e3fbd60 Proxies working 2025-07-25 16:10:53 -07:00
Owen
493831b5f0 Pm working 2025-07-25 13:09:11 -07:00
Owen
9fc692c090 Proxy working? 2025-07-25 12:00:09 -07:00
Owen
ccb7008579 Just hp like olm 2025-07-25 11:42:36 -07:00
Owen
f17dbe1fef Use normal udp 2025-07-25 11:05:24 -07:00
Owen
27561f52ca Dont restart netstack 2025-07-25 11:01:54 -07:00
Owen
499ebcd928 Maybe its working? 2025-07-25 10:59:34 -07:00
Owen
40dfab31a5 Maybe basic func 2025-07-25 10:50:02 -07:00
Owen
56377ec87e Exit well 2025-07-24 20:46:33 -07:00
Owen
008be54c55 Add get config 2025-07-24 12:40:14 -07:00
Owen
64c22a94a4 Log to file optionally and update config locations 2025-07-24 12:01:53 -07:00
Owen Schwartz
468c93c581 Merge pull request #91 from fosrl/dependabot/go_modules/prod-minor-updates-17f8beca3b
Bump software.sslmate.com/src/go-pkcs12 from 0.5.0 to 0.6.0 in the prod-minor-updates group
2025-07-23 11:26:32 -07:00
Owen Schwartz
c53b859cda Merge pull request #92 from nepthar/patch-1
Nit: Typo fix in help string
2025-07-23 11:26:15 -07:00
Jordan Parker
6cd824baf2 Nit: Typo fix in help string 2025-07-23 10:25:11 -04:00
dependabot[bot]
d8c5182acd Bump software.sslmate.com/src/go-pkcs12 in the prod-minor-updates group
Bumps the prod-minor-updates group with 1 update: software.sslmate.com/src/go-pkcs12.


Updates `software.sslmate.com/src/go-pkcs12` from 0.5.0 to 0.6.0

---
updated-dependencies:
- dependency-name: software.sslmate.com/src/go-pkcs12
  dependency-version: 0.6.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod-minor-updates
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-07-23 10:01:13 +00:00
Owen
c8c4666d63 Change rm to keep 2025-07-22 11:36:31 -07:00
Owen
f1fcc13e66 Holepunch to the right endpoint 2025-07-21 17:04:05 -07:00
Owen Schwartz
52bbc2fe31 Merge pull request #90 from firecat53/main
Update flake.nix to 1.3.4
2025-07-21 14:54:22 -07:00
Scott Hansen
b5ee12f84a Update flake.nix for 1.3.4 2025-07-21 11:23:50 -07:00
Owen
510e78437c Add client type 2025-07-18 16:55:38 -07:00
Owen
e14cffce1c Merge branch 'main' into clients-fr 2025-07-18 16:53:27 -07:00
Owen
629a92ee81 Make client work for olm 2025-07-18 16:53:13 -07:00
Owen
56df75544d Adjust logging 2025-07-18 16:52:59 -07:00
Owen
5b2e743470 Remove defers causing bad file descriptor issues 2025-07-18 15:49:57 -07:00
Owen
b5025c142f Working on it 2025-07-18 15:25:39 -07:00
Owen
cd86e6b6de Dont ping if there is just 1 2025-07-17 15:01:02 -07:00
Owen
230c34e4e0 Make sure to only exclude if there is anouther 2025-07-16 21:15:03 -07:00
Owen
a038ce1458 Move docker messages to debug for #86 2025-07-14 10:16:06 -07:00
Owen Schwartz
cd83efd365 Merge pull request #85 from firecat53/main
Update flake to 1.3.2
2025-07-13 19:47:06 -07:00
Scott Hansen
702f39e870 Update flake to 1.3.2 2025-07-13 18:47:20 -07:00
Owen Schwartz
02b7ea51af Merge pull request #84 from fosrl/dev
Ping improvements, dependabot, hostname in docker
2025-07-13 16:26:11 -07:00
Owen
e8421364fc Merge branch 'woutervanelten-patch-2' into dev 2025-07-13 16:12:42 -07:00
Owen
7264bb7001 Merge branch 'patch-2' of github.com:woutervanelten/newt into woutervanelten-patch-2 2025-07-13 16:10:23 -07:00
Owen
86e262ac1e Merge branch 'main' into dev 2025-07-13 16:10:02 -07:00
Owen Schwartz
dcacc03e96 Merge pull request #81 from fosrl/dependabot/go_modules/prod-patch-updates-f7fa3bf88c
Bump github.com/vishvananda/netlink from 1.3.0 to 1.3.1 in the prod-patch-updates group
2025-07-13 16:09:48 -07:00
Owen
6f4469a5a4 Merge branch 'woutervanelten-patch-5' into dev 2025-07-13 16:09:33 -07:00
Owen
663e28329b Fix typo with _ 2025-07-13 16:08:32 -07:00
Owen
f513f97fc3 Working on better ping 2025-07-13 16:07:46 -07:00
dependabot[bot]
ce4f3e4cdf Bump github.com/vishvananda/netlink in the prod-patch-updates group
Bumps the prod-patch-updates group with 1 update: [github.com/vishvananda/netlink](https://github.com/vishvananda/netlink).


Updates `github.com/vishvananda/netlink` from 1.3.0 to 1.3.1
- [Release notes](https://github.com/vishvananda/netlink/releases)
- [Commits](https://github.com/vishvananda/netlink/compare/v1.3.0...v1.3.1)

---
updated-dependencies:
- dependency-name: github.com/vishvananda/netlink
  dependency-version: 1.3.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod-patch-updates
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-07-13 23:07:14 +00:00
Owen
58a74fce6f Merge branch 'patch-5' of github.com:woutervanelten/newt into woutervanelten-patch-5 2025-07-13 16:06:54 -07:00
Owen Schwartz
fc965abbc4 Merge pull request #82 from fosrl/dependabot/go_modules/prod-minor-updates-d992c0ea53
Bump the prod-minor-updates group with 2 updates
2025-07-13 16:05:52 -07:00
Owen
b881808cae Loosen up the ping intervals 2025-07-11 16:35:39 -07:00
dependabot[bot]
6160e4c8a6 Bump the prod-minor-updates group with 2 updates
Bumps the prod-minor-updates group with 2 updates: [golang.org/x/crypto](https://github.com/golang/crypto) and [golang.org/x/net](https://github.com/golang/net).


Updates `golang.org/x/crypto` from 0.39.0 to 0.40.0
- [Commits](https://github.com/golang/crypto/compare/v0.39.0...v0.40.0)

Updates `golang.org/x/net` from 0.41.0 to 0.42.0
- [Commits](https://github.com/golang/net/compare/v0.41.0...v0.42.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.40.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod-minor-updates
- dependency-name: golang.org/x/net
  dependency-version: 0.42.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod-minor-updates
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-07-11 09:32:02 +00:00
Wouter van Elten
4d343e3541 Update README.md for health check
Added explanation of health_file
2025-07-11 08:14:32 +02:00
Wouter van Elten
9348842e2c added use of hostname if available 2025-07-01 17:26:49 +02:00
24 changed files with 2391 additions and 396 deletions

View File

@@ -33,7 +33,7 @@ jobs:
- name: Install Go
uses: actions/setup-go@v4
with:
go-version: 1.23.1
go-version: 1.24
- name: Update version in main.go
run: |

View File

@@ -16,7 +16,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v4
with:
go-version: '1.23'
go-version: '1.24'
- name: Build go
run: go build

View File

@@ -1 +1 @@
1.23.2
1.24

View File

@@ -1,4 +1,4 @@
FROM golang:1.24.5-alpine AS builder
FROM golang:1.24-alpine AS builder
# Set the working directory inside the container
WORKDIR /app

250
README.md
View File

@@ -12,7 +12,7 @@ Newt is used with Pangolin and Gerbil as part of the larger system. See document
<img src="public/screenshots/preview.png" alt="Preview"/>
_Sample output of a Newt container connected to Pangolin and hosting various resource target proxies._
_Sample output of a Newt connected to Pangolin and hosting various resource target proxies._
## Key Functions
@@ -22,7 +22,7 @@ Using the Newt ID and a secret, the client will make HTTP requests to Pangolin t
### Receives WireGuard Control Messages
When Newt receives WireGuard control messages, it will use the information encoded (endpoint, public key) to bring up a WireGuard tunnel using [netstack](https://github.com/WireGuard/wireguard-go/blob/master/tun/netstack/examples/http_server.go) fully in user space. It will ping over the tunnel to ensure the peer on the Gerbil side is brought up.
When Newt receives WireGuard control messages, it will use the information encoded (endpoint, public key) to bring up a WireGuard tunnel using [netstack](https://github.com/WireGuard/wireguard-go/blob/master/tun/netstack/examples/http_server.go) fully in user space. It will ping over the tunnel to ensure the peer on the Gerbil side is brought up.
### Receives Proxy Control Messages
@@ -30,21 +30,56 @@ When Newt receives WireGuard control messages, it will use the information encod
## CLI Args
- `endpoint`: The endpoint where both Gerbil and Pangolin reside in order to connect to the websocket.
- `id`: Newt ID generated by Pangolin to identify the client.
- `secret`: A unique secret (not shared and kept private) used to authenticate the client ID with the websocket in order to receive commands.
- `mtu`: MTU for the internal WG interface. Default: 1280
- `dns`: DNS server to use to resolve the endpoint
- `log-level` (optional): The log level to use. Default: INFO
- `updown` (optional): A script to be called when targets are added or removed.
- `tls-client-cert` (optional): Client certificate (p12 or pfx) for mTLS. See [mTLS](#mtls)
- `docker-socket` (optional): Set the Docker socket to use the container discovery integration
- `docker-enforce-network-validation` (optional): Validate the container target is on the same network as the newt process
- `id`: Newt ID generated by Pangolin to identify the client.
- `secret`: A unique secret (not shared and kept private) used to authenticate the client ID with the websocket in order to receive commands.
- `endpoint`: The endpoint where both Gerbil and Pangolin reside in order to connect to the websocket.
- Example:
- `mtu` (optional): MTU for the internal WG interface. Default: 1280
- `dns` (optional): DNS server to use to resolve the endpoint. Default: 8.8.8.8
- `log-level` (optional): The log level to use (DEBUG, INFO, WARN, ERROR, FATAL). Default: INFO
- `docker-socket` (optional): Set the Docker socket to use the container discovery integration
- `ping-interval` (optional): Interval for pinging the server. Default: 3s
- `ping-timeout` (optional): Timeout for each ping. Default: 5s
- `updown` (optional): A script to be called when targets are added or removed.
- `tls-client-cert` (optional): Client certificate (p12 or pfx) for mTLS. See [mTLS](#mtls)
- `docker-enforce-network-validation` (optional): Validate the container target is on the same network as the newt process. Default: false
- `health-file` (optional): Check if connection to WG server (pangolin) is ok. creates a file if ok, removes it if not ok. Can be used with docker healtcheck to restart newt
- `accept-clients` (optional): Enable WireGuard server mode to accept incoming olm client connections. Default: false
- `generateAndSaveKeyTo` (optional): Path to save generated private key
- `native` (optional): Use native WireGuard interface when accepting clients (requires WireGuard kernel module and Linux, must run as root). Default: false (uses userspace netstack)
- `interface` (optional): Name of the WireGuard interface. Default: newt
- `keep-interface` (optional): Keep the WireGuard interface. Default: false
## Environment Variables
All CLI arguments can be set using environment variables as an alternative to command line flags. Environment variables are particularly useful when running Newt in containerized environments.
- `PANGOLIN_ENDPOINT`: Endpoint of your pangolin server (equivalent to `--endpoint`)
- `NEWT_ID`: Newt ID generated by Pangolin (equivalent to `--id`)
- `NEWT_SECRET`: Newt secret for authentication (equivalent to `--secret`)
- `MTU`: MTU for the internal WG interface. Default: 1280 (equivalent to `--mtu`)
- `DNS`: DNS server to use to resolve the endpoint. Default: 8.8.8.8 (equivalent to `--dns`)
- `LOG_LEVEL`: Log level (DEBUG, INFO, WARN, ERROR, FATAL). Default: INFO (equivalent to `--log-level`)
- `DOCKER_SOCKET`: Path to Docker socket for container discovery (equivalent to `--docker-socket`)
- `PING_INTERVAL`: Interval for pinging the server. Default: 3s (equivalent to `--ping-interval`)
- `PING_TIMEOUT`: Timeout for each ping. Default: 5s (equivalent to `--ping-timeout`)
- `UPDOWN_SCRIPT`: Path to updown script for target add/remove events (equivalent to `--updown`)
- `TLS_CLIENT_CERT`: Path to client certificate for mTLS (equivalent to `--tls-client-cert`)
- `DOCKER_ENFORCE_NETWORK_VALIDATION`: Validate container targets are on same network. Default: false (equivalent to `--docker-enforce-network-validation`)
- `HEALTH_FILE`: Path to health file for connection monitoring (equivalent to `--health-file`)
- `ACCEPT_CLIENTS`: Enable WireGuard server mode. Default: false (equivalent to `--accept-clients`)
- `GENERATE_AND_SAVE_KEY_TO`: Path to save generated private key (equivalent to `--generateAndSaveKeyTo`)
- `USE_NATIVE_INTERFACE`: Use native WireGuard interface (Linux only). Default: false (equivalent to `--native`)
- `INTERFACE`: Name of the WireGuard interface. Default: newt (equivalent to `--interface`)
- `KEEP_INTERFACE`: Keep the WireGuard interface after shutdown. Default: false (equivalent to `--keep-interface`)
- `CONFIG_FILE`: Load the config json from this file instead of in the home folder.
**Note**: When both environment variables and CLI arguments are provided, CLI arguments take precedence.
- Example:
```bash
./newt \
newt \
--id 31frd0uzbjvp721 \
--secret h51mmlknrvrwv8s4r1i210azhumt6isgbpyavxodibx1k2d6 \
--endpoint https://example.com
@@ -54,30 +89,116 @@ You can also run it with Docker compose. For example, a service in your `docker-
```yaml
services:
newt:
image: fosrl/newt
container_name: newt
restart: unless-stopped
environment:
- PANGOLIN_ENDPOINT=https://example.com
- NEWT_ID=2ix2t8xk22ubpfy
- NEWT_SECRET=nnisrfsdfc7prqsp9ewo1dvtvci50j5uiqotez00dgap0ii2
newt:
image: fosrl/newt
container_name: newt
restart: unless-stopped
environment:
- PANGOLIN_ENDPOINT=https://example.com
- NEWT_ID=2ix2t8xk22ubpfy
- NEWT_SECRET=nnisrfsdfc7prqsp9ewo1dvtvci50j5uiqotez00dgap0ii2
- HEALTH_FILE=/tmp/healthy
```
You can also pass the CLI args to the container:
```yaml
services:
newt:
image: fosrl/newt
container_name: newt
restart: unless-stopped
command:
- --id 31frd0uzbjvp721
- --secret h51mmlknrvrwv8s4r1i210azhumt6isgbpyavxodibx1k2d6
- --endpoint https://example.com
newt:
image: fosrl/newt
container_name: newt
restart: unless-stopped
command:
- --id 31frd0uzbjvp721
- --secret h51mmlknrvrwv8s4r1i210azhumt6isgbpyavxodibx1k2d6
- --endpoint https://example.com
- --health-file /tmp/healthy
```
## Accept Client Connections
When the `--accept-clients` flag is enabled (or `ACCEPT_CLIENTS=true` environment variable is set), Newt operates as a WireGuard server that can accept incoming client connections from other devices. This enables peer-to-peer connectivity through the Newt instance.
### How It Works
In client acceptance mode, Newt:
- **Creates a WireGuard service** that can accept incoming connections from other WireGuard clients
- **Starts a connection testing server** (WGTester) that responds to connectivity checks from remote clients
- **Manages peer configurations** dynamically based on Pangolin's instructions
- **Enables bidirectional communication** between the Newt instance and connected clients
### Use Cases
- **Site-to-site connectivity**: Connect multiple locations through a central Newt instance
- **Client access to private networks**: Allow remote clients to access resources behind the Newt instance
- **Development environments**: Provide developers secure access to internal services
### Client Tunneling Modes
Newt supports two WireGuard tunneling modes:
#### Userspace Mode (Default)
By default, Newt uses a fully userspace WireGuard implementation using [netstack](https://github.com/WireGuard/wireguard-go/blob/master/tun/netstack/examples/http_server.go). This mode:
- **Does not require root privileges**
- **Works on all supported platforms** (Linux, Windows, macOS)
- **Does not require WireGuard kernel module** to be installed
- **Runs entirely in userspace** - no system network interface is created
- **Is containerization-friendly** - works seamlessly in Docker containers
This is the recommended mode for most deployments, especially containerized environments.
In this mode, TCP and UDP is proxied out of newt from the remote client using TCP/UDP resources in Pangolin.
#### Native Mode (Linux only)
When using the `--native` flag or setting `USE_NATIVE_INTERFACE=true`, Newt uses the native WireGuard kernel module. This mode:
- **Requires root privileges** to create and manage network interfaces
- **Only works on Linux** with the WireGuard kernel module installed
- **Creates a real network interface** (e.g., `newt0`) on the system
- **May offer better performance** for high-throughput scenarios
- **Requires proper network permissions** and may conflict with existing network configurations
In this mode it functions like a traditional VPN interface - all data arrives on the interface and you must get it to the destination (or access things locally).
#### Native Mode Requirements
To use native mode:
1. Run on a Linux system
2. Install the WireGuard kernel module
3. Run Newt as root (`sudo`)
4. Ensure the system allows creation of network interfaces
Docker Compose example:
```yaml
services:
newt:
image: fosrl/newt
container_name: newt
restart: unless-stopped
environment:
- PANGOLIN_ENDPOINT=https://example.com
- NEWT_ID=2ix2t8xk22ubpfy
- NEWT_SECRET=nnisrfsdfc7prqsp9ewo1dvtvci50j5uiqotez00dgap0ii2
- ACCEPT_CLIENTS=true
```
### Technical Details
When client acceptance is enabled:
- **WGTester Server**: Runs on `port + 1` (e.g., if WireGuard uses port 51820, WGTester uses 51821)
- **Connection Testing**: Responds to UDP packets with magic header `0xDEADBEEF` for connectivity verification
- **Dynamic Configuration**: Peer configurations are managed remotely through Pangolin
- **Proxy Integration**: Can work with both userspace (netstack) and native WireGuard modes
**Note**: Client acceptance mode requires coordination with Pangolin for peer management and configuration distribution.
### Docker Socket Integration
Newt can integrate with the Docker socket to provide remote inspection of Docker containers. This allows Pangolin to query and retrieve detailed information about containers running on the Newt client, including metadata, network configuration, port mappings, and more.
@@ -88,26 +209,27 @@ You can specify the Docker socket path using the `--docker-socket` CLI argument
```yaml
services:
newt:
image: fosrl/newt
container_name: newt
restart: unless-stopped
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- PANGOLIN_ENDPOINT=https://example.com
- NEWT_ID=2ix2t8xk22ubpfy
- NEWT_SECRET=nnisrfsdfc7prqsp9ewo1dvtvci50j5uiqotez00dgap0ii2
- DOCKER_SOCKET=/var/run/docker.sock
newt:
image: fosrl/newt
container_name: newt
restart: unless-stopped
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- PANGOLIN_ENDPOINT=https://example.com
- NEWT_ID=2ix2t8xk22ubpfy
- NEWT_SECRET=nnisrfsdfc7prqsp9ewo1dvtvci50j5uiqotez00dgap0ii2
- DOCKER_SOCKET=/var/run/docker.sock
```
#### Hostnames vs IPs
When the Docker Socket Integration is used, depending on the network which Newt is run with, either the hostname (generally considered the container name) or the IP address of the container will be sent to Pangolin. Here are some of the scenarios where IPs or hostname of the container will be utilised:
- **Running in Network Mode 'host'**: IP addresses will be used
- **Running in Network Mode 'bridge'**: IP addresses will be used
- **Running in docker-compose without a network specification**: Docker compose creates a network for the compose by default, hostnames will be used
- **Running on docker-compose with defined network**: Hostnames will be used
- **Running in Network Mode 'host'**: IP addresses will be used
- **Running in Network Mode 'bridge'**: IP addresses will be used
- **Running in docker-compose without a network specification**: Docker compose creates a network for the compose by default, hostnames will be used
- **Running on docker-compose with defined network**: Hostnames will be used
### Docker Enforce Network Validation
@@ -127,7 +249,7 @@ You can pass in a updown script for Newt to call when it is adding or removing a
`--updown "python3 test.py"`
It will get called with args when a target is added:
It will get called with args when a target is added:
`python3 test.py add tcp localhost:8556`
`python3 test.py remove tcp localhost:8556`
@@ -136,18 +258,20 @@ Returning a string from the script in the format of a target (`ip:dst` so `10.0.
You can look at updown.py as a reference script to get started!
### mTLS
Newt supports mutual TLS (mTLS) authentication, if the server has been configured to request a client certificate.
* Only PKCS12 (.p12 or .pfx) file format is accepted
* The PKCS12 file must contain:
* Private key
* Public certificate
* CA certificate
* Encrypted PKCS12 files are currently not supported
- Only PKCS12 (.p12 or .pfx) file format is accepted
- The PKCS12 file must contain:
- Private key
- Public certificate
- CA certificate
- Encrypted PKCS12 files are currently not supported
Examples:
```bash
./newt \
newt \
--id 31frd0uzbjvp721 \
--secret h51mmlknrvrwv8s4r1i210azhumt6isgbpyavxodibx1k2d6 \
--endpoint https://example.com \
@@ -156,20 +280,20 @@ Examples:
```yaml
services:
newt:
image: fosrl/newt
container_name: newt
restart: unless-stopped
environment:
- PANGOLIN_ENDPOINT=https://example.com
- NEWT_ID=2ix2t8xk22ubpfy
- NEWT_SECRET=nnisrfsdfc7prqsp9ewo1dvtvci50j5uiqotez00dgap0ii2
- TLS_CLIENT_CERT=./client.p12
newt:
image: fosrl/newt
container_name: newt
restart: unless-stopped
environment:
- PANGOLIN_ENDPOINT=https://example.com
- NEWT_ID=2ix2t8xk22ubpfy
- NEWT_SECRET=nnisrfsdfc7prqsp9ewo1dvtvci50j5uiqotez00dgap0ii2
- TLS_CLIENT_CERT=./client.p12
```
## Build
### Container
### Container
Ensure Docker is installed.

132
clients.go Normal file
View File

@@ -0,0 +1,132 @@
package main
import (
"fmt"
"strings"
"github.com/fosrl/newt/logger"
"github.com/fosrl/newt/proxy"
"github.com/fosrl/newt/websocket"
"golang.zx2c4.com/wireguard/tun/netstack"
"github.com/fosrl/newt/wgnetstack"
"github.com/fosrl/newt/wgtester"
)
var wgService *wgnetstack.WireGuardService
var wgTesterServer *wgtester.Server
var ready bool
func setupClients(client *websocket.Client) {
var host = endpoint
if strings.HasPrefix(host, "http://") {
host = strings.TrimPrefix(host, "http://")
} else if strings.HasPrefix(host, "https://") {
host = strings.TrimPrefix(host, "https://")
}
host = strings.TrimSuffix(host, "/")
if useNativeInterface {
setupClientsNative(client, host)
} else {
setupClientsNetstack(client, host)
}
ready = true
}
func setupClientsNetstack(client *websocket.Client, host string) {
logger.Info("Setting up clients with netstack...")
// Create WireGuard service
wgService, err = wgnetstack.NewWireGuardService(interfaceName, mtuInt, generateAndSaveKeyTo, host, id, client, "8.8.8.8")
if err != nil {
logger.Fatal("Failed to create WireGuard service: %v", err)
}
// // Set up callback to restart wgtester with netstack when WireGuard is ready
wgService.SetOnNetstackReady(func(tnet *netstack.Net) {
wgTesterServer = wgtester.NewServerWithNetstack("0.0.0.0", wgService.Port, id, tnet) // TODO: maybe make this the same ip of the wg server?
err := wgTesterServer.Start()
if err != nil {
logger.Error("Failed to start WireGuard tester server: %v", err)
}
})
wgService.SetOnNetstackClose(func() {
if wgTesterServer != nil {
wgTesterServer.Stop()
wgTesterServer = nil
}
})
client.OnTokenUpdate(func(token string) {
wgService.SetToken(token)
})
}
func setDownstreamTNetstack(tnet *netstack.Net) {
if wgService != nil {
wgService.SetOthertnet(tnet)
}
}
func closeClients() {
logger.Info("Closing clients...")
if wgService != nil {
wgService.Close(!keepInterface)
wgService = nil
}
closeWgServiceNative()
if wgTesterServer != nil {
wgTesterServer.Stop()
wgTesterServer = nil
}
}
func clientsHandleNewtConnection(publicKey string, endpoint string) {
if !ready {
return
}
// split off the port from the endpoint
parts := strings.Split(endpoint, ":")
if len(parts) < 2 {
logger.Error("Invalid endpoint format: %s", endpoint)
return
}
endpoint = strings.Join(parts[:len(parts)-1], ":")
if wgService != nil {
wgService.StartHolepunch(publicKey, endpoint)
}
clientsHandleNewtConnectionNative(publicKey, endpoint)
}
func clientsOnConnect() {
if !ready {
return
}
if wgService != nil {
wgService.LoadRemoteConfig()
}
clientsOnConnectNative()
}
func clientsAddProxyTarget(pm *proxy.ProxyManager, tunnelIp string) {
if !ready {
return
}
// add a udp proxy for localost and the wgService port
// TODO: make sure this port is not used in a target
if wgService != nil {
pm.AddTarget("udp", tunnelIp, int(wgService.Port), fmt.Sprintf("127.0.0.1:%d", wgService.Port))
}
clientsAddProxyTargetNative(pm, tunnelIp)
}

View File

@@ -26,6 +26,8 @@ type Container struct {
Labels map[string]string `json:"labels"`
Created int64 `json:"created"`
Networks map[string]Network `json:"networks"`
Hostname string `json:"hostname"` // added to use hostname if available instead of network address
}
// Port represents a port mapping for a Docker container
@@ -173,6 +175,14 @@ func ListContainers(socketPath string, enforceNetworkValidation bool) ([]Contain
// Short ID like docker ps
shortId := c.ID[:12]
// Inspect container to get hostname
hostname := ""
containerInfo, err := cli.ContainerInspect(ctx, c.ID)
if err == nil && containerInfo.Config != nil {
hostname = containerInfo.Config.Hostname
}
// Skip host container if set
if hostContainerId != "" && c.ID == hostContainerId {
continue
@@ -238,6 +248,7 @@ func ListContainers(socketPath string, enforceNetworkValidation bool) ([]Contain
Labels: c.Labels,
Created: c.Created,
Networks: networks,
Hostname: hostname, // added
}
dockerContainers = append(dockerContainers, dockerContainer)

6
flake.lock generated
View File

@@ -2,11 +2,11 @@
"nodes": {
"nixpkgs": {
"locked": {
"lastModified": 1749086602,
"narHash": "sha256-DJcgJMekoxVesl9kKjfLPix2Nbr42i7cpEHJiTnBUwU=",
"lastModified": 1753489912,
"narHash": "sha256-uDCFHeXdRIgJpYmtcUxGEsZ+hYlLPBhR83fdU+vbC1s=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "4792576cb003c994bd7cc1edada3129def20b27d",
"rev": "13e8d35b7d6028b7198f8186bc0347c6abaa2701",
"type": "github"
},
"original": {

View File

@@ -27,11 +27,11 @@
default = self.packages.${system}.pangolin-newt;
pangolin-newt = pkgs.buildGoModule {
pname = "pangolin-newt";
version = "1.2.1";
version = "1.4.0";
src = ./.;
vendorHash = "sha256-Yc5IXnShciek/bKkVezkAcaq47zGiZP8vUHFb9p09LI=";
vendorHash = "sha256-V8sq7XD/HJFKjhggrDWPdEEq3hjz0IHzpybQXA8Z/pg=";
meta = with pkgs.lib; {
description = "A tunneling client for Pangolin";

42
go.mod
View File

@@ -1,35 +1,33 @@
module github.com/fosrl/newt
go 1.23.1
toolchain go1.23.2
go 1.24
require (
github.com/docker/docker v28.3.2+incompatible
github.com/docker/docker v28.3.3+incompatible
github.com/google/gopacket v1.1.19
github.com/gorilla/websocket v1.5.3
github.com/vishvananda/netlink v1.3.0
golang.org/x/crypto v0.39.0
golang.org/x/exp v0.0.0-20250218142911-aa4b98e5adaa
golang.org/x/net v0.41.0
github.com/vishvananda/netlink v1.3.1
golang.org/x/crypto v0.40.0
golang.org/x/exp v0.0.0-20250718183923-645b1fa84792
golang.org/x/net v0.42.0
golang.zx2c4.com/wireguard v0.0.0-20250521234502-f333402bd9cb
golang.zx2c4.com/wireguard/wgctrl v0.0.0-20241231184526-a9ab2273dd10
gvisor.dev/gvisor v0.0.0-20250503011706-39ed1f5ac29c
software.sslmate.com/src/go-pkcs12 v0.5.0
software.sslmate.com/src/go-pkcs12 v0.6.0
)
require (
github.com/Microsoft/go-winio v0.6.0 // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/containerd/errdefs v1.0.0 // indirect
github.com/containerd/errdefs/pkg v0.3.0 // indirect
github.com/distribution/reference v0.6.0 // indirect
github.com/docker/go-connections v0.5.0 // indirect
github.com/docker/go-units v0.4.0 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/go-logr/logr v1.4.2 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/google/btree v1.1.2 // indirect
github.com/google/btree v1.1.3 // indirect
github.com/google/go-cmp v0.7.0 // indirect
github.com/josharian/native v1.1.0 // indirect
github.com/mdlayher/genetlink v1.3.2 // indirect
@@ -42,17 +40,15 @@ require (
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.1 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/vishvananda/netns v0.0.4 // indirect
github.com/vishvananda/netns v0.0.5 // indirect
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 // indirect
go.opentelemetry.io/otel v1.36.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.62.0 // indirect
go.opentelemetry.io/otel v1.37.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.36.0 // indirect
go.opentelemetry.io/otel/metric v1.36.0 // indirect
go.opentelemetry.io/otel/trace v1.36.0 // indirect
golang.org/x/mod v0.23.0 // indirect
golang.org/x/sync v0.11.0 // indirect
golang.org/x/sys v0.33.0 // indirect
golang.org/x/time v0.7.0 // indirect
golang.org/x/tools v0.30.0 // indirect
go.opentelemetry.io/otel/metric v1.37.0 // indirect
go.opentelemetry.io/otel/trace v1.37.0 // indirect
golang.org/x/sync v0.16.0 // indirect
golang.org/x/sys v0.34.0 // indirect
golang.org/x/time v0.12.0 // indirect
golang.zx2c4.com/wintun v0.0.0-20230126152724-0fa3db229ce2 // indirect
)

88
go.sum
View File

@@ -1,7 +1,7 @@
github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c h1:udKWzYgxTojEKWjV8V+WSxDXJ4NFATAsZjh8iIbsQIg=
github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E=
github.com/Microsoft/go-winio v0.6.0 h1:slsWYD/zyx7lCXoZVlvQrj0hPTM1HI4+v1sIda2yDvg=
github.com/Microsoft/go-winio v0.6.0/go.mod h1:cTAf44im0RAYeL23bpB+fzCyDH2MJiz2BO69KH/soAE=
github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
github.com/cenkalti/backoff v2.2.1+incompatible h1:tNowT99t7UNflLxfYYSlKYsBpXdEet03Pg2g16Swow4=
github.com/cenkalti/backoff/v5 v5.0.2 h1:rIfFVxEf1QsI7E1ZHfp/B4DF/6QBAUhmgkxc0H7Zss8=
github.com/cenkalti/backoff/v5 v5.0.2/go.mod h1:rkhZdG3JZukswDf7f0cwqPNk4K0sa+F97BxZthm/crw=
@@ -15,23 +15,23 @@ github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk=
github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
github.com/docker/docker v28.3.2+incompatible h1:wn66NJ6pWB1vBZIilP8G3qQPqHy5XymfYn5vsqeA5oA=
github.com/docker/docker v28.3.2+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker v28.3.3+incompatible h1:Dypm25kh4rmk49v1eiVbsAtpAsYURjYkaKubwuBdxEI=
github.com/docker/docker v28.3.3+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/go-connections v0.5.0 h1:USnMq7hx7gwdVZq1L49hLXaFtUdTADjXGp+uj1Br63c=
github.com/docker/go-connections v0.5.0/go.mod h1:ov60Kzw0kKElRwhNs9UlUHAE/F9Fe6GLaXnqyDdmEXc=
github.com/docker/go-units v0.4.0 h1:3uh0PgVws3nIA0Q+MwDC8yjEPf9zjRfZZWXZYDct3Tw=
github.com/docker/go-units v0.4.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY=
github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
github.com/google/btree v1.1.2 h1:xf4v41cLI2Z6FxbKm+8Bu+m8ifhj15JuZ9sa0jZCMUU=
github.com/google/btree v1.1.2/go.mod h1:qOPhT0dTNdNzV6Z/lhRX0YXUafgPLFUh+gZMl761Gm4=
github.com/google/btree v1.1.3 h1:CVpQJjYgC4VbzxeGVHfvZrv1ctoYCAI8vbl07Fcxlyg=
github.com/google/btree v1.1.3/go.mod h1:qOPhT0dTNdNzV6Z/lhRX0YXUafgPLFUh+gZMl761Gm4=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/gopacket v1.1.19 h1:ves8RnFZPGiFnTS0uPQStjwru6uO6h+nlr9j6fL7kF8=
@@ -76,76 +76,72 @@ github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/vishvananda/netlink v1.3.0 h1:X7l42GfcV4S6E4vHTsw48qbrV+9PVojNfIhZcwQdrZk=
github.com/vishvananda/netlink v1.3.0/go.mod h1:i6NetklAujEcC6fK0JPjT8qSwWyO0HLn4UKG+hGqeJs=
github.com/vishvananda/netns v0.0.4 h1:Oeaw1EM2JMxD51g9uhtC0D7erkIjgmj8+JZc26m1YX8=
github.com/vishvananda/netns v0.0.4/go.mod h1:SpkAiCQRtJ6TvvxPnOSyH3BMl6unz3xZlaprSwhNNJM=
github.com/vishvananda/netlink v1.3.1 h1:3AEMt62VKqz90r0tmNhog0r/PpWKmrEShJU0wJW6bV0=
github.com/vishvananda/netlink v1.3.1/go.mod h1:ARtKouGSTGchR8aMwmkzC0qiNPrrWO5JS/XMVl45+b4=
github.com/vishvananda/netns v0.0.5 h1:DfiHV+j8bA32MFM7bfEunvT8IAqQ/NzSJHtcmW5zdEY=
github.com/vishvananda/netns v0.0.5/go.mod h1:SpkAiCQRtJ6TvvxPnOSyH3BMl6unz3xZlaprSwhNNJM=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA=
go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 h1:F7Jx+6hwnZ41NSFTO5q4LYDtJRXBf2PD0rNBkeB/lus=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0/go.mod h1:UHB22Z8QsdRDrnAtX4PntOl36ajSxcdUMt1sF7Y6E7Q=
go.opentelemetry.io/otel v1.36.0 h1:UumtzIklRBY6cI/lllNZlALOF5nNIzJVb16APdvgTXg=
go.opentelemetry.io/otel v1.36.0/go.mod h1:/TcFMXYjyRNh8khOAO9ybYkqaDBb/70aVwkNML4pP8E=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.62.0 h1:Hf9xI/XLML9ElpiHVDNwvqI0hIFlzV8dgIr35kV1kRU=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.62.0/go.mod h1:NfchwuyNoMcZ5MLHwPrODwUF1HWCXWrL31s8gSAdIKY=
go.opentelemetry.io/otel v1.37.0 h1:9zhNfelUvx0KBfu/gb+ZgeAfAgtWrfHJZcAqFC228wQ=
go.opentelemetry.io/otel v1.37.0/go.mod h1:ehE/umFRLnuLa/vSccNq9oS1ErUlkkK71gMcN34UG8I=
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.36.0 h1:dNzwXjZKpMpE2JhmO+9HsPl42NIXFIFSUSSs0fiqra0=
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.36.0/go.mod h1:90PoxvaEB5n6AOdZvi+yWJQoE95U8Dhhw2bSyRqnTD0=
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.36.0 h1:nRVXXvf78e00EwY6Wp0YII8ww2JVWshZ20HfTlE11AM=
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.36.0/go.mod h1:r49hO7CgrxY9Voaj3Xe8pANWtr0Oq916d0XAmOoCZAQ=
go.opentelemetry.io/otel/metric v1.36.0 h1:MoWPKVhQvJ+eeXWHFBOPoBOi20jh6Iq2CcCREuTYufE=
go.opentelemetry.io/otel/metric v1.36.0/go.mod h1:zC7Ks+yeyJt4xig9DEw9kuUFe5C3zLbVjV2PzT6qzbs=
go.opentelemetry.io/otel/sdk v1.36.0 h1:b6SYIuLRs88ztox4EyrvRti80uXIFy+Sqzoh9kFULbs=
go.opentelemetry.io/otel/sdk v1.36.0/go.mod h1:+lC+mTgD+MUWfjJubi2vvXWcVxyr9rmlshZni72pXeY=
go.opentelemetry.io/otel/sdk/metric v1.36.0 h1:r0ntwwGosWGaa0CrSt8cuNuTcccMXERFwHX4dThiPis=
go.opentelemetry.io/otel/sdk/metric v1.36.0/go.mod h1:qTNOhFDfKRwX0yXOqJYegL5WRaW376QbB7P4Pb0qva4=
go.opentelemetry.io/otel/trace v1.36.0 h1:ahxWNuqZjpdiFAyrIoQ4GIiAIhxAunQR6MUoKrsNd4w=
go.opentelemetry.io/otel/trace v1.36.0/go.mod h1:gQ+OnDZzrybY4k4seLzPAWNwVBBVlF2szhehOBB/tGA=
go.opentelemetry.io/otel/metric v1.37.0 h1:mvwbQS5m0tbmqML4NqK+e3aDiO02vsf/WgbsdpcPoZE=
go.opentelemetry.io/otel/metric v1.37.0/go.mod h1:04wGrZurHYKOc+RKeye86GwKiTb9FKm1WHtO+4EVr2E=
go.opentelemetry.io/otel/sdk v1.37.0 h1:ItB0QUqnjesGRvNcmAcU0LyvkVyGJ2xftD29bWdDvKI=
go.opentelemetry.io/otel/sdk v1.37.0/go.mod h1:VredYzxUvuo2q3WRcDnKDjbdvmO0sCzOvVAiY+yUkAg=
go.opentelemetry.io/otel/sdk/metric v1.37.0 h1:90lI228XrB9jCMuSdA0673aubgRobVZFhbjxHHspCPc=
go.opentelemetry.io/otel/sdk/metric v1.37.0/go.mod h1:cNen4ZWfiD37l5NhS+Keb5RXVWZWpRE+9WyVCpbo5ps=
go.opentelemetry.io/otel/trace v1.37.0 h1:HLdcFNbRQBE2imdSEgm/kwqmQj1Or1l/7bW6mxVK7z4=
go.opentelemetry.io/otel/trace v1.37.0/go.mod h1:TlgrlQ+PtQO5XFerSPUYG0JSgGyryXewPGyayAWSBS0=
go.opentelemetry.io/proto/otlp v1.6.0 h1:jQjP+AQyTf+Fe7OKj/MfkDrmK4MNVtw2NpXsf9fefDI=
go.opentelemetry.io/proto/otlp v1.6.0/go.mod h1:cicgGehlFuNdgZkcALOCh3VE6K/u2tAjzlRhDwmVpZc=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.39.0 h1:SHs+kF4LP+f+p14esP5jAoDpHU8Gu/v9lFRK6IT5imM=
golang.org/x/crypto v0.39.0/go.mod h1:L+Xg3Wf6HoL4Bn4238Z6ft6KfEpN0tJGo53AAPC632U=
golang.org/x/exp v0.0.0-20250218142911-aa4b98e5adaa h1:t2QcU6V556bFjYgu4L6C+6VrCPyJZ+eyRsABUPs1mz4=
golang.org/x/exp v0.0.0-20250218142911-aa4b98e5adaa/go.mod h1:BHOTPb3L19zxehTsLoJXVaTktb06DFgmdW6Wb9s8jqk=
golang.org/x/crypto v0.40.0 h1:r4x+VvoG5Fm+eJcxMaY8CQM7Lb0l1lsmjGBQ6s8BfKM=
golang.org/x/crypto v0.40.0/go.mod h1:Qr1vMER5WyS2dfPHAlsOj01wgLbsyWtFn/aY+5+ZdxY=
golang.org/x/exp v0.0.0-20250718183923-645b1fa84792 h1:R9PFI6EUdfVKgwKjZef7QIwGcBKu86OEFpJ9nUEP2l4=
golang.org/x/exp v0.0.0-20250718183923-645b1fa84792/go.mod h1:A+z0yzpGtvnG90cToK5n2tu8UJVP2XUATh+r+sfOOOc=
golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.23.0 h1:Zb7khfcRGKk+kqfxFaP5tZqCnDZMjC5VtUBs87Hr6QM=
golang.org/x/mod v0.23.0/go.mod h1:6SkKJ3Xj0I0BrPOZoBy3bdMptDDU9oJrpohJ3eWZ1fY=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.41.0 h1:vBTly1HeNPEn3wtREYfy4GZ/NECgw2Cnl+nK6Nz3uvw=
golang.org/x/net v0.41.0/go.mod h1:B/K4NNqkfmg07DQYrbwvSluqCJOOXwUjeb/5lOisjbA=
golang.org/x/net v0.42.0 h1:jzkYrhi3YQWD6MLBJcsklgQsoAcw89EcZbJw8Z614hs=
golang.org/x/net v0.42.0/go.mod h1:FF1RA5d3u7nAYA4z2TkclSCKh68eSXtiFwcWQpPXdt8=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.11.0 h1:GGz8+XQP4FvTTrjZPzNKTMFtSXH80RAzG+5ghFPgK9w=
golang.org/x/sync v0.11.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sync v0.16.0 h1:ycBJEhp9p4vXvUZNszeOq0kGTPghopOL8q0fq3vstxw=
golang.org/x/sync v0.16.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.10.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.33.0 h1:q3i8TbbEz+JRD9ywIRlyRAQbM0qF7hu24q3teo2hbuw=
golang.org/x/sys v0.33.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/sys v0.34.0 h1:H5Y5sJ2L2JRdyv7ROF1he/lPdvFsd0mJHFw2ThKHxLA=
golang.org/x/sys v0.34.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.26.0 h1:P42AVeLghgTYr4+xUnTRKDMqpar+PtX7KWuNQL21L8M=
golang.org/x/text v0.26.0/go.mod h1:QK15LZJUUQVJxhz7wXgxSy/CJaTFjd0G+YLonydOVQA=
golang.org/x/time v0.7.0 h1:ntUhktv3OPE6TgYxXWv9vKvUSJyIFJlyohwbkEwPrKQ=
golang.org/x/time v0.7.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
golang.org/x/text v0.27.0 h1:4fGWRpyh641NLlecmyl4LOe6yDdfaYNrGb2zdfo4JV4=
golang.org/x/text v0.27.0/go.mod h1:1D28KMCvyooCX9hBiosv5Tz/+YLxj0j7XhWjpSUF7CU=
golang.org/x/time v0.12.0 h1:ScB/8o8olJvc+CQPWrK3fPZNfh7qgwCrY0zJmoEQLSE=
golang.org/x/time v0.12.0/go.mod h1:CDIdPxbZBQxdj6cxyCIdrNogrJKMJ7pr37NYpMcMDSg=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.30.0 h1:BgcpHewrV5AUp2G9MebG4XPFI1E2W41zU1SaqVA9vJY=
golang.org/x/tools v0.30.0/go.mod h1:c347cR/OJfw5TI+GfX7RUPNMdDRRbjvYTS0jPyvsVtY=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
@@ -171,5 +167,5 @@ gotest.tools/v3 v3.4.0 h1:ZazjZUfuVeZGLAmlKKuyv3IKP5orXcwtOwDQH6YVr6o=
gotest.tools/v3 v3.4.0/go.mod h1:CtbdzLSsqVhDgMtKsx03ird5YTGB3ar27v0u/yKBW5g=
gvisor.dev/gvisor v0.0.0-20250503011706-39ed1f5ac29c h1:m/r7OM+Y2Ty1sgBQ7Qb27VgIMBW8ZZhT4gLnUyDIhzI=
gvisor.dev/gvisor v0.0.0-20250503011706-39ed1f5ac29c/go.mod h1:3r5CMtNQMKIvBlrmM9xWUNamjKBYPOWyXOjmg5Kts3g=
software.sslmate.com/src/go-pkcs12 v0.5.0 h1:EC6R394xgENTpZ4RltKydeDUjtlM5drOYIG9c6TVj2M=
software.sslmate.com/src/go-pkcs12 v0.5.0/go.mod h1:Qiz0EyvDRJjjxGyUQa2cCNZn/wMyzrRJ/qcDXOQazLI=
software.sslmate.com/src/go-pkcs12 v0.6.0 h1:f3sQittAeF+pao32Vb+mkli+ZyT+VwKaD014qFGq6oU=
software.sslmate.com/src/go-pkcs12 v0.6.0/go.mod h1:Qiz0EyvDRJjjxGyUQa2cCNZn/wMyzrRJ/qcDXOQazLI=

View File

@@ -4,7 +4,8 @@ package main
import (
"fmt"
"strings"
"os"
"runtime"
"github.com/fosrl/newt/logger"
"github.com/fosrl/newt/proxy"
@@ -13,71 +14,61 @@ import (
"github.com/fosrl/newt/wgtester"
)
var wgService *wg.WireGuardService
var wgTesterServer *wgtester.Server
var wgServiceNative *wg.WireGuardService
func setupClients(client *websocket.Client) {
var host = endpoint
if strings.HasPrefix(host, "http://") {
host = strings.TrimPrefix(host, "http://")
} else if strings.HasPrefix(host, "https://") {
host = strings.TrimPrefix(host, "https://")
func setupClientsNative(client *websocket.Client, host string) {
if runtime.GOOS != "linux" {
logger.Fatal("Tunnel management is only supported on Linux right now!")
os.Exit(1)
}
host = strings.TrimSuffix(host, "/")
// make sure we are sudo
if os.Geteuid() != 0 {
logger.Fatal("You must run this program as root to manage tunnels on Linux.")
os.Exit(1)
}
// Create WireGuard service
wgService, err = wg.NewWireGuardService(interfaceName, mtuInt, generateAndSaveKeyTo, host, id, client)
wgServiceNative, err = wg.NewWireGuardService(interfaceName, mtuInt, generateAndSaveKeyTo, host, id, client)
if err != nil {
logger.Fatal("Failed to create WireGuard service: %v", err)
}
defer wgService.Close(rm)
wgTesterServer = wgtester.NewServer("0.0.0.0", wgService.Port, id) // TODO: maybe make this the same ip of the wg server?
wgTesterServer = wgtester.NewServer("0.0.0.0", wgServiceNative.Port, id) // TODO: maybe make this the same ip of the wg server?
err := wgTesterServer.Start()
if err != nil {
logger.Error("Failed to start WireGuard tester server: %v", err)
} else {
// Make sure to stop the server on exit
defer wgTesterServer.Stop()
}
client.OnTokenUpdate(func(token string) {
wgService.SetToken(token)
wgServiceNative.SetToken(token)
})
}
func closeClients() {
if wgService != nil {
wgService.Close(rm)
wgService = nil
}
if wgTesterServer != nil {
wgTesterServer.Stop()
wgTesterServer = nil
func closeWgServiceNative() {
if wgServiceNative != nil {
wgServiceNative.Close(!keepInterface)
wgServiceNative = nil
}
}
func clientsHandleNewtConnection(publicKey string) {
if wgService == nil {
return
func clientsOnConnectNative() {
if wgServiceNative != nil {
wgServiceNative.LoadRemoteConfig()
}
wgService.SetServerPubKey(publicKey)
}
func clientsOnConnect() {
if wgService == nil {
return
func clientsHandleNewtConnectionNative(publicKey, endpoint string) {
if wgServiceNative != nil {
wgServiceNative.StartHolepunch(publicKey, endpoint)
}
wgService.LoadRemoteConfig()
}
func clientsAddProxyTarget(pm *proxy.ProxyManager, tunnelIp string) {
if wgService == nil {
return
}
func clientsAddProxyTargetNative(pm *proxy.ProxyManager, tunnelIp string) {
// add a udp proxy for localost and the wgService port
// TODO: make sure this port is not used in a target
pm.AddTarget("udp", tunnelIp, int(wgService.Port), fmt.Sprintf("127.0.0.1:%d", wgService.Port))
if wgServiceNative != nil {
pm.AddTarget("udp", tunnelIp, int(wgServiceNative.Port), fmt.Sprintf("127.0.0.1:%d", wgServiceNative.Port))
}
}

View File

@@ -2,6 +2,7 @@ package logger
import (
"fmt"
"io"
"log"
"os"
"sync"
@@ -48,6 +49,11 @@ func (l *Logger) SetLevel(level LogLevel) {
l.level = level
}
// SetOutput sets the output destination for the logger
func (l *Logger) SetOutput(w io.Writer) {
l.logger.SetOutput(w)
}
// log handles the actual logging
func (l *Logger) log(level LogLevel, format string, args ...interface{}) {
if level < l.level {
@@ -120,3 +126,8 @@ func Error(format string, args ...interface{}) {
func Fatal(format string, args ...interface{}) {
GetLogger().Fatal(format, args...)
}
// SetOutput sets the output destination for the default logger
func SetOutput(w io.Writer) {
GetLogger().SetOutput(w)
}

292
main.go
View File

@@ -9,7 +9,7 @@ import (
"net/netip"
"os"
"os/signal"
"runtime"
"path/filepath"
"strconv"
"strings"
"syscall"
@@ -49,6 +49,10 @@ type ExitNodeData struct {
ExitNodes []ExitNode `json:"exitNodes"`
}
type SSHPublicKeyData struct {
PublicKey string `json:"publicKey"`
}
// ExitNode represents an exit node with an ID, endpoint, and weight.
type ExitNode struct {
ID int `json:"exitNodeId"`
@@ -80,19 +84,21 @@ var (
logLevel string
interfaceName string
generateAndSaveKeyTo string
rm bool
keepInterface bool
acceptClients bool
updownScript string
tlsPrivateKey string
dockerSocket string
dockerEnforceNetworkValidation string
dockerEnforceNetworkValidationBool bool
pingInterval = 2 * time.Second
pingTimeout = 3 * time.Second
pingInterval time.Duration
pingTimeout time.Duration
publicKey wgtypes.Key
pingStopChan chan struct{}
stopFunc func()
healthFile string
useNativeInterface bool
authorizedKeysFile string
)
func main() {
@@ -104,16 +110,24 @@ func main() {
dns = os.Getenv("DNS")
logLevel = os.Getenv("LOG_LEVEL")
updownScript = os.Getenv("UPDOWN_SCRIPT")
// interfaceName = os.Getenv("INTERFACE")
// generateAndSaveKeyTo = os.Getenv("GENERATE_AND_SAVE_KEY_TO")
// rm = os.Getenv("RM") == "true"
// acceptClients = os.Getenv("ACCEPT_CLIENTS") == "true"
interfaceName = os.Getenv("INTERFACE")
generateAndSaveKeyTo = os.Getenv("GENERATE_AND_SAVE_KEY_TO")
keepInterfaceEnv := os.Getenv("KEEP_INTERFACE")
acceptClientsEnv := os.Getenv("ACCEPT_CLIENTS")
useNativeInterfaceEnv := os.Getenv("USE_NATIVE_INTERFACE")
keepInterface = keepInterfaceEnv == "true"
acceptClients = acceptClientsEnv == "true"
useNativeInterface = useNativeInterfaceEnv == "true"
tlsPrivateKey = os.Getenv("TLS_CLIENT_CERT")
dockerSocket = os.Getenv("DOCKER_SOCKET")
pingIntervalStr := os.Getenv("PING_INTERVAL")
pingTimeoutStr := os.Getenv("PING_TIMEOUT")
dockerEnforceNetworkValidation = os.Getenv("DOCKER_ENFORCE_NETWORK_VALIDATION")
healthFile = os.Getenv("HEALTH_FILE")
// authorizedKeysFile = os.Getenv("AUTHORIZED_KEYS_FILE")
authorizedKeysFile = ""
if endpoint == "" {
flag.StringVar(&endpoint, "endpoint", "", "Endpoint of your pangolin server")
@@ -136,14 +150,21 @@ func main() {
if updownScript == "" {
flag.StringVar(&updownScript, "updown", "", "Path to updown script to be called when targets are added or removed")
}
// if interfaceName == "" {
// flag.StringVar(&interfaceName, "interface", "wg1", "Name of the WireGuard interface")
// }
// if generateAndSaveKeyTo == "" {
// flag.StringVar(&generateAndSaveKeyTo, "generateAndSaveKeyTo", "/tmp/newtkey", "Path to save generated private key")
// }
// flag.BoolVar(&rm, "rm", false, "Remove the WireGuard interface")
// flag.BoolVar(&acceptClients, "accept-clients", false, "Accept clients on the WireGuard interface")
if interfaceName == "" {
flag.StringVar(&interfaceName, "interface", "newt", "Name of the WireGuard interface")
}
if generateAndSaveKeyTo == "" {
flag.StringVar(&generateAndSaveKeyTo, "generateAndSaveKeyTo", "", "Path to save generated private key")
}
if keepInterfaceEnv == "" {
flag.BoolVar(&keepInterface, "keep-interface", false, "Keep the WireGuard interface")
}
if useNativeInterfaceEnv == "" {
flag.BoolVar(&useNativeInterface, "native", false, "Use native WireGuard interface (requires WireGuard kernel module) and linux")
}
if acceptClientsEnv == "" {
flag.BoolVar(&acceptClients, "accept-clients", false, "Accept clients on the WireGuard interface")
}
if tlsPrivateKey == "" {
flag.StringVar(&tlsPrivateKey, "tls-client-cert", "", "Path to client certificate used for mTLS")
}
@@ -151,26 +172,33 @@ func main() {
flag.StringVar(&dockerSocket, "docker-socket", "", "Path to Docker socket (typically /var/run/docker.sock)")
}
if pingIntervalStr == "" {
flag.StringVar(&pingIntervalStr, "ping-interval", "1s", "Interval for pinging the server (default 1s)")
flag.StringVar(&pingIntervalStr, "ping-interval", "3s", "Interval for pinging the server (default 3s)")
}
if pingTimeoutStr == "" {
flag.StringVar(&pingTimeoutStr, "ping-timeout", "2s", " Timeout for each ping (default 2s)")
flag.StringVar(&pingTimeoutStr, "ping-timeout", "5s", " Timeout for each ping (default 5s)")
}
// if authorizedKeysFile == "" {
// flag.StringVar(&authorizedKeysFile, "authorized-keys-file", "~/.ssh/authorized_keys", "Path to authorized keys file (if unset, no keys will be authorized)")
// }
if pingIntervalStr != "" {
pingInterval, err = time.ParseDuration(pingIntervalStr)
if err != nil {
fmt.Printf("Invalid PING_INTERVAL value: %s, using default 1 second\n", pingIntervalStr)
pingInterval = 1 * time.Second
fmt.Printf("Invalid PING_INTERVAL value: %s, using default 3 seconds\n", pingIntervalStr)
pingInterval = 3 * time.Second
}
} else {
pingInterval = 3 * time.Second
}
if pingTimeoutStr != "" {
pingTimeout, err = time.ParseDuration(pingTimeoutStr)
if err != nil {
fmt.Printf("Invalid PING_TIMEOUT value: %s, using default 2 seconds\n", pingTimeoutStr)
pingTimeout = 2 * time.Second
fmt.Printf("Invalid PING_TIMEOUT value: %s, using default 5 seconds\n", pingTimeoutStr)
pingTimeout = 5 * time.Second
}
} else {
pingTimeout = 5 * time.Second
}
if dockerEnforceNetworkValidation == "" {
@@ -224,6 +252,7 @@ func main() {
}
// Create a new client
client, err := websocket.NewClient(
"newt",
id, // CLI arg takes precedence
secret, // CLI arg takes precedence
endpoint,
@@ -234,6 +263,8 @@ func main() {
if err != nil {
logger.Fatal("Failed to create client: %v", err)
}
endpoint = client.GetConfig().Endpoint // Update endpoint from config
id = client.GetConfig().ID // Update ID from config
// output env var values if set
logger.Debug("Endpoint: %v", endpoint)
@@ -262,12 +293,6 @@ func main() {
var wgData WgData
if acceptClients {
// make sure we are running on linux
if runtime.GOOS != "linux" {
logger.Fatal("Tunnel management is only supported on Linux right now!")
os.Exit(1)
}
setupClients(client)
}
@@ -329,8 +354,6 @@ func main() {
return
}
clientsHandleNewtConnection(wgData.PublicKey)
logger.Debug("Received: %+v", msg)
tun, tnet, err = netstack.CreateNetTUN(
[]netip.Addr{netip.MustParseAddr(wgData.TunnelIP)},
@@ -340,6 +363,8 @@ func main() {
logger.Error("Failed to create TUN device: %v", err)
}
setDownstreamTNetstack(tnet)
// Create WireGuard device
dev = device.NewDevice(tun, conn.NewDefaultBind(), device.NewLogger(
mapToWireGuardLogLevel(loggerLevel),
@@ -360,6 +385,8 @@ func main() {
return
}
clientsHandleNewtConnection(wgData.PublicKey, endpoint)
// Configure WireGuard
config := fmt.Sprintf(`private_key=%s
public_key=%s
@@ -386,6 +413,15 @@ persistent_keepalive_interval=5`, fixKey(privateKey.String()), fixKey(wgData.Pub
close(pingWithRetryStopChan)
pingWithRetryStopChan = nil
}
// Use reliable ping for initial connection test
logger.Debug("Testing initial connection with reliable ping...")
_, err = reliablePing(tnet, wgData.ServerIP, pingTimeout, 5)
if err != nil {
logger.Warn("Initial reliable ping failed, but continuing: %v", err)
} else {
logger.Info("Initial connection test successful!")
}
pingWithRetryStopChan, _ = pingWithRetry(tnet, wgData.ServerIP, pingTimeout)
// Always mark as connected and start the proxy manager regardless of initial ping result
@@ -403,10 +439,18 @@ persistent_keepalive_interval=5`, fixKey(privateKey.String()), fixKey(wgData.Pub
// add the targets if there are any
if len(wgData.Targets.TCP) > 0 {
updateTargets(pm, "add", wgData.TunnelIP, "tcp", TargetData{Targets: wgData.Targets.TCP})
// Also update wgnetstack proxy manager
// if wgService != nil {
// updateTargets(wgService.GetProxyManager(), "add", wgData.TunnelIP, "tcp", TargetData{Targets: wgData.Targets.TCP})
// }
}
if len(wgData.Targets.UDP) > 0 {
updateTargets(pm, "add", wgData.TunnelIP, "udp", TargetData{Targets: wgData.Targets.UDP})
// Also update wgnetstack proxy manager
// if wgService != nil {
// updateTargets(wgService.GetProxyManager(), "add", wgData.TunnelIP, "udp", TargetData{Targets: wgData.Targets.UDP})
// }
}
clientsAddProxyTarget(pm, wgData.TunnelIP)
@@ -443,6 +487,11 @@ persistent_keepalive_interval=5`, fixKey(privateKey.String()), fixKey(wgData.Pub
// Close the WireGuard device and TUN
closeWgTunnel()
if stopFunc != nil {
stopFunc() // stop the ws from sending more requests
stopFunc = nil // reset stopFunc to nil to avoid double stopping
}
// Mark as disconnected
connected = false
@@ -475,6 +524,32 @@ persistent_keepalive_interval=5`, fixKey(privateKey.String()), fixKey(wgData.Pub
return
}
// If there is just one exit node, we can skip pinging it and use it directly
if len(exitNodes) == 1 {
logger.Debug("Only one exit node available, using it directly: %s", exitNodes[0].Endpoint)
// Prepare data to send to the cloud for selection
pingResults := []ExitNodePingResult{
{
ExitNodeID: exitNodes[0].ID,
LatencyMs: 0, // No ping latency since we are using it directly
Weight: exitNodes[0].Weight,
Error: "",
Name: exitNodes[0].Name,
Endpoint: exitNodes[0].Endpoint,
WasPreviouslyConnected: exitNodes[0].WasPreviouslyConnected,
},
}
stopFunc = client.SendMessageInterval("newt/wg/register", map[string]interface{}{
"publicKey": publicKey.String(),
"pingResults": pingResults,
"newtVersion": newtVersion,
}, 1*time.Second)
return
}
type nodeResult struct {
Node ExitNode
Latency time.Duration
@@ -484,11 +559,6 @@ persistent_keepalive_interval=5`, fixKey(privateKey.String()), fixKey(wgData.Pub
results := make([]nodeResult, len(exitNodes))
const pingAttempts = 3
for i, node := range exitNodes {
if connected && node.WasPreviouslyConnected {
logger.Info("Skipping ping for previously connected exit node so we pick another %d (%s)", node.ID, node.Endpoint)
continue
}
var totalLatency time.Duration
var lastErr error
successes := 0
@@ -544,6 +614,35 @@ persistent_keepalive_interval=5`, fixKey(privateKey.String()), fixKey(wgData.Pub
})
}
// If we were previously connected and there is at least one other good node,
// exclude the previously connected node from pingResults sent to the cloud so we don't try to reconnect to it
// This is to avoid issues where the previously connected node might be down or unreachable
if connected {
var filteredPingResults []ExitNodePingResult
previouslyConnectedNodeIdx := -1
for i, res := range pingResults {
if res.WasPreviouslyConnected {
previouslyConnectedNodeIdx = i
}
}
// Count good nodes (latency > 0, no error, not previously connected)
goodNodeCount := 0
for i, res := range pingResults {
if i != previouslyConnectedNodeIdx && res.LatencyMs > 0 && res.Error == "" {
goodNodeCount++
}
}
if previouslyConnectedNodeIdx != -1 && goodNodeCount > 0 {
for i, res := range pingResults {
if i != previouslyConnectedNodeIdx {
filteredPingResults = append(filteredPingResults, res)
}
}
pingResults = filteredPingResults
logger.Info("Excluding previously connected exit node from ping results due to other available nodes")
}
}
// Send the ping results to the cloud for selection
stopFunc = client.SendMessageInterval("newt/wg/register", map[string]interface{}{
"publicKey": publicKey.String(),
@@ -571,6 +670,11 @@ persistent_keepalive_interval=5`, fixKey(privateKey.String()), fixKey(wgData.Pub
if len(targetData.Targets) > 0 {
updateTargets(pm, "add", wgData.TunnelIP, "tcp", targetData)
// Also update wgnetstack proxy manager
// if wgService != nil && wgService.GetNetstackNet() != nil && wgService.GetProxyManager() != nil {
// updateTargets(wgService.GetProxyManager(), "add", wgData.TunnelIP, "tcp", targetData)
// }
}
})
@@ -591,6 +695,11 @@ persistent_keepalive_interval=5`, fixKey(privateKey.String()), fixKey(wgData.Pub
if len(targetData.Targets) > 0 {
updateTargets(pm, "add", wgData.TunnelIP, "udp", targetData)
// Also update wgnetstack proxy manager
// if wgService != nil && wgService.GetNetstackNet() != nil && wgService.GetProxyManager() != nil {
// updateTargets(wgService.GetProxyManager(), "add", wgData.TunnelIP, "udp", targetData)
// }
}
})
@@ -611,6 +720,11 @@ persistent_keepalive_interval=5`, fixKey(privateKey.String()), fixKey(wgData.Pub
if len(targetData.Targets) > 0 {
updateTargets(pm, "remove", wgData.TunnelIP, "udp", targetData)
// Also update wgnetstack proxy manager
// if wgService != nil && wgService.GetNetstackNet() != nil && wgService.GetProxyManager() != nil {
// updateTargets(wgService.GetProxyManager(), "remove", wgData.TunnelIP, "udp", targetData)
// }
}
})
@@ -631,15 +745,20 @@ persistent_keepalive_interval=5`, fixKey(privateKey.String()), fixKey(wgData.Pub
if len(targetData.Targets) > 0 {
updateTargets(pm, "remove", wgData.TunnelIP, "tcp", targetData)
// Also update wgnetstack proxy manager
// if wgService != nil && wgService.GetNetstackNet() != nil && wgService.GetProxyManager() != nil {
// updateTargets(wgService.GetProxyManager(), "remove", wgData.TunnelIP, "tcp", targetData)
// }
}
})
// Register handler for Docker socket check
client.RegisterHandler("newt/socket/check", func(msg websocket.WSMessage) {
logger.Info("Received Docker socket check request")
logger.Debug("Received Docker socket check request")
if dockerSocket == "" {
logger.Info("Docker socket path is not set")
logger.Debug("Docker socket path is not set")
err := client.SendMessage("newt/socket/status", map[string]interface{}{
"available": false,
"socketPath": dockerSocket,
@@ -667,10 +786,10 @@ persistent_keepalive_interval=5`, fixKey(privateKey.String()), fixKey(wgData.Pub
// Register handler for Docker container listing
client.RegisterHandler("newt/socket/fetch", func(msg websocket.WSMessage) {
logger.Info("Received Docker container fetch request")
logger.Debug("Received Docker container fetch request")
if dockerSocket == "" {
logger.Info("Docker socket path is not set")
logger.Debug("Docker socket path is not set")
return
}
@@ -696,6 +815,94 @@ persistent_keepalive_interval=5`, fixKey(privateKey.String()), fixKey(wgData.Pub
}
})
// EXPERIMENTAL: WHAT SHOULD WE DO ABOUT SECURITY?
client.RegisterHandler("newt/send/ssh/publicKey", func(msg websocket.WSMessage) {
logger.Debug("Received SSH public key request")
var sshPublicKeyData SSHPublicKeyData
jsonData, err := json.Marshal(msg.Data)
if err != nil {
logger.Info("Error marshaling data: %v", err)
return
}
if err := json.Unmarshal(jsonData, &sshPublicKeyData); err != nil {
logger.Info("Error unmarshaling SSH public key data: %v", err)
return
}
sshPublicKey := sshPublicKeyData.PublicKey
if authorizedKeysFile == "" {
logger.Debug("No authorized keys file set, skipping public key response")
return
}
// Expand tilde to home directory if present
expandedPath := authorizedKeysFile
if strings.HasPrefix(authorizedKeysFile, "~/") {
homeDir, err := os.UserHomeDir()
if err != nil {
logger.Error("Failed to get user home directory: %v", err)
return
}
expandedPath = filepath.Join(homeDir, authorizedKeysFile[2:])
}
// if it is set but the file does not exist, create it
if _, err := os.Stat(expandedPath); os.IsNotExist(err) {
logger.Debug("Authorized keys file does not exist, creating it: %s", expandedPath)
if err := os.MkdirAll(filepath.Dir(expandedPath), 0755); err != nil {
logger.Error("Failed to create directory for authorized keys file: %v", err)
return
}
if _, err := os.Create(expandedPath); err != nil {
logger.Error("Failed to create authorized keys file: %v", err)
return
}
}
// Check if the public key already exists in the file
fileContent, err := os.ReadFile(expandedPath)
if err != nil {
logger.Error("Failed to read authorized keys file: %v", err)
return
}
// Check if the key already exists (trim whitespace for comparison)
existingKeys := strings.Split(string(fileContent), "\n")
keyAlreadyExists := false
trimmedNewKey := strings.TrimSpace(sshPublicKey)
for _, existingKey := range existingKeys {
if strings.TrimSpace(existingKey) == trimmedNewKey && trimmedNewKey != "" {
keyAlreadyExists = true
break
}
}
if keyAlreadyExists {
logger.Info("SSH public key already exists in authorized keys file, skipping")
return
}
// append the public key to the authorized keys file
logger.Debug("Appending public key to authorized keys file: %s", sshPublicKey)
file, err := os.OpenFile(expandedPath, os.O_APPEND|os.O_WRONLY, 0644)
if err != nil {
logger.Error("Failed to open authorized keys file: %v", err)
return
}
defer file.Close()
if _, err := file.WriteString(sshPublicKey + "\n"); err != nil {
logger.Error("Failed to write public key to authorized keys file: %v", err)
return
}
logger.Info("SSH public key appended to authorized keys file")
})
client.OnConnect(func() error {
publicKey = privateKey.PublicKey()
logger.Debug("Public key: %s", publicKey)
@@ -734,10 +941,13 @@ persistent_keepalive_interval=5`, fixKey(privateKey.String()), fixKey(wgData.Pub
signal.Notify(sigCh, syscall.SIGINT, syscall.SIGTERM)
<-sigCh
dev.Close()
// Close clients first (including WGTester)
closeClients()
if dev != nil {
dev.Close()
}
if pm != nil {
pm.Stop()
}

View File

@@ -41,6 +41,23 @@ func NewProxyManager(tnet *netstack.Net) *ProxyManager {
}
}
// init function without tnet
func NewProxyManagerWithoutTNet() *ProxyManager {
return &ProxyManager{
tcpTargets: make(map[string]map[int]string),
udpTargets: make(map[string]map[int]string),
listeners: make([]*gonet.TCPListener, 0),
udpConns: make([]*gonet.UDPConn, 0),
}
}
// Function to add tnet to existing ProxyManager
func (pm *ProxyManager) SetTNet(tnet *netstack.Net) {
pm.mutex.Lock()
defer pm.mutex.Unlock()
pm.tnet = tnet
}
// AddTarget adds as new target for proxying
func (pm *ProxyManager) AddTarget(proto, listenIP string, port int, targetAddr string) error {
pm.mutex.Lock()
@@ -174,13 +191,13 @@ func (pm *ProxyManager) Stop() error {
pm.udpConns = append(pm.udpConns[:i], pm.udpConns[i+1:]...)
}
// Clear the target maps
for k := range pm.tcpTargets {
delete(pm.tcpTargets, k)
}
for k := range pm.udpTargets {
delete(pm.udpTargets, k)
}
// // Clear the target maps
// for k := range pm.tcpTargets {
// delete(pm.tcpTargets, k)
// }
// for k := range pm.udpTargets {
// delete(pm.udpTargets, k)
// }
// Give active connections a chance to close gracefully
time.Sleep(100 * time.Millisecond)
@@ -351,3 +368,23 @@ func (pm *ProxyManager) handleUDPProxy(conn *gonet.UDPConn, targetAddr string) {
}
}
}
// write a function to print out the current targets in the ProxyManager
func (pm *ProxyManager) PrintTargets() {
pm.mutex.RLock()
defer pm.mutex.RUnlock()
logger.Info("Current TCP Targets:")
for listenIP, targets := range pm.tcpTargets {
for port, targetAddr := range targets {
logger.Info("TCP %s:%d -> %s", listenIP, port, targetAddr)
}
}
logger.Info("Current UDP Targets:")
for listenIP, targets := range pm.udpTargets {
for port, targetAddr := range targets {
logger.Info("UDP %s:%d -> %s", listenIP, port, targetAddr)
}
}
}

Binary file not shown.

Before

Width:  |  Height:  |  Size: 774 KiB

After

Width:  |  Height:  |  Size: 93 KiB

18
stub.go
View File

@@ -7,26 +7,26 @@ import (
"github.com/fosrl/newt/websocket"
)
func setupClients(client *websocket.Client) {
func setupClientsNative(client *websocket.Client, host string) {
return // This function is not implemented for non-Linux systems.
}
func closeClients() {
// This function is not implemented for non-Linux systems.
func closeWgServiceNative() {
// No-op for non-Linux systems
return
}
func clientsHandleNewtConnection(publicKey string) {
// This function is not implemented for non-Linux systems.
func clientsOnConnectNative() {
// No-op for non-Linux systems
return
}
func clientsOnConnect() {
// This function is not implemented for non-Linux systems.
func clientsHandleNewtConnectionNative(publicKey, endpoint string) {
// No-op for non-Linux systems
return
}
func clientsAddProxyTarget(pm *proxy.ProxyManager, tunnelIp string) {
// This function is not implemented for non-Linux systems.
func clientsAddProxyTargetNative(pm *proxy.ProxyManager, tunnelIp string) {
// No-op for non-Linux systems
return
}

131
util.go
View File

@@ -45,9 +45,17 @@ func ping(tnet *netstack.Net, dst string, timeout time.Duration) (time.Duration,
}
defer socket.Close()
// Set socket buffer sizes to handle high bandwidth scenarios
if tcpConn, ok := socket.(interface{ SetReadBuffer(int) error }); ok {
tcpConn.SetReadBuffer(64 * 1024)
}
if tcpConn, ok := socket.(interface{ SetWriteBuffer(int) error }); ok {
tcpConn.SetWriteBuffer(64 * 1024)
}
requestPing := icmp.Echo{
Seq: rand.Intn(1 << 16),
Data: []byte("f"),
Data: []byte("newtping"),
}
icmpBytes, err := (&icmp.Message{Type: ipv4.ICMPTypeEcho, Code: 0, Body: &requestPing}).Marshal(nil)
@@ -65,12 +73,14 @@ func ping(tnet *netstack.Net, dst string, timeout time.Duration) (time.Duration,
return 0, fmt.Errorf("failed to write ICMP packet: %w", err)
}
n, err := socket.Read(icmpBytes[:])
// Use larger buffer for reading to handle potential network congestion
readBuffer := make([]byte, 1500)
n, err := socket.Read(readBuffer)
if err != nil {
return 0, fmt.Errorf("failed to read ICMP packet: %w", err)
}
replyPacket, err := icmp.ParseMessage(1, icmpBytes[:n])
replyPacket, err := icmp.ParseMessage(1, readBuffer[:n])
if err != nil {
return 0, fmt.Errorf("failed to parse ICMP packet: %w", err)
}
@@ -92,6 +102,51 @@ func ping(tnet *netstack.Net, dst string, timeout time.Duration) (time.Duration,
return latency, nil
}
// reliablePing performs multiple ping attempts with adaptive timeout
func reliablePing(tnet *netstack.Net, dst string, baseTimeout time.Duration, maxAttempts int) (time.Duration, error) {
var lastErr error
var totalLatency time.Duration
successCount := 0
for attempt := 1; attempt <= maxAttempts; attempt++ {
// Adaptive timeout: increase timeout for later attempts
timeout := baseTimeout + time.Duration(attempt-1)*500*time.Millisecond
// Add jitter to prevent thundering herd
jitter := time.Duration(rand.Intn(100)) * time.Millisecond
timeout += jitter
latency, err := ping(tnet, dst, timeout)
if err != nil {
lastErr = err
logger.Debug("Ping attempt %d/%d failed: %v", attempt, maxAttempts, err)
// Brief pause between attempts with exponential backoff
if attempt < maxAttempts {
backoff := time.Duration(attempt) * 50 * time.Millisecond
time.Sleep(backoff)
}
continue
}
totalLatency += latency
successCount++
// If we get at least one success, we can return early for health checks
if successCount > 0 {
avgLatency := totalLatency / time.Duration(successCount)
logger.Debug("Reliable ping succeeded after %d attempts, avg latency: %v", attempt, avgLatency)
return avgLatency, nil
}
}
if successCount == 0 {
return 0, fmt.Errorf("all %d ping attempts failed, last error: %v", maxAttempts, lastErr)
}
return totalLatency / time.Duration(successCount), nil
}
func pingWithRetry(tnet *netstack.Net, dst string, timeout time.Duration) (stopChan chan struct{}, err error) {
if healthFile != "" {
@@ -175,12 +230,14 @@ func pingWithRetry(tnet *netstack.Net, dst string, timeout time.Duration) (stopC
}
func startPingCheck(tnet *netstack.Net, serverIP string, client *websocket.Client) chan struct{} {
initialInterval := pingInterval
maxInterval := 3 * time.Second
currentInterval := initialInterval
maxInterval := 6 * time.Second
currentInterval := pingInterval
consecutiveFailures := 0
connectionLost := false
// Track recent latencies for adaptive timeout calculation
recentLatencies := make([]time.Duration, 0, 10)
pingStopChan := make(chan struct{})
go func() {
@@ -189,18 +246,52 @@ func startPingCheck(tnet *netstack.Net, serverIP string, client *websocket.Clien
for {
select {
case <-ticker.C:
_, err := ping(tnet, serverIP, pingTimeout)
// Calculate adaptive timeout based on recent latencies
adaptiveTimeout := pingTimeout
if len(recentLatencies) > 0 {
var sum time.Duration
for _, lat := range recentLatencies {
sum += lat
}
avgLatency := sum / time.Duration(len(recentLatencies))
// Use 3x average latency as timeout, with minimum of pingTimeout
adaptiveTimeout = avgLatency * 3
if adaptiveTimeout < pingTimeout {
adaptiveTimeout = pingTimeout
}
if adaptiveTimeout > 15*time.Second {
adaptiveTimeout = 15 * time.Second
}
}
// Use reliable ping with multiple attempts
maxAttempts := 2
if consecutiveFailures > 4 {
maxAttempts = 4 // More attempts when connection is unstable
}
latency, err := reliablePing(tnet, serverIP, adaptiveTimeout, maxAttempts)
if err != nil {
consecutiveFailures++
if consecutiveFailures == 1 {
// Track recent latencies (add a high value for failures)
recentLatencies = append(recentLatencies, adaptiveTimeout)
if len(recentLatencies) > 10 {
recentLatencies = recentLatencies[1:]
}
if consecutiveFailures < 2 {
logger.Debug("Periodic ping failed (%d consecutive failures): %v", consecutiveFailures, err)
} else {
logger.Warn("Periodic ping failed (%d consecutive failures): %v", consecutiveFailures, err)
}
if consecutiveFailures >= 3 && currentInterval < maxInterval {
// More lenient threshold for declaring connection lost under load
failureThreshold := 4
if consecutiveFailures >= failureThreshold && currentInterval < maxInterval {
if !connectionLost {
connectionLost = true
logger.Warn("Connection to server lost. Continuous reconnection attempts will be made.")
logger.Warn("Connection to server lost after %d failures. Continuous reconnection attempts will be made.", consecutiveFailures)
stopFunc = client.SendMessageInterval("newt/ping/request", map[string]interface{}{}, 3*time.Second)
// Send registration message to the server for backward compatibility
err := client.SendMessage("newt/wg/register", map[string]interface{}{
@@ -217,7 +308,7 @@ func startPingCheck(tnet *netstack.Net, serverIP string, client *websocket.Clien
}
}
}
currentInterval = time.Duration(float64(currentInterval) * 1.5)
currentInterval = time.Duration(float64(currentInterval) * 1.3) // Slower increase
if currentInterval > maxInterval {
currentInterval = maxInterval
}
@@ -225,9 +316,15 @@ func startPingCheck(tnet *netstack.Net, serverIP string, client *websocket.Clien
logger.Debug("Increased ping check interval to %v due to consecutive failures", currentInterval)
}
} else {
// Track recent latencies
recentLatencies = append(recentLatencies, latency)
if len(recentLatencies) > 10 {
recentLatencies = recentLatencies[1:]
}
if connectionLost {
connectionLost = false
logger.Info("Connection to server restored!")
logger.Info("Connection to server restored after %d failures!", consecutiveFailures)
if healthFile != "" {
err := os.WriteFile(healthFile, []byte("ok"), 0644)
if err != nil {
@@ -235,13 +332,13 @@ func startPingCheck(tnet *netstack.Net, serverIP string, client *websocket.Clien
}
}
}
if currentInterval > initialInterval {
currentInterval = time.Duration(float64(currentInterval) * 0.8)
if currentInterval < initialInterval {
currentInterval = initialInterval
if currentInterval > pingInterval {
currentInterval = time.Duration(float64(currentInterval) * 0.9) // Slower decrease
if currentInterval < pingInterval {
currentInterval = pingInterval
}
ticker.Reset(currentInterval)
logger.Info("Decreased ping check interval to %v after successful ping", currentInterval)
logger.Debug("Decreased ping check interval to %v after successful ping", currentInterval)
}
consecutiveFailures = 0
}

View File

@@ -34,6 +34,7 @@ type Client struct {
onConnect func() error
onTokenUpdate func(token string)
writeMux sync.Mutex
clientType string // Type of client (e.g., "newt", "olm")
}
type ClientOption func(*Client)
@@ -61,10 +62,10 @@ func (c *Client) OnTokenUpdate(callback func(token string)) {
c.onTokenUpdate = callback
}
// NewClient creates a new Newt client
func NewClient(newtID, secret string, endpoint string, pingInterval time.Duration, pingTimeout time.Duration, opts ...ClientOption) (*Client, error) {
// NewClient creates a new websocket client
func NewClient(clientType string, ID, secret string, endpoint string, pingInterval time.Duration, pingTimeout time.Duration, opts ...ClientOption) (*Client, error) {
config := &Config{
NewtID: newtID,
ID: ID,
Secret: secret,
Endpoint: endpoint,
}
@@ -78,6 +79,7 @@ func NewClient(newtID, secret string, endpoint string, pingInterval time.Duratio
isConnected: false,
pingInterval: pingInterval,
pingTimeout: pingTimeout,
clientType: clientType,
}
// Apply options before loading config
@@ -96,6 +98,10 @@ func NewClient(newtID, secret string, endpoint string, pingInterval time.Duratio
return client, nil
}
func (c *Client) GetConfig() *Config {
return c.config
}
// Connect establishes the WebSocket connection
func (c *Client) Connect() error {
go c.connectWithRetry()
@@ -199,12 +205,22 @@ func (c *Client) getToken() (string, error) {
}
}
var tokenData map[string]interface{}
// Get a new token
tokenData := map[string]interface{}{
"newtId": c.config.NewtID,
"secret": c.config.Secret,
if c.clientType == "newt" {
tokenData = map[string]interface{}{
"newtId": c.config.ID,
"secret": c.config.Secret,
}
} else if c.clientType == "olm" {
tokenData = map[string]interface{}{
"olmId": c.config.ID,
"secret": c.config.Secret,
}
}
jsonData, err := json.Marshal(tokenData)
if err != nil {
return "", fmt.Errorf("failed to marshal token request data: %w", err)
}
@@ -212,7 +228,7 @@ func (c *Client) getToken() (string, error) {
// Create a new request
req, err := http.NewRequest(
"POST",
baseEndpoint+"/api/v1/auth/newt/get-token",
baseEndpoint+"/api/v1/auth/"+c.clientType+"/get-token",
bytes.NewBuffer(jsonData),
)
if err != nil {
@@ -310,7 +326,7 @@ func (c *Client) establishConnection() error {
// Add token to query parameters
q := u.Query()
q.Set("token", token)
q.Set("clientType", "newt")
q.Set("clientType", c.clientType)
u.RawQuery = q.Encode()
// Connect to WebSocket

View File

@@ -8,30 +8,36 @@ import (
"runtime"
)
func getConfigPath() string {
var configDir string
switch runtime.GOOS {
case "darwin":
configDir = filepath.Join(os.Getenv("HOME"), "Library", "Application Support", "newt-client")
case "windows":
configDir = filepath.Join(os.Getenv("APPDATA"), "newt-client")
default: // linux and others
configDir = filepath.Join(os.Getenv("HOME"), ".config", "newt-client")
func getConfigPath(clientType string) string {
configFile := os.Getenv("CONFIG_FILE")
if configFile == "" {
var configDir string
switch runtime.GOOS {
case "darwin":
configDir = filepath.Join(os.Getenv("HOME"), "Library", "Application Support", clientType+"-client")
case "windows":
logDir := filepath.Join(os.Getenv("PROGRAMDATA"), "olm")
configDir = filepath.Join(logDir, clientType+"-client")
default: // linux and others
configDir = filepath.Join(os.Getenv("HOME"), ".config", clientType+"-client")
}
if err := os.MkdirAll(configDir, 0755); err != nil {
log.Printf("Failed to create config directory: %v", err)
}
return filepath.Join(configDir, "config.json")
}
if err := os.MkdirAll(configDir, 0755); err != nil {
log.Printf("Failed to create config directory: %v", err)
}
return filepath.Join(configDir, "config.json")
return configFile
}
func (c *Client) loadConfig() error {
if c.config.NewtID != "" && c.config.Secret != "" && c.config.Endpoint != "" {
if c.config.ID != "" && c.config.Secret != "" && c.config.Endpoint != "" {
return nil
}
configPath := getConfigPath()
configPath := getConfigPath(c.clientType)
data, err := os.ReadFile(configPath)
if err != nil {
if os.IsNotExist(err) {
@@ -45,8 +51,8 @@ func (c *Client) loadConfig() error {
return err
}
if c.config.NewtID == "" {
c.config.NewtID = config.NewtID
if c.config.ID == "" {
c.config.ID = config.ID
}
if c.config.Secret == "" {
c.config.Secret = config.Secret
@@ -63,7 +69,7 @@ func (c *Client) loadConfig() error {
}
func (c *Client) saveConfig() error {
configPath := getConfigPath()
configPath := getConfigPath(c.clientType)
data, err := json.MarshalIndent(c.config, "", " ")
if err != nil {
return err

View File

@@ -1,7 +1,7 @@
package websocket
type Config struct {
NewtID string `json:"newtId"`
ID string `json:"id"`
Secret string `json:"secret"`
Endpoint string `json:"endpoint"`
TlsClientCert string `json:"tlsClientCert"`

202
wg/wg.go
View File

@@ -4,6 +4,7 @@ package wg
import (
"encoding/json"
"errors"
"fmt"
"net"
"os"
@@ -48,21 +49,24 @@ type PeerReading struct {
}
type WireGuardService struct {
interfaceName string
mtu int
client *websocket.Client
wgClient *wgctrl.Client
config WgConfig
key wgtypes.Key
newtId string
lastReadings map[string]PeerReading
mu sync.Mutex
Port uint16
stopHolepunch chan struct{}
host string
serverPubKey string
token string
stopGetConfig chan struct{}
interfaceName string
mtu int
client *websocket.Client
wgClient *wgctrl.Client
config WgConfig
key wgtypes.Key
keyFilePath string
newtId string
lastReadings map[string]PeerReading
mu sync.Mutex
Port uint16
stopHolepunch chan struct{}
host string
serverPubKey string
holePunchEndpoint string
token string
stopGetConfig func()
interfaceCreated bool
}
// Add this type definition
@@ -149,25 +153,27 @@ func NewWireGuardService(interfaceName string, mtu int, generateAndSaveKeyTo str
var key wgtypes.Key
// if generateAndSaveKeyTo is provided, generate a private key and save it to the file. if the file already exists, load the key from the file
if _, err := os.Stat(generateAndSaveKeyTo); os.IsNotExist(err) {
// generate a new private key
key, err = wgtypes.GeneratePrivateKey()
if err != nil {
logger.Fatal("Failed to generate private key: %v", err)
}
// save the key to the file
err = os.WriteFile(generateAndSaveKeyTo, []byte(key.String()), 0644)
if err != nil {
logger.Fatal("Failed to save private key: %v", err)
}
} else {
keyData, err := os.ReadFile(generateAndSaveKeyTo)
if err != nil {
logger.Fatal("Failed to read private key: %v", err)
}
key, err = wgtypes.ParseKey(string(keyData))
if err != nil {
logger.Fatal("Failed to parse private key: %v", err)
key, err = wgtypes.GeneratePrivateKey()
if err != nil {
return nil, fmt.Errorf("failed to generate private key: %v", err)
}
// Load or generate private key
if generateAndSaveKeyTo != "" {
if _, err := os.Stat(generateAndSaveKeyTo); os.IsNotExist(err) {
keyData, err := os.ReadFile(generateAndSaveKeyTo)
if err != nil {
return nil, fmt.Errorf("failed to read private key: %v", err)
}
key, err = wgtypes.ParseKey(strings.TrimSpace(string(keyData)))
if err != nil {
return nil, fmt.Errorf("failed to parse private key: %v", err)
}
} else {
err = os.WriteFile(generateAndSaveKeyTo, []byte(key.String()), 0600)
if err != nil {
return nil, fmt.Errorf("failed to save private key: %v", err)
}
}
}
@@ -177,18 +183,26 @@ func NewWireGuardService(interfaceName string, mtu int, generateAndSaveKeyTo str
client: wsClient,
wgClient: wgClient,
key: key,
keyFilePath: generateAndSaveKeyTo,
newtId: newtId,
host: host,
lastReadings: make(map[string]PeerReading),
stopHolepunch: make(chan struct{}),
stopGetConfig: make(chan struct{}),
}
// Get the existing wireguard port (keep this part)
device, err := service.wgClient.Device(service.interfaceName)
if err == nil {
service.Port = uint16(device.ListenPort)
logger.Info("WireGuard interface %s already exists with port %d\n", service.interfaceName, service.Port)
if service.Port != 0 {
logger.Info("WireGuard interface %s already exists with port %d\n", service.interfaceName, service.Port)
} else {
service.Port, err = FindAvailableUDPPort(49152, 65535)
if err != nil {
fmt.Printf("Error finding available port: %v\n", err)
return nil, err
}
}
} else {
service.Port, err = FindAvailableUDPPort(49152, 65535)
if err != nil {
@@ -203,22 +217,13 @@ func NewWireGuardService(interfaceName string, mtu int, generateAndSaveKeyTo str
wsClient.RegisterHandler("newt/wg/peer/remove", service.handleRemovePeer)
wsClient.RegisterHandler("newt/wg/peer/update", service.handleUpdatePeer)
if err := service.sendUDPHolePunch(service.host + ":21820"); err != nil {
logger.Error("Failed to send UDP hole punch: %v", err)
}
// start the UDP holepunch
go service.keepSendingUDPHolePunch(service.host)
return service, nil
}
func (s *WireGuardService) Close(rm bool) {
select {
case <-s.stopGetConfig:
// Already closed, do nothing
default:
close(s.stopGetConfig)
if s.stopGetConfig != nil {
s.stopGetConfig()
s.stopGetConfig = nil
}
s.wgClient.Close()
@@ -229,14 +234,29 @@ func (s *WireGuardService) Close(rm bool) {
}
// Remove the private key file
if err := os.Remove(s.key.String()); err != nil {
logger.Error("Failed to remove private key file: %v", err)
}
// if s.keyFilePath != "" {
// if err := os.Remove(s.keyFilePath); err != nil {
// logger.Error("Failed to remove private key file: %v", err)
// }
// }
}
}
func (s *WireGuardService) SetServerPubKey(serverPubKey string) {
func (s *WireGuardService) StartHolepunch(serverPubKey string, endpoint string) {
// if the device is already created dont start a new holepunch
if s.interfaceCreated {
return
}
s.serverPubKey = serverPubKey
s.holePunchEndpoint = endpoint
logger.Debug("Starting UDP hole punch to %s", s.holePunchEndpoint)
s.stopHolepunch = make(chan struct{})
// start the UDP holepunch
go s.keepSendingUDPHolePunch(s.holePunchEndpoint)
}
func (s *WireGuardService) SetToken(token string) {
@@ -244,16 +264,12 @@ func (s *WireGuardService) SetToken(token string) {
}
func (s *WireGuardService) LoadRemoteConfig() error {
// Send the initial message
err := s.sendGetConfigMessage()
if err != nil {
logger.Error("Failed to send initial get-config message: %v", err)
return err
}
// Start goroutine to periodically send the message until config is received
go s.keepSendingGetConfig()
s.stopGetConfig = s.client.SendMessageInterval("newt/wg/get-config", map[string]interface{}{
"publicKey": s.key.PublicKey().String(),
"port": s.Port,
}, 2*time.Second)
logger.Info("Requesting WireGuard configuration from remote server")
go s.periodicBandwidthCheck()
return nil
@@ -262,7 +278,8 @@ func (s *WireGuardService) LoadRemoteConfig() error {
func (s *WireGuardService) handleConfig(msg websocket.WSMessage) {
var config WgConfig
logger.Info("Received message: %v", msg)
logger.Debug("Received message: %v", msg)
logger.Info("Received WireGuard clients configuration from remote server")
jsonData, err := json.Marshal(msg.Data)
if err != nil {
@@ -276,7 +293,10 @@ func (s *WireGuardService) handleConfig(msg websocket.WSMessage) {
}
s.config = config
close(s.stopGetConfig)
if s.stopGetConfig != nil {
s.stopGetConfig()
s.stopGetConfig = nil
}
// Ensure the WireGuard interface and peers are configured
if err := s.ensureWireguardInterface(config); err != nil {
@@ -298,6 +318,7 @@ func (s *WireGuardService) ensureWireguardInterface(wgconfig WgConfig) error {
if err != nil {
logger.Fatal("Failed to create WireGuard interface: %v", err)
}
s.interfaceCreated = true
logger.Info("Created WireGuard interface %s\n", s.interfaceName)
} else {
logger.Fatal("Error checking for WireGuard interface: %v", err)
@@ -315,9 +336,16 @@ func (s *WireGuardService) ensureWireguardInterface(wgconfig WgConfig) error {
s.Port = uint16(device.ListenPort)
logger.Info("WireGuard interface %s already exists with port %d\n", s.interfaceName, s.Port)
s.interfaceCreated = true
return nil
}
// stop the holepunch its a channel
if s.stopHolepunch != nil {
close(s.stopHolepunch)
s.stopHolepunch = nil
}
logger.Info("Assigning IP address %s to interface %s\n", wgconfig.IpAddress, s.interfaceName)
// Assign IP address to the interface
err = s.assignIPAddress(wgconfig.IpAddress)
@@ -328,7 +356,10 @@ func (s *WireGuardService) ensureWireguardInterface(wgconfig WgConfig) error {
// Check if the interface already exists
_, err = s.wgClient.Device(s.interfaceName)
if err != nil {
return fmt.Errorf("interface %s does not exist", s.interfaceName)
if errors.Is(err, os.ErrNotExist) {
return fmt.Errorf("interface %s does not exist", s.interfaceName)
}
return fmt.Errorf("failed to get device: %v", err)
}
// Parse the private key
@@ -447,7 +478,7 @@ func (s *WireGuardService) ensureWireguardPeers(peers []Peer) error {
}
func (s *WireGuardService) handleAddPeer(msg websocket.WSMessage) {
logger.Info("Received message: %v", msg.Data)
logger.Debug("Received message: %v", msg.Data)
var peer Peer
jsonData, err := json.Marshal(msg.Data)
@@ -520,7 +551,7 @@ func (s *WireGuardService) addPeer(peer Peer) error {
}
func (s *WireGuardService) handleRemovePeer(msg websocket.WSMessage) {
logger.Info("Received message: %v", msg.Data)
logger.Debug("Received message: %v", msg.Data)
// parse the publicKey from the message which is json { "publicKey": "asdfasdfl;akjsdf" }
type RemoveRequest struct {
PublicKey string `json:"publicKey"`
@@ -568,7 +599,7 @@ func (s *WireGuardService) removePeer(publicKey string) error {
}
func (s *WireGuardService) handleUpdatePeer(msg websocket.WSMessage) {
logger.Info("Received message: %v", msg.Data)
logger.Debug("Received message: %v", msg.Data)
// Define a struct to match the incoming message structure with optional fields
type UpdatePeerRequest struct {
PublicKey string `json:"publicKey"`
@@ -629,7 +660,7 @@ func (s *WireGuardService) handleUpdatePeer(msg websocket.WSMessage) {
}
// Only update AllowedIPs if provided in the request
if request.AllowedIPs != nil && len(request.AllowedIPs) > 0 {
if len(request.AllowedIPs) > 0 {
var allowedIPs []net.IPNet
for _, ipStr := range request.AllowedIPs {
_, ipNet, err := net.ParseCIDR(ipStr)
@@ -917,6 +948,11 @@ func (s *WireGuardService) encryptPayload(payload []byte) (interface{}, error) {
}
func (s *WireGuardService) keepSendingUDPHolePunch(host string) {
// send initial hole punch
if err := s.sendUDPHolePunch(host + ":21820"); err != nil {
logger.Error("Failed to send initial UDP hole punch: %v", err)
}
ticker := time.NewTicker(3 * time.Second)
defer ticker.Stop()
@@ -949,33 +985,3 @@ func (s *WireGuardService) removeInterface() error {
return nil
}
func (s *WireGuardService) sendGetConfigMessage() error {
err := s.client.SendMessage("newt/wg/get-config", map[string]interface{}{
"publicKey": fmt.Sprintf("%s", s.key.PublicKey().String()),
"port": s.Port,
})
if err != nil {
logger.Error("Failed to send get-config message: %v", err)
return err
}
logger.Info("Requesting WireGuard configuration from remote server")
return nil
}
func (s *WireGuardService) keepSendingGetConfig() {
ticker := time.NewTicker(3 * time.Second)
defer ticker.Stop()
for {
select {
case <-s.stopGetConfig:
logger.Info("Stopping get-config messages")
return
case <-ticker.C:
if err := s.sendGetConfigMessage(); err != nil {
logger.Error("Failed to send periodic get-config: %v", err)
}
}
}
}

1284
wgnetstack/wgnetstack.go Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -8,6 +8,8 @@ import (
"time"
"github.com/fosrl/newt/logger"
"golang.zx2c4.com/wireguard/tun/netstack"
"gvisor.dev/gvisor/pkg/tcpip/adapters/gonet"
)
const (
@@ -26,7 +28,9 @@ const (
// Server handles listening for connection check requests using UDP
type Server struct {
conn *net.UDPConn
conn net.Conn // Generic net.Conn interface (could be *net.UDPConn or *gonet.UDPConn)
udpConn *net.UDPConn // Regular UDP connection (when not using netstack)
netstackConn interface{} // Netstack UDP connection (when using netstack)
serverAddr string
serverPort uint16
shutdownCh chan struct{}
@@ -34,6 +38,8 @@ type Server struct {
runningLock sync.Mutex
newtID string
outputPrefix string
useNetstack bool
tnet interface{} // Will be *netstack.Net when using netstack
}
// NewServer creates a new connection test server using UDP
@@ -44,6 +50,21 @@ func NewServer(serverAddr string, serverPort uint16, newtID string) *Server {
shutdownCh: make(chan struct{}),
newtID: newtID,
outputPrefix: "[WGTester] ",
useNetstack: false,
tnet: nil,
}
}
// NewServerWithNetstack creates a new connection test server using WireGuard netstack
func NewServerWithNetstack(serverAddr string, serverPort uint16, newtID string, tnet *netstack.Net) *Server {
return &Server{
serverAddr: serverAddr,
serverPort: serverPort + 1, // use the next port for the server
shutdownCh: make(chan struct{}),
newtID: newtID,
outputPrefix: "[WGTester] ",
useNetstack: true,
tnet: tnet,
}
}
@@ -59,18 +80,30 @@ func (s *Server) Start() error {
//create the address to listen on
addr := net.JoinHostPort(s.serverAddr, fmt.Sprintf("%d", s.serverPort))
// Create UDP address to listen on
udpAddr, err := net.ResolveUDPAddr("udp", addr)
if err != nil {
return err
}
if s.useNetstack && s.tnet != nil {
// Use WireGuard netstack
tnet := s.tnet.(*netstack.Net)
udpAddr := &net.UDPAddr{Port: int(s.serverPort)}
netstackConn, err := tnet.ListenUDP(udpAddr)
if err != nil {
return err
}
s.netstackConn = netstackConn
s.conn = netstackConn
} else {
// Use regular UDP socket
udpAddr, err := net.ResolveUDPAddr("udp", addr)
if err != nil {
return err
}
// Create UDP connection
conn, err := net.ListenUDP("udp", udpAddr)
if err != nil {
return err
udpConn, err := net.ListenUDP("udp", udpAddr)
if err != nil {
return err
}
s.udpConn = udpConn
s.conn = udpConn
}
s.conn = conn
s.isRunning = true
go s.handleConnections()
@@ -96,6 +129,26 @@ func (s *Server) Stop() {
logger.Info(s.outputPrefix + "Server stopped")
}
// RestartWithNetstack stops the current server and restarts it with netstack
func (s *Server) RestartWithNetstack(tnet *netstack.Net) error {
s.Stop()
// Update configuration to use netstack
s.useNetstack = true
s.tnet = tnet
// Clear previous connections
s.conn = nil
s.udpConn = nil
s.netstackConn = nil
// Create new shutdown channel
s.shutdownCh = make(chan struct{})
// Restart the server
return s.Start()
}
// handleConnections processes incoming packets
func (s *Server) handleConnections() {
buffer := make([]byte, 2000) // Buffer large enough for any UDP packet
@@ -112,14 +165,30 @@ func (s *Server) handleConnections() {
continue
}
// Read from UDP connection
n, addr, err := s.conn.ReadFromUDP(buffer)
// Read from UDP connection - handle both regular UDP and netstack UDP
var n int
var addr net.Addr
if s.useNetstack {
// Use netstack UDP connection
netstackConn := s.netstackConn.(*gonet.UDPConn)
n, addr, err = netstackConn.ReadFrom(buffer)
} else {
// Use regular UDP connection
n, addr, err = s.udpConn.ReadFromUDP(buffer)
}
if err != nil {
if netErr, ok := err.(net.Error); ok && netErr.Timeout() {
// Just a timeout, keep going
continue
}
logger.Error(s.outputPrefix+"Error reading from UDP: %v", err)
// Check if we're shutting down and the connection was closed
select {
case <-s.shutdownCh:
return // Don't log error if we're shutting down
default:
logger.Error(s.outputPrefix+"Error reading from UDP: %v", err)
}
continue
}
@@ -152,8 +221,17 @@ func (s *Server) handleConnections() {
// Log response being sent for debugging
logger.Debug(s.outputPrefix+"Sending response to %s", addr.String())
// Send the response packet directly to the source address
_, err = s.conn.WriteToUDP(responsePacket, addr)
// Send the response packet - handle both regular UDP and netstack UDP
if s.useNetstack {
// Use netstack UDP connection
netstackConn := s.netstackConn.(*gonet.UDPConn)
_, err = netstackConn.WriteTo(responsePacket, addr)
} else {
// Use regular UDP connection
udpAddr := addr.(*net.UDPAddr)
_, err = s.udpConn.WriteToUDP(responsePacket, udpAddr)
}
if err != nil {
logger.Error(s.outputPrefix+"Error sending response: %v", err)
} else {