[PR #889] 2.0.0 #808

New Issue

GiteaMirror · 2025-10-31T15:22:24-05:00

GiteaMirror commented

2025-10-31 15:22:24 -05:00

📋 Pull Request Information

Original PR: https://github.com/moghtech/komodo/pull/889
Author: @mbecker20
Created: 9/28/2025
Status: 🔄 Open

Base: main ← Head: 2.0.0

📝 Commits (10+)

37ac0dc update deploy
7422c07 deploy 2.0.0-dev-19
e2680d0 fix deploy
4feecb4 write key pem files by default when not otherwise provided.
f228cd3 deploy 2.0.0-dev-20
c9c4ac4 fix clippy
7fb902b deploy 2.0.0-dev-21
21ea469 add login message 2 sec timeout
859bfe6 Improve Core side connection handling and fix Periphery -> Core error report
9eb32f9 store attempted public keys

📊 Changes

408 files changed (+26170 additions, -14284 deletions)

View changed files

📝 .vscode/resolver.code-snippets (+5 -5)
📝 Cargo.lock (+873 -811)
📝 Cargo.toml (+44 -34)
➕ action/build.ts (+2 -0)
➕ action/deno.json (+5 -0)
➕ action/deploy-fe.ts (+4 -0)
➕ action/deploy.ts (+2 -0)
➕ action/run.ts (+52 -0)
📝 bin/binaries.Dockerfile (+3 -3)
📝 bin/chef.binaries.Dockerfile (+3 -3)
📝 bin/cli/Cargo.toml (+3 -0)
📝 bin/cli/aio.Dockerfile (+3 -3)
📝 bin/cli/multi-arch.Dockerfile (+2 -2)
📝 bin/cli/single-arch.Dockerfile (+2 -2)
📝 bin/cli/src/command/container.rs (+4 -2)
📝 bin/cli/src/command/database.rs (+46 -0)
📝 bin/cli/src/command/execute.rs (+14 -0)
📝 bin/cli/src/command/list.rs (+52 -6)
📝 bin/cli/src/command/mod.rs (+1 -0)
➕ bin/cli/src/command/terminal.rs (+334 -0)

...and 80 more files

📄 Description

🚨 Both Core and Periphery must be upgraded to v2 for the connection to work.
🚨 Action users: komodo.execute_terminal / komodo.execute_container_exec changes -- see Terminals section below.

Changelog

I'm very excited to bring you Komodo v2. The major improvements are:

Enable "reversing" the Periphery connection.
Easy upgrade to public key authentication.
Implement Server onboarding flow.
Improve the Terminal feature set.

Periphery → Core Connection 🦎

One of the major limitations in Komodo v1 was that Core always had to establish the connection to Periphery. In many setups, this was not the optimal connection strategy, and would often require the usage of a VPN to establish the connection securely.

Now with v2, Periphery can be configured to establish an outbound connection to Core, and communication with Core will be multiplexed over this connection. #281

Existing Core → Periphery connections also continue working, and no configuration changes are required to update from Komodo v1.

Ideally Komodo Core should be served over HTTPS with certificates recognized by Periphery, however it also works for Periphery to connect to Core over ws://..., as well as insecure TLS (untrusted certificates) if you set PERIPHERY_CORE_TLS_INSECURE_SKIP_VERIFY=true.

It is usually both easier and more secure to use this Periphery → Core connection method, and there isn't much other difference or downside. The default compose configuration has changed to this method, and I recommend most users migrate their setups when convenient.

periphery.config.toml

## The address of Komodo Core. Must be reachable from this host.
## If provided, Periphery will operate in outbound mode.
## Use `ws://12.34.56.78:9120` if you don't have TLS enabled Core.
## Env: PERIPHERY_CORE_ADDRESS
## Default: None
core_address = "example.komodo.com"

## The Server this Periphery agent should connect as.
## If matching Server doesn't exist yet, see "Server Onboarding".
## Using `$(hostname)` will use the output of shell `hostname` command.
## Env: PERIPHERY_CONNECT_AS
## Default: None
connect_as = "$(hostname)"

If using a reverse proxy with Komodo Core, you can limit the IPs which can connect to the Periphery endpoint. For example with Caddy:

(reject-ips) {
	@externalIp not remote_ip 192.168.0.0/16 12.34.56.78/32
	respond @externalIp 403
}

komodo.example.com {
	handle /ws/periphery {
		import reject-ips
		reverse_proxy komodo-core:9120
	}
	handle {
		reverse_proxy komodo-core:9120
	}
}

Private / Public Key Authentication 🦎

The other limitation was the usage of a passkey for Core to authenticate to Periphery. This was not a secure usage pattern for 2 reasons:

The passkey had to be sent over the network rather than using cryptographic methods.
A single compromised Server would expose the passkey, which could be used to compromise other Servers.

While this passkey pattern remains supported for backward compatibility (only Core -> Periphery direction), users looking to move to Periphery -> Core (or increase Core -> Periphery security) can now replace their passkeys with paired private / public keys.

If you point to key files that don't exist yet, they will be generated. While the keys are openssl compatible, there is no need to manually generate keys.

core.config.toml

## Private key to use with Noise handshake to authenticate with Periphery agents.
## To load from file, use `file:/config/keys/core.key`, and mount `- keys:/config/keys`.
## If it doesn't exist yet, Core will generate a new key pair and store it for persistence.
## Env: KOMODO_PRIVATE_KEY or KOMODO_PRIVATE_KEY_FILE
private_key = "file:/config/keys/core.key"

periphery.config.toml

## Private key to use with Noise handshake to authenticate with Periphery agents.
## To load from file, use `file:/path/to/periphery.key`.
## If it doesn't exist yet, Periphery will generate a new key pair and store it for persistence.
private_key = "file:/config/keys/periphery.key"

## Optional. Accepted Core public key to allow connection.
## To load from file, use `file:/path/to/core.pub`
## Env: PERIPHERY_CORE_PUBLIC_KEY
core_public_key = "file:/config/keys/core.pub"

The authentication is accomplished using a Noise XX handshake via the snow library.

Automatic Key Rotation 🦎

Storing the private keys in dedicated files allows for easy automatic rotation of key pairs. To rotate a Server's key, Core sends the RotatePrivateKey request to Periphery, which then generates a new key pair, and returns the public key back to Core to update the allowed public key in the Database. The new private key stays local to each Periphery agent and is never exposed to the network. Note that it only works for Servers that are currently connected, if a Server cannot be reached, the rotation will be skipped for that Server.

Similar to the "Backup Core Database" and "Global Auto Update" default system Procedures, one for key rotation will now be generated during first database initialization. Here is the TOML:

[[procedure]]
name = "Rotate Server Keys"
description = "Rotates all currently connected Server keys."
tags = ["system"]
config.schedule = "Every day at 06:00"

[[procedure.config.stage]]
name = "Stage 1"
enabled = true
executions = [
  { execution.type = "RotateAllServerKeys", execution.params = {}, enabled = true }
]

Server Onboarding 🦎

You can now make Onboarding Keys in Settings / Onboarding page. Giving one of these to Periphery allows itself to "onboard" as a new Server in Komodo. These keys are only used for this purpose, and cannot be used as the Periphery private key itself. They aren't needed if the Server already exists.

If you would like, you can make multiple onboarding keys, and depending on the key used to onboard, the created Server will inherit the configured Tags, as well as the configuration from the optional template Server.

periphery.config.toml

## Make Onboarding Keys in Server settings.
## Not needed if Server already exists.
## Env: PERIPHERY_ONBOARDING_KEY
onboarding_key = "MC4CAQAwBQYDK2VuBCIEIH1+pLGMYkT36gNj5fRWrgRenBjaTfuUVoScW1a9XG8y"

Terminals

In v1, the Terminal feature set grew slowly over the releases, and the implementations needed refinement. For example, Servers could create multiple Terminal tabs, but the Container was limited to a single Terminal. Additionally, access to Terminals was hidden behind multiple clicks -- first navigate to Server / Container, then Terminal tab, etc.

The issue stemmed from the implementation placing "Container" type Terminals as a subset of "Server" type Terminals. This is now refactored to "flatten" the implementation -- There is now a single "Terminal" type, with possible sub-types being "Server" or "Container". In other words, the different Terminal types move from a "Parent - Child" relationship to a "Sibling" relationship, making the implementations between Server and Container sub-types more straightforward and consistent.

The first thing to notice is the new dedicated "Terminals" page. This provides a birds-eye view of active Terminal sessions for easier management. This page includes all the expected "Create", "Connect", "Delete" functionality, enabling usage of this feature entirely from this page.

Container type terminals (so also Stack service / Deployment) now support multiple active Terminals. The UI is now identical to the Server terminals for maximum flexibility.
docker attach ... is now supported.
The Komodo CLI now supports direct ssh analog -- km ssh <SERVER>.

🚨Breaking Action change

Instead of komodo.execute_terminal and komodo.execute_container_exec, use the updated methods

await komodo.execute_server_terminal(
  {
    server: "my-server",
    terminal: "custom-name",
    command: "ls -l",
    // New. Provide session init args.
    init: {
       command: "bash",
       recreate: Types.TerminalRecreateMode.Always
    }
  },
  // Callbacks unchanged
  {
    onLine: console.log
  }
);
await komodo.execute_container_terminal(
  {
    server: "my-server",
    container: "my-container",
    terminal: "custom-name",
    command: "ls -l",
    init: {
       command: "sh",
       recreate: Types.TerminalRecreateMode.Always
    }
  },
  // Callbacks unchanged
  {
    onLine: console.log
  }
);
// Also can use komodo.execute_stack_service_terminal and komodo.execute_deployment_terminal

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/moghtech/komodo/pull/889 **Author:** [@mbecker20](https://github.com/mbecker20) **Created:** 9/28/2025 **Status:** 🔄 Open **Base:** `main` ← **Head:** `2.0.0` --- ### 📝 Commits (10+) - [`37ac0dc`](https://github.com/moghtech/komodo/commit/37ac0dc7e372702d047e62cb6e4517bc8e5a76b2) update deploy - [`7422c07`](https://github.com/moghtech/komodo/commit/7422c0730de9089953bb3fc52df1b86725818baa) deploy 2.0.0-dev-19 - [`e2680d0`](https://github.com/moghtech/komodo/commit/e2680d0942155e22501e27fd7e8eb6e59c69ba6b) fix deploy - [`4feecb4`](https://github.com/moghtech/komodo/commit/4feecb4b97b07c54c2df32be4f55109d71bc487b) write key pem files by default when not otherwise provided. - [`f228cd3`](https://github.com/moghtech/komodo/commit/f228cd31f370506e396e5ab026e1d1401f67d5ce) deploy 2.0.0-dev-20 - [`c9c4ac4`](https://github.com/moghtech/komodo/commit/c9c4ac47eec3ee15f457fed3b8a79f81adb09f3b) fix clippy - [`7fb902b`](https://github.com/moghtech/komodo/commit/7fb902b89278a8c96630a1d71f2205ef1d1bfc81) deploy 2.0.0-dev-21 - [`21ea469`](https://github.com/moghtech/komodo/commit/21ea469cd44b354abf4bf7e7edb6e6806fc7b3c5) add login message 2 sec timeout - [`859bfe6`](https://github.com/moghtech/komodo/commit/859bfe67ef64b4368d4726ac3d8ff5b9bd5451ef) Improve Core side connection handling and fix Periphery -> Core error report - [`9eb32f9`](https://github.com/moghtech/komodo/commit/9eb32f9ff5d9cb69413090a66e1277ff38225c02) store attempted public keys ### 📊 Changes **408 files changed** (+26170 additions, -14284 deletions) <details> <summary>View changed files</summary> 📝 `.vscode/resolver.code-snippets` (+5 -5) 📝 `Cargo.lock` (+873 -811) 📝 `Cargo.toml` (+44 -34) ➕ `action/build.ts` (+2 -0) ➕ `action/deno.json` (+5 -0) ➕ `action/deploy-fe.ts` (+4 -0) ➕ `action/deploy.ts` (+2 -0) ➕ `action/run.ts` (+52 -0) 📝 `bin/binaries.Dockerfile` (+3 -3) 📝 `bin/chef.binaries.Dockerfile` (+3 -3) 📝 `bin/cli/Cargo.toml` (+3 -0) 📝 `bin/cli/aio.Dockerfile` (+3 -3) 📝 `bin/cli/multi-arch.Dockerfile` (+2 -2) 📝 `bin/cli/single-arch.Dockerfile` (+2 -2) 📝 `bin/cli/src/command/container.rs` (+4 -2) 📝 `bin/cli/src/command/database.rs` (+46 -0) 📝 `bin/cli/src/command/execute.rs` (+14 -0) 📝 `bin/cli/src/command/list.rs` (+52 -6) 📝 `bin/cli/src/command/mod.rs` (+1 -0) ➕ `bin/cli/src/command/terminal.rs` (+334 -0) _...and 80 more files_ </details> ### 📄 Description - 🚨 Both **Core** and **Periphery** must be upgraded to v2 for the connection to work. - 🚨 Action users: **`komodo.execute_terminal` / `komodo.execute_container_exec`** changes -- see **Terminals** section below. # Changelog I'm very excited to bring you Komodo v2. The major improvements are: - Enable "reversing" the Periphery connection. - Easy upgrade to public key authentication. - Implement Server onboarding flow. - Improve the Terminal feature set. ### **Periphery → Core Connection** 🦎 One of the major limitations in Komodo v1 was that Core always had to establish the connection to Periphery. In many setups, this was not the optimal connection strategy, and would often require the usage of a VPN to establish the connection securely. Now with v2, **Periphery can be configured to establish an outbound connection to Core**, and communication with Core will be multiplexed over this connection. #281 **Existing Core → Periphery connections also continue working, and no configuration changes are required to update from Komodo v1.** Ideally Komodo Core should be served over HTTPS with certificates recognized by Periphery, however it also works for Periphery to connect to Core over `ws://...`, as well as insecure TLS (untrusted certificates) if you set `PERIPHERY_CORE_TLS_INSECURE_SKIP_VERIFY=true`. **It is usually both easier and more secure to use this Periphery → Core connection method**, and there isn't much other difference or downside. The default compose configuration has changed to this method, and I recommend most users migrate their setups when convenient. #### periphery.config.toml ```toml ## The address of Komodo Core. Must be reachable from this host. ## If provided, Periphery will operate in outbound mode. ## Use `ws://12.34.56.78:9120` if you don't have TLS enabled Core. ## Env: PERIPHERY_CORE_ADDRESS ## Default: None core_address = "example.komodo.com" ## The Server this Periphery agent should connect as. ## If matching Server doesn't exist yet, see "Server Onboarding". ## Using `$(hostname)` will use the output of shell `hostname` command. ## Env: PERIPHERY_CONNECT_AS ## Default: None connect_as = "$(hostname)" ``` If using a reverse proxy with Komodo Core, you can limit the IPs which can connect to the Periphery endpoint. For example with Caddy: ```caddyfile (reject-ips) { @externalIp not remote_ip 192.168.0.0/16 12.34.56.78/32 respond @externalIp 403 } komodo.example.com { handle /ws/periphery { import reject-ips reverse_proxy komodo-core:9120 } handle { reverse_proxy komodo-core:9120 } } ``` ### **Private / Public Key Authentication** 🦎 The other limitation was the usage of a `passkey` for Core to authenticate to Periphery. This was not a secure usage pattern for 2 reasons: 1. The passkey had to be sent over the network rather than using cryptographic methods. 2. A single compromised Server would expose the passkey, which could be used to compromise other Servers. While this `passkey` pattern remains supported for backward compatibility (only Core -> Periphery direction), users looking to move to Periphery -> Core (or increase Core -> Periphery security) can now replace their passkeys with paired private / public keys. If you point to key files that don't exist yet, they will be generated. While the keys are `openssl` compatible, there is no need to manually generate keys. #### core.config.toml ```toml ## Private key to use with Noise handshake to authenticate with Periphery agents. ## To load from file, use `file:/config/keys/core.key`, and mount `- keys:/config/keys`. ## If it doesn't exist yet, Core will generate a new key pair and store it for persistence. ## Env: KOMODO_PRIVATE_KEY or KOMODO_PRIVATE_KEY_FILE private_key = "file:/config/keys/core.key" ``` #### periphery.config.toml ```toml ## Private key to use with Noise handshake to authenticate with Periphery agents. ## To load from file, use `file:/path/to/periphery.key`. ## If it doesn't exist yet, Periphery will generate a new key pair and store it for persistence. private_key = "file:/config/keys/periphery.key" ## Optional. Accepted Core public key to allow connection. ## To load from file, use `file:/path/to/core.pub` ## Env: PERIPHERY_CORE_PUBLIC_KEY core_public_key = "file:/config/keys/core.pub" ``` The authentication is accomplished using a [Noise XX handshake](https://noiseprotocol.org/noise.html#interactive-handshake-patterns-fundamental) via the [snow](https://github.com/mcginty/snow) library. ### **Automatic Key Rotation** 🦎 Storing the private keys in dedicated files allows for easy automatic rotation of key pairs. To rotate a Server's key, Core sends the `RotatePrivateKey` request to Periphery, which then generates a new key pair, and returns the public key back to Core to update the allowed public key in the Database. The new private key stays local to each Periphery agent and is never exposed to the network. Note that it only works for Servers that are *currently* connected, if a Server cannot be reached, the rotation will be skipped for that Server. Similar to the "Backup Core Database" and "Global Auto Update" default system Procedures, one for key rotation will now be generated during first database initialization. Here is the TOML: ```toml [[procedure]] name = "Rotate Server Keys" description = "Rotates all currently connected Server keys." tags = ["system"] config.schedule = "Every day at 06:00" [[procedure.config.stage]] name = "Stage 1" enabled = true executions = [ { execution.type = "RotateAllServerKeys", execution.params = {}, enabled = true } ] ``` ### **Server Onboarding** 🦎 You can now make Onboarding Keys in Settings / Onboarding page. Giving one of these to Periphery allows itself to "onboard" as a new Server in Komodo. These keys are *only* used for this purpose, and cannot be used as the Periphery private key itself. They aren't needed if the Server already exists. If you would like, you can make multiple onboarding keys, and depending on the key used to onboard, the created Server will inherit the configured Tags, as well as the configuration from the optional template Server. #### periphery.config.toml ```toml ## Make Onboarding Keys in Server settings. ## Not needed if Server already exists. ## Env: PERIPHERY_ONBOARDING_KEY onboarding_key = "MC4CAQAwBQYDK2VuBCIEIH1+pLGMYkT36gNj5fRWrgRenBjaTfuUVoScW1a9XG8y" ``` <img width="1463" height="367" alt="Screenshot 2025-10-05 at 3 01 34 PM" src="https://github.com/user-attachments/assets/d675d012-46ca-453f-a91c-7ae652d83654" /> ### **Terminals** In v1, the Terminal feature set grew slowly over the releases, and the implementations needed refinement. For example, Servers could create multiple Terminal tabs, but the Container was limited to a single Terminal. Additionally, access to Terminals was hidden behind multiple clicks -- first navigate to Server / Container, then Terminal tab, etc. The issue stemmed from the implementation placing "Container" type Terminals as a _subset_ of "Server" type Terminals. This is now refactored to "flatten" the implementation -- There is now a single "Terminal" type, with possible sub-types being "Server" or "Container". In other words, the different Terminal types move from a "Parent - Child" relationship to a "Sibling" relationship, making the implementations between Server and Container sub-types more straightforward and consistent. The first thing to notice is the new dedicated "Terminals" page. This provides a birds-eye view of active Terminal sessions for easier management. This page includes all the expected "Create", "Connect", "Delete" functionality, enabling usage of this feature entirely from this page. <img width="1689" height="567" alt="Screenshot 2025-10-28 at 12 50 25 PM" src="https://github.com/user-attachments/assets/d7fdfcba-8bc7-4e54-88b9-0bd1f503d559" /> - Container type terminals (so also Stack service / Deployment) now support multiple active Terminals. The UI is now identical to the Server terminals for maximum flexibility. - `docker attach ...` is now supported. - The Komodo CLI now supports direct `ssh` analog -- `km ssh <SERVER>`. <img width="424" height="298" alt="Screenshot 2025-10-28 at 12 53 56 PM" src="https://github.com/user-attachments/assets/e6a6ba1b-b723-45ee-835b-dcc94a795519" /> ### 🚨Breaking Action change Instead of `komodo.execute_terminal` and `komodo.execute_container_exec`, use the updated methods ```ts await komodo.execute_server_terminal( { server: "my-server", terminal: "custom-name", command: "ls -l", // New. Provide session init args. init: { command: "bash", recreate: Types.TerminalRecreateMode.Always } }, // Callbacks unchanged { onLine: console.log } ); await komodo.execute_container_terminal( { server: "my-server", container: "my-container", terminal: "custom-name", command: "ls -l", init: { command: "sh", recreate: Types.TerminalRecreateMode.Always } }, // Callbacks unchanged { onLine: console.log } ); // Also can use komodo.execute_stack_service_terminal and komodo.execute_deployment_terminal ``` --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

GiteaMirror added the

pull-request

label 2025-10-31 15:22:24 -05:00

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/komodo#808