Files
komodo/bin/cli/src/command/database.rs
Maxwell Becker 2fa9d9ecce 1.19.0 (#722)
* start 1.18.5

* prevent empty additional permission check (ie for new resources)

* dev-2

* bump rust to 1.88

* tweaks

* repo based stack commit happens from core repo cache rather than on server to simplify

* clippy auto fix

* clippy lints periphery

* clippy fix komodo_client

* dev-3

* emphasize ferret version pinning

* bump svi with PR fix

* dev-4

* webhook disabled early return

* Fix missing alert types for whitelist

* add "ScheduleRun"

* fix status cache not cleaning on resource delete

* dev-5

* forgot to pipe through poll in previous refactor

* refetch given in ms

* fix configure build extra args

* reorder resource sync config

* Implement ability to run actions at startup (#664)

* Implement ability to run actions at startup

* run post-startup actions after server is listening

* startup use action query

* fmt

* Fix Google Login enabled message (#668)

- it was showing "Github Login" instead of "Google Login"

* Allow CIDR ranges in Allowed IPs (#666)

* Allow CIDR ranges in Allowed IPs

* Catch mixed IPv4/IPv6 mappings that are probably intended to match

* forgiving vec

* dev-6

* forgiving vec log. allowed ips docs

* server stats UI: move current disk breakdown above charts

* searchable container stats, toggle collaple container / disk sections

* Add Clear repo cache method

* fix execute usage docs

* Komodo managed env-file should take precedence in all cases (ie come last in env file list)

* tag include unused flag for future use

* combine users page search

* util backup / restore

* refactor backup/restore duplication

* cleanup restore

* core image include util binary

* dev-7

* back to LinesCodec

* dev-8

* clean up

* clean up logs

* rename to komodo-util

* dev-9

* enable_fance_toml

* dev-10 enable fancy toml

* add user agent to oidc requests (#701)

Co-authored-by: eleith <online-github@eleith.com>

* fmt

* use database library

* clippy lint

* consolidate and standardize cli

* dev-11

* dev-12 implement backup using cli

* dev-13 logs

* command variant fields need to be #[arg]

* tweak cli

* gen client

* fix terminal reconnect issue

* rename cli to `km`

* tweaks for the cli logs

* wait for enter on --yes empty println

* fix --yes

* dev-15

* bump deps

* update croner to latest, use static parser

* dev-16

* cli execute polls updates until complete before logging

* remove repo cache mount

* cli nice

* /backup -> /backups

* dev-17 config loading preserves CONFIG_PATHS precedence

* update dockerfile default docker cli config keywords

* dev-18

* support .kmignore

* add ignores log

* Implement automatic backup pruning, default 14 backups before prune

* db copy / restore uses idempotent upsert

* cli update variable - "km set var VAR value"

* improve cli initial logs

* time the executions

* implement update for most resources

* dev 20

* add update page

* dev 21 support cli update link

* dev-22 test the deploy

* dev-23 use indexmap

* install-cli.py

* Frontend mobile fixes (#714)

* Allow ResourcePageHeader items to wrap

* Allow CardHeader items to wrap

* Increase z-index of sticky TableHeader, fixes #690

* Remove fixed widths from ActionButton, let them flex more to fit more layouts

* Make Section scroll overflow

* Remove grid class from Tabs, seems to prevent them from overflowing at small sizes

* deploy 1.18.5-dev-24

* auto version increment and deploy

* cli: profiles support aliases and merge on top of Default (root) config

* fix page set titles

* rust 1.89 and improve config logs

* skip serializing for proper merge

* fix clippy lints re 1.89

* remove layouts overflow-x-scroll

* deploy 1.18.5-dev-25

* 1.89 docker images not ready yet

* km cfg -a (print all profiles)

* include commit variables

* skip serializing profiles when empty

* skip serialize default db / log configs

* km cfg --debug print mode

* correct defaults for CLI and only can pass restore folder from cli arg

* some more skip serialization

* db restore / copy index optional

* add runfile command aliases

* remove second schedule updating loop, can causes some schedules to be missed

* deploy 1.18.5-dev-26

* add log when target db indexing disabled

* cli: user password reset, update user super admin

* Add manual network interface configuration for multi-NIC Docker environments (#719)

* Add iproute2 to debian-debs

* feat: Add manual network interface configuration for multi-NIC support

Complete implementation of manual interface configuration:
- Add internet_interface config option
- Implement manual gateway routing
- Add NET_ADMIN capability requirement
- Clean up codebase changes

* fix: Update internet interface handling for multi-NIC support

* refactor: Enhance error messages and logging in networking module

* refactor: Simplify interface argument handling and improve logging in network configuration and cleanup

* refactor(network): simplify startup integration and improve error handling

- Move config access and error handling into network::configure_internet_gateway()
- Simplify startup.rs to single function call without parameters
- Remove redundant check_network_privileges() function
- Improve error handling by checking actual command output instead of pre-validation
- Better separation of concerns between startup and network modules

Addresses feedback from PR discussion:
https://github.com/moghtech/komodo/pull/719#discussion_r2261542921

* fix(config): update default internet interface setting
Addresses feedback from PR discussion:
https://github.com/moghtech/komodo/pull/719#discussion_r2261552279

* fix(config): remove custom default for internet interface in CoreConfig

* move mod.rs -> network.rs
Addresses feedback from PR discussion:
https://github.com/moghtech/komodo/pull/719#discussion_r2261558332

* add internet interface example

* docs(build-images): document multi-platform builds with Docker Buildx (#721)

* docs(build-images): add multi-platform buildx guide to builders.md

* docs(build-images): add multi-platform buildx guide and clarify platform selection in Komodo UI Extra Args field

* move to 1.19.0

* core support reading from multiple config files

* config support yaml

* deploy 1.19.0-dev-1

* deploy 1.19.0-dev-2

* add default komodo cli config

* better config merge with base

* no need to panic if empty config paths

* improve km --help

* prog on cli docs

* tweak cli docs

* tweak doc

* split the runfile commands

* update docsite deps

* km ps initial

* km ls

* list resource apis

* km con inspect

* deploy 1.19.0-dev-3

* fix: need serde default

* dev-4 fix container parsing issue

* tweak

* use include-based file finding for much faster discovery

* just move to standard config dir .config/komodo/komodo.cli.*

* update fe w/ new contianer info minimal serialization

* add links to table names

* deploy 1.19.0-dev-5

* links in tables

* backend for Action arguments

* deploy 1.19.0-dev-6

* deploy 1.19.0-dev-7

* deploy 1.19.0-dev-8

* no space at front of KeyValue default args

* webhook branch / body optional

* The incoming arguments

* deploy 1.19.0-dev-9

* con -> cn

* add config -> cf alias

* .kmignore

* .peripheryinclude

* outdated

* optional links, configurable table format

* table_format -> table_borders

* get types

* include docsite in yarn install

* update runnables command in docs

* tweak

* improve km ls only show important stuff

* Add BackupCoreDatabase

* deploy 1.19.0-dev-10

* backup command needs "--yes"

* deploy 1.19.0-dev-11

* update rustc 1.89.0

* cli tweak

* try chef

* Fix chef (after dependencies)

* try other compile command

* fix

* fix comment

* cleanup stats page

* ensure database backup procedure

* UI allow configure Backup Core Database in Procedures

* procedure description

* deploy 1.19.0-dev-12

* deploy 1.19.0-dev-13

* GlobalAutoUpdate

* deploy 1.19.0-dev-14

* default tags and global auto update procedure

* deploy 1.19.0-dev-15

* trim the default procedure descriptions

* deploy 1.19.0-dev-16

* in "system" theme, also poll for updates to the theme based on time.

* Add next run to Action / Procedure column

* km ls support filter by templates

* fix procedure toml serialization when params = {}

* deploy 1.19.0-dev-17

* KOMODO_INIT_ADMIN_USERNAME

* KOMODO_FIRST_SERVER_NAME

* add server.config.external_address for use with links

* deploy 1.19.0-dev-18

* improve auto prune

* fix system theme auto update

* deploy 1.19.0-dev-19

* rename auth/CreateLocalUser -> SignUpLocalUser. Add write/CreateLocalUser for in-ui initialization.

* deploy 1.19.0-dev-20

* UI can handle multiple active logins

* deploy 1.19.0-dev-21

* fix

* add logout function

* fix oauth redirect

* fix multi user exchange token function

* default external address

* just Add

* style account switcher

* backup and restore docs

* rework docsite file / sidebar structure, start auto update docs

* auto update docs

* tweak

* fix doc links

* only pull / update running stacks / deployments images

* deploy 1.19.0-dev-22

* deploy 1.19.0-dev-23

* fix #737

* community docs

* add BackupCoreDatabase link to docs

* update ferret v2 update guide using komodo-cli

* fix data table headers overlapping topbar

* don't alert when deploying

* CommitSync returns Update

* deploy 1.19.0-dev-24

* trim the decoded branch

* action uses file contents deserializer

* deploy 1.19.0-dev-25

* remove Toml from action args format

* clarify External Address purpose

* Fix podman compatibility in `get_container_stats` (#739)

* Add podman compability for querying stats

Podman and docker stats differ in results in significant ways but this filter change they will output the same stats

* syntax fix

* feat(dashboard): display CPU, memory, and disk usage on server cards (#729)

* feat: mini-stats-card: Expose Server CPU , Memory, Disk Usage to Dashboard View

* comment: resolved

* Feat: fix overflow card , DRY stats-mini, add unreachable mini stats

* lint: fix

* deploy 1.19.0-dev-26

* 1.19.0

* linux, macos container install

* cli main config

---------

Co-authored-by: Brian Bradley <brian.bradley.p@gmail.com>
Co-authored-by: Daniel <daniel.barabasa@gmail.com>
Co-authored-by: eleith <eleith@users.noreply.github.com>
Co-authored-by: eleith <online-github@eleith.com>
Co-authored-by: Sam Edwards <sam@samedwards.ca>
Co-authored-by: Marcel Pfennig <82059270+MP-Tool@users.noreply.github.com>
Co-authored-by: itsmesid <693151+arevindh@users.noreply.github.com>
Co-authored-by: mbecker20 <max@mogh.tech>
Co-authored-by: Rhyn <Rhyn@users.noreply.github.com>
Co-authored-by: Anh Nguyen <tuananh131001@gmail.com>
2025-08-17 17:25:45 -07:00

321 lines
7.7 KiB
Rust

use std::path::Path;
use anyhow::Context;
use colored::Colorize;
use komodo_client::entities::{
config::cli::args::database::DatabaseCommand, optional_string,
};
use crate::{command::sanitize_uri, config::cli_config};
pub async fn handle(command: &DatabaseCommand) -> anyhow::Result<()> {
match command {
DatabaseCommand::Backup { yes, .. } => backup(*yes).await,
DatabaseCommand::Restore {
restore_folder,
index,
yes,
..
} => restore(restore_folder.as_deref(), *index, *yes).await,
DatabaseCommand::Prune { yes, .. } => prune(*yes).await,
DatabaseCommand::Copy { yes, index, .. } => {
copy(*index, *yes).await
}
}
}
async fn backup(yes: bool) -> anyhow::Result<()> {
let config = cli_config();
println!(
"\n🦎 {} Database {} Utility 🦎",
"Komodo".bold(),
"Backup".green().bold()
);
println!(
"\n{}\n",
" - Backup all database contents to gzip compressed files."
.dimmed()
);
if let Some(uri) = optional_string(&config.database.uri) {
println!("{}: {}", " - Source URI".dimmed(), sanitize_uri(&uri));
}
if let Some(address) = optional_string(&config.database.address) {
println!("{}: {address}", " - Source Address".dimmed());
}
if let Some(username) = optional_string(&config.database.username) {
println!("{}: {username}", " - Source Username".dimmed());
}
println!(
"{}: {}\n",
" - Source Db Name".dimmed(),
config.database.db_name,
);
println!(
"{}: {:?}",
" - Backups Folder".dimmed(),
config.backups_folder
);
if config.max_backups == 0 {
println!(
"{}{}",
" - Backup pruning".dimmed(),
"disabled".red().dimmed()
);
} else {
println!("{}: {}", " - Max Backups".dimmed(), config.max_backups);
}
crate::command::wait_for_enter("start backup", yes)?;
let db = database::init(&config.database).await?;
database::utils::backup(&db, &config.backups_folder).await?;
// Early return if backup pruning disabled
if config.max_backups == 0 {
return Ok(());
}
// Know that new backup was taken successfully at this point,
// safe to prune old backup folders
prune_inner().await
}
async fn restore(
restore_folder: Option<&Path>,
index: bool,
yes: bool,
) -> anyhow::Result<()> {
let config = cli_config();
println!(
"\n🦎 {} Database {} Utility 🦎",
"Komodo".bold(),
"Restore".purple().bold()
);
println!(
"\n{}\n",
" - Restores database contents from gzip compressed files."
.dimmed()
);
if let Some(uri) = optional_string(&config.database_target.uri) {
println!("{}: {}", " - Target URI".dimmed(), sanitize_uri(&uri));
}
if let Some(address) =
optional_string(&config.database_target.address)
{
println!("{}: {address}", " - Target Address".dimmed());
}
if let Some(username) =
optional_string(&config.database_target.username)
{
println!("{}: {username}", " - Target Username".dimmed());
}
println!(
"{}: {}",
" - Target Db Name".dimmed(),
config.database_target.db_name,
);
if !index {
println!(
"{}: {}",
" - Target Db Indexing".dimmed(),
"DISABLED".red(),
);
}
println!(
"\n{}: {:?}",
" - Backups Folder".dimmed(),
config.backups_folder
);
if let Some(restore_folder) = restore_folder {
println!("{}: {restore_folder:?}", " - Restore Folder".dimmed());
}
crate::command::wait_for_enter("start restore", yes)?;
let db = if index {
database::Client::new(&config.database_target).await?.db
} else {
database::init(&config.database_target).await?
};
database::utils::restore(
&db,
&config.backups_folder,
restore_folder,
)
.await
}
async fn prune(yes: bool) -> anyhow::Result<()> {
let config = cli_config();
println!(
"\n🦎 {} Database {} Utility 🦎",
"Komodo".bold(),
"Backup Prune".cyan().bold()
);
println!(
"\n{}\n",
" - Prunes database backup folders when greater than the configured amount."
.dimmed()
);
println!(
"{}: {:?}",
" - Backups Folder".dimmed(),
config.backups_folder
);
if config.max_backups == 0 {
println!(
"{}{}",
" - Backup pruning".dimmed(),
"disabled".red().dimmed()
);
} else {
println!("{}: {}", " - Max Backups".dimmed(), config.max_backups);
}
// Early return if backup pruning disabled
if config.max_backups == 0 {
info!(
"Backup pruning is disabled, enabled using 'max_backups' (KOMODO_CLI_MAX_BACKUPS)"
);
return Ok(());
}
crate::command::wait_for_enter("start backup prune", yes)?;
prune_inner().await
}
async fn prune_inner() -> anyhow::Result<()> {
let config = cli_config();
let mut backups_dir =
match tokio::fs::read_dir(&config.backups_folder)
.await
.context("Failed to read backups folder for prune")
{
Ok(backups_dir) => backups_dir,
Err(e) => {
warn!("{e:#}");
return Ok(());
}
};
let mut backup_folders = Vec::new();
loop {
match backups_dir.next_entry().await {
Ok(Some(entry)) => {
let Ok(metadata) = entry.metadata().await else {
continue;
};
if metadata.is_dir() {
backup_folders.push(entry.path());
}
}
Ok(None) => break,
Err(_) => {
continue;
}
}
}
// Ordered from oldest -> newest
backup_folders.sort();
let max_backups = config.max_backups as usize;
let backup_folders_len = backup_folders.len();
// Early return if under the backup count threshold
if backup_folders_len <= max_backups {
info!("No backups to prune");
return Ok(());
}
let to_delete =
&backup_folders[..(backup_folders_len - max_backups)];
info!("Pruning old backups: {to_delete:?}");
for path in to_delete {
if let Err(e) =
tokio::fs::remove_dir_all(path).await.with_context(|| {
format!("Failed to delete backup folder at {path:?}")
})
{
warn!("{e:#}");
}
}
Ok(())
}
async fn copy(index: bool, yes: bool) -> anyhow::Result<()> {
let config = cli_config();
println!(
"\n🦎 {} Database {} Utility 🦎",
"Komodo".bold(),
"Copy".blue().bold()
);
println!(
"\n{}\n",
" - Copies database contents to another database.".dimmed()
);
if let Some(uri) = optional_string(&config.database.uri) {
println!("{}: {}", " - Source URI".dimmed(), sanitize_uri(&uri));
}
if let Some(address) = optional_string(&config.database.address) {
println!("{}: {address}", " - Source Address".dimmed());
}
if let Some(username) = optional_string(&config.database.username) {
println!("{}: {username}", " - Source Username".dimmed());
}
println!(
"{}: {}\n",
" - Source Db Name".dimmed(),
config.database.db_name,
);
if let Some(uri) = optional_string(&config.database_target.uri) {
println!("{}: {}", " - Target URI".dimmed(), sanitize_uri(&uri));
}
if let Some(address) =
optional_string(&config.database_target.address)
{
println!("{}: {address}", " - Target Address".dimmed());
}
if let Some(username) =
optional_string(&config.database_target.username)
{
println!("{}: {username}", " - Target Username".dimmed());
}
println!(
"{}: {}",
" - Target Db Name".dimmed(),
config.database_target.db_name,
);
if !index {
println!(
"{}: {}",
" - Target Db Indexing".dimmed(),
"DISABLED".red(),
);
}
crate::command::wait_for_enter("start copy", yes)?;
let source_db = database::init(&config.database).await?;
let target_db = if index {
database::Client::new(&config.database_target).await?.db
} else {
database::init(&config.database_target).await?
};
database::utils::copy(&source_db, &target_db).await
}