forked from github-starred/komodo
* start 1.18.5 * prevent empty additional permission check (ie for new resources) * dev-2 * bump rust to 1.88 * tweaks * repo based stack commit happens from core repo cache rather than on server to simplify * clippy auto fix * clippy lints periphery * clippy fix komodo_client * dev-3 * emphasize ferret version pinning * bump svi with PR fix * dev-4 * webhook disabled early return * Fix missing alert types for whitelist * add "ScheduleRun" * fix status cache not cleaning on resource delete * dev-5 * forgot to pipe through poll in previous refactor * refetch given in ms * fix configure build extra args * reorder resource sync config * Implement ability to run actions at startup (#664) * Implement ability to run actions at startup * run post-startup actions after server is listening * startup use action query * fmt * Fix Google Login enabled message (#668) - it was showing "Github Login" instead of "Google Login" * Allow CIDR ranges in Allowed IPs (#666) * Allow CIDR ranges in Allowed IPs * Catch mixed IPv4/IPv6 mappings that are probably intended to match * forgiving vec * dev-6 * forgiving vec log. allowed ips docs * server stats UI: move current disk breakdown above charts * searchable container stats, toggle collaple container / disk sections * Add Clear repo cache method * fix execute usage docs * Komodo managed env-file should take precedence in all cases (ie come last in env file list) * tag include unused flag for future use * combine users page search * util backup / restore * refactor backup/restore duplication * cleanup restore * core image include util binary * dev-7 * back to LinesCodec * dev-8 * clean up * clean up logs * rename to komodo-util * dev-9 * enable_fance_toml * dev-10 enable fancy toml * add user agent to oidc requests (#701) Co-authored-by: eleith <online-github@eleith.com> * fmt * use database library * clippy lint * consolidate and standardize cli * dev-11 * dev-12 implement backup using cli * dev-13 logs * command variant fields need to be #[arg] * tweak cli * gen client * fix terminal reconnect issue * rename cli to `km` * tweaks for the cli logs * wait for enter on --yes empty println * fix --yes * dev-15 * bump deps * update croner to latest, use static parser * dev-16 * cli execute polls updates until complete before logging * remove repo cache mount * cli nice * /backup -> /backups * dev-17 config loading preserves CONFIG_PATHS precedence * update dockerfile default docker cli config keywords * dev-18 * support .kmignore * add ignores log * Implement automatic backup pruning, default 14 backups before prune * db copy / restore uses idempotent upsert * cli update variable - "km set var VAR value" * improve cli initial logs * time the executions * implement update for most resources * dev 20 * add update page * dev 21 support cli update link * dev-22 test the deploy * dev-23 use indexmap * install-cli.py * Frontend mobile fixes (#714) * Allow ResourcePageHeader items to wrap * Allow CardHeader items to wrap * Increase z-index of sticky TableHeader, fixes #690 * Remove fixed widths from ActionButton, let them flex more to fit more layouts * Make Section scroll overflow * Remove grid class from Tabs, seems to prevent them from overflowing at small sizes * deploy 1.18.5-dev-24 * auto version increment and deploy * cli: profiles support aliases and merge on top of Default (root) config * fix page set titles * rust 1.89 and improve config logs * skip serializing for proper merge * fix clippy lints re 1.89 * remove layouts overflow-x-scroll * deploy 1.18.5-dev-25 * 1.89 docker images not ready yet * km cfg -a (print all profiles) * include commit variables * skip serializing profiles when empty * skip serialize default db / log configs * km cfg --debug print mode * correct defaults for CLI and only can pass restore folder from cli arg * some more skip serialization * db restore / copy index optional * add runfile command aliases * remove second schedule updating loop, can causes some schedules to be missed * deploy 1.18.5-dev-26 * add log when target db indexing disabled * cli: user password reset, update user super admin * Add manual network interface configuration for multi-NIC Docker environments (#719) * Add iproute2 to debian-debs * feat: Add manual network interface configuration for multi-NIC support Complete implementation of manual interface configuration: - Add internet_interface config option - Implement manual gateway routing - Add NET_ADMIN capability requirement - Clean up codebase changes * fix: Update internet interface handling for multi-NIC support * refactor: Enhance error messages and logging in networking module * refactor: Simplify interface argument handling and improve logging in network configuration and cleanup * refactor(network): simplify startup integration and improve error handling - Move config access and error handling into network::configure_internet_gateway() - Simplify startup.rs to single function call without parameters - Remove redundant check_network_privileges() function - Improve error handling by checking actual command output instead of pre-validation - Better separation of concerns between startup and network modules Addresses feedback from PR discussion: https://github.com/moghtech/komodo/pull/719#discussion_r2261542921 * fix(config): update default internet interface setting Addresses feedback from PR discussion: https://github.com/moghtech/komodo/pull/719#discussion_r2261552279 * fix(config): remove custom default for internet interface in CoreConfig * move mod.rs -> network.rs Addresses feedback from PR discussion: https://github.com/moghtech/komodo/pull/719#discussion_r2261558332 * add internet interface example * docs(build-images): document multi-platform builds with Docker Buildx (#721) * docs(build-images): add multi-platform buildx guide to builders.md * docs(build-images): add multi-platform buildx guide and clarify platform selection in Komodo UI Extra Args field * move to 1.19.0 * core support reading from multiple config files * config support yaml * deploy 1.19.0-dev-1 * deploy 1.19.0-dev-2 * add default komodo cli config * better config merge with base * no need to panic if empty config paths * improve km --help * prog on cli docs * tweak cli docs * tweak doc * split the runfile commands * update docsite deps * km ps initial * km ls * list resource apis * km con inspect * deploy 1.19.0-dev-3 * fix: need serde default * dev-4 fix container parsing issue * tweak * use include-based file finding for much faster discovery * just move to standard config dir .config/komodo/komodo.cli.* * update fe w/ new contianer info minimal serialization * add links to table names * deploy 1.19.0-dev-5 * links in tables * backend for Action arguments * deploy 1.19.0-dev-6 * deploy 1.19.0-dev-7 * deploy 1.19.0-dev-8 * no space at front of KeyValue default args * webhook branch / body optional * The incoming arguments * deploy 1.19.0-dev-9 * con -> cn * add config -> cf alias * .kmignore * .peripheryinclude * outdated * optional links, configurable table format * table_format -> table_borders * get types * include docsite in yarn install * update runnables command in docs * tweak * improve km ls only show important stuff * Add BackupCoreDatabase * deploy 1.19.0-dev-10 * backup command needs "--yes" * deploy 1.19.0-dev-11 * update rustc 1.89.0 * cli tweak * try chef * Fix chef (after dependencies) * try other compile command * fix * fix comment * cleanup stats page * ensure database backup procedure * UI allow configure Backup Core Database in Procedures * procedure description * deploy 1.19.0-dev-12 * deploy 1.19.0-dev-13 * GlobalAutoUpdate * deploy 1.19.0-dev-14 * default tags and global auto update procedure * deploy 1.19.0-dev-15 * trim the default procedure descriptions * deploy 1.19.0-dev-16 * in "system" theme, also poll for updates to the theme based on time. * Add next run to Action / Procedure column * km ls support filter by templates * fix procedure toml serialization when params = {} * deploy 1.19.0-dev-17 * KOMODO_INIT_ADMIN_USERNAME * KOMODO_FIRST_SERVER_NAME * add server.config.external_address for use with links * deploy 1.19.0-dev-18 * improve auto prune * fix system theme auto update * deploy 1.19.0-dev-19 * rename auth/CreateLocalUser -> SignUpLocalUser. Add write/CreateLocalUser for in-ui initialization. * deploy 1.19.0-dev-20 * UI can handle multiple active logins * deploy 1.19.0-dev-21 * fix * add logout function * fix oauth redirect * fix multi user exchange token function * default external address * just Add * style account switcher * backup and restore docs * rework docsite file / sidebar structure, start auto update docs * auto update docs * tweak * fix doc links * only pull / update running stacks / deployments images * deploy 1.19.0-dev-22 * deploy 1.19.0-dev-23 * fix #737 * community docs * add BackupCoreDatabase link to docs * update ferret v2 update guide using komodo-cli * fix data table headers overlapping topbar * don't alert when deploying * CommitSync returns Update * deploy 1.19.0-dev-24 * trim the decoded branch * action uses file contents deserializer * deploy 1.19.0-dev-25 * remove Toml from action args format * clarify External Address purpose * Fix podman compatibility in `get_container_stats` (#739) * Add podman compability for querying stats Podman and docker stats differ in results in significant ways but this filter change they will output the same stats * syntax fix * feat(dashboard): display CPU, memory, and disk usage on server cards (#729) * feat: mini-stats-card: Expose Server CPU , Memory, Disk Usage to Dashboard View * comment: resolved * Feat: fix overflow card , DRY stats-mini, add unreachable mini stats * lint: fix * deploy 1.19.0-dev-26 * 1.19.0 * linux, macos container install * cli main config --------- Co-authored-by: Brian Bradley <brian.bradley.p@gmail.com> Co-authored-by: Daniel <daniel.barabasa@gmail.com> Co-authored-by: eleith <eleith@users.noreply.github.com> Co-authored-by: eleith <online-github@eleith.com> Co-authored-by: Sam Edwards <sam@samedwards.ca> Co-authored-by: Marcel Pfennig <82059270+MP-Tool@users.noreply.github.com> Co-authored-by: itsmesid <693151+arevindh@users.noreply.github.com> Co-authored-by: mbecker20 <max@mogh.tech> Co-authored-by: Rhyn <Rhyn@users.noreply.github.com> Co-authored-by: Anh Nguyen <tuananh131001@gmail.com>
321 lines
7.7 KiB
Rust
321 lines
7.7 KiB
Rust
use std::path::Path;
|
|
|
|
use anyhow::Context;
|
|
use colored::Colorize;
|
|
use komodo_client::entities::{
|
|
config::cli::args::database::DatabaseCommand, optional_string,
|
|
};
|
|
|
|
use crate::{command::sanitize_uri, config::cli_config};
|
|
|
|
pub async fn handle(command: &DatabaseCommand) -> anyhow::Result<()> {
|
|
match command {
|
|
DatabaseCommand::Backup { yes, .. } => backup(*yes).await,
|
|
DatabaseCommand::Restore {
|
|
restore_folder,
|
|
index,
|
|
yes,
|
|
..
|
|
} => restore(restore_folder.as_deref(), *index, *yes).await,
|
|
DatabaseCommand::Prune { yes, .. } => prune(*yes).await,
|
|
DatabaseCommand::Copy { yes, index, .. } => {
|
|
copy(*index, *yes).await
|
|
}
|
|
}
|
|
}
|
|
|
|
async fn backup(yes: bool) -> anyhow::Result<()> {
|
|
let config = cli_config();
|
|
|
|
println!(
|
|
"\n🦎 {} Database {} Utility 🦎",
|
|
"Komodo".bold(),
|
|
"Backup".green().bold()
|
|
);
|
|
println!(
|
|
"\n{}\n",
|
|
" - Backup all database contents to gzip compressed files."
|
|
.dimmed()
|
|
);
|
|
if let Some(uri) = optional_string(&config.database.uri) {
|
|
println!("{}: {}", " - Source URI".dimmed(), sanitize_uri(&uri));
|
|
}
|
|
if let Some(address) = optional_string(&config.database.address) {
|
|
println!("{}: {address}", " - Source Address".dimmed());
|
|
}
|
|
if let Some(username) = optional_string(&config.database.username) {
|
|
println!("{}: {username}", " - Source Username".dimmed());
|
|
}
|
|
println!(
|
|
"{}: {}\n",
|
|
" - Source Db Name".dimmed(),
|
|
config.database.db_name,
|
|
);
|
|
println!(
|
|
"{}: {:?}",
|
|
" - Backups Folder".dimmed(),
|
|
config.backups_folder
|
|
);
|
|
if config.max_backups == 0 {
|
|
println!(
|
|
"{}{}",
|
|
" - Backup pruning".dimmed(),
|
|
"disabled".red().dimmed()
|
|
);
|
|
} else {
|
|
println!("{}: {}", " - Max Backups".dimmed(), config.max_backups);
|
|
}
|
|
|
|
crate::command::wait_for_enter("start backup", yes)?;
|
|
|
|
let db = database::init(&config.database).await?;
|
|
|
|
database::utils::backup(&db, &config.backups_folder).await?;
|
|
|
|
// Early return if backup pruning disabled
|
|
if config.max_backups == 0 {
|
|
return Ok(());
|
|
}
|
|
|
|
// Know that new backup was taken successfully at this point,
|
|
// safe to prune old backup folders
|
|
|
|
prune_inner().await
|
|
}
|
|
|
|
async fn restore(
|
|
restore_folder: Option<&Path>,
|
|
index: bool,
|
|
yes: bool,
|
|
) -> anyhow::Result<()> {
|
|
let config = cli_config();
|
|
|
|
println!(
|
|
"\n🦎 {} Database {} Utility 🦎",
|
|
"Komodo".bold(),
|
|
"Restore".purple().bold()
|
|
);
|
|
println!(
|
|
"\n{}\n",
|
|
" - Restores database contents from gzip compressed files."
|
|
.dimmed()
|
|
);
|
|
if let Some(uri) = optional_string(&config.database_target.uri) {
|
|
println!("{}: {}", " - Target URI".dimmed(), sanitize_uri(&uri));
|
|
}
|
|
if let Some(address) =
|
|
optional_string(&config.database_target.address)
|
|
{
|
|
println!("{}: {address}", " - Target Address".dimmed());
|
|
}
|
|
if let Some(username) =
|
|
optional_string(&config.database_target.username)
|
|
{
|
|
println!("{}: {username}", " - Target Username".dimmed());
|
|
}
|
|
println!(
|
|
"{}: {}",
|
|
" - Target Db Name".dimmed(),
|
|
config.database_target.db_name,
|
|
);
|
|
if !index {
|
|
println!(
|
|
"{}: {}",
|
|
" - Target Db Indexing".dimmed(),
|
|
"DISABLED".red(),
|
|
);
|
|
}
|
|
println!(
|
|
"\n{}: {:?}",
|
|
" - Backups Folder".dimmed(),
|
|
config.backups_folder
|
|
);
|
|
if let Some(restore_folder) = restore_folder {
|
|
println!("{}: {restore_folder:?}", " - Restore Folder".dimmed());
|
|
}
|
|
|
|
crate::command::wait_for_enter("start restore", yes)?;
|
|
|
|
let db = if index {
|
|
database::Client::new(&config.database_target).await?.db
|
|
} else {
|
|
database::init(&config.database_target).await?
|
|
};
|
|
|
|
database::utils::restore(
|
|
&db,
|
|
&config.backups_folder,
|
|
restore_folder,
|
|
)
|
|
.await
|
|
}
|
|
|
|
async fn prune(yes: bool) -> anyhow::Result<()> {
|
|
let config = cli_config();
|
|
|
|
println!(
|
|
"\n🦎 {} Database {} Utility 🦎",
|
|
"Komodo".bold(),
|
|
"Backup Prune".cyan().bold()
|
|
);
|
|
println!(
|
|
"\n{}\n",
|
|
" - Prunes database backup folders when greater than the configured amount."
|
|
.dimmed()
|
|
);
|
|
println!(
|
|
"{}: {:?}",
|
|
" - Backups Folder".dimmed(),
|
|
config.backups_folder
|
|
);
|
|
if config.max_backups == 0 {
|
|
println!(
|
|
"{}{}",
|
|
" - Backup pruning".dimmed(),
|
|
"disabled".red().dimmed()
|
|
);
|
|
} else {
|
|
println!("{}: {}", " - Max Backups".dimmed(), config.max_backups);
|
|
}
|
|
|
|
// Early return if backup pruning disabled
|
|
if config.max_backups == 0 {
|
|
info!(
|
|
"Backup pruning is disabled, enabled using 'max_backups' (KOMODO_CLI_MAX_BACKUPS)"
|
|
);
|
|
return Ok(());
|
|
}
|
|
|
|
crate::command::wait_for_enter("start backup prune", yes)?;
|
|
|
|
prune_inner().await
|
|
}
|
|
|
|
async fn prune_inner() -> anyhow::Result<()> {
|
|
let config = cli_config();
|
|
|
|
let mut backups_dir =
|
|
match tokio::fs::read_dir(&config.backups_folder)
|
|
.await
|
|
.context("Failed to read backups folder for prune")
|
|
{
|
|
Ok(backups_dir) => backups_dir,
|
|
Err(e) => {
|
|
warn!("{e:#}");
|
|
return Ok(());
|
|
}
|
|
};
|
|
|
|
let mut backup_folders = Vec::new();
|
|
loop {
|
|
match backups_dir.next_entry().await {
|
|
Ok(Some(entry)) => {
|
|
let Ok(metadata) = entry.metadata().await else {
|
|
continue;
|
|
};
|
|
if metadata.is_dir() {
|
|
backup_folders.push(entry.path());
|
|
}
|
|
}
|
|
Ok(None) => break,
|
|
Err(_) => {
|
|
continue;
|
|
}
|
|
}
|
|
}
|
|
// Ordered from oldest -> newest
|
|
backup_folders.sort();
|
|
|
|
let max_backups = config.max_backups as usize;
|
|
let backup_folders_len = backup_folders.len();
|
|
|
|
// Early return if under the backup count threshold
|
|
if backup_folders_len <= max_backups {
|
|
info!("No backups to prune");
|
|
return Ok(());
|
|
}
|
|
|
|
let to_delete =
|
|
&backup_folders[..(backup_folders_len - max_backups)];
|
|
|
|
info!("Pruning old backups: {to_delete:?}");
|
|
|
|
for path in to_delete {
|
|
if let Err(e) =
|
|
tokio::fs::remove_dir_all(path).await.with_context(|| {
|
|
format!("Failed to delete backup folder at {path:?}")
|
|
})
|
|
{
|
|
warn!("{e:#}");
|
|
}
|
|
}
|
|
|
|
Ok(())
|
|
}
|
|
|
|
async fn copy(index: bool, yes: bool) -> anyhow::Result<()> {
|
|
let config = cli_config();
|
|
|
|
println!(
|
|
"\n🦎 {} Database {} Utility 🦎",
|
|
"Komodo".bold(),
|
|
"Copy".blue().bold()
|
|
);
|
|
println!(
|
|
"\n{}\n",
|
|
" - Copies database contents to another database.".dimmed()
|
|
);
|
|
|
|
if let Some(uri) = optional_string(&config.database.uri) {
|
|
println!("{}: {}", " - Source URI".dimmed(), sanitize_uri(&uri));
|
|
}
|
|
if let Some(address) = optional_string(&config.database.address) {
|
|
println!("{}: {address}", " - Source Address".dimmed());
|
|
}
|
|
if let Some(username) = optional_string(&config.database.username) {
|
|
println!("{}: {username}", " - Source Username".dimmed());
|
|
}
|
|
println!(
|
|
"{}: {}\n",
|
|
" - Source Db Name".dimmed(),
|
|
config.database.db_name,
|
|
);
|
|
|
|
if let Some(uri) = optional_string(&config.database_target.uri) {
|
|
println!("{}: {}", " - Target URI".dimmed(), sanitize_uri(&uri));
|
|
}
|
|
if let Some(address) =
|
|
optional_string(&config.database_target.address)
|
|
{
|
|
println!("{}: {address}", " - Target Address".dimmed());
|
|
}
|
|
if let Some(username) =
|
|
optional_string(&config.database_target.username)
|
|
{
|
|
println!("{}: {username}", " - Target Username".dimmed());
|
|
}
|
|
println!(
|
|
"{}: {}",
|
|
" - Target Db Name".dimmed(),
|
|
config.database_target.db_name,
|
|
);
|
|
if !index {
|
|
println!(
|
|
"{}: {}",
|
|
" - Target Db Indexing".dimmed(),
|
|
"DISABLED".red(),
|
|
);
|
|
}
|
|
|
|
crate::command::wait_for_enter("start copy", yes)?;
|
|
|
|
let source_db = database::init(&config.database).await?;
|
|
let target_db = if index {
|
|
database::Client::new(&config.database_target).await?.db
|
|
} else {
|
|
database::init(&config.database_target).await?
|
|
};
|
|
|
|
database::utils::copy(&source_db, &target_db).await
|
|
}
|