Files
komodo/bin/core/src/cloud/hetzner/mod.rs
Maxwell Becker 7d4376f426 1.15.0 (#90)
* attach env_file to compose build and compose pull stages

* fmt and bump rust version

* bump dependencies

* ignored for Sqlite message

* fix Build secret args info

* improve secret arguments info

* improve environment, ports, volumes deserializers

* rename `mongo` to `database` in config

* support _FILE in secret env vars

* improve setup - simpler compose

* remove aws ecr container registry support, alpine dockerfiles

* log periphery config

* ssl_enabled mode

* log http vs https

* periphery client accept untrust ssl certs

* fix nav issue from links

* configurable ssl

* KOMODO_ENSURE_SERVER -> KOMODO_FIRST_SERVER

* mount proc and ssl volume

* managed sync

* validate files on host resource path

* remove sync repo not configured guards

* disable confirm dialog

* fix sync hash / message Option

* try dev dockerfile

* refresh sync resources after commit

* socket invalidate handling

* delete dev dockerfile

* Commit Changes

* Add Info tab to syncs

* fix new Info parsing issue with serde default

* refresh stack cache on create / update

* managed syncs can't sync themselves

* managed syncs seems to work

* bump thiserror

* use alpine as main dockerfile

* apt add --no-cache

* disable user write perms, super admin perms to manage admins

* manage admin user UI

* implement disable non admin create frontend

* disable create non admin

* Copy button shown based on permission

* warning message on managed sync

* implement monaco editor

* impl simple match tags config

* resource sync support match tags

* more match tag filtering

* improve config with better saving diffs

* export button use monaco

* deser Conversions with wrapping strings

* envs editing

* don't delete variables / user groups if match tags defined

* env from_str improve

* improve dashboards

* remove core ca stuff for now

* move periphery ssl gen to dedicated file

* default server address periphery:8120

* clean up ssl configs

* server dashboard

* nice test compose

* add discord alerter

* discord alerter

* stack hideInfo logic

* compose setup

* alert table

* improve config hover card style

* update min editor height and stack config

* Feat: Styling Updates (#94)

* sidebar takes full screen height

* add bg accent to navbar

* add aschild prop to topbar alerts trigger

* stylize resource rows

* internally scrollable data tables

* better hover color for outlined button

* always show scrollbar to prevent layout shift

* better hover color for navbar

* rearrange buttons

* fix table and resource row styles

* cleanup scrollbar css

* use page for dashboard instead of section

* fix padding

* resource sync refactor and env keep comments

* frontend build

* improve configs

* config nice

* Feat/UI (#95)

* stylize resource rows

* internally scrollable data tables

* fix table and resource row styles

* use page for dashboard instead of section

* fix padding

* add `ResourcePageHeader` to required components

* add generic resource page header component

* add resource page headers for all components

* add resource notificaitons component

* add `TextUpdateMenu2` for use in resource page

* cleanup resource notificaitons

* update resource page layout

* ui edits

* sync kind of work

* clean up unused import

* syncs seem to work

* new sync pending

* monaco diff hide unchanged regions

* update styling all in config  resource select links

* confirm update default strings

* move procedure Add Stage to left

* update colors / styles

* frontend build

* backend for write file contents to host

* compose reference ports comment out

* server config

* ensure parent directory created

* fix frontend build

* remove default stack run_directory

* fix periphery compose deploy response set

* update compose files

* move server stats under tabs

* fix deployment list item getting correct image when not deployed

* stack updates cache after file write

* edit files on host

* clean up unused imports

* top level config update assignment must be spread

* update deps, move alert module

* move stack module

* move sync module

* move to sync db_client usage after init

* support generic OIDC provider

* init builders / server templates specifying https

* special cases for server / deployment state

* improve alert details

* add builder template `use_https` config

* try downgrade aws sdk ec2 for x86 build

* update debian dockerfiles to rm lists/*

* optionally configure seperate KOMODO_OIDC_REDIRECT

* add defaults to compose.env

* keep tags / search right aligned when view only

* clean up configs

* remove unused migrator deps

* update roadmap support generic OIDC

* initialize sync use confirm button

* key_value syntax highlighting

* smaller debian dockerfiles

* clean up deps.sh

* debian dockerifle

* New config layout (#96)

* new config layout

* fix image config layout and components config

* fix dom nesting and cleanup components

* fix label, make switches flex row

* ensure smooth scroll on hash navigations

* width 180 on config sidebar

* slight edits to config

* log whether https builder

* DISABLED <switch> ENABLED

* fix some more config

* smaller checked component

* server config looking good

* auto initialize compose files when files on host

* stack files on host good

* stack config nice

* remove old config

* deployments looking good

* build looking good

* Repo good

* nice config for builders

* alerter good

* server template config

* syncs good

* tweak stack config

* use status badge for update tables

* unified update page using router params

* replace /updates with unified updates page

* redirect all resource updates to unified update page

* fix reset handling

* unmount legacy page

* try periphery rustls

* rm unused import

* fix broken deps

* add unified alerts apge

* mount new alerts, remove old alerts page

* reroute resource alerts to unified alerts page

* back to periphery openssl

* ssl_enabled defaults to false for backward compat

* reqwest need json feature

* back to og yaml monaco

* Uncomment config fields for clearer config

* clean up compose env

* implement pull or clone, avoid deleting repo directory

* refactor mongo configuration params

* all configs respect empty string null

* add back status to header

* build toml don't have version if not auto incrementing

* fix comile

* fix repo pull cd to correct dir

* fix core pull_or_clone directory

* improve statuses

* remove ' ' from kv list parser

* longer CSRF valid for, to give time to login / accept

* don't compute diff / execute if there are any file_errors

* PartialBuilderConfig enum user inner option

* move errors to top

* fix toml init serializer

* server template and bulder manually add config.params line

* better way to check builder / template params empty

* improve build configs

* merge links into network area deployment

* default periphery config

* improve SystemCommand editor

* better Repo server / builder Info

* improve Alerts / Updates with ResourceSelector

* fix unused frontend

* update ResourceSync description

* toml use [resource.config] syntax

* update toml syntax

* update Build.image_registry schema

* fix repo / stack resource link alias

* reorder image registry

* align toml / yaml parser style

* some config updates

---------

Co-authored-by: Karamvir Singh <67458484+karamvirsingh98@users.noreply.github.com>
Co-authored-by: kv <karamvir.singh98@gmail.com>
2024-10-06 23:54:23 -07:00

282 lines
7.8 KiB
Rust

use std::{
sync::{Arc, Mutex, OnceLock},
time::Duration,
};
use anyhow::{anyhow, Context};
use futures::future::join_all;
use komodo_client::entities::server_template::hetzner::{
HetznerDatacenter, HetznerServerTemplateConfig, HetznerServerType,
HetznerVolumeFormat,
};
use crate::{
cloud::hetzner::{
common::HetznerServerStatus, create_server::CreateServerBody,
create_volume::CreateVolumeBody,
},
config::core_config,
};
use self::{client::HetznerClient, common::HetznerVolumeStatus};
mod client;
mod common;
mod create_server;
mod create_volume;
fn hetzner() -> Option<&'static HetznerClient> {
static HETZNER_CLIENT: OnceLock<Option<HetznerClient>> =
OnceLock::new();
HETZNER_CLIENT
.get_or_init(|| {
let token = &core_config().hetzner.token;
(!token.is_empty()).then(|| HetznerClient::new(token))
})
.as_ref()
}
pub struct HetznerServerMinimal {
pub id: i64,
pub ip: String,
}
const POLL_RATE_SECS: u64 = 3;
const MAX_POLL_TRIES: usize = 100;
#[instrument]
pub async fn launch_hetzner_server(
name: &str,
config: HetznerServerTemplateConfig,
) -> anyhow::Result<HetznerServerMinimal> {
let hetzner =
*hetzner().as_ref().context("Hetzner token not configured")?;
let HetznerServerTemplateConfig {
image,
datacenter,
private_network_ids,
placement_group,
enable_public_ipv4,
enable_public_ipv6,
firewall_ids,
server_type,
ssh_keys,
user_data,
use_public_ip,
labels,
volumes,
port: _,
use_https: _,
} = config;
let datacenter = hetzner_datacenter(datacenter);
// Create volumes and get their ids
let mut volume_ids = Vec::new();
for volume in volumes {
let body = CreateVolumeBody {
name: volume.name,
format: Some(hetzner_format(volume.format)),
location: Some(datacenter.into()),
labels: volume.labels,
size: volume.size_gb,
automount: None,
server: None,
};
let id = hetzner
.create_volume(&body)
.await
.context("failed to create hetzner volume")?
.volume
.id;
volume_ids.push(id);
}
// Make sure volumes are available before continue
let vol_ids_poll = Arc::new(Mutex::new(volume_ids.clone()));
for _ in 0..MAX_POLL_TRIES {
if vol_ids_poll.lock().unwrap().is_empty() {
break;
}
tokio::time::sleep(Duration::from_secs(POLL_RATE_SECS)).await;
let ids = vol_ids_poll.lock().unwrap().clone();
let futures = ids.into_iter().map(|id| {
let vol_ids = vol_ids_poll.clone();
async move {
let Ok(res) = hetzner.get_volume(id).await else {
return;
};
if matches!(res.volume.status, HetznerVolumeStatus::Available)
{
vol_ids.lock().unwrap().retain(|_id| *_id != id);
}
}
});
join_all(futures).await;
}
if !vol_ids_poll.lock().unwrap().is_empty() {
return Err(anyhow!("Volumes not ready after poll"));
}
let body = CreateServerBody {
name: name.to_string(),
automount: None,
datacenter: Some(datacenter),
location: None,
firewalls: firewall_ids
.into_iter()
.map(|firewall| create_server::Firewall { firewall })
.collect(),
image,
labels,
networks: private_network_ids,
placement_group: (placement_group > 0).then_some(placement_group),
public_net: create_server::PublicNet {
enable_ipv4: enable_public_ipv4,
enable_ipv6: enable_public_ipv6,
ipv4: None,
ipv6: None,
},
server_type: hetzner_server_type(server_type),
ssh_keys,
start_after_create: true,
user_data: (!user_data.is_empty()).then_some(user_data),
volumes: volume_ids,
};
let server_id = hetzner
.create_server(&body)
.await
.context("failed to create hetnzer server")?
.server
.id;
for _ in 0..MAX_POLL_TRIES {
tokio::time::sleep(Duration::from_secs(POLL_RATE_SECS)).await;
let Ok(res) = hetzner.get_server(server_id).await else {
continue;
};
if matches!(res.server.status, HetznerServerStatus::Running) {
let ip = if use_public_ip {
res
.server
.public_net
.ipv4
.context("instance does not have public ipv4 attached")?
.ip
} else {
res
.server
.private_net
.first()
.context("no private networks attached")?
.ip
.to_string()
};
let server = HetznerServerMinimal { id: server_id, ip };
return Ok(server);
}
}
Err(anyhow!(
"failed to verify server running after polling status"
))
}
fn hetzner_format(
format: HetznerVolumeFormat,
) -> common::HetznerVolumeFormat {
match format {
HetznerVolumeFormat::Xfs => common::HetznerVolumeFormat::Xfs,
HetznerVolumeFormat::Ext4 => common::HetznerVolumeFormat::Ext4,
}
}
fn hetzner_datacenter(
datacenter: HetznerDatacenter,
) -> common::HetznerDatacenter {
match datacenter {
HetznerDatacenter::Nuremberg1Dc3 => {
common::HetznerDatacenter::Nuremberg1Dc3
}
HetznerDatacenter::Helsinki1Dc2 => {
common::HetznerDatacenter::Helsinki1Dc2
}
HetznerDatacenter::Falkenstein1Dc14 => {
common::HetznerDatacenter::Falkenstein1Dc14
}
HetznerDatacenter::AshburnDc1 => {
common::HetznerDatacenter::AshburnDc1
}
HetznerDatacenter::HillsboroDc1 => {
common::HetznerDatacenter::HillsboroDc1
}
HetznerDatacenter::SingaporeDc1 => {
common::HetznerDatacenter::SingaporeDc1
}
}
}
fn hetzner_server_type(
server_type: HetznerServerType,
) -> common::HetznerServerType {
match server_type {
HetznerServerType::SharedAmd2Core2Ram40Disk => {
common::HetznerServerType::SharedAmd2Core2Ram40Disk
}
HetznerServerType::SharedArm2Core4Ram40Disk => {
common::HetznerServerType::SharedArm2Core4Ram40Disk
}
HetznerServerType::SharedIntel2Core4Ram40Disk => {
common::HetznerServerType::SharedIntel2Core4Ram40Disk
}
HetznerServerType::SharedAmd3Core4Ram80Disk => {
common::HetznerServerType::SharedAmd3Core4Ram80Disk
}
HetznerServerType::SharedArm4Core8Ram80Disk => {
common::HetznerServerType::SharedArm4Core8Ram80Disk
}
HetznerServerType::SharedIntel4Core8Ram80Disk => {
common::HetznerServerType::SharedIntel4Core8Ram80Disk
}
HetznerServerType::SharedAmd4Core8Ram160Disk => {
common::HetznerServerType::SharedAmd4Core8Ram160Disk
}
HetznerServerType::SharedArm8Core16Ram160Disk => {
common::HetznerServerType::SharedArm8Core16Ram160Disk
}
HetznerServerType::SharedIntel8Core16Ram160Disk => {
common::HetznerServerType::SharedIntel8Core16Ram160Disk
}
HetznerServerType::SharedAmd8Core16Ram240Disk => {
common::HetznerServerType::SharedAmd8Core16Ram240Disk
}
HetznerServerType::SharedArm16Core32Ram320Disk => {
common::HetznerServerType::SharedArm16Core32Ram320Disk
}
HetznerServerType::SharedIntel16Core32Ram320Disk => {
common::HetznerServerType::SharedIntel16Core32Ram320Disk
}
HetznerServerType::SharedAmd16Core32Ram360Disk => {
common::HetznerServerType::SharedAmd16Core32Ram360Disk
}
HetznerServerType::DedicatedAmd2Core8Ram80Disk => {
common::HetznerServerType::DedicatedAmd2Core8Ram80Disk
}
HetznerServerType::DedicatedAmd4Core16Ram160Disk => {
common::HetznerServerType::DedicatedAmd4Core16Ram160Disk
}
HetznerServerType::DedicatedAmd8Core32Ram240Disk => {
common::HetznerServerType::DedicatedAmd8Core32Ram240Disk
}
HetznerServerType::DedicatedAmd16Core64Ram360Disk => {
common::HetznerServerType::DedicatedAmd16Core64Ram360Disk
}
HetznerServerType::DedicatedAmd32Core128Ram600Disk => {
common::HetznerServerType::DedicatedAmd32Core128Ram600Disk
}
HetznerServerType::DedicatedAmd48Core192Ram960Disk => {
common::HetznerServerType::DedicatedAmd48Core192Ram960Disk
}
}
}