Compare commits

...

126 Commits

Author SHA1 Message Date
Maxwell Becker
765e5a0df1 1.17.4 (#446)
* add terminal (ssh) apis

* add core terminal exec method

* terminal typescript client method

* terminals WIP

* backend for pty

* add ts responses

* about wire everything

* add new blog

* credit Skyfay

* working

* regen lock

* 1.17.4-dev-1

* pty history

* replace the test terminal impl with websocket (pty)

* create api and improve frontend

* fix fe

* terminals

* disable terminal api on periphery

* implement write level terminal perms

* remove unneeded

* fix clippy

* delete unneeded

* fix waste cpu cycles

* set TERM and COLORTERM for shell environment

* fix xterm scrolling behavior

* starship promp in periphery container terminal

* kill all terminals on periphery shutdown signal

* improve starship config and enable ssl in compose

* use same scrollTop setter

* fix periphery container distribution link

* support custom command / args to init terminal

* allow fully configurable init command

* docker exec into container

* add permissioning for container exec

* add starship to core container

* add delete all terminals

* dev-2

* finished gen client

* core need curl

* hide Terminal trigger if disabled

* 1.17.4
2025-04-27 15:53:23 -07:00
mbecker20
76f2f61be5 1.17.3 fix Build pre_build functionality. 2025-04-24 22:03:46 -04:00
Maxwell Becker
b43e2918da 1.17.2 (#409)
* start on cron schedules

* rust 1.86.0

* config periphery directories easier with PERIPHERY_ROOT_DIRECTORY

* schedule backend

* fix config switch toggling through disabled

* procedure schedule working

* implement schedules for actions

* update schedule immediately after last run

* improve config update logs using toml diffs backend

* improve the config update logs with TOML diff view

* add schedule alerting

* version 1.17.2

* Set TZ in core env

* dev-1

* better term signal labels

* sync configurable pending alert send

* fix monaco editor height on larger screen

* poll update until complete on client

update lib

* add logger.pretty option for both core and periphery

* fix pretty

* configure schedule alert

* configure failure alert

* dev-3

* 1.17.2

* fmt

* added pushover alerter (#421)

* fix up pushover

* fix some clippy

---------

Co-authored-by: Alex Shore <alex@shore.me.uk>
2025-04-18 23:14:10 -07:00
Etienne
f45205011e Fix compose.env (#415)
Flip "stacks" and "repos" paths
2025-04-15 12:11:48 -07:00
Maxwell Becker
5b211fb8f0 1.17.1 (#383)
* interpolate into slack / discord url

* fix js client docs

* js client should be type: module

* click table tags to toggle tag filter

* git token helper early return when empty provider

* reorder Stack fields

* action support interpolation doc

* Fix for the {account} login fails when the account name contains '$' (#385)

* GetDeploymentsSummary (#386)

* added GetDockerContainersSummary endpoint in rust api

* typescript stuff

* more autogenned typescript stuff

* fixed comments to be in line with actual behaviour

* fixed ReadResponse for GetDockerContainersSummary

* I64 -> u32 for response types

* more accurate error context

* backend for build files on host / ui defined

* core api supports non repo based build

* Ntfy as Alerter (#404)

* add ntfy alerter

* add ntfy alerter

* add ntfy alerter

---------

Co-authored-by: GFXSpeed <github@schafhaupt.com>

* Improve docs around running periphery in a container (#402)

* Add ports that need to be exposed if periphery is remote from core

* Spelling: overide -> override

* Add info about using a custom config file when running periphery in docker

* clean up ntfy alerter

* clean up ResourceSyncConfig

* update build cache after create / update

* refresh stack cache log

* Build UI Defined / file on host frontend

* update clap + rustls

* don't cleanup build files

* clean up dockerfile full path

* update BuildListItemInfo + UI table

* add Other Resources page

* add 5 second ws reconnection timeout

* Make listener address configurable & Add support for IPv6 (#405)

* Make listener address configurable

* Make listener address configurable for periphery

* rename listener_address -> bind_ip

---------

Co-authored-by: Jacky Fong <hello@huzky.dev>
Co-authored-by: Alex Shore <kntrllr@gmail.com>
Co-authored-by: Niklas <108990063+GFXSpeed@users.noreply.github.com>
Co-authored-by: GFXSpeed <github@schafhaupt.com>
Co-authored-by: theRAAPster <theraapster@gmail.com>
Co-authored-by: Daniel Vos <45820840+vosdev@users.noreply.github.com>
2025-04-14 14:48:12 -07:00
Justin Goette
3f04614303 docs: Typo (#392)
FerretDB vs. FerritDB
2025-04-02 00:05:29 -07:00
mbecker20
4c3210f70a update dashboard screenshots with colored tags 2025-03-23 23:17:33 -07:00
mbecker20
2c37ac26a5 ts client 1.17.0 2025-03-23 17:42:47 -07:00
Maxwell Becker
db1cf786ac 1.17.0 (#248)
* resolver v3

add new ec2 instance types

clean up testing config

document the libraries a bit

clean up main

update sysinfo and otel

update client resolver 3.0

resolver v3 prog

clean up gitignore

implement periphery resolver v3

clean up

core read api v3

more prog

execute api

missing apis

compiling

1.16.13

work on more granular traits

prog on crud

* fmt

* format

* resource2 not really a benefit

* axum to 0.8

* bump aws deps

* just make it 1.17.0

* clean up cors

* the komodo env file should be highest priority over additional files

* add entities / message for test alerter

* test alert implementation

* rust 1.84.0

* axum update :param to {param} syntax

* fix last axum updates

* Add test alerter button

* higher quality / colored icons

* komodo-logo

* simplify network stats

* rename Test Alerter button

* escape incoming sync backslashes (BREAKING)

* clean up rust client websocket subscription

* finish oidc comment

* show update available stack table

* update available deployment table

* feature: use the repo path instead of name in GetLatestCommit (#282)

* Update repo path handling in commit fetching

- Changed `name` to `path` for repository identification.
- Updated cache update function to use the new path field.
- Improved error message for non-directory repo paths.

* feat: use optional name and path in GetLatestCommit

* review: don't use optional for name

* review: use helper

* review: remove redundant to_string()

* 1.17.0-dev

* feature: add post_deploy command (#288)

* feature: add post_deploy command

* review: do not run post_deploy if deploy failed

* feature: interpolate secrets in custom alerter (#289)

* feature: interpolate secrets in custom alerter

* fix rust warning

* review: sanitize errors

* review: sanitize error message

* Remove .git from remote_url (#299)

Remove .git from remote_url

Co-authored-by: Deon Marshall <dmarshall@ccp.com.au>

* mbecker20 -> moghtech

* remove example from cargo toml workspace

* dev-1

* fix login screen logo

* more legible favicon

* fix new compose images

* docs new organization

* typescript subscribe_to_update_websocket

* add donate button docsite

* add config save button in desktop sidebar navigator

* add save button to config bottom

* feature: allow docker image text to overflow in table (#301)

* feature: allow docker image text to overflow in table

* review: use break-words

* wip: revert line break in css file

* feature: update devcontainer node release

* improve First Login docs

* FIx PullStack re #302 and record docker compose config on stack deploy

* requery alerts more often

* improve update indicator style and also put on home screen

* Add all services stack log

* 1.17.0-dev-2

* fix api name chnage

* choose which stack services to include in logs

* feature: improve tables quick actions on mobile (#312)

* feature: improve tables quick actions on mobile

* review: fix gap4

* review: use flex-wrap

* improve pull to git init on existing folder without .git

* Fix unclear ComposePull log re #244

* use komodo_client.subscribe_to_update_websocket, and click indicator to reconnect

* dev-3

* ServerTemplate description

* improve WriteComposeContentsToHost instrument fields

* give server stat charts labels

* filters wrap

* show provider usernames from config file

* Stack: Fix git repo new compose file initialization

* init sync file new repo

* set branch on git init folder

* ResourceSync: pending view toggle between "Execute" vs "Commit" sync direction

* Improve resource sync Execute / Pending view selector

* standardize running commands with interpolation / output sanitizations

* fix all clippy lints

* fix rand

* lock certain users username / password, prevent demo creds from being changed.

* revert to login screen whenever the call to check login fails

* ResourceSync state resolution refinement

* make sure parent directories exist whenever writing files

* don't prune images if server not enabled

* update most deps

* update openidconnect dependency, and use reqwest rustls-tls-native-roots

* dev-4

* resource sync only add escaping on toml between the """

* Stacks executions take list of services -- Auto update only redeploys services with update

* auto update all service deploy option

* dev-5 fix the stack service executions

* clean up service_args

* rust 1.85

* store sync edits on localstorage

* stack edits on localstorage and show last deployed config

* add yarn install to runfile

* Fix actions when core on https

* add update_available query parameter to filter for only stacks /deployments with available update

* rust 2024 and fmt

* rename test.compose.yaml to dev.compose.yaml, and update runfile

* update .devcontainer / dev docs for updated runfile

* use png in topbar logo, svg quality sometimes bad

* OIDC: Support PKCE auth (secret optional)

* update docs on OIDC and client secret

* cycle the oidc client on interval to ensure up to date JWKs

* add KOMODO_LOCK_LOGIN_CREDENTIALS_FOR in config doc

* update deps

* resource sync toggle resource / variable / user group inclusion independantly

* use jsonwebtoken

* improve variable value table overflow

* colored tags

* fix sync summary count ok

* default new tag colors to grey

* soften tag opacity a bit

* Update config.tsx (#358)

* isolate stacks / deployments with pending updates

* update some deps

* use Tooltip component instead of HoverCard for mobile compatibility

* batch Build builds

* link to typescript client in the intro

* add link to main docs from client docs

* doc tweaks

* use moghtech/komodo-core and moghtech/komodo-periphery as images

* remove unnecessary explicit network

* periphery.compose.yaml

* clean up periphery compose

* add link to config

* update periphery container compose config

* rust 1.85.1

* update sync docs

* 1.17.0

---------

Co-authored-by: unsync <1211591+unsync@users.noreply.github.com>
Co-authored-by: Deon Marshall <dmarshall@ccp.com.au>
Co-authored-by: komodo <komodo@komo.do>
Co-authored-by: wlatic <jamesoh@gmail.com>
2025-03-23 16:47:06 -07:00
Maarten Kossen
9c841e5bdc Change amd64 to arm64 to prevent installing aarch64 binary on an x86_64 system. (#357) 2025-03-12 19:20:49 -07:00
mbecker20
e385c6e722 use ferretdb:1 2025-02-26 14:55:34 -08:00
Maxwell Becker
9ef25e7575 Create FUNDING.yml 2025-02-13 12:07:48 -08:00
boomam
f945a3014a Update index.mdx (#306)
Added small note on initial login steps.
2025-02-11 00:03:39 -08:00
mbecker20
fdad04d6cb fix KOMODO_DB_USERNAME compose files 2025-02-08 18:45:02 -08:00
mbecker20
c914f23aa8 update compose files re #180 2025-02-08 12:29:26 -08:00
Maarten Kossen
82b2e68cd3 Adding Resource Sync documentation. (#259) 2025-01-11 21:32:02 -08:00
rita7lopes
e274d6f7c8 Network Usage - Ingress Egress per interface and global usage (#229)
* Add network io stats

Add network usage graph and current status

Change network graphs to use network interface from drop down menu

Adjust the type to be able to get general network stats for the general
UI view

Working setup with a working builder

remove changes to these dockerfile

remove lock changes

* change network hashmap to Vector

* adjust all the network_usage_interface to be a vector rather than a hash map

* PR requested changes applied

* Change net_ingress_bytes and egress to network_ingress_bytes egress respectively

* final gen-client types

---------

Co-authored-by: mbecker20 <becker.maxh@gmail.com>
2024-12-21 08:15:21 -08:00
Maxwell Becker
ab8777460d Simplify periphery aio.Dockerfile 2024-12-14 09:13:01 -08:00
Maxwell Becker
7e030e702f Simplify aio.Dockerfile 2024-12-14 09:12:32 -08:00
Maxwell Becker
a869a74002 Fix test.compose.yaml (use aio.Dockerfiles) 2024-12-14 09:11:24 -08:00
mbecker20
1d31110f8c fix multi arch built var reference 2024-12-02 03:33:11 -05:00
mbecker20
bb63892e10 periphery -> periphery-x86_64 setup script 2024-12-02 03:31:55 -05:00
mbecker20
4e554eb2a7 add labels to binary / frontend images 2024-12-02 03:02:00 -05:00
Maxwell Becker
00968b6ea1 1.16.12 (#209)
* inc version

* Komodo interp in ui compose file

* fix auto update when image doesn't specify tag by defaulting to latest

* Pull image buttons don't need safety dialog

* WIP crosscompile

* rename

* entrypoint

* fix copy

* remove example/* from workspace

* add targets

* multiarch pkg config

* use specific COPY

* update deps

* multiarch build command

* pre compile deps

* cross compile

* enable-linger

* remove spammed log when server doesn't have docker

* add multiarch.Dockerfile

* fix casing

* fix tag

* try not let COPY fail

* try

* ARG TARGETPLATFORM

* use /app for consistency

* try

* delete cross-compile approach

* add multiarch core build

* multiarch Deno

* single arch multi arch

* typeshare cli note

* new typeshare

* remove note about aarch64 image

* test configs

* fix config file headers

* binaries dockerfile

* update cargo build

* docs

* simple

* just simple

* use -p

* add configurable binaries tag

* add multi-arch

* allow copy to fail

* fix binary paths

* frontend Dockerfiel

* use dedicated static frontend build

* auto retry getting instance state from aws

* retry 5 times

* cleanup

* simplify binary build

* try alpine and musl

* install alpine deps

* back to debian, try rustls

* move fully to rustls

* single arch builds using single binary image

* default IMAGE_TAG

* cleanup

* try caching deps

* single arch add frontend build

* rustls::crypto::ring::default_provider()

* back to simple

* comment dockerfile

* add select options prop, render checkboxes if present

* add allowSelectedIf to enable / disable rows where necessary

* rename allowSelectIf to isSelectable, allow false as global disable, disable checkboxes when not allowed

* rename isSelectable to disableRow (it works the oppsite way lol)

* selected resources hook, start deployment batch execute component

* add deployment group actions

* add deployment group actions

* add default (empty) group actions for other resources

* fix checkbox header styles

* explicitly check if disableRow is passed (this prop is cursed)

* don't disable row selection for deployments table

* don't need id for groupactions

* add group actions to resources page

* fix row checkbox (prop not cursed, i dumb)

* re-implement group action list using dropdown menu

* only make group actions clickable when at least one row selected

* add loading indicator

* gap betwen new resource and group actions

* refactor group actions

* remove "Batch" from action labels

* add group actions for relevant resources

* fix hardcode

* add selectOptions to relevant tables

* select by name not id

* expect selected to be names

* add note re selection state init for future reference

* multi select working nicely for all resources

* configure server health check timeout

* config message

* refresh processes remove dead processes

* simplify the build args

* default timeout seconds 3

---------

Co-authored-by: kv <karamvir.singh98@gmail.com>
2024-12-01 23:34:07 -08:00
mbecker20
a8050db5f6 1.16.11 bump version 2024-11-14 02:18:26 -05:00
Maxwell Becker
bf0a972ec2 1.16.11 (#187)
* fix discord stack auto updated link

* action only log completion correctly

* add containers to omni search

* periphery build use --push

* use --password-stdin to login

* docker login stdin
2024-11-13 23:17:35 -08:00
mbecker20
23c1a08c87 fix Action success log being triggered even when there is error. 2024-11-12 19:43:55 -05:00
mbecker20
2b6b8a21ec revert monaco scroll past last line 2024-11-08 03:47:35 -05:00
mbecker20
02974b9adb monaco enable scroll beyond last line 2024-11-08 03:44:54 -05:00
Maxwell Becker
64d13666a9 1.16.10 (#178)
* send alert on auto update

* scrolling / capturing monaco editors

* deployed services has correct image

* serde default services for backward compat

* improve auto update config
2024-11-07 23:59:52 -08:00
mbecker20
2b2f354a3c add ImageUpdateAvailable filter to alert page 2024-11-05 01:30:55 -05:00
Maxwell Becker
aea5441466 1.16.9 (#172)
* BatchDestroyDeployment

* periphery image pull api

* Add Pull apis

* Add PullStack / PullDeployment

* improve init deploy from container

* stacks + deployments update_available source

* Fix deploy / destroy stack service

* updates available indicator

* add poll for updates and auto update options

* use interval to handle waiting between resource refresh

* stack auto update deploy whole stack

* format

* clean up the docs

* update available alerts

* update alerting format

* fix most clippy
2024-11-04 20:28:31 -08:00
mbecker20
97ced3b2cb frontend allow Alerter configure StackStateChanged, include Stacks and Repos in whitelist 2024-11-02 21:00:38 -04:00
Maxwell Becker
1f79987c58 1.16.8 (#170)
* update configs

* bump to 1.16.8
2024-11-01 14:35:12 -07:00
Maxwell Becker
e859a919c5 1.16.8 (#169)
* use this to extract from path

* Fix references to __ALL__
2024-11-01 14:33:41 -07:00
mbecker20
2a1270dd74 webhook check will return better status codex 2024-11-01 15:57:36 -04:00
Maxwell Becker
f5a59b0333 1.16.7 (#167)
* 1.16.7

* increase builder max poll to allow User Data more time to setup periphery

* rework to KOMODO_OIDC_REDIRECT_HOST
2024-10-31 21:06:01 -07:00
mbecker20
cacea235f9 replace networks empty with network_mode, replace container: network mode 2024-10-30 02:58:27 -04:00
mbecker20
54ba31dca9 gen ts types 2024-10-30 02:18:57 -04:00
Maxwell Becker
17d7ecb419 1.16.6 (#163)
* remove instrument from validate_cancel_build

* use type safe AllResources map - Action not showing omnisearch

* Stack support replicated services

* server docker nested tables

* fix container networks which use network of another container

* bump version

* add 'address' to ServerListItemInfo

* secrets list on variables page wraps

* fix user data script

* update default template user data

* improve sidebar layout styling

* fix network names shown on containers

* improve stack service / container page

* deleted resource log records Toml backup for later reference

* align all the tables

* add Url Builder type
2024-10-29 23:17:10 -07:00
mbecker20
38f3448790 add Procedures and Actions page 2024-10-28 00:57:42 -04:00
Maxwell Becker
ec88a6fa5a 1.16.5 (#159)
* repo pull lock

* implement BatchRunAction

* other batch methods
2024-10-27 20:56:56 -07:00
mbecker20
3820cd0ca2 make Build Organization configurable with custom value 2024-10-27 14:42:06 -04:00
mbecker20
419aa87bbb update resource toml examples to latest standard 2024-10-26 16:22:12 -04:00
Maxwell Becker
7a9ad42203 1.16.4 (#151)
* rust client improvements and docs

* sync rust client

* version 1.16.4

* UI support YAML / TOML utils, typed Deno namespace

* add ResourcesToml to typeshare

* add YAML and TOML convenience

* make the types available globally

* preload container with @std/yaml and @std/toml, clean up genned files

* add deno setup to alpine dockerfile
2024-10-26 12:15:34 -07:00
mbecker20
3f1cfa9064 update docs. Add Variables / Secrets docs 2024-10-25 00:02:12 -04:00
Maxwell Becker
d05c81864e 1.16.3 (#150)
* refactor listener api implementation for Gitlab integration

* version 1.16.3

* builder delete id link cleanup

* refactor and add "__ALL__" branch to avoid branch filtering

* frontend config the webhook url

* action webhook config

* clean up webhook url copy

* add __ALL__ branch switch for Actions / Procedures
2024-10-24 16:03:00 -07:00
mbecker20
f1a09f34ab tweak dev docs and runfile 2024-10-22 17:04:49 -04:00
mbecker20
23c6e6306d fix usage of runnables-cli in dev docs 2024-10-22 16:36:25 -04:00
mbecker20
800da90561 tweaks to dev docs 2024-10-22 15:25:33 -04:00
mbecker20
b24bf6ed89 fix docsite broken links 2024-10-22 15:17:10 -04:00
Matt Foxx
d66a781a13 docs: Add development docs (#136)
* docs: Add development POC

* docs: Flesh out full build/run steps

* feat: Add mergeable compose file to expose port and internal periphery url

* feat: Add .devcontainer and VSCode Tasks for developing Komodo

* Make cargo cache persistent in devcontainer

* Add deno to devcontainer

* Update tasks to include TS client copy to frontend before run

* Recommend extensions for used dependencies in vscode workspace

All extensions recommended are included in the devcontainer. This makes it easier for users not using devcontainer to get lang support.

* Update local `run` sequence for development docs
2024-10-22 12:09:26 -07:00
Maxwell Becker
f9b2994d44 1.16.2 (#145)
* Env vars written using same quotes (single vs double) as the user passes

* fmt

* trim start matches '-'

* ts client version
2024-10-22 11:41:17 -07:00
mbecker20
c0d6d96b64 get username works for service users 2024-10-22 03:36:20 -04:00
mbecker20
34496b948a bump ts client to 1.16.1 2024-10-22 02:58:42 -04:00
mbecker20
90c6adf923 fix periphery installer force file recreation command 2024-10-22 02:55:39 -04:00
mbecker20
3b72dc65cc remove "Overviews" label from sidebar 2024-10-22 02:27:22 -04:00
mbecker20
05f38d02be bump version to 1.16.1 2024-10-22 02:21:16 -04:00
Maxwell Becker
ea5506c202 1.16.1 (#143)
* ensure sync state cache is refreshed on sync create / copy

* clean up resources post_create

* show sidebar if element length > 1

* update `run_komodo_command` command

* rename all resources

* refresh repo cache after clone / pull

* improve rename repo log
2024-10-21 23:19:40 -07:00
mbecker20
64b0a5c9d2 delete unrelated caddy compose 2024-10-21 00:30:54 -04:00
mbecker20
93cc6a3a6e Add running Action to dashboard "Active" 2024-10-21 00:21:17 -04:00
mbecker20
7ae69cf33b ignore top level return linting 2024-10-20 05:10:28 -04:00
mbecker20
404e00cc64 move action info inline 2024-10-20 04:43:29 -04:00
mbecker20
6fe5bc7420 properly host client lib for deno importing (types working) 2024-10-20 03:17:58 -04:00
mbecker20
82324b00ee typescript komodo_client v 1.16.0 2024-10-20 02:35:43 -04:00
Maxwell Becker
5daba3a557 1.16.0 (#140)
* consolidate deserializers

* key value list doc

* use string list deserializers for all entity Vec<String>

* add additional env files support

* plumbing for Action resource

* js client readme indentation

* regen lock

* add action UI

* action backend

* start on action frontend

* update lock

* get up to speed

* get action started

* clean up default action file

* seems to work

* toml export include action

* action works

* action works part 2

* bump rust version to 1.82.0

* copy deno bin from bin image

* action use local dir

* update not having changes doesn't return error

* format with prettier

* support yaml formatting with prettier

* variable no change is Ok
2024-10-19 23:27:28 -07:00
mbecker20
020cdc06fd remove migrator link in readme 2024-10-18 21:23:02 -04:00
Maxwell Becker
cb270f4dff 1.15.12 (#139)
* add containers link to mobile dropdown

* fix update / alert not showing permission issue

* prevent disk alert back and forth

* improve user group pending toml
2024-10-18 17:14:22 -07:00
Matt Foxx
21666cf9b3 feat: Add docs link to topbar (#134) 2024-10-18 16:10:01 -07:00
mbecker20
a417926690 1.15.11 allow adding stack to user group 2024-10-16 23:09:06 -04:00
mbecker20
293b36fae4 Allow adding Stack to User Group 2024-10-16 23:08:44 -04:00
mbecker20
dca37e9ba8 1.15.10 connect with http using DOCKER_HOST 2024-10-16 22:16:07 -04:00
Morgan Wyatt
1cc302fcbf Update docker.rs to allow http docker socket connection (#131)
* Update docker.rs to allow http docker socket connection

Add or_else to allow attempt to connect to docker socket proxy via http if local connection fails

* Update docker.rs

Change two part connection to use connect_with_defaults instead, per review on PR.
2024-10-16 19:13:19 -07:00
mbecker20
febcf739d0 Remove Comma from installer: thanks @PiotrBzdrega 2024-10-16 10:43:54 -04:00
mbecker20
cb79e00794 update systemd service file 2024-10-15 17:35:54 -04:00
mbecker20
869b397596 force service file recreation docs 2024-10-15 17:25:29 -04:00
Maxwell Becker
41d1ff9760 1.15.9 (#127)
* add close alert threshold to prevent Ok - Warning back and forth

* remove part about repo being deleted, no longer behavior

* resource sync share general common

* remove this changelog. use releases

* remove changelog from readme

* write commit file clean up path

* docs: supports any git provider repo

* fix docs: authorization

* multiline command supports escaped newlines

* move webhook to build config advanced

* parser comments with escaped newline

* improve parser

* save use Enter. escape monaco using escape

* improve logic when deployment / stack action buttons shown

* used_mem = total - available

* Fix unrecognized path have 404

* webhooks will 404 if misconfigured

* move update logger / alerter

* delete migrator

* update examples

* publish typescript client komodo_client
2024-10-14 23:04:49 -07:00
mbecker20
dfafadf57b demo / build username pw 2024-10-14 11:49:44 -04:00
mbecker20
538a79b8b5 fix upausing all container action state 2024-10-13 18:11:09 -04:00
Maxwell Becker
5088dc5c3c 1.15.8 (#124)
* fix all containers restart and unpause

* add CommitSync to Procedure

* validate resource query tags causes failure on non exist

* files on host init working. match tags fail if tag doesnt exist

* intelligent sync match tag selector

* fix linting

* Wait for user initialize file on host
2024-10-13 15:03:16 -07:00
mbecker20
581d7e0b2c fix Procedure sync log 2024-10-13 04:21:03 -04:00
mbecker20
657298041f remove unneeded syncs volume 2024-10-13 04:03:09 -04:00
mbecker20
d71e9dca11 fix version 2024-10-13 03:21:56 -04:00
Maxwell Becker
165131bdf8 1.15.7 (#119)
* 1.15.7-dev ensure git config set

* add username to commit msg
2024-10-13 00:01:14 -07:00
mbecker20
0a81d2a0d0 add labels to mongo compose 2024-10-13 00:57:13 -04:00
Maxwell Becker
44ab5eb804 1.15.6 (#117)
* add periphery.skip label, skip in StopAllContainers

* add core config sync directory

* deploy stack if changed

* fix stack env_file_path when git repo and using run_directory

* deploy stack if changed

* write sync contents

* commit to git based sync, managed git based sync

* can sync non UI defined resource syncs

* sync UI control

* clippy

* init new stack compose file in repo

* better error message when attached Server / Builder invalid

* specify multiple resource file paths (mixed files + folders)

* use react charts

* tweak stats charts

* add Containers page

* 1.15.6

* stack deploy check if deployes vs remote has changed

* improve ux with loading indicators

* sync diff accounts for deploy / after

* fix new chart time axes
2024-10-12 21:42:46 -07:00
Maxwell Becker
e3d8e603ec 1.15.5 (#116)
* 1.15.5
- Update your user's username and password
- **Admin**: Delete Users

* update username / password / delete user backend

* bump version

* alerter default disabled

* delete users and update username / password

* set password "" after update
2024-10-11 19:42:43 -07:00
mbecker20
8b5c179473 account recover note 2024-10-11 19:16:01 -04:00
mbecker20
8582bc92da fix Destroy Before Deploy config 2024-10-10 04:17:17 -04:00
Maxwell Becker
8ee270d045 1.15.4 (#114)
* stack destroy before deploy option

* add timestamps. Fix log polling even when poll not selected

* Add build [[$VERSION]] support. VERSION build arg default

* fix clippy lint

* initialize `first_builder`

* run_komodo_command uses parse_multiline_command

* comment UI for $VERSION and new command feature

* bump some deps

* support multiline commands in pre_deploy / pre_build
2024-10-10 00:37:23 -07:00
Maxwell Becker
2cfae525e9 1.15.3 (#109)
* fix parser support single quote '

* add stack reclone toggle

* git clone with token uses token:<TOKEN> for gitlab compatability

* support stack pre deploy shell command

* rename compose down update log stage

* deployment configure registry login account

* local testing setup

* bump version to 1.15.3

* new resources auto assign server if only one

* better error log when try to create resource with duplicate name

* end description with .

* ConfirmUpdate multi language

* fix compose write to host logic

* improve instrumentation

* improve update diff when small array

improve 2

* fix compose env file passing when repo_dir is not absolute
2024-10-08 23:07:38 -07:00
mbecker20
80e5d2a972 frontend dev setup guide 2024-10-08 16:55:24 -04:00
mbecker20
6f22c011a6 builder / server template add correct additional line if empty params 2024-10-07 22:55:48 -04:00
mbecker20
401cccee79 config nav buttons secondary 2024-10-07 21:55:14 -04:00
mbecker20
654b923f98 fix broken link to periphery setup 2024-10-07 18:56:14 -04:00
mbecker20
61261be70f update docs, split connecting servers out of Core Setup 2024-10-07 18:54:00 -04:00
mbecker20
46418125e3 update docs for periphery systemd --user install 2024-10-07 18:53:43 -04:00
mbecker20
e029e94f0d 1.15.2 Pass KOMODO_OIDC_ADDITIONAL_AUDIENCES 2024-10-07 15:44:51 -04:00
mbecker20
3be2b5163b 1.15.1 do not add trailing slash OIDC provider 2024-10-07 13:23:40 -04:00
mbecker20
6a145f58ff pass provider as-is. Authentik users should add a trailing slash 2024-10-07 13:16:25 -04:00
mbecker20
f1cede2ebd update dark / light stack screenshot to have action buttons 2024-10-07 08:05:39 -04:00
mbecker20
a5cfa1d412 update screenshots 2024-10-07 07:30:18 -04:00
mbecker20
a0674654c1 update screenshots 2024-10-07 07:30:11 -04:00
mbecker20
3faa1c58c1 update screenshots 2024-10-07 07:30:05 -04:00
mbecker20
7e296f34af screenshots 2024-10-07 07:29:58 -04:00
mbecker20
9f8ced190c update screenshots 2024-10-07 07:29:02 -04:00
mbecker20
c194bb16d8 update screenshots 2024-10-07 07:28:45 -04:00
mbecker20
39fec9b55e update screenshots 2024-10-07 07:27:52 -04:00
mbecker20
e97ed9888d update screenshots 1 2024-10-07 07:27:16 -04:00
mbecker20
559102ffe3 update readme 2024-10-07 07:25:36 -04:00
mbecker20
6bf80ddcc7 update screenshots readme 2024-10-07 07:25:24 -04:00
mbecker20
89dbe1b4d9 stack file_contents editor respects readOnly / disabled 2024-10-07 06:58:00 -04:00
mbecker20
334e16d646 OIDC use preferred username 2024-10-07 06:35:46 -04:00
mbecker20
a7bbe519f4 add build server link 2024-10-07 06:15:53 -04:00
mbecker20
5827486c5a add redirect uri for OIDC 2024-10-07 06:15:00 -04:00
mbecker20
8ca8f7eddd add context to oidc init error 2024-10-07 06:10:12 -04:00
mbecker20
0600276b43 fix parse KOMODO_MONGO_ in envs 2024-10-07 05:43:09 -04:00
mbecker20
a77a1495c7 active resources mb-12 not always there 2024-10-07 05:14:54 -04:00
mbecker20
021ed5d15f ActiveResources margin bottom 2024-10-07 03:24:57 -04:00
Maxwell Becker
7d4376f426 1.15.0 (#90)
* attach env_file to compose build and compose pull stages

* fmt and bump rust version

* bump dependencies

* ignored for Sqlite message

* fix Build secret args info

* improve secret arguments info

* improve environment, ports, volumes deserializers

* rename `mongo` to `database` in config

* support _FILE in secret env vars

* improve setup - simpler compose

* remove aws ecr container registry support, alpine dockerfiles

* log periphery config

* ssl_enabled mode

* log http vs https

* periphery client accept untrust ssl certs

* fix nav issue from links

* configurable ssl

* KOMODO_ENSURE_SERVER -> KOMODO_FIRST_SERVER

* mount proc and ssl volume

* managed sync

* validate files on host resource path

* remove sync repo not configured guards

* disable confirm dialog

* fix sync hash / message Option

* try dev dockerfile

* refresh sync resources after commit

* socket invalidate handling

* delete dev dockerfile

* Commit Changes

* Add Info tab to syncs

* fix new Info parsing issue with serde default

* refresh stack cache on create / update

* managed syncs can't sync themselves

* managed syncs seems to work

* bump thiserror

* use alpine as main dockerfile

* apt add --no-cache

* disable user write perms, super admin perms to manage admins

* manage admin user UI

* implement disable non admin create frontend

* disable create non admin

* Copy button shown based on permission

* warning message on managed sync

* implement monaco editor

* impl simple match tags config

* resource sync support match tags

* more match tag filtering

* improve config with better saving diffs

* export button use monaco

* deser Conversions with wrapping strings

* envs editing

* don't delete variables / user groups if match tags defined

* env from_str improve

* improve dashboards

* remove core ca stuff for now

* move periphery ssl gen to dedicated file

* default server address periphery:8120

* clean up ssl configs

* server dashboard

* nice test compose

* add discord alerter

* discord alerter

* stack hideInfo logic

* compose setup

* alert table

* improve config hover card style

* update min editor height and stack config

* Feat: Styling Updates (#94)

* sidebar takes full screen height

* add bg accent to navbar

* add aschild prop to topbar alerts trigger

* stylize resource rows

* internally scrollable data tables

* better hover color for outlined button

* always show scrollbar to prevent layout shift

* better hover color for navbar

* rearrange buttons

* fix table and resource row styles

* cleanup scrollbar css

* use page for dashboard instead of section

* fix padding

* resource sync refactor and env keep comments

* frontend build

* improve configs

* config nice

* Feat/UI (#95)

* stylize resource rows

* internally scrollable data tables

* fix table and resource row styles

* use page for dashboard instead of section

* fix padding

* add `ResourcePageHeader` to required components

* add generic resource page header component

* add resource page headers for all components

* add resource notificaitons component

* add `TextUpdateMenu2` for use in resource page

* cleanup resource notificaitons

* update resource page layout

* ui edits

* sync kind of work

* clean up unused import

* syncs seem to work

* new sync pending

* monaco diff hide unchanged regions

* update styling all in config  resource select links

* confirm update default strings

* move procedure Add Stage to left

* update colors / styles

* frontend build

* backend for write file contents to host

* compose reference ports comment out

* server config

* ensure parent directory created

* fix frontend build

* remove default stack run_directory

* fix periphery compose deploy response set

* update compose files

* move server stats under tabs

* fix deployment list item getting correct image when not deployed

* stack updates cache after file write

* edit files on host

* clean up unused imports

* top level config update assignment must be spread

* update deps, move alert module

* move stack module

* move sync module

* move to sync db_client usage after init

* support generic OIDC provider

* init builders / server templates specifying https

* special cases for server / deployment state

* improve alert details

* add builder template `use_https` config

* try downgrade aws sdk ec2 for x86 build

* update debian dockerfiles to rm lists/*

* optionally configure seperate KOMODO_OIDC_REDIRECT

* add defaults to compose.env

* keep tags / search right aligned when view only

* clean up configs

* remove unused migrator deps

* update roadmap support generic OIDC

* initialize sync use confirm button

* key_value syntax highlighting

* smaller debian dockerfiles

* clean up deps.sh

* debian dockerifle

* New config layout (#96)

* new config layout

* fix image config layout and components config

* fix dom nesting and cleanup components

* fix label, make switches flex row

* ensure smooth scroll on hash navigations

* width 180 on config sidebar

* slight edits to config

* log whether https builder

* DISABLED <switch> ENABLED

* fix some more config

* smaller checked component

* server config looking good

* auto initialize compose files when files on host

* stack files on host good

* stack config nice

* remove old config

* deployments looking good

* build looking good

* Repo good

* nice config for builders

* alerter good

* server template config

* syncs good

* tweak stack config

* use status badge for update tables

* unified update page using router params

* replace /updates with unified updates page

* redirect all resource updates to unified update page

* fix reset handling

* unmount legacy page

* try periphery rustls

* rm unused import

* fix broken deps

* add unified alerts apge

* mount new alerts, remove old alerts page

* reroute resource alerts to unified alerts page

* back to periphery openssl

* ssl_enabled defaults to false for backward compat

* reqwest need json feature

* back to og yaml monaco

* Uncomment config fields for clearer config

* clean up compose env

* implement pull or clone, avoid deleting repo directory

* refactor mongo configuration params

* all configs respect empty string null

* add back status to header

* build toml don't have version if not auto incrementing

* fix comile

* fix repo pull cd to correct dir

* fix core pull_or_clone directory

* improve statuses

* remove ' ' from kv list parser

* longer CSRF valid for, to give time to login / accept

* don't compute diff / execute if there are any file_errors

* PartialBuilderConfig enum user inner option

* move errors to top

* fix toml init serializer

* server template and bulder manually add config.params line

* better way to check builder / template params empty

* improve build configs

* merge links into network area deployment

* default periphery config

* improve SystemCommand editor

* better Repo server / builder Info

* improve Alerts / Updates with ResourceSelector

* fix unused frontend

* update ResourceSync description

* toml use [resource.config] syntax

* update toml syntax

* update Build.image_registry schema

* fix repo / stack resource link alias

* reorder image registry

* align toml / yaml parser style

* some config updates

---------

Co-authored-by: Karamvir Singh <67458484+karamvirsingh98@users.noreply.github.com>
Co-authored-by: kv <karamvir.singh98@gmail.com>
2024-10-06 23:54:23 -07:00
Brad Lugo
7e9b406a34 fix: change edit url for docsite (#91)
Changes the default edit url provided by the docusaurus template to the
correct komodo url.
2024-09-22 23:08:34 -07:00
Brad Lugo
dcf78b05b3 fix: check if systemd is available for install (#89)
Adds a check at the beginning of setup-periphery.py to verify if
`systemctl` is executable by the current user and if systemd is the
init system in use. These changes will inform the user we've decided
systemd is unavailable and exits before we attempt to use any systemd
functionality, but modify this logic to configure other init systems in
the future.

Relates to (but doesn't close)
https://github.com/mbecker20/komodo/issues/66.
2024-09-22 19:34:56 -07:00
Jérémy
3236302d05 Reduce periphery image size (#82)
* Reduce periphery image size

* rename new alpine Dockerfile as slim.Dockerfile

* bump slim dockerfile rust version

---------

Co-authored-by: mbecker20 <becker.maxh@gmail.com>
2024-09-21 14:57:51 -07:00
mbecker20
fc41258d6c fix docs deploy command --env-file 2024-09-13 12:12:01 +03:00
mbecker20
ae8df90361 uncomment compose envs set by variables for .env edit only 2024-09-12 11:11:28 +03:00
mbecker20
7d05b2677f explicit yaml 2024-09-12 00:52:35 +03:00
623 changed files with 86496 additions and 30331 deletions

View File

@@ -0,0 +1,33 @@
services:
dev:
image: mcr.microsoft.com/devcontainers/rust:1-1-bullseye
volumes:
# Mount the root folder that contains .git
- ../:/workspace:cached
- /var/run/docker.sock:/var/run/docker.sock
- /proc:/proc
- repos:/etc/komodo/repos
- stacks:/etc/komodo/stacks
command: sleep infinity
ports:
- "9121:9121"
environment:
KOMODO_FIRST_SERVER: http://localhost:8120
KOMODO_DATABASE_ADDRESS: db
KOMODO_ENABLE_NEW_USERS: true
KOMODO_LOCAL_AUTH: true
KOMODO_JWT_SECRET: a_random_secret
links:
- db
# ...
db:
extends:
file: ../dev.compose.yaml
service: ferretdb
volumes:
data:
repo-cache:
repos:
stacks:

View File

@@ -0,0 +1,46 @@
// For format details, see https://aka.ms/devcontainer.json. For config options, see the
// README at: https://github.com/devcontainers/templates/tree/main/src/rust
{
"name": "Komodo",
// Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
//"image": "mcr.microsoft.com/devcontainers/rust:1-1-bullseye",
"dockerComposeFile": ["dev.compose.yaml"],
"workspaceFolder": "/workspace",
"service": "dev",
// Features to add to the dev container. More info: https://containers.dev/features.
"features": {
"ghcr.io/devcontainers/features/node:1": {
"version": "20.12.2"
},
"ghcr.io/devcontainers-community/features/deno:1": {
}
},
// Use 'mounts' to make the cargo cache persistent in a Docker Volume.
"mounts": [
{
"source": "devcontainer-cargo-cache-${devcontainerId}",
"target": "/usr/local/cargo",
"type": "volume"
}
],
// Use 'forwardPorts' to make a list of ports inside the container available locally.
"forwardPorts": [
9121
],
// Use 'postCreateCommand' to run commands after the container is created.
"postCreateCommand": "./.devcontainer/postCreate.sh",
"runServices": [
"db"
]
// Configure tool-specific properties.
// "customizations": {},
// Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root.
// "remoteUser": "root"
}

3
.devcontainer/postCreate.sh Executable file
View File

@@ -0,0 +1,3 @@
#!/bin/sh
cargo install typeshare-cli

View File

@@ -1,8 +0,0 @@
/target
readme.md
typeshare.toml
LICENSE
*.code-workspace
*/node_modules
*/dist

2
.github/FUNDING.yml vendored Normal file
View File

@@ -0,0 +1,2 @@
# https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/displaying-a-sponsor-button-in-your-repository
open_collective: komodo

11
.gitignore vendored
View File

@@ -1,10 +1,13 @@
target
/frontend/build
node_modules
/lib/ts_client/build
dist
.env
.env.development
.DS_Store
.idea
/frontend/build
/lib/ts_client/build
creds.toml
.syncs
.stacks
.dev

8
.vscode/extensions.json vendored Normal file
View File

@@ -0,0 +1,8 @@
{
"recommendations": [
"rust-lang.rust-analyzer",
"tamasfe.even-better-toml",
"vadimcn.vscode-lldb",
"denoland.vscode-deno"
]
}

179
.vscode/tasks.json vendored Normal file
View File

@@ -0,0 +1,179 @@
{
"version": "2.0.0",
"tasks": [
{
"label": "Run Core",
"command": "cargo",
"args": [
"run",
"-p",
"komodo_core",
"--release"
],
"options": {
"cwd": "${workspaceFolder}",
"env": {
"KOMODO_CONFIG_PATH": "test.core.config.toml"
}
},
"problemMatcher": [
"$rustc"
]
},
{
"label": "Build Core",
"command": "cargo",
"args": [
"build",
"-p",
"komodo_core",
"--release"
],
"options": {
"cwd": "${workspaceFolder}",
"env": {
"KOMODO_CONFIG_PATH": "test.core.config.toml"
}
},
"problemMatcher": [
"$rustc"
]
},
{
"label": "Run Periphery",
"command": "cargo",
"args": [
"run",
"-p",
"komodo_periphery",
"--release"
],
"options": {
"cwd": "${workspaceFolder}",
"env": {
"KOMODO_CONFIG_PATH": "test.periphery.config.toml"
}
},
"problemMatcher": [
"$rustc"
]
},
{
"label": "Build Periphery",
"command": "cargo",
"args": [
"build",
"-p",
"komodo_periphery",
"--release"
],
"options": {
"cwd": "${workspaceFolder}",
"env": {
"KOMODO_CONFIG_PATH": "test.periphery.config.toml"
}
},
"problemMatcher": [
"$rustc"
]
},
{
"label": "Run Backend",
"dependsOn": [
"Run Core",
"Run Periphery"
],
"problemMatcher": [
"$rustc"
]
},
{
"label": "Build TS Client Types",
"type": "process",
"command": "node",
"args": [
"./client/core/ts/generate_types.mjs"
],
"problemMatcher": []
},
{
"label": "Init TS Client",
"type": "shell",
"command": "yarn && yarn build && yarn link",
"options": {
"cwd": "${workspaceFolder}/client/core/ts",
},
"problemMatcher": []
},
{
"label": "Init Frontend Client",
"type": "shell",
"command": "yarn link komodo_client && yarn install",
"options": {
"cwd": "${workspaceFolder}/frontend",
},
"problemMatcher": []
},
{
"label": "Init Frontend",
"dependsOn": [
"Build TS Client Types",
"Init TS Client",
"Init Frontend Client"
],
"dependsOrder": "sequence",
"problemMatcher": []
},
{
"label": "Build Frontend",
"type": "shell",
"command": "yarn build",
"options": {
"cwd": "${workspaceFolder}/frontend",
},
"problemMatcher": []
},
{
"label": "Prepare Frontend For Run",
"type": "shell",
"command": "cp -r ./client/core/ts/dist/. frontend/public/client/.",
"options": {
"cwd": "${workspaceFolder}",
},
"dependsOn": [
"Build TS Client Types",
"Build Frontend"
],
"dependsOrder": "sequence",
"problemMatcher": []
},
{
"label": "Run Frontend",
"type": "shell",
"command": "yarn dev",
"options": {
"cwd": "${workspaceFolder}/frontend",
},
"dependsOn": ["Prepare Frontend For Run"],
"problemMatcher": []
},
{
"label": "Init",
"dependsOn": [
"Build Backend",
"Init Frontend"
],
"dependsOrder": "sequence",
"problemMatcher": []
},
{
"label": "Run Komodo",
"dependsOn": [
"Run Core",
"Run Periphery",
"Run Frontend"
],
"problemMatcher": []
},
]
}

3577
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,32 +1,36 @@
[workspace]
resolver = "2"
members = ["bin/*", "lib/*", "client/core/rs", "client/periphery/rs"]
members = [
"bin/*",
"lib/*",
"client/core/rs",
"client/periphery/rs",
]
[workspace.package]
version = "1.14.2"
edition = "2021"
version = "1.17.4"
edition = "2024"
authors = ["mbecker20 <becker.maxh@gmail.com>"]
license = "GPL-3.0-or-later"
repository = "https://github.com/mbecker20/komodo"
repository = "https://github.com/moghtech/komodo"
homepage = "https://komo.do"
[patch.crates-io]
# komodo_client = { path = "client/core/rs" }
[workspace.dependencies]
# LOCAL
# komodo_client = "1.14.2"
komodo_client = { path = "client/core/rs" }
periphery_client = { path = "client/periphery/rs" }
environment_file = { path = "lib/environment_file" }
formatting = { path = "lib/formatting" }
response = { path = "lib/response" }
command = { path = "lib/command" }
logger = { path = "lib/logger" }
cache = { path = "lib/cache" }
git = { path = "lib/git" }
# MOGH
run_command = { version = "0.0.6", features = ["async_tokio"] }
serror = { version = "0.4.6", default-features = false }
slack = { version = "0.1.0", package = "slack_client_rs" }
serror = { version = "0.5.0", default-features = false }
slack = { version = "0.4.0", package = "slack_client_rs", default-features = false, features = ["rustls"] }
derive_default_builder = "0.1.8"
derive_empty_traits = "0.1.0"
merge_config_files = "0.1.5"
@@ -34,76 +38,90 @@ async_timing_util = "1.0.0"
partial_derive2 = "0.4.3"
derive_variants = "1.0.0"
mongo_indexed = "2.0.1"
resolver_api = "1.1.1"
resolver_api = "3.0.0"
toml_pretty = "1.1.2"
mungos = "1.1.0"
mungos = "3.2.0"
svi = "1.0.1"
# ASYNC
tokio = { version = "1.40.0", features = ["full"] }
reqwest = { version = "0.12.7", features = ["json"] }
tokio-util = "0.7.11"
futures = "0.3.30"
futures-util = "0.3.30"
reqwest = { version = "0.12.15", default-features = false, features = ["json", "stream", "rustls-tls-native-roots"] }
tokio = { version = "1.44.1", features = ["full"] }
tokio-util = { version = "0.7.14", features = ["io", "codec"] }
futures = "0.3.31"
futures-util = "0.3.31"
arc-swap = "1.7.1"
# SERVER
axum = { version = "0.7.5", features = ["ws", "json"] }
axum-extra = { version = "0.9.3", features = ["typed-header"] }
tower-http = { version = "0.5.2", features = ["fs", "cors"] }
tokio-tungstenite = "0.23.1"
tokio-tungstenite = { version = "0.26.2", features = ["rustls-tls-native-roots"] }
axum-extra = { version = "0.10.0", features = ["typed-header"] }
tower-http = { version = "0.6.2", features = ["fs", "cors"] }
axum-server = { version = "0.7.2", features = ["tls-rustls"] }
axum = { version = "0.8.1", features = ["ws", "json", "macros"] }
# SER/DE
ordered_hash_map = { version = "0.4.0", features = ["serde"] }
serde = { version = "1.0.209", features = ["derive"] }
strum = { version = "0.26.3", features = ["derive"] }
serde_json = "1.0.127"
serde = { version = "1.0.219", features = ["derive"] }
strum = { version = "0.27.1", features = ["derive"] }
serde_json = "1.0.140"
serde_yaml = "0.9.34"
toml = "0.8.19"
serde_qs = "0.14.0"
toml = "0.8.20"
# ERROR
anyhow = "1.0.86"
thiserror = "1.0.63"
anyhow = "1.0.98"
thiserror = "2.0.12"
# LOGGING
opentelemetry_sdk = { version = "0.24.1", features = ["rt-tokio"] }
tracing-subscriber = { version = "0.3.18", features = ["json"] }
opentelemetry-semantic-conventions = "0.16.0"
tracing-opentelemetry = "0.25.0"
opentelemetry-otlp = "0.17.0"
opentelemetry = "0.24.0"
tracing = "0.1.40"
opentelemetry-otlp = { version = "0.29.0", features = ["tls-roots", "reqwest-rustls"] }
opentelemetry_sdk = { version = "0.29.0", features = ["rt-tokio"] }
tracing-subscriber = { version = "0.3.19", features = ["json"] }
opentelemetry-semantic-conventions = "0.29.0"
tracing-opentelemetry = "0.30.0"
opentelemetry = "0.29.0"
tracing = "0.1.41"
# CONFIG
clap = { version = "4.5.16", features = ["derive"] }
clap = { version = "4.5.37", features = ["derive"] }
dotenvy = "0.15.7"
envy = "0.4.2"
# CRYPTO
uuid = { version = "1.10.0", features = ["v4", "fast-rng", "serde"] }
# CRYPTO / AUTH
uuid = { version = "1.16.0", features = ["v4", "fast-rng", "serde"] }
jsonwebtoken = { version = "9.3.1", default-features = false }
openidconnect = "4.0.0"
urlencoding = "2.1.3"
nom_pem = "4.0.0"
bcrypt = "0.15.1"
bcrypt = "0.17.0"
base64 = "0.22.1"
rustls = "0.23.26"
hmac = "0.12.1"
sha2 = "0.10.8"
rand = "0.8.5"
jwt = "0.16.0"
rand = "0.9.1"
hex = "0.4.3"
# SYSTEM
bollard = "0.17.1"
sysinfo = "0.31.4"
portable-pty = "0.9.0"
bollard = "0.18.1"
sysinfo = "0.34.2"
# CLOUD
aws-config = "1.5.5"
aws-sdk-ec2 = "1.70.0"
aws-sdk-ecr = "1.42.0"
aws-config = "1.6.1"
aws-sdk-ec2 = "1.121.1"
aws-credential-types = "1.2.2"
## CRON
english-to-cron = "0.1.4"
chrono-tz = "0.10.3"
chrono = "0.4.40"
croner = "2.1.0"
# MISC
derive_builder = "0.20.1"
typeshare = "1.0.3"
octorust = "0.7.0"
colored = "2.1.0"
regex = "1.10.6"
bson = "2.11.0"
derive_builder = "0.20.2"
typeshare = "1.0.4"
octorust = "0.10.0"
dashmap = "6.1.0"
wildcard = "0.3.0"
colored = "3.0.0"
regex = "1.11.1"
bytes = "1.10.1"
bson = "2.14.0"

27
bin/binaries.Dockerfile Normal file
View File

@@ -0,0 +1,27 @@
## Builds the Komodo Core and Periphery binaries
## for a specific architecture.
FROM rust:1.86.0-bullseye AS builder
WORKDIR /builder
COPY Cargo.toml Cargo.lock ./
COPY ./lib ./lib
COPY ./client/core/rs ./client/core/rs
COPY ./client/periphery ./client/periphery
COPY ./bin/core ./bin/core
COPY ./bin/periphery ./bin/periphery
# Compile bin
RUN \
cargo build -p komodo_core --release && \
cargo build -p komodo_periphery --release
# Copy just the binaries to scratch image
FROM scratch
COPY --from=builder /builder/target/release/core /core
COPY --from=builder /builder/target/release/periphery /periphery
LABEL org.opencontainers.image.source=https://github.com/moghtech/komodo
LABEL org.opencontainers.image.description="Komodo Binaries"
LABEL org.opencontainers.image.licenses=GPL-3.0

View File

@@ -16,6 +16,7 @@ path = "src/main.rs"
[dependencies]
# local
# komodo_client = "1.16.12"
komodo_client.workspace = true
# external
tracing-subscriber.workspace = true

View File

@@ -1,13 +1,21 @@
use std::time::Duration;
use colored::Colorize;
use komodo_client::api::execute::Execution;
use komodo_client::{
api::execute::{BatchExecutionResponse, Execution},
entities::update::Update,
};
use crate::{
helpers::wait_for_enter,
state::{cli_args, komodo_client},
};
pub enum ExecutionResult {
Single(Update),
Batch(BatchExecutionResponse),
}
pub async fn run(execution: Execution) -> anyhow::Result<()> {
if matches!(execution, Execution::None(_)) {
println!("Got 'none' execution. Doing nothing...");
@@ -21,18 +29,36 @@ pub async fn run(execution: Execution) -> anyhow::Result<()> {
Execution::None(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::RunAction(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::BatchRunAction(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::RunProcedure(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::BatchRunProcedure(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::RunBuild(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::BatchRunBuild(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::CancelBuild(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::Deploy(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::BatchDeploy(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::PullDeployment(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::StartDeployment(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
@@ -51,15 +77,27 @@ pub async fn run(execution: Execution) -> anyhow::Result<()> {
Execution::DestroyDeployment(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::BatchDestroyDeployment(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::CloneRepo(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::BatchCloneRepo(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::PullRepo(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::BatchPullRepo(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::BuildRepo(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::BatchBuildRepo(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::CancelRepoBuild(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
@@ -129,9 +167,24 @@ pub async fn run(execution: Execution) -> anyhow::Result<()> {
Execution::RunSync(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::CommitSync(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::DeployStack(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::BatchDeployStack(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::DeployStackIfChanged(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::BatchDeployStackIfChanged(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::PullStack(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::StartStack(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
@@ -150,6 +203,12 @@ pub async fn run(execution: Execution) -> anyhow::Result<()> {
Execution::DestroyStack(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::BatchDestroyStack(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::TestAlerter(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
Execution::Sleep(data) => {
println!("{}: {data:?}", "Data".dimmed())
}
@@ -162,135 +221,246 @@ pub async fn run(execution: Execution) -> anyhow::Result<()> {
info!("Running Execution...");
let res = match execution {
Execution::RunProcedure(request) => {
komodo_client().execute(request).await
}
Execution::RunBuild(request) => {
komodo_client().execute(request).await
}
Execution::CancelBuild(request) => {
komodo_client().execute(request).await
}
Execution::Deploy(request) => {
komodo_client().execute(request).await
}
Execution::StartDeployment(request) => {
komodo_client().execute(request).await
}
Execution::RestartDeployment(request) => {
komodo_client().execute(request).await
}
Execution::PauseDeployment(request) => {
komodo_client().execute(request).await
}
Execution::UnpauseDeployment(request) => {
komodo_client().execute(request).await
}
Execution::StopDeployment(request) => {
komodo_client().execute(request).await
}
Execution::DestroyDeployment(request) => {
komodo_client().execute(request).await
}
Execution::CloneRepo(request) => {
komodo_client().execute(request).await
}
Execution::PullRepo(request) => {
komodo_client().execute(request).await
}
Execution::BuildRepo(request) => {
komodo_client().execute(request).await
}
Execution::CancelRepoBuild(request) => {
komodo_client().execute(request).await
}
Execution::StartContainer(request) => {
komodo_client().execute(request).await
}
Execution::RestartContainer(request) => {
komodo_client().execute(request).await
}
Execution::PauseContainer(request) => {
komodo_client().execute(request).await
}
Execution::UnpauseContainer(request) => {
komodo_client().execute(request).await
}
Execution::StopContainer(request) => {
komodo_client().execute(request).await
}
Execution::DestroyContainer(request) => {
komodo_client().execute(request).await
}
Execution::StartAllContainers(request) => {
komodo_client().execute(request).await
}
Execution::RestartAllContainers(request) => {
komodo_client().execute(request).await
}
Execution::PauseAllContainers(request) => {
komodo_client().execute(request).await
}
Execution::UnpauseAllContainers(request) => {
komodo_client().execute(request).await
}
Execution::StopAllContainers(request) => {
komodo_client().execute(request).await
}
Execution::PruneContainers(request) => {
komodo_client().execute(request).await
}
Execution::DeleteNetwork(request) => {
komodo_client().execute(request).await
}
Execution::PruneNetworks(request) => {
komodo_client().execute(request).await
}
Execution::DeleteImage(request) => {
komodo_client().execute(request).await
}
Execution::PruneImages(request) => {
komodo_client().execute(request).await
}
Execution::DeleteVolume(request) => {
komodo_client().execute(request).await
}
Execution::PruneVolumes(request) => {
komodo_client().execute(request).await
}
Execution::PruneDockerBuilders(request) => {
komodo_client().execute(request).await
}
Execution::PruneBuildx(request) => {
komodo_client().execute(request).await
}
Execution::PruneSystem(request) => {
komodo_client().execute(request).await
}
Execution::RunSync(request) => {
komodo_client().execute(request).await
}
Execution::DeployStack(request) => {
komodo_client().execute(request).await
}
Execution::StartStack(request) => {
komodo_client().execute(request).await
}
Execution::RestartStack(request) => {
komodo_client().execute(request).await
}
Execution::PauseStack(request) => {
komodo_client().execute(request).await
}
Execution::UnpauseStack(request) => {
komodo_client().execute(request).await
}
Execution::StopStack(request) => {
komodo_client().execute(request).await
}
Execution::DestroyStack(request) => {
komodo_client().execute(request).await
}
Execution::RunAction(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::BatchRunAction(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Batch),
Execution::RunProcedure(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::BatchRunProcedure(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Batch),
Execution::RunBuild(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::BatchRunBuild(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Batch),
Execution::CancelBuild(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::Deploy(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::BatchDeploy(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Batch),
Execution::PullDeployment(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::StartDeployment(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::RestartDeployment(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::PauseDeployment(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::UnpauseDeployment(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::StopDeployment(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::DestroyDeployment(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::BatchDestroyDeployment(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Batch),
Execution::CloneRepo(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::BatchCloneRepo(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Batch),
Execution::PullRepo(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::BatchPullRepo(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Batch),
Execution::BuildRepo(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::BatchBuildRepo(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Batch),
Execution::CancelRepoBuild(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::StartContainer(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::RestartContainer(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::PauseContainer(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::UnpauseContainer(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::StopContainer(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::DestroyContainer(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::StartAllContainers(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::RestartAllContainers(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::PauseAllContainers(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::UnpauseAllContainers(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::StopAllContainers(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::PruneContainers(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::DeleteNetwork(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::PruneNetworks(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::DeleteImage(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::PruneImages(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::DeleteVolume(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::PruneVolumes(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::PruneDockerBuilders(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::PruneBuildx(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::PruneSystem(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::RunSync(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::CommitSync(request) => komodo_client()
.write(request)
.await
.map(ExecutionResult::Single),
Execution::DeployStack(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::BatchDeployStack(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Batch),
Execution::DeployStackIfChanged(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::BatchDeployStackIfChanged(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Batch),
Execution::PullStack(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::StartStack(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::RestartStack(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::PauseStack(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::UnpauseStack(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::StopStack(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::DestroyStack(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::BatchDestroyStack(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Batch),
Execution::TestAlerter(request) => komodo_client()
.execute(request)
.await
.map(ExecutionResult::Single),
Execution::Sleep(request) => {
let duration =
Duration::from_millis(request.duration_ms as u64);
@@ -302,7 +472,12 @@ pub async fn run(execution: Execution) -> anyhow::Result<()> {
};
match res {
Ok(update) => println!("\n{}: {update:#?}", "SUCCESS".green()),
Ok(ExecutionResult::Single(update)) => {
println!("\n{}: {update:#?}", "SUCCESS".green())
}
Ok(ExecutionResult::Batch(update)) => {
println!("\n{}: {update:#?}", "SUCCESS".green())
}
Err(e) => println!("{}\n\n{e:#?}", "ERROR".red()),
}

View File

@@ -40,7 +40,9 @@ pub fn komodo_client() -> &'static KomodoClient {
creds
}
};
futures::executor::block_on(KomodoClient::new(url, key, secret))
.expect("failed to initialize Komodo client")
futures::executor::block_on(
KomodoClient::new(url, key, secret).with_healthcheck(),
)
.expect("failed to initialize Komodo client")
})
}

View File

@@ -17,8 +17,12 @@ path = "src/main.rs"
# local
komodo_client = { workspace = true, features = ["mongo"] }
periphery_client.workspace = true
environment_file.workspace = true
formatting.workspace = true
response.workspace = true
command.workspace = true
logger.workspace = true
cache.workspace = true
git.workspace = true
# mogh
serror = { workspace = true, features = ["axum"] }
@@ -29,15 +33,19 @@ derive_variants.workspace = true
mongo_indexed.workspace = true
resolver_api.workspace = true
toml_pretty.workspace = true
run_command.workspace = true
mungos.workspace = true
slack.workspace = true
svi.workspace = true
# external
aws-credential-types.workspace = true
tokio-tungstenite.workspace = true
ordered_hash_map.workspace = true
english-to-cron.workspace = true
openidconnect.workspace = true
jsonwebtoken.workspace = true
axum-server.workspace = true
urlencoding.workspace = true
aws-sdk-ec2.workspace = true
aws-sdk-ecr.workspace = true
aws-config.workspace = true
tokio-util.workspace = true
axum-extra.workspace = true
@@ -45,15 +53,22 @@ tower-http.workspace = true
serde_json.workspace = true
serde_yaml.workspace = true
typeshare.workspace = true
chrono-tz.workspace = true
octorust.workspace = true
wildcard.workspace = true
arc-swap.workspace = true
dashmap.workspace = true
tracing.workspace = true
reqwest.workspace = true
futures.workspace = true
nom_pem.workspace = true
anyhow.workspace = true
dotenvy.workspace = true
anyhow.workspace = true
croner.workspace = true
chrono.workspace = true
bcrypt.workspace = true
base64.workspace = true
rustls.workspace = true
tokio.workspace = true
serde.workspace = true
regex.workspace = true
@@ -64,5 +79,4 @@ envy.workspace = true
rand.workspace = true
hmac.workspace = true
sha2.workspace = true
jwt.workspace = true
hex.workspace = true

View File

@@ -1,39 +0,0 @@
# Build Core
FROM rust:1.80.1-bookworm AS core-builder
WORKDIR /builder
COPY . .
RUN cargo build -p komodo_core --release
# Build Frontend
FROM node:20.12-alpine AS frontend-builder
WORKDIR /builder
COPY ./frontend ./frontend
COPY ./client/core/ts ./client
RUN cd client && yarn && yarn build && yarn link
RUN cd frontend && yarn link @komodo/client && yarn && yarn build
# Final Image
FROM debian:bookworm-slim
# Install Deps
RUN apt update && apt install -y git curl unzip ca-certificates && \
curl -SL https://github.com/docker/compose/releases/download/v2.29.1/docker-compose-linux-x86_64 -o /usr/local/bin/docker-compose && \
chmod +x /usr/local/bin/docker-compose && \
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" && \
unzip awscliv2.zip && \
./aws/install
# Copy
COPY ./config/core.config.toml /config/config.toml
COPY --from=core-builder /builder/target/release/core /
COPY --from=frontend-builder /builder/frontend/dist /frontend
# Hint at the port
EXPOSE 9000
# Label for Ghcr
LABEL org.opencontainers.image.source=https://github.com/mbecker20/komodo
LABEL org.opencontainers.image.description="Komodo Core"
LABEL org.opencontainers.image.licenses=GPL-3.0
CMD ["./core"]

54
bin/core/aio.Dockerfile Normal file
View File

@@ -0,0 +1,54 @@
## All in one, multi stage compile + runtime Docker build for your architecture.
# Build Core
FROM rust:1.86.0-bullseye AS core-builder
WORKDIR /builder
COPY Cargo.toml Cargo.lock ./
COPY ./lib ./lib
COPY ./client/core/rs ./client/core/rs
COPY ./client/periphery ./client/periphery
COPY ./bin/core ./bin/core
# Compile app
RUN cargo build -p komodo_core --release
# Build Frontend
FROM node:20.12-alpine AS frontend-builder
WORKDIR /builder
COPY ./frontend ./frontend
COPY ./client/core/ts ./client
RUN cd client && yarn && yarn build && yarn link
RUN cd frontend && yarn link komodo_client && yarn && yarn build
# Final Image
FROM debian:bullseye-slim
COPY ./bin/core/starship.toml /config/starship.toml
COPY ./bin/core/debian-deps.sh .
RUN sh ./debian-deps.sh && rm ./debian-deps.sh
# Setup an application directory
WORKDIR /app
# Copy
COPY ./config/core.config.toml /config/config.toml
COPY --from=frontend-builder /builder/frontend/dist /app/frontend
COPY --from=core-builder /builder/target/release/core /usr/local/bin/core
COPY --from=denoland/deno:bin /deno /usr/local/bin/deno
# Set $DENO_DIR and preload external Deno deps
ENV DENO_DIR=/action-cache/deno
RUN mkdir /action-cache && \
cd /action-cache && \
deno install jsr:@std/yaml jsr:@std/toml
# Hint at the port
EXPOSE 9120
# Label for Ghcr
LABEL org.opencontainers.image.source=https://github.com/moghtech/komodo
LABEL org.opencontainers.image.description="Komodo Core"
LABEL org.opencontainers.image.licenses=GPL-3.0
ENTRYPOINT [ "core" ]

14
bin/core/debian-deps.sh Normal file
View File

@@ -0,0 +1,14 @@
#!/bin/bash
## Core deps installer
apt-get update
apt-get install -y git curl ca-certificates
rm -rf /var/lib/apt/lists/*
# Starship prompt
curl -sS https://starship.rs/install.sh | sh -s -- --yes --bin-dir /usr/local/bin
echo 'export STARSHIP_CONFIG=/config/starship.toml' >> /root/.bashrc
echo 'eval "$(starship init bash)"' >> /root/.bashrc

View File

@@ -0,0 +1,49 @@
## Assumes the latest binaries for x86_64 and aarch64 are already built (by binaries.Dockerfile).
## Sets up the necessary runtime container dependencies for Komodo Core.
## Since theres no heavy build here, QEMU multi-arch builds are fine for this image.
ARG BINARIES_IMAGE=ghcr.io/moghtech/komodo-binaries:latest
ARG FRONTEND_IMAGE=ghcr.io/moghtech/komodo-frontend:latest
ARG X86_64_BINARIES=${BINARIES_IMAGE}-x86_64
ARG AARCH64_BINARIES=${BINARIES_IMAGE}-aarch64
# This is required to work with COPY --from
FROM ${X86_64_BINARIES} AS x86_64
FROM ${AARCH64_BINARIES} AS aarch64
FROM ${FRONTEND_IMAGE} AS frontend
# Final Image
FROM debian:bullseye-slim
COPY ./bin/core/starship.toml /config/starship.toml
COPY ./bin/core/debian-deps.sh .
RUN sh ./debian-deps.sh && rm ./debian-deps.sh
WORKDIR /app
# Copy both binaries initially, but only keep appropriate one for the TARGETPLATFORM.
COPY --from=x86_64 /core /app/arch/linux/amd64
COPY --from=aarch64 /core /app/arch/linux/arm64
ARG TARGETPLATFORM
RUN mv /app/arch/${TARGETPLATFORM} /usr/local/bin/core && rm -r /app/arch
# Copy default config / static frontend / deno binary
COPY ./config/core.config.toml /config/config.toml
COPY --from=frontend /frontend /app/frontend
COPY --from=denoland/deno:bin /deno /usr/local/bin/deno
# Set $DENO_DIR and preload external Deno deps
ENV DENO_DIR=/action-cache/deno
RUN mkdir /action-cache && \
cd /action-cache && \
deno install jsr:@std/yaml jsr:@std/toml
# Hint at the port
EXPOSE 9120
# Label for Ghcr
LABEL org.opencontainers.image.source=https://github.com/moghtech/komodo
LABEL org.opencontainers.image.description="Komodo Core"
LABEL org.opencontainers.image.licenses=GPL-3.0
CMD [ "core" ]

View File

@@ -0,0 +1,43 @@
## Assumes the latest binaries for the required arch are already built (by binaries.Dockerfile).
## Sets up the necessary runtime container dependencies for Komodo Core.
ARG BINARIES_IMAGE=ghcr.io/moghtech/komodo-binaries:latest
# This is required to work with COPY --from
FROM ${BINARIES_IMAGE} AS binaries
# Build Frontend
FROM node:20.12-alpine AS frontend-builder
WORKDIR /builder
COPY ./frontend ./frontend
COPY ./client/core/ts ./client
RUN cd client && yarn && yarn build && yarn link
RUN cd frontend && yarn link komodo_client && yarn && yarn build
FROM debian:bullseye-slim
COPY ./bin/core/starship.toml /config/starship.toml
COPY ./bin/core/debian-deps.sh .
RUN sh ./debian-deps.sh && rm ./debian-deps.sh
# Copy
COPY ./config/core.config.toml /config/config.toml
COPY --from=frontend-builder /builder/frontend/dist /app/frontend
COPY --from=binaries /core /usr/local/bin/core
COPY --from=denoland/deno:bin /deno /usr/local/bin/deno
# Set $DENO_DIR and preload external Deno deps
ENV DENO_DIR=/action-cache/deno
RUN mkdir /action-cache && \
cd /action-cache && \
deno install jsr:@std/yaml jsr:@std/toml
# Hint at the port
EXPOSE 9120
# Label for Ghcr
LABEL org.opencontainers.image.source=https://github.com/moghtech/komodo
LABEL org.opencontainers.image.description="Komodo Core"
LABEL org.opencontainers.image.licenses=GPL-3.0
CMD [ "core" ]

View File

@@ -0,0 +1,277 @@
use std::sync::OnceLock;
use serde::Serialize;
use super::*;
#[instrument(level = "debug")]
pub async fn send_alert(
url: &str,
alert: &Alert,
) -> anyhow::Result<()> {
let level = fmt_level(alert.level);
let content = match &alert.data {
AlertData::Test { id, name } => {
let link = resource_link(ResourceTargetVariant::Alerter, id);
format!(
"{level} | If you see this message, then Alerter **{name}** is **working**\n{link}"
)
}
AlertData::ServerUnreachable {
id,
name,
region,
err,
} => {
let region = fmt_region(region);
let link = resource_link(ResourceTargetVariant::Server, id);
match alert.level {
SeverityLevel::Ok => {
format!(
"{level} | **{name}**{region} is now **reachable**\n{link}"
)
}
SeverityLevel::Critical => {
let err = err
.as_ref()
.map(|e| format!("\n**error**: {e:#?}"))
.unwrap_or_default();
format!(
"{level} | **{name}**{region} is **unreachable** ❌\n{link}{err}"
)
}
_ => unreachable!(),
}
}
AlertData::ServerCpu {
id,
name,
region,
percentage,
} => {
let region = fmt_region(region);
let link = resource_link(ResourceTargetVariant::Server, id);
format!(
"{level} | **{name}**{region} cpu usage at **{percentage:.1}%**\n{link}"
)
}
AlertData::ServerMem {
id,
name,
region,
used_gb,
total_gb,
} => {
let region = fmt_region(region);
let link = resource_link(ResourceTargetVariant::Server, id);
let percentage = 100.0 * used_gb / total_gb;
format!(
"{level} | **{name}**{region} memory usage at **{percentage:.1}%** 💾\n\nUsing **{used_gb:.1} GiB** / **{total_gb:.1} GiB**\n{link}"
)
}
AlertData::ServerDisk {
id,
name,
region,
path,
used_gb,
total_gb,
} => {
let region = fmt_region(region);
let link = resource_link(ResourceTargetVariant::Server, id);
let percentage = 100.0 * used_gb / total_gb;
format!(
"{level} | **{name}**{region} disk usage at **{percentage:.1}%** 💿\nmount point: `{path:?}`\nusing **{used_gb:.1} GiB** / **{total_gb:.1} GiB**\n{link}"
)
}
AlertData::ContainerStateChange {
id,
name,
server_id: _server_id,
server_name,
from,
to,
} => {
let link = resource_link(ResourceTargetVariant::Deployment, id);
let to = fmt_docker_container_state(to);
format!(
"📦 Deployment **{name}** is now **{to}**\nserver: **{server_name}**\nprevious: **{from}**\n{link}"
)
}
AlertData::DeploymentImageUpdateAvailable {
id,
name,
server_id: _server_id,
server_name,
image,
} => {
let link = resource_link(ResourceTargetVariant::Deployment, id);
format!(
"⬆ Deployment **{name}** has an update available\nserver: **{server_name}**\nimage: **{image}**\n{link}"
)
}
AlertData::DeploymentAutoUpdated {
id,
name,
server_id: _server_id,
server_name,
image,
} => {
let link = resource_link(ResourceTargetVariant::Deployment, id);
format!(
"⬆ Deployment **{name}** was updated automatically ⏫\nserver: **{server_name}**\nimage: **{image}**\n{link}"
)
}
AlertData::StackStateChange {
id,
name,
server_id: _server_id,
server_name,
from,
to,
} => {
let link = resource_link(ResourceTargetVariant::Stack, id);
let to = fmt_stack_state(to);
format!(
"🥞 Stack **{name}** is now {to}\nserver: **{server_name}**\nprevious: **{from}**\n{link}"
)
}
AlertData::StackImageUpdateAvailable {
id,
name,
server_id: _server_id,
server_name,
service,
image,
} => {
let link = resource_link(ResourceTargetVariant::Stack, id);
format!(
"⬆ Stack **{name}** has an update available\nserver: **{server_name}**\nservice: **{service}**\nimage: **{image}**\n{link}"
)
}
AlertData::StackAutoUpdated {
id,
name,
server_id: _server_id,
server_name,
images,
} => {
let link = resource_link(ResourceTargetVariant::Stack, id);
let images_label =
if images.len() > 1 { "images" } else { "image" };
let images = images.join(", ");
format!(
"⬆ Stack **{name}** was updated automatically ⏫\nserver: **{server_name}**\n{images_label}: **{images}**\n{link}"
)
}
AlertData::AwsBuilderTerminationFailed {
instance_id,
message,
} => {
format!(
"{level} | Failed to terminated AWS builder instance\ninstance id: **{instance_id}**\n{message}"
)
}
AlertData::ResourceSyncPendingUpdates { id, name } => {
let link =
resource_link(ResourceTargetVariant::ResourceSync, id);
format!(
"{level} | Pending resource sync updates on **{name}**\n{link}"
)
}
AlertData::BuildFailed { id, name, version } => {
let link = resource_link(ResourceTargetVariant::Build, id);
format!(
"{level} | Build **{name}** failed\nversion: **v{version}**\n{link}"
)
}
AlertData::RepoBuildFailed { id, name } => {
let link = resource_link(ResourceTargetVariant::Repo, id);
format!("{level} | Repo build for **{name}** failed\n{link}")
}
AlertData::ProcedureFailed { id, name } => {
let link = resource_link(ResourceTargetVariant::Procedure, id);
format!("{level} | Procedure **{name}** failed\n{link}")
}
AlertData::ActionFailed { id, name } => {
let link = resource_link(ResourceTargetVariant::Action, id);
format!("{level} | Action **{name}** failed\n{link}")
}
AlertData::ScheduleRun {
resource_type,
id,
name,
} => {
let link = resource_link(*resource_type, id);
format!(
"{level} | **{name}** ({resource_type}) | Scheduled run started 🕝\n{link}"
)
}
AlertData::None {} => Default::default(),
};
if !content.is_empty() {
let vars_and_secrets = get_variables_and_secrets().await?;
let mut global_replacers = HashSet::new();
let mut secret_replacers = HashSet::new();
let mut url_interpolated = url.to_string();
// interpolate variables and secrets into the url
interpolate_variables_secrets_into_string(
&vars_and_secrets,
&mut url_interpolated,
&mut global_replacers,
&mut secret_replacers,
)?;
send_message(&url_interpolated, &content)
.await
.map_err(|e| {
let replacers =
secret_replacers.into_iter().collect::<Vec<_>>();
let sanitized_error =
svi::replace_in_string(&format!("{e:?}"), &replacers);
anyhow::Error::msg(format!(
"Error with slack request: {}",
sanitized_error
))
})?;
}
Ok(())
}
async fn send_message(
url: &str,
content: &str,
) -> anyhow::Result<()> {
let body = DiscordMessageBody { content };
let response = http_client()
.post(url)
.json(&body)
.send()
.await
.context("Failed to send message")?;
let status = response.status();
if status.is_success() {
Ok(())
} else {
let text = response.text().await.with_context(|| {
format!("Failed to send message to Discord | {status} | failed to get response text")
})?;
Err(anyhow::anyhow!(
"Failed to send message to Discord | {status} | {text}"
))
}
}
fn http_client() -> &'static reqwest::Client {
static CLIENT: OnceLock<reqwest::Client> = OnceLock::new();
CLIENT.get_or_init(reqwest::Client::new)
}
#[derive(Serialize)]
struct DiscordMessageBody<'a> {
content: &'a str,
}

272
bin/core/src/alert/mod.rs Normal file
View File

@@ -0,0 +1,272 @@
use ::slack::types::Block;
use anyhow::{Context, anyhow};
use derive_variants::ExtractVariant;
use futures::future::join_all;
use komodo_client::entities::{
ResourceTargetVariant,
alert::{Alert, AlertData, AlertDataVariant, SeverityLevel},
alerter::*,
deployment::DeploymentState,
stack::StackState,
};
use mungos::{find::find_collect, mongodb::bson::doc};
use std::collections::HashSet;
use tracing::Instrument;
use crate::helpers::interpolate::interpolate_variables_secrets_into_string;
use crate::helpers::query::get_variables_and_secrets;
use crate::{config::core_config, state::db_client};
mod discord;
mod ntfy;
mod pushover;
mod slack;
#[instrument(level = "debug")]
pub async fn send_alerts(alerts: &[Alert]) {
if alerts.is_empty() {
return;
}
let span =
info_span!("send_alerts", alerts = format!("{alerts:?}"));
async {
let Ok(alerters) = find_collect(
&db_client().alerters,
doc! { "config.enabled": true },
None,
)
.await
.inspect_err(|e| {
error!(
"ERROR sending alerts | failed to get alerters from db | {e:#}"
)
}) else {
return;
};
let handles =
alerts.iter().map(|alert| send_alert(&alerters, alert));
join_all(handles).await;
}
.instrument(span)
.await
}
#[instrument(level = "debug")]
async fn send_alert(alerters: &[Alerter], alert: &Alert) {
if alerters.is_empty() {
return;
}
let handles = alerters
.iter()
.map(|alerter| send_alert_to_alerter(alerter, alert));
join_all(handles)
.await
.into_iter()
.filter_map(|res| res.err())
.for_each(|e| error!("{e:#}"));
}
pub async fn send_alert_to_alerter(
alerter: &Alerter,
alert: &Alert,
) -> anyhow::Result<()> {
// Don't send if not enabled
if !alerter.config.enabled {
return Ok(());
}
let alert_type = alert.data.extract_variant();
// In the test case, we don't want the filters inside this
// block to stop the test from being sent to the alerting endpoint.
if alert_type != AlertDataVariant::Test {
// Don't send if alert type not configured on the alerter
if !alerter.config.alert_types.is_empty()
&& !alerter.config.alert_types.contains(&alert_type)
{
return Ok(());
}
// Don't send if resource is in the blacklist
if alerter.config.except_resources.contains(&alert.target) {
return Ok(());
}
// Don't send if whitelist configured and target is not included
if !alerter.config.resources.is_empty()
&& !alerter.config.resources.contains(&alert.target)
{
return Ok(());
}
}
match &alerter.config.endpoint {
AlerterEndpoint::Custom(CustomAlerterEndpoint { url }) => {
send_custom_alert(url, alert).await.with_context(|| {
format!(
"Failed to send alert to Custom Alerter {}",
alerter.name
)
})
}
AlerterEndpoint::Slack(SlackAlerterEndpoint { url }) => {
slack::send_alert(url, alert).await.with_context(|| {
format!(
"Failed to send alert to Slack Alerter {}",
alerter.name
)
})
}
AlerterEndpoint::Discord(DiscordAlerterEndpoint { url }) => {
discord::send_alert(url, alert).await.with_context(|| {
format!(
"Failed to send alert to Discord Alerter {}",
alerter.name
)
})
}
AlerterEndpoint::Ntfy(NtfyAlerterEndpoint { url }) => {
ntfy::send_alert(url, alert).await.with_context(|| {
format!(
"Failed to send alert to ntfy Alerter {}",
alerter.name
)
})
}
AlerterEndpoint::Pushover(PushoverAlerterEndpoint { url }) => {
pushover::send_alert(url, alert).await.with_context(|| {
format!(
"Failed to send alert to Pushover Alerter {}",
alerter.name
)
})
}
}
}
#[instrument(level = "debug")]
async fn send_custom_alert(
url: &str,
alert: &Alert,
) -> anyhow::Result<()> {
let vars_and_secrets = get_variables_and_secrets().await?;
let mut global_replacers = HashSet::new();
let mut secret_replacers = HashSet::new();
let mut url_interpolated = url.to_string();
// interpolate variables and secrets into the url
interpolate_variables_secrets_into_string(
&vars_and_secrets,
&mut url_interpolated,
&mut global_replacers,
&mut secret_replacers,
)?;
let res = reqwest::Client::new()
.post(url_interpolated)
.json(alert)
.send()
.await
.map_err(|e| {
let replacers =
secret_replacers.into_iter().collect::<Vec<_>>();
let sanitized_error =
svi::replace_in_string(&format!("{e:?}"), &replacers);
anyhow::Error::msg(format!(
"Error with request: {}",
sanitized_error
))
})
.context("failed at post request to alerter")?;
let status = res.status();
if !status.is_success() {
let text = res
.text()
.await
.context("failed to get response text on alerter response")?;
return Err(anyhow!(
"post to alerter failed | {status} | {text}"
));
}
Ok(())
}
fn fmt_region(region: &Option<String>) -> String {
match region {
Some(region) => format!(" ({region})"),
None => String::new(),
}
}
fn fmt_docker_container_state(state: &DeploymentState) -> String {
match state {
DeploymentState::Running => String::from("Running ▶️"),
DeploymentState::Exited => String::from("Exited 🛑"),
DeploymentState::Restarting => String::from("Restarting 🔄"),
DeploymentState::NotDeployed => String::from("Not Deployed"),
_ => state.to_string(),
}
}
fn fmt_stack_state(state: &StackState) -> String {
match state {
StackState::Running => String::from("Running ▶️"),
StackState::Stopped => String::from("Stopped 🛑"),
StackState::Restarting => String::from("Restarting 🔄"),
StackState::Down => String::from("Down ⬇️"),
_ => state.to_string(),
}
}
fn fmt_level(level: SeverityLevel) -> &'static str {
match level {
SeverityLevel::Critical => "CRITICAL 🚨",
SeverityLevel::Warning => "WARNING ‼️",
SeverityLevel::Ok => "OK ✅",
}
}
fn resource_link(
resource_type: ResourceTargetVariant,
id: &str,
) -> String {
let path = match resource_type {
ResourceTargetVariant::System => unreachable!(),
ResourceTargetVariant::Build => format!("/builds/{id}"),
ResourceTargetVariant::Builder => {
format!("/builders/{id}")
}
ResourceTargetVariant::Deployment => {
format!("/deployments/{id}")
}
ResourceTargetVariant::Stack => {
format!("/stacks/{id}")
}
ResourceTargetVariant::Server => {
format!("/servers/{id}")
}
ResourceTargetVariant::Repo => format!("/repos/{id}"),
ResourceTargetVariant::Alerter => {
format!("/alerters/{id}")
}
ResourceTargetVariant::Procedure => {
format!("/procedures/{id}")
}
ResourceTargetVariant::Action => {
format!("/actions/{id}")
}
ResourceTargetVariant::ServerTemplate => {
format!("/server-templates/{id}")
}
ResourceTargetVariant::ResourceSync => {
format!("/resource-syncs/{id}")
}
};
format!("{}{path}", core_config().host)
}

266
bin/core/src/alert/ntfy.rs Normal file
View File

@@ -0,0 +1,266 @@
use std::sync::OnceLock;
use super::*;
#[instrument(level = "debug")]
pub async fn send_alert(
url: &str,
alert: &Alert,
) -> anyhow::Result<()> {
let level = fmt_level(alert.level);
let content = match &alert.data {
AlertData::Test { id, name } => {
let link = resource_link(ResourceTargetVariant::Alerter, id);
format!(
"{level} | If you see this message, then Alerter {} is working\n{link}",
name,
)
}
AlertData::ServerUnreachable {
id,
name,
region,
err,
} => {
let region = fmt_region(region);
let link = resource_link(ResourceTargetVariant::Server, id);
match alert.level {
SeverityLevel::Ok => {
format!(
"{level} | {}{} is now reachable\n{link}",
name, region
)
}
SeverityLevel::Critical => {
let err = err
.as_ref()
.map(|e| format!("\nerror: {:#?}", e))
.unwrap_or_default();
format!(
"{level} | {}{} is unreachable ❌\n{link}{err}",
name, region
)
}
_ => unreachable!(),
}
}
AlertData::ServerCpu {
id,
name,
region,
percentage,
} => {
let region = fmt_region(region);
let link = resource_link(ResourceTargetVariant::Server, id);
format!(
"{level} | {}{} cpu usage at {percentage:.1}%\n{link}",
name, region,
)
}
AlertData::ServerMem {
id,
name,
region,
used_gb,
total_gb,
} => {
let region = fmt_region(region);
let link = resource_link(ResourceTargetVariant::Server, id);
let percentage = 100.0 * used_gb / total_gb;
format!(
"{level} | {}{} memory usage at {percentage:.1}%💾\n\nUsing {used_gb:.1} GiB / {total_gb:.1} GiB\n{link}",
name, region,
)
}
AlertData::ServerDisk {
id,
name,
region,
path,
used_gb,
total_gb,
} => {
let region = fmt_region(region);
let link = resource_link(ResourceTargetVariant::Server, id);
let percentage = 100.0 * used_gb / total_gb;
format!(
"{level} | {}{} disk usage at {percentage:.1}%💿\nmount point: {:?}\nusing {used_gb:.1} GiB / {total_gb:.1} GiB\n{link}",
name, region, path,
)
}
AlertData::ContainerStateChange {
id,
name,
server_id: _server_id,
server_name,
from,
to,
} => {
let link = resource_link(ResourceTargetVariant::Deployment, id);
let to_state = fmt_docker_container_state(to);
format!(
"📦Deployment {} is now {}\nserver: {}\nprevious: {}\n{link}",
name, to_state, server_name, from,
)
}
AlertData::DeploymentImageUpdateAvailable {
id,
name,
server_id: _server_id,
server_name,
image,
} => {
let link = resource_link(ResourceTargetVariant::Deployment, id);
format!(
"⬆ Deployment {} has an update available\nserver: {}\nimage: {}\n{link}",
name, server_name, image,
)
}
AlertData::DeploymentAutoUpdated {
id,
name,
server_id: _server_id,
server_name,
image,
} => {
let link = resource_link(ResourceTargetVariant::Deployment, id);
format!(
"⬆ Deployment {} was updated automatically\nserver: {}\nimage: {}\n{link}",
name, server_name, image,
)
}
AlertData::StackStateChange {
id,
name,
server_id: _server_id,
server_name,
from,
to,
} => {
let link = resource_link(ResourceTargetVariant::Stack, id);
let to_state = fmt_stack_state(to);
format!(
"🥞 Stack {} is now {}\nserver: {}\nprevious: {}\n{link}",
name, to_state, server_name, from,
)
}
AlertData::StackImageUpdateAvailable {
id,
name,
server_id: _server_id,
server_name,
service,
image,
} => {
let link = resource_link(ResourceTargetVariant::Stack, id);
format!(
"⬆ Stack {} has an update available\nserver: {}\nservice: {}\nimage: {}\n{link}",
name, server_name, service, image,
)
}
AlertData::StackAutoUpdated {
id,
name,
server_id: _server_id,
server_name,
images,
} => {
let link = resource_link(ResourceTargetVariant::Stack, id);
let images_label =
if images.len() > 1 { "images" } else { "image" };
let images_str = images.join(", ");
format!(
"⬆ Stack {} was updated automatically ⏫\nserver: {}\n{}: {}\n{link}",
name, server_name, images_label, images_str,
)
}
AlertData::AwsBuilderTerminationFailed {
instance_id,
message,
} => {
format!(
"{level} | Failed to terminate AWS builder instance\ninstance id: {}\n{}",
instance_id, message,
)
}
AlertData::ResourceSyncPendingUpdates { id, name } => {
let link =
resource_link(ResourceTargetVariant::ResourceSync, id);
format!(
"{level} | Pending resource sync updates on {}\n{link}",
name,
)
}
AlertData::BuildFailed { id, name, version } => {
let link = resource_link(ResourceTargetVariant::Build, id);
format!(
"{level} | Build {} failed\nversion: v{}\n{link}",
name, version,
)
}
AlertData::RepoBuildFailed { id, name } => {
let link = resource_link(ResourceTargetVariant::Repo, id);
format!("{level} | Repo build for {} failed\n{link}", name,)
}
AlertData::ProcedureFailed { id, name } => {
let link = resource_link(ResourceTargetVariant::Procedure, id);
format!("{level} | Procedure {name} failed\n{link}")
}
AlertData::ActionFailed { id, name } => {
let link = resource_link(ResourceTargetVariant::Action, id);
format!("{level} | Action {name} failed\n{link}")
}
AlertData::ScheduleRun {
resource_type,
id,
name,
} => {
let link = resource_link(*resource_type, id);
format!(
"{level} | {name} ({resource_type}) | Scheduled run started 🕝\n{link}"
)
}
AlertData::None {} => Default::default(),
};
if !content.is_empty() {
send_message(url, content).await?;
}
Ok(())
}
async fn send_message(
url: &str,
content: String,
) -> anyhow::Result<()> {
let response = http_client()
.post(url)
.header("Title", "ntfy Alert")
.body(content)
.send()
.await
.context("Failed to send message")?;
let status = response.status();
if status.is_success() {
debug!("ntfy alert sent successfully: {}", status);
Ok(())
} else {
let text = response.text().await.with_context(|| {
format!(
"Failed to send message to ntfy | {} | failed to get response text",
status
)
})?;
Err(anyhow!(
"Failed to send message to ntfy | {} | {}",
status,
text
))
}
}
fn http_client() -> &'static reqwest::Client {
static CLIENT: OnceLock<reqwest::Client> = OnceLock::new();
CLIENT.get_or_init(reqwest::Client::new)
}

View File

@@ -0,0 +1,270 @@
use std::sync::OnceLock;
use super::*;
#[instrument(level = "debug")]
pub async fn send_alert(
url: &str,
alert: &Alert,
) -> anyhow::Result<()> {
let level = fmt_level(alert.level);
let content = match &alert.data {
AlertData::Test { id, name } => {
let link = resource_link(ResourceTargetVariant::Alerter, id);
format!(
"{level} | If you see this message, then Alerter {} is working\n{link}",
name,
)
}
AlertData::ServerUnreachable {
id,
name,
region,
err,
} => {
let region = fmt_region(region);
let link = resource_link(ResourceTargetVariant::Server, id);
match alert.level {
SeverityLevel::Ok => {
format!(
"{level} | {}{} is now reachable\n{link}",
name, region
)
}
SeverityLevel::Critical => {
let err = err
.as_ref()
.map(|e| format!("\nerror: {:#?}", e))
.unwrap_or_default();
format!(
"{level} | {}{} is unreachable ❌\n{link}{err}",
name, region
)
}
_ => unreachable!(),
}
}
AlertData::ServerCpu {
id,
name,
region,
percentage,
} => {
let region = fmt_region(region);
let link = resource_link(ResourceTargetVariant::Server, id);
format!(
"{level} | {}{} cpu usage at {percentage:.1}%\n{link}",
name, region,
)
}
AlertData::ServerMem {
id,
name,
region,
used_gb,
total_gb,
} => {
let region = fmt_region(region);
let link = resource_link(ResourceTargetVariant::Server, id);
let percentage = 100.0 * used_gb / total_gb;
format!(
"{level} | {}{} memory usage at {percentage:.1}%💾\n\nUsing {used_gb:.1} GiB / {total_gb:.1} GiB\n{link}",
name, region,
)
}
AlertData::ServerDisk {
id,
name,
region,
path,
used_gb,
total_gb,
} => {
let region = fmt_region(region);
let link = resource_link(ResourceTargetVariant::Server, id);
let percentage = 100.0 * used_gb / total_gb;
format!(
"{level} | {}{} disk usage at {percentage:.1}%💿\nmount point: {:?}\nusing {used_gb:.1} GiB / {total_gb:.1} GiB\n{link}",
name, region, path,
)
}
AlertData::ContainerStateChange {
id,
name,
server_id: _server_id,
server_name,
from,
to,
} => {
let link = resource_link(ResourceTargetVariant::Deployment, id);
let to_state = fmt_docker_container_state(to);
format!(
"📦Deployment {} is now {}\nserver: {}\nprevious: {}\n{link}",
name, to_state, server_name, from,
)
}
AlertData::DeploymentImageUpdateAvailable {
id,
name,
server_id: _server_id,
server_name,
image,
} => {
let link = resource_link(ResourceTargetVariant::Deployment, id);
format!(
"⬆ Deployment {} has an update available\nserver: {}\nimage: {}\n{link}",
name, server_name, image,
)
}
AlertData::DeploymentAutoUpdated {
id,
name,
server_id: _server_id,
server_name,
image,
} => {
let link = resource_link(ResourceTargetVariant::Deployment, id);
format!(
"⬆ Deployment {} was updated automatically\nserver: {}\nimage: {}\n{link}",
name, server_name, image,
)
}
AlertData::StackStateChange {
id,
name,
server_id: _server_id,
server_name,
from,
to,
} => {
let link = resource_link(ResourceTargetVariant::Stack, id);
let to_state = fmt_stack_state(to);
format!(
"🥞 Stack {} is now {}\nserver: {}\nprevious: {}\n{link}",
name, to_state, server_name, from,
)
}
AlertData::StackImageUpdateAvailable {
id,
name,
server_id: _server_id,
server_name,
service,
image,
} => {
let link = resource_link(ResourceTargetVariant::Stack, id);
format!(
"⬆ Stack {} has an update available\nserver: {}\nservice: {}\nimage: {}\n{link}",
name, server_name, service, image,
)
}
AlertData::StackAutoUpdated {
id,
name,
server_id: _server_id,
server_name,
images,
} => {
let link = resource_link(ResourceTargetVariant::Stack, id);
let images_label =
if images.len() > 1 { "images" } else { "image" };
let images_str = images.join(", ");
format!(
"⬆ Stack {} was updated automatically ⏫\nserver: {}\n{}: {}\n{link}",
name, server_name, images_label, images_str,
)
}
AlertData::AwsBuilderTerminationFailed {
instance_id,
message,
} => {
format!(
"{level} | Failed to terminate AWS builder instance\ninstance id: {}\n{}",
instance_id, message,
)
}
AlertData::ResourceSyncPendingUpdates { id, name } => {
let link =
resource_link(ResourceTargetVariant::ResourceSync, id);
format!(
"{level} | Pending resource sync updates on {}\n{link}",
name,
)
}
AlertData::BuildFailed { id, name, version } => {
let link = resource_link(ResourceTargetVariant::Build, id);
format!(
"{level} | Build {name} failed\nversion: v{version}\n{link}",
)
}
AlertData::RepoBuildFailed { id, name } => {
let link = resource_link(ResourceTargetVariant::Repo, id);
format!("{level} | Repo build for {} failed\n{link}", name,)
}
AlertData::ProcedureFailed { id, name } => {
let link = resource_link(ResourceTargetVariant::Procedure, id);
format!("{level} | Procedure {name} failed\n{link}")
}
AlertData::ActionFailed { id, name } => {
let link = resource_link(ResourceTargetVariant::Action, id);
format!("{level} | Action {name} failed\n{link}")
}
AlertData::ScheduleRun {
resource_type,
id,
name,
} => {
let link = resource_link(*resource_type, id);
format!(
"{level} | {name} ({resource_type}) | Scheduled run started 🕝\n{link}"
)
}
AlertData::None {} => Default::default(),
};
if !content.is_empty() {
send_message(url, content).await?;
}
Ok(())
}
async fn send_message(
url: &str,
content: String,
) -> anyhow::Result<()> {
// pushover needs all information to be encoded in the URL. At minimum they need
// the user key, the application token, and the message (url encoded).
// other optional params here: https://pushover.net/api (just add them to the
// webhook url along with the application token and the user key).
let content = [("message", content)];
let response = http_client()
.post(url)
.form(&content)
.send()
.await
.context("Failed to send message")?;
let status = response.status();
if status.is_success() {
debug!("pushover alert sent successfully: {}", status);
Ok(())
} else {
let text = response.text().await.with_context(|| {
format!(
"Failed to send message to pushover | {} | failed to get response text",
status
)
})?;
Err(anyhow!(
"Failed to send message to pushover | {} | {}",
status,
text
))
}
}
fn http_client() -> &'static reqwest::Client {
static CLIENT: OnceLock<reqwest::Client> = OnceLock::new();
CLIENT.get_or_init(reqwest::Client::new)
}

View File

@@ -1,135 +1,28 @@
use anyhow::{anyhow, Context};
use derive_variants::ExtractVariant;
use futures::future::join_all;
use komodo_client::entities::{
alert::{Alert, AlertData, SeverityLevel},
alerter::*,
deployment::DeploymentState,
stack::StackState,
ResourceTargetVariant,
};
use mungos::{find::find_collect, mongodb::bson::doc};
use slack::types::Block;
use crate::{config::core_config, state::db_client};
#[instrument]
pub async fn send_alerts(alerts: &[Alert]) {
if alerts.is_empty() {
return;
}
let Ok(alerters) = find_collect(
&db_client().await.alerters,
doc! { "config.enabled": true },
None,
)
.await
.inspect_err(|e| {
error!(
"ERROR sending alerts | failed to get alerters from db | {e:#}"
)
}) else {
return;
};
let handles =
alerts.iter().map(|alert| send_alert(&alerters, alert));
join_all(handles).await;
}
use super::*;
#[instrument(level = "debug")]
async fn send_alert(alerters: &[Alerter], alert: &Alert) {
if alerters.is_empty() {
return;
}
let alert_type = alert.data.extract_variant();
let handles = alerters.iter().map(|alerter| async {
// Don't send if not enabled
if !alerter.config.enabled {
return Ok(());
}
// Don't send if alert type not configured on the alerter
if !alerter.config.alert_types.is_empty()
&& !alerter.config.alert_types.contains(&alert_type)
{
return Ok(());
}
// Don't send if resource is in the blacklist
if alerter.config.except_resources.contains(&alert.target) {
return Ok(());
}
// Don't send if whitelist configured and target is not included
if !alerter.config.resources.is_empty()
&& !alerter.config.resources.contains(&alert.target)
{
return Ok(());
}
match &alerter.config.endpoint {
AlerterEndpoint::Slack(SlackAlerterEndpoint { url }) => {
send_slack_alert(url, alert).await.with_context(|| {
format!(
"failed to send alert to slack alerter {}",
alerter.name
)
})
}
AlerterEndpoint::Custom(CustomAlerterEndpoint { url }) => {
send_custom_alert(url, alert).await.with_context(|| {
format!(
"failed to send alert to custom alerter {}",
alerter.name
)
})
}
}
});
join_all(handles)
.await
.into_iter()
.filter_map(|res| res.err())
.for_each(|e| error!("{e:#}"));
}
#[instrument(level = "debug")]
async fn send_custom_alert(
url: &str,
alert: &Alert,
) -> anyhow::Result<()> {
let res = reqwest::Client::new()
.post(url)
.json(alert)
.send()
.await
.context("failed at post request to alerter")?;
let status = res.status();
if !status.is_success() {
let text = res
.text()
.await
.context("failed to get response text on alerter response")?;
return Err(anyhow!(
"post to alerter failed | {status} | {text}"
));
}
Ok(())
}
#[instrument(level = "debug")]
async fn send_slack_alert(
pub async fn send_alert(
url: &str,
alert: &Alert,
) -> anyhow::Result<()> {
let level = fmt_level(alert.level);
let (text, blocks): (_, Option<_>) = match &alert.data {
AlertData::Test { id, name } => {
let text = format!(
"{level} | If you see this message, then Alerter *{name}* is *working*"
);
let blocks = vec![
Block::header(level),
Block::section(format!(
"If you see this message, then Alerter *{name}* is *working*"
)),
Block::section(resource_link(
ResourceTargetVariant::Alerter,
id,
)),
];
(text, blocks.into())
}
AlertData::ServerUnreachable {
id,
name,
@@ -180,7 +73,9 @@ async fn send_slack_alert(
let region = fmt_region(region);
match alert.level {
SeverityLevel::Ok => {
let text = format!("{level} | *{name}*{region} cpu usage at *{percentage:.1}%*");
let text = format!(
"{level} | *{name}*{region} cpu usage at *{percentage:.1}%*"
);
let blocks = vec![
Block::header(level),
Block::section(format!(
@@ -194,7 +89,9 @@ async fn send_slack_alert(
(text, blocks.into())
}
_ => {
let text = format!("{level} | *{name}*{region} cpu usage at *{percentage:.1}%* 📈");
let text = format!(
"{level} | *{name}*{region} cpu usage at *{percentage:.1}%* 📈"
);
let blocks = vec![
Block::header(level),
Block::section(format!(
@@ -220,7 +117,9 @@ async fn send_slack_alert(
let percentage = 100.0 * used_gb / total_gb;
match alert.level {
SeverityLevel::Ok => {
let text = format!("{level} | *{name}*{region} memory usage at *{percentage:.1}%* 💾");
let text = format!(
"{level} | *{name}*{region} memory usage at *{percentage:.1}%* 💾"
);
let blocks = vec![
Block::header(level),
Block::section(format!(
@@ -237,7 +136,9 @@ async fn send_slack_alert(
(text, blocks.into())
}
_ => {
let text = format!("{level} | *{name}*{region} memory usage at *{percentage:.1}%* 💾");
let text = format!(
"{level} | *{name}*{region} memory usage at *{percentage:.1}%* 💾"
);
let blocks = vec![
Block::header(level),
Block::section(format!(
@@ -267,7 +168,9 @@ async fn send_slack_alert(
let percentage = 100.0 * used_gb / total_gb;
match alert.level {
SeverityLevel::Ok => {
let text = format!("{level} | *{name}*{region} disk usage at *{percentage:.1}%* | mount point: *{path:?}* 💿");
let text = format!(
"{level} | *{name}*{region} disk usage at *{percentage:.1}%* | mount point: *{path:?}* 💿"
);
let blocks = vec![
Block::header(level),
Block::section(format!(
@@ -276,12 +179,17 @@ async fn send_slack_alert(
Block::section(format!(
"mount point: {path:?} | using *{used_gb:.1} GiB* / *{total_gb:.1} GiB*"
)),
Block::section(resource_link(ResourceTargetVariant::Server, id)),
Block::section(resource_link(
ResourceTargetVariant::Server,
id,
)),
];
(text, blocks.into())
}
_ => {
let text = format!("{level} | *{name}*{region} disk usage at *{percentage:.1}%* | mount point: *{path:?}* 💿");
let text = format!(
"{level} | *{name}*{region} disk usage at *{percentage:.1}%* | mount point: *{path:?}* 💿"
);
let blocks = vec![
Block::header(level),
Block::section(format!(
@@ -290,7 +198,10 @@ async fn send_slack_alert(
Block::section(format!(
"mount point: {path:?} | using *{used_gb:.1} GiB* / *{total_gb:.1} GiB*"
)),
Block::section(resource_link(ResourceTargetVariant::Server, id)),
Block::section(resource_link(
ResourceTargetVariant::Server,
id,
)),
];
(text, blocks.into())
}
@@ -305,7 +216,7 @@ async fn send_slack_alert(
..
} => {
let to = fmt_docker_container_state(to);
let text = format!("📦 Container *{name}* is now {to}");
let text = format!("📦 Container *{name}* is now *{to}*");
let blocks = vec![
Block::header(text.clone()),
Block::section(format!(
@@ -318,6 +229,48 @@ async fn send_slack_alert(
];
(text, blocks.into())
}
AlertData::DeploymentImageUpdateAvailable {
id,
name,
server_name,
server_id: _server_id,
image,
} => {
let text =
format!("⬆ Deployment *{name}* has an update available");
let blocks = vec![
Block::header(text.clone()),
Block::section(format!(
"server: *{server_name}*\nimage: *{image}*",
)),
Block::section(resource_link(
ResourceTargetVariant::Deployment,
id,
)),
];
(text, blocks.into())
}
AlertData::DeploymentAutoUpdated {
id,
name,
server_name,
server_id: _server_id,
image,
} => {
let text =
format!("⬆ Deployment *{name}* was updated automatically ⏫");
let blocks = vec![
Block::header(text.clone()),
Block::section(format!(
"server: *{server_name}*\nimage: *{image}*",
)),
Block::section(resource_link(
ResourceTargetVariant::Deployment,
id,
)),
];
(text, blocks.into())
}
AlertData::StackStateChange {
name,
server_name,
@@ -327,11 +280,56 @@ async fn send_slack_alert(
..
} => {
let to = fmt_stack_state(to);
let text = format!("🥞 Stack *{name}* is now {to}");
let text = format!("🥞 Stack *{name}* is now *{to}*");
let blocks = vec![
Block::header(text.clone()),
Block::section(format!(
"server: {server_name}\nprevious: {from}",
"server: *{server_name}*\nprevious: *{from}*",
)),
Block::section(resource_link(
ResourceTargetVariant::Stack,
id,
)),
];
(text, blocks.into())
}
AlertData::StackImageUpdateAvailable {
id,
name,
server_name,
server_id: _server_id,
service,
image,
} => {
let text = format!("⬆ Stack *{name}* has an update available");
let blocks = vec![
Block::header(text.clone()),
Block::section(format!(
"server: *{server_name}*\nservice: *{service}*\nimage: *{image}*",
)),
Block::section(resource_link(
ResourceTargetVariant::Stack,
id,
)),
];
(text, blocks.into())
}
AlertData::StackAutoUpdated {
id,
name,
server_name,
server_id: _server_id,
images,
} => {
let text =
format!("⬆ Stack *{name}* was updated automatically ⏫");
let images_label =
if images.len() > 1 { "images" } else { "image" };
let images = images.join(", ");
let blocks = vec![
Block::header(text.clone()),
Block::section(format!(
"server: *{server_name}*\n{images_label}: *{images}*",
)),
Block::section(resource_link(
ResourceTargetVariant::Stack,
@@ -356,8 +354,9 @@ async fn send_slack_alert(
(text, blocks.into())
}
AlertData::ResourceSyncPendingUpdates { id, name } => {
let text =
format!("{level} | Pending resource sync updates on {name}");
let text = format!(
"{level} | Pending resource sync updates on *{name}*"
);
let blocks = vec![
Block::header(text.clone()),
Block::section(format!(
@@ -374,21 +373,19 @@ async fn send_slack_alert(
let text = format!("{level} | Build {name} has failed");
let blocks = vec![
Block::header(text.clone()),
Block::section(format!(
"build id: *{id}*\nbuild name: *{name}*\nversion: v{version}",
Block::section(format!("version: *v{version}*",)),
Block::section(resource_link(
ResourceTargetVariant::Build,
id,
)),
Block::section(resource_link(ResourceTargetVariant::Build, id))
];
(text, blocks.into())
}
AlertData::RepoBuildFailed { id, name } => {
let text =
format!("{level} | Repo build for {name} has failed");
format!("{level} | Repo build for *{name}* has *failed*");
let blocks = vec![
Block::header(text.clone()),
Block::section(format!(
"repo id: *{id}*\nrepo name: *{name}*",
)),
Block::section(resource_link(
ResourceTargetVariant::Repo,
id,
@@ -396,83 +393,69 @@ async fn send_slack_alert(
];
(text, blocks.into())
}
AlertData::ProcedureFailed { id, name } => {
let text = format!("{level} | Procedure *{name}* has *failed*");
let blocks = vec![
Block::header(text.clone()),
Block::section(resource_link(
ResourceTargetVariant::Procedure,
id,
)),
];
(text, blocks.into())
}
AlertData::ActionFailed { id, name } => {
let text = format!("{level} | Action *{name}* has *failed*");
let blocks = vec![
Block::header(text.clone()),
Block::section(resource_link(
ResourceTargetVariant::Action,
id,
)),
];
(text, blocks.into())
}
AlertData::ScheduleRun {
resource_type,
id,
name,
} => {
let text = format!(
"{level} | *{name}* ({resource_type}) | Scheduled run started 🕝"
);
let blocks = vec![
Block::header(text.clone()),
Block::section(resource_link(*resource_type, id)),
];
(text, blocks.into())
}
AlertData::None {} => Default::default(),
};
if !text.is_empty() {
let slack = slack::Client::new(url);
slack.send_message(text, blocks).await?;
let vars_and_secrets = get_variables_and_secrets().await?;
let mut global_replacers = HashSet::new();
let mut secret_replacers = HashSet::new();
let mut url_interpolated = url.to_string();
// interpolate variables and secrets into the url
interpolate_variables_secrets_into_string(
&vars_and_secrets,
&mut url_interpolated,
&mut global_replacers,
&mut secret_replacers,
)?;
let slack = ::slack::Client::new(url_interpolated);
slack.send_message(text, blocks).await.map_err(|e| {
let replacers =
secret_replacers.into_iter().collect::<Vec<_>>();
let sanitized_error =
svi::replace_in_string(&format!("{e:?}"), &replacers);
anyhow::Error::msg(format!(
"Error with slack request: {}",
sanitized_error
))
})?;
}
Ok(())
}
fn fmt_region(region: &Option<String>) -> String {
match region {
Some(region) => format!(" ({region})"),
None => String::new(),
}
}
fn fmt_docker_container_state(state: &DeploymentState) -> String {
match state {
DeploymentState::Running => String::from("Running ▶️"),
DeploymentState::Exited => String::from("Exited 🛑"),
DeploymentState::Restarting => String::from("Restarting 🔄"),
DeploymentState::NotDeployed => String::from("Not Deployed"),
_ => state.to_string(),
}
}
fn fmt_stack_state(state: &StackState) -> String {
match state {
StackState::Running => String::from("Running ▶️"),
StackState::Stopped => String::from("Stopped 🛑"),
StackState::Restarting => String::from("Restarting 🔄"),
StackState::Down => String::from("Down ⬇️"),
_ => state.to_string(),
}
}
fn fmt_level(level: SeverityLevel) -> &'static str {
match level {
SeverityLevel::Critical => "CRITICAL 🚨",
SeverityLevel::Warning => "WARNING ‼️",
SeverityLevel::Ok => "OK ✅",
}
}
fn resource_link(
resource_type: ResourceTargetVariant,
id: &str,
) -> String {
let path = match resource_type {
ResourceTargetVariant::System => unreachable!(),
ResourceTargetVariant::Build => format!("/builds/{id}"),
ResourceTargetVariant::Builder => {
format!("/builders/{id}")
}
ResourceTargetVariant::Deployment => {
format!("/deployments/{id}")
}
ResourceTargetVariant::Stack => {
format!("/stacks/{id}")
}
ResourceTargetVariant::Server => {
format!("/servers/{id}")
}
ResourceTargetVariant::Repo => format!("/repos/{id}"),
ResourceTargetVariant::Alerter => {
format!("/alerters/{id}")
}
ResourceTargetVariant::Procedure => {
format!("/procedures/{id}")
}
ResourceTargetVariant::ServerTemplate => {
format!("/server-templates/{id}")
}
ResourceTargetVariant::ResourceSync => {
format!("/resource-syncs/{id}")
}
};
format!("{}{path}", core_config().host)
}

View File

@@ -1,13 +1,12 @@
use std::{sync::OnceLock, time::Instant};
use anyhow::anyhow;
use axum::{http::HeaderMap, routing::post, Router};
use axum_extra::{headers::ContentType, TypedHeader};
use axum::{Router, http::HeaderMap, routing::post};
use derive_variants::{EnumVariants, ExtractVariant};
use komodo_client::{api::auth::*, entities::user::User};
use reqwest::StatusCode;
use resolver_api::{derive::Resolver, Resolve, Resolver};
use resolver_api::Resolve;
use response::Response;
use serde::{Deserialize, Serialize};
use serror::{AddStatusCode, Json};
use serror::Json;
use typeshare::typeshare;
use uuid::Uuid;
@@ -16,16 +15,25 @@ use crate::{
get_user_id_from_headers,
github::{self, client::github_oauth_client},
google::{self, client::google_oauth_client},
oidc,
},
config::core_config,
helpers::query::get_user,
state::{jwt_client, State},
state::jwt_client,
};
pub struct AuthArgs {
pub headers: HeaderMap,
}
#[typeshare]
#[derive(Serialize, Deserialize, Debug, Clone, Resolver)]
#[resolver_target(State)]
#[resolver_args(HeaderMap)]
#[derive(
Serialize, Deserialize, Debug, Clone, Resolve, EnumVariants,
)]
#[args(AuthArgs)]
#[response(Response)]
#[error(serror::Error)]
#[variant_derive(Debug)]
#[serde(tag = "type", content = "params")]
#[allow(clippy::enum_variant_names, clippy::large_enum_variant)]
pub enum AuthRequest {
@@ -39,14 +47,25 @@ pub enum AuthRequest {
pub fn router() -> Router {
let mut router = Router::new().route("/", post(handler));
if core_config().local_auth {
info!("🔑 Local Login Enabled");
}
if github_oauth_client().is_some() {
info!("🔑 Github Login Enabled");
router = router.nest("/github", github::router())
}
if google_oauth_client().is_some() {
info!("🔑 Github Login Enabled");
router = router.nest("/google", google::router())
}
if core_config().oidc_enabled {
info!("🔑 OIDC Login Enabled");
router = router.nest("/oidc", oidc::router())
}
router
}
@@ -54,27 +73,20 @@ pub fn router() -> Router {
async fn handler(
headers: HeaderMap,
Json(request): Json<AuthRequest>,
) -> serror::Result<(TypedHeader<ContentType>, String)> {
) -> serror::Result<axum::response::Response> {
let timer = Instant::now();
let req_id = Uuid::new_v4();
debug!("/auth request {req_id} | METHOD: {}", request.req_type());
let res = State.resolve_request(request, headers).await.map_err(
|e| match e {
resolver_api::Error::Serialization(e) => {
anyhow!("{e:?}").context("response serialization error")
}
resolver_api::Error::Inner(e) => e,
},
debug!(
"/auth request {req_id} | METHOD: {:?}",
request.extract_variant()
);
let res = request.resolve(&AuthArgs { headers }).await;
if let Err(e) = &res {
debug!("/auth request {req_id} | error: {e:#}");
debug!("/auth request {req_id} | error: {:#}", e.error);
}
let elapsed = timer.elapsed();
debug!("/auth request {req_id} | resolve time: {elapsed:?}");
Ok((
TypedHeader(ContentType::json()),
res.status_code(StatusCode::UNAUTHORIZED)?,
))
res.map(|res| res.0)
}
fn login_options_reponse() -> &'static GetLoginOptionsResponse {
@@ -91,43 +103,42 @@ fn login_options_reponse() -> &'static GetLoginOptionsResponse {
google: config.google_oauth.enabled
&& !config.google_oauth.id.is_empty()
&& !config.google_oauth.secret.is_empty(),
oidc: config.oidc_enabled
&& !config.oidc_provider.is_empty()
&& !config.oidc_client_id.is_empty(),
registration_disabled: config.disable_user_registration,
}
})
}
impl Resolve<GetLoginOptions, HeaderMap> for State {
impl Resolve<AuthArgs> for GetLoginOptions {
#[instrument(name = "GetLoginOptions", level = "debug", skip(self))]
async fn resolve(
&self,
_: GetLoginOptions,
_: HeaderMap,
) -> anyhow::Result<GetLoginOptionsResponse> {
self,
_: &AuthArgs,
) -> serror::Result<GetLoginOptionsResponse> {
Ok(*login_options_reponse())
}
}
impl Resolve<ExchangeForJwt, HeaderMap> for State {
impl Resolve<AuthArgs> for ExchangeForJwt {
#[instrument(name = "ExchangeForJwt", level = "debug", skip(self))]
async fn resolve(
&self,
ExchangeForJwt { token }: ExchangeForJwt,
_: HeaderMap,
) -> anyhow::Result<ExchangeForJwtResponse> {
let jwt = jwt_client().redeem_exchange_token(&token).await?;
let res = ExchangeForJwtResponse { jwt };
Ok(res)
self,
_: &AuthArgs,
) -> serror::Result<ExchangeForJwtResponse> {
let jwt = jwt_client().redeem_exchange_token(&self.token).await?;
Ok(ExchangeForJwtResponse { jwt })
}
}
impl Resolve<GetUser, HeaderMap> for State {
impl Resolve<AuthArgs> for GetUser {
#[instrument(name = "GetUser", level = "debug", skip(self))]
async fn resolve(
&self,
GetUser {}: GetUser,
headers: HeaderMap,
) -> anyhow::Result<User> {
let user_id = get_user_id_from_headers(&headers).await?;
get_user(&user_id).await
self,
AuthArgs { headers }: &AuthArgs,
) -> serror::Result<User> {
let user_id = get_user_id_from_headers(headers).await?;
Ok(get_user(&user_id).await?)
}
}

View File

@@ -0,0 +1,370 @@
use std::{
collections::HashSet,
path::{Path, PathBuf},
str::FromStr,
sync::OnceLock,
};
use anyhow::Context;
use command::run_komodo_command;
use komodo_client::{
api::{
execute::{BatchExecutionResponse, BatchRunAction, RunAction},
user::{CreateApiKey, CreateApiKeyResponse, DeleteApiKey},
},
entities::{
action::Action,
alert::{Alert, AlertData, SeverityLevel},
config::core::CoreConfig,
komodo_timestamp,
permission::PermissionLevel,
update::Update,
user::action_user,
},
};
use mungos::{by_id::update_one_by_id, mongodb::bson::to_document};
use resolver_api::Resolve;
use tokio::fs;
use crate::{
alert::send_alerts,
api::{execute::ExecuteRequest, user::UserArgs},
config::core_config,
helpers::{
interpolate::{
add_interp_update_log,
interpolate_variables_secrets_into_string,
},
query::get_variables_and_secrets,
random_string,
update::update_update,
},
resource::{self, refresh_action_state_cache},
state::{action_states, db_client},
};
use super::ExecuteArgs;
impl super::BatchExecute for BatchRunAction {
type Resource = Action;
fn single_request(action: String) -> ExecuteRequest {
ExecuteRequest::RunAction(RunAction { action })
}
}
impl Resolve<ExecuteArgs> for BatchRunAction {
#[instrument(name = "BatchRunAction", skip(self, user), fields(user_id = user.id))]
async fn resolve(
self,
ExecuteArgs { user, .. }: &ExecuteArgs,
) -> serror::Result<BatchExecutionResponse> {
Ok(
super::batch_execute::<BatchRunAction>(&self.pattern, user)
.await?,
)
}
}
impl Resolve<ExecuteArgs> for RunAction {
#[instrument(name = "RunAction", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let mut action = resource::get_check_permissions::<Action>(
&self.action,
user,
PermissionLevel::Execute,
)
.await?;
// get the action state for the action (or insert default).
let action_state = action_states()
.action
.get_or_insert_default(&action.id)
.await;
// This will set action state back to default when dropped.
// Will also check to ensure action not already busy before updating.
let _action_guard =
action_state.update(|state| state.running = true)?;
let mut update = update.clone();
update_update(update.clone()).await?;
let CreateApiKeyResponse { key, secret } = CreateApiKey {
name: update.id.clone(),
expires: 0,
}
.resolve(&UserArgs {
user: action_user().to_owned(),
})
.await?;
let contents = &mut action.config.file_contents;
// Wrap the file contents in the execution context.
*contents = full_contents(contents, &key, &secret);
let replacers =
interpolate(contents, &mut update, key.clone(), secret.clone())
.await?
.into_iter()
.collect::<Vec<_>>();
let file = format!("{}.ts", random_string(10));
let path = core_config().action_directory.join(&file);
if let Some(parent) = path.parent() {
fs::create_dir_all(parent)
.await
.with_context(|| format!("Failed to initialize Action file parent directory {parent:?}"))?;
}
fs::write(&path, contents).await.with_context(|| {
format!("Failed to write action file to {path:?}")
})?;
let CoreConfig { ssl_enabled, .. } = core_config();
let https_cert_flag = if *ssl_enabled {
" --unsafely-ignore-certificate-errors=localhost"
} else {
""
};
let mut res = run_komodo_command(
// Keep this stage name as is, the UI will find the latest update log by matching the stage name
"Execute Action",
None,
format!(
"deno run --allow-all{https_cert_flag} {}",
path.display()
),
)
.await;
res.stdout = svi::replace_in_string(&res.stdout, &replacers)
.replace(&key, "<ACTION_API_KEY>");
res.stderr = svi::replace_in_string(&res.stderr, &replacers)
.replace(&secret, "<ACTION_API_SECRET>");
cleanup_run(file + ".js", &path).await;
if let Err(e) = (DeleteApiKey { key })
.resolve(&UserArgs {
user: action_user().to_owned(),
})
.await
{
warn!(
"Failed to delete API key after action execution | {:#}",
e.error
);
};
update.logs.push(res);
update.finalize();
// Need to manually update the update before cache refresh,
// and before broadcast with update_update.
// The Err case of to_document should be unreachable,
// but will fail to update cache in that case.
if let Ok(update_doc) = to_document(&update) {
let _ = update_one_by_id(
&db_client().updates,
&update.id,
mungos::update::Update::Set(update_doc),
None,
)
.await;
refresh_action_state_cache().await;
}
update_update(update.clone()).await?;
if !update.success && action.config.failure_alert {
warn!("action unsuccessful, alerting...");
let target = update.target.clone();
tokio::spawn(async move {
let alert = Alert {
id: Default::default(),
target,
ts: komodo_timestamp(),
resolved_ts: Some(komodo_timestamp()),
resolved: true,
level: SeverityLevel::Warning,
data: AlertData::ActionFailed {
id: action.id,
name: action.name,
},
};
send_alerts(&[alert]).await
});
}
Ok(update)
}
}
async fn interpolate(
contents: &mut String,
update: &mut Update,
key: String,
secret: String,
) -> serror::Result<HashSet<(String, String)>> {
let mut vars_and_secrets = get_variables_and_secrets().await?;
vars_and_secrets
.secrets
.insert(String::from("ACTION_API_KEY"), key);
vars_and_secrets
.secrets
.insert(String::from("ACTION_API_SECRET"), secret);
let mut global_replacers = HashSet::new();
let mut secret_replacers = HashSet::new();
interpolate_variables_secrets_into_string(
&vars_and_secrets,
contents,
&mut global_replacers,
&mut secret_replacers,
)?;
add_interp_update_log(update, &global_replacers, &secret_replacers);
Ok(secret_replacers)
}
fn full_contents(contents: &str, key: &str, secret: &str) -> String {
let CoreConfig {
port, ssl_enabled, ..
} = core_config();
let protocol = if *ssl_enabled { "https" } else { "http" };
let base_url = format!("{protocol}://localhost:{port}");
format!(
"import {{ KomodoClient }} from '{base_url}/client/lib.js';
import * as __YAML__ from 'jsr:@std/yaml';
import * as __TOML__ from 'jsr:@std/toml';
const YAML = {{
stringify: __YAML__.stringify,
parse: __YAML__.parse,
parseAll: __YAML__.parseAll,
parseDockerCompose: __YAML__.parse,
}}
const TOML = {{
stringify: __TOML__.stringify,
parse: __TOML__.parse,
parseResourceToml: __TOML__.parse,
parseCargoToml: __TOML__.parse,
}}
const komodo = KomodoClient('{base_url}', {{
type: 'api-key',
params: {{ key: '{key}', secret: '{secret}' }}
}});
async function main() {{
{contents}
console.log('🦎 Action completed successfully 🦎');
}}
main()
.catch(error => {{
console.error('🚨 Action exited early with errors 🚨')
if (error.status !== undefined && error.result !== undefined) {{
console.error('Status:', error.status);
console.error(JSON.stringify(error.result, null, 2));
}} else {{
console.error(JSON.stringify(error, null, 2));
}}
Deno.exit(1)
}});"
)
}
/// Cleans up file at given path.
/// ALSO if $DENO_DIR is set,
/// will clean up the generated file matching "file"
async fn cleanup_run(file: String, path: &Path) {
if let Err(e) = fs::remove_file(path).await {
warn!(
"Failed to delete action file after action execution | {e:#}"
);
}
// If $DENO_DIR is set (will be in container),
// will clean up the generated file matching "file" (NOT under path)
let Some(deno_dir) = deno_dir() else {
return;
};
delete_file(deno_dir.join("gen/file"), file).await;
}
fn deno_dir() -> Option<&'static Path> {
static DENO_DIR: OnceLock<Option<PathBuf>> = OnceLock::new();
DENO_DIR
.get_or_init(|| {
let deno_dir = std::env::var("DENO_DIR").ok()?;
PathBuf::from_str(&deno_dir).ok()
})
.as_deref()
}
/// file is just the terminating file path,
/// it may be nested multiple folder under path,
/// this will find the nested file and delete it.
/// Assumes the file is only there once.
fn delete_file(
dir: PathBuf,
file: String,
) -> std::pin::Pin<Box<dyn std::future::Future<Output = bool> + Send>>
{
Box::pin(async move {
let Ok(mut dir) = fs::read_dir(dir).await else {
return false;
};
// Collect the nested folders for recursing
// only after checking all the files in directory.
let mut folders = Vec::<PathBuf>::new();
while let Ok(Some(entry)) = dir.next_entry().await {
let Ok(meta) = entry.metadata().await else {
continue;
};
if meta.is_file() {
let Ok(name) = entry.file_name().into_string() else {
continue;
};
if name == file {
if let Err(e) = fs::remove_file(entry.path()).await {
warn!(
"Failed to clean up generated file after action execution | {e:#}"
);
};
return true;
}
} else {
folders.push(entry.path());
}
}
if folders.len() == 1 {
// unwrap ok, folders definitely is not empty
let folder = folders.pop().unwrap();
delete_file(folder, file).await
} else {
// Check folders with file.clone
for folder in folders {
if delete_file(folder, file.clone()).await {
return true;
}
}
false
}
})
}

View File

@@ -0,0 +1,73 @@
use formatting::format_serror;
use komodo_client::{
api::execute::TestAlerter,
entities::{
alert::{Alert, AlertData, SeverityLevel},
alerter::Alerter,
komodo_timestamp,
permission::PermissionLevel,
},
};
use resolver_api::Resolve;
use crate::{
alert::send_alert_to_alerter, helpers::update::update_update,
resource::get_check_permissions,
};
use super::ExecuteArgs;
impl Resolve<ExecuteArgs> for TestAlerter {
#[instrument(name = "TestAlerter", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> Result<Self::Response, Self::Error> {
let alerter = get_check_permissions::<Alerter>(
&self.alerter,
user,
PermissionLevel::Execute,
)
.await?;
let mut update = update.clone();
if !alerter.config.enabled {
update.push_error_log(
"Test Alerter",
String::from(
"Alerter is disabled. Enable the Alerter to send alerts.",
),
);
update.finalize();
update_update(update.clone()).await?;
return Ok(update);
}
let ts = komodo_timestamp();
let alert = Alert {
id: Default::default(),
ts,
resolved: true,
level: SeverityLevel::Ok,
target: update.target.clone(),
data: AlertData::Test {
id: alerter.id.clone(),
name: alerter.name.clone(),
},
resolved_ts: Some(ts),
};
if let Err(e) = send_alert_to_alerter(&alerter, &alert).await {
update.push_error_log("Test Alerter", format_serror(&e.into()));
} else {
update.push_simple_log("Test Alerter", String::from("Alert sent successfully. It should be visible at your alerting destination."));
};
update.finalize();
update_update(update.clone()).await?;
Ok(update)
}
}

View File

@@ -1,22 +1,23 @@
use std::{collections::HashSet, future::IntoFuture, time::Duration};
use anyhow::{anyhow, Context};
use anyhow::{Context, anyhow};
use formatting::format_serror;
use futures::future::join_all;
use komodo_client::{
api::execute::{CancelBuild, Deploy, RunBuild},
api::execute::{
BatchExecutionResponse, BatchRunBuild, CancelBuild, Deploy,
RunBuild,
},
entities::{
alert::{Alert, AlertData, SeverityLevel},
all_logs_success,
build::{Build, ImageRegistry, StandardRegistryConfig},
build::{Build, BuildConfig, ImageRegistryConfig},
builder::{Builder, BuilderConfig},
config::core::{AwsEcrConfig, AwsEcrConfigWithCredentials},
deployment::DeploymentState,
komodo_timestamp,
permission::PermissionLevel,
to_komodo_name,
update::{Log, Update},
user::{auto_redeploy_user, User},
user::auto_redeploy_user,
},
};
use mungos::{
@@ -32,17 +33,15 @@ use resolver_api::Resolve;
use tokio_util::sync::CancellationToken;
use crate::{
cloud::aws::ecr,
config::core_config,
alert::send_alerts,
helpers::{
alert::send_alerts,
builder::{cleanup_builder_instance, get_builder_periphery},
channel::build_cancel_channel,
git_token,
interpolate::{
add_interp_update_log,
interpolate_variables_secrets_into_environment,
interpolate_variables_secrets_into_extra_args,
interpolate_variables_secrets_into_string,
interpolate_variables_secrets_into_system_command,
},
query::{get_deployment_state, get_variables_and_secrets},
@@ -50,28 +49,53 @@ use crate::{
update::{init_execution_update, update_update},
},
resource::{self, refresh_build_state_cache},
state::{action_states, db_client, State},
state::{action_states, db_client},
};
use super::ExecuteRequest;
use super::{ExecuteArgs, ExecuteRequest};
impl Resolve<RunBuild, (User, Update)> for State {
#[instrument(name = "RunBuild", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl super::BatchExecute for BatchRunBuild {
type Resource = Build;
fn single_request(build: String) -> ExecuteRequest {
ExecuteRequest::RunBuild(RunBuild { build })
}
}
impl Resolve<ExecuteArgs> for BatchRunBuild {
#[instrument(name = "BatchRunBuild", skip(user), fields(user_id = user.id))]
async fn resolve(
&self,
RunBuild { build }: RunBuild,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, .. }: &ExecuteArgs,
) -> serror::Result<BatchExecutionResponse> {
Ok(
super::batch_execute::<BatchRunBuild>(&self.pattern, user)
.await?,
)
}
}
impl Resolve<ExecuteArgs> for RunBuild {
#[instrument(name = "RunBuild", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let mut build = resource::get_check_permissions::<Build>(
&build,
&user,
&self.build,
user,
PermissionLevel::Execute,
)
.await?;
let vars_and_secrets = get_variables_and_secrets().await?;
let mut vars_and_secrets = get_variables_and_secrets().await?;
// Add the $VERSION to variables. Use with [[$VERSION]]
vars_and_secrets.variables.insert(
String::from("$VERSION"),
build.config.version.to_string(),
);
if build.config.builder_id.is_empty() {
return Err(anyhow!("Must attach builder to RunBuild"));
return Err(anyhow!("Must attach builder to RunBuild").into());
}
// get the action state for the build (or insert default).
@@ -86,6 +110,9 @@ impl Resolve<RunBuild, (User, Update)> for State {
if build.config.auto_increment_version {
build.config.version.increment();
}
let mut update = update.clone();
update.version = build.config.version;
update_update(update.clone()).await?;
@@ -99,8 +126,8 @@ impl Resolve<RunBuild, (User, Update)> for State {
|| format!("Failed to get git token in call to db. This is a database error, not a token exisitence error. Stopping run. | {} | {}", build.config.git_provider, build.config.git_account),
)?;
let (registry_token, aws_ecr) =
validate_account_extract_registry_token_aws_ecr(&build).await?;
let registry_token =
validate_account_extract_registry_token(&build).await?;
let cancel = CancellationToken::new();
let cancel_clone = cancel.clone();
@@ -148,7 +175,6 @@ impl Resolve<RunBuild, (User, Update)> for State {
});
// GET BUILDER PERIPHERY
let (periphery, cleanup_data) = match get_builder_periphery(
build.name.clone(),
Some(build.config.version),
@@ -174,10 +200,8 @@ impl Resolve<RunBuild, (User, Update)> for State {
}
};
// CLONE REPO
// INTERPOLATE VARIABLES
let secret_replacers = if !build.config.skip_secret_interp {
// Interpolate variables / secrets into pre build command
let mut global_replacers = HashSet::new();
let mut secret_replacers = HashSet::new();
@@ -188,6 +212,34 @@ impl Resolve<RunBuild, (User, Update)> for State {
&mut secret_replacers,
)?;
interpolate_variables_secrets_into_string(
&vars_and_secrets,
&mut build.config.build_args,
&mut global_replacers,
&mut secret_replacers,
)?;
interpolate_variables_secrets_into_string(
&vars_and_secrets,
&mut build.config.secret_args,
&mut global_replacers,
&mut secret_replacers,
)?;
interpolate_variables_secrets_into_string(
&vars_and_secrets,
&mut build.config.dockerfile,
&mut global_replacers,
&mut secret_replacers,
)?;
interpolate_variables_secrets_into_extra_args(
&vars_and_secrets,
&mut build.config.extra_args,
&mut global_replacers,
&mut secret_replacers,
)?;
add_interp_update_log(
&mut update,
&global_replacers,
@@ -199,90 +251,62 @@ impl Resolve<RunBuild, (User, Update)> for State {
Default::default()
};
let res = tokio::select! {
res = periphery
.request(api::git::CloneRepo {
args: (&build).into(),
git_token,
environment: Default::default(),
env_file_path: Default::default(),
skip_secret_interp: Default::default(),
replacers: secret_replacers.into_iter().collect(),
}) => res,
_ = cancel.cancelled() => {
debug!("build cancelled during clone, cleaning up builder");
update.push_error_log("build cancelled", String::from("user cancelled build during repo clone"));
cleanup_builder_instance(periphery, cleanup_data, &mut update)
.await;
info!("builder cleaned up");
return handle_early_return(update, build.id, build.name, true).await
},
};
let commit_message = match res {
Ok(res) => {
debug!("finished repo clone");
update.logs.extend(res.logs);
update.commit_hash =
res.commit_hash.unwrap_or_default().to_string();
res.commit_message.unwrap_or_default()
}
Err(e) => {
warn!("failed build at clone repo | {e:#}");
update.push_error_log(
"clone repo",
format_serror(&e.context("failed to clone repo").into()),
);
Default::default()
}
};
update_update(update.clone()).await?;
if all_logs_success(&update.logs) {
let secret_replacers = if !build.config.skip_secret_interp {
// Interpolate variables / secrets into build args
let mut global_replacers = HashSet::new();
let mut secret_replacers = HashSet::new();
interpolate_variables_secrets_into_environment(
&vars_and_secrets,
&mut build.config.build_args,
&mut global_replacers,
&mut secret_replacers,
)?;
interpolate_variables_secrets_into_environment(
&vars_and_secrets,
&mut build.config.secret_args,
&mut global_replacers,
&mut secret_replacers,
)?;
interpolate_variables_secrets_into_extra_args(
&vars_and_secrets,
&mut build.config.extra_args,
&mut global_replacers,
&mut secret_replacers,
)?;
add_interp_update_log(
&mut update,
&global_replacers,
&secret_replacers,
);
secret_replacers
} else {
Default::default()
let commit_message = if !build.config.files_on_host
&& !build.config.repo.is_empty()
{
// CLONE REPO
let res = tokio::select! {
res = periphery
.request(api::git::CloneRepo {
args: (&build).into(),
git_token,
environment: Default::default(),
env_file_path: Default::default(),
skip_secret_interp: Default::default(),
replacers: Default::default(),
}) => res,
_ = cancel.cancelled() => {
debug!("build cancelled during clone, cleaning up builder");
update.push_error_log("build cancelled", String::from("user cancelled build during repo clone"));
cleanup_builder_instance(cleanup_data, &mut update)
.await;
info!("builder cleaned up");
return handle_early_return(update, build.id, build.name, true).await
},
};
let commit_message = match res {
Ok(res) => {
debug!("finished repo clone");
update.logs.extend(res.logs);
update.commit_hash =
res.commit_hash.unwrap_or_default().to_string();
res.commit_message.unwrap_or_default()
}
Err(e) => {
warn!("failed build at clone repo | {e:#}");
update.push_error_log(
"clone repo",
format_serror(&e.context("failed to clone repo").into()),
);
Default::default()
}
};
update_update(update.clone()).await?;
Some(commit_message)
} else {
None
};
if all_logs_success(&update.logs) {
// RUN BUILD
let res = tokio::select! {
res = periphery
.request(api::build::Build {
build: build.clone(),
registry_token,
aws_ecr,
replacers: secret_replacers.into_iter().collect(),
// Push a commit hash tagged image
additional_tags: if update.commit_hash.is_empty() {
@@ -294,7 +318,7 @@ impl Resolve<RunBuild, (User, Update)> for State {
_ = cancel.cancelled() => {
info!("build cancelled during build, cleaning up builder");
update.push_error_log("build cancelled", String::from("user cancelled build during docker build"));
cleanup_builder_instance(periphery, cleanup_data, &mut update)
cleanup_builder_instance(cleanup_data, &mut update)
.await;
return handle_early_return(update, build.id, build.name, true).await
},
@@ -317,7 +341,7 @@ impl Resolve<RunBuild, (User, Update)> for State {
update.finalize();
let db = db_client().await;
let db = db_client();
if update.success {
let _ = db
@@ -338,8 +362,9 @@ impl Resolve<RunBuild, (User, Update)> for State {
// stop the cancel listening task from going forever
cancel.cancel();
cleanup_builder_instance(periphery, cleanup_data, &mut update)
.await;
// If building on temporary cloud server (AWS),
// this will terminate the server.
cleanup_builder_instance(cleanup_data, &mut update).await;
// Need to manually update the update before cache refresh,
// and before broadcast with add_update.
@@ -385,7 +410,7 @@ impl Resolve<RunBuild, (User, Update)> for State {
});
}
Ok(update)
Ok(update.clone())
}
}
@@ -395,7 +420,7 @@ async fn handle_early_return(
build_id: String,
build_name: String,
is_cancel: bool,
) -> anyhow::Result<Update> {
) -> serror::Result<Update> {
update.finalize();
// Need to manually update the update before cache refresh,
// and before broadcast with add_update.
@@ -403,7 +428,7 @@ async fn handle_early_return(
// but will fail to update cache in that case.
if let Ok(update_doc) = to_document(&update) {
let _ = update_one_by_id(
&db_client().await.updates,
&db_client().updates,
&update.id,
mungos::update::Update::Set(update_doc),
None,
@@ -433,17 +458,16 @@ async fn handle_early_return(
send_alerts(&[alert]).await
});
}
Ok(update)
Ok(update.clone())
}
#[instrument(skip_all)]
pub async fn validate_cancel_build(
request: &ExecuteRequest,
) -> anyhow::Result<()> {
if let ExecuteRequest::CancelBuild(req) = request {
let build = resource::get::<Build>(&req.build).await?;
let db = db_client().await;
let db = db_client();
let (latest_build, latest_cancel) = tokio::try_join!(
db.updates
@@ -483,16 +507,15 @@ pub async fn validate_cancel_build(
Ok(())
}
impl Resolve<CancelBuild, (User, Update)> for State {
#[instrument(name = "CancelBuild", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for CancelBuild {
#[instrument(name = "CancelBuild", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
CancelBuild { build }: CancelBuild,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let build = resource::get_check_permissions::<Build>(
&build,
&user,
&self.build,
user,
PermissionLevel::Execute,
)
.await?;
@@ -505,9 +528,11 @@ impl Resolve<CancelBuild, (User, Update)> for State {
.and_then(|s| s.get().ok().map(|s| s.building))
.unwrap_or_default()
{
return Err(anyhow!("Build is not building."));
return Err(anyhow!("Build is not building.").into());
}
let mut update = update.clone();
update.push_simple_log(
"cancel triggered",
"the build cancel has been triggered",
@@ -526,14 +551,16 @@ impl Resolve<CancelBuild, (User, Update)> for State {
tokio::spawn(async move {
tokio::time::sleep(Duration::from_secs(60)).await;
if let Err(e) = update_one_by_id(
&db_client().await.updates,
&db_client().updates,
&update_id,
doc! { "$set": { "status": "Complete" } },
None,
)
.await
{
warn!("failed to set CancelBuild Update status Complete after timeout | {e:#}")
warn!(
"failed to set CancelBuild Update status Complete after timeout | {e:#}"
)
}
});
@@ -544,7 +571,7 @@ impl Resolve<CancelBuild, (User, Update)> for State {
#[instrument]
async fn handle_post_build_redeploy(build_id: &str) {
let Ok(redeploy_deployments) = find_collect(
&db_client().await.deployments,
&db_client().deployments,
doc! {
"config.image.params.build_id": build_id,
"config.redeploy_on_build": true
@@ -571,16 +598,13 @@ async fn handle_post_build_redeploy(build_id: &str) {
let user = auto_redeploy_user().to_owned();
let res = async {
let update = init_execution_update(&req, &user).await?;
State
.resolve(
Deploy {
deployment: deployment.id.clone(),
stop_signal: None,
stop_time: None,
},
(user, update),
)
.await
Deploy {
deployment: deployment.id.clone(),
stop_signal: None,
stop_time: None,
}
.resolve(&ExecuteArgs { user, update })
.await
}
.await;
Some((deployment.id.clone(), res))
@@ -594,71 +618,45 @@ async fn handle_post_build_redeploy(build_id: &str) {
continue;
};
if let Err(e) = res {
warn!("failed post build redeploy for deployment {id}: {e:#}");
warn!(
"failed post build redeploy for deployment {id}: {:#}",
e.error
);
}
}
}
/// This will make sure that a build with non-none image registry has an account attached,
/// and will check the core config for a token / aws ecr config matching requirements.
/// and will check the core config for a token matching requirements.
/// Otherwise it is left to periphery.
async fn validate_account_extract_registry_token_aws_ecr(
build: &Build,
) -> anyhow::Result<(Option<String>, Option<AwsEcrConfig>)> {
let (domain, account) = match &build.config.image_registry {
// Early return for None
ImageRegistry::None(_) => return Ok((None, None)),
// Early return for AwsEcr
ImageRegistry::AwsEcr(label) => {
// Note that aws ecr config still only lives in config file
let config = core_config()
.aws_ecr_registries
.iter()
.find(|reg| &reg.label == label);
let token = match config {
Some(AwsEcrConfigWithCredentials {
region,
access_key_id,
secret_access_key,
..
}) => {
let token = ecr::get_ecr_token(
region,
access_key_id,
secret_access_key,
)
.await
.context("failed to get aws ecr token")?;
ecr::maybe_create_repo(
&to_komodo_name(&build.name),
region.to_string(),
access_key_id,
secret_access_key,
)
.await
.context("failed to create aws ecr repo")?;
Some(token)
}
None => None,
};
return Ok((token, config.map(AwsEcrConfig::from)));
}
ImageRegistry::Standard(StandardRegistryConfig {
domain,
account,
..
}) => (domain.as_str(), account),
};
async fn validate_account_extract_registry_token(
Build {
config:
BuildConfig {
image_registry:
ImageRegistryConfig {
domain, account, ..
},
..
},
..
}: &Build,
) -> serror::Result<Option<String>> {
if domain.is_empty() {
return Ok(None);
}
if account.is_empty() {
return Err(anyhow!(
"Must attach account to use registry provider {domain}"
));
return Err(
anyhow!(
"Must attach account to use registry provider {domain}"
)
.into(),
);
}
let registry_token = registry_token(domain, account).await.with_context(
|| format!("Failed to get registry token in call to db. Stopping run. | {domain} | {account}"),
)?;
Ok((registry_token, None))
Ok(registry_token)
}

View File

@@ -1,35 +1,32 @@
use std::collections::HashSet;
use std::{collections::HashSet, sync::OnceLock};
use anyhow::{anyhow, Context};
use anyhow::{Context, anyhow};
use cache::TimeoutCache;
use formatting::format_serror;
use komodo_client::{
api::execute::*,
entities::{
build::{Build, ImageRegistry},
config::core::AwsEcrConfig,
Version,
build::{Build, ImageRegistryConfig},
deployment::{
extract_registry_domain, Deployment, DeploymentImage,
Deployment, DeploymentImage, extract_registry_domain,
},
get_image_name,
get_image_name, komodo_timestamp, optional_string,
permission::PermissionLevel,
server::Server,
update::{Log, Update},
user::User,
Version,
},
};
use periphery_client::api;
use resolver_api::Resolve;
use crate::{
cloud::aws::ecr,
config::core_config,
helpers::{
interpolate::{
add_interp_update_log,
interpolate_variables_secrets_into_container_command,
interpolate_variables_secrets_into_environment,
interpolate_variables_secrets_into_extra_args,
interpolate_variables_secrets_into_string,
},
periphery_client,
query::get_variables_and_secrets,
@@ -38,9 +35,35 @@ use crate::{
},
monitor::update_cache_for_server,
resource,
state::{action_states, State},
state::action_states,
};
use super::{ExecuteArgs, ExecuteRequest};
impl super::BatchExecute for BatchDeploy {
type Resource = Deployment;
fn single_request(deployment: String) -> ExecuteRequest {
ExecuteRequest::Deploy(Deploy {
deployment,
stop_signal: None,
stop_time: None,
})
}
}
impl Resolve<ExecuteArgs> for BatchDeploy {
#[instrument(name = "BatchDeploy", skip(user), fields(user_id = user.id))]
async fn resolve(
self,
ExecuteArgs { user, .. }: &ExecuteArgs,
) -> serror::Result<BatchExecutionResponse> {
Ok(
super::batch_execute::<BatchDeploy>(&self.pattern, user)
.await?,
)
}
}
async fn setup_deployment_execution(
deployment: &str,
user: &User,
@@ -53,28 +76,27 @@ async fn setup_deployment_execution(
.await?;
if deployment.config.server_id.is_empty() {
return Err(anyhow!("deployment has no server configured"));
return Err(anyhow!("Deployment has no Server configured"));
}
let server =
resource::get::<Server>(&deployment.config.server_id).await?;
if !server.config.enabled {
return Err(anyhow!("Attached Server is not enabled"));
}
Ok((deployment, server))
}
impl Resolve<Deploy, (User, Update)> for State {
#[instrument(name = "Deploy", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for Deploy {
#[instrument(name = "Deploy", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
Deploy {
deployment,
stop_signal,
stop_time,
}: Deploy,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let (mut deployment, server) =
setup_deployment_execution(&deployment, &user).await?;
setup_deployment_execution(&self.deployment, user).await?;
// get the action state for the deployment (or insert default).
let action_state = action_states()
@@ -87,42 +109,23 @@ impl Resolve<Deploy, (User, Update)> for State {
let _action_guard =
action_state.update(|state| state.deploying = true)?;
let mut update = update.clone();
// Send update after setting action state, this way frontend gets correct state.
update_update(update.clone()).await?;
let periphery = periphery_client(&server)?;
periphery
.health_check()
.await
.context("Failed server health check, stopping run.")?;
// This block resolves the attached Build to an actual versioned image
let (version, registry_token, aws_ecr) = match &deployment
.config
.image
{
let (version, registry_token) = match &deployment.config.image {
DeploymentImage::Build { build_id, version } => {
let build = resource::get::<Build>(build_id).await?;
let image_name = get_image_name(&build, |label| {
core_config()
.aws_ecr_registries
.iter()
.find(|reg| &reg.label == label)
.map(AwsEcrConfig::from)
})
.context("failed to create image name")?;
let image_name = get_image_name(&build)
.context("failed to create image name")?;
let version = if version.is_none() {
build.config.version
} else {
*version
};
// Remove ending patch if it is 0, this means use latest patch.
let version_str = if version.patch == 0 {
format!("{}.{}", version.major, version.minor)
} else {
version.to_string()
};
let version_str = version.to_string();
// Potentially add the build image_tag postfix
let version_str = if build.config.image_tag.is_empty() {
version_str
@@ -133,45 +136,27 @@ impl Resolve<Deploy, (User, Update)> for State {
deployment.config.image = DeploymentImage::Image {
image: format!("{image_name}:{version_str}"),
};
match build.config.image_registry {
ImageRegistry::None(_) => (version, None, None),
ImageRegistry::AwsEcr(label) => {
let config = core_config()
.aws_ecr_registries
.iter()
.find(|reg| reg.label == label)
.with_context(|| {
format!(
"did not find config for aws ecr registry {label}"
)
})?;
let token = ecr::get_ecr_token(
&config.region,
&config.access_key_id,
&config.secret_access_key,
)
.await
.context("failed to create aws ecr login token")?;
(version, Some(token), Some(AwsEcrConfig::from(config)))
}
ImageRegistry::Standard(params) => {
if deployment.config.image_registry_account.is_empty() {
deployment.config.image_registry_account =
params.account
}
let token = if !deployment
.config
.image_registry_account
.is_empty()
{
registry_token(&params.domain, &deployment.config.image_registry_account).await.with_context(
|| format!("Failed to get git token in call to db. Stopping run. | {} | {}", params.domain, deployment.config.image_registry_account),
)?
} else {
None
};
(version, token, None)
if build.config.image_registry.domain.is_empty() {
(version, None)
} else {
let ImageRegistryConfig {
domain, account, ..
} = build.config.image_registry;
if deployment.config.image_registry_account.is_empty() {
deployment.config.image_registry_account = account
}
let token = if !deployment
.config
.image_registry_account
.is_empty()
{
registry_token(&domain, &deployment.config.image_registry_account).await.with_context(
|| format!("Failed to get git token in call to db. Stopping run. | {domain} | {}", deployment.config.image_registry_account),
)?
} else {
None
};
(version, token)
}
}
DeploymentImage::Image { image } => {
@@ -187,7 +172,7 @@ impl Resolve<Deploy, (User, Update)> for State {
} else {
None
};
(Version::default(), token, None)
(Version::default(), token)
}
};
@@ -199,13 +184,27 @@ impl Resolve<Deploy, (User, Update)> for State {
let mut global_replacers = HashSet::new();
let mut secret_replacers = HashSet::new();
interpolate_variables_secrets_into_environment(
interpolate_variables_secrets_into_string(
&vars_and_secrets,
&mut deployment.config.environment,
&mut global_replacers,
&mut secret_replacers,
)?;
interpolate_variables_secrets_into_string(
&vars_and_secrets,
&mut deployment.config.ports,
&mut global_replacers,
&mut secret_replacers,
)?;
interpolate_variables_secrets_into_string(
&vars_and_secrets,
&mut deployment.config.volumes,
&mut global_replacers,
&mut secret_replacers,
)?;
interpolate_variables_secrets_into_extra_args(
&vars_and_secrets,
&mut deployment.config.extra_args,
@@ -213,7 +212,7 @@ impl Resolve<Deploy, (User, Update)> for State {
&mut secret_replacers,
)?;
interpolate_variables_secrets_into_container_command(
interpolate_variables_secrets_into_string(
&vars_and_secrets,
&mut deployment.config.command,
&mut global_replacers,
@@ -234,13 +233,12 @@ impl Resolve<Deploy, (User, Update)> for State {
update.version = version;
update_update(update.clone()).await?;
match periphery
match periphery_client(&server)?
.request(api::container::Deploy {
deployment,
stop_signal,
stop_time,
stop_signal: self.stop_signal,
stop_time: self.stop_time,
registry_token,
aws_ecr,
replacers: secret_replacers.into_iter().collect(),
})
.await
@@ -248,10 +246,8 @@ impl Resolve<Deploy, (User, Update)> for State {
Ok(log) => update.logs.push(log),
Err(e) => {
update.push_error_log(
"deploy container",
format_serror(
&e.context("failed to deploy container").into(),
),
"Deploy Container",
format_serror(&e.into()),
);
}
};
@@ -265,15 +261,164 @@ impl Resolve<Deploy, (User, Update)> for State {
}
}
impl Resolve<StartDeployment, (User, Update)> for State {
#[instrument(name = "StartDeployment", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
/// Wait this long after a pull to allow another pull through
const PULL_TIMEOUT: i64 = 5_000;
type ServerId = String;
type Image = String;
type PullCache = TimeoutCache<(ServerId, Image), Log>;
fn pull_cache() -> &'static PullCache {
static PULL_CACHE: OnceLock<PullCache> = OnceLock::new();
PULL_CACHE.get_or_init(Default::default)
}
pub async fn pull_deployment_inner(
deployment: Deployment,
server: &Server,
) -> anyhow::Result<Log> {
let (image, account, token) = match deployment.config.image {
DeploymentImage::Build { build_id, version } => {
let build = resource::get::<Build>(&build_id).await?;
let image_name = get_image_name(&build)
.context("failed to create image name")?;
let version = if version.is_none() {
build.config.version.to_string()
} else {
version.to_string()
};
// Potentially add the build image_tag postfix
let version = if build.config.image_tag.is_empty() {
version
} else {
format!("{version}-{}", build.config.image_tag)
};
// replace image with corresponding build image.
let image = format!("{image_name}:{version}");
if build.config.image_registry.domain.is_empty() {
(image, None, None)
} else {
let ImageRegistryConfig {
domain, account, ..
} = build.config.image_registry;
let account =
if deployment.config.image_registry_account.is_empty() {
account
} else {
deployment.config.image_registry_account
};
let token = if !account.is_empty() {
registry_token(&domain, &account).await.with_context(
|| format!("Failed to get git token in call to db. Stopping run. | {domain} | {account}"),
)?
} else {
None
};
(image, optional_string(&account), token)
}
}
DeploymentImage::Image { image } => {
let domain = extract_registry_domain(&image)?;
let token = if !deployment
.config
.image_registry_account
.is_empty()
{
registry_token(&domain, &deployment.config.image_registry_account).await.with_context(
|| format!("Failed to get git token in call to db. Stopping run. | {domain} | {}", deployment.config.image_registry_account),
)?
} else {
None
};
(
image,
optional_string(&deployment.config.image_registry_account),
token,
)
}
};
// Acquire the pull lock for this image on the server
let lock = pull_cache()
.get_lock((server.id.clone(), image.clone()))
.await;
// Lock the path lock, prevents simultaneous pulls by
// ensuring simultaneous pulls will wait for first to finish
// and checking cached results.
let mut locked = lock.lock().await;
// Early return from cache if lasted pulled with PULL_TIMEOUT
if locked.last_ts + PULL_TIMEOUT > komodo_timestamp() {
return locked.clone_res();
}
let res = async {
let log = match periphery_client(server)?
.request(api::image::PullImage {
name: image,
account,
token,
})
.await
{
Ok(log) => log,
Err(e) => Log::error("Pull image", format_serror(&e.into())),
};
update_cache_for_server(server).await;
anyhow::Ok(log)
}
.await;
// Set the cache with results. Any other calls waiting on the lock will
// then immediately also use this same result.
locked.set(&res, komodo_timestamp());
res
}
impl Resolve<ExecuteArgs> for PullDeployment {
#[instrument(name = "PullDeployment", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
StartDeployment { deployment }: StartDeployment,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let (deployment, server) =
setup_deployment_execution(&deployment, &user).await?;
setup_deployment_execution(&self.deployment, user).await?;
// get the action state for the deployment (or insert default).
let action_state = action_states()
.deployment
.get_or_insert_default(&deployment.id)
.await;
// Will check to ensure deployment not already busy before updating, and return Err if so.
// The returned guard will set the action state back to default when dropped.
let _action_guard =
action_state.update(|state| state.pulling = true)?;
let mut update = update.clone();
// Send update after setting action state, this way frontend gets correct state.
update_update(update.clone()).await?;
let log = pull_deployment_inner(deployment, &server).await?;
update.logs.push(log);
update.finalize();
update_update(update.clone()).await?;
Ok(update)
}
}
impl Resolve<ExecuteArgs> for StartDeployment {
#[instrument(name = "StartDeployment", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let (deployment, server) =
setup_deployment_execution(&self.deployment, user).await?;
// get the action state for the deployment (or insert default).
let action_state = action_states()
@@ -286,12 +431,12 @@ impl Resolve<StartDeployment, (User, Update)> for State {
let _action_guard =
action_state.update(|state| state.starting = true)?;
let mut update = update.clone();
// Send update after setting action state, this way frontend gets correct state.
update_update(update.clone()).await?;
let periphery = periphery_client(&server)?;
let log = match periphery
let log = match periphery_client(&server)?
.request(api::container::StartContainer {
name: deployment.name,
})
@@ -313,15 +458,14 @@ impl Resolve<StartDeployment, (User, Update)> for State {
}
}
impl Resolve<RestartDeployment, (User, Update)> for State {
#[instrument(name = "RestartDeployment", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for RestartDeployment {
#[instrument(name = "RestartDeployment", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
RestartDeployment { deployment }: RestartDeployment,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let (deployment, server) =
setup_deployment_execution(&deployment, &user).await?;
setup_deployment_execution(&self.deployment, user).await?;
// get the action state for the deployment (or insert default).
let action_state = action_states()
@@ -334,12 +478,12 @@ impl Resolve<RestartDeployment, (User, Update)> for State {
let _action_guard =
action_state.update(|state| state.restarting = true)?;
let mut update = update.clone();
// Send update after setting action state, this way frontend gets correct state.
update_update(update.clone()).await?;
let periphery = periphery_client(&server)?;
let log = match periphery
let log = match periphery_client(&server)?
.request(api::container::RestartContainer {
name: deployment.name,
})
@@ -363,15 +507,14 @@ impl Resolve<RestartDeployment, (User, Update)> for State {
}
}
impl Resolve<PauseDeployment, (User, Update)> for State {
#[instrument(name = "PauseDeployment", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for PauseDeployment {
#[instrument(name = "PauseDeployment", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
PauseDeployment { deployment }: PauseDeployment,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let (deployment, server) =
setup_deployment_execution(&deployment, &user).await?;
setup_deployment_execution(&self.deployment, user).await?;
// get the action state for the deployment (or insert default).
let action_state = action_states()
@@ -384,12 +527,12 @@ impl Resolve<PauseDeployment, (User, Update)> for State {
let _action_guard =
action_state.update(|state| state.pausing = true)?;
let mut update = update.clone();
// Send update after setting action state, this way frontend gets correct state.
update_update(update.clone()).await?;
let periphery = periphery_client(&server)?;
let log = match periphery
let log = match periphery_client(&server)?
.request(api::container::PauseContainer {
name: deployment.name,
})
@@ -411,15 +554,14 @@ impl Resolve<PauseDeployment, (User, Update)> for State {
}
}
impl Resolve<UnpauseDeployment, (User, Update)> for State {
#[instrument(name = "UnpauseDeployment", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for UnpauseDeployment {
#[instrument(name = "UnpauseDeployment", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
UnpauseDeployment { deployment }: UnpauseDeployment,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let (deployment, server) =
setup_deployment_execution(&deployment, &user).await?;
setup_deployment_execution(&self.deployment, user).await?;
// get the action state for the deployment (or insert default).
let action_state = action_states()
@@ -432,12 +574,12 @@ impl Resolve<UnpauseDeployment, (User, Update)> for State {
let _action_guard =
action_state.update(|state| state.unpausing = true)?;
let mut update = update.clone();
// Send update after setting action state, this way frontend gets correct state.
update_update(update.clone()).await?;
let periphery = periphery_client(&server)?;
let log = match periphery
let log = match periphery_client(&server)?
.request(api::container::UnpauseContainer {
name: deployment.name,
})
@@ -461,19 +603,14 @@ impl Resolve<UnpauseDeployment, (User, Update)> for State {
}
}
impl Resolve<StopDeployment, (User, Update)> for State {
#[instrument(name = "StopDeployment", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for StopDeployment {
#[instrument(name = "StopDeployment", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
StopDeployment {
deployment,
signal,
time,
}: StopDeployment,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let (deployment, server) =
setup_deployment_execution(&deployment, &user).await?;
setup_deployment_execution(&self.deployment, user).await?;
// get the action state for the deployment (or insert default).
let action_state = action_states()
@@ -486,18 +623,20 @@ impl Resolve<StopDeployment, (User, Update)> for State {
let _action_guard =
action_state.update(|state| state.stopping = true)?;
let mut update = update.clone();
// Send update after setting action state, this way frontend gets correct state.
update_update(update.clone()).await?;
let periphery = periphery_client(&server)?;
let log = match periphery
let log = match periphery_client(&server)?
.request(api::container::StopContainer {
name: deployment.name,
signal: signal
signal: self
.signal
.unwrap_or(deployment.config.termination_signal)
.into(),
time: time
time: self
.time
.unwrap_or(deployment.config.termination_timeout)
.into(),
})
@@ -519,19 +658,41 @@ impl Resolve<StopDeployment, (User, Update)> for State {
}
}
impl Resolve<DestroyDeployment, (User, Update)> for State {
#[instrument(name = "DestroyDeployment", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
DestroyDeployment {
impl super::BatchExecute for BatchDestroyDeployment {
type Resource = Deployment;
fn single_request(deployment: String) -> ExecuteRequest {
ExecuteRequest::DestroyDeployment(DestroyDeployment {
deployment,
signal,
time,
}: DestroyDeployment,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
signal: None,
time: None,
})
}
}
impl Resolve<ExecuteArgs> for BatchDestroyDeployment {
#[instrument(name = "BatchDestroyDeployment", skip(user), fields(user_id = user.id))]
async fn resolve(
self,
ExecuteArgs { user, .. }: &ExecuteArgs,
) -> serror::Result<BatchExecutionResponse> {
Ok(
super::batch_execute::<BatchDestroyDeployment>(
&self.pattern,
user,
)
.await?,
)
}
}
impl Resolve<ExecuteArgs> for DestroyDeployment {
#[instrument(name = "DestroyDeployment", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let (deployment, server) =
setup_deployment_execution(&deployment, &user).await?;
setup_deployment_execution(&self.deployment, user).await?;
// get the action state for the deployment (or insert default).
let action_state = action_states()
@@ -544,18 +705,20 @@ impl Resolve<DestroyDeployment, (User, Update)> for State {
let _action_guard =
action_state.update(|state| state.destroying = true)?;
let mut update = update.clone();
// Send update after setting action state, this way frontend gets correct state.
update_update(update.clone()).await?;
let periphery = periphery_client(&server)?;
let log = match periphery
let log = match periphery_client(&server)?
.request(api::container::RemoveContainer {
name: deployment.name,
signal: signal
signal: self
.signal
.unwrap_or(deployment.config.termination_signal)
.into(),
time: time
time: self
.time
.unwrap_or(deployment.config.termination_timeout)
.into(),
})

View File

@@ -1,17 +1,22 @@
use std::time::Instant;
use std::{pin::Pin, time::Instant};
use anyhow::{anyhow, Context};
use axum::{middleware, routing::post, Extension, Router};
use anyhow::Context;
use axum::{Extension, Router, middleware, routing::post};
use axum_extra::{TypedHeader, headers::ContentType};
use derive_variants::{EnumVariants, ExtractVariant};
use formatting::format_serror;
use futures::future::join_all;
use komodo_client::{
api::execute::*,
entities::{
Operation,
update::{Log, Update},
user::User,
},
};
use mungos::by_id::find_one_by_id;
use resolver_api::{derive::Resolver, Resolver};
use resolver_api::Resolve;
use response::JsonString;
use serde::{Deserialize, Serialize};
use serror::Json;
use typeshare::typeshare;
@@ -20,9 +25,12 @@ use uuid::Uuid;
use crate::{
auth::auth_request,
helpers::update::{init_execution_update, update_update},
state::{db_client, State},
resource::{KomodoResource, list_full_for_user_using_pattern},
state::db_client,
};
mod action;
mod alerter;
mod build;
mod deployment;
mod procedure;
@@ -32,10 +40,23 @@ mod server_template;
mod stack;
mod sync;
pub use {
deployment::pull_deployment_inner, stack::pull_stack_inner,
};
pub struct ExecuteArgs {
pub user: User,
pub update: Update,
}
#[typeshare]
#[derive(Serialize, Deserialize, Debug, Clone, Resolver)]
#[resolver_target(State)]
#[resolver_args((User, Update))]
#[derive(
Serialize, Deserialize, Debug, Clone, Resolve, EnumVariants,
)]
#[variant_derive(Debug)]
#[args(ExecuteArgs)]
#[response(JsonString)]
#[error(serror::Error)]
#[serde(tag = "type", content = "params")]
pub enum ExecuteRequest {
// ==== SERVER ====
@@ -63,38 +84,58 @@ pub enum ExecuteRequest {
// ==== DEPLOYMENT ====
Deploy(Deploy),
BatchDeploy(BatchDeploy),
PullDeployment(PullDeployment),
StartDeployment(StartDeployment),
RestartDeployment(RestartDeployment),
PauseDeployment(PauseDeployment),
UnpauseDeployment(UnpauseDeployment),
StopDeployment(StopDeployment),
DestroyDeployment(DestroyDeployment),
BatchDestroyDeployment(BatchDestroyDeployment),
// ==== STACK ====
DeployStack(DeployStack),
BatchDeployStack(BatchDeployStack),
DeployStackIfChanged(DeployStackIfChanged),
BatchDeployStackIfChanged(BatchDeployStackIfChanged),
PullStack(PullStack),
StartStack(StartStack),
RestartStack(RestartStack),
StopStack(StopStack),
PauseStack(PauseStack),
UnpauseStack(UnpauseStack),
DestroyStack(DestroyStack),
BatchDestroyStack(BatchDestroyStack),
// ==== BUILD ====
RunBuild(RunBuild),
BatchRunBuild(BatchRunBuild),
CancelBuild(CancelBuild),
// ==== REPO ====
CloneRepo(CloneRepo),
BatchCloneRepo(BatchCloneRepo),
PullRepo(PullRepo),
BatchPullRepo(BatchPullRepo),
BuildRepo(BuildRepo),
BatchBuildRepo(BatchBuildRepo),
CancelRepoBuild(CancelRepoBuild),
// ==== PROCEDURE ====
RunProcedure(RunProcedure),
BatchRunProcedure(BatchRunProcedure),
// ==== ACTION ====
RunAction(RunAction),
BatchRunAction(BatchRunAction),
// ==== SERVER TEMPLATE ====
LaunchServer(LaunchServer),
// ==== ALERTER ====
TestAlerter(TestAlerter),
// ==== SYNC ====
RunSync(RunSync),
}
@@ -108,53 +149,99 @@ pub fn router() -> Router {
async fn handler(
Extension(user): Extension<User>,
Json(request): Json<ExecuteRequest>,
) -> serror::Result<Json<Update>> {
let req_id = Uuid::new_v4();
// need to validate no cancel is active before any update is created.
build::validate_cancel_build(&request).await?;
let update = init_execution_update(&request, &user).await?;
let handle =
tokio::spawn(task(req_id, request, user, update.clone()));
tokio::spawn({
let update_id = update.id.clone();
async move {
let log = match handle.await {
Ok(Err(e)) => {
warn!("/execute request {req_id} task error: {e:#}",);
Log::error("task error", format_serror(&e.into()))
}
Err(e) => {
warn!("/execute request {req_id} spawn error: {e:?}",);
Log::error("spawn error", format!("{e:#?}"))
}
_ => return,
};
let res = async {
let mut update =
find_one_by_id(&db_client().await.updates, &update_id)
.await
.context("failed to query to db")?
.context("no update exists with given id")?;
update.logs.push(log);
update.finalize();
update_update(update).await
}
.await;
if let Err(e) = res {
warn!("failed to update update with task error log | {e:#}");
}
}
});
Ok(Json(update))
) -> serror::Result<(TypedHeader<ContentType>, String)> {
let res = match inner_handler(request, user).await? {
ExecutionResult::Single(update) => serde_json::to_string(&update)
.context("Failed to serialize Update")?,
ExecutionResult::Batch(res) => res,
};
Ok((TypedHeader(ContentType::json()), res))
}
#[instrument(name = "ExecuteRequest", skip(user, update), fields(user_id = user.id, update_id = update.id))]
pub enum ExecutionResult {
Single(Update),
/// The batch contents will be pre serialized here
Batch(String),
}
pub fn inner_handler(
request: ExecuteRequest,
user: User,
) -> Pin<
Box<
dyn std::future::Future<Output = anyhow::Result<ExecutionResult>>
+ Send,
>,
> {
Box::pin(async move {
let req_id = Uuid::new_v4();
// need to validate no cancel is active before any update is created.
build::validate_cancel_build(&request).await?;
let update = init_execution_update(&request, &user).await?;
// This will be the case for the Batch exections,
// they don't have their own updates.
// The batch calls also call "inner_handler" themselves,
// and in their case will spawn tasks, so that isn't necessary
// here either.
if update.operation == Operation::None {
return Ok(ExecutionResult::Batch(
task(req_id, request, user, update).await?,
));
}
let handle =
tokio::spawn(task(req_id, request, user, update.clone()));
tokio::spawn({
let update_id = update.id.clone();
async move {
let log = match handle.await {
Ok(Err(e)) => {
warn!("/execute request {req_id} task error: {e:#}",);
Log::error("task error", format_serror(&e.into()))
}
Err(e) => {
warn!("/execute request {req_id} spawn error: {e:?}",);
Log::error("spawn error", format!("{e:#?}"))
}
_ => return,
};
let res = async {
let mut update =
find_one_by_id(&db_client().updates, &update_id)
.await
.context("failed to query to db")?
.context("no update exists with given id")?;
update.logs.push(log);
update.finalize();
update_update(update).await
}
.await;
if let Err(e) = res {
warn!(
"failed to update update with task error log | {e:#}"
);
}
}
});
Ok(ExecutionResult::Single(update))
})
}
#[instrument(
name = "ExecuteRequest",
skip(user, update),
fields(
user_id = user.id,
update_id = update.id,
request = format!("{:?}", request.extract_variant()))
)
]
async fn task(
req_id: Uuid,
request: ExecuteRequest,
@@ -164,15 +251,14 @@ async fn task(
info!("/execute request {req_id} | user: {}", user.username);
let timer = Instant::now();
let res = State
.resolve_request(request, (user, update))
.await
.map_err(|e| match e {
resolver_api::Error::Serialization(e) => {
anyhow!("{e:?}").context("response serialization error")
}
resolver_api::Error::Inner(e) => e,
});
let res = match request.resolve(&ExecuteArgs { user, update }).await
{
Err(e) => Err(e.error),
Ok(JsonString::Err(e)) => Err(
anyhow::Error::from(e).context("failed to serialize response"),
),
Ok(JsonString::Ok(res)) => Ok(res),
};
if let Err(e) = &res {
warn!("/execute request {req_id} error: {e:#}");
@@ -183,3 +269,40 @@ async fn task(
res
}
trait BatchExecute {
type Resource: KomodoResource;
fn single_request(name: String) -> ExecuteRequest;
}
async fn batch_execute<E: BatchExecute>(
pattern: &str,
user: &User,
) -> anyhow::Result<BatchExecutionResponse> {
let resources = list_full_for_user_using_pattern::<E::Resource>(
pattern,
Default::default(),
user,
&[],
)
.await?;
let futures = resources.into_iter().map(|resource| {
let user = user.clone();
async move {
inner_handler(E::single_request(resource.name.clone()), user)
.await
.map(|r| {
let ExecutionResult::Single(update) = r else {
unreachable!()
};
update
})
.map_err(|e| BatchExecutionResponseItemErr {
name: resource.name,
error: e.into(),
})
.into()
}
});
Ok(join_all(futures).await)
}

View File

@@ -1,11 +1,17 @@
use std::pin::Pin;
use formatting::{bold, colored, format_serror, muted, Color};
use formatting::{Color, bold, colored, format_serror, muted};
use komodo_client::{
api::execute::RunProcedure,
api::execute::{
BatchExecutionResponse, BatchRunProcedure, RunProcedure,
},
entities::{
permission::PermissionLevel, procedure::Procedure,
update::Update, user::User,
alert::{Alert, AlertData, SeverityLevel},
komodo_timestamp,
permission::PermissionLevel,
procedure::Procedure,
update::Update,
user::User,
},
};
use mungos::{by_id::update_one_by_id, mongodb::bson::to_document};
@@ -13,19 +19,44 @@ use resolver_api::Resolve;
use tokio::sync::Mutex;
use crate::{
alert::send_alerts,
helpers::{procedure::execute_procedure, update::update_update},
resource::{self, refresh_procedure_state_cache},
state::{action_states, db_client, State},
state::{action_states, db_client},
};
impl Resolve<RunProcedure, (User, Update)> for State {
#[instrument(name = "RunProcedure", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
use super::{ExecuteArgs, ExecuteRequest};
impl super::BatchExecute for BatchRunProcedure {
type Resource = Procedure;
fn single_request(procedure: String) -> ExecuteRequest {
ExecuteRequest::RunProcedure(RunProcedure { procedure })
}
}
impl Resolve<ExecuteArgs> for BatchRunProcedure {
#[instrument(name = "BatchRunProcedure", skip(user), fields(user_id = user.id))]
async fn resolve(
&self,
RunProcedure { procedure }: RunProcedure,
(user, update): (User, Update),
) -> anyhow::Result<Update> {
resolve_inner(procedure, user, update).await
self,
ExecuteArgs { user, .. }: &ExecuteArgs,
) -> serror::Result<BatchExecutionResponse> {
Ok(
super::batch_execute::<BatchRunProcedure>(&self.pattern, user)
.await?,
)
}
}
impl Resolve<ExecuteArgs> for RunProcedure {
#[instrument(name = "RunProcedure", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
Ok(
resolve_inner(self.procedure, user.clone(), update.clone())
.await?,
)
}
}
@@ -50,7 +81,7 @@ fn resolve_inner(
// assumes first log is already created
// and will panic otherwise.
update.push_simple_log(
"execute_procedure",
"Execute procedure",
format!(
"{}: executing procedure '{}'",
muted("INFO"),
@@ -80,9 +111,9 @@ fn resolve_inner(
match res {
Ok(_) => {
update.push_simple_log(
"execution ok",
"Execution ok",
format!(
"{}: the procedure has {} with no errors",
"{}: The procedure has {} with no errors",
muted("INFO"),
colored("completed", Color::Green)
),
@@ -100,7 +131,7 @@ fn resolve_inner(
// but will fail to update cache in that case.
if let Ok(update_doc) = to_document(&update) {
let _ = update_one_by_id(
&db_client().await.updates,
&db_client().updates,
&update.id,
mungos::update::Update::Set(update_doc),
None,
@@ -111,6 +142,26 @@ fn resolve_inner(
update_update(update.clone()).await?;
if !update.success && procedure.config.failure_alert {
warn!("procedure unsuccessful, alerting...");
let target = update.target.clone();
tokio::spawn(async move {
let alert = Alert {
id: Default::default(),
target,
ts: komodo_timestamp(),
resolved_ts: Some(komodo_timestamp()),
resolved: true,
level: SeverityLevel::Warning,
data: AlertData::ProcedureFailed {
id: procedure.id,
name: procedure.name,
},
};
send_alerts(&[alert]).await
});
}
Ok(update)
})
}

View File

@@ -1,18 +1,17 @@
use std::{collections::HashSet, future::IntoFuture, time::Duration};
use anyhow::{anyhow, Context};
use anyhow::{Context, anyhow};
use formatting::format_serror;
use komodo_client::{
api::execute::*,
api::{execute::*, write::RefreshRepoCache},
entities::{
alert::{Alert, AlertData, SeverityLevel},
builder::{Builder, BuilderConfig},
komodo_timestamp, optional_string,
komodo_timestamp,
permission::PermissionLevel,
repo::Repo,
server::Server,
update::{Log, Update},
user::User,
},
};
use mungos::{
@@ -27,14 +26,15 @@ use resolver_api::Resolve;
use tokio_util::sync::CancellationToken;
use crate::{
alert::send_alerts,
api::write::WriteArgs,
helpers::{
alert::send_alerts,
builder::{cleanup_builder_instance, get_builder_periphery},
channel::repo_cancel_channel,
git_token,
interpolate::{
add_interp_update_log,
interpolate_variables_secrets_into_environment,
interpolate_variables_secrets_into_string,
interpolate_variables_secrets_into_system_command,
},
periphery_client,
@@ -42,21 +42,40 @@ use crate::{
update::update_update,
},
resource::{self, refresh_repo_state_cache},
state::{action_states, db_client, State},
state::{action_states, db_client},
};
use super::ExecuteRequest;
use super::{ExecuteArgs, ExecuteRequest};
impl Resolve<CloneRepo, (User, Update)> for State {
#[instrument(name = "CloneRepo", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl super::BatchExecute for BatchCloneRepo {
type Resource = Repo;
fn single_request(repo: String) -> ExecuteRequest {
ExecuteRequest::CloneRepo(CloneRepo { repo })
}
}
impl Resolve<ExecuteArgs> for BatchCloneRepo {
#[instrument(name = "BatchCloneRepo", skip( user), fields(user_id = user.id))]
async fn resolve(
&self,
CloneRepo { repo }: CloneRepo,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<BatchExecutionResponse> {
Ok(
super::batch_execute::<BatchCloneRepo>(&self.pattern, user)
.await?,
)
}
}
impl Resolve<ExecuteArgs> for CloneRepo {
#[instrument(name = "CloneRepo", skip( user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let mut repo = resource::get_check_permissions::<Repo>(
&repo,
&user,
&self.repo,
user,
PermissionLevel::Execute,
)
.await?;
@@ -70,8 +89,13 @@ impl Resolve<CloneRepo, (User, Update)> for State {
let _action_guard =
action_state.update(|state| state.cloning = true)?;
let mut update = update.clone();
update_update(update.clone()).await?;
if repo.config.server_id.is_empty() {
return Err(anyhow!("repo has no server attached").into());
}
let git_token = git_token(
&repo.config.git_provider,
&repo.config.git_account,
@@ -82,10 +106,6 @@ impl Resolve<CloneRepo, (User, Update)> for State {
|| format!("Failed to get git token in call to db. This is a database error, not a token exisitence error. Stopping run. | {} | {}", repo.config.git_provider, repo.config.git_account),
)?;
if repo.config.server_id.is_empty() {
return Err(anyhow!("repo has no server attached"));
}
let server =
resource::get::<Server>(&repo.config.server_id).await?;
@@ -100,7 +120,7 @@ impl Resolve<CloneRepo, (User, Update)> for State {
.request(api::git::CloneRepo {
args: (&repo).into(),
git_token,
environment: repo.config.environment,
environment: repo.config.env_vars()?,
env_file_path: repo.config.env_file_path,
skip_secret_interp: repo.config.skip_secret_interp,
replacers: secret_replacers.into_iter().collect(),
@@ -123,20 +143,51 @@ impl Resolve<CloneRepo, (User, Update)> for State {
update_last_pulled_time(&repo.name).await;
}
if let Err(e) = (RefreshRepoCache { repo: repo.id })
.resolve(&WriteArgs { user: user.clone() })
.await
.map_err(|e| e.error)
.context("Failed to refresh repo cache")
{
update.push_error_log(
"Refresh Repo cache",
format_serror(&e.into()),
);
};
handle_server_update_return(update).await
}
}
impl Resolve<PullRepo, (User, Update)> for State {
#[instrument(name = "PullRepo", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl super::BatchExecute for BatchPullRepo {
type Resource = Repo;
fn single_request(repo: String) -> ExecuteRequest {
ExecuteRequest::CloneRepo(CloneRepo { repo })
}
}
impl Resolve<ExecuteArgs> for BatchPullRepo {
#[instrument(name = "BatchPullRepo", skip(user), fields(user_id = user.id))]
async fn resolve(
&self,
PullRepo { repo }: PullRepo,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, .. }: &ExecuteArgs,
) -> serror::Result<BatchExecutionResponse> {
Ok(
super::batch_execute::<BatchPullRepo>(&self.pattern, user)
.await?,
)
}
}
impl Resolve<ExecuteArgs> for PullRepo {
#[instrument(name = "PullRepo", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let mut repo = resource::get_check_permissions::<Repo>(
&repo,
&user,
&self.repo,
user,
PermissionLevel::Execute,
)
.await?;
@@ -150,12 +201,24 @@ impl Resolve<PullRepo, (User, Update)> for State {
let _action_guard =
action_state.update(|state| state.pulling = true)?;
let mut update = update.clone();
update_update(update.clone()).await?;
if repo.config.server_id.is_empty() {
return Err(anyhow!("repo has no server attached"));
return Err(anyhow!("repo has no server attached").into());
}
let git_token = git_token(
&repo.config.git_provider,
&repo.config.git_account,
|https| repo.config.git_https = https,
)
.await
.with_context(
|| format!("Failed to get git token in call to db. This is a database error, not a token exisitence error. Stopping run. | {} | {}", repo.config.git_provider, repo.config.git_account),
)?;
let server =
resource::get::<Server>(&repo.config.server_id).await?;
@@ -168,12 +231,9 @@ impl Resolve<PullRepo, (User, Update)> for State {
let logs = match periphery
.request(api::git::PullRepo {
name: repo.name.clone(),
branch: optional_string(&repo.config.branch),
commit: optional_string(&repo.config.commit),
path: optional_string(&repo.config.path),
on_pull: repo.config.on_pull.into_option(),
environment: repo.config.environment,
args: (&repo).into(),
git_token,
environment: repo.config.env_vars()?,
env_file_path: repo.config.env_file_path,
skip_secret_interp: repo.config.skip_secret_interp,
replacers: secret_replacers.into_iter().collect(),
@@ -200,6 +260,18 @@ impl Resolve<PullRepo, (User, Update)> for State {
update_last_pulled_time(&repo.name).await;
}
if let Err(e) = (RefreshRepoCache { repo: repo.id })
.resolve(&WriteArgs { user: user.clone() })
.await
.map_err(|e| e.error)
.context("Failed to refresh repo cache")
{
update.push_error_log(
"Refresh Repo cache",
format_serror(&e.into()),
);
};
handle_server_update_return(update).await
}
}
@@ -207,14 +279,14 @@ impl Resolve<PullRepo, (User, Update)> for State {
#[instrument(skip_all, fields(update_id = update.id))]
async fn handle_server_update_return(
update: Update,
) -> anyhow::Result<Update> {
) -> serror::Result<Update> {
// Need to manually update the update before cache refresh,
// and before broadcast with add_update.
// The Err case of to_document should be unreachable,
// but will fail to update cache in that case.
if let Ok(update_doc) = to_document(&update) {
let _ = update_one_by_id(
&db_client().await.updates,
&db_client().updates,
&update.id,
mungos::update::Update::Set(update_doc),
None,
@@ -229,7 +301,6 @@ async fn handle_server_update_return(
#[instrument]
async fn update_last_pulled_time(repo_name: &str) {
let res = db_client()
.await
.repos
.update_one(
doc! { "name": repo_name },
@@ -243,22 +314,41 @@ async fn update_last_pulled_time(repo_name: &str) {
}
}
impl Resolve<BuildRepo, (User, Update)> for State {
#[instrument(name = "BuildRepo", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl super::BatchExecute for BatchBuildRepo {
type Resource = Repo;
fn single_request(repo: String) -> ExecuteRequest {
ExecuteRequest::CloneRepo(CloneRepo { repo })
}
}
impl Resolve<ExecuteArgs> for BatchBuildRepo {
#[instrument(name = "BatchBuildRepo", skip(user), fields(user_id = user.id))]
async fn resolve(
&self,
BuildRepo { repo }: BuildRepo,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, .. }: &ExecuteArgs,
) -> serror::Result<BatchExecutionResponse> {
Ok(
super::batch_execute::<BatchBuildRepo>(&self.pattern, user)
.await?,
)
}
}
impl Resolve<ExecuteArgs> for BuildRepo {
#[instrument(name = "BuildRepo", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let mut repo = resource::get_check_permissions::<Repo>(
&repo,
&user,
&self.repo,
user,
PermissionLevel::Execute,
)
.await?;
if repo.config.builder_id.is_empty() {
return Err(anyhow!("Must attach builder to BuildRepo"));
return Err(anyhow!("Must attach builder to BuildRepo").into());
}
// get the action state for the repo (or insert default).
@@ -270,6 +360,7 @@ impl Resolve<BuildRepo, (User, Update)> for State {
let _action_guard =
action_state.update(|state| state.building = true)?;
let mut update = update.clone();
update_update(update.clone()).await?;
let git_token = git_token(
@@ -363,7 +454,7 @@ impl Resolve<BuildRepo, (User, Update)> for State {
.request(api::git::CloneRepo {
args: (&repo).into(),
git_token,
environment: repo.config.environment,
environment: repo.config.env_vars()?,
env_file_path: repo.config.env_file_path,
skip_secret_interp: repo.config.skip_secret_interp,
replacers: secret_replacers.into_iter().collect()
@@ -371,7 +462,7 @@ impl Resolve<BuildRepo, (User, Update)> for State {
_ = cancel.cancelled() => {
debug!("build cancelled during clone, cleaning up builder");
update.push_error_log("build cancelled", String::from("user cancelled build during repo clone"));
cleanup_builder_instance(periphery, cleanup_data, &mut update)
cleanup_builder_instance(cleanup_data, &mut update)
.await;
info!("builder cleaned up");
return handle_builder_early_return(update, repo.id, repo.name, true).await
@@ -396,7 +487,7 @@ impl Resolve<BuildRepo, (User, Update)> for State {
update.finalize();
let db = db_client().await;
let db = db_client();
if update.success {
let _ = db
@@ -415,8 +506,9 @@ impl Resolve<BuildRepo, (User, Update)> for State {
// stop the cancel listening task from going forever
cancel.cancel();
cleanup_builder_instance(periphery, cleanup_data, &mut update)
.await;
// If building on temporary cloud server (AWS),
// this will terminate the server.
cleanup_builder_instance(cleanup_data, &mut update).await;
// Need to manually update the update before cache refresh,
// and before broadcast with add_update.
@@ -465,7 +557,7 @@ async fn handle_builder_early_return(
repo_id: String,
repo_name: String,
is_cancel: bool,
) -> anyhow::Result<Update> {
) -> serror::Result<Update> {
update.finalize();
// Need to manually update the update before cache refresh,
// and before broadcast with add_update.
@@ -473,7 +565,7 @@ async fn handle_builder_early_return(
// but will fail to update cache in that case.
if let Ok(update_doc) = to_document(&update) {
let _ = update_one_by_id(
&db_client().await.updates,
&db_client().updates,
&update.id,
mungos::update::Update::Set(update_doc),
None,
@@ -511,7 +603,7 @@ pub async fn validate_cancel_repo_build(
if let ExecuteRequest::CancelRepoBuild(req) = request {
let repo = resource::get::<Repo>(&req.repo).await?;
let db = db_client().await;
let db = db_client();
let (latest_build, latest_cancel) = tokio::try_join!(
db.updates
@@ -553,16 +645,15 @@ pub async fn validate_cancel_repo_build(
Ok(())
}
impl Resolve<CancelRepoBuild, (User, Update)> for State {
#[instrument(name = "CancelRepoBuild", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for CancelRepoBuild {
#[instrument(name = "CancelRepoBuild", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
CancelRepoBuild { repo }: CancelRepoBuild,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let repo = resource::get_check_permissions::<Repo>(
&repo,
&user,
&self.repo,
user,
PermissionLevel::Execute,
)
.await?;
@@ -575,9 +666,11 @@ impl Resolve<CancelRepoBuild, (User, Update)> for State {
.and_then(|s| s.get().ok().map(|s| s.building))
.unwrap_or_default()
{
return Err(anyhow!("Repo is not building."));
return Err(anyhow!("Repo is not building.").into());
}
let mut update = update.clone();
update.push_simple_log(
"cancel triggered",
"the repo build cancel has been triggered",
@@ -596,14 +689,16 @@ impl Resolve<CancelRepoBuild, (User, Update)> for State {
tokio::spawn(async move {
tokio::time::sleep(Duration::from_secs(60)).await;
if let Err(e) = update_one_by_id(
&db_client().await.updates,
&db_client().updates,
&update_id,
doc! { "$set": { "status": "Complete" } },
None,
)
.await
{
warn!("failed to set CancelRepoBuild Update status Complete after timeout | {e:#}")
warn!(
"failed to set CancelRepoBuild Update status Complete after timeout | {e:#}"
)
}
});
@@ -621,7 +716,7 @@ async fn interpolate(
let mut global_replacers = HashSet::new();
let mut secret_replacers = HashSet::new();
interpolate_variables_secrets_into_environment(
interpolate_variables_secrets_into_string(
&vars_and_secrets,
&mut repo.config.environment,
&mut global_replacers,

View File

@@ -7,7 +7,6 @@ use komodo_client::{
permission::PermissionLevel,
server::Server,
update::{Log, Update},
user::User,
},
};
use periphery_client::api;
@@ -17,19 +16,20 @@ use crate::{
helpers::{periphery_client, update::update_update},
monitor::update_cache_for_server,
resource,
state::{action_states, State},
state::action_states,
};
impl Resolve<StartContainer, (User, Update)> for State {
use super::ExecuteArgs;
impl Resolve<ExecuteArgs> for StartContainer {
#[instrument(name = "StartContainer", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
StartContainer { server, container }: StartContainer,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let server = resource::get_check_permissions::<Server>(
&server,
&user,
&self.server,
user,
PermissionLevel::Execute,
)
.await?;
@@ -45,13 +45,17 @@ impl Resolve<StartContainer, (User, Update)> for State {
let _action_guard = action_state
.update(|state| state.starting_containers = true)?;
let mut update = update.clone();
// Send update after setting action state, this way frontend gets correct state.
update_update(update.clone()).await?;
let periphery = periphery_client(&server)?;
let log = match periphery
.request(api::container::StartContainer { name: container })
.request(api::container::StartContainer {
name: self.container,
})
.await
{
Ok(log) => log,
@@ -71,16 +75,15 @@ impl Resolve<StartContainer, (User, Update)> for State {
}
}
impl Resolve<RestartContainer, (User, Update)> for State {
impl Resolve<ExecuteArgs> for RestartContainer {
#[instrument(name = "RestartContainer", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
RestartContainer { server, container }: RestartContainer,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let server = resource::get_check_permissions::<Server>(
&server,
&user,
&self.server,
user,
PermissionLevel::Execute,
)
.await?;
@@ -96,13 +99,17 @@ impl Resolve<RestartContainer, (User, Update)> for State {
let _action_guard = action_state
.update(|state| state.restarting_containers = true)?;
let mut update = update.clone();
// Send update after setting action state, this way frontend gets correct state.
update_update(update.clone()).await?;
let periphery = periphery_client(&server)?;
let log = match periphery
.request(api::container::RestartContainer { name: container })
.request(api::container::RestartContainer {
name: self.container,
})
.await
{
Ok(log) => log,
@@ -124,16 +131,15 @@ impl Resolve<RestartContainer, (User, Update)> for State {
}
}
impl Resolve<PauseContainer, (User, Update)> for State {
#[instrument(name = "PauseContainer", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for PauseContainer {
#[instrument(name = "PauseContainer", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
PauseContainer { server, container }: PauseContainer,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let server = resource::get_check_permissions::<Server>(
&server,
&user,
&self.server,
user,
PermissionLevel::Execute,
)
.await?;
@@ -149,13 +155,17 @@ impl Resolve<PauseContainer, (User, Update)> for State {
let _action_guard =
action_state.update(|state| state.pausing_containers = true)?;
let mut update = update.clone();
// Send update after setting action state, this way frontend gets correct state.
update_update(update.clone()).await?;
let periphery = periphery_client(&server)?;
let log = match periphery
.request(api::container::PauseContainer { name: container })
.request(api::container::PauseContainer {
name: self.container,
})
.await
{
Ok(log) => log,
@@ -175,16 +185,15 @@ impl Resolve<PauseContainer, (User, Update)> for State {
}
}
impl Resolve<UnpauseContainer, (User, Update)> for State {
#[instrument(name = "UnpauseContainer", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for UnpauseContainer {
#[instrument(name = "UnpauseContainer", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
UnpauseContainer { server, container }: UnpauseContainer,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let server = resource::get_check_permissions::<Server>(
&server,
&user,
&self.server,
user,
PermissionLevel::Execute,
)
.await?;
@@ -200,13 +209,17 @@ impl Resolve<UnpauseContainer, (User, Update)> for State {
let _action_guard = action_state
.update(|state| state.unpausing_containers = true)?;
let mut update = update.clone();
// Send update after setting action state, this way frontend gets correct state.
update_update(update.clone()).await?;
let periphery = periphery_client(&server)?;
let log = match periphery
.request(api::container::UnpauseContainer { name: container })
.request(api::container::UnpauseContainer {
name: self.container,
})
.await
{
Ok(log) => log,
@@ -228,21 +241,15 @@ impl Resolve<UnpauseContainer, (User, Update)> for State {
}
}
impl Resolve<StopContainer, (User, Update)> for State {
#[instrument(name = "StopContainer", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for StopContainer {
#[instrument(name = "StopContainer", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
StopContainer {
server,
container,
signal,
time,
}: StopContainer,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let server = resource::get_check_permissions::<Server>(
&server,
&user,
&self.server,
user,
PermissionLevel::Execute,
)
.await?;
@@ -258,6 +265,8 @@ impl Resolve<StopContainer, (User, Update)> for State {
let _action_guard = action_state
.update(|state| state.stopping_containers = true)?;
let mut update = update.clone();
// Send update after setting action state, this way frontend gets correct state.
update_update(update.clone()).await?;
@@ -265,9 +274,9 @@ impl Resolve<StopContainer, (User, Update)> for State {
let log = match periphery
.request(api::container::StopContainer {
name: container,
signal,
time,
name: self.container,
signal: self.signal,
time: self.time,
})
.await
{
@@ -288,21 +297,21 @@ impl Resolve<StopContainer, (User, Update)> for State {
}
}
impl Resolve<DestroyContainer, (User, Update)> for State {
#[instrument(name = "DestroyContainer", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for DestroyContainer {
#[instrument(name = "DestroyContainer", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
DestroyContainer {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let DestroyContainer {
server,
container,
signal,
time,
}: DestroyContainer,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
} = self;
let server = resource::get_check_permissions::<Server>(
&server,
&user,
user,
PermissionLevel::Execute,
)
.await?;
@@ -318,6 +327,8 @@ impl Resolve<DestroyContainer, (User, Update)> for State {
let _action_guard =
action_state.update(|state| state.pruning_containers = true)?;
let mut update = update.clone();
// Send update after setting action state, this way frontend gets correct state.
update_update(update.clone()).await?;
@@ -348,16 +359,15 @@ impl Resolve<DestroyContainer, (User, Update)> for State {
}
}
impl Resolve<StartAllContainers, (User, Update)> for State {
#[instrument(name = "StartAllContainers", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for StartAllContainers {
#[instrument(name = "StartAllContainers", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
StartAllContainers { server }: StartAllContainers,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let server = resource::get_check_permissions::<Server>(
&server,
&user,
&self.server,
user,
PermissionLevel::Execute,
)
.await?;
@@ -373,6 +383,8 @@ impl Resolve<StartAllContainers, (User, Update)> for State {
let _action_guard = action_state
.update(|state| state.starting_containers = true)?;
let mut update = update.clone();
update_update(update.clone()).await?;
let logs = periphery_client(&server)?
@@ -397,16 +409,15 @@ impl Resolve<StartAllContainers, (User, Update)> for State {
}
}
impl Resolve<RestartAllContainers, (User, Update)> for State {
#[instrument(name = "RestartAllContainers", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for RestartAllContainers {
#[instrument(name = "RestartAllContainers", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
RestartAllContainers { server }: RestartAllContainers,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let server = resource::get_check_permissions::<Server>(
&server,
&user,
&self.server,
user,
PermissionLevel::Execute,
)
.await?;
@@ -422,10 +433,12 @@ impl Resolve<RestartAllContainers, (User, Update)> for State {
let _action_guard = action_state
.update(|state| state.restarting_containers = true)?;
let mut update = update.clone();
update_update(update.clone()).await?;
let logs = periphery_client(&server)?
.request(api::container::StartAllContainers {})
.request(api::container::RestartAllContainers {})
.await
.context("failed to restart all containers on host")?;
@@ -448,16 +461,15 @@ impl Resolve<RestartAllContainers, (User, Update)> for State {
}
}
impl Resolve<PauseAllContainers, (User, Update)> for State {
#[instrument(name = "PauseAllContainers", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for PauseAllContainers {
#[instrument(name = "PauseAllContainers", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
PauseAllContainers { server }: PauseAllContainers,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let server = resource::get_check_permissions::<Server>(
&server,
&user,
&self.server,
user,
PermissionLevel::Execute,
)
.await?;
@@ -473,6 +485,8 @@ impl Resolve<PauseAllContainers, (User, Update)> for State {
let _action_guard =
action_state.update(|state| state.pausing_containers = true)?;
let mut update = update.clone();
update_update(update.clone()).await?;
let logs = periphery_client(&server)?
@@ -497,16 +511,15 @@ impl Resolve<PauseAllContainers, (User, Update)> for State {
}
}
impl Resolve<UnpauseAllContainers, (User, Update)> for State {
#[instrument(name = "UnpauseAllContainers", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for UnpauseAllContainers {
#[instrument(name = "UnpauseAllContainers", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
UnpauseAllContainers { server }: UnpauseAllContainers,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let server = resource::get_check_permissions::<Server>(
&server,
&user,
&self.server,
user,
PermissionLevel::Execute,
)
.await?;
@@ -520,12 +533,14 @@ impl Resolve<UnpauseAllContainers, (User, Update)> for State {
// Will check to ensure server not already busy before updating, and return Err if so.
// The returned guard will set the action state back to default when dropped.
let _action_guard = action_state
.update(|state| state.starting_containers = true)?;
.update(|state| state.unpausing_containers = true)?;
let mut update = update.clone();
update_update(update.clone()).await?;
let logs = periphery_client(&server)?
.request(api::container::StartAllContainers {})
.request(api::container::UnpauseAllContainers {})
.await
.context("failed to unpause all containers on host")?;
@@ -548,16 +563,15 @@ impl Resolve<UnpauseAllContainers, (User, Update)> for State {
}
}
impl Resolve<StopAllContainers, (User, Update)> for State {
#[instrument(name = "StopAllContainers", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for StopAllContainers {
#[instrument(name = "StopAllContainers", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
StopAllContainers { server }: StopAllContainers,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let server = resource::get_check_permissions::<Server>(
&server,
&user,
&self.server,
user,
PermissionLevel::Execute,
)
.await?;
@@ -573,6 +587,8 @@ impl Resolve<StopAllContainers, (User, Update)> for State {
let _action_guard = action_state
.update(|state| state.stopping_containers = true)?;
let mut update = update.clone();
update_update(update.clone()).await?;
let logs = periphery_client(&server)?
@@ -597,16 +613,15 @@ impl Resolve<StopAllContainers, (User, Update)> for State {
}
}
impl Resolve<PruneContainers, (User, Update)> for State {
#[instrument(name = "PruneContainers", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for PruneContainers {
#[instrument(name = "PruneContainers", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
PruneContainers { server }: PruneContainers,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let server = resource::get_check_permissions::<Server>(
&server,
&user,
&self.server,
user,
PermissionLevel::Execute,
)
.await?;
@@ -622,6 +637,8 @@ impl Resolve<PruneContainers, (User, Update)> for State {
let _action_guard =
action_state.update(|state| state.pruning_containers = true)?;
let mut update = update.clone();
update_update(update.clone()).await?;
let periphery = periphery_client(&server)?;
@@ -652,37 +669,43 @@ impl Resolve<PruneContainers, (User, Update)> for State {
}
}
impl Resolve<DeleteNetwork, (User, Update)> for State {
#[instrument(name = "DeleteNetwork", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for DeleteNetwork {
#[instrument(name = "DeleteNetwork", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
DeleteNetwork { server, name }: DeleteNetwork,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let server = resource::get_check_permissions::<Server>(
&server,
&user,
&self.server,
user,
PermissionLevel::Execute,
)
.await?;
let mut update = update.clone();
update_update(update.clone()).await?;
let periphery = periphery_client(&server)?;
let log = match periphery
.request(api::network::DeleteNetwork { name: name.clone() })
.request(api::network::DeleteNetwork {
name: self.name.clone(),
})
.await
.context(format!(
"failed to delete network {name} on server {}",
server.name
"failed to delete network {} on server {}",
self.name, server.name
)) {
Ok(log) => log,
Err(e) => Log::error(
"delete network",
format_serror(
&e.context(format!("failed to delete network {name}"))
.into(),
&e.context(format!(
"failed to delete network {}",
self.name
))
.into(),
),
),
};
@@ -697,16 +720,15 @@ impl Resolve<DeleteNetwork, (User, Update)> for State {
}
}
impl Resolve<PruneNetworks, (User, Update)> for State {
#[instrument(name = "PruneNetworks", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for PruneNetworks {
#[instrument(name = "PruneNetworks", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
PruneNetworks { server }: PruneNetworks,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let server = resource::get_check_permissions::<Server>(
&server,
&user,
&self.server,
user,
PermissionLevel::Execute,
)
.await?;
@@ -722,6 +744,8 @@ impl Resolve<PruneNetworks, (User, Update)> for State {
let _action_guard =
action_state.update(|state| state.pruning_networks = true)?;
let mut update = update.clone();
update_update(update.clone()).await?;
let periphery = periphery_client(&server)?;
@@ -750,36 +774,40 @@ impl Resolve<PruneNetworks, (User, Update)> for State {
}
}
impl Resolve<DeleteImage, (User, Update)> for State {
#[instrument(name = "DeleteImage", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for DeleteImage {
#[instrument(name = "DeleteImage", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
DeleteImage { server, name }: DeleteImage,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let server = resource::get_check_permissions::<Server>(
&server,
&user,
&self.server,
user,
PermissionLevel::Execute,
)
.await?;
let mut update = update.clone();
update_update(update.clone()).await?;
let periphery = periphery_client(&server)?;
let log = match periphery
.request(api::image::DeleteImage { name: name.clone() })
.request(api::image::DeleteImage {
name: self.name.clone(),
})
.await
.context(format!(
"failed to delete image {name} on server {}",
server.name
"failed to delete image {} on server {}",
self.name, server.name
)) {
Ok(log) => log,
Err(e) => Log::error(
"delete image",
format_serror(
&e.context(format!("failed to delete image {name}")).into(),
&e.context(format!("failed to delete image {}", self.name))
.into(),
),
),
};
@@ -794,16 +822,15 @@ impl Resolve<DeleteImage, (User, Update)> for State {
}
}
impl Resolve<PruneImages, (User, Update)> for State {
#[instrument(name = "PruneImages", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for PruneImages {
#[instrument(name = "PruneImages", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
PruneImages { server }: PruneImages,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let server = resource::get_check_permissions::<Server>(
&server,
&user,
&self.server,
user,
PermissionLevel::Execute,
)
.await?;
@@ -819,6 +846,8 @@ impl Resolve<PruneImages, (User, Update)> for State {
let _action_guard =
action_state.update(|state| state.pruning_images = true)?;
let mut update = update.clone();
update_update(update.clone()).await?;
let periphery = periphery_client(&server)?;
@@ -845,37 +874,43 @@ impl Resolve<PruneImages, (User, Update)> for State {
}
}
impl Resolve<DeleteVolume, (User, Update)> for State {
#[instrument(name = "DeleteVolume", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for DeleteVolume {
#[instrument(name = "DeleteVolume", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
DeleteVolume { server, name }: DeleteVolume,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let server = resource::get_check_permissions::<Server>(
&server,
&user,
&self.server,
user,
PermissionLevel::Execute,
)
.await?;
let mut update = update.clone();
update_update(update.clone()).await?;
let periphery = periphery_client(&server)?;
let log = match periphery
.request(api::volume::DeleteVolume { name: name.clone() })
.request(api::volume::DeleteVolume {
name: self.name.clone(),
})
.await
.context(format!(
"failed to delete volume {name} on server {}",
server.name
"failed to delete volume {} on server {}",
self.name, server.name
)) {
Ok(log) => log,
Err(e) => Log::error(
"delete volume",
format_serror(
&e.context(format!("failed to delete volume {name}"))
.into(),
&e.context(format!(
"failed to delete volume {}",
self.name
))
.into(),
),
),
};
@@ -890,16 +925,15 @@ impl Resolve<DeleteVolume, (User, Update)> for State {
}
}
impl Resolve<PruneVolumes, (User, Update)> for State {
#[instrument(name = "PruneVolumes", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for PruneVolumes {
#[instrument(name = "PruneVolumes", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
PruneVolumes { server }: PruneVolumes,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let server = resource::get_check_permissions::<Server>(
&server,
&user,
&self.server,
user,
PermissionLevel::Execute,
)
.await?;
@@ -915,6 +949,8 @@ impl Resolve<PruneVolumes, (User, Update)> for State {
let _action_guard =
action_state.update(|state| state.pruning_volumes = true)?;
let mut update = update.clone();
update_update(update.clone()).await?;
let periphery = periphery_client(&server)?;
@@ -941,16 +977,15 @@ impl Resolve<PruneVolumes, (User, Update)> for State {
}
}
impl Resolve<PruneDockerBuilders, (User, Update)> for State {
#[instrument(name = "PruneDockerBuilders", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for PruneDockerBuilders {
#[instrument(name = "PruneDockerBuilders", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
PruneDockerBuilders { server }: PruneDockerBuilders,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let server = resource::get_check_permissions::<Server>(
&server,
&user,
&self.server,
user,
PermissionLevel::Execute,
)
.await?;
@@ -966,20 +1001,23 @@ impl Resolve<PruneDockerBuilders, (User, Update)> for State {
let _action_guard =
action_state.update(|state| state.pruning_builders = true)?;
let mut update = update.clone();
update_update(update.clone()).await?;
let periphery = periphery_client(&server)?;
let log = match periphery.request(api::build::PruneBuilders {}).await {
Ok(log) => log,
Err(e) => Log::error(
"prune builders",
format!(
"failed to docker builder prune on server {} | {e:#?}",
server.name
let log =
match periphery.request(api::build::PruneBuilders {}).await {
Ok(log) => log,
Err(e) => Log::error(
"prune builders",
format!(
"failed to docker builder prune on server {} | {e:#?}",
server.name
),
),
),
};
};
update.logs.push(log);
update_cache_for_server(&server).await;
@@ -991,16 +1029,15 @@ impl Resolve<PruneDockerBuilders, (User, Update)> for State {
}
}
impl Resolve<PruneBuildx, (User, Update)> for State {
#[instrument(name = "PruneBuildx", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for PruneBuildx {
#[instrument(name = "PruneBuildx", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
PruneBuildx { server }: PruneBuildx,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let server = resource::get_check_permissions::<Server>(
&server,
&user,
&self.server,
user,
PermissionLevel::Execute,
)
.await?;
@@ -1016,20 +1053,23 @@ impl Resolve<PruneBuildx, (User, Update)> for State {
let _action_guard =
action_state.update(|state| state.pruning_buildx = true)?;
let mut update = update.clone();
update_update(update.clone()).await?;
let periphery = periphery_client(&server)?;
let log = match periphery.request(api::build::PruneBuildx {}).await {
Ok(log) => log,
Err(e) => Log::error(
"prune buildx",
format!(
"failed to docker buildx prune on server {} | {e:#?}",
server.name
let log =
match periphery.request(api::build::PruneBuildx {}).await {
Ok(log) => log,
Err(e) => Log::error(
"prune buildx",
format!(
"failed to docker buildx prune on server {} | {e:#?}",
server.name
),
),
),
};
};
update.logs.push(log);
update_cache_for_server(&server).await;
@@ -1041,16 +1081,15 @@ impl Resolve<PruneBuildx, (User, Update)> for State {
}
}
impl Resolve<PruneSystem, (User, Update)> for State {
#[instrument(name = "PruneSystem", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for PruneSystem {
#[instrument(name = "PruneSystem", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
PruneSystem { server }: PruneSystem,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let server = resource::get_check_permissions::<Server>(
&server,
&user,
&self.server,
user,
PermissionLevel::Execute,
)
.await?;
@@ -1066,6 +1105,8 @@ impl Resolve<PruneSystem, (User, Update)> for State {
let _action_guard =
action_state.update(|state| state.pruning_system = true)?;
let mut update = update.clone();
update_update(update.clone()).await?;
let periphery = periphery_client(&server)?;

View File

@@ -1,4 +1,4 @@
use anyhow::{anyhow, Context};
use anyhow::{Context, anyhow};
use formatting::format_serror;
use komodo_client::{
api::{execute::LaunchServer, write::CreateServer},
@@ -7,52 +7,51 @@ use komodo_client::{
server::PartialServerConfig,
server_template::{ServerTemplate, ServerTemplateConfig},
update::Update,
user::User,
},
};
use mungos::mongodb::bson::doc;
use resolver_api::Resolve;
use crate::{
api::write::WriteArgs,
cloud::{
aws::ec2::launch_ec2_instance, hetzner::launch_hetzner_server,
},
helpers::update::update_update,
resource,
state::{db_client, State},
state::db_client,
};
impl Resolve<LaunchServer, (User, Update)> for State {
#[instrument(name = "LaunchServer", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
use super::ExecuteArgs;
impl Resolve<ExecuteArgs> for LaunchServer {
#[instrument(name = "LaunchServer", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
LaunchServer {
name,
server_template,
}: LaunchServer,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
// validate name isn't already taken by another server
if db_client()
.await
.servers
.find_one(doc! {
"name": &name
"name": &self.name
})
.await
.context("failed to query db for servers")?
.is_some()
{
return Err(anyhow!("name is already taken"));
return Err(anyhow!("name is already taken").into());
}
let template = resource::get_check_permissions::<ServerTemplate>(
&server_template,
&user,
&self.server_template,
user,
PermissionLevel::Execute,
)
.await?;
let mut update = update.clone();
update.push_simple_log(
"launching server",
format!("{:#?}", template.config),
@@ -62,63 +61,77 @@ impl Resolve<LaunchServer, (User, Update)> for State {
let config = match template.config {
ServerTemplateConfig::Aws(config) => {
let region = config.region.clone();
let instance = match launch_ec2_instance(&name, config).await
{
Ok(instance) => instance,
Err(e) => {
update.push_error_log(
"launch server",
format!("failed to launch aws instance\n\n{e:#?}"),
);
update.finalize();
update_update(update.clone()).await?;
return Ok(update);
}
};
let use_https = config.use_https;
let port = config.port;
let instance =
match launch_ec2_instance(&self.name, config).await {
Ok(instance) => instance,
Err(e) => {
update.push_error_log(
"launch server",
format!("failed to launch aws instance\n\n{e:#?}"),
);
update.finalize();
update_update(update.clone()).await?;
return Ok(update);
}
};
update.push_simple_log(
"launch server",
format!(
"successfully launched server {name} on ip {}",
instance.ip
"successfully launched server {} on ip {}",
self.name, instance.ip
),
);
let protocol = if use_https { "https" } else { "http" };
PartialServerConfig {
address: format!("http://{}:8120", instance.ip).into(),
address: format!("{protocol}://{}:{port}", instance.ip)
.into(),
region: region.into(),
..Default::default()
}
}
ServerTemplateConfig::Hetzner(config) => {
let datacenter = config.datacenter;
let server = match launch_hetzner_server(&name, config).await
{
Ok(server) => server,
Err(e) => {
update.push_error_log(
"launch server",
format!("failed to launch hetzner server\n\n{e:#?}"),
);
update.finalize();
update_update(update.clone()).await?;
return Ok(update);
}
};
let use_https = config.use_https;
let port = config.port;
let server =
match launch_hetzner_server(&self.name, config).await {
Ok(server) => server,
Err(e) => {
update.push_error_log(
"launch server",
format!("failed to launch hetzner server\n\n{e:#?}"),
);
update.finalize();
update_update(update.clone()).await?;
return Ok(update);
}
};
update.push_simple_log(
"launch server",
format!(
"successfully launched server {name} on ip {}",
server.ip
"successfully launched server {} on ip {}",
self.name, server.ip
),
);
let protocol = if use_https { "https" } else { "http" };
PartialServerConfig {
address: format!("http://{}:8120", server.ip).into(),
address: format!("{protocol}://{}:{port}", server.ip)
.into(),
region: datacenter.as_ref().to_string().into(),
..Default::default()
}
}
};
match self.resolve(CreateServer { name, config }, user).await {
match (CreateServer {
name: self.name,
config,
})
.resolve(&WriteArgs { user: user.clone() })
.await
{
Ok(server) => {
update.push_simple_log(
"create server",
@@ -129,7 +142,9 @@ impl Resolve<LaunchServer, (User, Update)> for State {
Err(e) => {
update.push_error_log(
"create server",
format_serror(&e.context("failed to create server").into()),
format_serror(
&e.error.context("failed to create server").into(),
),
);
}
};

View File

@@ -3,10 +3,12 @@ use std::collections::HashSet;
use anyhow::Context;
use formatting::format_serror;
use komodo_client::{
api::execute::*,
api::{execute::*, write::RefreshStackCache},
entities::{
permission::PermissionLevel, stack::StackInfo, update::Update,
user::User,
permission::PermissionLevel,
server::Server,
stack::{Stack, StackInfo},
update::{Log, Update},
},
};
use mungos::mongodb::bson::{doc, to_document};
@@ -14,34 +16,59 @@ use periphery_client::api::compose::*;
use resolver_api::Resolve;
use crate::{
api::write::WriteArgs,
helpers::{
interpolate::{
add_interp_update_log,
interpolate_variables_secrets_into_environment,
interpolate_variables_secrets_into_extra_args,
interpolate_variables_secrets_into_string,
interpolate_variables_secrets_into_system_command,
},
periphery_client,
query::get_variables_and_secrets,
stack::{
execute::execute_compose, get_stack_and_server,
services::extract_services_into_res,
},
update::update_update,
update::{add_update_without_send, update_update},
},
monitor::update_cache_for_server,
state::{action_states, db_client, State},
resource,
stack::{execute::execute_compose, get_stack_and_server},
state::{action_states, db_client},
};
impl Resolve<DeployStack, (User, Update)> for State {
#[instrument(name = "DeployStack", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
use super::{ExecuteArgs, ExecuteRequest};
impl super::BatchExecute for BatchDeployStack {
type Resource = Stack;
fn single_request(stack: String) -> ExecuteRequest {
ExecuteRequest::DeployStack(DeployStack {
stack,
services: Vec::new(),
stop_time: None,
})
}
}
impl Resolve<ExecuteArgs> for BatchDeployStack {
#[instrument(name = "BatchDeployStack", skip(user), fields(user_id = user.id))]
async fn resolve(
&self,
DeployStack { stack, stop_time }: DeployStack,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, .. }: &ExecuteArgs,
) -> serror::Result<BatchExecutionResponse> {
Ok(
super::batch_execute::<BatchDeployStack>(&self.pattern, user)
.await?,
)
}
}
impl Resolve<ExecuteArgs> for DeployStack {
#[instrument(name = "DeployStack", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let (mut stack, server) = get_stack_and_server(
&stack,
&user,
&self.stack,
user,
PermissionLevel::Execute,
true,
)
@@ -56,8 +83,20 @@ impl Resolve<DeployStack, (User, Update)> for State {
let _action_guard =
action_state.update(|state| state.deploying = true)?;
let mut update = update.clone();
update_update(update.clone()).await?;
if !self.services.is_empty() {
update.logs.push(Log::simple(
"Service/s",
format!(
"Execution requested for Stack service/s {}",
self.services.join(", ")
),
))
}
let git_token = crate::helpers::git_token(
&stack.config.git_provider,
&stack.config.git_account,
@@ -81,7 +120,14 @@ impl Resolve<DeployStack, (User, Update)> for State {
let mut global_replacers = HashSet::new();
let mut secret_replacers = HashSet::new();
interpolate_variables_secrets_into_environment(
interpolate_variables_secrets_into_string(
&vars_and_secrets,
&mut stack.config.file_contents,
&mut global_replacers,
&mut secret_replacers,
)?;
interpolate_variables_secrets_into_string(
&vars_and_secrets,
&mut stack.config.environment,
&mut global_replacers,
@@ -102,6 +148,20 @@ impl Resolve<DeployStack, (User, Update)> for State {
&mut secret_replacers,
)?;
interpolate_variables_secrets_into_system_command(
&vars_and_secrets,
&mut stack.config.pre_deploy,
&mut global_replacers,
&mut secret_replacers,
)?;
interpolate_variables_secrets_into_system_command(
&vars_and_secrets,
&mut stack.config.post_deploy,
&mut global_replacers,
&mut secret_replacers,
)?;
add_interp_update_log(
&mut update,
&global_replacers,
@@ -116,15 +176,17 @@ impl Resolve<DeployStack, (User, Update)> for State {
let ComposeUpResponse {
logs,
deployed,
services,
file_contents,
missing_files,
remote_errors,
compose_config,
commit_hash,
commit_message,
} = periphery_client(&server)?
.request(ComposeUp {
stack: stack.clone(),
service: None,
services: self.services,
git_token,
registry_token,
replacers: secret_replacers.into_iter().collect(),
@@ -134,37 +196,28 @@ impl Resolve<DeployStack, (User, Update)> for State {
update.logs.extend(logs);
let update_info = async {
let latest_services = if !file_contents.is_empty() {
let mut services = Vec::new();
for contents in &file_contents {
if let Err(e) = extract_services_into_res(
&stack.project_name(true),
&contents.contents,
&mut services,
) {
update.push_error_log(
"extract services",
format_serror(&e.context(format!("Failed to extract stack services for compose file path {}. Things probably won't work correctly", contents.path)).into())
);
}
}
services
} else {
let latest_services = if services.is_empty() {
// maybe better to do something else here for services.
stack.info.latest_services.clone()
} else {
services
};
// This ensures to get the latest project name,
// as it may have changed since the last deploy.
let project_name = stack.project_name(true);
let (
deployed_services,
deployed_contents,
deployed_config,
deployed_hash,
deployed_message,
) = if deployed {
(
Some(latest_services.clone()),
Some(file_contents.clone()),
compose_config,
commit_hash.clone(),
commit_message.clone(),
)
@@ -172,6 +225,7 @@ impl Resolve<DeployStack, (User, Update)> for State {
(
stack.info.deployed_services,
stack.info.deployed_contents,
stack.info.deployed_config,
stack.info.deployed_hash,
stack.info.deployed_message,
)
@@ -182,6 +236,7 @@ impl Resolve<DeployStack, (User, Update)> for State {
deployed_project_name: project_name.into(),
deployed_services,
deployed_contents,
deployed_config,
deployed_hash,
deployed_message,
latest_services,
@@ -203,7 +258,6 @@ impl Resolve<DeployStack, (User, Update)> for State {
.context("failed to serialize stack info to bson")?;
db_client()
.await
.stacks
.update_one(
doc! { "name": &stack.name },
@@ -234,126 +288,359 @@ impl Resolve<DeployStack, (User, Update)> for State {
}
}
impl Resolve<StartStack, (User, Update)> for State {
#[instrument(name = "StartStack", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl super::BatchExecute for BatchDeployStackIfChanged {
type Resource = Stack;
fn single_request(stack: String) -> ExecuteRequest {
ExecuteRequest::DeployStackIfChanged(DeployStackIfChanged {
stack,
stop_time: None,
})
}
}
impl Resolve<ExecuteArgs> for BatchDeployStackIfChanged {
#[instrument(name = "BatchDeployStackIfChanged", skip(user), fields(user_id = user.id))]
async fn resolve(
&self,
StartStack { stack, service }: StartStack,
(user, update): (User, Update),
) -> anyhow::Result<Update> {
execute_compose::<StartStack>(
&stack,
service,
&user,
|state| state.starting = true,
update,
(),
self,
ExecuteArgs { user, .. }: &ExecuteArgs,
) -> serror::Result<BatchExecutionResponse> {
Ok(
super::batch_execute::<BatchDeployStackIfChanged>(
&self.pattern,
user,
)
.await?,
)
}
}
impl Resolve<ExecuteArgs> for DeployStackIfChanged {
#[instrument(name = "DeployStackIfChanged", skip(user, update), fields(user_id = user.id))]
async fn resolve(
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let stack = resource::get_check_permissions::<Stack>(
&self.stack,
user,
PermissionLevel::Execute,
)
.await?;
RefreshStackCache {
stack: stack.id.clone(),
}
.resolve(&WriteArgs { user: user.clone() })
.await?;
let stack = resource::get::<Stack>(&stack.id).await?;
let changed = match (
&stack.info.deployed_contents,
&stack.info.remote_contents,
) {
(Some(deployed_contents), Some(latest_contents)) => {
let changed = || {
for latest in latest_contents {
let Some(deployed) = deployed_contents
.iter()
.find(|c| c.path == latest.path)
else {
return true;
};
if latest.contents != deployed.contents {
return true;
}
}
false
};
changed()
}
(None, _) => true,
_ => false,
};
let mut update = update.clone();
if !changed {
update.push_simple_log(
"Diff compose files",
String::from("Deploy cancelled after no changes detected."),
);
update.finalize();
return Ok(update);
}
// Don't actually send it here, let the handler send it after it can set action state.
// This is usually done in crate::helpers::update::init_execution_update.
update.id = add_update_without_send(&update).await?;
DeployStack {
stack: stack.name,
services: Vec::new(),
stop_time: self.stop_time,
}
.resolve(&ExecuteArgs {
user: user.clone(),
update,
})
.await
}
}
impl Resolve<RestartStack, (User, Update)> for State {
#[instrument(name = "RestartStack", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
pub async fn pull_stack_inner(
mut stack: Stack,
services: Vec<String>,
server: &Server,
mut update: Option<&mut Update>,
) -> anyhow::Result<ComposePullResponse> {
if let Some(update) = update.as_mut() {
if !services.is_empty() {
update.logs.push(Log::simple(
"Service/s",
format!(
"Execution requested for Stack service/s {}",
services.join(", ")
),
))
}
}
let git_token = crate::helpers::git_token(
&stack.config.git_provider,
&stack.config.git_account,
|https| stack.config.git_https = https,
).await.with_context(
|| format!("Failed to get git token in call to db. Stopping run. | {} | {}", stack.config.git_provider, stack.config.git_account),
)?;
let registry_token = crate::helpers::registry_token(
&stack.config.registry_provider,
&stack.config.registry_account,
).await.with_context(
|| format!("Failed to get registry token in call to db. Stopping run. | {} | {}", stack.config.registry_provider, stack.config.registry_account),
)?;
// interpolate variables / secrets
if !stack.config.skip_secret_interp {
let vars_and_secrets = get_variables_and_secrets().await?;
let mut global_replacers = HashSet::new();
let mut secret_replacers = HashSet::new();
interpolate_variables_secrets_into_string(
&vars_and_secrets,
&mut stack.config.file_contents,
&mut global_replacers,
&mut secret_replacers,
)?;
interpolate_variables_secrets_into_string(
&vars_and_secrets,
&mut stack.config.environment,
&mut global_replacers,
&mut secret_replacers,
)?;
if let Some(update) = update {
add_interp_update_log(
update,
&global_replacers,
&secret_replacers,
);
}
};
let res = periphery_client(server)?
.request(ComposePull {
stack,
services,
git_token,
registry_token,
})
.await?;
// Ensure cached stack state up to date by updating server cache
update_cache_for_server(server).await;
Ok(res)
}
impl Resolve<ExecuteArgs> for PullStack {
#[instrument(name = "PullStack", skip(user, update), fields(user_id = user.id))]
async fn resolve(
&self,
RestartStack { stack, service }: RestartStack,
(user, update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let (stack, server) = get_stack_and_server(
&self.stack,
user,
PermissionLevel::Execute,
true,
)
.await?;
// get the action state for the stack (or insert default).
let action_state =
action_states().stack.get_or_insert_default(&stack.id).await;
// Will check to ensure stack not already busy before updating, and return Err if so.
// The returned guard will set the action state back to default when dropped.
let _action_guard =
action_state.update(|state| state.pulling = true)?;
let mut update = update.clone();
update_update(update.clone()).await?;
let res = pull_stack_inner(
stack,
self.services,
&server,
Some(&mut update),
)
.await?;
update.logs.extend(res.logs);
update.finalize();
update_update(update.clone()).await?;
Ok(update)
}
}
impl Resolve<ExecuteArgs> for StartStack {
#[instrument(name = "StartStack", skip(user, update), fields(user_id = user.id))]
async fn resolve(
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
execute_compose::<StartStack>(
&self.stack,
self.services,
user,
|state| state.starting = true,
update.clone(),
(),
)
.await
.map_err(Into::into)
}
}
impl Resolve<ExecuteArgs> for RestartStack {
#[instrument(name = "RestartStack", skip(user, update), fields(user_id = user.id))]
async fn resolve(
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
execute_compose::<RestartStack>(
&stack,
service,
&user,
&self.stack,
self.services,
user,
|state| {
state.restarting = true;
},
update,
update.clone(),
(),
)
.await
.map_err(Into::into)
}
}
impl Resolve<PauseStack, (User, Update)> for State {
#[instrument(name = "PauseStack", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for PauseStack {
#[instrument(name = "PauseStack", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
PauseStack { stack, service }: PauseStack,
(user, update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
execute_compose::<PauseStack>(
&stack,
service,
&user,
&self.stack,
self.services,
user,
|state| state.pausing = true,
update,
update.clone(),
(),
)
.await
.map_err(Into::into)
}
}
impl Resolve<UnpauseStack, (User, Update)> for State {
#[instrument(name = "UnpauseStack", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for UnpauseStack {
#[instrument(name = "UnpauseStack", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
UnpauseStack { stack, service }: UnpauseStack,
(user, update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
execute_compose::<UnpauseStack>(
&stack,
service,
&user,
&self.stack,
self.services,
user,
|state| state.unpausing = true,
update,
update.clone(),
(),
)
.await
.map_err(Into::into)
}
}
impl Resolve<StopStack, (User, Update)> for State {
#[instrument(name = "StopStack", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
impl Resolve<ExecuteArgs> for StopStack {
#[instrument(name = "StopStack", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
StopStack {
stack,
stop_time,
service,
}: StopStack,
(user, update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
execute_compose::<StopStack>(
&stack,
service,
&user,
&self.stack,
self.services,
user,
|state| state.stopping = true,
update,
stop_time,
update.clone(),
self.stop_time,
)
.await
.map_err(Into::into)
}
}
impl Resolve<DestroyStack, (User, Update)> for State {
#[instrument(name = "DestroyStack", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
DestroyStack {
impl super::BatchExecute for BatchDestroyStack {
type Resource = Stack;
fn single_request(stack: String) -> ExecuteRequest {
ExecuteRequest::DestroyStack(DestroyStack {
stack,
remove_orphans,
stop_time,
}: DestroyStack,
(user, update): (User, Update),
) -> anyhow::Result<Update> {
execute_compose::<DestroyStack>(
&stack,
None,
&user,
|state| state.destroying = true,
update,
(stop_time, remove_orphans),
)
.await
services: Vec::new(),
remove_orphans: false,
stop_time: None,
})
}
}
impl Resolve<ExecuteArgs> for BatchDestroyStack {
#[instrument(name = "BatchDestroyStack", skip(user), fields(user_id = user.id))]
async fn resolve(
self,
ExecuteArgs { user, .. }: &ExecuteArgs,
) -> serror::Result<BatchExecutionResponse> {
super::batch_execute::<BatchDestroyStack>(&self.pattern, user)
.await
.map_err(Into::into)
}
}
impl Resolve<ExecuteArgs> for DestroyStack {
#[instrument(name = "DestroyStack", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
execute_compose::<DestroyStack>(
&self.stack,
self.services,
user,
|state| state.destroying = true,
update.clone(),
(self.stop_time, self.remove_orphans),
)
.await
.map_err(Into::into)
}
}

View File

@@ -1,12 +1,12 @@
use std::collections::HashMap;
use std::{collections::HashMap, str::FromStr};
use anyhow::{anyhow, Context};
use formatting::{colored, format_serror, Color};
use mongo_indexed::doc;
use anyhow::{Context, anyhow};
use formatting::{Color, colored, format_serror};
use komodo_client::{
api::{execute::RunSync, write::RefreshResourceSyncPending},
entities::{
self,
self, ResourceTargetVariant,
action::Action,
alerter::Alerter,
build::Build,
builder::Builder,
@@ -18,70 +18,180 @@ use komodo_client::{
server::Server,
server_template::ServerTemplate,
stack::Stack,
sync::ResourceSync,
update::{Log, Update},
user::{sync_user, User},
user::sync_user,
},
};
use mungos::{by_id::update_one_by_id, mongodb::bson::to_document};
use mongo_indexed::doc;
use mungos::{by_id::update_one_by_id, mongodb::bson::oid::ObjectId};
use resolver_api::Resolve;
use crate::{
helpers::{
query::get_id_to_tags,
sync::{
deploy::{
build_deploy_cache, deploy_from_cache, SyncDeployParams,
},
resource::{
get_updates_for_execution, AllResourcesById, ResourceSync,
},
api::write::WriteArgs,
helpers::{query::get_id_to_tags, update::update_update},
resource,
state::{action_states, db_client},
sync::{
AllResourcesById, ResourceSyncTrait,
deploy::{
SyncDeployParams, build_deploy_cache, deploy_from_cache,
},
update::update_update,
execute::{ExecuteResourceSync, get_updates_for_execution},
remote::RemoteResources,
},
resource::{self, refresh_resource_sync_state_cache},
state::{db_client, State},
};
impl Resolve<RunSync, (User, Update)> for State {
#[instrument(name = "RunSync", skip(self, user, update), fields(user_id = user.id, update_id = update.id))]
use super::ExecuteArgs;
impl Resolve<ExecuteArgs> for RunSync {
#[instrument(name = "RunSync", skip(user, update), fields(user_id = user.id, update_id = update.id))]
async fn resolve(
&self,
RunSync { sync }: RunSync,
(user, mut update): (User, Update),
) -> anyhow::Result<Update> {
self,
ExecuteArgs { user, update }: &ExecuteArgs,
) -> serror::Result<Update> {
let RunSync {
sync,
resource_type: match_resource_type,
resources: match_resources,
} = self;
let sync = resource::get_check_permissions::<
entities::sync::ResourceSync,
>(&sync, &user, PermissionLevel::Execute)
>(&sync, user, PermissionLevel::Execute)
.await?;
if sync.config.repo.is_empty() {
return Err(anyhow!("resource sync repo not configured"));
}
// get the action state for the sync (or insert default).
let action_state = action_states()
.resource_sync
.get_or_insert_default(&sync.id)
.await;
// This will set action state back to default when dropped.
// Will also check to ensure sync not already busy before updating.
let _action_guard =
action_state.update(|state| state.syncing = true)?;
let mut update = update.clone();
// Send update here for FE to recheck action state
update_update(update.clone()).await?;
let (res, logs, hash, message) =
crate::helpers::sync::remote::get_remote_resources(&sync)
.await
.context("failed to get remote resources")?;
let RemoteResources {
resources,
logs,
hash,
message,
file_errors,
..
} = crate::sync::remote::get_remote_resources(&sync)
.await
.context("failed to get remote resources")?;
update.logs.extend(logs);
update_update(update.clone()).await?;
let resources = res?;
if !file_errors.is_empty() {
return Err(
anyhow!("Found file errors. Cannot execute sync.").into(),
);
}
let resources = resources?;
let id_to_tags = get_id_to_tags(None).await?;
let all_resources = AllResourcesById::load().await?;
// Convert all match_resources to names
let match_resources = match_resources.map(|resources| {
resources
.into_iter()
.filter_map(|name_or_id| {
let Some(resource_type) = match_resource_type else {
return Some(name_or_id);
};
match ObjectId::from_str(&name_or_id) {
Ok(_) => match resource_type {
ResourceTargetVariant::Alerter => all_resources
.alerters
.get(&name_or_id)
.map(|a| a.name.clone()),
ResourceTargetVariant::Build => all_resources
.builds
.get(&name_or_id)
.map(|b| b.name.clone()),
ResourceTargetVariant::Builder => all_resources
.builders
.get(&name_or_id)
.map(|b| b.name.clone()),
ResourceTargetVariant::Deployment => all_resources
.deployments
.get(&name_or_id)
.map(|d| d.name.clone()),
ResourceTargetVariant::Procedure => all_resources
.procedures
.get(&name_or_id)
.map(|p| p.name.clone()),
ResourceTargetVariant::Action => all_resources
.actions
.get(&name_or_id)
.map(|p| p.name.clone()),
ResourceTargetVariant::Repo => all_resources
.repos
.get(&name_or_id)
.map(|r| r.name.clone()),
ResourceTargetVariant::Server => all_resources
.servers
.get(&name_or_id)
.map(|s| s.name.clone()),
ResourceTargetVariant::ServerTemplate => all_resources
.templates
.get(&name_or_id)
.map(|t| t.name.clone()),
ResourceTargetVariant::Stack => all_resources
.stacks
.get(&name_or_id)
.map(|s| s.name.clone()),
ResourceTargetVariant::ResourceSync => all_resources
.syncs
.get(&name_or_id)
.map(|s| s.name.clone()),
ResourceTargetVariant::System => None,
},
Err(_) => Some(name_or_id),
}
})
.collect::<Vec<_>>()
});
let deployments_by_name = all_resources
.deployments
.values()
.filter(|deployment| {
Deployment::include_resource(
&deployment.name,
&deployment.config,
match_resource_type,
match_resources.as_deref(),
&deployment.tags,
&id_to_tags,
&sync.config.match_tags,
)
})
.map(|deployment| (deployment.name.clone(), deployment.clone()))
.collect::<HashMap<_, _>>();
let stacks_by_name = all_resources
.stacks
.values()
.filter(|stack| {
Stack::include_resource(
&stack.name,
&stack.config,
match_resource_type,
match_resources.as_deref(),
&stack.tags,
&id_to_tags,
&sync.config.match_tags,
)
})
.map(|stack| (stack.name.clone(), stack.clone()))
.collect::<HashMap<_, _>>();
@@ -94,149 +204,209 @@ impl Resolve<RunSync, (User, Update)> for State {
})
.await?;
let (servers_to_create, servers_to_update, servers_to_delete) =
let delete = sync.config.managed || sync.config.delete;
let server_deltas = if sync.config.include_resources {
get_updates_for_execution::<Server>(
resources.servers,
sync.config.delete,
delete,
&all_resources,
match_resource_type,
match_resources.as_deref(),
&id_to_tags,
&sync.config.match_tags,
)
.await?;
let (
deployments_to_create,
deployments_to_update,
deployments_to_delete,
) = get_updates_for_execution::<Deployment>(
resources.deployments,
sync.config.delete,
&all_resources,
&id_to_tags,
)
.await?;
let (stacks_to_create, stacks_to_update, stacks_to_delete) =
.await?
} else {
Default::default()
};
let stack_deltas = if sync.config.include_resources {
get_updates_for_execution::<Stack>(
resources.stacks,
sync.config.delete,
delete,
&all_resources,
match_resource_type,
match_resources.as_deref(),
&id_to_tags,
&sync.config.match_tags,
)
.await?;
let (builds_to_create, builds_to_update, builds_to_delete) =
.await?
} else {
Default::default()
};
let deployment_deltas = if sync.config.include_resources {
get_updates_for_execution::<Deployment>(
resources.deployments,
delete,
&all_resources,
match_resource_type,
match_resources.as_deref(),
&id_to_tags,
&sync.config.match_tags,
)
.await?
} else {
Default::default()
};
let build_deltas = if sync.config.include_resources {
get_updates_for_execution::<Build>(
resources.builds,
sync.config.delete,
delete,
&all_resources,
match_resource_type,
match_resources.as_deref(),
&id_to_tags,
&sync.config.match_tags,
)
.await?;
let (repos_to_create, repos_to_update, repos_to_delete) =
.await?
} else {
Default::default()
};
let repo_deltas = if sync.config.include_resources {
get_updates_for_execution::<Repo>(
resources.repos,
sync.config.delete,
delete,
&all_resources,
match_resource_type,
match_resources.as_deref(),
&id_to_tags,
&sync.config.match_tags,
)
.await?;
let (
procedures_to_create,
procedures_to_update,
procedures_to_delete,
) = get_updates_for_execution::<Procedure>(
resources.procedures,
sync.config.delete,
&all_resources,
&id_to_tags,
)
.await?;
let (builders_to_create, builders_to_update, builders_to_delete) =
.await?
} else {
Default::default()
};
let procedure_deltas = if sync.config.include_resources {
get_updates_for_execution::<Procedure>(
resources.procedures,
delete,
&all_resources,
match_resource_type,
match_resources.as_deref(),
&id_to_tags,
&sync.config.match_tags,
)
.await?
} else {
Default::default()
};
let action_deltas = if sync.config.include_resources {
get_updates_for_execution::<Action>(
resources.actions,
delete,
&all_resources,
match_resource_type,
match_resources.as_deref(),
&id_to_tags,
&sync.config.match_tags,
)
.await?
} else {
Default::default()
};
let builder_deltas = if sync.config.include_resources {
get_updates_for_execution::<Builder>(
resources.builders,
sync.config.delete,
delete,
&all_resources,
match_resource_type,
match_resources.as_deref(),
&id_to_tags,
&sync.config.match_tags,
)
.await?;
let (alerters_to_create, alerters_to_update, alerters_to_delete) =
.await?
} else {
Default::default()
};
let alerter_deltas = if sync.config.include_resources {
get_updates_for_execution::<Alerter>(
resources.alerters,
sync.config.delete,
delete,
&all_resources,
match_resource_type,
match_resources.as_deref(),
&id_to_tags,
&sync.config.match_tags,
)
.await?;
let (
server_templates_to_create,
server_templates_to_update,
server_templates_to_delete,
) = get_updates_for_execution::<ServerTemplate>(
resources.server_templates,
sync.config.delete,
&all_resources,
&id_to_tags,
)
.await?;
let (
resource_syncs_to_create,
resource_syncs_to_update,
resource_syncs_to_delete,
) = get_updates_for_execution::<entities::sync::ResourceSync>(
resources.resource_syncs,
sync.config.delete,
&all_resources,
&id_to_tags,
)
.await?;
.await?
} else {
Default::default()
};
let server_template_deltas = if sync.config.include_resources {
get_updates_for_execution::<ServerTemplate>(
resources.server_templates,
delete,
&all_resources,
match_resource_type,
match_resources.as_deref(),
&id_to_tags,
&sync.config.match_tags,
)
.await?
} else {
Default::default()
};
let resource_sync_deltas = if sync.config.include_resources {
get_updates_for_execution::<entities::sync::ResourceSync>(
resources.resource_syncs,
delete,
&all_resources,
match_resource_type,
match_resources.as_deref(),
&id_to_tags,
&sync.config.match_tags,
)
.await?
} else {
Default::default()
};
let (
variables_to_create,
variables_to_update,
variables_to_delete,
) = crate::helpers::sync::variables::get_updates_for_execution(
resources.variables,
sync.config.delete,
)
.await?;
) = if match_resource_type.is_none()
&& match_resources.is_none()
&& sync.config.include_variables
{
crate::sync::variables::get_updates_for_execution(
resources.variables,
delete,
)
.await?
} else {
Default::default()
};
let (
user_groups_to_create,
user_groups_to_update,
user_groups_to_delete,
) = crate::helpers::sync::user_groups::get_updates_for_execution(
resources.user_groups,
sync.config.delete,
&all_resources,
)
.await?;
) = if match_resource_type.is_none()
&& match_resources.is_none()
&& sync.config.include_user_groups
{
crate::sync::user_groups::get_updates_for_execution(
resources.user_groups,
delete,
&all_resources,
)
.await?
} else {
Default::default()
};
if deploy_cache.is_empty()
&& resource_syncs_to_create.is_empty()
&& resource_syncs_to_update.is_empty()
&& resource_syncs_to_delete.is_empty()
&& server_templates_to_create.is_empty()
&& server_templates_to_update.is_empty()
&& server_templates_to_delete.is_empty()
&& servers_to_create.is_empty()
&& servers_to_update.is_empty()
&& servers_to_delete.is_empty()
&& deployments_to_create.is_empty()
&& deployments_to_update.is_empty()
&& deployments_to_delete.is_empty()
&& stacks_to_create.is_empty()
&& stacks_to_update.is_empty()
&& stacks_to_delete.is_empty()
&& builds_to_create.is_empty()
&& builds_to_update.is_empty()
&& builds_to_delete.is_empty()
&& builders_to_create.is_empty()
&& builders_to_update.is_empty()
&& builders_to_delete.is_empty()
&& alerters_to_create.is_empty()
&& alerters_to_update.is_empty()
&& alerters_to_delete.is_empty()
&& repos_to_create.is_empty()
&& repos_to_update.is_empty()
&& repos_to_delete.is_empty()
&& procedures_to_create.is_empty()
&& procedures_to_update.is_empty()
&& procedures_to_delete.is_empty()
&& resource_sync_deltas.no_changes()
&& server_template_deltas.no_changes()
&& server_deltas.no_changes()
&& deployment_deltas.no_changes()
&& stack_deltas.no_changes()
&& build_deltas.no_changes()
&& builder_deltas.no_changes()
&& alerter_deltas.no_changes()
&& repo_deltas.no_changes()
&& procedure_deltas.no_changes()
&& action_deltas.no_changes()
&& user_groups_to_create.is_empty()
&& user_groups_to_update.is_empty()
&& user_groups_to_delete.is_empty()
@@ -261,7 +431,7 @@ impl Resolve<RunSync, (User, Update)> for State {
// No deps
maybe_extend(
&mut update.logs,
crate::helpers::sync::variables::run_updates(
crate::sync::variables::run_updates(
variables_to_create,
variables_to_update,
variables_to_delete,
@@ -270,7 +440,7 @@ impl Resolve<RunSync, (User, Update)> for State {
);
maybe_extend(
&mut update.logs,
crate::helpers::sync::user_groups::run_updates(
crate::sync::user_groups::run_updates(
user_groups_to_create,
user_groups_to_update,
user_groups_to_delete,
@@ -279,108 +449,63 @@ impl Resolve<RunSync, (User, Update)> for State {
);
maybe_extend(
&mut update.logs,
entities::sync::ResourceSync::run_updates(
resource_syncs_to_create,
resource_syncs_to_update,
resource_syncs_to_delete,
)
.await,
ResourceSync::execute_sync_updates(resource_sync_deltas).await,
);
maybe_extend(
&mut update.logs,
ServerTemplate::run_updates(
server_templates_to_create,
server_templates_to_update,
server_templates_to_delete,
)
.await,
ServerTemplate::execute_sync_updates(server_template_deltas)
.await,
);
maybe_extend(
&mut update.logs,
Server::run_updates(
servers_to_create,
servers_to_update,
servers_to_delete,
)
.await,
Server::execute_sync_updates(server_deltas).await,
);
maybe_extend(
&mut update.logs,
Alerter::run_updates(
alerters_to_create,
alerters_to_update,
alerters_to_delete,
)
.await,
Alerter::execute_sync_updates(alerter_deltas).await,
);
maybe_extend(
&mut update.logs,
Action::execute_sync_updates(action_deltas).await,
);
// Dependent on server
maybe_extend(
&mut update.logs,
Builder::run_updates(
builders_to_create,
builders_to_update,
builders_to_delete,
)
.await,
Builder::execute_sync_updates(builder_deltas).await,
);
maybe_extend(
&mut update.logs,
Repo::run_updates(
repos_to_create,
repos_to_update,
repos_to_delete,
)
.await,
Repo::execute_sync_updates(repo_deltas).await,
);
// Dependant on builder
maybe_extend(
&mut update.logs,
Build::run_updates(
builds_to_create,
builds_to_update,
builds_to_delete,
)
.await,
Build::execute_sync_updates(build_deltas).await,
);
// Dependant on server / build
maybe_extend(
&mut update.logs,
Deployment::run_updates(
deployments_to_create,
deployments_to_update,
deployments_to_delete,
)
.await,
Deployment::execute_sync_updates(deployment_deltas).await,
);
// stack only depends on server, but maybe will depend on build later.
maybe_extend(
&mut update.logs,
Stack::run_updates(
stacks_to_create,
stacks_to_update,
stacks_to_delete,
)
.await,
Stack::execute_sync_updates(stack_deltas).await,
);
// Dependant on everything
maybe_extend(
&mut update.logs,
Procedure::run_updates(
procedures_to_create,
procedures_to_update,
procedures_to_delete,
)
.await,
Procedure::execute_sync_updates(procedure_deltas).await,
);
// Execute the deploy cache
deploy_from_cache(deploy_cache, &mut update.logs).await;
let db = db_client().await;
let db = db_client();
if let Err(e) = update_one_by_id(
&db.resource_syncs,
@@ -402,39 +527,27 @@ impl Resolve<RunSync, (User, Update)> for State {
)
}
if let Err(e) = State
.resolve(
RefreshResourceSyncPending { sync: sync.id },
sync_user().to_owned(),
)
if let Err(e) = (RefreshResourceSyncPending { sync: sync.id })
.resolve(&WriteArgs {
user: sync_user().to_owned(),
})
.await
{
warn!("failed to refresh sync {} after run | {e:#}", sync.name);
warn!(
"failed to refresh sync {} after run | {:#}",
sync.name, e.error
);
update.push_error_log(
"refresh sync",
format_serror(
&e.context("failed to refresh sync pending after run")
&e.error
.context("failed to refresh sync pending after run")
.into(),
),
);
}
update.finalize();
// Need to manually update the update before cache refresh,
// and before broadcast with add_update.
// The Err case of to_document should be unreachable,
// but will fail to update cache in that case.
if let Ok(update_doc) = to_document(&update) {
let _ = update_one_by_id(
&db.updates,
&update.id,
mungos::update::Update::Set(update_doc),
None,
)
.await;
refresh_resource_sync_state_cache().await;
}
update_update(update.clone()).await?;
Ok(update)

View File

@@ -0,0 +1,137 @@
use anyhow::Context;
use komodo_client::{
api::read::*,
entities::{
action::{
Action, ActionActionState, ActionListItem, ActionState,
},
permission::PermissionLevel,
},
};
use resolver_api::Resolve;
use crate::{
helpers::query::get_all_tags,
resource,
state::{action_state_cache, action_states},
};
use super::ReadArgs;
impl Resolve<ReadArgs> for GetAction {
async fn resolve(
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<Action> {
Ok(
resource::get_check_permissions::<Action>(
&self.action,
user,
PermissionLevel::Read,
)
.await?,
)
}
}
impl Resolve<ReadArgs> for ListActions {
async fn resolve(
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<Vec<ActionListItem>> {
let all_tags = if self.query.tags.is_empty() {
vec![]
} else {
get_all_tags(None).await?
};
Ok(
resource::list_for_user::<Action>(self.query, user, &all_tags)
.await?,
)
}
}
impl Resolve<ReadArgs> for ListFullActions {
async fn resolve(
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListFullActionsResponse> {
let all_tags = if self.query.tags.is_empty() {
vec![]
} else {
get_all_tags(None).await?
};
Ok(
resource::list_full_for_user::<Action>(
self.query, user, &all_tags,
)
.await?,
)
}
}
impl Resolve<ReadArgs> for GetActionActionState {
async fn resolve(
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ActionActionState> {
let action = resource::get_check_permissions::<Action>(
&self.action,
user,
PermissionLevel::Read,
)
.await?;
let action_state = action_states()
.action
.get(&action.id)
.await
.unwrap_or_default()
.get()?;
Ok(action_state)
}
}
impl Resolve<ReadArgs> for GetActionsSummary {
async fn resolve(
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<GetActionsSummaryResponse> {
let actions = resource::list_full_for_user::<Action>(
Default::default(),
user,
&[],
)
.await
.context("failed to get actions from db")?;
let mut res = GetActionsSummaryResponse::default();
let cache = action_state_cache();
let action_states = action_states();
for action in actions {
res.total += 1;
match (
cache.get(&action.id).await.unwrap_or_default(),
action_states
.action
.get(&action.id)
.await
.unwrap_or_default()
.get()?,
) {
(_, action_states) if action_states.running => {
res.running += 1;
}
(ActionState::Ok, _) => res.ok += 1,
(ActionState::Failed, _) => res.failed += 1,
(ActionState::Unknown, _) => res.unknown += 1,
// will never come off the cache in the running state, since that comes from action states
(ActionState::Running, _) => unreachable!(),
}
}
Ok(res)
}
}

View File

@@ -3,7 +3,10 @@ use komodo_client::{
api::read::{
GetAlert, GetAlertResponse, ListAlerts, ListAlertsResponse,
},
entities::{deployment::Deployment, server::Server, user::User},
entities::{
deployment::Deployment, server::Server, stack::Stack,
sync::ResourceSync,
},
};
use mungos::{
by_id::find_one_by_id,
@@ -13,40 +16,46 @@ use mungos::{
use resolver_api::Resolve;
use crate::{
config::core_config,
resource::get_resource_ids_for_user,
state::{db_client, State},
config::core_config, resource::get_resource_ids_for_user,
state::db_client,
};
use super::ReadArgs;
const NUM_ALERTS_PER_PAGE: u64 = 100;
impl Resolve<ListAlerts, User> for State {
impl Resolve<ReadArgs> for ListAlerts {
async fn resolve(
&self,
ListAlerts { query, page }: ListAlerts,
user: User,
) -> anyhow::Result<ListAlertsResponse> {
let mut query = query.unwrap_or_default();
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListAlertsResponse> {
let mut query = self.query.unwrap_or_default();
if !user.admin && !core_config().transparent_mode {
let server_ids =
get_resource_ids_for_user::<Server>(&user).await?;
get_resource_ids_for_user::<Server>(user).await?;
let stack_ids =
get_resource_ids_for_user::<Stack>(user).await?;
let deployment_ids =
get_resource_ids_for_user::<Deployment>(&user).await?;
get_resource_ids_for_user::<Deployment>(user).await?;
let sync_ids =
get_resource_ids_for_user::<ResourceSync>(user).await?;
query.extend(doc! {
"$or": [
{ "target.type": "Server", "target.id": { "$in": &server_ids } },
{ "target.type": "Stack", "target.id": { "$in": &stack_ids } },
{ "target.type": "Deployment", "target.id": { "$in": &deployment_ids } },
{ "target.type": "ResourceSync", "target.id": { "$in": &sync_ids } },
]
});
}
let alerts = find_collect(
&db_client().await.alerts,
&db_client().alerts,
query,
FindOptions::builder()
.sort(doc! { "ts": -1 })
.limit(NUM_ALERTS_PER_PAGE as i64)
.skip(page * NUM_ALERTS_PER_PAGE)
.skip(self.page * NUM_ALERTS_PER_PAGE)
.build(),
)
.await
@@ -55,7 +64,7 @@ impl Resolve<ListAlerts, User> for State {
let next_page = if alerts.len() < NUM_ALERTS_PER_PAGE as usize {
None
} else {
Some((page + 1) as i64)
Some((self.page + 1) as i64)
};
let res = ListAlertsResponse { next_page, alerts };
@@ -64,15 +73,16 @@ impl Resolve<ListAlerts, User> for State {
}
}
impl Resolve<GetAlert, User> for State {
impl Resolve<ReadArgs> for GetAlert {
async fn resolve(
&self,
GetAlert { id }: GetAlert,
_: User,
) -> anyhow::Result<GetAlertResponse> {
find_one_by_id(&db_client().await.alerts, &id)
.await
.context("failed to query db for alert")?
.context("no alert found with given id")
self,
_: &ReadArgs,
) -> serror::Result<GetAlertResponse> {
Ok(
find_one_by_id(&db_client().alerts, &self.id)
.await
.context("failed to query db for alert")?
.context("no alert found with given id")?,
)
}
}

View File

@@ -1,73 +1,89 @@
use anyhow::Context;
use mongo_indexed::Document;
use komodo_client::{
api::read::*,
entities::{
alerter::{Alerter, AlerterListItem},
permission::PermissionLevel,
user::User,
},
};
use mongo_indexed::Document;
use mungos::mongodb::bson::doc;
use resolver_api::Resolve;
use crate::{
resource,
state::{db_client, State},
helpers::query::get_all_tags, resource, state::db_client,
};
impl Resolve<GetAlerter, User> for State {
use super::ReadArgs;
impl Resolve<ReadArgs> for GetAlerter {
async fn resolve(
&self,
GetAlerter { alerter }: GetAlerter,
user: User,
) -> anyhow::Result<Alerter> {
resource::get_check_permissions::<Alerter>(
&alerter,
&user,
PermissionLevel::Read,
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<Alerter> {
Ok(
resource::get_check_permissions::<Alerter>(
&self.alerter,
user,
PermissionLevel::Read,
)
.await?,
)
.await
}
}
impl Resolve<ListAlerters, User> for State {
impl Resolve<ReadArgs> for ListAlerters {
async fn resolve(
&self,
ListAlerters { query }: ListAlerters,
user: User,
) -> anyhow::Result<Vec<AlerterListItem>> {
resource::list_for_user::<Alerter>(query, &user).await
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<Vec<AlerterListItem>> {
let all_tags = if self.query.tags.is_empty() {
vec![]
} else {
get_all_tags(None).await?
};
Ok(
resource::list_for_user::<Alerter>(self.query, user, &all_tags)
.await?,
)
}
}
impl Resolve<ListFullAlerters, User> for State {
impl Resolve<ReadArgs> for ListFullAlerters {
async fn resolve(
&self,
ListFullAlerters { query }: ListFullAlerters,
user: User,
) -> anyhow::Result<ListFullAlertersResponse> {
resource::list_full_for_user::<Alerter>(query, &user).await
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListFullAlertersResponse> {
let all_tags = if self.query.tags.is_empty() {
vec![]
} else {
get_all_tags(None).await?
};
Ok(
resource::list_full_for_user::<Alerter>(
self.query, user, &all_tags,
)
.await?,
)
}
}
impl Resolve<GetAlertersSummary, User> for State {
impl Resolve<ReadArgs> for GetAlertersSummary {
async fn resolve(
&self,
GetAlertersSummary {}: GetAlertersSummary,
user: User,
) -> anyhow::Result<GetAlertersSummaryResponse> {
let query =
match resource::get_resource_ids_for_user::<Alerter>(&user)
.await?
{
Some(ids) => doc! {
"_id": { "$in": ids }
},
None => Document::new(),
};
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<GetAlertersSummaryResponse> {
let query = match resource::get_resource_object_ids_for_user::<
Alerter,
>(user)
.await?
{
Some(ids) => doc! {
"_id": { "$in": ids }
},
None => Document::new(),
};
let total = db_client()
.await
.alerters
.count_documents(query)
.await

View File

@@ -6,12 +6,11 @@ use futures::TryStreamExt;
use komodo_client::{
api::read::*,
entities::{
Operation,
build::{Build, BuildActionState, BuildListItem, BuildState},
config::core::CoreConfig,
permission::PermissionLevel,
update::UpdateStatus,
user::User,
Operation,
},
};
use mungos::{
@@ -22,56 +21,75 @@ use resolver_api::Resolve;
use crate::{
config::core_config,
helpers::query::get_all_tags,
resource,
state::{
action_states, build_state_cache, db_client, github_client, State,
action_states, build_state_cache, db_client, github_client,
},
};
impl Resolve<GetBuild, User> for State {
use super::ReadArgs;
impl Resolve<ReadArgs> for GetBuild {
async fn resolve(
&self,
GetBuild { build }: GetBuild,
user: User,
) -> anyhow::Result<Build> {
resource::get_check_permissions::<Build>(
&build,
&user,
PermissionLevel::Read,
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<Build> {
Ok(
resource::get_check_permissions::<Build>(
&self.build,
user,
PermissionLevel::Read,
)
.await?,
)
.await
}
}
impl Resolve<ListBuilds, User> for State {
impl Resolve<ReadArgs> for ListBuilds {
async fn resolve(
&self,
ListBuilds { query }: ListBuilds,
user: User,
) -> anyhow::Result<Vec<BuildListItem>> {
resource::list_for_user::<Build>(query, &user).await
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<Vec<BuildListItem>> {
let all_tags = if self.query.tags.is_empty() {
vec![]
} else {
get_all_tags(None).await?
};
Ok(
resource::list_for_user::<Build>(self.query, user, &all_tags)
.await?,
)
}
}
impl Resolve<ListFullBuilds, User> for State {
impl Resolve<ReadArgs> for ListFullBuilds {
async fn resolve(
&self,
ListFullBuilds { query }: ListFullBuilds,
user: User,
) -> anyhow::Result<ListFullBuildsResponse> {
resource::list_full_for_user::<Build>(query, &user).await
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListFullBuildsResponse> {
let all_tags = if self.query.tags.is_empty() {
vec![]
} else {
get_all_tags(None).await?
};
Ok(
resource::list_full_for_user::<Build>(
self.query, user, &all_tags,
)
.await?,
)
}
}
impl Resolve<GetBuildActionState, User> for State {
impl Resolve<ReadArgs> for GetBuildActionState {
async fn resolve(
&self,
GetBuildActionState { build }: GetBuildActionState,
user: User,
) -> anyhow::Result<BuildActionState> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<BuildActionState> {
let build = resource::get_check_permissions::<Build>(
&build,
&user,
&self.build,
user,
PermissionLevel::Read,
)
.await?;
@@ -85,15 +103,15 @@ impl Resolve<GetBuildActionState, User> for State {
}
}
impl Resolve<GetBuildsSummary, User> for State {
impl Resolve<ReadArgs> for GetBuildsSummary {
async fn resolve(
&self,
GetBuildsSummary {}: GetBuildsSummary,
user: User,
) -> anyhow::Result<GetBuildsSummaryResponse> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<GetBuildsSummaryResponse> {
let builds = resource::list_full_for_user::<Build>(
Default::default(),
&user,
user,
&[],
)
.await
.context("failed to get all builds")?;
@@ -132,20 +150,18 @@ impl Resolve<GetBuildsSummary, User> for State {
const ONE_DAY_MS: i64 = 86400000;
impl Resolve<GetBuildMonthlyStats, User> for State {
impl Resolve<ReadArgs> for GetBuildMonthlyStats {
async fn resolve(
&self,
GetBuildMonthlyStats { page }: GetBuildMonthlyStats,
_: User,
) -> anyhow::Result<GetBuildMonthlyStatsResponse> {
self,
_: &ReadArgs,
) -> serror::Result<GetBuildMonthlyStatsResponse> {
let curr_ts = unix_timestamp_ms() as i64;
let next_day = curr_ts - curr_ts % ONE_DAY_MS + ONE_DAY_MS;
let close_ts = next_day - page as i64 * 30 * ONE_DAY_MS;
let close_ts = next_day - self.page as i64 * 30 * ONE_DAY_MS;
let open_ts = close_ts - 30 * ONE_DAY_MS;
let mut build_updates = db_client()
.await
.updates
.find(doc! {
"start_ts": {
@@ -190,21 +206,21 @@ fn ms_to_hour(duration: i64) -> f64 {
duration as f64 / MS_TO_HOUR_DIVISOR
}
impl Resolve<ListBuildVersions, User> for State {
impl Resolve<ReadArgs> for ListBuildVersions {
async fn resolve(
&self,
ListBuildVersions {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<Vec<BuildVersionResponseItem>> {
let ListBuildVersions {
build,
major,
minor,
patch,
limit,
}: ListBuildVersions,
user: User,
) -> anyhow::Result<Vec<BuildVersionResponseItem>> {
} = self;
let build = resource::get_check_permissions::<Build>(
&build,
&user,
user,
PermissionLevel::Read,
)
.await?;
@@ -229,7 +245,7 @@ impl Resolve<ListBuildVersions, User> for State {
}
let versions = find_collect(
&db_client().await.updates,
&db_client().updates,
filter,
FindOptions::builder()
.sort(doc! { "_id": -1 })
@@ -247,15 +263,21 @@ impl Resolve<ListBuildVersions, User> for State {
}
}
impl Resolve<ListCommonBuildExtraArgs, User> for State {
impl Resolve<ReadArgs> for ListCommonBuildExtraArgs {
async fn resolve(
&self,
ListCommonBuildExtraArgs { query }: ListCommonBuildExtraArgs,
user: User,
) -> anyhow::Result<ListCommonBuildExtraArgsResponse> {
let builds = resource::list_full_for_user::<Build>(query, &user)
.await
.context("failed to get resources matching query")?;
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListCommonBuildExtraArgsResponse> {
let all_tags = if self.query.tags.is_empty() {
vec![]
} else {
get_all_tags(None).await?
};
let builds = resource::list_full_for_user::<Build>(
self.query, user, &all_tags,
)
.await
.context("failed to get resources matching query")?;
// first collect with guaranteed uniqueness
let mut res = HashSet::<String>::new();
@@ -272,12 +294,11 @@ impl Resolve<ListCommonBuildExtraArgs, User> for State {
}
}
impl Resolve<GetBuildWebhookEnabled, User> for State {
impl Resolve<ReadArgs> for GetBuildWebhookEnabled {
async fn resolve(
&self,
GetBuildWebhookEnabled { build }: GetBuildWebhookEnabled,
user: User,
) -> anyhow::Result<GetBuildWebhookEnabledResponse> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<GetBuildWebhookEnabledResponse> {
let Some(github) = github_client() else {
return Ok(GetBuildWebhookEnabledResponse {
managed: false,
@@ -286,8 +307,8 @@ impl Resolve<GetBuildWebhookEnabled, User> for State {
};
let build = resource::get_check_permissions::<Build>(
&build,
&user,
&self.build,
user,
PermissionLevel::Read,
)
.await?;
@@ -328,7 +349,11 @@ impl Resolve<GetBuildWebhookEnabled, User> for State {
..
} = core_config();
let host = webhook_base_url.as_ref().unwrap_or(host);
let host = if webhook_base_url.is_empty() {
host
} else {
webhook_base_url
};
let url = format!("{host}/listener/github/build/{}", build.id);
for webhook in webhooks {

View File

@@ -1,73 +1,89 @@
use anyhow::Context;
use mongo_indexed::Document;
use komodo_client::{
api::read::*,
entities::{
builder::{Builder, BuilderListItem},
permission::PermissionLevel,
user::User,
},
};
use mongo_indexed::Document;
use mungos::mongodb::bson::doc;
use resolver_api::Resolve;
use crate::{
resource,
state::{db_client, State},
helpers::query::get_all_tags, resource, state::db_client,
};
impl Resolve<GetBuilder, User> for State {
use super::ReadArgs;
impl Resolve<ReadArgs> for GetBuilder {
async fn resolve(
&self,
GetBuilder { builder }: GetBuilder,
user: User,
) -> anyhow::Result<Builder> {
resource::get_check_permissions::<Builder>(
&builder,
&user,
PermissionLevel::Read,
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<Builder> {
Ok(
resource::get_check_permissions::<Builder>(
&self.builder,
user,
PermissionLevel::Read,
)
.await?,
)
.await
}
}
impl Resolve<ListBuilders, User> for State {
impl Resolve<ReadArgs> for ListBuilders {
async fn resolve(
&self,
ListBuilders { query }: ListBuilders,
user: User,
) -> anyhow::Result<Vec<BuilderListItem>> {
resource::list_for_user::<Builder>(query, &user).await
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<Vec<BuilderListItem>> {
let all_tags = if self.query.tags.is_empty() {
vec![]
} else {
get_all_tags(None).await?
};
Ok(
resource::list_for_user::<Builder>(self.query, user, &all_tags)
.await?,
)
}
}
impl Resolve<ListFullBuilders, User> for State {
impl Resolve<ReadArgs> for ListFullBuilders {
async fn resolve(
&self,
ListFullBuilders { query }: ListFullBuilders,
user: User,
) -> anyhow::Result<ListFullBuildersResponse> {
resource::list_full_for_user::<Builder>(query, &user).await
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListFullBuildersResponse> {
let all_tags = if self.query.tags.is_empty() {
vec![]
} else {
get_all_tags(None).await?
};
Ok(
resource::list_full_for_user::<Builder>(
self.query, user, &all_tags,
)
.await?,
)
}
}
impl Resolve<GetBuildersSummary, User> for State {
impl Resolve<ReadArgs> for GetBuildersSummary {
async fn resolve(
&self,
GetBuildersSummary {}: GetBuildersSummary,
user: User,
) -> anyhow::Result<GetBuildersSummaryResponse> {
let query =
match resource::get_resource_ids_for_user::<Builder>(&user)
.await?
{
Some(ids) => doc! {
"_id": { "$in": ids }
},
None => Document::new(),
};
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<GetBuildersSummaryResponse> {
let query = match resource::get_resource_object_ids_for_user::<
Builder,
>(user)
.await?
{
Some(ids) => doc! {
"_id": { "$in": ids }
},
None => Document::new(),
};
let total = db_client()
.await
.builders
.count_documents(query)
.await

View File

@@ -1,6 +1,6 @@
use std::{cmp, collections::HashSet};
use anyhow::{anyhow, Context};
use anyhow::{Context, anyhow};
use komodo_client::{
api::read::*,
entities::{
@@ -12,62 +12,89 @@ use komodo_client::{
permission::PermissionLevel,
server::Server,
update::Log,
user::User,
},
};
use periphery_client::api;
use resolver_api::Resolve;
use crate::{
helpers::periphery_client,
helpers::{periphery_client, query::get_all_tags},
resource,
state::{action_states, deployment_status_cache, State},
state::{action_states, deployment_status_cache},
};
impl Resolve<GetDeployment, User> for State {
use super::ReadArgs;
impl Resolve<ReadArgs> for GetDeployment {
async fn resolve(
&self,
GetDeployment { deployment }: GetDeployment,
user: User,
) -> anyhow::Result<Deployment> {
resource::get_check_permissions::<Deployment>(
&deployment,
&user,
PermissionLevel::Read,
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<Deployment> {
Ok(
resource::get_check_permissions::<Deployment>(
&self.deployment,
user,
PermissionLevel::Read,
)
.await?,
)
.await
}
}
impl Resolve<ListDeployments, User> for State {
impl Resolve<ReadArgs> for ListDeployments {
async fn resolve(
&self,
ListDeployments { query }: ListDeployments,
user: User,
) -> anyhow::Result<Vec<DeploymentListItem>> {
resource::list_for_user::<Deployment>(query, &user).await
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<Vec<DeploymentListItem>> {
let all_tags = if self.query.tags.is_empty() {
vec![]
} else {
get_all_tags(None).await?
};
let only_update_available = self.query.specific.update_available;
let deployments = resource::list_for_user::<Deployment>(
self.query, user, &all_tags,
)
.await?;
let deployments = if only_update_available {
deployments
.into_iter()
.filter(|deployment| deployment.info.update_available)
.collect()
} else {
deployments
};
Ok(deployments)
}
}
impl Resolve<ListFullDeployments, User> for State {
impl Resolve<ReadArgs> for ListFullDeployments {
async fn resolve(
&self,
ListFullDeployments { query }: ListFullDeployments,
user: User,
) -> anyhow::Result<ListFullDeploymentsResponse> {
resource::list_full_for_user::<Deployment>(query, &user).await
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListFullDeploymentsResponse> {
let all_tags = if self.query.tags.is_empty() {
vec![]
} else {
get_all_tags(None).await?
};
Ok(
resource::list_full_for_user::<Deployment>(
self.query, user, &all_tags,
)
.await?,
)
}
}
impl Resolve<GetDeploymentContainer, User> for State {
impl Resolve<ReadArgs> for GetDeploymentContainer {
async fn resolve(
&self,
GetDeploymentContainer { deployment }: GetDeploymentContainer,
user: User,
) -> anyhow::Result<GetDeploymentContainerResponse> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<GetDeploymentContainerResponse> {
let deployment = resource::get_check_permissions::<Deployment>(
&deployment,
&user,
&self.deployment,
user,
PermissionLevel::Read,
)
.await?;
@@ -85,19 +112,23 @@ impl Resolve<GetDeploymentContainer, User> for State {
const MAX_LOG_LENGTH: u64 = 5000;
impl Resolve<GetDeploymentLog, User> for State {
impl Resolve<ReadArgs> for GetDeploymentLog {
async fn resolve(
&self,
GetDeploymentLog { deployment, tail }: GetDeploymentLog,
user: User,
) -> anyhow::Result<Log> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<Log> {
let GetDeploymentLog {
deployment,
tail,
timestamps,
} = self;
let Deployment {
name,
config: DeploymentConfig { server_id, .. },
..
} = resource::get_check_permissions::<Deployment>(
&deployment,
&user,
user,
PermissionLevel::Read,
)
.await?;
@@ -105,34 +136,37 @@ impl Resolve<GetDeploymentLog, User> for State {
return Ok(Log::default());
}
let server = resource::get::<Server>(&server_id).await?;
periphery_client(&server)?
let res = periphery_client(&server)?
.request(api::container::GetContainerLog {
name,
tail: cmp::min(tail, MAX_LOG_LENGTH),
timestamps,
})
.await
.context("failed at call to periphery")
.context("failed at call to periphery")?;
Ok(res)
}
}
impl Resolve<SearchDeploymentLog, User> for State {
impl Resolve<ReadArgs> for SearchDeploymentLog {
async fn resolve(
&self,
SearchDeploymentLog {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<Log> {
let SearchDeploymentLog {
deployment,
terms,
combinator,
invert,
}: SearchDeploymentLog,
user: User,
) -> anyhow::Result<Log> {
timestamps,
} = self;
let Deployment {
name,
config: DeploymentConfig { server_id, .. },
..
} = resource::get_check_permissions::<Deployment>(
&deployment,
&user,
user,
PermissionLevel::Read,
)
.await?;
@@ -140,54 +174,57 @@ impl Resolve<SearchDeploymentLog, User> for State {
return Ok(Log::default());
}
let server = resource::get::<Server>(&server_id).await?;
periphery_client(&server)?
let res = periphery_client(&server)?
.request(api::container::GetContainerLogSearch {
name,
terms,
combinator,
invert,
timestamps,
})
.await
.context("failed at call to periphery")
.context("failed at call to periphery")?;
Ok(res)
}
}
impl Resolve<GetDeploymentStats, User> for State {
impl Resolve<ReadArgs> for GetDeploymentStats {
async fn resolve(
&self,
GetDeploymentStats { deployment }: GetDeploymentStats,
user: User,
) -> anyhow::Result<ContainerStats> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ContainerStats> {
let Deployment {
name,
config: DeploymentConfig { server_id, .. },
..
} = resource::get_check_permissions::<Deployment>(
&deployment,
&user,
&self.deployment,
user,
PermissionLevel::Read,
)
.await?;
if server_id.is_empty() {
return Err(anyhow!("deployment has no server attached"));
return Err(
anyhow!("deployment has no server attached").into(),
);
}
let server = resource::get::<Server>(&server_id).await?;
periphery_client(&server)?
let res = periphery_client(&server)?
.request(api::container::GetContainerStats { name })
.await
.context("failed to get stats from periphery")
.context("failed to get stats from periphery")?;
Ok(res)
}
}
impl Resolve<GetDeploymentActionState, User> for State {
impl Resolve<ReadArgs> for GetDeploymentActionState {
async fn resolve(
&self,
GetDeploymentActionState { deployment }: GetDeploymentActionState,
user: User,
) -> anyhow::Result<DeploymentActionState> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<DeploymentActionState> {
let deployment = resource::get_check_permissions::<Deployment>(
&deployment,
&user,
&self.deployment,
user,
PermissionLevel::Read,
)
.await?;
@@ -201,15 +238,15 @@ impl Resolve<GetDeploymentActionState, User> for State {
}
}
impl Resolve<GetDeploymentsSummary, User> for State {
impl Resolve<ReadArgs> for GetDeploymentsSummary {
async fn resolve(
&self,
GetDeploymentsSummary {}: GetDeploymentsSummary,
user: User,
) -> anyhow::Result<GetDeploymentsSummaryResponse> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<GetDeploymentsSummaryResponse> {
let deployments = resource::list_full_for_user::<Deployment>(
Default::default(),
&user,
user,
&[],
)
.await
.context("failed to get deployments from db")?;
@@ -223,14 +260,17 @@ impl Resolve<GetDeploymentsSummary, User> for State {
DeploymentState::Running => {
res.running += 1;
}
DeploymentState::Unknown => {
res.unknown += 1;
DeploymentState::Exited | DeploymentState::Paused => {
res.stopped += 1;
}
DeploymentState::NotDeployed => {
res.not_deployed += 1;
}
DeploymentState::Unknown => {
res.unknown += 1;
}
_ => {
res.stopped += 1;
res.unhealthy += 1;
}
}
}
@@ -238,16 +278,21 @@ impl Resolve<GetDeploymentsSummary, User> for State {
}
}
impl Resolve<ListCommonDeploymentExtraArgs, User> for State {
impl Resolve<ReadArgs> for ListCommonDeploymentExtraArgs {
async fn resolve(
&self,
ListCommonDeploymentExtraArgs { query }: ListCommonDeploymentExtraArgs,
user: User,
) -> anyhow::Result<ListCommonDeploymentExtraArgsResponse> {
let deployments =
resource::list_full_for_user::<Deployment>(query, &user)
.await
.context("failed to get resources matching query")?;
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListCommonDeploymentExtraArgsResponse> {
let all_tags = if self.query.tags.is_empty() {
vec![]
} else {
get_all_tags(None).await?
};
let deployments = resource::list_full_for_user::<Deployment>(
self.query, user, &all_tags,
)
.await
.context("failed to get resources matching query")?;
// first collect with guaranteed uniqueness
let mut res = HashSet::<String>::new();

View File

@@ -1,11 +1,11 @@
use std::{collections::HashSet, sync::OnceLock, time::Instant};
use anyhow::{anyhow, Context};
use axum::{middleware, routing::post, Extension, Router};
use axum_extra::{headers::ContentType, TypedHeader};
use anyhow::{Context, anyhow};
use axum::{Extension, Router, middleware, routing::post};
use komodo_client::{
api::read::*,
entities::{
ResourceTarget,
build::Build,
builder::{Builder, BuilderConfig},
config::{DockerRegistry, GitProvider},
@@ -13,12 +13,10 @@ use komodo_client::{
server::Server,
sync::ResourceSync,
user::User,
ResourceTarget,
},
};
use resolver_api::{
derive::Resolver, Resolve, ResolveToString, Resolver,
};
use resolver_api::Resolve;
use response::Response;
use serde::{Deserialize, Serialize};
use serror::Json;
use typeshare::typeshare;
@@ -26,9 +24,10 @@ use uuid::Uuid;
use crate::{
auth::auth_request, config::core_config, helpers::periphery_client,
resource, state::State,
resource,
};
mod action;
mod alert;
mod alerter;
mod build;
@@ -38,7 +37,6 @@ mod permission;
mod procedure;
mod provider;
mod repo;
mod search;
mod server;
mod server_template;
mod stack;
@@ -50,18 +48,19 @@ mod user;
mod user_group;
mod variable;
pub struct ReadArgs {
pub user: User,
}
#[typeshare]
#[derive(Serialize, Deserialize, Debug, Clone, Resolver)]
#[resolver_target(State)]
#[resolver_args(User)]
#[derive(Serialize, Deserialize, Debug, Clone, Resolve)]
#[args(ReadArgs)]
#[response(Response)]
#[error(serror::Error)]
#[serde(tag = "type", content = "params")]
enum ReadRequest {
#[to_string_resolver]
GetVersion(GetVersion),
#[to_string_resolver]
GetCoreInfo(GetCoreInfo),
#[to_string_resolver]
ListAwsEcrLabels(ListAwsEcrLabels),
ListSecrets(ListSecrets),
ListGitProvidersFromConfig(ListGitProvidersFromConfig),
ListDockerRegistriesFromConfig(ListDockerRegistriesFromConfig),
@@ -80,9 +79,6 @@ enum ReadRequest {
GetUserGroup(GetUserGroup),
ListUserGroups(ListUserGroups),
// ==== SEARCH ====
FindResources(FindResources),
// ==== PROCEDURE ====
GetProceduresSummary(GetProceduresSummary),
GetProcedure(GetProcedure),
@@ -90,6 +86,13 @@ enum ReadRequest {
ListProcedures(ListProcedures),
ListFullProcedures(ListFullProcedures),
// ==== ACTION ====
GetActionsSummary(GetActionsSummary),
GetAction(GetAction),
GetActionActionState(GetActionActionState),
ListActions(ListActions),
ListFullActions(ListFullActions),
// ==== SERVER TEMPLATE ====
GetServerTemplate(GetServerTemplate),
GetServerTemplatesSummary(GetServerTemplatesSummary),
@@ -113,16 +116,14 @@ enum ReadRequest {
InspectDockerImage(InspectDockerImage),
ListDockerImageHistory(ListDockerImageHistory),
InspectDockerVolume(InspectDockerVolume),
#[to_string_resolver]
GetDockerContainersSummary(GetDockerContainersSummary),
ListAllDockerContainers(ListAllDockerContainers),
ListDockerContainers(ListDockerContainers),
#[to_string_resolver]
ListDockerNetworks(ListDockerNetworks),
#[to_string_resolver]
ListDockerImages(ListDockerImages),
#[to_string_resolver]
ListDockerVolumes(ListDockerVolumes),
#[to_string_resolver]
ListComposeProjects(ListComposeProjects),
ListTerminals(ListTerminals),
// ==== DEPLOYMENT ====
GetDeploymentsSummary(GetDeploymentsSummary),
@@ -168,8 +169,8 @@ enum ReadRequest {
GetStack(GetStack),
GetStackActionState(GetStackActionState),
GetStackWebhooksEnabled(GetStackWebhooksEnabled),
GetStackServiceLog(GetStackServiceLog),
SearchStackServiceLog(SearchStackServiceLog),
GetStackLog(GetStackLog),
SearchStackLog(SearchStackLog),
ListStacks(ListStacks),
ListFullStacks(ListFullStacks),
ListStackServices(ListStackServices),
@@ -205,11 +206,8 @@ enum ReadRequest {
GetAlert(GetAlert),
// ==== SERVER STATS ====
#[to_string_resolver]
GetSystemInformation(GetSystemInformation),
#[to_string_resolver]
GetSystemStats(GetSystemStats),
#[to_string_resolver]
ListSystemProcesses(ListSystemProcesses),
// ==== VARIABLE ====
@@ -233,127 +231,82 @@ pub fn router() -> Router {
async fn handler(
Extension(user): Extension<User>,
Json(request): Json<ReadRequest>,
) -> serror::Result<(TypedHeader<ContentType>, String)> {
) -> serror::Result<axum::response::Response> {
let timer = Instant::now();
let req_id = Uuid::new_v4();
debug!("/read request | user: {}", user.username);
let res =
State
.resolve_request(request, user)
.await
.map_err(|e| match e {
resolver_api::Error::Serialization(e) => {
anyhow!("{e:?}").context("response serialization error")
}
resolver_api::Error::Inner(e) => e,
});
let res = request.resolve(&ReadArgs { user }).await;
if let Err(e) = &res {
debug!("/read request {req_id} error: {e:#}");
debug!("/read request {req_id} error: {:#}", e.error);
}
let elapsed = timer.elapsed();
debug!("/read request {req_id} | resolve time: {elapsed:?}");
Ok((TypedHeader(ContentType::json()), res?))
res.map(|res| res.0)
}
fn version() -> &'static String {
static VERSION: OnceLock<String> = OnceLock::new();
VERSION.get_or_init(|| {
serde_json::to_string(&GetVersionResponse {
impl Resolve<ReadArgs> for GetVersion {
async fn resolve(
self,
_: &ReadArgs,
) -> serror::Result<GetVersionResponse> {
Ok(GetVersionResponse {
version: env!("CARGO_PKG_VERSION").to_string(),
})
.context("failed to serialize GetVersionResponse")
.unwrap()
})
}
impl ResolveToString<GetVersion, User> for State {
async fn resolve_to_string(
&self,
GetVersion {}: GetVersion,
_: User,
) -> anyhow::Result<String> {
Ok(version().to_string())
}
}
fn core_info() -> &'static String {
static CORE_INFO: OnceLock<String> = OnceLock::new();
fn core_info() -> &'static GetCoreInfoResponse {
static CORE_INFO: OnceLock<GetCoreInfoResponse> = OnceLock::new();
CORE_INFO.get_or_init(|| {
let config = core_config();
let info = GetCoreInfoResponse {
GetCoreInfoResponse {
title: config.title.clone(),
monitoring_interval: config.monitoring_interval,
webhook_base_url: config
.webhook_base_url
.clone()
.unwrap_or_else(|| config.host.clone()),
webhook_base_url: if config.webhook_base_url.is_empty() {
config.host.clone()
} else {
config.webhook_base_url.clone()
},
transparent_mode: config.transparent_mode,
ui_write_disabled: config.ui_write_disabled,
disable_confirm_dialog: config.disable_confirm_dialog,
disable_non_admin_create: config.disable_non_admin_create,
github_webhook_owners: config
.github_webhook_app
.installations
.iter()
.map(|i| i.namespace.to_string())
.collect(),
};
serde_json::to_string(&info)
.context("failed to serialize GetCoreInfoResponse")
.unwrap()
}
})
}
impl ResolveToString<GetCoreInfo, User> for State {
async fn resolve_to_string(
&self,
GetCoreInfo {}: GetCoreInfo,
_: User,
) -> anyhow::Result<String> {
Ok(core_info().to_string())
}
}
fn ecr_labels() -> &'static String {
static ECR_LABELS: OnceLock<String> = OnceLock::new();
ECR_LABELS.get_or_init(|| {
serde_json::to_string(
&core_config()
.aws_ecr_registries
.iter()
.map(|reg| reg.label.clone())
.collect::<Vec<_>>(),
)
.context("failed to serialize ecr registries")
.unwrap()
})
}
impl ResolveToString<ListAwsEcrLabels, User> for State {
async fn resolve_to_string(
&self,
ListAwsEcrLabels {}: ListAwsEcrLabels,
_: User,
) -> anyhow::Result<String> {
Ok(ecr_labels().to_string())
}
}
impl Resolve<ListSecrets, User> for State {
impl Resolve<ReadArgs> for GetCoreInfo {
async fn resolve(
&self,
ListSecrets { target }: ListSecrets,
_: User,
) -> anyhow::Result<ListSecretsResponse> {
self,
_: &ReadArgs,
) -> serror::Result<GetCoreInfoResponse> {
Ok(core_info().clone())
}
}
impl Resolve<ReadArgs> for ListSecrets {
async fn resolve(
self,
_: &ReadArgs,
) -> serror::Result<ListSecretsResponse> {
let mut secrets = core_config()
.secrets
.keys()
.cloned()
.collect::<HashSet<_>>();
if let Some(target) = target {
if let Some(target) = self.target {
let server_id = match target {
ResourceTarget::Server(id) => Some(id),
ResourceTarget::Builder(id) => {
match resource::get::<Builder>(&id).await?.config {
BuilderConfig::Url(_) => None,
BuilderConfig::Server(config) => Some(config.server_id),
BuilderConfig::Aws(config) => {
secrets.extend(config.secrets);
@@ -362,7 +315,9 @@ impl Resolve<ListSecrets, User> for State {
}
}
_ => {
return Err(anyhow!("target must be `Server` or `Builder`"))
return Err(
anyhow!("target must be `Server` or `Builder`").into(),
);
}
};
if let Some(id) = server_id {
@@ -387,21 +342,21 @@ impl Resolve<ListSecrets, User> for State {
}
}
impl Resolve<ListGitProvidersFromConfig, User> for State {
impl Resolve<ReadArgs> for ListGitProvidersFromConfig {
async fn resolve(
&self,
ListGitProvidersFromConfig { target }: ListGitProvidersFromConfig,
user: User,
) -> anyhow::Result<ListGitProvidersFromConfigResponse> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListGitProvidersFromConfigResponse> {
let mut providers = core_config().git_providers.clone();
if let Some(target) = target {
if let Some(target) = self.target {
match target {
ResourceTarget::Server(id) => {
merge_git_providers_for_server(&mut providers, &id).await?;
}
ResourceTarget::Builder(id) => {
match resource::get::<Builder>(&id).await?.config {
BuilderConfig::Url(_) => {}
BuilderConfig::Server(config) => {
merge_git_providers_for_server(
&mut providers,
@@ -418,7 +373,9 @@ impl Resolve<ListGitProvidersFromConfig, User> for State {
}
}
_ => {
return Err(anyhow!("target must be `Server` or `Builder`"))
return Err(
anyhow!("target must be `Server` or `Builder`").into(),
);
}
}
}
@@ -426,12 +383,18 @@ impl Resolve<ListGitProvidersFromConfig, User> for State {
let (builds, repos, syncs) = tokio::try_join!(
resource::list_full_for_user::<Build>(
Default::default(),
&user
user,
&[]
),
resource::list_full_for_user::<Repo>(
Default::default(),
user,
&[]
),
resource::list_full_for_user::<Repo>(Default::default(), &user),
resource::list_full_for_user::<ResourceSync>(
Default::default(),
&user
user,
&[]
),
)?;
@@ -478,15 +441,14 @@ impl Resolve<ListGitProvidersFromConfig, User> for State {
}
}
impl Resolve<ListDockerRegistriesFromConfig, User> for State {
impl Resolve<ReadArgs> for ListDockerRegistriesFromConfig {
async fn resolve(
&self,
ListDockerRegistriesFromConfig { target }: ListDockerRegistriesFromConfig,
_: User,
) -> anyhow::Result<ListDockerRegistriesFromConfigResponse> {
self,
_: &ReadArgs,
) -> serror::Result<ListDockerRegistriesFromConfigResponse> {
let mut registries = core_config().docker_registries.clone();
if let Some(target) = target {
if let Some(target) = self.target {
match target {
ResourceTarget::Server(id) => {
merge_docker_registries_for_server(&mut registries, &id)
@@ -494,6 +456,7 @@ impl Resolve<ListDockerRegistriesFromConfig, User> for State {
}
ResourceTarget::Builder(id) => {
match resource::get::<Builder>(&id).await?.config {
BuilderConfig::Url(_) => {}
BuilderConfig::Server(config) => {
merge_docker_registries_for_server(
&mut registries,
@@ -510,7 +473,9 @@ impl Resolve<ListDockerRegistriesFromConfig, User> for State {
}
}
_ => {
return Err(anyhow!("target must be `Server` or `Builder`"))
return Err(
anyhow!("target must be `Server` or `Builder`").into(),
);
}
}
}
@@ -524,7 +489,7 @@ impl Resolve<ListDockerRegistriesFromConfig, User> for State {
async fn merge_git_providers_for_server(
providers: &mut Vec<GitProvider>,
server_id: &str,
) -> anyhow::Result<()> {
) -> serror::Result<()> {
let server = resource::get::<Server>(server_id).await?;
let more = periphery_client(&server)?
.request(periphery_client::api::ListGitProviders {})
@@ -562,7 +527,7 @@ fn merge_git_providers(
async fn merge_docker_registries_for_server(
registries: &mut Vec<DockerRegistry>,
server_id: &str,
) -> anyhow::Result<()> {
) -> serror::Result<()> {
let server = resource::get::<Server>(server_id).await?;
let more = periphery_client(&server)?
.request(periphery_client::api::ListDockerRegistries {})

View File

@@ -1,28 +1,28 @@
use anyhow::{anyhow, Context};
use anyhow::{Context, anyhow};
use komodo_client::{
api::read::{
GetPermissionLevel, GetPermissionLevelResponse, ListPermissions,
ListPermissionsResponse, ListUserTargetPermissions,
ListUserTargetPermissionsResponse,
},
entities::{permission::PermissionLevel, user::User},
entities::permission::PermissionLevel,
};
use mungos::{find::find_collect, mongodb::bson::doc};
use resolver_api::Resolve;
use crate::{
helpers::query::get_user_permission_on_target,
state::{db_client, State},
helpers::query::get_user_permission_on_target, state::db_client,
};
impl Resolve<ListPermissions, User> for State {
use super::ReadArgs;
impl Resolve<ReadArgs> for ListPermissions {
async fn resolve(
&self,
ListPermissions {}: ListPermissions,
user: User,
) -> anyhow::Result<ListPermissionsResponse> {
find_collect(
&db_client().await.permissions,
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListPermissionsResponse> {
let res = find_collect(
&db_client().permissions,
doc! {
"user_target.type": "User",
"user_target.id": &user.id
@@ -30,35 +30,34 @@ impl Resolve<ListPermissions, User> for State {
None,
)
.await
.context("failed to query db for permissions")
.context("failed to query db for permissions")?;
Ok(res)
}
}
impl Resolve<GetPermissionLevel, User> for State {
impl Resolve<ReadArgs> for GetPermissionLevel {
async fn resolve(
&self,
GetPermissionLevel { target }: GetPermissionLevel,
user: User,
) -> anyhow::Result<GetPermissionLevelResponse> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<GetPermissionLevelResponse> {
if user.admin {
return Ok(PermissionLevel::Write);
}
get_user_permission_on_target(&user, &target).await
Ok(get_user_permission_on_target(user, &self.target).await?)
}
}
impl Resolve<ListUserTargetPermissions, User> for State {
impl Resolve<ReadArgs> for ListUserTargetPermissions {
async fn resolve(
&self,
ListUserTargetPermissions { user_target }: ListUserTargetPermissions,
user: User,
) -> anyhow::Result<ListUserTargetPermissionsResponse> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListUserTargetPermissionsResponse> {
if !user.admin {
return Err(anyhow!("this method is admin only"));
return Err(anyhow!("this method is admin only").into());
}
let (variant, id) = user_target.extract_variant_id();
find_collect(
&db_client().await.permissions,
let (variant, id) = self.user_target.extract_variant_id();
let res = find_collect(
&db_client().permissions,
doc! {
"user_target.type": variant.as_ref(),
"user_target.id": id
@@ -66,6 +65,7 @@ impl Resolve<ListUserTargetPermissions, User> for State {
None,
)
.await
.context("failed to query db for permissions")
.context("failed to query db for permissions")?;
Ok(res)
}
}

View File

@@ -4,60 +4,81 @@ use komodo_client::{
entities::{
permission::PermissionLevel,
procedure::{Procedure, ProcedureState},
user::User,
},
};
use resolver_api::Resolve;
use crate::{
helpers::query::get_all_tags,
resource,
state::{action_states, procedure_state_cache, State},
state::{action_states, procedure_state_cache},
};
impl Resolve<GetProcedure, User> for State {
use super::ReadArgs;
impl Resolve<ReadArgs> for GetProcedure {
async fn resolve(
&self,
GetProcedure { procedure }: GetProcedure,
user: User,
) -> anyhow::Result<GetProcedureResponse> {
resource::get_check_permissions::<Procedure>(
&procedure,
&user,
PermissionLevel::Read,
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<GetProcedureResponse> {
Ok(
resource::get_check_permissions::<Procedure>(
&self.procedure,
user,
PermissionLevel::Read,
)
.await?,
)
.await
}
}
impl Resolve<ListProcedures, User> for State {
impl Resolve<ReadArgs> for ListProcedures {
async fn resolve(
&self,
ListProcedures { query }: ListProcedures,
user: User,
) -> anyhow::Result<ListProceduresResponse> {
resource::list_for_user::<Procedure>(query, &user).await
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListProceduresResponse> {
let all_tags = if self.query.tags.is_empty() {
vec![]
} else {
get_all_tags(None).await?
};
Ok(
resource::list_for_user::<Procedure>(
self.query, user, &all_tags,
)
.await?,
)
}
}
impl Resolve<ListFullProcedures, User> for State {
impl Resolve<ReadArgs> for ListFullProcedures {
async fn resolve(
&self,
ListFullProcedures { query }: ListFullProcedures,
user: User,
) -> anyhow::Result<ListFullProceduresResponse> {
resource::list_full_for_user::<Procedure>(query, &user).await
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListFullProceduresResponse> {
let all_tags = if self.query.tags.is_empty() {
vec![]
} else {
get_all_tags(None).await?
};
Ok(
resource::list_full_for_user::<Procedure>(
self.query, user, &all_tags,
)
.await?,
)
}
}
impl Resolve<GetProceduresSummary, User> for State {
impl Resolve<ReadArgs> for GetProceduresSummary {
async fn resolve(
&self,
GetProceduresSummary {}: GetProceduresSummary,
user: User,
) -> anyhow::Result<GetProceduresSummaryResponse> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<GetProceduresSummaryResponse> {
let procedures = resource::list_full_for_user::<Procedure>(
Default::default(),
&user,
user,
&[],
)
.await
.context("failed to get procedures from db")?;
@@ -94,15 +115,14 @@ impl Resolve<GetProceduresSummary, User> for State {
}
}
impl Resolve<GetProcedureActionState, User> for State {
impl Resolve<ReadArgs> for GetProcedureActionState {
async fn resolve(
&self,
GetProcedureActionState { procedure }: GetProcedureActionState,
user: User,
) -> anyhow::Result<GetProcedureActionStateResponse> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<GetProcedureActionStateResponse> {
let procedure = resource::get_check_permissions::<Procedure>(
&procedure,
&user,
&self.procedure,
user,
PermissionLevel::Read,
)
.await?;

View File

@@ -1,116 +1,115 @@
use anyhow::{anyhow, Context};
use mongo_indexed::{doc, Document};
use komodo_client::{
api::read::{
GetDockerRegistryAccount, GetDockerRegistryAccountResponse,
GetGitProviderAccount, GetGitProviderAccountResponse,
ListDockerRegistryAccounts, ListDockerRegistryAccountsResponse,
ListGitProviderAccounts, ListGitProviderAccountsResponse,
},
entities::user::User,
};
use anyhow::{Context, anyhow};
use komodo_client::api::read::*;
use mongo_indexed::{Document, doc};
use mungos::{
by_id::find_one_by_id, find::find_collect,
mongodb::options::FindOptions,
};
use resolver_api::Resolve;
use crate::state::{db_client, State};
use crate::state::db_client;
impl Resolve<GetGitProviderAccount, User> for State {
use super::ReadArgs;
impl Resolve<ReadArgs> for GetGitProviderAccount {
async fn resolve(
&self,
GetGitProviderAccount { id }: GetGitProviderAccount,
user: User,
) -> anyhow::Result<GetGitProviderAccountResponse> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<GetGitProviderAccountResponse> {
if !user.admin {
return Err(anyhow!(
"Only admins can read git provider accounts"
));
return Err(
anyhow!("Only admins can read git provider accounts").into(),
);
}
find_one_by_id(&db_client().await.git_accounts, &id)
let res = find_one_by_id(&db_client().git_accounts, &self.id)
.await
.context("failed to query db for git provider accounts")?
.context("did not find git provider account with the given id")
}
}
impl Resolve<ListGitProviderAccounts, User> for State {
async fn resolve(
&self,
ListGitProviderAccounts { domain, username }: ListGitProviderAccounts,
user: User,
) -> anyhow::Result<ListGitProviderAccountsResponse> {
if !user.admin {
return Err(anyhow!(
"Only admins can read git provider accounts"
));
}
let mut filter = Document::new();
if let Some(domain) = domain {
filter.insert("domain", domain);
}
if let Some(username) = username {
filter.insert("username", username);
}
find_collect(
&db_client().await.git_accounts,
filter,
FindOptions::builder()
.sort(doc! { "domain": 1, "username": 1 })
.build(),
)
.await
.context("failed to query db for git provider accounts")
}
}
impl Resolve<GetDockerRegistryAccount, User> for State {
async fn resolve(
&self,
GetDockerRegistryAccount { id }: GetDockerRegistryAccount,
user: User,
) -> anyhow::Result<GetDockerRegistryAccountResponse> {
if !user.admin {
return Err(anyhow!(
"Only admins can read docker registry accounts"
));
}
find_one_by_id(&db_client().await.registry_accounts, &id)
.await
.context("failed to query db for docker registry accounts")?
.context(
"did not find docker registry account with the given id",
)
"did not find git provider account with the given id",
)?;
Ok(res)
}
}
impl Resolve<ListDockerRegistryAccounts, User> for State {
impl Resolve<ReadArgs> for ListGitProviderAccounts {
async fn resolve(
&self,
ListDockerRegistryAccounts { domain, username }: ListDockerRegistryAccounts,
user: User,
) -> anyhow::Result<ListDockerRegistryAccountsResponse> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListGitProviderAccountsResponse> {
if !user.admin {
return Err(anyhow!(
"Only admins can read docker registry accounts"
));
return Err(
anyhow!("Only admins can read git provider accounts").into(),
);
}
let mut filter = Document::new();
if let Some(domain) = domain {
if let Some(domain) = self.domain {
filter.insert("domain", domain);
}
if let Some(username) = username {
if let Some(username) = self.username {
filter.insert("username", username);
}
find_collect(
&db_client().await.registry_accounts,
let res = find_collect(
&db_client().git_accounts,
filter,
FindOptions::builder()
.sort(doc! { "domain": 1, "username": 1 })
.build(),
)
.await
.context("failed to query db for docker registry accounts")
.context("failed to query db for git provider accounts")?;
Ok(res)
}
}
impl Resolve<ReadArgs> for GetDockerRegistryAccount {
async fn resolve(
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<GetDockerRegistryAccountResponse> {
if !user.admin {
return Err(
anyhow!("Only admins can read docker registry accounts")
.into(),
);
}
let res =
find_one_by_id(&db_client().registry_accounts, &self.id)
.await
.context("failed to query db for docker registry accounts")?
.context(
"did not find docker registry account with the given id",
)?;
Ok(res)
}
}
impl Resolve<ReadArgs> for ListDockerRegistryAccounts {
async fn resolve(
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListDockerRegistryAccountsResponse> {
if !user.admin {
return Err(
anyhow!("Only admins can read docker registry accounts")
.into(),
);
}
let mut filter = Document::new();
if let Some(domain) = self.domain {
filter.insert("domain", domain);
}
if let Some(username) = self.username {
filter.insert("username", username);
}
let res = find_collect(
&db_client().registry_accounts,
filter,
FindOptions::builder()
.sort(doc! { "domain": 1, "username": 1 })
.build(),
)
.await
.context("failed to query db for docker registry accounts")?;
Ok(res)
}
}

View File

@@ -5,61 +5,79 @@ use komodo_client::{
config::core::CoreConfig,
permission::PermissionLevel,
repo::{Repo, RepoActionState, RepoListItem, RepoState},
user::User,
},
};
use resolver_api::Resolve;
use crate::{
config::core_config,
helpers::query::get_all_tags,
resource,
state::{action_states, github_client, repo_state_cache, State},
state::{action_states, github_client, repo_state_cache},
};
impl Resolve<GetRepo, User> for State {
use super::ReadArgs;
impl Resolve<ReadArgs> for GetRepo {
async fn resolve(
&self,
GetRepo { repo }: GetRepo,
user: User,
) -> anyhow::Result<Repo> {
resource::get_check_permissions::<Repo>(
&repo,
&user,
PermissionLevel::Read,
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<Repo> {
Ok(
resource::get_check_permissions::<Repo>(
&self.repo,
user,
PermissionLevel::Read,
)
.await?,
)
.await
}
}
impl Resolve<ListRepos, User> for State {
impl Resolve<ReadArgs> for ListRepos {
async fn resolve(
&self,
ListRepos { query }: ListRepos,
user: User,
) -> anyhow::Result<Vec<RepoListItem>> {
resource::list_for_user::<Repo>(query, &user).await
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<Vec<RepoListItem>> {
let all_tags = if self.query.tags.is_empty() {
vec![]
} else {
get_all_tags(None).await?
};
Ok(
resource::list_for_user::<Repo>(self.query, user, &all_tags)
.await?,
)
}
}
impl Resolve<ListFullRepos, User> for State {
impl Resolve<ReadArgs> for ListFullRepos {
async fn resolve(
&self,
ListFullRepos { query }: ListFullRepos,
user: User,
) -> anyhow::Result<ListFullReposResponse> {
resource::list_full_for_user::<Repo>(query, &user).await
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListFullReposResponse> {
let all_tags = if self.query.tags.is_empty() {
vec![]
} else {
get_all_tags(None).await?
};
Ok(
resource::list_full_for_user::<Repo>(
self.query, user, &all_tags,
)
.await?,
)
}
}
impl Resolve<GetRepoActionState, User> for State {
impl Resolve<ReadArgs> for GetRepoActionState {
async fn resolve(
&self,
GetRepoActionState { repo }: GetRepoActionState,
user: User,
) -> anyhow::Result<RepoActionState> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<RepoActionState> {
let repo = resource::get_check_permissions::<Repo>(
&repo,
&user,
&self.repo,
user,
PermissionLevel::Read,
)
.await?;
@@ -73,16 +91,18 @@ impl Resolve<GetRepoActionState, User> for State {
}
}
impl Resolve<GetReposSummary, User> for State {
impl Resolve<ReadArgs> for GetReposSummary {
async fn resolve(
&self,
GetReposSummary {}: GetReposSummary,
user: User,
) -> anyhow::Result<GetReposSummaryResponse> {
let repos =
resource::list_full_for_user::<Repo>(Default::default(), &user)
.await
.context("failed to get repos from db")?;
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<GetReposSummaryResponse> {
let repos = resource::list_full_for_user::<Repo>(
Default::default(),
user,
&[],
)
.await
.context("failed to get repos from db")?;
let mut res = GetReposSummaryResponse::default();
@@ -126,12 +146,11 @@ impl Resolve<GetReposSummary, User> for State {
}
}
impl Resolve<GetRepoWebhooksEnabled, User> for State {
impl Resolve<ReadArgs> for GetRepoWebhooksEnabled {
async fn resolve(
&self,
GetRepoWebhooksEnabled { repo }: GetRepoWebhooksEnabled,
user: User,
) -> anyhow::Result<GetRepoWebhooksEnabledResponse> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<GetRepoWebhooksEnabledResponse> {
let Some(github) = github_client() else {
return Ok(GetRepoWebhooksEnabledResponse {
managed: false,
@@ -142,8 +161,8 @@ impl Resolve<GetRepoWebhooksEnabled, User> for State {
};
let repo = resource::get_check_permissions::<Repo>(
&repo,
&user,
&self.repo,
user,
PermissionLevel::Read,
)
.await?;
@@ -188,7 +207,11 @@ impl Resolve<GetRepoWebhooksEnabled, User> for State {
..
} = core_config();
let host = webhook_base_url.as_ref().unwrap_or(host);
let host = if webhook_base_url.is_empty() {
host
} else {
webhook_base_url
};
let clone_url =
format!("{host}/listener/github/repo/{}/clone", repo.id);
let pull_url =

View File

@@ -1,82 +0,0 @@
use komodo_client::{
api::read::{FindResources, FindResourcesResponse},
entities::{
build::Build, deployment::Deployment, procedure::Procedure,
repo::Repo, server::Server, user::User, ResourceTargetVariant,
},
};
use resolver_api::Resolve;
use crate::{resource, state::State};
const FIND_RESOURCE_TYPES: [ResourceTargetVariant; 5] = [
ResourceTargetVariant::Server,
ResourceTargetVariant::Build,
ResourceTargetVariant::Deployment,
ResourceTargetVariant::Repo,
ResourceTargetVariant::Procedure,
];
impl Resolve<FindResources, User> for State {
async fn resolve(
&self,
FindResources { query, resources }: FindResources,
user: User,
) -> anyhow::Result<FindResourcesResponse> {
let mut res = FindResourcesResponse::default();
let resource_types = if resources.is_empty() {
FIND_RESOURCE_TYPES.to_vec()
} else {
resources
.into_iter()
.filter(|r| {
!matches!(
r,
ResourceTargetVariant::System
| ResourceTargetVariant::Builder
| ResourceTargetVariant::Alerter
)
})
.collect()
};
for resource_type in resource_types {
match resource_type {
ResourceTargetVariant::Server => {
res.servers = resource::list_for_user_using_document::<
Server,
>(query.clone(), &user)
.await?;
}
ResourceTargetVariant::Deployment => {
res.deployments = resource::list_for_user_using_document::<
Deployment,
>(query.clone(), &user)
.await?;
}
ResourceTargetVariant::Build => {
res.builds =
resource::list_for_user_using_document::<Build>(
query.clone(),
&user,
)
.await?;
}
ResourceTargetVariant::Repo => {
res.repos = resource::list_for_user_using_document::<Repo>(
query.clone(),
&user,
)
.await?;
}
ResourceTargetVariant::Procedure => {
res.procedures = resource::list_for_user_using_document::<
Procedure,
>(query.clone(), &user)
.await?;
}
_ => {}
}
}
Ok(res)
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -1,64 +1,82 @@
use anyhow::Context;
use mongo_indexed::Document;
use komodo_client::{
api::read::*,
entities::{
permission::PermissionLevel, server_template::ServerTemplate,
user::User,
},
};
use mongo_indexed::Document;
use mungos::mongodb::bson::doc;
use resolver_api::Resolve;
use crate::{
resource,
state::{db_client, State},
helpers::query::get_all_tags, resource, state::db_client,
};
impl Resolve<GetServerTemplate, User> for State {
use super::ReadArgs;
impl Resolve<ReadArgs> for GetServerTemplate {
async fn resolve(
&self,
GetServerTemplate { server_template }: GetServerTemplate,
user: User,
) -> anyhow::Result<GetServerTemplateResponse> {
resource::get_check_permissions::<ServerTemplate>(
&server_template,
&user,
PermissionLevel::Read,
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<GetServerTemplateResponse> {
Ok(
resource::get_check_permissions::<ServerTemplate>(
&self.server_template,
user,
PermissionLevel::Read,
)
.await?,
)
.await
}
}
impl Resolve<ListServerTemplates, User> for State {
impl Resolve<ReadArgs> for ListServerTemplates {
async fn resolve(
&self,
ListServerTemplates { query }: ListServerTemplates,
user: User,
) -> anyhow::Result<ListServerTemplatesResponse> {
resource::list_for_user::<ServerTemplate>(query, &user).await
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListServerTemplatesResponse> {
let all_tags = if self.query.tags.is_empty() {
vec![]
} else {
get_all_tags(None).await?
};
Ok(
resource::list_for_user::<ServerTemplate>(
self.query, user, &all_tags,
)
.await?,
)
}
}
impl Resolve<ListFullServerTemplates, User> for State {
impl Resolve<ReadArgs> for ListFullServerTemplates {
async fn resolve(
&self,
ListFullServerTemplates { query }: ListFullServerTemplates,
user: User,
) -> anyhow::Result<ListFullServerTemplatesResponse> {
resource::list_full_for_user::<ServerTemplate>(query, &user).await
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListFullServerTemplatesResponse> {
let all_tags = if self.query.tags.is_empty() {
vec![]
} else {
get_all_tags(None).await?
};
Ok(
resource::list_full_for_user::<ServerTemplate>(
self.query, user, &all_tags,
)
.await?,
)
}
}
impl Resolve<GetServerTemplatesSummary, User> for State {
impl Resolve<ReadArgs> for GetServerTemplatesSummary {
async fn resolve(
&self,
GetServerTemplatesSummary {}: GetServerTemplatesSummary,
user: User,
) -> anyhow::Result<GetServerTemplatesSummaryResponse> {
let query = match resource::get_resource_ids_for_user::<
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<GetServerTemplatesSummaryResponse> {
let query = match resource::get_resource_object_ids_for_user::<
ServerTemplate,
>(&user)
>(user)
.await?
{
Some(ids) => doc! {
@@ -67,7 +85,6 @@ impl Resolve<GetServerTemplatesSummary, User> for State {
None => Document::new(),
};
let total = db_client()
.await
.server_templates
.count_documents(query)
.await

View File

@@ -7,45 +7,47 @@ use komodo_client::{
config::core::CoreConfig,
permission::PermissionLevel,
stack::{Stack, StackActionState, StackListItem, StackState},
user::User,
},
};
use periphery_client::api::compose::{
GetComposeServiceLog, GetComposeServiceLogSearch,
GetComposeLog, GetComposeLogSearch,
};
use resolver_api::Resolve;
use crate::{
config::core_config,
helpers::{periphery_client, stack::get_stack_and_server},
helpers::{periphery_client, query::get_all_tags},
resource,
state::{action_states, github_client, stack_status_cache, State},
stack::get_stack_and_server,
state::{action_states, github_client, stack_status_cache},
};
impl Resolve<GetStack, User> for State {
use super::ReadArgs;
impl Resolve<ReadArgs> for GetStack {
async fn resolve(
&self,
GetStack { stack }: GetStack,
user: User,
) -> anyhow::Result<Stack> {
resource::get_check_permissions::<Stack>(
&stack,
&user,
PermissionLevel::Read,
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<Stack> {
Ok(
resource::get_check_permissions::<Stack>(
&self.stack,
user,
PermissionLevel::Read,
)
.await?,
)
.await
}
}
impl Resolve<ListStackServices, User> for State {
impl Resolve<ReadArgs> for ListStackServices {
async fn resolve(
&self,
ListStackServices { stack }: ListStackServices,
user: User,
) -> anyhow::Result<ListStackServicesResponse> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListStackServicesResponse> {
let stack = resource::get_check_permissions::<Stack>(
&stack,
&user,
&self.stack,
user,
PermissionLevel::Read,
)
.await?;
@@ -62,75 +64,79 @@ impl Resolve<ListStackServices, User> for State {
}
}
impl Resolve<GetStackServiceLog, User> for State {
impl Resolve<ReadArgs> for GetStackLog {
async fn resolve(
&self,
GetStackServiceLog {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<GetStackLogResponse> {
let GetStackLog {
stack,
service,
services,
tail,
}: GetStackServiceLog,
user: User,
) -> anyhow::Result<GetStackServiceLogResponse> {
let (stack, server) = get_stack_and_server(
&stack,
&user,
PermissionLevel::Read,
true,
)
.await?;
periphery_client(&server)?
.request(GetComposeServiceLog {
timestamps,
} = self;
let (stack, server) =
get_stack_and_server(&stack, user, PermissionLevel::Read, true)
.await?;
let res = periphery_client(&server)?
.request(GetComposeLog {
project: stack.project_name(false),
service,
services,
tail,
timestamps,
})
.await
.context("failed to get stack service log from periphery")
.context("Failed to get stack log from periphery")?;
Ok(res)
}
}
impl Resolve<SearchStackServiceLog, User> for State {
impl Resolve<ReadArgs> for SearchStackLog {
async fn resolve(
&self,
SearchStackServiceLog {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<SearchStackLogResponse> {
let SearchStackLog {
stack,
service,
services,
terms,
combinator,
invert,
}: SearchStackServiceLog,
user: User,
) -> anyhow::Result<SearchStackServiceLogResponse> {
let (stack, server) = get_stack_and_server(
&stack,
&user,
PermissionLevel::Read,
true,
)
.await?;
periphery_client(&server)?
.request(GetComposeServiceLogSearch {
timestamps,
} = self;
let (stack, server) =
get_stack_and_server(&stack, user, PermissionLevel::Read, true)
.await?;
let res = periphery_client(&server)?
.request(GetComposeLogSearch {
project: stack.project_name(false),
service,
services,
terms,
combinator,
invert,
timestamps,
})
.await
.context("failed to get stack service log from periphery")
.context("Failed to search stack log from periphery")?;
Ok(res)
}
}
impl Resolve<ListCommonStackExtraArgs, User> for State {
impl Resolve<ReadArgs> for ListCommonStackExtraArgs {
async fn resolve(
&self,
ListCommonStackExtraArgs { query }: ListCommonStackExtraArgs,
user: User,
) -> anyhow::Result<ListCommonStackExtraArgsResponse> {
let stacks = resource::list_full_for_user::<Stack>(query, &user)
.await
.context("failed to get resources matching query")?;
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListCommonStackExtraArgsResponse> {
let all_tags = if self.query.tags.is_empty() {
vec![]
} else {
get_all_tags(None).await?
};
let stacks = resource::list_full_for_user::<Stack>(
self.query, user, &all_tags,
)
.await
.context("failed to get resources matching query")?;
// first collect with guaranteed uniqueness
let mut res = HashSet::<String>::new();
@@ -147,15 +153,21 @@ impl Resolve<ListCommonStackExtraArgs, User> for State {
}
}
impl Resolve<ListCommonStackBuildExtraArgs, User> for State {
impl Resolve<ReadArgs> for ListCommonStackBuildExtraArgs {
async fn resolve(
&self,
ListCommonStackBuildExtraArgs { query }: ListCommonStackBuildExtraArgs,
user: User,
) -> anyhow::Result<ListCommonStackBuildExtraArgsResponse> {
let stacks = resource::list_full_for_user::<Stack>(query, &user)
.await
.context("failed to get resources matching query")?;
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListCommonStackBuildExtraArgsResponse> {
let all_tags = if self.query.tags.is_empty() {
vec![]
} else {
get_all_tags(None).await?
};
let stacks = resource::list_full_for_user::<Stack>(
self.query, user, &all_tags,
)
.await
.context("failed to get resources matching query")?;
// first collect with guaranteed uniqueness
let mut res = HashSet::<String>::new();
@@ -172,35 +184,65 @@ impl Resolve<ListCommonStackBuildExtraArgs, User> for State {
}
}
impl Resolve<ListStacks, User> for State {
impl Resolve<ReadArgs> for ListStacks {
async fn resolve(
&self,
ListStacks { query }: ListStacks,
user: User,
) -> anyhow::Result<Vec<StackListItem>> {
resource::list_for_user::<Stack>(query, &user).await
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<Vec<StackListItem>> {
let all_tags = if self.query.tags.is_empty() {
vec![]
} else {
get_all_tags(None).await?
};
let only_update_available = self.query.specific.update_available;
let stacks =
resource::list_for_user::<Stack>(self.query, user, &all_tags)
.await?;
let stacks = if only_update_available {
stacks
.into_iter()
.filter(|stack| {
stack
.info
.services
.iter()
.any(|service| service.update_available)
})
.collect()
} else {
stacks
};
Ok(stacks)
}
}
impl Resolve<ListFullStacks, User> for State {
impl Resolve<ReadArgs> for ListFullStacks {
async fn resolve(
&self,
ListFullStacks { query }: ListFullStacks,
user: User,
) -> anyhow::Result<ListFullStacksResponse> {
resource::list_full_for_user::<Stack>(query, &user).await
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListFullStacksResponse> {
let all_tags = if self.query.tags.is_empty() {
vec![]
} else {
get_all_tags(None).await?
};
Ok(
resource::list_full_for_user::<Stack>(
self.query, user, &all_tags,
)
.await?,
)
}
}
impl Resolve<GetStackActionState, User> for State {
impl Resolve<ReadArgs> for GetStackActionState {
async fn resolve(
&self,
GetStackActionState { stack }: GetStackActionState,
user: User,
) -> anyhow::Result<StackActionState> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<StackActionState> {
let stack = resource::get_check_permissions::<Stack>(
&stack,
&user,
&self.stack,
user,
PermissionLevel::Read,
)
.await?;
@@ -214,15 +256,15 @@ impl Resolve<GetStackActionState, User> for State {
}
}
impl Resolve<GetStacksSummary, User> for State {
impl Resolve<ReadArgs> for GetStacksSummary {
async fn resolve(
&self,
GetStacksSummary {}: GetStacksSummary,
user: User,
) -> anyhow::Result<GetStacksSummaryResponse> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<GetStacksSummaryResponse> {
let stacks = resource::list_full_for_user::<Stack>(
Default::default(),
&user,
user,
&[],
)
.await
.context("failed to get stacks from db")?;
@@ -236,15 +278,10 @@ impl Resolve<GetStacksSummary, User> for State {
match cache.get(&stack.id).await.unwrap_or_default().curr.state
{
StackState::Running => res.running += 1,
StackState::Paused => res.paused += 1,
StackState::Stopped => res.stopped += 1,
StackState::Restarting => res.restarting += 1,
StackState::Created => res.created += 1,
StackState::Removing => res.removing += 1,
StackState::Dead => res.dead += 1,
StackState::Unhealthy => res.unhealthy += 1,
StackState::Stopped | StackState::Paused => res.stopped += 1,
StackState::Down => res.down += 1,
StackState::Unknown => res.unknown += 1,
_ => res.unhealthy += 1,
}
}
@@ -252,12 +289,11 @@ impl Resolve<GetStacksSummary, User> for State {
}
}
impl Resolve<GetStackWebhooksEnabled, User> for State {
impl Resolve<ReadArgs> for GetStackWebhooksEnabled {
async fn resolve(
&self,
GetStackWebhooksEnabled { stack }: GetStackWebhooksEnabled,
user: User,
) -> anyhow::Result<GetStackWebhooksEnabledResponse> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<GetStackWebhooksEnabledResponse> {
let Some(github) = github_client() else {
return Ok(GetStackWebhooksEnabledResponse {
managed: false,
@@ -267,8 +303,8 @@ impl Resolve<GetStackWebhooksEnabled, User> for State {
};
let stack = resource::get_check_permissions::<Stack>(
&stack,
&user,
&self.stack,
user,
PermissionLevel::Read,
)
.await?;
@@ -311,7 +347,11 @@ impl Resolve<GetStackWebhooksEnabled, User> for State {
..
} = core_config();
let host = webhook_base_url.as_ref().unwrap_or(host);
let host = if webhook_base_url.is_empty() {
host
} else {
webhook_base_url
};
let refresh_url =
format!("{host}/listener/github/stack/{}/refresh", stack.id);
let deploy_url =

View File

@@ -5,66 +5,83 @@ use komodo_client::{
config::core::CoreConfig,
permission::PermissionLevel,
sync::{
PendingSyncUpdatesData, ResourceSync, ResourceSyncActionState,
ResourceSyncListItem, ResourceSyncState,
ResourceSync, ResourceSyncActionState, ResourceSyncListItem,
},
user::User,
},
};
use resolver_api::Resolve;
use crate::{
config::core_config,
helpers::query::get_all_tags,
resource,
state::{
action_states, github_client, resource_sync_state_cache, State,
},
state::{action_states, github_client},
};
impl Resolve<GetResourceSync, User> for State {
use super::ReadArgs;
impl Resolve<ReadArgs> for GetResourceSync {
async fn resolve(
&self,
GetResourceSync { sync }: GetResourceSync,
user: User,
) -> anyhow::Result<ResourceSync> {
resource::get_check_permissions::<ResourceSync>(
&sync,
&user,
PermissionLevel::Read,
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ResourceSync> {
Ok(
resource::get_check_permissions::<ResourceSync>(
&self.sync,
user,
PermissionLevel::Read,
)
.await?,
)
.await
}
}
impl Resolve<ListResourceSyncs, User> for State {
impl Resolve<ReadArgs> for ListResourceSyncs {
async fn resolve(
&self,
ListResourceSyncs { query }: ListResourceSyncs,
user: User,
) -> anyhow::Result<Vec<ResourceSyncListItem>> {
resource::list_for_user::<ResourceSync>(query, &user).await
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<Vec<ResourceSyncListItem>> {
let all_tags = if self.query.tags.is_empty() {
vec![]
} else {
get_all_tags(None).await?
};
Ok(
resource::list_for_user::<ResourceSync>(
self.query, user, &all_tags,
)
.await?,
)
}
}
impl Resolve<ListFullResourceSyncs, User> for State {
impl Resolve<ReadArgs> for ListFullResourceSyncs {
async fn resolve(
&self,
ListFullResourceSyncs { query }: ListFullResourceSyncs,
user: User,
) -> anyhow::Result<ListFullResourceSyncsResponse> {
resource::list_full_for_user::<ResourceSync>(query, &user).await
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListFullResourceSyncsResponse> {
let all_tags = if self.query.tags.is_empty() {
vec![]
} else {
get_all_tags(None).await?
};
Ok(
resource::list_full_for_user::<ResourceSync>(
self.query, user, &all_tags,
)
.await?,
)
}
}
impl Resolve<GetResourceSyncActionState, User> for State {
impl Resolve<ReadArgs> for GetResourceSyncActionState {
async fn resolve(
&self,
GetResourceSyncActionState { sync }: GetResourceSyncActionState,
user: User,
) -> anyhow::Result<ResourceSyncActionState> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ResourceSyncActionState> {
let sync = resource::get_check_permissions::<ResourceSync>(
&sync,
&user,
&self.sync,
user,
PermissionLevel::Read,
)
.await?;
@@ -78,76 +95,63 @@ impl Resolve<GetResourceSyncActionState, User> for State {
}
}
impl Resolve<GetResourceSyncsSummary, User> for State {
impl Resolve<ReadArgs> for GetResourceSyncsSummary {
async fn resolve(
&self,
GetResourceSyncsSummary {}: GetResourceSyncsSummary,
user: User,
) -> anyhow::Result<GetResourceSyncsSummaryResponse> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<GetResourceSyncsSummaryResponse> {
let resource_syncs =
resource::list_full_for_user::<ResourceSync>(
Default::default(),
&user,
user,
&[],
)
.await
.context("failed to get resource_syncs from db")?;
let mut res = GetResourceSyncsSummaryResponse::default();
let cache = resource_sync_state_cache();
let action_states = action_states();
for resource_sync in resource_syncs {
res.total += 1;
match resource_sync.info.pending.data {
PendingSyncUpdatesData::Ok(data) => {
if !data.no_updates() {
res.pending += 1;
continue;
}
}
PendingSyncUpdatesData::Err(_) => {
res.failed += 1;
continue;
}
if !(resource_sync.info.pending_deploy.to_deploy == 0
&& resource_sync.info.resource_updates.is_empty()
&& resource_sync.info.variable_updates.is_empty()
&& resource_sync.info.user_group_updates.is_empty())
{
res.pending += 1;
continue;
} else if resource_sync.info.pending_error.is_some()
|| !resource_sync.info.remote_errors.is_empty()
{
res.failed += 1;
continue;
}
match (
cache.get(&resource_sync.id).await.unwrap_or_default(),
action_states
.resource_sync
.get(&resource_sync.id)
.await
.unwrap_or_default()
.get()?,
) {
(_, action_states) if action_states.syncing => {
res.syncing += 1;
}
(ResourceSyncState::Ok, _) => res.ok += 1,
(ResourceSyncState::Failed, _) => res.failed += 1,
(ResourceSyncState::Unknown, _) => res.unknown += 1,
// will never come off the cache in the building state, since that comes from action states
(ResourceSyncState::Syncing, _) => {
unreachable!()
}
(ResourceSyncState::Pending, _) => {
unreachable!()
}
if action_states
.resource_sync
.get(&resource_sync.id)
.await
.unwrap_or_default()
.get()?
.syncing
{
res.syncing += 1;
continue;
}
res.ok += 1;
}
Ok(res)
}
}
impl Resolve<GetSyncWebhooksEnabled, User> for State {
impl Resolve<ReadArgs> for GetSyncWebhooksEnabled {
async fn resolve(
&self,
GetSyncWebhooksEnabled { sync }: GetSyncWebhooksEnabled,
user: User,
) -> anyhow::Result<GetSyncWebhooksEnabledResponse> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<GetSyncWebhooksEnabledResponse> {
let Some(github) = github_client() else {
return Ok(GetSyncWebhooksEnabledResponse {
managed: false,
@@ -157,8 +161,8 @@ impl Resolve<GetSyncWebhooksEnabled, User> for State {
};
let sync = resource::get_check_permissions::<ResourceSync>(
&sync,
&user,
&self.sync,
user,
PermissionLevel::Read,
)
.await?;
@@ -201,7 +205,11 @@ impl Resolve<GetSyncWebhooksEnabled, User> for State {
..
} = core_config();
let host = webhook_base_url.as_ref().unwrap_or(host);
let host = if webhook_base_url.is_empty() {
host
} else {
webhook_base_url
};
let refresh_url =
format!("{host}/listener/github/sync/{}/refresh", sync.id);
let sync_url =

View File

@@ -1,39 +1,31 @@
use anyhow::Context;
use mongo_indexed::doc;
use komodo_client::{
api::read::{GetTag, ListTags},
entities::{tag::Tag, user::User},
entities::tag::Tag,
};
use mongo_indexed::doc;
use mungos::{find::find_collect, mongodb::options::FindOptions};
use resolver_api::Resolve;
use crate::{
helpers::query::get_tag,
state::{db_client, State},
};
use crate::{helpers::query::get_tag, state::db_client};
impl Resolve<GetTag, User> for State {
async fn resolve(
&self,
GetTag { tag }: GetTag,
_: User,
) -> anyhow::Result<Tag> {
get_tag(&tag).await
use super::ReadArgs;
impl Resolve<ReadArgs> for GetTag {
async fn resolve(self, _: &ReadArgs) -> serror::Result<Tag> {
Ok(get_tag(&self.tag).await?)
}
}
impl Resolve<ListTags, User> for State {
async fn resolve(
&self,
ListTags { query }: ListTags,
_: User,
) -> anyhow::Result<Vec<Tag>> {
find_collect(
&db_client().await.tags,
query,
impl Resolve<ReadArgs> for ListTags {
async fn resolve(self, _: &ReadArgs) -> serror::Result<Vec<Tag>> {
let res = find_collect(
&db_client().tags,
self.query,
FindOptions::builder().sort(doc! { "name": 1 }).build(),
)
.await
.context("failed to get tags from db")
.context("failed to get tags from db")?;
Ok(res)
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -1,9 +1,11 @@
use std::collections::HashMap;
use anyhow::{anyhow, Context};
use anyhow::{Context, anyhow};
use komodo_client::{
api::read::{GetUpdate, ListUpdates, ListUpdatesResponse},
entities::{
ResourceTarget,
action::Action,
alerter::Alerter,
build::Build,
builder::Builder,
@@ -17,7 +19,6 @@ use komodo_client::{
sync::ResourceSync,
update::{Update, UpdateListItem},
user::User,
ResourceTarget,
},
};
use mungos::{
@@ -27,25 +28,22 @@ use mungos::{
};
use resolver_api::Resolve;
use crate::{
config::core_config,
resource,
state::{db_client, State},
};
use crate::{config::core_config, resource, state::db_client};
use super::ReadArgs;
const UPDATES_PER_PAGE: i64 = 100;
impl Resolve<ListUpdates, User> for State {
impl Resolve<ReadArgs> for ListUpdates {
async fn resolve(
&self,
ListUpdates { query, page }: ListUpdates,
user: User,
) -> anyhow::Result<ListUpdatesResponse> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListUpdatesResponse> {
let query = if user.admin || core_config().transparent_mode {
query
self.query
} else {
let server_query =
resource::get_resource_ids_for_user::<Server>(&user)
resource::get_resource_ids_for_user::<Server>(user)
.await?
.map(|ids| {
doc! {
@@ -55,7 +53,7 @@ impl Resolve<ListUpdates, User> for State {
.unwrap_or_else(|| doc! { "target.type": "Server" });
let deployment_query =
resource::get_resource_ids_for_user::<Deployment>(&user)
resource::get_resource_ids_for_user::<Deployment>(user)
.await?
.map(|ids| {
doc! {
@@ -65,7 +63,7 @@ impl Resolve<ListUpdates, User> for State {
.unwrap_or_else(|| doc! { "target.type": "Deployment" });
let stack_query =
resource::get_resource_ids_for_user::<Stack>(&user)
resource::get_resource_ids_for_user::<Stack>(user)
.await?
.map(|ids| {
doc! {
@@ -75,7 +73,7 @@ impl Resolve<ListUpdates, User> for State {
.unwrap_or_else(|| doc! { "target.type": "Stack" });
let build_query =
resource::get_resource_ids_for_user::<Build>(&user)
resource::get_resource_ids_for_user::<Build>(user)
.await?
.map(|ids| {
doc! {
@@ -85,7 +83,7 @@ impl Resolve<ListUpdates, User> for State {
.unwrap_or_else(|| doc! { "target.type": "Build" });
let repo_query =
resource::get_resource_ids_for_user::<Repo>(&user)
resource::get_resource_ids_for_user::<Repo>(user)
.await?
.map(|ids| {
doc! {
@@ -95,7 +93,7 @@ impl Resolve<ListUpdates, User> for State {
.unwrap_or_else(|| doc! { "target.type": "Repo" });
let procedure_query =
resource::get_resource_ids_for_user::<Procedure>(&user)
resource::get_resource_ids_for_user::<Procedure>(user)
.await?
.map(|ids| {
doc! {
@@ -104,8 +102,18 @@ impl Resolve<ListUpdates, User> for State {
})
.unwrap_or_else(|| doc! { "target.type": "Procedure" });
let action_query =
resource::get_resource_ids_for_user::<Action>(user)
.await?
.map(|ids| {
doc! {
"target.type": "Action", "target.id": { "$in": ids }
}
})
.unwrap_or_else(|| doc! { "target.type": "Action" });
let builder_query =
resource::get_resource_ids_for_user::<Builder>(&user)
resource::get_resource_ids_for_user::<Builder>(user)
.await?
.map(|ids| {
doc! {
@@ -115,7 +123,7 @@ impl Resolve<ListUpdates, User> for State {
.unwrap_or_else(|| doc! { "target.type": "Builder" });
let alerter_query =
resource::get_resource_ids_for_user::<Alerter>(&user)
resource::get_resource_ids_for_user::<Alerter>(user)
.await?
.map(|ids| {
doc! {
@@ -124,29 +132,29 @@ impl Resolve<ListUpdates, User> for State {
})
.unwrap_or_else(|| doc! { "target.type": "Alerter" });
let server_template_query = resource::get_resource_ids_for_user::<ServerTemplate>(
&user,
)
.await?
.map(|ids| {
doc! {
"target.type": "ServerTemplate", "target.id": { "$in": ids }
}
})
.unwrap_or_else(|| doc! { "target.type": "ServerTemplate" });
let server_template_query =
resource::get_resource_ids_for_user::<ServerTemplate>(user)
.await?
.map(|ids| {
doc! {
"target.type": "ServerTemplate", "target.id": { "$in": ids }
}
})
.unwrap_or_else(|| doc! { "target.type": "ServerTemplate" });
let resource_sync_query = resource::get_resource_ids_for_user::<ResourceSync>(
&user,
)
.await?
.map(|ids| {
doc! {
"target.type": "ResourceSync", "target.id": { "$in": ids }
}
})
.unwrap_or_else(|| doc! { "target.type": "ResourceSync" });
let resource_sync_query =
resource::get_resource_ids_for_user::<ResourceSync>(
user,
)
.await?
.map(|ids| {
doc! {
"target.type": "ResourceSync", "target.id": { "$in": ids }
}
})
.unwrap_or_else(|| doc! { "target.type": "ResourceSync" });
let mut query = query.unwrap_or_default();
let mut query = self.query.unwrap_or_default();
query.extend(doc! {
"$or": [
server_query,
@@ -155,6 +163,7 @@ impl Resolve<ListUpdates, User> for State {
build_query,
repo_query,
procedure_query,
action_query,
alerter_query,
builder_query,
server_template_query,
@@ -164,20 +173,19 @@ impl Resolve<ListUpdates, User> for State {
query.into()
};
let usernames =
find_collect(&db_client().await.users, None, None)
.await
.context("failed to pull users from db")?
.into_iter()
.map(|u| (u.id, u.username))
.collect::<HashMap<_, _>>();
let usernames = find_collect(&db_client().users, None, None)
.await
.context("failed to pull users from db")?
.into_iter()
.map(|u| (u.id, u.username))
.collect::<HashMap<_, _>>();
let updates = find_collect(
&db_client().await.updates,
&db_client().updates,
query,
FindOptions::builder()
.sort(doc! { "start_ts": -1 })
.skip(page as u64 * UPDATES_PER_PAGE as u64)
.skip(self.page as u64 * UPDATES_PER_PAGE as u64)
.limit(UPDATES_PER_PAGE)
.build(),
)
@@ -209,7 +217,7 @@ impl Resolve<ListUpdates, User> for State {
.collect::<Vec<_>>();
let next_page = if updates.len() == UPDATES_PER_PAGE as usize {
Some(page + 1)
Some(self.page + 1)
} else {
None
};
@@ -218,13 +226,12 @@ impl Resolve<ListUpdates, User> for State {
}
}
impl Resolve<GetUpdate, User> for State {
impl Resolve<ReadArgs> for GetUpdate {
async fn resolve(
&self,
GetUpdate { id }: GetUpdate,
user: User,
) -> anyhow::Result<Update> {
let update = find_one_by_id(&db_client().await.updates, &id)
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<Update> {
let update = find_one_by_id(&db_client().updates, &self.id)
.await
.context("failed to query to db")?
.context("no update exists with given id")?;
@@ -233,14 +240,14 @@ impl Resolve<GetUpdate, User> for State {
}
match &update.target {
ResourceTarget::System(_) => {
return Err(anyhow!(
"user must be admin to view system updates"
))
return Err(
anyhow!("user must be admin to view system updates").into(),
);
}
ResourceTarget::Server(id) => {
resource::get_check_permissions::<Server>(
id,
&user,
user,
PermissionLevel::Read,
)
.await?;
@@ -248,7 +255,7 @@ impl Resolve<GetUpdate, User> for State {
ResourceTarget::Deployment(id) => {
resource::get_check_permissions::<Deployment>(
id,
&user,
user,
PermissionLevel::Read,
)
.await?;
@@ -256,7 +263,7 @@ impl Resolve<GetUpdate, User> for State {
ResourceTarget::Build(id) => {
resource::get_check_permissions::<Build>(
id,
&user,
user,
PermissionLevel::Read,
)
.await?;
@@ -264,7 +271,7 @@ impl Resolve<GetUpdate, User> for State {
ResourceTarget::Repo(id) => {
resource::get_check_permissions::<Repo>(
id,
&user,
user,
PermissionLevel::Read,
)
.await?;
@@ -272,7 +279,7 @@ impl Resolve<GetUpdate, User> for State {
ResourceTarget::Builder(id) => {
resource::get_check_permissions::<Builder>(
id,
&user,
user,
PermissionLevel::Read,
)
.await?;
@@ -280,7 +287,7 @@ impl Resolve<GetUpdate, User> for State {
ResourceTarget::Alerter(id) => {
resource::get_check_permissions::<Alerter>(
id,
&user,
user,
PermissionLevel::Read,
)
.await?;
@@ -288,7 +295,15 @@ impl Resolve<GetUpdate, User> for State {
ResourceTarget::Procedure(id) => {
resource::get_check_permissions::<Procedure>(
id,
&user,
user,
PermissionLevel::Read,
)
.await?;
}
ResourceTarget::Action(id) => {
resource::get_check_permissions::<Action>(
id,
user,
PermissionLevel::Read,
)
.await?;
@@ -296,7 +311,7 @@ impl Resolve<GetUpdate, User> for State {
ResourceTarget::ServerTemplate(id) => {
resource::get_check_permissions::<ServerTemplate>(
id,
&user,
user,
PermissionLevel::Read,
)
.await?;
@@ -304,7 +319,7 @@ impl Resolve<GetUpdate, User> for State {
ResourceTarget::ResourceSync(id) => {
resource::get_check_permissions::<ResourceSync>(
id,
&user,
user,
PermissionLevel::Read,
)
.await?;
@@ -312,7 +327,7 @@ impl Resolve<GetUpdate, User> for State {
ResourceTarget::Stack(id) => {
resource::get_check_permissions::<Stack>(
id,
&user,
user,
PermissionLevel::Read,
)
.await?;

View File

@@ -1,4 +1,4 @@
use anyhow::{anyhow, Context};
use anyhow::{Context, anyhow};
use komodo_client::{
api::read::{
FindUser, FindUserResponse, GetUsername, GetUsernameResponse,
@@ -6,7 +6,7 @@ use komodo_client::{
ListApiKeysForServiceUserResponse, ListApiKeysResponse,
ListUsers, ListUsersResponse,
},
entities::user::{User, UserConfig},
entities::user::{UserConfig, admin_service_user},
};
use mungos::{
by_id::find_one_by_id,
@@ -15,18 +15,23 @@ use mungos::{
};
use resolver_api::Resolve;
use crate::{
helpers::query::get_user,
state::{db_client, State},
};
use crate::{helpers::query::get_user, state::db_client};
impl Resolve<GetUsername, User> for State {
use super::ReadArgs;
impl Resolve<ReadArgs> for GetUsername {
async fn resolve(
&self,
GetUsername { user_id }: GetUsername,
_: User,
) -> anyhow::Result<GetUsernameResponse> {
let user = find_one_by_id(&db_client().await.users, &user_id)
self,
_: &ReadArgs,
) -> serror::Result<GetUsernameResponse> {
if let Some(user) = admin_service_user(&self.user_id) {
return Ok(GetUsernameResponse {
username: user.username,
avatar: None,
});
}
let user = find_one_by_id(&db_client().users, &self.user_id)
.await
.context("failed at mongo query for user")?
.context("no user found with id")?;
@@ -44,30 +49,30 @@ impl Resolve<GetUsername, User> for State {
}
}
impl Resolve<FindUser, User> for State {
impl Resolve<ReadArgs> for FindUser {
async fn resolve(
&self,
FindUser { user }: FindUser,
admin: User,
) -> anyhow::Result<FindUserResponse> {
self,
ReadArgs { user: admin }: &ReadArgs,
) -> serror::Result<FindUserResponse> {
if !admin.admin {
return Err(anyhow!("This method is admin only."));
return Err(anyhow!("This method is admin only.").into());
}
get_user(&user).await
Ok(get_user(&self.user).await?)
}
}
impl Resolve<ListUsers, User> for State {
impl Resolve<ReadArgs> for ListUsers {
async fn resolve(
&self,
ListUsers {}: ListUsers,
user: User,
) -> anyhow::Result<ListUsersResponse> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListUsersResponse> {
if !user.admin {
return Err(anyhow!("this route is only accessable by admins"));
return Err(
anyhow!("this route is only accessable by admins").into(),
);
}
let mut users = find_collect(
&db_client().await.users,
&db_client().users,
None,
FindOptions::builder().sort(doc! { "username": 1 }).build(),
)
@@ -78,14 +83,13 @@ impl Resolve<ListUsers, User> for State {
}
}
impl Resolve<ListApiKeys, User> for State {
impl Resolve<ReadArgs> for ListApiKeys {
async fn resolve(
&self,
ListApiKeys {}: ListApiKeys,
user: User,
) -> anyhow::Result<ListApiKeysResponse> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListApiKeysResponse> {
let api_keys = find_collect(
&db_client().await.api_keys,
&db_client().api_keys,
doc! { "user_id": &user.id },
FindOptions::builder().sort(doc! { "name": 1 }).build(),
)
@@ -101,23 +105,22 @@ impl Resolve<ListApiKeys, User> for State {
}
}
impl Resolve<ListApiKeysForServiceUser, User> for State {
impl Resolve<ReadArgs> for ListApiKeysForServiceUser {
async fn resolve(
&self,
ListApiKeysForServiceUser { user }: ListApiKeysForServiceUser,
admin: User,
) -> anyhow::Result<ListApiKeysForServiceUserResponse> {
self,
ReadArgs { user: admin }: &ReadArgs,
) -> serror::Result<ListApiKeysForServiceUserResponse> {
if !admin.admin {
return Err(anyhow!("This method is admin only."));
return Err(anyhow!("This method is admin only.").into());
}
let user = get_user(&user).await?;
let user = get_user(&self.user).await?;
let UserConfig::Service { .. } = user.config else {
return Err(anyhow!("Given user is not service user"));
return Err(anyhow!("Given user is not service user").into());
};
let api_keys = find_collect(
&db_client().await.api_keys,
&db_client().api_keys,
doc! { "user_id": &user.id },
None,
)

View File

@@ -1,65 +1,60 @@
use std::str::FromStr;
use anyhow::Context;
use komodo_client::{
api::read::{
GetUserGroup, GetUserGroupResponse, ListUserGroups,
ListUserGroupsResponse,
},
entities::user::User,
};
use komodo_client::api::read::*;
use mungos::{
find::find_collect,
mongodb::{
bson::{doc, oid::ObjectId, Document},
bson::{Document, doc, oid::ObjectId},
options::FindOptions,
},
};
use resolver_api::Resolve;
use crate::state::{db_client, State};
use crate::state::db_client;
impl Resolve<GetUserGroup, User> for State {
use super::ReadArgs;
impl Resolve<ReadArgs> for GetUserGroup {
async fn resolve(
&self,
GetUserGroup { user_group }: GetUserGroup,
user: User,
) -> anyhow::Result<GetUserGroupResponse> {
let mut filter = match ObjectId::from_str(&user_group) {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<GetUserGroupResponse> {
let mut filter = match ObjectId::from_str(&self.user_group) {
Ok(id) => doc! { "_id": id },
Err(_) => doc! { "name": &user_group },
Err(_) => doc! { "name": &self.user_group },
};
// Don't allow non admin users to get UserGroups they aren't a part of.
if !user.admin {
// Filter for only UserGroups which contain the users id
filter.insert("users", &user.id);
}
db_client()
.await
let res = db_client()
.user_groups
.find_one(filter)
.await
.context("failed to query db for user groups")?
.context("no UserGroup found with given name or id")
.context("no UserGroup found with given name or id")?;
Ok(res)
}
}
impl Resolve<ListUserGroups, User> for State {
impl Resolve<ReadArgs> for ListUserGroups {
async fn resolve(
&self,
ListUserGroups {}: ListUserGroups,
user: User,
) -> anyhow::Result<ListUserGroupsResponse> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListUserGroupsResponse> {
let mut filter = Document::new();
if !user.admin {
filter.insert("users", &user.id);
}
find_collect(
&db_client().await.user_groups,
let res = find_collect(
&db_client().user_groups,
filter,
FindOptions::builder().sort(doc! { "name": 1 }).build(),
)
.await
.context("failed to query db for UserGroups")
.context("failed to query db for UserGroups")?;
Ok(res)
}
}

View File

@@ -1,27 +1,19 @@
use anyhow::Context;
use komodo_client::api::read::*;
use mongo_indexed::doc;
use komodo_client::{
api::read::{
GetVariable, GetVariableResponse, ListVariables,
ListVariablesResponse,
},
entities::user::User,
};
use mungos::{find::find_collect, mongodb::options::FindOptions};
use resolver_api::Resolve;
use crate::{
helpers::query::get_variable,
state::{db_client, State},
};
use crate::{helpers::query::get_variable, state::db_client};
impl Resolve<GetVariable, User> for State {
use super::ReadArgs;
impl Resolve<ReadArgs> for GetVariable {
async fn resolve(
&self,
GetVariable { name }: GetVariable,
user: User,
) -> anyhow::Result<GetVariableResponse> {
let mut variable = get_variable(&name).await?;
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<GetVariableResponse> {
let mut variable = get_variable(&self.name).await?;
if !variable.is_secret || user.admin {
return Ok(variable);
}
@@ -30,14 +22,13 @@ impl Resolve<GetVariable, User> for State {
}
}
impl Resolve<ListVariables, User> for State {
impl Resolve<ReadArgs> for ListVariables {
async fn resolve(
&self,
ListVariables {}: ListVariables,
user: User,
) -> anyhow::Result<ListVariablesResponse> {
self,
ReadArgs { user }: &ReadArgs,
) -> serror::Result<ListVariablesResponse> {
let variables = find_collect(
&db_client().await.variables,
&db_client().variables,
None,
FindOptions::builder().sort(doc! { "name": 1 }).build(),
)

View File

@@ -1,20 +1,16 @@
use std::{collections::VecDeque, time::Instant};
use anyhow::{anyhow, Context};
use axum::{middleware, routing::post, Extension, Json, Router};
use axum_extra::{headers::ContentType, TypedHeader};
use anyhow::{Context, anyhow};
use axum::{Extension, Json, Router, middleware, routing::post};
use derive_variants::EnumVariants;
use komodo_client::{
api::user::{
CreateApiKey, CreateApiKeyResponse, DeleteApiKey,
DeleteApiKeyResponse, PushRecentlyViewed,
PushRecentlyViewedResponse, SetLastSeenUpdate,
SetLastSeenUpdateResponse,
},
api::user::*,
entities::{api_key::ApiKey, komodo_timestamp, user::User},
};
use mongo_indexed::doc;
use mungos::{by_id::update_one_by_id, mongodb::bson::to_bson};
use resolver_api::{derive::Resolver, Resolve, Resolver};
use resolver_api::Resolve;
use response::Response;
use serde::{Deserialize, Serialize};
use typeshare::typeshare;
use uuid::Uuid;
@@ -22,13 +18,20 @@ use uuid::Uuid;
use crate::{
auth::auth_request,
helpers::{query::get_user, random_string},
state::{db_client, State},
state::db_client,
};
pub struct UserArgs {
pub user: User,
}
#[typeshare]
#[derive(Serialize, Deserialize, Debug, Clone, Resolver)]
#[resolver_target(State)]
#[resolver_args(User)]
#[derive(
Serialize, Deserialize, Debug, Clone, Resolve, EnumVariants,
)]
#[args(UserArgs)]
#[response(Response)]
#[error(serror::Error)]
#[serde(tag = "type", content = "params")]
enum UserRequest {
PushRecentlyViewed(PushRecentlyViewed),
@@ -47,47 +50,37 @@ pub fn router() -> Router {
async fn handler(
Extension(user): Extension<User>,
Json(request): Json<UserRequest>,
) -> serror::Result<(TypedHeader<ContentType>, String)> {
) -> serror::Result<axum::response::Response> {
let timer = Instant::now();
let req_id = Uuid::new_v4();
debug!(
"/user request {req_id} | user: {} ({})",
user.username, user.id
);
let res =
State
.resolve_request(request, user)
.await
.map_err(|e| match e {
resolver_api::Error::Serialization(e) => {
anyhow!("{e:?}").context("response serialization error")
}
resolver_api::Error::Inner(e) => e,
});
let res = request.resolve(&UserArgs { user }).await;
if let Err(e) = &res {
warn!("/user request {req_id} error: {e:#}");
warn!("/user request {req_id} error: {:#}", e.error);
}
let elapsed = timer.elapsed();
debug!("/user request {req_id} | resolve time: {elapsed:?}");
Ok((TypedHeader(ContentType::json()), res?))
res.map(|res| res.0)
}
const RECENTLY_VIEWED_MAX: usize = 10;
impl Resolve<PushRecentlyViewed, User> for State {
impl Resolve<UserArgs> for PushRecentlyViewed {
#[instrument(
name = "PushRecentlyViewed",
level = "debug",
skip(self, user)
skip(user)
)]
async fn resolve(
&self,
PushRecentlyViewed { resource }: PushRecentlyViewed,
user: User,
) -> anyhow::Result<PushRecentlyViewedResponse> {
self,
UserArgs { user }: &UserArgs,
) -> serror::Result<PushRecentlyViewedResponse> {
let user = get_user(&user.id).await?;
let (resource_type, id) = resource.extract_variant_id();
let (resource_type, id) = self.resource.extract_variant_id();
let update = match user.recents.get(&resource_type) {
Some(recents) => {
let mut recents = recents
@@ -103,7 +96,7 @@ impl Resolve<PushRecentlyViewed, User> for State {
}
};
update_one_by_id(
&db_client().await.users,
&db_client().users,
&user.id,
mungos::update::Update::Set(update),
None,
@@ -117,19 +110,18 @@ impl Resolve<PushRecentlyViewed, User> for State {
}
}
impl Resolve<SetLastSeenUpdate, User> for State {
impl Resolve<UserArgs> for SetLastSeenUpdate {
#[instrument(
name = "SetLastSeenUpdate",
level = "debug",
skip(self, user)
skip(user)
)]
async fn resolve(
&self,
SetLastSeenUpdate {}: SetLastSeenUpdate,
user: User,
) -> anyhow::Result<SetLastSeenUpdateResponse> {
self,
UserArgs { user }: &UserArgs,
) -> serror::Result<SetLastSeenUpdateResponse> {
update_one_by_id(
&db_client().await.users,
&db_client().users,
&user.id,
mungos::update::Update::Set(doc! {
"last_update_view": komodo_timestamp()
@@ -145,17 +137,12 @@ impl Resolve<SetLastSeenUpdate, User> for State {
const SECRET_LENGTH: usize = 40;
const BCRYPT_COST: u32 = 10;
impl Resolve<CreateApiKey, User> for State {
#[instrument(
name = "CreateApiKey",
level = "debug",
skip(self, user)
)]
impl Resolve<UserArgs> for CreateApiKey {
#[instrument(name = "CreateApiKey", level = "debug", skip(user))]
async fn resolve(
&self,
CreateApiKey { name, expires }: CreateApiKey,
user: User,
) -> anyhow::Result<CreateApiKeyResponse> {
self,
UserArgs { user }: &UserArgs,
) -> serror::Result<CreateApiKeyResponse> {
let user = get_user(&user.id).await?;
let key = format!("K-{}", random_string(SECRET_LENGTH));
@@ -164,15 +151,14 @@ impl Resolve<CreateApiKey, User> for State {
.context("failed at hashing secret string")?;
let api_key = ApiKey {
name,
name: self.name,
key: key.clone(),
secret: secret_hash,
user_id: user.id.clone(),
created_at: komodo_timestamp(),
expires,
expires: self.expires,
};
db_client()
.await
.api_keys
.insert_one(api_key)
.await
@@ -181,26 +167,21 @@ impl Resolve<CreateApiKey, User> for State {
}
}
impl Resolve<DeleteApiKey, User> for State {
#[instrument(
name = "DeleteApiKey",
level = "debug",
skip(self, user)
)]
impl Resolve<UserArgs> for DeleteApiKey {
#[instrument(name = "DeleteApiKey", level = "debug", skip(user))]
async fn resolve(
&self,
DeleteApiKey { key }: DeleteApiKey,
user: User,
) -> anyhow::Result<DeleteApiKeyResponse> {
let client = db_client().await;
self,
UserArgs { user }: &UserArgs,
) -> serror::Result<DeleteApiKeyResponse> {
let client = db_client();
let key = client
.api_keys
.find_one(doc! { "key": &key })
.find_one(doc! { "key": &self.key })
.await
.context("failed at db query")?
.context("no api key with key found")?;
if user.id != key.user_id {
return Err(anyhow!("api key does not belong to user"));
return Err(anyhow!("api key does not belong to user").into());
}
client
.api_keys

View File

@@ -0,0 +1,71 @@
use komodo_client::{
api::write::*,
entities::{
action::Action, permission::PermissionLevel, update::Update,
},
};
use resolver_api::Resolve;
use crate::resource;
use super::WriteArgs;
impl Resolve<WriteArgs> for CreateAction {
#[instrument(name = "CreateAction", skip(user))]
async fn resolve(
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Action> {
Ok(
resource::create::<Action>(&self.name, self.config, user)
.await?,
)
}
}
impl Resolve<WriteArgs> for CopyAction {
#[instrument(name = "CopyAction", skip(user))]
async fn resolve(
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Action> {
let Action { config, .. } =
resource::get_check_permissions::<Action>(
&self.id,
user,
PermissionLevel::Write,
)
.await?;
Ok(
resource::create::<Action>(&self.name, config.into(), user)
.await?,
)
}
}
impl Resolve<WriteArgs> for UpdateAction {
#[instrument(name = "UpdateAction", skip(user))]
async fn resolve(
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Action> {
Ok(resource::update::<Action>(&self.id, self.config, user).await?)
}
}
impl Resolve<WriteArgs> for RenameAction {
#[instrument(name = "RenameAction", skip(user))]
async fn resolve(
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Update> {
Ok(resource::rename::<Action>(&self.id, &self.name, user).await?)
}
}
impl Resolve<WriteArgs> for DeleteAction {
#[instrument(name = "DeleteAction", skip(args))]
async fn resolve(self, args: &WriteArgs) -> serror::Result<Action> {
Ok(resource::delete::<Action>(&self.id, args).await?)
}
}

View File

@@ -1,61 +1,77 @@
use komodo_client::{
api::write::{
CopyAlerter, CreateAlerter, DeleteAlerter, UpdateAlerter,
},
api::write::*,
entities::{
alerter::Alerter, permission::PermissionLevel, user::User,
alerter::Alerter, permission::PermissionLevel, update::Update,
},
};
use resolver_api::Resolve;
use crate::{resource, state::State};
use crate::resource;
impl Resolve<CreateAlerter, User> for State {
#[instrument(name = "CreateAlerter", skip(self, user))]
async fn resolve(
&self,
CreateAlerter { name, config }: CreateAlerter,
user: User,
) -> anyhow::Result<Alerter> {
resource::create::<Alerter>(&name, config, &user).await
}
}
use super::WriteArgs;
impl Resolve<CopyAlerter, User> for State {
#[instrument(name = "CopyAlerter", skip(self, user))]
impl Resolve<WriteArgs> for CreateAlerter {
#[instrument(name = "CreateAlerter", skip(user))]
async fn resolve(
&self,
CopyAlerter { name, id }: CopyAlerter,
user: User,
) -> anyhow::Result<Alerter> {
let Alerter { config, .. } = resource::get_check_permissions::<
Alerter,
>(
&id, &user, PermissionLevel::Write
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Alerter> {
Ok(
resource::create::<Alerter>(&self.name, self.config, user)
.await?,
)
.await?;
resource::create::<Alerter>(&name, config.into(), &user).await
}
}
impl Resolve<DeleteAlerter, User> for State {
#[instrument(name = "DeleteAlerter", skip(self, user))]
impl Resolve<WriteArgs> for CopyAlerter {
#[instrument(name = "CopyAlerter", skip(user))]
async fn resolve(
&self,
DeleteAlerter { id }: DeleteAlerter,
user: User,
) -> anyhow::Result<Alerter> {
resource::delete::<Alerter>(&id, &user).await
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Alerter> {
let Alerter { config, .. } =
resource::get_check_permissions::<Alerter>(
&self.id,
user,
PermissionLevel::Write,
)
.await?;
Ok(
resource::create::<Alerter>(&self.name, config.into(), user)
.await?,
)
}
}
impl Resolve<UpdateAlerter, User> for State {
#[instrument(name = "UpdateAlerter", skip(self, user))]
impl Resolve<WriteArgs> for DeleteAlerter {
#[instrument(name = "DeleteAlerter", skip(args))]
async fn resolve(
&self,
UpdateAlerter { id, config }: UpdateAlerter,
user: User,
) -> anyhow::Result<Alerter> {
resource::update::<Alerter>(&id, config, &user).await
self,
args: &WriteArgs,
) -> serror::Result<Alerter> {
Ok(resource::delete::<Alerter>(&self.id, args).await?)
}
}
impl Resolve<WriteArgs> for UpdateAlerter {
#[instrument(name = "UpdateAlerter", skip(user))]
async fn resolve(
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Alerter> {
Ok(
resource::update::<Alerter>(&self.id, self.config, user)
.await?,
)
}
}
impl Resolve<WriteArgs> for RenameAlerter {
#[instrument(name = "RenameAlerter", skip(user))]
async fn resolve(
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Update> {
Ok(resource::rename::<Alerter>(&self.id, &self.name, user).await?)
}
}

View File

@@ -1,150 +1,404 @@
use anyhow::{anyhow, Context};
use mongo_indexed::doc;
use std::{path::PathBuf, str::FromStr, time::Duration};
use anyhow::{Context, anyhow};
use formatting::format_serror;
use git::GitRes;
use komodo_client::{
api::write::*,
entities::{
CloneArgs, FileContents, NoData, Operation, all_logs_success,
build::{Build, BuildInfo, PartialBuildConfig},
builder::{Builder, BuilderConfig},
config::core::CoreConfig,
permission::PermissionLevel,
user::User,
CloneArgs, NoData,
server::ServerState,
update::Update,
},
};
use mongo_indexed::doc;
use mungos::mongodb::bson::to_document;
use octorust::types::{
ReposCreateWebhookRequest, ReposCreateWebhookRequestConfig,
};
use periphery_client::{
PeripheryClient,
api::build::{
GetDockerfileContentsOnHost, WriteDockerfileContentsToHost,
},
};
use resolver_api::Resolve;
use tokio::fs;
use crate::{
config::core_config,
helpers::{git_token, random_string},
helpers::{
git_token, periphery_client,
query::get_server_with_state,
update::{add_update, make_update},
},
resource,
state::{db_client, github_client, State},
state::{db_client, github_client},
};
impl Resolve<CreateBuild, User> for State {
#[instrument(name = "CreateBuild", skip(self, user))]
use super::WriteArgs;
impl Resolve<WriteArgs> for CreateBuild {
#[instrument(name = "CreateBuild", skip(user))]
async fn resolve(
&self,
CreateBuild { name, config }: CreateBuild,
user: User,
) -> anyhow::Result<Build> {
resource::create::<Build>(&name, config, &user).await
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Build> {
Ok(
resource::create::<Build>(&self.name, self.config, user)
.await?,
)
}
}
impl Resolve<CopyBuild, User> for State {
#[instrument(name = "CopyBuild", skip(self, user))]
impl Resolve<WriteArgs> for CopyBuild {
#[instrument(name = "CopyBuild", skip(user))]
async fn resolve(
&self,
CopyBuild { name, id }: CopyBuild,
user: User,
) -> anyhow::Result<Build> {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Build> {
let Build { mut config, .. } =
resource::get_check_permissions::<Build>(
&id,
&user,
&self.id,
user,
PermissionLevel::Write,
)
.await?;
// reset version to 0.0.0
config.version = Default::default();
resource::create::<Build>(&name, config.into(), &user).await
Ok(
resource::create::<Build>(&self.name, config.into(), user)
.await?,
)
}
}
impl Resolve<DeleteBuild, User> for State {
#[instrument(name = "DeleteBuild", skip(self, user))]
impl Resolve<WriteArgs> for DeleteBuild {
#[instrument(name = "DeleteBuild", skip(args))]
async fn resolve(self, args: &WriteArgs) -> serror::Result<Build> {
Ok(resource::delete::<Build>(&self.id, args).await?)
}
}
impl Resolve<WriteArgs> for UpdateBuild {
#[instrument(name = "UpdateBuild", skip(user))]
async fn resolve(
&self,
DeleteBuild { id }: DeleteBuild,
user: User,
) -> anyhow::Result<Build> {
resource::delete::<Build>(&id, &user).await
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Build> {
Ok(resource::update::<Build>(&self.id, self.config, user).await?)
}
}
impl Resolve<UpdateBuild, User> for State {
#[instrument(name = "UpdateBuild", skip(self, user))]
impl Resolve<WriteArgs> for RenameBuild {
#[instrument(name = "RenameBuild", skip(user))]
async fn resolve(
&self,
UpdateBuild { id, config }: UpdateBuild,
user: User,
) -> anyhow::Result<Build> {
resource::update::<Build>(&id, config, &user).await
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Update> {
Ok(resource::rename::<Build>(&self.id, &self.name, user).await?)
}
}
impl Resolve<RefreshBuildCache, User> for State {
impl Resolve<WriteArgs> for WriteBuildFileContents {
#[instrument(name = "WriteBuildFileContents", skip(args))]
async fn resolve(self, args: &WriteArgs) -> serror::Result<Update> {
let build = resource::get_check_permissions::<Build>(
&self.build,
&args.user,
PermissionLevel::Write,
)
.await?;
if !build.config.files_on_host && build.config.repo.is_empty() {
return Err(anyhow!(
"Build is not configured to use Files on Host or Git Repo, can't write dockerfile contents"
).into());
}
let mut update =
make_update(&build, Operation::WriteDockerfile, &args.user);
update.push_simple_log("Dockerfile to write", &self.contents);
if build.config.files_on_host {
match get_on_host_periphery(&build)
.await?
.request(WriteDockerfileContentsToHost {
name: build.name,
build_path: build.config.build_path,
dockerfile_path: build.config.dockerfile_path,
contents: self.contents,
})
.await
.context("Failed to write dockerfile contents to host")
{
Ok(log) => {
update.logs.push(log);
}
Err(e) => {
update.push_error_log(
"Write Dockerfile Contents",
format_serror(&e.into()),
);
}
};
if !all_logs_success(&update.logs) {
update.finalize();
update.id = add_update(update.clone()).await?;
return Ok(update);
}
if let Err(e) =
(RefreshBuildCache { build: build.id }).resolve(args).await
{
update.push_error_log(
"Refresh build cache",
format_serror(&e.error.into()),
);
}
update.finalize();
update.id = add_update(update.clone()).await?;
Ok(update)
} else {
write_dockerfile_contents_git(self, args, build, update).await
}
}
}
async fn write_dockerfile_contents_git(
req: WriteBuildFileContents,
args: &WriteArgs,
build: Build,
mut update: Update,
) -> serror::Result<Update> {
let WriteBuildFileContents { build: _, contents } = req;
let mut clone_args: CloneArgs = (&build).into();
let root = clone_args.unique_path(&core_config().repo_directory)?;
let build_path = build
.config
.build_path
.parse::<PathBuf>()
.context("Invalid build path")?;
let dockerfile_path = build
.config
.dockerfile_path
.parse::<PathBuf>()
.context("Invalid dockerfile path")?;
let full_path = root.join(&build_path).join(&dockerfile_path);
if let Some(parent) = full_path.parent() {
fs::create_dir_all(parent).await.with_context(|| {
format!(
"Failed to initialize dockerfile parent directory {parent:?}"
)
})?;
}
// Ensure the folder is initialized as git repo.
// This allows a new file to be committed on a branch that may not exist.
if !root.join(".git").exists() {
let access_token = if let Some(account) = &clone_args.account {
git_token(&clone_args.provider, account, |https| clone_args.https = https)
.await
.with_context(
|| format!("Failed to get git token in call to db. Stopping run. | {} | {account}", clone_args.provider),
)?
} else {
None
};
git::init_folder_as_repo(
&root,
&clone_args,
access_token.as_deref(),
&mut update.logs,
)
.await;
if !all_logs_success(&update.logs) {
update.finalize();
update.id = add_update(update.clone()).await?;
return Ok(update);
}
}
if let Err(e) =
fs::write(&full_path, &contents).await.with_context(|| {
format!("Failed to write dockerfile contents to {full_path:?}")
})
{
update
.push_error_log("Write Dockerfile", format_serror(&e.into()));
} else {
update.push_simple_log(
"Write Dockerfile",
format!("File written to {full_path:?}"),
);
};
if !all_logs_success(&update.logs) {
update.finalize();
update.id = add_update(update.clone()).await?;
return Ok(update);
}
let commit_res = git::commit_file(
&format!("{}: Commit Dockerfile", args.user.username),
&root,
&build_path.join(&dockerfile_path),
&build.config.branch,
)
.await;
update.logs.extend(commit_res.logs);
if let Err(e) = (RefreshBuildCache { build: build.name })
.resolve(args)
.await
{
update.push_error_log(
"Refresh build cache",
format_serror(&e.error.into()),
);
}
update.finalize();
update.id = add_update(update.clone()).await?;
Ok(update)
}
impl Resolve<WriteArgs> for RefreshBuildCache {
#[instrument(
name = "RefreshBuildCache",
level = "debug",
skip(self, user)
skip(user)
)]
async fn resolve(
&self,
RefreshBuildCache { build }: RefreshBuildCache,
user: User,
) -> anyhow::Result<NoData> {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<NoData> {
// Even though this is a write request, this doesn't change any config. Anyone that can execute the
// build should be able to do this.
let build = resource::get_check_permissions::<Build>(
&build,
&user,
&self.build,
user,
PermissionLevel::Execute,
)
.await?;
if build.config.repo.is_empty() {
// Nothing to do here
return Ok(NoData {})
}
let config = core_config();
let repo_dir = config.repo_directory.join(random_string(10));
let mut clone_args: CloneArgs = (&build).into();
// Don't want to run these on core.
clone_args.on_clone = None;
clone_args.on_pull = None;
clone_args.destination = Some(repo_dir.display().to_string());
let access_token = match (&clone_args.account, &clone_args.provider)
{
(None, _) => None,
(Some(_), None) => {
return Err(anyhow!(
"Account is configured, but provider is empty"
))
let (
remote_path,
remote_contents,
remote_error,
latest_hash,
latest_message,
) = if build.config.files_on_host {
// =============
// FILES ON HOST
// =============
match get_on_host_dockerfile(&build).await {
Ok(FileContents { path, contents }) => {
(Some(path), Some(contents), None, None, None)
}
Err(e) => {
(None, None, Some(format_serror(&e.into())), None, None)
}
}
(Some(username), Some(provider)) => {
git_token(provider, username, |https| {
} else if !build.config.repo.is_empty() {
// ================
// REPO BASED BUILD
// ================
if build.config.git_provider.is_empty() {
// Nothing to do here
return Ok(NoData {});
}
let config = core_config();
let mut clone_args: CloneArgs = (&build).into();
let repo_path =
clone_args.unique_path(&core_config().repo_directory)?;
clone_args.destination = Some(repo_path.display().to_string());
// Don't want to run these on core.
clone_args.on_clone = None;
clone_args.on_pull = None;
let access_token = if let Some(username) = &clone_args.account {
git_token(&clone_args.provider, username, |https| {
clone_args.https = https
})
.await
.with_context(
|| format!("Failed to get git token in call to db. Stopping run. | {provider} | {username}"),
|| format!("Failed to get git token in call to db. Stopping run. | {} | {username}", clone_args.provider),
)?
}
};
} else {
None
};
let (_, latest_hash, latest_message, _) = git::clone(
clone_args,
&config.repo_directory,
access_token,
&[],
"",
None,
&[],
)
.await
.context("failed to clone build repo")?;
let GitRes { hash, message, .. } = git::pull_or_clone(
clone_args,
&config.repo_directory,
access_token,
&[],
"",
None,
&[],
)
.await
.context("failed to clone build repo")?;
let relative_path = PathBuf::from_str(&build.config.build_path)
.context("Invalid build path")?
.join(&build.config.dockerfile_path);
let full_path = repo_path.join(&relative_path);
let (contents, error) = match fs::read_to_string(&full_path)
.await
.with_context(|| {
format!(
"Failed to read dockerfile contents at {full_path:?}"
)
}) {
Ok(contents) => (Some(contents), None),
Err(e) => (None, Some(format_serror(&e.into()))),
};
(
Some(relative_path.display().to_string()),
contents,
error,
hash,
message,
)
} else {
// =============
// UI BASED FILE
// =============
(None, None, None, None, None)
};
let info = BuildInfo {
last_built_at: build.info.last_built_at,
built_hash: build.info.built_hash,
built_message: build.info.built_message,
built_contents: build.info.built_contents,
remote_path,
remote_contents,
remote_error,
latest_hash,
latest_message,
};
@@ -153,7 +407,6 @@ impl Resolve<RefreshBuildCache, User> for State {
.context("failed to serialize build info to bson")?;
db_client()
.await
.builds
.update_one(
doc! { "name": &build.name },
@@ -162,49 +415,105 @@ impl Resolve<RefreshBuildCache, User> for State {
.await
.context("failed to update build info on db")?;
if repo_dir.exists() {
if let Err(e) = std::fs::remove_dir_all(&repo_dir) {
warn!("failed to remove build cache update repo directory | {e:?}")
}
}
Ok(NoData {})
}
}
impl Resolve<CreateBuildWebhook, User> for State {
#[instrument(name = "CreateBuildWebhook", skip(self, user))]
async fn get_on_host_periphery(
build: &Build,
) -> anyhow::Result<PeripheryClient> {
if build.config.builder_id.is_empty() {
return Err(anyhow!("No builder associated with build"));
}
let builder = resource::get::<Builder>(&build.config.builder_id)
.await
.context("Failed to get builder")?;
match builder.config {
BuilderConfig::Aws(_) => {
Err(anyhow!("Files on host doesn't work with AWS builder"))
}
BuilderConfig::Url(config) => {
let periphery = PeripheryClient::new(
config.address,
config.passkey,
Duration::from_secs(3),
);
periphery.health_check().await?;
Ok(periphery)
}
BuilderConfig::Server(config) => {
if config.server_id.is_empty() {
return Err(anyhow!(
"Builder is type server, but has no server attached"
));
}
let (server, state) =
get_server_with_state(&config.server_id).await?;
if state != ServerState::Ok {
return Err(anyhow!(
"Builder server is disabled or not reachable"
));
};
periphery_client(&server)
}
}
}
/// The successful case will be included as Some(remote_contents).
/// The error case will be included as Some(remote_error)
async fn get_on_host_dockerfile(
build: &Build,
) -> anyhow::Result<FileContents> {
get_on_host_periphery(build)
.await?
.request(GetDockerfileContentsOnHost {
name: build.name.clone(),
build_path: build.config.build_path.clone(),
dockerfile_path: build.config.dockerfile_path.clone(),
})
.await
}
impl Resolve<WriteArgs> for CreateBuildWebhook {
#[instrument(name = "CreateBuildWebhook", skip(args))]
async fn resolve(
&self,
CreateBuildWebhook { build }: CreateBuildWebhook,
user: User,
) -> anyhow::Result<CreateBuildWebhookResponse> {
self,
args: &WriteArgs,
) -> serror::Result<CreateBuildWebhookResponse> {
let Some(github) = github_client() else {
return Err(anyhow!(
"github_webhook_app is not configured in core config toml"
));
return Err(
anyhow!(
"github_webhook_app is not configured in core config toml"
)
.into(),
);
};
let WriteArgs { user } = args;
let build = resource::get_check_permissions::<Build>(
&build,
&user,
&self.build,
user,
PermissionLevel::Write,
)
.await?;
if build.config.repo.is_empty() {
return Err(anyhow!(
"No repo configured, can't create webhook"
));
return Err(
anyhow!("No repo configured, can't create webhook").into(),
);
}
let mut split = build.config.repo.split('/');
let owner = split.next().context("Build repo has no owner")?;
let Some(github) = github.get(owner) else {
return Err(anyhow!(
"Cannot manage repo webhooks under owner {owner}"
));
return Err(
anyhow!("Cannot manage repo webhooks under owner {owner}")
.into(),
);
};
let repo =
@@ -232,7 +541,11 @@ impl Resolve<CreateBuildWebhook, User> for State {
&build.config.webhook_secret
};
let host = webhook_base_url.as_ref().unwrap_or(host);
let host = if webhook_base_url.is_empty() {
host
} else {
webhook_base_url
};
let url = format!("{host}/listener/github/build/{}", build.id);
for webhook in webhooks {
@@ -261,64 +574,65 @@ impl Resolve<CreateBuildWebhook, User> for State {
.context("failed to create webhook")?;
if !build.config.webhook_enabled {
self
.resolve(
UpdateBuild {
id: build.id,
config: PartialBuildConfig {
webhook_enabled: Some(true),
..Default::default()
},
},
user,
)
.await
.context("failed to update build to enable webhook")?;
UpdateBuild {
id: build.id,
config: PartialBuildConfig {
webhook_enabled: Some(true),
..Default::default()
},
}
.resolve(args)
.await
.map_err(|e| e.error)
.context("failed to update build to enable webhook")?;
}
Ok(NoData {})
}
}
impl Resolve<DeleteBuildWebhook, User> for State {
#[instrument(name = "DeleteBuildWebhook", skip(self, user))]
impl Resolve<WriteArgs> for DeleteBuildWebhook {
#[instrument(name = "DeleteBuildWebhook", skip(user))]
async fn resolve(
&self,
DeleteBuildWebhook { build }: DeleteBuildWebhook,
user: User,
) -> anyhow::Result<DeleteBuildWebhookResponse> {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<DeleteBuildWebhookResponse> {
let Some(github) = github_client() else {
return Err(anyhow!(
"github_webhook_app is not configured in core config toml"
));
return Err(
anyhow!(
"github_webhook_app is not configured in core config toml"
)
.into(),
);
};
let build = resource::get_check_permissions::<Build>(
&build,
&user,
&self.build,
user,
PermissionLevel::Write,
)
.await?;
if build.config.git_provider != "github.com" {
return Err(anyhow!(
"Can only manage github.com repo webhooks"
));
return Err(
anyhow!("Can only manage github.com repo webhooks").into(),
);
}
if build.config.repo.is_empty() {
return Err(anyhow!(
"No repo configured, can't delete webhook"
));
return Err(
anyhow!("No repo configured, can't delete webhook").into(),
);
}
let mut split = build.config.repo.split('/');
let owner = split.next().context("Build repo has no owner")?;
let Some(github) = github.get(owner) else {
return Err(anyhow!(
"Cannot manage repo webhooks under owner {owner}"
));
return Err(
anyhow!("Cannot manage repo webhooks under owner {owner}")
.into(),
);
};
let repo =
@@ -338,7 +652,11 @@ impl Resolve<DeleteBuildWebhook, User> for State {
..
} = core_config();
let host = webhook_base_url.as_ref().unwrap_or(host);
let host = if webhook_base_url.is_empty() {
host
} else {
webhook_base_url
};
let url = format!("{host}/listener/github/build/{}", build.id);
for webhook in webhooks {

View File

@@ -1,59 +1,77 @@
use komodo_client::{
api::write::*,
entities::{
builder::Builder, permission::PermissionLevel, user::User,
builder::Builder, permission::PermissionLevel, update::Update,
},
};
use resolver_api::Resolve;
use crate::{resource, state::State};
use crate::resource;
impl Resolve<CreateBuilder, User> for State {
#[instrument(name = "CreateBuilder", skip(self, user))]
async fn resolve(
&self,
CreateBuilder { name, config }: CreateBuilder,
user: User,
) -> anyhow::Result<Builder> {
resource::create::<Builder>(&name, config, &user).await
}
}
use super::WriteArgs;
impl Resolve<CopyBuilder, User> for State {
#[instrument(name = "CopyBuilder", skip(self, user))]
impl Resolve<WriteArgs> for CreateBuilder {
#[instrument(name = "CreateBuilder", skip(user))]
async fn resolve(
&self,
CopyBuilder { name, id }: CopyBuilder,
user: User,
) -> anyhow::Result<Builder> {
let Builder { config, .. } = resource::get_check_permissions::<
Builder,
>(
&id, &user, PermissionLevel::Write
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Builder> {
Ok(
resource::create::<Builder>(&self.name, self.config, user)
.await?,
)
.await?;
resource::create::<Builder>(&name, config.into(), &user).await
}
}
impl Resolve<DeleteBuilder, User> for State {
#[instrument(name = "DeleteBuilder", skip(self, user))]
impl Resolve<WriteArgs> for CopyBuilder {
#[instrument(name = "CopyBuilder", skip(user))]
async fn resolve(
&self,
DeleteBuilder { id }: DeleteBuilder,
user: User,
) -> anyhow::Result<Builder> {
resource::delete::<Builder>(&id, &user).await
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Builder> {
let Builder { config, .. } =
resource::get_check_permissions::<Builder>(
&self.id,
user,
PermissionLevel::Write,
)
.await?;
Ok(
resource::create::<Builder>(&self.name, config.into(), user)
.await?,
)
}
}
impl Resolve<UpdateBuilder, User> for State {
#[instrument(name = "UpdateBuilder", skip(self, user))]
impl Resolve<WriteArgs> for DeleteBuilder {
#[instrument(name = "DeleteBuilder", skip(args))]
async fn resolve(
&self,
UpdateBuilder { id, config }: UpdateBuilder,
user: User,
) -> anyhow::Result<Builder> {
resource::update::<Builder>(&id, config, &user).await
self,
args: &WriteArgs,
) -> serror::Result<Builder> {
Ok(resource::delete::<Builder>(&self.id, args).await?)
}
}
impl Resolve<WriteArgs> for UpdateBuilder {
#[instrument(name = "UpdateBuilder", skip(user))]
async fn resolve(
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Builder> {
Ok(
resource::update::<Builder>(&self.id, self.config, user)
.await?,
)
}
}
impl Resolve<WriteArgs> for RenameBuilder {
#[instrument(name = "RenameBuilder", skip(user))]
async fn resolve(
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Update> {
Ok(resource::rename::<Builder>(&self.id, &self.name, user).await?)
}
}

View File

@@ -1,19 +1,22 @@
use anyhow::{anyhow, Context};
use anyhow::{Context, anyhow};
use komodo_client::{
api::write::*,
entities::{
deployment::{Deployment, DeploymentState},
Operation,
deployment::{
Deployment, DeploymentImage, DeploymentState,
PartialDeploymentConfig, RestartMode,
},
docker::container::RestartPolicyNameEnum,
komodo_timestamp,
permission::PermissionLevel,
server::Server,
server::{Server, ServerState},
to_komodo_name,
update::Update,
user::User,
Operation,
},
};
use mungos::{by_id::update_one_by_id, mongodb::bson::doc};
use periphery_client::api;
use periphery_client::api::{self, container::InspectContainer};
use resolver_api::Resolve;
use crate::{
@@ -23,70 +26,171 @@ use crate::{
update::{add_update, make_update},
},
resource,
state::{action_states, db_client, State},
state::{action_states, db_client, server_status_cache},
};
impl Resolve<CreateDeployment, User> for State {
#[instrument(name = "CreateDeployment", skip(self, user))]
use super::WriteArgs;
impl Resolve<WriteArgs> for CreateDeployment {
#[instrument(name = "CreateDeployment", skip(user))]
async fn resolve(
&self,
CreateDeployment { name, config }: CreateDeployment,
user: User,
) -> anyhow::Result<Deployment> {
resource::create::<Deployment>(&name, config, &user).await
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Deployment> {
Ok(
resource::create::<Deployment>(&self.name, self.config, user)
.await?,
)
}
}
impl Resolve<CopyDeployment, User> for State {
#[instrument(name = "CopyDeployment", skip(self, user))]
impl Resolve<WriteArgs> for CopyDeployment {
#[instrument(name = "CopyDeployment", skip(user))]
async fn resolve(
&self,
CopyDeployment { name, id }: CopyDeployment,
user: User,
) -> anyhow::Result<Deployment> {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Deployment> {
let Deployment { config, .. } =
resource::get_check_permissions::<Deployment>(
&id,
&user,
&self.id,
user,
PermissionLevel::Write,
)
.await?;
resource::create::<Deployment>(&name, config.into(), &user).await
Ok(
resource::create::<Deployment>(&self.name, config.into(), user)
.await?,
)
}
}
impl Resolve<DeleteDeployment, User> for State {
#[instrument(name = "DeleteDeployment", skip(self, user))]
impl Resolve<WriteArgs> for CreateDeploymentFromContainer {
#[instrument(name = "CreateDeploymentFromContainer", skip(user))]
async fn resolve(
&self,
DeleteDeployment { id }: DeleteDeployment,
user: User,
) -> anyhow::Result<Deployment> {
resource::delete::<Deployment>(&id, &user).await
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Deployment> {
let server = resource::get_check_permissions::<Server>(
&self.server,
user,
PermissionLevel::Write,
)
.await?;
let cache = server_status_cache()
.get_or_insert_default(&server.id)
.await;
if cache.state != ServerState::Ok {
return Err(
anyhow!(
"Cannot inspect container: server is {:?}",
cache.state
)
.into(),
);
}
let container = periphery_client(&server)?
.request(InspectContainer {
name: self.name.clone(),
})
.await
.context("Failed to inspect container")?;
let mut config = PartialDeploymentConfig {
server_id: server.id.into(),
..Default::default()
};
if let Some(container_config) = container.config {
config.image = container_config
.image
.map(|image| DeploymentImage::Image { image });
config.command = container_config.cmd.join(" ").into();
config.environment = container_config
.env
.into_iter()
.map(|env| format!(" {env}"))
.collect::<Vec<_>>()
.join("\n")
.into();
config.labels = container_config
.labels
.into_iter()
.map(|(key, val)| format!(" {key}: {val}"))
.collect::<Vec<_>>()
.join("\n")
.into();
}
if let Some(host_config) = container.host_config {
config.volumes = host_config
.binds
.into_iter()
.map(|bind| format!(" {bind}"))
.collect::<Vec<_>>()
.join("\n")
.into();
config.network = host_config.network_mode;
config.ports = host_config
.port_bindings
.into_iter()
.filter_map(|(container, mut host)| {
let host = host.pop()?.host_port?;
Some(format!(" {host}:{}", container.replace("/tcp", "")))
})
.collect::<Vec<_>>()
.join("\n")
.into();
config.restart = host_config.restart_policy.map(|restart| {
match restart.name {
RestartPolicyNameEnum::Always => RestartMode::Always,
RestartPolicyNameEnum::No
| RestartPolicyNameEnum::Empty => RestartMode::NoRestart,
RestartPolicyNameEnum::UnlessStopped => {
RestartMode::UnlessStopped
}
RestartPolicyNameEnum::OnFailure => RestartMode::OnFailure,
}
});
}
Ok(
resource::create::<Deployment>(&self.name, config, user)
.await?,
)
}
}
impl Resolve<UpdateDeployment, User> for State {
#[instrument(name = "UpdateDeployment", skip(self, user))]
impl Resolve<WriteArgs> for DeleteDeployment {
#[instrument(name = "DeleteDeployment", skip(args))]
async fn resolve(
&self,
UpdateDeployment { id, config }: UpdateDeployment,
user: User,
) -> anyhow::Result<Deployment> {
resource::update::<Deployment>(&id, config, &user).await
self,
args: &WriteArgs,
) -> serror::Result<Deployment> {
Ok(resource::delete::<Deployment>(&self.id, args).await?)
}
}
impl Resolve<RenameDeployment, User> for State {
#[instrument(name = "RenameDeployment", skip(self, user))]
impl Resolve<WriteArgs> for UpdateDeployment {
#[instrument(name = "UpdateDeployment", skip(user))]
async fn resolve(
&self,
RenameDeployment { id, name }: RenameDeployment,
user: User,
) -> anyhow::Result<Update> {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Deployment> {
Ok(
resource::update::<Deployment>(&self.id, self.config, user)
.await?,
)
}
}
impl Resolve<WriteArgs> for RenameDeployment {
#[instrument(name = "RenameDeployment", skip(user))]
async fn resolve(
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Update> {
let deployment = resource::get_check_permissions::<Deployment>(
&id,
&user,
&self.id,
user,
PermissionLevel::Write,
)
.await?;
@@ -102,21 +206,24 @@ impl Resolve<RenameDeployment, User> for State {
let _action_guard =
action_state.update(|state| state.renaming = true)?;
let name = to_komodo_name(&name);
let name = to_komodo_name(&self.name);
let container_state = get_deployment_state(&deployment).await?;
if container_state == DeploymentState::Unknown {
return Err(anyhow!(
"cannot rename deployment when container status is unknown"
));
return Err(
anyhow!(
"Cannot rename Deployment when container status is unknown"
)
.into(),
);
}
let mut update =
make_update(&deployment, Operation::RenameDeployment, &user);
make_update(&deployment, Operation::RenameDeployment, user);
update_one_by_id(
&db_client().await.deployments,
&db_client().deployments,
&deployment.id,
mungos::update::Update::Set(
doc! { "name": &name, "updated_at": komodo_timestamp() },
@@ -124,7 +231,7 @@ impl Resolve<RenameDeployment, User> for State {
None,
)
.await
.context("failed to update deployment name on db")?;
.context("Failed to update Deployment name on db")?;
if container_state != DeploymentState::NotDeployed {
let server =
@@ -135,20 +242,19 @@ impl Resolve<RenameDeployment, User> for State {
new_name: name.clone(),
})
.await
.context("failed to rename container on server")?;
.context("Failed to rename container on server")?;
update.logs.push(log);
}
update.push_simple_log(
"rename deployment",
"Rename Deployment",
format!(
"renamed deployment from {} to {}",
"Renamed Deployment from {} to {}",
deployment.name, name
),
);
update.finalize();
add_update(update.clone()).await?;
update.id = add_update(update.clone()).await?;
Ok(update)
}

View File

@@ -2,109 +2,118 @@ use anyhow::anyhow;
use komodo_client::{
api::write::{UpdateDescription, UpdateDescriptionResponse},
entities::{
alerter::Alerter, build::Build, builder::Builder,
deployment::Deployment, procedure::Procedure, repo::Repo,
server::Server, server_template::ServerTemplate, stack::Stack,
sync::ResourceSync, user::User, ResourceTarget,
ResourceTarget, action::Action, alerter::Alerter, build::Build,
builder::Builder, deployment::Deployment, procedure::Procedure,
repo::Repo, server::Server, server_template::ServerTemplate,
stack::Stack, sync::ResourceSync,
},
};
use resolver_api::Resolve;
use crate::{resource, state::State};
use crate::resource;
impl Resolve<UpdateDescription, User> for State {
#[instrument(name = "UpdateDescription", skip(self, user))]
use super::WriteArgs;
impl Resolve<WriteArgs> for UpdateDescription {
#[instrument(name = "UpdateDescription", skip(user))]
async fn resolve(
&self,
UpdateDescription {
target,
description,
}: UpdateDescription,
user: User,
) -> anyhow::Result<UpdateDescriptionResponse> {
match target {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<UpdateDescriptionResponse> {
match self.target {
ResourceTarget::System(_) => {
return Err(anyhow!(
"cannot update description of System resource target"
))
return Err(
anyhow!(
"cannot update description of System resource target"
)
.into(),
);
}
ResourceTarget::Server(id) => {
resource::update_description::<Server>(
&id,
&description,
&user,
&self.description,
user,
)
.await?;
}
ResourceTarget::Deployment(id) => {
resource::update_description::<Deployment>(
&id,
&description,
&user,
&self.description,
user,
)
.await?;
}
ResourceTarget::Build(id) => {
resource::update_description::<Build>(
&id,
&description,
&user,
&self.description,
user,
)
.await?;
}
ResourceTarget::Repo(id) => {
resource::update_description::<Repo>(
&id,
&description,
&user,
&self.description,
user,
)
.await?;
}
ResourceTarget::Builder(id) => {
resource::update_description::<Builder>(
&id,
&description,
&user,
&self.description,
user,
)
.await?;
}
ResourceTarget::Alerter(id) => {
resource::update_description::<Alerter>(
&id,
&description,
&user,
&self.description,
user,
)
.await?;
}
ResourceTarget::Procedure(id) => {
resource::update_description::<Procedure>(
&id,
&description,
&user,
&self.description,
user,
)
.await?;
}
ResourceTarget::Action(id) => {
resource::update_description::<Action>(
&id,
&self.description,
user,
)
.await?;
}
ResourceTarget::ServerTemplate(id) => {
resource::update_description::<ServerTemplate>(
&id,
&description,
&user,
&self.description,
user,
)
.await?;
}
ResourceTarget::ResourceSync(id) => {
resource::update_description::<ResourceSync>(
&id,
&description,
&user,
&self.description,
user,
)
.await?;
}
ResourceTarget::Stack(id) => {
resource::update_description::<Stack>(
&id,
&description,
&user,
&self.description,
user,
)
.await?;
}

View File

@@ -1,18 +1,19 @@
use std::time::Instant;
use anyhow::{anyhow, Context};
use axum::{middleware, routing::post, Extension, Router};
use axum_extra::{headers::ContentType, TypedHeader};
use anyhow::Context;
use axum::{Extension, Router, middleware, routing::post};
use derive_variants::{EnumVariants, ExtractVariant};
use komodo_client::{api::write::*, entities::user::User};
use resolver_api::{derive::Resolver, Resolver};
use resolver_api::Resolve;
use response::Response;
use serde::{Deserialize, Serialize};
use serror::Json;
use typeshare::typeshare;
use uuid::Uuid;
use crate::{auth::auth_request, state::State};
use crate::auth::auth_request;
mod action;
mod alerter;
mod build;
mod builder;
@@ -28,18 +29,29 @@ mod service_user;
mod stack;
mod sync;
mod tag;
mod user;
mod user_group;
mod variable;
pub struct WriteArgs {
pub user: User,
}
#[typeshare]
#[derive(
Serialize, Deserialize, Debug, Clone, Resolver, EnumVariants,
Serialize, Deserialize, Debug, Clone, Resolve, EnumVariants,
)]
#[variant_derive(Debug)]
#[resolver_target(State)]
#[resolver_args(User)]
#[args(WriteArgs)]
#[response(Response)]
#[error(serror::Error)]
#[serde(tag = "type", content = "params")]
pub enum WriteRequest {
// ==== USER ====
UpdateUserUsername(UpdateUserUsername),
UpdateUserPassword(UpdateUserPassword),
DeleteUser(DeleteUser),
// ==== SERVICE USER ====
CreateServiceUser(CreateServiceUser),
UpdateServiceUserDescription(UpdateServiceUserDescription),
@@ -55,6 +67,7 @@ pub enum WriteRequest {
SetUsersInUserGroup(SetUsersInUserGroup),
// ==== PERMISSIONS ====
UpdateUserAdmin(UpdateUserAdmin),
UpdateUserBasePermissions(UpdateUserBasePermissions),
UpdatePermissionOnResourceType(UpdatePermissionOnResourceType),
UpdatePermissionOnTarget(UpdatePermissionOnTarget),
@@ -68,10 +81,14 @@ pub enum WriteRequest {
UpdateServer(UpdateServer),
RenameServer(RenameServer),
CreateNetwork(CreateNetwork),
CreateTerminal(CreateTerminal),
DeleteTerminal(DeleteTerminal),
DeleteAllTerminals(DeleteAllTerminals),
// ==== DEPLOYMENT ====
CreateDeployment(CreateDeployment),
CopyDeployment(CopyDeployment),
CreateDeploymentFromContainer(CreateDeploymentFromContainer),
DeleteDeployment(DeleteDeployment),
UpdateDeployment(UpdateDeployment),
RenameDeployment(RenameDeployment),
@@ -81,6 +98,8 @@ pub enum WriteRequest {
CopyBuild(CopyBuild),
DeleteBuild(DeleteBuild),
UpdateBuild(UpdateBuild),
RenameBuild(RenameBuild),
WriteBuildFileContents(WriteBuildFileContents),
RefreshBuildCache(RefreshBuildCache),
CreateBuildWebhook(CreateBuildWebhook),
DeleteBuildWebhook(DeleteBuildWebhook),
@@ -90,18 +109,21 @@ pub enum WriteRequest {
CopyBuilder(CopyBuilder),
DeleteBuilder(DeleteBuilder),
UpdateBuilder(UpdateBuilder),
RenameBuilder(RenameBuilder),
// ==== SERVER TEMPLATE ====
CreateServerTemplate(CreateServerTemplate),
CopyServerTemplate(CopyServerTemplate),
DeleteServerTemplate(DeleteServerTemplate),
UpdateServerTemplate(UpdateServerTemplate),
RenameServerTemplate(RenameServerTemplate),
// ==== REPO ====
CreateRepo(CreateRepo),
CopyRepo(CopyRepo),
DeleteRepo(DeleteRepo),
UpdateRepo(UpdateRepo),
RenameRepo(RenameRepo),
RefreshRepoCache(RefreshRepoCache),
CreateRepoWebhook(CreateRepoWebhook),
DeleteRepoWebhook(DeleteRepoWebhook),
@@ -111,18 +133,30 @@ pub enum WriteRequest {
CopyAlerter(CopyAlerter),
DeleteAlerter(DeleteAlerter),
UpdateAlerter(UpdateAlerter),
RenameAlerter(RenameAlerter),
// ==== PROCEDURE ====
CreateProcedure(CreateProcedure),
CopyProcedure(CopyProcedure),
DeleteProcedure(DeleteProcedure),
UpdateProcedure(UpdateProcedure),
RenameProcedure(RenameProcedure),
// ==== ACTION ====
CreateAction(CreateAction),
CopyAction(CopyAction),
DeleteAction(DeleteAction),
UpdateAction(UpdateAction),
RenameAction(RenameAction),
// ==== SYNC ====
CreateResourceSync(CreateResourceSync),
CopyResourceSync(CopyResourceSync),
DeleteResourceSync(DeleteResourceSync),
UpdateResourceSync(UpdateResourceSync),
RenameResourceSync(RenameResourceSync),
WriteSyncFileContents(WriteSyncFileContents),
CommitSync(CommitSync),
RefreshResourceSyncPending(RefreshResourceSyncPending),
CreateSyncWebhook(CreateSyncWebhook),
DeleteSyncWebhook(DeleteSyncWebhook),
@@ -133,6 +167,7 @@ pub enum WriteRequest {
DeleteStack(DeleteStack),
UpdateStack(UpdateStack),
RenameStack(RenameStack),
WriteStackFileContents(WriteStackFileContents),
RefreshStackCache(RefreshStackCache),
CreateStackWebhook(CreateStackWebhook),
DeleteStackWebhook(DeleteStackWebhook),
@@ -141,6 +176,7 @@ pub enum WriteRequest {
CreateTag(CreateTag),
DeleteTag(DeleteTag),
RenameTag(RenameTag),
UpdateTagColor(UpdateTagColor),
UpdateTagsOnResource(UpdateTagsOnResource),
// ==== VARIABLE ====
@@ -168,51 +204,41 @@ pub fn router() -> Router {
async fn handler(
Extension(user): Extension<User>,
Json(request): Json<WriteRequest>,
) -> serror::Result<(TypedHeader<ContentType>, String)> {
) -> serror::Result<axum::response::Response> {
let req_id = Uuid::new_v4();
let res = tokio::spawn(task(req_id, request, user))
.await
.context("failure in spawned task");
if let Err(e) = &res {
warn!("/write request {req_id} spawn error: {e:#}");
}
Ok((TypedHeader(ContentType::json()), res??))
res?
}
#[instrument(
name = "WriteRequest",
skip(user, request),
fields(user_id = user.id, request = format!("{:?}", request.extract_variant()))
fields(
user_id = user.id,
request = format!("{:?}", request.extract_variant())
)
)]
async fn task(
req_id: Uuid,
request: WriteRequest,
user: User,
) -> anyhow::Result<String> {
) -> serror::Result<axum::response::Response> {
info!("/write request | user: {}", user.username);
let timer = Instant::now();
let res =
State
.resolve_request(request, user)
.await
.map_err(|e| match e {
resolver_api::Error::Serialization(e) => {
anyhow!("{e:?}").context("response serialization error")
}
resolver_api::Error::Inner(e) => e,
});
let res = request.resolve(&WriteArgs { user }).await;
if let Err(e) = &res {
warn!("/write request {req_id} error: {e:#}");
warn!("/write request {req_id} error: {:#}", e.error);
}
let elapsed = timer.elapsed();
debug!("/write request {req_id} | resolve time: {elapsed:?}");
res
res.map(|res| res.0)
}

View File

@@ -1,57 +1,97 @@
use std::str::FromStr;
use anyhow::{anyhow, Context};
use anyhow::{Context, anyhow};
use komodo_client::{
api::write::{
UpdatePermissionOnResourceType,
UpdatePermissionOnResourceTypeResponse, UpdatePermissionOnTarget,
UpdatePermissionOnTargetResponse, UpdateUserBasePermissions,
UpdateUserBasePermissionsResponse,
},
api::write::*,
entities::{
permission::{UserTarget, UserTargetVariant},
user::User,
ResourceTarget, ResourceTargetVariant,
permission::{UserTarget, UserTargetVariant},
},
};
use mungos::{
by_id::{find_one_by_id, update_one_by_id},
mongodb::{
bson::{doc, oid::ObjectId, Document},
bson::{Document, doc, oid::ObjectId},
options::UpdateOptions,
},
};
use resolver_api::Resolve;
use crate::{
helpers::query::get_user,
state::{db_client, State},
};
use crate::{helpers::query::get_user, state::db_client};
impl Resolve<UpdateUserBasePermissions, User> for State {
#[instrument(name = "UpdateUserBasePermissions", skip(self, admin))]
use super::WriteArgs;
impl Resolve<WriteArgs> for UpdateUserAdmin {
#[instrument(name = "UpdateUserAdmin", skip(super_admin))]
async fn resolve(
&self,
UpdateUserBasePermissions {
self,
WriteArgs { user: super_admin }: &WriteArgs,
) -> serror::Result<UpdateUserAdminResponse> {
if !super_admin.super_admin {
return Err(
anyhow!("Only super admins can call this method.").into(),
);
}
let user = find_one_by_id(&db_client().users, &self.user_id)
.await
.context("failed to query mongo for user")?
.context("did not find user with given id")?;
if !user.enabled {
return Err(
anyhow!("User is disabled. Enable user first.").into(),
);
}
if user.super_admin {
return Err(anyhow!("Cannot update other super admins").into());
}
update_one_by_id(
&db_client().users,
&self.user_id,
doc! { "$set": { "admin": self.admin } },
None,
)
.await?;
Ok(UpdateUserAdminResponse {})
}
}
impl Resolve<WriteArgs> for UpdateUserBasePermissions {
#[instrument(name = "UpdateUserBasePermissions", skip(admin))]
async fn resolve(
self,
WriteArgs { user: admin }: &WriteArgs,
) -> serror::Result<UpdateUserBasePermissionsResponse> {
let UpdateUserBasePermissions {
user_id,
enabled,
create_servers,
create_builds,
}: UpdateUserBasePermissions,
admin: User,
) -> anyhow::Result<UpdateUserBasePermissionsResponse> {
} = self;
if !admin.admin {
return Err(anyhow!("this method is admin only"));
return Err(anyhow!("this method is admin only").into());
}
let user = find_one_by_id(&db_client().await.users, &user_id)
let user = find_one_by_id(&db_client().users, &user_id)
.await
.context("failed to query mongo for user")?
.context("did not find user with given id")?;
if user.admin {
if user.super_admin {
return Err(
anyhow!(
"Cannot use this method to update super admins permissions"
)
.into(),
);
}
if user.admin && !admin.super_admin {
return Err(anyhow!(
"cannot use this method to update other admins permissions"
));
"Only super admins can use this method to update other admins permissions"
).into());
}
let mut update_doc = Document::new();
if let Some(enabled) = enabled {
@@ -65,7 +105,7 @@ impl Resolve<UpdateUserBasePermissions, User> for State {
}
update_one_by_id(
&db_client().await.users,
&db_client().users,
&user_id,
mungos::update::Update::Set(update_doc),
None,
@@ -76,34 +116,35 @@ impl Resolve<UpdateUserBasePermissions, User> for State {
}
}
impl Resolve<UpdatePermissionOnResourceType, User> for State {
#[instrument(
name = "UpdatePermissionOnResourceType",
skip(self, admin)
)]
impl Resolve<WriteArgs> for UpdatePermissionOnResourceType {
#[instrument(name = "UpdatePermissionOnResourceType", skip(admin))]
async fn resolve(
&self,
UpdatePermissionOnResourceType {
self,
WriteArgs { user: admin }: &WriteArgs,
) -> serror::Result<UpdatePermissionOnResourceTypeResponse> {
let UpdatePermissionOnResourceType {
user_target,
resource_type,
permission,
}: UpdatePermissionOnResourceType,
admin: User,
) -> anyhow::Result<UpdatePermissionOnResourceTypeResponse> {
} = self;
if !admin.admin {
return Err(anyhow!("this method is admin only"));
return Err(anyhow!("this method is admin only").into());
}
// Some extra checks if user target is an actual User
if let UserTarget::User(user_id) = &user_target {
let user = get_user(user_id).await?;
if user.admin {
return Err(anyhow!(
return Err(
anyhow!(
"cannot use this method to update other admins permissions"
));
)
.into(),
);
}
if !user.enabled {
return Err(anyhow!("user not enabled"));
return Err(anyhow!("user not enabled").into());
}
}
@@ -119,7 +160,6 @@ impl Resolve<UpdatePermissionOnResourceType, User> for State {
match user_target_variant {
UserTargetVariant::User => {
db_client()
.await
.users
.update_one(filter, update)
.await
@@ -129,7 +169,6 @@ impl Resolve<UpdatePermissionOnResourceType, User> for State {
}
UserTargetVariant::UserGroup => {
db_client()
.await
.user_groups
.update_one(filter, update)
.await
@@ -143,31 +182,35 @@ impl Resolve<UpdatePermissionOnResourceType, User> for State {
}
}
impl Resolve<UpdatePermissionOnTarget, User> for State {
#[instrument(name = "UpdatePermissionOnTarget", skip(self, admin))]
impl Resolve<WriteArgs> for UpdatePermissionOnTarget {
#[instrument(name = "UpdatePermissionOnTarget", skip(admin))]
async fn resolve(
&self,
UpdatePermissionOnTarget {
self,
WriteArgs { user: admin }: &WriteArgs,
) -> serror::Result<UpdatePermissionOnTargetResponse> {
let UpdatePermissionOnTarget {
user_target,
resource_target,
permission,
}: UpdatePermissionOnTarget,
admin: User,
) -> anyhow::Result<UpdatePermissionOnTargetResponse> {
} = self;
if !admin.admin {
return Err(anyhow!("this method is admin only"));
return Err(anyhow!("this method is admin only").into());
}
// Some extra checks if user target is an actual User
if let UserTarget::User(user_id) = &user_target {
let user = get_user(user_id).await?;
if user.admin {
return Err(anyhow!(
return Err(
anyhow!(
"cannot use this method to update other admins permissions"
));
)
.into(),
);
}
if !user.enabled {
return Err(anyhow!("user not enabled"));
return Err(anyhow!("user not enabled").into());
}
}
@@ -181,7 +224,6 @@ impl Resolve<UpdatePermissionOnTarget, User> for State {
(user_target_variant.as_ref(), resource_variant.as_ref());
db_client()
.await
.permissions
.update_one(
doc! {
@@ -210,7 +252,7 @@ impl Resolve<UpdatePermissionOnTarget, User> for State {
/// checks if inner id is actually a `name`, and replaces it with id if so.
async fn extract_user_target_with_validation(
user_target: &UserTarget,
) -> anyhow::Result<(UserTargetVariant, String)> {
) -> serror::Result<(UserTargetVariant, String)> {
match user_target {
UserTarget::User(ident) => {
let filter = match ObjectId::from_str(ident) {
@@ -218,7 +260,6 @@ async fn extract_user_target_with_validation(
Err(_) => doc! { "username": ident },
};
let id = db_client()
.await
.users
.find_one(filter)
.await
@@ -233,7 +274,6 @@ async fn extract_user_target_with_validation(
Err(_) => doc! { "name": ident },
};
let id = db_client()
.await
.user_groups
.find_one(filter)
.await
@@ -248,7 +288,7 @@ async fn extract_user_target_with_validation(
/// checks if inner id is actually a `name`, and replaces it with id if so.
async fn extract_resource_target_with_validation(
resource_target: &ResourceTarget,
) -> anyhow::Result<(ResourceTargetVariant, String)> {
) -> serror::Result<(ResourceTargetVariant, String)> {
match resource_target {
ResourceTarget::System(_) => {
let res = resource_target.extract_variant_id();
@@ -260,7 +300,6 @@ async fn extract_resource_target_with_validation(
Err(_) => doc! { "name": ident },
};
let id = db_client()
.await
.builds
.find_one(filter)
.await
@@ -275,7 +314,6 @@ async fn extract_resource_target_with_validation(
Err(_) => doc! { "name": ident },
};
let id = db_client()
.await
.builders
.find_one(filter)
.await
@@ -290,7 +328,6 @@ async fn extract_resource_target_with_validation(
Err(_) => doc! { "name": ident },
};
let id = db_client()
.await
.deployments
.find_one(filter)
.await
@@ -305,7 +342,6 @@ async fn extract_resource_target_with_validation(
Err(_) => doc! { "name": ident },
};
let id = db_client()
.await
.servers
.find_one(filter)
.await
@@ -320,7 +356,6 @@ async fn extract_resource_target_with_validation(
Err(_) => doc! { "name": ident },
};
let id = db_client()
.await
.repos
.find_one(filter)
.await
@@ -335,7 +370,6 @@ async fn extract_resource_target_with_validation(
Err(_) => doc! { "name": ident },
};
let id = db_client()
.await
.alerters
.find_one(filter)
.await
@@ -350,7 +384,6 @@ async fn extract_resource_target_with_validation(
Err(_) => doc! { "name": ident },
};
let id = db_client()
.await
.procedures
.find_one(filter)
.await
@@ -359,13 +392,26 @@ async fn extract_resource_target_with_validation(
.id;
Ok((ResourceTargetVariant::Procedure, id))
}
ResourceTarget::Action(ident) => {
let filter = match ObjectId::from_str(ident) {
Ok(id) => doc! { "_id": id },
Err(_) => doc! { "name": ident },
};
let id = db_client()
.actions
.find_one(filter)
.await
.context("failed to query db for actions")?
.context("no matching action found")?
.id;
Ok((ResourceTargetVariant::Action, id))
}
ResourceTarget::ServerTemplate(ident) => {
let filter = match ObjectId::from_str(ident) {
Ok(id) => doc! { "_id": id },
Err(_) => doc! { "name": ident },
};
let id = db_client()
.await
.server_templates
.find_one(filter)
.await
@@ -380,7 +426,6 @@ async fn extract_resource_target_with_validation(
Err(_) => doc! { "name": ident },
};
let id = db_client()
.await
.resource_syncs
.find_one(filter)
.await
@@ -395,7 +440,6 @@ async fn extract_resource_target_with_validation(
Err(_) => doc! { "name": ident },
};
let id = db_client()
.await
.stacks
.find_one(filter)
.await

View File

@@ -1,60 +1,80 @@
use komodo_client::{
api::write::*,
entities::{
permission::PermissionLevel, procedure::Procedure, user::User,
permission::PermissionLevel, procedure::Procedure, update::Update,
},
};
use resolver_api::Resolve;
use crate::{resource, state::State};
use crate::resource;
impl Resolve<CreateProcedure, User> for State {
#[instrument(name = "CreateProcedure", skip(self, user))]
use super::WriteArgs;
impl Resolve<WriteArgs> for CreateProcedure {
#[instrument(name = "CreateProcedure", skip(user))]
async fn resolve(
&self,
CreateProcedure { name, config }: CreateProcedure,
user: User,
) -> anyhow::Result<CreateProcedureResponse> {
resource::create::<Procedure>(&name, config, &user).await
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<CreateProcedureResponse> {
Ok(
resource::create::<Procedure>(&self.name, self.config, user)
.await?,
)
}
}
impl Resolve<CopyProcedure, User> for State {
#[instrument(name = "CopyProcedure", skip(self, user))]
impl Resolve<WriteArgs> for CopyProcedure {
#[instrument(name = "CopyProcedure", skip(user))]
async fn resolve(
&self,
CopyProcedure { name, id }: CopyProcedure,
user: User,
) -> anyhow::Result<CopyProcedureResponse> {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<CopyProcedureResponse> {
let Procedure { config, .. } =
resource::get_check_permissions::<Procedure>(
&id,
&user,
&self.id,
user,
PermissionLevel::Write,
)
.await?;
resource::create::<Procedure>(&name, config.into(), &user).await
Ok(
resource::create::<Procedure>(&self.name, config.into(), user)
.await?,
)
}
}
impl Resolve<UpdateProcedure, User> for State {
#[instrument(name = "UpdateProcedure", skip(self, user))]
impl Resolve<WriteArgs> for UpdateProcedure {
#[instrument(name = "UpdateProcedure", skip(user))]
async fn resolve(
&self,
UpdateProcedure { id, config }: UpdateProcedure,
user: User,
) -> anyhow::Result<UpdateProcedureResponse> {
resource::update::<Procedure>(&id, config, &user).await
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<UpdateProcedureResponse> {
Ok(
resource::update::<Procedure>(&self.id, self.config, user)
.await?,
)
}
}
impl Resolve<DeleteProcedure, User> for State {
#[instrument(name = "DeleteProcedure", skip(self, user))]
impl Resolve<WriteArgs> for RenameProcedure {
#[instrument(name = "RenameProcedure", skip(user))]
async fn resolve(
&self,
DeleteProcedure { id }: DeleteProcedure,
user: User,
) -> anyhow::Result<DeleteProcedureResponse> {
resource::delete::<Procedure>(&id, &user).await
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Update> {
Ok(
resource::rename::<Procedure>(&self.id, &self.name, user)
.await?,
)
}
}
impl Resolve<WriteArgs> for DeleteProcedure {
#[instrument(name = "DeleteProcedure", skip(args))]
async fn resolve(
self,
args: &WriteArgs,
) -> serror::Result<DeleteProcedureResponse> {
Ok(resource::delete::<Procedure>(&self.id, args).await?)
}
}

View File

@@ -1,10 +1,9 @@
use anyhow::{anyhow, Context};
use anyhow::{Context, anyhow};
use komodo_client::{
api::write::*,
entities::{
provider::{DockerRegistryAccount, GitProviderAccount},
user::User,
Operation, ResourceTarget,
provider::{DockerRegistryAccount, GitProviderAccount},
},
};
use mungos::{
@@ -15,39 +14,40 @@ use resolver_api::Resolve;
use crate::{
helpers::update::{add_update, make_update},
state::{db_client, State},
state::db_client,
};
impl Resolve<CreateGitProviderAccount, User> for State {
use super::WriteArgs;
impl Resolve<WriteArgs> for CreateGitProviderAccount {
async fn resolve(
&self,
CreateGitProviderAccount { account }: CreateGitProviderAccount,
user: User,
) -> anyhow::Result<CreateGitProviderAccountResponse> {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<CreateGitProviderAccountResponse> {
if !user.admin {
return Err(anyhow!(
"only admins can create git provider accounts"
));
return Err(
anyhow!("only admins can create git provider accounts")
.into(),
);
}
let mut account: GitProviderAccount = account.into();
let mut account: GitProviderAccount = self.account.into();
if account.domain.is_empty() {
return Err(anyhow!("domain cannot be empty string."));
return Err(anyhow!("domain cannot be empty string.").into());
}
if account.username.is_empty() {
return Err(anyhow!("username cannot be empty string."));
return Err(anyhow!("username cannot be empty string.").into());
}
let mut update = make_update(
ResourceTarget::system(),
Operation::CreateGitProviderAccount,
&user,
user,
);
account.id = db_client()
.await
.git_accounts
.insert_one(&account)
.await
@@ -78,62 +78,63 @@ impl Resolve<CreateGitProviderAccount, User> for State {
}
}
impl Resolve<UpdateGitProviderAccount, User> for State {
impl Resolve<WriteArgs> for UpdateGitProviderAccount {
async fn resolve(
&self,
UpdateGitProviderAccount { id, mut account }: UpdateGitProviderAccount,
user: User,
) -> anyhow::Result<UpdateGitProviderAccountResponse> {
mut self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<UpdateGitProviderAccountResponse> {
if !user.admin {
return Err(anyhow!(
"only admins can update git provider accounts"
));
return Err(
anyhow!("only admins can update git provider accounts")
.into(),
);
}
if let Some(domain) = &account.domain {
if let Some(domain) = &self.account.domain {
if domain.is_empty() {
return Err(anyhow!(
"cannot update git provider with empty domain"
));
return Err(
anyhow!("cannot update git provider with empty domain")
.into(),
);
}
}
if let Some(username) = &account.username {
if let Some(username) = &self.account.username {
if username.is_empty() {
return Err(anyhow!(
"cannot update git provider with empty username"
));
return Err(
anyhow!("cannot update git provider with empty username")
.into(),
);
}
}
// Ensure update does not change id
account.id = None;
self.account.id = None;
let mut update = make_update(
ResourceTarget::system(),
Operation::UpdateGitProviderAccount,
&user,
user,
);
let account = to_document(&account).context(
let account = to_document(&self.account).context(
"failed to serialize partial git provider account to bson",
)?;
let db = db_client().await;
let db = db_client();
update_one_by_id(
&db.git_accounts,
&id,
&self.id,
doc! { "$set": account },
None,
)
.await
.context("failed to update git provider account on db")?;
let Some(account) =
find_one_by_id(&db.git_accounts, &id)
.await
.context("failed to query db for git accounts")?
let Some(account) = find_one_by_id(&db.git_accounts, &self.id)
.await
.context("failed to query db for git accounts")?
else {
return Err(anyhow!("no account found with given id"));
return Err(anyhow!("no account found with given id").into());
};
update.push_simple_log(
@@ -157,33 +158,32 @@ impl Resolve<UpdateGitProviderAccount, User> for State {
}
}
impl Resolve<DeleteGitProviderAccount, User> for State {
impl Resolve<WriteArgs> for DeleteGitProviderAccount {
async fn resolve(
&self,
DeleteGitProviderAccount { id }: DeleteGitProviderAccount,
user: User,
) -> anyhow::Result<DeleteGitProviderAccountResponse> {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<DeleteGitProviderAccountResponse> {
if !user.admin {
return Err(anyhow!(
"only admins can delete git provider accounts"
));
return Err(
anyhow!("only admins can delete git provider accounts")
.into(),
);
}
let mut update = make_update(
ResourceTarget::system(),
Operation::UpdateGitProviderAccount,
&user,
user,
);
let db = db_client().await;
let Some(account) =
find_one_by_id(&db.git_accounts, &id)
.await
.context("failed to query db for git accounts")?
let db = db_client();
let Some(account) = find_one_by_id(&db.git_accounts, &self.id)
.await
.context("failed to query db for git accounts")?
else {
return Err(anyhow!("no account found with given id"));
return Err(anyhow!("no account found with given id").into());
};
delete_one_by_id(&db.git_accounts, &id, None)
delete_one_by_id(&db.git_accounts, &self.id, None)
.await
.context("failed to delete git account on db")?;
@@ -208,36 +208,37 @@ impl Resolve<DeleteGitProviderAccount, User> for State {
}
}
impl Resolve<CreateDockerRegistryAccount, User> for State {
impl Resolve<WriteArgs> for CreateDockerRegistryAccount {
async fn resolve(
&self,
CreateDockerRegistryAccount { account }: CreateDockerRegistryAccount,
user: User,
) -> anyhow::Result<CreateDockerRegistryAccountResponse> {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<CreateDockerRegistryAccountResponse> {
if !user.admin {
return Err(anyhow!(
"only admins can create docker registry account accounts"
));
return Err(
anyhow!(
"only admins can create docker registry account accounts"
)
.into(),
);
}
let mut account: DockerRegistryAccount = account.into();
let mut account: DockerRegistryAccount = self.account.into();
if account.domain.is_empty() {
return Err(anyhow!("domain cannot be empty string."));
return Err(anyhow!("domain cannot be empty string.").into());
}
if account.username.is_empty() {
return Err(anyhow!("username cannot be empty string."));
return Err(anyhow!("username cannot be empty string.").into());
}
let mut update = make_update(
ResourceTarget::system(),
Operation::CreateDockerRegistryAccount,
&user,
user,
);
account.id = db_client()
.await
.registry_accounts
.insert_one(&account)
.await
@@ -270,50 +271,56 @@ impl Resolve<CreateDockerRegistryAccount, User> for State {
}
}
impl Resolve<UpdateDockerRegistryAccount, User> for State {
impl Resolve<WriteArgs> for UpdateDockerRegistryAccount {
async fn resolve(
&self,
UpdateDockerRegistryAccount { id, mut account }: UpdateDockerRegistryAccount,
user: User,
) -> anyhow::Result<UpdateDockerRegistryAccountResponse> {
mut self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<UpdateDockerRegistryAccountResponse> {
if !user.admin {
return Err(anyhow!(
"only admins can update docker registry accounts"
));
return Err(
anyhow!("only admins can update docker registry accounts")
.into(),
);
}
if let Some(domain) = &account.domain {
if let Some(domain) = &self.account.domain {
if domain.is_empty() {
return Err(anyhow!(
"cannot update docker registry account with empty domain"
));
return Err(
anyhow!(
"cannot update docker registry account with empty domain"
)
.into(),
);
}
}
if let Some(username) = &account.username {
if let Some(username) = &self.account.username {
if username.is_empty() {
return Err(anyhow!(
"cannot update docker registry account with empty username"
));
return Err(
anyhow!(
"cannot update docker registry account with empty username"
)
.into(),
);
}
}
account.id = None;
self.account.id = None;
let mut update = make_update(
ResourceTarget::system(),
Operation::UpdateDockerRegistryAccount,
&user,
user,
);
let account = to_document(&account).context(
let account = to_document(&self.account).context(
"failed to serialize partial docker registry account account to bson",
)?;
let db = db_client().await;
let db = db_client();
update_one_by_id(
&db.registry_accounts,
&id,
&self.id,
doc! { "$set": account },
None,
)
@@ -322,11 +329,12 @@ impl Resolve<UpdateDockerRegistryAccount, User> for State {
"failed to update docker registry account account on db",
)?;
let Some(account) = find_one_by_id(&db.registry_accounts, &id)
.await
.context("failed to query db for registry accounts")?
let Some(account) =
find_one_by_id(&db.registry_accounts, &self.id)
.await
.context("failed to query db for registry accounts")?
else {
return Err(anyhow!("no account found with given id"));
return Err(anyhow!("no account found with given id").into());
};
update.push_simple_log(
@@ -350,32 +358,33 @@ impl Resolve<UpdateDockerRegistryAccount, User> for State {
}
}
impl Resolve<DeleteDockerRegistryAccount, User> for State {
impl Resolve<WriteArgs> for DeleteDockerRegistryAccount {
async fn resolve(
&self,
DeleteDockerRegistryAccount { id }: DeleteDockerRegistryAccount,
user: User,
) -> anyhow::Result<DeleteDockerRegistryAccountResponse> {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<DeleteDockerRegistryAccountResponse> {
if !user.admin {
return Err(anyhow!(
"only admins can delete docker registry accounts"
));
return Err(
anyhow!("only admins can delete docker registry accounts")
.into(),
);
}
let mut update = make_update(
ResourceTarget::system(),
Operation::UpdateDockerRegistryAccount,
&user,
user,
);
let db = db_client().await;
let Some(account) = find_one_by_id(&db.registry_accounts, &id)
.await
.context("failed to query db for git accounts")?
let db = db_client();
let Some(account) =
find_one_by_id(&db.registry_accounts, &self.id)
.await
.context("failed to query db for git accounts")?
else {
return Err(anyhow!("no account found with given id"));
return Err(anyhow!("no account found with given id").into());
};
delete_one_by_id(&db.registry_accounts, &id, None)
delete_one_by_id(&db.registry_accounts, &self.id, None)
.await
.context("failed to delete registry account on db")?;

View File

@@ -1,135 +1,211 @@
use anyhow::{anyhow, Context};
use mongo_indexed::doc;
use anyhow::{Context, anyhow};
use formatting::format_serror;
use git::GitRes;
use komodo_client::{
api::write::*,
entities::{
CloneArgs, NoData, Operation,
config::core::CoreConfig,
komodo_timestamp,
permission::PermissionLevel,
repo::{PartialRepoConfig, Repo, RepoInfo},
user::User,
CloneArgs, NoData,
server::Server,
to_komodo_name,
update::{Log, Update},
},
};
use mungos::mongodb::bson::to_document;
use mongo_indexed::doc;
use mungos::{by_id::update_one_by_id, mongodb::bson::to_document};
use octorust::types::{
ReposCreateWebhookRequest, ReposCreateWebhookRequestConfig,
};
use periphery_client::api;
use resolver_api::Resolve;
use crate::{
config::core_config,
helpers::{git_token, random_string},
helpers::{
git_token, periphery_client,
update::{add_update, make_update},
},
resource,
state::{db_client, github_client, State},
state::{action_states, db_client, github_client},
};
impl Resolve<CreateRepo, User> for State {
#[instrument(name = "CreateRepo", skip(self, user))]
use super::WriteArgs;
impl Resolve<WriteArgs> for CreateRepo {
#[instrument(name = "CreateRepo", skip(user))]
async fn resolve(
&self,
CreateRepo { name, config }: CreateRepo,
user: User,
) -> anyhow::Result<Repo> {
resource::create::<Repo>(&name, config, &user).await
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Repo> {
Ok(resource::create::<Repo>(&self.name, self.config, user).await?)
}
}
impl Resolve<CopyRepo, User> for State {
#[instrument(name = "CopyRepo", skip(self, user))]
impl Resolve<WriteArgs> for CopyRepo {
#[instrument(name = "CopyRepo", skip(user))]
async fn resolve(
&self,
CopyRepo { name, id }: CopyRepo,
user: User,
) -> anyhow::Result<Repo> {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Repo> {
let Repo { config, .. } =
resource::get_check_permissions::<Repo>(
&id,
&user,
&self.id,
user,
PermissionLevel::Write,
)
.await?;
resource::create::<Repo>(&name, config.into(), &user).await
Ok(
resource::create::<Repo>(&self.name, config.into(), user)
.await?,
)
}
}
impl Resolve<DeleteRepo, User> for State {
#[instrument(name = "DeleteRepo", skip(self, user))]
impl Resolve<WriteArgs> for DeleteRepo {
#[instrument(name = "DeleteRepo", skip(args))]
async fn resolve(self, args: &WriteArgs) -> serror::Result<Repo> {
Ok(resource::delete::<Repo>(&self.id, args).await?)
}
}
impl Resolve<WriteArgs> for UpdateRepo {
#[instrument(name = "UpdateRepo", skip(user))]
async fn resolve(
&self,
DeleteRepo { id }: DeleteRepo,
user: User,
) -> anyhow::Result<Repo> {
resource::delete::<Repo>(&id, &user).await
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Repo> {
Ok(resource::update::<Repo>(&self.id, self.config, user).await?)
}
}
impl Resolve<UpdateRepo, User> for State {
#[instrument(name = "UpdateRepo", skip(self, user))]
impl Resolve<WriteArgs> for RenameRepo {
#[instrument(name = "RenameRepo", skip(user))]
async fn resolve(
&self,
UpdateRepo { id, config }: UpdateRepo,
user: User,
) -> anyhow::Result<Repo> {
resource::update::<Repo>(&id, config, &user).await
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Update> {
let repo = resource::get_check_permissions::<Repo>(
&self.id,
user,
PermissionLevel::Write,
)
.await?;
if repo.config.server_id.is_empty()
|| !repo.config.path.is_empty()
{
return Ok(
resource::rename::<Repo>(&repo.id, &self.name, user).await?,
);
}
// get the action state for the repo (or insert default).
let action_state =
action_states().repo.get_or_insert_default(&repo.id).await;
// Will check to ensure repo not already busy before updating, and return Err if so.
// The returned guard will set the action state back to default when dropped.
let _action_guard =
action_state.update(|state| state.renaming = true)?;
let name = to_komodo_name(&self.name);
let mut update = make_update(&repo, Operation::RenameRepo, user);
update_one_by_id(
&db_client().repos,
&repo.id,
mungos::update::Update::Set(
doc! { "name": &name, "updated_at": komodo_timestamp() },
),
None,
)
.await
.context("Failed to update Repo name on db")?;
let server =
resource::get::<Server>(&repo.config.server_id).await?;
let log = match periphery_client(&server)?
.request(api::git::RenameRepo {
curr_name: to_komodo_name(&repo.name),
new_name: name.clone(),
})
.await
.context("Failed to rename Repo directory on Server")
{
Ok(log) => log,
Err(e) => Log::error(
"Rename Repo directory failure",
format_serror(&e.into()),
),
};
update.logs.push(log);
update.push_simple_log(
"Rename Repo",
format!("Renamed Repo from {} to {}", repo.name, name),
);
update.finalize();
update.id = add_update(update.clone()).await?;
Ok(update)
}
}
impl Resolve<RefreshRepoCache, User> for State {
impl Resolve<WriteArgs> for RefreshRepoCache {
#[instrument(
name = "RefreshRepoCache",
level = "debug",
skip(self, user)
skip(user)
)]
async fn resolve(
&self,
RefreshRepoCache { repo }: RefreshRepoCache,
user: User,
) -> anyhow::Result<NoData> {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<NoData> {
// Even though this is a write request, this doesn't change any config. Anyone that can execute the
// repo should be able to do this.
let repo = resource::get_check_permissions::<Repo>(
&repo,
&user,
&self.repo,
user,
PermissionLevel::Execute,
)
.await?;
if repo.config.repo.is_empty() {
if repo.config.git_provider.is_empty()
|| repo.config.repo.is_empty()
{
// Nothing to do
return Ok(NoData {});
}
let config = core_config();
let repo_dir = config.repo_directory.join(random_string(10));
let mut clone_args: CloneArgs = (&repo).into();
// No reason to to the commands here.
let repo_path =
clone_args.unique_path(&core_config().repo_directory)?;
clone_args.destination = Some(repo_path.display().to_string());
// Don't want to run these on core.
clone_args.on_clone = None;
clone_args.on_pull = None;
clone_args.destination = Some(repo_dir.display().to_string());
let access_token = match (&clone_args.account, &clone_args.provider)
{
(None, _) => None,
(Some(_), None) => {
return Err(anyhow!(
"Account is configured, but provider is empty"
))
}
(Some(username), Some(provider)) => {
git_token(provider, username, |https| {
let access_token = if let Some(username) = &clone_args.account {
git_token(&clone_args.provider, username, |https| {
clone_args.https = https
})
.await
.with_context(
|| format!("Failed to get git token in call to db. Stopping run. | {provider} | {username}"),
|| format!("Failed to get git token in call to db. Stopping run. | {} | {username}", clone_args.provider),
)?
}
} else {
None
};
let (_, latest_hash, latest_message, _) = git::clone(
let GitRes { hash, message, .. } = git::pull_or_clone(
clone_args,
&config.repo_directory,
&core_config().repo_directory,
access_token,
&[],
"",
@@ -137,22 +213,23 @@ impl Resolve<RefreshRepoCache, User> for State {
&[],
)
.await
.context("failed to clone repo (the resource) repo")?;
.with_context(|| {
format!("Failed to update repo at {repo_path:?}")
})?;
let info = RepoInfo {
last_pulled_at: repo.info.last_pulled_at,
last_built_at: repo.info.last_built_at,
built_hash: repo.info.built_hash,
built_message: repo.info.built_message,
latest_hash,
latest_message,
latest_hash: hash,
latest_message: message,
};
let info = to_document(&info)
.context("failed to serialize repo info to bson")?;
db_client()
.await
.repos
.update_one(
doc! { "name": &repo.name },
@@ -161,51 +238,46 @@ impl Resolve<RefreshRepoCache, User> for State {
.await
.context("failed to update repo info on db")?;
if repo_dir.exists() {
if let Err(e) = std::fs::remove_dir_all(&repo_dir) {
warn!(
"failed to remove repo (resource) cache update repo directory | {e:?}"
)
}
}
Ok(NoData {})
}
}
impl Resolve<CreateRepoWebhook, User> for State {
#[instrument(name = "CreateRepoWebhook", skip(self, user))]
impl Resolve<WriteArgs> for CreateRepoWebhook {
#[instrument(name = "CreateRepoWebhook", skip(args))]
async fn resolve(
&self,
CreateRepoWebhook { repo, action }: CreateRepoWebhook,
user: User,
) -> anyhow::Result<CreateRepoWebhookResponse> {
self,
args: &WriteArgs,
) -> serror::Result<CreateRepoWebhookResponse> {
let Some(github) = github_client() else {
return Err(anyhow!(
"github_webhook_app is not configured in core config toml"
));
return Err(
anyhow!(
"github_webhook_app is not configured in core config toml"
)
.into(),
);
};
let repo = resource::get_check_permissions::<Repo>(
&repo,
&user,
&self.repo,
&args.user,
PermissionLevel::Write,
)
.await?;
if repo.config.repo.is_empty() {
return Err(anyhow!(
"No repo configured, can't create webhook"
));
return Err(
anyhow!("No repo configured, can't create webhook").into(),
);
}
let mut split = repo.config.repo.split('/');
let owner = split.next().context("Repo repo has no owner")?;
let Some(github) = github.get(owner) else {
return Err(anyhow!(
"Cannot manage repo webhooks under owner {owner}"
));
return Err(
anyhow!("Cannot manage repo webhooks under owner {owner}")
.into(),
);
};
let repo_name =
@@ -233,8 +305,12 @@ impl Resolve<CreateRepoWebhook, User> for State {
&repo.config.webhook_secret
};
let host = webhook_base_url.as_ref().unwrap_or(host);
let url = match action {
let host = if webhook_base_url.is_empty() {
host
} else {
webhook_base_url
};
let url = match self.action {
RepoWebhookAction::Clone => {
format!("{host}/listener/github/repo/{}/clone", repo.id)
}
@@ -272,64 +348,65 @@ impl Resolve<CreateRepoWebhook, User> for State {
.context("failed to create webhook")?;
if !repo.config.webhook_enabled {
self
.resolve(
UpdateRepo {
id: repo.id,
config: PartialRepoConfig {
webhook_enabled: Some(true),
..Default::default()
},
},
user,
)
.await
.context("failed to update repo to enable webhook")?;
UpdateRepo {
id: repo.id,
config: PartialRepoConfig {
webhook_enabled: Some(true),
..Default::default()
},
}
.resolve(args)
.await
.map_err(|e| e.error)
.context("failed to update repo to enable webhook")?;
}
Ok(NoData {})
}
}
impl Resolve<DeleteRepoWebhook, User> for State {
#[instrument(name = "DeleteRepoWebhook", skip(self, user))]
impl Resolve<WriteArgs> for DeleteRepoWebhook {
#[instrument(name = "DeleteRepoWebhook", skip(user))]
async fn resolve(
&self,
DeleteRepoWebhook { repo, action }: DeleteRepoWebhook,
user: User,
) -> anyhow::Result<DeleteRepoWebhookResponse> {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<DeleteRepoWebhookResponse> {
let Some(github) = github_client() else {
return Err(anyhow!(
"github_webhook_app is not configured in core config toml"
));
return Err(
anyhow!(
"github_webhook_app is not configured in core config toml"
)
.into(),
);
};
let repo = resource::get_check_permissions::<Repo>(
&repo,
&user,
&self.repo,
user,
PermissionLevel::Write,
)
.await?;
if repo.config.git_provider != "github.com" {
return Err(anyhow!(
"Can only manage github.com repo webhooks"
));
return Err(
anyhow!("Can only manage github.com repo webhooks").into(),
);
}
if repo.config.repo.is_empty() {
return Err(anyhow!(
"No repo configured, can't create webhook"
));
return Err(
anyhow!("No repo configured, can't create webhook").into(),
);
}
let mut split = repo.config.repo.split('/');
let owner = split.next().context("Repo repo has no owner")?;
let Some(github) = github.get(owner) else {
return Err(anyhow!(
"Cannot manage repo webhooks under owner {owner}"
));
return Err(
anyhow!("Cannot manage repo webhooks under owner {owner}")
.into(),
);
};
let repo_name =
@@ -350,8 +427,12 @@ impl Resolve<DeleteRepoWebhook, User> for State {
..
} = core_config();
let host = webhook_base_url.as_ref().unwrap_or(host);
let url = match action {
let host = if webhook_base_url.is_empty() {
host
} else {
webhook_base_url
};
let url = match self.action {
RepoWebhookAction::Clone => {
format!("{host}/listener/github/repo/{}/clone", repo.id)
}

View File

@@ -3,15 +3,12 @@ use formatting::format_serror;
use komodo_client::{
api::write::*,
entities::{
komodo_timestamp,
NoData, Operation,
permission::PermissionLevel,
server::Server,
update::{Update, UpdateStatus},
user::User,
Operation,
},
};
use mungos::{by_id::update_one_by_id, mongodb::bson::doc};
use periphery_client::api;
use resolver_api::Resolve;
@@ -21,81 +18,59 @@ use crate::{
update::{add_update, make_update, update_update},
},
resource,
state::{db_client, State},
};
impl Resolve<CreateServer, User> for State {
#[instrument(name = "CreateServer", skip(self, user))]
async fn resolve(
&self,
CreateServer { name, config }: CreateServer,
user: User,
) -> anyhow::Result<Server> {
resource::create::<Server>(&name, config, &user).await
}
}
use super::WriteArgs;
impl Resolve<DeleteServer, User> for State {
#[instrument(name = "DeleteServer", skip(self, user))]
impl Resolve<WriteArgs> for CreateServer {
#[instrument(name = "CreateServer", skip(user))]
async fn resolve(
&self,
DeleteServer { id }: DeleteServer,
user: User,
) -> anyhow::Result<Server> {
resource::delete::<Server>(&id, &user).await
}
}
impl Resolve<UpdateServer, User> for State {
#[instrument(name = "UpdateServer", skip(self, user))]
async fn resolve(
&self,
UpdateServer { id, config }: UpdateServer,
user: User,
) -> anyhow::Result<Server> {
resource::update::<Server>(&id, config, &user).await
}
}
impl Resolve<RenameServer, User> for State {
#[instrument(name = "RenameServer", skip(self, user))]
async fn resolve(
&self,
RenameServer { id, name }: RenameServer,
user: User,
) -> anyhow::Result<Update> {
let server = resource::get_check_permissions::<Server>(
&id,
&user,
PermissionLevel::Write,
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Server> {
Ok(
resource::create::<Server>(&self.name, self.config, user)
.await?,
)
.await?;
let mut update =
make_update(&server, Operation::RenameServer, &user);
update_one_by_id(&db_client().await.servers, &id, mungos::update::Update::Set(doc! { "name": &name, "updated_at": komodo_timestamp() }), None)
.await
.context("failed to update server on db. this name may already be taken.")?;
update.push_simple_log(
"rename server",
format!("renamed server {id} from {} to {name}", server.name),
);
update.finalize();
update.id = add_update(update.clone()).await?;
Ok(update)
}
}
impl Resolve<CreateNetwork, User> for State {
#[instrument(name = "CreateNetwork", skip(self, user))]
impl Resolve<WriteArgs> for DeleteServer {
#[instrument(name = "DeleteServer", skip(args))]
async fn resolve(self, args: &WriteArgs) -> serror::Result<Server> {
Ok(resource::delete::<Server>(&self.id, args).await?)
}
}
impl Resolve<WriteArgs> for UpdateServer {
#[instrument(name = "UpdateServer", skip(user))]
async fn resolve(
&self,
CreateNetwork { server, name }: CreateNetwork,
user: User,
) -> anyhow::Result<Update> {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Server> {
Ok(resource::update::<Server>(&self.id, self.config, user).await?)
}
}
impl Resolve<WriteArgs> for RenameServer {
#[instrument(name = "RenameServer", skip(user))]
async fn resolve(
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Update> {
Ok(resource::rename::<Server>(&self.id, &self.name, user).await?)
}
}
impl Resolve<WriteArgs> for CreateNetwork {
#[instrument(name = "CreateNetwork", skip(user))]
async fn resolve(
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Update> {
let server = resource::get_check_permissions::<Server>(
&server,
&user,
&self.server,
user,
PermissionLevel::Write,
)
.await?;
@@ -103,12 +78,15 @@ impl Resolve<CreateNetwork, User> for State {
let periphery = periphery_client(&server)?;
let mut update =
make_update(&server, Operation::CreateNetwork, &user);
make_update(&server, Operation::CreateNetwork, user);
update.status = UpdateStatus::InProgress;
update.id = add_update(update.clone()).await?;
match periphery
.request(api::network::CreateNetwork { name, driver: None })
.request(api::network::CreateNetwork {
name: self.name,
driver: None,
})
.await
{
Ok(log) => update.logs.push(log),
@@ -124,3 +102,81 @@ impl Resolve<CreateNetwork, User> for State {
Ok(update)
}
}
impl Resolve<WriteArgs> for CreateTerminal {
#[instrument(name = "CreateTerminal", skip(user))]
async fn resolve(
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<NoData> {
let server = resource::get_check_permissions::<Server>(
&self.server,
user,
PermissionLevel::Write,
)
.await?;
let periphery = periphery_client(&server)?;
periphery
.request(api::terminal::CreateTerminal {
name: self.name,
command: self.command,
recreate: self.recreate,
})
.await
.context("Failed to create terminal on periphery")?;
Ok(NoData {})
}
}
impl Resolve<WriteArgs> for DeleteTerminal {
#[instrument(name = "DeleteTerminal", skip(user))]
async fn resolve(
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<NoData> {
let server = resource::get_check_permissions::<Server>(
&self.server,
user,
PermissionLevel::Write,
)
.await?;
let periphery = periphery_client(&server)?;
periphery
.request(api::terminal::DeleteTerminal {
terminal: self.terminal,
})
.await
.context("Failed to delete terminal on periphery")?;
Ok(NoData {})
}
}
impl Resolve<WriteArgs> for DeleteAllTerminals {
#[instrument(name = "DeleteAllTerminals", skip(user))]
async fn resolve(
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<NoData> {
let server = resource::get_check_permissions::<Server>(
&self.server,
user,
PermissionLevel::Write,
)
.await?;
let periphery = periphery_client(&server)?;
periphery
.request(api::terminal::DeleteAllTerminals {})
.await
.context("Failed to delete all terminals on periphery")?;
Ok(NoData {})
}
}

View File

@@ -1,65 +1,92 @@
use komodo_client::{
api::write::{
CopyServerTemplate, CreateServerTemplate, DeleteServerTemplate,
UpdateServerTemplate,
RenameServerTemplate, UpdateServerTemplate,
},
entities::{
permission::PermissionLevel, server_template::ServerTemplate,
user::User,
update::Update,
},
};
use resolver_api::Resolve;
use crate::{resource, state::State};
use crate::resource;
impl Resolve<CreateServerTemplate, User> for State {
#[instrument(name = "CreateServerTemplate", skip(self, user))]
use super::WriteArgs;
impl Resolve<WriteArgs> for CreateServerTemplate {
#[instrument(name = "CreateServerTemplate", skip(user))]
async fn resolve(
&self,
CreateServerTemplate { name, config }: CreateServerTemplate,
user: User,
) -> anyhow::Result<ServerTemplate> {
resource::create::<ServerTemplate>(&name, config, &user).await
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<ServerTemplate> {
Ok(
resource::create::<ServerTemplate>(
&self.name,
self.config,
user,
)
.await?,
)
}
}
impl Resolve<CopyServerTemplate, User> for State {
#[instrument(name = "CopyServerTemplate", skip(self, user))]
impl Resolve<WriteArgs> for CopyServerTemplate {
#[instrument(name = "CopyServerTemplate", skip(user))]
async fn resolve(
&self,
CopyServerTemplate { name, id }: CopyServerTemplate,
user: User,
) -> anyhow::Result<ServerTemplate> {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<ServerTemplate> {
let ServerTemplate { config, .. } =
resource::get_check_permissions::<ServerTemplate>(
&id,
&user,
&self.id,
user,
PermissionLevel::Write,
)
.await?;
resource::create::<ServerTemplate>(&name, config.into(), &user)
.await
Ok(
resource::create::<ServerTemplate>(
&self.name,
config.into(),
user,
)
.await?,
)
}
}
impl Resolve<DeleteServerTemplate, User> for State {
#[instrument(name = "DeleteServerTemplate", skip(self, user))]
impl Resolve<WriteArgs> for DeleteServerTemplate {
#[instrument(name = "DeleteServerTemplate", skip(args))]
async fn resolve(
&self,
DeleteServerTemplate { id }: DeleteServerTemplate,
user: User,
) -> anyhow::Result<ServerTemplate> {
resource::delete::<ServerTemplate>(&id, &user).await
self,
args: &WriteArgs,
) -> serror::Result<ServerTemplate> {
Ok(resource::delete::<ServerTemplate>(&self.id, args).await?)
}
}
impl Resolve<UpdateServerTemplate, User> for State {
#[instrument(name = "UpdateServerTemplate", skip(self, user))]
impl Resolve<WriteArgs> for UpdateServerTemplate {
#[instrument(name = "UpdateServerTemplate", skip(user))]
async fn resolve(
&self,
UpdateServerTemplate { id, config }: UpdateServerTemplate,
user: User,
) -> anyhow::Result<ServerTemplate> {
resource::update::<ServerTemplate>(&id, config, &user).await
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<ServerTemplate> {
Ok(
resource::update::<ServerTemplate>(&self.id, self.config, user)
.await?,
)
}
}
impl Resolve<WriteArgs> for RenameServerTemplate {
#[instrument(name = "RenameServerTemplate", skip(user))]
async fn resolve(
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Update> {
Ok(
resource::rename::<ServerTemplate>(&self.id, &self.name, user)
.await?,
)
}
}

View File

@@ -1,17 +1,8 @@
use std::str::FromStr;
use anyhow::{anyhow, Context};
use anyhow::{Context, anyhow};
use komodo_client::{
api::{
user::CreateApiKey,
write::{
CreateApiKeyForServiceUser, CreateApiKeyForServiceUserResponse,
CreateServiceUser, CreateServiceUserResponse,
DeleteApiKeyForServiceUser, DeleteApiKeyForServiceUserResponse,
UpdateServiceUserDescription,
UpdateServiceUserDescriptionResponse,
},
},
api::{user::CreateApiKey, write::*},
entities::{
komodo_timestamp,
user::{User, UserConfig},
@@ -23,31 +14,34 @@ use mungos::{
};
use resolver_api::Resolve;
use crate::state::{db_client, State};
use crate::{api::user::UserArgs, state::db_client};
impl Resolve<CreateServiceUser, User> for State {
#[instrument(name = "CreateServiceUser", skip(self, user))]
use super::WriteArgs;
impl Resolve<WriteArgs> for CreateServiceUser {
#[instrument(name = "CreateServiceUser", skip(user))]
async fn resolve(
&self,
CreateServiceUser {
username,
description,
}: CreateServiceUser,
user: User,
) -> anyhow::Result<CreateServiceUserResponse> {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<CreateServiceUserResponse> {
if !user.admin {
return Err(anyhow!("user not admin"));
return Err(anyhow!("user not admin").into());
}
if ObjectId::from_str(&username).is_ok() {
return Err(anyhow!("username cannot be valid ObjectId"));
if ObjectId::from_str(&self.username).is_ok() {
return Err(
anyhow!("username cannot be valid ObjectId").into(),
);
}
let config = UserConfig::Service { description };
let config = UserConfig::Service {
description: self.description,
};
let mut user = User {
id: Default::default(),
username,
username: self.username,
config,
enabled: true,
admin: false,
super_admin: false,
create_server_permissions: false,
create_build_permissions: false,
last_update_view: 0,
@@ -56,7 +50,6 @@ impl Resolve<CreateServiceUser, User> for State {
updated_at: komodo_timestamp(),
};
user.id = db_client()
.await
.users
.insert_one(&user)
.await
@@ -69,102 +62,94 @@ impl Resolve<CreateServiceUser, User> for State {
}
}
impl Resolve<UpdateServiceUserDescription, User> for State {
#[instrument(
name = "UpdateServiceUserDescription",
skip(self, user)
)]
impl Resolve<WriteArgs> for UpdateServiceUserDescription {
#[instrument(name = "UpdateServiceUserDescription", skip(user))]
async fn resolve(
&self,
UpdateServiceUserDescription {
username,
description,
}: UpdateServiceUserDescription,
user: User,
) -> anyhow::Result<UpdateServiceUserDescriptionResponse> {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<UpdateServiceUserDescriptionResponse> {
if !user.admin {
return Err(anyhow!("user not admin"));
return Err(anyhow!("user not admin").into());
}
let db = db_client().await;
let db = db_client();
let service_user = db
.users
.find_one(doc! { "username": &username })
.find_one(doc! { "username": &self.username })
.await
.context("failed to query db for user")?
.context("no user with given username")?;
let UserConfig::Service { .. } = &service_user.config else {
return Err(anyhow!("user is not service user"));
return Err(anyhow!("user is not service user").into());
};
db.users
.update_one(
doc! { "username": &username },
doc! { "$set": { "config.data.description": description } },
doc! { "username": &self.username },
doc! { "$set": { "config.data.description": self.description } },
)
.await
.context("failed to update user on db")?;
db.users
.find_one(doc! { "username": &username })
let res = db
.users
.find_one(doc! { "username": &self.username })
.await
.context("failed to query db for user")?
.context("user with username not found")
.context("user with username not found")?;
Ok(res)
}
}
impl Resolve<CreateApiKeyForServiceUser, User> for State {
#[instrument(name = "CreateApiKeyForServiceUser", skip(self, user))]
impl Resolve<WriteArgs> for CreateApiKeyForServiceUser {
#[instrument(name = "CreateApiKeyForServiceUser", skip(user))]
async fn resolve(
&self,
CreateApiKeyForServiceUser {
user_id,
name,
expires,
}: CreateApiKeyForServiceUser,
user: User,
) -> anyhow::Result<CreateApiKeyForServiceUserResponse> {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<CreateApiKeyForServiceUserResponse> {
if !user.admin {
return Err(anyhow!("user not admin"));
return Err(anyhow!("user not admin").into());
}
let service_user =
find_one_by_id(&db_client().await.users, &user_id)
find_one_by_id(&db_client().users, &self.user_id)
.await
.context("failed to query db for user")?
.context("no user found with id")?;
let UserConfig::Service { .. } = &service_user.config else {
return Err(anyhow!("user is not service user"));
return Err(anyhow!("user is not service user").into());
};
self
.resolve(CreateApiKey { name, expires }, service_user)
.await
CreateApiKey {
name: self.name,
expires: self.expires,
}
.resolve(&UserArgs { user: service_user })
.await
}
}
impl Resolve<DeleteApiKeyForServiceUser, User> for State {
#[instrument(name = "DeleteApiKeyForServiceUser", skip(self, user))]
impl Resolve<WriteArgs> for DeleteApiKeyForServiceUser {
#[instrument(name = "DeleteApiKeyForServiceUser", skip(user))]
async fn resolve(
&self,
DeleteApiKeyForServiceUser { key }: DeleteApiKeyForServiceUser,
user: User,
) -> anyhow::Result<DeleteApiKeyForServiceUserResponse> {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<DeleteApiKeyForServiceUserResponse> {
if !user.admin {
return Err(anyhow!("user not admin"));
return Err(anyhow!("user not admin").into());
}
let db = db_client().await;
let db = db_client();
let api_key = db
.api_keys
.find_one(doc! { "key": &key })
.find_one(doc! { "key": &self.key })
.await
.context("failed to query db for api key")?
.context("did not find matching api key")?;
let service_user =
find_one_by_id(&db_client().await.users, &api_key.user_id)
find_one_by_id(&db_client().users, &api_key.user_id)
.await
.context("failed to query db for user")?
.context("no user found with id")?;
let UserConfig::Service { .. } = &service_user.config else {
return Err(anyhow!("user is not service user"));
return Err(anyhow!("user is not service user").into());
};
db.api_keys
.delete_one(doc! { "key": key })
.delete_one(doc! { "key": self.key })
.await
.context("failed to delete api key on db")?;
Ok(DeleteApiKeyForServiceUserResponse {})

View File

@@ -1,161 +1,247 @@
use anyhow::{anyhow, Context};
use anyhow::{Context, anyhow};
use formatting::format_serror;
use komodo_client::{
api::write::*,
entities::{
FileContents, NoData, Operation,
config::core::CoreConfig,
komodo_timestamp,
permission::PermissionLevel,
server::ServerState,
stack::{ComposeContents, PartialStackConfig, Stack, StackInfo},
stack::{PartialStackConfig, Stack, StackInfo},
update::Update,
user::User,
NoData, Operation,
user::stack_user,
},
};
use mungos::{
by_id::update_one_by_id,
mongodb::bson::{doc, to_document},
};
use mungos::mongodb::bson::{doc, to_document};
use octorust::types::{
ReposCreateWebhookRequest, ReposCreateWebhookRequestConfig,
};
use periphery_client::api::compose::{
GetComposeContentsOnHost, GetComposeContentsOnHostResponse,
WriteCommitComposeContents, WriteComposeContentsToHost,
};
use resolver_api::Resolve;
use crate::{
api::execute::pull_stack_inner,
config::core_config,
helpers::{
periphery_client,
git_token, periphery_client,
query::get_server_with_state,
stack::{
remote::get_remote_compose_contents,
services::extract_services_into_res,
},
update::{add_update, make_update},
},
resource,
state::{db_client, github_client, State},
stack::{
get_stack_and_server,
remote::{RemoteComposeContents, get_repo_compose_contents},
services::extract_services_into_res,
},
state::{db_client, github_client},
};
impl Resolve<CreateStack, User> for State {
#[instrument(name = "CreateStack", skip(self, user))]
use super::WriteArgs;
impl Resolve<WriteArgs> for CreateStack {
#[instrument(name = "CreateStack", skip(user))]
async fn resolve(
&self,
CreateStack { name, config }: CreateStack,
user: User,
) -> anyhow::Result<Stack> {
resource::create::<Stack>(&name, config, &user).await
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Stack> {
Ok(
resource::create::<Stack>(&self.name, self.config, user)
.await?,
)
}
}
impl Resolve<CopyStack, User> for State {
#[instrument(name = "CopyStack", skip(self, user))]
impl Resolve<WriteArgs> for CopyStack {
#[instrument(name = "CopyStack", skip(user))]
async fn resolve(
&self,
CopyStack { name, id }: CopyStack,
user: User,
) -> anyhow::Result<Stack> {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Stack> {
let Stack { config, .. } =
resource::get_check_permissions::<Stack>(
&id,
&user,
&self.id,
user,
PermissionLevel::Write,
)
.await?;
resource::create::<Stack>(&name, config.into(), &user).await
Ok(
resource::create::<Stack>(&self.name, config.into(), user)
.await?,
)
}
}
impl Resolve<DeleteStack, User> for State {
#[instrument(name = "DeleteStack", skip(self, user))]
async fn resolve(
&self,
DeleteStack { id }: DeleteStack,
user: User,
) -> anyhow::Result<Stack> {
resource::delete::<Stack>(&id, &user).await
impl Resolve<WriteArgs> for DeleteStack {
#[instrument(name = "DeleteStack", skip(args))]
async fn resolve(self, args: &WriteArgs) -> serror::Result<Stack> {
Ok(resource::delete::<Stack>(&self.id, args).await?)
}
}
impl Resolve<UpdateStack, User> for State {
#[instrument(name = "UpdateStack", skip(self, user))]
impl Resolve<WriteArgs> for UpdateStack {
#[instrument(name = "UpdateStack", skip(user))]
async fn resolve(
&self,
UpdateStack { id, config }: UpdateStack,
user: User,
) -> anyhow::Result<Stack> {
resource::update::<Stack>(&id, config, &user).await
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Stack> {
Ok(resource::update::<Stack>(&self.id, self.config, user).await?)
}
}
impl Resolve<RenameStack, User> for State {
#[instrument(name = "RenameStack", skip(self, user))]
impl Resolve<WriteArgs> for RenameStack {
#[instrument(name = "RenameStack", skip(user))]
async fn resolve(
&self,
RenameStack { id, name }: RenameStack,
user: User,
) -> anyhow::Result<Update> {
let stack = resource::get_check_permissions::<Stack>(
&id,
&user,
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Update> {
Ok(resource::rename::<Stack>(&self.id, &self.name, user).await?)
}
}
impl Resolve<WriteArgs> for WriteStackFileContents {
#[instrument(name = "WriteStackFileContents", skip(user))]
async fn resolve(
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Update> {
let WriteStackFileContents {
stack,
file_path,
contents,
} = self;
let (mut stack, server) = get_stack_and_server(
&stack,
user,
PermissionLevel::Write,
true,
)
.await?;
if !stack.config.files_on_host && stack.config.repo.is_empty() {
return Err(anyhow!(
"Stack is not configured to use Files on Host or Git Repo, can't write file contents"
).into());
}
let mut update =
make_update(&stack, Operation::RenameStack, &user);
make_update(&stack, Operation::WriteStackContents, user);
update_one_by_id(
&db_client().await.stacks,
&stack.id,
mungos::update::Update::Set(
doc! { "name": &name, "updated_at": komodo_timestamp() },
),
None,
)
.await
.context("failed to update stack name on db")?;
update.push_simple_log("File contents to write", &contents);
let stack_id = stack.id.clone();
if stack.config.files_on_host {
match periphery_client(&server)?
.request(WriteComposeContentsToHost {
name: stack.name,
run_directory: stack.config.run_directory,
file_path,
contents,
})
.await
.context("Failed to write contents to host")
{
Ok(log) => {
update.logs.push(log);
}
Err(e) => {
update.push_error_log(
"Write File Contents",
format_serror(&e.into()),
);
}
};
} else {
let git_token = if !stack.config.git_account.is_empty() {
git_token(
&stack.config.git_provider,
&stack.config.git_account,
|https| stack.config.git_https = https,
)
.await
.with_context(|| {
format!(
"Failed to get git token. | {} | {}",
stack.config.git_account, stack.config.git_provider
)
})?
} else {
None
};
match periphery_client(&server)?
.request(WriteCommitComposeContents {
stack,
username: Some(user.username.clone()),
file_path,
contents,
git_token,
})
.await
.context("Failed to write contents to host")
{
Ok(res) => {
update.logs.extend(res.logs);
}
Err(e) => {
update.push_error_log(
"Write File Contents",
format_serror(&e.into()),
);
}
};
}
if let Err(e) = (RefreshStackCache { stack: stack_id })
.resolve(&WriteArgs {
user: stack_user().to_owned(),
})
.await
.map_err(|e| e.error)
.context(
"Failed to refresh stack cache after writing file contents",
)
{
update.push_error_log(
"Refresh stack cache",
format_serror(&e.into()),
);
}
update.push_simple_log(
"rename stack",
format!("renamed stack from {} to {}", stack.name, name),
);
update.finalize();
add_update(update.clone()).await?;
update.id = add_update(update.clone()).await?;
Ok(update)
}
}
impl Resolve<RefreshStackCache, User> for State {
impl Resolve<WriteArgs> for RefreshStackCache {
#[instrument(
name = "RefreshStackCache",
level = "debug",
skip(self, user)
skip(user)
)]
async fn resolve(
&self,
RefreshStackCache { stack }: RefreshStackCache,
user: User,
) -> anyhow::Result<NoData> {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<NoData> {
// Even though this is a write request, this doesn't change any config. Anyone that can execute the
// stack should be able to do this.
let stack = resource::get_check_permissions::<Stack>(
&stack,
&user,
&self.stack,
user,
PermissionLevel::Execute,
)
.await?;
let file_contents_empty = stack.config.file_contents.is_empty();
let repo_empty = stack.config.repo.is_empty();
if !stack.config.files_on_host
&& file_contents_empty
&& stack.config.repo.is_empty()
&& repo_empty
{
// Nothing to do without one of these
return Ok(NoData {});
@@ -173,69 +259,70 @@ impl Resolve<RefreshStackCache, User> for State {
// =============
// FILES ON HOST
// =============
if stack.config.server_id.is_empty() {
(vec![], None, None, None, None)
let (server, state) = if stack.config.server_id.is_empty() {
(None, ServerState::Disabled)
} else {
let (server, status) =
let (server, state) =
get_server_with_state(&stack.config.server_id).await?;
if status != ServerState::Ok {
(vec![], None, None, None, None)
} else {
let GetComposeContentsOnHostResponse { contents, errors } =
match periphery_client(&server)?
.request(GetComposeContentsOnHost {
file_paths: stack.file_paths().to_vec(),
name: stack.name.clone(),
run_directory: stack.config.run_directory.clone(),
})
.await
.context(
"failed to get compose file contents from host",
) {
Ok(res) => res,
Err(e) => GetComposeContentsOnHostResponse {
contents: Default::default(),
errors: vec![ComposeContents {
path: stack.config.run_directory.clone(),
contents: format_serror(&e.into()),
}],
},
};
(Some(server), state)
};
if state != ServerState::Ok {
(vec![], None, None, None, None)
} else if let Some(server) = server {
let GetComposeContentsOnHostResponse { contents, errors } =
match periphery_client(&server)?
.request(GetComposeContentsOnHost {
file_paths: stack.file_paths().to_vec(),
name: stack.name.clone(),
run_directory: stack.config.run_directory.clone(),
})
.await
.context("failed to get compose file contents from host")
{
Ok(res) => res,
Err(e) => GetComposeContentsOnHostResponse {
contents: Default::default(),
errors: vec![FileContents {
path: stack.config.run_directory.clone(),
contents: format_serror(&e.into()),
}],
},
};
let project_name = stack.project_name(true);
let project_name = stack.project_name(true);
let mut services = Vec::new();
let mut services = Vec::new();
for contents in &contents {
if let Err(e) = extract_services_into_res(
&project_name,
&contents.contents,
&mut services,
) {
warn!(
"failed to extract stack services, things won't works correctly. stack: {} | {e:#}",
stack.name
);
}
for contents in &contents {
if let Err(e) = extract_services_into_res(
&project_name,
&contents.contents,
&mut services,
) {
warn!(
"failed to extract stack services, things won't works correctly. stack: {} | {e:#}",
stack.name
);
}
(services, Some(contents), Some(errors), None, None)
}
(services, Some(contents), Some(errors), None, None)
} else {
(vec![], None, None, None, None)
}
} else if file_contents_empty {
} else if !repo_empty {
// ================
// REPO BASED STACK
// ================
let (
remote_contents,
remote_errors,
_,
latest_hash,
latest_message,
) =
get_remote_compose_contents(&stack, Some(&mut missing_files))
.await
.context("failed to clone remote compose file")?;
let RemoteComposeContents {
successful: remote_contents,
errored: remote_errors,
hash: latest_hash,
message: latest_message,
..
} = get_repo_compose_contents(&stack, Some(&mut missing_files))
.await?;
let project_name = stack.project_name(true);
let mut services = Vec::new();
@@ -272,21 +359,22 @@ impl Resolve<RefreshStackCache, User> for State {
&mut services,
) {
warn!(
"failed to extract stack services, things won't works correctly. stack: {} | {e:#}",
"Failed to extract Stack services for {}, things may not work correctly. | {e:#}",
stack.name
);
services.extend(stack.info.latest_services);
services.extend(stack.info.latest_services.clone());
};
(services, None, None, None, None)
};
let info = StackInfo {
missing_files,
deployed_services: stack.info.deployed_services,
deployed_project_name: stack.info.deployed_project_name,
deployed_contents: stack.info.deployed_contents,
deployed_hash: stack.info.deployed_hash,
deployed_message: stack.info.deployed_message,
deployed_services: stack.info.deployed_services.clone(),
deployed_project_name: stack.info.deployed_project_name.clone(),
deployed_contents: stack.info.deployed_contents.clone(),
deployed_config: stack.info.deployed_config.clone(),
deployed_hash: stack.info.deployed_hash.clone(),
deployed_message: stack.info.deployed_message.clone(),
latest_services,
remote_contents,
remote_errors,
@@ -298,7 +386,6 @@ impl Resolve<RefreshStackCache, User> for State {
.context("failed to serialize stack info to bson")?;
db_client()
.await
.stacks
.update_one(
doc! { "name": &stack.name },
@@ -307,43 +394,65 @@ impl Resolve<RefreshStackCache, User> for State {
.await
.context("failed to update stack info on db")?;
if (stack.config.poll_for_updates || stack.config.auto_update)
&& !stack.config.server_id.is_empty()
{
let (server, state) =
get_server_with_state(&stack.config.server_id).await?;
if state == ServerState::Ok {
let name = stack.name.clone();
if let Err(e) =
pull_stack_inner(stack, Vec::new(), &server, None).await
{
warn!(
"Failed to pull latest images for Stack {name} | {e:#}",
);
}
}
}
Ok(NoData {})
}
}
impl Resolve<CreateStackWebhook, User> for State {
#[instrument(name = "CreateStackWebhook", skip(self, user))]
impl Resolve<WriteArgs> for CreateStackWebhook {
#[instrument(name = "CreateStackWebhook", skip(args))]
async fn resolve(
&self,
CreateStackWebhook { stack, action }: CreateStackWebhook,
user: User,
) -> anyhow::Result<CreateStackWebhookResponse> {
self,
args: &WriteArgs,
) -> serror::Result<CreateStackWebhookResponse> {
let WriteArgs { user } = args;
let Some(github) = github_client() else {
return Err(anyhow!(
"github_webhook_app is not configured in core config toml"
));
return Err(
anyhow!(
"github_webhook_app is not configured in core config toml"
)
.into(),
);
};
let stack = resource::get_check_permissions::<Stack>(
&stack,
&user,
&self.stack,
user,
PermissionLevel::Write,
)
.await?;
if stack.config.repo.is_empty() {
return Err(anyhow!(
"No repo configured, can't create webhook"
));
return Err(
anyhow!("No repo configured, can't create webhook").into(),
);
}
let mut split = stack.config.repo.split('/');
let owner = split.next().context("Stack repo has no owner")?;
let Some(github) = github.get(owner) else {
return Err(anyhow!(
"Cannot manage repo webhooks under owner {owner}"
));
return Err(
anyhow!("Cannot manage repo webhooks under owner {owner}")
.into(),
);
};
let repo =
@@ -371,8 +480,12 @@ impl Resolve<CreateStackWebhook, User> for State {
&stack.config.webhook_secret
};
let host = webhook_base_url.as_ref().unwrap_or(host);
let url = match action {
let host = if webhook_base_url.is_empty() {
host
} else {
webhook_base_url
};
let url = match self.action {
StackWebhookAction::Refresh => {
format!("{host}/listener/github/stack/{}/refresh", stack.id)
}
@@ -407,64 +520,65 @@ impl Resolve<CreateStackWebhook, User> for State {
.context("failed to create webhook")?;
if !stack.config.webhook_enabled {
self
.resolve(
UpdateStack {
id: stack.id,
config: PartialStackConfig {
webhook_enabled: Some(true),
..Default::default()
},
},
user,
)
.await
.context("failed to update stack to enable webhook")?;
UpdateStack {
id: stack.id,
config: PartialStackConfig {
webhook_enabled: Some(true),
..Default::default()
},
}
.resolve(args)
.await
.map_err(|e| e.error)
.context("failed to update stack to enable webhook")?;
}
Ok(NoData {})
}
}
impl Resolve<DeleteStackWebhook, User> for State {
#[instrument(name = "DeleteStackWebhook", skip(self, user))]
impl Resolve<WriteArgs> for DeleteStackWebhook {
#[instrument(name = "DeleteStackWebhook", skip(user))]
async fn resolve(
&self,
DeleteStackWebhook { stack, action }: DeleteStackWebhook,
user: User,
) -> anyhow::Result<DeleteStackWebhookResponse> {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<DeleteStackWebhookResponse> {
let Some(github) = github_client() else {
return Err(anyhow!(
"github_webhook_app is not configured in core config toml"
));
return Err(
anyhow!(
"github_webhook_app is not configured in core config toml"
)
.into(),
);
};
let stack = resource::get_check_permissions::<Stack>(
&stack,
&user,
&self.stack,
user,
PermissionLevel::Write,
)
.await?;
if stack.config.git_provider != "github.com" {
return Err(anyhow!(
"Can only manage github.com repo webhooks"
));
return Err(
anyhow!("Can only manage github.com repo webhooks").into(),
);
}
if stack.config.repo.is_empty() {
return Err(anyhow!(
"No repo configured, can't create webhook"
));
return Err(
anyhow!("No repo configured, can't create webhook").into(),
);
}
let mut split = stack.config.repo.split('/');
let owner = split.next().context("Stack repo has no owner")?;
let Some(github) = github.get(owner) else {
return Err(anyhow!(
"Cannot manage repo webhooks under owner {owner}"
));
return Err(
anyhow!("Cannot manage repo webhooks under owner {owner}")
.into(),
);
};
let repo =
@@ -485,8 +599,12 @@ impl Resolve<DeleteStackWebhook, User> for State {
..
} = core_config();
let host = webhook_base_url.as_ref().unwrap_or(host);
let url = match action {
let host = if webhook_base_url.is_empty() {
host
} else {
webhook_base_url
};
let url = match self.action {
StackWebhookAction::Refresh => {
format!("{host}/listener/github/stack/{}/refresh", stack.id)
}

File diff suppressed because it is too large Load Diff

View File

@@ -1,17 +1,26 @@
use std::str::FromStr;
use anyhow::{anyhow, Context};
use anyhow::{Context, anyhow};
use komodo_client::{
api::write::{
CreateTag, DeleteTag, RenameTag, UpdateTagsOnResource,
UpdateTagsOnResourceResponse,
CreateTag, DeleteTag, RenameTag, UpdateTagColor,
UpdateTagsOnResource, UpdateTagsOnResourceResponse,
},
entities::{
alerter::Alerter, build::Build, builder::Builder,
deployment::Deployment, permission::PermissionLevel,
procedure::Procedure, repo::Repo, server::Server,
server_template::ServerTemplate, stack::Stack,
sync::ResourceSync, tag::Tag, user::User, ResourceTarget,
ResourceTarget,
action::Action,
alerter::Alerter,
build::Build,
builder::Builder,
deployment::Deployment,
permission::PermissionLevel,
procedure::Procedure,
repo::Repo,
server::Server,
server_template::ServerTemplate,
stack::Stack,
sync::ResourceSync,
tag::{Tag, TagColor},
},
};
use mungos::{
@@ -23,28 +32,29 @@ use resolver_api::Resolve;
use crate::{
helpers::query::{get_tag, get_tag_check_owner},
resource,
state::{db_client, State},
state::db_client,
};
impl Resolve<CreateTag, User> for State {
#[instrument(name = "CreateTag", skip(self, user))]
use super::WriteArgs;
impl Resolve<WriteArgs> for CreateTag {
#[instrument(name = "CreateTag", skip(user))]
async fn resolve(
&self,
CreateTag { name }: CreateTag,
user: User,
) -> anyhow::Result<Tag> {
if ObjectId::from_str(&name).is_ok() {
return Err(anyhow!("tag name cannot be ObjectId"));
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Tag> {
if ObjectId::from_str(&self.name).is_ok() {
return Err(anyhow!("tag name cannot be ObjectId").into());
}
let mut tag = Tag {
id: Default::default(),
name,
name: self.name,
color: TagColor::Slate,
owner: user.id.clone(),
};
tag.id = db_client()
.await
.tags
.insert_one(&tag)
.await
@@ -58,158 +68,191 @@ impl Resolve<CreateTag, User> for State {
}
}
impl Resolve<RenameTag, User> for State {
#[instrument(name = "RenameTag", skip(self, user))]
impl Resolve<WriteArgs> for RenameTag {
#[instrument(name = "RenameTag", skip(user))]
async fn resolve(
&self,
RenameTag { id, name }: RenameTag,
user: User,
) -> anyhow::Result<Tag> {
if ObjectId::from_str(&name).is_ok() {
return Err(anyhow!("tag name cannot be ObjectId"));
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Tag> {
if ObjectId::from_str(&self.name).is_ok() {
return Err(anyhow!("tag name cannot be ObjectId").into());
}
get_tag_check_owner(&id, &user).await?;
get_tag_check_owner(&self.id, user).await?;
update_one_by_id(
&db_client().await.tags,
&id,
doc! { "$set": { "name": name } },
&db_client().tags,
&self.id,
doc! { "$set": { "name": self.name } },
None,
)
.await
.context("failed to rename tag on db")?;
get_tag(&id).await
Ok(get_tag(&self.id).await?)
}
}
impl Resolve<DeleteTag, User> for State {
#[instrument(name = "DeleteTag", skip(self, user))]
impl Resolve<WriteArgs> for UpdateTagColor {
#[instrument(name = "UpdateTagColor", skip(user))]
async fn resolve(
&self,
DeleteTag { id }: DeleteTag,
user: User,
) -> anyhow::Result<Tag> {
let tag = get_tag_check_owner(&id, &user).await?;
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Tag> {
let tag = get_tag_check_owner(&self.tag, user).await?;
update_one_by_id(
&db_client().tags,
&tag.id,
doc! { "$set": { "color": self.color.as_ref() } },
None,
)
.await
.context("failed to rename tag on db")?;
Ok(get_tag(&self.tag).await?)
}
}
impl Resolve<WriteArgs> for DeleteTag {
#[instrument(name = "DeleteTag", skip(user))]
async fn resolve(
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<Tag> {
let tag = get_tag_check_owner(&self.id, user).await?;
tokio::try_join!(
resource::remove_tag_from_all::<Server>(&id),
resource::remove_tag_from_all::<Deployment>(&id),
resource::remove_tag_from_all::<Stack>(&id),
resource::remove_tag_from_all::<Build>(&id),
resource::remove_tag_from_all::<Repo>(&id),
resource::remove_tag_from_all::<Builder>(&id),
resource::remove_tag_from_all::<Alerter>(&id),
resource::remove_tag_from_all::<Procedure>(&id),
resource::remove_tag_from_all::<ServerTemplate>(&id),
resource::remove_tag_from_all::<Server>(&self.id),
resource::remove_tag_from_all::<Deployment>(&self.id),
resource::remove_tag_from_all::<Stack>(&self.id),
resource::remove_tag_from_all::<Build>(&self.id),
resource::remove_tag_from_all::<Repo>(&self.id),
resource::remove_tag_from_all::<Builder>(&self.id),
resource::remove_tag_from_all::<Alerter>(&self.id),
resource::remove_tag_from_all::<Procedure>(&self.id),
resource::remove_tag_from_all::<ServerTemplate>(&self.id),
)?;
delete_one_by_id(&db_client().await.tags, &id, None).await?;
delete_one_by_id(&db_client().tags, &self.id, None).await?;
Ok(tag)
}
}
impl Resolve<UpdateTagsOnResource, User> for State {
#[instrument(name = "UpdateTagsOnResource", skip(self, user))]
impl Resolve<WriteArgs> for UpdateTagsOnResource {
#[instrument(name = "UpdateTagsOnResource", skip(args))]
async fn resolve(
&self,
UpdateTagsOnResource { target, tags }: UpdateTagsOnResource,
user: User,
) -> anyhow::Result<UpdateTagsOnResourceResponse> {
match target {
ResourceTarget::System(_) => return Err(anyhow!("")),
self,
args: &WriteArgs,
) -> serror::Result<UpdateTagsOnResourceResponse> {
let WriteArgs { user } = args;
match self.target {
ResourceTarget::System(_) => {
return Err(anyhow!("Invalid target type: System").into());
}
ResourceTarget::Build(id) => {
resource::get_check_permissions::<Build>(
&id,
&user,
user,
PermissionLevel::Write,
)
.await?;
resource::update_tags::<Build>(&id, tags, user).await?;
resource::update_tags::<Build>(&id, self.tags, args).await?;
}
ResourceTarget::Builder(id) => {
resource::get_check_permissions::<Builder>(
&id,
&user,
user,
PermissionLevel::Write,
)
.await?;
resource::update_tags::<Builder>(&id, tags, user).await?
resource::update_tags::<Builder>(&id, self.tags, args).await?
}
ResourceTarget::Deployment(id) => {
resource::get_check_permissions::<Deployment>(
&id,
&user,
user,
PermissionLevel::Write,
)
.await?;
resource::update_tags::<Deployment>(&id, tags, user).await?
resource::update_tags::<Deployment>(&id, self.tags, args)
.await?
}
ResourceTarget::Server(id) => {
resource::get_check_permissions::<Server>(
&id,
&user,
user,
PermissionLevel::Write,
)
.await?;
resource::update_tags::<Server>(&id, tags, user).await?
resource::update_tags::<Server>(&id, self.tags, args).await?
}
ResourceTarget::Repo(id) => {
resource::get_check_permissions::<Repo>(
&id,
&user,
user,
PermissionLevel::Write,
)
.await?;
resource::update_tags::<Repo>(&id, tags, user).await?
resource::update_tags::<Repo>(&id, self.tags, args).await?
}
ResourceTarget::Alerter(id) => {
resource::get_check_permissions::<Alerter>(
&id,
&user,
user,
PermissionLevel::Write,
)
.await?;
resource::update_tags::<Alerter>(&id, tags, user).await?
resource::update_tags::<Alerter>(&id, self.tags, args).await?
}
ResourceTarget::Procedure(id) => {
resource::get_check_permissions::<Procedure>(
&id,
&user,
user,
PermissionLevel::Write,
)
.await?;
resource::update_tags::<Procedure>(&id, tags, user).await?
resource::update_tags::<Procedure>(&id, self.tags, args)
.await?
}
ResourceTarget::Action(id) => {
resource::get_check_permissions::<Action>(
&id,
user,
PermissionLevel::Write,
)
.await?;
resource::update_tags::<Action>(&id, self.tags, args).await?
}
ResourceTarget::ServerTemplate(id) => {
resource::get_check_permissions::<ServerTemplate>(
&id,
&user,
user,
PermissionLevel::Write,
)
.await?;
resource::update_tags::<ServerTemplate>(&id, tags, user)
resource::update_tags::<ServerTemplate>(&id, self.tags, args)
.await?
}
ResourceTarget::ResourceSync(id) => {
resource::get_check_permissions::<ResourceSync>(
&id,
&user,
user,
PermissionLevel::Write,
)
.await?;
resource::update_tags::<ResourceSync>(&id, tags, user).await?
resource::update_tags::<ResourceSync>(&id, self.tags, args)
.await?
}
ResourceTarget::Stack(id) => {
resource::get_check_permissions::<Stack>(
&id,
&user,
user,
PermissionLevel::Write,
)
.await?;
resource::update_tags::<Stack>(&id, tags, user).await?
resource::update_tags::<Stack>(&id, self.tags, args).await?
}
};
Ok(UpdateTagsOnResourceResponse {})

View File

@@ -0,0 +1,150 @@
use std::str::FromStr;
use anyhow::{Context, anyhow};
use komodo_client::{
api::write::{
DeleteUser, DeleteUserResponse, UpdateUserPassword,
UpdateUserPasswordResponse, UpdateUserUsername,
UpdateUserUsernameResponse,
},
entities::{NoData, user::UserConfig},
};
use mungos::mongodb::bson::{doc, oid::ObjectId};
use resolver_api::Resolve;
use crate::{
config::core_config, helpers::hash_password, state::db_client,
};
use super::WriteArgs;
//
impl Resolve<WriteArgs> for UpdateUserUsername {
async fn resolve(
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<UpdateUserUsernameResponse> {
for locked_username in &core_config().lock_login_credentials_for {
if locked_username == "__ALL__"
|| *locked_username == user.username
{
return Err(
anyhow!("User not allowed to update their username.")
.into(),
);
}
}
if self.username.is_empty() {
return Err(anyhow!("Username cannot be empty.").into());
}
let db = db_client();
if db
.users
.find_one(doc! { "username": &self.username })
.await
.context("Failed to query for existing users")?
.is_some()
{
return Err(anyhow!("Username already taken.").into());
}
let id = ObjectId::from_str(&user.id)
.context("User id not valid ObjectId.")?;
db.users
.update_one(
doc! { "_id": id },
doc! { "$set": { "username": self.username } },
)
.await
.context("Failed to update user username on database.")?;
Ok(NoData {})
}
}
//
impl Resolve<WriteArgs> for UpdateUserPassword {
async fn resolve(
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<UpdateUserPasswordResponse> {
for locked_username in &core_config().lock_login_credentials_for {
if locked_username == "__ALL__"
|| *locked_username == user.username
{
return Err(
anyhow!("User not allowed to update their password.")
.into(),
);
}
}
let UserConfig::Local { .. } = user.config else {
return Err(anyhow!("User is not local user").into());
};
if self.password.is_empty() {
return Err(anyhow!("Password cannot be empty.").into());
}
let id = ObjectId::from_str(&user.id)
.context("User id not valid ObjectId.")?;
let hashed_password = hash_password(self.password)?;
db_client()
.users
.update_one(
doc! { "_id": id },
doc! { "$set": {
"config.data.password": hashed_password
} },
)
.await
.context("Failed to update user password on database.")?;
Ok(NoData {})
}
}
//
impl Resolve<WriteArgs> for DeleteUser {
async fn resolve(
self,
WriteArgs { user: admin }: &WriteArgs,
) -> serror::Result<DeleteUserResponse> {
if !admin.admin {
return Err(anyhow!("Calling user is not admin.").into());
}
if admin.username == self.user || admin.id == self.user {
return Err(anyhow!("User cannot delete themselves.").into());
}
let query = if let Ok(id) = ObjectId::from_str(&self.user) {
doc! { "_id": id }
} else {
doc! { "username": self.user }
};
let db = db_client();
let Some(user) = db
.users
.find_one(query.clone())
.await
.context("Failed to query database for users.")?
else {
return Err(
anyhow!("No user found with given id / username").into(),
);
};
if user.super_admin {
return Err(
anyhow!("Cannot delete a super admin user.").into(),
);
}
if user.admin && !admin.super_admin {
return Err(
anyhow!("Only a Super Admin can delete an admin user.")
.into(),
);
}
db.users
.delete_one(query)
.await
.context("Failed to delete user from database")?;
Ok(user)
}
}

View File

@@ -1,12 +1,12 @@
use std::{collections::HashMap, str::FromStr};
use anyhow::{anyhow, Context};
use anyhow::{Context, anyhow};
use komodo_client::{
api::write::{
AddUserToUserGroup, CreateUserGroup, DeleteUserGroup,
RemoveUserFromUserGroup, RenameUserGroup, SetUsersInUserGroup,
},
entities::{komodo_timestamp, user::User, user_group::UserGroup},
entities::{komodo_timestamp, user_group::UserGroup},
};
use mungos::{
by_id::{delete_one_by_id, find_one_by_id, update_one_by_id},
@@ -15,25 +15,26 @@ use mungos::{
};
use resolver_api::Resolve;
use crate::state::{db_client, State};
use crate::state::db_client;
impl Resolve<CreateUserGroup, User> for State {
use super::WriteArgs;
impl Resolve<WriteArgs> for CreateUserGroup {
async fn resolve(
&self,
CreateUserGroup { name }: CreateUserGroup,
admin: User,
) -> anyhow::Result<UserGroup> {
self,
WriteArgs { user: admin }: &WriteArgs,
) -> serror::Result<UserGroup> {
if !admin.admin {
return Err(anyhow!("This call is admin-only"));
return Err(anyhow!("This call is admin-only").into());
}
let user_group = UserGroup {
id: Default::default(),
users: Default::default(),
all: Default::default(),
updated_at: komodo_timestamp(),
name,
name: self.name,
};
let db = db_client().await;
let db = db_client();
let id = db
.user_groups
.insert_one(user_group)
@@ -43,63 +44,63 @@ impl Resolve<CreateUserGroup, User> for State {
.as_object_id()
.context("inserted id is not ObjectId")?
.to_string();
find_one_by_id(&db.user_groups, &id)
let res = find_one_by_id(&db.user_groups, &id)
.await
.context("failed to query db for user groups")?
.context("user group at id not found")
.context("user group at id not found")?;
Ok(res)
}
}
impl Resolve<RenameUserGroup, User> for State {
impl Resolve<WriteArgs> for RenameUserGroup {
async fn resolve(
&self,
RenameUserGroup { id, name }: RenameUserGroup,
admin: User,
) -> anyhow::Result<UserGroup> {
self,
WriteArgs { user: admin }: &WriteArgs,
) -> serror::Result<UserGroup> {
if !admin.admin {
return Err(anyhow!("This call is admin-only"));
return Err(anyhow!("This call is admin-only").into());
}
let db = db_client().await;
let db = db_client();
update_one_by_id(
&db.user_groups,
&id,
doc! { "$set": { "name": name } },
&self.id,
doc! { "$set": { "name": self.name } },
None,
)
.await
.context("failed to rename UserGroup on db")?;
find_one_by_id(&db.user_groups, &id)
let res = find_one_by_id(&db.user_groups, &self.id)
.await
.context("failed to query db for UserGroups")?
.context("no user group with given id")
.context("no user group with given id")?;
Ok(res)
}
}
impl Resolve<DeleteUserGroup, User> for State {
impl Resolve<WriteArgs> for DeleteUserGroup {
async fn resolve(
&self,
DeleteUserGroup { id }: DeleteUserGroup,
admin: User,
) -> anyhow::Result<UserGroup> {
self,
WriteArgs { user: admin }: &WriteArgs,
) -> serror::Result<UserGroup> {
if !admin.admin {
return Err(anyhow!("This call is admin-only"));
return Err(anyhow!("This call is admin-only").into());
}
let db = db_client().await;
let db = db_client();
let ug = find_one_by_id(&db.user_groups, &id)
let ug = find_one_by_id(&db.user_groups, &self.id)
.await
.context("failed to query db for UserGroups")?
.context("no UserGroup found with given id")?;
delete_one_by_id(&db.user_groups, &id, None)
delete_one_by_id(&db.user_groups, &self.id, None)
.await
.context("failed to delete UserGroup from db")?;
db.permissions
.delete_many(doc! {
"user_target.type": "UserGroup",
"user_target.id": id,
"user_target.id": self.id,
})
.await
.context("failed to clean up UserGroups permissions. User Group has been deleted")?;
@@ -108,21 +109,20 @@ impl Resolve<DeleteUserGroup, User> for State {
}
}
impl Resolve<AddUserToUserGroup, User> for State {
impl Resolve<WriteArgs> for AddUserToUserGroup {
async fn resolve(
&self,
AddUserToUserGroup { user_group, user }: AddUserToUserGroup,
admin: User,
) -> anyhow::Result<UserGroup> {
self,
WriteArgs { user: admin }: &WriteArgs,
) -> serror::Result<UserGroup> {
if !admin.admin {
return Err(anyhow!("This call is admin-only"));
return Err(anyhow!("This call is admin-only").into());
}
let db = db_client().await;
let db = db_client();
let filter = match ObjectId::from_str(&user) {
let filter = match ObjectId::from_str(&self.user) {
Ok(id) => doc! { "_id": id },
Err(_) => doc! { "username": &user },
Err(_) => doc! { "username": &self.user },
};
let user = db
.users
@@ -131,9 +131,9 @@ impl Resolve<AddUserToUserGroup, User> for State {
.context("failed to query mongo for users")?
.context("no matching user found")?;
let filter = match ObjectId::from_str(&user_group) {
let filter = match ObjectId::from_str(&self.user_group) {
Ok(id) => doc! { "_id": id },
Err(_) => doc! { "name": &user_group },
Err(_) => doc! { "name": &self.user_group },
};
db.user_groups
.update_one(
@@ -142,32 +142,30 @@ impl Resolve<AddUserToUserGroup, User> for State {
)
.await
.context("failed to add user to group on db")?;
db.user_groups
let res = db
.user_groups
.find_one(filter)
.await
.context("failed to query db for UserGroups")?
.context("no user group with given id")
.context("no user group with given id")?;
Ok(res)
}
}
impl Resolve<RemoveUserFromUserGroup, User> for State {
impl Resolve<WriteArgs> for RemoveUserFromUserGroup {
async fn resolve(
&self,
RemoveUserFromUserGroup {
user_group,
user,
}: RemoveUserFromUserGroup,
admin: User,
) -> anyhow::Result<UserGroup> {
self,
WriteArgs { user: admin }: &WriteArgs,
) -> serror::Result<UserGroup> {
if !admin.admin {
return Err(anyhow!("This call is admin-only"));
return Err(anyhow!("This call is admin-only").into());
}
let db = db_client().await;
let db = db_client();
let filter = match ObjectId::from_str(&user) {
let filter = match ObjectId::from_str(&self.user) {
Ok(id) => doc! { "_id": id },
Err(_) => doc! { "username": &user },
Err(_) => doc! { "username": &self.user },
};
let user = db
.users
@@ -176,9 +174,9 @@ impl Resolve<RemoveUserFromUserGroup, User> for State {
.context("failed to query mongo for users")?
.context("no matching user found")?;
let filter = match ObjectId::from_str(&user_group) {
let filter = match ObjectId::from_str(&self.user_group) {
Ok(id) => doc! { "_id": id },
Err(_) => doc! { "name": &user_group },
Err(_) => doc! { "name": &self.user_group },
};
db.user_groups
.update_one(
@@ -187,25 +185,26 @@ impl Resolve<RemoveUserFromUserGroup, User> for State {
)
.await
.context("failed to add user to group on db")?;
db.user_groups
let res = db
.user_groups
.find_one(filter)
.await
.context("failed to query db for UserGroups")?
.context("no user group with given id")
.context("no user group with given id")?;
Ok(res)
}
}
impl Resolve<SetUsersInUserGroup, User> for State {
impl Resolve<WriteArgs> for SetUsersInUserGroup {
async fn resolve(
&self,
SetUsersInUserGroup { user_group, users }: SetUsersInUserGroup,
admin: User,
) -> anyhow::Result<UserGroup> {
self,
WriteArgs { user: admin }: &WriteArgs,
) -> serror::Result<UserGroup> {
if !admin.admin {
return Err(anyhow!("This call is admin-only"));
return Err(anyhow!("This call is admin-only").into());
}
let db = db_client().await;
let db = db_client();
let all_users = find_collect(&db.users, None, None)
.await
@@ -215,7 +214,8 @@ impl Resolve<SetUsersInUserGroup, User> for State {
.collect::<HashMap<_, _>>();
// Make sure all users are user ids
let users = users
let users = self
.users
.into_iter()
.filter_map(|user| match ObjectId::from_str(&user) {
Ok(_) => Some(user),
@@ -223,18 +223,20 @@ impl Resolve<SetUsersInUserGroup, User> for State {
})
.collect::<Vec<_>>();
let filter = match ObjectId::from_str(&user_group) {
let filter = match ObjectId::from_str(&self.user_group) {
Ok(id) => doc! { "_id": id },
Err(_) => doc! { "name": &user_group },
Err(_) => doc! { "name": &self.user_group },
};
db.user_groups
.update_one(filter.clone(), doc! { "$set": { "users": users } })
.await
.context("failed to set users on user group")?;
db.user_groups
let res = db
.user_groups
.find_one(filter)
.await
.context("failed to query db for UserGroups")?
.context("no user group with given id")
.context("no user group with given id")?;
Ok(res)
}
}

View File

@@ -1,15 +1,7 @@
use anyhow::{anyhow, Context};
use anyhow::{Context, anyhow};
use komodo_client::{
api::write::{
CreateVariable, CreateVariableResponse, DeleteVariable,
DeleteVariableResponse, UpdateVariableDescription,
UpdateVariableDescriptionResponse, UpdateVariableIsSecret,
UpdateVariableIsSecretResponse, UpdateVariableValue,
UpdateVariableValueResponse,
},
entities::{
user::User, variable::Variable, Operation, ResourceTarget,
},
api::write::*,
entities::{Operation, ResourceTarget, variable::Variable},
};
use mungos::mongodb::bson::doc;
use resolver_api::Resolve;
@@ -19,23 +11,26 @@ use crate::{
query::get_variable,
update::{add_update, make_update},
},
state::{db_client, State},
state::db_client,
};
impl Resolve<CreateVariable, User> for State {
#[instrument(name = "CreateVariable", skip(self, user, value))]
use super::WriteArgs;
impl Resolve<WriteArgs> for CreateVariable {
#[instrument(name = "CreateVariable", skip(user, self), fields(name = &self.name))]
async fn resolve(
&self,
CreateVariable {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<CreateVariableResponse> {
let CreateVariable {
name,
value,
description,
is_secret,
}: CreateVariable,
user: User,
) -> anyhow::Result<CreateVariableResponse> {
} = self;
if !user.admin {
return Err(anyhow!("only admins can create variables"));
return Err(anyhow!("only admins can create variables").into());
}
let variable = Variable {
@@ -46,7 +41,6 @@ impl Resolve<CreateVariable, User> for State {
};
db_client()
.await
.variables
.insert_one(&variable)
.await
@@ -55,7 +49,7 @@ impl Resolve<CreateVariable, User> for State {
let mut update = make_update(
ResourceTarget::system(),
Operation::CreateVariable,
&user,
user,
);
update
@@ -64,29 +58,29 @@ impl Resolve<CreateVariable, User> for State {
add_update(update).await?;
get_variable(&variable.name).await
Ok(get_variable(&variable.name).await?)
}
}
impl Resolve<UpdateVariableValue, User> for State {
#[instrument(name = "UpdateVariableValue", skip(self, user, value))]
impl Resolve<WriteArgs> for UpdateVariableValue {
#[instrument(name = "UpdateVariableValue", skip(user, self), fields(name = &self.name))]
async fn resolve(
&self,
UpdateVariableValue { name, value }: UpdateVariableValue,
user: User,
) -> anyhow::Result<UpdateVariableValueResponse> {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<UpdateVariableValueResponse> {
if !user.admin {
return Err(anyhow!("only admins can update variables"));
return Err(anyhow!("only admins can update variables").into());
}
let UpdateVariableValue { name, value } = self;
let variable = get_variable(&name).await?;
if value == variable.value {
return Err(anyhow!("no change"));
return Ok(variable);
}
db_client()
.await
.variables
.update_one(
doc! { "name": &name },
@@ -98,7 +92,7 @@ impl Resolve<UpdateVariableValue, User> for State {
let mut update = make_update(
ResourceTarget::system(),
Operation::UpdateVariableValue,
&user,
user,
);
let log = if variable.is_secret {
@@ -118,77 +112,71 @@ impl Resolve<UpdateVariableValue, User> for State {
add_update(update).await?;
get_variable(&name).await
Ok(get_variable(&name).await?)
}
}
impl Resolve<UpdateVariableDescription, User> for State {
#[instrument(name = "UpdateVariableDescription", skip(self, user))]
impl Resolve<WriteArgs> for UpdateVariableDescription {
#[instrument(name = "UpdateVariableDescription", skip(user))]
async fn resolve(
&self,
UpdateVariableDescription { name, description }: UpdateVariableDescription,
user: User,
) -> anyhow::Result<UpdateVariableDescriptionResponse> {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<UpdateVariableDescriptionResponse> {
if !user.admin {
return Err(anyhow!("only admins can update variables"));
return Err(anyhow!("only admins can update variables").into());
}
db_client()
.await
.variables
.update_one(
doc! { "name": &name },
doc! { "$set": { "description": &description } },
doc! { "name": &self.name },
doc! { "$set": { "description": &self.description } },
)
.await
.context("failed to update variable description on db")?;
get_variable(&name).await
Ok(get_variable(&self.name).await?)
}
}
impl Resolve<UpdateVariableIsSecret, User> for State {
#[instrument(name = "UpdateVariableIsSecret", skip(self, user))]
impl Resolve<WriteArgs> for UpdateVariableIsSecret {
#[instrument(name = "UpdateVariableIsSecret", skip(user))]
async fn resolve(
&self,
UpdateVariableIsSecret { name, is_secret }: UpdateVariableIsSecret,
user: User,
) -> anyhow::Result<UpdateVariableIsSecretResponse> {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<UpdateVariableIsSecretResponse> {
if !user.admin {
return Err(anyhow!("only admins can update variables"));
return Err(anyhow!("only admins can update variables").into());
}
db_client()
.await
.variables
.update_one(
doc! { "name": &name },
doc! { "$set": { "is_secret": is_secret } },
doc! { "name": &self.name },
doc! { "$set": { "is_secret": self.is_secret } },
)
.await
.context("failed to update variable is secret on db")?;
get_variable(&name).await
Ok(get_variable(&self.name).await?)
}
}
impl Resolve<DeleteVariable, User> for State {
impl Resolve<WriteArgs> for DeleteVariable {
async fn resolve(
&self,
DeleteVariable { name }: DeleteVariable,
user: User,
) -> anyhow::Result<DeleteVariableResponse> {
self,
WriteArgs { user }: &WriteArgs,
) -> serror::Result<DeleteVariableResponse> {
if !user.admin {
return Err(anyhow!("only admins can delete variables"));
return Err(anyhow!("only admins can delete variables").into());
}
let variable = get_variable(&name).await?;
let variable = get_variable(&self.name).await?;
db_client()
.await
.variables
.delete_one(doc! { "name": &name })
.delete_one(doc! { "name": &self.name })
.await
.context("failed to delete variable on db")?;
let mut update = make_update(
ResourceTarget::system(),
Operation::DeleteVariable,
&user,
user,
);
update

View File

@@ -1,11 +1,11 @@
use std::sync::OnceLock;
use anyhow::{anyhow, Context};
use anyhow::{Context, anyhow};
use komodo_client::entities::config::core::{
CoreConfig, OauthCredentials,
};
use reqwest::StatusCode;
use serde::{de::DeserializeOwned, Deserialize, Serialize};
use serde::{Deserialize, Serialize, de::DeserializeOwned};
use tokio::sync::Mutex;
use crate::{
@@ -47,15 +47,21 @@ impl GithubOauthClient {
return None;
}
if host.is_empty() {
warn!("github oauth is enabled, but 'config.host' is not configured");
warn!(
"github oauth is enabled, but 'config.host' is not configured"
);
return None;
}
if id.is_empty() {
warn!("github oauth is enabled, but 'config.github_oauth.id' is not configured");
warn!(
"github oauth is enabled, but 'config.github_oauth.id' is not configured"
);
return None;
}
if secret.is_empty() {
warn!("github oauth is enabled, but 'config.github_oauth.secret' is not configured");
warn!(
"github oauth is enabled, but 'config.github_oauth.secret' is not configured"
);
return None;
}
GithubOauthClient {

View File

@@ -1,6 +1,6 @@
use anyhow::{anyhow, Context};
use anyhow::{Context, anyhow};
use axum::{
extract::Query, response::Redirect, routing::get, Router,
Router, extract::Query, response::Redirect, routing::get,
};
use komodo_client::entities::{
komodo_timestamp,
@@ -64,15 +64,15 @@ async fn callback(
let github_user =
client.get_github_user(&token.access_token).await?;
let github_id = github_user.id.to_string();
let db_client = db_client().await;
let db_client = db_client();
let user = db_client
.users
.find_one(doc! { "config.data.github_id": &github_id })
.await
.context("failed at find user query from mongo")?;
.context("failed at find user query from database")?;
let jwt = match user {
Some(user) => jwt_client()
.generate(user.id)
.encode(user.id)
.context("failed to generate jwt")?,
None => {
let ts = komodo_timestamp();
@@ -87,6 +87,7 @@ async fn callback(
username: github_user.login,
enabled: no_users_exist || core_config.enable_new_users,
admin: no_users_exist,
super_admin: no_users_exist,
create_server_permissions: no_users_exist,
create_build_permissions: no_users_exist,
updated_at: ts,
@@ -108,7 +109,7 @@ async fn callback(
.context("inserted_id is not ObjectId")?
.to_string();
jwt_client()
.generate(user_id)
.encode(user_id)
.context("failed to generate jwt")?
}
};

View File

@@ -1,13 +1,12 @@
use std::sync::OnceLock;
use anyhow::{anyhow, Context};
use jwt::Token;
use anyhow::{Context, anyhow};
use jsonwebtoken::{DecodingKey, Validation, decode};
use komodo_client::entities::config::core::{
CoreConfig, OauthCredentials,
};
use reqwest::StatusCode;
use serde::{de::DeserializeOwned, Deserialize};
use serde_json::Value;
use serde::{Deserialize, de::DeserializeOwned};
use tokio::sync::Mutex;
use crate::{
@@ -49,15 +48,21 @@ impl GoogleOauthClient {
return None;
}
if host.is_empty() {
warn!("google oauth is enabled, but 'config.host' is not configured");
warn!(
"google oauth is enabled, but 'config.host' is not configured"
);
return None;
}
if id.is_empty() {
warn!("google oauth is enabled, but 'config.google_oauth.id' is not configured");
warn!(
"google oauth is enabled, but 'config.google_oauth.id' is not configured"
);
return None;
}
if secret.is_empty() {
warn!("google oauth is enabled, but 'config.google_oauth.secret' is not configured");
warn!(
"google oauth is enabled, but 'config.google_oauth.secret' is not configured"
);
return None;
}
let scopes = urlencoding::encode(
@@ -139,10 +144,16 @@ impl GoogleOauthClient {
&self,
id_token: &str,
) -> anyhow::Result<GoogleUser> {
let t: Token<Value, GoogleUser, jwt::Unverified> =
Token::parse_unverified(id_token)
.context("failed to parse id_token")?;
Ok(t.claims().to_owned())
let mut v = Validation::new(Default::default());
v.insecure_disable_signature_validation();
v.validate_aud = false;
let res = decode::<GoogleUser>(
id_token,
&DecodingKey::from_secret(b""),
&v,
)
.context("failed to decode google id token")?;
Ok(res.claims)
}
#[instrument(level = "debug", skip(self))]

View File

@@ -1,7 +1,7 @@
use anyhow::{anyhow, Context};
use anyhow::{Context, anyhow};
use async_timing_util::unix_timestamp_ms;
use axum::{
extract::Query, response::Redirect, routing::get, Router,
Router, extract::Query, response::Redirect, routing::get,
};
use komodo_client::entities::user::{User, UserConfig};
use mongo_indexed::Document;
@@ -73,7 +73,7 @@ async fn callback(
.await?;
let google_user = client.get_google_user(&token.id_token)?;
let google_id = google_user.id.to_string();
let db_client = db_client().await;
let db_client = db_client();
let user = db_client
.users
.find_one(doc! { "config.data.google_id": &google_id })
@@ -81,7 +81,7 @@ async fn callback(
.context("failed at find user query from mongo")?;
let jwt = match user {
Some(user) => jwt_client()
.generate(user.id)
.encode(user.id)
.context("failed to generate jwt")?,
None => {
let ts = unix_timestamp_ms() as i64;
@@ -102,6 +102,7 @@ async fn callback(
.to_string(),
enabled: no_users_exist || core_config.enable_new_users,
admin: no_users_exist,
super_admin: no_users_exist,
create_server_permissions: no_users_exist,
create_build_permissions: no_users_exist,
updated_at: ts,
@@ -123,7 +124,7 @@ async fn callback(
.context("inserted_id is not ObjectId")?
.to_string();
jwt_client()
.generate(user_id)
.encode(user_id)
.context("failed to generate jwt")?
}
};

View File

@@ -1,15 +1,15 @@
use std::collections::HashMap;
use anyhow::{anyhow, Context};
use anyhow::{Context, anyhow};
use async_timing_util::{
get_timelength_in_ms, unix_timestamp_ms, Timelength,
Timelength, get_timelength_in_ms, unix_timestamp_ms,
};
use jsonwebtoken::{
DecodingKey, EncodingKey, Header, Validation, decode, encode,
};
use hmac::{Hmac, Mac};
use jwt::SignWithKey;
use komodo_client::entities::config::core::CoreConfig;
use mungos::mongodb::bson::doc;
use serde::{Deserialize, Serialize};
use sha2::Sha256;
use tokio::sync::Mutex;
use crate::helpers::random_string;
@@ -24,7 +24,10 @@ pub struct JwtClaims {
}
pub struct JwtClient {
pub key: Hmac<Sha256>,
header: Header,
validation: Validation,
encoding_key: EncodingKey,
decoding_key: DecodingKey,
ttl_ms: u128,
exchange_tokens: ExchangeTokenMap,
}
@@ -36,10 +39,11 @@ impl JwtClient {
} else {
config.jwt_secret.clone()
};
let key = Hmac::new_from_slice(secret.as_bytes())
.context("failed at taking HmacSha256 of jwt secret")?;
Ok(JwtClient {
key,
header: Header::default(),
validation: Validation::new(Default::default()),
encoding_key: EncodingKey::from_secret(secret.as_bytes()),
decoding_key: DecodingKey::from_secret(secret.as_bytes()),
ttl_ms: get_timelength_in_ms(
config.jwt_ttl.to_string().parse()?,
),
@@ -47,7 +51,7 @@ impl JwtClient {
})
}
pub fn generate(&self, user_id: String) -> anyhow::Result<String> {
pub fn encode(&self, user_id: String) -> anyhow::Result<String> {
let iat = unix_timestamp_ms();
let exp = iat + self.ttl_ms;
let claims = JwtClaims {
@@ -55,10 +59,14 @@ impl JwtClient {
iat,
exp,
};
let jwt = claims
.sign_with_key(&self.key)
.context("failed at signing claim")?;
Ok(jwt)
encode(&self.header, &claims, &self.encoding_key)
.context("failed at signing claim")
}
pub fn decode(&self, jwt: &str) -> anyhow::Result<JwtClaims> {
decode::<JwtClaims>(jwt, &self.decoding_key, &self.validation)
.map(|res| res.claims)
.context("failed to decode token claims")
}
#[instrument(level = "debug", skip_all)]

View File

@@ -1,9 +1,7 @@
use std::str::FromStr;
use anyhow::{anyhow, Context};
use anyhow::{Context, anyhow};
use async_timing_util::unix_timestamp_ms;
use axum::http::HeaderMap;
use mongo_indexed::Document;
use komodo_client::{
api::auth::{
CreateLocalUser, CreateLocalUserResponse, LoginLocalUser,
@@ -11,74 +9,72 @@ use komodo_client::{
},
entities::user::{User, UserConfig},
};
use mongo_indexed::Document;
use mungos::mongodb::bson::{doc, oid::ObjectId};
use resolver_api::Resolve;
use crate::{
api::auth::AuthArgs,
config::core_config,
state::State,
helpers::hash_password,
state::{db_client, jwt_client},
};
const BCRYPT_COST: u32 = 10;
impl Resolve<CreateLocalUser, HeaderMap> for State {
impl Resolve<AuthArgs> for CreateLocalUser {
#[instrument(name = "CreateLocalUser", skip(self))]
async fn resolve(
&self,
CreateLocalUser { username, password }: CreateLocalUser,
_: HeaderMap,
) -> anyhow::Result<CreateLocalUserResponse> {
self,
_: &AuthArgs,
) -> serror::Result<CreateLocalUserResponse> {
let core_config = core_config();
if !core_config.local_auth {
return Err(anyhow!("Local auth is not enabled"));
return Err(anyhow!("Local auth is not enabled").into());
}
if username.is_empty() {
return Err(anyhow!("Username cannot be empty string"));
if self.username.is_empty() {
return Err(anyhow!("Username cannot be empty string").into());
}
if ObjectId::from_str(&username).is_ok() {
return Err(anyhow!("Username cannot be valid ObjectId"));
if ObjectId::from_str(&self.username).is_ok() {
return Err(
anyhow!("Username cannot be valid ObjectId").into(),
);
}
if password.is_empty() {
return Err(anyhow!("Password cannot be empty string"));
if self.password.is_empty() {
return Err(anyhow!("Password cannot be empty string").into());
}
let password = bcrypt::hash(password, BCRYPT_COST)
.context("failed to hash password")?;
let hashed_password = hash_password(self.password)?;
let no_users_exist = db_client()
.await
.users
.find_one(Document::new())
.await?
.is_none();
let no_users_exist =
db_client().users.find_one(Document::new()).await?.is_none();
if !no_users_exist && core_config.disable_user_registration {
return Err(anyhow!("User registration is disabled"));
return Err(anyhow!("User registration is disabled").into());
}
let ts = unix_timestamp_ms() as i64;
let user = User {
id: Default::default(),
username,
username: self.username,
enabled: no_users_exist || core_config.enable_new_users,
admin: no_users_exist,
super_admin: no_users_exist,
create_server_permissions: no_users_exist,
create_build_permissions: no_users_exist,
updated_at: ts,
last_update_view: 0,
recents: Default::default(),
all: Default::default(),
config: UserConfig::Local { password },
config: UserConfig::Local {
password: hashed_password,
},
};
let user_id = db_client()
.await
.users
.insert_one(user)
.await
@@ -89,52 +85,53 @@ impl Resolve<CreateLocalUser, HeaderMap> for State {
.to_string();
let jwt = jwt_client()
.generate(user_id)
.encode(user_id)
.context("failed to generate jwt for user")?;
Ok(CreateLocalUserResponse { jwt })
}
}
impl Resolve<LoginLocalUser, HeaderMap> for State {
impl Resolve<AuthArgs> for LoginLocalUser {
#[instrument(name = "LoginLocalUser", level = "debug", skip(self))]
async fn resolve(
&self,
LoginLocalUser { username, password }: LoginLocalUser,
_: HeaderMap,
) -> anyhow::Result<LoginLocalUserResponse> {
self,
_: &AuthArgs,
) -> serror::Result<LoginLocalUserResponse> {
if !core_config().local_auth {
return Err(anyhow!("local auth is not enabled"));
return Err(anyhow!("local auth is not enabled").into());
}
let user = db_client()
.await
.users
.find_one(doc! { "username": &username })
.find_one(doc! { "username": &self.username })
.await
.context("failed at db query for users")?
.with_context(|| {
format!("did not find user with username {username}")
format!("did not find user with username {}", self.username)
})?;
let UserConfig::Local {
password: user_pw_hash,
} = user.config
else {
return Err(anyhow!(
"non-local auth users can not log in with a password"
));
return Err(
anyhow!(
"non-local auth users can not log in with a password"
)
.into(),
);
};
let verified = bcrypt::verify(password, &user_pw_hash)
let verified = bcrypt::verify(self.password, &user_pw_hash)
.context("failed at verify password")?;
if !verified {
return Err(anyhow!("invalid credentials"));
return Err(anyhow!("invalid credentials").into());
}
let jwt = jwt_client()
.generate(user.id)
.encode(user.id)
.context("failed at generating jwt for user")?;
Ok(LoginLocalUserResponse { jwt })

View File

@@ -1,5 +1,4 @@
use ::jwt::VerifyWithKey;
use anyhow::{anyhow, Context};
use anyhow::{Context, anyhow};
use async_timing_util::unix_timestamp_ms;
use axum::{
extract::Request, http::HeaderMap, middleware::Next,
@@ -21,14 +20,15 @@ use self::jwt::JwtClaims;
pub mod github;
pub mod google;
pub mod jwt;
pub mod oidc;
mod local;
const STATE_PREFIX_LENGTH: usize = 20;
#[derive(Deserialize)]
pub struct RedirectQuery {
pub redirect: Option<String>,
#[derive(Debug, Deserialize)]
struct RedirectQuery {
redirect: Option<String>,
}
#[instrument(level = "debug")]
@@ -70,7 +70,9 @@ pub async fn get_user_id_from_headers(
}
_ => {
// AUTH FAIL
Err(anyhow!("must attach either AUTHORIZATION header with jwt OR pass X-API-KEY and X-API-SECRET"))
Err(anyhow!(
"must attach either AUTHORIZATION header with jwt OR pass X-API-KEY and X-API-SECRET"
))
}
}
}
@@ -92,9 +94,7 @@ pub async fn authenticate_check_enabled(
pub async fn auth_jwt_get_user_id(
jwt: &str,
) -> anyhow::Result<String> {
let claims: JwtClaims = jwt
.verify_with_key(&jwt_client().key)
.context("failed to verify claims")?;
let claims: JwtClaims = jwt_client().decode(jwt)?;
if claims.exp > unix_timestamp_ms() {
Ok(claims.id)
} else {
@@ -116,7 +116,6 @@ pub async fn auth_api_key_get_user_id(
secret: &str,
) -> anyhow::Result<String> {
let key = db_client()
.await
.api_keys
.find_one(doc! { "key": key })
.await

View File

@@ -0,0 +1,94 @@
use std::{sync::OnceLock, time::Duration};
use anyhow::Context;
use arc_swap::ArcSwapOption;
use openidconnect::{
Client, ClientId, ClientSecret, EmptyAdditionalClaims,
EndpointMaybeSet, EndpointNotSet, EndpointSet, IssuerUrl,
RedirectUrl, StandardErrorResponse, core::*,
};
use crate::config::core_config;
type OidcClient = Client<
EmptyAdditionalClaims,
CoreAuthDisplay,
CoreGenderClaim,
CoreJweContentEncryptionAlgorithm,
CoreJsonWebKey,
CoreAuthPrompt,
StandardErrorResponse<CoreErrorResponseType>,
CoreTokenResponse,
CoreTokenIntrospectionResponse,
CoreRevocableToken,
CoreRevocationErrorResponse,
EndpointSet,
EndpointNotSet,
EndpointNotSet,
EndpointNotSet,
EndpointMaybeSet,
EndpointMaybeSet,
>;
pub fn oidc_client() -> &'static ArcSwapOption<OidcClient> {
static OIDC_CLIENT: OnceLock<ArcSwapOption<OidcClient>> =
OnceLock::new();
OIDC_CLIENT.get_or_init(Default::default)
}
/// The OIDC client must be reinitialized to
/// pick up the latest provider JWKs. This
/// function spawns a management thread to do this
/// on a loop.
pub async fn spawn_oidc_client_management() {
let config = core_config();
if !config.oidc_enabled
|| config.oidc_provider.is_empty()
|| config.oidc_client_id.is_empty()
{
return;
}
reset_oidc_client()
.await
.context("Failed to initialize OIDC client.")
.unwrap();
tokio::spawn(async move {
loop {
tokio::time::sleep(Duration::from_secs(60)).await;
if let Err(e) = reset_oidc_client().await {
warn!("Failed to reinitialize OIDC client | {e:#}");
}
}
});
}
async fn reset_oidc_client() -> anyhow::Result<()> {
let config = core_config();
// Use OpenID Connect Discovery to fetch the provider metadata.
let provider_metadata = CoreProviderMetadata::discover_async(
IssuerUrl::new(config.oidc_provider.clone())?,
super::reqwest_client(),
)
.await
.context("Failed to get OIDC /.well-known/openid-configuration")?;
let client = CoreClient::from_provider_metadata(
provider_metadata,
ClientId::new(config.oidc_client_id.to_string()),
// The secret may be empty / ommitted if auth provider supports PKCE
if config.oidc_client_secret.is_empty() {
None
} else {
Some(ClientSecret::new(config.oidc_client_secret.to_string()))
},
)
// Set the URL the user will be redirected to after the authorization process.
.set_redirect_uri(RedirectUrl::new(format!(
"{}/auth/oidc/callback",
core_config().host
))?);
oidc_client().store(Some(client.into()));
Ok(())
}

View File

@@ -0,0 +1,287 @@
use std::sync::OnceLock;
use anyhow::{Context, anyhow};
use axum::{
Router, extract::Query, response::Redirect, routing::get,
};
use client::oidc_client;
use dashmap::DashMap;
use komodo_client::entities::{
komodo_timestamp,
user::{User, UserConfig},
};
use mungos::mongodb::bson::{Document, doc};
use openidconnect::{
AccessTokenHash, AuthorizationCode, CsrfToken, Nonce,
OAuth2TokenResponse, PkceCodeChallenge, PkceCodeVerifier, Scope,
TokenResponse, core::CoreAuthenticationFlow,
};
use reqwest::StatusCode;
use serde::Deserialize;
use serror::AddStatusCode;
use crate::{
config::core_config,
state::{db_client, jwt_client},
};
use super::RedirectQuery;
pub mod client;
fn reqwest_client() -> &'static reqwest::Client {
static REQWEST: OnceLock<reqwest::Client> = OnceLock::new();
REQWEST.get_or_init(|| {
reqwest::Client::builder()
.redirect(reqwest::redirect::Policy::none())
.build()
.expect("Invalid OIDC reqwest client")
})
}
/// CSRF tokens can only be used once from the callback,
/// and must be used within this timeframe
const CSRF_VALID_FOR_MS: i64 = 120_000; // 2 minutes for user to log in.
type RedirectUrl = Option<String>;
/// Maps the csrf secrets to other information added in the "login" method (before auth provider redirect).
/// This information is retrieved in the "callback" method (after auth provider redirect).
type VerifierMap =
DashMap<String, (PkceCodeVerifier, Nonce, RedirectUrl, i64)>;
fn verifier_tokens() -> &'static VerifierMap {
static VERIFIERS: OnceLock<VerifierMap> = OnceLock::new();
VERIFIERS.get_or_init(Default::default)
}
pub fn router() -> Router {
Router::new()
.route(
"/login",
get(|query| async {
login(query).await.status_code(StatusCode::UNAUTHORIZED)
}),
)
.route(
"/callback",
get(|query| async {
callback(query).await.status_code(StatusCode::UNAUTHORIZED)
}),
)
}
#[instrument(name = "OidcRedirect", level = "debug")]
async fn login(
Query(RedirectQuery { redirect }): Query<RedirectQuery>,
) -> anyhow::Result<Redirect> {
let client = oidc_client().load();
let client =
client.as_ref().context("OIDC Client not configured")?;
let (pkce_challenge, pkce_verifier) =
PkceCodeChallenge::new_random_sha256();
// Generate the authorization URL.
let (auth_url, csrf_token, nonce) = client
.authorize_url(
CoreAuthenticationFlow::AuthorizationCode,
CsrfToken::new_random,
Nonce::new_random,
)
.set_pkce_challenge(pkce_challenge)
.add_scope(Scope::new("openid".to_string()))
.add_scope(Scope::new("email".to_string()))
.url();
// Data inserted here will be matched on callback side for csrf protection.
verifier_tokens().insert(
csrf_token.secret().clone(),
(
pkce_verifier,
nonce,
redirect,
komodo_timestamp() + CSRF_VALID_FOR_MS,
),
);
let config = core_config();
let redirect = if !config.oidc_redirect_host.is_empty() {
let auth_url = auth_url.as_str();
let (protocol, rest) = auth_url
.split_once("://")
.context("Invalid URL: Missing protocol (eg 'https://')")?;
let host = rest
.split_once(['/', '?'])
.map(|(host, _)| host)
.unwrap_or(rest);
Redirect::to(&auth_url.replace(
&format!("{protocol}://{host}"),
&config.oidc_redirect_host,
))
} else {
Redirect::to(auth_url.as_str())
};
Ok(redirect)
}
#[derive(Debug, Deserialize)]
struct CallbackQuery {
state: Option<String>,
code: Option<String>,
error: Option<String>,
}
#[instrument(name = "OidcCallback", level = "debug")]
async fn callback(
Query(query): Query<CallbackQuery>,
) -> anyhow::Result<Redirect> {
let client = oidc_client().load();
let client =
client.as_ref().context("OIDC Client not configured")?;
if let Some(e) = query.error {
return Err(anyhow!("Provider returned error: {e}"));
}
let code = query.code.context("Provider did not return code")?;
let state = CsrfToken::new(
query.state.context("Provider did not return state")?,
);
let (_, (pkce_verifier, nonce, redirect, valid_until)) =
verifier_tokens()
.remove(state.secret())
.context("CSRF token invalid")?;
if komodo_timestamp() > valid_until {
return Err(anyhow!(
"CSRF token invalid (Timed out). The token must be used within 2 minutes."
));
}
let token_response = client
.exchange_code(AuthorizationCode::new(code))
.context("Failed to get Oauth token at exchange code")?
.set_pkce_verifier(pkce_verifier)
.request_async(reqwest_client())
.await
.context("Failed to get Oauth token")?;
// Extract the ID token claims after verifying its authenticity and nonce.
let id_token = token_response
.id_token()
.context("OIDC Server did not return an ID token")?;
// Some providers attach additional audiences, they must be added here
// so token verification succeeds.
let verifier = client.id_token_verifier();
let additional_audiences = &core_config().oidc_additional_audiences;
let verifier = if additional_audiences.is_empty() {
verifier
} else {
verifier.set_other_audience_verifier_fn(|aud| {
additional_audiences.contains(aud)
})
};
let claims = id_token
.claims(&verifier, &nonce)
.context("Failed to verify token claims. This issue may be temporary (60 seconds max).")?;
// Verify the access token hash to ensure that the access token hasn't been substituted for
// another user's.
if let Some(expected_access_token_hash) = claims.access_token_hash()
{
let actual_access_token_hash = AccessTokenHash::from_token(
token_response.access_token(),
id_token.signing_alg()?,
id_token.signing_key(&verifier)?,
)?;
if actual_access_token_hash != *expected_access_token_hash {
return Err(anyhow!("Invalid access token"));
}
}
let user_id = claims.subject().as_str();
let db_client = db_client();
let user = db_client
.users
.find_one(doc! {
"config.data.provider": &core_config().oidc_provider,
"config.data.user_id": user_id
})
.await
.context("failed at find user query from database")?;
let jwt = match user {
Some(user) => jwt_client()
.encode(user.id)
.context("failed to generate jwt")?,
None => {
let ts = komodo_timestamp();
let no_users_exist =
db_client.users.find_one(Document::new()).await?.is_none();
let core_config = core_config();
if !no_users_exist && core_config.disable_user_registration {
return Err(anyhow!("User registration is disabled"));
}
// Will use preferred_username, then email, then user_id if it isn't available.
let username = claims
.preferred_username()
.map(|username| username.to_string())
.unwrap_or_else(|| {
let email = claims
.email()
.map(|email| email.as_str())
.unwrap_or(user_id);
if core_config.oidc_use_full_email {
email
} else {
email
.split_once('@')
.map(|(username, _)| username)
.unwrap_or(email)
}
.to_string()
});
let user = User {
id: Default::default(),
username,
enabled: no_users_exist || core_config.enable_new_users,
admin: no_users_exist,
super_admin: no_users_exist,
create_server_permissions: no_users_exist,
create_build_permissions: no_users_exist,
updated_at: ts,
last_update_view: 0,
recents: Default::default(),
all: Default::default(),
config: UserConfig::Oidc {
provider: core_config.oidc_provider.clone(),
user_id: user_id.to_string(),
},
};
let user_id = db_client
.users
.insert_one(user)
.await
.context("failed to create user on database")?
.inserted_id
.as_object_id()
.context("inserted_id is not ObjectId")?
.to_string();
jwt_client()
.encode(user_id)
.context("failed to generate jwt")?
}
};
let exchange_token = jwt_client().create_exchange_token(jwt).await;
let redirect_url = if let Some(redirect) = redirect {
let splitter = if redirect.contains('?') { '&' } else { '?' };
format!("{}{splitter}token={exchange_token}", redirect)
} else {
format!("{}?token={exchange_token}", core_config().host)
};
Ok(Redirect::to(&redirect_url))
}

View File

@@ -1,25 +1,25 @@
use std::{str::FromStr, time::Duration};
use anyhow::{anyhow, Context};
use anyhow::{Context, anyhow};
use aws_config::{BehaviorVersion, Region};
use aws_sdk_ec2::{
Client,
types::{
BlockDeviceMapping, EbsBlockDevice,
InstanceNetworkInterfaceSpecification, InstanceStateChange,
InstanceStateName, InstanceStatus, InstanceType, ResourceType,
Tag, TagSpecification, VolumeType,
},
Client,
};
use base64::Engine;
use komodo_client::entities::{
ResourceTarget,
alert::{Alert, AlertData, SeverityLevel},
komodo_timestamp,
server_template::aws::AwsServerTemplateConfig,
ResourceTarget,
};
use crate::{config::core_config, helpers::alert::send_alerts};
use crate::{alert::send_alerts, config::core_config};
const POLL_RATE_SECS: u64 = 2;
const MAX_POLL_TRIES: usize = 30;
@@ -29,20 +29,40 @@ pub struct Ec2Instance {
pub ip: String,
}
/// Provides credentials in the core config file to the AWS client
#[derive(Debug)]
struct CredentialsFromConfig;
impl aws_credential_types::provider::ProvideCredentials
for CredentialsFromConfig
{
fn provide_credentials<'a>(
&'a self,
) -> aws_credential_types::provider::future::ProvideCredentials<'a>
where
Self: 'a,
{
aws_credential_types::provider::future::ProvideCredentials::new(
async {
let config = core_config();
Ok(aws_credential_types::Credentials::new(
&config.aws.access_key_id,
&config.aws.secret_access_key,
None,
None,
"komodo-config",
))
},
)
}
}
#[instrument]
async fn create_ec2_client(region: String) -> Client {
// There may be a better way to pass these keys to client
std::env::set_var(
"AWS_ACCESS_KEY_ID",
&core_config().aws.access_key_id,
);
std::env::set_var(
"AWS_SECRET_ACCESS_KEY",
&core_config().aws.secret_access_key,
);
let region = Region::new(region);
let config = aws_config::defaults(BehaviorVersion::v2024_03_28())
let config = aws_config::defaults(BehaviorVersion::latest())
.region(region)
.credentials_provider(CredentialsFromConfig)
.load()
.await;
Client::new(&config)
@@ -65,6 +85,7 @@ pub async fn launch_ec2_instance(
use_public_ip,
user_data,
port: _,
use_https: _,
} = config;
let instance_type = handle_unknown_instance_type(
InstanceType::from(instance_type.as_str()),
@@ -211,21 +232,37 @@ async fn terminate_ec2_instance_inner(
Ok(res)
}
/// Automatically retries 5 times, waiting 2 sec in between
#[instrument(level = "debug")]
async fn get_ec2_instance_status(
client: &Client,
instance_id: &str,
) -> anyhow::Result<Option<InstanceStatus>> {
let status = client
.describe_instance_status()
.instance_ids(instance_id)
.send()
let mut try_count = 1;
loop {
match async {
anyhow::Ok(
client
.describe_instance_status()
.instance_ids(instance_id)
.send()
.await
.context("failed to describe instance status from aws")?
.instance_statuses()
.first()
.cloned(),
)
}
.await
.context("failed to get instance status from aws")?
.instance_statuses()
.first()
.cloned();
Ok(status)
{
Ok(res) => return Ok(res),
Err(e) if try_count > 4 => return Err(e),
Err(_) => {
tokio::time::sleep(Duration::from_secs(2)).await;
try_count += 1;
}
}
}
}
#[instrument(level = "debug")]
@@ -247,28 +284,43 @@ async fn get_ec2_instance_state_name(
Ok(Some(state))
}
/// Automatically retries 5 times, waiting 2 sec in between
#[instrument(level = "debug")]
async fn get_ec2_instance_public_ip(
client: &Client,
instance_id: &str,
) -> anyhow::Result<String> {
let ip = client
.describe_instances()
.instance_ids(instance_id)
.send()
let mut try_count = 1;
loop {
match async {
anyhow::Ok(
client
.describe_instances()
.instance_ids(instance_id)
.send()
.await
.context("failed to describe instances from aws")?
.reservations()
.first()
.context("instance reservations is empty")?
.instances()
.first()
.context("instances is empty")?
.public_ip_address()
.context("instance has no public ip")?
.to_string(),
)
}
.await
.context("failed to get instance status from aws")?
.reservations()
.first()
.context("instance reservations is empty")?
.instances()
.first()
.context("instances is empty")?
.public_ip_address()
.context("instance has no public ip")?
.to_string();
Ok(ip)
{
Ok(res) => return Ok(res),
Err(e) if try_count > 4 => return Err(e),
Err(_) => {
tokio::time::sleep(Duration::from_secs(2)).await;
try_count += 1;
}
}
}
}
fn handle_unknown_instance_type(
@@ -1055,7 +1107,90 @@ fn handle_unknown_instance_type(
| InstanceType::Z1d6xlarge
| InstanceType::Z1dLarge
| InstanceType::Z1dMetal
| InstanceType::Z1dXlarge => Ok(instance_type),
| InstanceType::Z1dXlarge
| InstanceType::C7gdMetal
| InstanceType::C7gnMetal
| InstanceType::C7iFlex2xlarge
| InstanceType::C7iFlex4xlarge
| InstanceType::C7iFlex8xlarge
| InstanceType::C7iFlexLarge
| InstanceType::C7iFlexXlarge
| InstanceType::C8g12xlarge
| InstanceType::C8g16xlarge
| InstanceType::C8g24xlarge
| InstanceType::C8g2xlarge
| InstanceType::C8g48xlarge
| InstanceType::C8g4xlarge
| InstanceType::C8g8xlarge
| InstanceType::C8gLarge
| InstanceType::C8gMedium
| InstanceType::C8gMetal24xl
| InstanceType::C8gMetal48xl
| InstanceType::C8gXlarge
| InstanceType::G612xlarge
| InstanceType::G616xlarge
| InstanceType::G624xlarge
| InstanceType::G62xlarge
| InstanceType::G648xlarge
| InstanceType::G64xlarge
| InstanceType::G68xlarge
| InstanceType::G6Xlarge
| InstanceType::G6e12xlarge
| InstanceType::G6e16xlarge
| InstanceType::G6e24xlarge
| InstanceType::G6e2xlarge
| InstanceType::G6e48xlarge
| InstanceType::G6e4xlarge
| InstanceType::G6e8xlarge
| InstanceType::G6eXlarge
| InstanceType::Gr64xlarge
| InstanceType::Gr68xlarge
| InstanceType::M7gdMetal
| InstanceType::M8g12xlarge
| InstanceType::M8g16xlarge
| InstanceType::M8g24xlarge
| InstanceType::M8g2xlarge
| InstanceType::M8g48xlarge
| InstanceType::M8g4xlarge
| InstanceType::M8g8xlarge
| InstanceType::M8gLarge
| InstanceType::M8gMedium
| InstanceType::M8gMetal24xl
| InstanceType::M8gMetal48xl
| InstanceType::M8gXlarge
| InstanceType::Mac2M1ultraMetal
| InstanceType::R7gdMetal
| InstanceType::R7izMetal16xl
| InstanceType::R7izMetal32xl
| InstanceType::R8g12xlarge
| InstanceType::R8g16xlarge
| InstanceType::R8g24xlarge
| InstanceType::R8g2xlarge
| InstanceType::R8g48xlarge
| InstanceType::R8g4xlarge
| InstanceType::R8g8xlarge
| InstanceType::R8gLarge
| InstanceType::R8gMedium
| InstanceType::R8gMetal24xl
| InstanceType::R8gMetal48xl
| InstanceType::R8gXlarge
| InstanceType::U7i12tb224xlarge
| InstanceType::U7ib12tb224xlarge
| InstanceType::U7in16tb224xlarge
| InstanceType::U7in24tb224xlarge
| InstanceType::U7in32tb224xlarge
| InstanceType::X8g12xlarge
| InstanceType::X8g16xlarge
| InstanceType::X8g24xlarge
| InstanceType::X8g2xlarge
| InstanceType::X8g48xlarge
| InstanceType::X8g4xlarge
| InstanceType::X8g8xlarge
| InstanceType::X8gLarge
| InstanceType::X8gMedium
| InstanceType::X8gMetal24xl
| InstanceType::X8gMetal48xl
| InstanceType::X8gXlarge => Ok(instance_type),
other => Err(anyhow!("unknown InstanceType: {other:?}")),
}
}

View File

@@ -1,82 +0,0 @@
use anyhow::{anyhow, Context};
use aws_config::{BehaviorVersion, Region};
use aws_sdk_ecr::Client as EcrClient;
use run_command::async_run_command;
#[tracing::instrument(skip(access_key_id, secret_access_key))]
async fn make_ecr_client(
region: String,
access_key_id: &str,
secret_access_key: &str,
) -> EcrClient {
std::env::set_var("AWS_ACCESS_KEY_ID", access_key_id);
std::env::set_var("AWS_SECRET_ACCESS_KEY", secret_access_key);
let region = Region::new(region);
let config = aws_config::defaults(BehaviorVersion::v2024_03_28())
.region(region)
.load()
.await;
EcrClient::new(&config)
}
#[tracing::instrument(skip(access_key_id, secret_access_key))]
pub async fn maybe_create_repo(
repo: &str,
region: String,
access_key_id: &str,
secret_access_key: &str,
) -> anyhow::Result<()> {
let client =
make_ecr_client(region, access_key_id, secret_access_key).await;
let existing = client
.describe_repositories()
.send()
.await
.context("failed to describe existing repositories")?
.repositories
.unwrap_or_default();
if existing.iter().any(|r| {
if let Some(name) = r.repository_name() {
name == repo
} else {
false
}
}) {
return Ok(());
};
client
.create_repository()
.repository_name(repo)
.send()
.await
.context("failed to create repository")?;
Ok(())
}
/// Gets a token docker login.
///
/// Requires the aws cli be installed on the host
#[tracing::instrument(skip(access_key_id, secret_access_key))]
pub async fn get_ecr_token(
region: &str,
access_key_id: &str,
secret_access_key: &str,
) -> anyhow::Result<String> {
let log = async_run_command(&format!(
"AWS_ACCESS_KEY_ID={access_key_id} AWS_SECRET_ACCESS_KEY={secret_access_key} aws ecr get-login-password --region {region}"
))
.await;
if log.success() {
Ok(log.stdout)
} else {
Err(
anyhow!("stdout: {} | stderr: {}", log.stdout, log.stderr)
.context("failed to get aws ecr login token"),
)
}
}

View File

@@ -1,2 +1 @@
pub mod ec2;
pub mod ecr;

View File

@@ -1,7 +1,7 @@
use anyhow::{anyhow, Context};
use anyhow::{Context, anyhow};
use axum::http::{HeaderName, HeaderValue};
use reqwest::{RequestBuilder, StatusCode};
use serde::{de::DeserializeOwned, Serialize};
use serde::{Serialize, de::DeserializeOwned};
use super::{
common::{

View File

@@ -3,7 +3,7 @@ use std::{
time::Duration,
};
use anyhow::{anyhow, Context};
use anyhow::{Context, anyhow};
use futures::future::join_all;
use komodo_client::entities::server_template::hetzner::{
HetznerDatacenter, HetznerServerTemplateConfig, HetznerServerType,
@@ -66,6 +66,7 @@ pub async fn launch_hetzner_server(
labels,
volumes,
port: _,
use_https: _,
} = config;
let datacenter = hetzner_datacenter(datacenter);

View File

@@ -5,6 +5,8 @@ pub mod hetzner;
#[derive(Debug)]
pub enum BuildCleanupData {
Server { repo_name: String },
/// Nothing to clean up
Server,
/// Clean up AWS instance
Aws { instance_id: String, region: String },
}

View File

@@ -1,38 +1,18 @@
use std::sync::OnceLock;
use anyhow::Context;
use merge_config_files::parse_config_file;
use environment_file::{
maybe_read_item_from_file, maybe_read_list_from_file,
};
use komodo_client::entities::{
config::core::{
AwsCredentials, CoreConfig, Env, GithubWebhookAppConfig,
GithubWebhookAppInstallationConfig, HetznerCredentials,
MongoConfig, OauthCredentials,
AwsCredentials, CoreConfig, DatabaseConfig, Env,
GithubWebhookAppConfig, GithubWebhookAppInstallationConfig,
HetznerCredentials, OauthCredentials,
},
logger::LogConfig,
};
use serde::Deserialize;
pub fn frontend_path() -> &'static String {
#[derive(Deserialize)]
struct FrontendEnv {
#[serde(default = "default_frontend_path")]
komodo_frontend_path: String,
}
fn default_frontend_path() -> String {
"/frontend".to_string()
}
static FRONTEND_PATH: OnceLock<String> = OnceLock::new();
FRONTEND_PATH.get_or_init(|| {
let FrontendEnv {
komodo_frontend_path,
} = envy::from_env()
.context("failed to parse FrontendEnv")
.unwrap();
komodo_frontend_path
})
}
use merge_config_files::parse_config_file;
pub fn core_config() -> &'static CoreConfig {
static CORE_CONFIG: OnceLock<CoreConfig> = OnceLock::new();
@@ -50,7 +30,7 @@ pub fn core_config() -> &'static CoreConfig {
.unwrap_or_else(|e| {
panic!("failed at parsing config at {config_path} | {e:#}")
});
let installations = match (env.komodo_github_webhook_app_installations_ids, env.komodo_github_webhook_app_installations_namespaces) {
let installations = match (maybe_read_list_from_file(env.komodo_github_webhook_app_installations_ids_file,env.komodo_github_webhook_app_installations_ids), env.komodo_github_webhook_app_installations_namespaces) {
(Some(ids), Some(namespaces)) => {
if ids.len() != namespaces.len() {
panic!("KOMODO_GITHUB_WEBHOOK_APP_INSTALLATIONS_IDS length and KOMODO_GITHUB_WEBHOOK_APP_INSTALLATIONS_NAMESPACES length mismatch. Got {ids:?} and {namespaces:?}")
@@ -71,35 +51,112 @@ pub fn core_config() -> &'static CoreConfig {
config.github_webhook_app.installations
}
};
// recreating CoreConfig here makes sure we apply all env overrides.
// recreating CoreConfig here makes sure apply all env overrides applied.
CoreConfig {
// Secret things overridden with file
jwt_secret: maybe_read_item_from_file(env.komodo_jwt_secret_file, env.komodo_jwt_secret).unwrap_or(config.jwt_secret),
passkey: maybe_read_item_from_file(env.komodo_passkey_file, env.komodo_passkey)
.unwrap_or(config.passkey),
webhook_secret: maybe_read_item_from_file(env.komodo_webhook_secret_file, env.komodo_webhook_secret)
.unwrap_or(config.webhook_secret),
database: DatabaseConfig {
uri: maybe_read_item_from_file(env.komodo_database_uri_file,env.komodo_database_uri).unwrap_or(config.database.uri),
address: env.komodo_database_address.unwrap_or(config.database.address),
username: maybe_read_item_from_file(env.komodo_database_username_file,env
.komodo_database_username)
.unwrap_or(config.database.username),
password: maybe_read_item_from_file(env.komodo_database_password_file,env
.komodo_database_password)
.unwrap_or(config.database.password),
app_name: env
.komodo_database_app_name
.unwrap_or(config.database.app_name),
db_name: env
.komodo_database_db_name
.unwrap_or(config.database.db_name),
},
oidc_enabled: env.komodo_oidc_enabled.unwrap_or(config.oidc_enabled),
oidc_provider: env.komodo_oidc_provider.unwrap_or(config.oidc_provider),
oidc_redirect_host: env.komodo_oidc_redirect_host.unwrap_or(config.oidc_redirect_host),
oidc_client_id: maybe_read_item_from_file(env.komodo_oidc_client_id_file,env
.komodo_oidc_client_id)
.unwrap_or(config.oidc_client_id),
oidc_client_secret: maybe_read_item_from_file(env.komodo_oidc_client_secret_file,env
.komodo_oidc_client_secret)
.unwrap_or(config.oidc_client_secret),
oidc_use_full_email: env.komodo_oidc_use_full_email
.unwrap_or(config.oidc_use_full_email),
oidc_additional_audiences: maybe_read_list_from_file(env.komodo_oidc_additional_audiences_file,env
.komodo_oidc_additional_audiences)
.unwrap_or(config.oidc_additional_audiences),
google_oauth: OauthCredentials {
enabled: env
.komodo_google_oauth_enabled
.unwrap_or(config.google_oauth.enabled),
id: maybe_read_item_from_file(env.komodo_google_oauth_id_file,env
.komodo_google_oauth_id)
.unwrap_or(config.google_oauth.id),
secret: maybe_read_item_from_file(env.komodo_google_oauth_secret_file,env
.komodo_google_oauth_secret)
.unwrap_or(config.google_oauth.secret),
},
github_oauth: OauthCredentials {
enabled: env
.komodo_github_oauth_enabled
.unwrap_or(config.github_oauth.enabled),
id: maybe_read_item_from_file(env.komodo_github_oauth_id_file,env
.komodo_github_oauth_id)
.unwrap_or(config.github_oauth.id),
secret: maybe_read_item_from_file(env.komodo_github_oauth_secret_file,env
.komodo_github_oauth_secret)
.unwrap_or(config.github_oauth.secret),
},
aws: AwsCredentials {
access_key_id: maybe_read_item_from_file(env.komodo_aws_access_key_id_file, env
.komodo_aws_access_key_id)
.unwrap_or(config.aws.access_key_id),
secret_access_key: maybe_read_item_from_file(env.komodo_aws_secret_access_key_file, env
.komodo_aws_secret_access_key)
.unwrap_or(config.aws.secret_access_key),
},
hetzner: HetznerCredentials {
token: maybe_read_item_from_file(env.komodo_hetzner_token_file, env
.komodo_hetzner_token)
.unwrap_or(config.hetzner.token),
},
github_webhook_app: GithubWebhookAppConfig {
app_id: maybe_read_item_from_file(env.komodo_github_webhook_app_app_id_file, env
.komodo_github_webhook_app_app_id)
.unwrap_or(config.github_webhook_app.app_id),
pk_path: env
.komodo_github_webhook_app_pk_path
.unwrap_or(config.github_webhook_app.pk_path),
installations,
},
// Non secrets
title: env.komodo_title.unwrap_or(config.title),
host: env.komodo_host.unwrap_or(config.host),
port: env.komodo_port.unwrap_or(config.port),
passkey: env.komodo_passkey.unwrap_or(config.passkey),
ensure_server: env.komodo_ensure_server.unwrap_or(config.ensure_server),
jwt_secret: env.komodo_jwt_secret.unwrap_or(config.jwt_secret),
bind_ip: env.komodo_bind_ip.unwrap_or(config.bind_ip),
first_server: env.komodo_first_server.unwrap_or(config.first_server),
frontend_path: env.komodo_frontend_path.unwrap_or(config.frontend_path),
jwt_ttl: env
.komodo_jwt_ttl
.unwrap_or(config.jwt_ttl),
sync_directory: env
.komodo_sync_directory
.unwrap_or(config.sync_directory),
repo_directory: env
.komodo_repo_directory
.map(|dir|
dir.parse()
.context("failed to parse env komodo_REPO_DIRECTORY as valid path").unwrap())
.unwrap_or(config.repo_directory),
stack_poll_interval: env
.komodo_stack_poll_interval
.unwrap_or(config.stack_poll_interval),
sync_poll_interval: env
.komodo_sync_poll_interval
.unwrap_or(config.sync_poll_interval),
build_poll_interval: env
.komodo_build_poll_interval
.unwrap_or(config.build_poll_interval),
repo_poll_interval: env
.komodo_repo_poll_interval
.unwrap_or(config.repo_poll_interval),
action_directory: env
.komodo_action_directory
.unwrap_or(config.action_directory),
resource_poll_interval: env
.komodo_resource_poll_interval
.unwrap_or(config.resource_poll_interval),
monitoring_interval: env
.komodo_monitoring_interval
.unwrap_or(config.monitoring_interval),
@@ -109,83 +166,27 @@ pub fn core_config() -> &'static CoreConfig {
keep_alerts_for_days: env
.komodo_keep_alerts_for_days
.unwrap_or(config.keep_alerts_for_days),
webhook_secret: env
.komodo_webhook_secret
.unwrap_or(config.webhook_secret),
webhook_base_url: env
.komodo_webhook_base_url
.or(config.webhook_base_url),
.unwrap_or(config.webhook_base_url),
transparent_mode: env
.komodo_transparent_mode
.unwrap_or(config.transparent_mode),
ui_write_disabled: env
.komodo_ui_write_disabled
.unwrap_or(config.ui_write_disabled),
disable_confirm_dialog: env.komodo_disable_confirm_dialog
.unwrap_or(config.disable_confirm_dialog),
enable_new_users: env.komodo_enable_new_users
.unwrap_or(config.enable_new_users),
disable_user_registration: env.komodo_disable_user_registration
.unwrap_or(config.disable_user_registration),
local_auth: env.komodo_local_auth.unwrap_or(config.local_auth),
google_oauth: OauthCredentials {
enabled: env
.komodo_google_oauth_enabled
.unwrap_or(config.google_oauth.enabled),
id: env
.komodo_google_oauth_id
.unwrap_or(config.google_oauth.id),
secret: env
.komodo_google_oauth_secret
.unwrap_or(config.google_oauth.secret),
},
github_oauth: OauthCredentials {
enabled: env
.komodo_github_oauth_enabled
.unwrap_or(config.github_oauth.enabled),
id: env
.komodo_github_oauth_id
.unwrap_or(config.github_oauth.id),
secret: env
.komodo_github_oauth_secret
.unwrap_or(config.github_oauth.secret),
},
github_webhook_app: GithubWebhookAppConfig {
app_id: env
.komodo_github_webhook_app_app_id
.unwrap_or(config.github_webhook_app.app_id),
pk_path: env
.komodo_github_webhook_app_pk_path
.unwrap_or(config.github_webhook_app.pk_path),
installations,
},
aws: AwsCredentials {
access_key_id: env
.komodo_aws_access_key_id
.unwrap_or(config.aws.access_key_id),
secret_access_key: env
.komodo_aws_secret_access_key
.unwrap_or(config.aws.secret_access_key),
},
hetzner: HetznerCredentials {
token: env
.komodo_hetzner_token
.unwrap_or(config.hetzner.token),
},
mongo: MongoConfig {
uri: env.komodo_mongo_uri.or(config.mongo.uri),
address: env.komodo_mongo_address.or(config.mongo.address),
username: env
.komodo_mongo_username
.or(config.mongo.username),
password: env
.komodo_mongo_password
.or(config.mongo.password),
app_name: env
.komodo_mongo_app_name
.unwrap_or(config.mongo.app_name),
db_name: env
.komodo_mongo_db_name
.unwrap_or(config.mongo.db_name),
},
disable_non_admin_create: env.komodo_disable_non_admin_create
.unwrap_or(config.disable_non_admin_create),
lock_login_credentials_for: env.komodo_lock_login_credentials_for
.unwrap_or(config.lock_login_credentials_for),
local_auth: env.komodo_local_auth
.unwrap_or(config.local_auth),
logging: LogConfig {
level: env
.komodo_logging_level
@@ -193,19 +194,22 @@ pub fn core_config() -> &'static CoreConfig {
stdio: env
.komodo_logging_stdio
.unwrap_or(config.logging.stdio),
pretty: env.komodo_logging_pretty.unwrap_or(config.logging.pretty),
otlp_endpoint: env
.komodo_logging_otlp_endpoint
.or(config.logging.otlp_endpoint),
.unwrap_or(config.logging.otlp_endpoint),
opentelemetry_service_name: env
.komodo_logging_opentelemetry_service_name
.unwrap_or(config.logging.opentelemetry_service_name),
},
ssl_enabled: env.komodo_ssl_enabled.unwrap_or(config.ssl_enabled),
ssl_key_file: env.komodo_ssl_key_file.unwrap_or(config.ssl_key_file),
ssl_cert_file: env.komodo_ssl_cert_file.unwrap_or(config.ssl_cert_file),
// These can't be overridden on env
secrets: config.secrets,
git_providers: config.git_providers,
docker_registries: config.docker_registries,
aws_ecr_registries: config.aws_ecr_registries,
}
})
}

View File

@@ -1,11 +1,11 @@
use mongo_indexed::{create_index, create_unique_index};
use komodo_client::entities::{
action::Action,
alert::Alert,
alerter::Alerter,
api_key::ApiKey,
build::Build,
builder::Builder,
config::core::MongoConfig,
config::core::DatabaseConfig,
deployment::Deployment,
permission::Permission,
procedure::Procedure,
@@ -22,11 +22,13 @@ use komodo_client::entities::{
user_group::UserGroup,
variable::Variable,
};
use mongo_indexed::{create_index, create_unique_index};
use mungos::{
init::MongoBuilder,
mongodb::{Collection, Database},
};
#[derive(Debug)]
pub struct DbClient {
pub users: Collection<User>,
pub user_groups: Collection<UserGroup>,
@@ -46,6 +48,7 @@ pub struct DbClient {
pub builders: Collection<Builder>,
pub repos: Collection<Repo>,
pub procedures: Collection<Procedure>,
pub actions: Collection<Action>,
pub alerters: Collection<Alerter>,
pub server_templates: Collection<ServerTemplate>,
pub resource_syncs: Collection<ResourceSync>,
@@ -56,32 +59,39 @@ pub struct DbClient {
impl DbClient {
pub async fn new(
MongoConfig {
DatabaseConfig {
uri,
address,
username,
password,
app_name,
db_name,
}: &MongoConfig,
}: &DatabaseConfig,
) -> anyhow::Result<DbClient> {
let mut client = MongoBuilder::default().app_name(app_name);
match (uri, address, username, password) {
(Some(uri), _, _, _) => {
match (
!uri.is_empty(),
!address.is_empty(),
!username.is_empty(),
!password.is_empty(),
) {
(true, _, _, _) => {
client = client.uri(uri);
}
(_, Some(address), Some(username), Some(password)) => {
(_, true, true, true) => {
client = client
.address(address)
.username(username)
.password(password);
}
(_, Some(address), _, _) => {
(_, true, _, _) => {
client = client.address(address);
}
_ => {
error!("config.mongo not configured correctly. must pass either config.mongo.uri, or config.mongo.address + config.mongo.username? + config.mongo.password?");
error!(
"config.mongo not configured correctly. must pass either config.mongo.uri, or config.mongo.address + config.mongo.username? + config.mongo.password?"
);
std::process::exit(1)
}
}
@@ -109,6 +119,7 @@ impl DbClient {
repos: resource_collection(&db, "Repo").await?,
alerters: resource_collection(&db, "Alerter").await?,
procedures: resource_collection(&db, "Procedure").await?,
actions: resource_collection(&db, "Action").await?,
server_templates: resource_collection(&db, "ServerTemplate")
.await?,
resource_syncs: resource_collection(&db, "ResourceSync")

View File

@@ -4,7 +4,8 @@ use anyhow::anyhow;
use komodo_client::{
busy::Busy,
entities::{
build::BuildActionState, deployment::DeploymentActionState,
action::ActionActionState, build::BuildActionState,
deployment::DeploymentActionState,
procedure::ProcedureActionState, repo::RepoActionState,
server::ServerActionState, stack::StackActionState,
sync::ResourceSyncActionState,
@@ -22,6 +23,7 @@ pub struct ActionStates {
pub repo: Cache<String, Arc<ActionState<RepoActionState>>>,
pub procedure:
Cache<String, Arc<ActionState<ProcedureActionState>>>,
pub action: Cache<String, Arc<ActionState<ActionActionState>>>,
pub resource_sync:
Cache<String, Arc<ActionState<ResourceSyncActionState>>>,
pub stack: Cache<String, Arc<ActionState<StackActionState>>>,
@@ -82,8 +84,8 @@ pub struct UpdateGuard<'a, States: Default + Send + 'static>(
&'a Mutex<States>,
);
impl<'a, States: Default + Send + 'static> Drop
for UpdateGuard<'a, States>
impl<States: Default + Send + 'static> Drop
for UpdateGuard<'_, States>
{
fn drop(&mut self) {
let mut lock = match self.0.lock() {

View File

@@ -1,49 +0,0 @@
use async_timing_util::{wait_until_timelength, Timelength};
use komodo_client::{
api::write::RefreshBuildCache, entities::user::build_user,
};
use mungos::find::find_collect;
use resolver_api::Resolve;
use crate::{
config::core_config,
state::{db_client, State},
};
pub fn spawn_build_refresh_loop() {
let interval: Timelength = core_config()
.build_poll_interval
.try_into()
.expect("Invalid build poll interval");
tokio::spawn(async move {
refresh_builds().await;
loop {
wait_until_timelength(interval, 2000).await;
refresh_builds().await;
}
});
}
async fn refresh_builds() {
let Ok(builds) =
find_collect(&db_client().await.builds, None, None)
.await
.inspect_err(|e| {
warn!("failed to get builds from db in refresh task | {e:#}")
})
else {
return;
};
for build in builds {
State
.resolve(
RefreshBuildCache { build: build.id },
build_user().clone(),
)
.await
.inspect_err(|e| {
warn!("failed to refresh build cache in refresh task | build: {} | {e:#}", build.name)
})
.ok();
}
}

View File

@@ -1,27 +1,27 @@
use std::time::Duration;
use anyhow::{anyhow, Context};
use anyhow::{Context, anyhow};
use formatting::muted;
use komodo_client::entities::{
Version,
builder::{AwsBuilderConfig, Builder, BuilderConfig},
komodo_timestamp,
server::Server,
server_template::aws::AwsServerTemplateConfig,
update::{Log, Update},
Version,
};
use periphery_client::{
api::{self, GetVersionResponse},
PeripheryClient,
api::{self, GetVersionResponse},
};
use crate::{
cloud::{
aws::ec2::{
launch_ec2_instance, terminate_ec2_instance_with_retry,
Ec2Instance,
},
BuildCleanupData,
aws::ec2::{
Ec2Instance, launch_ec2_instance,
terminate_ec2_instance_with_retry,
},
},
config::core_config,
helpers::update::update_update,
@@ -31,7 +31,7 @@ use crate::{
use super::periphery_client;
const BUILDER_POLL_RATE_SECS: u64 = 2;
const BUILDER_POLL_MAX_TRIES: usize = 30;
const BUILDER_POLL_MAX_TRIES: usize = 60;
#[instrument(skip_all, fields(builder_id = builder.id, update_id = update.id))]
pub async fn get_builder_periphery(
@@ -42,18 +42,34 @@ pub async fn get_builder_periphery(
update: &mut Update,
) -> anyhow::Result<(PeripheryClient, BuildCleanupData)> {
match builder.config {
BuilderConfig::Url(config) => {
if config.address.is_empty() {
return Err(anyhow!(
"Builder has not yet configured an address"
));
}
let periphery = PeripheryClient::new(
config.address,
if config.passkey.is_empty() {
core_config().passkey.clone()
} else {
config.passkey
},
Duration::from_secs(3),
);
periphery
.health_check()
.await
.context("Url Builder failed health check")?;
Ok((periphery, BuildCleanupData::Server))
}
BuilderConfig::Server(config) => {
if config.server_id.is_empty() {
return Err(anyhow!("builder has not configured a server"));
return Err(anyhow!("Builder has not configured a server"));
}
let server = resource::get::<Server>(&config.server_id).await?;
let periphery = periphery_client(&server)?;
Ok((
periphery,
BuildCleanupData::Server {
repo_name: resource_name,
},
))
Ok((periphery, BuildCleanupData::Server))
}
BuilderConfig::Aws(config) => {
get_aws_builder(&resource_name, version, config, update).await
@@ -93,9 +109,14 @@ async fn get_aws_builder(
update_update(update.clone()).await?;
let periphery_address = format!("http://{ip}:{}", config.port);
let periphery =
PeripheryClient::new(&periphery_address, &core_config().passkey);
let protocol = if config.use_https { "https" } else { "http" };
let periphery_address =
format!("{protocol}://{ip}:{}", config.port);
let periphery = PeripheryClient::new(
&periphery_address,
&core_config().passkey,
Duration::from_secs(3),
);
let start_connect_ts = komodo_timestamp();
let mut res = Ok(GetVersionResponse {
@@ -148,17 +169,14 @@ async fn get_aws_builder(
)
}
#[instrument(skip(periphery, update))]
#[instrument(skip(update))]
pub async fn cleanup_builder_instance(
periphery: PeripheryClient,
cleanup_data: BuildCleanupData,
update: &mut Update,
) {
match cleanup_data {
BuildCleanupData::Server { repo_name } => {
let _ = periphery
.request(api::git::DeleteRepo { name: repo_name })
.await;
BuildCleanupData::Server => {
// Nothing to clean up
}
BuildCleanupData::Aws {
instance_id,
@@ -191,6 +209,7 @@ pub fn start_aws_builder_log(
assign_public_ip,
security_group_ids,
use_public_ip,
use_https,
..
} = config;
@@ -206,6 +225,7 @@ pub fn start_aws_builder_log(
format!("{}: {readable_sec_group_ids}", muted("security groups")),
format!("{}: {assign_public_ip}", muted("assign public ip")),
format!("{}: {use_public_ip}", muted("use public ip")),
format!("{}: {use_https}", muted("use https")),
]
.join("\n")
}

Some files were not shown because too many files have changed in this diff Show More