ComputerSurge/komodo
3
0
Fork 0
forked from github-starred/komodo
Code Pull Requests 1 Actions Packages Releases Activity

delete old docs and provide link to docsite

Browse Source
This commit is contained in:
beckerinj
2023-04-03 11:28:50 -04:00
parent af597eb3c7
commit 6239da45f4
8 changed files with 1 additions and 350 deletions
Download Patch File Download Diff File Expand all files Collapse all files

56
docs/builds.md
View File

@@ -1,56 +0,0 @@
# building images
Monitor builds docker images by cloning the source repository from Github, running ```docker build```, and pushing the resulting image to docker hub. Any repo containing a 'Dockerfile' is buildable using this method.
Build configuration involves passing file / directory paths, for more details about passing file paths, see the [file paths doc](https://github.com/mbecker20/monitor/blob/main/docs/paths.md).
## repo configuration
To specify the github repo to build, just give it the name of the repo and the branch under *repo config*. The name is given like ```mbecker20/monitor```, it includes the username / organization that owns the repo.
Many repos are private, in this case a Github access token is required in the periphery.config.toml of the building server. these are specified in the config like ```username = "access_token"```. An account which has access to the repo and is available on the periphery server can be selected to use via the *github account* dropdown menu.
## docker build configuration
In order to docker build, monitor just needs to know the build directory and the path of the Dockerfile, you can configure these in the *build config* section.
If the build directory is the root of the repository, you pass the build path as ```.```. If the build directory is some folder of the repo, just pass the name of the the folder. Do not pass the preceding "/". for example ```build/directory```
The dockerfile's path is given relative to the build directory. So if your build directory is ```build/directory``` and the dockerfile is in ```build/directory/Dockerfile.example```, you give the dockerfile path simply as ```Dockerfile.example```.
Just as with private repos, you will need to select a docker account to use with ```docker push```.
## running a pre build command
Sometimes a command needs to be run before running ```docker build```, you can configure this in the *pre build* section.
There are two fields to pass for *pre build*. the first is *path*, which changes the working directory. To run the command in the root of the repo, just pass ```.```. The second field is *command*, this is the shell command to be executed after the repo is cloned.
For example, say your repo had a folder in it called ```scripts``` with a shell script ```on-clone.sh```. You would give *path* as ```scripts``` and command as ```sh on-clone.sh```. Or you could make *path* just ```.``` and then the command would be ```sh scripts/on-clone.sh```. Either way works fine.
## adding build args
The Dockerfile may make use of [build args](https://docs.docker.com/engine/reference/builder/#arg). Build args can be passed using the gui by pressing the ```edit``` button. They are passed in the menu just like in the would in a .env file:
```
BUILD_ARG1=some_value
BUILD_ARG2=some_other_value
```
## builder configuration
A builder is a machine running monitor periphery and docker. Any server connected to monitor can be chosen as the builder for a build.
Building on a machine running production software is usually not a great idea, as this process can use a lot of the system resources. It is better to start up a temporary cloud machine dedicated for the build, then shut it down when the build is finished. Right now monitor supports AWS ec2 for this task.
### AWS builder
You can choose to build on AWS on the "builder" tab on the build's page. From here you can configure the AMI to use as a base to build the image. These must be configured in the monitor core configuration along with other information like defaults to use, AWS credentials, etc. This is explained on the [core setup page](https://github.com/mbecker20/monitor/blob/main/docs/setup.md).
## versioning
Monitor uses a major.minor.patch versioning scheme. Every build will auto increment the patch number, and push the image to docker hub with the version tag as well as the "latest" tag.
[next: deploying](https://github.com/mbecker20/monitor/blob/main/docs/deployments.md)
[back to table of contents](https://github.com/mbecker20/monitor/blob/main/readme.md)

103
docs/deployments.md
View File

@@ -1,103 +0,0 @@
# deploying applications
Monitor can deploy any docker images that it can access with the configured docker accounts. It works by parsing the deployment configuration into a ```docker run``` command. The configuration is stored on MongoDB, and records of all actions (update config, deploy, stop, etc.) are stored as well.
Deployment configuration involves passing file / directory paths, for more details about passing file paths, see the [file paths doc](https://github.com/mbecker20/monitor/blob/main/docs/paths.md).
## configuring the image
There are two options to configure the docker image to deploy.
### attaching a monitor build
If the software you want to deploy is built by monitor, you can attach the build directly to the deployment.
By default, monitor will deploy the latest available version of the build, or you can specify a specific version using the version dropdown.
Also by default, monitor will use the same docker account that is attached to the build in order to pull the image on the periphery server. If that account is not available on the server, you can specify another available account to use instead, this account just needs to have read access to the docker repository.
### using a custom image
You can also manually specify an image name, like ```mongo``` or ```mbecker2020/random_image:0.1.1```.
If the image repository is private, you can select an available docker account to use to pull the image.
## configuring the network
One feature of docker is that it allows for the creation of [virtual networks between containers](https://docs.docker.com/network/). Monitor allows you to specify a docker virtual network to connect the container to, or to use the host system networking to bypass the docker virtual network.
The default selection is ```host```, which bypasses the docker virtual network layer.
If you do select select a network other than host, you can specify port bindings with the GUI. For example, if you are running mongo (which defaults to port 27017), you could use the mapping:
```
27018 : 27017
```
In this case, you would access mongo from outside of the container on port ```27018```.
Note that this is not the only affect of using a network other than ```host```. For example, containers running on different networks can not communicate, and ones on the same network can not reach other containers on ```localhost``` even when they are running on the same system. This behavior can be a bit confusing if you are not familiar with it, and it can be bypassed entirely by just using ```host``` network.
## configuring restart behavior
Docker, like systemd, has a couple options for handling when a container exits. See [docker restart policies](https://docs.docker.com/config/containers/start-containers-automatically/). Monitor allows you to select the appropriate restart behavior from these options.
## configuring environment variables
Monitor enables you to easily manage environment variables passed to the container. In the GUI, click the 'edit' button on the 'environment' card, this will bring up the environment menu.
You pass environment variables just as you would with a ```.env``` file:
```
ENV_VAR_1=some_value
ENV_VAR_2=some_other_value
```
## configuring volumes
A docker container's filesystem is segregated from that of the host. However, it is still possible for a container to access system files and directories, this is accomplished by using [bind mounts](https://docs.docker.com/storage/bind-mounts/).
Say your container needs to read a config file located on the system at ```/home/ubuntu/config.toml```. You can specify the bind mount to be:
```
/home/ubuntu/config.toml : /config/config.toml
```
The first path is the one on the system, the second is the path in the container. Your application would then read the file at ```/config/config.toml``` in order to load its contents.
These can be configured easily with the GUI in the 'volumes' card. You can configure as many bind mounts as you need.
## extra args
Not all features of docker are mapped directly by monitor, only the most common. You can still specify any custom flags for monitor to include in the ```docker run``` command by utilizing 'extra args'. For example, you can enable log rotation using these two extra args:
```
--log-opt max-size=10M
```
```
--log-opt max-file=3
```
## post image
Sometimes you need to specify some flags to be passed directly to the application. What is put here is inserted into the docker run command after the image. For example, to pass the ```--quiet``` flag to MongoDB, the docker run command would be:
```
docker run -d --name mongo-db mongo:6.0.3 --quiet
```
In order to achieve this with monitor, just pass ```--quiet``` to 'post image'.
## container lifetime management
The lifetime of a docker container is more like a virtual machine. They can be created, started, stopped, and destroyed. The lifetime management actions monitor presents to the user is relative to the containers state. For example, when the container is ```running```, you can either stop it, destroy it, or redeploy it.
### stopping a container
Sometimes you want to stop a running application but preserve its logs and configuration, either to be restarted later or to view the logs at a later time. It is more like *pausing* the application with its current config, as no configuration (like environment variable, volume mounts, etc.) will be changed when the container is started again. In order to restart an application with updated configuration, it must be *redeployed*.
### container redeploy
redeploying is the action of destroying a container and recreating it. If you update deployment config, these changes will not take effect until the container is redeployed. Just note this will destroy the previous containers logs along with the container itself.
[next: permissions](https://github.com/mbecker20/monitor/blob/main/docs/permissions.md)
[back to table of contents](https://github.com/mbecker20/monitor/blob/main/readme.md)

44
docs/introduction.md
View File

@@ -1,44 +0,0 @@
# introduction
If you have many servers running many applications, it can be a challenge to keep things organized and easily accessible. Without structure, things can become messy quickly, which means operational issues are more likely to arise and they can take longer to resolve. Ultimately these issues hinder productivity and waste valuable time. Monitor is a web app to provide this structure for how applications are built, deployed, and managed across many servers.
## docker
Monitor is opinionated by design, and [docker](https://docs.docker.com/) is the tool of choice. Docker provides the ability to package applications and their runtime dependencies into a standalone bundle, called an *image*. This makes them easy to "ship" to any server and run without the hassle of setting up the runtime environment. Docker uses the image as a sort of template to create *containers*. Containers are kind of like virtual machines but with different performance characteristics, namely that processes contained still run natively on the system kernel. The file system is seperate though, and like virtual machines, they can be created, started, stopped, and destroyed.
## monitor
Monitor is a solution for handling for the following:
1. Build application source into auto-versioned images.
2. Create, start, stop, and restart Docker containers, and view their status and logs.
3. Keep a record of all the actions that are performed and by whom.
4. View realtime and historical system resource usage.
5. Alerting for server health, like high cpu, memory, disk, etc.
## architecture and components
Monitor is composed of a single core and any amount of connected servers running the periphery application.
### monitor core
The core is a web server that hosts the core API and serves the frontend to be accessed in a web browser. All user interaction with the connected servers flow through the core. It is the stateful part of the system, with the application state stored on an instance of MongoDB.
### monitor periphery
The periphery is a stateless web server that exposes API called by the core. The core calls this API to get system usage and container status / logs, clone git repos, and perform docker actions. It is only intended to be reached from the core, and has an address whitelist to limit the IPs allowed to call this API.
### monitor cli
This is a simple standalone cli that helps perform some actions required to setup monitor core and periphery, like generating config files.
## core API
Monitor exposes powerful functionality over the core's REST API, enabling infrastructure engineers to manage deployments programmatically in addition to with the GUI. There is a [rust crate](https://crates.io/crates/monitor_client) to simplify programmatic interaction with the API, but in general this can be accomplished using any programming language that can make REST requests.
## permissioning
Monitor is a system designed to be used by many users, whether they are developers, operations personnel, or administrators. The ability to affect an applications state is very powerful, so monitor has a granular permissioning system to only provide this functionality to the intended users. The permissioning system is explained in detail in the [permissioning](https://github.com/mbecker20/monitor/blob/main/docs/permissions.md) section.
User sign-on is possible using username / password, or with Oauth (Github and Google). Allowed login methods can be configured from the [core config](https://github.com/mbecker20/monitor/blob/main/config_example/core.config.example.toml).
[next: connecting servers](https://github.com/mbecker20/monitor/blob/main/docs/servers.md)
[back to table of contents](https://github.com/mbecker20/monitor/blob/main/readme.md)

38
docs/paths.md
View File

@@ -1,38 +0,0 @@
# File Paths
When working with monitor, you might have to configure file or directory paths.
## Relative Paths
Where possible, it is better to use relative file paths. Using relative file paths removes the connection between the process being run and the particular server it runs one, making it easier to move things between servers.
Where you see relative paths:
- setting the build directory and path of the Dockerfile
- setting a pre build command path
- configuring a frontend mount (used for web apps)
For all of the above, the path can be given relative to the root of the configured repo
The one exception is the Dockerfile path, which is given relative to the build directory (This is done by Docker itself, and this pattern matches usage of the Docker CLI).
There are 3 kinds of paths to pass:
1. to specify the root of the repo, use ```.``` as the path
2. to specify a folder in the repo, pass it with **no** preceding ```/```. For example, ```example_folder``` or ```folder1/folder2```
3. to specify an absolute path on the servers filesystem, use a preceding slash, eg. ```/home/ubuntu/example```. This way should only be used if absolutely necessary.
### Implementation
relative file paths are joined with the path of the repo on the system using a Rust [PathBuf](https://doc.rust-lang.org/std/path/struct.PathBuf.html#method.push).
## Docker Volume Paths
These are passed directly to the Docker CLI using ```--volume /path/on/system:/path/in/container```. So for these, the same rules apply as when using Docker on the command line. Paths here should be given as absolute, don't use ```~``` or even ```$HOME```.

32
docs/permissions.md
View File

@@ -1,32 +0,0 @@
# permissioning resources
All monitor resources (servers, builds, deployment) have independant permission tables to allow for users to have granular access to these resources. By default, users do not see any resources until they are given at least read permissions.
## permission levels
There are 4 levels of permissions a user can have on a resource:
1. **None**. This is the lowest permission level, and means the user will not have any access to this resource. They will not see it in the GUI, and it will not show up if the user queries the core API directly. All attempts to view or update the resource will be blocked.
2. **Read**. This is the first permission level that grants any access. It will enable the user to see the resource in the GUI, read the configuration, and see any logs. Any attempts to update configuration or trigger any action will be blocked.
3. **Execute**. This level will allow the user to execute actions on the resource, like send a build command or trigger a redeploy. The user will still be blocked from updating configuration on the resource.
4. **Update**. The user has full access to the resource, they can execute any actions, update the configuration, and delete the resource.
## Administration
Users can be given admin priviledges by accessing the monitor MongoDB and setting ```admin: true``` on the intended user document. These users have unrestricted access to all monitor resources, like servers, builds, and deployments. Additionally, only these users can update other (non-admin) user's permissions on resources, an action not available to regular users even with **Update** level permissions.
Monitor admins are responsible for managing user accounts as well. When a user logs into monitor for the first time, they will not immediately be granted access. An admin must first **enable** the user, which can be done from the 'manage users' page (found in the user dropdown menu in the topbar). Users can also be **disabled** by an admin at any time, which blocks all their access to the GUI and API.
Users also have some configurable global permissions, these are:
- create server permission
- create build permission
Only users with these permissions (as well as admins) can add additional servers to monitor, and can create additional builds, respectively.
[next: core setup](https://github.com/mbecker20/monitor/blob/main/docs/setup.md)
[back to table of contents](https://github.com/mbecker20/monitor/blob/main/readme.md)

62
docs/servers.md
View File

@@ -1,62 +0,0 @@
# connecting servers
Integrating a device into the monitor system has 2 steps:
1. Setup and start the periphery agent on the server
2. Adding the server to monitor via the core API
## setup monitor periphery
The easiest way to do this is to follow the [monitor guide](https://github.com/mbecker20/monitor-guide). This is a repo containing directions and scripts enabling command line installation via ssh or remotely.
### manual install steps
1. Download the periphery binary from the latest [release](https://github.com/mbecker20/monitor/releases).
2. Create and edit your config files, following the [config example](https://github.com/mbecker20/monitor/blob/main/config_example/periphery.config.example.toml). The monitor cli can be used to add the boilerplate: ```monitor periphery gen-config --path /path/to/config.toml```. The files can be anywhere, and can be passed to periphery via the ```--config-path``` argument.
3. Ensure that inbound connectivity is allowed on the port specified in periphery.config.toml (default 8000).
4. Install docker. Make sure whatever user periphery is run as has access to the docker group without sudo.
5. Start the periphery binary with your preferred process manager, like systemd. The config read from the file is printed on startup, ensure that it is as expected.
## example periphery start command
```
periphery \
--config-path /path/to/periphery.config.base.toml \
--config-path /other_path/to/overide-periphery-config-directory \
--config-keyword periphery \
--config-keyword config \
--merge-nested-config \
--home_dir /home/username
```
## passing config files
Either file paths or directory paths can be passed to ```--config-path```.
When using directories, the file entries can be filtered by name with the ```--config-keyword``` argument, which can be passed multiple times to add more keywords. If passed, then only config files with file names that contain all keywords will be merged.
When passing multiple config files, later --config-path given in the command will always overide previous ones. Directory config files are merged in alphabetical order by name, so ```config_b.toml``` will overide ```config_a.toml```.
There are two ways to merge config files. The default behavior is to completely replace any base fields with whatever fields are present in the overide config. So if you pass ```allowed_ips = []``` in your overide config, the final allowed_ips will be an empty list as well.
```--merge-nested-config``` will merge config fields recursively and extend config array fields.
For example, with ```--merge-nested-config``` you can specify an allowed ip in the base config, and another in the overide config, they will both be present in the final config.
Similarly, you can specify a base docker / github account pair, and extend them with additional accounts in the overide config.
## adding the server to monitor
The easiest way to add the server is with the GUI. On the home page, click the + button to the right of the server search bar, configure the name and address of the server. The address is the full http/s url to the periphery server, eg http://12.34.56.78:8000.
Once it is added, you can use access the GUI to modify some config, like the alerting thresholds for cpu, memory and disk usage. A server can also be temporarily disabled, this will prevent alerting if it goes offline.
Since no state is stored on the periphery servers, you can easily redirect all deployments to be hosted on a different server. Just update the address to point to the new server.
[next: building](https://github.com/mbecker20/monitor/blob/main/docs/builds.md)
[back to table of contents](https://github.com/mbecker20/monitor/blob/main/readme.md)

5
docs/setup.md
View File

@@ -1,5 +0,0 @@
# setting up monitor core
[back to table of contents](https://github.com/mbecker20/monitor/blob/main/readme.md)

11
readme.md
View File

@@ -1,12 +1,3 @@
# monitor 🦎
a tool to build and deploy software across many servers
## table of contents
1. [introduction](https://github.com/mbecker20/monitor/blob/main/docs/introduction.md)
2. [connecting servers](https://github.com/mbecker20/monitor/blob/main/docs/servers.md)
3. [building](https://github.com/mbecker20/monitor/blob/main/docs/builds.md)
4. [deploying](https://github.com/mbecker20/monitor/blob/main/docs/deployments.md)
5. [permissioning](https://github.com/mbecker20/monitor/blob/main/docs/permissions.md)
6. [core setup](https://github.com/mbecker20/monitor/blob/main/docs/setup.md)
a tool to build and deploy software across many servers. [docs](https://mbecker20.github.io/monitor)