mirror of
https://github.com/moghtech/komodo.git
synced 2026-03-09 07:13:36 -05:00
[Feature] Procedure filters #26
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @C0untZero on GitHub (Aug 23, 2024).
At the moment creating a procedure to, for instance, deploy or redeploy all stacks on a server is tedious - you have to specifically add a DeployStack action for each stack on the server. And this is not limited to stacks only, any procedure action with a target resource is the same.
Based on our discussion on Discord, filters could help with this. The idea is to create one action with a filter target which could then filter on any resource property, like name, server, tag, image registry, git provider/account etc. The target dropdown should have a use custom filter option at the top which when pressed shows a query building window. This window should have a list of properties we can filter on for the relevant resource type (based on selected action). Each property to filter on would support exact name matching, simple wildcard search, or advanced regex matching. An example toml for a filter would be any of the three options below:
Once a filter is created and the query window is closed, the target column for the configured row should show the filter query instead of a specific resource name:

Another nice to have feature would also be to show resource names that matched the query when the filter is hovered over, or maybe in the query config window at the bottom or something.
A simple implementation for this would be wildcard matching multiple resources based on one property.
A more advanced implementation could be built from this to support regex as well as matching on multiple properties using logic operators AND, OR, NOT, etc. Something akin to JQL - KQL (Komodo Query Language) maybe? ;)
@mbecker20 commented on GitHub (Aug 23, 2024):
Thanks for the write up C0unt!
It's going to be a great feature and I'm really excited by the possibilities here. I'm not sure how I will include it in the upcoming versions at this point, maybe hesitantly in 1.15. For now I will leave it as an enhancement.
🦎
@mbecker20 commented on GitHub (Aug 23, 2024):
Regex support at the library level will be using the already included regex crate.
In terms of "Wildcard" syntax, we can use the cloudflare standard handled by their wilcard crate.
@mbecker20 commented on GitHub (Oct 28, 2024):
1.16.5 gets most of the way there. It doesn't allow you to match on anything other than the resource name though, and currently doesn't have any preview of which resources are matched.
@C0untZero commented on GitHub (Oct 28, 2024):
Looking good, thanks for the progress on this.
I'm not sure about the decision of introducing duplicate actions with batch in name. Kinda crowds the action list imo.
Is this just temporary until the full query builder interface is built?
@mbecker20 commented on GitHub (Oct 28, 2024):
Thanks for the quick feedback. There is a reason it was done this way, I can try to explain.
There is another issue #93 which request to be able to make executions on more than one Stacks / Containers by selecting multiple in the table. This is best accomplished by adding ability to eg
DeployStack, but target more than one Stack in the call.Looking at the API for DeployStack, you see it returns a single Update. This is an issue, because a batch call to this API will under the hood call
DeployStackon the individual stacks, each will produce anUpdate.If you call to
DeployStacktargeting multiple Stacks, the return type of the data should actually be something likeVec<Update>, a list of all of the Updates for each Stacks deploy. So either the return type ofDeployStackneeds to change, or it needs another api. I thinkDeployStack->Updateis still a good API to expose and want to keep it as it is, soBatchDeployStackand the otherBatchexecutions were created.They all have this return type: BatchExecutionResponse. It is basically
Vec<Update>but with some error handling, each underlyingUpdatein the batch call may instead be some Error case which needs to be returned to user.With this
Batchexecution handling setup, the frontend can use it to handle the batch executions when user select multiple resources in a table, and alsoActionscan call this API directly.For
Procedures, I felt it was clearer to make this matching behavior explicit at API method level. This way,Procedurescontinue to map 1-to-1 with the behavior of the API when called outside of Procedures. Ie, it won't work to call theDeployStackapi directly passingarr-*as thestack, but using that withBatchDeployStackwill work. Procedures just follow this standard, users should use theBatchapis if they intend to do a batch execution targetting multiple underlying resources.The only downside I can see is the crowding of action list. But there is a search feature there, its not too bad when you start typing
deploy.@C0untZero commented on GitHub (Oct 29, 2024):
Ok, now that I understand the backend part it makes sense. The search feature does make it easy though, so I guess it's all good.
@oblivioncth commented on GitHub (Jun 25, 2025):
This somewhat it's own issue, but overlaps with this an some others:
It would be nice to have a way to exclude stacks from the batch actions, or any procedure really, altogether. For example, I use
BatchPullStack+BatchDeployStackin one procedure, andBatchDeployStackIfChangedin another to blanket update all of my containers; however, I have a couple stacks that I generally want to stay stopped as I only spin them up manually on occasion and then stop them when I'm done using them.As things stand, unless I add some kind of static identifier to the name of all the stacks I want to update, like "-update-" (ugly), then those few stacks I want to stay stopped will be deployed every time the procedure runs and I have to go out of my way to keep stopping them.
@samumatic commented on GitHub (Jun 26, 2025):
This is the same for me, i was trying to get the stacks excluded via regex (
^(?!.*\btemplate\b).*) but no success there. Especially with the new "Templates" feature in the newest release theBatchDeployStackIfChangedtries to start my template stack because of my*regex.I'm using renovate to update my compose files in GitHub and then
BatchDeployStackIfChangedevery night to update my stacks on the servers.@oblivioncth commented on GitHub (Jun 26, 2025):
@samumatic
Temporary workaround until this is hopefully a built-in feature:
Add this as an Action, then add a Variable called
ADHOC_STACKS(or whatever you want and change it in the script) to your instance, containing a whitespace delimited list of stacks you don't want deployed.Then just run this action in your procedure instead of BatchDeployStackIfChanged. You can also swap the call in the script to simply
DeployStackif you wanted to use that for routine/unconditional redeploys also.I also use renovate, as well as do weekly redeploys for the stacks with "latest" tags, hence why the variable, so it can be used in both actions.
I'd like to use a variable with newlines for this, but it causes a minor bug with syncs if you use those, so for now I'm just using plain spaces to separate the name (you could change the script to use another delimiter if you like as well): https://github.com/moghtech/komodo/issues/627
@oblivioncth commented on GitHub (Jun 27, 2025):
It's in the linked issue, but for clarity I just wanted to add this here:
@mbecker20 indirectly suggested using tags for this, which I do think can be a lot better depending on your preference. It looks like this:
Just make the matching tag and tag your stacks with it.
@shadybraden commented on GitHub (Aug 29, 2025):
This works, but the issue I am having now is that everytime a
DeployStackIfChangedruns my CPU pins for ~15 seconds as it runsdeployon every container (except ad-hoc) even if there are no changes.Any clues on how to debug this?
@mbecker20 commented on GitHub (Aug 29, 2025):
@shadybraden
Is it definitely every time?
I should have mentioned this in the notes on this. The first deploy after you make a configuration change affecting the files, they still use old deploy cache files. So immediately this triggers deploy on all stacks, because there is Komodo cache diff.
After the first deploy, the caches will be updated, and subsequent deploy if changed will respect only doing configured executions with actual diffs to the files.
@shadybraden commented on GitHub (Aug 30, 2025):
Yup. Every time I run the following procedure or action:
Here's a quick video of what I'm seeing:
https://github.com/user-attachments/assets/76706045-ea14-4bd4-bb38-04a4e266dae0
@mbecker20 commented on GitHub (Aug 31, 2025):
@shadybraden I think it would help you figure out what is going on if you break it down and make sure it works one stack at a time. For example, have webhook only trigger DeployStackIfChanged on a single Stack, and try committing to files that are associated with that stack, and ones that are not associated with that stack. You should find that commits to files not associated with that stack do not trigger deploy.
@shadybraden commented on GitHub (Aug 31, 2025):
Hmm. I did a bunch more troubleshooting, and got nowhere. Then I stepped away and came back, and it began working exactly as expected...So perhaps it was a cache issue...time to take a backup