Improve Config Management/Stateless Runner Deploy Workflows #10513

@cthu1hoo commented on GitHub (Mar 27, 2023):

I too ran into this problem while writing my ansible role for runner deployment, reusable tokens (similar to the way gitlab does it) sound like the best approach to me.

@cthu1hoo commented on GitHub (Mar 27, 2023): I too ran into this problem while writing my ansible role for runner deployment, reusable tokens (similar to the way gitlab does it) sound like the best approach to me.

GiteaMirror commented

@benyanke commented on GitHub (Mar 27, 2023):

Exactly where I ran into it as well - writing an idiomatic ansible role was almost impossible as-is.

@benyanke commented on GitHub (Mar 27, 2023): Exactly where I ran into it as well - writing an idiomatic ansible role was almost impossible as-is.

GiteaMirror commented

@garymoon commented on GitHub (May 22, 2023):

Woodpecker and Drone accept a symmetic secret via environment variables for both the application and the runners, making automated provisioning zero-touch. Could such a flow be considered for Gitea?

@garymoon commented on GitHub (May 22, 2023): [Woodpecker](https://woodpecker-ci.org/docs/administration/setup) and [Drone](https://docs.drone.io/runner/docker/installation/linux/) accept a symmetic secret via environment variables for both the application and the runners, making automated provisioning zero-touch. Could such a flow be considered for Gitea?

GiteaMirror commented

@sebthom commented on GitHub (May 30, 2023):

I don't really understand why a separate token mechanism was implemented for runner registration in the first place. I would suggest to remove it and instead introduce a new scope into the gitea access tokens that allows registering runners, e.g. admin:register_runner or register:runner? This reusable token could then be feed into the act_runner for self-registration. Nice side effect, the token management UI is already there. You see when the token was last used and can easily revoke it.

@sebthom commented on GitHub (May 30, 2023): I don't really understand why a separate token mechanism was implemented for runner registration in the first place. I would suggest to remove it and instead introduce a new scope into the gitea access tokens that allows registering runners, e.g. `admin:register_runner` or `register:runner`? This reusable token could then be feed into the act_runner for self-registration. Nice side effect, the token management UI is already there. You see when the token was last used and can easily revoke it.

GiteaMirror commented

@sillyguodong commented on GitHub (May 31, 2023):

I would suggest to remove it and instead introduce a new scope into the gitea access tokens that allows registering runners

If you only use PAT to register runners, how do Gitea know the scope level (org, repo, global) of the runners.

@sillyguodong commented on GitHub (May 31, 2023): > I would suggest to remove it and instead introduce a new scope into the gitea access tokens that allows registering runners If you only use PAT to register runners, how do Gitea know the scope level (org, repo, global) of the runners.

GiteaMirror commented

@sebthom commented on GitHub (May 31, 2023):

@sillyguodong True, then maybe the Gitea token can be extended to be "fine grained". https://github.blog/2022-10-18-introducing-fine-grained-personal-access-tokens-for-github/ instead of adding/maintaining a parallel token infrastructure only for the purpose of runner registration tokens.

@sebthom commented on GitHub (May 31, 2023): @sillyguodong True, then maybe the Gitea token can be extended to be "fine grained". https://github.blog/2022-10-18-introducing-fine-grained-personal-access-tokens-for-github/ instead of adding/maintaining a parallel token infrastructure only for the purpose of runner registration tokens.

GiteaMirror commented

@nodiscc commented on GitHub (Oct 19, 2023):

deploy runners with config management

This role can deploy act-runner and register it automatically on a gitea instance. In particular have a look at the registration part [1].

@nodiscc commented on GitHub (Oct 19, 2023): > deploy runners with config management [This role](https://github.com/nodiscc/xsrv/tree/master/roles/gitea_act_runner) can deploy act-runner and register it automatically on a gitea instance. In particular have a look at the registration part [[1]](https://github.com/nodiscc/xsrv/blob/master/roles/gitea_act_runner/tasks/act-runner.yml#L93).

GiteaMirror commented

@jtran commented on GitHub (Feb 16, 2024):

@sillyguodong True, then maybe the Gitea token can be extended to be "fine grained". https://github.blog/2022-10-18-introducing-fine-grained-personal-access-tokens-for-github/ instead of adding/maintaining a parallel token infrastructure only for the purpose of runner registration tokens.

Wasn't this done in https://github.com/go-gitea/gitea/pull/24767?

@jtran commented on GitHub (Feb 16, 2024): > @sillyguodong True, then maybe the Gitea token can be extended to be "fine grained". https://github.blog/2022-10-18-introducing-fine-grained-personal-access-tokens-for-github/ instead of adding/maintaining a parallel token infrastructure only for the purpose of runner registration tokens. Wasn't this done in https://github.com/go-gitea/gitea/pull/24767?

GiteaMirror commented

@lunny commented on GitHub (Mar 27, 2024):

I think this has been resolved by #23762 and #27143

@lunny commented on GitHub (Mar 27, 2024): I think this has been resolved by #23762 and #27143

GiteaMirror commented

@KyleGalvin commented on GitHub (Apr 4, 2024):

How does this solve the issue in a containerized environment, where we usually just have a single docker command/entrypoint?

It would be really nice to have exactly what @garymoon suggests: inject the secret at startup from the environment. That way the deployment can give both gitea and the runner the same shared key and they can authenticate automatically after that.

No command line. No side channel communication. No post-deploy setup. Just give both the key.

@KyleGalvin commented on GitHub (Apr 4, 2024): How does this solve the issue in a containerized environment, where we usually just have a single docker command/entrypoint? It would be really nice to have exactly what @garymoon suggests: inject the secret at startup from the environment. That way the deployment can give both gitea and the runner the same shared key and they can authenticate automatically after that. No command line. No side channel communication. No post-deploy setup. Just give both the key.

GiteaMirror commented

@KyleGalvin commented on GitHub (Jun 20, 2024):

I was searching for a solution to this, and found myself landing on this same thread 2 months down the line.

I have hacks and workarounds for the time being, but I want to describe my use-case to demonstrate how a shared environment variable between gitea and runners for initial setup can help:

I'm building a helm chart that includes gitea and act runners, and I want to automate the setup between the two.

I believe I could make a sidecar job that connects them using the API, but frankly this approach is more convoluted than I think it should be, and I'm using the following workaround hoping that a better solution comes down the pipeline.

Currently, I install the chart with an empty environment variable for the runner token.

I then wait for gitea to install and generate its own runner token. At this point, the runners are failing due to no valid token.

I go into the UI and copy that token out, pasting it into my chart values.yaml

I then update my helm chart with the new values, at which point the runners stop thrashing and connect.

This process isnt overly complicated for someone who knows how the setup works under the hood, but it would not be easy to point to my helm chart and tell another dev how to setup my environment.

@KyleGalvin commented on GitHub (Jun 20, 2024): I was searching for a solution to this, and found myself landing on this same thread 2 months down the line. I have hacks and workarounds for the time being, but I want to describe my use-case to demonstrate how a shared environment variable between gitea and runners for initial setup can help: I'm building a helm chart that includes gitea and act runners, and I want to automate the setup between the two. I believe I could make a sidecar job that connects them using the API, but frankly this approach is more convoluted than I think it should be, and I'm using the following workaround hoping that a better solution comes down the pipeline. Currently, I install the chart with an empty environment variable for the runner token. I then wait for gitea to install and generate its own runner token. At this point, the runners are failing due to no valid token. I go into the UI and copy that token out, pasting it into my chart values.yaml I then update my helm chart with the new values, at which point the runners stop thrashing and connect. This process isnt overly complicated for someone who knows how the setup works under the hood, but it would not be easy to point to my helm chart and tell another dev how to setup my environment.

GiteaMirror commented

@AlexMikhalev commented on GitHub (Dec 17, 2024):

Since it's the second time I've hit the same issue with Gitea and it's a show-stopper for using it, I found that fork of gitea supports setting up runners via shared secrets of the box:
https://forgejo.org/docs/latest/admin/runner-installation/#offline-registration
Example of docker compose: https://code.forgejo.org/forgejo/runner/src/branch/main/examples/docker-compose/README.md
Example of Kubernetes: https://code.forgejo.org/forgejo/runner/src/branch/main/examples/kubernetes

@AlexMikhalev commented on GitHub (Dec 17, 2024): Since it's the second time I've hit the same issue with Gitea and it's a show-stopper for using it, I found that fork of gitea supports setting up runners via shared secrets of the box: https://forgejo.org/docs/latest/admin/runner-installation/#offline-registration Example of docker compose: https://code.forgejo.org/forgejo/runner/src/branch/main/examples/docker-compose/README.md Example of Kubernetes: https://code.forgejo.org/forgejo/runner/src/branch/main/examples/kubernetes

GiteaMirror commented

@lunny commented on GitHub (Dec 17, 2024):

Since it's the second time I've hit the same issue with Gitea and it's a show-stopper for using it, I found that fork of gitea supports setting up runners via shared secrets of the box: forgejo.org/docs/latest/admin/runner-installation#offline-registration Example of docker compose: code.forgejo.org/forgejo/runner/src/branch/main/examples/docker-compose/README.md Example of Kubernetes: code.forgejo.org/forgejo/runner/src/branch/main/examples/kubernetes

I think this issue has been resolved by

You can find these examples in Gitea's act_runner repository.
https://gitea.com/gitea/act_runner/src/branch/main/examples

All of these features in the fork are copied from this repository except the shared secrets feature.

@lunny commented on GitHub (Dec 17, 2024): > Since it's the second time I've hit the same issue with Gitea and it's a show-stopper for using it, I found that fork of gitea supports setting up runners via shared secrets of the box: [forgejo.org/docs/latest/admin/runner-installation#offline-registration](https://forgejo.org/docs/latest/admin/runner-installation/#offline-registration) Example of docker compose: [code.forgejo.org/forgejo/runner/src/branch/main/examples/docker-compose/README.md](https://code.forgejo.org/forgejo/runner/src/branch/main/examples/docker-compose/README.md) Example of Kubernetes: [code.forgejo.org/forgejo/runner/src/branch/main/examples/kubernetes](https://code.forgejo.org/forgejo/runner/src/branch/main/examples/kubernetes) I think this issue has been resolved by - https://github.com/go-gitea/gitea/pull/23762 - https://github.com/go-gitea/gitea/pull/27144 - https://github.com/go-gitea/gitea/pull/27143 You can find these examples in Gitea's act_runner repository. https://gitea.com/gitea/act_runner/src/branch/main/examples All of these features in the fork are copied from this repository except the shared secrets feature.

GiteaMirror commented

@lunny commented on GitHub (Dec 17, 2024):

I propose adding an additional parameter to both the Gitea CLI command and the Gitea API for registering runners. This enhancement would allow users to seamlessly configure predefined secrets for deployment on both the Gitea server and the runner sides.

By reusing the existing command-line tools and API endpoints, we can maintain consistency while extending functionality, ensuring a streamlined and efficient process for managing secrets.

@lunny commented on GitHub (Dec 17, 2024): I propose adding an additional parameter to both the Gitea CLI command and the Gitea API for registering runners. This enhancement would allow users to seamlessly configure predefined secrets for deployment on both the Gitea server and the runner sides. By reusing the existing command-line tools and API endpoints, we can maintain consistency while extending functionality, ensuring a streamlined and efficient process for managing secrets.

GiteaMirror commented

@lunny commented on GitHub (Dec 17, 2024):

I created the pull request #32878 and will add some tests and docs. Please help to review.

@lunny commented on GitHub (Dec 17, 2024): I created the pull request #32878 and will add some tests and docs. Please help to review.

GiteaMirror commented

@AlexMikhalev commented on GitHub (Dec 17, 2024):

Thank you. How can I test it?

@AlexMikhalev commented on GitHub (Dec 17, 2024): Thank you. How can I test it?

GiteaMirror commented

@wxiaoguang commented on GitHub (Dec 22, 2024):

The proper fix could be like this: Use env GITEA_RUNNER_REGISTRATION_TOKEN as global runner token #32946

Now, we could just set GITEA_RUNNER_REGISTRATION_TOKEN or GITEA_RUNNER_REGISTRATION_TOKEN_FILE (the same as runner) to make Gitea server pre-init the runner token. Then everything should work out-of-box.

(I think we will have in 1.23 stable)

@wxiaoguang commented on GitHub (Dec 22, 2024): The proper fix could be like this: Use env GITEA_RUNNER_REGISTRATION_TOKEN as global runner token #32946 Now, we could just set `GITEA_RUNNER_REGISTRATION_TOKEN` or `GITEA_RUNNER_REGISTRATION_TOKEN_FILE` (the same as runner) to make Gitea server pre-init the runner token. Then everything should work out-of-box. (I think we will have in 1.23 stable)

GiteaMirror commented