Gitea inconsistencies with act_runner when pods are scaled #11369

New Issue

GiteaMirror · 2025-11-02T09:35:47-06:00

GiteaMirror commented

2025-11-02 09:35:47 -06:00

Originally created by @ccureau on GitHub (Jul 30, 2023).

Description

Issue description

When a Kubernetes Pod is deleted, or when a Deployment, StatefulSet, or ReplicaSet is scaled down and up again, the pod name changes. This causes issues with the act_runner for logging and availability.

Issue #1: When the deployment is scaled down and then up, further jobs do not run on the pod.

This is a really weird one. After the pod connects to the runner again, the pod no longer receives jobs. When a new commit is made or a job is re-run, the job is sent to a completely different runner with labels that do not match.

The scenario looks like this:

There are two runners attached to Gitea. One has label rootless-22.04 and one has label runner-22.04. I'm pretty sure the labels don't really matter in this scenario, but adding for completeness.
The deployment for the label rootless-22.04 is scaled to 0, and then scaled back to 1.
Jobs with the runs-on: rootless-22.04 instruction now run on runner-22.04!

Issue #2: When the name is changed in .runner before act_runner is started, the runner shows that it is connected to Gitea, but Gitea does not update the runner name.

This issue might be corrected in Gitea by searching for the registration key on startup, updating the hostname in the database, and then marking the runner as available.

Steps to reproduce

set up a Kubernetes deployment and connect to gitea
scale the deployment to have zero replicas (kubectl scale --replicas=0 deployment/<deployment-name>)
scale the deployment up to one replica (kubectl scale --replicas=1 deployment/<deployment-name>)

A suitable Kubernetes deployment for the runner can be found here

Expected outcomes

Issue #1: I expect jobs that contain runs-on: specific-label to always run on specific-label, and if specific-label is not available, the job should block.

Issue #2: I expect the runner name to be consistent with the hostname, which will always be an issue for any Kubernetes type besides a Pod.

Gitea Version

1.20.1

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

No response

Git Version

No response

Operating System

No response

How are you running Gitea?

Kubernetes Server Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.6", GitCommit:"11902a838028edef305dfe2f96be929bc4d114d8", GitTreeState:"clean", BuildDate:"2023-06-14T09:49:08Z", GoVersion:"go1.19.10", Compiler:"gc", Platform:"linux/amd64"}

Gitea running in its own namespace, with external high-availablility Postgres backend

Database

PostgreSQL

Originally created by @ccureau on GitHub (Jul 30, 2023). ### Description ### Issue description When a Kubernetes `Pod` is deleted, or when a `Deployment`, `StatefulSet`, or `ReplicaSet` is scaled down and up again, the pod name changes. This causes issues with the act_runner for logging and availability. **Issue #1**: When the deployment is scaled down and then up, further jobs do not run on the pod. This is a really weird one. After the pod connects to the runner again, the pod no longer receives jobs. When a new commit is made or a job is re-run, the job is sent to a completely different runner with labels that do not match. The scenario looks like this: - There are two runners attached to Gitea. One has label `rootless-22.04` and one has label `runner-22.04`. I'm pretty sure the labels don't really matter in this scenario, but adding for completeness. - The deployment for the label `rootless-22.04` is scaled to 0, and then scaled back to 1. - Jobs with the `runs-on: rootless-22.04` instruction now run on `runner-22.04`! **Issue #2**: When the name is changed in `.runner` before `act_runner` is started, the runner shows that it is connected to Gitea, but Gitea does not update the runner name. This issue might be corrected in Gitea by searching for the registration key on startup, updating the hostname in the database, and then marking the runner as available. ### Steps to reproduce - set up a Kubernetes deployment and connect to gitea - scale the deployment to have zero replicas (`kubectl scale --replicas=0 deployment/<deployment-name>`) - scale the deployment up to one replica (`kubectl scale --replicas=1 deployment/<deployment-name>`) A suitable Kubernetes deployment for the runner can be found [here](https://gitea.com/gitea/act_runner/src/branch/main/examples/kubernetes/dind-docker.yaml) ### Expected outcomes **Issue #1**: I expect jobs that contain `runs-on: specific-label` to always run on `specific-label`, and if `specific-label` is not available, the job should block. **Issue #2**: I expect the runner name to be consistent with the hostname, which will always be an issue for any Kubernetes type besides a `Pod`. ### Gitea Version 1.20.1 ### Can you reproduce the bug on the Gitea demo site? No ### Log Gist _No response_ ### Screenshots _No response_ ### Git Version _No response_ ### Operating System _No response_ ### How are you running Gitea? Kubernetes Server Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.6", GitCommit:"11902a838028edef305dfe2f96be929bc4d114d8", GitTreeState:"clean", BuildDate:"2023-06-14T09:49:08Z", GoVersion:"go1.19.10", Compiler:"gc", Platform:"linux/amd64"} Gitea running in its own namespace, with external high-availablility Postgres backend ### Database PostgreSQL

GiteaMirror added the topic/gitea-actions type/bug labels 2025-11-02 09:35:47 -06:00

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/gitea#11369