Cron: Update Mirrors cancelled: dial tcp {EXTERNAL IP ADRESS}:3306: connect: connection timed out #11936

New Issue

GiteaMirror · 2025-11-02T09:52:14-06:00

GiteaMirror commented

2025-11-02 09:52:14 -06:00

Originally created by @blade1989 on GitHub (Oct 30, 2023).

Description

The connection to the external database server often fails with a connection timed out message.

I have a MariaDB Galera Cluster with MaxScale. And In the MariaDB logs I can't seem to find the fault.

I read somewhere that the issue could be the max_allowed_packet which might be set too low. I set it to 1073741824 (1024 mebibytes), but no luck.

I have been trying to fix this for a while now, My System Notices are in total 938, with most of them being the connection time out message.

Gitea Version

1.20.5

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

No response

Git Version

built in

Operating System

Ubuntu 22.04.3 LTS

How are you running Gitea?

Docker rootless

Database

MySQL/MariaDB

Originally created by @blade1989 on GitHub (Oct 30, 2023). ### Description The connection to the external database server often fails with a `connection timed out` message. I have a MariaDB Galera Cluster with MaxScale. And In the MariaDB logs I can't seem to find the fault. I read somewhere that the issue could be the `max_allowed_packet` which might be set too low. I set it to `1073741824` (1024 mebibytes), but no luck. I have been trying to fix this for a while now, My System Notices are in total 938, with most of them being the connection time out message. ### Gitea Version 1.20.5 ### Can you reproduce the bug on the Gitea demo site? No ### Log Gist _No response_ ### Screenshots _No response_ ### Git Version built in ### Operating System Ubuntu 22.04.3 LTS ### How are you running Gitea? Docker rootless ### Database MySQL/MariaDB

GiteaMirror added the issue/needs-feedback type/bug labels 2025-11-02 09:52:14 -06:00

GiteaMirror closed this issue

2025-11-02 09:52:14 -06:00

GiteaMirror commented

2025-11-02 09:52:15 -06:00

@lunny commented on GitHub (Nov 1, 2023):

Please paste your logs here.

@lunny commented on GitHub (Nov 1, 2023): Please paste your logs here.

GiteaMirror commented

2025-11-02 09:52:15 -06:00

@blade1989 commented on GitHub (Nov 4, 2023):

@lunny

From Gitea System Notices:

...
Cron: Synchronize external user data cancelled: dial tcp {EXTERNAL MariaDB IP}:3306: connect: connection timed out
...
Cron: Update Mirrors cancelled: dial tcp {EXTERNAL MariaDB IP}:3306: connect: connection timed out
...

Gitea was found 369 times in the MariaDB Slow Logs

# Time: 230928 21:46:38
# User@Host: {gitea db user}[{gitea db user}] @ maxscale.{domain.tld} [{db IP}]
# Thread_id: 563559  Schema: gitea_db  QC_hit: No
# Query_time: 1.098845  Lock_time: 0.000054  Rows_sent: 0  Rows_examined: 1
# Rows_affected: 1  Bytes_sent: 0
#
# explain: id   select_type     table   type    possible_keys   key     key_len ref     rows    r_rows  filtered        r_filtered      Extra
# explain: 1    SIMPLE  action_runner   range   PRIMARY PRIMARY 8       NULL    1       1.00    100.00  100.00  Using where
#
SET timestamp=1695930398;
UPDATE `action_runner` SET `last_online` = 1695930396, `updated` = 1695930396 WHERE `id`=7 AND (`deleted`=0 OR `deleted` IS NULL);

...
...
...

# User@Host: {gitea db user}[{gitea db user}] @ maxscale.{domain.tld} [{db IP}]
# Thread_id: 507125  Schema: gitea_db  QC_hit: No
# Query_time: 3.353802  Lock_time: 0.003429  Rows_sent: 0  Rows_examined: 1
# Rows_affected: 1  Bytes_sent: 0
#
# explain: id   select_type     table   type    possible_keys   key     key_len ref     rows    r_rows  filtered        r_filtered      Extra
# explain: 1    SIMPLE  action_runner   range   PRIMARY PRIMARY 8       NULL    1       1.00    100.00  100.00  Using where
#
SET timestamp=1695813917;
UPDATE `action_runner` SET `last_online` = 1695813914, `updated` = 1695813914 WHERE `id`=7 AND (`deleted`=0 OR `deleted` IS NULL);

docker-compose logs:

gitea  | 2023/11/04 03:41:39 ...eb/routing/logger.go:102:func1() [I] router: completed POST /api/actions/runner.v1.RunnerService/FetchTask for 172.19.0.1:48940, 500 Internal Server Error in 30.6ms @ <autogenerated>:1(http.Handler.ServeHTTP-fm)

@blade1989 commented on GitHub (Nov 4, 2023): @lunny From Gitea System Notices: ```bash ... Cron: Synchronize external user data cancelled: dial tcp {EXTERNAL MariaDB IP}:3306: connect: connection timed out ... Cron: Update Mirrors cancelled: dial tcp {EXTERNAL MariaDB IP}:3306: connect: connection timed out ... ``` Gitea was found 369 times in the MariaDB Slow Logs ```bash # Time: 230928 21:46:38 # User@Host: {gitea db user}[{gitea db user}] @ maxscale.{domain.tld} [{db IP}] # Thread_id: 563559 Schema: gitea_db QC_hit: No # Query_time: 1.098845 Lock_time: 0.000054 Rows_sent: 0 Rows_examined: 1 # Rows_affected: 1 Bytes_sent: 0 # # explain: id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra # explain: 1 SIMPLE action_runner range PRIMARY PRIMARY 8 NULL 1 1.00 100.00 100.00 Using where # SET timestamp=1695930398; UPDATE `action_runner` SET `last_online` = 1695930396, `updated` = 1695930396 WHERE `id`=7 AND (`deleted`=0 OR `deleted` IS NULL); ... ... ... # User@Host: {gitea db user}[{gitea db user}] @ maxscale.{domain.tld} [{db IP}] # Thread_id: 507125 Schema: gitea_db QC_hit: No # Query_time: 3.353802 Lock_time: 0.003429 Rows_sent: 0 Rows_examined: 1 # Rows_affected: 1 Bytes_sent: 0 # # explain: id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra # explain: 1 SIMPLE action_runner range PRIMARY PRIMARY 8 NULL 1 1.00 100.00 100.00 Using where # SET timestamp=1695813917; UPDATE `action_runner` SET `last_online` = 1695813914, `updated` = 1695813914 WHERE `id`=7 AND (`deleted`=0 OR `deleted` IS NULL); ``` docker-compose logs: ```bash gitea | 2023/11/04 03:41:39 ...eb/routing/logger.go:102:func1() [I] router: completed POST /api/actions/runner.v1.RunnerService/FetchTask for 172.19.0.1:48940, 500 Internal Server Error in 30.6ms @ <autogenerated>:1(http.Handler.ServeHTTP-fm) ```

GiteaMirror commented

2025-11-02 09:52:15 -06:00

@lunny commented on GitHub (Nov 4, 2023):

How many records on your action_runner table?

@lunny commented on GitHub (Nov 4, 2023): How many records on your `action_runner` table?

GiteaMirror commented

2025-11-02 09:52:16 -06:00

@blade1989 commented on GitHub (Nov 5, 2023):

@lunny The are 9 records in the action_runner table.

The rest of my actions table count:

table	count
action	163095
action_artifact	0
action_run	84
action_run_index	6
action_run_job	84
action_runner	9
action_runner_token	20
action_task	85
action_task_output	0
action_task_step	483

@blade1989 commented on GitHub (Nov 5, 2023): @lunny The are 9 records in the `action_runner` table. The rest of my actions table count: | table | count | |----------|----------| | action | 163095 | | action_artifact | 0 | | action_run | 84 | | action_run_index | 6 | | action_run_job | 84 | | action_runner | 9 | | action_runner_token | 20 | | action_task | 85 | | action_task_output | 0 | | action_task_step | 483 |

GiteaMirror commented

2025-11-02 09:52:16 -06:00

@blade1989 commented on GitHub (Nov 13, 2023):

@lunny do you need any more information?

@blade1989 commented on GitHub (Nov 13, 2023): @lunny do you need any more information?

GiteaMirror commented

2025-11-02 09:52:16 -06:00

@lunny commented on GitHub (Nov 14, 2023):

Looks like it's normal.

What's your database version?

@lunny commented on GitHub (Nov 14, 2023): Looks like it's normal. What's your database version?

GiteaMirror commented

2025-11-02 09:52:16 -06:00

@blade1989 commented on GitHub (Nov 14, 2023):

@lunny the mariadb --version output:

mariadb from 11.0.3-MariaDB, client 15.2 for debian-linux-gnu (x86_64) using  EditLine wrapper

@blade1989 commented on GitHub (Nov 14, 2023): @lunny the `mariadb --version` output: ```bash mariadb from 11.0.3-MariaDB, client 15.2 for debian-linux-gnu (x86_64) using EditLine wrapper ```

GiteaMirror commented

2025-11-02 09:52:17 -06:00

@blade1989 commented on GitHub (Nov 27, 2023):

@lunny any luck on figuring this out?

@blade1989 commented on GitHub (Nov 27, 2023): @lunny any luck on figuring this out?

GiteaMirror commented

2025-11-02 09:52:17 -06:00

@lunny commented on GitHub (Nov 27, 2023):

I think these timeout messages are because it's too slow visiting from gitea to database? Maybe you can login to gitea container and have a test to visit the database.

@lunny commented on GitHub (Nov 27, 2023): I think these timeout messages are because it's too slow visiting from gitea to database? Maybe you can login to gitea container and have a test to visit the database.

GiteaMirror commented

2025-11-02 09:52:17 -06:00

@lunny commented on GitHub (Jan 19, 2024):

I think this is a network problem or mysql side problem but not related with Gitea.

@lunny commented on GitHub (Jan 19, 2024): I think this is a network problem or mysql side problem but not related with Gitea.

GiteaMirror commented

2025-11-02 09:52:17 -06:00

@GiteaBot commented on GitHub (Feb 18, 2024):

We close issues that need feedback from the author if there were no new comments for a month. 🍵

@GiteaBot commented on GitHub (Feb 18, 2024): We close issues that need feedback from the author if there were no new comments for a month. :tea:

GiteaMirror commented

2025-11-02 09:52:17 -06:00

@github-actions[bot] commented on GitHub (Feb 29, 2024):

Automatically locked because of our CONTRIBUTING guidelines

@github-actions[bot] commented on GitHub (Feb 29, 2024): Automatically locked because of our [CONTRIBUTING guidelines](https://github.com/go-gitea/gitea/blob/main/CONTRIBUTING.md#issue-locking)

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/gitea#11936