Enable cron.git_gc_repos by default to avoid mirror syncing errors #10547

New Issue

GiteaMirror · 2025-11-02T09:10:56-06:00

GiteaMirror commented

2025-11-02 09:10:56 -06:00

Originally created by @pboguslawski on GitHub (Mar 29, 2023).

Description

Just noticed https://github.com/nextcloud/contacts mirror sync error:

2023/03/29 08:08:31 ...irror/mirror_pull.go:279:runSync() [E] [6423d5db-14] SyncMirrors [repo: 38:nc/nextcloud-contacts]: unable to write commit-graph for '/myreporoot/nc/nextcloud-contacts.git' : exit status 128 - error: Could not read 544c4227b061eb5331617430a3896872fd3915ab
2023-03-29    08:08:31    gitea[123]: error: Could not read a3a65b5a36972ed4c50d461e9fe6127a66c2ea05
2023-03-29    08:08:31    gitea[123]: error: Could not read 681e22178c7cefeb3033740f2cbb93312a615fc9
2023-03-29    08:08:31    gitea[123]: error: Could not read 24df40d4cf028ba93ff68f7230e059dce1374c2f
2023-03-29    08:08:31    gitea[123]: error: Could not read 82fb27f04da07758971fff6f9d99d30e21f61295
2023-03-29    08:08:31    gitea[123]: error: Could not read 46ae973a9d8c6abf9b07da4ce0a27014c275f459
[...]
2023-03-29    08:08:31    gitea[123]: error: Could not read e4b86229e61575ab45542e6d337ddf3892a78f4f
2023-03-29    08:08:31    gitea[123]: error: Could not read 03a21dd37f6f01da2d637f789baa4066271d1080
2023-03-29    08:08:31    gitea[123]: fatal: unable to parse commit 03a21dd37f6f01da2d637f789baa4066271d1080
2023-03-29    08:08:31    gitea[123]: - error: Could not read 544c4227b061eb5331617430a3896872fd3915ab
2023-03-29    08:08:31    gitea[123]: error: Could not read a3a65b5a36972ed4c50d461e9fe6127a66c2ea05
2023-03-29    08:08:31    gitea[123]: error: Could not read 681e22178c7cefeb3033740f2cbb93312a615fc9
[...]
2023-03-29    08:08:31    gitea[123]: fatal: unable to parse commit 03a21dd37f6f01da2d637f789baa4066271d1080
2023-03-29    08:08:31    gitea[123]:

Mirror was created many months ago and worked fine (everyday sync) till today. Manual resync throws same error in this mirror.

Commit 544c4227b061eb5331617430a3896872fd3915ab mentioned in errors is present in github but not in mirror (looks like orphaned on github).

Fresh https://github.com/nextcloud/contacts mirror setup on other system with same gitea+git versions does not throw such error on sync.

Gitea Version

v1.18.3

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

No response

Git Version

2.30.2

Operating System

No response

How are you running Gitea?

Compiled from sources.

Database

MySQL

Originally created by @pboguslawski on GitHub (Mar 29, 2023). ### Description Just noticed https://github.com/nextcloud/contacts mirror sync error: ``` 2023/03/29 08:08:31 ...irror/mirror_pull.go:279:runSync() [E] [6423d5db-14] SyncMirrors [repo: 38:nc/nextcloud-contacts]: unable to write commit-graph for '/myreporoot/nc/nextcloud-contacts.git' : exit status 128 - error: Could not read 544c4227b061eb5331617430a3896872fd3915ab 2023-03-29 08:08:31 gitea[123]: error: Could not read a3a65b5a36972ed4c50d461e9fe6127a66c2ea05 2023-03-29 08:08:31 gitea[123]: error: Could not read 681e22178c7cefeb3033740f2cbb93312a615fc9 2023-03-29 08:08:31 gitea[123]: error: Could not read 24df40d4cf028ba93ff68f7230e059dce1374c2f 2023-03-29 08:08:31 gitea[123]: error: Could not read 82fb27f04da07758971fff6f9d99d30e21f61295 2023-03-29 08:08:31 gitea[123]: error: Could not read 46ae973a9d8c6abf9b07da4ce0a27014c275f459 [...] 2023-03-29 08:08:31 gitea[123]: error: Could not read e4b86229e61575ab45542e6d337ddf3892a78f4f 2023-03-29 08:08:31 gitea[123]: error: Could not read 03a21dd37f6f01da2d637f789baa4066271d1080 2023-03-29 08:08:31 gitea[123]: fatal: unable to parse commit 03a21dd37f6f01da2d637f789baa4066271d1080 2023-03-29 08:08:31 gitea[123]: - error: Could not read 544c4227b061eb5331617430a3896872fd3915ab 2023-03-29 08:08:31 gitea[123]: error: Could not read a3a65b5a36972ed4c50d461e9fe6127a66c2ea05 2023-03-29 08:08:31 gitea[123]: error: Could not read 681e22178c7cefeb3033740f2cbb93312a615fc9 [...] 2023-03-29 08:08:31 gitea[123]: fatal: unable to parse commit 03a21dd37f6f01da2d637f789baa4066271d1080 2023-03-29 08:08:31 gitea[123]: ``` Mirror was created many months ago and worked fine (everyday sync) till today. Manual resync throws same error in this mirror. Commit [544c4227b061eb5331617430a3896872fd3915ab](https://github.com/nextcloud/contacts/commit/544c4227b061eb5331617430a3896872fd3915ab) mentioned in errors is present in github but not in mirror (looks like orphaned on github). Fresh https://github.com/nextcloud/contacts mirror setup on other system with same gitea+git versions does not throw such error on sync. ### Gitea Version v1.18.3 ### Can you reproduce the bug on the Gitea demo site? No ### Log Gist _No response_ ### Screenshots _No response_ ### Git Version 2.30.2 ### Operating System _No response_ ### How are you running Gitea? Compiled from sources. ### Database MySQL

GiteaMirror added the type/proposal type/bug labels 2025-11-02 09:10:56 -06:00

GiteaMirror commented

2025-11-02 09:10:57 -06:00

@wxiaoguang commented on GitHub (Mar 29, 2023):

A quick google search says that git fsck or git gc might help. It looks like some git internal data corrupted (or out of sync)

@wxiaoguang commented on GitHub (Mar 29, 2023): A quick google search says that `git fsck` or `git gc` might help. It looks like some git internal data corrupted (or out of sync)

GiteaMirror commented

2025-11-02 09:10:57 -06:00

@pboguslawski commented on GitHub (Mar 30, 2023):

Running Garbage collect all repositories manually from Monitoring resolved the issue, thank you.

git gc is fired automatically only during some operations, so probably is not fired on mirror update and that caused the problem. Please consider enabling cron.git_gc_repos by default to avoid such surprises. gc and fsck jobs should probably have longer timeout by default (1800s maybe?) to handle bigger repos on slower hardware by default.

@pboguslawski commented on GitHub (Mar 30, 2023): Running `Garbage collect all repositories` manually from `Monitoring` resolved the issue, thank you. `git gc` [is fired automatically only during some operations](https://git-scm.com/docs/git-gc), so probably is not fired on mirror update and that caused the problem. Please consider enabling [`cron.git_gc_repos`](https://github.com/go-gitea/gitea/blob/main/custom/conf/app.example.ini#L2134) by default to avoid such surprises. gc and fsck jobs should probably have longer timeout by default (`1800s` maybe?) to handle bigger repos on slower hardware by default.

GiteaMirror commented

2025-11-02 09:10:58 -06:00

@lunny commented on GitHub (Mar 30, 2023):

Running Garbage collect all repositories manually from Monitoring resolved the issue, thank you.

git gc is fired automatically only during some operations, so probably is not fired on mirror update and that caused the problem. Please consider enabling cron.git_gc_repos by default to avoid such surprises. gc and fsck jobs should probably have longer timeout by default (1800s maybe?) to handle bigger repos on slower hardware by default.

Or maybe we can change the issue's title to proposal a default value change?

@lunny commented on GitHub (Mar 30, 2023): > Running `Garbage collect all repositories` manually from `Monitoring` resolved the issue, thank you. > > `git gc` [is fired automatically only during some operations](https://git-scm.com/docs/git-gc), so probably is not fired on mirror update and that caused the problem. Please consider enabling [`cron.git_gc_repos`](https://github.com/go-gitea/gitea/blob/main/custom/conf/app.example.ini#L2134) by default to avoid such surprises. gc and fsck jobs should probably have longer timeout by default (`1800s` maybe?) to handle bigger repos on slower hardware by default. Or maybe we can change the issue's title to proposal a default value change?

GiteaMirror commented

2025-11-02 09:10:58 -06:00

@pboguslawski commented on GitHub (Mar 30, 2023):

Done.

@pboguslawski commented on GitHub (Mar 30, 2023): Done.

GiteaMirror commented

2025-11-02 09:10:58 -06:00

@adlternative commented on GitHub (Apr 5, 2023):

Concurrent use of git gc and some git commands may lead to data corruption. I don't know if it is related to what is happening here: https://lore.kernel.org/git/CAOLTT8Tt3jW2yvm6BRU3yG+EvW1WG9wWFq6PuOcaHNNLQAaGjg@mail.gmail.com/

@adlternative commented on GitHub (Apr 5, 2023): Concurrent use of git gc and some git commands may lead to data corruption. I don't know if it is related to what is happening here: https://lore.kernel.org/git/CAOLTT8Tt3jW2yvm6BRU3yG+EvW1WG9wWFq6PuOcaHNNLQAaGjg@mail.gmail.com/

GiteaMirror commented

2025-11-02 09:10:58 -06:00

@pboguslawski commented on GitHub (Apr 5, 2023):

Concurrent use of git gc and some git commands may lead to data corruption.

To avoid problems consider:

adding warning to cron.git_gc_repos docs about corruption possibility with link to mentioned page,
introducing checkbox in every repo config to explicitly allow cron.git_gc_repos to process it (similar to existing Enable Repository Health Checks (git fsck) checkbox).

With such checkboxes admin may enable cron.git_gc_repos only for repos they consider as non-critical and safe (i.e. read only mirrors).

@pboguslawski commented on GitHub (Apr 5, 2023): > Concurrent use of git gc and some git commands may lead to data corruption. To avoid problems consider: * adding warning to `cron.git_gc_repos` docs about corruption possibility with link to [mentioned page](https://lore.kernel.org/git/CAOLTT8Tt3jW2yvm6BRU3yG+EvW1WG9wWFq6PuOcaHNNLQAaGjg@mail.gmail.com/), * introducing checkbox in every repo config to explicitly allow `cron.git_gc_repos` to process it (similar to existing `Enable Repository Health Checks (git fsck)` checkbox). With such checkboxes admin may enable `cron.git_gc_repos` only for repos they consider as non-critical and safe (i.e. read only mirrors).

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/gitea#10547