github-starred/gitea
2
0
Fork 0
mirror of https://github.com/go-gitea/gitea.git synced 2026-07-15 20:09:18 -05:00
Code Issues 2.6k Packages Projects Releases 100 Wiki Activity

gitea hanging with git 2.43.1 #12471

New Issue
Closed
opened 2025-11-02 10:10:51 -06:00 by GiteaMirror · 11 comments
GiteaMirror commented 2025-11-02 10:10:51 -06:00
Owner
Copy Link

Originally created by @tycho on GitHub (Feb 12, 2024).

Description

I noticed that gitea was hanging when attempting to view commits (via /<team>/<repo>/commit/<hash>), but only with git 2.43.1 installed. "ps" showed a bunch of cat-file --batch-check, cat-file --batch, and check-attr invocations of git, which were making no progress. I attached strace to one of the git child processes and it was simply blocked on reading from stdin. I downgraded to git 2.43.0 and gitea started being able to view commits and index repositories.

I tried cranking the log level up to Debug and other than reporting "router: slow" messages for the commit URLs, it didn't seem to have any complaints, e.g. here's what happens on attempting to view a commit URL:

Feb 12 01:25:10 korriban gitea[14813]: 2024/02/12 01:25:10 ...dules/git/command.go:290:Run() [D] git.Command.RunDir(/var/lib/gitea/repos/organization/repo.git): /usr/bin/git -c protocol.version=2 -c credential.helper= read-tree ca9da37ae9c977b9ab06fe21d515b1cd9f1ba563
Feb 12 01:25:10 korriban gitea[14813]: 2024/02/12 01:25:10 ...dules/git/command.go:290:Run() [D] git.Command.RunDir(/var/lib/gitea/repos/organization/repo.git): /usr/bin/git -c protocol.version=2 -c credential.helper= check-attr --stdin -z --cached linguist-vendored linguist-generated linguist-language gitlab-language
Feb 12 01:25:13 korriban gitea[14813]: 2024/02/12 01:25:13 ...eb/routing/logger.go:68:func1() [I] router: polling   GET /user/events for [2604:4080:xxxx]:0, elapsed 3211.7ms @ events/events.go:18(events.Events)
Feb 12 01:25:14 korriban gitea[14813]: 2024/02/12 01:25:14 ...eb/routing/logger.go:68:func1() [W] router: slow      GET /Organization/Repo/commit/ca9da37ae9c977b9ab06fe21d515b1cd9f1ba563 for [2604:4080:xxxx]:0, elapsed 3675.9ms @ repo/commit.go:271(repo.Diff)

And again this only seems to happen with git 2.43.1 installed. Any idea what could be going on? Is this an upstream git regression?

Gitea Version

All

Git Version

2.43.1

Workaround

Downgrade git to <=2.43.0 or upgrade to >= 2.43.2

Originally created by @tycho on GitHub (Feb 12, 2024). ### Description I noticed that gitea was hanging when attempting to view commits (via `/<team>/<repo>/commit/<hash>`), but only with git 2.43.1 installed. "ps" showed a bunch of `cat-file --batch-check`, `cat-file --batch`, and `check-attr` invocations of git, which were making no progress. I attached strace to one of the git child processes and it was simply blocked on reading from stdin. I downgraded to git 2.43.0 and gitea started being able to view commits and index repositories. I tried cranking the log level up to Debug and other than reporting "router: slow" messages for the commit URLs, it didn't seem to have any complaints, e.g. here's what happens on attempting to view a commit URL: ``` Feb 12 01:25:10 korriban gitea[14813]: 2024/02/12 01:25:10 ...dules/git/command.go:290:Run() [D] git.Command.RunDir(/var/lib/gitea/repos/organization/repo.git): /usr/bin/git -c protocol.version=2 -c credential.helper= read-tree ca9da37ae9c977b9ab06fe21d515b1cd9f1ba563 Feb 12 01:25:10 korriban gitea[14813]: 2024/02/12 01:25:10 ...dules/git/command.go:290:Run() [D] git.Command.RunDir(/var/lib/gitea/repos/organization/repo.git): /usr/bin/git -c protocol.version=2 -c credential.helper= check-attr --stdin -z --cached linguist-vendored linguist-generated linguist-language gitlab-language Feb 12 01:25:13 korriban gitea[14813]: 2024/02/12 01:25:13 ...eb/routing/logger.go:68:func1() [I] router: polling GET /user/events for [2604:4080:xxxx]:0, elapsed 3211.7ms @ events/events.go:18(events.Events) Feb 12 01:25:14 korriban gitea[14813]: 2024/02/12 01:25:14 ...eb/routing/logger.go:68:func1() [W] router: slow GET /Organization/Repo/commit/ca9da37ae9c977b9ab06fe21d515b1cd9f1ba563 for [2604:4080:xxxx]:0, elapsed 3675.9ms @ repo/commit.go:271(repo.Diff) ``` And again this only seems to happen with git 2.43.1 installed. Any idea what could be going on? Is this an upstream git regression? ### Gitea Version All ### Git Version 2.43.1 ## Workaround Downgrade git to <=2.43.0 or upgrade to >= 2.43.2
GiteaMirror commented 2025-11-02 10:10:52 -06:00
Author
Owner
Copy Link

@Bodenhaltung commented on GitHub (Feb 12, 2024):

Same here but as DB sqlite3, updated Git to git 2.43.1-1 and running Gitea 1.21.4-1 (Gitea version 1.21.5 built with GNU Make 4.4.1, go1.21.6 : bindata, sqlite, sqlite_unlock_notify, pam).

Feb 12 11:36:37 srv01 gitea[1566129]: 2024/02/12 11:36:37 ...eb/routing/logger.go:68:func1() [W] router: slow      GET /**removed**/**removed**/commit/38c1d2267aaf3d03675b3700b06226d7e9c4e5ec for 127.0.0.1:0, elapsed 3046.2ms @ repo/commit.go:270(repo.Diff)
Feb 12 11:42:34 srv01 gitea[1566129]: 2024/02/12 11:42:34 ...it/repo_attribute.go:306:func2() [E] Unable to open checker for 38c1d2267aaf3d03675b3700b06226d7e9c4e5ec. Error: failed to run attr-check. Error: context deadline exceeded
Feb 12 11:42:34 srv01 gitea[1566129]: Stderr:
Feb 12 11:42:34 srv01 gitea[1566129]: 2024/02/12 11:42:34 ...eb/routing/logger.go:102:func1() [I] router: completed GET /**removed**/**removed**/commit/38c1d2267aaf3d03675b3700b06226d7e9c4e5ec for 127.0.0.1:0, 200 OK in 360178.3ms @ repo/commit.go:270(repo.Diff)
@Bodenhaltung commented on GitHub (Feb 12, 2024): Same here but as DB `sqlite3`, updated Git to `git 2.43.1-1` and running Gitea `1.21.4-1` (`Gitea version 1.21.5 built with GNU Make 4.4.1, go1.21.6 : bindata, sqlite, sqlite_unlock_notify, pam`). ``` Feb 12 11:36:37 srv01 gitea[1566129]: 2024/02/12 11:36:37 ...eb/routing/logger.go:68:func1() [W] router: slow GET /**removed**/**removed**/commit/38c1d2267aaf3d03675b3700b06226d7e9c4e5ec for 127.0.0.1:0, elapsed 3046.2ms @ repo/commit.go:270(repo.Diff) Feb 12 11:42:34 srv01 gitea[1566129]: 2024/02/12 11:42:34 ...it/repo_attribute.go:306:func2() [E] Unable to open checker for 38c1d2267aaf3d03675b3700b06226d7e9c4e5ec. Error: failed to run attr-check. Error: context deadline exceeded Feb 12 11:42:34 srv01 gitea[1566129]: Stderr: Feb 12 11:42:34 srv01 gitea[1566129]: 2024/02/12 11:42:34 ...eb/routing/logger.go:102:func1() [I] router: completed GET /**removed**/**removed**/commit/38c1d2267aaf3d03675b3700b06226d7e9c4e5ec for 127.0.0.1:0, 200 OK in 360178.3ms @ repo/commit.go:270(repo.Diff) ```
GiteaMirror commented 2025-11-02 10:10:53 -06:00
Author
Owner
Copy Link

@wxiaoguang commented on GitHub (Feb 12, 2024):

I think it is a regression bug of Git. I have sent an email to the git mail list, quote it below.

(the only workaround at the moment is to downgrade the git to <=2.43.0 , well, I am not sure whether we should propose a dirty patch to bypass the regression).

Incorrect GIT_FLUSH behavior in 2.43.1 (regression and breaking)

Ref: 556e68032f

If GIT_FLUSH=true, it should mean to "do the flush". But that commit made skip_stdout_flush=true when GIT_FLUSH=true.

And by the way, only accepting GIT_FLUSH=true is quite breaking, it drops the compatibility of GIT_FLUSH=1: it causes the existing programs which depend on the "flushing(GIT_FLUSH=1)" behavior would hang forever if they use the new git binary, because the program won't see any flushed output .... (not a problem if the filpping bug could be fixed)

@wxiaoguang commented on GitHub (Feb 12, 2024): I think it is a regression bug of Git. I have sent an email to the git mail list, quote it below. (the only workaround at the moment is to downgrade the git to <=2.43.0 , well, I am not sure whether we should propose a dirty patch to bypass the regression). ---- Incorrect GIT_FLUSH behavior in 2.43.1 (regression and breaking) Ref: https://github.com/git/git/commit/556e68032f8248c831e48207e5cb923c9fe0e42c If GIT_FLUSH=true, it should mean to "do the flush". But that commit made skip_stdout_flush=true when GIT_FLUSH=true. ~~And by the way, only accepting GIT_FLUSH=true is quite breaking, it drops the compatibility of GIT_FLUSH=1: it causes the existing programs which depend on the "flushing(GIT_FLUSH=1)" behavior would hang forever if they use the new git binary, because the program won't see any flushed output ....~~ (not a problem if the filpping bug could be fixed)
GiteaMirror commented 2025-11-02 10:10:53 -06:00
Author
Owner
Copy Link

@tycho commented on GitHub (Feb 12, 2024):

I am not sure whether we should propose a dirty patch to bypass the regression

Probably shouldn't, as long as the git team resolves it quickly with a revert/patch and new release.

But is there an opportunity for better logging on the gitea side for this issue? Even with Debug level logging there wasn't much to report aside from the "router: slow" messages, which were at least somewhat helpful in figuring it out. But maybe there should be something more here?

@tycho commented on GitHub (Feb 12, 2024): > I am not sure whether we should propose a dirty patch to bypass the regression Probably shouldn't, as long as the git team resolves it quickly with a revert/patch and new release. But is there an opportunity for better logging on the gitea side for this issue? Even with Debug level logging there wasn't much to report aside from the "router: slow" messages, which were at least somewhat helpful in figuring it out. But maybe there should be something more here?
GiteaMirror commented 2025-11-02 10:10:53 -06:00
Author
Owner
Copy Link

@wxiaoguang commented on GitHub (Feb 13, 2024):

But is there an opportunity for better logging on the gitea side for this issue?

IMO there is no general/easy way to do the log for such hanging at the moment (feel free to propose some approaches), because the problem is that Gitea couldn't read the git program's output. Actually, there are many clues, eg: the "running process" (in the admin panel), the log about "failed to run attr-check". The real difficult step is how to figure out why it hangs.

Probably shouldn't, as long as the git team resolves it quickly with a revert/patch and new release.

Maybe we could do this: Refactor git version functions and check compatibility #29155

Tell end users to upgrade or downgrade the git if there is some known issues.

@wxiaoguang commented on GitHub (Feb 13, 2024): > But is there an opportunity for better logging on the gitea side for this issue? IMO there is no general/easy way to do the log for such hanging at the moment (feel free to propose some approaches), because the problem is that Gitea couldn't read the git program's output. Actually, there are many clues, eg: the "running process" (in the admin panel), the log about "failed to run attr-check". The real difficult step is how to figure out why it hangs. > Probably shouldn't, as long as the git team resolves it quickly with a revert/patch and new release. Maybe we could do this: Refactor git version functions and check compatibility #29155 Tell end users to upgrade or downgrade the git if there is some known issues.
GiteaMirror commented 2025-11-02 10:10:53 -06:00
Author
Owner
Copy Link

@VitaliyAT commented on GitHub (Feb 15, 2024):

I have the same problems with git version 2.43.0.windows.1

@VitaliyAT commented on GitHub (Feb 15, 2024): I have the same problems with git version 2.43.0.windows.1
GiteaMirror commented 2025-11-02 10:10:53 -06:00
Author
Owner
Copy Link

@milosivanovic commented on GitHub (Feb 16, 2024):

@tycho @wxiaoguang thanks for figuring this out - the new hangs were baffling me. I didn't immediately use gitea after updating git from 2.43.0 to 2.43.1, so I didn't correlate updating git with the hangs that occurred while trying to view diffs until I found this issue.

I am running gitea behind an nginx reverse proxy, and after clicking a commit to view a diff, nginx returned HTTP 504 after gitea failed to respond within 60s. @wxiaoguang It would be useful if gitea could be configured to time out the git command after a shorter time so that it could return a meaningful error for users who are still trying to figure this issue out. Would that be possible?

@milosivanovic commented on GitHub (Feb 16, 2024): @tycho @wxiaoguang thanks for figuring this out - the new hangs were baffling me. I didn't immediately use gitea after updating git from 2.43.0 to 2.43.1, so I didn't correlate updating git with the hangs that occurred while trying to view diffs until I found this issue. I am running gitea behind an nginx reverse proxy, and after clicking a commit to view a diff, nginx returned HTTP 504 after gitea failed to respond within 60s. @wxiaoguang It would be useful if gitea could be configured to time out the git command after a shorter time so that it could return a meaningful error for users who are still trying to figure this issue out. Would that be possible?
GiteaMirror commented 2025-11-02 10:10:53 -06:00
Author
Owner
Copy Link

@wxiaoguang commented on GitHub (Feb 16, 2024):

I am running gitea behind an nginx reverse proxy, and after clicking a commit to view a diff, nginx returned HTTP 504 after gitea failed to respond within 60s. @wxiaoguang It would be useful if gitea could be configured to time out the git command after a shorter time so that it could return a meaningful error for users who are still trying to figure this issue out. Would that be possible?

Actually there is one: [git.timeout].DEFAULT 6d4dc16c72/custom/conf/app.example.ini (L708C2-L708C15)

But I think you shouldn't touch it ..... because it's difficult to fine-tune a proper timeout for various cases at the moment.

@wxiaoguang commented on GitHub (Feb 16, 2024): > I am running gitea behind an nginx reverse proxy, and after clicking a commit to view a diff, nginx returned HTTP 504 after gitea failed to respond within 60s. @wxiaoguang It would be useful if gitea could be configured to time out the git command after a shorter time so that it could return a meaningful error for users who are still trying to figure this issue out. Would that be possible? Actually there is one: `[git.timeout].DEFAULT` https://github.com/go-gitea/gitea/blob/6d4dc16c726dd0be8d0f56405ba396d44dfd04ac/custom/conf/app.example.ini#L708C2-L708C15 But I think you shouldn't touch it ..... because it's difficult to fine-tune a proper timeout for various cases at the moment.
GiteaMirror commented 2025-11-02 10:10:54 -06:00
Author
Owner
Copy Link

@wxiaoguang commented on GitHub (Feb 16, 2024):

I have the same problems with git version 2.43.0.windows.1

Does an old version (say , git 2.42.x) work for you? If no, then it could be a different problem (not the GIT_FLUSH regression)

@wxiaoguang commented on GitHub (Feb 16, 2024): > I have the same problems with git version 2.43.0.windows.1 Does an old version (say , git 2.42.x) work for you? If no, then it could be a different problem (not the GIT_FLUSH regression)
GiteaMirror commented 2025-11-02 10:10:54 -06:00
Author
Owner
Copy Link

@tycho commented on GitHub (Feb 16, 2024):

Also 2.43.2 is out, which fixes this bug. Just have to wait for vendors to build binary packages for it.

@tycho commented on GitHub (Feb 16, 2024): Also 2.43.2 is out, which fixes this bug. Just have to wait for vendors to build binary packages for it.
GiteaMirror commented 2025-11-02 10:10:54 -06:00
Author
Owner
Copy Link

@silverwind commented on GitHub (Feb 16, 2024):

That, and all our fixes have merged as well, so let's close.

@silverwind commented on GitHub (Feb 16, 2024): That, and all our fixes have merged as well, so let's close.
GiteaMirror commented 2025-11-02 10:10:54 -06:00
Author
Owner
Copy Link

@github-actions[bot] commented on GitHub (Feb 28, 2024):

Automatically locked because of our CONTRIBUTING guidelines

@github-actions[bot] commented on GitHub (Feb 28, 2024): Automatically locked because of our [CONTRIBUTING guidelines](https://github.com/go-gitea/gitea/blob/main/CONTRIBUTING.md#issue-locking)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
$20
$250
$50
$500
backport/done
💎 Bounty
docs-update-needed
good first issue
hacktoberfest
issue/bounty
issue/confirmed
issue/critical
issue/duplicate
issue/needs-feedback
issue/not-a-bug
issue/regression
issue/stale
issue/workaround
lgtm/need 2
modifies/api
modifies/translation
outdated/backport/v1.18
outdated/theme/markdown
outdated/theme/timetracker
performance/bigrepo
performance/cpu
performance/memory
performance/speed
pr/breaking
proposal/accepted
proposal/rejected
pr/wip
pull-request
Mirrored from GitHub Pull Request
 reviewed/wontfix
💰 Rewarded
skip-changelog
status/blocked
topic/accessibility
topic/api
topic/authentication
topic/build
topic/code-linting
topic/commit-signing
topic/content-rendering
topic/deployment
topic/distribution
topic/federation
topic/gitea-actions
topic/issues
topic/lfs
topic/mobile
topic/moderation
topic/packages
topic/pr
topic/projects
topic/repo
topic/repo-migration
topic/security
topic/theme
topic/ui
topic/ui-interaction
topic/ux
topic/webhooks
topic/wiki
type/bug
type/deprecation
type/docs
type/enhancement
type/feature
type/miscellaneous
type/proposal
type/question
type/refactoring
type/summary
type/testing
type/upstream
No Label issue/confirmed issue/critical issue/workaround type/bug type/upstream
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/gitea#12471
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?