github-starred/gitea
2
0
Fork 0
mirror of https://github.com/go-gitea/gitea.git synced 2026-03-22 14:34:54 -05:00
Code Issues 2.6k Packages Projects Releases 100 Wiki Activity

Some existing and all new PRs for a given repository are broken #12807

New Issue
Open
opened 2025-11-02 10:21:36 -06:00 by GiteaMirror · 13 comments
GiteaMirror commented 2025-11-02 10:21:36 -06:00
Owner
Copy Link

Originally created by @Chippit on GitHub (Apr 9, 2024).

Description

One of our Gitea repositories has found itself in a strange state where:

  • New PRs cannot be opened. The task fails with a 500 Internal Server Error when submitting the new PR.
  • Some (but not all) pre-existing PRs display the following message (I think these are ones that have had new commits pushed to them since whatever event happened that caused the repository to enter this state.):

This pull request is broken due to missing fork information.

  • At least one of the non-broken PRs was still able to be merged successfully despite this state.

Some supplementary information based on a minor investigation I did:

  • We can still open and continue working on PRs in other repositories.
  • We can still pull, push and clone the broken repository in question.
  • Other repositories that have had PRs contain several subdirectories within their refs/pull directory.
  • The failing repository contains a refs/pull directory but it is empty.
    • PR Refs still appear in info/refs
    • I have attempted to remove the PR entries and the empty refs/pull directory with no change in behaviour.
  • Repositories that have never had a PR do not seem to have a refs/pull.

The attached logs were captured in trace mode and are all entries printed around when a new PR was opened (and the user receives a 500 error)

Gitea Version

1.21.1

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

https://gist.github.com/Chippit/be5a1616958d74a4aa61954bd520e32b

Screenshots

No response

Git Version

git version 2.25.1

Operating System

Ubuntu 20.04.6 LTS

How are you running Gitea?

Our instance is deployed from binary distributions retrieved via github, and is managed via a Systemd unit, on virtualised hardware.

Database

MySQL/MariaDB

Originally created by @Chippit on GitHub (Apr 9, 2024). ### Description One of our Gitea repositories has found itself in a strange state where: - New PRs cannot be opened. The task fails with a 500 Internal Server Error when submitting the new PR. - Some (but not all) pre-existing PRs display the following message (I _think_ these are ones that have had new commits pushed to them since whatever event happened that caused the repository to enter this state.): > This pull request is broken due to missing fork information. - At least one of the non-broken PRs was still able to be merged successfully despite this state. Some supplementary information based on a minor investigation I did: - We can still open and continue working on PRs in other repositories. - We can still pull, push and clone the broken repository in question. - Other repositories that have had PRs contain several subdirectories within their `refs/pull` directory. - The failing repository contains a `refs/pull` directory but it is empty. - PR Refs still appear in `info/refs` - I have attempted to remove the PR entries and the empty `refs/pull` directory with no change in behaviour. - Repositories that have never had a PR do not seem to have a `refs/pull`. The attached logs were captured in trace mode and are all entries printed around when a new PR was opened (and the user receives a 500 error) ### Gitea Version 1.21.1 ### Can you reproduce the bug on the Gitea demo site? No ### Log Gist https://gist.github.com/Chippit/be5a1616958d74a4aa61954bd520e32b ### Screenshots _No response_ ### Git Version git version 2.25.1 ### Operating System Ubuntu 20.04.6 LTS ### How are you running Gitea? Our instance is deployed from binary distributions retrieved via github, and is managed via a Systemd unit, on virtualised hardware. ### Database MySQL/MariaDB
GiteaMirror added the type/bug label 2025-11-02 10:21:36 -06:00
GiteaMirror commented 2025-11-02 10:21:37 -06:00
Author
Owner
Copy Link

@lunny commented on GitHub (Apr 9, 2024):

Push failed because of some reason. Can you check whether the repository is normal? /var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git

Apr 09 12:20:55 <redacted system hostname> gitea[2767615]: 2024/04/09 12:20:55 ...ervices/pull/pull.go:507:pushToBaseRepoHelper() [E] Unable to push PR head for <redacted>/<redacted project name>#20 (<Repository 60:<redacted>/<redacted project name>>:refs/pull/20/head) due to Error: push failed: exit status 1 - Invalid remote name "/var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git": Invalid remote name: "/var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git"
Apr 09 12:20:55 <redacted system hostname> gitea[2767615]: error: failed to push some refs to '/var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git'
Apr 09 12:20:55 <redacted system hostname> gitea[2767615]:  - Invalid remote name "/var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git": Invalid remote name: "/var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git"
Apr 09 12:20:55 <redacted system hostname> gitea[2767615]: error: failed to push some refs to '/var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git'
Apr 09 12:20:55 <redacted system hostname> gitea[2767615]:  - Invalid remote name "/var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git": Invalid remote name: "/var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git"
Apr 09 12:20:55 <redacted system hostname> gitea[2767615]: error: failed to push some refs to '/var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git'
@lunny commented on GitHub (Apr 9, 2024): Push failed because of some reason. Can you check whether the repository is normal? `/var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git` ```log Apr 09 12:20:55 <redacted system hostname> gitea[2767615]: 2024/04/09 12:20:55 ...ervices/pull/pull.go:507:pushToBaseRepoHelper() [E] Unable to push PR head for <redacted>/<redacted project name>#20 (<Repository 60:<redacted>/<redacted project name>>:refs/pull/20/head) due to Error: push failed: exit status 1 - Invalid remote name "/var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git": Invalid remote name: "/var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git" Apr 09 12:20:55 <redacted system hostname> gitea[2767615]: error: failed to push some refs to '/var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git' Apr 09 12:20:55 <redacted system hostname> gitea[2767615]: - Invalid remote name "/var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git": Invalid remote name: "/var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git" Apr 09 12:20:55 <redacted system hostname> gitea[2767615]: error: failed to push some refs to '/var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git' Apr 09 12:20:55 <redacted system hostname> gitea[2767615]: - Invalid remote name "/var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git": Invalid remote name: "/var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git" Apr 09 12:20:55 <redacted system hostname> gitea[2767615]: error: failed to push some refs to '/var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git' ```
GiteaMirror commented 2025-11-02 10:21:38 -06:00
Author
Owner
Copy Link

@Chippit commented on GitHub (Apr 9, 2024):

Yeah, I don't see anything wrong with it. Pushing and pulling from clients works fine too. All the expected bare repo files are intact at that path, and I can execute stuff like git log from within it and it works as expected. File permissions are all okay, as well.

Is there anything else I should check?

@Chippit commented on GitHub (Apr 9, 2024): Yeah, I don't see anything wrong with it. Pushing and pulling from clients works fine too. All the expected bare repo files are intact at that path, and I can execute stuff like `git log` from within it and it works as expected. File permissions are all okay, as well. Is there anything else I should check?
GiteaMirror commented 2025-11-02 10:21:38 -06:00
Author
Owner
Copy Link

@lunny commented on GitHub (Apr 9, 2024):

Maybe you can check the permission of refs/pull/*.*.

@lunny commented on GitHub (Apr 9, 2024): Maybe you can check the permission of `refs/pull/*.*`.
GiteaMirror commented 2025-11-02 10:21:38 -06:00
Author
Owner
Copy Link

@Chippit commented on GitHub (Apr 9, 2024):

Unfortunately, as I mentioned in my description above, refs/pull is completely empty. (Though corresponding items still existed in info/refs)

@Chippit commented on GitHub (Apr 9, 2024): Unfortunately, as I mentioned in my description above, `refs/pull` is completely empty. (Though corresponding items still existed in `info/refs`)
GiteaMirror commented 2025-11-02 10:21:38 -06:00
Author
Owner
Copy Link

@lunny commented on GitHub (Apr 9, 2024):

So maybe you can check the permission of refs/pull Maybe the dirs/files cannot be created under that directory.

@lunny commented on GitHub (Apr 9, 2024): So maybe you can check the permission of `refs/pull` Maybe the dirs/files cannot be created under that directory.
GiteaMirror commented 2025-11-02 10:21:38 -06:00
Author
Owner
Copy Link

@Chippit commented on GitHub (Apr 10, 2024):

Permissions all seem fine:

$ echo $USER
gitea
$ pwd
/var/lib/gitea/data/gitea-repositories/<redacted>/<redacted-project-name>.git/refs/pull
$ ls -la
total 8
drwxr-xr-x 2 gitea gitea 4096 Apr  9 14:06 .
drwxr-xr-x 5 gitea gitea 4096 Apr  9 13:34 ..
$ touch test
$ ls -la
total 8
drwxr-xr-x 2 gitea gitea 4096 Apr 10 09:07 .
drwxr-xr-x 5 gitea gitea 4096 Apr  9 13:34 ..
-rw-rw-r-- 1 gitea gitea    0 Apr 10 09:07 test

It also seems like this issue has now extended to another repository as well. That one still has all its prior entries in refs/pull but now presents this new issue. Some existing PRs there are broken, and no new ones can be made, with the same error in the log and the same 500 error in the frontend.

@Chippit commented on GitHub (Apr 10, 2024): Permissions all seem fine: ``` $ echo $USER gitea $ pwd /var/lib/gitea/data/gitea-repositories/<redacted>/<redacted-project-name>.git/refs/pull $ ls -la total 8 drwxr-xr-x 2 gitea gitea 4096 Apr 9 14:06 . drwxr-xr-x 5 gitea gitea 4096 Apr 9 13:34 .. $ touch test $ ls -la total 8 drwxr-xr-x 2 gitea gitea 4096 Apr 10 09:07 . drwxr-xr-x 5 gitea gitea 4096 Apr 9 13:34 .. -rw-rw-r-- 1 gitea gitea 0 Apr 10 09:07 test ``` It also seems like this issue has now extended to another repository as well. That one still has all its prior entries in `refs/pull` but now presents this new issue. Some existing PRs there are broken, and no new ones can be made, with the same error in the log and the same 500 error in the frontend.
GiteaMirror commented 2025-11-02 10:21:39 -06:00
Author
Owner
Copy Link

@lunny commented on GitHub (Apr 10, 2024):

How does your Gitea instance run? With docker or systemd?

@lunny commented on GitHub (Apr 10, 2024): How does your Gitea instance run? With docker or systemd?
GiteaMirror commented 2025-11-02 10:21:39 -06:00
Author
Owner
Copy Link

@Chippit commented on GitHub (Apr 10, 2024):

As mentioned in the ticket, we are running with systemd.

I think I have solved the problem however.

I tried running the command from the trace log interactively. It fails the same way.

$ git push --dry-run -- /var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git mb-fix-inputfield-nullref:refs/pull/20/head
Invalid remote name "/var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git": Invalid remote name: "/var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git"

However, executing it with --no-verify worked fine.

$ git push --no-verify --dry-run -- /var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git mb-fix-inputfield-nullref:refs/pull/20/head
To /var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git
 * [new branch]      mb-fix-inputfield-nullref -> refs/pull/20/head

I went to check and the failing repos all contained git-lfs hook files. These are not present in other repos.

$ ls
applypatch-msg.sample      post-checkout  post-receive        pre-applypatch.sample    prepare-commit-msg.sample  pre-rebase.sample  pre-receive.sample  update.sample
commit-msg.sample          post-commit    post-receive.d      pre-commit.sample        pre-push                   pre-receive        update
fsmonitor-watchman.sample  post-merge     post-update.sample  pre-merge-commit.sample  pre-push.sample            pre-receive.d      update.d

After removing them, it seems the PRs once again work correctly.

$ mv post-checkout post-checkout.bak
$ mv post-commit post-commit.bak
$ mv post-merge post-merge.bak
$ mv pre-push pre-push.bak

This ticket and this one made me a bit suspicious of the hooks. While I did try reset all the hooks from the admin interface when this issue first came up, as prompted by the first issue, I didn't scrutinize the ones present inside the broken repos particularly. (I'm not that familiar with which ones are expected and which ones aren't so I'm not sure I'd have spotted anything wrong at first glance anyway). But it seems the lfs hooks were causing the pushes to fail.

After doing this, we are once again able to open new ones. And previously broken ones work again after new commits are pushed to their branches.

It does raise the question, though: how did these get here? Are they SUPPOSED to be there? We have been using this gitea instance for about a year now with no issues, until yesterday, so something must have triggered this behaviour.

Our system runs the latest available git-lfs from the debian/ubuntu repositories, but it does seem like there is a much newer version of git lfs available via packagecloud repos. Should we install those instead?

$ git lfs --version
git-lfs/2.9.2 (GitHub; linux amd64; go 1.13.5)
@Chippit commented on GitHub (Apr 10, 2024): As mentioned in the ticket, we are running with systemd. I think I have solved the problem however. I tried running the command from the trace log interactively. It fails the same way. ``` $ git push --dry-run -- /var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git mb-fix-inputfield-nullref:refs/pull/20/head Invalid remote name "/var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git": Invalid remote name: "/var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git" ``` However, executing it with `--no-verify` worked fine. ``` $ git push --no-verify --dry-run -- /var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git mb-fix-inputfield-nullref:refs/pull/20/head To /var/lib/gitea/data/gitea-repositories/<redacted>/<redacted project name>.git * [new branch] mb-fix-inputfield-nullref -> refs/pull/20/head ``` I went to check and the failing repos all contained git-lfs hook files. These are not present in other repos. ``` $ ls applypatch-msg.sample post-checkout post-receive pre-applypatch.sample prepare-commit-msg.sample pre-rebase.sample pre-receive.sample update.sample commit-msg.sample post-commit post-receive.d pre-commit.sample pre-push pre-receive update fsmonitor-watchman.sample post-merge post-update.sample pre-merge-commit.sample pre-push.sample pre-receive.d update.d ``` After removing them, it seems the PRs once again work correctly. ``` $ mv post-checkout post-checkout.bak $ mv post-commit post-commit.bak $ mv post-merge post-merge.bak $ mv pre-push pre-push.bak ``` [This ticket](https://github.com/go-gitea/gitea/issues/11105#issuecomment-615228895) and [this one](https://github.com/git-lfs/git-lfs/issues/4065) made me a bit suspicious of the hooks. While I did try reset all the hooks from the admin interface when this issue first came up, as prompted by the first issue, I didn't scrutinize the ones present inside the broken repos particularly. (I'm not that familiar with which ones are expected and which ones aren't so I'm not sure I'd have spotted anything wrong at first glance anyway). But it seems the lfs hooks were causing the pushes to fail. After doing this, we are once again able to open new ones. And previously broken ones work again after new commits are pushed to their branches. --- It does raise the question, though: how did these get here? Are they SUPPOSED to be there? We have been using this gitea instance for about a year now with no issues, until yesterday, so _something_ must have triggered this behaviour. Our system runs the latest available git-lfs from the debian/ubuntu repositories, but it does seem like there is a much newer version of git lfs available via packagecloud repos. Should we install those instead? ``` $ git lfs --version git-lfs/2.9.2 (GitHub; linux amd64; go 1.13.5) ```
GiteaMirror commented 2025-11-02 10:21:39 -06:00
Author
Owner
Copy Link

@lunny commented on GitHub (Apr 11, 2024):

From the design of Gitea, those hooks should not be there.

@lunny commented on GitHub (Apr 11, 2024): From the design of Gitea, those hooks should not be there.
GiteaMirror commented 2025-11-02 10:21:39 -06:00
Author
Owner
Copy Link

@Chippit commented on GitHub (Apr 11, 2024):

Okay, well it seems this is a valid resolution then.

While I am concerned about how this happened in the first place, and it would be nice to know how to ensure that it doesn't happen again both for us and for other users, I suppose it's okay to close this issue. Thank you for your help.

@Chippit commented on GitHub (Apr 11, 2024): Okay, well it seems this is a valid resolution then. While I am concerned about how this happened in the first place, and it would be nice to know how to ensure that it doesn't happen again both for us and for other users, I suppose it's okay to close this issue. Thank you for your help.
GiteaMirror commented 2025-11-02 10:21:39 -06:00
Author
Owner
Copy Link

@Chippit commented on GitHub (Apr 12, 2024):

Okay, I have figured out how to reproduce this.

If I use this feature to download an archive of a repository as a zip file, then those hooks are recreated in the repo. Afterwards, this issue recurs.
image

@Chippit commented on GitHub (Apr 12, 2024): Okay, I have figured out how to reproduce this. If I use this feature to download an archive of a repository as a zip file, then those hooks are recreated in the repo. Afterwards, this issue recurs. ![image](https://github.com/go-gitea/gitea/assets/870779/daa93dcf-4653-4df2-a80a-ede2cb595986)
GiteaMirror commented 2025-11-02 10:21:40 -06:00
Author
Owner
Copy Link

@lenianiva commented on GitHub (Aug 4, 2024):

I have a similar problem, but the hooks mentioned above don't exist:

$ ls repositories/<repo-name>/hooks
applypatch-msg.sample	   post-receive.d	  pre-merge-commit.sample    pre-receive	 proc-receive.d		    update.d
commit-msg.sample	   post-update.sample	  prepare-commit-msg.sample  pre-receive.d	 push-to-checkout.sample    update.sample
fsmonitor-watchman.sample  pre-applypatch.sample  pre-push.sample	     pre-receive.sample  sendemail-validate.sample
post-receive		   pre-commit.sample	  pre-rebase.sample	     proc-receive	 update
@lenianiva commented on GitHub (Aug 4, 2024): I have a similar problem, but the hooks mentioned above don't exist: ``` $ ls repositories/<repo-name>/hooks applypatch-msg.sample post-receive.d pre-merge-commit.sample pre-receive proc-receive.d update.d commit-msg.sample post-update.sample prepare-commit-msg.sample pre-receive.d push-to-checkout.sample update.sample fsmonitor-watchman.sample pre-applypatch.sample pre-push.sample pre-receive.sample sendemail-validate.sample post-receive pre-commit.sample pre-rebase.sample proc-receive update ```
GiteaMirror commented 2025-11-02 10:21:40 -06:00
Author
Owner
Copy Link

@lunny commented on GitHub (Aug 29, 2024):

Maybe fixed by #31931

@lunny commented on GitHub (Aug 29, 2024): Maybe fixed by #31931
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
$20
$250
$50
$500
backport/done
💎 Bounty
docs-update-needed
good first issue
hacktoberfest
issue/bounty
issue/confirmed
issue/critical
issue/duplicate
issue/needs-feedback
issue/not-a-bug
issue/regression
issue/stale
issue/workaround
lgtm/need 2
modifies/api
modifies/translation
outdated/backport/v1.18
outdated/theme/markdown
outdated/theme/timetracker
performance/bigrepo
performance/cpu
performance/memory
performance/speed
pr/breaking
proposal/accepted
proposal/rejected
pr/wip
pull-request
Mirrored from GitHub Pull Request
 reviewed/wontfix
💰 Rewarded
skip-changelog
status/blocked
topic/accessibility
topic/api
topic/authentication
topic/build
topic/code-linting
topic/commit-signing
topic/content-rendering
topic/deployment
topic/distribution
topic/federation
topic/gitea-actions
topic/issues
topic/lfs
topic/mobile
topic/moderation
topic/packages
topic/pr
topic/projects
topic/repo
topic/repo-migration
topic/security
topic/theme
topic/ui
topic/ui-interaction
topic/ux
topic/webhooks
topic/wiki
type/bug
type/deprecation
type/docs
type/enhancement
type/feature
type/miscellaneous
type/proposal
type/question
type/refactoring
type/summary
type/testing
type/upstream
No Label type/bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/gitea#12807
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?