github-starred/gitea
2
0
Fork 0
mirror of https://github.com/go-gitea/gitea.git synced 2026-03-22 06:24:14 -05:00
Code Issues 2.6k Packages Projects Releases 100 Wiki Activity

Git LFS Push 500 Internal Server Error #10631

New Issue
Closed
opened 2025-11-02 09:13:32 -06:00 by GiteaMirror · 5 comments
GiteaMirror commented 2025-11-02 09:13:32 -06:00
Owner
Copy Link

Originally created by @Shinmera on GitHub (Apr 11, 2023).

Description

When attempting to push new LFS artefacts, the internal logs only show error messages like so, with the last error repeating over and over:

Apr 11 13:10:40 imai gitea[556860]: 2023/04/11 13:10:40 [64354030] router: completed GET /api/internal/serv/command/10/shinmera/trial-deployment?mode=2&verb=git-receive-pack for 31.164.105.196:0, 200 OK in 2.4ms @ private/serv.go:81(private.ServCommand)
Apr 11 13:10:41 imai gitea[556860]: 2023/04/11 13:10:41 [64354031] router: completed POST /api/internal/ssh/log for 31.164.105.196:0, 200 OK in 0.1ms @ private/ssh_log.go:18(private.SSHLog)
Apr 11 13:10:41 imai gitea[556860]: 2023/04/11 13:10:41 [64354031-2] router: completed GET /api/internal/serv/command/10/shinmera/trial-deployment?mode=1&verb=git-upload-pack for 31.164.105.196:0, 200 OK in 2.1ms @ private/serv.go:81(private.ServCommand)
Apr 11 13:10:42 imai gitea[556860]: 2023/04/11 13:10:42 [64354031-3] router: completed POST /api/internal/ssh/10/update/106 for 31.164.105.196:0, 200 OK in 129.7ms @ private/key.go:18(private.UpdatePublicKeyInRepo)
Apr 11 13:10:42 imai gitea[556860]: 2023/04/11 13:10:42 [64354032] router: completed POST /api/internal/ssh/log for 31.164.105.196:0, 200 OK in 0.1ms @ private/ssh_log.go:18(private.SSHLog)
Apr 11 13:10:42 imai gitea[556860]: 2023/04/11 13:10:42 [64354032-2] router: completed GET /api/internal/serv/command/10/shinmera/trial-deployment?mode=2&verb=git-lfs-authenticate&verb=upload for 31.164.105.196:0, 200 OK in 2.7ms @ private/serv.go:81(private.ServCommand)
Apr 11 13:10:43 imai gitea[556860]: 2023/04/11 13:10:43 [64354033] router: completed POST /shinmera/trial-deployment.git/info/lfs/locks/verify for 10.1.2.4:50662, 200 OK in 2.6ms @ lfs/locks.go:200(lfs.VerifyLockHandler)
Apr 11 13:10:43 imai gitea[556860]: 2023/04/11 13:10:43 [64354033-2] router: completed POST /shinmera/trial-deployment.git/info/lfs/objects/batch for 10.1.2.4:50664, 200 OK in 25.2ms @ lfs/server.go:154(lfs.BatchHandler)
Apr 11 13:10:44 imai gitea[556860]: 2023/04/11 13:10:44 [64354034-18] router: completed PUT /shinmera/trial-deployment.git/info/lfs/objects/fc663907272b18264b5e35c8d428aa378948d57a532f01452287d6fcdfad463c/49906 for 10.1.2.4:50968, 500 Internal Server Error in 10.0ms @ lfs/server.go:270(lfs.UploadHandler)

The logs contain nothing more useful, even when run in dev mode.

Pushing non-LFS content works just fine. I checked the file access permissions, everything is owned by the group and user git, and gitea is running under that user and group. I also tried setting the safe.directory git config setting to * or to include all relevant gitea storage directories, without success.

Gitea version 1.18.5 built with GNU Make 4.4.1, go1.20.1 : bindata, sqlite, sqlite_unlock_notify, pam

app.ini, passwords etc replaced by ...:

APP_NAME = TyNET Gitea
RUN_USER = git
RUN_MODE = prod

[security]
INTERNAL_TOKEN = ...
INSTALL_LOCK   = true
SECRET_KEY     = ...

[server]
LOCAL_ROOT_URL   = http://localhost:80/
SSH_DOMAIN       = git.tymoon.eu
DOMAIN           = localhost
HTTP_PORT        = 80
HTTP_ADDR        = 0.0.0.0
ROOT_URL         = https://gitea.tymoon.eu/
DISABLE_SSH      = false
SSH_PORT         = 10422
LFS_START_SERVER = true
LFS_JWT_SECRET   = ...

[lfs]
PATH = /var/lib/gitea/data/lfs

[database]
DB_TYPE  = postgres
HOST     = ...
NAME     = gitea
USER     = gitea
PASSWD   = ...
SSL_MODE = disable

[repository]
ROOT = /var/lib/gitea/repos

[service]
REGISTER_EMAIL_CONFIRM            = false
ENABLE_NOTIFY_MAIL                = true
DISABLE_REGISTRATION              = true
ALLOW_ONLY_EXTERNAL_REGISTRATION  = false
ENABLE_CAPTCHA                    = false
REQUIRE_SIGNIN_VIEW               = false
DEFAULT_KEEP_EMAIL_PRIVATE        = false
DEFAULT_ALLOW_CREATE_ORGANIZATION = true
DEFAULT_ENABLE_TIMETRACKING       = true
NO_REPLY_ADDRESS                  = noreply.example.org

[picture]
DISABLE_GRAVATAR        = false
ENABLE_FEDERATED_AVATAR = true

[openid]
ENABLE_OPENID_SIGNIN = false
ENABLE_OPENID_SIGNUP = false

[session]
PROVIDER = file

[log]
MODE      = file
LEVEL     = Debug
ROOT_PATH = /var/lib/gitea/log

[log.sublogger.macaron]
LEVEL = WARN

[ui]
DEFAULT_THEME = arc-green

[repository.upload]
ALLOWED_TYPES =
FILE_MAX_SIZE = 1000
MAX_FILES     = 100

[attachment]
ALLOWED_TYPES = image/jpeg|image/png|application/zip|application/gzip|application/x-xz|application/zlib|application/x-dosexec|application/octet-stream|application/x-msdownload|application/vnd.microsoft.portable-executable|application/x-apple-diskimage|application/x-compressed-tar|application/exe|application/x-exe|application/dos-exeapplication/x-winexe|application/msdos-windows|application/x-msdos-program|application/pdf
MAX_SIZE      = 1000
MAX_FILES     = 100

[oauth2]
JWT_SECRET = ...

Gitea Version

Gitea version 1.18.5 built with GNU Make 4.4.1, go1.20.1 : bindata, sqlite, sqlite_unlock_notify, pam

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

No response

Git Version

git version 2.40.0

Operating System

Linux imai 6.2.6-arch1-1 #1 SMP PREEMPT_DYNAMIC Mon, 13 Mar 2023 17:02:08 +0000 x86_64 GNU/Linux

How are you running Gitea?

Using official packages from the Arch Linux repositories, run as a systemd service.

Database

PostgreSQL

Originally created by @Shinmera on GitHub (Apr 11, 2023). ### Description When attempting to push new LFS artefacts, the internal logs only show error messages like so, with the last error repeating over and over: ``` Apr 11 13:10:40 imai gitea[556860]: 2023/04/11 13:10:40 [64354030] router: completed GET /api/internal/serv/command/10/shinmera/trial-deployment?mode=2&verb=git-receive-pack for 31.164.105.196:0, 200 OK in 2.4ms @ private/serv.go:81(private.ServCommand) Apr 11 13:10:41 imai gitea[556860]: 2023/04/11 13:10:41 [64354031] router: completed POST /api/internal/ssh/log for 31.164.105.196:0, 200 OK in 0.1ms @ private/ssh_log.go:18(private.SSHLog) Apr 11 13:10:41 imai gitea[556860]: 2023/04/11 13:10:41 [64354031-2] router: completed GET /api/internal/serv/command/10/shinmera/trial-deployment?mode=1&verb=git-upload-pack for 31.164.105.196:0, 200 OK in 2.1ms @ private/serv.go:81(private.ServCommand) Apr 11 13:10:42 imai gitea[556860]: 2023/04/11 13:10:42 [64354031-3] router: completed POST /api/internal/ssh/10/update/106 for 31.164.105.196:0, 200 OK in 129.7ms @ private/key.go:18(private.UpdatePublicKeyInRepo) Apr 11 13:10:42 imai gitea[556860]: 2023/04/11 13:10:42 [64354032] router: completed POST /api/internal/ssh/log for 31.164.105.196:0, 200 OK in 0.1ms @ private/ssh_log.go:18(private.SSHLog) Apr 11 13:10:42 imai gitea[556860]: 2023/04/11 13:10:42 [64354032-2] router: completed GET /api/internal/serv/command/10/shinmera/trial-deployment?mode=2&verb=git-lfs-authenticate&verb=upload for 31.164.105.196:0, 200 OK in 2.7ms @ private/serv.go:81(private.ServCommand) Apr 11 13:10:43 imai gitea[556860]: 2023/04/11 13:10:43 [64354033] router: completed POST /shinmera/trial-deployment.git/info/lfs/locks/verify for 10.1.2.4:50662, 200 OK in 2.6ms @ lfs/locks.go:200(lfs.VerifyLockHandler) Apr 11 13:10:43 imai gitea[556860]: 2023/04/11 13:10:43 [64354033-2] router: completed POST /shinmera/trial-deployment.git/info/lfs/objects/batch for 10.1.2.4:50664, 200 OK in 25.2ms @ lfs/server.go:154(lfs.BatchHandler) Apr 11 13:10:44 imai gitea[556860]: 2023/04/11 13:10:44 [64354034-18] router: completed PUT /shinmera/trial-deployment.git/info/lfs/objects/fc663907272b18264b5e35c8d428aa378948d57a532f01452287d6fcdfad463c/49906 for 10.1.2.4:50968, 500 Internal Server Error in 10.0ms @ lfs/server.go:270(lfs.UploadHandler) ``` The logs contain nothing more useful, even when run in ``dev`` mode. Pushing non-LFS content works just fine. I checked the file access permissions, everything is owned by the group and user ``git``, and gitea is running under that user and group. I also tried setting the `safe.directory` git config setting to `*` or to include all relevant gitea storage directories, without success. ``` Gitea version 1.18.5 built with GNU Make 4.4.1, go1.20.1 : bindata, sqlite, sqlite_unlock_notify, pam ``` app.ini, passwords etc replaced by `...`: ``` APP_NAME = TyNET Gitea RUN_USER = git RUN_MODE = prod [security] INTERNAL_TOKEN = ... INSTALL_LOCK = true SECRET_KEY = ... [server] LOCAL_ROOT_URL = http://localhost:80/ SSH_DOMAIN = git.tymoon.eu DOMAIN = localhost HTTP_PORT = 80 HTTP_ADDR = 0.0.0.0 ROOT_URL = https://gitea.tymoon.eu/ DISABLE_SSH = false SSH_PORT = 10422 LFS_START_SERVER = true LFS_JWT_SECRET = ... [lfs] PATH = /var/lib/gitea/data/lfs [database] DB_TYPE = postgres HOST = ... NAME = gitea USER = gitea PASSWD = ... SSL_MODE = disable [repository] ROOT = /var/lib/gitea/repos [service] REGISTER_EMAIL_CONFIRM = false ENABLE_NOTIFY_MAIL = true DISABLE_REGISTRATION = true ALLOW_ONLY_EXTERNAL_REGISTRATION = false ENABLE_CAPTCHA = false REQUIRE_SIGNIN_VIEW = false DEFAULT_KEEP_EMAIL_PRIVATE = false DEFAULT_ALLOW_CREATE_ORGANIZATION = true DEFAULT_ENABLE_TIMETRACKING = true NO_REPLY_ADDRESS = noreply.example.org [picture] DISABLE_GRAVATAR = false ENABLE_FEDERATED_AVATAR = true [openid] ENABLE_OPENID_SIGNIN = false ENABLE_OPENID_SIGNUP = false [session] PROVIDER = file [log] MODE = file LEVEL = Debug ROOT_PATH = /var/lib/gitea/log [log.sublogger.macaron] LEVEL = WARN [ui] DEFAULT_THEME = arc-green [repository.upload] ALLOWED_TYPES = FILE_MAX_SIZE = 1000 MAX_FILES = 100 [attachment] ALLOWED_TYPES = image/jpeg|image/png|application/zip|application/gzip|application/x-xz|application/zlib|application/x-dosexec|application/octet-stream|application/x-msdownload|application/vnd.microsoft.portable-executable|application/x-apple-diskimage|application/x-compressed-tar|application/exe|application/x-exe|application/dos-exeapplication/x-winexe|application/msdos-windows|application/x-msdos-program|application/pdf MAX_SIZE = 1000 MAX_FILES = 100 [oauth2] JWT_SECRET = ... ``` ### Gitea Version Gitea version 1.18.5 built with GNU Make 4.4.1, go1.20.1 : bindata, sqlite, sqlite_unlock_notify, pam ### Can you reproduce the bug on the Gitea demo site? No ### Log Gist _No response_ ### Screenshots _No response_ ### Git Version git version 2.40.0 ### Operating System Linux imai 6.2.6-arch1-1 #1 SMP PREEMPT_DYNAMIC Mon, 13 Mar 2023 17:02:08 +0000 x86_64 GNU/Linux ### How are you running Gitea? Using official packages from the Arch Linux repositories, run as a systemd service. ### Database PostgreSQL
GiteaMirror added the type/bug label 2025-11-02 09:13:32 -06:00
GiteaMirror commented 2025-11-02 09:13:33 -06:00
Author
Owner
Copy Link

@sillyguodong commented on GitHub (Apr 12, 2023):

Are there any associated errors in gitea.log?

@sillyguodong commented on GitHub (Apr 12, 2023): Are there any associated errors in `gitea.log`?
GiteaMirror commented 2025-11-02 09:13:33 -06:00
Author
Owner
Copy Link

@Shinmera commented on GitHub (Apr 12, 2023):

Not as far as I can see. I've uploaded the corresponding timestamp from the log:

https://gist.github.com/Shinmera/13f31bf25a27da09e7a82cbff7cdeb38

@Shinmera commented on GitHub (Apr 12, 2023): Not as far as I can see. I've uploaded the corresponding timestamp from the log: https://gist.github.com/Shinmera/13f31bf25a27da09e7a82cbff7cdeb38
GiteaMirror commented 2025-11-02 09:13:33 -06:00
Author
Owner
Copy Link

@wxiaoguang commented on GitHub (Apr 12, 2023):

The most suspicious part is that there are a lot of ROLLBACK commands in the SQL log.

And it explains that why there is no error log, because at the moment, the code is, the database failure doesn't print the error .....

Do you know if there is anything wrong with the lfs_meta_object table?

@wxiaoguang commented on GitHub (Apr 12, 2023): The most suspicious part is that there are a lot of `ROLLBACK` commands in the SQL log. And it explains that why there is no error log, because at the moment, the code is, the database failure doesn't print the error ..... Do you know if there is anything wrong with the lfs_meta_object table?
GiteaMirror commented 2025-11-02 09:13:33 -06:00
Author
Owner
Copy Link

@Shinmera commented on GitHub (Apr 12, 2023):

Ah, good shout!

I had a peek around with pgadmin, and it seems the lfs_meta_object sequence somehow got out of sync, possibly due to a bad backup job at some point? Fixing the sequence resolved the issue.

Thanks a lot for the debugging help, and my apologies for the noise!

@Shinmera commented on GitHub (Apr 12, 2023): Ah, good shout! I had a peek around with pgadmin, and it seems the lfs_meta_object sequence somehow got out of sync, possibly due to a bad backup job at some point? Fixing the sequence resolved the issue. Thanks a lot for the debugging help, and my apologies for the noise!
GiteaMirror commented 2025-11-02 09:13:34 -06:00
Author
Owner
Copy Link

@wxiaoguang commented on GitHub (Apr 12, 2023):

Never mind, I also find that the log is pretty unfriendly .... so I propose https://github.com/go-gitea/gitea/pull/24072 😁

@wxiaoguang commented on GitHub (Apr 12, 2023): Never mind, I also find that the log is pretty unfriendly .... so I propose https://github.com/go-gitea/gitea/pull/24072 😁
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
$20
$250
$50
$500
backport/done
💎 Bounty
docs-update-needed
good first issue
hacktoberfest
issue/bounty
issue/confirmed
issue/critical
issue/duplicate
issue/needs-feedback
issue/not-a-bug
issue/regression
issue/stale
issue/workaround
lgtm/need 2
modifies/api
modifies/translation
outdated/backport/v1.18
outdated/theme/markdown
outdated/theme/timetracker
performance/bigrepo
performance/cpu
performance/memory
performance/speed
pr/breaking
proposal/accepted
proposal/rejected
pr/wip
pull-request
Mirrored from GitHub Pull Request
 reviewed/wontfix
💰 Rewarded
skip-changelog
status/blocked
topic/accessibility
topic/api
topic/authentication
topic/build
topic/code-linting
topic/commit-signing
topic/content-rendering
topic/deployment
topic/distribution
topic/federation
topic/gitea-actions
topic/issues
topic/lfs
topic/mobile
topic/moderation
topic/packages
topic/pr
topic/projects
topic/repo
topic/repo-migration
topic/security
topic/theme
topic/ui
topic/ui-interaction
topic/ux
topic/webhooks
topic/wiki
type/bug
type/deprecation
type/docs
type/enhancement
type/feature
type/miscellaneous
type/proposal
type/question
type/refactoring
type/summary
type/testing
type/upstream
No Label type/bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/gitea#10631
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?