Out of memory while browsing large repositories (pprof report attached) #8342

New Issue

GiteaMirror · 2025-11-02T08:03:06-06:00

GiteaMirror commented

2025-11-02 08:03:06 -06:00

Originally created by @MatthiasPetermann on GitHub (Jan 12, 2022).

Gitea Version

1.13.4

Git Version

2.34.1

Operating System

NetBSD 9.2

How are you running Gitea?

Gitea built via pkgsrc framework
Run on a virtual server (Xen) from NetBSD rc-Script

Database

SQLite

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Description

Hello all,

when browsing a larger repository (NetBSD sources, about 2.5 GB with very many subdirectories and small files) the memory used by Gitea increases continuously and rapidly until finally even the swap is not enough anymore and the system has to kill the process because of OOM.

The Gitea version I use is a bit older (1.13.4) since there is no newer one in pkgsrc.

Gitea is configured with Redis as cache and cache for last commits.

I found a similar ticket #753 on this topic, which was closed due to lack of contributors. I would like to follow up on this one by attaching a report from pprof showing the state of the heap memory after browsing a larger repository (only home page open and a few directories clicked). The report shows about 1292 MB of memory usage. However, on the system (NetBSD) the process shows a size of 2241 MB (RES) and 3722 MB (SIZE).

profile006.pdf

Meaning of SIZE in NetBSD: Total size of the process (text, data, and stack) given in kilobytes

Meaning of RES in NetBSD: Resident memory: current amount of process memory that resides in physical memory, given in kilobytes.

Can someone possibly help me here and clarify on which side there is a problem here? Obviously Gitea does not release memory once allocated, but I can't explain the big differences between the report and the output of top. I also like to run more reports here, if that is useful for the cause research. And if the problem is already known and by chance already addressed in a newer release, I would of course also try to get an update with higher urgency. But at the moment I'm not quite sure if it's worth the effort or if I'm reaching a performance limit of Gitea that it's not built for?

I would be very happy to receive an answer.

Many greetings
Matthias

Screenshots

No response

Originally created by @MatthiasPetermann on GitHub (Jan 12, 2022). ### Gitea Version 1.13.4 ### Git Version 2.34.1 ### Operating System NetBSD 9.2 ### How are you running Gitea? - Gitea built via pkgsrc framework - Run on a virtual server (Xen) from NetBSD rc-Script ### Database SQLite ### Can you reproduce the bug on the Gitea demo site? No ### Log Gist _No response_ ### Description Hello all, when browsing a larger repository (NetBSD sources, about 2.5 GB with very many subdirectories and small files) the memory used by Gitea increases continuously and rapidly until finally even the swap is not enough anymore and the system has to kill the process because of OOM. The Gitea version I use is a bit older (1.13.4) since there is no newer one in pkgsrc. Gitea is configured with Redis as cache and cache for last commits. I found a similar ticket #753 on this topic, which was closed due to lack of contributors. I would like to follow up on this one by attaching a report from pprof showing the state of the heap memory after browsing a larger repository (only home page open and a few directories clicked). The report shows about 1292 MB of memory usage. However, on the system (NetBSD) the process shows a size of 2241 MB (RES) and 3722 MB (SIZE). [profile006.pdf](https://github.com/go-gitea/gitea/files/7853364/profile006.pdf) Meaning of SIZE in NetBSD: Total size of the process (text, data, and stack) given in kilobytes Meaning of RES in NetBSD: Resident memory: current amount of process memory that resides in physical memory, given in kilobytes. Can someone possibly help me here and clarify on which side there is a problem here? Obviously Gitea does not release memory once allocated, but I can't explain the big differences between the report and the output of top. I also like to run more reports here, if that is useful for the cause research. And if the problem is already known and by chance already addressed in a newer release, I would of course also try to get an update with higher urgency. But at the moment I'm not quite sure if it's worth the effort or if I'm reaching a performance limit of Gitea that it's not built for? I would be very happy to receive an answer. Many greetings Matthias ### Screenshots _No response_

GiteaMirror added the reviewed/wontfix label 2025-11-02 08:03:07 -06:00

GiteaMirror closed this issue

2025-11-02 08:03:07 -06:00

GiteaMirror commented

2025-11-02 08:03:07 -06:00

@zeripath commented on GitHub (Jan 12, 2022):

There have been many many improvements since 1.13.4. Please upgrade to 1.15.x or even main and see if you can reproduce there.

@zeripath commented on GitHub (Jan 12, 2022): There have been many many improvements since 1.13.4. Please upgrade to 1.15.x or even main and see if you can reproduce there.

GiteaMirror commented

2025-11-02 08:03:08 -06:00

@zeripath commented on GitHub (Jan 12, 2022):

Looking at your provided mem profile the issue appears to be related to memory use from earlier versions of go-git in those versions of gitea.

This has been addressed (or at least attempted to be addressed) in the many intervening changes between this 1.13.4 and 1.15.x.

$ git diff  --shortstat v1.15.9..v1.13.4                                                     [12:07:54]
 6766 files changed, 350744 insertions(+), 654174 deletions(-)
warning: exhaustive rename detection was skipped due to too many files.
warning: you may want to set your diff.renameLimit variable to at least 2140 and retry the command.

 $ git diff  --shortstat origin/release/v1.15..v1.13.4                                        [12:08:03]
 6767 files changed, 350906 insertions(+), 654787 deletions(-)
warning: exhaustive rename detection was skipped due to too many files.
warning: you may want to set your diff.renameLimit variable to at least 2140 and retry the command.

$ git diff  --shortstat main..v1.13.4                                                        [12:33:09]
 7714 files changed, 416025 insertions(+), 758405 deletions(-)
warning: exhaustive rename detection was skipped due to too many files.
warning: you may want to set your diff.renameLimit variable to at least 2754 and retry the command.

With such extensive changes in the codebase you just need to upgrade.

If you come to discord or use the matrix bridge #gitea:matrix.org and ask for help on how to compile gitea we would be glad to help you to compile and build a newer version but otherwise there's nothing we can.

Marking as wontfix as 1.13 is out of support.

@zeripath commented on GitHub (Jan 12, 2022): Looking at your provided mem profile the issue appears to be related to memory use from earlier versions of go-git in those versions of gitea. This has been addressed (or at least attempted to be addressed) in the many intervening changes between this 1.13.4 and 1.15.x. ```bash $ git diff --shortstat v1.15.9..v1.13.4 [12:07:54] 6766 files changed, 350744 insertions(+), 654174 deletions(-) warning: exhaustive rename detection was skipped due to too many files. warning: you may want to set your diff.renameLimit variable to at least 2140 and retry the command. $ git diff --shortstat origin/release/v1.15..v1.13.4 [12:08:03] 6767 files changed, 350906 insertions(+), 654787 deletions(-) warning: exhaustive rename detection was skipped due to too many files. warning: you may want to set your diff.renameLimit variable to at least 2140 and retry the command. $ git diff --shortstat main..v1.13.4 [12:33:09] 7714 files changed, 416025 insertions(+), 758405 deletions(-) warning: exhaustive rename detection was skipped due to too many files. warning: you may want to set your diff.renameLimit variable to at least 2754 and retry the command. ``` With such extensive changes in the codebase you just need to upgrade. If you come to discord or use the matrix bridge #gitea:matrix.org and ask for help on how to compile gitea we would be glad to help you to compile and build a newer version but otherwise there's nothing we can. Marking as wontfix as 1.13 is out of support.

GiteaMirror commented

2025-11-02 08:03:08 -06:00

@MatthiasPetermann commented on GitHub (Jan 12, 2022):

Hello, thank you for the feedback and your assessment. You have definitely convinced me to try a new version. Maybe I can take this opportunity to contribute some updates for the pkgsrc right away and will give it a try :-) In any case, thanks again for taking the trouble to look at my memory dump anyway - I really appreciate it.

@MatthiasPetermann commented on GitHub (Jan 12, 2022): Hello, thank you for the feedback and your assessment. You have definitely convinced me to try a new version. Maybe I can take this opportunity to contribute some updates for the pkgsrc right away and will give it a try :-) In any case, thanks again for taking the trouble to look at my memory dump anyway - I really appreciate it.

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/gitea#8342