[PR #2990] [MERGED] Improve memory usage when reaching diff limits #16701

New Issue

GiteaMirror · 2025-11-02T12:16:42-06:00

GiteaMirror commented

2025-11-02 12:16:42 -06:00

📋 Pull Request Information

Original PR: https://github.com/go-gitea/gitea/pull/2990
Author: @mrexodia
Created: 11/27/2017
Status: ✅ Merged
Merged: 11/28/2017
Merged by: @lafriks

Base: master ← Head: diff-memory-usage

📝 Commits (3)

1031bc2 Improve memory usage when reaching diff limits
520c5e8 Merge branch 'master' into diff-memory-usage
2ade5b3 Merge branch 'master' into diff-memory-usage

📊 Changes

1 file changed (+20 additions, -12 deletions)

View changed files

📝 models/git_diff.go (+20 -12)

📄 Description

Related to #2669, also related to https://github.com/go-gitea/git/pull/93

This change introduces a hand-rolled implementation of input.ReadString() that stops reading if the line buffer gets bigger than maxLineCharacters.

I thought the performance of this would be terrible, but actually it appears to be ~~slightly faster~~ slower when timing the ParsePatch function (with real loose timing so might be good to check out).

For commits that are shown completely the memory usage is slightly better, but for commits where files are hidden because they are too big, the memory usage is much, much better (my test was a 100mb one-liner which went from 322mb to 55mb).

The ParsePatch function does not behave completely identical because line is now truncated to whatever the user sets as maximum. I checked all usages a little and it does not appear to matter (diffs are only shown to the user and if IsIncomplete is set the line member is not used), however this needs some attention during review.

Possible follow-up for this is to completely rewrite the diff functions to use things like git diff --numstat to figure out if diffs have to be truncated and which files are part of a diff instead of this ugly parser.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/go-gitea/gitea/pull/2990 **Author:** [@mrexodia](https://github.com/mrexodia) **Created:** 11/27/2017 **Status:** ✅ Merged **Merged:** 11/28/2017 **Merged by:** [@lafriks](https://github.com/lafriks) **Base:** `master` ← **Head:** `diff-memory-usage` --- ### 📝 Commits (3) - [`1031bc2`](https://github.com/go-gitea/gitea/commit/1031bc2bbc6e3d78fddcbd92911b559358187960) Improve memory usage when reaching diff limits - [`520c5e8`](https://github.com/go-gitea/gitea/commit/520c5e82e45cbf51cf23bb324445290f998c9018) Merge branch 'master' into diff-memory-usage - [`2ade5b3`](https://github.com/go-gitea/gitea/commit/2ade5b34dea5ba0f1f1de61e2d23e7cf6998d5cd) Merge branch 'master' into diff-memory-usage ### 📊 Changes **1 file changed** (+20 additions, -12 deletions) <details> <summary>View changed files</summary> 📝 `models/git_diff.go` (+20 -12) </details> ### 📄 Description Related to #2669, also related to https://github.com/go-gitea/git/pull/93 This change introduces a hand-rolled implementation of `input.ReadString()` that stops reading if the line buffer gets bigger than `maxLineCharacters`. I thought the performance of this would be terrible, but actually it appears to be ~slightly faster~ **slower** when timing the `ParsePatch` function (with real loose timing so might be good to check out). For commits that are shown completely the memory usage is slightly better, but for commits where files are hidden because they are too big, the memory usage is much, much better (my test was a 100mb one-liner which went from 322mb to 55mb). The `ParsePatch` function does not behave completely identical because `line` is now truncated to whatever the user sets as maximum. I checked all usages a little and it does not appear to matter (diffs are only shown to the user and if `IsIncomplete` is set the `line` member is not used), however this needs some attention during review. Possible follow-up for this is to completely rewrite the diff functions to use things like `git diff --numstat` to figure out if diffs have to be truncated and which files are part of a diff instead of this ugly parser. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

GiteaMirror added the pull-request label 2025-11-02 12:16:42 -06:00

GiteaMirror closed this issue

2025-11-02 12:16:43 -06:00

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/gitea#16701