LFS: unexpected EOF despite good integrity of the file on the server #2452

Closed
opened 2025-11-02 04:36:45 -06:00 by GiteaMirror · 10 comments
Owner

Originally created by @qzmfranklin on GitHub (Oct 25, 2018).

  • Gitea version (or commit ref):
    git-lfs/2.5.2
  • Git version:
    git version 2.17.1
  • Operating system:
    Ubuntu 18.04.1
  • Database (use [x]):
    • PostgreSQL
    • MySQL
    • MSSQL
    • SQLite
  • Can you reproduce the bug at https://try.gitea.io:
    • Yes (provide example URL)
    • No
    • Not relevant
  • Log gist:
git-lfs/2.5.2 (GitHub; linux amd64; go 1.10.3; git 8e3c5c93)
git version 2.17.1

$ git-lfs filter-process
Error downloading object: it/docs/jenkins/step1.jpg (c5e6043): Smudge error: Error downloading it/docs/jenkins/step1.jpg (c5e60437c9e7596445342c5adc91f6535e7565fab96ff200112ed8b2aabdbf47): cannot write data to tempfile "/home/zhongming/git/logi3/.git/lfs/incomplete/c5e60437c9e7596445342c5adc91f6535e7565fab96ff200112ed8b2aabdbf47.tmp": LFS: unexpected EOF

unexpected EOF
LFS
github.com/git-lfs/git-lfs/errors.newWrappedError
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/errors/types.go:188
github.com/git-lfs/git-lfs/errors.NewRetriableError
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/errors/types.go:363
github.com/git-lfs/git-lfs/tools.(*RetriableReader).Read
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/tools/iotools.go:99
github.com/git-lfs/git-lfs/tools.(*HashingReader).Read
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/tools/iotools.go:66
github.com/git-lfs/git-lfs/tools.(*CallbackReader).Read
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/tools/copycallback.go:73
io.copyBuffer
        /usr/local/Cellar/go/1.10.3/libexec/src/io/io.go:400
io.Copy
        /usr/local/Cellar/go/1.10.3/libexec/src/io/io.go:362
github.com/git-lfs/git-lfs/tools.CopyWithCallback
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/tools/iotools.go:39
github.com/git-lfs/git-lfs/tq.(*basicDownloadAdapter).download
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/tq/basic_download.go:205
github.com/git-lfs/git-lfs/tq.(*basicDownloadAdapter).DoTransfer
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/tq/basic_download.go:48
github.com/git-lfs/git-lfs/tq.(*adapterBase).worker
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/tq/adapterbase.go:176
runtime.goexit
        /usr/local/Cellar/go/1.10.3/libexec/src/runtime/asm_amd64.s:2361
cannot write data to tempfile "/home/zhongming/git/logi3/.git/lfs/incomplete/c5e60437c9e7596445342c5adc91f6535e7565fab96ff200112ed8b2aabdbf47.tmp"
github.com/git-lfs/git-lfs/errors.newWrappedError
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/errors/types.go:184
github.com/git-lfs/git-lfs/errors.Wrapf
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/errors/errors.go:85
github.com/git-lfs/git-lfs/tq.(*basicDownloadAdapter).download
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/tq/basic_download.go:207
github.com/git-lfs/git-lfs/tq.(*basicDownloadAdapter).DoTransfer
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/tq/basic_download.go:48
github.com/git-lfs/git-lfs/tq.(*adapterBase).worker
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/tq/adapterbase.go:176
runtime.goexit
        /usr/local/Cellar/go/1.10.3/libexec/src/runtime/asm_amd64.s:2361
Error downloading it/docs/jenkins/step1.jpg (c5e60437c9e7596445342c5adc91f6535e7565fab96ff200112ed8b2aabdbf47)
github.com/git-lfs/git-lfs/errors.newWrappedError
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/errors/types.go:184
github.com/git-lfs/git-lfs/errors.Wrapf
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/errors/errors.go:85
github.com/git-lfs/git-lfs/lfs.(*GitFilter).downloadFile
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/lfs/gitfilter_smudge.go:110
github.com/git-lfs/git-lfs/lfs.(*GitFilter).Smudge
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/lfs/gitfilter_smudge.go:71
github.com/git-lfs/git-lfs/commands.smudge
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/commands/command_smudge.go:127
github.com/git-lfs/git-lfs/commands.filterCommand
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/commands/command_filter_process.go:118
github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).execute
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:477
github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).Execute
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:551
github.com/git-lfs/git-lfs/commands.Run
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/commands/run.go:77
main.main
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/git-lfs.go:35
runtime.main
        /usr/local/Cellar/go/1.10.3/libexec/src/runtime/proc.go:198
runtime.goexit
        /usr/local/Cellar/go/1.10.3/libexec/src/runtime/asm_amd64.s:2361
Smudge error
github.com/git-lfs/git-lfs/errors.newWrappedError
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/errors/types.go:184
github.com/git-lfs/git-lfs/errors.NewSmudgeError
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/errors/types.go:270
github.com/git-lfs/git-lfs/lfs.(*GitFilter).Smudge
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/lfs/gitfilter_smudge.go:80
github.com/git-lfs/git-lfs/commands.smudge
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/commands/command_smudge.go:127
github.com/git-lfs/git-lfs/commands.filterCommand
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/commands/command_filter_process.go:118
github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).execute
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:477
github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).Execute
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:551
github.com/git-lfs/git-lfs/commands.Run
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/commands/run.go:77
main.main
        /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/git-lfs.go:35
runtime.main
        /usr/local/Cellar/go/1.10.3/libexec/src/runtime/proc.go:198
runtime.goexit
        /usr/local/Cellar/go/1.10.3/libexec/src/runtime/asm_amd64.s:2361

Current time in UTC: 
2018-10-25 14:11:01

ENV:
LocalWorkingDir=/home/zhongming/git/logi3
LocalGitDir=/home/zhongming/git/logi3/.git
LocalGitStorageDir=/home/zhongming/git/logi3/.git
LocalMediaDir=/home/zhongming/git/logi3/.git/lfs/objects
LocalReferenceDirs=
TempDir=/home/zhongming/git/logi3/.git/lfs/tmp
ConcurrentTransfers=3
TusTransfers=false
BasicTransfersOnly=false
SkipDownloadErrors=false
FetchRecentAlways=false
FetchRecentRefsDays=7
FetchRecentCommitsDays=0
FetchRecentRefsIncludeRemotes=true
PruneOffsetDays=3
PruneVerifyRemoteAlways=false
PruneRemoteName=origin
LfsStorageDir=/home/zhongming/git/logi3/.git/lfs
AccessDownload=none
AccessUpload=none
DownloadTransfers=basic
UploadTransfers=basic
GIT_DIR=.git
GIT_PROMPT_ONLY_IN_REPO=1
GIT_BRANCH=master
__GIT_PROMPT_IGNORE_SUBMODULES__=1
__GIT_PROMPT_SHOW_CHANGED_FILES_COUNT=1
__GIT_PROMPT_SHOW_UPSTREAM=
__GIT_PROMPT_IGNORE_SUBMODULES=
__GIT_PROMPT_SHOW_UNTRACKED_FILES=normal
__GIT_PROMPT_IGNORE_STASH=

Description

I have a local gitea server set up at host http://git.corp.logiocean.com:3000.

I pushed an LFS-enabled repo, named logi to the gitea server. Everything worked fine until I tried git clone, which produced the above described error.

I pushed the same repo (at the same commit) to github and git clone from github, everything worked.

I have verified that the git commit hash on github and on my local gitea server are exactly the same.

I have also verified that the LFS file that failed to download are intact on the server. Here is how I verified it:

  • ssh into the server
  • go to the lfs data directory /home/git/data/lfs.
  • use the oid of the lfs file to locate the file.
  • use scp to copy the file in question to my own desktop and verified (using diff and other tools) that the size of the file and the content of the file are indeed intact.

I have searched for similar issues across this repo and in many other places. Have not found one with exactly the same syndrome.

Posting to hopefully help others who are having the same problem.

Any help would be much appreciated! Thanks in advance.

Originally created by @qzmfranklin on GitHub (Oct 25, 2018). <!-- 1. Please speak English, this is the language all of us can speak and write. 2. Please ask questions or configuration/deploy problems on our Discord server (https://discord.gg/NsatcWJ) or forum (https://discourse.gitea.io). 3. Please take a moment to check that your issue doesn't already exist. 4. Please give all relevant information below for bug reports, because incomplete details will be handled as an invalid report. --> - Gitea version (or commit ref): git-lfs/2.5.2 - Git version: git version 2.17.1 - Operating system: Ubuntu 18.04.1 - Database (use `[x]`): - [x] PostgreSQL - [ ] MySQL - [ ] MSSQL - [ ] SQLite - Can you reproduce the bug at https://try.gitea.io: - [ ] Yes (provide example URL) - [ ] No - [x] Not relevant - Log gist: ```text git-lfs/2.5.2 (GitHub; linux amd64; go 1.10.3; git 8e3c5c93) git version 2.17.1 $ git-lfs filter-process Error downloading object: it/docs/jenkins/step1.jpg (c5e6043): Smudge error: Error downloading it/docs/jenkins/step1.jpg (c5e60437c9e7596445342c5adc91f6535e7565fab96ff200112ed8b2aabdbf47): cannot write data to tempfile "/home/zhongming/git/logi3/.git/lfs/incomplete/c5e60437c9e7596445342c5adc91f6535e7565fab96ff200112ed8b2aabdbf47.tmp": LFS: unexpected EOF unexpected EOF LFS github.com/git-lfs/git-lfs/errors.newWrappedError /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/errors/types.go:188 github.com/git-lfs/git-lfs/errors.NewRetriableError /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/errors/types.go:363 github.com/git-lfs/git-lfs/tools.(*RetriableReader).Read /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/tools/iotools.go:99 github.com/git-lfs/git-lfs/tools.(*HashingReader).Read /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/tools/iotools.go:66 github.com/git-lfs/git-lfs/tools.(*CallbackReader).Read /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/tools/copycallback.go:73 io.copyBuffer /usr/local/Cellar/go/1.10.3/libexec/src/io/io.go:400 io.Copy /usr/local/Cellar/go/1.10.3/libexec/src/io/io.go:362 github.com/git-lfs/git-lfs/tools.CopyWithCallback /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/tools/iotools.go:39 github.com/git-lfs/git-lfs/tq.(*basicDownloadAdapter).download /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/tq/basic_download.go:205 github.com/git-lfs/git-lfs/tq.(*basicDownloadAdapter).DoTransfer /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/tq/basic_download.go:48 github.com/git-lfs/git-lfs/tq.(*adapterBase).worker /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/tq/adapterbase.go:176 runtime.goexit /usr/local/Cellar/go/1.10.3/libexec/src/runtime/asm_amd64.s:2361 cannot write data to tempfile "/home/zhongming/git/logi3/.git/lfs/incomplete/c5e60437c9e7596445342c5adc91f6535e7565fab96ff200112ed8b2aabdbf47.tmp" github.com/git-lfs/git-lfs/errors.newWrappedError /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/errors/types.go:184 github.com/git-lfs/git-lfs/errors.Wrapf /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/errors/errors.go:85 github.com/git-lfs/git-lfs/tq.(*basicDownloadAdapter).download /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/tq/basic_download.go:207 github.com/git-lfs/git-lfs/tq.(*basicDownloadAdapter).DoTransfer /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/tq/basic_download.go:48 github.com/git-lfs/git-lfs/tq.(*adapterBase).worker /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/tq/adapterbase.go:176 runtime.goexit /usr/local/Cellar/go/1.10.3/libexec/src/runtime/asm_amd64.s:2361 Error downloading it/docs/jenkins/step1.jpg (c5e60437c9e7596445342c5adc91f6535e7565fab96ff200112ed8b2aabdbf47) github.com/git-lfs/git-lfs/errors.newWrappedError /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/errors/types.go:184 github.com/git-lfs/git-lfs/errors.Wrapf /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/errors/errors.go:85 github.com/git-lfs/git-lfs/lfs.(*GitFilter).downloadFile /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/lfs/gitfilter_smudge.go:110 github.com/git-lfs/git-lfs/lfs.(*GitFilter).Smudge /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/lfs/gitfilter_smudge.go:71 github.com/git-lfs/git-lfs/commands.smudge /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/commands/command_smudge.go:127 github.com/git-lfs/git-lfs/commands.filterCommand /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/commands/command_filter_process.go:118 github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).execute /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:477 github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).Execute /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:551 github.com/git-lfs/git-lfs/commands.Run /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/commands/run.go:77 main.main /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/git-lfs.go:35 runtime.main /usr/local/Cellar/go/1.10.3/libexec/src/runtime/proc.go:198 runtime.goexit /usr/local/Cellar/go/1.10.3/libexec/src/runtime/asm_amd64.s:2361 Smudge error github.com/git-lfs/git-lfs/errors.newWrappedError /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/errors/types.go:184 github.com/git-lfs/git-lfs/errors.NewSmudgeError /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/errors/types.go:270 github.com/git-lfs/git-lfs/lfs.(*GitFilter).Smudge /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/lfs/gitfilter_smudge.go:80 github.com/git-lfs/git-lfs/commands.smudge /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/commands/command_smudge.go:127 github.com/git-lfs/git-lfs/commands.filterCommand /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/commands/command_filter_process.go:118 github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).execute /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:477 github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).Execute /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:551 github.com/git-lfs/git-lfs/commands.Run /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/commands/run.go:77 main.main /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/git-lfs.go:35 runtime.main /usr/local/Cellar/go/1.10.3/libexec/src/runtime/proc.go:198 runtime.goexit /usr/local/Cellar/go/1.10.3/libexec/src/runtime/asm_amd64.s:2361 Current time in UTC: 2018-10-25 14:11:01 ENV: LocalWorkingDir=/home/zhongming/git/logi3 LocalGitDir=/home/zhongming/git/logi3/.git LocalGitStorageDir=/home/zhongming/git/logi3/.git LocalMediaDir=/home/zhongming/git/logi3/.git/lfs/objects LocalReferenceDirs= TempDir=/home/zhongming/git/logi3/.git/lfs/tmp ConcurrentTransfers=3 TusTransfers=false BasicTransfersOnly=false SkipDownloadErrors=false FetchRecentAlways=false FetchRecentRefsDays=7 FetchRecentCommitsDays=0 FetchRecentRefsIncludeRemotes=true PruneOffsetDays=3 PruneVerifyRemoteAlways=false PruneRemoteName=origin LfsStorageDir=/home/zhongming/git/logi3/.git/lfs AccessDownload=none AccessUpload=none DownloadTransfers=basic UploadTransfers=basic GIT_DIR=.git GIT_PROMPT_ONLY_IN_REPO=1 GIT_BRANCH=master __GIT_PROMPT_IGNORE_SUBMODULES__=1 __GIT_PROMPT_SHOW_CHANGED_FILES_COUNT=1 __GIT_PROMPT_SHOW_UPSTREAM= __GIT_PROMPT_IGNORE_SUBMODULES= __GIT_PROMPT_SHOW_UNTRACKED_FILES=normal __GIT_PROMPT_IGNORE_STASH= ``` ## Description I have a local gitea server set up at host http://git.corp.logiocean.com:3000. I pushed an LFS-enabled repo, named `logi` to the gitea server. Everything worked fine until I tried `git clone`, which produced the above described error. I pushed the same repo (at the same commit) to github and `git clone` from github, everything worked. I have verified that the git commit hash on github and on my local gitea server are exactly the same. I have also verified that the LFS file that failed to download are intact on the server. Here is how I verified it: - ssh into the server - go to the lfs data directory `/home/git/data/lfs`. - use the `oid` of the lfs file to locate the file. - use `scp` to copy the file in question to my own desktop and verified (using diff and other tools) that the size of the file and the content of the file are indeed intact. I have searched for similar issues across this repo and in many other places. Have not found one with exactly the same syndrome. Posting to hopefully help others who are having the same problem. Any help would be much appreciated! Thanks in advance.
GiteaMirror added the issue/confirmedtype/bug labels 2025-11-02 04:36:45 -06:00
Author
Owner

@zeripath commented on GitHub (Oct 25, 2018):

hmm... looking at your logs I see: /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/errors/types.go:188. Are you running Gitea on a Mac?

I wonder if the file that's being downloaded to be smudged by git-lfs has the wrong line-endings?

git lfs install --skip-smudge

should allow you to skip the smudge action and interrogate the smudge files to check how they are formatted - although they might actually already be in your checkout for you to take a look at.

@zeripath commented on GitHub (Oct 25, 2018): hmm... looking at your logs I see: `/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/errors/types.go:188`. Are you running Gitea on a Mac? I wonder if the file that's being downloaded to be smudged by git-lfs has the wrong line-endings? ``` git lfs install --skip-smudge ``` should allow you to skip the smudge action and interrogate the smudge files to check how they are formatted - although they might actually already be in your checkout for you to take a look at.
Author
Owner

@qzmfranklin commented on GitHub (Oct 26, 2018):

Gitea was running on a Ubuntu 18.04.1 server.

This issue happens for only a few files. I tried --skip-smudge and verified the content of the lfs object file. The file has exactly the same content as it should (verified by xxd and diff).

To get around this issue, I have removed those faulting files. Running out of ideas to try.

@qzmfranklin commented on GitHub (Oct 26, 2018): Gitea was running on a Ubuntu 18.04.1 server. This issue happens for only a few files. I tried `--skip-smudge` and verified the content of the lfs object file. The file has exactly the same content as it should (verified by `xxd` and `diff`). To get around this issue, I have removed those faulting files. Running out of ideas to try.
Author
Owner

@zeripath commented on GitHub (Oct 27, 2018):

Are the faulting files proprietary? Would it be possible to post them so I could try them. Things that come to mind are:

  • Is there a buffer overrun/underrun problem? I.e. a file with the exact same length should also fail. These problems can be hard to find - but can be exploitable (!)
  • Is there a problem with something replacing part of the file. I.e. is something filtering out say the null bytes, or is the stream stopping too early due to null bytes.

The problem could be anywhere, but it's unlikely to be on your client end if you say it works for other providers. However it might be good to try a different client machine to see if it's restricted to a single client.

@zeripath commented on GitHub (Oct 27, 2018): Are the faulting files proprietary? Would it be possible to post them so I could try them. Things that come to mind are: * Is there a buffer overrun/underrun problem? I.e. a file with the exact same length should also fail. These problems can be hard to find - but can be exploitable (!) * Is there a problem with something replacing part of the file. I.e. is something filtering out say the null bytes, or is the stream stopping too early due to null bytes. The problem could be anywhere, but it's unlikely to be on your client end if you say it works for other providers. However it might be good to try a different client machine to see if it's restricted to a single client.
Author
Owner

@stale[bot] commented on GitHub (Jan 5, 2019):

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs during the next 2 weeks. Thank you for your contributions.

@stale[bot] commented on GitHub (Jan 5, 2019): This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs during the next 2 weeks. Thank you for your contributions.
Author
Owner

@ghost commented on GitHub (Jan 7, 2019):

I have the exact same issue. This is related to gzip.
When I disable ENABLE_GZIP in the config and the issue will be solved.
But when I enable the ENABLE_GZIP option, I won't be able to download some files via git-lfs or web (ERR_CONTENT_LENGTH_MISMATCH)

Similar issue: #4853

@ghost commented on GitHub (Jan 7, 2019): I have the exact same issue. This is related to gzip. When I disable `ENABLE_GZIP` in the config and the issue will be solved. But when I enable the `ENABLE_GZIP` option, I won't be able to download some files via git-lfs or web (ERR_CONTENT_LENGTH_MISMATCH) Similar issue: #4853
Author
Owner

@qzmfranklin commented on GitHub (Jan 8, 2019):

@yasuokav : Thanks for confirming and providing a workaround. This is infinitely helpful to us.

@qzmfranklin commented on GitHub (Jan 8, 2019): @yasuokav : Thanks for confirming and providing a workaround. This is infinitely helpful to us.
Author
Owner

@zeripath commented on GitHub (Jan 8, 2019):

An example file would really help in debugging this. If the problem files are all the same length that would also help identify this.

I am concerned that this implies there is either a buffer overflow or underrun here - those tend to be exploitable, so it would be helpful to get this identified. I guess we need to look at where gzip is enabled and see if there's an problem there.

@zeripath commented on GitHub (Jan 8, 2019): An example file would really help in debugging this. If the problem files are all the same length that would also help identify this. I am concerned that this implies there is either a buffer overflow or underrun here - those tend to be exploitable, so it would be helpful to get this identified. I guess we need to look at where gzip is enabled and see if there's an problem there.
Author
Owner

@ghost commented on GitHub (Jan 8, 2019):

@zeripath
1.zip
SHA1(1.png)= b9a88e4d43568845689750acbec93de346dbc8a4
Size: 39,683 bytes
Maybe Gitea sent the content-length of original file size but it's a gzipped response. Is this the problem?

Gitea(Gzip enabled):

Content-Encoding: gzip
Content-Length: 39683

Apache(Gzip enabled):

Content-Encoding: gzip
Content-Length: 39459
@ghost commented on GitHub (Jan 8, 2019): @zeripath [1.zip](https://github.com/go-gitea/gitea/files/2738381/1.zip) SHA1(1.png)= b9a88e4d43568845689750acbec93de346dbc8a4 Size: 39,683 bytes Maybe Gitea sent the content-length of original file size but it's a gzipped response. Is this the problem? Gitea(Gzip enabled): ``` Content-Encoding: gzip Content-Length: 39683 ``` Apache(Gzip enabled): ``` Content-Encoding: gzip Content-Length: 39459 ```
Author
Owner

@zeripath commented on GitHub (Jan 8, 2019):

Ah ok. That makes things interesting.

It appears that gziper.Gziper is supposed to reset the content-length.

I wonder if the lfs.getContentHandler is somehow overriding this.

@zeripath commented on GitHub (Jan 8, 2019): Ah ok. That makes things interesting. It appears that [gziper.Gziper](https://github.com/go-macaron/gzip/blob/master/gzip.go#L99) is supposed to reset the content-length. I wonder if the [lfs.getContentHandler](https://github.com/go-gitea/gitea/blob/eabbddcd98717ef20d8475e819f403c50f4a9787/modules/lfs/server.go#L171) is somehow overriding this.
Author
Owner

@zeripath commented on GitHub (Jan 8, 2019):

Certainly the range header code will not work in a gzip environment so even if this isn't relevant to the bug you're noticing it's definitely something.

@zeripath commented on GitHub (Jan 8, 2019): Certainly the range header code will not work in a gzip environment so even if this isn't relevant to the bug you're noticing it's definitely something.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/gitea#2452