move lfs to S3 (minio) #2652

New Issue

GiteaMirror · 2025-11-02T04:43:35-06:00

GiteaMirror commented

2025-11-02 04:43:35 -06:00

Originally created by @projectDaemon on GitHub (Dec 12, 2018).

It would be nice to have minio backend for lfs

Originally created by @projectDaemon on GitHub (Dec 12, 2018). It would be nice to have minio backend for lfs

GiteaMirror added the type/feature issue/confirmed labels 2025-11-02 04:43:35 -06:00

GiteaMirror closed this issue

2025-11-02 04:43:36 -06:00

GiteaMirror commented

2025-11-02 04:43:36 -06:00

@sapk commented on GitHub (Dec 13, 2018):

Is it to store lfs data via gitea in minio or use minio as external lfs backend ?

I haven't tested but external lfs backend can be set manually via :

git config -f .lfsconfig lfs.url https://my_other_server.example.com/foo/bar/info/lfs
git add .lfsconfig

@sapk commented on GitHub (Dec 13, 2018): Is it to store lfs data via gitea in minio or use minio as external lfs backend ? I haven't tested but external lfs backend can be set manually via : ``` git config -f .lfsconfig lfs.url https://my_other_server.example.com/foo/bar/info/lfs git add .lfsconfig ```

GiteaMirror commented

2025-11-02 04:43:36 -06:00

@lunny commented on GitHub (Dec 13, 2018):

I think he means store lfs files to s3 storage server but not local disk.

@lunny commented on GitHub (Dec 13, 2018): I think he means store lfs files to s3 storage server but not local disk.

GiteaMirror commented

2025-11-02 04:43:36 -06:00

@stale[bot] commented on GitHub (Feb 11, 2019):

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs during the next 2 weeks. Thank you for your contributions.

@stale[bot] commented on GitHub (Feb 11, 2019): This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs during the next 2 weeks. Thank you for your contributions.

GiteaMirror commented

2025-11-02 04:43:37 -06:00

@segevfiner commented on GitHub (Mar 12, 2019):

Rather than the minio library, which only supports S3 like providers, we can choose a library that supports multiple different cloud storage services (And S3 clones), wrapping it's configuration in Gitea's configuration. And if that library also supports local files, even better, as there will be no need to maintain two implementations. The local file implementation will have to scale properly, that is, support splitting the objects into subdirectories as currently done. And there is also the issue of migrating existing data if needed.

This needs to be something that the Gitea developers are comfortable using and having a proper license. Here are a few options:

(There may be more)

The configuration format for this is highly affected by the chosen library.

There is also.the issue of what key format to use. Cloud storage doesn't need the subdirectory splitting trick, though it probably doesn't hurt them either, and still using it there might allow easier migration using a simple recursive cp to the cloud storage provider.

I would love to hear Gitea's maintainers take on this.

@segevfiner commented on GitHub (Mar 12, 2019): Rather than the minio library, which only supports S3 like providers, we can choose a library that supports multiple different cloud storage services (And S3 clones), wrapping it's configuration in Gitea's configuration. And if that library also supports local files, even better, as there will be no need to maintain two implementations. The local file implementation will have to scale properly, that is, support splitting the objects into subdirectories as currently done. And there is also the issue of migrating existing data if needed. This needs to be something that the Gitea developers are comfortable using and having a proper license. Here are a few options: * https://github.com/google/go-cloud * https://github.com/graymeta/stow * https://github.com/lytics/cloudstorage (There may be more) The configuration format for this is highly affected by the chosen library. There is also.the issue of what key format to use. Cloud storage doesn't need the subdirectory splitting trick, though it probably doesn't hurt them either, and still using it there might allow easier migration using a simple recursive cp to the cloud storage provider. I would love to hear Gitea's maintainers take on this.

GiteaMirror commented

2025-11-02 04:43:37 -06:00

@clarfonthey commented on GitHub (Nov 25, 2019):

I agree that having some sort of object storage support for LFS would be really nice. S3-compatible APIs are one option, but really, having a generic API that allows us to plug in any cloud provider would be better.

@clarfonthey commented on GitHub (Nov 25, 2019): I agree that having some sort of object storage support for LFS would be really nice. S3-compatible APIs are one option, but really, having a generic API that allows us to plug in any cloud provider would be better.

GiteaMirror commented

2025-11-02 04:43:37 -06:00

@i0x71 commented on GitHub (Apr 27, 2020):

Was there any progress on this ?
This would be a killer feature.
If you could store all git data on minio that would be even better.
Gitea combined with minio distributed amd fault tolerant setup would be awesome.

@i0x71 commented on GitHub (Apr 27, 2020): Was there any progress on this ? This would be a killer feature. If you could store all git data on minio that would be even better. Gitea combined with minio distributed amd fault tolerant setup would be awesome.

GiteaMirror commented

2025-11-02 04:43:37 -06:00

@jolheiser commented on GitHub (Apr 27, 2020):

I believe this would be taken care of by https://github.com/go-gitea/gitea/pull/9567

@jolheiser commented on GitHub (Apr 27, 2020): I believe this would be taken care of by https://github.com/go-gitea/gitea/pull/9567

GiteaMirror commented

2025-11-02 04:43:38 -06:00

@i0x71 commented on GitHub (Apr 27, 2020):

Awesome, thanks

@i0x71 commented on GitHub (Apr 27, 2020): Awesome, thanks

GiteaMirror commented

2025-11-02 04:43:38 -06:00