move git backend behind an interface #11181

New Issue

GiteaMirror · 2025-11-02T09:30:00-06:00

GiteaMirror commented

2025-11-02 09:30:00 -06:00

Originally created by @6543 on GitHub (Jul 7, 2023).

The Issue

The current issue involves the handling of interactions with bare Git repositories.
These interactions are currently integrated throughout the entire codebase.

The challenge revolves around unifying interactions with bare Git repositories while accommodating various backends and storage locations (for addressing storage locations, refer to Issue #25070).

The main objective is to create a central interface that consolidates actions related to Git, allowing for their distribution across different services. The aim is not an immediate replacement of the current implementation, but rather to enable such a transition in the future if necessary.

This proposed solution would also lay the foundation for potentially separating the handling of git-bare-repositories into its own microservice at a later point.

This approach brings several benefits, including the ability for independent development and maintenance of various Git implementations, improved security through Separation of Concerns, and overall enhanced maintainability.

Furthermore, this approach could make Gitea more scalable in conjunction with Issue #25070, enabling the possibility of utilizing different microservices for different repositories, potentially divided by patterns or settings like the starting letters of repositories.

Proposed Solution

The proposed solution involves a phased implementation approach:

Initial Streamlining: The first step is to create a more concise implementation. This includes relocating the specific functions currently present in the modules/git package to a lower package level.

Interface Abstraction & Encapsulation: Following this, a fundamental interface will be established within the modules/git package at a higher package level. This interface will abstract essential functionalities, providing a clear structure for interaction.

Roadmap to Implementation

Module Restructuring: Move the Git module (e.g., module/git) to a submodule within module/git, and initiate the construction of the fundamental interface.
Interface Definition: Define the interface for Git operations (e.g., Git open) incrementally, piece by piece, at module/git.
Backend Segmentation: Segment the implementation of GoGit and NoGoGit, transitioning from using compile flags to run time settings.
Completing the Interface: Finalize the comprehensive interface and align it with the abstraction of Git storage sources (Issue #25070). This enables the assignment of distinct storage areas to various microservices based on repository names, enhancing scalability.
Integration Consideration: Evaluate the integration of libgit2 bindings as another backend behind a compile flag like sqlite support

tldr: This proposal aims to streamline our Git-Interaction in Gitea, leading to improved maintainability, flexibility, security and overall performance.

Originally created by @6543 on GitHub (Jul 7, 2023). # The Issue The current issue involves the handling of interactions with bare Git repositories. These interactions are currently integrated throughout the entire codebase. The challenge revolves around unifying interactions with bare Git repositories while accommodating various backends and storage locations (for addressing storage locations, refer to [Issue #25070](https://github.com/go-gitea/gitea/issues/25070)). The main objective is to create a central interface that consolidates actions related to Git, allowing for their distribution across different services. The aim is not an immediate replacement of the current implementation, but rather to enable such a transition in the future if necessary. This proposed solution would also lay the foundation for potentially separating the handling of git-bare-repositories into its own microservice at a later point. This approach brings several benefits, including the ability for independent development and maintenance of various Git implementations, improved security through Separation of Concerns, and overall enhanced maintainability. Furthermore, this approach could make Gitea more scalable in conjunction with [Issue #25070](https://github.com/go-gitea/gitea/issues/25070), enabling the possibility of utilizing different microservices for different repositories, potentially divided by patterns or settings like the starting letters of repositories. # Proposed Solution The proposed solution involves a phased implementation approach: **Initial Streamlining:** The first step is to create a more concise implementation. This includes relocating the specific functions currently present in the `modules/git` package to a lower package level. **Interface Abstraction & Encapsulation:** Following this, a fundamental interface will be established within the `modules/git` package at a higher package level. This interface will abstract essential functionalities, providing a clear structure for interaction. ## Roadmap to Implementation 1. **Module Restructuring:** Move the Git module (e.g., `module/git`) to a submodule within `module/git`, and initiate the construction of the fundamental interface. 2. **Interface Definition:** Define the interface for Git operations (e.g., Git open) incrementally, piece by piece, at `module/git`. 3. **Backend Segmentation:** Segment the implementation of GoGit and NoGoGit, transitioning from using compile flags to run time settings. 4. **Completing the Interface:** Finalize the comprehensive interface and align it with the abstraction of Git storage sources ([Issue #25070](https://github.com/go-gitea/gitea/issues/25070)). This enables the assignment of distinct storage areas to various microservices based on repository names, enhancing scalability. 5. **Integration Consideration:** Evaluate the integration of libgit2 bindings as another backend behind a compile flag like sqlite support **tldr: This proposal aims to streamline our Git-Interaction in Gitea, leading to improved maintainability, flexibility, security and overall performance.**

GiteaMirror added the type/refactoring type/summary labels 2025-11-02 09:30:00 -06:00

GiteaMirror commented

2025-11-02 09:30:01 -06:00

@6543 commented on GitHub (Jul 7, 2023):

this also might enable to use https://gitlab.com/gitlab-org/gitaly for us ... or split gitea int microservices ... (as one option)

@6543 commented on GitHub (Jul 7, 2023): this also might enable to use https://gitlab.com/gitlab-org/gitaly for us ... or split gitea int microservices ... (as one option)

GiteaMirror commented

2025-11-02 09:30:02 -06:00

@techknowlogick commented on GitHub (Jul 7, 2023):

closing as dupe of https://github.com/go-gitea/gitea/issues/25070

@techknowlogick commented on GitHub (Jul 7, 2023): closing as dupe of https://github.com/go-gitea/gitea/issues/25070

GiteaMirror commented

2025-11-02 09:30:02 -06:00

@6543 commented on GitHub (Aug 10, 2023):

Readun all the related issues&comments this is no dublicate ...
mentiined issue is about storage of git data to be more portable ... the other is about our git functionalitys to move them behind an interface

@6543 commented on GitHub (Aug 10, 2023): Readun all the related issues&comments this is no dublicate ... mentiined issue is about storage of git data to be more portable ... the other is about our git functionalitys to move them behind an interface

GiteaMirror commented

2025-11-02 09:30:02 -06:00

@6543 commented on GitHub (Aug 10, 2023):

Wip because of revormulate it

@6543 commented on GitHub (Aug 10, 2023): Wip because of revormulate it

GiteaMirror commented

2025-11-02 09:30:02 -06:00

@lunny commented on GitHub (Aug 11, 2023):

I don't think we need an abstract layer for all the git operations, we could just have one for git command.
The first important thing is #25071, we use reporootpath everywhere which will prevent the abstract layer.

@lunny commented on GitHub (Aug 11, 2023): I don't think we need an abstract layer for all the git operations, we could just have one for git command. The first important thing is #25071, we use reporootpath everywhere which will prevent the abstract layer.

GiteaMirror commented

2025-11-02 09:30:03 -06:00

@6543 commented on GitHub (Aug 11, 2023):

The problem with the git.Command is, that it allows arbitrary functionality to git ... yes we can do cover it as interface ... and I would do so first ... but it makes a hard requirement for the git binary and that that related git backend implementation. So in the long run I want to have it used only within the git modules specific implementation.

I agree that https://github.com/go-gitea/gitea/pull/25071 is higher in priority. This is about a long term goal, I want to refactor for the next versions ongoing (similar to how we migrated the http router) @lunny

PS: we already only use the git.Command in our code and not provide any external interaction (witch would be bad security wise) ... so it's just about moving that specific functions into the same package

@6543 commented on GitHub (Aug 11, 2023): The problem with the git.Command is, that it allows arbitrary functionality to git ... yes we can do cover it as interface ... and I would do so first ... but it makes a hard requirement for the git binary and that that related git backend implementation. So in the long run I want to have it used only within the git modules specific implementation. I agree that https://github.com/go-gitea/gitea/pull/25071 is higher in priority. This is about a long term goal, I want to refactor for the next versions ongoing (similar to how we migrated the http router) @lunny PS: we already only use the git.Command in our code and not provide any external interaction (witch would be bad security wise) ... so it's just about moving that specific functions into the same package

GiteaMirror commented

2025-11-02 09:30:03 -06:00

@wxiaoguang commented on GitHub (Aug 14, 2023):

IMO step 1 (Module Restructuring) is not necessary at the moment, while step 2 (Interface Definition) should be done first and needs more details.

The reason IMO is: Module Restructuring could be done at any time, even without Module Restructuring, the new package could be named as modules/gitapi or some funny name modules/gitter. After the new package works well, then "modules/git" could be moved/renamed easily and safely.

@wxiaoguang commented on GitHub (Aug 14, 2023): IMO step 1 (Module Restructuring) is not necessary at the moment, while step 2 (Interface Definition) should be done first and needs more details. The reason IMO is: `Module Restructuring` could be done at any time, even without Module Restructuring, the new package could be named as `modules/gitapi` or some funny name `modules/gitter`. After the new package works well, then "modules/git" could be moved/renamed easily and safely.

GiteaMirror commented

2025-11-02 09:30:03 -06:00

@6543 commented on GitHub (Aug 18, 2023):

not a bad idea too :)

I'll start with a smal pull and see if we can take it from there :D

@6543 commented on GitHub (Aug 18, 2023): not a bad idea too :) I'll start with a smal pull and see if we can take it from there :D

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/gitea#11181