Plugin system for obscure/non-universal features #923

@sapk commented on GitHub (Jul 28, 2017):

@lafriks I have to correct what I have said on compiled plugin and vendored deps : https://github.com/golang/go/issues/20481

This should be fix but not yet ready.

@sapk commented on GitHub (Jul 28, 2017): @lafriks I have to correct what I have said on compiled plugin and vendored deps : https://github.com/golang/go/issues/20481 This should be fix but not yet ready.

GiteaMirror commented

@sapk commented on GitHub (Jul 28, 2017):

Windows and macos plugin build are an objectif but not yet planned and no one on it. 😞
I would still suggest it and explain that it is only possible for linux host in the wait of golang improvement.
Maybe there are better solutions ?

@sapk commented on GitHub (Jul 28, 2017): Windows and macos plugin build are an objectif but not yet planned and no one on it. :disappointed: I would still suggest it and explain that it is only possible for linux host in the wait of golang improvement. Maybe there are better solutions ?

GiteaMirror commented

@andreynering commented on GitHub (Jul 28, 2017):

Gogs cross-reference: https://github.com/gogits/gogs/issues/2438

First, I completely agree the we should not bloat Gitea with more and more small features.

About the plugin system, it's also hard to implement. Go has interpreters for other languages we could look:

JavaScript: https://github.com/robertkrimen/otto
Lua: https://github.com/yuin/gopher-lua
Lua: https://github.com/Shopify/go-lua
Bash: https://github.com/mvdan/sh (this one may not be suitable for the plugin system, but for allowing running bash commands that will run on Windows, too)

We should look for an approach that is either easy to maintain in the Gitea core and also for plugin developers.

@andreynering commented on GitHub (Jul 28, 2017): Gogs cross-reference: https://github.com/gogits/gogs/issues/2438 First, I completely agree the we should not bloat Gitea with more and more small features. About the plugin system, it's also hard to implement. Go has interpreters for other languages we could look: - JavaScript: https://github.com/robertkrimen/otto - Lua: https://github.com/yuin/gopher-lua - Lua: https://github.com/Shopify/go-lua - Bash: https://github.com/mvdan/sh (this one may not be suitable for the plugin system, but for allowing running bash commands that will run on Windows, too) We should look for an approach that is either easy to maintain in the Gitea core and also for plugin developers.

GiteaMirror commented

@sapk commented on GitHub (Jul 28, 2017):

https://github.com/hashicorp/go-plugin suggested on gogs issue is also a good solution. It support some few good feature like negotiation for compatibility between main and plugins (and is planning "plugin fetching"). And plugin could be develop in various languages just need gRPC interface.

@sapk commented on GitHub (Jul 28, 2017): https://github.com/hashicorp/go-plugin suggested on gogs issue is also a good solution. It support some few good feature like negotiation for compatibility between main and plugins (and is planning "plugin fetching"). And plugin could be develop in various languages just need gRPC interface.

GiteaMirror commented

@sapk commented on GitHub (Jul 28, 2017):

gRPC permit support for plugin triggerring event and not wait to be called.

@sapk commented on GitHub (Jul 28, 2017): gRPC permit support for plugin triggerring event and not wait to be called.

GiteaMirror commented

@jonasfranz commented on GitHub (Jul 28, 2017):

I think that we only need a better filtering of features and we have to know what we want to implement and what not. But I disagree with you statement that #2211 is only used be a "small minority of users" because it makes gitea interesting for small businesses that need such features. And my apprehension is that the features will not be such good integrated or have a good quality if they come via plugin.
I agree that there should be a simple possibility to add small additions like #2221 but I do not think that it would be a good option for (in relation) big features like #2211.

@jonasfranz commented on GitHub (Jul 28, 2017): I think that we only need a better filtering of features and we have to know what we want to implement and what not. But I disagree with you statement that #2211 is only used be a "small minority of users" because it makes gitea interesting for small businesses that need such features. And my apprehension is that the features will not be such good integrated or have a good quality if they come via plugin. I agree that there should be a simple possibility to add small additions like #2221 but I do not think that it would be a good option for (in relation) big features like #2211.

GiteaMirror commented

@lunny commented on GitHub (Jul 29, 2017):

I agree with @JonasFranzDEV enhancement of issues will let small company remove their redmine and such others.

@lunny commented on GitHub (Jul 29, 2017): I agree with @JonasFranzDEV enhancement of issues will let small company remove their redmine and such others.

GiteaMirror commented

@ethantkoenig commented on GitHub (Jul 29, 2017):

@JonasFranzDEV @lunny Apologies if I came across as unfairly criticizing the timetracking feature. I didn't mean to say that the feature is useless, but only that it is peripheral to the main purpose of gitea, and that large subset of gitea users do not need it. I've updated the wording in my description accordingly.

I'm not sure "we just need to do a better job deciding what to implement" is a viable solution. Who decides what is worth implementing? What if user X needs feature Y, but it is decided that feature Y will not be implemented? User X's only option is to implement feature Y on his/her own fork, which is problematic as I described in the description.

Finally, I disagree with the "too big for a plugin" argument. The fact that the timetracking feature is large means it will add even more overhead and complexity to the codebase, which I believe further underscores the need for moving it to a plugin.

@ethantkoenig commented on GitHub (Jul 29, 2017): @JonasFranzDEV @lunny Apologies if I came across as unfairly criticizing the timetracking feature. I didn't mean to say that the feature is useless, but only that it is peripheral to the main purpose of gitea, and that large subset of gitea users do not need it. I've updated the wording in my description accordingly. I'm not sure "we just need to do a better job deciding what to implement" is a viable solution. Who decides what is worth implementing? What if user X needs feature Y, but it is decided that feature Y will not be implemented? User X's only option is to implement feature Y on his/her own fork, which is problematic as I described in the description. Finally, I disagree with the "too big for a plugin" argument. The fact that the timetracking feature is large means it will add even more overhead and complexity to the codebase, which I believe further underscores the need for moving it to a plugin.

GiteaMirror commented

@lafriks commented on GitHub (Jul 29, 2017):

@ethantkoenig plugins are good thing to have but also come with additional problems like it is harder to change core functionality/design/structure as such changes can and most probably will break plugins and is harder to test

@lafriks commented on GitHub (Jul 29, 2017): @ethantkoenig plugins are good thing to have but also come with additional problems like it is harder to change core functionality/design/structure as such changes can and most probably will break plugins and is harder to test

GiteaMirror commented

@sapk commented on GitHub (Jul 30, 2017):

@lafriks It also give us room for change because with too much function integrate in core we lock the structure of code and giving a standard interface (with versioning) for plugin will allow function to evolve at there own rythm. If a user want to keep a func implemented by a plugin that does not support the last version of core, it become the choice of the user to stay with an older version in the wait of an update of the plugin (or contribute to the update).

I think we all agree to had plugin, we just need to decide what type and how to limit bruden for gitea and plugins developer by starting a roadmap and defining interface and what a plugin can do.

@sapk commented on GitHub (Jul 30, 2017): @lafriks It also give us room for change because with too much function integrate in core we lock the structure of code and giving a standard interface (with versioning) for plugin will allow function to evolve at there own rythm. If a user want to keep a func implemented by a plugin that does not support the last version of core, it become the choice of the user to stay with an older version in the wait of an update of the plugin (or contribute to the update). I think we all agree to had plugin, we just need to decide what type and how to limit bruden for gitea and plugins developer by starting a roadmap and defining interface and what a plugin can do.

GiteaMirror commented

@lafriks commented on GitHub (Jul 30, 2017):

I agree that there could be plugins for webhooks/ci integration but for extending core functionality and UI it will be too much work to implement that correctly, especially to support multiple platforms etc. Also if plugins starts to be created that adds UI functionality we will start to get bug reports that will be hard to track down if bugs are only reproducible with plugin or combination of plugins enabled.

I think that we should continue to keep Gitea easy to install, run and packed everything is single binary. To make plugins right it would mean to rewrite gitea core to allow that and most probably it would be quite big task with no direct benefit.

@lafriks commented on GitHub (Jul 30, 2017): I agree that there could be plugins for webhooks/ci integration but for extending core functionality and UI it will be too much work to implement that correctly, especially to support multiple platforms etc. Also if plugins starts to be created that adds UI functionality we will start to get bug reports that will be hard to track down if bugs are only reproducible with plugin or combination of plugins enabled. I think that we should continue to keep Gitea easy to install, run and packed everything is single binary. To make plugins right it would mean to rewrite gitea core to allow that and most probably it would be quite big task with no direct benefit.

GiteaMirror commented

@tboerger commented on GitHub (Aug 21, 2017):

I just want to mention that we already had https://github.com/go-gitea/gitea/issues/661 ;)

@tboerger commented on GitHub (Aug 21, 2017): I just want to mention that we already had https://github.com/go-gitea/gitea/issues/661 ;)

GiteaMirror commented

2025-11-02 03:41:44 -06:00

@tboerger commented on GitHub (Aug 21, 2017):

Now my two cents for this topic. I wish we had an grpc based plugin system that is comparable to packer by hashicorp. Of course it's a lot of work, but we can even simplify the core and extract more parts into plugins. To avoid problems with plugin versions we have to apply versioned apis.

When we built the basics we can even offer some kind of marketplace where everybody can offer his stuff.

@tboerger commented on GitHub (Aug 21, 2017): Now my two cents for this topic. I wish we had an grpc based plugin system that is comparable to packer by hashicorp. Of course it's a lot of work, but we can even simplify the core and extract more parts into plugins. To avoid problems with plugin versions we have to apply versioned apis. When we built the basics we can even offer some kind of marketplace where everybody can offer his stuff.

GiteaMirror commented

@jonasfranz commented on GitHub (Aug 21, 2017):

@tboerger Will the plugins be compatible to newer / older "versioned apis" if gitea updates and the plugin not?
Which parts of the core should be extracted into plugins?

@jonasfranz commented on GitHub (Aug 21, 2017): @tboerger Will the plugins be compatible to newer / older "versioned apis" if gitea updates and the plugin not? Which parts of the core should be extracted into plugins?

GiteaMirror commented

2025-11-02 03:41:44 -06:00

@tboerger commented on GitHub (Aug 21, 2017):

@tboerger Will the plugins be compatible to newer / older "versioned apis" if gitea updates and the plugin not?

That's the whole thing of versioned APIs, older plugins should work with newer core versions, they just can not use newer core API parts.

Which parts of the core should be extracted into plugins?

No fixed list, maybe wikis, timetracking or other future additions like pages and so on. Keeping the core minimalistic and extracting many things into plugins should bring in some additional overhead, but every feature/plugin can be developed and published independent of the core as long as the core already provides the API for it.

@tboerger commented on GitHub (Aug 21, 2017): > @tboerger Will the plugins be compatible to newer / older "versioned apis" if gitea updates and the plugin not? That's the whole thing of versioned APIs, older plugins should work with newer core versions, they just can not use newer core API parts. > Which parts of the core should be extracted into plugins? No fixed list, maybe wikis, timetracking or other future additions like pages and so on. Keeping the core minimalistic and extracting many things into plugins should bring in some additional overhead, but every feature/plugin can be developed and published independent of the core as long as the core already provides the API for it.

GiteaMirror commented

2025-11-02 03:41:44 -06:00

@jonasfranz commented on GitHub (Oct 13, 2017):

I think that a concept like drone's plugin system might be a good idea. But we need a better API since we do not only provide "build steps". So my idea is that every plugin is started as docker container and connected to the plugin API via gRPC like @sapk pointed out (https://github.com/go-gitea/gitea/issues/2222#issuecomment-318653276). There is no installation process for the admin like JDK installation or other dependencies. But perhaps is this concept a little bit to complex.

@jonasfranz commented on GitHub (Oct 13, 2017): I think that a concept like drone's plugin system might be a good idea. But we need a better API since we do not only provide "build steps". So my idea is that every plugin is started as docker container and connected to the plugin API via gRPC like @sapk pointed out (https://github.com/go-gitea/gitea/issues/2222#issuecomment-318653276). There is no installation process for the admin like JDK installation or other dependencies. But perhaps is this concept a little bit to complex.

GiteaMirror commented

@daviian commented on GitHub (Oct 13, 2017):

Since we don't depend on Go 1.7 anymore, at least I thought so we could use the introducted go plugin architecture from 1.8.

A blogpost as an example: https://medium.com/learning-the-go-programming-language/writing-modular-go-programs-with-plugins-ec46381ee1a9

@daviian commented on GitHub (Oct 13, 2017): Since we don't depend on Go 1.7 anymore, at least I thought so we could use the introducted go plugin architecture from 1.8. A blogpost as an example: https://medium.com/learning-the-go-programming-language/writing-modular-go-programs-with-plugins-ec46381ee1a9

GiteaMirror commented

@lafriks commented on GitHub (Oct 13, 2017):

@daviian we can not use built-in golang plugin architecture as it is not supported on Windows and other gitea build targers

@lafriks commented on GitHub (Oct 13, 2017): @daviian we can not use built-in golang plugin architecture as it is not supported on Windows and other gitea build targers

GiteaMirror commented

@lunny commented on GitHub (Oct 14, 2017):

@daviian we should declare v1.3 only support go1.8+ on README or something place.

@lunny commented on GitHub (Oct 14, 2017): @daviian we should declare v1.3 only support go1.8+ on README or something place.

GiteaMirror commented

@tboerger commented on GitHub (Oct 14, 2017):

I would use some plugin system like packer, that's also grpc based and works also on Windows.

@tboerger commented on GitHub (Oct 14, 2017): I would use some plugin system like packer, that's also grpc based and works also on Windows.

GiteaMirror commented

@sapk commented on GitHub (Oct 17, 2017):

I could try to make a test with https://github.com/hashicorp/go-plugin. Any idea of a plugin that could be implemented ?
I was thinking of adding a file interpreter like markdown.

For the plugin architecture, I was thinking of allowing plugin to declare file handling based on regex/ file extension, declare url to be able to responds, declare event to listen for, be able to override some defined function.

@sapk commented on GitHub (Oct 17, 2017): I could try to make a test with https://github.com/hashicorp/go-plugin. Any idea of a plugin that could be implemented ? I was thinking of adding a file interpreter like markdown. For the plugin architecture, I was thinking of allowing plugin to declare file handling based on regex/ file extension, declare url to be able to responds, declare event to listen for, be able to override some defined function.

GiteaMirror commented

@lavvy commented on GitHub (Nov 20, 2017):

@sapk There are a ton of plugins here http://gitbucket-plugins.github.io/

@lavvy commented on GitHub (Nov 20, 2017): @sapk There are a ton of plugins here http://gitbucket-plugins.github.io/

GiteaMirror commented

@techknowlogick commented on GitHub (Jun 1, 2018):

Another plugin system we could consider is the one that Caddy uses: https://github.com/mholt/caddy/wiki/Extending-Caddy

For the webserver for the docs site it is already using Caddy and its plugin system: https://github.com/go-gitea/gitea/blob/master/docs/Dockerfile#L8

@techknowlogick commented on GitHub (Jun 1, 2018): Another plugin system we could consider is the one that Caddy uses: https://github.com/mholt/caddy/wiki/Extending-Caddy For the webserver for the docs site it is already using Caddy and its plugin system: https://github.com/go-gitea/gitea/blob/master/docs/Dockerfile#L8

GiteaMirror commented

@tboerger commented on GitHub (Jun 1, 2018):

A compile time plugin system is not something I really like

@tboerger commented on GitHub (Jun 1, 2018): A compile time plugin system is not something I really like

GiteaMirror commented

@stevegt commented on GitHub (Jul 9, 2018):

Trying to summarize this thread so far -- let me know what I'm missing. I see two viable alternatives for a plugin system:

A single-process internal plugin system similar to Caddy's, where everything runs in one process and each module registers its presence at startup. We could optionally refactor some or all of existing Gitea to make "everything a plugin". In addition to enabling unforseen features while keeping core code small, it enables less risk in refactoring major subsystems, since it encourages the use of internal API between modules.
A microservices architecture, where plugins each run in their own process and use the swagger API to interact with gitea and its db, which keeps gitea monolithic and its bugs relatively well-defined. This also means that, for instance, any plugin would have to maintain its own db for any custom tables or columns, and each plugin might best be run in its own container. We'd provide HTTP callback hooks so gitea can trigger plugins to do their jobs, and plugins would have to have some way of modifying Gitea's HTML.

Variations on these two themes might include runtime versus compile-time linking of the single-process system, or some RPC protocol other than HTTP for the multi-process system -- I'd describe a POSIX IPC plugin system as a variation of the second option, for instance. It would also be possible, though probably not desirable, to break existing Gitea up into microservices as well -- just tossing that out there as a strawman so it can get knocked down for performance reasons. ;-)

But in general, do these two pretty much cover all of the alternatives discussed so far? (I can't imagine any other category being computationally possible, but am willing to be surprised.) ;-)

If nobody can think of anything else, then we may want to split these two alternatives into two different issues, and see where they lead. Right now it's hard to tell which way the consensus is going, I think because we're actually talking about two completely different definitions of what a "plugin" even is.

Thoughts?

@stevegt commented on GitHub (Jul 9, 2018): Trying to summarize this thread so far -- let me know what I'm missing. I see two viable alternatives for a plugin system: - A single-process internal plugin system similar to Caddy's, where everything runs in one process and each module registers its presence at startup. We could optionally refactor some or all of existing Gitea to make "everything a plugin". In addition to enabling unforseen features while keeping core code small, it enables less risk in refactoring major subsystems, since it encourages the use of internal API between modules. - A microservices architecture, where plugins each run in their own process and use the swagger API to interact with gitea and its db, which keeps gitea monolithic and its bugs relatively well-defined. This also means that, for instance, any plugin would have to maintain its own db for any custom tables or columns, and each plugin might best be run in its own container. We'd provide HTTP callback hooks so gitea can trigger plugins to do their jobs, and plugins would have to have some way of modifying Gitea's HTML. Variations on these two themes might include runtime versus compile-time linking of the single-process system, or some RPC protocol other than HTTP for the multi-process system -- I'd describe a POSIX IPC plugin system as a variation of the second option, for instance. It would also be possible, though probably not desirable, to break existing Gitea up into microservices as well -- just tossing that out there as a strawman so it can get knocked down for performance reasons. ;-) But in general, do these two pretty much cover all of the alternatives discussed so far? (I can't imagine any other category being computationally possible, but am willing to be surprised.) ;-) If nobody can think of anything else, then we may want to split these two alternatives into two different issues, and see where they lead. Right now it's hard to tell which way the consensus is going, I think because we're actually talking about two completely different definitions of what a "plugin" even is. Thoughts?

GiteaMirror commented

@jonasfranz commented on GitHub (Jan 6, 2019):

I've tested the built-in plugin system of golang and discovered that it might fit very well in our use case. Plugins will be using the already existing database functions in models without having a seperate plugin api interface.

Plugins are going to be ~39MiB big since they include the gitea sources. This is a problem in my mind.
Any ideas on this?

@jonasfranz commented on GitHub (Jan 6, 2019): I've tested the built-in plugin system of golang and discovered that it might fit very well in our use case. Plugins will be using the already existing database functions in models without having a seperate plugin api interface. Plugins are going to be ~39MiB big since they include the gitea sources. This is a problem in my mind. Any ideas on this?

GiteaMirror commented

@tboerger commented on GitHub (Jan 6, 2019):

The builtin plugin system doesn't support windows, or has that changed?

@tboerger commented on GitHub (Jan 6, 2019): The builtin plugin system doesn't support windows, or has that changed?

GiteaMirror commented

Plugin support in v1.13

@Fardinak commented on GitHub (Dec 24, 2019):

Plugin support has indeed improved since 1.8, but Windows is still unsupported at this time and it doesn't seem like there's any active effort towards making it happen. However the community seems keen on accepting a pull request from someone who knows actually how to implement it…

Currently plugins are only supported on Linux and macOS.

Plugin support in master

Currently plugins are only supported on Linux, FreeBSD, and macOS.

And for reference, here's the issue regarding Windows support: golang/go#19282

Personally I'm mostly in favor of golang plugins or some sort of scripting. RPC just seems like too much trouble to manage. But in any case, I'm looking forward to plugin support.

@Fardinak commented on GitHub (Dec 24, 2019): Plugin support has indeed improved since 1.8, but Windows is still unsupported at this time and it doesn't seem like there's any active effort towards making it happen. However the community seems keen on accepting a pull request from someone who knows actually how to implement it… [Plugin support in `v1.13`](https://github.com/golang/go/blob/b0a2a817571c52341603d7d089c4138b048c67c3/src/plugin/plugin.go#L16) ``` Currently plugins are only supported on Linux and macOS. ``` [Plugin support in `master`](https://github.com/golang/go/blob/48ed1e61139d4089d8991e1387cec392f0b8378b/src/plugin/plugin.go#L16) ``` Currently plugins are only supported on Linux, FreeBSD, and macOS. ``` And for reference, here's the issue regarding Windows support: golang/go#19282 Personally I'm mostly in favor of golang plugins or some sort of scripting. RPC just seems like too much trouble to manage. But in any case, I'm looking forward to plugin support.

GiteaMirror commented

@zeripath commented on GitHub (Dec 24, 2019):

Oops, didn't mean to close this.

One thing I think would help would be a more extensible markdown renderer. I understand Hugo has moved to the goldmark renderer.

@zeripath commented on GitHub (Dec 24, 2019): Oops, didn't mean to close this. One thing I think would help would be a more extensible markdown renderer. I understand Hugo has moved to the goldmark renderer.

GiteaMirror commented

@tboerger commented on GitHub (Dec 26, 2019):

I still think it would make sense to implement the plugin api based on grpc and use the hashicorp plugin lib, this is already heavily battle tested.

@tboerger commented on GitHub (Dec 26, 2019): I still think it would make sense to implement the plugin api based on grpc and use the hashicorp plugin lib, this is already heavily battle tested.

GiteaMirror commented

@lunny commented on GitHub (Dec 27, 2019):

https://github.com/hashicorp/go-plugin allow only an executable binary plugin and running on the local network. So the plugin have be run on a sandbox(i.e. docker) to avoid it read/write filesystem/database directly.

@lunny commented on GitHub (Dec 27, 2019): https://github.com/hashicorp/go-plugin allow only an executable binary plugin and running on the local network. So the plugin have be run on a sandbox(i.e. docker) to avoid it read/write filesystem/database directly.

GiteaMirror commented

@sapk commented on GitHub (Dec 27, 2019):

@lunny I also think that the grpc solution is good because it allow local and remote. If plugin dev choose to acces database and file directly when run locally it is their concern. Maybe later we could choose to isolate (chroot, container, ...) local plugin but I think it would be extra first since it would be platform dependant.

@sapk commented on GitHub (Dec 27, 2019): @lunny I also think that the grpc solution is good because it allow local and remote. If plugin dev choose to acces database and file directly when run locally it is their concern. Maybe later we could choose to isolate (chroot, container, ...) local plugin but I think it would be extra first since it would be platform dependant.

GiteaMirror commented

@lunny commented on GitHub (Dec 27, 2019):

@sapk I don't think that's their concern. I think the security should be considered when we design the plugin system.
Once you choose to enable plugin system, gitea could require you installed docker or k8s. All plugins should declare their permissions(read disk/visit network and other etc.)

@lunny commented on GitHub (Dec 27, 2019): @sapk I don't think that's their concern. I think the security should be considered when we design the plugin system. Once you choose to enable plugin system, gitea could require you installed docker or k8s. All plugins should declare their permissions(read disk/visit network and other etc.)

GiteaMirror commented

@tboerger commented on GitHub (Dec 27, 2019):

I don't get your concerns... That doesn't make sense to me.

@tboerger commented on GitHub (Dec 27, 2019): I don't get your concerns... That doesn't make sense to me.

GiteaMirror commented

@lunny commented on GitHub (Dec 27, 2019):

@tboerger Plugins will run on your gitea server machine, it can do anything that's what my concerns. Could you explain why did you think that's not a problem?

@lunny commented on GitHub (Dec 27, 2019): @tboerger Plugins will run on your gitea server machine, it can do anything that's what my concerns. Could you explain why did you think that's not a problem?

GiteaMirror commented

@tboerger commented on GitHub (Dec 27, 2019):

They can do whatever they implement... That's the use case for plugins 😂

@tboerger commented on GitHub (Dec 27, 2019): They can do whatever they implement... That's the use case for plugins 😂

GiteaMirror commented

@zeripath commented on GitHub (Dec 27, 2019):

Depending on the level of plugin integration, there is a slight issue to do with plugins being run whilst there is an ongoing transaction with the db - but we'd just have to be explicit about when this happens. I would argue that level of integration would be too much to begin with.

If a server owner wants to blow themselves up by having a plugin that erases their entire db that's their call. We should however, try to proffer reasonable interfaces to the plugins to allow them to behave in a safe manner - so for example, offer sensible sanitisation services, perhaps offer some kind of limited js framework or even just best practices, or make it explicit which plugins extra js on the server - I dunno - I think the idea needs some concrete fleshing out to see what would need extra thought.

One thing that would make very tight plugin integration possible would be to move to a (configurable) dependency injection framework (There would be numerous benefits from this - including making it easier to test gitea properly). Go Wire is preferred by the go developers but I can't see how to make that make sense in a very configurable / plugin dominated system - something like a java Spring framework configurable file makes for very configurable system albeit with the costs of domain specific language. Again I dunno - just thoughts.

@zeripath commented on GitHub (Dec 27, 2019): Depending on the level of plugin integration, there is a slight issue to do with plugins being run whilst there is an ongoing transaction with the db - but we'd just have to be explicit about when this happens. I would argue that level of integration would be too much to begin with. If a server owner wants to blow themselves up by having a plugin that erases their entire db that's their call. We should however, try to proffer reasonable interfaces to the plugins to allow them to behave in a safe manner - so for example, offer sensible sanitisation services, perhaps offer some kind of limited js framework or even just best practices, or make it explicit which plugins extra js on the server - I dunno - I think the idea needs some concrete fleshing out to see what would need extra thought. One thing that would make very tight plugin integration possible would be to move to a (configurable) dependency injection framework (There would be numerous benefits from this - including making it easier to test gitea properly). Go Wire is preferred by the go developers but I can't see how to make that make sense in a very configurable / plugin dominated system - something like a java Spring framework configurable file makes for very configurable system albeit with the costs of domain specific language. Again I dunno - just thoughts.

GiteaMirror commented

@tboerger commented on GitHub (Dec 27, 2019):

Dependency injection doesn't exist in go... How to use wire you can see within drone.

Define some hook endpoints via interfaces, there drone could also give some advice. Start with first parts that can be extended and increase the hooks step by step

@tboerger commented on GitHub (Dec 27, 2019): Dependency injection doesn't exist in go... How to use wire you can see within drone. Define some hook endpoints via interfaces, there drone could also give some advice. Start with first parts that can be extended and increase the hooks step by step

GiteaMirror commented

@Fardinak commented on GitHub (Dec 27, 2019):

I think @lunny's concerns are valid, but that problem also exists with Go plugins or most other methods. I don't find requiring docker to be a viable option for a user that needs only a couple of plugins with their instance either. Also since OS support is clearly important, we need to remember that Docker still requires running a resource intensive VM on macOS.

Running plugins in a safe sandbox would be ideal. I consider WASI to be the ultimate solution, but unfortunately it just doesn't seem to be there yet. The closest option would be one of the interpreters mentioned by @andreynering:

About the plugin system, it's also hard to implement. Go has interpreters for other languages we could look:

* JavaScript: https://github.com/robertkrimen/otto
* Lua: https://github.com/yuin/gopher-lua
* Lua: https://github.com/Shopify/go-lua
* Bash: https://github.com/mvdan/sh (this one may not be suitable for the plugin system, but for allowing running bash commands that will run on Windows, too)

I must say, otto looks kinda promising.

Otherwise, it seems that GRPC is the most solid solution currently available. Paired with supervisord and it's RPC API (or maybe a golang implementation of it), they should provide a robust plugin solution for Gitea.

@Fardinak commented on GitHub (Dec 27, 2019): I think @lunny's concerns are valid, but that problem also exists with Go plugins or most other methods. I don't find requiring docker to be a viable option for a user that needs only a couple of plugins with their instance either. Also since OS support is clearly important, we need to remember that Docker still requires running a resource intensive VM on macOS. Running plugins in a safe sandbox would be ideal. I consider [WASI](https://github.com/WebAssembly/WASI) to be the ultimate solution, but unfortunately it just doesn't seem to be there yet. The closest option would be one of the interpreters [mentioned](https://github.com/go-gitea/gitea/issues/2222#issuecomment-318637397) by @andreynering: > About the plugin system, it's also hard to implement. Go has interpreters for other languages we could look: > > * JavaScript: https://github.com/robertkrimen/otto > * Lua: https://github.com/yuin/gopher-lua > * Lua: https://github.com/Shopify/go-lua > * Bash: https://github.com/mvdan/sh (this one may not be suitable for the plugin system, but for allowing running bash commands that will run on Windows, too) I must say, `otto` looks kinda promising. Otherwise, it seems that GRPC is the most solid solution currently available. Paired with supervisord and it's [RPC API](http://supervisord.org/api.html) (or maybe a [golang implementation](https://github.com/ochinchina/supervisord) of it), they should provide a robust plugin solution for Gitea.

GiteaMirror commented

@andreynering commented on GitHub (Dec 27, 2019):

Yaegi (a Go interpreter written in Go) is also a thing nowadays.

@andreynering commented on GitHub (Dec 27, 2019): [Yaegi](https://github.com/containous/yaegi) (a Go interpreter written in Go) is also a thing nowadays.

GiteaMirror commented

@Fardinak commented on GitHub (Dec 27, 2019):

Oh that looks really interesting. How feasible would it be to create a limited subset of the stdlib? Although I don't know how that would affect the rest of the libraries, it could for example help disable the plugins from accessing the filesystem, or limit them to only the interface provided by Gitea.

@Fardinak commented on GitHub (Dec 27, 2019): Oh that looks really interesting. How feasible would it be to create a limited subset of the stdlib? Although I don't know how that would affect the rest of the libraries, it could for example help disable the plugins from accessing the filesystem, or limit them to only the interface provided by Gitea.

GiteaMirror commented

@tboerger commented on GitHub (Dec 27, 2019):

With all the concerns you should continue with the mono binary. I will unsubscribe from this issue because I currently don't believe this will lead to something useful.

@tboerger commented on GitHub (Dec 27, 2019): With all the concerns you should continue with the mono binary. I will unsubscribe from this issue because I currently don't believe this will lead to something useful.

GiteaMirror commented

@Fardinak commented on GitHub (Dec 28, 2019):

Well it seems like the GRPC option makes sense to most. Let me know if I can be of help in any way. We're currently using Go+GRPC on a company project so I have some experience with it.

@Fardinak commented on GitHub (Dec 28, 2019): Well it seems like the GRPC option makes sense to most. Let me know if I can be of help in any way. We're currently using Go+GRPC on a company project so I have some experience with it.

GiteaMirror commented

@mewalig commented on GitHub (Jun 20, 2021):

Most of comments thus far seem to assume that the solution must be for gitea to dynamically load a module which is not incorporated into the original build of the gitea binary. That seems to have few advantages and many disadvantages, at least as a starting place.

What about just providing a format for modularizing the customization code, which allows for automating the integration of that code into the build process?

So instead of:

Build your gitea binary without customizations
Create your plug-in code, and build the plug-in binaries
Start gitea in some way that dynamically loads the above (assuming you can get this step to work)

you just do:

create your plug-in code, in some specific and known format (maybe this involves specific naming conventions, file + code snippet organization, etc)
Run a generic "merge" script that merges the plug-in code with the base code to generate composite code
Build your gitea binary from the composite code, creating a gitea binary with your customizations directly compiled in

This shifts the problem/challenge from dynamic loading, and all of the associated difficult problems, to a much simpler problem, which is to modify the build process to automatically merge the plug-in code with the base code, to generate the code that is ultimately compiled into the binary. This is not very hard, especially when one can easily define the initial scope to be as narrow as desired, and can also define the organizational structure of the new code to be whatever form is easiest to merge. Furthermore, this can be fairly easily done initially with plain text utilities, and while a more sophisticated approach to merging go code is not particularly trivial, it is also not that hard.

@mewalig commented on GitHub (Jun 20, 2021): Most of comments thus far seem to assume that the solution must be for gitea to dynamically load a module which is not incorporated into the original build of the gitea binary. That seems to have few advantages and many disadvantages, at least as a starting place. What about just providing a format for modularizing the customization code, which allows for automating the integration of that code into the build process? So instead of: 1. Build your gitea binary without customizations 2. Create your plug-in code, and build the plug-in binaries 3. Start gitea in some way that dynamically loads the above (assuming you can get this step to work) you just do: 1. create your plug-in code, in some specific and known format (maybe this involves specific naming conventions, file + code snippet organization, etc) 2. Run a generic "merge" script that merges the plug-in code with the base code to generate composite code 3. Build your gitea binary from the composite code, creating a gitea binary with your customizations directly compiled in This shifts the problem/challenge from dynamic loading, and all of the associated difficult problems, to a much simpler problem, which is to modify the build process to automatically merge the plug-in code with the base code, to generate the code that is ultimately compiled into the binary. This is not very hard, especially when one can easily define the initial scope to be as narrow as desired, and can also define the organizational structure of the new code to be whatever form is easiest to merge. Furthermore, this can be fairly easily done initially with plain text utilities, and while a more sophisticated approach to merging go code is not particularly trivial, it is also not that hard.

GiteaMirror commented

@lafriks commented on GitHub (Sep 17, 2021):

This could be an option: https://scriggo.com/

@lafriks commented on GitHub (Sep 17, 2021): This could be an option: https://scriggo.com/

GiteaMirror commented

@mewalig commented on GitHub (Oct 11, 2021):

I have a budget of a few hundred thousand USD for developing this gitea code base further, and that number is likely to go higher, and that does not include a decent amount of $ that has already been spent. It's not a huge sum of money, but it can certainly fund a heck of a lot of the requested improvements that have not moved very quickly on this project.

We have already implemented several features that have been requested but not implemented in this code base, and we'd be interested in contributing portions of our changes to this code base, if for no other reason than to benefit from the shared maintenance burden and to give back a bit.

However, the lack of a plug-in system is making it challenging for us to do that in a manner that is not cost-prohibitive.

Any thoughts / suggestions as to how this can be moved forward?

@mewalig commented on GitHub (Oct 11, 2021): I have a budget of a few hundred thousand USD for developing this gitea code base further, and that number is likely to go higher, and that does not include a decent amount of $ that has already been spent. It's not a huge sum of money, but it can certainly fund a heck of a lot of the requested improvements that have not moved very quickly on this project. We have already implemented several features that have been requested but not implemented in this code base, and we'd be interested in contributing portions of our changes to this code base, if for no other reason than to benefit from the shared maintenance burden and to give back a bit. However, the lack of a plug-in system is making it challenging for us to do that in a manner that is not cost-prohibitive. Any thoughts / suggestions as to how this can be moved forward?

GiteaMirror commented

@techknowlogick commented on GitHub (Oct 12, 2021):

We have already implemented several features that have been requested but not implemented in this code base, and we'd be interested in contributing portions of our changes to this code base, if for no other reason than to benefit from the shared maintenance burden and to give back a bit.

Even if it is a PR with your current code that may have merge conflicts that would be beneficial for the public good, or even just a description of what you have done and how you have implemented it would be good too.

As it seems you have invested $$ of your engineering time into building stuff for gitea for yourself, perhaps funding Gitea maintainers directly for maintenance of Gitea would be beneficial as well, so that you can ensure continued development and support of Gitea. Several of us have Github sponsors, liberapay's, and patreons listed in our profile.

Any thoughts / suggestions as to how this can be moved forward?

Several of us have previously engaged in contract work to expand/maintain Gitea if there is a specific contract you'd like to discuss my email is in my profile, although I can also connect you with others who may be interested as well.

@techknowlogick commented on GitHub (Oct 12, 2021): > We have already implemented several features that have been requested but not implemented in this code base, and we'd be interested in contributing portions of our changes to this code base, if for no other reason than to benefit from the shared maintenance burden and to give back a bit. Even if it is a PR with your current code that may have merge conflicts that would be beneficial for the public good, or even just a description of what you have done and how you have implemented it would be good too. As it seems you have invested $$ of your engineering time into building stuff for gitea for yourself, perhaps funding Gitea maintainers directly for maintenance of Gitea would be beneficial as well, so that you can ensure continued development and support of Gitea. Several of us have Github sponsors, liberapay's, and patreons listed in our profile. > Any thoughts / suggestions as to how this can be moved forward? Several of us have previously engaged in contract work to expand/maintain Gitea if there is a specific contract you'd like to discuss my email is in my profile, although I can also connect you with others who may be interested as well.

GiteaMirror commented

@mewalig commented on GitHub (Nov 23, 2021):

My company has made $ contributions to the project but we'd be more inclined to do so in larger numbers if there could be a prior understanding or discussion as to how those funds get used. I've tried some of the bounty websites and so far my experience with them has been "offputting", to put it mildly. Do the existing funding options allow bounties to be placed on specific feature requests? I have not seen that but maybe I'm missing.

@mewalig commented on GitHub (Nov 23, 2021): My company has made $ contributions to the project but we'd be more inclined to do so in larger numbers if there could be a prior understanding or discussion as to how those funds get used. I've tried some of the bounty websites and so far my experience with them has been "offputting", to put it mildly. Do the existing funding options allow bounties to be placed on specific feature requests? I have not seen that but maybe I'm missing.

GiteaMirror commented