Discussion of v1.15.x Roadmap #6766

@lunny commented on GitHub (Jan 27, 2021):

I will work on supporting Github Actions or implementing Gitea Bots . Another interesting topic is distributed git storage system.

@lunny commented on GitHub (Jan 27, 2021): I will work on supporting Github Actions or implementing Gitea Bots . Another interesting topic is distributed git storage system.

GiteaMirror commented

@lafriks commented on GitHub (Jan 27, 2021):

Yeah, I'm working on adding secret management that could be used for builds and probably other stuff also

@lafriks commented on GitHub (Jan 27, 2021): Yeah, I'm working on adding secret management that could be used for builds and probably other stuff also

GiteaMirror commented

@6543 commented on GitHub (Jan 27, 2021):

Added, @lafriks is there a issue or pull?
if not just alter the issue above and write the title into it

@6543 commented on GitHub (Jan 27, 2021): Added, @lafriks is there a issue or pull? if not just alter the issue above and write the title into it

GiteaMirror commented

@noerw commented on GitHub (Jan 27, 2021):

Might not hit most people's sentiment here, but:

I'd like to see a milestone heavily focusing on bugfixes & performance optimization.

With the current pace of the project it seems many more bugs are added than fixed, including some heavy performance regressions.
Which is worrysome, especially when considering that Gitea's selling points are 'it just works' and 'low-powered environment friendly'

@noerw commented on GitHub (Jan 27, 2021): Might not hit most people's sentiment here, but: I'd like to see a milestone heavily focusing on bugfixes & performance optimization. With the current pace of the project it seems many [more bugs](https://github.com/go-gitea/gitea/issues?q=is%3Aissue+is%3Aopen+label%3Akind%2Fbug) are added than fixed, including some heavy performance regressions. Which is worrysome, especially when considering that Gitea's selling points are 'it just works' and 'low-powered environment friendly'

GiteaMirror commented

@lunny commented on GitHub (Jan 27, 2021):

Might not hit most people's sentiment here, but:

I'd like to see a milestone heavily focusing on bugfixes & performance optimization.

With the current pace of the project it seems many more bugs are added than fixed, including some heavy performance regressions.
Which is worrysome, especially when considering that Gitea's selling points are 'it just works' and 'low-powered environment friendly'

I think v1.14 should do that. :)

@lunny commented on GitHub (Jan 27, 2021): > Might not hit most people's sentiment here, but: > > I'd like to see a milestone heavily focusing on bugfixes & performance optimization. > > With the current pace of the project it seems many [more bugs](https://github.com/go-gitea/gitea/issues?q=is%3Aissue+is%3Aopen+label%3Akind%2Fbug) are added than fixed, including some heavy performance regressions. > Which is worrysome, especially when considering that Gitea's selling points are 'it just works' and 'low-powered environment friendly' I think v1.14 should do that. :)

GiteaMirror commented

@nedvedad commented on GitHub (Jan 27, 2021):

my request:

Automated Kanban board #14359

(who would've guessed given that I even opened a feature request for it…)

I second this.

There are also a couple of simple proposals related to projects, that would greatly increase their usability:

@nedvedad commented on GitHub (Jan 27, 2021): > my request: > > * Automated Kanban board #14359 > > (who would've guessed given that I even opened a feature request for it…) I second this. There are also a couple of simple proposals related to projects, that would greatly increase their usability: - https://github.com/go-gitea/gitea/issues/14298 - https://github.com/go-gitea/gitea/issues/13814 - https://github.com/go-gitea/gitea/issues/13520 - https://github.com/go-gitea/gitea/issues/13519 - https://github.com/go-gitea/gitea/issues/13518 - https://github.com/go-gitea/gitea/issues/13517

GiteaMirror commented

@lafriks commented on GitHub (Jan 27, 2021):

Added, @lafriks is there a issue or pull?
if not just alter the issue above and write the title into it

@6543 I created draft PR #14483

@lafriks commented on GitHub (Jan 27, 2021): > Added, @lafriks is there a issue or pull? > if not just alter the issue above and write the title into it @6543 I created draft PR #14483

GiteaMirror commented

@a1012112796 commented on GitHub (Jan 27, 2021):

my draft pulls:

reply issue by email (#13585)(will finish test code soon)
agit flow (#14295)

FRs:

bot account(or something like github apps) #13044
More vue ....

@a1012112796 commented on GitHub (Jan 27, 2021): my draft pulls: - reply issue by email (#13585)(will finish test code soon) - agit flow (#14295) FRs: - bot account(or something like github apps) #13044 - More vue ....

GiteaMirror commented

@jpraet commented on GitHub (Jan 27, 2021):

My request is for improving the pull request experience:

Mark files as "Viewed" #11730 + new changes since you last viewed (like on github)
Suggesting code changes #9388
Fix false positives in merge conflict detection #13679
Resolve merge conflicts from web UI #9014
Cherry-pick commit to other branch #3880
Reverting a pull request #6375

@jpraet commented on GitHub (Jan 27, 2021): My request is for improving the pull request experience: * Mark files as "Viewed" #11730 + new changes since you last viewed (like on github) * Suggesting code changes #9388 * Fix false positives in merge conflict detection #13679 * Resolve merge conflicts from web UI #9014 * Cherry-pick commit to other branch #3880 * Reverting a pull request #6375

GiteaMirror commented

@delvh commented on GitHub (Jan 28, 2021):

As an only partially related side note, for 1.15 maybe even #1029 can finally be closed, as the dependency of the last remaining unchecked action (#12244) has been merged…
Or is that already intended for 1.14?

@delvh commented on GitHub (Jan 28, 2021): As an only partially related side note, for 1.15 maybe even #1029 can finally be closed, as the dependency of the last remaining unchecked action (#12244) has been merged… Or is that already intended for 1.14?

GiteaMirror commented

@lunny commented on GitHub (Jan 28, 2021):

As an only partially related side note, for 1.15 maybe even #1029 can finally be closed, as the dependency of the last remaining unchecked action (#12244) has been merged…
Or is that already intended for 1.14?

Yeah. Maybe 1.14 or 1.15 which depends on how long we should spend on the migrations.

@lunny commented on GitHub (Jan 28, 2021): > As an only partially related side note, for 1.15 maybe even #1029 can finally be closed, as the dependency of the last remaining unchecked action (#12244) has been merged… > Or is that already intended for 1.14? Yeah. Maybe 1.14 or 1.15 which depends on how long we should spend on the migrations.

GiteaMirror commented

@a96219 commented on GitHub (Jan 29, 2021):

About TOC tags
Directories are a relatively important feature for long documents
#7225
#7182

@a96219 commented on GitHub (Jan 29, 2021): About TOC tags Directories are a relatively important feature for long documents #7225 #7182

GiteaMirror commented

@OCram85 commented on GitHub (Jan 30, 2021):

It really would be awesome, if wiki pages could be managed via the REST API 😃

@OCram85 commented on GitHub (Jan 30, 2021): It really would be awesome, if wiki pages could be managed via the REST API :smiley:

GiteaMirror commented

@jsil3601 commented on GitHub (Feb 1, 2021):

My suggestion https://github.com/go-gitea/gitea/issues/2529 wildcard protected branches

@jsil3601 commented on GitHub (Feb 1, 2021): My suggestion https://github.com/go-gitea/gitea/issues/2529 wildcard protected branches

GiteaMirror commented

@Reinitialized commented on GitHub (Feb 9, 2021):

Below is a list of features I consider to essential quality of life improvements. Most of these features are already implemented in either GitHub or GitLab, and thus use them for references on how I consider the feature to be implemented.

#14622 #11888 #14714 #14717 #10808 #14776 #12120 #15577

As I play around with and integrate Gitea into my productivity flow, I'll add more here

@Reinitialized commented on GitHub (Feb 9, 2021): Below is a list of features I consider to essential quality of life improvements. Most of these features are already implemented in either GitHub or GitLab, and thus use them for references on how I consider the feature to be implemented. ___ #14622 #11888 #14714 #14717 #10808 #14776 #12120 #15577 ___ As I play around with and integrate Gitea into my productivity flow, I'll add more here

GiteaMirror commented

@zeripath commented on GitHub (Feb 11, 2021):

@noerw I agree. We've unfortunately made a very bad call with defaulting to argon2 which has caused a lot of problems.

I have ummed and ahhed about whether we should change the default back and I've come to the conclusion that I've spent way too much time dealing with it so I think we should just drop that.

My recent work on returning to git should also represent a significant improvement in memory use. (Albeit I need to think more about lastcommitcache.)

In terms of performance I think we definitely should focus on pushing more things out to Ajax rendering.

I'm more than happy to work with people looking specifically at performance prs though. So In particular I think we desperately need to look at two things:

The history / lastcommitcache should not hold up rendering it should be ajaxed in like in gitlab.
Diff pages need to infinite scrolled and ajaxed in to.

In terms of bugs - both the git and chi prs are likely to be a source of a few bugs. We really need to put some effort into looking for regressions due to these before 1.14 is released.

@zeripath commented on GitHub (Feb 11, 2021): @noerw I agree. We've unfortunately made a very bad call with defaulting to argon2 which has caused a lot of problems. I have ummed and ahhed about whether we should change the default back and I've come to the conclusion that I've spent way too much time dealing with it so I think we should just drop that. My recent work on returning to git should also represent a significant improvement in memory use. (Albeit I need to think more about lastcommitcache.) In terms of performance I think we definitely should focus on pushing more things out to Ajax rendering. I'm more than happy to work with people looking specifically at performance prs though. So In particular I think we desperately need to look at two things: * The history / lastcommitcache should not hold up rendering it should be ajaxed in like in gitlab. * Diff pages need to infinite scrolled and ajaxed in to. In terms of bugs - both the git and chi prs are likely to be a source of a few bugs. We really need to put some effort into looking for regressions due to these before 1.14 is released.

GiteaMirror commented

@yoyoyonas commented on GitHub (Apr 20, 2021):

Regarding API feature completeness, I'd propose to add #14248 to the list.

@yoyoyonas commented on GitHub (Apr 20, 2021): Regarding API feature completeness, I'd propose to add #14248 to the list.

GiteaMirror commented

@vchirkov commented on GitHub (Apr 21, 2021):

It would be really awesome if create from template would be available from API #9126

@vchirkov commented on GitHub (Apr 21, 2021): It would be really awesome if create from template would be available from API #9126

GiteaMirror commented

@yoyoyonas commented on GitHub (Apr 21, 2021):

Regarding API feature completeness, I'd propose to add #14248 to the list.

I just opened #15571 to which the same applies.

@yoyoyonas commented on GitHub (Apr 21, 2021): > Regarding API feature completeness, I'd propose to add #14248 to the list. I just opened #15571 to which the same applies.

GiteaMirror commented

@dcorral commented on GitHub (May 4, 2021):

Add admin email notifications on new user registration and user notification for activated accounts would be just amazing #15147
😍😍

@dcorral commented on GitHub (May 4, 2021): Add admin email notifications on new user registration and user notification for activated accounts would be just amazing #15147 😍😍

GiteaMirror commented

https://github.com/go-gitea/gitea/pull/15547
https://github.com/go-gitea/gitea/pull/15180
https://github.com/go-gitea/gitea/pull/14054

@zeripath commented on GitHub (May 9, 2021):

I suppose I should update with some of the work I've been doing.

Most of my recent PRs have been looking specifically at issues of performance.

git performance

Unfortunately Windows doesn't tolerate calling git processes as much as Linux so the native git PR has had a detrimental effect for them - but I do have good information that the native git backend has substantially reduced memory load in one of our larger users. #15482 means that gogit variants are built for Windows.

I've written several PRs eg #15455 that have sped up the native git backend, but most notably #15667 will have significant improvements once it's merged. #15653 will improve things gogit users too. There were a couple of other smaller improvement to commit list info - but it seems that there are likely still a few issues there which will need better adjustments to the algorithm.

I've written a pr to gogit to improve its memory use (https://github.com/go-git/go-git/pull/303) ~~but I need to look at it again and double check the caching issues~~ (I think this is now fine). I think we'll end up having to keep both backends around though.

It would great if people could try

PR checker

Another performance pr that's waiting review is #15314. This will significantly speed up the conflict testing of PRs — especially if they contain large binary data.

Background baseline load

One other common complaint is that Gitea now has too high a background unused load. I've put several PRs in which are looking at this: #15724, #15696, #15725 and #15693. (Last two are still awaiting to be merged) With these patches and a change to configuration we should find a quiescent gitea dropping back down to almost no background load. #15693 seriously drops the number of background goroutines from >150 to 50 with current default config. Once the last two are in - I will propose a pr to change the default config to match the low load option with a background goroutine count of 29. More can be done however, but not until those two PRs are in.

In a related vein #15733 fixes an issue with flushing loops in queues which may loop rapidly without it. This represents more of an issue in testing than real world but it's still an improvement.

Windows SVC

The Windows SVC control of Gitea suffered a regression due to moving to IsWindowsService from the now deprecated IsAnInteractiveSession. In the end we've had to move back to use the deprecated function. (#15749)

Authentication and routing

One of the downgrades with the move to chi is that the stack became much larger due to the use of http.Handlers. #15301 represents a refactor reducing these. There should be small performance improvements because part of the problem with the handlers is their repeated individual testing of the URL instead of using a trie to be efficient.

Another problem we have is that the "SSO" module has resulted in authentication occurring twice for git & lfs paths. This duplication of code is error prone and they both do things slightly differently and they both may result in significant loading and hashing issues. #15303 will stop this. It's also apparent that ReverseProxy users aren't having a session created for them on the UI which #15304 will fix. I would like to move this slightly misnamed package out of modules/auth/SSO and into services/auth and reintegrate a lot of the URL checking that occurs in these functions back into routers but I'm held back by these two PRs. I have added a pr to add trace logging to this authentication system #15803 which will help with tracing bugs in this.

One other pr that is speculative is the idea of limiting the number of concurrent hashes. I think this is part of the way we can safely switch to allow argon2 but I can understand the reluctance to merge it right now #14713 . Any move to argon2 in future would mean that we would also need to disallow basic authentication.

In order to make disabling Basic Authentication by default workable a pr I would like to make would be in creating specific oauth2 grants on a per repo push pull basis. But progress is stalled here by the other two PRs.

app.example.ini

A common cause of problems has been the copying of app.example.ini as app.ini. This was an unintended use. Today I worked on commenting out most of this file so the documentation component is correct and users can then simply copy it as app.ini if they do desire. (You probably shouldn't though as it's a big file and big files are always slow to parse.) #15807

Better cross db dumps

One of the things I've worked on recently is within gitea dump. Although we provide the option to cross db dump it's not entirely perfect. One of the issues is that the dumper converts the db schema without reference to the objects. This leads to problems with bools being incorrectly converted properly. The next version of xorm should contain a cleverer dump that references the object that to generate the target schema.

other features

The following are feature PRs are complete and awaiting:

@zeripath commented on GitHub (May 9, 2021): I suppose I should update with some of the work I've been doing. Most of my recent PRs have been looking specifically at issues of performance. ### git performance Unfortunately Windows doesn't tolerate calling git processes as much as Linux so the native git PR has had a detrimental effect for them - but I do have good information that the native git backend has substantially reduced memory load in one of our larger users. #15482 means that gogit variants are built for Windows. I've written several PRs eg #15455 that have sped up the native git backend, but most notably #15667 will have significant improvements once it's merged. #15653 will improve things gogit users too. There were a couple of other smaller improvement to commit list info - but it seems that there are likely still a few issues there which will need better adjustments to the algorithm. I've written a pr to gogit to improve its memory use (https://github.com/go-git/go-git/pull/303) ~~but I need to look at it again and double check the caching issues~~ (I think this is now fine). I think we'll end up having to keep both backends around though. It would great if people could try ### PR checker Another performance pr that's waiting review is #15314. This will significantly speed up the conflict testing of PRs — especially if they contain large binary data. ### Background baseline load One other common complaint is that Gitea now has too high a background unused load. I've put several PRs in which are looking at this: #15724, #15696, #15725 and #15693. (Last two are still awaiting to be merged) With these patches and a change to configuration we should find a quiescent gitea dropping back down to almost no background load. #15693 seriously drops the number of background goroutines from >150 to 50 with current default config. Once the last two are in - I will propose a pr to change the default config to match the low load option with a background goroutine count of 29. More can be done however, but not until those two PRs are in. In a related vein #15733 fixes an issue with flushing loops in queues which may loop rapidly without it. This represents more of an issue in testing than real world but it's still an improvement. ### Windows SVC The Windows SVC control of Gitea suffered a regression due to moving to `IsWindowsService` from the now deprecated `IsAnInteractiveSession`. In the end we've had to move back to use the deprecated function. (#15749) ### Authentication and routing One of the downgrades with the move to chi is that the stack became much larger due to the use of http.Handlers. #15301 represents a refactor reducing these. There should be small performance improvements because part of the problem with the handlers is their repeated individual testing of the URL instead of using a trie to be efficient. Another problem we have is that the "SSO" module has resulted in authentication occurring twice for git & lfs paths. This duplication of code is error prone and they both do things slightly differently and they both may result in significant loading and hashing issues. #15303 will stop this. It's also apparent that ReverseProxy users aren't having a session created for them on the UI which #15304 will fix. I would like to move this slightly misnamed package out of modules/auth/SSO and into services/auth and reintegrate a lot of the URL checking that occurs in these functions back into routers but I'm held back by these two PRs. I have added a pr to add trace logging to this authentication system #15803 which will help with tracing bugs in this. One other pr that is speculative is the idea of limiting the number of concurrent hashes. I think this is part of the way we can safely switch to allow argon2 but I can understand the reluctance to merge it right now #14713 . Any move to argon2 in future would mean that we would also need to disallow basic authentication. In order to make disabling Basic Authentication by default workable a pr I would like to make would be in creating specific oauth2 grants on a per repo push pull basis. But progress is stalled here by the other two PRs. ### app.example.ini A common cause of problems has been the copying of app.example.ini as app.ini. This was an unintended use. Today I worked on commenting out most of this file so the documentation component is correct and users can then simply copy it as app.ini if they do desire. (You probably shouldn't though as it's a big file and big files are always slow to parse.) #15807 ### Better cross db dumps One of the things I've worked on recently is within gitea dump. Although we provide the option to cross db dump it's not entirely perfect. One of the issues is that the dumper converts the db schema without reference to the objects. This leads to problems with bools being incorrectly converted properly. The next version of xorm should contain a cleverer dump that references the object that to generate the target schema. ### other features The following are feature PRs are complete and awaiting: https://github.com/go-gitea/gitea/pull/15547 https://github.com/go-gitea/gitea/pull/15180 https://github.com/go-gitea/gitea/pull/14054

GiteaMirror commented