settings refactoring #2400

Open
opened 2025-11-02 04:34:50 -06:00 by GiteaMirror · 11 comments
Owner

Originally created by @renothing on GitHub (Oct 11, 2018).

there are so many global options for gitea. and the variables grew up more and more. it's so boring for upgrading, user have to check every config variable before upgrade. maybe it's better to make a distinction. we can make two part:

  • 1: readonly: like database settings, ssh server settings, domains
  • 2: writeable: like smtp, security etc...

and only keep part 1 in config file or read from enviroments, write part2 into database with default values.
then we can keep the docker image clearly and upgrading safe than before.

What do you think?

Originally created by @renothing on GitHub (Oct 11, 2018). there are so many global options for gitea. and the variables grew up more and more. it's so boring for upgrading, user have to check every config variable before upgrade. maybe it's better to make a distinction. we can make two part: - 1: readonly: like database settings, ssh server settings, domains - 2: writeable: like smtp, security etc... and only keep part 1 in config file or read from enviroments, write part2 into database with default values. then we can keep the docker image clearly and upgrading safe than before. What do you think?
GiteaMirror added the type/proposaltype/refactoring labels 2025-11-02 04:34:50 -06:00
Author
Owner

@lafriks commented on GitHub (Oct 14, 2018):

I would like to see writeable settings moved to configurable storage (file, databse, etc)

@lafriks commented on GitHub (Oct 14, 2018): I would like to see writeable settings moved to configurable storage (file, databse, etc)
Author
Owner

@lunny commented on GitHub (May 9, 2023):

I think we already start moving part of configurations to database.

@lunny commented on GitHub (May 9, 2023): I think we already start moving part of configurations to database.
Author
Owner

@TheFox0x7 commented on GitHub (Dec 21, 2024):

I have something related to this proposal since the topic of refactoring settings is really broad. Can we consider taking a look at making better use of sections? Also using toml1 but that's another can of worms which should gets its own issue and discussion, if that hasn't happened yet.

To maybe show what I'm talking about take a look at ui settings and compare it to server. UI is more separated into sections making it smaller to look through, while server feels bloated and very large

Server would IMO benefit from making it more compact - for example all ssh related settings could easily be separated into server.ssh. Of course it's a two edge sword - too many sections make the overall readability worse.
Similar situation is with server.LFS_* parameters, shouldn't they be in [lfs], or at least [server.lfs]?
The other thing would be standardizing ENABLE_/DISABLE_ - to use only one of those. Personally I find ENABLE_CODE_PAGE = true more readable than than DISABLE_CODE_PAGE = false.

Is there a document/draft on the direction for settings? Maybe such ideas were discussed before and I missed it.


  1. main benefit would be possibility of having json-schema for settings and have taplo take advantage of it. Possibly even having a cue (or similar) layer to validate config or check for deprecated/invalid keys, which is related to https://github.com/go-gitea/gitea/issues/2762 ↩︎

@TheFox0x7 commented on GitHub (Dec 21, 2024): I have something related to this proposal since the topic of refactoring settings is really broad. Can we consider taking a look at making better use of sections? Also using toml[^1] but that's another can of worms which should gets its own issue and discussion, if that hasn't happened yet. To maybe show what I'm talking about take a look at [ui settings](https://docs.gitea.com/1.23/administration/config-cheat-sheet#ui-ui) and compare it to [server](https://docs.gitea.com/1.23/administration/config-cheat-sheet#server-server). UI is more separated into sections making it smaller to look through, while server feels bloated and very large Server would IMO benefit from making it more compact - for example all ssh related settings could easily be separated into `server.ssh`. Of course it's a two edge sword - too many sections make the overall readability worse. Similar situation is with `server.LFS_*` parameters, shouldn't they be in `[lfs]`, or at least `[server.lfs]`? The other thing would be standardizing `ENABLE_/DISABLE_` - to use only one of those. Personally I find `ENABLE_CODE_PAGE = true` more readable than than `DISABLE_CODE_PAGE = false`. Is there a document/draft on the direction for settings? Maybe such ideas were discussed before and I missed it. [^1]: main benefit would be possibility of having json-schema for settings and have [taplo](https://taplo.tamasfe.dev/) take advantage of it. Possibly even having a cue (or similar) layer to validate config or check for deprecated/invalid keys, which is related to https://github.com/go-gitea/gitea/issues/2762
Author
Owner

@wxiaoguang commented on GitHub (Dec 22, 2024):

Actually as now, question is not "whether we should do it", but actually "how to do it clearly without breaking and it must succeed".

Take the "removal of jQuery" as an example, everyone agrees, it is much simpler than "refactoring settings" (no need to consider about breaking), it still takes very long time and it still doesn't get full success, various regressions during the refactoring. The real challenge is that some legacy code needs to be completely rewritten (simple search&replace doesn't work), it seems that only me is the person who keeps really rewriting legacy code .........


The same question as jQuery removal: if we start the "settings refactoring", we can't stop halfway, otherwise it would be a bigger disaster.

@wxiaoguang commented on GitHub (Dec 22, 2024): Actually as now, question is not "whether we should do it", but actually "how to do it clearly without breaking and it must succeed". Take the "removal of jQuery" as an example, everyone agrees, it is much simpler than "refactoring settings" (no need to consider about breaking), it still takes very long time and it still doesn't get full success, various regressions during the refactoring. The real challenge is that some legacy code needs to be completely rewritten (simple search&replace doesn't work), it seems that only me is the person who keeps really rewriting legacy code ......... ---- The same question as jQuery removal: if we start the "settings refactoring", we can't stop halfway, otherwise it would be a bigger disaster.
Author
Owner

@wxiaoguang commented on GitHub (Dec 22, 2024):

And there are a lot of deprecated settings which "should be removed in 1.19/1.20/1.21/....", I had never seen who really started removing them

Please forgive me about my straightforward: are there enough developers having interests/time on this? It is not an easy task.

Screenshot: the deprecated settings

image

@wxiaoguang commented on GitHub (Dec 22, 2024): And there are a lot of deprecated settings which "should be removed in 1.19/1.20/1.21/....", I had never seen who really started removing them Please forgive me about my straightforward: are there enough developers having interests/time on this? It is not an easy task. <details> <summary>Screenshot: the deprecated settings</summary> ![image](https://github.com/user-attachments/assets/2dd3c9d3-f711-4180-80ba-0a024bdfb024) </details>
Author
Owner

@lunny commented on GitHub (Dec 22, 2024):

And there are a lot of deprecated settings which "should be removed in 1.19/1.20/1.21/....", I had never seen who really started removing them

Please forgive me about my straightforward: are there enough developers having interests/time on this? It is not an easy task.

Screenshot: the deprecated settings

image

I think I should add a task for releasing issue so that we can check them once we will release a new version.

@lunny commented on GitHub (Dec 22, 2024): > And there are a lot of deprecated settings which "should be removed in 1.19/1.20/1.21/....", I had never seen who really started removing them > > > > Please forgive me about my straightforward: are there enough developers having interests/time on this? It is not an easy task. > > > > > > <details> > > > > <summary>Screenshot: the deprecated settings</summary> > > > > ![image](https://github.com/user-attachments/assets/2dd3c9d3-f711-4180-80ba-0a024bdfb024) > > > > </details> I think I should add a task for releasing issue so that we can check them once we will release a new version.
Author
Owner

@wxiaoguang commented on GitHub (Dec 22, 2024):

I think I should add a task for releasing issue so that we can check them once we will release a new version.

Have you thought it through about how to improve the config options without breaking existing users?

@wxiaoguang commented on GitHub (Dec 22, 2024): > I think I should add a task for releasing issue so that we can check them once we will release a new version. Have you thought it through about how to improve the config options without breaking existing users?
Author
Owner

@lunny commented on GitHub (Dec 22, 2024):

How about moving most of the configurations to the database? We can have a notification or top bar about settings break changes for everyone and only the administrator can visit the configuration admin page to change them.

@lunny commented on GitHub (Dec 22, 2024): How about moving most of the configurations to the database? We can have a notification or top bar about settings break changes for everyone and only the administrator can visit the configuration admin page to change them.
Author
Owner

@wxiaoguang commented on GitHub (Dec 22, 2024):

How about moving most of the configurations to the database? We can have a notification or top bar about settings break changes for everyone and only the administrator can visit the configuration admin page to change them.

But I do not see you have the determination, for example: https://github.com/go-gitea/gitea/pull/32288#issuecomment-2421312408

@wxiaoguang commented on GitHub (Dec 22, 2024): > How about moving most of the configurations to the database? We can have a notification or top bar about settings break changes for everyone and only the administrator can visit the configuration admin page to change them. But I do not see you have the determination, for example: https://github.com/go-gitea/gitea/pull/32288#issuecomment-2421312408
Author
Owner

@TheFox0x7 commented on GitHub (Dec 22, 2024):

It would need an /api/admin route, though I think it already is a thing to read config. There's a benefit of that with being able to make a validation layer on the request and making the config resistant to misconfiguration - such as using a storage which doesn't exist, etc.

On the other hand, it makes reading and updating the configuration more difficult as it essentially makes 2 sources for it - the config which still needs to exist for init and the database for other parts, so the end user who would want an 1:1 replica would find it more difficult. And it would require an request/cli command instead of looking though template and restarting.
I'm not the largest fan of moving settings into a database though there are benefits of that. Is there any larger application which does this?

Also based on your comments it seems that config in db already is a thing, but it's the first I hear of it. I might not be only one who had no idea that there's something besides app.ini for configuration. (Found it, system_setting table)


The real challenge is that some legacy code needs to be completely rewritten (simple search&replace doesn't work), it seems that only me is the person who keeps really rewriting legacy code .........

I'd be happy to help but I don't have a good grasp at what's legacy and what isn't. If you have some areas which might be worth looking at refactors/rewrites I can take a look at them.

Have you thought it through about how to improve the config options without breaking existing users?

With app.ini one option would be to have a "simple" script in python or something which would move keys to their new places. Not the cleanest/greatest idea but an idea.

@TheFox0x7 commented on GitHub (Dec 22, 2024): It would need an /api/admin route, though I think it already is a thing to read config. There's a benefit of that with being able to make a validation layer on the request and making the config resistant to misconfiguration - such as using a storage which doesn't exist, etc. On the other hand, it makes reading and updating the configuration more difficult as it essentially makes 2 sources for it - the config which still needs to exist for init and the database for other parts, so the end user who would want an 1:1 replica would find it more difficult. And it would require an request/cli command instead of looking though template and restarting. I'm not the largest fan of moving settings into a database though there are benefits of that. Is there any larger application which does this? Also based on your comments it seems that config in db already is a thing, but it's the first I hear of it. I might not be only one who had no idea that there's something besides `app.ini` for configuration. (Found it, `system_setting` table) --- > The real challenge is that some legacy code needs to be completely rewritten (simple search&replace doesn't work), it seems that only me is the person who keeps really rewriting legacy code ......... I'd be happy to help but I don't have a good grasp at what's legacy and what isn't. If you have some areas which might be worth looking at refactors/rewrites I can take a look at them. > Have you thought it through about how to improve the config options without breaking existing users? With `app.ini` one option would be to have a "simple" script in python or something which would move keys to their new places. Not the cleanest/greatest idea but an idea.
Author
Owner

@wxiaoguang commented on GitHub (Dec 22, 2024):

Some brief thoughts about the problems (not deeply, just to share):

  1. We need a new INI file reader/writer first (I think we need to write it by ourselves)
    • It should be able to handle all legacy INI package quirks
    • It should be able to fully manage the INI files (including keeping the comments)
    • We could strictly check the config file to find all mistakes.
  2. Fully manage the "app.ini" by the Gitea program
    • Migrate legacy options to new sections if possible, or remove them
    • Report fatal error if an option is not migrate-able and let site admin to handle it manually (this is really a bad user experience and we should avoid it as much as possible)
  3. Classify the config options in "app.ini"
    • Prerequisite: without them, Gitea won't start: various paths, DB config, queue config, storage, some secrets, etc
    • Internal configs: although users could set them, actually they are only used by Gitea internally and could be fully managed by Gitea itself: internal token, jwt secret, etc
    • Database config options: for example: most UI settings, mailer, etc. They could be fully stored in database.
    • Git hook related configs: I haven't really looked into it, but I guess we need to support it: some options could/should be saved in database, but the git hook sub-command needs it, maybe we need to sync the config from database config to ini config and/or find some other approaches.

I think step 1 is a must, it could help to avoid various regressions. Without it, I have no confidence to refactor the setting system

@wxiaoguang commented on GitHub (Dec 22, 2024): Some brief thoughts about the problems (not deeply, just to share): 1. We need a new INI file reader/writer first (I think we need to write it by ourselves) * It should be able to handle all legacy INI package quirks * It should be able to fully manage the INI files (including keeping the comments) * We could strictly check the config file to find all mistakes. 2. Fully manage the "app.ini" by the Gitea program * Migrate legacy options to new sections if possible, or remove them * Report fatal error if an option is not migrate-able and let site admin to handle it manually (this is really a bad user experience and we should avoid it as much as possible) 3. Classify the config options in "app.ini" * Prerequisite: without them, Gitea won't start: various paths, DB config, queue config, storage, some secrets, etc * Internal configs: although users could set them, actually they are only used by Gitea internally and could be fully managed by Gitea itself: internal token, jwt secret, etc * Database config options: for example: most UI settings, mailer, etc. They could be fully stored in database. * Git hook related configs: I haven't really looked into it, but I guess we need to support it: some options could/should be saved in database, but the git hook sub-command needs it, maybe we need to sync the config from database config to ini config and/or find some other approaches. ---- I think step 1 is a must, it could help to avoid various regressions. Without it, I have no confidence to refactor the setting system
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/gitea#2400