Proposal: remove unpopular licenses #14080

New Issue

GiteaMirror · 2025-11-02T11:02:20-06:00

GiteaMirror commented

2025-11-02 11:02:20 -06:00

Originally created by @wxiaoguang on GitHub (Feb 1, 2025).

There are more than 720 licenses in Gitea https://github.com/go-gitea/gitea/tree/main/options/license

Most of them are out-dated/inactive/unpopular

We could just keep about 20-40 popular licenses:

Benefits:

reduce binary size
speed up execution time
save memory
- especially the License Detector, it consumes more than 100MB memory at the moment
- by removing most unnecessary licenses, License Detector could only consume 3-5MB memory then

Originally created by @wxiaoguang on GitHub (Feb 1, 2025). There are more than 720 licenses in Gitea https://github.com/go-gitea/gitea/tree/main/options/license Most of them are out-dated/inactive/unpopular We could just keep about 20-40 popular licenses: * https://innovationgraph.github.com/global-metrics/licenses * https://github.com/github/innovationgraph/blob/main/data/licenses.csv Benefits: * reduce binary size * speed up execution time * save memory * especially the License Detector, it consumes more than 100MB memory at the moment * by removing most unnecessary licenses, License Detector could only consume 3-5MB memory then

GiteaMirror added the proposal/accepted type/proposal labels 2025-11-02 11:02:20 -06:00

GiteaMirror closed this issue

2025-11-02 11:02:20 -06:00

GiteaMirror commented

2025-11-02 11:02:21 -06:00

@delvh commented on GitHub (Feb 2, 2025):

Another benefit:

users are not overloaded with possible options anymore (I've often asked myself "which of these licenses could be the one I want")

@delvh commented on GitHub (Feb 2, 2025): Another benefit: - users are not overloaded with possible options anymore (I've often asked myself "which of these licenses could be the one I want")

GiteaMirror commented

2025-11-02 11:02:21 -06:00

@wxiaoguang commented on GitHub (Feb 3, 2025):

So, as the first step, need to stop this:

@lunny @techknowlogick

@wxiaoguang commented on GitHub (Feb 3, 2025): So, as the first step, need to stop this: @lunny @techknowlogick ![Image](https://github.com/user-attachments/assets/5b8e468b-0642-40a9-a52c-5a66e15ed050)

GiteaMirror commented

2025-11-02 11:02:21 -06:00

@lunny commented on GitHub (Feb 3, 2025):

So, as the first step, need to stop this:

@lunny @techknowlogick

A pull request could be sent to change cron-licenses.yml to only manually. #33486

@lunny commented on GitHub (Feb 3, 2025): > So, as the first step, need to stop this: > > [<img alt="" width="16" height="16" src="https://avatars.githubusercontent.com/u/81045?s=64&u=99b64f0ca6ef63643c7583ab87dd31c52d28e673&v=4">@lunny](https://github.com/lunny) [<img alt="" width="16" height="16" src="https://avatars.githubusercontent.com/techknowlogick?size=32">@techknowlogick](https://github.com/techknowlogick) > > ![Image](https://github.com/user-attachments/assets/5b8e468b-0642-40a9-a52c-5a66e15ed050) A pull request could be sent to change `cron-licenses.yml` to only manually. #33486

GiteaMirror commented

2025-11-02 11:02:21 -06:00

@silverwind commented on GitHub (Feb 3, 2025):

especially the License Detector, it consumes more than 100MB memory at the moment

100MB persistent? Excluding actual license data, it ideally should have close to zero persistent memory, only run when needed (e.g. when a repo was pushed to, does not matter if license update takes a few seconds in such cases, ideally it start off a lazy goroutine, debounced on like ~10s of repo push inactivity).

@silverwind commented on GitHub (Feb 3, 2025): > especially the License Detector, it consumes more than 100MB memory at the moment 100MB persistent? Excluding actual license data, it ideally should have close to zero persistent memory, only run when needed (e.g. when a repo was pushed to, does not matter if license update takes a few seconds in such cases, ideally it start off a lazy goroutine, debounced on like ~10s of repo push inactivity).

GiteaMirror commented

2025-11-02 11:02:21 -06:00

@silverwind commented on GitHub (Feb 3, 2025):

20-40 popular licenses

How about keeping 50 most popular? Gitea's usersbase is diverse, 20 sounds definitely too low.

@silverwind commented on GitHub (Feb 3, 2025): > 20-40 popular licenses How about keeping 50 most popular? Gitea's usersbase is diverse, 20 sounds definitely too low.

GiteaMirror commented

2025-11-02 11:02:22 -06:00

@eeyrjmr commented on GitHub (Feb 7, 2025):

20-40 popular licenses

How about keeping 50 most popular? Gitea's usersbase is diverse, 20 sounds definitely too low.

maybe...
There only really appears to be ~30 that are really used on github,

import pandas as pd
url= "https://raw.githubusercontent.com/github/innovationgraph/refs/heads/main/data/licenses.csv"
tables= pd.read_csv(url)
tables[(tables.year == 2024)].pivot_table(values="num_pushers", index="spdx_license",aggfunc="sum").sort_values('num_pushers',ascending=False).plot(kind='barh',figsize=(9,9))

2024 data

2023 data

@eeyrjmr commented on GitHub (Feb 7, 2025): > > 20-40 popular licenses > > How about keeping 50 most popular? Gitea's usersbase is diverse, 20 sounds definitely too low. maybe... There only really appears to be ~30 that are really used on github, ``` import pandas as pd url= "https://raw.githubusercontent.com/github/innovationgraph/refs/heads/main/data/licenses.csv" tables= pd.read_csv(url) tables[(tables.year == 2024)].pivot_table(values="num_pushers", index="spdx_license",aggfunc="sum").sort_values('num_pushers',ascending=False).plot(kind='barh',figsize=(9,9)) ``` 2024 data ![Image](https://github.com/user-attachments/assets/ada641a2-34eb-40dd-b0e8-4439ce5081cc) 2023 data ![Image](https://github.com/user-attachments/assets/da0d5896-121a-46a2-943c-19dcad1d256e)

GiteaMirror commented

2025-11-02 11:02:22 -06:00

@silverwind commented on GitHub (Feb 11, 2025):

If there is a reliable data source on license usage, could define a threshold of let's say >= 0.1% usage above which to include a certain license. I would commit the popularity source data into the repo with some instructions on how to update it.

@silverwind commented on GitHub (Feb 11, 2025): If there is a reliable data source on license usage, could define a threshold of let's say `>= 0.1%` usage above which to include a certain license. I would commit the popularity source data into the repo with some instructions on how to update it.

GiteaMirror commented

2025-11-02 11:02:22 -06:00

@yp05327 commented on GitHub (Mar 6, 2025):

another benefit: we can remove the codes for detecting same license contents with similar license name. Actually, these codes are “magic codes”, it is better to remove them.
I think this is a good solution, the only one point is how we maintain these popular licenses list in the future. e.g. how to define the license is a popular license, are there any third part resources we can refer, are there any rules when the license is not popular any more.

@yp05327 commented on GitHub (Mar 6, 2025): another benefit: we can remove the codes for detecting same license contents with similar license name. Actually, these codes are “magic codes”, it is better to remove them. I think this is a good solution, the only one point is how we maintain these popular licenses list in the future. e.g. how to define the license is a popular license, are there any third part resources we can refer, are there any rules when the license is not popular any more.

GiteaMirror commented

2025-11-02 11:02:22 -06:00

@wxiaoguang commented on GitHub (Mar 9, 2025):

-> Only keep popular licenses #33832

@wxiaoguang commented on GitHub (Mar 9, 2025): -> Only keep popular licenses #33832

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/gitea#14080