Issue search for part of a word, case sensitivity #6367

Closed
opened 2025-11-02 06:53:41 -06:00 by GiteaMirror · 17 comments
Owner

Originally created by @johannes-nivus on GitHub (Nov 21, 2020).

  • Gitea version (or commit ref): 1.12.5 built with GNU Make 4.1, go1.14.9
  • Git version: 2.25.1
  • Operating system: Ubuntu 20.04.1 LTS, no docker
  • Database (use [x]):
    • PostgreSQL
    • MySQL
    • MSSQL
    • SQLite
  • Can you reproduce the bug at https://try.gitea.io:
    • Yes (provide example URL)
    • No
  • Log gist:

Description

Searching for parts of a word in Issues is not possible. I tried both bleeve and db as indexer. Even worse: Using db, the search becomes case sensitive.
Since elastic search seems to have this problem as well (see #11977), I have not tried this yet.

For us it's really a problem, since German language uses many compound nouns. And you would always have to search for the exact composition now.

Since it seems to work in https://try.gitea.io, which search indexer and which database is used there?

Screenshots

All screenshots done with db as indexer, but search for part of word is not possible with bleeve as well.
Temperatur
Temp
Temperatur_small

Originally created by @johannes-nivus on GitHub (Nov 21, 2020). <!-- NOTE: If your issue is a security concern, please send an email to security@gitea.io instead of opening a public issue --> <!-- 1. Please speak English, this is the language all maintainers can speak and write. 2. Please ask questions or configuration/deploy problems on our Discord server (https://discord.gg/gitea) or forum (https://discourse.gitea.io). 3. Please take a moment to check that your issue doesn't already exist. 4. Please give all relevant information below for bug reports, because incomplete details will be handled as an invalid report. --> - Gitea version (or commit ref): 1.12.5 built with GNU Make 4.1, go1.14.9 - Git version: 2.25.1 - Operating system: Ubuntu 20.04.1 LTS, no docker <!-- Please include information on whether you built gitea yourself, used one of our downloads or are using some other package --> <!-- Please also tell us how you are running gitea, e.g. if it is being run from docker, a command-line, systemd etc. ---> <!-- If you are using a package or systemd tell us what distribution you are using --> - Database (use `[x]`): - [x] PostgreSQL - [ ] MySQL - [ ] MSSQL - [ ] SQLite - Can you reproduce the bug at https://try.gitea.io: - [ ] Yes (provide example URL) - [x] No - Log gist: <!-- It really is important to provide pertinent logs --> <!-- Please read https://docs.gitea.io/en-us/logging-configuration/#debugging-problems --> <!-- In addition, if your problem relates to git commands set `RUN_MODE=dev` at the top of app.ini --> ## Description Searching for parts of a word in Issues is not possible. I tried both bleeve and db as indexer. Even worse: Using db, the search becomes case sensitive. Since elastic search seems to have this problem as well (see #11977), I have not tried this yet. For us it's really a problem, since German language uses many compound nouns. And you would always have to search for the exact composition now. Since it seems to work in https://try.gitea.io, which search indexer and which database is used there? ## Screenshots All screenshots done with db as indexer, but search for part of word is not possible with bleeve as well. ![Temperatur](https://user-images.githubusercontent.com/19533883/99872313-43333f00-2be1-11eb-8174-5f98ef66d34c.PNG) ![Temp](https://user-images.githubusercontent.com/19533883/99872306-39114080-2be1-11eb-8bbf-6ecb420233d4.PNG) ![Temperatur_small](https://user-images.githubusercontent.com/19533883/99872316-462e2f80-2be1-11eb-9f57-def248b8c35e.PNG) <!-- **If this issue involves the Web Interface, please include a screenshot** -->
GiteaMirror added the type/bug label 2025-11-02 06:53:41 -06:00
Author
Owner

@lunny commented on GitHub (Nov 21, 2020):

try.gitea.io use sqlite as database.

@lunny commented on GitHub (Nov 21, 2020): try.gitea.io use sqlite as database.
Author
Owner

@johannes-nivus commented on GitHub (Nov 21, 2020):

try.gitea.io use sqlite as database.

Is there a possibility to migrate an existing database from postgres to sqlite? I had only chosen postgres because of a parallel Open Project installation.

@johannes-nivus commented on GitHub (Nov 21, 2020): > > > try.gitea.io use sqlite as database. Is there a possibility to migrate an existing database from postgres to sqlite? I had only chosen postgres because of a parallel Open Project installation.
Author
Owner

@johannes-nivus commented on GitHub (Nov 21, 2020):

Attention: I have neither experience with databases nor with go.
But nevertheless a question:
What would need to be done to use pg_trgm and the % operator for issue searching? Am I right that currently LIKE is used (which would explain the sudden case sensitivity with postgresql)?
If this would be possible at all, it would be nice to have this configurable in app.ini.

@johannes-nivus commented on GitHub (Nov 21, 2020): Attention: I have neither experience with databases nor with go. But nevertheless a question: What would need to be done to use pg_trgm and the % operator for issue searching? Am I right that currently LIKE is used (which would explain the sudden case sensitivity with postgresql)? If this would be possible at all, it would be nice to have this configurable in app.ini.
Author
Owner

@johannes-nivus commented on GitHub (Nov 22, 2020):

Is there a possibility to migrate an existing database from postgres to sqlite? I had only chosen postgres because of a parallel Open Project installation.

I managed to migrate from postgres to sqlite, by using
./gitea dump -c /path/to/app.ini -d sqlite3
Then I extracted only the database dump and used
sqlite3 gitea.db <gitea-db.sql
After editing app.ini and restarting gitea everything is working for now.

@johannes-nivus commented on GitHub (Nov 22, 2020): > Is there a possibility to migrate an existing database from postgres to sqlite? I had only chosen postgres because of a parallel Open Project installation. I managed to migrate from postgres to sqlite, by using ```./gitea dump -c /path/to/app.ini -d sqlite3``` Then I extracted only the database dump and used ```sqlite3 gitea.db <gitea-db.sql``` After editing app.ini and restarting gitea everything is working for now.
Author
Owner

@ngugcx commented on GitHub (Dec 18, 2020):

Why is part-word search not supported when using Postgres?

@ngugcx commented on GitHub (Dec 18, 2020): Why is part-word search not supported when using Postgres?
Author
Owner

@ngugcx commented on GitHub (Dec 18, 2020):

Is there a possibility to migrate an existing database from postgres to sqlite? I had only chosen postgres because of a parallel Open Project installation.

I managed to migrate from postgres to sqlite, by using
./gitea dump -c /path/to/app.ini -d sqlite3
Then I extracted only the database dump and used
sqlite3 gitea.db <gitea-db.sql
After editing app.ini and restarting gitea everything is working for now.

Did you meet this problem?

bash-5.0$ sqlite3 gitea.db <gitea-db.sql
Error: near line 16: no such function: setval
Error: near line 23: no such function: setval
Error: near line 30: no such function: setval
Error: near line 72: no such function: setval
Error: near line 78: no such function: setval
Error: near line 84: no such function: setval
Error: near line 116: no such function: setval
Error: near line 120: no such function: setval
Error: near line 153: no such function: setval
Error: near line 157: no such function: setval
Error: near line 161: no such function: setval
Error: near line 1380: no such function: setval
@ngugcx commented on GitHub (Dec 18, 2020): > > Is there a possibility to migrate an existing database from postgres to sqlite? I had only chosen postgres because of a parallel Open Project installation. > > I managed to migrate from postgres to sqlite, by using > `./gitea dump -c /path/to/app.ini -d sqlite3` > Then I extracted only the database dump and used > `sqlite3 gitea.db <gitea-db.sql` > After editing app.ini and restarting gitea everything is working for now. Did you meet this problem? ``` bash-5.0$ sqlite3 gitea.db <gitea-db.sql Error: near line 16: no such function: setval Error: near line 23: no such function: setval Error: near line 30: no such function: setval Error: near line 72: no such function: setval Error: near line 78: no such function: setval Error: near line 84: no such function: setval Error: near line 116: no such function: setval Error: near line 120: no such function: setval Error: near line 153: no such function: setval Error: near line 157: no such function: setval Error: near line 161: no such function: setval Error: near line 1380: no such function: setval ```
Author
Owner

@lunny commented on GitHub (Dec 18, 2020):

Why is part-word search not supported when using Postgres?

part word should have been supported by Postgres but not Case sensitivity. Maybe we need a PR to improve issue db indexer to support that.

@lunny commented on GitHub (Dec 18, 2020): > Why is part-word search not supported when using Postgres? part word should have been supported by Postgres but not Case sensitivity. Maybe we need a PR to improve issue db indexer to support that.
Author
Owner

@johannes-nivus commented on GitHub (Dec 18, 2020):

part word should have been supported by Postgres but not Case sensitivity. Maybe we need a PR to improve issue db indexer to support that.

Part of word search had not been possible with Postgres when I opened the issue (see screenshots).
Postgres: No part of word, case sensitive.
Bleve: No part of word.

@johannes-nivus commented on GitHub (Dec 18, 2020): > part word should have been supported by Postgres but not Case sensitivity. Maybe we need a PR to improve issue db indexer to support that. Part of word search had not been possible with Postgres when I opened the issue (see screenshots). Postgres: No part of word, case sensitive. Bleve: No part of word.
Author
Owner

@johannes-nivus commented on GitHub (Dec 18, 2020):

Did you meet this problem?

bash-5.0$ sqlite3 gitea.db <gitea-db.sql
Error: near line 16: no such function: setval
Error: near line 23: no such function: setval
Error: near line 30: no such function: setval
Error: near line 72: no such function: setval
Error: near line 78: no such function: setval
Error: near line 84: no such function: setval
Error: near line 116: no such function: setval
Error: near line 120: no such function: setval
Error: near line 153: no such function: setval
Error: near line 157: no such function: setval
Error: near line 161: no such function: setval
Error: near line 1380: no such function: setval

Yes, but I ignored it. And it worked.

@johannes-nivus commented on GitHub (Dec 18, 2020): > Did you meet this problem? > > ``` > bash-5.0$ sqlite3 gitea.db <gitea-db.sql > Error: near line 16: no such function: setval > Error: near line 23: no such function: setval > Error: near line 30: no such function: setval > Error: near line 72: no such function: setval > Error: near line 78: no such function: setval > Error: near line 84: no such function: setval > Error: near line 116: no such function: setval > Error: near line 120: no such function: setval > Error: near line 153: no such function: setval > Error: near line 157: no such function: setval > Error: near line 161: no such function: setval > Error: near line 1380: no such function: setval > ``` Yes, but I ignored it. And it worked.
Author
Owner

@ngugcx commented on GitHub (Mar 1, 2021):

Any update here?

@ngugcx commented on GitHub (Mar 1, 2021): Any update here?
Author
Owner

@zeripath commented on GitHub (Mar 1, 2021):

For postgres we would just have to emit ILIKE instead of LIKE

@zeripath commented on GitHub (Mar 1, 2021): For postgres we would just have to emit ILIKE instead of LIKE
Author
Owner

@zeripath commented on GitHub (Mar 1, 2021):

However a cross dB solution would be to change

models.SearchIssueIDsByKeyword()

To use lower(...) on the fields and lower case the keyword too.

@zeripath commented on GitHub (Mar 1, 2021): However a cross dB solution would be to change models.SearchIssueIDsByKeyword() To use lower(...) on the fields and lower case the keyword too.
Author
Owner

@zeripath commented on GitHub (Mar 1, 2021):

This would be a three line pr

@zeripath commented on GitHub (Mar 1, 2021): This would be a three line pr
Author
Owner
@ngugcx commented on GitHub (Mar 2, 2021): @zeripath , I prefer upper() over lower(). https://docs.microsoft.com/en-us/dotnet/fundamentals/code-analysis/quality-rules/ca1308 https://en.wikipedia.org/wiki/%C3%9F#Capital_form
Author
Owner

@ngugcx commented on GitHub (Apr 19, 2021):

After upgrading to v1.14.1, search for part of a word is still not working.
I'm using postgres.

@ngugcx commented on GitHub (Apr 19, 2021): After upgrading to v1.14.1, search for part of a word is still not working. I'm using postgres.
Author
Owner

@6543 commented on GitHub (Apr 19, 2021):

@ngugcx can you file a new issue & reverence this closed one :)

@6543 commented on GitHub (Apr 19, 2021): @ngugcx can you file a new issue & reverence this closed one :)
Author
Owner

@ngugcx commented on GitHub (Apr 22, 2021):

@6543 , filed a new one.

@ngugcx commented on GitHub (Apr 22, 2021): @6543 , filed a new one.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/gitea#6367