ISSUE_INDEXER_QUEUE_CONN_STR used as ISSUE_INDEXER_QUEUE_DIR #6647

New Issue

GiteaMirror · 2025-11-02T07:02:22-06:00

GiteaMirror commented

2025-11-02 07:02:22 -06:00

Originally created by @nuno-silva on GitHub (Jan 6, 2021).

Gitea version (or commit ref): 1.13.1 (and also master as of 8688c2be95)
Git version: NA
Operating system: docker
Database (use [x]): NA
Can you reproduce the bug at https://try.gitea.io: NA
Log gist: NA

Description

It seems ISSUE_INDEXER_QUEUE_CONN_STR is being used as ISSUE_INDEXER_QUEUE_DIR.
Steps to reproduce (from source):

set this in app.ini (and some sane values for the other ini sections):

[indexer]
ISSUE_INDEXER_TYPE = bleve
ISSUE_INDEXER_PATH = /data/gitea/indexers/issues.bleve

start gitea web (this one was compiled from source as of 8688c2be95):

/data $ GITEA_WORK_DIR=/data/gitea ~/Downloads/gitea/gitea web --config /data/gitea/conf/app.ini

Notice issues.queue is created, as per the defaults:

/data $ tree gitea/indexers/
gitea/indexers/
├── issues.bleve
│   ├── index_meta.json
│   ├── rupture_meta.json
│   └── store
└── issues.queue
    ├── 000002.ldb
    ├── 000003.log
    ├── CURRENT
    ├── CURRENT.bak
    ├── LOCK
    ├── LOG
    └── MANIFEST-000004

stop gitea
rm -r /data/gitea/indexers/
change the indexer config section to

[indexer]
ISSUE_INDEXER_TYPE = bleve
ISSUE_INDEXER_PATH = /data/gitea/indexers/issues.bleve
ISSUE_INDEXER_QUEUE_CONN_STR = "addrs=127.0.0.1:6379 db=0"

start gitea web again
notice that issues.queue does not exist, but a 'addrs=127.0.0.1:6379 db=0' dir was created:

/data $ tree gitea/indexers/
gitea/indexers/
└── issues.bleve
    ├── index_meta.json
    ├── rupture_meta.json
    └── store

1 directory, 3 files
/data $ ls
'addrs=127.0.0.1:6379 db=0'   git   gitea
/data $ ls addrs\=127.0.0.1\:6379\ db\=0/
000001.log  CURRENT  LOCK  LOG  MANIFEST-000000

Notice I did not set ISSUE_INDEXER_QUEUE_TYPE nor ISSUE_INDEXER_QUEUE_DIR anywhere.

The config-cheat-sheet does mention that ISSUE_INDEXER_QUEUE_TYPE and ISSUE_INDEXER_QUEUE_DIR are deprecated, but they are present in the example config file:
3c96a37162/custom/conf/app.example.ini (L453-L460)

I was able to reproduce this using:

gitea 1.13.1, docker, sqlite
gitea 1.13.1, Gentoo, MySQL (production install)
master as of 8688c2be95), Ubuntu 18, sqlite

Screenshots

NA

Originally created by @nuno-silva on GitHub (Jan 6, 2021). - Gitea version (or commit ref): 1.13.1 (and also master as of 8688c2be95f1d11fadd6cbb979eafe0bb817589a) - Git version: NA - Operating system: docker - Database (use `[x]`): NA - Can you reproduce the bug at https://try.gitea.io: NA - Log gist: NA ## Description It seems `ISSUE_INDEXER_QUEUE_CONN_STR` is being used as `ISSUE_INDEXER_QUEUE_DIR`. Steps to reproduce (from source): * set this in app.ini (and some sane values for the other ini sections): ``` [indexer] ISSUE_INDEXER_TYPE = bleve ISSUE_INDEXER_PATH = /data/gitea/indexers/issues.bleve ``` * start gitea web (this one was compiled from source as of 8688c2be95f1d11fadd6cbb979eafe0bb817589a): ``` /data $ GITEA_WORK_DIR=/data/gitea ~/Downloads/gitea/gitea web --config /data/gitea/conf/app.ini ``` * Notice `issues.queue` is created, as per the defaults: ``` /data $ tree gitea/indexers/ gitea/indexers/ ├── issues.bleve │ ├── index_meta.json │ ├── rupture_meta.json │ └── store └── issues.queue ├── 000002.ldb ├── 000003.log ├── CURRENT ├── CURRENT.bak ├── LOCK ├── LOG └── MANIFEST-000004 ``` * stop gitea * `rm -r /data/gitea/indexers/` * change the indexer config section to ``` [indexer] ISSUE_INDEXER_TYPE = bleve ISSUE_INDEXER_PATH = /data/gitea/indexers/issues.bleve ISSUE_INDEXER_QUEUE_CONN_STR = "addrs=127.0.0.1:6379 db=0" ``` * start gitea web again * notice that `issues.queue` does not exist, but a 'addrs=127.0.0.1:6379 db=0' dir was created: ``` /data $ tree gitea/indexers/ gitea/indexers/ └── issues.bleve ├── index_meta.json ├── rupture_meta.json └── store 1 directory, 3 files /data $ ls 'addrs=127.0.0.1:6379 db=0' git gitea /data $ ls addrs\=127.0.0.1\:6379\ db\=0/ 000001.log CURRENT LOCK LOG MANIFEST-000000 ``` Notice I did not set `ISSUE_INDEXER_QUEUE_TYPE` nor `ISSUE_INDEXER_QUEUE_DIR` anywhere. --- The [config-cheat-sheet](https://docs.gitea.io/en-us/config-cheat-sheet/#indexer-indexer) does mention that `ISSUE_INDEXER_QUEUE_TYPE` and `ISSUE_INDEXER_QUEUE_DIR` are deprecated, but they are present in the example config file: https://github.com/go-gitea/gitea/blob/3c96a3716288b0ae1ae65df44cc92218b489b77f/custom/conf/app.example.ini#L453-L460 --- I was able to reproduce this using: - gitea 1.13.1, docker, sqlite - gitea 1.13.1, Gentoo, MySQL (production install) - master as of 8688c2be95f1d11fadd6cbb979eafe0bb817589a), Ubuntu 18, sqlite ## Screenshots NA

GiteaMirror added the type/docs label 2025-11-02 07:02:23 -06:00

GiteaMirror closed this issue

2025-11-02 07:02:23 -06:00

GiteaMirror commented

2025-11-02 07:02:24 -06:00

@zeripath commented on GitHub (Jan 6, 2021):

this is intentional.

@zeripath commented on GitHub (Jan 6, 2021): this is intentional.

GiteaMirror commented

2025-11-02 07:02:24 -06:00

@nuno-silva commented on GitHub (Jan 6, 2021):

What's the reasoning behind this being intentional? Shouldn't ISSUE_INDEXER_QUEUE_CONN_STR only do anything at all when ISSUE_INDEXER_QUEUE_TYPE is redis?

@nuno-silva commented on GitHub (Jan 6, 2021): What's the reasoning behind this being intentional? Shouldn't `ISSUE_INDEXER_QUEUE_CONN_STR` only do anything at all when `ISSUE_INDEXER_QUEUE_TYPE` is `redis`?

GiteaMirror commented

2025-11-02 07:02:24 -06:00

@nuno-silva commented on GitHub (Jan 7, 2021):

CONN_STR in the queue section is also affected (this is from a prod instance running 1.13.1):

[queue]
; Specific queues can be individually configured with [queue.name]. [queue] provides defaults
;
; General queue queue type, currently support: persistable-channel, channel, level, redis, dummy
; default to persistable-channel
TYPE = persistable-channel
; data-dir for storing persistable queues and level queues, individual queues will be named by their type
DATADIR = /var/lib/gitea/queues/
; Default queue length before a channel queue will block
LENGTH = 20 
; Batch size to send for batched queues
BATCH_LENGTH = 20 
; Connection string for redis queues this will store the redis connection string.
CONN_STR = "addrs=127.0.0.1:6379 db=0"

# lsof | grep 6379 | head -2
gitea     16162                      git   22r      REG              254,0      162     918641 /var/lib/gitea/addrs=127.0.0.1:6379 db=0/000010.ldb
gitea     16162                      git   23uW     REG              254,0        0     914026 /var/lib/gitea/addrs=127.0.0.1:6379 db=0/LOCK

...and if I comment out CONN_STR and restart gitea:

# lsof | grep 6379; echo $?
1

Note that it's not hard to end up with CONN_STR defined even if not using redis, since that's how it's done in the example config:
3c96a37162/custom/conf/app.example.ini (L484-L497)

So, if this is intentional, it should at least be documented :/

@nuno-silva commented on GitHub (Jan 7, 2021): `CONN_STR` in the `queue` section is also affected (this is from a prod instance running 1.13.1): ``` [queue] ; Specific queues can be individually configured with [queue.name]. [queue] provides defaults ; ; General queue queue type, currently support: persistable-channel, channel, level, redis, dummy ; default to persistable-channel TYPE = persistable-channel ; data-dir for storing persistable queues and level queues, individual queues will be named by their type DATADIR = /var/lib/gitea/queues/ ; Default queue length before a channel queue will block LENGTH = 20 ; Batch size to send for batched queues BATCH_LENGTH = 20 ; Connection string for redis queues this will store the redis connection string. CONN_STR = "addrs=127.0.0.1:6379 db=0" ``` ``` # lsof | grep 6379 | head -2 gitea 16162 git 22r REG 254,0 162 918641 /var/lib/gitea/addrs=127.0.0.1:6379 db=0/000010.ldb gitea 16162 git 23uW REG 254,0 0 914026 /var/lib/gitea/addrs=127.0.0.1:6379 db=0/LOCK ``` ...and if I comment out `CONN_STR` and restart gitea: ``` # lsof | grep 6379; echo $? 1 ``` Note that it's not hard to end up with `CONN_STR` defined **even if not using redis**, since **that's how it's done in the example config**: https://github.com/go-gitea/gitea/blob/3c96a3716288b0ae1ae65df44cc92218b489b77f/custom/conf/app.example.ini#L484-L497 So, if this is intentional, it should at least be documented :/

GiteaMirror commented

2025-11-02 07:02:24 -06:00

@zeripath commented on GitHub (Jan 7, 2021):

because it can be used to provide a lot more options to the leveldb queue.

The app.example.ini is NOT supposed to be copied. If you don't know what a setting is you should not set it.

@zeripath commented on GitHub (Jan 7, 2021): because it can be used to provide a lot more options to the leveldb queue. The app.example.ini is **NOT** supposed to be copied. If you don't know what a setting is you should not set it.

GiteaMirror commented

2025-11-02 07:02:24 -06:00

@nuno-silva commented on GitHub (Jan 7, 2021):

sorry to insist, but...

The app.example.ini is NOT supposed to be copied. If you don't know what a setting is you should not set it.

Fair enough. I can simply not define CONN_STR nor ISSUE_INDEXER_QUEUE_CONN_STR. However, it is not mentioned anywhere in the config-cheat-sheet nor in the example config that the directory settings are overridden by these unrelated "connection" settings. This "intentional behaviour" is totally unexpected and should at least be documented, even if done for legacy/compatibility reasons.

because it can be used to provide a lot more options to the leveldb queue.

sure, but why should it override the directory when leveldb is not used?

@nuno-silva commented on GitHub (Jan 7, 2021): sorry to insist, but... > The app.example.ini is NOT supposed to be copied. If you don't know what a setting is you should not set it. Fair enough. I can simply not define `CONN_STR` nor `ISSUE_INDEXER_QUEUE_CONN_STR`. However, it is not mentioned anywhere in the [config-cheat-sheet](https://docs.gitea.io/en-us/config-cheat-sheet/) nor in the example config that the directory settings are overridden by these _unrelated_ "connection" settings. This "intentional behaviour" is totally unexpected and should at least be documented, even if done for legacy/compatibility reasons. --- > because it can be used to provide a lot more options to the leveldb queue. sure, but why should it override the directory when leveldb is not used?

GiteaMirror commented

2025-11-02 07:02:24 -06:00

@zeripath commented on GitHub (Jan 9, 2021):

are overridden by these unrelated "connection" settings.

They are not unrelated settings at all. Just because you don't know how they relate does not mean that they aren't and aren't that way for a good reason.

sure, but why should it override the directory when leveldb is not used?

How do you think the persistable-channel is persisted? We use a leveldb internally - I'm fairly certain that the docs state that.

Anyway let's stop being so aggressive.

The CONN_STR for an underlying level db is a directory or, something of the form:

leveldb://path/to/db?option=value&.... or leveldb:///absolute/path/to/db?option=value&....

with options as per:

d989247bb0/modules/nosql/manager_leveldb.go (L59-L132)

I'm sorry it's incompletely documented but it's the only way to allow any of these options to be set.

This "intentional behaviour" is totally unexpected and should at least be documented, even if done for legacy/compatibility reasons.

I don't completely understand how can we be expected that you would have incorrect values in your app.ini. However, feel free to provide a PR improving the documentation.

If you don't understand what is in your app.ini please don't set it. Please go through the rest of your app.ini - I suspect you have a section [log.x] that does nothing amongst lots of other unhelpful logging configuration - https://docs.gitea.io/en-us/logging-configuration may be helpful.

@zeripath commented on GitHub (Jan 9, 2021): > are overridden by these unrelated "connection" settings. They are not unrelated settings at all. Just because you don't know how they relate does not mean that they aren't and aren't that way for a good reason. > sure, but why should it override the directory when leveldb is not used? How do you think the persistable-channel is persisted? We use a leveldb internally - I'm fairly certain that the docs state that. --- Anyway let's stop being so aggressive. The CONN_STR for an underlying level db is a directory or, something of the form: `leveldb://path/to/db?option=value&....` or `leveldb:///absolute/path/to/db?option=value&....` with options as per: https://github.com/go-gitea/gitea/blob/d989247bb08d2b8eb144e7a0edeaedfc26d08175/modules/nosql/manager_leveldb.go#L59-L132 I'm sorry it's incompletely documented but it's the only way to allow any of these options to be set. --- > This "intentional behaviour" is totally unexpected and should at least be documented, even if done for legacy/compatibility reasons. I don't completely understand how can we be expected that you would have incorrect values in your app.ini. However, feel free to provide a PR improving the documentation. If you don't understand what is in your app.ini please don't set it. Please go through the rest of your app.ini - I suspect you have a section `[log.x]` that does nothing amongst lots of other unhelpful logging configuration - https://docs.gitea.io/en-us/logging-configuration may be helpful.

GiteaMirror commented

2025-11-02 07:02:25 -06:00

@nuno-silva commented on GitHub (Jan 10, 2021):

Thanks for clearing that up. It makes a bit more sense now that you explained it.
I'm sorry if I was aggressive.

How do you think the persistable-channel is persisted? We use a leveldb internally - I'm fairly certain that the docs state that.

Sorry, I can't find it anywhere :/

Anyway, I think it's clear now that this is a documentation issue.
I'll try to open a PR (probably next week) to improve the docs and the example config to make this a bit more clear.

I suspect you have a section [log.x] that does nothing

yep, you're right :/ I'll review my config. Thank you.

@nuno-silva commented on GitHub (Jan 10, 2021): Thanks for clearing that up. It makes a bit more sense now that you explained it. I'm sorry if I was aggressive. --- > How do you think the persistable-channel is persisted? We use a leveldb internally - I'm fairly certain that the docs state that. Sorry, I can't find it anywhere :/ Anyway, I think it's clear now that this is a documentation issue. I'll try to open a PR (probably next week) to improve the docs and the example config to make this a bit more clear. --- > I suspect you have a section `[log.x]` that does nothing yep, you're right :/ I'll review my config. Thank you.

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/gitea#6647