Adjust database connection attempts #11656

Closed
opened 2025-11-02 09:43:37 -06:00 by GiteaMirror · 7 comments
Owner

Originally created by @tacerus on GitHub (Sep 14, 2023).

Feature Description

Hi,

currently, if the database backend is unreachable for a longer period of time (for example during maintenance on the database server), Gitea will crash after ten connection attempts:

Sep 14 09:40:57 gitea01 gitea[1165]: 2023/09/14 09:40:57 routers/common/db.go:35:InitDBEngine() [I] Backing off for 3 seconds
Sep 14 09:41:00 gitea01 gitea[1165]: 2023/09/14 09:41:00 routers/common/db.go:28:InitDBEngine() [I] ORM engine initialization attempt #9/10...
Sep 14 09:41:00 gitea01 gitea[1165]: 2023/09/14 09:41:00 cmd/web.go:175:serveInstalled() [I] PING DATABASE mysql
Sep 14 09:41:03 gitea01 gitea[1165]: 2023/09/14 09:41:03 routers/common/db.go:34:InitDBEngine() [E] ORM engine initialization attempt #9/10 failed. Error: dial tcp [2a07:>
Sep 14 09:41:03 gitea01 gitea[1165]: 2023/09/14 09:41:03 routers/common/db.go:35:InitDBEngine() [I] Backing off for 3 seconds
Sep 14 09:41:06 gitea01 gitea[1165]: 2023/09/14 09:41:06 routers/common/db.go:28:InitDBEngine() [I] ORM engine initialization attempt #10/10...
Sep 14 09:41:06 gitea01 gitea[1165]: 2023/09/14 09:41:06 cmd/web.go:175:serveInstalled() [I] PING DATABASE mysql
Sep 14 09:41:06 gitea01 gitea[1165]: 2023/09/14 09:41:06 routers/init.go:69:mustInitCtx() [F] code.gitea.io/gitea/routers/common.InitDBEngine(ctx) failed: dial tcp [2a07:>
Sep 14 09:41:06 gitea01 systemd[1]: gitea.service: Main process exited, code=exited, status=1/FAILURE
Sep 14 09:41:06 gitea01 systemd[1]: gitea.service: Failed with result 'exit-code'.

It would be great if the delay between the attempts and the amount of attempts could be adjusted, to have Gitea come online again automatically after the backend is back.
A workaround is to have systemd restart the service on failure, but that is prone to cause issues if the failure is caused by something other than the database being offline.

Screenshots

No response

Originally created by @tacerus on GitHub (Sep 14, 2023). ### Feature Description Hi, currently, if the database backend is unreachable for a longer period of time (for example during maintenance on the database server), Gitea will crash after ten connection attempts: ``` Sep 14 09:40:57 gitea01 gitea[1165]: 2023/09/14 09:40:57 routers/common/db.go:35:InitDBEngine() [I] Backing off for 3 seconds Sep 14 09:41:00 gitea01 gitea[1165]: 2023/09/14 09:41:00 routers/common/db.go:28:InitDBEngine() [I] ORM engine initialization attempt #9/10... Sep 14 09:41:00 gitea01 gitea[1165]: 2023/09/14 09:41:00 cmd/web.go:175:serveInstalled() [I] PING DATABASE mysql Sep 14 09:41:03 gitea01 gitea[1165]: 2023/09/14 09:41:03 routers/common/db.go:34:InitDBEngine() [E] ORM engine initialization attempt #9/10 failed. Error: dial tcp [2a07:> Sep 14 09:41:03 gitea01 gitea[1165]: 2023/09/14 09:41:03 routers/common/db.go:35:InitDBEngine() [I] Backing off for 3 seconds Sep 14 09:41:06 gitea01 gitea[1165]: 2023/09/14 09:41:06 routers/common/db.go:28:InitDBEngine() [I] ORM engine initialization attempt #10/10... Sep 14 09:41:06 gitea01 gitea[1165]: 2023/09/14 09:41:06 cmd/web.go:175:serveInstalled() [I] PING DATABASE mysql Sep 14 09:41:06 gitea01 gitea[1165]: 2023/09/14 09:41:06 routers/init.go:69:mustInitCtx() [F] code.gitea.io/gitea/routers/common.InitDBEngine(ctx) failed: dial tcp [2a07:> Sep 14 09:41:06 gitea01 systemd[1]: gitea.service: Main process exited, code=exited, status=1/FAILURE Sep 14 09:41:06 gitea01 systemd[1]: gitea.service: Failed with result 'exit-code'. ``` It would be great if the delay between the attempts and the amount of attempts could be adjusted, to have Gitea come online again automatically after the backend is back. A workaround is to have systemd restart the service on failure, but that is prone to cause issues if the failure is caused by something other than the database being offline. ### Screenshots _No response_
GiteaMirror added the issue/needs-feedback label 2025-11-02 09:43:37 -06:00
Author
Owner

@CaiCandong commented on GitHub (Sep 14, 2023):

I think so. We may need a Back-off algorithm to determine the retry interval.
How about this ?https://github.com/jpillora/backoff

@CaiCandong commented on GitHub (Sep 14, 2023): I think so. We may need a Back-off algorithm to determine the retry interval. How about this ?https://github.com/jpillora/backoff
Author
Owner

@tacerus commented on GitHub (Sep 14, 2023):

Hi @CaiCandong, the project you linked seems to be about HTTP, not TCP or MySQL?

@tacerus commented on GitHub (Sep 14, 2023): Hi @CaiCandong, the project you linked seems to be about HTTP, not TCP or MySQL?
Author
Owner

@CaiCandong commented on GitHub (Sep 14, 2023):

Hi @CaiCandong, the project you linked seems to be about HTTP, not TCP or MySQL?

Sorry, I just searched back off the keyword; I didn't look hard enough.

@CaiCandong commented on GitHub (Sep 14, 2023): > Hi @CaiCandong, the project you linked seems to be about HTTP, not TCP or MySQL? Sorry, I just searched `back off` the keyword; I didn't look hard enough.
Author
Owner

@wxiaoguang commented on GitHub (Sep 14, 2023):

Backoff is quite simple

https://github.com/go-gitea/gitea/blob/main/modules/queue/backoff.go

@wxiaoguang commented on GitHub (Sep 14, 2023): Backoff is quite simple https://github.com/go-gitea/gitea/blob/main/modules/queue/backoff.go
Author
Owner

@wxiaoguang commented on GitHub (Sep 17, 2023):

And it is already adjustable.

47b878858a/custom/conf/app.example.ini (L398-L399)

@wxiaoguang commented on GitHub (Sep 17, 2023): And it is already adjustable. https://github.com/go-gitea/gitea/blob/47b878858ada27fc4c74eeadcc1e467d2da90e04/custom/conf/app.example.ini#L398-L399
Author
Owner

@wxiaoguang commented on GitHub (Sep 21, 2023):

Close as completed because it is adjustable.

@wxiaoguang commented on GitHub (Sep 21, 2023): Close as completed because it is adjustable.
Author
Owner

@tacerus commented on GitHub (Sep 24, 2023):

Thank you, I missed this!

@tacerus commented on GitHub (Sep 24, 2023): Thank you, I missed this!
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/gitea#11656