random database lock when sqlite database #837

GiteaMirror commented

@lunny commented on GitHub (Jun 23, 2017):

And maybe we should pause make test-sqlite util this issue is resolved.

@lunny commented on GitHub (Jun 23, 2017): And maybe we should pause `make test-sqlite` util this issue is resolved.

GiteaMirror commented

@typeless commented on GitHub (Jun 27, 2017):

Reference: https://sqlite.org/unlock_notify.html

@typeless commented on GitHub (Jun 27, 2017): Reference: https://sqlite.org/unlock_notify.html

GiteaMirror commented

@bkcsoft commented on GitHub (Jun 28, 2017):

@typeless Unless that is available in https://github.com/mattn/go-sqlite3 and https://github.com/go-xorm/xorm/ we can't use it 😞

@bkcsoft commented on GitHub (Jun 28, 2017): @typeless Unless that is available in https://github.com/mattn/go-sqlite3 _and_ https://github.com/go-xorm/xorm/ we can't use it 😞

GiteaMirror commented

@typeless commented on GitHub (Jun 29, 2017):

@bkcsoft Agreed. This would probably need to be fixed upstream.

@typeless commented on GitHub (Jun 29, 2017): @bkcsoft Agreed. This would probably need to be fixed upstream.

GiteaMirror commented

@andreynering commented on GitHub (Jul 5, 2017):

I hope #2116 fixes the issue

@andreynering commented on GitHub (Jul 5, 2017): I hope #2116 fixes the issue

GiteaMirror commented

@andreynering commented on GitHub (Jul 6, 2017):

Reopening, since it seems it not really fixed.

@andreynering commented on GitHub (Jul 6, 2017): Reopening, since it seems it not really fixed.

GiteaMirror commented

@PaulBINK commented on GitHub (Jun 18, 2018):

We are getting this problem frequently when working with issues: add label, comment, close issue, reopen. Sooner or later it will 500 on us. According to the logs it happens when calling GetNotificationCount.

@PaulBINK commented on GitHub (Jun 18, 2018): We are getting this problem frequently when working with issues: add label, comment, close issue, reopen. Sooner or later it will 500 on us. According to the logs it happens when calling GetNotificationCount.

GiteaMirror commented

@jonasfranz commented on GitHub (Jun 18, 2018):

Enabling WAL mode worked for me. https://www.sqlite.org/wal.html

@jonasfranz commented on GitHub (Jun 18, 2018): Enabling WAL mode worked for me. https://www.sqlite.org/wal.html

GiteaMirror commented

@hoxnox commented on GitHub (Aug 30, 2018):

Same for me.

Caused by: hudson.plugins.git.GitException: Command "/usr/local/git/bin/git fetch --tags --progress ssh://sources.acme.com/repo.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout: 
stderr: Gitea: Internal error
Failed to get repository: database is locked

@hoxnox commented on GitHub (Aug 30, 2018): Same for me. ``` Caused by: hudson.plugins.git.GitException: Command "/usr/local/git/bin/git fetch --tags --progress ssh://sources.acme.com/repo.git +refs/heads/*:refs/remotes/origin/*" returned status code 128: stdout: stderr: Gitea: Internal error Failed to get repository: database is locked ```

GiteaMirror commented

https://github.com/bvinc/go-sqlite-lite
https://github.com/crawshaw/sqlite

@typeless commented on GitHub (Aug 31, 2018):

Potential alternatives that could fix this:

@typeless commented on GitHub (Aug 31, 2018): Potential alternatives that could fix this: https://github.com/bvinc/go-sqlite-lite https://github.com/crawshaw/sqlite

GiteaMirror commented

https://github.com/mattn/go-sqlite3#faq

@bvinc commented on GitHub (Sep 1, 2018):

Author of go-sqlite-lite here. My driver, and also crawshaw's driver, don't provide a database/sql interface, so they would be extremely inconvenient to use. But for a few reasons, it would probably help with locking errors. My driver gives complete control over connections, which means you'll only get locking errors if you're doing something directly to cause them. The crawshaw driver defaults to using shared cache mode and WAL mode, and uses the unlock_notify API to prevent locking issues with shared cache mode.

I think the current mattn driver is the best one to use if you require a database/sql driver. Their FAQ mentions setting the the maximum number of open connections to 1. Have you tried that?

@bvinc commented on GitHub (Sep 1, 2018): Author of `go-sqlite-lite` here. My driver, and also crawshaw's driver, don't provide a `database/sql` interface, so they would be extremely inconvenient to use. But for a few reasons, it would probably help with locking errors. My driver gives complete control over connections, which means you'll only get locking errors if you're doing something directly to cause them. The crawshaw driver defaults to using shared cache mode and WAL mode, and uses the unlock_notify API to prevent locking issues with shared cache mode. I think the current mattn driver is the best one to use if you require a `database/sql` driver. Their FAQ mentions setting the the maximum number of open connections to 1. Have you tried that? https://github.com/mattn/go-sqlite3#faq

GiteaMirror commented

@lunny commented on GitHub (Sep 2, 2018):

@bvinc thanks for your reply, It's a good idea to send a PR to set open connections to 1 when using mattn/go-sqlite3

@lunny commented on GitHub (Sep 2, 2018): @bvinc thanks for your reply, It's a good idea to send a PR to set open connections to 1 when using mattn/go-sqlite3

GiteaMirror commented

@hoxnox commented on GitHub (Sep 4, 2018):

What is the best way to migrate from sqlite to postgresql?

@hoxnox commented on GitHub (Sep 4, 2018): What is the best way to migrate from sqlite to postgresql?

GiteaMirror commented

@typeless commented on GitHub (Oct 21, 2018):

Now that https://github.com/mattn/go-sqlite3/pull/439 is merged, we can either use the tip right away or wait for the new release of go-sqlite3 to fix the issue. Note that to enable this functionality, we have to add a new build tag sqlite_unlock_notify.

Edit: I hope the users don't have to specify -tags sqlite sqlite_unlock_notify besides just -tags sqlite in that it would be better to not expose the nuances of a dependent package. I don't know how it should be done though.

Edit2: I changed my mind. If one wants to build Gitea from source, he should be capable of handling the additional flag. We should update the documentation accordingly though.

@typeless commented on GitHub (Oct 21, 2018): Now that https://github.com/mattn/go-sqlite3/pull/439 is merged, we can either use the tip right away or wait for the new release of `go-sqlite3` to fix the issue. Note that to enable this functionality, we have to add a new build tag `sqlite_unlock_notify.` Edit: I hope the users don't have to specify `-tags sqlite sqlite_unlock_notify` besides just `-tags sqlite` in that it would be better to not expose the nuances of a dependent package. I don't know how it should be done though. Edit2: I changed my mind. If one wants to build Gitea from source, he should be capable of handling the additional flag. We should update the documentation accordingly though.

GiteaMirror commented

@lunny commented on GitHub (Oct 21, 2018):

at first, we need a PR to update go-sqlite3 library.

@lunny commented on GitHub (Oct 21, 2018): at first, we need a PR to update go-sqlite3 library.

GiteaMirror commented

@lunny commented on GitHub (Oct 23, 2018):

Even #5144 resolved single process issue. There is some issues when pushing via SSH. This should be closed by #4886.

@lunny commented on GitHub (Oct 23, 2018): Even #5144 resolved single process issue. There is some issues when pushing via SSH. This should be closed by #4886.

GiteaMirror commented

@sapk commented on GitHub (Oct 23, 2018):

@typeless go-sqlite3 library with sqlite_unlock_notify tag seems to not build on win 32bit https://drone.gitea.io/go-gitea/gitea/3536/18

@sapk commented on GitHub (Oct 23, 2018): @typeless go-sqlite3 library with sqlite_unlock_notify tag seems to not build on win 32bit https://drone.gitea.io/go-gitea/gitea/3536/18

GiteaMirror commented

@typeless commented on GitHub (Oct 23, 2018):

@sapk Similar to https://github.com/mattn/go-sqlite3/issues/238. Will submit a fix soon.

@typeless commented on GitHub (Oct 23, 2018): @sapk Similar to https://github.com/mattn/go-sqlite3/issues/238. Will submit a fix soon.

GiteaMirror commented

@sapk commented on GitHub (Oct 23, 2018):

@typeless exactly what I was searching but I use the terms uint too large and I found nothing ...

@sapk commented on GitHub (Oct 23, 2018): @typeless exactly what I was searching but I use the terms `uint too large` and I found nothing ...

GiteaMirror commented

@typeless commented on GitHub (Oct 23, 2018):

I have submitted https://github.com/mattn/go-sqlite3/pull/654.
However, I cannot find a computer with Windows/386 to test. 😨

@typeless commented on GitHub (Oct 23, 2018): I have submitted https://github.com/mattn/go-sqlite3/pull/654. However, I cannot find a computer with Windows/386 to test. 😨

GiteaMirror commented

@sapk commented on GitHub (Oct 23, 2018):

@typeless use gox to cross-compile ?

@sapk commented on GitHub (Oct 23, 2018): @typeless use gox to cross-compile ?

GiteaMirror commented

@typeless commented on GitHub (Oct 24, 2018):

@sapk I just installed it. But after typing gox -osarch="windows/linux" -tags sqlite_unlock_notify
, it only prints Number of parallel builds: 31. I've not figured out how to use it to run go test either.

Edit: fixed at https://github.com/mattn/go-sqlite3/pull/655

@typeless commented on GitHub (Oct 24, 2018): @sapk I just installed it. But after typing `gox -osarch="windows/linux" -tags sqlite_unlock_notify` , it only prints `Number of parallel builds: 31`. I've not figured out how to use it to run `go test` either. Edit: fixed at https://github.com/mattn/go-sqlite3/pull/655

GiteaMirror commented

@lunny commented on GitHub (Oct 24, 2018):

Just type make release-windows on gitea?

@lunny commented on GitHub (Oct 24, 2018): Just type `make release-windows` on gitea?

GiteaMirror commented

@typeless commented on GitHub (Oct 24, 2018):

https://github.com/go-gitea/gitea/pull/5162 fixes the CI failure and TAGS='bindata sqlite sqlite_unlock_notify' make release-windows passes.

@typeless commented on GitHub (Oct 24, 2018): https://github.com/go-gitea/gitea/pull/5162 fixes the CI failure and `TAGS='bindata sqlite sqlite_unlock_notify' make release-windows` passes.

GiteaMirror commented