HTTP 500 Error and site unresponsive when using assignment notifications via slack webhook #2921

New Issue

GiteaMirror · 2025-11-02T04:54:10-06:00

GiteaMirror commented

2025-11-02 04:54:10 -06:00

Originally created by @jimpson on GitHub (Feb 15, 2019).

Gitea version (or commit ref): 1.7.1
Git version: 2.14.1
Operating system: Windows Server 2012 R2
Database (use [x]):
- PostgreSQL
- MySQL
- MSSQL
- SQLite
Can you reproduce the bug at https://try.gitea.io:
- Yes (provide example URL)
- No
- Not relevant
Log gist:

2019/02/15 11:12:52 [...s/issue_assignees.go:141 ChangeAssignee()] [E] PrepareWebhooks [is_pull: true, remove_assignee: false]: GetSlackPayload: read tcp SERVER_IP_ADDRESS_REDACTED->SQL_IP_ADDRESS_REDACTED:1433: i/o timeout
2019/02/15 11:13:05 [...user/notification.go:33 GetNotificationCount()] [E] GetNotificationCount: sql: expected 5 destination arguments in Scan, not 1

#5639 Assigning a PR causes the server to hang

Description

When a slack webhook is defined to send notifications related to pull requests, specifically assignment events, the Gitea server takes a long time (sql timeout length) to respond and once it does, there appears to no longer be any data being pulled into the web view. I have tested this with all other events and there does not seem to be any issues with the webhook and PRs other than the PR assignment event. It should be noted that after restarting the server the assignee on the PR is sometimes populated and sometimes missing. I'm guessing there is a deadlock on the SQL side of things, but that goes beyond my expertise.
...

Steps To Reproduce

Setup a slack webhook and select the "Pull Request" custom event
Open a PR and assign it to a user
Note the unresponsiveness
After ~30 seconds refresh the page
Note the appearance of missing data
Restart the server
Go back to the PR that was opened previously
Note if the assignment was successful or failed

Screenshots

Originally created by @jimpson on GitHub (Feb 15, 2019).  - Gitea version (or commit ref): 1.7.1 - Git version: 2.14.1 - Operating system: Windows Server 2012 R2 - Database (use `[x]`): - [ ] PostgreSQL - [ ] MySQL - [x] MSSQL - [ ] SQLite - Can you reproduce the bug at https://try.gitea.io: - [ ] Yes (provide example URL) - [ ] No - [x] Not relevant - Log gist: ```sh 2019/02/15 11:12:52 [...s/issue_assignees.go:141 ChangeAssignee()] [E] PrepareWebhooks [is_pull: true, remove_assignee: false]: GetSlackPayload: read tcp SERVER_IP_ADDRESS_REDACTED->SQL_IP_ADDRESS_REDACTED:1433: i/o timeout 2019/02/15 11:13:05 [...user/notification.go:33 GetNotificationCount()] [E] GetNotificationCount: sql: expected 5 destination arguments in Scan, not 1 ``` ## Possibly Related To #5639 Assigning a PR causes the server to hang ## Description When a slack webhook is defined to send notifications related to pull requests, specifically assignment events, the Gitea server takes a long time (sql timeout length) to respond and once it does, there appears to no longer be any data being pulled into the web view. I have tested this with all other events and there does not seem to be any issues with the webhook and PRs other than the PR assignment event. It should be noted that after restarting the server the assignee on the PR is sometimes populated and sometimes missing. I'm guessing there is a deadlock on the SQL side of things, but that goes beyond my expertise. ... ## Steps To Reproduce - Setup a slack webhook and select the "Pull Request" custom event - Open a PR and assign it to a user - Note the unresponsiveness - After ~30 seconds refresh the page - Note the appearance of missing data - Restart the server - Go back to the PR that was opened previously - Note if the assignment was successful or failed ## Screenshots ![webhook](https://user-images.githubusercontent.com/17748853/52869534-d9d0f300-3113-11e9-8ee1-dd8efcdcaef5.PNG) ![500](https://user-images.githubusercontent.com/17748853/52869561-e9e8d280-3113-11e9-8fb5-2c7323c77a85.PNG) ![missing-data](https://user-images.githubusercontent.com/17748853/52869563-ec4b2c80-3113-11e9-9c84-f5b24816b5dd.PNG) ![missing-data-2](https://user-images.githubusercontent.com/17748853/52869565-eead8680-3113-11e9-9f59-9486a9e87db9.PNG)

GiteaMirror added the issue/confirmed type/bug labels 2025-11-02 04:54:10 -06:00

GiteaMirror closed this issue

2025-11-02 04:54:10 -06:00

GiteaMirror commented

2025-11-02 04:54:11 -06:00

@zeripath commented on GitHub (Feb 17, 2019):

This is suspicious for a sqlite deadlock bug.

I think a few log lines above the error might be helpful too. I suspect that the log lines you've reported might already be deadlocked but I'll look into them.

@zeripath commented on GitHub (Feb 17, 2019): This is suspicious for a sqlite deadlock bug. I think a few log lines above the error might be helpful too. I suspect that the log lines you've reported might already be deadlocked but I'll look into them.

GiteaMirror commented

2025-11-02 04:54:11 -06:00

@lunny commented on GitHub (Feb 17, 2019):

@zeripath but his database is mssql.

@lunny commented on GitHub (Feb 17, 2019): @zeripath but his database is mssql.

GiteaMirror commented

2025-11-02 04:54:11 -06:00

@zeripath commented on GitHub (Feb 17, 2019):

Oh yeah.

Still suspicious for an SQL deadlock bug though...

@zeripath commented on GitHub (Feb 17, 2019): Oh yeah. Still suspicious for an SQL deadlock bug though...

GiteaMirror commented

2025-11-02 04:54:12 -06:00

@zeripath commented on GitHub (Feb 17, 2019):

Yup there's a deadlock:

https://github.com/go-gitea/gitea/blob/master/models/webhook_slack.go#L304

Calls list, err := MakeAssigneeList(&Issue{ID: p.PullRequest.ID}) which is a database function, whilst still within the transaction opened in:

https://github.com/go-gitea/gitea/blob/master/models/issue_assignees.go#L133

@zeripath commented on GitHub (Feb 17, 2019): Yup there's a deadlock: https://github.com/go-gitea/gitea/blob/master/models/webhook_slack.go#L304 Calls `list, err := MakeAssigneeList(&Issue{ID: p.PullRequest.ID})` which is a database function, whilst still within the transaction opened in: https://github.com/go-gitea/gitea/blob/master/models/issue_assignees.go#L133

GiteaMirror commented

2025-11-02 04:54:12 -06:00

@zeripath commented on GitHub (Feb 17, 2019):

Ceterum autem censeo x exempla monstrabit esse delendam.

@zeripath commented on GitHub (Feb 17, 2019): Ceterum autem censeo x exempla monstrabit esse delendam.

GiteaMirror commented

2025-11-02 04:54:12 -06:00

@zeripath commented on GitHub (Feb 17, 2019):

OK, it's not just Slack, Discord and Dingtalk have the problem too.

Now strangely there is an Assignees member in api.PullRequest which has been there for approximately as long as these lines - so I'm not sure why this wasn't used - perhaps it wasn't there...

Now, there's a valid question about whether webhook event generation should occur within the same transaction and the same request as the actual assignment but I'm going to ignore that for the moment and fix the issue at hand.

@zeripath commented on GitHub (Feb 17, 2019): OK, it's not just Slack, Discord and Dingtalk have the problem too. Now strangely there is an `Assignees` member in `api.PullRequest` which has been there for approximately as long as these lines - so I'm not sure why this wasn't used - perhaps it wasn't there... Now, there's a valid question about whether webhook event generation should occur within the same transaction and the same request as the actual assignment but I'm going to ignore that for the moment and fix the issue at hand.

GiteaMirror referenced this issue

2025-11-02 12:15:51 -06:00

[PR #2921] [MERGED] Add changelog for v1.3.0-rc1 #16664

GiteaMirror referenced this issue

2025-11-02 12:20:01 -06:00

[PR #3209] [CLOSED] Please check the following: #16827

Sign in to join this conversation.