github-starred/gitea
2
0
Fork 0
mirror of https://github.com/go-gitea/gitea.git synced 2026-03-12 02:24:21 -05:00
Code Issues 2.6k Packages Projects Releases 100 Wiki Activity

Gitlab importer worker crashes #6119

New Issue
Closed
opened 2025-11-02 06:45:39 -06:00 by GiteaMirror · 5 comments
GiteaMirror commented 2025-11-02 06:45:39 -06:00
Owner
Copy Link

Originally created by @Codeberg-org on GitHub (Oct 9, 2020).

Originally assigned to: @6543 on GitHub.

  • Gitea version (or commit ref): 1.12.5
  • Git version: 2.20.1
  • Operating system: Linux/Debian stable
  • Database (use [x]):
    • PostgreSQL
    • MySQL
    • MSSQL
    • SQLite
  • Can you reproduce the bug at https://try.gitea.io:
    • Yes (provide example URL)
    • No
    • Not relevant
  • Log gist:

Description

We see some dozen worker crashes seemingly related to the gitlab importer in our logs. Related issue: https://codeberg.org/Codeberg/Community/issues/304

Logs

2020/10/09 11:52:43 modules/task/task.go:51:handle() [E] Run task failed: Handler crashed with error: /build/go/src/runtime/panic.go:212 (0x44d709)
        /build/go/src/runtime/signal_unix.go:695 (0x44d558)
        /build/gitea/src/code.gitea.io/gitea/modules/migrations/gitlab.go:126 (0x19089c5)
        /build/gitea/src/code.gitea.io/gitea/modules/migrations/base/downloader.go:58 (0x1872e16)
        /build/gitea/src/code.gitea.io/gitea/modules/migrations/migrate.go:73 (0x190c76d)
        /build/gitea/src/code.gitea.io/gitea/modules/task/migrate.go:99 (0x192042f)
        /build/gitea/src/code.gitea.io/gitea/modules/task/task.go:28 (0x1920c97)
        /build/gitea/src/code.gitea.io/gitea/modules/task/task.go:50 (0x1920f51)
        /build/gitea/src/code.gitea.io/gitea/modules/queue/workerpool.go:383 (0x168de51)
        /build/gitea/src/code.gitea.io/gitea/modules/queue/workerpool.go:238 (0x16901d2)
        /build/go/src/runtime/asm_amd64.s:1373 (0x469720)
Originally created by @Codeberg-org on GitHub (Oct 9, 2020). Originally assigned to: @6543 on GitHub. - Gitea version (or commit ref): 1.12.5 - Git version: 2.20.1 - Operating system: Linux/Debian stable - Database (use `[x]`): - [ ] PostgreSQL - [X] MySQL - [ ] MSSQL - [ ] SQLite - Can you reproduce the bug at https://try.gitea.io: - [ ] Yes (provide example URL) - [X] No - [ ] Not relevant - Log gist: ## Description We see some dozen worker crashes seemingly related to the gitlab importer in our logs. Related issue: https://codeberg.org/Codeberg/Community/issues/304 ## Logs ```raw 2020/10/09 11:52:43 modules/task/task.go:51:handle() [E] Run task failed: Handler crashed with error: /build/go/src/runtime/panic.go:212 (0x44d709) /build/go/src/runtime/signal_unix.go:695 (0x44d558) /build/gitea/src/code.gitea.io/gitea/modules/migrations/gitlab.go:126 (0x19089c5) /build/gitea/src/code.gitea.io/gitea/modules/migrations/base/downloader.go:58 (0x1872e16) /build/gitea/src/code.gitea.io/gitea/modules/migrations/migrate.go:73 (0x190c76d) /build/gitea/src/code.gitea.io/gitea/modules/task/migrate.go:99 (0x192042f) /build/gitea/src/code.gitea.io/gitea/modules/task/task.go:28 (0x1920c97) /build/gitea/src/code.gitea.io/gitea/modules/task/task.go:50 (0x1920f51) /build/gitea/src/code.gitea.io/gitea/modules/queue/workerpool.go:383 (0x168de51) /build/gitea/src/code.gitea.io/gitea/modules/queue/workerpool.go:238 (0x16901d2) /build/go/src/runtime/asm_amd64.s:1373 (0x469720) ```
GiteaMirror added the type/bug label 2025-11-02 06:45:39 -06:00
GiteaMirror commented 2025-11-02 06:45:40 -06:00
Author
Owner
Copy Link

@zeripath commented on GitHub (Oct 11, 2020):

This is very odd...

The line marked responsible for the panic is:

aa1d9ef6cb/modules/migrations/gitlab.go (L126)

Which can only normally panic if g is nil.

aa1d9ef6cb/modules/migrations/base/downloader.go (L58)

Which would imply d.Downloader was nil here

aa1d9ef6cb/modules/migrations/migrate.go (L73)

Which would imply that downloader.Downloader was nil here.

aa1d9ef6cb/modules/migrations/migrate.go (L70)

Meaning downloader was nil here.

But that doesn't make sense because:

aa1d9ef6cb/modules/migrations/migrate.go (L52)

Should stop that...

However if we look again at the panic stack we see:

build/go/src/runtime/signal_unix.go:695

Which makes me suspicious that we're not seeing an npe here but rather makes me suspicious that it's something to do with the context and its channel being assigned to another struct.

I wonder if you're hitting some notify limit in regards to channels.There may be a hard limit of how many channels can be assigned to structs.

If there is we're in for a world of trouble as we'll have to consider migrating off channels on to atomic flags and some kind of polling but also consider if you've hit a go bug.

(There is an issue in that we've not got the recover result logged here which is strange because I was sure it should be...)

@zeripath commented on GitHub (Oct 11, 2020): This is very odd... The line marked responsible for the panic is: https://github.com/go-gitea/gitea/blob/aa1d9ef6cba71c80170ec8c9ef424cb780fa6481/modules/migrations/gitlab.go#L126 Which can only normally panic if g is nil. https://github.com/go-gitea/gitea/blob/aa1d9ef6cba71c80170ec8c9ef424cb780fa6481/modules/migrations/base/downloader.go#L58 Which would imply d.Downloader was nil here https://github.com/go-gitea/gitea/blob/aa1d9ef6cba71c80170ec8c9ef424cb780fa6481/modules/migrations/migrate.go#L73 Which would imply that downloader.Downloader was nil here. https://github.com/go-gitea/gitea/blob/aa1d9ef6cba71c80170ec8c9ef424cb780fa6481/modules/migrations/migrate.go#L70 Meaning downloader was nil here. But that doesn't make sense because: https://github.com/go-gitea/gitea/blob/aa1d9ef6cba71c80170ec8c9ef424cb780fa6481/modules/migrations/migrate.go#L52 Should stop that... However if we look again at the panic stack we see: build/go/src/runtime/signal_unix.go:695 Which makes me suspicious that we're not seeing an npe here but rather makes me suspicious that it's something to do with the context and its channel being assigned to another struct. I wonder if you're hitting some notify limit in regards to channels.There may be a hard limit of how many channels can be assigned to structs. If there is we're in for a world of trouble as we'll have to consider migrating off channels on to atomic flags and some kind of polling but also consider if you've hit a go bug. (There is an issue in that we've not got the recover result logged here which is strange because I was sure it should be...)
GiteaMirror commented 2025-11-02 06:45:40 -06:00
Author
Owner
Copy Link

@zeripath commented on GitHub (Oct 11, 2020):

What version of go are you building with?

@zeripath commented on GitHub (Oct 11, 2020): What version of go are you building with?
GiteaMirror commented 2025-11-02 06:45:40 -06:00
Author
Owner
Copy Link

@zeripath commented on GitHub (Oct 11, 2020):

(yup there's an issue with our recoverer here meaning we're not logging the underlying problem here.

9066d09c57/modules/task/migrate.go (L43)

Needs to log e as a %v argument just before the stack.)

@zeripath commented on GitHub (Oct 11, 2020): (yup there's an issue with our recoverer here meaning we're not logging the underlying problem here. https://github.com/go-gitea/gitea/blob/9066d09c57d3eaddd27c4986900e6afc0885e0c7/modules/task/migrate.go#L43 Needs to log e as a %v argument just before the stack.)
GiteaMirror commented 2025-11-02 06:45:40 -06:00
Author
Owner
Copy Link

@Codeberg-org commented on GitHub (Oct 11, 2020):

What version of go are you building with?

go1.14.9

@Codeberg-org commented on GitHub (Oct 11, 2020): > What version of go are you building with? go1.14.9
GiteaMirror commented 2025-11-02 06:45:40 -06:00
Author
Owner
Copy Link

@6543 commented on GitHub (Oct 11, 2020):

I have found 2 bugs (only in v1.12.x):

GitLab like token to be passed via:

 u.User = url.UserPassword("", opts.AuthUsername)

there is no option to pass a token as token-option to the migrator in v1.12

@6543 commented on GitHub (Oct 11, 2020): I have found 2 bugs (only in v1.12.x): * lable color with short-code do not work (e.g. `#234`) must be normalized to `#223344` * git clone from private repos pass token wrong way : GitHub Like it that way: https://github.com/go-gitea/gitea/blob/a0a77c9401c6ba89e9b24b609d455a7ab7303136/modules/migrations/gitea.go#L101 GitLab like token to be passed via: ```go u.User = url.UserPassword("", opts.AuthUsername) ``` there is no option to pass a token as token-option to the migrator in v1.12
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
$20
$250
$50
$500
backport/done
💎 Bounty
docs-update-needed
good first issue
hacktoberfest
issue/bounty
issue/confirmed
issue/critical
issue/duplicate
issue/needs-feedback
issue/not-a-bug
issue/regression
issue/stale
issue/workaround
lgtm/need 2
modifies/api
modifies/translation
outdated/backport/v1.18
outdated/theme/markdown
outdated/theme/timetracker
performance/bigrepo
performance/cpu
performance/memory
performance/speed
pr/breaking
proposal/accepted
proposal/rejected
pr/wip
pull-request
Mirrored from GitHub Pull Request
 reviewed/wontfix
💰 Rewarded
skip-changelog
status/blocked
topic/accessibility
topic/api
topic/authentication
topic/build
topic/code-linting
topic/commit-signing
topic/content-rendering
topic/deployment
topic/distribution
topic/federation
topic/gitea-actions
topic/issues
topic/lfs
topic/mobile
topic/moderation
topic/packages
topic/pr
topic/projects
topic/repo
topic/repo-migration
topic/security
topic/theme
topic/ui
topic/ui-interaction
topic/ux
topic/webhooks
topic/wiki
type/bug
type/deprecation
type/docs
type/enhancement
type/feature
type/miscellaneous
type/proposal
type/question
type/refactoring
type/summary
type/testing
type/upstream
No Label type/bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/gitea#6119
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?