drone.io integration broken after migration from gogs #5583

Closed
opened 2025-11-02 06:29:41 -06:00 by GiteaMirror · 7 comments
Owner

Originally created by @moepman on GitHub (Jun 19, 2020).

  • Gitea version: 1.12.0
  • Git version: 2.20.1
  • Operating system: Debian Buster (10.4)
  • Database:
    • PostgreSQL
    • MySQL
    • MSSQL
    • SQLite
  • Can you reproduce the bug at https://try.gitea.io:
    • Yes
    • No
    • Not relevant
  • Log gist:
gitea[1938]: [Macaron] 2020-06-19 07:48:33: Started GET /api/v1/user/repos for ::42
gitea[1938]: [Macaron] 2020-06-19 07:48:33: Completed GET /api/v1/user/repos 200 OK in 46.905635ms
gitea[1938]: [Macaron] 2020-06-19 07:48:33: Started GET /api/v1/user/repos for ::42
gitea[1938]: [Macaron] 2020-06-19 07:48:33: Completed GET /api/v1/user/repos 200 OK in 54.820797ms
[...]

Description

Note: the instance was migrated from Gogs 0.11.91 to Gitea 1.12.0. All old webhooks were deleted.

I setup a new drone server using the current version, created it as an oauth application as per its documentation and then allowed it to access gitea as my user. After the inital login it stays in "syncing" forever producing about 1000 log entries as seen above on my gitea server without ever finishing. The requests only stop once I stop the drone server. Drone itself does not seem to log anything useful, even with DRONE_LOGGING_TRACE=true.

I suspect this is an incompatibility with the paging in the repo API. Is there anything I can do to debug and help fix this problem?

Originally created by @moepman on GitHub (Jun 19, 2020). - Gitea version: 1.12.0 - Git version: 2.20.1 - Operating system: Debian Buster (10.4) - Database: - [x] PostgreSQL - [ ] MySQL - [ ] MSSQL - [ ] SQLite - Can you reproduce the bug at https://try.gitea.io: - [ ] Yes - [x] No - [ ] Not relevant - Log gist: ``` gitea[1938]: [Macaron] 2020-06-19 07:48:33: Started GET /api/v1/user/repos for ::42 gitea[1938]: [Macaron] 2020-06-19 07:48:33: Completed GET /api/v1/user/repos 200 OK in 46.905635ms gitea[1938]: [Macaron] 2020-06-19 07:48:33: Started GET /api/v1/user/repos for ::42 gitea[1938]: [Macaron] 2020-06-19 07:48:33: Completed GET /api/v1/user/repos 200 OK in 54.820797ms [...] ``` ## Description Note: the instance was migrated from Gogs 0.11.91 to Gitea 1.12.0. All old webhooks were deleted. I setup a new drone server using the current version, created it as an oauth application as per its documentation and then allowed it to access gitea as my user. After the inital login it stays in "syncing" forever producing about 1000 log entries as seen above on my gitea server without ever finishing. The requests only stop once I stop the drone server. Drone itself does not seem to log anything useful, even with `DRONE_LOGGING_TRACE=true`. I suspect this is an incompatibility with the paging in the repo API. Is there anything I can do to debug and help fix this problem?
GiteaMirror added the type/upstream label 2025-11-02 06:29:41 -06:00
Author
Owner

@lunny commented on GitHub (Jun 19, 2020):

Any logs?

@lunny commented on GitHub (Jun 19, 2020): Any logs?
Author
Owner

@moepman commented on GitHub (Jun 19, 2020):

On the drone.io side I have this:

drone-server[6999]: DEBU[0040] begin synchronization                         login=moepman
drone-server[6999]: DEBU[0040] syncer: begin repository sync                 login=moepman
drone-server[6999]: DEBU[0040]                                               fields.time="2020-06-19T08:26:33+02:00" latency="924.594µs" method=GET remote="127.0.0.1:50522" request=/ request-id=1dWjxxtquwp0KVvVOXLs0YBean1
drone-server[6999]: DEBU[0040] events: stream cancelled                      request-id=1dWjxqBT7v1p6leyTs6n13NTPJk
drone-server[6999]: DEBU[0040] events: stream closed                         request-id=1dWjxqBT7v1p6leyTs6n13NTPJk
drone-server[6999]: DEBU[0040] api: guest access                             request-id=1dWjxqBT7v1p6leyTs6n13NTPJk
drone-server[6999]: DEBU[0040]                                               fields.time="2020-06-19T08:26:33+02:00" latency=306.966922ms method=GET remote="[::1]:33174" request=/api/stream request-id=1dWjxqBT7v1p6leyTs6n13NTPJk
drone-server[6999]: DEBU[0040]                                               fields.time="2020-06-19T08:26:33+02:00" latency=1.413933ms method=GET remote="[::1]:33186" request=/api/user request-id=1dWjy1t1FtyWDp47SPBKJ3Au8y1
drone-server[6999]: DEBU[0040]                                               fields.time="2020-06-19T08:26:33+02:00" latency=9.526562ms method=GET remote="127.0.0.1:50526" request="/api/user/repos?latest=true" request-id=1dWjy2pmhPi23wDfdTLou2ct6EH
drone-server[6999]: DEBU[0040]                                               fields.time="2020-06-19T08:26:33+02:00" latency=3.516751ms method=GET remote="[::1]:33190" request=/api/user/builds/recent request-id=1dWjy3KEAGkXwa3Mdls4Yf8lF7O
drone-server[6999]: DEBU[0040]                                               fields.time="2020-06-19T08:26:33+02:00" latency=13.200574ms method=GET remote="127.0.0.1:50530" request="/api/user/repos?latest=true" request-id=1dWjy1XnkSjmn4UIK4kH1i50o7P
drone-server[6999]: DEBU[0040] events: stream opened                         request-id=1dWjxxWsALH4Qzd7ywkzv4xDEAL user.admin=true user.login=moepman
drone-server[6999]: DEBU[0060]                                               fields.time="2020-06-19T08:26:53+02:00" latency=2.180165ms method=GET remote="127.0.0.1:50542" request=/api/user request-id=1dWk0THrJnHS7PLvYl1pp4qf1jk
drone-server[6999]: DEBU[0070]                                               fields.time="2020-06-19T08:27:03+02:00" latency=1.845105ms method=GET remote="[::1]:33210" request=/api/user request-id=1dWk1iX75jgkOFZUNKNWDI272GL
drone-server[6999]: DEBU[0080]                                               fields.time="2020-06-19T08:27:13+02:00" latency=2.060026ms method=GET remote="127.0.0.1:50554" request=/api/user request-id=1dWk2yL5aVuPtO5ga5SJ9i2Tpjm
drone-server[6999]: DEBU[0090]                                               fields.time="2020-06-19T08:27:23+02:00" latency=2.069861ms method=GET remote="[::1]:33222" request=/api/user request-id=1dWk4Hpaj87f5yIwIsilaWQgiiu
drone-server[6999]: DEBU[0100]                                               fields.time="2020-06-19T08:27:33+02:00" latency=1.010055ms method=GET remote="127.0.0.1:50566" request=/api/user request-id=1dWk5W6IcoKJrOmkvzaXthaOygu
drone-server[6999]: DEBU[0110]                                               fields.time="2020-06-19T08:27:43+02:00" latency=2.088476ms method=GET remote="[::1]:33234" request=/api/user request-id=1dWk6quCYGeHpWnjvdfk0qqvyz5
drone-server[6999]: DEBU[0120]                                               fields.time="2020-06-19T08:27:53+02:00" latency=2.089175ms method=GET remote="127.0.0.1:50578" request=/api/user request-id=1dWk82BvmboTTbTL9WTp7zv02ok

On the gitea side I can enable SQL logging which gives me a ton of the same queries with each of the /api/v1/user/repos requests. Should I get them for a few of those and attach?

@moepman commented on GitHub (Jun 19, 2020): On the drone.io side I have this: ``` drone-server[6999]: DEBU[0040] begin synchronization login=moepman drone-server[6999]: DEBU[0040] syncer: begin repository sync login=moepman drone-server[6999]: DEBU[0040] fields.time="2020-06-19T08:26:33+02:00" latency="924.594µs" method=GET remote="127.0.0.1:50522" request=/ request-id=1dWjxxtquwp0KVvVOXLs0YBean1 drone-server[6999]: DEBU[0040] events: stream cancelled request-id=1dWjxqBT7v1p6leyTs6n13NTPJk drone-server[6999]: DEBU[0040] events: stream closed request-id=1dWjxqBT7v1p6leyTs6n13NTPJk drone-server[6999]: DEBU[0040] api: guest access request-id=1dWjxqBT7v1p6leyTs6n13NTPJk drone-server[6999]: DEBU[0040] fields.time="2020-06-19T08:26:33+02:00" latency=306.966922ms method=GET remote="[::1]:33174" request=/api/stream request-id=1dWjxqBT7v1p6leyTs6n13NTPJk drone-server[6999]: DEBU[0040] fields.time="2020-06-19T08:26:33+02:00" latency=1.413933ms method=GET remote="[::1]:33186" request=/api/user request-id=1dWjy1t1FtyWDp47SPBKJ3Au8y1 drone-server[6999]: DEBU[0040] fields.time="2020-06-19T08:26:33+02:00" latency=9.526562ms method=GET remote="127.0.0.1:50526" request="/api/user/repos?latest=true" request-id=1dWjy2pmhPi23wDfdTLou2ct6EH drone-server[6999]: DEBU[0040] fields.time="2020-06-19T08:26:33+02:00" latency=3.516751ms method=GET remote="[::1]:33190" request=/api/user/builds/recent request-id=1dWjy3KEAGkXwa3Mdls4Yf8lF7O drone-server[6999]: DEBU[0040] fields.time="2020-06-19T08:26:33+02:00" latency=13.200574ms method=GET remote="127.0.0.1:50530" request="/api/user/repos?latest=true" request-id=1dWjy1XnkSjmn4UIK4kH1i50o7P drone-server[6999]: DEBU[0040] events: stream opened request-id=1dWjxxWsALH4Qzd7ywkzv4xDEAL user.admin=true user.login=moepman drone-server[6999]: DEBU[0060] fields.time="2020-06-19T08:26:53+02:00" latency=2.180165ms method=GET remote="127.0.0.1:50542" request=/api/user request-id=1dWk0THrJnHS7PLvYl1pp4qf1jk drone-server[6999]: DEBU[0070] fields.time="2020-06-19T08:27:03+02:00" latency=1.845105ms method=GET remote="[::1]:33210" request=/api/user request-id=1dWk1iX75jgkOFZUNKNWDI272GL drone-server[6999]: DEBU[0080] fields.time="2020-06-19T08:27:13+02:00" latency=2.060026ms method=GET remote="127.0.0.1:50554" request=/api/user request-id=1dWk2yL5aVuPtO5ga5SJ9i2Tpjm drone-server[6999]: DEBU[0090] fields.time="2020-06-19T08:27:23+02:00" latency=2.069861ms method=GET remote="[::1]:33222" request=/api/user request-id=1dWk4Hpaj87f5yIwIsilaWQgiiu drone-server[6999]: DEBU[0100] fields.time="2020-06-19T08:27:33+02:00" latency=1.010055ms method=GET remote="127.0.0.1:50566" request=/api/user request-id=1dWk5W6IcoKJrOmkvzaXthaOygu drone-server[6999]: DEBU[0110] fields.time="2020-06-19T08:27:43+02:00" latency=2.088476ms method=GET remote="[::1]:33234" request=/api/user request-id=1dWk6quCYGeHpWnjvdfk0qqvyz5 drone-server[6999]: DEBU[0120] fields.time="2020-06-19T08:27:53+02:00" latency=2.089175ms method=GET remote="127.0.0.1:50578" request=/api/user request-id=1dWk82BvmboTTbTL9WTp7zv02ok ``` On the gitea side I can enable SQL logging which gives me a ton of the same queries with each of the `/api/v1/user/repos` requests. Should I get them for a few of those and attach?
Author
Owner

@moepman commented on GitHub (Jun 19, 2020):

I have run tcpdump to see what gitea returns to drone: it will always send a json object containing 30 repositories of 33 in total (X-Total-Count: 33). I still don't know why drone just won't show only those 30 but rather fetches the list in an infinite loop.

@moepman commented on GitHub (Jun 19, 2020): I have run tcpdump to see what gitea returns to drone: it will always send a json object containing 30 repositories of 33 in total (`X-Total-Count: 33`). I still don't know why drone just won't show only those 30 but rather fetches the list in an infinite loop.
Author
Owner

@CirnoT commented on GitHub (Jun 19, 2020):

Does drone attempt to request another page, or the request URI query stays the same all the time?

@CirnoT commented on GitHub (Jun 19, 2020): Does drone attempt to request another page, or the request URI query stays the same all the time?
Author
Owner

@moepman commented on GitHub (Jun 19, 2020):

As far as I can tell drone always requests the same URL, even after minutes.

@CirnoT your PR for drone seems to suggest that this is indeed purely a problem with drone?

@moepman commented on GitHub (Jun 19, 2020): As far as I can tell drone always requests the same URL, even after minutes. @CirnoT your PR for drone seems to suggest that this is indeed purely a problem with drone?
Author
Owner

@CirnoT commented on GitHub (Jun 19, 2020):

Most precisely with go-scm, and us adding proper pagination in v1.12 caused the issue to manifest between Drone and go-scm, because pagination handling is done in Drone (via checking Link header) but then request for page 2 is passed to go-scm which in turn would return page 1

@CirnoT commented on GitHub (Jun 19, 2020): Most precisely with go-scm, and us adding proper pagination in v1.12 caused the issue to manifest between Drone and go-scm, because pagination handling is done in Drone (via checking Link header) but then request for page 2 is passed to go-scm which in turn would return page 1
Author
Owner

@moepman commented on GitHub (Jun 29, 2020):

I can confirm that after upgrading to the current version of drone (which includes https://github.com/drone/go-scm/pull/66) everything works fine.

@moepman commented on GitHub (Jun 29, 2020): I can confirm that after upgrading to the current version of drone (which includes https://github.com/drone/go-scm/pull/66) everything works fine.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/gitea#5583