github-starred/pangolin
2
0
Fork 0
mirror of https://github.com/fosrl/pangolin.git synced 2026-05-22 09:32:36 -05:00
Code Issues 959 Packages Projects Releases 74 Wiki Activity

[GH-ISSUE #92] File transfer limit #1282

New Issue
Closed
opened 2026-04-16 07:55:02 -05:00 by GiteaMirror · 24 comments
GiteaMirror commented 2026-04-16 07:55:02 -05:00
Owner
Copy Link

Originally created by @ItsSK on GitHub (Jan 21, 2025).
Original GitHub issue: https://github.com/fosrl/pangolin/issues/92

I wanted to replace my Cloudflare Tunnel with Pangolin since Cloudflare has a 100MB limit. I noticed that Pangolin has a similar limit at around 370MB, is it possible to remove the limit?

Originally created by @ItsSK on GitHub (Jan 21, 2025). Original GitHub issue: https://github.com/fosrl/pangolin/issues/92 I wanted to replace my Cloudflare Tunnel with Pangolin since Cloudflare has a 100MB limit. I noticed that Pangolin has a similar limit at around 370MB, is it possible to remove the limit?
GiteaMirror commented 2026-04-16 07:55:03 -05:00
Author
Owner
Copy Link

@oschwartz10612 commented on GitHub (Jan 22, 2025):

Hi,

I am not sure what limit you are referring to? We should have no baked in data transfer limits.

What is happening when you get to 370? What service are you proxying that fails? Nextcloud?

<!-- gh-comment-id:2606370431 --> @oschwartz10612 commented on GitHub (Jan 22, 2025): Hi, I am not sure what limit you are referring to? We should have no baked in data transfer limits. What is happening when you get to 370? What service are you proxying that fails? Nextcloud?
GiteaMirror commented 2026-04-16 07:55:03 -05:00
Author
Owner
Copy Link

@ItsSK commented on GitHub (Jan 22, 2025):

Heyy, I did some testing with multiple docker applications, the test was preformed on the same application with Pangolin & NginxProxyManager being the only difference, Just sharing my findings

File: 1.16GB GoPro video
Pangolin:
Nextcloud: Uploaded successfully and playback works
Immich: fails to upload (Error in Immich log: [39m Request error while uploading file, cleaning up)
Synology Photos: says uploaded but file is nowhere to be found.

Nginx Proxy Manager:
Nextcloud: Uploaded successfully and playback works
Immich: Uploaded successfully and playback works
Synology Photos: Uploaded successfully and playback works

<!-- gh-comment-id:2608238231 --> @ItsSK commented on GitHub (Jan 22, 2025): Heyy, I did some testing with multiple docker applications, the test was preformed on the same application with Pangolin & NginxProxyManager being the only difference, Just sharing my findings File: 1.16GB GoPro video Pangolin: Nextcloud: Uploaded successfully and playback works Immich: fails to upload (Error in Immich log: [39m Request error while uploading file, cleaning up) Synology Photos: says uploaded but file is nowhere to be found. Nginx Proxy Manager: Nextcloud: Uploaded successfully and playback works Immich: Uploaded successfully and playback works Synology Photos: Uploaded successfully and playback works
GiteaMirror commented 2026-04-16 07:55:04 -05:00
Author
Owner
Copy Link

@netxer commented on GitHub (Jan 22, 2025):

Heyy, I did some testing with multiple docker applications, the test was preformed on the same application with Pangolin & NginxProxyManager being the only difference, Just sharing my findings

File: 1.16GB GoPro video Pangolin: Nextcloud: Uploaded successfully and playback works Immich: fails to upload (Error in Immich log: [39m �[33mRequest error while uploading file, cleaning up�[39m) Synology: Photos: says uploaded but file is nowhere to be found.

Nginx Proxy Manager: Nextcloud: Uploaded successfully and playback works Immich: Uploaded successfully and playback works Synology: Uploaded successfully and playback works

I had issue with synology photo as well, tried to upload new pic failed half way.

<!-- gh-comment-id:2608244850 --> @netxer commented on GitHub (Jan 22, 2025): > Heyy, I did some testing with multiple docker applications, the test was preformed on the same application with Pangolin & NginxProxyManager being the only difference, Just sharing my findings > > File: 1.16GB GoPro video Pangolin: Nextcloud: Uploaded successfully and playback works Immich: fails to upload (Error in Immich log: [39m �[33mRequest error while uploading file, cleaning up�[39m) Synology: Photos: says uploaded but file is nowhere to be found. > > Nginx Proxy Manager: Nextcloud: Uploaded successfully and playback works Immich: Uploaded successfully and playback works Synology: Uploaded successfully and playback works I had issue with synology photo as well, tried to upload new pic failed half way.
GiteaMirror commented 2026-04-16 07:55:05 -05:00
Author
Owner
Copy Link

@oschwartz10612 commented on GitHub (Jan 23, 2025):

This is all very interesting thanks for the feedback. I just want to confirm: are you running the latest version of gerbil and newt? Should be beta.3 and beta.4 respectively. I fixed many issues around MTU, MSS, and the TCP proxy.

<!-- gh-comment-id:2608845403 --> @oschwartz10612 commented on GitHub (Jan 23, 2025): This is all very interesting thanks for the feedback. I just want to confirm: are you running the latest version of gerbil and newt? Should be `beta.3` and `beta.4` respectively. I fixed many issues around MTU, MSS, and the TCP proxy.
GiteaMirror commented 2026-04-16 07:55:05 -05:00
Author
Owner
Copy Link

@ItsSK commented on GitHub (Jan 23, 2025):

For gerbil I'm running beta.3 and for newt all I show is fosrl/newt:latest

<!-- gh-comment-id:2609036803 --> @ItsSK commented on GitHub (Jan 23, 2025): For gerbil I'm running `beta.3` and for newt all I show is `fosrl/newt:latest`
GiteaMirror commented 2026-04-16 07:55:06 -05:00
Author
Owner
Copy Link

@netxer commented on GitHub (Jan 23, 2025):

I had issue with synology photo as well, tried to upload new pic failed half way.

did try again today, it seemed to work fine as long long the speed bandwidth is above 100kb as sec( was testing yesterday on 4g phone connection)

<!-- gh-comment-id:2609759436 --> @netxer commented on GitHub (Jan 23, 2025): > I had issue with synology photo as well, tried to upload new pic failed half way. did try again today, it seemed to work fine as long long the speed bandwidth is above 100kb as sec( was testing yesterday on 4g phone connection)
GiteaMirror commented 2026-04-16 07:55:07 -05:00
Author
Owner
Copy Link

@oschwartz10612 commented on GitHub (Jan 23, 2025):

For gerbil I'm running beta.3 and for newt all I show is fosrl/newt:latest

Just to be 100% sure have you docker pulled the latest container if you are using the latest tag? Not sure when you set it up...

<!-- gh-comment-id:2610002543 --> @oschwartz10612 commented on GitHub (Jan 23, 2025): > For gerbil I'm running `beta.3` and for newt all I show is `fosrl/newt:latest` Just to be 100% sure have you docker pulled the latest container if you are using the latest tag? Not sure when you set it up...
GiteaMirror commented 2026-04-16 07:55:07 -05:00
Author
Owner
Copy Link

@dhanadhan commented on GitHub (Jan 23, 2025):

I am also experiencing the same issue uploading bigger files/videos to Immich. Running all containers with the latest tag.

<!-- gh-comment-id:2610848301 --> @dhanadhan commented on GitHub (Jan 23, 2025): I am also experiencing the same issue uploading bigger files/videos to Immich. Running all containers with the latest tag.
GiteaMirror commented 2026-04-16 07:55:08 -05:00
Author
Owner
Copy Link

@oschwartz10612 commented on GitHub (Jan 24, 2025):

One thing you could try but I would not expect to instantly solve the issue is the MTU. I set it pretty low by default on the latest builds at 1280 to try to support all kinds of networks. You can use the --mtu or the MTU env var to set it higher to something like 1420 on BOTH gerbil and newt and see if it helps.

At the least it should speed it up a bit.

<!-- gh-comment-id:2611467190 --> @oschwartz10612 commented on GitHub (Jan 24, 2025): One thing you could try but I would not expect to instantly solve the issue is the MTU. I set it pretty low by default on the latest builds at 1280 to try to support all kinds of networks. You can use the `--mtu` or the `MTU` env var to set it higher to something like 1420 on BOTH gerbil and newt and see if it helps. At the least it should speed it up a bit.
GiteaMirror commented 2026-04-16 07:55:09 -05:00
Author
Owner
Copy Link

@dhanadhan commented on GitHub (Jan 24, 2025):

One thing you could try but I would not expect to instantly solve the issue is the MTU. I set it pretty low by default on the latest builds at 1280 to try to support all kinds of networks. You can use the --mtu or the MTU env var to set it higher to something like 1420 on BOTH gerbil and newt and see if it helps.

At the least it should speed it up a bit.

Just tried with MTU 1420 and 1400 set on gerbil and newt, nothing changed. Upload still failed after about 370 MB.

<!-- gh-comment-id:2611680424 --> @dhanadhan commented on GitHub (Jan 24, 2025): > One thing you could try but I would not expect to instantly solve the issue is the MTU. I set it pretty low by default on the latest builds at 1280 to try to support all kinds of networks. You can use the `--mtu` or the `MTU` env var to set it higher to something like 1420 on BOTH gerbil and newt and see if it helps. > > At the least it should speed it up a bit. Just tried with MTU 1420 and 1400 set on gerbil and newt, nothing changed. Upload still failed after about 370 MB.
GiteaMirror commented 2026-04-16 07:55:09 -05:00
Author
Owner
Copy Link

@oschwartz10612 commented on GitHub (Jan 25, 2025):

I am currently investigating if we need some Traefik config to resolve this. I am looking at a comment in this thread: https://community.traefik.io/t/increase-maximum-body-size-without-buffering/15928/6

I will do some experimentation on my side, but in the meantime could anyone here try to reproduce and see if there is any output in the Traefik logs? You might want to turn on DEBUG for the log level in traefik_config.yml

<!-- gh-comment-id:2614042348 --> @oschwartz10612 commented on GitHub (Jan 25, 2025): I am currently investigating if we need some Traefik config to resolve this. I am looking at a comment in this thread: https://community.traefik.io/t/increase-maximum-body-size-without-buffering/15928/6 I will do some experimentation on my side, but in the meantime could anyone here try to reproduce and see if there is any output in the Traefik logs? You might want to turn on DEBUG for the log level in `traefik_config.yml`
GiteaMirror commented 2026-04-16 07:55:10 -05:00
Author
Owner
Copy Link

@dhanadhan commented on GitHub (Jan 26, 2025):

I am currently investigating if we need some Traefik config to resolve this. I am looking at a comment in this thread: https://community.traefik.io/t/increase-maximum-body-size-without-buffering/15928/6

I will do some experimentation on my side, but in the meantime could anyone here try to reproduce and see if there is any output in the Traefik logs? You might want to turn on DEBUG for the log level in traefik_config.yml

Not sure if I am reading the right logs. I changed traefik config to add DEBUG under logs. Then watched for traefik container logs while uploading. This is what it showed when it failed to upload.

DBG github.com/traefik/traefik/v3/pkg/server/service/proxy.go:100 > 502 Bad Gateway error="readfrom tcp 100.89.128.1:56390->100.89.128.4:47672: i/o timeout

<!-- gh-comment-id:2614181854 --> @dhanadhan commented on GitHub (Jan 26, 2025): > I am currently investigating if we need some Traefik config to resolve this. I am looking at a comment in this thread: https://community.traefik.io/t/increase-maximum-body-size-without-buffering/15928/6 > > I will do some experimentation on my side, but in the meantime could anyone here try to reproduce and see if there is any output in the Traefik logs? You might want to turn on DEBUG for the log level in `traefik_config.yml` Not sure if I am reading the right logs. I changed traefik config to add DEBUG under logs. Then watched for traefik container logs while uploading. This is what it showed when it failed to upload. > DBG github.com/traefik/traefik/v3/pkg/server/service/proxy.go:100 > 502 Bad Gateway error="readfrom tcp 100.89.128.1:56390->100.89.128.4:47672: i/o timeout
GiteaMirror commented 2026-04-16 07:55:11 -05:00
Author
Owner
Copy Link

@dhanadhan commented on GitHub (Jan 26, 2025):

Made the change mentioned in the above link, and upload was Successful!

transport:
  respondingTimeouts:
    readTimeout: 30m
<!-- gh-comment-id:2614185754 --> @dhanadhan commented on GitHub (Jan 26, 2025): Made the change mentioned in the above link, and upload was Successful! transport: respondingTimeouts: readTimeout: 30m
GiteaMirror commented 2026-04-16 07:55:11 -05:00
Author
Owner
Copy Link

@oschwartz10612 commented on GitHub (Jan 26, 2025):

@dhanadhan Fantastic!

IDK if anyone else can confirm if this helps them. I will put this into the Traefik config for the next release.

<!-- gh-comment-id:2614186713 --> @oschwartz10612 commented on GitHub (Jan 26, 2025): @dhanadhan Fantastic! IDK if anyone else can confirm if this helps them. I will put this into the Traefik config for the next release.
GiteaMirror commented 2026-04-16 07:55:12 -05:00
Author
Owner
Copy Link

@ItsSK commented on GitHub (Jan 26, 2025):

Made the change mentioned in the above link, and upload was Successful!

websecure: address: ":443" transport: respondingTimeouts: readTimeout: 30m

What was the exact change?

<!-- gh-comment-id:2614186867 --> @ItsSK commented on GitHub (Jan 26, 2025): > Made the change mentioned in the above link, and upload was Successful! > > websecure: address: ":443" transport: respondingTimeouts: readTimeout: 30m What was the exact change?
GiteaMirror commented 2026-04-16 07:55:13 -05:00
Author
Owner
Copy Link

@dhanadhan commented on GitHub (Jan 26, 2025):

Made the change mentioned in the above link, and upload was Successful!
websecure: address: ":443" transport: respondingTimeouts: readTimeout: 30m

What was the exact change?

Check the below link and look in the last comment for formatting. Update that in traefik_config.yml. Then reboot the stack.
https://community.traefik.io/t/increase-maximum-body-size-without-buffering/15928/6

<!-- gh-comment-id:2614187232 --> @dhanadhan commented on GitHub (Jan 26, 2025): > > Made the change mentioned in the above link, and upload was Successful! > > websecure: address: ":443" transport: respondingTimeouts: readTimeout: 30m > > What was the exact change? Check the below link and look in the last comment for formatting. Update that in traefik_config.yml. Then reboot the stack. https://community.traefik.io/t/increase-maximum-body-size-without-buffering/15928/6
GiteaMirror commented 2026-04-16 07:55:14 -05:00
Author
Owner
Copy Link

@oschwartz10612 commented on GitHub (Jan 26, 2025):

Here is the change I made, It didnt work.

Bummer! Just checking you did restart Traefik right? Does the upload fail at the same point?

<!-- gh-comment-id:2614190307 --> @oschwartz10612 commented on GitHub (Jan 26, 2025): > Here is the change I made, It didnt work. Bummer! Just checking you did restart Traefik right? Does the upload fail at the same point?
GiteaMirror commented 2026-04-16 07:55:14 -05:00
Author
Owner
Copy Link

@ItsSK commented on GitHub (Jan 26, 2025):

Sorry I spoke too soon, I did restart Traefik and the fix did work. I tested both Immich Web and Mobile app.
websecure: address: ":443" transport: respondingTimeouts: readTimeout: 30m

Image
<!-- gh-comment-id:2614193307 --> @ItsSK commented on GitHub (Jan 26, 2025): Sorry I spoke too soon, I did restart Traefik and the fix did work. I tested both Immich Web and Mobile app. `websecure: address: ":443" transport: respondingTimeouts: readTimeout: 30m` <img width="449" alt="Image" src="https://github.com/user-attachments/assets/2e22e498-9a45-4d92-852d-4b1eedee26f8" />
GiteaMirror commented 2026-04-16 07:55:15 -05:00
Author
Owner
Copy Link

@oschwartz10612 commented on GitHub (Jan 26, 2025):

Great! I will include this in the next release and close this when it is out. Thanks for testing guys!

<!-- gh-comment-id:2614193539 --> @oschwartz10612 commented on GitHub (Jan 26, 2025): Great! I will include this in the next release and close this when it is out. Thanks for testing guys!
GiteaMirror commented 2026-04-16 07:55:16 -05:00
Author
Owner
Copy Link

@ItsSK commented on GitHub (Jan 26, 2025):

I tested Synology photos as well and it did upload successfully, though the progress bar was not showing and then jumped from 0 to 100...

Image

<!-- gh-comment-id:2614200817 --> @ItsSK commented on GitHub (Jan 26, 2025): I tested Synology photos as well and it did upload successfully, though the progress bar was not showing and then jumped from 0 to 100... ![Image](https://github.com/user-attachments/assets/bebe2208-538e-4802-92a8-d2885154828b)
GiteaMirror commented 2026-04-16 07:55:17 -05:00
Author
Owner
Copy Link

@miloschwartz commented on GitHub (Jan 30, 2025):

Added readTimeout in beta.9!

<!-- gh-comment-id:2624991658 --> @miloschwartz commented on GitHub (Jan 30, 2025): Added `readTimeout` in beta.9!
GiteaMirror commented 2026-04-16 07:55:18 -05:00
Author
Owner
Copy Link

@Cantello commented on GitHub (Feb 22, 2025):

Would this fix be present just by updating to the the latest version or do I need to edit traefik_config.yaml as well? I checked and readTimeout: 30m is already present. However, I am the one with the problem mentioned above with pdfding and uploading files larger than ~100MB. Setting traefik logging to DEBUG does not show anything unusual after the error message.

Any idea if this could be caused by pangolin/traefik?

<!-- gh-comment-id:2676428181 --> @Cantello commented on GitHub (Feb 22, 2025): Would this fix be present just by updating to the the latest version or do I need to edit traefik_config.yaml as well? I checked and `readTimeout: 30m` is already present. However, I am the one with the problem mentioned above with pdfding and uploading files larger than ~100MB. Setting traefik logging to DEBUG does not show anything unusual after the error message. Any idea if this could be caused by pangolin/traefik?
GiteaMirror commented 2026-04-16 07:55:18 -05:00
Author
Owner
Copy Link

@TuncTaylan commented on GitHub (Apr 10, 2025):

I also have this problem, even with the readTimeout set to 30m.

Edit: With enough RAM I'm gettings following log:
crowdsec | time="2025-04-10T16:28:59Z" level=warning msg="Disrupting transaction with body size above the configured limit (Action Reject)" band=inband chain_rule_id=3231508716 name=myAppSecComponent runner_uuid=1111e24a-bc7f-4862-9709-b69e66382a5a tx_id=ed578e15-f17c-4166-9172-75816b4dff66 type=appsec

The question is, where is that configuration?

<!-- gh-comment-id:2794415307 --> @TuncTaylan commented on GitHub (Apr 10, 2025): I also have this problem, even with the readTimeout set to 30m. Edit: With enough RAM I'm gettings following log: `crowdsec | time="2025-04-10T16:28:59Z" level=warning msg="Disrupting transaction with body size above the configured limit (Action Reject)" band=inband chain_rule_id=3231508716 name=myAppSecComponent runner_uuid=1111e24a-bc7f-4862-9709-b69e66382a5a tx_id=ed578e15-f17c-4166-9172-75816b4dff66 type=appsec` The question is, where is that configuration?
GiteaMirror commented 2026-04-16 07:55:19 -05:00
Author
Owner
Copy Link

@TuncTaylan commented on GitHub (Apr 11, 2025):

I solved this (my issue at least), check these out:

<!-- gh-comment-id:2798142991 --> @TuncTaylan commented on GitHub (Apr 11, 2025): I solved this (my issue at least), check these out: - https://github.com/fosrl/pangolin/issues/436#issuecomment-2797924344 - https://github.com/fosrl/pangolin/pull/515#issue-2989648287
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
api
authentication
bug
config
dependencies
docker
documentation
enhancement
good first issue
help wanted
Improvement
Look Into
needs investigating
networking
new feature
non-critical bug
potential bug
pull-request
Mirrored from GitHub Pull Request
 question
reverse proxy
Security
stale
ui
wontfix
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/pangolin#1282
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?