github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 16:11:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #10331] Client2 Feedback #53297

New Issue
Open
opened 2026-04-29 02:30:58 -05:00 by GiteaMirror · 28 comments
GiteaMirror commented 2026-04-29 02:30:58 -05:00
Owner
Copy Link

Originally created by @bmizerany on GitHub (Apr 18, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/10331

This is a tracking issue for gathering in-the-wild community feedback for the client2 experiment available in Ollama v0.6.6.

Background

Ollama v0.6.6 ships with a new experimental client for powering all model download operations. It is currently experimental and hidden behind a feature flag. Please see below for information on how to enable it.

Improvements

A lot of work went into increasing download speed AND reliability. Client2 should show, in most cases, 20-16,000% download-speed increases, and strong resilience in the face of flakey and/or slow network conditions.

Trying it out

To use client2 for pulls, you'll need to start the Ollama server with the client2 experiment enabled.

Example:

$ OLLAMA_EXPERIMENT=client2 ollama serve

NOTE: The OLLAMA_EXPERIMENT=client2 is only necessary when starting the server. All other operations do not require it.

With the above running, in another terminal sessions, try:

$ ollama pull gemma3

or

$ ollama run gemma3

Controlling Speed and Bandwidth

If reducing bandwidth usage is desired, the environment variable OLLAMA_REGISTRY_MAXSTREAMS may be set. By default it is equal to the number of processes you have available, but can be changed to reduce (or increase) the number of concurrent streams used for downloading. It must be set for ollama serve:

$ OLLAMA_REGISTRY_MAXSTREAMS=1 OLLAMA_EXPERIMENT=client2 ollama serve

Feedback wanted

We're excited to hear from you! Please give Client2 a try and let us know how it went here. All feedback and questions are super valuable to us and help us ensure client2 becomes the default client ASAP!

Originally created by @bmizerany on GitHub (Apr 18, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/10331 This is a tracking issue for gathering in-the-wild community feedback for the client2 experiment available in Ollama v0.6.6. ## Background Ollama v0.6.6 ships with a new experimental client for powering all model download operations. It is currently experimental and hidden behind a feature flag. Please see below for information on how to enable it. ## Improvements A lot of work went into increasing download speed AND reliability. Client2 should show, in most cases, 20-16,000% download-speed increases, and strong resilience in the face of flakey and/or slow network conditions. ## Trying it out To use client2 for pulls, you'll need to start the Ollama server with the client2 experiment enabled. Example: ``` $ OLLAMA_EXPERIMENT=client2 ollama serve ``` > NOTE: The `OLLAMA_EXPERIMENT=client2` is only necessary when starting the server. All other operations do not require it. With the above running, in another terminal sessions, try: ``` $ ollama pull gemma3 ``` or ``` $ ollama run gemma3 ``` # Controlling Speed and Bandwidth If reducing bandwidth usage is desired, the environment variable `OLLAMA_REGISTRY_MAXSTREAMS` may be set. By default it is equal to the number of processes you have available, but can be changed to reduce (or increase) the number of concurrent streams used for downloading. It must be set for `ollama serve`: ``` $ OLLAMA_REGISTRY_MAXSTREAMS=1 OLLAMA_EXPERIMENT=client2 ollama serve ``` # Feedback wanted We're excited to hear from you! Please give Client2 a try and let us know how it went here. All feedback *and* questions are super valuable to us and help us ensure client2 becomes the default client ASAP!
GiteaMirror commented 2026-04-29 02:31:00 -05:00
Author
Owner
Copy Link

@anikwadehra commented on GitHub (Apr 18, 2025):

Not able to start download with client2 in windows 11.

time=2025-04-18T07:09:30.141+05:30 level=INFO source=gpu.go:319 msg="detected OS VRAM overhead" id=GPU-36450fb2-7398-5410-e7a3-24a015407ac9 library=cuda compute=8.6 driver=12.8 name="NVIDIA GeForce RTX 3090" overhead="233.0 MiB" time=2025-04-18T07:09:30.142+05:30 level=INFO source=types.go:130 msg="inference compute" id=GPU-36450fb2-7398-5410-e7a3-24a015407ac9 library=cuda variant=v12 compute=8.6 driver=12.8 name="NVIDIA GeForce RTX 3090" total="24.0 GiB" available="22.8 GiB" [GIN] 2025/04/18 - 07:09:47 | 200 | 0s | 127.0.0.1 | HEAD "/" [GIN] 2025/04/18 - 07:09:47 | 404 | 1.0396ms | 127.0.0.1 | POST "/api/show" time=2025-04-18T07:10:25.907+05:30 level=INFO source=server.go:156 msg=http status=200 method=POST path=/api/pull content-length=60 remote=127.0.0.1:50385 proto=HTTP/1.1 query="" [GIN] 2025/04/18 - 07:13:52 | 200 | 0s | 127.0.0.1 | HEAD "/" time=2025-04-18T07:13:55.348+05:30 level=INFO source=server.go:156 msg=http status=200 method=POST path=/api/pull content-length=60 remote=127.0.0.1:50416 proto=HTTP/1.1 query=""

client output

C:\Windows\System32>ollama run gemma3:27b pulling e796792eba26... 0% ▕ ▏ 0 B/ 17 GB pulling e0a42594d802... 0% ▕ ▏ 0 B/ 358 B pulling dd084c7d92a3... 0% ▕ ▏ 0 B/8.4 KB pulling 3116c5225075... 0% ▕ ▏ 0 B/ 77 B pulling f838f048d368... 0% ▕ ▏ 0 B/ 490 B error: request error https://dd20bb891979d25aebc8bec07b2b3bbc.r2.cloudflarestorage.com/ollama/docker/registry/v2/blobs/sha256/e7/e796792eba26c4d3b04b0ac5adb01a453dd9ec2dfd83b6c59cbf6fe5f30b0f68/data?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=66040c77ac1b787c3af820529859349a%2F20250418%2Fauto%2Fs3%2Faws4_request&X-Amz-Date=20250418T013947Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=e187737570985971b34f538c3181cbc3a4037374bcf2f90272d40f592055b943: Get "https://dd20bb891979d25aebc8bec07b2b3bbc.r2.cloudflarestorage.com/ollama/docker/registry/v2/blobs/sha256/e7/e796792eba26c4d3b04b0ac5adb01a453dd9ec2dfd83b6c59cbf6fe5f30b0f68/data?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=66040c77ac1b787c3af820529859349a%2F20250418%2Fauto%2Fs3%2Faws4_request&X-Amz-Date=20250418T013947Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=e187737570985971b34f538c3181cbc3a4037374bcf2f90272d40f592055b943": dial tcp: lookup dd20bb891979d25aebc8bec07b2b3bbc.r2.cloudflarestorage.com: no such host Error: model 'gemma3:27b' not found

<!-- gh-comment-id:2814333875 --> @anikwadehra commented on GitHub (Apr 18, 2025): Not able to start download with client2 in windows 11. `time=2025-04-18T07:09:30.141+05:30 level=INFO source=gpu.go:319 msg="detected OS VRAM overhead" id=GPU-36450fb2-7398-5410-e7a3-24a015407ac9 library=cuda compute=8.6 driver=12.8 name="NVIDIA GeForce RTX 3090" overhead="233.0 MiB" time=2025-04-18T07:09:30.142+05:30 level=INFO source=types.go:130 msg="inference compute" id=GPU-36450fb2-7398-5410-e7a3-24a015407ac9 library=cuda variant=v12 compute=8.6 driver=12.8 name="NVIDIA GeForce RTX 3090" total="24.0 GiB" available="22.8 GiB" [GIN] 2025/04/18 - 07:09:47 | 200 | 0s | 127.0.0.1 | HEAD "/" [GIN] 2025/04/18 - 07:09:47 | 404 | 1.0396ms | 127.0.0.1 | POST "/api/show" time=2025-04-18T07:10:25.907+05:30 level=INFO source=server.go:156 msg=http status=200 method=POST path=/api/pull content-length=60 remote=127.0.0.1:50385 proto=HTTP/1.1 query="" [GIN] 2025/04/18 - 07:13:52 | 200 | 0s | 127.0.0.1 | HEAD "/" time=2025-04-18T07:13:55.348+05:30 level=INFO source=server.go:156 msg=http status=200 method=POST path=/api/pull content-length=60 remote=127.0.0.1:50416 proto=HTTP/1.1 query=""` client output `C:\Windows\System32>ollama run gemma3:27b pulling e796792eba26... 0% ▕ ▏ 0 B/ 17 GB pulling e0a42594d802... 0% ▕ ▏ 0 B/ 358 B pulling dd084c7d92a3... 0% ▕ ▏ 0 B/8.4 KB pulling 3116c5225075... 0% ▕ ▏ 0 B/ 77 B pulling f838f048d368... 0% ▕ ▏ 0 B/ 490 B error: request error https://dd20bb891979d25aebc8bec07b2b3bbc.r2.cloudflarestorage.com/ollama/docker/registry/v2/blobs/sha256/e7/e796792eba26c4d3b04b0ac5adb01a453dd9ec2dfd83b6c59cbf6fe5f30b0f68/data?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=66040c77ac1b787c3af820529859349a%2F20250418%2Fauto%2Fs3%2Faws4_request&X-Amz-Date=20250418T013947Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=e187737570985971b34f538c3181cbc3a4037374bcf2f90272d40f592055b943: Get "https://dd20bb891979d25aebc8bec07b2b3bbc.r2.cloudflarestorage.com/ollama/docker/registry/v2/blobs/sha256/e7/e796792eba26c4d3b04b0ac5adb01a453dd9ec2dfd83b6c59cbf6fe5f30b0f68/data?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=66040c77ac1b787c3af820529859349a%2F20250418%2Fauto%2Fs3%2Faws4_request&X-Amz-Date=20250418T013947Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=e187737570985971b34f538c3181cbc3a4037374bcf2f90272d40f592055b943": dial tcp: lookup dd20bb891979d25aebc8bec07b2b3bbc.r2.cloudflarestorage.com: no such host Error: model 'gemma3:27b' not found`
GiteaMirror commented 2026-04-29 02:31:03 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Apr 18, 2025):

This is likely not a problem with client2, but rather an ongoing issue with dd20bb891979d25aebc8bec07b2b3bbc.r2.cloudflarestorage.com, see #8605.

<!-- gh-comment-id:2814925568 --> @rick-github commented on GitHub (Apr 18, 2025): This is likely not a problem with client2, but rather an ongoing issue with dd20bb891979d25aebc8bec07b2b3bbc.r2.cloudflarestorage.com, see #8605.
GiteaMirror commented 2026-04-29 02:31:08 -05:00
Author
Owner
Copy Link

@kkm5848 commented on GitHub (Apr 18, 2025):

I am unable to run inference in qwen2.5 on GPU with cuda toolkit 12.8.

<!-- gh-comment-id:2815856397 --> @kkm5848 commented on GitHub (Apr 18, 2025): I am unable to run inference in qwen2.5 on GPU with cuda toolkit 12.8.
GiteaMirror commented 2026-04-29 02:31:10 -05:00
Author
Owner
Copy Link

@mchiang0610 commented on GitHub (Apr 18, 2025):

@kkm5848 sorry about that. This is for downloading the model when using ollama run?

<!-- gh-comment-id:2815929873 --> @mchiang0610 commented on GitHub (Apr 18, 2025): @kkm5848 sorry about that. This is for downloading the model when using `ollama run`?
GiteaMirror commented 2026-04-29 02:31:11 -05:00
Author
Owner
Copy Link

@kkm5848 commented on GitHub (Apr 18, 2025):

Downloading the model works (using ollama pull). When serving the model, it inferences on CPU (and not on GPU) which makes it impossible to use it for anything that has a reasonable model size.

Best Regards,

Krishna

From: Michael @.>
Sent: Friday, April 18, 2025 10:55 AM
To: ollama/ollama @.>
Cc: kkm5848 @.>; Mention @.>
Subject: Re: [ollama/ollama] Client2 Feedback (Issue #10331)

@kkm5848 https://github.com/kkm5848 sorry about that. This is for downloading the model when using ollama run?

—
Reply to this email directly, view it on GitHub https://github.com/ollama/ollama/issues/10331#issuecomment-2815929873 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AGW5DDI3E4WTVBL3CLO4MVD22E353AVCNFSM6AAAAAB3MCTQLKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQMJVHEZDSOBXGM .
You are receiving this because you were mentioned.Message ID: @.***>

mchiang0610 left a comment (ollama/ollama#10331) https://github.com/ollama/ollama/issues/10331#issuecomment-2815929873

@kkm5848 https://github.com/kkm5848 sorry about that. This is for downloading the model when using ollama run?

—
Reply to this email directly, view it on GitHub https://github.com/ollama/ollama/issues/10331#issuecomment-2815929873 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AGW5DDI3E4WTVBL3CLO4MVD22E353AVCNFSM6AAAAAB3MCTQLKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQMJVHEZDSOBXGM .
You are receiving this because you were mentioned.Message ID: @.***>

<!-- gh-comment-id:2815966154 --> @kkm5848 commented on GitHub (Apr 18, 2025): Downloading the model works (using ollama pull). When serving the model, it inferences on CPU (and not on GPU) which makes it impossible to use it for anything that has a reasonable model size. Best Regards, Krishna From: Michael ***@***.***> Sent: Friday, April 18, 2025 10:55 AM To: ollama/ollama ***@***.***> Cc: kkm5848 ***@***.***>; Mention ***@***.***> Subject: Re: [ollama/ollama] Client2 Feedback (Issue #10331) @kkm5848 <https://github.com/kkm5848> sorry about that. This is for downloading the model when using ollama run? — Reply to this email directly, view it on GitHub <https://github.com/ollama/ollama/issues/10331#issuecomment-2815929873> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGW5DDI3E4WTVBL3CLO4MVD22E353AVCNFSM6AAAAAB3MCTQLKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQMJVHEZDSOBXGM> . You are receiving this because you were mentioned.Message ID: ***@***.***> mchiang0610 left a comment (ollama/ollama#10331) <https://github.com/ollama/ollama/issues/10331#issuecomment-2815929873> @kkm5848 <https://github.com/kkm5848> sorry about that. This is for downloading the model when using ollama run? — Reply to this email directly, view it on GitHub <https://github.com/ollama/ollama/issues/10331#issuecomment-2815929873> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGW5DDI3E4WTVBL3CLO4MVD22E353AVCNFSM6AAAAAB3MCTQLKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQMJVHEZDSOBXGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>
GiteaMirror commented 2026-04-29 02:31:16 -05:00
Author
Owner
Copy Link

@kkm5848 commented on GitHub (Apr 18, 2025):

I upgraded Ubuntu to 24.04 from 22.04 and now it seems to be working out of the box.

Downloading the model works (using ollama pull). When serving the model, it inferences on CPU (and not on GPU) which makes it impossible to use it for anything that has a reasonable model size.

Best Regards,

Krishna

From: Michael @.>
Sent: Friday, April 18, 2025 10:55 AM
To: ollama/ollama @.>
Cc: kkm5848 @.>; Mention @.>
Subject: Re: [ollama/ollama] Client2 Feedback (Issue #10331)

@kkm5848 https://github.com/kkm5848 sorry about that. This is for downloading the model when using ollama run?

—
Reply to this email directly, view it on GitHub <#10331 (comment)> , or unsubscribe https://github.com/notifications/unsubscribe-auth/AGW5DDI3E4WTVBL3CLO4MVD22E353AVCNFSM6AAAAAB3MCTQLKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQMJVHEZDSOBXGM .
You are receiving this because you were mentioned.Message ID: @.***>

mchiang0610 left a comment (ollama/ollama#10331) <#10331 (comment)>

@kkm5848 https://github.com/kkm5848 sorry about that. This is for downloading the model when using ollama run?

—
Reply to this email directly, view it on GitHub <#10331 (comment)> , or unsubscribe https://github.com/notifications/unsubscribe-auth/AGW5DDI3E4WTVBL3CLO4MVD22E353AVCNFSM6AAAAAB3MCTQLKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQMJVHEZDSOBXGM .
You are receiving this because you were mentioned.Message ID: @.***>

<!-- gh-comment-id:2816223613 --> @kkm5848 commented on GitHub (Apr 18, 2025): I upgraded Ubuntu to 24.04 from 22.04 and now it seems to be working out of the box. > Downloading the model works (using ollama pull). When serving the model, it inferences on CPU (and not on GPU) which makes it impossible to use it for anything that has a reasonable model size. > > > > Best Regards, > > > > Krishna > > > > From: Michael ***@***.***> > Sent: Friday, April 18, 2025 10:55 AM > To: ollama/ollama ***@***.***> > Cc: kkm5848 ***@***.***>; Mention ***@***.***> > Subject: Re: [ollama/ollama] Client2 Feedback (Issue [#10331](https://github.com/ollama/ollama/issues/10331)) > > > > [@kkm5848](https://github.com/kkm5848) <https://github.com/kkm5848> sorry about that. This is for downloading the model when using ollama run? > > — > Reply to this email directly, view it on GitHub <[#10331 (comment)](https://github.com/ollama/ollama/issues/10331#issuecomment-2815929873)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGW5DDI3E4WTVBL3CLO4MVD22E353AVCNFSM6AAAAAB3MCTQLKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQMJVHEZDSOBXGM> . > You are receiving this because you were mentioned.Message ID: ***@***.***> > > mchiang0610 left a comment ([ollama/ollama#10331](https://github.com/ollama/ollama/issues/10331)) <[#10331 (comment)](https://github.com/ollama/ollama/issues/10331#issuecomment-2815929873)> > > [@kkm5848](https://github.com/kkm5848) <https://github.com/kkm5848> sorry about that. This is for downloading the model when using ollama run? > > — > Reply to this email directly, view it on GitHub <[#10331 (comment)](https://github.com/ollama/ollama/issues/10331#issuecomment-2815929873)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGW5DDI3E4WTVBL3CLO4MVD22E353AVCNFSM6AAAAAB3MCTQLKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQMJVHEZDSOBXGM> . > You are receiving this because you were mentioned.Message ID: ***@***.***>
GiteaMirror commented 2026-04-29 02:31:20 -05:00
Author
Owner
Copy Link

@Maltz42 commented on GitHub (Apr 18, 2025):

Faster pulls? My problem is that ollama already destroys my internet connection when pulling a model by opening a ridiculous number of simultaneous streams. I'd much, much rather have it play nice on the network than be "faster". It already pulls at full gigabit speed (while generating 15% packet loss for other processes) so I'm not sure how much faster it could go anyway. I'm curious to know more specifics about how client2 works, and whether it can be configured to use a single network-friendly download stream.

<!-- gh-comment-id:2816348568 --> @Maltz42 commented on GitHub (Apr 18, 2025): Faster pulls? My problem is that ollama already destroys my internet connection when pulling a model by opening a ridiculous number of simultaneous streams. I'd much, much rather have it play nice on the network than be "faster". It already pulls at full gigabit speed (while generating 15% packet loss for other processes) so I'm not sure how much faster it could go anyway. I'm curious to know more specifics about how client2 works, and whether it can be configured to use a *single* network-friendly download stream.
GiteaMirror commented 2026-04-29 02:31:22 -05:00
Author
Owner
Copy Link

@bmizerany commented on GitHub (Apr 19, 2025):

@Maltz42 I hear ya, which is why I added:

OLLAMA_REGISTRY_MAXSTREAMS=1 OLLAMA_EXPERIMENT=client2 ollama serve

:)

<!-- gh-comment-id:2816374640 --> @bmizerany commented on GitHub (Apr 19, 2025): @Maltz42 I hear ya, which is why I added: ``` OLLAMA_REGISTRY_MAXSTREAMS=1 OLLAMA_EXPERIMENT=client2 ollama serve ``` :)
GiteaMirror commented 2026-04-29 02:31:26 -05:00
Author
Owner
Copy Link

@bmizerany commented on GitHub (Apr 19, 2025):

@Maltz42 I just added the Controlling Speed and Bandwidth section above.

<!-- gh-comment-id:2816379140 --> @bmizerany commented on GitHub (Apr 19, 2025): @Maltz42 I just added the Controlling Speed and Bandwidth section above.
GiteaMirror commented 2026-04-29 02:31:30 -05:00
Author
Owner
Copy Link

@Maltz42 commented on GitHub (Apr 19, 2025):

I've set OLLAMA_REGISTRY_MAXSTREAMS=3, which gives me about 650Mbps throughput with 2% packet loss. That's tolerable on both counts. I'll take that all day long over line-speed 1Gbps @ 15% packet loss. HUGE improvement! Thanks!

<!-- gh-comment-id:2816738335 --> @Maltz42 commented on GitHub (Apr 19, 2025): I've set OLLAMA_REGISTRY_MAXSTREAMS=3, which gives me about 650Mbps throughput with 2% packet loss. That's tolerable on both counts. I'll take that all day long over line-speed 1Gbps @ 15% packet loss. HUGE improvement! Thanks!
GiteaMirror commented 2026-04-29 02:31:32 -05:00
Author
Owner
Copy Link

@ChromiaCat commented on GitHub (Apr 24, 2025):

Can these be added as environment path variables to not have to retype them on each command prompt session?

<!-- gh-comment-id:2825813182 --> @ChromiaCat commented on GitHub (Apr 24, 2025): Can these be added as environment path variables to not have to retype them on each command prompt session?
GiteaMirror commented 2026-04-29 02:31:34 -05:00
Author
Owner
Copy Link

@EntropyYue commented on GitHub (Apr 24, 2025):

If client2 is enabled, my model will disappear, and it will show that there is no model. If client2 is disabled and ollama is restarted, it will return to normal.
Other information: I used the OLLAMA_MODELS variable to modify the model save path.

<!-- gh-comment-id:2825948748 --> @EntropyYue commented on GitHub (Apr 24, 2025): If client2 is enabled, my model will disappear, and it will show that there is no model. If client2 is disabled and ollama is restarted, it will return to normal. Other information: I used the OLLAMA_MODELS variable to modify the model save path.
GiteaMirror commented 2026-04-29 02:31:36 -05:00
Author
Owner
Copy Link

@mann1x commented on GitHub (Apr 25, 2025):

Not much difference here where I have a 1Gb fiber.
It goes thru a WiFi link and a 1Gb Coax adapter, reducing the bandwidth to 500-600 Mbps.

Normal: 1m17s
Client2 1 stream: 1m16s
Client2 3 streams: 1m14s

<!-- gh-comment-id:2830812689 --> @mann1x commented on GitHub (Apr 25, 2025): Not much difference here where I have a 1Gb fiber. It goes thru a WiFi link and a 1Gb Coax adapter, reducing the bandwidth to 500-600 Mbps. Normal: 1m17s Client2 1 stream: 1m16s Client2 3 streams: 1m14s
GiteaMirror commented 2026-04-29 02:31:37 -05:00
Author
Owner
Copy Link

@Maltz42 commented on GitHub (Apr 25, 2025):

Don't measure internet bandwidth through WiFi - there are WAY too many variables. All you're measuring is your own WiFi speed. At that location. In that room. At that moment. etc...

<!-- gh-comment-id:2830827956 --> @Maltz42 commented on GitHub (Apr 25, 2025): Don't measure internet bandwidth through WiFi - there are WAY too many variables. All you're measuring is your own WiFi speed. At that location. In that room. At that moment. etc...
GiteaMirror commented 2026-04-29 02:31:39 -05:00
Author
Owner
Copy Link

@mann1x commented on GitHub (Apr 25, 2025):

Don't measure internet bandwidth through WiFi - there are WAY too many variables. All you're measuring is your own WiFi speed. At that location. In that room. At that moment. etc...

In general yes but in my case it's a very stable 4x4 WiFi AP to AP link, speed of the Coax adapter is lower than the WiFi uplink.

<!-- gh-comment-id:2831014205 --> @mann1x commented on GitHub (Apr 25, 2025): > Don't measure internet bandwidth through WiFi - there are WAY too many variables. All you're measuring is your own WiFi speed. At that location. In that room. At that moment. etc... In general yes but in my case it's a very stable 4x4 WiFi AP to AP link, speed of the Coax adapter is lower than the WiFi uplink.
GiteaMirror commented 2026-04-29 02:31:41 -05:00
Author
Owner
Copy Link

@FelikZ commented on GitHub (Apr 28, 2025):

When trying to download huggingface model:

export OLLAMA_NUM_PARALLEL=1
export OLLAMA_MAX_LOADED_MODELS=2
export OLLAMA_FLASH_ATTENTION=1
export OLLAMA_EXPERIMENT=client2

launchctl setenv OLLAMA_KV_CACHE_TYPE "$OLLAMA_KV_CACHE_TYPE";
launchctl setenv OLLAMA_NUM_PARALLEL "$OLLAMA_NUM_PARALLEL"
launchctl setenv OLLAMA_MAX_LOADED_MODELS "$OLLAMA_MAX_LOADED_MODELS"
launchctl setenv OLLAMA_FLASH_ATTENTION "$OLLAMA_FLASH_ATTENTION"
launchctl setenv OLLAMA_EXPERIMENT "$OLLAMA_EXPERIMENT"

# reload ollama

$ ollama pull hf.co/ibm-granite/granite-3.3-8b-instruct-GGUF:Q8_0
pulling manifest
Error: internal server error

Works fine without experiment flags.

<!-- gh-comment-id:2834546883 --> @FelikZ commented on GitHub (Apr 28, 2025): When trying to download huggingface model: ``` export OLLAMA_NUM_PARALLEL=1 export OLLAMA_MAX_LOADED_MODELS=2 export OLLAMA_FLASH_ATTENTION=1 export OLLAMA_EXPERIMENT=client2 launchctl setenv OLLAMA_KV_CACHE_TYPE "$OLLAMA_KV_CACHE_TYPE"; launchctl setenv OLLAMA_NUM_PARALLEL "$OLLAMA_NUM_PARALLEL" launchctl setenv OLLAMA_MAX_LOADED_MODELS "$OLLAMA_MAX_LOADED_MODELS" launchctl setenv OLLAMA_FLASH_ATTENTION "$OLLAMA_FLASH_ATTENTION" launchctl setenv OLLAMA_EXPERIMENT "$OLLAMA_EXPERIMENT" # reload ollama ``` ``` $ ollama pull hf.co/ibm-granite/granite-3.3-8b-instruct-GGUF:Q8_0 pulling manifest Error: internal server error ``` Works fine without experiment flags.
GiteaMirror commented 2026-04-29 02:31:44 -05:00
Author
Owner
Copy Link

@mann1x commented on GitHub (Apr 28, 2025):

Made some more testing with a different machine, Win10+powershell+ptime

Normal: 60.6s
Client2 1 stream: 64.7s
Client2 3 streams: 64.1s
Client2 6 streams: 51.7s
Client2 9 streams: 52.8s

Seems at this speed, 500-600 Mbps, the ideal amount of streams is around 6.

There's something off; I've been testing randomly throughout the day with client2 and 1/3/6 streams.
With 1/3 streams, slightly more often with 1 stream, the speed gets "capped" at about 25/30 MB/s.
Instead of the usual ~60s it takes 130-150s to pull gemma3.
Never had this issue with 6 streams.

<!-- gh-comment-id:2834809302 --> @mann1x commented on GitHub (Apr 28, 2025): Made some more testing with a different machine, Win10+powershell+ptime Normal: 60.6s Client2 1 stream: 64.7s Client2 3 streams: 64.1s Client2 6 streams: 51.7s Client2 9 streams: 52.8s Seems at this speed, 500-600 Mbps, the ideal amount of streams is around 6. There's something off; I've been testing randomly throughout the day with client2 and 1/3/6 streams. With 1/3 streams, slightly more often with 1 stream, the speed gets "capped" at about 25/30 MB/s. Instead of the usual ~60s it takes 130-150s to pull gemma3. Never had this issue with 6 streams.
GiteaMirror commented 2026-04-29 02:31:47 -05:00
Author
Owner
Copy Link

@Maltz42 commented on GitHub (May 2, 2025):

So, I'm downloading llama4 maverick, and I'm seeing tens of thousands of 186-byte files filling up my Ollama folder as it builds the main blob. Please don't tell me Ollama downloads files in 186-byte chunks...

<!-- gh-comment-id:2847977251 --> @Maltz42 commented on GitHub (May 2, 2025): So, I'm downloading llama4 maverick, and I'm seeing tens of thousands of 186-byte files filling up my Ollama folder as it builds the main blob. Please don't tell me Ollama downloads files in 186-byte chunks...
GiteaMirror commented 2026-04-29 02:31:49 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (May 2, 2025):

They are markers for chunks to be downloaded - they hold the state of the chunk download.

<!-- gh-comment-id:2848069426 --> @rick-github commented on GitHub (May 2, 2025): They are markers for chunks to be downloaded - they hold the state of the chunk download.
GiteaMirror commented 2026-04-29 02:31:50 -05:00
Author
Owner
Copy Link

@Maltz42 commented on GitHub (May 2, 2025):

... and then it didn't delete them when the transfer was over. I turned off "client2" and restarted Ollama, and all the temp files vanished. Without "client2", the transfer adds a few dozen 70 such tiny partial-download files (these were even smaller at around 70 bytes) as it downloads, but it deletes them as it goes. This is in Ollama 0.6.7

What kind of crazy download algorithm does Ollama use? I'd really like to get more in-the-weeds details on this.

<!-- gh-comment-id:2848072020 --> @Maltz42 commented on GitHub (May 2, 2025): ... and then it didn't delete them when the transfer was over. I turned off "client2" and restarted Ollama, and all the temp files vanished. Without "client2", the transfer adds a few dozen 70 such tiny partial-download files (these were even smaller at around 70 bytes) as it downloads, but it deletes them as it goes. This is in Ollama 0.6.7 What kind of crazy download algorithm does Ollama use? I'd really like to get more in-the-weeds details on this.
GiteaMirror commented 2026-04-29 02:31:51 -05:00
Author
Owner
Copy Link

@Maltz42 commented on GitHub (May 2, 2025):

They are markers for chunks to be downloaded - they hold the state of the chunk download.

Ah, that explains it. Still, why is this done on-disk instead of in RAM? That's a lot of writing and rather SSD-unfriendly.

In any case, client2 is very broken in this regard. It shouldn't leave tens of thousands of them laying around, especially after the transfer is complete.

<!-- gh-comment-id:2848080423 --> @Maltz42 commented on GitHub (May 2, 2025): > They are markers for chunks to be downloaded - they hold the state of the chunk download. Ah, that explains it. Still, why is this done on-disk instead of in RAM? That's a lot of writing and rather SSD-unfriendly. In any case, client2 is very broken in this regard. It shouldn't leave tens of thousands of them laying around, especially after the transfer is complete.
GiteaMirror commented 2026-04-29 02:31:51 -05:00
Author
Owner
Copy Link

@fangzm0805 commented on GitHub (May 2, 2025):

I set the OLLAMA_EXPERIMENT in the Windows environment variable to show "Error: internal server error" after pulling the model for client2.

<!-- gh-comment-id:2848291979 --> @fangzm0805 commented on GitHub (May 2, 2025): I set the OLLAMA_EXPERIMENT in the Windows environment variable to show "Error: internal server error" after pulling the model for client2.
GiteaMirror commented 2026-04-29 02:31:53 -05:00
Author
Owner
Copy Link

@ioquatix commented on GitHub (Jul 26, 2025):

I can confirm that running ollama with OLLAMA_EXPERIMENT=client2 fixes any performance issues I was experiencing before. Not only am I not experiencing the same degradation, it felt a bit faster, I was seeing 12x+MB/s where as before I never really saw it get above 11x MB/s

<!-- gh-comment-id:3121385649 --> @ioquatix commented on GitHub (Jul 26, 2025): I can confirm that running ollama with `OLLAMA_EXPERIMENT=client2` fixes any performance issues I was experiencing before. Not only am I not experiencing the same degradation, it felt a bit faster, I was seeing 12x+MB/s where as before I never really saw it get above 11x MB/s
GiteaMirror commented 2026-04-29 02:31:55 -05:00
Author
Owner
Copy Link

@lnicola commented on GitHub (Oct 23, 2025):

Both with and without OLLAMA_EXPERIMENT=client2 my downloads start at a good speed (25 MB/s), but then they slow down to practically 0 and never finish. I had qwen3:14b get stuck right at 100% and gpt-oss:20b is now at 3.5 MB/s with 9.2/13 GB done.

<!-- gh-comment-id:3438456257 --> @lnicola commented on GitHub (Oct 23, 2025): Both with and without `OLLAMA_EXPERIMENT=client2` my downloads start at a good speed (25 MB/s), but then they slow down to practically 0 and never finish. I had `qwen3:14b` get stuck right at 100% and `gpt-oss:20b` is now at 3.5 MB/s with 9.2/13 GB done.
GiteaMirror commented 2026-04-29 02:31:58 -05:00
Author
Owner
Copy Link

@lnicola commented on GitHub (Oct 23, 2025):

Yeah, so it gets up to the end, then it goes back to 12 / 13 GB as if it's retrying that last part. The logs say msg="e7b273f96360 part 9 stalled; retrying. If this persists, press ctrl-c to exit, then 'ollama pull' to find a faster connection."

I stopped it and restarted, but it still says the same thing (again on part 9).

EDIT: it finally finished after restarting it again and another dozen retries.

EDIT 2: looks like I didn't actually test client2, I messed up the systemd unit syntax.

<!-- gh-comment-id:3438780804 --> @lnicola commented on GitHub (Oct 23, 2025): Yeah, so it gets up to the end, then it goes back to 12 / 13 GB as if it's retrying that last part. The logs say msg="e7b273f96360 part 9 stalled; retrying. If this persists, press ctrl-c to exit, then 'ollama pull' to find a faster connection." I stopped it and restarted, but it still says the same thing (again on part 9). EDIT: it finally finished after restarting it again and another dozen retries. EDIT 2: looks like I didn't actually test `client2`, I messed up the systemd unit syntax.
GiteaMirror commented 2026-04-29 02:32:00 -05:00
Author
Owner
Copy Link

@mknwebsolutions commented on GitHub (Oct 24, 2025):

Confirming same issue here exactly, the client2 setting randomly helped

<!-- gh-comment-id:3440117880 --> @mknwebsolutions commented on GitHub (Oct 24, 2025): Confirming same issue here exactly, the client2 setting randomly helped
GiteaMirror commented 2026-04-29 02:32:06 -05:00
Author
Owner
Copy Link

@Chaoses-Ib commented on GitHub (Nov 14, 2025):

I set the OLLAMA_EXPERIMENT in the Windows environment variable to show "Error: internal server error" after pulling the model for client2.

I have the same problem. ollama run qwen2.5-coder:32b works but ollama run modelscope.cn/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF doesn't. Looks like the modelscope.cn part is broken or not supported yet?

time=2025-11-14T11:08:19.931+08:00 level=ERROR source=server.go:304 msg=pulling model=modelscope.cn/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF error.status=404 error.code="" error.message="404 page not found"
<!-- gh-comment-id:3530624339 --> @Chaoses-Ib commented on GitHub (Nov 14, 2025): > I set the OLLAMA_EXPERIMENT in the Windows environment variable to show "Error: internal server error" after pulling the model for client2. I have the same problem. `ollama run qwen2.5-coder:32b` works but `ollama run modelscope.cn/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF` doesn't. Looks like the `modelscope.cn` part is broken or not supported yet? ```log time=2025-11-14T11:08:19.931+08:00 level=ERROR source=server.go:304 msg=pulling model=modelscope.cn/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF error.status=404 error.code="" error.message="404 page not found" ```
GiteaMirror commented 2026-04-29 02:32:08 -05:00
Author
Owner
Copy Link

@mverrilli commented on GitHub (Feb 16, 2026):

Hi, just wanted to post my experience here in case it helps. client2 did get me pretty close, everything downloaded fine except for one file it seems. This was for qwen3-coder:30b. One file repeatedly came up 0 bytes and it was giving me an Error: internal server error.

-rw-r--r-- 1 ollama ollama           0 Feb 16 20:11 sha256-24a94682582c6045f4950846fc7711479dcecb478b86759f0306a2ef8484d318

Checking the daemon log:

Feb 16 20:08:46 myhost ollama[132435]: time=2026-02-16T20:08:46.526Z level=ERROR source=server.go:304 msg=pulling model=qwen3-coder:30b error="file content changed underfoot"
Feb 16 20:08:46 myhost ollama[132435]: time=2026-02-16T20:08:46.526Z level=INFO source=server.go:164 msg=http error="file content changed underfoot" status=200 method=POST path=/api/pull content-length=65 remote=127.0.0.1:41964 proto=HTTP/1.1 query=""

I had prune disabled, and so tried continuing it and got the same error repeatedly (even after deleting the 0 byte file, it kept replacing it with one). I turned off client2, deleted the zero byte file and tried again, this time it continued and downloaded the full file:

-rw-r--r-- 1 ollama ollama 542 Feb 16 20:14 sha256-24a94682582c6045f4950846fc7711479dcecb478b86759f0306a2ef8484d318

and installed the model.

ollama version is 0.16.1
Ubuntu 24.04

Best of luck!

<!-- gh-comment-id:3910379282 --> @mverrilli commented on GitHub (Feb 16, 2026): Hi, just wanted to post my experience here in case it helps. `client2` did get me pretty close, everything downloaded fine except for one file it seems. This was for `qwen3-coder:30b`. One file repeatedly came up 0 bytes and it was giving me an `Error: internal server error`. ``` -rw-r--r-- 1 ollama ollama 0 Feb 16 20:11 sha256-24a94682582c6045f4950846fc7711479dcecb478b86759f0306a2ef8484d318 ``` Checking the daemon log: ``` Feb 16 20:08:46 myhost ollama[132435]: time=2026-02-16T20:08:46.526Z level=ERROR source=server.go:304 msg=pulling model=qwen3-coder:30b error="file content changed underfoot" Feb 16 20:08:46 myhost ollama[132435]: time=2026-02-16T20:08:46.526Z level=INFO source=server.go:164 msg=http error="file content changed underfoot" status=200 method=POST path=/api/pull content-length=65 remote=127.0.0.1:41964 proto=HTTP/1.1 query="" ``` I had prune disabled, and so tried continuing it and got the same error repeatedly (even after deleting the 0 byte file, it kept replacing it with one). I turned off `client2`, deleted the zero byte file and tried again, this time it continued and downloaded the full file: ``` -rw-r--r-- 1 ollama ollama 542 Feb 16 20:14 sha256-24a94682582c6045f4950846fc7711479dcecb478b86759f0306a2ef8484d318 ``` and installed the model. ollama version is 0.16.1 Ubuntu 24.04 Best of luck!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#53297
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?