github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-15 20:54:54 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #16030] MLX - nvfp4 models on MacOs extremely slow #87893

New Issue
Open
opened 2026-05-10 06:33:12 -05:00 by GiteaMirror · 8 comments
GiteaMirror commented 2026-05-10 06:33:12 -05:00
Owner
Copy Link

Originally created by @alemata2006 on GitHub (May 6, 2026).
Original GitHub issue: https://github.com/ollama/ollama/issues/16030

What is the issue?

When running models (like qwen3.6:27b-nvfp4, qwen3.5:27b-coding-nvfp4, qwen3.6:35b-a3b-nvfp4) until version 0.20.0 they worked fine. A simple prompt took around 2 minutes to produce a result (ollama run MODEL "give me a definition for strategy and for tactics. Provide at least 3 examples to differentiate the concepts."

Ollama: 0.20.1: All models took around 1 minute 50 seconds to 2 minutes 25 seconds to answer (results between 120 and 250 lines with content).
Ollama 0.23.1: These models takes 45 to write a single line.

Rest of the models with the same test work with the basically same time and result as before (no notable changes), only MLX models seems affected.

Relevant log output

No visible errors, seems to work ok, but extremely slow.

time=2026-05-06T19:47:02.529-03:00 level=INFO source=client.go:359 msg="starting mlx runner subprocess" model=qwen3.5:27b-coding-nvfp4 port=58467
time=2026-05-06T19:47:02.532-03:00 level=INFO source=sched.go:561 msg="loaded runners" count=1
time=2026-05-06T19:47:02.587-03:00 level=INFO source=server.go:44 msg="MLX engine initialized" "MLX version"=0.31.2 device=gpu
time=2026-05-06T19:47:02.667-03:00 level=INFO source=base.go:110 msg="Model architecture" arch=Qwen3_5ForConditionalGeneration
time=2026-05-06T19:47:02.966-03:00 level=INFO source=runner.go:159 msg="Loaded tensors from manifest" count=1584
time=2026-05-06T19:47:09.859-03:00 level=INFO source=runner.go:194 msg="Starting HTTP server" host=127.0.0.1 port=58467
time=2026-05-06T19:47:09.980-03:00 level=INFO source=server.go:213 msg=ServeHTTP method=GET path=/v1/status took=10.853875ms status="200 OK"

OS

macOS

GPU

Apple

CPU

Apple

Ollama version

0.23.1

Originally created by @alemata2006 on GitHub (May 6, 2026). Original GitHub issue: https://github.com/ollama/ollama/issues/16030 ### What is the issue? When running models (like qwen3.6:27b-nvfp4, qwen3.5:27b-coding-nvfp4, qwen3.6:35b-a3b-nvfp4) until version 0.20.0 they worked fine. A simple prompt took around 2 minutes to produce a result (ollama run MODEL "give me a definition for strategy and for tactics. Provide at least 3 examples to differentiate the concepts." Ollama: 0.20.1: All models took around 1 minute 50 seconds to 2 minutes 25 seconds to answer (results between 120 and 250 lines with content). Ollama 0.23.1: These models takes 45 to write a single line. Rest of the models with the same test work with the basically same time and result as before (no notable changes), only MLX models seems affected. ### Relevant log output ```shell No visible errors, seems to work ok, but extremely slow. time=2026-05-06T19:47:02.529-03:00 level=INFO source=client.go:359 msg="starting mlx runner subprocess" model=qwen3.5:27b-coding-nvfp4 port=58467 time=2026-05-06T19:47:02.532-03:00 level=INFO source=sched.go:561 msg="loaded runners" count=1 time=2026-05-06T19:47:02.587-03:00 level=INFO source=server.go:44 msg="MLX engine initialized" "MLX version"=0.31.2 device=gpu time=2026-05-06T19:47:02.667-03:00 level=INFO source=base.go:110 msg="Model architecture" arch=Qwen3_5ForConditionalGeneration time=2026-05-06T19:47:02.966-03:00 level=INFO source=runner.go:159 msg="Loaded tensors from manifest" count=1584 time=2026-05-06T19:47:09.859-03:00 level=INFO source=runner.go:194 msg="Starting HTTP server" host=127.0.0.1 port=58467 time=2026-05-06T19:47:09.980-03:00 level=INFO source=server.go:213 msg=ServeHTTP method=GET path=/v1/status took=10.853875ms status="200 OK" ``` ### OS macOS ### GPU Apple ### CPU Apple ### Ollama version 0.23.1
GiteaMirror added the bugneeds more info labels 2026-05-10 06:33:13 -05:00
GiteaMirror commented 2026-05-10 06:33:19 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (May 8, 2026):

@alemata2006 Can you say what kind of Mac you're using? I think this probably happened as a result of bumping the MLX library to the most recent version.

<!-- gh-comment-id:4410357653 --> @pdevine commented on GitHub (May 8, 2026): @alemata2006 Can you say what kind of Mac you're using? I think this probably happened as a result of bumping the MLX library to the most recent version.
GiteaMirror commented 2026-05-10 06:33:21 -05:00
Author
Owner
Copy Link

@alemata2006 commented on GitHub (May 9, 2026):

@pdevine I'm using an Mac Pro Max M1 with 32gb of RAM as home lab for testing new versions before using in a M5 Pro Max (you can understand why a simple prompt could take more than 1 minute in a M1). Let me know if I can help you with more info. I don't want to try on the M5 Pro Max because I'm working with that machine (that one still have 0.19.0).

<!-- gh-comment-id:4411291161 --> @alemata2006 commented on GitHub (May 9, 2026): @pdevine I'm using an Mac Pro Max M1 with 32gb of RAM as home lab for testing new versions before using in a M5 Pro Max (you can understand why a simple prompt could take more than 1 minute in a M1). Let me know if I can help you with more info. I don't want to try on the M5 Pro Max because I'm working with that machine (that one still have 0.19.0).
GiteaMirror commented 2026-05-10 06:33:22 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (May 9, 2026):

@alemata2006 are you using MacOS Sequoia or Sonoma? I think we have a packaging bug which is only working correctly on Tahoe.

<!-- gh-comment-id:4413485310 --> @pdevine commented on GitHub (May 9, 2026): @alemata2006 are you using MacOS Sequoia or Sonoma? I think we have a packaging bug which is only working correctly on Tahoe.
GiteaMirror commented 2026-05-10 06:33:23 -05:00
Author
Owner
Copy Link

@alemata2006 commented on GitHub (May 9, 2026):

Hi @pdevine ! I'm using Tahoe (26.4.1)

The problem seems to be with the MLX integration somehow. I tried older versions, and found that there's an issue with the use of the library, but I didn't notice it before.
If you run 2 prompts one after the other (in older versions), it fails. I didn't notice because I mix models for tasks. In other words, if you use a nvfp4 model twice (one after another while the model is still on memory) it fails.

Test with 0.20.3

I downgraded and tested it with 0.20.3 twice, with the following prompt. First time it was really fast, second time really slow and after a couple of lines of thinking, gave an error:

ollama run qwen3.6:35b-a3b-nvfp4 --verbose "create an software architecture document to use as template, with all the necessary content (should include triggers, decisions, etc). Make a complete document, with all the elements a SAD should have, wiht an explanation in each point on how to use that section"

This it the first time

[GIN] 2026/05/09 - 15:56:45 | 200 |  422.095417ms |       127.0.0.1 | POST     "/api/show"
time=2026-05-09T15:56:45.845-03:00 level=INFO source=sched.go:484 msg="system memory" total="32.0 GiB" free="21.7 GiB" free_swap="0 B"
time=2026-05-09T15:56:45.845-03:00 level=INFO source=sched.go:491 msg="gpu memory" id=0 library=Metal available="24.5 GiB" free="25.0 GiB" minimum="512.0 MiB" overhead="0 B"
time=2026-05-09T15:56:45.845-03:00 level=INFO source=client.go:432 msg="starting mlx runner subprocess" model=qwen3.6:35b-a3b-nvfp4 port=62854
time=2026-05-09T15:56:45.847-03:00 level=INFO source=sched.go:561 msg="loaded runners" count=1
time=2026-05-09T15:56:45.895-03:00 level=INFO source=server.go:32 msg="MLX engine initialized" "MLX version"=0.31.1-23-g38ad257 device=gpu
time=2026-05-09T15:56:45.960-03:00 level=INFO source=base.go:67 msg="Model architecture" arch=Qwen3_5MoeForConditionalGeneration
time=2026-05-09T15:56:46.296-03:00 level=INFO source=runner.go:135 msg="Loaded tensors from manifest" count=1436
time=2026-05-09T15:56:52.659-03:00 level=INFO source=runner.go:169 msg="Starting HTTP server" host=127.0.0.1 port=62854
time=2026-05-09T15:56:52.763-03:00 level=INFO source=server.go:183 msg=ServeHTTP method=GET path=/v1/status took=3.341375ms status="200 OK"
time=2026-05-09T15:56:52.767-03:00 level=INFO source=client.go:147 msg="mlx runner is ready" port=62854
time=2026-05-09T15:56:52.788-03:00 level=INFO source=cache.go:126 msg="cache miss" total=61 matched=0 cached=0 left=61
time=2026-05-09T15:57:03.842-03:00 level=INFO source=pipeline.go:134 msg="Prompt processing progress" processed=57 total=61
time=2026-05-09T15:57:08.540-03:00 level=INFO source=pipeline.go:134 msg="Prompt processing progress" processed=60 total=61
time=2026-05-09T15:59:16.807-03:00 level=INFO source=pipeline.go:55 msg="peak memory" size="19.77 GiB"
time=2026-05-09T15:59:16.807-03:00 level=INFO source=server.go:183 msg=ServeHTTP method=POST path=/v1/completions took=2m24.033328917s status="200 OK"
time=2026-05-09T15:59:16.816-03:00 level=INFO source=server.go:183 msg=ServeHTTP method=POST path=/v1/tokenize took=7.379792ms status="200 OK"
[GIN] 2026/05/09 - 15:59:16 | 200 |         2m31s |       127.0.0.1 | POST     "/api/generate"

The result of the verbose:

Verbose:
total duration: 2m31.029591375s
load duration: 6.991995708s
prompt eval count: 61 token(s)
prompt eval duration: 16.582592209s
prompt eval rate: 3.68 tokens/s
eval count: 5907 token(s)
eval duration: 2m7.435055792s
eval rate: 46.35 tokens/s

This is the second time (again, extremely slow):

[GIN] 2026/05/09 - 15:59:16 | 200 |         2m31s |       127.0.0.1 | POST     "/api/generate"
[GIN] 2026/05/09 - 16:01:56 | 200 |     684.333µs |       127.0.0.1 | HEAD     "/"
[GIN] 2026/05/09 - 16:01:57 | 200 |  600.327583ms |       127.0.0.1 | POST     "/api/show"
time=2026-05-09T16:01:57.348-03:00 level=INFO source=server.go:183 msg=ServeHTTP method=GET path=/v1/status took=307.166µs status="200 OK"
time=2026-05-09T16:01:57.354-03:00 level=INFO source=cache.go:126 msg="cache hit" total=61 matched=61 cached=57 left=4
time=2026-05-09T16:02:08.724-03:00 level=INFO source=pipeline.go:134 msg="Prompt processing progress" processed=3 total=4
time=2026-05-09T16:11:57.302-03:00 level=INFO source=server.go:183 msg=ServeHTTP method=POST path=/v1/completions took=10m0.002687667s status="200 OK"
[GIN] 2026/05/09 - 16:11:57 | 200 |         10m0s |       127.0.0.1 | POST     "/api/generate"
time=2026-05-09T16:12:00.964-03:00 level=INFO source=pipeline.go:55 msg="peak memory" size="20.04 GiB"
time=2026-05-09T16:12:00.964-03:00 level=INFO source=runner.go:149 msg="Request terminated" error="context canceled"

And there was no verbose, but this error:

Error: mlx runner failed: time=2026-05-09T16:02:08.724-03:00 level=INFO source=pipeline.go:134 msg="Prompt processing progress" processed=3 total=4

Test with 0.19.0

I uninstall 0.20.3 and installed 0.19.0. I did the same test with the same model+prompt and verbose with 0.19.0 twice as with 0.20.3. First try was successful and really fast. Second was slow and

Log when I started ollama:

time=2026-05-09T16:21:11.625-03:00 level=INFO source=routes.go:1742 msg="server config" env="map[HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:0 OLLAMA_DEBUG:INFO OLLAMA_DEBUG_LOG_REQUESTS:false OLLAMA_EDITOR: OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/Users/MYSELF/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NO_CLOUD:true OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[ollama.com] OLLAMA_SCHED_SPREAD:false http_proxy: https_proxy: no_proxy:]"
time=2026-05-09T16:21:11.625-03:00 level=INFO source=routes.go:1744 msg="Ollama cloud disabled: true"
time=2026-05-09T16:21:11.814-03:00 level=INFO source=images.go:477 msg="total blobs: 9150"
time=2026-05-09T16:21:11.841-03:00 level=INFO source=images.go:484 msg="total unused blobs removed: 0"
time=2026-05-09T16:21:11.845-03:00 level=INFO source=routes.go:1800 msg="Listening on 127.0.0.1:11434 (version 0.19.0)"
time=2026-05-09T16:21:11.846-03:00 level=INFO source=runner.go:67 msg="discovering available GPUs..."
time=2026-05-09T16:21:11.846-03:00 level=INFO source=server.go:432 msg="starting runner" cmd="/Applications/Ollama.app/Contents/Resources/ollama runner --ollama-engine --port 63882"
time=2026-05-09T16:21:11.929-03:00 level=INFO source=types.go:42 msg="inference compute" id=0 filter_id=0 library=Metal compute=0.0 name=Metal description="Apple M1 Max" libdirs="" driver=0.0 pci_id="" type=discrete total="25.0 GiB" available="25.0 GiB"
time=2026-05-09T16:21:11.929-03:00 level=INFO source=routes.go:1850 msg="vram-based default context" total_vram="25.0 GiB" default_num_ctx=32768

** First Try **
Log during the prompt:

[GIN] 2026/05/09 - 16:24:09 | 200 |      29.041µs |       127.0.0.1 | HEAD     "/"
[GIN] 2026/05/09 - 16:24:09 | 200 |  468.322833ms |       127.0.0.1 | POST     "/api/show"
time=2026-05-09T16:24:09.900-03:00 level=INFO source=sched.go:484 msg="system memory" total="32.0 GiB" free="20.9 GiB" free_swap="0 B"
time=2026-05-09T16:24:09.900-03:00 level=INFO source=sched.go:491 msg="gpu memory" id=0 library=Metal available="24.5 GiB" free="25.0 GiB" minimum="512.0 MiB" overhead="0 B"
time=2026-05-09T16:24:09.901-03:00 level=INFO source=client.go:432 msg="starting mlx runner subprocess" model=qwen3.6:35b-a3b-nvfp4 port=63933
time=2026-05-09T16:24:09.903-03:00 level=INFO source=sched.go:561 msg="loaded runners" count=1
time=2026-05-09T16:24:09.945-03:00 level=INFO source=server.go:32 msg="MLX engine initialized" "MLX version"=0.31.1-23-g38ad257 device=gpu
time=2026-05-09T16:24:09.993-03:00 level=INFO source=base.go:67 msg="Model architecture" arch=Qwen3_5MoeForConditionalGeneration
time=2026-05-09T16:24:10.257-03:00 level=INFO source=runner.go:135 msg="Loaded tensors from manifest" count=1436
time=2026-05-09T16:24:16.663-03:00 level=INFO source=runner.go:169 msg="Starting HTTP server" host=127.0.0.1 port=63933
time=2026-05-09T16:24:16.716-03:00 level=INFO source=server.go:183 msg=ServeHTTP method=GET path=/v1/status took=3.929375ms status="200 OK"
time=2026-05-09T16:24:16.720-03:00 level=INFO source=client.go:147 msg="mlx runner is ready" port=63933
time=2026-05-09T16:24:16.752-03:00 level=INFO source=cache.go:126 msg="cache miss" total=62 matched=0 cached=0 left=62
time=2026-05-09T16:24:29.101-03:00 level=INFO source=pipeline.go:134 msg="Prompt processing progress" processed=58 total=62
time=2026-05-09T16:24:34.687-03:00 level=INFO source=pipeline.go:134 msg="Prompt processing progress" processed=61 total=62
time=2026-05-09T16:26:09.718-03:00 level=INFO source=pipeline.go:55 msg="peak memory" size="19.74 GiB"
time=2026-05-09T16:26:09.719-03:00 level=INFO source=server.go:183 msg=ServeHTTP method=POST path=/v1/completions took=1m52.992475875s status="200 OK"
time=2026-05-09T16:26:09.721-03:00 level=INFO source=server.go:183 msg=ServeHTTP method=POST path=/v1/tokenize took=3.043375ms status="200 OK"
[GIN] 2026/05/09 - 16:26:09 | 200 |         1m59s |       127.0.0.1 | POST     "/api/generate"

And the verbose:

total duration: 1m59.878254625s
load duration: 6.881631792s
prompt eval count: 62 token(s)
prompt eval duration: 19.073144375s
prompt eval rate: 3.25 tokens/s
eval count: 4792 token(s)
eval duration: 1m33.893201416s
eval rate: 51.04 tokens/s

** Second try **
I used the same prompt+model and verbose on the same terminal, after the other finished.

The log

time=2026-05-09T16:31:09.727-03:00 level=INFO source=client.go:189 msg="stopping mlx runner subprocess" pid=52781
[GIN] 2026/05/09 - 16:31:28 | 200 |    1.048125ms |       127.0.0.1 | HEAD     "/"
[GIN] 2026/05/09 - 16:31:28 | 200 |  458.950291ms |       127.0.0.1 | POST     "/api/show"
time=2026-05-09T16:31:28.585-03:00 level=INFO source=sched.go:484 msg="system memory" total="32.0 GiB" free="22.7 GiB" free_swap="0 B"
time=2026-05-09T16:31:28.586-03:00 level=INFO source=sched.go:491 msg="gpu memory" id=0 library=Metal available="24.5 GiB" free="25.0 GiB" minimum="512.0 MiB" overhead="0 B"
time=2026-05-09T16:31:28.587-03:00 level=INFO source=client.go:432 msg="starting mlx runner subprocess" model=qwen3.6:35b-a3b-nvfp4 port=64176
time=2026-05-09T16:31:28.590-03:00 level=INFO source=sched.go:561 msg="loaded runners" count=1
time=2026-05-09T16:31:28.631-03:00 level=INFO source=server.go:32 msg="MLX engine initialized" "MLX version"=0.31.1-23-g38ad257 device=gpu
time=2026-05-09T16:31:28.681-03:00 level=INFO source=base.go:67 msg="Model architecture" arch=Qwen3_5MoeForConditionalGeneration
time=2026-05-09T16:31:28.954-03:00 level=INFO source=runner.go:135 msg="Loaded tensors from manifest" count=1436
time=2026-05-09T16:31:34.749-03:00 level=INFO source=runner.go:169 msg="Starting HTTP server" host=127.0.0.1 port=64176
time=2026-05-09T16:31:34.806-03:00 level=INFO source=server.go:183 msg=ServeHTTP method=GET path=/v1/status took=4.523834ms status="200 OK"
time=2026-05-09T16:31:34.810-03:00 level=INFO source=client.go:147 msg="mlx runner is ready" port=64176
time=2026-05-09T16:31:34.826-03:00 level=INFO source=cache.go:126 msg="cache miss" total=62 matched=0 cached=0 left=62
time=2026-05-09T16:31:46.961-03:00 level=INFO source=pipeline.go:134 msg="Prompt processing progress" processed=58 total=62
time=2026-05-09T16:31:53.768-03:00 level=INFO source=pipeline.go:134 msg="Prompt processing progress" processed=61 total=62
time=2026-05-09T16:41:34.781-03:00 level=INFO source=server.go:183 msg=ServeHTTP method=POST path=/v1/completions took=10m0.000853833s status="200 OK"
[GIN] 2026/05/09 - 16:41:34 | 200 |         10m6s |       127.0.0.1 | POST     "/api/generate"
time=2026-05-09T16:41:38.768-03:00 level=INFO source=pipeline.go:55 msg="peak memory" size="19.68 GiB"
time=2026-05-09T16:41:38.769-03:00 level=INFO source=runner.go:149 msg="Request terminated" error="context canceled"

Same error in the terminal:

Error: mlx runner failed: time=2026-05-09T16:31:53.768-03:00 level=INFO source=pipeline.go:134 msg="Prompt processing progress" processed=61 total=62

3rd scenario / attemp 0.19.0

I run the same prompt with another non nvfp4 model and it worked.
The run again using the prompt with the original nvfp4 model:

[GIN] 2026/05/09 - 16:46:37 | 200 |  454.443417ms |       127.0.0.1 | POST     "/api/show"
time=2026-05-09T16:46:37.572-03:00 level=INFO source=sched.go:627 msg="updated VRAM based on existing loaded models" gpu=0 library=Metal total="25.0 GiB" available="17.3 GiB"
time=2026-05-09T16:46:37.574-03:00 level=INFO source=sched.go:484 msg="system memory" total="32.0 GiB" free="18.8 GiB" free_swap="0 B"
time=2026-05-09T16:46:37.574-03:00 level=INFO source=sched.go:491 msg="gpu memory" id=0 library=Metal available="16.8 GiB" free="17.3 GiB" minimum="512.0 MiB" overhead="0 B"
time=2026-05-09T16:46:37.616-03:00 level=INFO source=sched.go:484 msg="system memory" total="32.0 GiB" free="20.4 GiB" free_swap="0 B"
time=2026-05-09T16:46:37.616-03:00 level=INFO source=sched.go:491 msg="gpu memory" id=0 library=Metal available="24.5 GiB" free="25.0 GiB" minimum="512.0 MiB" overhead="0 B"
time=2026-05-09T16:46:37.620-03:00 level=INFO source=client.go:432 msg="starting mlx runner subprocess" model=qwen3.6:35b-a3b-nvfp4 port=64833
time=2026-05-09T16:46:37.623-03:00 level=INFO source=sched.go:561 msg="loaded runners" count=1
time=2026-05-09T16:46:37.668-03:00 level=INFO source=server.go:32 msg="MLX engine initialized" "MLX version"=0.31.1-23-g38ad257 device=gpu
time=2026-05-09T16:46:37.725-03:00 level=INFO source=base.go:67 msg="Model architecture" arch=Qwen3_5MoeForConditionalGeneration
time=2026-05-09T16:46:37.985-03:00 level=INFO source=runner.go:135 msg="Loaded tensors from manifest" count=1436
time=2026-05-09T16:46:43.882-03:00 level=INFO source=runner.go:169 msg="Starting HTTP server" host=127.0.0.1 port=64833
time=2026-05-09T16:46:43.940-03:00 level=INFO source=server.go:183 msg=ServeHTTP method=GET path=/v1/status took=3.692791ms status="200 OK"
time=2026-05-09T16:46:43.950-03:00 level=INFO source=client.go:147 msg="mlx runner is ready" port=64833
time=2026-05-09T16:46:43.976-03:00 level=INFO source=cache.go:126 msg="cache miss" total=62 matched=0 cached=0 left=62
time=2026-05-09T16:46:53.894-03:00 level=INFO source=pipeline.go:134 msg="Prompt processing progress" processed=58 total=62
time=2026-05-09T16:46:58.414-03:00 level=INFO source=pipeline.go:134 msg="Prompt processing progress" processed=61 total=62
time=2026-05-09T16:48:23.118-03:00 level=INFO source=pipeline.go:55 msg="peak memory" size="19.73 GiB"
time=2026-05-09T16:48:23.119-03:00 level=INFO source=server.go:183 msg=ServeHTTP method=POST path=/v1/completions took=1m39.152013917s status="200 OK"
time=2026-05-09T16:48:23.122-03:00 level=INFO source=server.go:183 msg=ServeHTTP method=POST path=/v1/tokenize took=2.897666ms status="200 OK"
[GIN] 2026/05/09 - 16:48:23 | 200 |         1m45s |       127.0.0.1 | POST     "/api/generate"

It worked, also fast:

total duration: 1m45.603657125s
load duration: 6.441641209s
prompt eval count: 62 token(s)
prompt eval duration: 15.097046083s
prompt eval rate: 4.11 tokens/s
eval count: 4300 token(s)
eval duration: 1m24.046400167s
eval rate: 51.16 tokens/s

another try:

Did the same another time, and same results:

total duration: 2m17.779411334s
load duration: 6.040227417s
prompt eval count: 62 token(s)
prompt eval duration: 14.429437333s
prompt eval rate: 4.30 tokens/s
eval count: 5948 token(s)
eval duration: 1m57.294200625s
eval rate: 50.71 tokens/s

Last test

I waited until the model was release from the memory (4 minutes) and after a "ollama ps" returned no model on memory, I tried again and it worked (still with 0.19.0).

total duration: 2m6.912727542s
load duration: 6.296107417s
prompt eval count: 62 token(s)
prompt eval duration: 15.165239709s
prompt eval rate: 4.09 tokens/s
eval count: 5370 token(s)
eval duration: 1m45.397576208s
eval rate: 50.95 tokens/s

<!-- gh-comment-id:4413559403 --> @alemata2006 commented on GitHub (May 9, 2026): Hi @pdevine ! I'm using Tahoe (26.4.1) The problem seems to be with the MLX integration somehow. I tried older versions, and found that there's an issue with the use of the library, but I didn't notice it before. If you run 2 prompts one after the other (in older versions), it fails. I didn't notice because I mix models for tasks. In other words, if you use a nvfp4 model twice (one after another while the model is still on memory) it fails. # Test with 0.20.3 I downgraded and tested it with 0.20.3 twice, with the following prompt. First time it was really fast, second time really slow and after a couple of lines of thinking, gave an error: ollama run qwen3.6:35b-a3b-nvfp4 --verbose "create an software architecture document to use as template, with all the necessary content (should include triggers, decisions, etc). Make a complete document, with all the elements a SAD should have, wiht an explanation in each point on how to use that section" ## This it the first time ``` [GIN] 2026/05/09 - 15:56:45 | 200 | 422.095417ms | 127.0.0.1 | POST "/api/show" time=2026-05-09T15:56:45.845-03:00 level=INFO source=sched.go:484 msg="system memory" total="32.0 GiB" free="21.7 GiB" free_swap="0 B" time=2026-05-09T15:56:45.845-03:00 level=INFO source=sched.go:491 msg="gpu memory" id=0 library=Metal available="24.5 GiB" free="25.0 GiB" minimum="512.0 MiB" overhead="0 B" time=2026-05-09T15:56:45.845-03:00 level=INFO source=client.go:432 msg="starting mlx runner subprocess" model=qwen3.6:35b-a3b-nvfp4 port=62854 time=2026-05-09T15:56:45.847-03:00 level=INFO source=sched.go:561 msg="loaded runners" count=1 time=2026-05-09T15:56:45.895-03:00 level=INFO source=server.go:32 msg="MLX engine initialized" "MLX version"=0.31.1-23-g38ad257 device=gpu time=2026-05-09T15:56:45.960-03:00 level=INFO source=base.go:67 msg="Model architecture" arch=Qwen3_5MoeForConditionalGeneration time=2026-05-09T15:56:46.296-03:00 level=INFO source=runner.go:135 msg="Loaded tensors from manifest" count=1436 time=2026-05-09T15:56:52.659-03:00 level=INFO source=runner.go:169 msg="Starting HTTP server" host=127.0.0.1 port=62854 time=2026-05-09T15:56:52.763-03:00 level=INFO source=server.go:183 msg=ServeHTTP method=GET path=/v1/status took=3.341375ms status="200 OK" time=2026-05-09T15:56:52.767-03:00 level=INFO source=client.go:147 msg="mlx runner is ready" port=62854 time=2026-05-09T15:56:52.788-03:00 level=INFO source=cache.go:126 msg="cache miss" total=61 matched=0 cached=0 left=61 time=2026-05-09T15:57:03.842-03:00 level=INFO source=pipeline.go:134 msg="Prompt processing progress" processed=57 total=61 time=2026-05-09T15:57:08.540-03:00 level=INFO source=pipeline.go:134 msg="Prompt processing progress" processed=60 total=61 time=2026-05-09T15:59:16.807-03:00 level=INFO source=pipeline.go:55 msg="peak memory" size="19.77 GiB" time=2026-05-09T15:59:16.807-03:00 level=INFO source=server.go:183 msg=ServeHTTP method=POST path=/v1/completions took=2m24.033328917s status="200 OK" time=2026-05-09T15:59:16.816-03:00 level=INFO source=server.go:183 msg=ServeHTTP method=POST path=/v1/tokenize took=7.379792ms status="200 OK" [GIN] 2026/05/09 - 15:59:16 | 200 | 2m31s | 127.0.0.1 | POST "/api/generate" ``` The result of the verbose: Verbose: total duration: 2m31.029591375s load duration: 6.991995708s prompt eval count: 61 token(s) prompt eval duration: 16.582592209s prompt eval rate: 3.68 tokens/s eval count: 5907 token(s) eval duration: 2m7.435055792s eval rate: 46.35 tokens/s ## This is the second time (again, extremely slow): ``` [GIN] 2026/05/09 - 15:59:16 | 200 | 2m31s | 127.0.0.1 | POST "/api/generate" [GIN] 2026/05/09 - 16:01:56 | 200 | 684.333µs | 127.0.0.1 | HEAD "/" [GIN] 2026/05/09 - 16:01:57 | 200 | 600.327583ms | 127.0.0.1 | POST "/api/show" time=2026-05-09T16:01:57.348-03:00 level=INFO source=server.go:183 msg=ServeHTTP method=GET path=/v1/status took=307.166µs status="200 OK" time=2026-05-09T16:01:57.354-03:00 level=INFO source=cache.go:126 msg="cache hit" total=61 matched=61 cached=57 left=4 time=2026-05-09T16:02:08.724-03:00 level=INFO source=pipeline.go:134 msg="Prompt processing progress" processed=3 total=4 time=2026-05-09T16:11:57.302-03:00 level=INFO source=server.go:183 msg=ServeHTTP method=POST path=/v1/completions took=10m0.002687667s status="200 OK" [GIN] 2026/05/09 - 16:11:57 | 200 | 10m0s | 127.0.0.1 | POST "/api/generate" time=2026-05-09T16:12:00.964-03:00 level=INFO source=pipeline.go:55 msg="peak memory" size="20.04 GiB" time=2026-05-09T16:12:00.964-03:00 level=INFO source=runner.go:149 msg="Request terminated" error="context canceled" ``` And there was no verbose, but this error: Error: mlx runner failed: time=2026-05-09T16:02:08.724-03:00 level=INFO source=pipeline.go:134 msg="Prompt processing progress" processed=3 total=4 # Test with 0.19.0 I uninstall 0.20.3 and installed 0.19.0. I did the same test with the same model+prompt and verbose with 0.19.0 twice as with 0.20.3. First try was successful and really fast. Second was slow and Log when I started ollama: ``` time=2026-05-09T16:21:11.625-03:00 level=INFO source=routes.go:1742 msg="server config" env="map[HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:0 OLLAMA_DEBUG:INFO OLLAMA_DEBUG_LOG_REQUESTS:false OLLAMA_EDITOR: OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/Users/MYSELF/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NO_CLOUD:true OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[ollama.com] OLLAMA_SCHED_SPREAD:false http_proxy: https_proxy: no_proxy:]" time=2026-05-09T16:21:11.625-03:00 level=INFO source=routes.go:1744 msg="Ollama cloud disabled: true" time=2026-05-09T16:21:11.814-03:00 level=INFO source=images.go:477 msg="total blobs: 9150" time=2026-05-09T16:21:11.841-03:00 level=INFO source=images.go:484 msg="total unused blobs removed: 0" time=2026-05-09T16:21:11.845-03:00 level=INFO source=routes.go:1800 msg="Listening on 127.0.0.1:11434 (version 0.19.0)" time=2026-05-09T16:21:11.846-03:00 level=INFO source=runner.go:67 msg="discovering available GPUs..." time=2026-05-09T16:21:11.846-03:00 level=INFO source=server.go:432 msg="starting runner" cmd="/Applications/Ollama.app/Contents/Resources/ollama runner --ollama-engine --port 63882" time=2026-05-09T16:21:11.929-03:00 level=INFO source=types.go:42 msg="inference compute" id=0 filter_id=0 library=Metal compute=0.0 name=Metal description="Apple M1 Max" libdirs="" driver=0.0 pci_id="" type=discrete total="25.0 GiB" available="25.0 GiB" time=2026-05-09T16:21:11.929-03:00 level=INFO source=routes.go:1850 msg="vram-based default context" total_vram="25.0 GiB" default_num_ctx=32768 ``` ** First Try ** Log during the prompt: ``` [GIN] 2026/05/09 - 16:24:09 | 200 | 29.041µs | 127.0.0.1 | HEAD "/" [GIN] 2026/05/09 - 16:24:09 | 200 | 468.322833ms | 127.0.0.1 | POST "/api/show" time=2026-05-09T16:24:09.900-03:00 level=INFO source=sched.go:484 msg="system memory" total="32.0 GiB" free="20.9 GiB" free_swap="0 B" time=2026-05-09T16:24:09.900-03:00 level=INFO source=sched.go:491 msg="gpu memory" id=0 library=Metal available="24.5 GiB" free="25.0 GiB" minimum="512.0 MiB" overhead="0 B" time=2026-05-09T16:24:09.901-03:00 level=INFO source=client.go:432 msg="starting mlx runner subprocess" model=qwen3.6:35b-a3b-nvfp4 port=63933 time=2026-05-09T16:24:09.903-03:00 level=INFO source=sched.go:561 msg="loaded runners" count=1 time=2026-05-09T16:24:09.945-03:00 level=INFO source=server.go:32 msg="MLX engine initialized" "MLX version"=0.31.1-23-g38ad257 device=gpu time=2026-05-09T16:24:09.993-03:00 level=INFO source=base.go:67 msg="Model architecture" arch=Qwen3_5MoeForConditionalGeneration time=2026-05-09T16:24:10.257-03:00 level=INFO source=runner.go:135 msg="Loaded tensors from manifest" count=1436 time=2026-05-09T16:24:16.663-03:00 level=INFO source=runner.go:169 msg="Starting HTTP server" host=127.0.0.1 port=63933 time=2026-05-09T16:24:16.716-03:00 level=INFO source=server.go:183 msg=ServeHTTP method=GET path=/v1/status took=3.929375ms status="200 OK" time=2026-05-09T16:24:16.720-03:00 level=INFO source=client.go:147 msg="mlx runner is ready" port=63933 time=2026-05-09T16:24:16.752-03:00 level=INFO source=cache.go:126 msg="cache miss" total=62 matched=0 cached=0 left=62 time=2026-05-09T16:24:29.101-03:00 level=INFO source=pipeline.go:134 msg="Prompt processing progress" processed=58 total=62 time=2026-05-09T16:24:34.687-03:00 level=INFO source=pipeline.go:134 msg="Prompt processing progress" processed=61 total=62 time=2026-05-09T16:26:09.718-03:00 level=INFO source=pipeline.go:55 msg="peak memory" size="19.74 GiB" time=2026-05-09T16:26:09.719-03:00 level=INFO source=server.go:183 msg=ServeHTTP method=POST path=/v1/completions took=1m52.992475875s status="200 OK" time=2026-05-09T16:26:09.721-03:00 level=INFO source=server.go:183 msg=ServeHTTP method=POST path=/v1/tokenize took=3.043375ms status="200 OK" [GIN] 2026/05/09 - 16:26:09 | 200 | 1m59s | 127.0.0.1 | POST "/api/generate" ``` And the verbose: total duration: 1m59.878254625s load duration: 6.881631792s prompt eval count: 62 token(s) prompt eval duration: 19.073144375s prompt eval rate: 3.25 tokens/s eval count: 4792 token(s) eval duration: 1m33.893201416s eval rate: 51.04 tokens/s ** Second try ** I used the same prompt+model and verbose on the same terminal, after the other finished. The log ``` time=2026-05-09T16:31:09.727-03:00 level=INFO source=client.go:189 msg="stopping mlx runner subprocess" pid=52781 [GIN] 2026/05/09 - 16:31:28 | 200 | 1.048125ms | 127.0.0.1 | HEAD "/" [GIN] 2026/05/09 - 16:31:28 | 200 | 458.950291ms | 127.0.0.1 | POST "/api/show" time=2026-05-09T16:31:28.585-03:00 level=INFO source=sched.go:484 msg="system memory" total="32.0 GiB" free="22.7 GiB" free_swap="0 B" time=2026-05-09T16:31:28.586-03:00 level=INFO source=sched.go:491 msg="gpu memory" id=0 library=Metal available="24.5 GiB" free="25.0 GiB" minimum="512.0 MiB" overhead="0 B" time=2026-05-09T16:31:28.587-03:00 level=INFO source=client.go:432 msg="starting mlx runner subprocess" model=qwen3.6:35b-a3b-nvfp4 port=64176 time=2026-05-09T16:31:28.590-03:00 level=INFO source=sched.go:561 msg="loaded runners" count=1 time=2026-05-09T16:31:28.631-03:00 level=INFO source=server.go:32 msg="MLX engine initialized" "MLX version"=0.31.1-23-g38ad257 device=gpu time=2026-05-09T16:31:28.681-03:00 level=INFO source=base.go:67 msg="Model architecture" arch=Qwen3_5MoeForConditionalGeneration time=2026-05-09T16:31:28.954-03:00 level=INFO source=runner.go:135 msg="Loaded tensors from manifest" count=1436 time=2026-05-09T16:31:34.749-03:00 level=INFO source=runner.go:169 msg="Starting HTTP server" host=127.0.0.1 port=64176 time=2026-05-09T16:31:34.806-03:00 level=INFO source=server.go:183 msg=ServeHTTP method=GET path=/v1/status took=4.523834ms status="200 OK" time=2026-05-09T16:31:34.810-03:00 level=INFO source=client.go:147 msg="mlx runner is ready" port=64176 time=2026-05-09T16:31:34.826-03:00 level=INFO source=cache.go:126 msg="cache miss" total=62 matched=0 cached=0 left=62 time=2026-05-09T16:31:46.961-03:00 level=INFO source=pipeline.go:134 msg="Prompt processing progress" processed=58 total=62 time=2026-05-09T16:31:53.768-03:00 level=INFO source=pipeline.go:134 msg="Prompt processing progress" processed=61 total=62 time=2026-05-09T16:41:34.781-03:00 level=INFO source=server.go:183 msg=ServeHTTP method=POST path=/v1/completions took=10m0.000853833s status="200 OK" [GIN] 2026/05/09 - 16:41:34 | 200 | 10m6s | 127.0.0.1 | POST "/api/generate" time=2026-05-09T16:41:38.768-03:00 level=INFO source=pipeline.go:55 msg="peak memory" size="19.68 GiB" time=2026-05-09T16:41:38.769-03:00 level=INFO source=runner.go:149 msg="Request terminated" error="context canceled" ``` Same error in the terminal: Error: mlx runner failed: time=2026-05-09T16:31:53.768-03:00 level=INFO source=pipeline.go:134 msg="Prompt processing progress" processed=61 total=62 # 3rd scenario / attemp 0.19.0 I run the same prompt with another non nvfp4 model and it worked. The run again using the prompt with the original nvfp4 model: ``` [GIN] 2026/05/09 - 16:46:37 | 200 | 454.443417ms | 127.0.0.1 | POST "/api/show" time=2026-05-09T16:46:37.572-03:00 level=INFO source=sched.go:627 msg="updated VRAM based on existing loaded models" gpu=0 library=Metal total="25.0 GiB" available="17.3 GiB" time=2026-05-09T16:46:37.574-03:00 level=INFO source=sched.go:484 msg="system memory" total="32.0 GiB" free="18.8 GiB" free_swap="0 B" time=2026-05-09T16:46:37.574-03:00 level=INFO source=sched.go:491 msg="gpu memory" id=0 library=Metal available="16.8 GiB" free="17.3 GiB" minimum="512.0 MiB" overhead="0 B" time=2026-05-09T16:46:37.616-03:00 level=INFO source=sched.go:484 msg="system memory" total="32.0 GiB" free="20.4 GiB" free_swap="0 B" time=2026-05-09T16:46:37.616-03:00 level=INFO source=sched.go:491 msg="gpu memory" id=0 library=Metal available="24.5 GiB" free="25.0 GiB" minimum="512.0 MiB" overhead="0 B" time=2026-05-09T16:46:37.620-03:00 level=INFO source=client.go:432 msg="starting mlx runner subprocess" model=qwen3.6:35b-a3b-nvfp4 port=64833 time=2026-05-09T16:46:37.623-03:00 level=INFO source=sched.go:561 msg="loaded runners" count=1 time=2026-05-09T16:46:37.668-03:00 level=INFO source=server.go:32 msg="MLX engine initialized" "MLX version"=0.31.1-23-g38ad257 device=gpu time=2026-05-09T16:46:37.725-03:00 level=INFO source=base.go:67 msg="Model architecture" arch=Qwen3_5MoeForConditionalGeneration time=2026-05-09T16:46:37.985-03:00 level=INFO source=runner.go:135 msg="Loaded tensors from manifest" count=1436 time=2026-05-09T16:46:43.882-03:00 level=INFO source=runner.go:169 msg="Starting HTTP server" host=127.0.0.1 port=64833 time=2026-05-09T16:46:43.940-03:00 level=INFO source=server.go:183 msg=ServeHTTP method=GET path=/v1/status took=3.692791ms status="200 OK" time=2026-05-09T16:46:43.950-03:00 level=INFO source=client.go:147 msg="mlx runner is ready" port=64833 time=2026-05-09T16:46:43.976-03:00 level=INFO source=cache.go:126 msg="cache miss" total=62 matched=0 cached=0 left=62 time=2026-05-09T16:46:53.894-03:00 level=INFO source=pipeline.go:134 msg="Prompt processing progress" processed=58 total=62 time=2026-05-09T16:46:58.414-03:00 level=INFO source=pipeline.go:134 msg="Prompt processing progress" processed=61 total=62 time=2026-05-09T16:48:23.118-03:00 level=INFO source=pipeline.go:55 msg="peak memory" size="19.73 GiB" time=2026-05-09T16:48:23.119-03:00 level=INFO source=server.go:183 msg=ServeHTTP method=POST path=/v1/completions took=1m39.152013917s status="200 OK" time=2026-05-09T16:48:23.122-03:00 level=INFO source=server.go:183 msg=ServeHTTP method=POST path=/v1/tokenize took=2.897666ms status="200 OK" [GIN] 2026/05/09 - 16:48:23 | 200 | 1m45s | 127.0.0.1 | POST "/api/generate" ``` It worked, also fast: total duration: 1m45.603657125s load duration: 6.441641209s prompt eval count: 62 token(s) prompt eval duration: 15.097046083s prompt eval rate: 4.11 tokens/s eval count: 4300 token(s) eval duration: 1m24.046400167s eval rate: 51.16 tokens/s ## another try: Did the same another time, and same results: total duration: 2m17.779411334s load duration: 6.040227417s prompt eval count: 62 token(s) prompt eval duration: 14.429437333s prompt eval rate: 4.30 tokens/s eval count: 5948 token(s) eval duration: 1m57.294200625s eval rate: 50.71 tokens/s # Last test I waited until the model was release from the memory (4 minutes) and after a "ollama ps" returned no model on memory, I tried again and it worked (still with 0.19.0). total duration: 2m6.912727542s load duration: 6.296107417s prompt eval count: 62 token(s) prompt eval duration: 15.165239709s prompt eval rate: 4.09 tokens/s eval count: 5370 token(s) eval duration: 1m45.397576208s eval rate: 50.95 tokens/s
GiteaMirror commented 2026-05-10 06:33:24 -05:00
Author
Owner
Copy Link

@alemata2006 commented on GitHub (May 9, 2026):

@alemata2006 are you using MacOS Sequoia or Sonoma? I think we have a packaging bug which is only working correctly on Tahoe.

@pdevine I'm using 26.4.1 (Tahoe with the latest update). All the issues I still can reproduce them. I'm only using right now Ollama, not using VSC or Rancher Desktop (to be sure nothing is consuming the RAM and creating issues).

<!-- gh-comment-id:4413563415 --> @alemata2006 commented on GitHub (May 9, 2026): > [@alemata2006](https://github.com/alemata2006) are you using MacOS Sequoia or Sonoma? I think we have a packaging bug which is only working correctly on Tahoe. @pdevine I'm using 26.4.1 (Tahoe with the latest update). All the issues I still can reproduce them. I'm only using right now Ollama, not using VSC or Rancher Desktop (to be sure nothing is consuming the RAM and creating issues).
GiteaMirror commented 2026-05-10 06:33:25 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (May 9, 2026):

I haven't been able to repro on an M1 mac with 32G with v26.4. Could you run the server with OLLAMA_DEBUG=1 and send the same prompt twice and share the whole log so we can see what's going on?

<!-- gh-comment-id:4413840244 --> @dhiltgen commented on GitHub (May 9, 2026): I haven't been able to repro on an M1 mac with 32G with v26.4. Could you run the server with OLLAMA_DEBUG=1 and send the same prompt twice and share the whole log so we can see what's going on?
GiteaMirror commented 2026-05-10 06:33:25 -05:00
Author
Owner
Copy Link

@alemata2006 commented on GitHub (May 9, 2026):

@dhiltgen, hi! Yes, I'll try that. In the meantime, I'm sharing the test with the latest version. As I comment below, it still working but after more than 2.5 hours, only printed 483 words (and still is in the "Thinking" stage).
I'll stop it and do the check with the debug and post it later.

Test with 0.23.2

Tested with 0.23.2, and started thinking and showing 1 word every 5 to 10 seconds. After 2 and a half hours, I did a "ollama ps" to check the model, and it failed and reflected that in the console. But the prompt is still ongoing. It has printed 438 words in 2 hours and 30 minutes and keeps going in a very slow manner.

Here's the log:

time=2026-05-09T17:13:09.955-03:00 level=INFO source=routes.go:1802 msg="server config" env="map[HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:0 OLLAMA_DEBUG:INFO OLLAMA_DEBUG_LOG_REQUESTS:false OLLAMA_EDITOR: OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/Users/MYSELF/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NO_CLOUD:true OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[ollama.com] OLLAMA_SCHED_SPREAD:false http_proxy: https_proxy: no_proxy:]"
time=2026-05-09T17:13:09.956-03:00 level=INFO source=routes.go:1804 msg="Ollama cloud disabled: true"
time=2026-05-09T17:13:10.157-03:00 level=INFO source=images.go:517 msg="total blobs: 9150"
time=2026-05-09T17:13:10.187-03:00 level=INFO source=images.go:524 msg="total unused blobs removed: 0"
time=2026-05-09T17:13:10.192-03:00 level=INFO source=routes.go:1864 msg="Listening on 127.0.0.1:11434 (version 0.23.2)"
time=2026-05-09T17:13:10.192-03:00 level=INFO source=model_recommendations.go:177 msg="model recommendations cache sleep scheduled" wait=3h43m29.601646378s consecutive_failures=0
time=2026-05-09T17:13:10.192-03:00 level=INFO source=runner.go:67 msg="discovering available GPUs..."
time=2026-05-09T17:13:10.193-03:00 level=INFO source=server.go:433 msg="starting runner" cmd="/Applications/Ollama.app/Contents/Resources/ollama runner --ollama-engine --port 50065"
time=2026-05-09T17:13:21.616-03:00 level=INFO source=types.go:42 msg="inference compute" id=0 filter_id=0 library=Metal compute=0.0 name=Metal description="Apple M1 Max" libdirs="" driver=0.0 pci_id="" type=discrete total="25.0 GiB" available="25.0 GiB"
time=2026-05-09T17:13:21.616-03:00 level=INFO source=routes.go:1914 msg="vram-based default context" total_vram="25.0 GiB" default_num_ctx=32768
[GIN] 2026/05/09 - 17:13:25 | 200 |     187.417µs |       127.0.0.1 | HEAD     "/"
[GIN] 2026/05/09 - 17:13:25 | 200 |   38.587625ms |       127.0.0.1 | POST     "/api/show"
time=2026-05-09T17:13:25.880-03:00 level=INFO source=sched.go:484 msg="system memory" total="32.0 GiB" free="19.9 GiB" free_swap="0 B"
time=2026-05-09T17:13:25.880-03:00 level=INFO source=sched.go:491 msg="gpu memory" id=0 library=Metal available="24.5 GiB" free="25.0 GiB" minimum="512.0 MiB" overhead="0 B"
time=2026-05-09T17:13:25.881-03:00 level=INFO source=client.go:359 msg="starting mlx runner subprocess" model=qwen3.6:35b-a3b-nvfp4 port=50074
time=2026-05-09T17:13:25.883-03:00 level=INFO source=sched.go:561 msg="loaded runners" count=1
time=2026-05-09T17:13:25.901-03:00 level=INFO source=server.go:44 msg="MLX engine initialized" "MLX version"=0.31.2-7-ge8ebdeb device=gpu
time=2026-05-09T17:13:25.969-03:00 level=INFO source=base.go:110 msg="Model architecture" arch=Qwen3_5MoeForConditionalGeneration
time=2026-05-09T17:13:26.231-03:00 level=INFO source=runner.go:159 msg="Loaded tensors from manifest" count=1436
time=2026-05-09T17:13:32.791-03:00 level=INFO source=runner.go:194 msg="Starting HTTP server" host=127.0.0.1 port=50074
time=2026-05-09T17:13:32.902-03:00 level=INFO source=server.go:213 msg=ServeHTTP method=GET path=/v1/status took=5.3535ms status="200 OK"
time=2026-05-09T17:13:32.909-03:00 level=INFO source=client.go:91 msg="mlx runner is ready" port=50074
time=2026-05-09T17:13:32.928-03:00 level=INFO source=cache.go:126 msg="cache miss" total=61 matched=0 cached=0 left=61
time=2026-05-09T17:13:45.158-03:00 level=INFO source=pipeline.go:135 msg="Prompt processing progress" processed=57 total=61
time=2026-05-09T17:13:50.629-03:00 level=INFO source=pipeline.go:135 msg="Prompt processing progress" processed=60 total=61
[GIN] 2026/05/09 - 19:56:37 | 200 |       1.439ms |       127.0.0.1 | HEAD     "/"
time=2026-05-09T19:56:38.660-03:00 level=ERROR source=server.go:76 msg="Failed to read MLX memory status" error="context canceled"
time=2026-05-09T19:56:38.660-03:00 level=ERROR source=server.go:213 msg=ServeHTTP method=GET path=/v1/status took=1.00096975s status="500 Internal Server Error"
[GIN] 2026/05/09 - 19:56:38 | 200 |  1.002356458s |       127.0.0.1 | GET      "/api/ps"
<!-- gh-comment-id:4413917672 --> @alemata2006 commented on GitHub (May 9, 2026): @dhiltgen, hi! Yes, I'll try that. In the meantime, I'm sharing the test with the latest version. As I comment below, it still working but after more than 2.5 hours, only printed 483 words (and still is in the "Thinking" stage). I'll stop it and do the check with the debug and post it later. # Test with 0.23.2 Tested with 0.23.2, and started thinking and showing 1 word every 5 to 10 seconds. After 2 and a half hours, I did a "ollama ps" to check the model, and it failed and reflected that in the console. But the prompt is still ongoing. It has printed 438 words in 2 hours and 30 minutes and keeps going in a very slow manner. Here's the log: ``` time=2026-05-09T17:13:09.955-03:00 level=INFO source=routes.go:1802 msg="server config" env="map[HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:0 OLLAMA_DEBUG:INFO OLLAMA_DEBUG_LOG_REQUESTS:false OLLAMA_EDITOR: OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/Users/MYSELF/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NO_CLOUD:true OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[ollama.com] OLLAMA_SCHED_SPREAD:false http_proxy: https_proxy: no_proxy:]" time=2026-05-09T17:13:09.956-03:00 level=INFO source=routes.go:1804 msg="Ollama cloud disabled: true" time=2026-05-09T17:13:10.157-03:00 level=INFO source=images.go:517 msg="total blobs: 9150" time=2026-05-09T17:13:10.187-03:00 level=INFO source=images.go:524 msg="total unused blobs removed: 0" time=2026-05-09T17:13:10.192-03:00 level=INFO source=routes.go:1864 msg="Listening on 127.0.0.1:11434 (version 0.23.2)" time=2026-05-09T17:13:10.192-03:00 level=INFO source=model_recommendations.go:177 msg="model recommendations cache sleep scheduled" wait=3h43m29.601646378s consecutive_failures=0 time=2026-05-09T17:13:10.192-03:00 level=INFO source=runner.go:67 msg="discovering available GPUs..." time=2026-05-09T17:13:10.193-03:00 level=INFO source=server.go:433 msg="starting runner" cmd="/Applications/Ollama.app/Contents/Resources/ollama runner --ollama-engine --port 50065" time=2026-05-09T17:13:21.616-03:00 level=INFO source=types.go:42 msg="inference compute" id=0 filter_id=0 library=Metal compute=0.0 name=Metal description="Apple M1 Max" libdirs="" driver=0.0 pci_id="" type=discrete total="25.0 GiB" available="25.0 GiB" time=2026-05-09T17:13:21.616-03:00 level=INFO source=routes.go:1914 msg="vram-based default context" total_vram="25.0 GiB" default_num_ctx=32768 [GIN] 2026/05/09 - 17:13:25 | 200 | 187.417µs | 127.0.0.1 | HEAD "/" [GIN] 2026/05/09 - 17:13:25 | 200 | 38.587625ms | 127.0.0.1 | POST "/api/show" time=2026-05-09T17:13:25.880-03:00 level=INFO source=sched.go:484 msg="system memory" total="32.0 GiB" free="19.9 GiB" free_swap="0 B" time=2026-05-09T17:13:25.880-03:00 level=INFO source=sched.go:491 msg="gpu memory" id=0 library=Metal available="24.5 GiB" free="25.0 GiB" minimum="512.0 MiB" overhead="0 B" time=2026-05-09T17:13:25.881-03:00 level=INFO source=client.go:359 msg="starting mlx runner subprocess" model=qwen3.6:35b-a3b-nvfp4 port=50074 time=2026-05-09T17:13:25.883-03:00 level=INFO source=sched.go:561 msg="loaded runners" count=1 time=2026-05-09T17:13:25.901-03:00 level=INFO source=server.go:44 msg="MLX engine initialized" "MLX version"=0.31.2-7-ge8ebdeb device=gpu time=2026-05-09T17:13:25.969-03:00 level=INFO source=base.go:110 msg="Model architecture" arch=Qwen3_5MoeForConditionalGeneration time=2026-05-09T17:13:26.231-03:00 level=INFO source=runner.go:159 msg="Loaded tensors from manifest" count=1436 time=2026-05-09T17:13:32.791-03:00 level=INFO source=runner.go:194 msg="Starting HTTP server" host=127.0.0.1 port=50074 time=2026-05-09T17:13:32.902-03:00 level=INFO source=server.go:213 msg=ServeHTTP method=GET path=/v1/status took=5.3535ms status="200 OK" time=2026-05-09T17:13:32.909-03:00 level=INFO source=client.go:91 msg="mlx runner is ready" port=50074 time=2026-05-09T17:13:32.928-03:00 level=INFO source=cache.go:126 msg="cache miss" total=61 matched=0 cached=0 left=61 time=2026-05-09T17:13:45.158-03:00 level=INFO source=pipeline.go:135 msg="Prompt processing progress" processed=57 total=61 time=2026-05-09T17:13:50.629-03:00 level=INFO source=pipeline.go:135 msg="Prompt processing progress" processed=60 total=61 [GIN] 2026/05/09 - 19:56:37 | 200 | 1.439ms | 127.0.0.1 | HEAD "/" time=2026-05-09T19:56:38.660-03:00 level=ERROR source=server.go:76 msg="Failed to read MLX memory status" error="context canceled" time=2026-05-09T19:56:38.660-03:00 level=ERROR source=server.go:213 msg=ServeHTTP method=GET path=/v1/status took=1.00096975s status="500 Internal Server Error" [GIN] 2026/05/09 - 19:56:38 | 200 | 1.002356458s | 127.0.0.1 | GET "/api/ps" ```
GiteaMirror commented 2026-05-10 06:33:26 -05:00
Author
Owner
Copy Link

@alemata2006 commented on GitHub (May 10, 2026):

@dhiltgen
Hold a second!!! This is just voodoo!! Still with the 0.23.2. The first time it ran, and the second time again working slow. With 0.23.2 without the "OLLAMA_DEBUG" it didn't even start.

First time

I used the OLLAMA_DEBUG=1 ollama serve to start it (I usually use only ollama serve), and not only it worked, it was FASSSSST the first time!!!

total duration: 2m1.633245708s
load duration: 6.736757958s
prompt eval count: 61 token(s)
prompt eval duration: 17.494932917s
prompt eval rate: 3.49 tokens/s
eval count: 4857 token(s)
eval duration: 1m37.339828541s
eval rate: 49.90 tokens/s

Second time:

Immediately after finish the other (with the model still in memory): I ran again, and started really slow again (1 word every 12 to 20 seconds, and after some words it stop for around 4 minutes and then start again the same cycle).
I has been running for 3 hours 15 minutes now, and stop writting the log right after starting.

Last login: Sat May  9 17:12:48 on ttys007
ollama % OLLAMA_DEBUG=1 ollama serve
time=2026-05-09T20:11:21.873-03:00 level=INFO source=routes.go:1802 msg="server config" env="map[HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:0 OLLAMA_DEBUG:DEBUG OLLAMA_DEBUG_LOG_REQUESTS:false OLLAMA_EDITOR: OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/Users/MYSELF/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NO_CLOUD:true OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[ollama.com] OLLAMA_SCHED_SPREAD:false http_proxy: https_proxy: no_proxy:]"
time=2026-05-09T20:11:21.873-03:00 level=INFO source=routes.go:1804 msg="Ollama cloud disabled: true"
time=2026-05-09T20:11:22.103-03:00 level=INFO source=images.go:517 msg="total blobs: 9150"
time=2026-05-09T20:11:22.135-03:00 level=INFO source=images.go:524 msg="total unused blobs removed: 0"
time=2026-05-09T20:11:22.139-03:00 level=DEBUG source=model_recommendations.go:57 msg="starting model recommendations cache" default_recommendations=6 refresh_interval=4h0m0s fetch_timeout=3s
time=2026-05-09T20:11:22.139-03:00 level=DEBUG source=model_show_cache.go:125 msg="starting model show cache"
time=2026-05-09T20:11:22.139-03:00 level=INFO source=routes.go:1864 msg="Listening on 127.0.0.1:11434 (version 0.23.2)"
time=2026-05-09T20:11:22.139-03:00 level=DEBUG source=sched.go:145 msg="starting llm scheduler"
time=2026-05-09T20:11:22.139-03:00 level=DEBUG source=model_show_cache.go:150 msg="skipping model show cloud cache hydration because cloud is disabled"
time=2026-05-09T20:11:22.139-03:00 level=DEBUG source=model_recommendations.go:262 msg="loaded model recommendations snapshot" path=/Users/MYSELF/.ollama/cache/model-recommendations.json count=7
time=2026-05-09T20:11:22.139-03:00 level=DEBUG source=model_recommendations.go:165 msg="skipping model recommendations refresh because cloud is disabled"
time=2026-05-09T20:11:22.139-03:00 level=INFO source=model_recommendations.go:177 msg="model recommendations cache sleep scheduled" wait=3h14m54.719905434s consecutive_failures=0
time=2026-05-09T20:11:22.139-03:00 level=INFO source=runner.go:67 msg="discovering available GPUs..."
time=2026-05-09T20:11:22.141-03:00 level=INFO source=server.go:433 msg="starting runner" cmd="/Applications/Ollama.app/Contents/Resources/ollama runner --ollama-engine --port 58115"
time=2026-05-09T20:11:22.141-03:00 level=DEBUG source=server.go:434 msg=subprocess PATH=/Users/MYSELF/.rd/bin:/Library/Frameworks/Python.framework/Versions/3.14/bin:/Library/Frameworks/Python.framework/Versions/3.13/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/opt/pkg/env/active/bin:/opt/pmk/env/global/bin:/usr/local/share/dotnet:~/.dotnet/tools OLLAMA_DEBUG=1 DYLD_LIBRARY_PATH=/Applications/Ollama.app/Contents/Resources:/Applications/Ollama.app/Contents/Resources/mlx_metal_v4 OLLAMA_LIBRARY_PATH=/Applications/Ollama.app/Contents/Resources
time=2026-05-09T20:11:22.236-03:00 level=DEBUG source=runner.go:433 msg="bootstrap discovery took" duration=96.015667ms OLLAMA_LIBRARY_PATH=[/Applications/Ollama.app/Contents/Resources] extra_envs=map[]
time=2026-05-09T20:11:22.236-03:00 level=DEBUG source=runner.go:123 msg="evaluating which, if any, devices to filter out" initial_count=1
time=2026-05-09T20:11:22.236-03:00 level=DEBUG source=runner.go:192 msg="adjusting filtering IDs" FilterID=0 new_ID=0
time=2026-05-09T20:11:22.236-03:00 level=DEBUG source=runner.go:40 msg="GPU bootstrap discovery took" duration=96.555167ms
time=2026-05-09T20:11:22.236-03:00 level=INFO source=types.go:42 msg="inference compute" id=0 filter_id=0 library=Metal compute=0.0 name=Metal description="Apple M1 Max" libdirs="" driver=0.0 pci_id="" type=discrete total="25.0 GiB" available="25.0 GiB"
time=2026-05-09T20:11:22.236-03:00 level=INFO source=routes.go:1914 msg="vram-based default context" total_vram="25.0 GiB" default_num_ctx=32768
time=2026-05-09T20:11:22.352-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:22.899-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:22.938-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:23.144-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:23.290-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:23.403-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:23.526-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:23.699-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:24.122-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:24.768-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:24.993-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:25.206-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:25.522-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:25.645-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:26.058-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:26.224-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:26.581-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:26.800-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:27.041-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:27.453-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:28.143-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:28.246-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:28.332-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:28.444-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:28.560-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:28.672-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:28.846-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:29.020-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:29.111-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:29.221-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:29.339-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
time=2026-05-09T20:11:29.469-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32
[GIN] 2026/05/09 - 20:12:16 | 200 |      43.916µs |       127.0.0.1 | HEAD     "/"
[GIN] 2026/05/09 - 20:12:16 | 200 |   40.339125ms |       127.0.0.1 | POST     "/api/show"
time=2026-05-09T20:12:16.517-03:00 level=DEBUG source=runner.go:40 msg="overall device VRAM discovery took" duration=1.166µs
time=2026-05-09T20:12:16.517-03:00 level=DEBUG source=sched.go:220 msg="updating default concurrency" OLLAMA_MAX_LOADED_MODELS=3 gpu_count=1
time=2026-05-09T20:12:16.517-03:00 level=DEBUG source=sched.go:229 msg="loading first model" model=""
time=2026-05-09T20:12:16.519-03:00 level=INFO source=sched.go:484 msg="system memory" total="32.0 GiB" free="21.3 GiB" free_swap="0 B"
time=2026-05-09T20:12:16.519-03:00 level=INFO source=sched.go:491 msg="gpu memory" id=0 library=Metal available="24.5 GiB" free="25.0 GiB" minimum="512.0 MiB" overhead="0 B"
time=2026-05-09T20:12:16.520-03:00 level=INFO source=client.go:359 msg="starting mlx runner subprocess" model=qwen3.6:35b-a3b-nvfp4 port=58131
time=2026-05-09T20:12:16.522-03:00 level=INFO source=sched.go:561 msg="loaded runners" count=1
time=2026-05-09T20:12:16.543-03:00 level=DEBUG source=dynamic.go:28 msg="MLX dynamic library loaded" path=/Applications/Ollama.app/Contents/Resources/mlx_metal_v4/libmlxc.dylib
time=2026-05-09T20:12:16.543-03:00 level=INFO source=server.go:44 msg="MLX engine initialized" "MLX version"=0.31.2-7-ge8ebdeb device=gpu
time=2026-05-09T20:12:16.639-03:00 level=INFO source=base.go:110 msg="Model architecture" arch=Qwen3_5MoeForConditionalGeneration
time=2026-05-09T20:12:16.987-03:00 level=INFO source=runner.go:159 msg="Loaded tensors from manifest" count=1436
time=2026-05-09T20:12:23.063-03:00 level=INFO source=runner.go:194 msg="Starting HTTP server" host=127.0.0.1 port=58131
time=2026-05-09T20:12:23.157-03:00 level=INFO source=server.go:213 msg=ServeHTTP method=GET path=/v1/status took=5.579833ms status="200 OK"
time=2026-05-09T20:12:23.173-03:00 level=INFO source=client.go:91 msg="mlx runner is ready" port=58131
time=2026-05-09T20:12:23.182-03:00 level=DEBUG source=sched.go:573 msg="finished setting up" runner.name=registry.ollama.ai/library/qwen3.6:35b-a3b-nvfp4 runner.size="20.4 GiB" runner.vram="20.4 GiB" runner.parallel=1 runner.pid=58767 runner.model=digest:1b50c6fdc2d4f75f94e6c19b49b20051bc6a7e73620d1ccdfc6687ec042ce754 runner.num_ctx=32768
time=2026-05-09T20:12:23.250-03:00 level=INFO source=cache.go:126 msg="cache miss" total=61 matched=0 cached=0 left=61
time=2026-05-09T20:12:34.068-03:00 level=INFO source=pipeline.go:135 msg="Prompt processing progress" processed=57 total=61
time=2026-05-09T20:12:34.068-03:00 level=DEBUG source=cache.go:401 msg="created snapshot" offset=57
time=2026-05-09T20:12:39.319-03:00 level=INFO source=pipeline.go:135 msg="Prompt processing progress" processed=60 total=61
time=2026-05-09T20:14:18.086-03:00 level=INFO source=pipeline.go:71 msg="peak memory" size="19.74 GiB"
time=2026-05-09T20:14:18.087-03:00 level=INFO source=server.go:213 msg=ServeHTTP method=POST path=/v1/completions took=1m54.868602292s status="200 OK"
time=2026-05-09T20:14:18.088-03:00 level=INFO source=server.go:213 msg=ServeHTTP method=POST path=/v1/tokenize took=2.894458ms status="200 OK"
[GIN] 2026/05/09 - 20:14:18 | 200 |          2m1s |       127.0.0.1 | POST     "/api/generate"
time=2026-05-09T20:14:18.090-03:00 level=DEBUG source=sched.go:581 msg="context for request finished"
time=2026-05-09T20:14:18.091-03:00 level=DEBUG source=sched.go:309 msg="runner with non-zero duration has gone idle, adding timer" runner.name=registry.ollama.ai/library/qwen3.6:35b-a3b-nvfp4 runner.size="20.4 GiB" runner.vram="20.4 GiB" runner.parallel=1 runner.pid=58767 runner.model=digest:1b50c6fdc2d4f75f94e6c19b49b20051bc6a7e73620d1ccdfc6687ec042ce754 runner.num_ctx=32768 duration=5m0s
time=2026-05-09T20:14:18.091-03:00 level=DEBUG source=sched.go:327 msg="after processing request finished event" runner.name=registry.ollama.ai/library/qwen3.6:35b-a3b-nvfp4 runner.size="20.4 GiB" runner.vram="20.4 GiB" runner.parallel=1 runner.pid=58767 runner.model=digest:1b50c6fdc2d4f75f94e6c19b49b20051bc6a7e73620d1ccdfc6687ec042ce754 runner.num_ctx=32768 refCount=0
[GIN] 2026/05/09 - 20:14:27 | 200 |     503.084µs |       127.0.0.1 | HEAD     "/"
[GIN] 2026/05/09 - 20:14:27 | 200 |    66.68675ms |       127.0.0.1 | POST     "/api/show"
time=2026-05-09T20:14:27.727-03:00 level=DEBUG source=sched.go:672 msg="evaluating already loaded" model=digest:1b50c6fdc2d4f75f94e6c19b49b20051bc6a7e73620d1ccdfc6687ec042ce754
time=2026-05-09T20:14:27.728-03:00 level=INFO source=server.go:213 msg=ServeHTTP method=GET path=/v1/status took=103.416µs status="200 OK"
time=2026-05-09T20:14:27.731-03:00 level=DEBUG source=cache.go:250 msg="switching cache path" page_out=1 page_in=0
time=2026-05-09T20:14:27.731-03:00 level=INFO source=cache.go:126 msg="cache hit" total=61 matched=61 cached=57 left=4
time=2026-05-09T20:14:33.373-03:00 level=INFO source=pipeline.go:135 msg="Prompt processing progress" processed=3 total=4
time=2026-05-09T20:14:33.374-03:00 level=DEBUG source=cache.go:401 msg="created snapshot" offset=60
<!-- gh-comment-id:4414258237 --> @alemata2006 commented on GitHub (May 10, 2026): @dhiltgen Hold a second!!! This is just voodoo!! Still with the 0.23.2. The first time it ran, and the second time again working slow. With 0.23.2 without the "OLLAMA_DEBUG" it didn't even start. # First time I used the OLLAMA_DEBUG=1 ollama serve to start it (I usually use only ollama serve), and not only it worked, it was FASSSSST the first time!!! total duration: 2m1.633245708s load duration: 6.736757958s prompt eval count: 61 token(s) prompt eval duration: 17.494932917s prompt eval rate: 3.49 tokens/s eval count: 4857 token(s) eval duration: 1m37.339828541s eval rate: 49.90 tokens/s # Second time: Immediately after finish the other (with the model still in memory): I ran again, and started really slow again (1 word every 12 to 20 seconds, and after some words it stop for around 4 minutes and then start again the same cycle). I has been running for 3 hours 15 minutes now, and stop writting the log right after starting. ``` Last login: Sat May 9 17:12:48 on ttys007 ollama % OLLAMA_DEBUG=1 ollama serve time=2026-05-09T20:11:21.873-03:00 level=INFO source=routes.go:1802 msg="server config" env="map[HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:0 OLLAMA_DEBUG:DEBUG OLLAMA_DEBUG_LOG_REQUESTS:false OLLAMA_EDITOR: OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/Users/MYSELF/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NO_CLOUD:true OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[ollama.com] OLLAMA_SCHED_SPREAD:false http_proxy: https_proxy: no_proxy:]" time=2026-05-09T20:11:21.873-03:00 level=INFO source=routes.go:1804 msg="Ollama cloud disabled: true" time=2026-05-09T20:11:22.103-03:00 level=INFO source=images.go:517 msg="total blobs: 9150" time=2026-05-09T20:11:22.135-03:00 level=INFO source=images.go:524 msg="total unused blobs removed: 0" time=2026-05-09T20:11:22.139-03:00 level=DEBUG source=model_recommendations.go:57 msg="starting model recommendations cache" default_recommendations=6 refresh_interval=4h0m0s fetch_timeout=3s time=2026-05-09T20:11:22.139-03:00 level=DEBUG source=model_show_cache.go:125 msg="starting model show cache" time=2026-05-09T20:11:22.139-03:00 level=INFO source=routes.go:1864 msg="Listening on 127.0.0.1:11434 (version 0.23.2)" time=2026-05-09T20:11:22.139-03:00 level=DEBUG source=sched.go:145 msg="starting llm scheduler" time=2026-05-09T20:11:22.139-03:00 level=DEBUG source=model_show_cache.go:150 msg="skipping model show cloud cache hydration because cloud is disabled" time=2026-05-09T20:11:22.139-03:00 level=DEBUG source=model_recommendations.go:262 msg="loaded model recommendations snapshot" path=/Users/MYSELF/.ollama/cache/model-recommendations.json count=7 time=2026-05-09T20:11:22.139-03:00 level=DEBUG source=model_recommendations.go:165 msg="skipping model recommendations refresh because cloud is disabled" time=2026-05-09T20:11:22.139-03:00 level=INFO source=model_recommendations.go:177 msg="model recommendations cache sleep scheduled" wait=3h14m54.719905434s consecutive_failures=0 time=2026-05-09T20:11:22.139-03:00 level=INFO source=runner.go:67 msg="discovering available GPUs..." time=2026-05-09T20:11:22.141-03:00 level=INFO source=server.go:433 msg="starting runner" cmd="/Applications/Ollama.app/Contents/Resources/ollama runner --ollama-engine --port 58115" time=2026-05-09T20:11:22.141-03:00 level=DEBUG source=server.go:434 msg=subprocess PATH=/Users/MYSELF/.rd/bin:/Library/Frameworks/Python.framework/Versions/3.14/bin:/Library/Frameworks/Python.framework/Versions/3.13/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/opt/pkg/env/active/bin:/opt/pmk/env/global/bin:/usr/local/share/dotnet:~/.dotnet/tools OLLAMA_DEBUG=1 DYLD_LIBRARY_PATH=/Applications/Ollama.app/Contents/Resources:/Applications/Ollama.app/Contents/Resources/mlx_metal_v4 OLLAMA_LIBRARY_PATH=/Applications/Ollama.app/Contents/Resources time=2026-05-09T20:11:22.236-03:00 level=DEBUG source=runner.go:433 msg="bootstrap discovery took" duration=96.015667ms OLLAMA_LIBRARY_PATH=[/Applications/Ollama.app/Contents/Resources] extra_envs=map[] time=2026-05-09T20:11:22.236-03:00 level=DEBUG source=runner.go:123 msg="evaluating which, if any, devices to filter out" initial_count=1 time=2026-05-09T20:11:22.236-03:00 level=DEBUG source=runner.go:192 msg="adjusting filtering IDs" FilterID=0 new_ID=0 time=2026-05-09T20:11:22.236-03:00 level=DEBUG source=runner.go:40 msg="GPU bootstrap discovery took" duration=96.555167ms time=2026-05-09T20:11:22.236-03:00 level=INFO source=types.go:42 msg="inference compute" id=0 filter_id=0 library=Metal compute=0.0 name=Metal description="Apple M1 Max" libdirs="" driver=0.0 pci_id="" type=discrete total="25.0 GiB" available="25.0 GiB" time=2026-05-09T20:11:22.236-03:00 level=INFO source=routes.go:1914 msg="vram-based default context" total_vram="25.0 GiB" default_num_ctx=32768 time=2026-05-09T20:11:22.352-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:22.899-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:22.938-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:23.144-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:23.290-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:23.403-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:23.526-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:23.699-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:24.122-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:24.768-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:24.993-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:25.206-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:25.522-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:25.645-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:26.058-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:26.224-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:26.581-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:26.800-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:27.041-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:27.453-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:28.143-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:28.246-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:28.332-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:28.444-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:28.560-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:28.672-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:28.846-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:29.020-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:29.111-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:29.221-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:29.339-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 time=2026-05-09T20:11:29.469-03:00 level=DEBUG source=ggml.go:326 msg="key with type not found" key=general.alignment default=32 [GIN] 2026/05/09 - 20:12:16 | 200 | 43.916µs | 127.0.0.1 | HEAD "/" [GIN] 2026/05/09 - 20:12:16 | 200 | 40.339125ms | 127.0.0.1 | POST "/api/show" time=2026-05-09T20:12:16.517-03:00 level=DEBUG source=runner.go:40 msg="overall device VRAM discovery took" duration=1.166µs time=2026-05-09T20:12:16.517-03:00 level=DEBUG source=sched.go:220 msg="updating default concurrency" OLLAMA_MAX_LOADED_MODELS=3 gpu_count=1 time=2026-05-09T20:12:16.517-03:00 level=DEBUG source=sched.go:229 msg="loading first model" model="" time=2026-05-09T20:12:16.519-03:00 level=INFO source=sched.go:484 msg="system memory" total="32.0 GiB" free="21.3 GiB" free_swap="0 B" time=2026-05-09T20:12:16.519-03:00 level=INFO source=sched.go:491 msg="gpu memory" id=0 library=Metal available="24.5 GiB" free="25.0 GiB" minimum="512.0 MiB" overhead="0 B" time=2026-05-09T20:12:16.520-03:00 level=INFO source=client.go:359 msg="starting mlx runner subprocess" model=qwen3.6:35b-a3b-nvfp4 port=58131 time=2026-05-09T20:12:16.522-03:00 level=INFO source=sched.go:561 msg="loaded runners" count=1 time=2026-05-09T20:12:16.543-03:00 level=DEBUG source=dynamic.go:28 msg="MLX dynamic library loaded" path=/Applications/Ollama.app/Contents/Resources/mlx_metal_v4/libmlxc.dylib time=2026-05-09T20:12:16.543-03:00 level=INFO source=server.go:44 msg="MLX engine initialized" "MLX version"=0.31.2-7-ge8ebdeb device=gpu time=2026-05-09T20:12:16.639-03:00 level=INFO source=base.go:110 msg="Model architecture" arch=Qwen3_5MoeForConditionalGeneration time=2026-05-09T20:12:16.987-03:00 level=INFO source=runner.go:159 msg="Loaded tensors from manifest" count=1436 time=2026-05-09T20:12:23.063-03:00 level=INFO source=runner.go:194 msg="Starting HTTP server" host=127.0.0.1 port=58131 time=2026-05-09T20:12:23.157-03:00 level=INFO source=server.go:213 msg=ServeHTTP method=GET path=/v1/status took=5.579833ms status="200 OK" time=2026-05-09T20:12:23.173-03:00 level=INFO source=client.go:91 msg="mlx runner is ready" port=58131 time=2026-05-09T20:12:23.182-03:00 level=DEBUG source=sched.go:573 msg="finished setting up" runner.name=registry.ollama.ai/library/qwen3.6:35b-a3b-nvfp4 runner.size="20.4 GiB" runner.vram="20.4 GiB" runner.parallel=1 runner.pid=58767 runner.model=digest:1b50c6fdc2d4f75f94e6c19b49b20051bc6a7e73620d1ccdfc6687ec042ce754 runner.num_ctx=32768 time=2026-05-09T20:12:23.250-03:00 level=INFO source=cache.go:126 msg="cache miss" total=61 matched=0 cached=0 left=61 time=2026-05-09T20:12:34.068-03:00 level=INFO source=pipeline.go:135 msg="Prompt processing progress" processed=57 total=61 time=2026-05-09T20:12:34.068-03:00 level=DEBUG source=cache.go:401 msg="created snapshot" offset=57 time=2026-05-09T20:12:39.319-03:00 level=INFO source=pipeline.go:135 msg="Prompt processing progress" processed=60 total=61 time=2026-05-09T20:14:18.086-03:00 level=INFO source=pipeline.go:71 msg="peak memory" size="19.74 GiB" time=2026-05-09T20:14:18.087-03:00 level=INFO source=server.go:213 msg=ServeHTTP method=POST path=/v1/completions took=1m54.868602292s status="200 OK" time=2026-05-09T20:14:18.088-03:00 level=INFO source=server.go:213 msg=ServeHTTP method=POST path=/v1/tokenize took=2.894458ms status="200 OK" [GIN] 2026/05/09 - 20:14:18 | 200 | 2m1s | 127.0.0.1 | POST "/api/generate" time=2026-05-09T20:14:18.090-03:00 level=DEBUG source=sched.go:581 msg="context for request finished" time=2026-05-09T20:14:18.091-03:00 level=DEBUG source=sched.go:309 msg="runner with non-zero duration has gone idle, adding timer" runner.name=registry.ollama.ai/library/qwen3.6:35b-a3b-nvfp4 runner.size="20.4 GiB" runner.vram="20.4 GiB" runner.parallel=1 runner.pid=58767 runner.model=digest:1b50c6fdc2d4f75f94e6c19b49b20051bc6a7e73620d1ccdfc6687ec042ce754 runner.num_ctx=32768 duration=5m0s time=2026-05-09T20:14:18.091-03:00 level=DEBUG source=sched.go:327 msg="after processing request finished event" runner.name=registry.ollama.ai/library/qwen3.6:35b-a3b-nvfp4 runner.size="20.4 GiB" runner.vram="20.4 GiB" runner.parallel=1 runner.pid=58767 runner.model=digest:1b50c6fdc2d4f75f94e6c19b49b20051bc6a7e73620d1ccdfc6687ec042ce754 runner.num_ctx=32768 refCount=0 [GIN] 2026/05/09 - 20:14:27 | 200 | 503.084µs | 127.0.0.1 | HEAD "/" [GIN] 2026/05/09 - 20:14:27 | 200 | 66.68675ms | 127.0.0.1 | POST "/api/show" time=2026-05-09T20:14:27.727-03:00 level=DEBUG source=sched.go:672 msg="evaluating already loaded" model=digest:1b50c6fdc2d4f75f94e6c19b49b20051bc6a7e73620d1ccdfc6687ec042ce754 time=2026-05-09T20:14:27.728-03:00 level=INFO source=server.go:213 msg=ServeHTTP method=GET path=/v1/status took=103.416µs status="200 OK" time=2026-05-09T20:14:27.731-03:00 level=DEBUG source=cache.go:250 msg="switching cache path" page_out=1 page_in=0 time=2026-05-09T20:14:27.731-03:00 level=INFO source=cache.go:126 msg="cache hit" total=61 matched=61 cached=57 left=4 time=2026-05-09T20:14:33.373-03:00 level=INFO source=pipeline.go:135 msg="Prompt processing progress" processed=3 total=4 time=2026-05-09T20:14:33.374-03:00 level=DEBUG source=cache.go:401 msg="created snapshot" offset=60 ```
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug needs more info
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#87893
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?