[GH-ISSUE #12560] 🤔 Does num_batch do anything? If so what and how can we use it? #70391

New Issue

GiteaMirror · 2026-05-04T21:23:32-05:00

GiteaMirror commented

2026-05-04 21:23:32 -05:00

Originally created by @FieldMouse-AI on GitHub (Oct 10, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/12560

What is the issue?

🤔 Does num_batch do anything? If so what and how can we use it?

I only know that setting it a value like 4096 makes the model so large that it does not even load at all into my CPU, but ends up 100% CPU bound.

I am trying small values, but I am not sure if they have any real effect.

So, what is the real story?

😯😯 UPDATE: New data and identification of a possible regression

🧐🧐 It would appear that I am both correct and incorrect at the same time.

The allocation within the GPU appears to be as expected when viewed using nvtop:

PARAMETER num_batch 17: VRAM 6.145Gi/12.000Gi
PARAMETER num_batch not specified (eg, the default): VRAM 8.116Gi/12.000Gi

So, the allocation is visibly different when viewed using nvtop.

However, under 0.12.5, ollama ps will report the SIZE as 8.5 GB in both cases.

But, under 0.12.3, ollama ps will report the SIZE as approximately 6.7 GB for PARAMETER num_batch 17, and 8.5 GBfor when PARAMETER num_batch not specified (eg, the default).

🤔🤔 New Concern is about `ollama ps`

ollama ps is not correctly reporting the VRAM allocation as of 0.12.5.

The last good version for reporting correct VRAM values is 0.12.3.

Any thoughts, please?

This really looks like a regression.

Relevant log output

OS

Linux, Docker

GPU

Nvidia

CPU

Intel

Ollama version

0.12.3, 0.12.5

Originally created by @FieldMouse-AI on GitHub (Oct 10, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/12560 ### What is the issue? 🤔 Does num_batch do anything? If so what and how can we use it? I only know that setting it a value like 4096 makes the model so large that it does not even load at all into my CPU, but ends up 100% CPU bound. I am trying small values, but I am not sure if they have any real effect. So, what is the real story? ### 😯😯 UPDATE: New data and identification of a possible regression 🧐🧐 It would appear that I am both correct **and** incorrect at the same time. The allocation within the GPU appears to be as expected when viewed using `nvtop`: - `PARAMETER num_batch 17`: VRAM 6.145Gi/12.000Gi - `PARAMETER num_batch` not specified (eg, the default): VRAM 8.116Gi/12.000Gi So, the allocation is visibly different when viewed using `nvtop`. However, under **0.12.5**, `ollama ps` will report the `SIZE` as `8.5 GB` in both cases. But, under **0.12.3**, `ollama ps` will report the `SIZE` as approximately `6.7 GB` for `PARAMETER num_batch 17`, and `8.5 GB`for when `PARAMETER num_batch` not specified (eg, the default). ### 🤔🤔 New Concern is about `ollama ps` `ollama ps` is not correctly reporting the VRAM allocation as of **0.12.5**. The last good version for reporting correct VRAM values is **0.12.3**. Any thoughts, please? This really looks like a regression. ### Relevant log output ```shell ``` ### OS Linux, Docker ### GPU Nvidia ### CPU Intel ### Ollama version 0.12.3, 0.12.5

GiteaMirror added the bug label 2026-05-04 21:23:32 -05:00

GiteaMirror closed this issue

2026-05-04 21:23:33 -05:00

GiteaMirror commented

2026-05-04 21:23:34 -05:00

@dragetd commented on GitHub (Oct 11, 2025):

Edit: I was confused and had the wrong parameter in mind - pleas ignore.

@dragetd commented on GitHub (Oct 11, 2025): Edit: I was confused and had the wrong parameter in mind - pleas ignore.

GiteaMirror commented

2026-05-04 21:23:34 -05:00

@FieldMouse-AI commented on GitHub (Oct 11, 2025):

If you read online about it, it is the number of token per batch generated for streaming responses. Default is 1 I believe. Setting it to 4 or 8 reduces the overhead of network calls vs. new tokens, but causes less frequent updates. If you have a fast model (e.g. > 30 token/s) it makes sense to increase it a bit. If you have a very fast model (e.g. >100 token/s) you might even want to set it to 32 or something like this, but more is rarely useful.

@dragetd , thanks for the reply!

Could you do me a favor and reply to the following?

First: Where did you find this information? Could you supply links, please? Unfortunately, my searches yielded non-conclusive references within the Ollama and llama.cpp documentation.

Second: In my testing, leaving the num_batch parameter unset, my model would use about 70% of my GPU card's 12GB, while setting PARAMETER num_batch 17 reduced my model's allocation down to only 50% of my GPU card's 12GB.

@FieldMouse-AI commented on GitHub (Oct 11, 2025): > If you read online about it, it is the number of token per batch generated for streaming responses. Default is 1 I believe. Setting it to 4 or 8 reduces the overhead of network calls vs. new tokens, but causes less frequent updates. If you have a fast model (e.g. > 30 token/s) it makes sense to increase it a bit. If you have a very fast model (e.g. >100 token/s) you might even want to set it to 32 or something like this, but more is rarely useful. @dragetd , thanks for the reply! Could you do me a favor and reply to the following? **First:** Where did you find this information? Could you supply links, please? Unfortunately, my searches yielded non-conclusive references within the Ollama and llama.cpp documentation. **Second:** In my testing, leaving the `num_batch` parameter unset, my model would use about 70% of my GPU card's 12GB, while setting `PARAMETER num_batch 17` **reduced** my model's allocation down to only 50% of my GPU card's 12GB.

GiteaMirror commented

2026-05-04 21:23:34 -05:00

@dragetd commented on GitHub (Oct 11, 2025):

@meidaid Okay, I actually confused the parameter myself and was thinking of the wrong one! I am very sorry, please ignore my post!

@dragetd commented on GitHub (Oct 11, 2025): @meidaid Okay, I actually confused the parameter myself and was thinking of the wrong one! I am very sorry, please ignore my post!

GiteaMirror commented

2026-05-04 21:23:35 -05:00

@FieldMouse-AI commented on GitHub (Oct 15, 2025):

It would appear that I am both correct and incorrect at the same time.

The allocation within the GPU appears to be as expected when viewed using nvtop:

PARAMETER num_batch 17: VRAM 6.145Gi/12.000Gi
PARAMETER num_batch not specified (eg, the default): VRAM 8.116Gi/12.000Gi

So, the allocation is visibly different when viewed using nvtop.

However, under 0.12.5, ollama ps will report the SIZE as 8.5 GB in both cases.

But, under 0.12.3, ollama ps will report the SIZE as approximately 6.7 GB for PARAMETER num_batch 17, and 8.5 GBfor when PARAMETER num_batch not specified (eg, the default).

New Concern is about `ollama ps`

ollama ps is not correctly reporting the VRAM allocation as of 0.12.5.

The last good version for reporting correct VRAM values is 0.12.3.

Any thoughts, please?

This really looks like a regression.

@FieldMouse-AI commented on GitHub (Oct 15, 2025): It would appear that I am both correct **and** incorrect at the same time. The allocation within the GPU appears to be as expected when viewed using `nvtop`: - `PARAMETER num_batch 17`: VRAM 6.145Gi/12.000Gi - `PARAMETER num_batch` not specified (eg, the default): VRAM 8.116Gi/12.000Gi So, the allocation is visibly different when viewed using `nvtop`. However, under **0.12.5**, `ollama ps` will report the `SIZE` as `8.5 GB` in both cases. But, under **0.12.3**, `ollama ps` will report the `SIZE` as approximately `6.7 GB` for `PARAMETER num_batch 17`, and `8.5 GB`for when `PARAMETER num_batch` not specified (eg, the default). ### New Concern is about `ollama ps` `ollama ps` is not correctly reporting the VRAM allocation as of **0.12.5**. The last good version for reporting correct VRAM values is **0.12.3**. Any thoughts, please? This really looks like a regression.

GiteaMirror commented

2026-05-04 21:23:36 -05:00

@jessegross commented on GitHub (Oct 16, 2025):

@meidaid Please attach logs for the scenarios where you see differences. Everything looks like it is working fine to me. Be aware that the memory impact of batch size also depends on the model and context size, so ensure that you are holding those constant.

@jessegross commented on GitHub (Oct 16, 2025): @meidaid Please attach logs for the scenarios where you see differences. Everything looks like it is working fine to me. Be aware that the memory impact of batch size also depends on the model and context size, so ensure that you are holding those constant.

GiteaMirror commented

2026-05-04 21:23:36 -05:00

@FieldMouse-AI commented on GitHub (Oct 16, 2025):

@meidaid Please attach logs for the scenarios where you see differences. Everything looks like it is working fine to me. Be aware that the memory impact of batch size also depends on the model and context size, so ensure that you are holding those constant.

@jessegross , thanks for your reply!

The good news is that I do keep the num_ctx 100% stable at PARAMETER num_ctx 22000.

I will work to gather the logs.

@FieldMouse-AI commented on GitHub (Oct 16, 2025): > [@meidaid](https://github.com/meidaid) Please attach logs for the scenarios where you see differences. Everything looks like it is working fine to me. Be aware that the memory impact of batch size also depends on the model and context size, so ensure that you are holding those constant. @jessegross , thanks for your reply! The good news is that I do keep the `num_ctx` 100% stable at `PARAMETER num_ctx 22000`. I will work to gather the logs.

GiteaMirror commented

2026-05-04 21:23:37 -05:00

@FieldMouse-AI commented on GitHub (Oct 16, 2025):

@meidaid Please attach logs for the scenarios where you see differences. Everything looks like it is working fine to me. Be aware that the memory impact of batch size also depends on the model and context size, so ensure that you are holding those constant.

@jessegross , thanks for your reply!

The good news is that I do keep the num_ctx 100% stable at PARAMETER num_ctx 22000.

I will work to gather the logs.

Hello, @jessegross , I've added the logs.

In both cases, the conditions are as follows:

Ollama v0.12.5
The model is based on qwen3:4b-instruct-2507-q4_K_M
PARAMETER num_ctx 22000

The test cases are:

Test Case 1: PARAMETER num_batch 17 exists in the Modelfile.
Test Case 2: PARAMETER num_batch does not exist in the Modelfile.

Attached logs are below:

num_batch-17.log
num_batch-unset.log

@FieldMouse-AI commented on GitHub (Oct 16, 2025): > > [@meidaid](https://github.com/meidaid) Please attach logs for the scenarios where you see differences. Everything looks like it is working fine to me. Be aware that the memory impact of batch size also depends on the model and context size, so ensure that you are holding those constant. > > [@jessegross](https://github.com/jessegross) , thanks for your reply! > > The good news is that I do keep the `num_ctx` 100% stable at `PARAMETER num_ctx 22000`. > > I will work to gather the logs. Hello, @jessegross , I've added the logs. In both cases, the conditions are as follows: - Ollama v0.12.5 - The model is based on `qwen3:4b-instruct-2507-q4_K_M` - `PARAMETER num_ctx 22000` The test cases are: 1. Test Case 1: `PARAMETER num_batch 17` exists in the `Modelfile`. 2. Test Case 2: `PARAMETER num_batch` does not exist in the `Modelfile`. Attached logs are below: [num_batch-17.log](https://github.com/user-attachments/files/22948234/num_batch-17.log) [num_batch-unset.log](https://github.com/user-attachments/files/22948233/num_batch-unset.log)

GiteaMirror commented

2026-05-04 21:23:38 -05:00

@jessegross commented on GitHub (Oct 16, 2025):

The difference is caused by the use of the old llama engine vs the new Ollama engine. Although both do account for batch size when calculating memory size, the Ollama engine is more accurate overall and should closely track nvidia-smi much more closely.

We are in the process of migrating models over to the Ollama engine incrementally and qwen3 was running on it in 0.12.3. However, it was temporarily moved back in 0.12.5 to fix a regression that was encountered. It will on the Ollama engine again in 0.12.6.

@jessegross commented on GitHub (Oct 16, 2025): The difference is caused by the use of the old llama engine vs the new Ollama engine. Although both do account for batch size when calculating memory size, the Ollama engine is more accurate overall and should closely track nvidia-smi much more closely. We are in the process of migrating models over to the Ollama engine incrementally and qwen3 was running on it in 0.12.3. However, it was temporarily moved back in 0.12.5 to fix a regression that was encountered. It will on the Ollama engine again in 0.12.6.

GiteaMirror commented

2026-05-04 21:23:39 -05:00

@FieldMouse-AI commented on GitHub (Oct 16, 2025):

The difference is caused by the use of the old llama engine vs the new Ollama engine. Although both do account for batch size when calculating memory size, the Ollama engine is more accurate overall and should closely track nvidia-smi much more closely.

We are in the process of migrating models over to the Ollama engine incrementally and qwen3 was running on it in 0.12.3. However, it was temporarily moved back in 0.12.5 to fix a regression that was encountered. It will on the Ollama engine again in 0.12.6.

Ah, thank you! So, in 0.12.6 ollama ps will match nvtop again, right?

If so, that would make reducing memory allocations and debugging things so much easier again!

I will install 0.12.6 as soon as it is available and retest!

@FieldMouse-AI commented on GitHub (Oct 16, 2025): > The difference is caused by the use of the old llama engine vs the new Ollama engine. Although both do account for batch size when calculating memory size, the Ollama engine is more accurate overall and should closely track nvidia-smi much more closely. > > We are in the process of migrating models over to the Ollama engine incrementally and qwen3 was running on it in 0.12.3. However, it was temporarily moved back in 0.12.5 to fix a regression that was encountered. It will on the Ollama engine again in 0.12.6. Ah, thank you! So, in 0.12.6 `ollama ps` will match `nvtop` again, right? If so, that would make reducing memory allocations and debugging things so much easier again! I will install 0.12.6 as soon as it is available and retest!

GiteaMirror commented

2026-05-04 21:23:40 -05:00

@FieldMouse-AI commented on GitHub (Oct 17, 2025):

The difference is caused by the use of the old llama engine vs the new Ollama engine. Although both do account for batch size when calculating memory size, the Ollama engine is more accurate overall and should closely track nvidia-smi much more closely.
We are in the process of migrating models over to the Ollama engine incrementally and qwen3 was running on it in 0.12.3. However, it was temporarily moved back in 0.12.5 to fix a regression that was encountered. It will on the Ollama engine again in 0.12.6.

Ah, thank you! So, in 0.12.6 ollama ps will match nvtop again, right?

If so, that would make reducing memory allocations and debugging things so much easier again!

I will install 0.12.6 as soon as it is available and retest!

Thank you again, @jessegross !!!

Ollama 0.12.6 was just released and I ran tests on this new version of Ollama.

I am happy to report that numbers from ollama ps are more in line with what is shown in nvtop.
(Please, keep reading... you will see where my little concern arises...)

While it is easier for me to make estimates about what models can fit into a single card, the numbers feel slightly off. This might be due to nvtop using GiB while ollama ps might be using tghe more traditional GB?

For example: The sum of my two models appear as the following:

ollama ps: 7.8GB
nvtop: 6.730Gi/12.000Gi

So, there is an improvement.
But I am curious: Why this difference in values? 🤔

PS: I went back and stopped the smaller embedding model...

I went back and stopped the smaller embedding model and found that the numbers for just the larger model looked more normal:

ollama ps: 6.1GB
nvtop: 6.012Gi/12.000Gi

So, perhaps it might be that my embedding model, bge-me, might be doing something different with allocations??? 🤔

@FieldMouse-AI commented on GitHub (Oct 17, 2025): > > The difference is caused by the use of the old llama engine vs the new Ollama engine. Although both do account for batch size when calculating memory size, the Ollama engine is more accurate overall and should closely track nvidia-smi much more closely. > > We are in the process of migrating models over to the Ollama engine incrementally and qwen3 was running on it in 0.12.3. However, it was temporarily moved back in 0.12.5 to fix a regression that was encountered. It will on the Ollama engine again in 0.12.6. > > Ah, thank you! So, in 0.12.6 `ollama ps` will match `nvtop` again, right? > > If so, that would make reducing memory allocations and debugging things so much easier again! > > I will install 0.12.6 as soon as it is available and retest! Thank you again, @jessegross !!! Ollama **0.12.6** was just released and I ran tests on this new version of Ollama. I am happy to report that numbers from `ollama ps` are _**more**_ in line with what is shown in `nvtop`. (Please, keep reading... you will see where my little concern arises...) While it is easier for me to make estimates about what models can fit into a single card, the numbers feel slightly off. This might be due to `nvtop` using `GiB` while `ollama ps` might be using tghe more traditional `GB`? For example: The sum of my two models appear as the following: - `ollama ps`: 7.8GB - `nvtop`: 6.730Gi/12.000Gi So, there is an improvement. But I am curious: Why this difference in values? 🤔 ### PS: I went back and stopped the smaller embedding model... I went back and stopped the smaller embedding model and found that the numbers for just the larger model looked more normal: - `ollama ps`: 6.1GB - `nvtop`: 6.012Gi/12.000Gi So, perhaps it might be that my embedding model, `bge-me`, might be doing something different with allocations??? 🤔

GiteaMirror commented

2026-05-04 21:23:41 -05:00

@FieldMouse-AI commented on GitHub (Oct 18, 2025):

Hello, @jessegross !

Thanks again for all of your help.

In testing 0.12.6, I've found it to be quite stable and the memory footprint is now well within my requirements for my workflow.

I just have one more question to ask, though: What exactly is num_batch used for and how does it really affect memory? I could not find a good explanation reading either the Ollama docs or the llama.cpp docs. Any references you might be able to share?

@FieldMouse-AI commented on GitHub (Oct 18, 2025): Hello, @jessegross ! Thanks again for all of your help. In testing **0.12.6**, I've found it to be quite stable and the memory footprint is now well within my requirements for my workflow. I just have one more question to ask, though: What exactly is `num_batch` used for and how does it really affect memory? I could not find a good explanation reading either the Ollama docs or the llama.cpp docs. Any references you might be able to share?

GiteaMirror commented

2026-05-04 21:23:42 -05:00

@FieldMouse-AI commented on GitHub (Oct 18, 2025):

PS: I went back and stopped the smaller embedding model...

I went back and stopped the smaller embedding model and found that the numbers for just the larger model looked more normal:

ollama ps: 6.1GB

nvtop: 6.012Gi/12.000Gi

So, perhaps it might be that my embedding model, bge-me, might be doing something different with allocations??? 🤔

Goodnews, @jessegross !

🤗 I did some testing with num_batch and my embedding model through the ollama.embed() API (https://github.com/ollama/ollama/blob/main/docs/api.md#generate-embeddings) and found that when I reduced it to num_batch: 2 that, according to nvtop, its memory footprint went down from 1.7GB down to just about 0.7GB!

😭 Unfortunately, similar to the problem that we say with regular models, the embedding model's reduced memory footprint was not reflected in ollama ps where it still reported the allocation as 1.7GB when it should have been 0.7GB.

🤔 Does this look like it would be a similar fix for the ollama.embed() API that was done for the ollama.chat() / ollama.generate() APIs?

Thanks in advance!

@FieldMouse-AI commented on GitHub (Oct 18, 2025): > ### PS: I went back and stopped the smaller embedding model... > I went back and stopped the smaller embedding model and found that the numbers for just the larger model looked more normal: > > * `ollama ps`: 6.1GB > * `nvtop`: 6.012Gi/12.000Gi > > So, perhaps it might be that my embedding model, `bge-me`, might be doing something different with allocations??? 🤔 Goodnews, @jessegross ! 🤗 I did some testing with `num_batch` and my embedding model through the `ollama.embed()` API (https://github.com/ollama/ollama/blob/main/docs/api.md#generate-embeddings) and found that when I reduced it to `num_batch: 2` that, according to `nvtop`, its memory footprint went **down from 1.7GB down to just about 0.7GB**! 😭 Unfortunately, similar to the problem that we say with regular models, the embedding model's reduced memory footprint was not reflected in `ollama ps` where it **still reported the allocation as 1.7GB when it should have been 0.7GB**. 🤔 Does this look like it would be a similar fix for the `ollama.embed()` API that was done for the `ollama.chat()` / `ollama.generate()` APIs? Thanks in advance!

GiteaMirror commented

2026-05-04 21:23:42 -05:00

@jessegross commented on GitHub (Oct 20, 2025):

It's a similar issue as before. bge-m3 runs on the old engine where memory allocation is not as good. Embedding models that run on the Ollama engine with better memory management include qwen3-embedding and embeddinggemma.

Regarding the original question, batch size mostly a tradeoff between speed and memory usage - larger batch sizes will process more tokens at once, which is faster but requirements more memory. However, there are situations which require all tokens to be present in the same batch, including embedding and image models. In general, I would recommend leaving it at the default.

@jessegross commented on GitHub (Oct 20, 2025): It's a similar issue as before. bge-m3 runs on the old engine where memory allocation is not as good. Embedding models that run on the Ollama engine with better memory management include qwen3-embedding and embeddinggemma. Regarding the original question, batch size mostly a tradeoff between speed and memory usage - larger batch sizes will process more tokens at once, which is faster but requirements more memory. However, there are situations which require all tokens to be present in the same batch, including embedding and image models. In general, I would recommend leaving it at the default.

GiteaMirror commented

2026-05-04 21:23:42 -05:00

@FieldMouse-AI commented on GitHub (Oct 21, 2025):

It's a similar issue as before. bge-m3 runs on the old engine where memory allocation is not as good. Embedding models that run on the Ollama engine with better memory management include qwen3-embedding and embeddinggemma.

Regarding the original question, batch size mostly a tradeoff between speed and memory usage - larger batch sizes will process more tokens at once, which is faster but requirements more memory. However, there are situations which require all tokens to be present in the same batch, including embedding and image models. In general, I would recommend leaving it at the default.

Hello, @jessegross , I see what you mean now.

I am already using Qwen3 for other things, so I can try replacing bge-m3 with qwen3-embedding and fully retest my system.

Thanks for the insights!

Do you think we should close this issue now or should we wait until after I have some statistics from after I replace bge-m3 with qwen3-embedding?

@FieldMouse-AI commented on GitHub (Oct 21, 2025): > It's a similar issue as before. bge-m3 runs on the old engine where memory allocation is not as good. Embedding models that run on the Ollama engine with better memory management include qwen3-embedding and embeddinggemma. > > Regarding the original question, batch size mostly a tradeoff between speed and memory usage - larger batch sizes will process more tokens at once, which is faster but requirements more memory. However, there are situations which require all tokens to be present in the same batch, including embedding and image models. In general, I would recommend leaving it at the default. Hello, @jessegross , I see what you mean now. I am already using Qwen3 for other things, so I can try replacing `bge-m3 ` with `qwen3-embedding` and fully retest my system. Thanks for the insights! Do you think we should close this issue now **or** should we wait until after I have some statistics from after I replace `bge-m3 ` with `qwen3-embedding`?

GiteaMirror commented

2026-05-04 21:23:44 -05:00

@jessegross commented on GitHub (Oct 21, 2025):

I think we can close the issue - it should have a similar impact to the chat model running on the Ollama engine.

@jessegross commented on GitHub (Oct 21, 2025): I think we can close the issue - it should have a similar impact to the chat model running on the Ollama engine.

Sign in to join this conversation.

Branches Tags

main

hoyyeva/anthropic-local-image-path

dhiltgen/ci

dhiltgen/llama-runner

parth-remove-claude-desktop-launch

hoyyeva/anthropic-reference-images-path

parth-anthropic-reference-images-path

brucemacd/download-before-remove

hoyyeva/editor-config-repair

parth-mlx-decode-checkpoints

parth-launch-codex-app

hoyyeva/fix-codex-model-metadata-warning

hoyyeva/qwen

parth/hide-claude-desktop-till-release

hoyyeva/opencode-image-modality

parth-add-claude-code-autoinstall

release_v0.22.0

pdevine/manifest-list

codex/fix-codex-model-metadata-warning

pdevine/addressable-manifest

brucemacd/launch-fetch-reccomended

jmorganca/llama-compat

launch-copilot-cli

hoyyeva/opencode-thinking

release_v0.20.7

parth-auto-save-backup

parth-test

jmorganca/gemma4-audio-replacements

fix-manifest-digest-on-pull

hoyyeva/vscode-improve

brucemacd/install-server-wait

parth/update-claude-docs

brucemac/start-ap-install

pdevine/mlx-update

pdevine/qwen35_vision

drifkin/api-show-fallback

mintlify/image-generation-1773352582

hoyyeva/server-context-length-local-config

jmorganca/faster-reptition-penalties

jmorganca/convert-nemotron

parth-pi-thinking

pdevine/sampling-penalties

jmorganca/fix-create-quantization-memory

dongchen/resumable_transfer_fix

pdevine/sampling-cache-error

jessegross/mlx-usage

hoyyeva/openclaw-config

hoyyeva/app-html

pdevine/qwen3next

brucemacd/sign-sh-install

brucemacd/tui-update

brucemacd/usage-api

jmorganca/launch-empty

fix-app-dist-embed

mxyng/mlx-compile

mxyng/mlx-quant

mxyng/mlx-glm4.7

mxyng/mlx

brucemacd/simplify-model-picker

jmorganca/qwen3-concurrent

fix-glm-4.7-flash-mla-config

drifkin/qwen3-coder-opening-tag

brucemacd/usage-cli

fix-cuda12-fattn-shmem

ollama-imagegen-docs

parth/fix-multiline-inputs

brucemacd/config-docs

mxyng/model-files

mxyng/simple-execute

fix-imagegen-ollama-models

mxyng/async-upload

jmorganca/lazy-no-dtype-changes

imagegen-auto-detect-create

parth/decrease-concurrent-download-hf

fix-mlx-quantize-init

jmorganca/x-cleanup

usage

imagegen-readme

jmorganca/glm-image

mlx-gpu-cd

jmorganca/imagegen-modelfile

parth/agent-skills

parth/agent-allowlist

parth/signed-in-offline

parth/agents

parth/fix-context-chopping

improve-cloud-flow

parth/add-models-websearch

parth/prompt-renderer-mcp

jmorganca/native-settings

jmorganca/download-stream-hash

jmorganca/client2-rebased

brucemacd/oai-chat-req-multipart

jessegross/multi_chunk_reserve

grace/additional-omit-empty

grace/mistral-3-large

mxyng/tokenizer2

mxyng/tokenizer

jessegross/flash

hoyyeva/windows-nacked-app

mxyng/cleanup-attention

grace/deepseek-parser

hoyyeva/remember-unsent-prompt

parth/add-lfs-pointer-error-conversion

parth/olmo2-test2

hoyyeva/ollama-launchagent-plist

nicole/olmo-model

parth/olmo-test

mxyng/remove-embedded

parth/render-template

jmorganca/intellect-3

parth/remove-prealloc-linter

jmorganca/cmd-eval

nicole/nomic-embed-text-fix

mxyng/lint-2

hoyyeva/add-gemini-3-pro-preview

hoyyeva/load-model-list

mxyng/expand-path

mxyng/environ-2

hoyyeva/deeplink-json-encoding

parth/improve-tool-calling-tests

hoyyeva/conversation

hoyyeva/assistant-edit-response

hoyyeva/thinking

origin/brucemacd/invalid-char-i-err

parth/improve-tool-calling

jmorganca/required-omitempty

grace/qwen3-vl-tests

mxyng/iter-client

parth/docs-readme

nicole/embed-test

pdevine/integration-benchstat

parth/remove-generate-cmd

parth/add-toolcall-id

mxyng/server-tests

jmorganca/glm-4.6

jmorganca/gin-h-compat

drifkin/stable-tool-args

pdevine/qwen3-more-thinking

parth/add-websearch-client

nicole/websearch_local

jmorganca/qwen3-coder-updates

grace/deepseek-v3-migration-tests

mxyng/fix-create

jmorganca/cloud-errors

pdevine/parser-tidy

revert-12233-parth/simplify-entrypoints-runner

parth/enable-so-gpt-oss

brucemacd/qwen3vl

jmorganca/readme-simplify

parth/gpt-oss-structured-outputs

revert-12039-jmorganca/tools-braces

mxyng/embeddings

mxyng/gguf

mxyng/benchmark

mxyng/types-null

parth/move-parsing

mxyng/gemma2

jmorganca/docs

mxyng/16-bit

mxyng/create-stdin

pdevine/authorizedkeys

mxyng/quant

parth/opt-in-error-context-window

brucemacd/cache-models

brucemacd/runner-completion

jmorganca/llama-update-6

brucemacd/benchmark-list

brucemacd/partial-read-caps

parth/deepseek-r1-tools

mxyng/omit-array

parth/tool-prefix-temp

brucemacd/runner-test

jmorganca/qwen25vl

brucemacd/model-forward-test-ext

parth/python-function-parsing

jmorganca/cuda-compression-none

drifkin/num-parallel

drifkin/chat-truncation-fix

jmorganca/sync

parth/python-tools-calling

drifkin/array-head-count

brucemacd/create-no-loop

parth/server-enable-content-stream-with-tools

qwen25omni

mxyng/v3

brucemacd/ropeconfig

jmorganca/silence-tokenizer

parth/sample-so-test

parth/sampling-structured-outputs

brucemacd/doc-go-engine

parth/constrained-sampling-json

jmorganca/mistral-wip

brucemacd/mistral-small-convert

parth/sample-unmarshal-json-for-params

brucemacd/jomorganca/mistral

pdevine/bfloat16

jmorganca/mistral

brucemacd/mistral

pdevine/logging

parth/sample-correctness-fix

parth/sample-fix-sorting

jmorgan/sample-fix-sorting-extras

jmorganca/temp-0-images

brucemacd/parallel-embed-models

brucemacd/shim-grammar

jmorganca/fix-gguf-error

bmizerany/nameswork

jmorganca/faster-releases

bmizerany/validatenames

brucemacd/err-no-vocab

brucemacd/rope-config

brucemacd/err-hint

brucemacd/qwen2_5

brucemacd/logprobs

brucemacd/new_runner_graph_bench

progress-flicker

brucemacd/forward-test

brucemacd/go_qwen2

pdevine/gemma2

jmorganca/add-missing-symlink-eval

mxyng/next-debug

parth/set-context-size-openai

brucemacd/next-bpe-bench

brucemacd/next-bpe-test

brucemacd/new_runner_e2e

brucemacd/new_runner_qwen2

pdevine/convert-cohere2

brucemacd/convert-cli

parth/log-probs

mxyng/next-mlx

mxyng/cmd-history

parth/templating

parth/tokenize-detokenize

brucemacd/check-key-register

bmizerany/grammar

jmorganca/vendor-081b29bd

mxyng/func-checks

jmorganca/fix-null-format

parth/fix-default-to-warn-json

jmorganca/qwen2vl

jmorganca/no-concat

parth/cmd-cleanup-SO

brucemacd/check-key-register-structured-err

parth/openai-stream-usage

parth/fix-referencing-so

stream-tools-stop

jmorganca/degin-1

brucemacd/install-path-clean

brucemacd/push-name-validation

brucemacd/browser-key-register

jmorganca/openai-fix-first-message

jmorganca/fix-proxy

jessegross/sample

parth/disallow-streaming-tools

dhiltgen/remove_submodule

jmorganca/ga

jmorganca/mllama

pdevine/newlines

pdevine/geems-2b

jmorganca/llama-bump

mxyng/modelname-7

mxyng/gin-slog

mxyng/modelname-6

jyan/convert-prog

jyan/quant5

paligemma-support

pdevine/import-docs

jmorganca/openai-context

jyan/paligemma

jyan/p2

jyan/palitest

bmizerany/embedspeedup

jmorganca/llama-vit

brucemacd/allow-ollama

royh/ep-methods

royh/whisper

mxyng/api-models

mxyng/fix-memory

jyan/q4_4/8

jyan/ollama-v

royh/stream-tools

roy-embed-parallel

bmizerany/hrm

revert-5963-revert-5924-mxyng/llama3.1-rope

royh/embed-viz

jyan/local2

jyan/auth

jyan/local

jyan/parse-temp

jmorganca/template-mistral

jyan/reord-g

royh-openai-suffixdocs

royh-imgembed

royh-embed-parallel

jyan/quant4

royh-precision

jyan/progress

pdevine/fix-template

jyan/quant3

pdevine/ggla

mxyng/update-registry-domain

jmorganca/ggml-static

mxyng/create-context

jyan/v0.146

mxyng/layers-from-files

build_dist

bmizerany/noseek

royh-ls

royh-name

timeout

mxyng/server-timestamp

bmizerany/nosillyggufslurps

royh-params

jmorganca/llama-cpp-7c26775

royh-openai-delete

royh-show-rigid

jmorganca/enable-fa

jmorganca/no-error-template

jyan/format

royh-testdelete

bmizerany/fastverify

language_support

pdevine/ps-glitches

brucemacd/tokenize

bruce/iq-quants

bmizerany/filepathwithcoloninhost

mxyng/split-bin

bmizerany/client-registry

jmorganca/if-none-match

native

jmorganca/native

jmorganca/batch-embeddings

jmorganca/initcmake

jmorganca/mm

pdevine/showggmlinfo

modenameenforcealphanum

bmizerany/modenameenforcealphanum

jmorganca/done-reason

jmorganca/llama-cpp-8960fe8

ollama.com

bmizerany/filepathnobuild

bmizerany/types/model/defaultfix

rmdisplaylong

nogogen

bmizerany/x

modelfile-readme

bmizerany/replacecolon

jmorganca/limit

jmorganca/execstack

jmorganca/replace-assets

mxyng/tune-concurrency

jmorganca/testing

whitespace-detection

jmorganca/options

upgrade-all

scratch

cuda-search

mattw/airenamer

mattw/allmodelsonhuggingface

mattw/quantcontext

mattw/whatneedstorun

brucemacd/llama-mem-calc

mattw/faq-context

mattw/communitylinks

mattw/noprune

mattw/python-functioncalling

rename

mxyng/install

pulse

remove-first

editor

mattw/selfqueryingretrieval

cgo

mattw/howtoquant

api

matt/streamingapi

format-config

mxyng/extra-args

shell

update-nous-hermes

cp-model

upload-progress

fix-unknown-model

fix-model-names

delete-fix

insecure-registry

ls

deletemodels

progressbar

readme-updates

license-layers

skip-list

list-models

modelpath

matt/examplemodelfiles

distribution

go-opts

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/ollama#70391

[GH-ISSUE #12560] 🤔 Does num_batch do anything? If so what and how can we use it? #70391

What is the issue?

😯😯 UPDATE: New data and identification of a possible regression

🤔🤔 New Concern is about ollama ps

Relevant log output

OS

GPU

CPU

Ollama version

New Concern is about ollama ps

PS: I went back and stopped the smaller embedding model...

PS: I went back and stopped the smaller embedding model...

🤔🤔 New Concern is about `ollama ps`

New Concern is about `ollama ps`