[GH-ISSUE #4170] OLLAMA_NUM_PARALLEL cannot be the solution. #28352

New Issue

GiteaMirror · 2026-04-22T06:28:34-05:00

GiteaMirror commented

2026-04-22 06:28:34 -05:00

Originally created by @KevinKrueger on GitHub (May 5, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/4170

Originally assigned to: @dhiltgen on GitHub.

You don't actually need this setting.
Ollama should actually know on its own how many parallel requests it can handle.

Because Ollama should know what hardware and corresponding performance is available. This information can be analyzed.

For example, if you send several requests where you already know in advance how much performance a request will consume, then an error message should come from the API "No further requests can be processed by the model due to the installed hardware." Then you could provide some good information on what you could do to avoid this or in the task you could say how much performance is required for each additional request so that a reasonably "fast" answer comes back.

Originally created by @KevinKrueger on GitHub (May 5, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/4170 Originally assigned to: @dhiltgen on GitHub. You don't actually need this setting. Ollama should actually know on its own how many parallel requests it can handle. Because Ollama should know what hardware and corresponding performance is available. This information can be analyzed. For example, if you send several requests where you already know in advance how much performance a request will consume, then an error message should come from the API "No further requests can be processed by the model due to the installed hardware." Then you could provide some good information on what you could do to avoid this or in the task you could say how much performance is required for each additional request so that a reasonably "fast" answer comes back.

GiteaMirror added the feature request label 2026-04-22 06:28:35 -05:00

GiteaMirror closed this issue

2026-04-22 06:28:35 -05:00

GiteaMirror commented

2026-04-22 06:28:37 -05:00

@jmorganca commented on GitHub (May 5, 2024):

Yes absolutely! You hit the nail on the head on how this feature will evolve 😊 . OLLAMA_NUM_PARALLEL is an experimental feature flag that will eventually be replaced with this sort of improvement.

@jmorganca commented on GitHub (May 5, 2024): Yes absolutely! You hit the nail on the head on how this feature will evolve 😊 . `OLLAMA_NUM_PARALLEL` is an experimental feature flag that will eventually be replaced with this sort of improvement.

GiteaMirror commented

2026-04-22 06:28:38 -05:00

@0sengseng0 commented on GitHub (May 6, 2024):

Yes absolutely! You hit the nail on the head on how this feature will evolve 😊 . OLLAMA_NUM_PARALLEL is an experimental feature flag that will eventually be replaced with this sort of improvement.

But why is the request still not parallel when I configure OLLAMA_NUM_PARALLEL?
And, the num_ctx I configured did not work either 😊

@0sengseng0 commented on GitHub (May 6, 2024): > Yes absolutely! You hit the nail on the head on how this feature will evolve 😊 . `OLLAMA_NUM_PARALLEL` is an experimental feature flag that will eventually be replaced with this sort of improvement. But why is the request still not parallel when I configure OLLAMA_NUM_PARALLEL? And, the num_ctx I configured did not work either 😊 ![image](https://github.com/ollama/ollama/assets/73268510/bc8df5eb-5863-450b-aaf0-ae6811cfd090) ![image](https://github.com/ollama/ollama/assets/73268510/ed411abe-dfc0-44f0-b8c9-9f9d2c115810)

GiteaMirror commented

2026-04-22 06:28:39 -05:00

@jmorganca commented on GitHub (May 6, 2024):

Hi @0sengseng0 it seems you're missing an O: OLLAMA_NUM_PARALLEL=3

Will close this as it is definitely plan A :)

@jmorganca commented on GitHub (May 6, 2024): Hi @0sengseng0 it seems you're missing an O: `OLLAMA_NUM_PARALLEL=3` Will close this as it is definitely plan A :)

GiteaMirror commented

2026-04-22 06:28:39 -05:00

@0sengseng0 commented on GitHub (May 7, 2024):

Hi @0sengseng0 it seems you're missing an O: OLLAMA_NUM_PARALLEL=3

Will close this as it is definitely plan A :)

According to the logs, n_ctx is still the default 2048 and has not been modified. What about large tokens? Is it truncated? (Because I see GPU usage is always fixed, even for large tokens, multi-threading, token usage is still the same.) OX

@0sengseng0 commented on GitHub (May 7, 2024): > Hi @0sengseng0 it seems you're missing an O: `OLLAMA_NUM_PARALLEL=3` > > Will close this as it is definitely plan A :) According to the logs, n_ctx is still the default 2048 and has not been modified. What about large tokens? Is it truncated? (Because I see GPU usage is always fixed, even for large tokens, multi-threading, token usage is still the same.) OX

GiteaMirror commented

2026-04-22 06:28:40 -05:00

@DirtyKnightForVi commented on GitHub (May 7, 2024):

Hi @0sengseng0 it seems you're missing an O: OLLAMA_NUM_PARALLEL=3
Will close this as it is definitely plan A :)

According to the logs, n_ctx is still the default 2048 and has not been modified. What about large tokens? Is it truncated? (Because I see GPU usage is always fixed, even for large tokens, multi-threading, token usage is still the same.) OX

OLLAMA_NUM_PARALLEL will split n_ctx

@DirtyKnightForVi commented on GitHub (May 7, 2024): > > Hi @0sengseng0 it seems you're missing an O: `OLLAMA_NUM_PARALLEL=3` > > Will close this as it is definitely plan A :) > > According to the logs, n_ctx is still the default 2048 and has not been modified. What about large tokens? Is it truncated? (Because I see GPU usage is always fixed, even for large tokens, multi-threading, token usage is still the same.) OX OLLAMA_NUM_PARALLEL will split n_ctx

GiteaMirror commented

2026-04-22 06:28:41 -05:00

@DirtyKnightForVi commented on GitHub (May 7, 2024):

in my case, my prompt as follows:

DDL

create table XXXXX
(about 12000 tokens)

HINT

(about 500 tokens, notes, etc)

if OLLAMA_NUM_PARALLEL=2，model can not get table name。 ifOLLAMA_NUM_PARALLEL=1, The model's output can accurately and flawlessly recognize table names.

@DirtyKnightForVi commented on GitHub (May 7, 2024): in my case, my prompt as follows: # DDL ```sql create table XXXXX (about 12000 tokens) ``` # HINT (about 500 tokens, notes, etc) if OLLAMA_NUM_PARALLEL=2，model can not get table name。 ifOLLAMA_NUM_PARALLEL=1, The model's output can accurately and flawlessly recognize table names.

GiteaMirror commented

2026-04-22 06:28:42 -05:00

@0sengseng0 commented on GitHub (May 7, 2024):

in my case, my prompt as follows:

DDL
create table XXXXX
(about 12000 tokens)
HINT

(about 500 tokens, notes, etc)

if OLLAMA_NUM_PARALLEL=2，model can not get table name。 ifOLLAMA_NUM_PARALLEL=1, The model's output can accurately and flawlessly recognize table names.

Do you mean that n_ctx is shared for requests received in a uniform time period? For example, if I send 10 requests in 1s with an n_ctx of 20480, does each request actually take only 2048 tokens?

@0sengseng0 commented on GitHub (May 7, 2024): > in my case, my prompt as follows: > > # DDL > ```sql > create table XXXXX > (about 12000 tokens) > ``` > > # HINT > (about 500 tokens, notes, etc) > > if OLLAMA_NUM_PARALLEL=2，model can not get table name。 ifOLLAMA_NUM_PARALLEL=1, The model's output can accurately and flawlessly recognize table names. Do you mean that n_ctx is shared for requests received in a uniform time period? For example, if I send 10 requests in 1s with an n_ctx of 20480, does each request actually take only 2048 tokens?

GiteaMirror commented

2026-04-22 06:28:43 -05:00

@DirtyKnightForVi commented on GitHub (May 7, 2024):

in my case, my prompt as follows:

DDL
create table XXXXX
(about 12000 tokens)
HINT

(about 500 tokens, notes, etc)
if OLLAMA_NUM_PARALLEL=2，model can not get table name。 ifOLLAMA_NUM_PARALLEL=1, The model's output can accurately and flawlessly recognize table names.
Do you mean that n_ctx is shared for requests received in a uniform time period? For example, if I send 10 requests in 1s with an n_ctx of 20480, does each request actually take only 2048 tokens?

Yeah, sure.

So, I use openweb UI with ollama.

Each model instance is set by parameters like n_ctx, while OLLAMA_NUM_PARALLEL is a shared parameter for all instances. If we take any two instances with n_ctx=A and n_ctx=B, then the actual context for each instance is calculated as: n_ctx / OLLAMA_NUM_PARALLEL

@DirtyKnightForVi commented on GitHub (May 7, 2024): > > in my case, my prompt as follows: > > # DDL > > ```sql > > create table XXXXX > > (about 12000 tokens) > > ``` > > > > > > > > > > > > > > > > > > > > > > > > # HINT > > (about 500 tokens, notes, etc) > > if OLLAMA_NUM_PARALLEL=2，model can not get table name。 ifOLLAMA_NUM_PARALLEL=1, The model's output can accurately and flawlessly recognize table names. > > Do you mean that n_ctx is shared for requests received in a uniform time period? For example, if I send 10 requests in 1s with an n_ctx of 20480, does each request actually take only 2048 tokens? Yeah, sure. So, I use openweb UI with ollama. Each model instance is set by parameters like `n_ctx`, while `OLLAMA_NUM_PARALLEL` is a shared parameter for all instances. If we take any two instances with `n_ctx=A` and `n_ctx=B`, then the actual context for each instance is calculated as: `n_ctx / OLLAMA_NUM_PARALLEL`

GiteaMirror commented

2026-04-22 06:28:44 -05:00

@0sengseng0 commented on GitHub (May 7, 2024):

in my case, my prompt as follows:就我而言，我的提示如下：

DDL 数据定义语言
create table XXXXX
(about 12000 tokens)
HINT 暗示

(about 500 tokens, notes, etc)（约500个代币、票据等）
if OLLAMA_NUM_PARALLEL=2，model can not get table name。 ifOLLAMA_NUM_PARALLEL=1, The model's output can accurately and flawlessly recognize table names.如果OLLAMA_NUM_PARALLEL=2，模型无法获取表名。如果OLLAMA_NUM_PARALLEL=1，模型的输出可以准确无误地识别表名。
Do you mean that n_ctx is shared for requests received in a uniform time period? For example, if I send 10 requests in 1s with an n_ctx of 20480, does each request actually take only 2048 tokens?您的意思是，对于在统一时间段内收到的请求，n_ctx 是共享的吗？例如，如果我在 1s 内发送 10 个请求，n_ctx 为 20480，那么每个请求实际上只需要 2048 个令牌吗？
Yeah, sure. 好，当然。

So, I use openweb UI with ollama.所以，我将 openweb UI 与 ollama 一起使用。

Each model instance is set by parameters like n_ctx, while OLLAMA_NUM_PARALLEL is a shared parameter for all instances. If we take any two instances with n_ctx=A and n_ctx=B, then the actual context for each instance is calculated as: n_ctx / OLLAMA_NUM_PARALLEL每个模型实例都由 n_ctx 等参数设置，而 OLLAMA_NUM_PARALLEL 是所有实例的共享参数。如果我们采用 n_ctx=A 和 n_ctx=B 的任意两个实例，则每个实例的实际上下文计算如下：n_ctx / OLLAMA_NUM_PARALLEL。

in my case, my prompt as follows:

DDL
create table XXXXX
(about 12000 tokens)
HINT

(about 500 tokens, notes, etc)
if OLLAMA_NUM_PARALLEL=2，model can not get table name。 ifOLLAMA_NUM_PARALLEL=1, The model's output can accurately and flawlessly recognize table names.
Do you mean that n_ctx is shared for requests received in a uniform time period? For example, if I send 10 requests in 1s with an n_ctx of 20480, does each request actually take only 2048 tokens?
Yeah, sure.

So, I use openweb UI with ollama.

Each model instance is set by parameters like n_ctx, while OLLAMA_NUM_PARALLEL is a shared parameter for all instances. If we take any two instances with n_ctx=A and n_ctx=B, then the actual context for each instance is calculated as: n_ctx / OLLAMA_NUM_PARALLEL

thannnnnnnnnnnnnnnnnnnks,you gave me a sense of why these two problems occurred:

When I set OLLAMA_NUM_PARALLEL=100, the response is only one sentence.
The GPU occupancy is constant all the time.

Now is there anything ollama can do to improve GPU usage? I changed these two parameters, but ollama still doesn't use more resources.

@0sengseng0 commented on GitHub (May 7, 2024): > > > in my case, my prompt as follows:就我而言，我的提示如下： > > > # DDL 数据定义语言 > > > ```sql > > > create table XXXXX > > > (about 12000 tokens) > > > ``` > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > # HINT 暗示 > > > (about 500 tokens, notes, etc)（约500个代币、票据等） > > > if OLLAMA_NUM_PARALLEL=2，model can not get table name。 ifOLLAMA_NUM_PARALLEL=1, The model's output can accurately and flawlessly recognize table names.如果OLLAMA_NUM_PARALLEL=2，模型无法获取表名。如果OLLAMA_NUM_PARALLEL=1，模型的输出可以准确无误地识别表名。 > > > > > > Do you mean that n_ctx is shared for requests received in a uniform time period? For example, if I send 10 requests in 1s with an n_ctx of 20480, does each request actually take only 2048 tokens?您的意思是，对于在统一时间段内收到的请求，n_ctx 是共享的吗？例如，如果我在 1s 内发送 10 个请求，n_ctx 为 20480，那么每个请求实际上只需要 2048 个令牌吗？ > > Yeah, sure. 好，当然。 > > So, I use openweb UI with ollama.所以，我将 openweb UI 与 ollama 一起使用。 > > Each model instance is set by parameters like `n_ctx`, while `OLLAMA_NUM_PARALLEL` is a shared parameter for all instances. If we take any two instances with `n_ctx=A` and `n_ctx=B`, then the actual context for each instance is calculated as: `n_ctx / OLLAMA_NUM_PARALLEL`每个模型实例都由 `n_ctx` 等参数设置，而 `OLLAMA_NUM_PARALLEL` 是所有实例的共享参数。如果我们采用 `n_ctx=A` 和 `n_ctx=B` 的任意两个实例，则每个实例的实际上下文计算如下：`n_ctx / OLLAMA_NUM_PARALLEL`。 > > > in my case, my prompt as follows: > > > # DDL > > > ```sql > > > create table XXXXX > > > (about 12000 tokens) > > > ``` > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > # HINT > > > (about 500 tokens, notes, etc) > > > if OLLAMA_NUM_PARALLEL=2，model can not get table name。 ifOLLAMA_NUM_PARALLEL=1, The model's output can accurately and flawlessly recognize table names. > > > > > > Do you mean that n_ctx is shared for requests received in a uniform time period? For example, if I send 10 requests in 1s with an n_ctx of 20480, does each request actually take only 2048 tokens? > > Yeah, sure. > > So, I use openweb UI with ollama. > > Each model instance is set by parameters like `n_ctx`, while `OLLAMA_NUM_PARALLEL` is a shared parameter for all instances. If we take any two instances with `n_ctx=A` and `n_ctx=B`, then the actual context for each instance is calculated as: `n_ctx / OLLAMA_NUM_PARALLEL` thannnnnnnnnnnnnnnnnnnks,you gave me a sense of why these two problems occurred: 1. When I set OLLAMA_NUM_PARALLEL=100, the response is only one sentence. 2. The GPU occupancy is constant all the time. Now is there anything ollama can do to improve GPU usage? I changed these two parameters, but ollama still doesn't use more resources. ![image](https://github.com/ollama/ollama/assets/73268510/148d7f87-9a55-450b-ad15-9b9e85f02291)

GiteaMirror commented

2026-04-22 06:28:45 -05:00

@DirtyKnightForVi commented on GitHub (May 7, 2024):

in my case, my prompt as follows:就我而言，我的提示如下：

DDL 数据定义语言
create table XXXXX
(about 12000 tokens)
HINT 暗示

(about 500 tokens, notes, etc)（约500个代币、票据等）
if OLLAMA_NUM_PARALLEL=2，model can not get table name。 ifOLLAMA_NUM_PARALLEL=1, The model's output can accurately and flawlessly recognize table names.如果OLLAMA_NUM_PARALLEL=2，模型无法获取表名。如果OLLAMA_NUM_PARALLEL=1，模型的输出可以准确无误地识别表名。
Do you mean that n_ctx is shared for requests received in a uniform time period? For example, if I send 10 requests in 1s with an n_ctx of 20480, does each request actually take only 2048 tokens?您的意思是，对于在统一时间段内收到的请求，n_ctx 是共享的吗？例如，如果我在 1s 内发送 10 个请求，n_ctx 为 20480，那么每个请求实际上只需要 2048 个令牌吗？
Yeah, sure. 好，当然。
So, I use openweb UI with ollama.所以，我将 openweb UI 与 ollama 一起使用。
Each model instance is set by parameters like n_ctx, while OLLAMA_NUM_PARALLEL is a shared parameter for all instances. If we take any two instances with n_ctx=A and n_ctx=B, then the actual context for each instance is calculated as: n_ctx / OLLAMA_NUM_PARALLEL每个模型实例都由 n_ctx 等参数设置，而 OLLAMA_NUM_PARALLEL 是所有实例的共享参数。如果我们采用 n_ctx=A 和 n_ctx=B 的任意两个实例，则每个实例的实际上下文计算如下：n_ctx / OLLAMA_NUM_PARALLEL。
in my case, my prompt as follows:

DDL
create table XXXXX
(about 12000 tokens)
HINT

(about 500 tokens, notes, etc)
if OLLAMA_NUM_PARALLEL=2，model can not get table name。 ifOLLAMA_NUM_PARALLEL=1, The model's output can accurately and flawlessly recognize table names.
Do you mean that n_ctx is shared for requests received in a uniform time period? For example, if I send 10 requests in 1s with an n_ctx of 20480, does each request actually take only 2048 tokens?
Yeah, sure.
So, I use openweb UI with ollama.
Each model instance is set by parameters like n_ctx, while OLLAMA_NUM_PARALLEL is a shared parameter for all instances. If we take any two instances with n_ctx=A and n_ctx=B, then the actual context for each instance is calculated as: n_ctx / OLLAMA_NUM_PARALLEL
thannnnnnnnnnnnnnnnnnnks,you gave me a sense of why these two problems occurred:

When I set OLLAMA_NUM_PARALLEL=100, the response is only one sentence.

The GPU occupancy is constant all the time.

Now is there anything ollama can do to improve GPU usage? I changed these two parameters, but ollama still doesn't use more resources.

I'm glad I could help you out. 😊
From what I've practiced and observed:

It seems that Ollama dynamically regulates resource allocation.
Once multiple instances are loaded, the resources they occupy remain constant within the time specified by the keep_alive parameter.
Different instances of the same model may have varying resource usage. For example, Model A with num_ctx set to 1024 or 2048 will have different memory consumption.

@DirtyKnightForVi commented on GitHub (May 7, 2024): > > > > in my case, my prompt as follows:就我而言，我的提示如下： > > > > # DDL 数据定义语言 > > > > ```sql > > > > create table XXXXX > > > > (about 12000 tokens) > > > > ``` > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > # HINT 暗示 > > > > (about 500 tokens, notes, etc)（约500个代币、票据等） > > > > if OLLAMA_NUM_PARALLEL=2，model can not get table name。 ifOLLAMA_NUM_PARALLEL=1, The model's output can accurately and flawlessly recognize table names.如果OLLAMA_NUM_PARALLEL=2，模型无法获取表名。如果OLLAMA_NUM_PARALLEL=1，模型的输出可以准确无误地识别表名。 > > > > > > > > > Do you mean that n_ctx is shared for requests received in a uniform time period? For example, if I send 10 requests in 1s with an n_ctx of 20480, does each request actually take only 2048 tokens?您的意思是，对于在统一时间段内收到的请求，n_ctx 是共享的吗？例如，如果我在 1s 内发送 10 个请求，n_ctx 为 20480，那么每个请求实际上只需要 2048 个令牌吗？ > > > > > > Yeah, sure. 好，当然。 > > So, I use openweb UI with ollama.所以，我将 openweb UI 与 ollama 一起使用。 > > Each model instance is set by parameters like `n_ctx`, while `OLLAMA_NUM_PARALLEL` is a shared parameter for all instances. If we take any two instances with `n_ctx=A` and `n_ctx=B`, then the actual context for each instance is calculated as: `n_ctx / OLLAMA_NUM_PARALLEL`每个模型实例都由 `n_ctx` 等参数设置，而 `OLLAMA_NUM_PARALLEL` 是所有实例的共享参数。如果我们采用 `n_ctx=A` 和 `n_ctx=B` 的任意两个实例，则每个实例的实际上下文计算如下：`n_ctx / OLLAMA_NUM_PARALLEL`。 > > > > > in my case, my prompt as follows: > > > > # DDL > > > > ```sql > > > > create table XXXXX > > > > (about 12000 tokens) > > > > ``` > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > # HINT > > > > (about 500 tokens, notes, etc) > > > > if OLLAMA_NUM_PARALLEL=2，model can not get table name。 ifOLLAMA_NUM_PARALLEL=1, The model's output can accurately and flawlessly recognize table names. > > > > > > > > > Do you mean that n_ctx is shared for requests received in a uniform time period? For example, if I send 10 requests in 1s with an n_ctx of 20480, does each request actually take only 2048 tokens? > > > > > > Yeah, sure. > > So, I use openweb UI with ollama. > > Each model instance is set by parameters like `n_ctx`, while `OLLAMA_NUM_PARALLEL` is a shared parameter for all instances. If we take any two instances with `n_ctx=A` and `n_ctx=B`, then the actual context for each instance is calculated as: `n_ctx / OLLAMA_NUM_PARALLEL` > > thannnnnnnnnnnnnnnnnnnks,you gave me a sense of why these two problems occurred: > > 1. When I set OLLAMA_NUM_PARALLEL=100, the response is only one sentence. > 2. The GPU occupancy is constant all the time. > > Now is there anything ollama can do to improve GPU usage? I changed these two parameters, but ollama still doesn't use more resources. ![image](https://private-user-images.githubusercontent.com/73268510/328370928-148d7f87-9a55-450b-ad15-9b9e85f02291.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTUwNTEwMTQsIm5iZiI6MTcxNTA1MDcxNCwicGF0aCI6Ii83MzI2ODUxMC8zMjgzNzA5MjgtMTQ4ZDdmODctOWE1NS00NTBiLWFkMTUtOWI5ZTg1ZjAyMjkxLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA1MDclMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNTA3VDAyNTgzNFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWVlZTIxZGU0MjRmODEwODM2OTNiOTM2MTUzM2M3ZTAwNDUwOTYzOTA3OTI4ODc3Y2ExMzJlZDNhMzgzZjFjMzAmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.OMr1YJP8Dyp6zMJOEv2vOl9RY2w1wD_3hSIQJhdNYl0) I'm glad I could help you out. 😊 From what I've practiced and observed: 1. It seems that Ollama dynamically regulates resource allocation. 2. Once multiple instances are loaded, the resources they occupy remain constant within the time specified by the `keep_alive` parameter. 3. Different instances of the same model may have varying resource usage. For example, Model A with `num_ctx` set to 1024 or 2048 will have different memory consumption.

GiteaMirror commented

2026-04-22 06:28:47 -05:00

@JedrzejMajko commented on GitHub (Jul 3, 2025):

Ollama cannot operate one model many times . OLLAMA_NUM_PARALLEL do not apply how you think.
OLLAMA_NUM_PARALLEL just adds more memory, instead of reusing existing model loaded onto memory.

@JedrzejMajko commented on GitHub (Jul 3, 2025): Ollama cannot operate one model many times . OLLAMA_NUM_PARALLEL do not apply how you think. OLLAMA_NUM_PARALLEL just adds more memory, instead of reusing existing model loaded onto memory.

GiteaMirror commented

2026-04-22 06:28:49 -05:00

@mirage335 commented on GitHub (Jul 18, 2025):

I think the complexities of GPU shared memory, sparse layers, and detachable eGPU, will make it VERY difficult to avoid the need for many users to just force OLLAMA_NUM_PARALLEL=1 until consumer laptops and GPUs begin offering less unserious amounts of VRAM (ie. at least 24GB, maybe 32GB, instead of 16GB).

@mirage335 commented on GitHub (Jul 18, 2025): I think the complexities of GPU shared memory, sparse layers, and detachable eGPU, will make it VERY difficult to avoid the need for many users to just force OLLAMA_NUM_PARALLEL=1 until consumer laptops and GPUs begin offering less unserious amounts of VRAM (ie. at least 24GB, maybe 32GB, instead of 16GB).

GiteaMirror commented

2026-04-22 06:28:51 -05:00

@colinemondswieprecht commented on GitHub (Dec 22, 2025):

#11977 was just closed as a duplicate of this issue. @dhiltgen @jmorganca Is this issue here (#4170) even still being worked on? Nothing appears to have moved since this was closed as soon-to-be-completed in May 2024.

@colinemondswieprecht commented on GitHub (Dec 22, 2025): #11977 was just closed as a duplicate of this issue. @dhiltgen @jmorganca Is this issue here (#4170) even still being worked on? Nothing appears to have moved since this was closed as soon-to-be-completed in May 2024.

Sign in to join this conversation.

Branches Tags

main

parth-mlx-decode-checkpoints

dhiltgen/ci

hoyyeva/editor-config-repair

parth-launch-codex-app

hoyyeva/fix-codex-model-metadata-warning

hoyyeva/qwen

hoyyeva/launch-backup-ux

parth/hide-claude-desktop-till-release

hoyyeva/opencode-image-modality

parth-add-claude-code-autoinstall

release_v0.22.0

pdevine/manifest-list

codex/fix-codex-model-metadata-warning

pdevine/addressable-manifest

brucemacd/launch-fetch-reccomended

jmorganca/llama-compat

launch-copilot-cli

hoyyeva/opencode-thinking

release_v0.20.7

parth-auto-save-backup

parth-test

jmorganca/gemma4-audio-replacements

fix-manifest-digest-on-pull

hoyyeva/vscode-improve

brucemacd/install-server-wait

brucemacd/download-before-remove

parth/update-claude-docs

parth-anthropic-reference-images-path

brucemac/start-ap-install

pdevine/mlx-update

pdevine/qwen35_vision

drifkin/api-show-fallback

mintlify/image-generation-1773352582

hoyyeva/server-context-length-local-config

jmorganca/faster-reptition-penalties

jmorganca/convert-nemotron

parth-pi-thinking

pdevine/sampling-penalties

jmorganca/fix-create-quantization-memory

dongchen/resumable_transfer_fix

pdevine/sampling-cache-error

jessegross/mlx-usage

hoyyeva/openclaw-config

hoyyeva/app-html

pdevine/qwen3next

brucemacd/sign-sh-install

brucemacd/tui-update

brucemacd/usage-api

jmorganca/launch-empty

fix-app-dist-embed

mxyng/mlx-compile

mxyng/mlx-quant

mxyng/mlx-glm4.7

mxyng/mlx

brucemacd/simplify-model-picker

jmorganca/qwen3-concurrent

fix-glm-4.7-flash-mla-config

drifkin/qwen3-coder-opening-tag

brucemacd/usage-cli

fix-cuda12-fattn-shmem

ollama-imagegen-docs

parth/fix-multiline-inputs

brucemacd/config-docs

mxyng/model-files

mxyng/simple-execute

fix-imagegen-ollama-models

mxyng/async-upload

jmorganca/lazy-no-dtype-changes

imagegen-auto-detect-create

parth/decrease-concurrent-download-hf

fix-mlx-quantize-init

jmorganca/x-cleanup

usage

imagegen-readme

jmorganca/glm-image

mlx-gpu-cd

jmorganca/imagegen-modelfile

parth/agent-skills

parth/agent-allowlist

parth/signed-in-offline

parth/agents

parth/fix-context-chopping

improve-cloud-flow

parth/add-models-websearch

parth/prompt-renderer-mcp

jmorganca/native-settings

jmorganca/download-stream-hash

jmorganca/client2-rebased

brucemacd/oai-chat-req-multipart

jessegross/multi_chunk_reserve

grace/additional-omit-empty

grace/mistral-3-large

mxyng/tokenizer2

mxyng/tokenizer

jessegross/flash

hoyyeva/windows-nacked-app

mxyng/cleanup-attention

grace/deepseek-parser

hoyyeva/remember-unsent-prompt

parth/add-lfs-pointer-error-conversion

parth/olmo2-test2

hoyyeva/ollama-launchagent-plist

nicole/olmo-model

parth/olmo-test

mxyng/remove-embedded

parth/render-template

jmorganca/intellect-3

parth/remove-prealloc-linter

jmorganca/cmd-eval

nicole/nomic-embed-text-fix

mxyng/lint-2

hoyyeva/add-gemini-3-pro-preview

hoyyeva/load-model-list

mxyng/expand-path

mxyng/environ-2

hoyyeva/deeplink-json-encoding

parth/improve-tool-calling-tests

hoyyeva/conversation

hoyyeva/assistant-edit-response

hoyyeva/thinking

origin/brucemacd/invalid-char-i-err

parth/improve-tool-calling

jmorganca/required-omitempty

grace/qwen3-vl-tests

mxyng/iter-client

parth/docs-readme

nicole/embed-test

pdevine/integration-benchstat

parth/remove-generate-cmd

parth/add-toolcall-id

mxyng/server-tests

jmorganca/glm-4.6

jmorganca/gin-h-compat

drifkin/stable-tool-args

pdevine/qwen3-more-thinking

parth/add-websearch-client

nicole/websearch_local

jmorganca/qwen3-coder-updates

grace/deepseek-v3-migration-tests

mxyng/fix-create

jmorganca/cloud-errors

pdevine/parser-tidy

revert-12233-parth/simplify-entrypoints-runner

parth/enable-so-gpt-oss

brucemacd/qwen3vl

jmorganca/readme-simplify

parth/gpt-oss-structured-outputs

revert-12039-jmorganca/tools-braces

mxyng/embeddings

mxyng/gguf

mxyng/benchmark

mxyng/types-null

parth/move-parsing

mxyng/gemma2

jmorganca/docs

mxyng/16-bit

mxyng/create-stdin

pdevine/authorizedkeys

mxyng/quant

parth/opt-in-error-context-window

brucemacd/cache-models

brucemacd/runner-completion

jmorganca/llama-update-6

brucemacd/benchmark-list

brucemacd/partial-read-caps

parth/deepseek-r1-tools

mxyng/omit-array

parth/tool-prefix-temp

brucemacd/runner-test

jmorganca/qwen25vl

brucemacd/model-forward-test-ext

parth/python-function-parsing

jmorganca/cuda-compression-none

drifkin/num-parallel

drifkin/chat-truncation-fix

jmorganca/sync

parth/python-tools-calling

drifkin/array-head-count

brucemacd/create-no-loop

parth/server-enable-content-stream-with-tools

qwen25omni

mxyng/v3

brucemacd/ropeconfig

jmorganca/silence-tokenizer

parth/sample-so-test

parth/sampling-structured-outputs

brucemacd/doc-go-engine

parth/constrained-sampling-json

jmorganca/mistral-wip

brucemacd/mistral-small-convert

parth/sample-unmarshal-json-for-params

brucemacd/jomorganca/mistral

pdevine/bfloat16

jmorganca/mistral

brucemacd/mistral

pdevine/logging

parth/sample-correctness-fix

parth/sample-fix-sorting

jmorgan/sample-fix-sorting-extras

jmorganca/temp-0-images

brucemacd/parallel-embed-models

brucemacd/shim-grammar

jmorganca/fix-gguf-error

bmizerany/nameswork

jmorganca/faster-releases

bmizerany/validatenames

brucemacd/err-no-vocab

brucemacd/rope-config

brucemacd/err-hint

brucemacd/qwen2_5

brucemacd/logprobs

brucemacd/new_runner_graph_bench

progress-flicker

brucemacd/forward-test

brucemacd/go_qwen2

pdevine/gemma2

jmorganca/add-missing-symlink-eval

mxyng/next-debug

parth/set-context-size-openai

brucemacd/next-bpe-bench

brucemacd/next-bpe-test

brucemacd/new_runner_e2e

brucemacd/new_runner_qwen2

pdevine/convert-cohere2

brucemacd/convert-cli

parth/log-probs

mxyng/next-mlx

mxyng/cmd-history

parth/templating

parth/tokenize-detokenize

brucemacd/check-key-register

bmizerany/grammar

jmorganca/vendor-081b29bd

mxyng/func-checks

jmorganca/fix-null-format

parth/fix-default-to-warn-json

jmorganca/qwen2vl

jmorganca/no-concat

parth/cmd-cleanup-SO

brucemacd/check-key-register-structured-err

parth/openai-stream-usage

parth/fix-referencing-so

stream-tools-stop

jmorganca/degin-1

brucemacd/install-path-clean

brucemacd/push-name-validation

brucemacd/browser-key-register

jmorganca/openai-fix-first-message

jmorganca/fix-proxy

jessegross/sample

parth/disallow-streaming-tools

dhiltgen/remove_submodule

jmorganca/ga

jmorganca/mllama

pdevine/newlines

pdevine/geems-2b

jmorganca/llama-bump

mxyng/modelname-7

mxyng/gin-slog

mxyng/modelname-6

jyan/convert-prog

jyan/quant5

paligemma-support

pdevine/import-docs

jmorganca/openai-context

jyan/paligemma

jyan/p2

jyan/palitest

bmizerany/embedspeedup

jmorganca/llama-vit

brucemacd/allow-ollama

royh/ep-methods

royh/whisper

mxyng/api-models

mxyng/fix-memory

jyan/q4_4/8

jyan/ollama-v

royh/stream-tools

roy-embed-parallel

bmizerany/hrm

revert-5963-revert-5924-mxyng/llama3.1-rope

royh/embed-viz

jyan/local2

jyan/auth

jyan/local

jyan/parse-temp

jmorganca/template-mistral

jyan/reord-g

royh-openai-suffixdocs

royh-imgembed

royh-embed-parallel

jyan/quant4

royh-precision

jyan/progress

pdevine/fix-template

jyan/quant3

pdevine/ggla

mxyng/update-registry-domain

jmorganca/ggml-static

mxyng/create-context

jyan/v0.146

mxyng/layers-from-files

build_dist

bmizerany/noseek

royh-ls

royh-name

timeout

mxyng/server-timestamp

bmizerany/nosillyggufslurps

royh-params

jmorganca/llama-cpp-7c26775

royh-openai-delete

royh-show-rigid

jmorganca/enable-fa

jmorganca/no-error-template

jyan/format

royh-testdelete

bmizerany/fastverify

language_support

pdevine/ps-glitches

brucemacd/tokenize

bruce/iq-quants

bmizerany/filepathwithcoloninhost

mxyng/split-bin

bmizerany/client-registry

jmorganca/if-none-match

native

jmorganca/native

jmorganca/batch-embeddings

jmorganca/initcmake

jmorganca/mm

pdevine/showggmlinfo

modenameenforcealphanum

bmizerany/modenameenforcealphanum

jmorganca/done-reason

jmorganca/llama-cpp-8960fe8

ollama.com

bmizerany/filepathnobuild

bmizerany/types/model/defaultfix

rmdisplaylong

nogogen

bmizerany/x

modelfile-readme

bmizerany/replacecolon

jmorganca/limit

jmorganca/execstack

jmorganca/replace-assets

mxyng/tune-concurrency

jmorganca/testing

whitespace-detection

jmorganca/options

upgrade-all

scratch

cuda-search

mattw/airenamer

mattw/allmodelsonhuggingface

mattw/quantcontext

mattw/whatneedstorun

brucemacd/llama-mem-calc

mattw/faq-context

mattw/communitylinks

mattw/noprune

mattw/python-functioncalling

rename

mxyng/install

pulse

remove-first

editor

mattw/selfqueryingretrieval

cgo

mattw/howtoquant

api

matt/streamingapi

format-config

mxyng/extra-args

shell

update-nous-hermes

cp-model

upload-progress

fix-unknown-model

fix-model-names

delete-fix

insecure-registry

ls

deletemodels

progressbar

readme-updates

license-layers

skip-list

list-models

modelpath

matt/examplemodelfiles

distribution

go-opts

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/ollama#28352