[GH-ISSUE #4912] Error: llama runner process has terminated: signal: aborted (core dumped) #3102

New Issue

GiteaMirror · 2026-04-12T13:32:36-05:00

GiteaMirror commented

2026-04-12 13:32:36 -05:00

Originally created by @mikestut on GitHub (Jun 7, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/4912

What is the issue?

When I run the CLI ollama run qwen2:72b-instruct-q2_K
then download the model and run the model。
rError: llama runner process has terminated: signal: aborted (core dumped)

OS

Linux

GPU

Nvidia

CPU

Intel

Ollama version

0.1.38

Originally created by @mikestut on GitHub (Jun 7, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/4912 ### What is the issue? When I run the CLI ollama run qwen2:72b-instruct-q2_K then download the model and run the model。 rError: llama runner process has terminated: signal: aborted (core dumped) ### OS Linux ### GPU Nvidia ### CPU Intel ### Ollama version 0.1.38

GiteaMirror added the bug label 2026-04-12 13:32:36 -05:00

GiteaMirror closed this issue

2026-04-12 13:32:37 -05:00

GiteaMirror commented

2026-04-12 13:32:38 -05:00

@informaticker commented on GitHub (Jun 7, 2024):

Upgrade your ollama using

curl -fsSL https://ollama.com/install.sh | sh

Should fix the issue, although with Qwen2 there's still a bug present which causes bugged output.

@informaticker commented on GitHub (Jun 7, 2024): Upgrade your ollama using ```bash curl -fsSL https://ollama.com/install.sh | sh ``` Should fix the issue, although with Qwen2 there's still a bug present which causes bugged output.

GiteaMirror commented

2026-04-12 13:32:38 -05:00

@mikestut commented on GitHub (Jun 7, 2024):

Thanks,update ollama newest version but the model still break work yet!!!

---Original---
From: @.>
Date: Sat, Jun 8, 2024 00:49 AM
To: @.>;
Cc: @.@.>;
Subject: Re: [ollama/ollama] Error: llama runner process has terminated:signal: aborted (core dumped) (Issue #4912)

Upgrade your ollama using
curl -fsSL https://ollama.com/install.sh | sh

Should fix the issue, although with Qwen2 there's still a bug present which causes bugged output.

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

@mikestut commented on GitHub (Jun 7, 2024): Thanks,update ollama newest version but the model still break work yet!!! ---Original--- From: ***@***.***> Date: Sat, Jun 8, 2024 00:49 AM To: ***@***.***>; Cc: ***@***.******@***.***>; Subject: Re: [ollama/ollama] Error: llama runner process has terminated:signal: aborted (core dumped) (Issue #4912) Upgrade your ollama using curl -fsSL https://ollama.com/install.sh | sh Should fix the issue, although with Qwen2 there's still a bug present which causes bugged output. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

GiteaMirror commented

2026-04-12 13:32:39 -05:00

@TonyHuang6666 commented on GitHub (Jun 7, 2024):

Bro, I am having the same problem too. My devices are AMD Ryzen 5700x +Nvidia RTX 4070 super 12GB + 64 GB RAM. Everytime I run models that are bigger than the VRAM in WSL2 or Ubuntu 22.04, the same problem as yours would occur, because Ollama fails to offload the model to RAM for some reason which shouldn't have happened. Once I switch to Windows, Ollama would run models using both CPU and GPU.

@TonyHuang6666 commented on GitHub (Jun 7, 2024): Bro, I am having the same problem too. My devices are AMD Ryzen 5700x +Nvidia RTX 4070 super 12GB + 64 GB RAM. Everytime I run models that are bigger than the VRAM in WSL2 or Ubuntu 22.04, the same problem as yours would occur, because Ollama fails to offload the model to RAM for some reason which shouldn't have happened. Once I switch to Windows, Ollama would run models using both CPU and GPU.

GiteaMirror commented

2026-04-12 13:32:40 -05:00

@mikestut commented on GitHub (Jun 8, 2024):

Bro, I am having the same problem too. My devices are AMD Ryzen 5700x +Nvidia RTX 4070 super 12GB + 64 GB RAM. Everytime I run models that are bigger than the VRAM in WSL2 or Ubuntu 22.04, the same problem as yours would occur, because Ollama fails to offload the model to RAM for some reason which shouldn't have happened. Once I switch to Windows, Ollama would run models using both CPU and GPU.

The reason is that Ubuntu 22.04 does not support Ollama running models using both CPU and GPU simultaneously?

@mikestut commented on GitHub (Jun 8, 2024): > Bro, I am having the same problem too. My devices are AMD Ryzen 5700x +Nvidia RTX 4070 super 12GB + 64 GB RAM. Everytime I run models that are bigger than the VRAM in WSL2 or Ubuntu 22.04, the same problem as yours would occur, because Ollama fails to offload the model to RAM for some reason which shouldn't have happened. Once I switch to Windows, Ollama would run models using both CPU and GPU. The reason is that Ubuntu 22.04 does not support Ollama running models using both CPU and GPU simultaneously?

GiteaMirror commented

2026-04-12 13:32:41 -05:00

@TonyHuang6666 commented on GitHub (Jun 8, 2024):

I found out why. You need to install Ollama as a startup service following
After doing so, Ollama could use both CPU and GPU on both WSL2 and other physical Linux devices

@TonyHuang6666 commented on GitHub (Jun 8, 2024): I found out why. You need to install Ollama as a startup service following![mmexport1717820606366.png](https://github.com/ollama/ollama/assets/87429109/9804bb80-7036-4803-901a-1114af63b91c) After doing so, Ollama could use both CPU and GPU on both WSL2 and other physical Linux devices

GiteaMirror commented

2026-04-12 13:32:43 -05:00

@mikestut commented on GitHub (Jun 8, 2024):

I found out why. You need to install Ollama as a startup service following After doing so, Ollama could use both CPU and GPU on both WSL2 and other physical Linux devices

I try this but the problem still here.(base) root@VenuePro:/home/tyn# ollama run qwen2:72b-instruct-q2_K
Error: llama runner process has terminated: signal: aborted (core dumped)

I save the file to /home/
paht:/home/ollama serve

The config :
[Unit]
Description=Ollama Service
After=network-online.target

[Service]
#ExecStart=/usr/local/bin/ollama serve
ExecStart=/home/ollama serve
User=ollama
Group=ollama
Restart=always
RestartSec=3
Environment="PATH=/usr/local/cuda/bin:/root/miniconda3/bin:/root/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin"
Environment="OLLAMA_HOST=0.0.0.0:11434"
[Install]
WantedBy=default.target

@mikestut commented on GitHub (Jun 8, 2024): > I found out why. You need to install Ollama as a startup service following![mmexport1717820606366.png](https://private-user-images.githubusercontent.com/87429109/337832305-9804bb80-7036-4803-901a-1114af63b91c.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTc4MjE0MDAsIm5iZiI6MTcxNzgyMTEwMCwicGF0aCI6Ii84NzQyOTEwOS8zMzc4MzIzMDUtOTgwNGJiODAtNzAzNi00ODAzLTkwMWEtMTExNGFmNjNiOTFjLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA2MDglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNjA4VDA0MzE0MFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWIxNGU5ZDAzNmJlYTUyNzExNTIyODZlNjI3ZGU4ODQyNzkyNDdkYzc0YWRjZDU2YmE5OTMyNTZjMGY4ZmJjMzImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.d9YaBV0VYA_Mj3vjVBZUz9RmIRB5LLHr2_l_9BavSWI) After doing so, Ollama could use both CPU and GPU on both WSL2 and other physical Linux devices I try this but the problem still here.(base) root@VenuePro:/home/tyn# ollama run qwen2:72b-instruct-q2_K Error: llama runner process has terminated: signal: aborted (core dumped) I save the file to /home/ paht:/home/ollama serve The config : [Unit] Description=Ollama Service After=network-online.target [Service] #ExecStart=/usr/local/bin/ollama serve ExecStart=/home/ollama serve User=ollama Group=ollama Restart=always RestartSec=3 Environment="PATH=/usr/local/cuda/bin:/root/miniconda3/bin:/root/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" Environment="OLLAMA_HOST=0.0.0.0:11434" [Install] WantedBy=default.target

GiteaMirror commented

2026-04-12 13:32:45 -05:00

@TonyHuang6666 commented on GitHub (Jun 8, 2024):

I'm sorry to hear that. It seems that we don't encounter the same problem.😢

@TonyHuang6666 commented on GitHub (Jun 8, 2024): I'm sorry to hear that. It seems that we don't encounter the same problem.😢

GiteaMirror commented

2026-04-12 13:32:48 -05:00

@igorschlum commented on GitHub (Jun 8, 2024):

Can you all try with version 0.1.42 of Ollama? Thanks

@igorschlum commented on GitHub (Jun 8, 2024): Can you all try with version 0.1.42 of Ollama? Thanks

GiteaMirror commented

2026-04-12 13:32:49 -05:00

@TonyHuang6666 commented on GitHub (Jun 8, 2024):

Can you all try with version 0.1.42 of Ollama? Thanks

I've tried it on Windows 11, WSL2 and Ubuntu 22.04 LTS. Nothing bad has happened yet.

@TonyHuang6666 commented on GitHub (Jun 8, 2024): > Can you all try with version 0.1.42 of Ollama? Thanks I've tried it on Windows 11, WSL2 and Ubuntu 22.04 LTS. Nothing bad has happened yet.

GiteaMirror commented

2026-04-12 13:32:49 -05:00

@TonyHuang6666 commented on GitHub (Jun 8, 2024):

I found out why. You need to install Ollama as a startup service following After doing so, Ollama could use both CPU and GPU on both WSL2 and other physical Linux devices

I try this but the problem still here.(base) root@VenuePro:/home/tyn# ollama run qwen2:72b-instruct-q2_K
Error: llama runner process has terminated: signal: aborted (core dumped)

I save the file to /home/
paht:/home/ollama serve

The config :
[Unit]
Description=Ollama Service
After=network-online.target

[Service]
#ExecStart=/usr/local/bin/ollama serve
ExecStart=/home/ollama serve
User=ollama
Group=ollama
Restart=always
RestartSec=3
Environment="PATH=/usr/local/cuda/bin:/root/miniconda3/bin:/root/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin"
Environment="OLLAMA_HOST=0.0.0.0:11434"
[Install]
WantedBy=default.target

Your log doesn't contain the key point about why the program aborted. If you don't install Ollama as a startup service, you need to type "ollama serve" in the Linux command console to start it. This console will then display detailed information about the running process. If you open another console to run a model, such as "ollama run qwen2," the output, whether successful or not, will be printed in the initial console. In my case, the console where I attempted to run a model simply stated "core dumped," which is vague and ambiguous. Meanwhile, the console where I started Ollama provided detailed information about insufficient VRAM, indicating that the process didn't utilize both the CPU and GPU.

Given that the model is much larger than my VRAM, if it fails to offload some parts to RAM and call the CPU to handle them, the whole program would definitely abort. Therefore, I suggest you uninstall Ollama completely following the official documentation, then install the latest release (without installing it as a startup service). After that, open two consoles as described above to monitor the process. You can also monitor VRAM and RAM usage to see if Ollama fails to offload the model to both VRAM and RAM. I believe you will identify the reason then.

@TonyHuang6666 commented on GitHub (Jun 8, 2024): > > I found out why. You need to install Ollama as a startup service following![mmexport1717820606366.png](https://private-user-images.githubusercontent.com/87429109/337832305-9804bb80-7036-4803-901a-1114af63b91c.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTc4MjE0MDAsIm5iZiI6MTcxNzgyMTEwMCwicGF0aCI6Ii84NzQyOTEwOS8zMzc4MzIzMDUtOTgwNGJiODAtNzAzNi00ODAzLTkwMWEtMTExNGFmNjNiOTFjLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA2MDglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNjA4VDA0MzE0MFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWIxNGU5ZDAzNmJlYTUyNzExNTIyODZlNjI3ZGU4ODQyNzkyNDdkYzc0YWRjZDU2YmE5OTMyNTZjMGY4ZmJjMzImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.d9YaBV0VYA_Mj3vjVBZUz9RmIRB5LLHr2_l_9BavSWI) After doing so, Ollama could use both CPU and GPU on both WSL2 and other physical Linux devices > > I try this but the problem still here.(base) root@VenuePro:/home/tyn# ollama run qwen2:72b-instruct-q2_K > Error: llama runner process has terminated: signal: aborted (core dumped) > > I save the file to /home/ > paht:/home/ollama serve > > The config : > [Unit] > Description=Ollama Service > After=network-online.target > > [Service] > #ExecStart=/usr/local/bin/ollama serve > ExecStart=/home/ollama serve > User=ollama > Group=ollama > Restart=always > RestartSec=3 > Environment="PATH=/usr/local/cuda/bin:/root/miniconda3/bin:/root/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" > Environment="OLLAMA_HOST=0.0.0.0:11434" > [Install] > WantedBy=default.target > > Your log doesn't contain the key point about why the program aborted. If you don't install Ollama as a startup service, you need to type "ollama serve" in the Linux command console to start it. This console will then display detailed information about the running process. If you open another console to run a model, such as "ollama run qwen2," the output, whether successful or not, will be printed in the initial console. In my case, the console where I attempted to run a model simply stated "core dumped," which is vague and ambiguous. Meanwhile, the console where I started Ollama provided detailed information about insufficient VRAM, indicating that the process didn't utilize both the CPU and GPU. Given that the model is much larger than my VRAM, if it fails to offload some parts to RAM and call the CPU to handle them, the whole program would definitely abort. Therefore, I suggest you uninstall Ollama completely following the official documentation, then install the latest release (without installing it as a startup service). After that, open two consoles as described above to monitor the process. You can also monitor VRAM and RAM usage to see if Ollama fails to offload the model to both VRAM and RAM. I believe you will identify the reason then.

GiteaMirror commented

2026-04-12 13:32:50 -05:00

@jmorganca commented on GitHub (Jun 9, 2024):

This should be fixed on 0.1.42. Let me know if that's not the case - thanks for the issue!

@jmorganca commented on GitHub (Jun 9, 2024): This should be fixed on 0.1.42. Let me know if that's not the case - thanks for the issue!

GiteaMirror commented

2026-04-12 13:32:50 -05:00

@mikestut commented on GitHub (Jun 10, 2024):

I found out why. You need to install Ollama as a startup service following After doing so, Ollama could use both CPU and GPU on both WSL2 and other physical Linux devices

I try this but the problem still here.(base) root@VenuePro:/home/tyn# ollama run qwen2:72b-instruct-q2_K
Error: llama runner process has terminated: signal: aborted (core dumped)
I save the file to /home/
paht:/home/ollama serve
The config :
[Unit]
Description=Ollama Service
After=network-online.target
[Service]
#ExecStart=/usr/local/bin/ollama serve
ExecStart=/home/ollama serve
User=ollama
Group=ollama
Restart=always
RestartSec=3
Environment="PATH=/usr/local/cuda/bin:/root/miniconda3/bin:/root/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin"
Environment="OLLAMA_HOST=0.0.0.0:11434"
[Install]
WantedBy=default.target

Your log doesn't contain the key point about why the program aborted. If you don't install Ollama as a startup service, you need to type "ollama serve" in the Linux command console to start it. This console will then display detailed information about the running process. If you open another console to run a model, such as "ollama run qwen2," the output, whether successful or not, will be printed in the initial console. In my case, the console where I attempted to run a model simply stated "core dumped," which is vague and ambiguous. Meanwhile, the console where I started Ollama provided detailed information about insufficient VRAM, indicating that the process didn't utilize both the CPU and GPU.

Given that the model is much larger than my VRAM, if it fails to offload some parts to RAM and call the CPU to handle them, the whole program would definitely abort. Therefore, I suggest you uninstall Ollama completely following the official documentation, then install the latest release (without installing it as a startup service). After that, open two consoles as described above to monitor the process. You can also monitor VRAM and RAM usage to see if Ollama fails to offload the model to both VRAM and RAM. I believe you will identify the reason then.

THANKS all Yours!!!
I retry again as follow yours but the GPU load all RAM

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 Tesla V100-PCIE-16GB Off | 00000000:02:00.0 Off | 0 |
| N/A 48C P0 42W / 250W | 8610MiB / 16384MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 1 Tesla V100-PCIE-16GB Off | 00000000:03:00.0 Off | 0 |
| N/A 42C P0 36W / 250W | 8974MiB / 16384MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 2 Tesla V100-PCIE-16GB Off | 00000000:81:00.0 Off | 0 |
| N/A 50C P0 39W / 250W | 8628MiB / 16384MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 3 Tesla V100-PCIE-16GB Off | 00000000:82:00.0 Off | 0 |
| N/A 47C P0 122W / 250W | 8628MiB / 16384MiB | 87% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+

but the end!!!

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 Tesla V100-PCIE-16GB Off | 00000000:02:00.0 Off | 0 |
| N/A 48C P0 31W / 250W | 118MiB / 16384MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 1 Tesla V100-PCIE-16GB Off | 00000000:03:00.0 Off | 0 |
| N/A 42C P0 37W / 250W | 6MiB / 16384MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 2 Tesla V100-PCIE-16GB Off | 00000000:81:00.0 Off | 0 |
| N/A 50C P0 34W / 250W | 6MiB / 16384MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 3 Tesla V100-PCIE-16GB Off | 00000000:82:00.0 Off | 0 |
| N/A 46C P0 32W / 250W | 6MiB / 16384MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+

AND

Error: llama runner process has terminated: signal: aborted (core dumped)

@mikestut commented on GitHub (Jun 10, 2024): > > > I found out why. You need to install Ollama as a startup service following![mmexport1717820606366.png](https://private-user-images.githubusercontent.com/87429109/337832305-9804bb80-7036-4803-901a-1114af63b91c.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTc4MjE0MDAsIm5iZiI6MTcxNzgyMTEwMCwicGF0aCI6Ii84NzQyOTEwOS8zMzc4MzIzMDUtOTgwNGJiODAtNzAzNi00ODAzLTkwMWEtMTExNGFmNjNiOTFjLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA2MDglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNjA4VDA0MzE0MFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWIxNGU5ZDAzNmJlYTUyNzExNTIyODZlNjI3ZGU4ODQyNzkyNDdkYzc0YWRjZDU2YmE5OTMyNTZjMGY4ZmJjMzImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.d9YaBV0VYA_Mj3vjVBZUz9RmIRB5LLHr2_l_9BavSWI) After doing so, Ollama could use both CPU and GPU on both WSL2 and other physical Linux devices > > > > > > I try this but the problem still here.(base) root@VenuePro:/home/tyn# ollama run qwen2:72b-instruct-q2_K > > Error: llama runner process has terminated: signal: aborted (core dumped) > > I save the file to /home/ > > paht:/home/ollama serve > > The config : > > [Unit] > > Description=Ollama Service > > After=network-online.target > > [Service] > > #ExecStart=/usr/local/bin/ollama serve > > ExecStart=/home/ollama serve > > User=ollama > > Group=ollama > > Restart=always > > RestartSec=3 > > Environment="PATH=/usr/local/cuda/bin:/root/miniconda3/bin:/root/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" > > Environment="OLLAMA_HOST=0.0.0.0:11434" > > [Install] > > WantedBy=default.target > > Your log doesn't contain the key point about why the program aborted. If you don't install Ollama as a startup service, you need to type "ollama serve" in the Linux command console to start it. This console will then display detailed information about the running process. If you open another console to run a model, such as "ollama run qwen2," the output, whether successful or not, will be printed in the initial console. In my case, the console where I attempted to run a model simply stated "core dumped," which is vague and ambiguous. Meanwhile, the console where I started Ollama provided detailed information about insufficient VRAM, indicating that the process didn't utilize both the CPU and GPU. > > Given that the model is much larger than my VRAM, if it fails to offload some parts to RAM and call the CPU to handle them, the whole program would definitely abort. Therefore, I suggest you uninstall Ollama completely following the official documentation, then install the latest release (without installing it as a startup service). After that, open two consoles as described above to monitor the process. You can also monitor VRAM and RAM usage to see if Ollama fails to offload the model to both VRAM and RAM. I believe you will identify the reason then. THANKS all Yours!!! I retry again as follow yours but the GPU load all RAM +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | |-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 Tesla V100-PCIE-16GB Off | 00000000:02:00.0 Off | 0 | | N/A 48C P0 42W / 250W | 8610MiB / 16384MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ | 1 Tesla V100-PCIE-16GB Off | 00000000:03:00.0 Off | 0 | | N/A 42C P0 36W / 250W | 8974MiB / 16384MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ | 2 Tesla V100-PCIE-16GB Off | 00000000:81:00.0 Off | 0 | | N/A 50C P0 39W / 250W | 8628MiB / 16384MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ | 3 Tesla V100-PCIE-16GB Off | 00000000:82:00.0 Off | 0 | | N/A 47C P0 122W / 250W | 8628MiB / 16384MiB | 87% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ but the end!!! +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | |-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 Tesla V100-PCIE-16GB Off | 00000000:02:00.0 Off | 0 | | N/A 48C P0 31W / 250W | 118MiB / 16384MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ | 1 Tesla V100-PCIE-16GB Off | 00000000:03:00.0 Off | 0 | | N/A 42C P0 37W / 250W | 6MiB / 16384MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ | 2 Tesla V100-PCIE-16GB Off | 00000000:81:00.0 Off | 0 | | N/A 50C P0 34W / 250W | 6MiB / 16384MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ | 3 Tesla V100-PCIE-16GB Off | 00000000:82:00.0 Off | 0 | | N/A 46C P0 32W / 250W | 6MiB / 16384MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ AND Error: llama runner process has terminated: signal: aborted (core dumped)

GiteaMirror commented

2026-04-12 13:32:51 -05:00

@mikestut commented on GitHub (Jun 10, 2024):

This should be fixed on 0.1.42. Let me know if that's not the case - thanks for the issue!

I upgraded the ollama version, but the problem still exists.

(base) root@Venue-vPro:/home/tyn# ollama run qwen2:72b-instruct-q2_K
Error: llama runner process has terminated: signal: aborted (core dumped)
(base) root@Venue-vPro:/home/tyn# ollama --version
ollama version is 0.1.42

The GPU memory loading status is recorded:
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 Tesla V100-PCIE-16GB Off | 00000000:02:00.0 Off | 0 |
| N/A 48C P0 42W / 250W | 8610MiB / 16384MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 1 Tesla V100-PCIE-16GB Off | 00000000:03:00.0 Off | 0 |
| N/A 42C P0 36W / 250W | 8974MiB / 16384MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 2 Tesla V100-PCIE-16GB Off | 00000000:81:00.0 Off | 0 |
| N/A 50C P0 39W / 250W | 8628MiB / 16384MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 3 Tesla V100-PCIE-16GB Off | 00000000:82:00.0 Off | 0 |
| N/A 47C P0 122W / 250W | 8628MiB / 16384MiB | 87% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+

1 second then

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 Tesla V100-PCIE-16GB Off | 00000000:02:00.0 Off | 0 |
| N/A 48C P0 31W / 250W | 118MiB / 16384MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 1 Tesla V100-PCIE-16GB Off | 00000000:03:00.0 Off | 0 |
| N/A 42C P0 37W / 250W | 6MiB / 16384MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 2 Tesla V100-PCIE-16GB Off | 00000000:81:00.0 Off | 0 |
| N/A 50C P0 34W / 250W | 6MiB / 16384MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 3 Tesla V100-PCIE-16GB Off | 00000000:82:00.0 Off | 0 |
| N/A 46C P0 32W / 250W | 6MiB / 16384MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
The model file is not downloaded from scratch. Do I need to download it again with the latest version?

GiteaMirror commented

2026-04-12 13:32:52 -05:00

@igorschlum commented on GitHub (Jun 11, 2024):

Hi @mikestut I'm on MacOS with 32 GB of RAM.

I could run qwen2:72b-instruct-q2-K it was very very slow (like hours to answer, but no crash).

Last login: Mon Jun 10 08:34:10 on ttys008
(base) igor@macIgor-2 ~ % ollama run qwen2:72b-instruct-q2_K

how can you help me?
As a large language model, I can provide assistance in various ways. Here
are some examples:

Answering questions: If you have any specific questions or need
information about a certain topic, feel free to ask and I will try my best
to provide accurate and relevant answers based on the data I've been
trained on.
Language translation: I can help translate sentences or phrases from
one language to another. Just let me know which languages you need
translated, and I'll do my best to assist you.
Writing assistance: Whether it's writing an essay, a story, or any
other type of text, I can provide suggestions for structure, grammar,
style, and even generate content based on your prompts.
Creative brainstorming: If you're stuck on ideas for a project, need
inspiration, or want to explore different perspectives, I can help by
suggesting creative concepts or offering alternative viewpoints.
Learning new skills: You can ask me questions about various topics to
learn something new or deepen your understanding of specific subjects.
Emotional support: While I am not a human and cannot provide the same
level of emotional support as a person, you can share your feelings with
me if you need someone to talk to. Remember that it's always important to
reach out to friends, family members, or professionals for more
personalized help in challenging situations.

Please feel free to let me know how I can assist you further!

@igorschlum commented on GitHub (Jun 11, 2024): Hi @mikestut I'm on MacOS with 32 GB of RAM. I could run qwen2:72b-instruct-q2-K it was very very slow (like hours to answer, but no crash). Last login: Mon Jun 10 08:34:10 on ttys008 (base) igor@macIgor-2 ~ % ollama run qwen2:72b-instruct-q2_K >>> how can you help me? As a large language model, I can provide assistance in various ways. Here are some examples: 1. Answering questions: If you have any specific questions or need information about a certain topic, feel free to ask and I will try my best to provide accurate and relevant answers based on the data I've been trained on. 2. Language translation: I can help translate sentences or phrases from one language to another. Just let me know which languages you need translated, and I'll do my best to assist you. 3. Writing assistance: Whether it's writing an essay, a story, or any other type of text, I can provide suggestions for structure, grammar, style, and even generate content based on your prompts. 4. Creative brainstorming: If you're stuck on ideas for a project, need inspiration, or want to explore different perspectives, I can help by suggesting creative concepts or offering alternative viewpoints. 5. Learning new skills: You can ask me questions about various topics to learn something new or deepen your understanding of specific subjects. 6. Emotional support: While I am not a human and cannot provide the same level of emotional support as a person, you can share your feelings with me if you need someone to talk to. Remember that it's always important to reach out to friends, family members, or professionals for more personalized help in challenging situations. Please feel free to let me know how I can assist you further!

GiteaMirror commented

2026-04-12 13:32:52 -05:00

@mikestut commented on GitHub (Jun 11, 2024):

I run the model on Ubuntu with ollama,so I Can't help you about your question.Maybe You can ask other guys.

---Original---
From: "Igor @.>
Date: Tue, Jun 11, 2024 13:47 PM
To: @.>;
Cc: @.@.>;
Subject: Re: [ollama/ollama] Error: llama runner process has terminated:signal: aborted (core dumped) (Issue #4912)

Hi @mikestut I'm on MacOS with 32 GB of RAM.

I could run qwen2:72b-instruct-q2-K it was very very slow (like hours to answer, but no crash).

Last login: Mon Jun 10 08:34:10 on ttys008
(base) @.*** ~ % ollama run qwen2:72b-instruct-q2_K

how can you help me?
As a large language model, I can provide assistance in various ways. Here
are some examples:

Answering questions: If you have any specific questions or need
information about a certain topic, feel free to ask and I will try my best
to provide accurate and relevant answers based on the data I've been
trained on.

Language translation: I can help translate sentences or phrases from
one language to another. Just let me know which languages you need
translated, and I'll do my best to assist you.

Writing assistance: Whether it's writing an essay, a story, or any
other type of text, I can provide suggestions for structure, grammar,
style, and even generate content based on your prompts.

Creative brainstorming: If you're stuck on ideas for a project, need
inspiration, or want to explore different perspectives, I can help by
suggesting creative concepts or offering alternative viewpoints.

Learning new skills: You can ask me questions about various topics to
learn something new or deepen your understanding of specific subjects.

Emotional support: While I am not a human and cannot provide the same
level of emotional support as a person, you can share your feelings with
me if you need someone to talk to. Remember that it's always important to
reach out to friends, family members, or professionals for more
personalized help in challenging situations.

Please feel free to let me know how I can assist you further!

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: @.***>

@mikestut commented on GitHub (Jun 11, 2024): I run the model on Ubuntu with ollama,so I Can't help you about your question.Maybe You can ask other guys. ---Original--- From: "Igor ***@***.***> Date: Tue, Jun 11, 2024 13:47 PM To: ***@***.***>; Cc: ***@***.******@***.***>; Subject: Re: [ollama/ollama] Error: llama runner process has terminated:signal: aborted (core dumped) (Issue #4912) Hi @mikestut I'm on MacOS with 32 GB of RAM. I could run qwen2:72b-instruct-q2-K it was very very slow (like hours to answer, but no crash). Last login: Mon Jun 10 08:34:10 on ttys008 (base) ***@***.*** ~ % ollama run qwen2:72b-instruct-q2_K how can you help me? As a large language model, I can provide assistance in various ways. Here are some examples: Answering questions: If you have any specific questions or need information about a certain topic, feel free to ask and I will try my best to provide accurate and relevant answers based on the data I've been trained on. Language translation: I can help translate sentences or phrases from one language to another. Just let me know which languages you need translated, and I'll do my best to assist you. Writing assistance: Whether it's writing an essay, a story, or any other type of text, I can provide suggestions for structure, grammar, style, and even generate content based on your prompts. Creative brainstorming: If you're stuck on ideas for a project, need inspiration, or want to explore different perspectives, I can help by suggesting creative concepts or offering alternative viewpoints. Learning new skills: You can ask me questions about various topics to learn something new or deepen your understanding of specific subjects. Emotional support: While I am not a human and cannot provide the same level of emotional support as a person, you can share your feelings with me if you need someone to talk to. Remember that it's always important to reach out to friends, family members, or professionals for more personalized help in challenging situations. Please feel free to let me know how I can assist you further! — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

GiteaMirror commented

2026-04-12 13:32:53 -05:00

@VirtualZardoz commented on GitHub (Jun 12, 2024):

On Windows 11, I have the same issue. I tried with several different Qwen2 models and quantizations. None worked:

PS C:\Users\Shahram> ollama run qwen2:7b-instruct-q8_0
Error: llama runner process has terminated: signal: aborted
PS C:\Users\Shahram> Ollama: 500, message='Internal Server Error', url=URL('http://host.docker.internal:11434/api/chat')

Nvidia RTX 3090
Windows 11
Ollama 0.1.42 or 0.1.38, I never know:

PS C:\Users\Shahram> ollama -v
ollama version is 0.1.38
Warning: client version is 0.1.42

@VirtualZardoz commented on GitHub (Jun 12, 2024): On Windows 11, I have the same issue. I tried with several different Qwen2 models and quantizations. None worked: ``` PS C:\Users\Shahram> ollama run qwen2:7b-instruct-q8_0 Error: llama runner process has terminated: signal: aborted PS C:\Users\Shahram> Ollama: 500, message='Internal Server Error', url=URL('http://host.docker.internal:11434/api/chat') ``` Nvidia RTX 3090 Windows 11 Ollama 0.1.42 or 0.1.38, I never know: ``` PS C:\Users\Shahram> ollama -v ollama version is 0.1.38 Warning: client version is 0.1.42 ```

GiteaMirror commented

2026-04-12 13:32:53 -05:00

@siavashmohammady66 commented on GitHub (Jun 14, 2024):

I have the same problem in the Manjaro Linux, my ollama version is 44 & qwen2:7b is not working with the same error

@siavashmohammady66 commented on GitHub (Jun 14, 2024): I have the same problem in the Manjaro Linux, my ollama version is 44 & qwen2:7b is not working with the same error

GiteaMirror commented

2026-04-12 13:32:54 -05:00

@igorschlum commented on GitHub (Jun 14, 2024):

@siavashmohammady66 can you share the logs of ollama running on your computer?

@igorschlum commented on GitHub (Jun 14, 2024): @siavashmohammady66 can you share the logs of ollama running on your computer?

GiteaMirror commented

2026-04-12 13:32:55 -05:00

@siavashmohammady66 commented on GitHub (Jun 16, 2024):

@siavashmohammady66 can you share the logs of ollama running on your computer?

Thank you for your response, how ever issue is solved after restarting my computer

@siavashmohammady66 commented on GitHub (Jun 16, 2024): > @siavashmohammady66 can you share the logs of ollama running on your computer? Thank you for your response, how ever issue is solved after restarting my computer

GiteaMirror commented

2026-04-12 13:32:55 -05:00

@CharlesHehe commented on GitHub (Jun 19, 2024):

curl -fsSL https://ollama.com/install.sh | sh

fixed my issue.

ubuntu 20.04
Nvidia RTX 3080

@CharlesHehe commented on GitHub (Jun 19, 2024): `curl -fsSL https://ollama.com/install.sh | sh` fixed my issue. ubuntu 20.04 Nvidia RTX 3080

GiteaMirror commented

2026-04-12 13:32:56 -05:00

@EduardDoronin commented on GitHub (Jun 19, 2024):

We are running Ubuntu as well and have our ollama version on 0.1.44. We used to get this error:

Error: llama runner process has terminated: signal: aborted (core dumped)

but after running curl -fsSL https://ollama.com/install.sh | sh

I started getting this error:

Error: llama runner process has terminated: signal: aborted (core dumped) error:failed to create context with model '/usr/share/ollama/.ollama/models/blobs/sha256-5ff0abeeac1d2dbdd5455c0b49ba3b29a9ce3c1fb181b2eef2e948689d55d046'

Any idea/fixes?

@EduardDoronin commented on GitHub (Jun 19, 2024): We are running Ubuntu as well and have our ollama version on 0.1.44. We used to get this error: Error: llama runner process has terminated: signal: aborted (core dumped) but after running `curl -fsSL https://ollama.com/install.sh | sh` I started getting this error: Error: llama runner process has terminated: signal: aborted (core dumped) error:failed to create context with model '/usr/share/ollama/.ollama/models/blobs/sha256-5ff0abeeac1d2dbdd5455c0b49ba3b29a9ce3c1fb181b2eef2e948689d55d046' Any idea/fixes?

GiteaMirror commented

2026-04-12 13:32:57 -05:00

@AlexFilipovici commented on GitHub (Jun 26, 2024):

Same here.

~$ollama list
NAME ID SIZE MODIFIED
mixtral:8x7b d39eb76ed9c5 26 GB 2 hours ago

~$ollama -v
ollama version is 0.1.46

~$curl -X POST -H "Content-Type: application/json" -d '{"model":"mixtral:8x7b", "prompt":"Why is the sky blue?", "stream": false}' http://127.0.0.1:11434/api/generate

{"error":"llama runner process has terminated: signal: aborted (core dumped) "}

@AlexFilipovici commented on GitHub (Jun 26, 2024): Same here. **~$ollama list** NAME ID SIZE MODIFIED mixtral:8x7b d39eb76ed9c5 26 GB 2 hours ago **~$ollama -v** ollama version is 0.1.46 **~$curl -X POST -H "Content-Type: application/json" -d '{"model":"mixtral:8x7b", "prompt":"Why is the sky blue?", "stream": false}' http://127.0.0.1:11434/api/generate** {"error":"llama runner process has terminated: signal: aborted (core dumped) "}

GiteaMirror commented

2026-04-12 13:32:57 -05:00

@AlexFilipovici commented on GitHub (Jun 27, 2024):

Same here.

~$ollama list NAME ID SIZE MODIFIED mixtral:8x7b d39eb76ed9c5 26 GB 2 hours ago

~$ollama -v ollama version is 0.1.46

~$curl -X POST -H "Content-Type: application/json" -d '{"model":"mixtral:8x7b", "prompt":"Why is the sky blue?", "stream": false}' http://127.0.0.1:11434/api/generate

{"error":"llama runner process has terminated: signal: aborted (core dumped) "}

Removed the model and gave it a go with ollama run qwen2:7b-instruct-q8_0.
It's working now.

@AlexFilipovici commented on GitHub (Jun 27, 2024): > Same here. > > **~$ollama list** NAME ID SIZE MODIFIED mixtral:8x7b d39eb76ed9c5 26 GB 2 hours ago > > **~$ollama -v** ollama version is 0.1.46 > > **~$curl -X POST -H "Content-Type: application/json" -d '{"model":"mixtral:8x7b", "prompt":"Why is the sky blue?", "stream": false}' http://127.0.0.1:11434/api/generate** > > {"error":"llama runner process has terminated: signal: aborted (core dumped) "} Removed the model and gave it a go with `ollama run qwen2:7b-instruct-q8_0`. It's working now.

GiteaMirror commented

2026-04-12 13:32:58 -05:00

@informaticker commented on GitHub (Jun 27, 2024):

Update your ollama.

-------- Original Message --------On 12.06.2024 09:26, VirtualZardoz wrote:
On Windows 11, I have the same issue. I tried with several different Qwen2 models and quantizations. None worked:
PS C:\Users\Shahram> ollama run qwen2:7b-instruct-q8_0
Error: llama runner process has terminated: signal: aborted
PS C:\Users\Shahram> Ollama: 500, message='Internal Server Error', url=URL('http://host.docker.internal:11434/api/chat')

Nvidia RTX 3090
Windows 11
Ollama 0.1.42 or 0.1.38, I never know:
PS C:\Users\Shahram> ollama -v
ollama version is 0.1.38
Warning: client version is 0.1.42

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.>
[
{
@.": "http://schema.org",
@.": "EmailMessage",
"potentialAction": {
@.": "ViewAction",
"target": "https://github.com/ollama/ollama/issues/4912#issuecomment-2162298408",
"url": "https://github.com/ollama/ollama/issues/4912#issuecomment-2162298408",
"name": "View Issue"
},
"description": "View this Issue on GitHub",
"publisher": {
@.***": "Organization",
"name": "GitHub",
"url": "https://github.com"
}
}
]

@informaticker commented on GitHub (Jun 27, 2024): Update your ollama. -------- Original Message --------On 12.06.2024 09:26, VirtualZardoz wrote: On Windows 11, I have the same issue. I tried with several different Qwen2 models and quantizations. None worked: PS C:\Users\Shahram> ollama run qwen2:7b-instruct-q8_0 Error: llama runner process has terminated: signal: aborted PS C:\Users\Shahram> Ollama: 500, message='Internal Server Error', url=URL('http://host.docker.internal:11434/api/chat') Nvidia RTX 3090 Windows 11 Ollama 0.1.42 or 0.1.38, I never know: PS C:\Users\Shahram> ollama -v ollama version is 0.1.38 Warning: client version is 0.1.42 —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: ***@***.***> [ { ***@***.***": "http://schema.org", ***@***.***": "EmailMessage", "potentialAction": { ***@***.***": "ViewAction", "target": "https://github.com/ollama/ollama/issues/4912#issuecomment-2162298408", "url": "https://github.com/ollama/ollama/issues/4912#issuecomment-2162298408", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { ***@***.***": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

GiteaMirror commented

2026-04-12 13:32:59 -05:00

@LegendNava commented on GitHub (Jun 30, 2024):

same issue with ollama 0.1.48, running dolphin-mixtral:8x7b-v2.5-q5_K_M
OS: Kali Linux

@LegendNava commented on GitHub (Jun 30, 2024): same issue with ollama 0.1.48, running dolphin-mixtral:8x7b-v2.5-q5_K_M OS: Kali Linux

GiteaMirror commented

2026-04-12 13:32:59 -05:00

@tomaszstachera commented on GitHub (Jul 3, 2024):

I've identified 2 root causes of llama runner process has terminated: signal: aborted (core dumped):

wrong name of the model in the request
not enough resources

@tomaszstachera commented on GitHub (Jul 3, 2024): I've identified 2 root causes of `llama runner process has terminated: signal: aborted (core dumped)`: 1. wrong name of the model in the request 2. not enough resources

GiteaMirror commented

2026-04-12 13:33:01 -05:00

@itsXactlY commented on GitHub (Jul 6, 2024):

Or, also, if NUM_CTX is too huge. Just noticed.

@itsXactlY commented on GitHub (Jul 6, 2024): Or, also, if NUM_CTX is too huge. Just noticed.

GiteaMirror commented

2026-04-12 13:33:01 -05:00

@mindkrypted commented on GitHub (Jul 9, 2024):

Seems like I'm encountering the same issue as stated by others.
In my case, it's when trying to set the context size manually for qwen:72b-chat-q4_0 using a customized modelfile with "parameter num_ctx 16384"

I have the resources available, if only the cpu&ram + gpu offloading would work...

base specs: 2x 3090 + 128gb of ram

ollama version 0.2

Loading model logs

``` Jul 09 01:07:19 AI-Station ollama[111814]: [GIN] 2024/07/09 - 01:07:19 | 200 | 17.010492ms | 127.0.0.1 | POST "/api/show" Jul 09 01:07:20 AI-Station ollama[111814]: time=2024-07-09T01:07:20.232-04:00 level=INFO source=memory.go:309 msg="offload to cuda" layers.requested=-1 layers.model=81 layers.offload=68 layers.split=34,34 memory.available="[23.4 GiB 23.4 GiB]" memory.required.full="53.3 GiB" memory.required.partial="46.8 GiB" memory.required.kv="640.0 MiB" memory.required.allocations="[23.4 GiB 23.4 GiB]" memory.weights.total="37.2 GiB" memory.weights.repeating="36.2 GiB" memory.weights.nonrepeating="974.6 MiB" memory.graph.full="6.7 GiB" memory.graph.partial="6.7 GiB" Jul 09 01:07:20 AI-Station ollama[111814]: time=2024-07-09T01:07:20.232-04:00 level=INFO source=server.go:375 msg="starting llama server" cmd="/tmp/ollama1733539351/runners/cuda_v11/ollama_llama_server --model /usr/share/ollama/.ollama/models/blobs/sha256-f749a67615fa575bd8d56d7b17b7064fc5c9489c03e0308f63d18885b55c8951 --ctx-size 16384 --batch-size 1024 --embedding --log-disable --n-gpu-layers 68 --parallel 1 --tensor-split 34,34 --tensor-split 34,34 --port 36743" Jul 09 01:07:20 AI-Station ollama[111814]: time=2024-07-09T01:07:20.232-04:00 level=INFO source=sched.go:477 msg="loaded runners" count=1 Jul 09 01:07:20 AI-Station ollama[111814]: time=2024-07-09T01:07:20.232-04:00 level=INFO source=server.go:563 msg="waiting for llama runner to start responding" Jul 09 01:07:20 AI-Station ollama[111814]: time=2024-07-09T01:07:20.232-04:00 level=INFO source=server.go:604 msg="waiting for server to become available" status="llm server error" Jul 09 01:07:20 AI-Station ollama[112171]: INFO [main] build info | build=1 commit="a8db2a9" tid="139680229888000" timestamp=1720501640 Jul 09 01:07:20 AI-Station ollama[112171]: INFO [main] system info | n_threads=16 n_threads_batch=-1 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 0 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 0 | " tid="139680229888000" timestamp=1720501640 total_threads=32 Jul 09 01:07:20 AI-Station ollama[112171]: INFO [main] HTTP server listening | hostname="127.0.0.1" n_threads_http="31" port="36743" tid="139680229888000" timestamp=1720501640 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: loaded meta data with 19 key-value pairs and 643 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-f749a67615fa575bd8d56d7b17b7064fc5c9489c03e0308f63d18885b55c8951 (version GGUF V3 (latest)) Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 0: general.architecture str = qwen Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 1: general.name str = Qwen Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 2: qwen.context_length u32 = 32768 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 3: qwen.block_count u32 = 80 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 4: qwen.embedding_length u32 = 8192 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 5: qwen.feed_forward_length u32 = 49152 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 6: qwen.rope.freq_base f32 = 1000000.000000 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 7: qwen.rope.dimension_count u32 = 128 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 8: qwen.attention.head_count u32 = 64 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 9: qwen.attention.layer_norm_rms_epsilon f32 = 0.000001 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 10: tokenizer.ggml.model str = gpt2 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 11: tokenizer.ggml.tokens arr[str,152064] = ["!", "\"", "#", "$", "%", "&", "'", ... Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 12: tokenizer.ggml.token_type arr[i32,152064] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 13: tokenizer.ggml.merges arr[str,151387] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",... Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 14: tokenizer.ggml.bos_token_id u32 = 151643 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 15: tokenizer.ggml.eos_token_id u32 = 151643 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 16: tokenizer.ggml.unknown_token_id u32 = 151643 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 17: general.quantization_version u32 = 2 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 18: general.file_type u32 = 2 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - type f32: 241 tensors Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - type q4_0: 401 tensors Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - type q6_K: 1 tensors Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_vocab: missing or unrecognized pre-tokenizer type, using: 'default' Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_vocab: special tokens cache size = 421 Jul 09 01:07:20 AI-Station ollama[111814]: time=2024-07-09T01:07:20.483-04:00 level=INFO source=server.go:604 msg="waiting for server to become available" status="llm server loading model" Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_vocab: token to piece cache size = 0.9355 MB Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: format = GGUF V3 (latest) Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: arch = qwen Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: vocab type = BPE Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_vocab = 152064 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_merges = 151387 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: vocab_only = 0 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_ctx_train = 32768 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_embd = 8192 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_layer = 80 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_head = 64 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_head_kv = 64 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_rot = 128 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_swa = 0 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_embd_head_k = 128 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_embd_head_v = 128 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_gqa = 1 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_embd_k_gqa = 8192 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_embd_v_gqa = 8192 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: f_norm_eps = 0.0e+00 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: f_norm_rms_eps = 1.0e-06 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: f_clamp_kqv = 0.0e+00 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: f_max_alibi_bias = 0.0e+00 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: f_logit_scale = 0.0e+00 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_ff = 49152 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_expert = 0 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_expert_used = 0 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: causal attn = 1 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: pooling type = 0 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: rope type = 2 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: rope scaling = linear Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: freq_base_train = 1000000.0 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: freq_scale_train = 1 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_ctx_orig_yarn = 32768 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: rope_finetuned = unknown Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: ssm_d_conv = 0 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: ssm_d_inner = 0 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: ssm_d_state = 0 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: ssm_dt_rank = 0 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: model type = ?B Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: model ftype = Q4_0 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: model params = 72.29 B Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: model size = 38.18 GiB (4.54 BPW) Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: general.name = Qwen Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: BOS token = 151643 '<|endoftext|>' Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: EOS token = 151643 '<|endoftext|>' Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: UNK token = 151643 '<|endoftext|>' Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: LF token = 148848 'ÄĬ' Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: EOT token = 151645 '<|im_end|>' Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: max token length = 256 Jul 09 01:07:20 AI-Station ollama[111814]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ: yes Jul 09 01:07:20 AI-Station ollama[111814]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no Jul 09 01:07:20 AI-Station ollama[111814]: ggml_cuda_init: found 2 CUDA devices: Jul 09 01:07:20 AI-Station ollama[111814]: Device 0: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes Jul 09 01:07:20 AI-Station ollama[111814]: Device 1: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_tensors: ggml ctx size = 0.93 MiB Jul 09 01:07:21 AI-Station ollama[111814]: time=2024-07-09T01:07:21.938-04:00 level=INFO source=server.go:604 msg="waiting for server to become available" status="llm server not responding" Jul 09 01:07:22 AI-Station ollama[111814]: time=2024-07-09T01:07:22.362-04:00 level=INFO source=server.go:604 msg="waiting for server to become available" status="llm server loading model" Jul 09 01:07:22 AI-Station ollama[111814]: llm_load_tensors: offloading 68 repeating layers to GPU Jul 09 01:07:22 AI-Station ollama[111814]: llm_load_tensors: offloaded 68/81 layers to GPU Jul 09 01:07:22 AI-Station ollama[111814]: llm_load_tensors: CPU buffer size = 39095.31 MiB Jul 09 01:07:22 AI-Station ollama[111814]: llm_load_tensors: CUDA0 buffer size = 15917.31 MiB Jul 09 01:07:22 AI-Station ollama[111814]: llm_load_tensors: CUDA1 buffer size = 15917.31 MiB Jul 09 01:07:25 AI-Station ollama[111814]: llama_new_context_with_model: n_ctx = 16384 Jul 09 01:07:25 AI-Station ollama[111814]: llama_new_context_with_model: n_batch = 1024 Jul 09 01:07:25 AI-Station ollama[111814]: llama_new_context_with_model: n_ubatch = 512 Jul 09 01:07:25 AI-Station ollama[111814]: llama_new_context_with_model: flash_attn = 0 Jul 09 01:07:25 AI-Station ollama[111814]: llama_new_context_with_model: freq_base = 1000000.0 Jul 09 01:07:25 AI-Station ollama[111814]: llama_new_context_with_model: freq_scale = 1 Jul 09 01:07:26 AI-Station ollama[111814]: time=2024-07-09T01:07:26.329-04:00 level=INFO source=server.go:604 msg="waiting for server to become available" status="llm server not responding" Jul 09 01:07:27 AI-Station ollama[111814]: time=2024-07-09T01:07:27.664-04:00 level=INFO source=server.go:604 msg="waiting for server to become available" status="llm server loading model" Jul 09 01:07:28 AI-Station ollama[111814]: llama_kv_cache_init: CUDA_Host KV buffer size = 6144.00 MiB Jul 09 01:07:28 AI-Station ollama[111814]: ggml_backend_cuda_buffer_type_alloc_buffer: allocating 17408.00 MiB on device 0: cudaMalloc failed: out of memory Jul 09 01:07:28 AI-Station ollama[111814]: llama_kv_cache_init: failed to allocate buffer for kv cache Jul 09 01:07:28 AI-Station ollama[111814]: llama_new_context_with_model: llama_kv_cache_init() failed for self-attention cache Jul 09 01:07:29 AI-Station ollama[111814]: llama_init_from_gpt_params: error: failed to create context with model '/usr/share/ollama/.ollama/models/blobs/sha256-f749a67615fa575bd8d56d7b17b7064fc5c9489c03e0308f63d18885b55c8951' Jul 09 01:07:30 AI-Station ollama[112171]: ERROR [load_model] unable to load model | model="/usr/share/ollama/.ollama/models/blobs/sha256-f749a67615fa575bd8d56d7b17b7064fc5c9489c03e0308f63d18885b55c8951" tid="139680229888000" timestamp=1720501650 Jul 09 01:07:30 AI-Station ollama[111814]: terminate called without an active exception Jul 09 01:07:30 AI-Station ollama[111814]: time=2024-07-09T01:07:30.175-04:00 level=ERROR source=sched.go:483 msg="error loading llama server" error="llama runner process has terminated: signal: aborted error:failed to create context with model '/usr/share/ollama/.ollama/models/blobs/sha256-f749a67615fa575bd8d56d7b17b7064fc5c9489c03e0308f63d18885b55c8951'" Jul 09 01:07:30 AI-Station ollama[111814]: [GIN] 2024/07/09 - 01:07:30 | 500 | 10.330751378s | 127.0.0.1 | POST "/api/chat" Jul 09 01:07:35 AI-Station ollama[111814]: time=2024-07-09T01:07:35.418-04:00 level=WARN source=sched.go:674 msg="gpu VRAM usage didn't recover within timeout" seconds=5.24330618 model=/usr/share/ollama/.ollama/models/blobs/sha256-f749a67615fa575bd8d56d7b17b7064fc5c9489c03e0308f63d18885b55c8951 Jul 09 01:07:35 AI-Station ollama[111814]: time=2024-07-09T01:07:35.669-04:00 level=WARN source=sched.go:674 msg="gpu VRAM usage didn't recover within timeout" seconds=5.494062518 model=/usr/share/ollama/.ollama/models/blobs/sha256-f749a67615fa575bd8d56d7b17b7064fc5c9489c03e0308f63d18885b55c8951 Jul 09 01:07:35 AI-Station ollama[111814]: time=2024-07-09T01:07:35.919-04:00 level=WARN source=sched.go:674 msg="gpu VRAM usage didn't recover within timeout" seconds=5.744309878 model=/usr/share/ollama/.ollama/models/blobs/sha256-f749a67615fa575bd8d56d7b17b7064fc5c9489c03e0308f63d18885b55c8951 Jul 09 01:08:52 AI-Station ollama[111814]: [GIN] 2024/07/09 - 01:08:52 | 200 | 18.04µs | 127.0.0.1 | HEAD "/" Jul 09 01:08:52 AI-Station ollama[111814]: [GIN] 2024/07/09 - 01:08:52 | 200 | 1.2676ms | 127.0.0.1 | GET "/api/tags" ```

@mindkrypted commented on GitHub (Jul 9, 2024): Seems like I'm encountering the same issue as stated by others. In my case, it's when trying to set the context size manually for qwen:72b-chat-q4_0 using a customized modelfile with "parameter num_ctx 16384" I have the resources available, if only the cpu&ram + gpu offloading would work... base specs: 2x 3090 + 128gb of ram ollama version 0.2 <details> <summary>Loading model logs</summary> ``` Jul 09 01:07:19 AI-Station ollama[111814]: [GIN] 2024/07/09 - 01:07:19 | 200 | 17.010492ms | 127.0.0.1 | POST "/api/show" Jul 09 01:07:20 AI-Station ollama[111814]: time=2024-07-09T01:07:20.232-04:00 level=INFO source=memory.go:309 msg="offload to cuda" layers.requested=-1 layers.model=81 layers.offload=68 layers.split=34,34 memory.available="[23.4 GiB 23.4 GiB]" memory.required.full="53.3 GiB" memory.required.partial="46.8 GiB" memory.required.kv="640.0 MiB" memory.required.allocations="[23.4 GiB 23.4 GiB]" memory.weights.total="37.2 GiB" memory.weights.repeating="36.2 GiB" memory.weights.nonrepeating="974.6 MiB" memory.graph.full="6.7 GiB" memory.graph.partial="6.7 GiB" Jul 09 01:07:20 AI-Station ollama[111814]: time=2024-07-09T01:07:20.232-04:00 level=INFO source=server.go:375 msg="starting llama server" cmd="/tmp/ollama1733539351/runners/cuda_v11/ollama_llama_server --model /usr/share/ollama/.ollama/models/blobs/sha256-f749a67615fa575bd8d56d7b17b7064fc5c9489c03e0308f63d18885b55c8951 --ctx-size 16384 --batch-size 1024 --embedding --log-disable --n-gpu-layers 68 --parallel 1 --tensor-split 34,34 --tensor-split 34,34 --port 36743" Jul 09 01:07:20 AI-Station ollama[111814]: time=2024-07-09T01:07:20.232-04:00 level=INFO source=sched.go:477 msg="loaded runners" count=1 Jul 09 01:07:20 AI-Station ollama[111814]: time=2024-07-09T01:07:20.232-04:00 level=INFO source=server.go:563 msg="waiting for llama runner to start responding" Jul 09 01:07:20 AI-Station ollama[111814]: time=2024-07-09T01:07:20.232-04:00 level=INFO source=server.go:604 msg="waiting for server to become available" status="llm server error" Jul 09 01:07:20 AI-Station ollama[112171]: INFO [main] build info | build=1 commit="a8db2a9" tid="139680229888000" timestamp=1720501640 Jul 09 01:07:20 AI-Station ollama[112171]: INFO [main] system info | n_threads=16 n_threads_batch=-1 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 0 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 0 | " tid="139680229888000" timestamp=1720501640 total_threads=32 Jul 09 01:07:20 AI-Station ollama[112171]: INFO [main] HTTP server listening | hostname="127.0.0.1" n_threads_http="31" port="36743" tid="139680229888000" timestamp=1720501640 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: loaded meta data with 19 key-value pairs and 643 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-f749a67615fa575bd8d56d7b17b7064fc5c9489c03e0308f63d18885b55c8951 (version GGUF V3 (latest)) Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 0: general.architecture str = qwen Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 1: general.name str = Qwen Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 2: qwen.context_length u32 = 32768 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 3: qwen.block_count u32 = 80 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 4: qwen.embedding_length u32 = 8192 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 5: qwen.feed_forward_length u32 = 49152 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 6: qwen.rope.freq_base f32 = 1000000.000000 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 7: qwen.rope.dimension_count u32 = 128 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 8: qwen.attention.head_count u32 = 64 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 9: qwen.attention.layer_norm_rms_epsilon f32 = 0.000001 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 10: tokenizer.ggml.model str = gpt2 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 11: tokenizer.ggml.tokens arr[str,152064] = ["!", "\"", "#", "$", "%", "&", "'", ... Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 12: tokenizer.ggml.token_type arr[i32,152064] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 13: tokenizer.ggml.merges arr[str,151387] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",... Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 14: tokenizer.ggml.bos_token_id u32 = 151643 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 15: tokenizer.ggml.eos_token_id u32 = 151643 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 16: tokenizer.ggml.unknown_token_id u32 = 151643 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 17: general.quantization_version u32 = 2 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - kv 18: general.file_type u32 = 2 Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - type f32: 241 tensors Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - type q4_0: 401 tensors Jul 09 01:07:20 AI-Station ollama[111814]: llama_model_loader: - type q6_K: 1 tensors Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_vocab: missing or unrecognized pre-tokenizer type, using: 'default' Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_vocab: special tokens cache size = 421 Jul 09 01:07:20 AI-Station ollama[111814]: time=2024-07-09T01:07:20.483-04:00 level=INFO source=server.go:604 msg="waiting for server to become available" status="llm server loading model" Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_vocab: token to piece cache size = 0.9355 MB Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: format = GGUF V3 (latest) Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: arch = qwen Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: vocab type = BPE Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_vocab = 152064 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_merges = 151387 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: vocab_only = 0 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_ctx_train = 32768 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_embd = 8192 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_layer = 80 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_head = 64 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_head_kv = 64 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_rot = 128 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_swa = 0 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_embd_head_k = 128 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_embd_head_v = 128 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_gqa = 1 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_embd_k_gqa = 8192 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_embd_v_gqa = 8192 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: f_norm_eps = 0.0e+00 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: f_norm_rms_eps = 1.0e-06 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: f_clamp_kqv = 0.0e+00 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: f_max_alibi_bias = 0.0e+00 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: f_logit_scale = 0.0e+00 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_ff = 49152 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_expert = 0 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_expert_used = 0 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: causal attn = 1 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: pooling type = 0 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: rope type = 2 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: rope scaling = linear Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: freq_base_train = 1000000.0 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: freq_scale_train = 1 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: n_ctx_orig_yarn = 32768 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: rope_finetuned = unknown Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: ssm_d_conv = 0 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: ssm_d_inner = 0 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: ssm_d_state = 0 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: ssm_dt_rank = 0 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: model type = ?B Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: model ftype = Q4_0 Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: model params = 72.29 B Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: model size = 38.18 GiB (4.54 BPW) Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: general.name = Qwen Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: BOS token = 151643 '<|endoftext|>' Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: EOS token = 151643 '<|endoftext|>' Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: UNK token = 151643 '<|endoftext|>' Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: LF token = 148848 'ÄĬ' Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: EOT token = 151645 '<|im_end|>' Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_print_meta: max token length = 256 Jul 09 01:07:20 AI-Station ollama[111814]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ: yes Jul 09 01:07:20 AI-Station ollama[111814]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no Jul 09 01:07:20 AI-Station ollama[111814]: ggml_cuda_init: found 2 CUDA devices: Jul 09 01:07:20 AI-Station ollama[111814]: Device 0: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes Jul 09 01:07:20 AI-Station ollama[111814]: Device 1: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes Jul 09 01:07:20 AI-Station ollama[111814]: llm_load_tensors: ggml ctx size = 0.93 MiB Jul 09 01:07:21 AI-Station ollama[111814]: time=2024-07-09T01:07:21.938-04:00 level=INFO source=server.go:604 msg="waiting for server to become available" status="llm server not responding" Jul 09 01:07:22 AI-Station ollama[111814]: time=2024-07-09T01:07:22.362-04:00 level=INFO source=server.go:604 msg="waiting for server to become available" status="llm server loading model" Jul 09 01:07:22 AI-Station ollama[111814]: llm_load_tensors: offloading 68 repeating layers to GPU Jul 09 01:07:22 AI-Station ollama[111814]: llm_load_tensors: offloaded 68/81 layers to GPU Jul 09 01:07:22 AI-Station ollama[111814]: llm_load_tensors: CPU buffer size = 39095.31 MiB Jul 09 01:07:22 AI-Station ollama[111814]: llm_load_tensors: CUDA0 buffer size = 15917.31 MiB Jul 09 01:07:22 AI-Station ollama[111814]: llm_load_tensors: CUDA1 buffer size = 15917.31 MiB Jul 09 01:07:25 AI-Station ollama[111814]: llama_new_context_with_model: n_ctx = 16384 Jul 09 01:07:25 AI-Station ollama[111814]: llama_new_context_with_model: n_batch = 1024 Jul 09 01:07:25 AI-Station ollama[111814]: llama_new_context_with_model: n_ubatch = 512 Jul 09 01:07:25 AI-Station ollama[111814]: llama_new_context_with_model: flash_attn = 0 Jul 09 01:07:25 AI-Station ollama[111814]: llama_new_context_with_model: freq_base = 1000000.0 Jul 09 01:07:25 AI-Station ollama[111814]: llama_new_context_with_model: freq_scale = 1 Jul 09 01:07:26 AI-Station ollama[111814]: time=2024-07-09T01:07:26.329-04:00 level=INFO source=server.go:604 msg="waiting for server to become available" status="llm server not responding" Jul 09 01:07:27 AI-Station ollama[111814]: time=2024-07-09T01:07:27.664-04:00 level=INFO source=server.go:604 msg="waiting for server to become available" status="llm server loading model" Jul 09 01:07:28 AI-Station ollama[111814]: llama_kv_cache_init: CUDA_Host KV buffer size = 6144.00 MiB Jul 09 01:07:28 AI-Station ollama[111814]: ggml_backend_cuda_buffer_type_alloc_buffer: allocating 17408.00 MiB on device 0: cudaMalloc failed: out of memory Jul 09 01:07:28 AI-Station ollama[111814]: llama_kv_cache_init: failed to allocate buffer for kv cache Jul 09 01:07:28 AI-Station ollama[111814]: llama_new_context_with_model: llama_kv_cache_init() failed for self-attention cache Jul 09 01:07:29 AI-Station ollama[111814]: llama_init_from_gpt_params: error: failed to create context with model '/usr/share/ollama/.ollama/models/blobs/sha256-f749a67615fa575bd8d56d7b17b7064fc5c9489c03e0308f63d18885b55c8951' Jul 09 01:07:30 AI-Station ollama[112171]: ERROR [load_model] unable to load model | model="/usr/share/ollama/.ollama/models/blobs/sha256-f749a67615fa575bd8d56d7b17b7064fc5c9489c03e0308f63d18885b55c8951" tid="139680229888000" timestamp=1720501650 Jul 09 01:07:30 AI-Station ollama[111814]: terminate called without an active exception Jul 09 01:07:30 AI-Station ollama[111814]: time=2024-07-09T01:07:30.175-04:00 level=ERROR source=sched.go:483 msg="error loading llama server" error="llama runner process has terminated: signal: aborted error:failed to create context with model '/usr/share/ollama/.ollama/models/blobs/sha256-f749a67615fa575bd8d56d7b17b7064fc5c9489c03e0308f63d18885b55c8951'" Jul 09 01:07:30 AI-Station ollama[111814]: [GIN] 2024/07/09 - 01:07:30 | 500 | 10.330751378s | 127.0.0.1 | POST "/api/chat" Jul 09 01:07:35 AI-Station ollama[111814]: time=2024-07-09T01:07:35.418-04:00 level=WARN source=sched.go:674 msg="gpu VRAM usage didn't recover within timeout" seconds=5.24330618 model=/usr/share/ollama/.ollama/models/blobs/sha256-f749a67615fa575bd8d56d7b17b7064fc5c9489c03e0308f63d18885b55c8951 Jul 09 01:07:35 AI-Station ollama[111814]: time=2024-07-09T01:07:35.669-04:00 level=WARN source=sched.go:674 msg="gpu VRAM usage didn't recover within timeout" seconds=5.494062518 model=/usr/share/ollama/.ollama/models/blobs/sha256-f749a67615fa575bd8d56d7b17b7064fc5c9489c03e0308f63d18885b55c8951 Jul 09 01:07:35 AI-Station ollama[111814]: time=2024-07-09T01:07:35.919-04:00 level=WARN source=sched.go:674 msg="gpu VRAM usage didn't recover within timeout" seconds=5.744309878 model=/usr/share/ollama/.ollama/models/blobs/sha256-f749a67615fa575bd8d56d7b17b7064fc5c9489c03e0308f63d18885b55c8951 Jul 09 01:08:52 AI-Station ollama[111814]: [GIN] 2024/07/09 - 01:08:52 | 200 | 18.04µs | 127.0.0.1 | HEAD "/" Jul 09 01:08:52 AI-Station ollama[111814]: [GIN] 2024/07/09 - 01:08:52 | 200 | 1.2676ms | 127.0.0.1 | GET "/api/tags" ``` </details>

GiteaMirror commented

2026-04-12 13:33:02 -05:00

@igorschlum commented on GitHub (Jul 9, 2024):

Hi @mindkrypted car you share your modelfile and the command line? I will try and see if I can reproduce the issue on MacOS.

@igorschlum commented on GitHub (Jul 9, 2024): Hi @mindkrypted car you share your modelfile and the command line? I will try and see if I can reproduce the issue on MacOS.

GiteaMirror commented

2026-04-12 13:33:02 -05:00

@mindkrypted commented on GitHub (Jul 10, 2024):

Hi @mindkrypted car you share your modelfile and the command line? I will try and see if I can reproduce the issue on MacOS.

ollama run qwen2-q4-16k:latest --verbose

FROM qwen:72b-chat-q4_0
TEMPLATE "<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"
PARAMETER num_batch 1024
PARAMETER num_ctx 16384
PARAMETER temperature 0.4
PARAMETER top_p 0.9
PARAMETER frequency_penalty 0.2
PARAMETER num_keep 256
PARAMETER presence_penalty 0.2
PARAMETER repeat_last_n 50

I'm able to reproduce the same behaviour using the base model "qwen:72b-chat-q4_0" with open-webui if I change the CTX from the Web app and then submit a prompt.
Using the "ollama run..." command then "/set parameter num_ctx 16384" does it too.

@mindkrypted commented on GitHub (Jul 10, 2024): > Hi @mindkrypted car you share your modelfile and the command line? I will try and see if I can reproduce the issue on MacOS. ollama run qwen2-q4-16k:latest --verbose ``` FROM qwen:72b-chat-q4_0 TEMPLATE "<|im_start|>system {{ .System }}<|im_end|> <|im_start|>user {{ .Prompt }}<|im_end|> <|im_start|>assistant " PARAMETER num_batch 1024 PARAMETER num_ctx 16384 PARAMETER temperature 0.4 PARAMETER top_p 0.9 PARAMETER frequency_penalty 0.2 PARAMETER num_keep 256 PARAMETER presence_penalty 0.2 PARAMETER repeat_last_n 50 ``` I'm able to reproduce the same behaviour using the base model "qwen:72b-chat-q4_0" with open-webui if I change the CTX from the Web app and then submit a prompt. Using the "ollama run..." command then "/set parameter num_ctx 16384" does it too.

GiteaMirror commented

2026-04-12 13:33:03 -05:00

@pleabargain commented on GitHub (Aug 26, 2024):

Trying to get Ollama v .3.6 to run on github codespaces. No joy.
ollama serve is running in a diff. term
errors there:
llm_load_print_meta: EOT token = 128009 '<|eot_id|>'
llm_load_print_meta: max token length = 256
llm_load_tensors: ggml ctx size = 0.14 MiB
llm_load_tensors: CPU buffer size = 4437.80 MiB
llama_new_context_with_model: n_ctx = 8192
llama_new_context_with_model: n_batch = 512
llama_new_context_with_model: n_ubatch = 512
llama_new_context_with_model: flash_attn = 0
llama_new_context_with_model: freq_base = 500000.0
llama_new_context_with_model: freq_scale = 1
time=2024-08-26T09:53:44.082Z level=INFO source=server.go:627 msg="waiting for server to become available" status="llm server not responding"
time=2024-08-26T09:53:44.346Z level=ERROR source=sched.go:451 msg="error loading llama server" error="llama runner process has terminated: signal: terminated"
[GIN] 2024/08/26 - 09:53:44 | 500 | 24.154367033s | 127.0.0.1 | POST "/api/chat"
[GIN] 2024/08/26 - 09:54:29 | 200 | 313.555µs | 127.0.0.1 | GET "/api/version"

error on
ollama run llama3

I would like to run Ollama on someone else's faster CPU/GPU :)

Constructive advice is appreciated!

@pleabargain commented on GitHub (Aug 26, 2024): Trying to get Ollama v .3.6 to run on github codespaces. No joy. ollama serve is running in a diff. term errors there: llm_load_print_meta: EOT token = 128009 '<|eot_id|>' llm_load_print_meta: max token length = 256 llm_load_tensors: ggml ctx size = 0.14 MiB llm_load_tensors: CPU buffer size = 4437.80 MiB llama_new_context_with_model: n_ctx = 8192 llama_new_context_with_model: n_batch = 512 llama_new_context_with_model: n_ubatch = 512 llama_new_context_with_model: flash_attn = 0 llama_new_context_with_model: freq_base = 500000.0 llama_new_context_with_model: freq_scale = 1 time=2024-08-26T09:53:44.082Z level=INFO source=server.go:627 msg="waiting for server to become available" status="llm server not responding" time=2024-08-26T09:53:44.346Z level=ERROR source=sched.go:451 msg="error loading llama server" error="llama runner process has terminated: signal: terminated" [GIN] 2024/08/26 - 09:53:44 | 500 | 24.154367033s | 127.0.0.1 | POST "/api/chat" [GIN] 2024/08/26 - 09:54:29 | 200 | 313.555µs | 127.0.0.1 | GET "/api/version" error on ollama run llama3 ![image](https://github.com/user-attachments/assets/70b29e5d-1448-4fc9-8b2a-d6d8febc50d5) I would like to run Ollama on someone else's faster CPU/GPU :) Constructive advice is appreciated!

GiteaMirror referenced this issue

2026-04-22 05:10:42 -05:00

[GH-ISSUE #3102] Response_format not supported #27665

GiteaMirror referenced this issue

2026-04-28 08:12:44 -05:00

[GH-ISSUE #3102] Response_format not supported #48417

GiteaMirror referenced this issue

2026-05-03 15:32:11 -05:00

[GH-ISSUE #3102] Response_format not supported #63943

Sign in to join this conversation.

Branches Tags

main

hoyyeva/anthropic-local-image-path

dhiltgen/ci

dhiltgen/llama-runner

parth-remove-claude-desktop-launch

hoyyeva/anthropic-reference-images-path

parth-anthropic-reference-images-path

brucemacd/download-before-remove

hoyyeva/editor-config-repair

parth-mlx-decode-checkpoints

parth-launch-codex-app

hoyyeva/fix-codex-model-metadata-warning

hoyyeva/qwen

parth/hide-claude-desktop-till-release

hoyyeva/opencode-image-modality

parth-add-claude-code-autoinstall

release_v0.22.0

pdevine/manifest-list

codex/fix-codex-model-metadata-warning

pdevine/addressable-manifest

brucemacd/launch-fetch-reccomended

jmorganca/llama-compat

launch-copilot-cli

hoyyeva/opencode-thinking

release_v0.20.7

parth-auto-save-backup

parth-test

jmorganca/gemma4-audio-replacements

fix-manifest-digest-on-pull

hoyyeva/vscode-improve

brucemacd/install-server-wait

parth/update-claude-docs

brucemac/start-ap-install

pdevine/mlx-update

pdevine/qwen35_vision

drifkin/api-show-fallback

mintlify/image-generation-1773352582

hoyyeva/server-context-length-local-config

jmorganca/faster-reptition-penalties

jmorganca/convert-nemotron

parth-pi-thinking

pdevine/sampling-penalties

jmorganca/fix-create-quantization-memory

dongchen/resumable_transfer_fix

pdevine/sampling-cache-error

jessegross/mlx-usage

hoyyeva/openclaw-config

hoyyeva/app-html

pdevine/qwen3next

brucemacd/sign-sh-install

brucemacd/tui-update

brucemacd/usage-api

jmorganca/launch-empty

fix-app-dist-embed

mxyng/mlx-compile

mxyng/mlx-quant

mxyng/mlx-glm4.7

mxyng/mlx

brucemacd/simplify-model-picker

jmorganca/qwen3-concurrent

fix-glm-4.7-flash-mla-config

drifkin/qwen3-coder-opening-tag

brucemacd/usage-cli

fix-cuda12-fattn-shmem

ollama-imagegen-docs

parth/fix-multiline-inputs

brucemacd/config-docs

mxyng/model-files

mxyng/simple-execute

fix-imagegen-ollama-models

mxyng/async-upload

jmorganca/lazy-no-dtype-changes

imagegen-auto-detect-create

parth/decrease-concurrent-download-hf

fix-mlx-quantize-init

jmorganca/x-cleanup

usage

imagegen-readme

jmorganca/glm-image

mlx-gpu-cd

jmorganca/imagegen-modelfile

parth/agent-skills

parth/agent-allowlist

parth/signed-in-offline

parth/agents

parth/fix-context-chopping

improve-cloud-flow

parth/add-models-websearch

parth/prompt-renderer-mcp

jmorganca/native-settings

jmorganca/download-stream-hash

jmorganca/client2-rebased

brucemacd/oai-chat-req-multipart

jessegross/multi_chunk_reserve

grace/additional-omit-empty

grace/mistral-3-large

mxyng/tokenizer2

mxyng/tokenizer

jessegross/flash

hoyyeva/windows-nacked-app

mxyng/cleanup-attention

grace/deepseek-parser

hoyyeva/remember-unsent-prompt

parth/add-lfs-pointer-error-conversion

parth/olmo2-test2

hoyyeva/ollama-launchagent-plist

nicole/olmo-model

parth/olmo-test

mxyng/remove-embedded

parth/render-template

jmorganca/intellect-3

parth/remove-prealloc-linter

jmorganca/cmd-eval

nicole/nomic-embed-text-fix

mxyng/lint-2

hoyyeva/add-gemini-3-pro-preview

hoyyeva/load-model-list

mxyng/expand-path

mxyng/environ-2

hoyyeva/deeplink-json-encoding

parth/improve-tool-calling-tests

hoyyeva/conversation

hoyyeva/assistant-edit-response

hoyyeva/thinking

origin/brucemacd/invalid-char-i-err

parth/improve-tool-calling

jmorganca/required-omitempty

grace/qwen3-vl-tests

mxyng/iter-client

parth/docs-readme

nicole/embed-test

pdevine/integration-benchstat

parth/remove-generate-cmd

parth/add-toolcall-id

mxyng/server-tests

jmorganca/glm-4.6

jmorganca/gin-h-compat

drifkin/stable-tool-args

pdevine/qwen3-more-thinking

parth/add-websearch-client

nicole/websearch_local

jmorganca/qwen3-coder-updates

grace/deepseek-v3-migration-tests

mxyng/fix-create

jmorganca/cloud-errors

pdevine/parser-tidy

revert-12233-parth/simplify-entrypoints-runner

parth/enable-so-gpt-oss

brucemacd/qwen3vl

jmorganca/readme-simplify

parth/gpt-oss-structured-outputs

revert-12039-jmorganca/tools-braces

mxyng/embeddings

mxyng/gguf

mxyng/benchmark

mxyng/types-null

parth/move-parsing

mxyng/gemma2

jmorganca/docs

mxyng/16-bit

mxyng/create-stdin

pdevine/authorizedkeys

mxyng/quant

parth/opt-in-error-context-window

brucemacd/cache-models

brucemacd/runner-completion

jmorganca/llama-update-6

brucemacd/benchmark-list

brucemacd/partial-read-caps

parth/deepseek-r1-tools

mxyng/omit-array

parth/tool-prefix-temp

brucemacd/runner-test

jmorganca/qwen25vl

brucemacd/model-forward-test-ext

parth/python-function-parsing

jmorganca/cuda-compression-none

drifkin/num-parallel

drifkin/chat-truncation-fix

jmorganca/sync

parth/python-tools-calling

drifkin/array-head-count

brucemacd/create-no-loop

parth/server-enable-content-stream-with-tools

qwen25omni

mxyng/v3

brucemacd/ropeconfig

jmorganca/silence-tokenizer

parth/sample-so-test

parth/sampling-structured-outputs

brucemacd/doc-go-engine

parth/constrained-sampling-json

jmorganca/mistral-wip

brucemacd/mistral-small-convert

parth/sample-unmarshal-json-for-params

brucemacd/jomorganca/mistral

pdevine/bfloat16

jmorganca/mistral

brucemacd/mistral

pdevine/logging

parth/sample-correctness-fix

parth/sample-fix-sorting

jmorgan/sample-fix-sorting-extras

jmorganca/temp-0-images

brucemacd/parallel-embed-models

brucemacd/shim-grammar

jmorganca/fix-gguf-error

bmizerany/nameswork

jmorganca/faster-releases

bmizerany/validatenames

brucemacd/err-no-vocab

brucemacd/rope-config

brucemacd/err-hint

brucemacd/qwen2_5

brucemacd/logprobs

brucemacd/new_runner_graph_bench

progress-flicker

brucemacd/forward-test

brucemacd/go_qwen2

pdevine/gemma2

jmorganca/add-missing-symlink-eval

mxyng/next-debug

parth/set-context-size-openai

brucemacd/next-bpe-bench

brucemacd/next-bpe-test

brucemacd/new_runner_e2e

brucemacd/new_runner_qwen2

pdevine/convert-cohere2

brucemacd/convert-cli

parth/log-probs

mxyng/next-mlx

mxyng/cmd-history

parth/templating

parth/tokenize-detokenize

brucemacd/check-key-register

bmizerany/grammar

jmorganca/vendor-081b29bd

mxyng/func-checks

jmorganca/fix-null-format

parth/fix-default-to-warn-json

jmorganca/qwen2vl

jmorganca/no-concat

parth/cmd-cleanup-SO

brucemacd/check-key-register-structured-err

parth/openai-stream-usage

parth/fix-referencing-so

stream-tools-stop

jmorganca/degin-1

brucemacd/install-path-clean

brucemacd/push-name-validation

brucemacd/browser-key-register

jmorganca/openai-fix-first-message

jmorganca/fix-proxy

jessegross/sample

parth/disallow-streaming-tools

dhiltgen/remove_submodule

jmorganca/ga

jmorganca/mllama

pdevine/newlines

pdevine/geems-2b

jmorganca/llama-bump

mxyng/modelname-7

mxyng/gin-slog

mxyng/modelname-6

jyan/convert-prog

jyan/quant5

paligemma-support

pdevine/import-docs

jmorganca/openai-context

jyan/paligemma

jyan/p2

jyan/palitest

bmizerany/embedspeedup

jmorganca/llama-vit

brucemacd/allow-ollama

royh/ep-methods

royh/whisper

mxyng/api-models

mxyng/fix-memory

jyan/q4_4/8

jyan/ollama-v

royh/stream-tools

roy-embed-parallel

bmizerany/hrm

revert-5963-revert-5924-mxyng/llama3.1-rope

royh/embed-viz

jyan/local2

jyan/auth

jyan/local

jyan/parse-temp

jmorganca/template-mistral

jyan/reord-g

royh-openai-suffixdocs

royh-imgembed

royh-embed-parallel

jyan/quant4

royh-precision

jyan/progress

pdevine/fix-template

jyan/quant3

pdevine/ggla

mxyng/update-registry-domain

jmorganca/ggml-static

mxyng/create-context

jyan/v0.146

mxyng/layers-from-files

build_dist

bmizerany/noseek

royh-ls

royh-name

timeout

mxyng/server-timestamp

bmizerany/nosillyggufslurps

royh-params

jmorganca/llama-cpp-7c26775

royh-openai-delete

royh-show-rigid

jmorganca/enable-fa

jmorganca/no-error-template

jyan/format

royh-testdelete

bmizerany/fastverify

language_support

pdevine/ps-glitches

brucemacd/tokenize

bruce/iq-quants

bmizerany/filepathwithcoloninhost

mxyng/split-bin

bmizerany/client-registry

jmorganca/if-none-match

native

jmorganca/native

jmorganca/batch-embeddings

jmorganca/initcmake

jmorganca/mm

pdevine/showggmlinfo

modenameenforcealphanum

bmizerany/modenameenforcealphanum

jmorganca/done-reason

jmorganca/llama-cpp-8960fe8

ollama.com

bmizerany/filepathnobuild

bmizerany/types/model/defaultfix

rmdisplaylong

nogogen

bmizerany/x

modelfile-readme

bmizerany/replacecolon

jmorganca/limit

jmorganca/execstack

jmorganca/replace-assets

mxyng/tune-concurrency

jmorganca/testing

whitespace-detection

jmorganca/options

upgrade-all

scratch

cuda-search

mattw/airenamer

mattw/allmodelsonhuggingface

mattw/quantcontext

mattw/whatneedstorun

brucemacd/llama-mem-calc

mattw/faq-context

mattw/communitylinks

mattw/noprune

mattw/python-functioncalling

rename

mxyng/install

pulse

remove-first

editor

mattw/selfqueryingretrieval

cgo

mattw/howtoquant

api

matt/streamingapi

format-config

mxyng/extra-args

shell

update-nous-hermes

cp-model

upload-progress

fix-unknown-model

fix-model-names

delete-fix

insecure-registry

ls

deletemodels

progressbar

readme-updates

license-layers

skip-list

list-models

modelpath

matt/examplemodelfiles

distribution

go-opts

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/ollama#3102