[GH-ISSUE #8299] Querying can cause c drive to max out 100% #5313

Closed
opened 2026-04-12 16:30:30 -05:00 by GiteaMirror · 16 comments
Owner

Originally created by @Bandit253 on GitHub (Jan 3, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/8299

What is the issue?

When using from langchain_community.document_loaders import WebBaseLoader - (maybe not cause)
I get ResponseError: llama runner process has terminated: error loading model: llama_model_loader: failed to load model from it doesn't even appear to finish the error message!?

but after retying, I get this, see image, I have taken the photo before it filled the graph. I had to take the photo because everything stopped with the only way to recover is a hard reset.
PXL_20250103_210900914

Langsmith reports:

ResponseError('llama runner process has terminated: error loading model: llama_model_loader: failed to load model from')Traceback (most recent call last):

  File "d:\_Ollama\.env312\Lib\site-packages\langchain_core\runnables\base.py", line 3024, in invoke
    input = context.run(step.invoke, input, config)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "d:\_Ollama\.env312\Lib\site-packages\langchain_core\language_models\chat_models.py", line 286, in invoke
    self.generate_prompt(

  File "d:\_Ollama\.env312\Lib\site-packages\langchain_core\language_models\chat_models.py", line 786, in generate_prompt
    return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "d:\_Ollama\.env312\Lib\site-packages\langchain_core\language_models\chat_models.py", line 643, in generate
    raise e

  File "d:\_Ollama\.env312\Lib\site-packages\langchain_core\language_models\chat_models.py", line 633, in generate
    self._generate_with_cache(

  File "d:\_Ollama\.env312\Lib\site-packages\langchain_core\language_models\chat_models.py", line 851, in _generate_with_cache
    result = self._generate(
             ^^^^^^^^^^^^^^^

  File "d:\_Ollama\.env312\Lib\site-packages\langchain_ollama\chat_models.py", line 644, in _generate
    final_chunk = self._chat_stream_with_aggregation(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "d:\_Ollama\.env312\Lib\site-packages\langchain_ollama\chat_models.py", line 545, in _chat_stream_with_aggregation
    for stream_resp in self._create_chat_stream(messages, stop, **kwargs):

  File "d:\_Ollama\.env312\Lib\site-packages\langchain_ollama\chat_models.py", line 527, in _create_chat_stream
    yield from self._client.chat(

  File "d:\_Ollama\.env312\Lib\site-packages\ollama\_client.py", line 85, in _stream
    raise ResponseError(e.response.text, e.response.status_code) from None

ollama._types.ResponseError: llama runner process has terminated: error loading model: llama_model_loader: failed to load model from

LangSmith metedata:

langchain_core_version: "0.3.28"
langchain_version: "0.3.13"
library: "langchain-core"
library_version: "0.3.28"
platform: "Windows-11-10.0.26100-SP0"
py_implementation: "CPython"
runtime: "python"
runtime_version: "3.12.3"
sdk: "langsmith-py"
sdk_version: "0.1.147"

OS

Windows

GPU

Nvidia

CPU

Intel

Ollama version

0.5.4

Originally created by @Bandit253 on GitHub (Jan 3, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/8299 ### What is the issue? When using from `langchain_community.document_loaders import WebBaseLoader` - (maybe not cause) I get `ResponseError: llama runner process has terminated: error loading model: llama_model_loader: failed to load model from` it doesn't even appear to finish the error message!? but after retying, I get this, see image, I have taken the photo before it filled the graph. I had to take the photo because everything stopped with the only way to recover is a hard reset. ![PXL_20250103_210900914](https://github.com/user-attachments/assets/6f0345fe-ee34-4461-a567-68e7cbf0b1e9) Langsmith reports: ``` ResponseError('llama runner process has terminated: error loading model: llama_model_loader: failed to load model from')Traceback (most recent call last): File "d:\_Ollama\.env312\Lib\site-packages\langchain_core\runnables\base.py", line 3024, in invoke input = context.run(step.invoke, input, config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "d:\_Ollama\.env312\Lib\site-packages\langchain_core\language_models\chat_models.py", line 286, in invoke self.generate_prompt( File "d:\_Ollama\.env312\Lib\site-packages\langchain_core\language_models\chat_models.py", line 786, in generate_prompt return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "d:\_Ollama\.env312\Lib\site-packages\langchain_core\language_models\chat_models.py", line 643, in generate raise e File "d:\_Ollama\.env312\Lib\site-packages\langchain_core\language_models\chat_models.py", line 633, in generate self._generate_with_cache( File "d:\_Ollama\.env312\Lib\site-packages\langchain_core\language_models\chat_models.py", line 851, in _generate_with_cache result = self._generate( ^^^^^^^^^^^^^^^ File "d:\_Ollama\.env312\Lib\site-packages\langchain_ollama\chat_models.py", line 644, in _generate final_chunk = self._chat_stream_with_aggregation( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "d:\_Ollama\.env312\Lib\site-packages\langchain_ollama\chat_models.py", line 545, in _chat_stream_with_aggregation for stream_resp in self._create_chat_stream(messages, stop, **kwargs): File "d:\_Ollama\.env312\Lib\site-packages\langchain_ollama\chat_models.py", line 527, in _create_chat_stream yield from self._client.chat( File "d:\_Ollama\.env312\Lib\site-packages\ollama\_client.py", line 85, in _stream raise ResponseError(e.response.text, e.response.status_code) from None ollama._types.ResponseError: llama runner process has terminated: error loading model: llama_model_loader: failed to load model from ``` LangSmith metedata: ``` langchain_core_version: "0.3.28" langchain_version: "0.3.13" library: "langchain-core" library_version: "0.3.28" platform: "Windows-11-10.0.26100-SP0" py_implementation: "CPython" runtime: "python" runtime_version: "3.12.3" sdk: "langsmith-py" sdk_version: "0.1.147" ``` ### OS Windows ### GPU Nvidia ### CPU Intel ### Ollama version 0.5.4
GiteaMirror added the bug label 2026-04-12 16:30:30 -05:00
Author
Owner

@rick-github commented on GitHub (Jan 3, 2025):

Server logs will aid in debugging. If your langchain app has logs, they would also be useful. The llama_model_loader: failed to load model from error is from llama.cpp, a missing model name seems to indicate that the constructor is instantiated with a broken argument.

Interesting that there is no read/write activity on the drive. https://www.reddit.com/r/techsupport/comments/6ciyje/disk_activity_on_ssd_spikes_to_100_and_freezes_at/

<!-- gh-comment-id:2569938732 --> @rick-github commented on GitHub (Jan 3, 2025): [Server logs](https://github.com/ollama/ollama/blob/main/docs/troubleshooting.md#how-to-troubleshoot-issues) will aid in debugging. If your langchain app has logs, they would also be useful. The `llama_model_loader: failed to load model from` error is from llama.cpp, a missing model name seems to indicate that the constructor is instantiated with a broken argument. Interesting that there is no read/write activity on the drive. https://www.reddit.com/r/techsupport/comments/6ciyje/disk_activity_on_ssd_spikes_to_100_and_freezes_at/
Author
Owner

@Bandit253 commented on GitHub (Jan 4, 2025):

That referenced article so far appears to have solved my problem...
Thank you so much!!

<!-- gh-comment-id:2570041244 --> @Bandit253 commented on GitHub (Jan 4, 2025): That referenced article so far appears to have solved my problem... Thank you so much!!
Author
Owner

@Bandit253 commented on GitHub (Jan 5, 2025):

Oh No! I thought it had been solved, and it was working well then, same thing c drive 100% no real activity.
Hard reset required!

server-1.log

<!-- gh-comment-id:2571459718 --> @Bandit253 commented on GitHub (Jan 5, 2025): Oh No! I thought it had been solved, and it was working well then, same thing c drive 100% no real activity. Hard reset required! [server-1.log](https://github.com/user-attachments/files/18308774/server-1.log)
Author
Owner

@rick-github commented on GitHub (Jan 5, 2025):

time=2025-01-04T14:53:45.579+11:00 level=WARN source=runner.go:129 msg="truncating input prompt" limit=2048 prompt=26491 keep=5 new=2048

The prompt you are sending is larger than the context window, see here, this can cause extra processing but shouldn't cause what you are seeing here.

time=2025-01-04T15:06:13.249+11:00 level=INFO source=server.go:376 msg="starting llama server" cmd="D:\\Ollama\\lib\\ollama\\runners\\cuda_v12\\ollama_llama_server.exe runner --model D:\\Ollama\\models\\blobs\\sha256-74701a8c35f6c8d9a4b91f3f3497643001d63e0c7a84e085bed452548fa88d45 --ctx-size 8192 --batch-size 512 --n-gpu-layers 17 --threads 10 --no-mmap --parallel 4 --port 58776"
...
gguf_init_from_file: failed to open '': 'Invalid argument'
llama_model_load: error loading model: llama_model_loader: failed to load model from 
...
time=2025-01-04T15:06:48.288+11:00 level=INFO source=server.go:376 msg="starting llama server" cmd="D:\\Ollama\\lib\\ollama\\runners\\cuda_v12_avx\\ollama_llama_server.exe runner --model D:\\Ollama\\models\\blobs\\sha256-74701a8c35f6c8d9a4b91f3f3497643001d63e0c7a84e085bed452548fa88d45 --ctx-size 8192 --batch-size 512 --n-gpu-layers 17 --threads 10 --no-mmap --parallel 4 --port 58796"

This is strange, the cuda_v12 runner was started with a --model parameter and crashed with Invalid argument, and then the cuda_v12_avx runner with the same arguments worked 35s later. Not sure what's going on here. Maybe the cuda_v12 runner is broken? You can try setting OLLAMA_LLM_LIBRARY=cuda_v12_avx in the server environment, that will make it skip over the cuda_v12 runner.

<!-- gh-comment-id:2571464672 --> @rick-github commented on GitHub (Jan 5, 2025): ``` time=2025-01-04T14:53:45.579+11:00 level=WARN source=runner.go:129 msg="truncating input prompt" limit=2048 prompt=26491 keep=5 new=2048 ``` The prompt you are sending is larger than the context window, see [here](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-can-i-specify-the-context-window-size), this can cause extra processing but shouldn't cause what you are seeing here. ``` time=2025-01-04T15:06:13.249+11:00 level=INFO source=server.go:376 msg="starting llama server" cmd="D:\\Ollama\\lib\\ollama\\runners\\cuda_v12\\ollama_llama_server.exe runner --model D:\\Ollama\\models\\blobs\\sha256-74701a8c35f6c8d9a4b91f3f3497643001d63e0c7a84e085bed452548fa88d45 --ctx-size 8192 --batch-size 512 --n-gpu-layers 17 --threads 10 --no-mmap --parallel 4 --port 58776" ... gguf_init_from_file: failed to open '': 'Invalid argument' llama_model_load: error loading model: llama_model_loader: failed to load model from ... time=2025-01-04T15:06:48.288+11:00 level=INFO source=server.go:376 msg="starting llama server" cmd="D:\\Ollama\\lib\\ollama\\runners\\cuda_v12_avx\\ollama_llama_server.exe runner --model D:\\Ollama\\models\\blobs\\sha256-74701a8c35f6c8d9a4b91f3f3497643001d63e0c7a84e085bed452548fa88d45 --ctx-size 8192 --batch-size 512 --n-gpu-layers 17 --threads 10 --no-mmap --parallel 4 --port 58796" ``` This is strange, the cuda_v12 runner was started with a `--model` parameter and crashed with `Invalid argument`, and then the cuda_v12_avx runner with the same arguments worked 35s later. Not sure what's going on here. Maybe the cuda_v12 runner is broken? You can try [setting](https://github.com/ollama/ollama/blob/main/docs/faq.md#setting-environment-variables-on-windows) `OLLAMA_LLM_LIBRARY=cuda_v12_avx` in the server environment, that will make it skip over the cuda_v12 runner.
Author
Owner

@Bandit253 commented on GitHub (Jan 5, 2025):

Unfortunately, OLLAMA_LLM_LIBRARY=cuda_v12_avx did not help

<!-- gh-comment-id:2571490267 --> @Bandit253 commented on GitHub (Jan 5, 2025): Unfortunately, `OLLAMA_LLM_LIBRARY=cuda_v12_avx` did not help
Author
Owner

@rick-github commented on GitHub (Jan 5, 2025):

Logs from failure of OLLAMA_LLM_LIBRARY=cuda_v12_avx?

<!-- gh-comment-id:2571490813 --> @rick-github commented on GitHub (Jan 5, 2025): Logs from failure of `OLLAMA_LLM_LIBRARY=cuda_v12_avx`?
Author
Owner

@Bandit253 commented on GitHub (Jan 5, 2025):

The last two logs,
server.log
server-1.log

Thanks of the help

<!-- gh-comment-id:2571496438 --> @Bandit253 commented on GitHub (Jan 5, 2025): The last two logs, [server.log](https://github.com/user-attachments/files/18309135/server.log) [server-1.log](https://github.com/user-attachments/files/18309134/server-1.log) Thanks of the help
Author
Owner

@rick-github commented on GitHub (Jan 5, 2025):

Well, things have improved in that the logs no longer show crashing runners. But there's nothing to indicate any reason for the system getting wedged and requiring a reset.

The thing is that once ollama has loaded the model, there's very little disk activity from the server - pretty much just the occasional log entry. I don't know what would cause 100% disk activity with zero read/write and also make the system unresponsive. From the screenshot, we can see the spike in GPU for the inference and a smaller spike in CPU as is processes the results, and then from that point on GPU and CPU slide away, memory is fairly static, network traffic goes to zero and disk activity goes to 100%. I'm pretty sure this is not directly caused by ollama. Can you give some details about the client that is using ollama? From the logs you are using nomic-embed-text-v1.5 and llama-3.2:3b-instruct-q4_K_M, so I'm guessing this is some sort of RAG system. Do you have a vectorstore backing the app? What sort of post-processing is done? Is the first time using this app, or have you you used it previously without having these issues?

<!-- gh-comment-id:2571517269 --> @rick-github commented on GitHub (Jan 5, 2025): Well, things have improved in that the logs no longer show crashing runners. But there's nothing to indicate any reason for the system getting wedged and requiring a reset. The thing is that once ollama has loaded the model, there's very little disk activity from the server - pretty much just the occasional log entry. I don't know what would cause 100% disk activity with zero read/write and also make the system unresponsive. From the screenshot, we can see the spike in GPU for the inference and a smaller spike in CPU as is processes the results, and then from that point on GPU and CPU slide away, memory is fairly static, network traffic goes to zero and disk activity goes to 100%. I'm pretty sure this is not directly caused by ollama. Can you give some details about the client that is using ollama? From the logs you are using nomic-embed-text-v1.5 and llama-3.2:3b-instruct-q4_K_M, so I'm guessing this is some sort of RAG system. Do you have a vectorstore backing the app? What sort of post-processing is done? Is the first time using this app, or have you you used it previously without having these issues?
Author
Owner

@Bandit253 commented on GitHub (Jan 6, 2025):

A bit of background I was completing an extensive training course which had aspects of all those things: RAG, Tools, vector store, agents and history. The model used most often was llama-3.2:3b and embedding done with nomic-embed-text.
I have now completed that course but intend to develop my own 'complete' application, which will not be quite as intensive, but I will very much be keeping an eye on when and where the problem source, if it occurs again and will likewise pass on my findings.

At this stage it is un-resolved but less critical, I hope.

Thank you for the help and suggestions.

<!-- gh-comment-id:2572424396 --> @Bandit253 commented on GitHub (Jan 6, 2025): A bit of background I was completing an extensive training course which had aspects of all those things: RAG, Tools, vector store, agents and history. The model used most often was llama-3.2:3b and embedding done with nomic-embed-text. I have now completed that course but intend to develop my own 'complete' application, which will not be quite as intensive, but I will very much be keeping an eye on when and where the problem source, if it occurs again and will likewise pass on my findings. At this stage it is un-resolved but less critical, I hope. Thank you for the help and suggestions.
Author
Owner

@rick-github commented on GitHub (Jan 6, 2025):

Closing this for now but feel free to re-open when you have new details.

<!-- gh-comment-id:2572486620 --> @rick-github commented on GitHub (Jan 6, 2025): Closing this for now but feel free to re-open when you have new details.
Author
Owner

@Bandit253 commented on GitHub (Jan 26, 2025):

Well, I thought I would give you an update and as they say in the classics, "It's not you, it's me". Since last report I have tried all sorts of things with essentially similar results:

  • Smaller models - breaks
  • Ollama in Docker worked but was unable to get GPU working - unworkable
  • Ollama in WSL - breaks
  • LM Studio - breaks same way

So, I think it is my hardware, either an issue with my C drive or with my GPU.

If you have any suggestions, happy to hear them. Next move probably replacing c drive.....

<!-- gh-comment-id:2614616246 --> @Bandit253 commented on GitHub (Jan 26, 2025): Well, I thought I would give you an update and as they say in the classics, "It's not you, it's me". Since last report I have tried all sorts of things with essentially similar results: - Smaller models - breaks - Ollama in Docker worked but was unable to get GPU working - unworkable - Ollama in WSL - breaks - LM Studio - breaks same way So, I think it is my hardware, either an issue with my C drive or with my GPU. If you have any suggestions, happy to hear them. Next move probably replacing c drive.....
Author
Owner

@rick-github commented on GitHub (Jan 27, 2025):

Thanks for the follow up. It's interesting that in worked in docker without the GPU. Could you run ollama natively and in your app set "options":{"num_gpu":0}? This will force the model to run in CPU, so it will be slow, but if it doesn't cause the drive issues it would confirm that it's the combination of GPU+drive that's causing the problem. Some sort of PCIe clash or lane saturation? What's your motherboard? Maybe moving the devices to different slots to change the pci lane mapping could help.

<!-- gh-comment-id:2615415469 --> @rick-github commented on GitHub (Jan 27, 2025): Thanks for the follow up. It's interesting that in worked in docker without the GPU. Could you run ollama natively and in your app set `"options":{"num_gpu":0}`? This will force the model to run in CPU, so it will be slow, but if it doesn't cause the drive issues it would confirm that it's the combination of GPU+drive that's causing the problem. Some sort of PCIe clash or lane saturation? What's your motherboard? Maybe moving the devices to different slots to change the pci lane mapping could help.
Author
Owner

@Bandit253 commented on GitHub (Jan 29, 2025):

Well, your suspicion was completely correct 100% CPU run fine if not really slowly but was unable to make it fail, big models small models, just kept working.
Change to using the GPU C drive maxed out and hard reset.

My motherboard is Gigabyte Z490 UD.

When you say "... moving the devices to different slots.." you are meaning physically?

<!-- gh-comment-id:2621084864 --> @Bandit253 commented on GitHub (Jan 29, 2025): Well, your suspicion was completely correct 100% CPU run fine if not really slowly but was unable to make it fail, big models small models, just kept working. Change to using the GPU C drive maxed out and hard reset. My motherboard is Gigabyte Z490 UD. When you say "... moving the devices to different slots.." you are meaning physically?
Author
Owner

@rick-github commented on GitHub (Jan 29, 2025):

When you say "... moving the devices to different slots.." you are meaning physically?

Yes, unfortunately looking at a pic of the board you don't have many options. An RTX3070 is a double-wide card so I assume it's plugged in to the white PCIEx16 slot. D: is SATA so it's either a 2.5/3.5" drive plugged into a SATA connector or it's an M.2 device in M2A_SB under the GPU, C: can be in either M2M_SB or M2A_SB depending on what D: is. Moving devices is going to require extracting the GPU which will be a pain. The simplest route may be getting another SATA drive and using one of the SATA ports (not number 1, see page 16 of the manual) as drive C:.

The BIOS has some options regarding SATA and PCIe devices but I don't think there's much that can be done there. If you've previously tweaked the settings (overclock, undervoltage, etc) setting default values may have an effect.

<!-- gh-comment-id:2621207773 --> @rick-github commented on GitHub (Jan 29, 2025): > When you say "... moving the devices to different slots.." you are meaning physically? Yes, unfortunately looking at a pic of the board you don't have many options. An RTX3070 is a double-wide card so I assume it's plugged in to the white PCIEx16 slot. D: is SATA so it's either a 2.5/3.5" drive plugged into a SATA connector or it's an M.2 device in M2A_SB under the GPU, C: can be in either M2M_SB or M2A_SB depending on what D: is. Moving devices is going to require extracting the GPU which will be a pain. The simplest route may be getting another SATA drive and using one of the SATA ports (not number 1, see page 16 of the manual) as drive C:. The BIOS has some options regarding SATA and PCIe devices but I don't think there's much that can be done there. If you've previously tweaked the settings (overclock, undervoltage, etc) setting default values may have an effect.
Author
Owner

@Bandit253 commented on GitHub (Feb 1, 2025):

Well, thank you so much after a small bit of open case surgery, all is well.
I merely swapped the C drive M2 slots and I have been doing a bit of testing that would have killed the PC prior and here I am still able to type.
You're a campion for going above beyond just the support of Ollama!!

<!-- gh-comment-id:2628704420 --> @Bandit253 commented on GitHub (Feb 1, 2025): Well, thank you so much after a small bit of open case surgery, all is well. I merely swapped the C drive M2 slots and I have been doing a bit of testing that would have killed the PC prior and here I am still able to type. You're a campion for going above beyond just the support of Ollama!!
Author
Owner

@rick-github commented on GitHub (Feb 1, 2025):

Glad you've resolved it, happy inferencing.

<!-- gh-comment-id:2628905927 --> @rick-github commented on GitHub (Feb 1, 2025): Glad you've resolved it, happy inferencing.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#5313