github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-17 05:36:16 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #3870] Failure to Load Llava in Ollama Windows Ver. #48909

New Issue
Closed
opened 2026-04-28 10:00:39 -05:00 by GiteaMirror · 5 comments
GiteaMirror commented 2026-04-28 10:00:39 -05:00
Owner
Copy Link

Originally created by @PasserDreamer on GitHub (Apr 24, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/3870

What is the issue?

I encountered an issue when attempting to load the 'llava' AI model; however, others such as 'Llama3' or 'Phi3' have no problem. Here are the details:

>>ollama run llava
Error: llama runner process no longer running: 1

server.log

...
clip_model_load: CLIP using CUDA backend
clip_model_load: text_encoder:   0
clip_model_load: vision_encoder: 1
clip_model_load: llava_projector:  1
clip_model_load: model size:     595.49 MB
clip_model_load: metadata size:  0.14 MB
clip_model_load: params backend buffer size =  595.49 MB (377 tensors)
cannot open model file for loading tensors
{"function":"load_model","level":"ERR","line":398,"model":"D:\\Llama\\models\\blobs\\sha256-72d6f08a42f656d36b356dbe0920675899a99ce21192fd66266fb7d82ed07539","msg":"unable to load clip model","tid":"162240","timestamp":1713942368}
time=2024-04-24T15:06:08.293+08:00 level=ERROR source=routes.go:120 msg="error loading llama server" error="llama runner process no longer running: 1 "

OS

Windows

GPU

Nvidia

CPU

AMD

Ollama version

0.1.32

Originally created by @PasserDreamer on GitHub (Apr 24, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/3870 ### What is the issue? I encountered an issue when attempting to load the 'llava' AI model; however, others such as 'Llama3' or 'Phi3' have no problem. Here are the details: ``` >>ollama run llava Error: llama runner process no longer running: 1 ``` server.log ``` ... clip_model_load: CLIP using CUDA backend clip_model_load: text_encoder: 0 clip_model_load: vision_encoder: 1 clip_model_load: llava_projector: 1 clip_model_load: model size: 595.49 MB clip_model_load: metadata size: 0.14 MB clip_model_load: params backend buffer size = 595.49 MB (377 tensors) cannot open model file for loading tensors {"function":"load_model","level":"ERR","line":398,"model":"D:\\Llama\\models\\blobs\\sha256-72d6f08a42f656d36b356dbe0920675899a99ce21192fd66266fb7d82ed07539","msg":"unable to load clip model","tid":"162240","timestamp":1713942368} time=2024-04-24T15:06:08.293+08:00 level=ERROR source=routes.go:120 msg="error loading llama server" error="llama runner process no longer running: 1 " ``` ### OS Windows ### GPU Nvidia ### CPU AMD ### Ollama version 0.1.32
GiteaMirror added the bug label 2026-04-28 10:00:39 -05:00
GiteaMirror commented 2026-04-28 10:00:52 -05:00
Author
Owner
Copy Link

@Elminsst commented on GitHub (Apr 25, 2024):

same issue
image

[GIN] 2024/04/27 - 20:56:35 | 500 | 1.5841313s | 127.0.0.1 | POST "/api/chat"
[GIN] 2024/04/27 - 20:56:48 | 200 | 0s | 127.0.0.1 | HEAD "/"
[GIN] 2024/04/27 - 20:56:48 | 200 | 504.6µs | 127.0.0.1 | POST "/api/show"
[GIN] 2024/04/27 - 20:56:48 | 200 | 523.9µs | 127.0.0.1 | POST "/api/show"
time=2024-04-27T20:56:50.001+08:00 level=INFO source=gpu.go:121 msg="Detecting GPU type"
time=2024-04-27T20:56:50.001+08:00 level=INFO source=gpu.go:268 msg="Searching for GPU management library cudart64_.dll"
time=2024-04-27T20:56:50.035+08:00 level=INFO source=gpu.go:314 msg="Discovered GPU libraries: [C:\Users\Elmin\AppData\Local\Programs\Ollama\cudart64_110.dll c:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin\cudart64_110.dll c:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\bin\cudart64_12.dll C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\bin\cudart64_12.dll C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin\cudart64_110.dll C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common\cudart64_65.dll]"
time=2024-04-27T20:56:50.038+08:00 level=INFO source=gpu.go:126 msg="Nvidia GPU detected via cudart"
time=2024-04-27T20:56:50.038+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-04-27T20:56:50.159+08:00 level=INFO source=gpu.go:202 msg="[cudart] CUDART CUDA Compute Capability detected: 8.9"
time=2024-04-27T20:56:50.176+08:00 level=INFO source=gpu.go:121 msg="Detecting GPU type"
time=2024-04-27T20:56:50.176+08:00 level=INFO source=gpu.go:268 msg="Searching for GPU management library cudart64_.dll"
time=2024-04-27T20:56:50.181+08:00 level=INFO source=gpu.go:314 msg="Discovered GPU libraries: [C:\Users\Elmin\AppData\Local\Programs\Ollama\cudart64_110.dll c:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin\cudart64_110.dll c:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\bin\cudart64_12.dll C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\bin\cudart64_12.dll C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin\cudart64_110.dll C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common\cudart64_65.dll]"
time=2024-04-27T20:56:50.181+08:00 level=INFO source=gpu.go:126 msg="Nvidia GPU detected via cudart"
time=2024-04-27T20:56:50.181+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-04-27T20:56:50.237+08:00 level=INFO source=gpu.go:202 msg="[cudart] CUDART CUDA Compute Capability detected: 8.9"
time=2024-04-27T20:56:50.275+08:00 level=INFO source=server.go:127 msg="offload to gpu" reallayers=33 layers=33 required="5708.0 MiB" used="5708.0 MiB" available="15225.0 MiB" kv="512.0 MiB" fulloffload="296.0 MiB" partialoffload="305.0 MiB"
time=2024-04-27T20:56:50.276+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-04-27T20:56:50.287+08:00 level=INFO source=server.go:264 msg="starting llama server" cmd="C:\Users\Elmin\AppData\Local\Temp\ollama2117875548\runners\cuda_v11.3\ollama_llama_server.exe --model D:\AI\语言模型\models\Repository\blobs\sha256-deb26e54ccebc9ce0665bf35dfdc73f2989b7308ecf9b08bd897a9a1ec9cb455 --ctx-size 4096 --batch-size 512 --embedding --log-disable --n-gpu-layers 33 --mmproj D:\AI\语言模型\models\Repository\blobs\sha256-addb9fdda3a5a9ffe3376f97583a2ea160a1050f1393ba4d45fa4a3e6c884867 --port 3382"
time=2024-04-27T20:56:50.291+08:00 level=INFO source=server.go:389 msg="waiting for llama runner to start responding"
{"function":"server_params_parse","level":"INFO","line":2603,"msg":"logging to file is disabled.","tid":"12596","timestamp":1714222610}
{"build":2679,"commit":"7593639","function":"wmain","level":"INFO","line":2820,"msg":"build info","tid":"12596","timestamp":1714222610}
{"function":"wmain","level":"INFO","line":2827,"msg":"system info","n_threads":10,"n_threads_batch":-1,"system_info":"AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 0 | SSSE3 = 0 | VSX = 0 | MATMUL_INT8 = 0 | ","tid":"12596","timestamp":1714222610,"total_threads":20}
{"function":"load_model","level":"INFO","line":395,"msg":"Multi Modal Mode Enabled","tid":"12596","timestamp":1714222610}
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: CUDA_USE_TENSOR_CORES: yes
ggml_cuda_init: found 1 CUDA devices:
Device 0: NVIDIA GeForce RTX 4060 Ti, compute capability 8.9, VMM: yes
time=2024-04-27T20:56:50.344+08:00 level=ERROR source=routes.go:120 msg="error loading llama server" error="llama runner process no longer running: 3221225477 "

<!-- gh-comment-id:2076647887 --> @Elminsst commented on GitHub (Apr 25, 2024): same issue ![image](https://github.com/ollama/ollama/assets/130235860/376a1339-d86f-47a4-b1b9-d6c346904d23) [GIN] 2024/04/27 - 20:56:35 | 500 | 1.5841313s | 127.0.0.1 | POST "/api/chat" [GIN] 2024/04/27 - 20:56:48 | 200 | 0s | 127.0.0.1 | HEAD "/" [GIN] 2024/04/27 - 20:56:48 | 200 | 504.6µs | 127.0.0.1 | POST "/api/show" [GIN] 2024/04/27 - 20:56:48 | 200 | 523.9µs | 127.0.0.1 | POST "/api/show" time=2024-04-27T20:56:50.001+08:00 level=INFO source=gpu.go:121 msg="Detecting GPU type" time=2024-04-27T20:56:50.001+08:00 level=INFO source=gpu.go:268 msg="Searching for GPU management library cudart64_*.dll" time=2024-04-27T20:56:50.035+08:00 level=INFO source=gpu.go:314 msg="Discovered GPU libraries: [C:\\Users\\Elmin\\AppData\\Local\\Programs\\Ollama\\cudart64_110.dll c:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.8\\bin\\cudart64_110.dll c:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v12.1\\bin\\cudart64_12.dll C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v12.1\\bin\\cudart64_12.dll C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.8\\bin\\cudart64_110.dll C:\\Program Files (x86)\\NVIDIA Corporation\\PhysX\\Common\\cudart64_65.dll]" time=2024-04-27T20:56:50.038+08:00 level=INFO source=gpu.go:126 msg="Nvidia GPU detected via cudart" time=2024-04-27T20:56:50.038+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2024-04-27T20:56:50.159+08:00 level=INFO source=gpu.go:202 msg="[cudart] CUDART CUDA Compute Capability detected: 8.9" time=2024-04-27T20:56:50.176+08:00 level=INFO source=gpu.go:121 msg="Detecting GPU type" time=2024-04-27T20:56:50.176+08:00 level=INFO source=gpu.go:268 msg="Searching for GPU management library cudart64_*.dll" time=2024-04-27T20:56:50.181+08:00 level=INFO source=gpu.go:314 msg="Discovered GPU libraries: [C:\\Users\\Elmin\\AppData\\Local\\Programs\\Ollama\\cudart64_110.dll c:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.8\\bin\\cudart64_110.dll c:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v12.1\\bin\\cudart64_12.dll C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v12.1\\bin\\cudart64_12.dll C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.8\\bin\\cudart64_110.dll C:\\Program Files (x86)\\NVIDIA Corporation\\PhysX\\Common\\cudart64_65.dll]" time=2024-04-27T20:56:50.181+08:00 level=INFO source=gpu.go:126 msg="Nvidia GPU detected via cudart" time=2024-04-27T20:56:50.181+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2024-04-27T20:56:50.237+08:00 level=INFO source=gpu.go:202 msg="[cudart] CUDART CUDA Compute Capability detected: 8.9" time=2024-04-27T20:56:50.275+08:00 level=INFO source=server.go:127 msg="offload to gpu" reallayers=33 layers=33 required="5708.0 MiB" used="5708.0 MiB" available="15225.0 MiB" kv="512.0 MiB" fulloffload="296.0 MiB" partialoffload="305.0 MiB" time=2024-04-27T20:56:50.276+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2024-04-27T20:56:50.287+08:00 level=INFO source=server.go:264 msg="starting llama server" cmd="C:\\Users\\Elmin\\AppData\\Local\\Temp\\ollama2117875548\\runners\\cuda_v11.3\\ollama_llama_server.exe --model D:\\AI\\语言模型\\models\\Repository\\blobs\\sha256-deb26e54ccebc9ce0665bf35dfdc73f2989b7308ecf9b08bd897a9a1ec9cb455 --ctx-size 4096 --batch-size 512 --embedding --log-disable --n-gpu-layers 33 --mmproj D:\\AI\\语言模型\\models\\Repository\\blobs\\sha256-addb9fdda3a5a9ffe3376f97583a2ea160a1050f1393ba4d45fa4a3e6c884867 --port 3382" time=2024-04-27T20:56:50.291+08:00 level=INFO source=server.go:389 msg="waiting for llama runner to start responding" {"function":"server_params_parse","level":"INFO","line":2603,"msg":"logging to file is disabled.","tid":"12596","timestamp":1714222610} {"build":2679,"commit":"7593639","function":"wmain","level":"INFO","line":2820,"msg":"build info","tid":"12596","timestamp":1714222610} {"function":"wmain","level":"INFO","line":2827,"msg":"system info","n_threads":10,"n_threads_batch":-1,"system_info":"AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 0 | SSSE3 = 0 | VSX = 0 | MATMUL_INT8 = 0 | ","tid":"12596","timestamp":1714222610,"total_threads":20} {"function":"load_model","level":"INFO","line":395,"msg":"Multi Modal Mode Enabled","tid":"12596","timestamp":1714222610} ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no ggml_cuda_init: CUDA_USE_TENSOR_CORES: yes ggml_cuda_init: found 1 CUDA devices: Device 0: NVIDIA GeForce RTX 4060 Ti, compute capability 8.9, VMM: yes time=2024-04-27T20:56:50.344+08:00 level=ERROR source=routes.go:120 msg="error loading llama server" error="llama runner process no longer running: 3221225477 "
GiteaMirror commented 2026-04-28 10:00:54 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Apr 28, 2024):

The pre-release for 0.1.33 is available now. I'm not sure what the root cause is on this issue, but we've had a number of stability bugs in 0.1.32 on windows, so this might be resolved. Please give it a try and let us know.

<!-- gh-comment-id:2081589905 --> @dhiltgen commented on GitHub (Apr 28, 2024): The pre-release for [0.1.33](https://github.com/ollama/ollama/releases) is available now. I'm not sure what the root cause is on this issue, but we've had a number of stability bugs in 0.1.32 on windows, so this might be resolved. Please give it a try and let us know.
GiteaMirror commented 2026-04-28 10:00:54 -05:00
Author
Owner
Copy Link

@Elminsst commented on GitHub (Apr 29, 2024):

image
I have update it to 0.1.33
but it still can't run

[GIN] 2024/04/29 - 15:04:35 | 200 | 0s | 127.0.0.1 | HEAD "/"
[GIN] 2024/04/29 - 15:04:35 | 200 | 614.6µs | 127.0.0.1 | POST "/api/show"
[GIN] 2024/04/29 - 15:04:35 | 200 | 1.0753ms | 127.0.0.1 | POST "/api/show"
time=2024-04-29T15:04:35.577+08:00 level=INFO source=gpu.go:96 msg="Detecting GPUs"
time=2024-04-29T15:04:35.620+08:00 level=INFO source=gpu.go:101 msg="detected GPUs" library=C:\Users\Elmin\AppData\Local\Programs\Ollama\cudart64_110.dll count=1
time=2024-04-29T15:04:35.620+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-04-29T15:04:36.634+08:00 level=INFO source=memory.go:147 msg="offload to gpu" layers.real=-1 layers.estimate=33 memory.available="15225.0 MiB" memory.required.full="5320.0 MiB" memory.required.partial="5320.0 MiB" memory.required.kv="256.0 MiB" memory.weights.total="3847.6 MiB" memory.weights.repeating="3745.0 MiB" memory.weights.nonrepeating="102.6 MiB" memory.graph.full="164.0 MiB" memory.graph.partial="181.0 MiB"
time=2024-04-29T15:04:36.642+08:00 level=INFO source=memory.go:147 msg="offload to gpu" layers.real=-1 layers.estimate=33 memory.available="15225.0 MiB" memory.required.full="5320.0 MiB" memory.required.partial="5320.0 MiB" memory.required.kv="256.0 MiB" memory.weights.total="3847.6 MiB" memory.weights.repeating="3745.0 MiB" memory.weights.nonrepeating="102.6 MiB" memory.graph.full="164.0 MiB" memory.graph.partial="181.0 MiB"
time=2024-04-29T15:04:36.642+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-04-29T15:04:36.659+08:00 level=INFO source=server.go:290 msg="starting llama server" cmd="C:\Users\Elmin\AppData\Local\Programs\Ollama\ollama_runners\cuda_v11.3\ollama_llama_server.exe --model D:\AI\语言模型\models\Repository\blobs\sha256-170370233dd5c5415250a2ecd5c71586352850729062ccef1496385647293868 --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 33 --mmproj D:\AI\语言模型\models\Repository\blobs\sha256-72d6f08a42f656d36b356dbe0920675899a99ce21192fd66266fb7d82ed07539 --parallel 1 --port 10094"
time=2024-04-29T15:04:36.665+08:00 level=INFO source=sched.go:327 msg="loaded runners" count=1
time=2024-04-29T15:04:36.665+08:00 level=INFO source=server.go:439 msg="waiting for llama runner to start responding"
{"function":"server_params_parse","level":"INFO","line":2603,"msg":"logging to file is disabled.","tid":"25312","timestamp":1714374276}
{"build":2737,"commit":"46e12c4","function":"wmain","level":"INFO","line":2820,"msg":"build info","tid":"25312","timestamp":1714374276}
{"function":"wmain","level":"INFO","line":2827,"msg":"system info","n_threads":10,"n_threads_batch":-1,"system_info":"AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 0 | SSSE3 = 0 | VSX = 0 | MATMUL_INT8 = 0 | LAMMAFILE = 1 | ","tid":"25312","timestamp":1714374276,"total_threads":20}
{"function":"load_model","level":"INFO","line":395,"msg":"Multi Modal Mode Enabled","tid":"25312","timestamp":1714374276}
{"function":"load_model","level":"ERR","line":398,"model":"D:\AI\语言模型\models\Repository\blobs\sha256-72d6f08a42f656d36b356dbe0920675899a99ce21192fd66266fb7d82ed07539","msg":"unable to load clip model","tid":"25312","timestamp":1714374276}
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: CUDA_USE_TENSOR_CORES: yes
ggml_cuda_init: found 1 CUDA devices:
Device 0: NVIDIA GeForce RTX 4060 Ti, compute capability 8.9, VMM: yes
time=2024-04-29T15:04:36.729+08:00 level=ERROR source=sched.go:333 msg="error loading llama server" error="llama runner process no longer running: 1 "
[GIN] 2024/04/29 - 15:04:36 | 500 | 1.1528845s | 127.0.0.1 | POST "/api/chat"
[GIN] 2024/04/29 - 15:04:49 | 200 | 513.7µs | 127.0.0.1 | HEAD "/"
[GIN] 2024/04/29 - 15:04:49 | 200 | 1.0227ms | 127.0.0.1 | POST "/api/show"
[GIN] 2024/04/29 - 15:04:49 | 200 | 509.6µs | 127.0.0.1 | POST "/api/show"
time=2024-04-29T15:04:49.034+08:00 level=INFO source=gpu.go:96 msg="Detecting GPUs"
time=2024-04-29T15:04:49.049+08:00 level=INFO source=gpu.go:101 msg="detected GPUs" library=C:\Users\Elmin\AppData\Local\Programs\Ollama\cudart64_110.dll count=1
time=2024-04-29T15:04:49.049+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-04-29T15:04:50.191+08:00 level=INFO source=memory.go:147 msg="offload to gpu" layers.real=-1 layers.estimate=41 memory.available="15225.0 MiB" memory.required.full="9812.5 MiB" memory.required.partial="9812.5 MiB" memory.required.kv="1600.0 MiB" memory.weights.total="6936.0 MiB" memory.weights.repeating="6807.8 MiB" memory.weights.nonrepeating="128.2 MiB" memory.graph.full="204.0 MiB" memory.graph.partial="244.1 MiB"
time=2024-04-29T15:04:50.198+08:00 level=INFO source=memory.go:147 msg="offload to gpu" layers.real=-1 layers.estimate=41 memory.available="15225.0 MiB" memory.required.full="9812.5 MiB" memory.required.partial="9812.5 MiB" memory.required.kv="1600.0 MiB" memory.weights.total="6936.0 MiB" memory.weights.repeating="6807.8 MiB" memory.weights.nonrepeating="128.2 MiB" memory.graph.full="204.0 MiB" memory.graph.partial="244.1 MiB"
time=2024-04-29T15:04:50.198+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-04-29T15:04:50.213+08:00 level=INFO source=server.go:290 msg="starting llama server" cmd="C:\Users\Elmin\AppData\Local\Programs\Ollama\ollama_runners\cuda_v11.3\ollama_llama_server.exe --model D:\AI\语言模型\models\Repository\blobs\sha256-87d5b13e5157d3a67f8e10a46d8a846ec2b68c1f731e3dfe1546a585432b8fa0 --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 41 --mmproj D:\AI\语言模型\models\Repository\blobs\sha256-42037f9f4c1b801eebaec1545ed144b8b0fa8259672158fb69c8c68f02cfe00c --parallel 1 --port 10111"
time=2024-04-29T15:04:50.218+08:00 level=INFO source=sched.go:327 msg="loaded runners" count=1
time=2024-04-29T15:04:50.218+08:00 level=INFO source=server.go:439 msg="waiting for llama runner to start responding"
{"function":"server_params_parse","level":"INFO","line":2603,"msg":"logging to file is disabled.","tid":"34736","timestamp":1714374290}
{"build":2737,"commit":"46e12c4","function":"wmain","level":"INFO","line":2820,"msg":"build info","tid":"34736","timestamp":1714374290}
{"function":"wmain","level":"INFO","line":2827,"msg":"system info","n_threads":10,"n_threads_batch":-1,"system_info":"AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 0 | SSSE3 = 0 | VSX = 0 | MATMUL_INT8 = 0 | LAMMAFILE = 1 | ","tid":"34736","timestamp":1714374290,"total_threads":20}
{"function":"load_model","level":"INFO","line":395,"msg":"Multi Modal Mode Enabled","tid":"34736","timestamp":1714374290}
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: CUDA_USE_TENSOR_CORES: yes
ggml_cuda_init: found 1 CUDA devices:
Device 0: NVIDIA GeForce RTX 4060 Ti, compute capability 8.9, VMM: yes
{"function":"load_model","level":"ERR","line":398,"model":"D:\AI\语言模型\models\Repository\blobs\sha256-42037f9f4c1b801eebaec1545ed144b8b0fa8259672158fb69c8c68f02cfe00c","msg":"unable to load clip model","tid":"34736","timestamp":1714374290}
time=2024-04-29T15:04:50.278+08:00 level=ERROR source=sched.go:333 msg="error loading llama server" error="llama runner process no longer running: 1 "
[GIN] 2024/04/29 - 15:04:50 | 500 | 1.2458195s | 127.0.0.1 | POST "/api/chat"

<!-- gh-comment-id:2082023327 --> @Elminsst commented on GitHub (Apr 29, 2024): ![image](https://github.com/ollama/ollama/assets/130235860/98fa66e5-9df9-44ad-bcb3-6424f534592c) I have update it to 0.1.33 but it still can't run [GIN] 2024/04/29 - 15:04:35 | 200 | 0s | 127.0.0.1 | HEAD "/" [GIN] 2024/04/29 - 15:04:35 | 200 | 614.6µs | 127.0.0.1 | POST "/api/show" [GIN] 2024/04/29 - 15:04:35 | 200 | 1.0753ms | 127.0.0.1 | POST "/api/show" time=2024-04-29T15:04:35.577+08:00 level=INFO source=gpu.go:96 msg="Detecting GPUs" time=2024-04-29T15:04:35.620+08:00 level=INFO source=gpu.go:101 msg="detected GPUs" library=C:\Users\Elmin\AppData\Local\Programs\Ollama\cudart64_110.dll count=1 time=2024-04-29T15:04:35.620+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2024-04-29T15:04:36.634+08:00 level=INFO source=memory.go:147 msg="offload to gpu" layers.real=-1 layers.estimate=33 memory.available="15225.0 MiB" memory.required.full="5320.0 MiB" memory.required.partial="5320.0 MiB" memory.required.kv="256.0 MiB" memory.weights.total="3847.6 MiB" memory.weights.repeating="3745.0 MiB" memory.weights.nonrepeating="102.6 MiB" memory.graph.full="164.0 MiB" memory.graph.partial="181.0 MiB" time=2024-04-29T15:04:36.642+08:00 level=INFO source=memory.go:147 msg="offload to gpu" layers.real=-1 layers.estimate=33 memory.available="15225.0 MiB" memory.required.full="5320.0 MiB" memory.required.partial="5320.0 MiB" memory.required.kv="256.0 MiB" memory.weights.total="3847.6 MiB" memory.weights.repeating="3745.0 MiB" memory.weights.nonrepeating="102.6 MiB" memory.graph.full="164.0 MiB" memory.graph.partial="181.0 MiB" time=2024-04-29T15:04:36.642+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2024-04-29T15:04:36.659+08:00 level=INFO source=server.go:290 msg="starting llama server" cmd="C:\\Users\\Elmin\\AppData\\Local\\Programs\\Ollama\\ollama_runners\\cuda_v11.3\\ollama_llama_server.exe --model D:\\AI\\语言模型\\models\\Repository\\blobs\\sha256-170370233dd5c5415250a2ecd5c71586352850729062ccef1496385647293868 --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 33 --mmproj D:\\AI\\语言模型\\models\\Repository\\blobs\\sha256-72d6f08a42f656d36b356dbe0920675899a99ce21192fd66266fb7d82ed07539 --parallel 1 --port 10094" time=2024-04-29T15:04:36.665+08:00 level=INFO source=sched.go:327 msg="loaded runners" count=1 time=2024-04-29T15:04:36.665+08:00 level=INFO source=server.go:439 msg="waiting for llama runner to start responding" {"function":"server_params_parse","level":"INFO","line":2603,"msg":"logging to file is disabled.","tid":"25312","timestamp":1714374276} {"build":2737,"commit":"46e12c4","function":"wmain","level":"INFO","line":2820,"msg":"build info","tid":"25312","timestamp":1714374276} {"function":"wmain","level":"INFO","line":2827,"msg":"system info","n_threads":10,"n_threads_batch":-1,"system_info":"AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 0 | SSSE3 = 0 | VSX = 0 | MATMUL_INT8 = 0 | LAMMAFILE = 1 | ","tid":"25312","timestamp":1714374276,"total_threads":20} {"function":"load_model","level":"INFO","line":395,"msg":"Multi Modal Mode Enabled","tid":"25312","timestamp":1714374276} {"function":"load_model","level":"ERR","line":398,"model":"D:\\AI\\语言模型\\models\\Repository\\blobs\\sha256-72d6f08a42f656d36b356dbe0920675899a99ce21192fd66266fb7d82ed07539","msg":"unable to load clip model","tid":"25312","timestamp":1714374276} ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no ggml_cuda_init: CUDA_USE_TENSOR_CORES: yes ggml_cuda_init: found 1 CUDA devices: Device 0: NVIDIA GeForce RTX 4060 Ti, compute capability 8.9, VMM: yes time=2024-04-29T15:04:36.729+08:00 level=ERROR source=sched.go:333 msg="error loading llama server" error="llama runner process no longer running: 1 " [GIN] 2024/04/29 - 15:04:36 | 500 | 1.1528845s | 127.0.0.1 | POST "/api/chat" [GIN] 2024/04/29 - 15:04:49 | 200 | 513.7µs | 127.0.0.1 | HEAD "/" [GIN] 2024/04/29 - 15:04:49 | 200 | 1.0227ms | 127.0.0.1 | POST "/api/show" [GIN] 2024/04/29 - 15:04:49 | 200 | 509.6µs | 127.0.0.1 | POST "/api/show" time=2024-04-29T15:04:49.034+08:00 level=INFO source=gpu.go:96 msg="Detecting GPUs" time=2024-04-29T15:04:49.049+08:00 level=INFO source=gpu.go:101 msg="detected GPUs" library=C:\Users\Elmin\AppData\Local\Programs\Ollama\cudart64_110.dll count=1 time=2024-04-29T15:04:49.049+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2024-04-29T15:04:50.191+08:00 level=INFO source=memory.go:147 msg="offload to gpu" layers.real=-1 layers.estimate=41 memory.available="15225.0 MiB" memory.required.full="9812.5 MiB" memory.required.partial="9812.5 MiB" memory.required.kv="1600.0 MiB" memory.weights.total="6936.0 MiB" memory.weights.repeating="6807.8 MiB" memory.weights.nonrepeating="128.2 MiB" memory.graph.full="204.0 MiB" memory.graph.partial="244.1 MiB" time=2024-04-29T15:04:50.198+08:00 level=INFO source=memory.go:147 msg="offload to gpu" layers.real=-1 layers.estimate=41 memory.available="15225.0 MiB" memory.required.full="9812.5 MiB" memory.required.partial="9812.5 MiB" memory.required.kv="1600.0 MiB" memory.weights.total="6936.0 MiB" memory.weights.repeating="6807.8 MiB" memory.weights.nonrepeating="128.2 MiB" memory.graph.full="204.0 MiB" memory.graph.partial="244.1 MiB" time=2024-04-29T15:04:50.198+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2024-04-29T15:04:50.213+08:00 level=INFO source=server.go:290 msg="starting llama server" cmd="C:\\Users\\Elmin\\AppData\\Local\\Programs\\Ollama\\ollama_runners\\cuda_v11.3\\ollama_llama_server.exe --model D:\\AI\\语言模型\\models\\Repository\\blobs\\sha256-87d5b13e5157d3a67f8e10a46d8a846ec2b68c1f731e3dfe1546a585432b8fa0 --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 41 --mmproj D:\\AI\\语言模型\\models\\Repository\\blobs\\sha256-42037f9f4c1b801eebaec1545ed144b8b0fa8259672158fb69c8c68f02cfe00c --parallel 1 --port 10111" time=2024-04-29T15:04:50.218+08:00 level=INFO source=sched.go:327 msg="loaded runners" count=1 time=2024-04-29T15:04:50.218+08:00 level=INFO source=server.go:439 msg="waiting for llama runner to start responding" {"function":"server_params_parse","level":"INFO","line":2603,"msg":"logging to file is disabled.","tid":"34736","timestamp":1714374290} {"build":2737,"commit":"46e12c4","function":"wmain","level":"INFO","line":2820,"msg":"build info","tid":"34736","timestamp":1714374290} {"function":"wmain","level":"INFO","line":2827,"msg":"system info","n_threads":10,"n_threads_batch":-1,"system_info":"AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 0 | SSSE3 = 0 | VSX = 0 | MATMUL_INT8 = 0 | LAMMAFILE = 1 | ","tid":"34736","timestamp":1714374290,"total_threads":20} {"function":"load_model","level":"INFO","line":395,"msg":"Multi Modal Mode Enabled","tid":"34736","timestamp":1714374290} ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no ggml_cuda_init: CUDA_USE_TENSOR_CORES: yes ggml_cuda_init: found 1 CUDA devices: Device 0: NVIDIA GeForce RTX 4060 Ti, compute capability 8.9, VMM: yes {"function":"load_model","level":"ERR","line":398,"model":"D:\\AI\\语言模型\\models\\Repository\\blobs\\sha256-42037f9f4c1b801eebaec1545ed144b8b0fa8259672158fb69c8c68f02cfe00c","msg":"unable to load clip model","tid":"34736","timestamp":1714374290} time=2024-04-29T15:04:50.278+08:00 level=ERROR source=sched.go:333 msg="error loading llama server" error="llama runner process no longer running: 1 " [GIN] 2024/04/29 - 15:04:50 | 500 | 1.2458195s | 127.0.0.1 | POST "/api/chat"
GiteaMirror commented 2026-04-28 10:00:55 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (May 5, 2024):

I haven't been able to reproduce this failure. I've tried on a Win 11 system with 0.1.33 and both llava and llava:13b run correctly on an Nvidia GPU.

@PasserDreamer does upgrading to 0.1.33 resolve the problem for you?

@Elminsst do smaller models work for you? Is it possible your system is running out of regular RAM while loading the models leading to a crash? Can you check Task Manager and see if there's anything else going on when this happens? Maybe an AV product is killing the process?

<!-- gh-comment-id:2094506903 --> @dhiltgen commented on GitHub (May 5, 2024): I haven't been able to reproduce this failure. I've tried on a Win 11 system with 0.1.33 and both llava and llava:13b run correctly on an Nvidia GPU. @PasserDreamer does upgrading to 0.1.33 resolve the problem for you? @Elminsst do smaller models work for you? Is it possible your system is running out of regular RAM while loading the models leading to a crash? Can you check Task Manager and see if there's anything else going on when this happens? Maybe an AV product is killing the process?
GiteaMirror commented 2026-04-28 10:00:56 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Oct 23, 2024):

If you're still having problems, please make sure you've upgraded to the latest version and if that doesn't clear it up, please share an updated server log and I'll reopen.

<!-- gh-comment-id:2433156790 --> @dhiltgen commented on GitHub (Oct 23, 2024): If you're still having problems, please make sure you've upgraded to the latest version and if that doesn't clear it up, please share an updated server log and I'll reopen.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#48909
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?