github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 08:02:14 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #14774] ollama 0.17.7 VRAM #56062

New Issue
Closed
opened 2026-04-29 10:13:08 -05:00 by GiteaMirror · 4 comments
GiteaMirror commented 2026-04-29 10:13:08 -05:00
Owner
Copy Link

Originally created by @sy960923 on GitHub (Mar 11, 2026).
Original GitHub issue: https://github.com/ollama/ollama/issues/14774

ollama 0.17.7版本 比ollama 0.9.5版本 相同模型，为什么相同并发下 显存占用高很多？

Originally created by @sy960923 on GitHub (Mar 11, 2026). Original GitHub issue: https://github.com/ollama/ollama/issues/14774 ollama 0.17.7版本 比ollama 0.9.5版本 相同模型，为什么相同并发下 显存占用高很多？
GiteaMirror commented 2026-04-29 10:13:09 -05:00
Author
Owner
Copy Link

@sy960923 commented on GitHub (Mar 11, 2026):

0.17.7 启动 qwne3:32b 日志

time=2026-03-11T03:02:00.576Z level=INFO source=routes.go:1658 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:0 OLLAMA_DEBUG:INFO OLLAMA_EDITOR: OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434/ OLLAMA_KEEP_ALIVE:2562047h47m16.854775807s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:1 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/root/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NO_CLOUD:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost/ https://localhost/ http://localhost/:* https://localhost/:* http://127.0.0.1/ https://127.0.0.1/ http://127.0.0.1/:* https://127.0.0.1/:* http://0.0.0.0/ https://0.0.0.0/ http://0.0.0.0/:* https://0.0.0.0/:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[[ollama.com](https://ollama.com/)] OLLAMA_SCHED_SPREAD:false OLLAMA_VULKAN:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"
time=2026-03-11T03:02:00.576Z level=INFO source=routes.go:1660 msg="Ollama cloud disabled: false"
time=2026-03-11T03:02:00.577Z level=INFO source=images.go:477 msg="total blobs: 24"
time=2026-03-11T03:02:00.577Z level=INFO source=images.go:484 msg="total unused blobs removed: 0"
time=2026-03-11T03:02:00.577Z level=INFO source=routes.go:1713 msg="Listening on [::]:11434 (version 0.17.7)"
time=2026-03-11T03:02:00.579Z level=INFO source=runner.go:67 msg="discovering available GPUs..."
time=2026-03-11T03:02:00.580Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 35037"
time=2026-03-11T03:02:00.661Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 46649"
time=2026-03-11T03:02:00.789Z level=INFO source=runner.go:106 msg="experimental Vulkan support disabled.  To enable, set OLLAMA_VULKAN=1"
time=2026-03-11T03:02:00.789Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 44775"
time=2026-03-11T03:02:00.867Z level=INFO source=types.go:42 msg="inference compute" id=GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 filter_id="" library=CUDA compute=8.6 name=CUDA0 description="NVIDIA RTX A6000" libdirs=ollama,cuda_v12 driver=12.4 pci_id=0000:17:00.0 type=discrete total="48.0 GiB" available="31.4 GiB"
time=2026-03-11T03:02:00.867Z level=INFO source=routes.go:1763 msg="vram-based default context" total_vram="48.0 GiB" default_num_ctx=262144
[GIN] 2026/03/11 - 03:02:12 | 200 |      47.736µs |       127.0.0.1 | HEAD     "/"
[GIN] 2026/03/11 - 03:02:12 | 200 |     836.993µs |       127.0.0.1 | GET      "/api/tags"
[GIN] 2026/03/11 - 03:02:19 | 200 |      31.779µs |       127.0.0.1 | HEAD     "/"
[GIN] 2026/03/11 - 03:02:20 | 200 |  212.596846ms |       127.0.0.1 | POST     "/api/show"
[GIN] 2026/03/11 - 03:02:20 | 200 |  209.781594ms |       127.0.0.1 | POST     "/api/show"
time=2026-03-11T03:02:20.472Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 44721"
time=2026-03-11T03:02:20.553Z level=WARN source=cpu_linux.go:130 msg="failed to parse CPU allowed micro secs" error="strconv.ParseInt: parsing \"max\": invalid syntax"
time=2026-03-11T03:02:20.699Z level=INFO source=server.go:246 msg="enabling flash attention"
time=2026-03-11T03:02:20.699Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --model /root/.ollama/models/blobs/sha256-d4b8b4f4c350f5d322dc8235175eeae02d32c6f3fd70bdb9ea481e3abb7d7fc4 --port 45551"
time=2026-03-11T03:02:20.700Z level=INFO source=sched.go:489 msg="system memory" total="124.8 GiB" free="124.6 GiB" free_swap="127.5 GiB"
time=2026-03-11T03:02:20.700Z level=INFO source=sched.go:496 msg="gpu memory" id=GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 library=CUDA available="31.0 GiB" free="31.4 GiB" minimum="457.0 MiB" overhead="0 B"
time=2026-03-11T03:02:20.700Z level=INFO source=server.go:757 msg="loading model" "model layers"=65 requested=-1
time=2026-03-11T03:02:20.712Z level=INFO source=runner.go:1429 msg="starting ollama engine"
time=2026-03-11T03:02:20.712Z level=INFO source=runner.go:1464 msg="Server listening on 127.0.0.1:45551"
time=2026-03-11T03:02:20.721Z level=INFO source=runner.go:1302 msg=load request="{Operation:fit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:262144 KvCacheType: NumThreads:24 GPULayers:65[ID:GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 Layers:65(0..64)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
time=2026-03-11T03:02:20.775Z level=INFO source=ggml.go:136 msg="" architecture=qwen35 file_type=Q4_K_M name="" description="" num_tensors=1307 num_key_values=53
load_backend: loaded CPU backend from /usr/lib/ollama/libggml-cpu-icelake.so
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA RTX A6000, compute capability 8.6, VMM: yes, ID: GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1
load_backend: loaded CUDA backend from /usr/lib/ollama/cuda_v12/libggml-cuda.so
time=2026-03-11T03:02:20.819Z level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.AVX512=1 CPU.0.AVX512_VBMI=1 CPU.0.AVX512_VNNI=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,520,600,610,700,750,800,860,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(gcc)
time=2026-03-11T03:02:23.093Z level=INFO source=runner.go:1302 msg=load request="{Operation:fit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:262144 KvCacheType: NumThreads:24 GPULayers:52[ID:GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 Layers:52(12..63)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
time=2026-03-11T03:02:25.152Z level=INFO source=runner.go:1302 msg=load request="{Operation:fit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:262144 KvCacheType: NumThreads:24 GPULayers:50[ID:GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 Layers:50(14..63)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
time=2026-03-11T03:02:27.147Z level=INFO source=runner.go:1302 msg=load request="{Operation:alloc LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:262144 KvCacheType: NumThreads:24 GPULayers:50[ID:GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 Layers:50(14..63)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
ggml_backend_cuda_buffer_type_alloc_buffer: allocating 4365.34 MiB on device 0: cudaMalloc failed: out of memory
ggml_gallocr_reserve_n_impl: failed to allocate CUDA0 buffer of size 4577394688
time=2026-03-11T03:02:31.141Z level=INFO source=server.go:879 msg="model layout did not fit, applying backoff" backoff=0.10
time=2026-03-11T03:02:31.142Z level=INFO source=runner.go:1302 msg=load request="{Operation:alloc LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:262144 KvCacheType: NumThreads:24 GPULayers:44[ID:GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 Layers:44(20..63)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
time=2026-03-11T03:02:36.861Z level=INFO source=runner.go:1302 msg=load request="{Operation:commit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:262144 KvCacheType: NumThreads:24 GPULayers:44[ID:GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 Layers:44(20..63)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
time=2026-03-11T03:02:36.861Z level=INFO source=ggml.go:482 msg="offloading 44 repeating layers to GPU"
time=2026-03-11T03:02:36.861Z level=INFO source=ggml.go:486 msg="offloading output layer to CPU"
time=2026-03-11T03:02:36.861Z level=INFO source=ggml.go:494 msg="offloaded 44/65 layers to GPU"
time=2026-03-11T03:02:36.861Z level=INFO source=device.go:240 msg="model weights" device=CUDA0 size="9.2 GiB"
time=2026-03-11T03:02:36.861Z level=INFO source=device.go:245 msg="model weights" device=CPU size="7.0 GiB"
time=2026-03-11T03:02:36.861Z level=INFO source=device.go:251 msg="kv cache" device=CUDA0 size="13.5 GiB"
time=2026-03-11T03:02:36.861Z level=INFO source=device.go:256 msg="kv cache" device=CPU size="6.1 GiB"
time=2026-03-11T03:02:36.861Z level=INFO source=device.go:262 msg="compute graph" device=CUDA0 size="4.6 GiB"
time=2026-03-11T03:02:36.861Z level=INFO source=device.go:267 msg="compute graph" device=CPU size="952.3 MiB"
time=2026-03-11T03:02:36.861Z level=INFO source=device.go:272 msg="total memory" size="41.4 GiB"
time=2026-03-11T03:02:36.861Z level=INFO source=sched.go:565 msg="loaded runners" count=1
time=2026-03-11T03:02:36.861Z level=INFO source=server.go:1350 msg="waiting for llama runner to start responding"
time=2026-03-11T03:02:36.862Z level=INFO source=server.go:1384 msg="waiting for server to become available" status="llm server loading model"
time=2026-03-11T03:02:44.635Z level=INFO source=server.go:1388 msg="llama runner started in 23.94 seconds"
[GIN] 2026/03/11 - 03:02:44 | 200 | 24.409871186s |       127.0.0.1 | POST     "/api/generate"
[GIN] 2026/03/11 - 03:03:37 | 200 |       52.64µs |       127.0.0.1 | HEAD     "/"
[GIN] 2026/03/11 - 03:03:37 | 200 |  215.183312ms |       127.0.0.1 | POST     "/api/show"
[GIN] 2026/03/11 - 03:03:37 | 200 |     693.633µs |       127.0.0.1 | POST     "/api/generate"
ggml_backend_cuda_device_get_memory device GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 utilizing NVML memory reporting free: 3141533696 total: 51527024640
time=2026-03-11T03:03:37.730Z level=ERROR source=server.go:303 msg="llama runner terminated" error="signal: killed"
time=2026-03-11T03:03:40.598Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 38139"
time=2026-03-11T03:10:10.432Z level=INFO source=routes.go:1658 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:0 OLLAMA_DEBUG:INFO OLLAMA_EDITOR: OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434/ OLLAMA_KEEP_ALIVE:2562047h47m16.854775807s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:1 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/root/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NO_CLOUD:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost/ https://localhost/ http://localhost/:* https://localhost/:* http://127.0.0.1/ https://127.0.0.1/ http://127.0.0.1/:* https://127.0.0.1/:* http://0.0.0.0/ https://0.0.0.0/ http://0.0.0.0/:* https://0.0.0.0/:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[[ollama.com](https://ollama.com/)] OLLAMA_SCHED_SPREAD:false OLLAMA_VULKAN:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"
time=2026-03-11T03:10:10.432Z level=INFO source=routes.go:1660 msg="Ollama cloud disabled: false"
time=2026-03-11T03:10:10.433Z level=INFO source=images.go:477 msg="total blobs: 24"
time=2026-03-11T03:10:10.433Z level=INFO source=images.go:484 msg="total unused blobs removed: 0"
time=2026-03-11T03:10:10.433Z level=INFO source=routes.go:1713 msg="Listening on [::]:11434 (version 0.17.7)"
time=2026-03-11T03:10:10.434Z level=INFO source=runner.go:67 msg="discovering available GPUs..."
time=2026-03-11T03:10:10.435Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 32817"
time=2026-03-11T03:10:10.514Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 41789"
time=2026-03-11T03:10:10.578Z level=INFO source=runner.go:106 msg="experimental Vulkan support disabled.  To enable, set OLLAMA_VULKAN=1"
time=2026-03-11T03:10:10.578Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 44083"
time=2026-03-11T03:10:10.655Z level=INFO source=types.go:42 msg="inference compute" id=GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 filter_id="" library=CUDA compute=8.6 name=CUDA0 description="NVIDIA RTX A6000" libdirs=ollama,cuda_v12 driver=12.4 pci_id=0000:17:00.0 type=discrete total="48.0 GiB" available="31.4 GiB"
time=2026-03-11T03:10:10.655Z level=INFO source=routes.go:1763 msg="vram-based default context" total_vram="48.0 GiB" default_num_ctx=262144
[GIN] 2026/03/11 - 03:10:18 | 200 |      51.236µs |       127.0.0.1 | HEAD     "/"
[GIN] 2026/03/11 - 03:10:18 | 200 |  110.201924ms |       127.0.0.1 | POST     "/api/show"
[GIN] 2026/03/11 - 03:10:18 | 200 |  110.707116ms |       127.0.0.1 | POST     "/api/show"
time=2026-03-11T03:10:19.006Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 43919"
time=2026-03-11T03:10:19.085Z level=WARN source=cpu_linux.go:130 msg="failed to parse CPU allowed micro secs" error="strconv.ParseInt: parsing \"max\": invalid syntax"
time=2026-03-11T03:10:19.160Z level=WARN source=server.go:168 msg="requested context size too large for model" num_ctx=262144 n_ctx_train=40960
time=2026-03-11T03:10:19.160Z level=INFO source=server.go:246 msg="enabling flash attention"
time=2026-03-11T03:10:19.160Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --model /root/.ollama/models/blobs/sha256-3291abe70f16ee9682de7bfae08db5373ea9d6497e614aaad63340ad421d6312 --port 45799"
time=2026-03-11T03:10:19.160Z level=INFO source=sched.go:489 msg="system memory" total="124.8 GiB" free="124.7 GiB" free_swap="127.5 GiB"
time=2026-03-11T03:10:19.160Z level=INFO source=sched.go:496 msg="gpu memory" id=GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 library=CUDA available="31.0 GiB" free="31.4 GiB" minimum="457.0 MiB" overhead="0 B"
time=2026-03-11T03:10:19.160Z level=INFO source=server.go:757 msg="loading model" "model layers"=65 requested=-1
time=2026-03-11T03:10:19.173Z level=INFO source=runner.go:1429 msg="starting ollama engine"
time=2026-03-11T03:10:19.173Z level=INFO source=runner.go:1464 msg="Server listening on 127.0.0.1:45799"
time=2026-03-11T03:10:19.182Z level=INFO source=runner.go:1302 msg=load request="{Operation:fit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:40960 KvCacheType: NumThreads:24 GPULayers:65[ID:GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 Layers:65(0..64)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
time=2026-03-11T03:10:19.211Z level=INFO source=ggml.go:136 msg="" architecture=qwen3 file_type=Q4_K_M name="Qwen3 32B" description="" num_tensors=707 num_key_values=28
load_backend: loaded CPU backend from /usr/lib/ollama/libggml-cpu-icelake.so
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA RTX A6000, compute capability 8.6, VMM: yes, ID: GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1
load_backend: loaded CUDA backend from /usr/lib/ollama/cuda_v12/libggml-cuda.so
time=2026-03-11T03:10:19.255Z level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.AVX512=1 CPU.0.AVX512_VBMI=1 CPU.0.AVX512_VNNI=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,520,600,610,700,750,800,860,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(gcc)
time=2026-03-11T03:10:19.476Z level=INFO source=runner.go:1302 msg=load request="{Operation:alloc LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:40960 KvCacheType: NumThreads:24 GPULayers:65[ID:GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 Layers:65(0..64)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
time=2026-03-11T03:10:19.640Z level=INFO source=runner.go:1302 msg=load request="{Operation:commit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:40960 KvCacheType: NumThreads:24 GPULayers:65[ID:GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 Layers:65(0..64)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
time=2026-03-11T03:10:19.640Z level=INFO source=device.go:240 msg="model weights" device=CUDA0 size="18.4 GiB"
time=2026-03-11T03:10:19.640Z level=INFO source=device.go:245 msg="model weights" device=CPU size="417.3 MiB"
time=2026-03-11T03:10:19.640Z level=INFO source=device.go:251 msg="kv cache" device=CUDA0 size="10.0 GiB"
time=2026-03-11T03:10:19.640Z level=INFO source=device.go:262 msg="compute graph" device=CUDA0 size="404.0 MiB"
time=2026-03-11T03:10:19.640Z level=INFO source=device.go:267 msg="compute graph" device=CPU size="10.0 MiB"
time=2026-03-11T03:10:19.640Z level=INFO source=device.go:272 msg="total memory" size="29.2 GiB"
time=2026-03-11T03:10:19.640Z level=INFO source=sched.go:565 msg="loaded runners" count=1
time=2026-03-11T03:10:19.640Z level=INFO source=server.go:1350 msg="waiting for llama runner to start responding"
time=2026-03-11T03:10:19.640Z level=INFO source=ggml.go:482 msg="offloading 64 repeating layers to GPU"
time=2026-03-11T03:10:19.640Z level=INFO source=ggml.go:489 msg="offloading output layer to GPU"
time=2026-03-11T03:10:19.640Z level=INFO source=ggml.go:494 msg="offloaded 65/65 layers to GPU"
time=2026-03-11T03:10:19.641Z level=INFO source=server.go:1384 msg="waiting for server to become available" status="llm server loading model"
time=2026-03-11T03:10:22.148Z level=INFO source=server.go:1388 msg="llama runner started in 2.99 seconds"
<!-- gh-comment-id:4035912073 --> @sy960923 commented on GitHub (Mar 11, 2026): 0.17.7 启动 qwne3:32b 日志 ``` time=2026-03-11T03:02:00.576Z level=INFO source=routes.go:1658 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:0 OLLAMA_DEBUG:INFO OLLAMA_EDITOR: OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434/ OLLAMA_KEEP_ALIVE:2562047h47m16.854775807s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:1 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/root/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NO_CLOUD:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost/ https://localhost/ http://localhost/:* https://localhost/:* http://127.0.0.1/ https://127.0.0.1/ http://127.0.0.1/:* https://127.0.0.1/:* http://0.0.0.0/ https://0.0.0.0/ http://0.0.0.0/:* https://0.0.0.0/:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[[ollama.com](https://ollama.com/)] OLLAMA_SCHED_SPREAD:false OLLAMA_VULKAN:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]" time=2026-03-11T03:02:00.576Z level=INFO source=routes.go:1660 msg="Ollama cloud disabled: false" time=2026-03-11T03:02:00.577Z level=INFO source=images.go:477 msg="total blobs: 24" time=2026-03-11T03:02:00.577Z level=INFO source=images.go:484 msg="total unused blobs removed: 0" time=2026-03-11T03:02:00.577Z level=INFO source=routes.go:1713 msg="Listening on [::]:11434 (version 0.17.7)" time=2026-03-11T03:02:00.579Z level=INFO source=runner.go:67 msg="discovering available GPUs..." time=2026-03-11T03:02:00.580Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 35037" time=2026-03-11T03:02:00.661Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 46649" time=2026-03-11T03:02:00.789Z level=INFO source=runner.go:106 msg="experimental Vulkan support disabled. To enable, set OLLAMA_VULKAN=1" time=2026-03-11T03:02:00.789Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 44775" time=2026-03-11T03:02:00.867Z level=INFO source=types.go:42 msg="inference compute" id=GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 filter_id="" library=CUDA compute=8.6 name=CUDA0 description="NVIDIA RTX A6000" libdirs=ollama,cuda_v12 driver=12.4 pci_id=0000:17:00.0 type=discrete total="48.0 GiB" available="31.4 GiB" time=2026-03-11T03:02:00.867Z level=INFO source=routes.go:1763 msg="vram-based default context" total_vram="48.0 GiB" default_num_ctx=262144 [GIN] 2026/03/11 - 03:02:12 | 200 | 47.736µs | 127.0.0.1 | HEAD "/" [GIN] 2026/03/11 - 03:02:12 | 200 | 836.993µs | 127.0.0.1 | GET "/api/tags" [GIN] 2026/03/11 - 03:02:19 | 200 | 31.779µs | 127.0.0.1 | HEAD "/" [GIN] 2026/03/11 - 03:02:20 | 200 | 212.596846ms | 127.0.0.1 | POST "/api/show" [GIN] 2026/03/11 - 03:02:20 | 200 | 209.781594ms | 127.0.0.1 | POST "/api/show" time=2026-03-11T03:02:20.472Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 44721" time=2026-03-11T03:02:20.553Z level=WARN source=cpu_linux.go:130 msg="failed to parse CPU allowed micro secs" error="strconv.ParseInt: parsing \"max\": invalid syntax" time=2026-03-11T03:02:20.699Z level=INFO source=server.go:246 msg="enabling flash attention" time=2026-03-11T03:02:20.699Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --model /root/.ollama/models/blobs/sha256-d4b8b4f4c350f5d322dc8235175eeae02d32c6f3fd70bdb9ea481e3abb7d7fc4 --port 45551" time=2026-03-11T03:02:20.700Z level=INFO source=sched.go:489 msg="system memory" total="124.8 GiB" free="124.6 GiB" free_swap="127.5 GiB" time=2026-03-11T03:02:20.700Z level=INFO source=sched.go:496 msg="gpu memory" id=GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 library=CUDA available="31.0 GiB" free="31.4 GiB" minimum="457.0 MiB" overhead="0 B" time=2026-03-11T03:02:20.700Z level=INFO source=server.go:757 msg="loading model" "model layers"=65 requested=-1 time=2026-03-11T03:02:20.712Z level=INFO source=runner.go:1429 msg="starting ollama engine" time=2026-03-11T03:02:20.712Z level=INFO source=runner.go:1464 msg="Server listening on 127.0.0.1:45551" time=2026-03-11T03:02:20.721Z level=INFO source=runner.go:1302 msg=load request="{Operation:fit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:262144 KvCacheType: NumThreads:24 GPULayers:65[ID:GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 Layers:65(0..64)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}" time=2026-03-11T03:02:20.775Z level=INFO source=ggml.go:136 msg="" architecture=qwen35 file_type=Q4_K_M name="" description="" num_tensors=1307 num_key_values=53 load_backend: loaded CPU backend from /usr/lib/ollama/libggml-cpu-icelake.so ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no ggml_cuda_init: found 1 CUDA devices: Device 0: NVIDIA RTX A6000, compute capability 8.6, VMM: yes, ID: GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 load_backend: loaded CUDA backend from /usr/lib/ollama/cuda_v12/libggml-cuda.so time=2026-03-11T03:02:20.819Z level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.AVX512=1 CPU.0.AVX512_VBMI=1 CPU.0.AVX512_VNNI=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,520,600,610,700,750,800,860,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(gcc) time=2026-03-11T03:02:23.093Z level=INFO source=runner.go:1302 msg=load request="{Operation:fit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:262144 KvCacheType: NumThreads:24 GPULayers:52[ID:GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 Layers:52(12..63)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}" time=2026-03-11T03:02:25.152Z level=INFO source=runner.go:1302 msg=load request="{Operation:fit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:262144 KvCacheType: NumThreads:24 GPULayers:50[ID:GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 Layers:50(14..63)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}" time=2026-03-11T03:02:27.147Z level=INFO source=runner.go:1302 msg=load request="{Operation:alloc LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:262144 KvCacheType: NumThreads:24 GPULayers:50[ID:GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 Layers:50(14..63)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}" ggml_backend_cuda_buffer_type_alloc_buffer: allocating 4365.34 MiB on device 0: cudaMalloc failed: out of memory ggml_gallocr_reserve_n_impl: failed to allocate CUDA0 buffer of size 4577394688 time=2026-03-11T03:02:31.141Z level=INFO source=server.go:879 msg="model layout did not fit, applying backoff" backoff=0.10 time=2026-03-11T03:02:31.142Z level=INFO source=runner.go:1302 msg=load request="{Operation:alloc LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:262144 KvCacheType: NumThreads:24 GPULayers:44[ID:GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 Layers:44(20..63)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}" time=2026-03-11T03:02:36.861Z level=INFO source=runner.go:1302 msg=load request="{Operation:commit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:262144 KvCacheType: NumThreads:24 GPULayers:44[ID:GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 Layers:44(20..63)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}" time=2026-03-11T03:02:36.861Z level=INFO source=ggml.go:482 msg="offloading 44 repeating layers to GPU" time=2026-03-11T03:02:36.861Z level=INFO source=ggml.go:486 msg="offloading output layer to CPU" time=2026-03-11T03:02:36.861Z level=INFO source=ggml.go:494 msg="offloaded 44/65 layers to GPU" time=2026-03-11T03:02:36.861Z level=INFO source=device.go:240 msg="model weights" device=CUDA0 size="9.2 GiB" time=2026-03-11T03:02:36.861Z level=INFO source=device.go:245 msg="model weights" device=CPU size="7.0 GiB" time=2026-03-11T03:02:36.861Z level=INFO source=device.go:251 msg="kv cache" device=CUDA0 size="13.5 GiB" time=2026-03-11T03:02:36.861Z level=INFO source=device.go:256 msg="kv cache" device=CPU size="6.1 GiB" time=2026-03-11T03:02:36.861Z level=INFO source=device.go:262 msg="compute graph" device=CUDA0 size="4.6 GiB" time=2026-03-11T03:02:36.861Z level=INFO source=device.go:267 msg="compute graph" device=CPU size="952.3 MiB" time=2026-03-11T03:02:36.861Z level=INFO source=device.go:272 msg="total memory" size="41.4 GiB" time=2026-03-11T03:02:36.861Z level=INFO source=sched.go:565 msg="loaded runners" count=1 time=2026-03-11T03:02:36.861Z level=INFO source=server.go:1350 msg="waiting for llama runner to start responding" time=2026-03-11T03:02:36.862Z level=INFO source=server.go:1384 msg="waiting for server to become available" status="llm server loading model" time=2026-03-11T03:02:44.635Z level=INFO source=server.go:1388 msg="llama runner started in 23.94 seconds" [GIN] 2026/03/11 - 03:02:44 | 200 | 24.409871186s | 127.0.0.1 | POST "/api/generate" [GIN] 2026/03/11 - 03:03:37 | 200 | 52.64µs | 127.0.0.1 | HEAD "/" [GIN] 2026/03/11 - 03:03:37 | 200 | 215.183312ms | 127.0.0.1 | POST "/api/show" [GIN] 2026/03/11 - 03:03:37 | 200 | 693.633µs | 127.0.0.1 | POST "/api/generate" ggml_backend_cuda_device_get_memory device GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 utilizing NVML memory reporting free: 3141533696 total: 51527024640 time=2026-03-11T03:03:37.730Z level=ERROR source=server.go:303 msg="llama runner terminated" error="signal: killed" time=2026-03-11T03:03:40.598Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 38139" time=2026-03-11T03:10:10.432Z level=INFO source=routes.go:1658 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:0 OLLAMA_DEBUG:INFO OLLAMA_EDITOR: OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434/ OLLAMA_KEEP_ALIVE:2562047h47m16.854775807s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:1 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/root/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NO_CLOUD:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost/ https://localhost/ http://localhost/:* https://localhost/:* http://127.0.0.1/ https://127.0.0.1/ http://127.0.0.1/:* https://127.0.0.1/:* http://0.0.0.0/ https://0.0.0.0/ http://0.0.0.0/:* https://0.0.0.0/:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[[ollama.com](https://ollama.com/)] OLLAMA_SCHED_SPREAD:false OLLAMA_VULKAN:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]" time=2026-03-11T03:10:10.432Z level=INFO source=routes.go:1660 msg="Ollama cloud disabled: false" time=2026-03-11T03:10:10.433Z level=INFO source=images.go:477 msg="total blobs: 24" time=2026-03-11T03:10:10.433Z level=INFO source=images.go:484 msg="total unused blobs removed: 0" time=2026-03-11T03:10:10.433Z level=INFO source=routes.go:1713 msg="Listening on [::]:11434 (version 0.17.7)" time=2026-03-11T03:10:10.434Z level=INFO source=runner.go:67 msg="discovering available GPUs..." time=2026-03-11T03:10:10.435Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 32817" time=2026-03-11T03:10:10.514Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 41789" time=2026-03-11T03:10:10.578Z level=INFO source=runner.go:106 msg="experimental Vulkan support disabled. To enable, set OLLAMA_VULKAN=1" time=2026-03-11T03:10:10.578Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 44083" time=2026-03-11T03:10:10.655Z level=INFO source=types.go:42 msg="inference compute" id=GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 filter_id="" library=CUDA compute=8.6 name=CUDA0 description="NVIDIA RTX A6000" libdirs=ollama,cuda_v12 driver=12.4 pci_id=0000:17:00.0 type=discrete total="48.0 GiB" available="31.4 GiB" time=2026-03-11T03:10:10.655Z level=INFO source=routes.go:1763 msg="vram-based default context" total_vram="48.0 GiB" default_num_ctx=262144 [GIN] 2026/03/11 - 03:10:18 | 200 | 51.236µs | 127.0.0.1 | HEAD "/" [GIN] 2026/03/11 - 03:10:18 | 200 | 110.201924ms | 127.0.0.1 | POST "/api/show" [GIN] 2026/03/11 - 03:10:18 | 200 | 110.707116ms | 127.0.0.1 | POST "/api/show" time=2026-03-11T03:10:19.006Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 43919" time=2026-03-11T03:10:19.085Z level=WARN source=cpu_linux.go:130 msg="failed to parse CPU allowed micro secs" error="strconv.ParseInt: parsing \"max\": invalid syntax" time=2026-03-11T03:10:19.160Z level=WARN source=server.go:168 msg="requested context size too large for model" num_ctx=262144 n_ctx_train=40960 time=2026-03-11T03:10:19.160Z level=INFO source=server.go:246 msg="enabling flash attention" time=2026-03-11T03:10:19.160Z level=INFO source=server.go:430 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --model /root/.ollama/models/blobs/sha256-3291abe70f16ee9682de7bfae08db5373ea9d6497e614aaad63340ad421d6312 --port 45799" time=2026-03-11T03:10:19.160Z level=INFO source=sched.go:489 msg="system memory" total="124.8 GiB" free="124.7 GiB" free_swap="127.5 GiB" time=2026-03-11T03:10:19.160Z level=INFO source=sched.go:496 msg="gpu memory" id=GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 library=CUDA available="31.0 GiB" free="31.4 GiB" minimum="457.0 MiB" overhead="0 B" time=2026-03-11T03:10:19.160Z level=INFO source=server.go:757 msg="loading model" "model layers"=65 requested=-1 time=2026-03-11T03:10:19.173Z level=INFO source=runner.go:1429 msg="starting ollama engine" time=2026-03-11T03:10:19.173Z level=INFO source=runner.go:1464 msg="Server listening on 127.0.0.1:45799" time=2026-03-11T03:10:19.182Z level=INFO source=runner.go:1302 msg=load request="{Operation:fit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:40960 KvCacheType: NumThreads:24 GPULayers:65[ID:GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 Layers:65(0..64)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}" time=2026-03-11T03:10:19.211Z level=INFO source=ggml.go:136 msg="" architecture=qwen3 file_type=Q4_K_M name="Qwen3 32B" description="" num_tensors=707 num_key_values=28 load_backend: loaded CPU backend from /usr/lib/ollama/libggml-cpu-icelake.so ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no ggml_cuda_init: found 1 CUDA devices: Device 0: NVIDIA RTX A6000, compute capability 8.6, VMM: yes, ID: GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 load_backend: loaded CUDA backend from /usr/lib/ollama/cuda_v12/libggml-cuda.so time=2026-03-11T03:10:19.255Z level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.AVX512=1 CPU.0.AVX512_VBMI=1 CPU.0.AVX512_VNNI=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,520,600,610,700,750,800,860,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(gcc) time=2026-03-11T03:10:19.476Z level=INFO source=runner.go:1302 msg=load request="{Operation:alloc LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:40960 KvCacheType: NumThreads:24 GPULayers:65[ID:GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 Layers:65(0..64)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}" time=2026-03-11T03:10:19.640Z level=INFO source=runner.go:1302 msg=load request="{Operation:commit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:40960 KvCacheType: NumThreads:24 GPULayers:65[ID:GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 Layers:65(0..64)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}" time=2026-03-11T03:10:19.640Z level=INFO source=device.go:240 msg="model weights" device=CUDA0 size="18.4 GiB" time=2026-03-11T03:10:19.640Z level=INFO source=device.go:245 msg="model weights" device=CPU size="417.3 MiB" time=2026-03-11T03:10:19.640Z level=INFO source=device.go:251 msg="kv cache" device=CUDA0 size="10.0 GiB" time=2026-03-11T03:10:19.640Z level=INFO source=device.go:262 msg="compute graph" device=CUDA0 size="404.0 MiB" time=2026-03-11T03:10:19.640Z level=INFO source=device.go:267 msg="compute graph" device=CPU size="10.0 MiB" time=2026-03-11T03:10:19.640Z level=INFO source=device.go:272 msg="total memory" size="29.2 GiB" time=2026-03-11T03:10:19.640Z level=INFO source=sched.go:565 msg="loaded runners" count=1 time=2026-03-11T03:10:19.640Z level=INFO source=server.go:1350 msg="waiting for llama runner to start responding" time=2026-03-11T03:10:19.640Z level=INFO source=ggml.go:482 msg="offloading 64 repeating layers to GPU" time=2026-03-11T03:10:19.640Z level=INFO source=ggml.go:489 msg="offloading output layer to GPU" time=2026-03-11T03:10:19.640Z level=INFO source=ggml.go:494 msg="offloaded 65/65 layers to GPU" time=2026-03-11T03:10:19.641Z level=INFO source=server.go:1384 msg="waiting for server to become available" status="llm server loading model" time=2026-03-11T03:10:22.148Z level=INFO source=server.go:1388 msg="llama runner started in 2.99 seconds" ```
GiteaMirror commented 2026-04-29 10:13:10 -05:00
Author
Owner
Copy Link

@sy960923 commented on GitHub (Mar 11, 2026):

0.9.5 qwen3:32b 启动日志

time=2026-03-11T03:07:53.764Z level=INFO source=routes.go:1235 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:4096 OLLAMA_DEBUG:INFO OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434/ OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:2562047h47m16.854775807s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:1 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/root/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost/ https://localhost/ http://localhost/:* https://localhost/:* http://127.0.0.1/ https://127.0.0.1/ http://127.0.0.1/:* https://127.0.0.1/:* http://0.0.0.0/ https://0.0.0.0/ http://0.0.0.0/:* https://0.0.0.0/:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_SCHED_SPREAD:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"
time=2026-03-11T03:07:53.766Z level=INFO source=images.go:476 msg="total blobs: 24"
time=2026-03-11T03:07:53.766Z level=INFO source=images.go:483 msg="total unused blobs removed: 0"
time=2026-03-11T03:07:53.767Z level=INFO source=routes.go:1288 msg="Listening on [::]:11434 (version 0.9.5)"
time=2026-03-11T03:07:53.767Z level=INFO source=gpu.go:217 msg="looking for compatible GPUs"
time=2026-03-11T03:07:53.856Z level=INFO source=types.go:130 msg="inference compute" id=GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 library=cuda variant=v12 compute=8.6 driver=12.4 name="NVIDIA RTX A6000" total="47.5 GiB" available="31.2 GiB"
[GIN] 2026/03/11 - 03:08:05 | 200 |      87.652µs |       127.0.0.1 | GET      "/api/version"
[GIN] 2026/03/11 - 03:08:19 | 200 |      36.786µs |       127.0.0.1 | HEAD     "/"
[GIN] 2026/03/11 - 03:08:19 | 200 |   47.596648ms |       127.0.0.1 | POST     "/api/show"
time=2026-03-11T03:08:20.001Z level=INFO source=sched.go:788 msg="new model will fit in available VRAM in single GPU, loading" model=/root/.ollama/models/blobs/sha256-3291abe70f16ee9682de7bfae08db5373ea9d6497e614aaad63340ad421d6312 gpu=GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 parallel=1 available=33451671552 required="21.5 GiB"
time=2026-03-11T03:08:20.074Z level=INFO source=server.go:135 msg="system memory" total="124.8 GiB" free="109.6 GiB" free_swap="127.5 GiB"
time=2026-03-11T03:08:20.075Z level=INFO source=server.go:175 msg=offload library=cuda layers.requested=-1 layers.model=65 layers.offload=65 layers.split="" memory.available="[31.2 GiB]" memory.gpu_overhead="0 B" memory.required.full="21.5 GiB" memory.required.partial="21.5 GiB" memory.required.kv="1.0 GiB" memory.required.allocations="[21.5 GiB]" memory.weights.total="18.4 GiB" memory.weights.repeating="17.8 GiB" memory.weights.nonrepeating="608.6 MiB" memory.graph.full="1.3 GiB" memory.graph.partial="1.3 GiB"
llama_model_loader: loaded meta data with 27 key-value pairs and 707 tensors from /root/.ollama/models/blobs/sha256-3291abe70f16ee9682de7bfae08db5373ea9d6497e614aaad63340ad421d6312 (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = qwen3
llama_model_loader: - kv   1:                               general.type str              = model
llama_model_loader: - kv   2:                               general.name str              = Qwen3 32B
llama_model_loader: - kv   3:                           general.basename str              = Qwen3
llama_model_loader: - kv   4:                         general.size_label str              = 32B
llama_model_loader: - kv   5:                          qwen3.block_count u32              = 64
llama_model_loader: - kv   6:                       qwen3.context_length u32              = 40960
llama_model_loader: - kv   7:                     qwen3.embedding_length u32              = 5120
llama_model_loader: - kv   8:                  qwen3.feed_forward_length u32              = 25600
llama_model_loader: - kv   9:                 qwen3.attention.head_count u32              = 64
llama_model_loader: - kv  10:              qwen3.attention.head_count_kv u32              = 8
llama_model_loader: - kv  11:                       qwen3.rope.freq_base f32              = 1000000.000000
llama_model_loader: - kv  12:     qwen3.attention.layer_norm_rms_epsilon f32              = 0.000001
llama_model_loader: - kv  13:                 qwen3.attention.key_length u32              = 128
llama_model_loader: - kv  14:               qwen3.attention.value_length u32              = 128
llama_model_loader: - kv  15:                       tokenizer.ggml.model str              = gpt2
llama_model_loader: - kv  16:                         tokenizer.ggml.pre str              = qwen2
llama_model_loader: - kv  17:                      tokenizer.ggml.tokens arr[str,151936]  = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv  18:                  tokenizer.ggml.token_type arr[i32,151936]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv  19:                      tokenizer.ggml.merges arr[str,151387]  = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
llama_model_loader: - kv  20:                tokenizer.ggml.eos_token_id u32              = 151645
llama_model_loader: - kv  21:            tokenizer.ggml.padding_token_id u32              = 151643
llama_model_loader: - kv  22:                tokenizer.ggml.bos_token_id u32              = 151643
llama_model_loader: - kv  23:               tokenizer.ggml.add_bos_token bool             = false
llama_model_loader: - kv  24:                    tokenizer.chat_template str              = {%- if tools %}\n    {{- '<|im_start|>...
llama_model_loader: - kv  25:               general.quantization_version u32              = 2
llama_model_loader: - kv  26:                          general.file_type u32              = 15
llama_model_loader: - type  f32:  257 tensors
llama_model_loader: - type  f16:   64 tensors
llama_model_loader: - type q4_K:  353 tensors
llama_model_loader: - type q6_K:   33 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type   = Q4_K - Medium
print_info: file size   = 18.81 GiB (4.93 BPW) 
load: special tokens cache size = 26
load: token to piece cache size = 0.9311 MB
print_info: arch             = qwen3
print_info: vocab_only       = 1
print_info: model type       = ?B
print_info: model params     = 32.76 B
print_info: general.name     = Qwen3 32B
print_info: vocab type       = BPE
print_info: n_vocab          = 151936
print_info: n_merges         = 151387
print_info: BOS token        = 151643 '<|endoftext|>'
print_info: EOS token        = 151645 '<|im_end|>'
print_info: EOT token        = 151645 '<|im_end|>'
print_info: PAD token        = 151643 '<|endoftext|>'
print_info: LF token         = 198 'Ċ'
print_info: FIM PRE token    = 151659 '<|fim_prefix|>'
print_info: FIM SUF token    = 151661 '<|fim_suffix|>'
print_info: FIM MID token    = 151660 '<|fim_middle|>'
print_info: FIM PAD token    = 151662 '<|fim_pad|>'
print_info: FIM REP token    = 151663 '<|repo_name|>'
print_info: FIM SEP token    = 151664 '<|file_sep|>'
print_info: EOG token        = 151643 '<|endoftext|>'
print_info: EOG token        = 151645 '<|im_end|>'
print_info: EOG token        = 151662 '<|fim_pad|>'
print_info: EOG token        = 151663 '<|repo_name|>'
print_info: EOG token        = 151664 '<|file_sep|>'
print_info: max token length = 256
llama_model_load: vocab only - skipping tensors
time=2026-03-11T03:08:20.282Z level=INFO source=server.go:438 msg="starting llama server" cmd="/usr/bin/ollama runner --model /root/.ollama/models/blobs/sha256-3291abe70f16ee9682de7bfae08db5373ea9d6497e614aaad63340ad421d6312 --ctx-size 4096 --batch-size 512 --n-gpu-layers 65 --threads 24 --parallel 1 --port 34965"
time=2026-03-11T03:08:20.283Z level=INFO source=sched.go:483 msg="loaded runners" count=1
time=2026-03-11T03:08:20.283Z level=INFO source=server.go:598 msg="waiting for llama runner to start responding"
time=2026-03-11T03:08:20.283Z level=INFO source=server.go:632 msg="waiting for server to become available" status="llm server not responding"
time=2026-03-11T03:08:20.294Z level=INFO source=runner.go:815 msg="starting go runner"
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA RTX A6000, compute capability 8.6, VMM: yes
load_backend: loaded CUDA backend from /usr/lib/ollama/libggml-cuda.so
load_backend: loaded CPU backend from /usr/lib/ollama/libggml-cpu-icelake.so
time=2026-03-11T03:08:20.337Z level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.AVX512=1 CPU.0.AVX512_VBMI=1 CPU.0.AVX512_VNNI=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(gcc)
time=2026-03-11T03:08:20.337Z level=INFO source=runner.go:874 msg="Server listening on 127.0.0.1:34965"
llama_model_load_from_file_impl: using device CUDA0 (NVIDIA RTX A6000) - 31902 MiB free
llama_model_loader: loaded meta data with 27 key-value pairs and 707 tensors from /root/.ollama/models/blobs/sha256-3291abe70f16ee9682de7bfae08db5373ea9d6497e614aaad63340ad421d6312 (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = qwen3
llama_model_loader: - kv   1:                               general.type str              = model
llama_model_loader: - kv   2:                               general.name str              = Qwen3 32B
llama_model_loader: - kv   3:                           general.basename str              = Qwen3
llama_model_loader: - kv   4:                         general.size_label str              = 32B
llama_model_loader: - kv   5:                          qwen3.block_count u32              = 64
llama_model_loader: - kv   6:                       qwen3.context_length u32              = 40960
llama_model_loader: - kv   7:                     qwen3.embedding_length u32              = 5120
llama_model_loader: - kv   8:                  qwen3.feed_forward_length u32              = 25600
llama_model_loader: - kv   9:                 qwen3.attention.head_count u32              = 64
llama_model_loader: - kv  10:              qwen3.attention.head_count_kv u32              = 8
llama_model_loader: - kv  11:                       qwen3.rope.freq_base f32              = 1000000.000000
llama_model_loader: - kv  12:     qwen3.attention.layer_norm_rms_epsilon f32              = 0.000001
llama_model_loader: - kv  13:                 qwen3.attention.key_length u32              = 128
llama_model_loader: - kv  14:               qwen3.attention.value_length u32              = 128
llama_model_loader: - kv  15:                       tokenizer.ggml.model str              = gpt2
llama_model_loader: - kv  16:                         tokenizer.ggml.pre str              = qwen2
llama_model_loader: - kv  17:                      tokenizer.ggml.tokens arr[str,151936]  = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv  18:                  tokenizer.ggml.token_type arr[i32,151936]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv  19:                      tokenizer.ggml.merges arr[str,151387]  = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
llama_model_loader: - kv  20:                tokenizer.ggml.eos_token_id u32              = 151645
llama_model_loader: - kv  21:            tokenizer.ggml.padding_token_id u32              = 151643
llama_model_loader: - kv  22:                tokenizer.ggml.bos_token_id u32              = 151643
llama_model_loader: - kv  23:               tokenizer.ggml.add_bos_token bool             = false
llama_model_loader: - kv  24:                    tokenizer.chat_template str              = {%- if tools %}\n    {{- '<|im_start|>...
llama_model_loader: - kv  25:               general.quantization_version u32              = 2
llama_model_loader: - kv  26:                          general.file_type u32              = 15
llama_model_loader: - type  f32:  257 tensors
llama_model_loader: - type  f16:   64 tensors
llama_model_loader: - type q4_K:  353 tensors
llama_model_loader: - type q6_K:   33 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type   = Q4_K - Medium
print_info: file size   = 18.81 GiB (4.93 BPW) 
time=2026-03-11T03:08:20.534Z level=INFO source=server.go:632 msg="waiting for server to become available" status="llm server loading model"
load: special tokens cache size = 26
load: token to piece cache size = 0.9311 MB
print_info: arch             = qwen3
print_info: vocab_only       = 0
print_info: n_ctx_train      = 40960
print_info: n_embd           = 5120
print_info: n_layer          = 64
print_info: n_head           = 64
print_info: n_head_kv        = 8
print_info: n_rot            = 128
print_info: n_swa            = 0
print_info: n_swa_pattern    = 1
print_info: n_embd_head_k    = 128
print_info: n_embd_head_v    = 128
print_info: n_gqa            = 8
print_info: n_embd_k_gqa     = 1024
print_info: n_embd_v_gqa     = 1024
print_info: f_norm_eps       = 0.0e+00
print_info: f_norm_rms_eps   = 1.0e-06
print_info: f_clamp_kqv      = 0.0e+00
print_info: f_max_alibi_bias = 0.0e+00
print_info: f_logit_scale    = 0.0e+00
print_info: f_attn_scale     = 0.0e+00
print_info: n_ff             = 25600
print_info: n_expert         = 0
print_info: n_expert_used    = 0
print_info: causal attn      = 1
print_info: pooling type     = 0
print_info: rope type        = 2
print_info: rope scaling     = linear
print_info: freq_base_train  = 1000000.0
print_info: freq_scale_train = 1
print_info: n_ctx_orig_yarn  = 40960
print_info: rope_finetuned   = unknown
print_info: ssm_d_conv       = 0
print_info: ssm_d_inner      = 0
print_info: ssm_d_state      = 0
print_info: ssm_dt_rank      = 0
print_info: ssm_dt_b_c_rms   = 0
print_info: model type       = 32B
print_info: model params     = 32.76 B
print_info: general.name     = Qwen3 32B
print_info: vocab type       = BPE
print_info: n_vocab          = 151936
print_info: n_merges         = 151387
print_info: BOS token        = 151643 '<|endoftext|>'
print_info: EOS token        = 151645 '<|im_end|>'
print_info: EOT token        = 151645 '<|im_end|>'
print_info: PAD token        = 151643 '<|endoftext|>'
print_info: LF token         = 198 'Ċ'
print_info: FIM PRE token    = 151659 '<|fim_prefix|>'
print_info: FIM SUF token    = 151661 '<|fim_suffix|>'
print_info: FIM MID token    = 151660 '<|fim_middle|>'
print_info: FIM PAD token    = 151662 '<|fim_pad|>'
print_info: FIM REP token    = 151663 '<|repo_name|>'
print_info: FIM SEP token    = 151664 '<|file_sep|>'
print_info: EOG token        = 151643 '<|endoftext|>'
print_info: EOG token        = 151645 '<|im_end|>'
print_info: EOG token        = 151662 '<|fim_pad|>'
print_info: EOG token        = 151663 '<|repo_name|>'
print_info: EOG token        = 151664 '<|file_sep|>'
print_info: max token length = 256
load_tensors: loading model tensors, this can take a while... (mmap = true)
load_tensors: offloading 64 repeating layers to GPU
load_tensors: offloading output layer to GPU
load_tensors: offloaded 65/65 layers to GPU
load_tensors:        CUDA0 model buffer size = 18842.40 MiB
load_tensors:   CPU_Mapped model buffer size =   417.30 MiB
llama_context: constructing llama_context
llama_context: n_seq_max     = 1
llama_context: n_ctx         = 4096
llama_context: n_ctx_per_seq = 4096
llama_context: n_batch       = 512
llama_context: n_ubatch      = 512
llama_context: causal_attn   = 1
llama_context: flash_attn    = 0
llama_context: freq_base     = 1000000.0
llama_context: freq_scale    = 1
llama_context: n_ctx_per_seq (4096) < n_ctx_train (40960) -- the full capacity of the model will not be utilized
llama_context:  CUDA_Host  output buffer size =     0.60 MiB
llama_kv_cache_unified: kv_size = 4096, type_k = 'f16', type_v = 'f16', n_layer = 64, can_shift = 1, padding = 32
llama_kv_cache_unified:      CUDA0 KV buffer size =  1024.00 MiB
llama_kv_cache_unified: KV self size  = 1024.00 MiB, K (f16):  512.00 MiB, V (f16):  512.00 MiB
llama_context:      CUDA0 compute buffer size =   572.00 MiB
llama_context:  CUDA_Host compute buffer size =    18.01 MiB
llama_context: graph nodes  = 2438
llama_context: graph splits = 2
time=2026-03-11T03:08:24.046Z level=INFO source=server.go:637 msg="llama runner started in 3.76 seconds"
<!-- gh-comment-id:4035915453 --> @sy960923 commented on GitHub (Mar 11, 2026): 0.9.5 qwen3:32b 启动日志 ``` time=2026-03-11T03:07:53.764Z level=INFO source=routes.go:1235 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:4096 OLLAMA_DEBUG:INFO OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434/ OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:2562047h47m16.854775807s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:1 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/root/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost/ https://localhost/ http://localhost/:* https://localhost/:* http://127.0.0.1/ https://127.0.0.1/ http://127.0.0.1/:* https://127.0.0.1/:* http://0.0.0.0/ https://0.0.0.0/ http://0.0.0.0/:* https://0.0.0.0/:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_SCHED_SPREAD:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]" time=2026-03-11T03:07:53.766Z level=INFO source=images.go:476 msg="total blobs: 24" time=2026-03-11T03:07:53.766Z level=INFO source=images.go:483 msg="total unused blobs removed: 0" time=2026-03-11T03:07:53.767Z level=INFO source=routes.go:1288 msg="Listening on [::]:11434 (version 0.9.5)" time=2026-03-11T03:07:53.767Z level=INFO source=gpu.go:217 msg="looking for compatible GPUs" time=2026-03-11T03:07:53.856Z level=INFO source=types.go:130 msg="inference compute" id=GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 library=cuda variant=v12 compute=8.6 driver=12.4 name="NVIDIA RTX A6000" total="47.5 GiB" available="31.2 GiB" [GIN] 2026/03/11 - 03:08:05 | 200 | 87.652µs | 127.0.0.1 | GET "/api/version" [GIN] 2026/03/11 - 03:08:19 | 200 | 36.786µs | 127.0.0.1 | HEAD "/" [GIN] 2026/03/11 - 03:08:19 | 200 | 47.596648ms | 127.0.0.1 | POST "/api/show" time=2026-03-11T03:08:20.001Z level=INFO source=sched.go:788 msg="new model will fit in available VRAM in single GPU, loading" model=/root/.ollama/models/blobs/sha256-3291abe70f16ee9682de7bfae08db5373ea9d6497e614aaad63340ad421d6312 gpu=GPU-743cd96c-f59a-1387-ad11-bb69cfd399c1 parallel=1 available=33451671552 required="21.5 GiB" time=2026-03-11T03:08:20.074Z level=INFO source=server.go:135 msg="system memory" total="124.8 GiB" free="109.6 GiB" free_swap="127.5 GiB" time=2026-03-11T03:08:20.075Z level=INFO source=server.go:175 msg=offload library=cuda layers.requested=-1 layers.model=65 layers.offload=65 layers.split="" memory.available="[31.2 GiB]" memory.gpu_overhead="0 B" memory.required.full="21.5 GiB" memory.required.partial="21.5 GiB" memory.required.kv="1.0 GiB" memory.required.allocations="[21.5 GiB]" memory.weights.total="18.4 GiB" memory.weights.repeating="17.8 GiB" memory.weights.nonrepeating="608.6 MiB" memory.graph.full="1.3 GiB" memory.graph.partial="1.3 GiB" llama_model_loader: loaded meta data with 27 key-value pairs and 707 tensors from /root/.ollama/models/blobs/sha256-3291abe70f16ee9682de7bfae08db5373ea9d6497e614aaad63340ad421d6312 (version GGUF V3 (latest)) llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. llama_model_loader: - kv 0: general.architecture str = qwen3 llama_model_loader: - kv 1: general.type str = model llama_model_loader: - kv 2: general.name str = Qwen3 32B llama_model_loader: - kv 3: general.basename str = Qwen3 llama_model_loader: - kv 4: general.size_label str = 32B llama_model_loader: - kv 5: qwen3.block_count u32 = 64 llama_model_loader: - kv 6: qwen3.context_length u32 = 40960 llama_model_loader: - kv 7: qwen3.embedding_length u32 = 5120 llama_model_loader: - kv 8: qwen3.feed_forward_length u32 = 25600 llama_model_loader: - kv 9: qwen3.attention.head_count u32 = 64 llama_model_loader: - kv 10: qwen3.attention.head_count_kv u32 = 8 llama_model_loader: - kv 11: qwen3.rope.freq_base f32 = 1000000.000000 llama_model_loader: - kv 12: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001 llama_model_loader: - kv 13: qwen3.attention.key_length u32 = 128 llama_model_loader: - kv 14: qwen3.attention.value_length u32 = 128 llama_model_loader: - kv 15: tokenizer.ggml.model str = gpt2 llama_model_loader: - kv 16: tokenizer.ggml.pre str = qwen2 llama_model_loader: - kv 17: tokenizer.ggml.tokens arr[str,151936] = ["!", "\"", "#", "$", "%", "&", "'", ... llama_model_loader: - kv 18: tokenizer.ggml.token_type arr[i32,151936] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... llama_model_loader: - kv 19: tokenizer.ggml.merges arr[str,151387] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",... llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151645 llama_model_loader: - kv 21: tokenizer.ggml.padding_token_id u32 = 151643 llama_model_loader: - kv 22: tokenizer.ggml.bos_token_id u32 = 151643 llama_model_loader: - kv 23: tokenizer.ggml.add_bos_token bool = false llama_model_loader: - kv 24: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>... llama_model_loader: - kv 25: general.quantization_version u32 = 2 llama_model_loader: - kv 26: general.file_type u32 = 15 llama_model_loader: - type f32: 257 tensors llama_model_loader: - type f16: 64 tensors llama_model_loader: - type q4_K: 353 tensors llama_model_loader: - type q6_K: 33 tensors print_info: file format = GGUF V3 (latest) print_info: file type = Q4_K - Medium print_info: file size = 18.81 GiB (4.93 BPW) load: special tokens cache size = 26 load: token to piece cache size = 0.9311 MB print_info: arch = qwen3 print_info: vocab_only = 1 print_info: model type = ?B print_info: model params = 32.76 B print_info: general.name = Qwen3 32B print_info: vocab type = BPE print_info: n_vocab = 151936 print_info: n_merges = 151387 print_info: BOS token = 151643 '<|endoftext|>' print_info: EOS token = 151645 '<|im_end|>' print_info: EOT token = 151645 '<|im_end|>' print_info: PAD token = 151643 '<|endoftext|>' print_info: LF token = 198 'Ċ' print_info: FIM PRE token = 151659 '<|fim_prefix|>' print_info: FIM SUF token = 151661 '<|fim_suffix|>' print_info: FIM MID token = 151660 '<|fim_middle|>' print_info: FIM PAD token = 151662 '<|fim_pad|>' print_info: FIM REP token = 151663 '<|repo_name|>' print_info: FIM SEP token = 151664 '<|file_sep|>' print_info: EOG token = 151643 '<|endoftext|>' print_info: EOG token = 151645 '<|im_end|>' print_info: EOG token = 151662 '<|fim_pad|>' print_info: EOG token = 151663 '<|repo_name|>' print_info: EOG token = 151664 '<|file_sep|>' print_info: max token length = 256 llama_model_load: vocab only - skipping tensors time=2026-03-11T03:08:20.282Z level=INFO source=server.go:438 msg="starting llama server" cmd="/usr/bin/ollama runner --model /root/.ollama/models/blobs/sha256-3291abe70f16ee9682de7bfae08db5373ea9d6497e614aaad63340ad421d6312 --ctx-size 4096 --batch-size 512 --n-gpu-layers 65 --threads 24 --parallel 1 --port 34965" time=2026-03-11T03:08:20.283Z level=INFO source=sched.go:483 msg="loaded runners" count=1 time=2026-03-11T03:08:20.283Z level=INFO source=server.go:598 msg="waiting for llama runner to start responding" time=2026-03-11T03:08:20.283Z level=INFO source=server.go:632 msg="waiting for server to become available" status="llm server not responding" time=2026-03-11T03:08:20.294Z level=INFO source=runner.go:815 msg="starting go runner" ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no ggml_cuda_init: found 1 CUDA devices: Device 0: NVIDIA RTX A6000, compute capability 8.6, VMM: yes load_backend: loaded CUDA backend from /usr/lib/ollama/libggml-cuda.so load_backend: loaded CPU backend from /usr/lib/ollama/libggml-cpu-icelake.so time=2026-03-11T03:08:20.337Z level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.AVX512=1 CPU.0.AVX512_VBMI=1 CPU.0.AVX512_VNNI=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(gcc) time=2026-03-11T03:08:20.337Z level=INFO source=runner.go:874 msg="Server listening on 127.0.0.1:34965" llama_model_load_from_file_impl: using device CUDA0 (NVIDIA RTX A6000) - 31902 MiB free llama_model_loader: loaded meta data with 27 key-value pairs and 707 tensors from /root/.ollama/models/blobs/sha256-3291abe70f16ee9682de7bfae08db5373ea9d6497e614aaad63340ad421d6312 (version GGUF V3 (latest)) llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. llama_model_loader: - kv 0: general.architecture str = qwen3 llama_model_loader: - kv 1: general.type str = model llama_model_loader: - kv 2: general.name str = Qwen3 32B llama_model_loader: - kv 3: general.basename str = Qwen3 llama_model_loader: - kv 4: general.size_label str = 32B llama_model_loader: - kv 5: qwen3.block_count u32 = 64 llama_model_loader: - kv 6: qwen3.context_length u32 = 40960 llama_model_loader: - kv 7: qwen3.embedding_length u32 = 5120 llama_model_loader: - kv 8: qwen3.feed_forward_length u32 = 25600 llama_model_loader: - kv 9: qwen3.attention.head_count u32 = 64 llama_model_loader: - kv 10: qwen3.attention.head_count_kv u32 = 8 llama_model_loader: - kv 11: qwen3.rope.freq_base f32 = 1000000.000000 llama_model_loader: - kv 12: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001 llama_model_loader: - kv 13: qwen3.attention.key_length u32 = 128 llama_model_loader: - kv 14: qwen3.attention.value_length u32 = 128 llama_model_loader: - kv 15: tokenizer.ggml.model str = gpt2 llama_model_loader: - kv 16: tokenizer.ggml.pre str = qwen2 llama_model_loader: - kv 17: tokenizer.ggml.tokens arr[str,151936] = ["!", "\"", "#", "$", "%", "&", "'", ... llama_model_loader: - kv 18: tokenizer.ggml.token_type arr[i32,151936] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... llama_model_loader: - kv 19: tokenizer.ggml.merges arr[str,151387] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",... llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151645 llama_model_loader: - kv 21: tokenizer.ggml.padding_token_id u32 = 151643 llama_model_loader: - kv 22: tokenizer.ggml.bos_token_id u32 = 151643 llama_model_loader: - kv 23: tokenizer.ggml.add_bos_token bool = false llama_model_loader: - kv 24: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>... llama_model_loader: - kv 25: general.quantization_version u32 = 2 llama_model_loader: - kv 26: general.file_type u32 = 15 llama_model_loader: - type f32: 257 tensors llama_model_loader: - type f16: 64 tensors llama_model_loader: - type q4_K: 353 tensors llama_model_loader: - type q6_K: 33 tensors print_info: file format = GGUF V3 (latest) print_info: file type = Q4_K - Medium print_info: file size = 18.81 GiB (4.93 BPW) time=2026-03-11T03:08:20.534Z level=INFO source=server.go:632 msg="waiting for server to become available" status="llm server loading model" load: special tokens cache size = 26 load: token to piece cache size = 0.9311 MB print_info: arch = qwen3 print_info: vocab_only = 0 print_info: n_ctx_train = 40960 print_info: n_embd = 5120 print_info: n_layer = 64 print_info: n_head = 64 print_info: n_head_kv = 8 print_info: n_rot = 128 print_info: n_swa = 0 print_info: n_swa_pattern = 1 print_info: n_embd_head_k = 128 print_info: n_embd_head_v = 128 print_info: n_gqa = 8 print_info: n_embd_k_gqa = 1024 print_info: n_embd_v_gqa = 1024 print_info: f_norm_eps = 0.0e+00 print_info: f_norm_rms_eps = 1.0e-06 print_info: f_clamp_kqv = 0.0e+00 print_info: f_max_alibi_bias = 0.0e+00 print_info: f_logit_scale = 0.0e+00 print_info: f_attn_scale = 0.0e+00 print_info: n_ff = 25600 print_info: n_expert = 0 print_info: n_expert_used = 0 print_info: causal attn = 1 print_info: pooling type = 0 print_info: rope type = 2 print_info: rope scaling = linear print_info: freq_base_train = 1000000.0 print_info: freq_scale_train = 1 print_info: n_ctx_orig_yarn = 40960 print_info: rope_finetuned = unknown print_info: ssm_d_conv = 0 print_info: ssm_d_inner = 0 print_info: ssm_d_state = 0 print_info: ssm_dt_rank = 0 print_info: ssm_dt_b_c_rms = 0 print_info: model type = 32B print_info: model params = 32.76 B print_info: general.name = Qwen3 32B print_info: vocab type = BPE print_info: n_vocab = 151936 print_info: n_merges = 151387 print_info: BOS token = 151643 '<|endoftext|>' print_info: EOS token = 151645 '<|im_end|>' print_info: EOT token = 151645 '<|im_end|>' print_info: PAD token = 151643 '<|endoftext|>' print_info: LF token = 198 'Ċ' print_info: FIM PRE token = 151659 '<|fim_prefix|>' print_info: FIM SUF token = 151661 '<|fim_suffix|>' print_info: FIM MID token = 151660 '<|fim_middle|>' print_info: FIM PAD token = 151662 '<|fim_pad|>' print_info: FIM REP token = 151663 '<|repo_name|>' print_info: FIM SEP token = 151664 '<|file_sep|>' print_info: EOG token = 151643 '<|endoftext|>' print_info: EOG token = 151645 '<|im_end|>' print_info: EOG token = 151662 '<|fim_pad|>' print_info: EOG token = 151663 '<|repo_name|>' print_info: EOG token = 151664 '<|file_sep|>' print_info: max token length = 256 load_tensors: loading model tensors, this can take a while... (mmap = true) load_tensors: offloading 64 repeating layers to GPU load_tensors: offloading output layer to GPU load_tensors: offloaded 65/65 layers to GPU load_tensors: CUDA0 model buffer size = 18842.40 MiB load_tensors: CPU_Mapped model buffer size = 417.30 MiB llama_context: constructing llama_context llama_context: n_seq_max = 1 llama_context: n_ctx = 4096 llama_context: n_ctx_per_seq = 4096 llama_context: n_batch = 512 llama_context: n_ubatch = 512 llama_context: causal_attn = 1 llama_context: flash_attn = 0 llama_context: freq_base = 1000000.0 llama_context: freq_scale = 1 llama_context: n_ctx_per_seq (4096) < n_ctx_train (40960) -- the full capacity of the model will not be utilized llama_context: CUDA_Host output buffer size = 0.60 MiB llama_kv_cache_unified: kv_size = 4096, type_k = 'f16', type_v = 'f16', n_layer = 64, can_shift = 1, padding = 32 llama_kv_cache_unified: CUDA0 KV buffer size = 1024.00 MiB llama_kv_cache_unified: KV self size = 1024.00 MiB, K (f16): 512.00 MiB, V (f16): 512.00 MiB llama_context: CUDA0 compute buffer size = 572.00 MiB llama_context: CUDA_Host compute buffer size = 18.01 MiB llama_context: graph nodes = 2438 llama_context: graph splits = 2 time=2026-03-11T03:08:24.046Z level=INFO source=server.go:637 msg="llama runner started in 3.76 seconds" ```
GiteaMirror commented 2026-04-29 10:13:10 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Mar 11, 2026):

time=2026-03-11T03:02:00.867Z level=INFO source=routes.go:1763 msg="vram-based default context" total_vram="48.0 GiB" default_num_ctx=262144

#14116

<!-- gh-comment-id:4036117961 --> @rick-github commented on GitHub (Mar 11, 2026): ``` time=2026-03-11T03:02:00.867Z level=INFO source=routes.go:1763 msg="vram-based default context" total_vram="48.0 GiB" default_num_ctx=262144 ``` #14116
GiteaMirror commented 2026-04-29 10:13:13 -05:00
Author
Owner
Copy Link

@sy960923 commented on GitHub (Mar 11, 2026):

time=2026-03-11T03:02:00.867Z level=INFO source=routes.go:1763 msg="vram-based default context" total_vram="48.0 GiB" default_num_ctx=262144

#14116

明白了

<!-- gh-comment-id:4036919609 --> @sy960923 commented on GitHub (Mar 11, 2026): > ``` > time=2026-03-11T03:02:00.867Z level=INFO source=routes.go:1763 msg="vram-based default context" total_vram="48.0 GiB" default_num_ctx=262144 > ``` > > [#14116](https://github.com/ollama/ollama/issues/14116) 明白了
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#56062
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?