github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-05 23:53:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #2598] Add ROCm support on windows #1530

New Issue
Closed
opened 2026-04-12 11:26:34 -05:00 by GiteaMirror · 16 comments
GiteaMirror commented 2026-04-12 11:26:34 -05:00
Owner
Copy Link

Originally created by @dhiltgen on GitHub (Feb 19, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/2598

Originally assigned to: @dhiltgen on GitHub.

Users with Radeon cards would like to be able to take advantage of the new native windows app and not have to resort to WSL2 to get support for their AMD GPUs.

Originally created by @dhiltgen on GitHub (Feb 19, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/2598 Originally assigned to: @dhiltgen on GitHub. Users with Radeon cards would like to be able to take advantage of the new native windows app and not have to resort to WSL2 to get support for their AMD GPUs.
GiteaMirror added the feature requestamd labels 2026-04-12 11:26:34 -05:00
GiteaMirror commented 2026-04-12 11:26:35 -05:00
Author
Owner
Copy Link

@zhiyuan1i commented on GitHub (Feb 21, 2024):

Could you please consider users who have other cards that support DirectX 12?

Some Intel, Qualcomm, and Chinese graphics card manufacturers often support DirectX 12, which can be utilized to accelerate computations for large models. May I ask if you have considered incorporating this functionality into the Windows client?

Thank you!

<!-- gh-comment-id:1956230360 --> @zhiyuan1i commented on GitHub (Feb 21, 2024): Could you please consider users who have other cards that support DirectX 12? Some Intel, Qualcomm, and Chinese graphics card manufacturers often support DirectX 12, which can be utilized to accelerate computations for large models. May I ask if you have considered incorporating this functionality into the Windows client? Thank you!
GiteaMirror commented 2026-04-12 11:26:35 -05:00
Author
Owner
Copy Link

@DK013 commented on GitHub (Feb 24, 2024):

As @uniartisan suggested, we would all love a backend that leverages DirectX 12 on windows machines, since it's widely available with almost all GPUs with windows drivers. and to be honest the list of ROCm supported cards are not that much. I'm sure this will take some time IF the team goes down this route.

However, here's a good news. recently AMD pulled out their support from the ZLUDA project and since then the author made the project source code available in Github. ZLUDA lets you run unmodified CUDA applications with near-native performance on AMD GPUs. Seems like the author was working on Intel GPU support as well but in last build he removed that.

Anyway, I tried ollama windows with zluda on my RX 5700XT, and the outcome was amazing !!
it's still not near as fast as the inference speed on my macbook pro, but it's much much faster than CPU inference on a windows pc with AMD cards. There're still a few caveats for different hardware scenarios but mostly it works.
So, anyone looking for a quick workaround may find this very helpful until further official support arrives for DIrectX 12.
#2529

<!-- gh-comment-id:1962267717 --> @DK013 commented on GitHub (Feb 24, 2024): As @uniartisan suggested, we would all love a backend that leverages DirectX 12 on windows machines, since it's widely available with almost all GPUs with windows drivers. and to be honest the list of ROCm supported cards are not that much. I'm sure this will take some time IF the team goes down this route. **However, here's a good news.** recently AMD pulled out their support from the [ZLUDA](https://github.com/vosen/ZLUDA) project and since then the author made the project source code available in Github. ZLUDA lets you run unmodified CUDA applications with near-native performance on AMD GPUs. Seems like the author was working on Intel GPU support as well but in last build he removed that. Anyway, I tried ollama windows with zluda on my RX 5700XT, and the outcome was amazing !! it's still not near as fast as the inference speed on my macbook pro, but it's much much faster than CPU inference on a windows pc with AMD cards. There're still a few caveats for different hardware scenarios but mostly it works. So, anyone looking for a quick workaround may find this very helpful until further official support arrives for DIrectX 12. #2529
GiteaMirror commented 2026-04-12 11:26:36 -05:00
Author
Owner
Copy Link

@ipfans commented on GitHub (Feb 29, 2024):

It seems like still fallback to CPU when I run ollam powered by zluda:

>zluda.exe -- %UserProfile%/AppData/Local/Programs/Ollama/ollama.exe serve
time=2024-02-29T09:31:02.916+08:00 level=INFO source=images.go:710 msg="total blobs: 5"
time=2024-02-29T09:31:02.928+08:00 level=INFO source=images.go:717 msg="total unused blobs removed: 0"
time=2024-02-29T09:31:02.929+08:00 level=INFO source=routes.go:1019 msg="Listening on 127.0.0.1:11434 (version 0.1.27)"
time=2024-02-29T09:31:02.929+08:00 level=INFO source=payload_common.go:107 msg="Extracting dynamic libraries..."
time=2024-02-29T09:31:03.053+08:00 level=INFO source=payload_common.go:146 msg="Dynamic LLM libraries [cpu cpu_avx2 cpu_avx cuda_v11.3]"

after I run ollama run mistral:

time=2024-02-29T09:32:24.173+08:00 level=INFO source=gpu.go:94 msg="Detecting GPU type"
time=2024-02-29T09:32:24.173+08:00 level=INFO source=gpu.go:265 msg="Searching for GPU management library nvml.dll"
time=2024-02-29T09:32:24.184+08:00 level=INFO source=gpu.go:311 msg="Discovered GPU libraries: [c:\\Windows\\System32\\nvml.dll C:\\Windows\\system32\\nvml.dll C:\\WINDOWS\\system32\\nvml.dll]"
time=2024-02-29T09:32:24.194+08:00 level=INFO source=gpu.go:99 msg="Nvidia GPU detected"
time=2024-02-29T09:32:24.194+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-02-29T09:32:24.194+08:00 level=INFO source=gpu.go:143 msg="error looking up CUDA GPU compute capability: device compute capability lookup failure 0: 3"
time=2024-02-29T09:32:24.195+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-02-29T09:32:24.195+08:00 level=INFO source=gpu.go:143 msg="error looking up CUDA GPU compute capability: device compute capability lookup failure 0: 3"
time=2024-02-29T09:32:24.195+08:00 level=INFO source=llm.go:77 msg="GPU not available, falling back to CPU"

Hardware: AMD Radeon RX 5950 XT

Updated: It seems affected by I previous installed nv drivers. After uninstall all nv softwares/drivers, the error messages changed:

time=2024-02-29T09:52:42.349+08:00 level=INFO source=gpu.go:94 msg="Detecting GPU type"
time=2024-02-29T09:52:42.349+08:00 level=INFO source=gpu.go:265 msg="Searching for GPU management library nvml.dll"
time=2024-02-29T09:52:42.359+08:00 level=INFO source=gpu.go:311 msg="Discovered GPU libraries: []"
time=2024-02-29T09:52:42.359+08:00 level=INFO source=gpu.go:265 msg="Searching for GPU management library rocm_smi64.dll"
time=2024-02-29T09:52:42.368+08:00 level=INFO source=gpu.go:311 msg="Discovered GPU libraries: []"
time=2024-02-29T09:52:42.368+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-02-29T09:52:42.368+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-02-29T09:52:42.369+08:00 level=INFO source=llm.go:77 msg="GPU not available, falling back to CPU"
<!-- gh-comment-id:1970237529 --> @ipfans commented on GitHub (Feb 29, 2024): It seems like still fallback to CPU when I run ollam powered by zluda: ``` >zluda.exe -- %UserProfile%/AppData/Local/Programs/Ollama/ollama.exe serve time=2024-02-29T09:31:02.916+08:00 level=INFO source=images.go:710 msg="total blobs: 5" time=2024-02-29T09:31:02.928+08:00 level=INFO source=images.go:717 msg="total unused blobs removed: 0" time=2024-02-29T09:31:02.929+08:00 level=INFO source=routes.go:1019 msg="Listening on 127.0.0.1:11434 (version 0.1.27)" time=2024-02-29T09:31:02.929+08:00 level=INFO source=payload_common.go:107 msg="Extracting dynamic libraries..." time=2024-02-29T09:31:03.053+08:00 level=INFO source=payload_common.go:146 msg="Dynamic LLM libraries [cpu cpu_avx2 cpu_avx cuda_v11.3]" ``` after I run `ollama run mistral`: ``` time=2024-02-29T09:32:24.173+08:00 level=INFO source=gpu.go:94 msg="Detecting GPU type" time=2024-02-29T09:32:24.173+08:00 level=INFO source=gpu.go:265 msg="Searching for GPU management library nvml.dll" time=2024-02-29T09:32:24.184+08:00 level=INFO source=gpu.go:311 msg="Discovered GPU libraries: [c:\\Windows\\System32\\nvml.dll C:\\Windows\\system32\\nvml.dll C:\\WINDOWS\\system32\\nvml.dll]" time=2024-02-29T09:32:24.194+08:00 level=INFO source=gpu.go:99 msg="Nvidia GPU detected" time=2024-02-29T09:32:24.194+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2024-02-29T09:32:24.194+08:00 level=INFO source=gpu.go:143 msg="error looking up CUDA GPU compute capability: device compute capability lookup failure 0: 3" time=2024-02-29T09:32:24.195+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2024-02-29T09:32:24.195+08:00 level=INFO source=gpu.go:143 msg="error looking up CUDA GPU compute capability: device compute capability lookup failure 0: 3" time=2024-02-29T09:32:24.195+08:00 level=INFO source=llm.go:77 msg="GPU not available, falling back to CPU" ``` Hardware: AMD Radeon RX 5950 XT Updated: It seems affected by I previous installed nv drivers. After uninstall all nv softwares/drivers, the error messages changed: ``` time=2024-02-29T09:52:42.349+08:00 level=INFO source=gpu.go:94 msg="Detecting GPU type" time=2024-02-29T09:52:42.349+08:00 level=INFO source=gpu.go:265 msg="Searching for GPU management library nvml.dll" time=2024-02-29T09:52:42.359+08:00 level=INFO source=gpu.go:311 msg="Discovered GPU libraries: []" time=2024-02-29T09:52:42.359+08:00 level=INFO source=gpu.go:265 msg="Searching for GPU management library rocm_smi64.dll" time=2024-02-29T09:52:42.368+08:00 level=INFO source=gpu.go:311 msg="Discovered GPU libraries: []" time=2024-02-29T09:52:42.368+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2024-02-29T09:52:42.368+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2024-02-29T09:52:42.369+08:00 level=INFO source=llm.go:77 msg="GPU not available, falling back to CPU" ```
GiteaMirror commented 2026-04-12 11:26:36 -05:00
Author
Owner
Copy Link

@Inokinoki commented on GitHub (Feb 29, 2024):

It seems like still fallback to CPU when I run ollam powered by zluda:

>zluda.exe -- %UserProfile%/AppData/Local/Programs/Ollama/ollama.exe serve
time=2024-02-29T09:31:02.916+08:00 level=INFO source=images.go:710 msg="total blobs: 5"
time=2024-02-29T09:31:02.928+08:00 level=INFO source=images.go:717 msg="total unused blobs removed: 0"
time=2024-02-29T09:31:02.929+08:00 level=INFO source=routes.go:1019 msg="Listening on 127.0.0.1:11434 (version 0.1.27)"
time=2024-02-29T09:31:02.929+08:00 level=INFO source=payload_common.go:107 msg="Extracting dynamic libraries..."
time=2024-02-29T09:31:03.053+08:00 level=INFO source=payload_common.go:146 msg="Dynamic LLM libraries [cpu cpu_avx2 cpu_avx cuda_v11.3]"

after I run ollama run mistral:

time=2024-02-29T09:32:24.173+08:00 level=INFO source=gpu.go:94 msg="Detecting GPU type"
time=2024-02-29T09:32:24.173+08:00 level=INFO source=gpu.go:265 msg="Searching for GPU management library nvml.dll"
time=2024-02-29T09:32:24.184+08:00 level=INFO source=gpu.go:311 msg="Discovered GPU libraries: [c:\\Windows\\System32\\nvml.dll C:\\Windows\\system32\\nvml.dll C:\\WINDOWS\\system32\\nvml.dll]"
time=2024-02-29T09:32:24.194+08:00 level=INFO source=gpu.go:99 msg="Nvidia GPU detected"
time=2024-02-29T09:32:24.194+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-02-29T09:32:24.194+08:00 level=INFO source=gpu.go:143 msg="error looking up CUDA GPU compute capability: device compute capability lookup failure 0: 3"
time=2024-02-29T09:32:24.195+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-02-29T09:32:24.195+08:00 level=INFO source=gpu.go:143 msg="error looking up CUDA GPU compute capability: device compute capability lookup failure 0: 3"
time=2024-02-29T09:32:24.195+08:00 level=INFO source=llm.go:77 msg="GPU not available, falling back to CPU"

Hardware: AMD Radeon RX 5950 XT

Updated: It seems affected by I previous installed nv drivers. After uninstall all nv softwares/drivers, the error messages changed:

time=2024-02-29T09:52:42.349+08:00 level=INFO source=gpu.go:94 msg="Detecting GPU type"
time=2024-02-29T09:52:42.349+08:00 level=INFO source=gpu.go:265 msg="Searching for GPU management library nvml.dll"
time=2024-02-29T09:52:42.359+08:00 level=INFO source=gpu.go:311 msg="Discovered GPU libraries: []"
time=2024-02-29T09:52:42.359+08:00 level=INFO source=gpu.go:265 msg="Searching for GPU management library rocm_smi64.dll"
time=2024-02-29T09:52:42.368+08:00 level=INFO source=gpu.go:311 msg="Discovered GPU libraries: []"
time=2024-02-29T09:52:42.368+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-02-29T09:52:42.368+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-02-29T09:52:42.369+08:00 level=INFO source=llm.go:77 msg="GPU not available, falling back to CPU"

I'm facing the similar issue.

AFAIK, there is a recent change explicitly detect the cuda compatibility. However, the operation to get cuda compatibility is not yet implemented by zluda, which fails the GPU detection.

I remove the code for compute compatibility detection and build locally, it works fine.

<!-- gh-comment-id:1970710747 --> @Inokinoki commented on GitHub (Feb 29, 2024): > It seems like still fallback to CPU when I run ollam powered by zluda: > > ``` > >zluda.exe -- %UserProfile%/AppData/Local/Programs/Ollama/ollama.exe serve > time=2024-02-29T09:31:02.916+08:00 level=INFO source=images.go:710 msg="total blobs: 5" > time=2024-02-29T09:31:02.928+08:00 level=INFO source=images.go:717 msg="total unused blobs removed: 0" > time=2024-02-29T09:31:02.929+08:00 level=INFO source=routes.go:1019 msg="Listening on 127.0.0.1:11434 (version 0.1.27)" > time=2024-02-29T09:31:02.929+08:00 level=INFO source=payload_common.go:107 msg="Extracting dynamic libraries..." > time=2024-02-29T09:31:03.053+08:00 level=INFO source=payload_common.go:146 msg="Dynamic LLM libraries [cpu cpu_avx2 cpu_avx cuda_v11.3]" > ``` > > after I run `ollama run mistral`: > > ``` > time=2024-02-29T09:32:24.173+08:00 level=INFO source=gpu.go:94 msg="Detecting GPU type" > time=2024-02-29T09:32:24.173+08:00 level=INFO source=gpu.go:265 msg="Searching for GPU management library nvml.dll" > time=2024-02-29T09:32:24.184+08:00 level=INFO source=gpu.go:311 msg="Discovered GPU libraries: [c:\\Windows\\System32\\nvml.dll C:\\Windows\\system32\\nvml.dll C:\\WINDOWS\\system32\\nvml.dll]" > time=2024-02-29T09:32:24.194+08:00 level=INFO source=gpu.go:99 msg="Nvidia GPU detected" > time=2024-02-29T09:32:24.194+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" > time=2024-02-29T09:32:24.194+08:00 level=INFO source=gpu.go:143 msg="error looking up CUDA GPU compute capability: device compute capability lookup failure 0: 3" > time=2024-02-29T09:32:24.195+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" > time=2024-02-29T09:32:24.195+08:00 level=INFO source=gpu.go:143 msg="error looking up CUDA GPU compute capability: device compute capability lookup failure 0: 3" > time=2024-02-29T09:32:24.195+08:00 level=INFO source=llm.go:77 msg="GPU not available, falling back to CPU" > ``` > > Hardware: AMD Radeon RX 5950 XT > > Updated: It seems affected by I previous installed nv drivers. After uninstall all nv softwares/drivers, the error messages changed: > > ``` > time=2024-02-29T09:52:42.349+08:00 level=INFO source=gpu.go:94 msg="Detecting GPU type" > time=2024-02-29T09:52:42.349+08:00 level=INFO source=gpu.go:265 msg="Searching for GPU management library nvml.dll" > time=2024-02-29T09:52:42.359+08:00 level=INFO source=gpu.go:311 msg="Discovered GPU libraries: []" > time=2024-02-29T09:52:42.359+08:00 level=INFO source=gpu.go:265 msg="Searching for GPU management library rocm_smi64.dll" > time=2024-02-29T09:52:42.368+08:00 level=INFO source=gpu.go:311 msg="Discovered GPU libraries: []" > time=2024-02-29T09:52:42.368+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" > time=2024-02-29T09:52:42.368+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" > time=2024-02-29T09:52:42.369+08:00 level=INFO source=llm.go:77 msg="GPU not available, falling back to CPU" > ``` I'm facing the similar issue. AFAIK, there is a recent change explicitly detect the cuda compatibility. However, the operation to get cuda compatibility is not yet implemented by zluda, which fails the GPU detection. I remove the code for compute compatibility detection and build locally, it works fine.
GiteaMirror commented 2026-04-12 11:26:36 -05:00
Author
Owner
Copy Link

@ipfans commented on GitHub (Feb 29, 2024):

@Inokinoki Thanks for you explanation. Could you also share the file/line that we can change?

<!-- gh-comment-id:1970960506 --> @ipfans commented on GitHub (Feb 29, 2024): @Inokinoki Thanks for you explanation. Could you also share the file/line that we can change?
GiteaMirror commented 2026-04-12 11:26:37 -05:00
Author
Owner
Copy Link

@Inokinoki commented on GitHub (Feb 29, 2024):

@Inokinoki Thanks for you explanation. Could you also share the file/line that we can change?

It's basically here: 076237b8ea/gpu/gpu_info_cuda.c (L199)
I by-passed this by manually setting major to 5 and minor to 0, which is the lowest required version.

In ZLUDA, nvmlDeviceGetCudaComputeCapability is not implemented:
af0216b1a0/zluda_ml/src/nvml.rs (L2871)

<!-- gh-comment-id:1971059013 --> @Inokinoki commented on GitHub (Feb 29, 2024): > @Inokinoki Thanks for you explanation. Could you also share the file/line that we can change? It's basically here: https://github.com/ollama/ollama/blob/076237b8ea8fbf686c6557012261d679b0254560/gpu/gpu_info_cuda.c#L199 I by-passed this by manually setting major to 5 and minor to 0, which is the lowest required version. In ZLUDA, `nvmlDeviceGetCudaComputeCapability` is not implemented: https://github.com/vosen/ZLUDA/blob/af0216b1a000cb2f8b705611a9198c916cca3146/zluda_ml/src/nvml.rs#L2871
GiteaMirror commented 2026-04-12 11:26:37 -05:00
Author
Owner
Copy Link

@shanoaice commented on GitHub (Mar 4, 2024):

I have noticed that AMD GPU does not actually have support on WSL2 (ROCm and clinfo cannot detect any capable device), and Windows currently lacks a ROCm v6 release, so does it mean that we still need to wait?

Also, I have installed Nvidia CUDA SDK on my machine, but during compilation GPU offloading doesn't seem to be enabled, and I didn't see any instruction in the document, what should I do?

<!-- gh-comment-id:1975765273 --> @shanoaice commented on GitHub (Mar 4, 2024): I have noticed that AMD GPU does not actually have support on WSL2 (ROCm and clinfo cannot detect any capable device), and Windows currently lacks a ROCm v6 release, so does it mean that we still need to wait? Also, I have installed Nvidia CUDA SDK on my machine, but during compilation GPU offloading doesn't seem to be enabled, and I didn't see any instruction in the document, what should I do?
GiteaMirror commented 2026-04-12 11:26:38 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Mar 5, 2024):

@shanoaice PR #2885 leverages ROCm v5.7 which is the latest version available on windows at the moment. We're still testing to validate the change, but the hope is to roll this into an upcoming release soon.

<!-- gh-comment-id:1979260055 --> @dhiltgen commented on GitHub (Mar 5, 2024): @shanoaice PR #2885 leverages ROCm v5.7 which is the latest version available on windows at the moment. We're still testing to validate the change, but the hope is to roll this into an upcoming release soon.
GiteaMirror commented 2026-04-12 11:26:38 -05:00
Author
Owner
Copy Link

@shanoaice commented on GitHub (Mar 6, 2024):

@dhiltgen Thanks, tried to build from #2885 and succeed. Only problem is that I got a RX 6600M, and it is gfx1032 instead of the officially supported gfx1030. ROCm will refrain from using my GPU if I directly run ollama server. If I set HSA_OVERRIDE_GFX_VERSION to 10.3.0 like on Linux, I get the following error:

rocBLAS error: Cannot read C:\Program Files\AMD\ROCm\5.7\bin\/rocblas/library/TensileLibrary.dat: No such file or directory for GPU arch : gfx1032

Full Log

If I then try to copy gfx1030 files in the ROCm directory and rename them to gfx1032, I got another error:

CUDA error: invalid device function
  current device: 0, in function ggml_cuda_op_flatten at D:/projects/build/ollama-rocm6/llm/llama.cpp/ggml-cuda.cu:10012
  hipGetLastError()

Full Log

<!-- gh-comment-id:1980107302 --> @shanoaice commented on GitHub (Mar 6, 2024): @dhiltgen Thanks, tried to build from #2885 and succeed. Only problem is that I got a RX 6600M, and it is `gfx1032` instead of the officially supported `gfx1030`. ROCm will refrain from using my GPU if I directly run ollama server. If I set `HSA_OVERRIDE_GFX_VERSION` to `10.3.0` like on Linux, I get the following error: ``` rocBLAS error: Cannot read C:\Program Files\AMD\ROCm\5.7\bin\/rocblas/library/TensileLibrary.dat: No such file or directory for GPU arch : gfx1032 ``` [Full Log](https://github.com/ollama/ollama/files/14504943/ollama-server-2024030602.log) If I then try to copy `gfx1030` files in the ROCm directory and rename them to `gfx1032`, I got another error: ``` CUDA error: invalid device function current device: 0, in function ggml_cuda_op_flatten at D:/projects/build/ollama-rocm6/llm/llama.cpp/ggml-cuda.cu:10012 hipGetLastError() ``` [Full Log](https://github.com/ollama/ollama/files/14504934/ollama-server-2024030601.log)
GiteaMirror commented 2026-04-12 11:26:38 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Mar 6, 2024):

@shanoaice with PR #2885 I'm changing around how we do ROCm discovery on linux in addition to adding windows support for Radeon GPUs. I'm not sure why it's not respecting the HSA_OVERRIDE_GFX_VERSION setting and still looking up the wrong tensile file. I'm hoping to get that PR merged soon and once we get a pre-release cut including it, I'll ask folks on this and a few other tickets to give it a try on their systems.

<!-- gh-comment-id:1981834049 --> @dhiltgen commented on GitHub (Mar 6, 2024): @shanoaice with PR #2885 I'm changing around how we do ROCm discovery on linux in addition to adding windows support for Radeon GPUs. I'm not sure why it's not respecting the `HSA_OVERRIDE_GFX_VERSION` setting and still looking up the wrong tensile file. I'm hoping to get that PR merged soon and once we get a pre-release cut including it, I'll ask folks on this and a few other tickets to give it a try on their systems.
GiteaMirror commented 2026-04-12 11:26:39 -05:00
Author
Owner
Copy Link

@ravarcade commented on GitHub (Mar 7, 2024):

@dhiltgen Thanks a lot for this!
I just tested your build on my 7800XT.

load duration:        25.1129ms
prompt eval count:    44 token(s)
prompt eval duration: 196.244ms
prompt eval rate:     224.21 tokens/s
eval count:           220 token(s)
eval duration:        4.653627s
eval rate:            47.27 tokens/s```

On windows. Codellama:13b model.
It works greate in VC with LLama Coder extension.
<!-- gh-comment-id:1984120321 --> @ravarcade commented on GitHub (Mar 7, 2024): @dhiltgen Thanks a lot for this! I just tested your build on my 7800XT. ```total duration: 4.8757218s load duration: 25.1129ms prompt eval count: 44 token(s) prompt eval duration: 196.244ms prompt eval rate: 224.21 tokens/s eval count: 220 token(s) eval duration: 4.653627s eval rate: 47.27 tokens/s``` On windows. Codellama:13b model. It works greate in VC with LLama Coder extension.
GiteaMirror commented 2026-04-12 11:26:39 -05:00
Author
Owner
Copy Link

@ipfans commented on GitHub (Mar 9, 2024):

Thanks a lot for your works. I just tried https://github.com/dhiltgen/ollama/releases for rocm support, but I found it will be failed when using mixtral model. Here is a log for this panic:

time=2024-03-09T10:13:54.011+08:00 level=INFO source=images.go:800 msg="total blobs: 8"
time=2024-03-09T10:13:54.022+08:00 level=INFO source=images.go:807 msg="total unused blobs removed: 0"
time=2024-03-09T10:13:54.023+08:00 level=INFO source=routes.go:1019 msg="Listening on 127.0.0.1:11434 (version 0.1.28-23-ge690faf)"
time=2024-03-09T10:13:54.023+08:00 level=INFO source=payload_common.go:109 msg="Extracting dynamic libraries to C:\\Users\\user\\AppData\\Local\\Temp\\ollama\\payloads ..."
time=2024-03-09T10:13:54.194+08:00 level=INFO source=payload_common.go:154 msg="Dynamic LLM libraries [rocm_v5.7 cuda_v11.3 cpu cpu_avx2 cpu_avx cuda_v12.3]"
[GIN] 2024/03/09 - 10:13:54 | 200 |            0s |       127.0.0.1 | HEAD     "/"
[GIN] 2024/03/09 - 10:13:54 | 200 |      1.0486ms |       127.0.0.1 | POST     "/api/show"
[GIN] 2024/03/09 - 10:13:54 | 200 |       525.7µs |       127.0.0.1 | POST     "/api/show"
time=2024-03-09T10:13:54.847+08:00 level=INFO source=gpu.go:77 msg="Detecting GPU type"
time=2024-03-09T10:13:54.847+08:00 level=INFO source=gpu.go:191 msg="Searching for GPU management library nvml.dll"
time=2024-03-09T10:13:54.858+08:00 level=INFO source=gpu.go:237 msg="Discovered GPU libraries: []"
time=2024-03-09T10:13:54.858+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-03-09T10:13:54.875+08:00 level=INFO source=amd_windows.go:40 msg="AMD Driver: 50732000"
time=2024-03-09T10:13:54.877+08:00 level=INFO source=amd_windows.go:69 msg="detected 1 hip devices"
time=2024-03-09T10:13:54.877+08:00 level=INFO source=amd_windows.go:87 msg="[0] Name: AMD Radeon RX 7900 XTX"
time=2024-03-09T10:13:54.877+08:00 level=INFO source=amd_windows.go:90 msg="[0] GcnArchName: gfx1100"
time=2024-03-09T10:13:54.877+08:00 level=INFO source=amd_windows.go:106 msg="amdgpu [0] gfx1100 is supported"
time=2024-03-09T10:13:55.129+08:00 level=INFO source=amd_windows.go:117 msg="[0] Total Mem: 25618808832"
time=2024-03-09T10:13:55.129+08:00 level=INFO source=amd_windows.go:118 msg="[0] Free Mem:  25753026560"
time=2024-03-09T10:13:55.129+08:00 level=INFO source=assets.go:36 msg="Updating PATH to C:\\Users\\user\\AppData\\Local\\Programs\\Ollama\\rocm;C:\\Python310\\Scripts\\;C:\\Python310\\;C:\\Program Files\\Zulu\\zulu-8\\bin\\;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Windows\\System32\\OpenSSH\\;C:\\Program Files\\Intel\\WiFi\\bin\\;C:\\Program Files\\Common Files\\Intel\\WirelessCommon\\;D:\\TDM-GCC-64\\bin;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\;C:\\WINDOWS\\System32\\OpenSSH\\;C:\\Program Files\\dotnet\\;C:\\ProgramData\\chocolatey\\bin;C:\\Program Files (x86)\\dotnet\\;C:\\Program Files\\nodejs\\;C:\\Program Files\\D2\\;C:\\Program Files\\Amazon\\AWSSAMCLI\\bin\\;C:\\Program Files\\Amazon\\AWSCLIV2\\;D:\\微信web开发者工具\\dll;C:\\Program Files (x86)\\Windows Kits\\10\\Windows Performance Toolkit\\;C:\\Program Files\\Docker\\Docker\\resources\\bin;C:\\Program Files\\Go\\bin;C:\\Program Files\\PowerShell\\7\\;C:\\Users\\user\\scoop\\shims;C:\\Users\\user\\AppData\\Local\\Microsoft\\WindowsApps;G:\\GOPATH\\bin;C:\\Program Files\\Intel\\WiFi\\bin\\;C:\\Program Files\\Common Files\\Intel\\WirelessCommon\\;C:\\Users\\user\\AppData\\Local\\GitHubDesktop\\bin;D:\\jdk-11\\bin;C:\\Program Files (x86)\\Graphviz2.38\\bin;C:\\Users\\user\\AppData\\Local\\Yarn\\bin;C:\\Users\\user\\.dotnet\\tools;D:\\flutter\\bin;D:\\Program Files\\Git\\usr\\bin;D:\\GitHub CLI\\;C:\\Program Files (x86)\\Nmap;C:\\Program Files\\Graphviz\\bin;C:\\Program Files\\Go\\bin;D:\\flutter\\.pub-cache\\bin;C:\\Users\\user\\go\\bin;C:\\Users\\user\\AppData\\Local\\Atlassian\\SourceTree\\git_local\\bin;C:\\Users\\user\\AppData\\Local\\JetBrains\\Toolbox\\scripts;C:\\Users\\user\\.dotnet\\tools;C:\\Users\\user\\AppData\\Roaming\\npm;C:\\Users\\user\\AppData\\Local\\Programs\\Microsoft VS Code\\bin;C:\\Users\\user\\go\\bin;C:\\Users\\user\\.detaspace\\bin;C:\\Users\\user\\.dotnet\\tools;C:\\Users\\user\\go\\bin;C:\\Users\\user\\AppData\\Local\\Programs\\Ollama"
time=2024-03-09T10:13:55.195+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-03-09T10:13:55.206+08:00 level=INFO source=amd_windows.go:40 msg="AMD Driver: 50732000"
time=2024-03-09T10:13:55.207+08:00 level=INFO source=amd_windows.go:69 msg="detected 1 hip devices"
time=2024-03-09T10:13:55.207+08:00 level=INFO source=amd_windows.go:87 msg="[0] Name: AMD Radeon RX 7900 XTX"
time=2024-03-09T10:13:55.207+08:00 level=INFO source=amd_windows.go:90 msg="[0] GcnArchName: gfx1100"
time=2024-03-09T10:13:55.207+08:00 level=INFO source=amd_windows.go:106 msg="amdgpu [0] gfx1100 is supported"
time=2024-03-09T10:13:55.443+08:00 level=INFO source=amd_windows.go:117 msg="[0] Total Mem: 25618808832"
time=2024-03-09T10:13:55.443+08:00 level=INFO source=amd_windows.go:118 msg="[0] Free Mem:  25753026560"
time=2024-03-09T10:13:55.509+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-03-09T10:13:55.509+08:00 level=INFO source=assets.go:36 msg="Updating PATH to C:\\Users\\user\\AppData\\Local\\Temp\\ollama\\payloads\\rocm_v5.7;C:\\Users\\user\\AppData\\Local\\Programs\\Ollama\\rocm;C:\\Python310\\Scripts\\;C:\\Python310\\;C:\\Program Files\\Zulu\\zulu-8\\bin\\;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Windows\\System32\\OpenSSH\\;C:\\Program Files\\Intel\\WiFi\\bin\\;C:\\Program Files\\Common Files\\Intel\\WirelessCommon\\;D:\\TDM-GCC-64\\bin;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\;C:\\WINDOWS\\System32\\OpenSSH\\;C:\\Program Files\\dotnet\\;C:\\ProgramData\\chocolatey\\bin;C:\\Program Files (x86)\\dotnet\\;C:\\Program Files\\nodejs\\;C:\\Program Files\\D2\\;C:\\Program Files\\Amazon\\AWSSAMCLI\\bin\\;C:\\Program Files\\Amazon\\AWSCLIV2\\;D:\\微信web开发者工具\\dll;C:\\Program Files (x86)\\Windows Kits\\10\\Windows Performance Toolkit\\;C:\\Program Files\\Docker\\Docker\\resources\\bin;C:\\Program Files\\Go\\bin;C:\\Program Files\\PowerShell\\7\\;C:\\Users\\user\\scoop\\shims;C:\\Users\\user\\AppData\\Local\\Microsoft\\WindowsApps;G:\\GOPATH\\bin;C:\\Program Files\\Intel\\WiFi\\bin\\;C:\\Program Files\\Common Files\\Intel\\WirelessCommon\\;C:\\Users\\user\\AppData\\Local\\GitHubDesktop\\bin;D:\\jdk-11\\bin;C:\\Program Files (x86)\\Graphviz2.38\\bin;C:\\Users\\user\\AppData\\Local\\Yarn\\bin;C:\\Users\\user\\.dotnet\\tools;D:\\flutter\\bin;D:\\Program Files\\Git\\usr\\bin;D:\\GitHub CLI\\;C:\\Program Files (x86)\\Nmap;C:\\Program Files\\Graphviz\\bin;C:\\Program Files\\Go\\bin;D:\\flutter\\.pub-cache\\bin;C:\\Users\\user\\go\\bin;C:\\Users\\user\\AppData\\Local\\Atlassian\\SourceTree\\git_local\\bin;C:\\Users\\user\\AppData\\Local\\JetBrains\\Toolbox\\scripts;C:\\Users\\user\\.dotnet\\tools;C:\\Users\\user\\AppData\\Roaming\\npm;C:\\Users\\user\\AppData\\Local\\Programs\\Microsoft VS Code\\bin;C:\\Users\\user\\go\\bin;C:\\Users\\user\\.detaspace\\bin;C:\\Users\\user\\.dotnet\\tools;C:\\Users\\user\\go\\bin;C:\\Users\\user\\AppData\\Local\\Programs\\Ollama"
time=2024-03-09T10:13:55.533+08:00 level=INFO source=dyn_ext_server.go:90 msg="Loading Dynamic llm server: C:\\Users\\user\\AppData\\Local\\Temp\\ollama\\payloads\\rocm_v5.7\\ext_server.dll"
time=2024-03-09T10:13:55.533+08:00 level=INFO source=dyn_ext_server.go:150 msg="Initializing llama server"
ggml_init_cublas: GGML_CUDA_FORCE_MMQ:   no
ggml_init_cublas: CUDA_USE_TENSOR_CORES: yes
ggml_init_cublas: found 1 ROCm devices:
  Device 0: AMD Radeon RX 7900 XTX, compute capability 11.0, VMM: no
llama_model_loader: loaded meta data with 24 key-value pairs and 291 tensors from D:\ollama-models\blobs\sha256-e8a35b5937a5e6d5c35d1f2a15f161e07eefe5e5bb0a3cdd42998ee79b057730 (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = llama
llama_model_loader: - kv   1:                               general.name str              = mistralai
llama_model_loader: - kv   2:                       llama.context_length u32              = 32768
llama_model_loader: - kv   3:                     llama.embedding_length u32              = 4096
llama_model_loader: - kv   4:                          llama.block_count u32              = 32
llama_model_loader: - kv   5:                  llama.feed_forward_length u32              = 14336
llama_model_loader: - kv   6:                 llama.rope.dimension_count u32              = 128
llama_model_loader: - kv   7:                 llama.attention.head_count u32              = 32
llama_model_loader: - kv   8:              llama.attention.head_count_kv u32              = 8
llama_model_loader: - kv   9:     llama.attention.layer_norm_rms_epsilon f32              = 0.000010
llama_model_loader: - kv  10:                       llama.rope.freq_base f32              = 1000000.000000
llama_model_loader: - kv  11:                          general.file_type u32              = 2
llama_model_loader: - kv  12:                       tokenizer.ggml.model str              = llama
llama_model_loader: - kv  13:                      tokenizer.ggml.tokens arr[str,32000]   = ["<unk>", "<s>", "</s>", "<0x00>", "<...
llama_model_loader: - kv  14:                      tokenizer.ggml.scores arr[f32,32000]   = [0.000000, 0.000000, 0.000000, 0.0000...
llama_model_loader: - kv  15:                  tokenizer.ggml.token_type arr[i32,32000]   = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ...
llama_model_loader: - kv  16:                      tokenizer.ggml.merges arr[str,58980]   = ["▁ t", "i n", "e r", "▁ a", "h e...
llama_model_loader: - kv  17:                tokenizer.ggml.bos_token_id u32              = 1
llama_model_loader: - kv  18:                tokenizer.ggml.eos_token_id u32              = 2
llama_model_loader: - kv  19:            tokenizer.ggml.unknown_token_id u32              = 0
llama_model_loader: - kv  20:               tokenizer.ggml.add_bos_token bool             = true
llama_model_loader: - kv  21:               tokenizer.ggml.add_eos_token bool             = false
llama_model_loader: - kv  22:                    tokenizer.chat_template str              = {{ bos_token }}{% for message in mess...
llama_model_loader: - kv  23:               general.quantization_version u32              = 2
llama_model_loader: - type  f32:   65 tensors
llama_model_loader: - type q4_0:  225 tensors
llama_model_loader: - type q6_K:    1 tensors
llm_load_vocab: special tokens definition check successful ( 259/32000 ).
llm_load_print_meta: format           = GGUF V3 (latest)
llm_load_print_meta: arch             = llama
llm_load_print_meta: vocab type       = SPM
llm_load_print_meta: n_vocab          = 32000
llm_load_print_meta: n_merges         = 0
llm_load_print_meta: n_ctx_train      = 32768
llm_load_print_meta: n_embd           = 4096
llm_load_print_meta: n_head           = 32
llm_load_print_meta: n_head_kv        = 8
llm_load_print_meta: n_layer          = 32
llm_load_print_meta: n_rot            = 128
llm_load_print_meta: n_embd_head_k    = 128
llm_load_print_meta: n_embd_head_v    = 128
llm_load_print_meta: n_gqa            = 4
llm_load_print_meta: n_embd_k_gqa     = 1024
llm_load_print_meta: n_embd_v_gqa     = 1024
llm_load_print_meta: f_norm_eps       = 0.0e+00
llm_load_print_meta: f_norm_rms_eps   = 1.0e-05
llm_load_print_meta: f_clamp_kqv      = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: n_ff             = 14336
llm_load_print_meta: n_expert         = 0
llm_load_print_meta: n_expert_used    = 0
llm_load_print_meta: pooling type     = 0
llm_load_print_meta: rope type        = 0
llm_load_print_meta: rope scaling     = linear
llm_load_print_meta: freq_base_train  = 1000000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: n_yarn_orig_ctx  = 32768
llm_load_print_meta: rope_finetuned   = unknown
llm_load_print_meta: model type       = 7B
llm_load_print_meta: model ftype      = Q4_0
llm_load_print_meta: model params     = 7.24 B
llm_load_print_meta: model size       = 3.83 GiB (4.54 BPW) 
llm_load_print_meta: general.name     = mistralai
llm_load_print_meta: BOS token        = 1 '<s>'
llm_load_print_meta: EOS token        = 2 '</s>'
llm_load_print_meta: UNK token        = 0 '<unk>'
llm_load_print_meta: LF token         = 13 '<0x0A>'
llm_load_tensors: ggml ctx size =    0.22 MiB
llm_load_tensors: offloading 32 repeating layers to GPU
llm_load_tensors: offloading non-repeating layers to GPU
llm_load_tensors: offloaded 33/33 layers to GPU
llm_load_tensors:      ROCm0 buffer size =  3847.55 MiB
llm_load_tensors:        CPU buffer size =    70.31 MiB
..................................................................................................
llama_new_context_with_model: n_ctx      = 2048
llama_new_context_with_model: freq_base  = 1000000.0
llama_new_context_with_model: freq_scale = 1
llama_kv_cache_init:      ROCm0 KV buffer size =   256.00 MiB
llama_new_context_with_model: KV self size  =  256.00 MiB, K (f16):  128.00 MiB, V (f16):  128.00 MiB
llama_new_context_with_model:  ROCm_Host input buffer size   =    13.02 MiB
llama_new_context_with_model:      ROCm0 compute buffer size =   164.00 MiB
llama_new_context_with_model:  ROCm_Host compute buffer size =     8.00 MiB
llama_new_context_with_model: graph splits (measure): 2
time=2024-03-09T10:13:59.231+08:00 level=INFO source=dyn_ext_server.go:162 msg="Starting llama main loop"
[GIN] 2024/03/09 - 10:13:59 | 200 |    4.8541149s |       127.0.0.1 | POST     "/api/chat"
[GIN] 2024/03/09 - 10:14:04 | 200 |    909.5642ms |       127.0.0.1 | POST     "/api/chat"
[GIN] 2024/03/09 - 10:14:40 | 200 |            0s |       127.0.0.1 | HEAD     "/"
[GIN] 2024/03/09 - 10:14:40 | 200 |      1.0474ms |       127.0.0.1 | POST     "/api/show"
[GIN] 2024/03/09 - 10:14:40 | 200 |       1.031ms |       127.0.0.1 | POST     "/api/show"
time=2024-03-09T10:14:40.272+08:00 level=INFO source=routes.go:76 msg="changing loaded model"
time=2024-03-09T10:14:40.952+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-03-09T10:14:40.952+08:00 level=INFO source=amd_windows.go:40 msg="AMD Driver: 50732000"
time=2024-03-09T10:14:40.953+08:00 level=INFO source=amd_windows.go:69 msg="detected 1 hip devices"
time=2024-03-09T10:14:40.953+08:00 level=INFO source=amd_windows.go:87 msg="[0] Name: AMD Radeon RX 7900 XTX"
time=2024-03-09T10:14:40.953+08:00 level=INFO source=amd_windows.go:90 msg="[0] GcnArchName: gfx1100"
time=2024-03-09T10:14:40.953+08:00 level=INFO source=amd_windows.go:106 msg="amdgpu [0] gfx1100 is supported"
time=2024-03-09T10:14:40.953+08:00 level=INFO source=amd_windows.go:117 msg="[0] Total Mem: 24946528256"
time=2024-03-09T10:14:40.953+08:00 level=INFO source=amd_windows.go:118 msg="[0] Free Mem:  25753026560"
time=2024-03-09T10:14:40.953+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-03-09T10:14:40.954+08:00 level=INFO source=amd_windows.go:40 msg="AMD Driver: 50732000"
time=2024-03-09T10:14:40.954+08:00 level=INFO source=amd_windows.go:69 msg="detected 1 hip devices"
time=2024-03-09T10:14:40.955+08:00 level=INFO source=amd_windows.go:87 msg="[0] Name: AMD Radeon RX 7900 XTX"
time=2024-03-09T10:14:40.955+08:00 level=INFO source=amd_windows.go:90 msg="[0] GcnArchName: gfx1100"
time=2024-03-09T10:14:40.955+08:00 level=INFO source=amd_windows.go:106 msg="amdgpu [0] gfx1100 is supported"
time=2024-03-09T10:14:40.955+08:00 level=INFO source=amd_windows.go:117 msg="[0] Total Mem: 24946528256"
time=2024-03-09T10:14:40.955+08:00 level=INFO source=amd_windows.go:118 msg="[0] Free Mem:  25753026560"
time=2024-03-09T10:14:40.955+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-03-09T10:14:40.955+08:00 level=INFO source=dyn_ext_server.go:90 msg="Loading Dynamic llm server: C:\\Users\\user\\AppData\\Local\\Temp\\ollama\\payloads\\rocm_v5.7\\ext_server.dll"
time=2024-03-09T10:14:40.955+08:00 level=INFO source=dyn_ext_server.go:150 msg="Initializing llama server"
llama_model_loader: loaded meta data with 26 key-value pairs and 995 tensors from D:\ollama-models\blobs\sha256-e9e56e8bb5f0fcd4860675e6837a8f6a94e659f5fa7dce6a1076279336320f2b (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = llama
llama_model_loader: - kv   1:                               general.name str              = mistralai
llama_model_loader: - kv   2:                       llama.context_length u32              = 32768
llama_model_loader: - kv   3:                     llama.embedding_length u32              = 4096
llama_model_loader: - kv   4:                          llama.block_count u32              = 32
llama_model_loader: - kv   5:                  llama.feed_forward_length u32              = 14336
llama_model_loader: - kv   6:                 llama.rope.dimension_count u32              = 128
llama_model_loader: - kv   7:                 llama.attention.head_count u32              = 32
llama_model_loader: - kv   8:              llama.attention.head_count_kv u32              = 8
llama_model_loader: - kv   9:                         llama.expert_count u32              = 8
llama_model_loader: - kv  10:                    llama.expert_used_count u32              = 2
llama_model_loader: - kv  11:     llama.attention.layer_norm_rms_epsilon f32              = 0.000010
llama_model_loader: - kv  12:                       llama.rope.freq_base f32              = 1000000.000000
llama_model_loader: - kv  13:                          general.file_type u32              = 2
llama_model_loader: - kv  14:                       tokenizer.ggml.model str              = llama
llama_model_loader: - kv  15:                      tokenizer.ggml.tokens arr[str,32000]   = ["<unk>", "<s>", "</s>", "<0x00>", "<...
llama_model_loader: - kv  16:                      tokenizer.ggml.scores arr[f32,32000]   = [0.000000, 0.000000, 0.000000, 0.0000...
llama_model_loader: - kv  17:                  tokenizer.ggml.token_type arr[i32,32000]   = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ...
llama_model_loader: - kv  18:                      tokenizer.ggml.merges arr[str,58980]   = ["▁ t", "i n", "e r", "▁ a", "h e...
llama_model_loader: - kv  19:                tokenizer.ggml.bos_token_id u32              = 1
llama_model_loader: - kv  20:                tokenizer.ggml.eos_token_id u32              = 2
llama_model_loader: - kv  21:            tokenizer.ggml.unknown_token_id u32              = 0
llama_model_loader: - kv  22:               tokenizer.ggml.add_bos_token bool             = true
llama_model_loader: - kv  23:               tokenizer.ggml.add_eos_token bool             = false
llama_model_loader: - kv  24:                    tokenizer.chat_template str              = {{ bos_token }}{% for message in mess...
llama_model_loader: - kv  25:               general.quantization_version u32              = 2
llama_model_loader: - type  f32:   65 tensors
llama_model_loader: - type  f16:   32 tensors
llama_model_loader: - type q4_0:  833 tensors
llama_model_loader: - type q8_0:   64 tensors
llama_model_loader: - type q6_K:    1 tensors
llm_load_vocab: special tokens definition check successful ( 259/32000 ).
llm_load_print_meta: format           = GGUF V3 (latest)
llm_load_print_meta: arch             = llama
llm_load_print_meta: vocab type       = SPM
llm_load_print_meta: n_vocab          = 32000
llm_load_print_meta: n_merges         = 0
llm_load_print_meta: n_ctx_train      = 32768
llm_load_print_meta: n_embd           = 4096
llm_load_print_meta: n_head           = 32
llm_load_print_meta: n_head_kv        = 8
llm_load_print_meta: n_layer          = 32
llm_load_print_meta: n_rot            = 128
llm_load_print_meta: n_embd_head_k    = 128
llm_load_print_meta: n_embd_head_v    = 128
llm_load_print_meta: n_gqa            = 4
llm_load_print_meta: n_embd_k_gqa     = 1024
llm_load_print_meta: n_embd_v_gqa     = 1024
llm_load_print_meta: f_norm_eps       = 0.0e+00
llm_load_print_meta: f_norm_rms_eps   = 1.0e-05
llm_load_print_meta: f_clamp_kqv      = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: n_ff             = 14336
llm_load_print_meta: n_expert         = 8
llm_load_print_meta: n_expert_used    = 2
llm_load_print_meta: pooling type     = 0
llm_load_print_meta: rope type        = 0
llm_load_print_meta: rope scaling     = linear
llm_load_print_meta: freq_base_train  = 1000000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: n_yarn_orig_ctx  = 32768
llm_load_print_meta: rope_finetuned   = unknown
llm_load_print_meta: model type       = 7B
llm_load_print_meta: model ftype      = Q4_0
llm_load_print_meta: model params     = 46.70 B
llm_load_print_meta: model size       = 24.62 GiB (4.53 BPW) 
llm_load_print_meta: general.name     = mistralai
llm_load_print_meta: BOS token        = 1 '<s>'
llm_load_print_meta: EOS token        = 2 '</s>'
llm_load_print_meta: UNK token        = 0 '<unk>'
llm_load_print_meta: LF token         = 13 '<0x0A>'
llm_load_tensors: ggml ctx size =    0.76 MiB
Exception 0xc0000005 0x1 0x10 0x7ffa12f9a390
PC=0x7ffa12f9a390
signal arrived during external code execution

runtime.cgocall(0x7ff6d5b764e0, 0xc001006760)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/cgocall.go:157 +0x3e fp=0xc001006738 sp=0xc001006700 pc=0x7ff6d50a953e
github.com/jmorganca/ollama/llm._Cfunc_dyn_llama_server_init({0x7ffa14a90000, 0x7ffa14baf250, 0x7ffa14bafd70, 0x7ffa14bafe20, 0x7ffa14bb0240, 0x7ffa14bb0470, 0x7ffa14bb14b0, 0x7ffa14bb1490, 0x7ffa14bb16e0, 0x7ffa14bb1d00, ...}, ...)
	_cgo_gotypes.go:282 +0x4d fp=0xc001006760 sp=0xc001006738 pc=0x7ff6d599b0ed
github.com/jmorganca/ollama/llm.newDynExtServer.func7(0xc00009c0f0, 0xc000be80d8)
	C:/Users/danie/code/ollama/llm/dyn_ext_server.go:154 +0x112 fp=0xc0010068a0 sp=0xc001006760 pc=0x7ff6d599c8b2
github.com/jmorganca/ollama/llm.newDynExtServer({0xc00050e180, 0x4b}, {0xc0004d8360, _}, {_, _, _}, {0x0, 0x0, 0x0}, ...)
	C:/Users/danie/code/ollama/llm/dyn_ext_server.go:154 +0xb50 fp=0xc001006ae8 sp=0xc0010068a0 pc=0x7ff6d599c4f0
github.com/jmorganca/ollama/llm.newLlmServer({{_, _, _}, {_, _}, {_, _}}, {_, _}, {0x0, ...}, ...)
	C:/Users/danie/code/ollama/llm/llm.go:158 +0x4c5 fp=0xc001006ca8 sp=0xc001006ae8 pc=0x7ff6d5998905
github.com/jmorganca/ollama/llm.New({0xc0004d8360, 0x5e}, {0x0, 0x0, 0x0}, {0x0, _, _}, {{0x0, 0x800, ...}, ...})
	C:/Users/danie/code/ollama/llm/llm.go:123 +0x76e fp=0xc001006f18 sp=0xc001006ca8 pc=0x7ff6d59981ce
github.com/jmorganca/ollama/server.load(0xc0000dc180?, 0xc0000dc180, {{0x0, 0x800, 0x200, 0x1, 0xffffffffffffffff, 0x0, 0x0, 0x1, ...}, ...}, ...)
	C:/Users/danie/code/ollama/server/routes.go:83 +0x325 fp=0xc001007068 sp=0xc001006f18 pc=0x7ff6d5b496e5
github.com/jmorganca/ollama/server.ChatHandler(0xc0000a2000)
	C:/Users/danie/code/ollama/server/routes.go:1173 +0xa37 fp=0xc001007770 sp=0xc001007068 pc=0x7ff6d5b54e57
github.com/gin-gonic/gin.(*Context).Next(...)
	C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174
github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func1(0xc0000a2000)
	C:/Users/danie/code/ollama/server/routes.go:943 +0x68 fp=0xc0010077a8 sp=0xc001007770 pc=0x7ff6d5b53628
github.com/gin-gonic/gin.(*Context).Next(...)
	C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174
github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1(0xc0000a2000)
	C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/recovery.go:102 +0x7a fp=0xc0010077f8 sp=0xc0010077a8 pc=0x7ff6d5b29eba
github.com/gin-gonic/gin.(*Context).Next(...)
	C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174
github.com/gin-gonic/gin.LoggerWithConfig.func1(0xc0000a2000)
	C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/logger.go:240 +0xdd fp=0xc0010079a8 sp=0xc0010077f8 pc=0x7ff6d5b28ffd
github.com/gin-gonic/gin.(*Context).Next(...)
	C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174
github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc000168ea0, 0xc0000a2000)
	C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/gin.go:620 +0x66e fp=0xc001007b28 sp=0xc0010079a8 pc=0x7ff6d5b2852e
github.com/gin-gonic/gin.(*Engine).ServeHTTP(0xc000168ea0, {0x7ff6d9539dd0, 0xc00009e0e0}, 0xc0000b45a0)
	C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/gin.go:576 +0x1b2 fp=0xc001007b60 sp=0xc001007b28 pc=0x7ff6d5b27cf2
net/http.serverHandler.ServeHTTP({0x7ff6d9537c48?}, {0x7ff6d9539dd0?, 0xc00009e0e0?}, 0x6?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:3137 +0x8e fp=0xc001007b90 sp=0xc001007b60 pc=0x7ff6d53aee2e
net/http.(*conn).serve(0xc0000e66c0, {0x7ff6d953c1c8, 0xc0001b8660})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:2039 +0x5e8 fp=0xc001007fb8 sp=0xc001007b90 pc=0x7ff6d53aa1e8
net/http.(*Server).Serve.gowrap3()
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:3285 +0x28 fp=0xc001007fe0 sp=0xc001007fb8 pc=0x7ff6d53af648
runtime.goexit({})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc001007fe8 sp=0xc001007fe0 pc=0x7ff6d5112901
created by net/http.(*Server).Serve in goroutine 1
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:3285 +0x4b4

goroutine 1 gp=0xc00007e000 m=nil [IO wait]:
runtime.gopark(0xc000600008?, 0x7ff6d5bfdb00?, 0x20?, 0x2a?, 0xc0005b2a50?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc0006a76f8 sp=0xc0006a76d8 pc=0x7ff6d50e1aae
runtime.netpollblock(0x1b4?, 0xd50a9046?, 0xf6?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/netpoll.go:573 +0xf7 fp=0xc0006a7730 sp=0xc0006a76f8 pc=0x7ff6d50d9357
internal/poll.runtime_pollWait(0x20177cadba0, 0x72)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/netpoll.go:345 +0x85 fp=0xc0006a7750 sp=0xc0006a7730 pc=0x7ff6d510c3a5
internal/poll.(*pollDesc).wait(0x7ff6d50c07b6?, 0x7ff6d9c800a0?, 0x0)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0006a7778 sp=0xc0006a7750 pc=0x7ff6d51b0647
internal/poll.execIO(0xc0005b2a20, 0xc0006a7818)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_windows.go:175 +0xe6 fp=0xc0006a77e8 sp=0xc0006a7778 pc=0x7ff6d51b1b26
internal/poll.(*FD).acceptOne(0xc0005b2a08, 0x268, {0xc00000c0f0?, 0x0?, 0x0?}, 0xc000600008?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_windows.go:944 +0x67 fp=0xc0006a7848 sp=0xc0006a77e8 pc=0x7ff6d51b61e7
internal/poll.(*FD).Accept(0xc0005b2a08, 0xc0006a79f8)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_windows.go:978 +0x1bc fp=0xc0006a7900 sp=0xc0006a7848 pc=0x7ff6d51b651c
net.(*netFD).accept(0xc0005b2a08)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/fd_windows.go:178 +0x54 fp=0xc0006a7a18 sp=0xc0006a7900 pc=0x7ff6d52476f4
net.(*TCPListener).accept(0xc0004405c0)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/tcpsock_posix.go:159 +0x1e fp=0xc0006a7a40 sp=0xc0006a7a18 pc=0x7ff6d525cb9e
net.(*TCPListener).Accept(0xc0004405c0)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/tcpsock.go:327 +0x30 fp=0xc0006a7a70 sp=0xc0006a7a40 pc=0x7ff6d525bc90
net/http.(*onceCloseListener).Accept(0xc0000e66c0?)
	<autogenerated>:1 +0x24 fp=0xc0006a7a88 sp=0xc0006a7a70 pc=0x7ff6d53d1804
net/http.(*Server).Serve(0xc0000c04b0, {0x7ff6d9539bc0, 0xc0004405c0})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:3255 +0x33e fp=0xc0006a7bb8 sp=0xc0006a7a88 pc=0x7ff6d53af25e
github.com/jmorganca/ollama/server.Serve({0x7ff6d9539bc0, 0xc0004405c0})
	C:/Users/danie/code/ollama/server/routes.go:1046 +0x468 fp=0xc0006a7cc0 sp=0xc0006a7bb8 pc=0x7ff6d5b53ae8
github.com/jmorganca/ollama/cmd.RunServer(0xc0001e6c00?, {0x7ff6d9c7e340?, 0x4?, 0x7ff6d5db3a7e?})
	C:/Users/danie/code/ollama/cmd/cmd.go:787 +0x1b9 fp=0xc0006a7d58 sp=0xc0006a7cc0 pc=0x7ff6d5b6d9b9
github.com/spf13/cobra.(*Command).execute(0xc0000b8f08, {0x7ff6d9c7e340, 0x0, 0x0})
	C:/Users/danie/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:940 +0x882 fp=0xc0006a7e78 sp=0xc0006a7d58 pc=0x7ff6d544afe2
github.com/spf13/cobra.(*Command).ExecuteC(0xc0000b8308)
	C:/Users/danie/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3a5 fp=0xc0006a7f30 sp=0xc0006a7e78 pc=0x7ff6d544b825
github.com/spf13/cobra.(*Command).Execute(...)
	C:/Users/danie/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:992
github.com/spf13/cobra.(*Command).ExecuteContext(...)
	C:/Users/danie/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:985
main.main()
	C:/Users/danie/code/ollama/main.go:11 +0x4d fp=0xc0006a7f50 sp=0xc0006a7f30 pc=0x7ff6d5b7624d
runtime.main()
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:271 +0x28b fp=0xc0006a7fe0 sp=0xc0006a7f50 pc=0x7ff6d50e16ab
runtime.goexit({})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0006a7fe8 sp=0xc0006a7fe0 pc=0x7ff6d5112901

goroutine 2 gp=0xc00007e700 m=nil [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000081fa8 sp=0xc000081f88 pc=0x7ff6d50e1aae
runtime.goparkunlock(...)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:408
runtime.forcegchelper()
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:326 +0xb8 fp=0xc000081fe0 sp=0xc000081fa8 pc=0x7ff6d50e1938
runtime.goexit({})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000081fe8 sp=0xc000081fe0 pc=0x7ff6d5112901
created by runtime.init.6 in goroutine 1
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:314 +0x1a

goroutine 3 gp=0xc00007ea80 m=nil [GC sweep wait]:
runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000083f80 sp=0xc000083f60 pc=0x7ff6d50e1aae
runtime.goparkunlock(...)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:408
runtime.bgsweep(0xc00008e000)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgcsweep.go:318 +0xdf fp=0xc000083fc8 sp=0xc000083f80 pc=0x7ff6d50cbb5f
runtime.gcenable.gowrap1()
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:203 +0x25 fp=0xc000083fe0 sp=0xc000083fc8 pc=0x7ff6d50c0405
runtime.goexit({})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000083fe8 sp=0xc000083fe0 pc=0x7ff6d5112901
created by runtime.gcenable in goroutine 1
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:203 +0x66

goroutine 4 gp=0xc00007ec40 m=nil [GC scavenge wait]:
runtime.gopark(0x991e0c?, 0x86fc3e?, 0x0?, 0x0?, 0x0?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000095f78 sp=0xc000095f58 pc=0x7ff6d50e1aae
runtime.goparkunlock(...)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:408
runtime.(*scavengerState).park(0x7ff6d9bf3020)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc000095fa8 sp=0xc000095f78 pc=0x7ff6d50c94e9
runtime.bgscavenge(0xc00008e000)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgcscavenge.go:658 +0x59 fp=0xc000095fc8 sp=0xc000095fa8 pc=0x7ff6d50c9a99
runtime.gcenable.gowrap2()
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:204 +0x25 fp=0xc000095fe0 sp=0xc000095fc8 pc=0x7ff6d50c03a5
runtime.goexit({})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000095fe8 sp=0xc000095fe0 pc=0x7ff6d5112901
created by runtime.gcenable in goroutine 1
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:204 +0xa5

goroutine 18 gp=0xc000104380 m=nil [finalizer wait]:
runtime.gopark(0xc000085e48?, 0x7ff6d50b37c5?, 0xa8?, 0x1?, 0xc00007e000?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000085e20 sp=0xc000085e00 pc=0x7ff6d50e1aae
runtime.runfinq()
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mfinal.go:194 +0x107 fp=0xc000085fe0 sp=0xc000085e20 pc=0x7ff6d50bf487
runtime.goexit({})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000085fe8 sp=0xc000085fe0 pc=0x7ff6d5112901
created by runtime.createfing in goroutine 1
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mfinal.go:164 +0x3d

goroutine 19 gp=0xc000105880 m=nil [GC worker (idle)]:
runtime.gopark(0x7ff6d9c800a0?, 0x1?, 0x0?, 0x0?, 0x0?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000091f50 sp=0xc000091f30 pc=0x7ff6d50e1aae
runtime.gcBgMarkWorker()
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000091fe0 sp=0xc000091f50 pc=0x7ff6d50c2545
runtime.goexit({})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000091fe8 sp=0xc000091fe0 pc=0x7ff6d5112901
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c

goroutine 34 gp=0xc000480000 m=nil [GC worker (idle)]:
runtime.gopark(0x3135b6217c2c?, 0x1?, 0x0?, 0x0?, 0x0?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000487f50 sp=0xc000487f30 pc=0x7ff6d50e1aae
runtime.gcBgMarkWorker()
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000487fe0 sp=0xc000487f50 pc=0x7ff6d50c2545
runtime.goexit({})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000487fe8 sp=0xc000487fe0 pc=0x7ff6d5112901
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c

goroutine 5 gp=0xc00007efc0 m=nil [GC worker (idle)]:
runtime.gopark(0x7ff6d9c800a0?, 0x3?, 0x0?, 0x0?, 0x0?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000097f50 sp=0xc000097f30 pc=0x7ff6d50e1aae
runtime.gcBgMarkWorker()
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000097fe0 sp=0xc000097f50 pc=0x7ff6d50c2545
runtime.goexit({})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000097fe8 sp=0xc000097fe0 pc=0x7ff6d5112901
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c

goroutine 35 gp=0xc0004801c0 m=nil [GC worker (idle)]:
runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000489f50 sp=0xc000489f30 pc=0x7ff6d50e1aae
runtime.gcBgMarkWorker()
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000489fe0 sp=0xc000489f50 pc=0x7ff6d50c2545
runtime.goexit({})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000489fe8 sp=0xc000489fe0 pc=0x7ff6d5112901
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c

goroutine 6 gp=0xc00007f180 m=nil [GC worker (idle)]:
runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000483f50 sp=0xc000483f30 pc=0x7ff6d50e1aae
runtime.gcBgMarkWorker()
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000483fe0 sp=0xc000483f50 pc=0x7ff6d50c2545
runtime.goexit({})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000483fe8 sp=0xc000483fe0 pc=0x7ff6d5112901
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c

goroutine 36 gp=0xc000480380 m=nil [GC worker (idle)]:
runtime.gopark(0x3135b6217c2c?, 0x1?, 0x0?, 0x0?, 0x0?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000491f50 sp=0xc000491f30 pc=0x7ff6d50e1aae
runtime.gcBgMarkWorker()
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000491fe0 sp=0xc000491f50 pc=0x7ff6d50c2545
runtime.goexit({})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000491fe8 sp=0xc000491fe0 pc=0x7ff6d5112901
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c

goroutine 7 gp=0xc00007f340 m=nil [GC worker (idle)]:
runtime.gopark(0x3135b6217c2c?, 0x3?, 0x1c?, 0xfa?, 0x0?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000485f50 sp=0xc000485f30 pc=0x7ff6d50e1aae
runtime.gcBgMarkWorker()
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000485fe0 sp=0xc000485f50 pc=0x7ff6d50c2545
runtime.goexit({})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000485fe8 sp=0xc000485fe0 pc=0x7ff6d5112901
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c

goroutine 37 gp=0xc000480540 m=nil [GC worker (idle)]:
runtime.gopark(0x7ff6d9c800a0?, 0x3?, 0x40?, 0xbd?, 0x0?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000493f50 sp=0xc000493f30 pc=0x7ff6d50e1aae
runtime.gcBgMarkWorker()
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000493fe0 sp=0xc000493f50 pc=0x7ff6d50c2545
runtime.goexit({})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000493fe8 sp=0xc000493fe0 pc=0x7ff6d5112901
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c

goroutine 8 gp=0xc00007f500 m=nil [GC worker (idle)]:
runtime.gopark(0x3135b6217c2c?, 0x1?, 0x0?, 0x0?, 0x0?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc00048df50 sp=0xc00048df30 pc=0x7ff6d50e1aae
runtime.gcBgMarkWorker()
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc00048dfe0 sp=0xc00048df50 pc=0x7ff6d50c2545
runtime.goexit({})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00048dfe8 sp=0xc00048dfe0 pc=0x7ff6d5112901
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c

goroutine 38 gp=0xc000480700 m=nil [GC worker (idle)]:
runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000499f50 sp=0xc000499f30 pc=0x7ff6d50e1aae
runtime.gcBgMarkWorker()
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000499fe0 sp=0xc000499f50 pc=0x7ff6d50c2545
runtime.goexit({})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000499fe8 sp=0xc000499fe0 pc=0x7ff6d5112901
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c

goroutine 9 gp=0xc00007f6c0 m=nil [GC worker (idle)]:
runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc00048ff50 sp=0xc00048ff30 pc=0x7ff6d50e1aae
runtime.gcBgMarkWorker()
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc00048ffe0 sp=0xc00048ff50 pc=0x7ff6d50c2545
runtime.goexit({})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00048ffe8 sp=0xc00048ffe0 pc=0x7ff6d5112901
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c

goroutine 39 gp=0xc0004808c0 m=nil [GC worker (idle)]:
runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc00049bf50 sp=0xc00049bf30 pc=0x7ff6d50e1aae
runtime.gcBgMarkWorker()
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc00049bfe0 sp=0xc00049bf50 pc=0x7ff6d50c2545
runtime.goexit({})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00049bfe8 sp=0xc00049bfe0 pc=0x7ff6d5112901
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c

goroutine 10 gp=0xc00007f880 m=nil [GC worker (idle)]:
runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000495f50 sp=0xc000495f30 pc=0x7ff6d50e1aae
runtime.gcBgMarkWorker()
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000495fe0 sp=0xc000495f50 pc=0x7ff6d50c2545
runtime.goexit({})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000495fe8 sp=0xc000495fe0 pc=0x7ff6d5112901
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c

goroutine 40 gp=0xc000480a80 m=nil [GC worker (idle)]:
runtime.gopark(0xc000059c50?, 0x1?, 0x2c?, 0x7c?, 0x0?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc0004a1f50 sp=0xc0004a1f30 pc=0x7ff6d50e1aae
runtime.gcBgMarkWorker()
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc0004a1fe0 sp=0xc0004a1f50 pc=0x7ff6d50c2545
runtime.goexit({})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0004a1fe8 sp=0xc0004a1fe0 pc=0x7ff6d5112901
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c

goroutine 11 gp=0xc00007fa40 m=nil [GC worker (idle)]:
runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000497f50 sp=0xc000497f30 pc=0x7ff6d50e1aae
runtime.gcBgMarkWorker()
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000497fe0 sp=0xc000497f50 pc=0x7ff6d50c2545
runtime.goexit({})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000497fe8 sp=0xc000497fe0 pc=0x7ff6d5112901
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c

goroutine 41 gp=0xc000480c40 m=nil [GC worker (idle)]:
runtime.gopark(0x3135b629396c?, 0x3?, 0x0?, 0x0?, 0x0?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc0004a3f50 sp=0xc0004a3f30 pc=0x7ff6d50e1aae
runtime.gcBgMarkWorker()
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc0004a3fe0 sp=0xc0004a3f50 pc=0x7ff6d50c2545
runtime.goexit({})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0004a3fe8 sp=0xc0004a3fe0 pc=0x7ff6d5112901
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c

goroutine 12 gp=0xc000481180 m=4 mp=0xc00008b808 [syscall]:
runtime.notetsleepg(0x7ff6d9c7eee0, 0xffffffffffffffff)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/lock_sema.go:296 +0x31 fp=0xc00049ffa0 sp=0xc00049ff68 pc=0x7ff6d50b1d91
os/signal.signal_recv()
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/sigqueue.go:152 +0x29 fp=0xc00049ffc0 sp=0xc00049ffa0 pc=0x7ff6d510e5e9
os/signal.loop()
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/os/signal/signal_unix.go:23 +0x13 fp=0xc00049ffe0 sp=0xc00049ffc0 pc=0x7ff6d53d3bb3
runtime.goexit({})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00049ffe8 sp=0xc00049ffe0 pc=0x7ff6d5112901
created by os/signal.Notify.func1.1 in goroutine 1
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/os/signal/signal.go:151 +0x1f

goroutine 13 gp=0xc000481340 m=nil [chan receive]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000093f18 sp=0xc000093ef8 pc=0x7ff6d50e1aae
runtime.chanrecv(0xc0001eafc0, 0x0, 0x1)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/chan.go:583 +0x3cd fp=0xc000093f90 sp=0xc000093f18 pc=0x7ff6d50abbcd
runtime.chanrecv1(0x0?, 0x0?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/chan.go:442 +0x12 fp=0xc000093fb8 sp=0xc000093f90 pc=0x7ff6d50ab7d2
github.com/jmorganca/ollama/server.Serve.func2()
	C:/Users/danie/code/ollama/server/routes.go:1028 +0x25 fp=0xc000093fe0 sp=0xc000093fb8 pc=0x7ff6d5b53b85
runtime.goexit({})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000093fe8 sp=0xc000093fe0 pc=0x7ff6d5112901
created by github.com/jmorganca/ollama/server.Serve in goroutine 1
	C:/Users/danie/code/ollama/server/routes.go:1027 +0x3f6

goroutine 83 gp=0xc0005048c0 m=nil [IO wait]:
runtime.gopark(0x0?, 0xc000698020?, 0xd0?, 0x80?, 0xc000698050?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc0004cbd28 sp=0xc0004cbd08 pc=0x7ff6d50e1aae
runtime.netpollblock(0x280?, 0xd50a9046?, 0xf6?)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/netpoll.go:573 +0xf7 fp=0xc0004cbd60 sp=0xc0004cbd28 pc=0x7ff6d50d9357
internal/poll.runtime_pollWait(0x20177cadaa8, 0x72)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/netpoll.go:345 +0x85 fp=0xc0004cbd80 sp=0xc0004cbd60 pc=0x7ff6d510c3a5
internal/poll.(*pollDesc).wait(0x10?, 0x10?, 0x0)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0004cbda8 sp=0xc0004cbd80 pc=0x7ff6d51b0647
internal/poll.execIO(0xc000698020, 0x7ff6d9419910)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_windows.go:175 +0xe6 fp=0xc0004cbe18 sp=0xc0004cbda8 pc=0x7ff6d51b1b26
internal/poll.(*FD).Read(0xc000698008, {0xc000f0c641, 0x1, 0x1})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_windows.go:436 +0x2b1 fp=0xc0004cbec0 sp=0xc0004cbe18 pc=0x7ff6d51b27d1
net.(*netFD).Read(0xc000698008, {0xc000f0c641?, 0xc0004cbf48?, 0x7ff6d510e330?})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/fd_posix.go:55 +0x25 fp=0xc0004cbf08 sp=0xc0004cbec0 pc=0x7ff6d5245805
net.(*conn).Read(0xc000110040, {0xc000f0c641?, 0xc0004cbeb8?, 0x7ff6d9c7e340?})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/net.go:179 +0x45 fp=0xc0004cbf50 sp=0xc0004cbf08 pc=0x7ff6d5254885
net.(*TCPConn).Read(0x7ff6d5bd7aa0?, {0xc000f0c641?, 0xc0000983c0?, 0x7ff6d54a25a0?})
	<autogenerated>:1 +0x25 fp=0xc0004cbf80 sp=0xc0004cbf50 pc=0x7ff6d52648a5
net/http.(*connReader).backgroundRead(0xc000f0c630)
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:681 +0x37 fp=0xc0004cbfc8 sp=0xc0004cbf80 pc=0x7ff6d53a4157
net/http.(*connReader).startBackgroundRead.gowrap2()
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:677 +0x25 fp=0xc0004cbfe0 sp=0xc0004cbfc8 pc=0x7ff6d53a4085
runtime.goexit({})
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0004cbfe8 sp=0xc0004cbfe0 pc=0x7ff6d5112901
created by net/http.(*connReader).startBackgroundRead in goroutine 21
	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:677 +0xba
rax     0x0
rbx     0x2012414aa68
rcx     0x2012414aa18
rdx     0xffffffdd00000000
rdi     0x2012414aa68
rsi     0x2012414aa18
rbp     0x0
rsp     0x8f23ffd7b0
r8      0x2012414aa68
r9      0x0
r10     0x96
r11     0x246
r12     0x0
r13     0x7ffa12a30000
r14     0x2014e56d9c0
r15     0x8f23ffd970
rip     0x7ffa12f9a390
rflags  0x10246
cs      0x33
fs      0x53
gs      0x2b
{"function":"initialize","level":"INFO","line":688,"msg":"initializing slots","n_slots":1,"tid":"19848","timestamp":1709950439}
{"function":"initialize","id_slot":0,"level":"INFO","line":699,"msg":"new slot","n_ctx_slot":2048,"tid":"19848","timestamp":1709950439}
{"function":"update_slots","level":"INFO","line":1593,"msg":"all slots are idle","tid":"5172","timestamp":1709950439}
{"function":"launch_slot_with_data","id_slot":0,"id_task":0,"level":"INFO","line":973,"msg":"slot is processing task","tid":"5172","timestamp":1709950443}
{"function":"update_slots","id_slot":0,"id_task":0,"level":"INFO","line":1845,"msg":"kv cache rm [p0, end)","p0":0,"tid":"5172","timestamp":1709950443}
{"function":"print_timings","id_slot":0,"id_task":0,"level":"INFO","line":311,"msg":"prompt eval time     =      63.01 ms /    11 tokens (    5.73 ms per token,   174.58 tokens per second)","n_prompt_tokens_processed":11,"n_tokens_second":174.58100558659217,"t_prompt_processing":63.008,"t_token":5.728000000000001,"tid":"5172","timestamp":1709950444}
{"function":"print_timings","id_slot":0,"id_task":0,"level":"INFO","line":327,"msg":"generation eval time =     845.31 ms /    95 runs   (    8.90 ms per token,   112.38 tokens per second)","n_decoded":95,"n_tokens_second":112.38480557428636,"t_token":8.898,"t_token_generation":845.31,"tid":"5172","timestamp":1709950444}
{"function":"print_timings","id_slot":0,"id_task":0,"level":"INFO","line":337,"msg":"          total time =     908.32 ms","t_prompt_processing":63.008,"t_token_generation":845.31,"t_total":908.318,"tid":"5172","timestamp":1709950444}
{"function":"update_slots","id_slot":0,"id_task":0,"level":"INFO","line":1575,"msg":"slot released","n_cache_tokens":105,"n_ctx":2048,"n_past":105,"n_system_tokens":0,"tid":"5172","timestamp":1709950444,"truncated":false}
{"function":"update_slots","level":"INFO","line":1593,"msg":"all slots are idle","tid":"5172","timestamp":1709950444}
{"function":"update_slots","level":"INFO","line":1593,"msg":"all slots are idle","tid":"5172","timestamp":1709950444}
{"function":"update_slots","level":"INFO","line":1593,"msg":"all slots are idle","tid":"5172","timestamp":1709950480}
loading library C:\Users\user\AppData\Local\Temp\ollama\payloads\rocm_v5.7\ext_server.dll
<!-- gh-comment-id:1986696035 --> @ipfans commented on GitHub (Mar 9, 2024): Thanks a lot for your works. I just tried https://github.com/dhiltgen/ollama/releases for rocm support, but I found it will be failed when using mixtral model. Here is a log for this panic: ``` time=2024-03-09T10:13:54.011+08:00 level=INFO source=images.go:800 msg="total blobs: 8" time=2024-03-09T10:13:54.022+08:00 level=INFO source=images.go:807 msg="total unused blobs removed: 0" time=2024-03-09T10:13:54.023+08:00 level=INFO source=routes.go:1019 msg="Listening on 127.0.0.1:11434 (version 0.1.28-23-ge690faf)" time=2024-03-09T10:13:54.023+08:00 level=INFO source=payload_common.go:109 msg="Extracting dynamic libraries to C:\\Users\\user\\AppData\\Local\\Temp\\ollama\\payloads ..." time=2024-03-09T10:13:54.194+08:00 level=INFO source=payload_common.go:154 msg="Dynamic LLM libraries [rocm_v5.7 cuda_v11.3 cpu cpu_avx2 cpu_avx cuda_v12.3]" [GIN] 2024/03/09 - 10:13:54 | 200 | 0s | 127.0.0.1 | HEAD "/" [GIN] 2024/03/09 - 10:13:54 | 200 | 1.0486ms | 127.0.0.1 | POST "/api/show" [GIN] 2024/03/09 - 10:13:54 | 200 | 525.7µs | 127.0.0.1 | POST "/api/show" time=2024-03-09T10:13:54.847+08:00 level=INFO source=gpu.go:77 msg="Detecting GPU type" time=2024-03-09T10:13:54.847+08:00 level=INFO source=gpu.go:191 msg="Searching for GPU management library nvml.dll" time=2024-03-09T10:13:54.858+08:00 level=INFO source=gpu.go:237 msg="Discovered GPU libraries: []" time=2024-03-09T10:13:54.858+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2024-03-09T10:13:54.875+08:00 level=INFO source=amd_windows.go:40 msg="AMD Driver: 50732000" time=2024-03-09T10:13:54.877+08:00 level=INFO source=amd_windows.go:69 msg="detected 1 hip devices" time=2024-03-09T10:13:54.877+08:00 level=INFO source=amd_windows.go:87 msg="[0] Name: AMD Radeon RX 7900 XTX" time=2024-03-09T10:13:54.877+08:00 level=INFO source=amd_windows.go:90 msg="[0] GcnArchName: gfx1100" time=2024-03-09T10:13:54.877+08:00 level=INFO source=amd_windows.go:106 msg="amdgpu [0] gfx1100 is supported" time=2024-03-09T10:13:55.129+08:00 level=INFO source=amd_windows.go:117 msg="[0] Total Mem: 25618808832" time=2024-03-09T10:13:55.129+08:00 level=INFO source=amd_windows.go:118 msg="[0] Free Mem: 25753026560" time=2024-03-09T10:13:55.129+08:00 level=INFO source=assets.go:36 msg="Updating PATH to C:\\Users\\user\\AppData\\Local\\Programs\\Ollama\\rocm;C:\\Python310\\Scripts\\;C:\\Python310\\;C:\\Program Files\\Zulu\\zulu-8\\bin\\;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Windows\\System32\\OpenSSH\\;C:\\Program Files\\Intel\\WiFi\\bin\\;C:\\Program Files\\Common Files\\Intel\\WirelessCommon\\;D:\\TDM-GCC-64\\bin;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\;C:\\WINDOWS\\System32\\OpenSSH\\;C:\\Program Files\\dotnet\\;C:\\ProgramData\\chocolatey\\bin;C:\\Program Files (x86)\\dotnet\\;C:\\Program Files\\nodejs\\;C:\\Program Files\\D2\\;C:\\Program Files\\Amazon\\AWSSAMCLI\\bin\\;C:\\Program Files\\Amazon\\AWSCLIV2\\;D:\\微信web开发者工具\\dll;C:\\Program Files (x86)\\Windows Kits\\10\\Windows Performance Toolkit\\;C:\\Program Files\\Docker\\Docker\\resources\\bin;C:\\Program Files\\Go\\bin;C:\\Program Files\\PowerShell\\7\\;C:\\Users\\user\\scoop\\shims;C:\\Users\\user\\AppData\\Local\\Microsoft\\WindowsApps;G:\\GOPATH\\bin;C:\\Program Files\\Intel\\WiFi\\bin\\;C:\\Program Files\\Common Files\\Intel\\WirelessCommon\\;C:\\Users\\user\\AppData\\Local\\GitHubDesktop\\bin;D:\\jdk-11\\bin;C:\\Program Files (x86)\\Graphviz2.38\\bin;C:\\Users\\user\\AppData\\Local\\Yarn\\bin;C:\\Users\\user\\.dotnet\\tools;D:\\flutter\\bin;D:\\Program Files\\Git\\usr\\bin;D:\\GitHub CLI\\;C:\\Program Files (x86)\\Nmap;C:\\Program Files\\Graphviz\\bin;C:\\Program Files\\Go\\bin;D:\\flutter\\.pub-cache\\bin;C:\\Users\\user\\go\\bin;C:\\Users\\user\\AppData\\Local\\Atlassian\\SourceTree\\git_local\\bin;C:\\Users\\user\\AppData\\Local\\JetBrains\\Toolbox\\scripts;C:\\Users\\user\\.dotnet\\tools;C:\\Users\\user\\AppData\\Roaming\\npm;C:\\Users\\user\\AppData\\Local\\Programs\\Microsoft VS Code\\bin;C:\\Users\\user\\go\\bin;C:\\Users\\user\\.detaspace\\bin;C:\\Users\\user\\.dotnet\\tools;C:\\Users\\user\\go\\bin;C:\\Users\\user\\AppData\\Local\\Programs\\Ollama" time=2024-03-09T10:13:55.195+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2024-03-09T10:13:55.206+08:00 level=INFO source=amd_windows.go:40 msg="AMD Driver: 50732000" time=2024-03-09T10:13:55.207+08:00 level=INFO source=amd_windows.go:69 msg="detected 1 hip devices" time=2024-03-09T10:13:55.207+08:00 level=INFO source=amd_windows.go:87 msg="[0] Name: AMD Radeon RX 7900 XTX" time=2024-03-09T10:13:55.207+08:00 level=INFO source=amd_windows.go:90 msg="[0] GcnArchName: gfx1100" time=2024-03-09T10:13:55.207+08:00 level=INFO source=amd_windows.go:106 msg="amdgpu [0] gfx1100 is supported" time=2024-03-09T10:13:55.443+08:00 level=INFO source=amd_windows.go:117 msg="[0] Total Mem: 25618808832" time=2024-03-09T10:13:55.443+08:00 level=INFO source=amd_windows.go:118 msg="[0] Free Mem: 25753026560" time=2024-03-09T10:13:55.509+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2024-03-09T10:13:55.509+08:00 level=INFO source=assets.go:36 msg="Updating PATH to C:\\Users\\user\\AppData\\Local\\Temp\\ollama\\payloads\\rocm_v5.7;C:\\Users\\user\\AppData\\Local\\Programs\\Ollama\\rocm;C:\\Python310\\Scripts\\;C:\\Python310\\;C:\\Program Files\\Zulu\\zulu-8\\bin\\;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Windows\\System32\\OpenSSH\\;C:\\Program Files\\Intel\\WiFi\\bin\\;C:\\Program Files\\Common Files\\Intel\\WirelessCommon\\;D:\\TDM-GCC-64\\bin;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\;C:\\WINDOWS\\System32\\OpenSSH\\;C:\\Program Files\\dotnet\\;C:\\ProgramData\\chocolatey\\bin;C:\\Program Files (x86)\\dotnet\\;C:\\Program Files\\nodejs\\;C:\\Program Files\\D2\\;C:\\Program Files\\Amazon\\AWSSAMCLI\\bin\\;C:\\Program Files\\Amazon\\AWSCLIV2\\;D:\\微信web开发者工具\\dll;C:\\Program Files (x86)\\Windows Kits\\10\\Windows Performance Toolkit\\;C:\\Program Files\\Docker\\Docker\\resources\\bin;C:\\Program Files\\Go\\bin;C:\\Program Files\\PowerShell\\7\\;C:\\Users\\user\\scoop\\shims;C:\\Users\\user\\AppData\\Local\\Microsoft\\WindowsApps;G:\\GOPATH\\bin;C:\\Program Files\\Intel\\WiFi\\bin\\;C:\\Program Files\\Common Files\\Intel\\WirelessCommon\\;C:\\Users\\user\\AppData\\Local\\GitHubDesktop\\bin;D:\\jdk-11\\bin;C:\\Program Files (x86)\\Graphviz2.38\\bin;C:\\Users\\user\\AppData\\Local\\Yarn\\bin;C:\\Users\\user\\.dotnet\\tools;D:\\flutter\\bin;D:\\Program Files\\Git\\usr\\bin;D:\\GitHub CLI\\;C:\\Program Files (x86)\\Nmap;C:\\Program Files\\Graphviz\\bin;C:\\Program Files\\Go\\bin;D:\\flutter\\.pub-cache\\bin;C:\\Users\\user\\go\\bin;C:\\Users\\user\\AppData\\Local\\Atlassian\\SourceTree\\git_local\\bin;C:\\Users\\user\\AppData\\Local\\JetBrains\\Toolbox\\scripts;C:\\Users\\user\\.dotnet\\tools;C:\\Users\\user\\AppData\\Roaming\\npm;C:\\Users\\user\\AppData\\Local\\Programs\\Microsoft VS Code\\bin;C:\\Users\\user\\go\\bin;C:\\Users\\user\\.detaspace\\bin;C:\\Users\\user\\.dotnet\\tools;C:\\Users\\user\\go\\bin;C:\\Users\\user\\AppData\\Local\\Programs\\Ollama" time=2024-03-09T10:13:55.533+08:00 level=INFO source=dyn_ext_server.go:90 msg="Loading Dynamic llm server: C:\\Users\\user\\AppData\\Local\\Temp\\ollama\\payloads\\rocm_v5.7\\ext_server.dll" time=2024-03-09T10:13:55.533+08:00 level=INFO source=dyn_ext_server.go:150 msg="Initializing llama server" ggml_init_cublas: GGML_CUDA_FORCE_MMQ: no ggml_init_cublas: CUDA_USE_TENSOR_CORES: yes ggml_init_cublas: found 1 ROCm devices: Device 0: AMD Radeon RX 7900 XTX, compute capability 11.0, VMM: no llama_model_loader: loaded meta data with 24 key-value pairs and 291 tensors from D:\ollama-models\blobs\sha256-e8a35b5937a5e6d5c35d1f2a15f161e07eefe5e5bb0a3cdd42998ee79b057730 (version GGUF V3 (latest)) llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. llama_model_loader: - kv 0: general.architecture str = llama llama_model_loader: - kv 1: general.name str = mistralai llama_model_loader: - kv 2: llama.context_length u32 = 32768 llama_model_loader: - kv 3: llama.embedding_length u32 = 4096 llama_model_loader: - kv 4: llama.block_count u32 = 32 llama_model_loader: - kv 5: llama.feed_forward_length u32 = 14336 llama_model_loader: - kv 6: llama.rope.dimension_count u32 = 128 llama_model_loader: - kv 7: llama.attention.head_count u32 = 32 llama_model_loader: - kv 8: llama.attention.head_count_kv u32 = 8 llama_model_loader: - kv 9: llama.attention.layer_norm_rms_epsilon f32 = 0.000010 llama_model_loader: - kv 10: llama.rope.freq_base f32 = 1000000.000000 llama_model_loader: - kv 11: general.file_type u32 = 2 llama_model_loader: - kv 12: tokenizer.ggml.model str = llama llama_model_loader: - kv 13: tokenizer.ggml.tokens arr[str,32000] = ["<unk>", "<s>", "</s>", "<0x00>", "<... llama_model_loader: - kv 14: tokenizer.ggml.scores arr[f32,32000] = [0.000000, 0.000000, 0.000000, 0.0000... llama_model_loader: - kv 15: tokenizer.ggml.token_type arr[i32,32000] = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ... llama_model_loader: - kv 16: tokenizer.ggml.merges arr[str,58980] = ["▁ t", "i n", "e r", "▁ a", "h e... llama_model_loader: - kv 17: tokenizer.ggml.bos_token_id u32 = 1 llama_model_loader: - kv 18: tokenizer.ggml.eos_token_id u32 = 2 llama_model_loader: - kv 19: tokenizer.ggml.unknown_token_id u32 = 0 llama_model_loader: - kv 20: tokenizer.ggml.add_bos_token bool = true llama_model_loader: - kv 21: tokenizer.ggml.add_eos_token bool = false llama_model_loader: - kv 22: tokenizer.chat_template str = {{ bos_token }}{% for message in mess... llama_model_loader: - kv 23: general.quantization_version u32 = 2 llama_model_loader: - type f32: 65 tensors llama_model_loader: - type q4_0: 225 tensors llama_model_loader: - type q6_K: 1 tensors llm_load_vocab: special tokens definition check successful ( 259/32000 ). llm_load_print_meta: format = GGUF V3 (latest) llm_load_print_meta: arch = llama llm_load_print_meta: vocab type = SPM llm_load_print_meta: n_vocab = 32000 llm_load_print_meta: n_merges = 0 llm_load_print_meta: n_ctx_train = 32768 llm_load_print_meta: n_embd = 4096 llm_load_print_meta: n_head = 32 llm_load_print_meta: n_head_kv = 8 llm_load_print_meta: n_layer = 32 llm_load_print_meta: n_rot = 128 llm_load_print_meta: n_embd_head_k = 128 llm_load_print_meta: n_embd_head_v = 128 llm_load_print_meta: n_gqa = 4 llm_load_print_meta: n_embd_k_gqa = 1024 llm_load_print_meta: n_embd_v_gqa = 1024 llm_load_print_meta: f_norm_eps = 0.0e+00 llm_load_print_meta: f_norm_rms_eps = 1.0e-05 llm_load_print_meta: f_clamp_kqv = 0.0e+00 llm_load_print_meta: f_max_alibi_bias = 0.0e+00 llm_load_print_meta: n_ff = 14336 llm_load_print_meta: n_expert = 0 llm_load_print_meta: n_expert_used = 0 llm_load_print_meta: pooling type = 0 llm_load_print_meta: rope type = 0 llm_load_print_meta: rope scaling = linear llm_load_print_meta: freq_base_train = 1000000.0 llm_load_print_meta: freq_scale_train = 1 llm_load_print_meta: n_yarn_orig_ctx = 32768 llm_load_print_meta: rope_finetuned = unknown llm_load_print_meta: model type = 7B llm_load_print_meta: model ftype = Q4_0 llm_load_print_meta: model params = 7.24 B llm_load_print_meta: model size = 3.83 GiB (4.54 BPW) llm_load_print_meta: general.name = mistralai llm_load_print_meta: BOS token = 1 '<s>' llm_load_print_meta: EOS token = 2 '</s>' llm_load_print_meta: UNK token = 0 '<unk>' llm_load_print_meta: LF token = 13 '<0x0A>' llm_load_tensors: ggml ctx size = 0.22 MiB llm_load_tensors: offloading 32 repeating layers to GPU llm_load_tensors: offloading non-repeating layers to GPU llm_load_tensors: offloaded 33/33 layers to GPU llm_load_tensors: ROCm0 buffer size = 3847.55 MiB llm_load_tensors: CPU buffer size = 70.31 MiB .................................................................................................. llama_new_context_with_model: n_ctx = 2048 llama_new_context_with_model: freq_base = 1000000.0 llama_new_context_with_model: freq_scale = 1 llama_kv_cache_init: ROCm0 KV buffer size = 256.00 MiB llama_new_context_with_model: KV self size = 256.00 MiB, K (f16): 128.00 MiB, V (f16): 128.00 MiB llama_new_context_with_model: ROCm_Host input buffer size = 13.02 MiB llama_new_context_with_model: ROCm0 compute buffer size = 164.00 MiB llama_new_context_with_model: ROCm_Host compute buffer size = 8.00 MiB llama_new_context_with_model: graph splits (measure): 2 time=2024-03-09T10:13:59.231+08:00 level=INFO source=dyn_ext_server.go:162 msg="Starting llama main loop" [GIN] 2024/03/09 - 10:13:59 | 200 | 4.8541149s | 127.0.0.1 | POST "/api/chat" [GIN] 2024/03/09 - 10:14:04 | 200 | 909.5642ms | 127.0.0.1 | POST "/api/chat" [GIN] 2024/03/09 - 10:14:40 | 200 | 0s | 127.0.0.1 | HEAD "/" [GIN] 2024/03/09 - 10:14:40 | 200 | 1.0474ms | 127.0.0.1 | POST "/api/show" [GIN] 2024/03/09 - 10:14:40 | 200 | 1.031ms | 127.0.0.1 | POST "/api/show" time=2024-03-09T10:14:40.272+08:00 level=INFO source=routes.go:76 msg="changing loaded model" time=2024-03-09T10:14:40.952+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2024-03-09T10:14:40.952+08:00 level=INFO source=amd_windows.go:40 msg="AMD Driver: 50732000" time=2024-03-09T10:14:40.953+08:00 level=INFO source=amd_windows.go:69 msg="detected 1 hip devices" time=2024-03-09T10:14:40.953+08:00 level=INFO source=amd_windows.go:87 msg="[0] Name: AMD Radeon RX 7900 XTX" time=2024-03-09T10:14:40.953+08:00 level=INFO source=amd_windows.go:90 msg="[0] GcnArchName: gfx1100" time=2024-03-09T10:14:40.953+08:00 level=INFO source=amd_windows.go:106 msg="amdgpu [0] gfx1100 is supported" time=2024-03-09T10:14:40.953+08:00 level=INFO source=amd_windows.go:117 msg="[0] Total Mem: 24946528256" time=2024-03-09T10:14:40.953+08:00 level=INFO source=amd_windows.go:118 msg="[0] Free Mem: 25753026560" time=2024-03-09T10:14:40.953+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2024-03-09T10:14:40.954+08:00 level=INFO source=amd_windows.go:40 msg="AMD Driver: 50732000" time=2024-03-09T10:14:40.954+08:00 level=INFO source=amd_windows.go:69 msg="detected 1 hip devices" time=2024-03-09T10:14:40.955+08:00 level=INFO source=amd_windows.go:87 msg="[0] Name: AMD Radeon RX 7900 XTX" time=2024-03-09T10:14:40.955+08:00 level=INFO source=amd_windows.go:90 msg="[0] GcnArchName: gfx1100" time=2024-03-09T10:14:40.955+08:00 level=INFO source=amd_windows.go:106 msg="amdgpu [0] gfx1100 is supported" time=2024-03-09T10:14:40.955+08:00 level=INFO source=amd_windows.go:117 msg="[0] Total Mem: 24946528256" time=2024-03-09T10:14:40.955+08:00 level=INFO source=amd_windows.go:118 msg="[0] Free Mem: 25753026560" time=2024-03-09T10:14:40.955+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2024-03-09T10:14:40.955+08:00 level=INFO source=dyn_ext_server.go:90 msg="Loading Dynamic llm server: C:\\Users\\user\\AppData\\Local\\Temp\\ollama\\payloads\\rocm_v5.7\\ext_server.dll" time=2024-03-09T10:14:40.955+08:00 level=INFO source=dyn_ext_server.go:150 msg="Initializing llama server" llama_model_loader: loaded meta data with 26 key-value pairs and 995 tensors from D:\ollama-models\blobs\sha256-e9e56e8bb5f0fcd4860675e6837a8f6a94e659f5fa7dce6a1076279336320f2b (version GGUF V3 (latest)) llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. llama_model_loader: - kv 0: general.architecture str = llama llama_model_loader: - kv 1: general.name str = mistralai llama_model_loader: - kv 2: llama.context_length u32 = 32768 llama_model_loader: - kv 3: llama.embedding_length u32 = 4096 llama_model_loader: - kv 4: llama.block_count u32 = 32 llama_model_loader: - kv 5: llama.feed_forward_length u32 = 14336 llama_model_loader: - kv 6: llama.rope.dimension_count u32 = 128 llama_model_loader: - kv 7: llama.attention.head_count u32 = 32 llama_model_loader: - kv 8: llama.attention.head_count_kv u32 = 8 llama_model_loader: - kv 9: llama.expert_count u32 = 8 llama_model_loader: - kv 10: llama.expert_used_count u32 = 2 llama_model_loader: - kv 11: llama.attention.layer_norm_rms_epsilon f32 = 0.000010 llama_model_loader: - kv 12: llama.rope.freq_base f32 = 1000000.000000 llama_model_loader: - kv 13: general.file_type u32 = 2 llama_model_loader: - kv 14: tokenizer.ggml.model str = llama llama_model_loader: - kv 15: tokenizer.ggml.tokens arr[str,32000] = ["<unk>", "<s>", "</s>", "<0x00>", "<... llama_model_loader: - kv 16: tokenizer.ggml.scores arr[f32,32000] = [0.000000, 0.000000, 0.000000, 0.0000... llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,32000] = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ... llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,58980] = ["▁ t", "i n", "e r", "▁ a", "h e... llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 1 llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 2 llama_model_loader: - kv 21: tokenizer.ggml.unknown_token_id u32 = 0 llama_model_loader: - kv 22: tokenizer.ggml.add_bos_token bool = true llama_model_loader: - kv 23: tokenizer.ggml.add_eos_token bool = false llama_model_loader: - kv 24: tokenizer.chat_template str = {{ bos_token }}{% for message in mess... llama_model_loader: - kv 25: general.quantization_version u32 = 2 llama_model_loader: - type f32: 65 tensors llama_model_loader: - type f16: 32 tensors llama_model_loader: - type q4_0: 833 tensors llama_model_loader: - type q8_0: 64 tensors llama_model_loader: - type q6_K: 1 tensors llm_load_vocab: special tokens definition check successful ( 259/32000 ). llm_load_print_meta: format = GGUF V3 (latest) llm_load_print_meta: arch = llama llm_load_print_meta: vocab type = SPM llm_load_print_meta: n_vocab = 32000 llm_load_print_meta: n_merges = 0 llm_load_print_meta: n_ctx_train = 32768 llm_load_print_meta: n_embd = 4096 llm_load_print_meta: n_head = 32 llm_load_print_meta: n_head_kv = 8 llm_load_print_meta: n_layer = 32 llm_load_print_meta: n_rot = 128 llm_load_print_meta: n_embd_head_k = 128 llm_load_print_meta: n_embd_head_v = 128 llm_load_print_meta: n_gqa = 4 llm_load_print_meta: n_embd_k_gqa = 1024 llm_load_print_meta: n_embd_v_gqa = 1024 llm_load_print_meta: f_norm_eps = 0.0e+00 llm_load_print_meta: f_norm_rms_eps = 1.0e-05 llm_load_print_meta: f_clamp_kqv = 0.0e+00 llm_load_print_meta: f_max_alibi_bias = 0.0e+00 llm_load_print_meta: n_ff = 14336 llm_load_print_meta: n_expert = 8 llm_load_print_meta: n_expert_used = 2 llm_load_print_meta: pooling type = 0 llm_load_print_meta: rope type = 0 llm_load_print_meta: rope scaling = linear llm_load_print_meta: freq_base_train = 1000000.0 llm_load_print_meta: freq_scale_train = 1 llm_load_print_meta: n_yarn_orig_ctx = 32768 llm_load_print_meta: rope_finetuned = unknown llm_load_print_meta: model type = 7B llm_load_print_meta: model ftype = Q4_0 llm_load_print_meta: model params = 46.70 B llm_load_print_meta: model size = 24.62 GiB (4.53 BPW) llm_load_print_meta: general.name = mistralai llm_load_print_meta: BOS token = 1 '<s>' llm_load_print_meta: EOS token = 2 '</s>' llm_load_print_meta: UNK token = 0 '<unk>' llm_load_print_meta: LF token = 13 '<0x0A>' llm_load_tensors: ggml ctx size = 0.76 MiB Exception 0xc0000005 0x1 0x10 0x7ffa12f9a390 PC=0x7ffa12f9a390 signal arrived during external code execution runtime.cgocall(0x7ff6d5b764e0, 0xc001006760) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/cgocall.go:157 +0x3e fp=0xc001006738 sp=0xc001006700 pc=0x7ff6d50a953e github.com/jmorganca/ollama/llm._Cfunc_dyn_llama_server_init({0x7ffa14a90000, 0x7ffa14baf250, 0x7ffa14bafd70, 0x7ffa14bafe20, 0x7ffa14bb0240, 0x7ffa14bb0470, 0x7ffa14bb14b0, 0x7ffa14bb1490, 0x7ffa14bb16e0, 0x7ffa14bb1d00, ...}, ...) _cgo_gotypes.go:282 +0x4d fp=0xc001006760 sp=0xc001006738 pc=0x7ff6d599b0ed github.com/jmorganca/ollama/llm.newDynExtServer.func7(0xc00009c0f0, 0xc000be80d8) C:/Users/danie/code/ollama/llm/dyn_ext_server.go:154 +0x112 fp=0xc0010068a0 sp=0xc001006760 pc=0x7ff6d599c8b2 github.com/jmorganca/ollama/llm.newDynExtServer({0xc00050e180, 0x4b}, {0xc0004d8360, _}, {_, _, _}, {0x0, 0x0, 0x0}, ...) C:/Users/danie/code/ollama/llm/dyn_ext_server.go:154 +0xb50 fp=0xc001006ae8 sp=0xc0010068a0 pc=0x7ff6d599c4f0 github.com/jmorganca/ollama/llm.newLlmServer({{_, _, _}, {_, _}, {_, _}}, {_, _}, {0x0, ...}, ...) C:/Users/danie/code/ollama/llm/llm.go:158 +0x4c5 fp=0xc001006ca8 sp=0xc001006ae8 pc=0x7ff6d5998905 github.com/jmorganca/ollama/llm.New({0xc0004d8360, 0x5e}, {0x0, 0x0, 0x0}, {0x0, _, _}, {{0x0, 0x800, ...}, ...}) C:/Users/danie/code/ollama/llm/llm.go:123 +0x76e fp=0xc001006f18 sp=0xc001006ca8 pc=0x7ff6d59981ce github.com/jmorganca/ollama/server.load(0xc0000dc180?, 0xc0000dc180, {{0x0, 0x800, 0x200, 0x1, 0xffffffffffffffff, 0x0, 0x0, 0x1, ...}, ...}, ...) C:/Users/danie/code/ollama/server/routes.go:83 +0x325 fp=0xc001007068 sp=0xc001006f18 pc=0x7ff6d5b496e5 github.com/jmorganca/ollama/server.ChatHandler(0xc0000a2000) C:/Users/danie/code/ollama/server/routes.go:1173 +0xa37 fp=0xc001007770 sp=0xc001007068 pc=0x7ff6d5b54e57 github.com/gin-gonic/gin.(*Context).Next(...) C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174 github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func1(0xc0000a2000) C:/Users/danie/code/ollama/server/routes.go:943 +0x68 fp=0xc0010077a8 sp=0xc001007770 pc=0x7ff6d5b53628 github.com/gin-gonic/gin.(*Context).Next(...) C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174 github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1(0xc0000a2000) C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/recovery.go:102 +0x7a fp=0xc0010077f8 sp=0xc0010077a8 pc=0x7ff6d5b29eba github.com/gin-gonic/gin.(*Context).Next(...) C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174 github.com/gin-gonic/gin.LoggerWithConfig.func1(0xc0000a2000) C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/logger.go:240 +0xdd fp=0xc0010079a8 sp=0xc0010077f8 pc=0x7ff6d5b28ffd github.com/gin-gonic/gin.(*Context).Next(...) C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174 github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc000168ea0, 0xc0000a2000) C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/gin.go:620 +0x66e fp=0xc001007b28 sp=0xc0010079a8 pc=0x7ff6d5b2852e github.com/gin-gonic/gin.(*Engine).ServeHTTP(0xc000168ea0, {0x7ff6d9539dd0, 0xc00009e0e0}, 0xc0000b45a0) C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/gin.go:576 +0x1b2 fp=0xc001007b60 sp=0xc001007b28 pc=0x7ff6d5b27cf2 net/http.serverHandler.ServeHTTP({0x7ff6d9537c48?}, {0x7ff6d9539dd0?, 0xc00009e0e0?}, 0x6?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:3137 +0x8e fp=0xc001007b90 sp=0xc001007b60 pc=0x7ff6d53aee2e net/http.(*conn).serve(0xc0000e66c0, {0x7ff6d953c1c8, 0xc0001b8660}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:2039 +0x5e8 fp=0xc001007fb8 sp=0xc001007b90 pc=0x7ff6d53aa1e8 net/http.(*Server).Serve.gowrap3() C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:3285 +0x28 fp=0xc001007fe0 sp=0xc001007fb8 pc=0x7ff6d53af648 runtime.goexit({}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc001007fe8 sp=0xc001007fe0 pc=0x7ff6d5112901 created by net/http.(*Server).Serve in goroutine 1 C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:3285 +0x4b4 goroutine 1 gp=0xc00007e000 m=nil [IO wait]: runtime.gopark(0xc000600008?, 0x7ff6d5bfdb00?, 0x20?, 0x2a?, 0xc0005b2a50?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc0006a76f8 sp=0xc0006a76d8 pc=0x7ff6d50e1aae runtime.netpollblock(0x1b4?, 0xd50a9046?, 0xf6?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/netpoll.go:573 +0xf7 fp=0xc0006a7730 sp=0xc0006a76f8 pc=0x7ff6d50d9357 internal/poll.runtime_pollWait(0x20177cadba0, 0x72) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/netpoll.go:345 +0x85 fp=0xc0006a7750 sp=0xc0006a7730 pc=0x7ff6d510c3a5 internal/poll.(*pollDesc).wait(0x7ff6d50c07b6?, 0x7ff6d9c800a0?, 0x0) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0006a7778 sp=0xc0006a7750 pc=0x7ff6d51b0647 internal/poll.execIO(0xc0005b2a20, 0xc0006a7818) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_windows.go:175 +0xe6 fp=0xc0006a77e8 sp=0xc0006a7778 pc=0x7ff6d51b1b26 internal/poll.(*FD).acceptOne(0xc0005b2a08, 0x268, {0xc00000c0f0?, 0x0?, 0x0?}, 0xc000600008?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_windows.go:944 +0x67 fp=0xc0006a7848 sp=0xc0006a77e8 pc=0x7ff6d51b61e7 internal/poll.(*FD).Accept(0xc0005b2a08, 0xc0006a79f8) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_windows.go:978 +0x1bc fp=0xc0006a7900 sp=0xc0006a7848 pc=0x7ff6d51b651c net.(*netFD).accept(0xc0005b2a08) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/fd_windows.go:178 +0x54 fp=0xc0006a7a18 sp=0xc0006a7900 pc=0x7ff6d52476f4 net.(*TCPListener).accept(0xc0004405c0) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/tcpsock_posix.go:159 +0x1e fp=0xc0006a7a40 sp=0xc0006a7a18 pc=0x7ff6d525cb9e net.(*TCPListener).Accept(0xc0004405c0) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/tcpsock.go:327 +0x30 fp=0xc0006a7a70 sp=0xc0006a7a40 pc=0x7ff6d525bc90 net/http.(*onceCloseListener).Accept(0xc0000e66c0?) <autogenerated>:1 +0x24 fp=0xc0006a7a88 sp=0xc0006a7a70 pc=0x7ff6d53d1804 net/http.(*Server).Serve(0xc0000c04b0, {0x7ff6d9539bc0, 0xc0004405c0}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:3255 +0x33e fp=0xc0006a7bb8 sp=0xc0006a7a88 pc=0x7ff6d53af25e github.com/jmorganca/ollama/server.Serve({0x7ff6d9539bc0, 0xc0004405c0}) C:/Users/danie/code/ollama/server/routes.go:1046 +0x468 fp=0xc0006a7cc0 sp=0xc0006a7bb8 pc=0x7ff6d5b53ae8 github.com/jmorganca/ollama/cmd.RunServer(0xc0001e6c00?, {0x7ff6d9c7e340?, 0x4?, 0x7ff6d5db3a7e?}) C:/Users/danie/code/ollama/cmd/cmd.go:787 +0x1b9 fp=0xc0006a7d58 sp=0xc0006a7cc0 pc=0x7ff6d5b6d9b9 github.com/spf13/cobra.(*Command).execute(0xc0000b8f08, {0x7ff6d9c7e340, 0x0, 0x0}) C:/Users/danie/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:940 +0x882 fp=0xc0006a7e78 sp=0xc0006a7d58 pc=0x7ff6d544afe2 github.com/spf13/cobra.(*Command).ExecuteC(0xc0000b8308) C:/Users/danie/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3a5 fp=0xc0006a7f30 sp=0xc0006a7e78 pc=0x7ff6d544b825 github.com/spf13/cobra.(*Command).Execute(...) C:/Users/danie/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:992 github.com/spf13/cobra.(*Command).ExecuteContext(...) C:/Users/danie/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:985 main.main() C:/Users/danie/code/ollama/main.go:11 +0x4d fp=0xc0006a7f50 sp=0xc0006a7f30 pc=0x7ff6d5b7624d runtime.main() C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:271 +0x28b fp=0xc0006a7fe0 sp=0xc0006a7f50 pc=0x7ff6d50e16ab runtime.goexit({}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0006a7fe8 sp=0xc0006a7fe0 pc=0x7ff6d5112901 goroutine 2 gp=0xc00007e700 m=nil [force gc (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000081fa8 sp=0xc000081f88 pc=0x7ff6d50e1aae runtime.goparkunlock(...) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:408 runtime.forcegchelper() C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:326 +0xb8 fp=0xc000081fe0 sp=0xc000081fa8 pc=0x7ff6d50e1938 runtime.goexit({}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000081fe8 sp=0xc000081fe0 pc=0x7ff6d5112901 created by runtime.init.6 in goroutine 1 C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:314 +0x1a goroutine 3 gp=0xc00007ea80 m=nil [GC sweep wait]: runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000083f80 sp=0xc000083f60 pc=0x7ff6d50e1aae runtime.goparkunlock(...) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:408 runtime.bgsweep(0xc00008e000) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgcsweep.go:318 +0xdf fp=0xc000083fc8 sp=0xc000083f80 pc=0x7ff6d50cbb5f runtime.gcenable.gowrap1() C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:203 +0x25 fp=0xc000083fe0 sp=0xc000083fc8 pc=0x7ff6d50c0405 runtime.goexit({}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000083fe8 sp=0xc000083fe0 pc=0x7ff6d5112901 created by runtime.gcenable in goroutine 1 C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:203 +0x66 goroutine 4 gp=0xc00007ec40 m=nil [GC scavenge wait]: runtime.gopark(0x991e0c?, 0x86fc3e?, 0x0?, 0x0?, 0x0?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000095f78 sp=0xc000095f58 pc=0x7ff6d50e1aae runtime.goparkunlock(...) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:408 runtime.(*scavengerState).park(0x7ff6d9bf3020) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc000095fa8 sp=0xc000095f78 pc=0x7ff6d50c94e9 runtime.bgscavenge(0xc00008e000) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgcscavenge.go:658 +0x59 fp=0xc000095fc8 sp=0xc000095fa8 pc=0x7ff6d50c9a99 runtime.gcenable.gowrap2() C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:204 +0x25 fp=0xc000095fe0 sp=0xc000095fc8 pc=0x7ff6d50c03a5 runtime.goexit({}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000095fe8 sp=0xc000095fe0 pc=0x7ff6d5112901 created by runtime.gcenable in goroutine 1 C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:204 +0xa5 goroutine 18 gp=0xc000104380 m=nil [finalizer wait]: runtime.gopark(0xc000085e48?, 0x7ff6d50b37c5?, 0xa8?, 0x1?, 0xc00007e000?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000085e20 sp=0xc000085e00 pc=0x7ff6d50e1aae runtime.runfinq() C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mfinal.go:194 +0x107 fp=0xc000085fe0 sp=0xc000085e20 pc=0x7ff6d50bf487 runtime.goexit({}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000085fe8 sp=0xc000085fe0 pc=0x7ff6d5112901 created by runtime.createfing in goroutine 1 C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mfinal.go:164 +0x3d goroutine 19 gp=0xc000105880 m=nil [GC worker (idle)]: runtime.gopark(0x7ff6d9c800a0?, 0x1?, 0x0?, 0x0?, 0x0?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000091f50 sp=0xc000091f30 pc=0x7ff6d50e1aae runtime.gcBgMarkWorker() C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000091fe0 sp=0xc000091f50 pc=0x7ff6d50c2545 runtime.goexit({}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000091fe8 sp=0xc000091fe0 pc=0x7ff6d5112901 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c goroutine 34 gp=0xc000480000 m=nil [GC worker (idle)]: runtime.gopark(0x3135b6217c2c?, 0x1?, 0x0?, 0x0?, 0x0?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000487f50 sp=0xc000487f30 pc=0x7ff6d50e1aae runtime.gcBgMarkWorker() C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000487fe0 sp=0xc000487f50 pc=0x7ff6d50c2545 runtime.goexit({}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000487fe8 sp=0xc000487fe0 pc=0x7ff6d5112901 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c goroutine 5 gp=0xc00007efc0 m=nil [GC worker (idle)]: runtime.gopark(0x7ff6d9c800a0?, 0x3?, 0x0?, 0x0?, 0x0?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000097f50 sp=0xc000097f30 pc=0x7ff6d50e1aae runtime.gcBgMarkWorker() C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000097fe0 sp=0xc000097f50 pc=0x7ff6d50c2545 runtime.goexit({}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000097fe8 sp=0xc000097fe0 pc=0x7ff6d5112901 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c goroutine 35 gp=0xc0004801c0 m=nil [GC worker (idle)]: runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000489f50 sp=0xc000489f30 pc=0x7ff6d50e1aae runtime.gcBgMarkWorker() C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000489fe0 sp=0xc000489f50 pc=0x7ff6d50c2545 runtime.goexit({}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000489fe8 sp=0xc000489fe0 pc=0x7ff6d5112901 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c goroutine 6 gp=0xc00007f180 m=nil [GC worker (idle)]: runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000483f50 sp=0xc000483f30 pc=0x7ff6d50e1aae runtime.gcBgMarkWorker() C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000483fe0 sp=0xc000483f50 pc=0x7ff6d50c2545 runtime.goexit({}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000483fe8 sp=0xc000483fe0 pc=0x7ff6d5112901 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c goroutine 36 gp=0xc000480380 m=nil [GC worker (idle)]: runtime.gopark(0x3135b6217c2c?, 0x1?, 0x0?, 0x0?, 0x0?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000491f50 sp=0xc000491f30 pc=0x7ff6d50e1aae runtime.gcBgMarkWorker() C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000491fe0 sp=0xc000491f50 pc=0x7ff6d50c2545 runtime.goexit({}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000491fe8 sp=0xc000491fe0 pc=0x7ff6d5112901 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c goroutine 7 gp=0xc00007f340 m=nil [GC worker (idle)]: runtime.gopark(0x3135b6217c2c?, 0x3?, 0x1c?, 0xfa?, 0x0?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000485f50 sp=0xc000485f30 pc=0x7ff6d50e1aae runtime.gcBgMarkWorker() C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000485fe0 sp=0xc000485f50 pc=0x7ff6d50c2545 runtime.goexit({}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000485fe8 sp=0xc000485fe0 pc=0x7ff6d5112901 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c goroutine 37 gp=0xc000480540 m=nil [GC worker (idle)]: runtime.gopark(0x7ff6d9c800a0?, 0x3?, 0x40?, 0xbd?, 0x0?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000493f50 sp=0xc000493f30 pc=0x7ff6d50e1aae runtime.gcBgMarkWorker() C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000493fe0 sp=0xc000493f50 pc=0x7ff6d50c2545 runtime.goexit({}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000493fe8 sp=0xc000493fe0 pc=0x7ff6d5112901 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c goroutine 8 gp=0xc00007f500 m=nil [GC worker (idle)]: runtime.gopark(0x3135b6217c2c?, 0x1?, 0x0?, 0x0?, 0x0?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc00048df50 sp=0xc00048df30 pc=0x7ff6d50e1aae runtime.gcBgMarkWorker() C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc00048dfe0 sp=0xc00048df50 pc=0x7ff6d50c2545 runtime.goexit({}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00048dfe8 sp=0xc00048dfe0 pc=0x7ff6d5112901 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c goroutine 38 gp=0xc000480700 m=nil [GC worker (idle)]: runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000499f50 sp=0xc000499f30 pc=0x7ff6d50e1aae runtime.gcBgMarkWorker() C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000499fe0 sp=0xc000499f50 pc=0x7ff6d50c2545 runtime.goexit({}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000499fe8 sp=0xc000499fe0 pc=0x7ff6d5112901 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c goroutine 9 gp=0xc00007f6c0 m=nil [GC worker (idle)]: runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc00048ff50 sp=0xc00048ff30 pc=0x7ff6d50e1aae runtime.gcBgMarkWorker() C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc00048ffe0 sp=0xc00048ff50 pc=0x7ff6d50c2545 runtime.goexit({}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00048ffe8 sp=0xc00048ffe0 pc=0x7ff6d5112901 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c goroutine 39 gp=0xc0004808c0 m=nil [GC worker (idle)]: runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc00049bf50 sp=0xc00049bf30 pc=0x7ff6d50e1aae runtime.gcBgMarkWorker() C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc00049bfe0 sp=0xc00049bf50 pc=0x7ff6d50c2545 runtime.goexit({}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00049bfe8 sp=0xc00049bfe0 pc=0x7ff6d5112901 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c goroutine 10 gp=0xc00007f880 m=nil [GC worker (idle)]: runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000495f50 sp=0xc000495f30 pc=0x7ff6d50e1aae runtime.gcBgMarkWorker() C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000495fe0 sp=0xc000495f50 pc=0x7ff6d50c2545 runtime.goexit({}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000495fe8 sp=0xc000495fe0 pc=0x7ff6d5112901 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c goroutine 40 gp=0xc000480a80 m=nil [GC worker (idle)]: runtime.gopark(0xc000059c50?, 0x1?, 0x2c?, 0x7c?, 0x0?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc0004a1f50 sp=0xc0004a1f30 pc=0x7ff6d50e1aae runtime.gcBgMarkWorker() C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc0004a1fe0 sp=0xc0004a1f50 pc=0x7ff6d50c2545 runtime.goexit({}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0004a1fe8 sp=0xc0004a1fe0 pc=0x7ff6d5112901 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c goroutine 11 gp=0xc00007fa40 m=nil [GC worker (idle)]: runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000497f50 sp=0xc000497f30 pc=0x7ff6d50e1aae runtime.gcBgMarkWorker() C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000497fe0 sp=0xc000497f50 pc=0x7ff6d50c2545 runtime.goexit({}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000497fe8 sp=0xc000497fe0 pc=0x7ff6d5112901 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c goroutine 41 gp=0xc000480c40 m=nil [GC worker (idle)]: runtime.gopark(0x3135b629396c?, 0x3?, 0x0?, 0x0?, 0x0?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc0004a3f50 sp=0xc0004a3f30 pc=0x7ff6d50e1aae runtime.gcBgMarkWorker() C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc0004a3fe0 sp=0xc0004a3f50 pc=0x7ff6d50c2545 runtime.goexit({}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0004a3fe8 sp=0xc0004a3fe0 pc=0x7ff6d5112901 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c goroutine 12 gp=0xc000481180 m=4 mp=0xc00008b808 [syscall]: runtime.notetsleepg(0x7ff6d9c7eee0, 0xffffffffffffffff) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/lock_sema.go:296 +0x31 fp=0xc00049ffa0 sp=0xc00049ff68 pc=0x7ff6d50b1d91 os/signal.signal_recv() C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/sigqueue.go:152 +0x29 fp=0xc00049ffc0 sp=0xc00049ffa0 pc=0x7ff6d510e5e9 os/signal.loop() C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/os/signal/signal_unix.go:23 +0x13 fp=0xc00049ffe0 sp=0xc00049ffc0 pc=0x7ff6d53d3bb3 runtime.goexit({}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00049ffe8 sp=0xc00049ffe0 pc=0x7ff6d5112901 created by os/signal.Notify.func1.1 in goroutine 1 C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/os/signal/signal.go:151 +0x1f goroutine 13 gp=0xc000481340 m=nil [chan receive]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000093f18 sp=0xc000093ef8 pc=0x7ff6d50e1aae runtime.chanrecv(0xc0001eafc0, 0x0, 0x1) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/chan.go:583 +0x3cd fp=0xc000093f90 sp=0xc000093f18 pc=0x7ff6d50abbcd runtime.chanrecv1(0x0?, 0x0?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/chan.go:442 +0x12 fp=0xc000093fb8 sp=0xc000093f90 pc=0x7ff6d50ab7d2 github.com/jmorganca/ollama/server.Serve.func2() C:/Users/danie/code/ollama/server/routes.go:1028 +0x25 fp=0xc000093fe0 sp=0xc000093fb8 pc=0x7ff6d5b53b85 runtime.goexit({}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000093fe8 sp=0xc000093fe0 pc=0x7ff6d5112901 created by github.com/jmorganca/ollama/server.Serve in goroutine 1 C:/Users/danie/code/ollama/server/routes.go:1027 +0x3f6 goroutine 83 gp=0xc0005048c0 m=nil [IO wait]: runtime.gopark(0x0?, 0xc000698020?, 0xd0?, 0x80?, 0xc000698050?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc0004cbd28 sp=0xc0004cbd08 pc=0x7ff6d50e1aae runtime.netpollblock(0x280?, 0xd50a9046?, 0xf6?) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/netpoll.go:573 +0xf7 fp=0xc0004cbd60 sp=0xc0004cbd28 pc=0x7ff6d50d9357 internal/poll.runtime_pollWait(0x20177cadaa8, 0x72) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/netpoll.go:345 +0x85 fp=0xc0004cbd80 sp=0xc0004cbd60 pc=0x7ff6d510c3a5 internal/poll.(*pollDesc).wait(0x10?, 0x10?, 0x0) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0004cbda8 sp=0xc0004cbd80 pc=0x7ff6d51b0647 internal/poll.execIO(0xc000698020, 0x7ff6d9419910) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_windows.go:175 +0xe6 fp=0xc0004cbe18 sp=0xc0004cbda8 pc=0x7ff6d51b1b26 internal/poll.(*FD).Read(0xc000698008, {0xc000f0c641, 0x1, 0x1}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_windows.go:436 +0x2b1 fp=0xc0004cbec0 sp=0xc0004cbe18 pc=0x7ff6d51b27d1 net.(*netFD).Read(0xc000698008, {0xc000f0c641?, 0xc0004cbf48?, 0x7ff6d510e330?}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/fd_posix.go:55 +0x25 fp=0xc0004cbf08 sp=0xc0004cbec0 pc=0x7ff6d5245805 net.(*conn).Read(0xc000110040, {0xc000f0c641?, 0xc0004cbeb8?, 0x7ff6d9c7e340?}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/net.go:179 +0x45 fp=0xc0004cbf50 sp=0xc0004cbf08 pc=0x7ff6d5254885 net.(*TCPConn).Read(0x7ff6d5bd7aa0?, {0xc000f0c641?, 0xc0000983c0?, 0x7ff6d54a25a0?}) <autogenerated>:1 +0x25 fp=0xc0004cbf80 sp=0xc0004cbf50 pc=0x7ff6d52648a5 net/http.(*connReader).backgroundRead(0xc000f0c630) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:681 +0x37 fp=0xc0004cbfc8 sp=0xc0004cbf80 pc=0x7ff6d53a4157 net/http.(*connReader).startBackgroundRead.gowrap2() C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:677 +0x25 fp=0xc0004cbfe0 sp=0xc0004cbfc8 pc=0x7ff6d53a4085 runtime.goexit({}) C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0004cbfe8 sp=0xc0004cbfe0 pc=0x7ff6d5112901 created by net/http.(*connReader).startBackgroundRead in goroutine 21 C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:677 +0xba rax 0x0 rbx 0x2012414aa68 rcx 0x2012414aa18 rdx 0xffffffdd00000000 rdi 0x2012414aa68 rsi 0x2012414aa18 rbp 0x0 rsp 0x8f23ffd7b0 r8 0x2012414aa68 r9 0x0 r10 0x96 r11 0x246 r12 0x0 r13 0x7ffa12a30000 r14 0x2014e56d9c0 r15 0x8f23ffd970 rip 0x7ffa12f9a390 rflags 0x10246 cs 0x33 fs 0x53 gs 0x2b {"function":"initialize","level":"INFO","line":688,"msg":"initializing slots","n_slots":1,"tid":"19848","timestamp":1709950439} {"function":"initialize","id_slot":0,"level":"INFO","line":699,"msg":"new slot","n_ctx_slot":2048,"tid":"19848","timestamp":1709950439} {"function":"update_slots","level":"INFO","line":1593,"msg":"all slots are idle","tid":"5172","timestamp":1709950439} {"function":"launch_slot_with_data","id_slot":0,"id_task":0,"level":"INFO","line":973,"msg":"slot is processing task","tid":"5172","timestamp":1709950443} {"function":"update_slots","id_slot":0,"id_task":0,"level":"INFO","line":1845,"msg":"kv cache rm [p0, end)","p0":0,"tid":"5172","timestamp":1709950443} {"function":"print_timings","id_slot":0,"id_task":0,"level":"INFO","line":311,"msg":"prompt eval time = 63.01 ms / 11 tokens ( 5.73 ms per token, 174.58 tokens per second)","n_prompt_tokens_processed":11,"n_tokens_second":174.58100558659217,"t_prompt_processing":63.008,"t_token":5.728000000000001,"tid":"5172","timestamp":1709950444} {"function":"print_timings","id_slot":0,"id_task":0,"level":"INFO","line":327,"msg":"generation eval time = 845.31 ms / 95 runs ( 8.90 ms per token, 112.38 tokens per second)","n_decoded":95,"n_tokens_second":112.38480557428636,"t_token":8.898,"t_token_generation":845.31,"tid":"5172","timestamp":1709950444} {"function":"print_timings","id_slot":0,"id_task":0,"level":"INFO","line":337,"msg":" total time = 908.32 ms","t_prompt_processing":63.008,"t_token_generation":845.31,"t_total":908.318,"tid":"5172","timestamp":1709950444} {"function":"update_slots","id_slot":0,"id_task":0,"level":"INFO","line":1575,"msg":"slot released","n_cache_tokens":105,"n_ctx":2048,"n_past":105,"n_system_tokens":0,"tid":"5172","timestamp":1709950444,"truncated":false} {"function":"update_slots","level":"INFO","line":1593,"msg":"all slots are idle","tid":"5172","timestamp":1709950444} {"function":"update_slots","level":"INFO","line":1593,"msg":"all slots are idle","tid":"5172","timestamp":1709950444} {"function":"update_slots","level":"INFO","line":1593,"msg":"all slots are idle","tid":"5172","timestamp":1709950480} loading library C:\Users\user\AppData\Local\Temp\ollama\payloads\rocm_v5.7\ext_server.dll ```
GiteaMirror commented 2026-04-12 11:26:39 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Mar 11, 2024):

The pre-release for 0.1.29 is live and ready for broader testing.

Windows users can install the OllamaSetup.exe from the 0.1.29 release page which includes ROCm v5.7 (latest at this time.)

We've updated our troubleshooting docs to include some pointers on Radeon GPU compatibility.

Please let us know if you run into any problems with the pre-release (on this issue, or open a new issue)

<!-- gh-comment-id:1989171492 --> @dhiltgen commented on GitHub (Mar 11, 2024): The pre-release for [0.1.29](https://github.com/ollama/ollama/releases/tag/v0.1.29) is live and ready for broader testing. Windows users can install the OllamaSetup.exe from the [0.1.29 release page](https://github.com/ollama/ollama/releases/tag/v0.1.29) which includes ROCm v5.7 (latest at this time.) We've updated our [troubleshooting docs](https://github.com/ollama/ollama/blob/main/docs/troubleshooting.md#amd-radeon-gpu-support) to include some pointers on Radeon GPU compatibility. Please let us know if you run into any problems with the pre-release (on this issue, or open a new issue)
GiteaMirror commented 2026-04-12 11:26:40 -05:00
Author
Owner
Copy Link

@sgwhat commented on GitHub (Mar 19, 2024):

Thanks a lot for your works. I just tried https://github.com/dhiltgen/ollama/releases for rocm support, but I found it will be failed when using mixtral model. Here is a log for this panic:


time=2024-03-09T10:13:54.011+08:00 level=INFO source=images.go:800 msg="total blobs: 8"

time=2024-03-09T10:13:54.022+08:00 level=INFO source=images.go:807 msg="total unused blobs removed: 0"

time=2024-03-09T10:13:54.023+08:00 level=INFO source=routes.go:1019 msg="Listening on 127.0.0.1:11434 (version 0.1.28-23-ge690faf)"

time=2024-03-09T10:13:54.023+08:00 level=INFO source=payload_common.go:109 msg="Extracting dynamic libraries to C:\\Users\\user\\AppData\\Local\\Temp\\ollama\\payloads ..."

time=2024-03-09T10:13:54.194+08:00 level=INFO source=payload_common.go:154 msg="Dynamic LLM libraries [rocm_v5.7 cuda_v11.3 cpu cpu_avx2 cpu_avx cuda_v12.3]"

[GIN] 2024/03/09 - 10:13:54 | 200 |            0s |       127.0.0.1 | HEAD     "/"

[GIN] 2024/03/09 - 10:13:54 | 200 |      1.0486ms |       127.0.0.1 | POST     "/api/show"

[GIN] 2024/03/09 - 10:13:54 | 200 |       525.7µs |       127.0.0.1 | POST     "/api/show"

time=2024-03-09T10:13:54.847+08:00 level=INFO source=gpu.go:77 msg="Detecting GPU type"

time=2024-03-09T10:13:54.847+08:00 level=INFO source=gpu.go:191 msg="Searching for GPU management library nvml.dll"

time=2024-03-09T10:13:54.858+08:00 level=INFO source=gpu.go:237 msg="Discovered GPU libraries: []"

time=2024-03-09T10:13:54.858+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"

time=2024-03-09T10:13:54.875+08:00 level=INFO source=amd_windows.go:40 msg="AMD Driver: 50732000"

time=2024-03-09T10:13:54.877+08:00 level=INFO source=amd_windows.go:69 msg="detected 1 hip devices"

time=2024-03-09T10:13:54.877+08:00 level=INFO source=amd_windows.go:87 msg="[0] Name: AMD Radeon RX 7900 XTX"

time=2024-03-09T10:13:54.877+08:00 level=INFO source=amd_windows.go:90 msg="[0] GcnArchName: gfx1100"

time=2024-03-09T10:13:54.877+08:00 level=INFO source=amd_windows.go:106 msg="amdgpu [0] gfx1100 is supported"

time=2024-03-09T10:13:55.129+08:00 level=INFO source=amd_windows.go:117 msg="[0] Total Mem: 25618808832"

time=2024-03-09T10:13:55.129+08:00 level=INFO source=amd_windows.go:118 msg="[0] Free Mem:  25753026560"

time=2024-03-09T10:13:55.129+08:00 level=INFO source=assets.go:36 msg="Updating PATH to C:\\Users\\user\\AppData\\Local\\Programs\\Ollama\\rocm;C:\\Python310\\Scripts\\;C:\\Python310\\;C:\\Program Files\\Zulu\\zulu-8\\bin\\;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Windows\\System32\\OpenSSH\\;C:\\Program Files\\Intel\\WiFi\\bin\\;C:\\Program Files\\Common Files\\Intel\\WirelessCommon\\;D:\\TDM-GCC-64\\bin;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\;C:\\WINDOWS\\System32\\OpenSSH\\;C:\\Program Files\\dotnet\\;C:\\ProgramData\\chocolatey\\bin;C:\\Program Files (x86)\\dotnet\\;C:\\Program Files\\nodejs\\;C:\\Program Files\\D2\\;C:\\Program Files\\Amazon\\AWSSAMCLI\\bin\\;C:\\Program Files\\Amazon\\AWSCLIV2\\;D:\\微信web开发者工具\\dll;C:\\Program Files (x86)\\Windows Kits\\10\\Windows Performance Toolkit\\;C:\\Program Files\\Docker\\Docker\\resources\\bin;C:\\Program Files\\Go\\bin;C:\\Program Files\\PowerShell\\7\\;C:\\Users\\user\\scoop\\shims;C:\\Users\\user\\AppData\\Local\\Microsoft\\WindowsApps;G:\\GOPATH\\bin;C:\\Program Files\\Intel\\WiFi\\bin\\;C:\\Program Files\\Common Files\\Intel\\WirelessCommon\\;C:\\Users\\user\\AppData\\Local\\GitHubDesktop\\bin;D:\\jdk-11\\bin;C:\\Program Files (x86)\\Graphviz2.38\\bin;C:\\Users\\user\\AppData\\Local\\Yarn\\bin;C:\\Users\\user\\.dotnet\\tools;D:\\flutter\\bin;D:\\Program Files\\Git\\usr\\bin;D:\\GitHub CLI\\;C:\\Program Files (x86)\\Nmap;C:\\Program Files\\Graphviz\\bin;C:\\Program Files\\Go\\bin;D:\\flutter\\.pub-cache\\bin;C:\\Users\\user\\go\\bin;C:\\Users\\user\\AppData\\Local\\Atlassian\\SourceTree\\git_local\\bin;C:\\Users\\user\\AppData\\Local\\JetBrains\\Toolbox\\scripts;C:\\Users\\user\\.dotnet\\tools;C:\\Users\\user\\AppData\\Roaming\\npm;C:\\Users\\user\\AppData\\Local\\Programs\\Microsoft VS Code\\bin;C:\\Users\\user\\go\\bin;C:\\Users\\user\\.detaspace\\bin;C:\\Users\\user\\.dotnet\\tools;C:\\Users\\user\\go\\bin;C:\\Users\\user\\AppData\\Local\\Programs\\Ollama"

time=2024-03-09T10:13:55.195+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"

time=2024-03-09T10:13:55.206+08:00 level=INFO source=amd_windows.go:40 msg="AMD Driver: 50732000"

time=2024-03-09T10:13:55.207+08:00 level=INFO source=amd_windows.go:69 msg="detected 1 hip devices"

time=2024-03-09T10:13:55.207+08:00 level=INFO source=amd_windows.go:87 msg="[0] Name: AMD Radeon RX 7900 XTX"

time=2024-03-09T10:13:55.207+08:00 level=INFO source=amd_windows.go:90 msg="[0] GcnArchName: gfx1100"

time=2024-03-09T10:13:55.207+08:00 level=INFO source=amd_windows.go:106 msg="amdgpu [0] gfx1100 is supported"

time=2024-03-09T10:13:55.443+08:00 level=INFO source=amd_windows.go:117 msg="[0] Total Mem: 25618808832"

time=2024-03-09T10:13:55.443+08:00 level=INFO source=amd_windows.go:118 msg="[0] Free Mem:  25753026560"

time=2024-03-09T10:13:55.509+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"

time=2024-03-09T10:13:55.509+08:00 level=INFO source=assets.go:36 msg="Updating PATH to C:\\Users\\user\\AppData\\Local\\Temp\\ollama\\payloads\\rocm_v5.7;C:\\Users\\user\\AppData\\Local\\Programs\\Ollama\\rocm;C:\\Python310\\Scripts\\;C:\\Python310\\;C:\\Program Files\\Zulu\\zulu-8\\bin\\;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Windows\\System32\\OpenSSH\\;C:\\Program Files\\Intel\\WiFi\\bin\\;C:\\Program Files\\Common Files\\Intel\\WirelessCommon\\;D:\\TDM-GCC-64\\bin;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\;C:\\WINDOWS\\System32\\OpenSSH\\;C:\\Program Files\\dotnet\\;C:\\ProgramData\\chocolatey\\bin;C:\\Program Files (x86)\\dotnet\\;C:\\Program Files\\nodejs\\;C:\\Program Files\\D2\\;C:\\Program Files\\Amazon\\AWSSAMCLI\\bin\\;C:\\Program Files\\Amazon\\AWSCLIV2\\;D:\\微信web开发者工具\\dll;C:\\Program Files (x86)\\Windows Kits\\10\\Windows Performance Toolkit\\;C:\\Program Files\\Docker\\Docker\\resources\\bin;C:\\Program Files\\Go\\bin;C:\\Program Files\\PowerShell\\7\\;C:\\Users\\user\\scoop\\shims;C:\\Users\\user\\AppData\\Local\\Microsoft\\WindowsApps;G:\\GOPATH\\bin;C:\\Program Files\\Intel\\WiFi\\bin\\;C:\\Program Files\\Common Files\\Intel\\WirelessCommon\\;C:\\Users\\user\\AppData\\Local\\GitHubDesktop\\bin;D:\\jdk-11\\bin;C:\\Program Files (x86)\\Graphviz2.38\\bin;C:\\Users\\user\\AppData\\Local\\Yarn\\bin;C:\\Users\\user\\.dotnet\\tools;D:\\flutter\\bin;D:\\Program Files\\Git\\usr\\bin;D:\\GitHub CLI\\;C:\\Program Files (x86)\\Nmap;C:\\Program Files\\Graphviz\\bin;C:\\Program Files\\Go\\bin;D:\\flutter\\.pub-cache\\bin;C:\\Users\\user\\go\\bin;C:\\Users\\user\\AppData\\Local\\Atlassian\\SourceTree\\git_local\\bin;C:\\Users\\user\\AppData\\Local\\JetBrains\\Toolbox\\scripts;C:\\Users\\user\\.dotnet\\tools;C:\\Users\\user\\AppData\\Roaming\\npm;C:\\Users\\user\\AppData\\Local\\Programs\\Microsoft VS Code\\bin;C:\\Users\\user\\go\\bin;C:\\Users\\user\\.detaspace\\bin;C:\\Users\\user\\.dotnet\\tools;C:\\Users\\user\\go\\bin;C:\\Users\\user\\AppData\\Local\\Programs\\Ollama"

time=2024-03-09T10:13:55.533+08:00 level=INFO source=dyn_ext_server.go:90 msg="Loading Dynamic llm server: C:\\Users\\user\\AppData\\Local\\Temp\\ollama\\payloads\\rocm_v5.7\\ext_server.dll"

time=2024-03-09T10:13:55.533+08:00 level=INFO source=dyn_ext_server.go:150 msg="Initializing llama server"

ggml_init_cublas: GGML_CUDA_FORCE_MMQ:   no

ggml_init_cublas: CUDA_USE_TENSOR_CORES: yes

ggml_init_cublas: found 1 ROCm devices:

  Device 0: AMD Radeon RX 7900 XTX, compute capability 11.0, VMM: no

llama_model_loader: loaded meta data with 24 key-value pairs and 291 tensors from D:\ollama-models\blobs\sha256-e8a35b5937a5e6d5c35d1f2a15f161e07eefe5e5bb0a3cdd42998ee79b057730 (version GGUF V3 (latest))

llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.

llama_model_loader: - kv   0:                       general.architecture str              = llama

llama_model_loader: - kv   1:                               general.name str              = mistralai

llama_model_loader: - kv   2:                       llama.context_length u32              = 32768

llama_model_loader: - kv   3:                     llama.embedding_length u32              = 4096

llama_model_loader: - kv   4:                          llama.block_count u32              = 32

llama_model_loader: - kv   5:                  llama.feed_forward_length u32              = 14336

llama_model_loader: - kv   6:                 llama.rope.dimension_count u32              = 128

llama_model_loader: - kv   7:                 llama.attention.head_count u32              = 32

llama_model_loader: - kv   8:              llama.attention.head_count_kv u32              = 8

llama_model_loader: - kv   9:     llama.attention.layer_norm_rms_epsilon f32              = 0.000010

llama_model_loader: - kv  10:                       llama.rope.freq_base f32              = 1000000.000000

llama_model_loader: - kv  11:                          general.file_type u32              = 2

llama_model_loader: - kv  12:                       tokenizer.ggml.model str              = llama

llama_model_loader: - kv  13:                      tokenizer.ggml.tokens arr[str,32000]   = ["<unk>", "<s>", "</s>", "<0x00>", "<...

llama_model_loader: - kv  14:                      tokenizer.ggml.scores arr[f32,32000]   = [0.000000, 0.000000, 0.000000, 0.0000...

llama_model_loader: - kv  15:                  tokenizer.ggml.token_type arr[i32,32000]   = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ...

llama_model_loader: - kv  16:                      tokenizer.ggml.merges arr[str,58980]   = ["▁ t", "i n", "e r", "▁ a", "h e...

llama_model_loader: - kv  17:                tokenizer.ggml.bos_token_id u32              = 1

llama_model_loader: - kv  18:                tokenizer.ggml.eos_token_id u32              = 2

llama_model_loader: - kv  19:            tokenizer.ggml.unknown_token_id u32              = 0

llama_model_loader: - kv  20:               tokenizer.ggml.add_bos_token bool             = true

llama_model_loader: - kv  21:               tokenizer.ggml.add_eos_token bool             = false

llama_model_loader: - kv  22:                    tokenizer.chat_template str              = {{ bos_token }}{% for message in mess...

llama_model_loader: - kv  23:               general.quantization_version u32              = 2

llama_model_loader: - type  f32:   65 tensors

llama_model_loader: - type q4_0:  225 tensors

llama_model_loader: - type q6_K:    1 tensors

llm_load_vocab: special tokens definition check successful ( 259/32000 ).

llm_load_print_meta: format           = GGUF V3 (latest)

llm_load_print_meta: arch             = llama

llm_load_print_meta: vocab type       = SPM

llm_load_print_meta: n_vocab          = 32000

llm_load_print_meta: n_merges         = 0

llm_load_print_meta: n_ctx_train      = 32768

llm_load_print_meta: n_embd           = 4096

llm_load_print_meta: n_head           = 32

llm_load_print_meta: n_head_kv        = 8

llm_load_print_meta: n_layer          = 32

llm_load_print_meta: n_rot            = 128

llm_load_print_meta: n_embd_head_k    = 128

llm_load_print_meta: n_embd_head_v    = 128

llm_load_print_meta: n_gqa            = 4

llm_load_print_meta: n_embd_k_gqa     = 1024

llm_load_print_meta: n_embd_v_gqa     = 1024

llm_load_print_meta: f_norm_eps       = 0.0e+00

llm_load_print_meta: f_norm_rms_eps   = 1.0e-05

llm_load_print_meta: f_clamp_kqv      = 0.0e+00

llm_load_print_meta: f_max_alibi_bias = 0.0e+00

llm_load_print_meta: n_ff             = 14336

llm_load_print_meta: n_expert         = 0

llm_load_print_meta: n_expert_used    = 0

llm_load_print_meta: pooling type     = 0

llm_load_print_meta: rope type        = 0

llm_load_print_meta: rope scaling     = linear

llm_load_print_meta: freq_base_train  = 1000000.0

llm_load_print_meta: freq_scale_train = 1

llm_load_print_meta: n_yarn_orig_ctx  = 32768

llm_load_print_meta: rope_finetuned   = unknown

llm_load_print_meta: model type       = 7B

llm_load_print_meta: model ftype      = Q4_0

llm_load_print_meta: model params     = 7.24 B

llm_load_print_meta: model size       = 3.83 GiB (4.54 BPW) 

llm_load_print_meta: general.name     = mistralai

llm_load_print_meta: BOS token        = 1 '<s>'

llm_load_print_meta: EOS token        = 2 '</s>'

llm_load_print_meta: UNK token        = 0 '<unk>'

llm_load_print_meta: LF token         = 13 '<0x0A>'

llm_load_tensors: ggml ctx size =    0.22 MiB

llm_load_tensors: offloading 32 repeating layers to GPU

llm_load_tensors: offloading non-repeating layers to GPU

llm_load_tensors: offloaded 33/33 layers to GPU

llm_load_tensors:      ROCm0 buffer size =  3847.55 MiB

llm_load_tensors:        CPU buffer size =    70.31 MiB

..................................................................................................

llama_new_context_with_model: n_ctx      = 2048

llama_new_context_with_model: freq_base  = 1000000.0

llama_new_context_with_model: freq_scale = 1

llama_kv_cache_init:      ROCm0 KV buffer size =   256.00 MiB

llama_new_context_with_model: KV self size  =  256.00 MiB, K (f16):  128.00 MiB, V (f16):  128.00 MiB

llama_new_context_with_model:  ROCm_Host input buffer size   =    13.02 MiB

llama_new_context_with_model:      ROCm0 compute buffer size =   164.00 MiB

llama_new_context_with_model:  ROCm_Host compute buffer size =     8.00 MiB

llama_new_context_with_model: graph splits (measure): 2

time=2024-03-09T10:13:59.231+08:00 level=INFO source=dyn_ext_server.go:162 msg="Starting llama main loop"

[GIN] 2024/03/09 - 10:13:59 | 200 |    4.8541149s |       127.0.0.1 | POST     "/api/chat"

[GIN] 2024/03/09 - 10:14:04 | 200 |    909.5642ms |       127.0.0.1 | POST     "/api/chat"

[GIN] 2024/03/09 - 10:14:40 | 200 |            0s |       127.0.0.1 | HEAD     "/"

[GIN] 2024/03/09 - 10:14:40 | 200 |      1.0474ms |       127.0.0.1 | POST     "/api/show"

[GIN] 2024/03/09 - 10:14:40 | 200 |       1.031ms |       127.0.0.1 | POST     "/api/show"

time=2024-03-09T10:14:40.272+08:00 level=INFO source=routes.go:76 msg="changing loaded model"

time=2024-03-09T10:14:40.952+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"

time=2024-03-09T10:14:40.952+08:00 level=INFO source=amd_windows.go:40 msg="AMD Driver: 50732000"

time=2024-03-09T10:14:40.953+08:00 level=INFO source=amd_windows.go:69 msg="detected 1 hip devices"

time=2024-03-09T10:14:40.953+08:00 level=INFO source=amd_windows.go:87 msg="[0] Name: AMD Radeon RX 7900 XTX"

time=2024-03-09T10:14:40.953+08:00 level=INFO source=amd_windows.go:90 msg="[0] GcnArchName: gfx1100"

time=2024-03-09T10:14:40.953+08:00 level=INFO source=amd_windows.go:106 msg="amdgpu [0] gfx1100 is supported"

time=2024-03-09T10:14:40.953+08:00 level=INFO source=amd_windows.go:117 msg="[0] Total Mem: 24946528256"

time=2024-03-09T10:14:40.953+08:00 level=INFO source=amd_windows.go:118 msg="[0] Free Mem:  25753026560"

time=2024-03-09T10:14:40.953+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"

time=2024-03-09T10:14:40.954+08:00 level=INFO source=amd_windows.go:40 msg="AMD Driver: 50732000"

time=2024-03-09T10:14:40.954+08:00 level=INFO source=amd_windows.go:69 msg="detected 1 hip devices"

time=2024-03-09T10:14:40.955+08:00 level=INFO source=amd_windows.go:87 msg="[0] Name: AMD Radeon RX 7900 XTX"

time=2024-03-09T10:14:40.955+08:00 level=INFO source=amd_windows.go:90 msg="[0] GcnArchName: gfx1100"

time=2024-03-09T10:14:40.955+08:00 level=INFO source=amd_windows.go:106 msg="amdgpu [0] gfx1100 is supported"

time=2024-03-09T10:14:40.955+08:00 level=INFO source=amd_windows.go:117 msg="[0] Total Mem: 24946528256"

time=2024-03-09T10:14:40.955+08:00 level=INFO source=amd_windows.go:118 msg="[0] Free Mem:  25753026560"

time=2024-03-09T10:14:40.955+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"

time=2024-03-09T10:14:40.955+08:00 level=INFO source=dyn_ext_server.go:90 msg="Loading Dynamic llm server: C:\\Users\\user\\AppData\\Local\\Temp\\ollama\\payloads\\rocm_v5.7\\ext_server.dll"

time=2024-03-09T10:14:40.955+08:00 level=INFO source=dyn_ext_server.go:150 msg="Initializing llama server"

llama_model_loader: loaded meta data with 26 key-value pairs and 995 tensors from D:\ollama-models\blobs\sha256-e9e56e8bb5f0fcd4860675e6837a8f6a94e659f5fa7dce6a1076279336320f2b (version GGUF V3 (latest))

llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.

llama_model_loader: - kv   0:                       general.architecture str              = llama

llama_model_loader: - kv   1:                               general.name str              = mistralai

llama_model_loader: - kv   2:                       llama.context_length u32              = 32768

llama_model_loader: - kv   3:                     llama.embedding_length u32              = 4096

llama_model_loader: - kv   4:                          llama.block_count u32              = 32

llama_model_loader: - kv   5:                  llama.feed_forward_length u32              = 14336

llama_model_loader: - kv   6:                 llama.rope.dimension_count u32              = 128

llama_model_loader: - kv   7:                 llama.attention.head_count u32              = 32

llama_model_loader: - kv   8:              llama.attention.head_count_kv u32              = 8

llama_model_loader: - kv   9:                         llama.expert_count u32              = 8

llama_model_loader: - kv  10:                    llama.expert_used_count u32              = 2

llama_model_loader: - kv  11:     llama.attention.layer_norm_rms_epsilon f32              = 0.000010

llama_model_loader: - kv  12:                       llama.rope.freq_base f32              = 1000000.000000

llama_model_loader: - kv  13:                          general.file_type u32              = 2

llama_model_loader: - kv  14:                       tokenizer.ggml.model str              = llama

llama_model_loader: - kv  15:                      tokenizer.ggml.tokens arr[str,32000]   = ["<unk>", "<s>", "</s>", "<0x00>", "<...

llama_model_loader: - kv  16:                      tokenizer.ggml.scores arr[f32,32000]   = [0.000000, 0.000000, 0.000000, 0.0000...

llama_model_loader: - kv  17:                  tokenizer.ggml.token_type arr[i32,32000]   = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ...

llama_model_loader: - kv  18:                      tokenizer.ggml.merges arr[str,58980]   = ["▁ t", "i n", "e r", "▁ a", "h e...

llama_model_loader: - kv  19:                tokenizer.ggml.bos_token_id u32              = 1

llama_model_loader: - kv  20:                tokenizer.ggml.eos_token_id u32              = 2

llama_model_loader: - kv  21:            tokenizer.ggml.unknown_token_id u32              = 0

llama_model_loader: - kv  22:               tokenizer.ggml.add_bos_token bool             = true

llama_model_loader: - kv  23:               tokenizer.ggml.add_eos_token bool             = false

llama_model_loader: - kv  24:                    tokenizer.chat_template str              = {{ bos_token }}{% for message in mess...

llama_model_loader: - kv  25:               general.quantization_version u32              = 2

llama_model_loader: - type  f32:   65 tensors

llama_model_loader: - type  f16:   32 tensors

llama_model_loader: - type q4_0:  833 tensors

llama_model_loader: - type q8_0:   64 tensors

llama_model_loader: - type q6_K:    1 tensors

llm_load_vocab: special tokens definition check successful ( 259/32000 ).

llm_load_print_meta: format           = GGUF V3 (latest)

llm_load_print_meta: arch             = llama

llm_load_print_meta: vocab type       = SPM

llm_load_print_meta: n_vocab          = 32000

llm_load_print_meta: n_merges         = 0

llm_load_print_meta: n_ctx_train      = 32768

llm_load_print_meta: n_embd           = 4096

llm_load_print_meta: n_head           = 32

llm_load_print_meta: n_head_kv        = 8

llm_load_print_meta: n_layer          = 32

llm_load_print_meta: n_rot            = 128

llm_load_print_meta: n_embd_head_k    = 128

llm_load_print_meta: n_embd_head_v    = 128

llm_load_print_meta: n_gqa            = 4

llm_load_print_meta: n_embd_k_gqa     = 1024

llm_load_print_meta: n_embd_v_gqa     = 1024

llm_load_print_meta: f_norm_eps       = 0.0e+00

llm_load_print_meta: f_norm_rms_eps   = 1.0e-05

llm_load_print_meta: f_clamp_kqv      = 0.0e+00

llm_load_print_meta: f_max_alibi_bias = 0.0e+00

llm_load_print_meta: n_ff             = 14336

llm_load_print_meta: n_expert         = 8

llm_load_print_meta: n_expert_used    = 2

llm_load_print_meta: pooling type     = 0

llm_load_print_meta: rope type        = 0

llm_load_print_meta: rope scaling     = linear

llm_load_print_meta: freq_base_train  = 1000000.0

llm_load_print_meta: freq_scale_train = 1

llm_load_print_meta: n_yarn_orig_ctx  = 32768

llm_load_print_meta: rope_finetuned   = unknown

llm_load_print_meta: model type       = 7B

llm_load_print_meta: model ftype      = Q4_0

llm_load_print_meta: model params     = 46.70 B

llm_load_print_meta: model size       = 24.62 GiB (4.53 BPW) 

llm_load_print_meta: general.name     = mistralai

llm_load_print_meta: BOS token        = 1 '<s>'

llm_load_print_meta: EOS token        = 2 '</s>'

llm_load_print_meta: UNK token        = 0 '<unk>'

llm_load_print_meta: LF token         = 13 '<0x0A>'

llm_load_tensors: ggml ctx size =    0.76 MiB

Exception 0xc0000005 0x1 0x10 0x7ffa12f9a390

PC=0x7ffa12f9a390

signal arrived during external code execution



runtime.cgocall(0x7ff6d5b764e0, 0xc001006760)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/cgocall.go:157 +0x3e fp=0xc001006738 sp=0xc001006700 pc=0x7ff6d50a953e

github.com/jmorganca/ollama/llm._Cfunc_dyn_llama_server_init({0x7ffa14a90000, 0x7ffa14baf250, 0x7ffa14bafd70, 0x7ffa14bafe20, 0x7ffa14bb0240, 0x7ffa14bb0470, 0x7ffa14bb14b0, 0x7ffa14bb1490, 0x7ffa14bb16e0, 0x7ffa14bb1d00, ...}, ...)

	_cgo_gotypes.go:282 +0x4d fp=0xc001006760 sp=0xc001006738 pc=0x7ff6d599b0ed

github.com/jmorganca/ollama/llm.newDynExtServer.func7(0xc00009c0f0, 0xc000be80d8)

	C:/Users/danie/code/ollama/llm/dyn_ext_server.go:154 +0x112 fp=0xc0010068a0 sp=0xc001006760 pc=0x7ff6d599c8b2

github.com/jmorganca/ollama/llm.newDynExtServer({0xc00050e180, 0x4b}, {0xc0004d8360, _}, {_, _, _}, {0x0, 0x0, 0x0}, ...)

	C:/Users/danie/code/ollama/llm/dyn_ext_server.go:154 +0xb50 fp=0xc001006ae8 sp=0xc0010068a0 pc=0x7ff6d599c4f0

github.com/jmorganca/ollama/llm.newLlmServer({{_, _, _}, {_, _}, {_, _}}, {_, _}, {0x0, ...}, ...)

	C:/Users/danie/code/ollama/llm/llm.go:158 +0x4c5 fp=0xc001006ca8 sp=0xc001006ae8 pc=0x7ff6d5998905

github.com/jmorganca/ollama/llm.New({0xc0004d8360, 0x5e}, {0x0, 0x0, 0x0}, {0x0, _, _}, {{0x0, 0x800, ...}, ...})

	C:/Users/danie/code/ollama/llm/llm.go:123 +0x76e fp=0xc001006f18 sp=0xc001006ca8 pc=0x7ff6d59981ce

github.com/jmorganca/ollama/server.load(0xc0000dc180?, 0xc0000dc180, {{0x0, 0x800, 0x200, 0x1, 0xffffffffffffffff, 0x0, 0x0, 0x1, ...}, ...}, ...)

	C:/Users/danie/code/ollama/server/routes.go:83 +0x325 fp=0xc001007068 sp=0xc001006f18 pc=0x7ff6d5b496e5

github.com/jmorganca/ollama/server.ChatHandler(0xc0000a2000)

	C:/Users/danie/code/ollama/server/routes.go:1173 +0xa37 fp=0xc001007770 sp=0xc001007068 pc=0x7ff6d5b54e57

github.com/gin-gonic/gin.(*Context).Next(...)

	C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174

github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func1(0xc0000a2000)

	C:/Users/danie/code/ollama/server/routes.go:943 +0x68 fp=0xc0010077a8 sp=0xc001007770 pc=0x7ff6d5b53628

github.com/gin-gonic/gin.(*Context).Next(...)

	C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174

github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1(0xc0000a2000)

	C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/recovery.go:102 +0x7a fp=0xc0010077f8 sp=0xc0010077a8 pc=0x7ff6d5b29eba

github.com/gin-gonic/gin.(*Context).Next(...)

	C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174

github.com/gin-gonic/gin.LoggerWithConfig.func1(0xc0000a2000)

	C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/logger.go:240 +0xdd fp=0xc0010079a8 sp=0xc0010077f8 pc=0x7ff6d5b28ffd

github.com/gin-gonic/gin.(*Context).Next(...)

	C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174

github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc000168ea0, 0xc0000a2000)

	C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/gin.go:620 +0x66e fp=0xc001007b28 sp=0xc0010079a8 pc=0x7ff6d5b2852e

github.com/gin-gonic/gin.(*Engine).ServeHTTP(0xc000168ea0, {0x7ff6d9539dd0, 0xc00009e0e0}, 0xc0000b45a0)

	C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/gin.go:576 +0x1b2 fp=0xc001007b60 sp=0xc001007b28 pc=0x7ff6d5b27cf2

net/http.serverHandler.ServeHTTP({0x7ff6d9537c48?}, {0x7ff6d9539dd0?, 0xc00009e0e0?}, 0x6?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:3137 +0x8e fp=0xc001007b90 sp=0xc001007b60 pc=0x7ff6d53aee2e

net/http.(*conn).serve(0xc0000e66c0, {0x7ff6d953c1c8, 0xc0001b8660})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:2039 +0x5e8 fp=0xc001007fb8 sp=0xc001007b90 pc=0x7ff6d53aa1e8

net/http.(*Server).Serve.gowrap3()

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:3285 +0x28 fp=0xc001007fe0 sp=0xc001007fb8 pc=0x7ff6d53af648

runtime.goexit({})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc001007fe8 sp=0xc001007fe0 pc=0x7ff6d5112901

created by net/http.(*Server).Serve in goroutine 1

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:3285 +0x4b4



goroutine 1 gp=0xc00007e000 m=nil [IO wait]:

runtime.gopark(0xc000600008?, 0x7ff6d5bfdb00?, 0x20?, 0x2a?, 0xc0005b2a50?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc0006a76f8 sp=0xc0006a76d8 pc=0x7ff6d50e1aae

runtime.netpollblock(0x1b4?, 0xd50a9046?, 0xf6?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/netpoll.go:573 +0xf7 fp=0xc0006a7730 sp=0xc0006a76f8 pc=0x7ff6d50d9357

internal/poll.runtime_pollWait(0x20177cadba0, 0x72)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/netpoll.go:345 +0x85 fp=0xc0006a7750 sp=0xc0006a7730 pc=0x7ff6d510c3a5

internal/poll.(*pollDesc).wait(0x7ff6d50c07b6?, 0x7ff6d9c800a0?, 0x0)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0006a7778 sp=0xc0006a7750 pc=0x7ff6d51b0647

internal/poll.execIO(0xc0005b2a20, 0xc0006a7818)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_windows.go:175 +0xe6 fp=0xc0006a77e8 sp=0xc0006a7778 pc=0x7ff6d51b1b26

internal/poll.(*FD).acceptOne(0xc0005b2a08, 0x268, {0xc00000c0f0?, 0x0?, 0x0?}, 0xc000600008?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_windows.go:944 +0x67 fp=0xc0006a7848 sp=0xc0006a77e8 pc=0x7ff6d51b61e7

internal/poll.(*FD).Accept(0xc0005b2a08, 0xc0006a79f8)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_windows.go:978 +0x1bc fp=0xc0006a7900 sp=0xc0006a7848 pc=0x7ff6d51b651c

net.(*netFD).accept(0xc0005b2a08)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/fd_windows.go:178 +0x54 fp=0xc0006a7a18 sp=0xc0006a7900 pc=0x7ff6d52476f4

net.(*TCPListener).accept(0xc0004405c0)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/tcpsock_posix.go:159 +0x1e fp=0xc0006a7a40 sp=0xc0006a7a18 pc=0x7ff6d525cb9e

net.(*TCPListener).Accept(0xc0004405c0)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/tcpsock.go:327 +0x30 fp=0xc0006a7a70 sp=0xc0006a7a40 pc=0x7ff6d525bc90

net/http.(*onceCloseListener).Accept(0xc0000e66c0?)

	<autogenerated>:1 +0x24 fp=0xc0006a7a88 sp=0xc0006a7a70 pc=0x7ff6d53d1804

net/http.(*Server).Serve(0xc0000c04b0, {0x7ff6d9539bc0, 0xc0004405c0})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:3255 +0x33e fp=0xc0006a7bb8 sp=0xc0006a7a88 pc=0x7ff6d53af25e

github.com/jmorganca/ollama/server.Serve({0x7ff6d9539bc0, 0xc0004405c0})

	C:/Users/danie/code/ollama/server/routes.go:1046 +0x468 fp=0xc0006a7cc0 sp=0xc0006a7bb8 pc=0x7ff6d5b53ae8

github.com/jmorganca/ollama/cmd.RunServer(0xc0001e6c00?, {0x7ff6d9c7e340?, 0x4?, 0x7ff6d5db3a7e?})

	C:/Users/danie/code/ollama/cmd/cmd.go:787 +0x1b9 fp=0xc0006a7d58 sp=0xc0006a7cc0 pc=0x7ff6d5b6d9b9

github.com/spf13/cobra.(*Command).execute(0xc0000b8f08, {0x7ff6d9c7e340, 0x0, 0x0})

	C:/Users/danie/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:940 +0x882 fp=0xc0006a7e78 sp=0xc0006a7d58 pc=0x7ff6d544afe2

github.com/spf13/cobra.(*Command).ExecuteC(0xc0000b8308)

	C:/Users/danie/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3a5 fp=0xc0006a7f30 sp=0xc0006a7e78 pc=0x7ff6d544b825

github.com/spf13/cobra.(*Command).Execute(...)

	C:/Users/danie/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:992

github.com/spf13/cobra.(*Command).ExecuteContext(...)

	C:/Users/danie/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:985

main.main()

	C:/Users/danie/code/ollama/main.go:11 +0x4d fp=0xc0006a7f50 sp=0xc0006a7f30 pc=0x7ff6d5b7624d

runtime.main()

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:271 +0x28b fp=0xc0006a7fe0 sp=0xc0006a7f50 pc=0x7ff6d50e16ab

runtime.goexit({})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0006a7fe8 sp=0xc0006a7fe0 pc=0x7ff6d5112901



goroutine 2 gp=0xc00007e700 m=nil [force gc (idle)]:

runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000081fa8 sp=0xc000081f88 pc=0x7ff6d50e1aae

runtime.goparkunlock(...)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:408

runtime.forcegchelper()

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:326 +0xb8 fp=0xc000081fe0 sp=0xc000081fa8 pc=0x7ff6d50e1938

runtime.goexit({})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000081fe8 sp=0xc000081fe0 pc=0x7ff6d5112901

created by runtime.init.6 in goroutine 1

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:314 +0x1a



goroutine 3 gp=0xc00007ea80 m=nil [GC sweep wait]:

runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000083f80 sp=0xc000083f60 pc=0x7ff6d50e1aae

runtime.goparkunlock(...)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:408

runtime.bgsweep(0xc00008e000)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgcsweep.go:318 +0xdf fp=0xc000083fc8 sp=0xc000083f80 pc=0x7ff6d50cbb5f

runtime.gcenable.gowrap1()

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:203 +0x25 fp=0xc000083fe0 sp=0xc000083fc8 pc=0x7ff6d50c0405

runtime.goexit({})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000083fe8 sp=0xc000083fe0 pc=0x7ff6d5112901

created by runtime.gcenable in goroutine 1

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:203 +0x66



goroutine 4 gp=0xc00007ec40 m=nil [GC scavenge wait]:

runtime.gopark(0x991e0c?, 0x86fc3e?, 0x0?, 0x0?, 0x0?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000095f78 sp=0xc000095f58 pc=0x7ff6d50e1aae

runtime.goparkunlock(...)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:408

runtime.(*scavengerState).park(0x7ff6d9bf3020)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc000095fa8 sp=0xc000095f78 pc=0x7ff6d50c94e9

runtime.bgscavenge(0xc00008e000)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgcscavenge.go:658 +0x59 fp=0xc000095fc8 sp=0xc000095fa8 pc=0x7ff6d50c9a99

runtime.gcenable.gowrap2()

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:204 +0x25 fp=0xc000095fe0 sp=0xc000095fc8 pc=0x7ff6d50c03a5

runtime.goexit({})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000095fe8 sp=0xc000095fe0 pc=0x7ff6d5112901

created by runtime.gcenable in goroutine 1

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:204 +0xa5



goroutine 18 gp=0xc000104380 m=nil [finalizer wait]:

runtime.gopark(0xc000085e48?, 0x7ff6d50b37c5?, 0xa8?, 0x1?, 0xc00007e000?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000085e20 sp=0xc000085e00 pc=0x7ff6d50e1aae

runtime.runfinq()

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mfinal.go:194 +0x107 fp=0xc000085fe0 sp=0xc000085e20 pc=0x7ff6d50bf487

runtime.goexit({})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000085fe8 sp=0xc000085fe0 pc=0x7ff6d5112901

created by runtime.createfing in goroutine 1

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mfinal.go:164 +0x3d



goroutine 19 gp=0xc000105880 m=nil [GC worker (idle)]:

runtime.gopark(0x7ff6d9c800a0?, 0x1?, 0x0?, 0x0?, 0x0?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000091f50 sp=0xc000091f30 pc=0x7ff6d50e1aae

runtime.gcBgMarkWorker()

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000091fe0 sp=0xc000091f50 pc=0x7ff6d50c2545

runtime.goexit({})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000091fe8 sp=0xc000091fe0 pc=0x7ff6d5112901

created by runtime.gcBgMarkStartWorkers in goroutine 1

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c



goroutine 34 gp=0xc000480000 m=nil [GC worker (idle)]:

runtime.gopark(0x3135b6217c2c?, 0x1?, 0x0?, 0x0?, 0x0?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000487f50 sp=0xc000487f30 pc=0x7ff6d50e1aae

runtime.gcBgMarkWorker()

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000487fe0 sp=0xc000487f50 pc=0x7ff6d50c2545

runtime.goexit({})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000487fe8 sp=0xc000487fe0 pc=0x7ff6d5112901

created by runtime.gcBgMarkStartWorkers in goroutine 1

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c



goroutine 5 gp=0xc00007efc0 m=nil [GC worker (idle)]:

runtime.gopark(0x7ff6d9c800a0?, 0x3?, 0x0?, 0x0?, 0x0?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000097f50 sp=0xc000097f30 pc=0x7ff6d50e1aae

runtime.gcBgMarkWorker()

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000097fe0 sp=0xc000097f50 pc=0x7ff6d50c2545

runtime.goexit({})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000097fe8 sp=0xc000097fe0 pc=0x7ff6d5112901

created by runtime.gcBgMarkStartWorkers in goroutine 1

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c



goroutine 35 gp=0xc0004801c0 m=nil [GC worker (idle)]:

runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000489f50 sp=0xc000489f30 pc=0x7ff6d50e1aae

runtime.gcBgMarkWorker()

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000489fe0 sp=0xc000489f50 pc=0x7ff6d50c2545

runtime.goexit({})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000489fe8 sp=0xc000489fe0 pc=0x7ff6d5112901

created by runtime.gcBgMarkStartWorkers in goroutine 1

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c



goroutine 6 gp=0xc00007f180 m=nil [GC worker (idle)]:

runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000483f50 sp=0xc000483f30 pc=0x7ff6d50e1aae

runtime.gcBgMarkWorker()

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000483fe0 sp=0xc000483f50 pc=0x7ff6d50c2545

runtime.goexit({})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000483fe8 sp=0xc000483fe0 pc=0x7ff6d5112901

created by runtime.gcBgMarkStartWorkers in goroutine 1

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c



goroutine 36 gp=0xc000480380 m=nil [GC worker (idle)]:

runtime.gopark(0x3135b6217c2c?, 0x1?, 0x0?, 0x0?, 0x0?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000491f50 sp=0xc000491f30 pc=0x7ff6d50e1aae

runtime.gcBgMarkWorker()

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000491fe0 sp=0xc000491f50 pc=0x7ff6d50c2545

runtime.goexit({})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000491fe8 sp=0xc000491fe0 pc=0x7ff6d5112901

created by runtime.gcBgMarkStartWorkers in goroutine 1

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c



goroutine 7 gp=0xc00007f340 m=nil [GC worker (idle)]:

runtime.gopark(0x3135b6217c2c?, 0x3?, 0x1c?, 0xfa?, 0x0?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000485f50 sp=0xc000485f30 pc=0x7ff6d50e1aae

runtime.gcBgMarkWorker()

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000485fe0 sp=0xc000485f50 pc=0x7ff6d50c2545

runtime.goexit({})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000485fe8 sp=0xc000485fe0 pc=0x7ff6d5112901

created by runtime.gcBgMarkStartWorkers in goroutine 1

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c



goroutine 37 gp=0xc000480540 m=nil [GC worker (idle)]:

runtime.gopark(0x7ff6d9c800a0?, 0x3?, 0x40?, 0xbd?, 0x0?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000493f50 sp=0xc000493f30 pc=0x7ff6d50e1aae

runtime.gcBgMarkWorker()

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000493fe0 sp=0xc000493f50 pc=0x7ff6d50c2545

runtime.goexit({})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000493fe8 sp=0xc000493fe0 pc=0x7ff6d5112901

created by runtime.gcBgMarkStartWorkers in goroutine 1

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c



goroutine 8 gp=0xc00007f500 m=nil [GC worker (idle)]:

runtime.gopark(0x3135b6217c2c?, 0x1?, 0x0?, 0x0?, 0x0?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc00048df50 sp=0xc00048df30 pc=0x7ff6d50e1aae

runtime.gcBgMarkWorker()

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc00048dfe0 sp=0xc00048df50 pc=0x7ff6d50c2545

runtime.goexit({})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00048dfe8 sp=0xc00048dfe0 pc=0x7ff6d5112901

created by runtime.gcBgMarkStartWorkers in goroutine 1

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c



goroutine 38 gp=0xc000480700 m=nil [GC worker (idle)]:

runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000499f50 sp=0xc000499f30 pc=0x7ff6d50e1aae

runtime.gcBgMarkWorker()

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000499fe0 sp=0xc000499f50 pc=0x7ff6d50c2545

runtime.goexit({})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000499fe8 sp=0xc000499fe0 pc=0x7ff6d5112901

created by runtime.gcBgMarkStartWorkers in goroutine 1

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c



goroutine 9 gp=0xc00007f6c0 m=nil [GC worker (idle)]:

runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc00048ff50 sp=0xc00048ff30 pc=0x7ff6d50e1aae

runtime.gcBgMarkWorker()

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc00048ffe0 sp=0xc00048ff50 pc=0x7ff6d50c2545

runtime.goexit({})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00048ffe8 sp=0xc00048ffe0 pc=0x7ff6d5112901

created by runtime.gcBgMarkStartWorkers in goroutine 1

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c



goroutine 39 gp=0xc0004808c0 m=nil [GC worker (idle)]:

runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc00049bf50 sp=0xc00049bf30 pc=0x7ff6d50e1aae

runtime.gcBgMarkWorker()

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc00049bfe0 sp=0xc00049bf50 pc=0x7ff6d50c2545

runtime.goexit({})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00049bfe8 sp=0xc00049bfe0 pc=0x7ff6d5112901

created by runtime.gcBgMarkStartWorkers in goroutine 1

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c



goroutine 10 gp=0xc00007f880 m=nil [GC worker (idle)]:

runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000495f50 sp=0xc000495f30 pc=0x7ff6d50e1aae

runtime.gcBgMarkWorker()

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000495fe0 sp=0xc000495f50 pc=0x7ff6d50c2545

runtime.goexit({})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000495fe8 sp=0xc000495fe0 pc=0x7ff6d5112901

created by runtime.gcBgMarkStartWorkers in goroutine 1

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c



goroutine 40 gp=0xc000480a80 m=nil [GC worker (idle)]:

runtime.gopark(0xc000059c50?, 0x1?, 0x2c?, 0x7c?, 0x0?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc0004a1f50 sp=0xc0004a1f30 pc=0x7ff6d50e1aae

runtime.gcBgMarkWorker()

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc0004a1fe0 sp=0xc0004a1f50 pc=0x7ff6d50c2545

runtime.goexit({})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0004a1fe8 sp=0xc0004a1fe0 pc=0x7ff6d5112901

created by runtime.gcBgMarkStartWorkers in goroutine 1

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c



goroutine 11 gp=0xc00007fa40 m=nil [GC worker (idle)]:

runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000497f50 sp=0xc000497f30 pc=0x7ff6d50e1aae

runtime.gcBgMarkWorker()

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000497fe0 sp=0xc000497f50 pc=0x7ff6d50c2545

runtime.goexit({})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000497fe8 sp=0xc000497fe0 pc=0x7ff6d5112901

created by runtime.gcBgMarkStartWorkers in goroutine 1

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c



goroutine 41 gp=0xc000480c40 m=nil [GC worker (idle)]:

runtime.gopark(0x3135b629396c?, 0x3?, 0x0?, 0x0?, 0x0?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc0004a3f50 sp=0xc0004a3f30 pc=0x7ff6d50e1aae

runtime.gcBgMarkWorker()

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc0004a3fe0 sp=0xc0004a3f50 pc=0x7ff6d50c2545

runtime.goexit({})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0004a3fe8 sp=0xc0004a3fe0 pc=0x7ff6d5112901

created by runtime.gcBgMarkStartWorkers in goroutine 1

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c



goroutine 12 gp=0xc000481180 m=4 mp=0xc00008b808 [syscall]:

runtime.notetsleepg(0x7ff6d9c7eee0, 0xffffffffffffffff)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/lock_sema.go:296 +0x31 fp=0xc00049ffa0 sp=0xc00049ff68 pc=0x7ff6d50b1d91

os/signal.signal_recv()

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/sigqueue.go:152 +0x29 fp=0xc00049ffc0 sp=0xc00049ffa0 pc=0x7ff6d510e5e9

os/signal.loop()

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/os/signal/signal_unix.go:23 +0x13 fp=0xc00049ffe0 sp=0xc00049ffc0 pc=0x7ff6d53d3bb3

runtime.goexit({})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00049ffe8 sp=0xc00049ffe0 pc=0x7ff6d5112901

created by os/signal.Notify.func1.1 in goroutine 1

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/os/signal/signal.go:151 +0x1f



goroutine 13 gp=0xc000481340 m=nil [chan receive]:

runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000093f18 sp=0xc000093ef8 pc=0x7ff6d50e1aae

runtime.chanrecv(0xc0001eafc0, 0x0, 0x1)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/chan.go:583 +0x3cd fp=0xc000093f90 sp=0xc000093f18 pc=0x7ff6d50abbcd

runtime.chanrecv1(0x0?, 0x0?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/chan.go:442 +0x12 fp=0xc000093fb8 sp=0xc000093f90 pc=0x7ff6d50ab7d2

github.com/jmorganca/ollama/server.Serve.func2()

	C:/Users/danie/code/ollama/server/routes.go:1028 +0x25 fp=0xc000093fe0 sp=0xc000093fb8 pc=0x7ff6d5b53b85

runtime.goexit({})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000093fe8 sp=0xc000093fe0 pc=0x7ff6d5112901

created by github.com/jmorganca/ollama/server.Serve in goroutine 1

	C:/Users/danie/code/ollama/server/routes.go:1027 +0x3f6



goroutine 83 gp=0xc0005048c0 m=nil [IO wait]:

runtime.gopark(0x0?, 0xc000698020?, 0xd0?, 0x80?, 0xc000698050?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc0004cbd28 sp=0xc0004cbd08 pc=0x7ff6d50e1aae

runtime.netpollblock(0x280?, 0xd50a9046?, 0xf6?)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/netpoll.go:573 +0xf7 fp=0xc0004cbd60 sp=0xc0004cbd28 pc=0x7ff6d50d9357

internal/poll.runtime_pollWait(0x20177cadaa8, 0x72)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/netpoll.go:345 +0x85 fp=0xc0004cbd80 sp=0xc0004cbd60 pc=0x7ff6d510c3a5

internal/poll.(*pollDesc).wait(0x10?, 0x10?, 0x0)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0004cbda8 sp=0xc0004cbd80 pc=0x7ff6d51b0647

internal/poll.execIO(0xc000698020, 0x7ff6d9419910)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_windows.go:175 +0xe6 fp=0xc0004cbe18 sp=0xc0004cbda8 pc=0x7ff6d51b1b26

internal/poll.(*FD).Read(0xc000698008, {0xc000f0c641, 0x1, 0x1})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_windows.go:436 +0x2b1 fp=0xc0004cbec0 sp=0xc0004cbe18 pc=0x7ff6d51b27d1

net.(*netFD).Read(0xc000698008, {0xc000f0c641?, 0xc0004cbf48?, 0x7ff6d510e330?})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/fd_posix.go:55 +0x25 fp=0xc0004cbf08 sp=0xc0004cbec0 pc=0x7ff6d5245805

net.(*conn).Read(0xc000110040, {0xc000f0c641?, 0xc0004cbeb8?, 0x7ff6d9c7e340?})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/net.go:179 +0x45 fp=0xc0004cbf50 sp=0xc0004cbf08 pc=0x7ff6d5254885

net.(*TCPConn).Read(0x7ff6d5bd7aa0?, {0xc000f0c641?, 0xc0000983c0?, 0x7ff6d54a25a0?})

	<autogenerated>:1 +0x25 fp=0xc0004cbf80 sp=0xc0004cbf50 pc=0x7ff6d52648a5

net/http.(*connReader).backgroundRead(0xc000f0c630)

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:681 +0x37 fp=0xc0004cbfc8 sp=0xc0004cbf80 pc=0x7ff6d53a4157

net/http.(*connReader).startBackgroundRead.gowrap2()

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:677 +0x25 fp=0xc0004cbfe0 sp=0xc0004cbfc8 pc=0x7ff6d53a4085

runtime.goexit({})

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0004cbfe8 sp=0xc0004cbfe0 pc=0x7ff6d5112901

created by net/http.(*connReader).startBackgroundRead in goroutine 21

	C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:677 +0xba

rax     0x0

rbx     0x2012414aa68

rcx     0x2012414aa18

rdx     0xffffffdd00000000

rdi     0x2012414aa68

rsi     0x2012414aa18

rbp     0x0

rsp     0x8f23ffd7b0

r8      0x2012414aa68

r9      0x0

r10     0x96

r11     0x246

r12     0x0

r13     0x7ffa12a30000

r14     0x2014e56d9c0

r15     0x8f23ffd970

rip     0x7ffa12f9a390

rflags  0x10246

cs      0x33

fs      0x53

gs      0x2b

{"function":"initialize","level":"INFO","line":688,"msg":"initializing slots","n_slots":1,"tid":"19848","timestamp":1709950439}

{"function":"initialize","id_slot":0,"level":"INFO","line":699,"msg":"new slot","n_ctx_slot":2048,"tid":"19848","timestamp":1709950439}

{"function":"update_slots","level":"INFO","line":1593,"msg":"all slots are idle","tid":"5172","timestamp":1709950439}

{"function":"launch_slot_with_data","id_slot":0,"id_task":0,"level":"INFO","line":973,"msg":"slot is processing task","tid":"5172","timestamp":1709950443}

{"function":"update_slots","id_slot":0,"id_task":0,"level":"INFO","line":1845,"msg":"kv cache rm [p0, end)","p0":0,"tid":"5172","timestamp":1709950443}

{"function":"print_timings","id_slot":0,"id_task":0,"level":"INFO","line":311,"msg":"prompt eval time     =      63.01 ms /    11 tokens (    5.73 ms per token,   174.58 tokens per second)","n_prompt_tokens_processed":11,"n_tokens_second":174.58100558659217,"t_prompt_processing":63.008,"t_token":5.728000000000001,"tid":"5172","timestamp":1709950444}

{"function":"print_timings","id_slot":0,"id_task":0,"level":"INFO","line":327,"msg":"generation eval time =     845.31 ms /    95 runs   (    8.90 ms per token,   112.38 tokens per second)","n_decoded":95,"n_tokens_second":112.38480557428636,"t_token":8.898,"t_token_generation":845.31,"tid":"5172","timestamp":1709950444}

{"function":"print_timings","id_slot":0,"id_task":0,"level":"INFO","line":337,"msg":"          total time =     908.32 ms","t_prompt_processing":63.008,"t_token_generation":845.31,"t_total":908.318,"tid":"5172","timestamp":1709950444}

{"function":"update_slots","id_slot":0,"id_task":0,"level":"INFO","line":1575,"msg":"slot released","n_cache_tokens":105,"n_ctx":2048,"n_past":105,"n_system_tokens":0,"tid":"5172","timestamp":1709950444,"truncated":false}

{"function":"update_slots","level":"INFO","line":1593,"msg":"all slots are idle","tid":"5172","timestamp":1709950444}

{"function":"update_slots","level":"INFO","line":1593,"msg":"all slots are idle","tid":"5172","timestamp":1709950444}

{"function":"update_slots","level":"INFO","line":1593,"msg":"all slots are idle","tid":"5172","timestamp":1709950480}

loading library C:\Users\user\AppData\Local\Temp\ollama\payloads\rocm_v5.7\ext_server.dll

Have you fixed it?

<!-- gh-comment-id:2007697121 --> @sgwhat commented on GitHub (Mar 19, 2024): > Thanks a lot for your works. I just tried https://github.com/dhiltgen/ollama/releases for rocm support, but I found it will be failed when using mixtral model. Here is a log for this panic: > > > > ``` > > time=2024-03-09T10:13:54.011+08:00 level=INFO source=images.go:800 msg="total blobs: 8" > > time=2024-03-09T10:13:54.022+08:00 level=INFO source=images.go:807 msg="total unused blobs removed: 0" > > time=2024-03-09T10:13:54.023+08:00 level=INFO source=routes.go:1019 msg="Listening on 127.0.0.1:11434 (version 0.1.28-23-ge690faf)" > > time=2024-03-09T10:13:54.023+08:00 level=INFO source=payload_common.go:109 msg="Extracting dynamic libraries to C:\\Users\\user\\AppData\\Local\\Temp\\ollama\\payloads ..." > > time=2024-03-09T10:13:54.194+08:00 level=INFO source=payload_common.go:154 msg="Dynamic LLM libraries [rocm_v5.7 cuda_v11.3 cpu cpu_avx2 cpu_avx cuda_v12.3]" > > [GIN] 2024/03/09 - 10:13:54 | 200 | 0s | 127.0.0.1 | HEAD "/" > > [GIN] 2024/03/09 - 10:13:54 | 200 | 1.0486ms | 127.0.0.1 | POST "/api/show" > > [GIN] 2024/03/09 - 10:13:54 | 200 | 525.7µs | 127.0.0.1 | POST "/api/show" > > time=2024-03-09T10:13:54.847+08:00 level=INFO source=gpu.go:77 msg="Detecting GPU type" > > time=2024-03-09T10:13:54.847+08:00 level=INFO source=gpu.go:191 msg="Searching for GPU management library nvml.dll" > > time=2024-03-09T10:13:54.858+08:00 level=INFO source=gpu.go:237 msg="Discovered GPU libraries: []" > > time=2024-03-09T10:13:54.858+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" > > time=2024-03-09T10:13:54.875+08:00 level=INFO source=amd_windows.go:40 msg="AMD Driver: 50732000" > > time=2024-03-09T10:13:54.877+08:00 level=INFO source=amd_windows.go:69 msg="detected 1 hip devices" > > time=2024-03-09T10:13:54.877+08:00 level=INFO source=amd_windows.go:87 msg="[0] Name: AMD Radeon RX 7900 XTX" > > time=2024-03-09T10:13:54.877+08:00 level=INFO source=amd_windows.go:90 msg="[0] GcnArchName: gfx1100" > > time=2024-03-09T10:13:54.877+08:00 level=INFO source=amd_windows.go:106 msg="amdgpu [0] gfx1100 is supported" > > time=2024-03-09T10:13:55.129+08:00 level=INFO source=amd_windows.go:117 msg="[0] Total Mem: 25618808832" > > time=2024-03-09T10:13:55.129+08:00 level=INFO source=amd_windows.go:118 msg="[0] Free Mem: 25753026560" > > time=2024-03-09T10:13:55.129+08:00 level=INFO source=assets.go:36 msg="Updating PATH to C:\\Users\\user\\AppData\\Local\\Programs\\Ollama\\rocm;C:\\Python310\\Scripts\\;C:\\Python310\\;C:\\Program Files\\Zulu\\zulu-8\\bin\\;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Windows\\System32\\OpenSSH\\;C:\\Program Files\\Intel\\WiFi\\bin\\;C:\\Program Files\\Common Files\\Intel\\WirelessCommon\\;D:\\TDM-GCC-64\\bin;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\;C:\\WINDOWS\\System32\\OpenSSH\\;C:\\Program Files\\dotnet\\;C:\\ProgramData\\chocolatey\\bin;C:\\Program Files (x86)\\dotnet\\;C:\\Program Files\\nodejs\\;C:\\Program Files\\D2\\;C:\\Program Files\\Amazon\\AWSSAMCLI\\bin\\;C:\\Program Files\\Amazon\\AWSCLIV2\\;D:\\微信web开发者工具\\dll;C:\\Program Files (x86)\\Windows Kits\\10\\Windows Performance Toolkit\\;C:\\Program Files\\Docker\\Docker\\resources\\bin;C:\\Program Files\\Go\\bin;C:\\Program Files\\PowerShell\\7\\;C:\\Users\\user\\scoop\\shims;C:\\Users\\user\\AppData\\Local\\Microsoft\\WindowsApps;G:\\GOPATH\\bin;C:\\Program Files\\Intel\\WiFi\\bin\\;C:\\Program Files\\Common Files\\Intel\\WirelessCommon\\;C:\\Users\\user\\AppData\\Local\\GitHubDesktop\\bin;D:\\jdk-11\\bin;C:\\Program Files (x86)\\Graphviz2.38\\bin;C:\\Users\\user\\AppData\\Local\\Yarn\\bin;C:\\Users\\user\\.dotnet\\tools;D:\\flutter\\bin;D:\\Program Files\\Git\\usr\\bin;D:\\GitHub CLI\\;C:\\Program Files (x86)\\Nmap;C:\\Program Files\\Graphviz\\bin;C:\\Program Files\\Go\\bin;D:\\flutter\\.pub-cache\\bin;C:\\Users\\user\\go\\bin;C:\\Users\\user\\AppData\\Local\\Atlassian\\SourceTree\\git_local\\bin;C:\\Users\\user\\AppData\\Local\\JetBrains\\Toolbox\\scripts;C:\\Users\\user\\.dotnet\\tools;C:\\Users\\user\\AppData\\Roaming\\npm;C:\\Users\\user\\AppData\\Local\\Programs\\Microsoft VS Code\\bin;C:\\Users\\user\\go\\bin;C:\\Users\\user\\.detaspace\\bin;C:\\Users\\user\\.dotnet\\tools;C:\\Users\\user\\go\\bin;C:\\Users\\user\\AppData\\Local\\Programs\\Ollama" > > time=2024-03-09T10:13:55.195+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" > > time=2024-03-09T10:13:55.206+08:00 level=INFO source=amd_windows.go:40 msg="AMD Driver: 50732000" > > time=2024-03-09T10:13:55.207+08:00 level=INFO source=amd_windows.go:69 msg="detected 1 hip devices" > > time=2024-03-09T10:13:55.207+08:00 level=INFO source=amd_windows.go:87 msg="[0] Name: AMD Radeon RX 7900 XTX" > > time=2024-03-09T10:13:55.207+08:00 level=INFO source=amd_windows.go:90 msg="[0] GcnArchName: gfx1100" > > time=2024-03-09T10:13:55.207+08:00 level=INFO source=amd_windows.go:106 msg="amdgpu [0] gfx1100 is supported" > > time=2024-03-09T10:13:55.443+08:00 level=INFO source=amd_windows.go:117 msg="[0] Total Mem: 25618808832" > > time=2024-03-09T10:13:55.443+08:00 level=INFO source=amd_windows.go:118 msg="[0] Free Mem: 25753026560" > > time=2024-03-09T10:13:55.509+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" > > time=2024-03-09T10:13:55.509+08:00 level=INFO source=assets.go:36 msg="Updating PATH to C:\\Users\\user\\AppData\\Local\\Temp\\ollama\\payloads\\rocm_v5.7;C:\\Users\\user\\AppData\\Local\\Programs\\Ollama\\rocm;C:\\Python310\\Scripts\\;C:\\Python310\\;C:\\Program Files\\Zulu\\zulu-8\\bin\\;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Windows\\System32\\OpenSSH\\;C:\\Program Files\\Intel\\WiFi\\bin\\;C:\\Program Files\\Common Files\\Intel\\WirelessCommon\\;D:\\TDM-GCC-64\\bin;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\;C:\\WINDOWS\\System32\\OpenSSH\\;C:\\Program Files\\dotnet\\;C:\\ProgramData\\chocolatey\\bin;C:\\Program Files (x86)\\dotnet\\;C:\\Program Files\\nodejs\\;C:\\Program Files\\D2\\;C:\\Program Files\\Amazon\\AWSSAMCLI\\bin\\;C:\\Program Files\\Amazon\\AWSCLIV2\\;D:\\微信web开发者工具\\dll;C:\\Program Files (x86)\\Windows Kits\\10\\Windows Performance Toolkit\\;C:\\Program Files\\Docker\\Docker\\resources\\bin;C:\\Program Files\\Go\\bin;C:\\Program Files\\PowerShell\\7\\;C:\\Users\\user\\scoop\\shims;C:\\Users\\user\\AppData\\Local\\Microsoft\\WindowsApps;G:\\GOPATH\\bin;C:\\Program Files\\Intel\\WiFi\\bin\\;C:\\Program Files\\Common Files\\Intel\\WirelessCommon\\;C:\\Users\\user\\AppData\\Local\\GitHubDesktop\\bin;D:\\jdk-11\\bin;C:\\Program Files (x86)\\Graphviz2.38\\bin;C:\\Users\\user\\AppData\\Local\\Yarn\\bin;C:\\Users\\user\\.dotnet\\tools;D:\\flutter\\bin;D:\\Program Files\\Git\\usr\\bin;D:\\GitHub CLI\\;C:\\Program Files (x86)\\Nmap;C:\\Program Files\\Graphviz\\bin;C:\\Program Files\\Go\\bin;D:\\flutter\\.pub-cache\\bin;C:\\Users\\user\\go\\bin;C:\\Users\\user\\AppData\\Local\\Atlassian\\SourceTree\\git_local\\bin;C:\\Users\\user\\AppData\\Local\\JetBrains\\Toolbox\\scripts;C:\\Users\\user\\.dotnet\\tools;C:\\Users\\user\\AppData\\Roaming\\npm;C:\\Users\\user\\AppData\\Local\\Programs\\Microsoft VS Code\\bin;C:\\Users\\user\\go\\bin;C:\\Users\\user\\.detaspace\\bin;C:\\Users\\user\\.dotnet\\tools;C:\\Users\\user\\go\\bin;C:\\Users\\user\\AppData\\Local\\Programs\\Ollama" > > time=2024-03-09T10:13:55.533+08:00 level=INFO source=dyn_ext_server.go:90 msg="Loading Dynamic llm server: C:\\Users\\user\\AppData\\Local\\Temp\\ollama\\payloads\\rocm_v5.7\\ext_server.dll" > > time=2024-03-09T10:13:55.533+08:00 level=INFO source=dyn_ext_server.go:150 msg="Initializing llama server" > > ggml_init_cublas: GGML_CUDA_FORCE_MMQ: no > > ggml_init_cublas: CUDA_USE_TENSOR_CORES: yes > > ggml_init_cublas: found 1 ROCm devices: > > Device 0: AMD Radeon RX 7900 XTX, compute capability 11.0, VMM: no > > llama_model_loader: loaded meta data with 24 key-value pairs and 291 tensors from D:\ollama-models\blobs\sha256-e8a35b5937a5e6d5c35d1f2a15f161e07eefe5e5bb0a3cdd42998ee79b057730 (version GGUF V3 (latest)) > > llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. > > llama_model_loader: - kv 0: general.architecture str = llama > > llama_model_loader: - kv 1: general.name str = mistralai > > llama_model_loader: - kv 2: llama.context_length u32 = 32768 > > llama_model_loader: - kv 3: llama.embedding_length u32 = 4096 > > llama_model_loader: - kv 4: llama.block_count u32 = 32 > > llama_model_loader: - kv 5: llama.feed_forward_length u32 = 14336 > > llama_model_loader: - kv 6: llama.rope.dimension_count u32 = 128 > > llama_model_loader: - kv 7: llama.attention.head_count u32 = 32 > > llama_model_loader: - kv 8: llama.attention.head_count_kv u32 = 8 > > llama_model_loader: - kv 9: llama.attention.layer_norm_rms_epsilon f32 = 0.000010 > > llama_model_loader: - kv 10: llama.rope.freq_base f32 = 1000000.000000 > > llama_model_loader: - kv 11: general.file_type u32 = 2 > > llama_model_loader: - kv 12: tokenizer.ggml.model str = llama > > llama_model_loader: - kv 13: tokenizer.ggml.tokens arr[str,32000] = ["<unk>", "<s>", "</s>", "<0x00>", "<... > > llama_model_loader: - kv 14: tokenizer.ggml.scores arr[f32,32000] = [0.000000, 0.000000, 0.000000, 0.0000... > > llama_model_loader: - kv 15: tokenizer.ggml.token_type arr[i32,32000] = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ... > > llama_model_loader: - kv 16: tokenizer.ggml.merges arr[str,58980] = ["▁ t", "i n", "e r", "▁ a", "h e... > > llama_model_loader: - kv 17: tokenizer.ggml.bos_token_id u32 = 1 > > llama_model_loader: - kv 18: tokenizer.ggml.eos_token_id u32 = 2 > > llama_model_loader: - kv 19: tokenizer.ggml.unknown_token_id u32 = 0 > > llama_model_loader: - kv 20: tokenizer.ggml.add_bos_token bool = true > > llama_model_loader: - kv 21: tokenizer.ggml.add_eos_token bool = false > > llama_model_loader: - kv 22: tokenizer.chat_template str = {{ bos_token }}{% for message in mess... > > llama_model_loader: - kv 23: general.quantization_version u32 = 2 > > llama_model_loader: - type f32: 65 tensors > > llama_model_loader: - type q4_0: 225 tensors > > llama_model_loader: - type q6_K: 1 tensors > > llm_load_vocab: special tokens definition check successful ( 259/32000 ). > > llm_load_print_meta: format = GGUF V3 (latest) > > llm_load_print_meta: arch = llama > > llm_load_print_meta: vocab type = SPM > > llm_load_print_meta: n_vocab = 32000 > > llm_load_print_meta: n_merges = 0 > > llm_load_print_meta: n_ctx_train = 32768 > > llm_load_print_meta: n_embd = 4096 > > llm_load_print_meta: n_head = 32 > > llm_load_print_meta: n_head_kv = 8 > > llm_load_print_meta: n_layer = 32 > > llm_load_print_meta: n_rot = 128 > > llm_load_print_meta: n_embd_head_k = 128 > > llm_load_print_meta: n_embd_head_v = 128 > > llm_load_print_meta: n_gqa = 4 > > llm_load_print_meta: n_embd_k_gqa = 1024 > > llm_load_print_meta: n_embd_v_gqa = 1024 > > llm_load_print_meta: f_norm_eps = 0.0e+00 > > llm_load_print_meta: f_norm_rms_eps = 1.0e-05 > > llm_load_print_meta: f_clamp_kqv = 0.0e+00 > > llm_load_print_meta: f_max_alibi_bias = 0.0e+00 > > llm_load_print_meta: n_ff = 14336 > > llm_load_print_meta: n_expert = 0 > > llm_load_print_meta: n_expert_used = 0 > > llm_load_print_meta: pooling type = 0 > > llm_load_print_meta: rope type = 0 > > llm_load_print_meta: rope scaling = linear > > llm_load_print_meta: freq_base_train = 1000000.0 > > llm_load_print_meta: freq_scale_train = 1 > > llm_load_print_meta: n_yarn_orig_ctx = 32768 > > llm_load_print_meta: rope_finetuned = unknown > > llm_load_print_meta: model type = 7B > > llm_load_print_meta: model ftype = Q4_0 > > llm_load_print_meta: model params = 7.24 B > > llm_load_print_meta: model size = 3.83 GiB (4.54 BPW) > > llm_load_print_meta: general.name = mistralai > > llm_load_print_meta: BOS token = 1 '<s>' > > llm_load_print_meta: EOS token = 2 '</s>' > > llm_load_print_meta: UNK token = 0 '<unk>' > > llm_load_print_meta: LF token = 13 '<0x0A>' > > llm_load_tensors: ggml ctx size = 0.22 MiB > > llm_load_tensors: offloading 32 repeating layers to GPU > > llm_load_tensors: offloading non-repeating layers to GPU > > llm_load_tensors: offloaded 33/33 layers to GPU > > llm_load_tensors: ROCm0 buffer size = 3847.55 MiB > > llm_load_tensors: CPU buffer size = 70.31 MiB > > .................................................................................................. > > llama_new_context_with_model: n_ctx = 2048 > > llama_new_context_with_model: freq_base = 1000000.0 > > llama_new_context_with_model: freq_scale = 1 > > llama_kv_cache_init: ROCm0 KV buffer size = 256.00 MiB > > llama_new_context_with_model: KV self size = 256.00 MiB, K (f16): 128.00 MiB, V (f16): 128.00 MiB > > llama_new_context_with_model: ROCm_Host input buffer size = 13.02 MiB > > llama_new_context_with_model: ROCm0 compute buffer size = 164.00 MiB > > llama_new_context_with_model: ROCm_Host compute buffer size = 8.00 MiB > > llama_new_context_with_model: graph splits (measure): 2 > > time=2024-03-09T10:13:59.231+08:00 level=INFO source=dyn_ext_server.go:162 msg="Starting llama main loop" > > [GIN] 2024/03/09 - 10:13:59 | 200 | 4.8541149s | 127.0.0.1 | POST "/api/chat" > > [GIN] 2024/03/09 - 10:14:04 | 200 | 909.5642ms | 127.0.0.1 | POST "/api/chat" > > [GIN] 2024/03/09 - 10:14:40 | 200 | 0s | 127.0.0.1 | HEAD "/" > > [GIN] 2024/03/09 - 10:14:40 | 200 | 1.0474ms | 127.0.0.1 | POST "/api/show" > > [GIN] 2024/03/09 - 10:14:40 | 200 | 1.031ms | 127.0.0.1 | POST "/api/show" > > time=2024-03-09T10:14:40.272+08:00 level=INFO source=routes.go:76 msg="changing loaded model" > > time=2024-03-09T10:14:40.952+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" > > time=2024-03-09T10:14:40.952+08:00 level=INFO source=amd_windows.go:40 msg="AMD Driver: 50732000" > > time=2024-03-09T10:14:40.953+08:00 level=INFO source=amd_windows.go:69 msg="detected 1 hip devices" > > time=2024-03-09T10:14:40.953+08:00 level=INFO source=amd_windows.go:87 msg="[0] Name: AMD Radeon RX 7900 XTX" > > time=2024-03-09T10:14:40.953+08:00 level=INFO source=amd_windows.go:90 msg="[0] GcnArchName: gfx1100" > > time=2024-03-09T10:14:40.953+08:00 level=INFO source=amd_windows.go:106 msg="amdgpu [0] gfx1100 is supported" > > time=2024-03-09T10:14:40.953+08:00 level=INFO source=amd_windows.go:117 msg="[0] Total Mem: 24946528256" > > time=2024-03-09T10:14:40.953+08:00 level=INFO source=amd_windows.go:118 msg="[0] Free Mem: 25753026560" > > time=2024-03-09T10:14:40.953+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" > > time=2024-03-09T10:14:40.954+08:00 level=INFO source=amd_windows.go:40 msg="AMD Driver: 50732000" > > time=2024-03-09T10:14:40.954+08:00 level=INFO source=amd_windows.go:69 msg="detected 1 hip devices" > > time=2024-03-09T10:14:40.955+08:00 level=INFO source=amd_windows.go:87 msg="[0] Name: AMD Radeon RX 7900 XTX" > > time=2024-03-09T10:14:40.955+08:00 level=INFO source=amd_windows.go:90 msg="[0] GcnArchName: gfx1100" > > time=2024-03-09T10:14:40.955+08:00 level=INFO source=amd_windows.go:106 msg="amdgpu [0] gfx1100 is supported" > > time=2024-03-09T10:14:40.955+08:00 level=INFO source=amd_windows.go:117 msg="[0] Total Mem: 24946528256" > > time=2024-03-09T10:14:40.955+08:00 level=INFO source=amd_windows.go:118 msg="[0] Free Mem: 25753026560" > > time=2024-03-09T10:14:40.955+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" > > time=2024-03-09T10:14:40.955+08:00 level=INFO source=dyn_ext_server.go:90 msg="Loading Dynamic llm server: C:\\Users\\user\\AppData\\Local\\Temp\\ollama\\payloads\\rocm_v5.7\\ext_server.dll" > > time=2024-03-09T10:14:40.955+08:00 level=INFO source=dyn_ext_server.go:150 msg="Initializing llama server" > > llama_model_loader: loaded meta data with 26 key-value pairs and 995 tensors from D:\ollama-models\blobs\sha256-e9e56e8bb5f0fcd4860675e6837a8f6a94e659f5fa7dce6a1076279336320f2b (version GGUF V3 (latest)) > > llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. > > llama_model_loader: - kv 0: general.architecture str = llama > > llama_model_loader: - kv 1: general.name str = mistralai > > llama_model_loader: - kv 2: llama.context_length u32 = 32768 > > llama_model_loader: - kv 3: llama.embedding_length u32 = 4096 > > llama_model_loader: - kv 4: llama.block_count u32 = 32 > > llama_model_loader: - kv 5: llama.feed_forward_length u32 = 14336 > > llama_model_loader: - kv 6: llama.rope.dimension_count u32 = 128 > > llama_model_loader: - kv 7: llama.attention.head_count u32 = 32 > > llama_model_loader: - kv 8: llama.attention.head_count_kv u32 = 8 > > llama_model_loader: - kv 9: llama.expert_count u32 = 8 > > llama_model_loader: - kv 10: llama.expert_used_count u32 = 2 > > llama_model_loader: - kv 11: llama.attention.layer_norm_rms_epsilon f32 = 0.000010 > > llama_model_loader: - kv 12: llama.rope.freq_base f32 = 1000000.000000 > > llama_model_loader: - kv 13: general.file_type u32 = 2 > > llama_model_loader: - kv 14: tokenizer.ggml.model str = llama > > llama_model_loader: - kv 15: tokenizer.ggml.tokens arr[str,32000] = ["<unk>", "<s>", "</s>", "<0x00>", "<... > > llama_model_loader: - kv 16: tokenizer.ggml.scores arr[f32,32000] = [0.000000, 0.000000, 0.000000, 0.0000... > > llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,32000] = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ... > > llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,58980] = ["▁ t", "i n", "e r", "▁ a", "h e... > > llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 1 > > llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 2 > > llama_model_loader: - kv 21: tokenizer.ggml.unknown_token_id u32 = 0 > > llama_model_loader: - kv 22: tokenizer.ggml.add_bos_token bool = true > > llama_model_loader: - kv 23: tokenizer.ggml.add_eos_token bool = false > > llama_model_loader: - kv 24: tokenizer.chat_template str = {{ bos_token }}{% for message in mess... > > llama_model_loader: - kv 25: general.quantization_version u32 = 2 > > llama_model_loader: - type f32: 65 tensors > > llama_model_loader: - type f16: 32 tensors > > llama_model_loader: - type q4_0: 833 tensors > > llama_model_loader: - type q8_0: 64 tensors > > llama_model_loader: - type q6_K: 1 tensors > > llm_load_vocab: special tokens definition check successful ( 259/32000 ). > > llm_load_print_meta: format = GGUF V3 (latest) > > llm_load_print_meta: arch = llama > > llm_load_print_meta: vocab type = SPM > > llm_load_print_meta: n_vocab = 32000 > > llm_load_print_meta: n_merges = 0 > > llm_load_print_meta: n_ctx_train = 32768 > > llm_load_print_meta: n_embd = 4096 > > llm_load_print_meta: n_head = 32 > > llm_load_print_meta: n_head_kv = 8 > > llm_load_print_meta: n_layer = 32 > > llm_load_print_meta: n_rot = 128 > > llm_load_print_meta: n_embd_head_k = 128 > > llm_load_print_meta: n_embd_head_v = 128 > > llm_load_print_meta: n_gqa = 4 > > llm_load_print_meta: n_embd_k_gqa = 1024 > > llm_load_print_meta: n_embd_v_gqa = 1024 > > llm_load_print_meta: f_norm_eps = 0.0e+00 > > llm_load_print_meta: f_norm_rms_eps = 1.0e-05 > > llm_load_print_meta: f_clamp_kqv = 0.0e+00 > > llm_load_print_meta: f_max_alibi_bias = 0.0e+00 > > llm_load_print_meta: n_ff = 14336 > > llm_load_print_meta: n_expert = 8 > > llm_load_print_meta: n_expert_used = 2 > > llm_load_print_meta: pooling type = 0 > > llm_load_print_meta: rope type = 0 > > llm_load_print_meta: rope scaling = linear > > llm_load_print_meta: freq_base_train = 1000000.0 > > llm_load_print_meta: freq_scale_train = 1 > > llm_load_print_meta: n_yarn_orig_ctx = 32768 > > llm_load_print_meta: rope_finetuned = unknown > > llm_load_print_meta: model type = 7B > > llm_load_print_meta: model ftype = Q4_0 > > llm_load_print_meta: model params = 46.70 B > > llm_load_print_meta: model size = 24.62 GiB (4.53 BPW) > > llm_load_print_meta: general.name = mistralai > > llm_load_print_meta: BOS token = 1 '<s>' > > llm_load_print_meta: EOS token = 2 '</s>' > > llm_load_print_meta: UNK token = 0 '<unk>' > > llm_load_print_meta: LF token = 13 '<0x0A>' > > llm_load_tensors: ggml ctx size = 0.76 MiB > > Exception 0xc0000005 0x1 0x10 0x7ffa12f9a390 > > PC=0x7ffa12f9a390 > > signal arrived during external code execution > > > > runtime.cgocall(0x7ff6d5b764e0, 0xc001006760) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/cgocall.go:157 +0x3e fp=0xc001006738 sp=0xc001006700 pc=0x7ff6d50a953e > > github.com/jmorganca/ollama/llm._Cfunc_dyn_llama_server_init({0x7ffa14a90000, 0x7ffa14baf250, 0x7ffa14bafd70, 0x7ffa14bafe20, 0x7ffa14bb0240, 0x7ffa14bb0470, 0x7ffa14bb14b0, 0x7ffa14bb1490, 0x7ffa14bb16e0, 0x7ffa14bb1d00, ...}, ...) > > _cgo_gotypes.go:282 +0x4d fp=0xc001006760 sp=0xc001006738 pc=0x7ff6d599b0ed > > github.com/jmorganca/ollama/llm.newDynExtServer.func7(0xc00009c0f0, 0xc000be80d8) > > C:/Users/danie/code/ollama/llm/dyn_ext_server.go:154 +0x112 fp=0xc0010068a0 sp=0xc001006760 pc=0x7ff6d599c8b2 > > github.com/jmorganca/ollama/llm.newDynExtServer({0xc00050e180, 0x4b}, {0xc0004d8360, _}, {_, _, _}, {0x0, 0x0, 0x0}, ...) > > C:/Users/danie/code/ollama/llm/dyn_ext_server.go:154 +0xb50 fp=0xc001006ae8 sp=0xc0010068a0 pc=0x7ff6d599c4f0 > > github.com/jmorganca/ollama/llm.newLlmServer({{_, _, _}, {_, _}, {_, _}}, {_, _}, {0x0, ...}, ...) > > C:/Users/danie/code/ollama/llm/llm.go:158 +0x4c5 fp=0xc001006ca8 sp=0xc001006ae8 pc=0x7ff6d5998905 > > github.com/jmorganca/ollama/llm.New({0xc0004d8360, 0x5e}, {0x0, 0x0, 0x0}, {0x0, _, _}, {{0x0, 0x800, ...}, ...}) > > C:/Users/danie/code/ollama/llm/llm.go:123 +0x76e fp=0xc001006f18 sp=0xc001006ca8 pc=0x7ff6d59981ce > > github.com/jmorganca/ollama/server.load(0xc0000dc180?, 0xc0000dc180, {{0x0, 0x800, 0x200, 0x1, 0xffffffffffffffff, 0x0, 0x0, 0x1, ...}, ...}, ...) > > C:/Users/danie/code/ollama/server/routes.go:83 +0x325 fp=0xc001007068 sp=0xc001006f18 pc=0x7ff6d5b496e5 > > github.com/jmorganca/ollama/server.ChatHandler(0xc0000a2000) > > C:/Users/danie/code/ollama/server/routes.go:1173 +0xa37 fp=0xc001007770 sp=0xc001007068 pc=0x7ff6d5b54e57 > > github.com/gin-gonic/gin.(*Context).Next(...) > > C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174 > > github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func1(0xc0000a2000) > > C:/Users/danie/code/ollama/server/routes.go:943 +0x68 fp=0xc0010077a8 sp=0xc001007770 pc=0x7ff6d5b53628 > > github.com/gin-gonic/gin.(*Context).Next(...) > > C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174 > > github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1(0xc0000a2000) > > C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/recovery.go:102 +0x7a fp=0xc0010077f8 sp=0xc0010077a8 pc=0x7ff6d5b29eba > > github.com/gin-gonic/gin.(*Context).Next(...) > > C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174 > > github.com/gin-gonic/gin.LoggerWithConfig.func1(0xc0000a2000) > > C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/logger.go:240 +0xdd fp=0xc0010079a8 sp=0xc0010077f8 pc=0x7ff6d5b28ffd > > github.com/gin-gonic/gin.(*Context).Next(...) > > C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174 > > github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc000168ea0, 0xc0000a2000) > > C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/gin.go:620 +0x66e fp=0xc001007b28 sp=0xc0010079a8 pc=0x7ff6d5b2852e > > github.com/gin-gonic/gin.(*Engine).ServeHTTP(0xc000168ea0, {0x7ff6d9539dd0, 0xc00009e0e0}, 0xc0000b45a0) > > C:/Users/danie/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/gin.go:576 +0x1b2 fp=0xc001007b60 sp=0xc001007b28 pc=0x7ff6d5b27cf2 > > net/http.serverHandler.ServeHTTP({0x7ff6d9537c48?}, {0x7ff6d9539dd0?, 0xc00009e0e0?}, 0x6?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:3137 +0x8e fp=0xc001007b90 sp=0xc001007b60 pc=0x7ff6d53aee2e > > net/http.(*conn).serve(0xc0000e66c0, {0x7ff6d953c1c8, 0xc0001b8660}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:2039 +0x5e8 fp=0xc001007fb8 sp=0xc001007b90 pc=0x7ff6d53aa1e8 > > net/http.(*Server).Serve.gowrap3() > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:3285 +0x28 fp=0xc001007fe0 sp=0xc001007fb8 pc=0x7ff6d53af648 > > runtime.goexit({}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc001007fe8 sp=0xc001007fe0 pc=0x7ff6d5112901 > > created by net/http.(*Server).Serve in goroutine 1 > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:3285 +0x4b4 > > > > goroutine 1 gp=0xc00007e000 m=nil [IO wait]: > > runtime.gopark(0xc000600008?, 0x7ff6d5bfdb00?, 0x20?, 0x2a?, 0xc0005b2a50?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc0006a76f8 sp=0xc0006a76d8 pc=0x7ff6d50e1aae > > runtime.netpollblock(0x1b4?, 0xd50a9046?, 0xf6?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/netpoll.go:573 +0xf7 fp=0xc0006a7730 sp=0xc0006a76f8 pc=0x7ff6d50d9357 > > internal/poll.runtime_pollWait(0x20177cadba0, 0x72) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/netpoll.go:345 +0x85 fp=0xc0006a7750 sp=0xc0006a7730 pc=0x7ff6d510c3a5 > > internal/poll.(*pollDesc).wait(0x7ff6d50c07b6?, 0x7ff6d9c800a0?, 0x0) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0006a7778 sp=0xc0006a7750 pc=0x7ff6d51b0647 > > internal/poll.execIO(0xc0005b2a20, 0xc0006a7818) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_windows.go:175 +0xe6 fp=0xc0006a77e8 sp=0xc0006a7778 pc=0x7ff6d51b1b26 > > internal/poll.(*FD).acceptOne(0xc0005b2a08, 0x268, {0xc00000c0f0?, 0x0?, 0x0?}, 0xc000600008?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_windows.go:944 +0x67 fp=0xc0006a7848 sp=0xc0006a77e8 pc=0x7ff6d51b61e7 > > internal/poll.(*FD).Accept(0xc0005b2a08, 0xc0006a79f8) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_windows.go:978 +0x1bc fp=0xc0006a7900 sp=0xc0006a7848 pc=0x7ff6d51b651c > > net.(*netFD).accept(0xc0005b2a08) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/fd_windows.go:178 +0x54 fp=0xc0006a7a18 sp=0xc0006a7900 pc=0x7ff6d52476f4 > > net.(*TCPListener).accept(0xc0004405c0) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/tcpsock_posix.go:159 +0x1e fp=0xc0006a7a40 sp=0xc0006a7a18 pc=0x7ff6d525cb9e > > net.(*TCPListener).Accept(0xc0004405c0) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/tcpsock.go:327 +0x30 fp=0xc0006a7a70 sp=0xc0006a7a40 pc=0x7ff6d525bc90 > > net/http.(*onceCloseListener).Accept(0xc0000e66c0?) > > <autogenerated>:1 +0x24 fp=0xc0006a7a88 sp=0xc0006a7a70 pc=0x7ff6d53d1804 > > net/http.(*Server).Serve(0xc0000c04b0, {0x7ff6d9539bc0, 0xc0004405c0}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:3255 +0x33e fp=0xc0006a7bb8 sp=0xc0006a7a88 pc=0x7ff6d53af25e > > github.com/jmorganca/ollama/server.Serve({0x7ff6d9539bc0, 0xc0004405c0}) > > C:/Users/danie/code/ollama/server/routes.go:1046 +0x468 fp=0xc0006a7cc0 sp=0xc0006a7bb8 pc=0x7ff6d5b53ae8 > > github.com/jmorganca/ollama/cmd.RunServer(0xc0001e6c00?, {0x7ff6d9c7e340?, 0x4?, 0x7ff6d5db3a7e?}) > > C:/Users/danie/code/ollama/cmd/cmd.go:787 +0x1b9 fp=0xc0006a7d58 sp=0xc0006a7cc0 pc=0x7ff6d5b6d9b9 > > github.com/spf13/cobra.(*Command).execute(0xc0000b8f08, {0x7ff6d9c7e340, 0x0, 0x0}) > > C:/Users/danie/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:940 +0x882 fp=0xc0006a7e78 sp=0xc0006a7d58 pc=0x7ff6d544afe2 > > github.com/spf13/cobra.(*Command).ExecuteC(0xc0000b8308) > > C:/Users/danie/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3a5 fp=0xc0006a7f30 sp=0xc0006a7e78 pc=0x7ff6d544b825 > > github.com/spf13/cobra.(*Command).Execute(...) > > C:/Users/danie/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:992 > > github.com/spf13/cobra.(*Command).ExecuteContext(...) > > C:/Users/danie/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:985 > > main.main() > > C:/Users/danie/code/ollama/main.go:11 +0x4d fp=0xc0006a7f50 sp=0xc0006a7f30 pc=0x7ff6d5b7624d > > runtime.main() > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:271 +0x28b fp=0xc0006a7fe0 sp=0xc0006a7f50 pc=0x7ff6d50e16ab > > runtime.goexit({}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0006a7fe8 sp=0xc0006a7fe0 pc=0x7ff6d5112901 > > > > goroutine 2 gp=0xc00007e700 m=nil [force gc (idle)]: > > runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000081fa8 sp=0xc000081f88 pc=0x7ff6d50e1aae > > runtime.goparkunlock(...) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:408 > > runtime.forcegchelper() > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:326 +0xb8 fp=0xc000081fe0 sp=0xc000081fa8 pc=0x7ff6d50e1938 > > runtime.goexit({}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000081fe8 sp=0xc000081fe0 pc=0x7ff6d5112901 > > created by runtime.init.6 in goroutine 1 > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:314 +0x1a > > > > goroutine 3 gp=0xc00007ea80 m=nil [GC sweep wait]: > > runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000083f80 sp=0xc000083f60 pc=0x7ff6d50e1aae > > runtime.goparkunlock(...) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:408 > > runtime.bgsweep(0xc00008e000) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgcsweep.go:318 +0xdf fp=0xc000083fc8 sp=0xc000083f80 pc=0x7ff6d50cbb5f > > runtime.gcenable.gowrap1() > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:203 +0x25 fp=0xc000083fe0 sp=0xc000083fc8 pc=0x7ff6d50c0405 > > runtime.goexit({}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000083fe8 sp=0xc000083fe0 pc=0x7ff6d5112901 > > created by runtime.gcenable in goroutine 1 > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:203 +0x66 > > > > goroutine 4 gp=0xc00007ec40 m=nil [GC scavenge wait]: > > runtime.gopark(0x991e0c?, 0x86fc3e?, 0x0?, 0x0?, 0x0?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000095f78 sp=0xc000095f58 pc=0x7ff6d50e1aae > > runtime.goparkunlock(...) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:408 > > runtime.(*scavengerState).park(0x7ff6d9bf3020) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc000095fa8 sp=0xc000095f78 pc=0x7ff6d50c94e9 > > runtime.bgscavenge(0xc00008e000) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgcscavenge.go:658 +0x59 fp=0xc000095fc8 sp=0xc000095fa8 pc=0x7ff6d50c9a99 > > runtime.gcenable.gowrap2() > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:204 +0x25 fp=0xc000095fe0 sp=0xc000095fc8 pc=0x7ff6d50c03a5 > > runtime.goexit({}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000095fe8 sp=0xc000095fe0 pc=0x7ff6d5112901 > > created by runtime.gcenable in goroutine 1 > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:204 +0xa5 > > > > goroutine 18 gp=0xc000104380 m=nil [finalizer wait]: > > runtime.gopark(0xc000085e48?, 0x7ff6d50b37c5?, 0xa8?, 0x1?, 0xc00007e000?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000085e20 sp=0xc000085e00 pc=0x7ff6d50e1aae > > runtime.runfinq() > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mfinal.go:194 +0x107 fp=0xc000085fe0 sp=0xc000085e20 pc=0x7ff6d50bf487 > > runtime.goexit({}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000085fe8 sp=0xc000085fe0 pc=0x7ff6d5112901 > > created by runtime.createfing in goroutine 1 > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mfinal.go:164 +0x3d > > > > goroutine 19 gp=0xc000105880 m=nil [GC worker (idle)]: > > runtime.gopark(0x7ff6d9c800a0?, 0x1?, 0x0?, 0x0?, 0x0?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000091f50 sp=0xc000091f30 pc=0x7ff6d50e1aae > > runtime.gcBgMarkWorker() > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000091fe0 sp=0xc000091f50 pc=0x7ff6d50c2545 > > runtime.goexit({}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000091fe8 sp=0xc000091fe0 pc=0x7ff6d5112901 > > created by runtime.gcBgMarkStartWorkers in goroutine 1 > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c > > > > goroutine 34 gp=0xc000480000 m=nil [GC worker (idle)]: > > runtime.gopark(0x3135b6217c2c?, 0x1?, 0x0?, 0x0?, 0x0?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000487f50 sp=0xc000487f30 pc=0x7ff6d50e1aae > > runtime.gcBgMarkWorker() > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000487fe0 sp=0xc000487f50 pc=0x7ff6d50c2545 > > runtime.goexit({}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000487fe8 sp=0xc000487fe0 pc=0x7ff6d5112901 > > created by runtime.gcBgMarkStartWorkers in goroutine 1 > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c > > > > goroutine 5 gp=0xc00007efc0 m=nil [GC worker (idle)]: > > runtime.gopark(0x7ff6d9c800a0?, 0x3?, 0x0?, 0x0?, 0x0?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000097f50 sp=0xc000097f30 pc=0x7ff6d50e1aae > > runtime.gcBgMarkWorker() > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000097fe0 sp=0xc000097f50 pc=0x7ff6d50c2545 > > runtime.goexit({}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000097fe8 sp=0xc000097fe0 pc=0x7ff6d5112901 > > created by runtime.gcBgMarkStartWorkers in goroutine 1 > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c > > > > goroutine 35 gp=0xc0004801c0 m=nil [GC worker (idle)]: > > runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000489f50 sp=0xc000489f30 pc=0x7ff6d50e1aae > > runtime.gcBgMarkWorker() > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000489fe0 sp=0xc000489f50 pc=0x7ff6d50c2545 > > runtime.goexit({}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000489fe8 sp=0xc000489fe0 pc=0x7ff6d5112901 > > created by runtime.gcBgMarkStartWorkers in goroutine 1 > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c > > > > goroutine 6 gp=0xc00007f180 m=nil [GC worker (idle)]: > > runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000483f50 sp=0xc000483f30 pc=0x7ff6d50e1aae > > runtime.gcBgMarkWorker() > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000483fe0 sp=0xc000483f50 pc=0x7ff6d50c2545 > > runtime.goexit({}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000483fe8 sp=0xc000483fe0 pc=0x7ff6d5112901 > > created by runtime.gcBgMarkStartWorkers in goroutine 1 > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c > > > > goroutine 36 gp=0xc000480380 m=nil [GC worker (idle)]: > > runtime.gopark(0x3135b6217c2c?, 0x1?, 0x0?, 0x0?, 0x0?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000491f50 sp=0xc000491f30 pc=0x7ff6d50e1aae > > runtime.gcBgMarkWorker() > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000491fe0 sp=0xc000491f50 pc=0x7ff6d50c2545 > > runtime.goexit({}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000491fe8 sp=0xc000491fe0 pc=0x7ff6d5112901 > > created by runtime.gcBgMarkStartWorkers in goroutine 1 > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c > > > > goroutine 7 gp=0xc00007f340 m=nil [GC worker (idle)]: > > runtime.gopark(0x3135b6217c2c?, 0x3?, 0x1c?, 0xfa?, 0x0?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000485f50 sp=0xc000485f30 pc=0x7ff6d50e1aae > > runtime.gcBgMarkWorker() > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000485fe0 sp=0xc000485f50 pc=0x7ff6d50c2545 > > runtime.goexit({}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000485fe8 sp=0xc000485fe0 pc=0x7ff6d5112901 > > created by runtime.gcBgMarkStartWorkers in goroutine 1 > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c > > > > goroutine 37 gp=0xc000480540 m=nil [GC worker (idle)]: > > runtime.gopark(0x7ff6d9c800a0?, 0x3?, 0x40?, 0xbd?, 0x0?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000493f50 sp=0xc000493f30 pc=0x7ff6d50e1aae > > runtime.gcBgMarkWorker() > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000493fe0 sp=0xc000493f50 pc=0x7ff6d50c2545 > > runtime.goexit({}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000493fe8 sp=0xc000493fe0 pc=0x7ff6d5112901 > > created by runtime.gcBgMarkStartWorkers in goroutine 1 > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c > > > > goroutine 8 gp=0xc00007f500 m=nil [GC worker (idle)]: > > runtime.gopark(0x3135b6217c2c?, 0x1?, 0x0?, 0x0?, 0x0?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc00048df50 sp=0xc00048df30 pc=0x7ff6d50e1aae > > runtime.gcBgMarkWorker() > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc00048dfe0 sp=0xc00048df50 pc=0x7ff6d50c2545 > > runtime.goexit({}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00048dfe8 sp=0xc00048dfe0 pc=0x7ff6d5112901 > > created by runtime.gcBgMarkStartWorkers in goroutine 1 > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c > > > > goroutine 38 gp=0xc000480700 m=nil [GC worker (idle)]: > > runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000499f50 sp=0xc000499f30 pc=0x7ff6d50e1aae > > runtime.gcBgMarkWorker() > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000499fe0 sp=0xc000499f50 pc=0x7ff6d50c2545 > > runtime.goexit({}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000499fe8 sp=0xc000499fe0 pc=0x7ff6d5112901 > > created by runtime.gcBgMarkStartWorkers in goroutine 1 > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c > > > > goroutine 9 gp=0xc00007f6c0 m=nil [GC worker (idle)]: > > runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc00048ff50 sp=0xc00048ff30 pc=0x7ff6d50e1aae > > runtime.gcBgMarkWorker() > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc00048ffe0 sp=0xc00048ff50 pc=0x7ff6d50c2545 > > runtime.goexit({}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00048ffe8 sp=0xc00048ffe0 pc=0x7ff6d5112901 > > created by runtime.gcBgMarkStartWorkers in goroutine 1 > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c > > > > goroutine 39 gp=0xc0004808c0 m=nil [GC worker (idle)]: > > runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc00049bf50 sp=0xc00049bf30 pc=0x7ff6d50e1aae > > runtime.gcBgMarkWorker() > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc00049bfe0 sp=0xc00049bf50 pc=0x7ff6d50c2545 > > runtime.goexit({}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00049bfe8 sp=0xc00049bfe0 pc=0x7ff6d5112901 > > created by runtime.gcBgMarkStartWorkers in goroutine 1 > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c > > > > goroutine 10 gp=0xc00007f880 m=nil [GC worker (idle)]: > > runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000495f50 sp=0xc000495f30 pc=0x7ff6d50e1aae > > runtime.gcBgMarkWorker() > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000495fe0 sp=0xc000495f50 pc=0x7ff6d50c2545 > > runtime.goexit({}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000495fe8 sp=0xc000495fe0 pc=0x7ff6d5112901 > > created by runtime.gcBgMarkStartWorkers in goroutine 1 > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c > > > > goroutine 40 gp=0xc000480a80 m=nil [GC worker (idle)]: > > runtime.gopark(0xc000059c50?, 0x1?, 0x2c?, 0x7c?, 0x0?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc0004a1f50 sp=0xc0004a1f30 pc=0x7ff6d50e1aae > > runtime.gcBgMarkWorker() > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc0004a1fe0 sp=0xc0004a1f50 pc=0x7ff6d50c2545 > > runtime.goexit({}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0004a1fe8 sp=0xc0004a1fe0 pc=0x7ff6d5112901 > > created by runtime.gcBgMarkStartWorkers in goroutine 1 > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c > > > > goroutine 11 gp=0xc00007fa40 m=nil [GC worker (idle)]: > > runtime.gopark(0x3135b6217c2c?, 0x3?, 0x0?, 0x0?, 0x0?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000497f50 sp=0xc000497f30 pc=0x7ff6d50e1aae > > runtime.gcBgMarkWorker() > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc000497fe0 sp=0xc000497f50 pc=0x7ff6d50c2545 > > runtime.goexit({}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000497fe8 sp=0xc000497fe0 pc=0x7ff6d5112901 > > created by runtime.gcBgMarkStartWorkers in goroutine 1 > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c > > > > goroutine 41 gp=0xc000480c40 m=nil [GC worker (idle)]: > > runtime.gopark(0x3135b629396c?, 0x3?, 0x0?, 0x0?, 0x0?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc0004a3f50 sp=0xc0004a3f30 pc=0x7ff6d50e1aae > > runtime.gcBgMarkWorker() > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1310 +0xe5 fp=0xc0004a3fe0 sp=0xc0004a3f50 pc=0x7ff6d50c2545 > > runtime.goexit({}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0004a3fe8 sp=0xc0004a3fe0 pc=0x7ff6d5112901 > > created by runtime.gcBgMarkStartWorkers in goroutine 1 > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/mgc.go:1234 +0x1c > > > > goroutine 12 gp=0xc000481180 m=4 mp=0xc00008b808 [syscall]: > > runtime.notetsleepg(0x7ff6d9c7eee0, 0xffffffffffffffff) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/lock_sema.go:296 +0x31 fp=0xc00049ffa0 sp=0xc00049ff68 pc=0x7ff6d50b1d91 > > os/signal.signal_recv() > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/sigqueue.go:152 +0x29 fp=0xc00049ffc0 sp=0xc00049ffa0 pc=0x7ff6d510e5e9 > > os/signal.loop() > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/os/signal/signal_unix.go:23 +0x13 fp=0xc00049ffe0 sp=0xc00049ffc0 pc=0x7ff6d53d3bb3 > > runtime.goexit({}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00049ffe8 sp=0xc00049ffe0 pc=0x7ff6d5112901 > > created by os/signal.Notify.func1.1 in goroutine 1 > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/os/signal/signal.go:151 +0x1f > > > > goroutine 13 gp=0xc000481340 m=nil [chan receive]: > > runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc000093f18 sp=0xc000093ef8 pc=0x7ff6d50e1aae > > runtime.chanrecv(0xc0001eafc0, 0x0, 0x1) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/chan.go:583 +0x3cd fp=0xc000093f90 sp=0xc000093f18 pc=0x7ff6d50abbcd > > runtime.chanrecv1(0x0?, 0x0?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/chan.go:442 +0x12 fp=0xc000093fb8 sp=0xc000093f90 pc=0x7ff6d50ab7d2 > > github.com/jmorganca/ollama/server.Serve.func2() > > C:/Users/danie/code/ollama/server/routes.go:1028 +0x25 fp=0xc000093fe0 sp=0xc000093fb8 pc=0x7ff6d5b53b85 > > runtime.goexit({}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000093fe8 sp=0xc000093fe0 pc=0x7ff6d5112901 > > created by github.com/jmorganca/ollama/server.Serve in goroutine 1 > > C:/Users/danie/code/ollama/server/routes.go:1027 +0x3f6 > > > > goroutine 83 gp=0xc0005048c0 m=nil [IO wait]: > > runtime.gopark(0x0?, 0xc000698020?, 0xd0?, 0x80?, 0xc000698050?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/proc.go:402 +0xce fp=0xc0004cbd28 sp=0xc0004cbd08 pc=0x7ff6d50e1aae > > runtime.netpollblock(0x280?, 0xd50a9046?, 0xf6?) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/netpoll.go:573 +0xf7 fp=0xc0004cbd60 sp=0xc0004cbd28 pc=0x7ff6d50d9357 > > internal/poll.runtime_pollWait(0x20177cadaa8, 0x72) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/netpoll.go:345 +0x85 fp=0xc0004cbd80 sp=0xc0004cbd60 pc=0x7ff6d510c3a5 > > internal/poll.(*pollDesc).wait(0x10?, 0x10?, 0x0) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0004cbda8 sp=0xc0004cbd80 pc=0x7ff6d51b0647 > > internal/poll.execIO(0xc000698020, 0x7ff6d9419910) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_windows.go:175 +0xe6 fp=0xc0004cbe18 sp=0xc0004cbda8 pc=0x7ff6d51b1b26 > > internal/poll.(*FD).Read(0xc000698008, {0xc000f0c641, 0x1, 0x1}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/internal/poll/fd_windows.go:436 +0x2b1 fp=0xc0004cbec0 sp=0xc0004cbe18 pc=0x7ff6d51b27d1 > > net.(*netFD).Read(0xc000698008, {0xc000f0c641?, 0xc0004cbf48?, 0x7ff6d510e330?}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/fd_posix.go:55 +0x25 fp=0xc0004cbf08 sp=0xc0004cbec0 pc=0x7ff6d5245805 > > net.(*conn).Read(0xc000110040, {0xc000f0c641?, 0xc0004cbeb8?, 0x7ff6d9c7e340?}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/net.go:179 +0x45 fp=0xc0004cbf50 sp=0xc0004cbf08 pc=0x7ff6d5254885 > > net.(*TCPConn).Read(0x7ff6d5bd7aa0?, {0xc000f0c641?, 0xc0000983c0?, 0x7ff6d54a25a0?}) > > <autogenerated>:1 +0x25 fp=0xc0004cbf80 sp=0xc0004cbf50 pc=0x7ff6d52648a5 > > net/http.(*connReader).backgroundRead(0xc000f0c630) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:681 +0x37 fp=0xc0004cbfc8 sp=0xc0004cbf80 pc=0x7ff6d53a4157 > > net/http.(*connReader).startBackgroundRead.gowrap2() > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:677 +0x25 fp=0xc0004cbfe0 sp=0xc0004cbfc8 pc=0x7ff6d53a4085 > > runtime.goexit({}) > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0004cbfe8 sp=0xc0004cbfe0 pc=0x7ff6d5112901 > > created by net/http.(*connReader).startBackgroundRead in goroutine 21 > > C:/Users/danie/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.windows-amd64/src/net/http/server.go:677 +0xba > > rax 0x0 > > rbx 0x2012414aa68 > > rcx 0x2012414aa18 > > rdx 0xffffffdd00000000 > > rdi 0x2012414aa68 > > rsi 0x2012414aa18 > > rbp 0x0 > > rsp 0x8f23ffd7b0 > > r8 0x2012414aa68 > > r9 0x0 > > r10 0x96 > > r11 0x246 > > r12 0x0 > > r13 0x7ffa12a30000 > > r14 0x2014e56d9c0 > > r15 0x8f23ffd970 > > rip 0x7ffa12f9a390 > > rflags 0x10246 > > cs 0x33 > > fs 0x53 > > gs 0x2b > > {"function":"initialize","level":"INFO","line":688,"msg":"initializing slots","n_slots":1,"tid":"19848","timestamp":1709950439} > > {"function":"initialize","id_slot":0,"level":"INFO","line":699,"msg":"new slot","n_ctx_slot":2048,"tid":"19848","timestamp":1709950439} > > {"function":"update_slots","level":"INFO","line":1593,"msg":"all slots are idle","tid":"5172","timestamp":1709950439} > > {"function":"launch_slot_with_data","id_slot":0,"id_task":0,"level":"INFO","line":973,"msg":"slot is processing task","tid":"5172","timestamp":1709950443} > > {"function":"update_slots","id_slot":0,"id_task":0,"level":"INFO","line":1845,"msg":"kv cache rm [p0, end)","p0":0,"tid":"5172","timestamp":1709950443} > > {"function":"print_timings","id_slot":0,"id_task":0,"level":"INFO","line":311,"msg":"prompt eval time = 63.01 ms / 11 tokens ( 5.73 ms per token, 174.58 tokens per second)","n_prompt_tokens_processed":11,"n_tokens_second":174.58100558659217,"t_prompt_processing":63.008,"t_token":5.728000000000001,"tid":"5172","timestamp":1709950444} > > {"function":"print_timings","id_slot":0,"id_task":0,"level":"INFO","line":327,"msg":"generation eval time = 845.31 ms / 95 runs ( 8.90 ms per token, 112.38 tokens per second)","n_decoded":95,"n_tokens_second":112.38480557428636,"t_token":8.898,"t_token_generation":845.31,"tid":"5172","timestamp":1709950444} > > {"function":"print_timings","id_slot":0,"id_task":0,"level":"INFO","line":337,"msg":" total time = 908.32 ms","t_prompt_processing":63.008,"t_token_generation":845.31,"t_total":908.318,"tid":"5172","timestamp":1709950444} > > {"function":"update_slots","id_slot":0,"id_task":0,"level":"INFO","line":1575,"msg":"slot released","n_cache_tokens":105,"n_ctx":2048,"n_past":105,"n_system_tokens":0,"tid":"5172","timestamp":1709950444,"truncated":false} > > {"function":"update_slots","level":"INFO","line":1593,"msg":"all slots are idle","tid":"5172","timestamp":1709950444} > > {"function":"update_slots","level":"INFO","line":1593,"msg":"all slots are idle","tid":"5172","timestamp":1709950444} > > {"function":"update_slots","level":"INFO","line":1593,"msg":"all slots are idle","tid":"5172","timestamp":1709950480} > > loading library C:\Users\user\AppData\Local\Temp\ollama\payloads\rocm_v5.7\ext_server.dll > > ``` Have you fixed it?
GiteaMirror commented 2026-04-12 11:26:40 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Mar 20, 2024):

@sgwhat please upgrade to 0.1.29 and if you're still seeing this crash, please go ahead and open a new issue so we can get to the bottom of why it's crashing for you.

<!-- gh-comment-id:2009122896 --> @dhiltgen commented on GitHub (Mar 20, 2024): @sgwhat please upgrade to 0.1.29 and if you're still seeing this crash, please go ahead and open a new issue so we can get to the bottom of why it's crashing for you.
GiteaMirror commented 2026-04-12 11:26:41 -05:00
Author
Owner
Copy Link

@likelovewant commented on GitHub (Mar 27, 2024):

zluda.exe -- %UserProfile%/AppData/Local/Programs/Ollama/ollama.exe serve
@Inokinoki
I manuly send the data as you suggest ,like this , replace the content from line 199 with "

resp->major = 5;  
 resp->minor = 0;

build and succuess ,
start :
zluda.exe -- %UserProfile%/AppData/Local/Programs/Ollama/ollama.exe serve

time=2024-03-27T13:07:01.957+08:00 level=INFO source=payload_common.go:139 msg="Dynamic LLM libraries [cpu_avx2 cpu_avx rocm_v5.7 cpu]"
3/27 - 13:09:17 | 200 | 3.9095ms | 127.0.0.1 | POST "/api/show"
[GIN] 2024/03/27 - 13:09:17 | 200 | 0s | 127.0.0.1 | POST "/api/show"
time=2024-03-27T13:09:18.120+08:00 level=INFO source=gpu.go:77 msg="Detecting GPU type"
time=2024-03-27T13:09:18.120+08:00 level=INFO source=gpu.go:191 msg="Searching for GPU management library nvml.dll"
time=2024-03-27T13:09:18.139+08:00 level=INFO source=gpu.go:237 msg="Discovered GPU libraries: [C:\ZLUDA\nvml.dll]"
time=2024-03-27T13:09:18.236+08:00 level=INFO source=gpu.go:82 msg="Nvidia GPU detected"
time=2024-03-27T13:09:18.237+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-03-27T13:09:18.237+08:00 level=INFO source=gpu.go:119 msg="CUDA Compute Capability detected: 5.0"
time=2024-03-27T13:09:18.238+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-03-27T13:09:18.238+08:00 level=INFO source=gpu.go:119 msg="CUDA Compute Capability detected: 5.0"
time=2024-03-27T13:09:18.238+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"

and start models by " zluda.exe -- %UserProfile%/AppData/Local/Programs/Ollama/ollama.exe run llama2
"
I got following response

llm_load_tensors:        CPU buffer size =  3647.87 MiB
..................................................................................................
llama_new_context_with_model: n_ctx      = 2048
llama_new_context_with_model: freq_base  = 10000.0
llama_new_context_with_model: freq_scale = 1
llama_kv_cache_init:        CPU KV buffer size =  1024.00 MiB
llama_new_context_with_model: KV self size  = 1024.00 MiB, K (f16):  512.00 MiB, V (f16):  512.00 MiB
llama_new_context_with_model:        CPU input buffer size   =    13.02 MiB
llama_new_context_with_model:        CPU compute buffer size =   160.00 MiB
llama_new_context_with_model: graph splits (measure): 1

it's seems the gpu not load . any idea why ? or how you did set the code

<!-- gh-comment-id:2022004710 --> @likelovewant commented on GitHub (Mar 27, 2024): zluda.exe -- %UserProfile%/AppData/Local/Programs/Ollama/ollama.exe serve @Inokinoki I manuly send the data as you suggest ,like this , replace the content from line 199 with " ``` resp->major = 5; resp->minor = 0; ``` build and succuess , start : zluda.exe -- %UserProfile%/AppData/Local/Programs/Ollama/ollama.exe serve ``` time=2024-03-27T13:07:01.957+08:00 level=INFO source=payload_common.go:139 msg="Dynamic LLM libraries [cpu_avx2 cpu_avx rocm_v5.7 cpu]" 3/27 - 13:09:17 | 200 | 3.9095ms | 127.0.0.1 | POST "/api/show" [GIN] 2024/03/27 - 13:09:17 | 200 | 0s | 127.0.0.1 | POST "/api/show" time=2024-03-27T13:09:18.120+08:00 level=INFO source=gpu.go:77 msg="Detecting GPU type" time=2024-03-27T13:09:18.120+08:00 level=INFO source=gpu.go:191 msg="Searching for GPU management library nvml.dll" time=2024-03-27T13:09:18.139+08:00 level=INFO source=gpu.go:237 msg="Discovered GPU libraries: [C:\ZLUDA\nvml.dll]" time=2024-03-27T13:09:18.236+08:00 level=INFO source=gpu.go:82 msg="Nvidia GPU detected" time=2024-03-27T13:09:18.237+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2024-03-27T13:09:18.237+08:00 level=INFO source=gpu.go:119 msg="CUDA Compute Capability detected: 5.0" time=2024-03-27T13:09:18.238+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2024-03-27T13:09:18.238+08:00 level=INFO source=gpu.go:119 msg="CUDA Compute Capability detected: 5.0" time=2024-03-27T13:09:18.238+08:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" ``` and start models by " zluda.exe -- %UserProfile%/AppData/Local/Programs/Ollama/ollama.exe run llama2 " I got following response ``` llm_load_tensors: CPU buffer size = 3647.87 MiB .................................................................................................. llama_new_context_with_model: n_ctx = 2048 llama_new_context_with_model: freq_base = 10000.0 llama_new_context_with_model: freq_scale = 1 llama_kv_cache_init: CPU KV buffer size = 1024.00 MiB llama_new_context_with_model: KV self size = 1024.00 MiB, K (f16): 512.00 MiB, V (f16): 512.00 MiB llama_new_context_with_model: CPU input buffer size = 13.02 MiB llama_new_context_with_model: CPU compute buffer size = 160.00 MiB llama_new_context_with_model: graph splits (measure): 1 ``` it's seems the gpu not load . any idea why ? or how you did set the code
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label amd feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#1530
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?