github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-16 05:05:37 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #4884] No proper response when IPEX-LLM setup with Ollama for intel cpu/gpu #3081

New Issue
Closed
opened 2026-04-12 13:31:09 -05:00 by GiteaMirror · 2 comments
GiteaMirror commented 2026-04-12 13:31:09 -05:00
Owner
Copy Link

Originally created by @filip-777 on GitHub (Jun 6, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/4884

What is the issue?

After setup of IPEX-LLM to work with ollama I see that output is wrong.

Example:

❯ ./ollama run phi3
>>> hi
<s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s>

Maybe I had setup something wrong...

When I serve ollama I have such logs:

2024/06/07 01:00:48 routes.go:1028: INFO server config env="map[OLLAMA_DEBUG:true OLLAMA_FLASH_ATTENTION:false OLLAMA_HOST: OLLAMA_KEEP_ALIVE: OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:1 OLLAMA_MAX_QUEUE:512 OLLAMA_MAX_VRAM:0 OLLAMA_MODELS: OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:*] OLLAMA_RUNNERS_DIR: OLLAMA_TMPDIR:]"
time=2024-06-07T01:00:48.471+02:00 level=INFO source=images.go:729 msg="total blobs: 38"
time=2024-06-07T01:00:48.471+02:00 level=INFO source=images.go:736 msg="total unused blobs removed: 0"
[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.

[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:   export GIN_MODE=release
 - using code:  gin.SetMode(gin.ReleaseMode)

[GIN-debug] POST   /api/pull                 --> github.com/ollama/ollama/server.(*Server).PullModelHandler-fm (5 handlers)
[GIN-debug] POST   /api/generate             --> github.com/ollama/ollama/server.(*Server).GenerateHandler-fm (5 handlers)
[GIN-debug] POST   /api/chat                 --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (5 handlers)
[GIN-debug] POST   /api/embeddings           --> github.com/ollama/ollama/server.(*Server).EmbeddingsHandler-fm (5 handlers)
[GIN-debug] POST   /api/create               --> github.com/ollama/ollama/server.(*Server).CreateModelHandler-fm (5 handlers)
[GIN-debug] POST   /api/push                 --> github.com/ollama/ollama/server.(*Server).PushModelHandler-fm (5 handlers)
[GIN-debug] POST   /api/copy                 --> github.com/ollama/ollama/server.(*Server).CopyModelHandler-fm (5 handlers)
[GIN-debug] DELETE /api/delete               --> github.com/ollama/ollama/server.(*Server).DeleteModelHandler-fm (5 handlers)
[GIN-debug] POST   /api/show                 --> github.com/ollama/ollama/server.(*Server).ShowModelHandler-fm (5 handlers)
[GIN-debug] POST   /api/blobs/:digest        --> github.com/ollama/ollama/server.(*Server).CreateBlobHandler-fm (5 handlers)
[GIN-debug] HEAD   /api/blobs/:digest        --> github.com/ollama/ollama/server.(*Server).HeadBlobHandler-fm (5 handlers)
[GIN-debug] GET    /api/ps                   --> github.com/ollama/ollama/server.(*Server).ProcessHandler-fm (5 handlers)
[GIN-debug] POST   /v1/chat/completions      --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (6 handlers)
[GIN-debug] GET    /                         --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers)
[GIN-debug] GET    /api/tags                 --> github.com/ollama/ollama/server.(*Server).ListModelsHandler-fm (5 handlers)
[GIN-debug] GET    /api/version              --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers)
[GIN-debug] HEAD   /                         --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers)
[GIN-debug] HEAD   /api/tags                 --> github.com/ollama/ollama/server.(*Server).ListModelsHandler-fm (5 handlers)
[GIN-debug] HEAD   /api/version              --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers)
time=2024-06-07T01:00:48.471+02:00 level=INFO source=routes.go:1074 msg="Listening on 127.0.0.1:11434 (version 0.0.0)"
time=2024-06-07T01:00:48.472+02:00 level=INFO source=payload.go:30 msg="extracting embedded files" dir=/tmp/ollama607982765/runners
time=2024-06-07T01:00:48.472+02:00 level=DEBUG source=payload.go:180 msg=extracting variant=cpu file=build/linux/x86_64/cpu/bin/ollama_llama_server.gz
time=2024-06-07T01:00:48.472+02:00 level=DEBUG source=payload.go:180 msg=extracting variant=cpu_avx file=build/linux/x86_64/cpu_avx/bin/ollama_llama_server.gz
time=2024-06-07T01:00:48.472+02:00 level=DEBUG source=payload.go:180 msg=extracting variant=cpu_avx2 file=build/linux/x86_64/cpu_avx2/bin/ollama_llama_server.gz
time=2024-06-07T01:00:48.523+02:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama607982765/runners/cpu
time=2024-06-07T01:00:48.523+02:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama607982765/runners/cpu_avx
time=2024-06-07T01:00:48.523+02:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama607982765/runners/cpu_avx2
time=2024-06-07T01:00:48.523+02:00 level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2]"
time=2024-06-07T01:00:48.523+02:00 level=DEBUG source=payload.go:45 msg="Override detection logic by setting OLLAMA_LLM_LIBRARY"
time=2024-06-07T01:00:48.523+02:00 level=DEBUG source=sched.go:90 msg="starting llm scheduler"
time=2024-06-07T01:00:48.523+02:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs"
time=2024-06-07T01:00:48.523+02:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so*
time=2024-06-07T01:00:48.523+02:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/opt/intel/oneapi/mkl/2024.0/lib/libcuda.so** /opt/intel/oneapi/compiler/2024.0/lib/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
time=2024-06-07T01:00:50.186+02:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[]
time=2024-06-07T01:00:50.186+02:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcudart.so*
time=2024-06-07T01:00:50.186+02:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/opt/intel/oneapi/mkl/2024.0/lib/libcudart.so** /opt/intel/oneapi/compiler/2024.0/lib/libcudart.so** /tmp/ollama607982765/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
time=2024-06-07T01:00:51.690+02:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/local/cuda/lib64/libcudart.so.12.5.39]
cudaSetDevice err: 35
time=2024-06-07T01:00:51.690+02:00 level=DEBUG source=gpu.go:325 msg="Unable to load cudart" library=/usr/local/cuda/lib64/libcudart.so.12.5.39 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
time=2024-06-07T01:00:51.690+02:00 level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-06-07T01:00:51.690+02:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu"
time=2024-06-07T01:00:51.690+02:00 level=INFO source=types.go:71 msg="inference compute" id=0 library=cpu compute="" driver=0.0 name="" total="7.6 GiB" available="3.6 GiB"```


### OS

WSL2

### GPU

Intel

### CPU

Intel

### Ollama version

0.1.39
Originally created by @filip-777 on GitHub (Jun 6, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/4884 ### What is the issue? After setup of IPEX-LLM to work with ollama I see that output is wrong. Example: ``` ❯ ./ollama run phi3 >>> hi <s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s> ``` Maybe I had setup something wrong... When I serve ollama I have such logs: ```❯ ./ollama serve 2024/06/07 01:00:48 routes.go:1028: INFO server config env="map[OLLAMA_DEBUG:true OLLAMA_FLASH_ATTENTION:false OLLAMA_HOST: OLLAMA_KEEP_ALIVE: OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:1 OLLAMA_MAX_QUEUE:512 OLLAMA_MAX_VRAM:0 OLLAMA_MODELS: OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:*] OLLAMA_RUNNERS_DIR: OLLAMA_TMPDIR:]" time=2024-06-07T01:00:48.471+02:00 level=INFO source=images.go:729 msg="total blobs: 38" time=2024-06-07T01:00:48.471+02:00 level=INFO source=images.go:736 msg="total unused blobs removed: 0" [GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached. [GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production. - using env: export GIN_MODE=release - using code: gin.SetMode(gin.ReleaseMode) [GIN-debug] POST /api/pull --> github.com/ollama/ollama/server.(*Server).PullModelHandler-fm (5 handlers) [GIN-debug] POST /api/generate --> github.com/ollama/ollama/server.(*Server).GenerateHandler-fm (5 handlers) [GIN-debug] POST /api/chat --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (5 handlers) [GIN-debug] POST /api/embeddings --> github.com/ollama/ollama/server.(*Server).EmbeddingsHandler-fm (5 handlers) [GIN-debug] POST /api/create --> github.com/ollama/ollama/server.(*Server).CreateModelHandler-fm (5 handlers) [GIN-debug] POST /api/push --> github.com/ollama/ollama/server.(*Server).PushModelHandler-fm (5 handlers) [GIN-debug] POST /api/copy --> github.com/ollama/ollama/server.(*Server).CopyModelHandler-fm (5 handlers) [GIN-debug] DELETE /api/delete --> github.com/ollama/ollama/server.(*Server).DeleteModelHandler-fm (5 handlers) [GIN-debug] POST /api/show --> github.com/ollama/ollama/server.(*Server).ShowModelHandler-fm (5 handlers) [GIN-debug] POST /api/blobs/:digest --> github.com/ollama/ollama/server.(*Server).CreateBlobHandler-fm (5 handlers) [GIN-debug] HEAD /api/blobs/:digest --> github.com/ollama/ollama/server.(*Server).HeadBlobHandler-fm (5 handlers) [GIN-debug] GET /api/ps --> github.com/ollama/ollama/server.(*Server).ProcessHandler-fm (5 handlers) [GIN-debug] POST /v1/chat/completions --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (6 handlers) [GIN-debug] GET / --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers) [GIN-debug] GET /api/tags --> github.com/ollama/ollama/server.(*Server).ListModelsHandler-fm (5 handlers) [GIN-debug] GET /api/version --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers) [GIN-debug] HEAD / --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers) [GIN-debug] HEAD /api/tags --> github.com/ollama/ollama/server.(*Server).ListModelsHandler-fm (5 handlers) [GIN-debug] HEAD /api/version --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers) time=2024-06-07T01:00:48.471+02:00 level=INFO source=routes.go:1074 msg="Listening on 127.0.0.1:11434 (version 0.0.0)" time=2024-06-07T01:00:48.472+02:00 level=INFO source=payload.go:30 msg="extracting embedded files" dir=/tmp/ollama607982765/runners time=2024-06-07T01:00:48.472+02:00 level=DEBUG source=payload.go:180 msg=extracting variant=cpu file=build/linux/x86_64/cpu/bin/ollama_llama_server.gz time=2024-06-07T01:00:48.472+02:00 level=DEBUG source=payload.go:180 msg=extracting variant=cpu_avx file=build/linux/x86_64/cpu_avx/bin/ollama_llama_server.gz time=2024-06-07T01:00:48.472+02:00 level=DEBUG source=payload.go:180 msg=extracting variant=cpu_avx2 file=build/linux/x86_64/cpu_avx2/bin/ollama_llama_server.gz time=2024-06-07T01:00:48.523+02:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama607982765/runners/cpu time=2024-06-07T01:00:48.523+02:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama607982765/runners/cpu_avx time=2024-06-07T01:00:48.523+02:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama607982765/runners/cpu_avx2 time=2024-06-07T01:00:48.523+02:00 level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2]" time=2024-06-07T01:00:48.523+02:00 level=DEBUG source=payload.go:45 msg="Override detection logic by setting OLLAMA_LLM_LIBRARY" time=2024-06-07T01:00:48.523+02:00 level=DEBUG source=sched.go:90 msg="starting llm scheduler" time=2024-06-07T01:00:48.523+02:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs" time=2024-06-07T01:00:48.523+02:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so* time=2024-06-07T01:00:48.523+02:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/opt/intel/oneapi/mkl/2024.0/lib/libcuda.so** /opt/intel/oneapi/compiler/2024.0/lib/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" time=2024-06-07T01:00:50.186+02:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[] time=2024-06-07T01:00:50.186+02:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcudart.so* time=2024-06-07T01:00:50.186+02:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/opt/intel/oneapi/mkl/2024.0/lib/libcudart.so** /opt/intel/oneapi/compiler/2024.0/lib/libcudart.so** /tmp/ollama607982765/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]" time=2024-06-07T01:00:51.690+02:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/local/cuda/lib64/libcudart.so.12.5.39] cudaSetDevice err: 35 time=2024-06-07T01:00:51.690+02:00 level=DEBUG source=gpu.go:325 msg="Unable to load cudart" library=/usr/local/cuda/lib64/libcudart.so.12.5.39 error="your nvidia driver is too old or missing. If you have a CUDA GPU please upgrade to run ollama" time=2024-06-07T01:00:51.690+02:00 level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2" time=2024-06-07T01:00:51.690+02:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu" time=2024-06-07T01:00:51.690+02:00 level=INFO source=types.go:71 msg="inference compute" id=0 library=cpu compute="" driver=0.0 name="" total="7.6 GiB" available="3.6 GiB"``` ### OS WSL2 ### GPU Intel ### CPU Intel ### Ollama version 0.1.39
GiteaMirror added the bug label 2026-04-12 13:31:09 -05:00
GiteaMirror commented 2026-04-12 13:31:13 -05:00
Author
Owner
Copy Link

@avimanyu786 commented on GitHub (Oct 7, 2024):

The primary reason for this bug seems to be the OS, which is WSL2.

To resolve this issue, perhaps using the windows preview installer would suffice.

This guide for IPEX-LLM would be helpful as well.

<!-- gh-comment-id:2395793985 --> @avimanyu786 commented on GitHub (Oct 7, 2024): The primary reason for this bug seems to be the OS, which is WSL2. To resolve this issue, perhaps using the [windows preview installer](https://ollama.com/download/OllamaSetup.exe) would suffice. This [guide for IPEX-LLM](https://github.com/intel-analytics/ipex-llm/blob/main/docs/mddocs/Quickstart/install_windows_gpu.md) would be helpful as well.
GiteaMirror commented 2026-04-12 13:31:15 -05:00
Author
Owner
Copy Link

@desmondsow commented on GitHub (May 28, 2025):

You may try the latest release https://github.com/ipex-llm/ipex-llm/blob/v2.3.0-nightly/docs/mddocs/Quickstart/ollama_quickstart.md for GPU setup. This should resolve the issue.

<!-- gh-comment-id:2915020492 --> @desmondsow commented on GitHub (May 28, 2025): You may try the latest release https://github.com/ipex-llm/ipex-llm/blob/v2.3.0-nightly/docs/mddocs/Quickstart/ollama_quickstart.md for GPU setup. This should resolve the issue.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#3081
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?