github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-15 12:48:05 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #8239] GPU is not being used on macOS when launching from CLI #5261

New Issue
Closed
opened 2026-04-12 16:24:23 -05:00 by GiteaMirror · 10 comments
GiteaMirror commented 2026-04-12 16:24:23 -05:00
Owner
Copy Link

Originally created by @Bhavya031 on GitHub (Dec 25, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/8239

What is the issue?

On macOS, if you use Ollama, it utilizes the GPU. However, when launching via CLI, it does not. I searched for GPU flags but couldn’t find any. We need default GPU support for macOS when using the CLI.

https://github.com/user-attachments/assets/26fd9f8a-94f8-458f-8482-bbb96ab40697

OS

macOS

GPU

Apple

CPU

Apple

Ollama version

0.5.4

Originally created by @Bhavya031 on GitHub (Dec 25, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/8239 ### What is the issue? On macOS, if you use Ollama, it utilizes the GPU. However, when launching via CLI, it does not. I searched for GPU flags but couldn’t find any. We need default GPU support for macOS when using the CLI. https://github.com/user-attachments/assets/26fd9f8a-94f8-458f-8482-bbb96ab40697 ### OS macOS ### GPU Apple ### CPU Apple ### Ollama version 0.5.4
GiteaMirror added the bug label 2026-04-12 16:24:23 -05:00
GiteaMirror commented 2026-04-12 16:24:24 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Dec 26, 2024):

Server logs will aid in debugging.

<!-- gh-comment-id:2562037898 --> @rick-github commented on GitHub (Dec 26, 2024): [Server logs](https://github.com/ollama/ollama/blob/main/docs/troubleshooting.md#how-to-troubleshoot-issues) will aid in debugging.
GiteaMirror commented 2026-04-12 16:24:25 -05:00
Author
Owner
Copy Link

@Oscarjia commented on GitHub (Dec 26, 2024):

Does Mac have GPU installed? In my memory, Mac don't have GPU? Am i right?

<!-- gh-comment-id:2562384459 --> @Oscarjia commented on GitHub (Dec 26, 2024): Does Mac have GPU installed? In my memory, Mac don't have GPU? Am i right?
GiteaMirror commented 2026-04-12 16:24:26 -05:00
Author
Owner
Copy Link

@Bhavya031 commented on GitHub (Dec 26, 2024):

Server logs will aid in debugging.

I figured out the cause: it is using a different runner when launched from the CLI compared to the application. Here are the server logs for both cases—first from the CLI, and then from the application.

cli(look at line 39:97):

󰀵 bhavya  ~/󰲋 /Collage   15:31 

ollama serve

2024/12/26 15:31:44 routes.go:1259: INFO server config env="map[HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE:f16 OLLAMA_LLM_LIBRARY:metal OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/Users/bhavya/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:4 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://*] OLLAMA_SCHED_SPREAD:false http_proxy: https_proxy: no_proxy:]"
time=2024-12-26T15:31:44.480+05:30 level=INFO source=images.go:757 msg="total blobs: 22"
time=2024-12-26T15:31:44.480+05:30 level=INFO source=images.go:764 msg="total unused blobs removed: 0"
[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.

[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:   export GIN_MODE=release
 - using code:  gin.SetMode(gin.ReleaseMode)

[GIN-debug] POST   /api/pull                 --> github.com/ollama/ollama/server.(*Server).PullHandler-fm (5 handlers)
[GIN-debug] POST   /api/generate             --> github.com/ollama/ollama/server.(*Server).GenerateHandler-fm (5 handlers)
[GIN-debug] POST   /api/chat                 --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (5 handlers)
[GIN-debug] POST   /api/embed                --> github.com/ollama/ollama/server.(*Server).EmbedHandler-fm (5 handlers)
[GIN-debug] POST   /api/embeddings           --> github.com/ollama/ollama/server.(*Server).EmbeddingsHandler-fm (5 handlers)
[GIN-debug] POST   /api/create               --> github.com/ollama/ollama/server.(*Server).CreateHandler-fm (5 handlers)
[GIN-debug] POST   /api/push                 --> github.com/ollama/ollama/server.(*Server).PushHandler-fm (5 handlers)
[GIN-debug] POST   /api/copy                 --> github.com/ollama/ollama/server.(*Server).CopyHandler-fm (5 handlers)
[GIN-debug] DELETE /api/delete               --> github.com/ollama/ollama/server.(*Server).DeleteHandler-fm (5 handlers)
[GIN-debug] POST   /api/show                 --> github.com/ollama/ollama/server.(*Server).ShowHandler-fm (5 handlers)
[GIN-debug] POST   /api/blobs/:digest        --> github.com/ollama/ollama/server.(*Server).CreateBlobHandler-fm (5 handlers)
[GIN-debug] HEAD   /api/blobs/:digest        --> github.com/ollama/ollama/server.(*Server).HeadBlobHandler-fm (5 handlers)
[GIN-debug] GET    /api/ps                   --> github.com/ollama/ollama/server.(*Server).PsHandler-fm (5 handlers)
[GIN-debug] POST   /v1/chat/completions      --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (6 handlers)
[GIN-debug] POST   /v1/completions           --> github.com/ollama/ollama/server.(*Server).GenerateHandler-fm (6 handlers)
[GIN-debug] POST   /v1/embeddings            --> github.com/ollama/ollama/server.(*Server).EmbedHandler-fm (6 handlers)
[GIN-debug] GET    /v1/models                --> github.com/ollama/ollama/server.(*Server).ListHandler-fm (6 handlers)
[GIN-debug] GET    /v1/models/:model         --> github.com/ollama/ollama/server.(*Server).ShowHandler-fm (6 handlers)
[GIN-debug] GET    /                         --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers)
[GIN-debug] GET    /api/tags                 --> github.com/ollama/ollama/server.(*Server).ListHandler-fm (5 handlers)
[GIN-debug] GET    /api/version              --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers)
[GIN-debug] HEAD   /                         --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers)
[GIN-debug] HEAD   /api/tags                 --> github.com/ollama/ollama/server.(*Server).ListHandler-fm (5 handlers)
[GIN-debug] HEAD   /api/version              --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers)
time=2024-12-26T15:31:44.482+05:30 level=INFO source=routes.go:1310 msg="Listening on 127.0.0.1:11434 (version 0.5.4)"
time=2024-12-26T15:31:44.483+05:30 level=INFO source=routes.go:1339 msg="Dynamic LLM libraries" runners=[cpu]
time=2024-12-26T15:31:44.483+05:30 level=INFO source=types.go:131 msg="inference compute" id="" library=cpu variant="no vector extensions" compute="" driver=0.0 name="" total="16.0 GiB" available="6.4 GiB"

ollama.app(look at line 34:97):

time=2024-12-26T15:35:15.806+05:30 level=INFO source=images.go:757 msg="total blobs: 22"
time=2024-12-26T15:35:15.806+05:30 level=INFO source=images.go:764 msg="total unused blobs removed: 0"
[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.

[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:   export GIN_MODE=release
 - using code:  gin.SetMode(gin.ReleaseMode)
[GIN-debug] POST   /api/pull                 --> github.com/ollama/ollama/server.(*Server).PullHandler-fm (5 handlers)
[GIN-debug] POST   /api/generate             --> github.com/ollama/ollama/server.(*Server).GenerateHandler-fm (5 handlers)
[GIN-debug] POST   /api/chat                 --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (5 handlers)
[GIN-debug] POST   /api/embed                --> github.com/ollama/ollama/server.(*Server).EmbedHandler-fm (5 handlers)
[GIN-debug] POST   /api/embeddings           --> github.com/ollama/ollama/server.(*Server).EmbeddingsHandler-fm (5 handlers)
[GIN-debug] POST   /api/create               --> github.com/ollama/ollama/server.(*Server).CreateHandler-fm (5 handlers)
[GIN-debug] POST   /api/push                 --> github.com/ollama/ollama/server.(*Server).PushHandler-fm (5 handlers)
[GIN-debug] POST   /api/copy                 --> github.com/ollama/ollama/server.(*Server).CopyHandler-fm (5 handlers)
[GIN-debug] DELETE /api/delete               --> github.com/ollama/ollama/server.(*Server).DeleteHandler-fm (5 handlers)
[GIN-debug] POST   /api/show                 --> github.com/ollama/ollama/server.(*Server).ShowHandler-fm (5 handlers)
[GIN-debug] POST   /api/blobs/:digest        --> github.com/ollama/ollama/server.(*Server).CreateBlobHandler-fm (5 handlers)
[GIN-debug] HEAD   /api/blobs/:digest        --> github.com/ollama/ollama/server.(*Server).HeadBlobHandler-fm (5 handlers)
[GIN-debug] GET    /api/ps                   --> github.com/ollama/ollama/server.(*Server).PsHandler-fm (5 handlers)
[GIN-debug] POST   /v1/chat/completions      --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (6 handlers)
[GIN-debug] POST   /v1/completions           --> github.com/ollama/ollama/server.(*Server).GenerateHandler-fm (6 handlers)
[GIN-debug] POST   /v1/embeddings            --> github.com/ollama/ollama/server.(*Server).EmbedHandler-fm (6 handlers)
[GIN-debug] GET    /v1/models                --> github.com/ollama/ollama/server.(*Server).ListHandler-fm (6 handlers)
[GIN-debug] GET    /v1/models/:model         --> github.com/ollama/ollama/server.(*Server).ShowHandler-fm (6 handlers)
[GIN-debug] GET    /                         --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers)
[GIN-debug] GET    /api/tags                 --> github.com/ollama/ollama/server.(*Server).ListHandler-fm (5 handlers)
[GIN-debug] GET    /api/version              --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers)
[GIN-debug] HEAD   /                         --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers)
[GIN-debug] HEAD   /api/tags                 --> github.com/ollama/ollama/server.(*Server).ListHandler-fm (5 handlers)
[GIN-debug] HEAD   /api/version              --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers)
time=2024-12-26T15:35:15.807+05:30 level=INFO source=routes.go:1310 msg="Listening on 127.0.0.1:11434 (version 0.5.4)"
time=2024-12-26T15:35:15.807+05:30 level=INFO source=routes.go:1339 msg="Dynamic LLM libraries" runners="[metal cpu_avx cpu_avx2]"
time=2024-12-26T15:35:15.873+05:30 level=INFO source=types.go:131 msg="inference compute" id=0 library=metal variant="" compute="" driver=0.0 name="" total="10.7 GiB" available="10.7 GiB"

As you can see, the application uses Metal, CPU_AVX, and CPU_AVX2, while the CLI uses only CPU. This issue originates from routes.go. If you don’t mind, I would like to work on solving it.

<!-- gh-comment-id:2562396710 --> @Bhavya031 commented on GitHub (Dec 26, 2024): > [Server logs](https://github.com/ollama/ollama/blob/main/docs/troubleshooting.md#how-to-troubleshoot-issues) will aid in debugging. I figured out the cause: it is using a different runner when launched from the CLI compared to the application. Here are the server logs for both cases—first from the CLI, and then from the application. ### cli(look at line 39:97): ``` 󰀵 bhavya  ~/󰲋 /Collage   15:31   ollama serve 2024/12/26 15:31:44 routes.go:1259: INFO server config env="map[HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE:f16 OLLAMA_LLM_LIBRARY:metal OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/Users/bhavya/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:4 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://*] OLLAMA_SCHED_SPREAD:false http_proxy: https_proxy: no_proxy:]" time=2024-12-26T15:31:44.480+05:30 level=INFO source=images.go:757 msg="total blobs: 22" time=2024-12-26T15:31:44.480+05:30 level=INFO source=images.go:764 msg="total unused blobs removed: 0" [GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached. [GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production. - using env: export GIN_MODE=release - using code: gin.SetMode(gin.ReleaseMode) [GIN-debug] POST /api/pull --> github.com/ollama/ollama/server.(*Server).PullHandler-fm (5 handlers) [GIN-debug] POST /api/generate --> github.com/ollama/ollama/server.(*Server).GenerateHandler-fm (5 handlers) [GIN-debug] POST /api/chat --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (5 handlers) [GIN-debug] POST /api/embed --> github.com/ollama/ollama/server.(*Server).EmbedHandler-fm (5 handlers) [GIN-debug] POST /api/embeddings --> github.com/ollama/ollama/server.(*Server).EmbeddingsHandler-fm (5 handlers) [GIN-debug] POST /api/create --> github.com/ollama/ollama/server.(*Server).CreateHandler-fm (5 handlers) [GIN-debug] POST /api/push --> github.com/ollama/ollama/server.(*Server).PushHandler-fm (5 handlers) [GIN-debug] POST /api/copy --> github.com/ollama/ollama/server.(*Server).CopyHandler-fm (5 handlers) [GIN-debug] DELETE /api/delete --> github.com/ollama/ollama/server.(*Server).DeleteHandler-fm (5 handlers) [GIN-debug] POST /api/show --> github.com/ollama/ollama/server.(*Server).ShowHandler-fm (5 handlers) [GIN-debug] POST /api/blobs/:digest --> github.com/ollama/ollama/server.(*Server).CreateBlobHandler-fm (5 handlers) [GIN-debug] HEAD /api/blobs/:digest --> github.com/ollama/ollama/server.(*Server).HeadBlobHandler-fm (5 handlers) [GIN-debug] GET /api/ps --> github.com/ollama/ollama/server.(*Server).PsHandler-fm (5 handlers) [GIN-debug] POST /v1/chat/completions --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (6 handlers) [GIN-debug] POST /v1/completions --> github.com/ollama/ollama/server.(*Server).GenerateHandler-fm (6 handlers) [GIN-debug] POST /v1/embeddings --> github.com/ollama/ollama/server.(*Server).EmbedHandler-fm (6 handlers) [GIN-debug] GET /v1/models --> github.com/ollama/ollama/server.(*Server).ListHandler-fm (6 handlers) [GIN-debug] GET /v1/models/:model --> github.com/ollama/ollama/server.(*Server).ShowHandler-fm (6 handlers) [GIN-debug] GET / --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers) [GIN-debug] GET /api/tags --> github.com/ollama/ollama/server.(*Server).ListHandler-fm (5 handlers) [GIN-debug] GET /api/version --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers) [GIN-debug] HEAD / --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers) [GIN-debug] HEAD /api/tags --> github.com/ollama/ollama/server.(*Server).ListHandler-fm (5 handlers) [GIN-debug] HEAD /api/version --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers) time=2024-12-26T15:31:44.482+05:30 level=INFO source=routes.go:1310 msg="Listening on 127.0.0.1:11434 (version 0.5.4)" time=2024-12-26T15:31:44.483+05:30 level=INFO source=routes.go:1339 msg="Dynamic LLM libraries" runners=[cpu] time=2024-12-26T15:31:44.483+05:30 level=INFO source=types.go:131 msg="inference compute" id="" library=cpu variant="no vector extensions" compute="" driver=0.0 name="" total="16.0 GiB" available="6.4 GiB" ``` ### ollama.app(look at line 34:97): ```2024/12/26 15:35:15 routes.go:1259: INFO server config env="map[HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/Users/bhavya/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://*] OLLAMA_SCHED_SPREAD:false http_proxy: https_proxy: no_proxy:]" time=2024-12-26T15:35:15.806+05:30 level=INFO source=images.go:757 msg="total blobs: 22" time=2024-12-26T15:35:15.806+05:30 level=INFO source=images.go:764 msg="total unused blobs removed: 0" [GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached. [GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production. - using env: export GIN_MODE=release - using code: gin.SetMode(gin.ReleaseMode) [GIN-debug] POST /api/pull --> github.com/ollama/ollama/server.(*Server).PullHandler-fm (5 handlers) [GIN-debug] POST /api/generate --> github.com/ollama/ollama/server.(*Server).GenerateHandler-fm (5 handlers) [GIN-debug] POST /api/chat --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (5 handlers) [GIN-debug] POST /api/embed --> github.com/ollama/ollama/server.(*Server).EmbedHandler-fm (5 handlers) [GIN-debug] POST /api/embeddings --> github.com/ollama/ollama/server.(*Server).EmbeddingsHandler-fm (5 handlers) [GIN-debug] POST /api/create --> github.com/ollama/ollama/server.(*Server).CreateHandler-fm (5 handlers) [GIN-debug] POST /api/push --> github.com/ollama/ollama/server.(*Server).PushHandler-fm (5 handlers) [GIN-debug] POST /api/copy --> github.com/ollama/ollama/server.(*Server).CopyHandler-fm (5 handlers) [GIN-debug] DELETE /api/delete --> github.com/ollama/ollama/server.(*Server).DeleteHandler-fm (5 handlers) [GIN-debug] POST /api/show --> github.com/ollama/ollama/server.(*Server).ShowHandler-fm (5 handlers) [GIN-debug] POST /api/blobs/:digest --> github.com/ollama/ollama/server.(*Server).CreateBlobHandler-fm (5 handlers) [GIN-debug] HEAD /api/blobs/:digest --> github.com/ollama/ollama/server.(*Server).HeadBlobHandler-fm (5 handlers) [GIN-debug] GET /api/ps --> github.com/ollama/ollama/server.(*Server).PsHandler-fm (5 handlers) [GIN-debug] POST /v1/chat/completions --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (6 handlers) [GIN-debug] POST /v1/completions --> github.com/ollama/ollama/server.(*Server).GenerateHandler-fm (6 handlers) [GIN-debug] POST /v1/embeddings --> github.com/ollama/ollama/server.(*Server).EmbedHandler-fm (6 handlers) [GIN-debug] GET /v1/models --> github.com/ollama/ollama/server.(*Server).ListHandler-fm (6 handlers) [GIN-debug] GET /v1/models/:model --> github.com/ollama/ollama/server.(*Server).ShowHandler-fm (6 handlers) [GIN-debug] GET / --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers) [GIN-debug] GET /api/tags --> github.com/ollama/ollama/server.(*Server).ListHandler-fm (5 handlers) [GIN-debug] GET /api/version --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers) [GIN-debug] HEAD / --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers) [GIN-debug] HEAD /api/tags --> github.com/ollama/ollama/server.(*Server).ListHandler-fm (5 handlers) [GIN-debug] HEAD /api/version --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers) time=2024-12-26T15:35:15.807+05:30 level=INFO source=routes.go:1310 msg="Listening on 127.0.0.1:11434 (version 0.5.4)" time=2024-12-26T15:35:15.807+05:30 level=INFO source=routes.go:1339 msg="Dynamic LLM libraries" runners="[metal cpu_avx cpu_avx2]" time=2024-12-26T15:35:15.873+05:30 level=INFO source=types.go:131 msg="inference compute" id=0 library=metal variant="" compute="" driver=0.0 name="" total="10.7 GiB" available="10.7 GiB" ``` As you can see, the application uses Metal, CPU_AVX, and CPU_AVX2, while the CLI uses only CPU. This issue originates from routes.go. If you don’t mind, I would like to work on solving it.
GiteaMirror commented 2026-04-12 16:24:26 -05:00
Author
Owner
Copy Link

@Bhavya031 commented on GitHub (Dec 26, 2024):

Does Mac have GPU installed? In my memory, Mac don't have GPU? Am i right?

hmm every mac has a gpu even intal one new macs from m2 uses apple slicon gpus which uses matel framwork.

<!-- gh-comment-id:2562399620 --> @Bhavya031 commented on GitHub (Dec 26, 2024): > Does Mac have GPU installed? In my memory, Mac don't have GPU? Am i right? hmm every mac has a gpu even intal one new macs from m2 uses apple slicon gpus which uses [matel](https://developer.apple.com/metal/) framwork.
GiteaMirror commented 2026-04-12 16:24:27 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Dec 26, 2024):

What are you trying to achieve by running ollama serve?

<!-- gh-comment-id:2562444589 --> @rick-github commented on GitHub (Dec 26, 2024): What are you trying to achieve by running `ollama serve`?
GiteaMirror commented 2026-04-12 16:24:27 -05:00
Author
Owner
Copy Link

@Bhavya031 commented on GitHub (Dec 26, 2024):

What are you trying to achieve by running ollama serve?

The main reason is that I have some Tmuxifier sessions to automate the starting point in the terminal, which is more convenient for me. Another reason, which might only be a problem for me, is that if I use Ollama from the application, I can’t access it using curl and Python. I also have to reset Ollama more frequently. However, using it from the CLI works well for me.

<!-- gh-comment-id:2562603255 --> @Bhavya031 commented on GitHub (Dec 26, 2024): > What are you trying to achieve by running `ollama serve`? The main reason is that I have some Tmuxifier sessions to automate the starting point in the terminal, which is more convenient for me. Another reason, which might only be a problem for me, is that if I use Ollama from the application, I can’t access it using curl and Python. I also have to reset Ollama more frequently. However, using it from the CLI works well for me.
GiteaMirror commented 2026-04-12 16:24:28 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Dec 26, 2024):

I'm not a Mac user so the following may be inaccurate. The ollama bundle includes the ollama server and the runners. When the server starts, it looks around for the runners, usually in a path relative to where the server binary is located. If you run the server with OLLAMA_DEBUG=1 in the environment, you will log lines from the search. I speculate that when you run ollama serve, the server is not looking in the right place and can't locate the runners.

<!-- gh-comment-id:2562741046 --> @rick-github commented on GitHub (Dec 26, 2024): I'm not a Mac user so the following may be inaccurate. The ollama bundle includes the ollama server and the runners. When the server starts, it looks around for the runners, usually in a path relative to where the server binary is located. If you run the server with `OLLAMA_DEBUG=1` in the environment, you will log lines from the search. I speculate that when you run `ollama serve`, the server is not looking in the right place and can't locate the runners.
GiteaMirror commented 2026-04-12 16:24:28 -05:00
Author
Owner
Copy Link

@Bhavya031 commented on GitHub (Dec 26, 2024):

I'm not a Mac user so the following may be inaccurate. The ollama bundle includes the ollama server and the runners. When the server starts, it looks around for the runners, usually in a path relative to where the server binary is located. If you run the server with OLLAMA_DEBUG=1 in the environment, you will log lines from the search. I speculate that when you run ollama serve, the server is not looking in the right place and can't locate the runners.

So here’s what I found: the file common.go determines which architecture to use, but it calls config.go to get the runner names. However, it doesn’t have any Darwin runner names. You can check that there’s also a function in common.go (link), but it doesn’t have support for Metal.

That said, I don’t think this is entirely relevant. The main relevant part is that I compiled the code myself, and aside from Metal, everything else is being detected. This block of code isn’t being used for some reason.023e4bca14/runners/common.go (L90)

Also, @rick-github, I’m trying to work on solving some real-life problems, and I came across this one while exploring Ollama’s architecture. If I open a PR for this, would it be accepted?

<!-- gh-comment-id:2562775660 --> @Bhavya031 commented on GitHub (Dec 26, 2024): > I'm not a Mac user so the following may be inaccurate. The ollama bundle includes the ollama server and the runners. When the server starts, it looks around for the runners, usually in a path relative to where the server binary is located. If you run the server with `OLLAMA_DEBUG=1` in the environment, you will log lines from the search. I speculate that when you run `ollama serve`, the server is not looking in the right place and can't locate the runners. So here’s what I found: the file [common.go](https://github.com/ollama/ollama/blob/main/runners/common.go) determines which architecture to use, but it calls [config.go](https://github.com/ollama/ollama/blob/main/envconfig/config.go) to get the runner names. However, it doesn’t have any Darwin runner names. You can check that there’s also a function in common.go ([link](https://github.com/ollama/ollama/blob/023e4bca144da199e6a6dc11d04c9e23623c9482/runners/common.go#L101)), but it doesn’t have support for Metal. That said, I don’t think this is entirely relevant. The main relevant part is that I compiled the code myself, and aside from Metal, everything else is being detected. This block of code isn’t being used for some reason.https://github.com/ollama/ollama/blob/023e4bca144da199e6a6dc11d04c9e23623c9482/runners/common.go#L90 Also, @rick-github, I’m trying to work on solving some real-life problems, and I came across this one while exploring Ollama’s architecture. If I open a PR for this, would it be accepted?
GiteaMirror commented 2026-04-12 16:24:28 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Dec 26, 2024):

You'll have to clear about what the PR is going to do. When run as an app, as is the usual way to run it, ollama detects the runners. So the issue is in the way ollama is being invoked from the command line. My understanding is that MacOS apps are an embedded mini-filesystem, so the command line invocation needs to access the app internals in order to find the runners and then make them available to the ollama server. This may fall afoul of the package signing that is supposed to protect the integrity of the app. But, not a MacOS user. so I may be mistaken. If you generate a PR that fixes a real problem, then there's a good chance it will be merged, but it's usually not a quick process.

<!-- gh-comment-id:2562861824 --> @rick-github commented on GitHub (Dec 26, 2024): You'll have to clear about what the PR is going to do. When run as an app, as is the usual way to run it, ollama detects the runners. So the issue is in the way ollama is being invoked from the command line. My understanding is that MacOS apps are an embedded mini-filesystem, so the command line invocation needs to access the app internals in order to find the runners and then make them available to the ollama server. This may fall afoul of the package signing that is supposed to protect the integrity of the app. But, not a MacOS user. so I may be mistaken. If you generate a PR that fixes a real problem, then there's a good chance it will be merged, but it's usually not a quick process.
GiteaMirror commented 2026-04-12 16:24:29 -05:00
Author
Owner
Copy Link

@Bhavya031 commented on GitHub (Dec 27, 2024):

You'll have to clear about what the PR is going to do. When run as an app, as is the usual way to run it, ollama detects the runners. So the issue is in the way ollama is being invoked from the command line. My understanding is that MacOS apps are an embedded mini-filesystem, so the command line invocation needs to access the app internals in order to find the runners and then make them available to the ollama server. This may fall afoul of the package signing that is supposed to protect the integrity of the app. But, not a MacOS user. so I may be mistaken. If you generate a PR that fixes a real problem, then there's a good chance it will be merged, but it's usually not a quick process.

So, I found the problem, and it wasn’t about runner detection. It was about where I installed it from. I installed it using Brew, but what I didn’t realize was that after installing the application, it was not using /opt/homebrew/bin/brew for Ollama but rather /usr/local/bin/brew.

Using both the Brew located in /opt/homebrew/bin/brew and the application solves the problem. I think this happened because the two versions are compiled differently—one for general use and the other specifically for Metal devices. Sorry for wasting your time!

<!-- gh-comment-id:2563538766 --> @Bhavya031 commented on GitHub (Dec 27, 2024): > You'll have to clear about what the PR is going to do. When run as an app, as is the usual way to run it, ollama detects the runners. So the issue is in the way ollama is being invoked from the command line. My understanding is that MacOS apps are an embedded mini-filesystem, so the command line invocation needs to access the app internals in order to find the runners and then make them available to the ollama server. This may fall afoul of the package signing that is supposed to protect the integrity of the app. But, not a MacOS user. so I may be mistaken. If you generate a PR that fixes a real problem, then there's a good chance it will be merged, but it's usually not a quick process. So, I found the problem, and it wasn’t about runner detection. It was about where I installed it from. I installed it using Brew, but what I didn’t realize was that after installing the application, it was not using /opt/homebrew/bin/brew for Ollama but rather /usr/local/bin/brew. Using both the Brew located in /opt/homebrew/bin/brew and the application solves the problem. I think this happened because the two versions are compiled differently—one for general use and the other specifically for Metal devices. Sorry for wasting your time!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#5261
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?