github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-15 20:54:54 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #15643] Apple M5 backend/runtime compatibility issue in Ollama #72039

New Issue
Open
opened 2026-05-05 03:23:12 -05:00 by GiteaMirror · 2 comments
GiteaMirror commented 2026-05-05 03:23:12 -05:00
Owner
Copy Link

Originally created by @yengda on GitHub (Apr 17, 2026).
Original GitHub issue: https://github.com/ollama/ollama/issues/15643

Summary

On Apple M5, Ollama fails to run qwen2.5-coder:14b due to a Metal backend initialization crash.
The server starts and /api/tags works, but /api/generate fails when loading the model.

Environment

  • Ollama: 0.20.7 (also reproduced with 0.21.0 binary from cached updater bundle)
  • OS: macOS 15.2 (Darwin 25.2.0)
  • Hardware: Apple M5
  • RAM: 32 GB
  • Model: qwen2.5-coder:14b

Repro Steps

  1. Ensure no existing instance:
    pkill -f "Ollama|ollama" || true
  2. Start server:
    OLLAMA_DEBUG=1 ollama serve > /tmp/ollama-m5.log 2>&1 &
  3. Confirm server readiness:
    curl -s http://127.0.0.1:11434/api/tags
  4. Trigger generation:
    curl -s http://127.0.0.1:11434/api/generate -d '{"model":"qwen2.5-coder:14b","prompt":"hello","stream":false}'

Expected

Model should load and return a normal response.

Actual

Returns server error:

  • {"error":"llama runner process has terminated: %!w()"}

Key Log Evidence

  • ggml_metal_init: picking default device: Apple M5
  • ggml_metal_init: the device does not have a precompiled Metal library - this is unexpected
  • ggml_metal_library_init: error: Error Domain=MTLLibraryErrorDomain Code=3
  • static_assert failed due to requirement '__tensor_ops_detail::__is_same_v<bfloat, half>' "Input types must match cooperative tensor types"
  • ggml_metal_init: error: failed to initialize the Metal library
  • llama_init_from_model: failed to initialize the context: failed to initialize Metal backend
  • panic: unable to create llama context
  • llama runner terminated (exit status 2)

Additional Notes

  • OLLAMA_LLM_LIBRARY=cpu was also tested, but the run path still attempts Metal initialization and fails with the same signature.
  • This appears to be a backend compatibility/regression issue on Apple M5 Metal path.

Control Tests

  • Same model works outside Ollama:
    • Using llama.cpp llama-server with the exact same GGUF blob from ~/.ollama/models/blobs/... serves requests successfully on /v1/chat/completions.
    • This indicates the model artifact itself is valid.
  • Different model also fails in Ollama:
    • tinyllama:latest was tested in Ollama and failed with the same llama runner process has terminated behavior.
    • This suggests the issue is runtime/backend-wide in Ollama on this machine, not specific to qwen2.5-coder:14b.
Originally created by @yengda on GitHub (Apr 17, 2026). Original GitHub issue: https://github.com/ollama/ollama/issues/15643 ## Summary On Apple M5, Ollama fails to run qwen2.5-coder:14b due to a Metal backend initialization crash. The server starts and /api/tags works, but /api/generate fails when loading the model. ## Environment - Ollama: 0.20.7 (also reproduced with 0.21.0 binary from cached updater bundle) - OS: macOS 15.2 (Darwin 25.2.0) - Hardware: Apple M5 - RAM: 32 GB - Model: qwen2.5-coder:14b ## Repro Steps 1. Ensure no existing instance: pkill -f "Ollama|ollama" || true 2. Start server: OLLAMA_DEBUG=1 ollama serve > /tmp/ollama-m5.log 2>&1 & 3. Confirm server readiness: curl -s http://127.0.0.1:11434/api/tags 4. Trigger generation: curl -s http://127.0.0.1:11434/api/generate -d '{"model":"qwen2.5-coder:14b","prompt":"hello","stream":false}' ## Expected Model should load and return a normal response. ## Actual Returns server error: - {"error":"llama runner process has terminated: %!w(<nil>)"} ## Key Log Evidence - ggml_metal_init: picking default device: Apple M5 - ggml_metal_init: the device does not have a precompiled Metal library - this is unexpected - ggml_metal_library_init: error: Error Domain=MTLLibraryErrorDomain Code=3 - static_assert failed due to requirement '__tensor_ops_detail::__is_same_v<bfloat, half>' "Input types must match cooperative tensor types" - ggml_metal_init: error: failed to initialize the Metal library - llama_init_from_model: failed to initialize the context: failed to initialize Metal backend - panic: unable to create llama context - llama runner terminated (exit status 2) ## Additional Notes - OLLAMA_LLM_LIBRARY=cpu was also tested, but the run path still attempts Metal initialization and fails with the same signature. - This appears to be a backend compatibility/regression issue on Apple M5 Metal path. ## Control Tests - Same model works outside Ollama: - Using llama.cpp `llama-server` with the exact same GGUF blob from `~/.ollama/models/blobs/...` serves requests successfully on `/v1/chat/completions`. - This indicates the model artifact itself is valid. - Different model also fails in Ollama: - `tinyllama:latest` was tested in Ollama and failed with the same `llama runner process has terminated` behavior. - This suggests the issue is runtime/backend-wide in Ollama on this machine, not specific to qwen2.5-coder:14b.
GiteaMirror commented 2026-05-05 03:23:17 -05:00
Author
Owner
Copy Link

@PureBlissAK commented on GitHub (Apr 18, 2026):

🤖 Automated Triage & Analysis Report

Issue: #15643
Analyzed: 2026-04-18T18:19:39.310862

Analysis

  • Type: unknown
  • Severity: medium
  • Components: unknown

Implementation Plan

  • Effort: medium
  • Steps:

This issue has been triaged and marked for implementation.

<!-- gh-comment-id:4274305260 --> @PureBlissAK commented on GitHub (Apr 18, 2026): <!-- ollama-issue-orchestrator:v1 issue:15643 --> ## 🤖 Automated Triage & Analysis Report **Issue**: #15643 **Analyzed**: 2026-04-18T18:19:39.310862 ### Analysis - **Type**: unknown - **Severity**: medium - **Components**: unknown ### Implementation Plan - **Effort**: medium - **Steps**: *This issue has been triaged and marked for implementation.*
GiteaMirror commented 2026-05-05 03:23:20 -05:00
Author
Owner
Copy Link

@asqararslonov commented on GitHub (May 3, 2026):

model failed to load on Apple M5 (Metal 4) — Metal shader half/bfloat type mismatch

Environment:

  • Apple M5 chip (Metal 4)
  • macOS Tahoe (latest)
  • Ollama 0.22.1 (stable / Ollama.app)
  • Any model (gemma4, qwen3, llama3, etc.)

Symptom

Every model fails immediately with:

Error: 500 Internal Server Error: model failed to load, this may be due to 
resource limitations or an internal error, check ollama server logs for details

This is not a RAM issue. It affects all models regardless of size.

Root Cause

The embedded Metal shader in Ollama's ggml backend fails to compile on Metal 4 (M5). Apple's MetalPerformancePrimitives framework on Metal 4 enforces strict type matching for cooperative tensor matmul — but the shader mixes half and bfloat types:

static_assert failed: "Input types must match cooperative tensor types" (half vs bfloat)
ggml_metal_library_init: error: Error Domain=MTLLibraryErrorDomain Code=3
ggml_backend_metal_device_init: error: failed to allocate context
GGML_ASSERT(backend) failed → exit status 2

The following workarounds do not help — Metal is always initialized regardless:

  • OLLAMA_METAL=0
  • OLLAMA_LLM_LIBRARY=cpu
  • OLLAMA_NEW_ENGINE=true

Fix

Install the HEAD build via Homebrew, which includes the Metal 4 shader patch:

# Quit Ollama.app first if it's running
brew unlink ollama        # only if stable was already installed via brew
brew install --HEAD ollama
brew services start ollama

The HEAD brew service automatically sets OLLAMA_FLASH_ATTENTION=1 and
OLLAMA_KV_CACHE_TYPE=q8_0 in the launchd plist.

Also remove Ollama.app from Login Items (System Settings → General → Login Items) to prevent conflicts with the brew service on reboot.

Verification

After the fix, all models load and run on GPU:

$ ollama ps
NAME             ID              SIZE     PROCESSOR    CONTEXT
gemma4:latest    c6eb396dbd59    10 GB    100% GPU     4096

Tested with gemma4:latest (9.6 GB) and qwen3:8b on Apple M5.

<!-- gh-comment-id:4366512095 --> @asqararslonov commented on GitHub (May 3, 2026): ## `model failed to load` on Apple M5 (Metal 4) — Metal shader `half`/`bfloat` type mismatch **Environment:** - Apple M5 chip (Metal 4) - macOS Tahoe (latest) - Ollama 0.22.1 (stable / Ollama.app) - Any model (gemma4, qwen3, llama3, etc.) --- ### Symptom Every model fails immediately with: ``` Error: 500 Internal Server Error: model failed to load, this may be due to resource limitations or an internal error, check ollama server logs for details ``` This is **not** a RAM issue. It affects all models regardless of size. --- ### Root Cause The embedded Metal shader in Ollama's ggml backend fails to compile on **Metal 4 (M5)**. Apple's `MetalPerformancePrimitives` framework on Metal 4 enforces strict type matching for cooperative tensor matmul — but the shader mixes `half` and `bfloat` types: ``` static_assert failed: "Input types must match cooperative tensor types" (half vs bfloat) ggml_metal_library_init: error: Error Domain=MTLLibraryErrorDomain Code=3 ggml_backend_metal_device_init: error: failed to allocate context GGML_ASSERT(backend) failed → exit status 2 ``` The following workarounds do **not** help — Metal is always initialized regardless: - `OLLAMA_METAL=0` - `OLLAMA_LLM_LIBRARY=cpu` - `OLLAMA_NEW_ENGINE=true` --- ### Fix Install the HEAD build via Homebrew, which includes the Metal 4 shader patch: ```bash # Quit Ollama.app first if it's running brew unlink ollama # only if stable was already installed via brew brew install --HEAD ollama brew services start ollama ``` > The HEAD brew service automatically sets `OLLAMA_FLASH_ATTENTION=1` and > `OLLAMA_KV_CACHE_TYPE=q8_0` in the launchd plist. Also remove **Ollama.app** from Login Items (**System Settings → General → Login Items**) to prevent conflicts with the brew service on reboot. --- ### Verification After the fix, all models load and run on GPU: ``` $ ollama ps NAME ID SIZE PROCESSOR CONTEXT gemma4:latest c6eb396dbd59 10 GB 100% GPU 4096 ``` Tested with `gemma4:latest` (9.6 GB) and `qwen3:8b` on Apple M5.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#72039
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?