github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-17 05:36:16 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #12131] gpt-oss-120b failing with Memory access fault by GPU node-1 #8061

New Issue
Closed
opened 2026-04-12 20:19:19 -05:00 by GiteaMirror · 3 comments
GiteaMirror commented 2026-04-12 20:19:19 -05:00
Owner
Copy Link

Originally created by @ulrichenslin on GitHub (Aug 31, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/12131

What is the issue?

After the last upgrade, the gpt-oss-120b model started failing, but only on bigger workloads.

When prompting "How far is the moon?", it works.
When including JSON or XML in the prompt, it fails with:
Memory access fault by GPU node-1 (Agent handle: 0x724828692930) on address 0x72370e95c000. Reason: Page not present or supervisor privilege.

Relevant log output

Here is rocminfo:
ROCk module version 6.12.12 is loaded
=====================    
HSA System Attributes    
=====================    
Runtime Version:         1.15
Runtime Ext Version:     1.7
System Timestamp Freq.:  1000.000000MHz
Sig. Max Wait Duration:  18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count)
Machine Model:           LARGE                              
System Endianness:       LITTLE                             
Mwaitx:                  DISABLED
XNACK enabled:           NO
DMAbuf Support:          YES
VMM Support:             YES

==========               
HSA Agents               
==========               
*******                  
Agent 1                  
*******                  
  Name:                    AMD RYZEN AI MAX+ 395 w/ Radeon 8060S
  Uuid:                    CPU-XX                             
  Marketing Name:          AMD RYZEN AI MAX+ 395 w/ Radeon 8060S
  Vendor Name:             CPU                                
  Feature:                 None specified                     
  Profile:                 FULL_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        0(0x0)                             
  Queue Min Size:          0(0x0)                             
  Queue Max Size:          0(0x0)                             
  Queue Type:              MULTI                              
  Node:                    0                                  
  Device Type:             CPU                                
  Cache Info:              
    L1:                      49152(0xc000) KB                   
  Chip ID:                 0(0x0)                             
  ASIC Revision:           0(0x0)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   5185                               
  BDFID:                   0                                  
  Internal Node ID:        0                                  
  Compute Unit:            32                                 
  SIMDs per CU:            0                                  
  Shader Engines:          0                                  
  Shader Arrs. per Eng.:   0                                  
  WatchPts on Addr. Ranges:1                                  
  Memory Properties:       
  Features:                None
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: FINE GRAINED        
      Size:                    32490472(0x1efc3e8) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    32490472(0x1efc3e8) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 3                   
      Segment:                 GLOBAL; FLAGS: KERNARG, FINE GRAINED
      Size:                    32490472(0x1efc3e8) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 4                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    32490472(0x1efc3e8) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
  ISA Info:                
*******                  
Agent 2                  
*******                  
  Name:                    gfx1151                            
  Uuid:                    GPU-XX                             
  Marketing Name:          AMD Radeon Graphics                
  Vendor Name:             AMD                                
  Feature:                 KERNEL_DISPATCH                    
  Profile:                 BASE_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        128(0x80)                          
  Queue Min Size:          64(0x40)                           
  Queue Max Size:          131072(0x20000)                    
  Queue Type:              MULTI                              
  Node:                    1                                  
  Device Type:             GPU                                
  Cache Info:              
    L1:                      32(0x20) KB                        
    L2:                      2048(0x800) KB                     
    L3:                      16384(0x4000) KB                   
  Chip ID:                 5510(0x1586)                       
  ASIC Revision:           0(0x0)                             
  Cacheline Size:          128(0x80)                          
  Max Clock Freq. (MHz):   2900                               
  BDFID:                   50432                              
  Internal Node ID:        1                                  
  Compute Unit:            40                                 
  SIMDs per CU:            2                                  
  Shader Engines:          2                                  
  Shader Arrs. per Eng.:   2                                  
  WatchPts on Addr. Ranges:4                                  
  Coherent Host Access:    FALSE                              
  Memory Properties:       APU
  Features:                KERNEL_DISPATCH 
  Fast F16 Operation:      TRUE                               
  Wavefront Size:          32(0x20)                           
  Workgroup Max Size:      1024(0x400)                        
  Workgroup Max Size per Dimension:
    x                        1024(0x400)                        
    y                        1024(0x400)                        
    z                        1024(0x400)                        
  Max Waves Per CU:        32(0x20)                           
  Max Work-item Per CU:    1024(0x400)                        
  Grid Max Size:           4294967295(0xffffffff)             
  Grid Max Size per Dimension:
    x                        4294967295(0xffffffff)             
    y                        4294967295(0xffffffff)             
    z                        4294967295(0xffffffff)             
  Max fbarriers/Workgrp:   32                                 
  Packet Processor uCode:: 26                                 
  SDMA engine uCode::      14                                 
  IOMMU Support::          None                               
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    100663296(0x6000000) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    100663296(0x6000000) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 3                   
      Segment:                 GROUP                              
      Size:                    64(0x40) KB                        
      Allocatable:             FALSE                              
      Alloc Granule:           0KB                                
      Alloc Recommended Granule:0KB                                
      Alloc Alignment:         0KB                                
      Accessible by all:       FALSE                              
  ISA Info:                
    ISA 1                    
      Name:                    amdgcn-amd-amdhsa--gfx1151         
      Machine Models:          HSA_MACHINE_MODEL_LARGE            
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                        
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                        
        y                        1024(0x400)                        
        z                        1024(0x400)                        
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        4294967295(0xffffffff)             
        y                        4294967295(0xffffffff)             
        z                        4294967295(0xffffffff)             
      FBarrier Max Size:       32                                 
    ISA 2                    
      Name:                    amdgcn-amd-amdhsa--gfx11-generic   
      Machine Models:          HSA_MACHINE_MODEL_LARGE            
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                        
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                        
        y                        1024(0x400)                        
        z                        1024(0x400)                        
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        4294967295(0xffffffff)             
        y                        4294967295(0xffffffff)             
        z                        4294967295(0xffffffff)             
      FBarrier Max Size:       32                                 
*** Done ***   

Here is the logs from Ollama:

OLLAMA_MODELS=/usr/share/ollama/.ollama/models /usr/local/bin/ollama serve 
time=2025-08-31T13:43:01.989Z level=INFO source=routes.go:1331 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:4096 OLLAMA_DEBUG:INFO OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE:q8_0 OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/usr/share/ollama/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NEW_ESTIMATES:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_SCHED_SPREAD:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"
time=2025-08-31T13:43:01.993Z level=INFO source=images.go:477 msg="total blobs: 168"
time=2025-08-31T13:43:01.994Z level=INFO source=images.go:484 msg="total unused blobs removed: 0"
time=2025-08-31T13:43:01.994Z level=INFO source=routes.go:1384 msg="Listening on 127.0.0.1:11434 (version 0.11.8)"
time=2025-08-31T13:43:01.994Z level=INFO source=gpu.go:217 msg="looking for compatible GPUs"
time=2025-08-31T13:43:02.000Z level=INFO source=amd_linux.go:389 msg="amdgpu is supported" gpu=0 gpu_type=gfx1151
time=2025-08-31T13:43:02.002Z level=INFO source=types.go:130 msg="inference compute" id=0 library=rocm variant="" compute=gfx1151 driver=6.12 name=1002:1586 total="96.0 GiB" available="95.9 GiB"
time=2025-08-31T13:43:04.850Z level=INFO source=server.go:199 msg="model wants flash attention"
time=2025-08-31T13:43:04.850Z level=INFO source=server.go:216 msg="enabling flash attention"
time=2025-08-31T13:43:04.850Z level=WARN source=ggml.go:764 msg="model only supports non-quantized cache types " mode=gptoss
time=2025-08-31T13:43:04.850Z level=WARN source=server.go:224 msg="kv cache type not supported by model" type=q8_0
time=2025-08-31T13:43:04.850Z level=INFO source=server.go:388 msg="starting runner" cmd="/usr/local/bin/ollama runner --ollama-engine --model /usr/share/ollama/.ollama/models/blobs/sha256-90a618fe6ff21b09ca968df959104eb650658b0bef0faef785c18c2795d993e3 --port 35733"
time=2025-08-31T13:43:04.851Z level=INFO source=server.go:493 msg="system memory" total="31.0 GiB" free="26.4 GiB" free_swap="6.3 GiB"
time=2025-08-31T13:43:04.851Z level=WARN source=ggml.go:764 msg="model only supports non-quantized cache types " mode=gptoss
time=2025-08-31T13:43:04.851Z level=INFO source=memory.go:36 msg="new model will fit in available VRAM across minimum required GPUs, loading" model=/usr/share/ollama/.ollama/models/blobs/sha256-90a618fe6ff21b09ca968df959104eb650658b0bef0faef785c18c2795d993e3 library=rocm parallel=1 required="62.6 GiB" gpus=1
time=2025-08-31T13:43:04.852Z level=WARN source=ggml.go:764 msg="model only supports non-quantized cache types " mode=gptoss
time=2025-08-31T13:43:04.852Z level=INFO source=server.go:533 msg=offload library=rocm layers.requested=-1 layers.model=37 layers.offload=37 layers.split=[37] memory.available="[95.9 GiB]" memory.gpu_overhead="0 B" memory.required.full="62.6 GiB" memory.required.partial="62.6 GiB" memory.required.kv="738.0 MiB" memory.required.allocations="[62.6 GiB]" memory.weights.total="59.7 GiB" memory.weights.repeating="58.6 GiB" memory.weights.nonrepeating="1.1 GiB" memory.graph.full="130.0 MiB" memory.graph.partial="130.0 MiB"
time=2025-08-31T13:43:04.859Z level=INFO source=runner.go:1006 msg="starting ollama engine"
time=2025-08-31T13:43:04.859Z level=INFO source=runner.go:1043 msg="Server listening on 127.0.0.1:35733"
time=2025-08-31T13:43:04.864Z level=INFO source=runner.go:925 msg=load request="{Operation:commit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:true KvSize:16384 KvCacheType: NumThreads:16 GPULayers:37[ID:0 Layers:37(0..36)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
time=2025-08-31T13:43:04.895Z level=INFO source=ggml.go:130 msg="" architecture=gptoss file_type=MXFP4 name="" description="" num_tensors=471 num_key_values=30
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
  Device 0: AMD Radeon Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32, ID: 0
load_backend: loaded ROCm backend from /usr/local/lib/ollama/libggml-hip.so
load_backend: loaded CPU backend from /usr/local/lib/ollama/libggml-cpu-icelake.so
time=2025-08-31T13:43:05.581Z level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.AVX512=1 CPU.0.AVX512_VBMI=1 CPU.0.AVX512_VNNI=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 ROCm.0.NO_VMM=1 ROCm.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(gcc)
time=2025-08-31T13:43:05.730Z level=INFO source=ggml.go:486 msg="offloading 36 repeating layers to GPU"
time=2025-08-31T13:43:05.730Z level=INFO source=ggml.go:492 msg="offloading output layer to GPU"
time=2025-08-31T13:43:05.730Z level=INFO source=ggml.go:497 msg="offloaded 37/37 layers to GPU"
time=2025-08-31T13:43:05.731Z level=INFO source=backend.go:310 msg="model weights" device=ROCm0 size="59.8 GiB"
time=2025-08-31T13:43:05.731Z level=INFO source=backend.go:315 msg="model weights" device=CPU size="1.1 GiB"
time=2025-08-31T13:43:05.731Z level=INFO source=backend.go:321 msg="kv cache" device=ROCm0 size="738.0 MiB"
time=2025-08-31T13:43:05.731Z level=INFO source=backend.go:332 msg="compute graph" device=ROCm0 size="129.8 MiB"
time=2025-08-31T13:43:05.731Z level=INFO source=backend.go:337 msg="compute graph" device=CPU size="5.6 MiB"
time=2025-08-31T13:43:05.731Z level=INFO source=backend.go:342 msg="total memory" size="61.7 GiB"
time=2025-08-31T13:43:05.731Z level=INFO source=sched.go:473 msg="loaded runners" count=1
time=2025-08-31T13:43:05.731Z level=INFO source=server.go:1236 msg="waiting for llama runner to start responding"
time=2025-08-31T13:43:05.732Z level=INFO source=server.go:1270 msg="waiting for server to become available" status="llm server loading model"
time=2025-08-31T13:43:18.525Z level=INFO source=server.go:1274 msg="llama runner started in 13.68 seconds"
Memory access fault by GPU node-1 (Agent handle: 0x724828692930) on address 0x72370e95c000. Reason: Page not present or supervisor privilege.
[GIN] 2025/08/31 - 13:43:37 | 500 | 32.614108095s |       127.0.0.1 | POST     "/api/generate"

OS

Linux

GPU

AMD

CPU

AMD

Ollama version

0.11.8

Originally created by @ulrichenslin on GitHub (Aug 31, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/12131 ### What is the issue? After the last upgrade, the gpt-oss-120b model started failing, but only on bigger workloads. When prompting "How far is the moon?", it works. When including JSON or XML in the prompt, it fails with: Memory access fault by GPU node-1 (Agent handle: 0x724828692930) on address 0x72370e95c000. Reason: Page not present or supervisor privilege. ### Relevant log output ```shell Here is rocminfo: ROCk module version 6.12.12 is loaded ===================== HSA System Attributes ===================== Runtime Version: 1.15 Runtime Ext Version: 1.7 System Timestamp Freq.: 1000.000000MHz Sig. Max Wait Duration: 18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count) Machine Model: LARGE System Endianness: LITTLE Mwaitx: DISABLED XNACK enabled: NO DMAbuf Support: YES VMM Support: YES ========== HSA Agents ========== ******* Agent 1 ******* Name: AMD RYZEN AI MAX+ 395 w/ Radeon 8060S Uuid: CPU-XX Marketing Name: AMD RYZEN AI MAX+ 395 w/ Radeon 8060S Vendor Name: CPU Feature: None specified Profile: FULL_PROFILE Float Round Mode: NEAR Max Queue Number: 0(0x0) Queue Min Size: 0(0x0) Queue Max Size: 0(0x0) Queue Type: MULTI Node: 0 Device Type: CPU Cache Info: L1: 49152(0xc000) KB Chip ID: 0(0x0) ASIC Revision: 0(0x0) Cacheline Size: 64(0x40) Max Clock Freq. (MHz): 5185 BDFID: 0 Internal Node ID: 0 Compute Unit: 32 SIMDs per CU: 0 Shader Engines: 0 Shader Arrs. per Eng.: 0 WatchPts on Addr. Ranges:1 Memory Properties: Features: None Pool Info: Pool 1 Segment: GLOBAL; FLAGS: FINE GRAINED Size: 32490472(0x1efc3e8) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Recommended Granule:4KB Alloc Alignment: 4KB Accessible by all: TRUE Pool 2 Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED Size: 32490472(0x1efc3e8) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Recommended Granule:4KB Alloc Alignment: 4KB Accessible by all: TRUE Pool 3 Segment: GLOBAL; FLAGS: KERNARG, FINE GRAINED Size: 32490472(0x1efc3e8) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Recommended Granule:4KB Alloc Alignment: 4KB Accessible by all: TRUE Pool 4 Segment: GLOBAL; FLAGS: COARSE GRAINED Size: 32490472(0x1efc3e8) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Recommended Granule:4KB Alloc Alignment: 4KB Accessible by all: TRUE ISA Info: ******* Agent 2 ******* Name: gfx1151 Uuid: GPU-XX Marketing Name: AMD Radeon Graphics Vendor Name: AMD Feature: KERNEL_DISPATCH Profile: BASE_PROFILE Float Round Mode: NEAR Max Queue Number: 128(0x80) Queue Min Size: 64(0x40) Queue Max Size: 131072(0x20000) Queue Type: MULTI Node: 1 Device Type: GPU Cache Info: L1: 32(0x20) KB L2: 2048(0x800) KB L3: 16384(0x4000) KB Chip ID: 5510(0x1586) ASIC Revision: 0(0x0) Cacheline Size: 128(0x80) Max Clock Freq. (MHz): 2900 BDFID: 50432 Internal Node ID: 1 Compute Unit: 40 SIMDs per CU: 2 Shader Engines: 2 Shader Arrs. per Eng.: 2 WatchPts on Addr. Ranges:4 Coherent Host Access: FALSE Memory Properties: APU Features: KERNEL_DISPATCH Fast F16 Operation: TRUE Wavefront Size: 32(0x20) Workgroup Max Size: 1024(0x400) Workgroup Max Size per Dimension: x 1024(0x400) y 1024(0x400) z 1024(0x400) Max Waves Per CU: 32(0x20) Max Work-item Per CU: 1024(0x400) Grid Max Size: 4294967295(0xffffffff) Grid Max Size per Dimension: x 4294967295(0xffffffff) y 4294967295(0xffffffff) z 4294967295(0xffffffff) Max fbarriers/Workgrp: 32 Packet Processor uCode:: 26 SDMA engine uCode:: 14 IOMMU Support:: None Pool Info: Pool 1 Segment: GLOBAL; FLAGS: COARSE GRAINED Size: 100663296(0x6000000) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Recommended Granule:2048KB Alloc Alignment: 4KB Accessible by all: FALSE Pool 2 Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED Size: 100663296(0x6000000) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Recommended Granule:2048KB Alloc Alignment: 4KB Accessible by all: FALSE Pool 3 Segment: GROUP Size: 64(0x40) KB Allocatable: FALSE Alloc Granule: 0KB Alloc Recommended Granule:0KB Alloc Alignment: 0KB Accessible by all: FALSE ISA Info: ISA 1 Name: amdgcn-amd-amdhsa--gfx1151 Machine Models: HSA_MACHINE_MODEL_LARGE Profiles: HSA_PROFILE_BASE Default Rounding Mode: NEAR Default Rounding Mode: NEAR Fast f16: TRUE Workgroup Max Size: 1024(0x400) Workgroup Max Size per Dimension: x 1024(0x400) y 1024(0x400) z 1024(0x400) Grid Max Size: 4294967295(0xffffffff) Grid Max Size per Dimension: x 4294967295(0xffffffff) y 4294967295(0xffffffff) z 4294967295(0xffffffff) FBarrier Max Size: 32 ISA 2 Name: amdgcn-amd-amdhsa--gfx11-generic Machine Models: HSA_MACHINE_MODEL_LARGE Profiles: HSA_PROFILE_BASE Default Rounding Mode: NEAR Default Rounding Mode: NEAR Fast f16: TRUE Workgroup Max Size: 1024(0x400) Workgroup Max Size per Dimension: x 1024(0x400) y 1024(0x400) z 1024(0x400) Grid Max Size: 4294967295(0xffffffff) Grid Max Size per Dimension: x 4294967295(0xffffffff) y 4294967295(0xffffffff) z 4294967295(0xffffffff) FBarrier Max Size: 32 *** Done *** Here is the logs from Ollama: OLLAMA_MODELS=/usr/share/ollama/.ollama/models /usr/local/bin/ollama serve time=2025-08-31T13:43:01.989Z level=INFO source=routes.go:1331 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:4096 OLLAMA_DEBUG:INFO OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE:q8_0 OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/usr/share/ollama/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NEW_ESTIMATES:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_SCHED_SPREAD:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]" time=2025-08-31T13:43:01.993Z level=INFO source=images.go:477 msg="total blobs: 168" time=2025-08-31T13:43:01.994Z level=INFO source=images.go:484 msg="total unused blobs removed: 0" time=2025-08-31T13:43:01.994Z level=INFO source=routes.go:1384 msg="Listening on 127.0.0.1:11434 (version 0.11.8)" time=2025-08-31T13:43:01.994Z level=INFO source=gpu.go:217 msg="looking for compatible GPUs" time=2025-08-31T13:43:02.000Z level=INFO source=amd_linux.go:389 msg="amdgpu is supported" gpu=0 gpu_type=gfx1151 time=2025-08-31T13:43:02.002Z level=INFO source=types.go:130 msg="inference compute" id=0 library=rocm variant="" compute=gfx1151 driver=6.12 name=1002:1586 total="96.0 GiB" available="95.9 GiB" time=2025-08-31T13:43:04.850Z level=INFO source=server.go:199 msg="model wants flash attention" time=2025-08-31T13:43:04.850Z level=INFO source=server.go:216 msg="enabling flash attention" time=2025-08-31T13:43:04.850Z level=WARN source=ggml.go:764 msg="model only supports non-quantized cache types " mode=gptoss time=2025-08-31T13:43:04.850Z level=WARN source=server.go:224 msg="kv cache type not supported by model" type=q8_0 time=2025-08-31T13:43:04.850Z level=INFO source=server.go:388 msg="starting runner" cmd="/usr/local/bin/ollama runner --ollama-engine --model /usr/share/ollama/.ollama/models/blobs/sha256-90a618fe6ff21b09ca968df959104eb650658b0bef0faef785c18c2795d993e3 --port 35733" time=2025-08-31T13:43:04.851Z level=INFO source=server.go:493 msg="system memory" total="31.0 GiB" free="26.4 GiB" free_swap="6.3 GiB" time=2025-08-31T13:43:04.851Z level=WARN source=ggml.go:764 msg="model only supports non-quantized cache types " mode=gptoss time=2025-08-31T13:43:04.851Z level=INFO source=memory.go:36 msg="new model will fit in available VRAM across minimum required GPUs, loading" model=/usr/share/ollama/.ollama/models/blobs/sha256-90a618fe6ff21b09ca968df959104eb650658b0bef0faef785c18c2795d993e3 library=rocm parallel=1 required="62.6 GiB" gpus=1 time=2025-08-31T13:43:04.852Z level=WARN source=ggml.go:764 msg="model only supports non-quantized cache types " mode=gptoss time=2025-08-31T13:43:04.852Z level=INFO source=server.go:533 msg=offload library=rocm layers.requested=-1 layers.model=37 layers.offload=37 layers.split=[37] memory.available="[95.9 GiB]" memory.gpu_overhead="0 B" memory.required.full="62.6 GiB" memory.required.partial="62.6 GiB" memory.required.kv="738.0 MiB" memory.required.allocations="[62.6 GiB]" memory.weights.total="59.7 GiB" memory.weights.repeating="58.6 GiB" memory.weights.nonrepeating="1.1 GiB" memory.graph.full="130.0 MiB" memory.graph.partial="130.0 MiB" time=2025-08-31T13:43:04.859Z level=INFO source=runner.go:1006 msg="starting ollama engine" time=2025-08-31T13:43:04.859Z level=INFO source=runner.go:1043 msg="Server listening on 127.0.0.1:35733" time=2025-08-31T13:43:04.864Z level=INFO source=runner.go:925 msg=load request="{Operation:commit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:true KvSize:16384 KvCacheType: NumThreads:16 GPULayers:37[ID:0 Layers:37(0..36)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}" time=2025-08-31T13:43:04.895Z level=INFO source=ggml.go:130 msg="" architecture=gptoss file_type=MXFP4 name="" description="" num_tensors=471 num_key_values=30 ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no ggml_cuda_init: found 1 ROCm devices: Device 0: AMD Radeon Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32, ID: 0 load_backend: loaded ROCm backend from /usr/local/lib/ollama/libggml-hip.so load_backend: loaded CPU backend from /usr/local/lib/ollama/libggml-cpu-icelake.so time=2025-08-31T13:43:05.581Z level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.AVX512=1 CPU.0.AVX512_VBMI=1 CPU.0.AVX512_VNNI=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 ROCm.0.NO_VMM=1 ROCm.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(gcc) time=2025-08-31T13:43:05.730Z level=INFO source=ggml.go:486 msg="offloading 36 repeating layers to GPU" time=2025-08-31T13:43:05.730Z level=INFO source=ggml.go:492 msg="offloading output layer to GPU" time=2025-08-31T13:43:05.730Z level=INFO source=ggml.go:497 msg="offloaded 37/37 layers to GPU" time=2025-08-31T13:43:05.731Z level=INFO source=backend.go:310 msg="model weights" device=ROCm0 size="59.8 GiB" time=2025-08-31T13:43:05.731Z level=INFO source=backend.go:315 msg="model weights" device=CPU size="1.1 GiB" time=2025-08-31T13:43:05.731Z level=INFO source=backend.go:321 msg="kv cache" device=ROCm0 size="738.0 MiB" time=2025-08-31T13:43:05.731Z level=INFO source=backend.go:332 msg="compute graph" device=ROCm0 size="129.8 MiB" time=2025-08-31T13:43:05.731Z level=INFO source=backend.go:337 msg="compute graph" device=CPU size="5.6 MiB" time=2025-08-31T13:43:05.731Z level=INFO source=backend.go:342 msg="total memory" size="61.7 GiB" time=2025-08-31T13:43:05.731Z level=INFO source=sched.go:473 msg="loaded runners" count=1 time=2025-08-31T13:43:05.731Z level=INFO source=server.go:1236 msg="waiting for llama runner to start responding" time=2025-08-31T13:43:05.732Z level=INFO source=server.go:1270 msg="waiting for server to become available" status="llm server loading model" time=2025-08-31T13:43:18.525Z level=INFO source=server.go:1274 msg="llama runner started in 13.68 seconds" Memory access fault by GPU node-1 (Agent handle: 0x724828692930) on address 0x72370e95c000. Reason: Page not present or supervisor privilege. [GIN] 2025/08/31 - 13:43:37 | 500 | 32.614108095s | 127.0.0.1 | POST "/api/generate" ``` ### OS Linux ### GPU AMD ### CPU AMD ### Ollama version 0.11.8
GiteaMirror added the bug label 2026-04-12 20:19:19 -05:00
GiteaMirror commented 2026-04-12 20:19:21 -05:00
Author
Owner
Copy Link

@ulrichenslin commented on GitHub (Aug 31, 2025):

It works when I start ollama from commandline setting the HSA_OVERRIDE_GFX_VERSION=11.0.2

HSA_OVERRIDE_GFX_VERSION=11.0.2 OLLAMA_MODELS=/usr/share/ollama/.ollama/models /usr/local/bin/ollama serve

If this an ollama issue with hardware detection, or a driver issue not presenting the write information to ollama?

<!-- gh-comment-id:3240296910 --> @ulrichenslin commented on GitHub (Aug 31, 2025): It works when I start ollama from commandline setting the HSA_OVERRIDE_GFX_VERSION=11.0.2 `HSA_OVERRIDE_GFX_VERSION=11.0.2 OLLAMA_MODELS=/usr/share/ollama/.ollama/models /usr/local/bin/ollama serve` If this an ollama issue with hardware detection, or a driver issue not presenting the write information to ollama?
GiteaMirror commented 2026-04-12 20:19:22 -05:00
Author
Owner
Copy Link

@ulrichenslin commented on GitHub (Sep 4, 2025):

This issue remains in version 0.11.9

The override does not work in 0.11.9
HSA_OVERRIDE_GFX_VERSION=11.0.2 OLLAMA_MODELS=/usr/share/ollama/.ollama/models /usr/local/bin/ollama serve

Now stepped back to 0.11.4, which is stable for my hardware.

<!-- gh-comment-id:3251956174 --> @ulrichenslin commented on GitHub (Sep 4, 2025): This issue remains in version 0.11.9 The override does not work in 0.11.9 HSA_OVERRIDE_GFX_VERSION=11.0.2 OLLAMA_MODELS=/usr/share/ollama/.ollama/models /usr/local/bin/ollama serve Now stepped back to 0.11.4, which is stable for my hardware.
GiteaMirror commented 2026-04-12 20:19:23 -05:00
Author
Owner
Copy Link

@ulrichenslin commented on GitHub (Oct 29, 2025):

Closing this issue. This is working on version 0.12.6 without any overrides.

From the logs:
library=ROCm compute=gfx1151 name=ROCm0 description="AMD Radeon Graphics" libdirs=ollama,rocm driver=60342.13 pci_id=c5:00.0 type=iGPU total="61.3 GiB" available="61.2 GiB"

Running 'gpt-oss-120b' successfully.

<!-- gh-comment-id:3460655494 --> @ulrichenslin commented on GitHub (Oct 29, 2025): Closing this issue. This is working on version 0.12.6 without any overrides. From the logs: library=ROCm compute=gfx1151 name=ROCm0 description="AMD Radeon Graphics" libdirs=ollama,rocm driver=60342.13 pci_id=c5:00.0 type=iGPU total="61.3 GiB" available="61.2 GiB" Running 'gpt-oss-120b' successfully.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#8061
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?