[GH-ISSUE #13075] Ollama will not build against CUDA later than 12.6 (fixed) #55170

New Issue

@abcbarryn commented on GitHub (Nov 13, 2025):

I am retrying with GCC 14 now.

@abcbarryn commented on GitHub (Nov 13, 2025): I am retrying with GCC 14 now.

GiteaMirror commented

@abcbarryn commented on GitHub (Nov 13, 2025):

Nope, same problem.

/usr/local/cuda/include/cuda/std/__cmath/traits.h(360): error: the global scope has no "isgreater"
    {return ::isgreater((type) __x, (type) __y);}
              ^

/usr/local/cuda/include/cuda/std/__cmath/traits.h(383): error: the global scope has no "isgreaterequal"
    {return ::isgreaterequal((type) __x, (type) __y);}
              ^

/usr/local/cuda/include/cuda/std/__cmath/traits.h(406): error: the global scope has no "isless"
    {return ::isless((type) __x, (type) __y);}
              ^

/usr/local/cuda/include/cuda/std/__cmath/traits.h(429): error: the global scope has no "islessequal"
    {return ::islessequal((type) __x, (type) __y);}
              ^

/usr/local/cuda/include/cuda/std/__cmath/traits.h(452): error: the global scope has no "islessgreater"
    {return ::islessgreater((type) __x, (type) __y);}
              ^

5 errors detected in the compilation of "/usr/src/ollama-0.12.10/ml/backend/ggml/ggml/src/ggml-cuda/mean.cu".
gmake[2]: *** [ml/backend/ggml/ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/build.make:440: ml/backend/ggml/ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/mean.cu.o] Error 2
gmake[1]: *** [CMakeFiles/Makefile2:801: ml/backend/ggml/ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2

@abcbarryn commented on GitHub (Nov 13, 2025): Nope, same problem. ``` /usr/local/cuda/include/cuda/std/__cmath/traits.h(360): error: the global scope has no "isgreater" {return ::isgreater((type) __x, (type) __y);} ^ /usr/local/cuda/include/cuda/std/__cmath/traits.h(383): error: the global scope has no "isgreaterequal" {return ::isgreaterequal((type) __x, (type) __y);} ^ /usr/local/cuda/include/cuda/std/__cmath/traits.h(406): error: the global scope has no "isless" {return ::isless((type) __x, (type) __y);} ^ /usr/local/cuda/include/cuda/std/__cmath/traits.h(429): error: the global scope has no "islessequal" {return ::islessequal((type) __x, (type) __y);} ^ /usr/local/cuda/include/cuda/std/__cmath/traits.h(452): error: the global scope has no "islessgreater" {return ::islessgreater((type) __x, (type) __y);} ^ 5 errors detected in the compilation of "/usr/src/ollama-0.12.10/ml/backend/ggml/ggml/src/ggml-cuda/mean.cu". gmake[2]: *** [ml/backend/ggml/ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/build.make:440: ml/backend/ggml/ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/mean.cu.o] Error 2 gmake[1]: *** [CMakeFiles/Makefile2:801: ml/backend/ggml/ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/all] Error 2 gmake: *** [Makefile:136: all] Error 2 ```

GiteaMirror commented

@abcbarryn commented on GitHub (Nov 13, 2025):

GCC 13 same issue...

[ 26%] Building CUDA object ml/backend/ggml/ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/mean.cu.o
nvcc warning : Support for offline compilation for architectures prior to '<compute/sm/lto>_75' will be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
/usr/local/cuda/include/cuda/std/__cmath/traits.h(360): error: the global scope has no "isgreater"
    {return ::isgreater((type) __x, (type) __y);}
              ^

/usr/local/cuda/include/cuda/std/__cmath/traits.h(383): error: the global scope has no "isgreaterequal"
    {return ::isgreaterequal((type) __x, (type) __y);}
              ^

/usr/local/cuda/include/cuda/std/__cmath/traits.h(406): error: the global scope has no "isless"
    {return ::isless((type) __x, (type) __y);}
              ^

/usr/local/cuda/include/cuda/std/__cmath/traits.h(429): error: the global scope has no "islessequal"
    {return ::islessequal((type) __x, (type) __y);}
              ^

/usr/local/cuda/include/cuda/std/__cmath/traits.h(452): error: the global scope has no "islessgreater"
    {return ::islessgreater((type) __x, (type) __y);}
              ^

5 errors detected in the compilation of "/usr/src/ollama-0.12.10/ml/backend/ggml/ggml/src/ggml-cuda/mean.cu".
gmake[2]: *** [ml/backend/ggml/ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/build.make:440: ml/backend/ggml/ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/mean.cu.o] Error 2
gmake[1]: *** [CMakeFiles/Makefile2:801: ml/backend/ggml/ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2

@abcbarryn commented on GitHub (Nov 13, 2025): GCC 13 same issue... ``` [ 26%] Building CUDA object ml/backend/ggml/ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/mean.cu.o nvcc warning : Support for offline compilation for architectures prior to '<compute/sm/lto>_75' will be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning). /usr/local/cuda/include/cuda/std/__cmath/traits.h(360): error: the global scope has no "isgreater" {return ::isgreater((type) __x, (type) __y);} ^ /usr/local/cuda/include/cuda/std/__cmath/traits.h(383): error: the global scope has no "isgreaterequal" {return ::isgreaterequal((type) __x, (type) __y);} ^ /usr/local/cuda/include/cuda/std/__cmath/traits.h(406): error: the global scope has no "isless" {return ::isless((type) __x, (type) __y);} ^ /usr/local/cuda/include/cuda/std/__cmath/traits.h(429): error: the global scope has no "islessequal" {return ::islessequal((type) __x, (type) __y);} ^ /usr/local/cuda/include/cuda/std/__cmath/traits.h(452): error: the global scope has no "islessgreater" {return ::islessgreater((type) __x, (type) __y);} ^ 5 errors detected in the compilation of "/usr/src/ollama-0.12.10/ml/backend/ggml/ggml/src/ggml-cuda/mean.cu". gmake[2]: *** [ml/backend/ggml/ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/build.make:440: ml/backend/ggml/ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/mean.cu.o] Error 2 gmake[1]: *** [CMakeFiles/Makefile2:801: ml/backend/ggml/ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/all] Error 2 gmake: *** [Makefile:136: all] Error 2 ```

GiteaMirror commented

2026-04-29 08:26:42 -05:00

@rick-github commented on GitHub (Nov 13, 2025):

$ mkdir 0.12.10 && cd 0.12.10 && git clone https://github.com/ollama/ollama -b v0.12.10 . && CC=gcc-14 CXX=g++-14 PATH=/usr/local/cuda-12.8/bin:$PATH cmake -B build && cmake --build build
Cloning into '.'...
...
-- The C compiler identification is GNU 14.2.0
-- The CXX compiler identification is GNU 14.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/gcc-14 - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/g++-14 - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- GGML_SYSTEM_ARCH: x86
-- Including CPU backend
-- x86 detected
-- Adding CPU backend variant ggml-cpu-x64:  
-- x86 detected
-- Adding CPU backend variant ggml-cpu-sse42: -msse4.2 GGML_SSE42
-- x86 detected
-- Adding CPU backend variant ggml-cpu-sandybridge: -msse4.2;-mavx GGML_SSE42;GGML_AVX
-- x86 detected
-- Adding CPU backend variant ggml-cpu-haswell: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2 GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2
-- x86 detected
-- Adding CPU backend variant ggml-cpu-skylakex: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavx512f;-mavx512cd;-mavx512vl;-mavx512dq;-mavx512bw GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX512
-- x86 detected
-- Adding CPU backend variant ggml-cpu-icelake: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavx512f;-mavx512cd;-mavx512vl;-mavx512dq;-mavx512bw;-mavx512vbmi;-mavx512vnni GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX512;GGML_AVX512_VBMI;GGML_AVX512_VNNI
-- x86 detected
-- Adding CPU backend variant ggml-cpu-alderlake: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavxvnni GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX_VNNI
-- Looking for a CUDA compiler
-- Looking for a CUDA compiler - /usr/local/cuda-12.8/bin/nvcc
-- Found CUDAToolkit: /usr/local/cuda-12.8/targets/x86_64-linux/include (found version "12.8.93") 
-- CUDA Toolkit found
-- Using CUDA architectures: native
-- The CUDA compiler identification is NVIDIA 12.8.93
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda-12.8/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Looking for a HIP compiler
-- Looking for a HIP compiler - NOTFOUND
-- Could NOT find Vulkan (missing: Vulkan_LIBRARY Vulkan_INCLUDE_DIR) (found version "")
-- Configuring done (7.2s)
-- Generating done (0.0s)
-- Build files have been written to: /home/rick/ollama/0.12.10/build
[  1%] Building C object ml/backend/ggml/ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o
...
[100%] Built target ggml-cuda

@rick-github commented on GitHub (Nov 13, 2025): ```console $ mkdir 0.12.10 && cd 0.12.10 && git clone https://github.com/ollama/ollama -b v0.12.10 . && CC=gcc-14 CXX=g++-14 PATH=/usr/local/cuda-12.8/bin:$PATH cmake -B build && cmake --build build Cloning into '.'... ... -- The C compiler identification is GNU 14.2.0 -- The CXX compiler identification is GNU 14.2.0 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /usr/bin/gcc-14 - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /usr/bin/g++-14 - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success -- Found Threads: TRUE -- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF -- CMAKE_SYSTEM_PROCESSOR: x86_64 -- GGML_SYSTEM_ARCH: x86 -- Including CPU backend -- x86 detected -- Adding CPU backend variant ggml-cpu-x64: -- x86 detected -- Adding CPU backend variant ggml-cpu-sse42: -msse4.2 GGML_SSE42 -- x86 detected -- Adding CPU backend variant ggml-cpu-sandybridge: -msse4.2;-mavx GGML_SSE42;GGML_AVX -- x86 detected -- Adding CPU backend variant ggml-cpu-haswell: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2 GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2 -- x86 detected -- Adding CPU backend variant ggml-cpu-skylakex: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavx512f;-mavx512cd;-mavx512vl;-mavx512dq;-mavx512bw GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX512 -- x86 detected -- Adding CPU backend variant ggml-cpu-icelake: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavx512f;-mavx512cd;-mavx512vl;-mavx512dq;-mavx512bw;-mavx512vbmi;-mavx512vnni GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX512;GGML_AVX512_VBMI;GGML_AVX512_VNNI -- x86 detected -- Adding CPU backend variant ggml-cpu-alderlake: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavxvnni GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX_VNNI -- Looking for a CUDA compiler -- Looking for a CUDA compiler - /usr/local/cuda-12.8/bin/nvcc -- Found CUDAToolkit: /usr/local/cuda-12.8/targets/x86_64-linux/include (found version "12.8.93") -- CUDA Toolkit found -- Using CUDA architectures: native -- The CUDA compiler identification is NVIDIA 12.8.93 -- Detecting CUDA compiler ABI info -- Detecting CUDA compiler ABI info - done -- Check for working CUDA compiler: /usr/local/cuda-12.8/bin/nvcc - skipped -- Detecting CUDA compile features -- Detecting CUDA compile features - done -- Looking for a HIP compiler -- Looking for a HIP compiler - NOTFOUND -- Could NOT find Vulkan (missing: Vulkan_LIBRARY Vulkan_INCLUDE_DIR) (found version "") -- Configuring done (7.2s) -- Generating done (0.0s) -- Build files have been written to: /home/rick/ollama/0.12.10/build [ 1%] Building C object ml/backend/ggml/ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o ... [100%] Built target ggml-cuda ```

GiteaMirror commented

@abcbarryn commented on GitHub (Nov 13, 2025):

Ok, well, something else must be different...
Last CUDA toolkit version that works to build Ollama 0.12.10 on my system is 12.6.
Using GCC 11,13 or 14
SuSE Leap 15.6 Linux

@abcbarryn commented on GitHub (Nov 13, 2025): Ok, well, something else must be different... Last CUDA toolkit version that works to build Ollama 0.12.10 on my system is 12.6. Using GCC 11,13 or 14 SuSE Leap 15.6 Linux

GiteaMirror commented

2026-04-29 08:26:42 -05:00

@rick-github commented on GitHub (Nov 13, 2025):

If you provide build information as demonstrated it might be possible to pinpoint the issue. The various CMakeConfigureLog.yaml files in the build (sub-)directories may also be useful.

@rick-github commented on GitHub (Nov 13, 2025): If you provide build information as demonstrated it might be possible to pinpoint the issue. The various `CMakeConfigureLog.yaml` files in the build (sub-)directories may also be useful.

GiteaMirror commented

2026-04-29 08:26:43 -05:00

@abcbarryn commented on GitHub (Nov 13, 2025):

Ok, well, I am done messing with it for now and I am using the build using CUDA 12.6 and GCC 13. I will probably try again later... What build information do you want?

@abcbarryn commented on GitHub (Nov 13, 2025): Ok, well, I am done messing with it for now and I am using the build using CUDA 12.6 and GCC 13. I will probably try again later... What build information do you want?

GiteaMirror commented

2026-04-29 08:26:43 -05:00

@rick-github commented on GitHub (Nov 13, 2025):

https://github.com/ollama/ollama/issues/13075#issuecomment-3527024193

@rick-github commented on GitHub (Nov 13, 2025): https://github.com/ollama/ollama/issues/13075#issuecomment-3527024193

GiteaMirror commented

2026-04-29 08:26:43 -05:00

@abcbarryn commented on GitHub (Nov 13, 2025):

I downloaded the Ollama 0.12.10 release source code from: https://github.com/ollama/ollama/archive/refs/tags/v0.12.10.tar.gz
I unpacked the .tar.gz file and cd'd to the folder.

# update-alternatives --config cuda
There are 4 choices for the alternative cuda (providing /usr/local/cuda).

  Selection    Path                  Priority   Status
------------------------------------------------------------
  0            /usr/local/cuda-13.0   130       auto mode
  1            /usr/local/cuda-11.8   118       manual mode
* 2            /usr/local/cuda-12.6   126       manual mode
  3            /usr/local/cuda-12.9   129       manual mode
  4            /usr/local/cuda-13.0   130       manual mode

Press <enter> to keep the current choice[*], or type selection number: 3
# update-alternatives --config gcc
There are 4 choices for the alternative gcc (providing /usr/bin/gcc).

  Selection    Path             Priority   Status
------------------------------------------------------------
  0            /usr/bin/gcc-14   14        auto mode
  1            /usr/bin/gcc-11   11        manual mode
* 2            /usr/bin/gcc-13   13        manual mode
  3            /usr/bin/gcc-14   14        manual mode
  4            /usr/bin/gcc-7    7         manual mode

Press <enter> to keep the current choice[*], or type selection number: 2
# export CMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc
# export INCLUDES="-I /usr/local/cuda/include"
# export PATH="/usr/local/cuda/nvvm/bin:/usr/local/cuda/bin:/usr/local/cuda/lib64:$PATH"
# export LD_LIBRARY_PATH="/lib64:/usr/lib64:/usr/local/cuda/lib64:/usr/local/cuda/targets/x86_64-linux/lib"
# cmake -B build
# cmake --build build
-- The C compiler identification is GNU 13.3.1
-- The CXX compiler identification is GNU 13.3.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- GGML_SYSTEM_ARCH: x86
-- Including CPU backend
-- x86 detected
-- Adding CPU backend variant ggml-cpu-x64:
-- x86 detected
-- Adding CPU backend variant ggml-cpu-sse42: -msse4.2 GGML_SSE42
-- x86 detected
-- Adding CPU backend variant ggml-cpu-sandybridge: -msse4.2;-mavx GGML_SSE42;GGML_AVX
-- x86 detected
-- Adding CPU backend variant ggml-cpu-haswell: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2 GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2
-- x86 detected
-- Adding CPU backend variant ggml-cpu-skylakex: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavx512f;-mavx512cd;-mavx512vl;-mavx512dq;-mavx512bw GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX512
-- x86 detected
-- Adding CPU backend variant ggml-cpu-icelake: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavx512f;-mavx512cd;-mavx512vl;-mavx512dq;-mavx512bw;-mavx512vbmi;-mavx512vnni GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX512;GGML_AVX512_VBMI;GGML_AVX512_VNNI
-- x86 detected
-- Adding CPU backend variant ggml-cpu-alderlake: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavxvnni GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX_VNNI
-- Looking for a CUDA compiler
-- Looking for a CUDA compiler - /usr/bin/nvcc
-- Found CUDAToolkit: /usr/include;/usr/src/ollama-0.12.10;/usr/local/cuda/include (found version "12.9.86")
-- CUDA Toolkit found
-- Using CUDA architectures: native
-- The CUDA compiler identification is NVIDIA 12.9.86 with host compiler GNU 13.3.1
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Looking for a HIP compiler
-- Looking for a HIP compiler - NOTFOUND
-- Found Vulkan: /usr/lib64/libvulkan.so (found version "1.3.275") found components: glslc missing components: glslangValidator
-- Vulkan found
-- GL_KHR_cooperative_matrix supported by glslc
-- GL_NV_cooperative_matrix2 supported by glslc
-- GL_EXT_integer_dot_product supported by glslc
-- GL_EXT_bfloat16 supported by glslc
-- Configuring done (11.9s)
-- Generating done (0.1s)
-- Build files have been written to: /usr/src/ollama-0.12.10/build
[  0%] Building C object ml/backend/ggml/ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o
[  0%] Building CXX object ml/backend/ggml/ggml/src/CMakeFiles/ggml-base.dir/ggml.cpp.o
[  1%] Building C object ml/backend/ggml/ggml/src/CMakeFiles/ggml-base.dir/ggml-alloc.c.o
[  1%] Building CXX object ml/backend/ggml/ggml/src/CMakeFiles/ggml-base.dir/ggml-backend.cpp.o
[  1%] Building CXX object ml/backend/ggml/ggml/src/CMakeFiles/ggml-base.dir/ggml-opt.cpp.o
[  1%] Building CXX object ml/backend/ggml/ggml/src/CMakeFiles/ggml-base.dir/ggml-threading.cpp.o
[  1%] Building C object ml/backend/ggml/ggml/src/CMakeFiles/ggml-base.dir/ggml-quants.c.o

[ 31%] Building CUDA object ml/backend/ggml/ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/mean.cu.o
nvcc warning : Support for offline compilation for architectures prior to '<compute/sm/lto>_75' will be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
/usr/local/cuda/include/cuda/std/__cmath/traits.h(360): error: the global scope has no "isgreater"
    {return ::isgreater((type) __x, (type) __y);}
              ^

/usr/local/cuda/include/cuda/std/__cmath/traits.h(383): error: the global scope has no "isgreaterequal"
    {return ::isgreaterequal((type) __x, (type) __y);}
              ^

/usr/local/cuda/include/cuda/std/__cmath/traits.h(406): error: the global scope has no "isless"
    {return ::isless((type) __x, (type) __y);}
              ^

/usr/local/cuda/include/cuda/std/__cmath/traits.h(429): error: the global scope has no "islessequal"
    {return ::islessequal((type) __x, (type) __y);}
              ^

/usr/local/cuda/include/cuda/std/__cmath/traits.h(452): error: the global scope has no "islessgreater"
    {return ::islessgreater((type) __x, (type) __y);}
              ^

5 errors detected in the compilation of "/usr/src/ollama-0.12.10/ml/backend/ggml/ggml/src/ggml-cuda/mean.cu".
gmake[2]: *** [ml/backend/ggml/ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/build.make:440: ml/backend/ggml/ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/mean.cu.o] Error 2
gmake[1]: *** [CMakeFiles/Makefile2:801: ml/backend/ggml/ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2

@abcbarryn commented on GitHub (Nov 13, 2025): I downloaded the Ollama 0.12.10 release source code from: https://github.com/ollama/ollama/archive/refs/tags/v0.12.10.tar.gz I unpacked the .tar.gz file and cd'd to the folder. ``` # update-alternatives --config cuda There are 4 choices for the alternative cuda (providing /usr/local/cuda). Selection Path Priority Status ------------------------------------------------------------ 0 /usr/local/cuda-13.0 130 auto mode 1 /usr/local/cuda-11.8 118 manual mode * 2 /usr/local/cuda-12.6 126 manual mode 3 /usr/local/cuda-12.9 129 manual mode 4 /usr/local/cuda-13.0 130 manual mode Press <enter> to keep the current choice[*], or type selection number: 3 # update-alternatives --config gcc There are 4 choices for the alternative gcc (providing /usr/bin/gcc). Selection Path Priority Status ------------------------------------------------------------ 0 /usr/bin/gcc-14 14 auto mode 1 /usr/bin/gcc-11 11 manual mode * 2 /usr/bin/gcc-13 13 manual mode 3 /usr/bin/gcc-14 14 manual mode 4 /usr/bin/gcc-7 7 manual mode Press <enter> to keep the current choice[*], or type selection number: 2 # export CMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc # export INCLUDES="-I /usr/local/cuda/include" # export PATH="/usr/local/cuda/nvvm/bin:/usr/local/cuda/bin:/usr/local/cuda/lib64:$PATH" # export LD_LIBRARY_PATH="/lib64:/usr/lib64:/usr/local/cuda/lib64:/usr/local/cuda/targets/x86_64-linux/lib" # cmake -B build # cmake --build build -- The C compiler identification is GNU 13.3.1 -- The CXX compiler identification is GNU 13.3.1 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /usr/bin/cc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /usr/bin/c++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success -- Found Threads: TRUE -- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF -- CMAKE_SYSTEM_PROCESSOR: x86_64 -- GGML_SYSTEM_ARCH: x86 -- Including CPU backend -- x86 detected -- Adding CPU backend variant ggml-cpu-x64: -- x86 detected -- Adding CPU backend variant ggml-cpu-sse42: -msse4.2 GGML_SSE42 -- x86 detected -- Adding CPU backend variant ggml-cpu-sandybridge: -msse4.2;-mavx GGML_SSE42;GGML_AVX -- x86 detected -- Adding CPU backend variant ggml-cpu-haswell: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2 GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2 -- x86 detected -- Adding CPU backend variant ggml-cpu-skylakex: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavx512f;-mavx512cd;-mavx512vl;-mavx512dq;-mavx512bw GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX512 -- x86 detected -- Adding CPU backend variant ggml-cpu-icelake: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavx512f;-mavx512cd;-mavx512vl;-mavx512dq;-mavx512bw;-mavx512vbmi;-mavx512vnni GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX512;GGML_AVX512_VBMI;GGML_AVX512_VNNI -- x86 detected -- Adding CPU backend variant ggml-cpu-alderlake: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavxvnni GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX_VNNI -- Looking for a CUDA compiler -- Looking for a CUDA compiler - /usr/bin/nvcc -- Found CUDAToolkit: /usr/include;/usr/src/ollama-0.12.10;/usr/local/cuda/include (found version "12.9.86") -- CUDA Toolkit found -- Using CUDA architectures: native -- The CUDA compiler identification is NVIDIA 12.9.86 with host compiler GNU 13.3.1 -- Detecting CUDA compiler ABI info -- Detecting CUDA compiler ABI info - done -- Check for working CUDA compiler: /usr/bin/nvcc - skipped -- Detecting CUDA compile features -- Detecting CUDA compile features - done -- Looking for a HIP compiler -- Looking for a HIP compiler - NOTFOUND -- Found Vulkan: /usr/lib64/libvulkan.so (found version "1.3.275") found components: glslc missing components: glslangValidator -- Vulkan found -- GL_KHR_cooperative_matrix supported by glslc -- GL_NV_cooperative_matrix2 supported by glslc -- GL_EXT_integer_dot_product supported by glslc -- GL_EXT_bfloat16 supported by glslc -- Configuring done (11.9s) -- Generating done (0.1s) -- Build files have been written to: /usr/src/ollama-0.12.10/build [ 0%] Building C object ml/backend/ggml/ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o [ 0%] Building CXX object ml/backend/ggml/ggml/src/CMakeFiles/ggml-base.dir/ggml.cpp.o [ 1%] Building C object ml/backend/ggml/ggml/src/CMakeFiles/ggml-base.dir/ggml-alloc.c.o [ 1%] Building CXX object ml/backend/ggml/ggml/src/CMakeFiles/ggml-base.dir/ggml-backend.cpp.o [ 1%] Building CXX object ml/backend/ggml/ggml/src/CMakeFiles/ggml-base.dir/ggml-opt.cpp.o [ 1%] Building CXX object ml/backend/ggml/ggml/src/CMakeFiles/ggml-base.dir/ggml-threading.cpp.o [ 1%] Building C object ml/backend/ggml/ggml/src/CMakeFiles/ggml-base.dir/ggml-quants.c.o ``` ``` [ 31%] Building CUDA object ml/backend/ggml/ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/mean.cu.o nvcc warning : Support for offline compilation for architectures prior to '<compute/sm/lto>_75' will be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning). /usr/local/cuda/include/cuda/std/__cmath/traits.h(360): error: the global scope has no "isgreater" {return ::isgreater((type) __x, (type) __y);} ^ /usr/local/cuda/include/cuda/std/__cmath/traits.h(383): error: the global scope has no "isgreaterequal" {return ::isgreaterequal((type) __x, (type) __y);} ^ /usr/local/cuda/include/cuda/std/__cmath/traits.h(406): error: the global scope has no "isless" {return ::isless((type) __x, (type) __y);} ^ /usr/local/cuda/include/cuda/std/__cmath/traits.h(429): error: the global scope has no "islessequal" {return ::islessequal((type) __x, (type) __y);} ^ /usr/local/cuda/include/cuda/std/__cmath/traits.h(452): error: the global scope has no "islessgreater" {return ::islessgreater((type) __x, (type) __y);} ^ 5 errors detected in the compilation of "/usr/src/ollama-0.12.10/ml/backend/ggml/ggml/src/ggml-cuda/mean.cu". gmake[2]: *** [ml/backend/ggml/ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/build.make:440: ml/backend/ggml/ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/mean.cu.o] Error 2 gmake[1]: *** [CMakeFiles/Makefile2:801: ml/backend/ggml/ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/all] Error 2 gmake: *** [Makefile:136: all] Error 2 ```

GiteaMirror commented

@abcbarryn commented on GitHub (Nov 13, 2025):

This could be related to: https://github.com/ollama/ollama/issues/12872

@abcbarryn commented on GitHub (Nov 13, 2025): This could be related to: https://github.com/ollama/ollama/issues/12872

GiteaMirror commented

@rick-github commented on GitHub (Nov 13, 2025):

$ curl -sL https://github.com/ollama/ollama/archive/refs/tags/v0.12.10.tar.gz | tar zxf - && cd ollama-0.12.10 && CC=gcc-13 CXX=g++-13 PATH=/usr/local/cuda-12.9/bin:$PATH cmake -B build && cmake --build build
-- The C compiler identification is GNU 13.2.0
-- The CXX compiler identification is GNU 13.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/gcc-13 - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/g++-13 - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- GGML_SYSTEM_ARCH: x86
-- Including CPU backend
-- x86 detected
-- Adding CPU backend variant ggml-cpu-x64:  
-- x86 detected
-- Adding CPU backend variant ggml-cpu-sse42: -msse4.2 GGML_SSE42
-- x86 detected
-- Adding CPU backend variant ggml-cpu-sandybridge: -msse4.2;-mavx GGML_SSE42;GGML_AVX
-- x86 detected
-- Adding CPU backend variant ggml-cpu-haswell: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2 GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2
-- x86 detected
-- Adding CPU backend variant ggml-cpu-skylakex: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavx512f;-mavx512cd;-mavx512vl;-mavx512dq;-mavx512bw GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX512
-- x86 detected
-- Adding CPU backend variant ggml-cpu-icelake: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavx512f;-mavx512cd;-mavx512vl;-mavx512dq;-mavx512bw;-mavx512vbmi;-mavx512vnni GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX512;GGML_AVX512_VBMI;GGML_AVX512_VNNI
-- x86 detected
-- Adding CPU backend variant ggml-cpu-alderlake: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavxvnni GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX_VNNI
-- Looking for a CUDA compiler
-- Looking for a CUDA compiler - /usr/local/cuda-12.9/bin/nvcc
-- Found CUDAToolkit: /usr/local/cuda-12.9/targets/x86_64-linux/include;/usr/include (found version "12.9.86") 
-- CUDA Toolkit found
-- Using CUDA architectures: native
-- The CUDA compiler identification is NVIDIA 12.9.86
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda-12.9/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Looking for a HIP compiler
-- Looking for a HIP compiler - NOTFOUND
-- Could NOT find Vulkan (missing: Vulkan_LIBRARY Vulkan_INCLUDE_DIR) (found version "")
-- Configuring done (7.3s)
-- Generating done (0.0s)
-- Build files have been written to: /home/rick/ollama/ollama-0.12.10/build
[  1%] Building C object ml/backend/ggml/ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o
...
[100%] Built target ggml-cuda

This could be related to: https://github.com/ollama/ollama/issues/12872

More likely that your build environment is inconsistent.

@rick-github commented on GitHub (Nov 13, 2025): ```console $ curl -sL https://github.com/ollama/ollama/archive/refs/tags/v0.12.10.tar.gz | tar zxf - && cd ollama-0.12.10 && CC=gcc-13 CXX=g++-13 PATH=/usr/local/cuda-12.9/bin:$PATH cmake -B build && cmake --build build -- The C compiler identification is GNU 13.2.0 -- The CXX compiler identification is GNU 13.2.0 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /usr/bin/gcc-13 - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /usr/bin/g++-13 - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success -- Found Threads: TRUE -- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF -- CMAKE_SYSTEM_PROCESSOR: x86_64 -- GGML_SYSTEM_ARCH: x86 -- Including CPU backend -- x86 detected -- Adding CPU backend variant ggml-cpu-x64: -- x86 detected -- Adding CPU backend variant ggml-cpu-sse42: -msse4.2 GGML_SSE42 -- x86 detected -- Adding CPU backend variant ggml-cpu-sandybridge: -msse4.2;-mavx GGML_SSE42;GGML_AVX -- x86 detected -- Adding CPU backend variant ggml-cpu-haswell: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2 GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2 -- x86 detected -- Adding CPU backend variant ggml-cpu-skylakex: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavx512f;-mavx512cd;-mavx512vl;-mavx512dq;-mavx512bw GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX512 -- x86 detected -- Adding CPU backend variant ggml-cpu-icelake: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavx512f;-mavx512cd;-mavx512vl;-mavx512dq;-mavx512bw;-mavx512vbmi;-mavx512vnni GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX512;GGML_AVX512_VBMI;GGML_AVX512_VNNI -- x86 detected -- Adding CPU backend variant ggml-cpu-alderlake: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavxvnni GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX_VNNI -- Looking for a CUDA compiler -- Looking for a CUDA compiler - /usr/local/cuda-12.9/bin/nvcc -- Found CUDAToolkit: /usr/local/cuda-12.9/targets/x86_64-linux/include;/usr/include (found version "12.9.86") -- CUDA Toolkit found -- Using CUDA architectures: native -- The CUDA compiler identification is NVIDIA 12.9.86 -- Detecting CUDA compiler ABI info -- Detecting CUDA compiler ABI info - done -- Check for working CUDA compiler: /usr/local/cuda-12.9/bin/nvcc - skipped -- Detecting CUDA compile features -- Detecting CUDA compile features - done -- Looking for a HIP compiler -- Looking for a HIP compiler - NOTFOUND -- Could NOT find Vulkan (missing: Vulkan_LIBRARY Vulkan_INCLUDE_DIR) (found version "") -- Configuring done (7.3s) -- Generating done (0.0s) -- Build files have been written to: /home/rick/ollama/ollama-0.12.10/build [ 1%] Building C object ml/backend/ggml/ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o ... [100%] Built target ggml-cuda ``` > This could be related to: https://github.com/ollama/ollama/issues/12872 More likely that your build environment is inconsistent.

GiteaMirror commented

@abcbarryn commented on GitHub (Nov 14, 2025):

Inconsistent how?
Ollama 0.12.10 builds with CUDA 11.8 and CUDA 12.0-12.6. What should I check? I installed the CUDA toolkits with the zypper package manager that SuSE Linux uses.

@abcbarryn commented on GitHub (Nov 14, 2025): Inconsistent how? Ollama 0.12.10 builds with CUDA 11.8 and CUDA 12.0-12.6. What should I check? I installed the CUDA toolkits with the zypper package manager that SuSE Linux uses.

GiteaMirror commented

2026-04-29 08:26:45 -05:00

@rick-github commented on GitHub (Nov 14, 2025):

You would be in a better position to determine inconsistency than I. For example, CMAKE_CUDA_COMPILER doesn't match the working CUDA compiler line in the build log. I don't know what other changes you've made over the course of trying to keep your K40 supported and now your Quadro, but it seems pretty apparent that the problem is in your environment and not in the build process. I can download and build the ollama binary with a one line shell command, the releases page has binary builds for every version of ollama on multiple hardware platforms, and nobody else out of the hundreds or thousands of users that are building from source has as many problems as you do. You've filed multiple bugs for build failures that nobody else is experiencing, perhaps it's time to look at the common factor, your build environment.

@rick-github commented on GitHub (Nov 14, 2025): You would be in a better position to determine inconsistency than I. For example, `CMAKE_CUDA_COMPILER` doesn't match the `working CUDA compiler` line in the build log. I don't know what other changes you've made over the course of trying to keep your K40 supported and now your Quadro, but it seems pretty apparent that the problem is in your environment and not in the build process. I can download and build the ollama binary with a one line shell command, the releases page has binary builds for every version of ollama on multiple hardware platforms, and nobody else out of the hundreds or thousands of users that are building from source has as many problems as you do. You've filed multiple bugs for build failures that nobody else is experiencing, perhaps it's time to look at the common factor, your build environment.

GiteaMirror commented

@abcbarryn commented on GitHub (Nov 14, 2025):

CMAKE_CUDA_COMPILER does match! Don't you know how update-alternatives works? The directory /usr/local/cuda is a symlink that points to the active CUDA toolkit directory. I backed out the changes for the K40 when I installed the P6000. Those changes were primarily choosing the versions of CUDA and drivers that were compatible and having to run a 6.2 version Linux kernel to be able to run the NVidia K40 kernel driver. There are over 1900 bugs filed in this project, so I am not sure where you are going with the statement that nobody else is experiencing build failures. Since you are the one claiming that my build environment is inconsistent, I was rather hoping that you had seen something in the build logs that justifies that conclusion. Since I am no longer running the K40, I have upgraded the kernel and switched to newer CUDA versions for compiling. All CUDA installations were done with my distributions package manager. I have avoided modifying system files except for issue #12872.

@abcbarryn commented on GitHub (Nov 14, 2025): CMAKE_CUDA_COMPILER does match! Don't you know how update-alternatives works? The directory /usr/local/cuda is a symlink that points to the active CUDA toolkit directory. I backed out the changes for the K40 when I installed the P6000. Those changes were primarily choosing the versions of CUDA and drivers that were compatible and having to run a 6.2 version Linux kernel to be able to run the NVidia K40 kernel driver. There are over 1900 bugs filed in this project, so I am not sure where you are going with the statement that nobody else is experiencing build failures. Since you are the one claiming that my build environment is inconsistent, I was rather hoping that you had seen something in the build logs that justifies that conclusion. Since I am no longer running the K40, I have upgraded the kernel and switched to newer CUDA versions for compiling. All CUDA installations were done with my distributions package manager. I have avoided modifying system files except for issue #12872.

GiteaMirror commented

2026-04-29 08:26:45 -05:00

@abcbarryn commented on GitHub (Nov 14, 2025):

Also, as far as others not having issues, read through this thread...
https://github.com/ollama/ollama/issues/12618
It is full of users having issues with recent versions of Ollama, there are users in the thread that are having trouble with both Windows and Linux systems and can't get Ollama to run properly at all. At least my system is working (very well, actually) with my build using CUDA 12.6 and the latest Ollama, 0.12.10. I really think my problem with CUDA versions later than 12.6 is related to issue #12872. The compile error points to a problem with the definitions in the include files. If I revert my system include file changes that I did I go back to issue #12872 where the build fails because it can't find math.h in the #include_next statement. This issue stated with Ollama version 0.11.3 and has not been fixed. It seems to be related to a change to the cmake build files that was done in Ollama version 0.11.3.

@abcbarryn commented on GitHub (Nov 14, 2025): Also, as far as others not having issues, read through this thread... https://github.com/ollama/ollama/issues/12618 It is full of users having issues with recent versions of Ollama, there are users in the thread that are having trouble with both Windows and Linux systems and can't get Ollama to run properly at all. At least my system is working (very well, actually) with my build using CUDA 12.6 and the latest Ollama, 0.12.10. I really think my problem with CUDA versions later than 12.6 is related to issue #12872. The compile error points to a problem with the definitions in the include files. If I revert my system include file changes that I did I go back to issue #12872 where the build fails because it can't find math.h in the #include_next statement. This issue stated with Ollama version 0.11.3 and has not been fixed. It seems to be related to a change to the cmake build files that was done in Ollama version 0.11.3.

GiteaMirror commented

2026-04-29 08:26:45 -05:00

@rick-github commented on GitHub (Nov 14, 2025):

CMAKE_CUDA_COMPILER does match! Don't you know how update-alternatives works? The directory /usr/local/cuda is a symlink that points to the active CUDA toolkit directory.

If it's the same, why are you setting CMAKE_CUDA_COMPILER? Why are you setting any of the environment variables?

Also, as far as others not having issues, read through this thread... #12618

Not a build failure.

If I revert my system include file changes that I did I go back to issue #12872 where the build fails because it can't find math.h in the #include_next statement. This issue stated with Ollama version 0.11.3 and has not been fixed. It seems to be related to a change to the cmake build files that was done in Ollama version 0.11.3.

And yet, a binary of ollama has been released at every version change for multiple hardware platforms. If 0.11.3 and later fails to build with CUDA, how is it that I can build 0.12.10 with CUDA?

Obviously, ollama is not bug free. But the build failures you are experiencing are not solely issues with ollama, they are a combination of ollama and your environment. If you can provide the requested information, then perhaps the inconsistencies can be uncovered. If it turns out to be an ollama issue, then it can be addressed. But pasting only the error error messages from bulld attempts where fundamental variables are being overridden is not testing the build process, it's playing a game of "guess what's changed". And since I'm not psychic, I have no insight into what those changes might be. First rule in experiments is only change one variable at a time.

@rick-github commented on GitHub (Nov 14, 2025): > CMAKE_CUDA_COMPILER does match! Don't you know how update-alternatives works? The directory /usr/local/cuda is a symlink that points to the active CUDA toolkit directory. If it's the same, why are you setting `CMAKE_CUDA_COMPILER`? Why are you setting any of the environment variables? > Also, as far as others not having issues, read through this thread... [#12618](https://github.com/ollama/ollama/issues/12618) Not a build failure. > If I revert my system include file changes that I did I go back to issue [#12872](https://github.com/ollama/ollama/issues/12872) where the build fails because it can't find math.h in the #include_next statement. This issue stated with Ollama version 0.11.3 and has not been fixed. It seems to be related to a change to the cmake build files that was done in Ollama version 0.11.3. And yet, a binary of ollama has been released at every version change for multiple hardware platforms. If 0.11.3 and later fails to build with CUDA, how is it that I can [build 0.12.10 with CUDA](https://github.com/ollama/ollama/issues/13075#issuecomment-3529210021)? Obviously, ollama is not bug free. But the build failures you are experiencing are not solely issues with ollama, they are a combination of ollama and your environment. If you can provide the [requested information](https://github.com/ollama/ollama/issues/13075#issuecomment-3527502373), then perhaps the inconsistencies can be uncovered. If it turns out to be an ollama issue, then it can be addressed. But pasting only the error error messages from bulld attempts where fundamental variables are being overridden is not testing the build process, it's playing a game of "guess what's changed". And since I'm not psychic, I have no insight into what those changes might be. [First rule](https://www.sciencing.com/should-only-test-one-variable-time-experiment-11414533/) in experiments is only change one variable at a time.

GiteaMirror commented

@abcbarryn commented on GitHub (Nov 14, 2025):

I am setting because /usr/local/cuda/bin is not in the system PATH, although I did try adding it to the PATH but that did not help. You can build Ollama because you are using a different Linux distribution and I can build Ollama 0.12.10 with CUDA, just not CUDA versions later than 12.6. The problem has something to do with the include path. You want the yaml files? I have been changing one thing at a time and testing. That's how I found that it would build with CUDA 12.6. I'll attach the yaml files when I have time to try another build with CUDA 12.9. As far as the issues not being solely with Ollama, I can and have compiled many other programs, including CUDA examples, and other Ollama versions. These problems are specific to certain Ollama versions when combined with certain CUDA versions. Other Ollama versions compile fine, and 0.12.10 compiles as long as I use CUDA 12.6 or earlier. It's something with the include path that cmake is setting up. You obviously don't know what the issue is. It could be something the SuSE Linux distribution is doing. I don't think there are many people running Ollama on SuSE. Maybe the yaml files will help.

@abcbarryn commented on GitHub (Nov 14, 2025): I am setting because /usr/local/cuda/bin is not in the system PATH, although I did try adding it to the PATH but that did not help. You can build Ollama because you are using a different Linux distribution and I can build Ollama 0.12.10 with CUDA, just not CUDA versions later than 12.6. The problem has something to do with the include path. You want the yaml files? I have been changing one thing at a time and testing. That's how I found that it would build with CUDA 12.6. I'll attach the yaml files when I have time to try another build with CUDA 12.9. As far as the issues not being solely with Ollama, I can and have compiled many other programs, including CUDA examples, and other Ollama versions. These problems are specific to certain Ollama versions when combined with certain CUDA versions. Other Ollama versions compile fine, and 0.12.10 compiles as long as I use CUDA 12.6 or earlier. It's something with the include path that cmake is setting up. You obviously don't know what the issue is. It could be something the SuSE Linux distribution is doing. I don't think there are many people running Ollama on SuSE. Maybe the yaml files will help.

GiteaMirror commented

@rick-github commented on GitHub (Nov 14, 2025):

Your're right, I don't SuSE. I had a little bit of spare time this evening and took a look. I spun up a SuSE VM.

rick@ddr4-suse:~> cat /etc/os-release 
NAME="openSUSE Leap"
VERSION="15.6"
ID="opensuse-leap"
ID_LIKE="suse opensuse"
VERSION_ID="15.6"
PRETTY_NAME="openSUSE Leap 15.6"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:opensuse:leap:15.6"
BUG_REPORT_URL="https://bugs.opensuse.org"
HOME_URL="https://www.opensuse.org/"
DOCUMENTATION_URL="https://en.opensuse.org/Portal:Leap"
LOGO="distributor-logo-Leap"

Installed dev packages.

rick@ddr4-suse:~> sudo -s
ddr4-suse:/home/rick # zypper addrepo https://developer.download.nvidia.com/compute/cuda/repos/opensuse15/x86_64/cuda-opensuse15.repo
ddr4-suse:/home/rick # zypper refresh
ddr4-suse:/home/rick # zypper install -y cmake-full gcc14 gcc14-c++ go1.25 cuda-toolkit-12-8
ddr4-suse:/home/rick # update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-14 143 --slave /usr/bin/cc cc /usr/bin/gcc-14
ddr4-suse:/home/rick # update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-14 143 --slave /usr/bin/c++ c++ /usr/bin/g++-14
ddr4-suse:/home/rick # exit
rick@ddr4-suse:~> echo 'export PATH=/usr/local/cuda/bin:$PATH' >> ~/.bashrc
rick@ddr4-suse:~> . ~/.bashrc

Set up the build area.

rick@ddr4-suse:~> mkdir ollama-13075 && cd $_
rick@ddr4-suse:~/ollama-13075> wget https://github.com/ollama/ollama/archive/refs/tags/v0.12.10.tar.gz
rick@ddr4-suse:~/ollama-13075> tar zxf v0.12.10.tar.gz
rick@ddr4-suse:~/ollama-13075> cd ollama-0.12.10/

Built ollama:

rick@ddr4-suse:~/ollama-13075/ollama-0.12.10> cmake -B build && cmake --build build && go run . serve
-- The C compiler identification is GNU 14.3.0
-- The CXX compiler identification is GNU 14.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- GGML_SYSTEM_ARCH: x86
-- Including CPU backend
-- x86 detected
-- Adding CPU backend variant ggml-cpu-x64:  
-- x86 detected
-- Adding CPU backend variant ggml-cpu-sse42: -msse4.2 GGML_SSE42
-- x86 detected
-- Adding CPU backend variant ggml-cpu-sandybridge: -msse4.2;-mavx GGML_SSE42;GGML_AVX
-- x86 detected
-- Adding CPU backend variant ggml-cpu-haswell: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2 GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2
-- x86 detected
-- Adding CPU backend variant ggml-cpu-skylakex: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavx512f;-mavx512cd;-mavx512vl;-mavx512dq;-mavx512bw GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX512
-- x86 detected
-- Adding CPU backend variant ggml-cpu-icelake: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavx512f;-mavx512cd;-mavx512vl;-mavx512dq;-mavx512bw;-mavx512vbmi;-mavx512vnni GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX512;GGML_AVX512_VBMI;GGML_AVX512_VNNI
-- x86 detected
-- Adding CPU backend variant ggml-cpu-alderlake: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavxvnni GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX_VNNI
-- Looking for a CUDA compiler
-- Looking for a CUDA compiler - /usr/local/cuda/bin/nvcc
-- Found CUDAToolkit: /usr/local/cuda/targets/x86_64-linux/include (found version "12.8.93") 
-- CUDA Toolkit found
-- Using CUDA architectures: native
-- The CUDA compiler identification is NVIDIA 12.8.93
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Looking for a HIP compiler
-- Looking for a HIP compiler - NOTFOUND
-- Could NOT find Vulkan (missing: Vulkan_LIBRARY Vulkan_INCLUDE_DIR) (found version "")
-- Configuring done (4.5s)
-- Generating done (0.0s)
-- Build files have been written to: /home/rick/ollama-13075/ollama-0.12.10/build
[  1%] Building C object ml/backend/ggml/ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o
...
[100%] Linking CUDA shared module ../../../../../../lib/ollama/libggml-cuda.so
[100%] Built target ggml-cuda
go: downloading github.com/spf13/cobra v1.7.0
...
go: downloading github.com/go-playground/locales v0.14.1
Couldn't find '/home/rick/.ollama/id_ed25519'. Generating new private key.
Your new public key is: 

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAh4OpoBmVwMev7ihFMP0AEi+tLl8Cwsxmjlc+uFNkek

time=2025-11-14T23:08:10.607+01:00 level=INFO source=routes.go:1525 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:4096 OLLAMA_DEBUG:INFO OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/home/rick/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[ollama.com] OLLAMA_SCHED_SPREAD:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"

Tested it:

rick@ddr4-suse:~/ollama-13075/ollama-0.12.10> go run . list
NAME    ID    SIZE    MODIFIED 
rick@ddr4-suse:~/ollama-13075/ollama-0.12.10> go run . run qwen2.5:0.5b hello
pulling manifest 
pulling c5396e06af29: 100% ▕██████████████████▏ 397 MB                         
pulling 66b9ea09bd5b: 100% ▕██████████████████▏   68 B                         
pulling eb4402837c78: 100% ▕██████████████████▏ 1.5 KB                         
pulling 832dd9e00a68: 100% ▕██████████████████▏  11 KB                         
pulling 005f95c74751: 100% ▕██████████████████▏  490 B                         
verifying sha256 digest 
writing manifest 
success 
Hello! How can I assist you today? Please let me know if there's anything 
specific you'd like to talk about or any questions you have.

The VM doesn't have a GPU so the test wasn't accelerated, but the CUDA library was built which seems to be the sticking point for your builds.

rick@ddr4-suse:~/ollama-13075/ollama-0.12.10> ls -l ./build/lib/ollama/libggml-cuda.so
-rwxr-xr-x 1 rick users 87433488 14. Nov 23:06 ./build/lib/ollama/libggml-cuda.so

@rick-github commented on GitHub (Nov 14, 2025): Your're right, I don't SuSE. I had a little bit of spare time this evening and took a look. I spun up a SuSE VM. ```console rick@ddr4-suse:~> cat /etc/os-release NAME="openSUSE Leap" VERSION="15.6" ID="opensuse-leap" ID_LIKE="suse opensuse" VERSION_ID="15.6" PRETTY_NAME="openSUSE Leap 15.6" ANSI_COLOR="0;32" CPE_NAME="cpe:/o:opensuse:leap:15.6" BUG_REPORT_URL="https://bugs.opensuse.org" HOME_URL="https://www.opensuse.org/" DOCUMENTATION_URL="https://en.opensuse.org/Portal:Leap" LOGO="distributor-logo-Leap" ``` Installed dev packages. ```console rick@ddr4-suse:~> sudo -s ddr4-suse:/home/rick # zypper addrepo https://developer.download.nvidia.com/compute/cuda/repos/opensuse15/x86_64/cuda-opensuse15.repo ddr4-suse:/home/rick # zypper refresh ddr4-suse:/home/rick # zypper install -y cmake-full gcc14 gcc14-c++ go1.25 cuda-toolkit-12-8 ddr4-suse:/home/rick # update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-14 143 --slave /usr/bin/cc cc /usr/bin/gcc-14 ddr4-suse:/home/rick # update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-14 143 --slave /usr/bin/c++ c++ /usr/bin/g++-14 ddr4-suse:/home/rick # exit rick@ddr4-suse:~> echo 'export PATH=/usr/local/cuda/bin:$PATH' >> ~/.bashrc rick@ddr4-suse:~> . ~/.bashrc ``` Set up the build area. ```console rick@ddr4-suse:~> mkdir ollama-13075 && cd $_ rick@ddr4-suse:~/ollama-13075> wget https://github.com/ollama/ollama/archive/refs/tags/v0.12.10.tar.gz rick@ddr4-suse:~/ollama-13075> tar zxf v0.12.10.tar.gz rick@ddr4-suse:~/ollama-13075> cd ollama-0.12.10/ ``` Built ollama: ```console rick@ddr4-suse:~/ollama-13075/ollama-0.12.10> cmake -B build && cmake --build build && go run . serve -- The C compiler identification is GNU 14.3.0 -- The CXX compiler identification is GNU 14.3.0 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /usr/bin/cc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /usr/bin/c++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success -- Found Threads: TRUE -- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF -- CMAKE_SYSTEM_PROCESSOR: x86_64 -- GGML_SYSTEM_ARCH: x86 -- Including CPU backend -- x86 detected -- Adding CPU backend variant ggml-cpu-x64: -- x86 detected -- Adding CPU backend variant ggml-cpu-sse42: -msse4.2 GGML_SSE42 -- x86 detected -- Adding CPU backend variant ggml-cpu-sandybridge: -msse4.2;-mavx GGML_SSE42;GGML_AVX -- x86 detected -- Adding CPU backend variant ggml-cpu-haswell: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2 GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2 -- x86 detected -- Adding CPU backend variant ggml-cpu-skylakex: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavx512f;-mavx512cd;-mavx512vl;-mavx512dq;-mavx512bw GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX512 -- x86 detected -- Adding CPU backend variant ggml-cpu-icelake: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavx512f;-mavx512cd;-mavx512vl;-mavx512dq;-mavx512bw;-mavx512vbmi;-mavx512vnni GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX512;GGML_AVX512_VBMI;GGML_AVX512_VNNI -- x86 detected -- Adding CPU backend variant ggml-cpu-alderlake: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavxvnni GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX_VNNI -- Looking for a CUDA compiler -- Looking for a CUDA compiler - /usr/local/cuda/bin/nvcc -- Found CUDAToolkit: /usr/local/cuda/targets/x86_64-linux/include (found version "12.8.93") -- CUDA Toolkit found -- Using CUDA architectures: native -- The CUDA compiler identification is NVIDIA 12.8.93 -- Detecting CUDA compiler ABI info -- Detecting CUDA compiler ABI info - done -- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped -- Detecting CUDA compile features -- Detecting CUDA compile features - done -- Looking for a HIP compiler -- Looking for a HIP compiler - NOTFOUND -- Could NOT find Vulkan (missing: Vulkan_LIBRARY Vulkan_INCLUDE_DIR) (found version "") -- Configuring done (4.5s) -- Generating done (0.0s) -- Build files have been written to: /home/rick/ollama-13075/ollama-0.12.10/build [ 1%] Building C object ml/backend/ggml/ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o ... [100%] Linking CUDA shared module ../../../../../../lib/ollama/libggml-cuda.so [100%] Built target ggml-cuda go: downloading github.com/spf13/cobra v1.7.0 ... go: downloading github.com/go-playground/locales v0.14.1 Couldn't find '/home/rick/.ollama/id_ed25519'. Generating new private key. Your new public key is: ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAh4OpoBmVwMev7ihFMP0AEi+tLl8Cwsxmjlc+uFNkek time=2025-11-14T23:08:10.607+01:00 level=INFO source=routes.go:1525 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:4096 OLLAMA_DEBUG:INFO OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/home/rick/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[ollama.com] OLLAMA_SCHED_SPREAD:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]" ``` Tested it: ```console rick@ddr4-suse:~/ollama-13075/ollama-0.12.10> go run . list NAME ID SIZE MODIFIED rick@ddr4-suse:~/ollama-13075/ollama-0.12.10> go run . run qwen2.5:0.5b hello pulling manifest pulling c5396e06af29: 100% ▕██████████████████▏ 397 MB pulling 66b9ea09bd5b: 100% ▕██████████████████▏ 68 B pulling eb4402837c78: 100% ▕██████████████████▏ 1.5 KB pulling 832dd9e00a68: 100% ▕██████████████████▏ 11 KB pulling 005f95c74751: 100% ▕██████████████████▏ 490 B verifying sha256 digest writing manifest success Hello! How can I assist you today? Please let me know if there's anything specific you'd like to talk about or any questions you have. ``` The VM doesn't have a GPU so the test wasn't accelerated, but the CUDA library was built which seems to be the sticking point for your builds. ```console rick@ddr4-suse:~/ollama-13075/ollama-0.12.10> ls -l ./build/lib/ollama/libggml-cuda.so -rwxr-xr-x 1 rick users 87433488 14. Nov 23:06 ./build/lib/ollama/libggml-cuda.so ```

GiteaMirror commented

@YonTracks commented on GitHub (Nov 15, 2025):

howdy, for me. WINDOWS, maybe will help.

[cmake] -- Adding CPU backend variant ggml-cpu-alderlake: /arch:AVX2 GGML_AVX2;GGML_FMA;GGML_F16C;__AVXVNNI__;GGML_AVX_VNNI;__BMI2__;GGML_BMI2
[cmake] -- Looking for a CUDA compiler
[cmake] -- Looking for a CUDA compiler - C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v13.0/bin/nvcc.exe
[cmake] -- Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v13.0/include (found version "13.0.48")
[cmake] -- CUDA Toolkit found
[cmake] -- Using CUDA architectures: native
[cmake] -- The CUDA compiler identification is NVIDIA 13.0.48
[cmake] -- Detecting CUDA compiler ABI info
[cmake] -- Detecting CUDA compiler ABI info - done

previous was auto build.

heres the actual full build.

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----        15/11/2025   5:12 PM                ollama
Preset CMake variables:

  CMAKE_BUILD_TYPE="Release"
  CMAKE_MSVC_RUNTIME_LIBRARY="MultiThreaded"

-- Selecting Windows SDK version 10.0.26100.0 to target Windows 10.0.26200.
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: AMD64
-- CMAKE_GENERATOR_PLATFORM:
-- GGML_SYSTEM_ARCH: x86
-- Including CPU backend
-- x86 detected
-- Adding CPU backend variant ggml-cpu-x64:
-- x86 detected
-- Adding CPU backend variant ggml-cpu-sse42: /arch:SSE4.2 GGML_SSE42
-- x86 detected
-- Adding CPU backend variant ggml-cpu-sandybridge: /arch:AVX GGML_AVX
-- x86 detected
-- Adding CPU backend variant ggml-cpu-haswell: /arch:AVX2 GGML_AVX2;GGML_FMA;GGML_F16C;__BMI2__;GGML_BMI2
-- x86 detected
-- Adding CPU backend variant ggml-cpu-skylakex: /arch:AVX512 GGML_AVX512;__BMI2__;GGML_BMI2
-- x86 detected
-- Adding CPU backend variant ggml-cpu-icelake: /arch:AVX512 GGML_AVX512;__AVX512VBMI__;__AVX512VNNI__;GGML_AVX512_VNNI;__BMI2__;GGML_BMI2
-- x86 detected
-- Adding CPU backend variant ggml-cpu-alderlake: /arch:AVX2 GGML_AVX2;GGML_FMA;GGML_F16C;__AVXVNNI__;GGML_AVX_VNNI;__BMI2__;GGML_BMI2
-- CUDA Toolkit found
-- Using CUDA architectures: native
-- Could NOT find Vulkan (missing: Vulkan_LIBRARY Vulkan_INCLUDE_DIR) (found version "")
-- Configuring done (1.2s)
-- Generating done (0.5s)
-- Build files have been written to: C:/ollama-yontracks/ollama-yontracks-test/build/cpu
MSBuild version 17.14.23+b0019275e for .NET Framework

  ggml-cpu-sandybridge-feats.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\ml\backend\ggml\ggml\src\ggml-cpu-sandybridge-feats.dir\Release\ggml-cpu-sandybridge-feats.lib
  ggml-cpu-sse42-feats.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\ml\backend\ggml\ggml\src\ggml-cpu-sse42-feats.dir\Release\ggml-cpu-sse42-feats.lib
  ggml-cpu-x64-feats.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\ml\backend\ggml\ggml\src\ggml-cpu-x64-feats.dir\Release\ggml-cpu-x64-feats.lib
  ggml-cpu-haswell-feats.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\ml\backend\ggml\ggml\src\ggml-cpu-haswell-feats.dir\Release\ggml-cpu-haswell-feats.lib
  ggml-cpu-alderlake-feats.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\ml\backend\ggml\ggml\src\ggml-cpu-alderlake-feats.dir\Release\ggml-cpu-alderlake-feats.lib
  ggml-cpu-icelake-feats.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\ml\backend\ggml\ggml\src\ggml-cpu-icelake-feats.dir\Release\ggml-cpu-icelake-feats.lib
  ggml-cpu-skylakex-feats.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\ml\backend\ggml\ggml\src\ggml-cpu-skylakex-feats.dir\Release\ggml-cpu-skylakex-feats.lib
  ggml-base.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\lib\ollama\ggml-base.dll
  ggml-cpu-sandybridge.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\lib\ollama\ggml-cpu-sandybridge.dll
  ggml-cpu-sse42.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\lib\ollama\ggml-cpu-sse42.dll
  ggml-cpu-x64.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\lib\ollama\ggml-cpu-x64.dll
  ggml-cpu-haswell.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\lib\ollama\ggml-cpu-haswell.dll
  ggml-cpu-alderlake.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\lib\ollama\ggml-cpu-alderlake.dll
  ggml-cpu-icelake.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\lib\ollama\ggml-cpu-icelake.dll
  ggml-cpu-skylakex.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\lib\ollama\ggml-cpu-skylakex.dll
-- Install configuration: "Release"
-- Installing: C:/ollama-yontracks/ollama-yontracks-test/dist/windows-amd64/lib/ollama/ggml-base.dll
-- Installing: C:/ollama-yontracks/ollama-yontracks-test/dist/windows-amd64/lib/ollama/ggml-cpu-alderlake.dll
-- Installing: C:/ollama-yontracks/ollama-yontracks-test/dist/windows-amd64/lib/ollama/ggml-cpu-haswell.dll
-- Installing: C:/ollama-yontracks/ollama-yontracks-test/dist/windows-amd64/lib/ollama/ggml-cpu-icelake.dll
-- Installing: C:/ollama-yontracks/ollama-yontracks-test/dist/windows-amd64/lib/ollama/ggml-cpu-sandybridge.dll
-- Installing: C:/ollama-yontracks/ollama-yontracks-test/dist/windows-amd64/lib/ollama/ggml-cpu-skylakex.dll
-- Installing: C:/ollama-yontracks/ollama-yontracks-test/dist/windows-amd64/lib/ollama/ggml-cpu-sse42.dll
-- Installing: C:/ollama-yontracks/ollama-yontracks-test/dist/windows-amd64/lib/ollama/ggml-cpu-x64.dll

what happens here^ if detected cuda 12 and or others with. issues for me if multi versions of cuda!

CUDA v12 not detected, skipping
^^^^^^^^^^^^^^^^^^^^^


Building CUDA v13 backend libraries C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.0
Preset CMake variables:

  CMAKE_BUILD_TYPE="Release"
  CMAKE_CUDA_ARCHITECTURES="75-virtual;80-virtual;86-virtual;87-virtual;89-virtual;90-virtual;90a-virtual;100-virtual;103-virtual;110-virtual;120-virtual;121-virtual"
  CMAKE_CUDA_FLAGS="-t 2"
  CMAKE_MSVC_RUNTIME_LIBRARY="MultiThreaded"
  OLLAMA_RUNNER_DIR="cuda_v13"

-- Selecting Windows SDK version 10.0.26100.0 to target Windows 10.0.26200.
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: AMD64
-- CMAKE_GENERATOR_PLATFORM:
-- GGML_SYSTEM_ARCH: x86
-- Including CPU backend
-- x86 detected
-- Adding CPU backend variant ggml-cpu-x64:
-- x86 detected
-- Adding CPU backend variant ggml-cpu-sse42: /arch:SSE4.2 GGML_SSE42
-- x86 detected
-- Adding CPU backend variant ggml-cpu-sandybridge: /arch:AVX GGML_AVX
-- x86 detected
-- Adding CPU backend variant ggml-cpu-haswell: /arch:AVX2 GGML_AVX2;GGML_FMA;GGML_F16C;__BMI2__;GGML_BMI2
-- x86 detected
-- Adding CPU backend variant ggml-cpu-skylakex: /arch:AVX512 GGML_AVX512;__BMI2__;GGML_BMI2
-- x86 detected
-- Adding CPU backend variant ggml-cpu-icelake: /arch:AVX512 GGML_AVX512;__AVX512VBMI__;__AVX512VNNI__;GGML_AVX512_VNNI;__BMI2__;GGML_BMI2
-- x86 detected
-- Adding CPU backend variant ggml-cpu-alderlake: /arch:AVX2 GGML_AVX2;GGML_FMA;GGML_F16C;__AVXVNNI__;GGML_AVX_VNNI;__BMI2__;GGML_BMI2
-- CUDA Toolkit found
-- Using CUDA architectures: 75-virtual;80-virtual;86-virtual;87-virtual;89-virtual;90-virtual;90a-virtual;100-virtual;103-virtual;110-virtual;120-virtual;121-virtual
-- Could NOT find Vulkan (missing: Vulkan_LIBRARY Vulkan_INCLUDE_DIR) (found version "")
-- Configuring done (1.2s)
-- Generating done (0.5s)
-- Build files have been written to: C:/ollama-yontracks/ollama-yontracks-test/build/cuda_v13
MSBuild version 17.14.23+b0019275e for .NET Framework

  ggml-base.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cuda_v13\lib\ollama\ggml-base.dll
  ggml-cuda.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cuda_v13\lib\ollama\ggml-cuda.dll
-- Install configuration: "Release"
-- Installing: C:/ollama-yontracks/ollama-yontracks-test/dist/windows-amd64/lib/ollama/cuda_v13/ggml-cuda.dll
-- Installing: C:/ollama-yontracks/ollama-yontracks-test/dist/windows-amd64/lib/ollama/cuda_v13/cublas64_13.dll
-- Installing: C:/ollama-yontracks/ollama-yontracks-test/dist/windows-amd64/lib/ollama/cuda_v13/cublasLt64_13.dll
ROCm not detected, skipping
Vulkan not detected, skipping
Building ollama CLI
Building Ollama App 0.12.11-yontracks with package version 0.12.11

do check your cuda toolkits and install etc.

start fresh!
I also remember nesting being a issue here, like ../../../../build folder.
good luck.

@YonTracks commented on GitHub (Nov 15, 2025): howdy, for me. WINDOWS, maybe will help. ``` [cmake] -- Adding CPU backend variant ggml-cpu-alderlake: /arch:AVX2 GGML_AVX2;GGML_FMA;GGML_F16C;__AVXVNNI__;GGML_AVX_VNNI;__BMI2__;GGML_BMI2 [cmake] -- Looking for a CUDA compiler [cmake] -- Looking for a CUDA compiler - C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v13.0/bin/nvcc.exe [cmake] -- Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v13.0/include (found version "13.0.48") [cmake] -- CUDA Toolkit found [cmake] -- Using CUDA architectures: native [cmake] -- The CUDA compiler identification is NVIDIA 13.0.48 [cmake] -- Detecting CUDA compiler ABI info [cmake] -- Detecting CUDA compiler ABI info - done ``` previous was auto build. heres the actual full build. ``` Mode LastWriteTime Length Name ---- ------------- ------ ---- d----- 15/11/2025 5:12 PM ollama Preset CMake variables: CMAKE_BUILD_TYPE="Release" CMAKE_MSVC_RUNTIME_LIBRARY="MultiThreaded" -- Selecting Windows SDK version 10.0.26100.0 to target Windows 10.0.26200. -- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF -- CMAKE_SYSTEM_PROCESSOR: AMD64 -- CMAKE_GENERATOR_PLATFORM: -- GGML_SYSTEM_ARCH: x86 -- Including CPU backend -- x86 detected -- Adding CPU backend variant ggml-cpu-x64: -- x86 detected -- Adding CPU backend variant ggml-cpu-sse42: /arch:SSE4.2 GGML_SSE42 -- x86 detected -- Adding CPU backend variant ggml-cpu-sandybridge: /arch:AVX GGML_AVX -- x86 detected -- Adding CPU backend variant ggml-cpu-haswell: /arch:AVX2 GGML_AVX2;GGML_FMA;GGML_F16C;__BMI2__;GGML_BMI2 -- x86 detected -- Adding CPU backend variant ggml-cpu-skylakex: /arch:AVX512 GGML_AVX512;__BMI2__;GGML_BMI2 -- x86 detected -- Adding CPU backend variant ggml-cpu-icelake: /arch:AVX512 GGML_AVX512;__AVX512VBMI__;__AVX512VNNI__;GGML_AVX512_VNNI;__BMI2__;GGML_BMI2 -- x86 detected -- Adding CPU backend variant ggml-cpu-alderlake: /arch:AVX2 GGML_AVX2;GGML_FMA;GGML_F16C;__AVXVNNI__;GGML_AVX_VNNI;__BMI2__;GGML_BMI2 -- CUDA Toolkit found -- Using CUDA architectures: native -- Could NOT find Vulkan (missing: Vulkan_LIBRARY Vulkan_INCLUDE_DIR) (found version "") -- Configuring done (1.2s) -- Generating done (0.5s) -- Build files have been written to: C:/ollama-yontracks/ollama-yontracks-test/build/cpu MSBuild version 17.14.23+b0019275e for .NET Framework ggml-cpu-sandybridge-feats.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\ml\backend\ggml\ggml\src\ggml-cpu-sandybridge-feats.dir\Release\ggml-cpu-sandybridge-feats.lib ggml-cpu-sse42-feats.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\ml\backend\ggml\ggml\src\ggml-cpu-sse42-feats.dir\Release\ggml-cpu-sse42-feats.lib ggml-cpu-x64-feats.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\ml\backend\ggml\ggml\src\ggml-cpu-x64-feats.dir\Release\ggml-cpu-x64-feats.lib ggml-cpu-haswell-feats.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\ml\backend\ggml\ggml\src\ggml-cpu-haswell-feats.dir\Release\ggml-cpu-haswell-feats.lib ggml-cpu-alderlake-feats.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\ml\backend\ggml\ggml\src\ggml-cpu-alderlake-feats.dir\Release\ggml-cpu-alderlake-feats.lib ggml-cpu-icelake-feats.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\ml\backend\ggml\ggml\src\ggml-cpu-icelake-feats.dir\Release\ggml-cpu-icelake-feats.lib ggml-cpu-skylakex-feats.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\ml\backend\ggml\ggml\src\ggml-cpu-skylakex-feats.dir\Release\ggml-cpu-skylakex-feats.lib ggml-base.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\lib\ollama\ggml-base.dll ggml-cpu-sandybridge.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\lib\ollama\ggml-cpu-sandybridge.dll ggml-cpu-sse42.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\lib\ollama\ggml-cpu-sse42.dll ggml-cpu-x64.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\lib\ollama\ggml-cpu-x64.dll ggml-cpu-haswell.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\lib\ollama\ggml-cpu-haswell.dll ggml-cpu-alderlake.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\lib\ollama\ggml-cpu-alderlake.dll ggml-cpu-icelake.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\lib\ollama\ggml-cpu-icelake.dll ggml-cpu-skylakex.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cpu\lib\ollama\ggml-cpu-skylakex.dll -- Install configuration: "Release" -- Installing: C:/ollama-yontracks/ollama-yontracks-test/dist/windows-amd64/lib/ollama/ggml-base.dll -- Installing: C:/ollama-yontracks/ollama-yontracks-test/dist/windows-amd64/lib/ollama/ggml-cpu-alderlake.dll -- Installing: C:/ollama-yontracks/ollama-yontracks-test/dist/windows-amd64/lib/ollama/ggml-cpu-haswell.dll -- Installing: C:/ollama-yontracks/ollama-yontracks-test/dist/windows-amd64/lib/ollama/ggml-cpu-icelake.dll -- Installing: C:/ollama-yontracks/ollama-yontracks-test/dist/windows-amd64/lib/ollama/ggml-cpu-sandybridge.dll -- Installing: C:/ollama-yontracks/ollama-yontracks-test/dist/windows-amd64/lib/ollama/ggml-cpu-skylakex.dll -- Installing: C:/ollama-yontracks/ollama-yontracks-test/dist/windows-amd64/lib/ollama/ggml-cpu-sse42.dll -- Installing: C:/ollama-yontracks/ollama-yontracks-test/dist/windows-amd64/lib/ollama/ggml-cpu-x64.dll what happens here^ if detected cuda 12 and or others with. issues for me if multi versions of cuda! CUDA v12 not detected, skipping ^^^^^^^^^^^^^^^^^^^^^ Building CUDA v13 backend libraries C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.0 Preset CMake variables: CMAKE_BUILD_TYPE="Release" CMAKE_CUDA_ARCHITECTURES="75-virtual;80-virtual;86-virtual;87-virtual;89-virtual;90-virtual;90a-virtual;100-virtual;103-virtual;110-virtual;120-virtual;121-virtual" CMAKE_CUDA_FLAGS="-t 2" CMAKE_MSVC_RUNTIME_LIBRARY="MultiThreaded" OLLAMA_RUNNER_DIR="cuda_v13" -- Selecting Windows SDK version 10.0.26100.0 to target Windows 10.0.26200. -- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF -- CMAKE_SYSTEM_PROCESSOR: AMD64 -- CMAKE_GENERATOR_PLATFORM: -- GGML_SYSTEM_ARCH: x86 -- Including CPU backend -- x86 detected -- Adding CPU backend variant ggml-cpu-x64: -- x86 detected -- Adding CPU backend variant ggml-cpu-sse42: /arch:SSE4.2 GGML_SSE42 -- x86 detected -- Adding CPU backend variant ggml-cpu-sandybridge: /arch:AVX GGML_AVX -- x86 detected -- Adding CPU backend variant ggml-cpu-haswell: /arch:AVX2 GGML_AVX2;GGML_FMA;GGML_F16C;__BMI2__;GGML_BMI2 -- x86 detected -- Adding CPU backend variant ggml-cpu-skylakex: /arch:AVX512 GGML_AVX512;__BMI2__;GGML_BMI2 -- x86 detected -- Adding CPU backend variant ggml-cpu-icelake: /arch:AVX512 GGML_AVX512;__AVX512VBMI__;__AVX512VNNI__;GGML_AVX512_VNNI;__BMI2__;GGML_BMI2 -- x86 detected -- Adding CPU backend variant ggml-cpu-alderlake: /arch:AVX2 GGML_AVX2;GGML_FMA;GGML_F16C;__AVXVNNI__;GGML_AVX_VNNI;__BMI2__;GGML_BMI2 -- CUDA Toolkit found -- Using CUDA architectures: 75-virtual;80-virtual;86-virtual;87-virtual;89-virtual;90-virtual;90a-virtual;100-virtual;103-virtual;110-virtual;120-virtual;121-virtual -- Could NOT find Vulkan (missing: Vulkan_LIBRARY Vulkan_INCLUDE_DIR) (found version "") -- Configuring done (1.2s) -- Generating done (0.5s) -- Build files have been written to: C:/ollama-yontracks/ollama-yontracks-test/build/cuda_v13 MSBuild version 17.14.23+b0019275e for .NET Framework ggml-base.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cuda_v13\lib\ollama\ggml-base.dll ggml-cuda.vcxproj -> C:\ollama-yontracks\ollama-yontracks-test\build\cuda_v13\lib\ollama\ggml-cuda.dll -- Install configuration: "Release" -- Installing: C:/ollama-yontracks/ollama-yontracks-test/dist/windows-amd64/lib/ollama/cuda_v13/ggml-cuda.dll -- Installing: C:/ollama-yontracks/ollama-yontracks-test/dist/windows-amd64/lib/ollama/cuda_v13/cublas64_13.dll -- Installing: C:/ollama-yontracks/ollama-yontracks-test/dist/windows-amd64/lib/ollama/cuda_v13/cublasLt64_13.dll ROCm not detected, skipping Vulkan not detected, skipping Building ollama CLI Building Ollama App 0.12.11-yontracks with package version 0.12.11 ``` do check your cuda toolkits and install etc. start fresh! I also remember nesting being a issue here, like ../../../../build folder. good luck.

GiteaMirror commented

2026-04-29 08:26:47 -05:00

@YonTracks commented on GitHub (Nov 15, 2025):

2 /usr/local/cuda-12.6 126 manual mode

  Selection    Path                  Priority   Status
------------------------------------------------------------
  0            /usr/local/cuda-13.0   130       auto mode
  1            /usr/local/cuda-11.8   118       manual mode
* 2            /usr/local/cuda-12.6   126       manual mode
  3            /usr/local/cuda-12.9   129       manual mode
  4            /usr/local/cuda-13.0   130       manual mode

here you selected 12.6? and manual mode? not sure, but it found 12.9?

-- Looking for a CUDA compiler - /usr/local/cuda-12.9/bin/nvcc
-- Found CUDAToolkit: /usr/local/cuda-12.9/targets/x86_64-linux/include;/usr/include (found version "12.9.86")

far out I only just realized this is linux lol, should I move this to windows? lol, sorry. bugger, maybe still relevant, I think.
not sure. good luck

@YonTracks commented on GitHub (Nov 15, 2025): > * 2 /usr/local/cuda-12.6 126 manual mode ``` Selection Path Priority Status ------------------------------------------------------------ 0 /usr/local/cuda-13.0 130 auto mode 1 /usr/local/cuda-11.8 118 manual mode * 2 /usr/local/cuda-12.6 126 manual mode 3 /usr/local/cuda-12.9 129 manual mode 4 /usr/local/cuda-13.0 130 manual mode ``` here you selected 12.6? and manual mode? not sure, but it found 12.9? ```-- Looking for a CUDA compiler -- Looking for a CUDA compiler - /usr/local/cuda-12.9/bin/nvcc -- Found CUDAToolkit: /usr/local/cuda-12.9/targets/x86_64-linux/include;/usr/include (found version "12.9.86") ``` far out I only just realized this is linux lol, should I move this to windows? lol, sorry. bugger, maybe still relevant, I think. not sure. good luck

GiteaMirror commented

@abcbarryn commented on GitHub (Nov 15, 2025):

Previous selection was 12.6, I selected 12.9 and it found 12.9. I saw that that rick-github built Ollama with CUDA support on a SuSE system with CUDA 13.0. I don't think I can use 13.0 because it doesn't support my GPU. I have been trying to build Ollama with CUDA 12.9. Ollama 0.12.10 with CUDA 12.6 will build on my system, 12.9 does not.

@abcbarryn commented on GitHub (Nov 15, 2025): Previous selection was 12.6, I selected 12.9 and it found 12.9. I saw that that [rick-github](https://github.com/rick-github) built Ollama with CUDA support on a SuSE system with CUDA 13.0. I don't think I can use 13.0 because it doesn't support my GPU. I have been trying to build Ollama with CUDA 12.9. Ollama 0.12.10 with CUDA 12.6 will build on my system, 12.9 does not.

GiteaMirror commented

2026-04-29 08:26:47 -05:00

@rick-github commented on GitHub (Nov 15, 2025):

built Ollama with CUDA support on a SuSE system with CUDA 13.0.

12.8

I don't think I can use 13.0 because it doesn't support my GPU.

13.0 supports compute capability from 7.5 so supports Quadro 6000.

Ollama 0.12.10 with CUDA 12.6 will build on my system, 12.9 does not.

Here's a build with CUDA 12.9:

rick@ddr4-suse:~/ollama> mkdir ollama-13075 && cd $_
rick@ddr4-suse:~/ollama/ollama-13075> sudo zypper install -y cuda-toolkit-12-9
rick@ddr4-suse:~/ollama/ollama-13075> nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2025 NVIDIA Corporation
Built on Tue_May_27_02:21:03_PDT_2025
Cuda compilation tools, release 12.9, V12.9.86
Build cuda_12.9.r12.9/compiler.36037853_0
rick@ddr4-suse:~/ollama/ollama-13075> tar zxf ~/v0.12.10.tar.gz
rick@ddr4-suse:~/ollama/ollama-13075> cd ollama-0.12.10/
rick@ddr4-suse:~/ollama/ollama-13075/ollama-0.12.10> cmake -B build && cmake --build build && go run . serve
-- The C compiler identification is GNU 14.3.0
-- The CXX compiler identification is GNU 14.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- GGML_SYSTEM_ARCH: x86
-- Including CPU backend
-- x86 detected
-- Adding CPU backend variant ggml-cpu-x64:  
-- x86 detected
-- Adding CPU backend variant ggml-cpu-sse42: -msse4.2 GGML_SSE42
-- x86 detected
-- Adding CPU backend variant ggml-cpu-sandybridge: -msse4.2;-mavx GGML_SSE42;GGML_AVX
-- x86 detected
-- Adding CPU backend variant ggml-cpu-haswell: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2 GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2
-- x86 detected
-- Adding CPU backend variant ggml-cpu-skylakex: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavx512f;-mavx512cd;-mavx512vl;-mavx512dq;-mavx512bw GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX512
-- x86 detected
-- Adding CPU backend variant ggml-cpu-icelake: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavx512f;-mavx512cd;-mavx512vl;-mavx512dq;-mavx512bw;-mavx512vbmi;-mavx512vnni GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX512;GGML_AVX512_VBMI;GGML_AVX512_VNNI
-- x86 detected
-- Adding CPU backend variant ggml-cpu-alderlake: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavxvnni GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX_VNNI
-- Looking for a CUDA compiler
-- Looking for a CUDA compiler - /usr/local/cuda/bin/nvcc
-- Found CUDAToolkit: /usr/local/cuda/targets/x86_64-linux/include (found version "12.9.86") 
-- CUDA Toolkit found
-- Using CUDA architectures: native
-- The CUDA compiler identification is NVIDIA 12.9.86
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Looking for a HIP compiler
-- Looking for a HIP compiler - NOTFOUND
-- Could NOT find Vulkan (missing: Vulkan_LIBRARY Vulkan_INCLUDE_DIR) (found version "")
-- Configuring done (4.9s)
-- Generating done (0.0s)
-- Build files have been written to: /home/rick/ollama/ollama-13075/ollama-0.12.10/build
[  1%] Building C object ml/backend/ggml/ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o
...
[100%] Linking CUDA shared module ../../../../../../lib/ollama/libggml-cuda.so
[100%] Built target ggml-cuda
go: downloading github.com/spf13/cobra v1.7.0
...
go: downloading github.com/go-playground/locales v0.14.1
time=2025-11-15T18:48:06.378+01:00 level=INFO source=routes.go:1525 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:4096 OLLAMA_DEBUG:INFO OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/home/rick/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[ollama.com] OLLAMA_SCHED_SPREAD:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"

rick@ddr4-suse:~/ollama/ollama-13075/ollama-0.12.10> go run . list
NAME            ID              SIZE      MODIFIED    
qwen2.5:0.5b    a8b0c5157701    397 MB    4 hours ago    
rick@ddr4-suse:~/ollama/ollama-13075/ollama-0.12.10> go run . run qwen2.5:0.5b hello
Hello! How can I assist you today?

An alternative to loading different compilers and toolkits into your system is to do the build in a docker container. That way the inconsistencies of the host system don't influence the build process.

rick@ddr4-suse:~> sudo -s
ddr4-suse:/home/rick # zypper install -y docker git-core
ddr4-suse:/home/rick # systemctl enable --now docker
ddr4-suse:/home/rick # exit
rick@ddr4-suse:~> sudo usermod --append --groups docker $USER
rick@ddr4-suse:~> exec su -l $USER    # to activate the group, logging out and in again will do the same.
rick@ddr4-suse:~> git clone https://github.com/ollama/ollama.git
rick@ddr4-suse:~> cd ollama
rick@ddr4-suse:~/ollama> PLATFORM=linux/amd64 scripts/build_linux.sh
Building Ollama
...
Compressing linux tar bundles...
rick@ddr4-suse:~/ollama> ./dist/bin/ollama serve
time=2025-11-15T15:25:38.422+01:00 level=INFO source=routes.go:1544 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:4096 OLLAMA_DEBUG:INFO OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/home/rick/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[ollama.com] OLLAMA_SCHED_SPREAD:false OLLAMA_VULKAN:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"

rick@ddr4-suse:~/ollama> ./dist/bin/ollama run qwen2.5:0.5b hello
Hello! How can I assist you today? If there's anything specific you'd like to know or discuss, feel free to ask.

@rick-github commented on GitHub (Nov 15, 2025): > built Ollama with CUDA support on a SuSE system with CUDA 13.0. 12.8 > I don't think I can use 13.0 because it doesn't support my GPU. 13.0 supports compute capability from 7.5 so [supports](https://developer.nvidia.com/cuda-gpus) Quadro 6000. > Ollama 0.12.10 with CUDA 12.6 will build on my system, 12.9 does not. Here's a build with CUDA 12.9: ```console rick@ddr4-suse:~/ollama> mkdir ollama-13075 && cd $_ rick@ddr4-suse:~/ollama/ollama-13075> sudo zypper install -y cuda-toolkit-12-9 rick@ddr4-suse:~/ollama/ollama-13075> nvcc --version nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2025 NVIDIA Corporation Built on Tue_May_27_02:21:03_PDT_2025 Cuda compilation tools, release 12.9, V12.9.86 Build cuda_12.9.r12.9/compiler.36037853_0 rick@ddr4-suse:~/ollama/ollama-13075> tar zxf ~/v0.12.10.tar.gz rick@ddr4-suse:~/ollama/ollama-13075> cd ollama-0.12.10/ rick@ddr4-suse:~/ollama/ollama-13075/ollama-0.12.10> cmake -B build && cmake --build build && go run . serve -- The C compiler identification is GNU 14.3.0 -- The CXX compiler identification is GNU 14.3.0 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /usr/bin/cc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /usr/bin/c++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success -- Found Threads: TRUE -- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF -- CMAKE_SYSTEM_PROCESSOR: x86_64 -- GGML_SYSTEM_ARCH: x86 -- Including CPU backend -- x86 detected -- Adding CPU backend variant ggml-cpu-x64: -- x86 detected -- Adding CPU backend variant ggml-cpu-sse42: -msse4.2 GGML_SSE42 -- x86 detected -- Adding CPU backend variant ggml-cpu-sandybridge: -msse4.2;-mavx GGML_SSE42;GGML_AVX -- x86 detected -- Adding CPU backend variant ggml-cpu-haswell: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2 GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2 -- x86 detected -- Adding CPU backend variant ggml-cpu-skylakex: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavx512f;-mavx512cd;-mavx512vl;-mavx512dq;-mavx512bw GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX512 -- x86 detected -- Adding CPU backend variant ggml-cpu-icelake: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavx512f;-mavx512cd;-mavx512vl;-mavx512dq;-mavx512bw;-mavx512vbmi;-mavx512vnni GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX512;GGML_AVX512_VBMI;GGML_AVX512_VNNI -- x86 detected -- Adding CPU backend variant ggml-cpu-alderlake: -msse4.2;-mf16c;-mfma;-mbmi2;-mavx;-mavx2;-mavxvnni GGML_SSE42;GGML_F16C;GGML_FMA;GGML_BMI2;GGML_AVX;GGML_AVX2;GGML_AVX_VNNI -- Looking for a CUDA compiler -- Looking for a CUDA compiler - /usr/local/cuda/bin/nvcc -- Found CUDAToolkit: /usr/local/cuda/targets/x86_64-linux/include (found version "12.9.86") -- CUDA Toolkit found -- Using CUDA architectures: native -- The CUDA compiler identification is NVIDIA 12.9.86 -- Detecting CUDA compiler ABI info -- Detecting CUDA compiler ABI info - done -- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped -- Detecting CUDA compile features -- Detecting CUDA compile features - done -- Looking for a HIP compiler -- Looking for a HIP compiler - NOTFOUND -- Could NOT find Vulkan (missing: Vulkan_LIBRARY Vulkan_INCLUDE_DIR) (found version "") -- Configuring done (4.9s) -- Generating done (0.0s) -- Build files have been written to: /home/rick/ollama/ollama-13075/ollama-0.12.10/build [ 1%] Building C object ml/backend/ggml/ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o ... [100%] Linking CUDA shared module ../../../../../../lib/ollama/libggml-cuda.so [100%] Built target ggml-cuda go: downloading github.com/spf13/cobra v1.7.0 ... go: downloading github.com/go-playground/locales v0.14.1 time=2025-11-15T18:48:06.378+01:00 level=INFO source=routes.go:1525 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:4096 OLLAMA_DEBUG:INFO OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/home/rick/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[ollama.com] OLLAMA_SCHED_SPREAD:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]" ``` ```console rick@ddr4-suse:~/ollama/ollama-13075/ollama-0.12.10> go run . list NAME ID SIZE MODIFIED qwen2.5:0.5b a8b0c5157701 397 MB 4 hours ago rick@ddr4-suse:~/ollama/ollama-13075/ollama-0.12.10> go run . run qwen2.5:0.5b hello Hello! How can I assist you today? ``` An alternative to loading different compilers and toolkits into your system is to do the build in a docker container. That way the inconsistencies of the host system don't influence the build process. ```console rick@ddr4-suse:~> sudo -s ddr4-suse:/home/rick # zypper install -y docker git-core ddr4-suse:/home/rick # systemctl enable --now docker ddr4-suse:/home/rick # exit rick@ddr4-suse:~> sudo usermod --append --groups docker $USER rick@ddr4-suse:~> exec su -l $USER # to activate the group, logging out and in again will do the same. rick@ddr4-suse:~> git clone https://github.com/ollama/ollama.git rick@ddr4-suse:~> cd ollama rick@ddr4-suse:~/ollama> PLATFORM=linux/amd64 scripts/build_linux.sh Building Ollama ... Compressing linux tar bundles... rick@ddr4-suse:~/ollama> ./dist/bin/ollama serve time=2025-11-15T15:25:38.422+01:00 level=INFO source=routes.go:1544 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:4096 OLLAMA_DEBUG:INFO OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/home/rick/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[ollama.com] OLLAMA_SCHED_SPREAD:false OLLAMA_VULKAN:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]" ``` ```console rick@ddr4-suse:~/ollama> ./dist/bin/ollama run qwen2.5:0.5b hello Hello! How can I assist you today? If there's anything specific you'd like to know or discuss, feel free to ask. ``` ```

GiteaMirror commented

2026-04-29 08:26:47 -05:00

@abcbarryn commented on GitHub (Nov 16, 2025):

The Quadro P6000 (note no RTX) has a compute capability of 6.1, not 7.5.

# nvidia-smi --query-gpu=compute_cap
compute_cap
6.1

So, I see you built Ollama 0.12.10 with CUDA 12.9. So, what the heck is going on with my environment? What version of gcc are you running? Something must be different but what? What cmake version are you using? What could it be? Scratches head...

@abcbarryn commented on GitHub (Nov 16, 2025): The Quadro P6000 (note no RTX) has a compute capability of 6.1, not 7.5. ``` # nvidia-smi --query-gpu=compute_cap compute_cap 6.1 ``` So, I see you built Ollama 0.12.10 with CUDA 12.9. So, what the heck is going on with my environment? What version of gcc are you running? Something must be different but what? What cmake version are you using? What could it be? Scratches head...

GiteaMirror commented

@abcbarryn commented on GitHub (Nov 16, 2025):

Maybe it has to do with the way I installed the GPU driver module using the NVidia .run installer. Or maybe not.

@abcbarryn commented on GitHub (Nov 16, 2025): Maybe it has to do with the way I installed the GPU driver module using the NVidia .run installer. Or maybe not.

GiteaMirror commented

@abcbarryn commented on GitHub (Nov 16, 2025):

Or maybe it has to do with #12872, which you are also not seeing, am I right?

@abcbarryn commented on GitHub (Nov 16, 2025): Or maybe it has to do with #12872, which you are also not seeing, am I right?

GiteaMirror commented

@abcbarryn commented on GitHub (Nov 16, 2025):

Maybe both are related to issue #13050, which I worked around by adding export INCLUDES="-I /usr/local/cuda/include". You don't seem to have that issue either.

@abcbarryn commented on GitHub (Nov 16, 2025): Maybe both are related to issue #13050, which I worked around by adding export INCLUDES="-I /usr/local/cuda/include". You don't seem to have that issue either.

GiteaMirror commented