github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 08:02:14 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #796] Support ppc64le architecture #26142

New Issue
Open
opened 2026-04-22 02:11:09 -05:00 by GiteaMirror · 25 comments
GiteaMirror commented 2026-04-22 02:11:09 -05:00
Owner
Copy Link

Originally created by @orkutmuratyilmaz on GitHub (Oct 16, 2023).
Original GitHub issue: https://github.com/ollama/ollama/issues/796

Hello all,

Thanks for Ollama, it is a great thing to use:)

I've installed it on my local (Manjaro) and it works nice. After that, I'm trying to install on a server, which is running with IBM POWER8NVL cpu and Ubuntu 18.04 is my latest updated choice. It means, I cannot run the install script, because of the script's demand for AMD64 cpu architecture. Then I've decided to build it.

First, I've installed gcc, cmake, nvidia-cuda-toolkit packages with apt and then, I've installed go with "snap install go --classic".

After that, I've downloaded Ollama with "wget https://github.com/jmorganca/ollama/archive/refs/heads/main.zip" and unzipped it. Then, I did "go generate ./..." in the unzipped directory, but at the end, I've received an error message, which is below:

go generate ./...
go: downloading gonum.org/v1/gonum v0.13.0
go: downloading github.com/spf13/cobra v1.7.0
go: downloading github.com/olekukonko/tablewriter v0.0.5
go: downloading github.com/dustin/go-humanize v1.0.1
go: downloading github.com/pdevine/readline v1.5.2
go: downloading golang.org/x/term v0.10.0
go: downloading golang.org/x/sync v0.3.0
go: downloading github.com/gin-contrib/cors v1.4.0
go: downloading github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db
go: downloading github.com/mattn/go-runewidth v0.0.14
go: downloading github.com/gin-gonic/gin v1.9.1
go: downloading golang.org/x/crypto v0.10.0
go: downloading golang.org/x/exp v0.0.0-20230817173708-d852ddb80c63
go: downloading github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58
go: downloading github.com/rivo/uniseg v0.2.0
go: downloading github.com/spf13/pflag v1.0.5
go: downloading github.com/gin-contrib/sse v0.1.0
go: downloading gopkg.in/yaml.v3 v3.0.1
go: downloading github.com/ugorji/go/codec v1.2.11
go: downloading golang.org/x/net v0.10.0
go: downloading github.com/mattn/go-isatty v0.0.19
go: downloading github.com/pelletier/go-toml/v2 v2.0.8
go: downloading google.golang.org/protobuf v1.30.0
go: downloading github.com/go-playground/validator/v10 v10.14.0
go: downloading golang.org/x/sys v0.11.0
go: downloading github.com/leodido/go-urn v1.2.4
go: downloading github.com/gabriel-vasile/mimetype v1.4.2
go: downloading github.com/go-playground/universal-translator v0.18.1
go: downloading golang.org/x/text v0.10.0
go: downloading github.com/go-playground/locales v0.14.1
fatal: not a git repository (or any of the parent directories): .git
llm/llama.cpp/generate_linux.go:3: running "git": exit status 128

I've also done some searching, but I couldn't find a solution. Do you have any ideas?

Best,
Orkut

Originally created by @orkutmuratyilmaz on GitHub (Oct 16, 2023). Original GitHub issue: https://github.com/ollama/ollama/issues/796 Hello all, Thanks for Ollama, it is a great thing to use:) I've installed it on my local (Manjaro) and it works nice. After that, I'm trying to install on a server, which is running with IBM POWER8NVL cpu and Ubuntu 18.04 is my latest updated choice. It means, I cannot run the install script, because of the script's demand for AMD64 cpu architecture. Then I've decided to build it. First, I've installed gcc, cmake, nvidia-cuda-toolkit packages with apt and then, I've installed go with "`snap install go --classic`". After that, I've downloaded Ollama with "`wget https://github.com/jmorganca/ollama/archive/refs/heads/main.zip`" and unzipped it. Then, I did "`go generate ./...`" in the unzipped directory, but at the end, I've received an error message, which is below: ``` go generate ./... go: downloading gonum.org/v1/gonum v0.13.0 go: downloading github.com/spf13/cobra v1.7.0 go: downloading github.com/olekukonko/tablewriter v0.0.5 go: downloading github.com/dustin/go-humanize v1.0.1 go: downloading github.com/pdevine/readline v1.5.2 go: downloading golang.org/x/term v0.10.0 go: downloading golang.org/x/sync v0.3.0 go: downloading github.com/gin-contrib/cors v1.4.0 go: downloading github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db go: downloading github.com/mattn/go-runewidth v0.0.14 go: downloading github.com/gin-gonic/gin v1.9.1 go: downloading golang.org/x/crypto v0.10.0 go: downloading golang.org/x/exp v0.0.0-20230817173708-d852ddb80c63 go: downloading github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58 go: downloading github.com/rivo/uniseg v0.2.0 go: downloading github.com/spf13/pflag v1.0.5 go: downloading github.com/gin-contrib/sse v0.1.0 go: downloading gopkg.in/yaml.v3 v3.0.1 go: downloading github.com/ugorji/go/codec v1.2.11 go: downloading golang.org/x/net v0.10.0 go: downloading github.com/mattn/go-isatty v0.0.19 go: downloading github.com/pelletier/go-toml/v2 v2.0.8 go: downloading google.golang.org/protobuf v1.30.0 go: downloading github.com/go-playground/validator/v10 v10.14.0 go: downloading golang.org/x/sys v0.11.0 go: downloading github.com/leodido/go-urn v1.2.4 go: downloading github.com/gabriel-vasile/mimetype v1.4.2 go: downloading github.com/go-playground/universal-translator v0.18.1 go: downloading golang.org/x/text v0.10.0 go: downloading github.com/go-playground/locales v0.14.1 fatal: not a git repository (or any of the parent directories): .git llm/llama.cpp/generate_linux.go:3: running "git": exit status 128 ``` I've also done some searching, but I couldn't find a solution. Do you have any ideas? Best, Orkut
GiteaMirror added the feature request label 2026-04-22 02:11:09 -05:00
GiteaMirror commented 2026-04-22 02:11:11 -05:00
Author
Owner
Copy Link

@orkutmuratyilmaz commented on GitHub (Oct 16, 2023):

I've added my SSH key and tried to pull repository after "git remote add" with a success. After that, I did "go generate ./..." again. Unfortunately, I'm receiving a new error message, which is below:

ollama$ go generate ./...
Submodule 'llm/llama.cpp/ggml' (https://github.com/ggerganov/llama.cpp.git) registered for path 'ggml'
Submodule 'llm/llama.cpp/gguf' (https://github.com/ggerganov/llama.cpp.git) registered for path 'gguf'
Cloning into '/home/username/ollama/llm/llama.cpp/ggml'...
remote: Enumerating objects: 4961, done.
remote: Counting objects: 100% (4961/4961), done.
remote: Compressing objects: 100% (1493/1493), done.
remote: Total 4815 (delta 3444), reused 4641 (delta 3291), pack-reused 0
Receiving objects: 100% (4815/4815), 3.26 MiB | 10.01 MiB/s, done.
Resolving deltas: 100% (3444/3444), completed with 102 local objects.
From https://github.com/ggerganov/llama.cpp
 * branch            9e232f0234073358e7031c1b8d7aa45020469a3b -> FETCH_HEAD
Submodule path 'ggml': checked out '9e232f0234073358e7031c1b8d7aa45020469a3b'
CMake Error: The source directory "/home/username/ollama/llm/llama.cpp/ggml/build/cpu" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
llm/llama.cpp/generate_linux.go:10: running "cmake": exit status 1

Is it again my CPU?

<!-- gh-comment-id:1764082155 --> @orkutmuratyilmaz commented on GitHub (Oct 16, 2023): I've added my SSH key and tried to pull repository after "`git remote add`" with a success. After that, I did "`go generate ./...`" again. Unfortunately, I'm receiving a new error message, which is below: ``` ollama$ go generate ./... Submodule 'llm/llama.cpp/ggml' (https://github.com/ggerganov/llama.cpp.git) registered for path 'ggml' Submodule 'llm/llama.cpp/gguf' (https://github.com/ggerganov/llama.cpp.git) registered for path 'gguf' Cloning into '/home/username/ollama/llm/llama.cpp/ggml'... remote: Enumerating objects: 4961, done. remote: Counting objects: 100% (4961/4961), done. remote: Compressing objects: 100% (1493/1493), done. remote: Total 4815 (delta 3444), reused 4641 (delta 3291), pack-reused 0 Receiving objects: 100% (4815/4815), 3.26 MiB | 10.01 MiB/s, done. Resolving deltas: 100% (3444/3444), completed with 102 local objects. From https://github.com/ggerganov/llama.cpp * branch 9e232f0234073358e7031c1b8d7aa45020469a3b -> FETCH_HEAD Submodule path 'ggml': checked out '9e232f0234073358e7031c1b8d7aa45020469a3b' CMake Error: The source directory "/home/username/ollama/llm/llama.cpp/ggml/build/cpu" does not exist. Specify --help for usage, or press the help button on the CMake GUI. llm/llama.cpp/generate_linux.go:10: running "cmake": exit status 1 ``` Is it again my CPU?
GiteaMirror commented 2026-04-22 02:11:11 -05:00
Author
Owner
Copy Link

@BruceMacD commented on GitHub (Oct 17, 2023):

Hi @orkutmuratyilmaz thanks for opening the issue. Right now Ollama only supports arm64 and aarch CPUs, I don't believe IBM Power8 CPU will be compatible with the library we use to run the language models.

<!-- gh-comment-id:1766937793 --> @BruceMacD commented on GitHub (Oct 17, 2023): Hi @orkutmuratyilmaz thanks for opening the issue. Right now Ollama only supports arm64 and aarch CPUs, I don't believe IBM Power8 CPU will be compatible with the library we use to run the language models.
GiteaMirror commented 2026-04-22 02:11:13 -05:00
Author
Owner
Copy Link

@orkutmuratyilmaz commented on GitHub (Oct 18, 2023):

Hello @BruceMacD, thanks for your kind reply. I'm still looking for a solution. Do I have a chance for compiling /building from the source, in order to make it work for my CPU? If so, where should I start reading? :)

<!-- gh-comment-id:1768873747 --> @orkutmuratyilmaz commented on GitHub (Oct 18, 2023): Hello @BruceMacD, thanks for your kind reply. I'm still looking for a solution. Do I have a chance for compiling /building from the source, in order to make it work for my CPU? If so, where should I start reading? :)
GiteaMirror commented 2026-04-22 02:11:13 -05:00
Author
Owner
Copy Link

@orkutmuratyilmaz commented on GitHub (Oct 26, 2023):

@jmorganca thanks for setting a better title for this issue:)

<!-- gh-comment-id:1780817685 --> @orkutmuratyilmaz commented on GitHub (Oct 26, 2023): @jmorganca thanks for setting a better title for this issue:)
GiteaMirror commented 2026-04-22 02:11:14 -05:00
Author
Owner
Copy Link

@cosmic-heart commented on GitHub (Mar 20, 2024):

Power 9 pc's are supported ? It will be good if build from source instructions are added to the repo until the ppc64le binaries are compiled. Atleast anyone can drop it here

<!-- gh-comment-id:2008506659 --> @cosmic-heart commented on GitHub (Mar 20, 2024): Power 9 pc's are supported ? It will be good if build from source instructions are added to the repo until the ppc64le binaries are compiled. Atleast anyone can drop it here
GiteaMirror commented 2026-04-22 02:11:15 -05:00
Author
Owner
Copy Link

@stormljor commented on GitHub (Apr 18, 2024):

I have a patched version that works on ppc64le; the few changes I had to do look like:
Essential patch:

diff --git a/llm/llm.go b/llm/llm.go
index 33949c7..17e9d1c 100644
--- a/llm/llm.go
+++ b/llm/llm.go
@@ -6,6 +6,7 @@ package llm
 // #cgo windows,amd64 LDFLAGS: ${SRCDIR}/build/windows/amd64_static/libllama.a -static -lstdc++
 // #cgo linux,amd64 LDFLAGS: ${SRCDIR}/build/linux/x86_64_static/libllama.a -lstdc++
 // #cgo linux,arm64 LDFLAGS: ${SRCDIR}/build/linux/arm64_static/libllama.a -lstdc++
+// #cgo linux,ppc64le LDFLAGS: ${SRCDIR}/build/linux/ppc64le_static/libllama.a -lstdc++
 // #include <stdlib.h>
 // #include "llama.h"
 import "C"

Only needed for ollama run:

diff --git a/readline/term_linux.go b/readline/term_linux.go
index 2d6211d..69e05bf 100644
--- a/readline/term_linux.go
+++ b/readline/term_linux.go
@@ -5,10 +5,11 @@ package readline
 import (
        "syscall"
        "unsafe"
+    "golang.org/x/sys/unix"
 )

-const tcgets = 0x5401
-const tcsets = 0x5402
+const tcgets = unix.TCGETS
+const tcsets = unix.TCSETSF

 func getTermios(fd int) (*Termios, error) {
        termios := new(Termios)

the first patch lets it find the llama static build, and the second part lets you run ollama run without getting Error: inappropriate ioctl for device.

I'm building in a conda environment so I can get a newer version of clang / cmake / gcc / g++ than what's in my base RHEL installation.
CC=clang CXX=clang++ NVCC_PREPEND_FLAGS=-allow-unsupported-compiler go generate ./...
(I'm using CUDA 11.4, and nvcc complained about "too new" compilers, but they were required to actually have golang 1.22)

(I'd do a PR with this, but I haven't had time to test it thoroughly enough)

<!-- gh-comment-id:2063697601 --> @stormljor commented on GitHub (Apr 18, 2024): I have a patched version that works on ppc64le; the few changes I had to do look like: Essential patch: ``` diff --git a/llm/llm.go b/llm/llm.go index 33949c7..17e9d1c 100644 --- a/llm/llm.go +++ b/llm/llm.go @@ -6,6 +6,7 @@ package llm // #cgo windows,amd64 LDFLAGS: ${SRCDIR}/build/windows/amd64_static/libllama.a -static -lstdc++ // #cgo linux,amd64 LDFLAGS: ${SRCDIR}/build/linux/x86_64_static/libllama.a -lstdc++ // #cgo linux,arm64 LDFLAGS: ${SRCDIR}/build/linux/arm64_static/libllama.a -lstdc++ +// #cgo linux,ppc64le LDFLAGS: ${SRCDIR}/build/linux/ppc64le_static/libllama.a -lstdc++ // #include <stdlib.h> // #include "llama.h" import "C" ``` Only needed for `ollama run`: ``` diff --git a/readline/term_linux.go b/readline/term_linux.go index 2d6211d..69e05bf 100644 --- a/readline/term_linux.go +++ b/readline/term_linux.go @@ -5,10 +5,11 @@ package readline import ( "syscall" "unsafe" + "golang.org/x/sys/unix" ) -const tcgets = 0x5401 -const tcsets = 0x5402 +const tcgets = unix.TCGETS +const tcsets = unix.TCSETSF func getTermios(fd int) (*Termios, error) { termios := new(Termios) ``` the first patch lets it find the llama static build, and the second part lets you run `ollama run` without getting `Error: inappropriate ioctl for device`. I'm building in a conda environment so I can get a newer version of clang / cmake / gcc / g++ than what's in my base RHEL installation. `CC=clang CXX=clang++ NVCC_PREPEND_FLAGS=-allow-unsupported-compiler go generate ./...` (I'm using CUDA 11.4, and nvcc complained about "too new" compilers, but they were required to actually have golang 1.22) (I'd do a PR with this, but I haven't had time to test it thoroughly enough)
GiteaMirror commented 2026-04-22 02:11:15 -05:00
Author
Owner
Copy Link

@ALutz273 commented on GitHub (Jun 30, 2024):

I am starting to test and write with but not a good developer only admin.
I am still trying to get a server with GPU and can then test if needed

<!-- gh-comment-id:2198566931 --> @ALutz273 commented on GitHub (Jun 30, 2024): I am starting to test and write with but not a good developer only admin. I am still trying to get a server with GPU and can then test if needed
GiteaMirror commented 2026-04-22 02:11:16 -05:00
Author
Owner
Copy Link

@kavita-rane2 commented on GitHub (Nov 18, 2024):

@stormljor @ALutz273 Are you actively working on this issue?

I have tested above fix with upstream code on GPU vm on PPC64LE architecture. and is working as expected.

@dhiltgen Along with above fix I have included workflow changes to enable CI for PPC64LE architecture. I have this changes in my fork branch.
CI changes will need enabling ppc64le self hoisted runner on main repository as mentioned in issue 7649

Can someone assist in configuring self hoisted runner to test CI workflows on the main branch and procced with PR

<!-- gh-comment-id:2481928322 --> @kavita-rane2 commented on GitHub (Nov 18, 2024): @stormljor @ALutz273 Are you actively working on this issue? I have tested above fix with upstream code on GPU vm on PPC64LE architecture. and is working as expected. @dhiltgen Along with above fix I have included workflow changes to enable CI for PPC64LE architecture. I have this changes in my fork branch. CI changes will need enabling ppc64le self hoisted runner on main repository as mentioned in issue [7649](https://github.com.mcas.ms/ollama/ollama/issues/7649) Can someone assist in configuring self hoisted runner to test CI workflows on the main branch and procced with PR
GiteaMirror commented 2026-04-22 02:11:16 -05:00
Author
Owner
Copy Link

@ALutz273 commented on GitHub (Nov 20, 2024):

hello @kavita-rane2
I was able to test it successfully with the change from @stormljor to Power9 and Power10. (ollama latest version (0.4.2)
You now have to introduce the change into the existing code, right?

<!-- gh-comment-id:2489110997 --> @ALutz273 commented on GitHub (Nov 20, 2024): hello @kavita-rane2 I was able to test it successfully with the change from @stormljor to Power9 and Power10. (ollama latest version (0.4.2) You now have to introduce the change into the existing code, right?
GiteaMirror commented 2026-04-22 02:11:17 -05:00
Author
Owner
Copy Link

@stormljor commented on GitHub (Nov 20, 2024):

I can turn my comment into a proper pull request; it's been working flawlessly so far at our end.

<!-- gh-comment-id:2489214235 --> @stormljor commented on GitHub (Nov 20, 2024): I can turn my comment into a proper pull request; it's been working flawlessly so far at our end.
GiteaMirror commented 2026-04-22 02:11:17 -05:00
Author
Owner
Copy Link

@kavita-rane2 commented on GitHub (Nov 21, 2024):

stormljor you can procced with your PR.

I have additional changes to enable CI on PPC64LE. CI changes will need enabling ppc64le self hoisted runner on ollama

dhiltgen Can we have CI changes deliver under the same issue ?

<!-- gh-comment-id:2490739167 --> @kavita-rane2 commented on GitHub (Nov 21, 2024): [stormljor](https://github.com/stormljor) you can procced with your PR. I have additional changes to enable CI on PPC64LE. CI changes will need enabling ppc64le self hoisted runner on ollama [dhiltgen](https://github.com/dhiltgen) Can we have CI changes deliver under the same issue ?
GiteaMirror commented 2026-04-22 02:11:18 -05:00
Author
Owner
Copy Link

@ALutz273 commented on GitHub (Jan 17, 2025):

Any news about this topic?

<!-- gh-comment-id:2598576815 --> @ALutz273 commented on GitHub (Jan 17, 2025): Any news about this topic?
GiteaMirror commented 2026-04-22 02:11:18 -05:00
Author
Owner
Copy Link

@Ry6000 commented on GitHub (Jan 28, 2025):

I would also be interested in ppc64le support! I have a Talos II system that I would love to run ollama on. 😄

<!-- gh-comment-id:2617231212 --> @Ry6000 commented on GitHub (Jan 28, 2025): I would also be interested in ppc64le support! I have a Talos II system that I would love to run ollama on. 😄
GiteaMirror commented 2026-04-22 02:11:19 -05:00
Author
Owner
Copy Link

@bmco321 commented on GitHub (Feb 1, 2025):

using IBM 8335-gtb with 4x p100
i would be very interested in the ppc64le support too!
i have not tested the above fixes i just got my hands on a server.

<!-- gh-comment-id:2629063438 --> @bmco321 commented on GitHub (Feb 1, 2025): using IBM 8335-gtb with 4x p100 i would be very interested in the ppc64le support too! i have not tested the above fixes i just got my hands on a server.
GiteaMirror commented 2026-04-22 02:11:19 -05:00
Author
Owner
Copy Link

@narendhar007 commented on GitHub (Feb 2, 2025):

I am also interested to try this out on Power 10. We can afford large amounts of memory and P10 procs on a E1080 system and keen to try this.

<!-- gh-comment-id:2629431959 --> @narendhar007 commented on GitHub (Feb 2, 2025): I am also interested to try this out on Power 10. We can afford large amounts of memory and P10 procs on a E1080 system and keen to try this.
GiteaMirror commented 2026-04-22 02:11:19 -05:00
Author
Owner
Copy Link

@mgiessing commented on GitHub (Feb 3, 2025):

Hi,

since it is currently a bit tricky to get started on Power systems here are two container images to have a quick start:

ollama built for Power8/9 with GPU support and CUDA ARCHITECTURES 60;70;75 (P100/V100/T4)

  • Requires RHEL8 host, CUDA 12.4.1, podman and the nvidia container toolkit
podman run -d --privileged \
  --device nvidia.com/gpu=all \
  -v ollama:/root/.ollama \
  -p 11434:11434 \
  --name ollama \
  quay.io/mgiessing/ollama:v0.5.7-cuda

ollama built with CPU support

  • Requires Power9 or Power10 system

currently only supports VSX/vector units; MMA is theoretically supported but this build is not using OpenBLAS (which has the Power10 MMA support for prompt processing)

podman run -d --privileged \
  -v ollama:/root/.ollama \
  -p 11434:11434 \
  --name ollama \
  quay.io/mgiessing/ollama:v0.5.7

These "old" systems still do a pretty good job on running LLMs due to their memory bandwidth :)

<!-- gh-comment-id:2631325618 --> @mgiessing commented on GitHub (Feb 3, 2025): Hi, since it is currently a bit tricky to get started on Power systems here are two container images to have a quick start: ### ollama built for Power8/9 with GPU support and CUDA ARCHITECTURES 60;70;75 (P100/V100/T4) - Requires RHEL8 host, CUDA 12.4.1, podman and the nvidia container toolkit ```bash podman run -d --privileged \ --device nvidia.com/gpu=all \ -v ollama:/root/.ollama \ -p 11434:11434 \ --name ollama \ quay.io/mgiessing/ollama:v0.5.7-cuda ``` ### ollama built with CPU support - Requires Power9 or Power10 system currently only supports VSX/vector units; MMA is theoretically supported but this build is not using OpenBLAS (which has the Power10 MMA support for prompt processing) ```bash podman run -d --privileged \ -v ollama:/root/.ollama \ -p 11434:11434 \ --name ollama \ quay.io/mgiessing/ollama:v0.5.7 ``` These "old" systems still do a pretty good job on running LLMs due to their memory bandwidth :)
GiteaMirror commented 2026-04-22 02:11:20 -05:00
Author
Owner
Copy Link

@ALutz273 commented on GitHub (Feb 6, 2025):

@mgiessing very good work. works really well. 👍

Now it “only” needs to be pushed upstream 🙈

<!-- gh-comment-id:2639159671 --> @ALutz273 commented on GitHub (Feb 6, 2025): @mgiessing very good work. works really well. 👍 Now it “only” needs to be pushed upstream 🙈
GiteaMirror commented 2026-04-22 02:11:20 -05:00
Author
Owner
Copy Link

@OverCanvassed commented on GitHub (Mar 5, 2025):

One more vote for mainlining support for ppc64le. These systems have much life yet to live.

Thank you all for the work you've done so far. Your efforts are greatly appreciated.

<!-- gh-comment-id:2701622328 --> @OverCanvassed commented on GitHub (Mar 5, 2025): One more vote for mainlining support for ppc64le. These systems have much life yet to live. Thank you all for the work you've done so far. Your efforts are greatly appreciated.
GiteaMirror commented 2026-04-22 02:11:20 -05:00
Author
Owner
Copy Link

@naveedus commented on GitHub (Mar 17, 2025):

I have an IBM AC922 P9 system equipped with four NVIDIA Tesla V100 SXM2 16GB GPUs. The system runs Red Hat Enterprise Linux (RHEL) 8.10 with CUDA Toolkit 11.8 Update 1 (version 11.8.0-1 ppc64le) installed.

The configuration successfully compiles and runs Ollama at #commit 53b460a8cf Results are impressive, utilizing the combined 64GB VRAM from four GPUs. The setup can handle inference workloads for up to 70 billion (70B) quantized models effortlessly.

<!-- gh-comment-id:2728085545 --> @naveedus commented on GitHub (Mar 17, 2025): I have an IBM AC922 P9 system equipped with four NVIDIA Tesla V100 SXM2 16GB GPUs. The system runs Red Hat Enterprise Linux (RHEL) 8.10 with CUDA Toolkit 11.8 Update 1 (version 11.8.0-1 ppc64le) installed. The configuration successfully compiles and runs Ollama at #commit 53b460a8cf22a67a7f77ddd1161de8117f58199b Results are impressive, utilizing the combined 64GB VRAM from four GPUs. The setup can handle inference workloads for up to 70 billion (70B) quantized models effortlessly.
GiteaMirror commented 2026-04-22 02:11:21 -05:00
Author
Owner
Copy Link

@iwaitlyd commented on GitHub (Sep 26, 2025):

I have an IBM AC922 P9 system equipped with four NVIDIA Tesla V100 SXM2 16GB GPUs. The system runs Red Hat Enterprise Linux (RHEL) 8.10 with CUDA Toolkit 11.8 Update 1 (version 11.8.0-1 ppc64le) installed.

The configuration successfully compiles and runs Ollama at #commit 53b460a Results are impressive, utilizing the combined 64GB VRAM from four GPUs. The setup can handle inference workloads for up to 70 billion (70B) quantized models effortlessly.

Hi, do you mind sharing which version of ollama you've tried? The there is no llm/llm.go file in latest version. Thank you!

<!-- gh-comment-id:3336680977 --> @iwaitlyd commented on GitHub (Sep 26, 2025): > I have an IBM AC922 P9 system equipped with four NVIDIA Tesla V100 SXM2 16GB GPUs. The system runs Red Hat Enterprise Linux (RHEL) 8.10 with CUDA Toolkit 11.8 Update 1 (version 11.8.0-1 ppc64le) installed. > > The configuration successfully compiles and runs Ollama at #commit [53b460a](https://github.com/ollama/ollama/commit/53b460a8cf22a67a7f77ddd1161de8117f58199b) Results are impressive, utilizing the combined 64GB VRAM from four GPUs. The setup can handle inference workloads for up to 70 billion (70B) quantized models effortlessly. Hi, do you mind sharing which version of ollama you've tried? The there is no llm/llm.go file in latest version. Thank you!
GiteaMirror commented 2026-04-22 02:11:21 -05:00
Author
Owner
Copy Link

@yaowenqiang commented on GitHub (Jan 27, 2026):

@mgiessing can you provide your Dockerfile?the docker image is a little old,many model require a higher version now.

<!-- gh-comment-id:3805040351 --> @yaowenqiang commented on GitHub (Jan 27, 2026): @mgiessing can you provide your Dockerfile?the docker image is a little old,many model require a higher version now.
GiteaMirror commented 2026-04-22 02:11:22 -05:00
Author
Owner
Copy Link

@mgiessing commented on GitHub (Jan 28, 2026):

@yaowenqiang I'm essentially using the official docker image with a lot of changes to make it compile & run on Power.

Btw. I have a newer image version (v0.11.4-cuda) that should be able to run many newer models - is that sufficient?
--> https://quay.io/repository/mgiessing/ollama?tab=tags&tag=latest

<!-- gh-comment-id:3810444376 --> @mgiessing commented on GitHub (Jan 28, 2026): @yaowenqiang I'm essentially using the official docker image with a lot of changes to make it compile & run on Power. Btw. I have a newer image version (v0.11.4-cuda) that should be able to run many newer models - is that sufficient? --> https://quay.io/repository/mgiessing/ollama?tab=tags&tag=latest
GiteaMirror commented 2026-04-22 02:11:22 -05:00
Author
Owner
Copy Link

@yaowenqiang commented on GitHub (Jan 28, 2026):

@mgiessing　I am just using that version,but still a little old, for example,i want' to run glm-4.7-flash:latest model,but got a error: The model you are attempting to pull requires a newer version of Ollama.

<!-- gh-comment-id:3811098567 --> @yaowenqiang commented on GitHub (Jan 28, 2026): @mgiessing　I am just using that version,but still a little old, for example,i want' to run glm-4.7-flash:latest model,but got a error: The model you are attempting to pull requires a newer version of Ollama.
GiteaMirror commented 2026-04-22 02:11:23 -05:00
Author
Owner
Copy Link

@mgiessing commented on GitHub (Jan 30, 2026):

@yaowenqiang sorry I'm currently kinda busy :-/

I'm usually editing the Dockerfile with a lot of sed commands, but this constantly breaks and requires trial and error :/

then build the image using:

PLATFORM=linux/ppc64le \
TARGETARCH=ppc64le \
VERSION=v0.5.12 \
FINAL_IMAGE_REPO=quay.io/mgiessing/ollama scripts/build_docker.sh
<!-- gh-comment-id:3823102393 --> @mgiessing commented on GitHub (Jan 30, 2026): @yaowenqiang sorry I'm currently kinda busy :-/ I'm usually editing the Dockerfile with a lot of `sed` commands, but this constantly breaks and requires trial and error :/ then build the image using: ```bash PLATFORM=linux/ppc64le \ TARGETARCH=ppc64le \ VERSION=v0.5.12 \ FINAL_IMAGE_REPO=quay.io/mgiessing/ollama scripts/build_docker.sh ```
GiteaMirror commented 2026-04-22 02:11:23 -05:00
Author
Owner
Copy Link

@naveedus commented on GitHub (Feb 12, 2026):

This latest readme provides instructions for building the latest Ollama version on an AC922 with CUDA.
https://github.com/naveedus/ollama-ppc64le/blob/AC922/README_AC922.md

<!-- gh-comment-id:3891025288 --> @naveedus commented on GitHub (Feb 12, 2026): This latest readme provides instructions for building the latest Ollama version on an AC922 with CUDA. https://github.com/naveedus/ollama-ppc64le/blob/AC922/README_AC922.md
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#26142
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?