github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-18 06:06:18 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[PR #5872] [Ascend ] add ascend npu support #37823

New Issue
Open
opened 2026-04-22 22:29:06 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-04-22 22:29:06 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/ollama/ollama/pull/5872
Author: @zhongTao99
Created: 7/23/2024
Status: 🔄 Open

Base: main ← Head: main

📝 Commits (10+)

  • f746fa2 Add CANN OP Interface
  • 828a7cc Add pool, matmul, copy Op implements
  • 8fad9f7 Add flash_attn, quant_mat_mal,rms_norml,... OP Implemention
  • e8f8345 Add tensor allocate implemention
  • c57de51 Add ascend discover implemention
  • 0f97aa4 Add Doxyfile for ggml-cann
  • 96cb01d Add CANN device init code
  • 8decb4a add memory pool and op dispatch implemention
  • 71b3aa4 Add ggmm-cann buffer manage and event implemention
  • 42b6985 Add op that cann supportted check support

📊 Changes

21 files changed (+11139 additions, -4 deletions)

View changed files

📝 CMakeLists.txt (+9 -0)
📝 CMakePresets.json (+33 -0)
📝 Dockerfile (+70 -0)
➕ discover/ascend_linux.go (+101 -0)
📝 discover/gpu.go (+49 -1)
📝 discover/gpu_info.h (+1 -0)
➕ discover/gpu_info_ascend.c (+178 -0)
➕ discover/gpu_info_ascend.h (+78 -0)
📝 llm/server.go (+2 -0)
📝 ml/backend/ggml/ggml/.rsync-filter (+1 -0)
➕ ml/backend/ggml/ggml/src/ggml-cann/CMakeLists.txt (+75 -0)
➕ ml/backend/ggml/ggml/src/ggml-cann/Doxyfile (+2579 -0)
➕ ml/backend/ggml/ggml/src/ggml-cann/acl_tensor.cpp (+181 -0)
➕ ml/backend/ggml/ggml/src/ggml-cann/acl_tensor.h (+258 -0)
➕ ml/backend/ggml/ggml/src/ggml-cann/aclnn_ops.cpp (+3193 -0)
➕ ml/backend/ggml/ggml/src/ggml-cann/aclnn_ops.h (+1125 -0)
➕ ml/backend/ggml/ggml/src/ggml-cann/common.h (+420 -0)
➕ ml/backend/ggml/ggml/src/ggml-cann/ggml-cann.cpp (+2607 -0)
📝 scripts/build_docker.sh (+23 -0)
📝 scripts/build_linux.sh (+24 -0)

...and 1 more files

📄 Description

It's a draft for ascend npu support, It can get gpu info for npu, and need to be optimization

fix:
https://github.com/ollama/ollama/issues/5315

The pre-builded ollama that support Huawei Atlas 800 A2 series and Atlas 300I Duo as the backend can be obtained from the following:
Pre-builded ollama ENV:
- Arch: linux/arm64
- CANN: 8.1.RC1,
- ollama and llama.cpp base code: 2025.5.26

Docker Image:
Atlas 800 A2:
docker pull leopony/ollama-cann-atlas-a2:latest
docker running command example:

docker run \
    --name ollama \
    --device /dev/davinci0 \
    --device /dev/davinci_manager \
    --device /dev/devmm_svm \
    --device /dev/hisi_hdc \
    -v /usr/local/dcmi:/usr/local/dcmi \
    -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
    -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
    -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
    -v /etc/ascend_install.info:/etc/ascend_install.info \
    -p 11434:11434  \
    -it leopony/ollama-cann-atlas-a2:latest /bin/bash

Atlas 300I Duo:
docker pull leopony/ollama-cann-300i-duo
docker running command example:

docker run \
    --name ollama \
    --device /dev/davinci0 \
    --device /dev/davinci_manager \
    --device /dev/devmm_svm \
    --device /dev/hisi_hdc \
    -v /usr/local/dcmi:/usr/local/dcmi \
    -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
    -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
    -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
    -v /etc/ascend_install.info:/etc/ascend_install.info \
    -p 11434:11434  \
    -it leopony/ollama-cann-300i-duo /bin/bash

Bianary tar:
linux-aarch64 Atlas 800 A2:
https://github.com/leo-pony/ollama/blob/ollama_bin/ollama-linux-arm64-cann-atlas-a2.tgz
linux-aarch64 Atlas 300I Duo:
https://github.com/leo-pony/ollama/blob/ollama_bin/ollama-linux-arm64-cann-300i-duo.tgz

Manully build guide:
The build step is same with ollama, details as following:
1) Binary build:
Cd the ollama project directory:
For Atlas 800 A2:

cmake --preset 'CANN Atlas 800 A2' 
cmake --build --parallel --preset 'CANN Atlas 800 A2' 
cmake --install build --component CANN

export ARG GOFLAGS="'-ldflags=-w -s'"
export ENV CGO_ENABLED=1
go build -trimpath -buildmode=pie -o /bin/ollama .

For Atlas 300I Duo：

cmake --preset 'CANN Atlas 300I Duo' 
cmake --build --parallel --preset 'CANN Atlas 300I Duo' 
cmake --install build --component CANN

export ARG GOFLAGS="'-ldflags=-w -s'"
export ENV CGO_ENABLED=1
go build -trimpath -buildmode=pie -o /bin/ollama .

2) Docker build:
If need net proxy, config docker proxy with guide: https://docs.docker.com/engine/daemon/proxy/
Cd the ollama project directory:
sh -x ./scripts/build_docker.sh

3) Ollama CANN release packages build:
If need net proxy, config docker proxy with guide: https://docs.docker.com/engine/daemon/proxy/
Cd the ollama project directory:
sh -x ./scripts/build_linux.sh

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/ollama/ollama/pull/5872 **Author:** [@zhongTao99](https://github.com/zhongTao99) **Created:** 7/23/2024 **Status:** 🔄 Open **Base:** `main` ← **Head:** `main` --- ### 📝 Commits (10+) - [`f746fa2`](https://github.com/ollama/ollama/commit/f746fa20cb8fe670175f558aab3fd135c0e90f47) Add CANN OP Interface - [`828a7cc`](https://github.com/ollama/ollama/commit/828a7cc45336aea1e43c9d1e41f80fa4829e4935) Add pool, matmul, copy Op implements - [`8fad9f7`](https://github.com/ollama/ollama/commit/8fad9f7a0f74f841a03a3661d69dc8ed7ae4e8d2) Add flash_attn, quant_mat_mal,rms_norml,... OP Implemention - [`e8f8345`](https://github.com/ollama/ollama/commit/e8f83455824716970d97c32a33b2fcc84f47c976) Add tensor allocate implemention - [`c57de51`](https://github.com/ollama/ollama/commit/c57de51c121e84bfa76c3cc36512998e531375c6) Add ascend discover implemention - [`0f97aa4`](https://github.com/ollama/ollama/commit/0f97aa48466bf8ddb6d2b56a4c3167699fdb0954) Add Doxyfile for ggml-cann - [`96cb01d`](https://github.com/ollama/ollama/commit/96cb01d8ddabbe8e31a850a04be61669b26fecfe) Add CANN device init code - [`8decb4a`](https://github.com/ollama/ollama/commit/8decb4af6217f8c8055bf1cdf8920a007b92a589) add memory pool and op dispatch implemention - [`71b3aa4`](https://github.com/ollama/ollama/commit/71b3aa4b7691c7deb96f5146e9b722cf2cc5550f) Add ggmm-cann buffer manage and event implemention - [`42b6985`](https://github.com/ollama/ollama/commit/42b69853634fcccc58752e782dfe22de160fa9b0) Add op that cann supportted check support ### 📊 Changes **21 files changed** (+11139 additions, -4 deletions) <details> <summary>View changed files</summary> 📝 `CMakeLists.txt` (+9 -0) 📝 `CMakePresets.json` (+33 -0) 📝 `Dockerfile` (+70 -0) ➕ `discover/ascend_linux.go` (+101 -0) 📝 `discover/gpu.go` (+49 -1) 📝 `discover/gpu_info.h` (+1 -0) ➕ `discover/gpu_info_ascend.c` (+178 -0) ➕ `discover/gpu_info_ascend.h` (+78 -0) 📝 `llm/server.go` (+2 -0) 📝 `ml/backend/ggml/ggml/.rsync-filter` (+1 -0) ➕ `ml/backend/ggml/ggml/src/ggml-cann/CMakeLists.txt` (+75 -0) ➕ `ml/backend/ggml/ggml/src/ggml-cann/Doxyfile` (+2579 -0) ➕ `ml/backend/ggml/ggml/src/ggml-cann/acl_tensor.cpp` (+181 -0) ➕ `ml/backend/ggml/ggml/src/ggml-cann/acl_tensor.h` (+258 -0) ➕ `ml/backend/ggml/ggml/src/ggml-cann/aclnn_ops.cpp` (+3193 -0) ➕ `ml/backend/ggml/ggml/src/ggml-cann/aclnn_ops.h` (+1125 -0) ➕ `ml/backend/ggml/ggml/src/ggml-cann/common.h` (+420 -0) ➕ `ml/backend/ggml/ggml/src/ggml-cann/ggml-cann.cpp` (+2607 -0) 📝 `scripts/build_docker.sh` (+23 -0) 📝 `scripts/build_linux.sh` (+24 -0) _...and 1 more files_ </details> ### 📄 Description It's a draft for ascend npu support, It can get gpu info for npu, and need to be optimization fix: https://github.com/ollama/ollama/issues/5315 The **pre-builded ollama** that support Huawei Atlas 800 A2 series and Atlas 300I Duo as the backend can be obtained from the following: Pre-builded ollama ENV: - Arch: linux/arm64 - CANN: 8.1.RC1, - ollama and llama.cpp base code: 2025.5.26 **Docker Image:** Atlas 800 A2: `docker pull leopony/ollama-cann-atlas-a2:latest` docker running command example: ``` docker run \ --name ollama \ --device /dev/davinci0 \ --device /dev/davinci_manager \ --device /dev/devmm_svm \ --device /dev/hisi_hdc \ -v /usr/local/dcmi:/usr/local/dcmi \ -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \ -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \ -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \ -v /etc/ascend_install.info:/etc/ascend_install.info \ -p 11434:11434 \ -it leopony/ollama-cann-atlas-a2:latest /bin/bash ``` Atlas 300I Duo: `docker pull leopony/ollama-cann-300i-duo` docker running command example: ``` docker run \ --name ollama \ --device /dev/davinci0 \ --device /dev/davinci_manager \ --device /dev/devmm_svm \ --device /dev/hisi_hdc \ -v /usr/local/dcmi:/usr/local/dcmi \ -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \ -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \ -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \ -v /etc/ascend_install.info:/etc/ascend_install.info \ -p 11434:11434 \ -it leopony/ollama-cann-300i-duo /bin/bash ``` **Bianary tar:** linux-aarch64 Atlas 800 A2: https://github.com/leo-pony/ollama/blob/ollama_bin/ollama-linux-arm64-cann-atlas-a2.tgz linux-aarch64 Atlas 300I Duo: https://github.com/leo-pony/ollama/blob/ollama_bin/ollama-linux-arm64-cann-300i-duo.tgz **Manully build guide:** The build step is same with ollama, details as following: **1) Binary build:** Cd the ollama project directory: For Atlas 800 A2: ``` cmake --preset 'CANN Atlas 800 A2' cmake --build --parallel --preset 'CANN Atlas 800 A2' cmake --install build --component CANN export ARG GOFLAGS="'-ldflags=-w -s'" export ENV CGO_ENABLED=1 go build -trimpath -buildmode=pie -o /bin/ollama . ``` For Atlas 300I Duo： ``` cmake --preset 'CANN Atlas 300I Duo' cmake --build --parallel --preset 'CANN Atlas 300I Duo' cmake --install build --component CANN export ARG GOFLAGS="'-ldflags=-w -s'" export ENV CGO_ENABLED=1 go build -trimpath -buildmode=pie -o /bin/ollama . ``` **2) Docker build:** If need net proxy, config docker proxy with guide: https://docs.docker.com/engine/daemon/proxy/ Cd the ollama project directory: sh -x ./scripts/build_docker.sh **3) Ollama CANN release packages build:** If need net proxy, config docker proxy with guide: https://docs.docker.com/engine/daemon/proxy/ Cd the ollama project directory: sh -x ./scripts/build_linux.sh --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-04-22 22:29:06 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#37823
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?