This Docker Compose will setup Ollama to run with AMD GPU support using the :rocm tagged image. Some GPUs seem to require this environment variable: HSA_OVERRIDE_GFX_VERSION
For AMD GPU Support: Some AMD GPUs require setting an environment variable for proper functionality:
HSA_OVERRIDE_GFX_VERSION=11.0.0 docker compose -f docker-compose.yaml -f docker-compose.amdgpu.yaml up -d --build
AMD GPU Support with HSA_OVERRIDE_GFX_VERSION
For AMD GPU users encountering compatibility issues, setting the HSA_OVERRIDE_GFX_VERSION environment variable is crucial. This variable instructs the ROCm platform to emulate a specific GPU architecture, ensuring compatibility with various AMD GPUs not officially supported. Depending on your GPU model, adjust the HSA_OVERRIDE_GFX_VERSION as follows:
For RDNA1 & RDNA2 GPUs (e.g., RX 6700, RX 680M): Use HSA_OVERRIDE_GFX_VERSION=10.3.0.
For RDNA3 GPUs: Set HSA_OVERRIDE_GFX_VERSION=11.0.0.
For older GCN (Graphics Core Next) GPUs: The version to use varies. GCN 4th gen and earlier might require different settings, such as ROC_ENABLE_PRE_VEGA=1 for GCN4, or HSA_OVERRIDE_GFX_VERSION=9.0.0 for Vega (GCN5.0) emulation.
Ensure to replace <version> with the appropriate version number based on your GPU model and the guidelines above. For a detailed list of compatible versions and more in-depth instructions, refer to the ROCm documentation and the openSUSE Wiki on AMD GPGPU.
Example command for RDNA1 & RDNA2 GPUs:
HSA_OVERRIDE_GFX_VERSION=10.3.0 docker compose -f docker-compose.yaml -f docker-compose.amdgpu.yaml up -d --build
Pull Request Checklist
Description: This pull request adds an AMD GPU Docker Compose file to the project. The file is configured to run the ollama/ollama:rocm Docker container, which should improve performance on systems with AMD GPUs.
Changelog:
Added
AMD Docker Compose file for Ollama on AMD GPUs
Changed
Updated documentation to include instructions for using the AMD Docker Compose file
Please review the changes and let me know if you have any questions or suggestions. Thank you!
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/open-webui/open-webui/pull/1439
**Author:** [@justinh-rahb](https://github.com/justinh-rahb)
**Created:** 4/6/2024
**Status:** ✅ Merged
**Merged:** 4/6/2024
**Merged by:** [@tjbck](https://github.com/tjbck)
**Base:** `dev` ← **Head:** `rocm-compose`
---
### 📝 Commits (3)
- [`075dddf`](https://github.com/open-webui/open-webui/commit/075dddf3d3e8492b8ff24c91d002065b73a1f015) Add AMD docker compose file
- [`8db03f3`](https://github.com/open-webui/open-webui/commit/8db03f3ab28b2ae98173352dafc189d0b7cd9837) Add variables
- [`f34b973`](https://github.com/open-webui/open-webui/commit/f34b9733e33ea6f525c3733c96720d45fe749535) Add Docker tag variables
### 📊 Changes
**2 files changed** (+10 additions, -2 deletions)
<details>
<summary>View changed files</summary>
➕ `docker-compose.amdgpu.yaml` (+8 -0)
📝 `docker-compose.yaml` (+2 -2)
</details>
### 📄 Description
This Docker Compose will setup Ollama to run with AMD GPU support using the `:rocm` tagged image. Some GPUs seem to require this environment variable: `HSA_OVERRIDE_GFX_VERSION`
Which could require different values:
https://github.com/ollama/ollama/issues/2637
---
- **For AMD GPU Support:** Some AMD GPUs require setting an environment variable for proper functionality:
```bash
HSA_OVERRIDE_GFX_VERSION=11.0.0 docker compose -f docker-compose.yaml -f docker-compose.amdgpu.yaml up -d --build
```
<details>
<summary>AMD GPU Support with HSA_OVERRIDE_GFX_VERSION</summary>
For AMD GPU users encountering compatibility issues, setting the `HSA_OVERRIDE_GFX_VERSION` environment variable is crucial. This variable instructs the ROCm platform to emulate a specific GPU architecture, ensuring compatibility with various AMD GPUs not officially supported. Depending on your GPU model, adjust the `HSA_OVERRIDE_GFX_VERSION` as follows:
- **For RDNA1 & RDNA2 GPUs** (e.g., RX 6700, RX 680M): Use `HSA_OVERRIDE_GFX_VERSION=10.3.0`.
- **For RDNA3 GPUs**: Set `HSA_OVERRIDE_GFX_VERSION=11.0.0`.
- **For older GCN (Graphics Core Next) GPUs**: The version to use varies. GCN 4th gen and earlier might require different settings, such as `ROC_ENABLE_PRE_VEGA=1` for GCN4, or `HSA_OVERRIDE_GFX_VERSION=9.0.0` for Vega (GCN5.0) emulation.
Ensure to replace `<version>` with the appropriate version number based on your GPU model and the guidelines above. For a detailed list of compatible versions and more in-depth instructions, refer to the [ROCm documentation](https://rocm.docs.amd.com) and the [openSUSE Wiki on AMD GPGPU](https://en.opensuse.org/SDB:AMD_GPGPU).
Example command for RDNA1 & RDNA2 GPUs:
```bash
HSA_OVERRIDE_GFX_VERSION=10.3.0 docker compose -f docker-compose.yaml -f docker-compose.amdgpu.yaml up -d --build
```
</details>
## Pull Request Checklist
- [x] **Description:** This pull request adds an AMD GPU Docker Compose file to the project. The file is configured to run the `ollama/ollama:rocm` Docker container, which should improve performance on systems with AMD GPUs.
- [x] **Changelog:**
### Added
- AMD Docker Compose file for Ollama on AMD GPUs
### Changed
- Updated documentation to include instructions for using the AMD Docker Compose file
---
Please review the changes and let me know if you have any questions or suggestions. Thank you!
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/open-webui/open-webui/pull/1439
Author: @justinh-rahb
Created: 4/6/2024
Status: ✅ Merged
Merged: 4/6/2024
Merged by: @tjbck
Base:
dev← Head:rocm-compose📝 Commits (3)
075dddfAdd AMD docker compose file8db03f3Add variablesf34b973Add Docker tag variables📊 Changes
2 files changed (+10 additions, -2 deletions)
View changed files
➕
docker-compose.amdgpu.yaml(+8 -0)📝
docker-compose.yaml(+2 -2)📄 Description
This Docker Compose will setup Ollama to run with AMD GPU support using the
:rocmtagged image. Some GPUs seem to require this environment variable:HSA_OVERRIDE_GFX_VERSIONWhich could require different values:
https://github.com/ollama/ollama/issues/2637
For AMD GPU Support: Some AMD GPUs require setting an environment variable for proper functionality:
AMD GPU Support with HSA_OVERRIDE_GFX_VERSION
For AMD GPU users encountering compatibility issues, setting the
HSA_OVERRIDE_GFX_VERSIONenvironment variable is crucial. This variable instructs the ROCm platform to emulate a specific GPU architecture, ensuring compatibility with various AMD GPUs not officially supported. Depending on your GPU model, adjust theHSA_OVERRIDE_GFX_VERSIONas follows:HSA_OVERRIDE_GFX_VERSION=10.3.0.HSA_OVERRIDE_GFX_VERSION=11.0.0.ROC_ENABLE_PRE_VEGA=1for GCN4, orHSA_OVERRIDE_GFX_VERSION=9.0.0for Vega (GCN5.0) emulation.Ensure to replace
<version>with the appropriate version number based on your GPU model and the guidelines above. For a detailed list of compatible versions and more in-depth instructions, refer to the ROCm documentation and the openSUSE Wiki on AMD GPGPU.Example command for RDNA1 & RDNA2 GPUs:
Pull Request Checklist
Description: This pull request adds an AMD GPU Docker Compose file to the project. The file is configured to run the
ollama/ollama:rocmDocker container, which should improve performance on systems with AMD GPUs.Changelog:
Added
Changed
Please review the changes and let me know if you have any questions or suggestions. Thank you!
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.