Description: Briefly describe the changes in this pull request.
Changelog: Ensure a changelog entry following the format of Keep a Changelog is added at the bottom of the PR description.
Documentation: Have you updated relevant documentation?
Dependencies: Are there any new dependencies? Have you updated the dependency versions in the documentation?
Description
Parallelizes the Docker image builds. All 6 possible builds (matrix of standard, cuda, ollama, on x86_64 and arm64) are run in parallel on separate build agents.
There are a few more drive by changes that I can back out if necessary:
In addition to the static cuda and ollama tags, build them as suffixes to existing tags (eg v0.1.118 would get v0.1.118, 0.1.118-cuda, and 0.1.118-ollama).
Setup image layer caching, with image layers being cached into GHA's cache.
Effectiveness of this might be minimal, the layers are big enough that we can blow through GitHub's 2GB cache limit.
cuda and ollama tags are built outside of the main branch as well.
Run the frontend JS builds on the build platform. As the final output are static HTML/JS bundles, we can avoid virtualization and compile it on native architecture.
Reordered the dependency install, fetch apt packages before python deps, since I believe that we're more likely to update Python dependencies than OS dependencies, so don't waste the apt cache.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/open-webui/open-webui/pull/1495
**Author:** [@cheahjs](https://github.com/cheahjs)
**Created:** 4/10/2024
**Status:** ✅ Merged
**Merged:** 4/10/2024
**Merged by:** [@tjbck](https://github.com/tjbck)
**Base:** `dev` ← **Head:** `feat/parallelize-docker-build`
---
### 📝 Commits (4)
- [`550fc63`](https://github.com/open-webui/open-webui/commit/550fc63ebd08de7c4108eef1aa302897db86aee8) feat: parallelize x86/arm docker builds
- [`eddaa7f`](https://github.com/open-webui/open-webui/commit/eddaa7fa89e05fe1b70045a39ea1c4dbd4b0b05b) cross compile node
- [`cd91d8a`](https://github.com/open-webui/open-webui/commit/cd91d8a987518ac1a050f24489ee9fc7c4e3c9a7) run apt install first for better potential layer caching
- [`7050d53`](https://github.com/open-webui/open-webui/commit/7050d5371883f0781bac275302962639e2727dcd) build ollama and cuda tags
### 📊 Changes
**2 files changed** (+351 additions, -54 deletions)
<details>
<summary>View changed files</summary>
📝 `.github/workflows/docker-build.yaml` (+319 -22)
📝 `Dockerfile` (+32 -32)
</details>
### 📄 Description
## Pull Request Checklist
- [ ] **Description:** Briefly describe the changes in this pull request.
- [ ] **Changelog:** Ensure a changelog entry following the format of [Keep a Changelog](https://keepachangelog.com/) is added at the bottom of the PR description.
- [ ] **Documentation:** Have you updated relevant documentation?
- [ ] **Dependencies:** Are there any new dependencies? Have you updated the dependency versions in the documentation?
---
## Description
Parallelizes the Docker image builds. All 6 possible builds (matrix of standard, cuda, ollama, on x86_64 and arm64) are run in parallel on separate build agents.
Example build: https://github.com/cheahjs/open-webui-fork/actions/runs/8631540693
There are a few more drive by changes that I can back out if necessary:
1. In addition to the static `cuda` and `ollama` tags, build them as suffixes to existing tags (eg `v0.1.118` would get `v0.1.118`, `0.1.118-cuda`, and `0.1.118-ollama`).
2. Setup image layer caching, with image layers being cached into GHA's cache.
* Effectiveness of this might be minimal, the layers are big enough that we can blow through GitHub's 2GB cache limit.
3. `cuda` and `ollama` tags are built outside of the `main` branch as well.
4. Run the frontend JS builds on the build platform. As the final output are static HTML/JS bundles, we can avoid virtualization and compile it on native architecture.
5. Reordered the dependency install, fetch apt packages before python deps, since I believe that we're more likely to update Python dependencies than OS dependencies, so don't waste the apt cache.
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/open-webui/open-webui/pull/1495
Author: @cheahjs
Created: 4/10/2024
Status: ✅ Merged
Merged: 4/10/2024
Merged by: @tjbck
Base:
dev← Head:feat/parallelize-docker-build📝 Commits (4)
550fc63feat: parallelize x86/arm docker buildseddaa7fcross compile nodecd91d8arun apt install first for better potential layer caching7050d53build ollama and cuda tags📊 Changes
2 files changed (+351 additions, -54 deletions)
View changed files
📝
.github/workflows/docker-build.yaml(+319 -22)📝
Dockerfile(+32 -32)📄 Description
Pull Request Checklist
Description
Parallelizes the Docker image builds. All 6 possible builds (matrix of standard, cuda, ollama, on x86_64 and arm64) are run in parallel on separate build agents.
Example build: https://github.com/cheahjs/open-webui-fork/actions/runs/8631540693
There are a few more drive by changes that I can back out if necessary:
cudaandollamatags, build them as suffixes to existing tags (egv0.1.118would getv0.1.118,0.1.118-cuda, and0.1.118-ollama).cudaandollamatags are built outside of themainbranch as well.🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.