[GH-ISSUE #13186] feat: update default cuda version to cu128 #32365

New Issue

GiteaMirror · 2026-04-25T06:16:22-05:00

GiteaMirror commented

2026-04-25 06:16:22 -05:00

Originally created by @Mister-Hope on GitHub (Apr 24, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/13186

Check Existing Issues

I have searched the existing issues and discussions.

Problem Description

Pytorch has release a new 2.7 stable version that support CUDA 12.8, which adds support for Blackwell GPUs (RTX 50 series and some other new server GPUs). These GPUs can not work with current cu121.

See https://pytorch.org/blog/pytorch-2-7/

Desired Solution you'd like

Set the default version of USE_CUDA_VER to cu128, so that users which has a Blackwell GPU can avoid forking and editing docker file, e.g.: https://github.com/Mister-Hope/open-webui-blackwell/tree/cu128

Alternatives Considered

No response

Additional Context

No response

Originally created by @Mister-Hope on GitHub (Apr 24, 2025). Original GitHub issue: https://github.com/open-webui/open-webui/issues/13186 ### Check Existing Issues - [x] I have searched the existing issues and discussions. ### Problem Description Pytorch has release a new 2.7 stable version that support CUDA 12.8, which adds support for Blackwell GPUs (RTX 50 series and some other new server GPUs). These GPUs can not work with current cu121. See https://pytorch.org/blog/pytorch-2-7/ ### Desired Solution you'd like Set the default version of `USE_CUDA_VER` to `cu128`, so that users which has a Blackwell GPU can avoid forking and editing docker file, e.g.: https://github.com/Mister-Hope/open-webui-blackwell/tree/cu128 ### Alternatives Considered _No response_ ### Additional Context _No response_

GiteaMirror closed this issue

2026-04-25 06:16:22 -05:00

GiteaMirror commented

2026-04-25 06:16:22 -05:00

@ER-EPR commented on GitHub (May 7, 2025):

Found GPU0 Tesla V100-SXM3-32GB which is of cuda capability 7.0.

PyTorch no longer supports this GPU because it is too old.

The minimum cuda capability supported by this library is 7.5.

What should I do now?

@ER-EPR commented on GitHub (May 7, 2025): Found GPU0 Tesla V100-SXM3-32GB which is of cuda capability 7.0. PyTorch no longer supports this GPU because it is too old. The minimum cuda capability supported by this library is 7.5. What should I do now?

GiteaMirror commented

2026-04-25 06:16:23 -05:00

@HotshotGT commented on GitHub (May 21, 2025):

I'd wager there are more people on Pascal that would no longer be supported by 12.8 than there are people on Blackwell that would benefit from changing the default.

@HotshotGT commented on GitHub (May 21, 2025): I'd wager there are more people on Pascal that would no longer be supported by 12.8 than there are people on Blackwell that would benefit from changing the default.

GiteaMirror commented

2026-04-25 06:16:24 -05:00

@Mister-Hope commented on GitHub (May 21, 2025):

I'd wager there are more people on Pascal that would no longer be supported by 12.8 than there are people on Blackwell that would benefit from changing the default.

The you should open a feature request to recommend the official to add an earlier CUDA version image, e.g.; cuda-11 cuda-12-6. I am not sure this is worthy and acceptable as some features with new cuda features may be blocked from open webui.

If latest cuda no longer gives compatibility to your hardware, then you should know that your device is being dropped by NVIDIA because of its age.

everything is rolling up with new features in both code and software, and you should expect you being left behind when you refuse to upgrade your hardware. Same thing happens commonly in many places, e.g.; earlier iPhone for iOS. It's not hard for anyone to fork this repo and build an image with early cuda version, until some new feature which is not runnable is introduced.

@Mister-Hope commented on GitHub (May 21, 2025): > I'd wager there are more people on Pascal that would no longer be supported by 12.8 than there are people on Blackwell that would benefit from changing the default. The you should open a feature request to recommend the official to add an earlier CUDA version image, e.g.; cuda-11 cuda-12-6. I am not sure this is worthy and acceptable as some features with new cuda features may be blocked from open webui. If latest cuda no longer gives compatibility to your hardware, then you should know that your device is being dropped by NVIDIA because of its age. everything is rolling up with new features in both code and software, and you should expect you being left behind when you refuse to upgrade your hardware. Same thing happens commonly in many places, e.g.; earlier iPhone for iOS. It's not hard for anyone to fork this repo and build an image with early cuda version, until some new feature which is not runnable is introduced.

GiteaMirror commented

2026-04-25 06:16:25 -05:00

@ER-EPR commented on GitHub (May 22, 2025):

I'd wager there are more people on Pascal that would no longer be supported by 12.8 than there are people on Blackwell that would benefit from changing the default.

The you should open a feature request to recommend the official to add an earlier CUDA version image, e.g.; cuda-11 cuda-12-6. I am not sure this is worthy and acceptable as some features with new cuda features may be blocked from open webui.

If latest cuda no longer gives compatibility to your hardware, then you should know that your device is being dropped by NVIDIA because of its age.

everything is rolling up with new features in both code and software, and you should expect you being left behind when you refuse to upgrade your hardware. Same thing happens commonly in many places, e.g.; earlier iPhone for iOS. It's not hard for anyone to fork this repo and build an image with early cuda version, until some new feature which is not runnable is introduced.

It's not cuda that is not supported, it is the Pytorch for cuda 12.8 (which is a prerelease) drop compatibility for device with capability below 7.5. Perhapes we need to ask the pytorch project whether it is possible to keep those compatibility in the newest package.

@ER-EPR commented on GitHub (May 22, 2025): > > I'd wager there are more people on Pascal that would no longer be supported by 12.8 than there are people on Blackwell that would benefit from changing the default. > > The you should open a feature request to recommend the official to add an earlier CUDA version image, e.g.; cuda-11 cuda-12-6. I am not sure this is worthy and acceptable as some features with new cuda features may be blocked from open webui. > > If latest cuda no longer gives compatibility to your hardware, then you should know that your device is being dropped by NVIDIA because of its age. > > everything is rolling up with new features in both code and software, and you should expect you being left behind when you refuse to upgrade your hardware. Same thing happens commonly in many places, e.g.; earlier iPhone for iOS. It's not hard for anyone to fork this repo and build an image with early cuda version, until some new feature which is not runnable is introduced. It's not cuda that is not supported, it is the Pytorch for cuda 12.8 (which is a prerelease) drop compatibility for device with capability below 7.5. Perhapes we need to ask the pytorch project whether it is possible to keep those compatibility in the newest package.

GiteaMirror commented

2026-04-25 06:16:25 -05:00

@HotshotGT commented on GitHub (May 22, 2025):

@Mister-Hope

I've already moved back to the non-cuda OWUI image and switched to external embedding and reranking engines, so I have no personal investment at this point.

From a common sense perspective though, I question the wisdom of changing the default build to deprecate a very widespread generation of GPUs and add preliminary support for Blackwell given its poor availability and reception.

It would make sense to change the default if new functionality actually required it.

@HotshotGT commented on GitHub (May 22, 2025): @Mister-Hope I've already moved back to the non-cuda OWUI image and switched to external embedding and reranking engines, so _I_ have no personal investment at this point. From a common sense perspective though, I question the wisdom of changing the default build to deprecate a _very_ widespread generation of GPUs and add preliminary support for Blackwell given its poor availability and reception. It would make sense to change the default if new functionality actually required it.

GiteaMirror commented

2026-04-25 06:16:26 -05:00

@qhaas commented on GitHub (Oct 27, 2025):

Perhapes we need to ask the pytorch project whether it is possible to keep those compatibility in the newest package.

For Pascal users finding this issue, here is the pytorch issue discussing that.

I assume one could build open-webui from source with the latest cuda 12.6 variant pytorch and get pascal support back since that variant of pytorch still supports pascal.

@qhaas commented on GitHub (Oct 27, 2025): > Perhapes we need to ask the pytorch project whether it is possible to keep those compatibility in the newest package. For Pascal users finding this issue, [here is the pytorch issue discussing that](https://github.com/pytorch/pytorch/issues/157517#issuecomment-3434764028). I assume one could build open-webui from source with the latest cuda 12.6 variant pytorch and get pascal support back since that variant of pytorch still supports pascal.

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#32365