Daniel Hiltgen 3f30836734 CUDA: filter devices on secondary discovery (#13317) ...

We now do a deeper probe of CUDA devices to verify the library version has
the correct compute capability coverage for the device.  Due to ROCm also
interpreting the CUDA env var to filter AMD devices, we try to avoid setting
it which leads to problems in mixed vendor systems.  However without setting
it for this deeper probe, each CUDA library subprocess discovers all CUDA GPUs
and on systems with lots of GPUs, this can lead to hitting timeouts.  The fix is
to turn on the CUDA visibility env var just for this deeper probe use-case.

2025-12-03 12:58:16 -08:00

..

llm_darwin.go

Optimize container images for startup (#6547)

2024-09-12 12:10:30 -07:00

llm_linux.go

Optimize container images for startup (#6547)

2024-09-12 12:10:30 -07:00

llm_windows.go

win: lint fix (#10571)

2025-05-05 11:08:12 -07:00

server_test.go

llm: Don't always evict models on CPU-only systems

2025-12-02 10:58:08 -08:00

server.go

CUDA: filter devices on secondary discovery (#13317)

2025-12-03 12:58:16 -08:00

status.go

logs: catch rocm errors (#12888)

2025-10-31 09:54:25 -07:00