docker-compose GPU used but only a little #404

New Issue

GiteaMirror · 2025-11-11T14:20:25-06:00

GiteaMirror commented

2025-11-11 14:20:25 -06:00

Originally created by @airdogvan on GitHub (Mar 3, 2024).

I've installed ollama on bare metal and in the terminal it runs blazingly fast, and I watch the CPU curve and notice that it's not used at all when ollama is busy answering queries.

I didn't succeed in having open-webui communicate with ollama on the host so I tried to run both ollama and open-webui in docker.
Here is my docker-compose file:

version: '3.8'

services:
  ollama:
    volumes:
      - ./ollama:/root/.ollama
    container_name: ollama
    pull_policy: always
    tty: true
    restart: unless-stopped
    image: ollama/ollama:latest
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]


  open-webui:
    build:
      context: .
      args:
        OLLAMA_API_BASE_URL: '/ollama/api'
      dockerfile: Dockerfile
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    volumes:
      - ./open-webui:/app/backend/data
    depends_on:
      - ollama
    ports:
      - ${OPEN_WEBUI_PORT-3000}:8080
    environment:
      - 'OLLAMA_API_BASE_URL=http://ollama:11434/api'
      - 'WEBUI_SECRET_KEY='
    extra_hosts:
      - host.docker.internal:host-gateway
    restart: unless-stopped

#volumes:
#  ollama: {}
#  open-webui: {}

The docker logs state that the GPU is being detected and if I do: nvidia-smi I see that the ollama code is loaded in VRAM.

But when ollama start answering queries, first it's much slower than bare metal and I see the CPU shoot to 75% use (from initial 3%). If fact there's almost no difference (if at all) between running docker-compose with the GPU section or without.

Running all this on Ubuntu 22.04 LTS.
Docker version 24.0.5, build ced0996
docker-compose version 1.29.2, build unknown

Again the logs do say that (if GPU section is included) the GPU is detected, and I verified that it is loaded in the GPU but the CPU usage and sluggishness of the output tell a different story.
Maybe I'm doing something wrong?

Originally created by @airdogvan on GitHub (Mar 3, 2024). I've installed ollama on bare metal and in the terminal it runs blazingly fast, and I watch the CPU curve and notice that it's not used at all when ollama is busy answering queries. I didn't succeed in having open-webui communicate with ollama on the host so I tried to run both ollama and open-webui in docker. Here is my docker-compose file: ``` version: '3.8' services: ollama: volumes: - ./ollama:/root/.ollama container_name: ollama pull_policy: always tty: true restart: unless-stopped image: ollama/ollama:latest deploy: resources: reservations: devices: - driver: nvidia count: 1 capabilities: [gpu] open-webui: build: context: . args: OLLAMA_API_BASE_URL: '/ollama/api' dockerfile: Dockerfile image: ghcr.io/open-webui/open-webui:main container_name: open-webui volumes: - ./open-webui:/app/backend/data depends_on: - ollama ports: - ${OPEN_WEBUI_PORT-3000}:8080 environment: - 'OLLAMA_API_BASE_URL=http://ollama:11434/api' - 'WEBUI_SECRET_KEY=' extra_hosts: - host.docker.internal:host-gateway restart: unless-stopped #volumes: # ollama: {} # open-webui: {} ``` The docker logs state that the GPU is being detected and if I do: nvidia-smi I see that the ollama code is loaded in VRAM. But when ollama start answering queries, first it's much slower than bare metal and I see the CPU shoot to 75% use (from initial 3%). If fact there's almost no difference (if at all) between running docker-compose with the GPU section or without. Running all this on Ubuntu 22.04 LTS. Docker version 24.0.5, build ced0996 docker-compose version 1.29.2, build unknown Again the logs do say that (if GPU section is included) the GPU is detected, and I verified that it is loaded in the GPU but the CPU usage and sluggishness of the output tell a different story. Maybe I'm doing something wrong?

GiteaMirror closed this issue

2025-11-11 14:20:25 -06:00

GiteaMirror referenced this issue

2025-11-11 17:16:54 -06:00

[PR #404] [MERGED] feat: chat history incognito mode #7109

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#404