[GH-ISSUE #562] litellm config file #12121

New Issue

GiteaMirror · 2026-04-19T18:55:29-05:00

GiteaMirror commented

2026-04-19 18:55:29 -05:00

Originally created by @mafrasiabi on GitHub (Jan 24, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/562

@justinh-rahb Could you please share the litellm config file? I've been struggling to make it work but no matter what I did, I can't see the models on the model list

Originally created by @mafrasiabi on GitHub (Jan 24, 2024). Original GitHub issue: https://github.com/open-webui/open-webui/issues/562 @justinh-rahb Could you please share the litellm config file? I've been struggling to make it work but no matter what I did, I can't see the models on the model list

GiteaMirror closed this issue

2026-04-19 18:55:31 -05:00

GiteaMirror commented

2026-04-19 18:55:32 -05:00

@justinh-rahb commented on GitHub (Jan 25, 2024):

@mafrasiabi I got everything I know about configuring this from their docs:
https://docs.litellm.ai/docs/proxy/configs

Assuming you're following my previously posted configs this should work:

model_list:
  - model_name: gpt-3.5-turbo
    litellm_params:
      model: openai/gpt-3.5-turbo-1106
      api_base: https://api.openai.com/v1
      api_key: "os.environ/OPENAI_API_KEY"
  - model_name: gpt-4-turbo
    litellm_params:
      model: openai/gpt-4-1106-preview
      api_base: https://api.openai.com/v1
      api_key: "os.environ/OPENAI_API_KEY"
  - model_name: mistral-small
    litellm_params:
      model: mistral/mistral-small
      api_base: https://api.mistral.ai/v1
      api_key: "os.environ/MISTRAL_API_KEY"
  - model_name: mistral-medium
    litellm_params:
      model: mistral/mistral-medium
      api_base: https://api.mistral.ai/v1
      api_key: "os.environ/MISTRAL_API_KEY"
  - model_name: claude-2.1
    litellm_params:
      model: claude-2.1
      api_key: "os.environ/ANTHROPIC_API_KEY"
  - model_name: claude-instant-1.2
    litellm_params:
      model: claude-instant-1.2
      api_key: "os.environ/ANTHROPIC_API_KEY"
  - model_name: cohere-command
    litellm_params:
      model: command
      api_key: "os.environ/COHERE_API_KEY"
  - model_name: cohere-command-light
    litellm_params:
      model: command-light
      api_key: "os.environ/COHERE_API_KEY"

general_settings:
  master_key: "os.environ/MASTER_KEY"

If you omit the general_settings: master_key: section you can test LiteLLM with a simple cURL command from the ollama-webui container:

curl http://litellm:8000/v1/models

This will verify that the containers are communicating successfully. If not, try using this instead:
http://host.docker.internal:publish_port/v1/models

Do note that the master_key is required for Ollama WebUI to use an OpenAI endpoint, with no key Ollama WebUI will not query the /v1/models endpoint. Took me a while to figure that one out and should probably be added to the README.md if it's intentional @tjbck Otherwise I can open an issue if this is unexpected behaviour.

@justinh-rahb commented on GitHub (Jan 25, 2024): @mafrasiabi I got everything I know about configuring this from their docs: https://docs.litellm.ai/docs/proxy/configs Assuming you're following my previously posted configs this should work: ```yml model_list: - model_name: gpt-3.5-turbo litellm_params: model: openai/gpt-3.5-turbo-1106 api_base: https://api.openai.com/v1 api_key: "os.environ/OPENAI_API_KEY" - model_name: gpt-4-turbo litellm_params: model: openai/gpt-4-1106-preview api_base: https://api.openai.com/v1 api_key: "os.environ/OPENAI_API_KEY" - model_name: mistral-small litellm_params: model: mistral/mistral-small api_base: https://api.mistral.ai/v1 api_key: "os.environ/MISTRAL_API_KEY" - model_name: mistral-medium litellm_params: model: mistral/mistral-medium api_base: https://api.mistral.ai/v1 api_key: "os.environ/MISTRAL_API_KEY" - model_name: claude-2.1 litellm_params: model: claude-2.1 api_key: "os.environ/ANTHROPIC_API_KEY" - model_name: claude-instant-1.2 litellm_params: model: claude-instant-1.2 api_key: "os.environ/ANTHROPIC_API_KEY" - model_name: cohere-command litellm_params: model: command api_key: "os.environ/COHERE_API_KEY" - model_name: cohere-command-light litellm_params: model: command-light api_key: "os.environ/COHERE_API_KEY" general_settings: master_key: "os.environ/MASTER_KEY" ``` If you omit the `general_settings: master_key:` section you can test LiteLLM with a simple cURL command from the ollama-webui container: ```bash curl http://litellm:8000/v1/models ``` This will verify that the containers are communicating successfully. If not, try using this instead: `http://host.docker.internal:publish_port/v1/models` Do note that the `master_key` is required for Ollama WebUI to use an OpenAI endpoint, with no key Ollama WebUI will not query the `/v1/models` endpoint. Took me a while to figure that one out and should probably be added to the README.md if it's intentional @tjbck Otherwise I can open an issue if this is unexpected behaviour.

GiteaMirror referenced this issue

2026-04-19 22:23:13 -05:00

[GH-ISSUE #12121] issue: Function deactivated after container restart due to missing extra dependencies #16474

GiteaMirror referenced this issue

2026-04-19 22:26:49 -05:00

[GH-ISSUE #12314] issue: when pod with openwebui restarts, dependencies installed on runtime are lost #16549

GiteaMirror referenced this issue

2026-04-19 22:36:13 -05:00

[GH-ISSUE #12888] issue: requirements for tools and filters aren't installed after update #16744

GiteaMirror referenced this issue

2026-04-20 04:40:31 -05:00

[PR #13358] [MERGED] fix: install external requirements of tools and functions to prevent deactivation #23164