[GH-ISSUE #9187] Adding Support for Nvidia Hosted Models #118728

New Issue

GiteaMirror · 2026-05-20T20:00:45-05:00

GiteaMirror commented

2026-05-20 20:00:45 -05:00

Originally created by @YashM8 on GitHub (Jan 31, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/9187

Nvidia follows a simple api calls using the OpenAI library but with a different base_url. For example -

from openai import OpenAI

client = OpenAI(
  base_url = "https://integrate.api.nvidia.com/v1",
  api_key = "$API_KEY_REQUIRED_IF_EXECUTING_OUTSIDE_NGC"
)

completion = client.chat.completions.create(
  model="deepseek-ai/deepseek-r1",
  messages=[{"role":"user","content":"Which number is larger, 9.11 or 9.8?"}],
  temperature=0.6,
  top_p=0.7,
  max_tokens=4096,
  stream=True
)

for chunk in completion:
  if chunk.choices[0].delta.content is not None:
    print(chunk.choices[0].delta.content, end="")

A UI modification that lets us change this base_url would be nice.

Originally created by @YashM8 on GitHub (Jan 31, 2025). Original GitHub issue: https://github.com/open-webui/open-webui/issues/9187 Nvidia follows a simple api calls using the OpenAI library but with a different `base_url`. For example - ``` from openai import OpenAI client = OpenAI( base_url = "https://integrate.api.nvidia.com/v1", api_key = "$API_KEY_REQUIRED_IF_EXECUTING_OUTSIDE_NGC" ) completion = client.chat.completions.create( model="deepseek-ai/deepseek-r1", messages=[{"role":"user","content":"Which number is larger, 9.11 or 9.8?"}], temperature=0.6, top_p=0.7, max_tokens=4096, stream=True ) for chunk in completion: if chunk.choices[0].delta.content is not None: print(chunk.choices[0].delta.content, end="") ``` A UI modification that lets us change this `base_url` would be nice.

GiteaMirror closed this issue

2026-05-20 20:00:45 -05:00

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#118728