feat: GroqCloud integration #936

New Issue

GiteaMirror · 2025-11-11T14:33:59-06:00

GiteaMirror commented

2025-11-11 14:33:59 -06:00

Originally created by @0x7CFE on GitHub (May 16, 2024).

Is your feature request related to a problem? Please describe.
GroqCloud now provides free access to its inference cluster which is too fast to ignore. Also it supports LLama3-70b out of the box with speeds up to 300 tokens/s! This is more than enough for a curious person's use.

Describe the solution you'd like
I'd like to be able to use Open Web UI to act as an interface to GroqCloud to keep all my chats in one place.

Describe alternatives you've considered

Use Groq's own UI. Inconvenient for free users, and feature poor.
Use Telegram chatbots. Work fine but again, I'd like to have it all in one place.
Use own ollama instance. Speed is considerably lower.

Additional context
https://groq.com/
https://console.groq.com/playground
https://console.groq.com/docs/quickstart

curl "https://api.groq.com/openai/v1/chat/completions" \
  -X POST \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${GROQ_API_KEY}" \
  -d '{
         "messages": [
           {
             "role": "user",
             "content": "Explain the importance of fast language models"
           },
         ],
         "model": "llama3-70b-8192",
         "temperature": 1,
         "max_tokens": 1024,
         "top_p": 1,
         "stream": true,
         "stop": null
       }'

Originally created by @0x7CFE on GitHub (May 16, 2024). **Is your feature request related to a problem? Please describe.** GroqCloud now provides free access to its inference cluster which is too fast to ignore. Also it supports LLama3-**70b** out of the box with speeds up to **300 tokens/s**! This is more than enough for a curious person's use. **Describe the solution you'd like** I'd like to be able to use Open Web UI to act as an interface to GroqCloud to keep all my chats in one place. **Describe alternatives you've considered** - Use Groq's own UI. Inconvenient for free users, and feature poor. - Use Telegram chatbots. Work fine but again, I'd like to have it all in one place. - Use own ollama instance. Speed is considerably lower. **Additional context** https://groq.com/ https://console.groq.com/playground https://console.groq.com/docs/quickstart ```bash curl "https://api.groq.com/openai/v1/chat/completions" \ -X POST \ -H "Content-Type: application/json" \ -H "Authorization: Bearer ${GROQ_API_KEY}" \ -d '{ "messages": [ { "role": "user", "content": "Explain the importance of fast language models" }, ], "model": "llama3-70b-8192", "temperature": 1, "max_tokens": 1024, "top_p": 1, "stream": true, "stop": null }' ```

GiteaMirror closed this issue

2025-11-11 14:33:59 -06:00

GiteaMirror referenced this issue

2025-11-11 17:23:45 -06:00

[PR #941] [MERGED] fix: added focus selector #7334

GiteaMirror referenced this issue

2025-11-11 17:23:46 -06:00

[PR #951] [MERGED] 0.1.106 #7335

Sign in to join this conversation.