Are we able to add Multi modal support? Ollama now supports this and it would be great to see it added to this very lovely UI I've just come across and started using 🙏 It would be great to be able to a) answer questions asked about images added to "documents" and b) generate new images from prompts.
Originally created by @prologic on GitHub (Feb 3, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/631
Are we able to add Multi modal support? Ollama now supports this and it would be great to see it added to this very lovely UI I've just come across and started using 🙏 It would be great to be able to a) answer questions asked about images added to "documents" and b) generate new images from prompts.
Hi, feature a) has implemented a long time ago! you should just be able to drag and drop images to your chat input, make sure to use multimodal models like llava. As for the feature b), generating images is not within the scopes of multimodal models and Ollama does not support diffusion models and only supports LLMs afaik, but we'll consider adding it in the future (maybe via stable-diffusion-webui apis). Thanks!
<!-- gh-comment-id:1925182223 -->
@tjbck commented on GitHub (Feb 3, 2024):
Hi, feature a) has implemented a long time ago! you should just be able to drag and drop images to your chat input, make sure to use multimodal models like llava. As for the feature b), generating images is not within the scopes of multimodal models and Ollama does not support diffusion models and only supports LLMs afaik, but we'll consider adding it in the future (maybe via stable-diffusion-webui apis). Thanks!
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @prologic on GitHub (Feb 3, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/631
Are we able to add Multi modal support? Ollama now supports this and it would be great to see it added to this very lovely UI I've just come across and started using 🙏 It would be great to be able to a) answer questions asked about images added to "documents" and b) generate new images from prompts.
@tjbck commented on GitHub (Feb 3, 2024):
Hi, feature a) has implemented a long time ago! you should just be able to drag and drop images to your chat input, make sure to use multimodal models like llava. As for the feature b), generating images is not within the scopes of multimodal models and Ollama does not support diffusion models and only supports LLMs afaik, but we'll consider adding it in the future (maybe via stable-diffusion-webui apis). Thanks!
@prologic commented on GitHub (Feb 3, 2024):
Thanks I realized the same after I posted this! Sorry 🙏