Is your feature request related to a problem? Please describe.
I want to use dunzhang/stella_en_400M_v5 - it is lightweight and high on MTEB leaderboard.
Describe the solution you'd like
I want xformers dependency added in Dockerfile, because there is an error when I set RAG_EMBEDDING_MODEL=dunzhang/stella_en_400M_v5 in .env file.
Describe alternatives you've considered
An alternative would be to serve this embedding model at RAG_OPENAI_API_BASE_URL value endpoint .
Additional context
The error mentioned above:
Traceback (most recent call last):
File "/usr/local/bin/uvicorn", line 8, in <module>
sys.exit(main())
^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main
rv= self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 410, in main
run(
File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 577, in run
server.run()
File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 65, in run
return asyncio.run(self.serve(sockets=sockets))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 69, in serve
await self._serve(sockets)
File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 76, in _serve
config.load()
File "/usr/local/lib/python3.11/site-packages/uvicorn/config.py", line 434, in load
self.loaded_app = import_from_string(self.app)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/uvicorn/importer.py", line 19, in import_from_string
module= importlib.import_module(module_str)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/app/backend/open_webui/main.py", line 49, in <module>
from open_webui.apps.retrieval.main import app as retrieval_app
File "/app/backend/open_webui/apps/retrieval/main.py", line 228, in <module>
update_embedding_model(
File "/app/backend/open_webui/apps/retrieval/main.py", line 185, in update_embedding_model
app.state.sentence_transformer_ef = SentenceTransformer(
^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 306, in __init__
modules, self.module_kwargs = self._load_sbert_model(
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 1722, in _load_sbert_model
module= module_class(model_name_or_path, cache_dir=cache_folder, backend=self.backend, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/sentence_transformers/models/Transformer.py", line 76, in __init__
self._load_model(model_name_or_path, config, cache_dir, backend, **model_args)
File "/usr/local/lib/python3.11/site-packages/sentence_transformers/models/Transformer.py", line 108, in _load_model
self.auto_model = AutoModel.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 559, in from_pretrained
return model_class.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/transformers/modeling_utils.py", line 4096, in from_pretrained
model= cls(config, *model_args, **model_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/data/cache/embedding/models/modules/transformers_modules/2aa5579fcae1c579de199a3866b6e514bbbf5d10/modeling.py", line 823, in __init__
self.encoder = NewEncoder(config)
^^^^^^^^^^^^^^^^^^
File "/app/backend/data/cache/embedding/models/modules/transformers_modules/2aa5579fcae1c579de199a3866b6e514bbbf5d10/modeling.py", line 696, in __init__
self.layer = nn.ModuleList([NewLayer(config)for _ in range(config.num_hidden_layers)])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/data/cache/embedding/models/modules/transformers_modules/2aa5579fcae1c579de199a3866b6e514bbbf5d10/modeling.py", line 696, in <listcomp>
self.layer = nn.ModuleList([NewLayer(config)for _ in range(config.num_hidden_layers)])
^^^^^^^^^^^^^^^^
File "/app/backend/data/cache/embedding/models/modules/transformers_modules/2aa5579fcae1c579de199a3866b6e514bbbf5d10/modeling.py", line 630, in __init__
self.attention = NEW_ATTENTION_CLASSES[attn_implementation](
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/data/cache/embedding/models/modules/transformers_modules/2aa5579fcae1c579de199a3866b6e514bbbf5d10/modeling.py", line 451, in __init__
assert self.memory_efficient_attention is not None, 'please install xformers'
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: please install xformers
Originally created by @ELigoP on GitHub (Nov 4, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/6693
# Feature Request
**Is your feature request related to a problem? Please describe.**
I want to use [dunzhang/stella_en_400M_v5](https://huggingface.co/dunzhang/stella_en_400M_v5) - it is lightweight and high on [MTEB](https://huggingface.co/spaces/mteb/leaderboard) leaderboard.
**Describe the solution you'd like**
I want `xformers` dependency added in Dockerfile, because there is an error when I set `RAG_EMBEDDING_MODEL=dunzhang/stella_en_400M_v5` in `.env` file.
**Describe alternatives you've considered**
An alternative would be to serve this embedding model at `RAG_OPENAI_API_BASE_URL` value endpoint .
**Additional context**
The error mentioned above:
```sh
Traceback (most recent call last):
File "/usr/local/bin/uvicorn", line 8, in <module>
sys.exit(main())
^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 410, in main
run(
File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 577, in run
server.run()
File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 65, in run
return asyncio.run(self.serve(sockets=sockets))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 69, in serve
await self._serve(sockets)
File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 76, in _serve
config.load()
File "/usr/local/lib/python3.11/site-packages/uvicorn/config.py", line 434, in load
self.loaded_app = import_from_string(self.app)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/uvicorn/importer.py", line 19, in import_from_string
module = importlib.import_module(module_str)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/app/backend/open_webui/main.py", line 49, in <module>
from open_webui.apps.retrieval.main import app as retrieval_app
File "/app/backend/open_webui/apps/retrieval/main.py", line 228, in <module>
update_embedding_model(
File "/app/backend/open_webui/apps/retrieval/main.py", line 185, in update_embedding_model
app.state.sentence_transformer_ef = SentenceTransformer(
^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 306, in __init__
modules, self.module_kwargs = self._load_sbert_model(
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 1722, in _load_sbert_model
module = module_class(model_name_or_path, cache_dir=cache_folder, backend=self.backend, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/sentence_transformers/models/Transformer.py", line 76, in __init__
self._load_model(model_name_or_path, config, cache_dir, backend, **model_args)
File "/usr/local/lib/python3.11/site-packages/sentence_transformers/models/Transformer.py", line 108, in _load_model
self.auto_model = AutoModel.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 559, in from_pretrained
return model_class.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/transformers/modeling_utils.py", line 4096, in from_pretrained
model = cls(config, *model_args, **model_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/data/cache/embedding/models/modules/transformers_modules/2aa5579fcae1c579de199a3866b6e514bbbf5d10/modeling.py", line 823, in __init__
self.encoder = NewEncoder(config)
^^^^^^^^^^^^^^^^^^
File "/app/backend/data/cache/embedding/models/modules/transformers_modules/2aa5579fcae1c579de199a3866b6e514bbbf5d10/modeling.py", line 696, in __init__
self.layer = nn.ModuleList([NewLayer(config) for _ in range(config.num_hidden_layers)])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/data/cache/embedding/models/modules/transformers_modules/2aa5579fcae1c579de199a3866b6e514bbbf5d10/modeling.py", line 696, in <listcomp>
self.layer = nn.ModuleList([NewLayer(config) for _ in range(config.num_hidden_layers)])
^^^^^^^^^^^^^^^^
File "/app/backend/data/cache/embedding/models/modules/transformers_modules/2aa5579fcae1c579de199a3866b6e514bbbf5d10/modeling.py", line 630, in __init__
self.attention = NEW_ATTENTION_CLASSES[attn_implementation](
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/data/cache/embedding/models/modules/transformers_modules/2aa5579fcae1c579de199a3866b6e514bbbf5d10/modeling.py", line 451, in __init__
assert self.memory_efficient_attention is not None, 'please install xformers'
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: please install xformers
```
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @ELigoP on GitHub (Nov 4, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/6693
Feature Request
Is your feature request related to a problem? Please describe.
I want to use dunzhang/stella_en_400M_v5 - it is lightweight and high on MTEB leaderboard.
Describe the solution you'd like
I want
xformersdependency added in Dockerfile, because there is an error when I setRAG_EMBEDDING_MODEL=dunzhang/stella_en_400M_v5in.envfile.Describe alternatives you've considered
An alternative would be to serve this embedding model at
RAG_OPENAI_API_BASE_URLvalue endpoint .Additional context
The error mentioned above: