Update: oookay. So, upgrading the remote ChromaDB server to 0.5.3 triggers embedding again. So, if you are using remote ChromaDB, it probably needs to be upgraded. And possibly the data nuked too.
Description
Bug Summary:
Two problems reported in this bug, though it's mostly about the RAG failure:
Original Sumary
I am running 0.3.8. However, the logging seems to have ... stopped. Basically nothing shows up in the logs, until I restart the container. Then, a little bit shows up.
I tried setting up Tika, and pointed OpenWebUI at it. It SEEMS like the context search is triggered, but my embedding model never gets loaded. I switched back to the default content extraction engine, and still no embeddings seem to be generated. Is there something odd going on?
I do have hybrid search turned on.
Document upload succeeds according to browser console. But it's... weirdly fast. Usually the content extraction is done immediately. But now I see no embeddings generated.
It seems like the LLM is being given a search context, but nothing's in it. No citations are given, and responses don't reference uploaded or referenced documents.
Environment
Open WebUI Version: 0.3.8
Ollama (if applicable): 0.2.2
Operating System: Gentoo Linux + Docker
Browser (if applicable): Firefox
Reproduction Details
Use remote ChromaDB server (not sure if this matters, but it's my setup).
Trigger document RAG (upload a file, go to #website, etc).
Make sure it is a fresh document, so something that won't be inserted into existing collection.
The store_docs_in_vector_db method fails with KeyError: 'dimension', seemingly around the create collection call.
I have read and followed all the instructions provided in the README.md.
I am on the latest version of both Open WebUI and Ollama.
Embedding engine (e.g. sentence-transfomers vs ollama) doesn't seem to matter for this issue.
Originally created by @ProjectMoon on GitHub (Jul 10, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/3764
# Bug Report
**Update:** oookay. So, upgrading the remote ChromaDB server to 0.5.3 triggers embedding again. So, if you are using remote ChromaDB, it probably needs to be upgraded. And possibly the data nuked too.
## Description
**Bug Summary:**
Two problems reported in this bug, though it's mostly about the RAG failure:
1. Logging stopped working.
2. Open WebUI seems to be affected by https://github.com/chroma-core/chroma/issues/2440.
**Original Sumary**
I am running 0.3.8. However, the logging seems to have ... stopped. Basically nothing shows up in the logs, until I restart the container. Then, a little bit shows up.
I tried setting up Tika, and pointed OpenWebUI at it. It SEEMS like the context search is triggered, but my embedding model never gets loaded. I switched back to the default content extraction engine, and still no embeddings seem to be generated. Is there something odd going on?
I do have hybrid search turned on.
Document upload succeeds according to browser console. But it's... weirdly fast. Usually the content extraction is done immediately. But now I see no embeddings generated.
It seems like the LLM is being given a search context, but nothing's in it. No citations are given, and responses don't reference uploaded or referenced documents.
## Environment
- **Open WebUI Version:** 0.3.8
- **Ollama (if applicable):** 0.2.2
- **Operating System:** Gentoo Linux + Docker
- **Browser (if applicable):** Firefox
## Reproduction Details
1. Use remote ChromaDB server (not sure if this matters, but it's my setup).
2. Trigger document RAG (upload a file, go to #website, etc).
4. Make sure it is a fresh document, so something that won't be inserted into existing collection.
5. The `store_docs_in_vector_db` method fails with `KeyError: 'dimension'`, seemingly around the create collection call.
- [X] I have read and followed all the instructions provided in the README.md.
- [X] I am on the latest version of both Open WebUI and Ollama.
- [X] I have included the browser console logs.
- [X] I have included the Docker container logs.
## Logs and Screenshots
https://github.com/open-webui/open-webui/issues/3764#issuecomment-2221279283
## Installation Method
Docker
## Additional Information
After some tracing with printlns and container restarts, I think that the _RAG failure_ is related to the ChromaDB create collection call. See: https://github.com/open-webui/open-webui/issues/3764#issuecomment-2221279283
Embedding engine (e.g. sentence-transfomers vs ollama) doesn't seem to matter for this issue.
I was literally just logging in to report this same issue. I can't seem to scan epub files anymore, and when i do docker logs -f and then the ID, i get this
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL
and that's it.
Mine will scan in pdfs, but i have a bunch of linux books i want to scan in that are epub, and it won't add them in.
<!-- gh-comment-id:2221237599 -->
@zono50 commented on GitHub (Jul 10, 2024):
I was literally just logging in to report this same issue. I can't seem to scan epub files anymore, and when i do docker logs -f and then the ID, i get this
/ _ \ _ __ ___ _ __ \ \ / /__| |__ | | | |_ _|
| | | | '_ \ / _ \ '_ \ \ \ /\ / / _ \ '_ \| | | || |
| |_| | |_) | __/ | | | \ V V / __/ |_) | |_| || |
\___/| .__/ \___|_| |_| \_/\_/ \___|_.__/ \___/|___|
|_|
v0.3.8 - building the best open-source AI user interface.
https://github.com/open-webui/open-webui
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL
and that's it.
Mine will scan in pdfs, but i have a bunch of linux books i want to scan in that are epub, and it won't add them in.
The logging is probably unrelated, but it doesn't help in diagnosing the problem with RAG. Python has a particular oddity when it comes to buffered output in Docker. This can usually be worked around with python -u. Not sure why it started in the latest release.
But the RAG pipeline not functioning is rather odd, and I am wondering if it's a problem with the certain combination of settings. Do you have an Ollama embedding model?
<!-- gh-comment-id:2221246160 -->
@ProjectMoon commented on GitHub (Jul 10, 2024):
The logging is probably unrelated, but it doesn't help in diagnosing the problem with RAG. Python has a particular oddity when it comes to buffered output in Docker. This can usually be worked around with `python -u`. Not sure why it started in the latest release.
But the RAG pipeline not functioning is rather odd, and I am wondering if it's a problem with the certain combination of settings. Do you have an Ollama embedding model?
I have an option for the ollama embedding model, also the default sentence transformers. Do i have to setup anything special for the ollama embedding model to choose that as the embedding model engine?
<!-- gh-comment-id:2221268429 -->
@zono50 commented on GitHub (Jul 10, 2024):
I have an option for the ollama embedding model, also the default sentence transformers. Do i have to setup anything special for the ollama embedding model to choose that as the embedding model engine?
Sorry, should have been more clear with my question. Are you using the ollama embedding model as the embedding engine? Or sentence transformers?
<!-- gh-comment-id:2221271499 -->
@ProjectMoon commented on GitHub (Jul 10, 2024):
Sorry, should have been more clear with my question. Are you using the ollama embedding model as the embedding engine? Or sentence transformers?
OK so after a bunch of tracing with print statements and container restarts, I have traced this down to something likely wrong with the ChromaDB create_collection call.
I should note that I am using a remote ChromaDB server, so I am using the HTTP client.
Here is (part of) the RAG code:
defstore_docs_in_vector_db(docs,collection_name,overwrite:bool=False)->bool:log.info(f"store_docs_in_vector_db {docs}{collection_name}")print("storing docs in vector db")texts=[doc.page_contentfordocindocs]metadatas=[doc.metadatafordocindocs]# ChromaDB does not like datetime formats# for meta-data so convert them to string.formetadatainmetadatas:forkey,valueinmetadata.items():ifisinstance(value,datetime):metadata[key]=str(value)try:print("in try")ifoverwrite:print("in overwrite")forcollectioninCHROMA_CLIENT.list_collections():ifcollection_name==collection.name:log.info(f"deleting existing collection {collection_name}")CHROMA_CLIENT.delete_collection(name=collection_name)collection=CHROMA_CLIENT.create_collection(name=collection_name)print("created collection")embedding_func=get_embedding_function(app.state.config.RAG_EMBEDDING_ENGINE,app.state.config.RAG_EMBEDDING_MODEL,app.state.sentence_transformer_ef,app.state.config.OPENAI_API_KEY,app.state.config.OPENAI_API_BASE_URL,app.state.config.RAG_EMBEDDING_OPENAI_BATCH_SIZE,)print("we got an embedding func",embedding_func)
Resulting logs, visible on container restart:
storing docs in vector db
in try
Hello exception 'dimension'
KeyError('dimension')
<!-- gh-comment-id:2221279283 -->
@ProjectMoon commented on GitHub (Jul 10, 2024):
OK so after a bunch of tracing with print statements and container restarts, I have traced this down to something likely wrong with the ChromaDB `create_collection` call.
I should note that I am using a remote ChromaDB server, so I am using the HTTP client.
Here is (part of) the RAG code:
```python
def store_docs_in_vector_db(docs, collection_name, overwrite: bool = False) -> bool:
log.info(f"store_docs_in_vector_db {docs} {collection_name}")
print("storing docs in vector db")
texts = [doc.page_content for doc in docs]
metadatas = [doc.metadata for doc in docs]
# ChromaDB does not like datetime formats
# for meta-data so convert them to string.
for metadata in metadatas:
for key, value in metadata.items():
if isinstance(value, datetime):
metadata[key] = str(value)
try:
print("in try")
if overwrite:
print("in overwrite")
for collection in CHROMA_CLIENT.list_collections():
if collection_name == collection.name:
log.info(f"deleting existing collection {collection_name}")
CHROMA_CLIENT.delete_collection(name=collection_name)
collection = CHROMA_CLIENT.create_collection(name=collection_name)
print("created collection")
embedding_func = get_embedding_function(
app.state.config.RAG_EMBEDDING_ENGINE,
app.state.config.RAG_EMBEDDING_MODEL,
app.state.sentence_transformer_ef,
app.state.config.OPENAI_API_KEY,
app.state.config.OPENAI_API_BASE_URL,
app.state.config.RAG_EMBEDDING_OPENAI_BATCH_SIZE,
)
print("we got an embedding func", embedding_func)
```
Resulting logs, visible on container restart:
```
storing docs in vector db
in try
Hello exception 'dimension'
KeyError('dimension')
```
This is a problem in upstream ChromaDB. OpenWebUI is using ChromaDB 0.5.3, which causes this problem.
<!-- gh-comment-id:2221291852 -->
@ProjectMoon commented on GitHub (Jul 10, 2024):
Aaaand... yes.
https://github.com/chroma-core/chroma/issues/2440
This is a problem in upstream ChromaDB. OpenWebUI is using ChromaDB 0.5.3, which causes this problem.
Edit: upgrading my ChromaDB server to 0.5.3 makes everything work again. So probably the release notes need to be updated to mention this for people using remote ChromaDB. It shouldn't be noticed by people using the in-memory one (I think).
Now, as for the logging... that's another story.
Edit 2: Also had to completely nuke the ChromaDB instance to make things work properly again, clearing it of documents that were already uploaded and failed.
<!-- gh-comment-id:2221294790 -->
@ProjectMoon commented on GitHub (Jul 10, 2024):
Are you using a remote ChromaDB?
Edit: upgrading my ChromaDB server to 0.5.3 makes everything work again. So probably the release notes need to be updated to mention this for people using remote ChromaDB. It shouldn't be noticed by people using the in-memory one (I think).
Now, as for the logging... that's another story.
Edit 2: Also had to completely nuke the ChromaDB instance to make things work properly again, clearing it of documents that were already uploaded and failed.
Thanks for the report, will promptly update the changelog to mention the ChromaDB update requirement!
<!-- gh-comment-id:2221331413 -->
@tjbck commented on GitHub (Jul 10, 2024):
Thanks for the report, will promptly update the changelog to mention the ChromaDB update requirement!
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @ProjectMoon on GitHub (Jul 10, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/3764
Bug Report
Update: oookay. So, upgrading the remote ChromaDB server to 0.5.3 triggers embedding again. So, if you are using remote ChromaDB, it probably needs to be upgraded. And possibly the data nuked too.
Description
Bug Summary:
Two problems reported in this bug, though it's mostly about the RAG failure:
Original Sumary
I am running 0.3.8. However, the logging seems to have ... stopped. Basically nothing shows up in the logs, until I restart the container. Then, a little bit shows up.
I tried setting up Tika, and pointed OpenWebUI at it. It SEEMS like the context search is triggered, but my embedding model never gets loaded. I switched back to the default content extraction engine, and still no embeddings seem to be generated. Is there something odd going on?
I do have hybrid search turned on.
Document upload succeeds according to browser console. But it's... weirdly fast. Usually the content extraction is done immediately. But now I see no embeddings generated.
It seems like the LLM is being given a search context, but nothing's in it. No citations are given, and responses don't reference uploaded or referenced documents.
Environment
Open WebUI Version: 0.3.8
Ollama (if applicable): 0.2.2
Operating System: Gentoo Linux + Docker
Browser (if applicable): Firefox
Reproduction Details
store_docs_in_vector_dbmethod fails withKeyError: 'dimension', seemingly around the create collection call.Logs and Screenshots
https://github.com/open-webui/open-webui/issues/3764#issuecomment-2221279283
Installation Method
Docker
Additional Information
After some tracing with printlns and container restarts, I think that the RAG failure is related to the ChromaDB create collection call. See: https://github.com/open-webui/open-webui/issues/3764#issuecomment-2221279283
Embedding engine (e.g. sentence-transfomers vs ollama) doesn't seem to matter for this issue.
@zono50 commented on GitHub (Jul 10, 2024):
I was literally just logging in to report this same issue. I can't seem to scan epub files anymore, and when i do docker logs -f and then the ID, i get this
/ _ \ _ __ ___ _ __ \ \ / /| | | | | |_ |
| | | | ' \ / _ \ '_ \ \ \ /\ / / _ \ '_ | | | || |
| || | |) | / | | | \ V V / / |) | || || |
_/| ./ _|| || _/_/ _|./ _/|_|
||
v0.3.8 - building the best open-source AI user interface.
https://github.com/open-webui/open-webui
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL
and that's it.
Mine will scan in pdfs, but i have a bunch of linux books i want to scan in that are epub, and it won't add them in.
@ProjectMoon commented on GitHub (Jul 10, 2024):
The logging is probably unrelated, but it doesn't help in diagnosing the problem with RAG. Python has a particular oddity when it comes to buffered output in Docker. This can usually be worked around with
python -u. Not sure why it started in the latest release.But the RAG pipeline not functioning is rather odd, and I am wondering if it's a problem with the certain combination of settings. Do you have an Ollama embedding model?
@zono50 commented on GitHub (Jul 10, 2024):
I have an option for the ollama embedding model, also the default sentence transformers. Do i have to setup anything special for the ollama embedding model to choose that as the embedding model engine?
@ProjectMoon commented on GitHub (Jul 10, 2024):
Sorry, should have been more clear with my question. Are you using the ollama embedding model as the embedding engine? Or sentence transformers?
@zono50 commented on GitHub (Jul 10, 2024):
i'm using sentence transformers
@ricky-davis commented on GitHub (Jul 10, 2024):
Can confirm I'm getting no logs after the github link as well
@ProjectMoon commented on GitHub (Jul 10, 2024):
OK so after a bunch of tracing with print statements and container restarts, I have traced this down to something likely wrong with the ChromaDB
create_collectioncall.I should note that I am using a remote ChromaDB server, so I am using the HTTP client.
Here is (part of) the RAG code:
Resulting logs, visible on container restart:
@ProjectMoon commented on GitHub (Jul 10, 2024):
Aaaand... yes.
https://github.com/chroma-core/chroma/issues/2440
This is a problem in upstream ChromaDB. OpenWebUI is using ChromaDB 0.5.3, which causes this problem.
@zono50 commented on GitHub (Jul 10, 2024):
gotcha, is there a work around for this currently?
@ProjectMoon commented on GitHub (Jul 10, 2024):
Are you using a remote ChromaDB?
Edit: upgrading my ChromaDB server to 0.5.3 makes everything work again. So probably the release notes need to be updated to mention this for people using remote ChromaDB. It shouldn't be noticed by people using the in-memory one (I think).
Now, as for the logging... that's another story.
Edit 2: Also had to completely nuke the ChromaDB instance to make things work properly again, clearing it of documents that were already uploaded and failed.
@tjbck commented on GitHub (Jul 10, 2024):
Thanks for the report, will promptly update the changelog to mention the ChromaDB update requirement!