github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 16:11:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #7668] is:issue iOllama Error - {json_chunk}s:open #4894

New Issue
Closed
opened 2026-04-12 15:55:59 -05:00 by GiteaMirror · 9 comments
GiteaMirror commented 2026-04-12 15:55:59 -05:00
Owner
Copy Link

Originally created by @papiche on GitHub (Nov 14, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/7668

What is the issue?

Using aider.chat with ollama, got

 Pourlitellm.APIConnectionError: Ollama Error - {'error': 'an unknown error was encountered while running the model '}
Traceback (most recent call last):
  File "/home/fred/.astro/lib/python3.12/site-packages/litellm/utils.py", line 7023, in chunk_creator
    response_obj = self.handle_ollama_stream(chunk)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fred/.astro/lib/python3.12/site-packages/litellm/utils.py", line 6566, in handle_ollama_stream
    raise e
  File "/home/fred/.astro/lib/python3.12/site-packages/litellm/utils.py", line 6541, in handle_ollama_stream
    raise Exception(f"Ollama Error - {json_chunk}")
Exception: Ollama Error - {'error': 'an unknown error was encountered while running the model '}

Retrying in 0.2 seconds...
 Pourlitellm.APIConnectionError: Ollama Error - {'error': 'an unknown error was encountered while running the model '}
Traceback (most recent call last):
  File "/home/fred/.astro/lib/python3.12/site-packages/litellm/utils.py", line 7023, in chunk_creator
    response_obj = self.handle_ollama_stream(chunk)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fred/.astro/lib/python3.12/site-packages/litellm/utils.py", line 6566, in handle_ollama_stream
    raise e
  File "/home/fred/.astro/lib/python3.12/site-packages/litellm/utils.py", line 6541, in handle_ollama_stream
    raise Exception(f"Ollama Error - {json_chunk}")
Exception: Ollama Error - {'error': 'an unknown error was encountered while running the model '}

https://github.com/Aider-AI/aider/issues/2372

any idea on what is wrong ?

OS

Linux

GPU

Nvidia

CPU

AMD

Ollama version

0.4.1

Originally created by @papiche on GitHub (Nov 14, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/7668 ### What is the issue? Using aider.chat with ollama, got ``` Pourlitellm.APIConnectionError: Ollama Error - {'error': 'an unknown error was encountered while running the model '} Traceback (most recent call last): File "/home/fred/.astro/lib/python3.12/site-packages/litellm/utils.py", line 7023, in chunk_creator response_obj = self.handle_ollama_stream(chunk) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/fred/.astro/lib/python3.12/site-packages/litellm/utils.py", line 6566, in handle_ollama_stream raise e File "/home/fred/.astro/lib/python3.12/site-packages/litellm/utils.py", line 6541, in handle_ollama_stream raise Exception(f"Ollama Error - {json_chunk}") Exception: Ollama Error - {'error': 'an unknown error was encountered while running the model '} Retrying in 0.2 seconds... Pourlitellm.APIConnectionError: Ollama Error - {'error': 'an unknown error was encountered while running the model '} Traceback (most recent call last): File "/home/fred/.astro/lib/python3.12/site-packages/litellm/utils.py", line 7023, in chunk_creator response_obj = self.handle_ollama_stream(chunk) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/fred/.astro/lib/python3.12/site-packages/litellm/utils.py", line 6566, in handle_ollama_stream raise e File "/home/fred/.astro/lib/python3.12/site-packages/litellm/utils.py", line 6541, in handle_ollama_stream raise Exception(f"Ollama Error - {json_chunk}") Exception: Ollama Error - {'error': 'an unknown error was encountered while running the model '} ``` https://github.com/Aider-AI/aider/issues/2372 any idea on what is wrong ? ### OS Linux ### GPU Nvidia ### CPU AMD ### Ollama version 0.4.1
GiteaMirror added the needs more infobug labels 2026-04-12 15:56:00 -05:00
GiteaMirror commented 2026-04-12 15:56:02 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Nov 14, 2024):

Server logs would aid in debugging.

<!-- gh-comment-id:2477233031 --> @rick-github commented on GitHub (Nov 14, 2024): [Server logs](https://github.com/ollama/ollama/blob/main/docs/troubleshooting.md#how-to-troubleshoot-issues) would aid in debugging.
GiteaMirror commented 2026-04-12 15:56:02 -05:00
Author
Owner
Copy Link

@krmao commented on GitHub (Dec 27, 2024):

please how to fix?
same here


                                                                                                                          Conclusion

In this tutorial, we have learned how to create a simple To Do App using HTML CSS JavaScript and Firebase Realtime Database. I hope you enjoyed this tutorial. If you have any questions or suggestions, please feel free to leave a comment below.


Tokens: 140 sent, 1.2k received.
Edit the files? (Y)es/(N)o [Yes]: y
Based                                                                                                                                                                                                                                                          litellm.APIConnectionError: Ollama Error - {'error': 'an error was encountered while running the model: unexpected EOF'}
Traceback (most recent call last):
  File "/Users/kr.mao/.local/share/uv/tools/aider-chat/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 1272, in chunk_creator
    response_obj = self.handle_ollama_chat_stream(chunk)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/kr.mao/.local/share/uv/tools/aider-chat/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 730, in handle_ollama_chat_stream
    raise e
  File "/Users/kr.mao/.local/share/uv/tools/aider-chat/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 705, in handle_ollama_chat_stream
    raise Exception(f"Ollama Error - {json_chunk}")
Exception: Ollama Error - {'error': 'an error was encountered while running the model: unexpected EOF'}

Retrying in 0.2 seconds...
Based                                                                                                                                                                                                                                                          litellm.APIConnectionError: Ollama Error - {'error': 'an error was encountered while running the model: unexpected EOF'}
Traceback (most recent call last):
  File "/Users/kr.mao/.local/share/uv/tools/aider-chat/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 1272, in chunk_creator
    response_obj = self.handle_ollama_chat_stream(chunk)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/kr.mao/.local/share/uv/tools/aider-chat/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 730, in handle_ollama_chat_stream
    raise e
  File "/Users/kr.mao/.local/share/uv/tools/aider-chat/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 705, in handle_ollama_chat_stream
    raise Exception(f"Ollama Error - {json_chunk}")
Exception: Ollama Error - {'error': 'an error was encountered while running the model: unexpected EOF'}

Retrying in 0.5 seconds...
Based
<!-- gh-comment-id:2563541039 --> @krmao commented on GitHub (Dec 27, 2024): please how to fix? same here ``` Conclusion In this tutorial, we have learned how to create a simple To Do App using HTML CSS JavaScript and Firebase Realtime Database. I hope you enjoyed this tutorial. If you have any questions or suggestions, please feel free to leave a comment below. Tokens: 140 sent, 1.2k received. Edit the files? (Y)es/(N)o [Yes]: y Based litellm.APIConnectionError: Ollama Error - {'error': 'an error was encountered while running the model: unexpected EOF'} Traceback (most recent call last): File "/Users/kr.mao/.local/share/uv/tools/aider-chat/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 1272, in chunk_creator response_obj = self.handle_ollama_chat_stream(chunk) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/kr.mao/.local/share/uv/tools/aider-chat/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 730, in handle_ollama_chat_stream raise e File "/Users/kr.mao/.local/share/uv/tools/aider-chat/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 705, in handle_ollama_chat_stream raise Exception(f"Ollama Error - {json_chunk}") Exception: Ollama Error - {'error': 'an error was encountered while running the model: unexpected EOF'} Retrying in 0.2 seconds... Based litellm.APIConnectionError: Ollama Error - {'error': 'an error was encountered while running the model: unexpected EOF'} Traceback (most recent call last): File "/Users/kr.mao/.local/share/uv/tools/aider-chat/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 1272, in chunk_creator response_obj = self.handle_ollama_chat_stream(chunk) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/kr.mao/.local/share/uv/tools/aider-chat/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 730, in handle_ollama_chat_stream raise e File "/Users/kr.mao/.local/share/uv/tools/aider-chat/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 705, in handle_ollama_chat_stream raise Exception(f"Ollama Error - {json_chunk}") Exception: Ollama Error - {'error': 'an error was encountered while running the model: unexpected EOF'} Retrying in 0.5 seconds... Based ```
GiteaMirror commented 2026-04-12 15:56:03 -05:00
Author
Owner
Copy Link

@krmao commented on GitHub (Dec 27, 2024):

server.log upload

server.log

@rick-github

<!-- gh-comment-id:2563543350 --> @krmao commented on GitHub (Dec 27, 2024): server.log upload [server.log](https://github.com/user-attachments/files/18259573/server.log) @rick-github
GiteaMirror commented 2026-04-12 15:56:04 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Dec 27, 2024):

llama.cpp:18996: The current context does not support K-shift

https://github.com/ollama/ollama/issues/5975

<!-- gh-comment-id:2563623179 --> @rick-github commented on GitHub (Dec 27, 2024): ``` llama.cpp:18996: The current context does not support K-shift ``` https://github.com/ollama/ollama/issues/5975
GiteaMirror commented 2026-04-12 15:56:05 -05:00
Author
Owner
Copy Link

@krmao commented on GitHub (Dec 30, 2024):

I did tried, not work

Modefile

FROM deepseek-coder-v2
PARAMETER num_ctx 24576
PARAMETER num_predict 8192


                                                                                                                            index.                                                                                                                             litellm.APIConnectionError: Ollama Error - {'error': 'an error was encountered while running the model: unexpected EOF'}
Traceback (most recent call last):
  File "/Users/kr.mao/.local/share/uv/tools/aider-chat/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 1272, in chunk_creator
    response_obj = self.handle_ollama_chat_stream(chunk)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/kr.mao/.local/share/uv/tools/aider-chat/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 730, in handle_ollama_chat_stream
    raise e
  File "/Users/kr.mao/.local/share/uv/tools/aider-chat/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 705, in handle_ollama_chat_stream
    raise Exception(f"Ollama Error - {json_chunk}")
Exception: Ollama Error - {'error': 'an error was encountered while running the model: unexpected EOF'}
Here are the updated files based on your request to create a simple To-Do list application using HTML, CSS, and JavaScript:

                                                                                                                            index.
litellm.APIConnectionError: Ollama Error - {'error': 'an error was encountered while running the model: unexpected EOF'}
Traceback (most recent call last):
  File "/Users/kr.mao/.local/share/uv/tools/aider-chat/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 1272, in chunk_creator
    response_obj = self.handle_ollama_chat_stream(chunk)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/kr.mao/.local/share/uv/tools/aider-chat/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 730, in handle_ollama_chat_stream
    raise e
  File "/Users/kr.mao/.local/share/uv/tools/aider-chat/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 705, in handle_ollama_chat_stream
    raise Exception(f"Ollama Error - {json_chunk}")
Exception: Ollama Error - {'error': 'an error was encountered while running the model: unexpected EOF'}



index.html
Create new file? (Y)es/(N)o [Yes]:
<!-- gh-comment-id:2564946244 --> @krmao commented on GitHub (Dec 30, 2024): I did tried, not work Modefile ``` FROM deepseek-coder-v2 PARAMETER num_ctx 24576 PARAMETER num_predict 8192 ``` ``` index. litellm.APIConnectionError: Ollama Error - {'error': 'an error was encountered while running the model: unexpected EOF'} Traceback (most recent call last): File "/Users/kr.mao/.local/share/uv/tools/aider-chat/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 1272, in chunk_creator response_obj = self.handle_ollama_chat_stream(chunk) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/kr.mao/.local/share/uv/tools/aider-chat/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 730, in handle_ollama_chat_stream raise e File "/Users/kr.mao/.local/share/uv/tools/aider-chat/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 705, in handle_ollama_chat_stream raise Exception(f"Ollama Error - {json_chunk}") Exception: Ollama Error - {'error': 'an error was encountered while running the model: unexpected EOF'} Here are the updated files based on your request to create a simple To-Do list application using HTML, CSS, and JavaScript: index. litellm.APIConnectionError: Ollama Error - {'error': 'an error was encountered while running the model: unexpected EOF'} Traceback (most recent call last): File "/Users/kr.mao/.local/share/uv/tools/aider-chat/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 1272, in chunk_creator response_obj = self.handle_ollama_chat_stream(chunk) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/kr.mao/.local/share/uv/tools/aider-chat/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 730, in handle_ollama_chat_stream raise e File "/Users/kr.mao/.local/share/uv/tools/aider-chat/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 705, in handle_ollama_chat_stream raise Exception(f"Ollama Error - {json_chunk}") Exception: Ollama Error - {'error': 'an error was encountered while running the model: unexpected EOF'} index.html Create new file? (Y)es/(N)o [Yes]: ```
GiteaMirror commented 2026-04-12 15:56:05 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Dec 30, 2024):

Server log from this recent failure?

<!-- gh-comment-id:2564964413 --> @rick-github commented on GitHub (Dec 30, 2024): Server log from this recent failure?
GiteaMirror commented 2026-04-12 15:56:06 -05:00
Author
Owner
Copy Link

@krmao commented on GitHub (Dec 30, 2024):

server.log

logs here

<!-- gh-comment-id:2564965169 --> @krmao commented on GitHub (Dec 30, 2024): [server.log](https://github.com/user-attachments/files/18271645/server.log) logs here
GiteaMirror commented 2026-04-12 15:56:06 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Dec 30, 2024):

time=2024-12-30T09:56:22.830+08:00 level=INFO source=server.go:376 msg="starting llama server" cmd="/Applications/Ollama.app/Contents/Resources/ollama runner --model /Users/kr.mao/.ollama/models/blobs/sha256-5ff0abeeac1d2dbdd5455c0b49ba3b29a9ce3c1fb181b2eef2e948689d55d046 --ctx-size 8192 --batch-size 512 --n-gpu-layers 28 --threads 6 --parallel 4 --port 54930"

The model that ollama is using has a context size of 2K (8192 = 2K * 4) so it's still the old model. After you created the Modelfile, what steps did you take to create the new model? Did you update your client to use the new model?

<!-- gh-comment-id:2564968717 --> @rick-github commented on GitHub (Dec 30, 2024): ``` time=2024-12-30T09:56:22.830+08:00 level=INFO source=server.go:376 msg="starting llama server" cmd="/Applications/Ollama.app/Contents/Resources/ollama runner --model /Users/kr.mao/.ollama/models/blobs/sha256-5ff0abeeac1d2dbdd5455c0b49ba3b29a9ce3c1fb181b2eef2e948689d55d046 --ctx-size 8192 --batch-size 512 --n-gpu-layers 28 --threads 6 --parallel 4 --port 54930" ``` The model that ollama is using has a context size of 2K (8192 = 2K * 4) so it's still the old model. After you created the Modelfile, what steps did you take to create the new model? Did you update your client to use the new model?
GiteaMirror commented 2026-04-12 15:56:07 -05:00
Author
Owner
Copy Link

@krmao commented on GitHub (Dec 30, 2024):

Thanks for response about the words old model, I find the issue key is command ollama create deepseek-coder-v2-fixed -f Modelfile that create a new ollama model.

I didn't understand it before and thought it was just a bug fix

So I should use the deepseek-coder-v2-fixed model and not use deepseek-coder-v2.
now it works!

aider --model ollama_chat/deepseek-coder-v2-fixed --architect

Thanks again !

<!-- gh-comment-id:2564987476 --> @krmao commented on GitHub (Dec 30, 2024): Thanks for response about the words ***old model***, I find the issue key is command `ollama create deepseek-coder-v2-fixed -f Modelfile` that create a new ollama model. > I didn't understand it before and thought it was just a bug fix So I should use the `deepseek-coder-v2-fixed` model and not use `deepseek-coder-v2`. now it works! ``` aider --model ollama_chat/deepseek-coder-v2-fixed --architect ``` Thanks again !
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug needs more info
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#4894
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?