github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 08:30:05 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #4203] panic: runtime error: invalid memory address or nil pointer dereference #64654

New Issue
Closed
opened 2026-05-03 18:26:45 -05:00 by GiteaMirror · 1 comment
GiteaMirror commented 2026-05-03 18:26:45 -05:00
Owner
Copy Link

Originally created by @inoue0426 on GitHub (May 6, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/4203

What is the issue?

I got below issue when we run the model on M2 Mac Studio with running 800K times.

{"function":"launch_slot_with_data","level":"INFO","line":833,"msg":"slot is processing task","slot_id":0,"task_id":1054116,"tid":"0x1fbd37ac0","timestamp":1714759585}
{"function":"update_slots","ga_i":0,"level":"INFO","line":1816,"msg":"slot progression","n_past":43,"n_past_se":0,"n_prompt_tokens_processed":45,"slot_id":0,"task_id":1054116,"tid":"0x1fbd37ac0","timestamp":1714759585}
{"function":"update_slots","level":"INFO","line":1840,"msg":"kv cache rm [p0, end)","p0":43,"slot_id":0,"task_id":1054116,"tid":"0x1fbd37ac0","timestamp":1714759585}
time=2024-05-03T14:06:25.278-04:00 level=ERROR source=server.go:559 msg="Failed to acquire semaphore" error="context canceled"
[GIN] 2024/05/03 - 14:06:25 | 200 |   2.48244975s |       127.0.0.1 | POST     "/api/generate"
time=2024-05-03T14:06:25.296-04:00 level=ERROR source=server.go:559 msg="Failed to acquire semaphore" error="context canceled"
[GIN] 2024/05/03 - 14:06:25 | 200 |  365.046333ms |       127.0.0.1 | POST     "/api/generate"
[GIN] 2024/05/03 - 14:06:25 | 200 |      15.209µs |       127.0.0.1 | HEAD     "/"
[GIN] 2024/05/03 - 14:06:25 | 200 |         400µs |       127.0.0.1 | POST     "/api/show"
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x0 pc=0x102ab4a90]

goroutine 13 [running]:
github.com/ollama/ollama/server.(*runnerRef).needsReload(0x14000470b40, {0x103099320, 0x1400004ebe0}, 0x140005fe0f0)
	github.c:442 +0x150
github.com/ollama/ollama/server.(*Scheduler).processPending(0x1400004ed20, {0x103099320, 0x1400004ebe0})
	github.com/ollama/ollama/server/sched.go:130 +0x390
github.com/ollama/ollama/server.(*Scheduler).Run.func1()
	github.com/ollama/ollama/server/sched.go:107 +0x28
created by github.com/ollama/ollama/server.(*Scheduler).Run in goroutine 1
	github.com/ollama/ollama/server/sched.go:106 +0xc4

Environment

  • Mac Studio 2022
  • Sonoma 14.4.1
  • ./ollama-darwin -v 0.1.33-rc5

code

  • Ollama server ./ollama-darwin run llama3:8b-instruct-q5_K_M
  • Python script
def run_test(t):
    subprocess.run(['./test.sh', str(t)])

Parallel(n_jobs=-1)(delayed(run_test)(t) for t in tqdm(remains))
  • Shell script
#!/bin/zsh

output=$(./ollama-darwin run llama3:8b-instruct-q5_K_M "does the INPUT TEXT related to biomedicine? answer Y for Yes, N for No. only output 1 character. do not provide explanation. do not include markdown formatting. INPUT TEXT: "${1}" ")

echo "\"${1}\",\"${output}\"" >> res.csv%

Python Script log

76%|█████████████████████████        | 447280/589304 [45:06:42<14:16:23,  2.76it/s] 77%|█████████████████████████▎       | 450940/589304 [45:28:16<13:18:45,  2.89it/s]Error: Post "http://127.0.0.1:11434/api/generate": EOF
Error: Post "http://127.0.0.1:11434/api/generate": EOF
Error: Post "http://127.0.0.1:11434/api/generate": EOF
Error: Post "http://127.0.0.1:11434/api/generate": EOF
Error: Post "http://127.0.0.1:11434/api/generate": EOF
Error: Post "http://127.0.0.1:11434/api/generate": EOF
Error: could not connect to ollama app, is it running?
joblib.externals.loky.process_executor._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/opt/anaconda3/lib/python3.9/site-packages/joblib/externals/loky/process_executor.py", line 436, in _process_worker
    r = call_item()
  File "/opt/anaconda3/lib/python3.9/site-packages/joblib/externals/loky/process_executor.py", line 288, in __call__
    return self.fn(*self.args, **self.kwargs)
  File "/opt/anaconda3/lib/python3.9/site-packages/joblib/_parallel_backends.py", line 595, in __call__
    return self.func(*args, **kwargs)
  File "/opt/anaconda3/lib/python3.9/site-packages/joblib/parallel.py", line 262, in __call__
    return [func(*args, **kwargs)
  File "/opt/anaconda3/lib/python3.9/site-packages/joblib/parallel.py", line 262, in <listcomp>
    return [func(*args, **kwargs)
  File "/Users/inouey2/code/review/llama3-server.py", line 37, in run_test
    subprocess.run(['./test.sh', str(t)])
  File "/opt/anaconda3/lib/python3.9/subprocess.py", line 505, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/opt/anaconda3/lib/python3.9/subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/opt/anaconda3/lib/python3.9/subprocess.py", line 1754, in _execute_child
    self.pid = _posixsubprocess.fork_exec(
ValueError: embedded null byte
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/inouey2/code/review/llama3-server.py", line 39, in <module>
Error: could not connect to ollama app, is it running?
    Parallel(n_jobs=-1)(delayed(run_test)(t) for t in tqdm(remains))
  File "/opt/anaconda3/lib/python3.9/site-packages/joblib/parallel.py", line 1056, in __call__
    self.retrieve()
  File "/opt/anaconda3/lib/python3.9/site-packages/joblib/parallel.py", line 935, in retrieve
Error: could not connect to ollama app, is it running?
    self._output.extend(job.get(timeout=self.timeout))
  File "/opt/anaconda3/lib/python3.9/site-packages/joblib/_parallel_backends.py", line 542, in wrap_future_result
    return future.result(timeout=timeout)
  File "/opt/anaconda3/lib/python3.9/concurrent/futures/_base.py", line 438, in result
    return self.__get_result()
  File "/opt/anaconda3/lib/python3.9/concurrent/futures/_base.py", line 390, in __get_result
    raise self._exception
ValueError: embedded null byte
 77%|█████████████████████████▎       | 452309/589304 [45:36:11<13:48:44,  2.76it/s]

OS

macOS

GPU

Apple

CPU

Apple

Ollama version

0.1.33-rc5

Originally created by @inoue0426 on GitHub (May 6, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/4203 ### What is the issue? I got below issue when we run the model on M2 Mac Studio with running 800K times. ``` {"function":"launch_slot_with_data","level":"INFO","line":833,"msg":"slot is processing task","slot_id":0,"task_id":1054116,"tid":"0x1fbd37ac0","timestamp":1714759585} {"function":"update_slots","ga_i":0,"level":"INFO","line":1816,"msg":"slot progression","n_past":43,"n_past_se":0,"n_prompt_tokens_processed":45,"slot_id":0,"task_id":1054116,"tid":"0x1fbd37ac0","timestamp":1714759585} {"function":"update_slots","level":"INFO","line":1840,"msg":"kv cache rm [p0, end)","p0":43,"slot_id":0,"task_id":1054116,"tid":"0x1fbd37ac0","timestamp":1714759585} time=2024-05-03T14:06:25.278-04:00 level=ERROR source=server.go:559 msg="Failed to acquire semaphore" error="context canceled" [GIN] 2024/05/03 - 14:06:25 | 200 | 2.48244975s | 127.0.0.1 | POST "/api/generate" time=2024-05-03T14:06:25.296-04:00 level=ERROR source=server.go:559 msg="Failed to acquire semaphore" error="context canceled" [GIN] 2024/05/03 - 14:06:25 | 200 | 365.046333ms | 127.0.0.1 | POST "/api/generate" [GIN] 2024/05/03 - 14:06:25 | 200 | 15.209µs | 127.0.0.1 | HEAD "/" [GIN] 2024/05/03 - 14:06:25 | 200 | 400µs | 127.0.0.1 | POST "/api/show" panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x2 addr=0x0 pc=0x102ab4a90] goroutine 13 [running]: github.com/ollama/ollama/server.(*runnerRef).needsReload(0x14000470b40, {0x103099320, 0x1400004ebe0}, 0x140005fe0f0) github.c:442 +0x150 github.com/ollama/ollama/server.(*Scheduler).processPending(0x1400004ed20, {0x103099320, 0x1400004ebe0}) github.com/ollama/ollama/server/sched.go:130 +0x390 github.com/ollama/ollama/server.(*Scheduler).Run.func1() github.com/ollama/ollama/server/sched.go:107 +0x28 created by github.com/ollama/ollama/server.(*Scheduler).Run in goroutine 1 github.com/ollama/ollama/server/sched.go:106 +0xc4 ``` # Environment - Mac Studio 2022 - Sonoma 14.4.1 - ./ollama-darwin -v ```0.1.33-rc5``` # code - Ollama server ```./ollama-darwin run llama3:8b-instruct-q5_K_M``` - Python script ```Python def run_test(t): subprocess.run(['./test.sh', str(t)]) Parallel(n_jobs=-1)(delayed(run_test)(t) for t in tqdm(remains)) ``` - Shell script ```Shell #!/bin/zsh output=$(./ollama-darwin run llama3:8b-instruct-q5_K_M "does the INPUT TEXT related to biomedicine? answer Y for Yes, N for No. only output 1 character. do not provide explanation. do not include markdown formatting. INPUT TEXT: "${1}" ") echo "\"${1}\",\"${output}\"" >> res.csv% ``` # Python Script log ``` 76%|█████████████████████████ | 447280/589304 [45:06:42<14:16:23, 2.76it/s] 77%|█████████████████████████▎ | 450940/589304 [45:28:16<13:18:45, 2.89it/s]Error: Post "http://127.0.0.1:11434/api/generate": EOF Error: Post "http://127.0.0.1:11434/api/generate": EOF Error: Post "http://127.0.0.1:11434/api/generate": EOF Error: Post "http://127.0.0.1:11434/api/generate": EOF Error: Post "http://127.0.0.1:11434/api/generate": EOF Error: Post "http://127.0.0.1:11434/api/generate": EOF Error: could not connect to ollama app, is it running? joblib.externals.loky.process_executor._RemoteTraceback: """ Traceback (most recent call last): File "/opt/anaconda3/lib/python3.9/site-packages/joblib/externals/loky/process_executor.py", line 436, in _process_worker r = call_item() File "/opt/anaconda3/lib/python3.9/site-packages/joblib/externals/loky/process_executor.py", line 288, in __call__ return self.fn(*self.args, **self.kwargs) File "/opt/anaconda3/lib/python3.9/site-packages/joblib/_parallel_backends.py", line 595, in __call__ return self.func(*args, **kwargs) File "/opt/anaconda3/lib/python3.9/site-packages/joblib/parallel.py", line 262, in __call__ return [func(*args, **kwargs) File "/opt/anaconda3/lib/python3.9/site-packages/joblib/parallel.py", line 262, in <listcomp> return [func(*args, **kwargs) File "/Users/inouey2/code/review/llama3-server.py", line 37, in run_test subprocess.run(['./test.sh', str(t)]) File "/opt/anaconda3/lib/python3.9/subprocess.py", line 505, in run with Popen(*popenargs, **kwargs) as process: File "/opt/anaconda3/lib/python3.9/subprocess.py", line 951, in __init__ self._execute_child(args, executable, preexec_fn, close_fds, File "/opt/anaconda3/lib/python3.9/subprocess.py", line 1754, in _execute_child self.pid = _posixsubprocess.fork_exec( ValueError: embedded null byte """ The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/Users/inouey2/code/review/llama3-server.py", line 39, in <module> Error: could not connect to ollama app, is it running? Parallel(n_jobs=-1)(delayed(run_test)(t) for t in tqdm(remains)) File "/opt/anaconda3/lib/python3.9/site-packages/joblib/parallel.py", line 1056, in __call__ self.retrieve() File "/opt/anaconda3/lib/python3.9/site-packages/joblib/parallel.py", line 935, in retrieve Error: could not connect to ollama app, is it running? self._output.extend(job.get(timeout=self.timeout)) File "/opt/anaconda3/lib/python3.9/site-packages/joblib/_parallel_backends.py", line 542, in wrap_future_result return future.result(timeout=timeout) File "/opt/anaconda3/lib/python3.9/concurrent/futures/_base.py", line 438, in result return self.__get_result() File "/opt/anaconda3/lib/python3.9/concurrent/futures/_base.py", line 390, in __get_result raise self._exception ValueError: embedded null byte 77%|█████████████████████████▎ | 452309/589304 [45:36:11<13:48:44, 2.76it/s] ``` ### OS macOS ### GPU Apple ### CPU Apple ### Ollama version 0.1.33-rc5
GiteaMirror added the bug label 2026-05-03 18:26:45 -05:00
GiteaMirror commented 2026-05-03 18:26:48 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (May 6, 2024):

Hi I'm so sorry about this error – this should be fixed in https://github.com/ollama/ollama/pull/4189 and will be in the next release 0.1.34

<!-- gh-comment-id:2097081767 --> @jmorganca commented on GitHub (May 6, 2024): Hi I'm so sorry about this error – this should be fixed in https://github.com/ollama/ollama/pull/4189 and will be in the next release 0.1.34
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#64654
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?