github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-17 13:47:08 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #3029] Ollama Hanging/Freeze On Embeddings API With Nomic-Embed-Text #1862

New Issue
Closed
opened 2026-04-12 11:55:38 -05:00 by GiteaMirror · 10 comments
GiteaMirror commented 2026-04-12 11:55:38 -05:00
Owner
Copy Link

Originally created by @matthewsmorrison on GitHub (Mar 9, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/3029

Originally assigned to: @jmorganca on GitHub.

I am running Ollama (0.1.28) on a Google Cloud VM (n1-standard-2, Intel Broadwell, NVIDIA T4 GPU, 7.5GB RAM). When I run the cURL command for the embeddings API with the nomic-embed-text model (version: nomic-embed-text:latest 0a109f422b47 )

curl http://localhost:11434/api/embeddings -d '{
  "model": "nomic-embed-text",
  "prompt": "Here is an article about llamas..."
}'

Ollama indefinitely hangs. The logs I am getting from the server:

llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = nomic-bert
llama_model_loader: - kv   1:                               general.name str              = nomic-embed-text-v1.5
llama_model_loader: - kv   2:                     nomic-bert.block_count u32              = 12
llama_model_loader: - kv   3:                  nomic-bert.context_length u32              = 2048
llama_model_loader: - kv   4:                nomic-bert.embedding_length u32              = 768
llama_model_loader: - kv   5:             nomic-bert.feed_forward_length u32              = 3072
llama_model_loader: - kv   6:            nomic-bert.attention.head_count u32              = 12
llama_model_loader: - kv   7:    nomic-bert.attention.layer_norm_epsilon f32              = 0.000000
llama_model_loader: - kv   8:                          general.file_type u32              = 1
llama_model_loader: - kv   9:                nomic-bert.attention.causal bool             = false
llama_model_loader: - kv  10:                    nomic-bert.pooling_type u32              = 1
llama_model_loader: - kv  11:                  nomic-bert.rope.freq_base f32              = 1000.000000
llama_model_loader: - kv  12:            tokenizer.ggml.token_type_count u32              = 2
llama_model_loader: - kv  13:                tokenizer.ggml.bos_token_id u32              = 101
llama_model_loader: - kv  14:                tokenizer.ggml.eos_token_id u32              = 102
llama_model_loader: - kv  15:                       tokenizer.ggml.model str              = bert
llama_model_loader: - kv  16:                      tokenizer.ggml.tokens arr[str,30522]   = ["[PAD]", "[unused0]", "[unused1]", "...
llama_model_loader: - kv  17:                      tokenizer.ggml.scores arr[f32,30522]   = [-1000.000000, -1000.000000, -1000.00...
llama_model_loader: - kv  18:                  tokenizer.ggml.token_type arr[i32,30522]   = [3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv  19:            tokenizer.ggml.unknown_token_id u32              = 100
llama_model_loader: - kv  20:          tokenizer.ggml.seperator_token_id u32              = 102
llama_model_loader: - kv  21:            tokenizer.ggml.padding_token_id u32              = 0
llama_model_loader: - kv  22:                tokenizer.ggml.cls_token_id u32              = 101
llama_model_loader: - kv  23:               tokenizer.ggml.mask_token_id u32              = 103
llama_model_loader: - type  f32:   51 tensors
llama_model_loader: - type  f16:   61 tensors
llm_load_vocab: mismatch in special tokens definition ( 7104/30522 vs 5/30522 ).
llm_load_print_meta: format           = GGUF V3 (latest)
llm_load_print_meta: arch             = nomic-bert
llm_load_print_meta: vocab type       = WPM
llm_load_print_meta: n_vocab          = 30522
llm_load_print_meta: n_merges         = 0
llm_load_print_meta: n_ctx_train      = 2048
llm_load_print_meta: n_embd           = 768
llm_load_print_meta: n_head           = 12
llm_load_print_meta: n_head_kv        = 12
llm_load_print_meta: n_layer          = 12
llm_load_print_meta: n_rot            = 64
llm_load_print_meta: n_embd_head_k    = 64
llm_load_print_meta: n_embd_head_v    = 64
llm_load_print_meta: n_gqa            = 1
llm_load_print_meta: n_embd_k_gqa     = 768
llm_load_print_meta: n_embd_v_gqa     = 768
llm_load_print_meta: f_norm_eps       = 1.0e-12
llm_load_print_meta: f_norm_rms_eps   = 0.0e+00
llm_load_print_meta: f_clamp_kqv      = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: n_ff             = 3072
llm_load_print_meta: n_expert         = 0
llm_load_print_meta: n_expert_used    = 0
llm_load_print_meta: pooling type     = 1
llm_load_print_meta: rope type        = 2
llm_load_print_meta: rope scaling     = linear
llm_load_print_meta: freq_base_train  = 1000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: n_yarn_orig_ctx  = 2048
llm_load_print_meta: rope_finetuned   = unknown
llm_load_print_meta: model type       = 137M
llm_load_print_meta: model ftype      = F16
llm_load_print_meta: model params     = 136.73 M
llm_load_print_meta: model size       = 260.86 MiB (16.00 BPW) 
llm_load_print_meta: general.name     = nomic-embed-text-v1.5
llm_load_print_meta: BOS token        = 101 '[CLS]'
llm_load_print_meta: EOS token        = 102 '[SEP]'
llm_load_print_meta: UNK token        = 100 '[UNK]'
llm_load_print_meta: SEP token        = 102 '[SEP]'
llm_load_print_meta: PAD token        = 0 '[PAD]'
llm_load_tensors: ggml ctx size =    0.09 MiB
llm_load_tensors: offloading 12 repeating layers to GPU
llm_load_tensors: offloading non-repeating layers to GPU
llm_load_tensors: offloaded 13/13 layers to GPU
llm_load_tensors:        CPU buffer size =    44.72 MiB
llm_load_tensors:      CUDA0 buffer size =   216.15 MiB
.......................................................
llama_new_context_with_model: n_ctx      = 2048
llama_new_context_with_model: freq_base  = 1000.0
llama_new_context_with_model: freq_scale = 1
ggml_init_cublas: GGML_CUDA_FORCE_MMQ:   yes
ggml_init_cublas: CUDA_USE_TENSOR_CORES: no
ggml_init_cublas: found 1 CUDA devices:
  Device 0: Tesla T4, compute capability 7.5, VMM: yes
llama_kv_cache_init:      CUDA0 KV buffer size =    72.00 MiB
llama_new_context_with_model: KV self size  =   72.00 MiB, K (f16):   36.00 MiB, V (f16):   36.00 MiB
llama_new_context_with_model:  CUDA_Host input buffer size   =     6.52 MiB
llama_new_context_with_model:      CUDA0 compute buffer size =    62.00 MiB
llama_new_context_with_model:  CUDA_Host compute buffer size =     1.50 MiB
llama_new_context_with_model: graph splits (measure): 2
{"function":"initialize","level":"INFO","line":433,"msg":"initializing slots","n_slots":1,"tid":"140541573117696","timestamp":1710020724}
{"function":"initialize","level":"INFO","line":442,"msg":"new slot","n_ctx_slot":2048,"slot_id":0,"tid":"140541573117696","timestamp":1710020724}
time=2024-03-09T21:45:24.886Z level=INFO source=dyn_ext_server.go:161 msg="Starting llama main loop"
{"function":"update_slots","level":"INFO","line":1565,"msg":"all slots are idle and system prompt is empty, clear the KV cache","tid":"140539684976384","timestamp":1710020724}
{"function":"launch_slot_with_data","level":"INFO","line":823,"msg":"slot is processing task","slot_id":0,"task_id":0,"tid":"140539684976384","timestamp":1710020724}

It is also important to add that this was running fine in my local development (CPU only, 4GB RAM).

Originally created by @matthewsmorrison on GitHub (Mar 9, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/3029 Originally assigned to: @jmorganca on GitHub. I am running Ollama (0.1.28) on a Google Cloud VM (n1-standard-2, Intel Broadwell, NVIDIA T4 GPU, 7.5GB RAM). When I run the cURL command for the embeddings API with the nomic-embed-text model (version: nomic-embed-text:latest 0a109f422b47 ) ``` curl http://localhost:11434/api/embeddings -d '{ "model": "nomic-embed-text", "prompt": "Here is an article about llamas..." }' ``` Ollama indefinitely hangs. The logs I am getting from the server: ``` llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. llama_model_loader: - kv 0: general.architecture str = nomic-bert llama_model_loader: - kv 1: general.name str = nomic-embed-text-v1.5 llama_model_loader: - kv 2: nomic-bert.block_count u32 = 12 llama_model_loader: - kv 3: nomic-bert.context_length u32 = 2048 llama_model_loader: - kv 4: nomic-bert.embedding_length u32 = 768 llama_model_loader: - kv 5: nomic-bert.feed_forward_length u32 = 3072 llama_model_loader: - kv 6: nomic-bert.attention.head_count u32 = 12 llama_model_loader: - kv 7: nomic-bert.attention.layer_norm_epsilon f32 = 0.000000 llama_model_loader: - kv 8: general.file_type u32 = 1 llama_model_loader: - kv 9: nomic-bert.attention.causal bool = false llama_model_loader: - kv 10: nomic-bert.pooling_type u32 = 1 llama_model_loader: - kv 11: nomic-bert.rope.freq_base f32 = 1000.000000 llama_model_loader: - kv 12: tokenizer.ggml.token_type_count u32 = 2 llama_model_loader: - kv 13: tokenizer.ggml.bos_token_id u32 = 101 llama_model_loader: - kv 14: tokenizer.ggml.eos_token_id u32 = 102 llama_model_loader: - kv 15: tokenizer.ggml.model str = bert llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,30522] = ["[PAD]", "[unused0]", "[unused1]", "... llama_model_loader: - kv 17: tokenizer.ggml.scores arr[f32,30522] = [-1000.000000, -1000.000000, -1000.00... llama_model_loader: - kv 18: tokenizer.ggml.token_type arr[i32,30522] = [3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... llama_model_loader: - kv 19: tokenizer.ggml.unknown_token_id u32 = 100 llama_model_loader: - kv 20: tokenizer.ggml.seperator_token_id u32 = 102 llama_model_loader: - kv 21: tokenizer.ggml.padding_token_id u32 = 0 llama_model_loader: - kv 22: tokenizer.ggml.cls_token_id u32 = 101 llama_model_loader: - kv 23: tokenizer.ggml.mask_token_id u32 = 103 llama_model_loader: - type f32: 51 tensors llama_model_loader: - type f16: 61 tensors llm_load_vocab: mismatch in special tokens definition ( 7104/30522 vs 5/30522 ). llm_load_print_meta: format = GGUF V3 (latest) llm_load_print_meta: arch = nomic-bert llm_load_print_meta: vocab type = WPM llm_load_print_meta: n_vocab = 30522 llm_load_print_meta: n_merges = 0 llm_load_print_meta: n_ctx_train = 2048 llm_load_print_meta: n_embd = 768 llm_load_print_meta: n_head = 12 llm_load_print_meta: n_head_kv = 12 llm_load_print_meta: n_layer = 12 llm_load_print_meta: n_rot = 64 llm_load_print_meta: n_embd_head_k = 64 llm_load_print_meta: n_embd_head_v = 64 llm_load_print_meta: n_gqa = 1 llm_load_print_meta: n_embd_k_gqa = 768 llm_load_print_meta: n_embd_v_gqa = 768 llm_load_print_meta: f_norm_eps = 1.0e-12 llm_load_print_meta: f_norm_rms_eps = 0.0e+00 llm_load_print_meta: f_clamp_kqv = 0.0e+00 llm_load_print_meta: f_max_alibi_bias = 0.0e+00 llm_load_print_meta: n_ff = 3072 llm_load_print_meta: n_expert = 0 llm_load_print_meta: n_expert_used = 0 llm_load_print_meta: pooling type = 1 llm_load_print_meta: rope type = 2 llm_load_print_meta: rope scaling = linear llm_load_print_meta: freq_base_train = 1000.0 llm_load_print_meta: freq_scale_train = 1 llm_load_print_meta: n_yarn_orig_ctx = 2048 llm_load_print_meta: rope_finetuned = unknown llm_load_print_meta: model type = 137M llm_load_print_meta: model ftype = F16 llm_load_print_meta: model params = 136.73 M llm_load_print_meta: model size = 260.86 MiB (16.00 BPW) llm_load_print_meta: general.name = nomic-embed-text-v1.5 llm_load_print_meta: BOS token = 101 '[CLS]' llm_load_print_meta: EOS token = 102 '[SEP]' llm_load_print_meta: UNK token = 100 '[UNK]' llm_load_print_meta: SEP token = 102 '[SEP]' llm_load_print_meta: PAD token = 0 '[PAD]' llm_load_tensors: ggml ctx size = 0.09 MiB llm_load_tensors: offloading 12 repeating layers to GPU llm_load_tensors: offloading non-repeating layers to GPU llm_load_tensors: offloaded 13/13 layers to GPU llm_load_tensors: CPU buffer size = 44.72 MiB llm_load_tensors: CUDA0 buffer size = 216.15 MiB ....................................................... llama_new_context_with_model: n_ctx = 2048 llama_new_context_with_model: freq_base = 1000.0 llama_new_context_with_model: freq_scale = 1 ggml_init_cublas: GGML_CUDA_FORCE_MMQ: yes ggml_init_cublas: CUDA_USE_TENSOR_CORES: no ggml_init_cublas: found 1 CUDA devices: Device 0: Tesla T4, compute capability 7.5, VMM: yes llama_kv_cache_init: CUDA0 KV buffer size = 72.00 MiB llama_new_context_with_model: KV self size = 72.00 MiB, K (f16): 36.00 MiB, V (f16): 36.00 MiB llama_new_context_with_model: CUDA_Host input buffer size = 6.52 MiB llama_new_context_with_model: CUDA0 compute buffer size = 62.00 MiB llama_new_context_with_model: CUDA_Host compute buffer size = 1.50 MiB llama_new_context_with_model: graph splits (measure): 2 {"function":"initialize","level":"INFO","line":433,"msg":"initializing slots","n_slots":1,"tid":"140541573117696","timestamp":1710020724} {"function":"initialize","level":"INFO","line":442,"msg":"new slot","n_ctx_slot":2048,"slot_id":0,"tid":"140541573117696","timestamp":1710020724} time=2024-03-09T21:45:24.886Z level=INFO source=dyn_ext_server.go:161 msg="Starting llama main loop" {"function":"update_slots","level":"INFO","line":1565,"msg":"all slots are idle and system prompt is empty, clear the KV cache","tid":"140539684976384","timestamp":1710020724} {"function":"launch_slot_with_data","level":"INFO","line":823,"msg":"slot is processing task","slot_id":0,"task_id":0,"tid":"140539684976384","timestamp":1710020724} ``` It is also important to add that this was running fine in my local development (CPU only, 4GB RAM).
GiteaMirror added the bug label 2026-04-12 11:55:38 -05:00
GiteaMirror commented 2026-04-12 11:55:40 -05:00
Author
Owner
Copy Link

@imrohankataria commented on GitHub (Mar 10, 2024):

Came Looking for same, running in Huggingface Spaces.

<!-- gh-comment-id:1987030976 --> @imrohankataria commented on GitHub (Mar 10, 2024): Came Looking for same, running in Huggingface Spaces.
GiteaMirror commented 2026-04-12 11:55:41 -05:00
Author
Owner
Copy Link

@neozhu commented on GitHub (Mar 10, 2024):

same issue to me, run at CPU docker containers

<!-- gh-comment-id:1987042938 --> @neozhu commented on GitHub (Mar 10, 2024): same issue to me, run at CPU docker containers
GiteaMirror commented 2026-04-12 11:55:42 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (Mar 10, 2024):

Sorry about this, working on a fix now

<!-- gh-comment-id:1987063250 --> @jmorganca commented on GitHub (Mar 10, 2024): Sorry about this, working on a fix now
GiteaMirror commented 2026-04-12 11:55:42 -05:00
Author
Owner
Copy Link

@imrohankataria commented on GitHub (Mar 10, 2024):

Seems like this is with all the embedding models, same behavior with all-minilm:latest
Freezes complete instance

<!-- gh-comment-id:1987100981 --> @imrohankataria commented on GitHub (Mar 10, 2024): Seems like this is with all the embedding models, same behavior with all-minilm:latest Freezes complete instance
GiteaMirror commented 2026-04-12 11:55:44 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (Mar 10, 2024):

This has been fixed in 908005d and will be available in the next release: 0.1.29. Sorry again that this happened.

<!-- gh-comment-id:1987104843 --> @jmorganca commented on GitHub (Mar 10, 2024): This has been fixed in [908005d](https://github.com/ollama/ollama/commit/908005d90b670f68cb8ee67c60b7a3733d3bffab) and will be available in the next release: [0.1.29](https://github.com/ollama/ollama/releases/tag/v0.1.29). Sorry again that this happened.
GiteaMirror commented 2026-04-12 11:55:45 -05:00
Author
Owner
Copy Link

@matthewsmorrison commented on GitHub (Mar 10, 2024):

@jmorganca Thank you so much for the quick fix. Any idea when 0.1.29 will be released?

<!-- gh-comment-id:1987161784 --> @matthewsmorrison commented on GitHub (Mar 10, 2024): @jmorganca Thank you so much for the quick fix. Any idea when 0.1.29 will be released?
GiteaMirror commented 2026-04-12 11:55:46 -05:00
Author
Owner
Copy Link

@matthewsmorrison commented on GitHub (Mar 10, 2024):

For anyone looking at this issue and wishing to use 0.1.29 in its current form.

  1. Uninstall Ollama on Linux (guide here: https://github.com/ollama/ollama/blob/main/docs/linux.md)
  2. Install 0.1.29 specific version:
curl -fsSL https://ollama.com/install.sh | sed 's#https://ollama.com/download#https://github.com/jmorganca/ollama/releases/download/v0.1.29#' | sh
  1. Run everything else as before (pulling models etc)
<!-- gh-comment-id:1987177366 --> @matthewsmorrison commented on GitHub (Mar 10, 2024): For anyone looking at this issue and wishing to use 0.1.29 in its current form. 1. Uninstall Ollama on Linux (guide here: https://github.com/ollama/ollama/blob/main/docs/linux.md) 2. Install 0.1.29 specific version: ``` curl -fsSL https://ollama.com/install.sh | sed 's#https://ollama.com/download#https://github.com/jmorganca/ollama/releases/download/v0.1.29#' | sh ``` 3. Run everything else as before (pulling models etc)
GiteaMirror commented 2026-04-12 11:55:47 -05:00
Author
Owner
Copy Link

@weisun10 commented on GitHub (Mar 13, 2024):

if you use docker image, note ollama/ollama:0.1.29 dosen't yet include the fix. 0.1.27 works.

<!-- gh-comment-id:1993714512 --> @weisun10 commented on GitHub (Mar 13, 2024): if you use docker image, note ollama/ollama:0.1.**29** dosen't yet include the fix. 0.1.27 works.
GiteaMirror commented 2026-04-12 11:55:49 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (Mar 13, 2024):

Correction: was fixed in main and the 0.1.29 prerelease will be updated soon with the fix, sorry for the confusion!

<!-- gh-comment-id:1994506914 --> @jmorganca commented on GitHub (Mar 13, 2024): Correction: was fixed in main and the 0.1.29 prerelease will be updated soon with the fix, sorry for the confusion!
GiteaMirror commented 2026-04-12 11:55:49 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (Mar 13, 2024):

Hi folks, 0.1.29 has this fixed now and will be released shortly: https://github.com/ollama/ollama/releases/tag/v0.1.29

<!-- gh-comment-id:1996052411 --> @jmorganca commented on GitHub (Mar 13, 2024): Hi folks, 0.1.29 has this fixed now and will be released shortly: https://github.com/ollama/ollama/releases/tag/v0.1.29
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#1862
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?