github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-16 05:05:37 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #9365] Failure when embedded with num_ctx #6115

New Issue
Closed
opened 2026-04-12 17:27:10 -05:00 by GiteaMirror · 1 comment
GiteaMirror commented 2026-04-12 17:27:10 -05:00
Owner
Copy Link

Originally created by @Marsman1996 on GitHub (Feb 26, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/9365

What is the issue?

When using model mxbai-embed-large to embed specific strings, it will fail.

The command is:

curl -X POST "http://localhost:11434/api/embed" \
  -d '{
    "model": "mxbai-embed-large", 
    "input": "0    12    3\n0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1\n       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\n       |      0xA1     |      0xB2     |     0xC3      |     0xCB      |\n       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\n       |      'c'      |      'B'      |     'P'       |     'F'       |\n       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\n       |   MajorVer=1  |    MinorVer   |     Flags       |\n       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\n       |    SnapLen       |\n       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\n       | LinkTypeValue       |       InstructionCount=n      |\n       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\n       |       |\n       | instruction 1       |\n       |       |\n       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\n       |       |\n       | instruction 2       |",
    "options":{"num_ctx": 40960}
  }'

However, if we remove the num_ctx option, it works fine.

Is it because of the large num_ctx value?
The log shows n_ctx_pre_seq (40960) > n_ctx_train (512) -- possible training context overflow

Relevant log output

Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.255+08:00 level=WARN source=sched.go:647 msg="gpu VRAM usage didn't recover within timeout" seconds=5.207563637 model=/usr/share/ollama/.ollama/models/blobs/sha256-819c2adf5ce6df2b6bd2ae4ca90d2a69f060afeb438d0c171db57daa02e39c3d
Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.504+08:00 level=WARN source=sched.go:647 msg="gpu VRAM usage didn't recover within timeout" seconds=5.457031768 model=/usr/share/ollama/.ollama/models/blobs/sha256-819c2adf5ce6df2b6bd2ae4ca90d2a69f060afeb438d0c171db57daa02e39c3d
Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.638+08:00 level=WARN source=ggml.go:132 msg="key not found" key=bert.attention.head_count_kv default=1
Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.638+08:00 level=WARN source=ggml.go:132 msg="key not found" key=bert.attention.key_length default=64
Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.638+08:00 level=WARN source=ggml.go:132 msg="key not found" key=bert.attention.value_length default=64
Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.639+08:00 level=WARN source=ggml.go:132 msg="key not found" key=bert.attention.head_count_kv default=1
Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.639+08:00 level=INFO source=sched.go:715 msg="new model will fit in available VRAM in single GPU, loading" model=/usr/share/ollama/.ollama/models/blobs/sha256-819c2adf5ce6df2b6bd2ae4ca90d2a69f060afeb438d0c171db57daa02e39c3d gpu=GPU-5e25c970-090c-8d60-a8f4-5983718231b2 parallel=1 available=49331830784 required="2.0 GiB"
Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.812+08:00 level=WARN source=sched.go:647 msg="gpu VRAM usage didn't recover within timeout" seconds=5.765118113 model=/usr/share/ollama/.ollama/models/blobs/sha256-819c2adf5ce6df2b6bd2ae4ca90d2a69f060afeb438d0c171db57daa02e39c3d
Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.995+08:00 level=INFO source=server.go:97 msg="system memory" total="125.5 GiB" free="90.9 GiB" free_swap="7.3 GiB"
Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.995+08:00 level=WARN source=ggml.go:132 msg="key not found" key=bert.attention.head_count_kv default=1
Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.995+08:00 level=WARN source=ggml.go:132 msg="key not found" key=bert.attention.key_length default=64
Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.995+08:00 level=WARN source=ggml.go:132 msg="key not found" key=bert.attention.value_length default=64
Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.995+08:00 level=WARN source=ggml.go:132 msg="key not found" key=bert.attention.head_count_kv default=1
Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.995+08:00 level=INFO source=server.go:130 msg=offload library=cuda layers.requested=-1 layers.model=25 layers.offload=25 layers.split="" memory.available="[45.9 GiB]" memory.gpu_overhead="0 B" memory.required.full="2.0 GiB" memory.required.partial="2.0 GiB" memory.required.kv="240.0 MiB" memory.required.allocations="[2.0 GiB]" memory.weights.total="817.2 MiB" memory.weights.repeating="757.6 MiB" memory.weights.nonrepeating="59.6 MiB" memory.graph.full="640.0 MiB" memory.graph.partial="640.0 MiB"
Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.995+08:00 level=INFO source=server.go:380 msg="starting llama server" cmd="/usr/local/bin/ollama runner --model /usr/share/ollama/.ollama/models/blobs/sha256-819c2adf5ce6df2b6bd2ae4ca90d2a69f060afeb438d0c171db57daa02e39c3d --ctx-size 40960 --batch-size 512 --n-gpu-layers 25 --threads 52 --parallel 1 --port 40221"
Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.996+08:00 level=INFO source=sched.go:450 msg="loaded runners" count=1
Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.996+08:00 level=INFO source=server.go:557 msg="waiting for llama runner to start responding"
Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.996+08:00 level=INFO source=server.go:591 msg="waiting for server to become available" status="llm server error"
Feb 26 23:30:20 ps ollama[908767]: time=2025-02-26T23:30:20.012+08:00 level=INFO source=runner.go:932 msg="starting go runner"
Feb 26 23:30:20 ps ollama[908767]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
Feb 26 23:30:20 ps ollama[908767]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
Feb 26 23:30:20 ps ollama[908767]: ggml_cuda_init: found 1 CUDA devices:
Feb 26 23:30:20 ps ollama[908767]:   Device 0: NVIDIA RTX A6000, compute capability 8.6, VMM: yes
Feb 26 23:30:20 ps ollama[908767]: load_backend: loaded CUDA backend from /usr/local/lib/ollama/cuda_v12/libggml-cuda.so
Feb 26 23:30:20 ps ollama[908767]: load_backend: loaded CPU backend from /usr/local/lib/ollama/libggml-cpu-skylakex.so
Feb 26 23:30:20 ps ollama[908767]: time=2025-02-26T23:30:20.071+08:00 level=INFO source=runner.go:935 msg=system info="CPU : LLAMAFILE = 1 | CPU : LLAMAFILE = 1 | CUDA : ARCHS = 600,610,620,700,720,750,800,860,870,890,900 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | AVX512 = 1 | LLAMAFILE = 1 | cgo(gcc)" threads=52
Feb 26 23:30:20 ps ollama[908767]: time=2025-02-26T23:30:20.071+08:00 level=INFO source=runner.go:993 msg="Server listening on 127.0.0.1:40221"
Feb 26 23:30:20 ps ollama[908767]: llama_load_model_from_file: using device CUDA0 (NVIDIA RTX A6000) - 47046 MiB free
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: loaded meta data with 23 key-value pairs and 389 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-819c2adf5ce6df2b6bd2ae4ca90d2a69f060afeb438d0c171db57daa02e39c3d (version GGUF V3 (latest))
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv   0:                       general.architecture str              = bert
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv   1:                               general.name str              = mxbai-embed-large-v1
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv   2:                           bert.block_count u32              = 24
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv   3:                        bert.context_length u32              = 512
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv   4:                      bert.embedding_length u32              = 1024
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv   5:                   bert.feed_forward_length u32              = 4096
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv   6:                  bert.attention.head_count u32              = 16
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv   7:          bert.attention.layer_norm_epsilon f32              = 0.000000
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv   8:                          general.file_type u32              = 1
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv   9:                      bert.attention.causal bool             = false
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv  10:                          bert.pooling_type u32              = 2
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv  11:            tokenizer.ggml.token_type_count u32              = 2
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv  12:                tokenizer.ggml.bos_token_id u32              = 101
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv  13:                tokenizer.ggml.eos_token_id u32              = 102
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv  14:                       tokenizer.ggml.model str              = bert
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv  15:                      tokenizer.ggml.tokens arr[str,30522]   = ["[PAD]", "[unused0]", "[unused1]", "...
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv  16:                      tokenizer.ggml.scores arr[f32,30522]   = [-1000.000000, -1000.000000, -1000.00...
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv  17:                  tokenizer.ggml.token_type arr[i32,30522]   = [3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv  18:            tokenizer.ggml.unknown_token_id u32              = 100
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv  19:          tokenizer.ggml.seperator_token_id u32              = 102
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv  20:            tokenizer.ggml.padding_token_id u32              = 0
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv  21:                tokenizer.ggml.cls_token_id u32              = 101
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv  22:               tokenizer.ggml.mask_token_id u32              = 103
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - type  f32:  243 tensors
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - type  f16:  146 tensors
Feb 26 23:30:20 ps ollama[908767]: llm_load_vocab: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
Feb 26 23:30:20 ps ollama[908767]: llm_load_vocab: special tokens cache size = 5
Feb 26 23:30:20 ps ollama[908767]: llm_load_vocab: token to piece cache size = 0.2032 MB
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: format           = GGUF V3 (latest)
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: arch             = bert
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: vocab type       = WPM
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_vocab          = 30522
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_merges         = 0
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: vocab_only       = 0
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_ctx_train      = 512
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_embd           = 1024
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_layer          = 24
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_head           = 16
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_head_kv        = 16
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_rot            = 64
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_swa            = 0
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_embd_head_k    = 64
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_embd_head_v    = 64
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_gqa            = 1
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_embd_k_gqa     = 1024
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_embd_v_gqa     = 1024
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: f_norm_eps       = 1.0e-12
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: f_norm_rms_eps   = 0.0e+00
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: f_clamp_kqv      = 0.0e+00
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: f_max_alibi_bias = 0.0e+00
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: f_logit_scale    = 0.0e+00
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_ff             = 4096
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_expert         = 0
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_expert_used    = 0
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: causal attn      = 0
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: pooling type     = 2
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: rope type        = 2
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: rope scaling     = linear
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: freq_base_train  = 10000.0
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: freq_scale_train = 1
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_ctx_orig_yarn  = 512
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: rope_finetuned   = unknown
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: ssm_d_conv       = 0
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: ssm_d_inner      = 0
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: ssm_d_state      = 0
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: ssm_dt_rank      = 0
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: ssm_dt_b_c_rms   = 0
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: model type       = 335M
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: model ftype      = F16
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: model params     = 334.09 M
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: model size       = 637.85 MiB (16.02 BPW)
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: general.name     = mxbai-embed-large-v1
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: BOS token        = 101 '[CLS]'
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: EOS token        = 102 '[SEP]'
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: UNK token        = 100 '[UNK]'
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: SEP token        = 102 '[SEP]'
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: PAD token        = 0 '[PAD]'
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: CLS token        = 101 '[CLS]'
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: MASK token       = 103 '[MASK]'
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: LF token         = 0 '[PAD]'
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: EOG token        = 102 '[SEP]'
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: max token length = 21
Feb 26 23:30:20 ps ollama[908767]: time=2025-02-26T23:30:20.247+08:00 level=INFO source=server.go:591 msg="waiting for server to become available" status="llm server loading model"
Feb 26 23:30:20 ps ollama[908767]: llm_load_tensors: offloading 24 repeating layers to GPU
Feb 26 23:30:20 ps ollama[908767]: llm_load_tensors: offloading output layer to GPU
Feb 26 23:30:20 ps ollama[908767]: llm_load_tensors: offloaded 25/25 layers to GPU
Feb 26 23:30:20 ps ollama[908767]: llm_load_tensors:        CUDA0 model buffer size =   577.22 MiB
Feb 26 23:30:20 ps ollama[908767]: llm_load_tensors:   CPU_Mapped model buffer size =    60.63 MiB
Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: n_seq_max     = 1
Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: n_ctx         = 40960
Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: n_ctx_per_seq = 40960
Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: n_batch       = 512
Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: n_ubatch      = 512
Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: flash_attn    = 0
Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: freq_base     = 10000.0
Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: freq_scale    = 1
Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: n_ctx_pre_seq (40960) > n_ctx_train (512) -- possible training context overflow
Feb 26 23:30:20 ps ollama[908767]: llama_kv_cache_init: kv_size = 40960, offload = 1, type_k = 'f16', type_v = 'f16', n_layer = 24, can_shift = 1
Feb 26 23:30:20 ps ollama[908767]: llama_kv_cache_init:      CUDA0 KV buffer size =  3840.00 MiB
Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: KV self size  = 3840.00 MiB, K (f16): 1920.00 MiB, V (f16): 1920.00 MiB
Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model:  CUDA_Host  output buffer size =     0.00 MiB
Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model:      CUDA0 compute buffer size =    25.01 MiB
Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model:  CUDA_Host compute buffer size =     5.01 MiB
Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: graph nodes  = 849
Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: graph splits = 4 (with bs=512), 2 (with bs=1)
Feb 26 23:30:20 ps ollama[908767]: time=2025-02-26T23:30:20.498+08:00 level=INFO source=server.go:596 msg="llama runner started in 0.50 seconds"
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: loaded meta data with 23 key-value pairs and 389 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-819c2adf5ce6df2b6bd2ae4ca90d2a69f060afeb438d0c171db57daa02e39c3d (version GGUF V3 (latest))
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv   0:                       general.architecture str              = bert
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv   1:                               general.name str              = mxbai-embed-large-v1
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv   2:                           bert.block_count u32              = 24
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv   3:                        bert.context_length u32              = 512
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv   4:                      bert.embedding_length u32              = 1024
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv   5:                   bert.feed_forward_length u32              = 4096
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv   6:                  bert.attention.head_count u32              = 16
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv   7:          bert.attention.layer_norm_epsilon f32              = 0.000000
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv   8:                          general.file_type u32              = 1
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv   9:                      bert.attention.causal bool             = false
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv  10:                          bert.pooling_type u32              = 2
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv  11:            tokenizer.ggml.token_type_count u32              = 2
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv  12:                tokenizer.ggml.bos_token_id u32              = 101
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv  13:                tokenizer.ggml.eos_token_id u32              = 102
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv  14:                       tokenizer.ggml.model str              = bert
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv  15:                      tokenizer.ggml.tokens arr[str,30522]   = ["[PAD]", "[unused0]", "[unused1]", "...
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv  16:                      tokenizer.ggml.scores arr[f32,30522]   = [-1000.000000, -1000.000000, -1000.00...
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv  17:                  tokenizer.ggml.token_type arr[i32,30522]   = [3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv  18:            tokenizer.ggml.unknown_token_id u32              = 100
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv  19:          tokenizer.ggml.seperator_token_id u32              = 102
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv  20:            tokenizer.ggml.padding_token_id u32              = 0
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv  21:                tokenizer.ggml.cls_token_id u32              = 101
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv  22:               tokenizer.ggml.mask_token_id u32              = 103
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - type  f32:  243 tensors
Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - type  f16:  146 tensors
Feb 26 23:30:20 ps ollama[908767]: llm_load_vocab: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
Feb 26 23:30:20 ps ollama[908767]: llm_load_vocab: special tokens cache size = 5
Feb 26 23:30:20 ps ollama[908767]: llm_load_vocab: token to piece cache size = 0.2032 MB
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: format           = GGUF V3 (latest)
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: arch             = bert
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: vocab type       = WPM
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_vocab          = 30522
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_merges         = 0
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: vocab_only       = 1
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: model type       = ?B
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: model ftype      = all F32
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: model params     = 334.09 M
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: model size       = 637.85 MiB (16.02 BPW)
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: general.name     = mxbai-embed-large-v1
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: BOS token        = 101 '[CLS]'
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: EOS token        = 102 '[SEP]'
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: UNK token        = 100 '[UNK]'
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: SEP token        = 102 '[SEP]'
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: PAD token        = 0 '[PAD]'
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: CLS token        = 101 '[CLS]'
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: MASK token       = 103 '[MASK]'
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: LF token         = 0 '[PAD]'
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: EOG token        = 102 '[SEP]'
Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: max token length = 21
Feb 26 23:30:20 ps ollama[908767]: llama_model_load: vocab only - skipping tensors
Feb 26 23:30:20 ps ollama[908767]: //ml/backend/ggml/ggml/src/ggml-cpu/ggml-cpu.c:8374: GGML_ASSERT(i01 >= 0 && i01 < ne01) failed
Feb 26 23:30:20 ps ollama[908767]: //ml/backend/ggml/ggml/src/ggml-cpu/ggml-cpu.c:8374: GGML_ASSERT(i01 >= 0 && i01 < ne01) failed
Feb 26 23:30:20 ps ollama[1363647]: Could not attach to process.  If your uid matches the uid of the target
Feb 26 23:30:20 ps ollama[1363647]: process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or try
Feb 26 23:30:20 ps ollama[1363647]: again as the root user.  For more details, see /etc/sysctl.d/10-ptrace.conf
Feb 26 23:30:20 ps ollama[908767]: ptrace: Inappropriate ioctl for device.
Feb 26 23:30:20 ps ollama[908767]: No stack.
Feb 26 23:30:20 ps ollama[908767]: The program is not being run.
Feb 26 23:30:20 ps ollama[1363642]: Could not attach to process.  If your uid matches the uid of the target
Feb 26 23:30:20 ps ollama[1363642]: process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or try
Feb 26 23:30:20 ps ollama[1363642]: again as the root user.  For more details, see /etc/sysctl.d/10-ptrace.conf
Feb 26 23:30:20 ps ollama[908767]: warning: process 1363105 is a zombie - the process has already terminated
Feb 26 23:30:20 ps ollama[908767]: ptrace: Inappropriate ioctl for device.
Feb 26 23:30:20 ps ollama[908767]: No stack.
Feb 26 23:30:20 ps ollama[908767]: The program is not being run.
Feb 26 23:30:20 ps ollama[908767]: time=2025-02-26T23:30:20.876+08:00 level=ERROR source=routes.go:478 msg="embedding generation failed" error="do embedding request: Post \"http://127.0.0.1:40221/embedding\": EOF"
Feb 26 23:30:20 ps ollama[908767]: [GIN] 2025/02/26 - 23:30:20 | 500 |   6.83067669s |             ::1 | POST     "/api/embed"
Feb 26 23:30:20 ps ollama[908767]: time=2025-02-26T23:30:20.941+08:00 level=ERROR source=server.go:421 msg="llama runner terminated" error="signal: aborted"

OS

Linux

GPU

Nvidia

CPU

Intel

Ollama version

0.5.12

Originally created by @Marsman1996 on GitHub (Feb 26, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/9365 ### What is the issue? When using model `mxbai-embed-large` to embed specific strings, it will fail. The command is: ```bash curl -X POST "http://localhost:11434/api/embed" \ -d '{ "model": "mxbai-embed-large", "input": "0 12 3\n0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1\n +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\n | 0xA1 | 0xB2 | 0xC3 | 0xCB |\n +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\n | 'c' | 'B' | 'P' | 'F' |\n +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\n | MajorVer=1 | MinorVer | Flags |\n +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\n | SnapLen |\n +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\n | LinkTypeValue | InstructionCount=n |\n +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\n | |\n | instruction 1 |\n | |\n +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\n | |\n | instruction 2 |", "options":{"num_ctx": 40960} }' ``` However, if we remove the `num_ctx` option, it works fine. --- Is it because of the large `num_ctx` value? The log shows `n_ctx_pre_seq (40960) > n_ctx_train (512) -- possible training context overflow` ### Relevant log output ```shell Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.255+08:00 level=WARN source=sched.go:647 msg="gpu VRAM usage didn't recover within timeout" seconds=5.207563637 model=/usr/share/ollama/.ollama/models/blobs/sha256-819c2adf5ce6df2b6bd2ae4ca90d2a69f060afeb438d0c171db57daa02e39c3d Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.504+08:00 level=WARN source=sched.go:647 msg="gpu VRAM usage didn't recover within timeout" seconds=5.457031768 model=/usr/share/ollama/.ollama/models/blobs/sha256-819c2adf5ce6df2b6bd2ae4ca90d2a69f060afeb438d0c171db57daa02e39c3d Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.638+08:00 level=WARN source=ggml.go:132 msg="key not found" key=bert.attention.head_count_kv default=1 Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.638+08:00 level=WARN source=ggml.go:132 msg="key not found" key=bert.attention.key_length default=64 Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.638+08:00 level=WARN source=ggml.go:132 msg="key not found" key=bert.attention.value_length default=64 Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.639+08:00 level=WARN source=ggml.go:132 msg="key not found" key=bert.attention.head_count_kv default=1 Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.639+08:00 level=INFO source=sched.go:715 msg="new model will fit in available VRAM in single GPU, loading" model=/usr/share/ollama/.ollama/models/blobs/sha256-819c2adf5ce6df2b6bd2ae4ca90d2a69f060afeb438d0c171db57daa02e39c3d gpu=GPU-5e25c970-090c-8d60-a8f4-5983718231b2 parallel=1 available=49331830784 required="2.0 GiB" Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.812+08:00 level=WARN source=sched.go:647 msg="gpu VRAM usage didn't recover within timeout" seconds=5.765118113 model=/usr/share/ollama/.ollama/models/blobs/sha256-819c2adf5ce6df2b6bd2ae4ca90d2a69f060afeb438d0c171db57daa02e39c3d Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.995+08:00 level=INFO source=server.go:97 msg="system memory" total="125.5 GiB" free="90.9 GiB" free_swap="7.3 GiB" Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.995+08:00 level=WARN source=ggml.go:132 msg="key not found" key=bert.attention.head_count_kv default=1 Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.995+08:00 level=WARN source=ggml.go:132 msg="key not found" key=bert.attention.key_length default=64 Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.995+08:00 level=WARN source=ggml.go:132 msg="key not found" key=bert.attention.value_length default=64 Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.995+08:00 level=WARN source=ggml.go:132 msg="key not found" key=bert.attention.head_count_kv default=1 Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.995+08:00 level=INFO source=server.go:130 msg=offload library=cuda layers.requested=-1 layers.model=25 layers.offload=25 layers.split="" memory.available="[45.9 GiB]" memory.gpu_overhead="0 B" memory.required.full="2.0 GiB" memory.required.partial="2.0 GiB" memory.required.kv="240.0 MiB" memory.required.allocations="[2.0 GiB]" memory.weights.total="817.2 MiB" memory.weights.repeating="757.6 MiB" memory.weights.nonrepeating="59.6 MiB" memory.graph.full="640.0 MiB" memory.graph.partial="640.0 MiB" Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.995+08:00 level=INFO source=server.go:380 msg="starting llama server" cmd="/usr/local/bin/ollama runner --model /usr/share/ollama/.ollama/models/blobs/sha256-819c2adf5ce6df2b6bd2ae4ca90d2a69f060afeb438d0c171db57daa02e39c3d --ctx-size 40960 --batch-size 512 --n-gpu-layers 25 --threads 52 --parallel 1 --port 40221" Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.996+08:00 level=INFO source=sched.go:450 msg="loaded runners" count=1 Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.996+08:00 level=INFO source=server.go:557 msg="waiting for llama runner to start responding" Feb 26 23:30:19 ps ollama[908767]: time=2025-02-26T23:30:19.996+08:00 level=INFO source=server.go:591 msg="waiting for server to become available" status="llm server error" Feb 26 23:30:20 ps ollama[908767]: time=2025-02-26T23:30:20.012+08:00 level=INFO source=runner.go:932 msg="starting go runner" Feb 26 23:30:20 ps ollama[908767]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no Feb 26 23:30:20 ps ollama[908767]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no Feb 26 23:30:20 ps ollama[908767]: ggml_cuda_init: found 1 CUDA devices: Feb 26 23:30:20 ps ollama[908767]: Device 0: NVIDIA RTX A6000, compute capability 8.6, VMM: yes Feb 26 23:30:20 ps ollama[908767]: load_backend: loaded CUDA backend from /usr/local/lib/ollama/cuda_v12/libggml-cuda.so Feb 26 23:30:20 ps ollama[908767]: load_backend: loaded CPU backend from /usr/local/lib/ollama/libggml-cpu-skylakex.so Feb 26 23:30:20 ps ollama[908767]: time=2025-02-26T23:30:20.071+08:00 level=INFO source=runner.go:935 msg=system info="CPU : LLAMAFILE = 1 | CPU : LLAMAFILE = 1 | CUDA : ARCHS = 600,610,620,700,720,750,800,860,870,890,900 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | AVX512 = 1 | LLAMAFILE = 1 | cgo(gcc)" threads=52 Feb 26 23:30:20 ps ollama[908767]: time=2025-02-26T23:30:20.071+08:00 level=INFO source=runner.go:993 msg="Server listening on 127.0.0.1:40221" Feb 26 23:30:20 ps ollama[908767]: llama_load_model_from_file: using device CUDA0 (NVIDIA RTX A6000) - 47046 MiB free Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: loaded meta data with 23 key-value pairs and 389 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-819c2adf5ce6df2b6bd2ae4ca90d2a69f060afeb438d0c171db57daa02e39c3d (version GGUF V3 (latest)) Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 0: general.architecture str = bert Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 1: general.name str = mxbai-embed-large-v1 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 2: bert.block_count u32 = 24 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 3: bert.context_length u32 = 512 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 4: bert.embedding_length u32 = 1024 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 5: bert.feed_forward_length u32 = 4096 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 6: bert.attention.head_count u32 = 16 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 7: bert.attention.layer_norm_epsilon f32 = 0.000000 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 8: general.file_type u32 = 1 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 9: bert.attention.causal bool = false Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 10: bert.pooling_type u32 = 2 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 11: tokenizer.ggml.token_type_count u32 = 2 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 12: tokenizer.ggml.bos_token_id u32 = 101 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 13: tokenizer.ggml.eos_token_id u32 = 102 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 14: tokenizer.ggml.model str = bert Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 15: tokenizer.ggml.tokens arr[str,30522] = ["[PAD]", "[unused0]", "[unused1]", "... Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 16: tokenizer.ggml.scores arr[f32,30522] = [-1000.000000, -1000.000000, -1000.00... Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,30522] = [3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 18: tokenizer.ggml.unknown_token_id u32 = 100 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 19: tokenizer.ggml.seperator_token_id u32 = 102 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 20: tokenizer.ggml.padding_token_id u32 = 0 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 21: tokenizer.ggml.cls_token_id u32 = 101 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 22: tokenizer.ggml.mask_token_id u32 = 103 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - type f32: 243 tensors Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - type f16: 146 tensors Feb 26 23:30:20 ps ollama[908767]: llm_load_vocab: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect Feb 26 23:30:20 ps ollama[908767]: llm_load_vocab: special tokens cache size = 5 Feb 26 23:30:20 ps ollama[908767]: llm_load_vocab: token to piece cache size = 0.2032 MB Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: format = GGUF V3 (latest) Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: arch = bert Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: vocab type = WPM Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_vocab = 30522 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_merges = 0 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: vocab_only = 0 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_ctx_train = 512 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_embd = 1024 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_layer = 24 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_head = 16 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_head_kv = 16 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_rot = 64 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_swa = 0 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_embd_head_k = 64 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_embd_head_v = 64 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_gqa = 1 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_embd_k_gqa = 1024 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_embd_v_gqa = 1024 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: f_norm_eps = 1.0e-12 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: f_norm_rms_eps = 0.0e+00 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: f_clamp_kqv = 0.0e+00 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: f_max_alibi_bias = 0.0e+00 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: f_logit_scale = 0.0e+00 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_ff = 4096 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_expert = 0 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_expert_used = 0 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: causal attn = 0 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: pooling type = 2 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: rope type = 2 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: rope scaling = linear Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: freq_base_train = 10000.0 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: freq_scale_train = 1 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_ctx_orig_yarn = 512 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: rope_finetuned = unknown Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: ssm_d_conv = 0 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: ssm_d_inner = 0 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: ssm_d_state = 0 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: ssm_dt_rank = 0 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: ssm_dt_b_c_rms = 0 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: model type = 335M Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: model ftype = F16 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: model params = 334.09 M Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: model size = 637.85 MiB (16.02 BPW) Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: general.name = mxbai-embed-large-v1 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: BOS token = 101 '[CLS]' Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: EOS token = 102 '[SEP]' Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: UNK token = 100 '[UNK]' Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: SEP token = 102 '[SEP]' Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: PAD token = 0 '[PAD]' Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: CLS token = 101 '[CLS]' Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: MASK token = 103 '[MASK]' Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: LF token = 0 '[PAD]' Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: EOG token = 102 '[SEP]' Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: max token length = 21 Feb 26 23:30:20 ps ollama[908767]: time=2025-02-26T23:30:20.247+08:00 level=INFO source=server.go:591 msg="waiting for server to become available" status="llm server loading model" Feb 26 23:30:20 ps ollama[908767]: llm_load_tensors: offloading 24 repeating layers to GPU Feb 26 23:30:20 ps ollama[908767]: llm_load_tensors: offloading output layer to GPU Feb 26 23:30:20 ps ollama[908767]: llm_load_tensors: offloaded 25/25 layers to GPU Feb 26 23:30:20 ps ollama[908767]: llm_load_tensors: CUDA0 model buffer size = 577.22 MiB Feb 26 23:30:20 ps ollama[908767]: llm_load_tensors: CPU_Mapped model buffer size = 60.63 MiB Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: n_seq_max = 1 Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: n_ctx = 40960 Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: n_ctx_per_seq = 40960 Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: n_batch = 512 Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: n_ubatch = 512 Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: flash_attn = 0 Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: freq_base = 10000.0 Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: freq_scale = 1 Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: n_ctx_pre_seq (40960) > n_ctx_train (512) -- possible training context overflow Feb 26 23:30:20 ps ollama[908767]: llama_kv_cache_init: kv_size = 40960, offload = 1, type_k = 'f16', type_v = 'f16', n_layer = 24, can_shift = 1 Feb 26 23:30:20 ps ollama[908767]: llama_kv_cache_init: CUDA0 KV buffer size = 3840.00 MiB Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: KV self size = 3840.00 MiB, K (f16): 1920.00 MiB, V (f16): 1920.00 MiB Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: CUDA_Host output buffer size = 0.00 MiB Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: CUDA0 compute buffer size = 25.01 MiB Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: CUDA_Host compute buffer size = 5.01 MiB Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: graph nodes = 849 Feb 26 23:30:20 ps ollama[908767]: llama_new_context_with_model: graph splits = 4 (with bs=512), 2 (with bs=1) Feb 26 23:30:20 ps ollama[908767]: time=2025-02-26T23:30:20.498+08:00 level=INFO source=server.go:596 msg="llama runner started in 0.50 seconds" Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: loaded meta data with 23 key-value pairs and 389 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-819c2adf5ce6df2b6bd2ae4ca90d2a69f060afeb438d0c171db57daa02e39c3d (version GGUF V3 (latest)) Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 0: general.architecture str = bert Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 1: general.name str = mxbai-embed-large-v1 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 2: bert.block_count u32 = 24 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 3: bert.context_length u32 = 512 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 4: bert.embedding_length u32 = 1024 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 5: bert.feed_forward_length u32 = 4096 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 6: bert.attention.head_count u32 = 16 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 7: bert.attention.layer_norm_epsilon f32 = 0.000000 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 8: general.file_type u32 = 1 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 9: bert.attention.causal bool = false Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 10: bert.pooling_type u32 = 2 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 11: tokenizer.ggml.token_type_count u32 = 2 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 12: tokenizer.ggml.bos_token_id u32 = 101 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 13: tokenizer.ggml.eos_token_id u32 = 102 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 14: tokenizer.ggml.model str = bert Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 15: tokenizer.ggml.tokens arr[str,30522] = ["[PAD]", "[unused0]", "[unused1]", "... Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 16: tokenizer.ggml.scores arr[f32,30522] = [-1000.000000, -1000.000000, -1000.00... Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,30522] = [3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 18: tokenizer.ggml.unknown_token_id u32 = 100 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 19: tokenizer.ggml.seperator_token_id u32 = 102 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 20: tokenizer.ggml.padding_token_id u32 = 0 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 21: tokenizer.ggml.cls_token_id u32 = 101 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - kv 22: tokenizer.ggml.mask_token_id u32 = 103 Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - type f32: 243 tensors Feb 26 23:30:20 ps ollama[908767]: llama_model_loader: - type f16: 146 tensors Feb 26 23:30:20 ps ollama[908767]: llm_load_vocab: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect Feb 26 23:30:20 ps ollama[908767]: llm_load_vocab: special tokens cache size = 5 Feb 26 23:30:20 ps ollama[908767]: llm_load_vocab: token to piece cache size = 0.2032 MB Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: format = GGUF V3 (latest) Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: arch = bert Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: vocab type = WPM Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_vocab = 30522 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: n_merges = 0 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: vocab_only = 1 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: model type = ?B Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: model ftype = all F32 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: model params = 334.09 M Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: model size = 637.85 MiB (16.02 BPW) Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: general.name = mxbai-embed-large-v1 Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: BOS token = 101 '[CLS]' Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: EOS token = 102 '[SEP]' Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: UNK token = 100 '[UNK]' Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: SEP token = 102 '[SEP]' Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: PAD token = 0 '[PAD]' Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: CLS token = 101 '[CLS]' Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: MASK token = 103 '[MASK]' Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: LF token = 0 '[PAD]' Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: EOG token = 102 '[SEP]' Feb 26 23:30:20 ps ollama[908767]: llm_load_print_meta: max token length = 21 Feb 26 23:30:20 ps ollama[908767]: llama_model_load: vocab only - skipping tensors Feb 26 23:30:20 ps ollama[908767]: //ml/backend/ggml/ggml/src/ggml-cpu/ggml-cpu.c:8374: GGML_ASSERT(i01 >= 0 && i01 < ne01) failed Feb 26 23:30:20 ps ollama[908767]: //ml/backend/ggml/ggml/src/ggml-cpu/ggml-cpu.c:8374: GGML_ASSERT(i01 >= 0 && i01 < ne01) failed Feb 26 23:30:20 ps ollama[1363647]: Could not attach to process. If your uid matches the uid of the target Feb 26 23:30:20 ps ollama[1363647]: process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or try Feb 26 23:30:20 ps ollama[1363647]: again as the root user. For more details, see /etc/sysctl.d/10-ptrace.conf Feb 26 23:30:20 ps ollama[908767]: ptrace: Inappropriate ioctl for device. Feb 26 23:30:20 ps ollama[908767]: No stack. Feb 26 23:30:20 ps ollama[908767]: The program is not being run. Feb 26 23:30:20 ps ollama[1363642]: Could not attach to process. If your uid matches the uid of the target Feb 26 23:30:20 ps ollama[1363642]: process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or try Feb 26 23:30:20 ps ollama[1363642]: again as the root user. For more details, see /etc/sysctl.d/10-ptrace.conf Feb 26 23:30:20 ps ollama[908767]: warning: process 1363105 is a zombie - the process has already terminated Feb 26 23:30:20 ps ollama[908767]: ptrace: Inappropriate ioctl for device. Feb 26 23:30:20 ps ollama[908767]: No stack. Feb 26 23:30:20 ps ollama[908767]: The program is not being run. Feb 26 23:30:20 ps ollama[908767]: time=2025-02-26T23:30:20.876+08:00 level=ERROR source=routes.go:478 msg="embedding generation failed" error="do embedding request: Post \"http://127.0.0.1:40221/embedding\": EOF" Feb 26 23:30:20 ps ollama[908767]: [GIN] 2025/02/26 - 23:30:20 | 500 | 6.83067669s | ::1 | POST "/api/embed" Feb 26 23:30:20 ps ollama[908767]: time=2025-02-26T23:30:20.941+08:00 level=ERROR source=server.go:421 msg="llama runner terminated" error="signal: aborted" ``` ### OS Linux ### GPU Nvidia ### CPU Intel ### Ollama version 0.5.12
GiteaMirror added the bug label 2026-04-12 17:27:10 -05:00
GiteaMirror commented 2026-04-12 17:27:12 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Feb 26, 2025):

Is it because of the large num_ctx value?

Yes. https://github.com/ollama/ollama/issues/7288#issuecomment-2591709109

<!-- gh-comment-id:2685445574 --> @rick-github commented on GitHub (Feb 26, 2025): > Is it because of the large num_ctx value? Yes. https://github.com/ollama/ollama/issues/7288#issuecomment-2591709109
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#6115
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?