github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 16:11:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #60] crash on large context sizes #25784

New Issue
Closed
opened 2026-04-22 01:36:49 -05:00 by GiteaMirror · 1 comment
GiteaMirror commented 2026-04-22 01:36:49 -05:00
Owner
Copy Link

Originally created by @jmorganca on GitHub (Jul 8, 2023).
Original GitHub issue: https://github.com/ollama/ollama/issues/60

For prompts larger than ~100 tokens the server will sometimes crash

Originally created by @jmorganca on GitHub (Jul 8, 2023). Original GitHub issue: https://github.com/ollama/ollama/issues/60 For prompts larger than ~100 tokens the server will sometimes crash
GiteaMirror added the bug label 2026-04-22 01:36:49 -05:00
GiteaMirror commented 2026-04-22 01:36:50 -05:00
Author
Owner
Copy Link

@mchiang0610 commented on GitHub (Jul 18, 2023):

Related to the large context size; this is after increasing n_ctx parameter to 8192.


[GIN] 2023/07/18 - 11:01:38 | 200 |    4.261792ms |       127.0.0.1 | POST     "/api/pull"
llama.cpp: loading model from /Users/mchiang/.ollama/models/blobs/sha256:e84705205f71dd55be7b24a778f248f0eda9999a125d313358c087e092d83148
llama_model_load_internal: format     = ggjt v3 (latest)
llama_model_load_internal: n_vocab    = 32000
llama_model_load_internal: n_ctx      = 8192
llama_model_load_internal: n_embd     = 3200
llama_model_load_internal: n_mult     = 240
llama_model_load_internal: n_head     = 32
llama_model_load_internal: n_layer    = 26
llama_model_load_internal: n_rot      = 100
llama_model_load_internal: ftype      = 2 (mostly Q4_0)
llama_model_load_internal: n_ff       = 8640
llama_model_load_internal: model size = 3B
llama_model_load_internal: ggml ctx size =    0.06 MB
llama_model_load_internal: mem required  = 2862.72 MB (+  682.00 MB per state)
llama_new_context_with_model: kv self size  = 2600.00 MB
ggml_metal_init: allocating
ggml_metal_init: using MPS
ggml_metal_init: loading '/Users/mchiang/Downloads/github/ollama/ggml-metal.metal'
ggml_metal_init: loaded kernel_add                            0x13f809930
ggml_metal_init: loaded kernel_mul                            0x13f80a4e0
ggml_metal_init: loaded kernel_mul_row                        0x13f80adb0
ggml_metal_init: loaded kernel_scale                          0x13f80b610
ggml_metal_init: loaded kernel_silu                           0x13f80bde0
ggml_metal_init: loaded kernel_relu                           0x13f80c5b0
ggml_metal_init: loaded kernel_gelu                           0x13f80cd80
ggml_metal_init: loaded kernel_soft_max                       0x13f80d2d0
ggml_metal_init: loaded kernel_diag_mask_inf                  0x13f80d820
ggml_metal_init: loaded kernel_get_rows_f16                   0x13ef058b0
ggml_metal_init: loaded kernel_get_rows_q4_0                  0x13ef05fb0
ggml_metal_init: loaded kernel_get_rows_q4_1                  0x13ef06670
ggml_metal_init: loaded kernel_get_rows_q2_K                  0x13ef06bc0
ggml_metal_init: loaded kernel_get_rows_q3_K                  0x13ef07110
ggml_metal_init: loaded kernel_get_rows_q4_K                  0x13ef07660
ggml_metal_init: loaded kernel_get_rows_q5_K                  0x13ef07bb0
ggml_metal_init: loaded kernel_get_rows_q6_K                  0x13ef08100
ggml_metal_init: loaded kernel_rms_norm                       0x13ef08650
ggml_metal_init: loaded kernel_norm                           0x13ef08ba0
ggml_metal_init: loaded kernel_mul_mat_f16_f32                0x13ef09260
ggml_metal_init: loaded kernel_mul_mat_q4_0_f32               0x13ef097b0
ggml_metal_init: loaded kernel_mul_mat_q4_1_f32               0x13ef09d00
ggml_metal_init: loaded kernel_mul_mat_q2_K_f32               0x13ef0a250
ggml_metal_init: loaded kernel_mul_mat_q3_K_f32               0x13f80ddd0
ggml_metal_init: loaded kernel_mul_mat_q4_K_f32               0x13f80e4a0
ggml_metal_init: loaded kernel_mul_mat_q5_K_f32               0x13ef0a680
ggml_metal_init: loaded kernel_mul_mat_q6_K_f32               0x13f904f90
ggml_metal_init: loaded kernel_rope                           0x13f9057b0
ggml_metal_init: loaded kernel_alibi_f32                      0x13ee051d0
ggml_metal_init: loaded kernel_cpy_f32_f16                    0x13ee05890
ggml_metal_init: loaded kernel_cpy_f32_f32                    0x13f905be0
ggml_metal_init: loaded kernel_cpy_f16_f16                    0x13f9062a0
ggml_metal_init: recommendedMaxWorkingSetSize = 10922.67 MB
ggml_metal_init: hasUnifiedMemory             = true
ggml_metal_init: maxTransferRate              = built-in GPU
llama_new_context_with_model: max tensor size =    54.93 MB
ggml_metal_add_buffer: allocated 'data            ' buffer, size =  1839.12 MB, ( 1839.52 / 10922.67)
ggml_metal_add_buffer: allocated 'eval            ' buffer, size =   512.00 MB, ( 2351.52 / 10922.67)
ggml_metal_add_buffer: allocated 'kv              ' buffer, size =  2602.00 MB, ( 4953.52 / 10922.67)
ggml_metal_add_buffer: allocated 'scr0            ' buffer, size =   256.00 MB, ( 5209.52 / 10922.67)
ggml_metal_add_buffer: allocated 'scr1            ' buffer, size =   256.00 MB, ( 5465.52 / 10922.67)

llama_print_timings:        load time =  5445.45 ms
llama_print_timings:      sample time =   832.48 ms /   345 runs   (    2.41 ms per token,   414.42 tokens per second)
llama_print_timings: prompt eval time =  9827.04 ms /   330 tokens (   29.78 ms per token,    33.58 tokens per second)
llama_print_timings:        eval time = 10982.84 ms /   344 runs   (   31.93 ms per token,    31.32 tokens per second)
llama_print_timings:       total time = 25044.42 ms
ggml_metal_free: deallocating
[GIN] 2023/07/18 - 11:02:11 | 200 | 30.556039791s |       127.0.0.1 | POST     "/api/generate"
llama.cpp: loading model from /Users/mchiang/.ollama/models/blobs/sha256:e84705205f71dd55be7b24a778f248f0eda9999a125d313358c087e092d83148
llama_model_load_internal: format     = ggjt v3 (latest)
llama_model_load_internal: n_vocab    = 32000
llama_model_load_internal: n_ctx      = 8192
llama_model_load_internal: n_embd     = 3200
llama_model_load_internal: n_mult     = 240
llama_model_load_internal: n_head     = 32
llama_model_load_internal: n_layer    = 26
llama_model_load_internal: n_rot      = 100
llama_model_load_internal: ftype      = 2 (mostly Q4_0)
llama_model_load_internal: n_ff       = 8640
llama_model_load_internal: model size = 3B
llama_model_load_internal: ggml ctx size =    0.06 MB
llama_model_load_internal: mem required  = 2862.72 MB (+  682.00 MB per state)
llama_new_context_with_model: kv self size  = 2600.00 MB
ggml_metal_init: allocating
ggml_metal_init: using MPS
ggml_metal_init: loading '/Users/mchiang/Downloads/github/ollama/ggml-metal.metal'
ggml_metal_init: loaded kernel_add                            0x13f9572a0
ggml_metal_init: loaded kernel_mul                            0x13f9569e0
ggml_metal_init: loaded kernel_mul_row                        0x13f906710
ggml_metal_init: loaded kernel_scale                          0x13f906980
ggml_metal_init: loaded kernel_silu                           0x13f957870
ggml_metal_init: loaded kernel_relu                           0x13f957ae0
ggml_metal_init: loaded kernel_gelu                           0x13f957d50
ggml_metal_init: loaded kernel_soft_max                       0x13f957fc0
ggml_metal_init: loaded kernel_diag_mask_inf                  0x13f958230
ggml_metal_init: loaded kernel_get_rows_f16                   0x13f9584a0
ggml_metal_init: loaded kernel_get_rows_q4_0                  0x13f958710
ggml_metal_init: loaded kernel_get_rows_q4_1                  0x13f958980
ggml_metal_init: loaded kernel_get_rows_q2_K                  0x13f958bf0
ggml_metal_init: loaded kernel_get_rows_q3_K                  0x13f958e60
ggml_metal_init: loaded kernel_get_rows_q4_K                  0x13f9590d0
ggml_metal_init: loaded kernel_get_rows_q5_K                  0x13f959340
ggml_metal_init: loaded kernel_get_rows_q6_K                  0x13f9595b0
ggml_metal_init: loaded kernel_rms_norm                       0x13f959820
ggml_metal_init: loaded kernel_norm                           0x13f959a90
ggml_metal_init: loaded kernel_mul_mat_f16_f32                0x13f959d00
ggml_metal_init: loaded kernel_mul_mat_q4_0_f32               0x13dbb8770
ggml_metal_init: loaded kernel_mul_mat_q4_1_f32               0x13dbb7eb0
ggml_metal_init: loaded kernel_mul_mat_q2_K_f32               0x13dbb8d40
ggml_metal_init: loaded kernel_mul_mat_q3_K_f32               0x13dbb8fb0
ggml_metal_init: loaded kernel_mul_mat_q4_K_f32               0x13dbb9220
ggml_metal_init: loaded kernel_mul_mat_q5_K_f32               0x13dbb9490
ggml_metal_init: loaded kernel_mul_mat_q6_K_f32               0x13dbb9700
ggml_metal_init: loaded kernel_rope                           0x13dbb9970
ggml_metal_init: loaded kernel_alibi_f32                      0x13dbb9be0
ggml_metal_init: loaded kernel_cpy_f32_f16                    0x13dbb9e50
ggml_metal_init: loaded kernel_cpy_f32_f32                    0x12ff15af0
ggml_metal_init: loaded kernel_cpy_f16_f16                    0x12ff15230
ggml_metal_init: recommendedMaxWorkingSetSize = 10922.67 MB
ggml_metal_init: hasUnifiedMemory             = true
ggml_metal_init: maxTransferRate              = built-in GPU
llama_new_context_with_model: max tensor size =    54.93 MB
ggml_metal_add_buffer: allocated 'data            ' buffer, size =  1839.12 MB, ( 1839.50 / 10922.67)
ggml_metal_add_buffer: allocated 'eval            ' buffer, size =   512.00 MB, ( 2351.50 / 10922.67)
ggml_metal_add_buffer: allocated 'kv              ' buffer, size =  2602.00 MB, ( 4953.50 / 10922.67)
ggml_metal_add_buffer: allocated 'scr0            ' buffer, size =   256.00 MB, ( 5209.50 / 10922.67)
ggml_metal_add_buffer: allocated 'scr1            ' buffer, size =   256.00 MB, ( 5465.50 / 10922.67)

llama_print_timings:        load time =  3873.08 ms
llama_print_timings:      sample time =   846.68 ms /   350 runs   (    2.42 ms per token,   413.38 tokens per second)
llama_print_timings: prompt eval time = 20146.86 ms /   690 tokens (   29.20 ms per token,    34.25 tokens per second)
llama_print_timings:        eval time = 12752.31 ms /   349 runs   (   36.54 ms per token,    27.37 tokens per second)
llama_print_timings:       total time = 34234.22 ms
ggml_metal_free: deallocating
[GIN] 2023/07/18 - 11:02:51 | 200 |  38.20611025s |       127.0.0.1 | POST     "/api/generate"
llama.cpp: loading model from /Users/mchiang/.ollama/models/blobs/sha256:e84705205f71dd55be7b24a778f248f0eda9999a125d313358c087e092d83148
llama_model_load_internal: format     = ggjt v3 (latest)
llama_model_load_internal: n_vocab    = 32000
llama_model_load_internal: n_ctx      = 8192
llama_model_load_internal: n_embd     = 3200
llama_model_load_internal: n_mult     = 240
llama_model_load_internal: n_head     = 32
llama_model_load_internal: n_layer    = 26
llama_model_load_internal: n_rot      = 100
llama_model_load_internal: ftype      = 2 (mostly Q4_0)
llama_model_load_internal: n_ff       = 8640
llama_model_load_internal: model size = 3B
llama_model_load_internal: ggml ctx size =    0.06 MB
llama_model_load_internal: mem required  = 2862.72 MB (+  682.00 MB per state)
llama_new_context_with_model: kv self size  = 2600.00 MB
ggml_metal_init: allocating
ggml_metal_init: using MPS
ggml_metal_init: loading '/Users/mchiang/Downloads/github/ollama/ggml-metal.metal'
ggml_metal_init: loaded kernel_add                            0x143888980
ggml_metal_init: loaded kernel_mul                            0x1438880c0
ggml_metal_init: loaded kernel_mul_row                        0x143888f50
ggml_metal_init: loaded kernel_scale                          0x1438891c0
ggml_metal_init: loaded kernel_silu                           0x143889430
ggml_metal_init: loaded kernel_relu                           0x1438896a0
ggml_metal_init: loaded kernel_gelu                           0x143889910
ggml_metal_init: loaded kernel_soft_max                       0x143889b80
ggml_metal_init: loaded kernel_diag_mask_inf                  0x143889df0
ggml_metal_init: loaded kernel_get_rows_f16                   0x14388a060
ggml_metal_init: loaded kernel_get_rows_q4_0                  0x14388a2d0
ggml_metal_init: loaded kernel_get_rows_q4_1                  0x14388a540
ggml_metal_init: loaded kernel_get_rows_q2_K                  0x14388a7b0
ggml_metal_init: loaded kernel_get_rows_q3_K                  0x14388aa20
ggml_metal_init: loaded kernel_get_rows_q4_K                  0x14388ac90
ggml_metal_init: loaded kernel_get_rows_q5_K                  0x14388af00
ggml_metal_init: loaded kernel_get_rows_q6_K                  0x14388b170
ggml_metal_init: loaded kernel_rms_norm                       0x14388b3e0
ggml_metal_init: loaded kernel_norm                           0x14388b650
ggml_metal_init: loaded kernel_mul_mat_f16_f32                0x14388b8c0
ggml_metal_init: loaded kernel_mul_mat_q4_0_f32               0x14388bb30
ggml_metal_init: loaded kernel_mul_mat_q4_1_f32               0x14388bda0
ggml_metal_init: loaded kernel_mul_mat_q2_K_f32               0x14388c010
ggml_metal_init: loaded kernel_mul_mat_q3_K_f32               0x14388c280
ggml_metal_init: loaded kernel_mul_mat_q4_K_f32               0x14388c4f0
ggml_metal_init: loaded kernel_mul_mat_q5_K_f32               0x14388c760
ggml_metal_init: loaded kernel_mul_mat_q6_K_f32               0x14388c9d0
ggml_metal_init: loaded kernel_rope                           0x14388cc40
ggml_metal_init: loaded kernel_alibi_f32                      0x14388ceb0
ggml_metal_init: loaded kernel_cpy_f32_f16                    0x14388d120
ggml_metal_init: loaded kernel_cpy_f32_f32                    0x14388d390
ggml_metal_init: loaded kernel_cpy_f16_f16                    0x14388d600
ggml_metal_init: recommendedMaxWorkingSetSize = 10922.67 MB
ggml_metal_init: hasUnifiedMemory             = true
ggml_metal_init: maxTransferRate              = built-in GPU
llama_new_context_with_model: max tensor size =    54.93 MB
ggml_metal_add_buffer: allocated 'data            ' buffer, size =  1839.12 MB, ( 1839.50 / 10922.67)
ggml_metal_add_buffer: allocated 'eval            ' buffer, size =   512.00 MB, ( 2351.50 / 10922.67)
ggml_metal_add_buffer: allocated 'kv              ' buffer, size =  2602.00 MB, ( 4953.50 / 10922.67)
ggml_metal_add_buffer: allocated 'scr0            ' buffer, size =   256.00 MB, ( 5209.50 / 10922.67)
ggml_metal_add_buffer: allocated 'scr1            ' buffer, size =   256.00 MB, ( 5465.50 / 10922.67)

llama_print_timings:        load time =  2970.37 ms
llama_print_timings:      sample time =   679.80 ms /   312 runs   (    2.18 ms per token,   458.96 tokens per second)
llama_print_timings: prompt eval time = 30586.76 ms /  1055 tokens (   28.99 ms per token,    34.49 tokens per second)
llama_print_timings:        eval time = 12229.11 ms /   311 runs   (   39.32 ms per token,    25.43 tokens per second)
llama_print_timings:       total time = 44055.38 ms
ggml_metal_free: deallocating
[GIN] 2023/07/18 - 11:03:40 | 200 |    47.156634s |       127.0.0.1 | POST     "/api/generate"
llama.cpp: loading model from /Users/mchiang/.ollama/models/blobs/sha256:e84705205f71dd55be7b24a778f248f0eda9999a125d313358c087e092d83148
llama_model_load_internal: format     = ggjt v3 (latest)
llama_model_load_internal: n_vocab    = 32000
llama_model_load_internal: n_ctx      = 8192
llama_model_load_internal: n_embd     = 3200
llama_model_load_internal: n_mult     = 240
llama_model_load_internal: n_head     = 32
llama_model_load_internal: n_layer    = 26
llama_model_load_internal: n_rot      = 100
llama_model_load_internal: ftype      = 2 (mostly Q4_0)
llama_model_load_internal: n_ff       = 8640
llama_model_load_internal: model size = 3B
llama_model_load_internal: ggml ctx size =    0.06 MB
llama_model_load_internal: mem required  = 2862.72 MB (+  682.00 MB per state)
llama_new_context_with_model: kv self size  = 2600.00 MB
ggml_metal_init: allocating
ggml_metal_init: using MPS
ggml_metal_init: loading '/Users/mchiang/Downloads/github/ollama/ggml-metal.metal'
ggml_metal_init: loaded kernel_add                            0x1445cbdc0
ggml_metal_init: loaded kernel_mul                            0x1445cb500
ggml_metal_init: loaded kernel_mul_row                        0x1445cc390
ggml_metal_init: loaded kernel_scale                          0x1445cc600
ggml_metal_init: loaded kernel_silu                           0x1445cc870
ggml_metal_init: loaded kernel_relu                           0x1445ccae0
ggml_metal_init: loaded kernel_gelu                           0x1445ccd50
ggml_metal_init: loaded kernel_soft_max                       0x1445ccfc0
ggml_metal_init: loaded kernel_diag_mask_inf                  0x1445cd230
ggml_metal_init: loaded kernel_get_rows_f16                   0x1445cd4a0
ggml_metal_init: loaded kernel_get_rows_q4_0                  0x1445cd710
ggml_metal_init: loaded kernel_get_rows_q4_1                  0x1445cd980
ggml_metal_init: loaded kernel_get_rows_q2_K                  0x1445cdbf0
ggml_metal_init: loaded kernel_get_rows_q3_K                  0x1445cde60
ggml_metal_init: loaded kernel_get_rows_q4_K                  0x1445ce0d0
ggml_metal_init: loaded kernel_get_rows_q5_K                  0x144e6bf00
ggml_metal_init: loaded kernel_get_rows_q6_K                  0x1445ce340
ggml_metal_init: loaded kernel_rms_norm                       0x1445ce5b0
ggml_metal_init: loaded kernel_norm                           0x1445ce820
ggml_metal_init: loaded kernel_mul_mat_f16_f32                0x1445cea90
ggml_metal_init: loaded kernel_mul_mat_q4_0_f32               0x1445ced00
ggml_metal_init: loaded kernel_mul_mat_q4_1_f32               0x1445cef70
ggml_metal_init: loaded kernel_mul_mat_q2_K_f32               0x1445cf1e0
ggml_metal_init: loaded kernel_mul_mat_q3_K_f32               0x1445cf450
ggml_metal_init: loaded kernel_mul_mat_q4_K_f32               0x1445cf6c0
ggml_metal_init: loaded kernel_mul_mat_q5_K_f32               0x1445cf930
ggml_metal_init: loaded kernel_mul_mat_q6_K_f32               0x1445cfba0
ggml_metal_init: loaded kernel_rope                           0x1445cfe10
ggml_metal_init: loaded kernel_alibi_f32                      0x1445d0080
ggml_metal_init: loaded kernel_cpy_f32_f16                    0x1445d02f0
ggml_metal_init: loaded kernel_cpy_f32_f32                    0x1445d0560
ggml_metal_init: loaded kernel_cpy_f16_f16                    0x1445d07d0
ggml_metal_init: recommendedMaxWorkingSetSize = 10922.67 MB
ggml_metal_init: hasUnifiedMemory             = true
ggml_metal_init: maxTransferRate              = built-in GPU
llama_new_context_with_model: max tensor size =    54.93 MB
ggml_metal_add_buffer: allocated 'data            ' buffer, size =  1839.12 MB, ( 1839.50 / 10922.67)
ggml_metal_add_buffer: allocated 'eval            ' buffer, size =   512.00 MB, ( 2351.50 / 10922.67)
ggml_metal_add_buffer: allocated 'kv              ' buffer, size =  2602.00 MB, ( 4953.50 / 10922.67)
ggml_metal_add_buffer: allocated 'scr0            ' buffer, size =   256.00 MB, ( 5209.50 / 10922.67)
ggml_metal_add_buffer: allocated 'scr1            ' buffer, size =   256.00 MB, ( 5465.50 / 10922.67)
ggml_new_tensor_impl: not enough space in the scratch memory pool (needed 336026880, available 268435456)
fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x2 addr=0x50 pc=0x10330775c]

runtime stack:
runtime.throw({0x103377dd6?, 0x17101ed00?})
	/usr/local/go/src/runtime/panic.go:1047 +0x40 fp=0x17101ec50 sp=0x17101ec20 pc=0x102e44b30
runtime.sigpanic()
	/usr/local/go/src/runtime/signal_unix.go:821 +0x244 fp=0x17101ec90 sp=0x17101ec50 pc=0x102e5b424

goroutine 67 [syscall]:
runtime.cgocall(0x103302a00, 0x14000062bb8)
	/usr/local/go/src/runtime/cgocall.go:157 +0x54 fp=0x14000062b80 sp=0x14000062b40 pc=0x102e14494
github.com/jmorganca/ollama/llama._Cfunc_llama_eval(0x140049600, 0x140005f8000, 0x56d, 0x1, 0xa)
	_cgo_gotypes.go:208 +0x38 fp=0x14000062bb0 sp=0x14000062b80 pc=0x1032f4698
github.com/jmorganca/ollama/llama.(*llama).generate.func2(0x140002761e0, 0x4?)
	/Users/mchiang/Downloads/github/ollama/llama/llama.go:211 +0xa0 fp=0x14000062c20 sp=0x14000062bb0 pc=0x1032f67d0
github.com/jmorganca/ollama/llama.(*llama).generate(0x140002761e0, {0x140005f8000, 0x56d, 0x800}, 0x14000062f90)
	/Users/mchiang/Downloads/github/ollama/llama/llama.go:211 +0x178 fp=0x14000062ec0 sp=0x14000062c20 pc=0x1032f6248
github.com/jmorganca/ollama/llama.(*llama).Predict(0x140002761e0?, {0x140006b0000, 0x556, 0x63f?}, {0x1400045e0a0?, 0x1b?}, 0x1400051af90?)
	/Users/mchiang/Downloads/github/ollama/llama/llama.go:161 +0x11c fp=0x14000062f30 sp=0x14000062ec0 pc=0x1032f5b3c
github.com/jmorganca/ollama/server.generate.func1()
	/Users/mchiang/Downloads/github/ollama/server/routes.go:80 +0xa8 fp=0x14000062fd0 sp=0x14000062f30 pc=0x1032ff228
runtime.goexit()
	/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x14000062fd0 sp=0x14000062fd0 pc=0x102e76934
created by github.com/jmorganca/ollama/server.generate
	/Users/mchiang/Downloads/github/ollama/server/routes.go:78 +0x8b0

goroutine 1 [IO wait, 2 minutes]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	/usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x140000c3860 sp=0x140000c3840 pc=0x102e47584
runtime.netpollblock(0x140000c38f8?, 0x2ef9774?, 0x1?)
	/usr/local/go/src/runtime/netpoll.go:527 +0x158 fp=0x140000c38a0 sp=0x140000c3860 pc=0x102e40c38
internal/poll.runtime_pollWait(0x12b00cb18, 0x72)
	/usr/local/go/src/runtime/netpoll.go:306 +0xa0 fp=0x140000c38d0 sp=0x140000c38a0 pc=0x102e707c0
internal/poll.(*pollDesc).wait(0x14000482380?, 0x102e1ce18?, 0x0)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x28 fp=0x140000c3900 sp=0x140000c38d0 pc=0x102ef4db8
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0x14000482380)
	/usr/local/go/src/internal/poll/fd_unix.go:614 +0x250 fp=0x140000c39b0 sp=0x140000c3900 pc=0x102ef9860
net.(*netFD).accept(0x14000482380)
	/usr/local/go/src/net/fd_unix.go:172 +0x28 fp=0x140000c3a70 sp=0x140000c39b0 pc=0x102f38738
net.(*TCPListener).accept(0x140000aed38)
	/usr/local/go/src/net/tcpsock_posix.go:148 +0x28 fp=0x140000c3aa0 sp=0x140000c3a70 pc=0x102f4dd38
net.(*TCPListener).Accept(0x140000aed38)
	/usr/local/go/src/net/tcpsock.go:297 +0x2c fp=0x140000c3ae0 sp=0x140000c3aa0 pc=0x102f4ceac
net/http.(*onceCloseListener).Accept(0x14000190240?)
	<autogenerated>:1 +0x30 fp=0x140000c3b00 sp=0x140000c3ae0 pc=0x1030b3290
net/http.(*Server).Serve(0x14000308ff0, {0x10356fe20, 0x140000aed38})
	/usr/local/go/src/net/http/server.go:3059 +0x304 fp=0x140000c3c30 sp=0x140000c3b00 pc=0x10308f7b4
github.com/jmorganca/ollama/server.Serve({0x10356fe20, 0x140000aed38})
	/Users/mchiang/Downloads/github/ollama/server/routes.go:201 +0x20c fp=0x140000c3ca0 sp=0x140000c3c30 pc=0x10330062c
github.com/jmorganca/ollama/cmd.RunServer(0x14000425200?, {0x1033532ac?, 0x0?, 0x0?})
	/Users/mchiang/Downloads/github/ollama/cmd/cmd.go:245 +0x114 fp=0x140000c3d20 sp=0x140000c3ca0 pc=0x103302194
github.com/spf13/cobra.(*Command).execute(0x14000425200, {0x1039cd908, 0x0, 0x0})
	/Users/mchiang/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:940 +0x5c8 fp=0x140000c3e60 sp=0x140000c3d20 pc=0x103122cd8
github.com/spf13/cobra.(*Command).ExecuteC(0x14000424900)
	/Users/mchiang/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:1068 +0x35c fp=0x140000c3f20 sp=0x140000c3e60 pc=0x10312342c
github.com/spf13/cobra.(*Command).Execute(...)
	/Users/mchiang/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:992
github.com/spf13/cobra.(*Command).ExecuteContext(0x14000054768?, {0x103570638?, 0x140000a0010?})
	/Users/mchiang/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:985 +0x50 fp=0x140000c3f40 sp=0x140000c3f20 pc=0x103122fc0
main.main()
	/Users/mchiang/Downloads/github/ollama/main.go:10 +0x34 fp=0x140000c3f70 sp=0x140000c3f40 pc=0x1033027d4
runtime.main()
	/usr/local/go/src/runtime/proc.go:250 +0x248 fp=0x140000c3fd0 sp=0x140000c3f70 pc=0x102e47158
runtime.goexit()
	/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x140000c3fd0 sp=0x140000c3fd0 pc=0x102e76934

goroutine 2 [force gc (idle), 2 minutes]:
runtime.gopark(0x61d96659a2f6?, 0x0?, 0x0?, 0x0?, 0x0?)
	/usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x14000054fa0 sp=0x14000054f80 pc=0x102e47584
runtime.goparkunlock(...)
	/usr/local/go/src/runtime/proc.go:387
runtime.forcegchelper()
	/usr/local/go/src/runtime/proc.go:305 +0xb8 fp=0x14000054fd0 sp=0x14000054fa0 pc=0x102e473c8
runtime.goexit()
	/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x14000054fd0 sp=0x14000054fd0 pc=0x102e76934
created by runtime.init.6
	/usr/local/go/src/runtime/proc.go:293 +0x24

goroutine 3 [GC sweep wait]:
runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
	/usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x14000055760 sp=0x14000055740 pc=0x102e47584
runtime.goparkunlock(...)
	/usr/local/go/src/runtime/proc.go:387
runtime.bgsweep(0x0?)
	/usr/local/go/src/runtime/mgcsweep.go:319 +0x110 fp=0x140000557b0 sp=0x14000055760 pc=0x102e34460
runtime.gcenable.func1()
	/usr/local/go/src/runtime/mgc.go:178 +0x28 fp=0x140000557d0 sp=0x140000557b0 pc=0x102e28f08
runtime.goexit()
	/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x140000557d0 sp=0x140000557d0 pc=0x102e76934
created by runtime.gcenable
	/usr/local/go/src/runtime/mgc.go:178 +0x74

goroutine 4 [GC scavenge wait]:
runtime.gopark(0x1e6ff8d?, 0x1d7328c?, 0x0?, 0x0?, 0x0?)
	/usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x14000055f50 sp=0x14000055f30 pc=0x102e47584
runtime.goparkunlock(...)
	/usr/local/go/src/runtime/proc.go:387
runtime.(*scavengerState).park(0x103912620)
	/usr/local/go/src/runtime/mgcscavenge.go:400 +0x5c fp=0x14000055f80 sp=0x14000055f50 pc=0x102e3226c
runtime.bgscavenge(0x0?)
	/usr/local/go/src/runtime/mgcscavenge.go:633 +0xac fp=0x14000055fb0 sp=0x14000055f80 pc=0x102e3284c
runtime.gcenable.func2()
	/usr/local/go/src/runtime/mgc.go:179 +0x28 fp=0x14000055fd0 sp=0x14000055fb0 pc=0x102e28ea8
runtime.goexit()
	/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x14000055fd0 sp=0x14000055fd0 pc=0x102e76934
created by runtime.gcenable
	/usr/local/go/src/runtime/mgc.go:179 +0xb8

goroutine 18 [finalizer wait]:
runtime.gopark(0x0?, 0x103569680?, 0x0?, 0x60?, 0x2000000020?)
	/usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x14000054580 sp=0x14000054560 pc=0x102e47584
runtime.runfinq()
	/usr/local/go/src/runtime/mfinal.go:193 +0x10c fp=0x140000547d0 sp=0x14000054580 pc=0x102e27f9c
runtime.goexit()
	/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x140000547d0 sp=0x140000547d0 pc=0x102e76934
created by runtime.createfing
	/usr/local/go/src/runtime/mfinal.go:163 +0x84

goroutine 25 [chan receive]:
runtime.gopark(0x1?, 0x0?, 0x38?, 0x73?, 0x102e25b00?)
	/usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x14000157300 sp=0x140001572e0 pc=0x102e47584
runtime.chanrecv(0x140004f2060, 0x14000157418, 0x1)
	/usr/local/go/src/runtime/chan.go:583 +0x45c fp=0x14000157390 sp=0x14000157300 pc=0x102e16fec
runtime.chanrecv2(0x140001573e8?, 0x1032ecc1c?)
	/usr/local/go/src/runtime/chan.go:447 +0x14 fp=0x140001573c0 sp=0x14000157390 pc=0x102e16b74
github.com/jmorganca/ollama/server.streamResponse.func1({0x12b00e4d8, 0x1400052e300})
	/Users/mchiang/Downloads/github/ollama/server/routes.go:206 +0x34 fp=0x14000157430 sp=0x140001573c0 pc=0x1033006f4
github.com/gin-gonic/gin.(*Context).Stream(0x102e501f0?, 0x14000157488)
	/Users/mchiang/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:1090 +0x70 fp=0x14000157470 sp=0x14000157430 pc=0x1032e45d0
github.com/jmorganca/ollama/server.streamResponse(0xffffffffffffffff?, 0x0?)
	/Users/mchiang/Downloads/github/ollama/server/routes.go:205 +0x34 fp=0x140001574a0 sp=0x14000157470 pc=0x103300694
github.com/jmorganca/ollama/server.generate(0x1400052e300)
	/Users/mchiang/Downloads/github/ollama/server/routes.go:91 +0x8bc fp=0x140001576e0 sp=0x140001574a0 pc=0x1032ff06c
github.com/gin-gonic/gin.(*Context).Next(...)
	/Users/mchiang/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174
github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1(0x1400052e300)
	/Users/mchiang/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/recovery.go:102 +0x7c fp=0x14000157730 sp=0x140001576e0 pc=0x1032ebd8c
github.com/gin-gonic/gin.(*Context).Next(...)
	/Users/mchiang/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174
github.com/gin-gonic/gin.LoggerWithConfig.func1(0x1400052e300)
	/Users/mchiang/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/logger.go:240 +0xac fp=0x140001578e0 sp=0x14000157730 pc=0x1032eb00c
github.com/gin-gonic/gin.(*Context).Next(...)
	/Users/mchiang/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174
github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0x14000476820, 0x1400052e300)
	/Users/mchiang/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/gin.go:620 +0x54c fp=0x14000157a70 sp=0x140001578e0 pc=0x1032ea11c
github.com/gin-gonic/gin.(*Engine).ServeHTTP(0x14000476820, {0x103570030?, 0x1400018c1c0}, 0x1400052e100)
	/Users/mchiang/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/gin.go:576 +0x1d4 fp=0x14000157ab0 sp=0x14000157a70 pc=0x1032e9a24
net/http.serverHandler.ServeHTTP({0x10356e1d0?}, {0x103570030, 0x1400018c1c0}, 0x1400052e100)
	/usr/local/go/src/net/http/server.go:2936 +0x2d8 fp=0x14000157b60 sp=0x14000157ab0 pc=0x10308f2e8
net/http.(*conn).serve(0x14000190240, {0x1035706a8, 0x14000475da0})
	/usr/local/go/src/net/http/server.go:1995 +0x560 fp=0x14000157fa0 sp=0x14000157b60 pc=0x10308afe0
net/http.(*Server).Serve.func3()
	/usr/local/go/src/net/http/server.go:3089 +0x30 fp=0x14000157fd0 sp=0x14000157fa0 pc=0x10308fb10
runtime.goexit()
	/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x14000157fd0 sp=0x14000157fd0 pc=0x102e76934
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:3089 +0x520

goroutine 34 [GC worker (idle)]:
runtime.gopark(0x61dba6af5d6a?, 0x0?, 0x0?, 0x0?, 0x0?)
	/usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x1400051e740 sp=0x1400051e720 pc=0x102e47584
runtime.gcBgMarkWorker()
	/usr/local/go/src/runtime/mgc.go:1275 +0xec fp=0x1400051e7d0 sp=0x1400051e740 pc=0x102e2ae4c
runtime.goexit()
	/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x1400051e7d0 sp=0x1400051e7d0 pc=0x102e76934
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1199 +0x28

goroutine 35 [GC worker (idle)]:
runtime.gopark(0x61dba6af6564?, 0x3?, 0x8d?, 0x1b?, 0x0?)
	/usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x1400051ef40 sp=0x1400051ef20 pc=0x102e47584
runtime.gcBgMarkWorker()
	/usr/local/go/src/runtime/mgc.go:1275 +0xec fp=0x1400051efd0 sp=0x1400051ef40 pc=0x102e2ae4c
runtime.goexit()
	/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x1400051efd0 sp=0x1400051efd0 pc=0x102e76934
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1199 +0x28

goroutine 22 [GC worker (idle)]:
runtime.gopark(0x61dba9264e16?, 0x1?, 0xcf?, 0x9b?, 0x0?)
	/usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x14000050f40 sp=0x14000050f20 pc=0x102e47584
runtime.gcBgMarkWorker()
	/usr/local/go/src/runtime/mgc.go:1275 +0xec fp=0x14000050fd0 sp=0x14000050f40 pc=0x102e2ae4c
runtime.goexit()
	/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x14000050fd0 sp=0x14000050fd0 pc=0x102e76934
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1199 +0x28

goroutine 5 [GC worker (idle)]:
runtime.gopark(0x61dba926079c?, 0x3?, 0x3f?, 0x28?, 0x0?)
	/usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x14000056740 sp=0x14000056720 pc=0x102e47584
runtime.gcBgMarkWorker()
	/usr/local/go/src/runtime/mgc.go:1275 +0xec fp=0x140000567d0 sp=0x14000056740 pc=0x102e2ae4c
runtime.goexit()
	/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x140000567d0 sp=0x140000567d0 pc=0x102e76934
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1199 +0x28

goroutine 36 [GC worker (idle)]:
runtime.gopark(0x61dba92cf124?, 0x3?, 0x30?, 0xeb?, 0x0?)
	/usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x1400051f740 sp=0x1400051f720 pc=0x102e47584
runtime.gcBgMarkWorker()
	/usr/local/go/src/runtime/mgc.go:1275 +0xec fp=0x1400051f7d0 sp=0x1400051f740 pc=0x102e2ae4c
runtime.goexit()
	/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x1400051f7d0 sp=0x1400051f7d0 pc=0x102e76934
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1199 +0x28

goroutine 23 [GC worker (idle)]:
runtime.gopark(0x61dba9263847?, 0x1?, 0xeb?, 0x29?, 0x0?)
	/usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x14000051740 sp=0x14000051720 pc=0x102e47584
runtime.gcBgMarkWorker()
	/usr/local/go/src/runtime/mgc.go:1275 +0xec fp=0x140000517d0 sp=0x14000051740 pc=0x102e2ae4c
runtime.goexit()
	/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x140000517d0 sp=0x140000517d0 pc=0x102e76934
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1199 +0x28

goroutine 6 [GC worker (idle)]:
runtime.gopark(0x61dba6af5888?, 0x1?, 0x19?, 0x56?, 0x0?)
	/usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x14000056f40 sp=0x14000056f20 pc=0x102e47584
runtime.gcBgMarkWorker()
	/usr/local/go/src/runtime/mgc.go:1275 +0xec fp=0x14000056fd0 sp=0x14000056f40 pc=0x102e2ae4c
runtime.goexit()
	/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x14000056fd0 sp=0x14000056fd0 pc=0x102e76934
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1199 +0x28

goroutine 37 [GC worker (idle)]:
runtime.gopark(0x61dba92cef07?, 0x3?, 0xdb?, 0xe?, 0x0?)
	/usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x1400051ff40 sp=0x1400051ff20 pc=0x102e47584
runtime.gcBgMarkWorker()
	/usr/local/go/src/runtime/mgc.go:1275 +0xec fp=0x1400051ffd0 sp=0x1400051ff40 pc=0x102e2ae4c
runtime.goexit()
	/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x1400051ffd0 sp=0x1400051ffd0 pc=0x102e76934
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1199 +0x28

goroutine 24 [GC worker (idle)]:
runtime.gopark(0x61dba9260197?, 0x1?, 0xdb?, 0xd3?, 0x0?)
	/usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x14000051f40 sp=0x14000051f20 pc=0x102e47584
runtime.gcBgMarkWorker()
	/usr/local/go/src/runtime/mgc.go:1275 +0xec fp=0x14000051fd0 sp=0x14000051f40 pc=0x102e2ae4c
runtime.goexit()
	/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x14000051fd0 sp=0x14000051fd0 pc=0x102e76934
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1199 +0x28

goroutine 38 [GC worker (idle)]:
runtime.gopark(0x61dba9260c55?, 0x3?, 0xd1?, 0x71?, 0x0?)
	/usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x14000520740 sp=0x14000520720 pc=0x102e47584
runtime.gcBgMarkWorker()
	/usr/local/go/src/runtime/mgc.go:1275 +0xec fp=0x140005207d0 sp=0x14000520740 pc=0x102e2ae4c
runtime.goexit()
	/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x140005207d0 sp=0x140005207d0 pc=0x102e76934
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1199 +0x28

goroutine 66 [IO wait]:
runtime.gopark(0xffffffffffffffff?, 0xffffffffffffffff?, 0x23?, 0x0?, 0x102e88b10?)
	/usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x14000520d40 sp=0x14000520d20 pc=0x102e47584
runtime.netpollblock(0x0?, 0x0?, 0x0?)
	/usr/local/go/src/runtime/netpoll.go:527 +0x158 fp=0x14000520d80 sp=0x14000520d40 pc=0x102e40c38
internal/poll.runtime_pollWait(0x12b00ca28, 0x72)
	/usr/local/go/src/runtime/netpoll.go:306 +0xa0 fp=0x14000520db0 sp=0x14000520d80 pc=0x102e707c0
internal/poll.(*pollDesc).wait(0x140004ee000?, 0x1400009ee81?, 0x0)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x28 fp=0x14000520de0 sp=0x14000520db0 pc=0x102ef4db8
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0x140004ee000, {0x1400009ee81, 0x1, 0x1})
	/usr/local/go/src/internal/poll/fd_unix.go:167 +0x200 fp=0x14000520e80 sp=0x14000520de0 pc=0x102ef6120
net.(*netFD).Read(0x140004ee000, {0x1400009ee81?, 0x0?, 0x0?})
	/usr/local/go/src/net/fd_posix.go:55 +0x28 fp=0x14000520ed0 sp=0x14000520e80 pc=0x102f36a98
net.(*conn).Read(0x14000010008, {0x1400009ee81?, 0x0?, 0x0?})
	/usr/local/go/src/net/net.go:183 +0x34 fp=0x14000520f20 sp=0x14000520ed0 pc=0x102f450a4
net.(*TCPConn).Read(0x0?, {0x1400009ee81?, 0x0?, 0x0?})
	<autogenerated>:1 +0x2c fp=0x14000520f50 sp=0x14000520f20 pc=0x102f575ec
net/http.(*connReader).backgroundRead(0x1400009ee70)
	/usr/local/go/src/net/http/server.go:674 +0x44 fp=0x14000520fb0 sp=0x14000520f50 pc=0x103085494
net/http.(*connReader).startBackgroundRead.func2()
	/usr/local/go/src/net/http/server.go:670 +0x28 fp=0x14000520fd0 sp=0x14000520fb0 pc=0x1030853b8
runtime.goexit()
	/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x14000520fd0 sp=0x14000520fd0 pc=0x102e76934
created by net/http.(*connReader).startBackgroundRead
	/usr/local/go/src/net/http/server.go:670 +0xcc
<!-- gh-comment-id:1640782760 --> @mchiang0610 commented on GitHub (Jul 18, 2023): Related to the large context size; this is after increasing n_ctx parameter to 8192. ``` [GIN] 2023/07/18 - 11:01:38 | 200 | 4.261792ms | 127.0.0.1 | POST "/api/pull" llama.cpp: loading model from /Users/mchiang/.ollama/models/blobs/sha256:e84705205f71dd55be7b24a778f248f0eda9999a125d313358c087e092d83148 llama_model_load_internal: format = ggjt v3 (latest) llama_model_load_internal: n_vocab = 32000 llama_model_load_internal: n_ctx = 8192 llama_model_load_internal: n_embd = 3200 llama_model_load_internal: n_mult = 240 llama_model_load_internal: n_head = 32 llama_model_load_internal: n_layer = 26 llama_model_load_internal: n_rot = 100 llama_model_load_internal: ftype = 2 (mostly Q4_0) llama_model_load_internal: n_ff = 8640 llama_model_load_internal: model size = 3B llama_model_load_internal: ggml ctx size = 0.06 MB llama_model_load_internal: mem required = 2862.72 MB (+ 682.00 MB per state) llama_new_context_with_model: kv self size = 2600.00 MB ggml_metal_init: allocating ggml_metal_init: using MPS ggml_metal_init: loading '/Users/mchiang/Downloads/github/ollama/ggml-metal.metal' ggml_metal_init: loaded kernel_add 0x13f809930 ggml_metal_init: loaded kernel_mul 0x13f80a4e0 ggml_metal_init: loaded kernel_mul_row 0x13f80adb0 ggml_metal_init: loaded kernel_scale 0x13f80b610 ggml_metal_init: loaded kernel_silu 0x13f80bde0 ggml_metal_init: loaded kernel_relu 0x13f80c5b0 ggml_metal_init: loaded kernel_gelu 0x13f80cd80 ggml_metal_init: loaded kernel_soft_max 0x13f80d2d0 ggml_metal_init: loaded kernel_diag_mask_inf 0x13f80d820 ggml_metal_init: loaded kernel_get_rows_f16 0x13ef058b0 ggml_metal_init: loaded kernel_get_rows_q4_0 0x13ef05fb0 ggml_metal_init: loaded kernel_get_rows_q4_1 0x13ef06670 ggml_metal_init: loaded kernel_get_rows_q2_K 0x13ef06bc0 ggml_metal_init: loaded kernel_get_rows_q3_K 0x13ef07110 ggml_metal_init: loaded kernel_get_rows_q4_K 0x13ef07660 ggml_metal_init: loaded kernel_get_rows_q5_K 0x13ef07bb0 ggml_metal_init: loaded kernel_get_rows_q6_K 0x13ef08100 ggml_metal_init: loaded kernel_rms_norm 0x13ef08650 ggml_metal_init: loaded kernel_norm 0x13ef08ba0 ggml_metal_init: loaded kernel_mul_mat_f16_f32 0x13ef09260 ggml_metal_init: loaded kernel_mul_mat_q4_0_f32 0x13ef097b0 ggml_metal_init: loaded kernel_mul_mat_q4_1_f32 0x13ef09d00 ggml_metal_init: loaded kernel_mul_mat_q2_K_f32 0x13ef0a250 ggml_metal_init: loaded kernel_mul_mat_q3_K_f32 0x13f80ddd0 ggml_metal_init: loaded kernel_mul_mat_q4_K_f32 0x13f80e4a0 ggml_metal_init: loaded kernel_mul_mat_q5_K_f32 0x13ef0a680 ggml_metal_init: loaded kernel_mul_mat_q6_K_f32 0x13f904f90 ggml_metal_init: loaded kernel_rope 0x13f9057b0 ggml_metal_init: loaded kernel_alibi_f32 0x13ee051d0 ggml_metal_init: loaded kernel_cpy_f32_f16 0x13ee05890 ggml_metal_init: loaded kernel_cpy_f32_f32 0x13f905be0 ggml_metal_init: loaded kernel_cpy_f16_f16 0x13f9062a0 ggml_metal_init: recommendedMaxWorkingSetSize = 10922.67 MB ggml_metal_init: hasUnifiedMemory = true ggml_metal_init: maxTransferRate = built-in GPU llama_new_context_with_model: max tensor size = 54.93 MB ggml_metal_add_buffer: allocated 'data ' buffer, size = 1839.12 MB, ( 1839.52 / 10922.67) ggml_metal_add_buffer: allocated 'eval ' buffer, size = 512.00 MB, ( 2351.52 / 10922.67) ggml_metal_add_buffer: allocated 'kv ' buffer, size = 2602.00 MB, ( 4953.52 / 10922.67) ggml_metal_add_buffer: allocated 'scr0 ' buffer, size = 256.00 MB, ( 5209.52 / 10922.67) ggml_metal_add_buffer: allocated 'scr1 ' buffer, size = 256.00 MB, ( 5465.52 / 10922.67) llama_print_timings: load time = 5445.45 ms llama_print_timings: sample time = 832.48 ms / 345 runs ( 2.41 ms per token, 414.42 tokens per second) llama_print_timings: prompt eval time = 9827.04 ms / 330 tokens ( 29.78 ms per token, 33.58 tokens per second) llama_print_timings: eval time = 10982.84 ms / 344 runs ( 31.93 ms per token, 31.32 tokens per second) llama_print_timings: total time = 25044.42 ms ggml_metal_free: deallocating [GIN] 2023/07/18 - 11:02:11 | 200 | 30.556039791s | 127.0.0.1 | POST "/api/generate" llama.cpp: loading model from /Users/mchiang/.ollama/models/blobs/sha256:e84705205f71dd55be7b24a778f248f0eda9999a125d313358c087e092d83148 llama_model_load_internal: format = ggjt v3 (latest) llama_model_load_internal: n_vocab = 32000 llama_model_load_internal: n_ctx = 8192 llama_model_load_internal: n_embd = 3200 llama_model_load_internal: n_mult = 240 llama_model_load_internal: n_head = 32 llama_model_load_internal: n_layer = 26 llama_model_load_internal: n_rot = 100 llama_model_load_internal: ftype = 2 (mostly Q4_0) llama_model_load_internal: n_ff = 8640 llama_model_load_internal: model size = 3B llama_model_load_internal: ggml ctx size = 0.06 MB llama_model_load_internal: mem required = 2862.72 MB (+ 682.00 MB per state) llama_new_context_with_model: kv self size = 2600.00 MB ggml_metal_init: allocating ggml_metal_init: using MPS ggml_metal_init: loading '/Users/mchiang/Downloads/github/ollama/ggml-metal.metal' ggml_metal_init: loaded kernel_add 0x13f9572a0 ggml_metal_init: loaded kernel_mul 0x13f9569e0 ggml_metal_init: loaded kernel_mul_row 0x13f906710 ggml_metal_init: loaded kernel_scale 0x13f906980 ggml_metal_init: loaded kernel_silu 0x13f957870 ggml_metal_init: loaded kernel_relu 0x13f957ae0 ggml_metal_init: loaded kernel_gelu 0x13f957d50 ggml_metal_init: loaded kernel_soft_max 0x13f957fc0 ggml_metal_init: loaded kernel_diag_mask_inf 0x13f958230 ggml_metal_init: loaded kernel_get_rows_f16 0x13f9584a0 ggml_metal_init: loaded kernel_get_rows_q4_0 0x13f958710 ggml_metal_init: loaded kernel_get_rows_q4_1 0x13f958980 ggml_metal_init: loaded kernel_get_rows_q2_K 0x13f958bf0 ggml_metal_init: loaded kernel_get_rows_q3_K 0x13f958e60 ggml_metal_init: loaded kernel_get_rows_q4_K 0x13f9590d0 ggml_metal_init: loaded kernel_get_rows_q5_K 0x13f959340 ggml_metal_init: loaded kernel_get_rows_q6_K 0x13f9595b0 ggml_metal_init: loaded kernel_rms_norm 0x13f959820 ggml_metal_init: loaded kernel_norm 0x13f959a90 ggml_metal_init: loaded kernel_mul_mat_f16_f32 0x13f959d00 ggml_metal_init: loaded kernel_mul_mat_q4_0_f32 0x13dbb8770 ggml_metal_init: loaded kernel_mul_mat_q4_1_f32 0x13dbb7eb0 ggml_metal_init: loaded kernel_mul_mat_q2_K_f32 0x13dbb8d40 ggml_metal_init: loaded kernel_mul_mat_q3_K_f32 0x13dbb8fb0 ggml_metal_init: loaded kernel_mul_mat_q4_K_f32 0x13dbb9220 ggml_metal_init: loaded kernel_mul_mat_q5_K_f32 0x13dbb9490 ggml_metal_init: loaded kernel_mul_mat_q6_K_f32 0x13dbb9700 ggml_metal_init: loaded kernel_rope 0x13dbb9970 ggml_metal_init: loaded kernel_alibi_f32 0x13dbb9be0 ggml_metal_init: loaded kernel_cpy_f32_f16 0x13dbb9e50 ggml_metal_init: loaded kernel_cpy_f32_f32 0x12ff15af0 ggml_metal_init: loaded kernel_cpy_f16_f16 0x12ff15230 ggml_metal_init: recommendedMaxWorkingSetSize = 10922.67 MB ggml_metal_init: hasUnifiedMemory = true ggml_metal_init: maxTransferRate = built-in GPU llama_new_context_with_model: max tensor size = 54.93 MB ggml_metal_add_buffer: allocated 'data ' buffer, size = 1839.12 MB, ( 1839.50 / 10922.67) ggml_metal_add_buffer: allocated 'eval ' buffer, size = 512.00 MB, ( 2351.50 / 10922.67) ggml_metal_add_buffer: allocated 'kv ' buffer, size = 2602.00 MB, ( 4953.50 / 10922.67) ggml_metal_add_buffer: allocated 'scr0 ' buffer, size = 256.00 MB, ( 5209.50 / 10922.67) ggml_metal_add_buffer: allocated 'scr1 ' buffer, size = 256.00 MB, ( 5465.50 / 10922.67) llama_print_timings: load time = 3873.08 ms llama_print_timings: sample time = 846.68 ms / 350 runs ( 2.42 ms per token, 413.38 tokens per second) llama_print_timings: prompt eval time = 20146.86 ms / 690 tokens ( 29.20 ms per token, 34.25 tokens per second) llama_print_timings: eval time = 12752.31 ms / 349 runs ( 36.54 ms per token, 27.37 tokens per second) llama_print_timings: total time = 34234.22 ms ggml_metal_free: deallocating [GIN] 2023/07/18 - 11:02:51 | 200 | 38.20611025s | 127.0.0.1 | POST "/api/generate" llama.cpp: loading model from /Users/mchiang/.ollama/models/blobs/sha256:e84705205f71dd55be7b24a778f248f0eda9999a125d313358c087e092d83148 llama_model_load_internal: format = ggjt v3 (latest) llama_model_load_internal: n_vocab = 32000 llama_model_load_internal: n_ctx = 8192 llama_model_load_internal: n_embd = 3200 llama_model_load_internal: n_mult = 240 llama_model_load_internal: n_head = 32 llama_model_load_internal: n_layer = 26 llama_model_load_internal: n_rot = 100 llama_model_load_internal: ftype = 2 (mostly Q4_0) llama_model_load_internal: n_ff = 8640 llama_model_load_internal: model size = 3B llama_model_load_internal: ggml ctx size = 0.06 MB llama_model_load_internal: mem required = 2862.72 MB (+ 682.00 MB per state) llama_new_context_with_model: kv self size = 2600.00 MB ggml_metal_init: allocating ggml_metal_init: using MPS ggml_metal_init: loading '/Users/mchiang/Downloads/github/ollama/ggml-metal.metal' ggml_metal_init: loaded kernel_add 0x143888980 ggml_metal_init: loaded kernel_mul 0x1438880c0 ggml_metal_init: loaded kernel_mul_row 0x143888f50 ggml_metal_init: loaded kernel_scale 0x1438891c0 ggml_metal_init: loaded kernel_silu 0x143889430 ggml_metal_init: loaded kernel_relu 0x1438896a0 ggml_metal_init: loaded kernel_gelu 0x143889910 ggml_metal_init: loaded kernel_soft_max 0x143889b80 ggml_metal_init: loaded kernel_diag_mask_inf 0x143889df0 ggml_metal_init: loaded kernel_get_rows_f16 0x14388a060 ggml_metal_init: loaded kernel_get_rows_q4_0 0x14388a2d0 ggml_metal_init: loaded kernel_get_rows_q4_1 0x14388a540 ggml_metal_init: loaded kernel_get_rows_q2_K 0x14388a7b0 ggml_metal_init: loaded kernel_get_rows_q3_K 0x14388aa20 ggml_metal_init: loaded kernel_get_rows_q4_K 0x14388ac90 ggml_metal_init: loaded kernel_get_rows_q5_K 0x14388af00 ggml_metal_init: loaded kernel_get_rows_q6_K 0x14388b170 ggml_metal_init: loaded kernel_rms_norm 0x14388b3e0 ggml_metal_init: loaded kernel_norm 0x14388b650 ggml_metal_init: loaded kernel_mul_mat_f16_f32 0x14388b8c0 ggml_metal_init: loaded kernel_mul_mat_q4_0_f32 0x14388bb30 ggml_metal_init: loaded kernel_mul_mat_q4_1_f32 0x14388bda0 ggml_metal_init: loaded kernel_mul_mat_q2_K_f32 0x14388c010 ggml_metal_init: loaded kernel_mul_mat_q3_K_f32 0x14388c280 ggml_metal_init: loaded kernel_mul_mat_q4_K_f32 0x14388c4f0 ggml_metal_init: loaded kernel_mul_mat_q5_K_f32 0x14388c760 ggml_metal_init: loaded kernel_mul_mat_q6_K_f32 0x14388c9d0 ggml_metal_init: loaded kernel_rope 0x14388cc40 ggml_metal_init: loaded kernel_alibi_f32 0x14388ceb0 ggml_metal_init: loaded kernel_cpy_f32_f16 0x14388d120 ggml_metal_init: loaded kernel_cpy_f32_f32 0x14388d390 ggml_metal_init: loaded kernel_cpy_f16_f16 0x14388d600 ggml_metal_init: recommendedMaxWorkingSetSize = 10922.67 MB ggml_metal_init: hasUnifiedMemory = true ggml_metal_init: maxTransferRate = built-in GPU llama_new_context_with_model: max tensor size = 54.93 MB ggml_metal_add_buffer: allocated 'data ' buffer, size = 1839.12 MB, ( 1839.50 / 10922.67) ggml_metal_add_buffer: allocated 'eval ' buffer, size = 512.00 MB, ( 2351.50 / 10922.67) ggml_metal_add_buffer: allocated 'kv ' buffer, size = 2602.00 MB, ( 4953.50 / 10922.67) ggml_metal_add_buffer: allocated 'scr0 ' buffer, size = 256.00 MB, ( 5209.50 / 10922.67) ggml_metal_add_buffer: allocated 'scr1 ' buffer, size = 256.00 MB, ( 5465.50 / 10922.67) llama_print_timings: load time = 2970.37 ms llama_print_timings: sample time = 679.80 ms / 312 runs ( 2.18 ms per token, 458.96 tokens per second) llama_print_timings: prompt eval time = 30586.76 ms / 1055 tokens ( 28.99 ms per token, 34.49 tokens per second) llama_print_timings: eval time = 12229.11 ms / 311 runs ( 39.32 ms per token, 25.43 tokens per second) llama_print_timings: total time = 44055.38 ms ggml_metal_free: deallocating [GIN] 2023/07/18 - 11:03:40 | 200 | 47.156634s | 127.0.0.1 | POST "/api/generate" llama.cpp: loading model from /Users/mchiang/.ollama/models/blobs/sha256:e84705205f71dd55be7b24a778f248f0eda9999a125d313358c087e092d83148 llama_model_load_internal: format = ggjt v3 (latest) llama_model_load_internal: n_vocab = 32000 llama_model_load_internal: n_ctx = 8192 llama_model_load_internal: n_embd = 3200 llama_model_load_internal: n_mult = 240 llama_model_load_internal: n_head = 32 llama_model_load_internal: n_layer = 26 llama_model_load_internal: n_rot = 100 llama_model_load_internal: ftype = 2 (mostly Q4_0) llama_model_load_internal: n_ff = 8640 llama_model_load_internal: model size = 3B llama_model_load_internal: ggml ctx size = 0.06 MB llama_model_load_internal: mem required = 2862.72 MB (+ 682.00 MB per state) llama_new_context_with_model: kv self size = 2600.00 MB ggml_metal_init: allocating ggml_metal_init: using MPS ggml_metal_init: loading '/Users/mchiang/Downloads/github/ollama/ggml-metal.metal' ggml_metal_init: loaded kernel_add 0x1445cbdc0 ggml_metal_init: loaded kernel_mul 0x1445cb500 ggml_metal_init: loaded kernel_mul_row 0x1445cc390 ggml_metal_init: loaded kernel_scale 0x1445cc600 ggml_metal_init: loaded kernel_silu 0x1445cc870 ggml_metal_init: loaded kernel_relu 0x1445ccae0 ggml_metal_init: loaded kernel_gelu 0x1445ccd50 ggml_metal_init: loaded kernel_soft_max 0x1445ccfc0 ggml_metal_init: loaded kernel_diag_mask_inf 0x1445cd230 ggml_metal_init: loaded kernel_get_rows_f16 0x1445cd4a0 ggml_metal_init: loaded kernel_get_rows_q4_0 0x1445cd710 ggml_metal_init: loaded kernel_get_rows_q4_1 0x1445cd980 ggml_metal_init: loaded kernel_get_rows_q2_K 0x1445cdbf0 ggml_metal_init: loaded kernel_get_rows_q3_K 0x1445cde60 ggml_metal_init: loaded kernel_get_rows_q4_K 0x1445ce0d0 ggml_metal_init: loaded kernel_get_rows_q5_K 0x144e6bf00 ggml_metal_init: loaded kernel_get_rows_q6_K 0x1445ce340 ggml_metal_init: loaded kernel_rms_norm 0x1445ce5b0 ggml_metal_init: loaded kernel_norm 0x1445ce820 ggml_metal_init: loaded kernel_mul_mat_f16_f32 0x1445cea90 ggml_metal_init: loaded kernel_mul_mat_q4_0_f32 0x1445ced00 ggml_metal_init: loaded kernel_mul_mat_q4_1_f32 0x1445cef70 ggml_metal_init: loaded kernel_mul_mat_q2_K_f32 0x1445cf1e0 ggml_metal_init: loaded kernel_mul_mat_q3_K_f32 0x1445cf450 ggml_metal_init: loaded kernel_mul_mat_q4_K_f32 0x1445cf6c0 ggml_metal_init: loaded kernel_mul_mat_q5_K_f32 0x1445cf930 ggml_metal_init: loaded kernel_mul_mat_q6_K_f32 0x1445cfba0 ggml_metal_init: loaded kernel_rope 0x1445cfe10 ggml_metal_init: loaded kernel_alibi_f32 0x1445d0080 ggml_metal_init: loaded kernel_cpy_f32_f16 0x1445d02f0 ggml_metal_init: loaded kernel_cpy_f32_f32 0x1445d0560 ggml_metal_init: loaded kernel_cpy_f16_f16 0x1445d07d0 ggml_metal_init: recommendedMaxWorkingSetSize = 10922.67 MB ggml_metal_init: hasUnifiedMemory = true ggml_metal_init: maxTransferRate = built-in GPU llama_new_context_with_model: max tensor size = 54.93 MB ggml_metal_add_buffer: allocated 'data ' buffer, size = 1839.12 MB, ( 1839.50 / 10922.67) ggml_metal_add_buffer: allocated 'eval ' buffer, size = 512.00 MB, ( 2351.50 / 10922.67) ggml_metal_add_buffer: allocated 'kv ' buffer, size = 2602.00 MB, ( 4953.50 / 10922.67) ggml_metal_add_buffer: allocated 'scr0 ' buffer, size = 256.00 MB, ( 5209.50 / 10922.67) ggml_metal_add_buffer: allocated 'scr1 ' buffer, size = 256.00 MB, ( 5465.50 / 10922.67) ggml_new_tensor_impl: not enough space in the scratch memory pool (needed 336026880, available 268435456) fatal error: unexpected signal during runtime execution [signal SIGSEGV: segmentation violation code=0x2 addr=0x50 pc=0x10330775c] runtime stack: runtime.throw({0x103377dd6?, 0x17101ed00?}) /usr/local/go/src/runtime/panic.go:1047 +0x40 fp=0x17101ec50 sp=0x17101ec20 pc=0x102e44b30 runtime.sigpanic() /usr/local/go/src/runtime/signal_unix.go:821 +0x244 fp=0x17101ec90 sp=0x17101ec50 pc=0x102e5b424 goroutine 67 [syscall]: runtime.cgocall(0x103302a00, 0x14000062bb8) /usr/local/go/src/runtime/cgocall.go:157 +0x54 fp=0x14000062b80 sp=0x14000062b40 pc=0x102e14494 github.com/jmorganca/ollama/llama._Cfunc_llama_eval(0x140049600, 0x140005f8000, 0x56d, 0x1, 0xa) _cgo_gotypes.go:208 +0x38 fp=0x14000062bb0 sp=0x14000062b80 pc=0x1032f4698 github.com/jmorganca/ollama/llama.(*llama).generate.func2(0x140002761e0, 0x4?) /Users/mchiang/Downloads/github/ollama/llama/llama.go:211 +0xa0 fp=0x14000062c20 sp=0x14000062bb0 pc=0x1032f67d0 github.com/jmorganca/ollama/llama.(*llama).generate(0x140002761e0, {0x140005f8000, 0x56d, 0x800}, 0x14000062f90) /Users/mchiang/Downloads/github/ollama/llama/llama.go:211 +0x178 fp=0x14000062ec0 sp=0x14000062c20 pc=0x1032f6248 github.com/jmorganca/ollama/llama.(*llama).Predict(0x140002761e0?, {0x140006b0000, 0x556, 0x63f?}, {0x1400045e0a0?, 0x1b?}, 0x1400051af90?) /Users/mchiang/Downloads/github/ollama/llama/llama.go:161 +0x11c fp=0x14000062f30 sp=0x14000062ec0 pc=0x1032f5b3c github.com/jmorganca/ollama/server.generate.func1() /Users/mchiang/Downloads/github/ollama/server/routes.go:80 +0xa8 fp=0x14000062fd0 sp=0x14000062f30 pc=0x1032ff228 runtime.goexit() /usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x14000062fd0 sp=0x14000062fd0 pc=0x102e76934 created by github.com/jmorganca/ollama/server.generate /Users/mchiang/Downloads/github/ollama/server/routes.go:78 +0x8b0 goroutine 1 [IO wait, 2 minutes]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x140000c3860 sp=0x140000c3840 pc=0x102e47584 runtime.netpollblock(0x140000c38f8?, 0x2ef9774?, 0x1?) /usr/local/go/src/runtime/netpoll.go:527 +0x158 fp=0x140000c38a0 sp=0x140000c3860 pc=0x102e40c38 internal/poll.runtime_pollWait(0x12b00cb18, 0x72) /usr/local/go/src/runtime/netpoll.go:306 +0xa0 fp=0x140000c38d0 sp=0x140000c38a0 pc=0x102e707c0 internal/poll.(*pollDesc).wait(0x14000482380?, 0x102e1ce18?, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x28 fp=0x140000c3900 sp=0x140000c38d0 pc=0x102ef4db8 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:89 internal/poll.(*FD).Accept(0x14000482380) /usr/local/go/src/internal/poll/fd_unix.go:614 +0x250 fp=0x140000c39b0 sp=0x140000c3900 pc=0x102ef9860 net.(*netFD).accept(0x14000482380) /usr/local/go/src/net/fd_unix.go:172 +0x28 fp=0x140000c3a70 sp=0x140000c39b0 pc=0x102f38738 net.(*TCPListener).accept(0x140000aed38) /usr/local/go/src/net/tcpsock_posix.go:148 +0x28 fp=0x140000c3aa0 sp=0x140000c3a70 pc=0x102f4dd38 net.(*TCPListener).Accept(0x140000aed38) /usr/local/go/src/net/tcpsock.go:297 +0x2c fp=0x140000c3ae0 sp=0x140000c3aa0 pc=0x102f4ceac net/http.(*onceCloseListener).Accept(0x14000190240?) <autogenerated>:1 +0x30 fp=0x140000c3b00 sp=0x140000c3ae0 pc=0x1030b3290 net/http.(*Server).Serve(0x14000308ff0, {0x10356fe20, 0x140000aed38}) /usr/local/go/src/net/http/server.go:3059 +0x304 fp=0x140000c3c30 sp=0x140000c3b00 pc=0x10308f7b4 github.com/jmorganca/ollama/server.Serve({0x10356fe20, 0x140000aed38}) /Users/mchiang/Downloads/github/ollama/server/routes.go:201 +0x20c fp=0x140000c3ca0 sp=0x140000c3c30 pc=0x10330062c github.com/jmorganca/ollama/cmd.RunServer(0x14000425200?, {0x1033532ac?, 0x0?, 0x0?}) /Users/mchiang/Downloads/github/ollama/cmd/cmd.go:245 +0x114 fp=0x140000c3d20 sp=0x140000c3ca0 pc=0x103302194 github.com/spf13/cobra.(*Command).execute(0x14000425200, {0x1039cd908, 0x0, 0x0}) /Users/mchiang/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:940 +0x5c8 fp=0x140000c3e60 sp=0x140000c3d20 pc=0x103122cd8 github.com/spf13/cobra.(*Command).ExecuteC(0x14000424900) /Users/mchiang/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:1068 +0x35c fp=0x140000c3f20 sp=0x140000c3e60 pc=0x10312342c github.com/spf13/cobra.(*Command).Execute(...) /Users/mchiang/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:992 github.com/spf13/cobra.(*Command).ExecuteContext(0x14000054768?, {0x103570638?, 0x140000a0010?}) /Users/mchiang/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:985 +0x50 fp=0x140000c3f40 sp=0x140000c3f20 pc=0x103122fc0 main.main() /Users/mchiang/Downloads/github/ollama/main.go:10 +0x34 fp=0x140000c3f70 sp=0x140000c3f40 pc=0x1033027d4 runtime.main() /usr/local/go/src/runtime/proc.go:250 +0x248 fp=0x140000c3fd0 sp=0x140000c3f70 pc=0x102e47158 runtime.goexit() /usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x140000c3fd0 sp=0x140000c3fd0 pc=0x102e76934 goroutine 2 [force gc (idle), 2 minutes]: runtime.gopark(0x61d96659a2f6?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x14000054fa0 sp=0x14000054f80 pc=0x102e47584 runtime.goparkunlock(...) /usr/local/go/src/runtime/proc.go:387 runtime.forcegchelper() /usr/local/go/src/runtime/proc.go:305 +0xb8 fp=0x14000054fd0 sp=0x14000054fa0 pc=0x102e473c8 runtime.goexit() /usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x14000054fd0 sp=0x14000054fd0 pc=0x102e76934 created by runtime.init.6 /usr/local/go/src/runtime/proc.go:293 +0x24 goroutine 3 [GC sweep wait]: runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x14000055760 sp=0x14000055740 pc=0x102e47584 runtime.goparkunlock(...) /usr/local/go/src/runtime/proc.go:387 runtime.bgsweep(0x0?) /usr/local/go/src/runtime/mgcsweep.go:319 +0x110 fp=0x140000557b0 sp=0x14000055760 pc=0x102e34460 runtime.gcenable.func1() /usr/local/go/src/runtime/mgc.go:178 +0x28 fp=0x140000557d0 sp=0x140000557b0 pc=0x102e28f08 runtime.goexit() /usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x140000557d0 sp=0x140000557d0 pc=0x102e76934 created by runtime.gcenable /usr/local/go/src/runtime/mgc.go:178 +0x74 goroutine 4 [GC scavenge wait]: runtime.gopark(0x1e6ff8d?, 0x1d7328c?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x14000055f50 sp=0x14000055f30 pc=0x102e47584 runtime.goparkunlock(...) /usr/local/go/src/runtime/proc.go:387 runtime.(*scavengerState).park(0x103912620) /usr/local/go/src/runtime/mgcscavenge.go:400 +0x5c fp=0x14000055f80 sp=0x14000055f50 pc=0x102e3226c runtime.bgscavenge(0x0?) /usr/local/go/src/runtime/mgcscavenge.go:633 +0xac fp=0x14000055fb0 sp=0x14000055f80 pc=0x102e3284c runtime.gcenable.func2() /usr/local/go/src/runtime/mgc.go:179 +0x28 fp=0x14000055fd0 sp=0x14000055fb0 pc=0x102e28ea8 runtime.goexit() /usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x14000055fd0 sp=0x14000055fd0 pc=0x102e76934 created by runtime.gcenable /usr/local/go/src/runtime/mgc.go:179 +0xb8 goroutine 18 [finalizer wait]: runtime.gopark(0x0?, 0x103569680?, 0x0?, 0x60?, 0x2000000020?) /usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x14000054580 sp=0x14000054560 pc=0x102e47584 runtime.runfinq() /usr/local/go/src/runtime/mfinal.go:193 +0x10c fp=0x140000547d0 sp=0x14000054580 pc=0x102e27f9c runtime.goexit() /usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x140000547d0 sp=0x140000547d0 pc=0x102e76934 created by runtime.createfing /usr/local/go/src/runtime/mfinal.go:163 +0x84 goroutine 25 [chan receive]: runtime.gopark(0x1?, 0x0?, 0x38?, 0x73?, 0x102e25b00?) /usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x14000157300 sp=0x140001572e0 pc=0x102e47584 runtime.chanrecv(0x140004f2060, 0x14000157418, 0x1) /usr/local/go/src/runtime/chan.go:583 +0x45c fp=0x14000157390 sp=0x14000157300 pc=0x102e16fec runtime.chanrecv2(0x140001573e8?, 0x1032ecc1c?) /usr/local/go/src/runtime/chan.go:447 +0x14 fp=0x140001573c0 sp=0x14000157390 pc=0x102e16b74 github.com/jmorganca/ollama/server.streamResponse.func1({0x12b00e4d8, 0x1400052e300}) /Users/mchiang/Downloads/github/ollama/server/routes.go:206 +0x34 fp=0x14000157430 sp=0x140001573c0 pc=0x1033006f4 github.com/gin-gonic/gin.(*Context).Stream(0x102e501f0?, 0x14000157488) /Users/mchiang/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:1090 +0x70 fp=0x14000157470 sp=0x14000157430 pc=0x1032e45d0 github.com/jmorganca/ollama/server.streamResponse(0xffffffffffffffff?, 0x0?) /Users/mchiang/Downloads/github/ollama/server/routes.go:205 +0x34 fp=0x140001574a0 sp=0x14000157470 pc=0x103300694 github.com/jmorganca/ollama/server.generate(0x1400052e300) /Users/mchiang/Downloads/github/ollama/server/routes.go:91 +0x8bc fp=0x140001576e0 sp=0x140001574a0 pc=0x1032ff06c github.com/gin-gonic/gin.(*Context).Next(...) /Users/mchiang/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174 github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1(0x1400052e300) /Users/mchiang/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/recovery.go:102 +0x7c fp=0x14000157730 sp=0x140001576e0 pc=0x1032ebd8c github.com/gin-gonic/gin.(*Context).Next(...) /Users/mchiang/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174 github.com/gin-gonic/gin.LoggerWithConfig.func1(0x1400052e300) /Users/mchiang/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/logger.go:240 +0xac fp=0x140001578e0 sp=0x14000157730 pc=0x1032eb00c github.com/gin-gonic/gin.(*Context).Next(...) /Users/mchiang/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174 github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0x14000476820, 0x1400052e300) /Users/mchiang/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/gin.go:620 +0x54c fp=0x14000157a70 sp=0x140001578e0 pc=0x1032ea11c github.com/gin-gonic/gin.(*Engine).ServeHTTP(0x14000476820, {0x103570030?, 0x1400018c1c0}, 0x1400052e100) /Users/mchiang/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/gin.go:576 +0x1d4 fp=0x14000157ab0 sp=0x14000157a70 pc=0x1032e9a24 net/http.serverHandler.ServeHTTP({0x10356e1d0?}, {0x103570030, 0x1400018c1c0}, 0x1400052e100) /usr/local/go/src/net/http/server.go:2936 +0x2d8 fp=0x14000157b60 sp=0x14000157ab0 pc=0x10308f2e8 net/http.(*conn).serve(0x14000190240, {0x1035706a8, 0x14000475da0}) /usr/local/go/src/net/http/server.go:1995 +0x560 fp=0x14000157fa0 sp=0x14000157b60 pc=0x10308afe0 net/http.(*Server).Serve.func3() /usr/local/go/src/net/http/server.go:3089 +0x30 fp=0x14000157fd0 sp=0x14000157fa0 pc=0x10308fb10 runtime.goexit() /usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x14000157fd0 sp=0x14000157fd0 pc=0x102e76934 created by net/http.(*Server).Serve /usr/local/go/src/net/http/server.go:3089 +0x520 goroutine 34 [GC worker (idle)]: runtime.gopark(0x61dba6af5d6a?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x1400051e740 sp=0x1400051e720 pc=0x102e47584 runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1275 +0xec fp=0x1400051e7d0 sp=0x1400051e740 pc=0x102e2ae4c runtime.goexit() /usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x1400051e7d0 sp=0x1400051e7d0 pc=0x102e76934 created by runtime.gcBgMarkStartWorkers /usr/local/go/src/runtime/mgc.go:1199 +0x28 goroutine 35 [GC worker (idle)]: runtime.gopark(0x61dba6af6564?, 0x3?, 0x8d?, 0x1b?, 0x0?) /usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x1400051ef40 sp=0x1400051ef20 pc=0x102e47584 runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1275 +0xec fp=0x1400051efd0 sp=0x1400051ef40 pc=0x102e2ae4c runtime.goexit() /usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x1400051efd0 sp=0x1400051efd0 pc=0x102e76934 created by runtime.gcBgMarkStartWorkers /usr/local/go/src/runtime/mgc.go:1199 +0x28 goroutine 22 [GC worker (idle)]: runtime.gopark(0x61dba9264e16?, 0x1?, 0xcf?, 0x9b?, 0x0?) /usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x14000050f40 sp=0x14000050f20 pc=0x102e47584 runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1275 +0xec fp=0x14000050fd0 sp=0x14000050f40 pc=0x102e2ae4c runtime.goexit() /usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x14000050fd0 sp=0x14000050fd0 pc=0x102e76934 created by runtime.gcBgMarkStartWorkers /usr/local/go/src/runtime/mgc.go:1199 +0x28 goroutine 5 [GC worker (idle)]: runtime.gopark(0x61dba926079c?, 0x3?, 0x3f?, 0x28?, 0x0?) /usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x14000056740 sp=0x14000056720 pc=0x102e47584 runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1275 +0xec fp=0x140000567d0 sp=0x14000056740 pc=0x102e2ae4c runtime.goexit() /usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x140000567d0 sp=0x140000567d0 pc=0x102e76934 created by runtime.gcBgMarkStartWorkers /usr/local/go/src/runtime/mgc.go:1199 +0x28 goroutine 36 [GC worker (idle)]: runtime.gopark(0x61dba92cf124?, 0x3?, 0x30?, 0xeb?, 0x0?) /usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x1400051f740 sp=0x1400051f720 pc=0x102e47584 runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1275 +0xec fp=0x1400051f7d0 sp=0x1400051f740 pc=0x102e2ae4c runtime.goexit() /usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x1400051f7d0 sp=0x1400051f7d0 pc=0x102e76934 created by runtime.gcBgMarkStartWorkers /usr/local/go/src/runtime/mgc.go:1199 +0x28 goroutine 23 [GC worker (idle)]: runtime.gopark(0x61dba9263847?, 0x1?, 0xeb?, 0x29?, 0x0?) /usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x14000051740 sp=0x14000051720 pc=0x102e47584 runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1275 +0xec fp=0x140000517d0 sp=0x14000051740 pc=0x102e2ae4c runtime.goexit() /usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x140000517d0 sp=0x140000517d0 pc=0x102e76934 created by runtime.gcBgMarkStartWorkers /usr/local/go/src/runtime/mgc.go:1199 +0x28 goroutine 6 [GC worker (idle)]: runtime.gopark(0x61dba6af5888?, 0x1?, 0x19?, 0x56?, 0x0?) /usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x14000056f40 sp=0x14000056f20 pc=0x102e47584 runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1275 +0xec fp=0x14000056fd0 sp=0x14000056f40 pc=0x102e2ae4c runtime.goexit() /usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x14000056fd0 sp=0x14000056fd0 pc=0x102e76934 created by runtime.gcBgMarkStartWorkers /usr/local/go/src/runtime/mgc.go:1199 +0x28 goroutine 37 [GC worker (idle)]: runtime.gopark(0x61dba92cef07?, 0x3?, 0xdb?, 0xe?, 0x0?) /usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x1400051ff40 sp=0x1400051ff20 pc=0x102e47584 runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1275 +0xec fp=0x1400051ffd0 sp=0x1400051ff40 pc=0x102e2ae4c runtime.goexit() /usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x1400051ffd0 sp=0x1400051ffd0 pc=0x102e76934 created by runtime.gcBgMarkStartWorkers /usr/local/go/src/runtime/mgc.go:1199 +0x28 goroutine 24 [GC worker (idle)]: runtime.gopark(0x61dba9260197?, 0x1?, 0xdb?, 0xd3?, 0x0?) /usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x14000051f40 sp=0x14000051f20 pc=0x102e47584 runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1275 +0xec fp=0x14000051fd0 sp=0x14000051f40 pc=0x102e2ae4c runtime.goexit() /usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x14000051fd0 sp=0x14000051fd0 pc=0x102e76934 created by runtime.gcBgMarkStartWorkers /usr/local/go/src/runtime/mgc.go:1199 +0x28 goroutine 38 [GC worker (idle)]: runtime.gopark(0x61dba9260c55?, 0x3?, 0xd1?, 0x71?, 0x0?) /usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x14000520740 sp=0x14000520720 pc=0x102e47584 runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1275 +0xec fp=0x140005207d0 sp=0x14000520740 pc=0x102e2ae4c runtime.goexit() /usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x140005207d0 sp=0x140005207d0 pc=0x102e76934 created by runtime.gcBgMarkStartWorkers /usr/local/go/src/runtime/mgc.go:1199 +0x28 goroutine 66 [IO wait]: runtime.gopark(0xffffffffffffffff?, 0xffffffffffffffff?, 0x23?, 0x0?, 0x102e88b10?) /usr/local/go/src/runtime/proc.go:381 +0xe4 fp=0x14000520d40 sp=0x14000520d20 pc=0x102e47584 runtime.netpollblock(0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/netpoll.go:527 +0x158 fp=0x14000520d80 sp=0x14000520d40 pc=0x102e40c38 internal/poll.runtime_pollWait(0x12b00ca28, 0x72) /usr/local/go/src/runtime/netpoll.go:306 +0xa0 fp=0x14000520db0 sp=0x14000520d80 pc=0x102e707c0 internal/poll.(*pollDesc).wait(0x140004ee000?, 0x1400009ee81?, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x28 fp=0x14000520de0 sp=0x14000520db0 pc=0x102ef4db8 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:89 internal/poll.(*FD).Read(0x140004ee000, {0x1400009ee81, 0x1, 0x1}) /usr/local/go/src/internal/poll/fd_unix.go:167 +0x200 fp=0x14000520e80 sp=0x14000520de0 pc=0x102ef6120 net.(*netFD).Read(0x140004ee000, {0x1400009ee81?, 0x0?, 0x0?}) /usr/local/go/src/net/fd_posix.go:55 +0x28 fp=0x14000520ed0 sp=0x14000520e80 pc=0x102f36a98 net.(*conn).Read(0x14000010008, {0x1400009ee81?, 0x0?, 0x0?}) /usr/local/go/src/net/net.go:183 +0x34 fp=0x14000520f20 sp=0x14000520ed0 pc=0x102f450a4 net.(*TCPConn).Read(0x0?, {0x1400009ee81?, 0x0?, 0x0?}) <autogenerated>:1 +0x2c fp=0x14000520f50 sp=0x14000520f20 pc=0x102f575ec net/http.(*connReader).backgroundRead(0x1400009ee70) /usr/local/go/src/net/http/server.go:674 +0x44 fp=0x14000520fb0 sp=0x14000520f50 pc=0x103085494 net/http.(*connReader).startBackgroundRead.func2() /usr/local/go/src/net/http/server.go:670 +0x28 fp=0x14000520fd0 sp=0x14000520fb0 pc=0x1030853b8 runtime.goexit() /usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x14000520fd0 sp=0x14000520fd0 pc=0x102e76934 created by net/http.(*connReader).startBackgroundRead /usr/local/go/src/net/http/server.go:670 +0xcc ```
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#25784
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?