mirror of
https://github.com/ollama/ollama.git
synced 2026-05-06 16:11:34 -05:00
Closed
opened 2026-04-12 16:25:19 -05:00 by GiteaMirror
·
67 comments
No Branch/Tag Specified
main
dhiltgen/ci
parth-launch-plan-gating
hoyyeva/anthropic-reference-images-path
parth-anthropic-reference-images-path
brucemacd/download-before-remove
hoyyeva/editor-config-repair
parth-mlx-decode-checkpoints
parth-launch-codex-app
hoyyeva/fix-codex-model-metadata-warning
hoyyeva/qwen
parth/hide-claude-desktop-till-release
hoyyeva/opencode-image-modality
parth-add-claude-code-autoinstall
release_v0.22.0
pdevine/manifest-list
codex/fix-codex-model-metadata-warning
pdevine/addressable-manifest
brucemacd/launch-fetch-reccomended
jmorganca/llama-compat
launch-copilot-cli
hoyyeva/opencode-thinking
release_v0.20.7
parth-auto-save-backup
parth-test
jmorganca/gemma4-audio-replacements
fix-manifest-digest-on-pull
hoyyeva/vscode-improve
brucemacd/install-server-wait
parth/update-claude-docs
brucemac/start-ap-install
pdevine/mlx-update
pdevine/qwen35_vision
drifkin/api-show-fallback
mintlify/image-generation-1773352582
hoyyeva/server-context-length-local-config
jmorganca/faster-reptition-penalties
jmorganca/convert-nemotron
parth-pi-thinking
pdevine/sampling-penalties
jmorganca/fix-create-quantization-memory
dongchen/resumable_transfer_fix
pdevine/sampling-cache-error
jessegross/mlx-usage
hoyyeva/openclaw-config
hoyyeva/app-html
pdevine/qwen3next
brucemacd/sign-sh-install
brucemacd/tui-update
brucemacd/usage-api
jmorganca/launch-empty
fix-app-dist-embed
mxyng/mlx-compile
mxyng/mlx-quant
mxyng/mlx-glm4.7
mxyng/mlx
brucemacd/simplify-model-picker
jmorganca/qwen3-concurrent
fix-glm-4.7-flash-mla-config
drifkin/qwen3-coder-opening-tag
brucemacd/usage-cli
fix-cuda12-fattn-shmem
ollama-imagegen-docs
parth/fix-multiline-inputs
brucemacd/config-docs
mxyng/model-files
mxyng/simple-execute
fix-imagegen-ollama-models
mxyng/async-upload
jmorganca/lazy-no-dtype-changes
imagegen-auto-detect-create
parth/decrease-concurrent-download-hf
fix-mlx-quantize-init
jmorganca/x-cleanup
usage
imagegen-readme
jmorganca/glm-image
mlx-gpu-cd
jmorganca/imagegen-modelfile
parth/agent-skills
parth/agent-allowlist
parth/signed-in-offline
parth/agents
parth/fix-context-chopping
improve-cloud-flow
parth/add-models-websearch
parth/prompt-renderer-mcp
jmorganca/native-settings
jmorganca/download-stream-hash
jmorganca/client2-rebased
brucemacd/oai-chat-req-multipart
jessegross/multi_chunk_reserve
grace/additional-omit-empty
grace/mistral-3-large
mxyng/tokenizer2
mxyng/tokenizer
jessegross/flash
hoyyeva/windows-nacked-app
mxyng/cleanup-attention
grace/deepseek-parser
hoyyeva/remember-unsent-prompt
parth/add-lfs-pointer-error-conversion
parth/olmo2-test2
hoyyeva/ollama-launchagent-plist
nicole/olmo-model
parth/olmo-test
mxyng/remove-embedded
parth/render-template
jmorganca/intellect-3
parth/remove-prealloc-linter
jmorganca/cmd-eval
nicole/nomic-embed-text-fix
mxyng/lint-2
hoyyeva/add-gemini-3-pro-preview
hoyyeva/load-model-list
mxyng/expand-path
mxyng/environ-2
hoyyeva/deeplink-json-encoding
parth/improve-tool-calling-tests
hoyyeva/conversation
hoyyeva/assistant-edit-response
hoyyeva/thinking
origin/brucemacd/invalid-char-i-err
parth/improve-tool-calling
jmorganca/required-omitempty
grace/qwen3-vl-tests
mxyng/iter-client
parth/docs-readme
nicole/embed-test
pdevine/integration-benchstat
parth/remove-generate-cmd
parth/add-toolcall-id
mxyng/server-tests
jmorganca/glm-4.6
jmorganca/gin-h-compat
drifkin/stable-tool-args
pdevine/qwen3-more-thinking
parth/add-websearch-client
nicole/websearch_local
jmorganca/qwen3-coder-updates
grace/deepseek-v3-migration-tests
mxyng/fix-create
jmorganca/cloud-errors
pdevine/parser-tidy
revert-12233-parth/simplify-entrypoints-runner
parth/enable-so-gpt-oss
brucemacd/qwen3vl
jmorganca/readme-simplify
parth/gpt-oss-structured-outputs
revert-12039-jmorganca/tools-braces
mxyng/embeddings
mxyng/gguf
mxyng/benchmark
mxyng/types-null
parth/move-parsing
mxyng/gemma2
jmorganca/docs
mxyng/16-bit
mxyng/create-stdin
pdevine/authorizedkeys
mxyng/quant
parth/opt-in-error-context-window
brucemacd/cache-models
brucemacd/runner-completion
jmorganca/llama-update-6
brucemacd/benchmark-list
brucemacd/partial-read-caps
parth/deepseek-r1-tools
mxyng/omit-array
parth/tool-prefix-temp
brucemacd/runner-test
jmorganca/qwen25vl
brucemacd/model-forward-test-ext
parth/python-function-parsing
jmorganca/cuda-compression-none
drifkin/num-parallel
drifkin/chat-truncation-fix
jmorganca/sync
parth/python-tools-calling
drifkin/array-head-count
brucemacd/create-no-loop
parth/server-enable-content-stream-with-tools
qwen25omni
mxyng/v3
brucemacd/ropeconfig
jmorganca/silence-tokenizer
parth/sample-so-test
parth/sampling-structured-outputs
brucemacd/doc-go-engine
parth/constrained-sampling-json
jmorganca/mistral-wip
brucemacd/mistral-small-convert
parth/sample-unmarshal-json-for-params
brucemacd/jomorganca/mistral
pdevine/bfloat16
jmorganca/mistral
brucemacd/mistral
pdevine/logging
parth/sample-correctness-fix
parth/sample-fix-sorting
jmorgan/sample-fix-sorting-extras
jmorganca/temp-0-images
brucemacd/parallel-embed-models
brucemacd/shim-grammar
jmorganca/fix-gguf-error
bmizerany/nameswork
jmorganca/faster-releases
bmizerany/validatenames
brucemacd/err-no-vocab
brucemacd/rope-config
brucemacd/err-hint
brucemacd/qwen2_5
brucemacd/logprobs
brucemacd/new_runner_graph_bench
progress-flicker
brucemacd/forward-test
brucemacd/go_qwen2
pdevine/gemma2
jmorganca/add-missing-symlink-eval
mxyng/next-debug
parth/set-context-size-openai
brucemacd/next-bpe-bench
brucemacd/next-bpe-test
brucemacd/new_runner_e2e
brucemacd/new_runner_qwen2
pdevine/convert-cohere2
brucemacd/convert-cli
parth/log-probs
mxyng/next-mlx
mxyng/cmd-history
parth/templating
parth/tokenize-detokenize
brucemacd/check-key-register
bmizerany/grammar
jmorganca/vendor-081b29bd
mxyng/func-checks
jmorganca/fix-null-format
parth/fix-default-to-warn-json
jmorganca/qwen2vl
jmorganca/no-concat
parth/cmd-cleanup-SO
brucemacd/check-key-register-structured-err
parth/openai-stream-usage
parth/fix-referencing-so
stream-tools-stop
jmorganca/degin-1
brucemacd/install-path-clean
brucemacd/push-name-validation
brucemacd/browser-key-register
jmorganca/openai-fix-first-message
jmorganca/fix-proxy
jessegross/sample
parth/disallow-streaming-tools
dhiltgen/remove_submodule
jmorganca/ga
jmorganca/mllama
pdevine/newlines
pdevine/geems-2b
jmorganca/llama-bump
mxyng/modelname-7
mxyng/gin-slog
mxyng/modelname-6
jyan/convert-prog
jyan/quant5
paligemma-support
pdevine/import-docs
jmorganca/openai-context
jyan/paligemma
jyan/p2
jyan/palitest
bmizerany/embedspeedup
jmorganca/llama-vit
brucemacd/allow-ollama
royh/ep-methods
royh/whisper
mxyng/api-models
mxyng/fix-memory
jyan/q4_4/8
jyan/ollama-v
royh/stream-tools
roy-embed-parallel
bmizerany/hrm
revert-5963-revert-5924-mxyng/llama3.1-rope
royh/embed-viz
jyan/local2
jyan/auth
jyan/local
jyan/parse-temp
jmorganca/template-mistral
jyan/reord-g
royh-openai-suffixdocs
royh-imgembed
royh-embed-parallel
jyan/quant4
royh-precision
jyan/progress
pdevine/fix-template
jyan/quant3
pdevine/ggla
mxyng/update-registry-domain
jmorganca/ggml-static
mxyng/create-context
jyan/v0.146
mxyng/layers-from-files
build_dist
bmizerany/noseek
royh-ls
royh-name
timeout
mxyng/server-timestamp
bmizerany/nosillyggufslurps
royh-params
jmorganca/llama-cpp-7c26775
royh-openai-delete
royh-show-rigid
jmorganca/enable-fa
jmorganca/no-error-template
jyan/format
royh-testdelete
bmizerany/fastverify
language_support
pdevine/ps-glitches
brucemacd/tokenize
bruce/iq-quants
bmizerany/filepathwithcoloninhost
mxyng/split-bin
bmizerany/client-registry
jmorganca/if-none-match
native
jmorganca/native
jmorganca/batch-embeddings
jmorganca/initcmake
jmorganca/mm
pdevine/showggmlinfo
modenameenforcealphanum
bmizerany/modenameenforcealphanum
jmorganca/done-reason
jmorganca/llama-cpp-8960fe8
ollama.com
bmizerany/filepathnobuild
bmizerany/types/model/defaultfix
rmdisplaylong
nogogen
bmizerany/x
modelfile-readme
bmizerany/replacecolon
jmorganca/limit
jmorganca/execstack
jmorganca/replace-assets
mxyng/tune-concurrency
jmorganca/testing
whitespace-detection
jmorganca/options
upgrade-all
scratch
cuda-search
mattw/airenamer
mattw/allmodelsonhuggingface
mattw/quantcontext
mattw/whatneedstorun
brucemacd/llama-mem-calc
mattw/faq-context
mattw/communitylinks
mattw/noprune
mattw/python-functioncalling
rename
mxyng/install
pulse
remove-first
editor
mattw/selfqueryingretrieval
cgo
mattw/howtoquant
api
matt/streamingapi
format-config
mxyng/extra-args
shell
update-nous-hermes
cp-model
upload-progress
fix-unknown-model
fix-model-names
delete-fix
insecure-registry
ls
deletemodels
progressbar
readme-updates
license-layers
skip-list
list-models
modelpath
matt/examplemodelfiles
distribution
go-opts
v0.23.1
v0.23.1-rc0
v0.23.0
v0.23.0-rc0
v0.22.1
v0.22.1-rc1
v0.22.1-rc0
v0.22.0
v0.22.0-rc1
v0.21.3-rc0
v0.21.2-rc1
v0.21.2
v0.21.2-rc0
v0.21.1
v0.21.1-rc1
v0.21.1-rc0
v0.21.0
v0.21.0-rc1
v0.21.0-rc0
v0.20.8-rc0
v0.20.7
v0.20.7-rc1
v0.20.7-rc0
v0.20.6
v0.20.6-rc1
v0.20.6-rc0
v0.20.5
v0.20.5-rc2
v0.20.5-rc1
v0.20.5-rc0
v0.20.4
v0.20.4-rc2
v0.20.4-rc1
v0.20.4-rc0
v0.20.3
v0.20.3-rc0
v0.20.2
v0.20.1
v0.20.1-rc2
v0.20.1-rc1
v0.20.1-rc0
v0.20.0
v0.20.0-rc1
v0.20.0-rc0
v0.19.0
v0.19.0-rc2
v0.19.0-rc1
v0.19.0-rc0
v0.18.4-rc1
v0.18.4-rc0
v0.18.3
v0.18.3-rc2
v0.18.3-rc1
v0.18.3-rc0
v0.18.2
v0.18.2-rc1
v0.18.2-rc0
v0.18.1
v0.18.1-rc1
v0.18.1-rc0
v0.18.0
v0.18.0-rc2
v0.18.0-rc1
v0.18.0-rc0
v0.17.8-rc4
v0.17.8-rc3
v0.17.8-rc2
v0.17.8-rc1
v0.17.8-rc0
v0.17.7
v0.17.7-rc2
v0.17.7-rc1
v0.17.7-rc0
v0.17.6
v0.17.5
v0.17.4
v0.17.3
v0.17.2
v0.17.1
v0.17.1-rc2
v0.17.1-rc1
v0.17.1-rc0
v0.17.0
v0.17.0-rc2
v0.17.0-rc1
v0.17.0-rc0
v0.16.3
v0.16.3-rc2
v0.16.3-rc1
v0.16.3-rc0
v0.16.2
v0.16.2-rc0
v0.16.1
v0.16.0
v0.16.0-rc2
v0.16.0-rc0
v0.16.0-rc1
v0.15.6
v0.15.5
v0.15.5-rc5
v0.15.5-rc4
v0.15.5-rc3
v0.15.5-rc2
v0.15.5-rc1
v0.15.5-rc0
v0.15.4
v0.15.3
v0.15.2
v0.15.1
v0.15.1-rc1
v0.15.1-rc0
v0.15.0-rc6
v0.15.0
v0.15.0-rc5
v0.15.0-rc4
v0.15.0-rc3
v0.15.0-rc2
v0.15.0-rc1
v0.15.0-rc0
v0.14.3
v0.14.3-rc3
v0.14.3-rc2
v0.14.3-rc1
v0.14.3-rc0
v0.14.2
v0.14.2-rc1
v0.14.2-rc0
v0.14.1
v0.14.0-rc11
v0.14.0
v0.14.0-rc10
v0.14.0-rc9
v0.14.0-rc8
v0.14.0-rc7
v0.14.0-rc6
v0.14.0-rc5
v0.14.0-rc4
v0.14.0-rc3
v0.14.0-rc2
v0.14.0-rc1
v0.14.0-rc0
v0.13.5
v0.13.5-rc1
v0.13.5-rc0
v0.13.4-rc2
v0.13.4
v0.13.4-rc1
v0.13.4-rc0
v0.13.3
v0.13.3-rc1
v0.13.3-rc0
v0.13.2
v0.13.2-rc2
v0.13.2-rc1
v0.13.2-rc0
v0.13.1
v0.13.1-rc2
v0.13.1-rc1
v0.13.1-rc0
v0.13.0
v0.13.0-rc0
v0.12.11
v0.12.11-rc1
v0.12.11-rc0
v0.12.10
v0.12.10-rc1
v0.12.10-rc0
v0.12.9-rc0
v0.12.9
v0.12.8
v0.12.8-rc0
v0.12.7
v0.12.7-rc1
v0.12.7-rc0
v0.12.7-citest0
v0.12.6
v0.12.6-rc1
v0.12.6-rc0
v0.12.5
v0.12.5-rc0
v0.12.4
v0.12.4-rc7
v0.12.4-rc6
v0.12.4-rc5
v0.12.4-rc4
v0.12.4-rc3
v0.12.4-rc2
v0.12.4-rc1
v0.12.4-rc0
v0.12.3
v0.12.2
v0.12.2-rc0
v0.12.1
v0.12.1-rc1
v0.12.1-rc2
v0.12.1-rc0
v0.12.0
v0.12.0-rc1
v0.12.0-rc0
v0.11.11
v0.11.11-rc3
v0.11.11-rc2
v0.11.11-rc1
v0.11.11-rc0
v0.11.10
v0.11.9
v0.11.9-rc0
v0.11.8
v0.11.8-rc0
v0.11.7-rc1
v0.11.7-rc0
v0.11.7
v0.11.6
v0.11.6-rc0
v0.11.5-rc4
v0.11.5-rc3
v0.11.5
v0.11.5-rc5
v0.11.5-rc2
v0.11.5-rc1
v0.11.5-rc0
v0.11.4
v0.11.4-rc0
v0.11.3
v0.11.3-rc0
v0.11.2
v0.11.1
v0.11.0-rc0
v0.11.0-rc1
v0.11.0-rc2
v0.11.0
v0.10.2-int1
v0.10.1
v0.10.0
v0.10.0-rc4
v0.10.0-rc3
v0.10.0-rc2
v0.10.0-rc1
v0.10.0-rc0
v0.9.7-rc1
v0.9.7-rc0
v0.9.6
v0.9.6-rc0
v0.9.6-ci0
v0.9.5
v0.9.4-rc5
v0.9.4-rc6
v0.9.4
v0.9.4-rc3
v0.9.4-rc4
v0.9.4-rc1
v0.9.4-rc2
v0.9.4-rc0
v0.9.3
v0.9.3-rc5
v0.9.4-citest0
v0.9.3-rc4
v0.9.3-rc3
v0.9.3-rc2
v0.9.3-rc1
v0.9.3-rc0
v0.9.2
v0.9.1
v0.9.1-rc1
v0.9.1-rc0
v0.9.1-ci1
v0.9.1-ci0
v0.9.0
v0.9.0-rc0
v0.8.0
v0.8.0-rc0
v0.7.1-rc2
v0.7.1
v0.7.1-rc1
v0.7.1-rc0
v0.7.0
v0.7.0-rc1
v0.7.0-rc0
v0.6.9-rc0
v0.6.8
v0.6.8-rc0
v0.6.7
v0.6.7-rc2
v0.6.7-rc1
v0.6.7-rc0
v0.6.6
v0.6.6-rc2
v0.6.6-rc1
v0.6.6-rc0
v0.6.5-rc1
v0.6.5
v0.6.5-rc0
v0.6.4-rc0
v0.6.4
v0.6.3-rc1
v0.6.3
v0.6.3-rc0
v0.6.2
v0.6.2-rc0
v0.6.1
v0.6.1-rc0
v0.6.0-rc0
v0.6.0
v0.5.14-rc0
v0.5.13
v0.5.13-rc6
v0.5.13-rc5
v0.5.13-rc4
v0.5.13-rc3
v0.5.13-rc2
v0.5.13-rc1
v0.5.13-rc0
v0.5.12
v0.5.12-rc1
v0.5.12-rc0
v0.5.11
v0.5.10
v0.5.9
v0.5.9-rc0
v0.5.8-rc13
v0.5.8
v0.5.8-rc12
v0.5.8-rc11
v0.5.8-rc10
v0.5.8-rc9
v0.5.8-rc8
v0.5.8-rc7
v0.5.8-rc6
v0.5.8-rc5
v0.5.8-rc4
v0.5.8-rc3
v0.5.8-rc2
v0.5.8-rc1
v0.5.8-rc0
v0.5.7
v0.5.6
v0.5.5
v0.5.5-rc0
v0.5.4
v0.5.3
v0.5.3-rc0
v0.5.2
v0.5.2-rc3
v0.5.2-rc2
v0.5.2-rc1
v0.5.2-rc0
v0.5.1
v0.5.0
v0.5.0-rc1
v0.4.8-rc0
v0.4.7
v0.4.6
v0.4.5
v0.4.4
v0.4.3
v0.4.3-rc0
v0.4.2
v0.4.2-rc1
v0.4.2-rc0
v0.4.1
v0.4.1-rc0
v0.4.0
v0.4.0-rc8
v0.4.0-rc7
v0.4.0-rc6
v0.4.0-rc5
v0.4.0-rc4
v0.4.0-rc3
v0.4.0-rc2
v0.4.0-rc1
v0.4.0-rc0
v0.4.0-ci3
v0.3.14
v0.3.14-rc0
v0.3.13
v0.3.12
v0.3.12-rc5
v0.3.12-rc4
v0.3.12-rc3
v0.3.12-rc2
v0.3.12-rc1
v0.3.11
v0.3.11-rc4
v0.3.11-rc3
v0.3.11-rc2
v0.3.11-rc1
v0.3.10
v0.3.10-rc1
v0.3.9
v0.3.8
v0.3.7
v0.3.7-rc6
v0.3.7-rc5
v0.3.7-rc4
v0.3.7-rc3
v0.3.7-rc2
v0.3.7-rc1
v0.3.6
v0.3.5
v0.3.4
v0.3.3
v0.3.2
v0.3.1
v0.3.0
v0.2.8
v0.2.8-rc2
v0.2.8-rc1
v0.2.7
v0.2.6
v0.2.5
v0.2.4
v0.2.3
v0.2.2
v0.2.2-rc2
v0.2.2-rc1
v0.2.1
v0.2.0
v0.1.49-rc14
v0.1.49-rc13
v0.1.49-rc12
v0.1.49-rc11
v0.1.49-rc10
v0.1.49-rc9
v0.1.49-rc8
v0.1.49-rc7
v0.1.49-rc6
v0.1.49-rc4
v0.1.49-rc5
v0.1.49-rc3
v0.1.49-rc2
v0.1.49-rc1
v0.1.48
v0.1.47
v0.1.46
v0.1.45-rc5
v0.1.45
v0.1.45-rc4
v0.1.45-rc3
v0.1.45-rc2
v0.1.45-rc1
v0.1.44
v0.1.43
v0.1.42
v0.1.41
v0.1.40
v0.1.40-rc1
v0.1.39
v0.1.39-rc2
v0.1.39-rc1
v0.1.38
v0.1.37
v0.1.36
v0.1.35
v0.1.35-rc1
v0.1.34
v0.1.34-rc1
v0.1.33
v0.1.33-rc7
v0.1.33-rc6
v0.1.33-rc5
v0.1.33-rc4
v0.1.33-rc3
v0.1.33-rc2
v0.1.33-rc1
v0.1.32
v0.1.32-rc2
v0.1.32-rc1
v0.1.31
v0.1.30
v0.1.29
v0.1.28
v0.1.27
v0.1.26
v0.1.25
v0.1.24
v0.1.23
v0.1.22
v0.1.21
v0.1.20
v0.1.19
v0.1.18
v0.1.17
v0.1.16
v0.1.15
v0.1.14
v0.1.13
v0.1.12
v0.1.11
v0.1.10
v0.1.9
v0.1.8
v0.1.7
v0.1.6
v0.1.5
v0.1.4
v0.1.3
v0.1.2
v0.1.1
v0.1.0
v0.0.21
v0.0.20
v0.0.19
v0.0.18
v0.0.17
v0.0.16
v0.0.15
v0.0.14
v0.0.13
v0.0.12
v0.0.11
v0.0.10
v0.0.9
v0.0.8
v0.0.7
v0.0.6
v0.0.5
v0.0.4
v0.0.3
v0.0.2
v0.0.1
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
Mirrored from GitHub Pull Request
No Label
model
Milestone
No items
No Milestone
Projects
Clear projects
No project
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: github-starred/ollama#5270
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @olumolu on GitHub (Dec 26, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/8249
https://huggingface.co/deepseek-ai/DeepSeek-V3-Base
https://huggingface.co/deepseek-ai/DeepSeek-V3
Kindly make it available to ollama so we can test..
Model paper
https://github.com/deepseek-ai/DeepSeek-V3/blob/main/DeepSeek_V3.pdf
https://x.com/deepseek_ai/status/1872242657348710721
@rick-github commented on GitHub (Dec 26, 2024):
https://github.com/ggerganov/llama.cpp/issues/10981
@x66ccff commented on GitHub (Jan 5, 2025):
https://github.com/ggerganov/llama.cpp/pull/11049 llama.cpp already merged now
@rick-github commented on GitHub (Jan 5, 2025):
#8308 will pull in deepseek3 support.
@ER-EPR commented on GitHub (Jan 9, 2025):
Now https://github.com/ollama/ollama/pull/8308 is merged, how can I use deepseekv3 with ollama? I searched in the library, but did not find it.
@rick-github commented on GitHub (Jan 9, 2025):
Build ollama from git head, download the safetensors, convert to GGUF, import to ollama.
@rick-github commented on GitHub (Jan 9, 2025):
Install v0.5.5-rc0, download the
GGUFshards, merge to a single GGUF, import to ollama.@rick-github commented on GitHub (Jan 9, 2025):
Sadly I get an error when loading:
So perhaps deepseek-v3 won't be supported in 0.5.5.
@rick-github commented on GitHub (Jan 9, 2025):
Whoops, mis-configured container.
@jjparady commented on GitHub (Jan 9, 2025):
@rick-github How did you fix the config? I’m getting the same error.
@rick-github commented on GitHub (Jan 9, 2025):
I had configured 4 GPUs instead of 8.
@jjparady commented on GitHub (Jan 10, 2025):
Hm, I am trying it on CPU only with no luck getting around this error.
I can get it working well on llama.cpp though. Maybe it has to do with how I am making the Binary?
@zengqingfu1442 commented on GitHub (Jan 13, 2025):
So, deepseek-v3 is already supported in 0.5.5 ?
@rick-github commented on GitHub (Jan 13, 2025):
Yes, but the model is not in the ollama library yet.
@zengqingfu1442 commented on GitHub (Jan 13, 2025):
I can run deepseek-v3 2bit quantizaton https://huggingface.co/unsloth/DeepSeek-V3-GGUF/tree/main/DeepSeek-V3-Q2_K_XS with ollama 0.5.5+CPU now.
@zengqingfu1442 commented on GitHub (Jan 13, 2025):
Where can i see the ollama library?
@rick-github commented on GitHub (Jan 13, 2025):
Ollama.com/models
@zengqingfu1442 commented on GitHub (Jan 13, 2025):
i searched this model https://ollama.com/nezahatkorkmaz/deepseek-v3
@rick-github commented on GitHub (Jan 13, 2025):
This model is only 2GB.
@bgort commented on GitHub (Jan 13, 2025):
I'm having trouble somewhere, but am unsure what's going on. Debug output isn't helpful.
I downloaded the Deepseek-V3 gguf shards from hf (unsloth/DeepSeek-V3-GGUF), merged them into a single file, created a model file:
Then 'create' fails:
The debug output (OLLAMA_DEBUG=1) shows nothing of use:
I'm still relatively new to ollama, so maybe I'm missing something obvious? Any help appreciated.
@rick-github commented on GitHub (Jan 14, 2025):
Official: https://ollama.com/library/deepseek-v3
@rick-github commented on GitHub (Jan 14, 2025):
The model create feature had a bit of an overhaul for 0.5.5, it has some rough edges. I'm not sure if it will work any better, but you could try downgrading to 0.5.4, doing the create, and then upgrading to 0.5.5 to actually run the model.
@pmarreck commented on GitHub (Jan 14, 2025):
what's the expected RAM reqs for this model?
@rick-github commented on GitHub (Jan 14, 2025):
More than 400GB. In my test above I ran the smaller q3 model and it used all 320G VRAM and still spilled into system RAM.
@bgort commented on GitHub (Jan 14, 2025):
Thanks @rick-github. When I downloaded the shards a few days ago there was no official Deepseek-V3 model, but I see now there is. It doesn't look like there's a q2 yet, so I assume it won't work very well for me; I only have 48gb GPU RAM at the moment, but will be doubling that in a few days so should be okay with the q4 at that point, I believe.
Creating the model with 0.5.4 worked - thanks for that - and am now attempting to run it with 0.5.5.
Loading the model (
ollama run) times out, as the process seems to be single-threaded and it maxes its single core out and then takes several minutes to load into the ~162GB of system ram. Once it's done that (stopping at ~162gb) it sits - with periodic single-core CPU spikes - and then times out (Error: timed out waiting for llama runner to start - progress 0.00 -). I've tried several times and can't get it to start. I've allocated 192GB of system RAM to this VM and 32 CPU cores (though it doesn't seem to use much CPU, overall - just the one full core and a partial second core sometimes). Unsloth says 40gb GPU RAM is enough for q2, but apparently not?@pmarreck, as mentioned above, ~162GB system RAM was required for the hf unsloth q2 gguf, for whatever it's worth.
@rick-github commented on GitHub (Jan 14, 2025):
Set
OLLAMA_LOAD_TIMEOUT=30m.@bgort commented on GitHub (Jan 14, 2025):
I think using DSV3 is beyond practical, at least for my current purposes and with my current hardware. I'll try again when I have a beefier system. Thanks much for your help.
@zhiyanliu commented on GitHub (Jan 14, 2025):
Awesome, just been notified to upgrade my local version, finally. Thanks!
@zengqingfu1442 commented on GitHub (Jan 14, 2025):
@rick-github i find that the model needs to be re-loaded into memory when i call the openai-api compatible server of ollama, how can i make ollama keep loading the model in memory?
@rick-github commented on GitHub (Jan 14, 2025):
By default ollama will keep a model in memory for 5 minutes. You can extend this by setting
keep_alive. Unfortunately the OpenAI API doesn't support this parameter, so you need to either set it server-wide withOLLAMA_KEEP_ALIVEor add it to the Modelfile.@zengqingfu1442 commented on GitHub (Jan 14, 2025):
i added this variable in
/etc/systemd/system/ollama.serviceand thensystemctl daemon-reload && systemctl restart ollama.serviceWill this work?
@rick-github commented on GitHub (Jan 14, 2025):
Yes. After you load the model,
ollama psshould showForever:@zengqingfu1442 commented on GitHub (Jan 14, 2025):
It shows

Forever, but the model is still off-loaded cause the memory decreased to 25GB.@rick-github commented on GitHub (Jan 14, 2025):
Then perhaps the runner crashed. The server log will show you more.
@zengqingfu1442 commented on GitHub (Jan 15, 2025):
Here is the ollama server logs when the runner crashed:
@zengqingfu1442 commented on GitHub (Jan 15, 2025):
@rick-github Can i set a larger input prompt size for ollama? The default size 2048 sometimes is too small.
@rick-github commented on GitHub (Jan 15, 2025):
https://github.com/ollama/ollama/issues/5975
The workaround increases the context size so allows a large prompt.
@zengqingfu1442 commented on GitHub (Jan 15, 2025):
PARAMETER num_ctx 24576in modelfile and then recreate the deepseek-v3 model.@rick-github commented on GitHub (Jan 15, 2025):
Set
num_predicttoo. The K-shift happens when the total input and output tokens exceeds the context window.ollama will use 100% of the GPU if 100% of the model fits in VRAM. If it doesn't fit all in VRAM part of it will be loaded in system RAM, and then CPU is used for inference computation.
If you find that 100% of the model is loaded in VRAM and only 50% of the GPU is being used, open a new ticket and include full system logs.
@zengqingfu1442 commented on GitHub (Jan 15, 2025):
There is no GPU on my machine. So i expect ollama to use 100% CPU for model inference.
@rick-github commented on GitHub (Jan 15, 2025):
I'm sorry, I misread. Set
num_threadin the API call or Modelfile.@zengqingfu1442 commented on GitHub (Jan 15, 2025):
The above is my cpu info. So i need to set
PARAMETER num_thread 112in my modelfile before creating the model?@zengqingfu1442 commented on GitHub (Jan 15, 2025):
i can't find the parameter
num_threadin this doc https://github.com/ollama/ollama/blob/main/docs/modelfile.md#parameter@zengqingfu1442 commented on GitHub (Jan 15, 2025):
i set these above parameters but when creating the model, it reports
Error: model requires more system memory (870.2 GiB) than is available (746.7 GiB). Can i change the modelfile and then re-apply the new modelfile? don't want to recreate the model cause i need to downgrade ollama to 0.5.4 and then upgrade it to 0.5.5 to run the model.@zengqingfu1442 commented on GitHub (Jan 15, 2025):
After i set the
num_threadto 112(total cores of cpu on my machine), i call the openai api and no response returned after 15 minutes.@corticalstack commented on GitHub (Jan 15, 2025):
Could the Ollama team also make the Q3 quant of DeepSeek V3 available please?
@Yooki-K commented on GitHub (Feb 6, 2025):
@Hugh-yw commented on GitHub (Feb 11, 2025):
GPU:H100 *8 Memory:640GB
@rick-github Hello, I would like to ask why an exception occurs when running ollama run 671b. The model is downloaded through ollama pull.
@moroboshiai commented on GitHub (Feb 11, 2025):
hello,I miss the same error. And I can't find the config file. how to set it?
@moroboshiai commented on GitHub (Feb 11, 2025):
the same error...
@Hugh-yw commented on GitHub (Feb 11, 2025):
Upgrade the ollama version to the latest 0.5.8 and you can run
But after running for a while, it crashed
@rick-github commented on GitHub (Feb 11, 2025):
https://github.com/ollama/ollama/issues/5975
@zengqingfu1442 commented on GitHub (Feb 13, 2025):
@rick-github If i want to use only number 4,5,6,7 GPUs on my machine, can i set
CUDA_VISIBLE_DEVICESin /etc/systemd/system/ollama.service like this?@rick-github commented on GitHub (Feb 13, 2025):
Yes.
@zengqingfu1442 commented on GitHub (Feb 13, 2025):
@rick-github
I use ollama 0.5.7 to run DeepSeek-R1-Q4_K_M from https://huggingface.co/unsloth/DeepSeek-R1-GGUF/tree/main/DeepSeek-R1-Q4_K_M, and put 12 layers onto GPU, the others on the CPU. When runing the inference, the CPU leverage rate is near 50%, but the GPU is only 0%-1%. And the generating speed is only about 3.7 tokens/s. So, how can i improve the speed?Thanks
@rick-github commented on GitHub (Feb 13, 2025):
From your screenshot, not all of VRAM is being used. What parameters have you set?
@zengqingfu1442 commented on GitHub (Feb 13, 2025):
Here is the modelfile.
@zengqingfu1442 commented on GitHub (Feb 13, 2025):
And the
/etc/systemd/system/ollama.serivcefile is@rick-github commented on GitHub (Feb 13, 2025):
You can increase speed by putting more layers in VRAM.
@zengqingfu1442 commented on GitHub (Feb 13, 2025):
But one RTX 4090 GPU can only be put on up to 2 layers.
@zengqingfu1442 commented on GitHub (Feb 13, 2025):
Six 4090 GPU is 12 layers at most.
@rick-github commented on GitHub (Feb 13, 2025):
Then you need more VRAM or a smaller model to increase speed.
@zengqingfu1442 commented on GitHub (Feb 13, 2025):
https://github.com/ollama/ollama/blob/main/docs/faq.md#how-can-i-set-the-quantization-type-for-the-kv-cache Would this
OLLAMA_KV_CACHE_TYPEparameter in ollama increase the speed if i quant the KV cache as 4 bitq4_0?@rick-github commented on GitHub (Feb 13, 2025):
Currently, KV quantization in ollama needs flash attention. Flash attention is not supported for deepseek architecture.
@rick-github commented on GitHub (Feb 13, 2025):
Try this: set
GGML_CUDA_ENABLE_UNIFIED_MEMORY=1in server environment. Then try offloading 3 layers per GPU. See here for explanation.@zengqingfu1442 commented on GitHub (Feb 13, 2025):
You mean i can offload 3 layers per 4090 GPU after setting GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 in ollama server environment?
@zengqingfu1442 commented on GitHub (Feb 13, 2025):
Why cannot see the logs of ollama 0.5.7? i remember i can see the logs of ollama when is is 0.5.5.
@zengqingfu1442 commented on GitHub (Feb 18, 2025):
I use ktransformers 0.2.0+14090+CPU, and the speed is about 5-8 token/s, larger than ollama 0.5.7+64090+cpu which is about 3.7 token/s. Same model, same machine. @rick-github