github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-15 20:54:54 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #15674] Jetson Orin Nano 8GB: improve defaults and diagnostics for reliable CUDA inference #72059

New Issue
Open
opened 2026-05-05 03:24:29 -05:00 by GiteaMirror · 4 comments
GiteaMirror commented 2026-05-05 03:24:29 -05:00
Owner
Copy Link

Originally created by @jammypate-1 on GitHub (Apr 18, 2026).
Original GitHub issue: https://github.com/ollama/ollama/issues/15674

Summary

Jetson users still run into recurring CUDA-init and runtime stability issues despite successful installs. I’d like to contribute a focused set of improvements for Orin-class devices (especially 8GB) that reduce setup friction and improve reliability without changing behavior on non-Jetson systems.

Motivation

There are multiple historical and active Jetson/Nvidia reports around detection, compatibility, and stability under load (for example #9503 and related Nvidia-labeled issues).

Proposed scope

  1. Docker JetPack auto-detection fallback

    • Infer JetPack major from /etc/nv_tegra_release
    • Preserve explicit JETSON_JETPACK override
    • Keep current behavior if detection is unavailable

  2. Scheduler warning guardrails for low-memory devices

    • Warn when effective memory pressure is likely unsafe (for example high OLLAMA_NUM_PARALLEL * OLLAMA_CONTEXT_LENGTH combinations)
    • Suggest safer values in warning text
    • Non-breaking: warning-only, no hard fail

  3. Jetson docs refresh (native + Docker)

    • Clear CUDA verification commands
    • expected log signatures for successful Jetson CUDA path
    • profile guidance for 8GB-class systems

Validation environment available

  • Device: Jetson Orin Nano 8GB
  • JetPack: 6.x
  • Ollama: 0.21.0
  • Verified CUDA runtime path via journalctl (inference compute ... library=CUDA ... Orin)

Contribution plan

If this direction is welcome, I can submit PRs in small sequence:

  1. docs + verification improvements
  2. Docker JetPack autodetect fallback
  3. scheduler warning guardrails

Happy to adjust scope to maintainer preference before opening PR 1.

Originally created by @jammypate-1 on GitHub (Apr 18, 2026). Original GitHub issue: https://github.com/ollama/ollama/issues/15674 ### Summary Jetson users still run into recurring CUDA-init and runtime stability issues despite successful installs. I’d like to contribute a focused set of improvements for Orin-class devices (especially 8GB) that reduce setup friction and improve reliability without changing behavior on non-Jetson systems. ### Motivation There are multiple historical and active Jetson/Nvidia reports around detection, compatibility, and stability under load (for example `#9503` and related Nvidia-labeled issues). ### Proposed scope 1. **Docker JetPack auto-detection fallback** - Infer JetPack major from `/etc/nv_tegra_release` - Preserve explicit `JETSON_JETPACK` override - Keep current behavior if detection is unavailable 2. **Scheduler warning guardrails for low-memory devices** - Warn when effective memory pressure is likely unsafe (for example high `OLLAMA_NUM_PARALLEL * OLLAMA_CONTEXT_LENGTH` combinations) - Suggest safer values in warning text - Non-breaking: warning-only, no hard fail 3. **Jetson docs refresh (native + Docker)** - Clear CUDA verification commands - expected log signatures for successful Jetson CUDA path - profile guidance for 8GB-class systems ### Validation environment available - Device: Jetson Orin Nano 8GB - JetPack: 6.x - Ollama: 0.21.0 - Verified CUDA runtime path via `journalctl` (`inference compute ... library=CUDA ... Orin`) ### Contribution plan If this direction is welcome, I can submit PRs in small sequence: 1. docs + verification improvements 2. Docker JetPack autodetect fallback 3. scheduler warning guardrails Happy to adjust scope to maintainer preference before opening PR 1.
GiteaMirror commented 2026-05-05 03:24:33 -05:00
Author
Owner
Copy Link

@PureBlissAK commented on GitHub (Apr 18, 2026):

🤖 Automated Triage & Analysis Report

Issue: #15674
Analyzed: 2026-04-18T18:13:44.899196

Analysis

  • Type: unknown
  • Severity: medium
  • Components: unknown

Implementation Plan

  • Effort: medium
  • Steps:

This issue has been triaged and marked for implementation.

<!-- gh-comment-id:4274294845 --> @PureBlissAK commented on GitHub (Apr 18, 2026): <!-- ollama-issue-orchestrator:v1 issue:15674 --> ## 🤖 Automated Triage & Analysis Report **Issue**: #15674 **Analyzed**: 2026-04-18T18:13:44.899196 ### Analysis - **Type**: unknown - **Severity**: medium - **Components**: unknown ### Implementation Plan - **Effort**: medium - **Steps**: *This issue has been triaged and marked for implementation.*
GiteaMirror commented 2026-05-05 03:24:35 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Apr 18, 2026):

@PureBlissAK Just stop.

<!-- gh-comment-id:4274316539 --> @rick-github commented on GitHub (Apr 18, 2026): @PureBlissAK Just stop.
GiteaMirror commented 2026-05-05 03:24:37 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Apr 18, 2026):

@PureBlissAK is spamming issues with their useless "Automated Triage & Analysis Report". It adds no value and creates pointless email notifications for those subscribed to the issue.

<!-- gh-comment-id:4274406494 --> @rick-github commented on GitHub (Apr 18, 2026): @PureBlissAK is spamming issues with their useless "Automated Triage & Analysis Report". It adds no value and creates pointless email notifications for those subscribed to the issue.
GiteaMirror commented 2026-05-05 03:24:39 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Apr 18, 2026):

Unless you are PureBlissAK you have nothing to be sorry about.

<!-- gh-comment-id:4274411877 --> @rick-github commented on GitHub (Apr 18, 2026): Unless you are PureBlissAK you have nothing to be sorry about.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#72059
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?