github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 08:02:14 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #4693] Add binary support for Nvidia Jetson Xavier- JetPack 5 #28714

New Issue
Closed
opened 2026-04-22 07:14:02 -05:00 by GiteaMirror · 6 comments
GiteaMirror commented 2026-04-22 07:14:02 -05:00
Owner
Copy Link

Originally created by @ZanMax on GitHub (May 29, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/4693

Originally assigned to: @dhiltgen on GitHub.

What is the issue?

After the update, it's just stuck with:

time=2024-05-28T20:08:58.624-07:00 level=INFO source=server.go:564 msg="waiting for server to become available" status="llm server loading model"
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: yes
ggml_cuda_init: CUDA_USE_TENSOR_CORES: no
ggml_cuda_init: found 1 CUDA devices:
Device 0: Xavier, compute capability 7.2, VMM: yes

After reinstalling 0.1.38 all works properly.

OS

Linux

GPU

Nvidia

CPU

Other

Ollama version

0.1.39

Originally created by @ZanMax on GitHub (May 29, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/4693 Originally assigned to: @dhiltgen on GitHub. ### What is the issue? After the update, it's just stuck with: > time=2024-05-28T20:08:58.624-07:00 level=INFO source=server.go:564 msg="waiting for server to become available" status="llm server loading model" > ggml_cuda_init: GGML_CUDA_FORCE_MMQ: yes > ggml_cuda_init: CUDA_USE_TENSOR_CORES: no > ggml_cuda_init: found 1 CUDA devices: > Device 0: Xavier, compute capability 7.2, VMM: yes After reinstalling 0.1.38 all works properly. ### OS Linux ### GPU Nvidia ### CPU Other ### Ollama version 0.1.39
GiteaMirror added the nvidiabug labels 2026-04-22 07:14:02 -05:00
GiteaMirror commented 2026-04-22 07:14:06 -05:00
Author
Owner
Copy Link

@mikeoleary commented on GitHub (May 30, 2024):

I think I've hit this same issue. I'm a noob with this, but following the instructions to deploy open-webui I have deployed ollama:latest and received the same error as ZanMax:
level=INFO source=server.go:564 msg="waiting for server to become available" status="llm server loading model"

But when I change my manifest to deploy ollama:0.1.38, all works fine.

<!-- gh-comment-id:2140630722 --> @mikeoleary commented on GitHub (May 30, 2024): I think I've hit this same issue. I'm a noob with this, but following the [instructions to deploy open-webui](https://github.com/open-webui/open-webui/blob/main/INSTALLATION.md#installing-both-ollama-and-open-webui-using-kustomize) I have deployed ollama:latest and received the same error as ZanMax: `level=INFO source=server.go:564 msg="waiting for server to become available" status="llm server loading model"` But when I change my manifest to deploy ollama:0.1.38, all works fine.
GiteaMirror commented 2026-04-22 07:14:08 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (May 31, 2024):

What driver are you running? There may be some issues with the latest 555 driver, so if you are running that, I'm curious if downgrading to an older driver gets 0.1.39 working.

Can you share the output of the server startup with OLLAMA_DEBUG=1 set?

<!-- gh-comment-id:2142864564 --> @dhiltgen commented on GitHub (May 31, 2024): What driver are you running? There may be some issues with the latest 555 driver, so if you are running that, I'm curious if downgrading to an older driver gets 0.1.39 working. Can you share the output of the server startup with OLLAMA_DEBUG=1 set?
GiteaMirror commented 2026-04-22 07:14:08 -05:00
Author
Owner
Copy Link

@ZanMax commented on GitHub (May 31, 2024):

I think Nvidia Xavier has other drivers instead of standard.
Log:

cat /var/log/Xorg.0.log
[    40.584] (--) Log file renamed from "/var/log/Xorg.pid-2645.log" to "/var/log/Xorg.0.log"
[    40.588]
X.Org X Server 1.20.13
X Protocol Version 11, Revision 0
[    40.588] Build Operating System: linux Ubuntu
[    40.588] Current Operating System: Linux dev-desktop 5.10.104-tegra #1 SMP PREEMPT Wed Aug 10 20:17:07 PDT 2022 aarch64
[    40.588] Kernel command line: root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyTCU0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0
[    40.588] Build Date: 09 April 2024  01:36:10AM
[    40.588] xorg-server 2:1.20.13-1ubuntu1~20.04.17 (For technical support please see http://www.ubuntu.com/support)
[    40.589] Current version of pixman: 0.38.4
[    40.589] 	Before reporting problems, check http://wiki.x.org
	to make sure that you have the latest version.
[    40.589] Markers: (--) probed, (**) from config file, (==) default setting,
	(++) from command line, (!!) notice, (II) informational,
	(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
[    40.589] (==) Log file: "/var/log/Xorg.0.log", Time: Thu May 30 09:06:55 2024
[    40.596] (==) Using config file: "/etc/X11/xorg.conf"
[    40.597] (==) Using config directory: "/etc/X11/xorg.conf.d"
[    40.597] (==) Using system config directory "/usr/share/X11/xorg.conf.d"
[    40.600] (==) No Layout section.  Using the first Screen section.
[    40.600] (==) No screen section available. Using defaults.
[    40.600] (**) |-->Screen "Default Screen Section" (0)
[    40.600] (**) |   |-->Monitor "<default monitor>"
[    40.602] (==) No device specified for screen "Default Screen Section".
	Using the first device section listed.
[    40.602] (**) |   |-->Device "Tegra0"
[    40.602] (==) No monitor specified for screen "Default Screen Section".
	Using a default monitor configuration.
[    40.602] (**) Option "AutoAddGPU" "false"
[    40.602] (==) Automatically adding devices
[    40.603] (==) Automatically enabling devices
[    40.603] (**) Not automatically adding GPU devices
[    40.603] (==) Automatically binding GPU devices
[    40.603] (==) Max clients allowed: 256, resource mask: 0x1fffff
[    40.609] (WW) The directory "/usr/share/fonts/X11/cyrillic" does not exist.
[    40.609] 	Entry deleted from font path.
[    40.609] (WW) The directory "/usr/share/fonts/X11/100dpi/" does not exist.
[    40.610] 	Entry deleted from font path.
[    40.610] (WW) The directory "/usr/share/fonts/X11/75dpi/" does not exist.
[    40.610] 	Entry deleted from font path.
[    40.611] (WW) The directory "/usr/share/fonts/X11/100dpi" does not exist.
[    40.611] 	Entry deleted from font path.
[    40.611] (WW) The directory "/usr/share/fonts/X11/75dpi" does not exist.
[    40.611] 	Entry deleted from font path.
[    40.612] (==) FontPath set to:
	/usr/share/fonts/X11/misc,
	/usr/share/fonts/X11/Type1,
	built-ins
[    40.612] (==) ModulePath set to "/usr/lib/xorg/modules"
[    40.612] (II) The server relies on udev to provide the list of input devices.
	If no devices become available, reconfigure udev or disable AutoAddDevices.
[    40.612] (II) Loader magic: 0xaaaab6cdc010
[    40.612] (II) Module ABI versions:
[    40.612] 	X.Org ANSI C Emulation: 0.4
[    40.612] 	X.Org Video Driver: 24.1
[    40.612] 	X.Org XInput driver : 24.1
[    40.612] 	X.Org Server Extension : 10.0
[    40.617] (++) using VT number 1

[    40.629] (II) systemd-logind: took control of session /org/freedesktop/login1/session/c1
[    40.633] (II) no primary bus or device found
[    40.633] (WW) "dri" will not be loaded unless you've specified it to be loaded elsewhere.
[    40.633] (II) "glx" will be loaded by default.
[    40.633] (II) LoadModule: "extmod"
[    40.634] (II) Module "extmod" already built-in
[    40.634] (II) LoadModule: "glx"
[    40.638] (II) Loading /usr/lib/xorg/modules/extensions/libglx.so
[    40.656] (II) Module glx: vendor="X.Org Foundation"
[    40.656] 	compiled for 1.20.13, module version = 1.0.0
[    40.656] 	ABI class: X.Org Server Extension, version 10.0
[    40.656] (II) LoadModule: "nvidia"
[    40.657] (II) Loading /usr/lib/xorg/modules/drivers/nvidia_drv.so
[    40.684] (II) Module nvidia: vendor="NVIDIA Corporation"
[    40.684] 	compiled for 1.6.99.901, module version = 1.0.0
[    40.684] 	Module class: X.Org Video Driver
[    40.690] (II) NVIDIA dlloader X Driver  35.1.0  Release Build  (bugfix_main)  (buildbrain@mobile-u64-5562-d3000)  Thu Aug 25 11:12:05 PDT 2022
[    40.690] (II) NVIDIA Unified Driver for all Supported NVIDIA GPUs
[    40.691] (WW) Falling back to old probe method for NVIDIA
[    40.693] (II) Loading sub module "fb"
[    40.693] (II) LoadModule: "fb"
[    40.693] (II) Loading /usr/lib/xorg/modules/libfb.so
[    40.695] (II) Module fb: vendor="X.Org Foundation"
[    40.695] 	compiled for 1.20.13, module version = 1.0.0
[    40.695] 	ABI class: X.Org ANSI C Emulation, version 0.4
[    40.695] (II) Loading sub module "wfb"
[    40.695] (II) LoadModule: "wfb"
[    40.698] (II) Loading /usr/lib/xorg/modules/libwfb.so
[    40.700] (II) Module wfb: vendor="X.Org Foundation"
[    40.700] 	compiled for 1.20.13, module version = 1.0.0
[    40.700] 	ABI class: X.Org ANSI C Emulation, version 0.4
[    40.700] (II) Loading sub module "ramdac"
[    40.700] (II) LoadModule: "ramdac"
[    40.700] (II) Module "ramdac" already built-in
[    40.705] (WW) VGA arbiter: cannot open kernel arbiter, no multi-card support
[    40.706] (II) NVIDIA(0): Creating default Display subsection in Screen section
	"Default Screen Section" for depth/fbbpp 24/32
[    40.706] (==) NVIDIA(0): Depth 24, (==) framebuffer bpp 32
[    40.706] (==) NVIDIA(0): RGB weight 888
[    40.706] (==) NVIDIA(0): Default visual is TrueColor
[    40.706] (==) NVIDIA(0): Using gamma correction (1.0, 1.0, 1.0)
[    40.706] (DB) xf86MergeOutputClassOptions unsupported bus type 0
[    40.706] (**) NVIDIA(0): Option "AllowEmptyInitialConfiguration" "true"
[    40.707] (**) NVIDIA(0): Enabling 2D acceleration
[    40.707] (II) Loading sub module "glxserver_nvidia"
[    40.707] (II) LoadModule: "glxserver_nvidia"
[    40.708] (II) Loading /usr/lib/xorg/modules/extensions/libglxserver_nvidia.so
[    40.802] (II) Module glxserver_nvidia: vendor="NVIDIA Corporation"
[    40.802] 	compiled for 1.6.99.901, module version = 1.0.0
[    40.802] 	Module class: X.Org Server Extension
[    40.802] (II) NVIDIA GLX Module  35.1.0  Release Build  (bugfix_main)  (buildbrain@mobile-u64-5562-d3000)  Thu Aug 25 11:11:11 PDT 2022
[    40.806] (II) NVIDIA: The X server supports PRIME Render Offload.
[    40.811] (--) NVIDIA(0): Valid display device(s) on GPU-0 at SoC
[    40.812] (--) NVIDIA(0):     DFP-0
[    40.812] (--) NVIDIA(0):     DFP-1
[    40.812] (WW) NVIDIA: No DRM device: No direct render devices found.
[    40.814] (II) NVIDIA(0): NVIDIA GPU Xavier (nvgpu) (GV11B) at SoC (GPU-0)
[    40.814] (--) NVIDIA(0): Memory: 7021844 kBytes
[    40.814] (--) NVIDIA(0): VideoBIOS:
[    40.815] (==) NVIDIA(0):
[    40.815] (==) NVIDIA(0): No modes were requested; the default mode "nvidia-auto-select"
[    40.815] (==) NVIDIA(0):     will be used as the requested mode.
[    40.815] (==) NVIDIA(0):
[    40.815] (--) NVIDIA(0): No enabled display devices found; starting anyway because
[    40.815] (--) NVIDIA(0):     AllowEmptyInitialConfiguration is enabled
[    40.815] (II) NVIDIA(0): Validated MetaModes:
[    40.815] (II) NVIDIA(0):     "NULL"
[    40.816] (II) NVIDIA(0): Virtual screen size determined to be 640 x 480
[    40.816] (WW) NVIDIA(0): Unable to get display device for DPI computation.
[    40.816] (==) NVIDIA(0): DPI set to (75, 75); computed from built-in default
[    40.817] (II) NVIDIA: Reserving 24576.00 MB of virtual memory for indirect memory
[    40.817] (II) NVIDIA:     access.
[    40.817] (WW) NVIDIA(GPU-0): Failed to enter interactive mode.
[    40.822] (EE) NVIDIA(0): Failed to allocate NVIDIA Error Handler
[    40.822] (II) NVIDIA(0): ACPI: failed to connect to the ACPI event daemon; the daemon
[    40.822] (II) NVIDIA(0):     may not be running or the "AcpidSocketPath" X
[    40.822] (II) NVIDIA(0):     configuration option may not be set correctly.  When the
[    40.822] (II) NVIDIA(0):     ACPI event daemon is available, the NVIDIA X driver will
[    40.823] (II) NVIDIA(0):     try to use it to receive ACPI event notifications.  For
[    40.823] (II) NVIDIA(0):     details, please see the "ConnectToAcpid" and
[    40.823] (II) NVIDIA(0):     "AcpidSocketPath" X configuration options in Appendix B: X
[    40.823] (II) NVIDIA(0):     Config Options in the README.
[    40.912] (II) NVIDIA(0): Setting mode "NULL"
[    40.931] (==) NVIDIA(0): Disabling shared memory pixmaps
[    40.931] (==) NVIDIA(0): Backing store enabled
[    40.932] (==) NVIDIA(0): Silken mouse enabled
[    40.935] (==) NVIDIA(0): DPMS enabled
[    40.936] (II) Loading sub module "dri2"
[    40.936] (II) LoadModule: "dri2"
[    40.936] (II) Module "dri2" already built-in
[    40.936] (II) NVIDIA(0): [DRI2] Setup complete
[    40.936] (II) NVIDIA(0): [DRI2]   VDPAU driver: nvidia
[    40.938] (II) Initializing extension Generic Event Extension
[    40.939] (II) Initializing extension SHAPE
[    40.940] (II) Initializing extension MIT-SHM
[    40.941] (II) Initializing extension XInputExtension
[    40.943] (II) Initializing extension XTEST
[    40.944] (II) Initializing extension BIG-REQUESTS
[    40.945] (II) Initializing extension SYNC
[    40.945] (II) Initializing extension XKEYBOARD
[    40.946] (II) Initializing extension XC-MISC
[    40.947] (II) Initializing extension SECURITY
[    40.947] (II) Initializing extension XFIXES
[    40.948] (II) Initializing extension RENDER
[    40.949] (II) Initializing extension RANDR
[    40.950] (II) Initializing extension COMPOSITE
[    40.950] (II) Initializing extension DAMAGE
[    40.951] (II) Initializing extension MIT-SCREEN-SAVER
[    40.952] (II) Initializing extension DOUBLE-BUFFER
[    40.952] (II) Initializing extension RECORD
[    40.953] (II) Initializing extension DPMS
[    40.954] (II) Initializing extension Present
[    40.954] (II) Initializing extension DRI3
[    40.955] (II) Initializing extension X-Resource
[    40.956] (II) Initializing extension XVideo
[    40.957] (II) Initializing extension XVideo-MotionCompensation
[    40.957] (II) Initializing extension SELinux
[    40.957] (II) SELinux: Disabled on system
[    40.957] (II) Initializing extension GLX
[    40.958] (II) Initializing extension GLX
[    40.958] (II) Indirect GLX disabled.
[    40.959] (II) GLX: Another vendor is already registered for screen 0
[    40.959] (II) Initializing extension XFree86-VidModeExtension
[    40.960] (II) Initializing extension XFree86-DGA
[    40.961] (II) Initializing extension XFree86-DRI
[    40.962] (II) Initializing extension DRI2
[    40.963] (II) Initializing extension NV-GLX
[    40.964] (II) Initializing extension NV-CONTROL
[    41.389] (II) config/udev: Adding input device NVIDIA Jetson Xavier NX HDA HDMI/DP,pcm=9 (/dev/input/event9)
[    41.389] (II) No input driver specified, ignoring this device.
[    41.389] (II) This device may have been added with another device file.
[    41.392] (II) config/udev: Adding input device NVIDIA Jetson Xavier NX HDA HDMI/DP,pcm=3 (/dev/input/event6)
[    41.392] (II) No input driver specified, ignoring this device.
[    41.392] (II) This device may have been added with another device file.
[    41.395] (II) config/udev: Adding input device NVIDIA Jetson Xavier NX HDA HDMI/DP,pcm=7 (/dev/input/event7)
[    41.396] (II) No input driver specified, ignoring this device.
[    41.396] (II) This device may have been added with another device file.
[    41.398] (II) config/udev: Adding input device NVIDIA Jetson Xavier NX HDA HDMI/DP,pcm=8 (/dev/input/event8)
[    41.398] (II) No input driver specified, ignoring this device.
[    41.398] (II) This device may have been added with another device file.
[    41.404] (II) config/udev: Adding input device 2.4G RF Keyboard & Mouse (/dev/input/event1)
[    41.404] (**) 2.4G RF Keyboard & Mouse: Applying InputClass "libinput keyboard catchall"
[    41.404] (II) LoadModule: "libinput"
[    41.406] (II) Loading /usr/lib/xorg/modules/input/libinput_drv.so
[    41.423] (II) Module libinput: vendor="X.Org Foundation"
[    41.423] 	compiled for 1.20.4, module version = 0.29.0
[    41.423] 	Module class: X.Org XInput Driver
[    41.424] 	ABI class: X.Org XInput driver, version 24.1
[    41.424] (II) Using input driver 'libinput' for '2.4G RF Keyboard & Mouse'
[    41.434] (II) systemd-logind: got fd for /dev/input/event1 13:65 fd 36 paused 0
[    41.434] (**) 2.4G RF Keyboard & Mouse: always reports core events
[    41.435] (**) Option "Device" "/dev/input/event1"
[    41.435] (**) Option "_source" "server/udev"
[    41.477] (II) event1  - 2.4G RF Keyboard & Mouse: is tagged by udev as: Keyboard
[    41.479] (II) event1  - 2.4G RF Keyboard & Mouse: device is a keyboard
[    41.480] (II) event1  - 2.4G RF Keyboard & Mouse: device removed
[    41.480] (**) Option "config_info" "udev:/sys/devices/platform/3610000.xhci/usb1/1-2/1-2.1/1-2.1:1.0/0003:3938:1192.0001/input/input1/event1"
[    41.481] (II) XINPUT: Adding extended input device "2.4G RF Keyboard & Mouse" (type: KEYBOARD, id 6)
[    41.481] (**) Option "xkb_model" "pc105"
[    41.481] (**) Option "xkb_layout" "us"
[    41.499] (II) event1  - 2.4G RF Keyboard & Mouse: is tagged by udev as: Keyboard
[    41.501] (II) event1  - 2.4G RF Keyboard & Mouse: device is a keyboard
[    41.512] (II) config/udev: Adding input device 2.4G RF Keyboard & Mouse (/dev/input/event2)
[    41.513] (II) No input driver specified, ignoring this device.
[    41.513] (II) This device may have been added with another device file.
[    41.519] (II) config/udev: Adding input device 2.4G RF Keyboard & Mouse (/dev/input/event3)
[    41.519] (**) 2.4G RF Keyboard & Mouse: Applying InputClass "libinput pointer catchall"
[    41.519] (II) Using input driver 'libinput' for '2.4G RF Keyboard & Mouse'
[    41.527] (II) systemd-logind: got fd for /dev/input/event3 13:67 fd 39 paused 0
[    41.527] (**) 2.4G RF Keyboard & Mouse: always reports core events
[    41.527] (**) Option "Device" "/dev/input/event3"
[    41.528] (**) Option "_source" "server/udev"
[    41.548] (II) event3  - 2.4G RF Keyboard & Mouse: is tagged by udev as: Mouse
[    41.549] (II) event3  - 2.4G RF Keyboard & Mouse: device is a pointer
[    41.550] (II) event3  - 2.4G RF Keyboard & Mouse: device removed
[    41.551] (**) Option "config_info" "udev:/sys/devices/platform/3610000.xhci/usb1/1-2/1-2.1/1-2.1:1.1/0003:3938:1192.0002/input/input3/event3"
[    41.551] (II) XINPUT: Adding extended input device "2.4G RF Keyboard & Mouse" (type: MOUSE, id 7)
[    41.552] (**) Option "AccelerationScheme" "none"
[    41.552] (**) 2.4G RF Keyboard & Mouse: (accel) selected scheme none/0
[    41.552] (**) 2.4G RF Keyboard & Mouse: (accel) acceleration factor: 2.000
[    41.552] (**) 2.4G RF Keyboard & Mouse: (accel) acceleration threshold: 4
[    41.568] (II) event3  - 2.4G RF Keyboard & Mouse: is tagged by udev as: Mouse
[    41.572] (II) event3  - 2.4G RF Keyboard & Mouse: device is a pointer
[    41.581] (II) config/udev: Adding input device 2.4G RF Keyboard & Mouse (/dev/input/mouse0)
[    41.581] (II) No input driver specified, ignoring this device.
[    41.581] (II) This device may have been added with another device file.
[    41.586] (II) config/udev: Adding input device 2.4G RF Keyboard & Mouse Consumer Control (/dev/input/event4)
[    41.587] (**) 2.4G RF Keyboard & Mouse Consumer Control: Applying InputClass "libinput keyboard catchall"
[    41.587] (II) Using input driver 'libinput' for '2.4G RF Keyboard & Mouse Consumer Control'
[    41.590] (II) systemd-logind: got fd for /dev/input/event4 13:68 fd 40 paused 0
[    41.591] (**) 2.4G RF Keyboard & Mouse Consumer Control: always reports core events
[    41.591] (**) Option "Device" "/dev/input/event4"
[    41.591] (**) Option "_source" "server/udev"
[    41.611] (II) event4  - 2.4G RF Keyboard & Mouse Consumer Control: is tagged by udev as: Keyboard
[    41.612] (II) event4  - 2.4G RF Keyboard & Mouse Consumer Control: device is a keyboard
[    41.613] (II) event4  - 2.4G RF Keyboard & Mouse Consumer Control: device removed
[    41.613] (II) libinput: 2.4G RF Keyboard & Mouse Consumer Control: needs a virtual subdevice
[    41.613] (**) Option "config_info" "udev:/sys/devices/platform/3610000.xhci/usb1/1-2/1-2.1/1-2.1:1.1/0003:3938:1192.0002/input/input4/event4"
[    41.613] (II) XINPUT: Adding extended input device "2.4G RF Keyboard & Mouse Consumer Control" (type: MOUSE, id 8)
[    41.614] (**) Option "AccelerationScheme" "none"
[    41.614] (**) 2.4G RF Keyboard & Mouse Consumer Control: (accel) selected scheme none/0
[    41.614] (**) 2.4G RF Keyboard & Mouse Consumer Control: (accel) acceleration factor: 2.000
[    41.614] (**) 2.4G RF Keyboard & Mouse Consumer Control: (accel) acceleration threshold: 4
[    41.625] (II) event4  - 2.4G RF Keyboard & Mouse Consumer Control: is tagged by udev as: Keyboard
[    41.625] (II) event4  - 2.4G RF Keyboard & Mouse Consumer Control: device is a keyboard
[    41.631] (II) config/udev: Adding input device 2.4G RF Keyboard & Mouse System Control (/dev/input/event5)
[    41.632] (**) 2.4G RF Keyboard & Mouse System Control: Applying InputClass "libinput keyboard catchall"
[    41.632] (II) Using input driver 'libinput' for '2.4G RF Keyboard & Mouse System Control'
[    41.636] (II) systemd-logind: got fd for /dev/input/event5 13:69 fd 41 paused 0
[    41.636] (**) 2.4G RF Keyboard & Mouse System Control: always reports core events
[    41.637] (**) Option "Device" "/dev/input/event5"
[    41.637] (**) Option "_source" "server/udev"
[    41.647] (II) event5  - 2.4G RF Keyboard & Mouse System Control: is tagged by udev as: Keyboard
[    41.648] (II) event5  - 2.4G RF Keyboard & Mouse System Control: device is a keyboard
[    41.648] (II) event5  - 2.4G RF Keyboard & Mouse System Control: device removed
[    41.649] (**) Option "config_info" "udev:/sys/devices/platform/3610000.xhci/usb1/1-2/1-2.1/1-2.1:1.1/0003:3938:1192.0002/input/input5/event5"
[    41.649] (II) XINPUT: Adding extended input device "2.4G RF Keyboard & Mouse System Control" (type: KEYBOARD, id 9)
[    41.649] (**) Option "xkb_model" "pc105"
[    41.649] (**) Option "xkb_layout" "us"
[    41.661] (II) event5  - 2.4G RF Keyboard & Mouse System Control: is tagged by udev as: Keyboard
[    41.661] (II) event5  - 2.4G RF Keyboard & Mouse System Control: device is a keyboard
[    41.665] (II) config/udev: Adding input device gpio-keys (/dev/input/event0)
[    41.666] (**) gpio-keys: Applying InputClass "libinput keyboard catchall"
[    41.666] (II) Using input driver 'libinput' for 'gpio-keys'
[    41.670] (II) systemd-logind: got fd for /dev/input/event0 13:64 fd 42 paused 0
[    41.670] (**) gpio-keys: always reports core events
[    41.670] (**) Option "Device" "/dev/input/event0"
[    41.670] (**) Option "_source" "server/udev"
[    41.678] (II) event0  - gpio-keys: is tagged by udev as: Keyboard
[    41.678] (II) event0  - gpio-keys: device is a keyboard
[    41.679] (II) event0  - gpio-keys: device removed
[    41.679] (**) Option "config_info" "udev:/sys/devices/platform/gpio-keys/input/input0/event0"
[    41.679] (II) XINPUT: Adding extended input device "gpio-keys" (type: KEYBOARD, id 10)
[    41.679] (**) Option "xkb_model" "pc105"
[    41.679] (**) Option "xkb_layout" "us"
[    41.684] (II) event0  - gpio-keys: is tagged by udev as: Keyboard
[    41.685] (II) event0  - gpio-keys: device is a keyboard
[    41.735] (**) 2.4G RF Keyboard & Mouse Consumer Control: Applying InputClass "libinput keyboard catchall"
[    41.735] (II) Using input driver 'libinput' for '2.4G RF Keyboard & Mouse Consumer Control'
[    41.735] (II) systemd-logind: returning pre-existing fd for /dev/input/event4 13:68
[    41.735] (**) 2.4G RF Keyboard & Mouse Consumer Control: always reports core events
[    41.735] (**) Option "Device" "/dev/input/event4"
[    41.736] (**) Option "_source" "_driver/libinput"
[    41.736] (II) libinput: 2.4G RF Keyboard & Mouse Consumer Control: is a virtual subdevice
[    41.736] (**) Option "config_info" "udev:/sys/devices/platform/3610000.xhci/usb1/1-2/1-2.1/1-2.1:1.1/0003:3938:1192.0002/input/input4/event4"
[    41.736] (II) XINPUT: Adding extended input device "2.4G RF Keyboard & Mouse Consumer Control" (type: KEYBOARD, id 11)
[    41.736] (**) Option "xkb_model" "pc105"
[    41.737] (**) Option "xkb_layout" "us"

OLLAMA LOG

May 31 12:57:03 dev-desktop ollama[18392]: time=2024-05-31T12:57:03.285-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs"
May 31 12:57:03 dev-desktop ollama[18392]: time=2024-05-31T12:57:03.287-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so*
May 31 12:57:03 dev-desktop ollama[18392]: time=2024-05-31T12:57:03.288-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li>
May 31 12:57:03 dev-desktop ollama[18392]: time=2024-05-31T12:57:03.306-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1]
May 31 12:57:03 dev-desktop ollama[18392]: CUDA driver version: 11.4
May 31 12:57:03 dev-desktop ollama[18392]: time=2024-05-31T12:57:03.306-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1
May 31 12:57:03 dev-desktop ollama[18392]: time=2024-05-31T12:57:03.306-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions"
May 31 12:57:03 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb
May 31 12:57:03 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5544 mb
May 31 12:57:03 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2
May 31 12:57:03 dev-desktop ollama[18392]: time=2024-05-31T12:57:03.426-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu"
May 31 12:57:03 dev-desktop ollama[18392]: releasing nvcuda library
May 31 12:57:03 dev-desktop ollama[18392]: time=2024-05-31T12:57:03.427-07:00 level=DEBUG source=gguf.go:57 msg="model = &llm.gguf{containerGGUF:(*llm.containerGGUF)(0x4000612000), kv:llm.KV{}, tensors:[]*llm.Te>
May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.471-07:00 level=DEBUG source=sched.go:153 msg="loading first model" model=/usr/share/ollama/.ollama/models/blobs/sha256-4fed7364ee3e0c7cb4fe088>
May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.471-07:00 level=DEBUG source=memory.go:44 msg=evaluating library=cuda gpu_count=1 available="5.4 GiB"
May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.474-07:00 level=INFO source=memory.go:133 msg="offload to gpu" layers.requested=-1 layers.real=33 memory.available="5.4 GiB" memory.required.fu>
May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.474-07:00 level=DEBUG source=sched.go:565 msg="new model will fit in available VRAM in single GPU, loading" model=/usr/share/ollama/.ollama/mod>
May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.475-07:00 level=DEBUG source=memory.go:44 msg=evaluating library=cuda gpu_count=1 available="5.4 GiB"
May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.477-07:00 level=INFO source=memory.go:133 msg="offload to gpu" layers.requested=-1 layers.real=33 memory.available="5.4 GiB" memory.required.fu>
May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.477-07:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3415314780/runners/cpu
May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.477-07:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3415314780/runners/cuda_v11
May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.477-07:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3415314780/runners/cpu
May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.477-07:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3415314780/runners/cuda_v11
May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.477-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions"
May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.478-07:00 level=INFO source=server.go:338 msg="starting llama server" cmd="/tmp/ollama3415314780/runners/cuda_v11/ollama_llama_server --model />
May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.478-07:00 level=DEBUG source=server.go:353 msg=subprocess environment="[PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/>
May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.480-07:00 level=INFO source=sched.go:338 msg="loaded runners" count=1
May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.480-07:00 level=INFO source=server.go:526 msg="waiting for llama runner to start responding"
May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.481-07:00 level=INFO source=server.go:564 msg="waiting for server to become available" status="llm server error"
May 31 12:57:04 dev-desktop ollama[18601]: INFO [main] build info | build=1 commit="74f33ad" tid="281472439437568" timestamp=1717185424
May 31 12:57:04 dev-desktop ollama[18601]: INFO [main] system info | n_threads=6 n_threads_batch=-1 system_info="AVX = 0 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 =>
May 31 12:57:04 dev-desktop ollama[18601]: INFO [main] HTTP server listening | hostname="127.0.0.1" n_threads_http="5" port="32905" tid="281472439437568" timestamp=1717185424
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: loaded meta data with 25 key-value pairs and 291 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-4fed7364ee3e0c7cb4fe0880148bfdfcd1b63098>
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv   0:                       general.architecture str              = llama
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv   1:                               general.name str              = LLaMA v2
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv   2:                           llama.vocab_size u32              = 32064
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv   3:                       llama.context_length u32              = 4096
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv   4:                     llama.embedding_length u32              = 3072
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv   5:                          llama.block_count u32              = 32
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv   6:                  llama.feed_forward_length u32              = 8192
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv   7:                 llama.rope.dimension_count u32              = 96
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv   8:                 llama.attention.head_count u32              = 32
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv   9:              llama.attention.head_count_kv u32              = 32
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv  10:     llama.attention.layer_norm_rms_epsilon f32              = 0.000010
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv  11:                       llama.rope.freq_base f32              = 10000.000000
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv  12:                          general.file_type u32              = 15
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv  13:                       tokenizer.ggml.model str              = llama
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv  14:                      tokenizer.ggml.tokens arr[str,32064]   = ["<unk>", "<s>", "</s>", "<0x00>", "<...
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv  15:                      tokenizer.ggml.scores arr[f32,32064]   = [0.000000, 0.000000, 0.000000, 0.0000...
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv  16:                  tokenizer.ggml.token_type arr[i32,32064]   = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ...
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv  17:                tokenizer.ggml.bos_token_id u32              = 1
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv  18:                tokenizer.ggml.eos_token_id u32              = 32000
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv  19:            tokenizer.ggml.unknown_token_id u32              = 0
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv  20:            tokenizer.ggml.padding_token_id u32              = 32000
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv  21:               tokenizer.ggml.add_bos_token bool             = true
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv  22:               tokenizer.ggml.add_eos_token bool             = false
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv  23:                    tokenizer.chat_template str              = {{ bos_token }}{% for message in mess...
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv  24:               general.quantization_version u32              = 2
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - type  f32:   65 tensors
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - type q4_K:  193 tensors
May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - type q6_K:   33 tensors
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_vocab: special tokens definition check successful ( 323/32064 ).
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: format           = GGUF V3 (latest)
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: arch             = llama
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: vocab type       = SPM
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_vocab          = 32064
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_merges         = 0
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_ctx_train      = 4096
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_embd           = 3072
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_head           = 32
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_head_kv        = 32
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_layer          = 32
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_rot            = 96
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_embd_head_k    = 96
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_embd_head_v    = 96
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_gqa            = 1
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_embd_k_gqa     = 3072
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_embd_v_gqa     = 3072
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: f_norm_eps       = 0.0e+00
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: f_norm_rms_eps   = 1.0e-05
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: f_clamp_kqv      = 0.0e+00
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: f_max_alibi_bias = 0.0e+00
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: f_logit_scale    = 0.0e+00
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_ff             = 8192
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_expert         = 0
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_expert_used    = 0
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: causal attn      = 1
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: pooling type     = 0
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: rope type        = 0
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: rope scaling     = linear
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: freq_base_train  = 10000.0
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: freq_scale_train = 1
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_yarn_orig_ctx  = 4096
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: rope_finetuned   = unknown
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: ssm_d_conv       = 0
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: ssm_d_inner      = 0
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: ssm_d_state      = 0
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: ssm_dt_rank      = 0
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: model type       = 7B
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: model ftype      = Q4_K - Medium
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: model params     = 3.82 B
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: model size       = 2.16 GiB (4.85 BPW)
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: general.name     = LLaMA v2
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: BOS token        = 1 '<s>'
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: EOS token        = 32000 '<|endoftext|>'
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: UNK token        = 0 '<unk>'
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: PAD token        = 32000 '<|endoftext|>'
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: LF token         = 13 '<0x0A>'
May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: EOT token        = 32007 '<|end|>'
May 31 12:57:04 dev-desktop ollama[18392]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ:   yes
May 31 12:57:04 dev-desktop ollama[18392]: ggml_cuda_init: CUDA_USE_TENSOR_CORES: no
May 31 12:57:04 dev-desktop ollama[18392]: ggml_cuda_init: found 1 CUDA devices:
May 31 12:57:04 dev-desktop ollama[18392]:   Device 0: Xavier, compute capability 7.2, VMM: yes
May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.734-07:00 level=INFO source=server.go:564 msg="waiting for server to become available" status="llm server loading model"
May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.390-07:00 level=WARN source=server.go:533 msg="client connection closed before server finished loading, aborting load"
May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.391-07:00 level=ERROR source=sched.go:344 msg="error loading llama server" error="timed out waiting for llama runner to start: context canceled"
May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.391-07:00 level=DEBUG source=sched.go:347 msg="triggering expiration for failed load" model=/usr/share/ollama/.ollama/models/blobs/sha256-4fed7>
May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.391-07:00 level=DEBUG source=sched.go:258 msg="runner expired event received" modelPath=/usr/share/ollama/.ollama/models/blobs/sha256-4fed7364e>
May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.391-07:00 level=DEBUG source=sched.go:274 msg="got lock to unload" modelPath=/usr/share/ollama/.ollama/models/blobs/sha256-4fed7364ee3e0c7cb4fe>
May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.391-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs"
May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.391-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so*
May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.391-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li>
May 31 12:57:23 dev-desktop ollama[18392]: [GIN] 2024/05/31 - 12:57:23 | 499 | 20.111113587s |       127.0.0.1 | POST     "/api/chat"
May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.408-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1]
May 31 12:57:23 dev-desktop ollama[18392]: CUDA driver version: 11.4
May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.409-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1
May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.409-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions"
May 31 12:57:23 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb
May 31 12:57:23 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5005 mb
May 31 12:57:23 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2
May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.531-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu"
May 31 12:57:23 dev-desktop ollama[18392]: releasing nvcuda library
May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.531-07:00 level=DEBUG source=server.go:987 msg="stopping llama server"
May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.532-07:00 level=DEBUG source=server.go:993 msg="waiting for llama server to exit"
May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.739-07:00 level=DEBUG source=server.go:997 msg="llama server stopped"
May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.740-07:00 level=DEBUG source=sched.go:279 msg="runner released" modelPath=/usr/share/ollama/.ollama/models/blobs/sha256-4fed7364ee3e0c7cb4fe088>
May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.783-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs"
May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.784-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so*
May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.784-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li>
May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.811-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1]
May 31 12:57:23 dev-desktop ollama[18392]: CUDA driver version: 11.4
May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.812-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1
May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.812-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions"
May 31 12:57:23 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb
May 31 12:57:23 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5554 mb
May 31 12:57:23 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2
May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.917-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu"
May 31 12:57:23 dev-desktop ollama[18392]: releasing nvcuda library
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.032-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs"
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.033-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so*
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.033-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li>
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.069-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1]
May 31 12:57:24 dev-desktop ollama[18392]: CUDA driver version: 11.4
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.070-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.071-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions"
May 31 12:57:24 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb
May 31 12:57:24 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5554 mb
May 31 12:57:24 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.173-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu"
May 31 12:57:24 dev-desktop ollama[18392]: releasing nvcuda library
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.283-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs"
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.283-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so*
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.283-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li>
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.303-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1]
May 31 12:57:24 dev-desktop ollama[18392]: CUDA driver version: 11.4
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.303-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.303-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions"
May 31 12:57:24 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb
May 31 12:57:24 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5554 mb
May 31 12:57:24 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.414-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu"
May 31 12:57:24 dev-desktop ollama[18392]: releasing nvcuda library
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.533-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs"
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.533-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so*
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.534-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li>
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.553-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1]
May 31 12:57:24 dev-desktop ollama[18392]: CUDA driver version: 11.4
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.554-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.554-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions"
May 31 12:57:24 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb
May 31 12:57:24 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5555 mb
May 31 12:57:24 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.655-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu"
May 31 12:57:24 dev-desktop ollama[18392]: releasing nvcuda library
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.782-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs"
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.782-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so*
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.782-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li>
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.801-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1]
May 31 12:57:24 dev-desktop ollama[18392]: CUDA driver version: 11.4
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.801-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.801-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions"
May 31 12:57:24 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb
May 31 12:57:24 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5555 mb
May 31 12:57:24 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2
May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.904-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu"
May 31 12:57:24 dev-desktop ollama[18392]: releasing nvcuda library
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.032-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs"
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.032-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so*
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.032-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li>
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.051-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1]
May 31 12:57:25 dev-desktop ollama[18392]: CUDA driver version: 11.4
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.051-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.051-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions"
May 31 12:57:25 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb
May 31 12:57:25 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5554 mb
May 31 12:57:25 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.154-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu"
May 31 12:57:25 dev-desktop ollama[18392]: releasing nvcuda library
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.283-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs"
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.283-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so*
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.283-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li>
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.304-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1]
May 31 12:57:25 dev-desktop ollama[18392]: CUDA driver version: 11.4
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.304-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.305-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions"
May 31 12:57:25 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb
May 31 12:57:25 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5555 mb
May 31 12:57:25 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.407-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu"
May 31 12:57:25 dev-desktop ollama[18392]: releasing nvcuda library
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.532-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs"
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.533-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so*
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.533-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li>
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.551-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1]
May 31 12:57:25 dev-desktop ollama[18392]: CUDA driver version: 11.4
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.551-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.551-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions"
May 31 12:57:25 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb
May 31 12:57:25 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5555 mb
May 31 12:57:25 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.653-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu"
May 31 12:57:25 dev-desktop ollama[18392]: releasing nvcuda library
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.782-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs"
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.782-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so*
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.783-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li>
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.802-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1]
May 31 12:57:25 dev-desktop ollama[18392]: CUDA driver version: 11.4
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.803-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.803-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions"
May 31 12:57:25 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb
May 31 12:57:25 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5555 mb
May 31 12:57:25 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2
May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.903-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu"
May 31 12:57:25 dev-desktop ollama[18392]: releasing nvcuda library
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.032-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs"
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.032-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so*
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.032-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li>
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.050-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1]
May 31 12:57:26 dev-desktop ollama[18392]: CUDA driver version: 11.4
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.051-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.051-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions"
May 31 12:57:26 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb
May 31 12:57:26 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5555 mb
May 31 12:57:26 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.152-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu"
May 31 12:57:26 dev-desktop ollama[18392]: releasing nvcuda library
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.283-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs"
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.283-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so*
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.283-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li>
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.301-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1]
May 31 12:57:26 dev-desktop ollama[18392]: CUDA driver version: 11.4
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.301-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.301-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions"
May 31 12:57:26 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb
May 31 12:57:26 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5554 mb
May 31 12:57:26 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.405-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu"
May 31 12:57:26 dev-desktop ollama[18392]: releasing nvcuda library
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.532-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs"
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.533-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so*
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.533-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li>
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.552-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1]
May 31 12:57:26 dev-desktop ollama[18392]: CUDA driver version: 11.4
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.553-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.553-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions"
May 31 12:57:26 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb
May 31 12:57:26 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5555 mb
May 31 12:57:26 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.655-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu"
May 31 12:57:26 dev-desktop ollama[18392]: releasing nvcuda library
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.782-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs"
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.782-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so*
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.782-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li>
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.801-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1]
May 31 12:57:26 dev-desktop ollama[18392]: CUDA driver version: 11.4
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.801-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.801-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions"
May 31 12:57:26 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb
May 31 12:57:26 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5555 mb
May 31 12:57:26 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2
May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.905-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu"
May 31 12:57:26 dev-desktop ollama[18392]: releasing nvcuda library
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.033-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs"
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.033-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so*
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.033-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li>
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.050-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1]
May 31 12:57:27 dev-desktop ollama[18392]: CUDA driver version: 11.4
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.051-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.051-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions"
May 31 12:57:27 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb
May 31 12:57:27 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5556 mb
May 31 12:57:27 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.151-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu"
May 31 12:57:27 dev-desktop ollama[18392]: releasing nvcuda library
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.283-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs"
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.283-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so*
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.283-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li>
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.301-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1]
May 31 12:57:27 dev-desktop ollama[18392]: CUDA driver version: 11.4
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.302-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.302-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions"
May 31 12:57:27 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb
May 31 12:57:27 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5556 mb
May 31 12:57:27 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.404-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu"
May 31 12:57:27 dev-desktop ollama[18392]: releasing nvcuda library
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.532-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs"
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.532-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so*
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.533-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li>
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.551-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1]
May 31 12:57:27 dev-desktop ollama[18392]: CUDA driver version: 11.4
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.552-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.552-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions"
May 31 12:57:27 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb
May 31 12:57:27 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5557 mb
May 31 12:57:27 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.651-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu"
May 31 12:57:27 dev-desktop ollama[18392]: releasing nvcuda library
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.782-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs"
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.782-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so*
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.782-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li>
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.802-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1]
May 31 12:57:27 dev-desktop ollama[18392]: CUDA driver version: 11.4
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.802-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.802-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions"
May 31 12:57:27 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb
May 31 12:57:27 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5557 mb
May 31 12:57:27 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2
May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.904-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu"
May 31 12:57:27 dev-desktop ollama[18392]: releasing nvcuda library
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.032-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs"
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.032-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so*
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.032-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li>
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.050-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1]
May 31 12:57:28 dev-desktop ollama[18392]: CUDA driver version: 11.4
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.051-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.051-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions"
May 31 12:57:28 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb
May 31 12:57:28 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5552 mb
May 31 12:57:28 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.176-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu"
May 31 12:57:28 dev-desktop ollama[18392]: releasing nvcuda library
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.282-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs"
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.283-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so*
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.283-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li>
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.304-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1]
May 31 12:57:28 dev-desktop ollama[18392]: CUDA driver version: 11.4
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.305-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.305-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions"
May 31 12:57:28 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb
May 31 12:57:28 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5552 mb
May 31 12:57:28 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.406-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu"
May 31 12:57:28 dev-desktop ollama[18392]: releasing nvcuda library
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.532-07:00 level=WARN source=sched.go:512 msg="gpu VRAM usage didn't recover within timeout" seconds=5.141294161
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.533-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs"
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.533-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so*
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.533-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li>
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.536-07:00 level=DEBUG source=sched.go:283 msg="sending an unloaded event" modelPath=/usr/share/ollama/.ollama/models/blobs/sha256-4fed7364ee3e0>
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.537-07:00 level=DEBUG source=sched.go:206 msg="ignoring unload event with no pending requests"
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.551-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1]
May 31 12:57:28 dev-desktop ollama[18392]: CUDA driver version: 11.4
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.551-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.551-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions"
May 31 12:57:28 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb
May 31 12:57:28 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5550 mb
May 31 12:57:28 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.661-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu"
May 31 12:57:28 dev-desktop ollama[18392]: releasing nvcuda library
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.782-07:00 level=WARN source=sched.go:512 msg="gpu VRAM usage didn't recover within timeout" seconds=5.391041361
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.782-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs"
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.782-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so*
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.782-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li>
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.800-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1]
May 31 12:57:28 dev-desktop ollama[18392]: CUDA driver version: 11.4
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.800-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.800-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions"
May 31 12:57:28 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb
May 31 12:57:28 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5550 mb
May 31 12:57:28 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2
May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.902-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu"
May 31 12:57:28 dev-desktop ollama[18392]: releasing nvcuda library
May 31 12:57:29 dev-desktop ollama[18392]: time=2024-05-31T12:57:29.032-07:00 level=WARN source=sched.go:512 msg="gpu VRAM usage didn't recover within timeout" seconds=5.640970132
May 31 12:57:31 dev-desktop ollama[18392]: [GIN] 2024/05/31 - 12:57:31 | 200 |     160.675µs |       127.0.0.1 | HEAD     "/"
May 31 12:57:31 dev-desktop ollama[18392]: [GIN] 2024/05/31 - 12:57:31 | 200 |    2.168833ms |       127.0.0.1 | POST     "/api/show"
May 31 12:57:31 dev-desktop ollama[18392]: [GIN] 2024/05/31 - 12:57:31 | 200 |     2.07632ms |       127.0.0.1 | POST     "/api/show"
May 31 12:57:31 dev-desktop ollama[18392]: time=2024-05-31T12:57:31.351-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs"
May 31 12:57:31 dev-desktop ollama[18392]: time=2024-05-31T12:57:31.351-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so*
May 31 12:57:31 dev-desktop ollama[18392]: time=2024-05-31T12:57:31.351-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li>
May 31 12:57:31 dev-desktop ollama[18392]: time=2024-05-31T12:57:31.369-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1]
May 31 12:57:31 dev-desktop ollama[18392]: CUDA driver version: 11.4
May 31 12:57:31 dev-desktop ollama[18392]: time=2024-05-31T12:57:31.370-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1
May 31 12:57:31 dev-desktop ollama[18392]: time=2024-05-31T12:57:31.370-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions"
May 31 12:57:31 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb
May 31 12:57:31 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5540 mb
May 31 12:57:31 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2
May 31 12:57:31 dev-desktop ollama[18392]: time=2024-05-31T12:57:31.473-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu"
May 31 12:57:31 dev-desktop ollama[18392]: releasing nvcuda library
May 31 12:57:31 dev-desktop ollama[18392]: time=2024-05-31T12:57:31.474-07:00 level=DEBUG source=gguf.go:57 msg="model = &llm.gguf{containerGGUF:(*llm.containerGGUF)(0x400049e000), kv:llm.KV{}, tensors:[]*llm.Te>
May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.449-07:00 level=DEBUG source=sched.go:153 msg="loading first model" model=/usr/share/ollama/.ollama/models/blobs/sha256-4fed7364ee3e0c7cb4fe088>
May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.450-07:00 level=DEBUG source=memory.go:44 msg=evaluating library=cuda gpu_count=1 available="5.4 GiB"
May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.453-07:00 level=INFO source=memory.go:133 msg="offload to gpu" layers.requested=-1 layers.real=33 memory.available="5.4 GiB" memory.required.fu>
May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.453-07:00 level=DEBUG source=sched.go:565 msg="new model will fit in available VRAM in single GPU, loading" model=/usr/share/ollama/.ollama/mod>
May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.454-07:00 level=DEBUG source=memory.go:44 msg=evaluating library=cuda gpu_count=1 available="5.4 GiB"
May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.456-07:00 level=INFO source=memory.go:133 msg="offload to gpu" layers.requested=-1 layers.real=33 memory.available="5.4 GiB" memory.required.fu>
May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.456-07:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3415314780/runners/cpu
May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.456-07:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3415314780/runners/cuda_v11
May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.457-07:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3415314780/runners/cpu
May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.457-07:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3415314780/runners/cuda_v11
May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.457-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions"
May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.457-07:00 level=INFO source=server.go:338 msg="starting llama server" cmd="/tmp/ollama3415314780/runners/cuda_v11/ollama_llama_server --model />
May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.457-07:00 level=DEBUG source=server.go:353 msg=subprocess environment="[PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/>
May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.459-07:00 level=INFO source=sched.go:338 msg="loaded runners" count=1
May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.459-07:00 level=INFO source=server.go:526 msg="waiting for llama runner to start responding"
May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.460-07:00 level=INFO source=server.go:564 msg="waiting for server to become available" status="llm server error"
May 31 12:57:32 dev-desktop ollama[18643]: INFO [main] build info | build=1 commit="74f33ad" tid="281472994298112" timestamp=1717185452
May 31 12:57:32 dev-desktop ollama[18643]: INFO [main] system info | n_threads=6 n_threads_batch=-1 system_info="AVX = 0 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 =>
May 31 12:57:32 dev-desktop ollama[18643]: INFO [main] HTTP server listening | hostname="127.0.0.1" n_threads_http="5" port="42441" tid="281472994298112" timestamp=1717185452
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: loaded meta data with 25 key-value pairs and 291 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-4fed7364ee3e0c7cb4fe0880148bfdfcd1b63098>
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv   0:                       general.architecture str              = llama
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv   1:                               general.name str              = LLaMA v2
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv   2:                           llama.vocab_size u32              = 32064
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv   3:                       llama.context_length u32              = 4096
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv   4:                     llama.embedding_length u32              = 3072
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv   5:                          llama.block_count u32              = 32
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv   6:                  llama.feed_forward_length u32              = 8192
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv   7:                 llama.rope.dimension_count u32              = 96
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv   8:                 llama.attention.head_count u32              = 32
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv   9:              llama.attention.head_count_kv u32              = 32
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv  10:     llama.attention.layer_norm_rms_epsilon f32              = 0.000010
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv  11:                       llama.rope.freq_base f32              = 10000.000000
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv  12:                          general.file_type u32              = 15
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv  13:                       tokenizer.ggml.model str              = llama
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv  14:                      tokenizer.ggml.tokens arr[str,32064]   = ["<unk>", "<s>", "</s>", "<0x00>", "<...
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv  15:                      tokenizer.ggml.scores arr[f32,32064]   = [0.000000, 0.000000, 0.000000, 0.0000...
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv  16:                  tokenizer.ggml.token_type arr[i32,32064]   = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ...
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv  17:                tokenizer.ggml.bos_token_id u32              = 1
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv  18:                tokenizer.ggml.eos_token_id u32              = 32000
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv  19:            tokenizer.ggml.unknown_token_id u32              = 0
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv  20:            tokenizer.ggml.padding_token_id u32              = 32000
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv  21:               tokenizer.ggml.add_bos_token bool             = true
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv  22:               tokenizer.ggml.add_eos_token bool             = false
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv  23:                    tokenizer.chat_template str              = {{ bos_token }}{% for message in mess...
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv  24:               general.quantization_version u32              = 2
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - type  f32:   65 tensors
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - type q4_K:  193 tensors
May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - type q6_K:   33 tensors
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_vocab: special tokens definition check successful ( 323/32064 ).
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: format           = GGUF V3 (latest)
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: arch             = llama
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: vocab type       = SPM
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_vocab          = 32064
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_merges         = 0
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_ctx_train      = 4096
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_embd           = 3072
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_head           = 32
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_head_kv        = 32
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_layer          = 32
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_rot            = 96
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_embd_head_k    = 96
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_embd_head_v    = 96
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_gqa            = 1
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_embd_k_gqa     = 3072
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_embd_v_gqa     = 3072
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: f_norm_eps       = 0.0e+00
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: f_norm_rms_eps   = 1.0e-05
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: f_clamp_kqv      = 0.0e+00
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: f_max_alibi_bias = 0.0e+00
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: f_logit_scale    = 0.0e+00
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_ff             = 8192
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_expert         = 0
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_expert_used    = 0
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: causal attn      = 1
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: pooling type     = 0
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: rope type        = 0
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: rope scaling     = linear
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: freq_base_train  = 10000.0
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: freq_scale_train = 1
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_yarn_orig_ctx  = 4096
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: rope_finetuned   = unknown
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: ssm_d_conv       = 0
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: ssm_d_inner      = 0
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: ssm_d_state      = 0
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: ssm_dt_rank      = 0
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: model type       = 7B
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: model ftype      = Q4_K - Medium
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: model params     = 3.82 B
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: model size       = 2.16 GiB (4.85 BPW)
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: general.name     = LLaMA v2
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: BOS token        = 1 '<s>'
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: EOS token        = 32000 '<|endoftext|>'
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: UNK token        = 0 '<unk>'
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: PAD token        = 32000 '<|endoftext|>'
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: LF token         = 13 '<0x0A>'
May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: EOT token        = 32007 '<|end|>'
May 31 12:57:32 dev-desktop ollama[18392]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ:   yes
May 31 12:57:32 dev-desktop ollama[18392]: ggml_cuda_init: CUDA_USE_TENSOR_CORES: no
May 31 12:57:32 dev-desktop ollama[18392]: ggml_cuda_init: found 1 CUDA devices:
May 31 12:57:32 dev-desktop ollama[18392]:   Device 0: Xavier, compute capability 7.2, VMM: yes
May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.713-07:00 level=INFO source=server.go:564 msg="waiting for server to become available" status="llm server loading model"
lines 1301-
<!-- gh-comment-id:2142905838 --> @ZanMax commented on GitHub (May 31, 2024): I think Nvidia Xavier has other drivers instead of standard. Log: ``` cat /var/log/Xorg.0.log [ 40.584] (--) Log file renamed from "/var/log/Xorg.pid-2645.log" to "/var/log/Xorg.0.log" [ 40.588] X.Org X Server 1.20.13 X Protocol Version 11, Revision 0 [ 40.588] Build Operating System: linux Ubuntu [ 40.588] Current Operating System: Linux dev-desktop 5.10.104-tegra #1 SMP PREEMPT Wed Aug 10 20:17:07 PDT 2022 aarch64 [ 40.588] Kernel command line: root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyTCU0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0 [ 40.588] Build Date: 09 April 2024 01:36:10AM [ 40.588] xorg-server 2:1.20.13-1ubuntu1~20.04.17 (For technical support please see http://www.ubuntu.com/support) [ 40.589] Current version of pixman: 0.38.4 [ 40.589] Before reporting problems, check http://wiki.x.org to make sure that you have the latest version. [ 40.589] Markers: (--) probed, (**) from config file, (==) default setting, (++) from command line, (!!) notice, (II) informational, (WW) warning, (EE) error, (NI) not implemented, (??) unknown. [ 40.589] (==) Log file: "/var/log/Xorg.0.log", Time: Thu May 30 09:06:55 2024 [ 40.596] (==) Using config file: "/etc/X11/xorg.conf" [ 40.597] (==) Using config directory: "/etc/X11/xorg.conf.d" [ 40.597] (==) Using system config directory "/usr/share/X11/xorg.conf.d" [ 40.600] (==) No Layout section. Using the first Screen section. [ 40.600] (==) No screen section available. Using defaults. [ 40.600] (**) |-->Screen "Default Screen Section" (0) [ 40.600] (**) | |-->Monitor "<default monitor>" [ 40.602] (==) No device specified for screen "Default Screen Section". Using the first device section listed. [ 40.602] (**) | |-->Device "Tegra0" [ 40.602] (==) No monitor specified for screen "Default Screen Section". Using a default monitor configuration. [ 40.602] (**) Option "AutoAddGPU" "false" [ 40.602] (==) Automatically adding devices [ 40.603] (==) Automatically enabling devices [ 40.603] (**) Not automatically adding GPU devices [ 40.603] (==) Automatically binding GPU devices [ 40.603] (==) Max clients allowed: 256, resource mask: 0x1fffff [ 40.609] (WW) The directory "/usr/share/fonts/X11/cyrillic" does not exist. [ 40.609] Entry deleted from font path. [ 40.609] (WW) The directory "/usr/share/fonts/X11/100dpi/" does not exist. [ 40.610] Entry deleted from font path. [ 40.610] (WW) The directory "/usr/share/fonts/X11/75dpi/" does not exist. [ 40.610] Entry deleted from font path. [ 40.611] (WW) The directory "/usr/share/fonts/X11/100dpi" does not exist. [ 40.611] Entry deleted from font path. [ 40.611] (WW) The directory "/usr/share/fonts/X11/75dpi" does not exist. [ 40.611] Entry deleted from font path. [ 40.612] (==) FontPath set to: /usr/share/fonts/X11/misc, /usr/share/fonts/X11/Type1, built-ins [ 40.612] (==) ModulePath set to "/usr/lib/xorg/modules" [ 40.612] (II) The server relies on udev to provide the list of input devices. If no devices become available, reconfigure udev or disable AutoAddDevices. [ 40.612] (II) Loader magic: 0xaaaab6cdc010 [ 40.612] (II) Module ABI versions: [ 40.612] X.Org ANSI C Emulation: 0.4 [ 40.612] X.Org Video Driver: 24.1 [ 40.612] X.Org XInput driver : 24.1 [ 40.612] X.Org Server Extension : 10.0 [ 40.617] (++) using VT number 1 [ 40.629] (II) systemd-logind: took control of session /org/freedesktop/login1/session/c1 [ 40.633] (II) no primary bus or device found [ 40.633] (WW) "dri" will not be loaded unless you've specified it to be loaded elsewhere. [ 40.633] (II) "glx" will be loaded by default. [ 40.633] (II) LoadModule: "extmod" [ 40.634] (II) Module "extmod" already built-in [ 40.634] (II) LoadModule: "glx" [ 40.638] (II) Loading /usr/lib/xorg/modules/extensions/libglx.so [ 40.656] (II) Module glx: vendor="X.Org Foundation" [ 40.656] compiled for 1.20.13, module version = 1.0.0 [ 40.656] ABI class: X.Org Server Extension, version 10.0 [ 40.656] (II) LoadModule: "nvidia" [ 40.657] (II) Loading /usr/lib/xorg/modules/drivers/nvidia_drv.so [ 40.684] (II) Module nvidia: vendor="NVIDIA Corporation" [ 40.684] compiled for 1.6.99.901, module version = 1.0.0 [ 40.684] Module class: X.Org Video Driver [ 40.690] (II) NVIDIA dlloader X Driver 35.1.0 Release Build (bugfix_main) (buildbrain@mobile-u64-5562-d3000) Thu Aug 25 11:12:05 PDT 2022 [ 40.690] (II) NVIDIA Unified Driver for all Supported NVIDIA GPUs [ 40.691] (WW) Falling back to old probe method for NVIDIA [ 40.693] (II) Loading sub module "fb" [ 40.693] (II) LoadModule: "fb" [ 40.693] (II) Loading /usr/lib/xorg/modules/libfb.so [ 40.695] (II) Module fb: vendor="X.Org Foundation" [ 40.695] compiled for 1.20.13, module version = 1.0.0 [ 40.695] ABI class: X.Org ANSI C Emulation, version 0.4 [ 40.695] (II) Loading sub module "wfb" [ 40.695] (II) LoadModule: "wfb" [ 40.698] (II) Loading /usr/lib/xorg/modules/libwfb.so [ 40.700] (II) Module wfb: vendor="X.Org Foundation" [ 40.700] compiled for 1.20.13, module version = 1.0.0 [ 40.700] ABI class: X.Org ANSI C Emulation, version 0.4 [ 40.700] (II) Loading sub module "ramdac" [ 40.700] (II) LoadModule: "ramdac" [ 40.700] (II) Module "ramdac" already built-in [ 40.705] (WW) VGA arbiter: cannot open kernel arbiter, no multi-card support [ 40.706] (II) NVIDIA(0): Creating default Display subsection in Screen section "Default Screen Section" for depth/fbbpp 24/32 [ 40.706] (==) NVIDIA(0): Depth 24, (==) framebuffer bpp 32 [ 40.706] (==) NVIDIA(0): RGB weight 888 [ 40.706] (==) NVIDIA(0): Default visual is TrueColor [ 40.706] (==) NVIDIA(0): Using gamma correction (1.0, 1.0, 1.0) [ 40.706] (DB) xf86MergeOutputClassOptions unsupported bus type 0 [ 40.706] (**) NVIDIA(0): Option "AllowEmptyInitialConfiguration" "true" [ 40.707] (**) NVIDIA(0): Enabling 2D acceleration [ 40.707] (II) Loading sub module "glxserver_nvidia" [ 40.707] (II) LoadModule: "glxserver_nvidia" [ 40.708] (II) Loading /usr/lib/xorg/modules/extensions/libglxserver_nvidia.so [ 40.802] (II) Module glxserver_nvidia: vendor="NVIDIA Corporation" [ 40.802] compiled for 1.6.99.901, module version = 1.0.0 [ 40.802] Module class: X.Org Server Extension [ 40.802] (II) NVIDIA GLX Module 35.1.0 Release Build (bugfix_main) (buildbrain@mobile-u64-5562-d3000) Thu Aug 25 11:11:11 PDT 2022 [ 40.806] (II) NVIDIA: The X server supports PRIME Render Offload. [ 40.811] (--) NVIDIA(0): Valid display device(s) on GPU-0 at SoC [ 40.812] (--) NVIDIA(0): DFP-0 [ 40.812] (--) NVIDIA(0): DFP-1 [ 40.812] (WW) NVIDIA: No DRM device: No direct render devices found. [ 40.814] (II) NVIDIA(0): NVIDIA GPU Xavier (nvgpu) (GV11B) at SoC (GPU-0) [ 40.814] (--) NVIDIA(0): Memory: 7021844 kBytes [ 40.814] (--) NVIDIA(0): VideoBIOS: [ 40.815] (==) NVIDIA(0): [ 40.815] (==) NVIDIA(0): No modes were requested; the default mode "nvidia-auto-select" [ 40.815] (==) NVIDIA(0): will be used as the requested mode. [ 40.815] (==) NVIDIA(0): [ 40.815] (--) NVIDIA(0): No enabled display devices found; starting anyway because [ 40.815] (--) NVIDIA(0): AllowEmptyInitialConfiguration is enabled [ 40.815] (II) NVIDIA(0): Validated MetaModes: [ 40.815] (II) NVIDIA(0): "NULL" [ 40.816] (II) NVIDIA(0): Virtual screen size determined to be 640 x 480 [ 40.816] (WW) NVIDIA(0): Unable to get display device for DPI computation. [ 40.816] (==) NVIDIA(0): DPI set to (75, 75); computed from built-in default [ 40.817] (II) NVIDIA: Reserving 24576.00 MB of virtual memory for indirect memory [ 40.817] (II) NVIDIA: access. [ 40.817] (WW) NVIDIA(GPU-0): Failed to enter interactive mode. [ 40.822] (EE) NVIDIA(0): Failed to allocate NVIDIA Error Handler [ 40.822] (II) NVIDIA(0): ACPI: failed to connect to the ACPI event daemon; the daemon [ 40.822] (II) NVIDIA(0): may not be running or the "AcpidSocketPath" X [ 40.822] (II) NVIDIA(0): configuration option may not be set correctly. When the [ 40.822] (II) NVIDIA(0): ACPI event daemon is available, the NVIDIA X driver will [ 40.823] (II) NVIDIA(0): try to use it to receive ACPI event notifications. For [ 40.823] (II) NVIDIA(0): details, please see the "ConnectToAcpid" and [ 40.823] (II) NVIDIA(0): "AcpidSocketPath" X configuration options in Appendix B: X [ 40.823] (II) NVIDIA(0): Config Options in the README. [ 40.912] (II) NVIDIA(0): Setting mode "NULL" [ 40.931] (==) NVIDIA(0): Disabling shared memory pixmaps [ 40.931] (==) NVIDIA(0): Backing store enabled [ 40.932] (==) NVIDIA(0): Silken mouse enabled [ 40.935] (==) NVIDIA(0): DPMS enabled [ 40.936] (II) Loading sub module "dri2" [ 40.936] (II) LoadModule: "dri2" [ 40.936] (II) Module "dri2" already built-in [ 40.936] (II) NVIDIA(0): [DRI2] Setup complete [ 40.936] (II) NVIDIA(0): [DRI2] VDPAU driver: nvidia [ 40.938] (II) Initializing extension Generic Event Extension [ 40.939] (II) Initializing extension SHAPE [ 40.940] (II) Initializing extension MIT-SHM [ 40.941] (II) Initializing extension XInputExtension [ 40.943] (II) Initializing extension XTEST [ 40.944] (II) Initializing extension BIG-REQUESTS [ 40.945] (II) Initializing extension SYNC [ 40.945] (II) Initializing extension XKEYBOARD [ 40.946] (II) Initializing extension XC-MISC [ 40.947] (II) Initializing extension SECURITY [ 40.947] (II) Initializing extension XFIXES [ 40.948] (II) Initializing extension RENDER [ 40.949] (II) Initializing extension RANDR [ 40.950] (II) Initializing extension COMPOSITE [ 40.950] (II) Initializing extension DAMAGE [ 40.951] (II) Initializing extension MIT-SCREEN-SAVER [ 40.952] (II) Initializing extension DOUBLE-BUFFER [ 40.952] (II) Initializing extension RECORD [ 40.953] (II) Initializing extension DPMS [ 40.954] (II) Initializing extension Present [ 40.954] (II) Initializing extension DRI3 [ 40.955] (II) Initializing extension X-Resource [ 40.956] (II) Initializing extension XVideo [ 40.957] (II) Initializing extension XVideo-MotionCompensation [ 40.957] (II) Initializing extension SELinux [ 40.957] (II) SELinux: Disabled on system [ 40.957] (II) Initializing extension GLX [ 40.958] (II) Initializing extension GLX [ 40.958] (II) Indirect GLX disabled. [ 40.959] (II) GLX: Another vendor is already registered for screen 0 [ 40.959] (II) Initializing extension XFree86-VidModeExtension [ 40.960] (II) Initializing extension XFree86-DGA [ 40.961] (II) Initializing extension XFree86-DRI [ 40.962] (II) Initializing extension DRI2 [ 40.963] (II) Initializing extension NV-GLX [ 40.964] (II) Initializing extension NV-CONTROL [ 41.389] (II) config/udev: Adding input device NVIDIA Jetson Xavier NX HDA HDMI/DP,pcm=9 (/dev/input/event9) [ 41.389] (II) No input driver specified, ignoring this device. [ 41.389] (II) This device may have been added with another device file. [ 41.392] (II) config/udev: Adding input device NVIDIA Jetson Xavier NX HDA HDMI/DP,pcm=3 (/dev/input/event6) [ 41.392] (II) No input driver specified, ignoring this device. [ 41.392] (II) This device may have been added with another device file. [ 41.395] (II) config/udev: Adding input device NVIDIA Jetson Xavier NX HDA HDMI/DP,pcm=7 (/dev/input/event7) [ 41.396] (II) No input driver specified, ignoring this device. [ 41.396] (II) This device may have been added with another device file. [ 41.398] (II) config/udev: Adding input device NVIDIA Jetson Xavier NX HDA HDMI/DP,pcm=8 (/dev/input/event8) [ 41.398] (II) No input driver specified, ignoring this device. [ 41.398] (II) This device may have been added with another device file. [ 41.404] (II) config/udev: Adding input device 2.4G RF Keyboard & Mouse (/dev/input/event1) [ 41.404] (**) 2.4G RF Keyboard & Mouse: Applying InputClass "libinput keyboard catchall" [ 41.404] (II) LoadModule: "libinput" [ 41.406] (II) Loading /usr/lib/xorg/modules/input/libinput_drv.so [ 41.423] (II) Module libinput: vendor="X.Org Foundation" [ 41.423] compiled for 1.20.4, module version = 0.29.0 [ 41.423] Module class: X.Org XInput Driver [ 41.424] ABI class: X.Org XInput driver, version 24.1 [ 41.424] (II) Using input driver 'libinput' for '2.4G RF Keyboard & Mouse' [ 41.434] (II) systemd-logind: got fd for /dev/input/event1 13:65 fd 36 paused 0 [ 41.434] (**) 2.4G RF Keyboard & Mouse: always reports core events [ 41.435] (**) Option "Device" "/dev/input/event1" [ 41.435] (**) Option "_source" "server/udev" [ 41.477] (II) event1 - 2.4G RF Keyboard & Mouse: is tagged by udev as: Keyboard [ 41.479] (II) event1 - 2.4G RF Keyboard & Mouse: device is a keyboard [ 41.480] (II) event1 - 2.4G RF Keyboard & Mouse: device removed [ 41.480] (**) Option "config_info" "udev:/sys/devices/platform/3610000.xhci/usb1/1-2/1-2.1/1-2.1:1.0/0003:3938:1192.0001/input/input1/event1" [ 41.481] (II) XINPUT: Adding extended input device "2.4G RF Keyboard & Mouse" (type: KEYBOARD, id 6) [ 41.481] (**) Option "xkb_model" "pc105" [ 41.481] (**) Option "xkb_layout" "us" [ 41.499] (II) event1 - 2.4G RF Keyboard & Mouse: is tagged by udev as: Keyboard [ 41.501] (II) event1 - 2.4G RF Keyboard & Mouse: device is a keyboard [ 41.512] (II) config/udev: Adding input device 2.4G RF Keyboard & Mouse (/dev/input/event2) [ 41.513] (II) No input driver specified, ignoring this device. [ 41.513] (II) This device may have been added with another device file. [ 41.519] (II) config/udev: Adding input device 2.4G RF Keyboard & Mouse (/dev/input/event3) [ 41.519] (**) 2.4G RF Keyboard & Mouse: Applying InputClass "libinput pointer catchall" [ 41.519] (II) Using input driver 'libinput' for '2.4G RF Keyboard & Mouse' [ 41.527] (II) systemd-logind: got fd for /dev/input/event3 13:67 fd 39 paused 0 [ 41.527] (**) 2.4G RF Keyboard & Mouse: always reports core events [ 41.527] (**) Option "Device" "/dev/input/event3" [ 41.528] (**) Option "_source" "server/udev" [ 41.548] (II) event3 - 2.4G RF Keyboard & Mouse: is tagged by udev as: Mouse [ 41.549] (II) event3 - 2.4G RF Keyboard & Mouse: device is a pointer [ 41.550] (II) event3 - 2.4G RF Keyboard & Mouse: device removed [ 41.551] (**) Option "config_info" "udev:/sys/devices/platform/3610000.xhci/usb1/1-2/1-2.1/1-2.1:1.1/0003:3938:1192.0002/input/input3/event3" [ 41.551] (II) XINPUT: Adding extended input device "2.4G RF Keyboard & Mouse" (type: MOUSE, id 7) [ 41.552] (**) Option "AccelerationScheme" "none" [ 41.552] (**) 2.4G RF Keyboard & Mouse: (accel) selected scheme none/0 [ 41.552] (**) 2.4G RF Keyboard & Mouse: (accel) acceleration factor: 2.000 [ 41.552] (**) 2.4G RF Keyboard & Mouse: (accel) acceleration threshold: 4 [ 41.568] (II) event3 - 2.4G RF Keyboard & Mouse: is tagged by udev as: Mouse [ 41.572] (II) event3 - 2.4G RF Keyboard & Mouse: device is a pointer [ 41.581] (II) config/udev: Adding input device 2.4G RF Keyboard & Mouse (/dev/input/mouse0) [ 41.581] (II) No input driver specified, ignoring this device. [ 41.581] (II) This device may have been added with another device file. [ 41.586] (II) config/udev: Adding input device 2.4G RF Keyboard & Mouse Consumer Control (/dev/input/event4) [ 41.587] (**) 2.4G RF Keyboard & Mouse Consumer Control: Applying InputClass "libinput keyboard catchall" [ 41.587] (II) Using input driver 'libinput' for '2.4G RF Keyboard & Mouse Consumer Control' [ 41.590] (II) systemd-logind: got fd for /dev/input/event4 13:68 fd 40 paused 0 [ 41.591] (**) 2.4G RF Keyboard & Mouse Consumer Control: always reports core events [ 41.591] (**) Option "Device" "/dev/input/event4" [ 41.591] (**) Option "_source" "server/udev" [ 41.611] (II) event4 - 2.4G RF Keyboard & Mouse Consumer Control: is tagged by udev as: Keyboard [ 41.612] (II) event4 - 2.4G RF Keyboard & Mouse Consumer Control: device is a keyboard [ 41.613] (II) event4 - 2.4G RF Keyboard & Mouse Consumer Control: device removed [ 41.613] (II) libinput: 2.4G RF Keyboard & Mouse Consumer Control: needs a virtual subdevice [ 41.613] (**) Option "config_info" "udev:/sys/devices/platform/3610000.xhci/usb1/1-2/1-2.1/1-2.1:1.1/0003:3938:1192.0002/input/input4/event4" [ 41.613] (II) XINPUT: Adding extended input device "2.4G RF Keyboard & Mouse Consumer Control" (type: MOUSE, id 8) [ 41.614] (**) Option "AccelerationScheme" "none" [ 41.614] (**) 2.4G RF Keyboard & Mouse Consumer Control: (accel) selected scheme none/0 [ 41.614] (**) 2.4G RF Keyboard & Mouse Consumer Control: (accel) acceleration factor: 2.000 [ 41.614] (**) 2.4G RF Keyboard & Mouse Consumer Control: (accel) acceleration threshold: 4 [ 41.625] (II) event4 - 2.4G RF Keyboard & Mouse Consumer Control: is tagged by udev as: Keyboard [ 41.625] (II) event4 - 2.4G RF Keyboard & Mouse Consumer Control: device is a keyboard [ 41.631] (II) config/udev: Adding input device 2.4G RF Keyboard & Mouse System Control (/dev/input/event5) [ 41.632] (**) 2.4G RF Keyboard & Mouse System Control: Applying InputClass "libinput keyboard catchall" [ 41.632] (II) Using input driver 'libinput' for '2.4G RF Keyboard & Mouse System Control' [ 41.636] (II) systemd-logind: got fd for /dev/input/event5 13:69 fd 41 paused 0 [ 41.636] (**) 2.4G RF Keyboard & Mouse System Control: always reports core events [ 41.637] (**) Option "Device" "/dev/input/event5" [ 41.637] (**) Option "_source" "server/udev" [ 41.647] (II) event5 - 2.4G RF Keyboard & Mouse System Control: is tagged by udev as: Keyboard [ 41.648] (II) event5 - 2.4G RF Keyboard & Mouse System Control: device is a keyboard [ 41.648] (II) event5 - 2.4G RF Keyboard & Mouse System Control: device removed [ 41.649] (**) Option "config_info" "udev:/sys/devices/platform/3610000.xhci/usb1/1-2/1-2.1/1-2.1:1.1/0003:3938:1192.0002/input/input5/event5" [ 41.649] (II) XINPUT: Adding extended input device "2.4G RF Keyboard & Mouse System Control" (type: KEYBOARD, id 9) [ 41.649] (**) Option "xkb_model" "pc105" [ 41.649] (**) Option "xkb_layout" "us" [ 41.661] (II) event5 - 2.4G RF Keyboard & Mouse System Control: is tagged by udev as: Keyboard [ 41.661] (II) event5 - 2.4G RF Keyboard & Mouse System Control: device is a keyboard [ 41.665] (II) config/udev: Adding input device gpio-keys (/dev/input/event0) [ 41.666] (**) gpio-keys: Applying InputClass "libinput keyboard catchall" [ 41.666] (II) Using input driver 'libinput' for 'gpio-keys' [ 41.670] (II) systemd-logind: got fd for /dev/input/event0 13:64 fd 42 paused 0 [ 41.670] (**) gpio-keys: always reports core events [ 41.670] (**) Option "Device" "/dev/input/event0" [ 41.670] (**) Option "_source" "server/udev" [ 41.678] (II) event0 - gpio-keys: is tagged by udev as: Keyboard [ 41.678] (II) event0 - gpio-keys: device is a keyboard [ 41.679] (II) event0 - gpio-keys: device removed [ 41.679] (**) Option "config_info" "udev:/sys/devices/platform/gpio-keys/input/input0/event0" [ 41.679] (II) XINPUT: Adding extended input device "gpio-keys" (type: KEYBOARD, id 10) [ 41.679] (**) Option "xkb_model" "pc105" [ 41.679] (**) Option "xkb_layout" "us" [ 41.684] (II) event0 - gpio-keys: is tagged by udev as: Keyboard [ 41.685] (II) event0 - gpio-keys: device is a keyboard [ 41.735] (**) 2.4G RF Keyboard & Mouse Consumer Control: Applying InputClass "libinput keyboard catchall" [ 41.735] (II) Using input driver 'libinput' for '2.4G RF Keyboard & Mouse Consumer Control' [ 41.735] (II) systemd-logind: returning pre-existing fd for /dev/input/event4 13:68 [ 41.735] (**) 2.4G RF Keyboard & Mouse Consumer Control: always reports core events [ 41.735] (**) Option "Device" "/dev/input/event4" [ 41.736] (**) Option "_source" "_driver/libinput" [ 41.736] (II) libinput: 2.4G RF Keyboard & Mouse Consumer Control: is a virtual subdevice [ 41.736] (**) Option "config_info" "udev:/sys/devices/platform/3610000.xhci/usb1/1-2/1-2.1/1-2.1:1.1/0003:3938:1192.0002/input/input4/event4" [ 41.736] (II) XINPUT: Adding extended input device "2.4G RF Keyboard & Mouse Consumer Control" (type: KEYBOARD, id 11) [ 41.736] (**) Option "xkb_model" "pc105" [ 41.737] (**) Option "xkb_layout" "us" ``` **OLLAMA LOG** ``` May 31 12:57:03 dev-desktop ollama[18392]: time=2024-05-31T12:57:03.285-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs" May 31 12:57:03 dev-desktop ollama[18392]: time=2024-05-31T12:57:03.287-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so* May 31 12:57:03 dev-desktop ollama[18392]: time=2024-05-31T12:57:03.288-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li> May 31 12:57:03 dev-desktop ollama[18392]: time=2024-05-31T12:57:03.306-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1] May 31 12:57:03 dev-desktop ollama[18392]: CUDA driver version: 11.4 May 31 12:57:03 dev-desktop ollama[18392]: time=2024-05-31T12:57:03.306-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1 May 31 12:57:03 dev-desktop ollama[18392]: time=2024-05-31T12:57:03.306-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions" May 31 12:57:03 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb May 31 12:57:03 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5544 mb May 31 12:57:03 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2 May 31 12:57:03 dev-desktop ollama[18392]: time=2024-05-31T12:57:03.426-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu" May 31 12:57:03 dev-desktop ollama[18392]: releasing nvcuda library May 31 12:57:03 dev-desktop ollama[18392]: time=2024-05-31T12:57:03.427-07:00 level=DEBUG source=gguf.go:57 msg="model = &llm.gguf{containerGGUF:(*llm.containerGGUF)(0x4000612000), kv:llm.KV{}, tensors:[]*llm.Te> May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.471-07:00 level=DEBUG source=sched.go:153 msg="loading first model" model=/usr/share/ollama/.ollama/models/blobs/sha256-4fed7364ee3e0c7cb4fe088> May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.471-07:00 level=DEBUG source=memory.go:44 msg=evaluating library=cuda gpu_count=1 available="5.4 GiB" May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.474-07:00 level=INFO source=memory.go:133 msg="offload to gpu" layers.requested=-1 layers.real=33 memory.available="5.4 GiB" memory.required.fu> May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.474-07:00 level=DEBUG source=sched.go:565 msg="new model will fit in available VRAM in single GPU, loading" model=/usr/share/ollama/.ollama/mod> May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.475-07:00 level=DEBUG source=memory.go:44 msg=evaluating library=cuda gpu_count=1 available="5.4 GiB" May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.477-07:00 level=INFO source=memory.go:133 msg="offload to gpu" layers.requested=-1 layers.real=33 memory.available="5.4 GiB" memory.required.fu> May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.477-07:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3415314780/runners/cpu May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.477-07:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3415314780/runners/cuda_v11 May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.477-07:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3415314780/runners/cpu May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.477-07:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3415314780/runners/cuda_v11 May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.477-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions" May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.478-07:00 level=INFO source=server.go:338 msg="starting llama server" cmd="/tmp/ollama3415314780/runners/cuda_v11/ollama_llama_server --model /> May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.478-07:00 level=DEBUG source=server.go:353 msg=subprocess environment="[PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/> May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.480-07:00 level=INFO source=sched.go:338 msg="loaded runners" count=1 May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.480-07:00 level=INFO source=server.go:526 msg="waiting for llama runner to start responding" May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.481-07:00 level=INFO source=server.go:564 msg="waiting for server to become available" status="llm server error" May 31 12:57:04 dev-desktop ollama[18601]: INFO [main] build info | build=1 commit="74f33ad" tid="281472439437568" timestamp=1717185424 May 31 12:57:04 dev-desktop ollama[18601]: INFO [main] system info | n_threads=6 n_threads_batch=-1 system_info="AVX = 0 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 => May 31 12:57:04 dev-desktop ollama[18601]: INFO [main] HTTP server listening | hostname="127.0.0.1" n_threads_http="5" port="32905" tid="281472439437568" timestamp=1717185424 May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: loaded meta data with 25 key-value pairs and 291 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-4fed7364ee3e0c7cb4fe0880148bfdfcd1b63098> May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv 0: general.architecture str = llama May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv 1: general.name str = LLaMA v2 May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv 2: llama.vocab_size u32 = 32064 May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv 3: llama.context_length u32 = 4096 May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv 4: llama.embedding_length u32 = 3072 May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv 5: llama.block_count u32 = 32 May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv 6: llama.feed_forward_length u32 = 8192 May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv 7: llama.rope.dimension_count u32 = 96 May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv 8: llama.attention.head_count u32 = 32 May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv 9: llama.attention.head_count_kv u32 = 32 May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv 10: llama.attention.layer_norm_rms_epsilon f32 = 0.000010 May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv 11: llama.rope.freq_base f32 = 10000.000000 May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv 12: general.file_type u32 = 15 May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv 13: tokenizer.ggml.model str = llama May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv 14: tokenizer.ggml.tokens arr[str,32064] = ["<unk>", "<s>", "</s>", "<0x00>", "<... May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv 15: tokenizer.ggml.scores arr[f32,32064] = [0.000000, 0.000000, 0.000000, 0.0000... May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv 16: tokenizer.ggml.token_type arr[i32,32064] = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ... May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv 17: tokenizer.ggml.bos_token_id u32 = 1 May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv 18: tokenizer.ggml.eos_token_id u32 = 32000 May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv 19: tokenizer.ggml.unknown_token_id u32 = 0 May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv 20: tokenizer.ggml.padding_token_id u32 = 32000 May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv 21: tokenizer.ggml.add_bos_token bool = true May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv 22: tokenizer.ggml.add_eos_token bool = false May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv 23: tokenizer.chat_template str = {{ bos_token }}{% for message in mess... May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - kv 24: general.quantization_version u32 = 2 May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - type f32: 65 tensors May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - type q4_K: 193 tensors May 31 12:57:04 dev-desktop ollama[18392]: llama_model_loader: - type q6_K: 33 tensors May 31 12:57:04 dev-desktop ollama[18392]: llm_load_vocab: special tokens definition check successful ( 323/32064 ). May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: format = GGUF V3 (latest) May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: arch = llama May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: vocab type = SPM May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_vocab = 32064 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_merges = 0 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_ctx_train = 4096 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_embd = 3072 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_head = 32 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_head_kv = 32 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_layer = 32 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_rot = 96 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_embd_head_k = 96 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_embd_head_v = 96 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_gqa = 1 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_embd_k_gqa = 3072 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_embd_v_gqa = 3072 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: f_norm_eps = 0.0e+00 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: f_norm_rms_eps = 1.0e-05 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: f_clamp_kqv = 0.0e+00 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: f_max_alibi_bias = 0.0e+00 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: f_logit_scale = 0.0e+00 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_ff = 8192 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_expert = 0 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_expert_used = 0 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: causal attn = 1 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: pooling type = 0 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: rope type = 0 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: rope scaling = linear May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: freq_base_train = 10000.0 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: freq_scale_train = 1 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: n_yarn_orig_ctx = 4096 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: rope_finetuned = unknown May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: ssm_d_conv = 0 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: ssm_d_inner = 0 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: ssm_d_state = 0 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: ssm_dt_rank = 0 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: model type = 7B May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: model ftype = Q4_K - Medium May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: model params = 3.82 B May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: model size = 2.16 GiB (4.85 BPW) May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: general.name = LLaMA v2 May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: BOS token = 1 '<s>' May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: EOS token = 32000 '<|endoftext|>' May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: UNK token = 0 '<unk>' May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: PAD token = 32000 '<|endoftext|>' May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: LF token = 13 '<0x0A>' May 31 12:57:04 dev-desktop ollama[18392]: llm_load_print_meta: EOT token = 32007 '<|end|>' May 31 12:57:04 dev-desktop ollama[18392]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ: yes May 31 12:57:04 dev-desktop ollama[18392]: ggml_cuda_init: CUDA_USE_TENSOR_CORES: no May 31 12:57:04 dev-desktop ollama[18392]: ggml_cuda_init: found 1 CUDA devices: May 31 12:57:04 dev-desktop ollama[18392]: Device 0: Xavier, compute capability 7.2, VMM: yes May 31 12:57:04 dev-desktop ollama[18392]: time=2024-05-31T12:57:04.734-07:00 level=INFO source=server.go:564 msg="waiting for server to become available" status="llm server loading model" May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.390-07:00 level=WARN source=server.go:533 msg="client connection closed before server finished loading, aborting load" May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.391-07:00 level=ERROR source=sched.go:344 msg="error loading llama server" error="timed out waiting for llama runner to start: context canceled" May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.391-07:00 level=DEBUG source=sched.go:347 msg="triggering expiration for failed load" model=/usr/share/ollama/.ollama/models/blobs/sha256-4fed7> May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.391-07:00 level=DEBUG source=sched.go:258 msg="runner expired event received" modelPath=/usr/share/ollama/.ollama/models/blobs/sha256-4fed7364e> May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.391-07:00 level=DEBUG source=sched.go:274 msg="got lock to unload" modelPath=/usr/share/ollama/.ollama/models/blobs/sha256-4fed7364ee3e0c7cb4fe> May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.391-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs" May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.391-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so* May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.391-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li> May 31 12:57:23 dev-desktop ollama[18392]: [GIN] 2024/05/31 - 12:57:23 | 499 | 20.111113587s | 127.0.0.1 | POST "/api/chat" May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.408-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1] May 31 12:57:23 dev-desktop ollama[18392]: CUDA driver version: 11.4 May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.409-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1 May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.409-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions" May 31 12:57:23 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb May 31 12:57:23 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5005 mb May 31 12:57:23 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2 May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.531-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu" May 31 12:57:23 dev-desktop ollama[18392]: releasing nvcuda library May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.531-07:00 level=DEBUG source=server.go:987 msg="stopping llama server" May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.532-07:00 level=DEBUG source=server.go:993 msg="waiting for llama server to exit" May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.739-07:00 level=DEBUG source=server.go:997 msg="llama server stopped" May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.740-07:00 level=DEBUG source=sched.go:279 msg="runner released" modelPath=/usr/share/ollama/.ollama/models/blobs/sha256-4fed7364ee3e0c7cb4fe088> May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.783-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs" May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.784-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so* May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.784-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li> May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.811-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1] May 31 12:57:23 dev-desktop ollama[18392]: CUDA driver version: 11.4 May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.812-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1 May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.812-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions" May 31 12:57:23 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb May 31 12:57:23 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5554 mb May 31 12:57:23 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2 May 31 12:57:23 dev-desktop ollama[18392]: time=2024-05-31T12:57:23.917-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu" May 31 12:57:23 dev-desktop ollama[18392]: releasing nvcuda library May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.032-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs" May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.033-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so* May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.033-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li> May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.069-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1] May 31 12:57:24 dev-desktop ollama[18392]: CUDA driver version: 11.4 May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.070-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1 May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.071-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions" May 31 12:57:24 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb May 31 12:57:24 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5554 mb May 31 12:57:24 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2 May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.173-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu" May 31 12:57:24 dev-desktop ollama[18392]: releasing nvcuda library May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.283-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs" May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.283-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so* May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.283-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li> May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.303-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1] May 31 12:57:24 dev-desktop ollama[18392]: CUDA driver version: 11.4 May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.303-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1 May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.303-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions" May 31 12:57:24 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb May 31 12:57:24 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5554 mb May 31 12:57:24 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2 May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.414-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu" May 31 12:57:24 dev-desktop ollama[18392]: releasing nvcuda library May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.533-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs" May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.533-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so* May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.534-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li> May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.553-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1] May 31 12:57:24 dev-desktop ollama[18392]: CUDA driver version: 11.4 May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.554-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1 May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.554-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions" May 31 12:57:24 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb May 31 12:57:24 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5555 mb May 31 12:57:24 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2 May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.655-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu" May 31 12:57:24 dev-desktop ollama[18392]: releasing nvcuda library May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.782-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs" May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.782-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so* May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.782-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li> May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.801-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1] May 31 12:57:24 dev-desktop ollama[18392]: CUDA driver version: 11.4 May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.801-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1 May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.801-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions" May 31 12:57:24 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb May 31 12:57:24 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5555 mb May 31 12:57:24 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2 May 31 12:57:24 dev-desktop ollama[18392]: time=2024-05-31T12:57:24.904-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu" May 31 12:57:24 dev-desktop ollama[18392]: releasing nvcuda library May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.032-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs" May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.032-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so* May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.032-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li> May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.051-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1] May 31 12:57:25 dev-desktop ollama[18392]: CUDA driver version: 11.4 May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.051-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1 May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.051-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions" May 31 12:57:25 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb May 31 12:57:25 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5554 mb May 31 12:57:25 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2 May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.154-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu" May 31 12:57:25 dev-desktop ollama[18392]: releasing nvcuda library May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.283-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs" May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.283-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so* May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.283-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li> May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.304-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1] May 31 12:57:25 dev-desktop ollama[18392]: CUDA driver version: 11.4 May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.304-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1 May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.305-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions" May 31 12:57:25 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb May 31 12:57:25 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5555 mb May 31 12:57:25 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2 May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.407-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu" May 31 12:57:25 dev-desktop ollama[18392]: releasing nvcuda library May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.532-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs" May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.533-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so* May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.533-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li> May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.551-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1] May 31 12:57:25 dev-desktop ollama[18392]: CUDA driver version: 11.4 May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.551-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1 May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.551-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions" May 31 12:57:25 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb May 31 12:57:25 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5555 mb May 31 12:57:25 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2 May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.653-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu" May 31 12:57:25 dev-desktop ollama[18392]: releasing nvcuda library May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.782-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs" May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.782-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so* May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.783-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li> May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.802-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1] May 31 12:57:25 dev-desktop ollama[18392]: CUDA driver version: 11.4 May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.803-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1 May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.803-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions" May 31 12:57:25 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb May 31 12:57:25 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5555 mb May 31 12:57:25 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2 May 31 12:57:25 dev-desktop ollama[18392]: time=2024-05-31T12:57:25.903-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu" May 31 12:57:25 dev-desktop ollama[18392]: releasing nvcuda library May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.032-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs" May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.032-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so* May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.032-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li> May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.050-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1] May 31 12:57:26 dev-desktop ollama[18392]: CUDA driver version: 11.4 May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.051-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1 May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.051-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions" May 31 12:57:26 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb May 31 12:57:26 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5555 mb May 31 12:57:26 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2 May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.152-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu" May 31 12:57:26 dev-desktop ollama[18392]: releasing nvcuda library May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.283-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs" May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.283-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so* May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.283-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li> May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.301-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1] May 31 12:57:26 dev-desktop ollama[18392]: CUDA driver version: 11.4 May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.301-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1 May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.301-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions" May 31 12:57:26 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb May 31 12:57:26 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5554 mb May 31 12:57:26 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2 May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.405-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu" May 31 12:57:26 dev-desktop ollama[18392]: releasing nvcuda library May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.532-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs" May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.533-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so* May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.533-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li> May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.552-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1] May 31 12:57:26 dev-desktop ollama[18392]: CUDA driver version: 11.4 May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.553-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1 May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.553-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions" May 31 12:57:26 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb May 31 12:57:26 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5555 mb May 31 12:57:26 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2 May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.655-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu" May 31 12:57:26 dev-desktop ollama[18392]: releasing nvcuda library May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.782-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs" May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.782-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so* May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.782-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li> May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.801-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1] May 31 12:57:26 dev-desktop ollama[18392]: CUDA driver version: 11.4 May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.801-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1 May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.801-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions" May 31 12:57:26 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb May 31 12:57:26 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5555 mb May 31 12:57:26 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2 May 31 12:57:26 dev-desktop ollama[18392]: time=2024-05-31T12:57:26.905-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu" May 31 12:57:26 dev-desktop ollama[18392]: releasing nvcuda library May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.033-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs" May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.033-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so* May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.033-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li> May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.050-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1] May 31 12:57:27 dev-desktop ollama[18392]: CUDA driver version: 11.4 May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.051-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1 May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.051-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions" May 31 12:57:27 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb May 31 12:57:27 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5556 mb May 31 12:57:27 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2 May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.151-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu" May 31 12:57:27 dev-desktop ollama[18392]: releasing nvcuda library May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.283-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs" May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.283-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so* May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.283-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li> May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.301-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1] May 31 12:57:27 dev-desktop ollama[18392]: CUDA driver version: 11.4 May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.302-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1 May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.302-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions" May 31 12:57:27 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb May 31 12:57:27 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5556 mb May 31 12:57:27 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2 May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.404-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu" May 31 12:57:27 dev-desktop ollama[18392]: releasing nvcuda library May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.532-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs" May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.532-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so* May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.533-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li> May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.551-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1] May 31 12:57:27 dev-desktop ollama[18392]: CUDA driver version: 11.4 May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.552-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1 May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.552-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions" May 31 12:57:27 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb May 31 12:57:27 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5557 mb May 31 12:57:27 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2 May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.651-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu" May 31 12:57:27 dev-desktop ollama[18392]: releasing nvcuda library May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.782-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs" May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.782-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so* May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.782-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li> May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.802-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1] May 31 12:57:27 dev-desktop ollama[18392]: CUDA driver version: 11.4 May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.802-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1 May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.802-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions" May 31 12:57:27 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb May 31 12:57:27 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5557 mb May 31 12:57:27 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2 May 31 12:57:27 dev-desktop ollama[18392]: time=2024-05-31T12:57:27.904-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu" May 31 12:57:27 dev-desktop ollama[18392]: releasing nvcuda library May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.032-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs" May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.032-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so* May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.032-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li> May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.050-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1] May 31 12:57:28 dev-desktop ollama[18392]: CUDA driver version: 11.4 May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.051-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1 May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.051-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions" May 31 12:57:28 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb May 31 12:57:28 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5552 mb May 31 12:57:28 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2 May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.176-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu" May 31 12:57:28 dev-desktop ollama[18392]: releasing nvcuda library May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.282-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs" May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.283-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so* May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.283-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li> May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.304-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1] May 31 12:57:28 dev-desktop ollama[18392]: CUDA driver version: 11.4 May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.305-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1 May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.305-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions" May 31 12:57:28 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb May 31 12:57:28 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5552 mb May 31 12:57:28 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2 May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.406-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu" May 31 12:57:28 dev-desktop ollama[18392]: releasing nvcuda library May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.532-07:00 level=WARN source=sched.go:512 msg="gpu VRAM usage didn't recover within timeout" seconds=5.141294161 May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.533-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs" May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.533-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so* May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.533-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li> May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.536-07:00 level=DEBUG source=sched.go:283 msg="sending an unloaded event" modelPath=/usr/share/ollama/.ollama/models/blobs/sha256-4fed7364ee3e0> May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.537-07:00 level=DEBUG source=sched.go:206 msg="ignoring unload event with no pending requests" May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.551-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1] May 31 12:57:28 dev-desktop ollama[18392]: CUDA driver version: 11.4 May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.551-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1 May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.551-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions" May 31 12:57:28 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb May 31 12:57:28 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5550 mb May 31 12:57:28 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2 May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.661-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu" May 31 12:57:28 dev-desktop ollama[18392]: releasing nvcuda library May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.782-07:00 level=WARN source=sched.go:512 msg="gpu VRAM usage didn't recover within timeout" seconds=5.391041361 May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.782-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs" May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.782-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so* May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.782-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li> May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.800-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1] May 31 12:57:28 dev-desktop ollama[18392]: CUDA driver version: 11.4 May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.800-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1 May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.800-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions" May 31 12:57:28 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb May 31 12:57:28 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5550 mb May 31 12:57:28 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2 May 31 12:57:28 dev-desktop ollama[18392]: time=2024-05-31T12:57:28.902-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu" May 31 12:57:28 dev-desktop ollama[18392]: releasing nvcuda library May 31 12:57:29 dev-desktop ollama[18392]: time=2024-05-31T12:57:29.032-07:00 level=WARN source=sched.go:512 msg="gpu VRAM usage didn't recover within timeout" seconds=5.640970132 May 31 12:57:31 dev-desktop ollama[18392]: [GIN] 2024/05/31 - 12:57:31 | 200 | 160.675µs | 127.0.0.1 | HEAD "/" May 31 12:57:31 dev-desktop ollama[18392]: [GIN] 2024/05/31 - 12:57:31 | 200 | 2.168833ms | 127.0.0.1 | POST "/api/show" May 31 12:57:31 dev-desktop ollama[18392]: [GIN] 2024/05/31 - 12:57:31 | 200 | 2.07632ms | 127.0.0.1 | POST "/api/show" May 31 12:57:31 dev-desktop ollama[18392]: time=2024-05-31T12:57:31.351-07:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs" May 31 12:57:31 dev-desktop ollama[18392]: time=2024-05-31T12:57:31.351-07:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so* May 31 12:57:31 dev-desktop ollama[18392]: time=2024-05-31T12:57:31.351-07:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-li> May 31 12:57:31 dev-desktop ollama[18392]: time=2024-05-31T12:57:31.369-07:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1] May 31 12:57:31 dev-desktop ollama[18392]: CUDA driver version: 11.4 May 31 12:57:31 dev-desktop ollama[18392]: time=2024-05-31T12:57:31.370-07:00 level=DEBUG source=gpu.go:127 msg="detected GPUs" count=1 library=/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1.1 May 31 12:57:31 dev-desktop ollama[18392]: time=2024-05-31T12:57:31.370-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions" May 31 12:57:31 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA totalMem 6857 mb May 31 12:57:31 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] CUDA freeMem 5540 mb May 31 12:57:31 dev-desktop ollama[18392]: [GPU-d1bf94b9-edc1-55a1-80b1-a3de76dd41c8] Compute Capability 7.2 May 31 12:57:31 dev-desktop ollama[18392]: time=2024-05-31T12:57:31.473-07:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu" May 31 12:57:31 dev-desktop ollama[18392]: releasing nvcuda library May 31 12:57:31 dev-desktop ollama[18392]: time=2024-05-31T12:57:31.474-07:00 level=DEBUG source=gguf.go:57 msg="model = &llm.gguf{containerGGUF:(*llm.containerGGUF)(0x400049e000), kv:llm.KV{}, tensors:[]*llm.Te> May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.449-07:00 level=DEBUG source=sched.go:153 msg="loading first model" model=/usr/share/ollama/.ollama/models/blobs/sha256-4fed7364ee3e0c7cb4fe088> May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.450-07:00 level=DEBUG source=memory.go:44 msg=evaluating library=cuda gpu_count=1 available="5.4 GiB" May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.453-07:00 level=INFO source=memory.go:133 msg="offload to gpu" layers.requested=-1 layers.real=33 memory.available="5.4 GiB" memory.required.fu> May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.453-07:00 level=DEBUG source=sched.go:565 msg="new model will fit in available VRAM in single GPU, loading" model=/usr/share/ollama/.ollama/mod> May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.454-07:00 level=DEBUG source=memory.go:44 msg=evaluating library=cuda gpu_count=1 available="5.4 GiB" May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.456-07:00 level=INFO source=memory.go:133 msg="offload to gpu" layers.requested=-1 layers.real=33 memory.available="5.4 GiB" memory.required.fu> May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.456-07:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3415314780/runners/cpu May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.456-07:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3415314780/runners/cuda_v11 May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.457-07:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3415314780/runners/cpu May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.457-07:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3415314780/runners/cuda_v11 May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.457-07:00 level=DEBUG source=cpu_common.go:18 msg="CPU does not have vector extensions" May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.457-07:00 level=INFO source=server.go:338 msg="starting llama server" cmd="/tmp/ollama3415314780/runners/cuda_v11/ollama_llama_server --model /> May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.457-07:00 level=DEBUG source=server.go:353 msg=subprocess environment="[PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/> May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.459-07:00 level=INFO source=sched.go:338 msg="loaded runners" count=1 May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.459-07:00 level=INFO source=server.go:526 msg="waiting for llama runner to start responding" May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.460-07:00 level=INFO source=server.go:564 msg="waiting for server to become available" status="llm server error" May 31 12:57:32 dev-desktop ollama[18643]: INFO [main] build info | build=1 commit="74f33ad" tid="281472994298112" timestamp=1717185452 May 31 12:57:32 dev-desktop ollama[18643]: INFO [main] system info | n_threads=6 n_threads_batch=-1 system_info="AVX = 0 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 => May 31 12:57:32 dev-desktop ollama[18643]: INFO [main] HTTP server listening | hostname="127.0.0.1" n_threads_http="5" port="42441" tid="281472994298112" timestamp=1717185452 May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: loaded meta data with 25 key-value pairs and 291 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-4fed7364ee3e0c7cb4fe0880148bfdfcd1b63098> May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv 0: general.architecture str = llama May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv 1: general.name str = LLaMA v2 May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv 2: llama.vocab_size u32 = 32064 May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv 3: llama.context_length u32 = 4096 May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv 4: llama.embedding_length u32 = 3072 May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv 5: llama.block_count u32 = 32 May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv 6: llama.feed_forward_length u32 = 8192 May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv 7: llama.rope.dimension_count u32 = 96 May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv 8: llama.attention.head_count u32 = 32 May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv 9: llama.attention.head_count_kv u32 = 32 May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv 10: llama.attention.layer_norm_rms_epsilon f32 = 0.000010 May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv 11: llama.rope.freq_base f32 = 10000.000000 May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv 12: general.file_type u32 = 15 May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv 13: tokenizer.ggml.model str = llama May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv 14: tokenizer.ggml.tokens arr[str,32064] = ["<unk>", "<s>", "</s>", "<0x00>", "<... May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv 15: tokenizer.ggml.scores arr[f32,32064] = [0.000000, 0.000000, 0.000000, 0.0000... May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv 16: tokenizer.ggml.token_type arr[i32,32064] = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ... May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv 17: tokenizer.ggml.bos_token_id u32 = 1 May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv 18: tokenizer.ggml.eos_token_id u32 = 32000 May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv 19: tokenizer.ggml.unknown_token_id u32 = 0 May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv 20: tokenizer.ggml.padding_token_id u32 = 32000 May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv 21: tokenizer.ggml.add_bos_token bool = true May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv 22: tokenizer.ggml.add_eos_token bool = false May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv 23: tokenizer.chat_template str = {{ bos_token }}{% for message in mess... May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - kv 24: general.quantization_version u32 = 2 May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - type f32: 65 tensors May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - type q4_K: 193 tensors May 31 12:57:32 dev-desktop ollama[18392]: llama_model_loader: - type q6_K: 33 tensors May 31 12:57:32 dev-desktop ollama[18392]: llm_load_vocab: special tokens definition check successful ( 323/32064 ). May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: format = GGUF V3 (latest) May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: arch = llama May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: vocab type = SPM May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_vocab = 32064 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_merges = 0 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_ctx_train = 4096 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_embd = 3072 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_head = 32 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_head_kv = 32 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_layer = 32 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_rot = 96 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_embd_head_k = 96 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_embd_head_v = 96 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_gqa = 1 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_embd_k_gqa = 3072 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_embd_v_gqa = 3072 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: f_norm_eps = 0.0e+00 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: f_norm_rms_eps = 1.0e-05 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: f_clamp_kqv = 0.0e+00 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: f_max_alibi_bias = 0.0e+00 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: f_logit_scale = 0.0e+00 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_ff = 8192 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_expert = 0 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_expert_used = 0 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: causal attn = 1 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: pooling type = 0 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: rope type = 0 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: rope scaling = linear May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: freq_base_train = 10000.0 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: freq_scale_train = 1 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: n_yarn_orig_ctx = 4096 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: rope_finetuned = unknown May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: ssm_d_conv = 0 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: ssm_d_inner = 0 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: ssm_d_state = 0 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: ssm_dt_rank = 0 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: model type = 7B May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: model ftype = Q4_K - Medium May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: model params = 3.82 B May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: model size = 2.16 GiB (4.85 BPW) May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: general.name = LLaMA v2 May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: BOS token = 1 '<s>' May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: EOS token = 32000 '<|endoftext|>' May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: UNK token = 0 '<unk>' May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: PAD token = 32000 '<|endoftext|>' May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: LF token = 13 '<0x0A>' May 31 12:57:32 dev-desktop ollama[18392]: llm_load_print_meta: EOT token = 32007 '<|end|>' May 31 12:57:32 dev-desktop ollama[18392]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ: yes May 31 12:57:32 dev-desktop ollama[18392]: ggml_cuda_init: CUDA_USE_TENSOR_CORES: no May 31 12:57:32 dev-desktop ollama[18392]: ggml_cuda_init: found 1 CUDA devices: May 31 12:57:32 dev-desktop ollama[18392]: Device 0: Xavier, compute capability 7.2, VMM: yes May 31 12:57:32 dev-desktop ollama[18392]: time=2024-05-31T12:57:32.713-07:00 level=INFO source=server.go:564 msg="waiting for server to become available" status="llm server loading model" lines 1301- ```
GiteaMirror commented 2026-04-22 07:14:09 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (May 31, 2024):

It looks like it's hanging in the cuda code, likely the result of our bundled cuda library not being compatible. If you can build from source it should work. Ultimately our goal is to try to bundle a cuda library that does work for these systems.

<!-- gh-comment-id:2142918924 --> @dhiltgen commented on GitHub (May 31, 2024): It looks like it's hanging in the cuda code, likely the result of our bundled cuda library not being compatible. If you can build from source it should work. Ultimately our goal is to try to bundle a cuda library that does work for these systems.
GiteaMirror commented 2026-04-22 07:14:09 -05:00
Author
Owner
Copy Link

@ZanMax commented on GitHub (May 31, 2024):

Is there a significant difference between versions 0.1.38 and 0.1.39?
Version 0.1.38 works great for me.

<!-- gh-comment-id:2142923226 --> @ZanMax commented on GitHub (May 31, 2024): Is there a significant difference between versions 0.1.38 and 0.1.39? Version 0.1.38 works great for me.
GiteaMirror commented 2026-04-22 07:14:09 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (May 31, 2024):

@ZanMax we bump llama.cpp in ~every new release, which has various bug fixes and features added that impact how the cuda libraries are called. Our current understanding is the binary cuda library we're bundling isn't compatible with JetPacks, and the failure modes are inconsistent, but often manifest as hangs during load. Building from source will get the right cuda library for your JetPack until we sort out the build rigging to bundle all the permutations into the binary release.

<!-- gh-comment-id:2143035706 --> @dhiltgen commented on GitHub (May 31, 2024): @ZanMax we bump llama.cpp in ~every new release, which has various bug fixes and features added that impact how the cuda libraries are called. Our current understanding is the binary cuda library we're bundling isn't compatible with JetPacks, and the failure modes are inconsistent, but often manifest as hangs during load. Building from source will get the right cuda library for your JetPack until we sort out the build rigging to bundle all the permutations into the binary release.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug nvidia
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#28714
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?