error loading model: llama.cpp: tensor 'layers.2.ffn_norm.weight' is missing from model #200

New Issue

2025-11-11T14:11:10-06:00

GiteaMirror commented

2025-11-11 14:11:10 -06:00

Originally created by @lpf763827726 on GitHub (Jan 18, 2024).

Bug Report

Description

Bug Summary:
docker always restarting

Steps to Reproduce:
sry, i dont know how to reproduce it, when i run ollama run llama2:13b and ollama run codellama download this two model, and use codellama ask 2~3 questions, it beginning restart it

Environment

Operating System: [e.g., Windows 10, macOS Big Sur, Ubuntu 20.04]
Ubuntu 23.04 (GNU/Linux 6.2.0-39-generic x86_64)
Browser (if applicable): [e.g., Chrome 100.0, Firefox 98.0]
Microsoft Edge 120.0.2210.133 x64

Reproduction Details

Confirmation:

[√] I have read and followed all the instructions provided in the README.md.
I have reviewed the troubleshooting.md document.
I have included the browser console logs.
[√] I have included the Docker container logs.

Logs and Screenshots

Browser Console Logs:
[Include relevant browser console logs, if applicable]

Docker Container Logs:

/models/llama-2-13b-chat.bin model found.
python3 setup.py develop
/usr/local/lib/python3.11/site-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` and ``easy_install``.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://github.com/pypa/setuptools/issues/917 for details.
        ********************************************************************************

!!
  easy_install.initialize_options(self)
[0/1] Install the project...
-- Install configuration: "Release"
-- Up-to-date: /app/_skbuild/linux-x86_64-3.11/cmake-install/llama_cpp/libllama.so
copying _skbuild/linux-x86_64-3.11/cmake-install/llama_cpp/libllama.so -> llama_cpp/libllama.so

running develop
/usr/local/lib/python3.11/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` directly.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
        ********************************************************************************

!!
  self.initialize_options()
running egg_info
writing llama_cpp_python.egg-info/PKG-INFO
writing dependency_links to llama_cpp_python.egg-info/dependency_links.txt
writing requirements to llama_cpp_python.egg-info/requires.txt
writing top-level names to llama_cpp_python.egg-info/top_level.txt
reading manifest file 'llama_cpp_python.egg-info/SOURCES.txt'
adding license file 'LICENSE.md'
writing manifest file 'llama_cpp_python.egg-info/SOURCES.txt'
running build_ext
Creating /usr/local/lib/python3.11/site-packages/llama-cpp-python.egg-link (link to .)
llama-cpp-python 0.1.78 is already the active version in easy-install.pth

Installed /app
Processing dependencies for llama-cpp-python==0.1.78
Searching for diskcache==5.6.1
Best match: diskcache 5.6.1
Processing diskcache-5.6.1-py3.11.egg
Adding diskcache 5.6.1 to easy-install.pth file

Using /usr/local/lib/python3.11/site-packages/diskcache-5.6.1-py3.11.egg
Searching for numpy==1.26.0b1
Best match: numpy 1.26.0b1
Processing numpy-1.26.0b1-py3.11-linux-x86_64.egg
Adding numpy 1.26.0b1 to easy-install.pth file
Installing f2py script to /usr/local/bin

Using /usr/local/lib/python3.11/site-packages/numpy-1.26.0b1-py3.11-linux-x86_64.egg
Searching for typing-extensions==4.7.1
Best match: typing-extensions 4.7.1
Adding typing-extensions 4.7.1 to easy-install.pth file

Using /usr/local/lib/python3.11/site-packages
Finished processing dependencies for llama-cpp-python==0.1.78
Initializing server with:
Batch size: 2096
Number of CPU threads: 24
Number of GPU layers: 0
Context window: 4096
/usr/local/lib/python3.11/site-packages/pydantic/_internal/_fields.py:127: UserWarning: Field "model_alias" has conflict with protected namespace "model_".

You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ('settings_',)`.
  warnings.warn(
llama.cpp: loading model from /models/llama-2-13b-chat.bin
llama_model_load_internal: format     = ggjt v3 (latest)
llama_model_load_internal: n_vocab    = 32032
llama_model_load_internal: n_ctx      = 4096
llama_model_load_internal: n_embd     = 5120
llama_model_load_internal: n_mult     = 256
llama_model_load_internal: n_head     = 40
llama_model_load_internal: n_head_kv  = 40
llama_model_load_internal: n_layer    = 40
llama_model_load_internal: n_rot      = 128
llama_model_load_internal: n_gqa      = 1
llama_model_load_internal: rnorm_eps  = 5.0e-06
llama_model_load_internal: n_ff       = 13824
llama_model_load_internal: freq_base  = 10000.0
llama_model_load_internal: freq_scale = 1
llama_model_load_internal: ftype      = 2 (mostly Q4_0)
llama_model_load_internal: model size = 13B
llama_model_load_internal: ggml ctx size =    0.01 MB
error loading model: llama.cpp: tensor 'layers.2.ffn_norm.weight' is missing from model
llama_load_model_from_file: failed to load model
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/app/llama_cpp/server/__main__.py", line 46, in <module>
    app = create_app(settings=settings)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/llama_cpp/server/app.py", line 317, in create_app
    llama = llama_cpp.Llama(
            ^^^^^^^^^^^^^^^^
  File "/app/llama_cpp/llama.py", line 328, in __init__
    assert self.model is not None
           ^^^^^^^^^^^^^^^^^^^^^^
AssertionError
/models/llama-2-13b-chat.bin model found.
python3 setup.py develop
/usr/local/lib/python3.11/site-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` and ``easy_install``.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://github.com/pypa/setuptools/issues/917 for details.
        ********************************************************************************

!!
  easy_install.initialize_options(self)
[0/1] Install the project...
-- Install configuration: "Release"
-- Up-to-date: /app/_skbuild/linux-x86_64-3.11/cmake-install/llama_cpp/libllama.so
copying _skbuild/linux-x86_64-3.11/cmake-install/llama_cpp/libllama.so -> llama_cpp/libllama.so

running develop
/usr/local/lib/python3.11/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` directly.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
        ********************************************************************************

!!
  self.initialize_options()
running egg_info
writing llama_cpp_python.egg-info/PKG-INFO
writing dependency_links to llama_cpp_python.egg-info/dependency_links.txt
writing requirements to llama_cpp_python.egg-info/requires.txt
writing top-level names to llama_cpp_python.egg-info/top_level.txt
reading manifest file 'llama_cpp_python.egg-info/SOURCES.txt'
adding license file 'LICENSE.md'
writing manifest file 'llama_cpp_python.egg-info/SOURCES.txt'
running build_ext
Creating /usr/local/lib/python3.11/site-packages/llama-cpp-python.egg-link (link to .)
llama-cpp-python 0.1.78 is already the active version in easy-install.pth

Installed /app
Processing dependencies for llama-cpp-python==0.1.78
Searching for diskcache==5.6.1
Best match: diskcache 5.6.1
Processing diskcache-5.6.1-py3.11.egg
Adding diskcache 5.6.1 to easy-install.pth file

Using /usr/local/lib/python3.11/site-packages/diskcache-5.6.1-py3.11.egg
Searching for numpy==1.26.0b1
Best match: numpy 1.26.0b1
Processing numpy-1.26.0b1-py3.11-linux-x86_64.egg
Adding numpy 1.26.0b1 to easy-install.pth file
Installing f2py script to /usr/local/bin

Using /usr/local/lib/python3.11/site-packages/numpy-1.26.0b1-py3.11-linux-x86_64.egg
Searching for typing-extensions==4.7.1
Best match: typing-extensions 4.7.1
Adding typing-extensions 4.7.1 to easy-install.pth file

Using /usr/local/lib/python3.11/site-packages
Finished processing dependencies for llama-cpp-python==0.1.78
Initializing server with:
Batch size: 2096
Number of CPU threads: 24
Number of GPU layers: 0
Context window: 4096
/usr/local/lib/python3.11/site-packages/pydantic/_internal/_fields.py:127: UserWarning: Field "model_alias" has conflict with protected namespace "model_".

You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ('settings_',)`.
  warnings.warn(
llama.cpp: loading model from /models/llama-2-13b-chat.bin
llama_model_load_internal: format     = ggjt v3 (latest)
llama_model_load_internal: n_vocab    = 32032
llama_model_load_internal: n_ctx      = 4096
llama_model_load_internal: n_embd     = 5120
llama_model_load_internal: n_mult     = 256
llama_model_load_internal: n_head     = 40
llama_model_load_internal: n_head_kv  = 40
llama_model_load_internal: n_layer    = 40
llama_model_load_internal: n_rot      = 128
llama_model_load_internal: n_gqa      = 1
llama_model_load_internal: rnorm_eps  = 5.0e-06
llama_model_load_internal: n_ff       = 13824
llama_model_load_internal: freq_base  = 10000.0
llama_model_load_internal: freq_scale = 1
llama_model_load_internal: ftype      = 2 (mostly Q4_0)
llama_model_load_internal: model size = 13B
llama_model_load_internal: ggml ctx size =    0.01 MB
error loading model: llama.cpp: tensor 'layers.2.ffn_norm.weight' is missing from model
llama_load_model_from_file: failed to load model
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/app/llama_cpp/server/__main__.py", line 46, in <module>
    app = create_app(settings=settings)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/llama_cpp/server/app.py", line 317, in create_app
    llama = llama_cpp.Llama(
            ^^^^^^^^^^^^^^^^
  File "/app/llama_cpp/llama.py", line 328, in __init__
    assert self.model is not None
           ^^^^^^^^^^^^^^^^^^^^^^
AssertionError
/models/llama-2-13b-chat.bin model found.
python3 setup.py develop
/usr/local/lib/python3.11/site-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` and ``easy_install``.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://github.com/pypa/setuptools/issues/917 for details.
        ********************************************************************************

!!
  easy_install.initialize_options(self)
[0/1] Install the project...
-- Install configuration: "Release"
-- Up-to-date: /app/_skbuild/linux-x86_64-3.11/cmake-install/llama_cpp/libllama.so
copying _skbuild/linux-x86_64-3.11/cmake-install/llama_cpp/libllama.so -> llama_cpp/libllama.so

running develop
/usr/local/lib/python3.11/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` directly.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
        ********************************************************************************

!!
  self.initialize_options()
running egg_info
writing llama_cpp_python.egg-info/PKG-INFO
writing dependency_links to llama_cpp_python.egg-info/dependency_links.txt
writing requirements to llama_cpp_python.egg-info/requires.txt
writing top-level names to llama_cpp_python.egg-info/top_level.txt
reading manifest file 'llama_cpp_python.egg-info/SOURCES.txt'
adding license file 'LICENSE.md'
writing manifest file 'llama_cpp_python.egg-info/SOURCES.txt'
running build_ext
Creating /usr/local/lib/python3.11/site-packages/llama-cpp-python.egg-link (link to .)
llama-cpp-python 0.1.78 is already the active version in easy-install.pth

Installed /app
Processing dependencies for llama-cpp-python==0.1.78
Searching for diskcache==5.6.1
Best match: diskcache 5.6.1
Processing diskcache-5.6.1-py3.11.egg
Adding diskcache 5.6.1 to easy-install.pth file

Using /usr/local/lib/python3.11/site-packages/diskcache-5.6.1-py3.11.egg
Searching for numpy==1.26.0b1
Best match: numpy 1.26.0b1
Processing numpy-1.26.0b1-py3.11-linux-x86_64.egg
Adding numpy 1.26.0b1 to easy-install.pth file
Installing f2py script to /usr/local/bin

Using /usr/local/lib/python3.11/site-packages/numpy-1.26.0b1-py3.11-linux-x86_64.egg
Searching for typing-extensions==4.7.1
Best match: typing-extensions 4.7.1
Adding typing-extensions 4.7.1 to easy-install.pth file

Using /usr/local/lib/python3.11/site-packages
Finished processing dependencies for llama-cpp-python==0.1.78
Initializing server with:
Batch size: 2096
Number of CPU threads: 24
Number of GPU layers: 0
Context window: 4096
/usr/local/lib/python3.11/site-packages/pydantic/_internal/_fields.py:127: UserWarning: Field "model_alias" has conflict with protected namespace "model_".

You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ('settings_',)`.
  warnings.warn(
llama.cpp: loading model from /models/llama-2-13b-chat.bin
llama_model_load_internal: format     = ggjt v3 (latest)
llama_model_load_internal: n_vocab    = 32032
llama_model_load_internal: n_ctx      = 4096
llama_model_load_internal: n_embd     = 5120
llama_model_load_internal: n_mult     = 256
llama_model_load_internal: n_head     = 40
llama_model_load_internal: n_head_kv  = 40
llama_model_load_internal: n_layer    = 40
llama_model_load_internal: n_rot      = 128
llama_model_load_internal: n_gqa      = 1
llama_model_load_internal: rnorm_eps  = 5.0e-06
llama_model_load_internal: n_ff       = 13824
llama_model_load_internal: freq_base  = 10000.0
llama_model_load_internal: freq_scale = 1
llama_model_load_internal: ftype      = 2 (mostly Q4_0)
llama_model_load_internal: model size = 13B
llama_model_load_internal: ggml ctx size =    0.01 MB
error loading model: llama.cpp: tensor 'layers.2.ffn_norm.weight' is missing from model
llama_load_model_from_file: failed to load model
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/app/llama_cpp/server/__main__.py", line 46, in <module>
    app = create_app(settings=settings)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/llama_cpp/server/app.py", line 317, in create_app
    llama = llama_cpp.Llama(
            ^^^^^^^^^^^^^^^^
  File "/app/llama_cpp/llama.py", line 328, in __init__
    assert self.model is not None
           ^^^^^^^^^^^^^^^^^^^^^^
AssertionError

Screenshots (if applicable):
[Attach any relevant screenshots to help illustrate the issue]

Installation Method

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v ollama-webui:/app/backend/data --name ollama-webui --restart always ghcr.io/ollama-webui/ollama-webui:main

Additional Information

[Include any additional details that may help in understanding and reproducing the issue. This could include specific configurations, error messages, or anything else relevant to the bug.]

Note

If the bug report is incomplete or does not follow the provided instructions, it may not be addressed. Please ensure that you have followed the steps outlined in the README.md and troubleshooting.md documents, and provide all necessary information for us to reproduce and address the issue. Thank you!

Originally created by @lpf763827726 on GitHub (Jan 18, 2024). # Bug Report ## Description **Bug Summary:** docker always restarting **Steps to Reproduce:** sry, i dont know how to reproduce it, when i run `ollama run llama2:13b` and `ollama run codellama` download this two model, and use codellama ask 2~3 questions, it beginning restart it ## Environment - **Operating System:** [e.g., Windows 10, macOS Big Sur, Ubuntu 20.04] Ubuntu 23.04 (GNU/Linux 6.2.0-39-generic x86_64) - **Browser (if applicable):** [e.g., Chrome 100.0, Firefox 98.0] Microsoft Edge 120.0.2210.133 x64 ## Reproduction Details **Confirmation:** - [√] I have read and followed all the instructions provided in the README.md. - [ ] I have reviewed the troubleshooting.md document. - [ ] I have included the browser console logs. - [√] I have included the Docker container logs. ## Logs and Screenshots **Browser Console Logs:** [Include relevant browser console logs, if applicable] **Docker Container Logs:** ``` /models/llama-2-13b-chat.bin model found. python3 setup.py develop /usr/local/lib/python3.11/site-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://github.com/pypa/setuptools/issues/917 for details. ******************************************************************************** !! easy_install.initialize_options(self) [0/1] Install the project... -- Install configuration: "Release" -- Up-to-date: /app/_skbuild/linux-x86_64-3.11/cmake-install/llama_cpp/libllama.so copying _skbuild/linux-x86_64-3.11/cmake-install/llama_cpp/libllama.so -> llama_cpp/libllama.so running develop /usr/local/lib/python3.11/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** !! self.initialize_options() running egg_info writing llama_cpp_python.egg-info/PKG-INFO writing dependency_links to llama_cpp_python.egg-info/dependency_links.txt writing requirements to llama_cpp_python.egg-info/requires.txt writing top-level names to llama_cpp_python.egg-info/top_level.txt reading manifest file 'llama_cpp_python.egg-info/SOURCES.txt' adding license file 'LICENSE.md' writing manifest file 'llama_cpp_python.egg-info/SOURCES.txt' running build_ext Creating /usr/local/lib/python3.11/site-packages/llama-cpp-python.egg-link (link to .) llama-cpp-python 0.1.78 is already the active version in easy-install.pth Installed /app Processing dependencies for llama-cpp-python==0.1.78 Searching for diskcache==5.6.1 Best match: diskcache 5.6.1 Processing diskcache-5.6.1-py3.11.egg Adding diskcache 5.6.1 to easy-install.pth file Using /usr/local/lib/python3.11/site-packages/diskcache-5.6.1-py3.11.egg Searching for numpy==1.26.0b1 Best match: numpy 1.26.0b1 Processing numpy-1.26.0b1-py3.11-linux-x86_64.egg Adding numpy 1.26.0b1 to easy-install.pth file Installing f2py script to /usr/local/bin Using /usr/local/lib/python3.11/site-packages/numpy-1.26.0b1-py3.11-linux-x86_64.egg Searching for typing-extensions==4.7.1 Best match: typing-extensions 4.7.1 Adding typing-extensions 4.7.1 to easy-install.pth file Using /usr/local/lib/python3.11/site-packages Finished processing dependencies for llama-cpp-python==0.1.78 Initializing server with: Batch size: 2096 Number of CPU threads: 24 Number of GPU layers: 0 Context window: 4096 /usr/local/lib/python3.11/site-packages/pydantic/_internal/_fields.py:127: UserWarning: Field "model_alias" has conflict with protected namespace "model_". You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ('settings_',)`. warnings.warn( llama.cpp: loading model from /models/llama-2-13b-chat.bin llama_model_load_internal: format = ggjt v3 (latest) llama_model_load_internal: n_vocab = 32032 llama_model_load_internal: n_ctx = 4096 llama_model_load_internal: n_embd = 5120 llama_model_load_internal: n_mult = 256 llama_model_load_internal: n_head = 40 llama_model_load_internal: n_head_kv = 40 llama_model_load_internal: n_layer = 40 llama_model_load_internal: n_rot = 128 llama_model_load_internal: n_gqa = 1 llama_model_load_internal: rnorm_eps = 5.0e-06 llama_model_load_internal: n_ff = 13824 llama_model_load_internal: freq_base = 10000.0 llama_model_load_internal: freq_scale = 1 llama_model_load_internal: ftype = 2 (mostly Q4_0) llama_model_load_internal: model size = 13B llama_model_load_internal: ggml ctx size = 0.01 MB error loading model: llama.cpp: tensor 'layers.2.ffn_norm.weight' is missing from model llama_load_model_from_file: failed to load model Traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "/app/llama_cpp/server/__main__.py", line 46, in <module> app = create_app(settings=settings) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/llama_cpp/server/app.py", line 317, in create_app llama = llama_cpp.Llama( ^^^^^^^^^^^^^^^^ File "/app/llama_cpp/llama.py", line 328, in __init__ assert self.model is not None ^^^^^^^^^^^^^^^^^^^^^^ AssertionError /models/llama-2-13b-chat.bin model found. python3 setup.py develop /usr/local/lib/python3.11/site-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://github.com/pypa/setuptools/issues/917 for details. ******************************************************************************** !! easy_install.initialize_options(self) [0/1] Install the project... -- Install configuration: "Release" -- Up-to-date: /app/_skbuild/linux-x86_64-3.11/cmake-install/llama_cpp/libllama.so copying _skbuild/linux-x86_64-3.11/cmake-install/llama_cpp/libllama.so -> llama_cpp/libllama.so running develop /usr/local/lib/python3.11/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** !! self.initialize_options() running egg_info writing llama_cpp_python.egg-info/PKG-INFO writing dependency_links to llama_cpp_python.egg-info/dependency_links.txt writing requirements to llama_cpp_python.egg-info/requires.txt writing top-level names to llama_cpp_python.egg-info/top_level.txt reading manifest file 'llama_cpp_python.egg-info/SOURCES.txt' adding license file 'LICENSE.md' writing manifest file 'llama_cpp_python.egg-info/SOURCES.txt' running build_ext Creating /usr/local/lib/python3.11/site-packages/llama-cpp-python.egg-link (link to .) llama-cpp-python 0.1.78 is already the active version in easy-install.pth Installed /app Processing dependencies for llama-cpp-python==0.1.78 Searching for diskcache==5.6.1 Best match: diskcache 5.6.1 Processing diskcache-5.6.1-py3.11.egg Adding diskcache 5.6.1 to easy-install.pth file Using /usr/local/lib/python3.11/site-packages/diskcache-5.6.1-py3.11.egg Searching for numpy==1.26.0b1 Best match: numpy 1.26.0b1 Processing numpy-1.26.0b1-py3.11-linux-x86_64.egg Adding numpy 1.26.0b1 to easy-install.pth file Installing f2py script to /usr/local/bin Using /usr/local/lib/python3.11/site-packages/numpy-1.26.0b1-py3.11-linux-x86_64.egg Searching for typing-extensions==4.7.1 Best match: typing-extensions 4.7.1 Adding typing-extensions 4.7.1 to easy-install.pth file Using /usr/local/lib/python3.11/site-packages Finished processing dependencies for llama-cpp-python==0.1.78 Initializing server with: Batch size: 2096 Number of CPU threads: 24 Number of GPU layers: 0 Context window: 4096 /usr/local/lib/python3.11/site-packages/pydantic/_internal/_fields.py:127: UserWarning: Field "model_alias" has conflict with protected namespace "model_". You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ('settings_',)`. warnings.warn( llama.cpp: loading model from /models/llama-2-13b-chat.bin llama_model_load_internal: format = ggjt v3 (latest) llama_model_load_internal: n_vocab = 32032 llama_model_load_internal: n_ctx = 4096 llama_model_load_internal: n_embd = 5120 llama_model_load_internal: n_mult = 256 llama_model_load_internal: n_head = 40 llama_model_load_internal: n_head_kv = 40 llama_model_load_internal: n_layer = 40 llama_model_load_internal: n_rot = 128 llama_model_load_internal: n_gqa = 1 llama_model_load_internal: rnorm_eps = 5.0e-06 llama_model_load_internal: n_ff = 13824 llama_model_load_internal: freq_base = 10000.0 llama_model_load_internal: freq_scale = 1 llama_model_load_internal: ftype = 2 (mostly Q4_0) llama_model_load_internal: model size = 13B llama_model_load_internal: ggml ctx size = 0.01 MB error loading model: llama.cpp: tensor 'layers.2.ffn_norm.weight' is missing from model llama_load_model_from_file: failed to load model Traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "/app/llama_cpp/server/__main__.py", line 46, in <module> app = create_app(settings=settings) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/llama_cpp/server/app.py", line 317, in create_app llama = llama_cpp.Llama( ^^^^^^^^^^^^^^^^ File "/app/llama_cpp/llama.py", line 328, in __init__ assert self.model is not None ^^^^^^^^^^^^^^^^^^^^^^ AssertionError /models/llama-2-13b-chat.bin model found. python3 setup.py develop /usr/local/lib/python3.11/site-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://github.com/pypa/setuptools/issues/917 for details. ******************************************************************************** !! easy_install.initialize_options(self) [0/1] Install the project... -- Install configuration: "Release" -- Up-to-date: /app/_skbuild/linux-x86_64-3.11/cmake-install/llama_cpp/libllama.so copying _skbuild/linux-x86_64-3.11/cmake-install/llama_cpp/libllama.so -> llama_cpp/libllama.so running develop /usr/local/lib/python3.11/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** !! self.initialize_options() running egg_info writing llama_cpp_python.egg-info/PKG-INFO writing dependency_links to llama_cpp_python.egg-info/dependency_links.txt writing requirements to llama_cpp_python.egg-info/requires.txt writing top-level names to llama_cpp_python.egg-info/top_level.txt reading manifest file 'llama_cpp_python.egg-info/SOURCES.txt' adding license file 'LICENSE.md' writing manifest file 'llama_cpp_python.egg-info/SOURCES.txt' running build_ext Creating /usr/local/lib/python3.11/site-packages/llama-cpp-python.egg-link (link to .) llama-cpp-python 0.1.78 is already the active version in easy-install.pth Installed /app Processing dependencies for llama-cpp-python==0.1.78 Searching for diskcache==5.6.1 Best match: diskcache 5.6.1 Processing diskcache-5.6.1-py3.11.egg Adding diskcache 5.6.1 to easy-install.pth file Using /usr/local/lib/python3.11/site-packages/diskcache-5.6.1-py3.11.egg Searching for numpy==1.26.0b1 Best match: numpy 1.26.0b1 Processing numpy-1.26.0b1-py3.11-linux-x86_64.egg Adding numpy 1.26.0b1 to easy-install.pth file Installing f2py script to /usr/local/bin Using /usr/local/lib/python3.11/site-packages/numpy-1.26.0b1-py3.11-linux-x86_64.egg Searching for typing-extensions==4.7.1 Best match: typing-extensions 4.7.1 Adding typing-extensions 4.7.1 to easy-install.pth file Using /usr/local/lib/python3.11/site-packages Finished processing dependencies for llama-cpp-python==0.1.78 Initializing server with: Batch size: 2096 Number of CPU threads: 24 Number of GPU layers: 0 Context window: 4096 /usr/local/lib/python3.11/site-packages/pydantic/_internal/_fields.py:127: UserWarning: Field "model_alias" has conflict with protected namespace "model_". You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ('settings_',)`. warnings.warn( llama.cpp: loading model from /models/llama-2-13b-chat.bin llama_model_load_internal: format = ggjt v3 (latest) llama_model_load_internal: n_vocab = 32032 llama_model_load_internal: n_ctx = 4096 llama_model_load_internal: n_embd = 5120 llama_model_load_internal: n_mult = 256 llama_model_load_internal: n_head = 40 llama_model_load_internal: n_head_kv = 40 llama_model_load_internal: n_layer = 40 llama_model_load_internal: n_rot = 128 llama_model_load_internal: n_gqa = 1 llama_model_load_internal: rnorm_eps = 5.0e-06 llama_model_load_internal: n_ff = 13824 llama_model_load_internal: freq_base = 10000.0 llama_model_load_internal: freq_scale = 1 llama_model_load_internal: ftype = 2 (mostly Q4_0) llama_model_load_internal: model size = 13B llama_model_load_internal: ggml ctx size = 0.01 MB error loading model: llama.cpp: tensor 'layers.2.ffn_norm.weight' is missing from model llama_load_model_from_file: failed to load model Traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "/app/llama_cpp/server/__main__.py", line 46, in <module> app = create_app(settings=settings) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/llama_cpp/server/app.py", line 317, in create_app llama = llama_cpp.Llama( ^^^^^^^^^^^^^^^^ File "/app/llama_cpp/llama.py", line 328, in __init__ assert self.model is not None ^^^^^^^^^^^^^^^^^^^^^^ AssertionError ``` **Screenshots (if applicable):** [Attach any relevant screenshots to help illustrate the issue] ## Installation Method ``` docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v ollama-webui:/app/backend/data --name ollama-webui --restart always ghcr.io/ollama-webui/ollama-webui:main ``` ## Additional Information [Include any additional details that may help in understanding and reproducing the issue. This could include specific configurations, error messages, or anything else relevant to the bug.] ## Note If the bug report is incomplete or does not follow the provided instructions, it may not be addressed. Please ensure that you have followed the steps outlined in the README.md and troubleshooting.md documents, and provide all necessary information for us to reproduce and address the issue. Thank you!

GiteaMirror closed this issue

2025-11-11 14:11:10 -06:00

GiteaMirror commented

2025-11-11 14:11:11 -06:00