github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-08 00:51:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #3206] 我将MiniCPM-2B-dpo-bf16-gguf.gguf模型成功的导入到了ollama中，并运行起来了。但是在推理的过程中，发现模型再说胡话，臆想比较严重。详见截图 #48489

New Issue
Closed
opened 2026-04-28 08:41:30 -05:00 by GiteaMirror · 13 comments
GiteaMirror commented 2026-04-28 08:41:30 -05:00
Owner
Copy Link

Originally created by @zhao1012 on GitHub (Mar 18, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/3206

          我将MiniCPM-2B-dpo-bf16-gguf.gguf模型成功的导入到了ollama中，并运行起来了。但是在推理的过程中，发现模型再说胡话，臆想比较严重。详见截图

miniCPM-2b 在ollama上乱说话

Originally posted by @zhao1012 in https://github.com/ollama/ollama/issues/2383#issuecomment-2002754353

Originally created by @zhao1012 on GitHub (Mar 18, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/3206 我将MiniCPM-2B-dpo-bf16-gguf.gguf模型成功的导入到了ollama中，并运行起来了。但是在推理的过程中，发现模型再说胡话，臆想比较严重。详见截图 ![miniCPM-2b 在ollama上乱说话](https://github.com/ollama/ollama/assets/38517343/8f68f251-d07d-4ea5-95a1-59dde7a84bf4) _Originally posted by @zhao1012 in https://github.com/ollama/ollama/issues/2383#issuecomment-2002754353_
GiteaMirror commented 2026-04-28 08:41:32 -05:00
Author
Owner
Copy Link

@zhao1012 commented on GitHub (Mar 18, 2024):

这个问题是因为转换成gguf格式中模型权重或者什么损坏了？模型为huggingface下载

<!-- gh-comment-id:2002756865 --> @zhao1012 commented on GitHub (Mar 18, 2024): 这个问题是因为转换成gguf格式中模型权重或者什么损坏了？模型为huggingface下载
GiteaMirror commented 2026-04-28 08:41:34 -05:00
Author
Owner
Copy Link

@gamersover commented on GitHub (Mar 18, 2024):

原始模型会输出什么？如果是一样的，那么可能是模型本身的效果不好，看起来模型大小只有2B。也可以给出你的转换代码或者模型参数（如temperature）

What will the original model output? If it's the same, then it's likely that the model itself iisn't performing well. It seems the model size is only 2B. You can also provide your conversion code to see if there are any issues or adjust parameters (such as temperature).

<!-- gh-comment-id:2003139425 --> @gamersover commented on GitHub (Mar 18, 2024): 原始模型会输出什么？如果是一样的，那么可能是模型本身的效果不好，看起来模型大小只有2B。也可以给出你的转换代码或者模型参数（如temperature） What will the original model output? If it's the same, then it's likely that the model itself iisn't performing well. It seems the model size is only 2B. You can also provide your conversion code to see if there are any issues or adjust parameters (such as temperature).
GiteaMirror commented 2026-04-28 08:41:36 -05:00
Author
Owner
Copy Link

@zhao1012 commented on GitHub (Mar 18, 2024):

原始模型应该是输出结果为2。不会存在后面一堆中文解释。模型来源于https://github.com/OpenBMB/MiniCPM?tab=readme-ov-file下提供的MiniCPM-2B-dpo-bf16-gguf.gguf（hunggingface下载）。
通过以下代码将MiniCPM-2B-dpo-bf16-gguf.gguf导入到了ollama中。

import ollama
if __name__ == "__main__":
    print("Procedures begin to execute！")
    Modelfile = 'FROM ./models/hungface_miniCPM2b_gguf/MiniCPM-2B-dpo-q4km-gguf.gguf'
    ollama.create(model='MiniCPM-2B-q4km',modelfile=Modelfile)

我通过MiniCPM-2B-dpo-bf16模型问了同样的问题回答是2.详细见截图。
miniCPM-2b 在ollama上乱说话1

<!-- gh-comment-id:2003185612 --> @zhao1012 commented on GitHub (Mar 18, 2024): 原始模型应该是输出结果为2。不会存在后面一堆中文解释。模型来源于https://github.com/OpenBMB/MiniCPM?tab=readme-ov-file下提供的MiniCPM-2B-dpo-bf16-gguf.gguf（hunggingface下载）。 通过以下代码将MiniCPM-2B-dpo-bf16-gguf.gguf导入到了ollama中。 ``` import ollama if __name__ == "__main__": print("Procedures begin to execute！") Modelfile = 'FROM ./models/hungface_miniCPM2b_gguf/MiniCPM-2B-dpo-q4km-gguf.gguf' ollama.create(model='MiniCPM-2B-q4km',modelfile=Modelfile) ``` 我通过MiniCPM-2B-dpo-bf16模型问了同样的问题回答是2.详细见截图。 ![miniCPM-2b 在ollama上乱说话1](https://github.com/ollama/ollama/assets/38517343/e757be79-aa98-4c24-b44a-64c603931644)
GiteaMirror commented 2026-04-28 08:41:37 -05:00
Author
Owner
Copy Link

@zhao1012 commented on GitHub (Mar 18, 2024):

补充一张照片，同样问题在ollama导入的MiniCPM-2B-dpo-bf16-gguf.gguf上的表现。
miniCPM-2b 在ollama上乱说话2

<!-- gh-comment-id:2003193077 --> @zhao1012 commented on GitHub (Mar 18, 2024): 补充一张照片，同样问题在ollama导入的MiniCPM-2B-dpo-bf16-gguf.gguf上的表现。 ![miniCPM-2b 在ollama上乱说话2](https://github.com/ollama/ollama/assets/38517343/ac9ff98b-f5fe-4149-95e6-c197ea3073cc)
GiteaMirror commented 2026-04-28 08:41:39 -05:00
Author
Owner
Copy Link

@gamersover commented on GitHub (Mar 18, 2024):

看起原始模型是bf16，而ollama的模型是4bit量化后的，模型回答有差异也合理，试试调整temperature等参数，参考这里https://github.com/ollama/ollama/blob/main/docs/modelfile.md#parameter

The original model appears to be in bf16, while Ollama's model is quantized to 4 bits. It's reasonable to expect differences in model responses. Try adjusting parameters such as temperature, referring to this link: https://github.com/ollama/ollama/blob/main/docs/modelfile.md#parameter

<!-- gh-comment-id:2003246948 --> @gamersover commented on GitHub (Mar 18, 2024): 看起原始模型是bf16，而ollama的模型是4bit量化后的，模型回答有差异也合理，试试调整temperature等参数，参考这里https://github.com/ollama/ollama/blob/main/docs/modelfile.md#parameter The original model appears to be in bf16, while Ollama's model is quantized to 4 bits. It's reasonable to expect differences in model responses. Try adjusting parameters such as temperature, referring to this link: https://github.com/ollama/ollama/blob/main/docs/modelfile.md#parameter
GiteaMirror commented 2026-04-28 08:41:41 -05:00
Author
Owner
Copy Link

@zhao1012 commented on GitHub (Mar 18, 2024):

我使用ollama导入了bf16（MiniCPM-2B-dpo-fp16-gguf.gguf（模型来源于[https://github.com/OpenBMB/MiniCPM?tab=readme-ov-file下提供的MiniCPM-2B-dpo-bf16-gguf.gguf（hunggingface下载））的模型，通过ollama运行相同的问题（1+1=?），依旧回答会如4bit量化后的一样。详细见如下图

import ollama
if __name__ == "__main__":
    print("Procedures begin to execute！")
    Modelfile = 'FROM ./models/hungface_miniCPM2b_gguf/MiniCPM-2B-dpo-fp16-gguf.gguf'
    ollama.create(model='MiniCPM-2B-dpo',modelfile=Modelfile)

这个问题让我很费解。
miniCPM-2b 在ollama上乱说话1
miniCPM-2b 在ollama上乱说话2
miniCPM-2b 在ollama上乱说话3

<!-- gh-comment-id:2003274747 --> @zhao1012 commented on GitHub (Mar 18, 2024): 我使用ollama导入了bf16（MiniCPM-2B-dpo-fp16-gguf.gguf（模型来源于[https://github.com/OpenBMB/MiniCPM?tab=readme-ov-file下提供的MiniCPM-2B-dpo-bf16-gguf.gguf（hunggingface下载））的模型，通过ollama运行相同的问题（1+1=?），依旧回答会如4bit量化后的一样。详细见如下图 ``` import ollama if __name__ == "__main__": print("Procedures begin to execute！") Modelfile = 'FROM ./models/hungface_miniCPM2b_gguf/MiniCPM-2B-dpo-fp16-gguf.gguf' ollama.create(model='MiniCPM-2B-dpo',modelfile=Modelfile) ``` 这个问题让我很费解。 ![miniCPM-2b 在ollama上乱说话1](https://github.com/ollama/ollama/assets/38517343/3b0f9e3f-73dd-4b4e-8d43-3c1f1447c1cc) ![miniCPM-2b 在ollama上乱说话2](https://github.com/ollama/ollama/assets/38517343/2a9135e9-6cc0-4fa8-b245-10770949a4ad) ![miniCPM-2b 在ollama上乱说话3](https://github.com/ollama/ollama/assets/38517343/cc96bd33-3ce4-46da-8046-65e728772bf3)
GiteaMirror commented 2026-04-28 08:41:46 -05:00
Author
Owner
Copy Link

@pretenderlu commented on GitHub (May 8, 2024):

我也遇到了类似的问题，我在lmstudio中使用相同的模型，并不会有这个问题。但是在ollama中却会疯狂输出。

I also encountered a similar problem. I use the same model in lmstudio and there is no such problem. But it will output wildly in ollama.

<!-- gh-comment-id:2100247740 --> @pretenderlu commented on GitHub (May 8, 2024): 我也遇到了类似的问题，我在lmstudio中使用相同的模型，并不会有这个问题。但是在ollama中却会疯狂输出。 -------- I also encountered a similar problem. I use the same model in lmstudio and there is no such problem. But it will output wildly in ollama.
GiteaMirror commented 2026-04-28 08:41:49 -05:00
Author
Owner
Copy Link

@amy108 commented on GitHub (May 23, 2024):

请问这个问题有没有解决，遇到同样问题

<!-- gh-comment-id:2126527428 --> @amy108 commented on GitHub (May 23, 2024): 请问这个问题有没有解决，遇到同样问题
GiteaMirror commented 2026-04-28 08:41:52 -05:00
Author
Owner
Copy Link

@a937983423 commented on GitHub (May 25, 2024):

请问这个问题有没有解决，我也遇到同样问题

<!-- gh-comment-id:2131383676 --> @a937983423 commented on GitHub (May 25, 2024): 请问这个问题有没有解决，我也遇到同样问题
GiteaMirror commented 2026-04-28 08:41:56 -05:00
Author
Owner
Copy Link

@boebo8 commented on GitHub (May 28, 2024):

有没有新版的minicpm llama3 V2.5的gguf格式可下载？

<!-- gh-comment-id:2134169067 --> @boebo8 commented on GitHub (May 28, 2024): 有没有新版的minicpm llama3 V2.5的gguf格式可下载？
GiteaMirror commented 2026-04-28 08:41:58 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (Jun 9, 2024):

Closing for https://github.com/ollama/ollama/issues/2383

<!-- gh-comment-id:2156703672 --> @jmorganca commented on GitHub (Jun 9, 2024): Closing for https://github.com/ollama/ollama/issues/2383
GiteaMirror commented 2026-04-28 08:42:00 -05:00
Author
Owner
Copy Link

@Az-CMQ commented on GitHub (Aug 9, 2024):

温度太高了，改一下modelfile就行了：

FROM model.gguf
PARAMETER temperature 0.7
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
TEMPLATE """
<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""
SYSTEM """You are a helpful assistant."""

<!-- gh-comment-id:2278278717 --> @Az-CMQ commented on GitHub (Aug 9, 2024): ### 温度太高了，改一下modelfile就行了： FROM model.gguf PARAMETER temperature 0.7 PARAMETER stop "<|im_start|>" PARAMETER stop "<|im_end|>" TEMPLATE """ <|im_start|>system {{ .System }}<|im_end|> <|im_start|>user {{ .Prompt }}<|im_end|> <|im_start|>assistant """ SYSTEM """You are a helpful assistant."""
GiteaMirror commented 2026-04-28 08:42:01 -05:00
Author
Owner
Copy Link

@Az-CMQ commented on GitHub (Aug 9, 2024):

我上次把温度改成了1直接胡言乱语

<!-- gh-comment-id:2278286376 --> @Az-CMQ commented on GitHub (Aug 9, 2024): 我上次把温度改成了1直接胡言乱语
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#48489
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?