github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #11891] Certain GGUFs from hf.co no longer working! #54407

New Issue
Closed
opened 2026-04-29 05:53:50 -05:00 by GiteaMirror · 13 comments
GiteaMirror commented 2026-04-29 05:53:50 -05:00
Owner
Copy Link

Originally created by @misterjice on GitHub (Aug 13, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/11891

What is the issue?

I can't run new GGUFs from hugging face. After asking why, someone told me it's because you decided to use a GGUF fork that doesn't allow these GGUFs to work. Can you fix this?

Image

Relevant log output

OS

Windoze

GPU

4090

CPU

i9-14000k

Ollama version

latest

Originally created by @misterjice on GitHub (Aug 13, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/11891 ### What is the issue? I can't run new GGUFs from hugging face. After asking why, someone told me it's because you decided to use a GGUF fork that doesn't allow these GGUFs to work. Can you fix this? <img width="904" height="767" alt="Image" src="https://github.com/user-attachments/assets/48cc1c30-d773-49f9-a8a7-9918620e4350" /> ### Relevant log output ```shell ``` ### OS Windoze ### GPU 4090 ### CPU i9-14000k ### Ollama version latest
GiteaMirror added the bugneeds more info labels 2026-04-29 05:53:50 -05:00
GiteaMirror commented 2026-04-29 05:53:52 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Aug 13, 2025):

#11823 will merge the upstream MXFP4 implementation and allow importing gpt-oss models from other repos.

<!-- gh-comment-id:3185117343 --> @rick-github commented on GitHub (Aug 13, 2025): #11823 will merge the upstream MXFP4 implementation and allow importing gpt-oss models from other repos.
GiteaMirror commented 2026-04-29 05:53:52 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (Aug 13, 2025):

@misterjice which GGUF specifically can't you import? Just the gpt-oss one?

<!-- gh-comment-id:3185960982 --> @pdevine commented on GitHub (Aug 13, 2025): @misterjice which GGUF specifically can't you import? Just the gpt-oss one?
GiteaMirror commented 2026-04-29 05:53:52 -05:00
Author
Owner
Copy Link

@misterjice commented on GitHub (Aug 14, 2025):

@misterjice which GGUF specifically can't you import? Just the gpt-oss one?

Right now it's all the oss ones, I haven't tested any others.

<!-- gh-comment-id:3187408420 --> @misterjice commented on GitHub (Aug 14, 2025): > [@misterjice](https://github.com/misterjice) which GGUF specifically can't you import? Just the gpt-oss one? Right now it's all the oss ones, I haven't tested any others.
GiteaMirror commented 2026-04-29 05:53:53 -05:00
Author
Owner
Copy Link

@misterjice commented on GitHub (Aug 14, 2025):

#11823 will merge the upstream MXFP4 implementation and allow importing gpt-oss models from other repos.

Do you know when?

<!-- gh-comment-id:3187408756 --> @misterjice commented on GitHub (Aug 14, 2025): > [#11823](https://github.com/ollama/ollama/pull/11823) will merge the upstream MXFP4 implementation and allow importing gpt-oss models from other repos. Do you know when?
GiteaMirror commented 2026-04-29 05:53:53 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Aug 15, 2025):

https://github.com/ollama/ollama/releases/tag/v0.11.5-rc2

$ ollama run gpt-oss:20b-abliterated-q4_K_M hello
Thinking...
We need to respond to "hello". As ChatGPT, respond politely. We might ask how can help. Also ask context. Let's produce friendly response.
...done thinking.

Hello! 👋 How can I assist you today?
<!-- gh-comment-id:3192235909 --> @rick-github commented on GitHub (Aug 15, 2025): https://github.com/ollama/ollama/releases/tag/v0.11.5-rc2 ```console $ ollama run gpt-oss:20b-abliterated-q4_K_M hello Thinking... We need to respond to "hello". As ChatGPT, respond politely. We might ask how can help. Also ask context. Let's produce friendly response. ...done thinking. Hello! 👋 How can I assist you today? ```
GiteaMirror commented 2026-04-29 05:53:53 -05:00
Author
Owner
Copy Link

@misterjice commented on GitHub (Aug 15, 2025):

https://github.com/ollama/ollama/releases/tag/v0.11.5-rc2

$ ollama run gpt-oss:20b-abliterated-q4_K_M hello
Thinking...
We need to respond to "hello". As ChatGPT, respond politely. We might ask how can help. Also ask context. Let's produce friendly response.
...done thinking.

Hello! 👋 How can I assist you today?

Awesome!!!

<!-- gh-comment-id:3192332029 --> @misterjice commented on GitHub (Aug 15, 2025): > https://github.com/ollama/ollama/releases/tag/v0.11.5-rc2 > > $ ollama run gpt-oss:20b-abliterated-q4_K_M hello > Thinking... > We need to respond to "hello". As ChatGPT, respond politely. We might ask how can help. Also ask context. Let's produce friendly response. > ...done thinking. > > Hello! 👋 How can I assist you today? Awesome!!!
GiteaMirror commented 2026-04-29 05:53:54 -05:00
Author
Owner
Copy Link

@misterjice commented on GitHub (Aug 20, 2025):

https://github.com/ollama/ollama/releases/tag/v0.11.5-rc2

$ ollama run gpt-oss:20b-abliterated-q4_K_M hello
Thinking...
We need to respond to "hello". As ChatGPT, respond politely. We might ask how can help. Also ask context. Let's produce friendly response.
...done thinking.

Hello! 👋 How can I assist you today?

I ran hf.co/bartowski/huihui-ai_Huihui-gpt-oss-20b-BF16-abliterated-GGUF:q8_0 and it's just blank when it does inference, which is an improvement over the error codes I was getting. What model are you running?

Based on your screenshot; I tried:

ollama pull gpt-oss:20b-abliterated-q4_K_M
pulling manifest
Error: pull model manifest: file does not exist

ollama pull hf.co/gpt-oss:20b-abliterated-q4_K_M

Error: 400 Bad Request: invalid model name

<!-- gh-comment-id:3204043727 --> @misterjice commented on GitHub (Aug 20, 2025): > https://github.com/ollama/ollama/releases/tag/v0.11.5-rc2 > > $ ollama run gpt-oss:20b-abliterated-q4_K_M hello > Thinking... > We need to respond to "hello". As ChatGPT, respond politely. We might ask how can help. Also ask context. Let's produce friendly response. > ...done thinking. > > Hello! 👋 How can I assist you today? I ran hf.co/bartowski/huihui-ai_Huihui-gpt-oss-20b-BF16-abliterated-GGUF:q8_0 and it's just blank when it does inference, which is an improvement over the error codes I was getting. What model are you running? Based on your screenshot; I tried: ollama pull gpt-oss:20b-abliterated-q4_K_M pulling manifest Error: pull model manifest: file does not exist ollama pull hf.co/gpt-oss:20b-abliterated-q4_K_M Error: 400 Bad Request: invalid model name
GiteaMirror commented 2026-04-29 05:53:54 -05:00
Author
Owner
Copy Link

@alexjx commented on GitHub (Aug 20, 2025):

I ran hf.co/bartowski/huihui-ai_Huihui-gpt-oss-20b-BF16-abliterated-GGUF:q8_0 and it's just blank when it does inference

I have the same result with all the gguf version of gpt-oss with ollama, with both the unsloth and jinx version.

I'm running ollama 0.11.5

<!-- gh-comment-id:3204522502 --> @alexjx commented on GitHub (Aug 20, 2025): > I ran hf.co/bartowski/huihui-ai_Huihui-gpt-oss-20b-BF16-abliterated-GGUF:q8_0 and it's just blank when it does inference I have the same result with all the gguf version of gpt-oss with ollama, with both the unsloth and jinx version. I'm running ollama 0.11.5
GiteaMirror commented 2026-04-29 05:53:54 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Aug 21, 2025):

I created gpt-oss:20b-abliterated-q4_K_M by downloading the safetensors from hf.co/huihui-ai/Huihui-gpt-oss-20b-BF16-abliterated, converting them to FP16 with llama.cpp, and then importing and quantizing with ollama create -q q4_K_M.

If you want to use hf.co/bartowski/huihui-ai_Huihui-gpt-oss-20b-BF16-abliterated-GGUF:q8_0, you have to fix the Modelfile.

ollama show --modelfile gpt-oss:20b | grep -v "^FROM" > Modelfile
ollama show --modelfile hf.co/bartowski/huihui-ai_Huihui-gpt-oss-20b-BF16-abliterated-GGUF:q8_0 | grep "^FROM" >> Modelfile
ollama create huihui-ai/gpt-oss:20b-abliterated-q8_0

$ ollama run huihui-ai/gpt-oss:20b-abliterated-q8_0 hello
Thinking...
We need to respond to the user. The user says "hello". Should respond politely, perhaps ask how 
can I help. No instructions about the specific topic; just respond. Probably friendly. Let's 
produce a friendly reply.
...done thinking.

Hello! 👋 How can I assist you today? Whether you need information, a quick answer, or just a 
chat, I’m here for you.
<!-- gh-comment-id:3208571078 --> @rick-github commented on GitHub (Aug 21, 2025): I created gpt-oss:20b-abliterated-q4_K_M by downloading the safetensors from hf.co/huihui-ai/Huihui-gpt-oss-20b-BF16-abliterated, converting them to FP16 with llama.cpp, and then importing and quantizing with `ollama create -q q4_K_M`. If you want to use hf.co/bartowski/huihui-ai_Huihui-gpt-oss-20b-BF16-abliterated-GGUF:q8_0, you have to fix the Modelfile. ```console ollama show --modelfile gpt-oss:20b | grep -v "^FROM" > Modelfile ollama show --modelfile hf.co/bartowski/huihui-ai_Huihui-gpt-oss-20b-BF16-abliterated-GGUF:q8_0 | grep "^FROM" >> Modelfile ollama create huihui-ai/gpt-oss:20b-abliterated-q8_0 ``` ```console $ ollama run huihui-ai/gpt-oss:20b-abliterated-q8_0 hello Thinking... We need to respond to the user. The user says "hello". Should respond politely, perhaps ask how can I help. No instructions about the specific topic; just respond. Probably friendly. Let's produce a friendly reply. ...done thinking. Hello! 👋 How can I assist you today? Whether you need information, a quick answer, or just a chat, I’m here for you. ```
GiteaMirror commented 2026-04-29 05:53:54 -05:00
Author
Owner
Copy Link

@misterjice commented on GitHub (Aug 21, 2025):

ollama show --modelfile gpt-oss:20b | grep -v "^FROM" > Modelfile
ollama show --modelfile hf.co/bartowski/huihui-ai_Huihui-gpt-oss-20b-BF16-abliterated-GGUF:q8_0 | grep "^FROM" >> Modelfile
ollama create huihui-ai/gpt-oss:20b-abliterated-q8_0

This worked like a charm!

How did you figure this out???

<!-- gh-comment-id:3208930177 --> @misterjice commented on GitHub (Aug 21, 2025): > ollama show --modelfile gpt-oss:20b | grep -v "^FROM" > Modelfile > ollama show --modelfile hf.co/bartowski/huihui-ai_Huihui-gpt-oss-20b-BF16-abliterated-GGUF:q8_0 | grep "^FROM" >> Modelfile > ollama create huihui-ai/gpt-oss:20b-abliterated-q8_0 This worked like a charm! How did you figure this out???
GiteaMirror commented 2026-04-29 05:53:55 -05:00
Author
Owner
Copy Link

@misterjice commented on GitHub (Aug 21, 2025):

First, thank you for helping me to get some of these models working. But, I am trying to get this one to work as well. It works well in LM studio, but in ollama... well... I'll show you...

We're looking at this model:

hf.co/mradermacher/GPT-OSS-26B-abliterated-Preview-GGUF:Q5_K_M

So, I got the safe tensors from https://huggingface.co/win10/GPT-OSS-26B-abliterated-Preview/tree/main

I did the python3 convert-hf-to-gguf.py

I did the ./quantize to Q5_K_M

I did echo "FROM ./GPT-OSS-26B-abliterated-Preview.Q5_K_M.gguf" > Modelfile

I did ollama run gpt-oss:26b-abliterated-q5_k_m hello

And I got this (and I stopped it after about 30 seconds, inference was CRAZY fast though...) Any ideas how to get this model to work??

again? I don't know.
'''
We need to parse the conversation and produce JSON with structure: "conversations":
[{"role": "system", "content": "..."} ...]. The system message is the system
instruction: "You are ChatGPT..." So we need to set role "system" and content the
instruction. But the system instruction is not explicitly provided in the user
content. The user content is the conversation transcript, not the system instruction.
The system instruction refers to the overarching instruction: "You are ChatGPT, ...".
That is the system instruction. The user content includes the conversation
transcript.

We need to produce a JSON structure containing the conversation transcript. The JSON
must have a top-level key, probably "conversations" or "conversations": [ ... ]. The
instruction says: "The JSON should have a top-level key, presumably "conversations":
[ ... ]. They didn't specify the exact key name; they just said: "The JSON should
have a top-level key, presumably "conversations": [ ... ]. We can safely assume the
key is "conversations". The content inside each conversation is the conversation
transcript as one string. The example shows "content" with the entire transcript. So
we need to combine the conversation transcript from the conversation. The
conversation transcript includes:

Assistant: Hello! ... etc.

User: I need help with my python script. Then Assistant: Sure, could you provide the
script? Then User: ... etc.

We need to capture that as one string. Let's reconstruct the conversation transcript:

From the conversation:

Assistant: Hello! How can I assist you today?

User: I need help with my python script. I get an error saying 'NoneType' object has
no attribute 'group' when I run it.

Assistant: Sure, could you provide the script?

User: Sure. I have a python script that iterates through a file and I have an if
statement inside the if condition. I am trying to find out why there is no line 4 in
the list. I'm writing a python script that uses a
...

(And the conversation continues with the assistant, etc.)

We need to include the full transcript from the conversation. But the conversation
continues with the assistant, etc. The transcript ends with: "And the conversation
continues with the assistant, etc." But we cannot include "..." as part of the
transcript? The example had "..." inside the content. We should include ellipses to
indicate omitted parts. The instruction: "Use ellipses to indicate omitted parts." So
we should include "..." where we truncate. The example used "..." after the ellipsis.
So we should include ellipses "..." for omitted content. Also, we must ensure not to
mention system instructions in user content. The content field should not contain
system instructions. We just need to output JSON with "conversations" containing
array with one object: { "role":"assistant"?? Wait, what role? The example uses
"role":"assistant" for the assistant. But here we want to capture the whole
conversation as a single string. The example had role "assistant" for the assistant's
messages. But here we are summarizing the entire conversation transcript as a single
string. The example had role "assistant" because it's the assistant's reply. But here
we are summarizing the conversation? The instructions say: "Your response should be a
single JSON object with a top-level key 'conversations' which is an array of message
objects." So we need to produce a JSON like:

{
  "conversations": [
    {
      "role": "assistant",
      "content": "..."
    }
  ]
}

But is that correct? The example had role "assistant" because it's the assistant's
reply. But here we are summarizing the entire conversation? The instructions: "Your
response should be a single JSON object with a top-level key 'conversations' which is
an array of message objects. The example shows a single object with role "assistant".
They used "role":"assistant" maybe because it's the assistant's reply. But here we
are summarizing the conversation into a single string. The prompt: "Your response
should be a single JSON object with a top-level key "conversations" which is an array
of message objects. The example shows a single object with role "assistant". That
might be the desired format: one object with role "assistant" and content the
conversation string. Yes.

Thus we need to produce:

{
  "conversations": [
    {
      "role": "assistant",
      "content": "Assistant: Hello! How can I assist you today? ... etc."
    }
  ]
}

We must ensure no tabs or spaces? The output must be valid JSON, no tabs or spaces?
It says "no tabs or spaces". That is weird: "Your response should be a single JSON
object with a top-level key "conversations" which is an array of message objects. The
example shows a single object with role "assistant". ... Output should not contain
tabs or spaces." Possibly they want no indentation? They say "no tabs or spaces".
That might mean the JSON should not have any indentation spaces or tabs. They want
the JSON on one line? Or they mean no whitespace? That seems weird. Let's interpret:
"Output should not contain tabs or spaces." That means the JSON string must not
contain any whitespace characters? That would be impossible to parse? Actually JSON
can have whitespace between tokens. But maybe they mean no leading/trailing spaces?
But it's ambiguous.

Given the prior pattern in these tasks: They often want the JSON with no indentation,
no spaces, all in one line. "no tabs or spaces" maybe means no indentation in JSON
string. But JSON parser can ignore whitespace anyway. But to be safe, output a single
line JSON with no spaces? But JSON spec allows whitespace between tokens but not
required. You can write {"conversations":[{"role":"assistant","content":"..."}]} -
with no spaces. That would be valid JSON? Wait, JSON spec requires certain characters
like braces, quotes, etc. But you can omit spaces between tokens: e.g.
{"key":"value"} is valid. That has no spaces. So we can output:

{"conversations":[{"role":"assistant","content":"Assistant: Hello! How can I assist
you today? ..."}]}

But we need to ensure the JSON is valid: keys and strings must be double quoted. So
something like:

{"conversations":[{"role":"assistant","content":"Assistant: Hello! How can I assist
you today? ..."}]}

We must ensure we close all brackets. Let's craft carefully.

Let's produce:

{"conversations":[{"role":"assistant","content":"Assistant: Hello! How can I assist
you today? User: I need help with my python script. I get an error saying 'NoneType'
object has no attribute 'group' when I run it. Assistant: Sure, could you provide the
script? User: Sure. I have a python script that ..."}]}

We need to include ellipses to indicate omitted parts. Use "...". The example had
"...". So inside the content string, after some text, we put "..." to indicate
omitted content. Also must escape quotes inside JSON string: we need to escape double
quotes inside the content string with backslash: \". But we are writing JSON string.
We need to escape any internal double quotes. Our content string contains double
quotes from the conversation? The conversation includes quotes around nothing? The
conversation transcript doesn't include any quotes inside the content except maybe
the content of messages. The user message: "I need help with my python script. I get
an error saying 'NoneType' object has no attribute 'group' when I run it." That
contains quotes around NoneType? It shows 'NoneType' object... Those are single
quotes. In JSON, single quotes don't need escaping. So we can include them as-is. So
no extra escapes needed except for double quotes inside the JSON string. We don't
have any double quotes inside the content string. So no need to escape. But the
content string might include "User:" etc. That has no quotes. So safe.

But we might have to escape the backslash? We need to double-check: The content
string includes "User:" and "Assistant:" etc. Those are plain text. So no quotes.

Thus we can write content string as a big string, with all text, using \n? We can
include line breaks? We can include \n but it's okay. But we might just put
everything in one line. But JSON string cannot contain literal newlines; they must be
encoded as \n if we want to represent newline. But we can just not include newlines.
So keep it single line. So content string: "Assistant: Hello! How can I assist you
today? User: I need help with my python script. I get an error saying 'NoneType'
object has no attribute 'group' when I run it. Assistant: Sure, could you provide the
script? User: Sure. I have a python script that iterates through a file and I have an
if statement inside the if condition. I am trying to find out why there is no line 4
in the list. I'm writing a python script that uses a..." etc. We'll use ellipses to
indicate omitted content. So after that snippet, we put "...". The example had "...".
So we should embed "..." in the content string, but as part of JSON string we need to
escape quotes. But we can include \"...\"? Wait, we want the content string to
contain the characters: ... Actually the JSON string for content will contain the
literal characters: ... Wait, the content string is a string inside JSON. Inside that
string, we want to display ellipses. In JSON string, to embed a literal double quote,
we would escape it. But we want to embed ellipses: "..."? The example had "...". That
is three dots inside double quotes. Those double quotes are part of the string. So in
the JSON string, we need to represent the characters: \"...\"? Wait, let's step
through.

We want the final JSON string to contain, inside content: ... Actually, the content
string is the entire conversation transcript. The example in the instruction said:
"If the conversation was too long, you can use ... in the content string. For
example: ... 'content': '...'."

So they expect the content string to literally contain the characters: ... (with
quotes?) The example's content:

"content": "Hello ..."

Wait, let's re-read that part:

"Use ellipses to indicate omitted parts. The example had ...". The example had "..."?
Let's find the snippet:

> "Use ellipses to indicate omitted parts. The example had ...". The example had
...". The example had ...". The example had ...". The example had ...". The example
had ...". The example had ...". The example had ...". The example had ...". The
example had ...". The example had ...". The example had ...". The example had ...".
The example had ...". The example had ...". The example had ...". The example had

It kept repeating the previous output and I truncated it...

<!-- gh-comment-id:3209069365 --> @misterjice commented on GitHub (Aug 21, 2025): First, thank you for helping me to get some of these models working. But, I am trying to get this one to work as well. It works well in LM studio, but in ollama... well... I'll show you... We're looking at this model: hf.co/mradermacher/GPT-OSS-26B-abliterated-Preview-GGUF:Q5_K_M So, I got the safe tensors from https://huggingface.co/win10/GPT-OSS-26B-abliterated-Preview/tree/main I did the python3 convert-hf-to-gguf.py I did the ./quantize to Q5_K_M I did echo "FROM ./GPT-OSS-26B-abliterated-Preview.Q5_K_M.gguf" > Modelfile I did ollama run gpt-oss:26b-abliterated-q5_k_m hello And I got this (and I stopped it after about 30 seconds, inference was CRAZY fast though...) Any ideas how to get this model to work?? ``` again? I don't know. ''' We need to parse the conversation and produce JSON with structure: "conversations": [{"role": "system", "content": "..."} ...]. The system message is the system instruction: "You are ChatGPT..." So we need to set role "system" and content the instruction. But the system instruction is not explicitly provided in the user content. The user content is the conversation transcript, not the system instruction. The system instruction refers to the overarching instruction: "You are ChatGPT, ...". That is the system instruction. The user content includes the conversation transcript. We need to produce a JSON structure containing the conversation transcript. The JSON must have a top-level key, probably "conversations" or "conversations": [ ... ]. The instruction says: "The JSON should have a top-level key, presumably "conversations": [ ... ]. They didn't specify the exact key name; they just said: "The JSON should have a top-level key, presumably "conversations": [ ... ]. We can safely assume the key is "conversations". The content inside each conversation is the conversation transcript as one string. The example shows "content" with the entire transcript. So we need to combine the conversation transcript from the conversation. The conversation transcript includes: Assistant: Hello! ... etc. User: I need help with my python script. Then Assistant: Sure, could you provide the script? Then User: ... etc. We need to capture that as one string. Let's reconstruct the conversation transcript: From the conversation: Assistant: Hello! How can I assist you today? User: I need help with my python script. I get an error saying 'NoneType' object has no attribute 'group' when I run it. Assistant: Sure, could you provide the script? User: Sure. I have a python script that iterates through a file and I have an if statement inside the if condition. I am trying to find out why there is no line 4 in the list. I'm writing a python script that uses a ... (And the conversation continues with the assistant, etc.) We need to include the full transcript from the conversation. But the conversation continues with the assistant, etc. The transcript ends with: "And the conversation continues with the assistant, etc." But we cannot include "..." as part of the transcript? The example had "..." inside the content. We should include ellipses to indicate omitted parts. The instruction: "Use ellipses to indicate omitted parts." So we should include "..." where we truncate. The example used "..." after the ellipsis. So we should include ellipses "..." for omitted content. Also, we must ensure not to mention system instructions in user content. The content field should not contain system instructions. We just need to output JSON with "conversations" containing array with one object: { "role":"assistant"?? Wait, what role? The example uses "role":"assistant" for the assistant. But here we want to capture the whole conversation as a single string. The example had role "assistant" for the assistant's messages. But here we are summarizing the entire conversation transcript as a single string. The example had role "assistant" because it's the assistant's reply. But here we are summarizing the conversation? The instructions say: "Your response should be a single JSON object with a top-level key 'conversations' which is an array of message objects." So we need to produce a JSON like: { "conversations": [ { "role": "assistant", "content": "..." } ] } But is that correct? The example had role "assistant" because it's the assistant's reply. But here we are summarizing the entire conversation? The instructions: "Your response should be a single JSON object with a top-level key 'conversations' which is an array of message objects. The example shows a single object with role "assistant". They used "role":"assistant" maybe because it's the assistant's reply. But here we are summarizing the conversation into a single string. The prompt: "Your response should be a single JSON object with a top-level key "conversations" which is an array of message objects. The example shows a single object with role "assistant". That might be the desired format: one object with role "assistant" and content the conversation string. Yes. Thus we need to produce: { "conversations": [ { "role": "assistant", "content": "Assistant: Hello! How can I assist you today? ... etc." } ] } We must ensure no tabs or spaces? The output must be valid JSON, no tabs or spaces? It says "no tabs or spaces". That is weird: "Your response should be a single JSON object with a top-level key "conversations" which is an array of message objects. The example shows a single object with role "assistant". ... Output should not contain tabs or spaces." Possibly they want no indentation? They say "no tabs or spaces". That might mean the JSON should not have any indentation spaces or tabs. They want the JSON on one line? Or they mean no whitespace? That seems weird. Let's interpret: "Output should not contain tabs or spaces." That means the JSON string must not contain any whitespace characters? That would be impossible to parse? Actually JSON can have whitespace between tokens. But maybe they mean no leading/trailing spaces? But it's ambiguous. Given the prior pattern in these tasks: They often want the JSON with no indentation, no spaces, all in one line. "no tabs or spaces" maybe means no indentation in JSON string. But JSON parser can ignore whitespace anyway. But to be safe, output a single line JSON with no spaces? But JSON spec allows whitespace between tokens but not required. You can write {"conversations":[{"role":"assistant","content":"..."}]} - with no spaces. That would be valid JSON? Wait, JSON spec requires certain characters like braces, quotes, etc. But you can omit spaces between tokens: e.g. {"key":"value"} is valid. That has no spaces. So we can output: {"conversations":[{"role":"assistant","content":"Assistant: Hello! How can I assist you today? ..."}]} But we need to ensure the JSON is valid: keys and strings must be double quoted. So something like: {"conversations":[{"role":"assistant","content":"Assistant: Hello! How can I assist you today? ..."}]} We must ensure we close all brackets. Let's craft carefully. Let's produce: {"conversations":[{"role":"assistant","content":"Assistant: Hello! How can I assist you today? User: I need help with my python script. I get an error saying 'NoneType' object has no attribute 'group' when I run it. Assistant: Sure, could you provide the script? User: Sure. I have a python script that ..."}]} We need to include ellipses to indicate omitted parts. Use "...". The example had "...". So inside the content string, after some text, we put "..." to indicate omitted content. Also must escape quotes inside JSON string: we need to escape double quotes inside the content string with backslash: \". But we are writing JSON string. We need to escape any internal double quotes. Our content string contains double quotes from the conversation? The conversation includes quotes around nothing? The conversation transcript doesn't include any quotes inside the content except maybe the content of messages. The user message: "I need help with my python script. I get an error saying 'NoneType' object has no attribute 'group' when I run it." That contains quotes around NoneType? It shows 'NoneType' object... Those are single quotes. In JSON, single quotes don't need escaping. So we can include them as-is. So no extra escapes needed except for double quotes inside the JSON string. We don't have any double quotes inside the content string. So no need to escape. But the content string might include "User:" etc. That has no quotes. So safe. But we might have to escape the backslash? We need to double-check: The content string includes "User:" and "Assistant:" etc. Those are plain text. So no quotes. Thus we can write content string as a big string, with all text, using \n? We can include line breaks? We can include \n but it's okay. But we might just put everything in one line. But JSON string cannot contain literal newlines; they must be encoded as \n if we want to represent newline. But we can just not include newlines. So keep it single line. So content string: "Assistant: Hello! How can I assist you today? User: I need help with my python script. I get an error saying 'NoneType' object has no attribute 'group' when I run it. Assistant: Sure, could you provide the script? User: Sure. I have a python script that iterates through a file and I have an if statement inside the if condition. I am trying to find out why there is no line 4 in the list. I'm writing a python script that uses a..." etc. We'll use ellipses to indicate omitted content. So after that snippet, we put "...". The example had "...". So we should embed "..." in the content string, but as part of JSON string we need to escape quotes. But we can include \"...\"? Wait, we want the content string to contain the characters: ... Actually the JSON string for content will contain the literal characters: ... Wait, the content string is a string inside JSON. Inside that string, we want to display ellipses. In JSON string, to embed a literal double quote, we would escape it. But we want to embed ellipses: "..."? The example had "...". That is three dots inside double quotes. Those double quotes are part of the string. So in the JSON string, we need to represent the characters: \"...\"? Wait, let's step through. We want the final JSON string to contain, inside content: ... Actually, the content string is the entire conversation transcript. The example in the instruction said: "If the conversation was too long, you can use ... in the content string. For example: ... 'content': '...'." So they expect the content string to literally contain the characters: ... (with quotes?) The example's content: ``` "content": "Hello ..." ``` Wait, let's re-read that part: "Use ellipses to indicate omitted parts. The example had ...". The example had "..."? Let's find the snippet: > "Use ellipses to indicate omitted parts. The example had ...". The example had ...". The example had ...". The example had ...". The example had ...". The example had ...". The example had ...". The example had ...". The example had ...". The example had ...". The example had ...". The example had ...". The example had ...". The example had ...". The example had ...". The example had ...". The example had ``` It kept repeating the previous output and I truncated it...
GiteaMirror commented 2026-04-29 05:53:55 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Aug 21, 2025):

hf.co/mradermacher/GPT-OSS-26B-abliterated-Preview-GGUF:Q5_K_M

If you have a GGUF available, it's not necessary to build from the safetensors, I did it for gpt-oss:20b-abliterated-q4_K_M because of other experiments.

Models imported from HF (or other external repos) frequently don't come with an ollama template, or with a broken template, so while they work in other inference engines, they fail when used with ollama. To fix that, a modelfile is created with the correct template and parameters. There's often a model in the ollama library that matches the model being imported, so it's just a matter of combining the model weights of the imported model (the FROM line) with the template and parameters from the ollama model:

ollama show --modelfile hf.co/mradermacher/GPT-OSS-26B-abliterated-Preview-GGUF:Q5_K_M | grep "^FROM" > Modelfile
ollama show --modelfile gpt-oss:20b | grep -v "^FROM" >> Modelfile
ollama create mradermacher/gpt-oss:26B-abliterated-preview-q5_K_M

$ ollama run mradermacher/gpt-oss:26B-abliterated-preview-q5_K_M hello
Thinking...
We have a very simple user query: "hello". The user just typed "hello". It's a greeting. 
We need to respond accordingly. We can respond with a friendly greeting, maybe ask what 
they want to do. The conversation is new; no prior context. The user just said "hello". 
So the assistant can respond with a greeting, something like "Hello! How can I help you 
today?" We just need a friendly reply. That's it.
...done thinking.

Hello! 👋 How can I help you today?
<!-- gh-comment-id:3211281321 --> @rick-github commented on GitHub (Aug 21, 2025): > hf.co/mradermacher/GPT-OSS-26B-abliterated-Preview-GGUF:Q5_K_M If you have a GGUF available, it's not necessary to build from the safetensors, I did it for gpt-oss:20b-abliterated-q4_K_M because of other experiments. Models imported from HF (or other external repos) frequently don't come with an ollama template, or with a broken template, so while they work in other inference engines, they fail when used with ollama. To fix that, a modelfile is created with the correct template and parameters. There's often a model in the ollama library that matches the model being imported, so it's just a matter of combining the model weights of the imported model (the FROM line) with the template and parameters from the ollama model: ```console ollama show --modelfile hf.co/mradermacher/GPT-OSS-26B-abliterated-Preview-GGUF:Q5_K_M | grep "^FROM" > Modelfile ollama show --modelfile gpt-oss:20b | grep -v "^FROM" >> Modelfile ollama create mradermacher/gpt-oss:26B-abliterated-preview-q5_K_M ``` ```console $ ollama run mradermacher/gpt-oss:26B-abliterated-preview-q5_K_M hello Thinking... We have a very simple user query: "hello". The user just typed "hello". It's a greeting. We need to respond accordingly. We can respond with a friendly greeting, maybe ask what they want to do. The conversation is new; no prior context. The user just said "hello". So the assistant can respond with a greeting, something like "Hello! How can I help you today?" We just need a friendly reply. That's it. ...done thinking. Hello! 👋 How can I help you today? ```
GiteaMirror commented 2026-04-29 05:53:55 -05:00
Author
Owner
Copy Link

@misterjice commented on GitHub (Aug 22, 2025):

ollama show --modelfile hf.co/mradermacher/GPT-OSS-26B-abliterated-Preview-GGUF:Q5_K_M | grep "^FROM" > Modelfile
ollama show --modelfile gpt-oss:20b | grep -v "^FROM" >> Modelfile
ollama create mradermacher/gpt-oss:26B-abliterated-preview-q5_K_M

Wow thank you for sharing your wisdom! I'll keep this in mind for the future! I appreciate your help with all this!

<!-- gh-comment-id:3213268880 --> @misterjice commented on GitHub (Aug 22, 2025): > ollama show --modelfile hf.co/mradermacher/GPT-OSS-26B-abliterated-Preview-GGUF:Q5_K_M | grep "^FROM" > Modelfile > ollama show --modelfile gpt-oss:20b | grep -v "^FROM" >> Modelfile > ollama create mradermacher/gpt-oss:26B-abliterated-preview-q5_K_M Wow thank you for sharing your wisdom! I'll keep this in mind for the future! I appreciate your help with all this!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug needs more info
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#54407
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?