github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-08 17:49:24 -05:00
Code Issues 14.5k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #6877] OpenAI o1-like Chain-of-thought (CoT) inference workflow #66384

New Issue
Open
opened 2026-05-04 03:24:02 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-05-04 03:24:02 -05:00
Owner
Copy Link

Originally created by @kozuch on GitHub (Sep 19, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/6877

Well, I am surprised that the "main" and "great" new feature of the new OpenAI o1 model is actually doing say "more sophisticated" inference workflow while employing something like Chain-of-thought process. Basically I understand it that even a "dumb" model can perform much better when it "thinks more" during inference. The great news they are telling us is that by "thinking more" you can get smarter, which is probably very true also for humans.

The o1 model is probably trained to come up with its own CoT workflow for any given prompt, but I think it could be interesting to try to even hardcode some kind of workflow which any standard LLM model may try to follow during inference. Basically let the model analyze the prompt from various perspectives first and then try to judge on what type of "inference workflow" it should employ.

The hardcoded workflow could look like this:

  1. Prompt is submitted to the model.
  2. The model asks itself couple of hard-coded questions about the prompt, maybe:
  • is that some light conversation (needing soft-skills like empathy etc)
  • does it look like a science problem (math, physics etc.)
  • can I break the prompt down to subtasks - if yes, the workflow will feed each subtask into the model separately, then combine the result etc.
  • is the problem easy/hard
  • do I have all information I need (do I need to ask the user for further input/clarification)
  1. The workflow would run, maybe in multiple iterations on various its levels, maybe trying to fit some "quality checks" for the answer
  2. The output is presented to the user (the "hidden" thinking may be optionally viewed by user)

Anyone having the same feelings as I do about the CoT thing? Looks like even a hard-coded process may give some interesting results.

Originally created by @kozuch on GitHub (Sep 19, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/6877 Well, I am surprised that the "main" and "great" new feature of the new OpenAI o1 model is actually doing say "more sophisticated" inference workflow while employing something like Chain-of-thought process. Basically I understand it that even a "dumb" model can perform much better when it "thinks more" during inference. The great news they are telling us is that by "thinking more" you can get smarter, which is probably very true also for humans. The o1 model is probably trained to come up with its own CoT workflow for any given prompt, but I think it could be interesting to try to even hardcode some kind of workflow which any standard LLM model may try to follow during inference. Basically let the model analyze the prompt from various perspectives first and then try to judge on what type of "inference workflow" it should employ. The hardcoded workflow could look like this: 1. Prompt is submitted to the model. 2. The model asks itself couple of hard-coded questions about the prompt, maybe: - is that some light conversation (needing soft-skills like empathy etc) - does it look like a science problem (math, physics etc.) - can I break the prompt down to subtasks - if yes, the workflow will feed each subtask into the model separately, then combine the result etc. - is the problem easy/hard - do I have all information I need (do I need to ask the user for further input/clarification) 3. The workflow would run, maybe in multiple iterations on various its levels, maybe trying to fit some "quality checks" for the answer 4. The output is presented to the user (the "hidden" thinking may be optionally viewed by user) Anyone having the same feelings as I do about the CoT thing? Looks like even a hard-coded process may give some interesting results.
GiteaMirror added the feature request label 2026-05-04 03:24:02 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#66384
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?