github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[PR #8672] [CLOSED] openai: set context_length size #59442

New Issue
Closed
opened 2026-04-29 14:22:42 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-04-29 14:22:42 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/ollama/ollama/pull/8672
Author: @ParthSareen
Created: 1/29/2025
Status: ❌ Closed

Base: main ← Head: parth/set-context-size-openai

📝 Commits (10+)

📊 Changes

3 files changed (+111 additions, -43 deletions)

View changed files

📝 docs/openai.md (+40 -24)
📝 openai/openai.go (+18 -5)
📝 openai/openai_test.go (+53 -14)

📄 Description

Important

Closed in favor of #8938

To change context window size using the openai supported endpoint:

curl

curl http://localhost:11434/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "llama3.2",
        "messages": [{"role": "user", "content": "Say this is a test"}],
        "context_length": 4096
    }'

OpenAI python library

  • OpenAI python library does not support setting context length, however this can be set for Ollama through the extra_body parameter
completion = client.chat.completions.create(
    model="llama3.1:8b",
    messages=[{"role": "user", "content": "Say this is a test"}],
    extra_body={"context_length": 4096},
)

OpenAI JavaScript library

  • OpenAI JavaScript library does not support setting context length, however this can be set for Ollama by passing context_length directly with a @ts-expect-error as an undocumented parameter in the OpenAI JavaScript library. See documentation here
const chatCompletion = await openai.chat.completions.create({
    messages: [{ role: 'user', content: 'Say this is a test' }],
    model: 'llama3.2',
    // @ts-expect-error context_length is an additional parameter 
    context_length: 4096,
})

This PR also adds support for max_completion_tokens to be on parity with OpenAI

closes: #6504, #8794, #5356

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/ollama/ollama/pull/8672 **Author:** [@ParthSareen](https://github.com/ParthSareen) **Created:** 1/29/2025 **Status:** ❌ Closed **Base:** `main` ← **Head:** `parth/set-context-size-openai` --- ### 📝 Commits (10+) - [`35e97db`](https://github.com/ollama/ollama/commit/35e97db03b07bba7151c0c09ff3f7faaabd2c019) set num_ctx through extra body - [`6de3227`](https://github.com/ollama/ollama/commit/6de32278417f05b41533f3b87890ee6fc783cb14) Cleanup api - [`5c2f35d`](https://github.com/ollama/ollama/commit/5c2f35d846a462c87155be512e652d774ddddb62) Add tests - [`16abd18`](https://github.com/ollama/ollama/commit/16abd181a91da9df62501503fd3b61449f849904) remove context shifting with max tokens and update docs - [`8c231b0`](https://github.com/ollama/ollama/commit/8c231b0826fed7ab6f27bbae10ea89fb3afe6010) Update openai/openai.go - [`4982089`](https://github.com/ollama/ollama/commit/4982089c8412a9a9b31f9e1a2f531bcd8ffa3884) Fix formatting - [`75f88e7`](https://github.com/ollama/ollama/commit/75f88e7aac09e1aa2c3226b69559acaabe779704) Update docs - [`53d2cf3`](https://github.com/ollama/ollama/commit/53d2cf37d2791eb7d5a7451e0d1d320af21cd9ab) update docs - [`61a5254`](https://github.com/ollama/ollama/commit/61a5254115d7f9684bfe57ccda82a616c02ad07a) context_window and addressing comments - [`b4de2e9`](https://github.com/ollama/ollama/commit/b4de2e91892772fc55257750e061fcc4ee7e3dc6) change name to context_length ### 📊 Changes **3 files changed** (+111 additions, -43 deletions) <details> <summary>View changed files</summary> 📝 `docs/openai.md` (+40 -24) 📝 `openai/openai.go` (+18 -5) 📝 `openai/openai_test.go` (+53 -14) </details> ### 📄 Description > [!IMPORTANT] > Closed in favor of #8938 To change context window size using the openai supported endpoint: #### `curl` ```shell curl http://localhost:11434/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "llama3.2", "messages": [{"role": "user", "content": "Say this is a test"}], "context_length": 4096 }' ``` #### OpenAI python library - OpenAI python library does not support setting context length, however this can be set for Ollama through the `extra_body` parameter ```py completion = client.chat.completions.create( model="llama3.1:8b", messages=[{"role": "user", "content": "Say this is a test"}], extra_body={"context_length": 4096}, ) ``` #### OpenAI JavaScript library - OpenAI JavaScript library does not support setting context length, however this can be set for Ollama by passing `context_length` directly with a `@ts-expect-error` as an undocumented parameter in the OpenAI JavaScript library. [See documentation here](https://github.com/openai/openai-node?tab=readme-ov-file#making-customundocumented-requests) ```ts const chatCompletion = await openai.chat.completions.create({ messages: [{ role: 'user', content: 'Say this is a test' }], model: 'llama3.2', // @ts-expect-error context_length is an additional parameter context_length: 4096, }) ``` This PR also adds support for `max_completion_tokens` to be on parity with OpenAI closes: #6504, #8794, #5356 --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-04-29 14:22:42 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#59442
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?