github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-16 13:19:40 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #912] API json output is tearing apart words. #443

New Issue
Closed
opened 2026-04-12 10:05:57 -05:00 by GiteaMirror · 6 comments
GiteaMirror commented 2026-04-12 10:05:57 -05:00
Owner
Copy Link

Originally created by @orkutmuratyilmaz on GitHub (Oct 26, 2023).
Original GitHub issue: https://github.com/ollama/ollama/issues/912

Hello all,

Thanks for this beautiful repo.

I've installed Ollama on my server and I'm making API calls with curl, as seen below:

curl -X POST http://172.16.41.91:11434/api/generate -d '{                                                                                                              ✔ 
  "model": "mistral-openorca",
  "prompt":"What is geography?"  
}'

It works, but on the output, words are divided, like this:


{"model":"mistral-openorca","created_at":"2023-10-26T08:29:58.98891761Z","response":" Ge","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.018778604Z","response":"ography","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.048504687Z","response":" is","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.078094116Z","response":" the","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.107457252Z","response":" study","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.13868812Z","response":" of","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.168320604Z","response":" the","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.197802662Z","response":" Earth","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.227216108Z","response":" and","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.256378119Z","response":" its","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.285954074Z","response":" features","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.315326903Z","response":",","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.344585824Z","response":" including","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.374070348Z","response":" natural","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.403537631Z","response":" phen","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.432798405Z","response":"omena","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.461998564Z","response":" such","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.491377633Z","response":" as","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.520541207Z","response":" land","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.550056497Z","response":"forms","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.579306529Z","response":",","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.608593343Z","response":" climate","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.638077188Z","response":",","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.667181329Z","response":" ecosystem","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.696505473Z","response":"s","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.725893347Z","response":",","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.75503674Z","response":" and","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.784494019Z","response":" human","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.813784319Z","response":"-","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.843240285Z","response":"made","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.872655231Z","response":" environments","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.90192139Z","response":".","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.931271624Z","response":" It","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.96165102Z","response":" en","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.990867302Z","response":"compass","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.020237128Z","response":"es","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.049652238Z","response":" various","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.079103562Z","response":" fields","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.108553919Z","response":" like","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.138109986Z","response":" physical","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.167264145Z","response":" ge","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.196610395Z","response":"ography","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.225985474Z","response":",","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.255268799Z","response":" human","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.28479104Z","response":" ge","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.313930302Z","response":"ography","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.343539271Z","response":",","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.372764036Z","response":" and","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.402296814Z","response":" environmental","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.431535875Z","response":" ge","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.461012368Z","response":"ography","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.490677883Z","response":".","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.520257262Z","response":" The","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.549584308Z","response":" goal","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.579018043Z","response":" of","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.608334651Z","response":" ge","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.638991298Z","response":"ography","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.668736969Z","response":" is","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.697926723Z","response":" to","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.727354031Z","response":" understand","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.756560779Z","response":" the","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.787057039Z","response":" inter","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.8165341Z","response":"play","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.846016458Z","response":" between","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.875269529Z","response":" the","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.904839891Z","response":" natural","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.934234445Z","response":" world","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.963685453Z","response":",","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.994070822Z","response":" human","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:01.023675167Z","response":" societies","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:01.05300978Z","response":",","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:01.082607386Z","response":" and","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:01.112238613Z","response":" their","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:01.141485031Z","response":" interactions","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:01.171112532Z","response":".","done":false}
{"model":"mistral-openorca","created_at":"2023-10-26T08:30:01.201667791Z","response":"","done":true,"context":[28705,13,28789,28766,321,28730,2521,28766,28767,1838,13,3195,349,2970,5064,28804,28789,28766,321,28730,416,28766,28767,13,28789,28766,321,28730,2521,28766,28767,489,11143,13,2404,5064,349,272,3881,302,272,8599,304,871,4190,28725,2490,4229,11315,26504,1259,390,2533,14367,28725,11259,28725,27374,28713,28725,304,2930,28733,15248,20031,28723,661,481,24975,274,4118,5080,737,5277,2970,5064,28725,2930,2970,5064,28725,304,12507,2970,5064,28723,415,5541,302,2970,5064,349,298,2380,272,791,1674,1444,272,4229,1526,28725,2930,25342,28725,304,652,14983,28723],"total_duration":7012283086,"load_duration":4646474654,"prompt_eval_count":35,"prompt_eval_duration":150529000,"eval_count":75,"eval_duration":2088593000}

What could be the reason for this?

Best,
Orkut

Originally created by @orkutmuratyilmaz on GitHub (Oct 26, 2023). Original GitHub issue: https://github.com/ollama/ollama/issues/912 Hello all, Thanks for this beautiful repo. I've installed Ollama on my server and I'm making API calls with curl, as seen below: ``` curl -X POST http://172.16.41.91:11434/api/generate -d '{  ✔ "model": "mistral-openorca", "prompt":"What is geography?" }' ``` It works, but on the output, words are divided, like this: ``` {"model":"mistral-openorca","created_at":"2023-10-26T08:29:58.98891761Z","response":" Ge","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.018778604Z","response":"ography","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.048504687Z","response":" is","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.078094116Z","response":" the","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.107457252Z","response":" study","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.13868812Z","response":" of","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.168320604Z","response":" the","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.197802662Z","response":" Earth","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.227216108Z","response":" and","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.256378119Z","response":" its","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.285954074Z","response":" features","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.315326903Z","response":",","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.344585824Z","response":" including","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.374070348Z","response":" natural","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.403537631Z","response":" phen","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.432798405Z","response":"omena","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.461998564Z","response":" such","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.491377633Z","response":" as","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.520541207Z","response":" land","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.550056497Z","response":"forms","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.579306529Z","response":",","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.608593343Z","response":" climate","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.638077188Z","response":",","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.667181329Z","response":" ecosystem","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.696505473Z","response":"s","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.725893347Z","response":",","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.75503674Z","response":" and","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.784494019Z","response":" human","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.813784319Z","response":"-","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.843240285Z","response":"made","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.872655231Z","response":" environments","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.90192139Z","response":".","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.931271624Z","response":" It","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.96165102Z","response":" en","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:29:59.990867302Z","response":"compass","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.020237128Z","response":"es","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.049652238Z","response":" various","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.079103562Z","response":" fields","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.108553919Z","response":" like","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.138109986Z","response":" physical","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.167264145Z","response":" ge","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.196610395Z","response":"ography","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.225985474Z","response":",","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.255268799Z","response":" human","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.28479104Z","response":" ge","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.313930302Z","response":"ography","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.343539271Z","response":",","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.372764036Z","response":" and","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.402296814Z","response":" environmental","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.431535875Z","response":" ge","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.461012368Z","response":"ography","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.490677883Z","response":".","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.520257262Z","response":" The","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.549584308Z","response":" goal","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.579018043Z","response":" of","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.608334651Z","response":" ge","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.638991298Z","response":"ography","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.668736969Z","response":" is","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.697926723Z","response":" to","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.727354031Z","response":" understand","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.756560779Z","response":" the","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.787057039Z","response":" inter","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.8165341Z","response":"play","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.846016458Z","response":" between","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.875269529Z","response":" the","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.904839891Z","response":" natural","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.934234445Z","response":" world","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.963685453Z","response":",","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:00.994070822Z","response":" human","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:01.023675167Z","response":" societies","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:01.05300978Z","response":",","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:01.082607386Z","response":" and","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:01.112238613Z","response":" their","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:01.141485031Z","response":" interactions","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:01.171112532Z","response":".","done":false} {"model":"mistral-openorca","created_at":"2023-10-26T08:30:01.201667791Z","response":"","done":true,"context":[28705,13,28789,28766,321,28730,2521,28766,28767,1838,13,3195,349,2970,5064,28804,28789,28766,321,28730,416,28766,28767,13,28789,28766,321,28730,2521,28766,28767,489,11143,13,2404,5064,349,272,3881,302,272,8599,304,871,4190,28725,2490,4229,11315,26504,1259,390,2533,14367,28725,11259,28725,27374,28713,28725,304,2930,28733,15248,20031,28723,661,481,24975,274,4118,5080,737,5277,2970,5064,28725,2930,2970,5064,28725,304,12507,2970,5064,28723,415,5541,302,2970,5064,349,298,2380,272,791,1674,1444,272,4229,1526,28725,2930,25342,28725,304,652,14983,28723],"total_duration":7012283086,"load_duration":4646474654,"prompt_eval_count":35,"prompt_eval_duration":150529000,"eval_count":75,"eval_duration":2088593000} ``` What could be the reason for this? Best, Orkut
GiteaMirror commented 2026-04-12 10:05:58 -05:00
Author
Owner
Copy Link

@orkutmuratyilmaz commented on GitHub (Oct 26, 2023):

I've also tried with llama2 and mistral models, the output is again divided words.

<!-- gh-comment-id:1780869428 --> @orkutmuratyilmaz commented on GitHub (Oct 26, 2023): I've also tried with llama2 and mistral models, the output is again divided words.
GiteaMirror commented 2026-04-12 10:05:59 -05:00
Author
Owner
Copy Link

@BruceMacD commented on GitHub (Oct 26, 2023):

Hi @orkutmuratyilmaz thanks for opening the issue.

What's going on here is that Ollama streams back the tokens as they are generated rather than the full words. Tokens from language models aren't always words, they can be parts of words or just letter groups based on patterns the model learned from data. There isn't a way to configure slicing the output by words rather than tokens at the moment.

<!-- gh-comment-id:1781257386 --> @BruceMacD commented on GitHub (Oct 26, 2023): Hi @orkutmuratyilmaz thanks for opening the issue. What's going on here is that Ollama streams back the tokens as they are generated rather than the full words. Tokens from language models aren't always words, they can be parts of words or just letter groups based on patterns the model learned from data. There isn't a way to configure slicing the output by words rather than tokens at the moment.
GiteaMirror commented 2026-04-12 10:05:59 -05:00
Author
Owner
Copy Link

@orkutmuratyilmaz commented on GitHub (Oct 26, 2023):

@BruceMacD thanks for your explanation. I'm also using Ollama on cli, but the output is not sliced on the cli. What makes the difference for the json output?

<!-- gh-comment-id:1781522497 --> @orkutmuratyilmaz commented on GitHub (Oct 26, 2023): @BruceMacD thanks for your explanation. I'm also using Ollama on cli, but the output is not sliced on the cli. What makes the difference for the json output?
GiteaMirror commented 2026-04-12 10:06:02 -05:00
Author
Owner
Copy Link

@BruceMacD commented on GitHub (Oct 26, 2023):

It actually is sliced on the CLI too, just hard to tell becuase usually responses start coming quickly once the model is warmed up. You can see the actual logic here:

fn := func(response api.GenerateResponse) error {
		if !spinner.IsFinished() {
			spinner.Finish()
		}

		latest = response

		if wordWrap {
			for _, ch := range response.Response {
				if currentLineLength+1 > termWidth-5 {
					// backtrack the length of the last word and clear to the end of the line
					fmt.Printf("\x1b[%dD\x1b[K\n", len(wordBuffer))
					fmt.Printf("%s%c", wordBuffer, ch)
					currentLineLength = len(wordBuffer) + 1
				} else {
					fmt.Print(string(ch))
					currentLineLength += 1

					switch ch {
					case ' ':
						wordBuffer = ""
					case '\n':
						currentLineLength = 0
					default:
						wordBuffer += string(ch)
					}
				}
			}
		} else {
			fmt.Print(response.Response)
		}

		return nil
	}
<!-- gh-comment-id:1781596528 --> @BruceMacD commented on GitHub (Oct 26, 2023): It actually is sliced on the CLI too, just hard to tell becuase usually responses start coming quickly once the model is warmed up. You can see the actual logic here: ``` fn := func(response api.GenerateResponse) error { if !spinner.IsFinished() { spinner.Finish() } latest = response if wordWrap { for _, ch := range response.Response { if currentLineLength+1 > termWidth-5 { // backtrack the length of the last word and clear to the end of the line fmt.Printf("\x1b[%dD\x1b[K\n", len(wordBuffer)) fmt.Printf("%s%c", wordBuffer, ch) currentLineLength = len(wordBuffer) + 1 } else { fmt.Print(string(ch)) currentLineLength += 1 switch ch { case ' ': wordBuffer = "" case '\n': currentLineLength = 0 default: wordBuffer += string(ch) } } } } else { fmt.Print(response.Response) } return nil } ```
GiteaMirror commented 2026-04-12 10:06:03 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (Oct 26, 2023):

@orkutmuratyilmaz you probably just want to turn streaming off. You can do this with:

% curl -X POST http://localhost:11434/api/generate -d '{
  "model": "mistral-openorca",
  "prompt":"What is geography?",
  "stream":false
}'

That will turn off streaming and return all of the tokens together as one big response. It will take a little longer to get some output, but you won't have to glue all of the tokens back together.

I'm going to go ahead and close the issue, but feel free to re-open it or reach out on the discord if you're having trouble!

<!-- gh-comment-id:1781817962 --> @pdevine commented on GitHub (Oct 26, 2023): @orkutmuratyilmaz you probably just want to turn streaming off. You can do this with: ``` % curl -X POST http://localhost:11434/api/generate -d '{ "model": "mistral-openorca", "prompt":"What is geography?", "stream":false }' ``` That will turn off streaming and return all of the tokens together as one big response. It will take a little longer to get some output, but you won't have to glue all of the tokens back together. I'm going to go ahead and close the issue, but feel free to re-open it or reach out on the discord if you're having trouble!
GiteaMirror commented 2026-04-12 10:06:04 -05:00
Author
Owner
Copy Link

@orkutmuratyilmaz commented on GitHub (Oct 27, 2023):

@BruceMacD @pdevine thanks for beautiful explanations and your guidance too. Turning the streaming off worked well.

I love free software<3

<!-- gh-comment-id:1783271605 --> @orkutmuratyilmaz commented on GitHub (Oct 27, 2023): @BruceMacD @pdevine thanks for beautiful explanations and your guidance too. Turning the streaming off worked well. I love free software<3
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#443
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?