[GH-ISSUE #7056] Undefined variable in this code file: convert/tokenizer_spm.go #4479

Closed
opened 2026-04-12 15:24:26 -05:00 by GiteaMirror · 2 comments
Owner

Originally created by @vignesh1507 on GitHub (Oct 1, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/7056

Originally assigned to: @dhiltgen on GitHub.

What is the issue?

The code doesn't currently define tokenTypeUserDefined, which will cause a compilation error.

How to fix?
Add a constant declaration to fix the issue, for example:

const tokenTypeUserDefined = int32(1)

OS

macOS

GPU

Apple

CPU

Apple

Ollama version

3.1

Originally created by @vignesh1507 on GitHub (Oct 1, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/7056 Originally assigned to: @dhiltgen on GitHub. ### What is the issue? The code doesn't currently define tokenTypeUserDefined, which will cause a compilation error. How to fix? Add a constant declaration to fix the issue, for example: `const tokenTypeUserDefined = int32(1) ` ### OS macOS ### GPU Apple ### CPU Apple ### Ollama version 3.1
GiteaMirror added the question label 2026-04-12 15:24:26 -05:00
Author
Owner

@dhiltgen commented on GitHub (Oct 1, 2024):

It's defined here - https://github.com/ollama/ollama/blob/main/convert/tokenizer.go#L22

Can you share the compiler error you're seeing?

<!-- gh-comment-id:2386385255 --> @dhiltgen commented on GitHub (Oct 1, 2024): It's defined here - https://github.com/ollama/ollama/blob/main/convert/tokenizer.go#L22 Can you share the compiler error you're seeing?
Author
Owner

@vignesh1507 commented on GitHub (Oct 1, 2024):

Thx for pointing it out, i thought the variable isn't defined so it might throw a compiler error(hypothetical).

<!-- gh-comment-id:2386522036 --> @vignesh1507 commented on GitHub (Oct 1, 2024): Thx for pointing it out, i thought the variable isn't defined so it might throw a compiler error(hypothetical).
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#4479