github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-08 08:59:13 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #7993] Structured generation cannot handle self referencing (recursion) #5114

New Issue
Open
opened 2026-04-12 16:12:42 -05:00 by GiteaMirror · 4 comments
GiteaMirror commented 2026-04-12 16:12:42 -05:00
Owner
Copy Link

Originally created by @CakeCrusher on GitHub (Dec 8, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/7993

Originally assigned to: @ParthSareen on GitHub.

What is the issue?

Ollama structured genereation cannot handle self referencing recursion

import json
from pydantic import BaseModel, Field
from typing import Optional

class Dossier(BaseModel):
    """Build a profile for the user"""
    name: str = Field(..., description="The name of the user")
    age: int = Field(..., description="The age of the user")
    friends: list["Dossier"] = []

print(json.dumps(Dossier.model_json_schema(), indent=2))

from ollama import chat

response = chat(
    messages=[
        {
            'role': 'user',
            'content': 'Hello my name is Tom I am 13 years old, my friend Bob is 14 years old'
        }
    ],
    model='llama3.1:8b-instruct-q2_K',
    format=Dossier.model_json_schema()
)
dossier = Dossier.model_validate_json(response.message.content)
print(dossier.model_dump_json(indent=2))

Output:

{
  "$defs": {
    "Dossier": {
      "description": "Build a profile for the user",
      "properties": {
        "name": {
          "description": "The name of the user",
          "title": "Name",
          "type": "string"
        },
        "age": {
          "description": "The age of the user",
          "title": "Age",
          "type": "integer"
        },
        "friends": {
          "default": [],
          "items": {
            "$ref": "#/$defs/Dossier"
          },
          "title": "Friends",
          "type": "array"
        }
      },
      "required": [
        "name",
        "age"
      ],
      "title": "Dossier",
      "type": "object"
    }
  },
  "$ref": "#/$defs/Dossier"
}

---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[54], [line 25](vscode-notebook-cell:?execution_count=54&line=25)
     [13](vscode-notebook-cell:?execution_count=54&line=13) from ollama import chat
     [15](vscode-notebook-cell:?execution_count=54&line=15) response = chat(
     [16](vscode-notebook-cell:?execution_count=54&line=16)     messages=[
     [17](vscode-notebook-cell:?execution_count=54&line=17)         {
   (...)
     [23](vscode-notebook-cell:?execution_count=54&line=23)     format=Dossier.model_json_schema()
     [24](vscode-notebook-cell:?execution_count=54&line=24) )
---> [25](vscode-notebook-cell:?execution_count=54&line=25) dossier = Dossier.model_validate_json(response.message.content)
     [26](vscode-notebook-cell:?execution_count=54&line=26) print(dossier.model_dump_json(indent=2))

File c:\Notes\ollama\.venv\lib\site-packages\pydantic\main.py:656, in BaseModel.model_validate_json(cls, json_data, strict, context)
    [654](file:///C:/Notes/ollama/.venv/lib/site-packages/pydantic/main.py:654) # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
    [655](file:///C:/Notes/ollama/.venv/lib/site-packages/pydantic/main.py:655) __tracebackhide__ = True
--> [656](file:///C:/Notes/ollama/.venv/lib/site-packages/pydantic/main.py:656) return cls.__pydantic_validator__.validate_json(json_data, strict=strict, context=context)

ValidationError: 1 validation error for Dossier
  Invalid JSON: expected value at line 1 column 1 [type=json_invalid, input_value="So you're 13 and your fr...m outside of class too?", input_type=str]
    For further information visit https://errors.pydantic.dev/2.10/v/json_invalid

When destructured there is no problem

import json
from pydantic import BaseModel, Field
from typing import Optional

class DossierInner(BaseModel):
    """Build a profile for the user"""
    name: str = Field(..., description="The name of the user")
    age: int = Field(..., description="The age of the user")

class Dossier(BaseModel):
    """Build a profile for the user"""
    name: str = Field(..., description="The name of the user")
    age: int = Field(..., description="The age of the user")
    friends: list[DossierInner] = []

print(json.dumps(Dossier.model_json_schema(), indent=2))

from ollama import chat

response = chat(
    messages=[
        {
            'role': 'user',
            'content': 'Hello my name is Tom I am 13 years old, my friend Bob is 14 years old'
        }
    ],
    model='llama3.1:8b-instruct-q2_K',
    format=Dossier.model_json_schema()
)
dossier = Dossier.model_validate_json(response.message.content)
print(dossier.model_dump_json(indent=2))

Output:

{
  "$defs": {
    "DossierInner": {
      "description": "Build a profile for the user",
      "properties": {
        "name": {
          "description": "The name of the user",
          "title": "Name",
          "type": "string"
        },
        "age": {
          "description": "The age of the user",
          "title": "Age",
          "type": "integer"
        }
      },
      "required": [
        "name",
        "age"
      ],
      "title": "DossierInner",
      "type": "object"
    }
  },
  "description": "Build a profile for the user",
  "properties": {
    "name": {
      "description": "The name of the user",
      "title": "Name",
      "type": "string"
    },
    "age": {
      "description": "The age of the user",
      "title": "Age",
      "type": "integer"
    },
    "friends": {
      "default": [],
      "items": {
        "$ref": "#/$defs/DossierInner"
      },
      "title": "Friends",
      "type": "array"
    }
  },
  "required": [
    "name",
    "age"
  ],
  "title": "Dossier",
  "type": "object"
}
{
  "name": "Tom",
  "age": -1,
  "friends": [
    {
      "name": "Bob",
      "age": -2
    }
  ]
}

OS

Windows

GPU

Nvidia

CPU

AMD

Ollama version

0.5.1

Originally created by @CakeCrusher on GitHub (Dec 8, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/7993 Originally assigned to: @ParthSareen on GitHub. ### What is the issue? Ollama structured genereation cannot handle self referencing recursion ```py import json from pydantic import BaseModel, Field from typing import Optional class Dossier(BaseModel): """Build a profile for the user""" name: str = Field(..., description="The name of the user") age: int = Field(..., description="The age of the user") friends: list["Dossier"] = [] print(json.dumps(Dossier.model_json_schema(), indent=2)) from ollama import chat response = chat( messages=[ { 'role': 'user', 'content': 'Hello my name is Tom I am 13 years old, my friend Bob is 14 years old' } ], model='llama3.1:8b-instruct-q2_K', format=Dossier.model_json_schema() ) dossier = Dossier.model_validate_json(response.message.content) print(dossier.model_dump_json(indent=2)) ``` Output: ``` { "$defs": { "Dossier": { "description": "Build a profile for the user", "properties": { "name": { "description": "The name of the user", "title": "Name", "type": "string" }, "age": { "description": "The age of the user", "title": "Age", "type": "integer" }, "friends": { "default": [], "items": { "$ref": "#/$defs/Dossier" }, "title": "Friends", "type": "array" } }, "required": [ "name", "age" ], "title": "Dossier", "type": "object" } }, "$ref": "#/$defs/Dossier" } ``` ``` --------------------------------------------------------------------------- ValidationError Traceback (most recent call last) Cell In[54], [line 25](vscode-notebook-cell:?execution_count=54&line=25) [13](vscode-notebook-cell:?execution_count=54&line=13) from ollama import chat [15](vscode-notebook-cell:?execution_count=54&line=15) response = chat( [16](vscode-notebook-cell:?execution_count=54&line=16) messages=[ [17](vscode-notebook-cell:?execution_count=54&line=17) { (...) [23](vscode-notebook-cell:?execution_count=54&line=23) format=Dossier.model_json_schema() [24](vscode-notebook-cell:?execution_count=54&line=24) ) ---> [25](vscode-notebook-cell:?execution_count=54&line=25) dossier = Dossier.model_validate_json(response.message.content) [26](vscode-notebook-cell:?execution_count=54&line=26) print(dossier.model_dump_json(indent=2)) File c:\Notes\ollama\.venv\lib\site-packages\pydantic\main.py:656, in BaseModel.model_validate_json(cls, json_data, strict, context) [654](file:///C:/Notes/ollama/.venv/lib/site-packages/pydantic/main.py:654) # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks [655](file:///C:/Notes/ollama/.venv/lib/site-packages/pydantic/main.py:655) __tracebackhide__ = True --> [656](file:///C:/Notes/ollama/.venv/lib/site-packages/pydantic/main.py:656) return cls.__pydantic_validator__.validate_json(json_data, strict=strict, context=context) ValidationError: 1 validation error for Dossier Invalid JSON: expected value at line 1 column 1 [type=json_invalid, input_value="So you're 13 and your fr...m outside of class too?", input_type=str] For further information visit https://errors.pydantic.dev/2.10/v/json_invalid ``` When destructured there is no problem ```py import json from pydantic import BaseModel, Field from typing import Optional class DossierInner(BaseModel): """Build a profile for the user""" name: str = Field(..., description="The name of the user") age: int = Field(..., description="The age of the user") class Dossier(BaseModel): """Build a profile for the user""" name: str = Field(..., description="The name of the user") age: int = Field(..., description="The age of the user") friends: list[DossierInner] = [] print(json.dumps(Dossier.model_json_schema(), indent=2)) from ollama import chat response = chat( messages=[ { 'role': 'user', 'content': 'Hello my name is Tom I am 13 years old, my friend Bob is 14 years old' } ], model='llama3.1:8b-instruct-q2_K', format=Dossier.model_json_schema() ) dossier = Dossier.model_validate_json(response.message.content) print(dossier.model_dump_json(indent=2)) ``` Output: ``` { "$defs": { "DossierInner": { "description": "Build a profile for the user", "properties": { "name": { "description": "The name of the user", "title": "Name", "type": "string" }, "age": { "description": "The age of the user", "title": "Age", "type": "integer" } }, "required": [ "name", "age" ], "title": "DossierInner", "type": "object" } }, "description": "Build a profile for the user", "properties": { "name": { "description": "The name of the user", "title": "Name", "type": "string" }, "age": { "description": "The age of the user", "title": "Age", "type": "integer" }, "friends": { "default": [], "items": { "$ref": "#/$defs/DossierInner" }, "title": "Friends", "type": "array" } }, "required": [ "name", "age" ], "title": "Dossier", "type": "object" } { "name": "Tom", "age": -1, "friends": [ { "name": "Bob", "age": -2 } ] } ``` ### OS Windows ### GPU Nvidia ### CPU AMD ### Ollama version 0.5.1
GiteaMirror added the bug label 2026-04-12 16:12:42 -05:00
GiteaMirror commented 2026-04-12 16:12:43 -05:00
Author
Owner
Copy Link

@zachwalton commented on GitHub (Jan 29, 2025):

Looks like it's still an issue on HEAD, though the linked PR was closed. Not sure if the refactor in the linked PR was supposed to fix this. Removing $ref makes the schema pass.

➜  git git clone https://github.com/ollama/ollama.git
Cloning into 'ollama'...
remote: Enumerating objects: 29806, done.
remote: Counting objects: 100% (324/324), done.
remote: Compressing objects: 100% (197/197), done.
remote: Total 29806 (delta 230), reused 127 (delta 127), pack-reused 29482 (from 4)
Receiving objects: 100% (29806/29806), 30.73 MiB | 23.80 MiB/s, done.
Resolving deltas: 100% (19150/19150), done.
➜  git cd ollama 
➜  ollama git:(main) go run main.go serve
# command-line-arguments

➜  devoid git:(main) ✗ go run main.go --project-path /tmp/foo "a sports cli"
...
 ERRO got an error during inference error="invalid JSON schema in format"

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://example.com/codebase-schema.json",
  "title": "Multi-Language Codebase Graph with Expanded Callees",
  "description": "A recursive structure representing a directory tree with call graph links for arbitrary languages, supporting access to constants, variables, and properties",
  "type": "object",
  "properties": {
    "language": {
      "type": "string",
      "description": "The programming language of the codebase (e.g., 'go', 'python', 'java', etc.)",
      "examples": ["go", "python", "java", "cpp", "typescript"]
    },
    "name": {
      "type": "string",
      "description": "The name of the directory, file, namespace, class, function, constant, variable, or property"
    },
    "type": {
      "type": "string",
      "enum": ["Dir", "File", "Namespace", "Class", "Interface", "Function", "Constant", "Variable", "Property"],
      "description": "The type of node in the structure"
    },
    "children": {
      "type": "array",
      "description": "Sub-nodes within this node",
      "items": { "$ref": "#" }
    },
    "callers": {
      "type": "array",
      "description": "Other symbols that reference this node (functions, properties, constants, etc.)",
      "items": { "type": "string" }
    },
    "callees": {
      "type": "array",
      "description": "Symbols accessed by this node (could be functions, constants, properties, etc.)",
      "items": { "type": "string" }
    },
    "metadata": {
      "type": "object",
      "description": "Optional metadata specific to the language (e.g., access modifiers, visibility, annotations)",
      "additionalProperties": true
    }
  },
  "required": ["name", "type"]
}
<!-- gh-comment-id:2621920426 --> @zachwalton commented on GitHub (Jan 29, 2025): Looks like it's still an issue on HEAD, though the linked PR was closed. Not sure if the refactor in the linked PR was supposed to fix this. Removing `$ref` makes the schema pass. ``` ➜ git git clone https://github.com/ollama/ollama.git Cloning into 'ollama'... remote: Enumerating objects: 29806, done. remote: Counting objects: 100% (324/324), done. remote: Compressing objects: 100% (197/197), done. remote: Total 29806 (delta 230), reused 127 (delta 127), pack-reused 29482 (from 4) Receiving objects: 100% (29806/29806), 30.73 MiB | 23.80 MiB/s, done. Resolving deltas: 100% (19150/19150), done. ➜ git cd ollama ➜ ollama git:(main) go run main.go serve # command-line-arguments ``` ``` ➜ devoid git:(main) ✗ go run main.go --project-path /tmp/foo "a sports cli" ... ERRO got an error during inference error="invalid JSON schema in format" ``` ``` { "$schema": "https://json-schema.org/draft/2020-12/schema", "$id": "https://example.com/codebase-schema.json", "title": "Multi-Language Codebase Graph with Expanded Callees", "description": "A recursive structure representing a directory tree with call graph links for arbitrary languages, supporting access to constants, variables, and properties", "type": "object", "properties": { "language": { "type": "string", "description": "The programming language of the codebase (e.g., 'go', 'python', 'java', etc.)", "examples": ["go", "python", "java", "cpp", "typescript"] }, "name": { "type": "string", "description": "The name of the directory, file, namespace, class, function, constant, variable, or property" }, "type": { "type": "string", "enum": ["Dir", "File", "Namespace", "Class", "Interface", "Function", "Constant", "Variable", "Property"], "description": "The type of node in the structure" }, "children": { "type": "array", "description": "Sub-nodes within this node", "items": { "$ref": "#" } }, "callers": { "type": "array", "description": "Other symbols that reference this node (functions, properties, constants, etc.)", "items": { "type": "string" } }, "callees": { "type": "array", "description": "Symbols accessed by this node (could be functions, constants, properties, etc.)", "items": { "type": "string" } }, "metadata": { "type": "object", "description": "Optional metadata specific to the language (e.g., access modifiers, visibility, annotations)", "additionalProperties": true } }, "required": ["name", "type"] } ```
GiteaMirror commented 2026-04-12 16:12:44 -05:00
Author
Owner
Copy Link

@ParthSareen commented on GitHub (Jan 29, 2025):

Yeah, haven't updated it on purpose. I'm writing a new sampler at the moment and am not sure if it is worthwhile to add this as there might be an accuracy hit. Will do some benchmarking and make a decision when I'm at that point. Thanks!

<!-- gh-comment-id:2622455397 --> @ParthSareen commented on GitHub (Jan 29, 2025): Yeah, haven't updated it on purpose. I'm writing a new sampler at the moment and am not sure if it is worthwhile to add this as there might be an accuracy hit. Will do some benchmarking and make a decision when I'm at that point. Thanks!
GiteaMirror commented 2026-04-12 16:12:44 -05:00
Author
Owner
Copy Link

@stevenwaterman commented on GitHub (Jun 19, 2025):

Is there any update on this? It still seems like recursive schemas are not supported

<!-- gh-comment-id:2988260380 --> @stevenwaterman commented on GitHub (Jun 19, 2025): Is there any update on this? It still seems like recursive schemas are not supported
GiteaMirror commented 2026-04-12 16:12:45 -05:00
Author
Owner
Copy Link

@dpolivaev commented on GitHub (Jan 13, 2026):

Happy new year to everyone.
@ParthSareen Could you please update us on this important topic?

<!-- gh-comment-id:3746745738 --> @dpolivaev commented on GitHub (Jan 13, 2026): Happy new year to everyone. @ParthSareen Could you please update us on this important topic?
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#5114
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?