github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-16 05:05:37 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #856] Implement Function call support for LLama2 models #412

New Issue
Closed
opened 2026-04-12 10:04:17 -05:00 by GiteaMirror · 2 comments
GiteaMirror commented 2026-04-12 10:04:17 -05:00
Owner
Copy Link

Originally created by @AlexandrePoisson on GitHub (Oct 20, 2023).
Original GitHub issue: https://github.com/ollama/ollama/issues/856

Implement Function call support

I want to use langchain's capability to create_tagging_chain with Ollama to constraint the output on a specific JSON format.

Problem is that it works only for models which supports OpenAI function calling API. see [related issue](<https://github.com/langchain-ai/langchain/issues/11847)
and I don't have access to OpenAI models, so I work with LLama2, codellama, ... and those apparently doesn't support it. See code and error below

Sample python code (here with ChatOllama class & llama2 - chat model, but same occurs with Ollama class & Llama model)

from langchain.llms import Ollama
from langchain.chat_models import ChatOllama
from langchain.chains import create_tagging_chain
from langchain.chains import create_extraction_chain
from langchain.callbacks.manager import CallbackManager
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler       
from langchain.schema import HumanMessage
#Schema
model = ChatOllama(model="llama2:7b-chat", 
                        callback_manager = CallbackManager([StreamingStdOutCallbackHandler()]))

schema = {
    "properties": {
        "sentiment": {"type": "string", 'description': 'the sentiment encountered in the passage'},
        "aggressiveness": {"type": "integer", 'description': 'a 0-10 score of how aggressive the passage is'},
        "language": {"type": "string", 'description': 'the language of the passage'},
    }
}
chain = create_tagging_chain(schema, model)
chain.run("give me your money")

Output Error :

/bin/python3 /home/alexandre/langchain/test2.py
Traceback (most recent call last):
File "/home/alexandre/.local/lib/python3.10/site-packages/langchain/output_parsers/openai_functions.py", line 76, in parse_result
    function_call = message.additional_kwargs["function_call"]
KeyError: 'function_call'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/alexandre/langchain/test2.py", line 20, in <module>
    chain.run("give me your money")
File "/home/alexandre/.local/lib/python3.10/site-packages/langchain/chains/base.py", line 501, in run
    return self(args[0], callbacks=callbacks, tags=tags, metadata=metadata)[
File "/home/alexandre/.local/lib/python3.10/site-packages/langchain/chains/base.py", line 306, in __call__
    raise e
File "/home/alexandre/.local/lib/python3.10/site-packages/langchain/chains/base.py", line 300, in __call__
    self._call(inputs, run_manager=run_manager)
File "/home/alexandre/.local/lib/python3.10/site-packages/langchain/chains/llm.py", line 94, in _call
    return self.create_outputs(response)[0]
File "/home/alexandre/.local/lib/python3.10/site-packages/langchain/chains/llm.py", line 222, in create_outputs
    result = [
File "/home/alexandre/.local/lib/python3.10/site-packages/langchain/chains/llm.py", line 225, in <listcomp>
    self.output_key: self.output_parser.parse_result(generation),
File "/home/alexandre/.local/lib/python3.10/site-packages/langchain/output_parsers/openai_functions.py", line 81, in parse_result
    raise OutputParserException(f"Could not parse function call: {exc}")
langchain.schema.output_parser.OutputParserException: Could not parse function call: 'function_call'

Would it be possible to implement function calling API for those models ? Would it be Ollama duty or LLama-cpp - or Llama-cpp-python ?

Alternatively, anyone knows another free model (free as a free beer) which I can use that supports OpenAI function calling ?

Originally created by @AlexandrePoisson on GitHub (Oct 20, 2023). Original GitHub issue: https://github.com/ollama/ollama/issues/856 # Implement Function call support I want to use langchain's capability to [create_tagging_chain](https://python.langchain.com/docs/use_cases/tagging) with Ollama to constraint the output on a specific JSON format. Problem is that it works only for models which supports OpenAI function calling API. see [related issue](<https://github.com/langchain-ai/langchain/issues/11847) and I don't have access to OpenAI models, so I work with LLama2, codellama, ... and those apparently doesn't support it. See code and error below Sample python code (here with ChatOllama class & llama2 - chat model, but same occurs with Ollama class & Llama model) from langchain.llms import Ollama from langchain.chat_models import ChatOllama from langchain.chains import create_tagging_chain from langchain.chains import create_extraction_chain from langchain.callbacks.manager import CallbackManager from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler from langchain.schema import HumanMessage #Schema model = ChatOllama(model="llama2:7b-chat", callback_manager = CallbackManager([StreamingStdOutCallbackHandler()])) schema = { "properties": { "sentiment": {"type": "string", 'description': 'the sentiment encountered in the passage'}, "aggressiveness": {"type": "integer", 'description': 'a 0-10 score of how aggressive the passage is'}, "language": {"type": "string", 'description': 'the language of the passage'}, } } chain = create_tagging_chain(schema, model) chain.run("give me your money") Output Error : /bin/python3 /home/alexandre/langchain/test2.py Traceback (most recent call last): File "/home/alexandre/.local/lib/python3.10/site-packages/langchain/output_parsers/openai_functions.py", line 76, in parse_result function_call = message.additional_kwargs["function_call"] KeyError: 'function_call' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/alexandre/langchain/test2.py", line 20, in <module> chain.run("give me your money") File "/home/alexandre/.local/lib/python3.10/site-packages/langchain/chains/base.py", line 501, in run return self(args[0], callbacks=callbacks, tags=tags, metadata=metadata)[ File "/home/alexandre/.local/lib/python3.10/site-packages/langchain/chains/base.py", line 306, in __call__ raise e File "/home/alexandre/.local/lib/python3.10/site-packages/langchain/chains/base.py", line 300, in __call__ self._call(inputs, run_manager=run_manager) File "/home/alexandre/.local/lib/python3.10/site-packages/langchain/chains/llm.py", line 94, in _call return self.create_outputs(response)[0] File "/home/alexandre/.local/lib/python3.10/site-packages/langchain/chains/llm.py", line 222, in create_outputs result = [ File "/home/alexandre/.local/lib/python3.10/site-packages/langchain/chains/llm.py", line 225, in <listcomp> self.output_key: self.output_parser.parse_result(generation), File "/home/alexandre/.local/lib/python3.10/site-packages/langchain/output_parsers/openai_functions.py", line 81, in parse_result raise OutputParserException(f"Could not parse function call: {exc}") langchain.schema.output_parser.OutputParserException: Could not parse function call: 'function_call' Would it be possible to implement function calling API for those models ? Would it be Ollama duty or LLama-cpp - or Llama-cpp-python ? Alternatively, anyone knows another free model (free as a free beer) which I can use that supports OpenAI function calling ?
GiteaMirror added the feature requestfeedback wanted labels 2026-04-12 10:04:17 -05:00
GiteaMirror commented 2026-04-12 10:04:18 -05:00
Author
Owner
Copy Link

@olafgeibig commented on GitHub (Oct 24, 2023):

Look at https://github.com/KillianLucas/open-interpreter It is a re-implementation of OpenAIs function calling and it can run with local models.

<!-- gh-comment-id:1776860730 --> @olafgeibig commented on GitHub (Oct 24, 2023): Look at https://github.com/KillianLucas/open-interpreter It is a re-implementation of OpenAIs function calling and it can run with local models.
GiteaMirror commented 2026-04-12 10:04:19 -05:00
Author
Owner
Copy Link

@technovangelist commented on GitHub (Dec 4, 2023):

Recently we added format: json via the API and cli that allows you to always output well formed JSON and to specify the schema. I think that addresses all the parts of your original issue so I will go ahead and close it now. If you think there is anything we left out, reopen and we can address. Thanks for being part of this great community.

<!-- gh-comment-id:1839406542 --> @technovangelist commented on GitHub (Dec 4, 2023): Recently we added `format: json` via the API and cli that allows you to always output well formed JSON and to specify the schema. I think that addresses all the parts of your original issue so I will go ahead and close it now. If you think there is anything we left out, reopen and we can address. Thanks for being part of this great community.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label feature request feedback wanted
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#412
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?