github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #10934] Feature Request: External Tool Integration for Executing and Using Common Lisp via <lisp> Tags and proper prompt prefix #7197

New Issue
Open
opened 2026-04-12 19:11:44 -05:00 by GiteaMirror · 6 comments
GiteaMirror commented 2026-04-12 19:11:44 -05:00
Owner
Copy Link

Originally created by @jordidelatorre on GitHub (May 31, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/10934

Summary

I propose adding support for integrating an external Common Lisp evaluation engine with Ollama. This would enable the model to evaluate Common Lisp code blocks dynamically and use the results to enhance reasoning, perform computations, and maintain contextual state.

🔧 Proposed Feature

Enable the use of a special syntax that signals the model to execute embedded Lisp code:

<lisp>
;; Common Lisp code here
(format t "Hello, world!")
</lisp>

When this block is encountered, Ollama should:

  1. Delegate the code to an external Common Lisp evaluator.
  2. Capture the output.
  3. Return the result in a dedicated tag:
<lisp-result>
Hello, world!
</lisp-result>

📌 Use Cases

  • Symbolic reasoning
    Execute logic or symbolic expressions with Lisp’s native capabilities.

  • Dynamic computation
    Perform real-time math or data structure manipulation using Lisp.

  • Session memory
    Maintain temporary state in the form of Lisp variables or structures.

✅ Benefits

  • Enhances interactivity, enabling live code execution.
  • Improves reasoning and dynamic response generation.
  • Enables temporary memory and stateful interactions using Lisp.
  • Serves as a blueprint for integrating other languages
  • Bridges the gap between code understanding and runtime execution.

💡 Why Lisp?

Lisp is a natural fit for LLM integration due to its strengths in symbolic reasoning, minimal and consistent syntax, and homoiconicity—where code is treated as data. This makes it easy for language models to generate, analyze, and manipulate Lisp code reliably. Its REPL-driven, stateful design aligns well with dynamic, interactive sessions, enabling real-time evaluation and temporary memory. These traits make Lisp not only efficient for logic-based tasks but also uniquely compatible with the way LLMs process and structure information.

🙏 Thanks

Thank you for considering this enhancement. It could be a powerful tool for users who rely on symbolic reasoning and functional programming.

Originally created by @jordidelatorre on GitHub (May 31, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/10934 ## Summary I propose adding support for integrating an external **Common Lisp evaluation engine** with Ollama. This would enable the model to **evaluate Common Lisp code blocks** dynamically and use the results to **enhance reasoning**, **perform computations**, and **maintain contextual state**. --- ## 🔧 Proposed Feature Enable the use of a special syntax that signals the model to execute embedded Lisp code: ```lisp <lisp> ;; Common Lisp code here (format t "Hello, world!") </lisp> ``` When this block is encountered, Ollama should: 1. **Delegate the code** to an external Common Lisp evaluator. 2. **Capture the output**. 3. **Return the result** in a dedicated tag: ```lisp <lisp-result> Hello, world! </lisp-result> ``` --- ## 📌 Use Cases - **Symbolic reasoning** Execute logic or symbolic expressions with Lisp’s native capabilities. - **Dynamic computation** Perform real-time math or data structure manipulation using Lisp. - **Session memory** Maintain temporary state in the form of Lisp variables or structures. --- ## ✅ Benefits - Enhances interactivity, enabling **live code execution**. - Improves reasoning and **dynamic response generation**. - Enables **temporary memory** and stateful interactions using Lisp. - Serves as a **blueprint for integrating other languages** - Bridges the gap between **code understanding** and **runtime execution**. --- ## 💡 Why Lisp? Lisp is a natural fit for LLM integration due to its strengths in symbolic reasoning, minimal and consistent syntax, and homoiconicity—where code is treated as data. This makes it easy for language models to generate, analyze, and manipulate Lisp code reliably. Its REPL-driven, stateful design aligns well with dynamic, interactive sessions, enabling real-time evaluation and temporary memory. These traits make Lisp not only efficient for logic-based tasks but also uniquely compatible with the way LLMs process and structure information. --- ## 🙏 Thanks Thank you for considering this enhancement. It could be a powerful tool for users who rely on symbolic reasoning and functional programming.
GiteaMirror added the feature request label 2026-04-12 19:11:44 -05:00
GiteaMirror commented 2026-04-12 19:11:45 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (May 31, 2025):

So the client creates a prompt/query that has embedded lisp code, sends it to the ollama server, the ollama server extracts the lisp and sends it to an evaluator, retrieves the result, appends it to the prompt, and then processes the prompt, and returns the result to the client?

<!-- gh-comment-id:2925780683 --> @rick-github commented on GitHub (May 31, 2025): So the client creates a prompt/query that has embedded lisp code, sends it to the ollama server, the ollama server extracts the lisp and sends it to an evaluator, retrieves the result, appends it to the prompt, and then processes the prompt, and returns the result to the client?
GiteaMirror commented 2026-04-12 19:11:45 -05:00
Author
Owner
Copy Link

@jordidelatorre commented on GitHub (Jun 1, 2025):

No, sorry — what I meant is this: during its reasoning process, the LLM can generate code enclosed in <lisp>...</lisp> tags. The Ollama server detects these code blocks and sends them to a Lisp interpreter for execution. Once the code is executed, the server appends the result within <lisp-result>...</lisp-result> tags and returns it to the LLM. The LLM then incorporates this result and continues its generation process.

This mechanism allows the LLM to interact with external tools via Lisp and store intermediate data in the Lisp interpreter’s temporary memory. That memory can be reused later in the same reasoning chain, enhancing the LLM's ability to "think" and manage context more effectively.

To enable the LLM to generate these <lisp> blocks, it must either be trained to do so or guided via an initial system prompt. I've chosen Common Lisp for this setup because it's a highly expressive and powerful language.

This architecture extends the LLM’s capabilities by integrating symbolic reasoning, persistent state, and executable logic into its generation process.

It's akin to giving a thinker a piece of paper, a pencil to sketch out ideas and a Turing-complete calculator.

<!-- gh-comment-id:2927025763 --> @jordidelatorre commented on GitHub (Jun 1, 2025): No, sorry — what I meant is this: during its reasoning process, the LLM can generate code enclosed in ```<lisp>...</lisp>``` tags. The Ollama server detects these code blocks and sends them to a Lisp interpreter for execution. Once the code is executed, the server appends the result within ```<lisp-result>...</lisp-result>``` tags and returns it to the LLM. The LLM then incorporates this result and continues its generation process. This mechanism allows the LLM to interact with external tools via Lisp and store intermediate data in the Lisp interpreter’s temporary memory. That memory can be reused later in the same reasoning chain, enhancing the LLM's ability to "think" and manage context more effectively. To enable the LLM to generate these ```<lisp>``` blocks, it must either be trained to do so or guided via an initial system prompt. I've chosen Common Lisp for this setup because it's a highly expressive and powerful language. This architecture extends the LLM’s capabilities by integrating symbolic reasoning, persistent state, and executable logic into its generation process. It's akin to giving a thinker a piece of paper, a pencil to sketch out ideas and a Turing-complete calculator.
GiteaMirror commented 2026-04-12 19:11:46 -05:00
Author
Owner
Copy Link

@AdamSobieski commented on GitHub (Jun 2, 2025):

Hello. If I understand correctly, the idea resembles LLMs being capable of first generating PHP-like content, then performing server-side processing upon it to produce resultant output to send to clients?

With respect to syntax possibilities, what about:

<?lisp (format t "Hello, world!") ?>

Then, we might also envision:

<?prolog write('Hello, world!'). ?>

<?php echo("Hello, world!"); ?>

<?python print("Hello, world!") ?>

<?javascript console.log('Hello, world!'); ?>

A solution could be extensible beyond Lisp. Developers could install modular plugins.

From preprocessing scripts, there would also be a means for interpreting multiple interconnected preprocessing scopes' contents in a result stream.

The above syntax possibilities, however, don't suggest clearly distinguishing preprocessed outputs from other outputs, that is automatically providing your <lisp-result>...</lisp-result> enclosure. Maybe a debugging mode or other toggle could add this for developers?

Also, a potential syntax would, I think, need to differ from the possibilities indicated, above, so that LLMs would still be able to output PHP source code and examples.

<!-- gh-comment-id:2928696933 --> @AdamSobieski commented on GitHub (Jun 2, 2025): Hello. If I understand correctly, the idea resembles LLMs being capable of first generating PHP-like content, then performing server-side processing upon it to produce resultant output to send to clients? With respect to syntax possibilities, what about: ```xml <?lisp (format t "Hello, world!") ?> ``` Then, we might also envision: ```xml <?prolog write('Hello, world!'). ?> ``` ```xml <?php echo("Hello, world!"); ?> ``` ```xml <?python print("Hello, world!") ?> ``` ```xml <?javascript console.log('Hello, world!'); ?> ``` A solution could be extensible beyond Lisp. Developers could install modular plugins. From preprocessing scripts, there would also be a means for interpreting multiple interconnected preprocessing scopes' contents in a result stream. The above syntax possibilities, however, don't suggest clearly distinguishing preprocessed outputs from other outputs, that is automatically providing your `<lisp-result>...</lisp-result>` enclosure. Maybe a debugging mode or other toggle could add this for developers? Also, a potential syntax would, I think, need to differ from the possibilities indicated, above, so that LLMs would still be able to output PHP source code and examples.
GiteaMirror commented 2026-04-12 19:11:46 -05:00
Author
Owner
Copy Link

@jordidelatorre commented on GitHub (Jun 2, 2025):

The idea is to allow the LLM to send calculations, program executions, and temporary memory load/store operations to an external source.

The syntax is only a proposal and can certainly be improved. The important point is to find the most suitable syntax to facilitate LLM generation. My basic proposal was based on the existing <thinking> ... </thinking> tags. Maybe we don't need the result tags and the value can be encoded in the existing <lisp> ... <lisp> placeholder.

As you also suggest, it is extensible beyond Lisp. In my opinion, the Lisp list syntax has its advantages. Furthermore, Common Lisp also has a huge library of existing, well-documented functions already encoded in LLMs training dataset by default.

<!-- gh-comment-id:2928923345 --> @jordidelatorre commented on GitHub (Jun 2, 2025): The idea is to allow the LLM to send calculations, program executions, and temporary memory load/store operations to an external source. The syntax is only a proposal and can certainly be improved. The important point is to find the most suitable syntax to facilitate LLM generation. My basic proposal was based on the existing ```<thinking> ... </thinking>``` tags. Maybe we don't need the result tags and the value can be encoded in the existing ```<lisp> ... <lisp>``` placeholder. As you also suggest, it is extensible beyond Lisp. In my opinion, the Lisp list syntax has its advantages. Furthermore, Common Lisp also has a huge library of existing, well-documented functions already encoded in LLMs training dataset by default.
GiteaMirror commented 2026-04-12 19:11:47 -05:00
Author
Owner
Copy Link

@AdamSobieski commented on GitHub (Jun 3, 2025):

@jdltg, thank you. It is an interesting idea.

I'm reviewing how agentic systems currently transform between source code as data resources and source code as interpretable or executable resources, e.g., providing generated source code to code-executing agents and receiving the results of computation within responses. I'm also reviewing how single agents can currently interpret or execute source code that they generate.

If you haven't already seen these, please feel free to check out:

Here is an example expressed using a number of syntax possibilities:

The answer is <lisp>(format t (* 6 7))</lisp>.

The answer is <|lisp_start|>(format t (* 6 7))<|lisp_end|>.

The answer is <|preprocessing_start|><|lisp|>(format t (* 6 7))<|preprocessing_end|>.

Also, interestingly, the results from LLM-generated preprocessing scopes could be multimodal, utilizing image-processing and/or drawing libraries.

... as one can observe in the following image: <lisp>(imago-llm:show-image ...)</lisp>.
<!-- gh-comment-id:2936529406 --> @AdamSobieski commented on GitHub (Jun 3, 2025): @jdltg, thank you. It is an interesting idea. I'm reviewing how agentic systems currently transform between source code as data resources and source code as interpretable or executable resources, e.g., providing generated source code to code-executing agents and receiving the results of computation within responses. I'm also reviewing how single agents can currently interpret or execute source code that they generate. If you haven't already seen these, please feel free to check out: * [LIDA: Automated Generation of Visualizations and Infographics with LLMs](https://microsoft.github.io/lida/) ([source code](https://github.com/microsoft/lida)) ([paper](https://aclanthology.org/2023.acl-demo.11/)). This paper is from July, 2023. * [Creating a Clever Code Interpreter with Langchain agents + Advanced Prompt Techniques](https://medium.com/latinxinai/creating-a-clever-code-interpreter-tool-with-langchain-agents-advanced-prompt-techniques-3d7b493cc580) from October 2, 2023. * [Build Your Own Code Interpreter - Dynamic Tool Generation and Execution with o3-mini](https://cookbook.openai.com/examples/object_oriented_agentic_approach/secure_code_interpreter_tool_for_llm_agents) from February 3, 2025. Here is an example expressed using a number of syntax possibilities: ``` The answer is <lisp>(format t (* 6 7))</lisp>. ``` ``` The answer is <|lisp_start|>(format t (* 6 7))<|lisp_end|>. ``` ``` The answer is <|preprocessing_start|><|lisp|>(format t (* 6 7))<|preprocessing_end|>. ``` Also, interestingly, the results from LLM-generated preprocessing scopes could be multimodal, utilizing image-processing and/or drawing libraries. ``` ... as one can observe in the following image: <lisp>(imago-llm:show-image ...)</lisp>. ```
GiteaMirror commented 2026-04-12 19:11:47 -05:00
Author
Owner
Copy Link

@jordidelatorre commented on GitHub (Jun 13, 2025):

https://arxiv.org/html/2506.10021v1

<!-- gh-comment-id:2970745963 --> @jordidelatorre commented on GitHub (Jun 13, 2025): https://arxiv.org/html/2506.10021v1
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#7197
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?