[GH-ISSUE #6925] Filter "body" should have access to "system message" #69140

New Issue

GiteaMirror · 2026-05-13T01:33:10-05:00

GiteaMirror commented

2026-05-13 01:33:10 -05:00

Originally created by @clang88 on GitHub (Nov 13, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/6925

Feature Request

I am trying to build a "token counter" filter for Open WebUI; it works fine, adding the "Tokens" at the end of each System output but removing them from the messages again before sending the message to the LLM.

The issue is: The token count is incorrect, because the "body" object does not seem to contain the system message:

Is your feature request related to a problem? Please describe.
This is particularly important when using RAG, as the context is added into the system message and is therefore not counted by my "token counter" filter.

Describe the solution you'd like
I believe it would be beneficial to also have acces to the system message with the filter, i.e. add the system message to the chain of messages.

Describe alternatives you've considered
Perhaps we should just add a "Token counter" for openAI models as is already present for Ollama models?

Originally created by @clang88 on GitHub (Nov 13, 2024). Original GitHub issue: https://github.com/open-webui/open-webui/issues/6925 # Feature Request I am trying to build a "token counter" filter for Open WebUI; it works fine, adding the "Tokens" at the end of each System output but removing them from the messages again before sending the message to the LLM. The issue is: The token count is incorrect, because the "body" object does not seem to contain the system message: ![image](https://github.com/user-attachments/assets/29ef64e0-e887-4d42-be17-df083c220ea5) --- **Is your feature request related to a problem? Please describe.** This is particularly important when using RAG, as the context is added into the system message and is therefore not counted by my "token counter" filter. **Describe the solution you'd like** I believe it would be beneficial to also have acces to the system message with the filter, i.e. add the system message to the chain of messages. **Describe alternatives you've considered** Perhaps we should just add a "Token counter" for openAI models as is already present for Ollama models?

GiteaMirror closed this issue

2026-05-13 01:33:11 -05:00

GiteaMirror commented

2026-05-13 01:33:12 -05:00

@clang88 commented on GitHub (Nov 13, 2024):

This is the code for the filter, btw. I commented out the trick I used to see the body contents.

"""
title: Token Counter
author: clang88
version: 1.0
"""

from pydantic import BaseModel, Field
from typing import Optional
import tiktoken
import json
import os



class Filter:
    class Valves(BaseModel):
        priority: int = Field(
            default=0, description="Priority level for the filter operations."
        )
        pass

    def __init__(self):
        self.valves = self.Valves()
        self.input_cost = 0
        self.output_cost = 0
        pass

    def inlet(self, body: dict, __user__: Optional[dict] = None) -> dict:
        if __user__.get("role", "admin") in ["user", "admin", "system"]:
            messages = body.get("messages", [])
            model = body.get("model")
            try:
                enc = tiktoken.encoding_for_model(model)
            except Exception as e:
                if "gpt-3.5-turbo" in model:
                    enc = tiktoken.encoding_for_model("gpt-3.5-turbo-")
                else:
                    print(f"Error: {e}")
                    enc = tiktoken.encoding_for_model("gpt-4o")

            for message in messages:
                if "\n\n**Token consumption:**" in message["content"]:
                    message["content"] = "".join(
                        message["content"].split("\n\n**Token consumption:**")
                    )[:-1]
        return body

    def outlet(self, body: dict, __user__: Optional[dict] = None) -> dict:
        messages = body.get("messages", [])
        model = body.get("model")
        try:
            enc = tiktoken.encoding_for_model(model)
        except Exception as e:
            if "gpt-3.5-turbo" in model:
                enc = tiktoken.encoding_for_model("gpt-3.5-turbo-")
            else:
                print(f"Error: {e}")
                enc = tiktoken.encoding_for_model("gpt-4o")

        total_tokens = sum(len(enc.encode(message["content"])) for message in messages)
        # messages[-1]["content"] = str(body)
        messages[-1]["content"] = str(
            messages[-1]["content"] + "\n\n**Token consumption:** " + str(total_tokens)
        )

        return body

@clang88 commented on GitHub (Nov 13, 2024): This is the code for the filter, btw. I commented out the trick I used to see the body contents. ```python """ title: Token Counter author: clang88 version: 1.0 """ from pydantic import BaseModel, Field from typing import Optional import tiktoken import json import os class Filter: class Valves(BaseModel): priority: int = Field( default=0, description="Priority level for the filter operations." ) pass def __init__(self): self.valves = self.Valves() self.input_cost = 0 self.output_cost = 0 pass def inlet(self, body: dict, __user__: Optional[dict] = None) -> dict: if __user__.get("role", "admin") in ["user", "admin", "system"]: messages = body.get("messages", []) model = body.get("model") try: enc = tiktoken.encoding_for_model(model) except Exception as e: if "gpt-3.5-turbo" in model: enc = tiktoken.encoding_for_model("gpt-3.5-turbo-") else: print(f"Error: {e}") enc = tiktoken.encoding_for_model("gpt-4o") for message in messages: if "\n\n**Token consumption:**" in message["content"]: message["content"] = "".join( message["content"].split("\n\n**Token consumption:**") )[:-1] return body def outlet(self, body: dict, __user__: Optional[dict] = None) -> dict: messages = body.get("messages", []) model = body.get("model") try: enc = tiktoken.encoding_for_model(model) except Exception as e: if "gpt-3.5-turbo" in model: enc = tiktoken.encoding_for_model("gpt-3.5-turbo-") else: print(f"Error: {e}") enc = tiktoken.encoding_for_model("gpt-4o") total_tokens = sum(len(enc.encode(message["content"])) for message in messages) # messages[-1]["content"] = str(body) messages[-1]["content"] = str( messages[-1]["content"] + "\n\n**Token consumption:** " + str(total_tokens) ) return body ```

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#69140