[GH-ISSUE #20655] feat: Add ENABLE_OTEL_REDIS env var to disable Redis instrumentation #57917

Closed
opened 2026-05-05 21:55:53 -05:00 by GiteaMirror · 1 comment
Owner

Originally created by @hcnimi on GitHub (Jan 14, 2026).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/20655

Check Existing Issues

  • I have searched for all existing open AND closed issues and discussions for similar requests. I have found none that is comparable to my request.

Verify Feature Scope

  • I have read through and understood the scope definition for feature requests in the Issues section. I believe my feature request meets the definition and belongs in the Issues section instead of the Discussions.

Problem Description

When ENABLE_OTEL=true, all instrumentors are unconditionally enabled via manual calls in backend/open_webui/utils/telemetry/instrumentors.py:

RedisInstrumentor().instrument(request_hook=redis_request_hook)
SQLAlchemyInstrumentor().instrument(engine=self.db_engine)

... etc

This causes 10,000+ spans per trace in high-traffic deployments, primarily from Redis operations. Each chat message triggers hundreds of Redis calls (session management, websocket coordination, caching), and every call creates a span.

Why this is a problem

  1. APM costs - Datadog/Grafana/etc. charge by span volume; 10k spans/trace is expensive
  2. Unreadable traces - Traces become impossible to analyze when flooded with Redis spans
  3. No workaround - OTEL_PYTHON_DISABLED_INSTRUMENTATIONS=redis doesn't work because open-webui uses manual instrumentation, bypassing the standard OTEL env var

Evidence

Datadog APM trace showing ~10k spans, mostly Redis GET/SET operations:

  • Service: open-webui
  • Most spans: redis.GET, redis.SET, redis.HGET, etc.

Desired Solution you'd like

Add granular environment variables following the existing pattern:

In env.py

ENABLE_OTEL_REDIS = os.environ.get("ENABLE_OTEL_REDIS", "true").lower() == "true"
ENABLE_OTEL_SQLALCHEMY = os.environ.get("ENABLE_OTEL_SQLALCHEMY", "true").lower() == "true"
ENABLE_OTEL_HTTPX = os.environ.get("ENABLE_OTEL_HTTPX", "true").lower() == "true"
ENABLE_OTEL_REQUESTS = os.environ.get("ENABLE_OTEL_REQUESTS", "true").lower() == "true"
ENABLE_OTEL_AIOHTTP = os.environ.get("ENABLE_OTEL_AIOHTTP", "true").lower() == "true"
ENABLE_OTEL_LOGGING = os.environ.get("ENABLE_OTEL_LOGGING", "true").lower() == "true"

In instrumentors.py

if ENABLE_OTEL_REDIS:
RedisInstrumentor().instrument(request_hook=redis_request_hook)
if ENABLE_OTEL_SQLALCHEMY:
SQLAlchemyInstrumentor().instrument(engine=self.db_engine)

... etc

This follows the existing pattern of ENABLE_OTEL_TRACES, ENABLE_OTEL_METRICS, ENABLE_OTEL_LOGS.

Alternatives Considered

Alternatively, respect the standard OpenTelemetry environment variable OTEL_PYTHON_DISABLED_INSTRUMENTATIONS by checking it before calling each instrumentor.

Environment

  • open-webui version: 0.6.22 (PyPI)
  • Deployment: Kubernetes with Redis for session/websocket management
  • OTEL exporter: Datadog via OTLP (port 4317)

Additional Context

No response

Originally created by @hcnimi on GitHub (Jan 14, 2026). Original GitHub issue: https://github.com/open-webui/open-webui/issues/20655 ### Check Existing Issues - [x] I have searched for all existing **open AND closed** issues and discussions for similar requests. I have found none that is comparable to my request. ### Verify Feature Scope - [x] I have read through and understood the scope definition for feature requests in the Issues section. I believe my feature request meets the definition and belongs in the Issues section instead of the Discussions. ### Problem Description When ENABLE_OTEL=true, all instrumentors are unconditionally enabled via manual calls in backend/open_webui/utils/telemetry/instrumentors.py: RedisInstrumentor().instrument(request_hook=redis_request_hook) SQLAlchemyInstrumentor().instrument(engine=self.db_engine) # ... etc This causes 10,000+ spans per trace in high-traffic deployments, primarily from Redis operations. Each chat message triggers hundreds of Redis calls (session management, websocket coordination, caching), and every call creates a span. Why this is a problem 1. APM costs - Datadog/Grafana/etc. charge by span volume; 10k spans/trace is expensive 2. Unreadable traces - Traces become impossible to analyze when flooded with Redis spans 3. No workaround - OTEL_PYTHON_DISABLED_INSTRUMENTATIONS=redis doesn't work because open-webui uses manual instrumentation, bypassing the standard OTEL env var Evidence Datadog APM trace showing ~10k spans, mostly Redis GET/SET operations: - Service: open-webui - Most spans: redis.GET, redis.SET, redis.HGET, etc. ### Desired Solution you'd like Add granular environment variables following the existing pattern: # In env.py ENABLE_OTEL_REDIS = os.environ.get("ENABLE_OTEL_REDIS", "true").lower() == "true" ENABLE_OTEL_SQLALCHEMY = os.environ.get("ENABLE_OTEL_SQLALCHEMY", "true").lower() == "true" ENABLE_OTEL_HTTPX = os.environ.get("ENABLE_OTEL_HTTPX", "true").lower() == "true" ENABLE_OTEL_REQUESTS = os.environ.get("ENABLE_OTEL_REQUESTS", "true").lower() == "true" ENABLE_OTEL_AIOHTTP = os.environ.get("ENABLE_OTEL_AIOHTTP", "true").lower() == "true" ENABLE_OTEL_LOGGING = os.environ.get("ENABLE_OTEL_LOGGING", "true").lower() == "true" # In instrumentors.py if ENABLE_OTEL_REDIS: RedisInstrumentor().instrument(request_hook=redis_request_hook) if ENABLE_OTEL_SQLALCHEMY: SQLAlchemyInstrumentor().instrument(engine=self.db_engine) # ... etc This follows the existing pattern of ENABLE_OTEL_TRACES, ENABLE_OTEL_METRICS, ENABLE_OTEL_LOGS. ### Alternatives Considered Alternatively, respect the standard OpenTelemetry environment variable OTEL_PYTHON_DISABLED_INSTRUMENTATIONS by checking it before calling each instrumentor. Environment - open-webui version: 0.6.22 (PyPI) - Deployment: Kubernetes with Redis for session/websocket management - OTEL exporter: Datadog via OTLP (port 4317) ### Additional Context _No response_
Author
Owner

@tjbck commented on GitHub (Jan 14, 2026):

PR Welcome.

<!-- gh-comment-id:3749656206 --> @tjbck commented on GitHub (Jan 14, 2026): PR Welcome.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#57917