Advanced local setup using MLX (Apple Silicon) + abliterated models + external harness (LHAI).
Current working architecture:
OpenWebUI → OpenAI-compatible endpoint (mlx_lm.server / vllm-mlx) → MLX models
Problem
OpenWebUI does not have a clear first-class pattern for:
Local MLX model usage
External runtime separation (UI vs inference)
Agent-oriented workflows with strict tool + JSON contracts
What works (documented setup)
Models stored locally (~/models/mlx/...)
MLX server exposes OpenAI-compatible API
OpenWebUI connects via custom endpoint
Gaps
No explicit docs for MLX workflows
No guidance for external runtime vs UI separation
No examples for agent-style usage
No mention of abliterated models and constraints
Proposed Improvements
Official MLX guide
Clear UI → API → Runtime architecture docs
Agent workflow examples (JSON, tools)
Sampling guidance for local models
Takeaway
Treat OpenWebUI as a client, not the runtime.
Meta (for agents)
UI is stateless
Runtime is external
Contracts enforced outside UI
Originally created by @kstabach on GitHub (Apr 30, 2026).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/24255
## Context
Advanced local setup using MLX (Apple Silicon) + abliterated models + external harness (LHAI).
Current working architecture:
OpenWebUI → OpenAI-compatible endpoint (mlx_lm.server / vllm-mlx) → MLX models
## Problem
OpenWebUI does not have a clear first-class pattern for:
- Local MLX model usage
- External runtime separation (UI vs inference)
- Agent-oriented workflows with strict tool + JSON contracts
## What works (documented setup)
- Models stored locally (~/models/mlx/...)
- MLX server exposes OpenAI-compatible API
- OpenWebUI connects via custom endpoint
## Gaps
1. No explicit docs for MLX workflows
2. No guidance for external runtime vs UI separation
3. No examples for agent-style usage
4. No mention of abliterated models and constraints
## Proposed Improvements
1. Official MLX guide
2. Clear UI → API → Runtime architecture docs
3. Agent workflow examples (JSON, tools)
4. Sampling guidance for local models
## Takeaway
Treat OpenWebUI as a client, not the runtime.
## Meta (for agents)
- UI is stateless
- Runtime is external
- Contracts enforced outside UI
@pr-validator-bot commented on GitHub (Apr 30, 2026):
⚠️ Missing Issue Title Prefix
@kstabach, your issue title is missing a prefix (e.g., bug:, feat:, docs:).
Please update your issue title to include one of the following prefixes:
bug: Bug report or error you've encountered
feat: Feature request or enhancement suggestion
docs: Documentation issue or improvement request
question: Question about usage or functionality
help: Request for help or support
Example: bug: Login fails when using special characters in password
<!-- gh-comment-id:4348789865 -->
@pr-validator-bot commented on GitHub (Apr 30, 2026):
# ⚠️ Missing Issue Title Prefix
@kstabach, your issue title is missing a prefix (e.g., `bug:`, `feat:`, `docs:`).
Please update your issue title to include one of the following prefixes:
- **bug**: Bug report or error you've encountered
- **feat**: Feature request or enhancement suggestion
- **docs**: Documentation issue or improvement request
- **question**: Question about usage or functionality
- **help**: Request for help or support
Example: `bug: Login fails when using special characters in password`
Leaving this here for context, but closing loop:\n\nThis was filed from a local agent/MLX workflow perspective and ended up being better tracked in my own stack (LHAI).\n\nThe core point still stands for anyone finding this:\n\n- OpenWebUI works best as a client\n- Local inference should live behind an OpenAI-compatible endpoint (MLX, vLLM, etc.)\n- Agent-style workflows (JSON, tools, strict schemas) are enforced outside the UI\n\nClosing internally and continuing iteration there. Thanks for the project — this is the best UI in this space right now.
<!-- gh-comment-id:4348948446 -->
@kstabach commented on GitHub (Apr 30, 2026):
Leaving this here for context, but closing loop:\n\nThis was filed from a local agent/MLX workflow perspective and ended up being better tracked in my own stack (LHAI).\n\nThe core point still stands for anyone finding this:\n\n- OpenWebUI works best as a **client**\n- Local inference should live behind an **OpenAI-compatible endpoint** (MLX, vLLM, etc.)\n- Agent-style workflows (JSON, tools, strict schemas) are enforced outside the UI\n\nClosing internally and continuing iteration there. Thanks for the project — this is the best UI in this space right now.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @kstabach on GitHub (Apr 30, 2026).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/24255
Context
Advanced local setup using MLX (Apple Silicon) + abliterated models + external harness (LHAI).
Current working architecture:
OpenWebUI → OpenAI-compatible endpoint (mlx_lm.server / vllm-mlx) → MLX models
Problem
OpenWebUI does not have a clear first-class pattern for:
What works (documented setup)
Gaps
Proposed Improvements
Takeaway
Treat OpenWebUI as a client, not the runtime.
Meta (for agents)
@pr-validator-bot commented on GitHub (Apr 30, 2026):
⚠️ Missing Issue Title Prefix
@kstabach, your issue title is missing a prefix (e.g.,
bug:,feat:,docs:).Please update your issue title to include one of the following prefixes:
Example:
bug: Login fails when using special characters in password@kstabach commented on GitHub (Apr 30, 2026):
Closing — filed in wrong repo. Moving to internal LHAI tracking.
@kstabach commented on GitHub (Apr 30, 2026):
Leaving this here for context, but closing loop:\n\nThis was filed from a local agent/MLX workflow perspective and ended up being better tracked in my own stack (LHAI).\n\nThe core point still stands for anyone finding this:\n\n- OpenWebUI works best as a client\n- Local inference should live behind an OpenAI-compatible endpoint (MLX, vLLM, etc.)\n- Agent-style workflows (JSON, tools, strict schemas) are enforced outside the UI\n\nClosing internally and continuing iteration there. Thanks for the project — this is the best UI in this space right now.