Target branch: Please verify that the pull request targets the dev branch.
Description: Provide a concise description of the changes made in this pull request.
Changelog: Ensure a changelog entry following the format of Keep a Changelog is added at the bottom of the PR description.
Documentation: Have you updated relevant documentation Open WebUI Docs, or other documentation sources?
Dependencies: Are there any new dependencies? Have you updated the dependency versions in the documentation?
Testing: Have you written and run sufficient tests for validating the changes?
Code review: Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards?
Prefix: To cleary categorize this pull request, prefix the pull request title, using one of the following:
BREAKING CHANGE: Significant changes that may affect compatibility
build: Changes that affect the build system or external dependencies
ci: Changes to our continuous integration processes or workflows
chore: Refactor, cleanup, or other non-functional code changes
docs: Documentation update or addition
feat: Introduces a new feature or enhancement to the codebase
fix: Bug fix or error correction
i18n: Internationalization or localization changes
perf: Performance improvement
refactor: Code restructuring for better maintainability, readability, or scalability
style: Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc.)
test: Adding missing tests or correcting existing tests
WIP: Work in progress, a temporary label for incomplete or ongoing work
Changelog Entry
Description
This feature introduces audit trails to enhance security auditing, debugging, and traceability within the OI API. It adds an ASGI middleware that intercepts HTTP metadata—including user details and, optionally, request/response payloads—and logs this information in a structured JSON format. The feature is configurable via several audit levels:
None (default): Audit logging is disabled.
METADATA: Logs only metadata related to the request (e.g., user, timestamp, resource, HTTP verb).
REQUEST: Logs metadata and the request body.
REQUEST_RESPONSE: Logs metadata, the request body, and the response body.
Important: Enabling audit logging may impact performance and increase memory consumption.
This update also introduces new environment variables to control audit logging behavior:
AUDIT_LOGS_FILE_PATH: Specifies the file path where audit logs will be stored (defaults to /data/audit.log).
AUDIT_LOG_FILE_ROTATION_SIZE: Defines the maximum size of a log file before it is rotated.
AUDIT_LOG_LEVEL: Sets the audit level. Acceptable values are METADATA, REQUEST, REQUEST_RESPONSE, or NONE (default is NONE).
MAX_BODY_LOG_SIZE: Specifies the maximum size of a request or response body to capture, preventing excessive memory usage.
AUDIT_EXCLUDED_PATHS: A comma-separated list of paths to exclude from audit logging (default excluded paths: /chats,/chat,/folders).
Added
AuditLoggingMiddleware: ASGI middleware that intercepts HTTP requests/responses to perform audit logging.
AuditLogger, AuditLogEntry, and AuditContext classes: Encapsulate audit logging functionality and structure audit data.
Custom Logger Configuration:
A console logger with a custom stdout_format for enhanced general log output.
Environment Variable Integration: Configurable variables such as AUDIT_LOG_LEVEL, AUDIT_LOGS_FILE_PATH, and AUDIT_LOG_FILE_ROTATION_SIZE to dynamically control logging behavior.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/open-webui/open-webui/pull/10436
**Author:** [@victorstevansuse](https://github.com/victorstevansuse)
**Created:** 2/20/2025
**Status:** ✅ Merged
**Merged:** 2/27/2025
**Merged by:** [@tjbck](https://github.com/tjbck)
**Base:** `audit-log-dev` ← **Head:** `feat/audits`
---
### 📝 Commits (1)
- [`39c6ecb`](https://github.com/open-webui/open-webui/commit/39c6ecb50d79111048fef7710edd935a6b98aea4) feat: add audit logger middleware and custom logger using loguru
### 📊 Changes
**5 files changed** (+434 additions, -0 deletions)
<details>
<summary>View changed files</summary>
📝 `backend/open_webui/env.py` (+22 -0)
📝 `backend/open_webui/main.py` (+20 -0)
➕ `backend/open_webui/utils/audit.py` (+249 -0)
➕ `backend/open_webui/utils/logger.py` (+140 -0)
📝 `backend/requirements.txt` (+3 -0)
</details>
### 📄 Description
# Pull Request Checklist
- [x] **Target branch:** Please verify that the pull request targets the `dev` branch.
- [x] **Description:** Provide a concise description of the changes made in this pull request.
- [ ] **Changelog:** Ensure a changelog entry following the format of [Keep a Changelog](https://keepachangelog.com/) is added at the bottom of the PR description.
- [ ] **Documentation:** Have you updated relevant documentation [Open WebUI Docs](https://github.com/open-webui/docs), or other documentation sources?
- [x] **Dependencies:** Are there any new dependencies? Have you updated the dependency versions in the documentation?
- [x] **Testing:** Have you written and run sufficient tests for validating the changes?
- [x] **Code review:** Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards?
- [x] **Prefix:** To cleary categorize this pull request, prefix the pull request title, using one of the following:
- **BREAKING CHANGE**: Significant changes that may affect compatibility
- **build**: Changes that affect the build system or external dependencies
- **ci**: Changes to our continuous integration processes or workflows
- **chore**: Refactor, cleanup, or other non-functional code changes
- **docs**: Documentation update or addition
- **feat**: Introduces a new feature or enhancement to the codebase
- **fix**: Bug fix or error correction
- **i18n**: Internationalization or localization changes
- **perf**: Performance improvement
- **refactor**: Code restructuring for better maintainability, readability, or scalability
- **style**: Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc.)
- **test**: Adding missing tests or correcting existing tests
- **WIP**: Work in progress, a temporary label for incomplete or ongoing work
# Changelog Entry
### Description
This feature introduces audit trails to enhance security auditing, debugging, and traceability within the OI API. It adds an ASGI middleware that intercepts HTTP metadata—including user details and, optionally, request/response payloads—and logs this information in a structured JSON format. The feature is configurable via several audit levels:
- **None (default):** Audit logging is disabled.
- **`METADATA`:** Logs only metadata related to the request (e.g., user, timestamp, resource, HTTP verb).
- **`REQUEST`:** Logs metadata and the request body.
- **`REQUEST_RESPONSE`:** Logs metadata, the request body, and the response body.
**Important:** Enabling audit logging may impact performance and increase memory consumption.
This update also introduces new environment variables to control audit logging behavior:
- **`AUDIT_LOGS_FILE_PATH`:** Specifies the file path where audit logs will be stored (defaults to `/data/audit.log`).
- **`AUDIT_LOG_FILE_ROTATION_SIZE`:** Defines the maximum size of a log file before it is rotated.
- **`AUDIT_LOG_LEVEL`:** Sets the audit level. Acceptable values are `METADATA`, `REQUEST`, `REQUEST_RESPONSE`, or `NONE` (default is `NONE`).
- **`MAX_BODY_LOG_SIZE`:** Specifies the maximum size of a request or response body to capture, preventing excessive memory usage.
- **`AUDIT_EXCLUDED_PATHS`:** A comma-separated list of paths to exclude from audit logging (default excluded paths: `/chats,/chat,/folders`).
### Added
- **AuditLoggingMiddleware:** ASGI middleware that intercepts HTTP requests/responses to perform audit logging.
- **AuditLogger, AuditLogEntry, and AuditContext classes:** Encapsulate audit logging functionality and structure audit data.
- **Custom Logger Configuration:**
- A console logger with a custom `stdout_format` for enhanced general log output.
- **Environment Variable Integration:** Configurable variables such as `AUDIT_LOG_LEVEL`, `AUDIT_LOGS_FILE_PATH`, and `AUDIT_LOG_FILE_ROTATION_SIZE` to dynamically control logging behavior.
- `loguru` library for improved logging features.
- `asgiref` for rigorous ASGI type hints (see https://www.starlette.io/middleware/#type-annotations)
### Changed
- **Logger Initialization**: Updated the logger setup to integrate Python’s standard logging and Uvicorn logging with Loguru improving standard output logs.
### Deprecated
- *None*
### Removed
- *None*
### Fixed
- *None*
### Security
- *None*
### Breaking Changes
- *None*
### Additional Information
- As mentioned, this could impact the API performance negatively depending on the configuration.
- Previous implementations streaming issues should be fixed.
- [Previous related PR](https://github.com/open-webui/open-webui/pull/8509)
### Screenshots or Videos
https://github.com/user-attachments/assets/ed4983f2-293f-49aa-9b38-54484f959cde
- Video terminal output:
```
$ jq . audit.log
{
"id": "a9a2f56a-521c-425f-9f03-1c8293a488ff",
"timestamp": 1740064037,
"user": {
"id": "e1d62734-9837-4e5e-9a5f-459d8f63dc62",
"name": "Victor",
"email": "viktor.stevan1999@gmail.com",
"role": "admin"
},
"audit_level": "REQUEST_RESPONSE",
"verb": "POST",
"request_uri": "http://localhost:8080/api/v1/auths/add",
"response_status_code": 200,
"source_ip": "127.0.0.1",
"user_agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36",
"request_object": "{\"name\":\"test\",\"email\":\"test@mail.com\",\"password\":\"1234567\",\"role\":\"user\"}",
"response_object": "{\"id\":\"7b4c4ce2-d66a-45cd-9746-dd0544a8f028\",\"email\":\"test@mail.com\",\"name\":\"test\",\"role\":\"user\",\"profile_image_url\":\"/user.png\",\"token\":\"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjdiNGM0Y2UyLWQ2NmEtNDVjZC05NzQ2LWRkMDU0NGE4ZjAyOCJ9.dtbs0xaeNZ1jVjHf9nz2YMfmUh39h7zHGeMusGFPIyM\",\"token_type\":\"Bearer\"}",
"extra": {}
}
{
"id": "91de517a-651e-4018-9455-e44368d2075a",
"timestamp": 1740064061,
"user": {
"id": "e1d62734-9837-4e5e-9a5f-459d8f63dc62",
"name": "Victor",
"email": "viktor.stevan1999@gmail.com",
"role": "admin"
},
"audit_level": "REQUEST_RESPONSE",
"verb": "POST",
"request_uri": "http://localhost:8080/api/v1/users/7b4c4ce2-d66a-45cd-9746-dd0544a8f028/update",
"response_status_code": 200,
"source_ip": "127.0.0.1",
"user_agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36",
"request_object": "{\"profile_image_url\":\"/user.png\",\"email\":\"test@mail.com\",\"name\":\"test123\",\"password\":\"1234567\"}",
"response_object": "{\"id\":\"7b4c4ce2-d66a-45cd-9746-dd0544a8f028\",\"name\":\"test123\",\"email\":\"test@mail.com\",\"role\":\"user\",\"profile_image_url\":\"/user.png\",\"last_active_at\":1740064037,\"updated_at\":1740064037,\"created_at\":1740064037,\"api_key\":null,\"settings\":null,\"info\":null,\"oauth_sub\":null}",
"extra": {}
}
```
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/open-webui/open-webui/pull/10436
Author: @victorstevansuse
Created: 2/20/2025
Status: ✅ Merged
Merged: 2/27/2025
Merged by: @tjbck
Base:
audit-log-dev← Head:feat/audits📝 Commits (1)
39c6ecbfeat: add audit logger middleware and custom logger using loguru📊 Changes
5 files changed (+434 additions, -0 deletions)
View changed files
📝
backend/open_webui/env.py(+22 -0)📝
backend/open_webui/main.py(+20 -0)➕
backend/open_webui/utils/audit.py(+249 -0)➕
backend/open_webui/utils/logger.py(+140 -0)📝
backend/requirements.txt(+3 -0)📄 Description
Pull Request Checklist
devbranch.Changelog Entry
Description
This feature introduces audit trails to enhance security auditing, debugging, and traceability within the OI API. It adds an ASGI middleware that intercepts HTTP metadata—including user details and, optionally, request/response payloads—and logs this information in a structured JSON format. The feature is configurable via several audit levels:
METADATA: Logs only metadata related to the request (e.g., user, timestamp, resource, HTTP verb).REQUEST: Logs metadata and the request body.REQUEST_RESPONSE: Logs metadata, the request body, and the response body.Important: Enabling audit logging may impact performance and increase memory consumption.
This update also introduces new environment variables to control audit logging behavior:
AUDIT_LOGS_FILE_PATH: Specifies the file path where audit logs will be stored (defaults to/data/audit.log).AUDIT_LOG_FILE_ROTATION_SIZE: Defines the maximum size of a log file before it is rotated.AUDIT_LOG_LEVEL: Sets the audit level. Acceptable values areMETADATA,REQUEST,REQUEST_RESPONSE, orNONE(default isNONE).MAX_BODY_LOG_SIZE: Specifies the maximum size of a request or response body to capture, preventing excessive memory usage.AUDIT_EXCLUDED_PATHS: A comma-separated list of paths to exclude from audit logging (default excluded paths:/chats,/chat,/folders).Added
stdout_formatfor enhanced general log output.AUDIT_LOG_LEVEL,AUDIT_LOGS_FILE_PATH, andAUDIT_LOG_FILE_ROTATION_SIZEto dynamically control logging behavior.logurulibrary for improved logging features.asgireffor rigorous ASGI type hints (see https://www.starlette.io/middleware/#type-annotations)Changed
Deprecated
Removed
Fixed
Security
Breaking Changes
Additional Information
As mentioned, this could impact the API performance negatively depending on the configuration.
Previous implementations streaming issues should be fixed.
Previous related PR
Screenshots or Videos
https://github.com/user-attachments/assets/ed4983f2-293f-49aa-9b38-54484f959cde
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.