Cache external renderers output #10115

New Issue

GiteaMirror · 2025-11-02T08:58:31-06:00

GiteaMirror commented

2025-11-02 08:58:31 -06:00

Originally created by @PlushBeaver on GitHub (Jan 17, 2023).

Feature Description

External rendering can consume a lot of CPU and produce results larger than the BLOB being rendered. Caching would reduce both the load and the bandwidth consumption.

The hard part is cache invalidation. External rendering inputs:

BLOB — has a hash.
Gitea code calling the external rendering — cannot be tracked directly, but can be identified by the Gitea version (development builds are not expected to rely on caching).
Relevant section of Gitea configuration — can be hashed on load.
External program and its dependencies — cannot possibly be tracked by Gitea currently, see below.

Add [markup.XXX.CACHE_SEED]: an arbitrary string that must identify all the rendering dependencies. If unset, the caching is disabled. Administrator must change it whenever they update the rendering program or its dependencies. For example, the last system upgrade time may be used if the rendering program does not use any non-distribution files. Its generation can be automated with tools like Ansible or Nix (the latter would probably generate a unique configuration anyway).
Combine the BLOB hash, Gitea version, and all [markup.XXX] parameters to produce ETag value for the caching.

https://github.com/go-gitea/gitea/issues/21391 — there is a demand for caching improvement
https://github.com/go-gitea/gitea/issues/17635 — heavy external rendering case

Screenshots

No response

Originally created by @PlushBeaver on GitHub (Jan 17, 2023). ### Feature Description External rendering can consume a lot of CPU and produce results larger than the BLOB being rendered. Caching would reduce both the load and the bandwidth consumption. The hard part is cache invalidation. External rendering inputs: * BLOB — has a hash. * Gitea code calling the external rendering — cannot be tracked directly, but can be identified by the Gitea version (development builds are not expected to rely on caching). * Relevant section of Gitea configuration — can be hashed on load. * External program and its dependencies — cannot possibly be tracked by Gitea currently, see below. 1. Add [markup.XXX.CACHE_SEED]: an arbitrary string that must identify all the rendering dependencies. **If unset, the caching is disabled.** Administrator must change it whenever they update the rendering program or its dependencies. For example, the last system upgrade time may be used if the rendering program does not use any non-distribution files. Its generation can be automated with tools like Ansible or Nix (the latter would probably generate a unique configuration anyway). 2. Combine the BLOB hash, Gitea version, and all [markup.XXX] parameters to produce ETag value for the caching. Related: * https://github.com/go-gitea/gitea/issues/21391 — there is a demand for caching improvement * https://github.com/go-gitea/gitea/issues/17635 — heavy external rendering case ### Screenshots _No response_

GiteaMirror added the type/proposal type/feature labels 2025-11-02 08:58:31 -06:00

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/gitea#10115