Rebuild association paths can emit thousands of websocket sends in bursts, which was creating Redis/websocket backpressure. This update introduces batched cross-node direct delivery and queue-level deduplication for rebuild jobs so duplicate queued work is dropped while preserving re-queue behavior for in-progress jobs.
Websocket cross-node batching (direct sends)
Added Redis direct-batch message support and typed it in the shared WS message contract.
Replaced per-message Redis publish for remote direct sends with buffered batching (size 250, short flush interval).
Added subscriber-side bulk handling to fan out batched direct messages efficiently to local connected clients.
Queue deduplication semantics for rebuilds
Added queued-job dedupe using Redis set membership keyed by type:id.
Skip enqueue when an identical job is already queued.
Remove dedupe key at dequeue time (not completion), so if a job is currently in progress, a new same-type/id job can still be queued.
Type/contract tightening
Refined RedisMessage into a discriminated union for direct, direct-batch, and broadcast, making batch payload shape explicit.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/fosrl/pangolin/pull/3294
**Author:** [@Copilot](https://github.com/apps/copilot-swe-agent)
**Created:** 6/16/2026
**Status:** 🔄 Open
**Base:** `queue` ← **Head:** `copilot/add-batching-mechanism-to-rebuild-client-associati`
---
### 📝 Commits (2)
- [`45158ff`](https://github.com/fosrl/pangolin/commit/45158ff45bbc7e70aa9174056345c9b91066b9f9) feat: batch redis ws direct messages and dedupe rebuild queue jobs
- [`b66140b`](https://github.com/fosrl/pangolin/commit/b66140bfd2c61dce94b16ff5f237239493cbd835) refactor: tighten ws batch typing and queue cleanup logging
### 📊 Changes
**3 files changed** (+170 additions, -27 deletions)
<details>
<summary>View changed files</summary>
📝 `server/private/lib/rebuildQueue.ts` (+29 -0)
📝 `server/private/routers/ws/ws.ts` (+118 -18)
📝 `server/routers/ws/types.ts` (+23 -9)
</details>
### 📄 Description
Rebuild association paths can emit thousands of websocket sends in bursts, which was creating Redis/websocket backpressure. This update introduces batched cross-node direct delivery and queue-level deduplication for rebuild jobs so duplicate queued work is dropped while preserving re-queue behavior for in-progress jobs.
- **Websocket cross-node batching (direct sends)**
- Added Redis `direct-batch` message support and typed it in the shared WS message contract.
- Replaced per-message Redis publish for remote direct sends with buffered batching (size `250`, short flush interval).
- Added subscriber-side bulk handling to fan out batched direct messages efficiently to local connected clients.
- **Queue deduplication semantics for rebuilds**
- Added queued-job dedupe using Redis set membership keyed by `type:id`.
- Skip enqueue when an identical job is already queued.
- Remove dedupe key at dequeue time (not completion), so if a job is currently in progress, a new same-type/id job can still be queued.
- **Type/contract tightening**
- Refined `RedisMessage` into a discriminated union for `direct`, `direct-batch`, and `broadcast`, making batch payload shape explicit.
```ts
// queued dedupe behavior
const dedupeKey = `${job.type}:${job.id}`;
const added = await redis.sadd(QUEUED_SET_KEY, dedupeKey);
if (added === 0) return; // already queued
await redis.rpush(QUEUE_KEY, JSON.stringify(job));
// allow requeue while in-progress
const payload = await redis.lpop(QUEUE_KEY);
await redis.srem(QUEUED_SET_KEY, `${job.type}:${job.id}`);
```
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/fosrl/pangolin/pull/3294
Author: @Copilot
Created: 6/16/2026
Status: 🔄 Open
Base:
queue← Head:copilot/add-batching-mechanism-to-rebuild-client-associati📝 Commits (2)
45158fffeat: batch redis ws direct messages and dedupe rebuild queue jobsb66140brefactor: tighten ws batch typing and queue cleanup logging📊 Changes
3 files changed (+170 additions, -27 deletions)
View changed files
📝
server/private/lib/rebuildQueue.ts(+29 -0)📝
server/private/routers/ws/ws.ts(+118 -18)📝
server/routers/ws/types.ts(+23 -9)📄 Description
Rebuild association paths can emit thousands of websocket sends in bursts, which was creating Redis/websocket backpressure. This update introduces batched cross-node direct delivery and queue-level deduplication for rebuild jobs so duplicate queued work is dropped while preserving re-queue behavior for in-progress jobs.
Websocket cross-node batching (direct sends)
direct-batchmessage support and typed it in the shared WS message contract.250, short flush interval).Queue deduplication semantics for rebuilds
type:id.Type/contract tightening
RedisMessageinto a discriminated union fordirect,direct-batch, andbroadcast, making batch payload shape explicit.🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.