3 people 2c7f3c7a3d [AI] Replace google-protobuf with @bufbuild/protobuf (#7535) ...

* [AI] crdt: typecheck test files and clean up lint issues

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [AI] Replace google-protobuf with @bufbuild/protobuf

Swap the google-protobuf + ts-protoc-gen + protoc-gen-js toolchain for
@bufbuild/protobuf + @bufbuild/protoc-gen-es. The generator now emits a
single pure-TS sync_pb.ts (no .js sidecar, no globalThis.proto hack)
and a thin wrapper in proto/compat.ts preserves the SyncProtoBuf /
SyncRequest / etc. API so call sites stay unchanged. Removes the
loot-core CommonJS require polyfill that only existed to service
google-protobuf.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [AI] Align @bufbuild/protobuf version ranges with installed

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [AI] crdt: drop the SyncProtoBuf compat layer

The proto/compat.ts wrapper was introduced alongside the bufbuild
migration to avoid touching call sites. With bufbuild messages already
exposing fields as plain mutable properties, the wrapper was just
boilerplate hiding direct reads and writes — and it had drifted (e.g.
setMessagesList was called in a test but never defined).

Delete compat.ts and migrate the six call sites in loot-core and
sync-server to use @bufbuild/protobuf directly. The crdt package now
re-exports the sync_pb types/schemas and the three bufbuild runtime
helpers (create, fromBinary, toBinary) so consumers keep a single
import source.

Also switch sync-server's @actual-app/crdt dependency from the pinned
"2.1.0" to "workspace:*", matching api/loot-core — the npm pin was
pulling the stale published copy instead of the workspace source.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [AI] CI: drive sync-server build through lage so crdt deps are built

Before: the server job ran `yarn workspace @actual-app/sync-server build`
directly, which invokes tsgo without first emitting the workspace
dependencies' declarations. That worked when sync-server pinned crdt to
the published npm version (declarations bundled in the tarball), but
with `workspace:*` it fails with TS6305 because packages/crdt/dist/*.d.ts
hasn't been built yet.

Switch the CI command to `yarn build --to=@actual-app/sync-server`.
Lage respects the `dependsOn: ['^build']` pipeline and builds
@actual-app/crdt (and the other transitive deps) before sync-server.

Using --to rather than --scope keeps the build set minimal; --scope
would also include dependents like desktop-electron.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [AI] sync-server: build project references via tsgo -b

The build script ran plain `tsgo`, which doesn't compile referenced
projects. With @actual-app/crdt now a `workspace:*` dep (no bundled
declarations from the npm tarball), the sync-server build fails with
TS6305 because packages/crdt/dist/index.d.ts doesn't exist yet.

Switch to `tsgo -b` so the sync-server build is self-contained: it
emits crdt's declarations into packages/crdt/dist on demand. This
mirrors what the sync-server `typecheck` script already does and fixes
all callers (`build:server`, docker-edge, publish workflows, the
direct `yarn workspace @actual-app/sync-server build` invocation in
build.yml) without needing per-workflow lage orchestration.

Revert the build.yml workaround added in the previous commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [AI] sync-server: build @actual-app/crdt before tsgo

The previous tsgo -b approach emitted crdt's .d.ts via the project
reference but never produced dist/index.js — tsgo respects crdt's
tsconfig which has emitDeclarationOnly: true, and the actual JS
runtime is emitted by Vite in crdt's build script. So sync-server
compiled cleanly but crashed at runtime when forked by desktop-electron
(require('@actual-app/crdt') resolved to a package whose main pointed
at a nonexistent file, surfaced in e2e as the onboarding screen never
leaving the "Configure your server" state).

Unlike packages/api (which uses Vite with noExternal: true and bundles
crdt's source inline), sync-server uses plain tsgo compilation and
keeps its deps external — so crdt must be built ahead of time and be
resolvable via node_modules at runtime.

Chain `yarn workspace @actual-app/crdt build` before tsgo so every
caller of sync-server's build (build:server, docker-edge, publish
workflows, direct invocations in CI) gets a complete crdt dist. Revert
tsgo -b back to plain tsgo since crdt's build step now emits both the
JS and the declarations.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [AI] crdt: expose dist/ via conditional exports so Node can load it

The package's `exports` field pointed straight at `./src/index.ts`,
which works for TS tooling and bundlers (vite with noExternal, vitest)
but breaks at plain-Node runtime — Node can't execute `.ts` files and
resolves dependent `./crdt` as a directory import, failing with
ERR_UNSUPPORTED_DIR_IMPORT.

That was invisible before because sync-server pinned
`@actual-app/crdt@2.1.0` and ran against the published npm tarball
(whose `publishConfig.exports` had already been promoted to the main
`exports` by yarn pack). Switching sync-server to `workspace:*` made
the raw workspace exports win at runtime: the compiled server imported
crdt when desktop-electron forked it, Node hit the `.ts` entry, the
utility process crashed before emitting `server-started`, and the
onboarding flow stalled on "Configure your server".

Switch to the same conditional-exports pattern packages/api already
uses: types → dist/index.d.ts, development → src/index.ts (for vitest
runs that enable the `development` condition), default → dist/index.js
(Node runtime and any other consumer). `publishConfig.exports` still
collapses this to just types + default for the npm tarball.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [AI] crdt: split exports per consumer (browser source, node dist)

Previous commit's conditional exports routed everything non-development
to ./dist/index.js. That broke the web build: rolldown runs with
conditions ['electron-renderer', 'module', 'browser', 'default'] — no
match for development, falls through to the dist entry, which isn't
built by bin/package-browser, and fails to resolve @actual-app/crdt
when bundling loot-core's server/undo.ts.

Split the entries so each consumer lands on the right artifact:

  types       → ./dist/index.d.ts   (TypeScript, project references)
  development → ./src/index.ts      (vitest — both configs include it)
  browser     → ./src/index.ts      (web rolldown bundles the source)
  node        → ./dist/index.js     (sync-server forked by Node at
                                     runtime — the failure that kicked
                                     off this whole saga)
  default     → ./src/index.ts      (fallback for bundlers like api's
                                     vite build with conditions=['api'])

Verified: node resolves to dist, yarn build:browser succeeds from a
clean crdt/, sync-server build produces both dist/index.js and
build/app.js, loot-core (552) + sync-server (386) tests pass, full
typecheck clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [AI] address review feedback on crdt/sync-server

- generate-proto: add `set -euo pipefail` so a protoc failure exits the
  script non-zero instead of silently running oxfmt on whatever is in
  src/proto/ from the previous run.
- sync.proto SyncRequest: field numbers jumped from 3 to 5; declare
  `reserved 4;` so the slot can't be silently reused for a new field
  with an incompatible type. Regenerated sync_pb.ts — the reservation
  shows up in the encoded file descriptor.
- sync-simple.js: SQLite stores is_encrypted as a 0/1 integer and
  better-sqlite3 hands it back as a number, but the bufbuild
  MessageEnvelope schema types isEncrypted as bool. Coerce to boolean
  when constructing the envelope so the JS value matches the field
  type before toBinary runs.

Skipped the suggested `types` → ./src/index.ts swap in crdt's exports:
packages/api uses the same `types` → dist pattern and TypeScript's
bundler resolution already falls through when dist/*.d.ts doesn't yet
exist (verified — loot-core typecheck passes with packages/crdt/dist
removed).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [AI] address review feedback on encoder/app-sync test

- encoder.ts: prefs.getPrefs().encryptKeyId is `string | undefined`
  (MetadataPrefs is a Partial<>). The bufbuild SyncRequestSchema's
  keyId field is a non-optional proto3 string. Current code worked by
  accident — passing undefined into `create(Schema, init)` falls back
  to the schema default '' — but relied on bufbuild's undef-handling
  and would break if someone dropped @ts-strict-ignore. Normalize to
  '' explicitly.
- app-sync.test.ts: add a short WHY comment next to
  `syncRequest.since = ''` in "returns 422 if since is not provided".
  The test's intent (missing since) only matches the handler's
  `requestPb.since || null` falsy-check because proto3 strips '' on
  the wire and decodes it back to ''. Not obvious without the comment.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [AI] crdt: load source directly in dev, only use dist when published

Local exports point at src/index.ts so consumers (sync-server in
particular) never load a stale Vite bundle. publishConfig keeps the
dist/ mapping for npm consumers. Switched the Vite output to ESM and
added "type": "module" so the published bundle stays consistent.

Sync-server's existing extension-resolution loader is extended to
handle directory imports and is now registered at runtime via
--import ./register-loader.mjs, matching how tests already load it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [AI] desktop-electron: register sync-server loader on the embedded fork

The Electron app starts the sync server via utilityProcess.fork, which
bypasses sync-server's `start` script. With crdt now loaded from
source, the fork needs the same `--import register-loader.mjs` that
the standalone server uses; otherwise it crashes on the extensionless
`from './crdt'` directory import. Adds the loader files to
sync-server's published `files` so they actually ship with the
packaged app.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [AI] sync-server: bootstrap entry that registers the loader for utilityProcess

Electron's utilityProcess.fork accepts execArgv but silently ignores
--import (verified with a minimal repro: the flag shows up in
process.execArgv but the preload module never executes), so the
previous attempt was a no-op and the embedded sync-server still
crashed on crdt's ESM directory imports. Add packages/sync-server/start.mjs
that statically imports register-loader.mjs and then dynamic-imports
build/app.js, so the loader is in place before the app's module graph
resolves. desktop-electron now points utilityProcess.fork at start.mjs
and drops the ineffective --import flag.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-12 17:55:52 +00:00

..

generate-proto

…