Skip to content

feat(plugins): hosted Tabularium registry + connection catalogue (fully BC)#299

Open
NewtTheWolf wants to merge 23 commits into
TabularisDB:mainfrom
NewtTheWolf:feat/registry-integration
Open

feat(plugins): hosted Tabularium registry + connection catalogue (fully BC)#299
NewtTheWolf wants to merge 23 commits into
TabularisDB:mainfrom
NewtTheWolf:feat/registry-integration

Conversation

@NewtTheWolf

@NewtTheWolf NewtTheWolf commented Jun 7, 2026

Copy link
Copy Markdown
Collaborator

What

Wires up the hosted Tabularium registry (registry.tabularis.dev) and rebuilds
the New Connection flow around a driver catalogue — without breaking the old
registry format.

70 files · +6311 / −314

Backend

  • Hosted registry + merge — new HTTP API catalogue merged with the legacy
    registry (union, API wins on id conflict), so old plugins (duckdb, …) stay
    visible. (registry.rs, commands.rs)
  • Release integrity — Ed25519 JWS verification against the registry's JWKS,
    with release-identity guards; the verified hash is what the installer checks
    the downloaded asset against. Unsigned legacy releases warn rather than
    refuse, since no live release is signed yet. (integrity.rs)
  • Deep-link installtabularis://install handler, version-aware via
    semver: Install / Update / already-installed. (deep_link.rs)
  • Tracked downloads — assets resolve through the registry redirect endpoint
    (/latest unpinned, /releases/{version} pinned) so installs get counted.
    (tabularium.rs)
  • Removable BC layer — all compat code in compat.rs, tagged
    COMPAT(registry-ga): at every call site. Legacy registry.json install +
    listing and legacy manifest.json still work; deletable once all plugins
    have migrated.

Frontend

  • Connection catalogue as step 1 of New Connection: search, paradigm
    facets, engine cards, version picker, install-gated uninstalled drivers with
    inline install. (ConnectionCatalogue, EngineCard, InstallGate, …)
  • PluginInstallConfirmModal renders the deep-link action (install / update /
    already-installed) and badges the release's signature status.
  • Plugins tab — the Installed tab now offers the update it advertises,
    gated on the release actually being installable (platform + app version), not
    just on a newer version existing.
  • i18n across all 8 locales.

Registry 0.13 compatibility

0.13 gates GET /api/plugins on a manifest having resolved at ingest, while
/{slug} stays ungated — a plugin can be invisible in the catalogue but still
resolvable. Installed-but-unlisted plugins are re-resolved by slug so they keep
their update path; dropping out of the list is not a deletion. Unknown slugs are
skipped, and when the API is down we don't fire lookups that cannot succeed.

manifest_raw needs no client work: no forge round-trip exists to drop, and the
null (unsigned legacy) branch is already handled — which is the only branch
live data currently exercises.

Fixes found while testing against the live registry

  • Legacy-only plugins were stamped with the API base, so their card linked to a
    <base>/plugins/<id> that 404s instead of falling back to their homepage.
  • has_manifest only knew .tabularium while read_manifest already had the
    legacy fallback, so manifest.json bundles were rejected on install and
    silently skipped by list_installed. Ten of the thirteen catalogue plugins
    still ship manifest.json.
  • A picked non-latest version survived its own install and pinned the card:
    neither "up to date" nor updatable, with the picker hidden — no way out.

Compatibility

Fully backwards-compatible: the legacy registry format and manifest.json are
still consumed. The in-repo plugins/registry.json seed is replaced by
tabularium-extensions.schema.json; the legacy format is read from the hosted
source via the BC layer.

Tests

Backend 940 ✓ · Frontend 2867 ✓ · typecheck + lint clean

Not covered

  • Signed installs are untested against real data. No live plugin on
    registry.tabularis.dev returns a jws — every install currently takes the
    unsigned legacy branch. The JWS path has unit coverage (tampered payload,
    wrong identity, unknown kid, malformed JWS) but no live exercise. This is a
    re-ingest problem on the registry side, not a client one.
  • Two published plugins ship inconsistent assets (cloudflare-d1@0.1.0 has no
    manifest in its archive; redis@0.4.1 ships a 0.4.0 manifest). Both are
    publisher-side; the client reports them correctly.

tabularium-sdk 0.2 exposes verified/extensions fields used by the
registry integration. semver is used for version-aware install
classification (install vs update vs already-installed).
…C layer

Integrate the hosted Tabularium registry (registry.tabularis.dev) alongside
the legacy static registry.json, with version-aware deep-link install.

- Merge new HTTP API catalogue with legacy registry.json (union, API wins
  on id conflict) so old plugins (duckdb, ...) stay visible.
- Semver-aware install classification: install / update / already-installed.
- Tracked asset downloads via the registry redirect endpoint:
  /api/plugins/{slug}/latest for unpinned, /releases/{version} for pinned.
- Host manifest parser migrated to .tabularium (JSON), legacy manifest.json
  read as fallback.

All backwards-compat code lives in plugins/compat.rs and is marked with
`COMPAT(registry-ga):` at every call site, so it can be removed mechanically
once all plugins have migrated to the hosted registry.
Redesign the New Connection flow around a searchable driver catalogue that
merges built-in drivers with registry plugins.

- EngineCard, ConnectionCatalogue (search + paradigm facets + grid),
  DriverVersionPicker, InstallBanner, InstallGate components.
- useConnectionCatalogue hook + connectionCatalogue normalization model.
- Catalogue is step 1 of the New Connection wizard; uninstalled drivers are
  install-gated with inline driver install.
PluginInstallConfirmModal now renders the install action returned by the
preview command: Install, Update (when a newer version exists), or an
already-installed banner for an up-to-date plugin.

- useDeepLinkInstall hook drives the tabularis://install flow.
- PluginsTab consumes the registry base URL per plugin for its registry link.
Add registry/catalogue and deepLink (updateConfirm, alreadyInstalled)
strings across all 7 locales.
- NewConnectionModal: remove unused PluginManifest import.
- PluginInstallConfirmModal: key the modal by request in App.tsx so each
  request remounts with fresh state, replacing the synchronous state reset
  inside the effect (react-hooks/set-state-in-effect). Loading state now
  initialises from the keyed mount; the effect only sets state in its async
  callbacks.
Comment thread src-tauri/src/plugins/commands.rs Outdated
Comment thread src-tauri/src/plugins/registry.rs Outdated
Comment thread src/hooks/useConnectionCatalogue.ts Outdated
Comment thread src/components/modals/PluginInstallConfirmModal.tsx Outdated
@kilo-code-bot

kilo-code-bot Bot commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Files Reviewed (2 files)
  • src-tauri/src/plugins/commands.rs
  • src-tauri/src/plugins/compat.rs
Previous Review Summaries (4 snapshots, latest commit d7b9dbb)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit d7b9dbb)

Status: No Issues Found | Recommendation: Merge

All 4 previously identified issues have been resolved in commit 4d4368c.

Resolved Issues
File Line Issue Status
src-tauri/src/plugins/commands.rs 45 Static registry plugins no longer receive broken detail-page links ✅ Fixed
src-tauri/src/plugins/registry.rs 185 Detail fetches now use futures::future::join_all for parallelism ✅ Fixed
src/hooks/useConnectionCatalogue.ts 71 Replaced in operator with Object.prototype.hasOwnProperty.call ✅ Fixed
src/components/modals/PluginInstallConfirmModal.tsx 70 Empty version string now treated as nullish via `
Files Reviewed (4 files)
  • src-tauri/src/plugins/commands.rs
  • src-tauri/src/plugins/registry.rs
  • src/hooks/useConnectionCatalogue.ts
  • src/components/modals/PluginInstallConfirmModal.tsx

Previous review (commit 9ff1fdd)

Status: No Issues Found | Recommendation: Merge

All 4 previously identified issues have been resolved in commit 4d4368c.

Resolved Issues
File Line Issue Status
src-tauri/src/plugins/commands.rs 45 Static registry plugins no longer receive broken detail-page links ✅ Fixed
src-tauri/src/plugins/registry.rs 185 Detail fetches now use futures::future::join_all for parallelism ✅ Fixed
src/hooks/useConnectionCatalogue.ts 71 Replaced in operator with Object.prototype.hasOwnProperty.call ✅ Fixed
src/components/modals/PluginInstallConfirmModal.tsx 70 Empty version string now treated as nullish via `
Files Reviewed (4 files)
  • src-tauri/src/plugins/commands.rs
  • src-tauri/src/plugins/registry.rs
  • src/hooks/useConnectionCatalogue.ts
  • src/components/modals/PluginInstallConfirmModal.tsx

Previous review (commit d113fa4)

Status: No Issues Found | Recommendation: Merge

All 4 previously identified issues have been resolved in commit 4d4368c.

Resolved Issues
File Line Issue Status
src-tauri/src/plugins/commands.rs 45 Static registry plugins no longer receive broken detail-page links ✅ Fixed
src-tauri/src/plugins/registry.rs 185 Detail fetches now use futures::future::join_all for parallelism ✅ Fixed
src/hooks/useConnectionCatalogue.ts 71 Replaced in operator with Object.prototype.hasOwnProperty.call ✅ Fixed
src/components/modals/PluginInstallConfirmModal.tsx 70 Empty version string now treated as nullish via `
Files Reviewed (4 files)
  • src-tauri/src/plugins/commands.rs
  • src-tauri/src/plugins/registry.rs
  • src/hooks/useConnectionCatalogue.ts
  • src/components/modals/PluginInstallConfirmModal.tsx

Previous review (commit 2cbb015)

Status: No Issues Found | Recommendation: Merge

All 4 previously identified issues have been resolved in commit 4d4368c.

Resolved Issues
File Line Issue Status
src-tauri/src/plugins/commands.rs 45 Static registry plugins no longer receive broken detail-page links ✅ Fixed
src-tauri/src/plugins/registry.rs 185 Detail fetches now use futures::future::join_all for parallelism ✅ Fixed
src/hooks/useConnectionCatalogue.ts 71 Replaced in operator with Object.prototype.hasOwnProperty.call ✅ Fixed
src/components/modals/PluginInstallConfirmModal.tsx 70 Empty version string now treated as nullish via `
Files Reviewed (4 files)
  • src-tauri/src/plugins/commands.rs
  • src-tauri/src/plugins/registry.rs
  • src/hooks/useConnectionCatalogue.ts
  • src/components/modals/PluginInstallConfirmModal.tsx

Reviewed by kimi-k2.6 · Input: 62.2K · Output: 9.1K · Cached: 219.1K

…y version)

- commands.rs: only set registry_base_url for a real Tabularium API, not a
  legacy `.json` base (avoids broken `…/registry.json/plugins/<id>` links).
- registry.rs: fetch plugin details concurrently via join_all instead of N
  sequential round-trips.
- useConnectionCatalogue: use hasOwnProperty instead of `in` so plugin ids
  like "constructor"/"toString" aren't matched against Object.prototype.
- PluginInstallConfirmModal: treat empty `?version=` as no pin (|| null) for
  both the preview fetch and the displayed target version.
Comment thread src-tauri/src/plugins/compat.rs
Comment thread src/utils/connectionCatalogue.ts Outdated
…catalogue sort

Addresses PR review (debba):
- Restore plugins/registry.json instead of deleting it. LEGACY_REGISTRY_URL
  points at this file on main; deleting it would 404 after merge, silently
  dropping un-migrated plugins from resolve_registry's merge and breaking
  already-shipped app versions that use it as their only registry source.
  Prune only when the whole COMPAT(registry-ga) layer is removed.
- Sort engine groups builtins-first then other-last (then alphabetical), so the
  catalogue leads with the SQL section and pushes the metadata-less 'Other'
  section to the end instead of opening on a wall of legacy plugins.
…-ready CI

- scaffold emits .tabularium (was manifest.json); keep id=slug/name=display
  so the host identity/display split is preserved
- add `migrate [path]` to convert legacy manifest.json plugins to .tabularium;
  drops a redundant id (id===name), refuses without a semver version
- `migrate --ci` regenerates release.yml from the registry-ready template
  (BIN_NAME from manifest.executable); without --ci the workflow is left as-is,
  only its manifest.json reference is renamed
- release.yml template: stash artifacts + single release job publishing
  .tabularium as a standalone asset (Tabularium registry resolves the manifest
  from release assets); assets/ dir support; action version bumps
- plugin-api: doc comments reference .tabularium
- bump @tabularis/create-plugin 0.1.1 -> 0.2.0
@NewtTheWolf

NewtTheWolf commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator Author

@debba small follow-up on this branch (2cbb015) — I aligned packages/ with the new .tabularium host contract and added tooling so plugin authors can keep up:

@tabularis/create-plugin (0.1.1 → 0.2.0)

  • Scaffold now emits .tabularium instead of manifest.json. id=slug / name=display are deliberately kept, because the host falls back to name as the identity when id is absent (otherwise the display name would become the slug).
  • New command migrate [path]: converts existing manifest.json plugins → .tabularium, drops a redundant id (only when id===name), and refuses without a semver version (registry requirement). justfile/README/release.yml references are carried along.
  • migrate --ci: additionally regenerates release.yml from the registry-ready template (BIN_NAME derived from manifest.executable). Without --ci the CI structure is left untouched — only its manifest.json reference is renamed so the build keeps working.

Registry-ready release workflow (template)
The previous template workflow only bundled .tabularium inside the platform zips. But the hosted Tabularium registry resolves the manifest from the release assets (since 0.8.0), not from the git ref. The workflow now stashes the artifacts and, in a separate release job, uploads the zips plus .tabularium as a standalone asset (modeled on my firestore plugin CI). I deliberately left out the Codeberg publish — that's a hosting choice, not generic.

@tabularis/plugin-api: doc comments updated to reference .tabularium only.

Tests: 32 green (8 new for migrate, incl. --ci regeneration + edge cases), tsc/build clean, generated YAML validated. Happy to hear if the migrate defaults (id handling, CI only with --ci) look right to you.

@debba

debba commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

That's awesome! Thanks @NewtTheWolf !

The published dist called copyTemplate('ui-extension-locales', ...) but that
template dir never existed, so --with-ui crashed; source had dropped the call,
so no locales were emitted at all. Restore the feature properly:
- add templates/ui-extension-locales/locales/en.json.tmpl (plugin-root locales)
- scaffold copies it under --with-ui
- ui template uses usePluginTranslation instead of a hardcoded string
- smoke asserts locales/en.json exists for --with-ui
…tion

# Conflicts:
#	src/components/modals/NewConnectionModal.tsx
…iver support

Surface registry engine/paradigms on manifests so locally-installed (not
yet published) plugin drivers appear in the connection picker instead of
loading invisibly. Add single_database capability to skip the database
tab/field for flat stores (e.g. Meilisearch).
…currency

WIP — review fixes for the hosted Tabularium registry. Not final;
JWS-signed install verification still pending (registry Phase-2 not released).

Frontend:
- i18n the connection catalogue components (ConnectionCatalogue, EngineCard,
  InstallGate, DriverVersionPicker) — were hardcoded English; add
  connectionCatalogue.* keys to all 7 locales + backfill changeDatabase /
  deepLink.updateConfirm / deepLink.alreadyInstalled
- delete dead InstallBanner (+ test); move InstallStatus into InstallGate
- EngineCard: aria-label uses displayName; surface platform-unsupported
  drivers via EngineGroup.platformSupported (badge + dim) instead of a
  click-through dead end
- useConnectionCatalogue: move setLoading out of the synchronous effect body

Backend:
- registry: bound the plugin-detail fan-out with buffered(8) instead of an
  uncapped join_all over up to 200 plugins
- deep_link: parse the slug from the no-`//` tabularis:install/<slug> form
- lib: handle a warm-launch deep link forwarded via argv (single-instance)

Tests updated to the key-echo i18n mock convention.

Still open (not in this commit): JWS install trust model, EngineGroup test
churn, cold-start double-dispatch.
…ration

# Conflicts:
#	src-tauri/Cargo.lock
#	src/App.tsx
#	src/components/modals/NewConnectionModal.tsx
#	src/i18n/locales/de.json
#	src/i18n/locales/en.json
#	src/i18n/locales/es.json
#	src/i18n/locales/fr.json
#	src/i18n/locales/it.json
#	src/i18n/locales/ja.json
#	src/i18n/locales/zh.json
#	tests/setup.ts
The merge with main left `cargo test` uncompilable: the test helper builds
PluginManifest with every field spelled out, and this branch added `engine`
and `paradigms` to the struct. The lib itself built fine, so only the test
profile broke.
During the registry transition the catalogue merges the Tabularium API with
the legacy static registry.json. merge_registries appends legacy-only
plugins and drops the distinction, after which fetch_plugin_registry stamped
`registry_base_url` onto every entry — its guard only checked whether the
configured base was a `.json`, which never holds in the merge case.

A not-yet-migrated plugin (e.g. redis) therefore claimed to be served by the
API, and the frontend linked its card to `<base>/plugins/redis`, which 404s.
Since registryPageUrl wins over the upstream homepage, the real homepage was
never reached.

Stamp the base in resolve_registry instead, where it's still known whether a
plugin came from the API or the legacy file. Legacy entries stay unstamped
and fall back to their homepage on their own.
read_manifest already falls back to the legacy manifest.json, but
has_manifest only looked for .tabularium. Install and list gate on
has_manifest first, so a not-yet-migrated bundle (e.g. redis) was rejected
as "does not contain a .tabularium manifest" — and, once installed, skipped
by list_installed — even though the read path would have loaded it fine.

Make the presence check agree with the read path via compat::has_legacy_
manifest, so both callers are fixed at the shared gate rather than each
symptom. Both disappear together at the cutover.
The Installed tab renders from driver data and showed the amber
update badge (which comes from the registry), but had no way to act on it —
the install button only exists on the catalogue cards. Users saw the problem
and were left without the fix.

Add the update button next to the existing per-plugin actions, gated by
canUpdateToLatest: update_available alone only means a newer version exists,
so the release must also ship a build for this platform and support this app
version. Notably the catalogue degrades to the release-less list item when a
detail fetch fails, where update_available stays true — the releases check
keeps us from offering an update we cannot perform.
Picking a non-latest version (e.g. firestore 0.3.9) left it in
selectedVersions after the install. The card then pinned itself to what was
already on disk: not "up to date" (it isn't latest) and not updatable
(isSelectedInstalled), so the action area rendered empty. The version picker
hides itself once only one other release remains, so there was no way back —
the update badge stayed on with nothing to click.

The picked version is spent once installed; clear it so the card falls back
to offering the latest release again.
Registry 0.13 gates GET /api/plugins on a manifest having resolved at ingest,
so approved is no longer enough to be listed. Detail and latest are not gated:
a plugin can be invisible in the catalogue while its slug still resolves.

The catalogue only ever iterated the listed set, so an installed plugin whose
publisher shipped a release with an unresolvable .tabularium would silently
lose its update path — no badge, no button, no error. Dropping out of the list
is not a deletion.

Re-resolve installed-but-unlisted plugins by slug and fold them back in.
Unknown slugs (locally-installed, legacy-only) are skipped quietly, and when
the API itself is down we return the legacy set as-is rather than firing
lookups that cannot succeed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants