You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Feldera authorizes API requests by tenant isolation alone: any principal that
resolves to a tenant may do anything to that tenant's resources. The read
scope is computed, stored, and attached to every request, then never checked.
This RFC introduces role-based access control with four totally ordered roles, read < write < admin < owner, enforced on every REST endpoint by a
deny-by-default, table-driven middleware. Per-(user, tenant) roles live in a new
Feldera-side membership table; the identity provider keeps deciding which
tenants a user may reach. API keys and OIDC trust relationships carry a role
capped at their creator's role. An owner is a platform operator who acts
inside any tenant by selecting it with the existing Feldera-Tenant header.
The RFC adds tenant and user management APIs plus a web-console admin UI.
Motivation
Authorization today is binary: hold a token that resolves to tenant T and you
have full read and write over everything in T (crates/pipeline-manager/src/auth.rs
inserts vec![Read, Write] on the login path. No handler
ever reads the permission vector back). This blocks several routine needs:
A monitoring or on-call user/service who should watch pipelines, stats, and logs but
must not edit code, start/stop pipelines, run ad-hoc SQL, or read row data.
A CI deploy credential that should manage pipelines but not mint further
credentials or manage users (least privilege).
A per-tenant administrator who manages that tenant's users, API keys, and
trust relationships without touching other tenants.
A platform operator (the team that installs and runs Feldera) who can act in
any tenant for maintenance, for example stopping a runaway pipeline after 24h.
The machinery to carry a permission already exists end to end; it simply has no
reader. This RFC improve today's state into enforced, least-privilege access.
Reference-level explanation
1. The role lattice
Roles form a single total order. A higher role can do everything a lower role
can, plus more. There is exactly one ordered Role type with a derived Ord,
defined once and reused for every comparison.
Role
Scope
Can do
read
one tenant
Monitor and diagnose: list/inspect pipelines, view SQL/UDF code and config, stats, metrics, logs, errors, events, and profiles (and trigger diagnostics like profiling). Never sees the data a pipeline processes; never changes a pipeline's definition, lifecycle, or data.
write
one tenant
Everything read can, plus everything that touches the data a pipeline processes (ad-hoc SQL, HTTP ingress/egress) and every change to pipelines (create/edit/delete, start/stop/pause/resume, checkpoints, transactions). Mints read/write API keys.
admin
one tenant
Everything write can, plus manages that tenant's users and their roles (up to admin) and its OIDC trust relationships. Grants admin to non-human principals through an OIDC trust; admin is not issuable as a static API key.
owner
all tenants
Everything admin can, in any tenant, selected per request with Feldera-Tenant. Plus platform-wide views (the tenant list) and creating owner OIDC trusts.
The line between read and write: a reader learns how a pipeline behaves and
what it is doing, but never learns anything (besides stats) about the data it processes and never
changes anything. Anything that reads or moves row-level data (ad-hoc SQL, HTTP
ingress, HTTP egress) is therefore write.
2. Where a principal's role comes from
A request is authenticated by one of four paths. Each resolves an AuthenticatedPrincipal { acting_tenant, home_tenant, role } and inserts it
into request extensions. The acting tenant is the tenant the request operates
in; for everyone but an owner it equals the home tenant.
Path
Today
New: role source
Login JWT (bearer_auth)
hardcodes [Read, Write] (auth.rs:314)
Look up tenant_membership(acting_tenant, user); if no row, the configured default role (read); if the login created the tenant, admin; if the identity is in FELDERA_OWNERS, owner.
API key (api_key_auth)
scopes from the key row
role column on the key row.
OIDC trust (oidc_trust_auth)
scopes from the trust row
role column on the trust row. owner trusts resolve cross-tenant (section 7).
No-auth dev (tag_with_default_tenant_id)
[Read, Write] on the nil tenant
admin of the single default tenant. Cross-tenant owner is meaningless with one tenant, so admin gives full control of it. Local single-node use only.
Tenant access stays exactly as documented today
(docs.feldera.com/.../authentication/index.mdx): the identity provider grants
which tenants a user can reach through the tenants claim, the issuer domain,
or the sub claim. Feldera owns only the role within a reachable tenant. A
user an admin "adds" is a membership row that takes effect when that user next
authenticates into the tenant through the IdP.
Provisioning rule that preserves today's single-user and individual-tenancy
behavior: the first principal to resolve to a tenant that does not yet exist
becomes that tenant's admin. Subsequent principals with no membership row get
the default role (read, configurable via FELDERA_AUTH_DEFAULT_ROLE).
3. Identity and data model
No user, role, or membership table exists today; the only principal is the
tenant, keyed (tenant, provider). RBAC introduces the user and the
(user, tenant, role) link, in migration VXX__rbac.sql.
CREATETABLEapp_user (
id uuid PRIMARY KEY,
provider varcharNOT NULL, -- OIDC issuer the subject came from
subject varcharNOT NULL, -- OIDC `sub`
email varchar, -- display only, may be null
UNIQUE (provider, subject)
);
CREATETABLEtenant_membership (
tenant_id uuid NOT NULLREFERENCES tenant(id) ON DELETE CASCADE,
user_id uuid NOT NULLREFERENCES app_user(id) ON DELETE CASCADE,
role textNOT NULL, -- 'read' | 'write' | 'admin'PRIMARY KEY (tenant_id, user_id)
);
ALTERTABLE api_key ADD COLUMN role textNOT NULL DEFAULT 'read';
ALTERTABLE oidc_trust_relationship ADD COLUMN role textNOT NULL DEFAULT 'read';
-- Backfill from the existing scopes column, then drop scopes:UPDATE api_key SET role = CASE WHEN 'write'= ANY(scopes) THEN 'write' ELSE 'read' END;
UPDATE oidc_trust_relationship SET role = CASE WHEN 'write'= ANY(scopes) THEN 'write' ELSE 'read' END;
ALTERTABLE api_key DROP COLUMN scopes;
ALTERTABLE oidc_trust_relationship DROP COLUMN scopes;
owner is never stored in tenant_membership (it is a platform property,
sourced from config or an owner OIDC trust) and never on an API key. tenant_membership.role is therefore capped at admin by a CHECK or by the
write path.
Parsing safety: the current ApiPermission::from_str(...).expect(...) sites
(db/operations/api_key.rs:30,52,141; oidc_trust.rs:23,180) panic on any
unknown string. Role::from_str must be total and an unknown value must fail
the request, never the process, because these parse inside the auth path.
New Storage methods (each following the existing trait / postgres / operations
trio): get_or_create_user, list_tenant_members, get_member_role, upsert_member_role, remove_member, list_tenants, get_tenant_id_by_name.
4. Enforcement architecture
request ─▶ auth middleware (auth_validator)
puts AuthenticatedPrincipal{acting_tenant, home_tenant, role} in extensions
─▶ rbac middleware
min = ROUTE_MIN_ROLE[(method, matched_route_pattern)] // deny if absent
if principal.role < min ─▶ 403 InsufficientPermissions
─▶ handler (reads acting_tenant via ReqData; queries WHERE tenant_id = acting_tenant)
Enforcement is a single middleware over the /v0 scope, driven by one static
table keyed on (method, actix matched route pattern). Two properties matter:
Deny-by-default. A route absent from the table is rejected, so a newly added
endpoint fails until classified. A meta-test should enumerate the routes
registered in api_scope() and fails the build if any has no entry.
The route table is the access-control table in section 5. The document and
the code share one source of truth.
HTTP method is not used as the read/write signal because POST /start, /stop, /pause are mutations and some queries are GET. Classification is
per route, not per verb.
A new DBError::InsufficientPermissions { required: Role } maps to HTTP 403
(db/error.rs: add the Display, error_code, and status_code arms; no 403
exists today). It flows to the wire through the existing From<DBError> for ManagerError and ResponseError.
Tenant isolation is unchanged: handlers still read one TenantId (now acting_tenant) and every query still filters WHERE tenant_id = $1. RBAC adds
a vertical check (role) on top of the existing horizontal check (tenant).
5. The access-control table
The minimum role to reach each endpoint. public endpoints are reachable before authentication and are not gated by any role.
Pipeline management
Method
Path
Min role
Purpose / nuance
GET
/v0/pipelines
read
List pipelines in the tenant (code, config, status). No access to pipeline data.
GET
/v0/pipelines/{pipeline_name}
read
Retrieve one pipeline (code, config, status). No access to pipeline data.
POST
/v0/pipelines
write
Create a new pipeline from the supplied SQL/config.
PUT
/v0/pipelines/{pipeline_name}
write
Upsert: fully update an existing pipeline or create it if absent.
PATCH
/v0/pipelines/{pipeline_name}
write
Partially update a pipeline's main fields (code, config, name, etc.).
POST
/v0/pipelines/{pipeline_name}/update_runtime
write
Recompile/bump a pipeline to the current platform (runtime) version.
DELETE
/v0/pipelines/{pipeline_name}
write
Delete a (fully stopped and cleared) pipeline by name.
POST
/v0/pipelines/{pipeline_name}/start
write
Asynchronously start a pipeline (initial=running/paused/standby), with bootstrap/dismiss_error options.
POST
/v0/pipelines/{pipeline_name}/stop
write
Asynchronously stop a pipeline; force=false checkpoints/suspends first, force=true deprovisions immediately.
POST
/v0/pipelines/{pipeline_name}/dismiss_error
write
Clear the pipeline's deployment_error so a subsequent start can proceed.
POST
/v0/pipelines/{pipeline_name}/clear
write
Asynchronously clear pipeline storage (disassociates and may delete the storage).
GET
/v0/pipelines/{pipeline_name}/logs
read
Stream a pipeline's logs (catch-up from circular buffer, then live tail).
POST
/v0/pipelines/{pipeline_name}/testing
write
Test-harness only (unstable feature); never runs outside testing, so write is fine.
GET
/v0/pipelines/{pipeline_name}/events
read
List recent pipeline monitor events (status history) in reverse chronological order.
GET
/v0/pipelines/{pipeline_name}/events/{event_id}
read
Get a single pipeline monitor event by id (or 'latest').
Pre-auth carve-out: the login page needs it before a token exists.
GET
/v0/config/demos
read
List the demos available in the WebConsole.
GET
/v0/config/session
read
Get current session info (tenant id and tenant name) for the calling user.
GET
/v0/metrics
read
Aggregate Prometheus metrics from all running/paused pipelines of the tenant by proxying each pipeline's /metrics.
GET
/v0/cluster/events
read
Platform telemetry (not tenant-scoped); could be restricted to owner pending discussion.
GET
/v0/cluster/events/{event_id}
read
Platform telemetry (not tenant-scoped); could be restricted to owner pending discussion.
GET
/v0/cluster_healthz
public
Liveness probe; stays reachable pre-auth. Detailed self_info clamped to owner.
6. Endpoints that behave differently by role
The mapping above flagged endpoints whose correct behavior depends on the
caller's role. Role-dependent behavior in endpoints is an API smell;
the resolutions below keep each endpoint single-purpose and make the difference
explicit and minimal rather than scattered through handler bodies.
Endpoint
Issue
Resolution
GET /v0/cluster/events, /v0/cluster/events/{id}, GET /v0/cluster_healthz
Not tenant-scoped: platform-wide telemetry exposed to any tenant user.
Kept read for now; cluster_healthz stays a public liveness probe. Restricting platform telemetry to owner depends on nature of data exposed here (see unresolved questions).
GET /v0/pipelines/{name}/query (ad-hoc SQL)
A GET that returns and can change the data a pipeline processes.
Requires write. A reader should not learn anything about processed data.
GET .../support_bundle, POST/GET .../samply_profile
Diagnostics that collect fresh data.
Kept read: they expose nothing a read user cannot already obtain. Whether all debug/collect endpoints should require write is an open question.
POST /v0/pipelines/{name}/testing
Test-harness hook (force-sets platform_version, gated by an unstable feature).
write: can be ignored, only runs in testing.
GET .../checkpoint_status, .../checkpoints
Documented as reads but call increment_notify_counter, mutating on a GET.
Stay read; the counter bump is an internal notify, acknowledged as benign.
GET /v0/config/authentication
Declares no security() and is mounted pre-auth; the login page needs it.
Stays public, carved out of the role table explicitly.
7. API keys, OIDC trust, and the mint cap
Both (API, OIDC trust) creations gain a role field; the hardcoded vec![Read, Write]
is removed. The minting rule, enforced in the handler and asserted again in the storage
layer (defense in depth):
minted_role <= caller.role else 403 (no silent downgrade)
API key: minted_role in {read, write} (admin and owner are never static secrets)
OIDC trust: minted_role in {read, write, admin}; owner only by a platform owner
A leaked static credential is much easier to happen than a leaked
interactive session, so admin and owner are expressible only as
signature-verified principals (a login JWT or an OIDC trust).
API keys carry only read or write. This is encoded as a type split: a MintableKeyRole (read or write) distinct from Role, so an admin or owner literal cannot reach store_api_key_hash. This allows development
to still test/script workflows using python/CLI etc.
OIDC trust creation is admin, not write: minting a durable login credential
for an external identity is identity administration. The trust's role is capped
at the creator's role (an admin may create up to an admin trust; only an owner
creates owner trusts). Wildcard-only subject/audience patterns are rejected
for any role above read, so a tenant admin cannot mint a "matches everyone"
trust. Granting admin to a non-human principal therefore goes through an OIDC
trust, surfaced in the admin UI, rather than a static key.
API keys and OIDC trusts are per-tenant principals (service accounts), not
people. That raises the question of whether minting them should require admin
at all, rather than write minting its own keys; this RFC keeps write
key-minting for now and revisits it in unresolved questions.
8. Owner and cross-tenant action
The first owner is bootstrapped from configuration: FELDERA_OWNERS lists
owner identities (provider:subject, or email). A login or OIDC-trust principal
that matches the list is granted owner. This breaks the chicken-and-egg
problem (creating an owner needs an owner) without a database seed. Existing
owners can then create additional owner OIDC trusts in the admin UI for
automation, for example the CI identity that runs the install.
An owner selects the tenant to act in with the existing Feldera-Tenant header.
This reuses the header, and MissingTenantHeader machinery already present
per-request UX (a web-console dropdown or fda --tenant X).
It is preferred over encoding the target tenant in the aud
claim (issue #6330): aud already carries audience meaning on the login path,
and a token-per-tenant defeats caching and forces a mint per target.
owner JWT / owner OIDC trust (signature-verified)
│ role = owner
▼
Feldera-Tenant: <name or uuid>
│ strict SELECT id FROM tenant WHERE ... (get_tenant_id_by_name, never get_or_create)
▼ 404 UnknownTenant on miss; never auto-create
AuthenticatedPrincipal { acting_tenant = resolved, home_tenant = owner's own, role = owner }
▼
handlers query WHERE tenant_id = acting_tenant (isolation invariant intact)
The Feldera-Tenant header always selects the acting tenant. A non-owner may
select only a tenant the IdP already authorizes for them (the tenants claim,
today's behavior at auth.rs), and their role is then looked up for that
selected tenant: a user can be admin in tenant A and read in tenant B. Only
an owner may select a tenant outside their authorized set; that widening is the
owner-only privilege, evaluated before the acting tenant is written to
extensions. Owner resolution is a strict lookup that errors on miss, never get_or_create_tenant_id, so a typo cannot silently create an empty tenant.
Every action is audit-logged with (user, target tenant, method, path), owner
cross-tenant actions especially.
owner OIDC trusts cannot be created by an ordinary tenant admin: they live in
a reserved platform tenant and are creatable only by an owner.
For an ordinary (non-owner) trust, the aud claim is the mechanism that maps a
federated token to one tenant, exactly as in GitHub Actions OIDC and GCP
Workload Identity Federation: a tenant scopes its trust with a tenant-specific
audience, and the external workload requests a token whose aud names that
tenant. Feldera matches (issuer, subject, audience), so the audience is the
disambiguator. Scoping a single-tenant workload identity is different from the
owner case above, where one principal must act across many tenants and a
per-request header beats a token-per-tenant aud.
Tenant-specific audiences are also why the wildcard subject/audience
restriction in section 7 matters. The current lookup, WHERE issuer = $1 with no tenant filter (oidc_trust.rs:151), is tightened to
match on issuer, subject, and audience together; if it still resolves to more
than one tenant (overlapping trusts with non-distinguishing audiences), it
rejects rather than picking an arbitrary first row, so a misconfiguration fails
closed instead of silently crossing tenants.
9. New REST APIs for tenant and user management
Method
Path
Min role
Purpose
GET
/v0/tenant/users
admin
List members of the acting tenant and their roles.
PUT
/v0/tenant/users/{user_id}
admin
Assign or change a member's role. Capped: an admin may grant at most admin; owner is never assignable here.
DELETE
/v0/tenant/users/{user_id}
admin
Remove a member from the acting tenant.
GET
/v0/tenants
owner
List all tenants in the installation (platform view).
POST
/v0/tenants
owner
Create a tenant explicitly (rather than by first login).
An owner manages any tenant's members by setting Feldera-Tenant to that tenant
and calling the same admin-level /v0/tenant/users endpoints, so no separate
cross-tenant user API is needed. GET /v0/config/session is extended to return
the caller's effective role in the acting tenant (and whether the caller is an
owner), so the web console can gate the admin UI without a role claim in the JWT.
10. Web console admin UI (prototype)
The web-console gains an /admin route, reachable from a new "Admin" entry in
the profile menu, shown only when page.data.feldera.role >= admin. The page has:
A user/role table listing members of the current tenant with a role
selector per row (GET/PUT/DELETE /v0/tenant/users), plus an "add user"
form for pre-provisioning.
An "Admin and owner access" section to grant admin (and, for owners, owner)
through OIDC trust relationships, since admin/owner are not issuable as
static API keys.
An owner-only "Tenants" section (GET /v0/tenants) with a tenant switcher that
sets Feldera-Tenant, letting an owner drop into any tenant's admin view.
An owner-only "Owner OIDC trust" section to create owner trust relationships.
The existing API-key and OIDC-trust creation forms gain a role selector:
the API-key form offers read/write; the trust form offers read/write/admin
(and owner for an owner), each bounded to the caller's role and defaulting to the minimum.
11. Required security invariants
The implementation must satisfy and test the following.
#
Invariant
I1
One Ord role type read < write < admin < owner, reused for every comparison.
I2
Minted key/trust role is always <= the creator's role; a violation is 403, not a silent downgrade. Enforced in handler and storage.
I3
Neither admin nor owner is mintable as an API key, by any caller, ever (type-level MintableKeyRole, i.e. read or write).
I4
admin and owner originate only from signature-verified principals (login JWT or OIDC trust).
I5
The Feldera-Tenant header selects the acting tenant; a non-owner may select only a tenant the IdP already authorizes (no widening), while an owner may select any tenant.
I6
Every /v0 route has an explicit minimum role; an unlisted route fails closed; a CI meta-test enforces coverage.
I7
Absent role in extensions is 403, never an implicit pass.
I8
Data-plane and mutating routes (ingress, egress, ad-hoc SQL, start/stop/clear/checkpoint/transaction) require >= write; read is monitoring.
I9
Every query filters by the resolved TenantId; the only exception is the pre-tenant auth lookup, which must resolve to exactly one tenant or reject.
I10
Foreign-object access (another tenant's id/name) returns 404, never reveals existence; the tenant is never derived from a client-supplied id.
I11
OIDC-trust creation is >= admin, capped at creator role, with no wildcard-only subject/audience above read, and owner forbidden except for platform owners.
I12
The no-auth default principal is a single fixed tenant with role at most admin, and cannot use the owner header path.
I13
Role parsing is total; no reachable panic on an unknown role string.
I14
Actions are audit-logged with (user, tenant, action); + owner cross-tenant actions.
I15
A negative test exists per invariant.
12. Corner cases
Existing deployments. Default role read is a behavior change: today every
logged-in user is write. Operators upgrading set FELDERA_AUTH_DEFAULT_ROLE=write
for a transition period, then tighten. The migration backfills existing API
keys and trusts from their scopes so no credential loses access.
Individual tenancy. A user whose first login creates their own tenant becomes
its admin, so they keep full control of their own workspace.
Role change after mint. An API key's role is a snapshot taken at mint; if the
creator is later downgraded, the key keeps its role until revoked (deleted).
Revocation, not downgrade, is the control. Re-evaluating a credential's role on
each use, and key expiry/rotation, are future work (see future possibilities).
Owner with no header. An owner without Feldera-Tenant acts in its home (or
the default) tenant; tenant-scoped actions in another tenant require the header.
Drawbacks
New tables and a membership-management surface to build, secure, and operate.
A behavior change (default read) that existing deployments must opt out of
during transition.
Two sources of truth for tenant participation (the IdP grants access, Feldera
grants role) can confuse: an admin can pre-assign a role to a user the IdP has
not yet authorized for the tenant, and the grant lies dormant until first login.
owner is effectively god-mode; a leaked owner login or owner trust is a
full-platform compromise, mitigated only by short-lived signed tokens and audit.
More enforcement code on the hot path (one table lookup per request, cheap, but
non-zero).
Rationale and alternatives
Ordered roles versus capability sets. The four roles are levels, not
independent permissions, so a totally ordered enum with Ord matches the
mental model and keeps comparisons trivial. Keeping the existing Vec<ApiPermission> and adding admin/owner doesn't seem like a good idea:
admin and owner are not additive permissions, and set semantics invite the
"has admin but not write" nonsense states.
Role source: Feldera-managed role with IdP-managed access, versus
Feldera-authoritative for both, versus IdP-claim-driven role. The chosen hybrid
is the least breaking: it keeps the three documented tenancy strategies intact
and adds roles additively, while still giving admins an in-product surface to
manage roles.
Owner tenant selection: Feldera-Tenant header versus the aud field of
issue [rfc] Support for OpenID Connect #6330. The header is per-request, reuses existing plumbing, and does not
overload the audience claim that the login path already validates.
admin/owner via OIDC trust only, never a static API key. A static secret
carrying admin or owner is easier to leak/abuse; OIDC trusts are
signature-verified and revocable at the IdP, so the elevated roles are granted
only through them. API keys still carry read or write.
Prior art
Kubernetes RBAC (Roles/ClusterRoles bound by RoleBindings, namespaced versus
cluster-scoped) is the direct analogue of tenant roles versus owner; feldera
has a coarser ordered model as it's model is arguably much simpler and doesn't
need as much complexity.
GitHub fine-grained personal access tokens are scoped at or below the granting
user, the same mint-cap principle as API keys here.
OIDC workload identity federation (GitHub Actions OIDC, GCP Workload Identity)
is the model the OIDC trust relationship already follows; this RFC adds a role
to it and a cross-tenant owner variant.
AWS IAM least-privilege keys and Postgres GRANT informed the deny-by-default
posture and the rule that credentials never exceed their issuer.
Unresolved questions
Should minting credentials require admin? API keys and OIDC trusts are
per-tenant principals (service accounts), not people, so an argument exists that
only an admin should create them and hand them out. This RFC keeps write
minting its own read/write API keys, with trusts at admin; whether to
require admin for all credential minting is open.
Another path for API keys being tenant-owned today; should they become user-owned
so "manage your own keys" is literal and each key's actions attribute to a person?
Should platform telemetry (/v0/cluster/*) be restricted to owner? It is not
tenant-scoped today. Kept read for now; tightening to owner is a judgment
call once the data exposed there is reviewed.
Durable audit log of all actions (user, tenant, action). Can wait as
follow up in v2.
Owner cross-tenant addressing. Tenants are keyed (name, provider). A tenant
an owner creates via POST /v0/tenants (provider manual) is a different row
from one users join under their OIDC issuer, so the same name can name two
tenants and an owner's Feldera-Tenant: <name> lookup becomes ambiguous. The
clean resolution is to address tenants by TenantId (UUID) in the header, or
make tenant names globally unique; the prototype resolves by name today.
Trusting the owner email claim. FELDERA_OWNERS may match a token's email,
but the prototype does not check an email_verified claim. If the login issuer
does not verify emails, owner designation should match the provider-qualified
subject instead, or require email_verified.
API-key default role. Keys now default to read (least privilege); clients
that previously relied on the implicit read+write key (e.g. the Python SDK's create_api_key) must pass an explicit role. Whether to default to write
for backward compatibility is open.
Last-admin protection. An admin can demote or remove the last admin of a
tenant (recoverable only by an owner acting in that tenant). A "cannot remove
the last admin" guard is future work.
Future possibilities
SCIM or group-claim provisioning to sync users and roles from the IdP, mapping
an IdP group to a role on top of the membership table.
API key expiry and rotation, reducing the blast radius of long-lived secrets.
Per-user (not per-tenant) API keys, for literal self-service key management and
per-user audit attribution.
A durable, queryable audit log of all actions (user, tenant, action).
Summary
Feldera authorizes API requests by tenant isolation alone: any principal that
resolves to a tenant may do anything to that tenant's resources. The
readscope is computed, stored, and attached to every request, then never checked.
This RFC introduces role-based access control with four totally ordered roles,
read < write < admin < owner, enforced on every REST endpoint by adeny-by-default, table-driven middleware. Per-(user, tenant) roles live in a new
Feldera-side membership table; the identity provider keeps deciding which
tenants a user may reach. API keys and OIDC trust relationships carry a role
capped at their creator's role. An
owneris a platform operator who actsinside any tenant by selecting it with the existing
Feldera-Tenantheader.The RFC adds tenant and user management APIs plus a web-console admin UI.
Motivation
Authorization today is binary: hold a token that resolves to tenant T and you
have full read and write over everything in T (
crates/pipeline-manager/src/auth.rsinserts
vec![Read, Write]on the login path. No handlerever reads the permission vector back). This blocks several routine needs:
must not edit code, start/stop pipelines, run ad-hoc SQL, or read row data.
credentials or manage users (least privilege).
trust relationships without touching other tenants.
any tenant for maintenance, for example stopping a runaway pipeline after 24h.
The machinery to carry a permission already exists end to end; it simply has no
reader. This RFC improve today's state into enforced, least-privilege access.
Reference-level explanation
1. The role lattice
Roles form a single total order. A higher role can do everything a lower role
can, plus more. There is exactly one ordered
Roletype with a derivedOrd,defined once and reused for every comparison.
readwritereadcan, plus everything that touches the data a pipeline processes (ad-hoc SQL, HTTP ingress/egress) and every change to pipelines (create/edit/delete, start/stop/pause/resume, checkpoints, transactions). Mintsread/writeAPI keys.adminwritecan, plus manages that tenant's users and their roles (up toadmin) and its OIDC trust relationships. Grantsadminto non-human principals through an OIDC trust;adminis not issuable as a static API key.owneradmincan, in any tenant, selected per request withFeldera-Tenant. Plus platform-wide views (the tenant list) and creatingownerOIDC trusts.The line between
readandwrite: a reader learns how a pipeline behaves andwhat it is doing, but never learns anything (besides stats) about the data it processes and never
changes anything. Anything that reads or moves row-level data (ad-hoc SQL, HTTP
ingress, HTTP egress) is therefore
write.2. Where a principal's role comes from
A request is authenticated by one of four paths. Each resolves an
AuthenticatedPrincipal { acting_tenant, home_tenant, role }and inserts itinto request extensions. The acting tenant is the tenant the request operates
in; for everyone but an
ownerit equals the home tenant.bearer_auth)[Read, Write](auth.rs:314)tenant_membership(acting_tenant, user); if no row, the configured default role (read); if the login created the tenant,admin; if the identity is inFELDERA_OWNERS,owner.api_key_auth)scopesfrom the key rowrolecolumn on the key row.oidc_trust_auth)scopesfrom the trust rowrolecolumn on the trust row.ownertrusts resolve cross-tenant (section 7).tag_with_default_tenant_id)[Read, Write]on the nil tenantadminof the single default tenant. Cross-tenantowneris meaningless with one tenant, soadmingives full control of it. Local single-node use only.Tenant access stays exactly as documented today
(
docs.feldera.com/.../authentication/index.mdx): the identity provider grantswhich tenants a user can reach through the
tenantsclaim, the issuer domain,or the
subclaim. Feldera owns only the role within a reachable tenant. Auser an admin "adds" is a membership row that takes effect when that user next
authenticates into the tenant through the IdP.
Provisioning rule that preserves today's single-user and individual-tenancy
behavior: the first principal to resolve to a tenant that does not yet exist
becomes that tenant's
admin. Subsequent principals with no membership row getthe default role (
read, configurable viaFELDERA_AUTH_DEFAULT_ROLE).3. Identity and data model
No user, role, or membership table exists today; the only principal is the
tenant, keyed
(tenant, provider). RBAC introduces the user and the(user, tenant, role) link, in migration
VXX__rbac.sql.owneris never stored intenant_membership(it is a platform property,sourced from config or an owner OIDC trust) and never on an API key.
tenant_membership.roleis therefore capped atadminby a CHECK or by thewrite path.
Parsing safety: the current
ApiPermission::from_str(...).expect(...)sites(
db/operations/api_key.rs:30,52,141;oidc_trust.rs:23,180) panic on anyunknown string.
Role::from_strmust be total and an unknown value must failthe request, never the process, because these parse inside the auth path.
New
Storagemethods (each following the existing trait / postgres / operationstrio):
get_or_create_user,list_tenant_members,get_member_role,upsert_member_role,remove_member,list_tenants,get_tenant_id_by_name.4. Enforcement architecture
Enforcement is a single middleware over the
/v0scope, driven by one statictable keyed on
(method, actix matched route pattern). Two properties matter:endpoint fails until classified. A meta-test should enumerate the routes
registered in
api_scope()and fails the build if any has no entry.the code share one source of truth.
HTTP method is not used as the read/write signal because
POST /start,/stop,/pauseare mutations and some queries areGET. Classification isper route, not per verb.
A new
DBError::InsufficientPermissions { required: Role }maps to HTTP 403(
db/error.rs: add theDisplay,error_code, andstatus_codearms; no 403exists today). It flows to the wire through the existing
From<DBError> for ManagerErrorandResponseError.Tenant isolation is unchanged: handlers still read one
TenantId(nowacting_tenant) and every query still filtersWHERE tenant_id = $1. RBAC addsa vertical check (role) on top of the existing horizontal check (tenant).
5. The access-control table
The minimum role to reach each endpoint.
publicendpoints are reachable before authentication and are not gated by any role.Pipeline management
/v0/pipelinesread/v0/pipelines/{pipeline_name}read/v0/pipelineswrite/v0/pipelines/{pipeline_name}write/v0/pipelines/{pipeline_name}write/v0/pipelines/{pipeline_name}/update_runtimewrite/v0/pipelines/{pipeline_name}write/v0/pipelines/{pipeline_name}/startwrite/v0/pipelines/{pipeline_name}/stopwrite/v0/pipelines/{pipeline_name}/dismiss_errorwrite/v0/pipelines/{pipeline_name}/clearwrite/v0/pipelines/{pipeline_name}/logsread/v0/pipelines/{pipeline_name}/testingwrite/v0/pipelines/{pipeline_name}/eventsread/v0/pipelines/{pipeline_name}/events/{event_id}readPipeline interaction (control + data plane)
/v0/pipelines/{pipeline_name}/ingress/{table_name}write/v0/pipelines/{pipeline_name}/egress/{table_name}write/v0/pipelines/{pipeline_name}/tables/{table_name}/connectors/{connector_name}/{action}write/v0/pipelines/{pipeline_name}/tables/{table_name}/connectors/{connector_name}/statsread/v0/pipelines/{pipeline_name}/views/{view_name}/connectors/{connector_name}/statsread/v0/pipelines/{pipeline_name}/views/{view_name}/connectors/{connector_name}/commandwrite/v0/pipelines/{pipeline_name}/statsread/v0/pipelines/{pipeline_name}/metricsread/v0/pipelines/{pipeline_name}/time_seriesread/v0/pipelines/{pipeline_name}/time_series_streamread/v0/pipelines/{pipeline_name}/circuit_profileread/v0/pipelines/{pipeline_name}/circuit_json_profileread/v0/pipelines/{pipeline_name}/dataflow_graphread/v0/pipelines/{pipeline_name}/rebalancewrite/v0/pipelines/{pipeline_name}/start_compactionwrite/v0/pipelines/{pipeline_name}/checkpoint/syncwrite/v0/pipelines/{pipeline_name}/checkpointwrite/v0/pipelines/{pipeline_name}/checkpoint_statusread/v0/pipelines/{pipeline_name}/checkpoint/sync_statusread/v0/pipelines/{pipeline_name}/checkpointsread/v0/pipelines/{pipeline_name}/samply_profileread/v0/pipelines/{pipeline_name}/samply_profileread/v0/pipelines/{pipeline_name}/heap_profileread/v0/pipelines/{pipeline_name}/pausewrite/v0/pipelines/{pipeline_name}/resumewrite/v0/pipelines/{pipeline_name}/activatewrite/v0/pipelines/{pipeline_name}/approvewrite/v0/pipelines/{pipeline_name}/querywrite/v0/pipelines/{pipeline_name}/tables/{table_name}/connectors/{connector_name}/completion_tokenwrite/v0/pipelines/{pipeline_name}/completion_statusread/v0/pipelines/{pipeline_name}/start_transactionwrite/v0/pipelines/{pipeline_name}/commit_transactionwrite/v0/pipelines/{pipeline_name}/clock/advancewrite/v0/pipelines/{pipeline_name}/support_bundlereadPlatform (keys, trust, config, metrics, cluster)
/v0/api_keyswrite/v0/api_keys/{api_key_name}write/v0/api_keyswrite/v0/api_keys/{api_key_name}write/v0/oidc_trustadmin/v0/oidc_trust/{name}admin/v0/oidc_trustadmin/v0/oidc_trust/{name}admin/v0/configread/v0/config/authenticationpublic/v0/config/demosread/v0/config/sessionread/v0/metricsread/v0/cluster/eventsread/v0/cluster/events/{event_id}read/v0/cluster_healthzpublic6. Endpoints that behave differently by role
The mapping above flagged endpoints whose correct behavior depends on the
caller's role. Role-dependent behavior in endpoints is an API smell;
the resolutions below keep each endpoint single-purpose and make the difference
explicit and minimal rather than scattered through handler bodies.
GET /v0/cluster/events,/v0/cluster/events/{id},GET /v0/cluster_healthzreadfor now;cluster_healthzstays apublicliveness probe. Restricting platform telemetry toownerdepends on nature of data exposed here (see unresolved questions).GET /v0/pipelines/{name}/query(ad-hoc SQL)GETthat returns and can change the data a pipeline processes.write. A reader should not learn anything about processed data.GET .../support_bundle,POST/GET .../samply_profileread: they expose nothing a read user cannot already obtain. Whether all debug/collect endpoints should requirewriteis an open question.POST /v0/pipelines/{name}/testingplatform_version, gated by an unstable feature).write: can be ignored, only runs in testing.GET .../checkpoint_status,.../checkpointsincrement_notify_counter, mutating on aGET.read; the counter bump is an internal notify, acknowledged as benign.GET /v0/config/authenticationsecurity()and is mounted pre-auth; the login page needs it.public, carved out of the role table explicitly.7. API keys, OIDC trust, and the mint cap
Both (API, OIDC trust) creations gain a
rolefield; the hardcodedvec![Read, Write]is removed. The minting rule, enforced in the handler and asserted again in the storage
layer (defense in depth):
A leaked static credential is much easier to happen than a leaked
interactive session, so
adminandownerare expressible only assignature-verified principals (a login JWT or an OIDC trust).
API keys carry only
readorwrite. This is encoded as a type split: aMintableKeyRole(readorwrite) distinct fromRole, so anadminorownerliteral cannot reachstore_api_key_hash. This allows developmentto still test/script workflows using python/CLI etc.
OIDC trust creation is
admin, notwrite: minting a durable login credentialfor an external identity is identity administration. The trust's role is capped
at the creator's role (an admin may create up to an
admintrust; only an ownercreates
ownertrusts). Wildcard-onlysubject/audiencepatterns are rejectedfor any role above
read, so a tenant admin cannot mint a "matches everyone"trust. Granting
adminto a non-human principal therefore goes through an OIDCtrust, surfaced in the admin UI, rather than a static key.
API keys and OIDC trusts are per-tenant principals (service accounts), not
people. That raises the question of whether minting them should require
adminat all, rather than
writeminting its own keys; this RFC keepswritekey-minting for now and revisits it in unresolved questions.
8. Owner and cross-tenant action
The first
owneris bootstrapped from configuration:FELDERA_OWNERSlistsowner identities (
provider:subject, or email). A login or OIDC-trust principalthat matches the list is granted
owner. This breaks the chicken-and-eggproblem (creating an owner needs an owner) without a database seed. Existing
owners can then create additional
ownerOIDC trusts in the admin UI forautomation, for example the CI identity that runs the install.
An owner selects the tenant to act in with the existing
Feldera-Tenantheader.This reuses the header, and
MissingTenantHeadermachinery already presentper-request UX (a web-console dropdown or
fda --tenant X).It is preferred over encoding the target tenant in the
audclaim (issue #6330):
audalready carries audience meaning on the login path,and a token-per-tenant defeats caching and forces a mint per target.
The
Feldera-Tenantheader always selects the acting tenant. A non-owner mayselect only a tenant the IdP already authorizes for them (the
tenantsclaim,today's behavior at
auth.rs), and their role is then looked up for thatselected tenant: a user can be
adminin tenant A andreadin tenant B. Onlyan
ownermay select a tenant outside their authorized set; that widening is theowner-only privilege, evaluated before the acting tenant is written to
extensions. Owner resolution is a strict lookup that errors on miss, never
get_or_create_tenant_id, so a typo cannot silently create an empty tenant.Every action is audit-logged with (user, target tenant, method, path), owner
cross-tenant actions especially.
ownerOIDC trusts cannot be created by an ordinary tenant admin: they live ina reserved platform tenant and are creatable only by an
owner.For an ordinary (non-owner) trust, the
audclaim is the mechanism that maps afederated token to one tenant, exactly as in GitHub Actions OIDC and GCP
Workload Identity Federation: a tenant scopes its trust with a tenant-specific
audience, and the external workload requests a token whose
audnames thattenant. Feldera matches
(issuer, subject, audience), so the audience is thedisambiguator. Scoping a single-tenant workload identity is different from the
owner case above, where one principal must act across many tenants and a
per-request header beats a token-per-tenant
aud.Tenant-specific audiences are also why the wildcard
subject/audiencerestriction in section 7 matters. The current lookup,
WHERE issuer = $1with no tenant filter (oidc_trust.rs:151), is tightened tomatch on issuer, subject, and audience together; if it still resolves to more
than one tenant (overlapping trusts with non-distinguishing audiences), it
rejects rather than picking an arbitrary first row, so a misconfiguration fails
closed instead of silently crossing tenants.
9. New REST APIs for tenant and user management
/v0/tenant/usersadmin/v0/tenant/users/{user_id}adminadmin;owneris never assignable here./v0/tenant/users/{user_id}admin/v0/tenantsowner/v0/tenantsownerAn owner manages any tenant's members by setting
Feldera-Tenantto that tenantand calling the same
admin-level/v0/tenant/usersendpoints, so no separatecross-tenant user API is needed.
GET /v0/config/sessionis extended to returnthe caller's effective role in the acting tenant (and whether the caller is an
owner), so the web console can gate the admin UI without a role claim in the JWT.
10. Web console admin UI (prototype)
The web-console gains an
/adminroute, reachable from a new "Admin" entry inthe profile menu, shown only when
page.data.feldera.role >= admin. The page has:selector per row (
GET/PUT/DELETE /v0/tenant/users), plus an "add user"form for pre-provisioning.
admin(and, for owners,owner)through OIDC trust relationships, since
admin/ownerare not issuable asstatic API keys.
GET /v0/tenants) with a tenant switcher thatsets
Feldera-Tenant, letting an owner drop into any tenant's admin view.ownertrust relationships.The existing API-key and OIDC-trust creation forms gain a role selector:
the API-key form offers
read/write; the trust form offersread/write/admin(and
ownerfor an owner), each bounded to the caller's role and defaulting to the minimum.11. Required security invariants
The implementation must satisfy and test the following.
Ordrole typeread < write < admin < owner, reused for every comparison.<=the creator's role; a violation is 403, not a silent downgrade. Enforced in handler and storage.adminnorowneris mintable as an API key, by any caller, ever (type-levelMintableKeyRole, i.e.readorwrite).adminandowneroriginate only from signature-verified principals (login JWT or OIDC trust).Feldera-Tenantheader selects the acting tenant; a non-owner may select only a tenant the IdP already authorizes (no widening), while anownermay select any tenant./v0route has an explicit minimum role; an unlisted route fails closed; a CI meta-test enforces coverage.>= write;readis monitoring.TenantId; the only exception is the pre-tenant auth lookup, which must resolve to exactly one tenant or reject.>= admin, capped at creator role, with no wildcard-onlysubject/audienceaboveread, andownerforbidden except for platform owners.admin, and cannot use the owner header path.12. Corner cases
readis a behavior change: today everylogged-in user is
write. Operators upgrading setFELDERA_AUTH_DEFAULT_ROLE=writefor a transition period, then tighten. The migration backfills existing API
keys and trusts from their
scopesso no credential loses access.its
admin, so they keep full control of their own workspace.creator is later downgraded, the key keeps its role until revoked (deleted).
Revocation, not downgrade, is the control. Re-evaluating a credential's role on
each use, and key expiry/rotation, are future work (see future possibilities).
Feldera-Tenantacts in its home (orthe default) tenant; tenant-scoped actions in another tenant require the header.
Drawbacks
read) that existing deployments must opt out ofduring transition.
grants role) can confuse: an admin can pre-assign a role to a user the IdP has
not yet authorized for the tenant, and the grant lies dormant until first login.
owneris effectively god-mode; a leaked owner login or owner trust is afull-platform compromise, mitigated only by short-lived signed tokens and audit.
non-zero).
Rationale and alternatives
independent permissions, so a totally ordered enum with
Ordmatches themental model and keeps comparisons trivial. Keeping the existing
Vec<ApiPermission>and addingadmin/ownerdoesn't seem like a good idea:admin and owner are not additive permissions, and set semantics invite the
"has admin but not write" nonsense states.
Feldera-authoritative for both, versus IdP-claim-driven role. The chosen hybrid
is the least breaking: it keeps the three documented tenancy strategies intact
and adds roles additively, while still giving admins an in-product surface to
manage roles.
Feldera-Tenantheader versus theaudfield ofissue [rfc] Support for OpenID Connect #6330. The header is per-request, reuses existing plumbing, and does not
overload the audience claim that the login path already validates.
admin/ownervia OIDC trust only, never a static API key. A static secretcarrying
adminorowneris easier to leak/abuse; OIDC trusts aresignature-verified and revocable at the IdP, so the elevated roles are granted
only through them. API keys still carry
readorwrite.Prior art
cluster-scoped) is the direct analogue of tenant roles versus owner; feldera
has a coarser ordered model as it's model is arguably much simpler and doesn't
need as much complexity.
user, the same mint-cap principle as API keys here.
is the model the OIDC trust relationship already follows; this RFC adds a role
to it and a cross-tenant owner variant.
GRANTinformed the deny-by-defaultposture and the rule that credentials never exceed their issuer.
Unresolved questions
admin? API keys and OIDC trusts areper-tenant principals (service accounts), not people, so an argument exists that
only an
adminshould create them and hand them out. This RFC keepswriteminting its own
read/writeAPI keys, with trusts atadmin; whether torequire
adminfor all credential minting is open.so "manage your own keys" is literal and each key's actions attribute to a person?
/v0/cluster/*) be restricted toowner? It is nottenant-scoped today. Kept
readfor now; tightening toowneris a judgmentcall once the data exposed there is reviewed.
follow up in v2.
(name, provider). A tenantan owner creates via
POST /v0/tenants(providermanual) is a different rowfrom one users join under their OIDC issuer, so the same name can name two
tenants and an owner's
Feldera-Tenant: <name>lookup becomes ambiguous. Theclean resolution is to address tenants by
TenantId(UUID) in the header, ormake tenant names globally unique; the prototype resolves by name today.
emailclaim.FELDERA_OWNERSmay match a token'semail,but the prototype does not check an
email_verifiedclaim. If the login issuerdoes not verify emails, owner designation should match the provider-qualified
subject instead, or require
email_verified.read(least privilege); clientsthat previously relied on the implicit read+write key (e.g. the Python SDK's
create_api_key) must pass an explicitrole. Whether to default towritefor backward compatibility is open.
admincan demote or remove the last admin of atenant (recoverable only by an
owneracting in that tenant). A "cannot removethe last admin" guard is future work.
Future possibilities
an IdP group to a role on top of the membership table.
per-user audit attribution.