Context
Follow-up from #5988. The initial config is intentionally minimal (`uri`, `publication`, `source_table`, `tls`). Several useful knobs are currently hidden or derived.
Candidates for exposure
Slot name customization
Currently derived from a deterministic hash of `(host:port/db, publication, source_table)`. Users can't:
- Choose a specific slot name for operational visibility
- Run two Feldera pipelines against the same publication + table (would collide)
Proposal: Optional `slot_name: Option` that overrides the derived hash.
Batch config
ETL's batch size and max fill duration are hardcoded to defaults. Power users may want to tune for their workload (e.g., high-throughput tables want larger batches).
Proposal: Optional `batch: Option` that overrides ETL defaults.
TCP keepalive
Hardcoded to `TcpKeepaliveConfig::default()`. Some networks need tuned keepalive.
Proposal: Optional `tcp_keepalive: Option`.
Memory backpressure
ETL's memory backpressure config is hardcoded. Users may want to tune it.
Proposal: Optional `memory_backpressure: Option`.
Open questions
- How much of ETL's config surface should we proxy through? All of it, or only the fields we think users need?
- For slot name: should we validate it's a legal Postgres identifier?
- Should these all go in the top level or be nested under an `advanced: {...}` config object to signal "most users don't need this"?
Context
Follow-up from #5988. The initial config is intentionally minimal (`uri`, `publication`, `source_table`, `tls`). Several useful knobs are currently hidden or derived.
Candidates for exposure
Slot name customization
Currently derived from a deterministic hash of `(host:port/db, publication, source_table)`. Users can't:
Proposal: Optional `slot_name: Option` that overrides the derived hash.
Batch config
ETL's batch size and max fill duration are hardcoded to defaults. Power users may want to tune for their workload (e.g., high-throughput tables want larger batches).
Proposal: Optional `batch: Option` that overrides ETL defaults.
TCP keepalive
Hardcoded to `TcpKeepaliveConfig::default()`. Some networks need tuned keepalive.
Proposal: Optional `tcp_keepalive: Option`.
Memory backpressure
ETL's memory backpressure config is hardcoded. Users may want to tune it.
Proposal: Optional `memory_backpressure: Option`.
Open questions