Tags: feldera/feldera
Tags
[ci] Skip a third timing-flaky adapter test proptest_kafka_input failed one run with the same mock_dezset wait panic as its previously skipped siblings while 418 tests passed; the branch's only code change is in dbsp circuit ids, nowhere near the kafka transport. Signed-off-by: Gerd Zellweger <mail@gerdzellweger.com>
[sql] Give the streams that leave a recursive circuit ids of their own
The compiler assigned the same ids to streams inside the recursive fragment and
the same streams exported outside the fragment:
```rust
let (s6, s9) = circuit.recursive(|circuit, (unused_0, s3)| {
...
let s6 = ...join...; s6.set_persistent_id(Some("ed5fc63e…")); // inside the scope
Ok((s6, s9))
}).unwrap();
s6.set_persistent_id(Some("ed5fc63e…")); // exported — same id
```
However these are different streams with different integrals as can be seen
from the circuit:
```
┌───────────────────────────────────────────────────────────────┐
│ │
i │ ┌───┐ |-s6 inside the scope │
────┼──►δ0─────────►│ │ V ┌────────┐ ┌───────────────┐ │ ┌───────────┐ s6 exported
│ │ f ├─────►│distinct├──┬───►│integrate_trace├───┼──►│consolidate├───────►
│ ┌──────►│ │ └────────┘ │ └───────────────┘ │ └───────────┘
│ │ └───┘ │ │
│ │ │ │
│ │ │ │
│ │ ┌────┐ │ │
│ └───────┤z^-1│◄────────────────┘ │
│ └────┘ │
│ │
└───────────────────────────────────────────────────────────────┘
```
Suffix the exported stream's id with '.export', in both the single-file and the
multi-crate writer, mirroring the '.delay' suffix that already distinguishes a
recursive input. The checkpoint then holds <id>.shard.accintegral for the inner
trace and <id>.export.shard.accintegral for the outer one.
Bump RECURSIVE_STATE_VERSION with it: the layout of what a recursive view stores
changes, so the ids in the dataflow graph have to change too, or the pipeline
manager would diff the views as unchanged while the runtime finds their state
missing, and refuse to start with UnexpectedBootstrap. The metadata test's
golden ids move for the same reason.
Signed-off-by: Leonid Ryzhyk <leonid@feldera.com>
[types] Serialize checkpoint fingerprint as OpenAPI format uint64 fda's REST client generates the CheckpointMetadata.fingerprint field as i64 from the OpenAPI schema, so a checkpoint whose 64-bit fingerprint has the high bit set breaks `fda pipelines`/`fda status` for the whole instance (#6841). Annotate the field with `#[schema(format = "uint64")]` so progenitor/typify generate `u64` instead, which accepts the full unsigned range without truncating or otherwise altering the fingerprint value. Signed-off-by: Ben Pfaff <blp@feldera.com>
python: retry connection resets on idempotent GET requests A connection reset mid-poll (e.g. status polling during compilation) was raised immediately as FelderaCommunicationError with no retry, turning a transient network blip into a hard client failure. GET is idempotent, so retry ConnectionError there; leave POST/PUT/PATCH/DELETE unretried since a lost response may hide an already-applied write. Signed-off-by: Ben Pfaff <blp@feldera.com>
[adapters] Unregister an input endpoint rejected for weak fault toler… …ance `add_input_endpoint` compares the endpoint's fault tolerance with the pipeline's only after it has registered and opened the endpoint, and the rejection path returned the error without undoing either step. The endpoint stayed in the status map, where it could never run because the error path also skipped the unpark that starts it, yet it still counted toward the pipeline's statistics and reserved its own name against a later attempt to add the connector. Disconnect it instead, which both removes the entry and shuts the reader down. Signed-off-by: Leonid Ryzhyk <ryzhyk@gmail.com>
ci: Fail faster and get more information if Rust build times ago A stalled network call (sccache's GCS backend exposes no client-side timeout) hung the arm64 build for 5h13m with zero output on 2026-07-28, until the merge queue's check_response_timeout_minutes force-cancelled the whole run. Add job- and step-level timeout-minutes so a stall fails fast, and keep the sccache stats step running on failure for diagnosis. Signed-off-by: Ben Pfaff <blp@feldera.com>
PreviousNext