fips: route cryptography through AWS-LC, remove OpenSSL from the dependency tree - #6828

Nobody uses it, and it was the only consumer of utoipa-swagger-ui, whose build dependency on reqwest pulled native-tls and with it the OpenSSL crate. The OpenAPI document itself is unaffected and still served.

Signed-off-by: Gerd Zellweger <mail@gerdzellweger.com>

native-tls links OpenSSL on Linux, giving a binary two cryptographic implementations that define the same symbols. Each of these dependencies reached it by defaulting to it, so each now selects rustls explicitly.

refinery moves to 0.9.2 because 0.9.0's tokio-postgres feature pulls postgres-native-tls unconditionally, and 0.9.2 adds tokio-postgres-rustls.

Signed-off-by: Gerd Zellweger <mail@gerdzellweger.com>

It is unused. Removing it drops three hardcoded DSNs, the FELDERA_SENTRY_ENABLED and SENTRY_ENVIRONMENT variables from integration CI, and the sentry dependency from five crates.

The rest-api client is generated, and its build script injected a trace header call into every generated request plus the trait import that backed it; both are gone. feldera-observability keeps its logging, system and FIPS modules.

Signed-off-by: Gerd Zellweger <mail@gerdzellweger.com>

AWS-LC keeps OpenSSL's symbol names, so a binary containing both resolves each name to whichever archive the linker reaches first, and memory allocated by one library gets freed by the other. That corrupts the heap and surfaces far from the cause, which is what made it expensive to diagnose.

The hook blocks openssl, native-tls and boring, asserts that openssl-sys resolves to an AWS-LC backend rather than the system OpenSSL, and reports ring without failing, since rustls still pulls it through sqlx. It reads the lockfile only, so it compiles nothing and runs in about a second.

Signed-off-by: Gerd Zellweger <mail@gerdzellweger.com>

sqlx's tls-rustls feature aliases tls-rustls-ring, and sqlx prefers ring whenever both backends are enabled, so a consumer cannot select aws-lc-rs by adding a feature; the choice has to be made where the dependency is declared. The fork pins the same upstream revision with that one line changed.

feldera/etl branch sqlx-aws-lc-rs-main carries the same change against upstream main, expressed as a feature so consumers can pick, ready to propose upstream.

Signed-off-by: Gerd Zellweger <mail@gerdzellweger.com>

0.1.2 declared reqwest without default-features = false, so it enabled default-tls, which pulls native-tls and with it the OpenSSL crate. 0.1.3 is built against reqwest 0.13 with rustls pinned explicitly.

This was the last dependency reintroducing OpenSSL: the crate, native-tls and hyper-tls are now all absent from the tree, and openssl-sys resolves to the AWS-LC FIPS backend.

Signed-off-by: Gerd Zellweger <mail@gerdzellweger.com>

async-nats defaults to ring and offers aws-lc-rs. aws-sigv4 1.4.5 dropped ring entirely, so a lockfile bump suffices. refinery needs no TLS backend at all, because migrations run on a client this crate creates, and its tokio-postgres-rustls feature pinned an older tokio-postgres-rustls that selects ring. reqwest builds without a provider and takes the process default, which every binary installs as aws-lc-rs; fda did not install one, so it does now.

object_store 0.14 selects aws-lc-rs where 0.12 used ring. The upgrade renamed one error variant and deprecated Path::child in favour of Path::join, which consumes self rather than borrowing.

Signed-off-by: Gerd Zellweger <mail@gerdzellweger.com>

rdkafka-sys otherwise compiles a vendored librdkafka against whatever OpenSSL pkg-config finds, which leaves Kafka TLS on a second cryptographic implementation no matter how the rest of the binary is built.

scripts/install-librdkafka.sh builds AWS-LC with BUILD_LIBSSL, then librdkafka against it. Both container images run the script, and so must developers, since the next commit links librdkafka dynamically. Distribution packages are built against OpenSSL and are older than the version rdkafka-sys requires, so they are not a substitute.

librdkafka 2.12.1 calls HMAC() without including <openssl/hmac.h>. OpenSSL supplies the declaration transitively through x509.h and AWS-LC does not, so the call would compile as an implicit declaration returning int, truncating the returned pointer. The script patches it; confluentinc/librdkafka#5552 fixes it upstream but is unmerged.

The librdkafka version comes from Cargo.lock, where rdkafka-sys names it in its own version as 4.10.0+2.12.1, so the library cannot drift from the crate expecting it. The configure flags mirror the cargo features rdkafka-sys used, and the script asserts each one: losing one is otherwise silent, and a codec or authentication mechanism simply stops being offered.

Signed-off-by: Gerd Zellweger <mail@gerdzellweger.com>

rdkafka's ssl feature depends on openssl-sys and configures the vendored librdkafka build. dynamic-linking consumes the library that scripts/install-librdkafka.sh builds against AWS-LC instead, so neither the feature nor the crate is needed, and Kafka TLS stops being the one part of the system on a different cryptographic implementation.

The ssl, gssapi, zstd and libz features only ever configured that vendored build; those capabilities now come from the installed library, where the script asserts them. Dropping the default libz feature also removes libz-sys, which was compiling a zlib nothing used.

default-features has to be set at the workspace root: a member cannot override it on an inherited dependency.

openssl-sys stays listed in Cargo.lock as an optional dependency of rdkafka-sys, but it is no longer in the build graph, so it is neither compiled nor linked.

Signed-off-by: Gerd Zellweger <mail@gerdzellweger.com>

openssl-sys can now be blocked outright rather than allowed with an assertion about its backend, since nothing needs it once rdkafka links librdkafka dynamically.

The second check is new. aws-lc-sys and aws-lc-fips-sys are separate crates, so cargo cannot unify them, and a tree holding both compiles and links the whole library twice. It happened here: aws-lc-rs selects its backend from the fips feature while openssl-sys selected its own, and the two disagreed. Nothing failed, because the symbol prefixes differ, so the duplication was invisible at run time.

Adopting the FIPS module means building AWS-LC with -DFIPS=1 and moving aws-lc-rs to aws-lc-fips-sys in one change. This check is what makes doing only half of it fail.

Signed-off-by: Gerd Zellweger <mail@gerdzellweger.com>

1.18.0 is the first release depending on aws-lc-fips-sys 0.14, which added AWS_LC_FIPS_SYS_SYSTEM_DIR and USE_SYSTEM. Earlier versions always compiled AWS-LC from source, so enabling the fips feature would have put Go and cmake back into the image that compiles pipelines, undoing #6822.

With 0.14 the validated module can be built once and linked, the way scripts/install-librdkafka.sh already builds AWS-LC for librdkafka. Nothing selects fips yet; this only makes that step possible without a toolchain in the shipped image.

Signed-off-by: Gerd Zellweger <mail@gerdzellweger.com>

Nothing in this repository links OpenSSL now that rdkafka consumes a prebuilt librdkafka, so the package looked removable. The cloud repository runs its Rust build in this same image, and its cluster-control-plane crate uses the openssl crate directly, in kubernetes_runner/runner_config.rs. Dropping the package would break that build the moment cloud bumps its image pin, which is far from this change.

Also names perl explicitly. scripts/install-librdkafka.sh needs it to patch librdkafka, and it was present only because Ubuntu marks it Essential.

Signed-off-by: Gerd Zellweger <mail@gerdzellweger.com>

rdkafka links librdkafka dynamically, so the CI image has to supply it. The previous image predates scripts/install-librdkafka.sh and every Rust job fails in it with "librdkafka 2.12.1 cannot be found on the system".

Built from 5b0f47e for amd64 and arm64 on native runners.

Signed-off-by: Gerd Zellweger <mail@gerdzellweger.com>

perl arrives with git either way, but awk keeps the script on tools the POSIX base guarantees, and nothing else here used perl. The insertion was validated against librdkafka 2.12.1: one hmac.h include, directly above x509.h.

Signed-off-by: Gerd Zellweger <mail@gerdzellweger.com>

The image carries librdkafka built against AWS-LC v5.5.0, so CI links
the same AWS-LC release the Rust dependency tree pins.

Signed-off-by: Gerd Zellweger <mail@gerdzellweger.com>