Skip to content

fips: route cryptography through AWS-LC, remove OpenSSL from the dependency tree #6828

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
gz wants to merge 19 commits into
base: main
Choose a base branch
from
Draft

fips: route cryptography through AWS-LC, remove OpenSSL from the dependency tree #6828

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
7601db1
fips: route all cryptography through AWS-LC
gz Aug 7, 2026
f2f6c12
api: remove the Swagger UI
gz Aug 8, 2026
a6a7287
deps: keep native-tls out of the dependency tree
gz Aug 8, 2026
58bef67
observability: remove Sentry
gz Aug 8, 2026
866ecd6
ci: fail when a second TLS implementation enters the dependency tree
gz Aug 8, 2026
aa754f5
deps: take etl from the feldera fork for its sqlx TLS backend
gz Aug 8, 2026
e3fb1d7
deps: take feldera-cloud1-client 0.1.3
gz Aug 8, 2026
309b4ba
deps: move the remaining rustls consumers to aws-lc-rs
gz Aug 8, 2026
9dba284
build: build librdkafka against AWS-LC
gz Aug 9, 2026
d34103b
deps: link librdkafka dynamically, dropping openssl-sys
gz Aug 9, 2026
c11fcae
ci: reject openssl-sys and a duplicate AWS-LC in the dependency tree
gz Aug 9, 2026
1834e30
deps: take aws-lc-rs 1.18.0
gz Aug 9, 2026
7d93901
ci: keep libssl-dev in the CI image for the cloud repository
gz Aug 9, 2026
315eb68
ci: take the feldera-dev image that carries librdkafka
gz Aug 9, 2026
fad7321
build: patch librdkafka with awk rather than perl
gz Aug 9, 2026
c001453
deps: floor aws-sigv4 at 1.4.5, where SigV4a left ring
gz Aug 9, 2026
fabf358
ci: drop libssl-dev from the build image
gz Aug 15, 2026
e9e4499
build: build librdkafka against AWS-LC v5.5.0, matching aws-lc-sys
gz Aug 15, 2026
010ab20
ci: take the feldera-dev image built without libssl-dev
gz Aug 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/build-docker-dev.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:
docker_platform: linux/arm64
runs-on: ${{ matrix.runner }}
container:
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-a5ab793b8d261068867b9fdfad3f04157b2536fc
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-e9e4499d6414ba10311bd7ef7cd20816e1d9f5c7

steps:
- name: Checkout Repository
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/build-docker.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ jobs:
docker_platform: linux/arm64
runs-on: ${{ matrix.runner }}
container:
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-a5ab793b8d261068867b9fdfad3f04157b2536fc
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-e9e4499d6414ba10311bd7ef7cd20816e1d9f5c7

steps:
- name: Checkout Repository
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/build-java.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:
AWS_ACCESS_KEY_ID: "${{ secrets.CI_GCS_HMAC_ACCESS_KEY_ID }}"
AWS_SECRET_ACCESS_KEY: "${{ secrets.CI_GCS_HMAC_SECRET_ACCESS_KEY }}"
container:
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-a5ab793b8d261068867b9fdfad3f04157b2536fc
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-e9e4499d6414ba10311bd7ef7cd20816e1d9f5c7
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
Expand Down Expand Up @@ -78,7 +78,7 @@ jobs:
id-token: write
contents: read
container:
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-a5ab793b8d261068867b9fdfad3f04157b2536fc
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-e9e4499d6414ba10311bd7ef7cd20816e1d9f5c7
steps:
- name: Download build artifact
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/build-rust.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ jobs:
runs-on: ${{ matrix.runner }}

container:
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-a5ab793b8d261068867b9fdfad3f04157b2536fc
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-e9e4499d6414ba10311bd7ef7cd20816e1d9f5c7

steps:
- name: Checkout repository
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci-post-release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ jobs:
adjust-versions:
runs-on: [gke-runners-amd64]
container:
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-a5ab793b8d261068867b9fdfad3f04157b2536fc
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-e9e4499d6414ba10311bd7ef7cd20816e1d9f5c7
steps:
- name: Generate GitHub App token
id: app-token
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci-pre-mergequeue.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
# because of how merge queues work: https://stackoverflow.com/a/78030618
main:
container:
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-a5ab793b8d261068867b9fdfad3f04157b2536fc
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-e9e4499d6414ba10311bd7ef7cd20816e1d9f5c7
runs-on: [gke-runners-amd64]
steps:
# Compute the internal-PR boolean once so individual steps can reference
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/publish-crates.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ jobs:
environment:
name: ${{ inputs.environment || 'release' }}
container:
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-a5ab793b8d261068867b9fdfad3f04157b2536fc
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-e9e4499d6414ba10311bd7ef7cd20816e1d9f5c7
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test-adapters.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ jobs:
target: aarch64-unknown-linux-gnu
runs-on: ${{ matrix.runner }}
container:
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-a5ab793b8d261068867b9fdfad3f04157b2536fc
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-e9e4499d6414ba10311bd7ef7cd20816e1d9f5c7
services:
postgres:
image: debezium/postgres:15
Expand Down
6 changes: 1 addition & 5 deletions .github/workflows/test-integration-platform.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,6 @@ on:
required: false
type: string

env:
FELDERA_SENTRY_ENABLED: 1
SENTRY_ENVIRONMENT: ci

jobs:
manager-no-network:
if: ${{ !contains(vars.CI_SKIP_JOBS, 'manager-no-network') }}
Expand Down Expand Up @@ -215,7 +211,7 @@ jobs:
FELDERA_HOST: https://pipeline-manager:8080

container:
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-a5ab793b8d261068867b9fdfad3f04157b2536fc
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-e9e4499d6414ba10311bd7ef7cd20816e1d9f5c7
services:
pipeline-manager:
image: ${{ vars.FELDERA_IMAGE_NAME }}:${{ inputs.image_tag || format('sha-{0}', github.sha) }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test-integration-runtime.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ jobs:
AWS_REQUEST_CHECKSUM_CALCULATION: WHEN_REQUIRED
AWS_RESPONSE_CHECKSUM_VALIDATION: WHEN_REQUIRED
container:
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-a5ab793b8d261068867b9fdfad3f04157b2536fc
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-e9e4499d6414ba10311bd7ef7cd20816e1d9f5c7
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test-java-nightly.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ jobs:
runs-on: ${{ matrix.runner }}

container:
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-a5ab793b8d261068867b9fdfad3f04157b2536fc
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-e9e4499d6414ba10311bd7ef7cd20816e1d9f5c7
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test-java.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ jobs:
runs-on: ${{ matrix.runner }}

container:
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-a5ab793b8d261068867b9fdfad3f04157b2536fc
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-e9e4499d6414ba10311bd7ef7cd20816e1d9f5c7
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test-unit.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
runs-on: ${{ matrix.runner }}

container:
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-a5ab793b8d261068867b9fdfad3f04157b2536fc
image: us-central1-docker.pkg.dev/feldera-ci/ghcr-remote/feldera/feldera-dev:sha-e9e4499d6414ba10311bd7ef7cd20816e1d9f5c7

steps:
- name: Checkout repository
Expand Down
7 changes: 7 additions & 0 deletions .pre-commit-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,13 @@ repos:
exclude: 'crates/pipeline-manager/migrations/.*|.*\.patch|.*\.svg|.*\.java'
- repo: local
hooks:
- id: validate-crypto-deps
name: Validate cryptography dependencies
description: Cryptography goes through AWS-LC; a second implementation corrupts the heap.
entry: scripts/validate-crypto-deps.sh
language: system
files: Cargo\.(toml|lock)$
pass_filenames: false
- id: fix-rust-mod-naming
name: Rust module naming
entry: scripts/fix-rust-module-naming
Expand Down
22 changes: 21 additions & 1 deletion CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,10 @@ Our dependencies for building the project are:

- C and C++ compiler toolchain (e.g., gcc, gcc++)
- cmake
- libssl-dev
- libsasl2-dev
- zlib1g-dev
- libzstd-dev
- Go, to build AWS-LC (see below)
- a Rust tool chain (install rustup and the default toolchain)
- a Java Virtual Machine (at least Java 19)
- maven
Expand All @@ -32,6 +33,25 @@ Our dependencies for building the project are:
Additional dependencies are automatically installed by the Rust,
maven, Python, and TypeScript build tools.

### librdkafka

The Kafka connectors link librdkafka dynamically, so it has to be installed
before the workspace will build:

```
./scripts/install-librdkafka.sh
```

The script builds librdkafka against AWS-LC, which is what keeps Kafka TLS on
the same cryptographic implementation as the rest of the system. Distribution
packages are built against OpenSSL and are usually older than the version the
`rdkafka-sys` crate requires, so installing one of those is not equivalent. Run
the script again after a `rdkafka` version bump; it reads the version it needs
from `Cargo.lock`.

Set `PREFIX` to install somewhere other than `/usr/local`, in which case
`PKG_CONFIG_PATH` has to point at `$PREFIX/lib/pkgconfig`.

## Contribution Flow

### Forking
Expand Down
Loading
Loading