feat: Add OnlineStore for MongoDB by caseyclements · Pull Request #6025 · feast-dev/feast

caseyclements · 2026-02-26T00:30:37Z

What this PR does / why we need it:

Adds a first-class MongoDB online store integration at feast.infra.online_stores.mongodb_online_store.

Schema

Each entity is stored as a single MongoDB document keyed by its serialized entity key:

{
  "_id": "<serialized_entity_key>",
  "features": { "<feature_view>": { "<feature>": <value> } },
  "event_timestamps": { "<feature_view>": "<datetime>" },
  "created_timestamp": "<datetime>"
}

Because MongoDB has a loose schema and supports upsert semantics natively, update() requires no
pre-creation — it only removes feature views named in tables_to_delete via $unset. Multiple
feature views for the same entity share one document.

Implementation highlights

Sync and async paths — implements online_write_batch, online_read,
online_write_batch_async, online_read_async, and async def close().
async_supported returns read=True, write=True.
requested_features projection — both read paths build a MongoDB field projection so only
the requested feature columns are returned from the server.
Efficient BSON → proto conversion — _convert_raw_docs_to_proto transforms column-wise to
minimise calls to python_values_to_proto_values (one call per feature across all entities,
rather than one call per entity × feature). Benchmarking confirmed this is ~3–4× faster than a
naïve row-wise approach across all scaling dimensions (entities, features, feature views).
Composite entity keys — entities are serialized with serialize_entity_key, so composite
keys (e.g. customer_id + driver_id) are handled without schema changes.

Tests

Docker integration test (test_mongodb_online_features) — spins up a real MongoDB container
via testcontainers, writes to three feature views (single int key, single string key, composite
key), and asserts correct retrieval including type coercion and missing-entity handling.
Pure Python unit tests (no Docker required):
- test_convert_raw_docs_missing_entity — entity absent from query results → (None, None)
- test_convert_raw_docs_partial_doc — entity present but a feature key missing → empty
  ValueProto (schema-migration safety)
- test_convert_raw_docs_ordering — result order follows the requested ids list regardless
  of MongoDB cursor order

All tests pass. Code is clean under mypy, ruff check, and ruff format.

shuchu · 2026-02-28T01:35:15Z

@caseyclements please check the errors in the tests :)

caseyclements · 2026-03-03T15:04:31Z

@shuchu Would you please help me investigate the CI timeout? I can run the universal tests for mongodb locally, like so:

$ cd /Users/casey.clements/src/feast/sdk/python && PYTHONPATH='../../' FULL_REPO_CONFIGS_MODULE=sdk.python.feast.infra.online_stores.mongodb_online_store.mongodb_repo_configuration PYTEST_PLUGINS=sdk.python.tests.integration.feature_repos.universal.online_store.mongodb ../../.venv/bin/python -m pytest --integration -v -k "MongoDBOnlineStoreCreator" tests/integration/online_store/test_universal_online.py
...
================================================================== 6 passed, 10 deselected, 37 warnings in 15.49s ===================================================================

ntkathole · 2026-03-03T15:54:29Z

@caseyclements Please rebase with master, it would solve the CI issue

caseyclements · 2026-03-03T19:06:22Z

Thanks @ntkathole , @shuchu . Would you please approve further workflows if applicable.

caseyclements · 2026-03-04T17:02:52Z

Hi @shuchu, @ntkathole, @oavdeev, @ankurs. All checks are now passing on approved workflows. Would you please kick off the others? Of course, your personal feedback would be great as well. In the meantime, I am working on an offline store.

caseyclements · 2026-03-04T22:15:24Z

Hi @shuchu, @ntkathole, @oavdeev, @ankurs. I made a small change in doctest.test_all.py that should fix the issue with pymongo not being available. Would you please approve the workflows again?

ntkathole · 2026-03-05T04:50:34Z

• [FAILED] [0.000 seconds]
FeatureStore Data Source Types When checking against the python code in feast.repo_config [It] should match defined onlineStore persistence types in the operator
/home/runner/work/feast/feast/infra/feast-operator/test/data-source-types/data_source_types_test.go:29

  [FAILED] Expected
      <[]string | len:1, cap:1>: ["mongodb"]
  to be empty
  In [It] at: /home/runner/work/feast/feast/infra/feast-operator/test/data-source-types/data_source_types_test.go:65 @ 03/05/26 02:45:55.162
------------------------------

Need to add mongodb in operator as well

caseyclements · 2026-03-05T19:07:45Z

• [FAILED] [0.000 seconds]
FeatureStore Data Source Types When checking against the python code in feast.repo_config [It] should match defined onlineStore persistence types in the operator
/home/runner/work/feast/feast/infra/feast-operator/test/data-source-types/data_source_types_test.go:29

[FAILED] Expected
<[]string | len:1, cap:1>: ["mongodb"]
to be empty
In [It] at: /home/runner/work/feast/feast/infra/feast-operator/test/data-source-types/data_source_types_test.go:65 @ 03/05/26 02:45:55.162

@ntkathole Thanks. The go files weren't on my radar. It looks like we're almost there. I updated a few small pieces based on feedback from our product team. Would you please kick off the workflows?

ntkathole · 2026-03-06T12:06:02Z

@caseyclements seems linting issue with operator code, you can run make lint in feast-operator dir to fix

caseyclements · 2026-03-06T17:41:49Z

@caseyclements seems linting issue with operator code, you can run make lint in feast-operator dir to fix

I believe this might be fixed by updating the GOLANGCI_LINT version. This fixes the issue which I could reproduce locally. No actual changes were needed though. It was a tooling issue.

caseyclements · 2026-03-06T18:48:01Z

@ntkathole Can I get a rerun of the workflows please? Is there someone on the west coast that can do another run?
CC: @shuchu, @oavdeev, @ankurs.

ntkathole · 2026-03-09T12:35:26Z

+        We remove any feature views named in tables_to_delete.
+        The Entities are serialized in the _id. No schema needs be adjusted.
+        """
+        if config.online_store.type != "mongodb":


Suggested change

if config.online_store.type != "mongodb":

if not isinstance(config.online_store, MongoDBOnlineStoreConfig):

Good to be consistent across

ntkathole · 2026-03-09T12:44:54Z

+    _client_async: Optional[AsyncMongoClient] = None
+    _collection_async: Optional[AsyncCollection] = None
+
+    def online_write_batch(


online_write_batch and online_write_batch_async has duplicated logic written in two different styles, we can extract the shared logic into a static helper that both paths call

caseyclements · 2026-03-09T14:38:58Z

HI @ntkathole I'll make the suggested changes. Is the one failing test a red-herring? It appears to be an unrelated timeout.

ntkathole · 2026-03-09T14:47:49Z

HI @ntkathole I'll make the suggested changes. Is the one failing test a red-herring? It appears to be an unrelated timeout.

yes, unrelated

ntkathole · 2026-03-09T14:48:27Z

Please rebase with master, seems there are some conflicts

caseyclements · 2026-03-09T14:58:31Z

Please rebase with master, seems there are some conflicts

Can I merge, or is only rebase permitted?

devin-ai-integration

Devin Review found 1 new potential issue.

View 19 additional findings in Devin Review.

devin-ai-integration · 2026-03-09T15:03:47Z

+    async def close(self) -> None:
+        """Close the async MongoDB client and release its resources."""
+        if self._client_async is not None:
+            await self._client_async.close()
+            self._client_async = None
+            self._collection_async = None


🟡 Resource leak: teardown() never closes async client, close() never closes sync client

The MongoDBOnlineStore maintains two independent clients: _client (sync MongoClient) and _client_async (async AsyncMongoClient). The teardown() method (line 273-276) only closes the sync _client and leaves _client_async open. Conversely, the close() method (line 280-283) only closes _client_async and leaves the sync _client open. If both clients are created during the lifetime of a store instance (e.g., sync writes via online_write_batch followed by async reads via online_read_async, or vice versa), the other client's connection pool will leak when cleanup runs through either path.

Suggested change

async def close(self) -> None:

"""Close the async MongoDB client and release its resources."""

if self._client_async is not None:

await self._client_async.close()

self._client_async = None

self._collection_async = None

async def close(self) -> None:

"""Close both async and sync MongoDB clients and release their resources."""

if self._client_async is not None:

await self._client_async.close()

self._client_async = None

self._collection_async = None

if self._client is not None:

self._client.close()

self._client = None

self._collection = None

Was this helpful? React with 👍 or 👎 to provide feedback.

…t.toml Any change to pyproject.toml invalidates the pixi.lock manifest hash, causing 'pixi install --locked' to fail in CI even when the changed section (mongodb optional extras) is not used by any pixi environment. Regenerated with: pixi install (v0.63.2) Signed-off-by: Casey Clements <casey.clements@mongodb.com>

Stores like mongodb, redis, and dynamodb raise FeastExtrasDependencyImportError at import time when their optional Python extras are not installed. test_all.py only caught ModuleNotFoundError, so any such import caused the entire test_docstrings() function to abort rather than gracefully skipping the module. Extend the except clause to also catch FeastExtrasDependencyImportError so the doctest run completes for all other modules when an optional extra is absent in the test environment. Signed-off-by: Casey Clements <casey.clements@mongodb.com>

Directory was renamed from tests/integration/feature_repos to tests/universal/feature_repos in upstream/master. Update the PYTEST_PLUGINS module path in test-python-universal-mongodb-online to match the new location. Signed-off-by: Casey Clements <casey.clements@mongodb.com>

…eError and other third-party import errors qdrant_client raises TypeError at import time when a gRPC EnumTypeWrapper is used with the | union operator on Python < 3.12. The previous fix only caught ModuleNotFoundError and FeastExtrasDependencyImportError, leaving the test runner vulnerable to any other exception raised by third-party libraries during pkgutil.walk_packages. Changes: - Catch bare Exception in the import try/except block so any import-time error from an optional dependency causes a graceful skip rather than an abort of the entire test run. - Initialize temp_module = None before the try block and add a continue guard so that a failed import never leaves a stale module reference to be used in the subsequent doctest execution block. Signed-off-by: Casey Clements <casey.clements@mongodb.com>

…r package import failures FeastExtrasDependencyImportError inherits from FeastError -> Exception, not from ImportError. pkgutil.walk_packages calls __import__ internally when recursing into sub-packages, and only silently swallows ImportError subclasses; any other exception is re-raised, crashing the entire test run. Passing onerror=lambda _: None tells walk_packages to skip any package that fails to import during the walk phase, regardless of the exception type. The inner importlib.import_module try/except already handles the same errors for the explicit import step used to collect doctests. Signed-off-by: Casey Clements <casey.clements@mongodb.com>

…iversal.feature_repos The upstream directory tests/integration/feature_repos was renamed to tests/universal/feature_repos. Three files still referenced the old path: - mongodb_repo_configuration.py: IntegrationTestRepoConfig and MongoDBOnlineStoreCreator imports - tests/.../online_store/mongodb.py: OnlineStoreCreator import - tests/unit/online_store/test_mongodb_online_retrieval.py: TAGS import Updating all three unblocks test collection and allows make test-python-universal-mongodb-online to run locally. Signed-off-by: Casey Clements <casey.clements@mongodb.com>

…-operator The Python ONLINE_STORE_CLASS_FOR_TYPE now includes 'mongodb', so the operator's parity check (test-datasources) fails unless the Go API also lists it. Add 'mongodb' to ValidOnlineStoreDBStorePersistenceTypes in both api/v1 and api/v1alpha1. Signed-off-by: Casey Clements <casey.clements@mongodb.com>

Add DriverInfo with the Feast name and version to both the sync MongoClient and async AsyncMongoClient so that MongoDB can identify traffic originating from a Feast AI integration. Signed-off-by: Casey Clements <casey.clements@mongodb.com>

Signed-off-by: Casey Clements <casey.clements@mongodb.com>

…rePersistence The +kubebuilder:validation:Enum annotation on the Type field of OnlineStoreDBStorePersistence was not updated when mongodb was added to ValidOnlineStoreDBStorePersistenceTypes. This annotation drives CRD OpenAPI schema validation at Kubernetes admission time, so any FeatureStore CR specifying type: mongodb would be rejected by the API server. Updated both api/v1 and api/v1alpha1. Signed-off-by: Casey Clements <casey.clements@mongodb.com>

Signed-off-by: Casey Clements <casey.clements@mongodb.com>

…onsistent with sync counterpart Signed-off-by: Casey Clements <casey.clements@mongodb.com>

Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>

Signed-off-by: Casey Clements <casey.clements@mongodb.com>

…ially identical. Signed-off-by: Casey Clements <casey.clements@mongodb.com>

caseyclements · 2026-03-10T13:34:39Z

@caseyclements Can you please rebase with master, I am not able to squash and merge.

Sure can. Until what time are you online today?

next 4 hrs, please take your time.

@ntkathole I've rebased all my commits from the latest master, and run tests locally. Please approve workflows and have a look.

caseyclements requested a review from a team as a code owner February 26, 2026 00:30

This comment was marked as resolved.

Sign in to view

caseyclements force-pushed the INTPYTHON-297-MongoDB-Feast-Integration branch from 5a815be to 6581159 Compare February 27, 2026 21:46

shuchu added the ok-to-test label Feb 28, 2026

This comment was marked as resolved.

Sign in to view

caseyclements force-pushed the INTPYTHON-297-MongoDB-Feast-Integration branch 2 times, most recently from 6aa5578 to 91dc75c Compare March 2, 2026 22:16

caseyclements force-pushed the INTPYTHON-297-MongoDB-Feast-Integration branch from 91dc75c to bcaa548 Compare March 3, 2026 16:18

This comment was marked as resolved.

Sign in to view

ntkathole reviewed Mar 9, 2026

View reviewed changes

devin-ai-integration Bot reviewed Mar 9, 2026

View reviewed changes

caseyclements and others added 15 commits March 10, 2026 09:02

docs: update MongoDB online store status from alpha to preview

1492a1e

Signed-off-by: Casey Clements <casey.clements@mongodb.com>

Update +GOLANGCI_LINT_VERSION to fix upstream issue golang/go#74462

7563f11

Signed-off-by: Casey Clements <casey.clements@mongodb.com>

fix: raise ValueError in _get_client_async for invalid config type, c…

26ad3bb

…onsistent with sync counterpart Signed-off-by: Casey Clements <casey.clements@mongodb.com>

fix: Added mongodb to operator yamls

2f92af9

Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>

Small change suggested by ntkathole

577f2a7

Signed-off-by: Casey Clements <casey.clements@mongodb.com>

Factor out write logic into utility function making sync/async essent…

e761e10

…ially identical. Signed-off-by: Casey Clements <casey.clements@mongodb.com>

caseyclements force-pushed the INTPYTHON-297-MongoDB-Feast-Integration branch from cb9e1d6 to e761e10 Compare March 10, 2026 13:13

ntkathole merged commit bf4e3fa into feast-dev:master Mar 10, 2026

	if config.online_store.type != "mongodb":
	if not isinstance(config.online_store, MongoDBOnlineStoreConfig):

Uh oh!

Conversation

caseyclements commented Feb 26, 2026 • edited by devin-ai-integration Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it:

Schema

Implementation highlights

Tests

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

shuchu commented Feb 28, 2026

Uh oh!

This comment was marked as resolved.

Uh oh!

caseyclements commented Mar 3, 2026

Uh oh!

ntkathole commented Mar 3, 2026

Uh oh!

This comment was marked as resolved.

Uh oh!

caseyclements commented Mar 3, 2026

Uh oh!

This comment was marked as resolved.

Uh oh!

caseyclements commented Mar 4, 2026

Uh oh!

caseyclements commented Mar 4, 2026

Uh oh!

ntkathole commented Mar 5, 2026

Uh oh!

This comment was marked as resolved.

Uh oh!

caseyclements commented Mar 5, 2026

[FAILED] Expected <[]string | len:1, cap:1>: ["mongodb"] to be empty In [It] at: /home/runner/work/feast/feast/infra/feast-operator/test/data-source-types/data_source_types_test.go:65 @ 03/05/26 02:45:55.162

Uh oh!

This comment was marked as resolved.

Uh oh!

ntkathole commented Mar 6, 2026

Uh oh!

caseyclements commented Mar 6, 2026

Uh oh!

caseyclements commented Mar 6, 2026

Uh oh!

ntkathole Mar 9, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ntkathole Mar 9, 2026

Choose a reason for hiding this comment

Uh oh!

caseyclements commented Mar 9, 2026

Uh oh!

ntkathole commented Mar 9, 2026

Uh oh!

ntkathole commented Mar 9, 2026

Uh oh!

caseyclements commented Mar 9, 2026

Uh oh!

devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot Mar 9, 2026

Choose a reason for hiding this comment

Uh oh!

caseyclements commented Mar 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

caseyclements commented Feb 26, 2026 •

edited by devin-ai-integration Bot

Loading

[FAILED] Expected
<[]string | len:1, cap:1>: ["mongodb"]
to be empty
In [It] at: /home/runner/work/feast/feast/infra/feast-operator/test/data-source-types/data_source_types_test.go:65 @ 03/05/26 02:45:55.162