feat: Add feature view versioning to HBase online store - #6755
Open
arose26 wants to merge 1 commit into
Open
Conversation
Thread registry.enable_online_feature_view_versioning through _table_id so that, when it is enabled, each feature view version gets its own HBase table (test_project:driver_stats_v2) instead of all versions sharing one. The version is appended via compute_versioned_name rather than by calling compute_table_id, because HBase addresses tables as namespace:table while compute_table_id joins with an underscore. Building the name here keeps the ':' separator, and with versioning disabled the table name is unchanged. Row keys stay unversioned. They are already suffixed with the feature view name to disambiguate views sharing a table, and since the table itself is now version-scoped the version would be redundant there; leaving them alone keeps existing keys stable and keeps the write and read paths computing identical keys, which a test pins. HbaseOnlineStore is deliberately not added to the versioned-read allowlist: online_read builds its result by iterating the rows HBase returned rather than the entity keys requested, so a missing row shortens the list instead of yielding (None, None) in place. That contract gap is pre-existing and independent of versioning. See the PR description. Part of feast-dev#2728. Closes feast-dev#6175 Signed-off-by: arose26 <145766958+arose26@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #6175. Part of #2728.
What this does
Threads
registry.enable_online_feature_view_versioningthroughHbaseOnlineStore._table_id,so that with versioning enabled each feature view version gets its own HBase table
(
test_project:driver_stats_v2) instead of all versions sharing one. The store alreadyfunnelled all five call sites —
online_write_batch,online_read,update(both the keepand delete loops) and
teardown— through that one method, so the change is localised there.Why not
compute_table_idThe merged Milvus (#6330) and FAISS (#6256) stores use
compute_table_id, which joins as{project}_{name}[_v{N}]. HBase addresses tables asnamespace:table, and this store hasalways produced
f"{project}:{table.name}". So the name is built here fromcompute_versioned_nameinstead, preserving the:separator and putting the version on thetable half where it belongs. With versioning disabled the result is byte-identical to today's,
which the
enable_versioning=Falsecases pin.Row keys are left unversioned
_hbase_row_keystill returns{entity_id}#{feature_view_name}. The existing comment explainsthe suffix disambiguates feature views that share a table; now that the table itself is
version-scoped, adding the version to the key would be redundant. Leaving it alone keeps
existing row keys stable, and — more importantly — keeps the write and read paths computing
identical keys, which
test_write_and_read_agree_on_row_keyspins directly.Why HBase is not added to the versioned-read allowlist
I checked the read path rather than assuming it.
HbaseOnlineStore.online_readbuilds itsresult by iterating the rows HBase returned, not the entity keys requested:
HBaseConnector.rowspasses straight through to happybase'sTable.rows, which omits keysthat do not exist. So a miss shortens the list rather than yielding
(None, None)in place,and the result stops corresponding positionally to
entity_keys— the contractsqlite.pyimplements. Demonstrated with mocks, no HBase required:
runnable reproduction
That gap is pre-existing and orthogonal to versioning, so I have left it out of scope rather
than widening this PR — versioned scalar reads stay correctly gated behind
VersionedOnlineReadNotSupporteduntil it is addressed. Happy to open a separate issue, or totake it in a follow-up.
Tests
New
sdk/python/tests/unit/infra/online_store/test_hbase_versioning.py, 15 tests,MagicMock-based in the same style as the mergedtest_milvus_versioning.py, andimportorskip-guarded onhappybase._v2when on;projection.version_tagtakes priority overcurrent_version_number; version0gets nosuffix; the
:namespace separator is preservedupdate(create and delete) andteardownall operate on theversioned table — write is parametrized so the unversioned case is asserted as the control
event_tsdecode path still round-trips
Verified red-before/green-after: with the source change reverted, the four routing tests fail
while the unversioned control and both invariant tests still pass, so they test the change
rather than the setup.
Regression check across the 55 unit test files touching online stores or versioning: the set of
failing and erroring tests is identical with and without this change (
diffof the sortedFAILED/ERRORlines is empty). Those pre-existing failures are missing optional dependenciesin my environment.
ruff check,ruff formatandmypyare clean on both files.🤖 Written with Claude Code (Claude Opus 5), reviewed by @arose26.