feat: Add HybridOfflineStore for supporting multiple offline store backends#5542
Closed
PrasannaKumarARDS wants to merge 3 commits intofeast-dev:hybrid-offline-storefrom
Closed
feat: Add HybridOfflineStore for supporting multiple offline store backends#5542PrasannaKumarARDS wants to merge 3 commits intofeast-dev:hybrid-offline-storefrom
PrasannaKumarARDS wants to merge 3 commits intofeast-dev:hybrid-offline-storefrom
Conversation
added 3 commits
June 21, 2025 09:58
- Implements HybridOnlineStore, enabling routing of online feature operations to different backends based on a configurable tag (e.g., tribe, team, or project) on the FeatureView. - Adds support for specifying the routing tag name via the 'routing_tag' field in the online_store config, allowing flexible backend selection. - Supports multi-tenancy and flexible data management by allowing multiple online store backends in a single Feast deployment. - added documentation - fixed linter raised issues Signed-off-by: r0b0fyi <renukaprasannakumar.badugu@walmart.com>
- Implements HybridOnlineStore, enabling routing of online feature operations to different backends based on a configurable tag (e.g., tribe, team, or project) on the FeatureView. - Adds support for specifying the routing tag name via the 'routing_tag' field in the online_store config, allowing flexible backend selection. - Supports multi-tenancy and flexible data management by allowing multiple online store backends in a single Feast deployment. - added documentation - fixed linter raised issues Signed-off-by: r0b0fyi <renukaprasannakumar.badugu@walmart.com>
The HybridOfflineStore allows routing operations to different offline store implementations based on data source type. This enables a single Feast deployment to work with multiple offline storage backends simultaneously, improving flexibility for organizations with heterogeneous data infrastructure. Key features: - Configure multiple offline stores in a single Feast deployment - Automatic routing of operations to the appropriate offline store - Support for all existing offline store operations - Transparent to feature views and other Feast components Signed-off-by: r0b0fyi <renukaprasannakumar.badugu@walmart.com>
Collaborator
|
Really appreciate your contribution. I made the first version of the hybrid store and already in the codebase now. Do you mind taking a look see if there is something missing and you want to optimize? |
Member
|
@PrasannaKumarARDS any update on this? |
Contributor
Author
|
Apologies for missing this earlier. I will review the version/implementation available at the earliest opportunity and provide feedback or recommendations for any potential improvements. Thank you for your patience, and I appreciate you highlighting this. |
|
I believe |
Member
|
@PrasannaKumarARDS any update on this? Want to see if we should aim to get it in the next release |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
This PR adds a new HybridOfflineStore implementation that can route operations to different offline stores based on data source type. This enables organizations with heterogeneous data infrastructure to use a single Feast deployment while storing features in multiple different storage systems.
Key features:
Which issue(s) this PR fixes:
Fixes #5541
Misc
This implementation follows the existing patterns in Feast's offline store design. It introduces a new configuration type
HybridOfflineStoreConfigthat contains a list of offline store configurations.Testing: