feat: Store default values for the feature in the registry by vanitabhagwat · Pull Request #336 · ExpediaGroup/feast

vanitabhagwat · 2026-01-23T18:00:28Z

What this PR does / why we need it:

Proto & pydantic model changes to add defaultvalue to registry.

Misc

EXPEbdodla · 2026-01-26T19:38:08Z

Add test cases for newly added fields to verify conversion from proto to python and backward compatibility
We need to make changes to feature.py for default values
Pydantic Models need changes as well

…east into registry_defaults

…e tests - Add Float32 default_value test - Add Float64 (Double) default_value test - Add Bytes default_value test - Add String array default_value test - Add Float32 array default_value test - Add Bool array default_value test - Updated imports to include Float64 and Bytes types Completes coverage gaps identified in Task 1 and approved by user.

- Add field_serializer to convert proto Value to JSON dict with camelCase keys - Add field_validator to accept both proto Value and dict for default_value - Use google.protobuf.json_format (MessageToDict/ParseDict) for conversion - Enables HTTP Registry endpoints to return defaultValue in JSON responses

- 11 unit tests covering all serialization scenarios - Tests primitive types: Int64, String, Float64, Bool - Tests None/null handling - Tests deserialization from dict and proto Value - Tests roundtrip: serialize -> deserialize - Tests Field bridge methods: to_field/from_field - Tests full roundtrip: Field -> FieldModel -> JSON -> FieldModel -> Field

- Add serialization_alias='defaultValue' for camelCase JSON output - Add validation_alias='defaultValue' to accept camelCase JSON input - Import Pydantic Field for alias configuration Enables HTTP REST API to use camelCase 'defaultValue' in JSON responses while maintaining snake_case 'default_value' in Python code.

- test_feature_view_proto_roundtrip_with_defaults: FeatureView with defaults - test_feature_view_proto_roundtrip_without_defaults: Backwards compatibility - test_feature_view_proto_roundtrip_mixed_defaults: Mixed fields with/without defaults - test_sorted_feature_view_proto_roundtrip_with_defaults: SortedFeatureView preserves sort_keys and defaults - test_feature_view_proto_bytes_identity: Verify proto wire format contains default_value All 5 tests pass. Verifies that FeatureView.to_proto() / from_proto() preserves Field.default_value through serialization, which is the path used by Remote Registry (feast serve_registry) gRPC communication.

- Configure FastAPI endpoints to exclude None values from JSON - Prevents defaultValue: null from appearing for fields without defaults - Maintains backwards compatibility with existing API consumers

- Removed prometheus import - Removed featureDefaultsApplied counter declaration and init() - Removed all 6 metric increment calls (preserved all debug logging)

- Removed prometheus import - Removed TestDefaultsMetricRegistered function - All existing tests pass

- Created 05-03-SUMMARY.md with execution details - Removed featureDefaultsApplied metric and test - Preserved all 6 debug logging statements - All verification checks passed

- Remove unused imports (pytest, PydanticField, Bytes) - Fix import ordering per ruff I001 rules - Move inline import to top of file in test_remote_registry_default_value.py All changes are auto-fixed by ruff --fix Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Format code per ruff/Black style guidelines - Add trailing commas to multiline dicts and function calls - Remove extra blank lines - Reformat long lines for better readability - Add blank lines after imports per PEP 8 All changes applied by: ruff format Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…hanges These changes are unrelated to the default value feature: - Restore debug logging in http.py - Restore comment punctuation in cassandra_online_store.py - Restore user_name fallback parsing in cassandraonlinestore.go - Restore 'Cassandra database' error message

…ed changes ## Critical Bug Fixes: 1. **Field equality now includes default_value comparison** - Fixed: Field.__eq__ was missing default_value in equality check - Impact: Two Fields with different defaults were incorrectly considered equal - Location: sdk/python/feast/field.py - Added proper proto Value comparison using SerializeToString() 2. **OUTSIDE_MAX_AGE status now handled in default logic** - Fixed: Expired features (OUTSIDE_MAX_AGE) were not replaced with defaults - Impact: Inconsistent behavior - missing features got defaults, expired didn't - Location: go/internal/feast/onlineserving/serving.go - Applied to both FLEXIBLE and STRICT modes in regular and range queries 3. **Invalid use_defaults mode now returns HTTP 400 error** - Fixed: Invalid mode values (e.g., "STRCT" typo) silently fell back to UNSPECIFIED - Impact: Users wouldn't know they made a mistake, defaults not applied when expected - Location: go/internal/feast/server/http_server.go - Changed parseUseDefaultsMode to return (UseDefaultsMode, error) - Both GetOnlineFeatures and GetOnlineFeaturesRange endpoints validate mode before execution ## Reverted Unrelated Changes: 4. **Reverted Go dependency changes (go.mod/go.sum)** - Reverted gocql version downgrade from 1.7.0 to 1.6.0 - Reverted scylladb/gocql replace directive - These changes are unrelated to default values and should be in separate PR 5. **Reverted SQL Registry breaking change** - Reverted: get_project_metadata_model → _get_project_metadata_model (private) - Impact: Breaking change for ExpediaGroup's eg-feature-store-registry - Unrelated to default values feature All changes focused on default value feature correctness and removing scope creep.

- Fix comment indentation in validate_default_value_type validator - Rename validator parameter from 'info' to 'values' (matches repo_config.py pattern) - Add default_value to Field.__repr__ (consistency with Feature.__repr__) All changes align with existing Feast codebase conventions.

The @field_serializer decorator only applies during JSON serialization. Changed model_dump() to model_dump(mode='json') to match existing patterns in test_pydantic_models.py. This fixes all 7 test failures in test_field_model_default_value.py.

…ut mode parameter) Changed test_field_model_default_value.py to use model_dump() instead of model_dump(mode='json') to match the existing pattern used across all other Pydantic model tests in test_pydantic_models.py. The @field_serializer decorator works correctly with both approaches, but using model_dump() without parameters is the established pattern in this codebase for dict serialization. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…lization The @field_serializer decorator needs when_used='always' to ensure it's consistently called across different environments when serializing arbitrary types (proto Value messages). Without this parameter, Pydantic may skip the serializer in certain contexts, causing the proto Value object to be returned directly instead of being converted to a JSON-compatible dict. This fixes all 7 failing unit tests in test_field_model_default_value.py: - test_field_model_serialize_int64_default - test_field_model_serialize_string_default - test_field_model_serialize_double_default - test_field_model_serialize_bool_default - test_field_model_deserialize_from_dict - test_field_model_roundtrip_json - test_field_model_full_roundtrip Tested locally: - All 11 tests in test_field_model_default_value.py pass - All 5 tests in test_remote_registry_default_value.py pass - All 16 tests in test_pydantic_models.py pass - Linting passes (ruff check, ruff format) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Remove OUTSIDE_MAX_AGE from default value application logic in serving.go (OUTSIDE_MAX_AGE represents stale data that should be returned as-is, not replaced with defaults) - Fix TestParseUseDefaultsMode to capture both return values (result and error) - Update comments and error messages to reflect correct default behavior Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

The @field_serializer with when_used='always' now works consistently, but mode='json' is still required for proper JSON serialization of proto Value objects to ensure they serialize as dicts rather than proto objects. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Python: - Remove mode='json' from all model_dump() calls in test_field_model_default_value.py - The @field_serializer with when_used='always' now works consistently without mode='json' - Using mode='json' causes Pydantic to serialize proto Values to raw values instead of dicts Go: - Update TestParseUseDefaultsMode to expect error for invalid input - The function now returns error for invalid strings instead of silently defaulting to UNSPECIFIED Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

The roundtrip tests were failing because model_dump() in python mode produces Python objects that don't properly round-trip through model_validate(). Changed to use: - model_dump_json() - serializes to JSON string - model_validate_json() - deserializes from JSON string This ensures the full JSON serialization/deserialization cycle works correctly with the @field_serializer and @field_validator for proto Value objects. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

The @field_serializer with when_used='always' wasn't consistently working in CI environments. Switching to @model_serializer with mode='wrap' which wraps the default serialization and explicitly converts the proto Value to a JSON-compatible dict using MessageToDict. This approach is more reliable because: - It intercepts the final serialization output - It explicitly handles the proto Value type - It works consistently across all Pydantic serialization modes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

The previous @model_serializer was trying to access the proto Value from the already-serialized data, but by that point Pydantic had already converted it to a raw value. Now we access self.default_value directly, which is the original proto Value object, before any serialization occurs. This ensures MessageToDict receives the actual proto Value and can properly convert it to a JSON-compatible dict. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Pydantic v2's PlainSerializer in Annotated type hints is the correct way to handle custom type serialization. This approach: 1. Applies to ALL serialization contexts (model_dump, model_dump_json, FastAPI) 2. Works with arbitrary_types_allowed for proto types 3. Follows Pydantic v2 best practices for custom serialization The PlainSerializer converts proto Value → dict for all outputs: - model_dump() → Python dict (for internal use, caching) - model_dump_json() → JSON string (for API responses) - FastAPI automatic serialization (for HTTP responses) The @field_validator handles the reverse: dict → proto Value for inputs. This matches the existing test pattern in test_pydantic_models.py where FieldModel/EntityModel roundtrip through model_dump() → model_validate(). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

PlainSerializer in Annotated types doesn't work consistently across Python versions. Using field_serializer with explicit return_type parameter is more compatible with Python 3.10 and Pydantic 2.10. The _info parameter is required by Pydantic's field_serializer signature even if not used. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Simplest approach that works across all Python versions: just override model_dump() to convert proto Value to dict after calling super(). This avoids all the complexity with field_serializer, PlainSerializer, etc. and works reliably. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

add default value field

a7d8b46

vanitabhagwat changed the title ~~add default value field~~ feat: feast proto & pydantic model changes to add defaultvalue to field Jan 23, 2026

fixing mypy issues

789aa14

vanitabhagwat changed the title ~~feat: feast proto & pydantic model changes to add defaultvalue to field~~ feat: Feast proto & pydantic model changes to add defaultvalue to field Jan 24, 2026

fixed formatting

46651f1

vbhagwat added 16 commits January 26, 2026 13:26

updated pydantic models

cefb1e1

add missing property

2a8317f

fix test failueres

4d75ee0

added unit tests

e022c03

adding compiled protos

377d8b0

generated protos

ff9655c

add default value field

df6f271

fixing mypy issues

bc9e537

fixed formatting

8b987e1

updated pydantic models

3395d2b

add missing property

fe771a7

fix test failueres

cde8f1e

added unit tests

c8dc514

adding compiled protos

af0b9ba

generated protos

ce5ca90

Merge branch 'registry_defaults' of https://github.com/ExpediaGroup/f…

9b0cb8b

…east into registry_defaults

vanitabhagwat changed the base branch from registry_defaults_feature to master January 29, 2026 17:51

vanitabhagwat changed the base branch from master to registry_defaults_feature January 29, 2026 17:52

vbhagwat added 6 commits February 27, 2026 00:30

fix(02-02): add response_model_exclude_none for backwards compatibility

f4c43b8

- Configure FastAPI endpoints to exclude None values from JSON - Prevents defaultValue: null from appearing for fields without defaults - Maintains backwards compatibility with existing API consumers

vbhagwat and others added 15 commits March 2, 2026 01:33

test(phase-05): diagnose UAT issue - metric properly exposed

39b4984

docs(05): create gap closure plan to remove Prometheus metric

bbc1008

refactor(05-03): remove Prometheus metric from serving.go

eefff29

- Removed prometheus import - Removed featureDefaultsApplied counter declaration and init() - Removed all 6 metric increment calls (preserved all debug logging)

test(05-03): remove Prometheus metric test from serving_test.go

d7bd99a

- Removed prometheus import - Removed TestDefaultsMetricRegistered function - All existing tests pass

docs(05-03): complete remove Prometheus metrics plan

4dc96cf

- Created 05-03-SUMMARY.md with execution details - Removed featureDefaultsApplied metric and test - Preserved all 6 debug logging statements - All verification checks passed

docs(phase-05): close gap - Prometheus metrics removed per user feedback

16b6ebb

removing docs

4a17666

docs(06): create phase plan for AI Workbench default value display

7321e56

style: apply ruff formatting to field.py

8a674cf

vanitabhagwat changed the title ~~feat: Feast proto & pydantic model changes to add defaultvalue to field~~ feat: Feast proto & pydantic model changes to add default value to field Mar 15, 2026

vanitabhagwat changed the title ~~feat: Feast proto & pydantic model changes to add default value to field~~ feat: Store default values for the feature in the registry Mar 15, 2026

vanitabhagwat and others added 13 commits March 15, 2026 16:15

fix: remove unused Callable import

1bca256

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

style: apply ruff formatting to field_model.py

1f51058

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Store default values for the feature in the registry#336

feat: Store default values for the feature in the registry#336
vanitabhagwat wants to merge 74 commits intoregistry_defaults_featurefrom
registry_defaults

vanitabhagwat commented Jan 23, 2026 •

edited

Loading

Uh oh!

EXPEbdodla commented Jan 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

vanitabhagwat commented Jan 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it:

Misc

Uh oh!

EXPEbdodla commented Jan 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

vanitabhagwat commented Jan 23, 2026 •

edited

Loading