Skip to content

dataconnect(chore): Add QueryId value type and use it instead of passing around raw byte arrays#8207

Open
dconeybe wants to merge 1 commit into
mainfrom
dconeybe/dataconnect/QueryId
Open

dataconnect(chore): Add QueryId value type and use it instead of passing around raw byte arrays#8207
dconeybe wants to merge 1 commit into
mainfrom
dconeybe/dataconnect/QueryId

Conversation

@dconeybe
Copy link
Copy Markdown
Contributor

@dconeybe dconeybe commented May 24, 2026

This PR introduces the type-safe internal value class QueryId for Data Connect to represent stable identifiers for query cache entries, replacing raw ImmutableByteArray instances in caching APIs. It also shifts the CPU-intensive query ID calculation to a background worker thread to improve overall responsiveness, and establishes comprehensive testing tools to verify the hash algorithm remains consistent over time.

Highlights

  • Type-Safe QueryId: Replaced generic ImmutableByteArray hashes with a dedicated QueryId value class to represent stable query identifiers, improving type safety across the database caching layer and simplifying logging statements.
  • Thread-Safe Computation: Moved the CPU-heavy SHA-512 stable query ID calculation to a background dispatcher (nonBlockingCoroutineDispatcher) via a new internal @WorkerThread function calculateQueryId.
  • Stable Hash Verification: Implemented strict testing checks, combining property-based testing and frozen static test cases (QueryIdUnitTestTestCases.dat.gz) to protect the persistent cache key generation algorithm against accidental changes.
  • Custom Protobuf Serialization: Added custom verbatim serialization utilities for Struct objects to enable the generation and testing of frozen static hash samples.
Changelog
  • DataConnectGrpcRPCs.kt
    • Updated query caching info to use the new QueryId type instead of raw ImmutableByteArray.
    • Offloaded the calculateQueryId execution to a background task running on nonBlockingCoroutineDispatcher.
  • QueryId.kt
    • Created the QueryId internal inline value class wrapping ImmutableByteArray.
    • Added the @WorkerThread utility function calculateQueryId to compute the SHA-512 query identifier based on query variables and operation name.
  • DataConnectCacheDatabase.kt
    • Refactored cache insertion and search interfaces to accept the new QueryId type.
    • Simplified log messages by printing queryId directly instead of manually formatting it as a hex string.
  • ProtoValueSerialization.kt
    • Added serializeStructVerbatim and deserializeStructVerbatim utility functions to consistently serialize and deserialize complex Struct objects for static test cases.
  • QueryIdUnitTest.kt
    • Added property-based tests to ensure the inline calculation matches an independent implementation.
    • Added regression tests that load, decode, and verify frozen test cases from pre-computed target parameters.
  • DataConnectCacheDatabaseUnitTest.kt
    • Updated SQLite database unit tests to mock and test cache insertion/retrieval with QueryId samples.
  • ProtoValueSerializationUnitTest.kt
    • Added round-trip tests to verify the new stable struct serialization routines.
  • QueryIdUnitTestTestCases.dat.gz
    • Added binary-encoded test cases file containing the stable pre-computed variables-to-query-ID assets.

@gemini-code-assist
Copy link
Copy Markdown
Contributor

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

@github-actions
Copy link
Copy Markdown
Contributor

📝 PRs merging into main branch

Our main branch should always be in a releasable state. If you are working on a larger change, or if you don't want this change to see the light of the day just yet, consider using a feature branch first, and only merge into the main branch when the code complete and ready to be released.

@dconeybe
Copy link
Copy Markdown
Contributor Author

/gemini review

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a dedicated QueryId value class to represent stable, SHA-512-based identifiers for Data Connect queries, replacing the previous use of raw ImmutableByteArray. The changes include refactoring DataConnectGrpcRPCs to compute these IDs on a non-blocking dispatcher and updating the SQLite cache database to utilize the new type. Additionally, new utilities for verbatim Struct serialization and comprehensive unit tests have been added. Review feedback suggests optimizing performance by calculating the query ID only when caching is active and ensuring deterministic serialization by sorting protobuf map keys.

@dconeybe dconeybe marked this pull request as ready for review May 25, 2026 04:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant