feat(ourlogs): add truncate RPC parameter for logs events query#116008
feat(ourlogs): add truncate RPC parameter for logs events query#116008JoshuaKGoldberg wants to merge 9 commits into
truncate RPC parameter for logs events query#116008Conversation
Adds a `truncate` query parameter to the organization events endpoint that truncates string column values at the RPC layer for the OurLogs dataset, reducing payload size for wide viewports.
github-actions
Bot
added
the
Scope: Backend
| "organizations:dynamic-sampling", | ||
| "organizations:on-demand-metrics-extraction", | ||
| "organizations:on-demand-metrics-extraction-widgets", | ||
| "organizations:on-demand-metrics-extraction-experimental", |
There was a problem hiding this comment.
🤔 some kind of merge oddity, will remove
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
JoshuaKGoldberg
changed the title
truncate RPC parameter for logs events query
| result_value = process_value(result_value) | ||
|
|
||
| # Note: post-query truncation may not be our preferred method long-term. | ||
| # We may want to set up a function that filters/truncates at the EAP side. |
There was a problem hiding this comment.
For my own learning, why is this not the preferred approach for this PR, time/scope?
There was a problem hiding this comment.
Please enjoy this 84-reply thread 😄 https://sentry.slack.com/archives/C08CR80T3RB/p1778786746769669
(Kevan and Will can speak to this better than me)
| truncate_str = request.GET.get("truncate") | ||
| if truncate_str is not None: | ||
| try: | ||
| max_string_length = int(truncate_str) |
There was a problem hiding this comment.
Ahh ok, this one does cast int here, so at least backend side is protected, sweet
adrianviquez
left a comment
adrianviquez
left a comment
There was a problem hiding this comment.
small q, otherwise 👍
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
![sentry-warden[bot]](https://avatars.githubusercontent.com/in/2755838?s=60&v=4){kind=small}
There was a problem hiding this comment.
max_string_length silently dropped in PreprodSize.run_table_query — truncation never applied (src/sentry/snuba/preprod_size.py:44)
The max_string_length parameter is accepted but not forwarded to rpc_dataset_common.TableQuery(...), so string truncation is never applied for this dataset even when the caller explicitly requests it.
Evidence
run_table_queryacceptsmax_string_length: int | None = Noneat line 44 but the value is unused.- The
TableQuerydataclass inrpc_dataset_common.py:94has amax_string_lengthfield;build_rpc_table_row_contextat line 246 propagates it to the context dict only when notNone. process_column_valuesat line 482 readsmax_string_lengthfrom context and applies truncation — unreachable here because it is never set.ourlogs.py:82shows the correct pattern:max_string_length=max_string_lengthpassed intoTableQuery.organization_events.py:617/638passes the user-suppliedmax_string_lengthto every dataset'srun_table_query, so the parameter reachesPreprodSizebut is then silently discarded.
Identified by Warden sentry-backend-bugs
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 155c952. Configure here.
JoshuaKGoldberg
force-pushed
the
logs-query-truncate-backend
branch
from
155c952 to
1a6ae6d
Compare
3 participants
Adds a
truncatequery parameter to the organization events endpoint for the OurLogs dataset. When provided, string column values are truncated at the RPC layer to the specified length. This can reduce response payload size nicely when there are huge log fields, such as -but not limited to-message.Split out of #115638. #116009 has the corresponding frontend changes.
Closes LOGS-816.