Skip to content

Add Format::Truncated — enforce maximum byte size on serialized output#86

Merged
ioquatix merged 1 commit into
mainfrom
add-format-truncated
Jun 2, 2026
Merged

Add Format::Truncated — enforce maximum byte size on serialized output#86
ioquatix merged 1 commit into
mainfrom
add-format-truncated

Conversation

@ioquatix
Copy link
Copy Markdown
Member

@ioquatix ioquatix commented Jun 1, 2026

Summary

Adds Console::Format::Truncated, a format wrapper that enforces a maximum byte size on serialized log output.

If the serialized record exceeds the limit, top-level fields are replaced with "[TRUNCATED]" one at a time, largest first, until the record fits within the budget. If truncating every field is still insufficient, a minimal {"truncated":true} fallback is returned.

Design

  • Single serialization pass@format.dump is called exactly once. All truncation is done by direct byte-range surgery on the already-serialized string; no Ruby objects are re-allocated or re-serialized.
  • Shallow JSON parser — a hand-rolled scanner (parse / scan_string / skip_value / skip_whitespace) locates top-level key→value byte ranges. It handles JSON whitespace correctly and is robust against any conformant JSON serializer, not just JSON.dump's compact output.
  • Largest-first replacement — fields are replaced in descending size order so the minimum number of fields are truncated.
  • Truncation flag — a ,"truncated":true fragment is injected before the closing } so consumers can detect that the record was modified.

Usage

format = Console::Format::Truncated.new(Console::Format::Safe.new)
# or with a custom limit:
format = Console::Format::Truncated.new(Console::Format::Safe.new, maximum_size: 4096)

Files

  • lib/console/format/truncated.rb — implementation
  • lib/console/format.rb — added require_relative "format/truncated"
  • test/console/format/truncated.rb — 7 tests covering pass-through, largest-first truncation, multi-field truncation, skip-if-replacement-is-larger, fallback, and default size behaviour

@samuel-williams-shopify samuel-williams-shopify force-pushed the add-format-truncated branch 7 times, most recently from b1d2a91 to 4ab1ca3 Compare June 2, 2026 04:36
Fold maximum-size enforcement into Safe rather than a separate wrapper:

- Rename `limit` to `depth_limit` (with a deprecated `limit:` alias) to clarify its purpose alongside the new `size_limit`.
- Add `size_limit` (default 16 KiB, `nil` disables). When the fast serialization exceeds it, the record is rebuilt field-by-field, keeping as many top-level fields as fit.
- Unify the failure and size diagnostics into a single `truncated` object mapping each degraded field to why: `true` (dropped for size) or its error (value recovered, could not serialize directly). Falls back to `truncated: true` when detail does not fit.
- Serialize hash-like records via `to_hash`.

Removes the separate Console::Format::Truncated class.
@ioquatix ioquatix merged commit 964ba00 into main Jun 2, 2026
34 of 39 checks passed
@ioquatix ioquatix deleted the add-format-truncated branch June 2, 2026 04:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant