Skip to content

Send writers messages as one batch#799

Merged
vgvoleg merged 1 commit intomainfrom
batch_send_messages
Apr 1, 2026
Merged

Send writers messages as one batch#799
vgvoleg merged 1 commit intomainfrom
batch_send_messages

Conversation

@vgvoleg
Copy link
Copy Markdown
Collaborator

@vgvoleg vgvoleg commented Apr 1, 2026

Pull request type

Please check the type of change your PR introduces:

  • Bugfix
  • Feature
  • Code style update (formatting, renaming)
  • Refactoring (no functional changes, no api changes)
  • Build related changes
  • Documentation content changes
  • Other (please describe):

What is the current behavior?

Issue Number: N/A

What is the new behavior?

Other information

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the topic writer’s asyncio reconnector to send queued messages in larger batches (instead of one gRPC write per message), and extends test coverage to validate batching and request-splitting behavior.

Changes:

  • Batch sends in WriterAsyncIOReconnector._send_loop by draining _new_messages and calling writer.write() once per batch.
  • Update asyncio reconnector tests to expect batched sends and batched re-sends after reconnect.
  • Add unit tests for _split_messages_for_send() and messages_to_proto_requests() (codec-run grouping + max gRPC size splitting).

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
ydb/_topic_writer/topic_writer_asyncio.py Batch messages in the reconnector send loop to reduce per-message write calls.
ydb/_topic_writer/topic_writer_asyncio_test.py Adjust expectations for batched sends; add batching-focused tests.
ydb/_topic_writer/topic_writer_test.py Add unit tests for message grouping/splitting into proto write requests.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@vgvoleg vgvoleg force-pushed the batch_send_messages branch from fffcbbf to c9eb77d Compare April 1, 2026 12:36
@vgvoleg vgvoleg merged commit 24162f8 into main Apr 1, 2026
32 of 34 checks passed
@vgvoleg vgvoleg deleted the batch_send_messages branch April 1, 2026 12:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants