Skip to content

api: add PoolAllocator for response buffers#543

Merged
oleg-jukovec merged 2 commits intomasterfrom
ermyar/gh-493-reduce-read-allocations
Apr 7, 2026
Merged

api: add PoolAllocator for response buffers#543
oleg-jukovec merged 2 commits intomasterfrom
ermyar/gh-493-reduce-read-allocations

Conversation

@ermyar
Copy link
Copy Markdown
Collaborator

@ermyar ermyar commented Mar 3, 2026

The new Allocator interface allows users to implement custom buffer
allocation and reuse strategies:

// Allocator is an interface for allocating and deallocating byte
// slices.
type Allocator interface {
    // Get returns a pointer to a byte slice of at least the given
    // length.
    // The caller should not assume anything about the slice's
    // capacity.
    //
    // If the allocator cannot allocate a buffer (e.g., invalid
    // length), it returns nil. The caller must handle this case
    // appropriately.
    Get(length int) *[]byte
    // Put returns a byte slice to the allocator for reuse.
    // After calling Put, the caller must not use the slice.
    //
    // The caller must ensure that the slice length remains unchanged
    // between Get and Put calls. Modifying the slice length before
    // calling Put may prevent the allocator from properly reusing the
    // buffer.
    Put(buf *[]byte)
}

The PoolAllocator type provides a ready-to-use implementation based
on sync.Pool for power-of-two sized byte slices.

The Opts.Allocator option enables configuring a custom allocator
for a connection. This is useful for applications that need to
optimize memory usage or integrate with custom memory management
systems.

Closes #493

@ermyar ermyar requested a review from oleg-jukovec March 3, 2026 14:55
@ermyar ermyar linked an issue Mar 3, 2026 that may be closed by this pull request
Base automatically changed from ermyar/gh-493-reduce-reader-allocations to master March 5, 2026 12:59
@oleg-jukovec
Copy link
Copy Markdown
Collaborator

Please, rebase the PR.

@ermyar ermyar force-pushed the ermyar/gh-493-reduce-read-allocations branch from e61434b to 05bb9da Compare March 5, 2026 13:34
Copy link
Copy Markdown
Collaborator

@oleg-jukovec oleg-jukovec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the patch!

Except for the comments, everything is fine here. But let's improve the solution a bit here before merge.

Let's use the approach from the grpc-go:

https://github.com/grpc/grpc-go/blob/master/mem/buffer_pool.go

Create multiple buffers with constant byte sizes instead of the single one. Response that does not fit into these buffers should be allocated separately without a pool and without a release.

It shouldn't be copy-paste, but inspirated the go-grpc's code.

Another option is that you can correct the current comments and we will merge the PR, and you will implement logic with buffer pools in another one.

Comment thread connection.go Outdated
Comment thread response.go Outdated
@ermyar ermyar force-pushed the ermyar/gh-493-reduce-read-allocations branch 5 times, most recently from 0bfee57 to 0d3cbe1 Compare March 12, 2026 00:30
@ermyar ermyar requested review from bigbes and oleg-jukovec March 12, 2026 06:35
Comment thread slice_pool.go Outdated
Comment thread slice_pool.go Outdated
Comment thread slice_pool.go Outdated
@ermyar ermyar force-pushed the ermyar/gh-493-reduce-read-allocations branch from 0d3cbe1 to 322ad09 Compare March 12, 2026 15:52
@ermyar ermyar requested a review from bigbes March 12, 2026 18:04
@oleg-jukovec oleg-jukovec force-pushed the ermyar/gh-493-reduce-read-allocations branch from 322ad09 to 9501898 Compare March 17, 2026 14:59
Copy link
Copy Markdown
Collaborator

@oleg-jukovec oleg-jukovec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. If a user do not use the Release() than the commits adds +1 allocation per request. Let the pool be a configurable feature with Opts. enabled by default.
  2. We need to update our code (in pool, queue, crud etc.). To release requests to be consistent with the feature.

Comment thread slice_pool.go Outdated
Comment thread connection.go Outdated
@ermyar ermyar force-pushed the ermyar/gh-493-reduce-read-allocations branch from 9501898 to 0c5a223 Compare March 25, 2026 22:19
@ermyar ermyar requested a review from oleg-jukovec March 26, 2026 06:20
Comment thread connection.go Outdated
Comment thread connection.go Outdated
Comment thread connection.go Outdated
Comment thread tarantool_test.go
@ermyar ermyar force-pushed the ermyar/gh-493-reduce-read-allocations branch 2 times, most recently from 9519cd2 to 27fd5b2 Compare April 1, 2026 22:22
@ermyar ermyar requested a review from oleg-jukovec April 2, 2026 08:06
The commit introduces ability to release resources allocated to
read response data.

Part of #493
@oleg-jukovec oleg-jukovec force-pushed the ermyar/gh-493-reduce-read-allocations branch 2 times, most recently from 5a31264 to ed58ed1 Compare April 2, 2026 15:33
@oleg-jukovec oleg-jukovec force-pushed the ermyar/gh-493-reduce-read-allocations branch 3 times, most recently from 73832e7 to e5ea54d Compare April 3, 2026 05:57
@oleg-jukovec oleg-jukovec changed the title api: added future/response releasing methods api: add Allocator interface for response buffers Apr 3, 2026
@oleg-jukovec oleg-jukovec changed the title api: add Allocator interface for response buffers api: add PoolAllocator for response buffers Apr 3, 2026
Comment thread poolalloc.go
Comment thread poolalloc.go
Comment thread poolalloc.go
Comment thread connection.go Outdated
Previously, response buffer allocation was hardcoded with an internal
pooler implementation. Users had no control over memory management
for responses, which could be problematic for applications with
specific memory requirements or custom allocation strategies.

The new Allocator interface allows users to implement custom buffer
allocation and reuse strategies:

```
// Allocator is an interface for allocating and deallocating byte
// slices.
type Allocator interface {
    // Get returns a pointer to a byte slice of at least the given
    // length.
    // The caller should not assume anything about the slice's
    // capacity.
    //
    // If the allocator cannot allocate a buffer (e.g., invalid
    // length), it returns nil. The caller must handle this case
    // appropriately.
    Get(length int) *[]byte
    // Put returns a byte slice to the allocator for reuse.
    // After calling Put, the caller must not use the slice.
    //
    // The caller must ensure that the slice length remains unchanged
    // between Get and Put calls. Modifying the slice length before
    // calling Put may prevent the allocator from properly reusing the
    // buffer.
    Put(buf *[]byte)
}
```

The PoolAllocator type provides a ready-to-use implementation based
on sync.Pool for power-of-two sized byte slices.

The Opts.Allocator option enables configuring a custom allocator
for a connection. This is useful for applications that need to
optimize memory usage or integrate with custom memory management
systems.

Closes #493
@oleg-jukovec oleg-jukovec force-pushed the ermyar/gh-493-reduce-read-allocations branch from e5ea54d to 1f0b9b9 Compare April 3, 2026 16:20
@oleg-jukovec oleg-jukovec requested a review from bigbes April 4, 2026 08:53
@oleg-jukovec oleg-jukovec merged commit 7858b1b into master Apr 7, 2026
27 checks passed
@oleg-jukovec oleg-jukovec deleted the ermyar/gh-493-reduce-read-allocations branch April 7, 2026 17:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

v3: design API to avoid allocations in responses

4 participants