Skip to content

Introduce context to golang APIs#7719

Open
dentiny wants to merge 14 commits into
apache:mainfrom
dentiny:hjiang/opendal-golang-context
Open

Introduce context to golang APIs#7719
dentiny wants to merge 14 commits into
apache:mainfrom
dentiny:hjiang/opendal-golang-context

Conversation

@dentiny

@dentiny dentiny commented Jun 10, 2026

Copy link
Copy Markdown
Member

Which issue does this PR close?

Closes #7608

Rationale for this change

Using context for IO operations is the general practice in golang; I cannot push forward adoption without this.

What changes are included in this PR?

This PR adds context to all golang interfaces.

High-level designs

Design principle:

  • for C binding, any existing documentation and functionality are kept unchanged for backward compatibility; we only expect async IO and cancellation related changes
  • for golang binding, it's intentional to do breaking API change to have context passed as the first argument; when context timeout or cancels, underlying IO operations should be cancelled in a blocking manner, without actual IO operation completes

There're a key changes:

  • Add cancellation token in C binding, so go binding layer could cancel the ongoing request when context done
  • In go binding layer, all IO operations are performed in a separate goroutine, with a select statement subscribing both completed IO operation signal, and finished context

Testing and code review

Consideration this is a huge PR, I did a few things to improve correctness guarantee:

  • General unit tests and behavior tests
  • Use both fable-5 and opus-4.7 for a few rounds of code review until they're happy with it
  • Test against real S3 with (1) normal IO operations, (2) with cancellation, (3) with timeout, for 20 minutes and runs fine (no unexpected behavior, no crash)
  • I manually review the core parts

Key code reference

How golang side handles IO and context finish
https://github.com/dentiny/opendal/blob/f9d3a2f877cdc41f71efca8631397a7ea516ebb7/bindings/go/ffi.go#L44-L93

How C binding handles IO and cancellation
https://github.com/dentiny/opendal/blob/f9d3a2f877cdc41f71efca8631397a7ea516ebb7/bindings/c/src/cancel.rs#L145-L159

How blocking IO is implemented
https://github.com/dentiny/opendal/blob/f9d3a2f877cdc41f71efca8631397a7ea516ebb7/bindings/c/src/cancel.rs#L165-L178

PR split considerations

Since it's a large PR, I considered to split it into smaller piece

Option-1: add C-binding part first, then go binding
My thought: viable, but C-binding alone is not straightforward enough to tell what implementation plan I'm doing; and two 2K LOC PR is not much easier to review than one 4K LOC PR

Option-2: implement one APIs one by one, for example, do read and core change first, then write, list, presign.
My thought: viable, but the biggest review difficulty still comes from the first large PR; if reviewers think it better I'm willing to split the PR into smaller pieces for easier review.

Are there any user-facing changes?

Yes

AI Usage Statement

Plan mode with GPT-5.5, with milestones broken down, and opus-4.7 for implementation

@dosubot dosubot Bot added size:XXL This PR changes 1000+ lines, ignoring generated files. releases-note/feat The PR implements a new feature or has a title that begins with "feat" labels Jun 10, 2026
@dentiny dentiny requested a review from yuchanns June 10, 2026 08:36
@yuchanns

yuchanns commented Jun 10, 2026

Copy link
Copy Markdown
Member

Thank you for your hard work bringing the Go binding into the real world!

I'm fine with the Go bindings, but I'd like to hear the members' views on the C part @apache/opendal-committers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

releases-note/feat The PR implements a new feature or has a title that begins with "feat" size:XXL This PR changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

new feature: go binding provides context-based API

2 participants