runtime,internal/poll,loader: wasip2 pollable poll-integration + TCP I/O by achille-roussel · Pull Request #5390 · tinygo-org/tinygo

achille-roussel · 2026-05-11T08:53:46Z

Stacked on #5386. Mirrors the wasip1 work for wasip2. The cooperative scheduler's idle path now calls wasi:io/poll.Poll over a combined list of (clock pollable, registered pollables) instead of blocking the wasm module on a single monotonic-clock subscription, so goroutines doing TCP I/O can park while the scheduler runs other goroutines.

End to end:

$ tinygo build -target=wasip2 -o tcpecho_wasip2.wasm ./tcpecho_wasip2.go
$ wasmtime run -Sinherit-network -Stcp ./tcpecho_wasip2.wasm &
listening on 127.0.0.1:9999
tick 1
tick 2
tick 3

$ echo hello | nc 127.0.0.1 9999
hello                                    # echoed by the wasm

Two concurrent nc clients both echo while the ticker keeps progressing — the cooperative scheduler keeps running goroutines parked on sock_recv via the new pollable registry.

How it works

Wasip2's polling primitive is wasi:io/poll.Poll(list<own<pollable>>) -> list<u32> taking a list of pollable resource handles. Each blocking wasi operation has a subscribe() that yields a pollable.

TCPConn.Read / Write   ─would-block─►  internal/poll registry  ─►  task.Pause()
                                                │
                                                ▼
                       scheduler idle  ──►  pollIO(timeoutNs)
                                                │
                                                ├─ build pollables: [clock?, p1, p2, …]
                                                ├─ wasi:io/poll.Poll(...)
                                                └─ wake matched tasks → run queue

The wasip2 path mirrors the wasip1 PR's structure file-for-file:

wasip1	wasip2
`runtime/netpoll_wasip1.go` (FD-keyed `poll_oneoff` registry)	`runtime/netpoll_wasip2.go` (pollable-keyed `wasi:io/poll.Poll` registry)
`runtime/scheduler_idle_wasip1.go` (cooperative `sleepTicks` routes via `pollIO`)	`runtime/scheduler_idle_wasip2.go` (cooperative `sleepTicks` routes via `pollIO`)
`runtime/scheduler_idle_wasip1_none.go` (non-coop fallback)	`runtime/scheduler_idle_wasip2_none.go` (non-coop fallback)
`internal/poll/fd_wasip1.go` (FD type)	`internal/poll/fd_wasip2.go` (WasipNFD over (TcpSocket, InputStream, OutputStream))
`syscall/syscall_libc_wasip1.go` park-on-EAGAIN	(not needed — wasip2 wasi calls return `would-block` directly into `internal/poll`)

What's added

src/runtime/netpoll_wasip2.go — pollable-keyed pollDesc registry, pollIO(timeoutNs) building one combined wasi:io/poll.Poll call (clock + active pollables), linkname-exposed wake helpers (runtime_netpoll_addpollable_wasip2 / done / pdfired / wake). Three timing cases handled like wasip1's pollIO.
src/runtime/scheduler_idle_wasip2.go + scheduler_idle_wasip2_none.go — cooperative-variant sleepTicks / waitForEvents that route through pollIO when pollables are registered; non-coop fallback uses monotonicclock.Block. Matches the wasip1 file split.
src/runtime/runtime_wasip2.go — sleepTicks moved out (now per-config in scheduler_idle_wasip2*.go).
src/runtime/wait_other.go — build tag tightened to exclude wasip2.
src/internal/poll/fd_wasip2.go — WasipNFD wraps (TcpSocket, InputStream, OutputStream). DialTCPWasip2, ListenTCPWasip2, Accept, Read, Write, Close, SetDeadline*. Each blocking op tries the wasi call, on would-block subscribes, parks via runtime_netpoll_addpollable_wasip2 + task.Pause, on resume drops pollable + retries. Deadline-aware variants use parkUntil + time.AfterFunc + runtime_netpoll_wake_wasip2. Linkname-friendly Wasip2TCP{Listen,Dial,Accept,Read,Write,Close,SetDeadline} wrappers for test / future net callers.
src/internal/poll/errors_wasip.go — error sentinels (ErrFileClosing, ErrNetClosing, ErrDeadlineExceeded, ErrNoDeadline) extracted from fd_wasip1.go to a wasip1 || wasip2 shared file.
src/os/poll_link_wasip2.go — pulls internal/poll into the wasip2 build (no-op blank import with justifying comment for the lint check).
loader/goroot.go — listGorootMergeLinks now filters TinyGo files by //go:build constraints (via go/build.Context.MatchFile) before deciding "TinyGo owns this directory". Files that don't match the current target no longer cause upstream Go files at the same level to be dropped. Unblocks per-target overrides for follow-up work without disturbing wasip1.

Non-goals (deferred)

net.Listen("tcp", ...) / net.Dial("tcp", ...) on wasip2 via the standard net package. Upstream Go's net doesn't currently build for wasip2 because its cgo_linux.go reaches for netdb.h even though TinyGo doesn't enable cgo. Bringing up upstream net on wasip2 (either by filtering its cgo files from the merge or providing a TinyGo-native net override) is its own piece of work — the linkname-friendly TCP helpers in this PR are the foundation. The synthetic test program calls them directly.
UDP / PacketConn and DNS resolution. Same shape will work (wasi:sockets/udp, wasi:sockets/ip-name-lookup) but not in scope here.

Wasip1 regression sweep: tcpecho.wasm from #5386 still passes (echo hi | nc 127.0.0.1 9998 round-trip + concurrent clients); time.Sleep / parkfile / parksynth unchanged.

deadprogram · 2026-05-27T18:35:42Z

@achille-roussel can you please rebase this PR against the latest dev.

Also, can you please remove Claude as your co-author?

Thank you!

Mirrors PR tinygo-org#5386's wasip1 work for wasip2. The cooperative scheduler's idle path now calls wasi:io/poll.Poll over a combined list of (clock pollable, registered pollables) instead of blocking the wasm module on a single monotonic-clock subscription, so goroutines doing TCP I/O can park while the scheduler runs other goroutines. Plumbing components: - runtime/netpoll_wasip2.go: pollable-keyed pollDesc registry; pollIO builds one combined wasi:io/poll.Poll call (clock pollable + active pollables). Linkname-exposed runtime_netpoll_addpollable_wasip2 / done / pdfired / wake for internal/poll and future net. - runtime/scheduler_idle_wasip2.go + scheduler_idle_wasip2_none.go: cooperative-variant sleepTicks / waitForEvents that route through pollIO; non-coop fallback uses monotonicclock.Block. Mirrors the wasip1 structure introduced in 7000e7b. - runtime/runtime_wasip2.go: sleepTicks moved out to the scheduler_idle_wasip2*.go files. - runtime/wait_other.go: build tag tightened to exclude wasip2. internal/poll surface: - internal/poll/fd_wasip2.go: WasipNFD wraps a (TcpSocket, InputStream, OutputStream) triple. DialTCPWasip2, ListenTCPWasip2, Accept, Read, Write, Close, SetDeadline*. Each blocking op tries the wasi call, on would-block subscribes, parks, retries — same pattern as the wasip1 internal/poll.FD but pollable-keyed. Linkname-friendly Wasip2TCP{Listen,Dial,Accept,Read,Write,Close,SetDeadline} wrappers for test / future net callers. - internal/poll/errors_wasip.go: ErrFileClosing / ErrNetClosing / ErrDeadlineExceeded / ErrNoDeadline extracted from fd_wasip1.go to a wasip1||wasip2 shared file. Loader change: - loader/goroot.go: listGorootMergeLinks now filters TinyGo files by //go:build constraints (via go/build.Context.MatchFile) before deciding "TinyGo owns this directory". Files that don't match the current target no longer cause upstream Go files at the same level to be dropped. Unblocks per-target overrides in directories like src/net/ for future net.wasip2 work without disturbing wasip1. End-to-end verification: $ wasmtime run -Sinherit-network -Stcp ./tcpecho_wasip2.wasm & listening on 127.0.0.1:9999 tick 1 tick 2 tick 3 $ echo hello | nc 127.0.0.1 9999 hello # echoed by the wasm $ # two concurrent clients echo cleanly while ticker keeps ticking The test program (not shipped) uses //go:linkname to drive the internal/poll TCP helpers directly, since TinyGo doesn't yet have a net.Listen / net.Dial path on wasip2 (upstream Go's net doesn't build for wasip2 due to cgo_linux.go reaching for Linux headers). The src/net/ wasip2 wrappers are out of scope for this PR and tracked as follow-up — once they land, callers will use net.Listen / Dial directly and the linkname wrappers can drop. Wasip1 regression sweep: tcpecho.wasm still passes; time.Sleep / parkfile / parksynth unchanged.

achille-roussel · 2026-05-27T22:19:03Z

Done.

I'm exploring how to add a few tests to verify the new behavior as well.

aykevl · 2026-05-28T10:22:17Z

+	// buildContext is used to evaluate //go:build constraints on TinyGo
+	// source files. A TinyGo file that doesn't match the current target
+	// (e.g. a *_wasip2.go file in a wasip1 build) must not be treated as
+	// "TinyGo owns this directory" — otherwise wasip1 builds would lose
+	// upstream Go files at the same level.
+	bctx := build.Default
+	bctx.GOOS = config.GOOS()
+	bctx.GOARCH = config.GOARCH()
+	bctx.BuildTags = config.BuildTags()
+	bctx.Compiler = "gc"


Why is this needed? (And why do normal build tags not work?)

The loader builds a merged GOROOT before the compiler runs: symlinks files from upstream Go and TinyGo into a temp dir. At that point Go's normal build-tag filtering hasn't happened yet.

Without the change: TinyGo's old rule was "if any TinyGo file exists in this dir, drop all upstream files at the same level." So a foo_wasip2.go sitting in src/runtime/ would make the wasip1 build also drop upstream files from that dir, even though foo_wasip2.go is invisible to a wasip1 compile.

The MatchFile call filters TinyGo files by //go:build constraints first, so only files actually compiled for the current target count as "TinyGo owns this dir."

Let me know if I overlooked something, happy to change the approach as needed.

Copilot

Pull request overview

This PR extends TinyGo’s WASI Preview 2 (“wasip2”) runtime and internal/poll to support cooperative-scheduler integration with wasi:io/poll.Poll and to enable TCP I/O that can park goroutines on pollables instead of blocking the entire module. It also adjusts the GOROOT merge logic so TinyGo per-target overrides don’t incorrectly suppress upstream Go files when build tags don’t match.

Changes:

Add a wasip2 pollable-keyed netpoll registry and route the cooperative scheduler’s idle path through a unified pollIO() (clock + registered pollables).
Introduce a wasip2 internal/poll TCP FD wrapper (WasipNFD) that retries would-block ops by subscribing + parking on pollables (with deadline wake support).
Fix GOROOT merging to ignore TinyGo files whose //go:build constraints don’t match the current target.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
`src/runtime/wait_other.go`	Excludes wasip2 from the generic deadlock-only `waitForEvents` implementation.
`src/runtime/scheduler_idle_wasip2.go`	Cooperative wasip2 idle path uses `pollIO()` when pollables are registered; otherwise uses monotonic-clock polling.
`src/runtime/scheduler_idle_wasip2_none.go`	Non-cooperative wasip2 fallback sleep/wait behavior.
`src/runtime/runtime_wasip2.go`	Removes `sleepTicks` from the shared file now that it’s scheduler-variant-specific.
`src/runtime/netpoll_wasip2.go`	New pollable-keyed registry + `pollIO(timeoutNs)` implementation and linkname entrypoints for `internal/poll`.
`src/os/poll_link_wasip2.go`	Blank-import hook to ensure wasip2 `internal/poll` is linkable for now.
`src/internal/poll/fd_wasip2.go`	New wasip2 TCP socket/stream wrapper integrating with runtime pollables and deadlines.
`src/internal/poll/fd_wasip1.go`	Removes duplicated wasip error sentinels now shared.
`src/internal/poll/errors_wasip.go`	Adds shared wasip1/wasip2 error sentinels.
`loader/goroot.go`	Filters TinyGo override files by build constraints so non-matching files don’t “claim” a directory.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+// ErrFileClosingWasip2 distinguishes the wasip2 close error; reusing the
+// wasip1 ErrFileClosing keeps callers in upstream net happy.
+


+		if r.IsErr() {
+			se := r.Err()
+			if se.Closed() {
+				return 0, nil // EOF
+			}
+			return 0, errors.New("wasip2 stream read failed")
+		}


+		if cw.IsErr() {
+			se := cw.Err()
+			if se.Closed() {
+				return nn, errors.New("wasip2 stream closed")
+			}
+			return nn, errors.New("wasip2 stream write check failed")
+		}


+		if wr.IsErr() {
+			se := wr.Err()
+			if se.Closed() {
+				return nn, errors.New("wasip2 stream closed")
+			}
+			return nn, errors.New("wasip2 stream write failed")
+		}


+func (fd *WasipNFD) Close() error {
+	if fd.closed {
+		return ErrFileClosing
+	}
+	fd.closed = true
+	// Drop streams first (they reference the socket).
+	var zeroIn wasistreams.InputStream
+	if fd.input != zeroIn {
+		fd.input.ResourceDrop()
+	}
+	var zeroOut wasistreams.OutputStream
+	if fd.output != zeroOut {
+		fd.output.ResourceDrop()
+	}
+	fd.socket.ResourceDrop()
+	return nil


+			if matched, _ := bctx.MatchFile(tinygoDir, name); !matched {
+				continue


achille-roussel force-pushed the wasip2-net branch from 3baa42c to 39b8954 Compare May 11, 2026 23:09

achille-roussel force-pushed the wasip2-net branch from 39b8954 to d9e0859 Compare May 27, 2026 21:08

achille-roussel force-pushed the wasip2-net branch from b14e4c3 to d9e0859 Compare May 27, 2026 23:40

aykevl reviewed May 28, 2026

View reviewed changes

deadprogram requested review from Copilot and dgryski June 5, 2026 10:07

Copilot started reviewing on behalf of deadprogram June 5, 2026 10:08 View session

Copilot AI reviewed Jun 5, 2026

View reviewed changes

achille-roussel marked this pull request as draft June 5, 2026 18:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

runtime,internal/poll,loader: wasip2 pollable poll-integration + TCP I/O#5390

runtime,internal/poll,loader: wasip2 pollable poll-integration + TCP I/O#5390
achille-roussel wants to merge 1 commit into
tinygo-org:devfrom
achille-roussel:wasip2-net

achille-roussel commented May 11, 2026

Uh oh!

deadprogram commented May 27, 2026

Uh oh!

achille-roussel commented May 27, 2026

Uh oh!

aykevl May 28, 2026

Uh oh!

achille-roussel May 30, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		// ErrFileClosingWasip2 distinguishes the wasip2 close error; reusing the
		// wasip1 ErrFileClosing keeps callers in upstream net happy.

		if matched, _ := bctx.MatchFile(tinygoDir, name); !matched {
		continue

Uh oh!

Conversation

achille-roussel commented May 11, 2026

How it works

What's added

Non-goals (deferred)

Uh oh!

deadprogram commented May 27, 2026

Uh oh!

achille-roussel commented May 27, 2026

Uh oh!

aykevl May 28, 2026

Choose a reason for hiding this comment

Uh oh!

achille-roussel May 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants