Add durable per-worker num and execution context#61
Open
udipl wants to merge 1 commit into
Open
Conversation
Containers expose nothing to a worker about which worker it is; the Child::Instance handed to the run block carries only `name`. Code that needs a stable per-worker identifier (e.g. a prometheus-client-mmap pid_provider, which keys mmap files per process) has nothing to key on, and under Falcon/async-service the app never holds the run block itself. Such an identifier needs to be both durable across a restart (so metrics survive a re-fork instead of fragmenting) and bounded in cardinality (drawn from 0..N-1 rather than the open-ended PID space, so the file and series count stays constant). A recycled worker ordinal satisfies both. Add a container-scoped `num` allocated by Generic (a counter plus a Set free-list; idempotent release), captured in the spawn closure so it is unchanged when a `restart: true` worker re-enters `start`. Expose `num` and `kind` on Child::Instance, `instance_num` on the parent-side Child, and a `parent` link plus a `context` Frame stack built from the object graph. Hybrid links each inner thread worker to its fork, so a Hybrid thread can reach its durable process num via `instance.parent.num` with no process- or thread-global state.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A container spawns workers but exposes nothing to a worker about which worker it is. The
Child::Instancehanded to the run block carries onlyname. Under a real deployment (Rails on Falcon, where the app never callscontainer.runitself), code that needs a stable per-worker identifier has nothing to key on.The concrete driver is GitLab's
prometheus-client-mmap, whose multiprocess mode writes one set of mmap files per process, keyed by a configurablepid_provider. For that keying to behave, the identifier needs two properties at once:0..N-1) rather than the open-ended space of OS PIDs, so the number of mmap files (and the metric series they back) stays constant instead of growing with every respawn.A worker ordinal that is recycled on restart and reused on re-fork satisfies both; a PID satisfies neither.
The natural access point already exists: async-service's
Managed::Environment#prepare!(instance)is called with the container instance at worker entry. What's missing is a durable ordinal on that instance, and — forHybrid— a way to reach the process number, sinceHybrid#runyields the inner thread instance to the block (the fork instance is consumed internally), so the thread worker can't see its fork's number.What this adds
Async::Container::Frame = Data.define(:kind, :num, :name).contextis built from the object graph (instance +parentchain) — no process- or thread-global state.Implementation
Generic— container-scoped allocator: a monotonic counter plus aSetfree-list.acquirereuses the lowest released num;releaseadds to the set (so a double-release can't hand the same num to two workers).spawnallocates before the fiber so the num is captured in the closure and is unchanged when arestart: trueworker re-entersstart; it releases in the fiber'sensure, only on permanent exit, and only for nums it allocated. Allocation runs on the single reactor thread, so no synchronisation.context.rb(new) —Frameand aContextmixin (parentaccessor + recursivecontext), included into eachChild::Instance.Forked/Threaded—instance_numthreaded throughstart→Child.fork→Instance.for;Instance#num,Instance#kind,numadded toas_json;Child#instance_numon the parent side. Signal andhandle_interruptpaths are unchanged.Hybrid—Hybrid#runsetsworker.parent = <fork instance>on each inner thread worker, so a Hybrid thread'scontextis[process, thread]and its durable fork number isinstance.parent.num.Tests
mark?-reused keyed child,num/kindvisible in Forked (:process) and Threaded (:thread) workers,numpreserved across a restart.[process]/[thread]withparent == nilfor plain containers;[process, thread]under Hybrid; a 2-fork Hybrid where both workers arethread/0but reach distinctparentnumsprocess/0/process/1— i.e.parent.numis the fork number, not the thread number.Scope / notes
nums are container-global, assigned in spawn order — not per-service0..N-1. Compose withnameat a higher layer for per-service numbering.execpath (Forked::Child.spawn) doesn't carry anum— it bypassesInstance.for. Left as-is.numto a new worker within one container lifetime; consumers needing exact isolation should fold a generation token into their key.Async::Container.contextconvenience for code with no instance handle could be a separate follow-up; it would need a process-global + thread-variable and isn't required for theprepare!-based use case above.🤖 Generated with Claude Code