feat: lazy per-route cache seeding for Workers

[NathanDrake2406]

Summary

• On Workers with MemoryCacheHandler (the default when KV is not configured), pre-rendered routes now lazily seed the memory cache on first request via env.ASSETS.fetch()
• Seeding is per-route (not eager startup) to respect Workers' ephemeral isolate model and 128MB memory limit
• Concurrent cold hits for the same route are deduped via an in-flight promise map

Pre-requisite

Depends on #645 for the manifest format changes (buildId, router, trailingSlash fields).

How it works

1. Deploy time (deploy.ts): After prerendering, copies vinext-prerender.json and HTML/RSC files to dist/client/__prerender/ so they're deployed as static assets
2. Runtime (seed-cache-workers.ts): On each request, checks if the route has prerendered data. On first miss, fetches HTML/RSC from the assets binding and populates MemoryCacheHandler. Subsequent requests in the same isolate get cache HITs
3. Generated worker entry: Calls seedRouteFromAssets() before delegating to the RSC handler

Changes

File	Change
src/server/seed-cache-workers.ts	New — lazy per-route seeding with manifest caching + concurrent dedup
src/deploy.ts	Copy prerender files to assets dir + add seeding call to generated worker entry
package.json	Export ./server/seed-cache-workers
tests/seed-cache-workers.test.ts	9 unit tests

Scope

This is Workers memory stub support only, not the final durable caching story. For persistent caching, users should use KVCacheHandler (opt-in) with TPR for deploy-time KV population. Issue #562 tracks the generalised remote cache seeding.

Ref #561

Test plan

• 9 unit tests: basic seeding, index route, dynamic routes, missing manifest, non-prerendered routes, already-cached no-op, concurrent dedup, manifest caching, missing RSC graceful degradation
• 216 deploy tests pass unchanged
• Type-aware lint clean

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/vinext@653

commit: ee4f85b

[james-elicx]

/bigbonk review

[ask-bonk]

Good PR — the lazy per-route seeding approach is well-suited for Workers' ephemeral isolate model, and the shared extraction from seed-cache.ts is a clean refactor. The test coverage is solid. A few issues to address, one of which could cause request failures in production.

Summary of findings:

1. P0 (error resilience): doSeedRoute has no try/catch, and the generated worker entry awaits without a safety net — a transient env.ASSETS.fetch() failure or handler.set() error would crash the request instead of gracefully degrading.
2. P1 (permanent null caching): loadManifest permanently caches null on transient fetch failure — the isolate can never seed any route after a single hiccup.
3. Minor: copyPrerenderToAssets flattens the manifest into the same directory as the route files, which works but could cause filename collisions on a route literally named /vinext-prerender (extremely unlikely, but worth noting).

[ask-bonk]

Error resilience: doSeedRoute can throw on fetchAsset() network errors, handler.set() failures, etc. Since the generated worker entry (line 503 of deploy.ts) does await seedRouteFromAssets(...) without try/catch, any throw here becomes an unhandled error that crashes the request with a 500.

Seeding is a best-effort optimization — a failure should log and degrade gracefully, not break the request. Wrap this in try/catch:

Suggested change

	).finally(() => seedInFlight.delete(htmlKey));
	const promise = doSeedRoute(
	loaded.manifest,
	pathname,
	baseKey,
	revalidateSeconds,
	fetchAsset,
	).catch(() => {/* seeding is best-effort */}).finally(() => seedInFlight.delete(htmlKey));

Alternatively (and perhaps better), wrap the entire seedRouteFromAssets body in try/catch so that any unexpected error — including from getCacheHandler().get() — is swallowed. The loadManifest function already follows this pattern.

[ask-bonk]

Permanent null caching on transient failure: If fetchAsset("/__prerender/vinext-prerender.json") fails transiently (e.g., a brief ASSETS binding hiccup, timeout), loadedPromise is permanently set to a promise that resolves to null. Every subsequent call to seedRouteFromAssets in this isolate will get null and no-op — the isolate can never recover.

Since the manifest is immutable per deployment, a permanent cache of the successful result is correct. But caching the failure permanently is not — a retry on the next request would likely succeed.

Consider only caching successful results:

Suggested change

	return null;
	loadedPromise = (async () => {
	try {
	const res = await fetchAsset("/__prerender/vinext-prerender.json");
	if (!res.ok) return null;
	const manifest: PrerenderManifest = await res.json();
	if (!manifest.buildId || !Array.isArray(manifest.routes)) return null;
	const lookup = new Map<string, PrerenderManifestRoute>();
	for (const route of manifest.routes) {
	if (route.status !== "rendered") continue;
	if (route.router !== "app") continue;
	lookup.set(route.path ?? route.route, route);
	}
	return { manifest, lookup };
	} catch {
	loadedPromise = null; // allow retry on next request
	return null;
	}
	})();

Note: the !res.ok and validation-failure cases are arguably permanent (malformed deployment), so you could choose to only reset loadedPromise in the catch branch. Either way, the catch case (network/timeout) should not be permanently cached.

[ask-bonk]

Nit: fs.cpSync(sourceDir, targetDir, { recursive: true }) copies the contents of sourceDir into targetDir where targetDir already exists. This means vinext-prerender.json (copied on line 1219) and route files (e.g., about.html) all live at the same level under __prerender/. This works, but means a pre-rendered route literally named /vinext-prerender (producing vinext-prerender.html) would coexist fine — just calling it out.

A more robust layout would be __prerender/manifest.json + __prerender/routes/..., but that's a future cleanup, not a blocker.

[ask-bonk]

The rscData parameter is typed as ArrayBuffer, which matches the CachedAppPageValue field type. But in seed-cache.ts:142-145, the Node.js codepath does rscBuffer.buffer.slice(byteOffset, byteOffset + byteLength) to extract a clean ArrayBuffer from the Node Buffer. Worth verifying that the ArrayBuffer from Response.arrayBuffer() in the Workers path (line 179) doesn't need similar treatment — in practice Workers' Response.arrayBuffer() returns a standalone ArrayBuffer with byteOffset === 0, so this should be fine. Just flagging the asymmetry.

[ask-bonk]

This await is on the critical path of every request — it blocks the response until seeding completes for pre-rendered routes. For the first cold request this adds the latency of two env.ASSETS.fetch() calls (HTML + RSC) plus two handler.set() calls.

This is probably acceptable since env.ASSETS.fetch() is local to the colo and fast, but worth a comment noting that this is intentionally blocking (to guarantee the cache is populated before the RSC handler runs). An alternative would be to let the RSC handler serve the request normally while seeding in the background via ctx.waitUntil(), but that would mean the first request always re-renders — so blocking is the right call here.

Also: should this be wrapped in try/catch as defense-in-depth? If seedRouteFromAssets throws (see other comment), the request dies.

Suggested change

	await seedRouteFromAssets(url.pathname, (p) => env.ASSETS.fetch(new Request(new URL(p, request.url))));
	// Seed the memory cache from pre-rendered assets (lazy, per-route).
	// On first request for a pre-rendered page, fetches HTML/RSC from the
	// assets binding and inserts into the cache. Subsequent requests in the
	// same isolate get cache HITs. No-op for non-prerendered routes.
	// Blocking (not waitUntil) so the RSC handler sees the seeded cache.
	try {
	await seedRouteFromAssets(url.pathname, (p) => env.ASSETS.fetch(new Request(new URL(p, request.url))));
	} catch { /* seeding is best-effort — fall through to normal render */ }

[ask-bonk]

Good test, but the dedup assertion depends on the mock fetcher resolving synchronously (via microtask). In a real Workers environment, env.ASSETS.fetch() would yield to the event loop, and the timing of when the second caller hits the seedInFlight.get() check vs the getCacheHandler().get() check could differ.

Consider adding a test variant where the mock fetcher introduces a small delay (e.g., a Promise that resolves after a setTimeout(0)) to verify the dedup still works when fetches genuinely yield. This would give more confidence in the concurrency model.

[ask-bonk]

Review submitted successfully. Here's a summary of the feedback I posted:

Key findings:

1. P0 — Error resilience: doSeedRoute lacks error handling, and the generated worker entry awaits seedRouteFromAssets without try/catch. Any transient env.ASSETS.fetch() failure or handler.set() error would crash the request with a 500 instead of gracefully falling through to normal rendering. Suggested wrapping in try/catch both in doSeedRoute (via .catch()) and in the generated worker entry.

2. P1 — Permanent null caching in loadManifest: If the manifest fetch fails transiently (timeout, brief ASSETS binding hiccup), loadedPromise is permanently set to a null-resolving promise. The isolate can never recover and seed any route. Suggested resetting loadedPromise = null in the catch branch to allow retries.

3. Minor observations:

   • The copyPrerenderToAssets layout puts manifest and route files at the same level (works fine, just noting the design choice)
   • The ArrayBuffer handling asymmetry between Node.js (Buffer.slice) and Workers (Response.arrayBuffer()) paths — not a bug, just worth being aware of
   • The blocking await on the critical path is the correct choice (vs waitUntil) but should be defended with try/catch
   • The concurrency dedup test could be strengthened with a delayed mock fetcher to better simulate real async behavior

github run

[NathanDrake2406]

@james-elicx been a good run bro! https://x.com/nextjs/status/2037186404116291603?s=46

[james-elicx]

/bigbonk review

[james-elicx]

@james-elicx been a good run bro! x.com/nextjs/status/2037186404116291603?s=46

Not over yet - the Adapter API has been in the works for a long time and we've discussed it on many occasions (I work on OpenNext as well)

[ask-bonk]

Well-structured PR. The lazy per-route seeding approach is a good fit for Workers' ephemeral isolate model, and the previous review's P0/P1 findings have been addressed — seedRouteFromAssets now has a top-level try/catch, doSeedRoute has .catch(() => {}), and loadManifest resets loadedPromise = null on transient failures while caching permanent failures (404, malformed JSON). The shared extraction into seed-cache-shared.ts is clean. Test coverage is thorough, including the error resilience and transient retry scenarios that were missing from the first revision.

A few observations, none blocking:

1. Concurrency comment accuracy (line 96-97): The comment says "between the await above and this sync check, no other microtask can interleave" — this is correct for the JS execution model, but worth noting that two requests can both pass the getCacheHandler().get() check if they interleave on that await. The second caller correctly falls into the seedInFlight.get() dedup path, so the behavior is correct. The comment could be slightly clearer that the dedup window is between the .get() await resumption and the seedInFlight.set() call.

2. doSeedRoute early return on RSC 404: When the RSC fetch returns 404, doSeedRoute returns early without logging. This is fine for the "RSC file missing" case (tested), but it means a transient RSC fetch failure (caught by .catch()) is indistinguishable from a permanent RSC absence. Not a problem in practice since env.ASSETS.fetch() is local, but worth knowing.

3. Pages Router: The seeding is App Router only (route.router !== "app" is skipped), and the Pages Router worker entry doesn't include the seedRouteFromAssets call. This is correct for the current scope.

[ask-bonk]

Nit: The comment says "no other microtask can interleave" between the await getCacheHandler().get() (line 93) and seedInFlight.get() (line 98). This is true — after the .get() await resumes, execution runs synchronously until the next await. But the scenario the dedup protects against is two requests that both pass the .get() check (because both interleave on that await and both see no cache entry). The comment could make this clearer:

Suggested change

	// no other microtask can interleave, so at most one caller creates the promise.
	// Dedup concurrent cold hits — two callers can both pass the cache-miss
	// check above (they interleave on the await), but after resuming from
	// await, execution is synchronous until the next yield. So at most one
	// caller creates the seeding promise; the second joins it here.

[ask-bonk]

Minor: When RSC fetch returns a non-OK response, doSeedRoute returns early. This means the HTML is seeded but RSC is not — which is the correct behavior (tested in "seeds HTML even when RSC file is missing"). However, the return here silently swallows the case where RSC should exist but the fetch had a transient error. Since the HTML is already cached at this point, a subsequent call to seedRouteFromAssets will see the HTML cache hit and no-op, so the RSC will never be retried for this isolate.

This is acceptable for the env.ASSETS.fetch() use case (local, reliable), but if the seeding mechanism is ever used with a remote asset store, you'd want to either:

• Seed HTML and RSC independently (so an RSC failure doesn't prevent RSC retry)
• Or track per-key seeding rather than per-route

Not a blocker — just something to keep in mind for #562.

[ask-bonk]

Observation: fs.cpSync(sourceDir, targetDir) merges the contents of prerendered-routes/ into __prerender/, which already contains vinext-prerender.json. Since route files are .html/.rsc and the manifest is .json, there's no collision risk. But if a future route produces a .json output file, it could theoretically overwrite the manifest. Very unlikely, but a __prerender/manifest/ vs __prerender/routes/ split would make this impossible. Low priority.

[ask-bonk]

Worth noting for future readers: rscData here accepts an ArrayBuffer directly. In the Workers codepath (seed-cache-workers.ts:187), Response.arrayBuffer() returns a standalone ArrayBuffer with byteOffset === 0, so no slicing is needed. In the Node.js codepath (seed-cache.ts:142-145), fs.readFileSync() returns a Buffer whose underlying ArrayBuffer may have a non-zero byteOffset, so the slice is required there. The asymmetry is correct but easy to miss.

[ask-bonk]

Review submitted successfully. Here's a summary of what I posted:

Verdict: Approved

The PR addresses all P0/P1 findings from the previous review — error resilience is now comprehensive with a top-level try/catch in seedRouteFromAssets, .catch(() => {}) on doSeedRoute, and the loadManifest transient-vs-permanent failure distinction. Test coverage is thorough (error resilience, transient retry, concurrent dedup with real async yields, graceful RSC-missing degradation).

4 inline comments posted (none blocking):

1. Line 97 of seed-cache-workers.ts: Suggested a clearer dedup comment that explains why two callers can both pass the cache-miss check (they interleave on the await .get()), rather than just stating the invariant.

2. Line 184 of seed-cache-workers.ts: Noted that RSC fetch failures are swallowed after HTML is already cached, meaning RSC can never be retried for that isolate. Fine for env.ASSETS.fetch() (local, reliable), but worth remembering for Populating remote cache during deployment #562.

3. Line 1224 of deploy.ts: Observed that fs.cpSync merges route files into the same directory as the manifest — no collision risk today, but a manifest/ vs routes/ split would make it structurally impossible.

4. Line 50 of seed-cache-shared.ts: Documented the ArrayBuffer handling asymmetry between Workers (Response.arrayBuffer() — standalone) and Node.js (Buffer.slice required for non-zero byteOffset).

github run