Skip to content

Fix stale glob cache causing empty tar archives in CacheStorage#1082

Merged
christiango merged 3 commits intomainfrom
christiango/fix-globby-caching
Mar 30, 2026
Merged

Fix stale glob cache causing empty tar archives in CacheStorage#1082
christiango merged 3 commits intomainfrom
christiango/fix-globby-caching

Conversation

@christiango
Copy link
Copy Markdown
Member

Problem

CacheStorage.put() uploads empty (~1KB) tar archives to Azure Blob cache when a worker thread handles multiple tasks that share the same outputGlob and cwd.

Root cause: @lage-run/globby wraps globby with a module-level Map cache keyed on (patterns, options). This cache never expires. When CacheStorage calls globAsync(outputGlob, { cwd }) and a previous call with the same key already ran (e.g., from an earlier task in the same worker thread), the cached result is returned — even though new files have been written to disk since then.

In practice this manifests as:

  1. Worker runs task A for a package → put() globs lib/** → caches result ["lib/a.js"]
  2. Worker runs task B for the same package → put() globs lib/** → cache hit returns stale ["lib/a.js"] (missing lib/b.js)
  3. tar-fs.pack(cwd, { entries: staleList }) produces an incomplete (or empty) tar
  4. The broken tar is uploaded to Azure Blob, poisoning the cache for all subsequent CI builds

Fix

  • Add globAsyncUncached() and globUncached() exports to @lage-run/globby that call globby/globbySync directly, bypassing the module-level cache.
  • Switch CacheStorage (backfill-cache) to use globAsyncUncached in both getHashesFor() and put(), since these always need fresh file system reads.

The existing cached globAsync/glob exports are left unchanged for callers where caching is beneficial (e.g., target graph resolution).

Test

Added cacheStorageGlobStaleness.test.ts — calls put() twice with the same outputGlob + cwd after writing new files between calls. Fails before fix (second put() returns stale single-file list), passes after.

@christiango christiango merged commit e29c80b into main Mar 30, 2026
14 checks passed
@christiango christiango deleted the christiango/fix-globby-caching branch March 30, 2026 21:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants