chore(e2e): drain test-user backlog in cleanup script#8530
Draft
jacekradko wants to merge 1 commit into
Draft
Conversation
The cleanup script used `getUserList` with `orderBy: '-created_at'` and a hard limit of 150 per query. On heavily-used dev instances, more than 150 test users would accumulate matching `clerkcookie` and the oldest never came back from the paginated head, so backlog grew indefinitely. On the shared `with-email-codes` (neutral-cub-53) instance this stranded a dozen test users going back to 2025-11. A second bug: only the current `+clerk_test@clerkcookie.com` email pattern matched the query, missing the older `test+clerk_test_<hash>@example.com` pattern (used through ~early 2026) entirely. Fix: - Paginate through every matching user (`+created_at`, page size 500) so the oldest entries actually get reached. - Run two email queries (`clerkcookie` + `clerk_test`) alongside the phone query, then gate deletion behind a strict test-domain allowlist so developer accounts like `fredrik+debugging-clerk_test@clerk.dev` that the broader query also matches are preserved. - Log the friendly env key name alongside the FAPI subdomain so it's obvious which entry corresponds to which test config.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
🦋 Changeset detectedLatest commit: b742fa9 The changes in this PR will be included in the next version bump. This PR includes changesets to release 0 packagesWhen changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The cleanup script used
getUserListwithorderBy: '-created_at'and a hard limit of 150 per query, so on heavily-used dev instances any test-user backlog beyond 150 never rotated out. The sharedwith-email-codesinstance had 13 stale users going back to November 2025; draining those locally just removed 12 of them.A second issue: the query
clerkcookieonly matched the current+clerk_test@clerkcookie.comemail pattern. The oldertest+clerk_test_<hash>@example.compattern (used through early 2026) wasn't matched at all, so those users were invisible to cleanup.The fix paginates through every matching user (
+created_at, page size 500), broadens the search by also queryingclerk_test, and then gates deletion behind a strict test-domain allowlist (clerkcookie.com,example.com,mailsac.com) plus the existing55501test-phone pattern. That way the broader query won't accidentally catch developer accounts likefredrik+debugging-clerk_test@clerk.dev.Also logs the friendly env key name alongside the FAPI subdomain so it's obvious which entry in the cleanup output corresponds to which test config.
Drafted while validating whether stale-user accumulation is what's pulling nextjs integration tests down on shared instances.